2023
First-Time Users Testing
My first project in House of Fun and Playtika LTD. was to analyse our First-Time User experience (FTUE) and compare it with other social casino games. This small task questioned many decisions made a few years ago, eventually leading to a more significant user testing project to improve our FTUE.
My Role
This project took place while I was working at Playtika Ltd.
My role was to decide who and how to test our FTUE. Later on, with the team's help, I analysed the results and created a report with my conclusion and the action items required.
Unfortunately, I cannot share the full report and some data collected during this project.
01
Who and How?
5 Social Casino players that never played House of Fun, Aged 50-70, 80% Women.
Each player will play for at least 15 minutes a day for 4 days.
All sessions (video and audio) will be recorded, and players should say their thoughts and actions out loud.
At the end of each session, players will answer a short survey so I can have their retrospective point of view.
02
What?
I start by talking with our Customer Insights department and see if they have something to share
FTUE is a vast expression, and I had to break it down to the answers we seek. When we know what they are, we will know which questions to ask or, in that case, what insights to look for.
First Impression
Tutorial
First Actions
Balance
Promotions
Notifications
News
Popups
Rewards/Gifts
Peak Moments
Features
Activity
Mechanics
Rewards
How to Win
Visibility
Slots
Preferences
Rules
Payouts
Mechanics
Bottom Bar
Graphics
Simplicity
Facebook Login
Storytelling
Characters
Environment
Levelling
Mechanics
Value
Orientation
Lobby
Hierarchy
Estimation
How many times...?
Memory
Can you recall when...?
General Experience
How did you feel when...?
Understanding
How do you...?
Reasoning
Why do you...?
Attention
What happened when...?
These notes helped me create a more focused list of subjects to test and methods to use
Using these two groups, I have decided on the subjects each session will focus on and the questions we will ask.
Day 1: First Impression
This session focuses on the players' game flow and feelings around their achievements, progression, and peak points.
Day 2: Slot Machines
What attributes in the slots are essential for first-time players? Are there preferred styles? Mechanics? Paylines?
Day 3: Orientation
Are players know where and when to collect rewards? Where to hear the news? How to navigate and find machines and features?
Day 4: Story and Features
Do players recognise HoF’s main characters? Do they feel in Vegas? Do they interact with features and use them to progress?
03
Collecting and Organizing the Data
Affinity Diagram
Under the three areas of subjects, every subject contained annotations from all the players and their survey answers
Slot Machines
General
Rules
Pick Reasons
Free Spins
Mechanics
Bets
Big Wins
Variety
Balance
Payouts
Graphics
Features
We collected almost 6 hours of recorded sessions to find insights. Playtest Cloud® platform has conducted the test and shared the players’ information, video and audio recording and the survey’s answers.
The team and I reviewed the videos and created annotations with timestamps to mark essential and insightful moments.
Features
Inbox
HoF Quest
Rush
Hourly Bonus
Pass
Album
Tutorial
Piggy
Wheel of Fun
Others
Progression
Popups
Technical
Exploration
Prizes
Insightful
^
Here is a glance at annotations in a unique group that contains surprisingly insightful notes
04
Analysis
Impression Test - 1st Session
Live Reactions
Survey's Retro
Gyla, F, 50
Corintha, F, 50
Robert, M, 42
Julia, F, 42
Kim, F, 66
During the session, most of the players had a lot of negative reactions. However, at the end of the session, some of those players said their experience was positive.
It means we were doing ok overall, but we can't settle for a generally positive experience and must improve.
Analytics - All Sessions
How do players pick their machine?
Browsing the Lobby
Promotions
First-time players should browse less and spin more.
What kind of content will attract First-time players to the lobby?
Picked machines from the loby had a...
Human figure in the thumbnail
Animal figure in the thumbnail
Las
Vegas theme
Catchy
name
Adorable character in the thumbnail
Promoting machines with a character or a "Vegas" logo will help shorten the browsing time.
We also need to ensure players keep spinning and not returning to browsing.
Why do players switch machines during a spin?
Interrupti-ons
"HoF Quest" notifications
Want to
explore more
Bad payouts
When players start losing coins, they will switch machines. If the will continue losing, they will stop playing. First-time players must play for a decent amount of time before they run out of things to do and must wait/purchase.
Further Exploration: Bets, Low Balance
Why do players change their bets?
Balance
Expectat-ions
Level progression
Super-stitions
Winning
Strike
Not surprisingly, higher balances raise the bets. When players raise the bet, they are engaged and have a better experience while playing.
Higher bets are awesome but can cause players to reach a low balance.
When did players reach a low balance?
Day 1
Robert
Day 2
Kim, Gyla
Day 3
Robert
Day 4
Gyla
Three of our testers lost all their coins after playing for less than 30 minutes. As testers, they had to continue playing and come back for the next sessions; normal players won't do that.
We cannot allow first-time players to run out of coins.
What is important when spinning?
Many general #positive or #negative annotations regarding the spinning experience were around #rules.
Further exploration: Rules clarity, Confusion
How important is it for players to understand the machine's rules?
In both machines, 75% of all annotations were
regarding the rules' coherency.
Everything that is luck-related (the reels) should remain unknown, but the rest should be known and easily understood.
Even when they didn't like it, players played in "Irish Princess" to win the "HoF Quest" grand reward. This feature has a lot of potential to help us attract players to the machines we pick.
85% Positive Feedback
-
75% #positive #rules
100% Negative Feedback
-
75% #negative #rules
Annotations regarding confusing popups
Out of context
Too many at once
Confusing
Interuupt spinning
Too many popups that appear out of context are another cause of the players' confusion.
I analysed more data regarding our game features’ potential, news and communications, rewards systems, and lobby navigation before summarising and coming up with conclusions
05
Identify the Issues
After examining all the charts and crossing the data, it was time to summarise the main pain points.
I have divided the points into two groups and ordered them by importance: Urgent, Essential, and Important.
Resources
Players cannot reach a low balance
Players don't notice the "Hourly Bonus" and not collecting it
Players don't notice the "Fun Bonus"
Rewards in "Pass" are unclear
Players might not enter the lobby during a session
Features' rewards do not necessarily keep high balances
Levelling up isn't attractive
Rewards in the Inbox look less luxurious
Communication
Machines' rules clarity is crucial
"HoF Quest" is engaging and focuses players on spinning
Machines with characters (especially adorable ones) attract players more
Players enjoy classic "Las Vegas" or "777" machine themes
Too many popups appear while players spin
Players ignore popups that appear one after the other
Technical issues cause popups disappearance
Players don't easily understand how "Pass" works
Time-based progression is confusing and inaccurate
Players spend much time browsing because the navigation in the lobby isn’t easy
06
Presenting the Solutions
Processes take time. I couldn't finish the research with only the pain points presented; instead, I came up with solutions.
To simplify it, I have suggested short-term quick patches, medium-sized fixes, and long-term ideas to solve some more extensive pain points.
Bets and Balances
Machine Rules
Progression
Popups
Continuously reward players who reach a low balance until they unlock features that will give them more coins.
Control "Max Bet" so players won't be able to lose their coins quickly.
When rewarding (thorough features, progression etc.), reward with only coins as they are the only thing that matters at this stage.
Relocate the "Game Rules" icon so players can find it easily.
Create a list of easy-to-understand machines and promote them during the FTUE period.
Machine Selection
Place machines with more suitable content for FTUE players in "HoF Quest" or "Adventure Emily".
Modify FTUE players' payouts so they will never reach a low balance.
Create a scenario that allows players to go up and down but never lose all their coins.
Use effort-based features for FTUE players so they will last longer, contain more machines and disappear when done.
Redesign the lobby so players can easily navigate and search for a machine or group of machines they like
All rights belong to House of Fun and Playtika 2023
©
Change level progression notifications to a more celebrative and informative event.
Change the progression from time to effort based; this gives the player an individual experience, not a generic one.
Features
Notify players who fail to collect the hourly bonus when their balance is low or when returning to the lobby.
Rethink which content is allowed to interrupt players while spinning.
Unlock "HoF Today" from day 0 to reduce the number of popups and teach the players where to hear the news.
Redesign popups that appear through the FTUE period so they will be informative.
Solve the overriding popups issue so messages will not disappear.
Modify the "Hourly Bonus" for FTUE players so it will be available more and give smaller amounts of coins.
Inbox Communication
Give less rewards through the inbox.
Use visually clear rewards (coins, gems) or an immediate mini-game.
Allow some time between features as players still don't recognise the difference between promotions.
Redesign the "Houly Bonus" icon to make it more prominent.
Teach players how to erase messages.
Adjust the notification badge to show only new messages.
Redesign the inbox to contain sections for different types of messages, and rewards won't erased easily.
Introduce every feature with an intro or tutorial.