Five Common Mistakes in Game Usability Testing And How To Avoid Them

As some of our readers might already know, a couple of weeks ago we held a massive playtesting session at the Gamefounders game accelerator (which is first of its kind in Europe) and let us just get this out of my chest: the whole experience was mind blowing!

Why, you ask? Well. The reason is simple. We didn´t expect to have so many test subjects and moderating the tests was a real challenge as you can see on the picture below.
 

It was a full day session and consisted of two groups of twenty-five youngsters, average age about fifteen. What surprised us the most was their good verbal command of the English language (because they were Estonian) and also their eagerness to let loose and just wreck the games completely. 
 

About the teams
What we did notice was a general lack of knowledge in how to moderate the testing sessions. One may think that it doesn´t matter how you ask the questions, but it matters a great deal.

Without further ado, here are the most frequent game usability testing errors and how to avoid them.

1. Too much guidance
When you are moderating a testing session, try to talk about the game or app as little as possible. It is perfectly fine to be mute and not give the player any background information about the game at all.

Let them figure it out by themselves. Players need to understand the mechanics of the game from the moment of installation to the very first moment of playing it. If they don´t then you have some work to do.

2. Assuming too much
Don´t assume that the player always understands your in-game menu. Before testing the game itself, try to get the test subjects to speak about the menus and items in the game. Do the players understand what each setting and button does? How do they think they can move around the different menu items?

During the sessions I witnessed teams skipping past the start screen and also the menu options, which is a bad move. You might have usability errors in the menu system and not even know it.
 

Do check if the options also include easy mode and handicapped mode for players with disabilities. The options might change the colour scheme for players with colour blindness, enable detailed subtitles for players who can`t hear and change the game pace for players with cognitive or mobility problems.
 

3. Testing with just one demographic
If you are making casual games which should appeal to several different age and gender groups then you should test the games with all of them. With learning games aimed at small children you might want to test the game with their parents who actually buy the games.

If they can´t understand the concept then chances are they won´t buy the game.

4. Talking too much
Don´t distract the players with too many questions and take them away from the game world.  Instead, ask them to play the game and verbalise their experience. Ask them what they are feeling at the moment and try to make the questions as open ended as possible. Is something frustrating them? Is something triggering a strong emotional response?
 

The less you interrupt the player, the better. Try to limit the number of moderators to 1 per player. The player gets confused if suddenly somebody else starts asking the questions.

Try to observe the body language - are they relaxed or tense, does their body react to certain actions in the game. When do they have that special glare in their eyes etc.

5. Not recording the sessions
Taking notes during the test sessions is a good thing, but do also record the actual tests. For games I suggest using a combination of a screen recorder (there`s a nifty app for iOS apps out which also records the face of the player with the inside camera) and a video camera aimed at the face of the test subject.

Couple this setup with a skin response sensor and you will have a great way of validating your test results.

Extra tip: Use skin response sensors and eye tracking tools if possible
The skin response sensor will show you what the player is actually feeling by measuring strong emotional responses. If you are testing video games for Xbox or PS3 then there is also a sensor available, which is built into the game controller itself, giving you a nice non-intrusive way to test your game.

Should you need the maximum results then you can also use an eye tracking device for in-depth data.

That`s it for now. Feel free to share the knowledge (along with this post, it`s a good post) and remember: don`t assume, test!