You can’t beat a user test

Here at iQ Content we do lots of ‘traditional’ user testing. This involves recording the on-screen actions of a user as they interact with a website. During each test, we ask the users to do what is called a ‘concurrent think aloud’. The user tells us what they are doing and thinking during the user test. This helps us identify usability issues on a site. We see great value in this, in what we consider to be a tried and tested approach. It gives us insights into user behaviour that we may not get if the user did not perform a think aloud.

Eye Tracking, another kind of user test

Last week I attended EyeTrackUX 2009 in Frankfurt. I was at some great (and some not so great) seminars, all related to the Eye Tracking method. In a nutshell, eye tracking allows analysts to track and record a users’ eye gaze whilst they interact with a website. This enables whoever is doing the research to see what the user looked at, what attracted their attention the most, and what elements on a page they completely missed. One of the outputs of a user test done using eye tracking is called a gaze replay. This is basically a replay of what the user looked at during the test. The video below is an example of this:

Is silence really golden?

At the end of the second day, I was lucky enough to speak with Rob Stevens from Bunnyfoot, and Guy Redwood from SimpleUsability. Both firmly agree, that when we are testing a website using Eye Tracking, the user should remain silent during the initial test. As Rob and Guy described, this is natural browsing behaviour, but by getting them to think aloud during the test, we are interfering with this natural behaviour.

Rob says “Generally, we don’t want users to think during their interaction with our websites, but that is exactly what we are forcing them to do when we get them to give concurrent protocols… and as Steve Krug so famously once said, “Dont make me think’’. With concurrent protocols, we may not be getting the right information from the user, and more critically, may be missing some some vital information all together.

Tell me later: Retrospective protocols

Once the eye tracking test is finished, a gaze replay is shown to the user. It is at this point that they get the user to give a ‘retrospective protocol’’, requiring the user to narrate their experience, as they see it being played back. The gaze replay reminds the user exactly what they did, what they looked at, and what they thought, at every point throughout the test. Guy Redwood said, “You get an insight into just how amazing the human brain is after watching a retrospective protocol… users correct themselves, and can instantly recall very precisely what happened”.

Having investigated Eye Tracking as a potential method for iQ, and having talked to many people at EyeTrackUX 09, I think it holds some merit over traditional user testing. Here are the reasons why;

• It eliminates the cognitive strain we place on the users when we force them to think aloud during a user test. This allows us to observe more natural browsing and interaction behaviours.
• It facilitates a more natural setting for the user test, where quite often, the analyst isn’t even required to be in the same room as the user. This reduces (although doesn’t eliminate) the likelihood of the Hawthorne effect taking place.
• It provides irrefutable proof about what our users looked at during the session which helps to reduce any suspicion that we, as analysts, are giving our subjective opinion about why or what a user did or said…. ‘here is the proof’!

With these in mind, I look forward to posting some insights and findings from our upcoming eye tracking studies here at iQ.

What do you think? Is it time to move on completely from the traditional testing approach? Or can we still get just as much insight from traditional user testing?