Member-only story
You need to observe your users, not just survey them
Surveys are not enough and don’t tell you the whole story.
Sometimes it feels like UX Research has become synonymous with surveys. They’re relatively cheap, easy and can reach large amounts of users. Sounds great, right?
Wrong. Surveys aren’t as easy to design as people assume. If you ask biased or leading questions, you cannot trust your results. My friend and former colleague, Mimi Turner wrote an excellent four-part series about this (links at the end).
Beyond possible biased results, you still may not get the full picture from a survey only. If you ask people how easy or difficult they find a task (or using your product) their answer may not be the same one you would draw if you observed them completing the task. This point can be illustrated with the exaggerated example of labour and delivery.
Labour and delivery
If you watched someone (without an epidural) go through labour and delivery you would probably observe they are in a great deal of pain. If you saw the same person shortly after birth they may seem pain-free and possibly overjoyed. This is because of the oxytocin released by the body and the fact that they (hopefully) have a healthy and loved baby (the halo effect). If you ask the parent how bad childbirth was after the fact, they may downplay the amount of pain they experienced at the moment.
Let’s take this example to the extreme, and say that we can only administer a survey and not observe any labouring and delivering people. We would determine that childbirth doesn’t hurt that much.
The same is true of user research. I have observed participants not complete tasks but still rate the task as ‘very easy’, 7 out of 7, on the SEQ*. I’ve also seen them struggle with every step of the process, only to state that a task was very easy.
*The Single Ease Question (SEQ) is a 7-point scale assessing how difficult or easy users find a task. It should be administered immediately after a user attempts the task in usability testing (Sauro…
