It's Not "User Testing"

Bevan Rudge

on

December 8, 2008

It's Not "User Testing"

It's not User Testing! A number of times recently I have heard folk in the Drupal community and elsewhere talk about User Testing. While many people use this term and it is not strictly incorrect it is misleading and also far from correct A better term to use is Usability testing or UI testing. Here's why; The point of usability testing is to evaluate the user interface of a piece of software. The subject of the test, the thing which has the ability to pass or fail, is the UI of the software. The user is not being tested. The user can do or say anything and they will not pass nor fail. Only the software's user interface is being tested and therefore only the UI can pass or fail – not the user. (Having said that, usability testing does not usually result in a pass or fail. Usability testing is rarely quantitive, and is difficult to do in this manner. Most usability testing is qualitative and results are a set of notes or issues, rather than ticks and crosses.) The term user testing suggests that the user is being tested, in much the same way as cosmetic testing suggests that a cosmetic product is being tested. While this might not be so problematic in other areas – i.e. calling cosmetic testing monkey testing when a cosmetic is applied to a monkey (not that we approve of that) – however in usability testing this misconception is the most common reason for wasted usability tests. Due to the nature of usability testing it is very easy for a participant – the user of the software which is the subject of the test – and even for designers of the software to mistakenly think that the participant is being tested. This can result in comments like "That participant was dumb" – which is utterly meaningless and usually means everyone's time was wasted. If the participant had a problem with the software, how can it possibly be the participant's fault? It is the fault of the software which is being tested. The participant must be made comfortable enough that they can express their true opinions about the software. They should be invited to blame and criticize the software and reminded that they are not being tested and nothing they say or do will be approved of nor disapproved of. Good usability test facilitators take this idea so far that they consciously weaken themselves around participant. They make themselves appear small in the chair, speak timidly, use body language expressing vulnerability, are indifferent but accepting to criticism, yet always open to listen. They never say anything like "congratulations", "you did well" or "you were supposed to do ... intead". The term User testing quickly creates misconceptions in software designers evaluators and participants as to what usability tests are for. To avoid these misconceptions, please use the term Usability Testing instead!

Share it!

Hehe, monkey testing - that sounds like something I would use. There is nothing natural about usability tests, so I don't think we should pretend it is - I never mention the word user testing to my participants so I don't think it really matters. For the client, it is far more clear to call it user testing then usability testing for obvious reasons. The developers "are" smart enough, we are not testing the actual user.
User-acceptance testing is another valuable related perspective. It's not so much testing of the interface/usability, so much as functionality "does it let the user do what the user needs/wants/expects?".