Usability Testing: Methods, Metrics, SUS Score and Checklist
Usability testing means watching real people try to complete real tasks with your product, so you find where they get stuck before your customers do. Learn the main methods, how to write tasks, the metrics that matter, how to score the System Usability Scale, how many users you need, and a ready checklist.
- Usability testing observes representative users attempting realistic tasks, then fixes what confuses them.
- Combine behaviour metrics (task success, time, errors) with an attitude score such as the System Usability Scale (SUS).
- In our worked example with sample data, the mean SUS was 71.5, but only 40 percent of participants could apply a coupon.
- About five users per round finds most problems in one user group, according to the widely used Nielsen and Landauer model; test again after fixing.
Your team knows the product too well to see its rough edges. You know the coupon box hides under “Order summary” and that “Proceed” means “pay”. Real users do not. Usability testing closes that gap by putting the product in front of the people it is built for and watching what actually happens, rather than asking what they think might happen.
This guide covers the methods, planning, metrics and reporting you need, plus a short Python script we ran to score the System Usability Scale and task metrics from sample data. It complements functional testing approaches such as exploratory testing and user acceptance testing.
What is usability testing?
Usability testing is a research method where participants who represent your users try to complete specific tasks while a facilitator observes, listens and records. It measures how effectively, efficiently and satisfactorily people can use a product, the three parts of the ISO 9241-11 definition of usability. It is not a survey, a focus group or a demo: the core is observing behaviour.
Why usability testing matters
- It finds problems that code reviews and automated tests cannot, because the code can be correct and still confusing.
- It is far cheaper to fix a prototype than shipped code and support tickets.
- It settles design debates with evidence instead of opinions.
- It is a form of validation: checking you built the right thing, as our verification vs validation guide explains.
Usability testing methods
Most methods combine two choices: moderated or unmoderated, and in person or remote.
- Moderated in person: a facilitator sits with the participant in a lab or office. Rich insight and follow-up questions; slower and costlier.
- Moderated remote: the same, over a video call with screen sharing. The most common form today.
- Unmoderated remote: participants complete tasks alone using a testing platform that records the screen and voice. Fast and scalable, but you cannot ask “why?” in the moment.
- Guerrilla testing: quick, informal sessions with people in a cafe or corridor. Great for early prototypes, weak on representativeness.
- Think-aloud protocol: participants narrate what they are thinking while working; used within most of the methods above.
- Benchmark testing: repeated, quantitative tests to compare versions or competitors over time.
- Tree testing and first-click tests: focused methods for navigation and information architecture.
Methods compared
| Method | Depth of insight | Cost and time | Best for |
|---|---|---|---|
| Moderated in person | Highest | High | Complex products, physical devices |
| Moderated remote | High | Medium | Most software products |
| Unmoderated remote | Medium | Low per session | Larger samples, simple tasks, metrics |
| Guerrilla | Low to medium | Very low | Early concepts and quick checks |
| Benchmark | Quantitative | Medium to high | Tracking improvement over releases |
Planning a usability test
- Define goals: what decisions will this test inform? For example, “Can new customers complete checkout with a coupon?”
- Pick participants: match your real audience on experience, devices and context. Screen out colleagues and power users unless they are the audience.
- Choose the method from the table above.
- Write 3 to 6 tasks tied to the goals.
- Define metrics and success criteria in advance.
- Prepare the environment: a stable build or prototype, test accounts and realistic data, recording consent.
- Run a pilot session to catch confusing tasks and technical issues.
Write this up in a one-page test plan; our test plan vs test strategy guide shows where it fits.
Writing good usability tasks
| Weak task | Better task |
|---|---|
| Click the Coupon link and enter WINTER. | You received a WINTER discount code by email. Use it on your current order. |
| Find the Settings page. | You moved house. Update where your next order will be delivered. |
| Do you like the new dashboard? | Find out whether yesterday’s order has shipped. |
Good tasks describe a goal in the user’s language, never name the UI element to click, have a clear end point, and are realistic. Avoid leading words that match your button labels, or participants simply hunt for the matching text.
Running a session
- Explain that you are testing the product, not the person, and that there are no wrong answers.
- Ask participants to think aloud.
- Stay quiet and neutral; answer questions with “What would you do if I were not here?”
- Note where they hesitate, backtrack, misread or give up, with timestamps.
- After the tasks, run a short questionnaire such as SUS and ask open questions.
- Thank them, and pay them if you promised an incentive.
Tip: have a second person take notes so the facilitator can focus on the participant. Invite developers to watch live; nothing builds empathy faster.
Usability metrics that matter
| Metric | What it measures |
|---|---|
| Task success rate | Share of participants who completed the task (effectiveness) |
| Time on task | How long successful attempts took (efficiency) |
| Error rate | Wrong clicks, invalid entries, dead ends |
| Single Ease Question | A 1 to 7 rating right after each task |
| SUS score | Overall perceived usability on a 0 to 100 scale (satisfaction) |
With small samples, treat numbers as signals and pair them with observed reasons. “2 of 5 completed checkout” means little without “3 people did not notice the coupon field”.
How to calculate the SUS score
The System Usability Scale, created by John Brooke in 1986, is a ten-statement questionnaire answered from 1 (strongly disagree) to 5 (strongly agree). Odd-numbered statements are positive and even-numbered ones negative, so scoring flips the even items:
def sus(r): # r = ten answers, each 1 to 5
odd = sum(r[i] - 1 for i in range(0, 10, 2)) # items 1, 3, 5, 7, 9
even = sum(5 - r[i] for i in range(1, 10, 2)) # items 2, 4, 6, 8, 10
return (odd + even) * 2.5 # scale 0 to 40 up to 0 to 100
The result is a score from 0 to 100, but it is not a percentage. Across many published studies the average SUS is about 68, so scores above 68 are better than average and those below it point to problems.
Worked example: SUS plus task metrics
We scored five participants’ questionnaires and task results with Python 3.13.6. The answers and timings are sample data created for illustration:
P1 85.0
P2 62.5
P3 100.0
P4 32.5
P5 77.5
mean SUS: 71.5
Find order status success 80% median time (successes) 48.5 s
Apply coupon success 40% median time (successes) 91.5 s
Change address success 100% median time (successes) 70 s
The mean SUS of 71.5 sits slightly above the 68 average and would look fine on a dashboard. The task data tells a different story: only two of five participants could apply a coupon, and they took about a minute and a half. One participant also scored the product just 32.5. That combination, an acceptable average hiding a failing core task, is why you should never rely on a single satisfaction score. The fix list starts with the coupon flow.
How many users do you need?
# Nielsen and Landauer model: share of problems found = 1 - (1 - L)^n, with L = 0.31
users 1 -> 31.0% of problems
users 3 -> 67.1% of problems
users 5 -> 84.4% of problems
users 8 -> 94.9% of problems
users 15 -> 99.6% of problems
Under this model, where each user reveals a given problem 31 percent of the time on average, five users uncover about 84 percent of problems, and returns diminish quickly after that. The practical advice is to test with about five users, fix what you found, and test again, rather than run one large study. The model assumes one fairly homogeneous user group and problems that are equally visible; with several distinct audiences, test about five from each, and use larger samples (20 or more) when you need reliable quantitative metrics.
Reporting findings
- List each problem with what happened, how many participants hit it, and a short clip or quote.
- Rate severity, for example blocker, major, minor and cosmetic, similar to bug severity.
- Recommend a fix, but separate the observation from the proposed solution.
- Highlight what worked well, so it is not “fixed” by accident.
- Track issues in your normal backlog, written like a good bug report.
Common usability testing mistakes
- Testing with colleagues instead of real users.
- Leading tasks that name the button to press.
- Helping participants when they struggle, which hides the problem.
- Asking opinions (“Would you use this?”) instead of observing behaviour.
- Waiting until the product is finished; test prototypes early.
- Collecting findings but never re-testing the fixes.
Usability testing checklist
- Goals and decisions defined.
- Participants match the real audience; consent and incentives ready.
- Three to six realistic, non-leading tasks with clear end points.
- Metrics chosen: success, time, errors, SUS.
- Pilot session completed.
- Facilitator and note-taker assigned; recording working.
- Findings rated, shared and turned into backlog items.
- Follow-up round scheduled to verify fixes.
Final thoughts
Usability testing is the fastest way to learn how your product feels to the people who matter. Watch a handful of representative users attempt realistic tasks, measure success and time, add a SUS questionnaire, and fix the biggest problems before testing again. As our sample data showed, a respectable average score can hide a broken core task, so let behaviour lead the analysis. Write tasks from the same user goals as your acceptance criteria to keep design and testing aligned.
Frequently asked questions
What is usability testing in software testing?
It is a method where representative users try to complete realistic tasks with a product while observers note where they succeed, struggle or fail, so the design can be improved.
How many users are needed for usability testing?
About five users per round usually reveals most problems for one user group. Test, fix and test again, and use larger samples when you need reliable statistics.
How is the SUS score calculated?
Subtract 1 from each odd item answer, subtract each even item answer from 5, add the ten results and multiply by 2.5. The score ranges from 0 to 100, with about 68 as the average.
What is the difference between usability testing and user acceptance testing?
Usability testing checks how easy and pleasant a product is to use. User acceptance testing checks whether it meets agreed business requirements before release.
What is the difference between moderated and unmoderated usability testing?
In moderated testing a facilitator guides the session live and can ask follow-up questions. In unmoderated testing participants work alone while software records their screen and voice.
Keep learning
Need help with testing, WordPress or SEO?
We help teams ship better software and grow their sites. Tell us what you are working on and we will suggest a clear next step.