1. Home
  2. Blog
  3. Software Testing

Software Testing

Usability Testing: Methods, Metrics, SUS Score and Checklist

Usability testing means watching real people try to complete real tasks with your product, so you find where they get stuck before your customers do. Learn the main methods, how to write tasks, the metrics that matter, how to score the System Usability Scale, how many users you need, and a ready checklist.

9 min read
Usability testing: methods, metrics, SUS score and checklist

Key takeaways

  • Usability testing observes representative users attempting realistic tasks, then fixes what confuses them.
  • Combine behaviour metrics (task success, time, errors) with an attitude score such as the System Usability Scale (SUS).
  • In our worked example with sample data, the mean SUS was 71.5, but only 40 percent of participants could apply a coupon.
  • About five users per round finds most problems in one user group, according to the widely used Nielsen and Landauer model; test again after fixing.

Your team knows the product too well to see its rough edges. You know the coupon box hides under “Order summary” and that “Proceed” means “pay”. Real users do not. Usability testing closes that gap by putting the product in front of the people it is built for and watching what actually happens, rather than asking what they think might happen.

This guide covers the methods, planning, metrics and reporting you need, plus a short Python script we ran to score the System Usability Scale and task metrics from sample data. It complements functional testing approaches such as exploratory testing and user acceptance testing.

What is usability testing?

Usability testing is a research method where participants who represent your users try to complete specific tasks while a facilitator observes, listens and records. It measures how effectively, efficiently and satisfactorily people can use a product, the three parts of the ISO 9241-11 definition of usability. It is not a survey, a focus group or a demo: the core is observing behaviour.

Why usability testing matters

  • It finds problems that code reviews and automated tests cannot, because the code can be correct and still confusing.
  • It is far cheaper to fix a prototype than shipped code and support tickets.
  • It settles design debates with evidence instead of opinions.
  • It is a form of validation: checking you built the right thing, as our verification vs validation guide explains.

Usability testing methods

Most methods combine two choices: moderated or unmoderated, and in person or remote.

  • Moderated in person: a facilitator sits with the participant in a lab or office. Rich insight and follow-up questions; slower and costlier.
  • Moderated remote: the same, over a video call with screen sharing. The most common form today.
  • Unmoderated remote: participants complete tasks alone using a testing platform that records the screen and voice. Fast and scalable, but you cannot ask “why?” in the moment.
  • Guerrilla testing: quick, informal sessions with people in a cafe or corridor. Great for early prototypes, weak on representativeness.
  • Think-aloud protocol: participants narrate what they are thinking while working; used within most of the methods above.
  • Benchmark testing: repeated, quantitative tests to compare versions or competitors over time.
  • Tree testing and first-click tests: focused methods for navigation and information architecture.

Methods compared

Method Depth of insight Cost and time Best for
Moderated in person Highest High Complex products, physical devices
Moderated remote High Medium Most software products
Unmoderated remote Medium Low per session Larger samples, simple tasks, metrics
Guerrilla Low to medium Very low Early concepts and quick checks
Benchmark Quantitative Medium to high Tracking improvement over releases

Planning a usability test

  1. Define goals: what decisions will this test inform? For example, “Can new customers complete checkout with a coupon?”
  2. Pick participants: match your real audience on experience, devices and context. Screen out colleagues and power users unless they are the audience.
  3. Choose the method from the table above.
  4. Write 3 to 6 tasks tied to the goals.
  5. Define metrics and success criteria in advance.
  6. Prepare the environment: a stable build or prototype, test accounts and realistic data, recording consent.
  7. Run a pilot session to catch confusing tasks and technical issues.

Write this up in a one-page test plan; our test plan vs test strategy guide shows where it fits.

Writing good usability tasks

Weak task Better task
Click the Coupon link and enter WINTER. You received a WINTER discount code by email. Use it on your current order.
Find the Settings page. You moved house. Update where your next order will be delivered.
Do you like the new dashboard? Find out whether yesterday’s order has shipped.

Good tasks describe a goal in the user’s language, never name the UI element to click, have a clear end point, and are realistic. Avoid leading words that match your button labels, or participants simply hunt for the matching text.

Running a session

  • Explain that you are testing the product, not the person, and that there are no wrong answers.
  • Ask participants to think aloud.
  • Stay quiet and neutral; answer questions with “What would you do if I were not here?”
  • Note where they hesitate, backtrack, misread or give up, with timestamps.
  • After the tasks, run a short questionnaire such as SUS and ask open questions.
  • Thank them, and pay them if you promised an incentive.

Tip: have a second person take notes so the facilitator can focus on the participant. Invite developers to watch live; nothing builds empathy faster.

Usability metrics that matter

Metric What it measures
Task success rate Share of participants who completed the task (effectiveness)
Time on task How long successful attempts took (efficiency)
Error rate Wrong clicks, invalid entries, dead ends
Single Ease Question A 1 to 7 rating right after each task
SUS score Overall perceived usability on a 0 to 100 scale (satisfaction)

With small samples, treat numbers as signals and pair them with observed reasons. “2 of 5 completed checkout” means little without “3 people did not notice the coupon field”.

How to calculate the SUS score

The System Usability Scale, created by John Brooke in 1986, is a ten-statement questionnaire answered from 1 (strongly disagree) to 5 (strongly agree). Odd-numbered statements are positive and even-numbered ones negative, so scoring flips the even items:

def sus(r):                       # r = ten answers, each 1 to 5
    odd  = sum(r[i] - 1 for i in range(0, 10, 2))   # items 1, 3, 5, 7, 9
    even = sum(5 - r[i] for i in range(1, 10, 2))   # items 2, 4, 6, 8, 10
    return (odd + even) * 2.5      # scale 0 to 40 up to 0 to 100

The result is a score from 0 to 100, but it is not a percentage. Across many published studies the average SUS is about 68, so scores above 68 are better than average and those below it point to problems.

Worked example: SUS plus task metrics

We scored five participants’ questionnaires and task results with Python 3.13.6. The answers and timings are sample data created for illustration:

P1 85.0
P2 62.5
P3 100.0
P4 32.5
P5 77.5
mean SUS: 71.5

Find order status  success 80%   median time (successes) 48.5 s
Apply coupon       success 40%   median time (successes) 91.5 s
Change address     success 100%  median time (successes) 70 s

The mean SUS of 71.5 sits slightly above the 68 average and would look fine on a dashboard. The task data tells a different story: only two of five participants could apply a coupon, and they took about a minute and a half. One participant also scored the product just 32.5. That combination, an acceptable average hiding a failing core task, is why you should never rely on a single satisfaction score. The fix list starts with the coupon flow.

How many users do you need?

# Nielsen and Landauer model: share of problems found = 1 - (1 - L)^n, with L = 0.31
users  1 -> 31.0% of problems
users  3 -> 67.1% of problems
users  5 -> 84.4% of problems
users  8 -> 94.9% of problems
users 15 -> 99.6% of problems

Under this model, where each user reveals a given problem 31 percent of the time on average, five users uncover about 84 percent of problems, and returns diminish quickly after that. The practical advice is to test with about five users, fix what you found, and test again, rather than run one large study. The model assumes one fairly homogeneous user group and problems that are equally visible; with several distinct audiences, test about five from each, and use larger samples (20 or more) when you need reliable quantitative metrics.

Reporting findings

  • List each problem with what happened, how many participants hit it, and a short clip or quote.
  • Rate severity, for example blocker, major, minor and cosmetic, similar to bug severity.
  • Recommend a fix, but separate the observation from the proposed solution.
  • Highlight what worked well, so it is not “fixed” by accident.
  • Track issues in your normal backlog, written like a good bug report.

Common usability testing mistakes

  • Testing with colleagues instead of real users.
  • Leading tasks that name the button to press.
  • Helping participants when they struggle, which hides the problem.
  • Asking opinions (“Would you use this?”) instead of observing behaviour.
  • Waiting until the product is finished; test prototypes early.
  • Collecting findings but never re-testing the fixes.

Usability testing checklist

  • Goals and decisions defined.
  • Participants match the real audience; consent and incentives ready.
  • Three to six realistic, non-leading tasks with clear end points.
  • Metrics chosen: success, time, errors, SUS.
  • Pilot session completed.
  • Facilitator and note-taker assigned; recording working.
  • Findings rated, shared and turned into backlog items.
  • Follow-up round scheduled to verify fixes.

Final thoughts

Usability testing is the fastest way to learn how your product feels to the people who matter. Watch a handful of representative users attempt realistic tasks, measure success and time, add a SUS questionnaire, and fix the biggest problems before testing again. As our sample data showed, a respectable average score can hide a broken core task, so let behaviour lead the analysis. Write tasks from the same user goals as your acceptance criteria to keep design and testing aligned.

Frequently asked questions

What is usability testing in software testing?

It is a method where representative users try to complete realistic tasks with a product while observers note where they succeed, struggle or fail, so the design can be improved.

How many users are needed for usability testing?

About five users per round usually reveals most problems for one user group. Test, fix and test again, and use larger samples when you need reliable statistics.

How is the SUS score calculated?

Subtract 1 from each odd item answer, subtract each even item answer from 5, add the ten results and multiply by 2.5. The score ranges from 0 to 100, with about 68 as the average.

What is the difference between usability testing and user acceptance testing?

Usability testing checks how easy and pleasant a product is to use. User acceptance testing checks whether it meets agreed business requirements before release.

What is the difference between moderated and unmoderated usability testing?

In moderated testing a facilitator guides the session live and can ask follow-up questions. In unmoderated testing participants work alone while software records their screen and voice.

Keep learning

Work with Stack Quora

Need help with testing, WordPress or SEO?

We help teams ship better software and grow their sites. Tell us what you are working on and we will suggest a clear next step.

Keep reading

More guides on testing, AI & modern development

Enjoyed this article? Pick a topic and jump straight into our newest tutorials.

Artificial Intelligence

Linear Regression in Python: Least Squares From Scratch, With R Squared

Linear regression in Python from scratch: closed form, NumPy least squares and gradient descent, R squared, RMSE, outliers and curved…

Read article →
Software Development

301 vs 302 Redirect: Differences, SEO Impact, 307 and 308

301 vs 302 redirect: the differences, SEO impact, 307 and 308, method changes, redirect chains and loops, all tested on…

Read article →
DevOps

Git Remote Explained: Add, Set-URL, Rename, Remove and Push URLs

Git remote explained: add, set-url, separate push URLs, rename, remove and prune, plus how to avoid leaking tokens in remote…

Read article →
Test Automation

Pytest Mock: mocker.patch, spy, side_effect and autospec Explained

Pytest mock explained: mocker.patch, return_value, side_effect, spy, autospec and PropertyMock, with tested examples for APIs, clocks and retries.

Read article →
Software Testing

System Testing Explained: Types, Process and a Real End-to-End Example

System testing explained: types, entry and exit criteria, the process, and a real end-to-end test suite run against a small…

Read article →
Databases & SQL

SQL Random Rows: How to Select Random Rows Fast and Fairly

SQL random rows: how to select random rows fast and fairly, timing ORDER BY RANDOM, offsets and ID jumps, and…

Read article →
Databases & SQL

SQL Median: 4 Ways to Calculate the Median, With Tested Queries

SQL median explained: four tested ways to calculate the median, including percentile functions, window ranks, offsets and per group medians.

Read article →
Software Development

JavaScript Regex Explained: Patterns, Flags, Groups and Pitfalls

JavaScript regex explained: literals, flags, groups, named groups, lookarounds, matchAll, replace and the catastrophic backtracking trap, tested in Node.

Read article →
Software Development

Python property: Getters, Setters, Validation and cached_property

Python property explained: getters, setters, deleters, validation, read-only and computed attributes, and cached_property, with tested code.

Read article →
Software Development

Python Enum Explained: auto, IntEnum, StrEnum, Flag and Best Practices

Python enum explained with tested examples: Enum, auto, IntEnum, StrEnum, Flag, aliases, unique, JSON serialisation and match statements.

Read article →
Artificial Intelligence

Naive Bayes Classifier Explained: Build a Spam Filter in Python

Naive Bayes classifier explained by building a spam filter in Python: Bayes rule, the naive assumption, Laplace smoothing, log probabilities…

Read article →
Software Development

401 vs 403: Unauthorized vs Forbidden, and When to Use Each

401 vs 403 explained: authentication vs authorization, the WWW-Authenticate header, when to return 404 instead, client reactions and a tested…

Read article →
DevOps

Git Branch Explained: Create, Switch, Rename, Delete and Recover

Git branch explained with tested commands: list, create, switch, rename, delete, merged checks, recovering deleted branches, remote tracking and naming.

Read article →
Test Automation

Pytest raises: Testing Exceptions, match and excinfo

Pytest raises explained with tested examples: match regexes, excinfo, custom and chained exceptions, parametrized error cases, pytest.warns and common mistakes.

Read article →
Software Testing

Non Functional Testing: Types, Metrics and a Real Load Test

Non functional testing explained: 10 key types, measurable requirements, latency percentiles and what a small real load test revealed about…

Read article →
Databases & SQL

SQL ROUND Function: Decimals, Ties, CEIL, FLOOR and Money

SQL ROUND explained with tested queries: decimals, tie rules, the 2.675 float trap, negative digits, CEIL, FLOOR, truncation and rounding…

Read article →
Databases & SQL

Recursive CTE in SQL: Hierarchies, Paths, Series and Cycles

Recursive CTE in SQL explained with tested queries: org charts with levels and paths, walking up a tree, team sizes,…

Read article →
Software Development

JavaScript Deep Copy: structuredClone vs JSON vs Spread, Tested

JavaScript deep copy compared: spread, Object.assign, the JSON trick, structuredClone and a custom copy, tested on Dates, Maps, Sets, BigInt…

Read article →
Artificial Intelligence

K Means Clustering Explained: How It Works, With Python Code

K means clustering explained: the algorithm, a from-scratch NumPy version, choosing k with the elbow and silhouette methods, k-means++, scaling…

Read article →
DevOps

Git Log Explained: 15 Useful Commands and Pretty Formats

Git log explained with 15 tested commands: oneline, graph, filters by author, date, message and code, file history, stats, pretty…

Read article →
Test Automation

Pytest conftest.py Explained: Shared Fixtures, Hooks and Overrides

Pytest conftest.py explained with a tested suite: shared fixtures, scope and teardown, folder overrides, autouse, custom options, markers and useful…

Read article →
Software Testing

Usability Testing: Methods, Metrics, SUS Score and Checklist

Usability testing explained: methods, planning, writing tasks, running sessions, metrics, SUS scoring, how many users you need and a practical…

Read article →
Databases & SQL

SQL UPDATE Statement: Safe Updates, UPDATE With JOIN and RETURNING

The SQL UPDATE statement explained with tested examples: WHERE guards, CASE updates, UPDATE with JOIN, the subquery NULL trap, RETURNING…

Read article →
Databases & SQL

SQL COUNT: COUNT(*) vs COUNT(column) vs COUNT(DISTINCT) Explained

SQL COUNT explained: COUNT(*) vs COUNT(column) vs COUNT(DISTINCT), NULLs and empty strings, conditional counts, HAVING, the LEFT JOIN trap and…

Read article →
Artificial Intelligence

K Nearest Neighbors (KNN) Explained: How It Works, With Python Code

K nearest neighbors explained: the KNN algorithm, a from-scratch NumPy version, choosing k, feature scaling, distance metrics, regression and vector…

Read article →
DevOps

Git Rebase Interactive: Squash, Fixup, Reword and Autosquash

Git rebase interactive explained step by step: pick, reword, squash, fixup, edit, drop, autosquash, conflicts and undoing a rebase with…

Read article →
Software Testing

Alpha vs Beta Testing: Key Differences, Exit Criteria and Checklist

Alpha vs beta testing compared: who tests, what each phase finds, entry and exit criteria, how to run a beta,…

Read article →
Test Automation

Pytest Monkeypatch: 10 Tested Examples for Env Vars, Time and APIs

Pytest monkeypatch explained with 10 passing tests: environment variables, functions, config, time, sleeps, paths, blocking the network and the wrong-target…

Read article →
Artificial Intelligence

Softmax Function Explained: Formula, Python Code and Stable Version

The softmax function explained: formula, worked example, NumPy code, the stable version that avoids NaN, temperature, cross-entropy and softmax vs…

Read article →
DevOps

Git Diff Explained: Staged Changes, Branches and Commits

Git diff explained: read the output, compare unstaged and staged changes, commits and branches, two dots vs three dots, and…

Read article →
Test Automation

Pytest Xdist: Run Tests in Parallel and Cut Test Time

Run pytest tests in parallel with pytest-xdist: install, the -n option, distribution modes, real timings and how to fix shared-state…

Read article →
Software Testing

Verification vs Validation in Software Testing: Example and Table

Verification vs validation in software testing explained with a comparison table and a tested coupon example where every spec test…

Read article →
Artificial Intelligence

Levenshtein Distance: Formula, Worked Table and Python Code

Levenshtein distance explained: the edit operations, the dynamic programming table for kitten and sitting, Python code with two-row memory, normalised…

Read article →
DevOps

Gitignore: Patterns, Negation and Fixing Tracked Files

A tested gitignore guide: pattern rules, leading and trailing slashes, negation with !, the ignored parent folder trap, git check-ignore…

Read article →
Software Testing

Acceptance Criteria: Given When Then Examples and Checklist

How to write acceptance criteria testers can run: rule-based and Given When Then formats, a worked discount code example turned…

Read article →
Test Automation

Python Unittest: Assert Methods, setUp and Mocks With Examples

A practical Python unittest tutorial: TestCase, assert methods, setUp and tearDown, subTest, skips, expected failures and mock, with a test…

Read article →