1. Home
  2. Blog
  3. Software Testing

Software Testing

Non Functional Testing: Types, Metrics and a Real Load Test

Non functional testing checks how well software works: how fast, how secure, how reliable, how usable and how well it scales, rather than what it does. Learn the ten types that matter most, how to write measurable requirements, which metrics to track, and what a small load test revealed about percentiles and saturation.

9 min read
Non functional testing: types, metrics and a real load test

Key takeaways

  • Non functional testing measures qualities such as performance, security, reliability, usability, compatibility and scalability.
  • Requirements must be measurable: “p95 response under 300 ms at 200 concurrent users”, not “the site should be fast”.
  • Look at percentiles, not averages: in our load test the mean was 32 ms while 1 in 100 requests took over 220 ms.
  • At 50 concurrent users our small server got slower overall: throughput fell from 292 to 190 requests per second and p99 jumped to about 2 seconds.

A checkout that calculates totals perfectly is still a failure if it takes eight seconds, leaks card data, crashes at lunchtime traffic or cannot be used with a screen reader. Functional tests would pass every one of those systems. This kind of testing is how teams check the qualities users actually feel: speed, safety, stability and ease of use.

This guide explains the main types, how to turn vague wishes into testable targets, and the metrics that matter. To make it concrete, we ran a small load test against a local Python server (Python 3.13.6, standard library only) and show the real numbers.

What is non functional testing?

It evaluates how a system performs its functions under given conditions, as opposed to whether it performs them. Standards such as ISO/IEC 25010 group these qualities into characteristics like performance efficiency, security, reliability, usability, compatibility, maintainability and portability. Non functional requirements are sometimes called quality attributes or “-ilities”.

Functional vs non functional testing

Aspect Functional testing Non functional testing
Question Does it do the right thing? How well does it do it?
Example Coupon applies 10 percent off Checkout responds in under 300 ms at peak
Result Pass or fail per case Measurements against thresholds
Typical tools Unit, API and UI tests Load tools, scanners, profilers, audits
Environment Any test environment Production-like, often dedicated

Both are needed. A useful mental model: functional tests protect correctness, non functional tests protect the experience and the business.

10 types of non functional testing

  1. Performance testing: response times and resource use under expected load.
  2. Load testing: behaviour at the expected peak number of users.
  3. Stress testing: pushing beyond capacity to find the breaking point and how the system fails. Our load vs stress testing guide compares the two.
  4. Scalability testing: whether adding servers or resources increases capacity as expected.
  5. Endurance (soak) testing: long runs to expose memory leaks and slow degradation.
  6. Security testing: vulnerabilities, authentication, authorisation and data protection.
  7. Usability testing: how easily real users complete tasks; see our usability testing guide.
  8. Accessibility testing: whether people using assistive technology can use the product, usually measured against WCAG.
  9. Compatibility testing: browsers, devices, operating systems and screen sizes.
  10. Reliability and recovery testing: uptime, failover, backups and how quickly the system recovers from crashes.

Other lists add installation, localisation, maintainability, portability, volume and compliance testing; one popular guide counts 51 types. Start with the ten above and add others when your product needs them.

Writing measurable non functional requirements

Vague Testable
The site should be fast. p95 page response under 300 ms with 200 concurrent users.
The app must be secure. No high or critical findings in the OWASP Top 10 scan; tokens expire after 15 minutes.
The service should be reliable. 99.9 percent monthly availability; recovery within 5 minutes of a node failure.
It should work on phones. Core flows pass on the latest two versions of iOS Safari and Android Chrome at 360 px width.

A good non functional requirement names a metric, a threshold and the conditions. Write them alongside acceptance criteria, so they are agreed before development rather than argued about before release.

Key non functional metrics

  • Latency percentiles: p50 (typical), p95 and p99 (slow tail).
  • Throughput: requests or transactions per second.
  • Error rate: failed requests as a share of all requests.
  • Resource use: CPU, memory, connections, disk and network.
  • Availability: uptime percentage over a period.
  • Recovery time and recovery point: how long until service returns and how much data could be lost.

Our small load test

We started a local HTTP server where each request did 20 ms of simulated work, and 3 percent of requests took an extra 200 ms, mimicking occasional slow database queries. Then we sent 400 requests at three concurrency levels and recorded every response time:

users | throughput   | p50     | p95     | p99       | max       | mean
    1 |  34.5 req/s  | 22.7 ms | 25.2 ms |  223.0 ms |  223.7 ms |  28.9 ms
   10 | 292.2 req/s  | 24.0 ms | 32.0 ms |  226.6 ms |  227.2 ms |  32.2 ms
   50 | 189.7 req/s  | 22.9 ms | 540.4 ms | 2048.4 ms | 2048.9 ms | 118.6 ms

This is a deliberately tiny, single-machine test, not a benchmark of anything real, but it shows patterns you will see in every serious load test. Real tests use dedicated tools, production-like environments and much longer runs.

What the load test showed

  • Averages hide the tail. At 10 users the mean was 32 ms, yet the p99 was 227 ms: one in a hundred users waited seven times longer. Our injected slow requests were invisible in the average.
  • Concurrency raises throughput, up to a point. Going from 1 to 10 users raised throughput about 8.5 times, because requests overlapped.
  • Past saturation, everything gets worse. At 50 users throughput dropped to 190 requests per second and the p99 reached about 2 seconds. The basic server and the operating system’s connection queue could not keep up, so requests waited before they were even handled.
  • The median stayed flat at about 23 ms throughout, which is why a dashboard showing only p50 would have missed the whole problem.

Tip: always report p95 and p99 alongside throughput and error rate. The tail is what your unluckiest, often most valuable, users experience.

A quick security check

security headers present:
  Strict-Transport-Security: None
  X-Content-Type-Options:    None
  Content-Security-Policy:   None
server header: BaseHTTP/0.6 Python/3.13.6

Our quick check of the same server found no security headers and a Server header revealing the exact runtime version, information that helps attackers. Checks like this are cheap to automate in every build. Full security testing goes much further, with dependency scanning, dynamic scanners and penetration tests; settings such as CORS and rate limiting belong on the same checklist.

When to run non functional tests

  • Early: agree measurable targets during requirements and design.
  • Continuously: run small performance smoke tests and security scans in CI on every change.
  • Before release: full load, stress and soak tests in a production-like environment.
  • In production: monitor real latency percentiles, errors and availability; see monitoring vs observability.

Common tools

Area Examples
Load and stress k6, JMeter, Gatling, Locust
Security OWASP ZAP, dependency scanners, static analysis
Accessibility axe, Lighthouse, screen readers
Front-end performance Lighthouse, WebPageTest
Reliability Chaos engineering tools, failover drills

We did not run these tools for this article; the load test above used only the Python standard library.

Common mistakes

  • Testing performance only once, right before launch.
  • Reporting averages instead of percentiles.
  • Load testing a laptop or a tiny environment and extrapolating to production.
  • Vague requirements that nobody can pass or fail.
  • Ignoring the load generator’s own limits, which can become the bottleneck.
  • Treating security and accessibility as one-off audits.

Non functional testing checklist

  • Measurable targets for latency, throughput, errors and availability.
  • Realistic load profile based on production traffic.
  • Production-like test environment and data volumes.
  • Security scans and header checks in CI.
  • Accessibility and compatibility checks for key flows.
  • Soak test for memory leaks before major releases.
  • Recovery drills with measured recovery time.

Non functional testing in agile teams

In short sprints, quality attributes are easy to postpone until a big pre-release test that nobody has time to act on. A better pattern is to treat them as part of the definition of done. Add performance budgets to stories that touch hot paths, such as “search stays under 200 ms at p95”, and check them in a small automated test. Run security scans and dependency checks on every pull request. Schedule a fuller load test each release, and keep its results so you can compare trends rather than single numbers. Assign an owner for each quality area, because shared responsibility often becomes nobody’s job. Finally, feed production data back into the tests: real traffic patterns make the next load test far more realistic.

Final thoughts

Non functional testing protects what users feel: speed, safety, stability and ease of use. Define measurable targets, test continuously as well as before release, and read results through percentiles rather than averages. Our tiny load test showed how a healthy median can hide a slow tail, and how adding load beyond capacity reduces throughput instead of increasing it. Those lessons hold for every system, from a small API to a global platform.

Frequently asked questions

What is non functional testing?

It is testing that checks how well a system works, such as its performance, security, reliability, usability and compatibility, rather than whether individual features give the correct result.

What are the main types of non functional testing?

The most common are performance, load, stress, scalability, endurance, security, usability, accessibility, compatibility and reliability or recovery testing.

What is the difference between functional and non functional testing?

Functional testing checks that features produce the right results. Non functional testing measures qualities such as speed, security and stability against agreed thresholds.

Is performance testing non functional testing?

Yes. Performance testing, along with its load, stress and endurance variants, is the most widely practised form of non functional testing.

How do you write a non functional requirement?

Name a measurable metric, a threshold and the conditions, for example p95 response time under 300 milliseconds with 200 concurrent users on the checkout page.

Keep learning

Work with Stack Quora

Need help with testing, WordPress or SEO?

We help teams ship better software and grow their sites. Tell us what you are working on and we will suggest a clear next step.

Keep reading

More guides on testing, AI & modern development

Enjoyed this article? Pick a topic and jump straight into our newest tutorials.

Artificial Intelligence

Random Forest in Python: Bagging, Feature Sampling and OOB From Scratch

Random forest in Python from scratch: bootstrap sampling, feature sampling, number of trees, out-of-bag evaluation and impurity vs permutation importance.

Read article →
Software Development

MIME Types Explained: Content-Type, nosniff and a Common Types List

MIME types explained and tested in a browser: Content-Type, nosniff, scripts and stylesheets with wrong types, a common MIME types…

Read article →
DevOps

Git Clean Explained: Remove Untracked Files Without Regret

Git clean explained with dry runs: -n, -f, -d, -x, -X, exclusions, paths, nested repositories and interactive mode, plus a…

Read article →
Test Automation

Pytest approx: Comparing Floats With rel, abs and Collections

Pytest approx explained: default tolerances, the near-zero trap, rel and abs, lists and dicts, NaN, NumPy, and how it differs…

Read article →
Software Testing

Accessibility Testing: WCAG Checklist, Automated Tools and Manual Checks

Accessibility testing explained: WCAG basics, an axe-core scan of a broken and a fixed page, what automation missed, contrast checks…

Read article →
Databases & SQL

SQL UNPIVOT: Turn Columns Into Rows, 4 Ways Tested

SQL UNPIVOT explained: turn columns into rows with UNION ALL, CROSS JOIN with CASE, JSON functions or native UNPIVOT, with…

Read article →
Software Development

Intl NumberFormat in JavaScript: Currency, Compact, Percent and Units

Intl.NumberFormat in JavaScript: locales, currency, compact notation, percent, units, rounding, formatToParts, the non-breaking space trap and formatter reuse.

Read article →
Software Development

Python Abstract Class: ABC, abstractmethod and When to Use Them

Python abstract class explained: ABC and abstractmethod, abstract properties, shared logic with super, register, collections.abc and ABC vs Protocol, tested.

Read article →
Software Development

Python Subprocess Explained: run, Capture Output, Timeouts and Safety

Python subprocess explained with tested examples: run, capture_output, check, timeouts, input, environment, shell injection, encoding on Windows and Popen streaming.

Read article →
Databases & SQL

SQL EXPLAIN Plan: How to Read Query Plans and Fix Slow Queries

SQL EXPLAIN plan explained: read scans, index searches, covering indexes and temporary sorts, with before and after timings for five…

Read article →
Artificial Intelligence

Decision Tree in Python: Gini, Entropy and Overfitting From Scratch

Decision tree in Python from scratch: Gini impurity, entropy, information gain, best splits, readable rules, and how depth and leaf…

Read article →
Software Development

URL Query Parameters: Encoding, URLSearchParams and Common Traps

URL query parameters explained: reading and building them with URLSearchParams and urllib, percent encoding, plus signs, repeated keys and nine…

Read article →
DevOps

Git Blame Explained: Find Who Changed a Line, and Why

Git blame explained with a real repository: ignore whitespace, line ranges and functions, ignore-revs files for formatting commits, moved code…

Read article →
Test Automation

Pytest Benchmark: Measure Speed and Catch Performance Regressions

Pytest benchmark tutorial: the benchmark fixture, reading results, parametrized and pedantic benchmarks, saving baselines and failing the build on regressions.

Read article →
Software Testing

Defect Density: Formula, Worked Examples and How to Use It

Defect density explained: the formula, a five-module worked example, per KLOC vs per function point, severity weighting, defect removal efficiency…

Read article →
Databases & SQL

Case Insensitive Search in SQL: LOWER, LIKE, COLLATE and Indexes

Case insensitive search in SQL: LOWER, LIKE, COLLATE NOCASE and expression indexes compared with query plans, plus accented letters and…

Read article →
Databases & SQL

ON DELETE CASCADE in SQL: CASCADE vs SET NULL vs RESTRICT

ON DELETE CASCADE in SQL tested: CASCADE vs SET NULL vs RESTRICT, multi-level cascades, the SQLite foreign key pragma, row…

Read article →
Software Development

JavaScript BigInt Explained: Big Numbers, Division and JSON

JavaScript BigInt explained: the 2^53 limit, creating and converting BigInt, division, mixing types, JSON stringify and parse without losing digits,…

Read article →
Artificial Intelligence

Logistic Regression in Python: Sigmoid, Log Loss and Gradient Descent From Scratch

Logistic regression in Python from scratch: sigmoid, log loss, gradient descent, odds ratios, thresholds, sigmoid overflow and separable data, tested…

Read article →
DevOps

Git Worktree Explained: Work on Two Branches at Once

Git worktree explained: add, list, remove, prune, lock and detached worktrees, a hotfix workflow, and the errors you will hit,…

Read article →
Test Automation

Pytest ini Explained: addopts, markers, testpaths and Precedence

Pytest ini explained: testpaths, addopts, strict markers, xfail_strict, filterwarnings, minversion, config file precedence and command-line overrides, tested.

Read article →
Software Testing

Test Estimation Techniques: WBS, Three Point PERT and Monte Carlo

Test estimation techniques worked through: WBS, three point PERT, total uncertainty, a Monte Carlo check, percentage rules and Wideband Delphi.

Read article →
Databases & SQL

Gaps and Islands in SQL: Find Streaks and Missing Ranges

Gaps and islands in SQL: find login streaks and missing ranges with ROW_NUMBER, DENSE_RANK, LAG and LEAD, plus the duplicate…

Read article →
Databases & SQL

INSERT INTO SELECT in SQL: Copy Rows Between Tables Safely

INSERT INTO SELECT in SQL: copy rows between tables with chosen columns, joins and summaries, avoid duplicates on reruns, upsert,…

Read article →
Artificial Intelligence

Linear Regression in Python: Least Squares From Scratch, With R Squared

Linear regression in Python from scratch: closed form, NumPy least squares and gradient descent, R squared, RMSE, outliers and curved…

Read article →
DevOps

Git Remote Explained: Add, Set-URL, Rename, Remove and Push URLs

Git remote explained: add, set-url, separate push URLs, rename, remove and prune, plus how to avoid leaking tokens in remote…

Read article →
Test Automation

Pytest Mock: mocker.patch, spy, side_effect and autospec Explained

Pytest mock explained: mocker.patch, return_value, side_effect, spy, autospec and PropertyMock, with tested examples for APIs, clocks and retries.

Read article →
Software Testing

System Testing Explained: Types, Process and a Real End-to-End Example

System testing explained: types, entry and exit criteria, the process, and a real end-to-end test suite run against a small…

Read article →
Artificial Intelligence

Naive Bayes Classifier Explained: Build a Spam Filter in Python

Naive Bayes classifier explained by building a spam filter in Python: Bayes rule, the naive assumption, Laplace smoothing, log probabilities…

Read article →
DevOps

Git Branch Explained: Create, Switch, Rename, Delete and Recover

Git branch explained with tested commands: list, create, switch, rename, delete, merged checks, recovering deleted branches, remote tracking and naming.

Read article →
Test Automation

Pytest raises: Testing Exceptions, match and excinfo

Pytest raises explained with tested examples: match regexes, excinfo, custom and chained exceptions, parametrized error cases, pytest.warns and common mistakes.

Read article →
Software Testing

Non Functional Testing: Types, Metrics and a Real Load Test

Non functional testing explained: 10 key types, measurable requirements, latency percentiles and what a small real load test revealed about…

Read article →
Artificial Intelligence

K Means Clustering Explained: How It Works, With Python Code

K means clustering explained: the algorithm, a from-scratch NumPy version, choosing k with the elbow and silhouette methods, k-means++, scaling…

Read article →
DevOps

Git Log Explained: 15 Useful Commands and Pretty Formats

Git log explained with 15 tested commands: oneline, graph, filters by author, date, message and code, file history, stats, pretty…

Read article →
Test Automation

Pytest conftest.py Explained: Shared Fixtures, Hooks and Overrides

Pytest conftest.py explained with a tested suite: shared fixtures, scope and teardown, folder overrides, autouse, custom options, markers and useful…

Read article →
Software Testing

Usability Testing: Methods, Metrics, SUS Score and Checklist

Usability testing explained: methods, planning, writing tasks, running sessions, metrics, SUS scoring, how many users you need and a practical…

Read article →