Non Functional Testing: Types, Metrics and a Real Load Test
Non functional testing checks how well software works: how fast, how secure, how reliable, how usable and how well it scales, rather than what it does. Learn the ten types that matter most, how to write measurable requirements, which metrics to track, and what a small load test revealed about percentiles and saturation.
- Non functional testing measures qualities such as performance, security, reliability, usability, compatibility and scalability.
- Requirements must be measurable: “p95 response under 300 ms at 200 concurrent users”, not “the site should be fast”.
- Look at percentiles, not averages: in our load test the mean was 32 ms while 1 in 100 requests took over 220 ms.
- At 50 concurrent users our small server got slower overall: throughput fell from 292 to 190 requests per second and p99 jumped to about 2 seconds.
A checkout that calculates totals perfectly is still a failure if it takes eight seconds, leaks card data, crashes at lunchtime traffic or cannot be used with a screen reader. Functional tests would pass every one of those systems. This kind of testing is how teams check the qualities users actually feel: speed, safety, stability and ease of use.
This guide explains the main types, how to turn vague wishes into testable targets, and the metrics that matter. To make it concrete, we ran a small load test against a local Python server (Python 3.13.6, standard library only) and show the real numbers.
What is non functional testing?
It evaluates how a system performs its functions under given conditions, as opposed to whether it performs them. Standards such as ISO/IEC 25010 group these qualities into characteristics like performance efficiency, security, reliability, usability, compatibility, maintainability and portability. Non functional requirements are sometimes called quality attributes or “-ilities”.
Functional vs non functional testing
| Aspect | Functional testing | Non functional testing |
|---|---|---|
| Question | Does it do the right thing? | How well does it do it? |
| Example | Coupon applies 10 percent off | Checkout responds in under 300 ms at peak |
| Result | Pass or fail per case | Measurements against thresholds |
| Typical tools | Unit, API and UI tests | Load tools, scanners, profilers, audits |
| Environment | Any test environment | Production-like, often dedicated |
Both are needed. A useful mental model: functional tests protect correctness, non functional tests protect the experience and the business.
10 types of non functional testing
- Performance testing: response times and resource use under expected load.
- Load testing: behaviour at the expected peak number of users.
- Stress testing: pushing beyond capacity to find the breaking point and how the system fails. Our load vs stress testing guide compares the two.
- Scalability testing: whether adding servers or resources increases capacity as expected.
- Endurance (soak) testing: long runs to expose memory leaks and slow degradation.
- Security testing: vulnerabilities, authentication, authorisation and data protection.
- Usability testing: how easily real users complete tasks; see our usability testing guide.
- Accessibility testing: whether people using assistive technology can use the product, usually measured against WCAG.
- Compatibility testing: browsers, devices, operating systems and screen sizes.
- Reliability and recovery testing: uptime, failover, backups and how quickly the system recovers from crashes.
Other lists add installation, localisation, maintainability, portability, volume and compliance testing; one popular guide counts 51 types. Start with the ten above and add others when your product needs them.
Writing measurable non functional requirements
| Vague | Testable |
|---|---|
| The site should be fast. | p95 page response under 300 ms with 200 concurrent users. |
| The app must be secure. | No high or critical findings in the OWASP Top 10 scan; tokens expire after 15 minutes. |
| The service should be reliable. | 99.9 percent monthly availability; recovery within 5 minutes of a node failure. |
| It should work on phones. | Core flows pass on the latest two versions of iOS Safari and Android Chrome at 360 px width. |
A good non functional requirement names a metric, a threshold and the conditions. Write them alongside acceptance criteria, so they are agreed before development rather than argued about before release.
Key non functional metrics
- Latency percentiles: p50 (typical), p95 and p99 (slow tail).
- Throughput: requests or transactions per second.
- Error rate: failed requests as a share of all requests.
- Resource use: CPU, memory, connections, disk and network.
- Availability: uptime percentage over a period.
- Recovery time and recovery point: how long until service returns and how much data could be lost.
Our small load test
We started a local HTTP server where each request did 20 ms of simulated work, and 3 percent of requests took an extra 200 ms, mimicking occasional slow database queries. Then we sent 400 requests at three concurrency levels and recorded every response time:
users | throughput | p50 | p95 | p99 | max | mean
1 | 34.5 req/s | 22.7 ms | 25.2 ms | 223.0 ms | 223.7 ms | 28.9 ms
10 | 292.2 req/s | 24.0 ms | 32.0 ms | 226.6 ms | 227.2 ms | 32.2 ms
50 | 189.7 req/s | 22.9 ms | 540.4 ms | 2048.4 ms | 2048.9 ms | 118.6 ms
This is a deliberately tiny, single-machine test, not a benchmark of anything real, but it shows patterns you will see in every serious load test. Real tests use dedicated tools, production-like environments and much longer runs.
What the load test showed
- Averages hide the tail. At 10 users the mean was 32 ms, yet the p99 was 227 ms: one in a hundred users waited seven times longer. Our injected slow requests were invisible in the average.
- Concurrency raises throughput, up to a point. Going from 1 to 10 users raised throughput about 8.5 times, because requests overlapped.
- Past saturation, everything gets worse. At 50 users throughput dropped to 190 requests per second and the p99 reached about 2 seconds. The basic server and the operating system’s connection queue could not keep up, so requests waited before they were even handled.
- The median stayed flat at about 23 ms throughout, which is why a dashboard showing only p50 would have missed the whole problem.
Tip: always report p95 and p99 alongside throughput and error rate. The tail is what your unluckiest, often most valuable, users experience.
A quick security check
security headers present:
Strict-Transport-Security: None
X-Content-Type-Options: None
Content-Security-Policy: None
server header: BaseHTTP/0.6 Python/3.13.6
Our quick check of the same server found no security headers and a Server header revealing the exact runtime version, information that helps attackers. Checks like this are cheap to automate in every build. Full security testing goes much further, with dependency scanning, dynamic scanners and penetration tests; settings such as CORS and rate limiting belong on the same checklist.
When to run non functional tests
- Early: agree measurable targets during requirements and design.
- Continuously: run small performance smoke tests and security scans in CI on every change.
- Before release: full load, stress and soak tests in a production-like environment.
- In production: monitor real latency percentiles, errors and availability; see monitoring vs observability.
Common tools
| Area | Examples |
|---|---|
| Load and stress | k6, JMeter, Gatling, Locust |
| Security | OWASP ZAP, dependency scanners, static analysis |
| Accessibility | axe, Lighthouse, screen readers |
| Front-end performance | Lighthouse, WebPageTest |
| Reliability | Chaos engineering tools, failover drills |
We did not run these tools for this article; the load test above used only the Python standard library.
Common mistakes
- Testing performance only once, right before launch.
- Reporting averages instead of percentiles.
- Load testing a laptop or a tiny environment and extrapolating to production.
- Vague requirements that nobody can pass or fail.
- Ignoring the load generator’s own limits, which can become the bottleneck.
- Treating security and accessibility as one-off audits.
Non functional testing checklist
- Measurable targets for latency, throughput, errors and availability.
- Realistic load profile based on production traffic.
- Production-like test environment and data volumes.
- Security scans and header checks in CI.
- Accessibility and compatibility checks for key flows.
- Soak test for memory leaks before major releases.
- Recovery drills with measured recovery time.
Non functional testing in agile teams
In short sprints, quality attributes are easy to postpone until a big pre-release test that nobody has time to act on. A better pattern is to treat them as part of the definition of done. Add performance budgets to stories that touch hot paths, such as “search stays under 200 ms at p95”, and check them in a small automated test. Run security scans and dependency checks on every pull request. Schedule a fuller load test each release, and keep its results so you can compare trends rather than single numbers. Assign an owner for each quality area, because shared responsibility often becomes nobody’s job. Finally, feed production data back into the tests: real traffic patterns make the next load test far more realistic.
Final thoughts
Non functional testing protects what users feel: speed, safety, stability and ease of use. Define measurable targets, test continuously as well as before release, and read results through percentiles rather than averages. Our tiny load test showed how a healthy median can hide a slow tail, and how adding load beyond capacity reduces throughput instead of increasing it. Those lessons hold for every system, from a small API to a global platform.
Frequently asked questions
What is non functional testing?
It is testing that checks how well a system works, such as its performance, security, reliability, usability and compatibility, rather than whether individual features give the correct result.
What are the main types of non functional testing?
The most common are performance, load, stress, scalability, endurance, security, usability, accessibility, compatibility and reliability or recovery testing.
What is the difference between functional and non functional testing?
Functional testing checks that features produce the right results. Non functional testing measures qualities such as speed, security and stability against agreed thresholds.
Is performance testing non functional testing?
Yes. Performance testing, along with its load, stress and endurance variants, is the most widely practised form of non functional testing.
How do you write a non functional requirement?
Name a measurable metric, a threshold and the conditions, for example p95 response time under 300 milliseconds with 200 concurrent users on the checkout page.
Keep learning
Need help with testing, WordPress or SEO?
We help teams ship better software and grow their sites. Tell us what you are working on and we will suggest a clear next step.