1. Home
  2. Blog
  3. Software Testing

Software Testing

Pairwise Testing Explained: How All-Pairs Cuts 216 Tests to 13, Tested

Pairwise testing is a test design technique that covers every combination of values for every pair of inputs, without testing every full combination. See how it shrank a checkout matrix from 216 cases to 13, how we verified the coverage, and where it falls short.

9 min read
Pairwise testing explained: how all-pairs testing cuts test cases

Key takeaways

  • Pairwise testing (all-pairs testing) makes sure every value of every input appears with every value of every other input at least once.
  • Our generator covered all 89 value pairs of a five-input checkout in 13 tests, against 216 exhaustive combinations.
  • With 13 inputs of 3 values each, 22 tests covered every pair of a space of 1,594,323 combinations.
  • It finds two-factor bugs reliably but can miss three-factor ones: our set had no Safari + UPI + French test.

Modern software runs on a mess of combinations: browsers, operating systems, languages, payment methods, account types, feature flags. Testing every combination quickly becomes impossible. Five settings with a handful of values each already produce hundreds of cases, and each new setting multiplies the total again. Pairwise testing is a systematic way to cut that number down while keeping the combinations most likely to reveal bugs.

We built a small pairwise generator in Python 3.13, ran it on a realistic checkout example, and wrote a separate check that every pair really is covered. The numbers below come from that run. If you are new to test design techniques, our guides to equivalence partitioning and boundary value analysis are good companions.

What is pairwise testing?

Pairwise testing is a black-box, combinatorial test design technique. For every pair of input parameters, it makes sure each possible combination of their values appears in at least one test. It does not try every combination of all parameters together.

Suppose you test browser and operating system. Pairwise testing guarantees that Chrome on Windows, Chrome on macOS, Safari on Android and every other browser-OS pair appear somewhere. Because one test case holds many pairs at once (browser-OS, browser-language, OS-payment and so on), a small set of tests can cover them all.

You may also hear it called all-pairs testing, or seen as a special case of combinatorial testing with strength 2. Orthogonal array testing is a related, more mathematical way of building such sets.

Why pairs are enough most of the time

The idea rests on an observation from field studies: most defects are triggered by a single input value or by the interaction of two inputs. Research by the US National Institute of Standards and Technology (NIST) on real failures found that the share of bugs needing three or more interacting factors falls off quickly. Pairwise testing aims at the large middle: bugs that appear only when two settings meet, such as a layout that breaks only in Safari when the language is French.

Single-factor bugs are covered automatically, because every value appears at least once. The technique is a bet on probability, not a guarantee, and that matters later.

A checkout example

We modelled the checkout page of an online shop with five parameters:

Parameter Values Count
Browser Chrome, Firefox, Safari 3
Operating system Windows, macOS, Android 3
Payment Card, UPI, Wallet 3
User type Guest, Member 2
Language en, hi, fr, es 4

Testing every combination means 3 × 3 × 3 × 2 × 4 = 216 tests. Across the ten pairs of parameters there are 89 value pairs to cover. The theoretical minimum is 12 tests, because the two largest parameters, language (4) and browser (3), alone form 12 pairs that each need their own row.

The 13 pairwise tests

Our greedy generator repeatedly picks the test case that covers the most still-uncovered pairs, until none remain. It produced 13 tests:

# Browser OS Payment User Language
1 Chrome Windows Card Member en
2 Firefox Windows UPI Guest es
3 Firefox macOS Wallet Member hi
4 Safari macOS Card Guest fr
5 Chrome Android UPI Member fr
6 Safari Android Wallet Guest en
7 Safari Windows UPI Guest hi
8 Chrome macOS Wallet Guest es
9 Safari Android Card Member es
10 Firefox Windows Wallet Guest fr
11 Firefox macOS UPI Guest en
12 Firefox Android Card Guest hi
13 Chrome Windows Card Guest hi

That is a 94% reduction, one test above the theoretical minimum. Some combinations are unrealistic, such as Safari on Windows or Android in row 6. In real projects you add constraints to exclude impossible pairs; dedicated tools support this directly.

Verifying the coverage

Never trust a generated set blindly. We checked it with an independent function that looks for every value pair in the tests:

import itertools

def covered(names, values, tests):
    for i, j in itertools.combinations(range(len(names)), 2):
        for a in values[i]:
            for b in values[j]:
                if not any(t[i] == a and t[j] == b for t in tests):
                    return False
    return True
exhaustive: 216 pairwise: 13 all pairs covered: True pairs: 89

A checker like this is also useful when someone hand-edits a pairwise table later, for example to swap an unrealistic row. One careless change can silently drop a pair.

How pairwise testing scales

The real win shows up as the number of inputs grows. Exhaustive testing grows exponentially, while pairwise sets grow roughly with the logarithm of the number of parameters. We ran our generator on three larger configurations:

Configuration Exhaustive tests Pairwise tests
10 parameters × 2 values 1,024 11
20 parameters × 2 values 1,048,576 16
13 parameters × 3 values 1,594,323 22

All three sets passed the coverage check. Twenty on/off feature flags have over a million combinations, yet 16 tests touched every pair of flag states. Mature tools often produce slightly smaller sets than our simple greedy approach, but the order of magnitude is the same.

What pairwise testing misses

Pairwise testing guarantees pairs, not triples. We simulated two bugs against our 13 tests:

  • A bug that needs Safari + UPI (two factors) was caught, by test 7.
  • A bug that needs Safari + UPI + French (three factors) was missed. No test combined all three.

A third simulated bug, Safari + UPI + Hindi, happened to be caught because test 7 includes Hindi, but that was luck rather than design. If interactions between three settings are risky in your system, raise the strength to 3-way testing. Tools support any strength, at the cost of more tests.

Tip: pairwise testing chooses which combinations to run, not which values. Pick values with equivalence partitioning and boundary analysis first, otherwise you will cover every pair of the wrong inputs.

Pairwise testing step by step

  1. List the parameters that change behaviour: environment, inputs, settings, user roles.
  2. Choose representative values for each, using partitions and boundaries rather than every possible value.
  3. Add constraints for impossible combinations, such as Safari only on macOS and iOS.
  4. Generate the set with a tool, and note its strength (2-way, 3-way).
  5. Verify coverage with the tool’s report or a checker like ours.
  6. Add must-have cases by hand: the most common real-world configuration and any combination that failed before.
  7. Define expected results for each row, then automate where possible, for example as a parameterised test.

Pairwise testing tools

You rarely need to write your own generator. Microsoft’s open-source PICT is a widely used command-line tool with constraint support. In Python, the allpairspy package generates pairs, and NIST’s ACTS tool supports higher strengths. We did not run these tools for this article; our numbers come from our own generator. Whatever you use, feed the result into data-driven tests, as shown in our test data generation guide.

When to use pairwise testing

Use it when you have several independent parameters with a few values each: browser and device matrices, configuration options, form fields with dropdowns, API query parameters and feature flag combinations. Skip it when parameters depend heavily on each other, when a single input has a huge range (use boundary analysis instead), or when the business rules are explicit conditions, which decision table testing handles better. For safety-critical systems, pairwise is a floor, not a ceiling.

Final thoughts

Pairwise testing turns an impossible matrix into a short, defensible list. In our checkout example, 13 tests covered every pair that 216 exhaustive tests would have covered, and the savings grew dramatically with more inputs. Choose good values first, add constraints and must-have cases, verify the coverage, and raise the strength where three-way interactions are a real risk. Then spend the time you saved on the riskiest areas, as described in our risk-based testing guide.

Frequently asked questions

What is pairwise testing in software testing?

It is a combinatorial test design technique that covers every combination of values for every pair of input parameters at least once, using far fewer tests than exhaustive testing.

Is pairwise testing the same as all-pairs testing?

Yes. All-pairs testing is another name for it. Both mean 2-way combinatorial testing, where every pair of parameter values appears in at least one test.

How many test cases does pairwise testing need?

At least the product of the two largest value counts. In our five-parameter checkout that minimum was 12, and our generator produced 13 tests instead of 216.

Can pairwise testing miss bugs?

Yes. It guarantees pairs, so bugs that need three or more specific values together can be missed. Use 3-way testing or targeted cases for those risks.

Which tools generate pairwise test cases?

Common options include Microsoft PICT, NIST ACTS and the Python package allpairspy. Many test management tools also include a pairwise generator.

Keep learning

Work with Stack Quora

Need help with testing, WordPress or SEO?

We help teams ship better software and grow their sites. Tell us what you are working on and we will suggest a clear next step.

Keep reading

More guides on testing, AI & modern development

Enjoyed this article? Pick a topic and jump straight into our newest tutorials.

Artificial Intelligence

BM25 Explained: How Keyword Search Ranks Results, With Python Code and Hybrid Search

BM25 explained with a pure Python implementation we ran: the formula, IDF, term saturation, the k1 and b parameters, why…

Read article →
Software Development

HTTP Methods Explained: GET, POST, PUT, PATCH, DELETE, Tested on a Live Server

HTTP methods explained with a small server we ran: GET, POST, PUT, PATCH, DELETE, HEAD and OPTIONS, safe vs idempotent,…

Read article →
DevOps

Git Bisect Explained: Find the Commit That Broke Your Code, Manually or Automatically

Git bisect explained with a tested repository: find the one bad commit among 41 in 5 test runs, automate it…

Read article →
Test Automation

pytest Markers Explained: Custom Marks, skip, skipif, xfail and -m Filters

pytest markers explained with a tested suite: register custom marks, run only smoke tests with -m, combine filters with and,…

Read article →
Software Testing

Positive vs Negative Testing: Differences, 24 Real Test Cases and a Checklist

Positive vs negative testing explained with a real signup form: 6 positive and 18 negative test cases run in pytest,…

Read article →
Databases & SQL

SQL ORDER BY Multiple Columns: ASC, DESC, NULLS LAST, Case and Custom Order

SQL ORDER BY with multiple columns explained with tested queries: mixed ASC and DESC, tie-breakers, NULLS FIRST and LAST, case-insensitive…

Read article →
Databases & SQL

SQL Running Total: SUM OVER, Per-Group Totals, Moving Sums and the RANGE Trap

How to calculate a SQL running total with SUM() OVER: per-group totals, daily cumulative sums, moving sums, percent of total,…

Read article →
Software Development

JavaScript Sort Array: Numbers, Objects, Strings and Why 10 Comes Before 5

How to sort an array in JavaScript correctly: numbers with a compare function, objects by one or more fields, strings…

Read article →
Software Development

Python String Methods: Cheat Sheet, Real Examples and the strip() Trap

Python string methods with tested examples: split, join, strip, replace, find, startswith, is-methods, casefold, partition and translate, plus the strip()…

Read article →
Software Development

Python zip() Explained: Loop Two Lists at Once, Unzip, strict=True and Traps

Python zip() pairs items from several lists in one loop. Tested examples for dicts, unzipping with *, strict=True, zip_longest, sliding…

Read article →
Artificial Intelligence

Cosine Similarity Explained: Formula, Worked Example and Python Code

Cosine similarity explained simply: the formula, a worked example by hand, pure Python and NumPy code, cosine vs Euclidean distance…

Read article →
Software Development

UTF-8 Encoding Explained: How Text Becomes Bytes and How to Fix Garbled Text

UTF-8 encoding explained with byte-level examples we ran in Python: how characters become 1 to 4 bytes, why café turns…

Read article →
DevOps

Git Squash Commits: 4 Easy Ways, Tested Step by Step (rebase -i, reset, merge)

Git squash commits explained with 4 tested methods: interactive rebase, reset –soft, merge –squash and fixup with autosquash. Before-and-after logs,…

Read article →
Test Automation

pytest parametrize Explained: 7 Patterns with Tested Examples

pytest parametrize explained with 7 tested patterns: basic cases, readable ids, stacked decorators, pytest.param with xfail and skip, indirect fixtures…

Read article →
Software Testing

Retesting vs Regression Testing: 8 Key Differences with a Real Bug Fix Example

Retesting vs regression testing explained with a real bug fix run in pytest: retesting confirmed the fix, the regression run…

Read article →
Databases & SQL

Delete Duplicate Rows in SQL: 5 Safe Methods, Tested Step by Step

How to delete duplicate rows in SQL safely: find them, preview them, then remove them with ROW_NUMBER, MIN(id), EXISTS, rowid…

Read article →
Databases & SQL

SQL RANK vs DENSE_RANK vs ROW_NUMBER: Differences with Tested Examples

SQL RANK vs DENSE_RANK vs ROW_NUMBER in one table: how each handles ties, top-N per group, the second-highest salary trap…

Read article →
Software Development

JavaScript Optional Chaining (?.) Explained: 8 Examples, ?? Defaults and Traps

JavaScript optional chaining stops “Cannot read properties of undefined” errors. 8 tested examples for properties, arrays and methods, ?? vs…

Read article →
Artificial Intelligence

RAG Chunking Strategies Explained: Fixed, Overlap, Sentence, Recursive and Headings, Measured

RAG chunking strategies compared on one document with tiktoken 0.14: fixed-size, overlap, sentence packing, recursive splitting and heading-based chunks, measuring…

Read article →
DevOps

Git Stash Explained: Save, Apply, Pop and Untracked Files, Tested

Git stash explained with commands we ran on Git 2.56: push with a message, untracked files, list and show, apply…

Read article →
Software Testing

Pairwise Testing Explained: How All-Pairs Cuts 216 Tests to 13, Tested

Pairwise testing explained with a generator we ran in Python 3.13: a checkout with 216 combinations covered by 13 tests,…

Read article →
Test Automation

Test Doubles Explained: Dummy, Stub, Spy, Mock and Fake, with Tested Python Code

Test doubles explained with one Python example run in pytest 9.2: dummy, stub, spy, mock and fake, when to use…

Read article →
Databases & SQL

SQL COALESCE Explained: Defaults, NULLIF, IFNULL vs ISNULL and Traps, Tested

SQL COALESCE explained with queries we ran in SQLite 3.50: first non-null values, empty strings, NULL in maths, NULLIF for…

Read article →
Databases & SQL

SQL Self Join Explained: Employee-Manager, Pairs and Row Comparisons, Tested

SQL self join explained with queries we ran in SQLite 3.50: employee and manager, earning more than the manager, unique…

Read article →
Artificial Intelligence

LLM Context Window Explained: Tokens, Limits and How to Fit More, Measured

The LLM context window explained with token counts we measured using tiktoken: words vs tokens for prose, code and Hindi,…

Read article →
DevOps

Git Cherry-Pick Explained: Copy the Commit You Need, Safely, Tested

Git cherry-pick explained with commands we ran in Git 2.56: copying a hotfix to a release branch, new hashes and…

Read article →
Test Automation

Snapshot Testing Explained: How It Works, When It Helps and When It Hurts

Snapshot testing explained with a pytest and syrupy demo we ran: creating snapshots, reading the diff after a code change,…

Read article →
Software Testing

Mutation Testing Explained: Why 100% Coverage Still Missed Our Bug

Mutation testing explained with a small mutation tool we built and ran: 14 mutants, a test suite with 100% line…

Read article →
Artificial Intelligence

Prompt Injection Explained: How AI Apps Get Tricked and How to Defend Them

Prompt injection explained for builders: direct and indirect attacks, why keyword filters fail (our test blocked 1 of 4), and…

Read article →
DevOps

DNS Records Explained: A, AAAA, CNAME, MX, TXT and TTL With Real Lookups

DNS records explained with lookups we ran: how resolution works, A and AAAA, CNAME rules, MX and null MX, TXT…

Read article →
DevOps

Feature Flags Explained: Safe Releases, Gradual Rollouts and Flag Hygiene

Feature flags explained with a rollout simulation we ran: flag types, percentage rollouts with stable hashing, kill switches, allow lists,…

Read article →
Test Automation

Test Data Generation With Python Faker: Realistic, Repeatable and Safe Data

Test data generation with Python Faker, run and measured: seeding for repeatable data, locales, unique values, domain-shaped records, edge cases…

Read article →
Software Testing

Shift-Left Testing Explained: Find Bugs Earlier Without Slowing the Team

Shift-left testing explained: what it means, why early defects are cheaper, practices from requirement reviews to CI checks, the tester's…

Read article →
Artificial Intelligence

LLM Structured Output Explained: Getting Reliable JSON From AI Models

LLM structured output explained with validation we ran in Pydantic: JSON mode and schemas, constrained decoding, the ways model output…

Read article →
Test Automation

API Contract Testing Explained: Catch Breaking Changes Before They Ship

API contract testing explained with a schema check we ran: what a contract is, consumer-driven contracts, Pact, OpenAPI, which changes…

Read article →
Software Testing

Risk-Based Testing Explained: Score Risks, Build a Matrix and Test What Matters

Risk-based testing explained step by step: identifying risks, scoring likelihood and impact, building a risk matrix, mapping risk to test…

Read article →