1. Home
  2. Blog
  3. Test Automation

Test Automation

Pytest Coverage With pytest-cov: Reports, Branch Gaps and CI Gates

Pytest coverage shows which lines and branches your tests actually run. Learn the pytest-cov commands, read the missing-lines report, switch on branch coverage, generate HTML, set a fail-under gate for CI, and see why 100% line coverage can still miss a branch.

8 min read
Pytest coverage with pytest-cov: reports and branch gaps

Key takeaways

  • Run pytest coverage with pytest --cov=yourpackage --cov-report=term-missing to see the percentage and the exact lines never run.
  • Add --cov-branch. In our demo, line coverage said 100% while branch coverage was 86% and pointed at the missed path.
  • --cov-fail-under=90 makes pytest exit with code 1 when coverage drops, which fails a CI build.
  • Coverage shows what ran, not what was checked. A test with no assertions still counts.

Your test suite is green. But does it test the error handling, the international shipping path, or that rarely used discount rule? Pytest coverage answers that question by recording which parts of your code ran during the tests. The standard tool is the pytest-cov plugin, a thin wrapper around coverage.py.

We ran every command below with pytest 9.2, pytest-cov 7.1.0 and coverage.py 7.17.3 on Python 3.13.6. The outputs are copied from those runs. If you are new to pytest, our pytest tutorial covers the basics first.

What does code coverage measure?

Coverage tools watch your program while tests run and record what executed. The two measures that matter in Python are:

  • Line (statement) coverage: the share of executable lines that ran at least once.
  • Branch coverage: the share of possible paths through each decision, such as both the true and false outcomes of an if, that were taken.

Our code coverage guide explains the theory and other types. Here we stay practical.

Install pytest-cov

pip install pytest-cov

This installs coverage.py as a dependency and adds the --cov family of options to pytest. Install it into your project’s virtual environment so CI and your laptop use the same version.

Your first coverage report

Here is a small module with a shipping rule:

# shipping.py
def shipping_cost(weight_kg, express=False, country="IN"):
    if weight_kg <= 0:
        raise ValueError("weight must be positive")
    cost = 40 if weight_kg <= 1 else 40 + (weight_kg - 1) * 25
    if express:
        cost *= 2
    if country != "IN":
        cost += 300
    return round(cost, 2)

def free_shipping(total):
    if total >= 499:
        return True
    return False

And three tests that look reasonable:

def test_light_parcel():
    assert shipping_cost(0.5) == 40
def test_express_heavy():
    assert shipping_cost(3, express=True) == 180
def test_free_shipping():
    assert free_shipping(600) is True

Run them with coverage:

pytest --cov=shipping --cov-report=term-missing
Name          Stmts   Miss  Cover   Missing
-------------------------------------------
shipping.py      13      3    77%   3, 8, 14
-------------------------------------------
TOTAL            13      3    77%

Reading the missing lines

The Missing column is the most useful part of the report. Each number is a line that never ran:

  • Line 3: the ValueError for zero or negative weight. No test sends a bad weight.
  • Line 8: the international surcharge. Every test ships within India.
  • Line 14: return False. No test checks a small order.

So 77% is not an abstract number. It is three named gaps, each a real behaviour nobody has verified. Pass --cov= the package or module you are testing, not your tests folder, or the tests themselves inflate the figure.

Turn on branch coverage

pytest --cov=shipping --cov-branch --cov-report=term-missing
Name          Stmts   Miss Branch BrPart  Cover   Missing
---------------------------------------------------------
shipping.py      13      3      8      3    71%   3, 8, 14
---------------------------------------------------------
TOTAL            13      3      8      3    71%

Two new columns appear. Branch counts possible branch outcomes (8 here), and BrPart counts decisions where only some outcomes were taken (3 here). The total drops to 71% because branch coverage is stricter. We recommend always switching it on.

Why 100% line coverage can still miss a branch

This tiny function shows the reason:

# label.py
def label(order):
    tag = "standard"
    if order.get("express"):
        tag = "express"
    return tag

# test_label.py
def test_express():
    assert label({"express": True}) == "express"

Line coverage only:

label.py       5      0   100%

With --cov-branch:

Name       Stmts   Miss Branch BrPart  Cover   Missing
label.py       5      0      2      1    86%   3->5

Every line ran, so line coverage reports 100%. But the if on line 3 was never false, so the jump straight from line 3 to line 5 (shown as 3->5) never happened. The standard order path is untested. Bugs love these invisible paths, such as an if without an else that should have set a default.

Enforce a minimum with fail-under

pytest --cov=shipping --cov-branch --cov-fail-under=90
FAIL Required test coverage of 90% not reached. Total coverage: 71.43%
3 passed in 0.22s

All three tests passed, yet the command exited with code 1, which we confirmed with echo $?. That is exactly what you want in CI: the build fails when coverage slips. We then added three tests for the gaps, a zero weight with pytest.raises(ValueError), an international parcel and a small order:

Name          Stmts   Miss Branch BrPart  Cover   Missing
---------------------------------------------------------
shipping.py      13      0      8      0   100%
---------------------------------------------------------
TOTAL            13      0      8      0   100%
Required test coverage of 90% reached. Total coverage: 100.00%
6 passed in 0.24s

Put the settings in pyproject.toml

Typing long flags gets old. Move them into configuration so every run, local or CI, behaves the same:

[tool.pytest.ini_options]
addopts = "--cov=shipping --cov-branch --cov-report=term-missing --cov-report=html"

[tool.coverage.report]
fail_under = 90
show_missing = true
exclude_also = ["if __name__ == .__main__.:"]

With this file in place, a plain pytest printed the report, wrote the HTML folder and confirmed Required test coverage of 90.0% reached. The [tool.coverage.*] sections are read by coverage.py itself, so they also apply if you run coverage run -m pytest without the plugin.

HTML and XML reports

  • --cov-report=html writes htmlcov/index.html. Open it in a browser to see each file with missed lines in red and partial branches in yellow. It is the fastest way to explore gaps.
  • --cov-report=xml writes coverage.xml in Cobertura format, which CI dashboards and services such as Codecov and SonarQube read.
  • --cov-report= with no value turns off the terminal report when you only want files.

You can pass --cov-report several times to get several formats in one run.

Excluding code on purpose

Some lines are not worth testing, such as debug helpers or if __name__ == "__main__": blocks. Exclude them explicitly:

  • Add # pragma: no cover to a line or block header.
  • Use exclude_also in config for patterns, as shown above.
  • Use omit under [tool.coverage.run] to skip whole files, such as migrations.

Be honest with exclusions. Excluding hard code to hit a target hides risk instead of removing it.

Tip: running tests in parallel with pytest-xdist? pytest-cov combines the data from each worker for you, so the same --cov flags still work.

Coverage in CI

A typical GitHub Actions step is a single command:

- run: pytest --cov=src --cov-branch --cov-report=xml --cov-fail-under=85

The step fails when coverage falls below 85%, and the XML file can be uploaded as an artifact or sent to a coverage service for pull request comments. Our GitHub Actions CI/CD guide shows the full workflow file.

What coverage target should you set?

There is no magic number. Many teams pick 80% to 90% for application code and higher for critical libraries. More important than the number:

  1. Do not let it fall. Set fail-under near today’s figure and raise it over time.
  2. Read the missing lines in every pull request, especially error handling.
  3. Remember coverage is not correctness. A test that calls a function without asserting anything still counts as coverage. Mutation testing checks whether your assertions would actually catch a bug.

Common pytest-cov problems

“Module was never imported” and “No data was collected”

We misspelled the target as --cov=shiping and got two warnings:

CoverageWarning: Module shiping was never imported. (module-not-imported)
CoverageWarning: No data was collected. (no-data-collected)

The tests still passed, so this is easy to miss in a long CI log. Check that the --cov value matches the importable package name, not a folder path with slashes. In a src/ layout, pass the package name, for example --cov=myapp.

Code imported before coverage starts

Lines that run at import time, such as module-level constants, can show as missed if something imports your package before pytest-cov starts measuring. A common cause is another plugin or a top-level import in conftest.py. pytest-cov starts as early as it can, but if the numbers look oddly low for module-level lines, check the import order.

Different numbers locally and in CI

Usually the two runs test different files, use different settings or skip different tests by marker. Keep every option in pyproject.toml instead of the CI command line, and see our pytest markers guide for how skipped groups affect totals.

Final thoughts

Pytest coverage with pytest-cov takes one install and one flag, and it turns “I think we test that” into a list of exact line numbers. Always add --cov-branch, read the Missing column, store settings in pyproject.toml, and use --cov-fail-under to stop coverage from quietly slipping. Then use those reports to find untested behaviour, not just to chase a percentage.

FAQ

How do I check code coverage in pytest?

Install pytest-cov and run pytest with the cov option pointing at your package, plus cov-report term-missing to list the lines that never ran.

What is the difference between line and branch coverage?

Line coverage counts lines that ran. Branch coverage also checks that each decision went both ways, so it catches untested paths that line coverage misses.

How do I fail a build when coverage is too low?

Add cov-fail-under with a percentage, or set fail_under in the coverage config. Pytest then exits with code 1 when total coverage is below the limit.

Does 100 percent coverage mean my code has no bugs?

No. Coverage shows what ran, not whether results were checked. Weak assertions or missing cases can still hide bugs.

How do I exclude code from coverage?

Add a pragma no cover comment to the line, use exclude_also patterns in the report config, or omit whole files in the run config.

Keep learning

Work with Stack Quora

Need help with testing, WordPress or SEO?

We help teams ship better software and grow their sites. Tell us what you are working on and we will suggest a clear next step.

Keep reading

More guides on testing, AI & modern development

Enjoyed this article? Pick a topic and jump straight into our newest tutorials.

Artificial Intelligence

Jaccard Similarity: Formula, Python, MinHash

Jaccard similarity measures overlap between two sets as intersection over union. Work the formula, compute it in Python for words…

Read article →
Software Development

Unix Timestamp Explained: Seconds vs ms

A Unix timestamp counts seconds since 1 January 1970 UTC. Learn seconds vs milliseconds, converting in Python, JavaScript and SQL,…

Read article →
DevOps

Git Hooks: Pre-commit and Commit-msg Examples

Git hooks run scripts at key moments such as commit and push. See tested pre-commit, commit-msg and prepare-commit-msg hooks, sharing…

Read article →
Test Automation

API Testing in Python: Pytest + Requests Suite

Build an API testing suite in Python with pytest and requests: fixtures, status codes, headers, JSON schema validation, CRUD flows,…

Read article →
Software Testing

Test Design Techniques: 9 Methods + Examples

Test design techniques help you write fewer tests that catch more bugs. Learn 9 black-box, white-box and experience-based methods with…

Read article →
Databases & SQL

SQL Temporary Table: Create, Use and Drop It

SQL temporary tables store interim results for one session. Learn syntax for SQL Server, PostgreSQL, MySQL, Oracle and SQLite, session…

Read article →
Databases & SQL

SQL CROSS JOIN: Every Combination + Examples

SQL CROSS JOIN returns every combination of rows from two tables. See the row math, product variants, calendar grids that…

Read article →
Software Development

JavaScript Date: Format, Parse, Timezone Bugs

JavaScript Date explained with tested Node.js code: zero-based months, the date-only UTC parsing trap, toISOString, getTimezoneOffset, Intl formatting, date math…

Read article →
Software Development

Python Requests: GET, POST, Timeouts, Retries

Python requests explained with code tested against a local API: GET with params, POST json vs data, headers and auth,…

Read article →
Software Development

Python Deepcopy vs Copy: Fix Nested Lists

Python deepcopy vs shallow copy explained with tested code: aliasing, nested lists, the [[0]*3]*3 trap, memo and cycles, custom __deepcopy__,…

Read article →
Artificial Intelligence

TF-IDF Explained: Formula and Worked Example

TF-IDF scores a word by how often it appears in a document and how rare it is across all documents….

Read article →
Software Development

Base64 Encoding Explained, With Python Code

Base64 encoding turns bytes into 64 safe text characters. See the bit-level steps, padding, URL-safe base64, the 33% size cost,…

Read article →
DevOps

Git Tag: Create, Push and Delete Release Tags

Git tag explained with tested commands: annotated vs lightweight tags, listing and sorting, pushing with follow-tags, deleting local and remote…

Read article →
Test Automation

Pytest Coverage: pytest-cov and Branch Gaps

Measure pytest coverage with pytest-cov: term-missing reports, branch coverage, HTML reports, pyproject settings and a fail-under gate for CI, with…

Read article →
Software Testing

Error Guessing in Testing: Checklist + Demo

Error guessing uses tester experience to predict where code breaks. Get a ready checklist of risky inputs, a step by…

Read article →
Databases & SQL

SQL LAG and LEAD: Compare Rows, 5 Examples

SQL LAG and LEAD read values from the previous or next row without a self join. Five tested examples: change,…

Read article →
Databases & SQL

SQL STRING_AGG: Combine Rows Into One String

SQL STRING_AGG joins values from many rows into one string per group. See syntax for PostgreSQL, SQL Server, MySQL GROUP_CONCAT,…

Read article →
Software Development

JavaScript Classes: Private Fields and Extends

JavaScript classes explained with tested Node.js code: constructors, methods, #private fields, getters, static members, extends and super, plus the errors…

Read article →
Artificial Intelligence

BM25 Explained: How Keyword Search Ranks Results, With Python Code and Hybrid Search

BM25 explained with a pure Python implementation we ran: the formula, IDF, term saturation, the k1 and b parameters, why…

Read article →
DevOps

Git Bisect Explained: Find the Commit That Broke Your Code, Manually or Automatically

Git bisect explained with a tested repository: find the one bad commit among 41 in 5 test runs, automate it…

Read article →
Test Automation

pytest Markers Explained: Custom Marks, skip, skipif, xfail and -m Filters

pytest markers explained with a tested suite: register custom marks, run only smoke tests with -m, combine filters with and,…

Read article →
Software Testing

Positive vs Negative Testing: Differences, 24 Real Test Cases and a Checklist

Positive vs negative testing explained with a real signup form: 6 positive and 18 negative test cases run in pytest,…

Read article →
Databases & SQL

SQL ORDER BY Multiple Columns: ASC, DESC, NULLS LAST, Case and Custom Order

SQL ORDER BY with multiple columns explained with tested queries: mixed ASC and DESC, tie-breakers, NULLS FIRST and LAST, case-insensitive…

Read article →
Databases & SQL

SQL Running Total: SUM OVER, Per-Group Totals, Moving Sums and the RANGE Trap

How to calculate a SQL running total with SUM() OVER: per-group totals, daily cumulative sums, moving sums, percent of total,…

Read article →
Artificial Intelligence

Cosine Similarity Explained: Formula, Worked Example and Python Code

Cosine similarity explained simply: the formula, a worked example by hand, pure Python and NumPy code, cosine vs Euclidean distance…

Read article →
DevOps

Git Squash Commits: 4 Easy Ways, Tested Step by Step (rebase -i, reset, merge)

Git squash commits explained with 4 tested methods: interactive rebase, reset –soft, merge –squash and fixup with autosquash. Before-and-after logs,…

Read article →
Test Automation

pytest parametrize Explained: 7 Patterns with Tested Examples

pytest parametrize explained with 7 tested patterns: basic cases, readable ids, stacked decorators, pytest.param with xfail and skip, indirect fixtures…

Read article →
Software Testing

Retesting vs Regression Testing: 8 Key Differences with a Real Bug Fix Example

Retesting vs regression testing explained with a real bug fix run in pytest: retesting confirmed the fix, the regression run…

Read article →
Artificial Intelligence

RAG Chunking Strategies Explained: Fixed, Overlap, Sentence, Recursive and Headings, Measured

RAG chunking strategies compared on one document with tiktoken 0.14: fixed-size, overlap, sentence packing, recursive splitting and heading-based chunks, measuring…

Read article →
DevOps

Git Stash Explained: Save, Apply, Pop and Untracked Files, Tested

Git stash explained with commands we ran on Git 2.56: push with a message, untracked files, list and show, apply…

Read article →
Software Testing

Pairwise Testing Explained: How All-Pairs Cuts 216 Tests to 13, Tested

Pairwise testing explained with a generator we ran in Python 3.13: a checkout with 216 combinations covered by 13 tests,…

Read article →
Test Automation

Test Doubles Explained: Dummy, Stub, Spy, Mock and Fake, with Tested Python Code

Test doubles explained with one Python example run in pytest 9.2: dummy, stub, spy, mock and fake, when to use…

Read article →
Artificial Intelligence

LLM Context Window Explained: Tokens, Limits and How to Fit More, Measured

The LLM context window explained with token counts we measured using tiktoken: words vs tokens for prose, code and Hindi,…

Read article →
DevOps

Git Cherry-Pick Explained: Copy the Commit You Need, Safely, Tested

Git cherry-pick explained with commands we ran in Git 2.56: copying a hotfix to a release branch, new hashes and…

Read article →
Test Automation

Snapshot Testing Explained: How It Works, When It Helps and When It Hurts

Snapshot testing explained with a pytest and syrupy demo we ran: creating snapshots, reading the diff after a code change,…

Read article →
Software Testing

Mutation Testing Explained: Why 100% Coverage Still Missed Our Bug

Mutation testing explained with a small mutation tool we built and ran: 14 mutants, a test suite with 100% line…

Read article →