1. Home
  2. Blog
  3. Software Development

Software Development

API Idempotency Explained: Idempotency Keys and Safe Retries, Tested

API idempotency means that sending the same request twice has the same effect as sending it once. Learn which HTTP methods are idempotent, why POST needs an idempotency key, and how to build one safely, with a Python demo that turned a double charge into a single one.

9 min read
API idempotency explained: idempotency keys and safe retries

Key takeaways

  • API idempotency means repeating a request does not repeat its side effect. GET, PUT and DELETE are idempotent by definition; POST is not.
  • In our demo, a lost response followed by a retry created two charges without a key and one with an Idempotency-Key.
  • Store a fingerprint of the request body with the key: reusing a key with a different body returned 422 in our server.
  • Claim the key atomically. Two simultaneous retries got 201 and 409, and only one charge was made.

Networks fail in the worst possible place: after the server has done the work but before the client hears back. The client sees a timeout, retries, and a customer is charged twice or an order ships twice. API idempotency is the design rule that makes those retries safe, and idempotency keys are the standard way to apply it to requests that create things.

To show the problem rather than describe it, we wrote a small payment API with Python 3.13’s built-in HTTP server and a client that retries after a dropped connection. Everything ran locally, and the outputs below are copied from that run. Real payment providers add much more, so treat the code as a teaching model. For general API design, see our REST API best practices.

What is idempotency?

An operation is idempotent if doing it once or many times leaves the system in the same state. Setting a light switch to “off” is idempotent: pressing “off” five times still leaves it off. Toggling the switch is not, because each press changes the result.

Note that idempotency is about the effect on the server, not the response. Deleting an order twice is idempotent even if the second call returns 404, because the order is gone either way. A related word, safe, means the request changes nothing at all, like GET.

Which HTTP methods are idempotent?

The HTTP specification, RFC 9110, defines these properties for each method:

Method Safe Idempotent Typical use
GET, HEAD, OPTIONS Yes Yes Read data
PUT No Yes Replace a resource at a known URL
DELETE No Yes Remove a resource
POST No No Create a resource or trigger an action
PATCH No Not guaranteed Partial update

The table describes what servers should do. A badly designed PUT that increments a counter breaks the promise. And PATCH depends on the payload: “set quantity to 3” is idempotent, while “add 1 to quantity” is not. Our HTTP status codes cheat sheet lists the responses each method usually returns.

The retry problem

A client sends POST /payments. Three things can happen:

  1. The request never reaches the server. Retrying is safe.
  2. The server processes it and the response arrives. No retry is needed.
  3. The server processes it, but the response is lost to a timeout or dropped connection. The client cannot tell this apart from case 1.

Case 3 is the dangerous one. If the client retries, the server creates a second payment. If it does not retry, a payment that failed in case 1 is never made. Without help from the server, the client has no safe choice.

How idempotency keys work

The fix is to give each logical operation a unique ID that the server remembers:

  1. The client generates a random key, usually a UUID, and sends it in a header such as Idempotency-Key.
  2. The server checks whether it has seen the key. If not, it records the key, does the work and stores the response with the key.
  3. If the same key arrives again, the server skips the work and returns the stored response.
  4. Keys expire after a retention window, commonly about 24 hours, though it varies by provider.

Payment providers popularised the pattern, and an IETF draft standard describes the Idempotency-Key header for HTTP APIs in general.

Our demo server

The heart of the server is a few lines that claim the key, then either do the work or replay the stored result:

key = self.headers.get("Idempotency-Key")
fp = hashlib.sha256(body).hexdigest()          # fingerprint of the request
if key:
    with lock:                                  # atomic claim
        rec = keys.get(key)
        if rec is None:
            keys[key] = {"fp": fp, "state": "processing"}
    if rec:
        if rec["fp"] != fp:
            return self.reply(422, {"error": "key reused with different body"})
        if rec["state"] == "processing":
            return self.reply(409, {"error": "request in progress"})
        return self.reply(rec["status"], rec["body"], replay=True)

charges.append(data["amount"])                  # the side effect
out = {"charge_id": f"ch_{len(charges)}", "amount": data["amount"]}
if key:
    keys[key].update(state="done", status=201, body=out)

To simulate case 3, the server can be told to complete the charge and then close the connection without replying. The client sees an error and retries with the identical request.

Results: with and without a key

no key, attempt 1: ('ERR', 'RemoteDisconnected', None)
no key, retry:     (201, {'charge_id': 'ch_2', 'amount': 500}, None)
charges without key: [500, 500]

key, attempt 1:    ('ERR', 'RemoteDisconnected', None)
key, retry:        (201, {'charge_id': 'ch_1', 'amount': 500}, 'true')
charges with key:  [500]

Without a key, the customer was charged 500 twice, and the client only ever saw charge ch_2; the first charge became invisible. With a key, the retry received the original response, ch_1, with an Idempotent-Replayed: true header, and only one charge exists. The client got exactly the answer it missed the first time.

Tip: replay the stored response, including the status code. Returning a different message such as “duplicate request” forces every client to write special handling and defeats the point.

Reusing a key with a different body

A key should identify one operation. If a client accidentally sends the same key with a different amount, replaying the old response would be wrong, and so would charging the new amount. Our server stores a SHA-256 fingerprint of the body with the key and compares it:

same key same body:      201 (replayed)
same key different body: (422, {'error': 'key reused with different body'})

Refusing loudly turns a silent client bug into an obvious one. Providers differ on the exact status code; the IETF draft suggests 422 for this case.

Concurrent retries

Aggressive clients sometimes retry before the first attempt has finished. If the server checks the key and then inserts it in two separate steps, both requests can pass the check and both can charge. We sent two identical requests with the same key at the same moment, with the work taking half a second:

concurrent same key: [201, 409] charges: [700]

One request won the atomic claim and charged; the other saw the key in the processing state and got 409 Conflict, which tells the client to wait and retry. In production, the lock is replaced by a database unique constraint or an atomic “insert if absent” operation such as Redis SET NX, so it works across many servers.

Server design checklist

  • Scope keys to the account or API key, so two customers cannot collide.
  • Claim before working, atomically, and record a processing state.
  • Store the full response, status and body, when the work finishes.
  • Fingerprint the request and reject mismatched reuse.
  • Handle failures deliberately. If the work fails because of a server error, release the key so a retry can try again; if it fails validation, store and replay the error.
  • Expire keys after a documented window, and keep the side effect and the key record in the same transaction where possible.
  • Make internal steps idempotent too. Message consumers and webhook handlers receive duplicates, so de-duplicate on an event ID.

Client side rules

  • Generate one key per logical operation, not per HTTP attempt, and reuse it for every retry of that operation.
  • Retry only on network errors, timeouts, 409, 429 and 5xx responses, with exponential backoff and jitter. Our API rate limiting guide explains backoff.
  • Never retry a 4xx validation error unchanged; it will fail the same way.
  • Persist the key with the pending operation, so a restarted app can retry safely.

How to test idempotency

Idempotency bugs hide until production traffic is messy, so test them on purpose. Send the same request twice and assert that one resource exists. Send the same key with a different body and assert the error. Fire parallel requests with one key and assert a single side effect, as our concurrent test did. Kill the connection after the server commits, using a proxy or a test hook, and check the retry gets the original response. These checks fit naturally into API tests built with the tools in our API testing with Postman guide.

Final thoughts

API idempotency turns retries from a risk into a routine. Use naturally idempotent methods where they fit, and give every POST that moves money or creates important records an idempotency key. On the server, claim keys atomically, store and replay full responses, and reject reuse with a different body. Our small demo showed the difference in one line: two charges became one.

Frequently asked questions

What is idempotency in an API?

An API operation is idempotent when making the same request several times has the same effect on the server as making it once, so clients can retry safely.

Is POST idempotent?

No. POST usually creates a new resource or triggers an action each time. Add an idempotency key so the server can recognise and safely ignore repeated attempts.

What is an idempotency key?

It is a unique value, often a UUID, that the client sends with a request. The server stores the result under that key and returns the same result if the key is seen again.

How long should idempotency keys be stored?

Long enough to cover every realistic retry. Around 24 hours is common, but providers differ, so document your window and expire old keys.

What should the server return for a duplicate request?

Return the stored original response with the same status code and body. If the original is still running, return 409 so the client retries later.

Keep learning

Work with Stack Quora

Need help with testing, WordPress or SEO?

We help teams ship better software and grow their sites. Tell us what you are working on and we will suggest a clear next step.

Keep reading

More guides on testing, AI & modern development

Enjoyed this article? Pick a topic and jump straight into our newest tutorials.

Artificial Intelligence

BM25 Explained: How Keyword Search Ranks Results, With Python Code and Hybrid Search

BM25 explained with a pure Python implementation we ran: the formula, IDF, term saturation, the k1 and b parameters, why…

Read article →
Software Development

HTTP Methods Explained: GET, POST, PUT, PATCH, DELETE, Tested on a Live Server

HTTP methods explained with a small server we ran: GET, POST, PUT, PATCH, DELETE, HEAD and OPTIONS, safe vs idempotent,…

Read article →
DevOps

Git Bisect Explained: Find the Commit That Broke Your Code, Manually or Automatically

Git bisect explained with a tested repository: find the one bad commit among 41 in 5 test runs, automate it…

Read article →
Test Automation

pytest Markers Explained: Custom Marks, skip, skipif, xfail and -m Filters

pytest markers explained with a tested suite: register custom marks, run only smoke tests with -m, combine filters with and,…

Read article →
Software Testing

Positive vs Negative Testing: Differences, 24 Real Test Cases and a Checklist

Positive vs negative testing explained with a real signup form: 6 positive and 18 negative test cases run in pytest,…

Read article →
Databases & SQL

SQL ORDER BY Multiple Columns: ASC, DESC, NULLS LAST, Case and Custom Order

SQL ORDER BY with multiple columns explained with tested queries: mixed ASC and DESC, tie-breakers, NULLS FIRST and LAST, case-insensitive…

Read article →
Databases & SQL

SQL Running Total: SUM OVER, Per-Group Totals, Moving Sums and the RANGE Trap

How to calculate a SQL running total with SUM() OVER: per-group totals, daily cumulative sums, moving sums, percent of total,…

Read article →
Software Development

JavaScript Sort Array: Numbers, Objects, Strings and Why 10 Comes Before 5

How to sort an array in JavaScript correctly: numbers with a compare function, objects by one or more fields, strings…

Read article →
Software Development

Python String Methods: Cheat Sheet, Real Examples and the strip() Trap

Python string methods with tested examples: split, join, strip, replace, find, startswith, is-methods, casefold, partition and translate, plus the strip()…

Read article →
Software Development

Python zip() Explained: Loop Two Lists at Once, Unzip, strict=True and Traps

Python zip() pairs items from several lists in one loop. Tested examples for dicts, unzipping with *, strict=True, zip_longest, sliding…

Read article →
Artificial Intelligence

Cosine Similarity Explained: Formula, Worked Example and Python Code

Cosine similarity explained simply: the formula, a worked example by hand, pure Python and NumPy code, cosine vs Euclidean distance…

Read article →
Software Development

UTF-8 Encoding Explained: How Text Becomes Bytes and How to Fix Garbled Text

UTF-8 encoding explained with byte-level examples we ran in Python: how characters become 1 to 4 bytes, why café turns…

Read article →
DevOps

Git Squash Commits: 4 Easy Ways, Tested Step by Step (rebase -i, reset, merge)

Git squash commits explained with 4 tested methods: interactive rebase, reset –soft, merge –squash and fixup with autosquash. Before-and-after logs,…

Read article →
Test Automation

pytest parametrize Explained: 7 Patterns with Tested Examples

pytest parametrize explained with 7 tested patterns: basic cases, readable ids, stacked decorators, pytest.param with xfail and skip, indirect fixtures…

Read article →
Software Testing

Retesting vs Regression Testing: 8 Key Differences with a Real Bug Fix Example

Retesting vs regression testing explained with a real bug fix run in pytest: retesting confirmed the fix, the regression run…

Read article →
Databases & SQL

Delete Duplicate Rows in SQL: 5 Safe Methods, Tested Step by Step

How to delete duplicate rows in SQL safely: find them, preview them, then remove them with ROW_NUMBER, MIN(id), EXISTS, rowid…

Read article →
Databases & SQL

SQL RANK vs DENSE_RANK vs ROW_NUMBER: Differences with Tested Examples

SQL RANK vs DENSE_RANK vs ROW_NUMBER in one table: how each handles ties, top-N per group, the second-highest salary trap…

Read article →
Software Development

JavaScript Optional Chaining (?.) Explained: 8 Examples, ?? Defaults and Traps

JavaScript optional chaining stops “Cannot read properties of undefined” errors. 8 tested examples for properties, arrays and methods, ?? vs…

Read article →
Artificial Intelligence

RAG Chunking Strategies Explained: Fixed, Overlap, Sentence, Recursive and Headings, Measured

RAG chunking strategies compared on one document with tiktoken 0.14: fixed-size, overlap, sentence packing, recursive splitting and heading-based chunks, measuring…

Read article →
DevOps

Git Stash Explained: Save, Apply, Pop and Untracked Files, Tested

Git stash explained with commands we ran on Git 2.56: push with a message, untracked files, list and show, apply…

Read article →
Software Testing

Pairwise Testing Explained: How All-Pairs Cuts 216 Tests to 13, Tested

Pairwise testing explained with a generator we ran in Python 3.13: a checkout with 216 combinations covered by 13 tests,…

Read article →
Test Automation

Test Doubles Explained: Dummy, Stub, Spy, Mock and Fake, with Tested Python Code

Test doubles explained with one Python example run in pytest 9.2: dummy, stub, spy, mock and fake, when to use…

Read article →
Databases & SQL

SQL COALESCE Explained: Defaults, NULLIF, IFNULL vs ISNULL and Traps, Tested

SQL COALESCE explained with queries we ran in SQLite 3.50: first non-null values, empty strings, NULL in maths, NULLIF for…

Read article →
Databases & SQL

SQL Self Join Explained: Employee-Manager, Pairs and Row Comparisons, Tested

SQL self join explained with queries we ran in SQLite 3.50: employee and manager, earning more than the manager, unique…

Read article →
Artificial Intelligence

LLM Context Window Explained: Tokens, Limits and How to Fit More, Measured

The LLM context window explained with token counts we measured using tiktoken: words vs tokens for prose, code and Hindi,…

Read article →
DevOps

Git Cherry-Pick Explained: Copy the Commit You Need, Safely, Tested

Git cherry-pick explained with commands we ran in Git 2.56: copying a hotfix to a release branch, new hashes and…

Read article →
Test Automation

Snapshot Testing Explained: How It Works, When It Helps and When It Hurts

Snapshot testing explained with a pytest and syrupy demo we ran: creating snapshots, reading the diff after a code change,…

Read article →
Software Testing

Mutation Testing Explained: Why 100% Coverage Still Missed Our Bug

Mutation testing explained with a small mutation tool we built and ran: 14 mutants, a test suite with 100% line…

Read article →
Artificial Intelligence

Prompt Injection Explained: How AI Apps Get Tricked and How to Defend Them

Prompt injection explained for builders: direct and indirect attacks, why keyword filters fail (our test blocked 1 of 4), and…

Read article →
DevOps

DNS Records Explained: A, AAAA, CNAME, MX, TXT and TTL With Real Lookups

DNS records explained with lookups we ran: how resolution works, A and AAAA, CNAME rules, MX and null MX, TXT…

Read article →
DevOps

Feature Flags Explained: Safe Releases, Gradual Rollouts and Flag Hygiene

Feature flags explained with a rollout simulation we ran: flag types, percentage rollouts with stable hashing, kill switches, allow lists,…

Read article →
Test Automation

Test Data Generation With Python Faker: Realistic, Repeatable and Safe Data

Test data generation with Python Faker, run and measured: seeding for repeatable data, locales, unique values, domain-shaped records, edge cases…

Read article →
Software Testing

Shift-Left Testing Explained: Find Bugs Earlier Without Slowing the Team

Shift-left testing explained: what it means, why early defects are cheaper, practices from requirement reviews to CI checks, the tester's…

Read article →
Artificial Intelligence

LLM Structured Output Explained: Getting Reliable JSON From AI Models

LLM structured output explained with validation we ran in Pydantic: JSON mode and schemas, constrained decoding, the ways model output…

Read article →
Test Automation

API Contract Testing Explained: Catch Breaking Changes Before They Ship

API contract testing explained with a schema check we ran: what a contract is, consumer-driven contracts, Pact, OpenAPI, which changes…

Read article →
Software Testing

Risk-Based Testing Explained: Score Risks, Build a Matrix and Test What Matters

Risk-based testing explained step by step: identifying risks, scoring likelihood and impact, building a risk matrix, mapping risk to test…

Read article →