API Rate Limiting Explained: Algorithms, Headers and Handling 429 Errors

One buggy script calls your API in a tight loop. Without limits, it can slow the service for every other user, or run up a large cloud bill. API rate limiting is the guard rail that keeps a single client from taking more than its fair share.

To show how the common algorithms behave, we wrote small Python versions and fed them simulated request timings. The outputs below come from that run. They are teaching models, not production code, and real gateways have more features.

What is rate limiting?

Rate limiting counts requests per client and rejects, or delays, requests over an agreed limit, such as 100 requests per minute. A related idea, throttling, often means slowing requests down rather than rejecting them. A quota is usually a larger allowance over a longer period, such as 10,000 calls per month on a pricing plan.

Why APIs need it

• Stability: traffic spikes cannot overwhelm databases or downstream services.
• Fairness: one heavy client cannot starve the others.
• Security: limits slow down brute-force login attempts and scraping.
• Cost control: expensive operations, such as AI model calls, stay within budget.

Rate limiting is a core control in the OWASP API Security Top 10, under unrestricted resource consumption. Our API testing with Postman guide shows how to send the repeated requests needed to check it.

Token bucket

Picture a bucket that holds up to 5 tokens. Each request takes a token. Tokens refill at 1 per second, and the bucket never holds more than 5. If the bucket is empty, the request is rejected.

class TokenBucket:
    def __init__(self, capacity, refill_per_sec):
        self.capacity = capacity
        self.rate = refill_per_sec
        self.tokens = capacity
        self.last = 0.0

    def allow(self, now):
        elapsed = now - self.last
        self.tokens = min(self.capacity, self.tokens + elapsed * self.rate)
        self.last = now
        if self.tokens >= 1:
            self.tokens -= 1
            return True
        return False

We sent 7 requests at once, then more later:

burst of 7 at t=0:  [True, True, True, True, True, False, False]
t=2.0:              [True, True, False]
t=10:               5 allowed of 10

The first 5 went through as a burst, then the bucket was empty. Two seconds later, 2 tokens had refilled, so 2 more requests passed. After a long pause, the bucket was full again but capped at 5. This burst-then-steady behaviour suits real traffic, which is why token buckets are common in API gateways. The leaky bucket is a close relative that processes requests at a fixed rate, smoothing output instead of allowing bursts.

Fixed window vs sliding window

A fixed window counter is the simplest approach: allow 10 requests per minute, and reset the counter when the minute changes. It has a known flaw at the boundary. We sent 10 requests at 59.9 seconds and 10 more at 60.0 seconds, with a limit of 10 per 60-second window:

fixed window allowed around boundary: 20 of 20
sliding log allowed around boundary:  10 of 20

The fixed window allowed all 20 requests in a tenth of a second, double the intended rate, because they fell into two different windows. A sliding log remembers the time of each recent request and counts those in the last 60 seconds, so it correctly allowed only 10.

The trade-off is memory. A sliding log stores a timestamp per request. The sliding window counter is a popular middle ground. It estimates the rate by blending the current and previous window counts, using little memory with good accuracy.

Algorithm comparison

What to limit by

• API key or user ID: the fairest option for authenticated APIs.
• IP address: useful for public endpoints, but many users can share one IP, and attackers can rotate them.
• Endpoint: expensive operations, such as search, exports or login, deserve tighter limits.
• Plan tier: free and paid customers often get different limits.

Login endpoints need special care. Limit both by account and by IP, so attackers cannot simply spread guesses across many IPs or many accounts.

The 429 response

When a client goes over the limit, return HTTP 429 Too Many Requests, not 500 or 403. Include headers that tell the client what happened and when to try again. Our simulation produced this set for a client with no requests left:

HTTP/1.1 429 Too Many Requests
RateLimit-Limit: 100
RateLimit-Remaining: 0
RateLimit-Reset: 30
Retry-After: 30

Retry-After is a standard HTTP header. The RateLimit-* headers come from an IETF draft, and many APIs use their own X-RateLimit-* variants. Whatever you choose, document it. For more on status codes, see our HTTP status codes cheat sheet.

Handling limits as a client

Retrying immediately after a 429 makes things worse. If thousands of clients do it at the same moment, they hit the server in synchronised waves. The fix is exponential backoff with jitter: wait longer after each failure, and add randomness.

def backoff(attempt, base=0.5, cap=30, rng=random.Random(4)):
    return round(min(cap, base * 2 ** attempt) * rng.random(), 2)

[backoff(a) for a in range(6)]
# [0.12, 0.1, 0.79, 0.62, 0.53, 6.43]

This is “full jitter”: a random wait between zero and an exponentially growing ceiling. The waits look irregular, and that is the point. Clients spread out instead of retrying together. We seeded the random generator so the output could be reproduced.

Distributed rate limiting

With several API servers behind a load balancer, each server counting on its own lets a client exceed the limit by spreading requests across them. Teams usually keep counters in a shared store such as Redis, with atomic operations or a short script so two servers cannot both take the last token. Gateways such as Kong, NGINX, Envoy and cloud API gateways provide this as configuration. We did not run a distributed setup for this article.

Testing rate limits

• Send requests up to the limit and confirm they succeed.
• Send one more and confirm a 429 with correct headers.
• Wait for the reset time and confirm requests succeed again.
• Check limits per key: one client’s usage must not block another.
• Test the boundary between windows for the double-burst problem.
• Use a load tool, as covered in load testing vs stress testing, to check behaviour under heavy concurrency.

In unit tests, inject the current time, as our simulation does with now, so tests never have to sleep.

Designing limits users can live with

Picking the numbers matters as much as picking the algorithm. Start from real traffic: look at how many requests your heaviest legitimate clients make per minute, and set limits comfortably above that. A limit that blocks normal use creates support tickets, not safety.

• Publish the limits in your API documentation, per endpoint and per plan.
• Warn before blocking. Headers showing remaining requests let well-behaved clients slow down on their own.
• Offer alternatives to heavy polling, such as webhooks, bulk endpoints or larger pages.
• Monitor 429 rates. A sudden rise can mean an attack, a buggy client release, or limits that are now too tight.
• Grant exceptions deliberately, for example raised limits for a partner, recorded in configuration rather than hard-coded.

Also decide what happens to requests that arrive just over the limit. Some APIs briefly queue a small number of extra requests instead of rejecting them, which smooths out short spikes. Others give priority to essential endpoints, such as payments, over less critical ones like analytics, so that important traffic keeps flowing when the system is under pressure.

Final thoughts

Good rate limiting is a contract between API and client. The server picks a fair algorithm, usually a token bucket or sliding window, and explains its limits clearly with a 429 and Retry-After. The client respects those signals and backs off with jitter. Start with generous limits per API key, watch real usage, and tighten only where you need to.

Frequently asked questions

Keep learning