Python Requests Tutorial: GET, POST, Timeouts, Sessions and Retries
Python requests is the most popular way to call web APIs from Python. Learn GET with query parameters, POST with JSON, headers and tokens, timeouts, raise_for_status, sessions and automatic retries, with every result taken from runs against a local test API.
- Python requests turns HTTP calls into one line:
requests.get(url, params=..., timeout=5). - A 404 or 500 is not an exception by default. Call
r.raise_for_status()or checkr.ok. - Always pass
timeout=. Requests has no default, so a stuck server can hang your program forever. - Use
json=for JSON bodies. In our test,data=sent a form-encoded body and the API answered 400.
Almost every modern application talks to an API: a payment gateway, a weather service, your own backend. In Python, the requests library makes that easy, with a friendly API over the low-level HTTP machinery. But easy is not the same as safe. Missing timeouts, unchecked status codes and the wrong body type cause real production bugs.
To show real behaviour without relying on a public service, we built a tiny JSON API with Python’s http.server on 127.0.0.1 and called it with requests 2.34.2 on Python 3.13.6. Every output below is copied from those runs. If HTTP verbs are new to you, read our HTTP methods guide first.
Setup and our test API
pip install requests
Our test server exposes a small book API: GET /books lists books, GET /books/1 fetches one, POST /books creates one with a bearer token, /slow waits two seconds, and /flaky fails twice with 503 before succeeding. Using a local server like this is also a great habit for your own tests, because it is fast and never rate-limits you.
Your first GET request
import requests
BASE = "http://127.0.0.1:8765"
r = requests.get(f"{BASE}/books", timeout=5)
print(r.status_code, r.ok, r.headers["Content-Type"])
print(r.json())
200 True application/json
[{'id': 1, 'title': 'Clean Tests', 'price': 450}, {'id': 2, 'title': 'SQL Basics', 'price': 299}]
r.json() parses the body into Python lists and dicts. It raises an error if the body is not valid JSON, so check the content type first when you are not sure what a server returns.
Query parameters
Do not build query strings by hand. Pass a dict to params and requests encodes it correctly, including spaces and special characters:
r = requests.get(f"{BASE}/books", params={"max_price": 300}, timeout=5)
print(r.url)
print(r.json())
http://127.0.0.1:8765/books?max_price=300
[{'id': 2, 'title': 'SQL Basics', 'price': 299}]
r.url shows the final URL, which is handy when debugging. A list value such as {"tag": ["a", "b"]} becomes tag=a&tag=b.
The Response object
| Attribute | What it holds |
|---|---|
r.status_code |
The HTTP status, such as 200 or 404 |
r.ok |
True when the status is below 400 |
r.json() |
The body parsed as JSON |
r.text |
The body as a str, decoded with r.encoding |
r.content |
The raw body as bytes, for images and files |
r.headers |
Response headers, case-insensitive |
r.elapsed |
Time from sending the request to receiving the headers |
r.request |
The request that was actually sent |
For one book, our run showed r.encoding as utf-8, len(r.content) as 47 bytes and r.elapsed well under a second. Our HTTP status codes cheat sheet explains what each code means.
Errors are not exceptions: raise_for_status
This surprises many people. Requests only raises for network problems. An error status comes back as a normal response:
r = requests.get(f"{BASE}/books/99", timeout=5)
print(r.status_code, r.ok, r.json())
r.raise_for_status()
404 False {'error': 'not found'}
HTTPError: 404 Client Error: Not Found for url: http://127.0.0.1:8765/books/99
If your code skipped the check, it would happily treat {'error': 'not found'} as a book. Call raise_for_status() right after every request where an error status should stop the flow, or branch on r.status_code when some errors are expected, such as a 404 meaning “create it”.
POST with a JSON body
r = requests.post(
f"{BASE}/books",
json={"title": "Git Pro", "price": 350},
headers={"Authorization": "Bearer test-token"},
timeout=5,
)
print(r.status_code, r.headers["Location"], r.json())
print(r.request.headers["Content-Type"])
201 /books/3 {'id': 3, 'title': 'Git Pro', 'price': 350}
application/json
The server answered 201 Created with a Location header pointing at the new resource. json= serialised the dict and set the Content-Type header for us. The token shown is a dummy value from our test server, never a real credential.
json= vs data=: a common mistake
r = requests.post(f"{BASE}/books", data={"title": "Git Pro", "price": 350},
headers={"Authorization": "Bearer test-token"}, timeout=5)
print(r.status_code, r.request.headers["Content-Type"], r.request.body)
400 application/x-www-form-urlencoded title=Git+Pro&price=350
With data= and a dict, requests sends an HTML form body, not JSON. Our API rejected it with 400. Use json= for JSON APIs and data= for form posts or raw bytes. Also note what happens without the token:
r = requests.post(f"{BASE}/books", json={"title": "x"}, timeout=5)
print(r.status_code, r.json())
# 401 {'error': 'missing or bad token'}
Headers, tokens and authentication
Pass headers as a dict. For bearer tokens, the header is Authorization: Bearer <token>. For HTTP Basic auth, use auth=("user", "password") and requests builds the header. Never hard-code real secrets in source files. Read them from environment variables with os.environ["API_TOKEN"], and keep them out of logs. Our JWT guide explains the tokens many APIs use.
Timeouts: the setting you must not skip
Requests waits forever by default. One hung server can freeze a worker, a cron job or a whole test suite. Pass a timeout on every call. A tuple sets separate limits for connecting and for waiting on the response:
import time
t = time.perf_counter()
try:
requests.get(f"{BASE}/slow", timeout=(3, 0.5))
except requests.Timeout as e:
print(type(e).__name__, round(time.perf_counter() - t, 2), "s")
ReadTimeout 0.51 s
The slow endpoint needed two seconds, but our read timeout of 0.5 seconds stopped the wait at 0.51. The timeout is not a limit on total download time. It limits how long requests waits between bytes from the server.
Tip: be careful retrying a POST after a read timeout. The server may have already processed it. Design write endpoints to be idempotent, as our API idempotency guide explains.
The exception hierarchy
try:
requests.get("http://127.0.0.1:9", timeout=2)
except requests.ConnectionError as e:
print("ConnectionError", type(e).__name__)
# ConnectionError ConnectTimeout
Nothing listens on port 9 on our machine. On Windows the connection attempt simply timed out, so we got ConnectTimeout, which is both a ConnectionError and a Timeout. On other systems you may see an immediate refusal instead. The main exceptions are:
requests.ConnectionError: DNS failure, refused or unreachable connection.requests.Timeout: coversConnectTimeoutandReadTimeout.requests.HTTPError: raised byraise_for_status().requests.RequestException: the base class of all of them.
Catch specific exceptions where you can handle them differently, and RequestException as a last resort. Our exception handling guide covers the patterns.
Sessions: shared settings and reused connections
with requests.Session() as s:
s.headers.update({"Authorization": "Bearer test-token",
"User-Agent": "demo-client/1.0"})
print(s.get(f"{BASE}/me", timeout=5).json())
print(requests.get(f"{BASE}/me", timeout=5).status_code)
{'user': 'demo'}
401
Headers set on the session go with every request it makes, while a plain requests.get did not carry the token. Sessions also keep cookies and reuse TCP connections. We timed 50 calls: 134 ms with a session against 158 ms without. On localhost the gain is small, because connecting is cheap. Over the internet, and especially with HTTPS handshakes, connection reuse saves much more. Note that a session has no default timeout setting, so still pass timeout= on each call.
Automatic retries with HTTPAdapter
Temporary failures such as 503 or a dropped connection are often worth retrying. Requests supports this through urllib3’s Retry, mounted on a session:
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
s = requests.Session()
retry = Retry(total=3, backoff_factor=0.1,
status_forcelist=[503], allowed_methods=["GET"])
s.mount("http://", HTTPAdapter(max_retries=retry))
r = s.get(f"{BASE}/flaky", timeout=5)
print("retry:", r.status_code, r.json())
retry: 200 {'attempt': 3}
The flaky endpoint failed twice, and the third attempt succeeded, all inside one call. backoff_factor adds a growing pause between attempts. Limiting allowed_methods to safe methods such as GET avoids repeating non-idempotent writes. Mount on "https://" too for real APIs, and respect Retry-After headers from rate-limited services, as covered in our API rate limiting guide.
Best practices checklist
- Pass
timeout=on every request. - Call
raise_for_status()or checkstatus_codebefore using the body. - Use
params=for query strings andjson=for JSON bodies. - Use a
Sessionfor repeated calls to the same host. - Retry only safe or idempotent requests, with backoff.
- Keep tokens in environment variables, never in code or logs.
- Test your client against a local or mocked server, as shown in our API testing in Python guide.
- For async code, consider
httpx, which offers a very similar API.
Final thoughts
Python requests is simple on the surface, and that is its strength. The habits that make it production-ready are just as simple: always set a timeout, always check the status, send JSON with json=, reuse a session, and retry only what is safe to retry. Build those into a small helper function and every API call in your project inherits them.
FAQ
How do I send a POST request with JSON in Python requests?
Use requests.post with the json argument set to a dict. Requests serialises it and sets the Content-Type header to application/json for you.
Does Python requests have a default timeout?
No. Without a timeout argument a request can wait forever. Pass a number of seconds, or a tuple of connect and read timeouts, on every call.
Why does requests not raise an error on 404?
An HTTP error status is a valid response, not a network failure. Call raise_for_status to turn 4xx and 5xx responses into an HTTPError.
When should I use a requests Session?
Use a Session for several calls to the same host. It reuses connections and keeps shared headers and cookies across requests.
How do I retry failed requests in Python?
Mount an HTTPAdapter with a urllib3 Retry object on a Session, list the status codes to retry, add a backoff factor and limit retries to safe methods.
Keep learning
Need help with testing, WordPress or SEO?
We help teams ship better software and grow their sites. Tell us what you are working on and we will suggest a clear next step.