1. Home
  2. Blog
  3. Software Development

Software Development

Python Requests Tutorial: GET, POST, Timeouts, Sessions and Retries

Python requests is the most popular way to call web APIs from Python. Learn GET with query parameters, POST with JSON, headers and tokens, timeouts, raise_for_status, sessions and automatic retries, with every result taken from runs against a local test API.

10 min read
Python requests: GET, POST, timeouts and retries

Key takeaways

  • Python requests turns HTTP calls into one line: requests.get(url, params=..., timeout=5).
  • A 404 or 500 is not an exception by default. Call r.raise_for_status() or check r.ok.
  • Always pass timeout=. Requests has no default, so a stuck server can hang your program forever.
  • Use json= for JSON bodies. In our test, data= sent a form-encoded body and the API answered 400.

Almost every modern application talks to an API: a payment gateway, a weather service, your own backend. In Python, the requests library makes that easy, with a friendly API over the low-level HTTP machinery. But easy is not the same as safe. Missing timeouts, unchecked status codes and the wrong body type cause real production bugs.

To show real behaviour without relying on a public service, we built a tiny JSON API with Python’s http.server on 127.0.0.1 and called it with requests 2.34.2 on Python 3.13.6. Every output below is copied from those runs. If HTTP verbs are new to you, read our HTTP methods guide first.

Setup and our test API

pip install requests

Our test server exposes a small book API: GET /books lists books, GET /books/1 fetches one, POST /books creates one with a bearer token, /slow waits two seconds, and /flaky fails twice with 503 before succeeding. Using a local server like this is also a great habit for your own tests, because it is fast and never rate-limits you.

Your first GET request

import requests
BASE = "http://127.0.0.1:8765"

r = requests.get(f"{BASE}/books", timeout=5)
print(r.status_code, r.ok, r.headers["Content-Type"])
print(r.json())
200 True application/json
[{'id': 1, 'title': 'Clean Tests', 'price': 450}, {'id': 2, 'title': 'SQL Basics', 'price': 299}]

r.json() parses the body into Python lists and dicts. It raises an error if the body is not valid JSON, so check the content type first when you are not sure what a server returns.

Query parameters

Do not build query strings by hand. Pass a dict to params and requests encodes it correctly, including spaces and special characters:

r = requests.get(f"{BASE}/books", params={"max_price": 300}, timeout=5)
print(r.url)
print(r.json())
http://127.0.0.1:8765/books?max_price=300
[{'id': 2, 'title': 'SQL Basics', 'price': 299}]

r.url shows the final URL, which is handy when debugging. A list value such as {"tag": ["a", "b"]} becomes tag=a&tag=b.

The Response object

Attribute What it holds
r.status_code The HTTP status, such as 200 or 404
r.ok True when the status is below 400
r.json() The body parsed as JSON
r.text The body as a str, decoded with r.encoding
r.content The raw body as bytes, for images and files
r.headers Response headers, case-insensitive
r.elapsed Time from sending the request to receiving the headers
r.request The request that was actually sent

For one book, our run showed r.encoding as utf-8, len(r.content) as 47 bytes and r.elapsed well under a second. Our HTTP status codes cheat sheet explains what each code means.

Errors are not exceptions: raise_for_status

This surprises many people. Requests only raises for network problems. An error status comes back as a normal response:

r = requests.get(f"{BASE}/books/99", timeout=5)
print(r.status_code, r.ok, r.json())
r.raise_for_status()
404 False {'error': 'not found'}
HTTPError: 404 Client Error: Not Found for url: http://127.0.0.1:8765/books/99

If your code skipped the check, it would happily treat {'error': 'not found'} as a book. Call raise_for_status() right after every request where an error status should stop the flow, or branch on r.status_code when some errors are expected, such as a 404 meaning “create it”.

POST with a JSON body

r = requests.post(
    f"{BASE}/books",
    json={"title": "Git Pro", "price": 350},
    headers={"Authorization": "Bearer test-token"},
    timeout=5,
)
print(r.status_code, r.headers["Location"], r.json())
print(r.request.headers["Content-Type"])
201 /books/3 {'id': 3, 'title': 'Git Pro', 'price': 350}
application/json

The server answered 201 Created with a Location header pointing at the new resource. json= serialised the dict and set the Content-Type header for us. The token shown is a dummy value from our test server, never a real credential.

json= vs data=: a common mistake

r = requests.post(f"{BASE}/books", data={"title": "Git Pro", "price": 350},
                  headers={"Authorization": "Bearer test-token"}, timeout=5)
print(r.status_code, r.request.headers["Content-Type"], r.request.body)
400 application/x-www-form-urlencoded title=Git+Pro&price=350

With data= and a dict, requests sends an HTML form body, not JSON. Our API rejected it with 400. Use json= for JSON APIs and data= for form posts or raw bytes. Also note what happens without the token:

r = requests.post(f"{BASE}/books", json={"title": "x"}, timeout=5)
print(r.status_code, r.json())
# 401 {'error': 'missing or bad token'}

Headers, tokens and authentication

Pass headers as a dict. For bearer tokens, the header is Authorization: Bearer <token>. For HTTP Basic auth, use auth=("user", "password") and requests builds the header. Never hard-code real secrets in source files. Read them from environment variables with os.environ["API_TOKEN"], and keep them out of logs. Our JWT guide explains the tokens many APIs use.

Timeouts: the setting you must not skip

Requests waits forever by default. One hung server can freeze a worker, a cron job or a whole test suite. Pass a timeout on every call. A tuple sets separate limits for connecting and for waiting on the response:

import time
t = time.perf_counter()
try:
    requests.get(f"{BASE}/slow", timeout=(3, 0.5))
except requests.Timeout as e:
    print(type(e).__name__, round(time.perf_counter() - t, 2), "s")
ReadTimeout 0.51 s

The slow endpoint needed two seconds, but our read timeout of 0.5 seconds stopped the wait at 0.51. The timeout is not a limit on total download time. It limits how long requests waits between bytes from the server.

Tip: be careful retrying a POST after a read timeout. The server may have already processed it. Design write endpoints to be idempotent, as our API idempotency guide explains.

The exception hierarchy

try:
    requests.get("http://127.0.0.1:9", timeout=2)
except requests.ConnectionError as e:
    print("ConnectionError", type(e).__name__)
# ConnectionError ConnectTimeout

Nothing listens on port 9 on our machine. On Windows the connection attempt simply timed out, so we got ConnectTimeout, which is both a ConnectionError and a Timeout. On other systems you may see an immediate refusal instead. The main exceptions are:

  • requests.ConnectionError: DNS failure, refused or unreachable connection.
  • requests.Timeout: covers ConnectTimeout and ReadTimeout.
  • requests.HTTPError: raised by raise_for_status().
  • requests.RequestException: the base class of all of them.

Catch specific exceptions where you can handle them differently, and RequestException as a last resort. Our exception handling guide covers the patterns.

Sessions: shared settings and reused connections

with requests.Session() as s:
    s.headers.update({"Authorization": "Bearer test-token",
                      "User-Agent": "demo-client/1.0"})
    print(s.get(f"{BASE}/me", timeout=5).json())
print(requests.get(f"{BASE}/me", timeout=5).status_code)
{'user': 'demo'}
401

Headers set on the session go with every request it makes, while a plain requests.get did not carry the token. Sessions also keep cookies and reuse TCP connections. We timed 50 calls: 134 ms with a session against 158 ms without. On localhost the gain is small, because connecting is cheap. Over the internet, and especially with HTTPS handshakes, connection reuse saves much more. Note that a session has no default timeout setting, so still pass timeout= on each call.

Automatic retries with HTTPAdapter

Temporary failures such as 503 or a dropped connection are often worth retrying. Requests supports this through urllib3’s Retry, mounted on a session:

from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

s = requests.Session()
retry = Retry(total=3, backoff_factor=0.1,
              status_forcelist=[503], allowed_methods=["GET"])
s.mount("http://", HTTPAdapter(max_retries=retry))
r = s.get(f"{BASE}/flaky", timeout=5)
print("retry:", r.status_code, r.json())
retry: 200 {'attempt': 3}

The flaky endpoint failed twice, and the third attempt succeeded, all inside one call. backoff_factor adds a growing pause between attempts. Limiting allowed_methods to safe methods such as GET avoids repeating non-idempotent writes. Mount on "https://" too for real APIs, and respect Retry-After headers from rate-limited services, as covered in our API rate limiting guide.

Best practices checklist

  1. Pass timeout= on every request.
  2. Call raise_for_status() or check status_code before using the body.
  3. Use params= for query strings and json= for JSON bodies.
  4. Use a Session for repeated calls to the same host.
  5. Retry only safe or idempotent requests, with backoff.
  6. Keep tokens in environment variables, never in code or logs.
  7. Test your client against a local or mocked server, as shown in our API testing in Python guide.
  8. For async code, consider httpx, which offers a very similar API.

Final thoughts

Python requests is simple on the surface, and that is its strength. The habits that make it production-ready are just as simple: always set a timeout, always check the status, send JSON with json=, reuse a session, and retry only what is safe to retry. Build those into a small helper function and every API call in your project inherits them.

FAQ

How do I send a POST request with JSON in Python requests?

Use requests.post with the json argument set to a dict. Requests serialises it and sets the Content-Type header to application/json for you.

Does Python requests have a default timeout?

No. Without a timeout argument a request can wait forever. Pass a number of seconds, or a tuple of connect and read timeouts, on every call.

Why does requests not raise an error on 404?

An HTTP error status is a valid response, not a network failure. Call raise_for_status to turn 4xx and 5xx responses into an HTTPError.

When should I use a requests Session?

Use a Session for several calls to the same host. It reuses connections and keeps shared headers and cookies across requests.

How do I retry failed requests in Python?

Mount an HTTPAdapter with a urllib3 Retry object on a Session, list the status codes to retry, add a backoff factor and limit retries to safe methods.

Keep learning

Work with Stack Quora

Need help with testing, WordPress or SEO?

We help teams ship better software and grow their sites. Tell us what you are working on and we will suggest a clear next step.

Keep reading

More guides on testing, AI & modern development

Enjoyed this article? Pick a topic and jump straight into our newest tutorials.

Artificial Intelligence

Jaccard Similarity: Formula, Python, MinHash

Jaccard similarity measures overlap between two sets as intersection over union. Work the formula, compute it in Python for words…

Read article →
Software Development

Unix Timestamp Explained: Seconds vs ms

A Unix timestamp counts seconds since 1 January 1970 UTC. Learn seconds vs milliseconds, converting in Python, JavaScript and SQL,…

Read article →
DevOps

Git Hooks: Pre-commit and Commit-msg Examples

Git hooks run scripts at key moments such as commit and push. See tested pre-commit, commit-msg and prepare-commit-msg hooks, sharing…

Read article →
Test Automation

API Testing in Python: Pytest + Requests Suite

Build an API testing suite in Python with pytest and requests: fixtures, status codes, headers, JSON schema validation, CRUD flows,…

Read article →
Software Testing

Test Design Techniques: 9 Methods + Examples

Test design techniques help you write fewer tests that catch more bugs. Learn 9 black-box, white-box and experience-based methods with…

Read article →
Databases & SQL

SQL Temporary Table: Create, Use and Drop It

SQL temporary tables store interim results for one session. Learn syntax for SQL Server, PostgreSQL, MySQL, Oracle and SQLite, session…

Read article →
Databases & SQL

SQL CROSS JOIN: Every Combination + Examples

SQL CROSS JOIN returns every combination of rows from two tables. See the row math, product variants, calendar grids that…

Read article →
Software Development

JavaScript Date: Format, Parse, Timezone Bugs

JavaScript Date explained with tested Node.js code: zero-based months, the date-only UTC parsing trap, toISOString, getTimezoneOffset, Intl formatting, date math…

Read article →
Software Development

Python Requests: GET, POST, Timeouts, Retries

Python requests explained with code tested against a local API: GET with params, POST json vs data, headers and auth,…

Read article →
Software Development

Python Deepcopy vs Copy: Fix Nested Lists

Python deepcopy vs shallow copy explained with tested code: aliasing, nested lists, the [[0]*3]*3 trap, memo and cycles, custom __deepcopy__,…

Read article →
Artificial Intelligence

TF-IDF Explained: Formula and Worked Example

TF-IDF scores a word by how often it appears in a document and how rare it is across all documents….

Read article →
Software Development

Base64 Encoding Explained, With Python Code

Base64 encoding turns bytes into 64 safe text characters. See the bit-level steps, padding, URL-safe base64, the 33% size cost,…

Read article →
DevOps

Git Tag: Create, Push and Delete Release Tags

Git tag explained with tested commands: annotated vs lightweight tags, listing and sorting, pushing with follow-tags, deleting local and remote…

Read article →
Test Automation

Pytest Coverage: pytest-cov and Branch Gaps

Measure pytest coverage with pytest-cov: term-missing reports, branch coverage, HTML reports, pyproject settings and a fail-under gate for CI, with…

Read article →
Software Testing

Error Guessing in Testing: Checklist + Demo

Error guessing uses tester experience to predict where code breaks. Get a ready checklist of risky inputs, a step by…

Read article →
Databases & SQL

SQL LAG and LEAD: Compare Rows, 5 Examples

SQL LAG and LEAD read values from the previous or next row without a self join. Five tested examples: change,…

Read article →
Databases & SQL

SQL STRING_AGG: Combine Rows Into One String

SQL STRING_AGG joins values from many rows into one string per group. See syntax for PostgreSQL, SQL Server, MySQL GROUP_CONCAT,…

Read article →
Software Development

JavaScript Classes: Private Fields and Extends

JavaScript classes explained with tested Node.js code: constructors, methods, #private fields, getters, static members, extends and super, plus the errors…

Read article →
Artificial Intelligence

BM25 Explained: How Keyword Search Ranks Results, With Python Code and Hybrid Search

BM25 explained with a pure Python implementation we ran: the formula, IDF, term saturation, the k1 and b parameters, why…

Read article →
DevOps

Git Bisect Explained: Find the Commit That Broke Your Code, Manually or Automatically

Git bisect explained with a tested repository: find the one bad commit among 41 in 5 test runs, automate it…

Read article →
Test Automation

pytest Markers Explained: Custom Marks, skip, skipif, xfail and -m Filters

pytest markers explained with a tested suite: register custom marks, run only smoke tests with -m, combine filters with and,…

Read article →
Software Testing

Positive vs Negative Testing: Differences, 24 Real Test Cases and a Checklist

Positive vs negative testing explained with a real signup form: 6 positive and 18 negative test cases run in pytest,…

Read article →
Databases & SQL

SQL ORDER BY Multiple Columns: ASC, DESC, NULLS LAST, Case and Custom Order

SQL ORDER BY with multiple columns explained with tested queries: mixed ASC and DESC, tie-breakers, NULLS FIRST and LAST, case-insensitive…

Read article →
Databases & SQL

SQL Running Total: SUM OVER, Per-Group Totals, Moving Sums and the RANGE Trap

How to calculate a SQL running total with SUM() OVER: per-group totals, daily cumulative sums, moving sums, percent of total,…

Read article →
Artificial Intelligence

Cosine Similarity Explained: Formula, Worked Example and Python Code

Cosine similarity explained simply: the formula, a worked example by hand, pure Python and NumPy code, cosine vs Euclidean distance…

Read article →
DevOps

Git Squash Commits: 4 Easy Ways, Tested Step by Step (rebase -i, reset, merge)

Git squash commits explained with 4 tested methods: interactive rebase, reset –soft, merge –squash and fixup with autosquash. Before-and-after logs,…

Read article →
Test Automation

pytest parametrize Explained: 7 Patterns with Tested Examples

pytest parametrize explained with 7 tested patterns: basic cases, readable ids, stacked decorators, pytest.param with xfail and skip, indirect fixtures…

Read article →
Software Testing

Retesting vs Regression Testing: 8 Key Differences with a Real Bug Fix Example

Retesting vs regression testing explained with a real bug fix run in pytest: retesting confirmed the fix, the regression run…

Read article →
Artificial Intelligence

RAG Chunking Strategies Explained: Fixed, Overlap, Sentence, Recursive and Headings, Measured

RAG chunking strategies compared on one document with tiktoken 0.14: fixed-size, overlap, sentence packing, recursive splitting and heading-based chunks, measuring…

Read article →
DevOps

Git Stash Explained: Save, Apply, Pop and Untracked Files, Tested

Git stash explained with commands we ran on Git 2.56: push with a message, untracked files, list and show, apply…

Read article →
Software Testing

Pairwise Testing Explained: How All-Pairs Cuts 216 Tests to 13, Tested

Pairwise testing explained with a generator we ran in Python 3.13: a checkout with 216 combinations covered by 13 tests,…

Read article →
Test Automation

Test Doubles Explained: Dummy, Stub, Spy, Mock and Fake, with Tested Python Code

Test doubles explained with one Python example run in pytest 9.2: dummy, stub, spy, mock and fake, when to use…

Read article →
Artificial Intelligence

LLM Context Window Explained: Tokens, Limits and How to Fit More, Measured

The LLM context window explained with token counts we measured using tiktoken: words vs tokens for prose, code and Hindi,…

Read article →
DevOps

Git Cherry-Pick Explained: Copy the Commit You Need, Safely, Tested

Git cherry-pick explained with commands we ran in Git 2.56: copying a hotfix to a release branch, new hashes and…

Read article →
Test Automation

Snapshot Testing Explained: How It Works, When It Helps and When It Hurts

Snapshot testing explained with a pytest and syrupy demo we ran: creating snapshots, reading the diff after a code change,…

Read article →
Software Testing

Mutation Testing Explained: Why 100% Coverage Still Missed Our Bug

Mutation testing explained with a small mutation tool we built and ran: 14 mutants, a test suite with 100% line…

Read article →