Introduction
Python’s requests library is the single most useful module outside the standard
library — it’s how you call APIs, scrape pages, send webhooks, integrate with any HTTP
service. If you can use it well, you can build against most of the internet.
This is the fifteen-minute version. URL anatomy, request and response structure, the four HTTP methods that cover 95% of work, query parameters, JSON bodies, headers, auth tokens, error handling. Everything you need to call any API from a Python script.
📚 Table of contents
- URL anatomy — domain, path, query parameters
- Request and response structure
- HTTP methods you’ll actually use
- Status codes worth memorising
- Installing requests
- Your first GET request
- Query parameters with
params= - POST requests with JSON bodies
- Headers and authorization tokens
- Error handling: timeouts, raise_for_status, exception types
- Common mistakes
- FAQs
URL anatomy — domain, path, query parameters
A URL splits into three labelled parts:
https://api.example.com/v1/users?page=2&sort=name
\__________________/\______/\_____________________/
domain path query params
- Domain — the host serving the request.
- Path — the resource or endpoint within that host (
/v1/users). - Query parameters — everything after the
?, key-value pairs separated by&. Optional. Used to filter, paginate, or modify behavior.
Request and response structure
Every HTTP exchange is one client request and one server response. A request contains:
- Method — GET, POST, PUT, PATCH, DELETE.
- Path — with optional query params.
- Headers — metadata: content type, auth token, custom keys.
- Body — payload data, usually JSON. Required for POST/PUT/PATCH, omitted for GET/DELETE.
A response contains:
- Status code — the result number. 200 OK, 404 Not Found, etc.
- Headers — server-side metadata.
- Body — the actual returned data.
HTTP methods you’ll actually use
| Method | Intent | Body? |
|---|---|---|
| GET | Read / fetch data | No |
| POST | Create a new resource | Yes |
| PUT | Replace a resource entirely | Yes |
| PATCH | Update part of a resource | Yes |
| DELETE | Remove a resource | Optional |
Status codes worth memorising
- 2xx success — 200 OK, 201 Created, 204 No Content.
- 3xx redirects — 301 Moved Permanently, 302 Found, 304 Not Modified.
- 4xx client errors — 400 Bad Request, 401 Unauthorized, 403 Forbidden, 404 Not Found, 429 Too Many Requests.
- 5xx server errors — 500 Internal Server Error, 502 Bad Gateway, 503 Service Unavailable, 504 Gateway Timeout.
Rule of thumb: 4xx means you did something wrong (bad input, no auth). 5xx means the server is having a bad time. Retry logic should be reserved for 5xx and 429, not for 4xx errors that won’t fix themselves.
Installing requests
# pip
pip install requests
# uv (recommended)
uv init .
uv add requests
uv run main.py
Your first GET request
import requests
url = "https://jsonplaceholder.typicode.com/posts/1"
response = requests.get(url)
print(response.status_code) # 200
print(response.headers["Content-Type"])# application/json; charset=utf-8
data = response.json() # parse JSON body into a dict
print(data["title"])
print(data) # full payload
The response object has the four things you actually use:
response.status_code, response.headers, response.json() to
parse JSON, and response.text for raw string bodies.
Query parameters with params=
Don’t hand-build query strings. Pass a dict to params= and let
requests encode it correctly:
params = {"page": 2, "limit": 25, "sort": "name"}
response = requests.get("https://api.example.com/users", params=params)
print(response.url)
# https://api.example.com/users?page=2&limit=25&sort=name
Special characters and spaces are URL-encoded for you. Hand-built strings break on the first space or ampersand in a value.
POST requests with JSON bodies
payload = {"title": "Hello from Python", "body": "First post", "userId": 1}
response = requests.post(
"https://jsonplaceholder.typicode.com/posts",
json=payload,
)
print(response.status_code) # 201 Created
print(response.json())
Important detail: use json=payload, not data=payload.
json= serialises to JSON and sets the Content-Type header automatically.
data= sends a form-urlencoded body, which most JSON APIs reject.
Headers and authorization tokens
Most real APIs require an auth token in the Authorization header. The convention is
Authorization: Bearer <token>:
import os
import requests
token = os.environ["X_API_TOKEN"]
headers = {"Authorization": f"Bearer {token}"}
response = requests.get(
"https://api.x.com/2/users/me",
headers=headers,
)
response.raise_for_status()
print(response.json())
Never hardcode tokens in source files. Use environment variables or a secrets manager. Committing a real API key to GitHub is one of the most common ways apps get breached — even well-funded companies make this mistake.
Error handling: timeouts, raise_for_status, exception types
A network call can fail in dozens of ways: timeout, DNS failure, connection reset, bad SSL, HTTP error code. Handle them explicitly:
import requests
from requests.exceptions import Timeout, RequestException
try:
response = requests.get(
"https://httpbin.org/delay/3",
timeout=5, # seconds; always set a timeout
)
response.raise_for_status() # raises HTTPError on 4xx/5xx
print(response.json())
except Timeout:
print("request timed out")
except RequestException as e:
print(f"request failed: {e}")
Three rules:
- Always pass
timeout=. Default is “wait forever,” which is wrong in production. - Call
response.raise_for_status()if you only care about success cases — it turns 4xx/5xx into exceptions. - Catch the specific exception you can handle, then a broader fallback. Don’t catch
Exception.
❌ Common mistakes
- Using
data=for JSON. Usejson=— it sets Content-Type and serialises for you. - Building query strings by hand. Use
params=. - Skipping
timeout=. A hung request can starve your worker for minutes. - Treating
response.json()as safe. Wrap in a try/except — HTML error pages are not JSON. - Hardcoding secrets in source. Use environment variables and add
.envto.gitignore. - Retrying 4xx responses. Those won’t resolve themselves; the bug is in your request.
- Calling APIs in a loop without rate-limit awareness. Hit the 429s and you’ll get throttled or banned.
💡 Pro tips
- Use
requests.Session()for multiple calls to the same host. It reuses TCP connections and shares cookies and headers. - Set a default header (
session.headers["User-Agent"] = "myapp/1.0") on the session so every request carries it. - For high-throughput work, switch to
httpx— same API surface, supports async. - Log
response.urlwhen debugging — you’ll catch params encoding issues fast. - For complex auth (OAuth, AWS SigV4), use a maintained auth helper rather than building it by hand.
Conclusion
Three patterns cover almost every real API call: GET with optional params, POST with a JSON body,
and any of them with an Authorization header. Wrap each in a timeout and an exception
handler and you have a production-shaped client.
Next step: turn the patterns into a small library for whatever API you actually need to call. The
Twitter (X) API, OpenAI, GitHub, Stripe — all of them are requests.get/post with
different paths and tokens.
Related reading: AI agents Day 2: tool calling with LangChain — how AI actually works (tokens & context) — LangChain review — deploy a Python AI agent with FastAPI + Vercel