DS DevShelfHub Projects · AI tools
Articles / Python Requests in 15 Minutes: Call Any API From Scratch

AI Engineering

Python Requests: Call Any API in 15 Minutes

By DevShelfHub

Everything you need to call any HTTP API from Python — URL anatomy, request and response structure, GET/POST/PUT/PATCH/DELETE, query parameters with params=, JSON bodies with json=, headers and Bearer tokens, error handling with timeouts and exception types, plus the common mistakes (data= vs json=, hardcoding secrets, no timeout) that turn a script into a production bug.

Python Requests: Call Any API in 15 Minutes

Introduction

Python’s requests library is the single most useful module outside the standard library — it’s how you call APIs, scrape pages, send webhooks, integrate with any HTTP service. If you can use it well, you can build against most of the internet.

This is the fifteen-minute version. URL anatomy, request and response structure, the four HTTP methods that cover 95% of work, query parameters, JSON bodies, headers, auth tokens, error handling. Everything you need to call any API from a Python script.

📚 Table of contents

  • URL anatomy — domain, path, query parameters
  • Request and response structure
  • HTTP methods you’ll actually use
  • Status codes worth memorising
  • Installing requests
  • Your first GET request
  • Query parameters with params=
  • POST requests with JSON bodies
  • Headers and authorization tokens
  • Error handling: timeouts, raise_for_status, exception types
  • Common mistakes
  • FAQs

URL anatomy — domain, path, query parameters

A URL splits into three labelled parts:

URL
https://api.example.com/v1/users?page=2&sort=name
\__________________/\______/\_____________________/
       domain        path          query params
  • Domain — the host serving the request.
  • Path — the resource or endpoint within that host (/v1/users).
  • Query parameters — everything after the ?, key-value pairs separated by &. Optional. Used to filter, paginate, or modify behavior.

Request and response structure

Every HTTP exchange is one client request and one server response. A request contains:

  • Method — GET, POST, PUT, PATCH, DELETE.
  • Path — with optional query params.
  • Headers — metadata: content type, auth token, custom keys.
  • Body — payload data, usually JSON. Required for POST/PUT/PATCH, omitted for GET/DELETE.

A response contains:

  • Status code — the result number. 200 OK, 404 Not Found, etc.
  • Headers — server-side metadata.
  • Body — the actual returned data.

HTTP methods you’ll actually use

Method Intent Body?
GETRead / fetch dataNo
POSTCreate a new resourceYes
PUTReplace a resource entirelyYes
PATCHUpdate part of a resourceYes
DELETERemove a resourceOptional

Status codes worth memorising

  • 2xx success — 200 OK, 201 Created, 204 No Content.
  • 3xx redirects — 301 Moved Permanently, 302 Found, 304 Not Modified.
  • 4xx client errors — 400 Bad Request, 401 Unauthorized, 403 Forbidden, 404 Not Found, 429 Too Many Requests.
  • 5xx server errors — 500 Internal Server Error, 502 Bad Gateway, 503 Service Unavailable, 504 Gateway Timeout.

Rule of thumb: 4xx means you did something wrong (bad input, no auth). 5xx means the server is having a bad time. Retry logic should be reserved for 5xx and 429, not for 4xx errors that won’t fix themselves.

Installing requests

Bash
# pip
pip install requests

# uv (recommended)
uv init .
uv add requests
uv run main.py

Your first GET request

Python
import requests

url = "https://jsonplaceholder.typicode.com/posts/1"
response = requests.get(url)

print(response.status_code)            # 200
print(response.headers["Content-Type"])# application/json; charset=utf-8
data = response.json()                  # parse JSON body into a dict
print(data["title"])
print(data)                             # full payload

The response object has the four things you actually use: response.status_code, response.headers, response.json() to parse JSON, and response.text for raw string bodies.

Query parameters with params=

Don’t hand-build query strings. Pass a dict to params= and let requests encode it correctly:

Python
params = {"page": 2, "limit": 25, "sort": "name"}
response = requests.get("https://api.example.com/users", params=params)
print(response.url)
# https://api.example.com/users?page=2&limit=25&sort=name

Special characters and spaces are URL-encoded for you. Hand-built strings break on the first space or ampersand in a value.

POST requests with JSON bodies

Python
payload = {"title": "Hello from Python", "body": "First post", "userId": 1}
response = requests.post(
    "https://jsonplaceholder.typicode.com/posts",
    json=payload,
)

print(response.status_code)   # 201 Created
print(response.json())

Important detail: use json=payload, not data=payload. json= serialises to JSON and sets the Content-Type header automatically. data= sends a form-urlencoded body, which most JSON APIs reject.

Headers and authorization tokens

Most real APIs require an auth token in the Authorization header. The convention is Authorization: Bearer <token>:

Python
import os
import requests

token = os.environ["X_API_TOKEN"]
headers = {"Authorization": f"Bearer {token}"}

response = requests.get(
    "https://api.x.com/2/users/me",
    headers=headers,
)
response.raise_for_status()
print(response.json())

Never hardcode tokens in source files. Use environment variables or a secrets manager. Committing a real API key to GitHub is one of the most common ways apps get breached — even well-funded companies make this mistake.

Error handling: timeouts, raise_for_status, exception types

A network call can fail in dozens of ways: timeout, DNS failure, connection reset, bad SSL, HTTP error code. Handle them explicitly:

Python
import requests
from requests.exceptions import Timeout, RequestException

try:
    response = requests.get(
        "https://httpbin.org/delay/3",
        timeout=5,           # seconds; always set a timeout
    )
    response.raise_for_status()  # raises HTTPError on 4xx/5xx
    print(response.json())
except Timeout:
    print("request timed out")
except RequestException as e:
    print(f"request failed: {e}")

Three rules:

  • Always pass timeout=. Default is “wait forever,” which is wrong in production.
  • Call response.raise_for_status() if you only care about success cases — it turns 4xx/5xx into exceptions.
  • Catch the specific exception you can handle, then a broader fallback. Don’t catch Exception.

❌ Common mistakes

  • Using data= for JSON. Use json= — it sets Content-Type and serialises for you.
  • Building query strings by hand. Use params=.
  • Skipping timeout=. A hung request can starve your worker for minutes.
  • Treating response.json() as safe. Wrap in a try/except — HTML error pages are not JSON.
  • Hardcoding secrets in source. Use environment variables and add .env to .gitignore.
  • Retrying 4xx responses. Those won’t resolve themselves; the bug is in your request.
  • Calling APIs in a loop without rate-limit awareness. Hit the 429s and you’ll get throttled or banned.

💡 Pro tips

  • Use requests.Session() for multiple calls to the same host. It reuses TCP connections and shares cookies and headers.
  • Set a default header (session.headers["User-Agent"] = "myapp/1.0") on the session so every request carries it.
  • For high-throughput work, switch to httpx — same API surface, supports async.
  • Log response.url when debugging — you’ll catch params encoding issues fast.
  • For complex auth (OAuth, AWS SigV4), use a maintained auth helper rather than building it by hand.

Conclusion

Three patterns cover almost every real API call: GET with optional params, POST with a JSON body, and any of them with an Authorization header. Wrap each in a timeout and an exception handler and you have a production-shaped client.

Next step: turn the patterns into a small library for whatever API you actually need to call. The Twitter (X) API, OpenAI, GitHub, Stripe — all of them are requests.get/post with different paths and tokens.

Related reading: AI agents Day 2: tool calling with LangChainhow AI actually works (tokens & context)LangChain reviewdeploy a Python AI agent with FastAPI + Vercel

Python Requests in 15 Minutes: Call Any API From Scratch FAQ

requests vs httpx vs urllib3?

requests is simple and synchronous—the default. httpx is a drop-in replacement with async and HTTP/2 support. urllib3 is the lower-level library underneath requests. Scripts: use requests. High-throughput async services: use httpx.

How do I upload a file?

Pass a files dict: requests.post(url, files={'file': open('a.png', 'rb')}). The library sets multipart encoding automatically.

How do I download a large file without loading it into memory?

Use stream=True and iterate with response.iter_content(chunk_size=8192). Each chunk writes straight to disk; you never hold the whole file in memory.

How do I retry on transient errors?

Use requests.adapters.HTTPAdapter with urllib3.util.Retry, or the tenacity library. Retry on 429, 502, 503, 504; never on 4xx errors (except 429).

Can I use requests in async code?

Not directly—requests is synchronous and blocks the event loop. Run it in a thread pool with run_in_executor, or switch to httpx which has a native async client.

How do I debug a failing request?

Print response.url, response.status_code, and response.text. For tougher cases, enable HTTP debug logging or use mitmproxy as a proxy to inspect the full request and response.