How to Deploy a LangChain App to Production

Deployment Options at a Glance

Option	Best For	Ops Overhead
LangGraph Platform	Managed agents, built-in streaming & persistence	Minimal
FastAPI + Docker	Full control, existing infra, custom APIs	Medium
Serverless (Lambda / Cloud Run)	Low traffic, event-driven, cost-sensitive	Low
Kubernetes	High traffic, complex scaling requirements	High

LangGraph Platform

LangGraph Platform is LangChain's managed hosting for LangGraph graphs. It adds persistence, streaming, cron jobs, webhooks, and a REST API with zero infrastructure setup.

bash

# langgraph.json — deployment manifest
{
  "dependencies": ["."],
  "graphs": {
    "agent": "./my_app/agent.py:graph"
  },
  "env": ".env"
}

# Deploy with the CLI
# pip install langgraph-cli
# langgraph build -t my-agent-image
# langgraph deploy

# The platform exposes a REST API automatically:
# POST /runs           — invoke the graph
# POST /runs/stream    — stream output tokens
# GET  /runs/{run_id}  — poll run status
# GET  /threads        — list conversation threads (with persistence)

Built-in persistence

Postgres-backed checkpointer — conversation state survives restarts automatically.

Streaming

Server-sent events for token streaming and graph state updates out of the box.

Human-in-the-loop

Pause a run, wait for human approval, then resume — all via the REST API.

FastAPI REST API

Wrap your LangChain chain or LangGraph graph in a FastAPI app for full control over the API surface, middleware, and auth.

python

from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
import asyncio

app = FastAPI()
llm   = ChatOpenAI(model="gpt-4o-mini", streaming=True)
chain = ChatPromptTemplate.from_template("Answer: {question}") | llm | StrOutputParser()

class ChatRequest(BaseModel):
    question: str

@app.post("/chat")
async def chat(req: ChatRequest):
    answer = await chain.ainvoke({"question": req.question})
    return {"answer": answer}

@app.post("/chat/stream")
async def chat_stream(req: ChatRequest):
    async def token_generator():
        async for chunk in chain.astream({"question": req.question}):
            yield f"data: {chunk}\n\n"
        yield "data: [DONE]\n\n"

    return StreamingResponse(token_generator(), media_type="text/event-stream")

# Run: uvicorn app:app --host 0.0.0.0 --port 8000 --workers 4

Dockerizing Your App

bash

# Dockerfile
FROM python:3.12-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 8000

# Use multiple workers in production
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "2"]

yaml

# docker-compose.yml
version: "3.9"
services:
  api:
    build: .
    ports: ["8000:8000"]
    env_file: .env
    depends_on: [redis, postgres]

  redis:
    image: redis:7-alpine
    ports: ["6379:6379"]

  postgres:
    image: pgvector/pgvector:pg16
    environment:
      POSTGRES_DB: langchain
      POSTGRES_USER: langchain
      POSTGRES_PASSWORD: secret
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

Serverless Deployment

For low-traffic or event-driven workloads, deploy as a serverless function. The main consideration is cold-start time — pre-loading the chain at module level helps.

python

# handler.py — AWS Lambda (Mangum adapter)
from mangum import Mangum
from fastapi import FastAPI
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

app = FastAPI()

# Pre-load at module level — survives warm container re-use
llm   = ChatOpenAI(model="gpt-4o-mini")
chain = ChatPromptTemplate.from_template("Answer: {question}") | llm | StrOutputParser()

@app.post("/chat")
async def chat(body: dict):
    return {"answer": await chain.ainvoke({"question": body["question"]})}

# Lambda entry point
handler = Mangum(app)

# Deploy: zip handler.py + dependencies → Lambda
# Or use: AWS SAM / Serverless Framework / Pulumi

Cold start tip: Lambda cold starts can add 2–5 s. Use Provisioned Concurrency for latency-sensitive endpoints, or prefer Cloud Run with minimum instances set to 1.

Environment & Config Management

python

# config.py — centralized settings with pydantic-settings
from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    openai_api_key:      str
    langchain_api_key:   str = ""
    langchain_tracing:   bool = False
    model_name:          str = "gpt-4o-mini"
    embedding_model:     str = "text-embedding-3-small"
    vector_store_url:    str = "postgresql://..."
    redis_url:           str = "redis://localhost:6379"
    max_tokens:          int = 2048
    temperature:         float = 0.0

    class Config:
        env_file = ".env"

settings = Settings()

# Use in app
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(
    model=settings.model_name,
    temperature=settings.temperature,
    max_tokens=settings.max_tokens,
    api_key=settings.openai_api_key,
)

Production Readiness Checklist

Observability

LangSmith tracing enabled in all environments
Structured logging (JSON) with request IDs
Token usage and cost tracked per request
Latency metrics exported to Prometheus / Datadog

Reliability

Retry with exponential backoff on all LLM calls
Fallback model configured for outages
Timeouts set on all external calls
Recursion limits on agent/graph loops

Security

API keys in secrets manager (not env files in prod)
Input sanitization before prompt injection
Rate limiting per user / IP
PII scrubbed from traces

Quality

Eval suite runs on every PR (CI/CD)
Golden dataset maintained and versioned
A/B test new models before full rollout
User feedback loop connected to dataset

You've Completed the LangChain Series!

You've covered the full LangChain stack — from model I/O and chains all the way to multi-agent systems, advanced RAG, evaluation, and production deployment. Before you ship, wire up Tracing and Evaluation so you can monitor live traffic, and revisit how to build a RAG chatbot if your deployment serves a retrieval app. The ecosystem moves fast; keep an eye on the official docs and the LangChain blog for new releases.

What to explore next

LangGraph Studio — visual debugging for graph-based agents
LangChain Templates — ready-made production starters
LangSmith Playground — iterate on prompts with side-by-side comparison
OpenGPTs — open-source GPTs built with LangGraph

LangChain Deployment FAQ

How do I serve a LangChain chain as an API?

Wrap the chain in a web framework and call it from a route handler. The simplest path is FastAPI: define a request model, call await chain.ainvoke(...) inside an async endpoint, and return the result as JSON. For a managed option, LangGraph Platform exposes your graph as a REST API automatically with no server code to write.

Should I use FastAPI or LangServe to deploy a LangChain app?

Use FastAPI when you want full control over routes, middleware, auth, and response shapes. LangServe sits on top of FastAPI and auto-generates invoke, batch, and stream endpoints plus a playground from a runnable, so it is faster to stand up. For agent workloads with persistence and streaming built in, LangGraph Platform is the managed alternative.

How do I stream tokens from a LangChain API?

Iterate over chain.astream(...) and yield each chunk from an async generator wrapped in a StreamingResponse with media_type text/event-stream. Clients consume the server-sent events as tokens arrive, which makes responses feel instant instead of waiting for the full completion.

How do I manage API keys and secrets in production?

Never commit keys to source control. Load them through pydantic-settings from environment variables during development, and in production inject them from a secrets manager such as AWS Secrets Manager, GCP Secret Manager, or Kubernetes secrets. Keep the OpenAI and LangSmith keys out of traces and logs.

How do I scale a LangChain app to handle more traffic?

Most LangChain latency is spent waiting on the model API, so favor async endpoints and run multiple uvicorn workers. Scale horizontally by adding container replicas behind a load balancer on Cloud Run or Kubernetes, set minimum instances to avoid cold starts, and add retries with backoff plus a fallback model for resilience.

How do I containerize a LangChain app with Docker?

Start from python:3.12-slim, copy and install requirements.txt, copy your code, expose the port, and run uvicorn with multiple workers as the CMD. Use docker-compose to bring up dependencies like Redis and a pgvector Postgres alongside the API, and pass secrets via env_file or the orchestrator.