Building AI Agents — Day 4: Routes, Subgraphs, and Deployment

Introduction

Day 4 of the Modern Route AI Agents crash course is where the conversation shifts. The first three days were about getting a single agent to do something useful — pick a tool, generate a reply, handle a state. Day 4 is about what happens when you have to put that agent in front of real users. Routes, services, retries, logging, cloud-integrated config, Docker, secret managers, observability, deployment topology. The unglamorous half of agent engineering.

The walkthrough is built around one concrete project — an autonomous research and analyst report generation system in LangGraph — but the lessons are framework-agnostic. By the end of this article you’ll have a picture of how a production-shaped agent codebase is organised, how a non-trivial LangGraph workflow with subgraphs, parallel branches, conditional edges, and a human-in-the-loop is wired up, and which production concerns (logging, secrets, cloud integration, tracing, deployment) you can’t skip if the agent is going to run for paying customers rather than a YouTube demo.

📌 Part of a 4-day crash course. Day 1 covered the agent mental model, Day 2 built the first LangGraph workflow, Day 3 layered project structure and config, and Day 4 (this article) is the production pass: routes, services, parallel subgraphs, cloud integration, and how to ship.

📚 Table of contents

🏗️ What changes when you move from a demo agent to production

A demo agent fits in one notebook cell. A production agent is an entire service. The shift looks obvious on paper and is painful in practice. Day 4 spends most of its time on the things that don’t fit in a notebook.

The job title shifts too. A demo agent is a prompt-engineering exercise. A production agent is an AI engineering exercise — the LLM is one component in a service that has the same operational concerns as any other backend.

🧭 The project: autonomous research and analyst report generator

The walkthrough is built around a system that takes a topic from the user, spins up multiple AI analyst personas, runs a parallel research interview for each, and stitches the outputs into a final report — downloadable as DOCX or PDF. It’s a deliberately non-trivial agent because every interesting production concern shows up in it.

Why this shape matters: it forces every production concern into the open. Multiple LLM calls per request (cost). Parallel branches (concurrency). Human approval (state pause). Tool calls to the open web (failure handling). File output (storage). Auth (DB). It’s a representative shape for the kind of agent product an enterprise actually pays for.

🧱 Codebase shape — routes, services, models, workflow

The folder layout is the first concrete production lesson. A real agent service is not one file. The walkthrough splits the code into four clear layers and that separation is what makes the system testable and deployable.

research_analyst/
├── api/
│   ├── main.py              # FastAPI app, mounts all routes
│   ├── routes.py            # Endpoint definitions
│   ├── services.py          # Business logic called by routes
│   └── models.py            # Pydantic request/response models
├── workflow/
│   ├── report_generation_workflow.py
│   └── interview_graph_builder.py
├── prompts/
│   └── prompt_templates.py
├── utils/
│   ├── model_loader.py
│   ├── config_loader.py
│   └── cloud.py             # S3, Secrets Manager, Bedrock helpers
├── exception/
│   └── exception.py
├── logger/
│   └── logger.py
├── generated_reports/       # output artefacts
├── logs/                    # per-run log files
├── requirements.txt
├── Dockerfile
└── .env                     # local only — never in prod

The win from this layout: the workflow file knows nothing about FastAPI, and the routes know nothing about LangGraph. You can swap one without touching the other. That’s ordinary backend hygiene — agent codebases skip it more often than they should.

🧠 Inside the LangGraph workflow

The workflow is built as a state graph. Each box on the diagram is a function (a node); each arrow is an edge. The state object — a typed dict in Python — flows through every node.

def build_graph(self):
    builder = StateGraph(ReportGenerationState)

    builder.add_node("create_analyst", self.create_analyst)
    builder.add_node("human_feedback", self.human_feedback)
    builder.add_node("conduct_interview", self.interview_graph)
    builder.add_node("write_report", self.write_report)
    builder.add_node("write_introduction", self.write_introduction)
    builder.add_node("write_conclusion", self.write_conclusion)
    builder.add_node("finalize_report", self.finalize_report)

    builder.add_edge(START, "create_analyst")
    builder.add_edge("create_analyst", "human_feedback")
    builder.add_conditional_edges(
        "human_feedback",
        self.initiate_all_interviews,
        ["create_analyst", "conduct_interview", END],
    )
    builder.add_edge("conduct_interview", "write_report")
    builder.add_edge("conduct_interview", "write_introduction")
    builder.add_edge("conduct_interview", "write_conclusion")
    builder.add_edge("write_report", "finalize_report")
    builder.add_edge("write_introduction", "finalize_report")
    builder.add_edge("write_conclusion", "finalize_report")
    builder.add_edge("finalize_report", END)

    return builder.compile(checkpointer=self.checkpointer)

The structure is doing real work. Three things in particular — subgraphs, conditional edges, and the send API — are what separate this from a toy graph.

🪆 Subgraphs, parallelization, and the send API

A subgraph is a self-contained graph used as a single node inside the parent. The interview logic — ask a question, search the web, generate an answer, save the interview, write a section — is its own state machine. Embedding it as a subgraph keeps the parent diagram readable and keeps the interview state isolated.

There’s an internal loop here: if the answer isn’t good enough, the subgraph routes back to generate_question for another round, capped at a max-turn budget. That cap matters — without it, an evaluator-LLM that’s feeling pedantic can keep the loop running until your token bill is unreadable.

Parallelization is what makes the system fast. With N analysts, you don’t want to run N interviews serially. LangGraph’s Send primitive fans the work out:

def initiate_all_interviews(self, state: ReportGenerationState):
    human_feedback = state.get("human_analyst_feedback", "")

    if human_feedback:
        return "create_analyst"  # rebuild personas based on feedback

    topic = state["topic"]
    return [
        Send(
            "conduct_interview",
            {
                "analyst": analyst,
                "messages": [HumanMessage(content=f"So you said you were writing about {topic}?")],
                "max_num_turns": 2,
                "context": [],
            },
        )
        for analyst in state["analysts"]
    ]

Each Send spawns an independent invocation of the subgraph with its own state. LangGraph executes them concurrently and waits for all branches to complete before moving to the report-writing stage. The downstream write_report, write_introduction, and write_conclusion nodes are also parallel — three edges from one source means three concurrent calls.

🔀 Conditional edges and the human-in-the-loop

Normal edges run sequentially. Conditional edges run a router function and pick the next node from a set of allowed targets. Most agent decisions — tool call vs final answer, retry vs continue, escalate vs resolve — are conditional edges.

The classic example is the ReAct loop. The LLM either answers directly, or it emits a tool call. A router inspects the message:

def route_tools(state: AgentState) -> Literal["tools", END]:
    last_message = state["messages"][-1]
    if hasattr(last_message, "tool_calls") and last_message.tool_calls:
        return "tools"
    return END

builder.add_conditional_edges("llm", route_tools, ["tools", END])
builder.add_edge("tools", "llm")  # back to the brain after the tool runs

In the report generator, the conditional edge sits after human_feedback. Three outcomes: feedback was given → rebuild personas; feedback was empty → start interviews; user aborted → end the run. The human-in-the-loop is implemented as a pause point in the graph — LangGraph’s checkpointer persists state, the UI submits feedback, the graph resumes from where it stopped.

🧬 Pydantic, state, and structured outputs

Four things show up in every well-built agent node, and the walkthrough is explicit about it:

class Analyst(BaseModel):
    name: str = Field(description="Name of the analyst")
    role: str = Field(description="Role of the analyst in the context of the topic")
    affiliation: str = Field(description="Primary affiliation of the analyst")
    description: str = Field(description="Description of the analyst's focus, concerns, and motives")

class Perspectives(BaseModel):
    analysts: List[Analyst] = Field(description="Comprehensive list of analysts with their roles")

def create_analyst(self, state: GenerateAnalystState):
    structured_llm = self.llm.with_structured_output(Perspectives)
    system_message = ANALYST_INSTRUCTIONS.format(
        topic=state["topic"],
        human_analyst_feedback=state.get("human_analyst_feedback", ""),
        max_analysts=state["max_analyst"],
    )
    analysts = structured_llm.invoke(
        [SystemMessage(content=system_message), HumanMessage(content="Generate the set of analysts.")]
    )
    return {"analysts": analysts.analysts}

Pydantic at every layer — HTTP request, internal state, LLM output — is the single biggest defence against silent agent breakage in prod.

📊 Logging, tracing, and observability

A demo agent that prints to stdout is useless the moment a real user hits it. Day 4’s repo bakes in two layers of visibility.

# .env (local) or AWS Secrets Manager (prod)
LANGCHAIN_TRACING_V2=true
LANGCHAIN_API_KEY=ls__xxx
LANGCHAIN_PROJECT=research-analyst-prod

The minimum bar to call an agent service production-ready: per-run trace, per-node latency, per-call token cost, retry counts, and a way to replay a failed run from its checkpoint.

☁️ Cloud-integrated config: secrets, storage, models

A .env file is fine on your laptop. In production it’s a security incident waiting to happen. Day 4 makes the explicit point that the same code path should pull secrets from a cloud-native store in prod, and from .env in local dev — with the switch driven by an environment flag, not a code change.

def load_secret(name: str) -> str:
    if os.getenv("APP_ENV") == "production":
        client = boto3.client("secretsmanager", region_name="ap-south-1")
        return json.loads(client.get_secret_value(SecretId=name)["SecretString"])["value"]
    return os.getenv(name, "")

🚢 Deployment topology: Docker, ECR, Lambda, ECS

The repo ships a Dockerfile for a reason. The deployment story isn’t “run uvicorn on an EC2 box” — it’s containerise, push, run.

FROM python:3.11-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

ENV APP_ENV=production
EXPOSE 8000

CMD ["uvicorn", "research_analyst.api.main:app", "--host", "0.0.0.0", "--port", "8000"]

The CI/CD path is standard: GitHub Actions builds the image, pushes to ECR, ECS pulls the new tag, rolling deploy. Nothing agent-specific — which is the point. An agent service is just a service.

💸 Cost and latency in a multi-LLM pipeline

A single report-generation run in this system fires a lot of LLM calls. Roughly: one to draft the personas, one to refine on feedback, three per analyst inside the interview loop, three section writes, plus intro and conclusion. With three analysts that’s comfortably 15+ calls per user request. Cost stops being a footnote.

Latency follows the same shape. The parallel Send fan-out is what keeps the total wall-clock time bounded — otherwise three analysts in series would triple the response time. Track p50 and p95 per node, not just per request.

🛡️ Guardrails, retries, and failure handling

Every node that calls something external — the LLM, a search API, a database — can fail. Production agents handle that explicitly.

• Retry on transient errors. Wrap LLM and tool calls with tenacity-style exponential backoff. Cap at three tries.
• Validate structured output. If pydantic parsing fails, retry once with a corrective prompt. After that, downgrade to a safe default and log the failure.
• Cap iteration counts. Max turns in the interview loop, max tool-call depth in the ReAct cycle. No unbounded loops in prod, ever.
• Timeouts. Per-LLM-call timeout, per-tool-call timeout, per- run timeout. The orchestrator should give up gracefully rather than hang.
• Input validation at the route. Pydantic in the API layer catches malformed requests before they reach the LLM. Cheap defence.
• Output filters. PII redaction on the final report, blocklist for sensitive terms, max length cap so the agent can’t generate a 50-page essay nobody asked for.

✅ Best practices for shipping agents

🚫 Common mistakes

• Skipping the API layer entirely. Exposing a LangGraph run directly to a frontend without a routes/services boundary makes the workflow impossible to test, version, or rate-limit.
• Running heavy agents inside the HTTP request. A 90-second LangGraph run inside a synchronous endpoint will time out behind any reasonable load balancer. Push it to a worker.
• No retry, no fallback. One transient Tavily 503 and the whole report fails. Wrap every external call.
• Storing checkpoints in SQLite in prod. Fine for dev, fatal under concurrency. Move to Postgres or Redis-backed checkpointers before launch.
• Ignoring token cost in design. The cheapest moment to reduce cost is at architecture time — pick which model runs which node before you have a bill.
• No human review of failed runs. If you don’t sample and replay a percentage of failures every week, regressions land silently.

Conclusion

Day 4 is the “everything you’d skip in a tutorial” pass. The LangGraph workflow itself is interesting — subgraphs, parallel Send, conditional edges, a human-in-the-loop checkpoint — but the bulk of the work is in the surrounding service: routes, services, pydantic at every boundary, structured logging, LangSmith tracing, cloud-native secrets, S3 for output, Postgres for checkpoints, and a container that ships cleanly into ECS or Cloud Run.

The mental model to take into your own builds: an agent in production is a regular backend service that happens to call an LLM. The same engineering hygiene you’d apply to any other service applies here. The LLM does not earn you an exemption from timeouts, validation, retries, or observability — it makes all of them more important.