Guardrails with LangChain: Building Safe AI Agents in Production

Introduction

Ship an AI agent without guardrails and you’re basically handing a customer a search bar with a credit card attached. The agent will happily answer “how do I hack into a database?”, echo back a user’s email and card number in plaintext, or fire off a DELETE FROM users because someone asked nicely. None of that is a model problem — it’s a missing safety layer.

Guardrails are the safety mechanisms that sit around your agent pipeline. They decide what inputs are safe enough to reach the LLM, which actions need a human in the loop, and what outputs are allowed to reach the user. This article is a hands-on crash course for building those guardrails inside LangChain — using the built-in middleware system, custom hooks, and a layered defense pattern you can drop into a real production agent.

Table of contents

What guardrails actually are

A clean working definition: guardrails are safety mechanisms that control what goes into and comes out of an AI agent. They sit around the agent pipeline and make sure three things hold:

Concretely, imagine the simplest agent loop: input → LLM → (optional tool call) → output. A guardrail is a check inserted at one or more of those edges. If a user types “how do I hack a server?”, a well-placed input guardrail flags the request before the model wastes a token on it. If the model tries to leak an email in its response, an output guardrail catches it on the way back.

Why guardrails matter for any agent in production

The cost of skipping guardrails is no longer theoretical. Agents now talk to databases, fire APIs, send transactional emails, and operate inside customer-support flows. The damage surface is wide:

• PII leakage. A logging pipeline that captures raw prompts ends up storing credit card numbers and emails in plaintext — a compliance nightmare under GDPR, HIPAA, or PCI-DSS.
• Prompt injection. Untrusted text retrieved from a webpage or document can hijack the agent into ignoring its system prompt and executing attacker instructions.
• Dangerous tool calls. An agent with delete or refund authority can be talked into wiping a table or issuing a payout without any human review.
• Hallucinated answers in regulated contexts. Legal, medical, and financial agents can’t simply make things up — outputs need quality and safety checks before they reach the user.

Guardrails aren’t a nice-to-have. They’re the difference between a demo and an agent you can hand to real users without losing sleep.

Two approaches: deterministic vs model-based

Every guardrail implementation lands in one of two buckets. The right answer in production is usually a mix.

A quick deterministic example in Python:

def deterministic_guardrail(text: str) -> bool:
    """Return True if content is blocked."""
    banned = ["hack", "exploit", "malware", "bomb"]
    text_lower = text.lower()
    return any(keyword in text_lower for keyword in banned)

for q in ["How do I hack into a database?",
          "What is the capital of France?",
          "Explain how malware spreads"]:
    status = "blocked" if deterministic_guardrail(q) else "allowed"
    print(f"{status}: {q}")

And the model-based equivalent:

from langchain_openai import ChatOpenAI

model = ChatOpenAI(model="gpt-4o-mini", temperature=0)

def model_guardrail(text: str) -> str:
    """Use an LLM to evaluate content safety. Returns 'safe' or 'unsafe'."""
    prompt = (
        "Is the following user input safe to process? "
        "Reply with only 'safe' or 'unsafe'.\n\n"
        f"Input: {text}"
    )
    return model.invoke(prompt).content.strip().lower()

Notice the difference on “explain how malware spreads” — the keyword filter blocks it, the model-based check correctly classifies it as a legitimate education request. That’s the whole point of layering both.

Why LangChain — the middleware model

LangChain handles guardrails through middleware: hooks you attach to an agent that fire at well-defined points in the execution loop. Instead of scattering safety checks across your code, you compose them as middleware instances and hand them to create_agent.

The mental model: middleware is just a list. The agent runs each one in order, and any of them can short-circuit the request or mutate its state. That makes it trivial to reason about — you can read the middleware stack top to bottom and know exactly what’s being checked at every stage.

Built-in PII middleware: redact, mask, block

Personally identifiable information (PII) is the most common guardrail failure in real systems — a user pastes a credit card or an SSN into a chat box, and it ends up in your logs. LangChain ships PIIMiddleware with built-in detectors for the obvious offenders.

The setup looks like this:

from langchain.agents import create_agent
from langchain.agents.middleware import PIIMiddleware
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool

@tool
def customer_lookup(query: str) -> str:
    """Look up a customer record."""
    return f"Customer record found for: {query}"

agent = create_agent(
    model=ChatOpenAI(model="gpt-4o"),
    tools=[customer_lookup],
    middleware=[
        PIIMiddleware("email", strategy="redact", apply_to_input=True),
        PIIMiddleware("credit_card", strategy="mask", apply_to_input=True),
        PIIMiddleware(
            "api_key",
            detector=r"sk-[A-Za-z0-9]{32}",
            strategy="block",
            apply_to_input=True,
        ),
    ],
)

Three middlewares, three different strategies. Email gets redacted, credit cards get masked, API keys are blocked outright with a raised exception. The apply_to_input flag controls whether the check runs on inbound messages — you can also apply it to outputs and tool calls.

Now run it:

result = agent.invoke({
    "messages": [{
        "role": "user",
        "content": "My email is john.doe@example.com and my card number "
                   "is 4111-1111-1111-5100. Can you help me?",
    }]
})

The agent never sees the raw email or card number. Internally the request becomes “My email is [REDACTED_EMAIL] and my card number is **** **** **** 5100”, and the response correctly references only the masked digits. Trying to pass an API key like sk-abc123... triggers the block strategy and raises an exception you catch in a try/except.

Human-in-the-loop middleware for sensitive tools

Some actions should never be fully automated. Financial transactions, outbound emails to external parties, production data deletions — anything with real-world consequences needs a human in the loop. LangChain’s HumanInTheLoopMiddleware pauses the agent before specific tool calls and waits for an approve / reject decision.

from langchain.agents import create_agent
from langchain.agents.middleware import HumanInTheLoopMiddleware
from langchain_core.tools import tool
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.types import Command

@tool
def search_web(query: str) -> str:
    return f"Search results for: {query}"

@tool
def send_email(to: str, subject: str, body: str) -> str:
    return f"Email sent to {to} with subject '{subject}'"

@tool
def delete_records(table: str, condition: str) -> str:
    return f"Deleted records from {table} where {condition}"

hitl_agent = create_agent(
    model=ChatOpenAI(model="gpt-4o"),
    tools=[search_web, send_email, delete_records],
    middleware=[
        HumanInTheLoopMiddleware(
            interrupt_on={
                "search_web": False,      # auto-approved
                "send_email": True,       # needs approval
                "delete_records": True,   # needs approval
            }
        )
    ],
    checkpointer=InMemorySaver(),
)

The checkpointer is required — the agent needs to persist state when it pauses, so it can resume on the same thread once a human decides. search_web runs without interruption because it’s read-only. send_email and delete_records pause and wait.

Approve a paused call by resuming the thread with a command:

config = {"configurable": {"thread_id": "session-123"}}

# First call - this will pause at send_email
hitl_agent.invoke(
    {"messages": [{"role": "user", "content": "Send an email to team@company.com about Q4 results"}]},
    config=config,
)

# Approve and resume
hitl_agent.invoke(
    Command(resume={"decision": {"type": "approve"}}),
    config=config,
)

# Or reject with a reason
hitl_agent.invoke(
    Command(resume={"decision": {"type": "reject", "reason": "Too risky, needs DBA review"}}),
    config=config,
)

Custom before-agent hook: cheap, cost-free filtering

When the built-in middleware doesn’t fit your case, write your own. A custom before-agent hook runs before any LLM call — perfect for keyword filters, auth checks, rate limiting, or category blocks. Because it short-circuits before the model fires, blocked requests cost zero tokens.

from langchain.agents.middleware import AgentMiddleware, AgentState, hook_config
from langgraph.runtime import Runtime
from langchain.agents import create_agent

class ContentFilterMiddleware(AgentMiddleware):
    """Deterministic guardrail: block requests containing banned keywords."""

    def __init__(self, banned_keywords: list[str]):
        super().__init__()
        self.banned_keywords = banned_keywords

    @hook_config(jump_to="end")
    def before_agent(self, state: AgentState, runtime: Runtime):
        if not state["messages"]:
            return None

        first = state["messages"][0]
        if first.type != "human":
            return None

        content = first.content.lower()
        for keyword in self.banned_keywords:
            if keyword in content:
                return {
                    "messages": [{
                        "role": "assistant",
                        "content": (
                            f"Blocked: keyword '{keyword}' detected. "
                            "I cannot process this request."
                        ),
                    }]
                }
        return None

The key piece is @hook_config(jump_to="end") — when the guardrail triggers, the agent jumps straight to the end node and returns the blocked-message response. No LLM call, no tool calls, no token spend.

Wire it into an agent:

filter_agent = create_agent(
    model=ChatOpenAI(model="gpt-4o"),
    tools=[customer_lookup],
    middleware=[
        ContentFilterMiddleware(
            banned_keywords=["hack", "exploit", "malware", "bomb"]
        )
    ],
)

# Safe request - runs normally
filter_agent.invoke({"messages": [{"role": "user", "content": "What is machine learning?"}]})

# Unsafe request - blocked before the model is called
filter_agent.invoke({"messages": [{"role": "user", "content": "How do I hack into a server?"}]})

For the second request, the response is the canned blocked message — no token spend, no risk of the model engaging with the prompt. That’s the entire value proposition of cheap deterministic guardrails at the front door.

Custom after-agent hook: validating the final output

The flip side: validate what the agent produced before the user sees it. An after-agent hook is the right place for model-based safety evaluation, compliance scanning (legal, medical, financial disclaimers), and removing any sensitive info that slipped through earlier stages.

class SafetyGuardrailMiddleware(AgentMiddleware):
    """Model-based output guardrail: rewrite or block unsafe responses."""

    def __init__(self):
        super().__init__()
        self.safety_model = ChatOpenAI(model="gpt-4o-mini", temperature=0)

    @hook_config
    def after_agent(self, state: AgentState, runtime: Runtime):
        last = state["messages"][-1]
        if last.type != "ai":
            return None

        prompt = (
            "Evaluate if this AI response is safe and appropriate for users. "
            "Reply only with 'safe' or 'unsafe'.\n\n"
            f"Response: {last.content}"
        )
        verdict = self.safety_model.invoke(prompt).content.strip().lower()

        if "unsafe" in verdict:
            return {
                "messages": [{
                    "role": "assistant",
                    "content": "I cannot share that response. Please rephrase your request.",
                }]
            }
        return None

A couple of details worth calling out:

• Use a cheap model for the check. The safety classifier doesn’t need to be your main model. A small, fast model (GPT-4o-mini, Haiku, Llama-3 8B) keeps cost and latency low.
• Temperature zero. You want deterministic verdicts, not creative ones.
• Mutate, don’t just block. An after-agent hook can rewrite the response — strip a phone number, add a disclaimer, soften tone — instead of just rejecting it.

Layered guardrails: stacking middleware for defense in depth

No single guardrail catches everything. Production systems stack them. Deterministic filters at the front door, PII redaction for compliance, human-in-the-loop on the dangerous tools, model-based safety check on the way out. Each layer is cheap because the previous one already filtered out the obvious cases.

layered_agent = create_agent(
    model=ChatOpenAI(model="gpt-4o"),
    tools=[search_web, send_email, customer_lookup],
    middleware=[
        # Layer 1: cheap deterministic filter
        ContentFilterMiddleware(banned_keywords=["hack", "exploit", "malware"]),

        # Layer 2: PII protection
        PIIMiddleware("email", strategy="redact", apply_to_input=True),
        PIIMiddleware("credit_card", strategy="mask", apply_to_input=True),

        # Layer 3: human approval for sensitive tools
        HumanInTheLoopMiddleware(
            interrupt_on={"send_email": True, "search_web": False}
        ),

        # Layer 4: model-based output safety
        SafetyGuardrailMiddleware(),
    ],
    checkpointer=InMemorySaver(),
)

Order matters. The list runs top to bottom, and each middleware can short-circuit the rest. Put the cheap, fast checks first so you’re not paying for an LLM-based check on a request that was going to be blocked by a regex anyway.

Best practices for shipping guardrails

Common mistakes to avoid

• Treating guardrails as a final-step add-on. Retro-fitting safety on an agent that’s already shipped is harder than building it in from day one. The middleware list should be part of the initial design.
• Trusting the LLM to refuse on its own. System prompts help, but they’re bypassed by prompt injection. A deterministic guardrail outside the model is non-negotiable for high-risk actions.
• Ignoring tool calls. PII can leak through the arguments of a tool call, not just the final response. Make sure your PII middleware is configured to inspect tool inputs too.
• No observability on blocks. If you can’t see which guardrail fired, on which request, and why, you can’t tune the rules. Wire blocked events into your usual logging and dashboards.
• Over-blocking. A guardrail that flags “explain how malware spreads” as unsafe is hostile to legitimate education use cases. Use model-based checks where intent matters.
• Single-language keyword lists. If your agent talks to users in Hindi, Spanish, or French, an English-only banlist won’t catch translated attack strings. Either expand the list or fall back to a multilingual model-based check.

Conclusion

Guardrails aren’t a single feature you bolt on once. They’re a layered, evolving safety surface that wraps your agent from input to output and at every tool call in between. LangChain makes that surface concrete: PIIMiddleware, HumanInTheLoopMiddleware, and the before_agent/after_agent hooks give you four well-defined slots to plug in safety checks.

A reasonable first pass for any production agent: deterministic keyword filter at the front, PII redaction on inputs and outputs, human-in-the-loop on every tool that can do real damage, and a cheap model-based safety check on the way out. Build the stack incrementally, log every block, and treat the guardrail config as code you review and test like any other part of your system.