Production RAG: Redis Semantic Caching, BetterDB, and Claude MCP

Introduction

Building a RAG application is the easy part. You chunk a PDF, push embeddings into a vector store, wire up a retrieval call, and ship a tidy notebook demo. The moment that pipeline meets real users, however, everything that looked clean in development starts misbehaving—latency creeps up, the same questions get re-computed thousands of times, LLM bills bloat, and nobody has any visibility into what the system is actually doing.

This article walks through a practical blueprint for taking a RAG pipeline to production using Redis (or Valkey) for semantic caching, agent memory, and rate limiting, and BetterDB as the AI-native observability layer on top of Redis. We’ll also hook the whole stack to Claude via MCP so you can debug your cache, anomalies, and memory in plain English.

📚 Table of contents

💥 Why a vanilla RAG pipeline breaks in production

A typical RAG demo has two endpoints: /ingest and /query. Ingest takes a PDF, chunks it, embeds it, and writes the vectors. Query takes a user question, embeds it, runs cosine similarity, and asks an LLM to ground its answer on the retrieved chunks. Clean. Linear. Easy to demo.

The problems start the day you hand the URL to anyone other than yourself:

👉 Production RAG is less about better embeddings and more about caching, observability, and discipline around what you don’t recompute.

🧭 The production architecture at a glance

The architecture we’re building keeps the RAG flow intact but slots in two new pieces around it.

Conceptually it’s a stack: RAG → Redis/Valkey → BetterDB → Claude. Every layer adds a capability that the previous one couldn’t do on its own.

🗄️ Redis and Valkey: the in-memory backbone

Redis (REmote DIctionary Server) is an open-source, in-memory data structure store used as a cache, a message broker, and a lightweight database. Valkey is the fully open-source fork of Redis that emerged after the licensing change—wire-compatible, drop-in, and maintained by the Linux Foundation. For the purposes of this project, anywhere you see “Redis” you can mentally substitute “Valkey” and vice versa.

Inside a RAG stack, Redis/Valkey ends up doing four jobs at once:

You can run it locally with a single docker compose up -d, use a managed service like Upstash, or self-host with persistence enabled—the rest of the stack is identical.

⚡ Semantic caching: how it actually works

A regular cache uses string equality. “What is RAG?” and “Explain retrieval augmented generation” are two different strings, so a regular cache misses both. A semantic cache compares queries by meaning, not text—both questions hash to nearby points in embedding space, so the second one hits.

The numbers in the demo are honest about the win: a fresh query took roughly 1.2 seconds, while a repeat (or a paraphrase) came back in around 220 milliseconds with cache_hit=true. That’s a 5–6x latency improvement and the LLM didn’t bill you for the second call.

👉 Pick your similarity threshold carefully. Too loose, and unrelated questions get the same answer. Too tight, and you barely cache anything. 0.93–0.97 is a reasonable starting band for most domains.

⏱️ TTL: the most overlooked knob in RAG

Every key written to Redis can carry a Time To Live—a duration after which the key is automatically deleted. Set TTL well and your cache stays fresh. Set it poorly and you either serve stale answers forever or pay for the LLM on every request.

In BetterDB you can see the TTL distribution of every key namespace at a glance—how many are expiring soon, how many will live forever, and how the average TTL has drifted over time. That single view catches a category of bugs you’d otherwise never notice.

🧠 Agent memory and session storage

Multi-turn conversations require state. The cleanest pattern is to push session history into Redis under a key shaped like session:<user_id>:<chat_id>, with a sliding TTL that resets each time the user sends another message. LangChain ships first-class Redis-backed memory classes that do exactly this, so the integration is two or three lines of code.

Three logical namespaces, one data store, one operational story. That consolidation is the real productivity win—you don’t need a separate cache, separate session DB, and separate vector store with three different SLAs to babysit.

🔭 BetterDB: observability built for Redis-backed AI

Once Redis is doing four jobs, the next question is “how do I see what’s actually happening in there?” That’s where BetterDB sits—a purpose-built observability platform for Redis and Valkey, with first-class support for AI workloads. Think of it as Datadog, but laser-focused on Redis and aware of patterns like RAG, semantic cache, agent memory, and MCP traffic.

Crucially, BetterDB also ships an AI agent and an MCP server—which means you can drive the whole dashboard from natural language inside Claude. We’ll come back to that.

🧰 Project walkthrough: cloning, env setup, Docker Compose

Time to wire it together. The full project is a deliberately simple FastAPI service with /ingest and /query endpoints, a Redis/Valkey container, and a BetterDB agent that connects out to the BetterDB cloud (or self-hosted).

git clone <repo-url> betterdb-redis
cd betterdb-redis
uv venv
source .venv/bin/activate
uv pip install -e .

OPENAI_API_KEY=sk-...
REDIS_URL=redis://localhost:6379
BETTERDB_URL=wss://your-workspace.betterdb.io
BETTERDB_TOKEN=<generated-in-betterdb-settings>

docker compose up -d
docker compose ps

docker run -d \
  --name betterdb-agent \
  -e BETTERDB_URL=wss://your-workspace.betterdb.io \
  -e BETTERDB_TOKEN=<your-token> \
  betterdb/agent:latest

🛠️ Running the ingest and query APIs

With Valkey and BetterDB connected, boot the FastAPI app:

uvicorn main:app --reload --port 8000

Open http://localhost:8000/docs for the automatic Swagger UI. You’ll see three endpoints:

Run an ingest, then run two queries: ask the same question twice. The first response will report cache_hit=false with multi-second latency, the second will report cache_hit=true with sub-300 ms latency. Open BetterDB and click Trigger collection—you’ll see the new keys, their TTLs, and the namespace breakdown update live.

🧩 Hooking BetterDB to Claude with MCP

The Model Context Protocol (MCP) is Anthropic’s open standard for letting tools expose themselves to AI assistants. BetterDB ships an MCP server—point Claude at it once and you can drive your cache analytics in natural language.

{
  "mcpServers": {
    "betterdb": {
      "command": "npx",
      "args": ["-y", "@betterdb/mcp-server"],
      "env": {
        "BETTERDB_URL": "wss://your-workspace.betterdb.io",
        "BETTERDB_TOKEN": "<your-mcp-token>"
      }
    }
  }
}

Claude calls the BetterDB MCP tools under the hood, fetches the live data, and answers in plain English—often with patterns you would’ve missed on a dashboard, like “two distinct anomaly clusters separated by 16 hours.” That kind of pattern recognition is why MCP-backed dashboards are so valuable for on-call.

✅ Best practices for production RAG

Common mistakes & pro tips

Conclusion

A RAG demo is a hundred lines of Python. A production RAG service is the same hundred lines plus the discipline of caching, observability, memory, and rate limiting. Redis and Valkey give you the in-memory backbone to do all four cleanly. BetterDB gives you the eyes to see what’s actually happening. Claude with MCP turns that visibility into a conversation instead of a chore.

Build the pipeline, but treat the layer around it as a first-class part of the architecture. The teams that ship reliable AI products are not the ones with the fanciest retrievers—they’re the ones who can answer “what is the cache doing right now?” in under a minute.

Related reading: RunPod serverless RAG pipeline guide — NVIDIA DGX Spark hands-on