OpenClaw Architecture: Memory, Context, Caching, and API Cost Cuts

Introduction

If your OpenClaw bill keeps creeping up, your agent forgets things you told it 72 hours ago, and the demos online seem to run faster than yours, the problem isn’t the model. It’s the architecture. OpenClaw is a layered system — an LLM at the front, a stack of memory files behind it, a context-window budget that gets eaten by tool outputs and session history, and a handful of optional optimizations that most people never enable.

This guide walks through the actual mechanics: how memory is stored on disk, why the LLM forgets, what compaction and memory flush do, when to switch to vector memory search, why hosting your tools through Composio is cheaper than wiring them yourself, and how prompt caching turns repeated input tokens into a 90% discount. By the end you should be able to look at your OpenClaw instance and know exactly which knobs to turn first.

📚 Table of contents

🔐 Security first — why not to run OpenClaw on your laptop

OpenClaw is bleeding-edge open source. Vulnerabilities have already been disclosed and patched in the short time it has existed. Treat any instance like a foreign process with hands on your keyboard.

• Don’t run it on the same machine as your primary accounts.
• Connect it to a sandboxed Google/GitHub/Slack account, not your real one.
• Run it in an isolated environment — ideally a VPS, ideally behind Docker.
• Assume any credential it stores in plain text is one prompt injection away from leaking.

A VPS is also cheaper than dedicated hardware once you factor in three years of electricity, backup, and insurance against theft or fire. Even with a high-end Mac mini on your desk, running OpenClaw against a $9–$25/month KVM plan is the safer call.

🚀 One-click deploy and SSH access via VS Code

Hostinger’s one-click OpenClaw template puts a Docker-isolated instance behind your gateway token in under five minutes. Pick the KVM2 plan, launch the template, drop in your Anthropic or OpenAI key, and grab the gateway token from the Docker Manager → Projects panel. Paste it into the OpenClaw control panel and you have a working agent.

Get a file-level view with VS Code over SSH

The terminal works, but you’ll spend most of your tuning time editing markdown and JSON. Connect VS Code to the VPS instead:

1. Set the VPS root password from the Hostinger overview panel.
2. Install the Remote — SSH extension in VS Code.
3. Hit Cmd/Ctrl + Shift + P → Remote-SSH: Connect to Host → enter root@<ip>.
4. Open the folder /docker/<container>/data/.openclaw (on a default Hostinger Docker deploy). On a non-Docker install, look in the user’s home folder.

You now see the full OpenClaw file tree — identity, workspace, agents, tools, and the memory files you’re about to learn to tune.

🧠 The memory architecture

The single most important sentence about OpenClaw memory: if it isn’t on disk, the LLM doesn’t remember it. The LLM is stateless. Anything that survives between sessions has been written to a file and read back into the context window on startup.

Read agents.md in the OpenClaw workspace to see how this is actually wired — the philosophy is explicit in the file, and you can change it if you want different defaults.

🪟 What’s actually in your context window

Every API call ships a stack of inputs to the model. Understanding the layers is the prerequisite to making it cheaper.

• System prompt — fixed, can’t change much.
• Bootstrap files — semi-fixed startup info.
• Memory files — variable, scales with your discipline.
• Skills — every active skill adds a description that gets considered on every call. 200 skills means 200 descriptions in your prompt.
• Conversation history — the number-one cost driver. A 30-minute uncompacted session keeps re-sending every previous turn.
• Tool outputs — results from prior tool calls hang around in the session.
• Compaction summary — the condensed replacement when you compact a session.

Slash commands for an audit

🗜️ Compaction and memory flush

Compaction is the safety valve. When a session gets too large, OpenClaw summarises the previous conversation into a small block of tokens and starts fresh. You want this enabled — without it, every reply re-sends the whole conversation.

The catch: a naive compaction drops information you cared about. Memory flush fixes that. Before compaction, the agent reviews the session and writes any durable facts it should remember into memory.md automatically — even if you never told it to.

Enable memory flush in openclaw.json

Open openclaw.json over your VS Code SSH session, find the agents.defaults block, and add a compaction object that sets a reserve token floor and enables the memory-flush prompt. A floor of around 20,000 tokens keeps useful context alive instead of nuking the whole session.

After saving the config, restart the gateway and ask the agent “is memory flush enabled?”. It will read the config and confirm. From now on, long sessions degrade gracefully instead of catastrophically.

🧭 Vector memory search with the QMD backend

By default OpenClaw scans memory using keyword matching on the markdown files. It works, but it’s blunt — you get a lot of irrelevant context and miss semantically related material.

Vector memory search swaps that out for a small local vector database with embeddings. Memories get converted to numeric vectors; queries are matched by semantic similarity instead of exact text. Searches are faster, more accurate, and pull in less noise. The OpenClaw docs ship an example config for the QMD backend — copy it, paste it into a chat with the agent, and say “enable the QMD backend.” The agent applies the config and restarts the gateway.

💡 The vector backend is experimental as of writing — treat it as a meaningful upgrade for instances with a lot of historical memory, less critical for fresh deploys.

🧰 Tools: why Composio beats hand-wired MCP

Tools are the other big bloat point. Every tool you register adds a schema to the prompt. Have 100 tools wired in, and every single API call asks the LLM to consider 100 tool descriptions before answering. Most of the time it doesn’t need 95 of them.

There’s also a security and reliability tax. Tools you connect by hand store credentials on disk in plain text. Google’s OAuth has started rejecting refresh tokens that originate from OpenClaw, and Anthropic’s terms restrict using subscription tokens against custom tool chains.

Composio’s on-demand tool discovery

Composio is a hosted MCP layer with thousands of integrations behind a single endpoint. Instead of loading 100 tool schemas into your prompt, OpenClaw sees about five Composio meta-tools, including a search-tool. When the agent needs to do something, it calls search-tool, Composio returns a short list of matching tools, and only those schemas come back into the LLM’s view.

The wins:

• Slimmer prompts — only the tools you’re about to use occupy context.
• Just-in-time discovery — new integrations don’t bloat every call.
• OAuth handled remotely — credentials live in Composio, not in plain text on your VPS.
• Free tier covers most users — 20,000 tool calls per month, no card required.

Wire it up

1. Sign up at composio.dev and open the dashboard.
2. Create a workspace, then connect the integrations you need (GitHub, Google, Slack, etc.).
3. Use the “Connect OpenClaw” prompt — it’s a one-line instruction that adds an MCP server called composio with the right HTTP transport and API key header.
4. Ask the agent to list your recent GitHub repos via Composio. The first run installs the connector skill; from there it just works.

💾 Prompt caching and warming the cache

Prompt caching is the single biggest lever between you and your invoice. Anthropic and OpenAI both support it. The math is simple:

The default cache TTL is about an hour. If you go idle long enough, the cache invalidates and the next call pays the write premium again. The fix is cache warming: add a heartbeat that fires every ~55 minutes and runs a trivial task. That keeps the cache fresh for busy windows.

👉 Enable caching in openclaw.json per provider, and add the heartbeat config if you want warming. Even without warming, the savings on any multi-turn session are immediate.

💸 Cost impact matrix

Rough ranking of what actually drives your bill, biggest first:

• Conversation history — every uncompacted turn re-ships everything before it.
• Tool output history — results pile up fast, especially with chatty tools.
• Skills you forgot you enabled — description bloat on every call.
• memory.md size — loaded on every main session.
• Daily logs — pruned automatically, but bloated logs cost on the days they’re in the window.
• System prompt and bootstrap — mostly fixed, modest tuning available.
• Embedding calls — usually local and free; trivial if done right.

🎯 Optimization strategy

1. Session hygiene — /new when a task is done, /compact when a session is mid-thread but getting long.
2. Cap the context window — drop the auto-compact threshold from the default 200k (or 1M on some models) to something tighter, so the agent compacts without you remembering.
3. Disable unused skills — or move tooling to Composio so schemas load on demand.
4. Spawn sub-agents for narrow delegated tasks — they start with fresh context and only get what they need.
5. Enable caching and memory flush — both are config-only changes with outsized impact.
6. Audit regularly — /context detail every few days, especially after adding new skills or memory.

🛣️ Model routing and provider fallbacks

Almost everyone deploys OpenClaw with one default model and uses it for everything — including the cheap, mechanical tasks that don’t deserve a frontier model. Pinning Opus 4.6 to a haiku-sized workload is the single most common reason bills get out of hand.

The fix is to register multiple models from at least two providers and let routing rules pick the right one for the task. The first provider gives you tier diversity (Haiku < Sonnet < Opus); the second provider gives you a rate-limit escape hatch.

Add the second provider’s API key to the env section of openclaw.json, expand the models list with aliases, and paste the rule set into soul.md under a Model routing heading. Restart the gateway and ask the agent “based on your routing rules, what model should you be using for this task?” to sanity-check.

# Model routing rules (read before every task)

- Default model: claude-haiku-4-5
- Switch to claude-sonnet-4-6 only when the task requires design,
  review, security analysis, or a major decision.
- Switch to claude-opus-4-6 only after Sonnet has failed at advanced
  reasoning twice on the same task.

# Rate-limit fallback (if Anthropic is unavailable)

- Cheap tasks  -> gpt-5-mini
- Complex tasks -> gpt-5.1

# Rules of engagement

- Never switch models mid-task unless a rate limit is hit.
- Never use a premium model for reading files, formatting, or status updates.

🧾 Soul.md guardrails — session init, spending limits, local heartbeats

soul.md is OpenClaw’s standing-orders file. Three short blocks of text inside it drastically reduce token waste without touching openclaw.json at all.

Install Ollama, pull a small tool-friendly model, and point the heartbeat at it:

curl -fsSL https://ollama.com/install.sh | sh
systemctl enable ollama
systemctl start ollama
ollama pull llama3.2:3b
ollama run llama3.2:3b "respond with only the word OK"

Then add a heartbeat block inside agents.defaults in openclaw.json that targets the local Ollama model:

{
  "heartbeat": {
    "enabled": true,
    "interval_minutes": 60,
    "model": "ollama:llama3.2:3b",
    "prompt": "Briefly check any blockers, pending tasks, or reminders."
  }
}

Pair that with a context-pruning policy that drops stale tool outputs after the cache TTL window, and your idle costs effectively go to zero.

🧮 Token audit workflow

Without an audit you’re tuning blind. Run this monthly or whenever the bill jumps:

👉 Useful one-line audit prompt to keep saved: “give me a token-usage and cost audit for this session and recent sessions — per-response breakdown, cost drivers, and recommendations.”

🚫 Common mistakes & pro tips

🏁 Conclusion

OpenClaw isn’t expensive because the model is expensive. It’s expensive because the default configuration loads more than you need, keeps it loaded longer than it should, and doesn’t cache what it repeats. Spend an hour with the memory layout, the slash commands, and the two or three config flags that turn on caching, compaction, and memory flush, and the bill drops by a multiple. Layer Composio for tools and you cut the prompt bloat at its source.

Treat the architecture as something you actively tune, not something that ships ready. The instance you deploy on Monday should look measurably different by Friday — smaller prompts, sharper memory, higher cache hit rate, and an agent that actually remembers what you told it on Tuesday.

Explore More on DevShelf

• →
  OpenClaw — Tool Profile

  Full overview of OpenClaw — skills, channels, pricing, and how the architecture deep-dive here affects your real setup.

• →
  Deploy OpenClaw to the Cloud

  The VPS deployment guide — covers the Docker paths and config files referenced in this architecture article.