Headroom: Open-Source AI Context Compression — 60–95% Fewer Tokens, Same Answers

Introduction

There is a number that has quietly become one of the most important line items in every serious AI engineering budget: the token count. Every call to GPT-4o, Claude Opus, or Gemini Ultra is billed by the token. A single token is roughly four characters of English text. That sounds benign until you run your first agentic workflow at scale and discover that a single SRE incident investigation can consume 65,000 input tokens in one session — the equivalent of the first 50 pages of a novel — just in log data, stack traces, and tool outputs.

The dominant response has been to wait for providers to offer longer, cheaper context windows. That strategy has worked partially. But bigger context windows have not reduced costs — they have enabled more tokens to be consumed at the same per-token price. A different approach is now emerging: compress what you send before it reaches the model. Process raw, verbose tool outputs locally, strip out redundancy, and ship only the signal to the LLM.

This is the philosophy behind Headroom — an open-source context compression layer with 37K+ GitHub stars, built for AI engineers who are serious about cost control without sacrificing answer quality.

Key takeaways

What is Headroom?

Headroom ( github.com/chopratejas/headroom, Apache 2.0) is an open-source context compression layer for AI agents and LLM-powered applications. Its core promise: 60–95% fewer tokens, same answers.

It compresses everything an AI agent reads — tool outputs, log files, RAG retrieval chunks, source code, conversation history, and structured JSON — before that content reaches the LLM provider. The LLM receives a semantically equivalent but dramatically smaller representation, processes it correctly, and answers as accurately as it would have with the full payload.

Headroom is not a model. It is not a provider. It is a middleware layer — a local, privacy-preserving processing pipeline that sits between your application and the cloud LLM. It ships in four deployment modes:

Why AI agents generate massive context

A traditional chatbot sends a question and gets an answer. The context is small. AI agents operate on an entirely different paradigm. A coding agent in a single session might:

• List all files in a repository (returns 500 filenames)
• Read five relevant files in full (returns 2,000–10,000 lines each)
• Run a search across the codebase (returns 100 matched snippets with context)
• Execute tests (returns thousands of lines of output, stack traces, and timing data)
• Query a GitHub issue tracker (returns 50 issue descriptions, comments, and metadata)

Each step appends a tool output to the conversation context, which is sent in full with the next step. By step 6, the agent may be carrying 150,000 tokens of accumulated context — most of it raw, unprocessed noise.

How Headroom works internally

At the highest level, Headroom applies a pipeline of transforms to the messages array before it leaves your system. The pipeline lifecycle has well-defined stages:

 Your Agent / App
 (Claude Code, Cursor, Codex, LangChain, Agno, your own code…)
       │   prompts · tool outputs · logs · RAG results · files
       ▼
   ┌────────────────────────────────────────────────────┐
   │  Headroom   (runs locally — your data stays here)  │
   │  ────────────────────────────────────────────────  │
   │  CacheAligner  →  ContentRouter  →  CCR            │
   │                    ├─ SmartCrusher   (JSON)        │
   │                    ├─ CodeCompressor (AST)         │
   │                    └─ Kompress-base  (text, HF)    │
   │                                                    │
   │  Cross-agent memory  ·  headroom learn  ·  MCP     │
   └────────────────────────────────────────────────────┘
       │   compressed prompt  +  retrieval tool
       ▼
 LLM provider  (Anthropic · OpenAI · Bedrock · …)

Each stage is extensible. You can attach pipeline extensions that observe or mutate content at any lifecycle event — a design that keeps core orchestration clean while supporting advanced customization.

Architecture deep dive

Headroom's architecture reflects three core design decisions that distinguish it from simpler approaches.

Implementation languages reflect the performance requirements: Python (78.7%) for orchestration, CLi, proxy and ML inference; Rust (16.8%) for performance-critical components (AST parser, CCR storage backend) exposed via PyO3 bindings; TypeScript (2.4%) for the Node.js SDK.

The six compression engines

[COMPRESSED — 94% reduction. Full original retrievable via headroom_retrieve("key:abc123") if needed]
Error spike detected in auth-service logs: 47 FATAL events, 12 unique error signatures.
Top error: NullPointerException in UserAuthHandler:127 (39 occurrences, 14:32–14:41 UTC)

Output token reduction

Everything above shrinks the tokens you send. But you also pay for every token the model writes back — and on Opus-class models, output costs 5× the input rate. A significant fraction of that output is waste: preambles, re-printed code, and deep "thinking" on routine steps.

Headroom trims this through two mechanisms:

export HEADROOM_OUTPUT_SHAPER=1
headroom proxy --port 8787

# See estimated savings
headroom output-savings
# Reduction: 31.7%  (95% CI 27.7% … 35.7%)   [estimated]

Savings are reported as estimates with confidence intervals since you cannot know what the model would have written without steering. For a measured number, use HEADROOM_OUTPUT_HOLDOUT=0.1 to leave 10% of conversations unshaped as a control group.

Headroom Learn

headroom learn mines your past agent sessions to automatically improve future ones. It scans local session history for failed or corrected steps, identifies the pattern (missing information, wrong assumption, misread tool output), and writes a correction to your agent’s instruction file: CLAUDE.md, AGENTS.md, or GEMINI.md.

headroom learn               # Preview what it found (dry run)
headroom learn --apply       # Write corrections to instruction files
headroom learn --verbosity   # Learn preferred output length from session history
headroom learn --verbosity --apply  # Save verbosity preference; proxy uses it immediately

The --verbosity flag reads your past sessions and infers how verbose you prefer answers by looking at how quickly you interrupted long replies or moved on before reading them. It configures verbosity steering accordingly — personalizing terseness without requiring you to articulate a preference.

Cross-agent memory

Modern development workflows often span multiple AI agents. Claude Code for architecture, Cursor for inline edits, Codex for automation. Each agent currently runs in isolation, unable to benefit from what the others learned. Headroom’s cross-agent memory changes this with a shared context store that persists facts and decisions across agent sessions, with automatic deduplication.

from headroom import SharedContext

ctx = SharedContext()

# In a Claude Code session
ctx.put("db_connection_format", "postgres://user:pass@host:5432/db?sslmode=require")

# Automatically available in any Cursor or Codex session (same project)
conn_format = ctx.get("db_connection_format")

The shared store includes agent provenance metadata — knowing that a fact came from Claude Code vs. Codex allows downstream agents to weight it appropriately.

MCP integration

The Model Context Protocol (MCP) is becoming the standard integration interface for AI agent ecosystems. Headroom ships a first-class MCP server that exposes its capabilities as native MCP tools.

headroom mcp install

This registers three tools for any MCP-compatible client (Claude Desktop, Cursor, custom MCP hosts):

Supported AI tools and frameworks

Agent compatibility

Framework SDK integrations

Installation and setup

Requires Python 3.10+. Optional: Apple Silicon GPU acceleration via [pytorch-mps].

# Full installation with all features
pip install "headroom-ai[all]"

# Granular extras
pip install "headroom-ai[proxy]"       # Proxy server
pip install "headroom-ai[mcp]"         # MCP server
pip install "headroom-ai[ml]"          # Kompress-base ML model
pip install "headroom-ai[code]"        # CodeCompressor (Rust AST parser)
pip install "headroom-ai[memory]"      # Cross-agent memory
pip install "headroom-ai[langchain]"   # LangChain integration
pip install "headroom-ai[pytorch-mps]" # Apple GPU acceleration

# TypeScript / Node.js
npm install headroom-ai

Quick start tutorial

headroom proxy --port 8787

# Point your agent at the proxy
export ANTHROPIC_BASE_URL=http://localhost:8787
claude  # or any other AI agent

headroom perf  # View real-time savings

headroom wrap claude    # Wraps Claude Code with compression + memory
headroom wrap codex     # Wraps OpenAI Codex
headroom wrap cursor    # Prints Cursor proxy config
headroom wrap aider     # Wraps Aider with compression

from headroom import compress
import anthropic

client = anthropic.Anthropic()
messages = [{"role": "user", "content": "Analyze these logs: " + very_long_log_string}]

result = compress(messages, model="claude-opus-4-8")
print(f"Tokens saved: {result.tokens_saved}")          # e.g. 55,200
print(f"Compression ratio: {result.compression_ratio}") # e.g. 0.08

response = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=2048,
    messages=result.messages
)

from langchain_anthropic import ChatAnthropic
from headroom.integrations.langchain import HeadroomChatModel

base_llm = ChatAnthropic(model="claude-opus-4-8")
llm = HeadroomChatModel(base_llm)

response = llm.invoke("Summarize these 100 search results...")

Real benchmark analysis

Headroom publishes benchmarks from real agent workloads — not synthetic datasets. These four workloads are representative of what engineering teams run daily.

Reproduce with: python -m headroom.evals suite --tier 1

The 47% ceiling on codebase exploration illustrates an honest limit: Headroom cannot compress genuinely novel, high-entropy content aggressively. When an agent is exploring a new codebase, it needs broad structural understanding — there is less redundancy to eliminate.

Accuracy preservation benchmarks

Token reduction is meaningless if it degrades answer quality. Headroom’s accuracy benchmarks use 100-sample runs on standard evaluation datasets:

The TruthfulQA improvement (+0.030) is counterintuitive: compression improves factual accuracy. The likely explanation: Headroom removes verbose but misleading context that was pushing the model toward plausible-sounding but incorrect answers. The BFCL result (97% tool-calling accuracy at 32% compression) is critical for agent builders — tool-calling accuracy is the most operationally important metric in agentic workloads.

Headroom vs. the competition

OpenAI Compaction is automatic and convenient but provider-locked, one-directional, and covers only conversation history — not tool outputs. RTK is effective for pure RAG pipelines but does not handle code, logs, or JSON tool outputs. lean-ctx is a truncation tool — it cannot distinguish a FATAL log line from an INFO line. Hosted services introduce data egress, per-call fees on top of LLM costs, and additional network latency.

Pros and cons

Cost-saving calculations

Concrete monthly cost model for a 10-person engineering team on Claude Opus 4.8-class models:

For teams where log analysis and code search dominate (90%+ compression achievable), savings can reach 70–80% of total LLM spend. Output token reduction alone on a $1,000/mo output bill saves ~$300/mo from a single configuration flag.

Frequently asked questions

Final verdict