Advanced technique 1: Chain-of-thought in system prompts
Chain-of-thought makes the AI reason step-by-step. It produces better answers and shows its work.
When analyzing a problem:
1. State your assumptions
2. Break the problem into parts
3. Solve each part
4. Verify the answer makes sense
5. State your final answer
Use phrases like: "Let me think through this step by step..."More tokens, better reasoning. Use when accuracy matters more than speed.
Advanced technique 2: Meta-prompting
Meta-prompting teaches the AI to improve its own prompts. Use it to iterate on system prompts automatically.
When a user asks you to do a task:
1. First, generate an ideal system prompt for that task
2. Explain why that prompt is effective
3. Then, perform the task using that prompt
4. Reflect: did it work? How could the prompt improve?Advanced technique 3: Dynamic system prompts
Instead of a static prompt, modify it based on context. Different users, different prompts.
Example:
- If user is a beginner → simpler language, more examples
- If user is advanced → technical depth, skip basics
- If previous response was misunderstood → adjust tone/clarity
- If user asked follow-up question → add previous context
This requires tracking user context and modifying the prompt at runtime.
Advanced technique 4: Prompt compression
Make prompts shorter without losing quality. Reduce token count and cost.
Compression strategies:
- Remove redundant examples
- Combine related constraints
- Use shorthand: "Return JSON" instead of "Return results in JSON format with..."
- Trust the model: fewer examples sometimes work as well as many
- Use prompt caching: keep system prompt static, only send new content
When to use which advanced technique
All four techniques have a real cost — extra tokens, extra latency, or extra engineering. Pick the one that matches the bottleneck you actually have.
| Technique | Best for | Cost | Skip when |
|---|---|---|---|
| Chain-of-thought | Math, logic, debugging, multi-step analysis | 2–5x output tokens, slower | Simple lookups, classification, summaries |
| Meta-prompting | Discovering better prompts during development | Engineering time, not runtime | Hot production paths — bake the result in |
| Dynamic prompts | Multi-tenant apps, personalised assistants | Template engine + breaks prompt caching | One prompt fits all your users |
| Compression + caching | High-traffic apps, long system prompts | One-off engineering pass | Low-volume prototypes |
The most common production combo is compression + prompt caching for the stable system prompt, plus dynamic per-request suffixes for personalisation. CoT is reserved for the few endpoints where reasoning quality is worth the token bill.
Putting it together: Python code samples
Two short snippets using the Anthropic Python SDK that show CoT and dynamic prompting in real code. Both patterns transfer directly to OpenAI and Gemini with minor naming changes.
Chain-of-thought in the system prompt
from anthropic import Anthropic
client = Anthropic()
system = """You are a careful reasoning assistant.
When given a problem:
1. State your assumptions explicitly.
2. Break the problem into 2-4 sub-problems.
3. Solve each sub-problem and show your work.
4. Combine the parts and sanity-check the answer.
5. State the final answer on its own line, prefixed with "ANSWER:".
Never skip the reasoning steps, even on questions that look easy."""
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=2048,
system=system,
messages=[{"role": "user", "content": "A train leaves Berlin at 09:00 ..."}],
)Dynamic prompt with cacheable prefix
from anthropic import Anthropic
client = Anthropic()
STABLE_SYSTEM = """You are Acme's support assistant. Cite docs by URL.
Never share roadmap or pricing details. Always end with a follow-up question."""
def build_system(user_tier: str, locale: str) -> list[dict]:
return [
{"type": "text", "text": STABLE_SYSTEM, "cache_control": {"type": "ephemeral"}},
{"type": "text", "text": f"User tier: {user_tier}. Locale: {locale}."},
]
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=1024,
system=build_system(user_tier="enterprise", locale="de-DE"),
messages=[{"role": "user", "content": "How do I rotate my API key?"}],
)The cacheable prefix pattern is the single highest-leverage optimisation for any production app: the long stable instructions get cached, the short per-request suffix doesn't, and you pay full-price tokens only for the dynamic part. For LangChain-native patterns of the same idea, see LangChain prompt templates.
Your system prompt journey: 10 key insights
What's next? Recommended tutorials
Prompt Engineering
Go deeper into user prompts (not system prompts). Learn techniques like few-shot learning, prompt templates, and optimization.
AI Agents
Use system prompts to build agents that can take actions, use tools, and reason over time. System prompts are the agent's "brain."
Fine-tuning
When system prompts aren't enough, fine-tune a model on your specific use case. More powerful but more expensive.
RAG Pipeline
Use system prompts to guide how a model uses retrieved documents. Combines prompting with knowledge retrieval.
Resources for learning more
Anthropic Prompt Library: Real-world system prompts and examples from Anthropic
OpenAI Cookbook: Practical examples of prompt engineering techniques
Promptfoo: Framework for testing and evaluating system prompts
Academic papers: "Chain-of-Thought Prompting" and "In-Context Learning" for deeper understanding
Final thoughts
You now know how to write system prompts that work. Start with the basics (clear identity, instructions, constraints), test with real inputs, iterate, and scale. The best prompts come from experimentation and refinement.
System prompts are a superpower. Use them to build better AI systems. Continue your learning with prompt engineering techniques, explore AI agents that use system prompts as their core logic, or learn about RAG pipelines that combine prompting with knowledge retrieval.
Advanced System Prompts FAQ
What is chain-of-thought in a system prompt?
Chain-of-thought instructs the AI to reason step by step before answering. You add instructions like 'break the problem into parts, solve each part, then verify' to the system prompt. It produces better answers on complex tasks at the cost of more tokens.
What is meta-prompting?
Meta-prompting teaches the AI to generate and improve its own prompts. The system prompt instructs the model to first draft an ideal prompt for a task, explain why it works, perform the task, and then reflect on how the prompt could be improved.
How do dynamic system prompts work?
Dynamic system prompts are built at runtime by injecting variables like user context, retrieved documents, or conversation state. Instead of a static string, you use a template that assembles the prompt based on the current request.
What is prompt compression and when should I use it?
Prompt compression reduces token count without losing meaning — by removing filler words, using abbreviations, or restructuring instructions. Use it when your system prompt approaches the context window limit or when you need to reduce API costs.
What should I learn after mastering system prompts?
Next steps include prompt engineering for user-side prompts, building AI agents that use system prompts as their core logic, fine-tuning models for tasks where prompting alone is insufficient, and RAG pipelines that combine system prompts with retrieved knowledge.
When should I use chain-of-thought vs a simple prompt?
Use chain-of-thought for multi-step reasoning tasks like math, logic, debugging, and analysis where accuracy matters more than latency or cost. Skip it for simple lookups, summaries, classification, and any path where extra output tokens are wasted. CoT typically adds 2x to 5x output tokens, so the cost is real.
Does prompt caching change how I write a system prompt?
Yes. Both Anthropic and OpenAI charge less and respond faster for cached prefix tokens, so keep the long, stable part of your system prompt at the top and put any per-request variation at the end. A 4,000-token cached system prompt costs roughly one-tenth of an uncached one on subsequent calls within the cache TTL.