Why memory?
LLMs are stateless by default — every call is independent. If you ask "What did I just say?", the model has no idea. Memory lets you inject previous conversation turns into the prompt so the model can refer back to them.
ConversationBufferMemory
The simplest approach — store the full message history and replay it every time. Works well for short conversations.
from langchain.memory import ConversationBufferMemory
from langchain.chains import ConversationChain
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(model="gpt-4o-mini")
memory = ConversationBufferMemory()
conversation = ConversationChain(llm=llm, memory=memory)
conversation.invoke("Hi, my name is Alice.")
conversation.invoke("What's my name?") # → "Your name is Alice."Every message is appended to the buffer. On the next call, the full history is injected into the prompt before the new message. Simple — but the prompt grows unboundedly.
The modern LCEL approach — RunnableWithMessageHistory
The ConversationChain pattern is legacy. With LCEL, you attach history via
RunnableWithMessageHistory — cleaner and more flexible.
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_community.chat_message_histories import ChatMessageHistory
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
store = {} # session_id → ChatMessageHistory
def get_history(session_id: str):
if session_id not in store:
store[session_id] = ChatMessageHistory()
return store[session_id]
prompt = ChatPromptTemplate.from_messages([
("system", "You are a helpful assistant."),
MessagesPlaceholder(variable_name="history"),
("human", "{input}"),
])
chain = prompt | llm | StrOutputParser()
with_history = RunnableWithMessageHistory(
chain,
get_history,
input_messages_key="input",
history_messages_key="history",
)
cfg = {"configurable": {"session_id": "user-123"}}
with_history.invoke({"input": "My name is Alice."}, config=cfg)
with_history.invoke({"input": "What's my name?"}, config=cfg)Each session gets its own history store. You can swap the in-memory store for Redis, DynamoDB, or any persistent backend by implementing the same interface.
ConversationSummaryMemory
Instead of storing every message, summarise the conversation so far using another LLM call. The prompt stays short regardless of conversation length.
from langchain.memory import ConversationSummaryMemory
memory = ConversationSummaryMemory(llm=llm)
conversation = ConversationChain(llm=llm, memory=memory, verbose=True)
conversation.invoke("I'm planning a trip to Japan.")
conversation.invoke("I want to visit Kyoto and Tokyo.")
conversation.invoke("What did we discuss?")Pros
- Prompt length stays constant
- Works for very long conversations
Cons
- Extra LLM call per turn (cost + latency)
- Summarisation loses detail
ConversationTokenBufferMemory
A sliding window that keeps as many recent messages as will fit within a token budget. When the limit is reached, old messages are dropped.
from langchain.memory import ConversationTokenBufferMemory
memory = ConversationTokenBufferMemory(
llm=llm,
max_token_limit=500, # keep the most recent messages that fit
)
conversation = ConversationChain(llm=llm, memory=memory)Good middle ground — predictable prompt size without the extra LLM call that summary memory requires. Older context is lost, but recent turns are always preserved.
Choosing a memory strategy
| Strategy | Best for | Watch out for |
|---|---|---|
| BufferMemory | Short conversations, demos | Prompt grows without bound |
| TokenBufferMemory | Predictable cost, recent context matters | Early context is silently dropped |
| SummaryMemory | Long conversations, big-picture recall | Extra LLM call; detail loss |
| RunnableWithMessageHistory | Production LCEL apps, multi-user | Need a persistent backend for scale |
When not to use memory
Memory is not always the right tool. Before reaching for it, consider these cases where you should skip it entirely.
Single-shot pipelines
If your chain processes one document or one request with no back-and-forth, memory adds overhead with zero benefit. Summarisation chains, classification pipelines, and batch jobs don't need it.
The context window is large enough
Modern models (GPT-4o, Claude 3.5) have context windows of 128k–200k tokens. For many use cases, you can just pass the full conversation without any memory management and it fits fine.
You need exact recall, not fuzzy summarisation
If precise past details matter (contract terms, exact numbers), summary memory will lose them. Use a database and retrieve the exact record instead of relying on the LLM to remember it.
Stateless API endpoints
If your backend handles many users, in-process memory (like ConversationBufferMemory) won't survive server restarts and can't scale across multiple instances. Use a proper persistent store or pass history from the client.
Quick summary
- LLMs are stateless — memory works by prepending conversation history to every prompt
BufferMemoryis simplest;TokenBufferMemorycaps cost;SummaryMemoryhandles long conversations- For LCEL, use
RunnableWithMessageHistorywith a persistent backend in production - Skip memory for single-shot pipelines, large-context models, exact-recall needs, and stateless APIs
LangChain Memory FAQ
What is memory in LangChain?
LLMs are stateless — every call is independent and the model cannot recall earlier turns. Memory in LangChain works by prepending a transcript or summary of the conversation to each prompt, so the model can refer back to what was said before.
What is the difference between buffer, summary, and token buffer memory?
ConversationBufferMemory stores the full message history and replays it, so the prompt grows without bound. ConversationTokenBufferMemory keeps only the most recent messages that fit a token budget. ConversationSummaryMemory uses an extra LLM call to compress the conversation into a short summary, keeping the prompt small for long chats.
How do I add memory to an LCEL chain?
Wrap your chain in RunnableWithMessageHistory, supply a function that returns a ChatMessageHistory per session_id, and set input_messages_key and history_messages_key. Add a MessagesPlaceholder to the prompt so the history is injected. Each session gets its own store, which you can back with Redis or DynamoDB.
Is ConversationChain deprecated in LangChain?
The ConversationChain and classic Memory classes are legacy. The recommended modern approach is to compose your chain with LCEL and attach history using RunnableWithMessageHistory, which is cleaner, more flexible, and works with any persistent backend.
When should I not use memory in LangChain?
Skip memory for single-shot pipelines like summarisation or classification, when the model's context window is large enough to pass the full conversation, when you need exact recall (use a database instead of fuzzy summaries), and for stateless API endpoints where in-process memory cannot scale across instances.