What is .stream()?
.stream() returns an iterator that yields output chunks as a Runnable produces them, rather than waiting for the full result. For chat models, each chunk is an AIMessageChunk whose .content field contains the latest token text. For LCEL chains, the iterator yields the output type of the final component in the pipeline, accumulating through each step. For LangGraph graphs, it yields state delta dicts at each node boundary.
Consuming a .stream() iterator is straightforward: iterate with a for-loop, write each chunk to the output immediately. The key footgun is treating stream() like invoke() — if you assign the return value to a variable without iterating it, you get a generator object, not the full text. The iterator is also consumed once: once you've iterated through it, it's exhausted. To accumulate the full response while also streaming, concatenate chunks as you go: full = ""; for chunk in model.stream(msg): full += chunk.content; print(chunk.content, end="", flush=True).
For production web endpoints, the async sibling .astream() is the right choice — it returns an async generator you consume with async for, which keeps the event loop free between I/O waits. Server-Sent Events (SSE) patterns with FastAPI use StreamingResponse wrapping an async generator that yields f"data: {chunk.content}\n\n". Not all providers support true token streaming — some providers batch the full response and "fake stream" by yielding a single chunk. Check the provider's streaming support in the LangChain documentation.
Use Cases
- • Chat UI
- • Token-by-token display
- • Long operations
- • Progressive rendering
- • UX improvement
- • Cost monitoring
Key Features
- ✓ Token streaming
- ✓ Real-time updates
- ✓ Iterator-based
- ✓ Low latency
- ✓ Works with chains
- ✓ Configurable
When NOT to Use
For batch processing—use batch(). For simple results—use invoke().
Notes
Assign to a variable and iterate — do not treat the return value as a string
stream() returns a generator object, not the final text. Writing result = model.stream(msg) stores the generator but does not start streaming. You must iterate it: for chunk in model.stream(msg): ..., or convert to a string with "".join(c.content for c in model.stream(msg)).
Use astream() in async contexts to keep the event loop free
Calling .stream() inside an async function works but blocks the event loop on each yield until the next token arrives. In FastAPI or asyncio applications, use async for chunk in model.astream(msg) instead — this suspends the coroutine between tokens, allowing other requests to be handled concurrently.
Not all providers support true token streaming
Some model providers (certain Anthropic SDK versions, some Azure deployments) batch the full response and emit it as a single chunk regardless of streaming settings. The .stream() call still returns an iterator, but it may yield only one large chunk. Verify streaming behavior against your specific provider and model version.
Use astream_events() for chunk metadata in complex chains
In LCEL chains with multiple steps, .stream() yields the output of the final component. If you need to know which step produced each chunk (for progress indicators or partial tool-call results), use .astream_events() instead — it emits structured event dicts with event type, run_id, name, and data fields on every emission.
Method Signature
for chunk in runnable.stream(input):
print(chunk)Parameters
| Parameter | Type | Required | Purpose |
|---|---|---|---|
| input | Any | Yes | Input data |
Return Value
Type:
Iterator[Any]
Description:
Iterator of chunks
Example Output:
for chunk in llm.stream('hello'): print(chunk.content)Code Examples
Basic token streaming
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage
model = ChatOpenAI(model='gpt-4o-mini')
for chunk in model.stream([HumanMessage(content='Write a haiku')]):
print(chunk.content, end='', flush=True)Accumulate full response while streaming
full_response = ''
for chunk in model.stream([HumanMessage(content='Explain RAG')]):
full_response += chunk.content
print(chunk.content, end='', flush=True)
print() # newline after streaming ends
print(f'Total chars: {len(full_response)}')FastAPI SSE endpoint with astream()
# FastAPI SSE endpoint
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
app = FastAPI()
@app.get('/chat')
async def chat(q: str):
async def generate():
async for chunk in model.astream([HumanMessage(content=q)]):
yield f'data: {chunk.content}\n\n'
return StreamingResponse(generate(), media_type='text/event-stream')Common Mistakes
❌ result = model.stream(msg) # Result is iterator
✅ for chunk in model.stream(msg): print(chunk)
Related LangChain References
Browse the full LangChain API reference index to explore more classes, methods, and decorators, or start with the LangChain introduction tutorial for end-to-end context on building with .stream() and the wider framework.
.stream() FAQ
What does .stream() do in LangChain?
Stream output tokens as they arrive. .stream() returns an iterator that yields output chunks as a Runnable produces them, rather than waiting for the full result. For chat models, each chunk is an AIMessageChunk whose .content field contains the latest token text. For LCEL chains, the iterator yields the output type of the final component in the pipeline, accumulating through each step. For LangGraph graphs, it yields state delta dicts at each node boundary. Consuming a .stream() iterator is straightforward: it…
Which LangChain classes support .stream()?
.stream() is available on All Runnables. Pin your installed LangChain version and verify the method exists in that release before deploying.
When should I use .stream()?
Use .stream() when your LangChain chains, agents, or pipelines need the behavior described in this guide.
What does .stream() return?
.stream() returns a Iterator[Any]. Iterator of chunks
Does .stream() have an async equivalent?
Yes — use astream() for async contexts such as FastAPI handlers or asyncio-based pipelines. It is the non-blocking counterpart to .stream().
Where can I explore more LangChain API reference pages?
Open the LangChain API reference index on DevShelfHub to browse classes, methods, and decorators, each with runnable examples, parameters, common mistakes, and cross-links.