The Default LLM
Out of the box, CrewAI uses gpt-4o-mini via OpenAI. To override globally, set OPENAI_MODEL_NAME:
.env
OPENAI_API_KEY=sk-...
OPENAI_MODEL_NAME=gpt-4oPer-Agent LLMs
Different roles benefit from different models. A classifier doesn't need GPT-4 — but a planner might.
Mixed LLMs
from langchain_openai import ChatOpenAI
from crewai import Agent
cheap = ChatOpenAI(model="gpt-4o-mini", temperature=0)
smart = ChatOpenAI(model="gpt-4o", temperature=0.2)
triager = Agent(role="Ticket Classifier", goal="Tag tickets", backstory="...", llm=cheap)
solver = Agent(role="Senior Engineer", goal="Fix bugs", backstory="...", llm=smart)Anthropic (Claude)
Claude integration
from langchain_anthropic import ChatAnthropic
from crewai import Agent
claude = ChatAnthropic(model="claude-sonnet-4-6", temperature=0.2)
writer = Agent(
role="Long-Form Writer",
goal="Produce 2000-word essays with strong narrative",
backstory="...",
llm=claude,
)Set ANTHROPIC_API_KEY in your .env.
Ollama (Local Models)
Run a crew entirely offline with a local model. Great for privacy-sensitive workloads.
Local Llama via Ollama
from langchain_community.chat_models import ChatOllama
from crewai import Agent
local_llm = ChatOllama(
model="llama3.1:8b",
base_url="http://localhost:11434",
temperature=0,
)
private_analyst = Agent(
role="Compliance Analyst",
goal="Review documents without sending data to third parties",
backstory="You handle regulated data and never use cloud LLMs.",
llm=local_llm,
)Groq (Fast Inference)
For latency-sensitive crews, Groq's hardware delivers very high tokens-per-second on open models.
Groq backend
from langchain_groq import ChatGroq
fast = ChatGroq(model="llama-3.1-70b-versatile", temperature=0)
reactive_agent = Agent(
role="Real-time Triager",
goal="Tag and route inbound messages within 200ms",
backstory="...",
llm=fast,
)Cost-Aware Routing
A common production pattern: a cheap classifier decides if the heavy model is needed.
Two-tier routing
triage = Agent(role="Router", goal="Decide if this needs the senior model", llm=cheap, backstory="...")
junior = Agent(role="Junior Solver", goal="Handle simple cases", llm=cheap, backstory="...")
senior = Agent(role="Senior Solver", goal="Handle complex cases", llm=smart, backstory="...")
# In your task descriptions, instruct the router to mark
# "easy" or "hard"; downstream tasks branch on that label.Notes
Per-agent routing beats one-size-fits-all
Frontier models for planning and small models for formatting is a common split. Document which agent uses which model so on-call can spot quota issues quickly.
Keep API keys out of trace payloads
Some clients echo configuration into logs when errors occur. Scrub headers and env dumps before shipping traces to vendors.
Fallback chains need health checks
Automatically switching providers can hide outages until bills spike. Emit metrics whenever fallbacks activate and page if the rate jumps.
Temperature and tool discipline interact
Higher creativity settings can increase spurious tool calls. Pair temperature tuning with guardrails when agents touch production systems.
CrewAI LLM integration FAQ
How do I configure multiple LLM providers in CrewAI?
Set provider credentials in your environment, then assign models per agent or task so simple steps use cheaper endpoints and complex steps use frontier models.
Can CrewAI run local models with Ollama?
Yes, when your hardware supports the model size. Local inference helps development loops and privacy-sensitive workloads but still needs capacity planning.
How do I reduce CrewAI LLM spend?
Cap iterations, shorten prompts, cache retrieval results, batch tool calls, and downgrade models for structured extraction tasks.
What happens when a provider rate limits CrewAI?
Calls fail or retry depending on your client settings. Add exponential backoff and queueing at the service boundary for production traffic.
Should every agent use the same model?
Not necessarily. Match model capability to task risk: use smaller models for formatting and larger models for reasoning-heavy steps.
See also: DevShelfHub's CrewAI tool review for a product-level comparison, pricing notes, and links back into this tutorial series.