DS DevShelfHub Projects · AI tools
Tutorials / CrewAI / Reference / Classes / BaseLLM
Class llm

BaseLLM: Reference Guide

By DevShelfHub

Subclass-this base for implementing custom LLM integrations.

See the CrewAI API reference index, CrewAI introduction, and core concepts for surrounding context.

What is BaseLLM?

BaseLLM is the extension point when CrewAI's built-in LLM wrapper or LiteLLM routing cannot talk to your gateway — private clusters, custom authentication handshakes, deterministic mock servers in tests, or corporate proxies that rewrite URLs. Subclasses implement call() for synchronous completions and stream() when token-by-token output matters; the framework expects the same message list / tool-call conventions the stock LLM class uses so hooks, guardrails, and token accounting stay wired.

Implementers should assume aggressive concurrency: multiple agents may invoke the same instance across threads or asyncio tasks, so keep mutable connection pools thread-safe and avoid global mutable caches without locking. Surface clear exceptions on transport failures so CrewAI's retry layers can behave predictably, and document whether your backend supports JSON mode or function calling because agents with tools will probe those paths early.

When a thin HTTP proxy in front of an otherwise standard provider is all you need, configuring LLM with extra_headers and a model string is usually simpler than a BaseLLM subclass. Reserve subclassing for behavior that truly cannot be expressed through parameters.

When to Use

In-house models, exotic gateways, deterministic fakes in CI, or custom routing layers that need code-level control.

Use Cases

  • • Custom inference servers
  • • Internal gateways
  • • Test doubles

Key Features

  • ✓ Override-friendly
  • ✓ Plays nicely with hooks/events
  • ✓ Streaming hook point

When NOT to Use

Mainstream hosted providers that LiteLLM already supports — pass a model string to LLM instead.

Notes

Streaming parity

If stream() yields nothing but call() works, UIs that rely on streaming will look hung. Either implement streaming faithfully or document that streaming is unsupported so callers disable stream=True upstream.

Tool and JSON contracts

Agents with tools expect function-calling or JSON-mode behavior depending on configuration. If your gateway strips tool schemas, disable tools for that agent or fix the proxy — otherwise you will see empty tool arguments and confusing retries.

Timeouts and cancellation

Respect kwargs.timeout when present and propagate cancellation where your HTTP client allows it. Hung gateways stall entire crews; aggressive timeouts plus fallbacks are usually cheaper than unbounded waits.

Import

python 
from crewai import BaseLLM

Code Examples

Minimal subclass (sync)

python 
from crewai import BaseLLM

class EchoLLM(BaseLLM):
    def call(self, messages, **kwargs):
        return messages[-1]['content'][::-1]

    def stream(self, messages, **kwargs):
        text = self.call(messages, **kwargs)
        yield text

Delegate to an internal HTTP gateway

python 
import requests
from crewai import BaseLLM

class GatewayLLM(BaseLLM):
    def call(self, messages, **kwargs):
        resp = requests.post(
            'https://llm-gw.corp/v1/chat',
            json={'messages': messages, **kwargs},
            timeout=kwargs.get('timeout', 60),
        )
        resp.raise_for_status()
        return resp.json()['text']

Wire into an Agent

python 
from crewai import Agent

agent = Agent(
    role='Researcher',
    goal='Summarize',
    backstory='You are concise.',
    llm=EchoLLM(),
)

Common Mistakes

❌ Sharing a non-thread-safe HTTP session across agents without locking

✅ Use thread-local sessions or a connection pool designed for concurrent use.

❌ Not handling streaming

✅ Implement stream() or document that streaming is unsupported and disable streaming callers.

BaseLLM FAQ

What is BaseLLM in CrewAI?

Subclass-this base for implementing custom LLM integrations. BaseLLM is the extension point when CrewAI's built-in LLM wrapper or LiteLLM routing cannot talk to your gateway — private clusters, custom authentication handshakes, deterministic mock servers in tests, or corporate proxies that rewrite URLs. Subclasses implement call() for synchronous completions and stream() when token-by-token output matters; the framework expects the same message list / tool-call conventions the stock LLM class uses so hooks, guardrails, and token accoun…

Which package defines the CrewAI class BaseLLM?

DevShelfHub maps BaseLLM to Python module crewai (package path crewai in this reference). Pin your installed crewai version and match imports to the snippet on this page.

When should I use BaseLLM?

In-house models, exotic gateways, deterministic fakes in CI, or custom routing layers that need code-level control.

When should I avoid using BaseLLM?

Mainstream hosted providers that LiteLLM already supports — pass a model string to LLM instead.

How do I import BaseLLM in Python?

from crewai import BaseLLM

Where can I explore more CrewAI API reference pages?

Open the CrewAI API reference index on DevShelfHub to search 58 classes, 30 methods, and 16 decorators, each with runnable examples, parameters, common mistakes, and cross-links.