What is .split_text()?
.split_text() is the lowest-level chunking method on LangChain text splitter classes — it takes a single raw string and returns a List[str] of text chunks according to the splitter's configured chunk_size, chunk_overlap, and separator hierarchy. Unlike .split_documents(), it has no concept of metadata: the result is plain strings with no source attribution.
This makes .split_text() the right starting point when your input text comes from a source that doesn't map cleanly to LangChain's Document model — a custom database query result, a string extracted from a proprietary API, or text you've assembled programmatically. After chunking, you typically wrap each chunk in a Document object manually: [Document(page_content=c, metadata={"source": "my-source"}) for c in chunks], which then makes the chunks compatible with .add_documents(), .from_documents(), and the rest of the LangChain pipeline.
For RecursiveCharacterTextSplitter (the most commonly used splitter), .split_text() tries to split on paragraph breaks ("\n\n") first, then on single newlines, then on spaces, then on individual characters — always keeping chunk size under chunk_size characters. The chunk_overlap parameter specifies how many characters from the end of one chunk are repeated at the start of the next, preventing ideas that span a chunk boundary from being split in half. The length_function parameter (default: Python's len) determines how chunk size is measured — swap it for a token-counting function if your embedding model has a token budget.
Use Cases
- • Split raw text
- • String chunking
- • Simple splitting
- • Non-document text
- • Preprocessing
- • Text preparation
Key Features
- ✓ String input
- ✓ Simple API
- ✓ Fast chunking
- ✓ No metadata
- ✓ Configurable size
- ✓ Overlap support
When NOT to Use
For Document objects—use split_documents().
Notes
Returns List[str] not List[Document] — wrap manually if you need metadata
split_text() discards any notion of source. If you need each chunk to carry a source reference (file path, URL, row ID), you must wrap the strings: docs = [Document(page_content=c, metadata={"source": src}) for c in chunks]. Without this step, vector store methods like add_documents() will receive strings instead of Document objects and raise a type error.
Use tiktoken length_function when targeting embedding models with token limits
OpenAI's text-embedding-3-small and text-embedding-ada-002 both have an 8191-token limit. Character-based chunk_size=1000 is safe for ASCII but can produce chunks exceeding 8191 tokens for dense technical text or non-ASCII content. Pass a tiktoken-based length_function to measure chunks in tokens and avoid silent truncation by the embedding API.
Overlap creates redundancy — that's by design
Chunk overlap means that some text appears in two adjacent chunks. This redundancy is intentional: it ensures that a sentence or concept at the boundary of a chunk is fully represented in at least one chunk for retrieval. Set overlap to 10–20% of chunk_size as a starting point and adjust based on retrieval evaluation metrics for your dataset.
split_text() is the primitive — split_documents() calls it internally
split_documents() calls split_text() on each Document's page_content and then re-wraps the resulting strings in new Documents. If you find yourself calling split_text() and then wrapping in Documents every time, switch to split_documents() and pass real Document objects from your loader instead.
Method Signature
chunks = splitter.split_text(text)Parameters
| Parameter | Type | Required | Purpose |
|---|---|---|---|
| text | str | Yes | Text to split |
Return Value
Type:
List[str]
Description:
Text chunks
Example Output:
['chunk1', 'chunk2', ...]Code Examples
Basic split_text usage
from langchain_text_splitters import RecursiveCharacterTextSplitter
text = 'Long text here...'
splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
chunks = splitter.split_text(text)
print(f"{len(chunks)} chunks")Wrap chunks in Documents for vector store
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_core.documents import Document
splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
raw_text = fetch_from_database(row_id=42)
chunks = splitter.split_text(raw_text)
docs = [
Document(page_content=c, metadata={"row_id": 42, "chunk": i})
for i, c in enumerate(chunks)
]
vector_store.add_documents(docs)Token-aware splitting with tiktoken
import tiktoken
enc = tiktoken.encoding_for_model("text-embedding-3-small")
def tiktoken_len(text):
return len(enc.encode(text))
splitter = RecursiveCharacterTextSplitter(
chunk_size=512, # 512 tokens
chunk_overlap=50,
length_function=tiktoken_len
)
chunks = splitter.split_text(long_text)Common Mistakes
❌ Lose original text structure with split_text()
✅ Use split_documents() to preserve metadata
Related LangChain References
Browse the full LangChain API reference index to explore more classes, methods, and decorators, or start with the LangChain introduction tutorial for end-to-end context on building with .split_text() and the wider framework.
.split_text() FAQ
What does .split_text() do in LangChain?
Split raw text string into chunks. .split_text() is the lowest-level chunking method on LangChain text splitter classes — it takes a single raw string and returns a List[str] of text chunks according to the splitter's configured chunk_size, chunk_overlap, and separator hierarchy. Unlike .split_documents(), it has no concept of metadata: the result is plain strings with no source attribution. This makes .split_text() the right starting point when your input text comes from a source that doesn't map cleanly to …
Which LangChain classes support .split_text()?
.split_text() is available on Text splitters. Pin your installed LangChain version and verify the method exists in that release before deploying.
When should I use .split_text()?
Use .split_text() when your LangChain chains, agents, or pipelines need the behavior described in this guide.
What does .split_text() return?
.split_text() returns a List[str]. Text chunks
Does .split_text() have an async equivalent?
.split_text() does not have a documented async variant. Avoid .split_text() For Document objects—use split_documents().
Where can I explore more LangChain API reference pages?
Open the LangChain API reference index on DevShelfHub to browse classes, methods, and decorators, each with runnable examples, parameters, common mistakes, and cross-links.