What is a Document Loader?
A Document Loader reads raw data from a source and returns a list of Document objects. Each Document has two fields: page_content (the text) and metadata (a dict with source info like file path, page number, or URL).
Data Source
PDF, URL, CSV, DB, S3…
Document Loader
.load() / .lazy_load()
Output
list[Document]
All loaders implement the same interface: .load() returns a list of Documents, and .lazy_load() returns a generator for memory-efficient processing of large sources.
from langchain_core.documents import Document
# A Document is just two fields
doc = Document(
page_content="LangChain makes building LLM apps easy.",
metadata={"source": "intro.txt", "page": 1},
)
print(doc.page_content) # LangChain makes building LLM apps easy.
print(doc.metadata) # {'source': 'intro.txt', 'page': 1}TextLoader
The simplest loader. Reads a plain text file and returns a single Document with the full file contents.
from langchain_community.document_loaders import TextLoader
loader = TextLoader("notes.txt", encoding="utf-8")
docs = loader.load()
print(len(docs)) # 1
print(docs[0].page_content[:80]) # first 80 chars of the file
print(docs[0].metadata) # {'source': 'notes.txt'}Tip: Always specify encoding explicitly. On Windows systems the default may not be UTF-8, causing errors with non-ASCII characters.
PyPDFLoader
Loads a PDF and returns one Document per page. Install pypdf first.
# pip3 install pypdf
from langchain_community.document_loaders import PyPDFLoader
loader = PyPDFLoader("research_paper.pdf")
pages = loader.load()
print(len(pages)) # one Document per page
print(pages[0].metadata) # {'source': 'research_paper.pdf', 'page': 0}
print(pages[0].page_content[:200])
# Lazy load for large PDFs (memory efficient)
for page in loader.lazy_load():
process(page)PyPDFLoader
Pure Python · No system deps · Good for most PDFs
PyMuPDFLoader
Faster · Better table extraction · Requires pymupdf
WebBaseLoader
Fetches one or more URLs and extracts their text content using BeautifulSoup. Good for scraping documentation, blog posts, or any public web page.
# pip3 install beautifulsoup4
from langchain_community.document_loaders import WebBaseLoader
# Single URL
loader = WebBaseLoader("https://python.langchain.com/docs/introduction/")
docs = loader.load()
print(docs[0].metadata["source"]) # the URL
# Multiple URLs — loaded concurrently
loader = WebBaseLoader([
"https://python.langchain.com/docs/introduction/",
"https://python.langchain.com/docs/concepts/",
])
docs = loader.load()
print(len(docs)) # 2
# Filter specific HTML tags with bs_kwargs
import bs4
loader = WebBaseLoader(
"https://example.com/blog",
bs_kwargs={"parse_only": bs4.SoupStrainer(class_="post-content")},
)
docs = loader.load()CSVLoader
Loads a CSV file and returns one Document per row. Each document's content is a key-value string of column names and values.
from langchain_community.document_loaders import CSVLoader
loader = CSVLoader("products.csv")
docs = loader.load()
# Each row becomes one Document
# page_content: "name: Widget\nprice: 9.99\ncategory: Tools"
print(docs[0].page_content)
# Use a specific column as the content
loader = CSVLoader(
"products.csv",
source_column="description", # use this column as page_content
)
docs = loader.load()DirectoryLoader
Load all files in a directory, optionally filtered by glob pattern. Automatically uses the right sub-loader for each file type.
from langchain_community.document_loaders import DirectoryLoader, TextLoader
# Load all .txt files in a directory (recursive)
loader = DirectoryLoader(
"docs/",
glob="**/*.txt",
loader_cls=TextLoader,
loader_kwargs={"encoding": "utf-8"},
show_progress=True, # tqdm progress bar
use_multithreading=True, # parallel loading
)
docs = loader.load()
print(len(docs)) # one Document per fileCommon patterns: **/*.pdf for PDFs, **/*.md for Markdown, **/*.py for source code. Combine use_multithreading=True with show_progress=True for fast bulk ingestion.
Loader Quick Reference
| Loader | Source | Docs per call |
|---|---|---|
| TextLoader | .txt file | 1 |
| PyPDFLoader | PDF file | 1 per page |
| WebBaseLoader | HTTP URL(s) | 1 per URL |
| CSVLoader | CSV file | 1 per row |
| DirectoryLoader | Folder of files | varies |
| UnstructuredLoader | Any file type | varies |
| NotionDirectoryLoader | Notion export | 1 per page |
| S3FileLoader | AWS S3 object | 1 |
Building a Custom Loader
When no built-in loader fits your source, subclass BaseLoader and implement lazy_load(). The load() method is implemented automatically.
from typing import Iterator
from langchain_core.document_loaders import BaseLoader
from langchain_core.documents import Document
class DatabaseLoader(BaseLoader):
def __init__(self, connection_string: str, query: str):
self.connection_string = connection_string
self.query = query
def lazy_load(self) -> Iterator[Document]:
import sqlite3
conn = sqlite3.connect(self.connection_string)
cursor = conn.execute(self.query)
columns = [desc[0] for desc in cursor.description]
for row in cursor:
content = "\n".join(f"{k}: {v}" for k, v in zip(columns, row))
yield Document(
page_content=content,
metadata={"source": self.connection_string},
)
conn.close()
# Use it like any built-in loader
loader = DatabaseLoader("mydb.sqlite", "SELECT title, body FROM articles")
docs = loader.load()Next steps
Once your data is loaded into Document objects, the usual next move is to split it into chunks with Text Splitters and then wire everything together in a RAG Pipeline.
LangChain Document Loaders FAQ
How do I load a PDF in LangChain?
Install pypdf, then use PyPDFLoader from langchain_community.document_loaders. Pass the file path to PyPDFLoader('research_paper.pdf') and call .load(), which returns one Document per page with the page number stored in metadata. For large PDFs, iterate over .lazy_load() instead so pages stream one at a time and you do not hold the whole document in memory.
How do I load a webpage into LangChain?
Use WebBaseLoader, which fetches one or more URLs and extracts their text with BeautifulSoup. Pass a single URL string or a list of URLs (loaded concurrently), then call .load() to get a Document per page. You can narrow the extracted content by passing bs_kwargs with a SoupStrainer so only the relevant HTML tags are parsed.
What file types can LangChain document loaders handle?
LangChain ships loaders for text files, PDFs, web pages, CSVs, directories, Notion exports, S3 objects, and many more. TextLoader, PyPDFLoader, WebBaseLoader, CSVLoader, and DirectoryLoader cover the most common cases, while UnstructuredLoader handles almost any file type. When nothing fits, you subclass BaseLoader to support your own source.
How do I load all files in a directory with LangChain?
Use DirectoryLoader with a glob pattern such as '**/*.txt' or '**/*.pdf' to match files recursively. Pass a loader_cls like TextLoader for the matched files, and enable show_progress=True for a progress bar and use_multithreading=True to load files in parallel for fast bulk ingestion.
What is a Document object in LangChain?
A Document is LangChain's standard container for a piece of loaded data. It has two fields: page_content, the text itself, and metadata, a dictionary of source information such as the file path, page number, or URL. Every loader returns a list of Document objects so the rest of your pipeline can split, embed, and store them the same way.
How do I handle very large files with LangChain loaders?
Use .lazy_load() instead of .load(). It returns a generator that yields one Document at a time, so you can process or stream pages without loading the entire source into memory. Most built-in loaders implement lazy_load(), and custom loaders get it automatically when you subclass BaseLoader and implement lazy_load().