Vector Stores in LangChain: FAISS and Chroma

What is a Vector Store?

A vector store is a database optimised for storing embeddings and running fast similarity searches. Instead of matching exact keywords, it finds documents whose meaning is closest to the query.

Regular Database

Exact keyword match. Query "LangChain" only returns rows with that exact word. Semantic variants are missed.

Vector Store

Semantic match. Query "AI orchestration framework" retrieves documents about LangChain even without the exact words.

In a RAG pipeline: embed documents → store in vector DB → embed user query → retrieve nearest neighbours → feed to LLM.

Provider Overview

Store	Package	Hosting	Best For
Chroma	langchain-chroma	Local	Dev / prototyping
FAISS	langchain-community	In-process	Offline / fast
Pinecone	langchain-pinecone	Cloud	Production scale
Qdrant	langchain-qdrant	Both	High performance
AstraDBVectorStore	langchain-astradb	Cloud	Serverless / hybrid
ElasticsearchStore	langchain-elasticsearch	Both	Enterprise
MongoDBAtlasVectorSearch	langchain-mongodb	Cloud	Existing Atlas users
PGVector (pgvector)	langchain-postgres	Self-hosted	SQL-first teams
Weaviate	langchain-weaviate	Both	Multimodal data
InMemoryVectorStore	langchain-core	In-process	Unit testing

Core Methods (Universal API)

Every vector store in LangChain shares this interface — swap providers without rewriting search logic.

python

from langchain_chroma import Chroma
from langchain_openai import OpenAIEmbeddings
from langchain_core.documents import Document

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")

# 1. Create from documents
docs = [Document(page_content="LangChain is a framework", metadata={"source": "docs"})]
store = Chroma.from_documents(docs, embeddings, persist_directory=".chroma_db")

# 2. Create from raw text
store = Chroma.from_texts(["text1", "text2"], embeddings, metadatas=[{"id": 1}, {"id": 2}])

# 3. Add to existing store
store.add_documents([Document(page_content="New document")])
store.add_texts(["Another text"])

# 4. Similarity search
results = store.similarity_search("what is langchain?", k=4)
for doc in results:
    print(doc.page_content, doc.metadata)

# 5. Search with relevance score (0.0–1.0)
results = store.similarity_search_with_score("langchain query", k=4)
for doc, score in results:
    print(f"Score: {score:.3f} — {doc.page_content[:80]}")

# 6. Delete documents
store.delete(ids=["doc-id-1", "doc-id-2"])

MMR Search — Diverse Results

Max Marginal Relevance (MMR) balances relevance with diversity. It avoids returning 4 near-identical chunks about the same sentence.

python

# fetch_k candidates, then pick k diverse ones
results = store.max_marginal_relevance_search(
    query="LangChain agents",
    k=4,           # final results returned
    fetch_k=20,    # candidate pool
    lambda_mult=0.5,  # 0.0 = max diversity, 1.0 = max relevance
)
for doc in results:
    print(doc.page_content[:100])

Rule of thumb: Use lambda_mult=0.5 as a default. Lower it when your documents are repetitive; raise it when you need the highest-relevance chunks regardless of diversity.

Converting to a Retriever

Any vector store can be converted to a Retriever object, which plugs into chains and agents via the standard .invoke() interface.

python

# Default: similarity search, k=4
retriever = store.as_retriever()

# Custom: MMR with k=6
retriever = store.as_retriever(
    search_type="mmr",
    search_kwargs={"k": 6, "fetch_k": 30, "lambda_mult": 0.6},
)

# Custom: similarity with score threshold
retriever = store.as_retriever(
    search_type="similarity_score_threshold",
    search_kwargs={"score_threshold": 0.7, "k": 4},
)

# Use in a chain
docs = retriever.invoke("What is LangChain?")

Chroma — Local Development

python

pip install langchain-chroma

from langchain_chroma import Chroma
from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")

# Persistent store — survives restarts
store = Chroma(
    collection_name="my_docs",
    embedding_function=embeddings,
    persist_directory=".chroma_db",
)

# Load existing persistent store
store = Chroma(embedding_function=embeddings, persist_directory=".chroma_db")

FAISS — Offline & Fast

python

pip install langchain-community faiss-cpu

from langchain_community.vectorstores import FAISS
from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
store = FAISS.from_texts(["doc1", "doc2", "doc3"], embeddings)

# Save to disk
store.save_local("faiss_index")

# Load from disk
store = FAISS.load_local("faiss_index", embeddings, allow_dangerous_deserialization=True)

Choosing a Vector Store

Decision Tree

Need embeddings?
├─ Development / Testing?
│  ├─ Single machine? → Chroma or FAISS
│  └─ Unit tests only? → InMemoryVectorStore
├─ Production / Scale?
│  ├─ Fully managed? → Pinecone
│  ├─ Self-hosted? → Qdrant or Elasticsearch
│  └─ Existing cloud?
│     ├─ AWS? → RDS pgvector or OpenSearch
│     ├─ Azure? → Azure AI Search
│     └─ GCP? → Vertex AI Vector Search
└─ Special needs?
   ├─ Hybrid (keyword + vector)? → Elasticsearch, AstraDB
   ├─ Existing MongoDB? → MongoDB Atlas Vector Search
   └─ Serverless? → AstraDB, Pinecone Serverless

Factor	Chroma	FAISS	Pinecone	Qdrant
Setup	✓ Easy	Medium	Cloud only	Medium
Persistence	✓ SQLite	Manual save	✓ Auto	✓ Auto
Max scale	~100K	~10M	Unlimited	~1B
Cost	Free	Free	$	Free (self) / $
Hybrid search	No	No	✓ Yes	✓ Yes
Real-time	✓ Yes	No	✓ Yes	✓ Yes

Common Mistake

Starting with Pinecone (costs add up fast). Or choosing a vector store, only to realize mid-project that you need hybrid search (keyword + semantic).

Best Practice

Start local with Chroma for prototyping. As you scale, migrate to Qdrant (self-hosted) or Pinecone (managed). Chroma's API is compatible.

Quick Decision Guide

Notebook / local dev: Chroma — zero setup, persists automatically
Python script (offline): FAISS — no external deps, load from disk
Production MVP: Pinecone Serverless — minimal config, auto-scaling
Self-hosted production: Qdrant — powerful, open-source, Kubernetes-ready
Hybrid search needed: Elasticsearch or AstraDB — full-text + semantic
Unit tests: InMemoryVectorStore — no I/O, deterministic
Existing cloud investment: Use AWS (RDS pgvector), Azure (AI Search), or GCP (Vertex AI)

LangChain Vector Stores FAQ

What is the difference between FAISS, Chroma, and Pinecone in LangChain?

FAISS is an in-process library that is fast and fully offline, but you save and load the index yourself. Chroma is a local-first store that persists to SQLite with almost no setup, which makes it ideal for development. Pinecone is a fully managed cloud service built for production scale and hybrid search. All three expose the same LangChain vector store interface, so you can prototype on FAISS or Chroma and swap to Pinecone later with minimal code changes.

How do I persist a vector store in LangChain so I do not re-embed every run?

With FAISS, call store.save_local('faiss_index') to write the index to disk and FAISS.load_local('faiss_index', embeddings, allow_dangerous_deserialization=True) to reload it. With Chroma, pass persist_directory='.chroma_db' when you create the store and it writes to SQLite automatically, so reopening it with the same directory restores your documents. Persisting avoids paying for embeddings again on every restart.

What is the difference between similarity search and MMR retrieval?

Similarity search returns the top-k chunks with the closest embeddings, which can be redundant when several chunks repeat the same idea. MMR (Maximum Marginal Relevance) balances relevance with diversity, so it returns varied supporting passages instead of near-duplicates. Use similarity for precise lookups and MMR (search_type='mmr', tuned with lambda_mult) when your documents contain overlapping content.

How do I add or delete documents in a LangChain vector store?

Use store.add_documents([Document(...)]) or store.add_texts(['...']) to insert new content into an existing store, and store.delete(ids=['doc-id-1', 'doc-id-2']) to remove specific entries by id. Because every LangChain vector store shares this interface, the same add and delete calls work whether you are using Chroma, FAISS, Qdrant, or a managed service.

How do I turn a vector store into a retriever in LangChain?

Call store.as_retriever() to get a retriever that runs similarity search with k=4 by default. Pass search_type and search_kwargs to customise it, for example search_type='mmr' with {'k': 6, 'fetch_k': 30, 'lambda_mult': 0.6}, or search_type='similarity_score_threshold' with a score_threshold. The retriever exposes a standard .invoke(query) method, so it plugs directly into LCEL chains and agents.

Which vector store should I use in production?

For production, prefer a store with automatic persistence and scale: Pinecone if you want a fully managed serverless service, Qdrant if you want a powerful self-hosted option, or Elasticsearch and AstraDB when you need hybrid keyword plus vector search. Start local with Chroma or FAISS for prototyping, then migrate once you need higher scale, real-time updates, or hybrid search.

Vector Stores in LangChain: FAISS, Chroma, and Retrieval

What is a Vector Store?

Regular Database

Vector Store

Provider Overview

Core Methods (Universal API)

MMR Search — Diverse Results

Converting to a Retriever

Chroma — Local Development

FAISS — Offline & Fast

Choosing a Vector Store

LangChain Vector Stores FAQ

What is the difference between FAISS, Chroma, and Pinecone in LangChain?

How do I persist a vector store in LangChain so I do not re-embed every run?

What is the difference between similarity search and MMR retrieval?

How do I add or delete documents in a LangChain vector store?

How do I turn a vector store into a retriever in LangChain?

Which vector store should I use in production?