Vector Database Landscape
Choosing a vector DB depends on your scale, latency needs, budget, and infrastructure.
🌐 Cloud-Hosted (Managed, no setup)
Pinecone — Most popular, serverless, easy to use
Good for: Quick start, ≤100M vectors, don't want ops burden
Weaviate — Open-source with cloud option, powerful filtering
Good for: Complex queries, multi-modal, hybrid search
Qdrant — High performance, payload-based filtering, great docs
Good for: Low-latency, fine-grained filtering, e-commerce
🏠 Self-Hosted (Full control, ops burden)
Milvus — Distributed, scales to billions, mature
Good for: Enterprise, massive scale, on-prem required
Vespa — Production-grade, exact & approximate search, complex features
Good for: Complex ranking, e-commerce, media search
FAISS — Research library, in-memory, blazingly fast
Good for: Offline, small datasets, research prototypes
💾 In-Memory / Lightweight (Dev & testing)
Chroma — Simple, embedded, great for prototyping
Good for: Dev/testing, <100K vectors, learn RAG basics
LanceDB — Fast, serverless, great for notebooks
🎯 My recommendation: Start with Pinecone (easiest) or Chroma (free). Move to Qdrant/Milvus if you need self-hosting or fine-grained control.
Comparison Matrix
| DB | Setup | Query Latency | Filtering | Cost |
|---|---|---|---|---|
| Pinecone | Minutes | ~50ms | Good | $0.12-1.5/M vectors |
| Qdrant | Hours | ~10ms | Excellent | Open-source |
| Weaviate | Hours | ~50ms | Excellent | Open-source |
| Milvus | Days | ~5ms | Good | Open-source |
| FAISS | Minutes | <1ms | None | Free |
| Chroma | Minutes | ~100ms | Basic | Free |
Implementation Examples
Using Pinecone (Easiest)
import pinecone
from openai import OpenAI
# Initialize
pinecone.init(api_key="your-key", environment="us-west1-gcp")
index = pinecone.Index("my-rag-index")
client = OpenAI()
# Upsert embeddings
embeddings = client.embeddings.create(
model="text-embedding-3-small",
input=["chunk 1", "chunk 2", "chunk 3"]
)
vectors = [(f"id-{i}", emb.embedding) for i, emb in enumerate(embeddings.data)]
index.upsert(vectors=vectors)
# Search
query_emb = client.embeddings.create(
model="text-embedding-3-small",
input="How do I file taxes?"
).data[0].embedding
results = index.query(vector=query_emb, top_k=5)
for match in results.matches:
print(f"ID: {match.id}, Score: {match.score}")Using Chroma (Free, Local)
import chromadb
# Create collection
client = chromadb.Client()
collection = client.create_collection(name="my-docs")
# Add documents
collection.add(
ids=["id1", "id2", "id3"],
embeddings=[[0.1, 0.2], [0.2, 0.3], [0.3, 0.4]],
documents=["chunk 1", "chunk 2", "chunk 3"]
)
# Query
results = collection.query(
query_embeddings=[[0.1, 0.2]],
n_results=5
)
print(results)Scaling Considerations
<100K vectors (Dev/MVP)
Chroma, FAISS, or Pinecone free tier. Single machine fine.
100K-10M vectors (Small production)
Pinecone or self-hosted Qdrant. Monitor memory/latency.
10M-1B vectors (Scale)
Milvus (distributed), Vespa, or Pinecone enterprise. Add caching.
1B+ vectors (Massive scale)
Custom infrastructure, distributed Milvus, or specialized systems like Faiss with sharding.
Notes
Index type selection has permanent consequences
Choosing between HNSW, IVF-Flat, and IVF-PQ at index creation time is not easily reversible — you'd need to re-embed and re-index to switch. HNSW is the right default for most RAG workloads: it's memory-efficient, supports incremental inserts, and delivers good recall at low latency. IVF-PQ saves memory at scale (>50M vectors) but requires training on a representative sample before use and adds recall loss from quantization.
Pinecone serverless vs. pod-based: understand the billing model
Pinecone's serverless tier bills per read unit and write unit, which is economical for bursty, low-volume workloads. Pod-based plans bill for reserved infrastructure — better for steady high-throughput use. A serverless index that gets 10,000 queries per day can be cheaper than pods; at 500,000 queries per day it can be more expensive. Benchmark your query volume before choosing, and set cost alerts from day one.
Metadata filtering is not free — it runs before ANN search
When you filter by metadata (e.g., category == "finance"), most vector stores first apply the filter and then run ANN search on the reduced set. If the filter is very selective (few matching documents), ANN recall drops because the candidate pool is too small for HNSW to explore effectively. Counter-intuitive result: very tight metadata filters can hurt retrieval quality. Use broader filters and let reranking narrow down results.
Chroma's default persistence mode is not production-safe
Chroma in in-memory mode loses all data on restart. In persistent mode (SQLite backend), it works fine for development but is not designed for concurrent writes from multiple processes — you'll see lock errors under load. For anything beyond a single-process dev environment, switch to Qdrant (local or cloud) or a fully managed store. Don't optimize for Chroma in production; it's a prototyping tool.
RAG Vector Stores FAQ
What is the difference between Pinecone, Qdrant, and FAISS?
Pinecone is a fully managed cloud service (easiest to start, monthly cost). Qdrant and Milvus are open-source with cloud and self-hosted options (more control, lower marginal cost). FAISS is a local library (free, no server, limited to single machine).
When should I use FAISS vs a cloud vector database?
Use FAISS for prototypes, local development, or datasets under 1 million vectors on a single machine. Switch to a hosted database (Pinecone, Qdrant Cloud) when you need multi-node scaling, persistence across restarts, or team access.
What is HNSW indexing in vector databases?
HNSW (Hierarchical Navigable Small World) is the most common ANN index type. It builds a multi-layer graph of vectors for fast approximate nearest-neighbor lookup. Most production vector databases use HNSW or a variant of it.
How do I choose between cosine similarity and dot product for RAG?
Use cosine similarity when your embedding model produces variable-magnitude vectors (most transformer models). Use dot product when vectors are L2-normalized (many open-source embedding models). Check your model's documentation.
How many vectors can each vector database handle?
FAISS handles billions of vectors on a single GPU node. Qdrant and Milvus scale to tens of billions with distributed deployments. Pinecone is limited to hundreds of millions per index on its starter plan but scales with higher tiers.