DS DevShelfHub Projects · AI tools
Tutorials / LangChain / Reference / Classes / NVIDIAEmbeddings
Embeddings langchain-nvidia-ai-endpoints Intermediate

NVIDIAEmbeddings: Reference Guide

By DevShelfHub

Use NVIDIA's high-performance embeddings.

What is NVIDIAEmbeddings?

NVIDIAEmbeddings wraps NVIDIA's NV-Embed family via the NVIDIA AI Endpoints API (api.nvcf.nvidia.com). These GPU-accelerated models are trained specifically for retrieval tasks — nv-embed-qa-4 is optimised for question-answer style retrieval while nvidia/nv-embedqa-mistral-7b-v2 targets longer document contexts. The API requires an NGC API key (set as NVIDIA_API_KEY).

The class supports both synchronous embed_query / embed_documents and async aembed_query / aembed_documents, making it suitable for async FastAPI or LangGraph workflows. Batch requests are handled automatically; the internal chunk size can be tuned via model_kwargs if you hit per-request payload limits.

NVIDIA's embedding API is priced per 1 000 tokens and is aimed at enterprise use cases where latency, throughput SLAs, and on-premise deployment (via NVIDIA NIM) matter more than per-unit cost. For on-premise deployment, point base_url at your NIM inference endpoint — the LangChain class does not care whether it is calling the hosted API or a local NIM container.

When to Use

You need high-performance embeddings. Use for latency-sensitive systems.

Use Cases

  • • High-performance
  • • GPU-optimized
  • • Real-time
  • • Large-scale
  • • Enterprise
  • • Performance critical

Key Features

  • ✓ GPU-optimized
  • ✓ Fast
  • ✓ High quality
  • ✓ Enterprise
  • ✓ Managed
  • ✓ Specialized

When NOT to Use

For cost-sensitive projects.

Notes

Set NVIDIA_API_KEY — no unauthenticated access

All NVIDIA AI Endpoints calls require an NGC API key. Set NVIDIA_API_KEY in your environment or pass api_key= directly. Get a key at build.nvidia.com. Without it, the first call raises an AuthenticationError with no useful message.

nv-embed-qa-4 vs nvidia/nv-embedqa-mistral-7b-v2

nv-embed-qa-4 is the default and suits general retrieval. nvidia/nv-embedqa-mistral-7b-v2 produces higher-quality embeddings for long documents (4096-token context) but is slower and costs more. Check the NVIDIA AI Catalog for the current model list — new models are added regularly.

base_url lets you run against a NIM container

NVIDIA NIM (Inference Microservices) lets you self-host the same models on your own GPU hardware. Set base_url to your NIM endpoint URL and the LangChain class will hit it instead of the cloud API — useful for air-gapped or high-throughput production deployments.

Dimensions are model-specific and non-configurable

Unlike text-embedding-3 models (which support Matryoshka truncation), NVIDIA embedding models return a fixed dimension. Check the model card before building your vector store index — changing models later requires re-embedding your entire corpus.

Import

python 
from langchain_nvidia_ai_endpoints import NVIDIAEmbeddings

Configuration

Parameter Type Default Purpose
model str nv-embed-qa-4 NVIDIA model

Usage Examples

Hosted NVIDIA Embeddings

python 
from langchain_nvidia_ai_endpoints import NVIDIAEmbeddings

embeddings = NVIDIAEmbeddings(model='nv-embed-qa-4')
vec = embeddings.embed_query('What is CUDA?')
print(len(vec))  # 2048 dimensions

Batch Document Embeddings

python 
# Batch embed documents for a vector store
docs = ['CUDA is a parallel programming platform', 'GPUs accelerate ML workloads']
embeddings = NVIDIAEmbeddings(model='nv-embed-qa-4')
vecs = embeddings.embed_documents(docs)
print(len(vecs), len(vecs[0]))  # 2, 2048

On-premise NIM Container

python 
# Point at a local NIM container instead of the hosted API
embeddings = NVIDIAEmbeddings(
    model='nv-embed-qa-4',
    base_url='http://localhost:8080/v1',
)
vec = embeddings.embed_query('on-premise query')

Common Pitfalls

❌ Expect lowest cost

✅ Premium for performance

Alternative Embedding Models

Model When to Use
OpenAIEmbeddings For cost-effective quality

Related LangChain References

Browse the full LangChain API reference index to explore more classes, methods, and decorators, or start with the LangChain introduction tutorial for end-to-end context on building with NVIDIAEmbeddings and the wider framework.

NVIDIAEmbeddings FAQ

What is NVIDIAEmbeddings in LangChain?

Use NVIDIA's high-performance embeddings. NVIDIAEmbeddings wraps NVIDIA's NV-Embed family via the NVIDIA AI Endpoints API (api.nvcf.nvidia.com). These GPU-accelerated models are trained specifically for retrieval tasks — nv-embed-qa-4 is optimised for question-answer style retrieval while nvidia/nv-embedqa-mistral-7b-v2 targets longer document contexts. The API requires an NGC API key (set as NVIDIA_API_KEY). The class supports both synchronous embed_query / embed_documents and async aembed_query / aembed_documents,…

Which package provides NVIDIAEmbeddings?

DevShelfHub documents NVIDIAEmbeddings from the langchain-nvidia-ai-endpoints package. Pin your installed LangChain version and match imports to the snippet on this page.

When should I use NVIDIAEmbeddings?

You need high-performance embeddings. Use for latency-sensitive systems.

When should I avoid using NVIDIAEmbeddings?

For cost-sensitive projects.

How do I import NVIDIAEmbeddings in Python?

from langchain_nvidia_ai_endpoints import NVIDIAEmbeddings

Where can I explore more LangChain API reference pages?

Open the LangChain API reference index on DevShelfHub to browse classes, methods, and decorators, each with runnable examples, parameters, common mistakes, and cross-links.