Perplexity Releases Two Multimodal Embedding Models for Edge and Cloud

Perplexity AI has released pplx-embed-v2-late, a pair of ColBERT-style multimodal embedding models designed for both edge devices and high-quality indexing. The models come in 0.6 billion and 9 billion parameter sizes, each optimized for different stages of a retrieval pipeline.

Two Models for Different Use Cases

The 0.6B model is intended for live queries on edge devices like laptops or small GPUs, using only about 1.2 GB of memory. The 9B model (listed as 8B on Hugging Face) serves as the document index builder, requiring 16–18 GB of GPU memory. Both models share a unified embedding space, meaning a 9B index can be queried with the 0.6B encoder without losing compatibility.

Both models are available under an MIT license, allowing commercial use. A hosted API endpoint is planned but not yet live.

Performance Highlights

On the MADQA benchmark (agentic PDF question-answering), the 9B model scores 92.4% accuracy, outperforming competitors like Mixedbread’s retriever (88.9%) ¹. The 0.6B model achieves 90.1% on the same task.

In domain-specific text retrieval across 72 tasks, the 9B model leads all tested models with an nDCG@10 of 81.3%, while the 0.6B model trails only Gemini Embedding 2 by 0.3 points.

Image retrieval shows a gap: Tencent’s EVIE model scores higher on ViDoRe v3, but the 0.6B model still ranks second with 62.3% nDCG@10.

A key advantage is mixing sizes: querying a 9B index with the 0.6B encoder recovers about half the quality gap at 0.6B query cost, making it efficient for latency-sensitive applications.

Technical Details

Both models use 128-dimensional token vectors—16× to 32× narrower than typical 2,048- to 4,096-dim rivals—which reduces storage and speeds up similarity searches.

They are distilled from a larger 18B teacher model using LEAF-style token-level training, which creates the shared embedding space. Pages are encoded as images, eliminating the need for OCR.

The 0.6B base is a pruned Qwen3.5-0.8B model, while the 9B version likely builds on a similar architecture. Both require sentence-transformers >= 6.0.0 and transformers >= 5.4.0.

One limitation is that a single input cannot mix text and images; they must be processed separately.

Best Use Cases

The models excel at visual document search over PDFs, slides, and scanned reports—especially in agentic RAG systems that handle large web or PDF collections.

The edge-friendly 0.6B variant enables low-latency search where the index lives in the cloud and queries run locally. This hybrid approach balances cost, speed, and quality.

All scores are self-reported; a technical report is expected later. Researchers can experiment with the weights now on Hugging Face.