Free Lesson

Searching 100 billion vectors in object storage

Part of Exploring Modern AI Search

60 min

Feb 26, 2026 2:00 PM

Virtual (Zoom)

In this video

What you'll learn

Architecting a modern search database from first principles

Learn how turbopuffer was built from first principles as a fast, yet stateless query tier on object storage like S3.

The critical compute vs. bandwidth tradeoffs and bottlenecks

Many engines over-rely on expensive memory or SSDs. Learn where the true bottlenecks lie and how to optimize for scale.

How hierarchical clustering overcomes bandwidth bottlenecks

See how multi-level centroid-based indexing overcomes typical bandwidth bottlenecks on slower object storage tiers.

Binary Quantization at 100 Billion Scale

Discover how RaBitQ compresses vectors 16-32x while preserving recall to power fast search over 100 billion vectors.

Why this topic matters

In this Lightning Lesson, Nathan will introduce turbopuffer, a new search database built from first principles on object storage. He will introduce the architecture of the system at a high level before diving deep on how it was recently enhanced to serve production search at 100 billion vector scale.