Back to Devoured - May 28, 2026
DEVOURED
How CockroachDB Built Vector Indexing at Scale

How CockroachDB Built Vector Indexing at Scale

Data databaseaivector-databasedistributed-systems ByteByteGo
CockroachDB developed C-SPANN, a novel distributed vector indexing system that treats index data as regular table data, overcoming scalability limitations of existing algorithms like HNSW and IVF.
What: CockroachDB’s engineering team built C-SPANN, a hierarchical K-means tree vector index stored as key-value rows, for real-time, sharding-compatible, distributed vector search. This custom solution addresses architectural requirements like no central coordinator, persistent storage, minimal network hops, and incremental updates, which popular methods like HNSW failed to meet.
Why it matters: This demonstrates the challenges of integrating advanced AI capabilities like vector search into existing, highly distributed, transactional database architectures, often requiring custom engineering rather than off-the-shelf solutions to maintain core guarantees.
Takeaway: If evaluating vector databases for a distributed transactional system with strict consistency and real-time update needs, consider systems that integrate vector indexing natively into their distributed storage model.
Deep dive
  • CockroachDB needed vector search but existing algorithms (HNSW, IVF) didn't fit its distributed SQL architecture.
  • Key architectural requirements for the vector index included: no central coordinator, no large in-memory caches, minimal network hops, sharding compatibility, hot spot avoidance, and real-time incremental updates.
  • HNSW was ruled out due to its in-memory graph structure and resistance to sharding; IVF struggled with single-node assumptions and dynamic updates.
  • CockroachDB developed C-SPANN, combining ideas from Microsoft's SPANN/SPFresh and Google's ScaNN, adapted for its distributed environment.
  • C-SPANN uses a hierarchical K-means tree where vectors are grouped into partitions with centroids.
  • The critical design choice is storing each partition as self-contained key-value rows within CockroachDB, treating the index as ordinary table data.
  • This allows C-SPANN to leverage CockroachDB's existing distributed machinery for splitting, rebalancing, caching, and replication automatically.
  • Index maintenance, like splitting overly large partitions and merging small ones, happens incrementally in the background using "nearest partition assignment" for accuracy.
  • Vectors are compressed using RaBitQ (reducing 1536-dimension 2-byte floats to ~200 bytes per vector) to save storage and improve scan speed.
  • Quantization loss is compensated by a reranking step: searching compressed vectors for candidates, then fetching full-precision vectors for exact distance calculation.
  • Multi-tenancy is handled via prefix columns (e.g., user_id or geographical regions) on the vector index, creating separate K-means trees per tenant for performance and security.
  • C-SPANN offers real-time transactional freshness, native scaling, and vectors coexisting with transactional data.
  • Current limitations (as of the 25.2 preview) include Euclidean distance only, limited filtering on non-prefix columns, and not winning on pure latency benchmarks against specialized in-memory systems.
Decoder
  • Vector embeddings: Numeric representations of objects (like images, text, audio) where similar objects have similar vectors, enabling semantic search.
  • Approximate Nearest Neighbor (ANN) search: Algorithms that find vectors "close" to a query vector, sacrificing some accuracy for significantly faster search performance on large datasets.
  • HNSW (Hierarchical Navigable Small World): A popular graph-based ANN algorithm known for high accuracy, often used in vector databases, but typically relies on large in-memory structures and is difficult to shard.
  • IVF (Inverted File Index): An ANN algorithm that clusters vectors into a predefined number of centroids; queries search only a subset of clusters, faster than brute force but can struggle with dynamic updates in distributed settings.
  • C-SPANN: CockroachDB's custom-built distributed vector indexing system, based on a hierarchical K-means tree structure.
  • K-means tree: A hierarchical clustering structure where vectors are grouped into partitions with representative centroids, and those centroids are further grouped, forming a tree.
  • Centroid: The mean position of a cluster of vectors, representing its "center of mass."
  • Quantization: A technique to reduce the precision (and thus size) of vector dimensions, trading some accuracy for storage and performance gains.
  • RaBitQ: A specific quantization technique used by C-SPANN that reduces each vector dimension to a single bit.
Original article

Full article content is not available for inline reading.

Read the original article →

Explore all 61 articles in today's digest

Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi