What Is a Vector Database?

A prototype built on an in-memory vector store worked fine at a small scale. At 50 million vectors, it crashes every six hours. This is the moment "which vector database" stops being a minor implementation detail and becomes a real infrastructure decision, one with genuine tradeoffs between speed, accuracy, and operational complexity.

The most obvious way to find the nearest neighbors of a query vector is to compute its distance to every single vector in the database and sort. This is exact, and it is also O(n) per query, computing 50 million distances for every single search. At any real scale, this is far too slow for an interactive product. A vector database's entire purpose is replacing that brute-force scan with an index structure that finds approximately the nearest neighbors, trading a small, usually negligible amount of accuracy for a dramatic speedup.

• -HNSW (Hierarchical Navigable Small World): builds a multi-layer graph where each vector is connected to its approximate neighbors. A search starts at a sparse top layer and navigates down through progressively denser layers, homing in on the right neighborhood fast. Strong query speed and accuracy, at the cost of higher memory usage and slower index-build time.
• -IVF (Inverted File Index): clusters the vector space into a fixed number of partitions (via something like k-means) ahead of time, and a query only searches the handful of partitions closest to it, instead of the whole space. Lower memory footprint than HNSW, faster to build, but recall depends heavily on how many partitions get probed per query.

Neither is universally "better." HNSW tends to win when query latency matters most and memory is available to spend. IVF (often combined with product quantization to compress vectors further) tends to win at truly massive scale where memory cost per vector becomes the dominant constraint. Most managed vector databases let you choose, or pick a sensible default and expose the tuning knobs (like how many HNSW graph connections per node, or how many IVF partitions to probe) for when the default is not good enough.

Recall cosine similarity from the previous tier. Vector databases typically support a small set of distance metrics, cosine similarity, dot product, and Euclidean (L2) distance, and the right choice usually follows directly from how the embedding model itself was trained. Most modern text embedding models are trained (and normalized) for cosine similarity specifically, so using a different metric against their output can silently degrade result quality even though nothing throws an error.