What is SQ8 scalar quantization?

SQ8 (Scalar Quantization 8-bit) compresses each float32 vector component into a single uint8 byte, reducing memory footprint to roughly one quarter of the original. DBX applies this quantization at storage time and uses asymmetric distance computation at query time — the query vector stays in float32 while stored vectors remain in uint8.

How much memory does SQ8 save compared to float32?

SQ8 reduces vector storage to approximately one quarter of float32 payload. For 100,000 vectors at 1536 dimensions, float32 requires ~600 MB while SQ8 requires ~150 MB. Combined with mmap, idle tenants reside in page cache rather than resident RAM, with approximately 14-17 MiB RSS per idle worker.

Does scalar quantization reduce search accuracy?

SQ8 with asymmetric distance computation preserves high recall quality for most workloads. The query vector remains in full float32 precision while only the stored vectors are quantized, which is more accurate than symmetric quantization where both sides are compressed.

How does mmap help with multi-tenant vector storage?

Memory-mapped (mmap) vector files allow the operating system to manage which tenant data is in RAM. Active tenants are paged in; idle tenants reside on disk in page cache. This makes the cost proportional to active tenants, not total tenants — critical for hosting many small customers on one node.

Storage & density

SQ8 Scalar Quantization: Memory-Efficient Vector Search at Scale

8-bit scalar-quantized vectors in mmap'd files with asymmetric distance computation. ~4x memory savings over float32. Idle tenants live in page cache, not resident RAM.

Why Memory Density Matters for Multi-Tenant Vector Search

When you host 100 tenants on a single node, each with up to 100,000 vectors, float32 storage becomes prohibitively expensive. At 1536 dimensions (OpenAI's embedding size), float32 requires ~600 MB per tenant — that's 60 GB of RAM for the vectors alone across 100 tenants.

Most of those tenants are idle at any moment. Paying for resident RAM for every idle customer's vectors is waste. The metric that matters is cost per active tenant, not cost per provisioned tenant.

How SQ8 + mmap Achieves Density

DBX stores vectors as 8-bit scalar-quantized (SQ8) rows in memory-mapped files:

SQ8 quantization — each float32 component is mapped to a uint8 byte, reducing payload to ~¼ of float32
Asymmetric distance — the query vector stays in float32 while stored vectors are in uint8, preserving better accuracy than symmetric quantization
mmap storage — the OS pages in active tenants and pages out idle ones automatically
Page cache residency — idle tenants sit in the kernel's page cache, evicted under memory pressure without process management

Measured density

Memory Footprint Numbers

Metric	Value
Idle worker RSS	~14–17 MiB
SQ8 vs float32 payload	~¼ size
Certified profile	100 tenants / node
Max vectors per tenant	100,000
Worst idle GET latency	~3–12 ms
ANN p50 latency	2.304 ms

Measured on the certification host and workload. Production numbers depend on your hardware, dataset, and access patterns. Run make soak on your target hardware.

How SQ8 Fits Into the Storage Architecture

Per-tenant data directory

Each tenant's SQ8 vector files, HNSW graph, WAL, and snapshots live in an isolated directory. Backup and delete operate on one customer's files.

Named spaces

Each named SPACE (text, image, code) creates separate mmap/HNSW files in the same tenant directory. All are SQ8. Quota and backup cover all spaces for that tenant.

Encryption boundary

In strict mode, the HNSW graph and WAL are encrypted under a per-tenant key. SQ8 .vec rows stay mmap plaintext so idle tenants remain cheap. Use fscrypt or LUKS for at-rest encryption of vector rows.

Host more tenants on less hardware.

Get started All features