Google TurboQuant: 6x Memory Reduction for AI Models With Zero Accuracy Loss

Google Research has released TurboQuant, a breakthrough compression algorithm for large language models (LLMs) that cuts memory requirements by up to six times without sacrificing accuracy. The algorithm is being presented at ICLR 2026 and could fundamentally change how AI models are deployed at scale.

TurboQuant compresses the KV cache in LLMs down to just 3 bits, compared to today's standard of 16–32 bits. On Nvidia H100 GPUs, benchmarks show up to 8x faster computation of attention logits. Crucially, this requires no model retraining or fine-tuning.

The technology works in two stages. The first, called PolarQuant, converts data vectors into polar coordinates to enable high-quality compression. The second applies a 1-bit QJL transform to the residual error, eliminating systematic bias in attention score calculations.

Google tested TurboQuant on open models including Gemma, Mistral, and Llama 3.1, across benchmarks such as LongBench, Needle In A Haystack, and RULER. Results show TurboQuant matches or outperforms existing methods like KIVI.

The implications extend far beyond Google. Memory-heavy models that currently require expensive server GPUs could potentially run on consumer hardware. Financial markets noticed immediately: shares in memory makers like Micron and SK Hynix fell as investors reassessed future AI memory demand.

For enterprises running AI at scale, TurboQuant represents a potential cost reduction. Cheaper inference lowers operating costs for everything from customer support bots to internal analytics tools.

The algorithm is available through Google Research, and framework support is expected to follow quickly.