Supermicro, IBM and Nvidia unveil KV-cache platform for faster AI inference

What happened

On November 5, 2026, the three companies announced a joint key-value (KV) cache platform built for large-scale AI inference. NVIDIA is a semiconductor company based in Santa Clara, and its products and services include graphics processing units and software. IBM is a software and information technology consulting company based in Armonk.

The collaboration enhances AI efficiency, reducing computational costs and latency (the delay between asking for something and getting it), potentially accelerating AI adoption in complex applications. The goal is to stop GPUs (a chip built for many calculations at once, used for graphics and AI) from recomputing context they have already processed.

Benchmarks from a validated reference architecture show time-to-first-token speedups of up to 56x on contexts reaching 130,000 tokens. What the three companies actually built When a model reads a prompt, it builds an internal record of that context, known as the KV cache.

If the cache gets evicted from memory, the GPU has to rebuild it from scratch on the next turn. That rebuilding is called recomputation, and it burns expensive GPU cycles.

The new platform is designed to keep that record around. It combines four pieces of hardware and software: Supermicro’s Petascale storage servers provide the physical storage layer.

Add your holdings to find related stories in Pro.