Lightbits Launches Inferra KV Cache Engine for AI Inference

Lightbits Labs released Inferra, a software engine that moves KV cache data beyond GPU high-bandwidth memory to improve AI inference economics and performance. The product uses predictive prefetching to stage data before the GPU needs it, claiming over 100-fold reduction in time to first token for long-context workloads.