GKE Inference Gateway cuts AI latency by 92%

Google's GKE Inference Gateway delivers 92.8% shorter wait times and 62.6% lower inter-token latency versus the next leading managed Kubernetes service, per an independent benchmark. The gateway uses prefix caching and model-aware routing to minimize accelerator idle time. Snap reported prefix cache hit rates of 75-80% using the system.