How NVIDIA's Inference Software Stack Powers the Lowest Token Cost

NVIDIA's full-stack inference software, codesigned with its GPUs and networking, reduced token costs by up to 5x on the DeepSeek V4 model in one month on the Blackwell platform. The stack connects three layers to compound individual optimizations, achieving up to 20x throughput gains through techniques like disaggregated serving and multi-token prediction.