Chip Huyen on Cutting AI Inference Costs

Ahead of her return to P99 CONF 2026, AI engineer Chip Huyen's advice on reducing inference costs is revisited. She argues inference costs, which can be 10 to 100 times training costs, are a major profitability hurdle. Her key strategies include model optimizations like quantization and distillation, and service optimizations like batching techniques to improve efficiency without new hardware.