DigitalOcean Gradient™ AI GPU Droplets Optimized for Inference: Increasing Throughput at Lower the Cost

DigitalOcean released an Inference Optimized Image for GPU Droplets that increased Llama 3.3 70B throughput by 143% to 2,000 tokens per second on 2 H100 GPUs. The image reduced cost per million tokens by 75% to $1.472 and lowered time to first token by 40.7%.