
A July 2025 METR randomized controlled trial of 16 developers on 246 real tasks found developers estimated AI made them 20% faster, but data showed they were 19% slower. A February 2026 follow-up failed because developers refused to work without AI. A newer 800+ task dataset showed a smaller -4% effect.
Tap to vote and see what everyone thinks.
Summary by ByteBrief
Estimating No-CoT Task-Completion Time Horizons of Frontier AI Models