OpenAI predicts AI model failure rates before launch

OpenAI researchers proposed Deployment Simulation, a method that uses real anonymized user conversations to predict how often a new AI model will fail after release. The approach addresses gaps in standard safety testing, which relies on synthetic questions and can miss real-world behavior. Tests on GPT-5 series models with 1.3 million conversations showed verifiable frequency estimates.