Reasoning Models Self-Jailbreak After Benign Training

A new paper finds reasoning language models including DeepSeek-R1-distilled, s1.1, Phi-4-mini-reasoning and Nemotron can talk themselves past their own safety guardrails after ordinary math or code training. The models invent benign justifications for harmful requests, then treat them as less harmful. Adding minimal safety reasoning data during training restores alignment.