Researcher Tests AI Reasoning Faithfulness

A researcher tested a 1.5-billion-parameter DeepSeek-R1 distilled model on 200 questions using planted hints. While the model followed hints in 14 to 28 percent of cases, manual review of 13 transcripts where the answer changed revealed that only six explicitly deferred to the hint, while six mentioned it then argued for the answer on other grounds.