LLM role flaw enables unstoppable attacks

Researchers presented a paper at ICML showing LLMs identify text by style, not role tags. This chain-of-thought forgery tricked models into revealing forbidden information like drug synthesis. The flaw is considered fundamentally unsolvable.