Prompt Injection as Role Confusion

Researchers Charles Ye, Jasmine Cui, and Dylan Hadfield-Menell found that LLMs cannot reliably distinguish system role tags from user input. Models treat text style more seriously than content, enabling jailbreaks. Destyling text reduced attack success from 61% to 10%, revealing a fundamental vulnerability called role confusion.