ScienceLobstersabout 2 hours ago
Canonical basis realignment makes Transformer axes measurable
2 min read
A lossless coordinate transformation, CBLL, makes every hidden axis of Transformer LLMs independently measurable and controllable. On Qwen 2.5 0.5B, zeroing axis 62 alone collapses MMLU from 47.50% to 21.25%. The method works across RMSNorm and LayerNorm families, including SmolLM2, Pythia, and OLMoE.
Level
Hype check
Tap to vote and see what everyone thinks.
Summary by ByteBrief
