TechTowards Data Scienceabout 8 hours ago
A Three-Phase Factual Recall Circuit in Gemma-2B and Gemma-12B-IT
1 min read
Activation patching reveals a three-phase factual recall circuit in Gemma-2B and Gemma-12B-IT. Facts are stored, routed, and read out across transformer layers, with the residual stream doing most of the work. The study used 60 prompt pairs across 20 knowledge categories to localize where factual knowledge lives inside a transformer.
Level
Hype check
Tap to vote and see what everyone thinks.
