Sakana AI repositions context for long LLM tasks

Sakana AI researchers developed Context Re-Positioning (RePo), a technique that lets LLMs dynamically reorganize internal token positions. RePo uses a lightweight neural network to move related tokens closer in attention space, reducing cognitive load from fixed position embeddings like RoPE. This improves handling of long-context tasks like code repositories and RAG pipelines.