How Memory Sparse Attention scales LLM memory to 100 million tokens

Memory Sparse Attention (MSA) extends LLM context windows to 100 million tokens while preserving reasoning accuracy. Developed by researchers at Evermind, Shanda Group, and Peking University, MSA uses a differentiable routing mechanism to compress documents into precomputed attention values and retrieve relevant chunks during generation.