New method cuts LLM input 16x without accuracy loss

A research team from NYU, Columbia, Princeton, University of Maryland, Harvard and Lawrence Livermore National Laboratory published a paper on a method that compresses LLM context input by 16x without degrading model accuracy. The approach addresses the computational bottleneck of growing context windows from agent runs, retrieved documents, and conversation history.