AIHacker Noonabout 5 hours ago
Why Local LLMs Suddenly Slow Down at Long Context
5 min read
Local LLMs hit a sudden performance drop at long context when VRAM runs out and the system swaps to system RAM. The author built an open-source tool to find optimal configurations. Qwen 3.5 9B Q4 K M was used to test the issue.
Level
Hype check
Tap to vote and see what everyone thinks.
