AITowards Data Scienceabout 2 hours ago
Local LLM Cost Measured on Apple Silicon
1 min read
A 27-billion-parameter dense model costs five to nine times more per token than a 120-billion-parameter mixture-of-experts model on an M3 Ultra Mac Studio, because cost tracks bytes streamed per token, not parameter count. The 27B model draws 138 watts at 21.5 tokens per second, while the 120B MoE draws 94 watts at 74 tok/s.
Level
Hype check
Tap to vote and see what everyone thinks.
In this storyApple
