AITowards Data Scienceabout 1 hour ago
Local LLM Energy Costs Measured Per Million Tokens
1 min read
A developer measured the actual GPU electricity cost of running eight local LLMs on a single RTX 3090. Five models were cheaper than a hosted cloud API, three were not. The cheapest model was not the smallest, nor the priciest the biggest. Costs were calculated using real power samples from nvidia-smi at Bulgarian electricity tariffs.
Level
Hype check
Tap to vote and see what everyone thinks.
Summary by ByteBrief
