AILobstersabout 4 hours ago
AI At Home Part 2: Multi GPU Drifting
1 min read
A home server built from e-waste GPUs runs AI language models, but token generation is limited by memory bandwidth, not compute. Layer and tensor parallelism split models across GPUs, with layer parallel capped at single-GPU speed.
Level
Hype check
Tap to vote and see what everyone thinks.
