AIHacker Newsabout 3 hours ago
Mesh LLM: distributed AI computing on iroh
1 min read
Mesh LLM pools GPUs and memory across machines into one OpenAI-compatible API. The system distributes model compute across iroh endpoints with no central server. Models up to 235B parameters can run split across modest machines. A mobile app built on iroh's Swift SDK is planned.
Level
Hype check
Tap to vote and see what everyone thinks.
