AIPandailyabout 2 hours ago
Moore Threads Open-Sources Trillion-Parameter MoE Stack
1 min read
Moore Threads open-sourced Megatron-MUSA-Titan, a large-model training infrastructure for its GPUs on GitHub. The stack supports trillion-parameter mixture-of-experts models on clusters of more than 10,000 GPUs, covering pretraining and reinforcement learning with stable training and high model FLOPs utilization.
Level
Hype check
Tap to vote and see what everyone thinks.
In this storyMeta
