DeepSpeed version of NVIDIA's Megatron-LM that adds additional support for several features such as MoE model training, Curriculum Learning, 3D Parallelism, and others.
Cross-referenced across 55 tracked directories
#3488
Popularity Rank
1 / 55
Listed In
Emerging
Adoption Stage
6/21/2021
Created
2,235
GitHub Stars
370
Forks
161
Open Issues
8/14/2025
Last Commit
Recently added to the ecosystem
Score: 100/100
0 dependency vulnerabilities found
Leandro von Werra <[email protected]>
Train transformer language models with reinforcement learning.
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.