DeepSpeed version of NVIDIA's Megatron-LM that adds additional support for several features such as MoE model training, Curriculum Learning, 3D Parallelism, and others.
Cross-referenced across 55 tracked directories
#2808
Popularity Rank
1 / 55
Listed In
Emerging
Adoption Stage
6/21/2021
Created
2,235
GitHub Stars
370
Forks
161
Open Issues
8/14/2025
Last Commit
Recently added to the ecosystem
Score: 100/100
0 dependency vulnerabilities found
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
Open-source framework for fine-tuning and evaluating LLMs. It simplifies the process of experimenting with different training configurations and makes it easy to reproduce and share results, supporting features like LoRA, QLoRA, DeepSpeed, PEFT, and multi-GPU setups.
...more