Agents

LLM application: DeepSeek-Coder-v2-16|236B-MOE

LLM application: RecurrentGemma-2B

LLM application: Pythia-1|1.4|2.8|6.9|12B

Open-Source Toolkit for Efficient Unstructured Data Processing with Pre-built Modules and Local to Cluster Scalability.

Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.

A powerful tool for creating high-quality training datasets for Large Language Models

a lightweight LLM evaluation suite that Hugging Face has been using internally.

Eval tools by OpenAI.

a repository for evaluating open language models.

This repository contains code to quantitatively evaluate instruction-tuned models such as Alpaca and Flan-T5 on held-out tasks.

a lean, efficient, and easy-to-hack codebase to research LLMs.

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

Minimalistic large language model 3D-parallelism training.

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Ongoing research training transformer models at scale.

A native PyTorch Library for large model training.

DeepSpeed version of NVIDIA's Megatron-LM that adds additional support for several features such as MoE model training, Curriculum Learning, 3D Parallelism, and others.

A Native-PyTorch Library for LLM Fine-tuning.

veRL is a flexible and efficient RL framework for LLMs.

Generative AI framework built for researchers and PyTorch developers working on Large Language Models (LLMs), Multimodal Models (MMs), Automatic Speech Recognition (ASR), Text to Speech (TTS), and Computer Vision (CV) domains.

Efficient Training for Big Models.

Mesh TensorFlow: Model Parallelism Made Easier.

A simple, performant and scalable Jax LLM!

An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.