Search

This repository contains code to quantitatively evaluate instruction-tuned models such as Alpaca and Flan-T5 on held-out tasks.

a lean, efficient, and easy-to-hack codebase to research LLMs.

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

Minimalistic large language model 3D-parallelism training.

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Ongoing research training transformer models at scale.

A native PyTorch Library for large model training.

DeepSpeed version of NVIDIA's Megatron-LM that adds additional support for several features such as MoE model training, Curriculum Learning, 3D Parallelism, and others.

A Native-PyTorch Library for LLM Fine-tuning.

Generative AI framework built for researchers and PyTorch developers working on Large Language Models (LLMs), Multimodal Models (MMs), Automatic Speech Recognition (ASR), Text to Speech (TTS), and Computer Vision (CV) domains.

Efficient Training for Big Models.

Mesh TensorFlow: Model Parallelism Made Easier.

An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.

A library for accelerating Transformer model training on NVIDIA GPUs.

An Easy-to-use, Scalable and High-performance RLHF Framework (70B+ PPO Full Tuning & Iterative DPO & LoRA & RingAttention & RFT).

A framework that specializes in efficient fine-tuning. On its GitHub page, you can find ready-to-use fine-tuning templates for various LLMs, allowing you to easily train your own data for free on the Google Colab cloud.

SGLang is a fast serving framework for large language models and vision language models.

a toolkit for deploying and serving Large Language Models (LLMs).

NVIDIA Framework for LLM Inference(Transitioned to TensorRT-LLM)

To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.