Search

veRL is a flexible and efficient RL framework for LLMs.

Making large AI models cheaper, faster, and more accessible.

A simple, performant and scalable Jax LLM!

Open-source framework for fine-tuning and evaluating LLMs. It simplifies the process of experimenting with different training configurations and makes it easy to reproduce and share results, supporting features like LoRA, QLoRA, DeepSpeed, PEFT, and multi-GPU setups.

LLM inference in C/C++.

Nvidia Framework for LLM Inference

A distributed multi-model LLM serving system with web UI and OpenAI-compatible RESTful APIs.

Run LLMs and batch jobs on any cloud. Get maximum cost savings, highest GPU availability, and managed execution -- all with a simple interface.

an open-source NLP framework that allows you to use LLMs and transformer-based models from Hugging Face, OpenAI and Cohere to interact with your own data.

Fine-tune, serve, deploy, and monitor any open-source LLMs in production. Used in production at [BentoML](https://bentoml.com/) for LLMs-based applications.

A high-throughput and low-latency inference and serving framework for LLMs and VLs

Efficient Triton Kernels for LLM Training.

Comprehensive set of tools for working with local LLMs for various tasks.

Locally running websearch using LLM chains

A python package for Txt-to-SQL with self hosting functionalities and RESTful APIs compatible with proprietary as well as open source LLM.

A paid product for building, comparing, and shipping language model apps.

A paid product for testing and improving prompts.

An open-source library for evaluating task performance of language models and prompts.

A programming language for LLM interaction with support for typed prompting, control flow, constraints, and tools.

An open-source LLM app for building multi-agent LLMs applications in an easy and lazy way, supports model deployment and fine-tuning.