TensorRT-LLM

A high-throughput and memory-efficient inference and serving engine for LLMs

Run LLMs and batch jobs on any cloud. Get maximum cost savings, highest GPU availability, and managed execution -- all with a simple interface.

A high-throughput and low-latency inference and serving framework for LLMs and VLs

Efficient Triton Kernels for LLM Training.