MII makes low-latency and high-throughput inference, similar to vLLM powered by DeepSpeed.
Cross-referenced across 55 tracked directories
#298
Popularity Rank
1 / 55
Listed In
Emerging
Adoption Stage
3/23/2022
Created
2,102
GitHub Stars
191
Forks
209
Open Issues
6/30/2025
Last Commit
Recently added to the ecosystem
Score: 100/100
0 dependency vulnerabilities found
Run an AI-powered security scan to analyze this package's source code for vulnerabilities, prompt injection vectors, data exfiltration risks, and behavior mismatches.
Scans fetch actual source code from the GitHub repository, not just the README.
Nvidia Framework for LLM Inference
A high-throughput and low-latency inference and serving framework for LLMs and VLs
SGLang is a fast serving framework for large language models and vision language models.
a toolkit for deploying and serving Large Language Models (LLMs).