LLM inference in C/C++.
Cross-referenced across 55 tracked directories
#322
Popularity Rank
2 / 55
Listed In
Growing
Adoption Stage
3/10/2023
Created
132,561
GitHub Stars
24,881
Forks
447
Contributors
2,590
Open Issues
10/27/2026
Last Commit
Gaining traction in the ecosystem
Cross-Posting Opportunities
Could also be listed in these directories:
Score: 100/100
0 dependency vulnerabilities found
vLLM Team
A high-throughput and memory-efficient inference and serving engine for LLMs
A distributed multi-model LLM serving system with web UI and OpenAI-compatible RESTful APIs.
an open-source NLP framework that allows you to use LLMs and transformer-based models from Hugging Face, OpenAI and Cohere to interact with your own data.
...more