A high-throughput and memory-efficient inference and serving engine for LLMs
Cross-referenced across 55 tracked directories
#104
Popularity Rank
3 / 55
Listed In
Mature
Adoption Stage
2/9/2023
Created
95,645
GitHub Stars
24,020
Forks
6,304,358
Monthly Downloads
9,538
Open Issues
10/27/2026
Last Commit
Widely adopted across the ecosystem
Cross-Posting Opportunities
Could also be listed in these directories:
Score: 100/100
0 dependency vulnerabilities found
SkyPilot Team
Run LLMs and batch jobs on any cloud. Get maximum cost savings, highest GPU availability, and managed execution -- all with a simple interface.
...moreA high-throughput and low-latency inference and serving framework for LLMs and VLs