Back to Agents
TGI
AgentLLM Inferencellmai-appawesome-list
a toolkit for deploying and serving Large Language Models (LLMs).
Directory Presence
Cross-referenced across 55 tracked directories
Adoption Metrics
#311
Popularity Rank
1 / 55
Listed In
Emerging
Adoption Stage
2d
Listed For
Recently added to the ecosystem
Related Agents
To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.
AgentLLM Inference
1 dir
SGLang is a fast serving framework for large language models and vision language models.
AgentLLM Inference
1 dir