kvcache.ai
Organization@kvcache-ai
KVCache.AI is a joint research project between MADSys and top industry collaborators, focusing on efficient LLM serving.
8
Published Tools
16,873
Total Stars
0
Weekly Downloads
1,239
GitHub Followers
15
Public Repos
Published Tools
8 Skillsacross 2 categoriesktransformers
kvcache-ai
KTransformers: CPU-GPU heterogeneous inference framework for LLMs
mooncake-transfer-engine-efa-non-cuda
Mooncake Authors
A KVCache-centric Disaggregated Architecture for large-scale LLM inference and training. (AWS EFA, Non-CUDA version)
mooncake-transfer-engine-non-cuda
Mooncake Authors
A KVCache-centric Disaggregated Architecture for large-scale LLM inference and training. (Non-CUDA version)
mooncake-transfer-engine-efa
Mooncake Authors
A KVCache-centric Disaggregated Architecture for large-scale LLM inference and training. (AWS EFA, CUDA version)
mooncake-transfer-engine
Mooncake Authors
A KVCache-centric Disaggregated Architecture for large-scale LLM inference and training.
mooncake-transfer-engine-npu
Mooncake Authors
A KVCache-centric Disaggregated Architecture for large-scale LLM inference and training. (Ascend NPU version)
mooncake-transfer-engine-cuda13
Mooncake Authors
A KVCache-centric Disaggregated Architecture for large-scale LLM inference and training. (CUDA 13 version)
mooncake-transfer-engine-musa
Mooncake Authors
A KVCache-centric Disaggregated Architecture for large-scale LLM inference and training. (MUSA version)