vllm-project/vllm
Fast LLM serving with PagedAttention
View on index · View in 3D Map
Related repos
- Serving framework for text, image, video, and audio models
- CLI tool for managing and serving LLMs with interactive terminal and server monitoring
- Minimal vLLM engine for fast offline LLM inference with a clean Python codebase
- Speeds up LLM inference with speculative decoding
- vLLM implementation for CosyVoice built from scratch
- Lifelong memory for LLM agents
- KVM management with a web UI, CLI, and API
- AIBrix manages and scales large language model inference in the cloud for enterprise and developer workloads
- Runs natural language workflows in cloud sandboxes
