vllm-project/vllm

Fast LLM inference and serving library

View on index · View in 3D Map

Related repos

• Minimal vLLM engine for fast offline LLM inference with a clean Python codebase
• AIBrix manages and scales large language model inference in the cloud for enterprise and developer workloads
• CLI tool for managing and serving LLMs with interactive terminal and server monitoring
• framework for fast, resource-efficient LLM inference on local hardware
• LLM inference in C/C++ for running and serving models locally on various hardware
• Build and train custom RL environments for LLM agents with modular tools and easy integration
• Lifelong memory for LLM agents
• run large language models on any device
• Benchmarks LLMs for end-to-end machine learning workflows