vllm-project/vllm

Library for running and serving large language models

View on index · View in 3D Map

Related repos

• AIBrix manages and scales large language model inference in the cloud for enterprise and developer workloads
• routes API requests to the best language model using semantic understanding
• Code repository for Hands-On Large Language Models book
• toolkit for compressing and deploying large language models
• Minimal vLLM engine for fast offline LLM inference with a clean Python codebase
• Curated resources and tools for building, fine-tuning, and deploying large language models
• run large language models on any device
• reinforcement learning library for training large language models, supports many RL algorithms, integrates with LLM tools, scales to models with up to 671B parameters
• Run large language models locally, manage and interact with different model types