vllm-project/aibrix

AIBrix manages and scales large language model inference in the cloud for enterprise and developer workloads

View on index

Related repos

• Library for running and serving large language models
• Toolkit for serving and running large language models through an inference server
• run large language models on any device
• open notes and guides for training large language models
• VILA: Visual language models for AI tasks
• Toolkit for distributed training and inference of large AI models using parallelism and memory management techniques
• AI framework for semantic search, workflows, and large language models
• reinforcement learning library for training large language models, supports many RL algorithms, integrates with LLM tools, scales to models with up to 671B parameters
• routes API requests to the best language model using semantic understanding