vllm-project/llm-compressor

Compresses large language models for efficient vLLM deployment

View on index · View in 3D Map

Related repos

• AIBrix manages and scales large language model inference in the cloud for enterprise and developer workloads
• Curated resources and tools for building, fine-tuning, and deploying large language models
• Compress prompts up to 20x while keeping LLM performance intact
• run large language models on any device
• Serving framework for text, image, video, and audio models
• curated list of large language model papers, tools, and resources
• Code and tutorials for building large language models from scratch in PyTorch
• Speeds up LLM inference with speculative decoding
• Integrates large language models into a spaCy pipeline
• no-code GUI for fine-tuning large language models