GeeeekExplorer/nano-vllm

Minimal vLLM engine for fast offline LLM inference with a clean Python codebase

framework for fast, resource-efficient LLM inference on local hardware
kvcache-ai/ktransformers
Library for running and serving large language models
vllm-project/vllm
Python toolkit for building LLM and RAG apps with lots of small, specialized models
llmware-ai/llmware
CLI tool for managing and serving LLMs with interactive terminal and server monitoring
Chen-zexi/vllm-cli
Open cookbook for building top-tier code LLMs with complete training data and reproducible pipeline
OpenCoder-llm/OpenCoder-llm
multimodal LLM inference engine in Rust and Python
EricLBuehler/mistral.rs
Serve LLMs, speech, and multimodal models with one command
xorbitsai/inference
Open-source tools for running and improving LLM applications—gateway, observability, data feedback, optimization
tensorzero/tensorzero
AIBrix manages and scales large language model inference in the cloud for enterprise and developer workloads
vllm-project/aibrix
LLM inference in C/C++ for running and serving models locally on various hardware
ggml-org/llama.cpp