GeeeekExplorer/nano-vllm

Minimal vLLM engine for fast offline LLM inference with a clean Python codebase

View on index · View in 3D Map

Related repos

• framework for fast, resource-efficient LLM inference on local hardware
• Library for running and serving large language models
• Python toolkit for building LLM and RAG apps with lots of small, specialized models
• CLI tool for managing and serving LLMs with interactive terminal and server monitoring
• Open cookbook for building top-tier code LLMs with complete training data and reproducible pipeline
• Lifelong memory for LLM agents
• multimodal LLM inference engine in Rust and Python
• Serve LLMs, speech, and multimodal models with one command
• Open-source tools for running and improving LLM applications—gateway, observability, data feedback, optimization
• Runs LLMs on AMD NPUs