Repository Showcase
A curated collection of interesting GitHub repositories
turboderp-org/exllamav2
run local LLMs fast on consumer GPUs with flexible quantization
Related repos
- Run 70B LLMs on a 4GB GPU with layer-wise inference and memory optimization, quantization optional
- Run huge LLMs with 100k context on an 8GB GPU using SSD offload, no quantization
- framework for fast, resource-efficient LLM inference on local hardware
- LLM inference in C/C++ for running and serving models locally on various hardware
- 1-bit LLM inference on CPUs and GPUs with BitNet
- LLMPEG: GPU-accelerated video transcoding tool
- local LLM serving with GPU and NPU acceleration
- run local LLMs fast on your Mac with Swift and MLX
- Open-source tools for running and improving LLM applications—gateway, observability, data feedback, optimization
- semantic cache for LLM queries, speeds up GPT apps, cuts API costs
