lyogavin/airllm
Runs 405B Llama3.1 on 8GB VRAM
View on index · View in 3D Map
Related repos
- Runs 70B models on 4GB GPUs
- run local LLMs fast on consumer GPUs with flexible quantization
- Run huge LLMs with 100k context on an 8GB GPU using SSD offload, no quantization
- Resources for running LLMs on local hardware
- Runs LLMs over a cluster of GPUs
- Trains LLMs with less VRAM
- Runs LLMs on AMD NPUs
- Self-hosted GPU video compressor targeting file sizes
- speeds up LLMs with fast key-value caching
