0xSojalSec/airllm
Runs 70B models on 4GB GPUs
View on index · View in 3D Map
Related repos
- Run 70B LLMs on a 4GB GPU with layer-wise inference and memory optimization, quantization optional
- Self-hosted GPU video compressor targeting file sizes
- Linux GPU configuration and monitoring tool
- Memvid stores data in video files (?) for AI memory
- Extension of Transformers for optimizing and running models on specific hardware
- Trains models exceeding GPU memory
- open source video generation for low VRAM GPUs
- personal cloud OS for home servers and Zima hardware
- Web desktop operating system for low power devices
- Run huge LLMs with 100k context on an 8GB GPU using SSD offload, no quantization
