Repository Showcase
A curated collection of interesting GitHub repositories
Mega4alik/ollm
Run huge LLMs with 100k context on an 8GB GPU using SSD offload, no quantization
Related repos
- Run 70B LLMs on a 4GB GPU with layer-wise inference and memory optimization, quantization optional
- run local LLMs fast on consumer GPUs with flexible quantization
- framework for fast, resource-efficient LLM inference on local hardware
- efficient attention for long-context LLMs
- OpenAI-compatible proxy that boosts LLM reasoning accuracy with 20+ optimization techniques, no retraining needed
- minimal 4D-parallelism framework for distributed LLM training and experimentation
- run large language models on any device
- open-source memory engine for LLMs and AI agents
- plug and play memory layer for LLMs
- GPU-optimized library for training massive transformer models at scale
