ggml-org/llama.cpp

Runs quantized LLMs on consumer hardware

View on index · View in 3D Map

Related repos

• Resources for running LLMs on local hardware
• turn home devices into a cluster for faster llama inference
• run local LLMs fast on consumer GPUs with flexible quantization
• Matches LLM models to hardware specifications
• Menu bar app for local LLMs
• run Llama 2 models in pure C with a single file
• Runs 405B Llama3.1 on 8GB VRAM
• Integrate llama.cpp into Go applications
• 1-bit LLM inference on CPUs and GPUs with BitNet