ggml-org/llama.cpp

LLaMA.cpp: C/C++ LLM inference for various hardware

View on index · View in 3D Map

Related repos

• 1-bit LLM inference on CPUs and GPUs with BitNet
• turn home devices into a cluster for faster llama inference
• framework for fast, resource-efficient LLM inference on local hardware
• run Llama 2 models in pure C with a single file
• Fast LLM inference and serving library
• Open-source toolkit for building scalable LLM inference infrastructure
• multimodal LLM inference engine in Rust and Python
• Serve LLMs, speech, and multimodal models with one command
• Menu bar app to run local LLMs on a Mac