ggml-org/llama.cpp

LLM inference in C/C++ for running and serving models locally on various hardware

View on index · View in 3D Map

Related repos

• 1-bit LLM inference on CPUs and GPUs with BitNet
• run Llama 2 models in pure C with a single file
• Fast LLM inference and serving library
• framework for fast, resource-efficient LLM inference on local hardware
• Serve LLMs, speech, and multimodal models with one command
• run large language models on any device
• Resources for running LLMs on local hardware
• Open-source toolkit for building scalable LLM inference infrastructure
• turn home devices into a cluster for faster llama inference