Repository Showcase
A curated collection of interesting GitHub repositories
ggml-org/llama.cpp
LLM inference in C/C++ for running and serving models locally on various hardware
Related repos
- 1-bit LLM inference on CPUs and GPUs with BitNet
- run Llama 2 models in pure C with a single file
- Fast LLM inference and serving library
- framework for fast, resource-efficient LLM inference on local hardware
- Serve LLMs, speech, and multimodal models with one command
- run large language models on any device
- Open-source toolkit for building scalable LLM inference infrastructure
- turn home devices into a cluster for faster llama inference
- CLI for interacting with Model Context Protocol servers and LLMs
