ggml-org/llama.cpp

LLM inference in C/C++ for running and serving models locally on various hardware