Repository Showcase
A curated collection of interesting GitHub repositories
microsoft/BitNet
1-bit LLM inference on CPUs and GPUs with BitNet
Related repos
- LLM inference in C/C++ for running and serving models locally on various hardware
- framework for fast, resource-efficient LLM inference on local hardware
- Run 70B LLMs on a 4GB GPU with layer-wise inference and memory optimization, quantization optional
- Open-source toolkit for building scalable LLM inference infrastructure
- run large language models on any device
- Code and tutorials for building GPT-style language models from the ground up in PyTorch
- run local LLMs fast on consumer GPUs with flexible quantization
- Open-source tools for running and improving LLM applications—gateway, observability, data feedback, optimization
- Benchmark for LLMs on software engineering tasks
