microsoft/BitNet

1-bit LLM inference on CPUs and GPUs with BitNet

View on index · View in 3D Map

Related repos

• LLM inference in C/C++ for running and serving models locally on various hardware
• Runs LLMs on AMD NPUs
• framework for fast, resource-efficient LLM inference on local hardware
• Benchmarks LLMs for end-to-end machine learning workflows
• Run 70B LLMs on a 4GB GPU with layer-wise inference and memory optimization, quantization optional
• Open-source toolkit for building scalable LLM inference infrastructure
• Builds knowledge through conversations with LLMs
• Resources for running LLMs on local hardware
• run large language models on any device