FastFlowLM/FastFlowLM

Runs LLMs on Ryzen NPUs without GPUs

View on index · View in 3D Map

Related repos

• Runs LLMs over a cluster of GPUs
• 1-bit LLM inference on CPUs and GPUs with BitNet
• Automates GPU kernel generation with LLMs and agents
• Run 70B LLMs on a 4GB GPU with layer-wise inference and memory optimization, quantization optional
• local LLM serving with GPU and NPU acceleration
• Resources for running LLMs on local hardware
• Matches LLM models to hardware specifications
• run local LLMs fast on consumer GPUs with flexible quantization
• Open-source tools for running and improving LLM applications—gateway, observability, data feedback, optimization