kyuz0/amd-r9700-vllm-toolboxes

Container for serving LLMs on AMD Radeon R9700 GPUs

View on index · View in 3D Map

Related repos

• Containers for LLMs on AMD Radeon GPUs
• system container and virtual machine manager for Linux containers and VMs
• Runs LLMs on AMD NPUs
• Fast LLM inference and serving library
• DiskStation Manager in a Docker container
• Run 70B LLMs on a 4GB GPU with layer-wise inference and memory optimization, quantization optional
• Open-source toolkit for building scalable LLM inference infrastructure
• Armbian build scripts and tools for running on Amlogic S9xxx and similar ARMboxes
• Resources for running LLMs on local hardware
• containerized server for fetching web content for LLMs