efeslab/fiddler

Run unquantized Mixtral-8x7B on a single 24GB GPU

View on index · View in 3D Map

Related repos

• Self-hosted GPU video compressor targeting file sizes
• Run 70B LLMs on a 4GB GPU with layer-wise inference and memory optimization, quantization optional
• Runs LLMs and RAG locally on desktops
• Chromium fork with performance tweaks, codec support, and builds for many platforms
• Run huge LLMs with 100k context on an 8GB GPU using SSD offload, no quantization
• DJ software for live mixing on Windows, macOS, and Linux
• run local LLMs fast on consumer GPUs with flexible quantization
• v86: x86 machine emulator running in the browser using JavaScript and WebAssembly
• Code editor with hardware acceleration
• Lightweight 2D graphics library for modern GPUs