ModelCloud/GPTQModel

Model quantization with hardware acceleration for Nvidia, AMD, and Intel

View on index · View in 3D Map

Related repos

• Applies GPTQ quantization to GGUF models
• competes to train GPT-2 models super fast on 8xH100 GPUs using speedrun benchmarks
• open-source GPU cluster manager for AI models
• open-weight GPT models for reasoning and agent tasks
• Run 70B LLMs on a 4GB GPU with layer-wise inference and memory optimization, quantization optional
• Runs large language models on consumer hardware without internet
• AI inference engine for Apple Silicon, hybrid GPU/MPSGraph, easy model support
• Modelica-based modeling and simulation environment
• Monitors hardware performance with a dashboard and usage graphs
• Python framework for fast GPU simulation and graphics