Zyora-Dev/zse

Runs LLM inference without PyTorch, Triton, or transformers, compiling kernels directly to GPU code for rapid model loading.

View on index · View in 3D Map

Related repos

• Zig inference engine for AMD GPUs
• run CUDA apps on AMD GPUs
• Code for building a GPT-like LLM from scratch in PyTorch
• Runs LLMs on Ryzen NPUs without GPUs
• Runs YOLOv8, YOLO11, and YOLO26 models via ONNX Runtime using pure Rust.
• Run 70B LLMs on a 4GB GPU with layer-wise inference and memory optimization, quantization optional
• Builds GPT-like LLMs from scratch in PyTorch
• framework for fast, resource-efficient LLM inference on local hardware
• 1-bit LLM inference on CPUs and GPUs with BitNet
• Code and tutorials for building large language models from scratch in PyTorch