llm-d/llm-d

Kubernetes-native framework for serving large language models at scale

View on index · View in 3D Map

Related repos

• Kubernetes-native distributed ML training and LLM fine-tuning
• run large language models on any device
• Runs large language models on iOS and macOS
• Automates large-scale application management on Kubernetes
• large language model tutorials and training basics from scratch
• Simulates thousands of Kubernetes nodes
• framework for fast, resource-efficient LLM inference on local hardware
• Curated resources and tools for building, fine-tuning, and deploying large language models
• Benchmarks storage performance in Kubernetes
• Code and tutorials for building large language models from scratch in PyTorch