xorbitsai/inference

Serve LLMs, speech, and multimodal models with one command

View on index · View in 3D Map

Related repos

• multimodal LLM inference engine in Rust and Python
• Toolkit for serving and running large language models through an inference server
• Multimodal LLM inference engine for mobile devices
• LLM inference in C/C++ for running and serving models locally on various hardware
• Open-source toolkit for building scalable LLM inference infrastructure
• Fast text embeddings inference server for popular open source models
• Controls drones with natural language through LLMs
• Code and tutorials for building large language models from scratch in PyTorch
• Skywork R1V: multimodal reasoning model extending R1-series LLMs to images and text, open-source, efficient transfer, 38B parameters
• Survey of unified multimodal models