google-ai-edge/LiteRT-LM

Inference framework for Large Language Models on edge devices

View on index · View in 3D Map

Related repos

• run large language models on any device
• Toolkit for serving and running large language models through an inference server
• Runs large language models in the browser with WebGPU acceleration
• Inference of language models on hardware with limited memory
• Large language model using diffusion for text generation
• open notes and guides for training large language models
• AIBrix manages and scales large language model inference in the cloud for enterprise and developer workloads
• Training and inference for multimodal language models
• litgpt: tools to train, finetune, and run large language models from the command line
• run language models in your browser with webgpu, openai api compatible