openai/evals

framework for testing and benchmarking large language models with custom or built-in evaluations

tools for finetuning language models with instruction datasets and preference optimization
allenai/open-instruct
evaluating large language models across tasks and datasets
open-compass/opencompass
framework for testing language models on benchmarks and tasks
EleutherAI/lm-evaluation-harness
Run large language models locally, manage and interact with different model types
ollama/ollama
Run open-source or custom language models as OpenAI-style APIs with one command
bentoml/OpenLLM
Benchmark for LLMs on software engineering tasks
openai/SWELancer-Benchmark
Open-source platform for training, evaluating, and deploying AI models with end-to-end tools
oumi-ai/oumi
Toolkit for serving and running large language models through an inference server
huggingface/text-generation-inference
run language models in your browser with webgpu, openai api compatible
mlc-ai/web-llm
Run AI models, agents, and tools locallywith an OpenAI-compatible API
mudler/LocalAI