LMCache/LMCache

speeds up LLMs with fast key-value caching

View on index · View in 3D Map

Related repos

• semantic cache for LLM queries, speeds up GPT apps, cuts API costs
• Resources for running LLMs on local hardware
• Benchmarks LLMs for end-to-end machine learning workflows
• Benchmark for evaluating LLMs on real-world GitHub issues
• Reduces LLM token usage compared to JSON
• Least Recently Used (LRU) cache implementation
• Benchmark for LLMs on software engineering tasks
• Runs LLMs offline on computers
• Compresses, deploys, and serves LLMs
• efficient attention for long-context LLMs