messkan/prompt-cache
Reduces LLM costs by up to 80% with intelligent semantic caching.
View on index · View in 3D Map
Related repos
- speeds up LLMs with fast key-value caching
- automates prompt tuning for LLMs with feedback and example generation
- semantic cache for LLM queries, speeds up GPT apps, cuts API costs
- Compress prompts up to 20x while keeping LLM performance intact
- test and compare LLM prompts and models locally
- alternative to RAG for LLMs, skips real-time retrieval by caching knowledge in context
- a fast, simple key-value cache
- Fast hybrid cache with resiliency features
- Estimates USD costs for LLM prompts and completions
