hhhuang/CAG

Cache-Augmented Generation for LLMs

View on index · View in 3D Map

Related repos

• speeds up LLMs with fast key-value caching
• Proxy for caching LLM prompts
• Hybrid cache with resiliency features
• semantic cache for LLM queries, speeds up GPT apps, cuts API costs
• Lifelong memory for LLM agents
• a fast, simple key-value cache
• Least Recently Used (LRU) cache implementation
• Cache-Control header management
• Runs LLMs and RAG locally on desktops