zilliztech/GPTCache

semantic cache for LLM queries, speeds up GPT apps, cuts API costs

View on index · View in 3D Map

Related repos

• speeds up LLMs with fast key-value caching
• Hybrid cache with resiliency features
• a fast, simple key-value cache
• Redis-compatible cache store with sub-300 microsecond latency
• open-weight GPT models for reasoning and agent tasks
• fastRAG helps you build efficient retrieval-augmented generation pipelines with optimized LLMs and search tools
• open source status page for incidents and outages
• 1-bit LLM inference on CPUs and GPUs with BitNet
• Benchmark for LLMs on software engineering tasks
• scans LLMs for vulnerabilities like prompt injection, hallucination, and data leaks