gptcache

A learned, cost-aware eviction policy for semantic LLM caches that replaces heuristic policies like LRU/FIFO with a lightweight offline-trained model, improving regeneration-token savings while maintaining safe LRU fallback.

Research behind CacheVerifier (cacheverifier.com): synchronous verifier gating for semantic cache hits vs. static and adaptive thresholds on ~210k real requests, 3 datasets. Fine-tuning vs. off-the-shelf, label noise, drift, conformal risk control (CRC) under shift, and online ACI driven by randomized audits.

Verification layer for semantic caches: checks whether a cached answer is right for the question, not just how similar it is, on top of the cache you already run. Randomized audits give an exact upper bound on the live error rate; CRC-certified and online ACI thresholds; drift detection with auto-recalibration; entries to refresh; local scoring.

Keep poisoned answers out of LLM semantic caches: a drop-in Deletion Gain hit filter for GPTCache

How often does your semantic cache serve the wrong answer? Benchmark, CI check and multi-turn guard.

⚡ LLM complexity & cost router with response cache (RouteLLM + GPTCache). Cut API bills by routing simple queries to $0 models.

FreCoS: freshness- and cost-aware extension to GPTCache (BSc course project)