cache-eviction

Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)

SIEVE cache - lockfree, concurrent cache - faster and simpler than LRU

A comprehensive open-source cache trace dataset

Virtual memory page replacement algorithms: Least Recently Used (LRU) and Second-Chance Clock algorithm.

In-memory LRU cache with per-key TTL expiration, deterministic eviction, and hit/miss/stale telemetry for LLMs

In-memory LRU cache with per-key TTL expiration, deterministic eviction, and hit/miss/stale telemetry for LLMs

Virtual memory page replacement algorithms: Least Recently Used (LRU) and Second-Chance Clock algorithm.

Hardware-level security protocol for mitigating cache-based side-channel attacks.

A prefix-cache advisor for LLM serving infrastructure that recommends KV-cache capacity and eviction policies from your request traces/logs.

A learned, cost-aware eviction policy for semantic LLM caches that replaces heuristic policies like LRU/FIFO with a lightweight offline-trained model, improving regeneration-token savings while maintaining safe LRU fallback.

A distributed gRPC cache that learns what to evict instead of guessing: a LightGBM model, trained offline on sampled access traces, predicts whether an object will be re-requested beyond a "Belady boundary," and the cache evicts the worst candidate from a small random sample.

Trace-driven research harness for KV-cache hierarchy policy evaluation in long-context LLM inference.

REAL: REtrieval-reAsoning and Logic-constructed Attention Behaviors for Long-Context KV Cache Compression

Idle but not forgotten: usage-driven agent memory forgets the dormant skills you'll need next quarter; NicheMem's competitive ownership retains them exactly. Mechanism-level benchmark + verified theory (sibling of GAUSE).

A high-performance, modular implementation of the Least Frequently Used (LFU) Cache in modern C++17. The project demonstrates efficient cache eviction using a combination of hash maps and doubly linked lists to achieve O(1) average time complexity for both get() and put() operations.

An implementation of W-TinyLFU policy

CLOCK replacement algorithm, an efficient, compact and concurrent alternative to LRU cache eviction policy.

Event-driven simulator for prefix KV-cache eviction policies in LLM serving systems

HybridCacheSystem is a modern C++ framework for building high-performance, multi-layered cache systems. It supports both LRU and LFU caching strategies, allows combining them for hybrid setups, and provides a unified interface for cache management and customization.

Five Twitter production cache traces replayed through FIFO, LRU, LFU, ARC and W-TinyLFU in Java 21, with every policy checked against a brute-force model before its hit rates are compared.