arXiv (OAI Expanded)
open access
Shared KV Caching for Replicated 27B Inference: Correctness Failures and Performance Boundaries
Shared host-memory caching can avoid repeated prefill when a request moves between inference replicas. Its usefulness depends on both correct state transfer and lost prefix locality. We study two single-GPU 27B vLLM replicas sharing a 256 GiB LMCache pool. After adopting an existing packed-page patc
Conceptio Open Archive
Analyze & explore related research in Conceptio
Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.
This document is indexed with metadata only: full text is not available in the archive for this record. Open the official source ↗
Related documents
- Shared KV Caching for Replicated 27B Inference: Correctness Failures and Performance Boundaries#919350
- ParaAnya: Accelerating Parallel Diffusion Sampling with Plug-and-Play Output Caching#1131723
- ParaAnya: Accelerating Parallel Diffusion Sampling with Plug-and-Play Output Caching#1122267
- Fixed State, Long Reach: What a Constant-Size Cache Buys Block Diffusion at Scale#999980
- The Golden Path Hypothesis: Reusable Schedules in Diffusion Caching#1144310
Record · ID 1084918
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Bring this data into your agent: Connect via MCP or API · 20 free credits to start · no card required
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive