EvoCache: Accelerating Browser Large Language Model Delivery with Public Cache Anchors

EvoCache: Accelerating Browser Large Language Model Delivery with Public Cache Anchors Zhiyu Wang

Rajkumar Buyya

zhiyu.wang4@unimelb.edu.au The University of Melbourne Melbourne, Australia

rbuyya@unimelb.edu.au The University of Melbourne Melbourne, Australia

arXiv:2611.04638v1 [cs.DC] 3 Nov 2026

Abstract

architecture and quantization method are identical, differences in calibration data, tensor types, metadata, and file layout change the final GGML Universal File (GGUF). An audit of 1,000 public model repositories finds that 61 of the 100 most popular LLMs are offered by multiple publishers. Among 738 groups containing the same LLM and quantization configuration from different publishers, 724 (98.1%) contain no byte-identical files. Ordinary whole-file caching can therefore reuse almost none of this previously downloaded data. A browser also cannot directly expose one publisher’s cached files to other sites. If model installation speed depends on whether a cache entry exists, a site can use timing to infer whether the user installed an LLM on another site. Modern browsers prevent such cross-site observation by partitioning storage [7, 16]. Existing sharing mechanisms handle only more limited cases. Cross-Origin Storage (COS) requires byte-identical objects [27]; Compression Dictionary Transport (CDT) requires a site to specify a dictionary in its partition [20, 22]; and cross-origin model caching likewise depends on identical complete model files [29]. None allows a site to specify only its desired target while the browser privately reuses another publisher’s model package. This gap lies in the Web’s own infrastructure: how browsers deliver, cache, and isolate large resources across sites, the focus of the Web Infrastructure and Agentic Systems track. This paper proposes EvoCache, which places a trusted broker inside the browser and uses public anchors that are provisioned by the browser and independent of site-visit history. A site submits only a publisher-authorized target. The broker matches corresponding tensors and, for each data chunk, selects exact reuse, compressed fetch, or lossless residual reconstruction. The final file must exactly match the target supplied by the publisher. The selected anchor, reconstruction plan, transferred bytes, and execution progress remain hidden from the site. A target reconstructed from a public anchor can be returned immediately. For cached data left by site activity, EvoCache returns results at a predetermined public time, thereby protecting users’ cross-site privacy. We implement EvoCache in a privileged Chromium extension. Across 23 pairs of LLM packages from different publishers that are difficult to reuse directly, EvoCache achieves a median traffic reduction of 77.3%, 27.9 percentage points more than the best generic file-differencing method given the same tensor-structure information, and leads on every pair. Its minimum traffic reduction remains 76.6% for larger LLMs. A further 240 real end-to-end browser experiments show that, relative to ordinary full downloads, EvoCache reduces network traffic by 78.9% and page-ready time by 64.2%. Relative to the best-performing comparator, it further reduces network traffic by 14.2% and page-ready time by 7.9%. This paper makes three contributions:

Large Language Models (LLMs) can run directly in a browser, but first use typically requires downloading model files that are hundreds of MiB or even several GiB in size. The same LLM is often repackaged by different publishers; these files share substantial content, yet byte differences prevent whole-file caches from reusing them. We propose EvoCache, a browser-managed LLM delivery system. EvoCache uses public cache anchors that are stored in advance by the browser and independent of browsing history to reconstruct losslessly the publisher version requested by a site. A site specifies only the target, while the browser selects the anchor and performs reconstruction, keeping both hidden from the site. An audit of 1,000 public repositories finds that 61 of the 100 most popular LLMs are offered by multiple publishers and that 98.1% of overlapping versions cannot be reused by whole-file caching. Across 23 crosspublisher model-package pairs, EvoCache achieves a median traffic reduction of 77.3% and outperforms every comparator. Across 240 real end-to-end browser experiments, EvoCache reduces network traffic by 78.9% and page-ready time by 64.2% relative to ordinary full downloads. For cached data left by other sites, EvoCache also protects users’ cross-site cache privacy by using the same return time for cache hits and misses.

CCS Concepts • Information systems → Browsers; • Computer systems organization → Client-server architectures; • Security and privacy → Web application security.

Keywords in-browser large language models, Web caching, delta compression, cache privacy ACM Reference Format: Zhiyu Wang and Rajkumar Buyya. 2027. EvoCache: Accelerating Browser Large Language Model Delivery with Public Cache Anchors. In Proceedings of the ACM Web Conference 2027 (WWW ’27). ACM, New York, NY, USA, 11 pages.

1

Introduction

Browser-based Large Language Models (LLMs) allow Web pages to run models directly on users’ devices, reducing server costs while keeping sensitive data local [2, 15, 30]. The browser must first download the model weights [31]. Even after quantization, a model package is typically hundreds of MiB, and larger models can reach several GiB [36]. Model downloads therefore often determine how long users must wait. Caching should reduce repeated downloads, but different publishers often package the same LLM differently. Even when the model 1

WWW ’27, May 10–14, 2027, Dublin, Ireland

Zhiyu Wang and Rajkumar Buyya

Table 1: Real model-package corpus.

• Through a large-scale repository audit and byte-level analysis, we show that cross-publisher repackaging is a widespread problem for browser caching and reveal that different GGUF files still contain substantial compressible redundancy at the tensor level. • We design EvoCache. The system selects reusable anchors inside the browser and chooses lossless reconstruction operations according to the characteristics of each data chunk. It both reproduces the publisher-authorized target exactly and keeps cache state and reconstruction hidden from the site. • We implement and evaluate a complete browser delivery path. The experiments cover 45 real model packages, publishers and LLMs not used during system development, larger LLMs, and 240 real end-to-end browser runs, in which EvoCache outperforms every comparator in both page-ready time and network traffic.

Value

LLM families

SmolLM2-135M; SmolLM2-360M; Qwen2.5-0.5B; Llama-3.2-1B 4 / 12

Publishers / immutable repositories Quantizations Model packages / evaluated pairs Corpus size Target sizes

2.3

Q3_K_M, Q4_K_M, Q5_K_M, Q8_0 45 / 42 18.4 GiB 89.2 MiB–1.23 GiB

Audited Cross-Publisher Corpus

Table 1 summarizes 45 GGUF model packages selected from public Hugging Face repositories. Each pair contains versions from two publishers for the same upstream LLM and quantization class, producing 42 directed pairs. The source and target files in every pair are different, so ordinary whole-file caching cannot produce a hit. Corresponding tensors are matched by name, shape, and quantization class to analyze calibration and per-tensor type differences introduced by different publishers. After dividing the model packages into chunks, the 42 pairs fall into two groups. In 19 pairs, at least 90% of chunks are identical and can be reused directly by EvoCache. In the other 23 pairs, hereafter called calibration-divergent pairs, different publishers used different calibration settings and identical chunks account for no more than 41.8%. Most content in these pairs must therefore be reconstructed from residuals, and they are used to evaluate EvoCache’s core reconstruction method.

The rest of the paper is organized as follows. Section 2 establishes the caching problem created by cross-publisher repackaging. Section 3 defines the system goals and security rules. Section 4 presents the design of EvoCache. Section 5 evaluates the system. Section 6 discusses related work. Section 7 concludes the paper.

2 Why Whole-File Caching Fails for LLMs 2.1 The Tension in Web Caching Unpartitioned caching accelerates identical third-party resources but turns cache hits into cross-site history signals, so modern browsers partition network and storage state [7, 16]. COS shares public-list-admitted, content-addressed files [27]. CDT reconstructs a response against a matching cached dictionary under HTTP partitioning and negotiation [20]; the server learns the selected dictionary, and Chromium caps a dictionary at 100 MiB [3]. Only 3 of the 45 model packages in the corpus fall below this limit. These mechanisms reuse a complete object or an explicitly negotiated dictionary. LLM packaging introduces another dimension: the same LLM may be quantized and converted independently by multiple publishers. GGUF stores typed metadata, tensor descriptions, alignment information, and tensor bytes [8]. Metadata order changes file offsets, while importance matrices and per-tensor type selection change the quantized bytes. A target site knows which publisher’s model package it needs, but it should not learn which other eligible model packages the browser holds.

2.2

Property

2.4

Why Quantized Bytes Differ

Why can two files differ while remaining reusable? Their publishers start from the same logical tensors, but calibration data can change small values inside the packed representation, and importanceaware policies can assign different types to some tensors. Across all 23 calibration-divergent pairs, publishers either differ in whether they use an importance matrix or use different calibration sources and chunk counts. Tensor names and shapes remain the same in every pair. Eleven pairs also retain every tensor type and extent; the other twelve change some type assignments. Within tensor bytes that remain positionally comparable (91.1% median coverage), 81.0% are equal at the median and only 5.79% of bits flip. These sparse, aligned changes explain why compressed residuals remain small even when nearly every 1-MiB chunk differs. EvoCache uses these shared bytes to reconstruct the publisher-selected package while preserving its quantization and calibration choices.

How Common Is Independent Repackaging?

We audit the 1,000 most-downloaded GGUF repositories returned by the public Hugging Face API on August 22, 2026, and group them by their declared upstream LLM [12]. Among the 100 mostdownloaded LLMs, 61 are offered by at least two independent publishers, with a median of three publishers. These multi-publisher LLMs account for 74.8% of downloads among the relevant repositories. Among 738 cross-publisher version groups for the same LLM and quantization configuration, 724 (98.1%) share no byte-identical Git Large File Storage (LFS) object. This result shows that independent repackaging of the same LLM is common, yet these model packages can almost never directly hit a whole-file cache.

3 Preventing Cross-Site Cache Probing 3.1 Timing Differences Expose Cache State When the browser already holds reusable data, installing an LLM is usually faster than downloading the complete file again. A malicious site can repeatedly request the same LLM and measure the completion time of each request. If cache hits are consistently faster, 2

EvoCache: Accelerating Browser Large Language Model Delivery with Public Cache Anchors

the site can determine whether the browser holds related data and infer whether the user previously installed the same LLM on another site. The threat model considers an attacker that controls a Web origin. It can request any authorized target, use high-resolution clocks, and observe API results, errors, and Performance entries available to the page. Normal browser isolation prevents it from directly reading process memory, another origin’s OPFS, or browserinternal requests, and the attacker cannot forge the publisher’s Ed25519 signature [1]. It can therefore probe the cross-site cache only through installation behavior visible to the page. We assume publisher endpoints do not collude with the calling origin, and strict release reuses history-derived data only for publishers on a browser-maintained list. EvoCache targets observations exposed through the page’s APIs and complements browser-level isolation of shared hardware resources.

3.2

uses immediate release and reuses only that anchor, while every other target from a listed publisher uses strict release whether or not history-derived data exists. Targets from unlisted publishers without an anchor reuse only data in the requesting site’s own partition. For a request, the public description 𝑝 contains the target identity and size together with a public CPU/network service class. Before cache lookup, the broker computes a release time 𝐷 (𝑝) from the time reserved for a cold download, commit, and a public safety margin. Because this calculation occurs before cache lookup, 𝐷 (𝑝) does not depend on whether reusable data exists. A cache hit may finish reconstruction early, but the broker waits until 𝐷 (𝑝) to return the handle. A cold download returns its handle at the same time. The admission check in Section 3.2 ensures that the cold path can finish before 𝐷 (𝑝). The page therefore observes the same completion time in both cases, while a cache hit still saves network bytes. Formally, for an admitted request class 𝑝, let 𝐶 0 and 𝐶 1 differ only in cross-origin eligible cache state. When both paths complete by 𝐷 (𝑝), the page observes 𝑇API (𝑝, 𝐶 0 ) ≡ 𝑇API (𝑝, 𝐶 1 )

The Page Can Specify Only the Target

To prevent a page from actively probing the browser cache, EvoCache allows it to specify only an authorized target, not the base package or reconstruction method. The page already knows the target it requested and the model packages it supplied. What must remain hidden is whether the browser found other reusable data, which base package it selected, and the reconstruction plan, transfer volume, and execution progress. The interface is:

= (accepted, 𝐷 (𝑝), target handle).

navigator.modelPackages.install({ root, manifestURL, publisherKey }) -> ModelPackageHandle Here root identifies the target, and manifestURL and publisherKey establish publisher authorization; the browser, not the page, selects the release policy described in Section 3.3. The returned ModelPackageHandle opens only the requested target. The interface exposes no has or list operation, so a page cannot query which LLM files the browser has cached. It also exposes no base-package parameter, byte counter, selected reconstruction method, or fine-grained progress. Base-package selection, planning, download, and reconstruction all remain inside the browser broker. The broker must also decide whether to accept the request before reading shared cache state. This decision uses only the security policy, publisher authorization, caller-owned quota, and public targetsize, device-capacity, and network classes. Shared storage has a separate accounting domain, so its occupancy cannot affect admission. If the browser cannot reserve enough time and resources for a full cold download and commit, it rejects the request before cache lookup. Acceptance or rejection therefore cannot reveal whether reusable data exists.

3.3

WWW ’27, May 10–14, 2027, Dublin, Ireland

Return Rules for Two Cache Classes

EvoCache defines one return rule for each of the two kinds of reusable data. Public anchors provisioned by the browser contain no browsing history, so their results can be returned as soon as reconstruction finishes. Cache left after another site installs an LLM may reflect browsing history, so the corresponding result can be returned only at a public deadline. EvoCache calls the latter rule strict release. The broker chooses the rule from public information before cache lookup: a target with a browser-provisioned anchor 3

This expression says that both requests are accepted, both return at 𝐷 (𝑝), and both produce a handle for the requested target. The relation ≡ also requires opaque handle identifiers to have the same distribution in both cases. The broker emits no intermediate API events, and corresponding failures expose the same errors. Appendix A gives the isolation and scheduling conditions needed for these properties. When those conditions hold, the page cannot distinguish 𝐶 0 from 𝐶 1 through any API-visible observation. Because 𝐷 (𝑝) adds a public safety margin to the cold-path estimate for the declared network class, the cold path completes before 𝐷 (𝑝) whenever the network meets that class; the guarantee therefore covers every request the broker admits under its declared class. A site-facing API cannot mix the two cache-entry classes or turn a history-dependent entry into a public anchor. Only browser policy can decide that an entry belongs in the public-anchor namespace.

4

EvoCache’s LLM Reconstruction Workflow

This section explains how EvoCache reconstructs and delivers a target LLM. The broker first verifies the target and aligns reusable model packages, then selects a reconstruction method for each data chunk, and finally reconstructs and verifies the target as a stream before delivering it. EvoCache can reuse two classes of data. The first is a public anchor provisioned in advance by the browser. Its presence is independent of which sites the user has visited, so the broker can return a verified target immediately. The second is cached data left after another site installs an LLM. Its presence may reflect browsing history, so the broker waits until the public deadline 𝐷 (𝑝) before returning the result. Both classes use the same lookup and reconstruction process; they differ in planning objective and return time. Figure 1 shows this common process and the return control used for strict release. Steps 1–8 show the complete process. The page submits an installation request for the target LLM (1). Before querying shared cache state, the broker decides whether to admit the request. Under

WWW ’27, May 10–14, 2027, Dublin, Ireland

Zhiyu Wang and Rajkumar Buyya

strict release, this step also sets 𝐷 (𝑝) (2), after which the broker prepares the return control (3). Once the request is admitted, the broker privately finds eligible cached data (4), selects an action for each chunk (5), fetches the required objects (6), reconstructs and verifies the target (7), and returns the LLM handle according to the release policy (8). Public-anchor mode returns the handle as soon as the target is verified, whereas strict release waits until 𝐷 (𝑝). The page cannot select the base package or observe the cache lookup, reconstruction plan, transfer volume, or execution progress.

4.1

change patterns and can produce a smaller XOR residual. The publisher precomputes and compresses both residuals, and EvoCache selects the one with the smaller complete transfer size for each chunk. Low-bit lane differences. GGUF packs several 2- or 4-bit quantized values into one byte, so comparing the whole byte can hide relationships between the individual values. Low-bit lane differences separate the packed 2-, 4-, or 8-bit values and compute a circular signed difference for each value. The result can also undergo bitplane transposition before zstd compression [4, 19]. The decoder reverses these operations in the opposite order, so the target data remain unchanged. Field separation. A quantized block commonly contains scales, subscales, and low-bit codes that represent LLM weights. These fields have different meanings and change patterns, so processing them as one byte stream can reduce compression effectiveness. For Q5_0, Q4_K, and Q6_K, EvoCache uses the validated GGML type to locate the fields and separates the scale, subscale, and packedcode streams. Each field can independently select exact reuse, compressed fetch, XOR, BMS, or a low-bit lane difference. A change in one field therefore does not prevent the other fields in the same block from being reused. Public-anchor planning. Public anchors are provisioned by the browser and do not reflect which sites a user has visited, so the result can be returned as soon as reconstruction finishes. Among memory-feasible candidates, the planner selects the action with the shortest estimated delivery time for each chunk:

Target Verification and Package Alignment

The publisher-signed manifest defines the target file together with its semantic regions and chunks, and the broker delivers only a result that exactly matches the signed content. The GGUF adapter validates the file structure, aligns packages from different publishers by tensor name and internal position, and divides large tensors into fixed 1-MiB chunks. Byte-identical chunks can be reused directly, aligned but different data uses residual reconstruction, and portions that cannot use equal-length reconstruction are obtained through compressed fetch. For each target chunk, the public catalog lists the permitted operations and binds each residual to its base package, target, transform, and data length.

4.2

Per-Chunk Reconstruction Selection

After verifying the target and aligning the model packages, EvoCache reads the precomputed reconstruction operations that the public catalog lists for each target chunk and selects the lowestcost one according to transfer volume, decoding time, and memory demand. Exact reuse and compressed fetch. If a target chunk is identical to local data, EvoCache uses exact reuse and reads the bytes locally without network transfer. If local data cannot be reused efficiently, the system uses compressed fetch and downloads the zstd-compressed target bytes from the publisher. Compressed fetch ensures that the system can always obtain the target when the other reconstruction actions do not reduce cost. XOR and BMS residuals. For chunks with equal length but different contents, EvoCache can transfer only the residual between the base package and target. Let 𝑏𝑖 be a base-package byte and 𝑡𝑖 the corresponding target byte. The publisher generates two basic residuals offline:

8(𝐵𝑎 + ℎ)/𝑟 + 𝑑𝑎 . Here 𝐵𝑎 is the action payload, ℎ is its framing overhead, 𝑟 is the network rate, and 𝑑𝑎 is the decode time. This cost model allows the planner to trade network transfer against local computation. Strict-release planning. When EvoCache reuses data created by another site’s activity, the final handle cannot be returned until the public deadline 𝐷 (𝑝). Finishing earlier therefore does not make the result available to the page sooner. The planner instead minimizes transferred bytes while ensuring that reconstruction finishes on time: ∑︁ min 𝐵𝑎 s.t. 𝑇b(𝜋) ≤ 𝐷 (𝑝) − 𝑡 0, 𝑀 (𝜋) ≤ 𝑀max . 𝜋

𝑎∈𝜋

𝑡𝑖 = (𝑏𝑖 + 𝑠𝑖 ) mod 256.

Here 𝑡 0 is the admission time, 𝜋 is the complete reconstruction plan, 𝑇b(𝜋) is its estimated completion time, including metadata processing, network transfer, round trips, decoding, and reserved local processing, and 𝑀 (𝜋) is its memory demand. The planner first chooses the lowest-traffic action for each chunk that satisfies the memory bound. If the complete plan cannot finish before 𝐷 (𝑝), it replaces selected actions with faster alternatives and can fetch the complete target when necessary. The internal choice of plan never changes the return time observed by the page.

Bytewise subtraction has previously been used in binary differencing, the Portable Network Graphics (PNG) Sub filter, and integermapped LLM deltas [24, 25, 28]. The key observation behind EvoCache is that independent publishers often produce only small numeric changes when they requantize the same LLM with different calibration data. BMS consequently produces a more compressible residual for many aligned chunks. Other chunks follow different

After selecting the reconstruction plan, the broker fetches data while reconstructing the target and verifies the complete file before delivering it to the page. Reconstruction uses a bounded stream and does not load the entire LLM file into memory. The broker reads only the required ranges of the base package, overlaps network

𝑥 𝑖 = 𝑏 𝑖 ⊕ 𝑡𝑖 ,

𝑠𝑖 = (𝑡𝑖 − 𝑏𝑖 ) mod 256.

Here 𝑥𝑖 is the XOR residual. Applying XOR again during decoding recovers the target byte. The value 𝑠𝑖 is the Bytewise Modular Subtraction (BMS) residual used by EvoCache for GGUF reconstruction. The decoder recovers the target byte as

4.3

4

Streaming Reconstruction and Delivery

EvoCache: Accelerating Browser Large Language Model Delivery with Public Cache Anchors

UNTRUSTED WEB ORIGIN

WWW ’27, May 10–14, 2027, Dublin, Ireland

TRUSTED BROWSER BROKER — BROWSER-OWNED PROCESS

PUBLISHER ENDPOINT / CDN manifest + catalog

1 install request root, manifestURL, publisherKey

hidden from script has() / list() base root, codec bytes transferred progress or errors cross-origin state

handle returned

2 admit, set D(p)

3 prepare release

4 index lookup

5 plan per chunk

regions + chunk table

admission reads only the caller's own quota, and runs before any cache lookup cached chunks already on the device cached bytes other origins 7 rebuild 16-MiB arena target file OPFS temp

signed manifest action catalog

inventory-independent 6 fetch ≤ 8 MiB

objects compressed chunks and residual objects content-addressed

reconstructed target, one bounded pass into an OPFS file

reconstruction remains broker-internal until the target is published

arrives at D(p) never earlier

8 release gate at D(p)

BMS residual

XOR residual

cache hit

held until D(p)

handle at D(p)

cache miss — full fetch + rebuild

schematic the physical work differs with the cache state; the release point does not exact reuse

compressed fetch

the server sees

the requested target requested object IDs

cross-origin state

Figure 1: EvoCache’s broker boundary and strict-release data path. The calling origin supplies public request metadata and receives a final handle only at the public deadline 𝐷 (𝑝). Eligible cached roots, the selected plan, byte counts, and cache-dependent progress remain inside the trusted broker and hidden from the calling origin.

5.1

transfer with reconstruction, and writes the result to a temporary file through synchronous OPFS access handles [33]. It verifies each chunk while writing, then rereads the complete file and verifies the final target against the publisher-authorized manifest. Only a file that matches exactly is committed and made available through the final handle. Public anchors are pinned and change only with browser updates, so their presence never depends on local storage pressure. The Rust/WebAssembly (Wasm) [10] kernel and download buffers use fixed memory, while per-site request limits, concurrentinstall limits, and global quotas control cache occupancy. We implemented a complete EvoCache prototype comprising a privileged Chromium extension, a Rust/WebAssembly reconstruction kernel, and publishing tools. Cache lookup, plan selection, reconstruction, and verification all execute inside the browser broker. Appendix B gives the planning budgets, resource limits, commit procedure, and prototype implementation details.

5

Experimental Setup

Datasets. The evaluation uses four datasets. The core dataset contains the 23 directed model-package pairs identified by the audit in Section 2.3, spanning 12 LLM-family/quantization groups. The independent validation set contains eight model packages covering two new LLMs and four publishers that were not involved in codec design. The 7B dataset consists of Qwen2.5-7BInstruct Q4_K_M model packages from three publishers, forming six directed pairs with a 4.36-GiB target file in each direction. The controlled-quantization dataset fixes the 16-bit floating-point weights of SmolLM2 and the llama.cpp version while changing only the calibration data or tensor-type policy [8]. The first three datasets use publicly released GGUF model packages from Hugging Face [13]. The end-to-end browser experiment selects one directed pair from each of the four LLM families in the core dataset and uses the same model-package sources. Comparators. The comparators fall into three groups. The first represents ordinary LLM delivery: a level-3 zstd full download [4] and browser-partitioned whole-file caching [16]. The second consists of generic binary-reuse methods: FastCDC [35], whole-file Xdelta, and tensor-aligned Xdelta [17, 18]. Tensor-aligned Xdelta receives the same tensor boundaries and complete equal-length base blocks as EvoCache, testing whether EvoCache’s advantage comes only from tensor alignment. The third group uses LLM structure: a ZipLLM/BitX-style bit shuffle [32], exact tensor reuse, fixed tensor XOR, and fixed BMS. The last three are internal EvoCache ablations that measure the contributions of exact reuse, XOR residuals, and BMS residuals, respectively. EvoCache core selects among

Performance Evaluation

The evaluation has two parts: offline evaluation and end-to-end browser experiments. The offline evaluation compares traffic savings across methods, analyzes why quantization residuals are compressible, tests the method on new publishers, new LLMs, and 7B LLMs, and measures reconstruction speed and resource overhead. The end-to-end browser experiments run the complete cross-site delivery workflow in Chromium and measure HTTP response-body bytes and page-ready time. 5

WWW ’27, May 10–14, 2027, Dublin, Ireland

Zhiyu Wang and Rajkumar Buyya

Table 2: Median traffic reduction on the 23 calibrationdivergent target pairs. Fixed-codec rows retain exact reuse and compressed fetch while fixing the delta transform. Method Whole-file cache / full zstd FastCDC, 1-MiB average FastCDC, 256-KiB average Tensor-reset FastCDC Whole-file Xdelta Tensor-aligned Xdelta Tensor chunks, exact Tensor chunks + BitX Tensor XOR (fixed codec) BMS (fixed codec) EvoCache core

of residual magnitudes are at most one, compared with 17.52% in XOR. On the median pair, the 90th percentiles of the two residualmagnitude distributions are 74 and 99, respectively. These results show that BMS preserves numerical proximity between quantized values. Figure 2 shows the operations selected by the latency-optimized policy. BMS handles most tensor data that has changed but remains aligned, XOR handles chunks better suited to bitwise comparison, identical chunks are reused locally, and only portions that cannot be reconstructed efficiently are fetched from the publisher. As bandwidth increases, the planner adjusts the share assigned to each operation, showing that operation selection considers both network transfer and local decoding cost. The strict-release policy, which minimizes transferred bytes, achieves the same or slightly higher traffic savings. Publisher sensitivity. To test whether the main result depends on any single publisher, the evaluation groups results by publisher combination and recomputes them after excluding each publisher in turn. EvoCache retains the best traffic reduction throughout these analyses. Appendix C reports the complete results.

Reduction (%) 0.0 23.8 23.9 24.0 30.1 49.4 23.9 68.6 75.4 76.4 77.3

exact reuse, compressed fetch, XOR, and BMS for each chunk; full EvoCache, evaluated in Section 5.5, additionally uses low-bit lane differences and field separation. Xdelta, XOR, and BMS may also use the same exact-reuse and level-3 zstd compressed-fetch operations, so their differences come from residual coding and operation selection. Environment and statistics. Both the offline evaluation and the end-to-end browser experiments use only CPUs. Appendix B gives the complete experimental environments, comparator parameters, statistical methods, and reproduction description.

5.2

5.3

Controlled Validation, Independent Validation, and 7B LLM Evaluation

Building on the core-dataset results in Section 5.2, this section uses controlled quantization to validate the relationship between calibration differences and residual savings, then evaluates EvoCache on new publishers, new LLMs, and larger LLMs. Controlled validation. The controlled experiment fixes the LLM weights and quantization toolchain and changes only the calibration method or tensor-type policy, allowing the effects of these two differences on reconstruction to be observed separately. Appendix D gives the complete model-package generation configuration and comparison directions. When only calibration changes, tensor types, sizes, and positions remain fixed; only the quantized values change. Table 3 shows that BMS produces smaller residuals than XOR in every calibration comparison, confirming that calibration differences create quantization changes suited to modular subtraction. When only the tensor-type policy changes, the storage format and length of some tensors also change. The planner fetches these tensors because equal-length residual coding cannot be applied, while exactly reusing tensors whose format is unchanged. The controlled experiment therefore validates the division of labor between the operations: BMS handles quantization changes that preserve layout, while fetch and exact reuse handle tensor-format changes. Independent validation. The independent validation set contains eight model packages covering two new LLMs (Qwen3-0.6B and TinyLlama-1.1B-Chat-v1.0) and four publishers that were not involved in codec design. The four Qwen3-0.6B directions contain large amounts of identical tensor data, yielding 99.52–99.57% exact reuse and 99.84–99.86% traffic savings with EvoCache. In the four TinyLlama-1.1B-Chat-v1.0 directions, nearly every chunk differs between publishers; EvoCache saves 97.8%, compared with 97.5% for fixed XOR and 95.0% for tensor-aligned Xdelta. These two groups validate EvoCache’s ability to reuse both identical tensor data and

Traffic Savings and Result Stability

EvoCache’s core planner achieves the greatest traffic reduction in Table 2, which reports network traffic saved relative to a level-3 zstd full download. The table includes the envelopes transmitted by the reconstruction operations; manifest and catalog traffic is accounted for separately. Every pair in the full core dataset saves at least 63.3%. Even when Xdelta receives the same tensor-alignment information, EvoCache wins on all 23 pairs and leads by at least 11.6 percentage points on every pair. Fixed BMS outperforms fixed XOR, showing that subtraction residuals further reduce traffic. EvoCache core selects a residual encoding for each chunk according to its contents and therefore obtains the best result in the table. Including manifests and catalogs reduces this result by only 0.3 percentage points. EvoCache outperforms fixed XOR on all 23 pairs and all 12 LLMfamily/quantization groups. When weighted by target-file size, EvoCache saves 76.2%, compared with 68.8% for the strongest external comparator (BitX-style bit shuffle) and 51.6% for tensor-aligned Xdelta. Appendix C reports the complete statistical results. Why BMS works. BMS gains its advantage from the distribution of residual values. XOR compares the bits of two bytes, whereas BMS computes the target byte minus the base byte modulo 256. When quantized values change only slightly, BMS residuals usually remain close to zero and are therefore easier to compress. On aligned tensor chunks whose contents differ, compressed BMS residuals are 8.89% smaller than XOR residuals, and BMS produces a smaller aggregate residual on all 23 pairs. For a nonzero residual byte 𝑣, define its circular magnitude as min(𝑣, 256 − 𝑣). In BMS, 31.59% 6

EvoCache: Accelerating Browser Large Language Model Delivery with Public Cache Anchors

BMS residual

XOR residual

WWW ’27, May 10–14, 2027, Dublin, Ireland

exact reuse

compressed fetch

SHARE OF TARGET BYTES

50 Mbit/s

65.4%

15.2%

16.1%

3.3%

300 Mbit/s

65.1%

16.3%

16.1%

2.5%

SHARE OF BYTES TRANSFERRED

50 Mbit/s

71.9%

15.8%

12.3% exact reuse: 0%

Figure 2: Operations selected by the latency-optimized policy across the 23 calibration-divergent pairs. The upper two bars show the share of target bytes handled by each operation at 50 and 300 Mbit/s. The lower bar shows each operation’s share of transmitted bytes at 50 Mbit/s. Exact reuse transfers no network data. Table 3: Controlled quantization with fixed weights and toolchain. Values are group medians over tensor bytes whose storage format and length remain unchanged, and BMS gain denotes residual reduction relative to XOR. Tensor policy

Calibration contrast

Mixed Mixed Pure Pure

None–calibrated Prose–code None–calibrated Prose–code

Directions

Bit flips

BMS gain

8 4 8 4

5.32% 3.15% 6.24% 3.91%

7.33% 7.08% 8.60% 9.19%

5.4

Reducing network traffic is useful in practice only if local reconstruction is fast enough and its memory and storage overheads remain controlled. This section evaluates EvoCache’s local reconstruction performance and its memory and storage overheads. Complete reconstruction performance. Across all 42 pairs, EvoCache achieves a median reconstruction throughput of 105.1 MiB/s. In repeated tests over six pairs covering all four LLM families, the median throughput is 196.2 MiB/s and maximum resident memory is 93.7 MiB. These results show that EvoCache can reconstruct LLM files as a stream without loading the complete file into memory. Codec overhead. With the same tensor alignment and equallength base blocks, the median decoding time for the residuals selected by EvoCache is 0.401 s, compared with 1.290 s for Xdelta, making EvoCache about 3.2 times as fast. Catalog and storage scaling. The 45 manifests and 42 catalogs occupy 28.55 MiB in total. Candidate residuals occupy 1.91 GiB, about 10% of the 18.4-GiB corpus; routing reconstruction through a shared hub package cuts this storage by 37.8% and adds only 2.9% median traffic. Appendix E reports the complete results.

Table 4: Traffic savings on six cross-publisher directions for 7B LLMs. Values are relative to a level-3 zstd full download. Method Exact tensor reuse Tensor-aligned Xdelta Fixed tensor XOR EvoCache core

Median (%)

Minimum (%)

16.0 43.9 73.1 76.6

9.5 43.9 73.1 76.6

Reconstruction Speed and Resource Overhead

data that changes in nearly every chunk, showing that its benefits extend to publishers and LLMs outside the core dataset. 7B LLM evaluation. The 7B experiment uses Qwen2.5-7BInstruct Q4_K_M model packages from three publishers. Each target file is 4.36 GiB, and the three model packages form six directed pairs. Table 4 shows that EvoCache outperforms tensor-aligned Xdelta and fixed tensor XOR in all six directions, with both its median and minimum traffic savings reaching 76.6%. The six directions form two groups. In the four directions that include importance-matrix calibration differences, EvoCache’s BMS residuals are 13.2% smaller than XOR residuals, and every direction saves 76.6% of traffic. In the other two directions, neither package records an importance matrix, so more tensor data can be reused exactly and the remaining changes produce only small residuals, raising savings to 99.9%. These results show that EvoCache handles both calibration-induced quantization changes and local differences between publisher packages at 7B scale. Appendix D reports the complete results for all six directions.

5.5

End-to-End Cross-Site Delivery

Purpose and setup. The preceding experiments measure traffic savings and local reconstruction performance separately. This section tests whether these advantages shorten page waiting time during complete browser delivery. The experiment selects one crosspublisher Q4_K_M pair from each of the four LLM families in the core dataset, with target sizes ranging from 100.6 to 770.3 MiB. Each pair runs five times under full EvoCache and each of the five comparator configurations described below, at both 50 Mbit/s and 100 Mbit/s with an 80-ms network round-trip time, producing 240 experiments. Each experiment starts with a public anchor already stored by the browser, namely the other publisher’s package of the same model and quantization. Because one browser-provisioned anchor serves every site that requests a matching target, its download is excluded from the measured traffic. Each experiment measures the complete time from the page submitting a target to receiving a 7

WWW ’27, May 10–14, 2027, Dublin, Ireland

Zhiyu Wang and Rajkumar Buyya

Table 5: Reductions of full EvoCache relative to five browser comparator configurations. Values are medians [minimum, maximum] across eight model-package-pair/bandwidth combinations.

Table 6: Browser delivery support for independently repackaged LLMs. ✓ means supported; – means unsupported.

Mechanism

Comparator configuration Ordinary partitioned full delivery Fixed tensor XOR Basic actions Basic actions plus low-bit lane differences Field-separated XOR

Page-ready time reduction (%)

HTTP response-body reduction (%)

64.2 [54.1, 77.5] 12.6 [4.1, 19.1] 9.2 [3.7, 14.7] 7.9 [4.8, 12.9]

78.9 [72.2, 86.5] 23.0 [15.6, 33.1] 18.5 [12.5, 23.2] 14.2 [12.3, 22.1]

9.6 [2.2, 15.5]

15.0 [9.2, 28.8]

HTTP delta encoding [21] Cross-Origin Storage (COS) [27] Cross-Origin Model Cache [29] Compression Dictionary Transport (CDT) [20] EvoCache

verified LLM, and counts all delivery data transferred during this process. Comparator configurations. The experiment compares full EvoCache with five configurations. Ordinary partitioned full delivery directly downloads the complete compressed target and measures the overall benefit of cross-publisher reuse. Fixed tensor XOR uses the same tensor alignment, exact reuse, and compressed fetch as EvoCache, but uses XOR for every residual; it tests the value of selecting different reconstruction operations for individual chunks. Three additional configurations analyze the quantizationspecific operations. Basic actions support exact reuse, compressed fetch, XOR, and BMS. Basic actions plus low-bit lane differences additionally process packed low-bit weights. Field-separated XOR separates scale values, subscale values, and weight codes, but uses XOR for every changed field. Full EvoCache may use all operations and selects the plan with the shortest estimated delivery time for each chunk or field. End-to-end results. As Table 5 shows, relative to ordinary partitioned full delivery, EvoCache reduces network traffic by 78.9% and page-ready time by 64.2%. Relative to fixed tensor XOR with the same tensor alignment, it further reduces network traffic by 23.0% and page-ready time by 12.6%. This shows that the traffic advantage in the offline evaluation translates into a real reduction in page waiting time. EvoCache also outperforms the three internal ablations. The results show that low-bit lane differences and field separation each reduce delivery cost further, while selecting different residuals for different fields outperforms using XOR throughout. Table 9 in Appendix F reports the complete results for each model-package-pair/bandwidth combination.

6

Cross-site reuse

Byte-different base

Browser selects and hides base

– ✓ ✓ –

✓ – – ✓

– – – –

✓

✓

✓

provisions one on-device model at the browser level [9], a precedent for browser-provisioned public anchors. EvoCache lets the page specify only a publisher-authorized target, while the browser independently selects a byte-different cross-publisher base package and hides that choice from the page. Table 6 summarizes this distinction. Reconstruction methods. LBFS and FastCDC use contentdefined chunking to find identical data [23, 35]; VCDIFF, Xdelta, and bsdiff generate deltas for related binary files [17, 18, 25]. LLMaware methods such as ZipNN, FM-Delta, QStore, and ZipLLM use weight or tensor structure but generally require a fixed representation, a known training relationship, matched precision, or a uniformly managed model collection [11, 24, 26, 32]. Independently published GGUF files may change metadata, calibration results, and tensor formats together and need not meet these assumptions. EvoCache targets this setting by using tensor structure and quantization layout to select exact reuse, compressed fetch, or residual reconstruction for each chunk or field inside the browser. Tables 10 and 11 in Appendix G give complete comparisons.

7

Conclusion and Future Work

Cross-publisher GGUF repackaging of the same LLM prevents whole-file cache reuse, but corresponding tensors retain substantial compressible information. EvoCache lets a page specify only an authorized target, while the browser internally selects a base package and a lossless reconstruction operation for each chunk. On the core dataset, EvoCache saves 77.3% of network traffic, exceeding Xdelta given the same tensor-alignment information (49.4%); it still saves 76.6% on 4.36-GiB 7B LLMs. Across 240 real end-to-end browser experiments, EvoCache reduces response-body bytes by 78.9% and page-ready time by 64.2% relative to ordinary full delivery. Results reconstructed from public anchors can be returned immediately, whereas history-related cache reuse follows a public deadline, thereby protecting cross-site cache privacy. Deployment requires little publisher effort: catalogs are static objects generated offline, and candidate residuals add about 10% storage that a hub package further reduces. Browsers can choose anchors from widely shared LLM families and bound anchor storage through browser provisioning policy. Although we evaluate GGUF, the same tensor alignment and per-chunk selection apply to other block-quantized formats. Future work includes popularity-driven anchor selection, cache-state-oblivious prefetching, private data retrieval, and multibase selection.

Related Work

EvoCache relates to two bodies of work: Web data delivery and cross-site reuse, and reconstruction of a target from an existing file. Web delivery mechanisms. Distributed Web caching studies object placement among servers [6], and Hugging Face’s Xet storage deduplicates model files at chunk level [14], which, like FastCDC, recovers little from calibration-divergent packages. On the client side, HTTP delta encoding transfers differences from a cached version, but the client must disclose that version to the server [21]. Cross-Origin Storage (COS) and Cross-Origin Model Cache allow sites to reuse data but require byte-identical files [27, 29]. Compression Dictionary Transport (CDT) can reconstruct a byte-different HTTP response, but the site must preselect the dictionary and can use it only within that site’s partition [20, 22]. Chrome’s built-in AI 8

EvoCache: Accelerating Browser Large Language Model Delivery with Public Cache Anchors

References

WWW ’27, May 10–14, 2027, Dublin, Ireland

USA, 66796–66825. [25] Colin Percival. 2003. An Automated Binary Security Update System for FreeBSD. In Proceedings of BSDCon. USENIX Association, Berkeley, CA, USA, 29–34. [26] Raunak Shah, Zhaoheng Li, and Yongjoo Park. 2025. QStore: Quantization-Aware Compressed Model Storage. Proceedings of the VLDB Endowment 19, 3 (2025), 388–398. [27] Thomas Steiner, Christian Liebel, and François Beaufort. 2026. Cross-Origin Storage. WICG Draft Community Group Report. https://wicg.github.io/crossorigin-storage/ [28] W3C Portable Network Graphics Working Group. 2025. Portable Network Graphics (PNG) Specification, Third Edition. W3C Recommendation. https: //www.w3.org/TR/2025/REC-png-3-20250624/ [29] W3C Web Machine Learning Community Group. 2025. Cross-Origin Model Cache. W3C Web Machine Learning Community Group Proposal. https://github. com/webmachinelearning/hybrid-ai/blob/main/proposals/cache.md [30] Qipeng Wang, Shiqi Jiang, Zhenpeng Chen, Xu Cao, Yuanchun Li, Aoyu Li, Yun Ma, Ting Cao, and Xuanzhe Liu. 2025. Anatomizing Deep Learning Inference in Web Browsers. ACM Transactions on Software Engineering and Methodology 34, 2, Article 47 (2025), 43 pages. [31] Yitong Wang, Pengfei Yu, Hao Han, Jingjing Gu, Fengyuan Xu, and Xinyi Huang. 2026. WAMO: Toward Secure Browser Inference via Web Model Obfuscation in WebAssembly. In Proceedings of the ACM Web Conference. Association for Computing Machinery, New York, NY, USA, 2812–2823. [32] Zirui Wang, Tingfeng Lan, Zhaoyuan Su, Juncheng Yang, and Yue Cheng. 2026. ZipLLM: Efficient LLM Storage via Model-Aware Synergistic Data Deduplication and Compression. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation. USENIX Association, Berkeley, CA, USA, 2371–2387. [33] WHATWG. 2026. File System Living Standard. WHATWG Living Standard. https://fs.spec.whatwg.org/ [34] Frank Wilcoxon. 1945. Individual Comparisons by Ranking Methods. Biometrics Bulletin 1, 6 (1945), 80–83. [35] Wen Xia, Yukun Zhou, Hong Jiang, Dan Feng, Yu Hua, Yuchong Hu, Qing Liu, and Yucheng Zhang. 2016. FastCDC: A Fast and Efficient Content-Defined Chunking Approach for Data Deduplication. In Proceedings of the USENIX Annual Technical Conference. USENIX Association, Berkeley, CA, USA, 101–114. [36] Siqi Zhong, Mugeng Liu, Haiyang Shen, Chongyang Pan, and Yun Ma. 2026. LaTune: Lightweight and Adaptive Configuration Tuning for LLM Inference on Edge Devices. In Proceedings of the ACM Web Conference. Association for Computing Machinery, New York, NY, USA, 5404–5414.

[1] Daniel J. Bernstein, Niels Duif, Tanja Lange, Peter Schwabe, and Bo-Yin Yang. 2011. High-Speed High-Security Signatures. In Proceedings of the International Workshop on Cryptographic Hardware and Embedded Systems. Springer, Berlin, Heidelberg, 124–142. [2] Zhiyang Chen, Yun Ma, Haiyang Shen, and Mugeng Liu. 2025. WeInfer: Unleashing the Power of WebGPU on LLM Inference in Web Browsers. In Proceedings of the ACM Web Conference. Association for Computing Machinery, New York, NY, USA, 4264–4273. Compression Dictionary Transport with [3] Chromium Project. 2026. Shared Brotli. Chromium Implementation Documentation. https: //chromium.googlesource.com/chromium/src/+/HEAD/services/network/ shared_dictionary/README.md [4] Yann Collet and Murray Kucherawy. 2021. Zstandard Compression and the ‘application/zstd’ Media Type. RFC 8878. Internet Engineering Task Force. [5] Bradley Efron and Robert J. Tibshirani. 1993. An Introduction to the Bootstrap. Chapman and Hall/CRC, New York, NY, USA. [6] S. Rasoul Etesami. 2024. Distributed Data Placement and Content Delivery in Web Caches with Non-Metric Access Costs. In Proceedings of the ACM Web Conference. Association for Computing Machinery, New York, NY, USA, 4340–4351. [7] Edward W. Felten and Michael A. Schneider. 2000. Timing Attacks on Web Privacy. In Proceedings of the ACM Conference on Computer and Communications Security. Association for Computing Machinery, New York, NY, USA, 25–32. [8] ggml contributors. 2026. GGUF Format Specification. ggml Project Documentation. https://github.com/ggml-org/ggml/blob/master/docs/gguf.md [9] Google Chrome. 2025. Get Started with Built-in AI. Chrome for Developers. https://developer.chrome.com/docs/ai/get-started [10] Andreas Haas, Andreas Rossberg, Derek L. Schuff, Ben L. Titzer, Michael Holman, Dan Gohman, Luke Wagner, Alon Zakai, and JF Bastien. 2017. Bringing the Web up to Speed with WebAssembly. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation. Association for Computing Machinery, New York, NY, USA, 185–200. [11] Moshik Hershcovitch, Andrew Wood, Leshem Choshen, Guy Girmonsky, Roy Leibovitz, Or Ozeri, Ilias Ennmouri, Michal Malka, Peter Chin, Swaminathan Sundararaman, and Danny Harnik. 2025. ZipNN: Lossless Compression for AI Models. In Proceedings of the IEEE International Conference on Cloud Computing. IEEE, Los Alamitos, CA, USA, 186–198. [12] Hugging Face. 2026. HfApi Client Reference. Hugging Face Documentation. https://huggingface.co/docs/huggingface_hub/package_reference/hf_api [13] Hugging Face. 2026. Hugging Face Hub. Hugging Face. https://huggingface.co/ [14] Hugging Face. 2026. Xet Storage. Hugging Face Documentation. https:// huggingface.co/docs/hub/xet [15] Fucheng Jia, Shiqi Jiang, Ting Cao, Wei Cui, Tianrui Xia, Xu Cao, Yuanchun Li, Qipeng Wang, Deyu Zhang, Ju Ren, Yunxin Liu, Lili Qiu, and Mao Yang. 2024. Empowering In-Browser Deep Learning Inference on Edge Devices with Just-inTime Kernel Optimizations. In Proceedings of the ACM International Conference on Mobile Systems, Applications and Services. Association for Computing Machinery, New York, NY, USA, 438–450. [16] Eiji Kitamura. 2020. Gaining Security and Privacy by Partitioning the Cache. Chrome for Developers. https://developer.chrome.com/blog/http-cachepartitioning [17] David Korn, Joshua MacDonald, Jeffrey Mogul, and Kiem-Phong Vo. 2002. The VCDIFF Generic Differencing and Compression Data Format. RFC 3284. Internet Engineering Task Force. [18] Joshua MacDonald. 2026. Xdelta: VCDIFF Delta Compression Tool. GitHub repository. https://github.com/jmacd/xdelta [19] Kiyoshi Masui, Mandana Amiri, Liam Connor, Meiling Deng, Mateus Fandino, Carolin Höfer, Mark Halpern, David Hanna, Adam D. Hincks, Gary Hinshaw, Juan Mena Parra, Laura B. Newburgh, J. Richard Shaw, and Keith Vanderlinde. 2015. A Compression Scheme for Radio Data in High Performance Computing. Astronomy and Computing 12 (2015), 181–190. [20] Patrick Meenan and Yoav Weiss. 2025. Compression Dictionary Transport. RFC 9842. Internet Engineering Task Force. [21] Jeffrey C. Mogul, Balachander Krishnamurthy, Fred Douglis, Anja Feldmann, Yaron Y. Goland, Arthur van Hoff, and Daniel M. Hellerstein. 2002. Delta Encoding in HTTP. RFC 3229. Internet Engineering Task Force. [22] Carlos Mateo Muñoz and Barry Pollard. 2025. Improving Google Search with Compression Dictionaries. Chrome for Developers. https://developer.chrome. com/blog/search-compression-dictionaries [23] Athicha Muthitacharoen, Benjie Chen, and David Mazières. 2001. A LowBandwidth Network File System. In Proceedings of the ACM Symposium on Operating Systems Principles. Association for Computing Machinery, New York, NY, USA, 174–187. [24] Wanyi Ning, Jingyu Wang, Qi Qi, Mengde Zhu, Haifeng Sun, Daixuan Cheng, Jianxin Liao, and Ce Zhang. 2024. FM-Delta: Lossless Compression for Storing Massive Fine-Tuned Foundation Models. In Proceedings of the Conference on Neural Information Processing Systems. Curran Associates, Inc., Red Hook, NY,

A

Conditions Required by Strict Release

Section 3.3’s conditional API equality requires: (1) admission, the release rule, and deadlines read only public request classes and caller-owned quota, never broker occupancy; (2) the shared store has an accounting domain independent of caller storage; (3) reconstruction requests and progress are absent from caller interception and Performance APIs; (4) both accepted paths complete by the deadline; and (5) neither path emits its handle or error before that time. Opaque handles are sampled independently of cache state and bound to the originating caller. Under these conditions, couple the two executions’ public admission, random handle generation, and release schedule. Their intermediate API sequences are empty, and their terminal values describe the same target at the same scheduled time, establishing equality in distribution of the API transcripts. Extending this equality to arbitrary page observations additionally requires cachestate-independent CPU, storage, and network service classes, which we leave to browser-level resource isolation.

B

Implementation and Evaluation Details

System implementation details. The EvoCache prototype comprises a Python reference implementation, a Rust/Wasm reconstruction kernel, a memory-bounded native Xdelta wrapper, and a privileged Chromium extension. In Chromium 151, the extension implements the browser broker using an offscreen document, a hidden worker, OPFS storage, and persistent model-package and 9

WWW ’27, May 10–14, 2027, Dublin, Ireland

Zhiyu Wang and Rajkumar Buyya

chunk indexes. Pages cannot inspect the internal indexes or specify the base package or reconstruction plan. The publisher generates the manifest, content-addressed chunks, and catalog offline and signs the manifest. Median packaging and catalog-generation times are 3.45 and 6.77 s, respectively. The generated static objects can be uploaded alongside ordinary LLM files, without online LLM execution or a specialized delivery server. Across 336 catalog and network combinations, the strict-release plan that prioritizes reducing transferred bytes satisfies the public time budget in every case. At 300 Mbit/s, its traffic reduction differs from that of the latency-optimized policy by 0.040 percentage points, with a maximum difference of 0.171 percentage points. The Rust/Wasm kernel uses a fixed 16-MiB memory region to hold one base-package chunk, the decoded residual, and the target chunk. Remote bundles do not exceed 8 MiB in either compressed or decoded form, and at most two decoded bundles are retained simultaneously. WebCrypto checks publisher authorization, the reconstructed result is written to a temporary OPFS file, and it is committed after verification. Per-site and global quotas jointly limit cache occupancy, and shared broker storage is accounted for separately. Experimental and reproduction details. The offline experiments, except the 7B evaluation, run on a Windows machine with an Intel Core i9-11900K processor and 32 GiB of memory. The 7B evaluation and the end-to-end browser experiments run on a Linux 6.8 virtual machine in Melbourne Research Cloud with eight AMD EPYC 7763 virtual CPUs and 32 GiB of memory. All codecs use only CPUs. Model-package versions, target files, tensor chunking, and comparator configurations were fixed before the full experiment and kept unchanged across the core dataset and independent validation set. Each directed pair constitutes one observation. The 95% confidence intervals use 10,000 cluster-bootstrap resamples over 12 LLM-family/quantization groups [5], and paired Wilcoxon tests compare EvoCache with fixed XOR [34]. FastCDC was tested with average chunk sizes of 256 KiB, 1 MiB, and 4 MiB and region-reset settings of 64 KiB, 256 KiB, and 1 MiB. Xdelta 3.2.0 uses its maximum compression level (-9) with DJW secondary compression; whole-file Xdelta uses a 16-MiB source window, the largest setting that keeps the smallest packages within a memory budget of twice the model size, and tensor-aligned Xdelta encodes each aligned block with a 4-MiB window. The end-to-end browser experiments use two independent HTTPS sites and one publisher server, with a fresh browser profile for every trial. The server provides all signed candidate objects but does not participate in selecting the base package or reconstruction plan. Across the complete process tree, peak resident memory is 1.42 GiB at the median and 1.49 GiB at maximum. We will release the EvoCache artifact after the review process. Every reconstructed result exactly matches the requested publisher model package.

C

LLM-family/quantization groups containing these pairs, with 𝑝 = 2.44 × 10−4 . The publisher bartowski appears in 20 of the 23 target pairs; Table 7 reports publisher-pair results, followed by leave-one-publisherout results. Table 7: Target-workload publisher-pair medians (traffic reduction, %), with one representative of each comparator group in Table 2; 𝑛 = 1 is descriptive. Publisher pair

𝑛

EvoCache

Fixed XOR

Aligned Xdelta

Tensor-reset FastCDC

bartowski–lmstudio bartowski–mradermacher bartowski–unsloth mradermacher–unsloth

1 11 8 3

82.4 77.6 73.4 73.1

81.1 75.4 72.3 70.4

58.7 49.4 51.5 48.3

36.4 27.5 20.3 3.0

Omitting bartowski/lmstudio/mradermacher/unsloth gives EvoCache medians of 73.1% (𝑛 = 3), 77.2% (𝑛 = 22), 77.3% (𝑛 = 9), and 78.1% (𝑛 = 12). In the bartowski-disjoint subset, EvoCache’s 73.1% savings alongside 3.0% for tensor-reset FastCDC demonstrate positional-delta reuse even with little exact chunk overlap.

D

Controlled-Experiment Configuration and Per-Direction 7B Results

Controlled-experiment configuration. Real model packages may differ simultaneously in calibration data, tensor types, and file layout, making it difficult to determine the effect of each difference on reconstruction directly. The controlled experiment fixes the 16bit floating-point weights of SmolLM2-135M and SmolLM2-360M and the llama.cpp b10816 toolchain, changing only the calibration method or tensor-type policy. For each LLM, it uses no importance matrix, a text-calibrated matrix, and a code-calibrated matrix under both pure-type and mixed-type policies, producing 12 model packages. Calibration processes 16 chunks of 128 tokens on the CPU. These model packages form 24 directions that change only calibration and 12 directions that change only the tensor-type policy. Table 8 gives the per-direction 7B results.

E

Catalog Scaling

For each LLM-family/quantization group, a public hub reduces pairwise catalogs from quadratic to linear edge count; a client reconstructs the hub and then the target. Across the 16 groups covering all 42 pairs, minimizing incident candidate bytes selects bartowski, mradermacher, and unsloth in seven, eight, and one groups. Edges fall from 42 to 29, candidate storage falls 37.8%, and thirteen two-hop routes add 2.9% median traffic. Summed measured local reconstruction times are 3.37 s median and 17.10 s maximum for two-hop routes, or 1.24× direct CPU time at the median. Two-hop reconstruction temporarily needs 0.54 GiB median and 2.46 GiB maximum additional disk space for the intermediate hub and the target; including the existing base, the maximum is 3.69 GiB. The replay uses observed directed edges as reverse proxies, preserving XOR payload sizes and estimating directed BMS residuals, metadata, and timing.

Statistical Results and Publisher-Pair Sensitivity Analysis

The 95% cluster-bootstrap confidence intervals are 66.5–78.8% for fixed XOR, 67.8–79.0% for fixed BMS, and 67.8–79.8% for EvoCache. EvoCache outperforms fixed XOR on all 23 pairs, with a pairedtest result of 𝑝 = 1.19 × 10−7 . Its median is also higher in all 12 10

EvoCache: Accelerating Browser Large Language Model Delivery with Public Cache Anchors

WWW ’27, May 10–14, 2027, Dublin, Ireland

Table 8: Direction-level traffic reduction (%) in the 7B evaluation. B/M/L denote bartowski, mradermacher, and lmstudiocommunity.

F

Direction

Exact

Aligned Xdelta

Fixed XOR

EvoCache

B→M M→B B→L L→B M→L L→M

9.538 9.538 15.982 15.979 26.979 26.980

43.924 43.910 43.936 43.923 99.861 99.861

73.068 73.081 73.077 73.091 99.896 99.896

76.614 76.626 76.623 76.635 99.899 99.899

Browser-Combination Results

Table 9: Medians for each setting after five repetitions. Page time runs from submission of the target request until the final handle is returned; savings are relative to fixed tensor XOR, and response-body reduction includes signed control metadata. Target

Mbit/s

Ordinary (s)

EvoCache (s)

XOR (s)

Page save

Body save

50 100 50 100 50 100 50 100

131.4234 69.0424 64.7999 34.1861 17.8674 9.7031 44.9995 24.1063

33.3618 24.3758 14.5626 10.8640 6.4882 4.4313 16.9995 11.0713

40.3131 28.3544 18.0101 12.2176 7.5712 4.6221 19.0816 12.0181

17.243% 14.032% 19.142% 11.079% 14.304% 4.128% 10.912% 7.878%

33.125% 33.123% 28.797% 28.745% 15.585% 15.585% 17.237% 17.237%

Llama-3.2-1B Llama-3.2-1B Qwen2.5-0.5B Qwen2.5-0.5B SmolLM2-135M SmolLM2-135M SmolLM2-360M SmolLM2-360M

G

Complete Related-Work Comparisons Table 10: Support for independently repackaged LLMs across browser delivery mechanisms.

Mechanism

Page request

Reusable data

HTTP delta encoding [21] HTTP response Cross-Origin Storage (COS) [27] Public object identifier Cross-Origin Model Cache [29] LLM identifier Compression Dictionary Transport HTTP response (CDT) [20] EvoCache Publisher-authorized target

Client-declared cached version Byte-identical public object Byte-identical complete LLM file Prearranged, site-partitioned dictionary Public anchor or eligible cross-site package

Cross-site reuse

Byte-different base

Base hidden from page

No Yes Yes No

Yes No No Yes

No No No No

Immediate Immediate Immediate Immediate

Yes

Yes

Yes

Public anchor: immediate; history-derived cache: public deadline

Release behavior

Table 11: File relationships and decision units supported by reconstruction methods. Method

Decision unit

Information used

Applicable source–target relation

Reconstruction method

LBFS [23]

Content-defined chunk

Byte content

FastCDC [35] Xdelta [18] bsdiff [25] ZipNN [11] FM-Delta [24] QStore [26] ZipLLM [32] EvoCache

Content-defined chunk Byte sequence Byte range Weight block Parameter tensor Quantization group Tensor Authenticated GGUF chunk or field

Byte content Generic byte information Generic byte information Tensor storage representation Numeric parameter type Precision correspondence Tensor identity and layout Tensor structure and quantization layout

Files retaining identical regions shifted by insertions Files containing identical chunks Related binary files Versions of the same software LLMs using one representation Pretrained and fine-tuned LLMs Matched high- and low-precision LLMs Related LLMs in a repository The same LLM independently repackaged by different publishers

Fast boundary selection VCDIFF copy/add operations [17] Suffix matching and byte differences Fixed lossless codec Integer-mapped parameter differences Quantization-conditioned residual Tensor deduplication and BitX coding Lowest-cost exact-reuse, compressed-fetch, or residual action

11

Exact-chunk lookup

Related documents

• CORE: COverage CAlibration and Evicted-Mass REdistribution for KV Cache#1186549
• Fast Forgiving Base64 Decoding with SIMD Instructions#1354975
• Fast Forgiving Base64 Decoding with SIMD Instructions#1318904
• Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs#1062720
• PeekKV: The First Content Token Already Knows Which Heads Matter#1388505