arXiv CSopen access

When Words Predict Workload

1

When Words Predict Workload

arXiv:2607.04951v1 [cs.DC] 6 Jul 2026

Anubhab Banerjee Nokia Germany E-mail: [email protected].

Abstract—Standard distributed schedulers for Large Language Model (LLM) inference rely on static token counts or rolling latency averages, making them susceptible to failures arising from statutorily constrained text or other linguistics properties. For example, on European Patent Office (EPO) claims, governed by the of Article 84 European Patent Convention (EPC), properties like rigidity make human and machine authorship are statistically indistinguishable. Resolving this ambiguity mid-flight forces the pipeline to dynamically expand into a heavy multi-model ensemble, triggering unpredictable KV-cache and weight-allocation spikes that saturate the VRAM ceiling of consumer-grade edge accelerators and cause severe out of memory (OOM) crashes and queue stalls. To prevent this hardware collapse, we propose a CPU-side Linguistic Resource Forecasting (LRF) gateway that extracts a 16-dimensional vector of text-structure features and processes them through an XGBoost predictor to forecast trapband membership. The resulting escalation probability (Pescalate ) is evaluated against a dynamic, closed-form routing threshold (τθ (t)), which is recomputed per request using real-time latency telemetry. Crucially, the gateway safely routes requests to either the local Qwen2.5-7B edge worker or a remote Binocularsstyle contrastive ensemble (Qwen2.5 7B + 32B) on an NVIDIA H100 before any edge GPU memory is unnecessarily allocated. In a 6,000-request live trial, the LRF gateway reduced the operational misroute fraction (Rmis ) to 0.087–0.095—an order of magnitude below the token-count baseline (0.849). Peak edge VRAM remained safely bounded at 4.82 GiB (out of 8 GiB) across a 27× variation in wide area network (WAN) conditions. The XGBoost predictor achieved a live-trial AUROC of 0.84, while the dynamic τθ (t) delivered an 8.2% relative reduction in misroutes compared to an equivalent static threshold. Index Terms—Distributed systems, edge–cloud routing, LLM inference, perplexity, linguistic feature extraction, XGBoost, GPU memory management, heterogeneous accelerators.

I. I NTRODUCTION RODUCTION serving stacks for LLMs (e.g., vLLM, Triton) predominantly manage computational resources through proxy heuristics like static token counts and rollinglatency averages. These heuristics are predicated on the assumption that the resource footprint of an LLM request is primarily a function of its length, rather than its semantic content. In open-domain natural language workloads, this assumption is statistically sound, as attention mechanisms and KV-cache growth scale linearly with sequence length, making “tokens” a reliable representation for “workload.”

P

This work has been submitted to the IEEE for possible publication. Personal use of this material is permitted. Permission from the author must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Copyright may be transferred without notice, after which this version may no longer be accessible.

However, this dependency on structural proxies creates problems in domains characterized by rigid, statutorilyconstrained text registers. EPO patent claims, for example, are legally compelled to follow narrow, prescriptive, rigid templates, strict antecedent-basis rules, and chained subordinate clauses. Our previous work [1] identified that these registers create a “perplexity trap”, where lightweight LLMs produce statistically indistinguishable likelihoods for human-authored and AI-rewritten text, causing even sophisticated detectors (e.g., Binoculars [2], DivScore [3]) to collapse. The perplexity trap is more of an imminent threat to operational stability than a failure of classification accuracy. In 2026-era of cascading-LLM stacks, the standard response to an ambiguous classification is to escalate the request to a heavier, cloud-based ensemble. Since the upstream scheduler routes based solely on token counts, it pays no attention to the claim’s internal structure. Consequently, when a claim falls into the trap band, the gateway invokes an expensive escalation mid-flight. This creates a sudden, unpredictable demand on memory and latency that saturates the consumergrade edge accelerators (e.g., NVIDIA GeForce GTX 1080), triggers OOM aborts, stalls the head-of-line queue, and forces a serial fallback to the cloud, where the system pays the full latency penalty of CUDA memory cleanup, state serialization, and remote re-transmission. The gap in current infrastructure sits exactly on the CPU side of the gateway, before the GPU memory is allocated. To close this gap, we present Linguistic Resource Forecasting (LRF), a sub-5 ms CPU pipeline that predicts hardware escalation probability Pescalate using a sixteen-dimensional vector of likelihood-orthogonal features. By coupling this predictor with a telemetry-driven, closed-form routing threshold τθ (t), our system resolves operational scenarios—from VRAM-imminent overflows to WAN partitions—by overriding the escalation logic before a physical crash occurs. A. Contributions Our contributions in this paper are three-fold: Linguistics-to-Hardware Telemetry. A deterministic CPUside mapping from sixteen static text-structure features to a downstream hardware-escalation probability Pescalate , computed in O(1) complexity. This feature space combines a baseline feature with seven repurposed signals from [1] and eight new EPC-register-aware features (e.g., boilerplate frequency, POS-bigram entropy). Closed-Form Dynamic Routing Threshold. A dynamic, telemetry-driven routing threshold that optimizes the balance between local edge processing and remote cloud offloading.

2

This threshold is governed by an operational override hierarchy, ensuring system safety and stability in edge cases—such as network partitions or hardware overloads—where standard latency-based decision logic would be insufficient. Crash-Free Heterogeneous Execution. A dual-mechanism VRAM safety interlock (closed-form pre-allocation gating + cooperative NVML-polling aborts) that maintains peak edge VRAM usage strictly within the 8 GB ceiling on a heterogeneous GTX 1080 + H100 cluster under sustained load. B. Relation to Prior Work This paper builds partly on the [1], which established the perplexity-trap phenomenon as a structural failure of likelihood-based AI-text detectors on EPO claims. While that work demonstrated the feasibility of detecting trap-band membership using a simple solution like logistic regression, the present work elevates the pipeline from offline detection to a live, distributed inference system. We upgrade the classifier from logistic regression to XGBoost, shift the target from semantic authorship to operational trap-band membership, and expand the topology from a single-machine bottleneck to a gateway-governed edge–cloud distributed architecture. C. Paper Organization The remainder of this paper is structured as follows: Section II situates our contribution against LLM serving and dynamic routing literature. Section III formalizes the perplexitytrap-to-hardware-collapse pipeline. Section IV details the LRF pipeline and the derivation of our routing threshold. Section V describes our heterogeneous corpus and workload harness. Section VI reports our experimental results, and Section VII concludes with a discussion of future applications.

a fixed validation set and applied as a static cutoff at inference time. None of them recomputes the threshold per request from live network and queue telemetry, and none of them recompute it from a closed-form equilibrium of expected latencies. The closed-form τθ (t) derived in §IV fills both gaps. C. AI-Text Detection and Likelihood-Based Classifiers The AI-text detection literature divides into two camps: zero-shot likelihood detectors that score a candidate text under a known scoring model and look for distributional anomalies (DetectGPT [9], Fast-DetectGPT [10], Binoculars [2], DivScore [3]), and supervised classifiers trained on labelled human/AI corpora. The previous paper [1] systematically benchmarked the zero-shot family on EPO patent claims and reported a structural failure mode, “the perplexity trap”, that survives detector substitution, scoring-head substitution, and LLM-family substitution. That paper’s resolution was diagnostic (a seven-feature off-axis logistic-regression detector that recovers signal where likelihood detectors fail); the present paper recasts that diagnosis as the operational input to a distributed-systems routing problem. The Binoculars [2] formulation remains our cloud-side authoritative classifier; the [1] Appendix B result that Binoculars at Falcon-7B failed catastrophically on EPO H04 (TP = 0, FP = 0, defaulting to “Human”) is an open empirical question at the Qwen2.5 7B + 32B scale used in this paper’s cloud tier (both Apache 2.0 and ungated, deliberately chosen to keep the full stack OSI-compatible open source), and the experimental design preserves the option to report either outcome (a successful escalation classifier or a failure that re-validates the hypothesis at frontier scale; see §V). D. Linguistic Register Analysis and Stylometry

II. BACKGROUND AND R ELATED W ORK A. LLM Serving and Scheduling Production LLM serving stacks of 2025–2026 are dominated by throughput-oriented architectures that batch independent requests and amortise GPU prefill cost. Representative systems include vLLM [4], NVIDIA Triton Inference Server [5], and the Orca distributed scheduler [6]. Each treats every request as a length-parameterised KV-cache budget problem and routes by token count (or a token-count-derived priority) for the edge-vs-cloud or replica-selection decision. The implicit assumption, that the per-token compute cost is content-invariant once Ntokens is fixed, is correct for opendomain workloads but breaks under cascading-ensemble escalation triggered by low-confidence verdicts on statutorily constrained text (this paper’s central observation).

Register-aware stylometry has a long history in computational linguistics. Patent-prosecution-specific work includes the M1–M6 markers identified in [1] §5.3 (rule-based regex anchors over EPC boilerplate phrases, claim-coverage symmetry, antecedent-basis violations). The present paper continues two of these as continuous features — F10 (EPC boilerplate frequency, lifted from M3) and F11 (boilerplate positional variance, lifted from M5) — and adds six new register-aware features (F3, F4, F12, F13, F14, F15) that were not in the AI4Law feature set. Broader stylometry literature (Type-Token Ratio, HapaxLegomena ratio, Flesch–Kincaid grade, dependency depth) supplies the remaining nine continuous features. We cite the canonical references for each feature at the point of definition in §IV; the citation list is short because every feature has at least a four-decade history of use in computational stylometry.

B. Edge–Cloud Cooperative Inference and Dynamic Routing A wave of edge–cloud cooperative inference systems (speculative decoding, draft-and-verify schemes, model cascades) frames the routing decision as a per-token confidence threshold problem (e.g. model cascades and frugality-driven escalation rules [7], speculative decoding with draft models [8]). The thresholds in these systems are typically learned offline against

E. What is Missing The four lineages above share a structural omission. The LLM-serving lineage routes on token count, agnostic to text register. The edge–cloud-cooperative lineage uses static thresholds, agnostic to live telemetry. The AI-text-detection lineage frames its output as a semantic verdict, agnostic to the

3

downstream hardware consequence. The stylometry lineage produces features, but does not connect them to hardwareallocation decisions. This paper closes the omission by composing the four lineages into a single CPU-side pipeline: stylometry produces sixteen text- structure features, an XGBoost predictor maps the features to a trap-band-membership probability, a closedform equilibrium threshold mixes that probability with live telemetry to produce a routing decision, and a two-mechanism safety interlock enforces the 8 GB edge ceiling as a contract rather than a hope. The composition is the contribution. III. P ROBLEM S TATEMENT A. The Statutory Constraint Under Article 84 of the EPC, inventors are forced to use a rigid, repetitive, and narrow syntactic language for their patents. A claim is formally a single sentence and is anchored by mandated boilerplate phrases (“characterized in”, “comprising at least one”, “wherein the”), enforces strict antecedent- basis between indefinite-introduction nouns (“a transceiver”) and their definite back-references (“the transceiver”), and chains subordinate clauses to more narrow scope. The result is a low-entropy text whose distributional structure is determined more by statute than by authorship. B. The “Perplexity Trap” Due to the highly constrained linguistic style of patent claims, lightweight edge models struggle to distinguish between human-authored and AI-generated text based on nexttoken likelihood alone. Previous work in [1] demonstrated this limitation across various detectors, scoring heads, and generative LLMs. Specifically, the Weitzman overlap coefficient (Ω) for per-token perplexity (PPL) distributions on the EPO H04 dataset ranges from 0.40 to 0.85. The most challenging generation techniques, such as standard zero-shot (Cat A) and iterative refinement (Cat D) of [1], produced overlaps exceeding 0.75 even when scored by GPT-2-medium. trap band [θmin , θmax ] = [4.046, 40.019]

human (n=960)

ai (n=973)

log-domain density

0.5 0.4 0.3 0.2 0.1 0.0 100

101 PPLedge (log scale)

102

Fig. 1. Density distributions of edge-computed perplexity for human and AIgenerated H04L claims

Figure 1 illustrates this overlap on our deployed edge stack, using a Qwen2.5-7B-Instruct (GGUF Q4 K M) model to score the H04L IID-test partition (for more details about the dataset, please refer to §V-B). The density distributions for human and AI claims overlap almost entirely across the calibrated trap band of [θmin , θmax ] = [4.046, 40.019]. Under

offline scoring, 82.8% of these claims (1601 out of 1933; 960 human, 973 AI) fall directly into this band, where the two classes become virtually indistinguishable. This is the primary reason for algorithmic failure: for the vast majority of H04L patent claims, the edge model’s perplexity scores land in a region of critical ambiguity, rendering the local evaluation insufficient. C. The Dynamic Graph Expansion When a patent text’s perplexity falls within the trap band, the edge node cannot issue a reliable decision. In modern cascading LLM architectures, the standard procedure for resolving this type of ambiguity is to dynamically escalate the request to a heavier, cloud-based ensemble mid-flight. While this dynamic escalation is the intended behavior of a cascading stack, it introduces a critical systems-level challenge: the mid-flight expansion occurs completely unannounced to the upstream scheduler. D. The Hardware Collapse Mid-flight ensemble escalation produces a sudden, unpredictable spike in VRAM consumption because the system must simultaneously hold the active scoring model’s KV-cache and load the newly invoked ensemble model’s weights. Our edge node operates on a consumer-grade NVIDIA GeForce GTX 1080 (Pascal sm 61, 8 GB VRAM, no tensor cores). For the quantized Qwen2.5-7B-Instruct (Q4 K M) model, the VRAM budget consists of three static components: ≈ 4.6 GB of model weights, ≈ 0.5 GB of llama.cpp runtime overhead, and the dynamic KV-cache Mkv (N ). As detailed in §IV-G, we compute this KV-cache conservatively using the dense query head count (hq = 28) rather than the model’s actual Grouped-Query Attention (GQA) count (hkv = 4) to maintain strict systems safety. This yields a pertoken footprint of Mkv (N ) ≈ 0.383 MiB × N . Figure 2 maps this closed-form mathematical envelope. The projected peak VRAM reaches ≈ 5.87 GiB at N = 2048 tokens and ≈ 6.63 GiB at N = 4096. Given the physical 8.0 GiB hardware ceiling and a 7.5 GiB safety margin, the gateway’s pre-allocation rule must refuse any request where N ≳ 6420. If an unannounced ensemble expansion bypasses this envelope, it immediately saturates the ceiling and trips a CUDA OOM abort. This failure stalls the head-of-line queue, forces a host process restart, and sequentially reissues the request to the cloud, incurring the severe Lstall penalty. Figure 3 complements this analytical envelope with highfrequency empirical NVML telemetry captured during a sustained burst stress test (Poisson arrivals at λ = 4 req/s). The unmitigated all-edge baseline (red trace) suffers catastrophic hardware collapse: the VRAM allocations form a destructive sawtooth pattern, repeatedly crashing into the absolute 8.0 GiB physical ceiling and triggering hard CUDA out-of-memory aborts (The secondary fragmented spike near t = 51 s visually captures the head-of-line queue stall, as the worker attempts to restart while the queue remains flooded). Conversely, when the LRF gateway and safety interlocks are engaged (blue trace), the system dynamically escalates marginal requests

4

runtime overhead projected peak

refuse N≈6420

in trap band

trap-band resident (jittered)

static weights KV cache

N=4096

8.0 7.5 7.0 6.5 6.0 5.5 5.0 4.5 4.0

N=2048

projected VRAM (GiB)

8.0 GiB ceiling 7.5 GiB safety margin

out of trap band

no

4000 5000 6000 7000 8000 input tokens N Fig. 2. Analytical VRAM envelope showing the edge GTX 1080 reaching its 7.5 GiB safety refusal threshold at N ≈ 6420 tokens

0

1000

2000

0

3000

100

200

300

Ntokens

edge VRAM used (GiB)

8.0 GiB physical ceiling 7.5 GiB Mechanism-B margin

all-edge (E9 stress) LRF (safety on)

trap-band resident (jittered)

to the cloud, cleanly intercepting the allocation spikes and maintaining edge stability well below the 7.5 GiB margin.

400

512

600

Fig. 4. Per-claim scatter of the token-count decision variable (Ntokens ) against the offline trap-band label. in trap band

8.0 7.5 7.0 6.5 6.0 5.5 5.0 4.5

Nthr candidates

yes

out of trap band

yes

no 0

1000

2000 3000 4000 edge compute time Llocal (ms, Phase-2 PPL pass)

5000

Fig. 5. Per-claim scatter of the edge compute time (Llocal ) against the offline trap-band label.

0

50

100 150 200 elapsed time (s) Fig. 3. Empirical edge VRAM trace under a sustained burst workload, contrasting the destructive OOM sawtooth of the all-edge policy against the stable, bounded allocations of the LRF-gated policy

long right tails (p95 of 2.48 s and 4.93 s, resp.). Since the distributions share the same domain, any vertical decision threshold on Llocal (edge computing time) will intersect both classes, rendering it useless as a predictive routing feature.

E. The Failure of Standard Schedulers

F. The Quantitative Gap and Research Question

Current schedulers typically rely on token count (Ntokens ) or a rolling mean of recent latencies, neither of which can distinguish a standard text from a patent claim which might cause a trap-band failure. Trap-band membership is caused by text structure, which is entirely different from those two metrics. Therefore, standard schedulers repeatedly route tasks to the edge cluster which triggers OOM aborts and degrades tail latency. Figure 4 plots the token-count baseline against the actual offline trap-band labels for the H04L IID-test partition (n = 1 933, with the binary y-axis jittered to reveal density). Since H04L claims are short (99.2% fall below 512 tokens), token count proves to be an ineffective decider for this workload. Moreover, any standard routing threshold, e.g., the evaluation grid candidates Nthr ∈ {512, 768, 1024, 1536, 2048}, defined in §V-D, simply routes everything to edge. Furthermore, even at the lowest candidate threshold (Nthr = 512), the inband fraction remains stagnant at 82.8%, with no operating point on the Ntokens axis that separates the two classes. This empirical failure justifies the need for the LRF gateway evaluated in §VI-C. Figure 5 demonstrates a similar failure for the latency-mean baseline, applying the same y-axis jittering. The class-conditional distributions heavily overlap. The median compute time for trap-band-resident claims is 596 ms, compared to 472 ms for out-of-band claims, and both exhibit

Patent offices, IP law firms, and SME R&D departments do not usually own cloud computing infrastructure of large scales, they generally use consumer-grade hardware. Since building and maintaining such infrastructure can be expensive, it is often found that they buy or rent cloud compute time whenever necessary [1]. This raises a big challenge: they cannot always use cloud-based deployments for every patent evaluation, otherwise it will be too expensive for them. So, from their perspective, they need a solution, which can be used already in consumer-grade hardware, and the cloud computation will only be used when necessary. On the other hand, the failure stages described above converge on the following: given a heterogeneous edge–cloud cluster, where the edge tier is a consumer-grade 8 GB accelerator and the cloud tier is a frontier-scale H100 ensemble, can a CPU-side gateway predict and prevent hardware collapse caused by trap-band-resident requests on highly structured text, while maintaining a perrequest CPU budget of < 5 ms, strictly respecting the 8 GB edge VRAM ceiling, and achieving a p99 tail latency near the theoretical network-bound minimum? The remainder of this paper addresses this question. We detail the gateway architecture (§IV), define the evaluation metrics (§V), and report the empirical results (§VI). Pilot Evidence: A 300-row pilot has been executed as a system wiring sanity-check and is detailed in §V-F. The

5

conclusive answers to the research question, including the full heterogeneous-mix workload, the open-loop Poisson sweep (λ ∈ {0.5, 1.0, 2.0, 4.0} req/s), and the baseline routing ablations, are presented in the final evaluation (§VI).

SpaCy Gateway (CPU)

KV-cache Estimator

Projected VRAM usage > 7.5 GB

B. The Sixteen-Feature LRF Vector The feature space is fixed at sixteen dimensions: one systems baseline (Ntokens ) and fifteen off-axis linguisticcomplexity features. Table II lists the vectors in index order; the order is canonical and matches the booster’s stored featurename list. Inherited features F1, F2, F5, F6, F7, F8, F9 lift the [1] F1–F7 set; the eight remaining off-axis features (F3, F4, F10–F15) are new for this paper. The extractor is single-threaded and stateless across requests. spaCy is configured to load the en_core_web_sm pipeline once at gateway boot, with NER, lemmatiser, and attribute-ruler disabled Configuration to meet the criteria of being lightweight with strict latency requirement. A pilot extraction over a 300-row sample of H04L IID claims completes

Qwen-7B (Observer)

Projected VRAM usage < 7.5 GB

Edge Worker (\PPLedge)

IV. P ROPOSED S OLUTION : T HE LRF G ATEWAY The LRF gateway sits in front of a heterogeneous two-tier cluster (Table I). When a patent claim arrives, the gateway executes the following pipeline on a single CPU thread, with a target budget of < 5 ms end-to-end: Pre-allocation safety check (Mechanism B §IV-G): A closed-form KV-cache size estimate confirms that the projected peak VRAM usage stays under 7.5 GB. If not, the request is routed the request directly to cloud. LRF feature extraction: Sixteen features (one systems baseline plus fifteen off-axis features) are extracted via a CPUbound spaCy pipeline. XGBoost inference: The feature vector is passed through a trained XGBoost hist-method booster configured for < 200 µs per-row latency on a single CPU core. The booster outputs the trap-band inclusion probability Pescalate ∈ [0, 1]. Closed-form threshold τroute (t): At the arrival of each request, the gateway captures four live telemetry metrics: Llocal (t), Lcloud , Lwan (t), and Lstall . These metrics feed into a closed-form equilibrium equation to calculate the baseline routing threshold. The output is then normalized to a scale of [0, 1]. To ensure system safety, this baseline is preempted by critical hardware conditions according to a strict override hierarchy (VRAM-imminent ≻ WAN-partition ≻ cold-start). Finally, the system handles cloud overloads by substituting the Lcloud metric upstream, which safely avoids adding complex branching logic to the core equation. Routing decision: If Pescalate > τ˜θ (t), the request is escalated to the cloud Binoculars ensemble; otherwise it is dispatched to the local edge worker. The edge worker computes PPLedge on the local GTX 1080 with cooperative VRAM-abort instrumentation (Mechanism A; §IV-G). The gateway then either returns a decision (when PPLedge ∈ / [θmin , θmax ]) or escalates to cloud (when PPLedge ∈ [θmin , θmax ]). This whole ensemble is depicted in Fig. 6.

Cloud GPU (H100)

Input (Raw UTF-8 claims)

KV-cache Estimator

A. System Architecture

Edge GPU (GTX 1080)

\PPLedge \in \Ltrap

Qwen-32B (Performer)

\PPLedge \notin \Ltrap

XGBoost (P_esc)

Terminal Verdict Terminal Verdict

Threshold Calculation (tau_route)

Routing Decision

\Pesc < \tilde\Tauroute(t) \Pesc > \tilde\Tauroute(t)

Fig. 6. End-to-end system architecture diagram

at 4.6 ms per row on the development host (a laptop CPU also feeding the GPU), while the deployment-grade gateway with a dedicated CPU core targets the documented < 5 ms envelope. C. XGBoost Predictor The predictor simply follows this equation: ( 1 if PPLedge (c) ∈ [θmin , θmax ] y(c) = 0 otherwise

(1)

The predictor estimates trap-band membership of the edgePPL for the deployed scoring model, not for human-vs-AI authorship. Under the operational target, a “false positive” is a wasted cloud escalation, not a false accusation of an author. Authorship classification is delivered downstream, either terminally at the edge (when PPLedge ∈ / [θmin , θmax ]) or at the cloud (when the Binoculars ensemble is invoked). Model configuration.: The XGBoost predictor is configured with a binary objective and a CPU-bound histogram tree method (hist). Model hyperparameters include a maximum depth of 6, a learning rate of 0.05, and a 0.8 ratio for both subsampling and column sampling. The model trains for a maximum of 500 boosting rounds, utilizing early stopping with a patience of 20 rounds on validation log-loss. To strictly prevent data leakage, training uses patents from five different patent families. During inference, the gateway calls Booster.inplace_predict on a pre-allocated NumPy buffer rather than constructing a new DMatrix per request; this optimization is critical for keeping single-row prediction latency below the 200 µs budget. D. Edge Perplexity Computation PPLedge For an input claim tokenised into X = (x1 , . . . , xN ) under the edge model’s vocabulary, the edge perplexity is the exponentiated mean negative log-likelihood: ! N 1 X PPLedge (X) = exp − log Pθ (xi | x<i ) . N i=1

(2)

We adopt the following five-step pipeline: (i) sub-word tokenisation, (ii) autoregressive forward pass, (iii) log-softmax in

6

TABLE I H ARDWARE ENVELOPE OF THE HETEROGENEOUS CLUSTER . Tier

Hardware

VRAM

Architecture

Role

Gateway Edge Cloud

Commodity x86 CPU + RAM NVIDIA GeForce GTX 1080 NVIDIA H100 (cross-region)

— 8 GB 80 GB HBM3

— Pascal sm 61 Hopper sm 90

LRF extraction, XGBoost inference, τθ recomputation Qwen2.5-7B-Instruct GGUF Q4 K M PPLedge scoring Binoculars-style Qwen2.5 7B + 32B contrastive-PPL ensemble (Apache 2.0)

TABLE II T HE SIXTEEN LRF FEATURES ( ONE SYSTEMS BASELINE PLUS FIFTEEN OFF - AXIS LINGUISTIC - COMPLEXITY FEATURES ) #

Feature

Source

Rationale

F0 F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 F11 F12 F13 F14 F15

Ntokens Type-Token Ratio Hapax-Legomena Ratio Functional Stop-Word Entropy POS Bigram Transition Entropy Mean Dependency Depth Subordinate-Clause Ratio Noun-Phrase Density Flesch–Kincaid Grade Sentence-Length Variance EPC Boilerplate Frequency Boilerplate Positional Variance Semicolon-to-Word Ratio Inter-Delimiter Length Variance Definite/Indefinite Noun Ratio Lexical Overlap Across Claims

Tiktoken / HF fast tokenizer Hash set over word tokens Word-frequency dict Shannon entropy over 50 stop-words spaCy POS tagger + bigram entropy spaCy parser, root-to-leaf BFS spaCy dependency tags spaCy noun chunks / words Syllable counter + Flesch formula Per-sentence word-count variance Regex over top-20 EPO/USPTO anchors Std. dev. of anchor token-index Character count Variance of semicolon-chunk lengths Article-tag counter Jaccard between first 25 % / last 75 %

Systems baseline; KV-cache linearity LLM-injected synonym variance vs. legal vocabulary reuse AI4Law’s strongest discriminator (|d| = 1.17) LLM-specific stop-word distribution skew Patent claim syntactic predictability Heavily-nested claim drafting depth Scope-narrowing clause chains Patent components as noun phrases LLM-driven jargon inflation LLM-style rhythmic cadence vs. legal uniformity Mandated anchor under-utilisation by LLMs Anchor-placement law violation by LLMs LLM normalisation of legal punctuation cadence Uneven LLM-generated claim-element lists Antecedent-basis drift in LLM rewrites Independent-claim vocabulary continuity

log-domain, (iv) Negative Log Likelihood (NLL) accumulation in float64 and (v) scalar exponentiation. Inputs longer than nctx = 4096 tokens are chunked with a 256-token rolling overlap, with per-chunk perplexities aggregated as a tokencount-weighted geometric mean. The edge engine is a llama-cpp-python wrapping the llama.cpp CUDA backend compiled for Pascal sm 61. At inference time the engine offloads all transformer layers to the GTX 1080 (n_gpu_layers = −1). The wrapper exposes a cooperative threading.Event abort flag that the inference loop checks between tokens; setting the flag from the VRAM watcher thread raises EdgeAbortError at the next token boundary (see §IV-G). Pilot measurement.: A 300-row class-balanced stratified sample of the H04L IID test partition (§V) was scored at 4.7 min of wall-clock time on the GTX 1080 after the CUDAenabled rebuild of llama-cpp-python (a CPU-only build of the same code completed the same claim at ≈ 5 tok/s versus ≈ 86 tok/s on the GPU, a ∼ 17× speedup). Class-conditional summary statistics are reported in §V-F. E. Weitzman α-Mass Trap-Band Calibration The perplexity trap band [θmin , θmax ] is derived offline through a five-step calibration procedure: Perplexity Computation: We compute the local perplexity PPLedge for every text sample in the training corpus (H04L samples consist of patent claims only, whereas G06F and A61K samples have patent abstracts to create a heterogeneous evaluation corpus). Density Estimation: We apply Gaussian kernel density estimation to model the probability distributions of the logperplexities for both human-authored and AI-generated texts.

Overlap Calculation: We compute the Weitzman overlap coefficient (Ω) by determining the intersection area of these two density curves. Band Extraction: We identify the narrowest contiguous log-perplexity interval that captures 90% (α = 0.90) of this total overlap area. Linear Conversion: The interval boundaries are scaled back to the linear domain to define the final thresholds θmin and θmax for request routing. A pilot run on a 300-sample subset yielded an overlap coefficient of Ω = 0.756, resulting in a calibrated trap band of θmin = 4.53 and θmax = 47.74. The at-scale deployment follows this exact procedure across the complete corpus. F. Closed-Form Dynamic Routing Threshold τθ (t) The routing threshold represents the closed-form equilibrium between two expected-latency equations under a Bernoulli escalation model: E[Ledge ] = (1 − Pescalate )Llocal (t) + Pescalate ·
Llocal (t) + Lstall + Lwan (t) + Lcloud , E[Lcloud ] = Lwan (t) + Lcloud .

(3) (4)

Equating these expectations and solving for the equilibrium escalation probability τθ (t) yields, after the Llocal terms cancel: τθ (t) =

Lwan (t) + Lcloud − Llocal (t) . Lstall + Lwan (t) + Lcloud

(5)

The operational threshold is normalized to τ˜θ (t) = min(1, max(0, τθ (t))). The router escalates the request to the cloud iff Pescalate > τ˜θ (t).

7

1) Operational Control Scenarios: Equation (5) remains valid only where all four telemetry quantities are finite and the cluster is stable. To maintain system safety across eight distinct operational scenarios (cold start, steady-state light queue, steady-state heavy queue, degraded WAN, WAN partition, VRAM-imminent override, cloud overload and joint stress), we implement a two-class override architecture. The first class consists of short-circuit overrides that completely bypass the closed-form equation, applying instead a deterministic constant for τ˜θ (t). These follow a strict priority hierarchy: VRAM-imminent (τ˜θ = 0) ≻ WAN-partition (0) (τ˜θ = 1 − 10−3 ) ≻ cold-start (τθ = 0.50) ≻ standard closedform evaluation. The second class is a telemetry-substitution override designed specifically to handle cloud overloads which sits outside the standard evaluation chain. Through passive RTTdecomposition, the gateway substitutes the live windowedmean Lcloud for its nominal static value upstream. As a result, the closed-form formula dynamically adapts its evaluation using the degraded Lcloud metric. 2) Per-request Update Cost: Recomputing τθ (t) incurs minimal computational overhead: four memory reads, one subtraction, two additions, one division, and one normalization. Empirically, this completes in < 100 µs on a commodity x86 CPU core, fitting comfortably within the gateway’s overarching < 5 ms latency budget. G. Hybrid VRAM Safety Interlock The 8 GB edge VRAM ceiling is strictly enforced by two cooperating mechanisms: a runtime polling watcher (Mechanism A) and a pre-allocation admission gate (Mechanism B). Mechanism A (Cooperative NVML polling): : To capture unpredictable memory spikes during execution, a background daemon thread runs continuously on the edge worker. Operating at a 50 ms cadence with a negligible 0.2 % CPU idle cost, the daemon polls the GPU memory state via NVML. If the utilized VRAM exceeds 95 %, the daemon atomically sets a shared abort flag. Since driver-level preemption is unavailable on the Pascal sm 61 architecture, the llama.cpp inference loop cooperatively checks this flag at every token boundary, raising an abort exception if tripped. To prevent rapid oscillation during the post-abort memory drain, the flag relies on a hysteretic clear threshold of 85 %. Each abort logs the full causal chain (completed tokens, VRAM state, subsequent route, and observed stall). Crucially, tripping this flag also broadcasts a VRAM-imminent override to the upstream τθ (t) pipeline, forcing all new requests to the cloud until the local edge VRAM falls back below the 85 % safe threshold. Mechanism B (Pre-allocation closed-form check): : Operating at the gateway prior to admission, Mechanism B computes the projected peak VRAM using the formula static weights + Mkv (N ) + runtime overhead. The KVcache size Mkv (N ) is calculated as 2 · B · N · llayers · hkv · dhead ·BPP. For the Qwen2.5-7B architecture, this uses l = 28, d = 128, batch size B = 1, and BPP = 2 (representing fp16 precision for keys and values in llama.cpp). Although Qwen2.5-7B utilizes GQA with hkv = 4, our safety check

intentionally applies the dense baseline of hkv = hq = 28. This conservative approach provides headroom to absorb second-order memory bloat caused by llama.cpp’s internal KV layout, cudaMalloc fragmentation, and runtime CUDA kernel scratch space on Pascal GPUs. This non-GQA constant yields a per-token KV footprint of approximately 0.383 MiB ×N , projecting a peak of ≈ 5.9 GiB at N = 2048 and ≈ 6.6 GiB at N = 4096. The router preemptively escalates any request to the cloud if this projected peak, plus a 0.5 GiB safety margin, exceeds the 8 GiB ceiling (which occurs at N ≈ 6 420 tokens). During live trials, the measured peak was Vpeak ≈ 4.82 GiB, confirming that while conservative, the bound does not trigger spurious cloud escalations. While Mechanism A acts as a runtime fail-safe against hardware-level memory fragmentation and overhead drift, Mechanism B intercepts mathematically unfeasible long claims at the gateway before they can consume bandwidth. V. E XPERIMENTAL M ETHODOLOGY A. Metric Framework To evaluate the system, we measure seven performance indicators categorized into three distinct domains as below. 1) ML Predictor Metrics (Gateway XGBoost): AUROC: The Area Under the Receiver Operating Characteristic curve for the XGBoost predictor. This tracks the model’s accuracy in identifying claims that fall into the ”trap band” (defined in §IV-C). Our performance target (operational green-zone) is an AUROC ≥ 0.85. This threshold acts as the operational gating requirement, ensuring the predictor possesses sufficient discriminatory power to balance edge-collapse avoidance (false negatives) against unnecessary WAN utilization (false positives). False Positive Rate (FPR) at TPR ≥ 0.80: The fraction of ”safe” claims that the gateway incorrectly escalates to the cloud when the system is tuned to capture at least 80% of true trap-band claims, quantifying the frequency of unnecessary escalations. Our performance target is an FPR < 0.25. This limit ensures that while the gateway aggressively intercepts at least 80% of hardware-threatening claims to maintain edge stability, it does not needlessly degrade into an all-cloud routing policy by wasting expensive H100 compute and WAN bandwidth on more than a quarter of safely edgeable requests. 2) Distributed-Systems Metrics: p99 tail latency: The endto-end RPC turnaround time, captured across the primary H04L trial (n = 6 000 requests §V-C) and remeasured per arrival-rate cell in the auxiliary λ-sweep harness (4 × 5 000 × 3 = 60 000 open-loop Poisson samples). Performance targets: ≤ 500 ms is the goal, whereas ≥ 2000 ms marks a legacy failure mode (see §VI-B for the target gating Table IV). trap Primary misrouting rate (Rmis ): Calculated as (LRF FP + LRF FN)/N , evaluated against the offlinemeasured trap-band membership of PPLedge as the ground truth. Performance target: < 0.15. lat Secondary misrouting rate (Rmis ): The fraction of requests where an offline oracle confirms the unchosen routing path would have been strictly faster. This is reported for analytical context without a binary pass/fail threshold.

8

Gateway extraction overhead: The total end-to-end CPU time required for LRF feature extraction, XGBoost inference, and τθ (t) recomputation. Performance target: < 5 ms per request. These targets act as practical engineering goals to govern system efficiency, providing baseline pass/fail criteria for our subsequent live-trial evaluations rather than universally mandated thresholds. 3) Hardware-Constraint Metrics.: Peak edge VRAM (Vpeak ): The maximum NVML-polled used_bytes on the GTX 1080 during the 6,000-request primary trial. (Note: The λ-sweep harness reports its own per-cell Vpeak , but Table IV utilizes the claim-level replay data). Performance targets: Vpeak < 7.5 GiB ensures operational safety, while Vpeak ≥ 8.0 GiB triggers a hard OOM failure. Cross-cluster WAN bandwidth: The total bytes transmitted from the gateway to the cloud over a sustained onehour run. Performance targets: < 50 MB h−1 indicates an effective filter, whereas > 500 MB h−1 suggests excessive cloud escalation. B. Corpus: Heterogeneous-Mix Design The dataset has a heterogeneous-mix to evaluate the predictor’s robustness across different International Patent Classifications (IPCs), generator families, and prompt categories. We utilize two data generation recipes: • Recipe A (Mixed-Generator): Categories A–E (from [1]) utilize a balanced 50/50 split of AI rewrites generated by Claude Opus 4.6 and Qwen 2.5-72B-Instruct. • Recipe B (Cross-Generator Probing): Categories A–E are exclusively generated by Claude Opus 4.6. A heldout Category F is generated entirely by Qwen 2.5-3BInstruct (a model family the classifier never observes during training). This recipe explicitly tests the predictor’s categorical cross-generator robustness. As detailed in Table III, the four-IPC main partition and the one-IPC long-tail diagnostic collectively yield 4,100 abstracts and 25,138 claims. AI-generated instances carry a categoryletter suffix (e.g., EP3533259B1F), ensuring deterministic traceability back to the human one. The dataset is split into training (70%), validation (15%), and IID-test (15%) partitions at the patent-family level using a fixed seed (seed = 42). The C07D IPC is strictly held out as an Out-Of-Distribution (OOD) test set, and F03D serves as a sparse-class diagnostic. C. Hardware Envelope and Workload The deployment cluster operates across a heterogeneous architecture. The gateway and edge worker are co-located onpremises in an EU-Central office premise (NVIDIA GeForce GTX 1080). The cloud worker operates on a single NVIDIA H100 80GB in a separate EU region, accessed via gRPC and TLS 1.3 over HTTP/2, utilizing gzip compression for payloads exceeding 4 KB. Cloud Kernel Selection: The cloud-side Qwen2.5 7B + 32B AWQ ensemble is executed using the pure-PyTorch AWQ_TORCH reference kernel, avoiding the Marlin (sm 80+)

JIT kernel to ensure strict environmental reproducibility. This deliberately inflates the cloud compute envelope (Lcloud ) by a factor of 2×–3× (≈ 80 ms versus ≈ 30 ms per claim on the 32B model). This conservative configuration ensures our evaluation is strictly pessimistic: by artificially inflating the penalty of every cloud escalation, the reported p99 tail latency represents a possible worst-case scenario. Workload Drivers: We evaluate the system using two distinct workload drivers: Arrival-Rate Variance Harness: Emits open-loop Poisson arrivals with inter-arrival times sampled from Exp(λ) across λ ∈ {0.5, 1.0, 2.0, 4.0} req/s. We generate 5,000 synthetic events per λ point across three variance seeds (42, 43, 44), totaling 60,000 RPCs per router configuration. This harness tests system stability under varying loads. Headline Claim-Level Replay: A deterministic, closedloop replay of the de-duplicated H04L claim pool. It evaluates 1,500 claims across four seeds (42, 43, 44, 45) for a total of 6,000 routed requests, completely isolating the threshold policy’s performance from arrival-distribution variance. Telemetry: The dynamic routing threshold τθ (t) consumes four atomically captured telemetry metrics: Llocal (t) via a queue-depth-scaled EWMA of recent edge completions; Lwan (t) from 1 Hz active gRPC probes fused with passive RTT decomposition; Lcloud from the cloud’s self-reported compute time; and Lstall derived from a forced CUDA OOM fault-injection test. Operating Point for the Headline Live Trial: The closedform τθ (t) supports two execution modes. To isolate the XGBoost predictor’s raw efficacy from the dynamic controller’s adjustments, the 6,000-request headline trial utilizes a frozenseed mode. The four latency arguments are pinned to static estimates: Llocal = 300 ms, Lwan = 80 ms, Lcloud = 1800 ms, and Lstall = 2000 ms. This yields a static operating equilibrium: 1580 Lwan + Lcloud − Llocal = = 0.407 τθ⋆ = Lstall + Lwan + Lcloud 3880 This isolates Pesc performance, allowing the AUROC and FPR@TPR≥ 0.80 metrics to reflect solely the classifier’s accuracy at a fixed decision boundary. Dynamic-τθ (t) Comparator Run: To evaluate the adaptive controller, a secondary 6,000-request trial enables the fully dynamic τθ (t) mode. Here, telemetry channels update continuously via an α = 0.2 EWMA. The gateway applies a cold-start prior of τθ = 0.50 until the warm-up thresholds (50 edge observations, 10 WAN probes) are met, after which τθ (t) drifts organically with live compute and network conditions. Comparing this artifact set against the frozen-seed trial allows us to quantify the exact benefit of time-evolution policy while holding the predictor, corpus, and cloud kernel perfectly constant. D. Baselines We evaluate the LRF gateway against two industry-standard comparator policies. Other theoretical baselines (such as allcloud routing, queue-aware least-loaded, or oracle routing) fall outside the operational scope of this hardware-constrained architecture.

9

TABLE III H ETEROGENEOUS - MIX CORPUS , PAIRING SYMMETRY IS ENFORCED AT THE ABSTRACT LEVEL ACROSS ALL IPC S Tier

IPC

Recipe

Notes

IID IID IID OOD long-tail

H04L G06F A61K C07D F03D

B A A-partial A++ minimal

Verbatim mixed + robustness pools Paired Claude+Qwen-72B Claude-only Claude-oversampled, mixed OOD Sparse-class diagnostic

Token-Count Router (Headline Baseline): Representing standard production practice, this scheduler routes to the edge if Ntokens < Nthr , and escalates to the cloud otherwise. We keep Nthr ∈ {512, 768, 1024, 1536, 2048} and report the headline result using the threshold that yields the lowest p99 latency, deliberately granting the baseline its most favorable tuning. All-Edge OOM-Reactive Ablation: This ablation routes every request to the edge. Upon encountering a CUDA OOM crash, the request is logged and serially re-submitted to the cloud, absorbing the full Lstall penalty. This baseline establishes the empirical severity of the hardware collapse. E. Ablation Schedule To isolate the contributions of the gateway’s subcomponents, we execute the following experiments: Feature-count ablation: LR-7 ( [1] linguistic features only) vs. XGBoost-7 (identical features, swapped classifier) vs. XGBoost-15 vs. XGBoost-16 (XGBoost-15 +Ntokens ). Classifier ablation: Logistic Regression vs. Random Forest vs. XGBoost across the full 16-feature set. Trap-band α sweep: Evaluated across {0.70, 0.80, 0.85, 0.90, 0.95}. KDE bandwidth sweep: Evaluated across {silverman, scott, 0.5 × silverman, 2 × silverman}. Trap-band calibration: Per-IPC calibration versus pooledcorpus calibration. Lstall sensitivity: Evaluated across {1 000, 2 000, 4 000, 8 000} ms. F. Pilot Run: End-to-End Integration Test Prior to the evaluation, a 300-row pilot was executed to validate the flow of the complete pipeline. This pilot does not serve as empirical proof for the primary research question (§III); rather, it confirms that the pipeline correctly emits and logs proper values across the heterogeneous architecture. Step 1: Edge Perplexity Pass: A stratified 300-row sample (150 human, 150 AI) from the H04L IID test partition was scored on the CUDA-enabled edge engine. The classconditional summary statistics (mean / median / standard deviation) are: Class

Mean

Median

Std. Dev.

Human (n = 150) AI (n = 150)

17.32 28.21

13.29 20.81

14.45 22.60

These results confirm the ”trap-inversion”: AI rewrites are systematically more surprising to the edge model than human ones. While raw perplexity separation is non-trivial, the

Humans

AI rewrites

500 abstracts + 6,613 claims 500 abstracts + 6,186 claims 280 abstracts + 0 claims 711 abstracts + 6,416 claims 9 abstracts + 0 claims

600 abstracts + 5,923 claims 500 abstracts + 0 claims 280 abstracts + 0 claims 711 abstracts + 0 claims 9 abstracts + 0 claims

overlap is wide, forcing the gateway to use orthogonal textstructure features. Step 2: Trap-Band Calibration: Calibrating the trap band at α = 0.90 on the 300-row sample produced a Weitzman overlap of Ω = 0.756 and a trap interval of (θmin , θmax ) = (4.53, 47.74). The medians of both classes fall squarely inside this trap band, confirming that a raw-perplexity threshold cannot resolve the ambiguity. Step 3 & 4: Feature Extraction and XGBoost Training: The 16-feature LRF vector was extracted for every row, yielding a mean extractor overhead of 4.6 ms per claim. An XGBoost predictor was then trained against the trapband target using 5 IPC cross-validation. The mean validation AUROC across the five folds was 0.814. Step 5: Live Trial Routing: A miniature trial routed 14 H04L claims through the complete operational pipeline (admission safety check → LRF extraction → XGBoost prediction → τθ (t) evaluation → edge/cloud dispatch → telemetry). A ”shadow-edge” mode was enabled to force GPU perplexity computation on every request, providing counterfactual ground truth for the misroute metrics. The integration metrics are reported below: Metric AUROC vs. trap-band FPR at TPR ≥ 0.80 p99 end-to-end latency trap Rmis Gateway overhead p99 Vpeak WAN egress

Value

Target Target

0.939 0.000 2189 ms 0.143 26.4 ms 5.33 GB 1.21 MB/h

≥ 0.85 < 0.25 < 500 ms < 0.15 < 5 ms < 7.5 GB < 50 MB/h

Pilot Limitations: Five of the seven metrics successfully meet the operational targets during this first integration pass. The two failures (p99 latency and gateway overhead) are known artifacts of the pilot configuration. First, the p99 latency is inflated because the shadow-edge mode effectively counts cloud requests twice (once for the remote ensemble and once for the local ground truth). Second, the gateway overhead p99 suffers from resource contention, as the CPU orchestrating the pipeline is simultaneously feeding the local GPU. Furthermore, this pilot is structurally optimistic because it calibrates the trap band on the same sample it scores. The full, at-scale evaluation reported in §VI resolves it by disabling shadow-edge mode for latency measurements, isolating the gateway process to a dedicated CPU core, and strictly enforcing the OOD boundaries across the full corpus.

10

trap band [θmin , θmax ]

human (n=4646)

XGBoost OOF (AUROC = 0.910) random

ai (n=4058)

TPR ≥ 0.80

FPR @ TPR=0.80: 0.155

1.0

0.5 0.4 0.3

0.8

0.2 0.1 0.0 101

PPLedge (log scale)

102

Fig. 7. Edge-side Qwen2.5-7B perplexity density on the pooled H04L training split (n = 8,704)

VI. E VALUATION We evaluate the system using two complete 6,000-request, multi-seed live trials: a controlled static-threshold (τθ⋆ ) trial and a self-adapting dynamic-threshold (τθ (t)) trial. Both configurations utilize the identical predictor (XGBoost trained on canonical LRF features), edge stack (Qwen2.5-7B-Instruct GGUF Q4 K M on a single GTX 1080), cloud stack (Qwen2.57B/32B Binoculars ensemble on an H100), and random seed schedule (42, 43, 44, 45). The only manipulated variable is the routing threshold policy. The trials comprise n = 6 000 requests, which comes to nunique = 1 924 distinct claims after first-observation deduplication for the threshold-free ML metrics. To establish a baseline, the token-count router (§V-D) is swept across Nthr ∈ {512, 768, 1024, 1536, 2048}, providing the standard comparator for the LRF measurements at Section §VI-C. Operational versus Aspirational Thresholds: Since the cloud deployment image relies on the PyTorch AWQ_TORCH backend rather than a highly optimized JIT kernel (§V-C), each cloud RPC incurs approximately 1.8 s of pure inference time. Consequently, the p99 ≤ 500 ms tail-latency target is structurally unreachable in this specific configuration. To maintain rigor, we classify the AUROC, FPR, misrouting rate (Rmis ), peak VRAM (Vpeak ), and WAN bandwidth as strict operational thresholds, while treating the p99 latency and gateway extraction overhead as descriptive metrics characterizing the current hardware limits. A. Predictor Characterization (Corpus Level) Prior to evaluating the dynamic controller, we isolate and characterize the standalone performance of the XGBoost trapband classifier. Figure 7 plots the canonical edge perplexity density over the IID training corpus, highlighting the calibrated trap band of [θmin , θmax ] = [4.046, 40.019]. Figure 8 presents the pooled five-fold cross-validation Receiver Operating Characteristic (ROC) curve for the classifier on the same training partition. (Note: §VI-B subsequently reports a deduplicated live-trial ROC, which acts as the deployment-level counterpart to this training-corpus curve). B. Static vs. Dynamic τθ : Ablation Table IV reports the seven target metrics alongside the route-fraction breakdown for both the static (τθ⋆ ) and dynamic

True Positive Rate

log-domain density

0.6

0.6 0.4 0.2 0.0

0.0

0.2

0.4 0.6 False Positive Rate

0.8

1.0

Fig. 8. Pooled five-fold out-of-fold ROC for the XGBoost trap-band classifier evaluated on the n = 8,704 training corpus

(τθ (t)) threshold policies. Both runs utilize the identical XGBoost predictor, corpus, cloud and edge stack; the threshold policy is the sole manipulated variable. The trial evaluates n = 6 000 pooled requests across four seeds ({42, 43, 44, 45} × 1 500 claims). For the threshold-free predictor metrics (AUROC and FPR@TPR≥ 0.80), the data is deduplicated by first-observation per claim_id (nunique = 1 924). Because changing the downstream routing policy cannot alter upstream threshold-free rankings, these predictor metrics are identical across both rows. The remaining five rows represent operational outcomes that respond directly to the controller’s policy. Furthermore, targets marked with a dagger († ) in Table IV represent aspirational goals under a highly optimized cloud scenario. Given the pure-inference overhead of the AWQ_TORCH kernel (§V-C), these specific targets are treated as descriptive limits rather than pass/fail gating criteria. Predictor Unchanged; Controller Measurable: Both configurations exhibit an AUROC of 0.840 and an FPR0.80 of 0.268. This isolation validates the two-run protocol established in §V-C. The reduction from the training-corpus performance (AUROC = 0.910; Fig. 8) to these live-trial values accurately reflects the deployment penalty: whereas training data is artificially balanced by authorship and IPC class, live requests are IID samples governed by lambda-distributed inter-arrival rates focused on a narrower IPC slice. Dynamic Threshold Reduces Operational Misroutes: The operational misroute fraction (Rmis ) drops from 0.0953 under the static policy to 0.0875 dynamically (572 → 525 misroutes). This represents an 8.2% relative reduction using the exact same predictor, corpus, and hardware stack. While both policies comfortably pass the ≤ 0.15 operational target, the dynamic formulation provides a measurable improvement due to the controller adapting to network telemetry. Note that Rmis strictly charges the controller for wasting WAN bandwidth on edge-executable cases,not for the correct routing low-confidence AI claims to the cloud.

11

TABLE IV H04L CLAIM - LEVEL LIVE TRIAL : STATIC τθ⋆ VERSUS DYNAMIC τθ (t) OVER n = 6 000 POOLED REQUESTS .

Static τθ⋆

Dynamic τθ (t)

Target

Verdict (s / d)

Class

0.840 0.268 0.0953 6 484 49.1 4.82 0.492

0.840 0.268 0.0875 6 532 51.8 4.82 0.503

≥ 0.85 ≤ 0.25 ≤ 0.15 ≤ 500† ≤ 5† < 7.5 ≤ 50

FAIL / FAIL FAIL / FAIL PASS / PASS FAIL / FAIL FAIL / FAIL PASS / PASS PASS / PASS

predictor predictor controller system system hardware hardware

589 (9.82%) 5 411 (90.18%) 572 4.01

393 (6.55%) 5 607 (93.45%) 525 4.09

— — — —

— — — —

accounting accounting accounting accounting

()

()

Metric AUROC FPR @ TPR ≥ 0.80 Rmis (operational misroutes) p99 end-to-end latency [ms] Gateway overhead p99 [ms] Vpeak [GB] WAN bandwidth [MB hr−1 ]

τθ̃ (t)

Edge route count (%) Cloud route count (%) Misroute count Trial wall-clock [h]

τθ̃ (t) (dynamic)

0.5 0.4 0.3 0.2 0.1 0.0

()

Llocal t

Lwan t

Lcloud t

ms

3000 2000 1000 0 1750

ms

1700 1650 1600 1550

0

1000

2000

3000 claim index

4000

5000

6000

Fig. 9. Dynamic τθ (t) trajectory under the EWMA-driven controller, settling steadily below the 0.407 static baseline.

Routing Fraction Shifts Toward Cloud: Under the dynamic policy, the edge routing fraction drops from 9.82% to 6.55%. The EWMA-tracked cloud latency (Lcloud (t)) converges to ≈ 1.8 s—the true computational cost of an AWQ_TORCH RPC. Figure 9 demonstrates that after a brief cold-start period (WAN warming in ∼ 10 s, edge warming after ∼ 50 requests), the dynamic threshold τθ (t) settles below the static reference of τθ⋆ = 0.407. This lower operational threshold enforces a stricter edge-feasibility test, actively shifting marginal requests to the cloud ensemble to preserve system stability. Latency, VRAM, and WAN Bandwidth Remain PolicyInvariant: The p99 end-to-end latency shifts marginally from 6 484 ms to 6 532 ms, remaining within expected trial variance. Figure 10 illustrates the per-route latency CDFs. The distributions are visually indistinguishable above the median; the dynamic policy simply truncates the slow tail of the

edge distribution by escalating those marginal requests to the cloud. Peak VRAM (Vpeak ) is bit-identical at 4.82 GiB across both runs because edge model weights are deterministic and routing decisions are executed before any GPU allocation. WAN bandwidth increases slightly (0.492 → 0.503 MB hr−1 ), directly proportional to the increased cloud-RPC volume (5 411 → 5 607). Aspirational Targets Remain Unmet: Both p99 end-toend latency (≈ 6.5 s) and gateway-overhead p99 (≈ 50 ms) fail their aspirational targets. The end-to-end tail is strictly bottlenecked by the pure inference cost of the AWQ_TORCH cloud kernel, while the CPU gateway overhead is bottlenecked by the XGBoost forward pass on unsorted feature batches. Neither metric is sensitive to the routing policy itself. The predictor metrics (AUROC = 0.840, FPR0.80 = 0.268) narrowly miss their operational targets, reflecting the honest ceiling of the current XGBoost configuration. Structural predictor mitigations—such as deeper trees and abstract-level cross-IPC training—are discussed in §VII. Overall Ranking: The dynamic τθ (t) controller demonstrates its value by yielding a lower misroute fraction with zero degradation to system safety limits. While the static τθ⋆ configuration successfully isolates and proves the predictor’s baseline quality, the dynamic EWMA-driven controller represents the superior, practical operating mode for live deployment. C. Token-Count Baseline Sweep The standard industry baseline, as defined in §V-D, routes requests strictly by input length: to the edge if Ntokens < Nthr , and to the cloud otherwise. To ensure a conservative comparison, we sweep the parameter Nthr ∈ {512, 768, 1024, 1536, 2048} across the identical four-seed H04L claim pool (n = 6 000 routed requests) used in §VI-B. We highlight the Nthr configuration that minimizes the baseline’s own p99 latency as the primary comparator. Result: Token-Count is Fast but Routes Blindly: Since H04L claims are structurally short, four of the five sweep thresholds (Nthr ≥ 768) fall entirely into an all-edge policy. Zero requests reach the cloud, meaning the routing decision is completely disconnected from the underlying trap-band signal. Only the Nthr = 512 row produces any non-trivial routing

12

cloud (n=5411)

1.0

edge (n=589)

overall p99 = 6484 ms

0.8

edge (n=393)

overall p99 = 6532 ms

0.8 empirical CDF

empirical CDF

cloud (n=5607)

1.0

0.6 0.4 0.2

0.6 0.4 0.2

0.0

103 end-to-end latency (ms, log scale)

0.0

104

103 end-to-end latency (ms, log scale)

(a) Static τθ⋆ = 0.407.

104

(b) Dynamic τθ (t).

Fig. 10. Per-route latency CDFs under the static (left) and dynamic (right) threshold policies. TABLE V T OKEN - COUNT ROUTER SWEEP ON THE H04L TRIAL . T HE Nthr = 512 THRESHOLD ( BOLDED ) MINIMIZES p99 LATENCY, SERVING AS THE CONSERVATIVE COMPARATOR .

Nthr

edge

cloud

edge %

Rmis

p99 [ms]

gw p99 [ms]

Vpeak [GB]

WAN [MB hr−1 ]

512 768 1024 1536 2048

5 979 6 000 6 000 6 000 6 000

21 0 0 0 0

99.65 100.00 100.00 100.00 100.00

0.8487 0.8462 0.8462 0.8462 0.8462

5 373.2 5 443.7 5 412.3 5 437.3 5 423.3

0.03 0.03 0.03 0.03 0.03

4.816 4.816 4.816 4.816 4.816

0.0441 0.0000 0.0000 0.0000 0.0000

sweep metric

5440

5410 5400

0.8475 0.8470

5390

0.8465 N_thr (tokens)

N_thr (tokens)

100.00 WAN bandwidth (MB/hr)

99.95 99.90 99.85 99.80 99.75 99.70 99.65

N_thr (tokens)

0.04

5.0

0.03

4.9

0.02

4.8 4.7

0.01

4.6

0.00 500 750 1000 1250 1500 1750 2000 N_thr (tokens)

0.0320 0.0315 0.0310 0.0305 0.0300 0.0295 0.0290 0.0285 0.0280

V_peak (GB)

5380

edge route fraction (%)

gateway overhead p99 (ms)

0.8480

5420 R_mis

p99 latency (ms)

5430

5370

headline Nthr = 512

0.8485

500 750 1000 1250 1500 1750 2000 N_thr (tokens)

500 750 1000 1250 1500 1750 2000 N_thr (tokens)

Fig. 11. Token-count sweep summary: p99 end-to-end latency (left axis) against operational misroute fraction Rmis (right axis).

structure, catching the long tail of the distribution by routing 21 out of 6 000 requests (0.35%) to the cloud. Figure 11 illustrates the severity of this failure. While the headline Nthr = 512 configuration yields the lowest p99 latency (5 373 ms), it incurs a massive operational misroute fraction of Rmis = 0.849. Thresholding by token length fails to produce a viable band of operating points; it either acts as a blunt all-edge filter or catches a statistically insignificant cloud tail, neither of which constitutes a functional routing strategy. Comparison Against the LRF Gateway: When compared against Table IV, the token-count baseline strictly trades correctness for latency. The LRF gateway incurs a ∆p99 premium of roughly 1.1 s (6 484–6 532 ms versus 5 373 ms) to evaluate the predictive features and execute the necessary cloud RPCs. In exchange for this latency cost, the LRF gateway cuts the misroute fraction by almost 10 times (dropping from 0.849 down to the 0.087–0.095 range).

The only metric where the baseline dominates is the gateway CPU overhead. The baseline evaluates a single integer comparison (0.03 ms), whereas the LRF gateway computes a 16dimensional XGBoost forward pass (49–52 ms). Both systems share an identical peak VRAM (Vpeak = 4.816 GB) because the model weights are deterministic and the routing divergence occurs upstream of GPU allocation. WAN bandwidth predictably scales with cloud escalation volume (0.044 MB hr−1 for the baseline versus ≈ 0.50 MB hr−1 for the LRF gateway). Summary: The LRF gateway outperforms the tokencount baseline by nearly a factor of 10 on Rmis —the only metric that actually measures routing correctness. The subsecond latency penalty and minor WAN/CPU overheads are the deliberate, acceptable costs of achieving system stability and preventing the edge hardware collapse outlined in §III-E. D. Cross-IPC Trap-Band Convergence (Abstract-Level Ablation) The static versus dynamic comparisons (§VI-B) rely on a single corpus: H04L claims. The trap band itself was calibrated entirely on a pooled H04L claim-level pass and reused verbatim for every routing decision. A critical deployment question follows: would this same calibrated band serve a different IPC, or a different text structure (e.g., abstracts), without recalibration? To answer this, we conduct a cross-IPC ablation. We first execute an edge perplexity pass across all canonical abstract rows for five IPCs: A61K, C07D, F03D, G06F, and H04L. We then derive a fresh trap band per IPC, holding the Weitzman overlap coefficient procedure constant at α = 0.90. Finally, we evaluate whether each abstract-level band falls within a ±5% relative tolerance of the H04L claim-level reference endpoints.

13

TABLE VI C OMPARISON OF PER -IPC ABSTRACT- LEVEL TRAP BANDS AGAINST THE ref = 4.046, θ ref = 40.019). H04L CLAIM - LEVEL REFERENCE (θmin max

IPC A61K G06F H04L C07D

nh 280 500 500 711

na 280 500 600 711

θmin 4.292 2.580 5.605 1.361

θmax 11.712 7.412 27.274 9.543

ω

in tol.?

0.650 0.527 0.724 0.516

no no no no

TABLE VII WAN- SHAPING ABLATION PER - REGIME AGGREGATES UNDER THE DYNAMIC -τθ (t) CONTROLLER (n = 500 H04L CLAIMS PER REGIME ). Regime healthy degraded heavy jitter severe loss

edge %

Lwan [ms]

τθmean

τθstd

p99 [ms]

Rmis

dur. [min]

7.6 10.4 13.4 12.8

219 1 670 3 156 6 000

0.251 0.439 0.552 0.500

0.109 0.056 0.047 0.000

6 087 7 003 13 790 5 715

0.108 0.114 0.114 0.118

21.9 32.4 46.3 15.6

healthy

5% tolerance vs ref.

H04L/claim reference band [4.05, 40.02]

A61K/abstract

τ̃ θ(t)

[2.58, 7.41]

H04L/abstract

[5.61, 27.27]

C07D/abstract

[1.36, 9.54]

2 × 100

3 × 1004 × 100

6 × 100

101 PPLedge (log scale)

heavy_jitter

severe_loss

200 300 Row index in trial

400

0.8

[4.29, 11.71]

G06F/abstract

degraded

1.0

0.6 0.4 0.2 0.0

2 × 101

3 × 1014 × 101

Fig. 12. Cross-IPC abstract-level trap bands plotted against the H04L claimlevel reference (dashed lines), demonstrating severe out-of-tolerance shifts across all domains.

0

100

500

Fig. 13. Per-regime τθ (t) trajectories demonstrating the controller’s adaptation to varying network conditions after a 10-request cold-start phase.

degrading the operational misroute fraction (Rmis ). Result: Zero of Four Per-IPC Bands Converge: Table VI and Figure 12 demonstrate that zero IPCs produce a trap band within the ±5% tolerance of the H04L claim reference. Crucially, this failure includes the same IPC measured at a different text granularity (H04L abstracts), which shifts θmin by +38.5% and θmax by −31.8%. Other domains exhibit even more drastic deviations: G06F shifts its endpoints by roughly −36% and −81%, respectively. The biomedical IPCs (A61K and C07D) collapse θmax to nearly a quarter of the reference value. The baseline Weitzman overlap coefficient (ω) also fluctuates significantly (0.516 for C07D up to 0.724 for H04L abstracts), proving that the distributional geometry between human and AI text is inherently IPC-dependent. Interpretation: These failures are driven by two compounding effects. First, the trap band is highly sensitive to text structure: a patent claim is a single, structurally rigid sentence governed by statutory constraints, whereas an abstract is a multi-sentence narrative. Their underlying perplexity distributions are fundamentally non-exchangeable. Second, the narrow linguistic register of pharmaceutical and chemistry abstracts (A61K, C07D) compresses the human perplexity distribution downward, pulling both trap band endpoints with it. Consequently, the single-band assumption that holds within the H04L claim distribution fails immediately when exposed to out-of-distribution text. Deployment Implications: Deploying the LRF gateway on a new corpus requires per-IPC and per-text-kind recalibration. The computational cost of this recalibration is marginal; the edge PPL pass requires a few hours on modern hardware, and deriving the actual band is sub-second. While the XGBoost predictor’s underlying 16-dimensional LRF feature space remains corpus-agnostic, the specific trap-band membership targets do not transfer. Reusing the H04L claim-level band on a corpus like C07D abstracts would systematically misclassify trap-band positives by a factor of 3 to 4, severely

E. WAN-Shaping Ablation (Controller Robustness) The static-versus-dynamic comparison in §VI-B operated over a low-latency loopback to the H100 pod’s port-forward. To strictly evaluate the dynamic-τθ (t) controller’s robustness, we must deliberately degrade the WAN channel to provide Lwan (t) with sufficient dynamic range, ensuring the controller’s adaptive behavior is not masked by predictor noise. We utilize the Linux kernel’s tc qdisc netem utility on the loopback interface, allowing ms-precise injection of delay, jitter, and packet loss without confounding external variables. We evaluate four regimes on a 500-claim H04L sample: healthy (80 ms delay, no jitter), degraded (800 ms delay, ±100 ms normal-distributed jitter), heavy jitter (1 500 ms delay, ±300 ms jitter, 2% random loss), and severe loss (6 000 ms delay, 50% random loss). The predictor, edge stack, cloud kernel, and seed schedule remain identical to §VI-B. Result: τθ (t) Tracks the WAN Cost as Designed: As illustrated in Figure 13, once the Lwan EWMA warms past the initial 10-request cold-start phase, the controller successfully separates the regimes. The mean τθ rises proportionally with the round-trip cost: 0.251 at Lwan = 219 ms (healthy), 0.439 at 1 670 ms (degraded), and 0.552 at 3 156 ms (heavy jitter). Note that the healthy trajectory settles well below the static trial’s reference of τθ⋆ = 0.407 as the empirical 219 ms RTT is much cheaper than the operator-seeded 1 800 ms baseline. The threshold drop in the severe loss regime (0.500) represents a degenerate operating point rather than a saturation effect. At 50% random loss, the gRPC RTT distribution is dominated by fast timeouts. Since netem produces isolated drops rather than a sustained partition, the gateway’s override heuristic does not trip. Consequently, the EWMA fails to converge, the standard deviation collapses to zero, and the system defaults to the cold-start prior of 0.5.

degraded

80 60 40 20 0

healt

hy

ded loss jitter degra heavy_ severe_

0.100 0.075 0.050 0.025 0.000

healt

hy

ded loss jitter degra heavy_ severe_

End-to-end p99 latency (ms)

healthy

100

heavy_jitter

12500 10000 7500 5000 2500 0 Gateway overhead p99 (ms)

Misroute fraction Rmis

Edge routing share (%)

14

healt

hy

severe_loss

ded loss jitter degra heavy_ severe_

within ±0.01 across extreme delay variances, adjusts the edge routing share monotonically to protect system throughput, and degrades safely to a mathematical prior during chaotic TCP timeout events. This completes the three-axis validation of the proposed architecture: it outperforms static routing internally (§VI-B), drastically reduces misroutes compared to industry baselines (§VI-C), and maintains stability under adverse network constraints.

50

VII. C ONCLUSION AND F UTURE W ORK

40

We presented a Linguistic Resource Forecasting (LRF) gateway that dynamically routes workload across a heterogeneous edge–cloud cluster before any GPU memory is allocated. The gateway extracts a 16-dimensional vector on a standard commodity CPU, predicts trap-band membership via an XGBoost classifier, and fuses the resulting escalation probability Pescalate with a closed-form, telemetry-driven routing threshold τθ (t) - which is recomputed per request using four latency metrics. To guarantee hardware safety, a dual-mechanism VRAM interlock, comprising a closed-form pre-allocation gate (Mechanism B) and a cooperative NVML watcher that preempts in-flight edge inferences at the next token boundary (Mechanism A), strictly limits the peak edge memory allocation within the physical limit.

30 20 10 0

healt

hy

ded loss jitter degra heavy_ severe_

Fig. 14. Four-panel summary of the shaped-WAN trials illustrating edge routing share, end-to-end p99 latency, misroute fraction (Rmis ), and gateway overhead.

Edge Fraction Rises Monotonically with Lwan : As cloud escalation becomes increasingly expensive, the controller enforces a stricter Pescalate routing threshold. The edge routing fraction rises accordingly from 7.6% (healthy) to 10.4% (degraded) to 13.4% (heavy jitter). The slight reversal at severe loss (12.8%) occurs because the threshold falls back to the 0.500 prior, capping the controller’s penalty for further WAN degradation. Rmis Remains Policy-Invariant: The operational misroute fraction (Rmis ) stays tightly bounded within [0.108, 0.118] across all four regimes. This demonstrates the controller’s core robustness capability: it maintains correctness QoS within statistical noise even when WAN costs fluctuate by a factor of 27× (219 ms to 6 000 ms). A purely static threshold would force either over-escalating during healthy periods or underescalating during outages, which The dynamic controller avoids. p99 Tail Reflects Network Reality: End-to-end p99 latency accurately tracks regime severity, scaling from 6 087 ms (healthy) to 13 790 ms (heavy jitter), as shown in the top-right panel of Figure 14. The anomalous drop to 5 715 ms under severe loss is a known artifact of extreme packet loss: the TCP stack collapses would-be slow cloud requests into fast gRPC timeouts, truncating the long tail of the distribution and heavily skewing successful completions toward the local edge path. Hardware Overheads Remain Network-Invariant: By design, the routing policy isolates edge hardware from network volatility. Peak VRAM (Vpeak ) is bit-identical at 4.816 GiB across all regimes because the static model weights define the envelope, and routing divergence occurs upstream of GPU allocation. Gateway overhead p99 strictly remains between 44.5 ms and 50.1 ms (Figure 14, bottom-right); this variance is driven entirely by the CPU’s XGBoost forward pass, independent of network conditions. Summary.: The dynamic-τθ (t) controller demonstrates highly adaptive behavior. It preserves the misroute fraction

A. Measured Outcomes The empirical evaluation on the H04L claim-level corpus (§VI) addresses the primary research question across three critical axes. Predictor: On the live-trial corpus (nunique = 1,924 claims), the deployed classifier yields an AUROC of 0.840 and an FPR0.80 of 0.268 (Table IV). While this narrowly misses the operational targets (≥ 0.85 and ≤ 0.25, respectively), it reflects the inherent modeling limits of the current classifier rather than a fundamental failure of the LRF feature space. The importance of this predictive routing is underscored by offline scoring: 82.8% of H04L IID-test claims fall into the trap band, an ambiguity that standard heuristics like token count or edge compute time fundamentally cannot resolve. Controller: The dynamic τθ (t) formulation reduces the operational misroute fraction Rmis from 0.0953 to 0.0875 (572 → 525 misroutes per 6 000 requests) with zero degradation in peak VRAM, WAN bandwidth, or p99 latency. Compared to the token-count baseline, the LRF gateway suppresses Rmis by almost 10 times (0.087–0.095 versus 0.849). This improvement incurs a ≈ 1.1 s premium on the p99 tail, an overhead justified by the cloud-RPC cost that the baseline entirely evades. Furthermore, under highly variable shapedWAN conditions encompassing a 27× range in effective round-trip delay (219 ms to 6 000 ms), the dynamic controller bounds Rmis fluctuations to within ±0.01 and successfully adapts τθ (t) monotonically with Lwan across all stable network regimes (Table VII). Hardware Safety: Peak VRAM (Vpeak ) remained at 4.82 GiB across all scenarios.This confirms that the conservative non-GQA VRAM projection (Figure 2) and the dual safety Mechanism A/B successfully prevented any spurious

15

cloud routing on the observed H04L workload. The empirical VRAM trace (Figure 3) demonstrates that both the all-edgelike baseline and the LRF-gated policies securely operated well below the 7.5 GiB safety margin. Cross-IPC Calibration: The abstract-level ablation (Table VI) validates the architectural hypothesis: the H04L claimlevel trap band does not natively transfer to other IPC domains or differing text structures. Zero of the four evaluated per-IPC abstract bands converged within a ±5% tolerance on either endpoint. Consequently, deploying the gateway on a novel IPC or text format strictly requires a one-time edge-perplexity recalibration pass. Once established, however, the underlying LRF feature extraction and XGBoost booster remain highly reusable simply by swapping in the newly derived trap band at inference time. B. Binoculars Cloud Outcome The remote Binoculars ensemble executed successfully across all escalations. Utilizing the AWQ_TORCH kernel on an NVIDIA H100, the ensemble completed RPCs with a compute latency of ≈ 1.8 s. The live trial’s massive cloud-route fraction (90% to 93%) highlights that escalations function as the primary operating path rather than an exception handling mechanism. Crucially, the Qwen2.5 architecture evades the classification failure previously reported in [1] for Falcon-7B on the EPO H04 corpus. Evaluating the Binoculars ensemble’s downstream classification accuracy on trap-band data, isolated from the upstream routing decision, remains an active area for subsequent cloud characterization. C. Future Work Several targeted engineering and scientific pathways remain open to extend this architecture. Engineering: (i) Port the spaCy-backed LRF to a highly optimized C++/pybind11 implementation. This aims to close the gap between the 5 ms design target and the current empirical gateway overhead p99 of ≈ 50 ms. (ii) Rebuild the cloud deployment utilizing the Marlin JIT kernel, which is expected to drop Lcloud from ≈ 1.8 s toward the hardwaresupported baseline of ≈ 30 ms - that would fundamentally reshape the aspirational p99 latency targets established in Table IV. (iii) Publish a comprehensive Docker reproducibility bundle, explicitly pinning GGUF weights, AWQ checkpoints, and calibration manifests to ensure independent verification of all reported metrics directly from artifact hashes. Science: Train and evaluate abstract-level or per-IPC classifiers against the recalibrated trap bands identified in Table VI. This will quantify the extent to which the live-trial AUROC degradation (0.910 → 0.840) is driven by IPC-slice mismatch v/s broader deployment distribution shifts. Closing Remark: The perplexity trap observed on patent text is not a calibration artifact - it exposes a structural collision between the legally mandated, low-entropy linguistic register of patent claims and the superficial heuristic assumptions embedded in standard LLM schedulers. The LRF gateway successfully demonstrates that combining a CPU-side linguistic forecast, a telemetry-driven dynamic threshold, and

a strict VRAM interlock can safely confine a consumer-grade edge accelerator within its physical limits. In doing so, it suppresses operational misroutes by a full order of magnitude compared to the static heuristics predominantly shipped by the industry today. R EFERENCES [1] A. Banerjee, “Perplexity trap: When patent law makes human writing look like ai,” June 2026, accepted at the ICML 2026 Workshop on AI for Law (AI4Law). Preprint available: https://anubhabbanerjee.github.io/ preprints/main paper.pdf. [2] A. Hans, A. Schwarzschild, V. Cherepanova, H. Kazemi, A. Saha, M. Goldblum, J. Geiping, and T. Goldstein, “Spotting llms with binoculars: Zero-shot detection of machine-generated text,” in Proceedings of the 41st International Conference on Machine Learning, 2024, pp. 17 519–17 537. [3] Z. Chen, K. He, Y. Huang, Y. Zhu, and M. Feng, “Divscore: Zero-shot detection of llm-generated text in specialized domains,” in Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025, pp. 19 242–19 264. [4] W. e. a. Kwon, “Efficient memory management for large language model serving with PagedAttention,” in Proceedings of the 29th Symposium on Operating Systems Principles (SOSP ’23). ACM, 2023, pp. 611–626. [5] NVIDIA Corporation, “Triton inference server: An optimized cloud and edge inferencing solution,” Open-source software, 2024, https://github. com/triton-inference-server/server. [6] G. e. a. Yu, “ORCA: A distributed serving system for Transformer-based generative models,” in 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’22). USENIX Association, 2022, pp. 521–538. [7] L. Chen, M. Zaharia, and J. Zou, “FrugalGPT: How to use large language models while reducing cost and improving performance,” arXiv preprint arXiv:2305.05176, 2023, https://arxiv.org/abs/2305.05176. [8] Y. Leviathan, M. Kalman, and Y. Matias, “Fast inference from transformers via speculative decoding,” in Proceedings of the 40th International Conference on Machine Learning (ICML ’23). PMLR, 2023, pp. 19 274–19 286. [9] E. Mitchell, Y. Lee, A. Khazatsky, C. D. Manning, and C. Finn, “Detectgpt: Zero-shot machine-generated text detection using probability curvature,” in International conference on machine learning. PMLR, 2023, pp. 24 950–24 962. [10] G. Bao, Y. Zhao, Z. Teng, L. Yang, and Y. Zhang, “Fast-detectgpt: Efficient zero-shot detection of machine-generated text via conditional probability curvature,” arXiv preprint arXiv:2310.05130, 2023.