Decoupling Inference from State Updates in Low-Latency Feature Engines via Probabilistic Thinning

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

Decoupling Inference from State Updates in Low-Latency Feature Engines via Probabilistic Thinning

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

databases datamanagement sql storage

databases, sql, data management, storage

arXiv:2606.16981v1 [cs.DB] 15 Jun 2026

Decoupling Inference from State Updates in Low-Latency Feature Engines via Probabilistic Thinning Augusto Peres

Iker Perez

Pedro Valdeira

Guilherme Jardim

Feedzai Lisbon, Portugal [email protected]

Feedzai London, UK [email protected]

Feedzai Lisbon, Portugal [email protected]

Ana Sofia Gomes

Hugo Ferreira

Pedro Bizarro

Feedzai Lisbon, Portugal [email protected]

Abstract Streaming data systems increasingly underpin Machine Learning workflows that maintain large numbers of continuously updated aggregations. In production settings, each incoming event typically triggers read–modify–write operations to persistent storage, making high-frequency state updates a dominant source of latency, contention, and operational cost. In this work, we decouple inference from state persistence in streaming Machine Learning pipelines via probabilistic thinning: every event is scored, but durable state updates are selectively triggered by informative events. Unlike approaches that shed input or state, we show that persistence-path control is achievable without a high-frequency in-memory control plane or cross-worker coordination, relying exclusively on approximate statistics retrieved from disk-backed key-value stores. We model the resulting stochastic processes, derive bounds on filtering rates, and prove that common time-based aggregations remain unbiased under variance-aware formulations, preventing systemic error accumulation. We evaluate the approach in a controlled setting that isolates per-event costs, demonstrating substantial reductions in storage Input/Output and serialization overhead. Across experiments, up to 90% of events are excluded from the persistence path while preserving and in some cases improving downstream utility.

Keywords Stream processing, probabilistic state management, approximate computation, event filtering, machine learning systems, key–value stores, variance reduction

Introduction

Streaming Machine Learning (ML) workflows underpin a wide range of critical applications, including fraud detection, recommender systems, and cybersecurity. In these settings, decisions must be made continuously over high-volume event streams under strict latency constraints. A common architectural pattern is to maintain features or profiles: temporal aggregations such as counts, sums, or averages computed per uniquely identifiable entities in the data, including payment cards, users, or IP addresses [19, 42, 64, 69, 70]. However, maintaining feature profiles at scale incurs substantial operational costs. In modern streaming architectures, each incoming event typically triggers a read–modify–write (RMW) interaction with persistent state: existing aggregates are retrieved, updated,

and written back to storage. As event rates increase, the tight coupling between event processing and state persistence becomes a dominant source of latency, contention, and infrastructure cost [35]. To manage costs, modern production systems rely on a combination of distributed stream processing and approximate computation. Streaming platforms such as Kafka Streams [38], Apache Flink [12], or Spark Structured Streaming [6] partition event streams across workers and maintain per-entity state locally. At the same time, approximation techniques, including sketches, recursive estimators, or sampling, are widely used to reduce memory footprint or per-update computation [16, 18, 31]. While effective, these approaches largely treat state updates as unavoidable side effects of event processing. In domains where discarding or deferring events is unacceptable, such as fraud detection, every event triggers a persistence update, leaving state maintenance as a fundamental scalability bottleneck. In this work, we show that this limitation is not fundamental. A key observation is that streaming ML workloads commonly exhibit pronounced data skew: a small fraction of entities account for a disproportionate share of events. While this is traditionally viewed as a systems challenge, we show that it enables selective suppression of persistence operations without sacrificing inference quality. We introduce persistence-path control, a probabilistic thinning and state management mechanism that decouples inference from persistence. Every incoming event is evaluated by the ML model, but only a subset triggers updates to feature profiles. In contrast to classical thinning or load-shedding based on external control or input-driven policies, inclusion decisions are driven exclusively by disk-backed approximate statistics, governed by a self-correcting stochastic process that preserves unbiasedness and provides explicit variance control. While prior work explores utility-driven shedding over intermediate state, we instead enable stateless orchestration embedded in the persistence path, eliminating high-frequency in-memory counters, coordination, or auxiliary control. The design targets a broad class of stateful, decayed and recursive per-entity aggregations, while preserving unbiased estimates, enabling variance reduction, and dramatically reducing the frequency of expensive write operations. Figure 1 previews the resulting trade-offs: large reductions in storage Input/Output (I/O) and serialization overhead translate into higher throughput and, remarkably, improved downstream ML performance due to an implicit regularization effect under highly skewed workloads.

Peres et al.

Recall improvment at processed volume

−10

Proposed filtering without control state Proposed filtering with variance reduction Filtering with in-memory control state Unfiltered Baseline

−20 0.0

0.2 0.4 0.6 Average processed volume

Throughput (Events/second)

Recall improvment (%)

Throughput at processed volume Filtered Unfiltered baseline

600 500 400 300

0.2

0.8

0.4 0.6 0.8 Average processed volume

1.0

Figure 1: Performance benefits of persistence-path control on real-world data. Left: ML utility vs. filtered data volume for a binary classification task. Selectively discarding events from persistence path operations, but still scoring every event, preserves or improves predictive performance. Right: System throughput vs. persistence-update frequency. Reducing the fraction of events that trigger costly state updates enables a substantial increase in end-to-end system capacity. Statement of Contributions. This paper demonstrates how probabilistic thinning can be designed and operationalized to deliver scalable streaming ML systems. Our main contributions are: • We identify the tight coupling between event processing and persistent state updates as a primary scalability bottleneck in modern streaming ML architectures; see Section 3. • We formulate persistence-path control as a stateless orchestration mechanism for thinning without local state or cross-worker coordination; see Section 4. • We model the resulting process and derive analytical bounds on filtering rates, showing unbiasedness and variance control for common aggregations; see Section 4. Finally, in Sections 5 and 6 we discuss operating assumptions, and evaluate the approach in controlled settings that isolate per-event persistence costs and avoid confounding runtime effects. Our results demonstrate reductions of up to 90% in storage I/O, serialization and replication traffic, as well as 45% reduction in overall system utilization, while preserving and in several cases improving downstream ML utility across domains.

Background

Streaming ML systems are designed to make predictions or decisions over continuous event flows under strict latency and reliability constraints. In domains such as fraud detection (a recurring example throughout this work), recommender systems, and cybersecurity, Service Level Agreements routinely require millisecond-level response times at extreme percentiles (e.g., 250 ms at the 99.9th) under sustained high throughput [5, 29, 52]. Meeting these requirements at scale places streaming ML at the intersection of distributed systems engineering and approximate computation.

2.1

State Management in Stream Processing

To sustain workloads processing millions of events per second, modern platforms such as Kafka Streams [38], Apache Flink [12],

Spark Structured Streaming [6], and related systems [5, 29, 45, 52] rely on distributed architectures that partition workloads across independent workers. Each worker maintains persistent local state in embedded key–value (KV) stores, commonly backed by LogStructured Merge Trees (LSM-Trees) [46] (e.g., RocksDB [47]). For every incoming event, a worker must deserialize historical state, apply an update, and serialize the result back to storage. While this architecture enables horizontal scalability and minimizes coordination, it incurs substantial overheads associated with the RMW execution model: • State amplification: LSM-Trees suffer from write amplification during compactions and read amplification from multi-level lookups and Bloom filter checks [22, 46]. • Serialization overhead: In JVM-based systems [43], serializing state into bytes for storage and deserializing it for updates often dominates CPU time, creating a pronounced serialization–deserialization (SerDe) bottleneck [35]. These costs compound for high-frequency keys, where even lightweight updates translate into sustained I/O pressure and elevated tail latency.

2.2

Mathematical Foundations for Efficiency

Approximate computation has long been employed to reduce the cost of maintaining state in large-scale data systems [16, 31]. Classical work in approximate query processing [16] and stochastic simulation [39] showed how controlled uncertainty can yield significant efficiency gains. In streaming settings, exponential decay mechanisms [15, 64], incremental estimators such as Welford’s method [68], and related recursive formulations [11, 14, 33] reduce aggregation updates to constant-time operations, enabling long event histories to be summarized in bounded memory. Probabilistic data structures, including Bloom filters [58], count– min sketches [18], and HyperLogLog [27], provide sublinear-memory

Decoupling Inference and State Updates

2.3

Pareto chart of transaction volume 105

80% volume

103

Entity Volume Cumulative % of volume 60 Top 4.1% of entitities account for 80% volume 40

101

20 0

Load Shedding and Stream Sampling

When input rates exceed capacity, streaming engines commonly resort to load shedding or sampling. Ingestion-level techniques such as reservoir sampling or threshold-based shedding discard events early to protect downstream operators [2, 7], while selective approaches prioritize events based on utility or query relevance [51, 60]. Alternatively, state-shedding methods leverage in-memory buffers to dynamically select events under pressure [56, 71], trading off inference completeness by evicting intermediate state under windowed, query-driven semantics. Although effective, shedding is incompatible with security and risk-sensitive ML applications. In fraud detection, for example, every transaction must be inspected: discarding an events at ingestion or evicting intermediate state compromises both inference coverage and long-lived feature consistency. Furthermore, reliance on complex in-memory control logic or cross-worker coordination to manage such shedding can become a scalability bottleneck. Our work builds on these foundations but instead applies stochastic thinning directly at the persistence layer. We treat state updates as probabilistic events decoupled from inference triggers, enabling persistencepath control that avoids both input dropping and state eviction, while eliminating high-frequency in-memory control. Thus, we reduce write frequency and mitigate the RMW and SerDe bottlenecks intrinsic to modern streaming engines.

100

20 40 60 80 Merchants ordered by volume (%)

100 0

Cumulative % of volume

Total volume (log scale)

summaries with bounded bias and variance. While these techniques optimize the cost per updates, even compact sketches must be retrieved from storage, deserialized, updated, and written back. For high-velocity entities, such as popular merchants in payment streams, the dominant bottleneck is therefore not storage capacity, but the sustained rate of persistence operations; such as I/O Operations Per Second (IOPS) and CPU SerDe processes. Our work targets the complementary dimension: reducing the frequency of state updates while preserving statistical correctness under controlled stochastic selection.

Figure 2: Merchant activity volume in a real-world transaction dataset, over a 7 month interval. Transaction counts (log scale) are shown for unique merchants ordered by volume, revealing a highly skewed distribution. maintaining unbiased aggregates. We first describe the process under the standard assumption that control statistics are continuously updated in-memory. Later, we address the significant challenges of maintaining these statistics in modern streaming systems.

3.1

Problem Formulation

For simplicity and without loss of generality, we consider a timeordered event stream E = {𝑒 1, 𝑒 2, . . . },

with

E (𝑡) = {𝑒𝑛 ∈ E : 𝑡𝑛 ≤ 𝑡 },

associated with a single entity, such as a merchant, card number, or device. Each event 𝑒𝑛 = (𝑞𝑛 , 𝑡𝑛 ) consists of a quantitative attribute 𝑞𝑛 ∈ R (e.g. transaction amount) and a timestamp 𝑡𝑛 ∈ R+ . The arrivals form a marked inhomogeneous Poisson Point Process [20, 34] with intensity function 𝜆(𝑡). Thus, the expected number of events 𝑁 in any interval (𝑡 1, 𝑡 2 ] is given by: ∫ 𝑡2 E[𝑁 (𝑡 1, 𝑡 2 )] = 𝜆(𝑡) 𝑑𝑡, 𝑡1

Adaptive Event Filtering

Streaming ML workloads exhibit strong heterogeneity in event arrival patterns across entities. In financial transaction monitoring, activity is often concentrated on a small subset of accounts, while most entities remain inactive for extended periods. Figure 2 illustrates this effect on real-world data: approximately 4% of merchant entities generate nearly 80% of incoming events. Under such skew, executing a full RMW cycle for every event yields diminishing returns. For high-volume entities, successive updates contribute little new statistical information while incurring substantial serialization, I/O, and replication cost. This motivates adaptive event filtering: a systems-level mechanism that suppresses persistent state updates while preserving inference on every event, under a controlled probabilistic thinning policy [20, 37]. In this work, we frame this as intensity-aware thinning of stochastic processes [20, 37], adapting ideas from load shedding [59], rate limiting [13], or adaptive sampling [16, 23, 62], but shifting their purpose. Rather than controlling ingestion, we bound write frequency and serialization cost under skewed ML workloads, while

In practice, 𝜆(𝑡) varies widely across entities and over time, reflecting behavioural heterogeneity. Figure 3 illustrates this effect for two entities with high and low arrival intensity. While every event 𝑒𝑛 arriving at the system triggers an inference task, we execute a persistent state update only if an auxiliary Bernoulli variable 𝑍𝑛 ∼ B (𝑝𝑛 ) evaluates to 1. This yields a filtered sub-stream of persisted events: I = {𝑒𝑛 ∈ E : 𝑍𝑛 = 1},

with

I (𝑡) = {𝑒𝑛 ∈ I : 𝑡𝑛 ≤ 𝑡 }.

The inclusion probability 𝑝𝑛 governs the expected write rate, and is determined by a policy ˆ 𝑛 ), 𝑒𝑛 ), 𝑝𝑛 = 𝑓 (Λ, 𝜆(𝑡 (1) which enforces a user-defined write budget Λ relative to an estimate ˆ 𝑛 ) of the unknown true event intensity1 . A common baseline is 𝜆(𝑡 ! Λ ˆ 𝑓 (Λ, 𝜆(𝑡𝑛 ), 𝑒𝑛 ) = min 1, , (2) ˆ 𝑛) 𝜆(𝑡

1 Processes to estimate 𝜆 (𝑡 ) from observed event histories are detailed in Section 4.

Peres et al.

Intensity estimates

True (High) Est. (High)

Decayed sum

Approximations (Low) 6 × 106 4 × 106 3 × 106

103

102

High volume

λ̂(t) > Λ Low volume Λ = 70.0

True (Low) Est. (Low)

100

150

200

Time

100 Time

150

200

Inclusion probability p

2 × 108

Sampling probabilities 1.0

Estimated intensity λ̂

Decayed sum

Approximations (High) 4 × 108 3 × 108

0.8 0.6

High volume Low volume

0.4 0.2 0.0 0

100 Time

150

200

Figure 3: Adaptive event filtering applied to high- and low-velocity entities. Left: Reconstructed decayed sums for LogNormal ˆ distributed {𝑞𝑛 }𝑛=1,2,... , averaged over 100 runs and cropped to 𝑡 ∈ [50, 200) for readability. Center: Estimated arrival rates 𝜆(𝑡) compared to a thinning threshold Λ. Right: Probability of triggering persistence-path write operations over time. High-intensity entities undergo aggressive filtering (fewer writes), while low-intensity entities remain largely unaffected. Í which guarantees E[ 𝑛𝑖=1 𝑍𝑖 ] ≤ Λ ·𝑡𝑛 and throttles persistence-path operations for high-velocity entities. Figure 4 illustrates a schematic of this naive workflow.

Arrival-rate intensity estimate

Read

Persistent Storage

Write Probability Score

Classification Nodes

Feature Profiles

Aggregation Estimates

Figure 4: Naive adaptive filtering. (1) Event arrives; (2) System retrieves aggregations from persistent storage; (3) Computes updates and writes back to persistence probabilistically; (4) Event is sent for scoring and (5) returned to the client.

3.2

Probabilistic State Management

Let 𝐴(𝑡) denote an aggregate computed over the full stream, such as a sum or count, with per-event contribution 𝑤 (𝑡, 𝑒𝑛 ). Under adaptive filtering, the system introduces stochastic sparsity by suppressing updates. We therefore approximate 𝐴(𝑡) using a Horvitz-Thompson estimator [32] over the sub-stream I (𝑡), i.e. ∑︁ 𝑤 (𝑡, 𝑒𝑛 ) b = 𝐴(𝑡) . 𝑝𝑛 𝑒𝑛 ∈ I (𝑡 )

3.3

Recursive Aggregates

Streaming feature stores typically maintain aggregates over complete histories, using exponentially decayed counts, sums, or averages, because they admit constant-space updates compatible with KV-based state stores [15, 17, 53]. Table 1 outlines standard perevent weights 𝑤 (𝑡, 𝑒𝑛 ), where 𝜏 > 0 represents a decay factor analogous to sliding-window intervals. Table 1: Examples of popular per-event contribution functions 𝑤 (𝑡, 𝑒𝑛 ) used in streaming feature stores. Metric Type All-Time Exponential Decay

Count Weight

Sum Weight

1 𝑛| exp − |𝑡 −𝑡 𝜏

𝑞𝑛 𝑛| 𝑞𝑛 · exp − |𝑡 −𝑡 𝜏

When enforcing thinning, update rules for recursive aggregates elegantly incorporate a Bernoulli mask, s.t. b 𝑛 ) = 𝑍𝑛 · 𝑤 (𝑡𝑛 , 𝑒𝑛 ) + exp − 𝑡𝑛 − 𝑡𝑛−1 𝐴(𝑡 b 𝑛−1 ). 𝐴(𝑡 𝑝𝑛 𝜏 This simple formulation supports, without loss of generality, for the measurement of averages, ratios, squared means, moments, or variances, to name only a few.

3.4

Variance-Aware Adaptive Filtering

This estimator inversely scales contributions by their inclusion probability, remains unbiased, and has variance: h i ∑︁ 1 b Var 𝐴(𝑡) = 𝑤 2 (𝑡, 𝑒𝑛 ) E −1 . (3) 𝑝𝑛

b of the true aggregate 𝐴, precision is For an unbiased estimator 𝐴 often measured by the coefficient of variation [25]; i.e. √︃ b Var[𝐴(𝑡)] 1 b CV 𝐴(𝑡) = ∝ √︁ , b E[𝐴(𝑡)] |I (𝑡)|

Thus, approximation error is dominated by large-magnitude contributions 𝑤 (𝑡, 𝑒𝑛 ) retained with low probability. We refer the reader ˆ 𝑛 ). to Appendix A for the derivation through conditioning on 𝜆(𝑡

where |I (𝑡)| is the number of retained events. As volume grows, the marginal utility of each update for precision decreases. Adaptive filtering exploits this by suppressing updates beyond a threshold, in order to achieve system-level savings.

𝑒𝑛 ∈ E (𝑡 )

Decoupling Inference and State Updates

Inclusion probability distribution

Relative error distribution

Self-correcting (SC) SC + variance reduction

Density

Error Self-correcting (SC) Error SC + variance reduction

2 0 0.0

0.2

0.4 0.6 Inclusion probabilities

0.8

1.0

0.0

0.2 0.4 0.6 Relative estimation error

0.8

Figure 5: Inclusion probabilities and estimator error across sum, count, and average aggregations 𝐴(𝑡). We compare standard (blue) and variance-reduced (red) filtering strategies. Left: Variance-aware strategies reallocate probability mass for write cycles while maintaining total write budget. Right: Targeted reallocation reduces estimation error without increasing system load.

Filtering under Compute-Side Constraints

A central challenge in operationalizing event filtering is the reliance on a continuously updated control state. Classical thinning ˆ or moments assumes access to accurate arrival-rate estimates 𝜆(𝑡) (𝜇 𝑤 , 𝜎𝑤 ), maintained eagerly in compute-adjacent storage. However, maintaining high-frequency, coordination-sensitive control state reintroduces the very write pressure that thinning is meant to eliminate. If filtering decisions require per-event state updates, the control plane becomes a bottleneck even when data-plane updates are suppressed, nullifying the intended efficiency gains. Design goal. We require a filtering mechanism where decisions depend exclusively on persistence-backed state already maintained for feature computation, without additional in-memory state or coordination. Additionally, statistical correctness and predictable bounds on write rates must be preserved.

4.1

Arrival-Rate Estimation as a Control Dependency

To adjust 𝑝𝑛 dynamically, adaptive filtering requires an online estimate of the arrival intensity 𝜆(𝑡) per entity. In practice, this is

𝑒𝑛 ∈𝐸 (𝑡 )

where ℎ > 0 is a bandwidth controlling temporal responsiveness. In a streaming context, this admits a simple recurrence across timestamps (𝑡𝑛 )𝑛=1,2,... : ˆ 𝑛 ) = 𝜈ˆ (𝑡𝑛 ) , 𝜈ˆ (𝑡𝑛 ) = 1 + exp − 𝑡𝑛 − 𝑡𝑛−1 𝜈ˆ (𝑡𝑛−1 ). 𝜆(𝑡 ℎ ℎ This estimator offers a low-variance, biased approximation of the true arrival intensity (see Appendix B) and is widely used in practice [3, 9, 41]. However, maintaining this estimate eagerly creates a circular dependency: the system must update compute-adjacent 1.0

1.0

Probability before variance reduction

where 𝜎 (𝑥) = 1/(1 + 𝑒 −𝑥 ) is the logistic function [8], and 𝜇 𝑤 , 𝜎𝑤 denote historical means and standard deviations of contributions. Here, 𝛼 ∈ R+ controls the trade-off between write-budget adherence and variance reduction. This mirrors classical importance sampling [28, 30, 63] and is particularly effective under heavytailed event magnitudes [66]. Figures 5–6 demonstrate that this rule reallocates update probability toward informative events while stabilizing total write volume, and we refer the reader to Appendix E for a sensitivity analysis.

derived using Kalman filters [67] or Kernel Density Estimation (KDE) with exponential smoothing [48, 54]: ∑︁ |𝑡 − 𝑡𝑛 | ˆ =1 exp − 𝜆(𝑡) , (5) ℎ ℎ

Probability after variance reduction

To counter the error spikes caused by large-magnitude events in Equation 3), we design inclusion rules that adjust 𝑝𝑛 in inverse proportion to squared contributions 𝑤 2 (𝑡, 𝑒𝑛 ), i.e. ! ! 𝑤 (𝑡, 𝑒𝑛 ) − 𝜇 𝑤 Λ 𝑝𝑛 = 𝜎 𝜎 −1 +𝛼 , (4) ˆ 𝑛) 𝜎𝑤 𝜆(𝑡

0.8

0.6

0.4

0.2

0.0 -4

-2 0 2 Deviation from historic mean

Figure 6: Variance-reduced probability of persistence-path write cycles (y-axis) relative to normalized event magnitude 𝑤 (𝑡, 𝑒𝑛 ) (x-axis). Blue indicates events unlikely to trigger writes prior to variance reduction; orange indicates likely events. Variance-aware filtering increases retention probability only for statistically influential events.

Peres et al.

Estimator error Mean error Spread (2σ)

−5

−10

Cumulative oversampling gap

150

100

1000

2000 Time

3000

4000

30 20 10 0

−10

0 0

Oversampling gap

Self-correcting λ̂F Threshold Λ

Vanilla λ̂(t)

Gap (NF − N )

True λ(t)

200

Intensity estimate

10 Error (λ̂F − λ̂)

Intensity estimate

200

400 Time

600

800

200

400 Time

600

800

ˆ over 1000 runs, illustrating cyclic self-correcting behavior. Center: True intensity of Figure 7: Left: Error bounds for 𝜆ˆF (𝑡) − 𝜆(𝑡) an arrival process with sampled events and corresponding estimators (blue line is hidden behind the red line). Right: Expected difference in write cycles under persistence-path control (NF ) and a vanilla in-memory control, with 95% confidence intervals. state just to decide if a disk-backed persistence update is necessary. Under high-throughput workloads, this feedback loop can dominate CPU time, serialization cost, and cache pressure.

4.2

Persistence-Path Control via Filtered Estimators

To break this dependency, we replace a full-stream KDE estimator with a filtered counterpart: 1 + exp(−(𝑡𝑛 − 𝑡𝑛−1 )/ℎ) 𝜈ˆF (𝑡𝑛−1 ) 𝜆ˆF (𝑡𝑛 ) = , ℎ where 𝜈ˆF is updated exclusively on the persistence path, i.e. 𝑡 −𝑡 𝑍𝑛 𝑛 𝑛−1 𝜈ˆF (𝑡𝑛−1 ). 𝜈ˆF (𝑡𝑛 ) = + exp − 𝑝𝑛 ℎ only when a write operation is executed across persisted events I = {𝑒𝑛 ∈ 𝐸 : 𝑍𝑛 = 1}. Consequently, thinning probabilities 𝑝𝑛 in Equation (1) are derived using only disk-backed aggregates already required for feature maintenance, fully decoupling the process from in-memory control panes and synchronization. The trade-off is increased stochasticity, since 𝜆ˆF (𝑡𝑛 ) is itself influenced by prior filtering decisions 𝑍 1, . . . , 𝑍𝑛−1 . Crucially, this interdependence remains stable and analytically tractable. Remark 4.1 (Stability of filtered estimation). Define the normalized deviation ˆ 𝑛) 𝜆ˆF (𝑡𝑛 ) − 𝜆(𝑡 𝑀𝑛 = exp(−𝑡𝑛 /ℎ) Then {𝑀𝑛 }𝑛≥0 is a martingale with respect to the natural filtration induced by event arrivals. The filtered estimator remains anchored in expectation to the ˆ in Equation (5). This property motivates the full-stream KDE 𝜆(𝑡) term self-correcting: if it overshoots, inclusion probabilities decrease, suppressing subsequent updates; when it undershoots, probabilities increase, restoring update frequency. Estimation errors cannot compound indefinitely, and formal proofs are provided in Appendix C.

Moreover, the variance introduced by filtered estimation provides an implicit safety guarantee through structural oversampling. Remark 4.2 (Safety via oversampling). Let NF and N denote the number of persistent updates triggered under filtered and fullˆ respectively. stream control, using intensity estimates 𝜆ˆF (𝑡) and 𝜆(𝑡) Then E[NF ] ≥ E[N ]. Persistence-path control may incur marginally higher write volume than an idealized in-memory design, but never fewer in expectation. As shown in Figure 7, oversampling is bounded, preserves statistical correctness, and can be managed by adjusting the global write budget Λ. Taken together, results demonstrate that adaptive event filtering can be implemented without high-frequency in-memory control state. By relying exclusively on persistence-backed estimates, the system preserves unbiasedness, ensures predictable write-rate bounds, and control-plane computation scales with the write budget rather than the raw event rate. This aligns with the execution model of modern streaming engines, where only persisted state is checkpointed, replicated, and migrated during failure recovery or rebalancing. Appendix D provides formal proofs and additional analysis of oversampling dynamics.

System Implementation

We describe a concrete system instantiating our probabilistic state management design. To isolate persistence-path control as a reusable primitive, we focus on a standard execution unit: a worker operating over partitioned key-–value state. This allows us to reason about the marginal cost of persistence operations while maintaining compatibility with distributed execution.

5.1

Component-Level Architecture

Figure 8 illustrates our design operating as an embedded feature aggregation component in a standard production-grade risk-scoring pipeline. The architecture consists of two primary layers:

Decoupling Inference and State Updates

Client

Partitioned Aggregation Engine

Model Serving Feature Aggregations

Orchestration Workflow

Cluster Worker 1

Key Routing

Recursive Updates

Event

Worker Node Filtering + Serialize

Deserialize

Worker 2

.. .

Parsing & Retrieval Assembly & Joins

Worker n

CEP & Decisioning

Persistent K/V Storage and

Figure 8: Logical architecture of a standard ML pipeline with persistence-path control embedded in the feature aggregation engine. A stream processing layer orchestrates feature retrieval and model inference, while state is maintained in partitioned key–value stores. (1) The orchestrator queries the engine for entity state; (2) requests are routed to the responsible partition, ensuring per-key ordering; (3) a worker retrieves feature aggregates and control statistics; (4) updates are probabilistically materialized via persistence-path control; (5) resulting features are returned to the orchestrator for downstream scoring. • A Stream Processing Layer orchestrates ingestion, feature retrieval, and model inference. This may include enrichment or Complex Event Processing (CEP) pattern detection [56, 71], and operates under strict latency constraints. • A Feature Aggregation Engine maintains per-entity state in a distributed store, enabling recursive updates via partitioned key–value stores. Here, our mechanism intercepts the RMW cycle associated with feature updates. Model inference is executed by a decoupled serving layer, which performs request routing, model selection, and stateless evaluation over the assembled feature vector, as is standard in modern online inference systems. In our partitioned system, for each event, a feature aggregation worker performs the following steps: (1) Retrieves feature state and control statistics from storage. (2) Materializes features for inference. (3) Derives an inclusion probability over disk-backed estimates. (4) Samples a Bernoulli decision. (5) Executes a write-back only if selected. Crucially, inference is performed for every event, while persistence updates are selectively suppressed. The mechanism operates entirely within the feature computation layer: its control logic is stateless, requires no auxiliary in-memory control or cross-worker synchronization, and avoids control-plane amplification under skewed workloads. The design aligns with execution models of Apache Flink, Kafka Streams, and Spark Structured Streaming, where workers process disjoint key partitions with local state.

5.2

Summary of Operating Assumptions

We summarize the assumptions underlying the design and applicability of persistence-path control for feature-store workloads: • Partitioned Storage: State is organized per-entity and processed independently without cross-key coordination. • Recursive Aggregations: We target decomposable feature updates of the form 𝐴(𝑡𝑛 ) = 𝑔(𝐴(𝑡𝑛−1 ), 𝑒𝑛 ),

for recursive operations 𝑔 including counts, sums or moments. Sequence-sensitive or pattern-based state (e.g., CEP operators) are out of scope. • Logical RMW: Processing follows a retrieval, update, and optional persistence pattern. Optimizations like batching or async I/O must preserve these semantics. • Skewed Distributions: We assume skewed workloads where a few keys drive most updates. Common condition essential for significant persistence reduction. • Approximation Tolerance: Features tolerate controlled stochastic approximation, typical in domains like fraud detection where full inference coverage is prioritized.

5.3

Controlled Execution Model

We implement an execution model that instantiates the per-partition abstraction described above, under varying levels of concurrency, load, and key distribution. Our objective is to isolate the impact of probabilistic filtering on persistence costs, under storage and compute resource contention: • Events are replayed from Parquet-backed datasets using a containerized load generator based on Locust [61]. • Requests are issued continuously and routed across worker instances via deterministic key partitioning, ensuring perkey ordering while enabling parallel processing. Workers are deployed as independent, horizontally scaled containers on shared hardware, enforcing strict request–response execution with no internal batching. Thus, the system is subject to CPU contention, I/O interference, and skew-induced load imbalance across partitions. Each process orchestrates feature retrieval, probabilistic filtering, and optional persistence via a single logical RMW operations per event. This ensures that write frequency, IOPS, and latency remain directly observable. State management is handled through embedded RocksDB instances [22] with leveled compaction and write-ahead logging, where updates are issued synchronously to SSD-backed storage to guarantee atomic, strongly ordered, and sequential execution per key.

Peres et al.

Interpretation. This execution model captures the fundamental unit of stateful processing in modern streaming systems: independent, partitioned workers operating over local key–value state. We thus preserve the persistence interface where RMW costs arise, while abstracting away runtime concerns such as operator scheduling or networked dataflows. Because partitions execute independently, the measured effects of reduced write frequency directly translate to improved throughput, reduced contention, and delayed saturation under scale-out. These effects compose across partitions and integrate with common dataflow, check-pointing, and fault-tolerance mechanisms without requiring changes to upstream or downstream operators.

Experimental Evaluation

Our evaluation reflects streaming ML settings where all events are processed for inference, while only a subset of events is required to update persistent feature state. We separate concerns: systemlevel results isolate persistence overheads under varying execution conditions, while ML evaluation assesses the predictive impact of self-correcting persistence-path thinning. All results are reported with approximate 95% confidence intervals, and experiments are reproducible and public.2

6.1

Experimental Setup

We report results on the datasets outlined in Table 2 to capture distinct operational regimes: • Financial Transaction Fraud: A proprietary real-world transaction monitoring dataset, with strong key skew, continuous anomalous activity, and heavy-tailed transaction amount distributions.3 • IBM Synthetic Fraud Dataset: Public benchmark exhibiting financial patterns with strong key skew and moderate heavy-tailed transaction values [4]. • Edge-IIoTset: Network intrusion data with bursty denial of service patterns, temporal locality of anomalies, and relatively symmetric packet size distributions [26]. • Wikipedia Vandalism: User edit streams with weak key skew and balanced edit distributions, where anomalous behavior is associated with short-lived throwaway accounts, limiting the utility of long-lived temporal profiles [50]. Thus, we evaluate persistence-path control under target conditions (high key skew), assumption violations (e.g., weak skew or limited temporal structure), and across varying label magnitude imbalance. In all datasets, events are partitioned by a primary identifier, e.g. merchant, user or IP address, and each key maintains temporal aggregations of activity patterns. Feature engineering is kept simple and production-representative. Persisted state consists exclusively of time-decayed aggregations, including counts, sums, and means, implemented via recursive decays as described in Section 3. We use decay factors approximating windows of 1 minute, 1 hour, and 1, 30, 60, and 120 days. This captures heterogeneous temporal dynamics, from short-term bursts to long-term trends, while remaining compatible with constant-space KV-store updates. 2 https://anonymous.4open.science/r/Decoupling-Inference-from-StateUpdates-via-Probabilistic-Thinning-in-Low-Latency-Feature-Engines-0319 3 Sourced for research purposes by Feedzai: https://www.feedzai.com

Table 2: Dataset characteristics. “80% Vol.” denotes the percentage of keys responsible for 80% of events; kurtosis refers to the aggregand distribution (e.g., transaction amounts). Dataset

Events

Keys

Anomaly %

80% Vol.

Kurtosis

Fraud IBM IIoTset Wikipedia

11M 9M 5M 6K

7K 7K 800K 3K

0.05 0.13 40.01 8.35

4.1% 1.5% 0.7% 23.6%

8 3 2 2

Experiments run on dedicated AWS instances, with ingestion workers on c6i.xlarge nodes and load generation on c5a.4xlarge nodes. Components are co-located in the same region and Virtual Private Cloud (VPC) to minimize network variability, and storage is backed by SSD EBS volumes.

6.2

Filtering Strategies and Baselines

We compare our design against a range of strategies that isolate ˆ for different system trade-offs, using arrival-intensity estimates 𝜆(𝑡) filtering decisions computed with the KDE estimator in Equation (5). All strategies enforce a user-defined upper bound Λ on the expected number of persistent write operations per minute and key. We evaluate the following strategies: (1) Persistence-Path Control. Filtering decisions depend solely on control statistics updated along the persistence path, without auxiliary in-memory control state. (2) Persistence-Path Control + Variance Reduction. Inclusion probabilities incorporate the variance-aware formulation in Equation (4), evaluated specifically for downstream ML fidelity and feature approximation quality. As benchmarks, we consider the following reference baselines and commonly used techniques for controlling write pressure: (1) Full-Stream Control. Reference baseline with computeadjacent control state, enabling filtering decisions without serialization overheads. (2) Naive Fixed-Rate Filtering. Stateless random persistent updates using a fixed global probability, independent of key activity or arrival rate. (3) Periodic Batching. Per-key buffering with periodic flushes, introducing feature staleness and bursty I/O. We use a small buffer (100 events) to approximate an upper bound on batching efficiency under minimal delay. Excluded Baselines. We exclude reservoir sampling [65] and its weighted variants [24]. These techniques maintain a fixed-size sample per key, requiring explicit in-memory buffers whose size must be chosen a priori. Under skewed key distributions, this leads to unbounded memory growth or biased eviction behavior, making reservoir sampling unsuitable for high-cardinality feature maintenance with strict write-rate constraints.

6.3

Intrinsic Execution

We first evaluate system-level efficiency in a controlled singlepartition setting that isolates the direct relationship between persistence frequency and worker-level performance. This captures

Decoupling Inference and State Updates

Table 3: Intrinsic efficiency metrics for different filtering strategies and baselines, on open-source IBM data. We use user-defined reference bounds for event volume processed, per minute and key. Full-stream filtering and periodic batching serve as idealized baselines that contextualize upper limits of achievable efficiency. Strategy

𝚲

Write

Throughput

(Ev/m)

(%)

(TPS)

Avg

p95

p99.99

WAF

Bps

Util (%)

–

100.00

226.82 ± 0.21

4.36 ± 0.00

5.00

10.00

2.6

3910.54 ± 17.45

Persistence-Path

0.001 0.005 0.010 0.050 0.100 1.000

5.91 ± 0.05 25.65 ± 0.07 44.70 ± 0.04 83.94 ± 0.02 91.70 ± 0.02 100.00 ± 0.00

621.05 ± 3.15 454.90 ± 1.80 352.57 ± 0.41 252.00 ± 0.14 239.41 ± 0.15 226.59 ± 0.27

1.56 ± 0.01 2.16 ± 0.00 2.79 ± 0.00 3.91 ± 0.00 4.13 ± 0.00 4.37 ± 0.00

4.00 4.00 4.33 5.00 5.00 5.00

6.50 8.25 8.80 9.80 9.93 10.00

1.7 2.3 2.3 2.7 2.6 2.7

247.07 ± 3.24 1029.48 ± 3.81 1782.59 ± 5.87 3317.67 ± 6.21 3625.38 ± 8.68 3947.12 ± 7.30

3 13 22 41 45 49

Full-Stream

0.010 0.050 0.100 1.000

13.57 ± 0.78 55.24 ± 0.04 73.34 ± 0.04 99.84 ± 0.03

510.78 ± 6.19 318.74 ± 0.35 274.31 ± 0.21 227.03 ± 0.43

1.91 ± 0.02 3.09 ± 0.00 3.60 ± 0.00 4.36 ± 0.00

4.00 4.73 5.00 5.00

6.40 9.13 9.33 9.93

2.1 2.3 2.5 2.6

781.32 ± 7.25 3269.48 ± 2.90 3755.13 ± 3.14 3970.73 ± 184.31

8 40 46 49

Fixed-Rate

– –

14.98 ± 0.08 44.94 ± 0.16

536.68 ± 2.04 360.56 ± 2.77

1.82 ± 0.01 2.73 ± 0.02

4.00 4.00

7.00 9.25

2.0 2.0

559.85 ± 3.67 1773.27 ± 6.79

7 22

Periodic Batching

–

703.12 ± 0.47

1.38 ± 0.00

1.00

6.00

2.4

128.84 ± 2.41

Unfiltered

intrinsic properties of the RMW execution model underlying modern streaming systems, abstracting away cross-partition contention, coordination, or scheduling effects. We use the IBM dataset. Since system behavior is primarily driven by event rate and persistence frequency rather than feature semantics, results are qualitatively consistent across datasets. We use two configurations: • Closed-loop load generation [55] to measure client-side peak throughput and latency, issuing each subsequent event only after receiving a response, and • Fixed-rate input (200 events/sec) to isolate system-side resource utilization. We report on: (i) write frequency, (ii) throughput, (iii) end-to-end latency, (iv) write amplification, (v) sustained storage bandwidth, and (vi) disk utilization. Figure 1 illustrates the throughput–volume trade-off, and Table 3 summarizes intrinsic performance results. Across all system-level metrics, persistence-path control achieves substantial reductions in write amplification, latency, and disk utilization compared to reference baselines, while increasing throughput, despite avoiding in-memory control state. Also, as Λ → 1, performance metrics converge to the unfiltered baseline, confirming negligible overhead. Client-Side Metrics. Unfiltered processing yields the minimum throughput (226.8 TPS), while decreasing the write budget Λ results in near-linear scaling: at write budget Λ = 0.001, throughput increases 2.7× and average latency drops 64%. Stable 𝑝99.99 tail latencies indicate that aggressive filtering mitigates I/O-induced head-of-line blocking in synchronous RMW execution, ultimately improving ML utility (Section 6.5).

Latency (ms)

Fixed Throughput @ 200 Ev/s

System-Side Metrics. Filtering reduces WAF from 2.6 to 1.7. This aligns with LSM-tree dynamics, where lower ingestion rates suppress compaction frequency and background amplification [22], extending SSD endurance. The non-linear rise in disk utilization at higher Λ reflects LSM-tree compaction thresholds, where background maintenance amplifies I/O disproportionately [21, 44]; filtering effectively keeps the system below these saturation points. Noticeably, periodic batching achieves high throughput by amortizing I/O, but at the cost of feature staleness and streaming ML inference degradation [1] (see results in Section 6.5). The strong taillatency results observed reflect small buffers and single-threaded execution; larger batch sizes or multi-partition deployments would induce bursty I/O and unbounded staleness.

6.4

Operational Scalability

We next demonstrate that the relative benefits of persistence-path control remain stable under operational conditions common in production streaming systems, including concurrency, asynchronous execution, and workload variability. In particular, we study: (i) concurrent execution under increasing parallelism, (ii) sensitivity to skewed key distributions, (iii) long-running execution stability, and (iv) behavior near saturation points. Concurrent Execution and Contention. Figure 9 evaluates throughput and latency while horizontally scaling independent partition workers across containerized RocksDB-backed instances. Events are injected concurrently across workers using deterministic key-toworker routing, following the partitioned aggregation architecture illustrated in Figure 8, while increasing aggregate load to expose the system to significant CPU and I/O contention. We observe throughput scaling consistently with worker parallelism, while average latency decreases despite higher aggregate

Peres et al.

Figure 9: Concurrent execution and contention evaluation under horizontally scaled partitioned workers. Events are routed to worker instances via deterministic key partitioning, preserving per-key ordering while enabling parallel execution over isolated RocksDB-backed state. Left: Throughput under increasing worker parallelism. Center: average latency under contention. Right: tail-latency behavior. Relative gains from persistence-path control remain stable under increasing CPU and I/O contention. load. Importantly, the relative gains from filtering remain stable under increasing contention. Stable tail latencies further suggest that reducing persistence frequency mitigates contention-induced queueing and head-of-line blocking during concurrent execution. Sensitivity to Skew, Stability, and Saturation. Table 4 summarizes worker-level analyses under this partitioned execution model, across varying workload regimes. To study skew sensitivity, we reduce key-frequency imbalance by removing transactions associated with high-volume entities, while preserving overall workload structure. This materially changes the realized write percentage under identical filtering budgets. Nevertheless, throughput and latency remain anchored to persistence frequency rather than the key distribution, suggesting that the dominant execution cost remains the number of completed RMW cycles. We further evaluate long-running execution by extending workload duration from 5 to 50 minutes across filtering regimes, under stabler request volumes. Here, throughput and latency remain effectively unchanged over time, indicating stable behavior and absence of progressive degradation under sustained operation. Finally, we evaluate saturation behavior by progressively increasing the intensity of asynchronous request streams until latency collapse (>500ms), in increments of 50 events/sec to estimate failure regions. The sustainable throughput prior to back-pressure onset increases substantially as filtering becomes more aggressive, with the failure threshold rising from approximately 200 events/sec to over 1350 events/sec at the lowest write rates. This indicates that reducing persistence pressure delays storage saturation and allows substantially higher inference throughput before contention dominates execution. More broadly, persistence-path control expands the feasible operating region of streaming inference systems by enabling higher event throughput before storage saturation or back-pressure compromise latency.

6.5

Downstream ML Utility

We next assess whether adaptive filtering: (i) degrades downstream ML performance, and (ii) preserves sufficient information to approximate full-stream feature aggregates.

Table 4: System evaluation under key skew variation, longrunning execution, and saturation analysis. Experiment

Throughput

Latency (ms) Avg

p95

p99.99

3.65 4.10 4.48

5 5 5

10 10 10

2.77 3.35 4.10

5 5 5

8.5 9.25 10

221.21 ± 0.54 221.44 ± 0.87 220.72 ± 0.63

4.47 4.47 4.48

5 5 5

9.75 9.75 10

343.59 ± 1.42 342.96 ± 1.73 342.15 ± 1.06

2.86 2.87 2.88

5 5 5

8.5 9 9

559.88 ± 1.77 559.00 ± 2.52 557.01 ± 1.17

1.74 1.74 1.75

4 4 4

5.5 6 6

Failure Thr. 200-250

–

350-400 600-650 1350-1400

– – –

(TPS) Sensitivity to Key Skew 10% keys → 80% volume Write ≈70% 270.54 ± 1.35 Write ≈85% 241.27 ± 1.07 Write ≈97% 220.76 ± 1.69 5% keys → 80% volume Write ≈41% 354.38 ± 1.13 Write ≈61% 294.57 ± 1.37 Write ≈87% 241.10 ± 0.52 Long-Running Stability Write ≈100% 5 Minutes 10 Minutes 50 Minutes Write ≈45% 5 Minutes 10 Minutes 50 Minutes Write ≈6% 5 Minutes 10 Minutes 50 Minutes

Backpressure and Saturation Write ≈100% 100% Filtered Execution Write ≈45% Write ≈26% Write ≈6%

Decoupling Inference and State Updates

Table 5: Downstream ML performance across datasets and experiments, under varying filtering strategies. As we relax humandefined RMW budgets, we measure write persistence operations and model recall at fixed FPR levels. Strategy

Real Fraud

IBM Fraud

Edge-IIoT

Wikipedia

Write %

Recall Δ

Write %

Recall Δ

Write %

Recall Δ

Write %

Recall Δ

Unfiltered

100.00

0.00

100.00

0.00

100.00

0.00

100.00

0.00

Persistence-Path

0.84 3.60 6.67 28.11 53.19

−5.61 ± 5.22 +7.06 ± 2.53 +7.83 ± 3.17 +4.55 ± 5.03 +2.57 ± 4.18

33.34 45.14 54.03 78.23 90.26

−2.94 ± 0.27 −1.11 ± 0.35 −0.40 ± 0.39 −0.18 ± 0.31 +0.45 ± 0.26

16.42 27.51 46.68 49.51 62.42

−10.29 ± 3.03 −7.19 ± 5.79 +0.81 ± 10.28 +2.46 ± 11.78 +8.73 ± 12.75

87.11 88.55 89.88 92.14 94.19

+0.05 ± 0.35 +0.09 ± 0.28 +0.05 ± 0.28 −0.05 ± 0.24 −0.17 ± 0.26

Persistence-Path + VR

1.10 4.00 7.23 29.00 53.52

−5.21 ± 5.94 +5.67 ± 3.07 +6.00 ± 3.76 +5.05 ± 4.78 +0.45 ± 2.77

34.35 46.30 55.06 78.62 89.97

−1.81 ± 0.26 −0.98 ± 0.34 +0.05 ± 0.34 +0.09 ± 0.41 +0.15 ± 0.27

16.89 28.39 47.67 50.51 62.73

−10.91 ± 0.02 −11.15 ± 0.03 +0.78 ± 11.31 +5.54 ± 11.74 +6.74 ± 12.40

86.68 88.33 89.80 92.07 94.13

−0.11 ± 0.34 +0.02 ± 0.27 +0.08 ± 0.18 +0.11 ± 0.30 +0.05 ± 0.35

Full-Stream

0.56 2.80 5.60 27.90 55.90

−0.27 ± 6.08 +4.07 ± 5.91 +4.49 ± 8.53 +3.98 ± 4.39 +2.14 ± 4.51

32.21 48.34 58.60 84.15 95.52

−2.77 ± 0.29 −1.05 ± 0.28 −0.10 ± 0.29 +0.10 ± 0.42 +0.12 ± 0.34

17.82 32.14 45.63 51.04 65.56

−13.17 ± 1.51 −8.33 ± 5.97 +0.32 ± 11.95 +2.79 ± 11.30 +4.89 ± 12.80

85.22 86.56 87.72 92.07 94.14

−0.21 ± 0.40 +0.26 ± 0.33 +0.24 ± 0.35 +0.07 ± 0.31 +0.22 ± 0.38

Fixed-Rate

1.00 4.00 7.00

+1.63 ± 3.74 +2.77 ± 3.90 +2.53 ± 4.64

33.11 54.36 94.92

−3.27 ± 0.31 −1.76 ± 0.18 −0.19 ± 0.36

16.32 28.02 45.87

−11.05 ± 4.79 −11.09 ± 6.91 −11.17 ± 3.09

87.01 92.31 94.91

−0.20 ± 0.24 −0.40 ± 0.21 −0.02 ± 0.31

Periodic Batching

−6.10 ± 1.30

−1.30 ± 0.25

−6.19 ± 4.95

−0.47 ± 0.30

Using the datasets and workload regimes in Table 2, we vary the write budget Λ to aling write percentages across strategies. These differ by datasets due to event rate and volume. We use multiple end-to-end simulations with temporal train–test splits, evaluating on all test events, including those that did not trigger persistent updates. This reflects realistic deployment conditions where inference operates on potentially stale profiles. To isolate the effect of filtering, models use only features derived from aggregations in persistent state; stateless attributes such as timestamps and categorical encodings are excluded. Thus, we directly attribute changes in ML utility to the fidelity of maintained state. Finally, given the stochastic nature of filtering and ML training and evaluation, our analysis prioritizes consistent trends over individual point estimates. Effect on ML Performance. In Table 5 we report the write volume and recall differential relative to an unfiltered baseline at a fixed False Positive Rate (FPR) of 1%, a standard operating point in anomaly detection [49]. Across datasets and operating regimes, persistence-path control largely preserves the performance of unfiltered pipelines despite substantial reductions in persistence volume. At moderate filtering levels (e.g., 3–7% write rates in transactional settings), recall improvements are sometimes observed, while aggressive filtering eventually degrades performance. This yields a non-monotonic trade-off, with a broad operating region where substantial write reduction is achieved with limited or no loss in predictive quality. While recall improvements are observed in several

workloads, other datasets primarily exhibit performance preservation under substantial write reduction, indicating that the benefits of adaptive filtering are workload-dependent rather than uniform. We attribute performance improvements to implicit regularization induced by probabilistic thinning, i.e. filtering reduces the dominance of high-velocity keys, whiles stochastic suppression can inject controlled noise into feature construction. This is consistent with known effects in stochastic regularization [10, 40, 57, 72]. However, we emphasize that improvements are workload-dependent rather than universal.

Approximation Fidelity and Variance Reduction. In Figure 10 we show the feature approximation error measured on sum aggregates, which are particularly sensitive to missed updates and therefore represent a worst-case proxy for fidelity. Across datasets, approximation error decreases monotonically with increasing write volume. Even under substantial filtering, errors remain bounded, indicating that recursive aggregates degrade gracefully under stochastic update suppression. Furthermore, variance-aware filtering (green lines) consistently improves approximation fidelity relative to standard persistence-path control (blue lines), at comparable write rates. Notably, persistence-path control remains competitive with the fullstream in-memory benchmark (orange lines), and in several regimes achieves comparable or lower approximation error despite relying exclusively on persistence-backed control statistics.

Peres et al.

Figure 10: Sum feature aggregation approximation error (average and 95th percentile) across datasets, under varying filtering strategies and write persistence operations. Baselines. Full-stream control achieves similar recall and approximation quality to persistence-path control, suggesting that persistence-backed control statistics are sufficient to drive effective filtering decisions without requiring high-frequency in-memory state or coordination. Also, naive fixed-rate filtering performs inconsistently across datasets, since it disproportionately under-samples high-activity keys, leading to degraded feature fidelity and weaker performance recall. Finally, periodic batching introduces systematic degradation in ML performance and feature aggregation accuracy across datasets, delaying state updates and inducing feature staleness.

Discussion

We have demonstrated that the tight coupling between inference and persistent state updates in streaming ML systems is not fundamental. By treating persistence updates as probabilistic events, rather than mandatory side effects of inference, we show that it is possible to substantially reduce write frequency, serialization overhead, and storage I/O while largely preserving (and in some workloads improving) downstream ML utility. Under skewed workloads, the marginal statistical value of successive updates for high-activity entities diminishes rapidly, whereas their systems cost grows linearly. The persistence-path control design avoids high-frequency inmemory control planes, coordination across workers, and additional fault-tolerance mechanisms. Despite operating on filtered control statistics, the resulting stochastic process is self-correcting, preserves unbiasedness, and enforces predictable write-rate bounds with only bounded oversampling. In our experiments, we show that persistence-path control remains competitive with full-stream

in-memory coordination while operating exclusively on persistencebacked statistics. As a result, control-plane overhead scales with the write budget rather than the raw event rate, aligning naturally with the execution and recovery models of modern streaming engines. From a broader systems perspective, our approach can be interpreted as introducing a probabilistic control layer within stateful stream processing. This is complementary to traditional complex event processing (CEP) and streaming dataflow systems, where operators typically assume deterministic state transitions. In contrast, persistence-path control relaxes this assumption by allowing controlled stochasticity in state updates, while preserving statistical correctness at the aggregate level. More generally, the results suggest that certain classes of stateful streaming workloads may tolerate substantially reduced persistence intensity before approximation effects materially impact downstream inference quality. While the performance gains are significant, several architectural considerations and broader implications warrant discussion. First, the benefits of probabilistic filtering are workload-dependent: some datasets exhibit measurable improvements in downstream ML utility, while others primarily maintain baseline performance under substantial write reduction. Second, the stochastic nature of filtering introduces bounded approximation error in maintained state, which may or may not be acceptable depending on application requirements. In domains such as fraud detection, these errors are often secondary to system responsiveness under load; however, in contexts involving compliance, financial auditing, or legal reporting, discarding updates may not be permissible. In such scenarios, our architecture can be deployed in a tiered fashion [36]: applying probabilistic filtering for low-latency inference while maintaining an unfiltered, strictly consistent path for auditability.

Decoupling Inference and State Updates

Future Extensions. Promising directions for future research include: • Integrating streaming sketches and probabilistic data structures that are explicitly aware of probabilistic downsampling. This would enable richer feature profiles (e.g., quantiles, distinct counts) while maintaining statistical guarantees under filtered updates. • Developing principled approaches to variance reduction, including analytical calibration of parameter 𝛼 in Equation 4 under different data distributions and feature objectives, as well as alternative formulations that better prioritize highimpact events when low approximation error is required. • Extending persistence-path control to more heterogeneous streaming deployments involving shared storage layers, multi-tenant contention, and distributed operator coordination.

References [1] Wei Dai 0003, Yi Zhou, Nanqing Dong, Hao Zhang 0025, and Eric P. Xing. 2019. Toward Understanding the Impact of Staleness in Distributed Machine Learning. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. https://openreview.net/ forum?id=BylQV305YQ [2] Daniel J. Abadi, Yanif Ahmad, Magdalena Balazinska, Uğur Çetintemel, Mitch Cherniack, Jeong-Hyon Hwang, Wolfgang Lindner, Anurag Maskey, Alexander Rasin, Esther Ryvkina, Nesime Tatbul, Ying Xing, and Stanley B. Zdonik. 2005. The design of the Borealis stream processing engine. In Proceedings of the Conference on Innovative Data Systems Research. http://cidrdb.org/cidr2005/papers/ P23.pdf [3] Avinash Achar, Dhivya Bharathi, Bachu Anil Kumar, and Lelitha Vanajakshi. 2019. Bus arrival time prediction: A spatial Kalman filter approach. IEEE Transactions on Intelligent Transportation Systems 21, 3 (2019), 1298–1307. https: //api.semanticscholar.org/CorpusID:182478323 [4] Erik R. Altman. 2019. Synthesizing credit card transactions. https://arxiv.org/ abs/1910.03033 arXiv:1910.03033. [5] Arvind Arasu, Brian Babcock, Shivnath Babu, Mayur Datar, Keith Ito, Itaru Nishizawa, Justin Rosenstein, and Jennifer Widom. 2003. STREAM: the Stanford stream data manager. In Proceedings of the ACM SIGMOD International Conference on Management of Data. 665. doi:10.1145/872757.872854 [6] Michael Armbrust, Tathagata Das, Joseph Torres, Burak Yavuz, Shixiong Zhu, Reynold Xin, Ali Ghodsi, Ion Stoica, and Matei Zaharia. 2018. Structured streaming: a declarative API for real-time applications in Apache Spark. In Proceedings of the ACM SIGMOD International Conference on Management of Data. 601–613. doi:10.1145/3183713.3190664 [7] Brian Babcock, Shivnath Babu, Mayur Datar, Rajeev Motwani, and Jennifer Widom. 2002. Models and issues in data stream systems. In Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems. 1–16. doi:10.1145/543613.543615 [8] Joseph Berkson. 1944. Application of the logistic function to bio-assay. Journal of the American statistical association 39, 227 (1944), 357–365. doi:10.1080/01621459. 1944.10500699 [9] Giuseppe Bianchi and Ilenia Tinnirello. 2003. Kalman filter estimation of the number of competing terminals in an IEEE 802.11 network. In IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No. 03CH37428), Vol. 2. IEEE, 844–852. http://www.aicit.org/ijact/ppl/8.pdf [10] Christopher M. Bishop. 1995. Training with noise is equivalent to Tikhonov regularization. Neural Computation 7, 1 (1995), 108–116. doi:10.1162/neco.1995.7. 1.108 [11] R. G. Brown. 1956. Exponential smoothing for predicting demand. Little. [12] Paris Carbone, Stephan Ewen, Gyula Fóra, Seif Haridi, Stefan Richter, and Kostas Tzoumas. 2017. State management in Apache Flink: consistent stateful distributed stream processing. Proceedings of the VLDB Endowment 10, 12 (2017), 1718–1729. doi:10.14778/3137765.3137777 [13] Valeria Cardellini, Francesco Lo Presti, Matteo Nardelli, and Gabriele Russo. 2022. Runtime adaptation of data stream processing systems: the state of the art. Comput. Surveys 54, 11S (2022), 1–36. doi:10.1145/3514496 [14] Tony F. Chan, Gene H. Golub, and Randall J. LeVeque. 1983. Algorithms for computing the sample variance: analysis and recommendations. The American Statistician 37, 3 (1983), 242–247. https://api.semanticscholar.org/CorpusID: 3593421

[15] Edith Cohen and Martin Strauss. 2003. Maintaining time-decaying stream aggregates. Journal of Algorithms 59 (2003), 223–233. doi:10.1016/j.jalgor.2005.01.006 [16] Graham Cormode, Minos Garofalakis, Peter J. Haas, and Chris Jermaine. 2012. Synopses for massive data: samples, histograms, wavelets, sketches. Foundations and Trends in Databases 4, 1–3 (2012), 1–294. doi:10.1561/1900000004 [17] Graham Cormode, Flip Korn, and Srikanta Tirthapura. 2008. Exponentially decayed aggregates on data streams. In 2008 IEEE 24th International Conference on Data Engineering. IEEE, 1379–1381. doi:10.1109/ICDE.2008.4497562 [18] Graham Cormode and S. Muthukrishnan. 2005. An improved data stream summary: the count-min sketch and its applications. Journal of Algorithms 55, 1 (2005), 58–75. doi:10.1016/j.jalgor.2003.12.001 [19] Andrea Dal Pozzolo, Olivier Caelen, Yann-Ael Le Borgne, Serge Waterschoot, and Gianluca Bontempi. 2014. Learned lessons in credit card fraud detection from a practitioner perspective. Expert systems with applications 41, 10 (2014), 4915–4928. doi:10.1016/j.eswa.2014.02.026 [20] Daryl J. Daley and David Vere-Jones. 2003. An introduction to the theory of point processes: volume I: elementary theory and methods. Springer. doi:10.1007/b97277 [21] Niv Dayan, Manos Athanassoulis, and Stratos Idreos. 2017. Monkey: Optimal Navigable Key-Value Store. In Proceedings of the 2017 ACM International Conference on Management of Data (Chicago, Illinois, USA) (SIGMOD ’17). Association for Computing Machinery, New York, NY, USA, 79–94. doi:10.1145/3035918.3064054 [22] Siying Dong, Andrew Kryczka, Yanqin Jin, and Michael Stumm. 2021. RocksDB: evolution of development priorities in a key-value store serving large-scale applications. ACM Transactions on Storage 17, 4 (2021), 1–32. doi:10.1145/3483840 [23] Nick Duffield, Yunhong Xu, Liangzhen Xia, Nesreen K. Ahmed, and Minlan Yu. 2017. Stream aggregation through order sampling. In Proceedings of the ACM International Conference on Information and Knowledge Management. 909–918. doi:10.48550/arXiv.1703.02693 [24] Pavlos S Efraimidis and Paul G Spirakis. 2006. Weighted random sampling with a reservoir. Information processing letters 97, 5 (2006), 181–185. doi:10.5555/ 1138831.1711169 [25] Brian S. Everitt and Anders Skrondal. 2010. The Cambridge dictionary of statistics. Cambridge University Press. https://books.google.pt/books?id= C98wSQAACAAJ [26] Mohamed Amine Ferrag, Othmane Friha, Djallel Hamouda, Leandros Maglaras, and Helge Janicke. 2022. Edge-IIoTset: A new comprehensive realistic cyber security dataset of IoT and IIoT applications for centralized and federated learning. IEEe Access 10 (2022), 40281–40306. [27] Philippe Flajolet, Éric Fusy, Olivier Gandouet, and Frédéric Meunier. 2007. Hyperloglog: the analysis of a near-optimal cardinality estimation algorithm. Discrete Mathematics and Theoretical Computer Science (2007). doi:10.46298/dmtcs.3545 [28] John Geweke. 1989. Bayesian inference in econometric models using Monte Carlo integration. Econometrica (1989), 1317–1339. https://ideas.repec.org/a/ ecm/emetrp/v57y1989i6p1317-39.html [29] Ana Sofia Gomes, João Oliveirinha, Pedro Cardoso, and Pedro Bizarro. 2021. Railgun: managing large streaming windows under MAD requirements. Proceedings of the VLDB Endowment 14, 12 (2021), 3069–3082. doi:10.14778/3476311.3476384 [30] J. M. Hammersley and D. C. Handscomb. 1964. The general nature of Monte Carlo methods. In Monte Carlo Methods. 1–9. doi:10.1007/978-94-009-5819-7_1 [31] Jie Han and Michael Orshansky. 2013. Approximate computing: an emerging paradigm for energy-efficient design. In Proceedings of the IEEE European Test Symposium. 1–6. doi:10.1109/ETS.2013.6569370 [32] Daniel G. Horvitz and Donovan J. Thompson. 1952. A generalization of sampling without replacement from a finite universe. J. Amer. Statist. Assoc. 47 (1952), 663–685. https://api.semanticscholar.org/CorpusID:120274071 [33] J. Stuart Hunter. 1986. The exponentially weighted moving average. Journal of Quality Technology 18, 4 (1986), 203–210. doi:10.1002/9780470400531 [34] Martin Jacobsen. 2006. Point process theory and applications: marked point and piecewise deterministic processes. Springer. doi:10.1007/0-8176-4463-6 [35] Jaeyoung Jang, Sung Jun Jung, Sunmin Jeong, Jun Heo, Hoon Shin, Tae Jun Ham, and Jae W. Lee. 2020. A specialized architecture for object serialization with applications to big data analytics. In Proceedings of the ACM/IEEE International Symposium on Computer Architecture. 322–334. doi:10.1109/ISCA45697.2020. 00036 [36] Charles Jaranilla, Hojin Shin, Seehwan Yoo, Seong-je Cho, and Jongmoo Choi. 2024. Tiered Storage in Modern Key-Value Stores: Performance, StorageEfficiency, and Cost-Efficiency Considerations. In 2024 IEEE International Conference on Big Data and Smart Computing (BigComp). 151–158. doi:10.1109/ BigComp60711.2024.00032 [37] John Frank Charles Kingman. 1992. Poisson processes. Clarendon Press. [38] Jay Kreps, Neha Narkhede, and Jun Rao. 2011. Kafka: a distributed messaging system for log processing. In Proceedings of the International Workshop on Networking Meets Databases. 1–7. https://api.semanticscholar.org/CorpusID:18534081 [39] Dirk P. Kroese, Thomas Taimre, and Zdravko I. Botev. 2013. Handbook of Monte Carlo methods. Wiley. [40] David Krueger, Tegan Maharaj, János Kramár, Mohammad Pezeshki, Nicolas Ballas, Nan Rosemary Ke, Anirudh Goyal, Yoshua Bengio, Aaron Courville, and

Peres et al.

Chris Pal. 2017. Zoneout: regularizing RNNs by randomly preserving hidden activations. https://arxiv.org/abs/1606.01305 arXiv:1606.01305. [41] Ove Daae Lampe and Helwig Hauser. 2011. Interactive visualization of streaming data with kernel density estimation. In 2011 IEEE pacific visualization symposium. IEEE, 171–178. [42] Yann-A Le Borgne and Gianluca Bontempi. 2004. Machine learning for credit card fraud detection-practical handbook. ACM SIGKDD explorations newsletter 6, 1 (2004), 1–6. [43] Tim Lindholm, Frank Yellin, Gilad Bracha, and Alex Buckley. 2014. The Java virtual machine specification: Java SE 8 edition. Addison-Wesley Professional. https://api.semanticscholar.org/CorpusID:60794508 [44] Lanyue Lu, Thanumalayan Sankaranarayana Pillai, Hariharan Gopalakrishnan, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. 2017. WiscKey: Separating Keys from Values in SSD-Conscious Storage. ACM Trans. Storage 13, 1, Article 5 (March 2017), 28 pages. doi:10.1145/3033273 [45] Shadi A. Noghabi, Kartik Paramasivam, Yi Pan, Navina Ramesh, Jon Bringhurst, Indranil Gupta, and Roy H. Campbell. 2017. Samza: stateful scalable stream processing at LinkedIn. Proceedings of the VLDB Endowment 10, 12 (2017), 1634– 1645. doi:10.14778/3137765.3137770 [46] Patrick O’Neil, Edward Cheng, Dieter Gawlick, and Elizabeth O’Neil. 1996. The log-structured merge-tree (LSM-tree). Acta Informatica 33, 4 (1996), 351–385. doi:10.1007/s002360050048 [47] Sandeep Pamarthi. 2024. Real-time state management techniques using RocksDB: a high-performance approach to scalable stream processing. International Journal of Science and Research Archive (2024). doi:10.30574/ijsra.2024.12.1.0867 [48] Emanuel Parzen. 1962. On estimation of a probability density function and mode. The Annals of Mathematical Statistics 33, 3 (1962), 1065–1076. http: //dx.doi.org/10.1214/aoms/1177704472 [49] Iker Perez, Jason Wong, Piotr Skalski, Stuart Burrell, Richard Mortier, Derek McAuley, and David Sutton. 2023. Locally differentially private embedding models in distributed fraud prevention systems. In Proceedings of the IEEE International Conference on Data Mining Workshops. 475–484. doi:10.1109/ICDMW60847. 2023.00068 [50] Martin Potthast. 2010. Crowdsourcing a Wikipedia vandalism corpus. In 33rd Annual International ACM SIGIR Conference. ACM, 789–790. [51] Do Le Quoc, Ruichuan Chen, Pramod Bhatotia, Christof Fetzer, Volker Hilt, and Thorsten Strufe. 2017. Streamapprox: approximate computing for stream analytics. In Proceedings of the ACM/IFIP/USENIX Middleware Conference. 185–197. doi:10.1145/3135974.3135989 [52] Karthik Ramasamy. 2019. Unifying messaging, queuing, streaming, and lightweight compute for online event processing. In Proceedings of the ACM International Conference on Distributed and Event-Based Systems. 5. doi:10.1145/3328905. 3338224 [53] Stuart W Roberts. 2000. Control chart tests based on geometric moving averages. Technometrics 42, 1 (2000), 97–101. doi:10.2307/1271439 [54] Murray Rosenblatt. 1956. Remarks on some nonparametric estimates of a density function. The Annals of Mathematical Statistics 27 (1956), 832–837. doi:10.1007/ 978-1-4419-8339-8_13 [55] Bianca Schroeder, Adam Wierman, and Mor Harchol-Balter. 2006. Open versus closed: a cautionary tale. In Proceedings of the 3rd Conference on Networked Systems Design & Implementation - Volume 3 (San Jose, CA) (NSDI’06). USENIX Association, USA, 18. doi:10.5555/1267680.1267698 [56] Ahmad Slo, Sukanya Bhowmik, and Kurt Rothermel. 2020. State-aware load shedding from input event streams in complex event processing. Transactions on Big Data 8, 5 (2020), 1340–1357. [57] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 15, 1 (2014), 1929–1958. http: //jmlr.org/papers/v15/srivastava14a.html [58] Sasu Tarkoma, Christian Esteve Rothenberg, and Eemil Lagerspetz. 2012. Theory and practice of bloom filters for distributed systems. IEEE Communications Surveys and Tutorials 14, 1 (2012), 131–155. doi:10.1109/SURV.2011.031611.00024 [59] Nesime Tatbul, Uğur Çetintemel, Stan Zdonik, Mitch Cherniack, and Michael Stonebraker. 2003. Load shedding in a data stream manager. In Proceedings of the VLDB Conference. 309–320. doi:10.1016/B978-012722442-8/50035-5 [60] Nesime Tatbul and Stan Zdonik. 2006. Window-aware load shedding for aggregation queries over data streams. In Proceedings of the 32nd international conference on Very large data bases. 799–810. http://dl.acm.org/citation.cfm?id=1164196 [61] The Locust Team. 2024. Locust: an open source load testing tool. https://locust.io/ Version 2.24.0. [62] Daniel Ting. 2022. Adaptive threshold sampling. In Proceedings of the ACM SIGMOD International Conference on Management of Data. 1612–1625. doi:10. 1145/3514221.3526122 [63] Surya T. Tokdar and Robert E. Kass. 2010. Importance sampling: a review. Wiley Interdisciplinary Reviews: Computational Statistics 2, 1 (2010), 54–60. doi:10.1002/ wics.56 [64] Juliane Verwiebe, Philipp M. Grulich, Jonas Traub, and Volker Markl. 2023. Survey of window types for aggregation in stream processing systems. The VLDB Journal

32, 5 (2023), 985–1011. doi:10.1007/s00778-022-00778-6 [65] Jeffrey S. Vitter. 1985. Random sampling with a reservoir. ACM Trans. Math. Software 11, 1 (1985), 37–57. doi:10.1145/3147.3165 [66] Richard M. Vogel, Simon Michael Papalexiou, Jonathan R. Lamontagne, and Flannery C. Dolan. 2025. When Heavy Tails Disrupt Statistical Inference. The American Statistician 79, 2 (2025), 221–235. doi:10.1080/00031305.2024.2402898 [67] Greg Welch and Gary Bishop. 1995. An introduction to the Kalman filter. University of North Carolina at Chapel Hill, Department of Computer Science (1995). https://api.semanticscholar.org/CorpusID:14848980 [68] B. P. Welford. 1962. Note on a method for calculating corrected sums of squares and products. Technometrics 4, 3 (1962), 419–420. doi:10.1080/00401706.1962. 10490022 [69] Christopher Whitrow, David J Hand, Piotr Juszczak, David Weston, and Niall M Adams. 2009. Transaction aggregation as a strategy for credit card fraud detection. Data mining and knowledge discovery 18, 1 (2009), 30–55. doi:10.1007/s10618008-0116-z [70] Kenny Wong, David Sutton, Iker Perez, and Alec Barns-Graham. 2024. Training a machine learning system for transaction data processing. US Patent 12,118,559. [71] Bo Zhao, Nguyen Quoc Viet Hung, and Matthias Weidlich. 2020. Load shedding for complex event processing: Input-based and state-based techniques. In 36th International Conference on Data Engineering (ICDE). IEEE, 1093–1104. [72] Richard Zur, Yulei Jiang, Lorenzo Pesce, and Karen Drukker. 2009. Noise injection for training artificial neural networks: a comparison with weight decay and early stopping. Medical Physics 36 (2009), 4810–4818. doi:10.1118/1.3213517

Properties of Filtered Estimators

Let E define a realization of fixed size from a Marked Inhomogeneous Counting Process E = {𝑒 1, 𝑒 2, . . . , 𝑒 𝑁 },

𝑒𝑛 = (𝑞𝑛 , 𝑡𝑛 ),

where 𝑡𝑛 ∈ R+ are arrival times drawn from a time-varying intensity function 𝜆(𝑡), with 𝑡 𝑁 = 𝑡. And consider the aggregate ∑︁ 𝐴(𝑡) = 𝑤 (𝑡, 𝑒𝑛 ). 𝑛=1,...,𝑁

for an arbitrary function 𝑤 (𝑡, 𝑒𝑛 ) at time 𝑡 > 0.

A.1

The Estimator is Unbiased

Let 𝑝𝑛 ∈ (0, 1] for 𝑛 = 1 . . . , 𝑁 be a sequence of random variables and 𝑍𝑛 ∼ Bernoulli(𝑝𝑛 ). We show that: ∑︁ 𝑍𝑛 ˆ = 𝐴(𝑡) 𝑤 (𝑡, 𝑒𝑛 ) · , 𝑝𝑛 𝑛=1,...,𝑁 is an unbiased estimator of 𝐴(𝑡), assuming that 𝑤 (𝑡, 𝑒𝑛 ) is a fixed sequence over 1, . . . , 𝑁 independent of 𝑝𝑛 , for all 𝑛 ≤ 𝑁 . Proof. Let F𝑛−1 = 𝜎 (𝑍 1, . . . , 𝑍𝑛−1 ) denote the natural filtration containing Bernoulli trials information available immediately before event 𝑒𝑛 , such that 𝑝𝑛 is is predictable. Note that " # ∑︁ ∑︁ 𝑍𝑛 𝑍𝑛 ˆ E 𝐴(𝑡) =E 𝑤 (𝑡, 𝑒𝑛 ) · = 𝑤 (𝑡, 𝑒𝑛 ) · E . 𝑝 𝑝𝑛 𝑛 𝑛=1,...,𝑁 𝑛=1,...,𝑁 By the law of total expectation, we derive ∑︁ 𝑍𝑛 ˆ E 𝐴(𝑡) = 𝑤 (𝑡, 𝑒𝑛 ) · E E F𝑛−1 = 𝐴(𝑡). 𝑝𝑛 𝑛=1,...,𝑁 □

Decoupling Inference and State Updates

A.2

Estimator Variance

which defines a smoothed, past-weighted-average count with bandwidth ℎ > 0. Note that, via Campbell’s Theorem, it holds ∫ 𝑡 −𝑠 1 𝑡 ˆ exp − E[𝜆(𝑡)] = 𝜆(𝑠) 𝑑𝑠 ℎ −∞ ℎ

We show that 1 ˆ Var 𝐴(𝑡) = 𝑤 2 (𝑡, 𝑒𝑛 ) E −1 . 𝑝𝑛

which defines a convolution of the true intensity 𝜆(𝑠) and an ˆ to be unbiased, the following exponential decay kernel. For 𝜆(𝑡) integral equation must hold: ∫ 𝑡 −𝑠 1 𝑡 exp − 𝜆(𝑡) = 𝜆(𝑠) 𝑑𝑠. ℎ −∞ ℎ

Proof. Define 𝑋𝑛 ≔ 𝑤 (𝑡, 𝑒𝑛 )

𝑍𝑛 , 𝑝𝑛

s.t. E[𝑋𝑛 ] = 𝑤 (𝑡, 𝑒𝑛 ). By definition: 2 ˆ ˆ Var 𝐴(𝑡) = E 𝐴(𝑡) − 𝐴(𝑡) ∑︁ = E (𝑋𝑛 − 𝑤 (𝑡, 𝑒𝑛 )) 2 {z } 𝑛 |

Above, differentiation under the integral sign implies 𝜆 ′ (𝑡) = 0. Thus, unbiased estimates are only retrieved in homogeneous settings with constant rate. In time-varying, inhomogeneous settings, the convolution operation introduces a smoothing effect.

V(𝑋𝑛 )

Unbiased Local Weighted Counts. . The estimator is widely used ˆ is a low-variance, approximately in practice because the term ℎ· 𝜆(𝑡) unbiased estimate of the count 𝑁 (𝑡 − ℎ, 𝑡). Indeed, ∫ ∞ ∫ ℎ ˆ E[ℎ𝜆(𝑡)] − E[𝑁 (𝑡 −ℎ, 𝑡)] = 𝑒 −𝑢/ℎ 𝜆(𝑡 −𝑢) 𝑑𝑢 − 𝜆(𝑡 −𝑢) 𝑑𝑢,

∑︁ ∑︁ 𝑛

E [(𝑋𝑛 − 𝑤 (𝑡, 𝑒𝑛 )) (𝑋𝑖 − 𝑤 (𝑡, 𝑒𝑖 ))] . {z } 𝑖≠𝑛 | Cov(𝑋𝑛 ,𝑋𝑖 )

Furthermore, " V(𝑋𝑛 ) = E[𝑋𝑛2 ] − E[𝑋𝑛 ] 2 = E

𝑍𝑛 𝑤 (𝑡, 𝑒𝑛 ) 𝑝𝑛

− 𝑤 2 (𝑡, 𝑒𝑛 )

2 𝑍 = 𝑤 2 (𝑡, 𝑒𝑛 ) E 𝑛2 − 1 𝑝𝑛 2 𝑍𝑛 2 = 𝑤 (𝑡, 𝑒𝑛 ) E E 2 F𝑛−1 − 1 𝑝𝑛 1 = 𝑤 2 (𝑡, 𝑒𝑛 ) E −1 . 𝑝𝑛 Additionally, " E

𝜆(𝑡 − 𝑢) = 𝜆(𝑡) − 𝑢𝜆 ′ (𝑡) + 𝑢2 𝜆 ′′ (𝑡) + · · · yields the leading-order approximation ℎ2 ′ 𝜆 (𝑡). 2 Hence the bias is 𝑂 (ℎ 2 ), small when 𝜆 varies slowly relative to ℎ. Furthermore, note that ∫ ∞ ˆ V ℎ𝜆(𝑡) = 𝑒 −2𝑢/ℎ 𝜆(𝑡 − 𝑢) 𝑑𝑢. ˆ E[ℎ𝜆(𝑡)] − E[𝑁 (𝑡 − ℎ, 𝑡)] ≈ −

# ∑︁ ∑︁

(𝑋𝑛 − 𝑤 (𝑡, 𝑒𝑛 )) (𝑋𝑖 − 𝑤 (𝑡, 𝑒𝑖 ))

For constant 𝜆, this becomes ∫ ∞ ℎ ˆ V ℎ𝜆(𝑡) =𝜆 𝑒 −2𝑢/ℎ 𝑑𝑢 = 𝜆 , 2 0

𝑛 𝑖≠𝑛

∑︁ ∑︁

which vanishes when 𝜆 is constant. For smoothly varying 𝜆, a Taylor expansion

E [(𝑋𝑛 − 𝑤 (𝑡, 𝑒𝑛 )) (𝑋𝑖 − 𝑤 (𝑡, 𝑒𝑖 ))] ,

𝑛 𝑖<𝑛

allowing us to focus on E [(𝑋𝑛 − 𝑤 (𝑡, 𝑒𝑛 )) (𝑋𝑖 − 𝑤 (𝑡, 𝑒𝑖 ))] for 𝑖 < 𝑛 in specific. Now: E [(𝑋𝑛 − 𝑤 (𝑡, 𝑒𝑛 )) (𝑋𝑖 − 𝑤 (𝑡, 𝑒𝑖 ))] h h ii = E E (𝑋𝑛 − 𝑤 (𝑡, 𝑒𝑛 )) (𝑋𝑖 − 𝑤 (𝑡, 𝑒𝑖 )) F𝑛−1 h h ii = E (𝑋𝑖 − 𝑤 (𝑡, 𝑒𝑖 )) E (𝑋𝑛 − 𝑤 (𝑡, 𝑒𝑛 )) F𝑛−1

while the variance of the true count is given by Var[𝑁 (𝑡 −ℎ, 𝑡)] = 𝜆ℎ. Thus, the kernel estimator reduces the variance by a factor of 1/2 in the homogeneous case. Empirical evidence suggests comparable variance reduction in slowly varying inhomogeneous settings.

Stability of filtered estimation

We prove Remark 4.1 by showing that persistence-path control induces a self-correcting estimation process.

Proof. Let by noting that E[𝑋𝑛 F𝑛−1 ] = 𝑤 (𝑡, 𝑒𝑛 ).

□

Kernel Density Intensity Estimators

Again, let E define a Marked Inhomogeneous Counting Process E = {𝑒 1, 𝑒 2, . . . },

𝑒𝑛 = (𝑞𝑛 , 𝑡𝑛 ),

where 𝑡𝑛 ∈ R+ are arrival times drawn from a time-varying intensity function 𝜆(𝑡). Consider the on-line kernel density estimator: 𝑡 −𝑡 ∑︁ 𝑛 ˆ =1 𝜆(𝑡) exp − , ℎ > 0, ℎ 𝑡 <𝑡 ℎ 𝑛

′ F𝑛−1 = 𝜎 (𝑒 1, . . . , 𝑒𝑛−1 ; 𝑍 1, . . . , 𝑍𝑛−1 )

denote the natural filtration containing all information available immediately before the arrival of event 𝑒𝑛 and the corresponding filtering decision 𝑍𝑛 . Define the exponential decay factor 𝛽𝑛 = exp(−(𝑡𝑛 − 𝑡𝑛−1 )/ℎ) with KDE bandwidth parameter ℎ > 0. The full-stream kernel estimator for arrival rates evolves deterministically as ˆ 𝑛 ) = 𝜈ˆ (𝑡𝑛 ) , 𝜆(𝑡 ℎ

𝜈ˆ (𝑡𝑛 ) = 1 + 𝛽𝑛 · 𝜈ˆ (𝑡𝑛−1 ).

Peres et al.

Under persistence-path control, the filtered arrival rate estimator updates stochastically, i.e. 𝜈ˆ𝐹 (𝑡𝑛 ) 𝜆ˆF (𝑡𝑛 ) = , ℎ

𝜈ˆF (𝑡𝑛 ) =

Safety via oversampling

We prove Remark 4.2, showing that persistence-path control cannot reduce the expected number of persistent updates relative to full-stream control, under a naive filtering formulation.

𝑍𝑛 + 𝛽𝑛 · 𝜈ˆ𝐹 (𝑡𝑛−1 ), 𝑝𝑛

′ -measurable. where 𝑍𝑛 ∼ Bernoulli(𝑝𝑛 ) and 𝑝𝑛 is F𝑛−1 By conditioning, and noting that 𝑍𝑛 ′ E F𝑛−1 =1 𝑝𝑛

Proof. Let NF and N denote the total number of updates triggered under filtered and full-stream control, respectively. Both mechanisms apply a thinning rule

we obtain ′ E[𝜈ˆF (𝑡𝑛 ) | F𝑛−1 ] = 𝛽𝑛 · 𝜈ˆF (𝑡𝑛−1 ) + 1.

(6) 𝑝𝑛 = min 1,

We now examine the martingale property of 𝑀𝑛 =

ˆ 𝑛) 𝜆ˆF (𝑡𝑛 ) − 𝜆(𝑡 . exp(−𝑡𝑛 /ℎ)

ˆ 𝑛) 𝜆(𝑡

with user-defined threshold Λ > 0, but differ in the intensity estiˆ 𝑛 ) is deterministic mates they rely on. A full-stream estimator 𝜆(𝑡 ˆ given the arrival history, and 𝜆F (𝑡𝑛 ) is a random estimator constructed via self-normalized filtering. ˆ 𝑛 ) ≥ Λ almost surely; and the reWe consider the regime 𝜆(𝑡 sult trivially holds otherwise. The expected number of persistent updates under filtered control is

Using (6) and the deterministic update of 𝜈ˆ (𝑡𝑛 ), 𝛽𝑛 (𝜈ˆF (𝑡𝑛−1 ) − 𝜈ˆ (𝑡𝑛−1 )) ℎ exp(−𝑡𝑛 /ℎ) ˆ ˆ 𝑛−1 ) 𝜆F (𝑡𝑛−1 ) − 𝜆(𝑡 = = 𝑀𝑛−1 . exp(−𝑡𝑛−1 /ℎ)

′ E[𝑀𝑛 | F𝑛−1 ]=

Thus, {𝑀𝑛 }𝑛≥0 is a martingale.

□ "

ˆ 0 ), it follows that Assuming identical initialization 𝜆ˆF (𝑡 0 ) = 𝜆(𝑡 E[𝑀𝑛 ] = 0 for all 𝑛. This establishes that deviations between filtered and full-stream intensity estimates do not compound over time, but are damped by the exponential decay of the estimator itself. In Figure 11 we illustrates the stability of this self-correcting process over time.

E[NF ] = Λ

∑︁ 𝑛

# .

𝜆ˆF (𝑡𝑛 )

Also, the function 𝑔(𝑥) = 1/𝑥 is strictly convex for 𝑥 > 0, and by Jensen’s inequality, " E

# ≥

𝜆ˆF (𝑡𝑛 )

1 . E[𝜆ˆF (𝑡𝑛 )]

By the martingale property established in Remark 4.1, 𝜆ˆF (𝑡𝑛 ) is an unbiased estimator of the full-stream intensity, i.e. ˆ 𝑛 ). E[𝜆ˆF (𝑡𝑛 )] = 𝜆(𝑡 Through substitution, E[NF ] ≥ Λ

∑︁ 𝑛

1 = E[N ]. ˆ 𝜆(𝑡𝑛 )

Thus, persistence-path control can only oversample relative to full-stream control in expectation. □

Figure 11: Top: Arrival intensity estimations under persistence-path and full-stream approaches. Bottom: Estimation difference across both approaches over a long period of time.

Variance reduction sensitivity analysis

We explore the impact of the variance reduction parameter 𝛼 in equation (4) over simulated financial transaction data. We sample transaction amounts from multiple Pareto and Log-normal distributions, and simulate our persistence path mechanism with variance reduction. We compute the relative error between estimated sum features aggregates and their true unfiltered profiles.

Decoupling Inference and State Updates

15.0

0.8

Error: Pareto (1, 8)

Volume: Pareto (1, 8)

Error: Pareto (2, 8)

Volume: Pareto (2, 8)

Error: LogNorm (1.0, 1.0)

Volume: LogNorm (1.0, 1.0)

Error: LogNorm (1.0, 1.5)

Volume: LogNorm (1.0, 1.5)

12.5 10.0 7.5

0.6

5.0

0.4

2.5 0.0

0.2

Included Volume (%)

Relative Error (|Â − A|/A)

Distributions & Metrics

Furthermore, increasing the effect of variance reduction yields noticeable improvements in the quality of retrieved estimates. Finally, Figure 13 shows the relative error distribution across individual feature aggregates, for different values of 𝛼. We notice that even midly increasing the variance reduction term can effectively ensure no extreme values are ever ignored during persistence-path aggregations; i.e. variance reduction operates by increasing the inclusion probability of outlier events. Error Distributions

0.0

0.5

1.0

1.5

2.0

2.5

3.0

Variance Reduction Parameter (α)

Figure 12: Estimation relative error for a decayed sum aggregation, as a function of variance-reduction parameter 𝛼. Figure 12 shows: • The relative feature aggregation error, with 95% confidence intervals, as the value of parameter 𝛼 is increased. • The volume of persistence-path operations processed, as the value of parameter 𝛼 is increased. The approximation error is noticeably higher for transaction amount distributions with considerable skew and kurtosis, while the number of processed persistence-path operations remains unchanged.

Relative Error Distribution

−2.5 0.0

15.0

12.5 10.0

7.5 4

5.0 2.5

0.0

Included Volume (%)

Multi-Distribution Variance Reduction Sensitivity 1.0

−2.5

0 0.0

0.5

1.0

1.5

2.0

2.5

3.0

Variance Reduction Parameter (α) Relative Error Distribution

Included Volume

Figure 13: Error distribution for profile estimations depending on variance reduction 𝛼 parameter.

Decoupling Inference from State Updates in Low-Latency Feature Engines via Probabilistic Thinning

Related documents