arXiv:2604.03855v1 [cs.DB] 4 Apr 2026
VectraFlow: Long-Horizon Semantic Processing over Data and Event Streams with LLMs Shu Chen∗
Junhan Liu∗
Brown University [email protected]
Brown University [email protected]
Deepti Raghavan
Ugur Cetintemel
Brown University [email protected]
Brown University [email protected]
ABSTRACT Monitoring continuous data for meaningful signals increasingly demands long-horizon, stateful reasoning over unstructured streams. However, today’s LLM frameworks remain stateless and one-shot, and traditional Complex Event Processing (CEP) systems, while capable of temporal pattern detection, assume structured, typed event streams that leave unstructured text out of reach. We demonstrate VectraFlow, a semantic streaming dataflow engine, to address both gaps. VectraFlow extends traditional relational operators with LLMpowered execution over free-text streams, offering a suite of continuous semantic operators—filter, map, aggregate, join, group-by, and window—each with configurable throughput–accuracy tradeoffs across LLM-based, embedding-based, and hybrid implementations. Building on this, a semantic event pattern operator lifts complex event processing to unstructured document streams, combining LLM-based event extraction with NFA-based temporal rule matching for stateful reasoning over sequences of semantic events. In this demonstration, users will interact with VectraFlow’s live query interface to compose semantic pipelines over clinical document streams. Attendees will compile natural language intents into executable operator graphs, inspect intermediate stateful outputs, and observe end-to-end temporal pattern detection, from raw text to matched event cohorts.
1
INTRODUCTION
Large language models (LLMs) have become the de facto tool for interpreting unstructured data, yet most LLM pipelines remain stateless and episodic, evaluating isolated prompts over static corpora without support for persistence or reasoning that unfolds over time. Many applications, however, require signals that emerge across sequences of events rather than single documents. Detecting a deteriorating patient, escalating compliance violations, or multi-step fraud patterns demands long-horizon, stateful reasoning over unstructured streams—a capability that remains beyond both one-shot LLM prompting and even today’s agentic models. Existing systems address only part of this challenge. LLM-based frameworks such as Palimpzest [9], LOTUS [11], and DocETL [12] introduce semantic operators for unstructured data, but operate in batch or one-shot settings with no cross-record state. In contrast, complex event processing systems [3, 5, 7, 15] enable long-horizon temporal reasoning but assume structured data. Neither approach
∗ Both authors contributed equally to this research.
suffices when events must first be inferred from unstructured text before being reasoned over. In this demo, we will showcase VectraFlow [6, 10], a semantic streaming dataflow engine that bridges this gap by unifying continuous semantic processing with complex event pattern detection in a single framework. Analogous to how continuous queries extend relational operators to unbounded streams, VectraFlow lifts LLMbased semantics into continuous, pipeline operators. It extends both relational processing and event pattern detection with LLMaugmented execution over unstructured streams, enabling queries that span from record-level semantic interpretation to long-horizon temporal pattern matching. We will showcase these features via: • An interactive, end-to-end demonstration enabling users to compose and observe semantic pipelines over unstructured streams. A natural language interface compiles queries into executable operator graphs with live visibility into intermediate states. • A suite of continuous semantic operators (e.g., filter, aggregate, window) supporting tunable throughput–accuracy tradeoffs via LLM-based, embedding-based, and hybrid execution strategies. • A semantic pattern operator that extends complex event processing to unstructured text, fusing LLM-based event extraction with NFA-based matching for long-horizon temporal reasoning.
2
VECTRAFLOW OVERVIEW
VectraFlow is a semantic streaming dataflow engine that extends traditional relational operators and complex event processing with LLM-backed execution over unstructured streams. Core Architecture. Figure 1 illustrates the overall VectraFlow architecture, which is built on three layers. At the foundation, the streaming dataflow engine models computation as a directed acyclic graph (DAG) of operators through which records flow continuously from sources to sinks. Above this layer, the LLM-based semantic operator layer extends traditional relational operators with LLM-powered execution over unstructured text. This layer also incorporates sem_pattern, which encapsulates event extraction and temporal rule matching behind a single composable operator. Finally, the natural language layer bridges the gap between user intent and executable pipelines through an agentic loop of structured critique, automatic repair, and targeted user clarification. Continuous Semantic Operators. VectraFlow provides a suite of semantic operators that extend their traditional relational counterparts with LLM-powered execution over unstructured text. Operators such as Semantic Filter, Map, Aggregate, Join, and Group-By
VectraFlow Data Flow Engine
Query Synthesizer Natural
Language
Input
Filter
Map
Group-By
Pattern
Top-k
Input Data
Output Data Streams
Streams
CEP
Quer y
Synthesizer
Operator
Quer y Synthesizer
Pipeline
Synthesizer
Compiled Pattern (AST)
LLM-Powered Event Extractor
DSL Parser &
AST Compiler
Unstructured
Document Stream (Clinical Notes)
Field Extraction Mode
Direct Judgement Mode
Populating
Attributes
Binary Classification
Layer 3: Stream Abstraction & NFA Runtime Layer
LLM Critique
Confident?
Layer 2: Atomic Event Extraction Layer
Layer 1: Declarative Pattern & AST Layer
Keyed Stream
Pattern AST Object
Keyed Stream
Pattern AST Object
Patient 1’s NFA-Based Rule Detector
No
LLM Refinement
Yes Pipeline Ready
Event Stream
Keyed Stream
Current State
Pattern AST Object
State 2
Initial State Event A followed
by Event B
within x days
Final
State Real-time
Emission
Complex Event
Detection Results
Figure 1: VectraFlow Data and Event Processing Architecture.
Semantics
sem_window
Adaptive segmentation Pairwise similarity, rolling summaries, based on topic or sentior embedding-based clustering. ment shifts. Online grouping by mean- LLM assignment/creation with periodic ing with evolving catemerge–split refinement; embedding gories. clustering + LLM labels. Continuously updates Adaptive prompting (unified or deretrieval context as query composed) with LLM or embedding scope evolves. retrieval. Temporal pattern detecLLM-based event extraction + NFA rule tion over events extracted matching (single pass). from text streams.
sem_groupby
cont_rag
sem_pattern
M1 Basic LLM M2 LLM+Refine M3 Embedding
0.8
Implementation
Score
Operator
(a) Metric Comparison
1.0 0.6 0.4 0.2 0.0
F1
ARI
(b) Throughput Throughput (tuples/s)
Table 1: Sample continuous semantic operators.
Purity
1.25 1.00 0.75 0.50 0.25 0.00
M1
M2
M3
Figure 2: Semantic group-by implementations on the MiDe22 dataset. Left: comparison across F1, ARI, and Purity (★ marks the best). Right: throughput in tuples/s.
are direct analogs of their relational counterparts, preserving familiar query semantics over free-text streams. Beyond these analogs, VectraFlow introduces operators designed for dynamic, open-ended streams: Semantic Window adjusts boundaries on topic or sentiment shifts, Semantic Group-By allows categories to emerge and dissolve over time, and Continuous RAG adapts retrieval context as query scope evolves. Table 1 provides details on some of VectraFlow’s key operators (full set of operators described in [6]). Experimental Results. VectraFlow operators typically support LLM-based, embedding-based, or hybrid implementations to trade off semantic fidelity against latency and token cost. Here we illustrate this tradeoff using sem_groupby as a representative example [6], where we compare the alternative implementations on a subset of MiDe22 [14], reporting F1, ARI, Purity, and throughput (tuples/s) in Figure 2. The LLM with Refinement (M2) method periodically issues an additional refinement prompt every 10 tuples. Embedding-based (M3) grouping is fast and achieves high itemlevel F1, but its over-segmentation produces fragmented events.
Basic LLM (M1) offers moderate coherence and competitive speed, whereas LLM with Refinement (M2) improves cluster coherence metrics at the cost of lower throughput, making it the preferred choice when preserving event structure is the primary objective.
3
SEMANTIC PATTERN OPERATOR
Real-world analytics often demand long-horizon temporal reasoning over sequences of events. Traditional CEP systems [1, 4, 5, 7, 13, 15] excel in this, but they rely on structured, explicitly typed event streams, which makes them essentially unable to detect events that are hidden within unstructured free text. Conversely, while semantic relational operators effectively interpret unstructured text, they process records in isolation and cannot express cross-record temporal constraints, such as sequential ordering or time-windowed conditions. To bridge this gap, VectraFlow introduces the sem_pattern 2
operator. It extends CEP to unstructured document streams by seamlessly fusing LLM-based event extraction with automaton-based temporal pattern matching. Model and Syntax. Unlike traditional CEP systems where events are emitted by instrumented infrastructure, the atomic unit of matching in VectraFlow is a semantic event: a foundational fact extracted on demand from unstructured text by an LLM and represented as a typed, timestamped tuple with a concise semantic description usable in pattern guards. VectraFlow supports the following constructs: • Sequence (𝐴 → 𝐵 → 𝐶): events occur in order with relaxed contiguity. • Conjunction (𝐴 ∧ 𝐵): sub-patterns match in either order within a window. • Disjunction (𝐴 ∨ 𝐵): either sub-pattern suffices. • Negation (¬𝐴): bounded absence of an event type [3, 5]. • Quantifiers times(𝑛): one_or_more, or optional. • Temporal Constraint (within Δ𝑡): bounds elapsed time between the first and last match. These constructs compose freely, enabling expressive rules such as a sequence with an embedded negation and a time bound, e.g., within(𝐴 → ¬𝐵, Δ𝑡). Execution Semantics. The sem_pattern operator runs in two stages: (1) an LLM extractor emits a typed, timestamped event stream keyed by entity; and (2) an Nondeterministic Finite Automaton (NFA) rule detector evaluates CEP rules over per-entity event sequences. The operator is stateful, maintaining active partial matches across arrivals. This design separates semantic interpretation from temporal validation. Following the SASE execution model [3, 15] as adopted by FlinkCEP [5], each rule is compiled into a shared NFA with three transition types: Take (consume), Ignore (bypass), and Proceed (𝜀transition). Any incoming event satisfying an initial transition spawns a lightweight NFA instance that shares the compiled automaton while maintaining isolated runtime state (current node, matched events, and elapsed window). This compilation directly captures two key semantics. Sequence patterns implement skip-tillany-match contiguity via Ignore edges, allowing partial matches to bypass irrelevant interleaved events. Negation is compiled as a bounded stop-state: an instance survives non-forbidden events but is killed upon a forbidden one. To ensure well-defined evaluation, negated patterns are bounded by a within window (enforced at compile time), and an instance becomes a match only if it survives the full window without triggering the negative guard. Experimental Results. We evaluated sem_pattern on 256 clinical notes from MIMIC-IV [8] across five complex event patterns involving sequences, negations, and time-bounded conjunctions. We compared four configurations. Baseline incrementally aggregates incoming documents and prompts the LLM to determine, at each step, whether the temporal pattern is satisfied over the expanding narrative. Baseline (+ RAG) augments each judgment with retrieved context to reduce the accumulated narrative length. sem_pattern instead extracts typed events from each document via LLM and delegates temporal reasoning to the NFA engine, decoupling semantic extraction from pattern matching. sem_pattern (+ RAG) further focuses each extraction call on relevant passages via
retrieval before invoking the LLM. Results are reported for GPT-4omini [2], Qwen3-8B, and Qwen3-4B [16]; GPT-4o-mini is deployed on a private Azure server for secure MIMIC-IV data processing. Table 2: Efficiency-accuracy tradeoffs for semantic CEP approaches. Method
Full Context Baseline Full Context Baseline (+ RAG) sem_pattern sem_pattern (+ RAG)
Total Tokens
14.6M∗ 7.0M 5.7M 3.1M
Average F1 Score GPT-4o-mini
Qwen3-8B
Qwen3-4B
0.675 0.787 0.844 0.848
OOM∗ 0.721 0.814 0.862
OOM∗ 0.812 0.791 0.822
∗ The full-context baseline exceeded the practical GPU VRAM limits of the local Qwen deployments.
Table 2 shows that sem_pattern consistently improves the tokenaccuracy tradeoff. Relying on a single stateless LLM call to parse accumulating text, the full-context Baseline suffers from severe context bloat (14.6M tokens) and triggers Out-Of-Memory (OOM) failures on local Qwen deployments. Even when GPU memory is not a bottleneck, direct LLM pattern judgment performs poorly (F1 = 0.675). RAG improves both experimental settings, yet sem_pattern (+ RAG) consistently delivers the best efficiency and accuracy across all models. The results show that sem_pattern attains a better balance between efficiency and accuracy than full-context prompting, allowing long-horizon pattern detection with fewer tokens and improved precision.
4
DEMONSTRATION SETUP
VectraFlow exposes the full lifecycle of semantic stream processing, from natural language specification to operator execution and temporal pattern detection, through a set of tightly integrated interactive views. Together, these views allow users not only to execute queries but also to understand, debug, and refine long-horizon semantic pipelines in real time. Pipeline Authoring. Figure 3(a) shows the NL & Config view, where users describe their pipeline as a free-form natural language task over a synthetic clinical document stream derived from MIMICIV [8]. The pipeline synthesizer compiles this into an executable operator sequence in which each operator’s type and bound LLM instruction prompt are editable inline. For sem_pattern, compilation exposes two components: a formal pattern expression (e.g., SEQ(Discharge, WITHIN(NOT(FollowUp), 30 days))) and perevent LLM extraction prompts defining what the extractor looks for in each document. Users can modify any step and recompile without restarting, enabling iterative refinement of both pipeline structure and event semantics. This tight edit-compile-execute loop enables rapid exploration of alternative semantic interpretations and pattern definitions, making the impact of prompt and operator changes immediately visible in downstream results. Pipeline Inspection. Figure 3(b) shows the Query Processing view, where intermediate results are inspectable at each pipeline stage via per-operator tabs. The Pattern operator exposes two layers: the extracted event stream and the completed rule matches emitted by the NFA-based rule detector. For each emitted match, the system shows the matched pattern expression and supporting evidence 3
Figure 3: VectraFlow interactive interface. (a) NL & Config View: Natural language to executable pipeline compilation. (b) Query Processing View: Per-operator traces and intermediate outputs. (c) Report View: End-to-end execution metrics and LLM profiling. span. This makes the full transformation from unstructured documents to structured events to matched temporal patterns directly observable at every step. By exposing both semantic extraction and temporal reasoning stages, the system makes it possible to diagnose errors arising from either misinterpreted text or incorrect pattern logic, a distinction that is typically opaque in end-to-end LLM pipelines. This fine-grained visibility enables users to reason about the cost-accuracy tradeoffs of different operator configurations, and to identify bottlenecks arising from LLM invocation patterns or dataflow structure. Execution Profiling. Figure 3(c) shows the Report view, presenting per-operator execution metrics across the full pipeline. Every operator reports wall time, I/O row counts, and throughput; LLMpowered operators (e.g., Filter, Pattern) additionally surface token usage, call latency, extraction throughput, and accuracy metrics where ground truth is available.
5
[5] Apache Flink. 2024. FlinkCEP — Complex Event Processing. https://nightlies. apache.org/flink/flink-docs-release-1.20/docs/libs/cep/. Accessed: 2024. [6] Shu Chen, Deepti Raghavan, and Uğur Çetintemel. 2025. Continuous Prompts: LLM-Augmented Pipeline Processing over Unstructured Streams. arXiv preprint arXiv:2512.03389 (2025). https://arxiv.org/abs/2512.03389 [7] EsperTech. 2023. Esper Reference Documentation — Event Pattern Operators. http://esper.espertech.com/release-9.0.0/reference-esper/html/event_ patterns.html. Accessed: 2024. [8] Alistair Johnson, Lucas Bulgarelli, Tom Pollard, Steven Horng, Leo Anthony Celi, and Roger Mark. 2020. MIMIC-IV. https://physionet.org/content/mimiciv/1.0/. Accessed: 2021-08-23. [9] Chunwei Liu, Matthew Russo, Michael Cafarella, Lei Cao, Peter Baille Chen, Zui Chen, Michael Franklin, Tim Kraska, Samuel Madden, Rana Shahout, and Gerardo Vitagliano. 2025. Palimpzest: Optimizing AI-Powered Analytics with Declarative Query Processing. In Proceedings of the 15th Conference on Innovative Data Systems Research (CIDR). [10] Duo Lu, Siming Feng, Jonathan Zhou, Franco Solleza, Malte Schwarzkopf, and Uğur Çetintemel. 2025. VectraFlow: Integrating Vectors into Stream Processing. In Proceedings of the 15th Annual Conference on Innovative Data Systems Research. [11] Liana Patel, Siddharth Jha, Melissa Pan, Harshit Gupta, Parth Asawa, Carlos Guestrin, and Matei Zaharia. 2025. Semantic operators and their optimization: Enabling LLM-powered analytics. Proceedings of the VLDB Endowment (PVLDB) 18, 3 (2025), 4171–4184. https://doi.org/10.14778/3749646.3749685 [12] Shreya Shankar, Tristan Chambers, Tarak Shah, Aditya G. Parameswaran, and Eugene Wu. 2025. DocETL: Agentic query rewriting and evaluation for complex document processing. Proceedings of the VLDB Endowment 18, 9 (2025). https: //doi.org/10.14778/3746405.3746426 [13] Snowflake Inc. 2024. MATCH_RECOGNIZE: Snowflake Documentation. https: //docs.snowflake.com/en/sql-reference/constructs/match_recognize. Accessed: 2024. [14] Cagri Toraman, Oguzhan Ozcelik, Furkan Sahinuç, and Fazli Can. 2024. MiDe22: An Annotated Multi-Event Tweet Dataset for Misinformation Detection. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). ELRA and ICCL, 11283– 11295. https://aclanthology.org/2024.lrec-main.986 [15] Eugene Wu, Yanlei Diao, and Shariq Rizvi. 2006. High-Performance Complex Event Processing over Streams. In Proceedings of the 2006 ACM SIGMOD International Conference on Management of Data. 407–418. [16] An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chao Gao, Cheng Huang, Chen Lv, Chen Zheng, and Zhenyu Qiu. 2025. Qwen3 Technical Report. arXiv preprint arXiv:2505.09388 (2025).
CONCLUSIONS
VectraFlow introduces a unified framework for continuous semantic processing and event pattern detection over unstructured streams, combining LLM-based operators with stateful, long-horizon temporal reasoning. Its key innovations include a suite of continuous semantic operators and a semantic pattern operator that integrates event extraction with temporal rule matching within a single dataflow abstraction. This demo of VectraFlow enables users to compose, execute, and observe semantic pipelines over free-form clinical document streams, exposing both intermediate operator behavior and end-to-end pattern detection.
REFERENCES [1] Daniel J. Abadi, Don Carney, Uğur Çetintemel, Mitch Cherniack, Christian Convey, Sangdon Lee, Michael Stonebraker, Nesime Tatbul, and Stan Zdonik. 2003. Aurora: A New Model and Architecture for Data Stream Management. The VLDB Journal 12, 2 (2003), 120–139. [2] Josh Achiam, Scott Adler, Sandhini Agarwal, Liane Ahmad, Ilge Akkaya, Felipe L. Aleman, Daniel Almeida, Johannes Altenschmidt, Sam Altman, Shantanu Anadkat, and Rafael Avila. 2023. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774 (2023). [3] Jagrati Agrawal, Yanlei Diao, Daniel Gyllstrom, and Neil Immerman. 2008. Efficient Pattern Matching over Event Streams. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. 147–160. [4] Mert Akdere, Uğur Çetintemel, and Nesime Tatbul. 2008. Plan-Based Complex Event Detection across Distributed Sources. PVLDB 1, 1 (2008), 66–77. 4