arXiv CSopen access

AAFLOW: Scalable Patterns for Agentic AI Workflows

arXiv:2605.02162v1 [cs.DC] 4 May 2026

AAFLOW: Scalable Patterns for Agentic AI Workflows Arup Kumar Sarker

Mills Staylor

Aymen Alsaadi

Department of Computer Science, Biocomplexity Institute and Initiative University of Virginia Charlottesville, VA, USA [email protected]

Department of Computer Science, Biocomplexity Institute and Initiative University of Virginia Charlottesville, VA, USA [email protected]

Department of Computer Science, Rutgers University New Brunswick, NJ, USA [email protected]

Gregor von Laszewski

Shantenu Jha

Geoffrey Fox

Biocomplexity Institute and Initiative University of Virginia Charlottesville, VA, USA [email protected]

Department of Computer Science Rutgers University Princeton Plasma Physics Laboratory Princeton, NJ, USA [email protected]

Department of Computer Science, Biocomplexity Institute and Initiative University of Virginia Charlottesville, VA, USA [email protected]

Abstract—Agentic workflows in large language model systems integrate retrieval, reasoning, and memory, but existing frameworks suffer from scalability and reproducibility limitations due to fragmented data orchestration, serialization overhead, and non-deterministic execution. Although these frameworks increase flexibility, they don’t have a formal execution model that adheres to the principles of high-performance computing. We introduce AAFLOW, a unified distributed runtime that creates communication-efficient execution plans by modeling agentic workflows as an operator abstraction. Using Apache Arrow and Cylon, AAFLOW creates a zero-copy data plane that allows direct interoperability between preprocessing, embedding, and vector retrieval without the need for serialization overhead. To lower coordination costs, it uses resource-deterministic scheduling and asynchronous batching. While retaining comparable LLM generation throughput, experimental results demonstrate up to 4.64× pipeline speedup and 2.8× gains in embedding and upsert phases. Rather than LLM inference acceleration, these advantages result from enhanced data flow, batching, and communication efficiency. Index Terms—Agentic AI, Retrieval-Augmented Generation (RAG), Large Language Models (LLMs), Distributed Data Processing, Cylon, LlamaIndex, High-Performance Computing (HPC), Data Orchestration, Reproducibility.

I. I NTRODUCTION Large language models (LLMs) are increasingly deployed within agentic workflows, where multiple components—including retrieval, reasoning, tool invocation, and memory—are dynamically orchestrated to solve complex tasks in scientific computing [1]. By adding adaptive control flow and multi-step reasoning to conventional inference pipelines, these workflows enable more expressive and context-aware AI systems. This change, however, highlights a basic systems problem: data infrastructures built for static, predictable workloads are piled on top of dynamic, non-deterministic agent

execution. There are two significant drawbacks to the current systems as a result of this mismatch. First, data orchestration bottlenecks dominate performance. Retrieval-Augmented Generation (RAG) pipelines require large-scale preprocessing, including document chunking, embedding generation, indexing, and vector retrieval [2], [3]. Significant data transport and nodeto-node communication are required for these processes. Due to serialization, object management, and fragmented execution pipelines, existing distributed frameworks like Dask and Spark have significant overhead, especially in communication-bound regimes [4]–[6]. Second, execution non-determinism limits reproducibility and optimization. Flexible orchestration is introduced by agentic frameworks like LangChain and LangGraph, but LLM-driven decision-making is given execution control [1]. In conventional HPC environments, this leads to dynamic execution routes that are challenging to replicate, profile, and optimize [7], [8]. Existing approaches address these issues separately. While agentic frameworks increase reasoning flexibility without taking underlying data movement and communication costs into account, distributed data systems maximize throughput for static workloads. Scaling these processes sometimes leads to incredibly disjointed architecture solutions, necessitating coordination between multiple services for retrieval, splitting, data storage, and embedding construction. Scaling costs and overall complexity are greatly increased by this [8], [9]. Consequently, agentic workflows cannot be mapped onto effective distributed execution models using a single abstraction. We argue that, similar to relational abstractions in data systems, agentic workflows can be formalized as a composition of operators. Embedding, retrieval, reasoning, memory access, and index updates are examples of agentic operations that can be mapped to well-defined distributed communication

patterns, such as broadcast, shuffle-compute, reduction, and while maintaining identical LLM generation throughput. embarrassingly parallel execution. This viewpoint allows for a fundamental change: agentic orchestration can be compiled A. Scope of Evaluation into a predictable execution plan via high-performance comWe highlight that LLM inference itself is still GPU-bound munication primitives rather than being treated as a black-box and comparable across frameworks, and AAFLOW does not process [10]. speed it up. Rather, the main bottlenecks in large-scale agentic In order to achieve this goal, we provide AAFLOW, a unified RAG systems are data orchestration, connectivity, and pipeline distributed runtime that serves as an agentic workflow compiler. execution, which may be optimized to improve performance. Using a zero-copy data plane based on Apache Arrow and But we will add and run experiments with multiple LLMs on Cylon, AAFLOW converts high-level agentic processes into GPU clusters in the future. communication-efficient execution graphs. AAFLOW allows II. A RCHITECTURAL D ESIGN for scalable and repeatable agentic pipelines by removing serialization overhead and separating logical execution from Conventional RAG systems are built as loosely connected resource scheduling. AAFLOW introduces three key design pipelines that combine the processes of generation, retrieval, emprinciples: Operator-driven execution: Agentic workflows are bedding, and preprocessing [8], [11]. These systems show three expressed as a composition of operators mapped to distributed basic drawbacks in distributed contexts, notwithstanding their communication patterns. Zero-copy data movement: Apache effectiveness at small scales: (1) fragmented data orchestration Arrow enables direct interoperability between data processing, resulting in high serialization and data movement overhead in embedding, and retrieval without serialization. Resource- distributed settings [4], [9], (2) non-deterministic execution due deterministic scheduling: Execution is decoupled from agent to dynamic agent-driven control flow [1], [12], [13], and (3) logic, enabling predictable and high-concurrency execution. lack of a unified abstraction for mapping agentic workflows to A multi-tier memory model is inherited partly from the distributed execution models [5]. The inconsistent generation data infrastructure layer, and an agentic orchestration layer of embeddings across scattered nodes causes semantic drift in in AAFLOW enables context-aware retrieval and execution vector indices. [1], [14]. The lack of a standard data abstraction continuity. In order to provide adaptive, multi-step reasoning across various data formats also causes additional serialization across sessions, the memory layer serves as a bidirectional link overhead, which slows down the training and inference stages. between agents and system state, preserving past context and intermediate outcomes. Workloads can be expressed as batchoriented tasks across transformation, embedding, and indexing stages thanks to AAFLOW’s integration of this orchestration with a distributed embedding and ingestion pipeline that uses asynchronous parallel batching and task-based execution to ensure scalability. This design decreases write amplification in vector databases, eliminates unnecessary vector operations, and permits high-concurrency execution. AAFLOW removes serialization and I/O bottlenecks by combining zero-copy data transfer with communication-efficient scheduling, offering a reliable and repeatable basis for extensive agentic workflows. This paper makes the following contributions: 1) Agentic Operator Abstraction: We introduce a formal Fig. 1. End-to-End code flow of AAFLOW with memory operation. The RAG abstraction that maps agentic workflow components context constructor uses the memory module through the retriever. (embedding, retrieval, reasoning, memory, upsert) to AAFLOW proposes a formal operator-driven execution distributed communication patterns, enabling systematic model in place of component-based architectures to overcome execution on HPC systems. 2) Unified Distributed Runtime: We design AAFLOW, these constraints. Agentic workflows are represented in this a zero-copy, communication-efficient runtime that inte- approach as collections of operators that are assembled into grates distributed data processing, embedding pipelines, execution graphs over HPC primitives that facilitate communication. and vector retrieval into a single execution model. 3) Resource-Deterministic Execution: We propose an execution model that separates logical agent behavior A. Agentic Operator Abstraction from resource scheduling, improving reproducibility and We define an agentic workflow as a set of operators that enabling high-concurrency batching. modify system state, context, and data. Every operator maps to 4) System-Level Performance Analysis: We demonstrate a particular distributed execution and communication pattern that AAFLOW reduces data movement and coordination while encapsulating a logical function. This abstraction allows overhead, achieving up to 4.64× pipeline speedup and agentic workflows to be compiled into deterministic execution 2.8× improvements in embedding and upsert stages, graphs.

Let a workflow be defined as: W = {Opembed , Opretrieve , Opreason , Opmemory , Opupsert } Each operator is defined as a tuple: Opi = (Ii , Oi , fi , Pi ) where Ii and Oi denote input and output data, fi is the transformation function, and Pi is the associated communication pattern. 1) Embedding Operator (Opembed ) The embedding operator transforms raw textual data into dense vector representations: E = Opembed (D) where D is a set of documents or chunks, and E is the corresponding embedding matrix. Fully parallel execution across nodes is made possible by the separate processing of each document or chunk. It will employ the Embarrassingly Parallel (EP) communication pattern, which makes it simple to divide workloads into smaller, independent subtasks that can be executed concurrently on several cores [15]. The benefits of batch processing and the computational cost are dominated by this operator. 2) Retrieval Operator (Opretrieve ) The retrieval operator performs distributed similarity search over partitioned vector indices: R = Opretrieve (q, E) where q is the query embedding and R is the set of retrieved top-k documents. Every partition receives the query. Local similarity scores are calculated by each node, and then global aggregation is performed. It employs a communication pattern called shuffle-compute, which is followed by a query broadcast and a partial Top-k reduction between nodes. This operator is dependent on communication and is susceptible to network latency and data partitioning. 3) Reasoning Operator (Opreason ) The reasoning operator aggregates retrieved context and produces a structured prompt or response: C = Opreason (R) For downstream LLM inference, partial contexts are created locally and merged into a single representation. Distributed context fragments are combined into a single global context using the reduction communication pattern. This operator filters and synthesizes pertinent data to assess the quality of downstream inference. 4) Memory Operator (Opmemory ) The memory operator manages persistent system state and contextual history: M ′ = Opmemory (M, C) Distributed memory stores are used for state updates and lookups. It employs a broadcast/exchange communication

pattern in which nodes selectively spread pertinent context or state updates. Adaptive behavior across sessions, context reuse, and multi-turn reasoning are all made possible by this operator. 5) Upsert Operator (Opupsert ) The upsert operator inserts or updates embeddings in distributed vector indices: E ′ = Opupsert (E) Updates for embedding are written to partitioned vector stores in batches. In order to preserve index consistency, updates are dispersed and condensed using a shuffle-reduce communication pattern. This operator directly affects ingestion throughput and controls indexing efficiency. AAFLOW is executed as a directed acyclic graph (DAG), where operators are composed as: Opembed → Opretrieve → Opreason → Opmemory → Opupsert While independent operators can be overlapped or parallelized, data dependencies specify the order of execution. This framework allows for explicit communication planning between operators, batching and data location optimization, and deterministic execution independent of LLM control flow. AAFLOW adapts dynamic agentic workflows into predictable distributed programs that may be optimized with HPC scheduling and communication primitives by mapping each operator to a known communication pattern. B. Compilation to Distributed Execution Given a workflow W, AAFLOW constructs a distributed execution plan: G = Compile(W) where data dependencies are represented by edges and operators are represented by nodes. Every operator is mapped to HPC communication primitives such RDMA-based data exchange, UCX [16] transfers, and MPI [17] collectives. This stage of compilation guarantees: 1) Deterministic execution pathways that are not dependent on LLM control flow, 2) Explicit communication structures that take the place of implicit framework coordination, 3) Partition-aware scheduling for optimized data locality. Data infrastructure, agentic orchestration, RAG, and a memory module (knowledge cache) are some of the components that make up AAFLOW, which uses distributed determinism and hybrid orchestration shown in Fig.-1. C. Data Infrastructure—Unified Zero-Copy Data Plane AAFLOW removes serialization overhead between pipeline stages by introducing a shared data plane based on Apache Arrow and Cylon [18]. When integrated into hybrid frameworks like as Deep Radical-Cylon (Deep RC) and RHAPSODY [19]– [21], Cylon provides the high-performance data base required for scalable, repeatable, and low-latency RAG pipelines operating on modern HPC and AI infrastructures. Arrow allows

columnar, zero-copy memory sharing between distributed data This formulation shows that parallelism reduces the execution preprocessing, embedding generation pipelines, and vector time of the useful work by a factor of P , while batching database indexing and retrieval, in contrast to conventional amortizes overhead by a factor of b. Across all models, frameworks that depend on object-based data transfer. In Fig. execution time can be expressed as 2, cylon’s dual-layer interface for Python and C++ guarantees Nβ Nα T ≈ + + Ω, (3) seamless integration with contemporary AI toolchains [21], P bP striking a balance between quick local execution and effective where Ω represents framework-specific overhead where Ω ≈ 0 inter-node communication for distributed workflows [22]–[24]. for ideal async batching. In traditional frameworks (Ray, Dask), Ω ≈ 0 includes serialization, task scheduling, and objectstore overhead. In contrast, AAFLOW minimizes Ω ≈ 0 by 1) eliminating serialization through zero-copy data exchange, 2) reducing coordination via operator-level batching, and 3) using explicit communication primitives instead of implicit task graphs. E. Agent-Based Retrieval-Augmented Generation The agents in AAFLOW dynamically decides when to retrieve, how to reformulate queries, and how to integrate intermediate results instead of carrying out a predetermined sequence. As a result, RAG becomes a decision-driven system with the ability to reason iteratively rather than a linear pipeline [12], [13]. Formally, given a user query q, the agent performs query Fig. 2. Cylon Layered Architecture. From the bottom-up view, the Hardware interpretation and planning, where it decides whether relayer is compatible with vendor-based or open-sourced transport layer [19] trieval is required and may decompose q into sub-queries Cylon uses network-level communication primitives based on {q1 , q2 , . . . , qn }. Each sub-query is embedded into a vector TCP, InfiniBand, and unified backends like MPI [17], UCX [16], representation and used to retrieve relevant documents from GLOO [25] and FMI [26] to satisfy the bandwidth and latency an external knowledge base (e.g., a vector database such as requirements of RAG preprocessing (Fig. 2). Data-intensive ChromaDB [29], [30], FAISS [31]). In contrast to static RAG, distributed operations that rely on these communication layers, multi-hop reasoning and better recall are made possible by such as shuffle, gather, and reduce, dominate RAG indexing the agent’s ability to iteratively adjust searches depending and embedding generation processes. The resulting abstraction on intermediate retrieval outcomes (using LlamaIndex agentic supports high-throughput, communication-aware data move- orchestration [32]). The agent then filters, ranks, and aggregates ment, which lowers I/O overhead and improves scalability in evidence in a context integration stage of processing the multi-node situations. [27], [28]. retrieved documents. This stage is essential for lowering noise and guaranteeing that the LLM receives only high-quality, D. Resource-Deterministic Execution Model task-relevant data (in Fig.-1). In reality, the agent might The fact that LLM-driven control flow determines execution additionally carry out further reasoning tasks before creation, order, which results in non-deterministic behavior, is a major including cross-document synthesis or summarization. The drawback of current agentic frameworks. This is addressed by LLM generates an output conditioned on the curated context AAFLOW by dividing: physical execution, which involves and the original query during response generation (Fig.-1). scheduling and resource allocation, and logical execution, III. I MPLEMENTATION which comprises agent decisions and reasoning stages. Batches of operator executions that are scheduled separately from the AAFLOW is a constructed distributed runtime over a zeroagent logic are used to express workloads. This allows for copy data plane that implements the operator abstraction reproducible execution traces, high-concurrency batching, and presented in Section II. The implementation converts each predictable execution delay. We consider an ingestion workload workflow into an execution DAG whose vertices are operator of N items processed in batches of size b with up to P parallel instances and whose edges are typed data dependencies, rather workers. The cost of processing a batch is modeled as than exposing orchestration as a set of loosely connected framework methods. The runtime then uses explicit communication Tbatch = α + βb, (1) primitives and batch-aware execution to schedule these operator where α denotes fixed per-request overhead and β denotes instances across distant resources and partitioned data. The implementation consists of four tightly coupled layers: per-item cost. With batching and asynchronous execution, the (1) an operator runtime, (2) a zero-copy distributed data plane, total runtime is approximated as (3) a memory-aware retrieval path, and (4) an asynchronous N Nα Nβ TAAFLOW ≈ (α + βb) = + . (2) batched execution engine. As illustrated in a layered figure in bP bP P

Fig. 3. AAFLOW design incorporating a multilayered architecture with agentic DAG execution supported by a zero-copy data plane, used by high-performance agentic execution, hosted in distributed storage, which is managed by a scalable adaptive workflow.

Fig. 3, these layers together achieve the architectural objectives of operator-driven execution, zero-copy data movement, and resource-deterministic scheduling.

a bounded context object rather than a free-form orchestration callback. Memory operator (Opmemory ). Memory is implemented by AAFLOW as a hierarchical state layer that includes longterm vectorized summaries, intermediate reasoning artifacts, and short-term interaction states. Two memory operations are carried out by the runtime: lookup before to reasoning and update following response production. While update employs batched state insertion and summary compaction, lookup follows the same partitioned retrieval path as knowledge search. Instead of treating memory as an ad hoc side channel, this gives it the same execution semantics as other operators. Upsert operator (Opupsert ). Embedding updates generated during ingestion or memory compaction are buffered and written in bulk to partitioned FAISS [31] indices. In order to minimize write amplification and per-item commit overhead, the runtime groups writes by destination shard and executes batched insertion. One of the primary causes of the observed ingestion benefits is the explicit implementation of this stage as a distributed batched write operator. Other distributed vector databases like ChromaDB [30], [33] or Pinecone [34] can also be partitioned for bulk indices.

A. Operator Runtime: Realizing the Agentic Abstraction AAFLOW materializes each abstract operator, Opembed , Opretrieve , Opreason , Opmemory , and Opupsert , as a concrete runtime primitive with explicit input/output schemas, partitioning semantics, and communication behavior. Embedding operator (Opembed ). Tokenization, chunking, and embedding generation are applied independently over each partition by the runtime after it receives document partitions created by preprocessing. A batched map over distributed Arrow [9] /Cylon [18] tables is used to implement this operator. The runtime arranges this stage as embarrassingly 4. Advanced Async with Parallel Pipeline of AAFLOW. With batching parallel work among workers because each chunk is embedded Fig. to measure execution time. BE = Size of the batch embeddings; BU = Size independently. To increase performance while maintaining of the upsert batching. TT otal = Load T ime + T ransf orm T ime(t1 + deterministic partition ownership, embedding models are called t2 ) + Index T ime using fixed-size micro-batches. Retrieval operator (Opretrieve ). The input query is embedded by the runtime and sent to partition-local vector indices B. Zero-Copy Data Plane and Distributed Storage AAFLOW employs Cylon as the distributed dataframe for query-time execution. Every worker compares its shard of the knowledge index or memory index to the local top-k substrate for partitioned execution and Apache Arrow as candidates. Local candidates are then globally merged by the the in-memory format for records, chunks, embeddings, and runtime to create a final ranked set. Instead of using implicit metadata. Between the preprocessing, embedding, retrieval, framework-managed object passing, this stage is implemented and indexing phases, this solution eliminates intermediary as a distributed top-k query with explicit partition-aware routing Python-object serialization. The runtime passes Arrow-backed buffers directly between stages [6], [18], rather than converting and reduction. Reasoning operator (Opreason ). A structured reasoning across framework-specific containers. This is significant since state is created from the recovered context. Before final AAFLOW’s architecture aims to eliminate serialization and aggregation, locally acquired evidence is sorted, filtered, and object-store overhead, two significant bottlenecks in distributed put together into a condensed context representation. AAFLOW RAG pipelines, in addition to parallelism. generates a deterministic context payload for downstream LLM Partitioned tabular datasets with raw text, chunk identifiers, inference by treating this step as a reduction over recovered source metadata, and routing information are used to store fragments. Therefore, reasoning in the implementation is a document collections. Embeddings are attached as columnar typed runtime stage that receives ranking evidence and outputs vector fields after chunking, and they stay in layouts suitable

with Arrow until they are committed to the vector index. To prevent repeated reconstruction during query execution, embeddings are maintained alongside metadata needed for subsequent retrieval, such as provenance, memory scope, and partition location. FAISS is the main similarity-search backend used by AAFLOW for vector storage. A knowledge index for static corpus retrieval and a memory index for historical interaction context are the two logical indices that are kept. Because both indices are divided across workers, the same distributed routing architecture may be used for both retrieval and update activities. This implementation detail is crucial because it simplifies scheduling and enhances repeatability by aligning the retrieval and memory operators with the same communication plan.

New contexts are selectively promoted into memory by the runtime after generation. While contextual embeddings and reusable summaries are compacted and added to the memory index, short-lived execution traces stay local. Controlling memory development and preventing needless upsert overhead depend on this judicious promotion. Therefore, in terms of implementation, memory is an organized and planned statemanagement operator rather than a passive cache. E. Asynchronous Batched Execution Engine

The key performance-critical portion of the implementation is the asynchronous execution engine used for ingestion and index building, described in Fig. 4. The pipeline is divided into four distinct steps by AAFLOW: Load, Transform, Embed, and Upsert. The runtime uses stage-local worker pools and C. Compiled Workflow Execution constrained queues to connect these instead of considering At runtime, an agentic workflow is first lowered into an them as synchronous barriers. Input files are divided and execution DAG: transformed into Arrow/Cylon tables during the load stage. G = (V, E), Partition localization and low-overhead ingestion are prioritized where each vertex v ∈ V corresponds to an instantiated in this step. The transform stage. applies chunk creation, noroperator and each edge e ∈ E represents a typed dependency malization, metadata alignment, and delimiter-based splitting between operator outputs and inputs. The compiler assigns each across partitions. This CPU-focused stage generates the chunk operator instance to a resource domain based on its execution records that embedding workers use. Chunks are transmitted to specialized embedding workers in fixed-size micro-batches characteristics: during the embed stage. While separate workers allow overlap • preprocessing and metadata transforms → CPU-distributed with previous and later stages, batching amortizes kernel launch partitions, and framework overhead. In the upsert stage, embedding • embedding generation → CPU-backed batched workers, outputs are combined into bigger write batches and added to • retrieval and merge → distributed vector shards + reducpartitioned FAISS shards. The runtime optimizes index-write tion, efficiency and underlying hardware utilization independently • reasoning/context assembly → bounded aggregation stage, by separating embedding batch size from upsert batch size. • memory updates and index insertion → batched distributed Bounded queues between stages and persistent worker writes. pools are used to implement this asynchronous design. While What sets AAFLOW apart from framework-level orches- bounded queues impose backpressure across the pipeline and tration is this reduction step. The runtime determines "how" stop unchecked expansion in the intermediate state, persistent a retrieval or memory action is carried out, but the LLM workers avoid recurring startup overhead. In practice, this may determine "what" is required. To put it another way, means that AAFLOW maintains operator-level determinism physical scheduling is not directly determined by logical agent while overlapping data movement, embedding, and indexing. action. Rather, a predetermined operator plan is carried out by the scheduler, guaranteeing stable performance traces, explicit IV. P ERFORMANCE E VALUATION communication, and repeatable stage boundaries. A. Experimental Setup D. Memory-Aware Retrieval Path Our assessment attempts to evaluate the systems claims Retrieval is implemented by AAFLOW as a dual-path query presented in Sections I–III. We quantify improvements in over persistent memory and static knowledge. Upon receiving preparation, embedding, retrieval, memory access, and index a question, the runtime creates a query embedding and sends updating stages, which dominate end-to-end execution in it to the memory index and the knowledge index [3], [35]. A distributed RAG systems, rather than trying to speed up LLM weighted ranking policy over semantic score, source type, and decoding itself, in accordance with the scope specified in recency is used to merge the resultant candidate sets. As a result, Section I. a composite context is created that incorporates both prior inThe parallel queue is used in experiments on the teraction state and factual foundation. Three classes of memory large-scale academic HPC cluster, where each node offers artifacts are supported by the implementation: persistent long- 40 CPU cores. To rigorously isolate data orchestration and term memory: vectorized interaction history or compressed framework communication overhead from GPU-bound model summaries; intermediate results: previously retrieved chunks, computation, we employ a synthetic systems microbenchmark. partial reasoning outputs, and generated context fragments; and Rather than using large, instruction-tuned LLMs that would agent state: previous tool calls, decisions, and summaries. heavily bottleneck the pipeline with raw compute time, we

TABLE I RAG P IPELINE B ENCHMARK ON 32768 TOKENS GENERATED FROM 256 DOCUMENTS WITH T UNED S TREAMING AAFLOW. A LL METRICS ARE MEASURED IN SECONDS AND TPS = T OKENS / S Load

Trans.

TPS

Embed

Upsert

Total

LangChain LangGraph CrewAI AutoGen AAFLOW

0.0236 0.0179 0.0108 0.0126 0.0113

0.0057 0.0075 0.0100 0.0102 0.0115

94823 93712 93955 94579 96556

1.1489 1.1396 1.1453 1.1362 0.4856

0.1403 0.1364 0.1326 0.1352 0.0488

1.6447 1.6142 1.6255 1.6135 0.8748

Note: TPS represents the aggregate, cluster-wide throughput across all parallel workers executing the lightweight DistilGPT-2 proxy to saturate the data plane.

Transform

Upsert

Embed

1 Seconds

Framework

Load

0.5

0

ain gCh Lan

ph

ra gG

Lan

I

wA Cre

en

toG

Au

W FLO

AA

substitute the generation and embedding stages with ultralightweight surrogates (distilgpt2 and LocalHashEmbedder). Fig. 5. Framework benchmarking in each vital stage in the RAG pipeline By actively reducing inference time to near-zero, we force under equal parallelism. AAFLOW beats all frameworks in two vital stages the frameworks into an extreme communication-bound regime, (embed and upsert) exposing their underlying scheduling, zero-copy data plane TABLE II efficiencies, and baseline systems overhead (Ω). The system S MALL - SCALE BENCHMARK COMPARISON ACROSS PARALLEL architecture remains fully compatible with production-scale CONFIGURATIONS (10 MILLION CHUNKS , 4096 FILES ). neural models. Three scenarios are used to assess AAFLOW. First, we compare end-to-end RAG execution against agentic Config Load Trans. Embed Upsert Total AAFLOW (s) (s) (s) (s) (s) (Boost) orchestration frameworks (LangChain [36], LangGraph [37], CrewAI [38], [39], AutoGen [40]) under equalized concurrency RayScalableRAG 57.636 0.032 22.914 3.151 84.136 24.12 AsyncParallelOnly 2.646 2.047 8.067 0.724 11.641 3.33 and batching configurations to isolate orchestration overhead. DaskScalableRAG 2.876 2.194 3.222 12.423 16.188 4.64 Second, we benchmark ingestion pipelines across multiple HigressRAG 0.890 2.108 1.765 0.634 4.439 1.28 distributed baselines (RayDataScalableRAG [41], AsyncParAAFLOW 0.952 1.277 1.507 0.437 3.487 1 allelOnly [42], DaskScalableRAG [43], HigressRAG [44]) to AAFLOW vs DaskScalableRAG and HigressRAG, total improvement: 4.64× and 1.28× evaluate stage-wise execution behavior. Third, we evaluate faster with 16 logical workers. Note that AAFLOW’s total execution time is less than the sum of its individual stages. This directly validates the effectiveness of our asynchronous retrieval and response performance, including semantic cache pipeline in successfully overlapping stage computation and masking latency. lookup, hybrid retrieval, and non-cached queries. Across all experiments, we measure three system-level properties: • Upsert: 0.0488s vs. 0.13–0.14s • Pipeline efficiency: stage-wise latency for Load, TransThe AAFLOW execution model explains these enhancements. form, Embed, and Upsert, In order to minimize per-request overhead (α in Eq. 1), embed• Retrieval performance: latency and effectiveness of dings are first processed in fixed-size micro-batches utilizing query-time execution, persistent workers. Second, write amplification and commit • Scalability: behavior under increasing parallelism (strong overhead are decreased by grouping upsert operations into scaling). Every system makes use of the same hardware configuration, bigger batched writes. Third, intermediary serialization between vector storage (FAISS), and embedding models. As a result, pipeline steps is eliminated via zero-copy data transmission. As execution-model variations are responsible for the observed illustrated in Fig. 5, token throughput is almost the same and gains, which include decreased serialization, increased batching Load and Transform stages are comparable across frameworks. Consequently, Embed and Upsert dominate the performance efficiency, and less coordination overhead. gap, indicating that end-to-end performance is determined by B. Framework Benchmarking for RAG Pipeline orchestration efficiency rather than model inference. AAFLOW is compared to LangChain, LangGraph, CrewAI, and AutoGen under equal parallelism in Table I. With an end-to- C. Hybrid Parallel Ingestion Pipeline Benchmarking end runtime of 0.8748 seconds, AAFLOW outperforms baseline The ingestion performance of various distributed configframeworks, which range from 1.6135 to 1.6447 seconds. urations is assessed in Table II. AAFLOW outperforms Relative to LangChain, the speedup is: DaskScalableRAG (16.188s), AsyncParallelOnly (11.641s), and HigressRAG (4.439s) with the lowest overall runtime (3.487s). 1.6447 ≈ 1.88× The Transform, Embed, and Upsert stages show the biggest 0.8748 gains: Since token throughput is still similar across frameworks, this • Transform: 1.277s vs. 2.108s (HigressRAG) improvement cannot be attributed to quicker token production. • Embed: 1.507s vs. 1.765s Instead, the gains arise from ingestion-heavy stages: • Upsert: 0.437s vs. 0.634s • Embed: 0.4856s vs. 1.13–1.15s

These gains follow directly from the execution model in Eq. (2)–(3). AAFLOW reduces total runtime by: • amortizing fixed overhead (α) through batching, • increasing parallelism (P ) via persistent workers, • minimizing framework overhead (Ω) through zero-copy data exchange and explicit communication. Conversely, RayScalableRAG, Dask and AsyncParallelonly experience increased Ω as a result of object-store overhead, job scheduling, and serialization. Although HigressRAG uses partial pipeline overlap to save runtime, stage-level synchronization still results in coordination costs. In the end-to-end pipeline, AAFLOW achieves: • 24.12× speedup over RayScalableRAG, • 4.64× speedup over DaskScalableRAG, • 1.28× speedup over HigressRAG. This indicates that enhancing ingestion performance is mostly achieved by reorganizing the execution model rather than by boosting raw parallelism.

AsyncParallelOnly HigressRAG

DaskScalableRAG AAFLOW

Load

Transform 1.5

10

1

Time (s)

10

101 100

1024

512

256

128

1024

512

256

128

100.5

Upsert

Embed 102

Time (s)

102

101

101

D. Strong and Weak Scaling of Parallel Ingestion

We compare AAFLOW and HigressRAG [44] on retrieval and reasoning operations. HigressRAG is treated as a thinner retrieval path derived from the Higress AI gateway abstraction [45]. Table III compares AAFLOW and HigressRAG across four scenarios: LLM generation (LLMG), non-cached complex queries (NCCQ), hybrid retrieval (HR), and semantic cache lookup (SCL). AAFLOW consistently reduces total latency: • LLMG: 68.23ms → 28.12ms (58.8% reduction)

1024

512

256

128

1024

512

No of Workers

No of Workers

Fig. 6. Strong scaling behavior across configurations for Load, Transform, Embed, Upsert operations. 100 million chunks (a 92 GB synthetic corpus generated from wikitext2_train) are used from 4096 files. Each node has 40 cores, totaling 128–1024 workers are used in multiple runs (e.g., 7 nodes * 40 cores = 280 workers) TABLE III R ESPONSE AND R ETRIEVAL B ENCHMARK R ESULTS WITH D ISTRIBUTED FAISS. Engine

Scenario

Retrieval (ms)

Mem. (ms)

LLM (ms)

Total (ms)

AAFLOW HigressRAG AAFLOW HigressRAG AAFLOW HigressRAG AAFLOW HigressRAG

LLMG LLMG NCCQ NCCQ HR HR SCL SCL

1.48 21.55 2.08 22.16 1.26 21.40 0.00 0.00

0.02 0.00 0.02 0.00 0.02 0.00 0.00 0.00

26.53 46.61 27.97 48.05 0.00 0.00 0.00 0.00

28.12 68.23 30.18 70.31 1.33 21.45 0.03 0.03

Here, LLM Generation = LLMG, Non-Cached Complex Query = NCCQ, Hybrid Retrieval = HR, Semantic Cache Lookup = SCL, No. of query = 2048

NCCQ: 70.31ms → 30.18ms (57.1% reduction) HR: 21.45ms → 1.33ms (93.8% reduction) There are two reasons behind these improvements. In order to minimize unnecessary data transfer, AAFLOW first performs retrieval over partitioned indices utilizing explicit routing and reduction. Second, intermediary conversions during query execution are eliminated by the zero-copy data plane. The 58.8% reduction in LLMG stage latency is due to the zero-copy data plane eliminating serialization and I/O staging overhead before the LLM engine begins generation, rather than an increase •

E. Retrieval and Reasoning Performance

256

100 128

We evaluate strong and weak scaling behavior by increasing the number of workers from 128 to 1024 (Fig. 6 and Fig. 7). AAFLOW reduces end-to-end latency from 30.944s to 4.505s (first graph in Fig. 8), achieving over 1.34× improvement compared to the nearest baseline in strong scaling. For weak scaling, end-to-end latency experiences an expected moderate increase (from 3.064s to 5.185s, second graph in Fig. 8) due to the global reduction overhead (Ω) scaling as a function of worker count. However, it maintains a much shallower degradation curve than baselines. AAFLOW performs ingestion as a non-blocking pipeline with bounded queues and stage-local worker pools, in contrast to conventional frameworks that rely on stage barriers and centralized coordination. This eliminates synchronization overhead and permits overlap between the Load, Transform, Embed, and Upsert stages. Consequently: • Load remains approximately constant, • Transform scales efficiently with CPU parallelism with minimal communication overhead, • Embed benefits from batched CPU execution, • Upsert latency decreases due to buffered writes. These findings validate the efficiency of operator-level scheduling and asynchronous batching by demonstrating that AAFLOW sustains near-linear scaling until the bottleneck moves from coordination overhead to intrinsic computation cost.

•

AsyncParallelOnly HigressRAG

improving retrieval latency and end-to-end response time.

DaskScalableRAG AAFLOW

Load

V. R ELATED W ORKS

Transform

Distributed data systems, workflow runtimes, LLM programming frameworks, LLM serving systems, and memory100.5 augmented retrieval architectures are five closely connected fields of research that are pertinent to AAFLOW. Distributed data systems and dataframe execution: 100 100.5 Effective preprocessing, partitioning, and communication-aware data transfer are essential for large-scale AI pipelines. While Dask [43] offers lightweight task-graph execution for Pythonnative analytics, Apache Spark [46] and Apache Flink [47] offer general-purpose distributed execution for batch and streaming Upsert Embed workloads. Modin provides a crucial precedent for treating high-level abstractions as compilable execution plans rather 1 1 than framework callbacks [48], arguing that scalable dataframe 10 10 systems should be based on a clear data model and algebra rather than ad hoc API-level parallelization. By focusing on 100 communication-efficient, Arrow-based distributed execution 100 for HPC and hybrid cloud environments [6], [18], [21], [27], 10−1 [49], Cylon and associated high-performance dataframe work further extend this direction. Building on this system’s legacy, AAFLOW focuses on agentic RAG processes rather than just dataframe programming. No of Workers No of Workers Workflow runtimes and distributed execution control: The scheduling of diverse workloads across dispersed resources Fig. 7. Weak scaling behavior across configurations for Load, Transform, is the subject of a second field of study. While ZMQ, Parsl, and Embed, and Upsert operations. 95000 chunks are used by each worker. Each RADICAL-Pilot concentrate on task-based workflow execution node has 40 cores, totaling 128–1024 workers are used in multiple runs (e.g. for HPC and heterogeneous platforms [24], [50], [51], Ray 7 nodes * 40 cores = 280 workers) offers a flexible runtime for distributed AI applications [41]. OneFlow and Google Pathways both investigate asynchronous AsyncParallelOnly DaskScalableRAG distributed execution for big machine learning programs [52], HigressRAG AAFLOW [53]. An agent-specific operator algebra that maps retrieval, Strong Scaling Total Weak Scaling Total reasoning, memory, and index updates onto communication patterns is not exposed by these systems, despite the fact that 101.5 they offer crucial execution substrates. In contrast, AAFLOW 102 compiles agentic processes into explicit operator DAGs with zero-copy data transfer and partition-aware scheduling. 1 10 LLM programming and agent frameworks: Multi-step 101 orchestration, tool use, and agent coordination are made possible at the application layer by LangChain [36], Lang100.5 Graph [37], CrewAI [38], [39], and AutoGen [40]. These frameworks facilitate the creation of agents, although they usually handle execution as a runtime orchestration issue rather No of Workers No of Workers than a communication-aware systems issue. By treating LM pipelines as declarative computational graphs and assembling Fig. 8. Strong and Weak scaling behavior across configurations for Total them into optimal LM programs, DSPy advances this trend. execution time. 95000 chunks are used by each worker. Each node has 40 This approach is spiritually similar to our operator-centric view cores, totaling 128–1024 workers are used in multiple runs (e.g. 7 nodes * 40 of agentic workflows [54]. By reorienting the emphasis from cores = 280 workers) prompt/program specification to distributed execution and data flow, AAFLOW enhances such systems. in underlying token generation speed. Semantic cache lookup LLM serving runtimes and inference systems: The latency is consistent across systems, indicating that execution execution of LLM serving itself is optimized by another efficiency, not caching variations, is the source of benefits. significant class of related technologies. By using PagedAttenThese findings show that AAFLOW maintains compatibility tion and effective KV-cache management, vLLM increases with current vector search and generation components while serving throughput and shows that careful runtime design 256

512

1024

128

512

1024

256

512

1024

128

256

512

1024

256

512

1024

128

256

512

1024

Time (s)

Time (s)

256

128 128

128

Time (s)

101

may significantly increase model serving efficiency [55]. Through a language/runtime co-design [56], SGLang also aims to execute structured language-model programs efficiently. Although these systems are quite important, their main focus is on organized LM execution on accelerators and model-serving efficiency. AAFLOW, on the other hand, concentrates on the surrounding distributed pipeline—preprocessing, retrieval, memory access, and index maintenance—where coordination and data movement account for the majority of overall costs. Retrieval-augmented generation and memory systems: To enhance factual grounding, classical RAG systems integrate generation and retrieval [14]. This model is expanded with more sophisticated retrieval and memory mechanisms in more recent study. Persistent or structured memory can enhance long-context and multi-step reasoning, as demonstrated by MemoRAG [57], RAG-Tuned-LLM [58], HippoRAG [59], and Cue RAG [60]. HigressRAG [44] concentrates on retrieval-path optimization using routing, hybrid retrieval, and semantic cache lookup. AAFLOW complements existing methods by offering a distributed systems architecture where retrieval, memory, and index updates are handled as first-class operators with explicit communication and batching semantics, as opposed to merely introducing a new retrieval heuristic. In general, current research has improved either the runtime layer of distributed systems (task execution, dataframe processing, model serving) or the application layer of agentic AI (prompting, orchestration, memory design). By treating agentic workflows as compilable distributed programs, AAFLOW bridges these two viewpoints by integrating resourcedeterministic scheduling, zero-copy data transfer, and operatordriven execution into a unified runtime. VI. D ISCUSSION AND F UTURE E XPERIMENTS The evaluation results show that rather than LLM inference acceleration, the main performance increases of AAFLOW come from enhancements in data movement, batching efficiency, and execution scheduling. Token throughput is similar across frameworks, as Table I and Table II demonstrate, although there are notable decreases in the embedding and upsert stages. This demonstrates that pipeline orchestration, not model execution, is the primary bottleneck in agentic RAG systems. The findings support the execution model presented in Section II from a systems perspective. The total runtime can be expressed as: Nβ Nα + + Ω, P bP where batching reduces per-request overhead (α), parallelism reduces the time spent on useful work (β), and framework overhead (Ω) captures serialization and coordination costs. By decreasing Ω through zero-copy data exchange and boosting effective batching (b) and parallelism (P ) through persistent workers and asynchronous execution, AAFLOW enhances performance. A crucial systems insight is further highlighted by the experimental results: greater parallelism by itself does not ensure better performance. Frameworks like AsyncParallel T ≈

and DaskScalableRAG only show increased overhead as a result of synchronization hurdles, object management, and job scheduling. By reorganizing execution around operatorlevel scheduling, clear communication patterns, and stage overlap, AAFLOW, on the other hand, delivers more efficiency. This minimizes redundant data transportation, cuts down on idle time, and improves compute and memory resource use. Improvements are centered on retrieval execution and communication-bound stages (Embed and Upsert), which is another significant finding. This is consistent with the architectural design, which minimizes network overhead and eliminates intermediary serialization through partition-aware routing and zerocopy data transfer. The efficiency of asynchronous batching and restricted pipeline execution is demonstrated by the strong and weak scaling findings (Fig. 6– 8), which further confirm that AAFLOW maintains near-linear scaling until computation becomes the major cost. A. Future Experiments Several extensions are required to further characterize system behavior, even if the current evaluation validates the fundamental design ideas. To separate the contribution of each operator (Opembed , Opretrieve , Opreason , Opmemory , Opupsert ), we first intend to carry out operator-level micro-benchmarking. This will make it possible to validate communication patterns (broadcast, shuffle, reduce) under different data distributions and to attribute performance gains more precisely. Second, in order to determine how indexing structures and partitioning techniques affect retrieval latency and ingestion performance, we will assess AAFLOW across several vector storage backends (such as FAISS, ChromaDB, and other distributed indices). Third, by contrasting execution with and without the memory operator, we hope to examine the effects of memory integration. This includes measuring the overhead of memory management and compaction as well as evaluating latency, retrieval quality, and system stability under multi-step query workloads. Fourth, in order to precisely measure serialization overhead, network transfer time, and coordination latency across frameworks, we shall do communication-cost profiling. This will improve the causal relationship between system design and observed performance gains by directly measuring the Ω term in Eq. (3). Lastly, we intend to compare execution traces under identical workloads and analyze variance across repeated runs in order to assess repeatability and execution determinism. In scientific computer contexts, where reliable execution and constant performance are essential, this is especially crucial. The influence of operator-driven execution and zero-copy data movement on large-scale agentic workflows will be better understood thanks to these upcoming experiments. VII. C ONCLUSION The unified distributed runtime AAFLOW, which reformulates agentic RAG workflows as operator-driven execution graphs over high-performance communication primitives, is presented in this paper. AAFLOW bridges the gap between adaptable agentic orchestration and effective distributed systems

design by offering an agentic operator algebra, a zero-copy data plane, and a resource-deterministic execution model. Without changing LLM inference, the evaluation shows that AAFLOW achieves significant performance improvements—up to 4.64× in ingestion pipelines and 1.88× in end-to-end RAG execution. Rather than modifications to model computation, these benefits result from less serialization overhead, enhanced batching efficiency, and lower coordination costs. AAFLOW ensures effective parallel execution by reducing synchronization costs and permitting stage overlap, as demonstrated by both strong and weak scaling results. This work emphasizes a change in system architecture for AI processes in a broader sense. AAFLOW aggregates agentic pipelines into organized, communication-aware execution plans rather than handling them as dynamic, framework-driven processes. This makes it possible to achieve scalability, repeatability, and predictable performance—all crucial for implementing agentic AI systems in high-performance computing and scientific settings. AAFLOW shows that changing the surrounding data and execution infrastructure—rather than speeding up model inference—is the key to expanding agentic workflows. AAFLOW offers a framework for creating effective, scalable, and repeatable AI systems by combining data processing, retrieval, memory, and reasoning under a unified execution paradigm.

[10] A. Alsaadi, L. Ward, A. Merzky, K. Chard, I. Foster, S. Jha, and M. Turilli, “Radical-pilot and parsl: Executing heterogeneous workflows on hpc platforms,” in 2022 IEEE/ACM Workshop on Workflows in Support of Large-Scale Science (WORKS). IEEE, 2022, pp. 27–34. [Online]. Available: https://doi.org/10.1109/WORKS56498.2022.00009 [11] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, S. Riedel, and D. Kiela, “Retrieval-augmented generation for knowledge-intensive nlp tasks,” in Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 9459–9474. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/ 2020/file/6b493230205f780e1bc26945df7481e5-Paper.pdf [12] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. R. Narasimhan, and Y. Cao, “React: Synergizing reasoning and acting in language models,” in The eleventh international conference on learning representations, 2023. [Online]. Available: https://openreview.net/pdf?id=WE_vluYUL-X [13] N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao, “Reflexion: language agents with verbal reinforcement learning,” in Proceedings of the 37th International Conference on Neural Information Processing Systems, ser. NIPS ’23. Red Hook, NY, USA: Curran Associates Inc., 2023. [Online]. Available: https://dl.acm.org/doi/10.5555/3666122.3666499 [14] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, S. Riedel, and D. Kiela, “Retrieval-augmented generation for knowledge-intensive nlp tasks,” in Proceedings of the 34th International Conference on Neural Information Processing Systems, ser. NIPS ’20. Red Hook, NY, USA: Curran Associates Inc., 2020. [Online]. Available: https://dl.acm.org/doi/abs/10.5555/3495724.3496517 [15] Q. Wang, J. Liu, X. Tang, F. Wang, G. Fu, and Z. Xing, “Accelerating embarrassingly parallel algorithm on intel mic,” in 2014 IEEE International Conference on Progress in Informatics and Computing, 2014, pp. 213–218. [Online]. Available: https: //doi.org/10.1109/PIC.2014.6972327 R EFERENCES [16] P. Shamis, M. G. Venkata, M. G. Lopez, M. B. Baker, O. Hernandez, Y. Itigin, M. Dubman, G. Shainer, R. L. Graham, L. Liss, Y. Shahar, [1] W. Ma, Y. Yang, Q. Hu, S. Ying, Z. Jin, B. Du, Z. Xing, T. Li, J. Shi, S. Potluri, D. Rossetti, D. Becker, D. Poole, C. Lamb, S. Kumar, Y. Liu et al., “Rethinking testing for llm applications: Characteristics, C. Stunkel, G. Bosilca, and A. Bouteiller, “Ucx: An open source challenges, and a lightweight interaction protocol,” 2025. [Online]. framework for hpc network apis and beyond,” in 2015 IEEE 23rd Available: https://arxiv.org/abs/2508.20737 Annual Symposium on High-Performance Interconnects, 2015, pp. 40–43. [2] E. R. Vanna Winland, “What is llamaindex ?” IBM, Tech. Rep., April [Online]. Available: https://doi.org/10.1109/HOTI.2015.13 2025. [Online]. Available: https://www.ibm.com/think/topics/llamaindex [17] L. Dalcin, R. Paz, and M. Storti, “Mpi for python,” Journal of Parallel [3] M. B. Cathy Zhang, “Optimize vector databases, enhance and Distributed Computing, vol. 65, no. 9, pp. 1108–1115, Sep. 2005. rag-driven generative ai,” Intel, Tech. Rep., March 2024. [Online]. Available: https://doi.org/10.1016/j.jpdc.2005.03.010 [Online]. Available: https://medium.com/intel-tech/optimize-vector- [18] C. Widanage, N. Perera, V. Abeykoon, S. Kamburugamuve, T. A. databases-enhance-rag-driven-generative-ai-90c10416cb9c Kanewala, H. Maithree, P. Wickramasinghe, A. Uyar, G. Gunduz, [4] M. Dugré, V. Hayot-Sasson, and T. Glatard, “Performance comparison and G. Fox, “High performance data engineering everywhere,” of dask and apache spark on hpc systems for neuroimaging,” p. e7635, in 2020 IEEE International Conference on Smart Data Services 2023. [Online]. Available: https://doi.org/10.1002/cpe.7635 (SMDS). IEEE, 2020, pp. 122–132. [Online]. Available: https: [5] H. Liang, X. Ma, Z. Liu, Z. H. Wong, Z. Zhao, Z. Meng, R. He, //doi.org/10.1109/SMDS49396.2020.00022 C. Shen, Q. Cai, Z. Han et al., “Dataflow: An llm-driven framework [19] N. Perera, A. K. Sarker, M. Staylor, G. von Laszewski, K. Shan, for unified data preparation and workflow automation in the era S. Kamburugamuve, C. Widanage, V. Abeykoon, T. A. Kanewela, of data-centric ai,” arXiv preprint arXiv:2512.16676, 2025. [Online]. and G. Fox, “In-depth analysis on parallel processing patterns Available: https://arxiv.org/abs/2512.16676 for high-performance dataframes,” Future Generation Computer [6] V. Abeykoon, P. Wickramasinghe, S. Kamburugamuve, H. Maithree, Systems, vol. 149, pp. 250–264, 2023. [Online]. Available: https: C. Widanage, N. Perera, T. A. Kanewala, A. Uyar, G. Gunduz, and //doi.org/10.1016/j.future.2023.07.007 G. Fox, “High performance dataframes from parallel processing patterns,” [20] A. K. Sarker, A. Alsaadi, A. J. Halpern, P. Tangella, M. Titov, N. Perera, in Parallel Processing and Applied Mathematics: 14th International M. Staylor, G. von Laszewski, S. Jha, and G. Fox, “Deep rc: A Conference, PPAM 2022, Gdansk, Poland, September 11–14, 2022, scalable data engineering and deep learning pipeline,” in Job Scheduling Revised Selected Papers, Part I. Springer Nature Switzerland, 2023, pp. Strategies for Parallel Processing: 28th International Workshop, JSSPP 291–304. 2025, Milan, Italy, June 3–4, 2025, Revised Selected Papers. Berlin, [7] B. Wang, D. Zhao, N. R. Tallent, and L. Guo, “On the Heidelberg: Springer-Verlag, 2025, p. 205–223. [Online]. Available: reproducibility limitations of rag systems,” 2025. [Online]. Available: https://doi.org/10.1007/978-3-032-10507-3_11 https://doi.org/10.48550/arXiv.2509.18869 [21] A. Alsaadi, M. Hooten, M. Goliyad, A. Merzky, A. Shao, M. Titov, [8] LlamaIndex, “Simplify your rag application architecture with llamainT. Wang, Y. Chen, M. Kalantzi, K. Lee et al., “Rhapsody: Execution of dex + postgresml,” https://www.llamaindex.ai/blog/simplify-your-raghybrid ai-hpc workflows at scale,” arXiv preprint arXiv:2512.20795, application-architecture-with-llamaindex-postgresml, accessed: October 2025. [Online]. Available: https://arxiv.org/abs/2512.20795 7, 2025. [22] S. K. Niranda Perera, “Architecture,” Cylon, Tech. Rep., May 2022. [Online]. Available: https://cylondata.org/docs/arch/ [9] R. K. Choudhary, “How to achieve 10x performance with vector database for llm using lancedb [23] A. K. Sarker, A. Alsaadi, N. Perera, M. Staylor, G. von and pyarrow,” https://www.rishabhxchoudhary.com/blog/ Laszewski, M. Turilli, O. O. Kilic, M. Titov, A. Merzky, S. Jha How_to_Achieve_10x_Performance_with_Vector_Database_for_LLM_using_LanceDB_and_PyArrow, et al., “Radical-cylon: A heterogeneous data pipeline for scientific accessed: October 7, 2025. computing,” in Job Scheduling Strategies for Parallel Processing.

Springer Nature Switzerland, 2024, pp. 84–102. [Online]. Available: https://doi.org/10.1007/978-3-031-74430-3_5 [24] A. Merzky, M. Turilli, M. Titov, A. Al-Saadi, and S. Jha, “Design and performance characterization of radical-pilot on leadership-class platforms,” IEEE Transactions on Parallel and amp; Distributed Systems, vol. 33, no. 04, pp. 818–829, apr 2022. [Online]. Available: https://doi.org/10.1109/TPDS.2021.3105994 [25] Facebookincubator, “Gloo: Collective communications library with various primitives for multi-machine training,” Facebook, Tech. Rep., March 2023. [Online]. Available: https://github.com/facebookincubator/ gloo" [26] M. Staylor, A. K. Sarker, G. von Laszewski, G. Fox, Y. Cheng, and J. Fox, “Combining serverless and high-performance computing paradigms to support ml data-intensive applications,” Frontiers in High Performance Computing, 2026. [27] K. Shan, N. Perera, D. Lenadora, T. Zhong, A. Kumar Sarker, S. Kamburugamuve, T. Amila Kanewela, C. Widanage, and G. Fox, “Hybrid cloud and hpc approach to high-performance dataframes,” in 2022 IEEE International Conference on Big Data (Big Data), 2022, pp. 2728–2736. [Online]. Available: https://doi.org/10.1109/ BigData55660.2022.10020958 [28] N. Perera, A. K. Sarker, K. Shan, A. Fetea, S. Kamburugamuve, T. A. Kanewala, C. Widanage, M. Staylor, T. Zhong, V. Abeykoon, G. von Laszewski, and G. Fox, “Supercharging distributed computing environments for high-performance data engineering,” Frontiers in High Performance Computing, vol. Volume 2 - 2024, 2024. [Online]. Available: https://doi.org/10.3389/fhpcp.2024.1384619 [29] K. Hong, A. Troynikov, and J. Huber, “Context rot: How increasing input tokens impacts llm performance,” Chroma, Tech. Rep., July 2025. [Online]. Available: https://trychroma.com/research/context-rot [30] K. Hong, A. Troynikov, J. Huber, and M. McGuire, “Generative benchmarking,” Chroma, Tech. Rep., April 2025. [Online]. Available: https://trychroma.com/research/generative-benchmarking [31] J. Johnson, M. Douze, and H. Jégou, “Billion-scale similarity search with gpus,” IEEE Transactions on Big Data, vol. 7, no. 3, pp. 535–547, 2021. [Online]. Available: https://doi.org/10.1109/TBDATA.2019.2921572 [32] J. Liu, “Llamaindex: Data framework for connecting large language models to data,” LlamaIndex, Tech. Rep., 2023. [Online]. Available: https://github.com/jerryjliu/llama_index [33] B. Smith and A. Troynikov, “Evaluating chunking strategies for retrieval,” Chroma, Tech. Rep., July 2024. [Online]. Available: https://trychroma.com/research/evaluating-chunking [34] P. Developers, “Pinecone: Scalable vector database for machine learning applications,” Pinecone Systems Inc., Tech. Rep., May 2023. [Online]. Available: https://www.pinecone.io/ [35] A. Karpathy, “From rag to agents: Building intelligent systems with memory and tools,” Eureka Labs, Tech. Rep., January 2022. [Online]. Available: https://karpathy.ai/agents-rag-memory [36] E. Davis, “Building custom ai workflows using langchain tools,” ThinkTide Global Research Journal, vol. 5, no. 4, pp. 54–62, 2024. [Online]. Available: https://thinktidejournal.com/index.php/TGRJ/article/ view/53/63 [37] L. Developer, “Langgraph: Stateful multi-agent workflows,” LangChain Inc, Tech. Rep., Jan 2024. [Online]. Available: https://blog.langchain.com/ langgraph-multi-agent-workflows [38] Z. Duan and J. Wang, “Exploration of llm multiagent application implementation based on langgraph+ crewai,” arXiv preprint arXiv:2411.18241, 2024. [Online]. Available: https://doi.org/10.32388/R27SW4 [39] CrewAI Team, Kickoff Crew Asynchronously, 2026, accessed: 2026-02-18. [Online]. Available: https://docs.crewai.com/en/learn/kickoff-async [40] Q. Wu, G. Bansal, J. Zhang, Y. Wu, B. Li, E. E. Zhu, L. Jiang, X. Zhang, S. Zhang, A. Awadallah, R. W. White, D. Burger, and C. Wang, “Autogen: Enabling next-gen llm applications via multi-agent conversation,” in COLM 2024, August 2024. [Online]. Available: https: //www.microsoft.com/en-us/research/publication/autogen-enablingnext-gen-llm-applications-via-multi-agent-conversation-framework/ [41] P. Moritz, R. Nishihara, S. Wang, A. Tumanov, R. Liaw, E. Liang, M. Elibol, Z. Yang, W. Paul, M. I. Jordan et al., “Ray: A distributed framework for emerging {AI} applications,” in 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), 2018, pp. 561–577. [Online]. Available: https://www.usenix.org/system/files/osdi18-moritz.pdf

Developers, Parallel Execution Ingestion [42] LlamaIndex Pipeline, LlamaIndex, 2024, accessed: 2026-02-13. [Online]. Available: https://developers.llamaindex.ai/python/examples/ingestion/ parallel_execution_ingestion_pipeline/ [43] M. Rocklin, “Dask: Parallel computation with blocked algorithms and task scheduling,” in Proceedings of the 14th python in science conference, vol. 130. Citeseer, 2015, p. 136. [Online]. Available: https://proceedings.scipy.org/articles/Majora-7b98e3ed-013.pdf [44] W. Lin, “Higress-rag: A holistic optimization framework for enterprise retrieval-augmented generation via dual hybrid retrieval, adaptive routing, and crag,” arXiv preprint arXiv:2602.23374, 2025. [Online]. Available: https://arxiv.org/abs/2602.23374 [45] Alibaba Cloud and the Higress Authors, “Higress,” https://github.com/ alibaba/higress, 2026, gitHub repository. Accessed: 2026-01-25. [46] M. Zaharia, R. S. Xin, P. Wendell, T. Das, M. Armbrust, A. Dave, X. Meng, J. Rosen, S. Venkataraman, M. J. Franklin, A. Ghodsi, J. Gonzalez, S. Shenker, and I. Stoica, “Apache spark: a unified engine for big data processing,” Commun. ACM, vol. 59, no. 11, p. 56–65, Oct. 2016. [Online]. Available: https://doi.org/10.1145/2934664 [47] P. Carbone, A. Katsifodimos, S. Ewen, V. Markl, S. Haridi, and K. Tzoumas, “Apache flink: Stream and batch processing in a single engine,” The Bulletin of the Technical Committee on Data Engineering, vol. 38, no. 4, 2015. [Online]. Available: https://asterios.katsifodimos.com/assets/publications/flink-deb.pdf [48] M. Petersohn, S. Macke, D. Xin, W. Ma, J. K. Wittenauer, S. Hoyer, R. Marcus, M. Zaharia, and B. Recht, “Towards scalable dataframe systems,” Proceedings of the VLDB Endowment (PVLDB), vol. 13, no. 12, pp. 2033–2046, 2020. [Online]. Available: https://doi.org/10.14778/3407790.3407807 [49] A. K. Sarker and F. X. Lin, “Incremental perception on real time 3d data,” in Proceedings of the 23rd Annual International Workshop on Mobile Computing Systems and Applications, ser. HotMobile ’22. New York, NY, USA: Association for Computing Machinery, 2022, p. 68–73. [Online]. Available: https://doi.org/10.1145/3508396.3512875 [50] Y. Babuji, A. Woodard, Z. Li, D. S. Katz, B. Clifford, R. Kumar, L. Lacinski, R. Chard, J. M. Wozniak, I. Foster, M. Wilde, and K. Chard, “Parsl: Pervasive parallel programming in python,” in Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing, ser. HPDC ’19. New York, NY, USA: Association for Computing Machinery, 2019, p. 25–36. [Online]. Available: https://doi.org/10.1145/3307681.3325400 [51] ZMQ, “High-level messaging patterns,” Zeromq, Tech. Rep., October 2021. [Online]. Available: https://zguide.zeromq.org/docs/chapter2/ #High-Level-Messaging-Patterns" [52] P. Barham, A. Chowdhery, J. Dean, S. Ghemawat, S. Hand, D. Hurt, M. Isard, H. Lim, R. Pang, S. Roy et al., “Pathways: Asynchronous distributed dataflow for ml,” Proceedings of Machine Learning and Systems, vol. 4, pp. 430–449, 2022. [Online]. Available: https://proceedings.mlsys.org/paper_files/paper/ 2022/file/37385144cac01dff38247ab11c119e3c-Paper.pdf [53] J. Yuan, X. Li, C. Cheng, J. Liu, R. Guo, S. Cai, C. Yao, F. Yang, X. Yi, C. Wu et al., “Oneflow: Redesign the distributed deep learning framework from scratch,” arXiv preprint arXiv:2110.15032, 2021. [Online]. Available: https://arxiv.org/abs/2110.15032 [54] O. Khattab, A. Singhvi, P. Maheshwari, Z. Zhang, K. Santhanam, S. Vardhamanan, S. Haq, A. Sharma, T. T. Joshi, H. Moazam, H. Miller, M. Zaharia, and C. Potts, “Dspy: Compiling declarative language model calls into self-improving pipelines,” The Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https://openreview.net/pdf?id=sY5N0zY5Od [55] W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Stoica, “Efficient memory management for large language model serving with pagedattention,” in Proceedings of the 29th Symposium on Operating Systems Principles, ser. SOSP ’23. New York, NY, USA: Association for Computing Machinery, 2023, p. 611–626. [Online]. Available: https://doi.org/10.1145/3600006.3613165 [56] L. Zheng, L. Yin, Z. Xie, C. Sun, J. Huang, C. H. Yu, S. Cao, C. Kozyrakis, I. Stoica, J. E. Gonzalez, C. Barrett, and Y. Sheng, “Sglang: efficient execution of structured language model programs,” in Proceedings of the 38th International Conference on Neural Information Processing Systems, ser. NIPS ’24. Red Hook, NY, USA: Curran Associates Inc., 2024. [Online]. Available: https://dl.acm.org/doi/10.5555/3737916.3739916

[57] H. Qian, Z. Liu, P. Zhang, K. Mao, D. Lian, Z. Dou, and T. Huang, “Memorag: Boosting long context processing with global memory-enhanced retrieval augmentation,” in Proceedings of the ACM on Web Conference 2025, ser. WWW ’25. New York, NY, USA: Association for Computing Machinery, 2025, p. 2366–2377. [Online]. Available: https://doi.org/10.1145/3696410.3714805 [58] J. Wei, S. Wu, R. Liu, X. Ying, J. Shang, and F. Tao, “Tuning llms by rag principles: Towards llm-native memory,” arXiv preprint arXiv:2503.16071, 2025. [Online]. Available: https: //arxiv.org/abs/2503.16071 [59] B. J. Gutiérrez, Y. Shu, W. Qi, S. Zhou, and Y. Su, “From RAG to memory: Non-parametric continual learning for large language models,” in Forty-second International Conference on Machine Learning, 2025. [Online]. Available: https://openreview.net/forum?id=LWH8yn4HS2 [60] Y. Fu, D. Liu, B. Zhang, Z. Jiang, H. Mei, and J. Guan, “Cue rag: Dynamic multi-output cue memory under h framework for retrieval-augmented generation,” Neurocomputing, vol. 639, p. 130235, 2025. [Online]. Available: https://www.sciencedirect.com/science/article/ pii/S0925231225009075