(POSTER) From Sensors to Insight: Rapid, Edge-to-Core Application Development for Sensor-Driven Applications Komal Thareja∗ , Anirban Mandal∗ , Ewa Deelman‡ ∗ Renaissance Computing Institute, University of North Carolina Chapel Hill, NC, USA
arXiv:2605.02844v1 [cs.DC] 4 May 2026
‡ Information Sciences Institute, University of Southern California, Marina del Rey, CA, USA
Abstract—Scientists increasingly rely on sensor-based data; however transforming raw streams into insights across the edge-to-cloud continuum remains difficult due to the breadth of expertise required to coordinate the necessary data and computation flow. This paper introduces a pattern-based, AIassisted methodology for rapid development of sensor-driven applications. Using Pegasus workflows executing on the FABRIC testbed, we demonstrate a 5-step development loop that shifts workflow construction and deployment from code-first to intentfirst design. Starting from an existing Orcasound hydrophone workflow as a reusable template, we generate and refine workflows for air quality, earthquake, and soil moisture monitoring applications. We further show how these workflows extend to edge resources—including BlueField-3 DPUs and Raspberry Pis—through configuration and placement rather than workflow redesign. Our evaluation, from the perspective of a novice Pegasus user, shows that AI-assisted pattern reuse compresses multi-stage workflow development to 1–1.5 days per workflow while preserving the rigor and portability of workflow-based execution. Index Terms—Sensor-based applications, Edge-to-core computing, Scientific workflows, AI-assisted development, User productivity
I. I NTRODUCTION AND M OTIVATION Sensor-driven applications are a cornerstone of environmental monitoring, smart infrastructure, and cyber-physical systems. These applications require the execution of endto-end processing pipelines that transform raw sensor data into actionable insight across the edge-to-cloud continuum. However, moving from a scientific question (e.g., “How does local air quality impact respiratory health trends?”) to a production workflow requires deep infrastructure expertise— what we term the “blank page bottleneck.” Scientists often spend more time on execution mechanics (workflow formats, container orchestration, infrastructure provisioning) than exploring and analyzing their data. Workflow management systems like Pegasus [1] and programmable infrastructures like FABRIC [2] offer powerful abstractions for managing distributed applications, including emerging hardware accelerators such as Data Processing Units (DPUs) [3]. However, effectively leveraging these tools requires understanding configuration models, deployment options, and infrastructure interfaces. Meanwhile, AI code assistants [4] show promise for boilerplate code generation
but produce fragile, standalone scripts when unconstrained, limiting portability, provenance tracking, and fault tolerance. We address this gap through pattern-based engineering with AI assistance: users provide an AI assistant (e.g., Claude [5]) with an existing Pegasus workflow pattern and describe a new use case in natural language. The AI adapts the pattern—retaining dependency logic while swapping domain-specific components—producing a working first draft. Grounding AI in validated patterns prevents hallucinations and ensures that the generated code follows established best practices. The paper makes the following contributions: (1) a pattern-based methodology and 5-step development loop for intent-first workflow construction; (2) four case studies (air quality, earthquake, soil moisture, Orcasound) deployed across edge-to-core continuum including Edge DPUs (e.g. BlueField-3); (3) a user-centric evaluation demonstrating that AI-assisted pattern reuse compresses development from weeks to days for novice users; (4) practical insights on why Pegasus workflows are synergistic with LLM-assisted generation due to their explicit, readable, composable, and execution-agnostic nature. II. M ETHODOLOGY Our methodology combines pattern-based workflow engineering with AI-assisted development, guided by four design principles: (1) Intent-first development—scientists express data sources, analysis steps, and expected outputs in natural language rather than low-level execution mechanics; (2) Pattern reuse over greenfield design—new applications are derived by adapting existing, validated workflow patterns; (3) Separation of concerns—workflow logic, execution placement, and infrastructure provisioning are explicitly separated; (4) Human-in-the-loop AI assistance—AI tools assist with workflow generation and debugging but do not replace workflow engines or schedulers. Figure 1 summarizes the approach. Our methodology is structured around a five-step development loop: (1) Describe Science—define data sources, analysis steps, and outputs in plain English; (2) Ground with Context—provide the AI with a proven Pegasus workflow as a structural template; (3) Generate Skeleton—AI produces
Fig. 1: Pattern-based, AI-assisted 5-step development loop. executable Python code with task definitions, I/O declarations, and dependency graphs; (4) Refine—human-guided iteration to adjust resource requirements, containerization, and parallelization; (5) Scale—deploy to edge, clusters, or clouds without workflow redesign using Pegasus catalogs and HTCondor [6] placement. All applications are represented as Pegasus Directed Acyclic Graphs (DAGs) with explicit data dependencies. This representation is particularly compatible with AI-assisted development because it is explicit, readable, composable, and execution-agnostic. It is also and open source project with documentation available online for over 20 years, making commercial LLMs work out of the box. For edge-to-cloud workflows, HTCondor ClassAds steer tasks to appropriate nodes (e.g., DPU-enabled edge workers vs. CPU-only cloud workers). Infrastructure is provisioned programmatically using FABlib on the FABRIC testbed, with typical deployments including a submit node hosting Pegasus/HTCondor, edge workers near data sources, and cloud workers for computeintensive stages. III. C ASE S TUDIES We demonstrate progressive workflow reuse through a chaining pattern: Orcasound → air quality → earthquake and soil moisture. Each workflow inherits validated structure from its predecessor, compounding productivity gains across successive applications. A. Orcasound: Baseline Pattern The Orcasound hydrophone workflow [7] served as the foundational pattern, implementing a canonical sensor-driven structure: ingest → preprocess → transform → infer → aggregate. Concretely, it ingests audio segments, converts them into analysis-ready formats, performs ML inference to detect Orca vocalizations, and aggregates predictions across sensors and time windows. Its explicit dependency graph, containerized transformations, and separation of workflow logic from execution placement make it ideal for crossdomain adaptation. B. Air Quality: Incremental Evolution Starting from Orcasound, the air quality workflow [8] was built incrementally: (i) a baseline ingest–analyze–detect
Fig. 2: Air quality workflow DAG.
Fig. 3: Representative AQI forecast and anomaly detection. pipeline using OpenAQ [9] data (using 3–4 AI prompts), (ii) extension to LSTM-based ML forecasting (3–4 additional prompts), and (iii) integration of SAGE [10] sensor network data (2–3 prompts). Each extension preserved upstream logic. Early executions exposed realistic runtime errors—e.g., missing API credentials inside execution containers causing HTCondor jobs to enter Held state—which were diagnosed through AI-assisted log analysis across Pegasus, HTCondor, and container layers. Figure 2 shows the resulting workflow DAG and Figure 3 shows a representative AQI forecast output. C. Earthquake and Soil Moisture Both workflows reused the validated air quality pipeline as their immediate template. The earthquake workflow [11] replaced ingestion with USGS [12] seismic feeds and introduced conditional branching to trigger aftershock analysis, pattern detection, and hazard assessment for high-magnitude events. The soil moisture workflow [13] adapted ingestion for in-situ soil sensors and weather data, followed by feature joining and regression-based prediction. In both cases, the underlying execution model and deployment configuration remained largely unchanged. D. Edge Integration: DPUs and Raspberry Pi After validating cloud-based executions, we extended workflows to incorporate edge resources without modifying the abstract workflow. For the soil moisture workflow, I/Ointensive preprocessing stages were offloaded to BlueField3 DPUs via Pegasus site mappings, enabling experimentation with alternative placements along the edge-to-cloud continuum. A Raspberry Pi was also configured as an edge
TABLE I: Development effort per workflow stage. Workflow / Stage
Prompts
Exec. Attempts
Dev. Time
AQ: Baseline AQ: ML extension AQ: SAGE data
3–4 3–4 2–3
7 2–3 2–3
0.5–1 day ∼0.5 day ∼0.25 day
Earthquake Soil moisture
similar similar
fewer fewer
∼1 day ∼1 day
Total (all workflows)
4–5 days
participant for data ingestion and lightweight preprocessing, with downstream analytics executing on cloud resources. Both cases demonstrate that heterogeneous edge devices can become first-class participants in end-to-end Pegasus workflows using standard scheduling and execution mechanisms—no bespoke edge orchestration layers are required. IV. E VALUATION Our evaluation targets user productivity from the perspective of a novice Pegasus user whose prior experience was limited to a single tutorial [14]. We used Claude [5] (Opus 4.5) as the primary AI assistant, supplemented by ChatGPT [15] and Gemini [16] during debugging sessions. All workflows were deployed on the FABRIC testbed using FABlib for provisioning and FABRIC notebooks [17] as contextual grounding. Key results. Table I summarizes the development effort. Cross-Workflow Observations. Across all workflows, we observed consistent trends: • Rapid time-to-first-workflow: Runnable workflows were generated within 3–4 prompts when grounded in an existing pattern. • Decreasing iteration cost: Later workflows required fewer execution attempts due to inherited configurations and prior debugging experience. • Debugging dominates effort: Most iteration effort was spent resolving execution-time configuration issues (credential propagation, data staging, container runtime) rather than correcting workflow logic. • Compounding productivity gains: Progressive reuse (Orcasound → AQ → earthquake/soil moisture) reduced development effort for each successive workflow. The total development effort across all workflows was approximately 4–5 days (24–40 person-hours), or 8–12 hours per workflow—a substantial reduction for a novice user who would otherwise need weeks to learn workflow abstractions, configure execution environments, and diagnose distributed runtime failures. V. C ONCLUSIONS AND F UTURE W ORK We presented an experience-driven methodology combining pattern-based workflow engineering with AI-assisted development for sensor-driven, edge-to-core applications. Using Pegasus on the FABRIC testbed, we demonstrated how validated workflow patterns can be incrementally adapted across domains—air quality analysis, earthquake processing, soil
moisture prediction, and hydrophone-based audio analytics— and execution environments, including BlueField-3 DPUs and Raspberry Pis, without redesigning the abstract workflow. Our evaluation highlights that AI assistance is most effective when grounded in proven workflow patterns and applied to iterative refinement and debugging. Progressive reuse compounds productivity gains: each successive workflow benefits from inherited structure, configuration, and debugging experience. This work has limitations, including an evaluation focus on development effort rather than application performance and reliance on a single user’s experience. Future directions include extending the methodology to team-based development effort evaluation and longer-lived workflows, incorporating execution feedback for automated workflow adaptation, comparative performance evaluation against manually constructed workflows by experts, integrating AI agents that proactively suggest optimizations and placement strategies, and leveraging tools like Kiso [18] for automated infrastructure provisioning across the edge-tocloud continuum. ACKNOWLEDGMENTS This work is supported by the US National Science Foundation grants #2403051 and #2513101. We acknowledge SAGE (NSF award #2436842 and FABRIC Testbed (NSF #2330891). R EFERENCES [1] E. Deelman et al., “Pegasus, a workflow management system for science automation,” Future Generation Computer Systems, vol. 46, pp. 17–35, 2015. [2] I. Baldin et al., “Fabric: A national-scale programmable experimental network infrastructure,” IEEE Internet Computing, vol. 23, no. 6, 2019. [3] NVIDIA, “BlueField-3: A high-performance DPU for data center infrastructure,” in Proceedings of the Hot Chips Symposium, 2022. [4] M. Chen et al., “Evaluating large language models trained on code,” arXiv preprint arXiv:2107.03374, 2021. [5] Anthropic, “Claude (anthropic),” https://www.anthropic.com/claude. [6] D. Thain, T. Tannenbaum, and M. Livny, “Distributed computing in practice: The condor experience,” Concurrency and Computation: Practice and Experience, vol. 17, no. 2–4, pp. 323–356, 2005. [7] Orcasound Project, “Orcasound: An open-source project for real-time bioacoustic research and conservation,” in Proceedings of the Marine Mammal Science Conference, 2020. [8] “Air quality forecasting workflow using pegasus wms,” 2025. [Online]. Available: https://github.com/pegasus-isi/airquality-workflow [9] “OpenAQ: Open air quality data,” https://openaq.org. [10] P. Beckman et al., “SAGE: A software-defined sensor network,” Computing in Science & Engineering, vol. 23, no. 3, pp. 95–101, 2021. [11] “Earthquake/seismic data workflow using pegasus wms,” 2025. [Online]. Available: https://github.com/pegasus-isi/earthquake-workflow [12] “USGS earthquake hazards program,” https://earthquake.usgs.gov. [13] “Soil moisture analysis and irrigation workflow using pegasus wms,” 2025. [Online]. Available: https://github.com/pegasus-isi/ soilmoisture-workflow [14] “Pegasus workflow management system via ACCESS,” 2024. [Online]. Available: https://support.access-ci.org/tools/pegasus [15] OpenAI, “Openai codex,” https://openai.com/research/codex. [16] Google, “Gemini (google),” https://ai.google/gemini/, accessed 2026. [17] “Deploying pegasus on fabric testbed,” 2026. [Online]. Available: https://tinyurl.com/fabric-artifacts [18] “Kiso: Edge to cloud workflows: Advancing workflow management in the computing continuum,” 2026. [Online]. Available: https: //kiso.scitech.group/