A Workflow-Oriented Framework for Asynchronous Human-AI Collaboration in Hybrid and Compute-Intensive HPC Environments Sergio Mendoza 1,* , Cedric Bhihe 1 , Natalia Zamora 1 , David Modesto 1 , Jose Martin Bugallo Batalla2 , Jesus Gomez Canovas2 , Rafel Palomo Avellaneda2 , and Miguel Perez Espinosa2
arXiv:2605.03743v1 [cs.DC] 5 May 2026
1
Barcelona Supercomputing Center, Spain 2 NTT DATA, Spain
Published version notice. This is the accepted manuscript of the paper published in the Proceedings of SPIE. DOI: 10.1117/12.3069887. © 2025 Society of Photo-Optical Instrumentation Engineers (SPIE). One print or electronic copy may be made for personal use only. Systematic reproduction and distribution, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper are prohibited.
ABSTRACT Human involvement is critical in training and deploying AI systems in high-stakes defence and security contexts. However, real-time interaction is impractical in HPC environments due to compute intensity and resource constraints. We present a workflow framework that enables asynchronous human-AI collaboration across hybrid infrastructures, including HPC clusters, local machines, and cloud platforms. Workflows can pause at defined checkpoints for human input without halting underlying compute jobs, preventing idle resources and enabling non-blocking supervision. The framework supports interaction with SLURM-based scheduling, containerized and native tasks, and is customized for scenarios requiring human judgment and adaptability. We demonstrate its application in model training on systems like MareNostrum 5, highlighting benefits in portability, efficiency, and oversight in operational AI workflows. Keywords: Human-in-the-Loop (HITL), Workflow execution management, Adaptability, Rapid Adaptation, High-Performance Computing (HPC), Hybrid Computing, Configuration-based Workflows
1. INTRODUCTION Artificial intelligence (AI) systems are increasingly deployed in domains where reliability, adaptability, and transparency are critical, including security and defence. In these high-stakes contexts, fully automated workflows are insufficient: human expertise remains essential for oversight, validation, and adjustment. This need is addressed by the Human-in-the-Loop (HITL) paradigm, which ensures that human judgment complements algorithmic computation. When data is scarce, as in frugal learning, HITL becomes critical, since even a very small amount of expert supervision can significantly accelerate model performance.1 However, traditional high-performance computing (HPC) infrastructures are not designed to allow asynchronous human participation. Once a job is submitted to a batch system such as SLURM, it is expected to execute without human interaction. This rigidity creates a misalignment between the need for human supervision and the operational characteristics of HPC. As a result, existing AI workflows cannot adapt dynamically to human input without costly interruptions, reducing both efficiency and flexibility. Existing workflow management frameworks—such as Apache Airflow, Nextflow, or COMPSs—partially address automation, portability, and scalability, but they remain limited by code-centric definitions or static execution models. Few allow workflows to pause logically for asynchronous human supervision, and even fewer *Correspondence: [email protected]
support runtime restructuring across heterogeneous infrastructures. These limits reduce their use in defenceinspired scenarios where rapid adaptation with human oversight is essential. In response to this gap, we introduce the Collaborative Innovation Framework (CIF). CIF is both: • A framework: a concrete software system that enables the execution of modular workflows across HPC, cloud, and local environments. • An architecture: a workflow-oriented architecture (WOA), where the workflow configuration file serves as the central element that defines the initial task dependencies, execution sites, and HITL checkpoints, while allowing runtime modifications that enable dynamic branching. This dual view is essential. As a framework, CIF shows practical value through implementation and validation. As an architecture, it defines a new paradigm where configuration-based specification, asynchronous HITL checkpoints, and runtime adaptability are built-in features. By treating the workflow as the central element, CIF makes sure that human supervision and computational processes act together as complementary rather than competing forces. The contribution of this work can be seen in two main aspects. First, it shows how workflows defined declaratively in configuration files can run across hybrid environments with integrated HITL checkpoints, allowing human supervision without stopping compute-intensive tasks. Second, it presents CIF as a workflow-oriented architecture that directly supports adaptability and rapid adaptation, ensuring that workflows can change dynamically in response to runtime conditions, such as human supervision, changing data or resource availability. The remainder of this paper is organized as follows: Section 2 reviews related work in workflow management, hybrid execution environments, and HITL integration. Section 3 explains the design principles of CIF as a workflow-oriented architecture. Section 4 describes its implementation. Section 5 introduces a case study that validates CIF in a defence-oriented scenario. Section 6 reports the validation methodology and results. Section 7 compares CIF with existing frameworks, and Section 8 concludes with directions for future research.
2. RELATED WORK Research on workflow management and execution frameworks has advanced significantly, but important gaps remain when adaptability, asynchronous HITL checkpoints, and hybrid HPC execution need to coexist. We evaluate previous work along three main areas: workflow frameworks, hybrid execution environments, and HITL integration.
2.1 Workflow Frameworks General-purpose systems such as Apache Airflow,2 Luigi,3 and Prefect4 are widely used for task automation. They provide dependency management and monitoring, but remain code-centric (Python DAGs) and have little or no integration with HPC schedulers such as SLURM. Airflow supports branching and sensors, but these are synchronous constructs defined in Python code, not declarative configuration. This makes them harder to adapt at runtime and less portable to hybrid HPC environments. In scientific computing, frameworks such as Nextflow,5 Snakemake,6 and Argo Workflows7 focus on containerization and reproducibility. These systems are effective for large-scale data pipelines but rely on fixed execution models, with no native support for asynchronous human supervision. Closer to HPC, systems such as COMPSs,8 Pegasus,9 and Kepler10 support large-scale distributed execution and parallelism. While powerful, their workflows remain fixed once launched and do not provide clear mechanisms to include HITL checkpoints. Contrast with CIF: Unlike these solutions, CIF uses a configuration-based TOML specification, includes asynchronous HITL tasks as explicit elements, and allows runtime restructuring, making it particularly suited for adaptive workflows in defence and security contexts.
2.2 Hybrid Execution Environments The efforts for unifying HPC, cloud, and local resources have led to orchestration strategies such as Kubernetes Federation,11 HPC-as-a-Service models,12 and scientific platforms like Galaxy.13 Community standards such as CWL14 and WDL15 reinforce reproducibility through declarative specifications. Although these systems enable portability, they remain mainly machine-driven and rarely integrate human checkpoints into execution workflows. In addition, secure transfer and supervision across defence and HPC environments remain underexplored. Contrast with CIF: CIF directly supports hybrid execution across tactical edge resources, HPC clusters, and cloud platforms, integrating with SLURM while keeps portability through containers and explicit HITL checkpoints.
2.3 HITL Integration in Workflows HITL methods are increasingly studied in medicine,16 autonomous driving,17 and defence operations.18 TThese approaches often rely on annotation or later validation, but interventions are synchronous, stopping the workflow until human feedback is received. Business Process Management (BPM) systems,19 as well as Airflow sensors20 or n8n manual triggers,21 support human interaction by pausing execution until input is received. These mechanisms stop progress in their branch of execution, while other branches may continue, but they remain blocking in nature and are tied to code- or GUI-based definitions. In contrast, CIF integrates HITL checkpoints as asynchronous, declarative tasks that can change parameters or add new tasks at runtime, making them better suited for compute-intensive and hybrid HPC contexts. Contrast with CIF: CIF defines HITL as asynchronous, non-blocking tasks that are defined in the workflow configuration. This design supports rapid adaptation: workflows can pause logically for human decisions while parallel HPC tasks continue, ensuring resource efficiency and adaptability.
2.4 Gap and Motivation Across these three areas of previous work (workflow frameworks, hybrid execution environments, and HITL integration), existing systems show two main patterns: (i) they support automation without human adaptability, or (ii) they include human oversight in ways that are not compatible with HPC. None combine configurationbased workflows, asynchronous HITL, runtime restructuring, and hybrid HPC execution in a single approach. The CIF addresses this gap by enabling dynamic workflows with HITL, ensuring adaptability and rapid adaptation in hybrid, compute-intensive environments. Current workflow systems deal with automation, hybrid execution, or human interaction only on their own, but none provide a unified solution that brings these aspects together. Many are optimized for deterministic automation without human adaptability, while others manage human oversight in ways that do not work well in HPC environments. This motivates the design of CIF, which defines asynchronous HITL checkpoints as workflow tasks, supports dynamic restructuring at runtime, and keeps execution running across different infrastructures. The following section introduces the architectural principles of CIF, showing how configuration-based specification, runtime orchestration, and hybrid execution are integrated into a workflow-oriented architecture designed for adaptability and rapid adaptation under human-AI collaboration.22, 23
3. FRAMEWORK ARCHITECTURE AND DESIGN 3.1 Workflow-Oriented Architecture (WOA) The CIF adopts a workflow-oriented architecture (WOA) in which the workflow configuration is not only an execution input but the element that organizes the system. Workflows are written declaratively in TOML files, defining tasks, dependencies, execution sites, and HITL checkpoints. At runtime, the TaskScheduler reads the configuration, activates tasks once dependencies are satisfied, and submits them to the right execution environment. By separating workflow logic from implementation code, CIF reduces barriers for non-developer users and improves portability.
A key property of this architecture is that HITL checkpoints are defined in the same configuration file as computational tasks, making supervision an integral part of workflow logic. In addition, workflows can be changed during execution: tasks can be added, parameters revised, or branches redirected based on human decisions. In short, WOA assures that CIF workflows remain adaptive and portable, while treating HITL checkpoints as structural rather than optional elements.
3.2 Workflow-Oriented System (WOS) The architecture is implemented through lightweight modules that together form a workflow-oriented system (WOS). At the entry point, CIF-CLI checks the TOML workflow file and submits it to the CIF core components. The TaskScheduler then interprets the configuration, resolves dependencies, coordinates task execution, and manages pauses at HITL checkpoints and branching. The TaskExecutor handles input/output transfers, execution environments, and interactions with local, cloud, or HPC schedulers such as SLURM. The WorkflowExecution component maintains the global workflow state. The Watcher checks completions asynchronously in HPC environments by polling for .done markers, while the JobRegistry tracks status of all tasks to ensure consistency across computational and HITL tasks. This modular design allows CIF to integrate external frameworks (e.g., TensorFlow, MPI, COMPSs) without modifying the workflow definition, while ensuring robust state management in hybrid and asynchronous contexts.
Figure 1: Layered architecture of CIF. At the top, CIF-CLI receives the workflow file and starts execution. The middle layer (TaskScheduler, TaskExecutor, WorkflowExecution) organizes tasks and controls their flow. The Watcher and JobRegistry manage monitoring and state across the system. At the bottom, tasks run on different sites (local, HPC, cloud). This layered view shows that all modules act according to the workflow, making CIF workflow-centric, adaptive, and ready for asynchronous HITL. Together, these modules formalize CIF’s capacity for adaptability, hybrid execution, and workflow management with HITL integration.
3.3 Workflow Execution Model CIF workflows follow a dependency-driven execution model: the TaskScheduler starts tasks when their prerequisites are complete, the TaskExecutor runs them, and WorkflowExecution keeps the global state updated. Unlike static directed acyclic graphs (DAGs), CIF supports runtime restructuring. HITL checkpoints can add new tasks or change execution paths while computation continues in parallel branches. In Figure 2, you can see that Task A runs locally while Task B is running in MareNostrum 5 (MN5)† , showing how the same workflow can run across different execution sites. This model makes sure that HITL steps do not block ongoing HPC jobs. While
one branch may wait for human feedback, others continue normally, keeping efficiency. In this way, CIF enables rapid adaptation by making sure that waiting for human feedback does not slow down the computing tasks.
3.4 Hybrid Execution and Containerization Workflows may extend across tactical edge resourcestactical cloudlets (deployable computing nodes operating near the mission area), HPC clusters, cloud platforms, and local machines. Execution sites are declared directly in the TOML file, allowing the same workflow to distribute tasks across heterogeneous infrastructures. Containerization is an explicit feature: Docker and Singularity images preserve dependencies and runtime consistency across environments. In CIF, each task can reference a specific container image in the workflow file. When the TaskExecutor runs the task, it uses the defined container to ensure the task always runs in the same environment. In this way, CIF avoids inconsistencies between environments and makes it easier to integrate external frameworks. This hybrid strategy guarantees portability and flexibility while supporting adaptability in mission-oriented contexts.
3.5 Asynchronous HITL Integration HITL checkpoints are defined as declarative tasks within the workflow configuration. When reached, the workflow pauses logically for human review (for example, validating inference or authorizing retraining) without stopping parallel jobs already running on HPC or cloud resources. The Watcher and JobRegistry manage the asynchronous use of human decisions, which may adjust parameters, trigger new tasks, or skip steps that are no longer needed. This guarantees supervision without leaving resources idle. By defining HITL as asynchronous, non-blocking tasks, CIF balances adaptability and efficiency, which is critical in defence and security operations.
3.6 Integrated State Management and Differentiation In HPC contexts, schedulers such as SLURM provide no workflow-level notifications. CIF addresses this with the Watcher, which detects completion signals, and the JobRegistry, which keeps the central record of workflow state. Together, these components ensure integrity across asynchronous tasks. Architecturally, CIF distinguishes itself by combining three principles: first, a workflow-oriented architecture, where workflows act as the structural axis rather than just execution inputs; second, asynchronous HITL checkpoints, which are non-blocking and declared in the workflow; and third, hybrid execution designed for HPC, achieved through direct integration with SLURM and containerization technologies. In short, CIF establishes a new model where configuration-based workflows can adapt dynamically under human supervision while keeping full use of compute resources.
4. IMPLEMENTATION This section describes how the CIF is implemented in practice. Building on the workflow-oriented architecture introduced in Section 3, the implementation shows how configuration-based workflow control, hybrid execution, and asynchronous HITL supervision are applied in workflows.
4.1 CIF-CLI and Workflow Submission The CIF command-line interface (CIF-CLI) is the entry point for workflow execution. It serves as the user interface for submitting workflows to the CIF framework. The workflow is described in a configuration file written in TOML, which specifies tasks, dependencies, and execution sites. Once submitted through CIF-CLI, the file is interpreted by the TaskScheduler, which parses the configuration and manages the execution flow by starting tasks when dependencies are satisfied and sending them to the correct execution environment. †
https://www.bsc.es/supportkc/docs/MareNostrum5/intro/
Figure 2: UML sequence of asynchronous HITL in CIF. Two stakeholders interact through the CIF-CLI: the CIF user (who submits the workflow) and the HITL supervisor (who reviews and adapts it). The user submits a configuration file in TOML, which is parsed by the TaskScheduler. Tasks are executed on different sites, such as MN5 via SLURM, through dedicated TaskExecutors. The Watcher polls HPC jobs and notifies the JobRegistry when tasks finish. At HITL checkpoints, the supervisor provides feedback through CIF-CLI, such as adding tasks or changing parameters. The TaskScheduler then updates the workflow and continues execution. This sequence shows how CIF keeps workflows running while separating human review time from compute time. Code 1: Minimal CIF configuration file: task and workflow (TOML). [ [ task ] ] name = ” i n f e r e n c e t a s k ” command = ” s i n g u l a r i t y run −−nv \ −−bind <DATA ROOT> <SIF IMAGE> \ s r c / main.py −−c o n f i g <CONFIG FILE>” execsite = ” local ” i n p u t . m o d e l = ” / path / t o / m o d e l . p t ” i n p u t . d a t a = ” / path / t o / images / ” o u t p u t . r e s u l t s = ” / path / t o / r e s u l t s . j s o n ” [ workflow ] tasks = [ ” inference task ” ] Code 1 shows the simplest configuration of a CIF workflow. It has two sections: first, the task definition, and second, the workflow description. The single task is named inference task. Its command is a Singularity execution line written in Bash, which will be launched in the operating system shell. The field execsite = "local" indicates that the task runs in the same machine where CIF is executed. The task requires two input paths: a model file (model.pt) and a dataset directory. As output, it produces a JSON file with the inference results. The [workflow] section lists this task for execution.
This design keeps the workflow description in a simple configuration file, while CIF modules (TaskScheduler, TaskExecutor ) take care of parsing and running the tasks. As a result, workflows are easy to read, portable across environments, and can be adapted at runtime without changing the underlying application code.
4.2 Multi-Stage HPC Workflow CIF supports multi-stage workflows on HPC systems by declaring dependencies between tasks such as training and evaluation. A multi-stage workflow is one where tasks are chained in order, with each stage depending on the output of the previous one. These tasks are run in the execsite specified in the TOML file. Unlike code-centric DAGs, CIF workflows remain declarative and portable. For example, a training task may generate a model, followed by an evaluation task that depends on it. Both are scheduled on the HPC cluster. While the detailed TOML definition is omitted here, the setup follows Code 1, with explicit dependencies and execution sites declared. This demonstrates CIF’s ability to scale workflows declaratively on HPC without changing the configuration model.
4.3 HITL Checkpoint Integration A key feature of CIF is the ability to include Human-in-the-Loop (HITL) checkpoints directly as tasks in the workflow configuration file. These checkpoints allow a human supervisor to review metrics or outputs and decide how the workflow should continue. Unlike blocking mechanisms in other systems, CIF HITL tasks are asynchronous: they pause only their workflow branch, while unrelated tasks continue executing in parallel. This behaviour is consistent with the execution model shown in Figure 2, where Task A and Task B run in parallel on different execution sites. Code 2: Declarative definition of an asynchronous HITL checkpoint task [ [ task ] ] name = ” t r a i n i n g t a s k ” command = ” s i n g u l a r i t y run −−nv \ −−bind <DATA ROOT> <SIF IMAGE> \ s r c / main.py −−c o n f i g <CONFIG FILE>” depends on = [ ” i n f e r e n c e t a s k ” ] e x e c s i t e = ”HPC−CLUSTER” i n p u t . c o n f i g = ”<CONFIG FILE>” o u t p u t . m e t r i c s = ”<OUTPUT MODELS DIR>” [ [ task ] ] name = ” h i t l r e v i e w e r e v a l u a t i o n ” command = ” m o d u l e s . c i f c o r e . h i t l r e v i e w . h i t l e p o c h r e v i e w e r ” e x e c s i t e = ”HPC−CLUSTER” depends on = [ ” t r a i n i n g t a s k ” ] add tasks = [ ” inference task ” ] hitl.enabled = true h i t l . i n p u t = ” / path / t o / output / m e t r i c s . t x t ” h i t l . m e s s a g e = ” I s t h e model good enough ?\n” output.hitl decision = ” hitl decision.json ” [ e x e c s i t e s . ”HPC−CLUSTER” ] h o s t = ”hpc−l o g i n . e x a m p l e . o r g ” key = ” /home/ u s e r / . s s h / i d e c d s a ” user = ” user1234 ” [ workflow ] tasks = [ ” training task ” , ” hitl reviewer evaluation ” ]
Code 2 shows how CIF defines an asynchronous HITL checkpoint. Two tasks are included. The first, training task, depends on the inference stage and is executed on the HPC cluster. Its command is a Singularity execution line in Bash that trains the model, using a configuration file as input and producing metrics as output. The second task, hitl reviewer evaluation, also runs on the HPC cluster and depends on the training task. Instead of a compute-intensive operation, it calls the CIF HITL review module, which presents metrics to the human supervisor. This task is marked with hitl.enabled = true, meaning that only this workflow branch pauses until the supervisor provides input. Other tasks without this dependency continue to run. The supervisor’s decision is saved in a JSON file (hitl decision.json) and may include adding new tasks through the add tasks field. The section [execsites."HPC-CLUSTER"] defines the CIF needs for using to the HPC system, showing that CIF itself runs externally and only submits tasks remotely. The workflow section at the bottom lists both tasks, confirming that HITL checkpoints are treated as regular workflow tasks. This example also shows that dependencies in CIF are not mandatory. Tasks can run independently, in parallel, or as chained stages. Workflows are not limited to linear pipelines, since HITL checkpoints allow branching and the dynamic addition of tasks. Loops could in principle be described directly in TOML by declaring cyclic dependencies, but this would be unsafe in HPC environments. For this reason, CIF discourages explicit loops and instead supports iterative processes through HITL-driven task addition. This mechanism enables retraining cycles while keeping execution predictable and safe. Overall, HITL checkpoints in CIF are integrated as asynchronous, non-blocking elements. They allow supervisors to influence workflow paths dynamically while other branches continue, supporting adaptability and rapid adaptation without stopping ongoing computation.
4.4 Execution Flow Example The CIF execution cycle can be summarized as: Parse → Schedule → Execute → HITL Review → Adapt → Continue This flow is illustrated in Figure 2. It shows how a workflow is first parsed, tasks are scheduled, execution starts, and then HITL checkpoints may pause the flow for review. Human input can adapt the workflow, after which execution continues. Unlike systems such as Airflow20 or n8n,21 where workflow control is either code-centric or limited to blocking steps, CIF defines HITL and hybrid execution as declarative features. This design combines portability with HPC-native execution and allows workflows to adapt at runtime by adding new tasks or changing parameters. As a result, CIF workflows remain dynamic, adaptive, and quick to reconfigure, which is especially important in mission-oriented HPC scenarios.
5. CASE STUDY: TACTICAL USE CASE EXECUTION 5.1 Description of the Use Case The tactical case study focuses on ship detection in maritime surveillance, where operational needs require both real-time inference at the edge and adaptive retraining on HPC resources. A Tactical Unit deployed in the field runs inference tasks on image streams, while supervisors review outputs asynchronously through HITL checkpoints. If inference performance is not good enough, CIF manages retraining and evaluation on the MN5 supercomputer. The overall workflow is illustrated in Figure 3, which shows how CIF links local inference, human supervision, and compute-intensive retraining across different environments.
Figure 3: Abstract workflow for the ship-detection case study. Two execution sites are used: the Tactical Unit and MN5 — HPC Resources in BSC. In a real mission, input images would come from drones, satellites, or other sensors (left side). Inference runs on the Tactical Unit because it is lightweight, while training is computeintensive and runs on MN5. Two HITL checkpoints are included: one where the supervisor reviews inference results and decides whether to trigger retraining, and another where the updated model is validated after training to decide if it can be deployed or needs improvement.
5.2 Workflow Structure and Execution Plan The workflow is written declaratively in a TOML configuration file and includes three categories of tasks: inference tasks that run locally on the Tactical Unit in Singularity containers; HITL checkpoints where supervisors validate detections asynchronously; and retraining and evaluation tasks that run on MN5, also in Singularity containers under SLURM. As shown in Figure 4, inference is followed by a HITL checkpoint (Inference Validation). If the results are accepted, the workflow ends. If they are rejected, CIF runs retraining and evaluation on MN5. A second HITL checkpoint validates the metrics before deployment. In short, CIF makes sure that the workflow can change dynamically under human oversight, combining local/edge inference with HPC-scale retraining.
Figure 4: Workflow structure for the ship-detection case study. The figure shows the initial configuration of the workflow defined in the configuration file. Dependencies between tasks are visible together with two HITL checkpoints. The Inference Reviewer can approve results or trigger retraining on MN5, while the Model Reviewer validates the retrained model. This structure illustrates how CIF workflows adapt dynamically under human supervision.
5.3 Role of HITL in Supervision and Rapid Adaptation HITL supervision was essential for adaptability. At runtime, supervisors could reject detections or request retraining directly through HITL checkpoints, without stopping other workflow branches. This made it possible to add new tasks, such as retraining jobs, into the workflow while it was running. This mechanism shows rapid adaptation: the workflow changes in response to human feedback while compute-intensive retraining continues asynchronously on MN5. Instead of re-submitting jobs or modifying code, supervisors influenced execution by providing decisions that updated the workflow state. In this way, CIF showed that asynchronous HITL checkpoints let mission workflows adapt in real time without reducing computational efficiency.
5.4 Integration with CIF and Containerization CIF managed the case study workflow through declarative specifications, using the same TOML file across the Tactical Unit and MN5. Workflow logic was kept separate from infrastructure, which ensured portability between local and HPC execution. Containerization provided consistent environments: on the Tactical Unit, Singularity was used for the Inference, Inference Validation, and Task Valid Metrics tasks. On MN5, Singularity was also used for the Evaluation Task when retraining the model. Native Python modules were also supported for evaluation steps, showing flexibility. Operational limits, such as CPU-only inference on the Tactical Unit and restricted network bandwidth, were reduced by CIF’s hybrid design. Inference was run locally on the Tactical Unit, while only model updates or evaluation results were sent to MN5. This avoided heavy data transfers and kept response times low. In contrast, training the model required many GPUs and was too compute-intensive to be executed on the Tactical Unit, so it was performed on MN5. The framework also supported effective cooperation between the Tactical Unit, HPC resources, and asynchronous human supervision. Feedback from NTTDATA confirmed that HITL checkpoints were essential for supervisors’ confidence in the detection results, and that defining tasks in CIF was simple and required little extra training.
6. VALIDATION AND DEMONSTRATION This section evaluates whether the CIF can run hybrid workflows with asynchronous HITL checkpoints while keeping computational efficiency. Building on the architectural principles in Section 3, the implementation in Section 4, and the case study in Section 5, we test whether CIF keeps execution continuous across heterogeneous resources and enables rapid adaptation to human feedback under operational constraints. We focus on three aspects: workflow continuity with HITL, resource efficiency during hybrid execution, and adaptability through runtime changes.
6.1 Objective of the Validation The validation aimed to confirm that CIF supports dynamic adaptation to human feedback without disrupting ongoing HPC workloads. In particular, we examined whether asynchronous HITL checkpoints could change execution paths, including triggering retraining, while parallel tasks remained active. We also tested whether this mechanism enabled timely model improvement in a mission-oriented setting. Success is defined as achieving human-guided reconfiguration of the workflow without leaving compute resources idle and without modifying code in running components.
6.2 Methodology of Demonstration Validation was conducted on a hybrid setup combining a field-deployed Tactical Unit for on-device inference and the MN5 supercomputer for retraining and evaluation. As shown in Figure 5, the workflow proceeded as follows: the Tactical Unit executed local inference on incoming imagery, after which an asynchronous HITL checkpoint allowed a supervisor to validate detections. If detections were rejected by the supervisor, the CIF automatically scheduled retraining and evaluation tasks on MN5. Once these tasks finished, a second HITL checkpoint determined whether the updated model was approved for deployment. Throughout, CIF’s Watcher polled for completion signals, and the JobRegistry tracked the status of running tasks. This information allowed WorkflowExecution to apply HITL decisions while keeping the workflow consistent and avoiding conflicts between human input and ongoing computation.
Figure 5: Validation workflow for CIF (flowchart view). Inference runs on the Tactical Unit, while training and evaluation are executed on MN5-ACC. The CIF framework and CIF-CLI operate on the Tactical Unit and control remote tasks without being installed on MN5. Two HITL checkpoints guide the flow: inference validation and model validation. This figure shows how CIF combines local inference on the Tactical Unit, remote HPC training, and asynchronous HITL to adapt workflows dynamically. This setup showed that human decisions were included in the workflow while HPC jobs kept running at full capacity. The workflow paused only in a logical way at HITL checkpoints, without stopping other tasks. This means that while the system waited for human feedback, other tasks could continue running on the supercomputer. In this way, CIF kept human review in the loop without wasting compute resources.
6.3 Adaptability through Iterative Retraining A key feature of CIF is its ability to let workflows be changed during execution through HITL input. HITL checkpoints allowed supervisors to reject outputs and start new retraining cycles without re-submitting jobs or changing source code. The initial TOML configuration listed the possible tasks and checkpoints, and the TaskScheduler chose the path based on human input saved in decision files. This design made rapid adaptation possible: when supervisors identified low confidence in detections or changes in the data (e.g. new types of ships, difficult weather conditions or suspicious camouflage), they could modify the workflow, while other computations continued on MN5 or on the Tactical Unit. Such adaptability is important in mission contexts where data or adversarial behavior may change unpredictably. By including HITL as an explicit, non-blocking element, CIF enabled supervisors to keep the system responsive without reducing throughput or wasting resources, supporting retraining when needed and avoiding unnecessary cycles when it was not.
6.4 Results CIF successfully coordinated inference on the Tactical Unit with retraining and evaluation on MN5, while keeping compute resources fully used. Human supervision guided the execution path at two points: initial inference validation and approval after retraining. When the first checkpoint showed poor metrics, the HITL told the CIF to run retraining and evaluation tasks without blocking parallel activity. When the metrics met operational thresholds, the updated model was approved for deployment, closing the adaptation loop. As shown in Figure 6a, the baseline model produced a generic “building” detection and missed ships of interest. After the HITL triggered the retraining task on MN5, Figure 6b shows that multiple ships were
(a) Baseline inference
(b) Post-retraining inference
Figure 6: Qualitative inference results before and after HITL-supervised retraining. (a) Baseline: generic “vehicle” detection, missing ships. (b) Post-retraining: multiple ships detected with higher precision. correctly detected with higher precision. These qualitative results confirm that CIF executed the adaptive loop as intended: HITL checkpoints identified errors, CIF scheduled retraining, the model was updated, and redeployment followed without interruption. Although precise model optimization was not the primary goal of this study, the improved detections illustrate CIF’s ability to support dynamic, human-guided adaptation while keeping computational efficiency. Overall, the demonstration validates that configuration-based workflows with asynchronous HITL can sustain execution continuity across heterogeneous infrastructures while enabling rapid, mission-relevant updates.
7. DISCUSSION: COMPARISON WITH EXISTING FRAMEWORKS AND LIMITATIONS This section contrasts the CIF with representative workflow systems used in data engineering and scientific computing, and then discusses limitations and applicability. Our analysis focuses on five dimensions that are central to this work: (i) configuration-based workflow specification, (ii) hybrid/HPC-native execution, (iii) containerization for reproducibility, (iv) asynchronous HITL checkpoints, and (v) runtime restructuring guided by human supervision. We select widely adopted workflow managers—Airflow,2 Luigi,3 Prefect,4 Nextflow,5 Snakemake,6 Argo,7 COMPSs,8 Pegasus,9 and Kepler10 —as baselines for comparison.
7.1 Comparative Analysis This comparison builds on the workflow systems introduced in Section 2, focusing on five aspects: configurationbased specification, hybrid execution, containerization, asynchronous HITL, and runtime adaptation. Table 1 shows the main differences. Table 1: Comparison of selected workflow frameworks against CIF. Framework
Workflow spec.
HPC/Hybrid
Containers
Async HITL
Runtime restruct.
Scheduler integ.
Airflow2 Luigi3 Prefect4 Nextflow5 Snakemake6 Argo7 COMPSs8 Pegasus9 Kepler10
Code (Python DAG) Code (Python) Code (Python) DSL / Config Rules (Py/Config) YAML (K8s) Annotations/Conf. DAX/Config Config/GUI
Limited∗ Limited Limited Yes (Cloud/HPC) Yes (HPC) Cloud/K8s Yes (HPC) Yes (HPC/Grid) Yes (HPC)
Partial Partial Partial Yes Yes Yes Partial Partial Partial
No No No No No No No No No
Limited (manual) Limited Limited Rare Rare Limited No No No
Plugins / ext. Plugins / ext. Plugins / ext. Native / adapters Native / adapters K8s-native Native (HPC) Native / schedulers Native / schedulers
CIF (this work)
Config (TOML)
Yes (Edge/HPC/Cloud)
Yes
Yes (non-blocking)
Yes (HITL-driven)
SLURM + ext.
General-purpose systems (Airflow, Luigi, Prefect) provide dependency management and monitoring, but they are code-centric and link to HPC only through extensions. Scientific workflow managers (Nextflow, Snakemake, Argo) focus on portability and containerization, making them effective for large-scale pipelines. However, human
oversight in these systems is usually synchronous and blocking or handled outside the workflow. They may allow manual approval steps, but they do not support runtime updates where human input changes the workflow itself. HPC-focused frameworks (COMPSs, Pegasus, Kepler) enable parallel execution and traceability, but workflows remain static after submission and do not include asynchronous HITL as an explicit feature. CIF differs in three ways. First, the workflow configuration file (TOML) is the central element of execution. Second, asynchronous HITL checkpoints are defined directly as workflow tasks, allowing human supervision without blocking other branches. Third, execution sites are declared explicitly, so the same configuration can run locally, on tactical units, HPC clusters under SLURM or cloud resources. The TaskScheduler, Watcher, and JobRegistry support runtime changes based on HITL decisions while keeping compute resources fully used. In this way, CIF separates the time needed for human review from the time needed for computation, which is essential in mission contexts where rapid adaptation is required. From an operational viewpoint, CIF’s main contribution is making HITL both declarative and asynchronous. Instead of stopping the whole workflow while waiting for approval, CIF lets human validation happen in parallel with other compute tasks. This design keeps HPC resources productive and enables rapid model updates when supervision detects errors, as shown in the ship-detection case study in Section 6. The configuration-based approach also makes CIF easier to adopt. Users can adjust execution by editing configuration or decision files, without having to modify the application code.
7.2 Limitations and Applicability Users of CIF must be aware of several constraints: (i) each execution site must support containers (Docker or Singularity); (ii) HPC systems must provide a batch scheduler (e.g., SLURM) with accessible login nodes; (iii) task completion is detected through file-based signals (.done markers) used by the Watcher, with custom extensions needed at sites that do not allow containerization or external polling; and (iv) users must have the necessary accounts and access rights to submit jobs on remote resources (e.g. MN5 or equivalent HPC systems). In addition to these technical constraints, the duration of the adaptive loop depends on external factors such as data transfer times and queue delays on shared HPC systems. These delays are outside CIF’s control and must be considered when setting service expectations. Security and compliance requirements in defence-oriented scenarios can also restrict network connectivity between exeuction sites, which may limit the exchange of data or the integration of human supervision tools. Our validation relied on secure transfer and authenticated endpoints, but larger deployments may require stronger safeguards, detailed logs of user actions for accountability, and integration with policy-aligned identity management.Finally, while CIF allows workflows to be changed at runtime, very large or uncontrolled modifications should be limited by safety rules to prevent errors or unstable states in long-running workflows.
7.3 Summary In summary, existing frameworks perform well in their own domains such as enterprise automation, containerbased workflows, or HPC scheduling, but they usually manage the human oversight as external or blocking, and they do not support runtime restructuring based on supervision. CIF combines configuration-based specification, asynchronous HITL, and hybrid HPC execution in a single workflow-oriented architecture. This makes it possible for workflows to adapt rapidly under operational conditions while keeping full use of computing resources. CIF can therefore serve as a practical foundation for mission-oriented workflows that require both adaptability and computational scale.
8. CONCLUSIONS AND FUTURE WORK This paper presented the CIF, a workflow-oriented architecture that uses the configuration file as the central element of execution. By declaring HITL checkpoints together as computational tasks, CIF turns supervision ∗
Hybrid execution in these systems is usually achieved by relying on external executors, instead of providing direct integration with HPC environments.
from a blocking step into an asynchronous mechanism that runs in parallel with large-scale computation. Validation on a tactical ship-detection case study showed that CIF maintains workflow execution across heterogeneous resources while enabling rapid, human-guided adaptation. The results highlight two main contributions. First, CIF puts adaptability into practice through a configurationbased specification that unifies task dependencies, execution sites, and HITL checkpoints in a single declarative file. Second, CIF separates human review time from compute time, allowing supervisors to guide iterative retraining on HPC without leaving resources idle. Compared with existing workflow systems, CIF makes HITL an explicit, asynchronous feature while providing configuration-level control over hybrid execution. The future work will extend CIF in different ways. As a short-term objective, we plan to demonstrate CIF across other domains, including disaster response and space situational awareness, to test its generality. Another interest is exploring interoperability with workflow standards (e.g. CWL, WDL, etc.), which would allow CIF to integrate more easily with existing ecosystems. As a mid-term objective, we plan to develop an API layer above the CIF-CLI that will allow users to integrate their own web applications as the user interface for interaction and visualization.
Acknowledgments This work was supported by the European Defence Fund (EDF) under Grant Agreement No. 101103770 (KOIOS Project). The authors acknowledge the resources provided by the Spanish Supercomputing Network (RES) at the Barcelona Supercomputing Center (BSC) on MareNostrum 5 [Activity ID: IM-2025-2-0033]. The authors also thank NTTDATA for their collaboration in defining and validating the tactical use case.
REFERENCES [1] J. Jakubik, B. Blumenstiel, M. Vössing, and P. Hemmer, “Instance selection mechanisms for human-in-theloop systems in few-shot learning,” 2022. [2] Apache Software Foundation, “Apache airflow documentation.” https://airflow.apache.org, 2025. [3] Spotify AB, “Luigi: Workflow management system for batch data processing.” https://github.com/ spotify/luigi, 2019. [4] Prefect Technologies, “Prefect: The modern workflow orchestration framework.” https://www.prefect.io, 2025. [5] P. Di Tommaso, M. Chatzou, E. W. Floden, P. Prieto Barja, E. Palumbo, and C. Notredame, “Nextflow enables reproducible computational workflows,” Nature Biotechnology 35(4), pp. 316–319, 2017. [6] J. Köster and S. Rahmann, “Snakemake—a scalable bioinformatics workflow engine,” Bioinformatics 28(19), pp. 2520–2522, 2012. [7] The Argo Project, “Argo workflows: Kubernetes-native workflow engine for orchestrating parallel jobs.” https://argoproj.github.io/workflows/, 2025. Accessed: 2025-08-25. [8] R. M. Badia, J. Conejero, C. Diaz, J. Ejarque, D. Lezzi, F. Lordan, C. Ramon-Corts, and R. Sirvent, “COMP Superscalar, an interoperable programming framework,” SoftwareX 3–4, pp. 32–36, 2015. [9] E. Deelman, K. Vahi, G. Juve, M. Rynge, S. Callaghan, P. J. Maechling, R. Mayani, W. Chen, R. Ferreira da Silva, M. Livny, and K. Wenger, “Pegasus: a workflow management system for science automation,” Future Generation Computer Systems 46, pp. 17–35, 2015. [10] I. Altintas, C. Berkley, E. Jaeger, M. Jones, B. Ludäscher, and S. Mock, “Kepler: An extensible system for design and execution of scientific workflows,” in Proceedings of the 16th International Conference on Scientific and Statistical Database Management (SSDBM), SSDBM ’04, pp. 423–424, IEEE, 2004. [11] Cloud Native Computing Foundation, “Kubernetes multi-cluster services documentation.” https:// kubernetes.io, 2025. [12] J. Ejarque and R. M. Badia, “Enabling dynamic and intelligent workflows for hpc, data analytics, and ai convergence,” Future Generation Computer Systems 134, pp. 414–429, 2022. [13] Galaxy Community, “The galaxy platform for accessible, reproducible and collaborative biomedical analyses: 2022 update,” Nucleic Acids Research 50(W1), pp. W345–W351, 2022.
[14] CWL Community, “Common workflow language v1.2 specification.” https://www.commonwl.org/v1.2/, 2023. Accessed: 2025-08-25. [15] OpenWDL Community, “Workflow description language (wdl) documentation.” https://docs.openwdl. org/, 2025. Accessed: 2025-08-25. [16] A. Holzinger, G. Langs, H. Denk, K. Zatloukal, and H. Müller, “Causability and explainability of artificial intelligence in medicine,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 9(4), p. e1312, 2019. Published online 2 April 2019; e-pub ahead of print Jul–Aug 2019 issue. [17] Y. Ma, X. Cao, W. Ye, C. Cui, K. Mei, and Z. Wang, “Learning autonomous driving tasks via human feedbacks with large language models,” in Findings of the Association for Computational Linguistics: EMNLP 2024, Y. Al-Onaizan, M. Bansal, and Y.-N. Chen, eds., pp. 4985–4995, Association for Computational Linguistics, (Miami, Florida, USA), Nov. 2024. [18] B. Shneiderman, “Human-centered artificial intelligence: Reliable, safe & trustworthy,” International Journal of Human–Computer Interaction 36(6), pp. 495–504, 2020. [19] W. M. P. van der Aalst, “Business process management: A comprehensive survey,” ISRN Software Engineering 2013, pp. Article ID 507984, 1–37, 2013. [20] Apache Software Foundation, “Airflow sensors documentation.” https://airflow.apache.org/docs/ apache-airflow/stable/core-concepts/sensors.html, 2025. Accessed: 2025-08-25. [21] n8n.io, “n8n documentation: Manual trigger node,” 2024. Accessed: 2025-08-24. [22] C. Gómez, S. M. Cho, S. Ke, C.-M. Huang, and M. Unberath, “Human-ai collaboration is not very collaborative yet: a taxonomy of interaction patterns in ai-assisted decision making from a systematic review,” Frontiers in Computer Science 6, p. 1521066, 2025. [23] P. Hemmer, M. Schemmer, N. Kühl, M. Vössing, and G. Satzger, “Complementarity in human-ai collaboration: Concept, sources, and evidence,” arXiv preprint arXiv:2404.00029 , 2024. Accessed: 2025-08-25.