arXiv CSopen access

Beyond Component Testing: Validating Agentic AI Systems

Beyond Component Testing: Validating Agentic AI Systems Fabio Orazio Mirto1,2 , Luca D’Agati2 , Giuseppe Tricomi2 , Stefano Silvestri3*, Francesco Longo2,4 , Antonio Puliafito2,4 , Giovanni Merlino2,4

arXiv:2607.29405v1 [cs.AI] 31 Jul 2026

1*

Department of Biomedical, Dental, and Morphological and Functional Imaging Sciences, University of Messina, A.O.U. Policlinico “G.Martino” - Via Consolare Valeria, Messina, 98125, Italy. 2 Department of Engineering, University of Messina, Contrada di Dio, Sant’Agata, Messina, 98158, Italy. 3 Institute for High Performance Computing and Networking of National Research Council of Italy , ICAR-CNR, Via Pietro Castellino 111, Naples, 80131, Italy. 4 National Interuniversity Consortium for Informatics, CINI, Via Ariosto, 25, Rome, 00185, Italy.

*Corresponding author(s). E-mail(s): [email protected]; Contributing authors: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Abstract Agentic AI systems act through multi-step trajectories that combine planning, tool use, memory, interaction, and adaptation. This behavior stretches validation practice beyond component testing and one-shot input–output evaluation, because acceptable system behavior now depends on how decisions unfold over time and under changing environmental conditions. This survey synthesizes 257 papers spanning agent evaluation, software assurance, cyber-physical systems, runtime monitoring, and regulatory guidance in order to characterize the validation problem for agentic systems. The review is organized around a five-dimension taxonomy covering behavioral, safety, temporal, regulatory, and multi-agent concerns, and uses that taxonomy to map current approaches and expose recurrent coverage gaps. The analysis shows that behavioral evaluation is comparatively

1

mature, while temporal validity, runtime evidence maintenance, regulatory legibility, and open-ended multi-agent systems assurance remain under-developed. Three cross-domain case studies (medical care, industrial operations, smartmobility systems) provide operational illustrations of how the five taxonomy dimensions recur in safety-critical settings, grounded in the failure patterns documented in the reviewed literature. The paper concludes with a lifecycle-oriented research agenda centered on bounded-autonomy specifications, adversarial trajectory generation, runtime monitoring, and audit-ready evidence structures. The central claim is that trustworthy deployment of agentic AI depends on validating trajectories in context rather than assessing isolated components alone. Keywords: agentic AI, agentic systems, runtime validation, behavioral evaluation, software verification and validation, AI assurance

1 Introduction 1.1 The Agentic Shift Agentic artificial intelligence (AI) systems are increasingly deployed as software systems that pursue goals through multi-step reasoning, planning, memory, tool use, and adaptation under changing context (Luo et al. 2025; Zou et al. 2026). This shift changes the validation target. Classical software components primarily implement bounded functions over inputs, state, and interfaces, whereas agentic systems must be evaluated as policies acting over trajectories in dynamic environments. The question is therefore no longer only whether a component returns the correct output, but whether the overall system behaves acceptably over time, across interactions, and under changing operational conditions. Recent agent architectures make this shift concrete. Contemporary frameworks organize execution around loops of perception, planning, tool invocation, reflection, memory update, and possible delegation to other agents or services (Wu et al. 2024; Luo et al. 2025). Even when built from familiar software parts, such as models, Application Programming Interfaces (APIs), retrieval modules, schedulers, and user interfaces, their system-level behavior emerges from how these parts are orchestrated across trajectories rather than from any single module in isolation. The result is a software architecture that is more open-ended, history-dependent, and environmentcoupled than the systems for which mainstream testing abstractions were originally designed.

1.2 Five Testing Mismatches Software engineering already provides mature foundations for unit and regression testing (Myers et al. 2004; Beizer 1990), as well as integration testing and specificationbased verification (Ammann and Offutt 2016). Those foundations remain necessary. However, they were largely developed for systems whose relevant behaviors are relatively reproducible, decomposable, and specifiable at the level of components and interfaces. Agentic systems stress those assumptions in at least five recurring ways. 2

Table 1 Five abstract mismatches between classical testing assumptions and agentic-system validation targets.

Mismatch Determinacy mismatch

Classical testing assumption Repeating the same test should yield the same or predictably equivalent result.

Decomposition mismatch

Correct components compose into correct system behavior.

Specification mismatch

Requirements can be stated as explicit input–output properties or interface contracts.

Environment mismatch

Operating conditions can be bounded into stable test fixtures and controlled mocks.

Temporal mismatch

Correctness is treated as a property established prior to deployment.

Agentic-system reality

Validation implication Stochastic inference, Validation contingent planning, and must tool availability can produce characterize different trajectories from acceptable the same starting state. variation, not only exact reproducibility. Failures often arise from Evidence orchestration across planner, must cover memory, tools, users, and end-to-end other agents. trajectories and crosscomponent interactions. Goals such as “act helpfully,” Validation “escalate appropriately,” or needs “use tools safely” are boundedpartial, contextual, and autonomy temporally extended. contracts, runtime constraints, and scenariobased evidence. Agent behavior depends on Assurance dynamic environments, must include external services, changing simulation, data, and human responses. adversarial scenarios, and monitoring in context. Agentic behavior unfolds Validation over long horizons and can must become drift with memory, updates, lifecycleand changing workflows. oriented, with postdeployment oversight and evidence refresh.

3

Table 1 shows that unit, regression, integration, and specification-based tests cover only part of the assurance space. The validating question expands from “Did the component return the correct output?” to “What evidence justifies trust in this system’s behavior over realistic trajectories and operating conditions?” Established softwareengineering practice therefore remains essential but requires extension for agentic autonomy. Section 5 returns to these same five abstract mismatches by examining them across classical testing methods. In that comparison, fault injection is listed separately because it operationalizes both decomposition and environment concerns within the five mismatches of Table 1.

1.3 Why a Validation-Centered Survey Now Three developments make a validation-centered synthesis timely. The first is a gap in the survey landscape. The agent surveys of 2024–2026 map architectures, capabilities, benchmarks, and evaluation practice with increasing precision (Yehudai et al. 2025; Luo et al. 2025), but they organize the field by what agents can do, not by what must be shown before and during consequential deployment. As Table 3 makes explicit, temporal validity and regulatory legibility are rarely treated as first-class axes among the closest adjacent surveys, and almost none rests on a systematically screened corpus. A reader of the existing literature can learn how agents are built and how well they score; what evidence package would justify trusting one in deployment is a question that literature does not yet organize itself around. The second development is that the governance clock is already running. FDA guidance on AI-enabled device software and predetermined change control, together with the MDCG positions on adaptive AI, has moved lifecycle evidence, traceability, and change control from aspiration to operative expectation. FDA guidance addresses AI-enabled devices and predetermined change control (U.S. Food and Drug Administration 2025a,b); MDCG guidance provides complementary European expectations (Medical Device Coordination Group 2025b,a). These documents state what must be demonstrated; the engineering literature that should supply the demonstration methods remains fragmented across the five streams surveyed here. The cost of that fragmentation is borne now, by teams deploying agentic systems in clinical, industrial, and mobility settings under evidence expectations that the technical literature does not yet operationalize. The third development is the rapid consolidation of the empirical base: 209 of the 257 included papers were published in 2025–2026 (Figure 2). This concentration makes it possible to synthesize agent-specific validation work while evaluation norms are still developing.

1.4 Contributions This survey makes four contributions: 1. Systematic literature review. We synthesize five partially disconnected bodies of literature: classical software engineering (SE) testing, agent-framework evaluation, Cyber-Physical Systems (CPS) validation, runtime assurance, and regulatory 4

guidance, through a PRISMA-inspired screening of 7,197 retrieved records down to 257 included papers. 2. Five-dimension taxonomy. We introduce a standalone taxonomy of validation challenges for agentic systems spanning behavioral, safety, temporal, regulatory, and multi-agent dimensions, with explicit validation objects, characteristic failure modes, and measurable metrics for each. Its distinguishing choice is to treat temporal validity and regulatory legibility as first-class axes rather than as remarks appended to capability valuation, the two axes on which adjacent surveys are weakest (Section 6). 3. Quantified gap analysis. We derive gaps from the coded corpus: a coverage matrix over approach families and validation dimensions identifies which pairings are mature and which remain structurally under-addressed, with adversarial sensitivity bounds showing that the directional claims survive worst-case reassignment of borderline papers (Section 7). 4. Research agenda. We translate the identified gaps into a four-direction validation stack with candidate metrics andsoftware-engineering targets, moving from a generic call for better evaluation toward a concrete research program (Section 9). The remainder of the paper is organized as follows. Section 2 describes the survey methodology. Section 3 organizes prior literature along five validation axes. Section 4 defines agentic systems and the expanded assurance target, while Section 5 analyzes which classical software-testing assumptions break and which still carry over. Section 6 presents the five-dimension taxonomy, and Section 7 maps existing approaches onto it to identify directional gaps. Section 8 grounds the taxonomy in three consequential domains, Section 9 develops the research agenda, Section 10 discusses open challenges, and Section 11 concludes.

2 Survey Methodology 2.1 Research Questions The review is structured by four research questions: which classical software-testing assumptions fail for agentic systems (RQ1), which validation dimensions a comprehensive framework must cover (RQ2), how existing approaches distribute across those dimensions (RQ3), and which research directions follow from the remaining gaps (RQ4). These RQs map directly onto Sections 5, 6, 7, and 9.

2.2 Search Strategy We searched five primary databases: ACM Digital Library, IEEE Xplore, Scopus, arXiv (cs.SE, cs.AI, cs.MA, cs.RO), and Semantic Scholar. The initial search was conducted between January and April 2025, followed by a refresh pass between January and March 2026 that re-interrogated the same five sources to capture newly indexed 2025–2026 publications. The review covers publications from January 2019 to March 2026, with selective inclusion of foundational earlier work where it defines the classical baseline being challenged. Unless noted otherwise, counts reported below refer to the consolidated retrieval set after both passes. 5

Each database was queried with a core string combining agent-identity terms (“agentic,” “LLM agent,” “language model agent,” “autonomous agent”), validation terms (“validation,” “assurance,” “verification,” “testing,” “evaluation”), and trajectory-level terms (“trajectory,” “tool use,” “multi-step,” “stateful,”) adapted to each database’s field-search syntax. Dimension-specific expansion added behavioral terms (e.g., “benchmark,” “trajectory consistency”), safety terms (e.g., “runtime assurance,” “safe autonomy”), temporal terms (e.g., “concept drift,” “evidence freshness”), regulatory terms (e.g., “assurance case,” “post-market surveillance”), and multi-agent terms (e.g., “coordination,” “emergent behavior”). We additionally performed forward and backward citation searches from agentevaluation anchor papers (Luo et al. 2025; Yehudai et al. 2025) and CPS-validation anchor papers (Collaco et al. 2026; Zhao et al. 2026), and included regulatory documents from the Food and Drug Administration (FDA) and Medical Device Coordination Group (MDCG) not indexed in academic databases. All retrieved records were exported with bibliographic metadata (including title, abstract, venue, year, and DOI where available) and consolidated into a unified dataset for screening and analysis. The review process follows a Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)-inspired structure (Page et al. 2021) with explicit reporting of deduplication, screening, and inclusion stages. The merged retrieval set used for screening contained 7,197 unique records after both passes. Of these, 7,125 carried a single-source provenance tag: 4,575 from IEEE Xplore, 1,194 from arXiv, 709 from the ACM Digital Library, 364 from Semantic Scholar, and 283 from Scopus. A further 72 unique records were retrieved from two or more of those sources and were retained as a separate multi-source provenance category for audit traceability. Regulatory and standards documents from the FDA and MDCG were retained as background references but are not counted in the screened paper corpus. IEEE Xplore alone supplies 64% of single-source records (4,575/7,125), well above arXiv (1,194); Section 2.8 returns to this skew as a threat to validity. Deduplication was performed using title and source normalization, followed by manual inspection for ambiguous cases and cross-source matches.

2.3 Inclusion and Exclusion Criteria The following criteria governed all three screening stages. They were defined and frozen before screening began and applied consistently across all records. Inclusion criteria. A paper was included if it satisfied all of the following conditions:

• I1: Validation relevance. The primary contribution had to address validation, testing, verification, runtime monitoring, assurance, benchmarking, safety enforcement, or auditable oversight of software behavior. • I2: Agentic execution feature. The evaluated system had to execute over a trajectory rather than a single inference step. For corpus-inclusion purposes, this required at least one agentic execution feature : explicit planning across steps, tool

6

invocation with feedback, persistent memory/state carry-over, closed-loop interaction with an external environment, or coordination among multiple autonomous components. These disjunctive criteria ensured that papers addressing trajectorylevel validation in any form could enter the corpus, even if they did not emphasize all three canonical agentic properties. • I3: Software-level scope. The paper had to contain a substantive software or systems contribution relevant to agentic AI, CPS, runtime assurance, or AI-enabled software engineering. • I4: Sufficient technical content. Empirical studies, benchmark papers, frameworks, formal methods papers, and technical reports were included only if they supplied enough methodological detail to support analytical coding. • I5: Time and language window. The source had to be published in English and fall within the January 2019 to March 2026 search window, unless it was a deliberately retained foundational baseline source.

Exclusion criteria. A paper was excluded if any of the following conditions held: • E1: Single-shot inference only. The paper evaluated a single forward pass, a static prompt-response exchange, or prompt quality without an autonomous trajectory. • E2: Prompt-only orchestration. The system used prompt templates, selfconsistency, or prompt ensembles but did not expose persistent state, tool-mediated feedback, iterative control, or interacting agents. • E3: Non-agentic adjacent AI. The contribution focused on perception-only natural language processing (NLP), vision, or prediction tasks not embedded in an autonomous action loop. • E4: Non-software assurance target. The paper addressed hardware reliability, sensor fault tolerance, or physical device dependability without a software-level validation contribution. • E5: Insufficient scholarly record. Editorials, blog posts, slide decks, and very short abstracts without retrievable technical content were excluded. Preprints were retained only when they were the principal technical record for a widely used agentic benchmark or framework. The review focuses on software-level validation of agentic systems. Formal verification of hybrid systems and model checking of finite-state software provide background context, while hardware reliability and sensor fault tolerance are included only when they directly affect software-level validation claims.

2.4 Paper Selection Selection proceeded in three sequential screening stages, each applying the criteria of Section 2.3 at increasing levels of granularity. Figure 1 reports record counts at each stage; the decision trace records exclusion criteria at the record level for audit purposes.

7

Stage 1: Title screening (7,197 to 3,214). Each of the 7,197 deduplicated records was assessed on title alone. A record was excluded only when its title provided unambiguous evidence of out-of-scope content: hardware reliability or sensor fault tolerance without a software validation contribution (E4); NLP/vision/speech work with no agentic framing and no assurance term (E3); or LLM capability studies without any testing or verification component (E1, E2). Ambiguous titles were retained for abstract review. This pass reduced the corpus from 7,197 to 3,214 records (55% reduction). Stage 2: Title-and-abstract screening (3,214 to 561). The 3,214 records were assessed on title and abstract together. The most frequent exclusion reasons were: papers using “agent” in a non-agentic sense such as reinforcement-learning (RL) agents on low-level state spaces (E2, E3); benchmark papers evaluating static model capabilities without a trajectory or tool-use component (E1); and papers addressing LLM evaluation without a lifecycle or assurance framing, where validation appeared as a secondary concern (I1 not satisfied). This pass reduced the corpus from 3,214 to 561 records (83% reduction). Stage 3: Full-text review (561 to 257). The 561 records were reviewed in full. Exclusions arose from three patterns: architecturally oriented papers whose validation component was too thin to constitute a primary contribution (I1, I4 not satisfied); papers proposing validation methods evaluated exclusively on static benchmarks with no agentic trace as the unit of assessment (E1); and workshop or position papers too preliminary to provide citable methodological detail (E5). Of the 257 retained papers, 237 were assigned high confidence and 20 medium confidence based on centrality to a taxonomy dimension.

Fig. 1 PRISMA-style workflow for literature identification, screening, eligibility assessment, and inclusion. The retrieval stage yielded 7,197 unique records after source merging and deduplication across five primary sources. Sequential screening reduced the corpus to 257 papers included in the final survey.

8

2.5 Coding Scheme The unit of analysis is the individual paper, even when multiple systems or evaluations are reported. Each paper was coded along three axes: primary validation dimension, approach family, and domain context. The validation dimensions (behavioral, safety, temporal, regulatory, and multi-agent) are defined operationally in Section 6 and applied consistently during coding. Papers could contribute evidence to more than one dimension, but a single primary dimension was assigned based on the main validation objective emphasized by the work; secondary contributions are recorded in the coding trace but do not duplicate counts across the primary-dimension totals reported in Section 7. The initial screening and study-selection stages were conducted by a single reviewer. To strengthen the reliability of the analytical phase, we ran two distinct reliability checks. The first is an inter-rater check: a second reviewer independently classified all 257 included papers, achieving substantial agreement with the primary reviewer (Cohen’s κ = 0.759) for this classification step. The second is an intra-rater check on the primary reviewer’s own consistency over time: the codebook was frozen before full-corpus coding and applied by the primary reviewer in three passes. Pass 1 assigned a primary dimension and confidence label (high/medium) to all 257 papers; Pass 2 re-coded, after a week-long interval, an 84-paper audit subset (all 20 mediumconfidence papers plus a 64-paper stratified high-confidence sample); Pass 3 reconciled discordant cases against the written codebook, logging each adjudication decision. Within-reviewer stability was 76/84 before adjudication; the eight discordant cases all fell at pre-specified borderlines (behavioral/safety, temporal/regulatory, safety/multiagent). This intra-rater procedure documents within-reviewer stability and remaining coding ambiguities; it is distinct from the independent classification check reported above and in Section 2.8.

2.6 Corpus Composition Two descriptive views of the 257-paper corpus ground the analysis in later sections. The first is primary-dimension composition: under the frozen codebook each included paper carries exactly one primary dimension, yielding 71 behavioral (27.6%), 62 temporal (24.1%), 50 multi-agent (19.5%), 42 safety (16.3%), and 32 regulatory (12.5%) papers. Behavioral work is the largest primary category, but the corpus is not dominated by a single validation style: temporal and multi-agent work occupy substantial shares, while regulatory work is the smallest primary category. The second view is recency: the corpus is heavily concentrated in recent years (1 paper pre-2019, 29 in 2019–2023, 18 in 2024, 189 in 2025, and 20 in January–March 2026), with the sharpest growth from 2024 onward. These patterns motivate the directional gap analysis in Section 7; Figure 2 reports the year-band composition.

2.7 Automated Classification Validation The second reviewer’s role was limited to validating the classification of the included studies rather than participating in the primary screening pass. Across the 257

9

Fig. 2 Primary-dimension composition of the corpus by publication-year band. Each included paper carries exactly one primary dimension, so the per-band counts (n) sum to the 257 included papers.

included papers, the two human reviewers achieved Cohen’s κ = 0.759. Four openweight classifiers (qwen2.5:3b, llama3.2:3b, ministral-3:3b, phi4-mini:3.8b) were added as auxiliary raters; their agreement improved from the zero-shot condition (Fleiss’ κ = 0.559) to the few-shot condition (Fleiss’ κ = 0.665), where the Fleiss coefficient is computed across the four classifiers alone. The primary reviewer enters the separate pairwise Cohen comparison against the second reviewer. These results show that the taxonomy supports independent classification while retaining borderline cases for human judgment.

2.8 Limitations and Threats to Validity The first limitation concerns corpus coverage and timing. Agentic AI validation remains terminologically unstable, and terms such as “assurance,” “evaluation,” and “monitoring” span overlapping research programs. The search therefore used expanded queries across five databases, citation chaining from anchor papers, and regulatory documents not consistently indexed in academic databases. Residual risks include terminology emerging after the March 2026 cut-off, metadata inconsistencies in preprint-heavy venues, and the dominance of IEEE Xplore in the merged retrieval set, which may under-represent some machine-learning venues.

10

The second limitation is reviewer dependence. A single reviewer conducted title, abstract, and full-text screening, creating a risk of systematic selection bias. Classification was independently validated by a second reviewer and is therefore less exposed to this limitation. For classification, the codebook was frozen before full-corpus coding; a second reviewer independently classified all 257 included papers (Cohen’s κ = 0.759); and four LLM classifiers served as auxiliary raters. The eight Pass 1/Pass 2 discordances occurred at the three pre-specified codebook boundaries (behavioral versus safety, temporal versus regulatory, and safety versus multi-agent). The adversarial sensitivity analysis in Table 2 shows that the directional claims persist under worst-case reassignment of all eight cases. Internal checks cannot exclude systematic interpretation bias in the single-reviewer selection stage, which may have shifted corpus composition. Accordingly, the analysis emphasizes directional patterns, and Table 2 reports how plausible reassignment changes the primary-dimension totals. Table 2 Sensitivity of paper-level primary-dimension counts to adversarial reassignment of the eight Pass 1/Pass 2 discordant cases documented in the adjudication record. “Nominal” is the adjudicated primary-dimension count; “Range” gives the minimum and maximum count obtained when each discordant case is reassigned to its non-adjudicated alternative in the direction least or most favorable to that dimension. Every reassignment preserves the 257-paper total.

Dimension

Nominal

Worst-case range

Behavioral Safety Temporal Regulatory Multi-agent

71 42 62 32 50

[70, 72] [40, 45] [61, 63] [30, 33] [48, 52]

3 Related Work This section positions the survey against five adjacent literatures that partially address agent validation but rarely yield a single assurance framework. The through-line is that each body of work captures one part of the problem, whether benchmarking, architectural evaluation, CPS validation, runtime assurance, or governance, while leaving other validation objects under-specified. Together, they show the need for a lifecycle account of what must be validated, when validation must be renewed, and what evidence must remain available for review.

11

3.1 Classical SE Testing Limits Recent evaluation work converges on a familiar software-engineering lesson: task success is not equivalent to system assurance. Agent-evaluation surveys show that prevailing benchmarks privilege outcome completion while under-specifying trace quality and tool-use correctness (Yehudai et al. 2025; Luo et al. 2025). Work on diagnosis makes the remaining limitation explicit: failure explanations are rarely part of the benchmark target (Mohammadi et al. 2025). Static benchmark suites are vulnerable to contamination and leaderboard aging (Liang et al. 2022; Lipton and Steinhardt 2018); recent evidence documents these effects directly in agent evaluation (Chen et al. 2025). AgentBench (Liu et al. 2023) established the baseline for multi-environment evaluation but reports task completion rather than trajectory acceptability; SWE-rebench (Badertdinov et al. 2025) and ITBench (Jha et al. 2025) show that contamination-free refresh can materially change ranking, confirming that outcome scores are environment-snapshot artefacts. Convergent benchmark work such as AgentBoard, TheAgentCompany, and AgentHarm is important here because it broadens evaluation from static question answering to multi-turn interaction, consequential office-style tasks, and harmfulaction exposure, yet still leaves intermediate-step legibility and trajectory acceptability only partially specified: AgentBoard and TheAgentCompany broaden the interaction setting (C. Ma et al. 2024; F. F. Xu et al. 2025), whereas AgentHarm foregrounds harmful-action exposure (Andriushchenko et al. 2024). Convergent evidence across clinical, cybersecurity, logistics, and financial deployments confirms that the outcomescore gap is domain-independent: clinical and cybersecurity studies report it in high-consequence settings (Arjun Warrier and Abhilash K S 2025; Anshuman Chhabra et al. 2026), and logistics and planning studies report the same limitation in operational workflows (Giuliano Lorenzoni et al. 2025; Xiaodong Zheng et al. 2025). The practical limit of this literature is therefore not lack of evaluation effort, but the narrow validation target encoded in most benchmark designs.

3.2 Agentic Frameworks Evaluation Gaps A second literature evaluates agent frameworks directly, but in terms of capability rather than assurance. Wang et al. (2024) provide the most comprehensive architectural survey of LLM-based agents, covering memory, planning, tool use, and interaction patterns, but do not define validation dimensions or conduct systematic corpus coding. Acharya et al. (2025) and Abou Ali et al. (2025) offer further architectural surveys with attention to safety risks and governance, but neither treats temporal lifecycle validity or regulatory evidence legibility as first-class assurance targets. More focused work adds trajectory awareness only partially: MultiAgentBench (Zhu et al. 2025) and MAST (Cemri et al. 2025) score coordination outcomes and failure modes, AutoGen-style framework studies (Wu et al. 2024) expose orchestration patterns, recent protocol-oriented work such as Autogenesis (Wentao Zhang et al. 2026) makes explicit that agent protocols themselves can evolve across deployment, and studies of asynchronous software-engineering agents show that handoff, parallelism, and coordination policy materially affect system behavior even when the base model is fixed (Jiayi Geng and Graham Neubig 2026), yet the field still lacks shared methods for validating

12

whether trajectories remain acceptable under changing tools, humans, and environments (Luo et al. 2025; Zou et al. 2026). In other words, framework surveys explain how agents are assembled, but still leave open how those assemblies should be assured once their components evolve in deployment.

3.3 CPS Trajectory Validation The CPS literature contributes a more deployment-grounded view by treating behavior as closed-loop, temporal, and environment-coupled. Work on digital twins, runtime safety, autonomy in medical and industrial systems, and International Conference on Cyber-Physical Systems (ICCPS)-style validation emphasizes that hazards emerge from feedback, delays, stale state, and coupled infrastructure, not only from wrong final outputs. Digital-twin and ICCPS work emphasize feedback, delays, and stale state (Veledar et al. 2019; Lin et al. 2025), while recent CPS validation studies examine coupled infrastructure and closed-loop autonomy (Brown et al. 2025; Collaco et al. 2026). Medical and industrial agent studies extend the evidence to deployment settings (Zhao et al. 2026; Sharma et al. 2026); healthcare simulation work shows related trajectory risks (Draelos et al. 2026; The MITRE Corporation 2026), including in gym-style evaluation environments (Kuo et al. 2022). This literature is closest to the present paper’s assurance target, but it is usually domain-specific and does not systematically connect CPS validation concerns to the broader wave of LLM-based agentic systems. Its importance here is conceptual: it shows that trajectory validity is inherently relational and time-dependent, even when the surrounding evaluation culture is not yet organized around LLM-agent terminology.

3.4 Runtime Assurance and Monitoring Runtime-assurance work addresses the gap that static testing cannot close once systems adapt in the field. Greshake et al. (2023) established the indirect promptinjection problem, where malicious content in tool outputs hijacks agent behavior, as one of the most studied runtime safety failures in the current corpus, motivating the trajectory-level containment gap identified in Section 7. Research on runtime verification, assurance monitors, drift management, policy shields, and expert monitoring shows how safety envelopes, conformance checks, and evidence refresh can intervene after deployment rather than treating release as the end of validation. Assurance and runtime-verification work supplies conformance and safety-envelope mechanisms (Hawkins et al. 2021; Zhong et al. 2025), while MLOps and expertmonitoring work focuses on evidence refresh in operation (Kreuzberger et al. 2023; Leest et al. 2024). Safe control and regulatory guidance specify complementary obligations (Alshiekh et al. 2018; U.S. Food and Drug Administration 2025b); MDCG guidance and anomaly monitoring extend these concerns to evolving deployments (Medical Device Coordination Group 2025a; Poenaru-Olaru et al. 2024). More agent-specific systems such as AgentGuard and ControlA move from static evaluation toward runtime verification and workflow control for autonomous or semi-autonomous agents, addressing local monitoring and containment problems within an agentic execution (Roham Koohestani 2025; Amal Gueroudji et al. 2025). What remains missing

13

is integration: most monitoring papers specify local safeguards, but not a full stack connecting requirements, simulation, runtime evidence, and audit-ready assurance artifacts. This is the clearest sign that runtime assurance is a necessary layer of validation, but not a substitute for the upstream and downstream evidence structures that make interventions explainable.

3.5 Regulatory and Assurance Frameworks Regulatory and assurance-case literatures specify the accountability requirements that technical evaluation must satisfy. FDA, MDCG, International Electrotechnical Commission (IEC), European Union (EU), and Institute of Electrical and Electronics Engineers (IEEE) guidance consistently foreground lifecycle evidence, change control, documentation, human oversight, and post-market surveillance. FDA guidance foregrounds lifecycle evidence and good machine-learning practice (U.S. Food and Drug Administration 2021; U.S. Food and Drug Administration et al. 2021), while its AI-enabled-device guidance treats adaptive change explicitly (U.S. Food and Drug Administration 2025a). MDCG documents address software and adaptive AI governance (Medical Device Coordination Group 2025b,a); the EU MDR, IEC 62304, and IEEE guidance provide the surrounding accountability requirements (European Union 2017; International Electrotechnical Commission 2006), with IEEE guidance providing an additional reference point (IEEE Standards Association 2025). Assurance-case methods, including Goal Structuring Notation (GSN)-style argumentation and recent responsible-AI frameworks for healthcare autonomy, show how heterogeneous evidence can be assembled into reviewable claims (Alelyani 2025). The unresolved problem is evidentiary generation: governance documents say what must be shown, but not how agentic systems should be tested and monitored so that those claims can be credibly made. Recent assurance-oriented work on agentic GenAI risk mapping and broader AI-safety assurance methods is therefore relevant to the present corpus because it starts to translate governance obligations into candidate evidence structures, even if the trajectory-level instrumentation problem remains open (Ray 2026; Schnitzer et al. 2024).

3.6 Positioning of This Paper Our paper sits at the intersection of these five axes. Relative to classical testing, it explains why component-oriented methods remain necessary but no longer define the full assurance target. Relative to agent-framework evaluation, it shifts attention from benchmark scores to trajectory validation in context. Relative to CPS work, it generalizes environment-coupled validation beyond any single domain. Relative to runtime-assurance research, it places monitoring inside a lifecycle stack rather than as an isolated safeguard. Relative to regulatory and assurance-case work, it translates governance demands into concrete validation objects. The integrative contribution is a unified view of the validation problem across the residual gaps of these literatures. To situate this survey relative to the closest adjacent reviews, we compare them along six properties: publication year, whether they release a systematically screened corpus, whether they organize the field by validation objects rather than capabilities or

14

Table 3 Comparison with adjacent surveys. Corpus: S = systematic screened corpus, O = curated overview. Organization: VO = validation objects, CAP = capabilities, DOM = domains. Primary dimensions: B = Behavioral, Sa = Safety, T = Temporal, R = Regulatory, M = Multi-agent. Gap analysis: Y = explicit, P = partial, N = absent. Agenda: M = directions with candidate metrics, D = directions without metrics, N = none.

Survey

Year Corpus Organization Evaluation dimensions / Agenda gap analysis

Wang et al. Luo et al. Yehudai et al. Acharya et al. Abou Ali et al. Zhao et al. Mohammadi et al. Zou et al. Yu et al. Akshata Kishore Moharir et al. Zha et al. Chen et al. Zhang et al. Schnitzer et al. Collaco et al.

2024 2025 2025 2025 2025 2026 2025 2026 2025 2025

O O O O O O O O O O

CAP CAP CAP CAP CAP DOM CAP CAP CAP CAP

{B} / P {B} / P {B} / P {B} / N ∅/N {B, Sa, R} / P {B, Sa} / P {B, M} / P {Sa, M} / P {Sa, M} / N

D D N N N D D D D N

2023 2025 2022 2024 2026

O O O O S

DOM DOM CAP VO DOM

{B} / N {T} / P {B, Sa} / N {Sa, R} / P {Sa} / P

D D D M D

This survey

2026

S

VO

{B, Sa, T, R, M} / Y

M

domains, which of the five validation dimensions they treat as primary axes, whether they contain an explicit gap analysis, and whether they produce a research agenda with candidate metrics. These are the properties that most clearly distinguish an inspectable synthesis survey from a high-level overview. Table 3 shows four recurring patterns: most adjacent surveys are curated overviews rather than systematic syntheses; the field is organized by capabilities or domains rather than validation objects; temporal, regulatory, and multi-agent coverage are rarely treated as co-equal axes; and few surveys combine explicit gap analysis with a metrics-based research agenda. These are the gaps this paper is structured to address.

4 Agentic Systems: Definitions and Validation Challenges 4.1 What Makes Systems “Agentic” We use agentic system to denote a software system that pursues a goal through temporally extended action rather than through a single inference. Three properties matter for validation. First, the system maintains state across steps, whether through memory, retrieved context, or workflow state. Second, it acts through tool use, delegation, or other environment-coupled interventions. Third, its relevant behavior is a trajectory: planning, execution, escalation, recovery, and stopping conditions may all matter independently of the final answer. This definition covers contemporary LLMbased agents while remaining broad enough to include software-intensive autonomy 15

in CPS and enterprise workflows. Agent-evaluation surveys support the former comparison (Luo et al. 2025; Zou et al. 2026), while CPS validation studies motivate the latter (Collaco et al. 2026; Zhao et al. 2026). Formally, we use trajectory to denote a finite temporally ordered sequence τ = ((s0 , a0 , m0 ), . . . , (sT , aT , mT )), where each st is the relevant system–environment state at step t, each at is the action selected at that step, and each mt records minimal execution annotations such as tool invocation, retrieved evidence, delegation, or human handoff. This definition is intentionally lightweight: the state may be partially observed, the action may be symbolic or natural-language, and the annotations need only be rich enough to reconstruct why a step occurred and what external intervention it triggered. We separate three boundary terms. An agentic execution feature is any trajectoryrelevant property used for screening: planning across steps, tool invocation with feedback, persistent state, closed-loop external interaction, or coordination among autonomous components. A canonical agentic system satisfies the three conceptual properties together: persistent state, environment-coupled action, and trajectory-level behavior. An LLM-based agent with planning loops, persistent memory, and toolcall sequences exemplifies this canonical case. A consequential assurance trigger arises when any agentic execution feature affects workflows, infrastructures, or physical processes strongly enough that lifecycle validation becomes necessary. This distinction matters operationally. The corpus-inclusion criterion (I2) is disjunctive because a paper enters the corpus if it addresses trajectory-level validation of any system exhibiting at least one agentic execution feature. A stateful dialog manager with no external tool calls occupies a boundary zone and is included only if the paper demonstrates that its lifecycle or trajectory-level properties matter for assurance. Conversely, a stateless tool dispatcher that invokes an external service is included if the paper treats the multi-step outcome as the unit of validation rather than only the final response. The canonical cases form the primary target of the taxonomy and most of the survey findings, while the permissive inclusion boundary captures consequential assurance triggers that do not emphasize all three canonical properties.

4.2 Architecture Typology For validation purposes, three architectural patterns matter. Pipeline systems follow predetermined stages with limited branching, so the main risks are interface mismatches, stale handoffs, and accumulation of stage-local errors. A clinical documentation pipeline that extracts findings, classifies urgency, and drafts a note in fixed sequence illustrates this pattern: errors in the extraction stage propagate silently into later stages, and the fixed stage order offers no mechanism for dynamic error recovery. Network systems route dynamically across tools, planners, or peer agents, shifting attention to coordination failures, non-local causality, and weak trace reproducibility. A code-repair agent that queries a retrieval index, invokes a linter, calls an LLM planner, and conditionally delegates to a test executor illustrates this pattern: the same defect report can trigger different tool invocation sequences across runs, making it difficult to define a canonical trajectory against which to evaluate correctness. Human-in-the-loop systems insert review, override, or escalation checkpoints, 16

making calibration of handoffs, authority boundaries, and accountability for mixed human–agent trajectories the dominant assurance concern. A triage agent that surfaces high-risk cases to a clinician while autonomously closing low-risk ones illustrates this pattern: validation must address not only whether the agent acts correctly in isolation, but whether its escalation threshold is well-calibrated, neither suppressing alerts that require human review nor generating so many interruptions that human oversight becomes ineffective in practice.

4.3 The Expanded Assurance Target Once a system is agentic, the assurance target expands from isolated component outputs to acceptable trajectories in context. The relevant questions become whether the system gathered the right evidence, used tools appropriately, respected safety and escalation boundaries, coordinated correctly with humans or peer agents, and remained valid as the environment changed. This is why the paper’s central validation object is not the model response alone, but the trajectory realized by the socio-technical system. This expansion has a direct methodological consequence: test adequacy can no longer be defined by coverage of input–output pairs. A test suite that exercises all intended tool calls in isolation may still fail to expose trajectory-level failures, such as incorrect tool ordering, unsafe intermediate states, or recovery breakdowns, that only become visible when the full action sequence is examined end to end. The fivedimension taxonomy in Section 6 articulates what trajectory-level adequacy requires across behavioral, safety, temporal, regulatory, and multi-agent concerns.

4.4 Scope and Boundary Conditions The paper focuses on software-intensive agentic systems whose actions are consequential because they affect workflows, infrastructures, or physical processes. Full lifecycle assurance applies when systems exhibit an agentic execution feature and act consequentially; in these settings, classical testing no longer covers the full assurance target. Table 1 and the five-dimension taxonomy in Section 6 specify where that coverage breaks. The dividing line is not model size or interface type, but whether the system’s assurance target requires reasoning about sequences of actions, state transitions, and environment feedback rather than about isolated responses. These trajectory-level assurance burdens are triggered disjunctively: any one of them can be sufficient to create a validation gap that classical component testing cannot close. That is why a system that exhibits only one of the three canonical agentic properties (persistent state, environment-coupled action, trajectory-level behavior) can still warrant lifecycle assurance if it operates consequentially. The inclusion boundary (I2) therefore remains disjunctive at the corpus level, while the conceptual core of agentic systems remains the canonical case where all three properties hold. The research agenda and gap analysis focus primarily on that canonical case, because that is where the convergence of all five validation dimensions becomes most pressing.

17

5 Why Classical SE Testing Assumptions Break 5.1 Core Assumptions of Classical Testing Classical software testing assumes that behavior is sufficiently stable, decomposable, specifiable, and environment-bounded that evidence from unit, integration, regression, and system tests can be composed into a credible judgment about the deployed system. Those assumptions remain valuable for deterministic subsystems, tool wrappers, validators, interfaces, and change control. They stop being sufficient when the system’s relevant behavior is an open-ended action trajectory. Each assumption encodes a tacit decomposability claim. Unit testing assumes that function-level correctness composes into system-level correctness. Integration testing assumes that interface contracts are stable enough that component interactions can be verified once and trusted thereafter. Regression testing assumes that prior behavior is a meaningful baseline against which to detect degradation. Specification-based testing assumes that requirements can be expressed as checkable pre- and postconditions. System testing assumes that a bounded set of representative scenarios covers the space of safety-relevant behaviors. Fault injection typically assumes that failures can be injected at identified components and traced through sufficiently bounded interactions. In agentic systems, none of these assumptions holds without qualification, as the structural argument below and the empirical examples below both show.

5.2 Empirical Mismatch Examples Table 4 summarizes the structural mismatch by operationalizing the five abstract mismatches from Table 1 across six familiar testing methods; the fault-injection row is separated as a method-level view of breakdowns that cut across decomposition and environment mismatch rather than as a sixth abstract mismatch. The key empirical point is that the failure is visible in real deployments, not only in abstract argument. Three short examples make the mismatch concrete. In vessel-traffic and smartmobility settings, superficially similar observations can induce different action sequences because timing, routing context, and human responses alter which trajectory is safe; validating the final maneuver alone misses the path-dependent hazard (Brown et al. 2025; Lin et al. 2025). In power and industrial systems, planners that look accurate on nominal scenarios can still fail when stale telemetry, delayed actuation, or tool disagreement shifts the safe operating window (Collaco et al. 2026; Sharma et al. 2026). In clinical workflows, an agent may produce a plausible recommendation while relying on stale lab data, mis-timed escalation, or incomplete chart retrieval; the clinically salient failure is the trajectory, not the surface plausibility of the endpoint. CPS and clinical studies demonstrate this concretely (Zhao et al. 2026; Draelos et al. 2026), while expert-monitoring work describes its operational consequence (Leest et al. 2024).

5.3 What Still Carries Over Classical methods remain essential: component testing, interface contracts, hazard analysis, fault injection, and change control retain their roles, while machine-learning (ML) engineering adds versioning and conservative deployment discipline (Zinkevich 18

Table 4 Why classical testing abstractions are insufficient for agentic systems. The six method rows operationalize, rather than extend, the five abstract mismatches in Table 1, and the table should be read as an author synthesis of the classical testing and agentic-systems assurance literatures discussed in this section and in Section 4.

Testing method

Core assumption

Why agentic systems break it

Operational implication

Unit testing

Reproducible function behavior

Passing micro-tests does not imply dependable autonomous behavior

Integration testing

Stable interfaces and predictable composition

Regression testing

Prior behavior is a stable baseline

Specificationbased testing

Requirements can be stated as complete properties

System testing

Representative endto-end scenarios can be bounded

Fault injection

Injected faults can be isolated to bounded components and propagated through tractable interactions

The same apparent state can induce different reasoning traces, tool calls, and memory updates Tool APIs, retrieval behavior, and orchestration policies evolve during deployment Memory, model updates, and environment change invalidate historical baselines Goals such as “escalate appropriately” and “use tools safely” are partial and contextual Context spans tools, humans, timing, and coordination effects that labs underapproximate Breakdowns can cascade across planner, tool, memory, and human couplings and become only partially attributable at component level

19

Workflow-dependent appear after release

failures

Evidence refreshed

unless

decays

Runtime constraints and scenario evidence become necessary

Safety-critical hazards escape fixed test suites

Root-cause analysis becomes trajectory-level and often requires cascade-aware tracing

2016; Zhang et al. 2022). Testing research provides further methods for the transition (Riccio et al. 2020; Braiek and Khomh 2020). What changes is their role: each method retains validity within a circumscribed scope, component tests for input–output-stable subsystems, contract tests at API boundaries, Failure Mode and Effects Analysis (FMEA)/Hazard and Operability Study (HAZOP) when scoped to specific tool invocations or handoff points, fault injection targeting tool-call responses and inter-agent messages, and change control for model checkpoints and tool-schema versions. What none of these supplies alone is the composition of component-level assurances into trajectory-level claims. That gap is the entry point for the taxonomy in Section 6.

6 A Taxonomy of Validation Dimensions We organize the validation problem into five dimensions derived from the primary assurance burdens that recur across the corpus: trajectory quality, bounded safety, persistence of evidence over time, regulatory legibility, and collective behavior under interaction. This taxonomy of validation objects defines what is validated and why; coverage across the literature is examined in Section 7. Table 5 summarizes the dimensions, their characteristic failure modes, and representative metrics. The five dimensions correspond to five recurring questions in the surveyed literature: whether the trajectory is acceptable, whether it stays within risk bounds, whether that judgment remains valid after change, whether the supporting evidence is auditable, and whether local policies compose into acceptable joint behavior. These questions are analytically distinct even when they co-occur in deployment. A system can be behaviorally effective yet unsafe, temporally stale yet still presently auditable, or individually well-behaved yet collectively hazardous. This is why the taxonomy is organized by failure mode rather than by capability list.

6.1 Behavioral Dimension The behavioral dimension evaluates whether an agent’s realized trajectory is competent, consistent, and justifiable rather than merely whether the final answer looks plausible. The literature shows this from several directions: evaluation surveys establish the gap between outcome scores and trajectory quality (Yehudai et al. 2025; Luo et al. 2025); failure-mode studies locate breakdowns in reasoning transitions and toolinvocation sequences rather than in final answers (Cemri et al. 2025); data-centric work shows how quality and provenance defects propagate through otherwise plausible traces (Zha et al. 2023; Thomas Cook et al. 2025). Applied pipeline studies confirm the same pattern in production-like settings: orchestration and simulation-optimization studies expose it in automated decision pipelines (Leander Melroy Maben et al. 2025; Mohammad Peyman and Yusef Ahsini 2025), while workflow-generation and monitoring studies observe it in evolving processes (Zhongjun Yang et al. 2025; Agrippina Mwangi et al. 2025). Domain-adapted deployments supply further evidence (Sabrine Amri et al. 2025; Sijin Sun et al. 2025). The characteristic gap is outcome-preserving but procedurally defective behavior.

20

Table 5 Standalone taxonomy of validation dimensions for agentic systems. Each row states a validation object, a characteristic failure mode, general metrics, and a brief example. Dimension

Behavioral

Safety

Temporal

Regulatory

Multi-agent

Conceptual General measurable Brief example definition metrics (validation object + failure mode) Task-level trajectory: Task-completion rate; A support agent action choice, tool constraint satisfaction; chooses different refund sequence, state trajectory consistency; procedures for updates, and recovery recovery success; toolequivalent complaints, under realistic selection precision/recall; producing inconsistent variation. Failure escalation appropriateness. service outcomes. appears as brittle or unjustified behavior across semantically similar situations. Risk boundary for Unsafe-action rate; A warehouse robot autonomy during error, near-miss rate; severityagent routes forklifts uncertainty, or weighted incidents; policy into a restricted area interface failure. Failure violations; time to safe after receiving appears as harmful stop; intervention success. corrupted sensor data. action, unsafe tool use, or uncontrolled degradation despite nominal competence. Persistence of Drift slope; calibration A rehabilitationassurance claims as drift; memory planning agent becomes inputs, tools, policies, contamination; evidence less reliable after and memory evolve. freshness; revalidation therapy-slot encodings Failure appears as interval; post-update change. evidence decay after regression. drift, updates, or accumulated history. Evidence package for Trace completeness; A loan-processing agent traceable, auditable, audit-log coverage; cannot reconstruct why and governable requirement-to-test it changed a credit behavior under linkage; provenance recommendation after a external oversight. completeness; model update. Failure appears as change-impact latency; missing justification for evidence-backed claims. intended use, change control, provenance, or accountability. A delivery-routing Collective behavior of Coordination success; agent and an inventory interacting agents, conflict frequency; agent issue conflicting services, and humans. deadlock/livelock Failure appears when incidence; duplicated shipment instructions, causing delays. locally acceptable actions; interaction policies combine into latency; resource conflict, oscillation, contention. duplication, deadlock, or unsafe delay.

The practical implication is that behavioral validation cannot be reduced to measuring task success rates at the endpoint. An agent that achieves the correct final output via an inconsistent, unjustified, or brittle trajectory offers weaker assurance guarantees than one whose intermediate steps are themselves verifiable. This matters most when intermediate tool calls produce external side effects, including database writes, API calls that trigger downstream processes, and escalation decisions that route work to human reviewers, because a procedurally defective trajectory may leave the environment in an inconsistent state even when the final answer appears correct. Trajectory consistency under repeated trials and recovery success after perturbation are therefore first-class behavioral metrics, not secondary proxies, for this dimension.

6.2 Safety Dimension The safety dimension evaluates bounded autonomy: whether the system can avoid harmful actions, unsafe tool use, and uncontrolled degradation under error, uncertainty, or attack. In this survey, Safety is an intentionally aggregated dimension

21

because functional safety and adversarial security both define the action boundary that keeps agent autonomy within acceptable risk limits. Foundational safety work and agent-specific safeguards articulate bounded action (Amodei et al. 2016; Koohestani 2025), whereas adversarial benchmarks and autonomy analyses expose ways those bounds can be crossed (Vishal Kharde 2026; Shadi Nashwan et al. 2025). Threatadaptation and observability studies add operational evidence (Vinod B Maniyat and Arun Kumar B R 2025; Rahul Kapoor and Miray Kas 2026), including multilingual and cross-agent attacks (Basma ElSaify and Mohamed Baderelden 2025; Toqeer Ali Syed et al. 2025). The former emphasizes hazard analysis, fail-safe behavior, bounded degradation, and safe-stop or handoff mechanisms; the latter emphasizes prompt injection, tool misuse, privilege escalation, data exfiltration, and adversarial manipulation. Because the corpus skews toward the security-oriented strand, Safety counts should be read as aggregate breadth rather than as uniform maturity across both substrands. The underspecified-goal problem, in which agents appear safe while exploiting loopholes in vaguely specified objectives, is documented empirically as specification gaming and is a structural precursor to both safe-stop failures and security policy bypass (Turner et al. 2020). The recurring gap is therefore not just hazard detection, but the specification of safe-stop, defer, or handoff behavior that remains robust under underspecified goals, partial observability, and adversarial interference.

6.3 Temporal Dimension Temporal validation asks whether assurance evidence persists after updates, drift, memory accumulation, workflow change, or benchmark aging. This dimension synthesizes work from dataset shift, MLOps, post-deployment monitoring, and production ML research, all of which show that validation is a moving target rather than a one-time pre-release event. Benchmark-contamination and MLOps studies show that evaluation conditions and models both change over time (Chen et al. 2025; Kreuzberger et al. 2023); expert-monitoring and dataset-shift work describe the resulting evidencemaintenance burden (Leest et al. 2024; Quionero-Candela et al. 2009). Production-ML research supplies the complementary account of hidden technical debt and operational regression (Sculley et al. 2015; Breck et al. 2017), including rules for dependable deployment (Zinkevich 2016). Its central failure mode is evidence decay: a system once judged acceptable no longer merits the same claim after its operating context changes. This dimension is especially consequential for agentic systems because many of their assurance claims are path-dependent rather than purely input–output based. A benchmark score or pre-deployment test result may look stable even while the underlying trajectory-generating process has changed: memory may accumulate stale assumptions, retrieved evidence may shift in quality or provenance, tool interfaces may silently change their semantics, and escalation behavior may drift as surrounding workflows adapt. Three channels recur: memory or retrieved-context drift, tool-version and schema change, and environment or workload shift. In each case, a release-time claim can become stale without the system ever failing a release-time test. Temporal adequacy therefore requires explicit evidence-expiry conditions: what kinds of change invalidate prior claims, which traces or monitoring signals count as

22

sufficient warning, and what revalidation scope follows. Without that linkage, monitoring remains observational rather than normative, able to report change, but not to determine when a previously accepted deployment is no longer adequately validated. Section 9 develops this as a lifecycle-assurance direction.

6.4 Regulatory Dimension The regulatory dimension evaluates whether validation evidence is traceable, reviewable, and governable under external oversight. This is where audit trails, provenance, requirement-to-evidence linkage, change control, and post-market processes become part of the validation target rather than administrative afterthoughts. FDA documents articulate lifecycle evidence and good machine-learning practice (U.S. Food and Drug Administration 2021; U.S. Food and Drug Administration et al. 2021), and FDA’s AI-device guidance makes change control central (U.S. Food and Drug Administration 2025a). MDCG guidance provides the parallel European perspective (Medical Device Coordination Group 2025b,a), while the EU MDR and IEC situate it in wider oversight practice (European Union 2017; International Electrotechnical Commission 2006), with IEEE guidance adding a further reference point (IEEE Standards Association 2025). Assurance and evidence-design studies show how these requirements can be made reviewable (Alelyani 2025; Ewen Denney and Ganesh Pai 2023), with concise agentic proposals extending the argument (Maxim Chernyshev et al. 2024). The gap repeatedly exposed in the corpus is that even technically strong systems often lack an evidence package that can be reconstructed and defended after deployment changes or incidents. This gap has a structural cause: regulatory evidence requirements are typically defined at system release, but agentic systems continue to evolve as model updates, tool schema changes, retrieval corpus refreshes, and policy revisions accumulate after the initial evidence package is assembled. Maintaining regulatory legibility therefore requires not only that the release-time evidence is complete and traceable, but also that a change-impact assessment process exists to determine when a post-release change is material enough to require evidence re-submission or re-evaluation. The FDA’s good machine learning practice guidance and the MDCG position papers on adaptive AI both identify this as an open operational challenge, and the corpus confirms that most technical validation frameworks address it incompletely if at all. The implication for this dimension is that audit-log coverage and provenance completeness must be treated as runtime properties maintained continuously, not as documentation artifacts assembled at release and left static thereafter.

6.5 Multi-Agent Dimension The multi-agent dimension evaluates the joint trajectory produced by interacting agents, services, and humans. It is structurally distinct from single-agent validation because acceptable local policies can still compose into conflict, deadlock, duplication, delay, or distributed risk. Benchmark and failure-mode evidence documents these compositional breakdowns directly (Zhu et al. 2025); work on cooperation and coordination mechanisms shows how they arise from interaction structure rather than from

23

agent-level defects. Collaborative and mean-field analyses relate failures to interaction structure (Bruce Fang and Danyi Gao 2025; Lin Huo et al. 2026); cooperative learning and strategic-interaction work identifies further coordination mechanisms (Lulu Li et al. 2025; Sungwook Yoon 2025), including explicit coordination models (Chenhang Xu et al. 2024). Studies of LLM-based frameworks and role design confirm the pattern in contemporary stacks (Y. Yoldas et al. 2024; Vasanth Rajendran et al. 2025), at increasing organizational scale (Abdullah Mushtaq et al. 2025; Chitiz Tayal et al. 2026), and in scientific multi-agent settings (Kim et al. 2026). The joint trajectory, not any component in isolation, is therefore the proper validation object. This makes the dimension asymmetric with the other four in a basic sense: behavioral, safety, temporal, and regulatory concerns all apply to single-agent systems as well, whereas multi-agent validation exists only once assurance claims depend on relations among multiple decision makers. The distinctive burden here is therefore not merely to add more agents to an existing validation frame, but to validate interaction-level properties, including coordination, delegation, conflict resolution, and joint accountability, that do not arise in isolated-agent settings. The structural challenge is that emergent coordination failures cannot generally be predicted from single-agent test results: an agent whose individual behavior is within specification may still produce unsafe outcomes when coupled with other agents whose policies interact with its own in unanticipated ways. Shared resource contention, implicit assumptions about message ordering, and uncoordinated escalation are recurring examples in the corpus. Validating this dimension therefore requires test scenarios that exercise inter-agent interactions explicitly, including adversarial compositions in which one agent’s well-intentioned action is precisely the trigger that causes another’s policy to fail. Blame resolution and causal attribution in such settings are particularly difficult because the observable failure may occur far removed from its initiating cause, both temporally and across a chain of interacting agents.

6.6 Dimension Intersections and Relation to Prior Surveys The most consequential failures often lie at dimension intersections: behavioral × safety yields competent-but-dangerous trajectories; temporal × regulatory yields stale but still-claimed evidence; and multi-agent interaction amplifies both behavioral and safety hazards through coordination breakdowns (Sambasivan et al. 2021). For behavioral × safety, a concrete failure mode is an agent that competently completes its assigned task while choosing an unsafe tool sequence or escalation path: the behavior is goal-effective, yet the trajectory violates the safety boundary precisely because competence and constraint satisfaction come apart. For temporal × regulatory, a concrete failure mode is a system whose release-time evidence package remains formally on file even after tool semantics, retrieved knowledge, or escalation workflows have changed, so the deployment still appears compliant while the evidential basis for that claim has already expired. Relative to existing surveys, our taxonomy differs by organizing the field around these validation objects rather than around capabilities or application domains. Luo et al. (2025) and Yehudai et al. (2025) foreground evaluation gaps but do not make temporal validity or regulatory legibility first-class dimensions; Zhao et al.

24

(2026) captures safety and governance pressures in healthcare but does not generalize them as a cross-domain taxonomy.

7 Survey of Existing Approaches This section maps concrete validation approaches onto the five taxonomy dimensions and identifies robust coverage gaps in the coded corpus (RQ3). Unlike Section 3, which positions adjacent literatures, the emphasis here is evidentiary: which approaches cover which validation objects, and where durable asymmetries remain.

7.1 Behavioral Validation Behavioral validation is the most developed dimension in the current literature. Benchmarks such as SWE-bench and ITBench, LLM-as-judge methods, and newer multi-agent suites have made capability measurement substantially richer, but the dominant target remains task success or broad process scoring rather than whether trajectories remain acceptable, consistent, and stable across semantically equivalent cases. SWE-bench and ITBench establish software-engineering task performance (Badertdinov et al. 2025; Jha et al. 2025), while LLM-as-judge and process-scoring methods broaden the measurement toolkit (Zheng et al. 2023; S. Pandruju 2025). Assessment frameworks offer further general evaluation designs (Akshathala et al. 2025; Rakesh More 2025). Multi-agent and interactive suites extend coverage to coordination and environments (Zhu et al. 2025; Cemri et al. 2025), including AgentVerse and Taubench (W. Chen et al. 2024; S. Yao et al. 2024). Benchmark-aging work qualifies the durability of those scores (Liang et al. 2022). Across domains, the pattern is the same: behavioral evaluation is broad in coverage but shallow in what it guarantees, because outcome scores scale more readily than trajectory auditing. This limitation is visible even in the strongest contemporary benchmark families. AgentBoard, TheAgentCompany, and AgentHarm extend behavioral evaluation toward multi-turn interaction, consequential workflows, and harmful-action measurement, yet they still stop short of specifying lifecycle criteria for whether trajectories remain acceptable under change, handoff, or tool failure. AgentBoard and TheAgentCompany expose the interaction and workflow dimensions (C. Ma et al. 2024; F. F. Xu et al. 2025), while AgentHarm measures harmful-action exposure (Andriushchenko et al. 2024). The same pattern recurs across domains: in clinical orchestration (Arjun Warrier and Abhilash K S 2025), software engineering benchmarks and analyses (Zhi Chen and Lingxiao Jiang 2025; M. Mahdy and C. Rubio-Medrano 2025), and planexecute-generate-judge pipelines (Alex Kaplunovich 2025), cybersecurity (Anshuman Chhabra et al. 2026; Masike Malatji 2025), and finance and logistics (Giuliano Lorenzoni et al. 2025; Kartik Rajesh et al. 2025), outcome scores do not expose intermediate-step inconsistencies, tool-use brittleness, or non-deterministic plan selection. Broader evaluations of agent-based and generative deployments report the same asymmetry (Pat Rondon et al. 2025; Dholakia et al. 2025), including autonomousagent deployments (Rupesh Kohli et al. 2025), and failure-analysis studies locate the breakdowns specifically in reasoning transitions and tool-invocation sequences rather than in final answers (Ruofan Lu et al. 2025; Yash Tripathi et al. 2025). Dynamic 25

suites such as RefuteBench 2.0 and AutoONBench probe consistency under perturbation (Jianhao Yan et al. 2025; Yihao Zhang et al. 2025). Agent-drift studies track degradation over repeated interaction (Rath 2026; Jilk 2016), including in self-evolving settings (Wang et al. 2025); trajectory-level auditing is nevertheless not a standard behavioral-validation requirement.

7.2 Safety Validation Safety validation is active but fragmented. As Section 6 noted, the Safety column aggregates functional-safety and adversarial-security work; in the approaches layer, most recent density comes from prompt injection, misuse containment, and runtime enforcement rather than explicit safe-stop or graceful-degradation design. RTBAS and G-Safeguard exemplify runtime enforcement (Zhong et al. 2025; Wang et al. 2025), whereas AgentGuard and foundational safety work motivate bounded autonomy (Koohestani 2025; Amodei et al. 2016). Partial progress comes from tool-use containment through access control, semantic gating, defense-in-depth architectures, and uncertainty-driven monitoring. MCP security and semantic gating constrain tool use (Gamini Singh and Vijay K. Madisetti 2026; Lucas G. M. Castro et al. 2026); defense-in-depth and uncertainty monitoring add complementary controls (Aruun K. Kumar et al. 2026; Vladimir Balditsyn et al. 2025). From verifiable safeguards and safety-requirement derivation. Explainable and unified-safeguard approaches make local controls inspectable (Satyanarayana Burugupalli 2025; Khan et al. 2025); security and assurance frameworks translate them into broader claims (Collins P Obeng et al. 2025; Himanshu Agarwal et al. 2021). Safety-requirement derivation and calibration work address the specification side (Badr et al. 2025; Kadavath et al. 2022), with debate-based approaches providing a related mechanism (Irving et al. 2018). Multimodal and multi-agent defenses broaden the threat coverage (Syed et al. 2025; Wang et al. 2026); runtime resonance and moderation systems add operational controls (Charles Hartsell et al. 2021; Juan Ren et al. 2025), alongside cybersecurity-focused defenses (V. Vinay 2026). The remaining gap is not intervention in general, but a unified specification for safestop, graceful degradation, and reviewable evidence generation across diverse agent architectures and toolchains. Even systems such as RTBAS, G-Safeguard, and AgentGuard treat agents as intervenable information-flow systems: RTBAS and G-Safeguard provide runtime controls (Zhong et al. 2025; Wang et al. 2025), and AgentGuard extends them to agentic execution (Koohestani 2025). They still leave open how deployments should hand off, roll back, or degrade when confidence collapses, or safeguards disagree.

7.3 Temporal and Lifecycle Validation Temporal validation remains the least consolidated dimension: temporal concerns attract substantial paper volume, but no mature, agent-specific validation family yet anchors them. Benchmark-contamination work shows that static suites age, while MLOps, CPS, and regulatory literatures show that deployed systems require evidence maintenance, change control, and post-deployment monitoring rather than one-time

26

benchmark results. Benchmark contamination and MLOps work show why one-time scores become stale (Chen et al. 2025; Kreuzberger et al. 2023); expert monitoring and dataset-shift research identify the signals that should prompt renewal (Leest et al. 2024; Quionero-Candela et al. 2009). Hidden-debt and production-ML studies explain the operational sources of this decay (Sculley et al. 2015; Breck et al. 2017), while FDA and MDCG guidance makes the resulting change-control obligation explicit (U.S. Food and Drug Administration 2025b; Medical Device Coordination Group 2025a). What is still missing is operationalization for agentic systems: what to monitor, when to revalidate, and how to refresh evidence when memory, tools, or workflows change. Recent work makes this gap concrete without closing it. Agent-drift and longhorizon studies show degradation over repeated interaction or after many locally acceptable steps (Rath 2026; Du et al. 2025), including explicitly temporal agent settings (Mitra 2025). MLOps monitoring detects performance change, feature drift, and retraining triggers, but usually assumes stable prediction targets rather than open-ended policies that evolve through tool use, interaction, and memory adaptation. Observability and monitoring-maintenance frameworks provide the core instruments (Preetam Dedu et al. 2025; Lorena Poenaru-Olaru et al. 2023); anomaly and reliability studies address their operational use (Poenaru-Olaru et al. 2024; Lorena Poenaru-Olaru et al. 2025), alongside model-updating methods (Poray et al. 2025). Agent-specific harnesses and protocols increasingly treat lifecycle change as expected (Jiahang Lin et al. 2026; Wentao Zhang et al. 2026), and CPS work connects temporal validity to control loops, delays, and operational constraints (Collaco et al. 2026; Zhao et al. 2026), as also shown by digital-twin work (Veledar et al. 2019). The core temporal gap is therefore the missing bridge between observability and obligation: when a tool-schema revision invalidates earlier evidence, when memory contamination requires a fresh evaluation window, or when monitored drift becomes assurance failure.

7.4 Regulatory and Assurance Approaches The regulatory dimension is addressed primarily by governance and assurance-case frameworks rather than technical methods. FDA guidance makes lifecycle evidence and change control explicit (U.S. Food and Drug Administration 2025a,b), while MDCG guidance supplies the European counterpart (Medical Device Coordination Group 2025b,a). IEEE guidance and assurance-case methods provide structured argumentation (IEEE Standards Association 2025; Alelyani 2025), but the engineering implementation remains underspecified for agentic systems. Put concretely, the surveyed governance documents are comparatively clear about the evidentiary requirements, such as traceability, change control, post-market evidence, and reviewable justification. However, they are far less specific about how an agentic system should operationally package trajectory evidence, monitoring outputs, override records, and revalidation triggers into a reviewable evidential payload. Mature argument patterns already exist across regulated domains, autonomous inspection, unmanned aircraft, spacecraft, automotive systems, and safety-critical machine learning. Reference patterns and interlocking-safety work establish reusable assurance structures (Dhaminda B. Abeywickrama et al. 2025; Michael Vierhauser et al. 2019); software and formal assurance-case studies carry them into regulated engineering (Ben Smith et al. 2020; 27

Ioannis Nearchou et al. 2023). Automotive and security applications demonstrate domain transfer (Luis-Pedro Cobos et al. 2022; Zhengshu Zhou et al. 2021), including for safety-critical machine learning (Saswata Paul et al. 2024). A second body of work makes the assurance stack more operational through automated case management and continuous assurance (Ran Wei et al. 2023; Fredrik Warg et al. 2019), model-risk governance and policy pipelines (Izunna Okpala et al. 2025; Talal Ashraf Butt et al. 2025), and automated interpretation (Rohlinger Tihomir 2024). Governance-oriented analyses extend the picture to liability and decentralized accountability (Asaro 2016; Chaffer et al. 2025), and agentic-specific assurance proposals begin to translate these obligations toward agent deployments. Reliability-by-design and safety-landscape studies make this move explicitly (Ray 2026; Schnitzer et al. 2024); agentic assurance cases and holistic frameworks extend it (Benjamin D. Werner et al. 2023; Uma D. Ferrell and Alfred H. Andy Anderegg 2022), alongside broader assurance-case work (Ferrell and Anderegg 2022). All of these still stop short of specifying the trajectory-evidence bundle for open-ended agentic systems.

7.5 Multi-Agent Validation Multi-agent validation is emerging but still immature. Benchmarks, failure-mode studies, and runtime monitoring now provide useful data on interaction quality, yet they do not yield compositional validation certificates for open-ended LLM coordination. MultiAgentBench and MAST measure coordination outcomes and failure modes (Zhu et al. 2025; Cemri et al. 2025), while G-Safeguard supplies a runtime perspective (Wang et al. 2025). Formal verification and distributed-systems work provides candidate compositional methods (Geeta Mahala et al. 2024; Sagir M. Yusuf and Chris Baber 2022), including engineering and multilevel protocol approaches (Zhenyu Mao et al. 2025; Wenbin Zhang et al. 2025). Classical coordination frameworks offer additional precedents (G. Li et al. 2023; Dahlquist et al. 2023), with recent design work extending them (Zahra Aminiranjbar et al. 2025). Recent evidence from asynchronous software-engineering agents makes the same point in a practically important setting: coordination quality depends on delegation structure, synchronization discipline, and handoff management, not only on single-agent competence (Jiayi Geng and Graham Neubig 2026). Classical multi-agent systems (MAS) verification remains methodologically relevant but only partially transferable because LLM-based agents operate over open-ended natural language and learned policies rather than bounded programs and finite vocabularies. The central gap is therefore structural: the challenge is not merely to test more agents, but to certify interaction-level properties that emerge only at the system level. Supporting evidence converges from three further directions. Studies of collaborative planning, auction-based allocation, and decentralized coordination show that individually competent agents still produce oscillation, deadlock, or degraded joint behavior. LLM and reactive-coordination studies document these effects (João Vitor de Carvalho Silva and Douglas G. Macharet 2025; Niklas Dahlquist et al. 2023), as do behavior-tree approaches (Niklas Dahlquist et al. 2024). Trust modeling and rolespecialized traceability improve partner selection and post-hoc auditability without yielding compositional guarantees. Task-specific and contextual trust models improve 28

Table 6 Coverage gap matrix across approach families and evaluation dimensions. Higher-intensity cells indicate denser coverage, while low-intensity cells highlight approach–dimension gaps.

partner selection (Botao Zhu et al. 2025; Kangning Gao et al. 2025); traceability and comparative studies improve auditability (Amine Barrak 2025; Akshata Kishore Moharir et al. 2025), including in scientific scaling settings (Kim et al. 2026). And digital-twin simulation together with cascade-aware analysis supplies the substrate on which interaction-level hazards can be exercised before deployment (Veledar et al. 2019; Sambasivan et al. 2021). The common conclusion is that collective failures and interaction-level hazards cannot be inferred from single-agent scores alone. Table 7 clarifies that the paper-level counts assign one primary dimension to each of the 257 papers, whereas Table 6 counts multi-label approach-family appearances. The latter therefore measures coverage intensity, not unique-paper frequency. On this reading, behavioral validation is strongest on both layers: 71 primary papers (27.6%) and 120 appearances, including 68 in benchmark-centric evaluation. Safety accounts for 42 primary papers (16.3%) but 85 appearances, with runtime monitoring as the clearest concentration and with the functional-safety/security split noted above. Temporal validation diverges most sharply: it is the second-largest primary category (62 papers, 24.1%) but produces only 40 coverage appearances, with no temporal cell above medium intensity. Regulatory evidence shows the opposite profile: 32 primary papers (12.5%) but 70 appearances in governance and assurance-case work, and multiagent validation remains under-covered on both layers (50 primary papers, 19.5%; 45 appearances). Overall, the matrix supports the same directional claim as the sensitivity

29

Table 7 How the two counting layers used in the paper relate to each other: paper-level primary-dimension counts (single label, total 257) and gap-matrix appearance counts (multi-label over approach families and dimension tags, total 360).

Counting layer

Behavioral / Safety / Temporal / Regulatory / Multi-agent

Total What is being counted

Primary dimension (paper-level; single label)

71 / 42 / 62 / 32 / 50

257

Gap-matrix coverage 120 / 85 / 40 / 70 / (approach-family × 45 dimension-tag appearances; multilabel on both axes)

360

Each included paper carries exactly one primary dimension under the frozen codebook; these counts ground all paper-level claims, the year-band composition in Figure 2, and the sensitivity bounds in Table 2. Reported in Section 2.6. Appearance counts used for the coverage matrix in Table 6: a paper may contribute to several approach-family cells and, through secondary dimension tags, to more than one dimension column. The six approach families are not exhaustive, so papers whose approach fits none of them contribute no appearances. The matrix therefore measures coverage intensity, not unique-paper frequency; the sensitivity analysis in Table 2 applies to the paperlevel primary-dimension counts, not to this multi-label appearance layer.

analysis: behavioral instruments are relatively strong, while lifecycle evidence decay, compositional multi-agent assurance, and audit-ready governance remain weaker.

8 Illustrative Case Studies Across Operational Domains The five-dimension taxonomy is domain-general, but its force is clearest when applied to deployment settings where agentic systems already act consequentially. We present three parallel case studies, namely medical care delivery, industrial operations, and smart-mobility CPS, as matched illustrations of the same validation pressures. Each surfaces heterogeneous tool use, temporally evolving evidence, safety-critical intervention, human override, and operational accountability. CPS and clinical case studies establish the basic pattern (Collaco et al. 2026; Zhao et al. 2026); industrial and healthcare work extends it to deployed workflows (Sharma et al. 2026; Draelos et al. 2026). ICCPS studies provide complementary evidence on closed-loop systems (Lin et al. 2025; Brown et al. 2025). The recurrence of the five validation questions across these domains indicates a deployment-level structure. Each case study is grounded in the reviewed literature and pairs the dimension mapping with a constructed trajectory trace: a step-by-step walkthrough using the

30

per-step (st , at , mt ) notation from the trajectory τ = ((s0 , a0 , m0 ), . . . , (sT , aT , mT )) defined in Section 4, consistent with the failure patterns documented in the cited deployment and failure studies. The constructed traces show where each latent defect would be intercepted and by which monitor. As illustrative operational constructions, they make the taxonomy’s dimensional structure concrete at step granularity; they are not incident reports or independent empirical evidence. The reviewed literature shows that components can pass their local tests while the trajectory as a whole fails.

8.1 Metric Definitions for the Case Studies Three metrics appear throughout the illustrative case studies and policy vignettes. All are operationally defined by the papers cited in the taxonomy section; they are reproduced here to clarify the numerical statements in the traces that follow. Table 8 Operational definitions of three illustrative metrics used in case-study traces. These definitions are derived from the taxonomy discussion in Section 6; the thresholds shown in the case studies (Section 8) are example policy values, not empirical benchmark results. Metric

Definition

Evaluation window and interpretation

Evidence Time elapsed since the last refresh of a data source (e.g., Freshness laboratory panel, sensor reading, or retrieved evidence) Interval (EFI) used in a trajectory decision. Measured in minutes or hours from the source’s timestamp to the decision point.

Evaluation window: from source creation to decision time. Interpretation: if EFI exceeds a policy-specified freshness limit (e.g., 60 minutes for medicationrelevant labs), the decision is flagged or blocked until the source is refreshed. Example threshold: EFI = 105 min > 60 min policy limit → block.

Unsafe-Action Proportion of executed actions (typically tool invocations Rate (UAR) or control commands) within a session or trajectory that violate safe-operating constraints or authorization policy. Numerator: count of unsafe actions. Denominator: total actions executed in the window.

Evaluation window: rolling window over the current session or trajectory (typically 10–100 steps). Interpretation: if UAR exceeds a threshold (e.g., 0.02 for safety-critical systems), containment or rollback is triggered. Example threshold: UAR = 0.08 > 0.02 plant threshold → block.

Coordination Success Rate (CSR)

Evaluation window: rolling count of handoffs in the current session or deployment period. Interpretation: if CSR falls below a threshold (e.g., 0.95 for safety-critical coordination), escalation is triggered to reduce the coordination load or involve human oversight. Example threshold: CSR = 0.91 < 0.95 (deployment threshold) → flag.

Proportion of inter-agent or human–agent handoff events that complete without conflict, deadlock, or timeout. Numerator: count of conflict-free transitions. Denominator: total handoff events attempted in the evaluation window.

31

Note on illustrative thresholds. The numerical examples in the metric definitions above (60 minutes, 0.02, 0.95) are representative policy-threshold values used only in the illustrative case-study traces. They are not empirical results from agentic-system benchmarks and should not be interpreted as recommendations for real deployments. Each operational context requires calibration of these thresholds based on risk tolerance, domain constraints, and deployment-specific data.

8.2 Medical Care Delivery A representative deployment target in the surveyed corpus is postoperative nursing support with LLM-agent systems, where agents assist observation, follow-up, and coordination around postoperative care workflows (Xue Qiao et al. 2025). The most directly documented failure evidence comes from Draelos et al., who show that large language models can provide unsafe answers to patient-posed medical questions, including advice that is clinically unsafe or insufficiently escalatory for the presenting condition (Draelos et al. 2026). While Draelos et al. examine LLM responses rather than full agentic trajectories, their findings document unsafe and insufficiently escalatory response patterns. In the constructed trace below, we treat those response-level failures as local breakdowns that an agentic clinical deployment could amplify through retrieval, escalation, and follow-up. The stale-evidence step is an illustrative trajectorylevel risk motivated by clinical workflow coupling rather than a direct incident reported by Draelos et al. In the five-dimension structure, this failure is behavioral because similar symptom descriptions can produce different advice trajectories; safety because the recommendation itself can be unsafe; temporal because the correctness of the advice depends on whether medication lists, recent vitals, or symptom progression are still current; regulatory because any recommendation that affects care requires reviewable justification; and multi-agent because the effective outcome depends on coordination across the agent, electronic health record (EHR) tools, nursing staff, physicians, and pharmacy. Zhao et al. likewise characterize healthcare agents as operating in tightly coupled socio-technical loops rather than as isolated predictors. Clinical and expertmonitoring studies motivate these indicators (Zhao et al. 2026; Leest et al. 2024); taxonomic and knowledge-action work specifies related measures (S. Vatsal et al. 2026; Qingrui Li et al. 2025), including convenient human-facing designs (Nejad Alagha et al. 2025). The validation lesson is that clinical usefulness cannot be separated from freshness, escalation timing, and documented responsibility for action. Table 9 makes the mapping operational through a constructed postoperativesupport trajectory. Every component in this trace can pass its local tests; the retrieval API returns well-formed records; the classifier meets its accuracy target on its test distribution; the generated message is fluent, while the trajectory as a whole is unsafe. Each latent defect is intercepted, if at all, by a different dimension, and the first interception (the freshness breach at t1 ) precedes the clinically dangerous step at t3 : trajectory-level validation matters precisely because it can stop a defective episode before its consequential action.

32

Table 9 Constructed trajectory trace for the medical case study, consistent with the response-level failure patterns documented in (Draelos et al. 2026) and the socio-technical coupling described in (Zhao et al. 2026). Each row is one step (st , at , mt ); the final column names the taxonomy dimension and monitor signal that intercepts the defect. Thresholds are illustrative policy values, not reported benchmark results. t

Trajectory event (st , at , mt )

Latent defect

Dimension → signal and outcome

0

Patient message reports a new post-operative symptom; the agent opens a session and initializes state s0 .

—

—

1

Agent retrieves chart and laboratory data via the EHR tool; m1 records that the renal panel timestamp is 105 minutes old.

Freshness policy for medication-relevant decisions is 60 minutes.

Temporal → EFI = 105 > 60 min: block pending refresh.

2

Planner classifies the report as routine discomfort; a replay run with a semantically equivalent phrasing yields a different classification.

Trajectory inconsistency under paraphrase.

Behavioral → consistency check on replayed trajectories: flag.

3

Agent drafts medication-adjustment advice and does not escalate to clinical staff.

Advice insufficiently escalatory for the presenting differential, the pattern documented in (Draelos et al. 2026).

Safety → escalation-appropriateness rule: escalate.

4

Memory update logs the episode as resolved without linking the advice to the chart snapshot used.

Decision rationale not reconstructable.

Regulatory → tracecompleteness audit: flag.

5

Follow-up task enters the nursing queue with low priority and misses the shift handover.

Mistimed coordination between agent and care workflow.

Multi-agent → handofflatency monitor: escalate.

8.3 Industrial Operations A representative industrial target is multi-agent support for commissioning and troubleshooting industrial drives, where specialized agents coordinate diagnosis steps, parameter checks, and operator-facing recommendations during equipment setup and fault analysis (Virendra Ashiwal et al. 2025). The five-dimensional structure recurs end to end: behavioral, because similar fault signatures can trigger different diagnostic decompositions or recovery plans; safety, because an incorrect recommendation can propagate into unsafe actuator settings or ill-timed restart advice; temporal, because telemetry, maintenance history, and machine state can drift while the troubleshooting loop is still running; regulatory, because industrial interventions require a reconstructable rationale for why a suggested action was issued; and multi-agent, because the outcome depends on coordination across specialized software agents, plant-control tools, operators, and the physical asset itself. Recent industrial AI-drift and uncertainty work reinforces this mapping: changing operating regimes and limited supervision undermine the validity of learned decision support if monitoring and recalibration are not built into the loop. Connection and boundary-aware approaches motivate this integration (Jesus L. Lobo et al. 2023; Haowen Tan 2025), and CPS

33

deployment studies demonstrate its need (Collaco et al. 2026; Sharma et al. 2026). Here the important shift is from evaluating whether a recommendation looks plausible to evaluating whether it remains safe and accountable while plant state, operators, and controllers continue to change. Table 10 traces a constructed commissioning episode. The trace makes the drift literature concrete: the defect introduced at t1 is temporal, but its consequential expression at t3 is a safety violation, and the only dimension that explains the duplicated restart at t4 is multi-agent coordination. A validation regime checking any single dimension would have caught one defect and missed the episode. Table 10 Constructed trajectory trace for the industrial case study, consistent with the multi-agent drive-commissioning setting of (Virendra Ashiwal et al. 2025) and the industrial drift and uncertainty findings of (Jesus L. Lobo et al. 2023; Haowen Tan 2025). Thresholds are illustrative policy values. t

Trajectory event (st , at , mt )

Latent defect

Dimension → signal and outcome

0

An intermittent overcurrent fault is reported during drive commissioning; the diagnostic session opens.

—

—

1

Telemetry tool returns a cached parameter snapshot predating a recent firmware update; m1 records the snapshot version.

Tool output reflects a superseded machine configuration.

Temporal → evidence-version check: flag.

2

Planner decomposes the diagnosis; a replay of the same fault signature yields a different tool-invocation order.

Non-deterministic decomposition of an identical fault state.

Behavioral → tool-sequence consistency under replay: flag.

3

Agent recommends raising the current limit and issuing a restart; the session’s unsafe-action rate over actuator-setting recommendations reaches 0.08.

Recommendation violates the post-update safe operating envelope.

Safety → UAR = 0.08 > 0.02 plant threshold: block.

4

The thermal-monitoring peer agent is not consulted; a duplicated restart instruction is issued through a second channel.

Conflicting interagent instructions to the same asset.

Multi-agent → duplicatedaction detector: flag.

5

Operator accepts the remaining suggestion; the log does not link the recommendation to the telemetry version that produced it.

Audit cannot reconstruct why the action was issued.

Regulatory → provenancecompleteness audit: flag.

8.4 Smart-Mobility CPS A representative mobility-side target is highway safety monitoring with multimodal agentic systems, where an agent consumes heterogeneous road, sensor, and situational signals to decide whether to flag hazards, request intervention, or prioritize follow-up analysis (Abdulla Almarzooqi et al. 2025). The validation issues again map cleanly onto five dimensions: behavioral, because small changes in perception inputs can alter downstream interpretations and recommended responses; safety, because missed or spurious hazard classification directly affects roadway risk; temporal, because the validity of a decision depends on whether traffic state, weather, and map context are still current; regulatory, because public-safety actions need inspectable justification and replayable evidence; and multi-agent, because the effective behavior is distributed

34

across perception modules, runtime monitors, infrastructure services, and human supervisors. CPS work on out-of-distribution safety monitoring and perception-based quantitative runtime verification treats smart-mobility assurance as a runtime problem rather than a pre-deployment benchmark problem. ICCPS studies describe the closed-loop requirement (Lin et al. 2025; Brown et al. 2025), and digital-twin work provides a complementary account (Veledar et al. 2019). This makes mobility the clearest reminder that locally sensible steps can still compose into globally unsafe behavior when timing, coordination, and intervention logic are wrong. Table 11 traces a constructed fog-onset episode. It differs from the previous two traces in that the earliest signal (t1 ) is already an escalation trigger: runtime outof-distribution monitoring is the most mature instrument in this domain (Lin et al. 2025; Brown et al. 2025), yet the trace still accumulates four further defects that OOD monitoring alone does not see. Maturity in one dimension does not substitute for the others. Table 11 Constructed trajectory trace for the smart-mobility case study, consistent with the multimodal highway-monitoring setting of (Abdulla Almarzooqi et al. 2025) and the runtime-assurance instruments of (Lin et al. 2025; Brown et al. 2025). Thresholds are illustrative policy values. t

Trajectory event (st , at , mt )

Latent defect

Dimension → signal and outcome

0

Highway monitoring agent ingests camera and roadside-sensor feeds as fog develops over a monitored segment.

—

—

1

Perception confidence degrades; the out-of-distribution score rises and persists; m1 logs the monitor output.

Inputs are leaving the validated operating domain.

Safety → OOD-persistence monitor: escalate.

2

Agent classifies the obstruction as low-priority debris; a near-identical scene in replay is classified as a hazard.

Timing-dependent divergent interpretation of similar scenes.

Behavioral → consistency under scene perturbation: flag.

3

The deferral decision relies on a map layer older than its freshness bound.

Stale context informs a safety-relevant deferral.

Temporal → map-freshness check: flag.

4

Handoff to the infrastructure service misses its latency budget; the session’s conflict-free handoff rate falls to 0.91.

Coordination below the deployment threshold.

Multi-agent → CSR = 0.91 < 0.95: flag.

5

Escalation eventually occurs, but the inputs that drove the earlier deferral are not replayable for review.

Intervention record not legible to oversight.

Regulatory → audit-replay check: flag.

8.5 Cross-Domain Summary Across all three domains, the common pattern is that the agent is not an isolated predictor but a participant in a closed-loop socio-technical system. Failures emerge from stale evidence, unsafe intermediate actions, mistimed escalation, and coordination breakdowns, precisely why behavioral, safety, temporal, regulatory, and multi-agent

35

Table 12 Cross-domain summary of the five validation dimensions. Dimension

Medical

Industrial

Smart Mobility

Behavioral

Divergent tool-use and escalation trajectories for similar patient states

Similar process states can trigger brittle planning or actuator sequencing

Similar traffic scenes can produce unsafe trajectory choices under different timing assumptions

Safety

Unsafe recommendations, delayed holds, or missed escalation affect care

Wrong sequencing or delayed intervention can damage equipment or violate limits

Navigation, routing, or handoff errors create system-level risk

Temporal

Labs, device feeds, and orders age asynchronously

Telemetry, controller state, and maintenance context drift during execution

Sensor latency, map freshness, and traffic evolution alter safe decisions over time

Regulatory

Clinical interventions require traceable justification and reviewability

Audits require change traceability, incident reconstruction, and operator accountability

Public-safety actions require legible intervention records

Multi-agent

Outcomes depend on coordination among agent, EHR tools, clinicians, and pharmacy

Agent, control tools, operators, and plant infrastructure form one coupled loop

Safety emerges from coordination among agents, platforms, infrastructure, and supervisors

Illustrative policy vignettes. Medical: EFI = 105 min against a 60-min limit → block. Industrial: UAR = 0.08 against a threshold of 0.02 → block. Smart mobility: CSR = 0.91 against a threshold of 0.95 → flag. These values are illustrative policy examples rather than reported benchmark results; they correspond to the threshold breaches at t1 , t3 , and t4 of the worked traces in Tables 9–11.

validation must be addressed together rather than as separate checklists. The trajectory, not the final output, is the unit at which the relevant failure modes become visible. The worked traces (Tables 9–11) make this concrete in a second way: the medical and industrial traces are both first intercepted by a temporal signal, yet only in the industrial trace does that stale-evidence defect go on to express itself as a downstream safety violation, while the mobility trace is instead intercepted first by runtime safety monitoring; no single instrument dominates across domains. The policy vignettes in Table 12 show how the same thresholds convert dimension signals into actionable outcomes such as block, flag, or escalate. What the case studies add beyond the taxonomy alone is cross-domain mapping of the taxonomy onto three unlike deployment domains. The reviewed literature provides empirical support for the claim that the same five validation questions recur independently in medical care (Draelos et al., Zhao et al.), industrial operations (Collaco et al., Sharma et al.), and smart mobility (Lin et al., Brown et al.) despite their different hazards, tools, oversight regimes, and time scales. The constructed traces operationalize that mapping by showing where latent defects would be intercepted and by which dimension’s monitor. The recurrence shows that the taxonomy captures deployment-level validation concerns across consequential domains. The illustrative traces make this literature-grounded mapping concrete.

36

9 Research Agenda: Building the Validation Stack The survey supports four cumulative directions for a validation stack: specification, stress exposure, runtime containment, and auditability. 1) Bounded-autonomy specifications. Which specification formalisms can express partial trajectory contracts for open-ended, tool-using agents without collapsing them into brittle scripts? This direction directly addresses the Behavioral × Assurance Cases cell in Table 6, which contains only 4 coded appearances and therefore remains red despite the apparent maturity of behavioral evaluation overall. The gap is not measuring whether an agent can solve a task once; it is stating, in machine-checkable form, what the agent is allowed, forbidden, or obligated to do while pursuing that task. The core research problem is to define partial contracts over trajectories: obligations to escalate, regions of forbidden action, constraints on delegation, and memory-update rules that survive policy variation and environment change. In agentic systems, engineers rarely know the exact action sequence in advance; what they know instead is the structure of acceptable behavior under uncertainty. That makes the problem a requirements-language challenge at the boundary of temporal logic, assume–guarantee reasoning, runtime shielding, and socio-technical systems engineering rather than a mere benchmark-design exercise. Runtime shielding and safety-of-the-intended-functionality work supplies formal starting points (Alshiekh et al. 2018; ISO 2022); CPS validation studies ground the problem in deployed autonomy (Zhao et al. 2026; Collaco et al. 2026), while constitutional approaches offer a related agentic framing (Torgbi Agbemabiese 2026). The expected artifact is an interchange format for bounded autonomy: a specification object that can be compiled into test oracles, runtime monitors, and assurance claims, evaluated through obligation coverage, violation-detection recall, and representational adequacy. 2) Adversarial trajectory generation. Which techniques can generate test trajectories that cover the fat-tail failure modes unique to agentic systems? This direction directly addresses the Temporal × Benchmarks cell in Table 6, which contains only 4 coded appearances and leaves long-horizon temporal failure discovery largely outside the benchmark mainstream. The immediate gap is not the absence of environments per se, but the absence of generators for rare, off-policy, state-dependent trajectories in which memory errors, tool misuse, delayed side effects, and adversarial perturbations combine over time. Existing environments provide useful starting points, including WebArena, WorkArena, and AgentDojo (S. Zhou et al. 2024; Debenedetti et al. 2024), together with the adversarial Nemesis benchmark (Vishal Kharde 2026), but they do not yet provide systematic coverage of temporally extended failure surfaces. What is needed is search procedures that preferentially discover tail failures through environment mutation, counterfactual replay, disturbance injection, and policy-guided stress generation (Collaco et al. 2026; Draelos et al. 2026). The expected artifact is a fat-tail generator : a replayable test generator that emits adversarial trajectory families and minimal counterexamples for debugging and audit. Progress is visible through rare-event coverage, adversarial failure discovery, and replay reproducibility. 37

3) Temporal runtime monitoring. Which runtime monitors can detect trajectory-level drift early enough to trigger containment before localized deviations compound into unsafe outcomes? This direction directly addresses the Temporal × CPS Validation cell in Table 6, which contains only 3 coded appearances, the sparsest cell among the engineering-oriented approach families. That scarcity matters because many agentic deployments are not one-shot services but closed-loop systems in which delayed tool effects, memory corruption, schema changes, and workload shift accumulate across operational time. Three monitoring channels recur across the surveyed literature: memory and retrieved-context drift, tool-version and schema change, and environment or workload shift. The unsolved problem is to turn those signals into revalidation triggers, degradation modes, rollback rules, and escalation paths that activate before error propagation becomes opaque. Assurance monitoring and runtime verification provide the initial containment mechanisms (Hawkins et al. 2021; Zhong et al. 2025); MLOps and expert-monitoring work identifies observable signals (Kreuzberger et al. 2023; Leest et al. 2024). FDA and MDCG guidance establishes the associated change obligations (U.S. Food and Drug Administration 2025b; Medical Device Coordination Group 2025a), while agent harnesses and evolving protocols make the need concrete (Jiahang Lin et al. 2026; Wentao Zhang et al. 2026). A key implication of the Safety split in Section 7 is that runtime monitors must distinguish security-triggered containment from functional-safety degradation. The expected artifact is an event calculus for revalidation : a monitor specification that binds observable drift events to containment actions and evidence refresh. It specifies which changes in memory state, tool schema, workload, or policy behavior are revalidation-relevant, how those signals combine across time, and which assurance action each pattern triggers. Progress can be assessed through drift-slope sensitivity, unsafe-action rate (UAR), and response-rule correctness: block, rollback, degrade, or escalate before downstream hazard propagation becomes opaque. 4) Human oversight and evidence legibility. Which oversight and evidencechain designs allow human reviewers to verify, contest, and renew trajectory-level assurance claims as systems evolve? This direction directly addresses the Regulatory × Runtime Monitors cell in Table 6, which contains only 7 coded appearances and marks a specific weakness in the current literature: operational traces are monitored, and regulatory claims are documented, but the connection between the two is rarely engineered as a reviewable chain of evidence. Two distinct concerns have to be separated here because they draw on different literatures and produce different artifacts. The first is human oversight : humancomputer interaction (HCI) and human-factors work asks when to interrupt, how to present uncertainty, what escalation path is usable, and how to avoid over-trust, alert fatigue, and nominal human-in-the-loop theater. Trust and human-factors research establishes the basic risks (Lee and See 2004; Parasuraman and Riley 1997); clinical deployment studies show their relevance to agents (Zhao et al. 2026; Draelos et al. 2026), and cascade analysis exposes their systemic consequences (Sambasivan et al. 2021). The second is evidence legibility : assurance-case, formal-methods, and traceability work asks whether a reviewer can reconstruct what claim was made,

38

which evidence supported it, which monitor fired, which override occurred, and whether that evidence remained valid at decision time. FDA documents establish the lifecycle-evidence expectation (U.S. Food and Drug Administration 2021; U.S. Food and Drug Administration et al. 2021); IEC and IEEE guidance specify complementary traceability practices (International Electrotechnical Commission 2006; IEEE Standards Association 2025). MDCG documents and the EU MDR provide the regulatory setting (Medical Device Coordination Group 2025b,a), with assurance-case work addressing reviewable arguments (European Union 2017; Alelyani 2025). Treating them as one problem obscures the fact that a usable override interface is not yet an audit-ready assurance package, and conversely that a formally traceable log may still be unusable for real-time human review. The expected artifacts are therefore an oversight interface specification for escalation, override, acknowledgment, and handoff, plus proof objects and linking schemas that bind specifications, test traces, runtime logs, overrides, and update events into audit-ready claims. Oversight quality is evaluated through reviewer uptake, correct override or escalation use, and low interruption burden; evidence legibility is evaluated through claim support, audit replay success, and freshness checks such as EFI. Together, these directions shift validation from measuring whether a component can succeed to demonstrating that trajectories remain acceptable, governable, and evidentially current in context.

10 Open Challenges and Methodological Directions Even with the proposed validation stack, several structural problems are likely to persist beyond any single research direction. Scalability of assurance. Assurance cost grows superlinearly as deployments expand from single agents to interacting ecologies of tools, humans, and organizations. Compositional methods can validate local properties, but system-level properties of multi-agent systems with dynamic membership and open communication vocabularies resist tractable state-space analysis (Kim et al. 2026; Abdullah Mushtaq et al. 2025). Better local validation does not automatically compose into system-level assurance, and each additional coordination or security layer adds obligations that do not aggregate cleanly (Aaluri Seenu et al. 2025; Akinyemi Arabambi et al. 2025). A general solution likely requires new assume-guarantee frameworks adapted to open-ended LLM interaction rather than bounded agent programs. Performance versus assurance overhead. Richer simulation, denser logging, monitoring, and more frequent human review improve evidence depth, but each creates a persistent tension with latency, usability, compute efficiency, and workflow acceptance. Work on sustainable monitoring explicitly quantifies this trade-off between accuracy and energy efficiency (Rafiullah Omar et al. 2024), while cost analyses of agentic reasoning show that assurance instrumentation carries measurable operational overhead (Jiin Kim et al. 2026). Adaptive protocols that reduce monitoring granularity during low-risk operation and scale up when risk indicators rise are a promising partial mitigation (Kumar Yashu et al. 2025), but the fundamental tension between evidentiary depth and performance remains unresolved.

39

Human-AI teaming as a moving target. Human-in-the-loop validation is a pillar of the research agenda, but it does not fully solve the problem that human behavior changes in response to automation. Over-trust, under-reliance, intervention fatigue, complacency, and shifting norms of responsibility are adaptive properties of socio-technical systems, not static interface issues. Mission-oriented and user-facing agent studies illustrate this shift (Swarnamouli Majumdar et al. 2025; Sidong Feng et al. 2025), including personalized-agent settings (Madhukar Nimbalkar et al. 2025). Validating the agent alone therefore undershoots the problem; validating the humanagent team is difficult precisely because the target of validation evolves through use. Studies designed to capture this dynamic, such as tracking override behavior longitudinally, varying automation reliability systematically, and measuring responsibility diffusion after incidents, are largely absent from the current corpus. Evidence portability and regulatory harmonization. Even well-formed assurance artifacts may not transfer across domains, jurisdictions, or procurement settings. Regulatory concepts such as acceptable risk, traceability, and post-deployment accountability are not interpreted identically across governance regimes, and benchmarks and architectures keep changing faster than evaluation norms can stabilize. Verifiable-semantics and accountability-oriented architectures provide candidate foundations (Schoenegger et al. 2026; Marco Becattini et al. 2025); agent fusion and team-aware methods extend them to multi-agent systems (Yasser Saeid and Thomas Kopinski 2024; Zhenfeng Su et al. 2025), alongside explicit verification models (Oleksandr Martynyuk et al. 2021). The engineering challenge is therefore not only how to produce evidence, but how to make assurance claims portable, comparable, and durable under heterogeneous external oversight. This ultimately requires coordination between technical standards bodies and governance institutions that lies beyond any single engineering contribution. The four-direction validation stack addresses engineering within a deployment lifecycle. Assurance at scale, overhead management, human-team dynamics, and evidence portability remain broader socio-technical and governance challenges.

11 Conclusion This survey argues that unit, integration, and benchmark results alone do not provide adequate validation coverage for agentic systems. Their relevant behavior is a trajectory realized through planning, tool use, memory, coordination, and adaptation in context. That shift changes the assurance target from isolated component correctness to bounded acceptable behavior over time. Drawing on a corpus of 257 papers, the survey makes four claims. First, classical testing abstractions remain necessary but stop short of the validation target once software acts autonomously in consequential environments. Second, the literature is most usefully organized by five validation dimensions: behavioral, safety, temporal, regulatory, and multi-agent, each capturing distinct failure modes and evidence needs. Third, existing work is unevenly distributed across those dimensions: behavioral evaluation is comparatively mature, while temporal validity, lifecycle evidence, and certifiable governance remain structurally underdeveloped. Fourth, the

40

survey translates those asymmetries into a four-direction lifecycle stack spanning bounded-autonomy specifications, adversarial trajectory generation, temporal runtime monitoring, and human-verified evidence chains, an interpretive synthesis of the surveyed gaps supported by the corpus-level counts. The broader implication is methodological as much as technical. Validation for agentic AI is no longer mainly about measuring whether a model can succeed on a task; it is about demonstrating that trajectories remain acceptable, inspectable, and governable as environments, interfaces, and evidence change. That is why agentic AI demands not a larger benchmark alone, but a new validation paradigm.

Statements and Declarations Funding This research received no external funding.

Ethics approval and consent to participate Not applicable

Consent for publication Not applicable

Competing interests The authors declare that they have no conflict of interest related to this work.

Data availability Not applicable

Author contributions F.O.M. conceived the original idea, designed the review methodology, developed the five-dimensional taxonomy, conducted the literature review and analysis, synthesized the findings, prepared the case studies, and drafted the manuscript. L.D. and G.T. contributed to the methodology, literature analysis, taxonomy refinement, interpretation of the findings, and critical revision of the manuscript. S.S., F.L., A.P., and G.M. supervised the work, contributed to the interpretation of the findings, and critically revised the manuscript for important intellectual content. All authors read and approved the final manuscript.

References Abou Ali, N., et al.: Agentic AI: A Comprehensive Survey of Architectures, Applications, and Future Directions. arXiv:2510.25445 (2025). https://doi.org/10.48550/a rXiv.2510.25445 41

Arjun Warrier, Abhilash K S: Autonomous Agentic AI for Clinical Workflow Orchestration: Self-Managing Healthcare Operations. doi: 10.1109/ICICNIS66685.2025.11315712 (2025). https://doi.org/10.1109/ICICNIS66685.2025.1131 5712 Abdulla Almarzooqi, Abid Abderrazek, Fakhri Karray: Multimodal Agentic System for Highway Safety Monitoring. doi: 10.1109/ComComAp68359.2025.11353134 (2025). https://doi.org/10.1109/ComComAp68359.2025.11353134 Akshathala, S., Adnan, B., Ramesh, M., Vaidhyanathan, K., Muhammed, B., Parthasarathy, K.: Beyond Task Completion: An Assessment Framework for Evaluating Agentic AI Systems (2025). https://arxiv.org/abs/2512.12791 Aaluri Seenu, Avinash Reddy Aitha, Vijaya Rama Raju Gottimukkala, Jeevani Singireddy, Raviteja Meda, Ravi Shankar Garapati: Hybrid Multi-Agent Reinforcement Learning and Blockchain Framework for Real-Time Transaction Integrity in CloudDriven Financial Systems. doi: 10.1109/GCWCN66157.2025.11448456 (2025). https: //doi.org/10.1109/GCWCN66157.2025.11448456 Alshiekh, M., Bloem, R., Ehlers, R., Könighofer, B., Niekum, S., Topcu, U.: Safe reinforcement learning via shielding. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18), pp. 2669–2678. AAAI Press, Palo Alto, California, USA (2018). https://doi.org/10.1609/aaai.v32i1.11797 Akinyemi Arabambi, Bisola Faith Kayode, Olamide Faroun, Fatimat Okeleye, Emereuwaonu Ezechukwu, Abiola Oludotun, Oluwatosin Oyeladun, Gabriel Aiyeetan: Decision Alignment Protocols: Harmonising AI Agents for Comprehensive Market Assessment. doi: 10.1109/ICECCE67514.2025.11257956 (2025). https://do i.org/10.1109/ICECCE67514.2025.11257956 Aruun K. Kumar, George Belsian, Santosh Srinivasan, Giridhar Shyam Sankararaman: Multi-Layered Security Framework for Financial AI Solutions - PISA. doi: 10.1109/ICAIC67076.2026.11395741 (2026). https://doi.org/10.1109/ICAIC67076 .2026.11395741 Akshata Kishore Moharir, Jay Prakash Thakur, Aditya Deshpande, Apoorva Phadnis: A Comparative Survey of Trust-Building Methods in Multi-Agent AI Systems. doi: 10.1109/CogMI67134.2025.00063 (2025). https://doi.org/10.1109/CogMI67134.202 5.00063 Acharya, D.B., Kuppan, K., Divya, B.: Agentic AI: Autonomous intelligence for complex goals–a comprehensive survey. IEEE Access 13, 18912–18936 (2025) https: //doi.org/10.1109/ACCESS.2025.3532853 Alelyani, T.: A validated framework for responsible AI in healthcare autonomous systems. Scientific Reports 15, 44432 (2025) https://doi.org/10.1038/s41598-025-2 5266-z

42

Alex Kaplunovich: Plan-Execute-Generate-Judge: A Self Verifying Multi Agent LLM Framework for Complex NoSQL Querying. doi: 10.1109/BigData66926.2025.11400819 (2025). https://doi.org/10.1109/BigData66926.2025.11 400819 Agrippina Mwangi, León Navarro-Hilfiker, Lukasz Brewka, Mikkel Gryning, Elena Fumagalli, Madeleine Gibescu: A Threshold-Triggered Deep Q-Network-Based Framework for Self-Healing in Autonomic Software-Defined IIoT-Edge Networks. doi: 10.1109/TNSM.2025.3647853 (2025). https://doi.org/10.1109/TNSM.2025.36 47853 Amine Barrak: Traceability and Accountability in Role-Specialized Multi-Agent LLM Pipelines. doi: 10.1109/ASEW67777.2025.00064 (2025). https://doi.org/10.1109/ ASEW67777.2025.00064 Ammann, P., Offutt, J.: Introduction to Software Testing, 2nd edn. Cambridge University Press, Cambridge, United Kingdom (2016) Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., Mané, D.: Concrete Problems in AI Safety. arXiv:1606.06565 (2016). https://doi.org/10.48550/arXiv.1 606.06565 Abdullah Mushtaq, Rafay Naeem, Ibrahim Ghaznavi, Imran Taj, Imran Hashmi, Junaid Qadir: Harnessing Multi-Agent LLMs for Complex Engineering Problem-Solving: A Framework for Senior Design Projects. doi: 10.1109/EDUCON62633.2025.11016653 (2025). https://doi.org/10.1109/EDUCON 62633.2025.11016653 Asaro, P.: The liability problem for autonomous artificial agents. In: 2016 AAAI Spring Symposium Series. AAAI Press, Palo Alto, California, USA (2016). AAAI Spring Symposium on AI, Ethics, and Society. https://cdn.aaai.org/ocs/12699/1269956141-1-PB.pdf Andriushchenko, M., Souly, A., Dziemian, M., Duenas, D., Lin, M., Wang, J., Hendrycks, D., Zou, A., Kolter, J.Z., Fredrikson, M., Winsor, E., Wynne, J., Gal, Y., Davies, X.: AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents. arXiv:2410.09024 (2024). https://doi.org/10.48550/arXiv.2410.09024 Anshuman Chhabra, Shrestha Datta, Shahriar Kabir Nahin, Prasant Mohapatra: Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges. doi: 10.1109/ACCESS.2026.3675554 (2026). https://doi.org/10.1109/ACCESS.2026.36 75554 Amal Gueroudji, Tanwi Mallick, Renan Souza, Rafael Ferreira Da Silva, Robert Ross, Matthieu Dorier, Philip Carns, Kyle Chard, Ian Foster: ControlA: Agentic Workflow Control Mechanisms for Reliable Science. doi: 10.1109/eScience65000.2025.00086 (2025). https://doi.org/10.1109/eScience65000.2025.00086

43

Benjamin D. Werner, Benjamin J. Schumeg, Tiffany M. Mills, Elizabeth V. Velilla: An Assurance Case for the DoD Ethical Principles of Artificial Intelligence. doi: 10.1109/RAMS51473.2023.10088273 (2023). https://doi.org/10.1109/RAMS51473. 2023.10088273 Breck, E., Cai, S., Nielsen, E., Salib, M., Sculley, D.: The ML test score: A rubric for ML production readiness and technical debt reduction. In: 2017 IEEE International Conference on Big Data (BigData). IEEE, Piscataway, New Jersey, USA (2017). https://doi.org/10.1109/BigData.2017.8258038 Bruce Fang, Danyi Gao: Collaborative Multi-Agent Reinforcement Learning Approach for Elastic Cloud Resource Scaling. doi: 10.1109/ICAITA67588.2025.11137847 (2025). https://doi.org/10.1109/ICAITA67588.2025.11137847 Beizer, B.: Software Testing Techniques, 2nd edn. Van Nostrand Reinhold, New York, New York, USA (1990) Badr, N., Giustiniani, A., Maes, D., Verschuure, M.: Deriving safety requirements for machine learning based systems using an attribute-driven goal structuring notation method. In: 2025 9th International Conference on System Reliability and Safety (ICSRS), pp. 111–116. IEEE, ??? (2025). https://doi.org/10.1109/ICSRS68021.202 5.11422049 Badertdinov, I., Golubev, A., Nekrashevich, M., Shevtsov, A., Karasik, S., Andriushchenko, A., Trofimova, M., Litvintseva, D., Yangel, B.: SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents. arXiv:2505.20411 (2025). https://doi.org/10.48550/arX iv.2505.20411 Braiek, H.B., Khomh, F.: On testing machine learning programs. Journal of Systems and Software 164, 110542 (2020) https://doi.org/10.1016/j.jss.2020.110542 Basma ElSaify, Mohamed Baderelden: Adversarial and Multilingual Threats in Retrieval-Augmented Generation: From Prompt Injection to Model Exploitation. doi: 10.1109/GACLM67198.2025.11231998 (2025). https://doi.org/10.1109/GACL M67198.2025.11231998 Ben Smith, Martin S. Feather, Terry Huntsberger, Robert Bocchino: Software Assurance of Autonomous Spacecraft Control. doi: 10.1109/RAMS48030.2020.9153674 (2020). https://doi.org/10.1109/RAMS48030.2020.9153674 Brown, R., Nguyen, L.V., Xiang, W., Wolf, M., Tran, H.-D.: Perception-based quantitative runtime verification for learning-enabled cyber-physical systems. In: Proceedings of the 16th ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS 2025), pp. 1–11. ACM, New York, New York, USA (2025). https://doi.org/10.1145/3716550.3722031

44

Botao Zhu, Xianbin Wang, Dusit Niyato: Task-Specific Trust Evaluation for Multi-Hop Collaborator Selection via GNN-Aided Distributed Agentic AI. doi: 10.1109/JSAC.2025.3642235 (2025). https://doi.org/10.1109/JSAC.2025.3642235 Collaco, B.G., Haider, S.A., Prabha, S., et al.: The role of agentic artificial intelligence in healthcare: A scoping review. npj Digital Medicine 9, 289 (2026) https://doi.or g/10.1038/s41746-026-02517-5 Chenhang Xu, Jia Wang, Xiaohui Zhu, Yong Yue, Jun Qi, Jieming Ma: Toward Multi-Agent Coordination in IoT via Prompt Pool-based Continual Reinforcement Learning. doi: 10.1109/ISPA63168.2024.00296 (2024). https://doi.org/10.1109/IS PA63168.2024.00296 C. Ma, J. Zhang, Z. Zhu, et al.: AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents. arXiv:2401.13178, in Proc. NeurIPS, 2024 (2024) Cemri, M., Lyu, Y., Lo, Y., Swaminathan, A., Nandwani, Y., Wang, Z., Gupta, S.: Why Do Multi-Agent LLM Systems Fail? arXiv:2503.13657 (2025). https://doi.or g/10.48550/arXiv.2503.13657 Collins P Obeng, Nethshan M Narasinghe, Ryan Striker, Enrique Alvarez Vazquez: Security Framework for Agentic Home AI in Preventive Healthcare: Cyber Threats Worth Noting. doi: 10.1109/CARS67163.2025.11337832 (2025). https://doi.org/10 .1109/CARS67163.2025.11337832 Charles Hartsell, Shreyas Ramakrishna, Abhishek Dubey, Daniel Stojcsics, Nagabhushan Mahadevan, Gabor Karsai: ReSonAte: A Runtime Risk Assessment Framework for Autonomous Systems. doi: 10.1109/SEAMS51251.2021.00025 (2021). http s://doi.org/10.1109/SEAMS51251.2021.00025 Chen, S., Singh, S., Pusarla, P., Hou, B., Bhat, S., Fields, B.A., Ray, B.: Benchmarking Large Language Models Under Data Contamination: A Survey from Static to Dynamic Evaluation. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP 2025) (2025). https://doi.org/10.18653/v 1/2025.emnlp-main.511 Chitiz Tayal, Sujit Murumkar, Sammip Biradar: Analysing the Role of Multi-Agent AI Models for Autonomous Business Decision Systems. doi: 10.1109/CCWC67433.2026.11393746 (2026). https://doi.org/10.1109/CCWC6743 3.2026.11393746 Chaffer, T.J., Goins II, C., Okusanya, B., Cotlage, D., Goldston, J.: Decentralized Governance of Autonomous AI Agents (2025). https://arxiv.org/abs/2412.17114 Draelos, R.L., Afreen, S., Blasko, B., et al.: Large language models provide unsafe answers to patient-posed medical questions. npj Digital Medicine 9, 241 (2026) https://doi.org/10.1038/s41746-026-02428-5

45

Dahlquist, N., Lindqvist, B., Saradagi, A., Nikolakopoulos, G.: Reactive Multi-agent Coordination using Auction-based Task Allocation and Behavior Trees (2023). http s://arxiv.org/abs/2304.01976 Dhaminda B. Abeywickrama, Michael Fisher, Frederic Wheeler, Louise Dennis: Towards Patterns for a Reference Assurance Case for Autonomous Inspection Robots. doi: 10.1109/ICSR66718.2025.00016 (2025). https://doi.org/10.1109/ICSR 66718.2025.00016 Dholakia, N., Shukla, M., Khan, S., Jadeja, R.: An overview of genai 2.0 partnering with digital twin to enhance decision making. IEEE Communications Standards Magazine PP, 1–10 (2025) https://doi.org/10.1109/MCOMSTD.2025.3644899 Du, Y., Wang, B., Xiang, Y., Wang, Z., Huang, W., Xue, B., Liang, B., Zeng, X., Mi, F., Bai, H., Shang, L., Pan, J.Z., Jiang, Y., Wong, K.-F.: Memory-T1: Reinforcement Learning for Temporal Reasoning in Multi-session Agents (2025). https://arxiv.or g/abs/2512.20092 Debenedetti, E., Zhang, J., Balunovic, M., Beurer-Kellner, L., Fischer, M., Tramèr, F.: AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents. arXiv:2406.13352 (2024). https://doi.org/10.48550/arX iv.2406.13352 Ewen Denney, Ganesh Pai: Assurance-driven Design of Machine Learning-based Functionality in an Aviation Systems Context. doi: 10.1109/DASC58513.2023.10311282 (2023). https://doi.org/10.1109/DASC58513.2023.10311282 European Union: Regulation (EU) 2017/745 of the European Parliament and of the Council of 5 April 2017 on Medical Devices. Official Journal of the European Union, L 117, 5 May 2017 (2017). https://eur-lex.europa.eu/eli/reg/2017/745/oj/eng Ferrell, U.D., Anderegg, A.H.A.: Validation of assurance case for dynamic systems. In: 2022 IEEE/AIAA 41st Digital Avionics Systems Conference (DASC), pp. 1–11. IEEE, Portsmouth, VA, USA (2022). https://doi.org/10.1109/DASC55683.2022.9 925731 Fredrik Warg, Hans Blom, Jonas Borg, Rolf Johansson: Continuous Deployment for Dependable Systems with Continuous Assurance Cases. doi: 10.1109/ISSREW.2019.00091 (2019). https://doi.org/10.1109/ISSREW.2019.00091 F. F. Xu, Y. Wang, X. Xiao, et al.: TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks. arXiv:2412.14161, 2025 (2025) Geeta Mahala, Aditya Ghose, Hoa Khanh Dam, Angela Consoli: Verifying Multi -Agent Coordination Correctness for BDI Agents. doi: 10.1109/COMPSAC61105.2024.00013 (2024). https://doi.org/10.1109/COMPSAC61105.2024.0001 3

46

Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., Fritz, M.: Not What You’ve Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. arXiv:2302.12173 (2023). https://doi.org/10.48550 /arXiv.2302.12173 G. Li, H. A. A. K. Hammoud, H. Itani, D. Khizbullin, B. Ghanem: CAMEL: Communicative Agents for “Mind” Exploration of Large Language Model Society. in Proc. NeurIPS, 2023, arXiv:2303.17760 (2023) Giuliano Lorenzoni, Paulo Alencar, Donald Cowan: ABL: An LLM-Based Agentic Framework for Applying Black-Litterman Portfolio Optimization. doi: 10.1109/BigData66926.2025.11402167 (2025). https://doi.org/10.1109/BigData66926.2025.11 402167 Gamini Singh, Vijay K. Madisetti: MCP-Secure: A Runtime Access Control Layer for Privilege-Aware LLM Agent Tooling. doi: 10.1109/OJCS.2026.3664314 (2026). https://doi.org/10.1109/OJCS.2026.3664314 Haowen Tan: Boundary-Aware Concept Drift Detection for Automated Industrial Systems with Limited Supervision. doi: 10.1109/IFEEA66847.2025.11388971 (2025). https://doi.org/10.1109/IFEEA66847.2025.11388971 Hawkins, R.D., Paterson, C., Picardi, C., Jia, Y., Calinescu, R., Habli, I.: Guidance on the assurance of machine learning in autonomous systems (AMLAS). Technical report, University of York (2021). https://doi.org/10.48550/arXiv.2102.01564 . arXiv:2102.01564 Himanshu Agarwal, Rafal Dorociak, Achim Rettberg: On safety assurance case for deep learning based image classification in highly automated driving. doi: 10.23919/DATE51398.2021.9473924 (2021). https://doi.org/10.23919/DATE51398 .2021.9473924 Izunna Okpala, Ashkan Golgoon, Arjun Ravi Kannan Discover: Model Risk Management in Finance: The Role of Agentic Systems in Risk Mitigation. doi: 10.1109/ICA67499.2025.00010 (2025). https://doi.org/10.1109/ICA67499.2025.00 010 Irving, G., Christiano, P., Amodei, D.: AI Safety via Debate. arXiv:1805.00899, 2018, doi: 10.48550/arXiv.1805.00899 (2018). https://doi.org/10.48550/arXiv.1805.00899 IEEE Standards Association: IEEE draft guide for verification of autonomous systems. Draft Guide IEEE P2817, IEEE Standards Association (2025) Ioannis Nearchou, Lance Rafalko, Ryan Phillips, Matthew Anderson, Wuwei Shen, Steve Drager: An Assurance Case Driven Development Paradigm for Autonomous Vehicles: An F1TENTH Racing Car Case Study. doi: 10.1109/SERA57763.2023.10197746 (2023). https://doi.org/10.1109/SERA57763.

47

2023.10197746 International Electrotechnical Commission: IEC 62304: Medical device software– software life cycle processes. International Standard IEC 62304:2006+A1:2015, International Electrotechnical Commission (2006) ISO: ISO 21448: Road vehicles–safety of the intended functionality (SOTIF). Standard 21448:2022, International Organization for Standardization (2022) Jiin Kim, Byeongjun Shin, Jinha Chung, Minsoo Rhu: The Cost of Dynamic Reasoning: Demystifying AI Agents and Test-Time Scaling from an AI Infrastructure Perspective. doi: 10.1109/HPCA68181.2026.11408569 (2026). https://doi.org/10.1 109/HPCA68181.2026.11408569 Jha, S., Bhat, P., Khanduja, A., Shenoy, P., Otte, M., Sreekanti, V., Tumanov, A., Posada, J., Mytkowicz, T., Sarma, S.: ITBench: Evaluating AI Agents Across Diverse Real-World IT Automation Tasks. arXiv:2502.05352 (2025). https://doi.or g/10.48550/arXiv.2502.05352 João Vitor de Carvalho Silva, Douglas G. Macharet: Can LLM Agents Solve Collaborative Tasks? A Study on Urgency-Aware Planning and Coordination. doi: 10.1109/ICAR65334.2025.11338680 (2025). https://doi.org/10.1109/ICAR65334.20 25.11338680 Jiayi Geng, Graham Neubig: Effective Strategies for Asynchronous Software Engineering Agents. arXiv:2603.21489v1 (2026). https://doi.org/10.48550/arXiv.2603.2148 9 Jesus L. Lobo, Ibai Laña, Eneko Osaba, Javier Del Ser: On the Connection between Concept Drift and Uncertainty in Industrial Artificial Intelligence. doi: 10.1109/CAI54212.2023.00081 (2023). https://doi.org/10.1109/CAI54212.2023.00 081 Jilk, D.J.: Limits to Verification and Validation of Agentic Behavior (2016). https: //arxiv.org/abs/1604.06963 Juan Ren, Mark Dras, Usman Naseem: Agentic Moderation: Multi-Agent Design for Safer Vision-Language Models. doi: 10.1109/BigData66926.2025.11401450 (2025). https://doi.org/10.1109/BigData66926.2025.11401450 Jiahang Lin, Shichun Liu, Chengjun Pan, Lizhi Lin, Shihan Dou, Xuanjing Huang, Hang Yan, Zhenhua Han, Tao Gui: Agentic Harness Engineering: ObservabilityDriven Automatic Evolution of Coding-Agent Harnesses. arXiv:2604.25850v3 (2026). https://doi.org/10.48550/arXiv.2604.25850 Jianhao Yan, Yun Luo, Yue Zhang: RefuteBench 2.0 – Agentic Benchmark for Dynamic

48

Evaluation of LLM Responses to Refutation Instruction. doi: 10.1109/TASLPRO.2025.3626926 (2025). https://doi.org/10.1109/TASLPRO.2025.36269 26 Kadavath, S., et al.: Language Models (Mostly) Know What They Know. arXiv:2207.05221 (2022). https://doi.org/10.48550/arXiv.2207.05221 . https: //arxiv.org/abs/2207.05221 Kim, Y., Gu, K., Park, C., Park, C., Schmidgall, S., Heydari, A.A., Yan, Y., Zhang, Z., Zhuang, Y., Liu, Y., Malhotra, M., Liang, P.P., Park, H.W., Yang, Y., Xu, X., Du, Y., Patel, S., Althoff, T., McDuff, D., Liu, X.: Towards a Science of Scaling Agent Systems (2026). https://arxiv.org/abs/2512.08296 Kangning Gao, Haotian Zhu, Rui Liu, Jinming Li, Xu Yan, Yi Hu: Contextual Trust Evaluation for Robust Coordination in Large Language Model Multi-Agent Systems. doi: 10.1109/EIECC67963.2025.11409558 (2025). https://doi.org/10.1109/EIECC6 7963.2025.11409558 Khan, R., Joyce, D., Habiba, M.: AGENTSAFE: A Unified Framework for Ethical Assurance and Governance in Agentic AI (2025). https://arxiv.org/abs/2512.03180 Kreuzberger, D., K”uhl, N., Hirschl, S.: Machine learning operations (MLOps): Overview, definition, and architecture. IEEE Access 11, 31866–31879 (2023) https: //doi.org/10.1109/ACCESS.2023.3262138 Koohestani, R.: AgentGuard: Runtime Verification of AI Agents. arXiv:2509.23864 (2025). https://doi.org/10.48550/arXiv.2509.23864 Kuo, N.I.-H., Polizzotto, M.N., Finfer, S., et al.: The health gym: Synthetic healthrelated datasets for the development of reinforcement learning algorithms. Scientific Data 9, 693 (2022) https://doi.org/10.1038/s41597-022-01784-7 Kartik Rajesh, Salonee Surve, Jish Joy, Ami Munshi: SmartOps: From Demand to Delivery. doi: 10.1109/ICAIQSA67794.2025.11440525 (2025). https://doi.org/10.1 109/ICAIQSA67794.2025.11440525 Kumar Yashu, Vyshali J Gogi, Azarudheen S: Adaptive Communication Protocols for Manager-Worker Small LLM Multi-Agent Systems in Resource-Constrained Environments. doi: 10.1109/DELCON68055.2025.11400330 (2025). https://doi.org/10.1 109/DELCON68055.2025.11400330 Lin, V., Brown, R., Jafarpour, S., Mangharam, R.: Safety monitoring for learningenabled cyber-physical systems in out-of-distribution scenarios. In: Proceedings of the 16th ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS 2025), pp. 5–1512. ACM, New York, New York, USA (2025). https://doi.org/10.1 145/3716550.3722022

49

Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., Zhang, Y., Narayanan, D., Wu, Y., Kumar, A., Newman, B., Yuan, B., Yan, B., Zhang, C., Cosgrove, C., Manning, C.D., Ré, C., Acosta-Navas, D., Hudson, D.A., Zelikman, E., Durmus, E., Ladhak, F., et al.: Holistic Evaluation of Language Models. arXiv:2211.09110 (2022). https://doi.org/10.48550/arXiv.2211.09110 Leander Melroy Maben, Gayathri Ganesh Lakshmy, Srijith Radhakrishnan, Siddhant Arora, Shinji Watanabe: AURA: Agent for Understanding, Reasoning, and Automated Tool Use in Voice-Driven Tasks. doi: 10.1109/ASRU65441.2025.11434725 (2025). https://doi.org/10.1109/ASRU65441.2025.11434725 Lin Huo, Jianlin Mao, Hongjun San, Ruiqi Li, Zhiwei Xuan: Mean-Field Deep Reinforcement Learning for Multi-Agent Path Finding. doi: 10.1109/LRA.2026.3669783 (2026). https://doi.org/10.1109/LRA.2026.3669783 Lorena Poenaru-Olaru, Luis Cruz, Jan S. Rellermeyer, Arie van Deursen: Maintaining and Monitoring AIOps Models Against Concept Drift. doi: 10.1109/CAIN58948.2023.00024 (2023). https://doi.org/10.1109/CAIN58948.2023 .00024 Lorena Poenaru-Olaru, Luis Cruz, Jan S. Rellermeyer, Arie van Deursen: Improving the Reliability of Failure Prediction Models through Concept Drift Monitoring. doi: 10.1109/DeepTest66595.2025.00006 (2025). https://doi.org/10.1109/DeepTest6659 5.2025.00006 Lucas G. M. Castro, Myke D. M. Valadão, Waldir S. S. Júnior, Celso B. Carvalho: Adaptive Semantic Gates for Reliable Multi-Agent LLMs. doi: 10.1109/ICCE67443.2026.11449653 (2026). https://doi.org/10.1109/ICCE67443.20 26.11449653 Leest, J., Raibulet, C., Gerostathopoulos, I., Lago, P.: Expert monitoring: Humancentered concept drift detection in machine learning operations. In: Proceedings of the 2024 ACM/IEEE 46th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER 2024), pp. 1–5. IEEE Computer Society, Los Alamitos, California, USA (2024). https://doi.org/10.1145/3639476.3639771 . Lisbon, Portugal Lee, J.D., See, K.A.: Trust in automation: Designing for appropriate reliance. Human Factors 46(1), 50–80 (2004) Lipton, Z.C., Steinhardt, J.: Troubling Trends in Machine Learning Scholarship. arXiv:1807.03341 (2018). https://doi.org/10.48550/arXiv.1807.03341 Luis-Pedro Cobos, Tianlei Miao, Kacper Sowka, Garikayi Madzudzo, Alastair R. Ruddle, Ehab El Amam: Application of an Automotive Assurance Case Approach to Autonomous Marine Vessel Security. doi: 10.1109/ICECCME55909.2022.9988376 (2022). https://doi.org/10.1109/ICECCME55909.2022.9988376

50

Liu, X., Xu, C., Zhang, J., Wang, J., Xu, Y., Wang, X., Qiu, Y., Zhang, J., Wang, Y., Zheng, K., et al.: AgentBench: Evaluating LLMs as Agents. arXiv:2308.03688 (2023). https://doi.org/10.48550/arXiv.2308.03688 Lulu Li, Yafei Li, Shaohui Zhang, Yuanyuan Jin, Shuo He, Ke Wang, Mingliang Xu: Efficient Cooperative Mechanism for Distributed Multi-Agent Traffic Signal Control. doi: 10.1109/TMC.2025.3587257 (2025). https://doi.org/10.1109/TMC.2025.35872 57 Luo, J., Zhang, W., Yuan, Y., Zhao, Y., et al.: Large Language Model Agent: A Survey on Methodology, Applications and Challenges. arXiv:2503.21460 (2025). https://do i.org/10.48550/arXiv.2503.21460 M. Mahdy and C. Rubio-Medrano: Beyond the Chatbox: An Exploratory Case Study of Autonomous Computer-Use Agents. in Proc. IEEE CogMI, 2025, doi: 10.1109/CogMI67134.2025.00062 (2025). https://doi.org/10.1109/CogMI67134.202 5.00062 Masike Malatji: Benchmarking Offensive AI Agents Through the Autonomous Attack Chain. doi: 10.1109/RAAI67517.2025.11423369 (2025). https://doi.org/10.1109/ RAAI67517.2025.11423369 Medical Device Coordination Group: MDCG 2025-10: Post-market surveillance considerations for ai-enabled medical device software. Guidance MDCG 2025-10, European Commission (2025). https://health.ec.europa.eu/document/download/a9ad86b7-1 b8e-4bae-beb4-48b2b3ed2f05 en?filename=mdcg 2025-10 en.pdf Medical Device Coordination Group: MDCG 2025-6: Interplay between the medical devices regulation and the AI act for medical device software. Guidance MDCG 2025-6, European Commission (2025). https://health.ec.europa.eu/document/dow nload/b78a17d7-e3cd-4943-851d-e02a2f22bbb4 en?filename=mdcg 2025-6 en.pdf Mitra, S.: Agentic Temporal Graph of Reasoning with Multimodal Language Models: A Potential AI Aid to Healthcare (2025). https://arxiv.org/abs/2509.11944 Mohammadi, M., Li, Y., et al.: Evaluation and benchmarking of LLM agents: A survey. In: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2025). ACM, New York, New York, USA (2025). https: //doi.org/10.1145/3711896.3736570 Madhukar Nimbalkar, Pankaj Chandre, Bhagyashree Shendkar, Sachin Jagdale, Renuka Arbat, Shilpa Dhopte: Agent AI for Personalized Healthcare: A Multi-Agent Framework for Real-Time Disease Detection and Patient Support. doi: 10.1109/ICTBIG68706.2025.11323744 (2025). https://doi.org/10.1109/ICTBIG68706.2025.113 23744

51

Marco Becattini, Roberto Verdecchia, Enrico Vicario: An Accountability-Based Architectural Tactic for Agent Cooperation in LLM-Based Multi-Agent Systems. doi: 10.1109/BigData66926.2025.11402340 (2025). https://doi.org/10.1109/BigData669 26.2025.11402340 Myers, G.J., Sandler, C., Badgett, T.: The Art of Software Testing, 2nd edn. Wiley, Hoboken, New Jersey, USA (2004) Michael Vierhauser, Sean Bayley, Jane Wyngaard, Wandi Xiong, Jinghui Cheng, Joshua Huseman, Robyn Lutz, Jane Cleland-Huang: Interlocking Safety Cases for Unmanned Autonomous Systems in Shared Airspaces. doi: 10.1109/TSE.2019.2907595 (2019). https://doi.org/10.1109/TSE.2019.2907595 Mohammad Peyman, Yusef Ahsini: Agentic Simheuristic: Integrating Generative AI and Simheuristic for a Team Orienteering Problem. doi: 10.1109/WSC68292.2025.11339023 (2025). https://doi.org/10.1109/WSC68292.2 025.11339023 Maxim Chernyshev, Zubair Baig, Robin Doss: [Short Paper] Forensic Analysis of Indirect Prompt Injection Attacks on LLM Agents. doi: 10.1109/TPSISA62245.2024.00053 (2024). https://doi.org/10.1109/TPS-ISA62245.2024.000 53 Nejad Alagha, Aya A. Elkhodiry, Abigail Copiaco, Yassine Himeur, Wathiq Mansoor, Christian Ritz, Valsamma Eapen, Ammar Albanna: Empowering Convenient Homebased Progressive Autism Diagnosis and Management System through Agentic AIpowered Assistive Technology. doi: 10.1109/HealthCom60686.2025.11343645 (2025). https://doi.org/10.1109/HealthCom60686.2025.11343645 Niklas Dahlquist, Akshit Saradagi, George Nikolakopoulos: Behavior Tree Based Decentralized Multi-agent Coordination for Balanced Servicing of Time Varying Task Queues. doi: 10.1109/IROS58592.2024.10801900 (2024). https://doi.org/10.1 109/IROS58592.2024.10801900 Niklas Dahlquist, Björn Lindqvist, Akshit Saradagi, George Nikolakopoulos: Reactive Multi-agent Coordination using Auction-based Task Allocation and Behavior Trees. doi: 10.1109/CCTA54093.2023.10252961 (2023). https://doi.org/10.1109/CCTA54 093.2023.10252961 Oleksandr Martynyuk, Oleksandr Drozd, Anatoliy Sachenko, Hanna Stepova, Dmitry Martynyuk, Lyudmila Sugak, Iryna Turchenko: Verification Model for Agent Coordination of Distributed Information Systems. doi: 10.1109/IDAACS53288.2021.9660961 (2021). https://doi.org/10.1109/IDAACS53 288.2021.9660961 Poray, J., Jungmann, P., Franczyk, B., Heller, T.: Updating machine learning models in real-world applications: Finding best deployment settings. In: 2025 International

52

Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA), pp. 1–9. IEEE, Antalya, Türkiye (2025). https://doi.org/10.1109/AC DSA65407.2025.11166305 Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D., Shamseer, L., Tetzlaff, J.M., Akl, E.A., Brennan, S.E., Chou, R., Glanville, J., Grimshaw, J.M., Hróbjartsson, A., Lalu, M.M., Li, T., Loder, E.W., MayoWilson, E., McDonald, S., McGuinness, L.A., Stewart, L.A., Thomas, J., Tricco, A.C., Welch, V.A., Whiting, P., Moher, D.: The prisma 2020 statement: an updated guideline for reporting systematic reviews. BMJ 372, 71 (2021) https://doi.org/10 .1136/bmj.n71 Poenaru-Olaru, L., Karpova, N., Cruz, L., Rellermeyer, J.S., Deursen, A.: Is your anomaly detector ready for change? adapting aiops solutions to the real world. In: Proceedings of the IEEE/ACM 3rd International Conference on AI Engineering - Software Engineering for AI. CAIN 2024, pp. 222–233. ACM, Lisbon, Portugal (2024). h t t p s : / / d o i . o r g / 1 0 . 1 1 4 5 / 3 6 4 4 8 1 5 . 3 6 4 4 9 6 1 . http://dx.doi.org/10.1145/3644815.3644961 Parasuraman, R., Riley, V.: Humans and automation: Use, misuse, disuse, abuse. Human Factors 39(2), 230–253 (1997) https://doi.org/10.1518/0018720977785438 86 Pat Rondon, Renyao Wei, José Cambronero, Jürgen Cito, Aaron Sun, Siddhant Sanyam, Michele Tufano, Satish Chandra: Evaluating Agent-Based Program Repair at Google. doi: 10.1109/ICSE-SEIP66354.2025.00038 (2025). https://doi.org/10.1 109/ICSE-SEIP66354.2025.00038 Preetam Dedu, Willison Lopes, Nishanth Sirikonda: An Observability Framework for Detecting Bias and Drift in AI-Based Student Early Alert Systems. doi: 10.1109/ICALTER69698.2025.11355034 (2025). https://doi.org/10.1109/ICALTE R69698.2025.11355034 Quionero-Candela, J., Sugiyama, M., Schwaighofer, A., Lawrence, N.D.: Dataset Shift in Machine Learning. MIT Press, Cambridge, Massachusetts, USA (2009) Qingrui Li, Kapileshwor Ray Amat, Juan Li: From Knowledge to Action: An Agentic AI Framework for Diabetes Management. doi: 10.1109/UEMCON67449.2025.11267605 (2025). https://doi.org/10.1109/UEMCON67449.2025.1 1267605 Rakesh More: A Unified Evaluation Framework for Grounded LLM Architectures: Comparative Analysis of RAG, Self-RAG, and Agentic RAG. doi: 10.1109/AISP68263.2025.11396270 (2025). https://doi.org/10.1109/AISP68263.20 25.11396270 Rath, A.: Agent Drift: Quantifying Behavioral Degradation in Multi-Agent LLM

53

Systems Over Extended Interactions (2026). https://arxiv.org/abs/2601.04170 Ray, P.: Reliability-by-design for agentic genai: Turning the ai risk atlas into an eu ai act-ready assurance case. IEEE Reliability Magazine PP, 1–10 (2026) https: //doi.org/10.1109/MRL.2026.3660197 Rafiullah Omar, Justus Bogner, Joran Leest, Vincenzo Stoico, Patricia Lago, Henry Muccini: How to Sustainably Monitor ML-Enabled Systems? Accuracy and Energy Efficiency Tradeoffs in Concept Drift Detection. doi: 10.1109/ICT4S64576.2024.00026 (2024). https://doi.org/10.1109/ICT4S64576.202 4.00026 Riccio, V., Jahangirova, G., Stocco, A., Humbatova, N., Weiss, M., Tonella, P.: Testing machine learning based systems: A systematic mapping. Empirical Software Engineering 25(6), 5193–5254 (2020) https://doi.org/10.1007/s10664-020-09881-0 Rahul Kapoor, Miray Kas: Breaking the Observability Tax: Dynamic Resolution Anomaly Detection via Topology-Aware Active LLM Agents. doi: 10.1109/ACCESS.2026.3675074 (2026). https://doi.org/10.1109/ACCESS.2026.367507 4 Rohlinger Tihomir: Automated Interpretation of Fleet Incidents to Enable System Level Runtime Assurance. doi: 10.1109/ISSREW63542.2024.00053 (2024). https: //doi.org/10.1109/ISSREW63542.2024.00053 Roham Koohestani: AgentGuard: Runtime Verification of AI Agents. doi: 10.1109/ASEW67777.2025.00023 (2025). https://doi.org/10.1109/ASEW67777.20 25.00023 Rupesh Kohli, Vishwajit K Barbudhe, Vijay M. Rakhade, Trushna M Bhanarkar, Nandini S. Patil, Shreya S. Deshmukh: Autonomous AI Agents in Scientific Discovery – How Agentic AI Accelerates Research in Drug Design, Materials Science, or Climate Modeling. doi: 10.1109/ICAIQSA67794.2025.11440385 (2025). https: //doi.org/10.1109/ICAIQSA67794.2025.11440385 Ruofan Lu, Yichen Li, Yintong Huo: Exploring Autonomous Agents: A Closer Look at Why They Fail When Completing Tasks. doi: 10.1109/ASE63991.2025.00330 (2025). https://doi.org/10.1109/ASE63991.2025.00330 Ran Wei, Zhe Jiang, Haitao Mei, Konstantinos Barmpis, Simon Foster, Tim Kelly, Yan Zhuang: Automated Model-Based Assurance Case Management Using Constrained Natural Language. doi: 10.1109/TCAD.2023.3303220 (2023). https://doi.org/10.1 109/TCAD.2023.3303220 S. Pandruju: COSTAR: Cloud-Observed Safety and Trust-Aware Agentic Reasoning for Enterprise Workflows. in Proc. ICAIDES, 2025, pp. 1–5, doi: 10.1109/ICAIDES67265.2025.11404085 (2025). https://doi.org/10.1109/ICAIDES6

54

7265.2025.11404085 Syed, T.A., Almutairi, M.A., Moaty, M.A.: Toward Trustworthy Agentic AI: A Multimodal Framework for Preventing Prompt Injection Attacks (2025). https://arxiv. org/abs/2512.23557 Satyanarayana Burugupalli: Explainable, Verifiable Agentic AI with Provably Reduced Failure Modes Using Quantum Verification Subroutines. doi: 10.1109/ISED67359.2025.11405134 (2025). https://doi.org/10.1109/ISED67359.20 25.11405134 Sagir M. Yusuf, Chris Baber: Formalizing Distributed Situation Awareness in MultiAgent Networks. doi: 10.1109/THMS.2022.3142109 (2022). https://doi.org/10.110 9/THMS.2022.3142109 Sidong Feng, Changhao Du, Huaxiao Liu, Qingnan Wang, Zhengwei Lv, Gang Huo, Xu Yang, Chunyang Chen: Agent for User: Testing Multi - User Interactive Features in TikTok. doi: 10.1109/ICSE-SEIP66354.2025.00011 (2025). https://doi.org/10.1 109/ICSE-SEIP66354.2025.00011 Schoenegger, P., Carlson, M., Schneider, C., Daly, C.: Verifiable Semantics for Agentto-Agent Communication (2026). https://arxiv.org/abs/2602.16424 S. Zhou, F. F. Xu, H. Zhu, et al.: WebArena: A Realistic Web Environment for Building Autonomous Agents. in Proc. ICLR, 2024, arXiv:2307.13854 (2024) S. Vatsal, H. Dubey, A. Singh: Agentic AI in Healthcare and Medicine: A SevenDimensional Taxonomy for Empirical Evaluation of LLM-Based Agents. IEEE Access, 2026, doi: 10.1109/ACCESS.2026.3651218 (2026). https://doi.org/10.1109/ ACCESS.2026.3651218 Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., Crespo, J.-F., Dennison, D.: Hidden technical debt in machine learning systems. In: Advances in Neural Information Processing Systems 28 (NIPS 2015). Curran Associates, Inc., Red Hook, New York, USA (2015) Shadi Nashwan, Issam Jebreen, Ahmad Al-Qerem, Ammar Almomani, Mohammed Saleh, Amjad Aldweesh: Autonomous Agents in a Zero-Trust World: Architecting Secure, Auditable AI Workflows. doi: 10.1109/ITT69610.2025.11352936 (2025). ht tps://doi.org/10.1109/ITT69610.2025.11352936 Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P.K., Aroyo, L.M.: Everyone wants to do the model work, not the data work: Data cascades in highstakes AI. In: CHI Conference on Human Factors in Computing Systems (CHI ’21). ACM, New York, New York, USA (2021). https://doi.org/10.1145/3411764.3445518 . Yokohama, Japan

55

Schnitzer, R., Kilian, L., Roessner, S., Theodorou, K., Zillner, S.: Landscape of AI safety concerns: A methodology to support safety assurance for AI-based autonomous systems. In: Proceedings of the 2024 International Conference on Safety and Reliability of Systems (ICSRS 2024). IEEE, Piscataway, New Jersey, USA (2024). https://doi.org/10.1109/ICSRS63046.2024.10927556 Sijin Sun, Liangbin Zhao, Ming Deng, Xiuju Fu: VTS-LLM: Domain-Adaptive LLM Agent for Enhancing Awareness in Vessel Traffic Services Through Natural Language. doi: 10.1109/ITSC60802.2025.11423850 (2025). https://doi.org/10.1109/IT SC60802.2025.11423850 Sharma, D., Meshkat, S., Perivolaris, A., et al.: Reimagining psychiatric care with agentic AI: Promise, challenges, and a roadmap forward. npj Digital Medicine 9, 252 (2026) https://doi.org/10.1038/s41746-026-02453-4 Saswata Paul, Naresh Iyer, Daniel Prince, Liang Tang, Michael Durling, Mike Meiners, Baoluo Meng, Nikita Visnevski, Udayan Mandal: Assurance of AI/ML-Based Aerospace Systems Using Overarching Properties. doi: 10.1109/DASC62030.2024.10749102 (2024). https://doi.org/10.1109/DASC62030. 2024.10749102 Sabrine Amri, Nora Boulahia-Cuppens, Frédéric Cuppens: PRISM-AI: A Dual-Stage Neuro-Symbolic Agentic Framework for Privacy Risk Mitigation in LLMs. doi: 10.1109/AICCSA66935.2025.11315208 (2025). https://doi.org/10.1109/AICCSA66 935.2025.11315208 S. Yao, N. Shinn, P. Razavi, K. Narasimhan: tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. arXiv:2406.12045, 2024 (2024) Swarnamouli Majumdar, Sonny Kirkley, Biswadip Basu Mallik, Arman Khan, Soumik Basu: Agentic AI for Mission Adaptation: A Distributed Cognition Framework for Air Force Operations. doi: 10.1109/InCIT66780.2025.11276050 (2025). https://doi. org/10.1109/InCIT66780.2025.11276050 Sungwook Yoon: Strategic Learning Under Linguistic and Contextual Constraints: A Theoretical Framework for LLM-Based Multi-Agent Coordination. doi: 10.1109/ACCESS.2025.3628927 (2025). https://doi.org/10.1109/ACCESS.2025.36 28927 Torgbi Agbemabiese, W.: Toward constitutional autonomy in ai systems: A theoretical framework for aligned agentic intelligence. IEEE Access 14, 11385–11402 (2026) https://doi.org/10.1109/ACCESS.2026.3654907 The MITRE Corporation: SyntheaTM and synthetic patient generation for healthcare simulation and evaluation. project documentation, 2026 (2026). https://github.com /synthetichealth/synthea

56

Toqeer Ali Syed, Mishal Ateeq Almutairi, Mahmoud Abdel Moaty: Cross-Agent Multimodal Provenance-Aware Framework for Robust Prompt Injection Defense in Large Language and Vision-Language Models. doi: 10.1109/ICCA66035.2025.11430791 (2025). https://doi.org/10.1109/ICCA66035.2025.11430791 Talal Ashraf Butt, Muhammad Iqbal, Noor Arshad: From Policy to Pipeline: A Governance Framework for AI Development and Operations Pipelines. doi: 10.1109/ACCESS.2025.3647479 (2025). https://doi.org/10.1109/ACCESS.2025.36 47479 Thomas Cook, Richard Osuagwu, Liman Tsatiashvili, Vrynsia Vrynsia, Koustav Ghosal, Maraim Masoud, Riccardo Mattivi: Retrieval Augmented Generation (RAG) for Fintech: Agentic Design and Evaluation. doi: 10.1109/FLLM67465.2025.11391155 (2025). https://doi.org/10.1109/FLLM67465. 2025.11391155 Turner, A., Ratzlaff, N., Tadepalli, P.: Avoiding side effects in complex environments. In: Advances in Neural Information Processing Systems 33 (NeurIPS 2020). Curran Associates, Inc., Red Hook, New York, USA (2020). https://doi.org/10.48550/arX iv.2006.06547 . https://arxiv.org/abs/2006.06547 Uma D. Ferrell, Alfred H. Andy Anderegg: Holistic Assurance Case for System-ofSystems. doi: 10.1109/DASC55683.2022.9925789 (2022). https://doi.org/10.1109/ DASC55683.2022.9925789 U.S. Food and Drug Administration, Health Canada, Medicines & Healthcare products Regulatory Agency: Good machine learning practice for medical device development: Guiding principles. Guiding principles, U.S. Food and Drug Administration (2021). https://www.fda.gov/medical-devices/software-medical-device-samd/good -machine-learning-practice-medical-device-development-guiding-principles U.S. Food and Drug Administration: Artificial intelligence/machine learning (AI/ML)based software as a medical device action plan. Action plan, U.S. Food and Drug Administration (2021). https://www.fda.gov/media/145022/download U.S. Food and Drug Administration: Artificial intelligence-enabled device software functions: Lifecycle management and marketing submission recommendations. Draft guidance, U.S. Food and Drug Administration (2025). https://www.fda.gov/regula tory-information/search-fda-guidance-documents/artificial-intelligence-enabled-d evice-software-functions-lifecycle-management-and-marketing U.S. Food and Drug Administration: Marketing submission recommendations for a predetermined change control plan for artificial intelligence-enabled device software functions. Final guidance, U.S. Food and Drug Administration (2025). https://www. fda.gov/regulatory-information/search-fda-guidance-documents/marketing-submi ssion-recommendations-predetermined-change-control-plan-artificial-intelligence

57

V. Vinay: The Evolution of Agentic AI in Cybersecurity: From Single LLM Reasoners to Multi-Agent Systems and Autonomous Pipelines. in Proc. IEEE ICAIC, 2026, doi: 10.1109/ICAIC67076.2026.11395809 (2026). https://doi.org/10.1109/ICAIC6 7076.2026.11395809 Vinod B Maniyat, Arun Kumar B R: Adaptive Threat Modeling with MITRE ATT&CK: A Machine Learning Framework for Real-Time Adversarial Detection. doi: 10.1109/CSITSS67709.2025.11295603 (2025). https://doi.org/10.1109/CSITSS 67709.2025.11295603 Veledar, O., Damjanovic-Behrendt, V., Macher, G.: Digital twins for dependability improvement of autonomous driving. In: Systems, Software and Services Process Improvement (EuroSPI 2019), pp. 415–426. Springer, Cham, Switzerland (2019). https://doi.org/10.1007/978-3-030-28005-5 32 Vasanth Rajendran, Dinesh Besiahgari, Sachin C. Patil, Manjunath Chandrashekaraiah, Vishnu Challagulla: A Multi-Agent LLM Environment for Software Design and Refactoring: A Conceptual Framework. doi: 10.1109/SoutheastCon56624.2025.10971563 (2025). https://doi.org/10.1109/SoutheastCon56624.2025 .10971563 Vishal Kharde: Nemesis: An Adversarial Multi-Agent AI Framework for Proactive Vulnerability Discovery and Patch Synthesis. doi: 10.1109/ICEI65890.2026.11448038 (2026). https://doi.org/10.1109/ICEI65890.2026.11448038 Virendra Ashiwal, Marcus Ritter, Sebastian Palacio, Nicolai Schoch: DriveAIAgent: A Multi-Agent System for Industrial Drive Commissioning and Troubleshooting. doi: 10.1109/ETFA65518.2025.11205685 (2025). https://doi.org/10.1109/ETFA65518. 2025.11205685 Vladimir Balditsyn, Philippe Lalanda, German Vega, Stéphanie Chollet: UncertaintyDriven Monitoring for ML-Based Autonomic Systems. doi: 10.1109/ACSOS66086.2025.00021 (2025). https://doi.org/10.1109/ACSOS66086.2025.0002 1 Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, C., Liu, S., Awadallah, A.H., White, R.W., Burger, D., Wang, C.: Autogen: Enabling next-gen LLM applications via multi-agent conversation. In: Proceedings of the First Conference on Language Modeling (COLM 2024). OpenReview.net, Online (2024). https://doi.org/10.48550/arXiv.2308.08155 . https://arxiv.org/abs/2308.08155 Wenbin Zhang, Haowen Chen, Yunfeng Wang, Wei Qiang: Multi-Level Protocol Consistency Verification Using Large Language Model Agents. doi: 10.1109/ISCIPT67144.2025.11265514 (2025). https://doi.org/10.1109/ISCIPT67144.2025.112 65514 Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen,

58

X., Lin, Y., Zhao, W.X., Wei, Z., Wen, J.-R.: A survey on large language model based autonomous agents. Frontiers of Computer Science 18(6), 186345 (2024) https: //doi.org/10.1007/s11704-024-40231-1 Wang, Y., Wang, X., Yao, Y., Li, X., Yang, X., Teng, Y., Ma, X., Wang, Y.: AgenticEval: Toward Agentic and Self-Evolving Safety Evaluation of Large Language Models (2025). https://arxiv.org/abs/2509.26100 W. Chen, Y. Su, J. Zuo, et al.: AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors. in Proc. ICLR, 2024, arXiv:2308.10848 (2024) Wang, S., Zhang, Y., Feng, D., Ge, Y., Zhou, M., Liu, Y., Zhang, N., Wang, H.: GSafeguard: A Topology-Guided Security Lens and Treatment on LLM-Based MultiAgent Systems. arXiv:2502.11127 (2025). https://doi.org/10.48550/arXiv.2502.11 127 Wentao Zhang, Zhe Zhao, Haibin Wen, Yingcheng Wu, Cankun Guo, Ming Yin, Bo An, Mengdi Wang: Autogenesis: A Self-Evolving Agent Protocol. arXiv:2604.15034v3 (2026). https://doi.org/10.48550/arXiv.2604.15034 Wang, K., Zeng, B., Wei, Z., Jin, C., Zhou, H., Li, X., Yang, C., Qu, J., Xu, X., Hu, X.: TrinityGuard: A Unified Framework for Safeguarding Multi-Agent Systems (2026). https://arxiv.org/abs/2603.15408 Xue Qiao, Jing Wang, Mengmeng Zhang, Fei Lin, Peijun Ye, Qinghua Ni, Fei-Yue Wang: Parallel Nursing: Enhancing Postoperative Nursing With LLM Agent Systems. doi: 10.1109/TCSS.2025.3605582 (2025). https://doi.org/10.1109/TCSS.202 5.3605582 Xiaodong Zheng, Siheng Zhao, Tianzhuo Shi, Tao Wang, Ruilin Chen, Shixuan Yu, Shuangsi Xue, Hui Cao: Agentic Planning for Power System Simulation Task Orchestrations Based on Multi-Agent Systems. doi: 10.1109/RCAE66389.2025.11355202 (2025). https://doi.org/10.1109/RCAE66389. 2025.11355202 Y. Yoldas, A. Onen, K. Alawasa, A. El Haffar, R. Ahshan, Md. R. Islam, S.M. Muyeen, N. Noorfatima, J. Jung: A Framework-Based Multi-Agent Coordination for Enhanced Microgrid Energy Management at the Secondary Control Layer. doi: 10.1109/TASC.2024.3468074 (2024). https://doi.org/10.1109/TASC.2024.3468074 Yehudai, A., Eden, L., Li, A., Uziel, G., Zhao, Y., Bar-Haim, R., Cohan, A., ShmueliScheuer, M.: Survey on Evaluation of LLM-Based Agents. arXiv:2503.16416 (2025). https://doi.org/10.48550/arXiv.2503.16416 Yu, M., Li, X., He, X., et al.: A survey on trustworthy LLM agents: Threats and countermeasures. In: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2025). ACM, New York, New York, USA (2025).

59

https://doi.org/10.1145/3711896.3736561 Yihao Zhang, Qizhi Qiu, Jiaping Wu, Xiaomin Liu, Weisheng Hu, Qunbi Zhuge: AutoONBench: a benchmark for large language model agents in autonomous optical networks. doi: 10.1364/JOCN.589201 (2025). https://doi.org/10.1364/JOCN.5 89201 Yasser Saeid, Thomas Kopinski: AgentFusion: A Multi-Agent Approach to Accurate Text Generation. doi: 10.1109/ICECER62944.2024.10920460 (2024). https://doi.or g/10.1109/ICECER62944.2024.10920460 Yash Tripathi, Vijayent Kohli, Shriya Misra: Confident but Incorrect: Mitigating Hallucination and Overconfidence in Agentic AI Coders. doi: 10.1109/ICIIP68302.2025.11346318 (2025). https://doi.org/10.1109/ICIIP68302.2 025.11346318 Zha, D., Bhat, Z.P., Lai, K.-H., Yang, F., Jiang, Z., Zhong, S., Hu, X.: Data-Centric Artificial Intelligence: A Survey. arXiv:2303.10158 (2023). https://doi.org/10.48550 /arXiv.2303.10158 Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E.P., Zhang, H., Gonzalez, J.E., Stoica, I.: Judging LLMas-a-Judge with MT-Bench and Chatbot Arena. arXiv:2306.05685, 2023, doi: 10.48550/arXiv.2306.05685 (2023). https://doi.org/10.48550/arXiv.2306.05685 Zhong, P.Y., Chen, S., Wang, R., McCall, M., Titzer, B.L., Miller, H., Gibbons, P.B.: RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage. arXiv:2502.08966 (2025). https://doi.org/10.48550/arXiv.2502.08966 Zhang, J.M., Harman, M., Ma, L., Liu, Y.: Machine learning testing: Survey, landscapes and horizons. IEEE Transactions on Software Engineering 48(1), 1–36 (2022) https://doi.org/10.1109/TSE.2019.2962027 Zou, H.P., Huang, W.-C., Wu, Y., Guo, J., Chen, Y., Miao, C., Nguyen, H., Zhou, Y., Zhang, W., Fang, L., Zhang, H., Wang, F., Zhang, P., Wang, H., He, L., Li, Y., Li, D., Jiang, R., Liu, X., Yu, P.S.: LLM-Based Human-Agent Collaboration and Interaction Systems: A Survey. arXiv:2505.00753 (2026). https://doi.org/10.48550 /arXiv.2505.00753 Zhongjun Yang, Haihong E, Zichen Tang, Jiacheng Liu, Yang Liu, Qianhe Zheng: AutoFlowgen: A Multi-Agent Framework for Automated Generation of Workflows from Natural Language. doi: 10.1109/CAIBDA65784.2025.11182955 (2025). https: //doi.org/10.1109/CAIBDA65784.2025.11182955 Zinkevich, M.: Rules of Machine Learning: Best Practices for ML Engineering. https: //developers.google.com/machine-learning/guides/rules-of-ml (2016). https: //developers.google.com/machine-learning/guides/rules-of-ml?hl=it

60

Zhenyu Mao, Jacky Keung, Fengji Zhang, Shuo Liu, Yifei Wang, Jialong Li: Towards Engineering Multi-Agent LLMs: A Protocol-Driven Approach. doi: 10.1109/APSEC66846.2025.00100 (2025). https://doi.org/10.1109/APSEC66846.2 025.00100 Zahra Aminiranjbar, Jianan Tang, Qiudan Wang, Shubha Pant, Mahesh Viswanathan: DAWN: Designing Distributed Agents in a Worldwide Network. doi: 10.1109/ACCESS.2025.3588425 (2025). https://doi.org/10.1109/ACCESS.2025.358842 5 Zhi Chen, Lingxiao Jiang: Evaluating Software Development Agents: Patch Patterns, Code Quality, and Issue Complexity in Real-World GitHub Scenarios. doi: 10.1109/SANER64311.2025.00068 (2025). https://doi.org/10.1109/SANER64311.2 025.00068 Zhao, L., Liu, S., Xin, T., et al.: Ai agent in healthcare: Applications, evaluations, and future directions. npj Artificial Intelligence 2, 31 (2026) https://doi.org/10.1038/s4 4387-026-00076-4 Zhengshu Zhou, Yutaka Matsubara, Hiroaki Takada: Quantitative Security Assurance Case for In-vehicle Embedded Systems. doi: 10.1109/DASC-PICom-CBDComCyberSciTech52372.2021.00022 (2021). https://doi.org/10.1109/DASC-PICom-C BDCom-CyberSciTech52372.2021.00022 Zhenfeng Su, Zhiming Zhou, Jiaosai Li, Ren Wen, Zhen Liu: TAAE: Team-Aware Attention Extraction for Generalized Agent Coordination in Multi-Agent Reinforcement Learning. doi: 10.1109/IJCNN64981.2025.11227327 (2025). https://doi.org/ 10.1109/IJCNN64981.2025.11227327 Zhu, K., Zhang, J., Xiao, Y., Liu, X., Song, K., Tan, C., Zeng, B., Wang, J., Xing, E.P.: Multiagentbench: Evaluating the collaboration and competition of LLM agents. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 8580–8622. Association for Computational Linguistics, Online (2025). https://doi.org/10.18653/v1/2025.acl-long.421

61