Dissecting Agentic Forensics: The Role of Triage, Prompting, and Evidence Arbitration in Open-World Fake Image Detection
arXiv:2609.24359v1 [cs.CV] 21 Sep 2026
Xianlong Li2 , Pietro Bongini1 , Niccoló Pancino1 , Marco Blanchini2 , Benedetta Tondi1 , and Mauro Barni1 University of Siena, Italy IMT School for Advanced Studies Lucca, Italy 1
2
Abstract. Image forensics is increasingly an open-world problem: manipulations range from fully synthetic images to localized edits, splicing and swapping, while most forensic detectors remain specialized to a single manipulation family. Agentic AI has recently emerged as a promising solution. In principle, such systems can assess the reliability of individual detectors, identify out-of-scope evidence, and arbitrate conflicting reports. However, it remains unclear which components actually drive performance and whether their benefits persist under distribution shift. To answer these questions, we study a training-free agentic framework built around specialist detectors, per-detector triage, and conflict-aware evidence arbitration. Using six configurations and three multimodal large language model backbones, we dissect the role of triage, prompting, and reasoning quality on both in-distribution and out-of-distribution data. Our results show that naive detector fusion suffers from severe falsepositive rates on authentic images. Triage and prompting consistently improve performance by filtering unreliable evidence and exposing detector limitations. However, the dominant factor is represented by reasoning itself: A stronger judge substantially outperforms a weaker one, particularly under distribution shift. Most notably, manipulation recall is nearly saturated across all configurations, indicating that the main challenge of open-world image forensics is not detecting manipulations, but calibrating trust in specialized forensic tools and arbitrating conflicting evidence. Keywords: Image Forensics · Agentic AI · Multi-Agent Systems
Accepted at the 2026 Workshop on AI for Multimedia Forensics and Disinformation Detection (AI4MFDD), ECCV 2026. This is the authors’ version of the contribution.
2
1
Li et al.
Introduction
Advances in generative modeling have made image manipulation more convincing, cheaper, and more pervasive, while broadening the forms it takes: an image may be synthesized in its entirety, edited locally by diffusion-based methods [17] or generative inpainting, assembled by splicing content from other sources, or altered by face swapping, among other techniques. Determining whether, and how, an image has been manipulated is therefore central to media authenticity and information integrity [21]. The difficulty is that image forensics is inherently an open-world problem: in realistic scenarios, a forensic system must accept an arbitrary image and remain reliable whichever of these manipulations, if any, it has undergone. Most forensic detectors are specialists: each targets one manipulation family and is accurate within it, but its behaviour outside that family is neither defined nor dependable, and such detectors are known not to generalise across generators or manipulation families [15,22]. The natural response - to run several specialists and fuse their outputs, an idea with a long history in forensic decision fusion [8] - breaks down precisely when they disagree. A detector applied to content it was never designed to judge may report a manipulation with high confidence, yielding a false positive on an authentic image, or stay silent on a manipulation beyond its scope, yielding a false negative. A flat aggregation of scores has no principled basis for deciding which detector to believe. Resolving such conflicts is therefore a matter of judgement rather than arithmetic: it requires assessing whether a detector is applicable to the image at hand and weighing its evidence against competing reports. Agentic approaches have recently applied multimodal large language models (MLLMs) to this judgement [14, 30]. However, they are typically developed and evaluated as monolithic pipelines. As a result, it remains unclear which components drive performance, how much of the observed gains stem from the underlying model rather than the pipeline design, and whether these gains persist under distribution shift. These questions are the subject of this paper. To study these questions, we adopt a deliberately simple agentic framework whose components can be isolated and evaluated independently. The design is guided by two observations. First, specialist detectors are frequently applied outside their intended scope, suggesting the need for an explicit triage stage that assesses whether a detector’s output should be considered evidence at all. Second, detectors may produce false positives, false negatives, and mutually conflicting reports, suggesting the need for a judge that reasons explicitly about detector limitations rather than blindly aggregating their outputs. The resulting framework requires no training of its own. A fixed pool of pretrained specialist detectors is applied to an input image; a per-detector auditor agent triages each output, deciding whether it should be admitted as evidence; and a conflict-aware judge arbitrates the retained evidence into a final verdict—a binary decision together with an attribution of the evidence on which that decision rests (Fig. 1).
Dissecting Agentic Forensics
3
We instantiate the framework with seven publicly available detectors spanning five manipulation families. The detector pool is extensible: supporting a new manipulation family requires only registering an additional specialist detector, with no retraining. We then use this framework as an experimental testbed to study three aspects of agentic forensic reasoning: evidence triage, explicit awareness of detector limitations, and conflict-aware arbitration. Through a controlled set of ablations, we isolate the contribution of each component and evaluate their interaction with different MLLM backbones. Because open-world forensic systems must operate on previously unseen content, all experiments are performed both in-distribution and out-of-distribution. This allows us to assess not only overall performance, but also the robustness of the different reasoning components under distribution shift. Our experiments reveal that the benefits of agentic forensics do not stem equally from all components. Evidence triage and explicit awareness of detector limitations provide consistent improvements by filtering unreliable evidence and exposing detector failure modes. However, these benefits can only be fully exploited by sufficiently capable reasoning models, whose quality ultimately dominates performance. We further find that manipulation recall is already close to saturation, and that performance differences arise primarily from the ability to reject false alarms on authentic images. These findings suggest that the central challenge of open-world image forensics is not manipulation detection per se, but the calibration of trust in specialized forensic tools. With the above ideas in mind, the contributions of the paper can be summarised as follows: – A simple, training-free agentic framework for open-world image forensics, designed as an experimental testbed for combining specialist detectors through evidence triage and arbitration. – A controlled dissection of the role of triage, prompting, and evidence arbitration across multiple MLLM backbones under both in-distribution and out-of-distribution conditions. – An empirical study of the factors driving performance in agentic image forensics. – A simple and measurable form of evidence attribution, identifying the detector deemed decisive by the arbitration process in addition to the final binary verdict.
2
Related Work
Our work builds on three principal lines of research, which we review in turn: forensic detectors specialised to a single manipulation family, approaches that jointly address multiple manipulation families, and the emerging integration of agentic reasoning with forensic tools, to which our method belongs. Single-Task Forensic Detection. The majority of forensic detectors are designed for a single manipulation family [21]. Whole-image generation detectors
4
Li et al.
determine whether an image is entirely synthetic, relying on generator-specific fingerprints [22], CLIP-based representations [3,15], cross-generator inconsistencies [28], or feature-subspace decompositions [27]. Face-manipulation detectors concentrate on a single tampered face and its associated blending artefacts, and are typically evaluated on the FaceForensics++ benchmark [18]. Localised-edit detectors, in contrast, produce pixel-level tampering maps for operations such as splicing, copy-move, and inpainting [10], with recent methods further specialising in diffusion-based inpainting [1, 2]. Each detector is thus calibrated to a specific manipulation family and data distribution, and its performance tends to deteriorate when applied to images subjected to other manipulations. Multi-Task Forensic Detection. Since the manipulation family is typically unknown a priori, a second major research direction addresses multiple manipulation families simultaneously. Classical approaches perform decision-level fusion over a set of specialised forensic tools, combining their outputs into a single verdict and weighing each according to its estimated reliability under the observed acquisition and processing conditions [7]; decision fusion based on Dempster– Shafer evidence theory constitutes a canonical formulation of this paradigm [8]. Deep learning approaches instead train a single network to jointly detect and localise multiple manipulation families without family-specific supervision [5, 25]. More recent work extends this with multimodal architectures that additionally produce a natural-language rationale [12, 26]. Although these approaches increase the coverage of manipulation families, the fusion rules in classical systems are manually specified, and the learned multimodal models are trained on, and therefore constrained by, a fixed and predefined manipulation taxonomy. Agentic Forensics. A third, more recent line of research endows the model with agency: rather than emitting a verdict in a single forward pass, the model iteratively reasons and invokes external tools [19]. This direction builds on agentic reasoning frameworks such as ReAct, which interleave explicit reasoning steps with tool calls [29], as well as on methods that enhance the robustness and reliability of tool usage via self-critique and inference-time feedback mechanisms [9,20]. In the forensic domain, AIFo casts AI-generated image detection as a multi-agent reasoning problem over provenance information, metadata, classifier outputs, and vision–language cues [14]; EvoGuard trains an agent via reinforcement learning to dynamically select among a set of off-the-shelf detectors [31]; and ForenAgent employs a multimodal model that iteratively invokes hand-crafted low-level forensic cues under reinforcement fine-tuning [30]. These systems are most closely related to our work, but they are all designed to answer a single binary decision problem—authentic versus manipulated—for a specific manipulation task, rather than to identify and localise the manipulation family in an open-world setting. They rely on task-specific training or explicitly engineered low-level cues. By contrast, our framework orchestrates a pool of pre-trained, state-of-the-art single-task detectors spanning multiple manipulation families without any additional training, identifies the manipulation’s family and region rather than only
Dissecting Agentic Forensics
5
its presence, and treats the scope conflicts and out-of-competence errors that combining specialists entails as first-class problems.
3
Method
3.1
Problem Setting and Overview
We address open-world image forensics. Given an image x of unknown provenance, the system returns a verdict (y, m): a binary authenticity label y ∈ {0, 1} (authentic or manipulated) and, when y = 1, the manipulation family m ∈ M responsible for the manipulation. We place no prior on the family, and we assume that a manipulated image belongs to a single family of M. Composite manipulations are beyond the scope of this paper. Covering M requires a pool of targeted detectors, each specialised to one manipulation family. Combining them raises the two problems set out in Sec. 1, both absent in the single-family case. First, a detector may be applied outside its domain of competence: when confronted with content that its design does not encompass, it may return an uninformative score, either producing a false positive on an authentic image or failing to flag a genuine manipulation. Second, because at most one family can correctly account for x, detectors targeting different manipulation families may yield conflicting evidence: several detectors can simultaneously report a manipulation even though they cannot all be correct. These two problems motivate the two principal components around which our framework is organised. Triage addresses the first: before a detector’s output is admitted as evidence, an agent first assesses whether that detector is applicable to the image under analysis. Arbitration addresses the second: a single agent aggregates the admitted, potentially conflicting reports and decides which of them, if any, to regard as reliable. Orthogonal to both is a third, equally important design dimension, prompting: what the agents are explicitly told about each detector’s domain of competence and about the possibility of conflict, rather than leaving such information implicit. We treat triage, prompting, and arbitration as the three central design choices to be analysed empirically (Sec. 4). Figure 1 shows the pipeline of our agentic framework, which requires no training of its own and proceeds in three stages. In Stage 0, the full pool of detectors is run on x; each returns a manipulation score, and the localisation detectors additionally return a tampering map. In Stage 1, a dedicated auditor agent triages each detector independently, discarding any whose output is not credible for the image at hand: for instance, a face manipulation detector applied to an image with no visible face, or a tampering map scattered over natural texture rather than concentrated on a plausible edit. In Stage 2, a single judge agent integrates the output of the retained detectors, resolves possible conflicts, and emits the verdict (y, m) including the indication of the detector whose evidence it deems decisive. The agents are MLLMs prompted to read the available evidence and produce these judgements. We describe the detector pool in Sec. 3.2 and the agents in Sec. 3.3.
6
Li et al. Stage 0: detector pool Whole-image generation
Stage 1 per-detector triage
Cozzolino et al . [3]
Æ
D3 [28]
Æ
Face manipulation Input image x
run on all
Effort [27]
Æ
ForensicsAdapter [4]
Æ
of unknown provenance
Local edits RADAR [2] diffusion inpainting
NFA-ViT [1] diffusion / GAN inpainting
TruFor [10] inpainting, splicing / copy-move
retained only
Æ
Æ Stage 2 conflictaware judge
Verdict y: authentic or manipulated m: manipulation family decisive detector
Æ Æ
Fig. 1: Our framework as a three-stage pipeline. Stage 0 runs the full pool of pretrained detectors on the input image, grouped by manipulation family: whole-image generation, face manipulation, and local edits, with each local-edit detector annotated by the manipulations it targets. Stage 1 audits each detector with its own agent (one auditor per detector), which keeps or discards that detector’s output; only the retained detectors are passed on. Stage 2 arbitrates them with a single conflict-aware judge, returning the verdict (y, m), where m is the family of the decisive detector. Splicing and copy-move are treated as a single family because TruFor is the only pooled detector covering them; the localisation detectors (RADAR, NFA-ViT, TruFor) additionally return a tampering map. Blue: automated steps; green: MLLM agents.
3.2
Detector Pool and Tool Cards
We instantiate the pool of detectors with seven publicly available methods whose combined competence spans the five families of M: whole-image generation, face manipulation, diffusion inpainting, GAN inpainting, and splicing/copy-move (Tab. 1). Each detector is treated as a black box that emits a scalar indication of manipulation and, for the three localisation detectors, a dense tampering map. Crucially, each detector is registered together with a tool card : a short, externally authored description that is the agents’ only knowledge of the detector. A card declares the detector’s scope (the manipulation families it targets), its decision threshold (fixed at 0.5 for every detector in our pool), and how to read its output: the score and, for the localisation detectors, the tampering heatmap (Tab. 2). It is therefore the card that lets an agent recognise a score as out of scope, and so uninformative, rather than as evidence. Exposing each detector’s competence in this way is a deliberate design choice rather than an implementation detail, and its effect is one of the objective of our study (Sec. 4). Table 2 shows one card in full; the remaining six are given as supplementary material. The detectors’ native outputs are heterogeneous and not directly comparable, so a per-detector calibration maps each score onto a common range s ∈ [0, 1], with a shared threshold equal to 0.5 separating a detector that reports a manipulation from a silent one. Because a detector’s family membership and output semantics reside entirely in its tool card rather than in the surrounding code, extending the pool to a new family reduces to registering one further detector and its card.
Dissecting Agentic Forensics
7
Table 1: The seven pre-trained detectors and their declared scope (the manipulation family or families each targets). Following our setting, splicing and copy-move form a single family, since TruFor is the only pooled detector covering them. Localisation detectors additionally produce a dense tampering map. Detector
Declared scope
Output
Cozzolino et al . [3] Whole-image generation score D3 [28] Whole-image generation score Effort [27] Face manipulation score ForensicsAdapter [4] Face manipulation score RADAR [2] Diffusion inpainting score + map NFA-ViT [1] Diffusion / GAN inpainting score + map TruFor [10] Diffusion / GAN inpainting, splicing/copy-move score + map
3.3
Agentic Triage and Arbitration
Triage: Per-Detector Auditing. Stage 1 addresses the first problem outlined before: out-of-competence detection. For every detector that produced an output, a dedicated auditor agent decides whether that output is credible evidence on the image at hand, and excludes it from all downstream processing if it is not. The audit turns on two questions: whether the image actually contains the kind of content the detector targets (a scope check, taken as a hard prerequisite), and, if so, whether the detector’s signal is internally coherent, meaning a score clearly separated from the common 0.5 decision threshold (Sec. 3.2): a score resting on this boundary leaves the detection undecided and hence uninformative, and, for localisation detectors, a map concentrated on a plausible edit rather than dispersed over natural texture. We assign one narrowly scoped agent to each detector, rather than reading the detector’s scalar directly, because deciding whether that scalar is credible for this particular image demands contextual judgement the detector cannot provide. Concretely, each auditor is an MLLM that reads the image, the detector’s score and map, and its tool card, and returns a verdict (keep or discard) together with a brief written justification. Assigning one agent per detector follows a standard multi-agent design, in which a task is decomposed into narrowly scoped, single-purpose agents rather than handled by one monolithic agent [11, 24]. The opposite choice, folding the audit into the judge so that a single agent both audits and arbitrates, is one of the comparison configurations consider in our experiments (Sec. 4). Arbitration: The Conflict-Aware Judge. Stage 2 addresses the second problem: conflicting evidence. A single judge agent receives, for each retained detector, its score, the output map (if any), and a tool card together with that detector’s auditor verdict and justification. The judge also has access to the analyzed image itself. From these data, the judge produces the verdict (y, m). Because at most one family can explain the image, detectors of differing scope that simultaneously report a manipulation cannot all be correct; the judge must decide
8
Li et al.
Table 2: The tool card of TruFor, shown to make concrete what every card declares: the detector’s scope and the attack classes it covers, its decision threshold, how to read its score, and how to read its heatmap. All seven cards share this structure. Field
Content (TruFor)
Local-manipulation localisation: splicing, copy-move, and inpainting (including diffusion-based); produces an image-level score and a pixel-level anomaly map. attack_classes inpaint_diffusion, inpaint_gan_traditional, splice_copymove threshold integrity_score in [0, 1]; decision boundary at 0.5 (higher means more likely tampered). signal_interp. A higher score means more likely tampered; a low score does not by itself rule out manipulation if the anomaly map shows a concentrated hot region at a plausible tamper location. heatmap_desc Two-panel figure: left, the original with the anomaly map overlaid; right, a pure jet heatmap (red = likely manipulated, blue = pristine). scope_in
which one, if any, reflects a genuine manipulation. That detector is the framework’s evidence attribution: the family m is read directly off its declared scope, so naming the decisive detector and naming m are equivalent operations. We make this attribution a primary output, not a by-product of the binary decision. A detector applied beyond its competence frequently reports a manipulation on out-of-distribution content, so a framework can return the correct binary label while crediting the wrong detector - a verdict that is right only by accident and whose underlying explanation is unsound. For this reason, in Sec. 4, we also verify that the decisive detector be appropriate for the image at hand, in addition to checking if y is correct or not. The manner in which the judge is prompted is fundamental to its behaviour. The prompt does not request a verdict directly; instead, it specifies an explicit, stepwise protocol: first, inspect the image directly; next, enumerate the crossscope conflicts among detectors that report a manipulation; then, evaluate each report in light of both the image content and the detector’s stated domain of competence; and only thereafter output a judgment. The judge’s reasoning relies critically on two forms of guidance encoded in this prompt. The first concerns conflict: since at most one family can explain the image, retained detectors whose declared scopes differ cannot jointly detect a true manipulation, and the prompt instructs the judge to identify such cross-scope disagreements and resolve them rather than average or otherwise smooth them away. The second concerns reliability: a detector remains an imperfect classifier, so a positive report may reflect either a genuine detection or a false alarm. Examples include near-threshold scores, tampering maps that highlight natural texture rather than plausible edits, or isolated positive reports unsupported by other detectors targeting the same manipulation family. The prompt therefore instructs the judge to recon-
Dissecting Agentic Forensics
9
cile each detector report with its own interpretation of the image, weighing the credibility of the evidence before reaching a final decision. Both are deliberate prompting design choices; removing them, so that the judge aggregates the same retained detector outputs without explicit awareness of conflicts or of residual false-positive risk, is the basis for the prompting ablation study described in Sec. 4. The complete protocol and all prompts are provided verbatim in the supplementary material.
4
Evaluation Design and Results
4.1
Datasets
All configurations (Sec. 4.3) are evaluated on two data collections, each covering the five manipulation families forming M alongside authentic imagery. Each collection is drawn from a larger annotated pool (7,000 images for the in-distribution collection, 7,496 for the out-of-distribution), by including a fixed number of images per manipulation family. In this way, we built an in-distribution evaluation set of 240 images (135 manipulated, 105 authentic) and an out-ofdistribution set of 200 images (100 manipulated - 20 per family - and 100 authentic). The in-distribution collection is organised by detector: for each of the seven pooled detectors, we sample authentic images and images of the manipulation family targeted by that detector from the benchmark on which the detector was originally evaluated, yielding seven detector-keyed groups. Crucially, the full pool of seven detectors is run on every input image: for an image drawn from detector d’s evaluation benchmark, detector d - and any other detector covering the same manipulation family - operates within its intended regime, while the remaining detectors are applied out of scope, generating the cross-scope conflicts the framework is designed to resolve. The out-of-distribution collection comprises images drawn from datasets lying outside the training and evaluation data of every pooled detector. Face manipulations are taken from Celeb-DF-v2 [13]; whole-image generations from Community Forensics [16], which spans a large and diverse set of recent generators; diffusion inpaintings from Adobe Firefly edits; GAN inpaintings from the six GAN inpainters of the IID-Net dataset [23]; and splicing and copy-move forgeries from CASIA v2.0 [6]. Authentic images are drawn from the corresponding source corpora - Celeb-DF-v2 source videos, Community Forensics reals, the originals underlying the Firefly edits, and CASIA v2.0 authentic images - so that, within each manipulation family, genuine and manipulated content share the same data domain. Consequently, none of the detectors can rely on prior exposure to the test distribution, and the framework is assessed on its ability to arbitrate among detectors that are themselves operating out of distribution, closely reflecting the conditions encountered in real-world deployment. 4.2
Experimental Setup and Metrics
Each agentic configuration (Sec. 4.3) is driven by an MLLM backbone and is evaluated on both the in-distribution and out-of-distribution collections of Sec. 4.1.
10
Li et al.
Table 3: The six variants of the agentic frameworks tested in our experiments. The first four form an ablation ladder on the Gemma-4-26B backbone; the last two vary the model. ✓/✗ indicate whether each component is present.
Variant Algorithmic fusion Single-agent Unguided judge Complete framework Complete framework Complete framework
MLLM Per-detector Judge agent auditors guidance Backbone ✗ ✓ ✓ ✓ ✓ ✓
✗ ✗ ✓ ✓ ✓ ✓
✗ ✗ ✗ ✓ ✓ ✓
none Gemma-4-26B Gemma-4-26B Gemma-4-26B GPT-4o GPT-5 (judge only)
The decision threshold of 0.5 applied to each detector’s normalised score is held fixed across all configurations - it is the natural midpoint of the [0, 1] score range and keeps the detector interface constant so that accuracy differences isolate the agentic components under study rather than reflect threshold calibration. We report two metrics: binary accuracy is the proportion of images correctly judged manipulated or authentic; tool-arbitration accuracy examines, for the images the framework judges manipulated, whether the detector identified by the judge as the decisive detector is appropriate for the analyzed image. The second metric is necessary because a high binary accuracy carries little weight if the framework reaches it for the wrong reason - attributing a manipulation to a detector whose competence does not cover the actual manipulation - and we therefore regard a verdict as fully correct only when both the binary decision and the arbitration are correct. Quantifying arbitration correctness requires a ground-truth notion of which detector is appropriate for a given image. Every test image is drawn from a source containing a single, known manipulation, so its true manipulation family is fixed by construction, without per-image annotation; each detector, in turn, declares the family or families that constitute its scope (Tab. 1). We count the arbitrated detector as correct when its declared scope contains the image’s true manipulation family (an in-scope decision). A manipulated image erroneously judged authentic yields no arbitration and counts against the metric. 4.3
Framework Variants
We compare six configurations (Tab. 3), each run on both data collections, for a total of twelve runs. The first four experiments constitute a progressive ablation study on the Gemma-4-26B backbone3 , allowing us to quantify the individual contributions of triage, prompting, and evidence arbitration. 3
Google DeepMind Gemma 4 26B A4B: a sparse mixture-of-experts model with 25.2B total parameters and ∼3.8B active per forward pass, released April 2026. https: //huggingface.co/google/gemma-4-26B-A4B-it
Dissecting Agentic Forensics
11
Table 4: Results (%), in and out of distribution. Manip. is recall over manipulated images, Pristine is specificity on authentic images, Overall is binary accuracy, and Arb. is in-scope tool-arbitration accuracy. Best Overall per regime in bold. In-distribution
Out-of-distribution
Method
Manip. Pristine Overall Arb. Manip. Pristine Overall Arb.
Algorithmic fusion Single-agent (Gemma-4-26B) Unguided judge (Gemma-4-26B) Complete framework (GPT-4o) Complete framework (Gemma-4-26B) Complete framework (GPT-5 judge)
100.0 98.5 99.3 97.8 97.0 90.4
9.5 49.5 50.5 31.4 61.0 86.7
60.4 77.1 77.9 68.8 81.2 88.8
71.1 65.9 71.1 64.4 68.2 68.9
100.0 96.0 98.0 99.0 95.0 86.0
7.0 37.0 39.0 28.0 47.0 82.0
53.5 66.5 68.5 63.5 71.0 84.0
61.0 70.0 77.0 63.0 74.0 72.0
Algorithmic fusion is the no-agent floor: it groups detectors by declared manipulation scope, averages their normalised scores within each scope, and declares the image manipulated in the scope with the highest average among those with at least one detector above threshold. A single agent reinstates model reasoning but audits and arbitrates in one call, collapsing triage and arbitration; the unguided judge restores per-detector triage but omits the prompting axis; and the complete framework adds it. The remaining two re-instantiate the complete framework with a different backbone - GPT-4o throughout, and GPT-5 as the judge with the auditors left on Gemma-4-26B. All six share the same detector pool and score normalisation. 4.4
Results
Table 4 summarises the results of all twelve experimental configurations, while Figs. 2 and 3 provide a visual overview. Since the relative trends are largely consistent across the two evaluation protocols, we focus the discussion on the out-of-distribution results, referring to the in-distribution setting only where differences are noteworthy. The discussion is organised around the four questions posed at the outset: the contributions of triage, prompting, and reasoning quality, together with the behaviour under distribution shift. For each, we consider two complementary perspectives: specificity on authentic images, and performance on manipulated images, assessed in terms of both manipulation recall and toolarbitration accuracy. The Role of Triage. We isolate the contribution of per-detector auditors by comparing the single-agent configuration, which performs auditing and arbitration in a single pass, with the complete framework instantiated on the same Gemma backbone. The impact of triage is clear and unidirectional. Specificity on authentic images increases substantially, by approximately 10%, while manipulation recall remains effectively unchanged and tool-arbitration accuracy improves by a further 3–5%. Triage thus yields a net benefit: it suppresses false positives on authentic content and sharpens detector attribution, with no meaningful loss in manipulation detection.
12
Li et al.
Fig. 2: Binary accuracy (left) and tool-arbitration accuracy (right) for the six configurations (Tab. 3), in-distribution (solid) and out-of-distribution (hatched). Binary accuracy is the overall rate of correct authentic/manipulated verdicts; tool-arbitration accuracy is the in-scope detector-selection rate on images judged manipulated (Sec. 4.2).
The Role of Prompting. We isolate the effect of the judge’s guidance by comparing the unguided judge with the complete framework, again using Gemma. Here the improvement comes with a mild trade-off: specificity on authentic images increases by roughly 8%, whereas manipulation recall and tool-arbitration accuracy each decrease slightly, by about 2–4%. The guidance makes the judge mildly sceptical of the detectors’ positive detector outputs, recovering additional authentic images at a modest cost on manipulated ones; the net effect on overall accuracy is nevertheless clearly favourable. The Role of Reasoning Quality. Reasoning quality is the dominant factor, as evidenced by the following three comparisons. Reasoning versus none. Relative to algorithmic fusion, the complete framework with Gemma delivers a substantial improvement on out of distribution experiments: specificity on authentic images increases from 7% to 47%, and tool-arbitration accuracy from 61% to 74%, at the expense of only a few percentage points of recall. Even a moderately capable reasoning agent is therefore what makes the detector pool practically usable. Backbone capability. The capability of the reasoning backbone is critical. The open-source Gemma 4 is more recent and empirically stronger than GPT-4o. The Gemma-based configuration outperforms the GPT-4o one on every metric: specificity (47% vs. 28%), overall accuracy (71% vs. 63.5%), and arbitration (74% vs. 63%), with manipulation recall already saturated in both cases. A more capable backbone therefore translates directly into a more accurate framework. Upgrading only the judge. Motivated by this, we upgrade only the judge - the most demanding component of the pipeline - to GPT-5, while retaining Gemma as the auditor backbone. Even this partial change is decisive: specificity on authentic images increases from 47% to 82%, manipulation recall decreases only slightly, and arbitration accuracy remains essentially unchanged. Overall accuracy rises to 84%, the highest among all configurations.
Dissecting Agentic Forensics
13
Fig. 3: Specificity on authentic images (left) and recall on manipulated images (right), in-distribution (solid) and out-of-distribution (hatched). These are the two class-wise components of the binary accuracy of Fig. 2: on the balanced sets their average is the binary accuracy. Manipulation recall is near-saturated everywhere, so the variation in binary accuracy—and the room for improvement—lies almost entirely in specificity on authentic images.
Robustness to Distribution Shift. Distribution shift does not affect all aspects of the system uniformly. The degradation is largely confined to binary accuracy and, within it, to its two class-wise components: specificity decreases appreciably, whereas recall, already near saturation, changes only marginally. The main difficulty out of distribution thus lies in recognising unfamiliar authentic content rather than in detecting manipulations. By contrast, tool-arbitration accuracy does not deteriorate under distribution shift; for all agentic configurations it remains stable or improves slightly. The GPT-5 judge is the most robust overall, with accuracy decreasing only from 88.8% to 84.0%. Algorithmic fusion is the only clear outlier. Its tool-arbitration accuracy drops from 71.1% for in-distribution to 61.0% for out-of-distribution images, despite achieving the highest in-distribution accuracy among all configurations. This behaviour stems directly from the selection rule, which assigns each manipulation to the family with the highest average normalised score. In the in-distribution setting, every image is drawn from the benchmark on which its corresponding detector was originally evaluated, allowing that detector to consistently achieve the highest score and be correctly selected. Under distribution shift, however, this assumption no longer holds, leading to substantially more frequent misattributions. In contrast, the agentic configurations do not rely on such a regularity, which explains why their tool-arbitration accuracy remains remarkably stable across the two evaluation settings. 4.5
Takeaways
Four observations emerge from our results. First, every component of the agentic layer contributes positively to performance. Both per-detector auditing and conflict-aware arbitration yield con-
14
Li et al.
sistent, albeit modest, gains, indicating that filtering unreliable evidence and reasoning over detector disagreements are beneficial in their own right. Second, the quality of the reasoning model exerts a larger impact than any individual architectural component. Stronger judges do not replace triage or prompting redundant; rather, they exploit the additional structure and contextual information these mechanisms provide more effectively. Their advantage is most evident on authentic images, where they recover many cases that weaker backbones incorrectly classify as manipulated. Third, the improvements introduced by the agentic layer arise primarily from a better handling of authentic content. Across configurations, gains in overall accuracy are consistently associated with substantial reductions in false positives, while changes in manipulation recall and tool-arbitration accuracy remain comparatively minor. Fourth, manipulation recall is uniformly high, exceeding 85% in all configurations and approaching saturation in most. Consequently, almost all variability in overall performance originates from the authentic class. This suggests that the central challenge of open-world image forensics is not the detection of manipulations per se, but the calibration of trust in forensic evidence and the reliable recognition of authentic content.
5
Conclusion
We have investigated open-world image forensics through a deliberately simple, training-free agentic framework that orchestrates a pool of pre-trained specialist detectors via per-detector triage and conflict-aware arbitration, and employed this framework as a testbed to identify the primary drivers of performance. Our dissection yields a single overarching conclusion: in the open world, the main difficulty does not lie in identifying that an image has been manipulated - a task for which the detectors already achieve near-saturated recall - but in determining when a detector’s output should be trusted. Nearly all variation in accuracy arises on the authentic class, where detectors operating outside their domain of competence generate spurious alarms. The agentic layer attains its accuracy precisely by improving this trust decision: triage filters out detectors operating out of scope, and conflict-aware prompting evaluates the remaining evidence in relation to the image and each detector’s declared domain. The quality of this decision, however, depends more on the reasoning capability of the model than on any particular architectural refinement: a stronger, better-calibrated judge improves specificity on authentic content far more than any architectural change, and upgrading the judge alone already suffices to obtain most of the benefit. We therefore anticipate that further progress in agentic forensics will stem less from increasingly elaborate pipelines than from stronger reasoning capabilities and more accurately calibrated trust in individual detectors. At the same time, the framework’s training-free and extensible design allows it to track the evolution of manipulation techniques by incorporating new detectors without additional training cost.
Dissecting Agentic Forensics
15
Relaxing the single-manipulation assumption to encompass composite manipulations, and replacing purely per-case reasoning with trust estimates calibrated from accumulated experience, constitute natural directions for future work.
References 1. Cai, L., Wang, H., Ji, J., ZhouMen, Y., Chen, S., Yao, T., Sun, X.: Zooming in on fakes: A novel dataset for localized ai-generated image detection with forgery amplification approach. In: Fortieth AAAI Conference on Artificial Intelligence, AAAI 2026, Singapore, January 20-27, 2026. pp. 2534–2542. AAAI Press (2026). https://doi.org/10.1609/AAAI.V40I4.37240 2. Costanzino, A., Bayliss, W., Sock, J., Blanch, M.G., Horak, D., Laptev, I., Torr, P., Pizzati, F.: Towards reliable identification of diffusion-based image manipulations. CoRR abs/2506.05466 (2025). https://doi.org/10.48550/ARXIV.2506.05466 3. Cozzolino, D., Poggi, G., Corvi, R., Nießner, M., Verdoliva, L.: Raising the bar of ai-generated image detection with CLIP. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024 - Workshops, Seattle, WA, USA, June 17-18, 2024. pp. 4356–4366. IEEE (2024). https://doi.org/10.1109/ CVPRW63382.2024.00439 4. Cui, X., Li, Y., Luo, A., Zhou, J., Dong, J.: Forensics adapter: Adapting CLIP for generalizable face forgery detection. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN, USA, June 11-15, 2025. pp. 19207–19217. Computer Vision Foundation / IEEE (2025). https:// doi.org/10.1109/CVPR52734.2025.01789 5. Dong, C., Chen, X., Hu, R., Cao, J., Li, X.: Mvss-net: Multi-view multi-scale supervised networks for image manipulation detection. IEEE Trans. Pattern Anal. Mach. Intell. 45(3), 3539–3553 (2023). https://doi.org/10.1109/TPAMI.2022. 3180556, https://doi.org/10.1109/TPAMI.2022.3180556 6. Dong, J., Wang, W., Tan, T.: CASIA image tampering detection evaluation database. In: 2013 IEEE China Summit and International Conference on Signal and Information Processing, ChinaSIP 2013, Beijing, China, July 6-10, 2013. pp. 422–426. IEEE (2013). https://doi.org/10.1109/CHINASIP.2013.6625374 7. Fontani, M., Argones-Rúa, E., Troncoso, C., Barni, M.: The watchful forensic analyst: Multi-clue information fusion with background knowledge. In: 2013 IEEE International Workshop on Information Forensics and Security, WIFS 2013, Guangzhou, China, November 18-21, 2013. pp. 120–125. IEEE (2013). https: //doi.org/10.1109/WIFS.2013.6707805 8. Fontani, M., Bianchi, T., Rosa, A.D., Piva, A., Barni, M.: A framework for decision fusion in image forensics based on dempster-shafer theory of evidence. IEEE Trans. Inf. Forensics Secur. 8(4), 593–607 (2013). https://doi.org/10.1109/TIFS.2013. 2248727 9. Gou, Z., Shao, Z., Gong, Y., Shen, Y., Yang, Y., Duan, N., Chen, W.: CRITIC: large language models can self-correct with tool-interactive critiquing. In: The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net (2024) 10. Guillaro, F., Cozzolino, D., Sud, A., Dufour, N., Verdoliva, L.: Trufor: Leveraging all-round clues for trustworthy image forgery detection and localization. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023. pp. 20606–20615. IEEE (2023). https://doi.org/10.1109/CVPR52729.2023.01974
16
Li et al.
11. Hong, S., Zhuge, M., Chen, J., Zheng, X., Cheng, Y., Wang, J., Zhang, C., Wang, Z., Yau, S.K.S., Lin, Z., Zhou, L., Ran, C., Xiao, L., Wu, C., Schmidhuber, J.: Metagpt: Meta programming for A multi-agent collaborative framework. In: The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net (2024) 12. Li, J., Zhang, F., Zhu, J., Sun, E., Zhang, Q., Zha, Z.: Forgerygpt: Multimodal large language model for explainable image forgery detection and localization. CoRR abs/2410.10238 (2024). https://doi.org/10.48550/ARXIV.2410.10238 13. Li, Y., Yang, X., Sun, P., Qi, H., Lyu, S.: Celeb-df: A large-scale challenging dataset for deepfake forensics. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020. pp. 3204– 3213. Computer Vision Foundation / IEEE (2020). https://doi.org/10.1109/ CVPR42600.2020.00327 14. Liang, M., Qu, Y., Jiang, Y., Backes, M., Zhang, Y.: From evidence to verdict: An agent-based forensic framework for ai-generated image detection. CoRR abs/2511.00181 (2025). https://doi.org/10.48550/ARXIV.2511.00181 15. Ojha, U., Li, Y., Lee, Y.J.: Towards universal fake image detectors that generalize across generative models. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023. pp. 24480–24489. IEEE (2023). https://doi.org/10.1109/CVPR52729.2023.02345 16. Park, J., Owens, A.: Community forensics: Using thousands of generators to train fake image detectors. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN, USA, June 11-15, 2025. pp. 8245– 8257. Computer Vision Foundation / IEEE (2025). https://doi.org/10.1109/ CVPR52734.2025.00772 17. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 1824, 2022. pp. 10674–10685. IEEE (2022). https://doi.org/10.1109/CVPR52688. 2022.01042 18. Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., Nießner, M.: Faceforensics++: Learning to detect manipulated facial images. In: 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019. pp. 1–11. IEEE (2019). https://doi.org/10. 1109/ICCV.2019.00009 19. Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., Scialom, T.: Toolformer: Language models can teach themselves to use tools. In: Advances in Neural Information Processing Systems 36, NeurIPS 2023, New Orleans, LA, USA, December 1016, 2023 (2023), http : / / papers . nips . cc / paper _ files / paper / 2023 / hash / d842425e4bf79ba039352da0f658a906- Abstract- Conference.html, accessed: 21 August 2026 20. Ta, A., Zhu, J., Shayandeh, S.: Reinforced agent: Inference-time feedback for tool-calling agents. CoRR abs/2604.27233 (2026). https://doi.org/10.48550/ ARXIV.2604.27233 21. Verdoliva, L.: Media forensics and deepfakes: An overview. IEEE J. Sel. Top. Signal Process. 14(5), 910–932 (2020). https://doi.org/10.1109/JSTSP.2020.3002101 22. Wang, S., Wang, O., Zhang, R., Owens, A., Efros, A.A.: Cnn-generated images are surprisingly easy to spot... for now. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020.
Dissecting Agentic Forensics
17
pp. 8692–8701. Computer Vision Foundation / IEEE (2020). https://doi.org/ 10.1109/CVPR42600.2020.00872 23. Wu, H., Zhou, J.: Iid-net: Image inpainting detection network via neural architecture search and attention. IEEE Trans. Circuits Syst. Video Technol. 32(3), 1172–1185 (2022). https://doi.org/10.1109/TCSVT.2021.3075039 24. Wu, Q., Bansal, G., Zhang, J., Wu, Y., Zhang, S., Zhu, E., Li, B., Jiang, L., Zhang, X., Wang, C.: Autogen: Enabling next-gen LLM applications via multiagent conversation framework. CoRR abs/2308.08155 (2023). https://doi.org/ 10.48550/ARXIV.2308.08155 25. Wu, Y., AbdAlmageed, W., Natarajan, P.: Mantra-net: Manipulation tracing network for detection and localization of image forgeries with anomalous features. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019. pp. 9543–9552. Computer Vision Foundation / IEEE (2019). https://doi.org/10.1109/CVPR.2019.00977, http://openaccess. thecvf.com/content_CVPR_2019/html/Wu_ManTra-Net_Manipulation_Tracing_ Network_for_Detection_and_Localization_of_Image_CVPR_2019_paper.html, accessed: 21 August 2026 26. Xu, Z., Zhang, X., Li, R., Tang, Z., Huang, Q., Zhang, J.: Fakeshield: Explainable image forgery detection and localization via multi-modal large language models. In: The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net (2025) 27. Yan, Z., Wang, J., Jin, P., Zhang, K., Liu, C., Chen, S., Yao, T., Ding, S., Wu, B., Yuan, L.: Orthogonal subspace decomposition for generalizable ai-generated image detection. In: Forty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025. Proceedings of Machine Learning Research, vol. 267. PMLR / OpenReview.net (2025) 28. Yang, Y., Qian, Z., Zhu, Y., Russakovsky, O., Wu, Y.: D3 : Scaling up deepfake detection by learning from discrepancy. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN, USA, June 11-15, 2025. pp. 23850–23859. Computer Vision Foundation / IEEE (2025). https:// doi.org/10.1109/CVPR52734.2025.02221 29. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K.R., Cao, Y.: React: Synergizing reasoning and acting in language models. In: The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net (2023) 30. Zhang, F., Zhang, Q., Zhou, S., Sun, J., Li, C., Ai, J., Feng, Y., Zhang, Y., Li, W., Li, Z., Chang, Y., Liu, J., Zhang, K.: Code-in-the-loop forensics: Agentic tool use for image forgery detection. CoRR abs/2512.16300 (2025). https://doi.org/ 10.48550/ARXIV.2512.16300 31. Zhu, C., Wang, M., Liu, J., Chang, C., Echizen, I.: Evoguard: An extensible agentic rl-based framework for practical and evolving ai-generated image detection. CoRR abs/2603.17343 (2026). https://doi.org/10.48550/ARXIV.2603.17343
18
Li et al.
Supplementary Material A
Overview
This supplement documents our full pipeline: the complete framework of the main paper, with all components (per-detector auditing and judge guidance) and a GPT-5 judge. The pipeline’s three stages—detector dispatch (Stage 0), per-detector auditing (Stage 1), and conflict-aware judging (Stage 2)—are defined in the main paper; here we detail their input-output mechanisms, showing some examples. Appendix B follows the pipeline stage by stage on a single running example (a pristine COCO photograph of zebras, hereafter the zebra case), presenting each stage’s exact inputs and real outputs in order, from the dispatch table through the auditor prompts and verdicts to the judge prompt and final output. Prompts and outputs are reproduced verbatim from the deployed system. Colour-coded annotations mark slot types throughout: red marks content that is the same across all images but varies across detectors (the per-detector slot); blue marks content that changes for every image (the per-case slot). Appendix C presents two representative out-of-distribution (OOD) cases: one authentic image and one manipulated image. Appendix D explains how the two ablated variants differ in their inputs. All traces use Gemma-4-26B auditors and a GPT-5 judge.
B
The Zebra Example: Stage-by-Stage Walkthrough
B.1
Stage 0: Detector Dispatch
The seven detectors described in the main paper run concurrently on the submitted image, without MLLM mediation/auditing. The verbatim traces below identify each detector by its lowercase system identifier (e.g. clipsid = the CLIP-based detector from Cozzolino et al .; d3 = D3 ; the other five are named as in the main paper). Table 5 shows the dispatch output for the zebra case.
Dissecting Agentic Forensics
19
Table 5: Stage 0 dispatch output for the zebra case (authentic). Score = is_fake_score ∈ [0, 1]: the unified manipulation score computed deterministically from each detector’s native metric (0 = strong evidence authentic; 1 = strong evidence manipulated). Fired (✓) = fired = True iff Score > 0.5, i.e., the detector reports manipulation; False (blank) = the detector stays silent. Both values are authoritative inputs to all downstream agents, and their variable names appear in all agent prompts and output traces below. Detector
Scope
Native metric (raw value) Score Fired
Cozzolino et al. Whole-image generation llr_score = −1.97 D3 Whole-image generation fake_score = 0.29 Effort Face manipulation fake_prob = 0.70 ForensicsAdapter Face manipulation fake_prob = 0.59 RADAR Diffusion inpainting tamper_score = 1.00 NFA-ViT Diffusion / GAN inpainting fake_score = 0.40 TruFor Inpainting, splicing/copy-move integrity_score = 0.53
B.2
0.122 0.291 0.699 0.585 1.000 0.399 0.531
✓ ✓ ✓ ✓
Stage 1: Per-Detector Auditor Agent
Images. Each auditor call receives exactly the images its detector needs: – Non-heatmap detectors (Cozzolino et al ., D3 , Effort, ForensicsAdapter): only Image 1 (the original photo). – Heatmap detectors (RADAR, NFA-ViT, TruFor): Image 1 plus Image 2, a tool-specific two-panel PNG generated by that detector’s server: left panel = original overlaid with the detector’s heatmap; right panel = pure jet heatmap. Each heatmap auditor sees only its own tool’s two-panel PNG, not the others’. Figure 4 shows Image 1 and RADAR’s Image 2 — the two images that the RADAR auditor receives.
(a) Image 1 — original photo, received by all seven auditors
(b) Image 2 for the RADAR auditor — two-panel PNG: left = original with RADAR’s overlay; right = pure jet heatmap
Fig. 4: The two images sent to the RADAR auditor for the zebra case. Non-heatmap auditors (Cozzolino et al ., D3 , Effort, ForensicsAdapter) receive only Image 1. Each heatmap auditor (RADAR, NFA-ViT, TruFor) additionally receives its own two-panel PNG; these are separate per-tool files, not the composite panel the judge sees.
20
Li et al.
Prompt. Listing 1.1 is the verbatim prompt sent to the RADAR auditor. Red: the Tool Card block — RADAR’s identity, scope, threshold, and heatmap description. This block is different for each of the seven detectors but stays fixed across all images. Blue: the This Run block — the dispatch output specific to this image. This block changes for every image (and also reflects the detector’s native output format). The TASK, YOUR REASONING, and OUTPUT sections are identical for all seven auditors on all images. Non-heatmap detectors (Cozzolino et al ., D3 , Effort, ForensicsAdapter) receive the same prompt structure but without the heatmap instructions in STEP 2. Listing 1.1: Verbatim prompt sent to the RADAR auditor for the zebra case. Red: per-detector tool card. Blue: per-case dispatch output. The remainder is the shared fixed template. Respond with ONLY a valid JSON object. No prose, no markdown, no code fences. You are the per-tool auditor agent for ONE specific detector in a multi-agent forensic pipeline. -------------- TASK -------------Decide whether THIS detector’s output is credible evidence on THIS image, and produce one ‘‘audit_verdict‘‘. Your verdict is the gatekeeper -- ‘‘invalid‘‘ removes the tool from every downstream stage (it will not be considered by any later agent); a live verdict (e.g. ‘‘valid‘‘) propagates the tool’s output downstream. The verdict integrates three checks IN ORDER, with check 1 as a HARD PREREQUISITE for checks 2 and 3: 1. SCOPE -- does the image actually contain the kind of content this tool is designed to detect? This is the prerequisite: **when SCOPE fails, the verdict is ‘‘invalid‘‘ regardless of anything checked later** (a tool applied outside its scope produces a meaningless score, no matter how the heatmap looks, no matter how far the score is from threshold, no matter what any additional rule below says). 2. SIGNAL -- if SCOPE passed, is the is_fake_score interpretable (not stuck right at threshold), and (for heatmap tools) does the spatial pattern coherently support the score? 3. CONTEXT -- if an additional verdict-rule block is rendered below, use it to refine the choice AMONG THE LIVE VERDICTS -- it does NOT override check 1. ("Live" means the verdicts that propagate the tool downstream; a refined choice within them just tells downstream how much weight to put on the tool.) -------------- TOOL CARD -------------Tool: RADAR Designed to detect: - Localised diffusion-based image manipulations: object insertion, object removal via background inpainting, and region-level edits produced by text-conditioned latent-diffusion inpainters. Produces both an image-level score and a per-pixel tampering probability map. Decision metric threshold: - tamper_score > 0.5 -> predicted tampered; < 0.5 -> predicted clean. The score is the mean of the top 1 % highest pixel values in the tampering probability map (paper-defined image-level aggregation). How to read the output: tamper_score is the paper-defined image-level aggregation over the 1 % most-suspicious pixels in the tampering map. The continuous tampering map explains where the evidence is spatially located; cross-validate by checking whether bright regions correspond to a semantically plausible edit target in the original image. Heatmap semantics: Two-panel figure. LEFT = original image. RIGHT = jet-coloured tampering probability map (red = high tampering probability, blue = clean). Use the RIGHT panel to localise the inpainted region; verify against the original image. Attached image (Image 2) is a 2-panel PNG for this tool: left = original
Dissecting Agentic Forensics
21
with overlay heatmap; right = pure heatmap. Hot (red/orange) regions indicate the tool’s suspected manipulation locations. -------------- THIS RUN -------------Tool ID: radar Raw tool output (sanitised; file paths removed): {"is_tampered": true, "runtime_s": 0.88, "schema": "1.2", "status": "ok", "tamper_score": 1.0, "tool": "radar"} Normalized decision signal (computed by code, unified across all tools): is_fake_score = 1.0000 (range [0, 1]; threshold 0.5) -> 0 means strong evidence REAL; 1 means strong evidence FAKE fired = True (= is_fake_score > 0.5) raw source = tamper_score=1.0000 (the tool’s native score before normalization) These two values (‘‘is_fake_score‘‘ and ‘‘fired‘‘) are AUTHORITATIVE and will be re-emitted by the code unchanged. Your job is NOT to recompute them -- your job is to judge whether the verdict they imply is CREDIBLE (per the TASK above). -------------- YOUR REASONING (follow in order) -------------STEP 1 -- Scope-input applicability (the TASK’s hard prerequisite) Look at Image 1 (the original photo). Does it contain the kind of content this tool is designed to detect (see "Designed to detect" above)? When the tool’s scope requires content that is absent from this image (e.g. a face-deepfake detector applied to a landscape or to animals rather than human faces; a local-edit detector applied to clearly synthetic whole-image content), the tool is not applicable here -- its score is meaningless on this image. Per the TASK, this case yields ‘‘audit_verdict = "invalid"‘‘ and any later step (SIGNAL, CONTEXT) cannot rescue it. Otherwise proceed to STEP 2. STEP 2 -- Output interpretation Given is_fake_score = 1.0000 and threshold 0.5: - How far is it from the threshold? ~0.5 = weak signal; near 0 or 1 = strong. - "fired = True" means the tool claims the image IS manipulated. - Look at the attached heatmap (Image 2). Describe the spatial pattern of the hot (red/orange) region neutrally: * Concentrated on a specific object/region in Image 1 -> potential support for the fire. * Scattered isolated pixels with no clear spatial structure -> potential noise pattern; raise the suspicion level. * Uniformly high across the whole heatmap -> low-information pattern (saturated everywhere ~= "no localised anomaly"); raise the suspicion level. - Ask whether the hot region corresponds to ANY plausible manipulation target visible in Image 1. A clearly-named target (a specific object, a clean region boundary) is potential support for the fire; absence of a nameable target raises the suspicion level. - Do NOT pre-judge specific visual patterns as "always FP" (e.g. activation on object edges, image borders, or fabric texture). Such patterns CAN host real manipulations. If a data-driven FP signal for THIS tool on a property visible in Image 1 is supplied elsewhere in this prompt, defer to that signal rather than inventing hard-coded rules of your own here. STEP 3 -- Credibility verdict You are this tool’s gatekeeper. Your verdict here determines whether the tool’s output is forwarded to downstream stages or dropped from the case entirely. Integrate your STEP 1 and STEP 2 observations and pick the verdict that best describes this tool’s output on this image. What each value MEANS to downstream stages: "valid" -- the tool’s output is credible evidence (in either direction). A credible NEGATIVE (low is_fake_score, fired=false) is just as useful as a credible positive; downstream agents read both faithfully. "invalid" -- the tool’s output is not credible enough to count as evidence (e.g. out-of-scope content, uninterpretable signal near the threshold, or STEP 2 suspicion signals that compoundingly undermine the score). Downstream agents will exclude it.
22
Li et al. When an additional verdict-rule block is rendered above (e.g. an extended enum or tier-specific guidance), apply the more specific rule.
-------------- OUTPUT (strict JSON, no prose, no markdown) -------------{ "tool_id": "radar", "audit_verdict": <one of the values defined by the verdict rules above>, "reasoning": "<concise but specific; cite which step led to the verdict; <=150 words>" }
Outputs. Listing 1.2 gives the verbatim outputs from all seven auditor calls. Effort and ForensicsAdapter report a manipulation4 at 0.70 and 0.59 respectively, yet are immediately invalidated because STEP 1 finds no single dominant human face in the image. The remaining five detectors — including RADAR at score 1.0 — are retained and forwarded to the judge. Listing 1.2: Stage 1 auditor outputs for the zebra case, verbatim from the trace (is_fake: code-computed normalised score; fired: is_fake>0.5; the native metric follows in brackets). Each verdict line is followed by that auditor’s reasoning. Effort and ForensicsAdapter are invalidated at STEP 1 for scope (no human face); the other five are retained. clipsid -> valid [is_fake=0.122 fired=False llr_score=-1.9724] STEP 1: The tool is designed for whole-image AI detection, which is applicable to this image of zebras. STEP 2: The signal is clear and interpretable; the is_fake_score of 0.1221 is well below the 0.5 threshold, and the llr_score of -1.9724 indicates a strong prediction that the image is real. STEP 3: Since the tool is within scope and provides a decisive, non-ambiguous signal, the output is credible evidence for downstream agents. d3 -> valid [is_fake=0.291 fired=False fake_score=0.2913] STEP 1: The image contains a whole-scene composition of zebras, which falls within the scope of a whole-image AI-generation detector (D3). STEP 2: The is_fake_score of 0.2913 is well below the 0.5 threshold, indicating a clear signal that the image is predicted to be real. The score is not stuck at the threshold, providing a confident negative result. STEP 3: Since the scope is correct and the signal is interpretable and clear, the output is credible evidence for downstream agents to consider as a ’real’ classification. nfavit -> valid [is_fake=0.399 fired=False fake_score=0.3985] STEP 1: The tool is designed to detect inpainting/local manipulations, which is applicable to the zebra image. STEP 2: The is_fake_score is 0.3985, which is below the 0.5 threshold, indicating a ’real’ classification. The heatmap shows diffuse, low-magnitude activations (cyan/blue) across the zebra bodies rather than high-confidence red/orange regions. While the signal is not extremely strong, it is not stuck exactly at the threshold, and the lack of a concentrated, high-magnitude heatmap is consistent with the tool’s decision that no significant inpainting was detected. Therefore, the output is a credible negative signal. radar -> valid [is_fake=1.000 fired=True tamper_score=1.0000] STEP 1: The image contains zebras, which are suitable subjects for detecting localized diffusion-based manipulations like object insertion or inpainting. STEP 2: The is_fake_score is 1.0, a very strong signal far from the 0.5 threshold. The heatmap (Image 2) shows highly concentrated, high-magnitude (red) regions specifically localized on the bodies of the first two zebras. This spatial coherence–where the ’tampering’ is mapped to specific semantic objects rather than being scattered noise or uniform across the image–provides strong support for the tool’s claim of localized 4
In the verbatim traces, fired = True (score > 0.5) denotes a detector reporting a manipulation; audit_verdict = "valid" / "invalid" corresponds to retained / excluded in the main paper’s language.
Dissecting Agentic Forensics
23
manipulation. STEP 3: The signal is interpretable and spatially consistent with the tool’s design, making it credible evidence. trufor -> valid [is_fake=0.531 fired=True integrity_score=0.5314] STEP 1: The image contains real animals (zebras) in a natural setting, which is within the scope of TruFor’s ability to detect local manipulations like splicing or inpainting. STEP 2: The is_fake_score (0.5314) is very close to the 0.5 threshold, indicating a weak signal. However, the heatmap (Image 2) shows highly concentrated, high-magnitude (red) regions specifically localized on the bodies of the zebras. This spatial coherence–where the anomaly is not scattered noise but follows the semantic structure of the subjects–provides interpretable support for the score. While the signal is weak, the pattern is not uninterpretable or nonsensical, making the output credible evidence for downstream agents to evaluate. effort -> invalid [is_fake=0.699 fired=True fake_prob=0.6993] STEP 1: The tool is designed to detect face-manipulation deepfakes on a single human face presented as the dominant subject. The provided image contains zebras, not human faces. Therefore, the image is outside the tool’s scope, making the output meaningless. Per the task instructions, a scope failure results in an ’invalid’ verdict regardless of the score. forensicsadapter -> invalid [is_fake=0.585 fired=True fake_prob=0.5851] STEP 1: The tool is designed specifically for face forgery detection on a single human face (portrait/headshot). The provided image contains zebras, not human faces. Therefore, the image is outside the tool’s scope, making the output meaningless. Per the task instructions, a scope failure results in an ’invalid’ verdict regardless of the score.
B.3
Stage 2: Final Judge Agent
Images. The judge receives three images: Image 1 (original photo, same as the auditors’ input), Image 2 (composite overlay panel), and Image 3 (composite raw heatmap panel). These composite panels are assembled from the three heatmap detectors’ individual outputs after all auditor calls complete. Figure 5 shows Images 2 and 3 for the zebra case.
24
Li et al.
(a) Image 2 — composite overlay panel (sent to judge)
(b) Image 3 — composite raw heatmap panel (sent to judge) Fig. 5: Images 2 and 3 received by the judge for the zebra case. In each panel, the top-left tile is the original photo for reference; the remaining three tiles show outputs from RADAR, NFA-ViT, and TruFor. These composite panels are only sent to the judge — individual auditors each see their own per-tool two-panel PNG (Fig. 4b).
Prompt. The judge prompt has a large fixed body (TASK, REASONING PROTOCOL STEP 1–3, DECISION instructions, and OUTPUT format) and two sections that change per case, shown in blue in Listing 1.3. The Tool Cards block (shown in red) lists one card per active detector — it is per-detector, but the set of active detectors varies per case as some are invalidated (here, two face detectors were excluded). The Audit Evidence block (blue) carries each retained auditor’s verbatim verdict and reasoning — fully per case; these are the Stage 1 auditor outputs listed in full in Listing 1.2. Listing 1.3: Verbatim judge prompt for the zebra case. Red: TOOL CARDS block (per active detector). Blue: AUDIT EVIDENCE block (per case). The remainder is the shared fixed template. You are the final judge agent. -------------- TASK -------------Integrate your own visual inspection of Image 1, the per-tool auditor agents’ verdicts, the tool cards, and the composite heatmap panels into a SINGLE forensic verdict for this image. A core part of the integration is identifying and resolving DISAGREEMENTS among detectors -- multiple detectors with overlapping or different scopes often produce inconsistent verdicts on the same image; your job is to read those inconsistencies and decide what they mean. You do NOT re-do the per-tool auditor agents’ work. The auditor agents already decided which tools’ outputs are credible on this image (you only see the surviving ones). You integrate what you have. Your output: - is_manipulated (true or false) - manipulation_locality / manipulation_region - arbitrated_tools (the tools whose evidence drove the verdict) - decision_path (short ordered bullets naming the steps you took) - reasoning (written FIRST as a narrative, then the structured fields fall out from it) You receive: - Image 1: the original photo under examination. - Image 2: composite OVERLAY panel -- for each heatmap-producing tool, its heatmap is blended on top of the original. Tile-by-tile, labeled by tool name. Top-left tile is the unaltered original for reference. - Image 3: composite RAW heatmap panel -- pure jet heatmaps from the same tools in the same tile layout, labeled. Top-left tile is the original.
Dissecting Agentic Forensics
25
- Tool cards: one card per ACTIVE tool -- its scope_in, threshold, and signal interpretation. These define what each tool is designed to detect; you’ll use them to reason about which detectors should agree vs disagree on this image. "Active" here means the per-tool auditor agent did not invalidate the tool; invalidated tools do NOT appear. - Per-tool audit table: verdicts from the per-tool auditor agents (one auditor per detector; each examined a single tool’s output in isolation). ‘‘audit_verdict in {valid, invalid}‘‘; tools the auditor agents invalidated do NOT appear in this table. -------------- REASONING PROTOCOL -------------STEP 1 -- Independent semantic visual inspection of Image 1 Describe what the image depicts and note anything visually inconsistent: - Lighting and shadow consistency across objects. - Texture uniformity (does grain/noise match across regions?). - Perspective and scale (do foreground/background elements relate correctly?). - Object boundary quality (sharp edges where they should not be, missing shadows, mismatched depth-of-field). - For potential AI generation: hand/finger anomalies, unnatural skin, implausible backgrounds, malformed text. Your semantic observation is INDEPENDENT EVIDENCE -- it can confirm or override tool verdicts. STEP 2 -- Identify potential conflicts among detectors Every surviving tool has two attributes that together determine whether its output is consistent with another tool’s output: - SCOPE (from the tool card’s scope_in): the kind of manipulation this tool is designed to detect. - OUTPUT (from the audit table): ‘‘fired‘‘ = claims manipulation; silent = claims no manipulation in its scope. A "conflict" exists when two or more tools’ (scope, output) pairs cannot be JOINTLY true on this image. principle:
The governing
At most ONE scope can be the true scope of a manipulation, if any manipulation is present at all. Therefore, detectors whose scopes overlap but whose outputs disagree, OR detectors whose scopes do not overlap but whose outputs both fire, cannot all be right on this image. Detectors that agree within a shared scope, or that all stay silent across scopes, are jointly consistent and need no resolution. Walk through the surviving tools, compare their (scope, output) pairs across one another, and list the conflict groups you find. The principle above is the criterion, not an exhaustive catalogue -- derive the conflict-vs-no-conflict distinction yourself for each combination you encounter. Briefly state the conflicts you identified in your reasoning output (this is your audit trail). Do NOT attempt resolution yet -- that is STEP 3. STEP 2b -- Reason about per-tool reliability on THIS image Each surviving tool is a statistical classifier whose paper-reported accuracy is meaningfully below 100%. A tool that fires is making a probabilistic claim -- sometimes a correct detection within its scope, sometimes a false alarm where the tool over-triggered on image content that resembled its training distribution. A silent tool is correspondingly sometimes a missed detection. When you weigh a surviving tool’s verdict, you are weighing the joint plausibility of "true detection on this image" against "this is one of the tool’s known failure modes". The lens for reasoning about whether THIS tool’s output on THIS image is credible evidence: - Scope vs image content (your STEP 1 visual reading): does Image 1 plausibly contain the manipulation type the tool was designed to detect? A face-manipulation detector firing on a landscape with no visible face has no place for a true positive to be; a whole-image AI-generation detector firing on a photograph with clear camera noise and consistent lighting is making a claim that the scene’s natural evidence contradicts. - Score magnitude relative to the decision boundary (from the tool card’s threshold): a score sitting close to the boundary ridge is qualitatively different from a saturated score -- borderline values are more often false alarms than confident detections. - Heatmap shape, for heatmap-producing tools (Images 2 and 3): a spatial pattern that lands on an attacker-plausible region (a single coherent object, a clear inpainting target, a face) is qualitatively different from a pattern that scatters across natural-texture content (water ripples, foliage, dense repeating patterns, sky) where spurious activations are well-known in the forensic literature. - Cross-tool corroboration within the same scope: a positive backed by another in-scope peer is more likely a true detection than a lone positive surrounded by silent peers sharing the same scope. These cues are NOT a checklist with a scored outcome -- they are the lens through which you re-read each surviving tool’s output. Your reading of each tool ("this output is credible evidence" vs "this output is likely a false alarm on this image") is what STEP 3 then integrates with your STEP 1 visual observations to resolve the STEP 2 conflicts. STEP 3 -- Resolve conflicts and integrate into the final verdict For each conflict you listed in STEP 2, decide which direction survives.
Inputs available for your decision:
- Tool cards’ scope_in -- which detector’s design scope actually matches the content visible in Image 1 (per your STEP 1 visual inspection)? - Composite heatmap panels (Image 2 overlay, Image 3 raw) -- do firing heatmap detectors highlight the SAME image structure (multi-tool spatial corroboration) or DIFFERENT structures (score-level agreement without spatial agreement)? - Per-tool auditor reasoning (in the audit table) -- what each auditor saw and why it gave its verdict. Integrate the surviving evidence + your STEP 1 visual observations into the final verdict. When the surviving tools and your visual observations point in different directions, you decide how to weigh them given the inputs available to you. TOOL CARDS (per active tool): clipsid (ClipSID): Detects: Whole-image classification: is the entire image AI-generated content?. Threshold: llr_score: signed scalar -- negative -> predicted real, positive -> predicted AI-generated (decision boundary at 0). Signal: llr_score is a signed scalar. Sign = predicted class; magnitude = relative distance from the decision boundary. It is NOT a calibrated probability. d3 (D3): Detects: Whole-image AI-generation detection (GAN and diffusion).. Threshold: fake_score > 0.5 -> predicted AI-generated; < 0.5 -> predicted real (fixed decision boundary). Signal: fake_score is a sigmoid scalar. Above 0.5 -> the model detected AI-generation artifacts; below 0.5 -> feature distributions look natural. Magnitude reflects the model’s confidence, not a calibrated probability. radar (RADAR):
26
Li et al.
Detects: Localised diffusion-based image manipulations: object insertion, object removal via background inpainting, and region-level edits produced by text-conditioned latent-diffusion inpainters. Produces both an image-level score and a per-pixel tampering probability map.. Threshold: tamper_score > 0.5 -> predicted tampered; < 0.5 -> predicted clean. The score is the mean of the top 1 % highest pixel values in the tampering probability map (paper-defined image-level aggregation). Signal: tamper_score is the paper-defined image-level aggregation over the 1 % most-suspicious pixels in the tampering map. The continuous tampering map explains where the evidence is spatially located; cross-validate by checking whether bright regions correspond to a semantically plausible edit target in the original image. nfavit (NFA-ViT): Detects: Image inpainting localization -- GAN-based, traditional, and diffusion-based inpainting alike. Produces both an image-level score and a per-pixel segmentation mask of the inpainted region.. Threshold: fake_score > 0.5 -> inpainting detected (paper-derived decision boundary). Values near 0.5 are uncertain; > 0.8 indicates high-confidence inpainting. Signal: fake_score is the top-1 % mean of pred_mask, the pixel-level segmentation output of a trained decoder head (not a saliency / gradient explanation). A high fake_score with no compact heatmap region is diagnostically inconsistent and should raise suspicion of a false positive. trufor (TruFor): Detects: Local-manipulation localization: splicing, copy-move, inpainting (including diffusion-based). Produces both an image-level score and a pixel-level anomaly map.. Threshold: integrity_score is in [0, 1]; decision boundary at 0.5 (higher -> more likely tampered). Signal: integrity_score -> higher means more likely tampered. anomaly_max = peak local suspicion; anomaly_mean = background floor. A LOW integrity score does NOT automatically rule out manipulation if the anomaly map shows a concentrated hot region at a semantically plausible tamper location. -------------- DECISION -------------After working through the reasoning protocol above, integrate to the final verdict. WRITE ‘‘reasoning’’ FIRST as a step-by-step narrative. Walk through every reasoning element rendered above IN THE ORDER IT APPEARED: each section / step / rule. For each, name what you considered, name the rule you applied, and name the surviving tool(s). Once the narrative is complete, the remaining fields fall out: - arbitrated_tools = a faithful transcription of the conclusion you just wrote -- the specific tool(s) your reasoning named as the DECISIVE evidence for the verdict, listed most-decisive FIRST. This is NOT a re-listing of every tool that fired, nor every tool the per-tool auditor agent left live; it is exactly the driver(s) your final reasoning step settled on. When your reasoning prioritized a scope or tool ("I favor the local detectors", "X is the most credible", "the face evidence is primary"), those favored tool(s) are the arbitrated ones and the first element is the one you treated as primary. A tool you described only as corroborating / supporting / consistent-with / secondary stays OUT -- it shaped your thinking but did not drive the verdict. Non-empty when is_manipulated=true; empty when is_manipulated=false. - is_manipulated = true iff ‘‘arbitrated_tools’’ is non-empty AND the winning evidence is not contradicted by the observations available to you. - manipulation_locality = "global" (whole-image AI-gen-style) or "local" (a specific region) or "n/a" (pristine). - manipulation_region = brief textual description of where the edit is, if local; null otherwise. - decision_path = ordered list of short bullet strings naming the reasoning steps you walked. Each bullet <= 25 words. AUDIT EVIDENCE: clipsid: valid fired=False score=0.122 -> STEP 1: The tool is designed for whole-image AI detection, which is applicable to this image of zebras. STEP 2: The signal is clear and interpretable; the is_fake_score of 0.1221 is well below the 0.5 threshold, and the llr_score of -1.9724 indicates a strong prediction that the image is real. STEP 3: Since the tool is within scope and provides a decisive, non-ambiguous signal, the output is credible evidence for downstream agents. d3: valid fired=False score=0.291 -> STEP 1: The image contains a whole-scene composition of zebras, which falls within the scope of a whole-image AI-generation detector (D3). STEP 2: The is_fake_score of 0.2913 is well below the 0.5 threshold, indicating a clear signal that the image is predicted to be real. The score is not stuck at the threshold, providing a confident negative result. STEP 3: Since the scope is correct and the signal is interpretable and clear, the output is credible evidence for downstream agents to consider as a ’real’ classification. radar: valid fired=True score=1.000 -> STEP 1: The image contains zebras, which are suitable subjects for detecting localized diffusion-based manipulations like object insertion or inpainting. STEP 2: The is_fake_score is 1.0, a very strong signal far from the 0.5 threshold. The heatmap (Image 2) shows highly concentrated, high-magnitude (red) regions specifically localized on the bodies of the first two zebras. This spatial coherence--where the ’tampering’ is mapped to specific semantic objects rather than being scattered noise or uniform across the image--provides strong support for the tool’s claim of localized manipulation. STEP 3: The signal is interpretable and spatially consistent with the tool’s design, making it credible evidence. nfavit: valid fired=False score=0.399 -> STEP 1: The tool is designed to detect inpainting/local manipulations, which is applicable to the zebra image. STEP 2: The is_fake_score is 0.3985, which is below the 0.5 threshold, indicating a ’real’ classification. The heatmap shows diffuse, low-magnitude activations (cyan/blue) across the zebra bodies rather than high-confidence red/orange regions. While the signal is not extremely strong, it is not stuck exactly at the threshold, and the lack of a concentrated, high-magnitude heatmap is consistent with the tool’s decision that no significant inpainting was detected. Therefore, the output is a credible negative signal. trufor: valid fired=True score=0.531 -> STEP 1: The image contains real animals (zebras) in a natural setting, which is within the scope of TruFor’s ability to detect local manipulations like splicing or inpainting. STEP 2: The is_fake_score (0.5314) is very close to the 0.5 threshold, indicating a weak signal. However, the heatmap (Image 2) shows highly concentrated, high-magnitude (red) regions specifically localized on the bodies of the zebras. This spatial coherence--where the anomaly is not scattered noise but follows the semantic structure of the subjects--provides interpretable support for the score. While the signal is weak, the pattern is not uninterpretable or nonsensical, making the output credible evidence for downstream agents to evaluate. Excluded by the per-tool auditor agent (do not consider): effort, forensicsadapter -------------- OUTPUT (strict JSON, no prose, no markdown) -------------{ "reasoning": "<<=200 words; step-by-step narrative in the order the prompt rendered the reasoning elements; cite the rule and the surviving tool(s) for each>", "is_manipulated": true or false, "manipulation_locality": "global" or "local" or "n/a", "manipulation_region": "<brief region description or null>", "arbitrated_tools": ["<tool_id>", ...], "decision_path": ["<step1>", "<step2>", ...] }
Dissecting Agentic Forensics
27
Output. Listing 1.4 shows the real GPT-5 output for the zebra case.5 Listing 1.4: Real GPT-5 judge output for the zebra case (verbatim from trace). reasoning: STEP 1 – Visual inspection: Photo of three zebras walking in daylight. Lighting and shadows are consistent under each zebra; textures/noise uniform; perspective and boundaries natural; no seams or depth-of-field mismatches. No obvious local edits. STEP 2 – Conflicts: Whole-image detectors (ClipSID, D3) both predict real (silent). Local tools disagree: RADAR fires strongly on the left zebra and center chest; TruFor barely above threshold and highlights mostly the right zebra; NFA-ViT is below threshold with diffuse maps. These constitute intra-scope conflicts (only one local-edit scope can be true). STEP 2b – Reliability on this image: RADAR’s saturated score but object-sized blobs align with high-contrast zebra stripes – a known false-positive surface. TruFor’s score is near the decision boundary and its hotspot is in a different place, weakening credibility. NFA-ViT’s negative with no compact mask argues against inpainting. Lack of spatial corroboration across local tools further suggests false alarms. STEP 3 – Resolution: Favor the consistent global-real signals and clean visual evidence over conflicting local positives. Conclude no manipulation. is_manipulated: false manipulation_locality: n/a arbitrated_tools: (none) decision_path: 1. Visually inspect: lighting, shadows, textures consistent; no seams. 2. Global AI detectors (ClipSID, D3) agree image is real. 3. Local tools conflict: RADAR strong, TruFor weak, NFA-ViT negative. 4. Heatmaps disagree spatially; activations follow zebra stripes – false-positive prone. 5. Integrate: global negatives + clean visuals outweigh local conflicts – pristine.
The judge receives RADAR at score 1.0 and TruFor reporting a manipulation, yet correctly identifies them as false alarms by noting that (a) the two positive heatmaps disagree spatially, (b) both global detectors report real, and (c) RADAR’s activation follows zebra stripes, a well-known false-positive surface.
C
Two Representative OOD Cases
The zebra walkthrough above used a single in-distribution image to expose every prompt and output in full. Here we complement it with two out-of-distribution cases—one authentic, one manipulated—presented more compactly: for each we give only the Stage-1 auditor outputs and the Stage-2 judge output, as the prompts are identical to those in Appendix B. C.1
Authentic: Backlit Cow at Sunset
Source: an authentic image from the CASIA v2.0 authentic-image pool. 5
arbitrated_tools = the decisive detector of the main paper (most-decisive first).
28
Li et al.
(a) Image 2 — composite overlay panel (sent to judge)
(b) Image 3 — composite raw heatmap panel (sent to judge) Fig. 6: Inputs for the authentic cow case (Case C.1). In each panel the top-left tile is the original; the others are RADAR, NFA-ViT, and TruFor. RADAR reports a manipulation at 1.0 with a whole-cow heatmap; NFA-ViT is borderline with a diffuse pattern; TruFor is below threshold.
Listing 1.5: Stage 1 auditor outputs — authentic cow field, verbatim from the trace. Three detectors are invalidated (Effort and ForensicsAdapter for scope, NFA-ViT for a borderline / spatially incoherent signal); RADAR reports a manipulation but is alone. clipsid -> valid [is_fake=0.297 fired=False llr_score=-0.8626] STEP 1: The tool is designed for whole-image AI detection, which is applicable to this photograph of cattle in a field. STEP 2: The is_fake_score of 0.2968 is well below the 0.5 threshold, and the llr_score of -0.8626 indicates a clear signal towards the ’real’ classification rather than being stuck at the decision boundary. STEP 3: Since the tool is in-scope and provides a coherent, interpretable signal (predicting the image is real), the output is credible evidence for downstream agents. d3 -> valid [is_fake=0.016 fired=False fake_score=0.0164] STEP 1: The image is a whole-image photograph of cattle in a field, which falls within the scope of D3’s design to detect whole-image AI generation (GAN/diffusion). STEP 2: The signal is highly interpretable; the is_fake_score of 0.0164 is far from the 0.5 threshold, indicating strong confidence from the model that the image is real. STEP 3: Since the tool is in-scope and provides a clear, non-ambiguous signal, the output is credible evidence and is marked as valid. radar -> valid [is_fake=1.000 fired=True tamper_score=1.0000] STEP 1: The image contains a natural scene with animals, which is within the scope of RADAR’s ability to detect localized object insertions or edits. STEP 2: The is_fake_score is 1.0, representing a maximum signal strength far from the 0.5 threshold. The heatmap (Image 2) shows a highly concentrated, coherent spatial pattern covering the primary subject (the foreground cow). This localized high-probability region corresponds directly to a specific object in the original image, providing strong spatial support for the score. The signal is interpretable and not noise-driven. STEP 3: Since the scope is valid and the signal is strong and spatially coherent, the verdict is valid. trufor -> valid [is_fake=0.333 fired=False integrity_score=0.3329] STEP 1: The image contains a natural scene with animals, which is within the scope of TruFor’s ability to detect local manipulations like splicing or inpainting. STEP 2: The is_fake_score (0.3329) is below the 0.5 threshold, indicating a ’real’ verdict. The heatmap shows highly localized, intense red/orange anomalies specifically on the
Dissecting Agentic Forensics
29
lower legs of the foreground cow. While these are ’hot’ regions, they are spatially concentrated and do not represent a global or noisy pattern. The low integrity score combined with these specific local anomalies suggests the tool is correctly identifying the image as largely authentic while noting minor local irregularities. The signal is interpretable and provides credible evidence for the downstream pipeline. effort -> invalid [is_fake=0.829 fired=True fake_prob=0.8291] STEP 1: The tool is designed to detect face-manipulation deepfakes in a single, centrally framed human face. The provided image contains cows in a field and no human faces. Therefore, the image is outside the tool’s scope, making the output meaningless. Per the task instructions, a scope failure requires an ’invalid’ verdict regardless of the score. forensicsadapter -> invalid [is_fake=0.947 fired=True fake_prob=0.9472] STEP 1: The tool is designed specifically for face forgery detection in single human portraits. The provided image contains cows in a field and no human faces. Therefore, the image is outside the tool’s scope, making the output meaningless. Per the task instructions, a scope failure results in an ’invalid’ verdict regardless of the score. nfavit -> invalid [is_fake=0.512 fired=True fake_score=0.5121] STEP 1: The tool is designed to detect inpainting (local object removal/replacement). The image contains a natural scene of cows in a field, which is within the scope of detecting local manipulations. STEP 2: The is_fake_score is 0.5121, which is extremely close to the 0.5 threshold, indicating a very weak and uncertain signal. Furthermore, the heatmap (Image 2) shows a highly diffuse, non-localized, and unstructured pattern that covers large portions of the background and the cow’s body without identifying a compact, contiguous region of tampering. This lack of spatial coherence, combined with the marginal score, suggests the signal is likely noise rather than credible evidence of inpainting. Therefore, the output is deemed uninterpretable and invalid.
Listing 1.6: Real GPT-5 judge output — authentic cow field (verbatim from trace). reasoning: STEP 1 – Visual read: Backlit cow in pasture at sunset. Shadows align across animals and ground; lens-flare orbs appear in front of the cow; edges, scale, and texture look natural. STEP 2 – Conflicts: Whole-image AI detectors (ClipSID, D3) are silent (predict real). Local tools disagree: RADAR fires strongly on the foreground cow; TruFor is below threshold with small hotspots on the lower legs. STEP 2b – Tool credibility: RADAR: score=1.0 with a heatmap blanketing the entire foreground cow. While attacker-plausible for splicing, such whole-object saturation is a known false-alarm pattern on salient, backlit objects; additionally, the in-front lens flare contradicts a simple post-hoc insertion. No in-scope corroboration. TruFor: integrity 0.333 (<0.5), hotspots confined to hooves – plausible natural artifacts (shadow/flare), not a coherent tamper region. ClipSID/D3: confidently real; they cannot confirm local edits but argue against full-image synthesis. STEP 3 – Resolution: Given consistent scene physics and lack of multi-tool local corroboration, downweight RADAR’s lone claim. Favor the combined evidence from TruFor (under-threshold) and global detectors. Verdict: no manipulation. is_manipulated: false manipulation_locality: n/a arbitrated_tools: (none) decision_path: 1. Visually check lighting, shadows, texture, boundaries; scene appears consistent with in-camera photo. 2. Record tool outputs: ClipSID real, D3 real, RADAR tampered, TruFor below threshold with leg hotspots. 3. Identify conflict within local scope: RADAR vs TruFor. 4. Assess credibility: RADAR blanket foreground-cow heatmap; no corroboration; lens flare in front contradicts splicing. 5. Integrate evidence; weight TruFor and global detectors; conclude pristine image.
Of the retained detectors, only RADAR reports a manipulation, blanketing the foreground cow at saturation. Lacking any in-scope corroboration and contra-
30
Li et al.
dicted by the consistent scene physics, the judge rejects it as a false alarm and returns authentic. C.2
Manipulated: Face Swap (Celeb-DF-v2)
Source: a face-swap manipulation from Celeb-DF-v2.
(a) Image 2 — composite overlay panel (sent to judge)
(b) Image 3 — composite raw heatmap panel (sent to judge) Fig. 7: Inputs for the face-swap case (Case C.2). In each panel the top-left tile is the original; the others (tiles 2–4) are RADAR, NFA-ViT, and TruFor. RADAR and NFAViT heatmaps show saturated full-face patterns characteristic of false positives; TruFor is low throughout.
Listing 1.7: Stage 1 auditor outputs — face swap, verbatim from the trace. RADAR and NFA-ViT report a manipulation at saturation but are invalidated for full-face / spatially incoherent heatmaps; the two face-forgery specialists (Effort, ForensicsAdapter) are retained with high-confidence corroborating positives. clipsid -> valid [is_fake=0.542 fired=True llr_score=0.1690] STEP 1: The tool is designed for whole-image AI generation detection. The image is a standard portrait, which falls within the scope of general image classification. STEP 2: The is_fake_score is 0.5421, which is slightly above the 0.5 threshold. While the signal is relatively weak (close to the decision boundary), it is a clear, interpretable value that indicates a prediction of AI-generated content. STEP 3: Since the tool is in-scope and provides a non-ambiguous signal, the output is considered credible evidence for downstream agents to weigh.
Dissecting Agentic Forensics
31
d3 -> valid [is_fake=0.005 fired=False fake_score=0.0048] STEP 1: The image is a portrait of a person, which falls within the scope of a whole-image AI-generation detector (D3) designed to identify GAN or diffusion artifacts. STEP 2: The signal is highly interpretable; the is_fake_score of 0.0048 is very far from the 0.5 threshold, indicating a strong, confident prediction that the image is real. STEP 3: Since the tool is in-scope and provides a clear, non-ambiguous signal, the output is credible evidence for downstream agents. effort -> valid [is_fake=0.982 fired=True fake_prob=0.9819] STEP 1: The image contains a single human face as the dominant subject, which is the required scope for this tool. STEP 2: The is_fake_score of 0.9819 is very far from the 0.5 threshold, indicating a strong and interpretable signal. STEP 3: Since the tool is in-scope and provides a high-confidence signal, the output is credible evidence and is marked as valid. forensicsadapter -> valid [is_fake=1.000 fired=True fake_prob=0.9997] STEP 1: The image contains a single human face as the dominant subject, centrally framed in a portrait-style shot, which falls squarely within the tool’s intended scope. STEP 2: The signal is extremely strong; the is_fake_score of 0.9997 is far from the 0.5 threshold, indicating high model confidence in detecting blending-boundary artifacts. STEP 3: Since the scope is met and the signal is highly interpretable and decisive, the output is credible evidence and is marked as valid. trufor -> valid [is_fake=0.211 fired=False integrity_score=0.2110] STEP 1: The tool is designed for local-manipulation detection (splicing, inpainting, etc.), which is applicable to this portrait image. STEP 2: The is_fake_score is 0.211, well below the 0.5 threshold, indicating a ’real’ prediction. The heatmap shows low-magnitude, diffuse activations primarily around facial features (eyes, nose, mouth) rather than concentrated, high-intensity red regions. This pattern is consistent with low-level noise or texture-based activations common in real images rather than a coherent manipulation signal. Since the signal is interpretable and the score is far from the decision boundary, the tool’s negative finding is credible evidence. nfavit -> invalid [is_fake=0.992 fired=True fake_score=0.9916] STEP 1: The tool is designed to detect localized inpainting (removing/replacing objects). While the image contains a face, the tool’s output in STEP 2 shows a massive, non-localized, and saturated heatmap covering almost the entire head and neck area. This ’full-face’ activation pattern is diagnostically inconsistent with the tool’s purpose of identifying specific inpainting regions; it behaves more like a global texture or compression detector rather than a localized inpainting detector. Because the signal is spatially incoherent and lacks a compact, nameable target region, it is considered a false positive/uninterpretable signal, leading to an ’invalid’ verdict. radar -> invalid [is_fake=1.000 fired=True tamper_score=1.0000] STEP 1: The tool is designed to detect localized diffusion-based manipulations (object insertion/removal). The input image is a portrait of a person, which is within the scope of detecting localized edits. STEP 2: The is_fake_score is 1.0, indicating a very strong signal. However, the heatmap (Image 2) shows a massive, saturated red region covering almost the entire subject, including the face and hair, with high-intensity edges around the silhouette. This uniform, high-magnitude pattern is characteristic of a low-information/saturated signal rather than a localized edit. Instead of identifying a specific manipulated object or region, the tool is essentially flagging the entire subject as anomalous, which suggests the signal is not interpretable as a specific localized tampering event. Therefore, the signal is not credible.
Listing 1.8: Real GPT-5 judge output — face swap (verbatim from trace). reasoning: STEP 1 – Visual: Single centrally framed face, low resolution/compressed. Lighting/background consistent; facial boundary (cheeks/jaw/forehead) looks unusually soft, compatible with blending. STEP 2 – Conflicts: (a) Whole-image AI-gen: ClipSID fired (0.542) vs D3 strongly real (0.005). (b) Local manipulation: face-forgery detectors Effort (0.982) and ForensicsAdapter (0.9997) fired, while TruFor predicted real (0.211).
32
Li et al. STEP 2b – Reliability: D3 is far below boundary – confident real; ClipSID is only slightly positive – likely false alarm. Effort and ForensicsAdapter are in-scope (single dominant face) and give saturated positives – credible and mutually corroborating. TruFor’s heatmap in the panels is diffuse/low around normal facial textures, a known weak pattern for face-swap detection at low resolution – down-weighted. STEP 3 – Resolution: Prefer D3 over ClipSID – not globally AI-generated. Prioritize the corroborated, high-confidence face-forgery signals (ForensicsAdapter, Effort) over TruFor’s weak negative. Integrated with visual cues, the most plausible explanation is a local face manipulation (swap/reenactment) centered on the face.
is_manipulated: true manipulation_locality: local manipulation_region: face region (central portrait), especially along the facial boundary around cheeks, jawline, and forehead arbitrated_tools: forensicsadapter, effort decision_path: 1. Visual review: single face; low-res; soft facial boundary suggests blending. 2. Conflict group: global AI-gen detectors disagree (ClipSID+, D3-). 3. Conflict group: face-forgery tools fired; TruFor negative. 4. Reliability: D3 strong real; ClipSID marginal – discount global AI-gen. 5. Reliability: ForensicsAdapter and Effort high-confidence, in-scope, corroborating. 6. Heatmaps: TruFor diffuse/low – likely miss; downweight. 7. Integrate – local face manipulation centered on face.
This is the only manipulated case. The auditor retains the two in-scope face specialists (Effort, ForensicsAdapter), which report the swap with high confidence, and excludes RADAR and NFA-ViT, whose saturated full-face heatmaps are out-of-scope false positives. The judge arbitrates to the face specialists and returns a local face manipulation—the correct verdict and the correct decisive detectors.
D
Variant Input Differences
The output schema is identical across all variants; only their inputs differ. Algorithmic fusion uses no MLLM and is described in the main paper. D.1
Unguided Judge
The unguided-judge variant is identical to the full pipeline except that three sections are removed from the judge’s REASONING PROTOCOL—this is the prompting axis of the main paper’s dissection (the judge no longer receives explicit false-positive guidance). Listing 1.9 shows the full pipeline’s REASONING PROTOCOL with the removed blocks in gray italic. After STEP 1 visual inspection the judge moves directly to a simplified one-step integration (the ablated replacement is shown in black after the gray blocks). Listing 1.9: Judge REASONING PROTOCOL. Gray italic: blocks present in the full pipeline but removed in the unguided-judge variant (the judge prompt stops after STEP 1 and jumps to the simplified STEP 2 shown in black). STEP 1 -- Independent semantic visual inspection of Image 1 [... identical in both variants: describe lighting, texture, perspective, boundaries, AI-generation cues ...] STEP 2 -- Identify potential conflicts among detectors
Dissecting Agentic Forensics
33
Every surviving tool has two attributes -- SCOPE (from the tool card’s scope_in) and OUTPUT (fired / silent) -- that determine whether its verdict is consistent with another tool’s. The governing principle: at most ONE scope can be the true scope of a manipulation. Walk through the surviving tools, compare their (scope, output) pairs, and list the conflict groups you find. Do NOT attempt resolution yet -- that is STEP 3. STEP 2b -- Reason about per-tool reliability on THIS image Each surviving tool is a statistical classifier whose accuracy is below 100 %. The reliability lens for this image: - Scope vs image content: does Image 1 plausibly contain the manipulation the tool detects? - Score magnitude relative to the decision boundary. - Heatmap shape (for heatmap tools): compact on an attacker-plausible region vs scattered on natural-texture content where spurious activations are well-documented? - Cross-tool corroboration within the same scope. STEP 3 -- Resolve conflicts and integrate into the final verdict For each conflict listed in STEP 2, decide which direction survives using: tool cards’ scope_in vs Image 1 content; composite heatmap panels; auditor reasoning from the audit table. Integrate surviving evidence with STEP 1 visual observations. STEP 2 -- Integrate tool evidence into the final verdict Using your STEP 1 visual observations, the tool cards, the audit table, and the composite heatmap panels, decide whether the image has been manipulated. When surviving tools and visual observations point in different directions, you decide how to weigh them given the inputs available to you.
The per-case inputs (retained tool cards, audit evidence) are identical to the full pipeline. D.2
Single-Agent
The seven per-detector auditor calls are removed entirely. A single judge agent receives all seven detectors’ raw scores and tool cards (no pre-filtering) and is tasked with doing both jobs in one pass. Listing 1.10: Judge prompt for the single-agent variant. The STEP 0 self-audit block replaces the separate auditor stage. You are the final judge agent. In this configuration there is NO separate per-tool auditor -- you do BOTH jobs in one pass: first assess each detector’s credibility on this image yourself, then integrate the credible ones into a single forensic verdict. -------------- TASK -------------[... identical structure but receives ALL seven detectors’ raw evidence (no pre-filtering) and all seven tool cards (not just the surviving ones).] -------------- REASONING PROTOCOL -------------STEP 0 -- Per-tool credibility (you are the auditor here) For EACH detector, decide whether its output is credible: - SCOPE: does Image 1 contain the content this detector was designed for? Out-of-scope score is meaningless -- set aside. - SIGNAL: if scope holds, is the score interpretable (not stuck near the 0.5 boundary)? For heatmap detectors (Images 2 and 3): does the spatial pattern land on an attacker-plausible region, or scatter across natural-texture content where spurious activations are well-documented? - Cross-tool corroboration within a shared scope: a lone positive among silent same-scope peers is more likely a false alarm. Name each detector as kept or set aside, with reason. Carry the credible detectors forward. STEP 1 -- Independent visual inspection of Image 1 [... same instructions as the full pipeline ...]
34
Li et al.
STEP 2 -- Identify conflicts among credible detectors [... same single-scope principle applied to STEP 0 survivors ...] STEP 3 -- Resolve conflicts and integrate into final verdict [... same as the full pipeline ...] [OUTPUT FORMAT -- identical to the full pipeline.]
The full pipeline runs seven parallel MLLM auditor calls plus one judge call (≈8 calls total); the single-agent variant collapses this to a single MLLM call. The judge prompt grows correspondingly (all seven tool cards, no pre-assigned verdicts), and STEP 0 adds a self-audit pass before the conflict-resolution steps.