Trusting the Tool: Asymmetric Deference of LLM Vulnerability Auditors to Static-Analysis Evidence

Trusting the Tool: Asymmetric Deference of LLM Vulnerability Auditors to Static-Analysis Evidence ZhiHong Liu1,2 and Shuyao Gao∗1

arXiv:2610.32876v1 [cs.SE] 23 Oct 2026

1

Seoul School of Integrated Sciences and Technologies (aSSIST University), Seoul, Republic of Korea 2 Liaoning Mechanical & Electrical College of Technology, Dandong, Liaoning, China

Abstract Pipelines that combine large language models (LLMs) with static analysis let the model consume the analyser’s output as evidence, but whether the model distinguishes true from false evidence, and whether its scrutiny depends on what the evidence claims, has not been measured. We built an intraprocedural analyser for C/C++ that produces proof obligations and audited vulnerable and patched functions from PrimeVul under six evidence conditions that differ only in the notes, including fabricated assurances that a missing check exists and fabricated alarms that an added check is missing. The study covers 916 pairs in four disjoint sets, eleven model configurations from four vendors and 16,572 audits; no model is trained or finetuned. True evidence barely improved accuracy (+0.02). False evidence acted asymmetrically: relative to true notes about the same operations, false assurances lowered the detection of real vulnerabilities by up to 19 points, whereas false alarms never significantly lowered accuracy and were disputed far more often. An internally registered replication on seven new models confirmed that false assurances move verdicts towards their claim more than false alarms do (mean asymmetry +0.16, 95% CI +0.10 to +0.24). Instructions in the audit prompt did not reliably reduce deference; the best one failed to replicate. Verifying each claim in a separate call and showing the refuted claims raised accuracy under false assurances by 12 points in a pre-specified test (p = 0.002) without loss under true evidence; for a second model the gain was smaller and not significant. Pipelines should therefore verify a tool’s guarantees before the model relies on them, and studies should measure deference in a direction-aware way.

Keywords: Automation bias, empirical study, human–AI interaction, large language models, static analysis, vulnerability detection.

1

Introduction

Large language models (LLMs) are increasingly used to audit code for vulnerabilities, and a natural way to make them more reliable is to give them the output of program analysis. Systems that combine LLMs with static analysis let the analyser enumerate candidate defects and ask the model to judge them [1,2], or let the model triage the analyser’s warnings [3,4]. In all of these designs the ∗

Corresponding author. E-mail: gaoshuyao.mk@gmail.com This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.

1

model consumes evidence produced by a tool, and the value of the combination depends on how the model treats that evidence: whether it uses true evidence, and whether it notices when the evidence is wrong. Static analysers are wrong often enough that developers cite false positives as a main reason not to use them [5], and the same analysers can also be wrong in the other direction, reporting that a guard exists or that a path is safe when it is not. How people treat the output of automated aids has been studied for decades under the name of automation bias: people over-rely on automation, accept its errors of commission, and miss what it fails to flag [6, 7]. Recent work in human–AI interaction shows the same pattern with AI assistants, where explanations can increase rather than reduce over-reliance [8] and fluent assistance can create confidence without competence [9]. LLMs that consume tool output are in the position of the human user. Work on knowledge conflicts shows that LLMs can be highly receptive to external evidence that contradicts what they know [10–12], and work on sycophancy shows that they tend to agree with the views put to them [13]. Concurrent work has begun to show that context outside the code, such as prior verdicts, pull-request metadata and comments, shifts the verdicts of LLM security reviewers [14–16]. These studies vary cues that are not tied to the truth of a specific claim, or study only claims that the code is safe. The closest, ALIBI [16], already embeds fabricated tool reports in code comments to evade LLM detectors, but it studies only claims of safety and does not compare a false claim with a true claim about the same operation. Our study adds three things: true and false evidence matched on the same operations, false claims in both directions, and mitigations evaluated on held-out data. Whether LLM auditors distinguish true from false analysis evidence about the same operations, and whether their scrutiny depends on the direction of the claim, has not been measured. This paper reports a controlled empirical study of that question. We built an intraprocedural analyser for C and C++ that produces proof obligations: for each sensitive operation, the guards that hold on every path to it and what the function does not guarantee. Running it on pairs of vulnerable and patched functions from PrimeVul [17], we audit the same function under six evidence conditions that vary only the notes: none, all true notes, true notes about the operations the fix touches, true notes about other operations, a fabricated assurance that the missing check exists, and a fabricated alarm that the check added by the fix is missing. Because both false conditions are compared with true notes about the same operations, differences in the verdicts can be attributed to the truth of the evidence. All models are used through their APIs for inference only; no model training or fine-tuning is performed. We ask four research questions: whether true evidence helps (RQ1), how relevance and correctness affect the verdicts (RQ2), whether the models check the evidence (RQ3), and whether the way evidence is presented can reduce deference to false evidence (RQ4). The main findings are as follows. • True evidence barely helps. Showing all true notes changes accuracy by +0.02 (95% CI −0.03 to +0.07); restricting them to the relevant operations gives a small, non-significant gain (RQ1). • False evidence acts asymmetrically. A fabricated assurance lowers the detection of real vulnerabilities (by 0.11, p = 0.012, for deepseek-chat, and by 0.13–0.19 for four of seven further models), whereas a fabricated alarm about a fixed function never significantly lowers accuracy and often raises it, because the model goes looking for the guard and finds it (RQ2). Models dispute false alarms in the text of their answers far more often than false assurances (RQ3). • The asymmetry generalises. Accuracy contrasts make deference look model-dependent, because models that flag nearly every function cannot be pushed further. A direction-aware 2

measure reduces this confound, although its size still depends on how much room a model’s leaning leaves in each direction. After deriving it on four configurations, we registered a replication internally on seven new models from four vendors, which confirmed it: false assurances move verdicts towards their claim more than false alarms do (mean asymmetry +0.16, 95% CI +0.10 to +0.24, p < 0.001). • Separate verification mitigates deference; instructions do not reliably. The best instructionlevel strategy on the study set (reframing assurances as claims to verify, +0.18) failed to replicate on held-out data. Checking each note against the code in a separate call and showing the refuted notes (verify-show) recovered 12 points of accuracy under false assurances in a pre-specified test on 300 unseen pairs (p = 0.002) at no cost under true evidence; for a second model the gain was half as large and not significant. For practitioners who build LLM–analysis pipelines, these findings point to a concrete risk: the input that most needs checking is the analyser’s guarantees, not its alarms, because guarantees are the evidence the models accept without looking. We return to this point in Section 7.3 and in the conclusion. The study is designed to support these claims with appropriate rigour. It comprises 916 PrimeVul pairs (1,832 functions) in four disjoint sets, eleven model configurations from four vendors, and 16,572 audits. All comparisons are paired on the same functions. Exploration and confirmation are separated: presentation strategies were developed on a development set, and the hypotheses of the confirmatory phases were fixed before the corresponding data were collected [18, 19]; Section 5.3 states how each was recorded. The behavioural coding is validated by two independent LLM annotators and a human check. We report every strategy we tried and every confirmatory test, including those that failed. The main contributions of this study are: 1. A controlled methodology for measuring how LLMs use program-analysis evidence. It consists of an obligation analyser for C/C++, six evidence conditions built from vulnerability-fixing pairs, matched contrasts that isolate the truth of the evidence, and a direction-aware measure of deference relative to true evidence about the same operations. 2. Empirical evidence of asymmetric deference. Across eleven configurations and four vendors, LLM auditors trust false assurances and check false alarms. The asymmetry was confirmed in an internally registered replication; its size varies between models and samples, and the study is limited to C/C++ functions audited in isolation. 3. An evaluation of evidence presentation. We evaluate eight ways of presenting evidence along five dimensions (content, framing, provenance, checking and timing), including a failed replication, and a pre-specified test showing that moving verification into a separate step reduces deference without losing the value of true evidence for one of two models. 4. A replication package. It contains the analyser, the evidence of every condition, all model responses, the annotations, the registrations and the analysis scripts that produce every number and figure in this paper.0 The remainder of the paper is organised as follows. Section 2 discusses related work. Section 3 reports a preliminary study that motivated the design. Section 4 describes the methodology and Section 5 the experimental setup, Section 6 the results, and Section 7 their implications and limitations. Section 8 concludes. 0

https://github.com/lzh0082-cloud/CODE-DETECTION

3

2

Related Work

Our study sits at the intersection of six strands of prior work. We summarise each briefly and indicate where the present paper departs from them.

2.1

LLMs for Vulnerability Detection

LLMs are now applied across software engineering tasks [20,21], and vulnerability detection is one of the most studied. Before LLMs, deep learning detectors learned from token sequences, code gadgets or program graphs [22–24], trained on datasets mined from vulnerability-fixing commits [25, 26]. Such detectors have repeatedly been found to perform far worse under realistic evaluation than on their original benchmarks [27]. Risse and Böhme showed that state-of-the-art models overfit to features unrelated to the vulnerability and cannot distinguish vulnerable functions from their patched versions [28]. PrimeVul [17] addressed label noise and data duplication and introduced paired evaluation; on it, code language models, including large general-purpose LLMs, rarely judge both versions of a pair correctly. Studies focusing on LLMs report the same picture: answers are non-deterministic, reasoning is often incorrect or unfaithful, and small semantics-preserving changes alter verdicts [29]. Prompting strategies and in-context information improve results only modestly [30, 31]. Steenhoek et al. argue that vulnerability detection requires multi-step reasoning about code semantics, such as relating bounds and null checks to the operations they protect, and that this reasoning is where LLMs fail [32]. Our preliminary study (Section 3) agrees: the models we study make almost no errors about local program structure, so their errors are judgements, not misreadings. We use PrimeVul’s paired data and its paired metrics, but our dependent variable is not detection performance; it is how detection changes when the model is given evidence.

2.2

Combining LLMs and Static Analysis

Static analysers trade completeness for precision in ways that limit their adoption. They miss many real bugs [33], and their false positives are a main reason why developers ignore or abandon them [5, 34, 35]. A body of work therefore post-processes static-analysis alarms to identify the actionable ones [36], although learned alarm classifiers have been found to benefit from data leakage in their evaluation [37]. LLMs offer a new way to perform this triage, and more generally to use tools as part of their reasoning [38,39]. A growing line of work combines LLMs with static analysis. LLift uses an LLM to examine the candidate bugs of a static analyser for the Linux kernel and to discard those whose preconditions are ruled out [1]; an earlier pilot used ChatGPT to prune the analyser’s false positives [3]. IRIS lets an LLM infer taint specifications that a static analyser then uses for whole-repository detection [2]. In industry, hybrid pipelines in which an LLM adjudicates static-analysis alarms remove most false positives while keeping high recall [4], and general-purpose coding agents can filter most false positives of security analysers, at the risk of also suppressing real vulnerabilities [40]. In all of these designs, analysis results enter the model’s context as evidence. Their evaluations measure the end result of the pipeline; they do not isolate how the model treats correct and incorrect evidence. Our study complements them: we hold the code and the model fixed and vary only the truth, relevance and presentation of the evidence.

2.3

LLMs and External Evidence

Natural language processing research has studied how LLMs resolve conflicts between their parametric knowledge and evidence in their context. LLMs can be highly receptive to coherent external evidence even when it contradicts what they know, while also showing confirmation bias towards 4

evidence that agrees with them [10]; the literature on such knowledge conflicts is surveyed by Xu et al. [11]. ClashEval found that models adopt incorrect retrieved content over their own correct answer in more than 60% of cases, and less often the more implausible the error is [12]. A related tendency, sycophancy, makes models agree with the views of the user [13, 41], and models often abandon a correct answer when it is challenged, even by invalid arguments [42]. These studies vary the content of factual evidence in question answering. We study evidence about program behaviour, which the model can check against the code it is given, and we find that the direction of the evidence (whether it says that the code is safe or that it is unsafe) determines whether it is checked.

2.4

Contextual Bias in LLM Security Review

Concurrent work shows that LLM security verdicts depend on context outside the code. Shahriar et al. add author attributions, task framings and prior analysis outcomes (a function “marked SAFE” or “VULNERABLE” in an automated analysis) to PrimeVul functions and find that all eight models they study are susceptible; they conclude that such cues mainly change how willing a model is to flag code rather than how well it discriminates [14]. Alexopoulos et al. show that crafted pull-request metadata biases LLM-based code review and, with iterative refinement, hides 32 of 33 real vulnerabilities from production pipelines, while blunt templates can backfire because they raise suspicion [15]. ALIBI uses code comments with fabricated safety claims, including fabricated tool reports, to evade LLM detectors, and finds that isolating the code from unverifiable comments defends better than instructions to distrust them [16]. In contrast, Thornton finds that adversarial comments have no significant effect on frontier models in a benchmark of 100 samples [43]. Beyond verdicts, correct vulnerability verdicts are often accompanied by fabricated or unverifiable reasoning [44], and in code smell detection, models adopt false premises in the prompt unless asked to reason from evidence first [45].

2.5

Hallucination, Reasoning and Verification

LLMs produce fluent statements that are unsupported by their input or by facts, a behaviour broadly termed hallucination [46, 47]. Generating a chain of reasoning improves multi-step reasoning [48], but the stated reasoning need not reflect what determines the answer [49]. Whether models can correct their own errors is contested: iterative self-feedback improves some tasks [50], but without external feedback models generally fail to correct their reasoning and may make it worse [51], and self-correction works best when reliable external feedback is available [52]. Chainof-Verification reduces hallucination by having the model plan verification questions about its draft, answer them independently and revise the draft [53]. Our verify-show strategy follows the same principle of independent verification, applied to the tool’s claims rather than to the model’s own draft; the code under audit serves as the external reference against which the claims are checked.

2.6

Automation Bias and Cognitive Forcing

Human factors research distinguishes misuse of automation, that is over-reliance, from disuse, which is often caused by false alarms [6]; appropriate reliance requires trust to be calibrated to the actual reliability of the aid [54]. Automation bias produces two kinds of error: errors of commission, when people follow incorrect advice, and errors of omission, when they miss problems the aid did not flag [55]. A systematic review of automation bias in clinical decision support found that incorrect advice leads to errors of commission, and identified the presentation of advice and the user’s verification effort as mediators [7]. In AI-assisted decision making, explanations can increase 5

over-reliance [56]; they reduce it when verifying the AI’s output is easy relative to solving the task alone [57]. Cognitive forcing functions, such as asking people to decide before they see the AI’s recommendation, also reduce over-reliance [8]. Recent work on LLM-assisted knowledge work finds that fluent assistance produces high perceived understanding with the lowest actual learning [9]. We transfer these concepts to an LLM that is the user of a tool: we ask whether the model over-relies on the tool’s assurances, and we test interventions analogous to those studied with people, namely framing, provenance, independent verification and a commit-first variant of cognitive forcing.

2.7

Pre-Registration in Software Engineering

Empirical software engineering has long-standing guidelines for controlled experiments [58] and for the statistical analysis of stochastic results [59]. LLM-based studies add threats of their own: closed models change behind their APIs, training data may overlap with evaluation data, and outputs are non-deterministic even at temperature zero [60, 61], while results can depend on seemingly irrelevant details of prompt formatting [62]. Pre-registration commits to hypotheses and analyses before the data are seen, which separates confirmatory from exploratory findings and prevents selective reporting [18]. It has been adopted by software engineering venues in the form of registered reports [19]. Studies of LLMs are especially exposed to the garden of forking paths, because prompts can be varied without limit; we therefore separate development and test data and fix the hypotheses of the confirmatory phases in advance.

2.8

Summary of the Gap

Work on LLM vulnerability detection measures how well models flag code, not how they use evidence; work that combines LLMs with static analysis evaluates whole pipelines, not the model’s treatment of correct and incorrect evidence; and work on knowledge conflicts studies factual evidence in question answering that the model cannot check against its input. Concurrent studies of contextual bias in security review come closest, and our study differs from them in three ways. First, our evidence consists of specific claims about the operations in the function, which the model can check against the code, and each false claim is compared with a true claim about the same operation; this isolates the truth of the evidence from its presence. Second, we compare false claims in both directions on matched functions and show that the models scrutinise them differently: they check alarms and trust assurances. A pure change in willingness to flag, as reported for generic cues [14], would move verdicts by similar amounts in both directions; our direction-aware measure shows that it does not. Third, we evaluate mitigations in a pre-specified test on held-out data. The adversarial studies above examine only claims that code is safe; our results explain why that is the direction an attacker would choose. The asymmetry also bears directly on alarm triage, where the model is asked to dismiss alarms, and on designs that pass the analyser’s guarantees to the model as context.

3

Preliminary Study: Where Do the Errors Come From?

The design of the main study grew out of two preliminary experiments with deepseek-chat. The first asked whether LLMs misread program structure, which static analysis could correct; the second whether static-analysis facts improve vulnerability audits.

6

3.1

Claim-Level Errors in Execution Reasoning

We asked the model to predict the output of Python programs from CRUXEval [63] (50 samples) and from the HumanEval part of REval [64] (85 samples), reasoning step by step. We extracted the factual claims of each reasoning chain into a small predicate language (for example, that a line executes, that a branch is taken, how often a loop iterates or what value a variable holds) and labelled each claim against the program’s control and data flow (static claims) or against an execution trace recorded in an isolated process (dynamic claims). With step-by-step reasoning, accuracy was saturated: 0.98 on CRUXEval and 1.00 on REval, against 0.88 and 0.89 for direct answers. Of 1,257 labelled claims, none of the 337 static claims was wrong; the errors were confined to run-specific values (3.0% of 234 dynamic claims on CRUXEval and 0.4% of 686 on REval), such as miscounted lengths or loop iterations, and several of them occurred in chains whose final answer was correct. A verifier that checked the static claims against the program graphs therefore had nothing to correct.

3.2

Static-Analysis Facts in Vulnerability Audits

We then gave the model the obligations of our C/C++ analyser (Section 4.3) while auditing 30 PrimeVul pairs. Plain step-by-step auditing judged 10 pairs correctly. Adding the facts gave 11; presenting them as a checklist of obligations to discharge reduced the number to 5–6, because the model judged more functions SAFE once it had worked through the listed obligations; a refinement step that asked the model to revisit its verdict in the light of the concerns gave 9, and five of its seven changed verdicts were patched functions turned into false alarms. Reading the answers, we found that the model’s local claims about the code were almost always correct; its errors were omissions and judgements, and in several cases it abandoned a correct judgement of its own in favour of a note.

3.3

Implications for the Main Study

The two experiments shifted our question. The models do not need static analysis to see the structure of the code; what matters is how they weigh what the analysis tells them. This question cannot be answered on 30 pairs or by measuring accuracy alone. The main study therefore manipulates the evidence while keeping the code fixed, scales the data and the models, and separates exploratory from confirmatory analyses.

4

Methodology

We study how an LLM that audits a C/C++ function for vulnerabilities uses static-analysis evidence about that function. The design manipulates the evidence while keeping the code, the model and the task fixed, so that differences in the model’s verdicts can be attributed to the evidence. This section describes the research questions (Section 4.1), the data (Section 4.2), how the evidence is produced and manipulated (Sections 4.3 and 4.4), the measures and statistics (Section 4.5), and the validation of the behavioural coding (Section 4.6); Section 5 describes the models, the audit task and the protocol that separates exploration from confirmation. Fig. 1 summarises the design.

4.1

Research Questions

RQ1 (usefulness). Does realistic static-analysis evidence improve an LLM’s vulnerability judgements? 7

PrimeVul pair vulnerable version

patched version

fix commit

(used only to build

RQ1 RQ2

Obligation analyser tree-sitter, C/C++, intraprocedural sensitive operation index, offset, deref, free, size, division holds here: dominating guards concern: what is not ensured

RQ1

C0

no notes

C1

all, true

C2

patch-relevant, true

C3

irrelevant, true

C4

false assurance

C5

false alarm

plain

RQ3

Presentation

LLM auditor deepseek-chat

quote

deepseek-flash (R)

alarms only

deepseek-v4-pro (R)

reframe

qwen-plus

provenance verify verify-show

final

commit

accuracy per version, true notes vs. none

RQ2

matched contrasts on the same operations: C4 vs C2, C5 vs C2

RQ3

disputes the note? lexical cue, 2 LLM annotators + author check

step-by-step reasoning ... [VERDICT] VULNERABLE | SAFE

commit-verify same function, only the notes vary

the conditions)

Analysis

RQ4

Evidence condition

RQ4

presentations vs. plain; dev → frozen → pre-specified test

Figure 1: Study overview. For each PrimeVul pair, an intraprocedural analyser produces proof obligations for both versions; the fix commit decides which obligations are relevant and how false evidence is built. The same function is audited under every evidence condition and, for RQ4, under different presentations of the same notes. RQ2 (relevance and correctness). How do the relevance and the correctness of the evidence change the judgements? In particular, do false assurances (“this access is checked”) and false alarms (“this access is not checked”) affect the model to the same degree? RQ3 (mechanism). When the evidence is false, does the model check it against the code, and does this differ between assurances and alarms and across models? RQ4 (presentation). Can the way evidence is presented reduce deference to false evidence without losing the value of true evidence?

4.2

Data

We use PrimeVul [17], a vulnerability dataset with de-duplicated, manually validated labels. Its paired subsets contain, for each vulnerability-fixing commit, the vulnerable version of a function and its patched version. Pairs are well suited to our questions: a model that truly reasons about the code should flag the vulnerable version and clear the patched one, and the patch localises the code that matters. We take the release v0.1 of the paired splits and link the two versions of a pair through their commit. Study set From the paired test split (433 linkable pairs) we keep pairs whose CWE (Common Weakness Enumeration) category describes a structural defect that intraprocedural evidence can speak to (out-of-bounds access: CWE-119/120/122/125/787; NULL dereference: CWE-476; use after free, double free and leaks: CWE-416/415/401; divide by zero and integer over/underflow: CWE-369/190/191; infinite loop: CWE-835), whose two versions differ, and whose versions both have at most 150 lines. This yields 187 pairs (374 functions) from 60 projects; the largest are TensorFlow (38 pairs), the Linux kernel (26) and Vim (14), and the most frequent CWEs are outof-bounds write (CWE-787, 46 pairs), NULL dereference (CWE-476, 32) and out-of-bounds read (CWE-125, 28).

8

Development and test sets To evaluate presentation strategies without fitting them to the data they are evaluated on, we use two further splits that we did not inspect before the corresponding phase (Section 5.3). The development set contains all 237 eligible pairs of the paired validation split. The test set is a random sample (seed 7) of 300 of the 1,390 eligible pairs of the paired training split; since we do not train models, the training split is simply unseen data. No function hash is shared between the three splits. Held-out replication set An earlier replication (Section 6.5) used the 192 pairs of the paired test split that are disjoint from the study set (any CWE, at most 300 lines).

4.3

Static-Analysis Evidence: Proof Obligations

To produce evidence whose truth we control, we implemented an intraprocedural analyser for C and C++ on top of tree-sitter [65]. For a function it lists the sensitive operations (array indexing, pointer offsets, dereferences of pointers obtained from calls, frees, divisions, copy and allocation sizes, and computed length arguments) and, for each operation, a proof obligation with two parts: • holds here: the conditions that hold on every path reaching the operation, derived from enclosing if/else arms, loop conditions, the left operands of short-circuit operators and ternaries, and earlier early-exit checks in enclosing blocks (after if (c) return; the condition !c holds); • concern: what the function itself does not guarantee, for example a missing upper or lower bound on an index, a signed index without a non-negativity check, a size computed with a multiplication of unbounded values, a use after a free, or a return path that drops an allocation. The analyser handles functions whose return type PrimeVul has stripped from the first line, attribute macros, sizeof operands, pool allocators and returns that follow a free. It is exact for the syntax it models and deliberately does not reason about callers and callees; the notes state this. A function’s obligations are rendered as a numbered list, operations with concerns first, at most twelve per function (Fig. 2). O1 [array index] line 13: ‘avc->pps[pps_id]‘ holds here: ‘!(pps_id > 255)‘ (early exit, line 11) concern: ‘pps_id‘ is signed (‘s32‘) and no lower-bound (>= 0) check dominates this access

Figure 2: Notes for the vulnerable version of a PrimeVul pair (gpac, CWE-120), wrapped to the column width. In the patched version the check at line 11 becomes (pps_id<0) || (pps_id > 255), the holds-here line changes accordingly and the concern disappears.

4.4

Evidence Conditions

For every pair we derive six evidence conditions (Table 1). Relevance is defined by the patch: an obligation is patch-relevant if its operation, or a line its concerns refer to, lies near a changed hunk (three lines before to eight lines after) and it mentions an identifier that occurs in the changed lines. The patch is used only to build the conditions; the model never sees it. 9

Table 1: Evidence conditions. Functions: number of study-set functions to which a condition applies. Evidence

Content

Functions

C0 C1 C2 C3 C4

none all, true relevant, true irrelevant, true false assurance

C5

false alarm

plain audit every obligation the analyser reports only patch-relevant obligations only obligations away from the patch that carry a concern vulnerable versions: relevant concerns replaced by fabricated, refutable guarantees patched versions: concerns the patch removed, guards deleted

374 374 189 183 83 97

False evidence is constructed so that it can be refuted by reading the function. In C4 every concern of the patch-relevant obligations of the vulnerable version is replaced by a fabricated guarantee that cites a concrete line (for instance “pps_id >= 0 is checked before this access (line 12)” where line 12 is return -1;). In C5 the patched version receives the concerns that the patch removed, transplanted onto the same kind of operation, with the contradicting guard facts deleted; when no such concern exists, a true guard of a patch-relevant (or, failing that, any) operation is declared not to protect it. We record which of the three constructions was used.

4.5

Measures and Statistics

A verdict is correct if it is VULNERABLE for a vulnerable version and SAFE for a patched one. We report accuracy separately for vulnerable versions (recall) and patched versions (specificity), and the paired outcome of a pair (both correct, both VULNERABLE, both SAFE, both wrong) following PrimeVul [17]. Responses without a verdict count as wrong; API failures are treated as missing. Following guidelines for experiments in software engineering [58, 59], all comparisons are paired on the same functions and we report effect sizes with confidence intervals, not only p-values. We test differences in accuracy with the exact McNemar test [66] and report bootstrap 95% confidence intervals (2,000 resamples) [67] of the difference; single proportions have Wilson intervals. Our main contrasts compare false evidence with true evidence about the same operations (C4 vs. C2 on vulnerable versions, C5 vs. C2 on patched versions), which holds the presence and placement of notes constant and varies only their truth. When several presentation strategies are compared with a baseline, p-values are Holm-corrected [68]. Direction-aware measure Accuracy contrasts conflate deference with a model’s leaning: a model that flags almost every function cannot be pushed further towards VULNERABLE by a false alarm, and loses little recall to a false assurance. We therefore measure how far false evidence moves verdicts towards its own claim, relative to true notes about the same operations: ∆as = P (SAFE | C4) − P (SAFE | C2),

(1)

∆al = P (VULN | C5) − P (VULN | C2),

(2)

A = ∆as − ∆al ,

(3)

where SAFE and VULN denote a SAFE and a VULNERABLE verdict. The assurance shift ∆as is computed on vulnerable versions and the alarm shift ∆al on patched versions, each on the functions that have both conditions. Each shift lies in [−1, 1]: a value of +1 means that the false evidence 10

turns every verdict towards its claim, 0 that it has the same effect as true notes about the same operations, and negative values that it pushes verdicts away from its claim (it backfires). The asymmetry A lies in [−2, 2]; it is positive when false assurances move verdicts towards their claim more than false alarms do, and zero when the two are followed equally, as a pure change in a model’s willingness to flag would produce. No smoothing or guard term is involved. Responses without a verdict count as neither SAFE nor VULNERABLE. The measure does not capture whether the model’s reasoning is correct, only where its verdicts move. When and why the measure was introduced is described in Section 5.3.

4.6

Validating the Behavioural Coding

For RQ3 we need to know whether an answer disputes a note. LLMs can match or exceed crowd workers on some annotation tasks [69], but their reliability for software engineering artefacts varies by task and must be checked against human labels [70]; we therefore validate both the automatic rule and the LLM annotators. A lexical rule flags answers that state that a note is wrong, inaccurate, a false positive or does not match the code. To validate it, we drew 80 C4/C5 answers stratified by the rule (40 flagged, 40 not). Two LLM annotators (Claude and deepseek-v4-pro) labelled each answer independently with a written guide (Appendix B): whether it checks a note against the code, whether it rejects a false note, and whether its verdict relies on a note at face value. Agreement is measured with Cohen’s κ [71]. The first author independently labelled a random subset of 20 answers. Population rates are estimated by reweighting the strata with the rule’s hit rate among all C4/C5 answers.

5

Experimental Setup

This section describes the model configurations compared, the audit task, and the protocol that separates exploration from confirmation.

5.1

Models Compared

We use four model configurations from two vendors: DeepSeek’s flash model without reasoning (deepseek-chat) and with reasoning (deepseek-flash), the larger deepseek-v4-pro with reasoning, and Alibaba’s qwen-plus-2025-12-01 without reasoning. Reasoning models run at the low reasoning effort; at the default effort their reasoning frequently exhausted a 32k-token budget before a verdict was produced. All calls use temperature 0; budgets are 8k tokens for non-reasoning and 32k for reasoning models. Because the larger reasoning model needs about six minutes per call, it was run on a random 60-pair subset of the study set and only on C0, C2, C4 and C5; qwen-plus was run on C2, C4 and C5. For the registered multi-vendor replication (H3, Section 5.3) we added seven further models from four vendors, all accessed through Alibaba Cloud Bailian’s OpenAI-compatible endpoint with reasoning disabled (Table 2). None of them had been run on any data of this study before the hypothesis was registered. They were run on the study set under C2, C4 and C5, which suffices for the contrasts of RQ2 and RQ3. In total the study covers eleven model configurations from four vendors. All models are used for inference only; no model training or fine-tuning is performed. Why manipulate the evidence rather than use a production analyser? Three reasons motivate this choice. First, only constructed evidence lets us know the truth of every note and compare a false note with a true note about the same operation; with a production tool, the truth 11

Table 2: Model configurations. R: reasoning enabled (effort low). Sets: S study, H held-out, D development, T test. Conditions in brackets were run on a subset only. Name in this paper API model identifier

R Vendor

Sets (conditions)

deepseek-chat deepseek-flash (R) deepseek-v4-pro (R) qwen-plus

DeepSeek ✓ DeepSeek ✓ DeepSeek Alibaba

S (all); H, D (C1, C2, C4, C5); T (C2, C4) S (all) S, 60 pairs (C0, C2, C4, C5) S (C2, C4, C5); T (C2, C4)

deepseek-chat deepseek-flash deepseek-v4-pro qwen-plus-2025-12-01

Registered replication (H3), via Alibaba Cloud Bailian glm-5.2 glm-5.2 kimi-k2.6 kimi-k2.6 kimi-k3 kimi-k3 qwen3.7-plus qwen3.7-plus-2026-05-26 qwen3.7-max qwen3.7-max-2026-06-08 qwen3.7-flash qwen3.7-flash-2026-07-15 deepseek-v4.1-flash deepseek-v4.1-flash

Zhipu AI Moonshot AI Moonshot AI Alibaba Alibaba Alibaba DeepSeek

S (C2, C4, C5) S (C2, C4, C5) S (C2, C4, C5) S (C2, C4, C5) S (C2, C4, C5) S (C2, C4, C5) S (C2, C4, C5)

of each warning would have to be established by hand and could not be varied on the same function. Second, the obligation format states guarantees and concerns explicitly, which is the information that pipelines pass to the model and the distinction our questions are about. Third, keeping the code, model and task fixed while only the notes change is what makes the contrasts paired and the effects attributable. We view evidence from production analysers as a complementary, not competing, direction (Section 7.4).

5.2

Audit Task

The audit prompt shows the numbered function, the project and file name and, except in C0, the notes with a one-paragraph legend. It asks the model to reason step by step about untrusted values, the operations they reach and the checks that guard them, and to end with [VERDICT] VULNERABLE or [VERDICT] SAFE. The prompts are listed in Appendix A.

5.3

Exploration and Confirmation

RQ1–RQ3 are answered on the study set. For RQ4 we first explored presentation strategies on the study set and tested the most promising one on the held-out replication set; it did not replicate (Section 6.5). We therefore adopted a stricter protocol (Fig. 3): every further strategy was developed and compared on the development set only; the best one was frozen and the following hypotheses and analyses were fixed before the test set was run once. Unlike H3 below, they were not written into a separate registration document with hashes, so we call this test pre-specified rather than registered: H1 With the frozen strategy, the accuracy of deepseek-chat on vulnerable versions under false assurances (C4) is higher than with the plain presentation (paired McNemar test, two-sided, α = 0.05, no further correction). H1b The same holds for qwen-plus. H2 With the plain presentation, accuracy under C4 is lower than under true relevant evidence (C2) on the same vulnerable functions. As a constraint we report the change in accuracy under true relevant evidence (C2). 12

1 Study set

2 Held-out

3 Development

4 Test (run once)

PrimeVul test, 187 pairs

192 disjoint pairs

PrimeVul valid, 237 pairs

PrimeVul train, 300 pairs

RQ1-RQ3

replicate reframe

verify, verify-show,

4 presentations explored

C4 +0.08, p=.302

commit, commit-verify

C4 +0.12, p=.002

freeze verify-show

H1b qwen-plus

best: reframe on C4 +0.18, p<.001

not replicated

stricter protocol

C4 +0.13, p=.011

H1 deepseek-chat

fix H1, H1b, H2

C4 +0.06, p=.134 H2 C4−C2, plain chat -0.06, p=.150 qwen -0.11, p=.005

Figure 3: Order of the RQ4 evaluations. After the strategy selected on the study set failed to replicate, strategies were compared on a separate development set only, and the frozen strategy was tested once on unseen pairs with pre-specified hypotheses. Values are accuracy changes under false assurances relative to the plain presentation (uncorrected McNemar p). Direction-aware measure and H3 The direction-aware measure (Equations (1) to (3)) was defined after the qwen-plus study-set run, that is, after seeing the data of the first four configurations; those results are therefore exploratory. Before running any further model, we wrote a registration document and recorded the SHA-256 hashes of it and of the analysis script: H3 Over seven new models, the mean asymmetry is positive: false assurances move verdicts towards their claim more than false alarms do (bootstrap over pairs, resampled jointly for all models, 2,000 resamples, one-sided, α = 0.05). The registration fixed the models, data, conditions, analysis script (with its hash) and the handling of missing data (Appendix C). The registration is internal: the document and the hashes were written before the data were collected, but they were first deposited publicly, with the replication package, after data collection, and no third party time-stamped the earlier date. We also tested, and rejected before registration, a different post-hoc explanation (that models follow evidence congruent with their leaning): flip rates conditioned on the baseline verdict were dominated by regression to the mean, as content-neutral changes of the prompt produced similar flip rates.

6

Results and Analysis

We first describe how true evidence affects the audits (RQ1), then contrast true and false evidence about the same operations (RQ2), examine whether the models check the evidence (RQ3), and finally report every presentation strategy we evaluated, including the ones that failed (RQ4). Unless stated otherwise, numbers refer to the study set (187 pairs) and differences are paired on the same functions.

6.1

RQ1: Does True Static-Analysis Evidence Help?

Fig. 4 shows accuracy per condition, separately for vulnerable and patched versions. Two observations stand out. First, the models differ mainly in their bias, not in their discrimination. Without evidence (C0), deepseek-chat judges 36% of the functions vulnerable and finds 45% of the vulnerable versions, whereas deepseek-flash with reasoning judges 69% vulnerable, finds 78% of the vulnerable versions, but clears only 36% of the patched ones. The larger deepseek-v4-pro is

13

Accuracy (95% Wilson CI)

1.0

Vulnerable versions (correct = VULNERABLE)

Patched versions (correct = SAFE)

0.8 0.6 0.4 deepseek-chat

0.2 0.0

deepseek-flash (R) qwen-plus

C0

C1

false

false

evidence

C2

C3

evidence

C4

C0

C1

C2

C3

C5

Figure 4: Accuracy per evidence condition on the study set, for vulnerable versions (left) and patched versions (right). Conditions apply to different subsets of functions (Table 1), so the figure is descriptive; paired comparisons are in Fig. 5. False evidence exists only for one version: false assurances (C4) for vulnerable versions and false alarms (C5) for patched ones. even more inclined to flag (79% on its 60-pair subset). Reasoning thus shifts the operating point towards suspicion rather than separating the two versions of a pair better. Second, true evidence barely moves accuracy. Showing every obligation the analyser reports (C1) changes accuracy by +0.02 for both DeepSeek flash configurations (95% CI −0.03 to +0.07 and −0.02 to +0.07; p = 0.52 and p = 0.41). Restricting the notes to the operations that the fix touches (C2) yields a larger but non-significant gain (+0.06, CI −0.01 to +0.12, for deepseek-chat; +0.04 for deepseek-flash), and true notes about other operations (C3) have no effect (+0.00 and +0.01). For deepseek-chat the small gain under C2 is spread over both versions (+0.06 on vulnerable and +0.05 on patched versions, both n.s.); for deepseek-flash relevant notes counteract its bias, raising accuracy on patched versions by 0.13 (p = 0.036) while lowering it on vulnerable ones by 0.04 (n.s.). Relevant notes thus seem to help by pointing the model at the operation that matters, not by telling it something it could not see in the code. (The models of the replication and qwen-plus were run only on the conditions needed for RQ2 and RQ3, so RQ1 rests on the two DeepSeek flash configurations and, for C2 vs C0, on deepseek-v4-pro: +0.04, CI −0.06 to +0.15.) Answer to RQ1. Realistic, correct static-analysis evidence does not make the audits measurably more accurate. Even evidence restricted to the operations that the fix touches yields only a small, non-significant gain, mostly by drawing attention to the right operation.

6.2

RQ2: Relevance and Correctness of the Evidence

To isolate the effect of correctness, we compare false and true notes about the same operations of the same functions (Fig. 5 and Table 3). The two kinds of false evidence have opposite effects. False assurances mislead. When the notes on a vulnerable function claim that the relevant check exists (C4), the accuracy of deepseek-chat drops from 0.75 to 0.64 relative to the true notes (C2): ten vulnerable functions that were found with true notes are cleared with false ones, and only one moves in the other direction (p = 0.012). The larger deepseek-v4-pro shows the same drop on its 60-pair subset (0.97 → 0.83, CI −0.28 to −0.03, p = 0.125 given only 29 functions). The reasoning variant of deepseek-flash and qwen-plus are not misled (+0.04 and −0.01). Both lean strongly towards VULNERABLE (qwen-plus flags 77% of the functions under C2 and clears only

14

Table 3: Matched contrasts on the study set. n: functions with both conditions. Discordant: functions correct only under the true notes / only under the false notes. Model

Contrast

n

True → false

Discordant

p

Difference (95% CI)

deepseek-chat

C4 vs C2 C5 vs C2 C4 vs C2 C5 vs C2 C4 vs C2 C5 vs C2 C4 vs C2 C5 vs C2

83 67 83 67 29 28 83 67

0.75 → 0.64 0.67 → 0.79 0.87 → 0.90 0.37 → 0.55 0.97 → 0.83 0.29 → 0.32 0.88 → 0.87 0.31 → 0.39

10 / 1 0/8 3/6 3 / 15 4/0 4/3 6/5 5 / 10

0.012 0.008 0.508 0.008 0.125 1.000 1.000 0.302

−0.11 (−0.18, −0.04) +0.12 (+0.04, +0.19) +0.04 (−0.04, +0.11) +0.18 (+0.06, +0.30) −0.14 (−0.28, −0.03) +0.04 (−0.14, +0.21) −0.01 (−0.08, +0.07) +0.07 (−0.03, +0.18)

deepseek-flash (R) deepseek-v4-pro (R)† qwen-plus †

60-pair random subset of the study set. (R): reasoning enabled.

31% of the patched versions), and both find 87–90% of the vulnerable versions whatever the notes say. False alarms do not. When the notes on a patched function claim that a check is missing (C5), accuracy does not fall; it rises, from 0.67 to 0.79 for deepseek-chat (0/8 discordant pairs, p = 0.008) and from 0.37 to 0.55 for deepseek-flash (p = 0.008). A false alarm draws the model to the operation, the model then finds the guard that the alarm denies, and it clears the function more often than when the true guard is merely listed. For deepseek-v4-pro (+0.04) and qwen-plus (+0.07) the change is not significant, but in no model does a false alarm lower accuracy. The effect of false evidence therefore depends on its direction. In all four configurations the effect of a false assurance is below that of a false alarm, and only false assurances ever lower accuracy; how much they do so differs between models, from no effect for the two models biased towards VULNERABLE to 11–14 points for the others. Answer to RQ2. Correctness matters asymmetrically. A fabricated alarm about a fixed function never significantly lowers accuracy and for two configurations improves it. Relative to true notes about the same operations, a fabricated assurance that a check exists lowers the detection of real vulnerabilities by 11 points for deepseek-chat (p = 0.012); deepseek-v4-pro shows a drop of similar size that is not significant on its 29 functions, and models that already flag most functions are less affected in accuracy terms. Section 6.4 examines the asymmetry across vendors with a direction-aware measure. True evidence helps only when it is relevant, and then only slightly.

6.3

RQ3: Do the Models Check the Evidence?

The asymmetry suggests that the models scrutinise alarms but accept assurances. Fig. 6(a) supports this with the lexical cue: deepseek-chat disputes a false alarm in 80% of its answers (78/97) but a false assurance in only 28% (23/83); for deepseek-flash the rates are 49% and 14%, and for deepseek-v4-pro 26% and 7%. qwen-plus is the exception: it disputes false assurances more often than false alarms (45%, 37/83, versus 36%, 35/97). When it disputes a false assurance, its verdict is almost always correct (35 of 37), which is consistent with its immunity to false assurances in RQ2; when it disputes a false alarm, it still judges the patched function vulnerable in 14 of 35 cases. We read a sample of the flagged qwen-plus answers to make sure that the cue, validated on DeepSeek answers, captures the same behaviour; it does (for instance, “line 18 is a comment, not a check. That means the analysis note is incorrect or misleading”). The cue is conservative: against the annotations its precision is 0.97–1.00 and its recall 0.70–0.76, so the absolute rates are lower bounds, but there is no reason for the misses to depend on the condition. 15

false assurance (C4 − C2, vulnerable) false alarm (C5 − C2, patched) -0.11*

deepseek-chat

+0.12** +0.04

deepseek-flash (R)

+0.18**

-0.14

deepseek-v4-pro (R) †

+0.04 -0.01

qwen-plus

exploratory replication (H3)

-0.19**

glm-5.2

+0.07 +0.07

-0.01 -0.04

kimi-k2.6 -0.19***

kimi-k3

+0.07

qwen3.7-plus

-0.06

qwen3.7-max

-0.13* -0.14*

qwen3.7-flash

+0.07 +0.06 +0.03

-0.05

deepseek-v4.1-flash −0.4

−0.2

0.0

+0.15*

0.2

0.4

Accuracy change vs. true evidence on the same operations Figure 5: Effect of false evidence relative to true evidence about the same operations, with 95% bootstrap CIs. Orange: false assurance on vulnerable versions (C4 − C2); blue: false alarm on patched versions (C5 − C2). The upper block shows the four exploratory configurations and the lower block the seven models of the registered replication (Section 6.4). * p < .05, ** p < .01 (exact McNemar). † 60-pair subset. 16

The annotations confirm the gap (Fig. 6(b)). The two LLM annotators agreed well on whether an answer rejects the false note (κ = 0.82) and on whether its verdict relies on a note at face value (κ = 0.75), less on whether an answer checks the notes at all (κ = 0.52, raw agreement 85%). The first author’s independent labels of 20 answers agreed with both annotators on every item. Reweighted to all C4/C5 answers, the share of answers that reject the false note is 0.43–0.50 for false assurances and 0.70–0.81 for false alarms, depending on the annotator. The answers show how the asymmetry arises. Faced with an alarm, the models typically go to the cited operation, quote the guard and conclude that the note “appears to be a false positive”. Faced with an assurance, they more often restate it as a fact (“the static analysis notes confirm . . . ”) and move on to other operations. A claim that a check is missing invites a search for the check; a claim that it is present ends the search. Can the models refute false assurances when asked to? A different reading is that false assurances are simply harder to refute: refuting an alarm means finding a check that exists, whereas refuting an assurance means establishing that a check is absent. To separate not checking from not being able to check, we ran the verification call of verify-show (Appendix A), unchanged, with deepseek-chat on every claim of the C2, C4 and C5 notes of the study set (357 calls). A claim of C4 or C5 counts as false if it does not occur among the true C2 claims of the same function. When asked to check, the model refuted false assurances about as often as false alarms (Table 4): 0.76 versus 0.79 of the claims. It also discriminated well between true and false assurances (2% of true holds-here claims refuted), whereas it refuted 45% of the true concerns, a general scepticism towards alarms that matches the backfire in RQ2. At the level of functions the picture is less even: on the 57 pairs with both conditions, all false claims were refuted for 36 vulnerable versions under C4 and for 48 patched versions under C5 (difference −0.21, 95% CI −0.37 to −0.05). Among the false alarms, those that negate a guard of an arbitrary operation were refuted less often (0.70) than transplanted concerns (0.93) and negated guards of relevant operations (1.00). Overall, the model is able to refute most false assurances once it is asked to, which supports the reading that the asymmetry in the audits arises mainly from whether the model checks a claim rather than from whether it can; the residual function-level difference shows that the two cannot be fully separated in this design, since the direction of a claim is tied to whether refuting it requires finding a check or establishing its absence. This check used one model. Table 4: Verification of true and false claims when the model is asked to check them (deepseek-c hat, study set). Shares of claims; refuted with Wilson 95% CI. Claims

n

Refuted

C4 false assurances 251 0.76 (0.70, 0.81) C5 false alarms 428 0.79 (0.75, 0.82) C2 true holds-here 203 0.02 (0.01, 0.06) C2 true concerns 488 0.45 (0.41, 0.50)

Confirmed Unclear 0.10 0.06 0.90 0.31

0.14 0.15 0.08 0.24

Answer to RQ3. The three DeepSeek configurations verify false alarms against the code far more often than false assurances (annotated: roughly three in four versus one in two answers). We call this pattern asymmetric scrutiny: evidence that the code is safe is trusted, evidence that it is unsafe is checked. It is not universal: qwen-plus checks assurances at least as often as alarms, and it is also the model that false assurances do not mislead. When explicitly asked to check, deepseek-chat refutes false assurances about as often as false alarms (0.76 vs. 0.79 of the claims), so the gap lies mainly in whether the claims are checked. 17

Answers disputing a note

false assurance (C4)

false alarm (C5)

0.8 0.6 0.4 0.2 0.0 e de

ps

ha k-c

t

R) h(

as ro k-fl 4-p s ee p k-v e ee s de ep de

ee

†

) (R

en qw

us -pl

2 2 .6 -5. i-k glm kim

3 us sh sh ax i-k fla fla -pl -m .1 .7 kim 3 .7 3 .7 v4 n3 en en e k w w q q qw s ee ep de

(b) Annotated, reweighted (95% bootstrap CI)

Answers rejecting the false note

(a) Lexical dispute cue, all answers (lower bound)

1.0

1.0

0.81 0.70

0.8 0.6

0.43

0.50

0.4 0.2 0.0 annotator 1

annotator 2

(Claude)

(deepseek-v4-pro)

Figure 6: (a) Share of answers that dispute a note according to the lexical cue, by model and type of false evidence (Wilson 95% CIs). (b) Share of answers that reject the false note according to each annotator, reweighted from the 80-answer stratified sample to all C4/C5 answers (bootstrap 95% CIs).

6.4

Replication Across Vendors (H3)

Fig. 7 shows the direction-aware measure (Equations (1) to (3)) for all eleven configurations. In the four exploratory configurations, false assurances moved verdicts towards SAFE by +0.03 on average, while false alarms moved verdicts away from VULNERABLE (−0.10); the mean asymmetry was +0.13 (95% CI +0.04 to +0.22). The registered replication confirmed H3. Over the seven new models the mean asymmetry was +0.16 (95% CI +0.10 to +0.24, one-sided bootstrap p < 0.001). It was positive for six of the seven models (sign test, secondary, p = 0.06) and its confidence interval excluded zero for five. False assurances shifted the new models towards SAFE by +0.11 on average, and false alarms shifted them away from VULNERABLE by −0.06: the backfire of false alarms that we observed for DeepSeek also appears in other vendors’ models. The exception is kimi-k2.6, which leans strongly towards VULNERABLE (79% under C2, 22% accuracy on patched versions), is moved by neither kind of false evidence, and gives no verdict in 24 of its 369 answers. In accuracy terms (Fig. 5), false assurances lowered the detection of vulnerable versions significantly for four of the seven new models: glm-5.2 and kimi-k3 by 0.19 (p = 0.002 and p = 0.001), qwen3.7-flash by 0.14 (p = 0.012) and qwen3.7-max by 0.13 (p = 0.035). No false alarm lowered accuracy significantly in any of the eleven configurations. The lexical cue repeats the pattern of RQ3: six of the seven new models disputed false alarms more often than false assurances (Fig. 6(a)), for instance kimi-k3 61% versus 24% and deepseek-v4.1-flash 75% versus 39%; kimi-k2.6 disputed both at similar rates (53% and 58%). Replication. Across eleven models from four vendors, false assurances pull verdicts towards their claim, while false alarms are checked and often backfire. The asymmetry was registered and confirmed on seven models not used to derive it (+0.16, 95% CI +0.10 to +0.24); its size varies between models, and a model that flags nearly every function is moved by neither.

6.5

RQ4: Presenting Evidence

Since the models already check alarms, we looked for presentations that make them check assurances too, without losing what true evidence offers. Table 5 lists the strategies along the dimensions of 18

Asymmetry (assurance − alarm)

Shift towards the claim of the false note Exploratory (hypothesis-generating) deepseek-chat

+0.18

deepseek-flash (R)

+0.11

†

+0.14

deepseek-v4-pro (R)

qwen-plus

+0.09

pooled (4 models)

+0.13

Registered replication (H3) +0.26

glm-5.2 kimi-k2.6

-0.09

kimi-k3

+0.30

qwen3.7-plus

+0.13

qwen3.7-max

+0.19

qwen3.7-flash

+0.17

deepseek-v4.1-flash

+0.19

pooled (7 models)

+0.16

−0.3

−0.2

−0.1

false assurance: share SAFE, C4 − C2

0.0

0.1

0.2

0.3

0.4

−0.2

0.0

0.2

0.4

0.6

false alarm: share VULNERABLE, C5 − C2

Figure 7: How far false evidence moves verdicts towards its own claim, relative to true notes about the same operations (left), and the asymmetry between false assurances and false alarms (right), with 95% bootstrap CIs over pairs. Top: the four configurations from which the measure was derived; bottom: the seven models of the registered replication. Diamonds: mean over the models of a group. † 60-pair subset. Table 5: Presentation strategies. All use the same notes; only their presentation changes. Strategy

Dimension

Change relative to plain

quote

framing

alarms reframe provenance verify

content framing source checking

verify-show

checking

commit

timing

commit-verify

timing + checking

instruction to treat notes as unverified claims and to quote the code before relying on one “holds here” assurances removed; operations and concerns kept assurances recast as claims to verify (“to verify: the tool reports . . . ”) the source text of every line a note cites is printed next to the citation a separate call checks each claim against the code; refuted claims are withheld as verify, but refuted claims are listed separately as contradicted by the code the model first audits without notes, then reconsiders with them (two turns) commit with the notes of verify-show

the design space: what is shown, how it is framed, whether its source is shown, and when and by whom it is checked. Fig. 9 reports every strategy we evaluated, in the order in which we evaluated them (Fig. 3). Exploration on the study set Of the first four strategies, reframing assurances as claims to verify was by far the most effective: accuracy under false assurances rose from 0.64 to 0.82 (16 functions gained, one lost; Holm-adjusted p = 0.001) with no significant loss under true evidence. Asking the model to quote the code before relying on a note helped less (+0.11, Holm p = 0.15). Removing assurances altogether (alarms) and showing provenance had no effect on false assurances, 19

Function (excerpt)

Call 1: verify each claim

Call 2: audit

10

pps_id = gf_bs_read_ue(bs);

Check every claim against the

code + confirmed notes

11

if (pps_id > 255)

code only; quote the deciding

+ claims the verifier refuted,

12

return -1;

line(s). CONFIRMED /

shown separately:

13

pps = &avc->pps[pps_id];

REFUTED / UNCLEAR. Claims ... CONTRADICTED by the code (do not rely on them):

pps_id is signed: a negative value

- array index line 13 --

reaches line 13 unchecked

pps_id >= 0 is checked ...

Analyser notes, C4 (false) O1 [array index] line 13 holds here: !(pps_id > 255) pps_id >= 0 is checked before this access (line 12)

Verifier output [1] CONFIRMED

... nothing prevents a negative pps_id from indexing pps[] ...

"if (pps_id > 255)" [2] REFUTED "12: return -1;"

[VERDICT] VULNERABLE

Figure 8: Verify-show on a false assurance (illustrative, simplified from a study-set pair). A first call checks each claim of the notes against the code and quotes the deciding line; the audit then sees the confirmed notes and, separately, the claims that the code contradicts. and removing assurances cost 10 points of recall under true relevant evidence (p = 0.021): the assurances that mislead under C4 are useful under C2. Showing provenance did not help even though the printed source line contradicted the false note; the models did not use the contradiction unless asked to check. A failed replication Reframing was selected on the same functions on which it was evaluated. To check that the gain was not an artefact of this selection, we pre-specified a replication on the 192 held-out pairs. It did not replicate: accuracy under false assurances rose from 0.38 to 0.45 (10 gained, 5 lost, p = 0.30). Deference itself was also weaker on these pairs (gap −0.09, p = 0.18), which leaves less to recover, but the gain was clearly smaller than on the study set. We take this as evidence that the selection had overestimated the effect, and we changed the protocol: all further strategies were developed on a separate development set, and one was frozen and tested once. Development On the development set we compared four strategies that move the checking out of the audit or before it (Fig. 9, development rows). Checking each claim in a separate call and listing the refuted claims next to the confirmed notes (verify-show, Fig. 8) raised accuracy under false assurances from 0.59 to 0.72 (18 gained, 5 lost; Holm-adjusted p = 0.032) without a significant loss under true evidence. Withholding the refuted claims instead (verify) helped less (+0.06): the explicit statement that the code contradicts a note appears to matter, not just its absence. Committing to an initial verdict before seeing the notes (commit) was the only strategy that made true evidence useful (+0.05 under C1, p = 0.023; +0.09 on patched versions under C2, p = 0.076), but it did little against false assurances (+0.05), and combining it with verification did not add the two effects (+0.06 under C4). Following the rule we had fixed in advance, verify-show became the final strategy. Pre-specified test On the 300 test pairs, run once after the hypotheses were fixed, verify-show raised the accuracy of deepseek-chat under false assurances from 0.54 to 0.66 (+0.12, CI +0.05 to +0.19; 23 functions gained, 6 lost; p = 0.002), confirming H1. Accuracy under true relevant 20

False assurance (C4): accuracy change vs. plain

True relevant evidence (C2): accuracy change vs. plain

quote [Study set]

+0.11* (n=83)

-0.05 (n=96)

alarms [Study set]

+0.04 (n=83)

-0.10* (n=96)

reframe [Study set]

+0.18*** (n=83)

-0.05 (n=96)

provenance [Study set]

+0.01 (n=83)

-0.04 (n=96)

reframe [Held-out]

+0.08 (n=66)

+0.02 (n=82)

verify [Development]

+0.06 (n=100)

-0.03 (n=120)

verify-show [Development]

+0.13* (n=100)

-0.03 (n=120)

commit [Development]

+0.05 (n=100)

+0.02 (n=120)

commit-verify [Development]

+0.06 (n=100)

verify-show [Test]

+0.12** (n=141)

+0.00 (n=167)

verify-show [Test, qwen-plus]

+0.06 (n=141)

-0.05 (n=167)

−0.2

0.0

0.2

0.4

exploratory (study / held-out / dev)

0.6

not run

−0.2

0.0

0.2

0.4

0.6

pre-specified test, run once

Paired difference in accuracy on vulnerable versions (95% bootstrap CI; * McNemar p < . 05, ** p < . 01, *** p < . 001, uncorrected)

Figure 9: Every presentation strategy evaluated, in the order of evaluation: accuracy change relative to the plain presentation on vulnerable versions under false assurances (left) and under true relevant evidence (right). Hollow markers: exploratory comparisons; filled markers: the pre-specified test, run once. Blue: deepseek-chat; pink: qwen-plus. evidence was unchanged (+0.00, CI −0.04 to +0.05, on 338 functions). The share of C4 answers that dispute a note doubled, from 56 to 114 of 141. H1b, the same hypothesis for qwen-plus, was not confirmed. Verify-show raised its accuracy under false assurances from 0.78 to 0.84 (+0.06, CI −0.01 to +0.13; 15 gained, 7 lost; p = 0.13), in the same direction as for deepseek-chat but half as large, and accuracy on vulnerable versions under true relevant evidence fell by 0.05 (CI −0.11 to +0.00, p = 0.11). Verification also changed the answers of qwen-plus less: the share of C4 answers that dispute a note did not increase (61 vs. 54 of 141). H2, the replication of deference, gave mixed results. For deepseek-chat, accuracy under false assurances was 0.06 lower than under true evidence on the same functions, but the difference was not significant (CI −0.14 to +0.01, p = 0.15); the direction agrees with the study set (−0.11) and the held-out set (−0.09). For qwen-plus, which false assurances had not misled on the study set, the drop on the test set was 0.11 (from 0.89 to 0.78, CI −0.19 to −0.04, p = 0.005). Deference to false assurances is therefore not a property of one model, but its size varies between models and samples. In an analysis that we did not pre-specify and therefore label post hoc, we pooled the disjoint sets on which each model was run under the plain presentation. For deepseek-chat (study, held-out and test sets; 290 vulnerable functions) false assurances lowered accuracy by 0.08 (from 0.61 to 0.53, CI −0.13 to −0.03; 40 lost, 16 gained; p = 0.002); for qwen-plus (study and test sets; 224 functions) by 0.08 as well (from 0.89 to 0.81, CI −0.13 to −0.02; 29 lost, 12 gained; p = 0.012).

21

Answer to RQ4. Instructions and framing inside the audit prompt did not reliably reduce deference: the one that worked on the study set did not replicate. Moving verification into a separate step and showing its refutations did: in a pre-specified test it recovered 12 points of accuracy under false assurances for deepseek-chat at no cost under true evidence. For qwen-plus the gain was smaller (+0.06) and not significant.

7

Discussion

7.1

Why Are Assurances Trusted and Alarms Checked?

The answers suggest a simple mechanism. An alarm states that something is missing at a specific operation; the natural response is to look for it, and when the model looks, it usually finds the guard that the alarm denies. An assurance states that the work has already been done; it gives the model a reason to stop looking at that operation and to spend its reasoning elsewhere. In this sense the asymmetry is not a failure to read the code (the models read the relevant lines correctly when they look at them, Section 3) but a failure to look. The same allocation of attention is familiar from human automation bias, where errors of commission arise when an aid’s output replaces the verification it was meant to support [6, 7, 55]. In the terms of Lee and See [54], the models’ reliance on the tool is miscalibrated in one direction only. We read the answers as consistent with this mechanism; the study does not demonstrate it directly. Other explanations are compatible with the data. Assistants trained to agree with their users may agree more readily with statements that reduce the scope of the task [13], and the securityoriented framing of our prompts may make alarms feel more salient than reassurances. Our study does not separate these explanations; it establishes that the asymmetry exists, that it is not specific to one model or vendor, and that it also appears in a direction-aware measure that compares each kind of false evidence with true notes about the same operations. It also qualifies the conclusion of concurrent work that contextual cues mainly change a model’s willingness to flag code [14]: for claims that can be checked against the code, the direction of the claim decides whether it is checked. The observation that blunt attempts to bias LLM code reviewers can raise suspicion [15] fits the same pattern: the models scrutinise what looks like an attempt to change their verdict, and an alarm is such an attempt in a way that a quiet assurance is not.

7.2

Failure Modes and Boundary Conditions

Two boundary conditions recur in the data. First, a model’s leaning limits how much any evidence can move it: the exception kimi-k2.6 flags almost every function, has little to gain from scrutiny and little to lose to an assurance, and is moved by neither kind of false evidence. Second, the size of deference varies between samples of the same model. On the study set qwen-plus appeared immune to false assurances because it flags most functions; on the test set the same model lost 11 points to them, and the direction-aware measure showed the same tendency as for the other models. Accuracy is therefore a poor measure of deference. The models we studied differ more in their leaning than in their discrimination (Section 6.1), and a model’s leaning determines how much room evidence has to move it in either direction. Studies that evaluate LLM pipelines with analysis evidence should report, at least, the shift towards the evidence’s claim relative to a matched baseline, separately for evidence that argues for and against a vulnerability. The mitigation has its own boundary: verify-show helped deepseek-chat in the pre-specified test but gave a smaller, non-significant gain for qwen-plus, whose answers it changed less.

22

7.3

Implications for LLM–Analysis Pipelines

Treat the tool’s guarantees as the riskiest input Pipelines that let an LLM triage staticanalysis alarms rely on the direction of evidence that the models already check. Pipelines that pass the analyser’s guarantees to the model (for example that a value is bounded, that a pointer is checked, or that a path is infeasible) rely on the direction that the models trust. An incorrect guarantee, whether caused by an unsound analysis, a stale result or an adversary, is likely to propagate into the model’s verdict. Guarantees should be verified before they reach the auditor, or presented with their provenance and an explicit instruction to verify them in a separate step. Separate verification from judgement Instructions inside the audit prompt were fragile: asking the model to quote the code before relying on a note helped little, and reframing assurances as claims to verify, the best instruction on the study set, did not replicate. Showing the source line that contradicts a note did not help either; the models did not use the contradiction unless asked to check. What worked was moving the check out of the audit: a separate call verifies each claim against the code, and the audit receives the result. This mirrors Chain-of-Verification, where answering verification questions independently of the draft is essential to its effect [53], and is consistent with the finding that self-correction succeeds mainly when it can draw on a reliable external reference [52], here the code itself. It also matches the human finding that explanations reduce over-reliance when they make verification easy [57], and the adversarial finding that isolating the code from unverifiable comments defends LLM detectors better than instructions to distrust them [16]. Verify-show doubles the number of calls; the verification call is short, since it returns a JSON array of statuses, but its cost is not negligible. Withholding refuted claims (verify) was less effective than showing them as contradicted: an explicit statement that the code contradicts a claim appears to correct the auditor, whereas the mere absence of a claim does not. Make true evidence useful True evidence did little in our study, and only the commit-first strategy, in which the model audits the function before seeing the notes and then reconsiders, made it measurably useful on the development set. Commit-first is a cognitive forcing function in the sense of Buçinca et al. [8]: it ensures that the model forms its own judgement before the tool can anchor it. Its effects did not add up with those of verification in our combination, and it was not tested on the test set; whether a better combination exists is an open question. Implications for research The failed replication of the reframing strategy is a cautionary example. Selected among four strategies on 83 functions, it raised accuracy by 18 points with a Holm-adjusted p of 0.001, yet on 66 held-out functions the gain fell to 8 points and was not significant. Prompt variants are cheap to generate and to evaluate, which makes selection effects of this kind easy to produce and hard to notice. Separating development and test data, freezing the strategy, and fixing the hypothesis before the test is run cost us one additional dataset and a few days; it is what allowed us to report the verify-show result with confidence, and the H1b result with honesty.

7.4

Limitations and Threats to Validity

Construct validity Our evidence is produced by our own analyser and manipulated by construction, not taken from a production tool. This is deliberate, since it lets us control the truth and relevance of every note, but real tools phrase their output differently and are wrong in different ways. Our false assurances are refutable by reading the function, and they cite a concrete line; real 23

incorrect guarantees may be harder to refute, which would make the deference we measure a lower bound. Relevance is defined through the fix, which approximates but does not equal what matters for the vulnerability. The behavioural coding of RQ3 relies on a lexical cue; we validated it against two independent LLM annotators and a human check of 20 items, found it conservative (precision 0.97–1.00, recall 0.70–0.76), and report reweighted annotated rates. The annotators are LLMs; the agreement with the first author’s labels was perfect on the checked subset, but the subset is small. The cue was validated on DeepSeek answers; we read a sample of flagged qwen-plus answers but did not formally validate it for the models of the replication. Internal validity All comparisons are paired on the same functions and differ only in the notes or their presentation. Responses at temperature 0 are not fully deterministic [61], so some discordant pairs reflect noise; noise affects both directions of a McNemar test equally and cannot create a systematic difference, but it lowers power. The direction-aware measure was defined after seeing the first four configurations; we therefore treat those results as exploratory and confirmed the measure on seven new models under an internal registration (Section 5.3). We also examined, and rejected before the registration, an alternative analysis (flip rates conditioned on the baseline verdict) after finding that it is dominated by regression to the mean. Responses without a verdict are counted as wrong in accuracy and as neither verdict in the direction-aware measure, which is why the two can differ (for deepseek-chat, two of the ten vulnerable functions lost under C4 were left without a verdict rather than judged SAFE); they are rare (at most 3.2% for all configurations except kimi-k2.6 with 6.5%). External validity We study C/C++ functions of at most 150 lines (300 on the held-out set) with structural CWEs, audited in isolation; longer functions, other languages, inter-procedural evidence and agentic settings may behave differently. PrimeVul’s labels are validated but not free of noise, and the vulnerable version of a pair may contain issues beyond the one fixed. The models are accessed through vendor APIs; we pin dated snapshots where the vendor offers them, but undated model names may change behind the API, and we cannot rule out that PrimeVul functions appear in the models’ training data [60]. Such contamination would affect all conditions of a function alike and cannot explain differences between them. Results may also depend on the wording of our prompts [62]; we used one wording per condition and presentation, and the replication across eleven configurations reduces, but does not eliminate, this threat. Models were run without reasoning, except for two DeepSeek configurations at low reasoning effort; the reasoning configurations showed the same asymmetry, but we did not vary the reasoning effort. Results for deepseek-v4-pro are based on a 60-pair subset. The mitigation was tested on two models; its effect on qwen-plus was half as large and not significant, so we do not claim that it generalises across models. Conclusion validity We report paired exact tests, bootstrap confidence intervals and Holm corrections where several strategies are compared with a baseline. The confirmatory tests use the analyses fixed in advance; only H3 has a registration document with hashes, and that registration is internal (Section 5.3). The pooled analysis of deference across the three sets (Section 6.5) was not pre-specified and is labelled post hoc. Several per-model results rest on 60–90 functions and have wide intervals; we base our conclusions on the pooled and confirmatory tests rather than on individual models.

24

7.5

Reproducibility of the Analysis Scripts

All code used in this study is released as a companion repository at https://github.com/lzh00 82-cloud/CODE-DETECTION under the MIT license. It contains the obligation analyser, the scripts that build the evidence conditions and presentations, run the audits and the verification calls, and compute every table and figure in this paper, together with the annotations and the registrations with their hashes. The full run records, with the evidence of all conditions and all model responses, and the cache of all API responses are archived separately and linked from the repository. PrimeVul is a public dataset and is not redistributed; the repository explains how to obtain its paired splits from the original release.

8

Conclusion

We studied how LLM vulnerability auditors use static-analysis evidence by auditing the same functions under evidence that differed only in relevance, truth and presentation, on 916 PrimeVul pairs with eleven model configurations from four vendors. • True evidence barely improved the audits. Showing all true notes changed accuracy by +0.02; relevant notes gave a small, non-significant gain. • False evidence acted asymmetrically. Fabricated assurances that a check exists pulled verdicts towards SAFE and lowered the detection of real vulnerabilities, whereas fabricated alarms were checked against the code and often made the model clear the function. The asymmetry was confirmed in an internally registered replication on seven models that were not used to derive it; its size varies between models and samples. • Separate verification helped; instructions did not reliably. Instructions within the audit prompt did not reliably reduce deference, and the most promising one failed to replicate; verifying each claim in a separate step and showing the refuted claims did, in a pre-specified test for one of two models, without reducing the value of true evidence. Practical implications • Treat the analyser’s guarantees, not its alarms, as the input that most needs checking in LLM–analysis pipelines. • Check guarantees outside the auditing prompt, in a separate verification step whose refutations are shown to the auditor. • Measure deference in a direction-aware way, and separate the development of prompts from their evaluation. Limitations and future work The findings are bounded by C/C++ functions audited in isolation, evidence from our own analyser, and models accessed through vendor APIs. Future work should (i) study evidence from production analysers; (ii) extend the study to inter-procedural and agentic settings; (iii) test verification strategies on more models; and (iv) find ways to make true evidence useful without making false evidence harmful.

25

Acknowledgment This work was supported by aSSIST University.

Ethics and Data Use The study uses only PrimeVul, a public dataset of open-source code, and the outputs of commercial LLM APIs. PrimeVul is used under the terms of its release and is not redistributed.

Disclosure on the Use of LLMs in Manuscript Preparation LLMs are the object of this study; the models described in the Experimental Setup were the subject of study and were not used for manuscript writing. All scientific claims, experimental designs, data analyses, and interpretations are the authors’ own, and the authors take full responsibility for the content of the paper. In addition, two LLMs (Claude and deepseek-v4-pro) served as annotators for the validation of the behavioural coding, as described in Section 4.6.

A

Prompts

All prompts are reproduced verbatim from the replication package. {...} marks inserted content.

A.1

Audit

System message

It reads:

You are an expert C/C++ security auditor who reasons precisely about code.

User message

The function is shown with line numbers. Under C0 the notes block is omitted.

Decide whether the following function (project ‘{project}‘, file ‘{file}‘) contains a security vulnerability. Line numbers are shown for reference only. ‘‘‘c {numbered function} ‘‘‘ Static analysis notes for reference. {legend} {notes} Reason step by step, citing line numbers: identify untrusted or externally controlled values, trace how they flow into memory accesses, pointer dereferences, frees, arithmetic and calls, and state which checks guard each of them. Then finish with a final line: [VERDICT] VULNERABLE or [VERDICT] SAFE

Legend (plain presentation)

It reads:

’holds here’ lists conditions guaranteed on every path that reaches the operation; ’concern’ states what this function itself does not guarantee. The analysis is exact for this function’s code but does not look into callers or called functions.

26

A.2

Presentation Strategies

quote

The following paragraph is inserted between the notes and the task:

Treat every note as an unverified claim, not as a fact. Before you rely on any ’holds here’ or ’concern’ line, quote the exact code it refers to and say whether that code confirms the claim. Do not rely on a note that you could not confirm in the code; reason from the code instead.

alarms

All holds here lines are removed, and the legend becomes:

Each ’concern’ states what this function itself does not guarantee at that operation. The analysis is exact for this function’s code but does not look into callers or called functions.

reframe

Every line holds here:

X becomes to verify:

the tool reports X.

provenance Every line number cited in a holds here or concern line is followed by “[line N is: ‘...‘]” with the source text of line N. verify and verify-show A separate call checks every claim (one holds here or concern line) before the audit. System message: “You check claims made by a static analyzer against C/C++ source code.” User message: ‘‘‘c {numbered function} ‘‘‘ A static analyzer made these claims about the function above. ’holds here’ claims that a condition is guaranteed on every path to the operation; ’concern’ claims that something is not guaranteed there. [1] {claim} [2] ...

(about {operation})

Check every claim against the code only. Quote the code line(s) that decide it. A claim is CONFIRMED if the code shows it is true, REFUTED if the code shows it is false (for example the cited line does not contain the claimed check), UNCLEAR if this function’s code cannot decide it. Return only a JSON array: [{"id": 1, "status": "CONFIRMED|REFUTED|UNCLEAR", "quote": "..."}]

In the audit, confirmed claims are kept, unclear claims are marked (not confirmed by the code), and refuted claims are removed (verify) or, in verify-show, listed after the notes under the heading Claims of the analyzer that verification found CONTRADICTED by the code (do not rely on them): - {operation} -- {claim}

The legend is extended by: “Every claim below was checked against the code in a separate verification step; claims the code did not confirm are marked.” commit and commit-verify

The first turn is the C0 audit. The model’s answer is followed by:

A static analysis tool, which is sometimes wrong, produced these notes about the function. {legend} {notes} Reconsider your analysis as a careful reviewer would treat advice from such a tool. For every note that bears on your verdict, quote the code it refers to and say whether the code confirms or contradicts it. Keep your earlier verdict unless a note that you confirmed in the code gives a concrete reason to change it. Finish with a final line: [VERDICT] VULNERABLE or [VERDICT] SAFE

In commit-verify the notes are those of verify-show. 27

B

Annotation Guide

The same guide was given to both LLM annotators and to the first author. The sheet showed, for each item, the notes given to the model and the model’s answer, but neither the condition nor the correctness of the verdict. Each row shows the static-analysis notes given to the model (some notes are deliberately FALSE) and the model’ s answer. Read the answer and fill three columns with Y or N. * checks_notes: the answer explicitly compares at least one note with the code (e.g. "line 8 is the loop condition, so the index is guarded"). * rejects_false_note: the answer says that a note is wrong, inaccurate or does not match the code. * verdict_relies_on_note: the final verdict is justified (at least partly) by a note, taken at face value. Label the three columns independently of whether the final verdict is right. When unsure, write ‘?‘ and add a remark in ‘comment‘.

C

Registration of H3

The registration of the multi-vendor replication (preregistration/H3_multimodel.md in the replication package) fixed, before any of the seven models was run: the models and API identifiers, the data and conditions, the measures and the primary test, the secondary analyses, and the handling of missing data. The SHA-256 hashes of the registration and of the analysis script, recorded at registration time, are 85A0D074...F7AD87E and 8047D475...A1927EFB; the script was not changed afterwards. The registration is internal: these files were first deposited publicly with the replication package after the data had been collected.

References [1] H. Li, Y. Hao, Y. Zhai, and Z. Qian, “Enhancing static analysis for practical bug detection: An LLM-integrated approach,” Proceedings of the ACM on Programming Languages, vol. 8, no. OOPSLA1, pp. 111:1–111:26, 2024. [2] Z. Li, S. Dutta, and M. Naik, “IRIS: LLM-assisted static analysis for detecting security vulnerabilities,” in International Conference on Learning Representations (ICLR), 2025. [3] H. Li, Y. Hao, Y. Zhai, and Z. Qian, “Assisting static analysis with large language models: A ChatGPT experiment,” in Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), Ideas, Visions and Reflections, 2023. [4] X. Du, J. Feng, Y. Zou, W. Xu, J. Ma, W. Zhang, S. Liu, X. Peng, and Y. Lou, “Reducing false positives in static bug detection with LLMs: An empirical study in industry,” arXiv preprint arXiv:2601.18844, 2026. [5] B. Johnson, Y. Song, E. Murphy-Hill, and R. Bowdidge, “Why don’t software developers use static analysis tools to find bugs?” in Proceedings of the 35th International Conference on Software Engineering (ICSE), 2013, pp. 672–681. [6] R. Parasuraman and V. Riley, “Humans and automation: Use, misuse, disuse, abuse,” Human Factors, vol. 39, no. 2, pp. 230–253, 1997.

28

[7] K. Goddard, A. Roudsari, and J. C. Wyatt, “Automation bias: A systematic review of frequency, effect mediators, and mitigators,” Journal of the American Medical Informatics Association, vol. 19, no. 1, pp. 121–127, 2012. [8] Z. Buçinca, M. B. Malaya, and K. Z. Gajos, “To trust or to think: Cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making,” Proceedings of the ACM on Human-Computer Interaction, vol. 5, no. CSCW1, pp. 188:1–188:21, 2021. [9] E. Eleftheriou, G. Pallis, and M. Constantinides, “Confidence without competence in AIassisted knowledge work,” arXiv preprint arXiv:2604.09444, 2026. [10] J. Xie, K. Zhang, J. Chen, R. Lou, and Y. Su, “Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts,” in International Conference on Learning Representations (ICLR), 2024. [11] R. Xu, Z. Qi, Z. Guo, C. Wang, H. Wang, Y. Zhang, and W. Xu, “Knowledge conflicts for LLMs: A survey,” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024, pp. 8541–8565. [12] K. Wu, E. Wu, and J. Zou, “ClashEval: Quantifying the tug-of-war between an LLM’s internal prior and external evidence,” in Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track, 2024. [13] M. Sharma, M. Tong, T. Korbak, D. Duvenaud, A. Askell, S. R. Bowman, E. Durmus, Z. Hatfield-Dodds, S. R. Johnston, S. Kravec, T. Maxwell, S. McCandlish, K. Ndousse, O. Rausch, N. Schiefer, D. Yan, M. Zhang, and E. Perez, “Towards understanding sycophancy in language models,” in International Conference on Learning Representations (ICLR), 2024. [14] A. Shahriar, H. Cai, H. Benkraouda, G. Wang, and Z. B. Celik, “Words speak louder than code: Investigating cognitive heuristics in LLM-based code vulnerability detection,” arXiv preprint arXiv:2606.30587, 2026. [15] G. Alexopoulos, N. Alexopoulos, D. Spinellis, and D. Mitropoulos, “Measuring and exploiting contextual bias in LLM-assisted security code review,” arXiv preprint arXiv:2603.18740, 2026. [16] Z. Wu and C. Nita-Rotaru, “ALIBI: Adaptive agentic attacks on LLM-based vulnerability detectors via adversarial code comments,” arXiv preprint arXiv:2607.24964, 2026. [17] Y. Ding, Y. Fu, O. Ibrahim, C. Sitawarin, X. Chen, B. Alomair, D. Wagner, B. Ray, and Y. Chen, “Vulnerability detection with code language models: How far are we?” in Proceedings of the IEEE/ACM 47th International Conference on Software Engineering (ICSE), 2025, arXiv:2403.18624. [18] B. A. Nosek, C. R. Ebersole, A. C. DeHaven, and D. T. Mellor, “The preregistration revolution,” Proceedings of the National Academy of Sciences, vol. 115, no. 11, pp. 2600–2606, 2018. [19] N. A. Ernst and M. T. Baldassarre, “Registered reports in software engineering,” Empirical Software Engineering, vol. 28, no. 2, p. 55, 2023. [20] A. Fan, B. Gokkaya, M. Harman, M. Lyubarskiy, S. Sengupta, S. Yoo, and J. M. Zhang, “Large language models for software engineering: Survey and open problems,” in IEEE/ACM 29

International Conference on Software Engineering: Future of Software Engineering (ICSEFoSE), 2023, pp. 31–53. [21] X. Hou, Y. Zhao, Y. Liu, Z. Yang, K. Wang, L. Li, X. Luo, D. Lo, J. Grundy, and H. Wang, “Large language models for software engineering: A systematic literature review,” ACM Transactions on Software Engineering and Methodology, vol. 33, no. 8, pp. 220:1–220:79, 2024. [22] Z. Li, D. Zou, S. Xu, X. Ou, H. Jin, S. Wang, Z. Deng, and Y. Zhong, “VulDeePecker: A deep learning-based system for vulnerability detection,” in Network and Distributed System Security Symposium (NDSS), 2018. [23] Y. Zhou, S. Liu, J. Siow, X. Du, and Y. Liu, “Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks,” in Advances in Neural Information Processing Systems (NeurIPS), 2019. [24] M. Fu and C. Tantithamthavorn, “LineVul: A transformer-based line-level vulnerability prediction,” in Proceedings of the 19th International Conference on Mining Software Repositories (MSR), 2022, pp. 608–620. [25] J. Fan, Y. Li, S. Wang, and T. N. Nguyen, “A C/C++ code vulnerability dataset with code changes and CVE summaries,” in Proceedings of the 17th International Conference on Mining Software Repositories (MSR), 2020, pp. 508–512. [26] Y. Chen, Z. Ding, L. Alowain, X. Chen, and D. Wagner, “DiverseVul: A new vulnerable source code dataset for deep learning based vulnerability detection,” in Proceedings of the 26th International Symposium on Research in Attacks, Intrusions and Defenses (RAID), 2023. [27] S. Chakraborty, R. Krishna, Y. Ding, and B. Ray, “Deep learning based vulnerability detection: Are we there yet?” IEEE Transactions on Software Engineering, vol. 48, no. 9, pp. 3280–3296, 2022. [28] N. Risse and M. Böhme, “Uncovering the limits of machine learning for automatic vulnerability detection,” in 33rd USENIX Security Symposium, 2024. [29] S. Ullah, M. Han, S. Pujar, H. Pearce, A. Coskun, and G. Stringhini, “LLMs cannot reliably identify and reason about security vulnerabilities (yet?): A comprehensive evaluation, framework, and benchmarks,” in IEEE Symposium on Security and Privacy (S&P), 2024. [30] X. Zhou, T. Zhang, and D. Lo, “Large language model for vulnerability detection: Emerging results and future directions,” in Proceedings of the 46th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER), 2024, pp. 47–51. [31] A. Khare, S. Dutta, Z. Li, A. Solko-Breslin, R. Alur, and M. Naik, “Understanding the effectiveness of large language models in detecting security vulnerabilities,” in IEEE Conference on Software Testing, Verification and Validation (ICST), 2025, pp. 103–114. [32] B. Steenhoek, M. M. Rahman, M. K. Roy, M. S. Alam, H. Tong, S. Das, E. T. Barr, and W. Le, “To err is machine: Vulnerability detection challenges LLM reasoning,” arXiv preprint arXiv:2403.17218, 2024. [33] A. Habib and M. Pradel, “How many of all bugs do we find? a study of static bug detectors,” in Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (ASE), 2018, pp. 317–328. 30

[34] M. Christakis and C. Bird, “What developers want and need from program analysis: An empirical study,” in Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering (ASE), 2016, pp. 332–343. [35] C. Sadowski, E. Aftandilian, A. Eagle, L. Miller-Cushon, and C. Jaspan, “Lessons from building static analysis tools at Google,” Communications of the ACM, vol. 61, no. 4, pp. 58–66, 2018. [36] S. Heckman and L. Williams, “A systematic literature review of actionable alert identification techniques for automated static code analysis,” Information and Software Technology, vol. 53, no. 4, pp. 363–387, 2011. [37] H. J. Kang, K. L. Aw, and D. Lo, “Detecting false alarms from automatic static analysis tools: How far are we?” in Proceedings of the 44th International Conference on Software Engineering (ICSE), 2022, pp. 698–709. [38] T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, and T. Scialom, “Toolformer: Language models can teach themselves to use tools,” in Advances in Neural Information Processing Systems (NeurIPS), 2023. [39] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, “ReAct: Synergizing reasoning and acting in language models,” in International Conference on Learning Representations (ICLR), 2023. [40] Y. Xiong and T. Zhang, “Sifting the noise: A comparative study of LLM agents in vulnerability false positive filtering,” arXiv preprint arXiv:2601.22952, 2026. [41] E. Perez, S. Ringer, K. Lukošiūtė, K. Nguyen, E. Chen, S. Heiner et al., “Discovering language model behaviors with model-written evaluations,” in Findings of the Association for Computational Linguistics: ACL 2023, 2023. [42] B. Wang, X. Yue, and H. Sun, “Can ChatGPT defend its belief in truth? evaluating LLM reasoning via debate,” in Findings of the Association for Computational Linguistics: EMNLP 2023, 2023, pp. 11 865–11 881. [43] S. Thornton, “Can adversarial code comments fool AI security reviewers? A large-scale empirical study of comment-based attacks and defenses against LLM code analysis,” arXiv preprint arXiv:2602.16741, 2026. [44] B. C. Hu, K. Ahir, R. Ni, and L. Jia, “Correct verdicts, flawed reasoning: Structured auditing of LLM-based vulnerability reasoning,” arXiv preprint arXiv:2610.06366, 2026. [45] I. A. Fahad, K. Asif, and M. N. A. Tawhid, “Mitigating LLM sycophancy in code smell detection using evidence-guided reasoning prompts,” arXiv preprint arXiv:2607.10411, 2026. [46] Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, and P. Fung, “Survey of hallucination in natural language generation,” ACM Computing Surveys, vol. 55, no. 12, pp. 248:1–248:38, 2023. [47] L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, and T. Liu, “A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions,” ACM Transactions on Information Systems, vol. 43, no. 2, 2025.

31

[48] J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. V. Le, and D. Zhou, “Chain-of-thought prompting elicits reasoning in large language models,” in Advances in Neural Information Processing Systems (NeurIPS), 2022. [49] M. Turpin, J. Michael, E. Perez, and S. R. Bowman, “Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting,” in Advances in Neural Information Processing Systems (NeurIPS), 2023. [50] A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang, S. Gupta, B. P. Majumder, K. Hermann, S. Welleck, A. Yazdanbakhsh, and P. Clark, “Self-refine: Iterative refinement with self-feedback,” in Advances in Neural Information Processing Systems (NeurIPS), 2023. [51] J. Huang, X. Chen, S. Mishra, H. S. Zheng, A. W. Yu, X. Song, and D. Zhou, “Large language models cannot self-correct reasoning yet,” in International Conference on Learning Representations (ICLR), 2024. [52] R. Kamoi, Y. Zhang, N. Zhang, J. Han, and R. Zhang, “When can LLMs actually correct their own mistakes? a critical survey of self-correction of LLMs,” Transactions of the Association for Computational Linguistics, vol. 12, pp. 1417–1440, 2024. [53] S. Dhuliawala, M. Komeili, J. Xu, R. Raileanu, X. Li, A. Celikyilmaz, and J. Weston, “Chainof-verification reduces hallucination in large language models,” in Findings of the Association for Computational Linguistics: ACL 2024, 2024, pp. 3563–3578. [54] J. D. Lee and K. A. See, “Trust in automation: Designing for appropriate reliance,” Human Factors, vol. 46, no. 1, pp. 50–80, 2004. [55] L. J. Skitka, K. L. Mosier, and M. Burdick, “Does automation bias decision-making?” International Journal of Human-Computer Studies, vol. 51, no. 5, pp. 991–1006, 1999. [56] G. Bansal, T. Wu, J. Zhou, R. Fok, B. Nushi, E. Kamar, M. T. Ribeiro, and D. Weld, “Does the whole exceed its parts? the effect of AI explanations on complementary team performance,” in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI), 2021. [57] H. Vasconcelos, M. Jörke, M. Grunde-McLaughlin, T. Gerstenberg, M. S. Bernstein, and R. Krishna, “Explanations can reduce overreliance on AI systems during decision-making,” Proceedings of the ACM on Human-Computer Interaction, vol. 7, no. CSCW1, pp. 129:1– 129:38, 2023. [58] C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, and A. Wesslén, Experimentation in Software Engineering. Springer, 2012. [59] A. Arcuri and L. Briand, “A practical guide for using statistical tests to assess randomized algorithms in software engineering,” in Proceedings of the 33rd International Conference on Software Engineering (ICSE), 2011, pp. 1–10. [60] J. Sallou, T. Durieux, and A. Panichella, “Breaking the silence: the threats of using LLMs in software engineering,” in Proceedings of the 46th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER), 2024.

32

[61] S. Ouyang, J. M. Zhang, M. Harman, and M. Wang, “An empirical study of the nondeterminism of ChatGPT in code generation,” ACM Transactions on Software Engineering and Methodology, vol. 34, no. 2, 2025. [62] M. Sclar, Y. Choi, Y. Tsvetkov, and A. Suhr, “Quantifying language models’ sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting,” in International Conference on Learning Representations (ICLR), 2024. [63] A. Gu, B. Rozière, H. J. Leather, A. Solar-Lezama, G. Synnaeve, and S. Wang, “CRUXEval: A benchmark for code reasoning, understanding and execution,” in Proceedings of the 41st International Conference on Machine Learning (ICML), ser. PMLR, vol. 235, 2024, pp. 16 568– 16 621. [64] J. Chen, Z. Pan, X. Hu, Z. Li, G. Li, and X. Xia, “Reasoning runtime behavior of a program with LLM: How far are we?” in Proceedings of the IEEE/ACM 47th International Conference on Software Engineering (ICSE), 2025. [65] Tree-sitter contributors, “Tree-sitter: a parser generator tool and an incremental parsing library,” https://tree-sitter.github.io/tree-sitter/, python bindings 0.26; grammars tree-sitter-c 0.24 and tree-sitter-cpp 0.23. [66] Q. McNemar, “Note on the sampling error of the difference between correlated proportions or percentages,” Psychometrika, vol. 12, no. 2, pp. 153–157, 1947. [67] B. Efron and R. J. Tibshirani, An Introduction to the Bootstrap.

Chapman & Hall, 1993.

[68] S. Holm, “A simple sequentially rejective multiple test procedure,” Scandinavian Journal of Statistics, vol. 6, no. 2, pp. 65–70, 1979. [69] F. Gilardi, M. Alizadeh, and M. Kubli, “ChatGPT outperforms crowd workers for textannotation tasks,” Proceedings of the National Academy of Sciences, vol. 120, no. 30, p. e2305016120, 2023. [70] T. Ahmed, P. Devanbu, C. Treude, and M. Pradel, “Can LLMs replace manual annotation of software engineering artifacts?” in Proceedings of the 22nd International Conference on Mining Software Repositories (MSR), 2025. [71] J. Cohen, “A coefficient of agreement for nominal scales,” Educational and Psychological Measurement, vol. 20, no. 1, pp. 37–46, 1960.

33

Related documents

• Trusting the Tool: Asymmetric Deference of LLM Vulnerability Auditors to Static-Analysis Evidence#1448391
• TRACE: Diagnosing Verifier Brittleness in Agentic Evaluation#1248958
• Towards Behavior Tree-Guided Vulnerability Detection with Lightweight LLMs#822187
• SERI v0.1: Structural Ethics Readiness Index for LLM Systems.#86355
• Beyond Predefined Sinks: Security-Aware Dependency Analysis for LLM Agents#1187692