Trust but Verify? Uncovering the Security Debt of Autonomous Coding Agents A H M Nazmus Sakib
University of Texas at San Antonio USA [email protected]
Dipayan Banik
Danovo Energy Solutions USA [email protected]
arXiv:2607.12428v1 [cs.CR] 14 Jul 2026
Abstract The increasing adoption of autonomous coding agents accelerates software development but also introduces scoped security risks within high-impact file paths that can outpace traditional human review capacity. While prior research has primarily evaluated these systems in terms of functional correctness and productivity, this paper presents a large-scale empirical study using the AIDev dataset to systematically characterize security code smells in agent-generated pull requests (PRs). Through a combination of a validated LLM-asa-judge framework and manual qualitative analysis, we identify and classify security misconfigurations across 16,112 file changes spanning 4,022 pull requests. Our results reveal that 38.9% of agentgenerated PRs contain at least one security smell, with supply chain integrity issues accounting for 82.3% of all detected security smells. Furthermore, hard-coded credentials constitute 99.6% of all critical-severity security smells. Crucially, we find that human collaborators are responsible for introducing 67.6% of genuine leaked secrets within these agent-assisted workflows, while existing automated and human review processes fail to detect 81.1% of these credentials prior to integration. These findings highlight substantial security risks in agent-assisted software development workflows and suggest a potential reduction in developer vigilance. They also underscore the urgent need for context-aware security guardrails implemented directly at the point of human-AI collaboration.
CCS Concepts • Security and privacy → Software security engineering.
Keywords Agentic AI, LLMs, Pull Requests, Empirical Study, Code Smells
1
Introduction
The increasing adoption of Artificial Intelligence (AI) tools in software engineering has contributed to the emergence of Software Engineering 3.0 (SE 3.0), a paradigm characterized by AI systems that can perform a range of software development tasks with limited human intervention[11, 14]. Systems such as GitHub Copilot[7], OpenAI Codex[19], Claude Code[2], Cursor[3], and Devin[5] can execute multi-step development workflows, including task planning, code generation, testing, and the creation of Pull Requests (PRs) for integration into existing codebases. Unfortunately, this acceleration often compromises system trustworthiness and code quality[17]: the scale of agent-generated PRs outpaces human review capacity, leaving maintainers fatigued and over-reliant on automated outputs[22, 26]. The mismatch between code production velocity and reviewer capacity has demonstrably compromised repository-level security
Murtuza Jadliwala
University of Texas at San Antonio USA [email protected]
integrity; nascent empirical analyses, albeit derived from limited datasets, suggest that AI-generated code exhibits a vulnerability introduction rate approximately 2.7 times that observed in humanwritten code[1, 15]. However, current software engineering literature predominantly evaluates Large Language Model (LLM) code generation with benchmarks focused on functional correctness rather than the security posture of the specific files and paths agents modify[4, 12]. Although emerging empirical studies have begun to characterize agentic pull requests by analyzing their general modification patterns[18], test generation behaviors[10, 29], and commit quality[24], there remains a critical research gap with respect to the introduction of structural security flaws within the specific code paths that agents touch. Conducting a large-scale, scoped empirical study on real-world data is imperative to document and understand the distribution, types, and severity of security code smells introduced by the use of autonomous agents. To address this gap, we present an empirical study leveraging the AIDev dataset: a large-scale repository of PRs generated by autonomous coding agents to systematically characterize their security implications. In this paper, we focus on ’security smells’ (structural patterns that indicate potential security risks), rather than confirmed exploitable vulnerabilities, which require contextspecific exploitability analysis. We utilize a combination of LLM-asa-judge evaluation, and manual qualitative analysis to investigate the following Research Questions (RQs): RQ1: Which security smells appear most in agentic PRs, and how do they distribute across categories and severity? RQ2: How many flagged hard-coded secrets are genuine, who introduced them, and are they identified during review before integration? This paper makes the following core contributions: • A six-category taxonomy of security smells in agent-generated PRs, grounded in OWASP secure-coding guidance, the Center for Internet Security Benchmarks, and GitHub hardening documentation. • An LLM-as-a-judge detection pipeline that categorizes security smells in agentic diffs with quantized open-source models, validated against a human-annotated gold standard using precision, recall, F1, and Cohen’s kappa. • A large-scale empirical characterization of the prevalence, category distribution, and severity of security smells across agentic PRs in the AIDev dataset[14]. • A qualitative analysis of the most critical smells that traces each genuine credential to its committer, the agent or the human collaborator, and assesses whether automated bots and human reviewers intercept it before integration.
Sakib et al.
Figure 1: Architecture of the proposed methodology.
2
Methodology
We use AIDev, a public dataset of PRs from five coding agents on GitHub. We draw on three tables: pull_request holds per-request metadata, pr_commit_details holds one row per changed file per commit with the unified-diff patch, and pr_review_comments holds inline review comments. Our methodology is shown in Figure 1.
Table 1: Security Smell Categories and Scope Category secrets_identity cleartext_transport over_privilege_execution
2.1
Detection and Quantification of Security Smells
Corpus construction. We keep files whose path matches a highrisk pattern set covering Continuous Integration (CI) definitions, container files, infrastructure-as-code, secret-bearing files, configuration files, shell scripts, and computational notebooks. We exclude lock files, minified assets, vendor directories, and binary formats. We assign each retained file to the file categories to support stratified analysis. We then reduce each patch to the lines added, retaining only + lines and discarding diff metadata and removed lines, so that every counted smell reflects introduced content. The corpus contains 16,370 file changes across 4,064 PRs. Because we filter for high-risk patterns and analyze only added lines, this work constitutes a scoped security-smell analysis rather than a comprehensive repository audit. Taxonomy. We define six smell categories, grounded in the OWASP guidance, the Center for Internet Security Benchmarks, and the GitHub hardening documentation[20], shown in Table 1. Detection. We label the corpus with a large language model acting as a judge. We prompt two open-source models, Qwen3.635B-A3B-FP8[23] and Gemma-4-26B-A4B-IT-FP8[8], with zero-shot prompting, at FP8 quantization through a local end-point at sampling temperature 0.1 (see Appendix A.1). The judge performs independent categorization over the whole corpus: for each file change, it receives the patch, file path, language, and PR title. The judge
permissive_network misconfig_hardening supply_chain_integrity
Scope Hard-coded keys, tokens, private keys, and credential-bearing connection strings. Cleartext endpoints, disabled TLS, and weak ciphers. Root containers, world-writable files, sudo, shell installers, broad CI scopes. All-interface binds, open CIDRs, and public-access flags. Debug modes, logged secrets, and disabled encryption. Mutable action/image tags and unpinned global installs.
marks the file clean or flags each smell with one of the six categories, a line number and snippet, a severity on a three-level scale, and a one-sentence rationale. Reliability and Validation. We merge the runs of the models as a union, marking a file dirty when at least one run flags it, and deduplicating smells on category, line number, and line snippet. We report inter-rater agreement with Cohen’s Kappa and validate the judge against a random sample of 376 file changes with 95% Confidence Level and 5% Margin of Error. Two authors independently assign a smell label to each file change, and a third author adjudicates disagreements to a gold label matched to the judge. We report precision, recall, the F1 score, and Cohen’s kappa against the gold label, together with inter-annotator agreement. Stratified Analysis. To examine which factors drive smell prevalence, we disaggregate the flagging results along four dimensions: agent type, repository primary language, file category and PR size. We take the agent and the repository identifier from the pull request table and the primary language from the repository table. We
Trust but Verify? Uncovering the Security Debt of Autonomous Coding Agents
supply_chain_integrity
n=7,160
over_privilege_execution
n=835
secrets_identity
n=294
misconfig_hardening
n=163
permissive_network
n=153
cleartext_transport
n=96
0
25
50
75
100
Share of smells by severity (%) Critical
Major
Minor
Figure 2: Severity composition of smells by category assign the file category during corpus construction. We compute the PR size as the sum of changed lines over the distinct commits of a PR and bin it into five tiers: XS (1-9), S (10-49), M (50-199), L (200-999), and XL (1000+). We report a file-level flagging rate as the share of files marked dirty, a PR-level flagging rate as the share of PRs with at least one dirty file, and the mean smells per file.
2.2
Validating and Reviewing Hard-Coded Secrets
A hard-coded secret is the highest-impact security smell, because a live credential committed to a public repository is exploitable the moment it is pushed. RQ2 examines the secrets_identity category in depth. We determine whether the secret is genuine, who committed it, and whether the existing reviewers catch it or not. We pay particular attention to automated security bots, since they are the layer expected to intercept a leaked credential before it causes harm. We extract the complete set of smells that the categorization judge flags under secrets_identity and manually examine each one. We carry no judge label into this stage so that the coders judge each secret on its own evidence. Two authors independently code each secret in two dimensions. Validity records whether the flagged value is a genuine credential rather than a placeholder, an example, or a misread. Reviewer detection records whether an automated security bot commented on the secret. The coders inspect the live repository, the pull-request discussion, and the commit history that follows the introducing commit.
3
Key Finding Supply_chain_integrity smell dominates at 82.3% of the 8,701 smells, and hard-coded secrets form 99.6% of the 253 critical smells. Prevalence rises with development activity from 16.2% to 53.6% of PRs by size, 87.6% of smells in GitHub Actions and Docker files, highest for Copilot (45.5%) and JavaScript repositories (55.3%). The judge matched the human gold labels at 0.908 precision and 0.836 F1 score.
Results
To support reproducibility, we provide a replication package containing the data, scripts, and documentation required to reproduce the analyses reported in this paper. The replication package is available in the following link[16].
3.1
for 258 file changes. The judge flagged 3,807 file changes as containing at least one smell, a rate of 23.6%, and recorded 8,701 distinct smells after deduplication. At the PR level, 1,563 of 4,022 requests contained at least one smell, a rate of 38.9%, and the remaining 2,459 requests stayed clean. The six categories differed by an order of magnitude in frequency, as Table 2 reports. As shown in Figure 2, the supply_chain_integrity category alone accounted for 7,160 of the 8,701 smells, a share of 82.3%, followed by over_privilege_execution at 9.6% and secrets_identity at 3.4%. The severity distribution placed 6,292 smells at the major level (72.3%), 2,156 at the minor level (24.7%), and 253 at the critical level (3.0%). Critical smells concentrated in one category: 252 of the 253 critical smells, a share of 99.6%, fell under secrets_identity. The judge thus reserved its highest severity almost entirely for hard-coded credentials. The human validation set comprised 376 file changes where the two coders manually labeled each item and reached a Cohen’s kappa of 0.929. The judge reached a precision of 0.908, a recall of 0.775, an F1 score of 0.836, and a Cohen’s kappa of 0.789 against the gold labels. The judge therefore identified smells at high precision while missing 22.5% of smells. To identify the factors that drive smell prevalence, we disaggregated the PR-level flagging rate by agent, repository primary language, file category and PR size. Figure 4 compares the flagging rate against the 38.9% corpus average by agent and language. Copilot PRs carried a smell most often at 45.5%, and OpenAI Codex PRs least often at 34.9%, yet Codex’s flagged files carried the most smells per file at 0.62. JavaScript repositories showed the highest prevalence at 55.3% and Python the lowest among the major languages at 31.8%, a spread of 23.5 percentage points. TypeScript sat at the corpus average of 38.9%. Table 3 disaggregates the file-level rate by file category. GitHub Actions workflows and Docker files together held 87.6% of all smells, consistent with the concentration of supply_chain_integrity smells. The application configuration files remained clean at a 2.2% dirty rate. Figure 5 groups the PRs by the total lines changed and shows the flagging rate increasing with the size of the PR, from 16.2% for the smallest PRs (1-9 lines) to 53.6% for the largest (1000+ lines), a spread of 37.4 percentage points.
RQ1: Distribution of Security Smells
The judge returned valid labels for 16,112 of the 16,370 file changes, which span 4,022 PRs. The judge failed to return parseable output
3.2
RQ2: Genuine Secrets and Review Outcomes
The coders assigned a validity label to 272 of the 294 secret smells. The remaining 22 could not be labeled because the PRs no longer exist. The coders confirmed 74 genuine credentials, a rate of 27.2% of the labeled set. The commit author of each genuine secret fell into one of two classes. A human author committed 50 of the 74 genuine
Sakib et al.
Table 2: Distribution of Smells by Category and Severity Category Smells Critical Major Minor supply_chain_integrity 7,160 0 5,018 2,142 over_privilege_execution 835 1 833 1 secrets_identity 294 252 41 1 misconfig_hardening 163 0 151 12 permissive_network 153 0 153 0 cleartext_transport 96 0 96 0 Table 3: Smell-Flagging Rate by File Category (file-level) Category GH Actions workflows Docker Other CI Shell scripts Terraform K8s/Helm Notebooks App config
Files 7,054 1,775 89 2,142 374 636 793 1,058
% Dirty 36.3% 36.4% 22.5% 12.0% 11.5% 11.2% 10.1% 2.2%
Smells 6,577 1,058 40 465 76 119 123 30
secrets, a rate of 67.6%, and an AI agent committed the remaining 24. The human collaborator therefore introduces genuine secrets into agent pull requests more often than the agent does. A security bot or a human reviewer commented on the secret in 14 of the 74 genuine cases, a rate of 18.9%, and the reviewers removed the other 60 secrets without a comment. The commenting actors comprised seven distinct tools and reviewers: GitHub Advanced Security, GitGuardian, Qodo, Copilot, Greptile, Gemini Code Assist, and Cursor bot. A summary of these results is depicted in Figure 3. Secret Smells Attrition
Total Secret Smells Labeled Valid Genuine Secrets
n=74
Silently Removed
n=60
Commented / Reported n=14 0
100
200
Reporting Tools (n=14)
n=4 n=3 n=3
Qodo n=294 GitHub Adv. Sec. n=272 Copilot GitGuardian Greptile Gemini Cursor 300
Number of Items
n=1 n=1 n=1 n=1 0
2
4
Number of Reports
Figure 3: Summary of RQ2 Investigation Outcomes Key Finding Coders confirmed 27.2% of labeled secret smells as genuine credentials. Human collaborators committed 67.6% of them, and 81.1% reached integration without a review comment from a bot or human.
4
Threats to Validity
The study relies on a manually annotated gold standard of 376 file changes, making results susceptible to annotation bias and human error. Similar risks apply to the manual validation of hardcoded secrets in repositories and commit histories. To reduce these threats, we used independent double-blind coding and third-author adjudication. The use of quantized LLMs (Qwen3.6 and Gemma-4) introduces limitations such as non-determinism and incomplete detection. With a recall of 0.775, the LLM judge missed 22.5% of security smells, meaning reported frequencies likely underestimate the true prevalence of security smells. By intentionally restricting the analyzed corpus to high-risk file paths and focusing exclusively on added lines, the findings characterize specific security smells introduced within targeted boundaries rather than providing a comprehensive audit of total repository security health. Security smells or misconfigurations present in excluded files or introduced through complex interactions across multiple existing files are not captured in this analysis. The findings are based on the AIDev dataset, which focuses on selected coding agents in open-source GitHub repositories. As a result, the observed security smells and review patterns may not generalize to other agents, enterprise environments, or development platforms.
5
Related Work
The advent of autonomous coding agents has prompted extensive empirical investigation into the differences between AI-generated and human-authored PRs. Ogenrwot[18] found substantial differences in commit counts, merge outcomes, lifecycle dynamics, and description-to-diff alignment compared to human baselines. Yoshioka et al.[30] identified that while submitter attributes dominate merge success in both groups, review-related features exert contrasting effects on agentic PRs. This builds upon Watanabe et al.[28], who established the distinct architectural footprint of AI contributors. Rahman et al.[24] conducted task-level evaluations demonstrating that while certain models achieve high PR acceptance rates, others trigger significantly higher volumes of human and automated review discussions. Haque et al.[10] and Yoshimoto et al.[29] observed that agent-generated test methods present distinct structural patterns. The human-AI interaction layer is equally critical; Haider et al.[9] taxonomized reviewer responses to AI-authored code, revealing that inline comments predominantly address logical and functional correctness rather than security posture. While AI applications provide a foundation for open-source software sustainability[13], system trustworthiness remains an underexplored bottleneck[25]. Prior security literature on PRs has primarily focused on human-authored vulnerabilities or automated patching mechanisms utilizing datasets like TREEVUL[21] or architectures like GraphSPD[27]. Wang et al.[6] recently addressed AI-specific security dynamics by examining automated versus human security patching patterns within the AIDev dataset. However, while these studies analyze patching efficacy, functional correctness, or general modification metrics, our work distinctly isolates the introduction of structural security smells (e.g., hard-coded secrets, supply chain misconfigurations) during autonomous generation.
Trust but Verify? Uncovering the Security Debt of Autonomous Coding Agents
By agent type
By repository language 45.5%
Copilot
55.3%
JavaScript Rust
47.5%
Other langs.
47.5%
41.2%
Claude Code Cursor
40.6%
Devin
39.7%
34.9%
OpenAI Codex 0
10
20
30
43.8%
Go C#
40.2%
TypeScript
38.9% 31.8%
Python 40
50
60
0
10
PRs flagged (%)
20
30
40
50
60
PRs flagged (%)
Corpus average (38.9%)
Above average
Below average
Figure 4: PR-level smell-flagging rate by agent type and by repository primary language, relative to the corpus average of 38.9%. +37.4 pp
PRs flagged (%)
60
53.6%
50 40
42.9%
42.9%
M (50–199)
L (200–999)
29.8%
30 20 16.2% 10 0 XS (1–9)
S (10–49)
XL (1000+)
PR size (lines changed)
Figure 5: PR-level smell-flagging rate by PR size, showing an upwards trend with the amount of code changed.
These results reveal a critical operational insight: the velocity of agentic code generation outpaces and compromises traditional human-in-the-loop review pipelines. The high rate of humanintroduced secrets suggests potential review fatigue or cognitive offloading, though further behavioral studies are required to determine why developers bypass standard security hygiene when interacting with autonomous workflows. Future research must focus on mitigating developer cognitive load by designing context-aware security interfaces that evaluate agentic workflows without exacerbating review fatigue. Investigations should also explore novel mechanisms for actively enforcing security hygiene at the point of human-AI collaboration, specifically targeting preventative measures against the initial injection of live credentials. Finally, expanding this empirical analysis to enterprise environments is necessary to determine if proprietary guardrails alter security smell distributions or better sustain developer vigilance.
References 6
Conclusion and Future Work
This study provides a large-scale empirical analysis of security code smells introduced within autonomous agent-generated pull requests. By analyzing the AIDev dataset using a combination of validated LLM-as-a-judge methodology and manual qualitative analysis, we established that agentic coding workflows carry significant security debt. Code smells were present in 38.9% of agent-authored PRs, with supply chain integrity misconfigurations constituting the vast majority (82.3%) of identified security smells. Furthermore, our investigation into critical-severity smells revealed that 99.6% are hard-coded credentials. Crucially, our findings show that human collaborators were mostly responsible, injecting 67.6% of genuine leaked secrets within agentic PRs. Compounding this risk, the current safety net of automated security bots and human reviewers failed to intercept 81.1% of these leaked credentials prior to integration.
[1] [n. d.]. AI Coding Agent Horror Stories: Security Risks Explained | Docker — docker.com. https://www.docker.com/blog/ai-coding-agent-horror-storiessecurity-risks/. [Accessed 01-06-2026]. [2] Anthropic. 2026. Claude Code. https://www.anthropic.com/product/claude-code. Accessed: 2026-06-01. [3] Anysphere. 2026. Cursor. https://cursor.com. Accessed: 2026-06-01. [4] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. 2021. Evaluating Large Language Models Trained on Code. arXiv:2107.03374 [cs.LG] https://arxiv.org/abs/2107.03374 [5] Cognition AI. 2026. Devin. https://devin.ai. Accessed: 2026-06-01. [6] Jacie Jermier Felix Wang, Brian Do. 2026. Automated vs. Human Security Patching Patterns in Pull Requests: Evidence from the AIDev Dataset. https://plg.uwaterloo.ca/~migod/846/current/projects/proposals/04FelixJacie-Brian-proposal.pdf. [Accessed 01-06-2026].
Sakib et al.
[7] GitHub. 2026. GitHub Copilot Documentation. https://docs.github.com/en/ copilot. Accessed: 2026-06-01. [8] Google DeepMind. 2026. Gemma-4-26B-A4B-IT-FP8. https://huggingface.co/ google/gemma-4-26b-it. Instruction-tuned FP8 checkpoint of Gemma 4 26B A4B Mixture-of-Experts model. [9] Md. Asif Haider and Thomas Zimmermann. 2026. Understanding Dominant Themes in Reviewing Agentic AI-authored Code. arXiv:2601.19287 [cs.SE] https://arxiv.org/abs/2601.19287 [10] Sabrina Haque, Sarvesh Ingale, and Christoph Csallner. 2026. Do Autonomous Agents Contribute Test Code? A Study of Tests in Agentic Pull Requests. arXiv:2601.03556 [cs.SE] https://arxiv.org/abs/2601.03556 [11] Ahmed E. Hassan, Gustavo A. Oliva, Dayi Lin, Boyuan Chen, and Zhen Ming (Jack) Jiang. 2026. Towards AI-Native Software Engineering (SE 3.0): A Vision and a Challenge Roadmap. ACM Trans. Softw. Eng. Methodol. (April 2026). doi:10.1145/3807901 Just Accepted. [12] Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. 2024. SWE-bench: Can Language Models Resolve Real-world Github Issues?. In The Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=VTF8yNQM66 [13] S M Rakib UI Karim, Wenyi Lu, and Sean Goggins. 2026. Artificial Intelligence in Open Source Software Engineering: A Foundation for Sustainability. arXiv:2602.07071 [cs.SE] https://arxiv.org/abs/2602.07071 [14] Hao Li, Haoxiang Zhang, and Ahmed E. Hassan. 2025. The Rise of AI Teammates in Software Engineering (SE) 3.0: How Autonomous Coding Agents Are Reshaping Software Engineering. arXiv:2507.15003 [cs.SE] https://arxiv.org/ abs/2507.15003 [15] Gary Marcus. [n. d.]. LLMs + Coding Agents = Security Nightmare — garymarcus.substack.com. https://garymarcus.substack.com/p/llms-coding-agentssecurity-nightmare. [Accessed 01-06-2026]. [16] nazanaza2970. [n. d.]. GitHub - nazanaza2970/agentic_se_kdd_submission: Codes and files for submission to agentic se@kdd workshop — github.com. https: //github.com/nazanaza2970/agentic_se_kdd_submission. [Accessed 14-07-2026]. [17] Claudia Negri-Ribalta, Rémi Geraud-Stewart, Anastasia Sergeeva, and Gabriele Lenzini. 2024. A systematic literature review on the impact of AI models on the security of code generation. Frontiers in Big Data Volume 7 - 2024 (2024). doi:10.3389/fdata.2024.1386720 [18] Daniel Ogenrwot. 2026. How AI Coding Agents Modify Code: A Large-Scale Study of GitHub Pull Requests. arXiv preprint arXiv:2601.17581 (2026). [19] OpenAI. 2025. Introducing Codex. https://openai.com/index/introducing-codex/. Accessed: 2026-06-01. [20] OWASP Foundation. 2026. OWASP Secure Coding Practices – Quick Reference Guide. https://owasp.org/www-project-secure-coding-practices-quickreference-guide/stable-en/01-introduction/05-introduction Accessed: 2026-0601. [21] Shengyi Pan, Lingfeng Bao, Xin Xia, David Lo, and Shanping Li. 2023. Finegrained Commit-level Vulnerability Type Prediction by CWE Tree Structure. 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) (2023), 957–969. https://api.semanticscholar.org/CorpusID:259860088 [22] Veronica Pimenova, Sarah Fakhoury, Christian Bird, Margaret-Anne Storey, and Madeline Endres. 2025. Good Vibrations? A Qualitative Study of Co-Creation, Communication, Flow, and Trust in Vibe Coding. arXiv:2509.12491 [cs.SE] https://arxiv.org/abs/2509.12491 [23] Qwen Team. 2026. Qwen3.6-35B-A3B-FP8. https://huggingface.co/Qwen/Qwen3. 6-35B-A3B-FP8. FP8 quantized multimodal Mixture-of-Experts model, 35B total parameters, 3B active parameters. [24] Shojibur Rahman, Md Fazle Rabbi, and Minhaz Zibran. 2026. A Task-Level Evaluation of AI Agents in Open-Source Projects. arXiv:2602.02345 [cs.SE] https://arxiv.org/abs/2602.02345 [25] Abhik Roychoudhury, Corina Pasareanu, Michael Pradel, and Baishakhi Ray. 2025. Agentic AI Software Engineers: Programming with Trust. arXiv:2502.13767 [cs.SE] https://arxiv.org/abs/2502.13767 [26] Abbas Sabra, Olivier Schmitt, and Joseph Tyler. 2025. Assessing the Quality and Security of AI-Generated Code: A Quantitative Analysis. arXiv:2508.14727 [cs.SE] https://arxiv.org/abs/2508.14727 [27] Shu Wang, Xinda Wang, Kun Sun, Sushil Jajodia, Haining Wang, and Qi Li. 2023. GraphSPD: Graph-Based Security Patch Detection with Enriched Code Semantics. 2023 IEEE Symposium on Security and Privacy (SP) (2023), 2409–2426. https://api.semanticscholar.org/CorpusID:253390023 [28] Miku Watanabe, Hao Li, Yutaro Kashiwa, Brittany Reid, Hajimu Iida, and Ahmed E. Hassan. 2026. On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub. arXiv:2509.14745 [cs.SE] https://arxiv.org/abs/2509.14745 [29] Suzuka Yoshimoto, Shun Fujita, Kosei Horikawa, Daniel Feitosa, Yutaro Kashiwa, and Hajimu Iida. 2026. Testing with AI Agents: An Empirical Study of Test Generation Frequency, Quality, and Coverage. arXiv:2603.13724 [cs.SE] https: //arxiv.org/abs/2603.13724 [30] Haruhiko Yoshioka, Takahiro Monno, Haruka Tokumasu, Taiki Wakamatsu, Yuki Ota, Nimmi Weeraddana, and Kenichi Matsumoto. 2026. Let’s Make Every Pull Request Meaningful: An Empirical Analysis of Developer and Agentic Pull
Requests. arXiv:2601.18749 [cs.SE] https://arxiv.org/abs/2601.18749
Trust but Verify? Uncovering the Security Debt of Autonomous Coding Agents
A Appendix A.1 LLM Setup Information A sample of the prompt provided to the LLM-as-a-judge system is shown in Figure 6. A sample response sent to LLM is shown in Figure 7.
A.2
Sample Secrets Found During Manual Verification
Figures 9 and 10 show two sample secrets found while manually verifying the results produced by the LLM-as-a-judge system.
SYSTEM_PROMPT = """You are a senior application-security engineer reviewing agent-generated diffs. You respond with one valid JSON object only. No markdown, no commentary outside the JSON.""" SMELL_GUIDE = """## Security / misconfiguration smell taxonomy You will receive a unified diff containing only the agent-introduced ('+') lines. Report each smell you find using one of the six categories below. Report a finding ONLY if the offending content is on a '+' line. Use the exact category id shown in the heading. ---------------------------------------------------------------------### secrets_identity (default severity: critical | major) Hard-coded credentials or identity material in source. POSITIVE cues — concrete shapes: - OpenAI-style key: sk-[A-Za-z0-9]{20,} - AWS access / session: AKIA[0-9A-Z]{16}, ASIA[0-9A-Z]{16} - GitHub PAT: ghp_..., github_pat_... - Google API key: AIzaSy... - Slack token: xox[baprs]-... - PEM block: -----BEGIN (RSA |EC |OPENSSH |)PRIVATE KEY----- Quoted password / api/secret key assignment with a non-trivial value. - DB connection string with user:password embedded.
Figure 9: Sample Secret 1
NOT a smell: - Obvious placeholders: REPLACE_ME, CHANGEME, <your-key>, xxxx, dummy, example, foo, bar, VAR, ENV, all-zero / repeating-char strings. - Reads from env / secret manager (os.environ, getenv, vault, KMS). - Test fixtures clearly labelled as such AND value is non-functional. """
Figure 6: Prompt for LLM-as-a-judge
response = client.chat.completions.create( model="/home/sakib/models/Qwen3.6-35B-A3B-FP8", messages=[ {"role": "system", "content": SYSTEM_PROMPT}, {"role": "user", "content": prompt}, ], temperature=0.1, max_tokens=8192, extra_body={"chat_template_kwargs": {"thinking": False}} )
Figure 7: Sample Response sent to LLM The LLMs were hosted locally using VLLM backend with the command shown in Figure 8. An NVIDIA RTX PRO 6000 Blackwell Workstation Edition Graphics Card with 96GB VRAM was used to host the LLMs.
nohup vllm serve ~/models/gemma-4-26B-A4B-it-FP8 --port 8000 --tensor-parallel-size 1 --max-model-len 32768 --gpu-memory-utilization 0.85 --language-model-only --enable-prefix-caching --kv-cache-dtype fp8 --max-num-seqs 64 --performance-mode throughput --max-num-batched-tokens 32768 > ~/vllm-gemma4.log 2>&1 &
Figure 8: Sample LLM Server Setup Command
Figure 10: Sample Secret 2