arXiv CSopen access

From Preventive to Reactive: How AI Coding Assistants Transform Developers' Security Awareness

From Preventive to Reactive: How AI Coding Assistants Transform Developers’ Security Awareness

arXiv:2605.23130v1 [cs.HC] 22 May 2026

Faisal Haque Bappy⋆ , Tahrim Hossain⋆ , Sidratul Muntaher Meheraj◦ , Annoor Sharara Akhand◦ , Tasfia Tabassum◦ , Tarannum Shaila Zaman⋆ , Raiful Hasan⋄ , Tariqul Islam⋆ ⋆ University of Maryland Baltimore County, ◦ University of Dhaka, ⋄ Kent State University

Abstract

1

AI coding assistants are now central to professional software development, yet their impact on how developers think about and practice security remains poorly understood. While prior work has documented vulnerability rates in AI-generated code, a more fundamental question persists: how do these tools transform security awareness in authentic, ongoing development practice? We conducted semi-structured interviews with 15 professional software engineers and observed them completing security-relevant coding tasks with AI assistance, spanning 3 experience cohorts defined by their relationship to AI tools during professional formation. We find that AI coding assistants reorganize rather than eliminate security thinking, shifting it from the act of writing code to the act of reviewing it. This transition from preventive to reactive security is structurally encouraged by interaction models that frame code generation as a functional task, leaving security as an afterthought. Notably, none of our coding session participants specified security requirements in their initial prompts, even when they possessed the relevant knowledge, revealing a decoupling of security awareness from security behavior. We further document informal coping strategies developers had independently invented to manage AI security risk, none of which are supported by current tools or organizations, and find that the experience cohort did not reliably predict security performance. This paper contributes a practice-grounded account of how AI-assisted development reshapes the human side of secure coding, offering empirical foundations for the design of more security-aware tools, training programs, and organizational policies.

Introduction

Software development is changing faster than our understanding of its consequences. AI coding assistants, including GitHub Copilot [17], ChatGPT [31], and Claude [3], have become part of the daily infrastructure of professional engineering work within just a few years [13, 21, 39]. Developers now routinely delegate code generation to these tools, accepting suggestions, iterating on prompts, and shipping the results at a pace that was previously impossible [6, 45]. The productivity case for this shift is well established [39]. The security case is considerably less so. What we already know is concerning. Pearce et al. [38] found that roughly 40% of code generated by GitHub Copilot across high-risk vulnerability scenarios was exploitable, spanning weakness categories well documented in the OWASP Top Ten [33]. Perry et al. [40] found that developers with access to an AI assistant produced less secure code than those without, and were simultaneously more likely to believe their code was secure. This combination of increased vulnerability and increased confidence points to something more troubling than simple tool failure: AI coding assistants may be quietly reshaping how developers perceive and engage with security, not just in the code they produce, but in the habits of mind they bring to reviewing it. Yet the existing literature stops short of explaining how or why this happens in practice. Technical evaluations measure code outputs in controlled settings [38,40], but say little about the cognitive and behavioral processes that produce those outputs in authentic, ongoing development work. The AIassisted programming literature has documented interaction patterns [6], usability expectations [45], and productivity outcomes [39, 51], but has largely treated security as peripheral. The developer security literature, meanwhile, has shown that security is difficult to sustain even without AI in the picture: developers avoid static analysis tools because the warnings lack actionable context [20], the information sources they consult shape the vulnerabilities they introduce [1], and the shift away from manual information foraging [8] toward automated

Copyright is held by the author/owner. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee. USENIX Symposium on Usable Privacy and Security (SOUPS) 2026. August 23–26, 2026, Hannover, Germany.

1

suggestion carries risks the field has not yet fully worked through. The automation trust literature adds another layer of concern, showing that people exhibit systematic biases toward trusting automated outputs [25, 35], biases that tend to grow stronger as tools perform well on average, making occasional failures harder to notice and more consequential when they slip through [5, 22]. None of these bodies of work, individually or together, provides a qualitative account of how AI coding assistants transform security awareness as a lived practice. The central question this paper addresses is: How do AI coding assistants influence the ways professional software engineers approach security, and how do these effects differ based on developers’ levels of exposure to AI tools during their professional formation? To answer this, we conducted a mixed-methods qualitative study with 15 professional software engineers, combining semi-structured interviews with observed coding sessions in which participants completed security-relevant implementation tasks using AI assistants. Participants were drawn from 3 experience cohorts: Pre-AI developers who built their foundational practices before AI tools were widely available, AIera developers who adopted AI mid-career, and AI-native developers who have used AI throughout their professional lives. Our analysis follows established thematic analysis procedures [9, 11], combining iterative coding of interview transcripts with systematic behavioral observation of the coding sessions. Our findings highlight several interconnected patterns that together point to a meaningful shift in how security awareness operates when AI enters the development workflow:

sistent with the broader automation literature [25, 35], participants applied similar levels of trust to both boilerplate (i.e., standardized template or scaffolding code reused across projects) and security-critical code, even though the tool provided no indication that the risks associated with these categories were different. Importantly, experience cohort did not reliably predict security performance during the coding sessions, suggesting that years of experience is a limited proxy for security-aware AI use. • Developers have built their own coping strategies that tools and organizations have not yet caught up with. Participants described independently developing practices such as restricting agent permissions, constructing security-oriented prompt templates, and explicitly asking AI whether any security concerns remained after completing a functional implementation. These strategies were effective when applied, but they existed entirely outside any formal guidance or tool support, leaving their consistent use to individual initiative. What these findings share is a common structural explanation: the shift from preventive to reactive security awareness is not primarily a knowledge problem or a carelessness problem. It is a structural outcome of how AI coding assistants are currently designed and adopted [2, 32], and addressing it will require intervention at the level of tools, training, and organizational practice. This paper takes a step toward that intervention by documenting, in developers’ own words and actions, how the shift is already underway.

• Security thinking moves from authorship to review. Security considerations, traditionally exercised during the act of writing code [29], now appear to be displaced downstream into the review phase. As a result, they are introduced later in the development cycle, where they must compete with time pressure and are applied less consistently. Notably, none of the participants in our coding sessions included security requirements in their initial prompts, even those who had just articulated the relevant vulnerability concerns in the interview, suggesting that security knowledge and security prompting behavior can come apart even among aware engineers.

2

Related Work

Our study draws on and critically engages with four bodies of literature: AI-assisted software development, developer security behavior and awareness, trust and reliance in AI systems, and the intersection of AI tools with security practices. While substantial work exists in each area, we identify significant gaps that our qualitative investigation addresses.

2.1

AI-Assisted Software Development

The emergence of large language models trained on code repositories has fundamentally altered how software is written [13, 51]. Tools like GitHub Copilot, ChatGPT, and Claude Code now offer real-time code suggestions within development environments, and empirical evaluations have demonstrated measurable productivity gains. Peng et al. [39] found that developers completed tasks significantly faster with Copilot, while Vaithilingam et al. [45] documented mixed results depending on task complexity and developer experience. However, the existing literature on AI coding assistants suffers from a narrow fixation on productivity metrics. Weisz

• Security outcomes depend heavily on what engineers bring to the interaction. AI coding assistants do not apply security constraints by default and produce functionally plausible but security-incomplete code unless explicitly directed otherwise. This places the burden of security squarely on the individual engineer’s knowledge and prompting habits, creating uneven outcomes that are difficult to address through tool adoption alone. • Trust is poorly calibrated to security sensitivity. Con2

et al. [49] investigated IBM’s AI coding assistant and found that developers mainly used it for code comprehension and explanation rather than generation, challenging the dominant framing of these tools as mere code producers. Critically, their participants described coding with AI as a collaborative process involving shared authorship and responsibility, yet the study stopped short of examining what happens to security-specific responsibilities in this shared model. Pu et al. [41] introduced Codellaborator, a proactive AI programming assistant, and acknowledged the tension between AI proactivity and user control, finding that users feared losing ownership and understanding. Their recommendation that AI tools should slow down generation to help users grasp code logic is well taken, but the study did not investigate whether this loss of understanding extends to security-critical reasoning, a gap our work directly addresses. The growing literature on proactive AI assistants further illustrates this blind spot. Chen et al. [14] explored proactive chat-based AI assistants for programming and confirmed productivity improvements, but their design considerations around user control and context awareness entirely omit security as a dimension. Similarly, ChainForge [4] and ChainBuddy [52] advanced prompt engineering toolkits for LLM pipelines, yet neither considered how developers verify the security properties of AI-generated outputs. Zhang et al. [52] even noted that participants using ChainBuddy became dependent on the AI and often missed AI-generated errors, a finding with alarming implications for security that the authors did not explore. Kretzer et al. [24] demonstrated LLM-powered GUI prototyping that reduced manual effort, but the question of whether reduced manual engagement also reduces security scrutiny remained unasked. Chen et al. [12] studied how novices and experts learn agent-based modeling with LLM companions and found that novices need substantially more support in planning, understanding, and debugging. Their finding that LLMs should adapt guidance based on skill level aligns with our observation that developers across experience cohorts interact with AI coding assistants in different ways. However, the prior study did not treat security knowledge as a distinct dimension of expertise that warrants adaptation. Across this body of work, the consistent omission of security as a variable of interest is itself a finding: the field has implicitly treated security as orthogonal to AI-assisted development, an assumption our study challenges.

remain stubbornly prevalent [33]. Johnson et al. [20] found that static analysis tools fail because developers misinterpret warnings, lack context for severity assessment, or face integration friction. Acar et al. [1] showed that the choice of information source significantly affects the security of code developers produce, with Stack Overflow’s copy-paste culture contributing to vulnerability propagation. The replication study by Ortloff et al. [32] on expert versus non-expert security practices provides important longitudinal context: over the past decade, non-experts have increasingly adopted expert-level practices like two-factor authentication, VPNs, and ad blockers, driven largely by improved usability and institutional mandates. However, the same study identified a troubling consequence of automation: as security processes become less visible, users lose their sense of control and awareness. This tension between automation benefit and awareness erosion is precisely what we observe in the AI coding assistant context, but manifested at a more granular, code-level scale. Mowar et al. [30] developed CodeA11y, a Copilot extension that reinforced accessible coding practices, demonstrating that AI-generated code still requires human review and that developers, especially novices, miss important issues without explicit guidance. While their focus was accessibility rather than security, the underlying dynamic is transferable: when AI handles generation, developers’ capacity to identify nonfunctional quality attributes degrades unless the tool actively surfaces them. Widder et al. [50] examined the ethical concerns of software engineers more broadly, finding that engineers’ power to resolve concerns was constrained by financial precarity, workplace culture, and organizational incentives. Their finding that attempts to address ethical issues were “often psychologically taxing and ineffective” helps explain why developers may deprioritize the security review of AI generated code. Such review introduces additional friction into workflows that AI tools were adopted specifically to streamline.

2.3 Trust, Reliance, and Transparency in AI Systems Understanding how developers calibrate their trust in AI coding assistants requires engaging with the broader trust and automation literature. Parasuraman and Riley’s foundational work on automation [35] identified systematic biases toward trusting automated outputs, while Lee and See [25] established frameworks for appropriate reliance. In AI-specific contexts, the question of when trust is justified has received renewed attention. Manzini et al. [28] argued that user trust in advanced AI assistants is only justified when there is evidence of both the AI’s competence and its alignment with user values, requiring interventions at the assistant design, organizational, and governance levels. This theoretical framework exposes a criti-

2.2 Developer Security Behavior and Awareness Security has long been recognized as a persistently challenging dimension of software development [2, 29]. Despite decades of research and extensive documentation of secure alternatives, common vulnerability categories such as injection flaws, broken authentication, and sensitive data exposure 3

cal gap in current AI coding assistant deployments: developers have limited evidence of either competence or alignment when it comes to security, yet trust levels remain high. Wang et al. [47] investigated trust formation in AI code generation tools specifically, finding that developers base trust on perceived ability, integrity, and benevolence, and that trust varies with task complexity and stakes. Their finding that current tools lack affordances for validating output quality directly supports our observation that developers lack mechanisms for security-specific trust calibration. Pareek et al. [36] showed that explicit trust repair strategies play a significant role in restoring trust after AI failures. Among the strategies they evaluated, model updates were the most effective in rebuilding trust, while denial tended to deepen distrust. For security contexts, this has practical implications: when AI-generated code introduces a vulnerability, the absence of any acknowledgment or repair mechanism may lead to sustained inappropriate trust. Lee et al. [26] found that presenting users with multiple, potentially inconsistent LLM outputs reduced overreliance by making the model’s limitations more visible. In fact, exposure to inconsistencies improved user comprehension rather than undermining it. This suggests that the current design pattern of presenting a single “best” code suggestion may actively suppress the critical evaluation that security review demands. Serafini et al. [43] conducted the most directly relevant experimental work, testing how security prompts and AI warning messages affect developers’ behavior when using insecure AI-generated code. Their findings are sobering: developers frequently trusted insecure AI suggestions like MD5 hashing and failed to verify outputs properly. While security prompts and guidelines reduced the use of insecure code, the study was conducted in a controlled experimental setting with manipulated ChatGPT responses. Our qualitative investigation complements this by examining how trust and verification behaviors manifest in developers’ authentic, ongoing practices rather than one-time experimental tasks. The broader literature on AI transparency further contextualizes our findings. Chan et al. [10] argued that autonomous AI agents introduce unique risks requiring visibility measures, including agent identifiers, real-time monitoring, and activity logging. Turri et al. [44] demonstrated through a case study that current methods for identifying user transparency needs are too narrow, requiring multi-method approaches to capture diverse stakeholder needs. For AI coding assistants, neither visibility into agent decision-making nor transparency about security-relevant choices is currently provided, leaving developers to rely on heuristic judgment.

2.4

SMS phishing detection with explainable AI, but identified the risk of overreliance on AI and difficulty in error correction when the AI misclassifies messages. These challenges mirror what we observe in code generation: high baseline performance breeds complacency, and the cognitive cost of overriding AI suggestions discourages critical evaluation. Research on integrating LLMs into security incident response [23] found that experts preferred AI-assisted summaries in the majority of cases, but over half of the summaries required factual corrections. The gap between perceived utility and actual reliability is a recurring theme in AI assisted security work. Our study shows that this gap also appears in code generation settings, where developers rely on AI outputs for security sensitive tasks without sufficient verification. Studies on AI adoption in cybersecurity settings reveal persistent barriers. Research examining security experts’ perspectives on augmented intelligence [42] found that only about a third of organizations use or plan to use AI for protection, with experts requiring adaptability, transparency, determinism, and openness as prerequisites for trusted collaboration. Research on AI adoption in industrial control systems [16] similarly found that human tasks relying on experience and intuition are better suited for AI-assisted rather than AI-automated support. These findings from security operations contexts reinforce our argument that security critical code generation requires human in the loop oversight. However, current AI coding assistants do not provide mechanisms to distinguish between security sensitive and routine code generation tasks. Madaio et al. [27] investigated how AI practitioners learn about responsible AI, finding that existing materials are predominantly oriented toward technical and checklist-based approaches, while practitioners aspire to sociotechnical understanding. This disconnect between available learning resources and needed competencies parallels what we observe: developers need to understand the security implications of AI-assisted development holistically, but available guidance remains narrowly technical.

2.5

Gaps Addressed by This Work

The literature reveals three critical gaps. First, research on AI coding assistants has overwhelmingly prioritized productivity, usability, and learning outcomes while treating security as peripheral. Second, the developer security literature has not yet reckoned with how AI-mediated code generation transforms the cognitive and practical dimensions of security awareness. Third, while trust and transparency research provides useful frameworks, it has not been applied to understanding how developers calibrate security-specific trust in AI-generated code across different experience levels. Our study bridges these gaps through a qualitative investigation that foregrounds security awareness as the central variable of interest, examines how it transforms rather than simply persists or disappears across developer experience

AI at the Intersection of Security

A small but growing body of work examines AI tools within security-specific contexts, though this literature remains fragmented. The SmishX system [48] achieved high accuracy in 4

cohorts, and grounds its findings in developers’ authentic practices and reasoning rather than controlled experimental tasks.

confirmed existing themes without introducing new ones.

3

Each interview followed a semi-structured guide designed to move from general reflections on security practices toward more specific questions about AI tool usage. We began by asking participants to describe how they approach security in their day-to-day work, whether they rely on any particular tools, frameworks, or mental habits, and how confident they feel about catching security issues before code ships. We then probed their use of AI coding assistants (including Claude Code, ChatGPT, Gemini, and Cursor) specifically, exploring whether and how these tools had changed their security workflows, any experiences where AI-generated code introduced or concealed a security problem, and how they evaluate the trustworthiness of AI output. Interviews were conducted remotely via video conferencing and recorded with explicit participant consent.

3.2

Methodology

The study consisted of 2 components conducted in sequence during a single session lasting approximately 45 to 60 minutes. In the first component, we conducted a conversational semi-structured interview exploring participants’ experiences maintaining security in their day-to-day development work, their familiarity with AI coding assistants, and how (if at all) these tools had changed how they think about security. In the second component, participants completed a Think-Aloud coding task using Gemini CLI [19] with the Gemini 2.5 Pro model, during which we observed their prompting strategies, decision-making processes, and any security-relevant behaviors or omissions. Both components were designed to surface not just what developers say about security, but what they do when AI assistance is readily available.

3.1

3.3

Participants

Interview Procedure

Think-Aloud Coding Task

Following the interview, participants were invited to complete a coding task using Gemini CLI with Gemini 2.5 Pro. Rather than assigning a fixed task, we offered 3 options aligned with common development activities: implementing a new feature (Task 1), initializing a new project (Task 2), or debugging existing code (Task 3). Participants selected whichever best matched their regular work to preserve ecological validity. One participant (P13) declined this component, and one participant (P5) attempted all of them. To observe participant behavior without introducing the overhead of screen sharing software, we hosted a collaborative development environment using a self-hosted VS Code server [15]. Participants accessed this environment through a shared link, and interviewers observed the session live from their own screen, recording their view throughout. Gemini CLI with Gemini 2.5 Pro was pre-installed and configured within the environment so that participants could begin working immediately without setup friction. This setup allowed us to capture prompting sequences, code edits, iteration patterns, and any security-relevant decisions or omissions in real time.

We recruited 15 professional software developers between November 2025 and January 2026 using snowball [18] and purposive sampling [37] o ensure diversity in roles, experience levels, and familiarity with AI coding tools. All participants were industry professionals; no students or university employees were included. Participation was voluntary and uncompensated. We did not collect employer or organizational identifiers, as discussing AI usage and security practices may carry workplace sensitivity. Because our study examines how developers engage with AI-assisted code generation, we grouped participants according to when they entered the profession relative to the emergence of large language model tools, allowing us to analyze how professional formation shapes interaction with AI systems. Participants were categorized into 3 AI relationship cohorts: Pre AI developers began their professional software engineering careers before LLM based tools became widely available; AI era developers were trained prior to this shift but entered the workforce as AI tools became mainstream; and AI native developers both trained and began their careers entirely within the era of accessible AI coding assistants. Cohort boundaries were set at pre-2022, 2022 to 2023, and 2024 onward, corresponding to the period before, during, and after widespread LLM tool adoption. Participants ranged from entry-level to mid-level engineers across roles, including full-stack development, machine learning engineering, frontend engineering, research and development, and general software engineering. Table 1 provides a complete summary of participant demographics and study characteristics. Thematic saturation was reached at approximately 13 participants, after which the final 2 interviews

3.3.1

Task Design Rationale

The 3 coding tasks were designed to probe security awareness across distinct phases of the software development lifecycle. Each was grounded in a realistic scenario with no explicit security requirements, so that participants’ natural reasoning would be observable rather than primed. Full task prompts are provided in Appendix C. The feature implementation task asked participants to build a /update-profile HTTP endpoint accepting a JSON 5

ID P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15

Gender* M F M M M M F M M F M M M M M

Table 1: Participant Demographics and Study Characteristics Role Experience AI Cohort Coding Task Selected Mid-Level Full-Stack Developer 5 yrs Pre-AI Feature Implementation Full-Stack Engineer 2 yrs Pre-AI Project Planning ML Engineer L2 4 yrs Pre-AI Project Planning Mid-Level R&D Engineer 2 yrs Pre-AI Debugging Mid-Level Full-Stack Developer 4 yrs Pre-AI All Three Tasks Mid-Level Frontend Engineer 4 yrs AI-era Feature Implementation Mid-Level ML Engineer 3 yrs AI-era Debugging Entry-Level R&D Engineer 1 yr AI-native Feature Implementation Mid-Level R&D Engineer 2 yrs AI-era Debugging Mid-Level Software Engineer 2 yrs AI-era Feature Implementation Full-Stack Developer 1 yr AI-native Debugging Software Engineer 3 yrs Pre-AI Debugging Entry-Level ML Engineer 1 yr AI-native Declined† Entry-Level Full-Stack Developer 1 yr AI-native Project Planning Entry-Level Full-Stack Developer 1 yr AI-native Feature Implementation

* The gender imbalance reflects broader trends in the software engineering profession and may affect the generalizability of findings. † P13 completed the interview but declined the coding task due to time constraints.

payload with user_id, display_name, and bio. It was designed to surface whether participants spontaneously applied input validation, authorization checks, or sanitization when handling untrusted user data, a persistent source of web application vulnerabilities [34] that developers frequently overlook even when aware of its importance [46] The project initialization task asked participants to scaffold a backend project with a configuration system, basic routing, and a /health endpoint. It probed whether participants spontaneously considered secure defaults, secret management, or environment separation at project inception, decisions that are substantially harder to retrofit later [29]. The debugging task presented a concurrency vulnerability in a Python file upload function that wrote all uploads to a shared temporary path before renaming, causing files to mix between users under simultaneous load. It probed reactive security reasoning: whether participants would recognize a latent race condition [7] as a security issue rather than treating it as a purely functional bug.

3.4

Data Analysis

3.4.1

Interview Analysis

decisions and track how themes evolved across the dataset. We paid particular attention to tensions between participants’ stated security practices and the behaviors they described when using AI tools. 3.4.2

Coding Session Analysis

Screen recordings of the coding sessions were reviewed alongside the source code files produced during each session. Analysis focused on three dimensions: (1) prompting strategies and iteration patterns, including how participants refined or escalated prompts in response to AI output; (2) securityrelevant decisions, including whether participants reviewed, questioned, or accepted AI-generated code that contained or omitted security considerations; and (3) task completion approach, including how participants structured their workflow and how much independent verification they performed. Coding session observations were triangulated with interview responses to identify participants whose stated practices aligned or diverged from their observed behavior.

3.5

Limitations

We want to acknowledge a few limitations in this study. The sample (N=15) is small and male-skewed, so the findings should be treated as exploratory rather than generalizable. The interview-first design may have primed participants to reflect on security before the coding task, though most still did not address security during it, suggesting any priming effect was minimal. We used Gemini CLI to ensure session consistency, which may not fully reflect other AI-native tools. Finally, the think-aloud setting may have introduced some deliberateness not present in routine work.

Interview transcripts were analyzed using reflexive thematic analysis [9]. The first author conducted open coding of all transcripts, generating an initial set of codes grounded in participants’ own language. The complete codebook, including all 20 codes across 6 thematic categories with definitions and representative quotes, is provided in Appendix A. These codes were then organized into candidate themes through iterative discussion with the broader research team. Analytical memos were maintained throughout to document interpretive 6

4

Ethics Statement

5.1.1

A particularly consistent pattern across participants was the framing of AI as a junior colleague rather than an authoritative system. P1 articulated this most explicitly, describing his approach as treating the AI “as my colleague who is slightly a junior engineer ... I trust him, but not 100%, I have to do my work and ensure credibility and reliability.” P2 echoed this framing, stating that engineers should “treat AI output as a reference, not the final code, and as an assistant, not a replacement.” This mental model appeared to function as a practical heuristic for managing trust: participants who held this view were more likely to describe careful review practices and less likely to report accepting suggestions wholesale. Notably, however, participants who endorsed this framing in the interview did not consistently translate it into security-oriented prompting behavior during the coding sessions, suggesting the mental model shapes how engineers describe their practice more reliably than it shapes the practice itself.

This study received approval from our Institution’s Review Board (IRB) prior to data collection. All 15 participants were industry professionals recruited through purposive and snowball sampling via professional networks. Participation was voluntary and uncompensated. Participants provided informed consent, were told they could withdraw at any time, and are referred to by anonymous codes throughout. No identifiable personal data beyond voluntary demographics was collected.

5

The Junior Engineer Mental Model

Findings

We discovered a consistent and recurring tension across all participant groups: software engineers simultaneously rely on AI coding assistants as indispensable productivity tools while harboring persistent doubts about the security and reliability of the code they produce. Participants with more years of experience tended to articulate more specific security concerns and name vulnerability classes more precisely, while junior engineers more often framed security as something to catch in review rather than something to specify upfront. What united all groups, however, was an acknowledgment that AI has fundamentally changed the nature of software development work, even if its implications for code security remain personally and professionally unsettled.

5.1.2

Task Allocation and the Limits of AI Delegation

The junior engineer framing also helps explain variation in how participants allocated AI assistance across task types, a pattern that became especially visible during the coding sessions. P3 described using AI primarily to generate logical outlines that he then rewrites almost entirely by hand, and during the session, he deliberately chose the project planning task, explaining that it was “the most fitting task for AI ... it initializes backend, generates boilerplate, makes project runnable. This is something I would use AI for without any thought.” He was equally explicit about what he would not delegate, stating that “the third task is something I would NOT use AI for ... debugging something, I do not trust AI to do.” P15 similarly avoided the debugging task on specification grounds, noting that “I don’t want to debug something that isn’t completely spec’d out.” The consensus across this group was that AI performs best on narrowly scoped, welldefined tasks and degrades significantly when given broader autonomy over complex or underspecified problems.

5.1 Productivity Gains and Shifting Development Workflows The most universally reported change brought about by AI coding assistants was a dramatic reduction in the time required to complete development tasks. Participants across all experience levels described AI as having compressed timelines that previously spanned days or weeks into hours. P2 captured this shift most directly, noting that tasks which once took a month or more could now be completed within a week. This acceleration was particularly pronounced for boilerplate code, utility functions, and routine feature implementation, tasks that participants consistently described as well-suited to AI delegation. Before the widespread availability of AI tools, participants described relying on a patchwork of resources, including Stack Overflow, documentation, YouTube tutorials, and online blogs, a process that AI has largely replaced with a single conversational interface. P5 noted that he still turns to Google first for simple problems but increasingly uses AI tools when a task requires planning, scoping, or exploration of unfamiliar libraries, explaining: “when I see it’s a bit complicated, or when I see that there is some sort of planning required, I use it to figure out the scope and then work on it manually.”

5.1.3

Changing Practices Around Code Review

The introduction of AI into development workflows also altered how participants approached code review. P1 described a consistent practice of reading through AI-generated code before running it, checking the workflow, function calls, and parameter handling before proceeding to local testing and then deployment, noting that “if the AI gives me code, I first go through the code ... what are the things happening, how is the workflow going through. If it seems reliable, then I run it locally.” P6 emphasized testing as the primary verification mechanism, stating that “testing is the main thing for AI stuff ... if you don’t validate the code it wrote and go through the code changes, then it’s not done. Manual testing is far better 7

5.2.2

than AI testing.” Several participants acknowledged that review practices tend to be more rigorous for production-bound code and more lenient for internal or experimental work, a distinction that introduces inconsistency into how thoroughly AI output is scrutinized across different development contexts.

A key motivating question for our study design was whether a developer’s relationship to AI tools during professional formation would predict their security behavior in AI-assisted work. Our coding sessions did not support this expectation. The 2 participants who caught unstated security issues (P10 and P12) represent different cohorts (AI-era and Pre-AI, respectively) and different levels of seniority. Participants who missed critical security issues were similarly distributed across cohorts. This null result is informative: it suggests that experience level and AI familiarity, as we operationalized them, are not reliable proxies for security-aware AI use. What appeared to differentiate effective from ineffective security behavior was not years of experience or cohort membership but the specific dimension of security knowledge and whether participants had developed habits of security-oriented prompting. We return to the implications of this finding in the Discussion section.

5.2 Trust Calibration and Security Awareness While productivity gains were celebrated, participants expressed pervasive concern about whether AI-generated code could be trusted from a security standpoint. The majority reported that they do not fully trust AI for security-sensitive implementation, and several described having caught specific vulnerabilities in AI-generated suggestions. P2 noted that AI tools tend to prioritize functionality and readability over security by default, observing that “the biggest concern for me is that code can look correct but still be insecure ... AI often focuses on functionality and readability, not security by default.” Trust ratings across participants clustered in a moderate range, with most assigning scores between 4 and 7 out of 10, reflecting a widespread sense that AI output requires verification but is not without value. Crucially, the coding sessions revealed a sharper picture: none of the 14 participants mentioned security in their initial prompts, and only 2 caught unstated security issues unprompted, a divergence between stated concern and observed behavior that held across experience levels. 5.2.1

Experience Cohort and Security Performance

5.2.3

The Role of Prompt Specificity in Security Outcomes

A secondary theme that emerged from both interviews and coding sessions was the critical importance of prompt specificity in determining the security quality of AI output. P5 described constructing detailed prompt templates that included application structure, API patterns, and testing frameworks as a persistent context layer before issuing any task-specific instructions. P11 took a structured approach during the coding session, creating an agent configuration file that established code style rules and error handling expectations before engaging with the task itself, noting that this helped constrain the AI to a predictable and reviewable mode of operation. P10 observed that “the more context you give, the more refined the answer ... but in the same chat, AI keeps the previous context and gives a mixture of both, even if you tell it not to,” suggesting that prompt discipline must be maintained not only at the outset but throughout an entire session. The coding sessions reinforced this finding: participants who iterated on their prompts with security in mind produced more secure outcomes than those who issued a single functional prompt and accepted the result, though the sample is too small to characterize the magnitude of this effect.

Vulnerability Detection in Practice

The coding sessions surfaced meaningful variation in how participants detected, or failed to detect, security issues in AI-generated code. P10 identified a critical access control flaw in an AI-generated profile update endpoint immediately and without prompting. The AI had retrieved the current user ID from the POST body rather than a secure session store, meaning a malicious user could trivially impersonate any account by manipulating the request. P10 explained: “AI didn’t implement any security feature ... there is no check for that, so any user can update the profile. We need to implement that portion as AI didn’t implement it.” P12 took a different but equally effective approach, explicitly asking the AI whether there were additional security concerns after completing the initial fix. This prompted the AI to surface DDoS vulnerabilities that P12 had not considered: “it also handles some DDoS attack issues as well, which, to be honest, I didn’t think of.” These 2 participants were the only ones to catch unstated security issues across the 14 coding sessions. The remaining 12 participants either did not review for security or expressed satisfaction with outputs that contained unaddressed vulnerabilities, underscoring how substantially security outcomes varied based on individual prompting habits rather than on any consistent behavior the tool itself elicited.

5.2.4

Patterns and Repetition in AI Suggestions

Participants also noted that AI tools exhibit recognizable patterns in their suggestions that can create a false sense of familiarity and reliability. P2 observed that AI tools sometimes appear to retrieve cached responses, repeatedly offering the same solution regardless of how the prompt was varied. P10 described a related issue in which the AI confidently asserted implementation details that did not exist in the actual codebase: “sometimes it says garbage very confidently ... it says 8

5.3.2

that’s how I can do it, but when I look into the codebase, I see there is no such thing like that.” This pattern of confident but incorrect output was raised unprompted by multiple participants as one of the more practically dangerous properties of current tools, because the surface presentation of confidence provides no reliable signal of underlying accuracy.

A specific and recurring security concern across participants was the risk of inadvertently exposing sensitive information, either by sharing proprietary code or credentials with AI tools or by accepting AI-generated code that fails to handle sensitive data appropriately. P5 described an organizational context in which senior leadership initially discouraged external AI tools out of concern that proprietary information would be leaked, noting that “the models aren’t really transparent about how they will use it, and malicious attackers might use that to steal information.” P10 offered a practical framework for managing this risk, advising that engineers should share with AI only what does not expose client details, security keys, or architectural specifics, and concluding that “security vulnerabilities come from people, not from the tools ... you need to be careful what you feed the AI.” P11 independently raised hard-coded API keys and plain-text password storage as the clearest markers of insecure code in his own review process, noting that these are precisely the kinds of issues that are easy to miss when reviewing AI-generated output quickly.

5.3 Diverging Views on Security Awareness Over Time One of the most contested questions across the interviews was whether AI tools make engineers more or less security-aware over time. Participants were genuinely divided, and several acknowledged the tension without resolving it. P3 was among the most candid, stating that AI tools represent a significant hindrance to his security awareness: “I become lazy with AI tools ... security considerations take a backseat in my head. I would overlook things, not read line by line.” He qualified this by noting that the productivity benefits still outweigh the security costs in his personal assessment, but the trade-off was acknowledged rather than dismissed. P5 extended this concern beyond his own practice, observing that “people are becoming less aware because they’re not reviewing the code ... they’re just blindly accepting the code, which shouldn’t be the case.”

5.3.1

Concerns About Data Exposure and Sensitive Information

5.3.3

Education and Security Knowledge as Mediating Factors

Participants consistently pointed to the engineer’s own security knowledge as the primary determinant of whether AIgenerated code ends up being secure or not. P4 described staying current on common vulnerability classes through blogs and capture-the-flag exercises, noting that this background knowledge is what allows him to recognize when AI output introduces broken access control, SQL injection risks, or supply chain vulnerabilities. The coding sessions reinforced this in a way that cuts against intuitive assumptions about experience: P10, despite having approximately 1.5 years of professional experience, caught a critical authorization flaw that more senior participants missed. Meanwhile, P12 (a more experienced Pre-AI developer) used a systematic post-implementation query strategy to surface additional vulnerabilities. Neither pattern aligns cleanly with cohort or seniority as predictors. Instead, both cases point toward specific security knowledge and deliberate verification habits as the operative factors. As P10 put it, “a person should crosscheck it themselves because they know the way it should be implemented ... AI doesn’t.” This convergence across participants suggests a troubling asymmetry: those most capable of catching AI-introduced vulnerabilities are those who already possess strong security knowledge, while those who most rely on AI may be least positioned to evaluate its security outputs critically.

The Risk of Uncritical Acceptance in Production Environments

Several participants identified an emerging pattern of speeddriven code acceptance, where engineers accept AI output wholesale without meaningful review. P5 attributed this partly to organizational pressure, arguing that executives are pushing AI adoption without adequate understanding of its security limitations, warning that “many executives are really pushing it too much without understanding the problems and vulnerabilities ... people are vibe coding production code, and that’s really creating more risks.” The coding sessions provided some direct evidence of this dynamic: 12 of the 14 participants included no security considerations in their initial prompts, and several expressed satisfaction with outputs that contained unaddressed vulnerabilities. P6 acknowledged that while he was satisfied with the output in the research context, “in actual job life, I wouldn’t be” satisfied with the same result, making explicit the gap between research task behavior and his own production standards. Whether this gap reflects the low-stakes research setting, time pressure, or something more persistent about how AI interaction is structured is a question our data cannot fully resolve. 9

5.4 Attitudes Toward AI Autonomy and Agentic Tools

regardless of its accuracy, and contrasting this unfavorably with tools that push back on incorrect premises: “I will 100% change it to be more critical of the user’s opinion.” P12’s practice during the coding session, explicitly asking the AI to identify remaining security concerns after completing a task, surfaced vulnerabilities that neither the engineer nor the initial AI output had raised, pointing to post-implementation security querying as a candidate design primitive. Across participants, the shared vision for improved AI tooling was one in which tools actively participate in quality and security enforcement rather than deferring entirely to engineer prompts, while preserving enough human oversight to catch errors before they reach production.

As participants described their use of agentic AI tools, a distinct set of concerns emerged around the question of how much control engineers should delegate to systems capable of taking actions, modifying files, and executing commands without explicit step-by-step authorization. P3 described a moment during the coding session when he realized the AI agent could execute bash scripts on his local machine, which he found genuinely alarming: “it can execute a bash script on my PC, it can do anything it wants ... that is a fairly strong security consideration for me.” He stated that any indication of unsolicited scripting would cause him to halt the agent immediately and inspect what had occurred before proceeding, reflecting a principled commitment to maintaining human authority over system-level actions. 5.4.1

6

Our findings reveal that the relationship between AI coding assistants and developer security awareness is neither straightforwardly harmful nor straightforwardly beneficial, but is instead deeply mediated by individual knowledge, prompting practice, and organizational context. This complexity resists simple characterization and demands a more nuanced account than either optimistic narratives of AI-assisted productivity or pessimistic warnings about automation complacency have so far provided.

Restricting Agent Permissions as a Coping Strategy

Several participants described actively restricting the permissions available to their AI agents as a practical response to concerns about unintended actions. P9 described removing file write permissions from his agent after repeated experiences of the tool making changes he had not intended, framing this as preserving his own authority over the codebase: “I don’t like to give you the autonomy to do what you want to do ... maybe you’re not right always.” P11 took a more structured approach during the coding session, creating an agent configuration file before beginning the task, using it to define code style expectations and constrain the agent’s behavior, noting that this gave him more confidence in reviewing the output that followed. P3 similarly described a preference for reviewing each proposed change individually rather than accepting batch edits, explaining that “I want nothing decided by it ... I want full control over the code it generates.” These strategies represent an emerging set of informal norms around agentic AI use that participants had developed independently, without formal guidance from their organizations. 5.4.2

Discussion

6.1 Security Awareness as a Function of the Individual, Not the Tool The most striking pattern in our coding sessions was how little the tool itself shaped security outcomes. None of the 14 participants included security requirements in their initial prompts, yet 2 caught critical vulnerabilities through independent knowledge and deliberate querying. P10, among the least experienced session participants, identified a fundamental authorization flaw that more senior developers missed. P12 surfaced additional vulnerabilities by explicitly asking the AI whether security concerns remained, a strategy no other participant employed. What differentiated these 2 participants was not seniority or AI familiarity but specific security knowledge applied at the right moment. This suggests that AI coding assistants amplify existing security awareness rather than supplanting it. Engineers with strong security intuitions extend that intuition into their AI interactions; those without it accept functionally plausible output with no tool-level signal that review is warranted. This dynamic mirrors the asymmetry Perry et al. [40] observed, in which developers using AI produced less secure code while rating it more highly, and extends it: the gap is not just in outputs but in the capacity to evaluate them. As AI lowers the barrier to producing functional code, it may simultaneously raise the effective barrier to producing secure code, concentrating risk among the engineers who most depend on the

Redesigning AI Tools for Greater Security Integration

When asked how they would redesign AI coding tools to better serve security goals, participants converged on several common themes. P2 argued for restricting the access AI tools have to project files and sensitive environment variables, reasoning that limiting exposure reduces the surface area for potential leakage. P5 emphasized efficiency and local execution, arguing that high API costs and cloud dependency create unnecessary risk and that tools should be redesigned to run on local hardware with modest specifications. P11 identified a more fundamental behavioral issue, observing that his preferred agentic tool would accept any input as ground truth 10

tool. Given that current tools provide no mechanism to distinguish security-sensitive from routine generation [42], this concentration is a structural outcome rather than an individual failure.

use consistent rather than contingent on individual resourcefulness.

6.4 Toward Security-Aware AI Coding Assistants

6.2 The Prompting Gap and Its Consequences

Our findings point toward design directions that address structural conditions rather than layering security warnings onto existing interaction patterns. Three directions warrant particular attention. First, security should be a first-class concern in the generation workflow. This could mean automatic security review passes following code generation, prompts requiring engineers to specify security requirements before generation begins in sensitive contexts, or inline flagging of patterns associated with OWASP vulnerability classes [33]. The goal is to make security consideration structurally embedded rather than optional, shifting the default from functional to holistic specification. Second, trust calibration mechanisms should be built into the review interface. Our participants applied similar confidence levels to boilerplate code and security-sensitive logic alike, with no tool signaling that the stakes differed. Lee et al. [26] found that exposing users to multiple potentially inconsistent outputs reduced overreliance by making model uncertainty visible. Applying this principle to securitycritical code sections by surfacing alternative implementations, flagged assumptions, or explicit confidence signals would help engineers calibrate scrutiny to context rather than applying uniform trust across all generated code [47]. Third, organizations must treat security-aware AI use as a competency requiring deliberate investment. Several participants described workplace cultures in which adoption pressure outpaced the development of security review practices, and in which the effort required for careful AI use was structurally discouraged. Madaio et al. [27] found that practitioners need sociotechnical understanding of AI rather than narrowly technical guidance, which suggests that training should address the behavioral and organizational dimensions of AIassisted development alongside the technical ones. Shared prompt templates, review checklists encoding the informal strategies our participants developed, and explicit policies governing what information may be shared with AI tools would institutionalize practices that currently exist only in individual workflows. Tool design alone cannot create these conditions, but guidance framing security-aware AI use as an organizational competency rather than a property that tools deliver automatically is a tractable first step.

A related finding is what we term the prompting gap: the consistent absence of security requirements from initial AI interactions, even among engineers who articulated clear security concerns in the interview. This was not a knowledge failure. Participants who named common vulnerability classes and described authorization best practices did not spontaneously import that knowledge into their prompts. Security was treated as a subsequent concern to catch in review rather than a first-class requirement to specify upfront. The structural explanation is that AI coding assistants are designed and evaluated around functional correctness. The prompt-then-suggest interaction model implicitly frames the engineer’s role as functional specification; non-functional properties like security, accessibility, and performance fall outside that frame unless actively imported [2]. Serafini et al. [43] demonstrated experimentally that even lightweight structural interventions such as security prompts and warning messages meaningfully reduced insecure code use, which suggests the prompting gap is addressable at the interaction level. The implication is that fixing it requires reconsidering the interaction model itself, not simply encouraging engineers to write more thorough prompts.

6.3 Informal Coping Strategies and Their Limits Without formal guidance or tool-level support, participants independently developed strategies for managing AI security risk: restricting agent permissions, creating configuration files to constrain agent behavior, reviewing changes individually rather than in bulk, and explicitly querying the AI for remaining security concerns after completing functional implementation. Where applied, these strategies were effective. Where absent, as was most often the case among less experienced participants, no mechanism existed to fill the gap. The prevalence of these self-invented strategies points to a genuine practitioner need that current tools and organizations are not meeting. This pattern is consistent with Widder et al.’s [50] finding that software engineers’ attempts to address quality concerns are constrained by organizational incentives and unsupported by institutional structures, leaving the burden on individual initiative. The informal strategies our participants developed, and particularly the practice of postimplementation security querying, are candidate patterns for formalization. Evaluated and incorporated into both tool design and training curricula, they could make security-aware AI

7

Design Implications

Our findings, taken together with the structural gaps identified in the existing literature, point toward concrete design 11

7.3

directions for AI coding assistants, developer tooling, and organizational practice. We organize these implications around three levels at which intervention is both warranted and feasible.

Organizations Should Formalize SecurityAware AI Use as a Competency

Tool-level interventions alone are insufficient if the organizational conditions surrounding AI adoption continue to reward speed over scrutiny. Several participants described workplace cultures where pressure to ship code quickly discouraged the additional effort required for security aware AI use. In these environments, informal coping strategies such as permission restriction, iterative security querying, and agent configuration were developed individually rather than shared or institutionalized. Earlier research [50] found that engineers’ attempts to address quality and ethical concerns were constrained by organizational incentives and often psychologically taxing, indicating that individual vigilance is not a sustainable substitute for institutional support. Organizations adopting AI coding assistants should treat security-aware prompting as a formal competency. They should develop shared prompt templates and review checklists that capture the informal strategies our participants described. They should also establish clear policies about what information may be shared with AI tools. Research shows [27] that practitioners need a sociotechnical understanding of AI rather than narrowly technical guidance. Therefore, training should address behavioral and organizational dimensions of AI-assisted development in addition to technical skills. Taken together, these three levels of intervention reflect a recognition that the security awareness gap we identified is not a property of individual engineers but a structural consequence of how AI coding assistants are designed, deployed, and governed.

7.1 Tools Should Treat Security as a Core Interaction Concern The most consequential design change would be to restructure the AI coding assistant interaction model so that security is surfaced proactively rather than left entirely to the engineer. Current tools present code generation as a functional task, implicitly framing security as a post-hoc concern. This framing is not neutral: it shapes behavior, as our coding sessions demonstrated, with zero participants including security requirements in initial prompts despite being able to articulate security concerns when directly asked. Serafini et al. [43] demonstrated experimentally that security prompts and warning messages reduce the use of insecure AI-generated code, suggesting that even lightweight structural interventions at the point of interaction can shift behavior meaningfully. Tools should implement automatic security review passes following code generation, require engineers to confirm security requirements before generation begins for sensitive code contexts, and surface inline warnings for patterns commonly associated with vulnerability classes, such as those documented by OWASP [33]. The goal is not to add friction indiscriminately but to make security considerations a structural part of the generation workflow rather than an optional addition.

8

Conclusion

This paper set out to examine not whether AI coding assistants generate vulnerable code, but how they reshape the security awareness of the developers who use them. The answer is nuanced. AI does not erode security knowledge, but it reorganizes when that knowledge is applied, shifting it from a preventive stance embedded in writing code to a reactive one exercised during review. That shift is a predictable consequence of interaction models that treat code generation as a functional task and leave security to be addressed afterward. Three findings stand out: security knowledge and prompting behavior are decoupled across experience cohorts; informal coping strategies point to a practitioner need that neither tools nor organizations are meeting; and experience level shows no reliable relationship to security performance during coding sessions. The goal is not to slow adoption but to shape it. Tool designers need to treat security as primary in the generation workflow, organizations need to formalize security-aware AI use as a competency rather than assuming it emerges naturally, and researchers need to keep examining how these tools transform practice in ways vulnerability benchmarks cannot capture.

7.2 Trust Calibration Mechanisms Should Be Built Into the Review Interface Poorly calibrated trust, rather than trust itself, poses the core design challenge our findings surface. When developers apply uniform confidence across boilerplate and security-sensitive logic alike, the interface bears partial responsibility: presenting a single best suggestion signals equivalence where none exists. AI coding assistants should differentiate output presentation by security sensitivity, surfacing alternative implementations, and flagging embedded assumptions precisely where the cost of misplaced trust is highest. This reframing shifts the intervention from the developer to the tool. Wang et al. [47] found that developers ground trust in perceived competence and integrity, yet current interfaces offer little to validate either. Making the AI’s security reasoning visible at the point of review would let engineers evaluate not just functional correctness but the soundness of the assumptions underneath it, turning code review from passive acceptance into active scrutiny. 12

References

[11] Kathy Charmaz. Constructing grounded theory (introducing qualitative methods series). Constr. grounded theory, 2014.

[1] Yasemin Acar, Michael Backes, Sascha Fahl, Doowon Kim, Michelle L Mazurek, and Christian Stransky. You get where you’re looking for: The impact of information sources on code security. In 2016 IEEE symposium on security and privacy (SP), pages 289–305. IEEE, 2016.

[12] John Chen, Xi Lu, Yuzhou Du, Michael Rejtig, Ruth Bagley, Mike Horn, and Uri Wilensky. Learning agentbased modeling with llm companions: Experiences of novices and experts using chatgpt & netlogo chat. In Proceedings of the 2024 CHI conference on human factors in computing systems, pages 1–18, 2024.

[2] Ross Anderson. Security engineering: a guide to building dependable distributed systems. John Wiley & Sons, 2010.

[13] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021.

[3] Anthropic. Claude. https://claude.com/product /overview, 2026. Accessed: 2026-02-19. [4] Ian Arawjo, Chelse Swoopes, Priyan Vaithilingam, Martin Wattenberg, and Elena L Glassman. Chainforge: A visual toolkit for prompt engineering and llm hypothesis testing. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, pages 1–18, 2024.

[14] Valerie Chen, Alan Zhu, Sebastian Zhao, Hussein Mozannar, David Sontag, and Ameet Talwalkar. Need help? designing proactive ai assistants for programming. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages 1–18, 2025. [15] Coder Technologies Inc. code-server: VS Code in the Browser. https://github.com/coder/code-ser ver, 2026. MIT License. GitHub repository. Accessed: 2026-02-19.

[5] Gagan Bansal, Besmira Nushi, Ece Kamar, Daniel S Weld, Walter S Lasecki, and Eric Horvitz. Updates in human-ai teams: Understanding and addressing the performance/compatibility tradeoff. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 2429–2437, 2019.

[16] Clement Fung, Eric Zeng, and Lujo Bauer. Adopting {AI} to protect industrial control systems: Assessing challenges and opportunities from the {Operators’} perspective. In Twenty-First Symposium on Usable Privacy and Security (SOUPS 2025), pages 555–573, 2025.

[6] Shraddha Barke, Michael B James, and Nadia Polikarpova. Grounded copilot: How programmers interact with code-generating models. Proceedings of the ACM on Programming Languages, 7(OOPSLA1):85– 111, 2023.

[17] GitHub. Github copilot: Your ai pair programmer. https://github.com/features/copilot, 2026. Accessed: 2026-02-19.

[7] Matt Bishop, Michael Dilger, et al. Checking for race conditions in file accesses. Computing systems, 2(2):131–152, 1996.

[18] Leo A Goodman. Snowball sampling. The annals of mathematical statistics, pages 148–170, 1961. [19] Google. Gemini CLI: An Open-Source AI Agent for the Terminal. https://github.com/google-gemin i/gemini-cli, 2025. Apache 2.0 License. GitHub repository. Accessed: 2026-02-19.

[8] Joel Brandt, Philip J Guo, Joel Lewenstein, Mira Dontcheva, and Scott R Klemmer. Two studies of opportunistic programming: interleaving web foraging, learning, and writing code. In Proceedings of the SIGCHI conference on human factors in computing systems, pages 1589–1598, 2009.

[20] Brittany Johnson, Yoonki Song, Emerson Murphy-Hill, and Robert Bowdidge. Why don’t software developers use static analysis tools to find bugs? In 2013 35th International Conference on Software Engineering (ICSE), pages 672–681. IEEE, 2013.

[9] Virginia Braun and Victoria Clarke. Using thematic analysis in psychology. Qualitative research in psychology, 3(2):77–101, 2006.

[21] Eirini Kalliamvakou. Research: Quantifying GitHub Copilot’s Impact on Developer Productivity and Happiness. https://github.blog/news-insights/re search/research-quantifying-github-copilot s-impact-on-developer-productivity-and-hap piness/, 2022. GitHub Blog. Updated May 21, 2024. Accessed: 2026-02-19.

[10] Alan Chan, Carson Ezell, Max Kaufmann, Kevin Wei, Lewis Hammond, Herbie Bradley, Emma Bluemke, Nitarshan Rajkumar, David Krueger, Noam Kolt, et al. Visibility into ai agents. In Proceedings of the 2024 ACM conference on fairness, accountability, and transparency, pages 958–973, 2024. 13

[22] Rafal Kocielnik, Saleema Amershi, and Paul N Bennett. Will you accept an imperfect ai? exploring designs for adjusting end-user expectations of ai systems. In Proceedings of the 2019 CHI conference on human factors in computing systems, pages 1–14, 2019.

[32] Anna-Marie Ortloff, Jenny Tang, Arthi Arumugam, Daniel Huschina, Lisa Geierhaas, Florin Martius, Luisa Jansen, Kolja Von Der Twer, Lilly Jungbluth, and Matthew Smith. Replication:{“No} one can hack my {mind”}-10 years later: An update and outlook on {experts’} and {non-experts’} security practices and advice. In Twenty-First Symposium on Usable Privacy and Security (SOUPS 2025), pages 435–454, 2025.

[23] Diana Kramer, Lambert Rosique, Ajay Narotam, Elie Bursztein, Patrick Gage Kelley, Kurt Thomas, and Allison Woodruff. Integrating large language models into security incident response. In Twenty-First Symposium on Usable Privacy and Security (SOUPS 2025), pages 133–148, 2025.

[33] OWASP Foundation. OWASP Top 10: 2021. https: //owasp.org/Top10/2021/, 2021. Accessed: 202602-18.

[24] Felix Kretzer, Kristian Kolthoff, Christian Bartelt, Simone Paolo Ponzetto, and Alexander Maedche. Closing the loop between user stories and gui prototypes: an llm-based assistant for cross-functional integration in software development. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages 1–19, 2025.

[34] OWASP Foundation. OWASP Secure Coding Practices – Quick Reference Guide. https://owasp.org/ww w-project-secure-coding-practices-quick-r eference-guide/, 2022. Version 2.0.1. Accessed: 2026-02-19. [35] Raja Parasuraman and Victor Riley. Humans and automation: Use, misuse, disuse, abuse. Human factors, 39(2):230–253, 1997.

[25] John D Lee and Katrina A See. Trust in automation: Designing for appropriate reliance. Human factors, 46(1):50–80, 2004.

[36] Saumya Pareek, Eduardo Velloso, and Jorge Goncalves. Trust development and repair in ai-assisted decisionmaking during complementary expertise. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 546–561, 2024.

[26] Yoonjoo Lee, Kihoon Son, Tae Soo Kim, Jisu Kim, John Joon Young Chung, Eytan Adar, and Juho Kim. One vs. many: Comprehending accurate information from multiple erroneous and inconsistent ai generations. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 2518–2531, 2024.

[37] Michael Quinn Patton. Qualitative evaluation and research methods. SAGE Publications, inc, 1990.

[27] Michael Madaio, Shivani Kapania, Rida Qadri, Ding Wang, Andrew Zaldivar, Remi Denton, and Lauren Wilcox. Learning about responsible ai on-the-job: Learning pathways, orientations, and aspirations. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 1544–1558, 2024.

[38] Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, and Ramesh Karri. Asleep at the keyboard? assessing the security of github copilot’s code contributions. Communications of the ACM, 68(2):96– 105, 2025.

[28] Arianna Manzini, Geoff Keeling, Nahema Marchal, Kevin R McKee, Verena Rieser, and Iason Gabriel. Should users trust advanced ai assistants? justified trust as a function of competence and alignment. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 1174–1186, 2024.

[39] Sida Peng, Eirini Kalliamvakou, Peter Cihon, and Mert Demirer. The impact of ai on developer productivity: Evidence from github copilot. arXiv preprint arXiv:2302.06590, 2023. [40] Neil Perry, Megha Srivastava, Deepak Kumar, and Dan Boneh. Do users write more insecure code with ai assistants? In Proceedings of the 2023 ACM SIGSAC conference on computer and communications security, pages 2785–2799, 2023.

[29] Gary McGraw. Software security. IEEE Security & Privacy, 2(2):80–83, 2004. [30] Peya Mowar, Yi-Hao Peng, Jason Wu, Aaron Steinfeld, and Jeffrey P Bigham. Codea11y: Making ai coding assistants useful for accessible web development. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages 1–15, 2025.

[41] Kevin Pu, Daniel Lazaro, Ian Arawjo, Haijun Xia, Ziang Xiao, Tovi Grossman, and Yan Chen. Assistance or disruption? exploring and evaluating the design and trade-offs of proactive ai programming support. In Proceedings of the 2025 CHI conference on human factors in computing systems, pages 1–21, 2025.

[31] OpenAI. Chatgpt. https://openai.com/chatgpt, 2026. Accessed: 2026-02-19. 14

[42] Neele Roch, Hannah Sievers, Lorin Schöni, and Verena Zimmermann. Navigating autonomy: unveiling security experts’ perspectives on augmented intelligence in cybersecurity. In Twentieth Symposium on Usable Privacy and Security (SOUPS 2024), pages 41–60, 2024.

2023 ACM Conference on Fairness, Accountability, and Transparency, pages 467–479, 2023. [51] Frank F Xu, Bogdan Vasilescu, and Graham Neubig. Inide code generation from natural language: Promise and challenges. ACM Transactions on Software Engineering and Methodology (TOSEM), 31(2):1–47, 2022.

[43] Raphael Serafini, Asli Yardim, and Alena Naiakshina. Exploring the impact of intervention methods on developers’ security behavior in a manipulated chatgpt study. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages 1–26, 2025.

[52] Jingyue Zhang and Ian Arawjo. Chainbuddy: An aiassisted agent system for generating llm pipelines. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages 1–21, 2025.

[44] Violet Turri, Katelyn Morrison, Katherine-Marie Robinson, Collin Abidi, Adam Perer, Jodi Forlizzi, and Rachel Dzombak. Transparency in the wild: Navigating transparency in a deployed ai system to broaden need-finding approaches. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 1494–1514, 2024.

A

Codebook for Interview Analysis

Table 2 presents the full codebook developed through open and focused coding of the 15 interview transcripts. Codes are organized into 6 thematic categories that emerged inductively from the data, yielding 20 codes in total. Each entry includes the code name, a definition, and a representative example quote from a participant.

[45] Priyan Vaithilingam, Tianyi Zhang, and Elena L Glassman. Expectation vs. experience: Evaluating the usability of code generation tools powered by large language models. In Chi conference on human factors in computing systems extended abstracts, pages 1–7, 2022. [46] Daniel Votipka, Rock Stevens, Elissa Redmiles, Jeremy Hu, and Michelle Mazurek. Hackers vs. testers: A comparison of software vulnerability discovery processes. In 2018 IEEE Symposium on Security and Privacy (SP), pages 374–391. IEEE, 2018. [47] Ruotong Wang, Ruijia Cheng, Denae Ford, and Thomas Zimmermann. Investigating and designing for trust in ai-powered code generation tools. In Proceedings of the 2024 ACM conference on fairness, accountability, and transparency, pages 1475–1493, 2024. [48] Yizhu Wang, Haoyu Zhai, Chenkai Wang, Qingying Hao, Nick A Cohen, Roopa Foulger, Jonathan A Handler, and Gang Wang. Can you walk me through it? explainable {SMS} phishing detection using {LLMbased} agents. In Twenty-First Symposium on Usable Privacy and Security (SOUPS 2025), pages 37–56, 2025. [49] Justin D Weisz, Shraddha Vijay Kumar, Michael Muller, Karen-Ellen Browne, Arielle Goldberg, Katrin Ellice Heintze, and Shagun Bajpai. Examining the use and impact of an ai code assistant on developer productivity and experience in the enterprise. In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, pages 1–13, 2025. [50] David Gray Widder, Derrick Zhen, Laura Dabbish, and James Herbsleb. It’s about power: What ethical concerns do software engineers have, and what do they (feel they can) do about them? In Proceedings of the 15

Table 2: Interview Codebook: Categories, Codes, Definitions, and Example Quotes Category Security Practices and Habits

Code Preventive security behavior

Security Practices and Habits

Reactive behavior

Security Practices and Habits

Security tooling

Security Practices and Habits

Security knowledge gaps

AI Tool Usage and Trust

Uncritical AI acceptance

AI Tool Usage and Trust

Critical AI evaluation

AI Tool Usage and Trust Security Awareness Shift with AI Security Awareness Shift with AI

AI-induced complacency Increased security awareness Decreased security awareness

Security Awareness Shift with AI Prompting and Interaction Behavior Prompting and Interaction Behavior

No perceived change Security-explicit prompting Security-absent prompting

Prompting and Interaction Behavior Experience and Cohort Effects Experience and Cohort Effects Experience and Cohort Effects

Iterative security correction Pre-AI security grounding AI-era hybrid approach AI-native dependency

Perceived Responsibility and Ownership

Personal security ownership

Perceived Responsibility and Ownership

Diffused responsibility

Perceived Responsibility and Ownership Perceived Responsibility and Ownership

Organizational security reliance Security as a secondary concern

security

Definition Proactive security measures integrated during development (e.g., validation, sanitization, access control). Security actions taken only after issues are detected via testing or reports. Use of automated tools (e.g., static analysis, dependency scanning) in the workflow. Self-reported uncertainty about vulnerability types or mitigation practices. Integration of AI-generated code with minimal review for correctness or security. Active inspection or modification of AI output before integration. Reduced vigilance attributed to reliance on AI suggestions. Reported growth in security attentiveness due to AI feedback. Reported decline in deliberate security reasoning after AI adoption. Security habits reported as unchanged despite AI use. Explicit inclusion of security requirements in AI prompts. Prompts focused only on functionality without security constraints. Follow-up prompts to correct security issues in AI output. Security mental models formed prior to AI tool adoption. Combined reliance on prior training and AI assistance. Heavy reliance on AI with limited independent security reasoning. Explicit acknowledgment of individual accountability for code security. Ambiguity about accountability when AI-generated code is involved. Dependence on team processes rather than individual initiative. Security framed as subordinate to functionality or deadlines.

16

Example Quote “I always validate what comes in from the client before I do anything with it.” (P1)

“Usually I find out there’s a problem when someone files a ticket.” (P12) “We run Snyk in CI, so vulnerabilities get caught before anything ships.” (P3) “I know SQL injection is a thing, but I wouldn’t know how to test for it.” (P8) “If it runs and tests pass, I usually just commit it.” (P15) “I always read what it gives me. I’ve caught things that looked fine.” (P1) “It should know about these things better than me.” (P11) “Sometimes it suggests something I hadn’t thought of.” (P6) “I used to think through edge cases more.” (P9)

“I do the same things I always did.” (P4) “Make sure this handles invalid input.” (P5) “I just describe what I want it to do.” (P14)

“I told it to add an auth check.” (P10) “I learned this stuff before these tools existed.” (P3) “I know enough to know when it’s wrong.” (P7) “I haven’t really worked without it.” (P8)

“If something is insecure, that’s on me.” (P5)

“I’m not sure who’s responsible.” (P11)

“That’s what code review is for.” (P9) “If the feature isn’t working, nothing else matters.” (P12)

B

Interview Protocol

Part D: Coding Task Reflection

The following semi-structured interview guide was used across all sessions. Questions were adapted conversationally based on participant responses. Interviews averaged 35 minutes.

• Walk me through the security considerations you had in mind during the task.

Part A: Background and Experience

• Are there security features you knew you should implement but did not? Why not?

• Did the AI assistant help or complicate security implementation in this task?

• How long have you been developing software professionally?

• If you were to do this task again, what would you do differently from a security standpoint?

• What types of applications or projects do you typically work on?

Closing

• When did you first start using AI coding assistants, and which tools have you used?

• Is there anything about AI coding tools and security that we did not discuss but you think is important?

• How frequently do you use AI coding assistants in your development work?

C

• Have you received any formal training in secure coding practices or application security?

Coding Task Problem Statements

The following task descriptions were provided to participants. No security-specific requirements were included. Participants were free to use any language or framework.

• How confident do you feel about catching security issues before code ships?

Task 1: Feature Implementation

Part B: AI Tool Usage and Trust

Prompt. Create an HTTP endpoint called /update-profile that allows a user to update their profile information. The client will send JSON containing:

• Walk me through your typical workflow when using an AI coding assistant. • How do you decide when to use AI assistance versus coding something manually?

{ "user_id": 123, "display_name": "Example Name", "bio": "Example Bio" }

• How much do you trust code generated by AI tools, and what influences that level of trust? • How do you evaluate whether AI-generated code is correct and safe to use?

Requirements. • Implement the endpoint in the language and framework of your choice.

• Can you recall a time when an AI tool suggested code that was problematic or insecure? How did you recognize the issue?

• Update the user’s profile in a persistent store or mock database.

• Do you review AI-generated code differently than code you write yourself?

• Handle the fields user_id, display_name, and bio. • Return a JSON response indicating success or failure.

Part C: Security Practices and AI

• Include any validation or request handling logic you consider appropriate.

• How has using AI coding assistants changed how you think about security, if at all? • Can you describe a situation where AI-generated code introduced or concealed a security problem?

Task 2: Project Initialization Prompt. Plan and initialize a backend project in the language and framework of your choice. The project should run with simple commands and include a basic structure for configuration and routing. Requirements.

• Are there security checks or practices you apply consistently regardless of whether code is AI-generated? • Do you feel more or less confident about the security of code when AI was involved in writing it? 17

• Create a project folder with all necessary setup files. • Add a configuration system for managing environmentspecific settings. • Implement a simple /health endpoint returning a status message. • Include instructions for running the project and supplying the configuration. • Provide a minimal but extensible project layout.

Task 3: Debugging Prompt. Users report that when multiple people upload files simultaneously, files occasionally end up in the wrong user’s folder. The following function is used to save uploaded files: def save_file(user_id, uploaded_file): temp_path = "/tmp/upload.tmp" with open(temp_path, "wb") as f: f.write(uploaded_file.read()) final_path = f"/data/{user_id}/{uploaded_file.filename}" os.rename(temp_path, final_path) Requirements. • Identify why files may be mixed between users. • Modify the function so that simultaneous uploads do not interfere. • Ensure files are saved under the correct user’s directory. • Ensure the revised version works reliably under concurrent uploads.

18