Prompt Coach: An Empirical Evaluation of an Agentic Tutor for Learning Prompt Engineering in Software Development Rohit Mehra† , Kapil Singi† , Vikrant Kaulgud† , Vibhu Saujanya Sharma† , Swapnajeet Gon Choudhury† , Swati Sharma‡ , Adam P. Burden*, Majd Sakr* † Accenture Labs, India ‡ Accenture, India *Accenture, USA {rohit.a.mehra,kapil.singi,vikrant.kaulgud,vibhu.sharma,s.g.choudhury}@accenture.com {swati.c.sharma,adam.p.burden,majd.sakr}@accenture.com
arXiv:2607.06074v1 [cs.SE] 7 Jul 2026
Abstract Prompt engineering has emerged as a critical yet undertaught skill for software developers, one that traditional learning approaches are ill-equipped to support given its evolving, interactive, and context-dependent nature. In this paper, we introduce Prompt Coach (PC), an agentic tutor that helps developers learn how to craft high-quality code-generation prompts through Socratic guidance embedded in-flow within their IDE. PC evaluates prompt quality across multiple dimensions and surfaces targeted questions to guide self-correction, grounded in the developer’s codebase and the behavior of the target LLM. We present an early empirical study with 15 professional developers combining quantitative prompt quality scoring with qualitative perception measures. Participants showed statistically significant improvements after a single 60minute session, with the largest gains across dimensions commonly overlooked by developers. They also reported strong trust, high adoption readiness, and unanimous agreement that PC improved their prompt-writing skills.
1
Introduction
With the rapid proliferation of generative and agentic AI, software engineering skills are undergoing a fundamental transformation [2]. Many long-standing skills are becoming obsolete, while a new class of skills is emerging and evolving faster than developers can adapt [14, 16]. Recent industry reports project that 80% of developers will need to upgrade their skills by 2027 to remain market relevant [13]. The focus is shifting from mastering static tools, languages, and frameworks to sustained cognitive collaboration with generative and agentic AI systems. Developers are now required not only to produce correct and efficient code, but also to explicitly articulate intent, constraints, context, and quality expectations through natural language interactions that shape AI-generated artifacts [35]. Prompt engineering has consequently emerged as a first-class engineering skill, with organizations increasingly seeking dedicated expertise in this area [12, 36]. Yet, acquiring such skills efficiently and effectively has become an urgent need, one that traditional learning approaches are ill-equipped to meet. We posit that, despite their widespread adoption and proven effectiveness in teaching stable programming concepts, traditional learning models (such as video tutorials, books, documentation, online courses, etc.) have fundamental limitations that make them illsuited for emerging human-AI collaboration skills such as prompt engineering. These limitations include:
• Lack of context: Prompt engineering is deeply tied to the developer’s task, codebase, and workflow, whereas videos and tutorials remain generic and detached from real development contexts [11]. • One-size-fits-all calibration: Developers vary widely in their baseline skills and experience, yet traditional content is typically designed for a single, assumed learner and lacks the ability to adapt dynamically to individual needs [23]. • Inherently interactive practice: Prompt engineering skill develops through iterative cycles of prompting, observing AI-generated code, and refining prompts, a process that static formats cannot effectively support. • Rapid content obsolescence: Static learning materials quickly become outdated as AI systems evolve, often failing to reflect current model behaviors and capabilities [20]. • Tacit, experiential expertise: Much of the expertise involves developing an intuition for model behavior and recognizing subtle failure modes, which is difficult to convey through passive instruction. • Delayed and generic feedback: Effective learning requires immediate, context-specific guidance on the prompt being crafted, whereas traditional formats provide delayed and generic feedback [21]. As a result of these limitations, learning remains decoupled from day-to-day AI-assisted software development, limiting skill transfer, adaptability, and long-term retention. What is needed is a fundamentally different learning paradigm, one that is adaptive, interactive, embedded within the very workflows in which these skills are exercised, and grounded in the developer’s own code and the behavior of the AI systems they target. We believe that recent advances in agentic AI systems have reached a juncture where they can be meaningfully leveraged to fundamentally transform software engineering education [19]. In particular, agentic tutors, defined as LLM-powered systems that can operate autonomously, reason over context, and engage learners through natural, goal-directed interaction, offer a compelling alternative to the traditional learning models discussed above. Unlike static instructional formats, such systems provide adaptive, interactive, and context-aware guidance embedded within the learner’s workflow. Early research has begun to explore this direction for software engineering tasks such as coding and debugging, demonstrating the potential of these systems to improve learning outcomes [17, 31, 37]. Importantly, agentic tutors differ from prior AI-based and GenAI-based tutoring systems, which were often limited to static knowledge models or reactive conversational exchanges. They instead combine multi-step reasoning, tool use, and
Accepted for publication at 41st IEEE/ACM International Conference on Automated Software Engineering (ASE 2026 - Industry Showcase)
ASE-IndustryShowcase ’26, October 12–16, 2026, Munich, Germany
dynamic context adaptation, enabling guidance that is grounded in the learner’s evolving working context. In this paper, we introduce Prompt Coach (PC), a novel agentic tutor that helps developers learn to craft high-quality code-generation prompts through Socratic guidance. PC is realized as a multi-agent system embedded within the integrated development environment (IDE) that reasons over the developer’s prompt in the context of the surrounding codebase and the behavior of the target LLM. It assesses prompt quality across multiple dimensions and surfaces targeted, context-aware questions that guide developers toward iterative refinement through self-correction rather than prescriptive instruction. By design, PC is adaptive, interactive, embedded within the developer’s workflow, and grounded in both the developer’s code and the behavior of the AI systems they target. To evaluate its impact on learning effectiveness, we present the methodology and results of an empirical study involving 15 professional software developers with diverse experience levels and AI exposure. The study shows measurable improvements in prompt-writing proficiency within a short learning session, along with increased developer confidence and evidence of deeper reflective reasoning about prompt construction.
2
Related Work
Traditional AI-based, and more recently LLM-based, tutoring systems have demonstrated strong learning effectiveness, with LLM tutors outperforming active learning classrooms on outcomes and engagement [18]. Within software engineering, recent works have explored agentic approaches, including guiding students through code debugging [17] and simulating multi-agent development teams for learning the software development lifecycle (DevCoach) [37]. Moreover, several AI-powered tools and resources have recently emerged to support prompt engineering. Anthropic and OpenAI provide interactive tutorials and automated prompt optimizers [3, 4, 28, 30]. Systematic surveys and pattern catalogs of prompting techniques have been proposed [33, 38], and empirical guidelines exist for code-generation prompts, although a gap persists between guidelines and practice [27]. Developers continue to struggle with prompt formulation during exploratory coding interactions [6]. Tools such as ChainForge enable prompt variation and hypothesis testing across models [5], while prompts are treated as structured IDE artifacts without integrated support for improvement [22]. Existing approaches to prompt engineering largely provide static guidance or automated optimization, without adapting to a developer’s task or skill gaps, or supporting the learning process itself. While some tools enable prompt exploration or management, they do not actively teach developers how to construct better prompts. Similarly, existing coaching interfaces offer generic elaboration cues but lack grounding in the developer’s working context and the behavior of the target model. To our knowledge, PC is among the first agentic tutoring systems to address these gaps, delivering adaptive, in-flow guidance grounded in the developer’s codebase and target model behavior, with empirical evidence of measurable improvements in prompt quality.
Mehra et al.
Figure 1: Prompt Coach architecture: An agentic tutor for learning prompt engineering using Socratic guidance.
3
Prompt Coach
PC focuses on helping developers learn the skill of crafting highquality code-generation prompts through Socratic guidance delivered within their IDE. It aims to replace passive, disconnected learning formats with adaptive, in-flow tutoring that responds to the developer’s own prompts. Figure 1 shows the high-level architecture of our approach.
3.1
Architecture
PC is designed as a multi-agent tutoring system that operates directly within the developer’s IDE, ensuring that tutoring occurs where code-generation prompts are actually crafted rather than in a separate learning environment. Upon activation, PC ingests the surrounding project context, including the codebase and accompanying artifacts such as design notes, style guides, and requirement specifications (if available). The Project Context Understanding Agent then processes this context to extract structural and semantic information, which is embedded into a project-specific knowledge store backed by a vector database. This knowledge store serves as the grounding that downstream agents query for context-aware reasoning. A central Tutoring Orchestrator coordinates interactions among agents, routes intermediate outputs, and manages the multi-turn flow between the developer and the system. Once the project-specific knowledge store is established, PC begins supporting the developer during prompt construction. When the developer submits a code-generation prompt intended for a target LLM, the Prompt Evaluator Agent assesses its quality. Rather than applying a monolithic notion of prompt quality, the evaluator scores the prompt across a set of explicitly defined dimensions configured in a Dimension Registry. Each dimension receives a score between 0-100, along with a brief rationale, using an LLMas-a-judge evaluation. For this specific implementation (discussed later in Section 3.2), we adopted the following eight representative dimensions, chosen because they are commonly discussed in the literature as properties of effective code-generation prompts: • Clarity: The prompt clearly communicates intent using welldefined terms and avoids ambiguous language [7].
Prompt Coach: An Empirical Evaluation of an Agentic Tutor for Learning Prompt Engineering
ASE-IndustryShowcase ’26, October 12–16, 2026, Munich, Germany
Figure 2: Prompt Coach in action inside the VSCode IDE for an example prompt. (a) Dimensional assessment of the developer’s prompt across the eight quality dimensions (b–c) Contextual Socratic guidance surfaced for two weak dimensions, Inclusion of Constraints and Error Handling. • Specificity: The prompt explicitly defines inputs, outputs, requirements, and, where applicable, concrete examples [7]. • Context Awareness: The prompt incorporates relevant development context, including environment, dependencies, and intended use case [7]. • Adaptability: The prompt supports variability in inputs and scenarios, and allows for flexible behavior [27]. • Inclusion of Constraints: The prompt specifies technical, performance, resource, sustainability, or security constraints that must be satisfied [34]. • Error Handling: The prompt accounts for edge cases, validation requirements, and expected behavior under failure conditions [32]. • Output Requirements: The prompt clearly defines the expected output format, structure, and success criteria [10]. • Testability: The prompt enables systematic verification of generated code through tests or clear validation criteria [24]. Once the prompt has been evaluated, the Socratic Guidance Agent transforms the dimensional scores into pedagogical interventions. Rooted in the Socratic method, a dialogue-based approach that guides learners toward insight through targeted questioning rather than direct instruction, this agent generates context-aware nudges that encourage developers to reflect on and refine weaknesses in their prompt instead of prescribing direct fixes [1]. To make this guidance grounded and actionable, the agent draws on two complementary sources of context: the project knowledge store established earlier, which anchors nudges in the developer’s own codebase and conventions, and the Consequence Preview Agent, which invokes the same class of LLM targeted by the developer to preview the code the current prompt would generate. This preview is not exposed to the developer, as doing so would bypass the learning process by providing a completed artifact rather than encouraging prompt refinement. Instead, it is used internally to identify concrete failure modes arising from the prompt. For example, if a prompt scores low on error handling and the preview indicates that the generated
code would fail on empty inputs, the system may ask: “What should the function do if the input list is empty? Return a default value, raise an exception, or skip silently?” By combining dimensional assessment with project and consequence-grounded reasoning, PC produces guidance that is both targeted to prompt weaknesses and anchored in the developer’s working context and the observable behavior of the target LLM, enabling developers to understand not only what to improve, but why those improvements matter. Finally, as the developer responds to these nudges and refines their prompt, the Developer Modeling and Tracking Agent observes each iteration and updates a dynamic representation of the developer’s evolving proficiency in prompt construction. It analyzes how the developer engages with each round of guidance, including which nudges are acted upon, which dimensions improve rapidly, and which remain persistent blind spots, to infer strengths, weaknesses, and areas of growth. This learner profile is continuously updated with each interaction, enabling PC to adapt both the depth and focus of subsequent guidance. For example, if the Developer Modeling Agent observes that the developer consistently overlooks error handling across multiple prompts, the Socratic Guidance Agent will emphasize questions in that dimension even when subsequent prompts have only minor error-handling gaps. Conversely, dimensions the developer has consistently mastered receive lighter attention, allowing guidance to focus where learning is still needed. By personalizing guidance based on observed behavior rather than a fixed learner profile, PC addresses variability in developer skill levels and supports adaptive, individualized learning. Over time, this enables PC to transition from a static evaluator to a personalized tutor tailored to the developer’s needs. PC is designed not merely to produce better prompts, but to develop more proficient prompt engineers. By integrating context understanding, adaptive developer modeling, structured prompt evaluation, and Socratic guidance within the IDE, the system turns each code-generation attempt into a reflective learning opportunity. Rather than prescribing what to write, it encourages developers
ASE-IndustryShowcase ’26, October 12–16, 2026, Munich, Germany
Table 1: Tasks used in the evaluation study, drawn from the APPS benchmark across three complexity levels. Phase Baselining
Post-Learning
APPS ID
Complexity
AI-Refined Description
4987
Introductory
1961
Interview
Write a function that takes two dates as input and calculates the difference in years. Create a program that simulates a browser with basic history navigation. It should allow starting from a homepage, visiting new pages, and moving backward or forward through the visited history. Write a program that simulates pings to two servers. Each ping sends multiple packets, some of which succeed and some fail. Based on the ping results, determine whether each server should be considered “alive” (at least half of its packets succeeded) or “dead”.
2207
Competition
2815
Introductory
1924
Interview
2016
Competition
Write a program that takes a sentence as input, builds a list of its words, and outputs a string representing the positions of each word in that list. Write a program that checks a list of transactions and flags ones that break certain validity rules, such as large amounts or conflicts with other transactions. Write a program to help Alex plan a trip across cities connected by roads. Each city has an associated fun score, and Alex wants to maximize the total fun score of the cities he visits. He starts in one city, can revisit cities, but may not immediately travel back on the same road he just used. The goal is to find the maximum possible score he can achieve.
to reason about what is missing, why it matters, and how the surrounding code and target model context shape the outcome. As developers engage with the system over time, this repeated reflection fosters the deeper intuition required for effective prompt engineering, addressing the limitations of traditional learning approaches and enabling in-flow skill acquisition.
3.2
Implementation
To demonstrate the applicability of our approach and evaluate its impact on learning, we implemented PC as a working prototype integrated within the VSCode IDE as an extension [26]. The system is built as a multi-agent architecture using the CrewAI framework, with backend services deployed on Azure [9, 25]. Project context is indexed and retrieved using a ChromaDB-based vector store, enabling context-aware reasoning across agents [8]. Prompt evaluation and guidance are powered by GPT-4.1 from OpenAI, which serves both as the target LLM and as the underlying model for LLMas-a-judge and consequence preview mechanisms [29]. Figure 2 presents snapshots of PC in action, illustrating dimensional scoring of prompts and the corresponding contextual Socratic guidance surfaced to the developer.
4
Empirical Evaluation
We conducted an early empirical evaluation to assess the impact of PC on developers’ prompt-engineering proficiency, using a mixedmethods design that combined quantitative measurement of promptwriting proficiency with qualitative assessment of developer perceptions. The study was guided by the following research questions: • RQ1: What is the baseline quality of prompts that developers write when performing code-generation tasks? • RQ2: What is the measured learning impact of PC on developers’ prompt-engineering proficiency? • RQ3: How do developers perceive and reflect on using PC as an agentic tutor embedded in their daily coding workflows?
4.1
Methodology
We recruited 15 professional software developers from our delivery centers to participate in the study. The participants recorded a
Mehra et al.
Table 2: Baseline and post-learning prompt-quality scores, with relative improvement and paired Wilcoxon signed-rank p − 𝑣𝑎𝑙𝑢𝑒𝑠. Avg. Baseline
Avg. Post-Learning
%Increase
p − 𝑣𝑎𝑙𝑢𝑒
Overall
63.04
71.69
+ 13.73 %
<.001
Complexity-Wise Introductory Interview Competition
65.63 62.56 60.66
74.48 69.05 71.34
+ 13.93 % + 10.38 % + 17.71 %
0.003 0.004 0.001
Dimension-Wise Clarity Specificity Context Awareness Adaptability Inclusion of Constraints Error Handling Output Requirements Testability
79.87 68.58 56.56 66.89 50.51 52.56 70.07 59.29
79.71 72.07 69.91 69.58 66.49 68.67 77.91 69.20
- 00.19% + 05.09% + 23.61% + 4.02 % + 31.63% + 30.66% + 11.20% + 16.72%
0.577 0.003 0.002 0.208 <.001 <.001 <.001 0.001
mean development experience of 9.6 years (ranging from 3 years to 22 years). Participants were selected only if they used AI-coding assistants in their day-to-day roles, regardless of whether they had previously taken prompt-engineering training. The study followed a single-arm, within-subjects, pre/post design conducted remotely via video conferencing, with a moderator guiding each session. This design was chosen to establish an initial within-subject measurement of PC’s learning impact before extending to comparative evaluations against alternative learning approaches in future work. The study was structured across five phases, with an average duration of approximately 2 hours and 30 minutes per participant: • Pre-Study Questionnaire: Participants completed a questionnaire capturing demographic and background information, including development experience, career level, prior prompt-engineering training, and frequency of AI coding assistant usage. • Baselining Phase: Participants wrote code-generation prompts for three tasks spanning introductory, interview, and competitionlevel complexity, sampled from the widely used APPS (Automated Programming Progress Standard) benchmark [15] (10 minutes per task). To prevent verbose task descriptions from artificially inflating prompt quality scores, each description was refined using an LLM to remove extraneous details while preserving task intent. Table 1 lists the tasks used in the study. Participants had no prior exposure to PC at this stage. • Learning Phase: Participants engaged in self-directed learning using PC with the three prompts crafted in the previous phase (60 minutes). The moderator first conducted an end-toend demonstration of PC and assisted participants in setting it up on their systems before the learning phase began. • Post-Learning Phase: Participants wrote code-generation prompts for three additional APPS tasks at the same complexity levels (10 minutes per task), with access to PC revoked during this phase. The tasks were distinct from those in the baselining phase to prevent memorization effects. • Post-Study Questionnaire: Finally, participants completed a questionnaire capturing their perceptions of PC and the
Prompt Coach: An Empirical Evaluation of an Agentic Tutor for Learning Prompt Engineering
ASE-IndustryShowcase ’26, October 12–16, 2026, Munich, Germany
Table 3: Post-study qualitative questionnaire results captured on a 7-point Likert scale and one-sided Wilcoxon signed-rank p − 𝑣𝑎𝑙𝑢𝑒𝑠 tested against the neutral midpoint. (% Agreement = participants responding ≥ 5, and IQR = Interquartile range) Category Learning Effectiveness
Definition Perceived improvement in prompt-writing skills and quality
Parameter Learning Intervention Prompt Quality Prompting Velocity
Perceived enhancement in thinking, Cognitive Elaboration reasoning, and self-reflection during prompt writing Confidence
Cognitive Depth
Trust and Personalization Adoption Readiness
Perceived trust in the tutor and personalization of its guidance. Likelihood of adopting, recommending, and continuing to use Prompt Coach
Trust Perceived Personalization Adoption Likelihood Recommendation Likelihood Engagement Comparative Perception
Question asked to the participant The learnings were effective in improving my prompt-writing skills. I believe that the prompts I created post-learning could lead to higher quality generated code. Post-learning, I spent more time in crafting high-quality code-generation prompts. Post-learning, I have started thinking more about what other details should be specified in my prompt. The learnings improved my confidence to write effective code-generation prompts. I trust these learnings to guide me in crafting better code-generation prompts. Learnings were contextual/personalized based on my inputs. I would consider integrating Prompt Coach into my regular development workflow for continuous learning. I would recommend Prompt Coach to other developers within my team for improving their prompt-writing skills. Prompt Coach’s Socratic guidance approach was engaging. Compared to other learning approaches (like books, blogs, articles, tutorials, and MOOCs), I perceive Prompt Coach as more useful for improving my prompt-writing skills.
learning experience. All responses were recorded on a 7point Likert scale (1 = Strongly Disagree, 7 = Strongly Agree). Prompts collected during the baselining and post-learning phases were subsequently scored across the eight quality dimensions defined in Section 3, using an LLM-as-judge evaluation with the same model and prompting setup as the Prompt Evaluator Agent. The resulting per-participant, per-dimension prompt-quality scores form the basis for the quantitative analysis addressing RQ1 and RQ2, while responses from the post-study questionnaire inform RQ3.
4.2
Results
RQ1: Baseline quality of developer-written prompts: Across the three baselining tasks, participants achieved a mean promptquality score of 63.04, indicating that professional software developers produce prompts of moderate quality. Table 2 summarizes the quantitative results. Further, baselines were broadly consistent across complexity levels (introductory: 65.63, interview: 62.56, competition: 60.66). A Pearson correlation (𝑟 = 0.11) between years of experience and baseline score indicates that prompt-writing proficiency does not meaningfully scale with programming experience, suggesting it is a distinct skill and an emerging area of expertise. Dimension-wise, however, variation was striking: developers scored highest on Clarity (79.87) and Output Requirements (70.07), but lower on Inclusion of Constraints (50.51), Error Handling (52.56), and Context Awareness (56.56). This pattern suggests that while developers naturally produce clear prompts that communicate intent, they tend to underspecify constraints, edge cases, and contextual details critical for downstream code generation. RQ2: Learning impact of PC: After a 60-minute learning session with PC, post-learning prompts achieved a mean score of 71.69, representing a 13.73% relative improvement over the baseline (ranging from -0.28% to 38.37%). Notably, 13 out of 15 participants showed improvements, while 2 participants exhibited no measurable gain on average. Importantly, no participant showed a meaningful decline in performance. The slight negative change (-0.28%) can be attributed to minor variability inherent in LLM-as-a-judge evaluations, which may introduce small fluctuations in scores across runs. Statistical significance was assessed using the paired Wilcoxon signed-rank test, with significant results (𝑝 < 0.05) highlighted in grey (ref. Table 2). These results highlight the positive impact of PC on improving prompt-writing proficiency within a relatively short
Mean 6.33 5.73
SD 0.62 1.79
Median 6.0 6.0
IQR 1.00 1.00
% Agreement 100% 86.7%
p−𝑣𝑎𝑙𝑢𝑒 <0.001 0.013
5.73
0.96
6.0
0.50
93.3%
0.002
6.20
1.57
7.0
1.00
93.3%
0.010
5.93
1.39
6.0
1.00
86.7%
0.005
6.00 5.73 5.73
1.07 1.16 1.39
6.0 6.0 6.0
1.00 1.50 2.00
93.3% 86.7% 86.7%
0.001 0.003 0.009
6.00
1.31
6.0
1.00
93.3%
0.004
5.67 5.80
1.63 1.47
6.0 6.0
1.50 1.50
80.0% 80.0%
0.013 0.009
learning period. Notably, these gains were achieved after a single 60-minute session, suggesting that sustained interaction with the system may yield further improvements. Furthermore, the largest gains were observed in Inclusion of Constraints, Error Handling, and Context Awareness, which were also the lowest-scoring dimensions at baseline, suggesting that PC is particularly effective at addressing developers’ cognitive blind spots. RQ3: Learner perceptions of PC: Participants reported strong positive sentiment across all learning dimensions. Table 3 summarizes the results. Statistical significance was assessed using a one-sample Wilcoxon signed-rank test against the neutral midpoint, with Holm-Bonferroni correction, with significant results (𝑝 < 0.05) highlighted in grey. All parameters exhibited mean scores ranging from 5.67 to 6.33 on a 7-point Likert scale. Participants unanimously agreed that PC improved their code-generation prompt-writing skills (100% agreement), expressed strong trust in the guidance provided, and reported increased cognitive elaboration during prompt construction. Adoption readiness was similarly high, with most participants indicating that they would integrate PC into their regular workflow and recommend it to colleagues.
4.3
Threats to Validity
Our findings are based on 15 developers from a single organization using a single code-generation benchmark (APPS), which may limit generalizability. To keep the study mechanics consistent across participants, the study was conducted in isolation without access to real project context, which PC is designed to leverage during actual development workflows. Finally, due to limited participant availability, the learning session was restricted to 60 minutes. Longer engagement may yield different patterns of impact. Despite these considerations, the large observed effect size and participant-level consistency support the robustness of our findings.
5
Conclusion and Future Work
This paper presents our work on leveraging agentic AI to transform how developers learn emerging skills such as prompt engineering. We introduced PC, an agentic tutor that delivers adaptive, contextual, Socratic guidance within the IDE to help developers learn how to craft high-quality code-generation prompts. An initial empirical evaluation with 15 professional developers demonstrated statistically significant improvements in prompt-writing proficiency after
ASE-IndustryShowcase ’26, October 12–16, 2026, Munich, Germany
a single 60-minute session. Participants also reported strong positive perceptions of the learning experience. These findings position agentic tutoring as a promising paradigm for enabling in-flow, context-aware learning of emerging software engineering skills. Looking ahead, we plan to conduct larger-scale studies with longer engagement durations and additional measures such as retention and downstream code quality. Comparative evaluations against traditional learning approaches and other GenAI-based tutors will help isolate the specific impact of PC. We also aim to deploy PC in real-world project settings to evaluate the benefits of context-aware reasoning.
Data Availability Statement The PC implementation and study data cannot be publicly released due to organizational IP and confidentiality constraints. All aggregate results are reported in full within the paper. Interested members may contact the authors for further discussion.
References [1] Erfan Al-Hossami, Razvan Bunescu, Justin Smith, and Ryan Teehan. 2024. Can Language Models Employ the Socratic Method? Experiments with Code Debugging. In Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE 2024). Association for Computing Machinery, New York, NY, USA, 53–59. doi:10.1145/3626252.3630799 [2] Mamdouh Alenezi and Mohammed Akour. 2025. AI-Driven Innovations in Software Engineering: A Review of Current Practices and Future Directions. Applied Sciences 15, 3 (2025). doi:10.3390/app15031344 [3] Anthropic. 2024. Prompt Engineering Interactive Tutorial. https://github.com/ anthropics/prompt-eng-interactive-tutorial Accessed: 2026-04-08. [4] Anthropic. 2026. Prompt Improver. https://console.anthropic.com Accessed: 2026-04-08. [5] Ian Arawjo, Chelse Wang, Andrew McNutt, Ryo Suzuki, Erik Argyle, Caleb Singh, and Bjoern Hartmann. 2024. ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’24). Association for Computing Machinery, New York, NY, USA. doi:10.1145/3613904.3642016 [6] Shraddha Barke, Michael B. James, and Nadia Polikarpova. 2023. Grounded Copilot: How Programmers Interact with Code-Generating Models. Proceedings of the ACM on Programming Languages 7, OOPSLA (2023), 85–111. doi:10.1145/ 3586030 [7] Zhenpeng Chen, Chong Wang, Weisong Sun, Xuanzhe Liu, Jie M. Zhang, and Yang Liu. 2026. Promptware Engineering: Software Engineering for Prompt-Enabled Systems. ACM Trans. Softw. Eng. Methodol. (Feb. 2026). doi:10.1145/3796535 [8] Chroma. 2026. Chroma: The Open-Source Embedding Database. https://www. trychroma.com Accessed: 2026-04-08. [9] CrewAI. 2026. CrewAI: Fast and Flexible Multi-Agent Automation Framework. https://github.com/crewAIInc/crewAI Accessed: 2026-04-08. [10] Sophia DiCuffa, Amanda Zambrana, Priyanshi Yadav, Sashidhar Madiraju, Khushi Suman, and Eman Abdullah AlOmar. 2025. Exploring Prompt Patterns in AIAssisted Code Generation: Towards Faster and More Effective Developer-AI Collaboration. In 2025 IEEE 4th International Conference on Computing and Machine Intelligence (ICMI). 1–7. doi:10.1109/ICMI65310.2025.11141320 [11] Ian Drosos, Advait Sarkar, and Andrew D. Gordon. 2024. "My toxic trait is thinking I’ll remember this": gaps in the learner experience of video tutorials for feature-rich software. arXiv:2404.07114 [cs.HC] https://arxiv.org/abs/2404.07114 [12] Denis Federiakin, Dimitri Molerov, Olga Zlatkin-Troitschanskaia, and Andreas Maur. 2024. Prompt engineering as a new 21st century skill. Frontiers in Education Volume 9 - 2024 (2024). doi:10.3389/feduc.2024.1366434 [13] Gartner. 2024. Gartner Says Generative AI will Require 80% of Engineering Workforce to Upskill Through 2027. https://www.gartner.com/en/newsroom/pressreleases/2024-10-03-gartner-says-generative-ai-will-require-80-percent-ofengineering-workforce-to-upskill-through-2027 Accessed: 2026-04-08. [14] GitHub. 2025. The new identity of a developer: What changes and what doesn’t in the AI era. https://github.blog/news-insights/octoverse/the-new-identity-of-adeveloper-what-changes-and-what-doesnt-in-the-ai-era/ Accessed: 2026-04-08. [15] Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt. 2021. Measuring Coding Challenge Competence With APPS. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, J. Vanschoren and S. Yeung (Eds.),
Mehra et al.
Vol. 1. https://datasets-benchmarks-proceedings.neurips.cc/paper_files/paper/ 2021/file/c24cd76e1ce41366a4bbe8a49b02a028-Paper-round2.pdf [16] Matthew Kam, Cody Miller, Miaoxin Wang, Abey Tidwell, Irene A. Lee, Joyce Malyn-Smith, Beatriz Perret, Vikram Tiwari, Joshua Kenitzer, Andrew Macvean, and Erin Barrar. 2025. What do professional software developers need to know to succeed in an age of Artificial Intelligence?. In Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering (Clarion Hotel Trondheim, Trondheim, Norway) (FSE Companion ’25). Association for Computing Machinery, New York, NY, USA, 947–958. doi:10.1145/3696630.3727251 [17] Priyanka Kargupta, Ishika Agarwal, Dilek Hakkani Tur, and Jiawei Han. 2024. Instruct, Not Assist: LLM-based Multi-Turn Planning and Hierarchical Questioning for Socratic Code Debugging. In Findings of the Association for Computational Linguistics: EMNLP 2024, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA. [18] Greg Kestin, Kelly Miller, Anna Klales, Timothy Milbourne, and Gregorio Ponti. 2025. AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authentic educational setting. Scientific Reports 15, 1 (3 6 2025), 17458. doi:10.1038/s41598-025-97652-6 [19] Georgios Kostopoulos, Vasileios Gkamas, Maria Rigou, and Sotiris Kotsiantis. 2025. Agentic AI in Education: State of the Art and Future Directions. IEEE Access 13 (2025), 177467–177491. doi:10.1109/ACCESS.2025.3620473 [20] Stefan Kuchemann, Karina E. Avila, Yavuz Dinc, Chiara Hortmann, Natalia Revenga, Verena Ruf, Niklas Stausberg, Steffen Steinert, Frank Fischer, Martin Fischer, Enkelejda Kasneci, Gjergji Kasneci, Thomas Kuhr, Gitta Kutyniok, Sarah Malone, Michael Sailer, Albrecht Schmidt, Matthias Stadler, Jochen Weller, and Jochen Kuhn. 2025. On opportunities and challenges of large multimodal foundation models in education. npj Science of Learning 10, 1 (Feb. 2025), 11. doi:10.1038/s41539-025-00301-w [21] Angélique Létourneau, Marion Deslandes Martineau, Patrick Charland, John Alexander Karran, Jared Boasen, and Pierre Majorique Léger. 2025. A systematic review of AI-driven intelligent tutoring systems (ITS) in K-12 education. npj Science of Learning 10, 1 (14 5 2025), 29. doi:10.1038/s41539-025-00320-7 [22] Ziyou Li, Agnia Sergeyuk, and Maliheh Izadi. 2025. Prompt-with-Me: in-IDE Structured Prompt Management for LLM-Driven Software Engineering. In 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE). 3346–3356. doi:10.1109/ASE63991.2025.00276 [23] Ben Liu, Jihai Zhang, Fangquan Lin, Xu Jia, and Min Peng. 2025. One Size doesn’t Fit All: A Personalized Conversational Tutoring Agent for Mathematics Instruction. In Companion Proceedings of the ACM on Web Conference 2025 (Sydney NSW, Australia) (WWW ’25). Association for Computing Machinery, New York, NY, USA, 2401–2410. doi:10.1145/3701716.3717527 [24] Noble Saji Mathews and Meiyappan Nagappan. 2024. Test-Driven Development and LLM-based Code Generation. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering (Sacramento, CA, USA) (ASE ’24). Association for Computing Machinery, New York, NY, USA, 1583–1594. doi:10.1145/3691620.3695527 [25] Microsoft. 2026. Microsoft Azure Cloud Computing Platform. https://azure. microsoft.com Accessed: 2026-04-08. [26] Microsoft. 2026. Visual Studio Code. https://code.visualstudio.com Accessed: 2026-04-08. [27] Alessandro Midolo, Alessandro Giagnorio, Fiorella Zampetti, Rosalia Tufano, Gabriele Bavota, and Massimiliano Di Penta. 2026. Guidelines to Prompt Large Language Models for Code Generation: An Empirical Characterization. arXiv:2601.13118 [cs.SE] https://arxiv.org/abs/2601.13118 [28] OpenAI. 2024. Prompt Engineering Guide. https://platform.openai.com/docs/ guides/prompt-engineering Accessed: 2026-04-08. [29] OpenAI. 2026. Introducing GPT-4.1 in the API. https://openai.com/index/gpt-4-1/ Accessed: 2026-04-08. [30] OpenAI. 2026. Prompt Optimizer. https://platform.openai.com Accessed: 202604-08. [31] Seetaram Rao Rayarao, Nilesh Dnyaneshwar Bhandarwar, Rahul Dattangire, and Sudhakar Vuriti. 2025. An Autonomous Agentic Framework for Interactive Code Mentoring and Error Resolution using Large Language Models. In 2025 Modern Electronics Devices and Intelligent Communication Systems (MEDCOM). 45–51. doi:10.1109/MEDCOM67532.2025.11404910 [32] Xiaoxue Ren, Xinyuan Ye, Dehai Zhao, Zhenchang Xing, and Xiaohu Yang. 2024. From Misuse to Mastery: Enhancing Code Generation with Knowledge-Driven AI Chaining. In Proceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering (Echternach, Luxembourg) (ASE ’23). IEEE Press, 976–987. doi:10.1109/ASE56229.2023.00143 [33] Sander Schulhoff, Michael Ilie, Nishant Balepur, Konstantine Kahadze, Amanda Liu, Chenglei Si, Yinheng Li, Aayush Gupta, HyoJung Han, Sevien Schulhoff, et al. 2024. The Prompt Report: A Systematic Survey of Prompting Techniques. arXiv:2406.06608 [cs.AI] https://arxiv.org/abs/2406.06608 [34] Catherine Tony, Nicolás E. Díaz Ferreyra, Markus Mutas, Salem Dhif, and Riccardo Scandariato. 2025. Prompting Techniques for Secure Code Generation: A Systematic Investigation. ACM Trans. Softw. Eng. Methodol. 34, 8, Article 225 (Oct. 2025), 53 pages. doi:10.1145/3722108
Prompt Coach: An Empirical Evaluation of an Agentic Tutor for Learning Prompt Engineering
[35] Christoph Treude and Margaret-Anne Storey. 2025. Generative AI and Empirical Software Engineering: A Paradigm Shift . In 2025 2nd IEEE/ACM International Conference on AI-powered Software (AIware). IEEE Computer Society, Los Alamitos, CA, USA, 233–239. doi:10.1109/AIware69974.2025.00033 [36] An Vu and Jonas Oppenlaender. 2026. Prompt Engineer: Analyzing Hard and Soft Skill Requirements in the AI Job Market. arXiv:2506.00058 [cs.CY] https: //arxiv.org/abs/2506.00058 [37] Tianjia Wang, Matthew Trimble, and Chris Brown. 2025. DevCoach: Supporting Students Learning the Software Development Life Cycle with a Generative
ASE-IndustryShowcase ’26, October 12–16, 2026, Munich, Germany
AI powered Multi-Agent System. In Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering (Clarion Hotel Trondheim, Trondheim, Norway) (FSE Companion ’25). Association for Computing Machinery, New York, NY, USA, 987–998. doi:10.1145/3696630.3727255 [38] Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, and Douglas C. Schmidt. 2023. A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT. arXiv:2302.11382 [cs.SE] https://arxiv.org/abs/2302.11382