Code, Commits, and Complexity in the GenAI Era: A Longitudinal Study of Team-Based Capstone Projects

arXiv:2611.07297v1 [cs.CY] 5 Nov 2026

Code, Commits, and Complexity in the GenAI Era: A Longitudinal Study of Team-Based Capstone Projects Asma Shakil

Claus Brabrand

Paul Denny

University of Auckland Auckland, New Zealand asma.shakil@auckland.ac.nz

IT University of Copenhagen Copenhagen, Denmark brabrand@itu.dk

University of Auckland Auckland, New Zealand paul@cs.auckland.ac.nz

Abstract

1

Generative AI tools are now widely available to students, but little is known about how team-based project codebases have changed since they appeared. This matters when project courses use repository activity as evidence of progress and learning. Existing studies have largely examined self-reports, controlled tasks, or short individual assignments, leaving open questions around how student codebases evolve in authentic project settings. We present a ten-semester analysis of N=260 capstone team repositories from 2021 to 2025, spanning pre- and post-AI cohorts. We examine three dimensions: code size (lines of code), code characteristics (comment density, errorhandling density, and cyclomatic complexity), and development behaviour (commit volume, edit size, and commit frequency). We find that end-of-semester codebases roughly triple, from about 3.4 KLOC in 2021–2023 to about 10 KLOC in 2024–2025. Code characteristics show a clear decoupling: cyclomatic complexity and error-handling density increase, while comment density does not. Later cohorts commit far more often and in greater volume, but individual commit sizes are largely unchanged, suggesting a shift in development rhythm rather than edit granularity. Most changes appear from 2024 onwards and are not uniformly monotonic. We did not directly observe AI use, so we report these as changes that coincide with increased GenAI availability rather than as effects of AI use. Our findings highlight the need to reconsider how studentproduced artefacts are used and assessed in project courses.

Generative AI (GenAI) tools, such as ChatGPT, Claude, and Gemini, are now widely available to students for programming tasks. These tools can generate, refactor, debug, and explain code on demand, raising key questions for education:

CCS Concepts • Social and professional topics → Computing education.

Keywords Generative Artificial Intelligence, GenAI, AI Code Generation, Large Language Models, CS Education, Capstones ACM Reference Format: Asma Shakil, Claus Brabrand, and Paul Denny. 2027. Code, Commits, and Complexity in the GenAI Era: A Longitudinal Study of Team-Based Capstone Projects. In Proceedings of the 58th ACM Technical Symposium on Computer Science Education V.1 (SIGCSE TS 2027), February 17–20, 2027, Sacramento, CA, USA. ACM, New York, NY, USA, 7 pages. https://doi.org/ 10.1145/3857222.3860095

SIGCSE TS 2027, Sacramento, CA, USA © 2027 Copyright held by the owner/author(s). This is the author’s version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in Proceedings of the 58th ACM Technical Symposium on Computer Science Education V.1 (SIGCSE TS 2027), February 17–20, 2027, Sacramento, CA, USA, https://doi.org/10.1145/3857222.3860095.

Introduction

How are student software development practices changing in the presence of GenAI, and how should educators interpret student-produced code in this new landscape? While early evidence suggests that GenAI tools may increase apparent productivity, they also complicate assumptions about effort, learning, and assessment [11, 14], and can expose misalignments between intended learning outcomes and assessment tasks [3]. Recent empirical work has begun to examine these changes. For example, Zhang et al. [24] report that students submit longer solutions, make larger edits, and iterate more frequently after the public release of ChatGPT, often without corresponding improvements in conceptual understanding or performance. However, such studies have mainly examined short-term, individual assignments with fixed specifications. Less is known about longer-running, open-ended, team-based projects, where students collaborate over many weeks, make architectural decisions, and build evolving codebases. Capstone projects therefore provide a useful context for examining how the size, structure, documentation, and commit history of student code have changed as GenAI tools have become available. Pre-AI vs Post-AI. In this paper, we address these gaps through a longitudinal analysis of student capstone repositories across ten semesters (2021:S1–2025:S2), covering the pre-AI (2021–2022) as well as the post-AI (2023–2025) eras. This boundary follows the public release of ChatGPT on November 30, 2022 [16]; since students adopted AI tools at different times, we treat it as approximate (Section 3.1). Using version-control histories from 260 team repositories, we analyse code size and code characteristics at two snapshots (mid- and end-semester), and development behaviour measured across the project window. Contributions. This paper makes four primary contributions: (1) Longitudinal Analysis: We provide one of the first multisemester, repository-level analyses of capstone projects spanning the pre-AI and post-AI eras (260 repositories across ten semesters, from 2021–2025). (2) Measurement Framework: We introduce a multi-dimensional measurement framework spanning i) code size, ii) code characteristics, and iii) development behaviour, enabling a richer characterisation of AI-era change than single-metric studies permit. (3) Silent Complexity: We identify a complexity-w/o-commentary phenomenon in post-AI cohorts, where cyclomatic complexity and error-handling density rise while comment density remains

SIGCSE TS 2027, February 17–20, 2027, Sacramento, CA, USA

flat—a finding with direct implications for code comprehension and assessment in AI-augmented codebases. (4) More, Not Bigger Commits: Later cohorts commit far more often and in greater volume, yet commit size shows no clear trend. This shift in frequency, not granularity, does not support the assumption that AI assistance yields fewer, larger “batch” commits.

2

Related Work

Our work sits at the intersection of three research strands: GenAI’s impact on programming behaviour, repository-level analysis of software development in educational settings, and the assessment of code quality in student projects. We review each in turn below. GenAI’s Impact on Programming Behaviour. The rapid availability of GenAI tools has created new challenges for programming education [18]. These tools can generate, explain, debug, and refactor code, making them highly relevant to tasks traditionally used to assess student understanding [2]. Recent reviews highlight both benefits and risks: GenAI tools may support productivity and feedback, but may also encourage over-reliance and make it harder for educators to judge what students have learned [15, 20]. Empirical studies have begun to examine these effects. Zhang et al. [24] provide a close point of comparison for our work, showing that after the release of ChatGPT, students submitted longer solutions, made larger edits between submissions, and iterated more frequently. However, their study focuses on fixed-assignment tasks, leaving open the question of whether similar changes occur in larger, open-ended projects. Other studies show that students use GenAI tools for both conceptual help and direct code generation [19], and that tools such as GitHub Copilot can accelerate task progress while raising concerns about students’ understanding of the generated suggestions [21]. This body of work shows that GenAI tools can change student programming behaviour. However, most studies focus on shortterm tasks, controlled settings, individual assignments, or selfreported use; less is known about how student codebases evolve over longer periods and across multiple cohorts. Our study addresses this gap through a five-year, ten-semester longitudinal repository-level analysis spanning pre-AI and post-AI cohorts. Repository Metrics in Education. Version-control data have long been used in computing education research to study how students work over time. Git and GitHub histories provide evidence of commit activity, contribution patterns, collaboration, and project progress. Prior studies have used these data to examine student development practices in software engineering and project-based courses. For example, Tushev et al. [22] show that GitHub activity can provide useful insight into student work, while cautioning against treating simple activity metrics as direct measures of learning or performance. Hamer et al. [9] use Git metrics to measure individual and team contributions, and Gitinabard et al. [8] show that GitHub logs can reveal meaningful patterns in teamwork and development behaviour. This prior work demonstrates the value of repository mining in educational contexts. However, much of it predates the widespread use of GenAI and focuses mainly on contribution, teamwork, or

Asma Shakil, Claus Brabrand, and Paul Denny

assessment. Our work builds on this tradition but applies it to longitudinal change across the pre- and post-GenAI eras. We combine behavioural metrics, such as commit volume, commit frequency, and edit size, with artefact-level metrics, such as lines of code, comment density, error-handling density, and cyclomatic complexity. Code Quality in Student Projects. A related stream of computing education research examines the quality and structure of student code. While correctness is often central to programming assessment, code quality is increasingly treated as an important learning outcome in its own right. Keuning et al. [13] show that it is multi-dimensional, spanning readability, documentation, decomposition, complexity, maintainability, and style—and these are precisely the dimensions along which student code tends to fall short. In another study, Keuning et al. [12] identify recurring quality problems in student programs and argue that students need support beyond simply producing working solutions, while Ihantola and Petersen [10] examine code complexity in introductory programming and highlight both the value and the limitations of applying traditional software metrics to student code. This same structural lens has more recently been turned on AI-generated code: in a controlled comparison of functionally equivalent programs, Azeem et al. [1] found that ChatGPT-generated code exhibited higher cyclomatic complexity and Halstead volume than human-written code while scoring lower on documentation and function structure. These studies support the use of static metrics to characterise student-produced software, but they do not directly examine whether such metrics shift in the GenAI era. This matters because AI assistance may affect not only how much code students produce, but also how that code is structured, documented, and maintained. We therefore use multiple complementary metrics to understand how student codebases have changed over time: lines of code for scale, and cyclomatic complexity, error-handling density and comment density for code characteristics. These metrics describe properties of a codebase; they are not measures of code quality, and each has known limitations (Section 3.5).

3 Method 3.1 Context & Study Design Our analysis draws on GitHub repositories from our capstone course, a compulsory final-year course in our undergraduate computer science degree offered twice a year. Students working in teams of five to six build software for real external clients over a 13-week period, following an agile process and using GitHub for version control. The projects span a broad range of domains (web and mobile applications, AI systems, games, simulations, and desktop software) and technologies. JavaScript or TypeScript is the main language in 42–58% of repositories in 2021–2023 and in 71–74% in 2024–2025; Python leads in 2022 (26 of 65 repositories). The working prototype is the primary assessment artefact. Our data spans ten consecutive semesters (S1 2021–S2 2025). We divide the ten semesters into a pre-AI period (2021–2022) and a post-AI period (2023–2025), with the public release of ChatGPT in November 2022 [16] as the boundary, after which general-purpose GenAI assistance became freely and widely available. The boundary is not sharp. GitHub Copilot was released as a technical preview in June 2021 [6] and became free for verified students in June 2022 [7],

Code, Commits, and Complexity in the GenAI Era

so some S2 2022 students may already have used AI code completion. Conversely, uptake after ChatGPT’s release was gradual. For example, among UK undergraduates, the share using GenAI for assessed work rose from 53% in late 2023 to 88% in late 2024 [5]. We therefore use “pre-AI” and “post-AI” as shorthand for periods of lower and higher GenAI availability, not as labels of actual use, and report per-cohort values so that the timing of change can be seen rather than assumed. We define two fixed measurement (snapshot) points within each semester: • Week-7 (mid), following the mid-semester break, when teams present their first minimum viable product; and • Week-13 (end), at the close of the project period, when teams present their final, complete solution. These milestones bracket the two main development phases: the mid-point captures the project at its first functional state, and the end-point captures the finished product. Comparing the two illustrates how projects evolve during their most active period. Snapshot dates are treated as end-of-day UTC cutoffs to ensure consistent and reproducible selection of commits across repositories.

3.2

Research Questions

Our study is guided by the following research questions: • RQ1 (Code Size): How has the volume of code produced in capstone projects changed between the pre-AI era (2021–2022) and the post-AI era (2023–2025), as measured by lines of code? • RQ2 (Code Characteristics): How have the qualitative characteristics of code produced in capstone projects changed between the pre-AI era and the post-AI era? • RQ3 (Coding Behaviour): To what extent has the development behaviour of capstone students shifted between the pre-AI and the post-AI era? RQ1–2 is about (code) product; RQ3 is about (development) process.

3.3

Repository Selection & Dataset

For each semester, we began with all repositories in the GitHub organisation for that cohort’s capstone offering, yielding 302 repositories across ten semesters. We then excluded those not representing active student projects according to three criteria: empty or inactive (no commits or no source code at any snapshot); non-project (testing, demonstrations, or templates and identified through naming conventions and manual inspection); and duplicate (forking or renaming, of which only one per team was retained). This produced a cleaned dataset of N=260 team repositories: 19 in 2021, 65 in 2022, 57 in 2023, 51 in 2024, and 68 in 2025. For each week 7 or 13 snapshot, we use the latest commit on the main branch at or before the cutoff. Repositories with no commit before a cutoff are treated as missing and disregarded. This affects only Week-7, where six 2022 repositories had not yet committed, yielding N=254 at Week-7 (versus the full N=260 at Week-13). Metrics that require both snapshots, such as the mid-semester ratio (Section 3.4) and edit size (Section 3.6), are computed only for repositories present in both snapshots. Further per-metric exclusions occur only when a metric is undefined, e.g. repositories with no functions for cyclomatic complexity.

SIGCSE TS 2027, February 17–20, 2027, Sacramento, CA, USA

3.4

Code Size (RQ1)

To address RQ1, we measure both how much code teams produce and how early in the project that code accumulates. Mid & End Code Size. We measure code size as the cumulative number of lines of code (LOC) present in a repository at two fixed snapshots during the semester: MidCodeSize = repository LOC at week 7 (mid); EndCodeSize = repository LOC at week 13(end). These metrics capture the cumulative amount of source code present at the mid-semester and end-of-semester checkpoints, respectively. At each resolved snapshot commit, the repository is checked out and analysed with tokei, a language-aware static analysis tool1 , which classifies each file’s lines as code, comment, or blank and ignores unrecognised file types. We sum code lines across all detected languages into a cumulative LOC value per repository per snapshot, excluding dependency lock files (e.g., package-lock.json) and the dependency and build directories (e.g., node_modules, virtual environments). Comment lines are retained separately for the comment-density metric (Section 3.5). All values are tied to explicit snapshot commits, ensuring measurements are reproducible and independently verifiable. Mid-Semester Progress. To capture not only how much code teams produce but when they produce it, we compute a mid-semester accumulation ratio as: MidCodeSize . EndCodeSize This metric isolates the timing of code production independently of absolute codebase size. MidSemesterProgress =

3.5

Code Characteristics (RQ2)

We describe three size-normalised characteristics of each codebase at both snapshots (week 7 and week 13): control-flow complexity (mean per-function CCN), explicit error handling code (per KLOC), and the share of comment lines (relative to LOC). Prior work uses similar measures to characterise student and AI-generated code [1, 13]. We treat none of them as a direct measure of quality. Cyclomatic Complexity. To characterise the structural complexity of code, we compute function-level cyclomatic complexity (CCN). CCN reflects branching structure only and relates weakly to how hard code is to understand [4]; we read it as a structural measure, not a measure of comprehensibility. At each snapshot, we analyse source files in the codebase using lizard 2 , which supports several languages including Python, JavaScript/TypeScript, Java, and C# and excludes vendor, build, and generated directories, minified bundles, and files exceeding 2 MB, under per-file and perrepository time limits. For each function lizard reports its CCN; the repository-level statistic is the mean CCN across all functions, 𝑓 ∈ F (the set of all functions): Í 𝑓 ∈F CCN(𝑓 ) MeanCyclomaticComplexity = . |F| Error-Handling Density. We measure how often code contains explicit error-handling constructs; more handling is not necessarily 1 https://github.com/XAMPPRocky/tokei 2 Lizard (language-independent analysis tool): https://github.com/terryyin/lizard

SIGCSE TS 2027, February 17–20, 2027, Sacramento, CA, USA

better, and we do not treat it as evidence of defensive design or of AIgenerated code. For each resolved snapshot commit, the repository is checked out and its source files scanned for language-specific error-handling patterns.3 Files under vendor or build directories, binary files, and files exceeding 2 MB are again excluded. A source line is counted once if it matches at least one pattern, and matched lines are summed across the repository. We normalise this count by the repository’s LOC (Section 3.4) to obtain error-handling density per thousand lines of code: #Error-Handling Lines ErrorHandlingDensity = . KLOC This metric is a lexical proxy: it captures the number of lines containing error-handling constructs rather than a semantic analysis of exception flow, and we treat it accordingly (Section 6). Comment Density. Comment density captures the quantity of in-code commentary, not its quality, and more comments are not necessarily better, as comments may become inconsistent with the code [23]. We compute it as: #Comment Lines , CommentDensity = LOC where blank lines are excluded, and LOC is as defined in Section 3.4.

3.6

Coding Behaviour (RQ3)

To address RQ3, we characterise how students develop their code, including the size and cadence of commits that drive codebase evolution. Median Edit Size. To characterise how code changes are introduced, we compute edit size, a commit-level metric that captures the magnitude of individual commits and, hence, the granularity of development activity. For each repository, we consider all commits after the resolved Week-7 snapshot through and including the Week-12 snapshot commit on the default branch, a window that captures the latter half of the semester, when teams typically focus on feature completion, refactoring, and integration. We use the Week-12 boundary (rather than the Week-13 end-of-semester snapshot used elsewhere) so that edit size spans a fixed five-week development window; repositories with missing or identical Week-7 and Week-12 commits are excluded. Edit size is computed per commit using Git’s diff --numstat, which reports lines added and deleted per file; binary files (for which line counts are undefined) are excluded. For a given commit 𝐶 ∈ C (in the set of all commits in the repository), edit size is the total number of modified lines across all affected files: ∑︁ ( |A (𝑓 )| + |D (𝑓 )| ) EditSize(𝐶) = 𝑓 ∈𝐶

MedianEditSize = median𝐶 ∈C EditSize(𝐶) , where A (𝑓 ) and D (𝑓 ) denote the sets of lines added to and deleted from file 𝑓 , respectively. EditSize therefore captures the total number of modified lines in a commit, whether the changes are additions, deletions, or refactorings. For each repository, we summarise the per-commit edit sizes by their median, which is robust to the extreme outliers common in student repositories (e.g. bulk imports). We report the cohort median of these repository medians. 3 Python

try/except/raise/finally ; JavaScript try/catch/throw/.catch() ; Go

if err != nil/return err ; Rust Result/match/unwrap/? ; etc.

Asma Shakil, Claus Brabrand, and Paul Denny

Commit Volume. Alongside edit size, we characterise development cadence over the full project window on the default branch. Commit volume is the total number of commits per repository: CommitVolume = |C|. Commit frequency is the total number of commits per active day, defined as the total commit count divided by the number of distinct (active) calendar days carrying at least one commit, D: CommitFreqency =

|C| . |D|

Commit frequency distinguishes teams that commit steadily from those that commit in concentrated sessions of the same total volume. We report the cohort median of each.

3.7

Statistical Analysis

We test each metric for a monotonic trend across the five cohorts (2021–2025), treating cohort year as an ordinal predictor at the repository level. We report Spearman’s rank correlation (𝜌) between cohort year and the per-repository value as the effect size, with the Theil–Sen slope as an interpretable estimate of change per year. To control the false discovery rate across the eight4 metrics, we apply the Benjamini–Hochberg procedure, reporting both uncorrected (𝑝) and adjusted (𝑞) values. As repositories are nested within semesters and teams, we restrict inference to repository-level trends and do not model within-team dependence.

4

Results

We examine whether each metric changes consistently across the five cohorts (2021—2025). For each metric, we treat cohort year as an ordered variable and test for a monotonic trend at the repository level. We report Spearman’s rank correlation (𝜌) as the effect size, and use the Benjamini–Hochberg procedure to control the false discovery rate across all metrics (𝑞). Snapshot metrics are measured at the end-of-semester checkpoint, week 13 (𝑁 = 260 repositories). Behavioural metrics are measured across the full project period. Full statistics are shown in Table 1.

4.1

Code Size (RQ1)

End-of-semester code size increases substantially across the study period (Figure 1, top rows of Table 1). The median is 2.3 KLOC in 2021, 3.7 in 2022 and 3.4 in 2023, then rises to 7.7 KLOC in 2024 and 11 KLOC in 2025; the 2024–2025 median is roughly three times the 2021–2023 median. The trend is statistically significant (𝜌 = +0.45, 𝑞 < 0.001), with a Theil–Sen slope of about +2.0 KLOC per year. Mid-semester code size follows the same pattern, rising from 1.4– 1.8 KLOC in 2021–2023 to 2.5 KLOC in 2024 and 4.8 KLOC in 2025. The proportion of final code present by mid-semester (week 7) shows no clear trend (𝜌 = −0.12, 𝑝 = 0.069): later cohorts produce more code overall, but not proportionately earlier in the semester. Result 1 (Code volume roughly triples from 2024): Median end-of-semester repository size rose from 3.4 KLOC in 2021–2023 to 10 KLOC in 2024–2025, with no increase in 2023. 4 MidSemesterProgress is deterministically derived from MidCodeSize and End-

CodeSize and is therefore not counted as an additional (ninth) test.

Code, Commits, and Complexity in the GenAI Era

SIGCSE TS 2027, February 17–20, 2027, Sacramento, CA, USA

Table 1: Per-cohort medians and five-cohort monotonic-trend statistics, grouped by research question. ★ marks trends significant after Benjamini–Hochberg correction (𝑞 ≤ 0.05); non-significant trends are in grey. Median by cohort RQ

Metric

RQ1 (Code Size)

MidCodeSize (KLOC) EndCodeSize (KLOC) MidSemesterProgress (%)

RQ2 (Code Characteristics)

MeanCyclomaticComplexity ErrorHandlingDensity (/KLOC) CommentDensity (%)

RQ3 (Coding Behaviour)

MedianEditSize CommitVolume CommitFreqency

Pre-AI

12

KLOC

10

2023

2024

2025

𝜌

Slope/yr

𝑝

𝑞

★ ★

1.5 2.3 73

1.4 3.7 46

1.8 3.4 58

2.5 7.7 40

4.8 11 51

+0.34 +0.45 −0.12

+0.66 +2.0 −2.8

<0.001 <0.001 0.069

<0.001 <0.001 0.092

★ ★

1.7 10 9.2

2.0 3.7 8.6

1.8 4.8 7.6

2.2 12 7.6

2.5 13 8.4

+0.27 +0.27 −0.04

+0.17 +1.9 −0.19

<0.001 <0.001 0.52

<0.001 <0.001 0.52

65 63 3.3

38 112 4.2

61 100 4.8

51 265 5.3

65 300 5.4

+0.09 +0.38 +0.26

+3.0 +46 +0.45

0.14 <0.001 <0.001

0.16 <0.001 <0.001

4.3 11

Mid (Week 7) End (Week 13) 7.7

6

4.8 3.7

4 2.3

0

2022

★ ★

Post-AI

8

2

2021

1.5

1.4

2021

2022

3.4 1.8

2.5

2023

2024

2025

Figure 1: Median KLOC per repository (RQ1) at mid (Week 7) and end (Week 13) snapshots (■ pre-AI; ■ post-AI).

4.2

Coding activity also increases across the study period (bottom rows of Table 1). The median number of commits per repository rises from 63 in 2021 and around 100–112 in the 2022–2023 cohorts to 265 in 2024 and 300 in 2025. Commit volume shows the strongest trend among the behavioural metrics (𝜌 = +0.38, 𝑞 < 0.001). Commit frequency also increases. The median number of commits per active day rises from 3.3 in 2021 to 5.4 in 2025 (𝜌 = +0.26, 𝑞 < 0.001). However, the size of each commit does not change much. The median number of lines changed per commit after mid-semester shows no clear trend across the 2021–2025 cohorts (𝜌 = +0.09, 𝑝 = 0.14; cohort medians 38–65 lines). This result holds even when repositories with very little activity are excluded. This indicates that overall later cohorts commit much more often and produce much more code, but they do so through a larger number of similarly sized commits, rather than through larger commits. Result 3 (More commits, not bigger ones): Median commit volume tripled, increasing from around 100 commits to around 300 commits per repository. Commits per active day also rose from 3.3 to 5.4. PostAI students commit more often, but not in larger batches.

Code Characteristics (RQ2)

The three code-characteristic metrics show different patterns (middle rows of Table 1). Cyclomatic complexity increases steadily across the cohorts. Mean per-function CCN rises from 1.7 in 2021 to 2.5 in 2025 (𝜌 = +0.27, 𝑞 < 0.001). This suggests that student code becomes more structurally complex over time. Error-handling density also increases, although the pattern is less smooth. It falls from 10 per KLOC in 2021 to 3.7 in 2022 and 4.8 in 2023, then rises to about 12–13 in 2024–2025. This trend is also statistically significant (𝜌 = +0.27, 𝑞 < 0.001). Comment density, in contrast, shows no clear trend (𝜌 = −0.04, 𝑝 = 0.52). It is highest in 2021 (9.2%), lowest in 2023–2024 (7.6%) and 8.4% in 2025, an uneven pattern rather than a steady decline. Overall, student code becomes larger, more complex and contains more error-handling constructs, but comment density does not increase. Result 2 (Complexity without commentary): Cyclomatic complexity and error-handling density both increase significantly. Comment density shows no clear trend. Post-AI code is more complex and contains more error-handling constructs, but is not more documented.

Coding Behaviour (RQ3)

5

Discussion

Our results suggest that the GenAI-era change in capstone development is best understood as multi-dimensional: codebases have grown, development activity has intensified, and code structure has shifted, but these changes occur unevenly across metrics. The Shape of Change: Large, Later, and Uneven. By the end of the semester, repositories in 2024–2025 are about three times the size of those in 2021–2023. This extends the pattern Zhang et al. [24] report from short individual assignments to longer-running team projects, and is consistent with evidence that AI assistance can increase coding throughput [17]. However, the change is not abrupt: code size, complexity, error-handling density, and commit volume change little in 2023 and then rise from 2024 onward. The 2023 cohort appears transitional: it was the first after the public release of ChatGPT, but use of AI tools was likely still uneven (Section 3.1).

SIGCSE TS 2027, February 17–20, 2027, Sacramento, CA, USA

Complexity Without Commentary. The code-characteristic metrics split: cyclomatic complexity and error-handling density rise, but comment density does not. Later cohorts thus submit code that is larger, more complex, and contains more error-handling constructs, but not better documented. This is consistent with evidence that AI-assisted code can increase structural complexity without a commensurate gain in maintainability: Azeem et al. [1] found ChatGPT-generated code to be more complex and less welldocumented than human-written code. The concern is not that students produce more code, but that it may be harder to inspect and maintain—sharpened by evidence that heavy reliance on AI can impair the very skills needed to review it [20]. In project courses, documentation, traceability, and the ability to explain one’s own code may therefore need to become explicit assessment criteria, through viva-style defences, code walkthroughs, or stronger documentation requirements. A Behavioural Signature of Frequency, Not Granularity. Later cohorts commit far more often, both in total and per active day, yet median commit size does not increase. This challenges the expectation that AI-generated code would appear as fewer, larger commits: instead, teams produce more code through more frequent, similarly sized commits. This differs from the assignment-level pattern in Zhang et al. [24], suggesting development behaviour may vary between short individual tasks and longer team projects, and aligns with Shihab et al. [21], who found Copilot shaped students’ processes, not just their outputs. AI assistance may speed how quickly students integrate and revise code without changing commit size, though as our study is correlational, we treat this as a hypothesis rather than a causal claim. Implications for Assessing AI-Era Artefacts. Our findings complicate artefact-based assessment. Metrics that educators may once have treated as signs of effort or progress, such as code volume and commit activity, have increased substantially, while comment density has not. A larger, more complex, more frequently committed codebase is therefore not necessarily evidence of greater learning. This reinforces concerns that traditional programming assessments become less reliable under GenAI: Prather et al. [18] argue for rethinking assessment, and Bouvier et al. [2] emphasise assessing students’ reasoning, process, and ability to explain their work. This does not mean AI use is harmful; rather, assessment is likely more robust when the final codebase is weighed alongside process evidence and students’ ability to justify their work.

6

Threats to Validity

This study has several limitations. First, we do not directly observe students’ use of GenAI. As discussed in Section 3.1, the pre/post boundary marks a change in availability rather than use, so our findings are observed changes during a period of increasing AI availability, not causal evidence of AI use. Other changes over the same period include changes in course staff, assessment criteria, and project types across semesters. The 2021 and some of the 2022 cohorts worked partly remotely under COVID-19 restrictions, and the 2021 cohort is much smaller (19 repositories) than later ones. Changes in frameworks and project templates could also add boilerplate code independently of AI use. Our design cannot separate these factors from changes in GenAI availability.

Asma Shakil, Claus Brabrand, and Paul Denny

Second, as noted in Section 3.5, our code-characteristic metrics are descriptive proxies. For example, our error-handling metric is lexical rather than semantic, and may count language-specific idioms, comments, or strings that do not represent genuine error handling. The language mix shifted towards JavaScript/TypeScript over the study period (see Section 3.1), but restricting the analysis to the 157 repositories whose main language is JavaScript or TypeScript, all five significant trends hold (e.g., EndCodeSize 𝜌 = +0.42, 𝑞 < 0.001). Also, LOC and comment density are derived from tokei, which covers a broader range of languages, whereas cyclomatic complexity is derived from lizard, which supports only a subset. The complexity-without-commentary pattern, which pairs a lizard-based measure with a tokei-based one, should therefore be interpreted at the cohort level, rather than as a direct within-file comparison. Finally, the findings come from a single team-based capstone course at a single institution. They may not transfer to courses with different assessment structures, languages, or student populations. Repositories are also nested within semesters and teams, and our analysis treats repository-level trends descriptively rather than modelling all sources of dependence. Although we control the false discovery rate across metrics, the monotonic trend tests have limited sensitivity to non-monotonic patterns, such as a low in 2022–2023 followed by a rise from 2024. Null results should therefore be read as the absence of a clear monotonic trend, not as evidence that no change occurred.

7

Conclusion

We have presented a ten-semester, repository-level analysis of 260 student capstone projects from 2021 to 2025, examining changes in student code and development practices during the rise of GenAI. We find substantial shifts across code size, code characteristics, and development behaviour. End-of-semester codebases in 2024–2025 are roughly three times the size of those in 2021–2023; cyclomatic complexity and error-handling density increase, while comment density does not. Students also commit more frequently, though individual commit sizes remain largely unchanged. These findings suggest that capstone project work since 2023 is characterised by larger, more complex codebases that contain more error-handling constructs, but are not necessarily better documented. This complexity-without-commentary pattern may make student code harder to understand and assess. Similarly, increased commit activity reflects frequency rather than granularity, meaning traditional indicators such as code volume and commit counts may no longer map cleanly onto student effort or learning. As AI use was not observed, these are changes that coincide with increased GenAI availability, not demonstrated effects of it. Future work should link repository-level trends to observed AI use, distinguish AI-authored from human-authored code, and examine whether rising complexity reflects sound engineering practice or the verbose style often associated with GenAI output.

Acknowledgments We are grateful to reviewers for their many helpful comments, especially regarding our metrics, that have enabled us to strengthen this work.

Code, Commits, and Complexity in the GenAI Era

References [1] Samina Azeem, Muhammad Shumail Naveed, Muhammad Sajid, and Imran Ali. 2025. AI vs. human programmers: Complexity and performance in code generation. VAWKUM Transactions on Computer Sciences 13, 1 (2025), 201–216. [2] Dennis J. Bouvier, Bruno Pereira Cipriano, Richard Glassey, Olga Petrovska, Emma Anderson, Anastasiia Birillo, Ryan Dougherty, Raymond Pettit, Nuno Pombo, Ebrahim Rahimi, Charanya Ramakrishnan, Alexander Steinmaurer, Shubbhi Taneja, Muhammad Usman, and Annapurna Vadaparty. 2026. The Rest of the Robots: Generative AI in Post-introductory Computing Education (ITiCSE-WGR 2025). Association for Computing Machinery, New York, NY, USA, 61–107. doi:10.1145/3760545.3783970 [3] Claus Brabrand and Paul Denny. 2026. Constructive Alignment in the Age of AI. EdArXiv preprint. doi:10.35542/osf.io/m9yfk_v2 [4] Dror G. Feitelson. 2023. From Code Complexity Metrics to Program Comprehension. Commun. ACM 66, 5 (2023), 52–61. doi:10.1145/3546576 [5] Josh Freeman. 2025. Student Generative AI Survey 2025. HEPI Policy Note 61. Higher Education Policy Institute. https://www.hepi.ac.uk/reports/studentgenerative-ai-survey-2025/ [6] GitHub. 2021. Introducing GitHub Copilot: your AI pair programmer. https://github.blog/news-insights/product-news/introducing-githubcopilot-ai-pair-programmer/. Accessed: 2026-10-25. [7] GitHub. 2022. GitHub Copilot is generally available to all developers. https://github.blog/news-insights/product-news/github-copilot-isgenerally-available-to-all-developers/. Accessed: 2026-10-25. [8] Niki Gitinabard, Ruth Okoilu, Yiqiao Xu, Sarah Heckman, Tiffany Barnes, and Collin Lynch. 2020. Student Teamwork on Programming Projects: What can GitHub logs show us?. In Proceedings of the 13th International Conference on Educational Data Mining (EDM 2020), Anna N. Rafferty, Jacob Whitehill, Violetta Cavalli-Sforza, and Cristobal Romero (Eds.). International Educational Data Mining Society, 409–416. [9] Sivana Hamer, Christian Quesada-López, Alexandra Martínez, and Marcelo Jenkins. 2021. Using git metrics to measure students’ and teams’ code contributions in software development projects. CLEI electronic journal 24, 2 (2021), 1–18. [10] Petri Ihantola and Andrew Petersen. 2019. Code complexity in introductory programming courses. In Proceedings of the 52nd Hawaii International Conference on System Sciences (HICSS-52). 7662–7670. https://hdl.handle.net/10125/60204 [11] Gregor Jošt, Viktor Taneski, and Sašo Karakatič. 2024. The impact of large language models on programming education and student learning outcomes. Applied Sciences 14, 10 (2024), 4115. [12] Hieke Keuning, Bastiaan Heeren, and Johan Jeuring. 2017. Code quality issues in student programs. In Proceedings of the 2017 ACM Conference on Innovation and

SIGCSE TS 2027, February 17–20, 2027, Sacramento, CA, USA

Technology in Computer Science Education. 110–115. [13] Hieke Keuning, Johan Jeuring, and Bastiaan Heeren. 2023. A systematic mapping study of code quality in education. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1. 5–11. [14] Wenhan Lyu, Yimeng Wang, Tingting Chung, Yifan Sun, and Yixuan Zhang. 2024. Evaluating the effectiveness of LLMs in introductory computer science education: A semester-long field study. In Proceedings of the eleventh ACM conference on learning@ scale. 63–74. [15] Jemimah Nathaniel, Solomon Sunday Oyelere, Jarkko Suhonen, and Matti Tedre. 2025. Literature Review on the Integration of Generative AI in Programming Education. International Journal of Artificial Intelligence in Education (2025), 1–32. [16] OpenAI. 2022. Introducing ChatGPT. https://openai.com/blog/chatgpt. Accessed: 2026-05-23. [17] Sida Peng, Eirini Kalliamvakou, Peter Cihon, and Mert Demirer. 2023. The impact of AI on developer productivity: Evidence from GitHub Copilot. arXiv preprint arXiv:2302.06590 (2023). [18] James Prather, Paul Denny, Juho Leinonen, Brett A Becker, Ibrahim Albluwi, Michelle Craig, Hieke Keuning, Natalie Kiesler, Tobias Kohn, Andrew LuxtonReilly, et al. 2023. The robots are here: Navigating the generative AI revolution in computing education. In Proceedings of the 2023 working group reports on innovation and technology in computer science education. 108–159. [19] Christian Rahe and Walid Maalej. 2025. How do programming students use generative AI? Proceedings of the ACM on Software Engineering 2, FSE (2025), 978–1000. [20] Judy Hanwen Shen and Alex Tamkin. 2026. How AI impacts skill formation. arXiv preprint arXiv:2601.20245 (2026). [21] Md Istiak Hossain Shihab, Christopher Hundhausen, Ahsun Tariq, Summit Haque, Yunhan Qiao, and Brian Wise Mulanda. 2025. The Effects of GitHub Copilot on Computing Students’ Programming Effectiveness, Efficiency, and Processes in Brownfield Coding Tasks. In Proceedings of the 2025 ACM Conference on International Computing Education Research V. 1. 407–420. [22] Miroslav Tushev, Grant Williams, and Anas Mahmoud. 2020. Using GitHub in large software engineering classes. An exploratory case study. Computer Science Education 30, 2 (2020), 155–186. [23] Fengcai Wen, Csaba Nagy, Gabriele Bavota, and Michele Lanza. 2019. A largescale empirical study on code-comment inconsistencies. In 2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC). IEEE, 53–64. [24] Yufan Zhang, Jaromir Savelka, Seth Goldstein, and Michael Conway. 2026. Changes in Coding Behavior and Performance Since the Introduction of LLMs. arXiv preprint arXiv:2601.11835 (2026).

Related documents

• Observing the Conduct of Systematic Reviews with Generative AI Support: An Experience Report from a Graduate Software Engineering Course#1061962
• VOTE: Group Editors Analyzing Tool#226745
• Beyond Human-in-the-Loop: Verification-by-Design for Generative AI-Assisted Research Data Analysis#1416439
• Counting and Sampling Git Graphs#1468812
• Ceci N'est Pas un Drone: Investigating the Impact of Design Representation on Design Decision Making When Using GenAI#1129134