arXiv:2604.10631v1 [cs.SE] 12 Apr 2026
Investigating CI/CD-based Technical Debt Management in Open-source Projects João Paulo Biazotto
Daniel Feitosa
Paris Avgeriou
Elisa Yumi Nakagawa
University of Groningen Groningen, The Netherlands University of São Paulo São Carlos - SP - Brazil [email protected]
University of Groningen Groningen, The Netherlands [email protected]
University of Groningen Groningen, The Netherlands [email protected]
University of São Paulo São Carlos-SP, Brazil [email protected]
Abstract—Managing technical debt (TD) is critical to ensure the sustainability of long-term software projects. However, the time and cost involved in technical debt management (TDM) often discourage practitioners from performing this activity consistently. Continuous Integration and Continuous Delivery (CI/CD) pipelines offer an opportunity to support TDM by embedding automated practices directly into the development workflow. Despite this potential, it remains unclear how TDM tools could be integrated into CI/CD pipelines, and we still lack established best practices for this process. To address this problem, the objective of this study is to understand how TDM tools have been used in CI/CD pipelines and also identify potential configuration anti-patterns. To this end, we conducted a largescale mining software repository (MSR) study on GitHub. In total, we collected around 600,000 Travis CI configuration files and 50,000 supporting scripts, and identified 3,684 pipelines that contain at least one TDM tool. We applied descriptive statistics to analyze the prevalence of tools and anti-patterns, and our findings show that most tools are executed and integrated using an external script; in addition, Absent Feedback is the most common configuration anti-pattern. We believe that researchers and practitioners can use the evidence of this study to further investigate how to improve both the tools that are integrated in CI/CD and the integration practices. Index Terms—Technical debt, Technical debt management automation, CI/CD, Travis CI.
I. I NTRODUCTION Over the years, several technical debt management (TDM) activities, such as identifying and measuring debt, have been developed to support developers and companies to keep technical debt (TD) under control [1], [2]. Although paramount for the long-term sustainability of software projects, such TDM activities are often effort-intensive and require substantial time that could potentially be used to develop new features [3]. In this context, tooling is essential to automate or semi-automate such activities, and several recent studies investigated available tools [4], practitioners’ concerns about those tools [5], and their usage [6]. Although such studies list many available tools, they also highlight that the adoption of tools is a complex matter for many reasons, such as usability and workflow friction [7], false positives [8], and integration overheads [9]. These barriers help explain why teams underuse tools even when they acknowledge their benefits [10].
To address some of those problems, the Manifesto on Reframing Technical Debt [11] explicitly calls for tools that are easily integrated into existing workflows. Specifically, the manifesto suggests to “develop workflow-based TDM tools”, emphasizing the need for solutions that integrate seamlessly into existing development processes. The same concern was reported by practitioners [5], who highlighted that TDM tools must be consistent with their workflows to reduce friction in tool adoption. One potential solution is to integrate TDM tools in Continuous Integration and Continuous Delivery (CI/CD) pipelines [12]. CI/CD aims to make the integration and delivery of changes safe, fast and routine [13] and is widely adopted in open-source and industrial projects [14]. In addition, a previous systematic review showed that adopting CI/CD pipelines that contain TDM tools (e.g., static analyzers, code smell detectors, architectural conformance checkers) can identify TD earlier and reduce manual effort, turning TDM from sporadic activity into a continuous practice [6]. Previous studies have reported that many existing TDM tools can be integrated into CI/CD pipelines [4]. However, we lack consolidated evidence on the extent that TDM tools are actually used in CI/CD pipelines. There is also little knowledge on how TDM tools could be integrated in such pipelines, and when the tools are executed within the pipelines (e.g., before or after deployment). To make matters worse, the pipelines themselves suffer from configuration problems, such as anti-patterns [14], [15], which degrade feedback quality, delay TD reports, and reduce developer trust. This suggests that simply adding a TDM tool to a CI/CD pipeline is insufficient; how the tool is wired and governed matters. In summary, the main problem we tackle in this study is the lack of largescale empirical knowledge on how and when TDM tools are actually integrated into CI/CD pipelines, and what antipatterns may occur and impede the pipelines. This can hinder the adoption of such tools and lead to less effective TDM using CI/CD pipelines. To address this problem and provide support to developers who aim at adopting TDM tools in CI/CD pipelines more systematically, we carried out a large-scale mining software repository (MSR) study on GitHub repositories. Specifically,
we analyzed around 600,000 CI/CD configuration files and 50,000 supporting scripts. We characterized the pipelines to understand how TDM tools are integrated, when this happens in the pipeline, and the prevalence of configuration antipatterns. Our study provides the following contributions: • We systematically identify how TDM tools are executed within pipelines (e.g., direct calls in the config vs. invocation through external scripts), revealing common integration patterns that could support developers planning the integration of tools. • We determine the specific pipeline stages, jobs, and phases in which TDM tools are typically run, providing a broader view of the tools within the CI/CD workflows1 . • We quantify how often key CI/CD anti-patterns (e.g., skipped failures) occur in configurations that involve TDM tools, offering actionable evidence for practitioners and tool vendors. The remainder of this paper is organized as follows: Section II discusses the related work and compares it to our results, highlighting our contributions. Section III describes the methods used in this study, including data source selection, data collection pipeline, and data analysis approaches. Section IV presents the results, while Section V discusses them and points out implications for researchers and practitioners. Section VI outlines the threats to the validity of this study and the actions taken to mitigate them. Finally, Section VII presents the conclusions of this study and discusses potential future research directions. II. BACKGROUND AND R ELATED W ORK A. Tooling for Technical Debt Management Tooling is central to contemporary engineering practices, including continuous software development, agile methods, and DevOps, and supports nearly every phase of the life cycle, from design to deployment and maintenance [16]. Such tools encompass a wide range of artifacts and services, such as integrated development environments (IDEs), project and issue trackers, static analyzers, and testing frameworks; collectively, these tools enable teams to create, analyze, and manage software products [16]. Despite the broad adoption of general-purpose tools in daily workflows, TDM tools remain comparatively underutilized and face notable limitations [10]. Systematic TD tracking is still uncommon, and although many TD tools rely on techniques like static analysis (similarly to other quality tools) [17], they often fall short of estimating the interest or likelihood associated with debt items, which hampers effective communication and prioritization [18]. Practitioners also report practical hurdles when adopting TDM tools, including high error or false positive rates [19], complex configuration [20], limited alignment with existing workflows (e.g., integration to IDE or CI/CD embedding, and insufficient explainability [20]. These limitations reduce both adoption and impact. Addressing them can enable [6], [20], 1 In this study, pipeline(s) and workflow(s) are used as synonyms.
[21]: (i) more contextually aware and usable tools that cover a wider set of TD types; (ii) the combination of multiple information sources; (iii) actionable guidance, particularly around impact and consequences; and (iv) seamless integration and higher levels of automation throughout the modern toolchains. Several studies have examined the usage of TDM tools in both industry and open-source projects [22]–[24]. Avgeriou et al. [18], for instance, assessed tools to measure code, design, and architecture-related debt, analyzing characteristics, popularity, and available empirical evidence (including evaluations in industrial settings). While that work catalogs and characterizes the tools, our focus is on concerns surrounding their usage, specially in open-source software (OSS) context), bringing the practitioner’s perspective. Other literature [4], [6], [21] shows that stakeholders often rely on isolated TDM tools. Junior et al. [21] highlighted the need for simpler integration of these tools into the prevailing development stack, moving toward a more holistic approach to TDM (i.e., supporting more activities with tools). Similarly, Biazotto et al. [6] argued for greater levels of automation to reduce the developer workload associated with TDM. While these contributions advance the theoretical understanding of tool usage, they offer limited practical guidance, an evidence gap that our study addresses. Our study tackles this gap by further investigating the usage of TDM tools in practice, helping both identify tools’ limitations and provide guidance for developing management strategies (e.g., by following common patterns to add TDM tools to CI/CD pipelines). B. CI/CD and Configuration Anti-patterns CI/CD aims to make integrating and releasing changes safe, fast, and routine, providing rapid feedback through automated build, test, and deployment stages [13]. In principle, such pipelines serve as quality gates that solve problems early and keep software in a releasable state. A CI/CD workflow (or pipeline) specifies an ordered and sequential set of stages that defines the process to build and deploy an application. Each stage has one or more jobs, which run in parallel. Such jobs have a set of phases (e.g., before install, script, after deploy), each one running a specific command2 . For example, consider Listing 1, which shows a .travis.yml file for a Python project. This file defines a list of stages (1) to build and deploy the project. In this example, three stages were defined: lint (2), test (3), and deploy (4). Each stage has at least one job (e.g., 2.1); the “test” stage (3) has two jobs (i.e., 3.1 and 3.2). During execution, stages (2), (3) and (4) are executed sequentially, while jobs (3.1) and (3.2) are executed in parallel within the same stage. Since a stage can have one or many jobs, it can be a dedicated stage (i.e., with a single job, as stage 2, which runs only Flake8) or mixed (i.e., many jobs, as stage (3)). The job itself can also be dedicated (i.e., executes only one action) or mixed (i.e., running multiple scripts). Although this structure provides significant flexibility for testing an 2 https://docs.travis-ci.com/user/job-lifecycle/
application, pipelines themselves are software artifacts that must be designed and maintained; when their configuration is degraded, the feedback becomes noisy, delayed, or misleading. Prior work has cataloged recurring configuration antipatterns (also called configuration smells or bad practices) that erode the potential benefits of CI/CD. Examples include overly long or monolithic builds, late integration/merging, silently ignored failures (e.g., allow failures), and broken or unstable main branches. Subsequent research proposed linters and analyzers to detect such problems. For instance, Vassallo et al. [15] introduced a reporting approach, CI-Odor, that automates the detection and longitudinal reporting of CI antipatterns and pipeline decay, helping teams monitor when and where configurations start to drift. Complementary studies further show that pipelines can produce misleading outcomes (for example, by masking failures), compromising developer trust, and ultimately the value of automation [14]. Listing 1: Example of a .travis.yml file for a Python project 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
language: python os: linux stages: [lint, test, deploy] (1) jobs: include: - stage: lint -- (2) -name: Lint -- (2.1) -install: pip install flake8 script: flake8 src tests - stage: test -- (3) -name: Unit tests -- (3.1) -python: "3.11" install: pip install -r r.txt script: pytest -q - stage: test -- (3) -name: integration tests -- (3.2) -python: "3.11" install: pip install -r req.txt script: pytest -q -r - stage: deploy (4) if: tag IS present script: skip deploy: -- (4.1) -provider: pypi username: "__token__" password: $PYPI_TOKEN
Despite this promising work, we still lack consolidated, large-scale evidence on how TDM tools are actually wired into real-world pipelines, which configuration choices and anti-patterns co-occur with their usage, and what actionable practices can help teams realize value consistently. The current study aims to address this gap. III. S TUDY D ESIGN This section presents the objective and research questions (RQs) of this study, the data collection process, and how we analyzed the data to answer the RQs. A. Objective and Research Questions Considering the problems and gaps we presented in the two previous sections, the objective of this study, structured according to the Goal-Question-Metric template [25], is to
“analyze CI/CD configuration files for the purpose of identifying and classifying TDM tools with respect to their integration and configuration from the point of view of software developers in the context of open-source projects in GitHub.” To achieve this objective, we defined the following RQs: • RQ1 - How are TDM tools integrated in CI/CD pipelines? In this RQ, we investigate which TDM tools are commonly integrated into CI/CD pipelines, and how this integration usually occurs (i.e., by directly calling the tool or in an external script). Investigating the type of execution helps to understand how easily TDM logic can be maintained within pipelines (e.g., if TDM tools are in external scripts, more artifacts should be updated to change TDM logic). We also investigate if tools are usually used individually or co-occur with other tools. The answer to RQ1 can help practitioners evaluate their own tools and pipelines and be aware of alternatives for configurating those pipelines. • RQ2 — When are TDM tools integrated into CI/CD pipelines? The stages, jobs, and phases at which a certain TDM tool is integrated into pipelines determine, among others, the visibility of the TDM feedback. For example, a phase indicates whether tools act as “gates” (i.e., the analysis happens before deployment and may block the build if there is any error) or as “reporting” (e.g., a linter that simply checks code quality, but does not affect the build process). In addition, the types of job/stage (e.g., dedicated or mixed) and their names help to organize TDM tools within the pipelines. Mapping tools across phases and job/stage structures can help to recommend strategies to integrate tools in TDM, increasing its value. • RQ3 - Which configuration anti-patterns are prevalent in pipelines with TDM tools? Configuration anti-patterns (e.g., skipped failures) can hinder the value of TDM tools even when those tools are present. Quantifying the prevalence of such issues in pipelines that include TDM tools identifies the primary obstacles to trustworthy feedback. Together with RQ1 and RQ2 , RQ3 completes the progression from “how” TDM is integrated to “when” integration occurs in the pipelines and “why” it may succeed or fail in real-world CI/CD settings. B. Data Collection and Analysis In this section, we describe the methods we used and decisions we made during the data collection and analysis. All scripts used for these processes are reported in a replication package3 . 1) Selecting Context and CI/CD Managers to Investigate We focus our investigation on GitHub, as it is one of the largest code repositories and has been extensively investigated 3 https://doi.org/10.5281/zenodo.17469262
in previous studies about TDM [26]–[28]. Those studies provided evidence that GitHub provides the necessary variety of projects and teams to investigate TDM. As for CI/CD managers, we needed to select a manager that can be integrated with GitHub. While we had several options (e.g., Circle CI, GitHub Actions, and Jenkins), we decided to focus on Travis CI. Travis CI is one of the most popular CI/CD managers in GitHub [29], [30]. Furthermore, since other CI services have a similar configuration syntax (YAML-based DSL), it is likely that our observations will be applicable to other CI services. In addition, previous work mentioned Travis CI as a good starting point for investigating tools in CI/CD pipelines [31]. Therefore, we argue that the focus on Travis CI and GitHub is sufficiently representative for a first exploratory study on TDM tools in CI/CD pipelines.
unique GitHub repositories. We scanned each repository for .travis.yml files within the project root directories, in the main branch (i.e., “master” or ”main”). In total, approximately 80,000 additional .travis.yml were recovered for analysis. Therefore, around 600,000 Travis CI configuration files were recovered. Considering that it is possible to use shell scripts7 to run TDM tools within Travis CI pipelines, we processed each .travis.yml file to extract the shell scripts (e.g., analysis.sh) that are executed in the CI/CD pipelines. The scripts must be stored within the repository, so we downloaded the scripts from GitHub. By doing this process, we collected around 50,000 scripts across the respositories that have a .travis.yml file. This expands the corpus for analysis and provides a further understanding on how the tools are executed.
2) Retrieving Tavis CI Configuration Files To answer the proposed RQs, we need to identify GitHub projects that use Travis CI. As reported in previous studies [14], [15], [31], the most efficient way to identify Travis CI is by searching for .travis.yml files within the projects’ root directories. To carry out this process, initially, we considered using the GitHub Search API4 , but that has significant limitations: first, it has a rate limit of 1,000 requests per hour, which deeply limits the feasibility to analyze a large amount of projects; second, as a HTTP-based API search, it imposes challenges for searching files with special characters (i.e., .travis.yml ). A second alternative is to use Google BigQuery5 , a serverless data warehouse on Google Cloud that lets one to run standard SQL queries over massive public datasets. Google BigQuery has a GitHub “files” dataset (e.g., bigquery-publicdata.github repos.files), which is a snapshot of the files of public GitHub repositories stored in BigQuery. Using Google BigQuery, we retrieved a list of 526,864 repositories that have a Travis CI configuration file. One potential issue with the set of retrieved repositories from BigQuery is that we could not be sure that the GitHub snapshot in Google BigQuery was up-to-date. To mitigate this and expand the set of repositories, we used GH Archive6 , a repository that registers GitHub events (e.g., PushEvent and IssueCommentEvent). GH Archive provides hourly-based json files that contain all events registered during that period. In our study, we collected all repositories that had events between January 1st, 2025 and August 31st, 2025. We made a decision to focus on active projects instead of focusing on historical data, which might include discontinued features and/or practices. We deemed that this decision enabled us to investigate only updated projects, which have more potential to provide the current overview of CI/CD pipelines. The data collected from GH Archive include 6,652 json files, comprising around 430 GB of data. The json files contain around 1.1 billion events involving around 22 million
3) Identifying TDM tools Identifying CI/CD pipelines that encompass TDM tools is crucial for our study. While we acknowledge that any quality gate can be used as an alternative to mitigate TD [32], there is a variety of possible patterns to configure and execute such quality gates, making it infeasible to identify every pattern for our study. Hence, we decided to carry our study based on a sample of TDM tools that can be integrated with CI/CD. To derive this sample of tools, we used a recent study on TDM [6], which reports on a set of 121 automation artifacts that can be used for TDM. From this set, we selected those tools which can be integrated with CI/CD pipelines as reported in [6]. For each of those tools, we identified the patterns for using/executing them in CI/CD pipelines. For example, SonarCloud can be executed within Travis CI by using the command “sonar-scanner”8 . Based on such patterns, we developed an analyzer that parses the Travis CI configuration files and searches for the patterns using regular expressions9 . During the analysis of the list of tools, we noted some linters, such as pylint and ESlint10 . A linter is a static analysis tool that automatically checks the source code for errors, style issues, and suspicious patterns against defined rules. Considering that the list of linters reported in [6] is limited, and aiming to increase the representativeness of our list of tools, we also identified patterns for running linters for the top-10 programming languages most used, according to the TIOBE raking11 . We are confident that our tool sample is representative for exploring TDM in CI/CD. 4) Identifying the Step that Executes the TDM tool The official documentation of Travis CI present an explanation of the jobs/stages of the CI/CD pipelines in Travis CI, as we described in Section II-B. Besides, previous studies [14], [31] discuss such phases, and explain the goal of each one. Inspired by both sources, we considered two characteristics of CI/CD steps to be investigated. First, the type of job/stage that 7 https://docs.travis-ci.com/user/deployment/script 8 https://docs.travis-ci.com/user/sonarcloud/
4 https://docs.github.com/en/rest?apiVersion=2022-11-28
9 https://docs.python.org/3/library/re.html
5 https://cloud.google.com/bigquery?hl=pt br
10 https://eslint.org/
6 https://www.gharchive.org
11 https://www.tiobe.com/tiobe-index/
runs a tool, i.e., dedicated stage, dedicated job within a stage (a job that runs only the TDM tool), and mixed job (a job that performs multiple actions). Second, regarding when the tool is executed, we consider if it is ran pre-deployment (i.e., as a quality gate) or after deployment (i.e., as a post-check to provide a quality report, but do not gate the build). 5) Selecting and Identifying Configuration Anti-patterns Anti-patterns can have direct implications for TDM tools in CI pipelines. For example, if TDM tools are wired into stages that run infrequently (or post-merge only), the feedback loop becomes too long and opportunities for early remediation are lost. In the context of our study, we used the list of nine anti-patterns reported by Vassallo et al. [15]. Specifically, we considered four of those anti-patterns that can directly impact TDM and can be identified from the Travis CI files. Table I presents: (a) the definition of each selected anti-pattern; and (b) a rationale for including it in our study, considering its potential impact on TDM. The selected anti-patterns can be identified considering only the .travis.yml file, which also aligns with out data collection strategy. For each anti-pattern, we defined a set of rules to identify them in the CI/CD pipelines: • Late Merging: stage/job is limited to type = push AND branch = main/master. • Skip-on-Failures: jobs.allow failures present • Absent Feedback: No notification section, or no tokens for notification tools, neither the command email on the notifications section. • Email-Only Notifications: notifications: email: ... 6) Analyzing Data To answer the RQs, we mainly relied on descriptive statistics. For RQ1 , we analyzed the share of adoption per tool, the type of execution (direct calls or script-based), as well as the co-integration of tools. To answer RQ2 , we analyzed the distribution of pre-deployment vs. post-deployment execution, the types of job (e.g., dedicated stage, dedicate jobs) as so as stage naming practices (i.e., the names of stages containing TDM tools). For RQ3 , we analyzed the prevalence of each antipattern, the co-occurrence of the anti-patterns, and the relation between tools and anti-patterns. We use frequency tables to carry out the analysis. IV. R ESULTS A. How TDM tools integrate in CI/CD pipelines To answer RQ1 , we analyze (a) which TDM tools have been integrated, (b) whether the CI/CD pipelines use direct or script-based invocations, and (c) whether multiple tools are integrated within the same pipeline. Regarding TDM tools, we identified a total of 38 tools integrated into CI/CD pipelines (Table II). Among them, Shellchek is the most prevalent, appearing in 727 repositories, followed by Flake8 (724), Cppcheck (332), Pylint (315), and Govet (292). Most of the identified tools are linters or static analyzers that primarily support the identification of TD, indicating
that TD detection is the dominant practice in CI/CD-based TDM. In contrast, fewer tools focus on measurement (e.g., SonarQube, Lattix) or prevention activities (e.g., Black, Clang_format). Concerning TD types, the vast majority of tools address code debt, followed by few tools targeting build debt. This distribution is expected, as source code is the primary artifact analyzed by automated CI/CD pipelines. Considering the type of invocation of each tool, 2,466/3,684 pipelines invoke tools using external scripts (66.9%), while 1,127/3,684 do so by calling the tool directly (30.6%), and 91/3,684 use both strategies (2.5%). Therefore, maintainers often “glue” TDM tools through shell/auxiliary scripts rather than declaring them inline in Travis CI. This approach decentralizes logic, i.e., practitioners have CI/CD logic in both the configuration file and the scripts. Therefore, using external scripts can hide TDM checks from the top-level pipeline view. This can be a problem when the TDM logic needs to be maintained: multiple execution points (e.g., multiple scripts) make it harder to change tool configuration and increase maintenance costs, which might decrease the perceived value of TDM. At the tool level, the highest volumes of direct calls (DC in Table II), are in Flake8 (310) and Cppcheck (129). As for script calls (SC in Table II), Shellcheck (672), Flake8 (422), and Govet (245) are the most prevalent. We note that 35/38 tools are invoked using both strategies, suggesting no practitioner preference. For instance, Tslint tends to be executed via direct calls because the integration between Travis CI and the tool is straightforward, whereas Sonarqube is mainly used in scripts, even though the integration of Sonarcloud with Travis CI is relatively simple. In summary, while highly context dependent, practitioners tend to use TDM tools mostly in script files. Finally, we checked how many tools are typically integrated within the same pipeline. Overall, 2,900/3,684 (78.7%) of repositories use exactly one TDM tool; 555/3,684 (15.1%) use two; 174/3,684 (4.7%) use three; and a 55/3,684 (1.5%) use four or more. Therefore, most pipelines adopt a single primary tool rather than a broad toolchain. To extend this analysis, we examined stacks (co-occurrence of tools within the same repository) to understand common combinations of tools. Figure 1 depicts the tools that co-occur in at least 10 pipelines. While a full list of co-occurrences can be found in our replication package12 , here we limit the analysis to pairs appearing in at least 10 pipelines to increase representativeness. From the 38 identified tools, 36 were found to co-occur in pipelines. In total, we identified 142 different combinations, with Flake8–Pylint (77) and Flake8–Shellcheck (67) as the most common. Across ecosystems, multiple combinations support Python and Java projects (e.g., Flake8–Pylint and Pmd–Checkstyle). In addition, some co-occurrences focus on checking project infrastructure (e.g., Shellcheck–Yamllint), which points to a 12 https://doi.org/10.5281/zenodo.17469262
TABLE I: CI/CD configuration anti-patterns (per Vassallo et al.) with selection and rationale for a TDM-in-CI/CD study Antipattern Late Merging
Definition (from Vassallo et al.)
Incl. Rationale for inclusion/exclusion in this study
Agile teams develop in feature branches; integration effort and conflict potential increase if completed features are not integrated in a timely manner.
Yes
Skip Failed Tests Absent Feedback
Yes
Broken Release
Skipping a previously failing test “fixes” the build symptom without addressing the cause, undermining the safety of the test suite. Developers miss required feedback if they are not automatically notified about relevant build events, especially failures. Email alone is an inappropriate single notification channel, as developers may lack access or notifications get lost among other messages. A broken build not fixed promptly prevents CI from assessing new changes.
Slow Build
Report increasing build times and outliers in build duration.
No
Aged Branches
Infrequently synced branches diverge and become hard to integrate; warn when an open branch hasn’t been merged for a release. Build artifacts/binaries should not be committed.
No
Email Notif.
Bloated Repo. Scheduled Builds
Scheduled builds either rebuild changes unnecessarily or indicate changes are not automatically built.
Yes Yes No
No No
Directly affects timeliness of CI feedback and makes TD signals stale, reducing flow and team velocity; it is aligned with the need for integrating TDM with CI/CD and actionable flow metrics [11]. Also matches Biazotto [5]’s “workflow consistency.” Hides debt and degrades trust in quality signals. Studying it supports the calls for reliable, visible data in CI/CD [11] and Biazotto’s concerns about tool reliability and the need for human control over automation outputs [5]. A core visibility gap: without feedback, TD information does not reach practitioners. Biazotto [5] shows many requirements are about communication (dashboards, IM, email) and notification configurability; Single-channel alerts exacerbate notification fatigue and are easy to miss. Biazotto reports preferences for configurable thresholds, summaries, and multiple channels [5]; Vassallo et al. [15] note teams are aware of this anti-pattern, thus reducing its risk to TDM. Besides, our study targets configuration choices that alter TD visibility and developer communication. Our study focuses on patterns that directly shape TD feedback quality and communication. Performance concerns are acknowledged, but less central to TDM signal visibility than the selected four [5]. Covered by Late Merging; Vassallo merges the two due to overlap [15]. Repository hygiene is important but less directly tied to TD communication/integration in CI. Also dropped by Vassallo after survey, showing low relevance [15]. Context-dependent and also dropped by Vassallo [15] due to low relevance. cpplint cpplint flake8 flake8
TABLE II: TDM tools in CI/CD pipelines Tool Tool type Shellcheck Linter Flake8 Linter Cppcheck Static analyzer Pylint Linter Govet Static analyzer Clang format Formatter Eslint Linter Phpcs Linter Black Formatter Sonarqube Static analyzer Checkstyle Linter Rubocop Linter Golangci lint Linter Clang tidy Linter/Analyzer Phpstan Static analyzer Cpplint Linter Pmd Static analyzer Sonarcloud Static analyzer Mypy Static analyzer Tslint Linter Coverity Static analyzer Swiftlint Linter Ruff Linter Spotbugs Static analyzer Yamllint Linter Phpmd Static analyzer Prettier Formatter Bandit Static analyzer Lattix Arch. analyzer Findbugs Static analyzer Staticcheck Static analyzer Stylelint Linter Psalm Static analyzer Hadolint Linter Detekt Linter/Analyzer Swiftformat Formatter Brakeman Static analyzer Ktlint Linter
TDM Activity Identification Identification Identification Identification Identification Prevention Identification Identification Prevention Ident./Measur. Identification Identification Identification Identification Identification Identification Identification Ident./Measur. Identification Identification Identification Identification Identification Identification Identification Identification Prevention Identification Measurement Identification Identification Identification Identification Identification Identification Prevention Identification Identification
Debt Type Build Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Build Code Code Security Arch. Code Code Code Code Build Code Code Code Code
DC SC Pip. 69 672 727 310 422 724 129 205 332 89 226 315 47 245 292 78 213 272 66 175 241 98 129 225 71 117 180 54 124 178 39 75 114 70 44 110 71 28 97 31 64 94 72 5 76 9 64 73 17 54 71 8 59 67 27 36 62 54 5 59 38 47 57 18 40 56 0 52 52 19 26 45 40 8 44 8 31 39 25 12 32 15 11 25 0 24 24 1 20 21 3 16 19 10 1 11 8 3 10 5 5 9 6 1 7 0 5 5 1 3 4 1 1 2
ruff ruff eslint eslint rubocop rubocop yamllint yamllint shellcheck shellcheck mypy mypy cppcheck cppcheck clang_tidy clang_tidy black black pylint pylint clang_format clang_format spotbugs spotbugs checkstyle checkstyle pmd pmd sonarcloud sonarcloud sonarqube sonarqube phpstan phpstan phpmd phpmd golangci_lint golangci_lint phpcs phpcs govet govet
pylint pylint phpcs phpcs yamllint yamllint black black rubocop rubocop shellcheck shellcheck clang_tidy clang_tidy spotbugs spotbugs govet govet cpplint cpplint mypy mypy phpstan phpstan phpmd phpmd eslint eslint golangci_lint golangci_lint ruff ruff clang_format clang_format flake8 flake8 cppcheck cppcheck pmd pmd checkstyle checkstyle sonarqube sonarqube sonarcloud sonarcloud
Fig. 1: Tools that co-occur in at least 20 different pipelines concern for keeping pipeline code consistent. Finally, we also noted cross-ecosystem co-occurrences, such as Flake8Cppcheck (for Python and C/C++) and Eslint–Pylint (for JavaScript to Python). While stacks are not prevalent in our dataset, when teams do stack tools, they usually combine tools that are complementary within a single ecosystem (e.g., linting and bug-finding for Python) and, sometimes, pair the tools with configuration/scripting checks (e.g., shell checkers). Answer to RQ1 : TDM in CI/CD is carried out predominantly by linters, and Flake8, Shellcheck, Cppcheck, Govet, and Pylint are the most integrated tools. Tools are mostly invoked through external scripts (67.7%), indicating a preference for a separation of concerns between CI/CD steps and quality checks. Typical stacks pair a linter with a tool that provides deeper code analysis (e.g., code metrics) or a formatter (e.g., Phpstan + Phpcs), but most projects use just one tool.
B. When TDM Tools integrate in CI/CD pipelines As we explained in Section II, a job is an isolated execution unit within a CI/CD pipeline (i.e., a self-contained set of steps), and jobs may be grouped into stages that run sequentially; a single pipeline may have multiple jobs and stages. To answer RQ2 , we analyze three aspects of jobs/stages: 1) To understand whether practitioners separate TDM checks from other processes in pipelines, we classify jobs/stages into three types: dedicated stage, dedicated job, and mixed job. This helps us to clarify how the TDM tools are usually spread over the CI/CD stages and jobs. 2) We also investigate whether TDM tools are run before the deployment, as a quality gate, or after the deployment, as a simple report. This helps us to understand the moment in which TDM processes are carried. 3) Finally, we investigate stage naming. As we discussed in Section I, the pipelines themselves are artifacts that must be maintained over time. Explicitly defining stage names for TDM could simplify the maintenance of such pipelines, and it is important to report current practices for naming. We split the data set according to the type of execution, that is, directly in .travis.yml (3,283 jobs) or through a external script (2,993 jobs), as we did in RQ1 . Since pipelines might have multiple jobs, the number of jobs (6,276 jobs) is higher than the number of pipelines (3,684 pipelines) First, we identified the types of jobs/stages: (i) dedicated stage, meaning the whole stage (i.e., group of jobs) focuses on configuring and running the TDM tool; (ii) dedicated job within stage, which means that a job within the stage focuses on running the TDM tool; and (iii) mixed job, where the TDM tool shares the main job phases with other tasks (e.g., tests / build). Figure 2 reports the distribution of types for script-based and direct executions. In direct calls, mixed job pipelines account for a little more than a half of the cases (1,963/3,283), while script-based setups raise this to about two-thirds (2,119/2,993). Direct calls show a larger portion of dedicated job within stage (870/3,283), whereas dedicated stage (450/3,283) is a minority. In script-based TDM, dedicated stages (836/2,993) are more common than in direct calls. Regarding the moment in which TDM tools are executed, pre-deployment executions are most prevalent in both direct calls (3,261/3,283) and script calls (2,907/2,993). This suggests that most TDM tools are used as quality gates. Optionally, some tools are also used for reporting after deployment. Such tools include Coverity and Flake8, indicating that tools providing reports that are not critical (e.g., test coverage) can be run later in the pipeline. The few post-deployment executions are mainly present in script-based setups. Since scripts are more flexible and enable combing runtime information (e.g., deployment duration) with other static checks, practitioners might prefer this type of execution for postchecks.
Type of Job mixed_job dedicated_job_within_stage dedicated_stage
3000
2500
2000
1500
1000
500
0
Direct
Script
Type of Execution
Fig. 2: Types of stage/job Regarding stage naming, most TDM executions live in an unnamed stage/job (5,057). This is largely a consequence of Travis CI’s defaults: when jobs are not explicitly grouped into stages, the pipeline runs in a single implicit stage, named test13 . In practice, the lack of stage names mean that many pipelines operate as a single, bulk CI stage. This explains why the vast majority of direct executions fall into unnamed stages, since practitioners are not forced to name stages, so the default behavior silently absorbs TDM into the implicit test stage, making configurations simpler to authors, but also easier to ignore and harder to review at a glance. When teams name stages explicitly, the labels usually indicate a quality-related concern (e.g., Lint/lint and Code Quality). Thus, when explicit, TDM tools appear to be assigned to stages more specific and appropriate, instead of more general (e.g., test) or mixed ones. Table III shows a list of stage names that appear in at least five pipelines. A complete list of names is available in our replication package. TABLE III: Number of jobs per stage name (at least 20 stages) Stage Name implicit lint test Lint macOS Build Code Quality build Coverity Build, Test & Check Code Quality Coding standard verify
Direct 2442 105 90 63 0 52 33 29 27 21 21
Scripts 2615 39 0 0 61 0 3 0 0 0 0
Total 5057 144 90 63 61 52 36 29 27 21 21
Answer to RQ2 Teams frequently place TDM tools to execute checks before the deployment. Moreover, most jobs are mixed, and TDM tools execute alongside with other tools or processes (e.g., pipeline configurations). In addition, most stages names are implicit; for explicit names, “lint” and “code quality” are the most common. 13 We decided to use “implicit” instead of “test” to avoid wrongly capturing named stages
C. CI/CD configuration anti-patterns in TDM Across 3,684 pipelines, Absent Feedback is the dominant anti-pattern (2,493 or 67.7%), while Skip-on-Failure (565 or 15.3%) and Late Merging (412 or 11.2%) are less frequent. Email-only notifications appear in 251 pipelines (6.8%). The distribution shows that most pipelines exhibit exactly one antipattern (2,482 or 65.7%), with fewer showing two (649 or 17.2%) and only a small fraction exhibiting all three (16 or 0.5%); 628 pipelines (16.6%) show none. We also identified some co-occurrences between the antipatterns, as presented in Table IV. Late Merging co-occurs with Absent Feedback in 226 cases (54.9% of Late Merging pipelines), and Skip-on-Failure co-occurs with Absent Feedback in 371 cases (65.7% of Skip-on-Failure pipelines). Late Merging and Skip-on-Failure rarely occur together (23 pipelines). Since the Absent Feedback is the most common anti-pattern, it was expected that it would co-occur with other anti-patterns more frequently. TABLE IV: Co-occurrences of anti-patterns
Absent Feedack Skip-on-Failure Late Merging Email-only Notifications
Absent Feedback
SkiponFailure
Late Merging
Emailonly
371 226 0
371 23 29
226 23 20
0 29 20 -
We also examine tools that are frequently related to antipatterns. For this, we use Tables V, VI, VII, and VIII that show the number of pipelines each tool is involved in, the number of pipelines with a certain anti-pattern for that tool, and the percentage of pipelines with that anti-pattern. For instance, in Table V, we show that Absent Feedback occurs in 94.9% of the pipelines with Tslint. TABLE V: Absent Feedback (AF) among projects using each tool (with AF ≥ 50). Tool
Pipelines
with AF
% AF
Tslint Clang format Golangci lint Clang tidy Black Sonarcloud Govet Phpstan Eslint Flake8 Sonarqube Checkstyle Phpcs Cppcheck Pylint Shellcheck Rubocop
59 272 97 94 180 67 292 76 241 724 178 114 225 332 315 727 110
56 239 85 81 151 53 230 59 179 517 124 79 141 199 182 407 56
94.9% 87.9% 87.6% 86.2% 83.9% 79.1% 78.8% 77.6% 74.3% 71.4% 69.7% 69.3% 62.7% 59.9% 57.8% 56.0% 50.9%
Absent Feedback is widespread among the pipelines, as Table V shows. The tools, such as Golangci_lint (87.9%) and TSLint (94.9%), are highly correlated with this pattern.
TABLE VI: Late Merging (LM) among projects using each tool (with LM ≥ 10). Tool
Pipelines
with LM
% LM
Black Pmd Checkstyle Sonarcloud Shellcheck Spotbugs Sonarqube Rubocop Eslint Clang format Govet Phpcs Flake8 Pylint
180 71 114 67 727 45 178 110 241 272 292 225 724 315
69 20 32 16 172 10 25 11 16 18 15 11 34 14
38.3% 28.2% 28.1% 23.9% 23.7% 22.2% 14.0% 10.0% 6.6% 6.6% 5.1% 4.9% 4.7% 4.4%
This might indicate that simpler checks, such as evaluating shell structures, do not impose strong concerns on developers. Therefore, notifications may be ignored, because immediate actions are not necessary. As listed in Table VI, Black (38.3%) and Pmd (28.2%) are likely to appear in pipelines with Late Merging. On the other hand, Pylint is much less frequent (4.4%). These higher rates among static code analyzers (e.g., Pmd) indicate that more complex analysis (e.g., identification of code smells with Checkstyle) are often performed only when the code is merged to the main branch, which might not be ideal and could potentially delay TDM. TABLE VII: Skip-on-Failure (SoF) among projects using each tool (with SoF ≥ 10). Tool
Pipelines
with SoF
% SoF
Coverity Phpstan Lattix Phpcs Clang tidy Cpplint Cppcheck Clang format Flake8 Shellcheck Pylint Govet Sonarqube Eslint
57 76 24 225 94 73 332 272 724 727 315 292 178 241
42 53 14 97 39 16 68 52 133 61 26 23 10 13
73.7% 69.7% 58.3% 43.1% 41.5% 21.9% 20.5% 19.1% 18.4% 8.4% 8.3% 7.9% 5.6% 5.4%
For Skip-on-Failure, practices are mixed, as Table VII shows. Tools such as PHPStan (69.7%) and Coverity (73.7%) are often configured to allow failures. Other tools, such as Eslint (5.4%) and Sonarqube (5.6%), usually block the execution in case of failures. This pattern indicates that tools focused on code style and formatting (e.g., Flake8) or test coverage (e.g., Coverity) are more frequently treated as non-blocking checks and do not enforce a pipeline to stop in case of violations. On the other hand, linters and static analyzers (e.g., Eslint), which provide more detailed reports on code quality (e.g., violations), are often configured to enforce the pipeline to stop in case of failures.
For Email-only Notifications, Table VIII shows that tools such as Cpplint (47.9%) and Spotbugs (31.1%) often rely on email as notification channel; Shellcheck (2.7%) and Cppcheck (3.0%) usually uses other channels, such as Slack. This pattern suggests that pipelines running tools with more detailed reports (e.g., the smells identified by Spotbugs) usually send results by email. TABLE VIII: Email-only Notifications (EoN) among projects using each tool (with EoN ≥ 10). Tool
Pipelines
with EoN
% EoN
Cpplint Spotbugs Pmd Pylint Phpcs Checkstyle Flake8 Govet Clang format Cppcheck Shellcheck
73 45 71 315 225 114 724 292 272 332 727
35 14 16 67 47 19 47 18 13 10 20
47.9% 31.1% 22.5% 21.3% 20.9% 16.7% 6.5% 6.2% 4.8% 3.0% 2.8%
Answer to RQ3 : Across 3,684 pipelines, Absent Feedback is the most common anti-pattern. Regarding the tools in pipelines with anti-patterns, linters and static code analyzers are less often related to Late Merging and Skip-on-Failure (i.e., such analysis runs earlier in the pipeline and enforces the build stop in case of failures). V. D ISCUSSION A. Interpretation of the results of RQ1 The results of RQ1 indicate that TD identification is the most prevalent activity in CI/CD pipelines, since most tools are linters/static analyzers (e.g., Flake8). These results are consistent with other recent literature [4], [5], [21], which states that identification is the most performed TDM activity. Although adding TD identification tools helps to improve awareness of TD, the potential for performing other TDM activities (e.g., TD repayment) is underexplored. Thus, we advise researchers to investigate how TD measurement (quality gates, trend tracking), prevention (auto-fixers, formatters), and repayment (e.g., refactoring suggestions) activities could be inserted into CI/CD pipelines. We also found that most pipelines run single TDM tools, while few TDM stacks, i.e., combinations of TDM tools, also appear. Considering the flexibility provided by CI/CD pipelines, it would be most beneficial if stacks focusing on multiple TDM activities were implemented. We thus recommend that researchers investigate defining, and testing TDM tool stacks. As for practitioners, a suggestion is to start integrating TDM into CI/CD with a static code analyzer and potentially combining other tools (such as a linter) for the same ecosystem (e.g., JavaScript). This would help to increase TD visibility without disrupting existing pipelines.
Regarding tool execution, most pipelines invoke tools via external scripts. This practice may help reusing commands across jobs and projects; it can also help scale TDM while reducing the effort to maintain TDM tools. In addition, external scripts might help keep the Travis CI YAML concise, promoting a more coherent separation of concerns. However, moving TDM logic to scripts reduces the visibility of quality, making reviews and quick comprehension harder, especially for newcomers. This trade-off may be more critical for OSS, because the number of (new) contributors is usually higher in OSS than in industry. Nevertheless, we advise all practitioners to pay attention to stage naming when using external scripts and to use mechanisms like log messages or notifications, to make the TD checks more visible. B. Interpretation of the results ot RQ2 The results from RQ2 show that most TDM tools run in predeployment phases, which is consistent with common CI/CD practice around quality gates. As discussed in Section III, any quality gate can support TDM [32], and our data corroborate this because most tools are actually executed before deployment (i.e., as a gate). As an implication, practitioners could increase the value of these gates by adopting TDM tools that further explain the bad effects of TD (e.g., interest or maintainability indexes). Practitioners could also combine pre-deployment checks, which make TD visible earlier, with simple post-deployment reports (e.g., test coverage), which provide additional information to support planning TDM. Finally, researchers can investigate whether using tools as gates vs. for reporting, impacts the perceived usefulness and value of TDM tools. Most TDM tools are present in mixed jobs, which once again indicates that TDM is treated as any other quality check within the pipeline. Furthermore, when TDM logic is pushed into external scripts, it might be mixed with other non qualityrelated tasks, such as environment preparation and caching. This somehow reduces the coherence of the TDM checks and might reduce the perception of its value. In direct calls, there is a relatively higher share of dedicated jobs within a stage. This could happen simply because, with seamless integration between the TDM tool and the pipeline (i.e., a single command), dedicated stages become simpler to maintain and make the configuration file more consistent. Based on these observations, we deem that practitioners should avoid mixing TDM with non-quality-related processes (e.g., pipeline configuration), ideally keeping TDM checks as a single stage in the pipeline. This would provide both more flexibility (e.g., multiple TDM checks can be combined) and more coherence, simplifying the maintenance of TDM tools and pipelines Finally, naming practices show that many tool executions occur in unnamed /implicit stages. In practice, this treats TDM tools as “just another step.” Similar to the problems identified in RQ1 , unnamed stages might hinder the understandability of YAML files and keep TDM hidden from maintainers. In contrast, when developers name stages, the labels are related to quality (e.g., Lint, Code Quality, Static Analysis). However,
we did not identify any specific stage name for TD (e.g., debt or tech debt). A good practice for practitioners would be to clarify the stage names and clearly define when TDM is being carried out. C. Interpretation of the results to RQ3 The prevalence of Absent Feedback (2,493/3,684) suggests that many teams successfully run TDM tools but fail to communicate their results. In practice, this reduces the effectiveness of the feedback and hinders awareness of TD. Hence, the support provided by static analysis and quality checks has not been exploited at its maximal potential. Researchers should investigate how to measure and improve the observability of TD-related failures in CI/CD pipelines. A potential direction is to define and track visibility metrics and evaluate their relationship with tool adoption and code review effectiveness (i.e., whether increased visibility leads to more effective remediation actions for TD). Regarding Late Merging (412/3,684), a core problem is that it shifts attention from prevention (that is, managing TD in pull requests) to detection after integration. The results also indicated that static analyzers (e.g., Pmd) are mostly executed in main branches only, while tools with less complex analysis (e.g., Pylint) tend to run in every branch. Overall, considering that Late Merging appears in less than 10% of all pipelines, we can infer that practitioners are already using TDM tools as gates (i.e., checking TD before merging the code). Nonetheless, the higher rates on static analyzers, such as Checkstyle, indicate that deeper TD analysis (e.g., smells detection) are being carried only when the code is merged to the main branch. Therefore, we caution practitioners that such practices might be risky and should be avoided. For researchers, they could investigate the combinations of preand post-merge checks. This would both raise the awareness of practitioners about the problems of Late Merging and define scenarios in which post-merge checks could be safely used. This understanding might lead to the definition of guidelines or even CI/CD templates focused on TDM; those templates could be easily replicated in different projects. Finally, Skip-on-Failure (565/3,684) hinders the “gatekeeping” aspect of TD tools in CI/CD pipelines. This can explain how pipelines can sometimes appear healthy while accumulating unmanaged debt. According to the results, linters appear to be less critical for practitioners and more prone to this antipattern. Specifically for TDM, we advise practitioners to avoid skipping failures, since this can hide the TD. Another suggestion for practitioners is to have separated jobs for higherrisk checks (e.g., architectural issues with Checkstyle) and avoid skipping failures on those. If necessary, allowing-failure policies must be rare, deliberate, and documented to ensure the reliability of the pipeline. VI. T HREATS TO VALIDITY Construct validity: Our identification of Travis CI configuration files relies on BigQuery and GitHub Archive, and
we may have missed projects (e.g., configuration files in nondefault branches) or included false positives (e.g., non-Travis CI YAMLs named as .travis.yml ). To mitigate this threat, we developed scripts that check whether the minimal configurations of Travis CI (e.g., a script section) are present and removed the files that did not have such configurations. The operationalization for identifying CI/CD anti-patterns (e.g., Late Merging, Skip-on-Failure) may over- or under-flag cases; we derived characteristics of each anti-pattern from published definitions (e.g., [15]) and defined the identification rules considering the structure of Travis CI files (as we describe in Section III). However, Travis’s configuration files are highly heterogeneous, and it is not possible to ensure that we were able to capture all possible types. As a mitigation action, the scripts we used to process the Travis CI files are based on the official documentation only14 . External validity: As our dataset focuses on GitHub repositories and Travis CI, we cannot generalize the results to other platforms (e.g., GitLab, Bitbucket) or CI/CD platforms (e.g., GitHub Actions, GitLab CI, CircleCI), although other CI/CD platforms are mostly similar to Travis CI (e.g., centered around the concept of jobs or configured using YAML files). Hence, replications are needed to increase external validity. Similarly, the set of tools we analyzed is limited. To mitigate this threat, we considered not only the set of tools identified in a previous study [6] but also the list of other linters (i.e., linters for the top-10 most used programming languages, according to the TIOBE raking15 ). While we acknowledge that the investigated tools are sufficiently representative for an initial exploratory study, replications are needed to improve the generalizability of our results. Reliability: To mitigate threats to the reliability of the study, we describe the data acquisition process in as much detail as possible. More importantly, the data set curated through our investigation and the scripts used for data collection and analysis are publicly available16 . VII. C ONCLUSION AND F UTURE W ORK The main finding of this study is that CI/CD-based TDM is mainly destined to detect TD, rather than communicate it or make it visible. Our results show that linters are the dominant tools, and nearly two-thirds of CI/CD pipelines invoke TDM tools via scripts. We see two research directions from our results: (i) cross-platform (e.g., using Gitlab or Github Actions) and longitudinal studies are essential to measure the benefits of TDM in CI/CD; and (ii) while integrating TDM tools using external or direct calls might be help the separation of concerns between pipelines and TDM check, it is still necessary to survey practitioners to further understand the rationale for such decisions. Both research directions can enable the definition of guidelines for adopting TD tools in continuous and agile environments. 14 https://docs.travis-ci.com/user/job-lifecycle/ 15 https://www.tiobe.com/tiobe-index/ 16 https://doi.org/10.5281/zenodo.17469262
R EFERENCES [1] Z. Li, P. Avgeriou, and P. Liang, “A systematic mapping study on technical debt and its management,” Journal of Systems and Software, vol. 101, pp. 193–220, Mar. 2015. [2] N. Rios, M. G. de Mendonça Neto, and R. O. Spı́nola, “A tertiary study on technical debt: Types, management strategies, research trends, and base information for practitioners,” Information and Software Technology, vol. 102, pp. 117–145, 2018. [3] T. Besker, A. Martini, and J. Bosch, “Technical debt cripples software developer productivity: A longitudinal study on developers’ daily software development work,” in International Conference on Technical Debt (TechDebt), 2018, p. 105–114. [4] J. D. S. Silva, J. G. Neto, U. Kulesza, G. Freitas, R. Rebouças, and R. Coelho, “Exploring technical debt tools: A systematic mapping study,” in International Conference on Enterprise Information Systems (ICEIS), 2022, pp. 280–303. [5] J. a. P. Biazotto, D. Feitosa, P. Avgeriou, and E. Y. Nakagawa, “Understanding practitioners’ reasoning and requirements for efficient tool support in technical debt management,” Empirical Software Engineering, vol. 30, no. 5, Jul. 2025. [6] ——, “Technical debt management automation: State of the art and future perspectives,” Information and Software Technology, p. 107375, Dec. 2023. [7] N. Rios, R. O. Spı́nola, M. Mendonça, and C. Seaman, “The practitioners’ point of view on the concept of technical debt and its causes and consequences: a design for a global family of industrial surveys and its first results from Brazil,” Empirical Software Engineering, vol. 25, no. 5, pp. 3216–3287, Jun. 2020. [8] D. Sas and P. Avgeriou, “An architectural technical debt index based on machine learning and architectural smells,” IEEE Transactions on Software Engineering, vol. 49, no. 8, pp. 4169–4195, 2023. [9] L. Ochoa, T. Degueule, and J.-R. Falleri, “Breakbot: analyzing the impact of breaking changes to assist library evolution,” in Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging Results, ser. ICSE ’22. ACM, May 2022. [10] A. Martini, T. Besker, and J. Bosch, “Technical debt tracking: Current state of practice: A survey and multiple case study in 15 large organizations,” Science of Computer Programming, vol. 163, pp. 42–61, 2018. [11] P. Avgeriou, I. Ozkaya, H. Koziolek, Z. Codabux, and N. Ernst, “Manifesto from dagstuhl perspectives workshop 24452 – reframing technical debt,” 2025. [12] L. d. O. Carvalho, J. P. Biazotto, D. Feitosa, and E. Y. Nakagawa, “Technical debt in continuous software engineering: An overview of the state of the art and future trends,” in Anais do XXVII Congresso Ibero-Americano em Engenharia de Software (CIbSE 2024), ser. CIbSE 2024. Sociedade Brasileira de Computação, May 2024, p. 313–326. [Online]. Available: http://dx.doi.org/10.5753/cibse.2024.28456 [13] B. Fitzgerald and K.-J. Stol, “Continuous software engineering: A roadmap and agenda,” Journal of Systems and Software, vol. 123, pp. 176–189, 2017. [14] K. Gallaba and S. McIntosh, “Use and misuse of continuous integration features: An empirical study of projects that (mis)use travis ci,” IEEE Transactions on Software Engineering, vol. 46, no. 1, p. 33–50, Jan. 2020. [Online]. Available: http://dx.doi.org/10.1109/TSE.2018.2838131 [15] C. Vassallo, S. Proksch, H. C. Gall, and M. Di Penta, “Automated reporting of anti-patterns and decay in continuous integration,” in 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, May 2019. [Online]. Available: http: //dx.doi.org/10.1109/ICSE.2019.00028 [16] T. Theunissen, S. Hoppenbrouwers, and S. Overbeek, “In continuous software development, tools are the message for documentation,” in Proceedings of the 23rd International Conference on Enterprise Information Systems. SCITEPRESS - Science and Technology Publications, 2021. [17] N. Rios, R. O. Spı́nola, M. Mendonça, and C. Seaman, “The most common causes and effects of technical debt: First results from a global family of industrial surveys,” in 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), 2018, pp. 1–10. [18] P. Avgeriou, D. Taibi, A. Ampatzoglou, F. A. Fontana, T. Besker, A. Chatzigeorgiou, V. Lenarduzzi, A. Martini, A. Moschou, I. Pigazzini, N. Saarimaki, D. D. Sas, S. S. de Toledo, and A. A. Tsintzira, “An overview and comparison of technical debt measurement tools,” IEEE Software, vol. 38, no. 3, pp. 61–71, May 2021.
[19] S. Malakuti and J. Heuschkel, “The need for holistic technical debt management across the value stream: Lessons learnt and open challenges,” in 2021 IEEE/ACM International Conference on Technical Debt (TechDebt), 2021, pp. 109–113. [20] J. a. Biazotto, D. Feitosa, P. Avgeriou, and E. Nakagawa, “Automating technical debt management: Insights from practitioner discussions in stack exchange,” in 2025 IEEE/ACM International Conference on Technical Debt (TechDebt), 2025. [21] H. Jeronimo Junior and G. H. Travassos, “Consolidating a common perspective on technical debt and its management through a tertiary study,” Information and Software Technology, vol. 149, p. 106964, 2022. [22] I. Khomyakov, Z. Makhmutov, R. Mirgalimova, and A. Sillitti, “Automated measurement of technical debt: A systematic literature review,” in 21st International Conference on Enterprise Information Systems (ICEIS), 2019, pp. 95–106. [23] A. Tornhill and M. Borg, “Code red: the business impact of code quality - a quantitative study of 39 proprietary production codebases,” in Proceedings of the International Conference on Technical Debt, ser. TechDebt ’22. New York, NY, USA: Association for Computing Machinery, 2022, p. 11–20. [24] R. Verdecchia, K. Maggi, L. Scommegna, and E. Vicario, Technical Debt in Microservices: A Mixed-Method Case Study. Springer Nature Switzerland, 2024, p. 217–236. [25] R. van Solingen, V. Basili, G. Caldiera, and H. D. Rombach, “Goal question metric (GQM) approach,” Jan. 2002. [26] Y. Kashiwa, R. Nishikawa, Y. Kamei, M. Kondo, E. Shihab, R. Sato, and N. Ubayashi, “An empirical study on self-admitted technical debt in modern code review,” Information and Software Technology, vol. 146, p. 106855, 2022. [27] J. Tan, D. Feitosa, and P. Avgeriou, “The lifecycle of technical debt that manifests in both source code and issue trackers,” Information and Software Technology, vol. 159, p. 107216, 2023. [28] A. Tommasel and J. A. Diaz-Pace, “Identifying emerging smells in software designs based on predicting package dependencies,” Engineering Applications of Artificial Intelligence, vol. 115, p. 105209, 2022. [29] D. E. Rzig, F. Hassan, C. Bansal, and N. Nagappan, “Characterizing the usage of ci tools in ml projects,” in Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement, ser. ESEM ’22. New York, NY, USA: Association for Computing Machinery, 2022, p. 69–79. [Online]. Available: https://doi.org/10.1145/3544902.3546237 [30] G. Freitas, J. a. H. Bernardo, G. Sizı́Lio, D. A. D. Costa, and U. Kulesza, “Analyzing the impact of ci sub-practices on continuous code quality in open-source projects: An empirical study,” in Proceedings of the XXXVII Brazilian Symposium on Software Engineering, ser. SBES ’23. New York, NY, USA: Association for Computing Machinery, 2023, p. 1–10. [Online]. Available: https://doi.org/10.1145/3613372.3613403 [31] L. Chomkatek, J. Papuga, P. Nowak, and A. Poniszewska-Marańda, “Decoding ci/cd practices in open-source projects with llm insights,” in Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering, ser. FSE Companion ’25. New York, NY, USA: Association for Computing Machinery, 2025, p. 1638–1644. [Online]. Available: https://doi.org/10.1145/3696630. 3728699 [32] E.-M. Arvanitou, A. Ampatzoglou, S. Bibi, A. Chatzigeorgiou, and I. Stamelos, “Monitoring technical debt in an industrial setting,” in Proceedings of the 23rd International Conference on Evaluation and Assessment in Software Engineering, ser. EASE ’19. New York, NY, USA: Association for Computing Machinery, 2019, p. 123–132. [Online]. Available: https://doi.org/10.1145/3319008.3319019