arXiv CSopen access

How Developers Adopt, Use, and Evolve CI/CD Caching: An Empirical Study on GitHub Actions

Noname manuscript No. (will be inserted by the editor)

How Developers Adopt, Use, and Evolve CI/CD Caching: An Empirical Study on GitHub Actions

arXiv:2604.13129v1 [cs.SE] 13 Apr 2026

Kazi Amit Hasan · Yuan Tian · Safwat Hassan · Steven H. H. Ding

Received: date / Accepted: date

Abstract Continuous Integration/Continuous Delivery (CI/CD) caching is widely used to reduce repeated computation and improve CI/CD efficiency, yet maintaining effective caching requires ongoing maintenance effort. In this paper, we present the first empirical study on how developers configure and evolve caching in CI/CD workflows on GitHub Actions. We analyze 952 GitHub repositories (266 cache adopters and 686 non-adopters), to compare repository characteristics, characterize caching usage at the job and step levels, uncover patterns in caching configuration evolution, and identify the drivers of cacherelated changes. Our analysis spans 1,556 workflow files, 10,373 commits, and 17,185 workflow configuration changes, including an average of 9.37 cacherelated changes per repository. Our main observations are: (1) cache-adopting repositories are more active and popular than non-adopters; (2) caching is used across multiple CI/CD job types through a variety of caching mechanisms rather than a single standardized approach; (3) caching configurations evolve through frequent, repetitive maintenance patterns, with rapid updates in build and test jobs and slower evolution in other job types; and (4) cacherelated modifications are driven by distinct maintenance needs: parameter updates are mainly human-driven to fix issues, while version updates occur later and are often bot-driven for dependency maintenance. Our findings quantify the substantial maintenance effort involved in CI/CD caching and highlight opportunities to improve reliability and tool support. Kazi Amit Hasan, Yuan Tian School of Computing, Queen’s University, ON, Canada E-mail: {kaziamit.hasan, y.tian}@queensu.ca Safwat Hassan Faculty of Information, University of Toronto, ON, Canada E-mail: [email protected] Steven H. H. Ding School of Information Studies, McGill University, Canada E-mail: [email protected]

2

Kazi Amit Hasan et al.

Keywords Caching · GitHub Actions · Continuous Integration (CI) · Continuous Delivery (CD) · Collaborative Software Development

1 Introduction Modern software development relies on Continuous Integration and Continuous Deployment (CI/CD) pipelines to automate integration, testing, and deployment activities, enabling teams to sustain fast iteration and gradually improve product quality (Shahin et al., 2017; Rahman, 2023; Yang, 2025). A variety of platforms support CI/CD, including Jenkins, Travis CI, CircleCI, and GitHub Actions (GHA) (Ghaleb et al., 2026a). As projects grow, CI/CD workflows scale in complexity, increasing in the number of jobs (sets of steps executed on dedicated runners), the volume of dependencies, and execution frequency. This growth makes repeated work, such as reinstalling dependencies and rebuilding intermediate outputs, a common source of slowdown.1 Even small inefficiencies accumulate into substantial delays and computational overhead when runs are frequent, extending feedback cycles and increasing computational costs (Gallaba et al., 2020; Bouzenia and Pradel, 2024). To address this, CI/CD platforms, such as GHA provide configurable caching mechanisms that allow developers to store and reuse artifacts (Gallaba et al., 2020; Ghaleb et al., 2026b). For many projects, caching is not just a performance optimization but a necessary mechanism to maintain fast development cycles as CI/CD workflows grow in size, complexity, and frequency (Gallaba, 2019). However, CI/CD caching is not a simple toggle. Implementing an effective caching strategy is often complex because developers must determine when caching should be applied, what artifacts should be stored, and how cached artifacts should be identified and reused across workflow runs. These decisions are critical for ensuring that cached artifacts are reused when appropriate while avoiding the reuse of outdated artifacts. For example, an improperly designed key can cause repeated cache misses2 and provide little performance benefit3 . As a result, what is intended as a performance boost4 often becomes a non-trivial maintenance burden that adds to the hidden costs of CI/CD (Valenzuela-Toledo et al., 2024). This burden is evident in real workflow maintenance activities, such as adjusting cache keys and paths, adding or removing caches when they prove ineffective, and updating cache-related configurations to remain compatible over time. Despite this practical complexity, caching has received limited attention in prior literature work as a configuration and maintenance concern in CI/CD workflows. In this work, we aim to better understand CI/CD caching, particularly the maintenance of caching configurations, by focusing on GHA, given its popularity 1

https://runs-on.com/github-actions/caching-dependencies/ https://www.tencentcloud.com/techpedia/130942 3 https://aws.amazon.com/caching/best-practices/ 4 https://blog.jetbrains.com/teamcity/2025/12/is-your-ci-cd-tool-helping-or-hinderingperformance/ 2

How Developers Adopt, Use, and Evolve CI/CD Caching

3

and rich caching mechanisms. Prior studies have examined GHA workflows and their evolution (Chen et al., 2021; Decan et al., 2022), but provide limited fine-grained analysis of caching itself. They treat caching as a simple CI/CD optimization, focusing primarily on whether it is enabled and reporting it as one of the most commonly used strategies. As a result, caching is often viewed as a one-time configuration choice rather than an ongoing maintenance concern, leaving its configuration and maintenance dynamics poorly understood. This gap is practically significant, as practitioners may underestimate the effort required to keep caching effective as projects evolve. To fill this gap, we present the first large-scale, cache-centric empirical study of GHA workflows. We analyze caching configurations and their evolution in 266 cache-adopting repositories, encompassing 1,556 workflow files, 10,373 commits, and 17,185 workflow configuration changes (2494 of them are caching-related), drawn from a broader sample of 952 GitHub repositories that use GHA. Our study addresses three research questions: RQ1: What are the characteristics of cache-adopting repositories? We find that cache-adopting repositories are more active and popular than non-adopters. Caching is used across diverse CI/CD job types, with build and test jobs being the most prominent. At the step level, developers employ multiple caching mechanisms, where explicit caching via actions/cache (70.9%) dominates, followed by package manager (24.5%) and Docker layer caching (0.2%). RQ2: How do caching configurations evolve over time across different CI/CD job types? We find that caching evolution exhibits both shared patterns and job-specific differences. Across job types, caching configurations evolve through frequent, iterative updates and corrective actions, such as removing and immediately re-adding caches. Build and test jobs exhibit rapid, repeated parameter updates, whereas job types such as release exhibit slower modifications to caching configurations, with longer gaps between changes. RQ3: Why do developers make cache-related modifications in GitHub Actions workflows? By linking observed quantitative caching modification patterns with qualitative evidence from commit messages and pull requests, we develop a taxonomy of the drivers behind these changes. We find that parameter updates are primarily human-driven and aimed at fixing caching issues, whereas cache version updates are often triggered by bot alerts and repository dependency maintenance activities. In general, developers perform most of the debugging, tuning, and restructuring that shape how caching evolves in practice. Our contributions: – We present the first large-scale empirical study of CI/CD caching in GHA, characterizing its adoption across repositories and languages, the evolution of caching configurations over time, and the drivers behind these changes.

4

Kazi Amit Hasan et al.

– We broaden prior work, which primarily focuses on explicit caching via actions/cache, by capturing implicit caching mechanisms such as package manager and Docker layer caching, providing a more comprehensive view of how caching is implemented in GHA workflows. – We quantify how caching configurations evolve over time by representing cache-related maintenance activities as state transitions and analyzing their evolution patterns across different job types. – We explain why cache-related modifications occur by linking observed transitions to commit messages and PR context and constructing a taxonomy of drivers grounded in that evidence. – To foster future research in the area, we have made our replication package publicly available on GitHub.5 We organize the remainder of the paper as follows. Section 3 introduces our dataset. Sections 4 to 6 present the methodology and answers to each research question. Section 7 discusses implications for practitioners and researchers. Section 8 presents threats to validity, and Section 9 presents related work. Finally, Section 10 concludes the paper. 2 Background 2.1 Caching in GitHub Actions GitHub Actions defines CI and CD processes through workflow files written in YAML and stored under the .github/workflows directory of a repository. A workflow can contain one or more jobs. Each job runs on a runner that provides the execution environment, and each job consists of an ordered sequence of steps that perform tasks such as checking out code, setting up languages, installing dependencies, building, and testing.6 Caching is a performance optimization mechanism that reuses files across workflow runs to reduce repeated work, such as reinstalling dependencies and rebuilding intermediate artifacts. In GHA, caching is typically configured as a dedicated step within a job using the available caching strategies. Cache configuration requires two user-specified parameters, path and key. The path identifies the file(s), directory(ies) to be cached, while the key serves as a unique identifier for locating and restoring cached artifacts across runs. Figure 1 shows an example workflow that illustrates how jobs and steps are structured and how caching appears as steps inside a job7 . The workflow defines a single job named test that runs on ubuntu-latest and uses a Java version from a small matrix. The job first checks out the repository, then sets up the JDK using actions/setup-java. It then includes two cache steps using actions/cache.One cache step stores Gradle cache files in the 5

https://github.com/RISElabQueens/caching-in-github-actions https://docs.github.com/en/actions/get-started/understand-github-actions 7 https://github.com/apache/incubator-tuweni/blob/8bb12d442f6ac43e1ea616e0cc2 e5d1fcb1e73ee/.github/workflows/master-pr-build.yml 6

How Developers Adopt, Use, and Evolve CI/CD Caching

5

Fig. 1: An example of GitHub Actions workflow illustrating jobs, steps, and cache steps. ~/.gradle/caches directory, while the other stores the local Maven repository in the ~/.m2 directory. Both cache steps define cache keys that include the runner operating system and a hash of dependency-related files. This setup illustrates how a job can include multiple caches and how keys can be tied to dependency definitions so that caches remain stable when dependencies do not change and are refreshed when dependency files are modified. 2.2 Cache-related maintenance activities and terminology To support our empirical analysis, we define terminology related to caching in GHA workflows. At the repository level, we distinguish between cache-adopting and non-adopters repositories. A repository is considered cache-adopting if it enables caching at any point during its lifecycle. Conversely, a repository is considered a non-adopter (for caching) if none of its workflows contain any cache configuration throughout the observed history. This repository-level classification is used in RQ1 to compare adopters and non-adopters, while RQ2 and RQ3 analyze cache-related changes only within repositories that adopt caching. Next, we define cache-related maintenance activities to capture how developers introduce, update, and manage caching in GHA workflows (Table 1). These activities serve as a common vocabulary across the paper and provide

6

Kazi Amit Hasan et al.

a consistent basis for repository characterization (RQ1), evolution analysis (RQ2), and driver analysis (RQ3).

Table 1: Taxonomy of cache-related maintenance activities. Category

Abbr. Sub-category Definition

Example (Diff )

Cache EC enablement

Introduction of caching - (no cache step) for the first time in a + uses: actions/cache@v3 workflow, corresponding with: to the initial addition of path: ~ /.cache/pip a cache step within a job.

Cache version update

Any modification to the cache action version specification.

C up

Update

Refers to modifications within the caching version configuration.

Upgrade

Represents incremental - uses: actions/cache@v2 + uses: actions/cache@v3 cache version updates, where developers move to a newer version of cache.

Downgrade

Occurs when developers - uses: actions/cache@v3 revert to an older version + uses: actions/cache@v2 within caching configurations.

- uses: actions/cache@v3 + uses: actions/cache@wgewetwetwe

Adding a new cache

C add

Introduction of an - (cache already exists) additional cache step + (adding additional cache) alongside existing caches, + uses: actions/cache@v3 typically to support other with: jobs or workflow stages. path: node modules

Cache removal

C rm

Removal or disabling of a - uses: actions/cache@v3 previously defined cache - with: step when it is no longer path: node modules beneficial. + (cache step removed)

Parameter update

P up

Modification of existing cache parameter values. - key: $ runner.os -deps-$ hashFiles(’**/yarn.lock’) + key: $ runner.os -deps-$ hashFiles (’**/package-lock.json’)

Update

Refers to textual modifications within the caching configuration parameters.

Upgrade

Refers incremental - with: { node-version: 16 } language version updates. + with: { node-version: 18 }

Downgrade

Refers to parameter specific version downgrades.

- with: { python-version: ’3.12’ } + with: { python-version: ’3.11’ }

Parameter addition

P add

Introduction of a new - (no parameters) cache parameter that was + restore-keys: $ runner.os not previously specified. -deps-

Parameter removal

P rm

Removal of an existing cache parameter that is no longer required.

- restore-keys: $ runner.os -deps+ (restore-keys removed)

How Developers Adopt, Use, and Evolve CI/CD Caching

7

Fig. 2: Overview of our study.

3 Data Collection and Preparation In this section, we describe how we collected and prepared data for our empirical study. Figure 2 presents an overview of the study design and illustrates how each step of the process and the resulting data artifacts contribute to each research question. Step 1: Extract repository metadata: Our study builds on the dataset curated by Bouzenia et al. (Bouzenia and Pradel, 2024), which contains 952 GitHub repositories that use GitHub Actions (GHA) as their CI/CD automation platform. For each repository, we collect repository-level metadata through the GitHub GraphQL API.8 The collected metadata includes the numbers of stars, forks, contributors, commits, closed pull requests, and issues, together with the repository’s primary programming language. This step produces the repository metadata artifact shown in Figure 2, which is later used in RQ1 to compare cache adopters and non-adopters. Step 2: Identify cache adopting repositories: Our analysis focuses on caching configurations in GHA workflows. We label a repository as 8

https://docs.github.com/en/graphql

8

Kazi Amit Hasan et al.

Table 2: Cache-related actions considered in our study and their default configurations. Caching Strategy

Ecosystem

Actions

Default Caching

Explicit caching

General

actions/cache

No

Package manager caching

Go Python Node.js Ruby Java

actions/setup-go@v4 actions/setup-python actions/setup-node ruby/setup-ruby actions/setup-java

Yes1 No No No No

Docker layer caching

Docker

docker/build-push-action

No

1 Caching is enabled by default only for actions/setup-go@v4, earlier versions do not enable

caching by default.

cache-adopting if caching is enabled in any job at any point in its workflow history. We refer to the first such event as cache enablement (EC), following the taxonomy in Table 1. We capture caching through three cases9 . First, we capture explicit caching, where a job includes an actions/cache step and developers specify its inputs (e.g., path and key). Second, we capture package manager caching, where caching is enabled through language/tool setup actions (e.g., setup-go, setup-node, setup-python) using their caching settings. Third, we capture docker layer caching, where caching is configured through Docker build tooling (e.g., docker/build-push-action) to reuse previously built layers. As caching behavior differs across setup actions and versions, we follow official documentation10 and treat caching as present only when it is actually enabled in the workflow configuration. In particular, actions/setup-go@v4 enables caching by default, while earlier versions of setup-go do not. Therefore, for Go, we treat caching as enabled by default only when the workflow uses actions/setup-go@v4 and anything after this. For other setup actions in our scope, caching is not enabled by default in the versions we observe, so we require an explicit caching configuration in the action inputs (e.g., enabling the caching option). Table 2 summarizes the setup actions considered and whether caching is enabled by default in the action version we observe in workflow files. Applying this identification procedure to the 952 repositories yields 266 cache-adopted repositories (27.9% of the dataset) that enabled caching at any point during their lifecycle, as recorded in their commit history, and 686 non-adopters.We retain both groups for RQ1. The 266 cache adopters then proceed to the subsequent preprocessing steps, because only these repositories contain caching configurations that can be examined longitudinally.

9 10

https://www.ubicloud.com/blog/github-actions-transparent-cache https://github.com/actions/cache

How Developers Adopt, Use, and Evolve CI/CD Caching

9

Step 3: Extract workflows and their change histories. For the 266 cache adopters, we reconstruct workflow histories from Git history using Gigawork11 . Gigawork extracts historical versions of workflow files together with commit-level metadata, including commit hash, timestamp, and file path. Our reconstructed workflow history includes commits up to May 5, 2025. This step yields 1,556 workflow files, 10,373 commits, and 17,185 configuration changes, as shown in Figure 2. These reconstructed histories allow us to trace how caching-related configurations evolve over time. Although the source dataset also contains workflow-run logs, our analyses focus on repository metadata and workflow configuration histories, as these are the data required to answer our research questions. Step 4: Map jobs to CI/CD job types. To support cross-repository analysis, we normalize developer-defined job names into eight CI/CD job types, adapting the taxonomy of Bouzenia and Pradel (2024). The resulting categories are build, test, integration, lint, release, linux, analyze, and sync. This normalization is necessary because GHA job names are not standardized across repositories, and direct aggregation of raw job names would not support meaningful cross-project comparison. Step 5: Detect cache-related maintenance activities. Finally, we analyze workflow histories and retain only changes that modify cachingrelated configurations. Through this process, we identify 2,494 cache-related changes, representing 14.53% of all workflow changes. Each such changes are mapped to one of the seven cache-related maintenance activity categories defined in Table 1: EC, C up, C add, C rm, P up, P add, and P rm. Workflow changes unrelated to caching are excluded at this stage. This step produces the activity-level dataset used to model cache evolution and to support the qualitative analysis of cache-related modifications. Preparing data for RQ1: RQ1 examines the characteristics of cache adopters and profiles how caching is used within adopting repositories. As shown in Figure 2, RQ1 combines four artifacts: (1) repository metadata for all 952 repositories, (2) workflow configurations of the 266 cache adopters, (3) steplevel cache configurations and strategies extracted from those adopters, and (4) the mapping of jobs to the eight CI/CD job types. Repository metadata is used to compare adopters and non-adopters, while workflow and step-level configurations are used to characterize caching prevalence and caching strategy usage at both the job and step levels. Preparing data for RQ2: RQ2 investigates how caching configurations evolve through different maintenance activities over time. For this analysis, we use the step-level cache configurations of the 266 cache adopters, the eight CI/CD job types, and the seven cache-related maintenance activity categories. After filtering workflow histories to retain only cache-related changes, we reconstruct 11

https://github.com/cardoeng/gigawork

10

Kazi Amit Hasan et al.

per-job sequences of cache maintenance activities over time and group them by the CI/CD job types. Preparing data for RQ3: RQ3 explains why cache-related modifications occur. This analysis is built on the cache-related commits identified in Step 5. We group them by their target activity type (e.g., C up, P up, C add, P add, C rm, P rm) and select a statistically significant random sample from each group (sample sizes reported in the RQ3 section). For each sampled commit, we collect the workflow-file diff, the commit message, and the linked pull request when available. We also record whether the change was introduced by a human or a bot based on author identity and automation indicators. These artifacts provide the qualitative evidence used to interpret the reasons behind cache-related modification activities. 4 RQ1: What are the characteristics of cache-adopting repositories? 4.1 Motivation This research question aims to understand how caching is configured in GHA workflows and how cache-adopting repositories differ from non-adopters. Answering this question provides insight into when caching becomes necessary in practice and the common configuration patterns that developers adopt. 4.2 Approach To characterize repositories that adopt GHA caching, we combine a repositorylevel comparison with a workflow-level analysis. Specifically, at the repository level, we compare cache-adopting and non-adopters using metadata (e.g., contributors, commits, stars) collected via the GitHub GraphQL API (ref. Section 3). For cache-adopting repositories, we further analyze their CI/CD workflows using the latest repository snapshots to capture their current configurations. We quantify workflow complexity by summarizing the number of workflow files and jobs per repository. To characterize caching practices, we conduct an analysis of caching configurations at two levels of granularity: Job-level: We categorize the jobs specified in GHA workflow files into highlevel types based on their developer-defined names using the keyword-based taxonomy proposed by Bouzenia et al. (Bouzenia and Pradel, 2024). For each job, we determine whether caching is used by checking for the presence of at least one cache-related configuration step. We then compute caching prevalence for each job type as the percentage of jobs that contain caching. Step-level: We extract cache-related configuration steps from workflow files and categorize them based on the caching mechanism used (e.g., explicit caching, package manager caching, and docker layer caching). We further analyze their distribution across job types.

How Developers Adopt, Use, and Evolve CI/CD Caching

11

Fig. 3: Distributions of repository characteristics for cache-adopting (with) and non-adopting (without) repositories. Values are plotted on a log scale, with medians annotated in the original scale.

4.3 Results The following observations summarize patterns identified from our analysis of the 266 cache-adopted repositories (1,556 YAML files) and 686 non-adopting repositories in our dataset. Observation 1.1: Cache-adopting repositories tend to be more active, collaborative, and popular than non-adopting repositories. Figure 3 shows the repository-level characteristics of cache-adopting and non-adopting repositories. Compared with non-adopters, cache-adopting repositories have more contributors, commits, closed PRs, stars, forks, and issues. They also exhibit greater community interest, as indicated by higher median numbers of stars and forks. These results suggest that repositories with higher activity and community engagement are more likely to adopt caching to help manage CI workloads.

12

Kazi Amit Hasan et al.

Table 3: Adoption rates of caching across primary programming languages. Language

Cache-adopting

Non-adopters

Adoption Rate

PHP Java Kotlin TypeScript Go JavaScript C++ Python C Shell Ruby Others

26 36 12 45 22 37 17 42 9 4 6 10

22 35 21 79 54 109 52 146 38 17 28 79

54.2% 50.7% 36.4% 36.3% 28.9% 25.3% 24.6% 22.3% 19.1% 19.0% 17.6% 11.2%

Observation 1.2: Caching adoption differs across primary languages. Table 3 compares the primary programming languages of cache-adopting repositories and non-adopters in our dataset. To account for differences in language prevalence in our sample, we report the within-language adoption rate, which captures the proportion of repositories in each language category that adopt caching. PHP and Java show the highest adoption rates (54.2% and 50.7%), while Python (22.3%), Ruby (17.6%) show comparatively lower adoption. A chi-square test of independence shows that caching adoption is significantly associated with programming language (p < 0.001). This result indicates that adoption rates of caching differ significantly across languages rather than being uniformly distributed. Observation 1.3: Cache-adopting repositories typically maintain a moderate number of workflow files and jobs, with substantial long-tail complexity. Table 4 presents the per-repository workflow structure of cacheadopting repositories. The median number of workflow files per repository is 3, indicating that most adopters maintain multiple workflow files rather than a single configuration. While the minimum is 1 file, the maximum reaches 75 files, and the standard deviation (8.02) exceeds the mean (5.85), suggesting a right-skewed distribution with a long tail of repositories exhibiting substantially more complex workflow setups. A similar pattern is observed in jobs. The median number of jobs per repository is 4, with an average of 6.57 jobs. Although many repositories operate with a relatively small number of jobs, the maximum reaches 116, indicating the presence of a small subset of repositories with highly complex CI/CD configurations. Observation 1.4: Caching is most prevalent in build, integration, lint, and test jobs. Table 5 reports the highest job-level caching prevalence in build (37.31%) and integration (36.58%). Test (25.96%) and lint (27.84%) also show notable prevalence. By contrast, analyze remains lower (15.87%), and

How Developers Adopt, Use, and Evolve CI/CD Caching

13

Table 4: Distribution of workflow files and jobs per repository in cache-adopting projects. Metric # of files per repository # of jobs per repository

Min

Mean

Std

25%

50%

75%

Max

1 1

5.85 6.57

8.02 11.14

2 2

3 4

6 7

75 116

Table 5: Prevalence of caching across different CI/CD job types. # of jobs with caching

Total jobs

Caching prevalence (%)

build integration lint test release linux analyze sync other

247 15 22 141 76 6 20 3 51

662 41 79 543 332 29 126 59 457

37.31% 36.58% 27.84% 25.96% 22.89% 20.69% 15.87% 5.08% 11.16%

Total

581

2,328

24.95%

CI/CD job types

sync is rare (5.08%). These phase differences indicate that caching is primarily deployed in phases where repeated dependency installation and artifact reuse is common (e.g., build and test), whereas jobs focused on synchronization or maintenance are less likely to benefit from caching. Observation 1.5: Repositories adopt caching mostly through three different caching strategies. We derived a taxonomy of cache types used in GHA workflows: - Explicit caching: Caching is configured through an explicit cache step, using actions/cache where developers provide cache paths and keys. We define this “explicit” because caching is enabled only when developers declare a cache step and specify its inputs, which offers flexibility to cache arbitrary directories (e.g., dependency folders, build outputs, tool caches), but requires the most manual configuration. - Package manager caching: Caching is enabled through language/tool setup actions (e.g., setup-go, setup-node, setup-python) that either expose a caching option or provide caching behavior as part of the setup interface. We separate this from explicit cache actions because the developer’s configuration is mediated by the setup action, and the cached content is typically tied to the dependency management conventions of that ecosystem. - Docker layer caching: Caching is configured through Docker build tooling (e.g., docker/build-push-action) to reuse previously built layers. This is distinct from the two above mechanisms because the cached units are Docker

14

Kazi Amit Hasan et al.

Table 6: Distribution of caching strategies across CI/CD job types. Counts represent step-level caching configurations extracted from workflow files; a single job may contribute multiple times due to multiple caching steps. CI/CD job types

Explicit Caching

build test release other analyze lint integration linux sync Total

Package Manager Caching

Docker Layer Caching

Others

Total

282 (73.1%) 178 (75.4%) 41 (41.0%) 75 (80.6%) 23 (74.2%) 19 (70.4%) 10 (66.7%) 7 (100.0%) 2 (66.7%)

71 (18.4%) 53 (22.5%) 59 (59.0%) 17 (18.3%) 6 (19.4%) 8 (29.6%) 5 (33.3%) 0 (0.0%) 1 (33.3%)

1 (0.3%) 0 (0.0%) 0 (0.0%) 1 (1.1%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%)

32 (8.3%) 5 (2.1%) 0 (0.0%) 0 (0.0%) 2 (6.5%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%)

386 236 100 93 31 27 15 7 3

637 (70.9%)

220 (24.5%)

2 (0.2%)

39 (4.3%)

898

image layers, and the configuration uses the Docker caching interface rather than file-path caching. - Others: Any caching configuration that does not fall into the above categories. Observation 1.6: Explicit caching strategies dominate at the step level. Table 6 shows that explicit caching via actions/cache accounts for the majority of cache-related steps (70.9%). Package-manager caching via setup actions is the second most common strategy (24.5%). A plausible explanation for the dominance of explicit caching is its flexibility and configurability. Developers can explicitly specify cache paths and keys, which enables caching beyond dependencies (e.g., build outputs and tool caches).12 Moreover, actions/cache is extensively documented and commonly referenced in tutorials, blog posts, and Q&A forums such as Stack Overflow13 , lowering the barrier to adoption through readily reusable examples (Baltes and Diehl, 2019). Its long-standing and stable interface may further make it an attractive choice from a maintainability and backward-compatibility perspective, whereas caching support in setup-* actions is mediated by ecosystem-specific conventions and may change over time.14 Finally, explicit caching is well aligned with iterative CI/CD optimization, as its finer-grained and more flexible configuration allows developers to iteratively refine cache keys and paths in response to cache misses and performance bottlenecks.15 12

https://dev.to/github/caching-dependencies-to-speed-up-workflows-in-github-actions-

3efl 13 https://stackoverflow.com/questions/74401969/what-is-the-logic-in-using-the-restorekeys-field-in-the-github-cache-action 14 https://github.com/actions/setup-node 15 https://docs.github.com/en/actions/reference/workflows-and-actions/dependencycaching

How Developers Adopt, Use, and Evolve CI/CD Caching

15

RQ1 Summary: Repositories that adopt caching are more active than nonadopters. PHP and Java exhibit the highest adoption rates of CI/CD caching. Cache-adopting repositories maintain a moderate number of workflow files and jobs. At the job level, caching is most prevalent in build, integration, lint, and test jobs. At the step level, developers implement caching mainly through explicit caching strategies, followed by package manager caching and Docker layer caching.

5 RQ2: How do caching configurations evolve over time across different CI/CD job types? 5.1 Motivation While RQ1 represents the current, static characteristics of cache-adopting repositories, it does not explain how these configurations evolve over time. In practice, caching is rarely static or a one-time setup. As projects evolve and dependencies change, developers must update cache keys, paths, and strategies to maintain effectiveness and reduce CI/CD time. This research question examines these dynamics by analyzing the evolution of caching configurations across different job types. Understanding these evolution patterns is important because they reveal which cache maintenance activities are most common, how frequently configuration updates occur, and whether these dynamics differ across job types (e.g., build vs. test). These insights benefit practitioners by setting realistic expectations about the maintenance efforts required to keep caching effective and stable, and benefit tool and platform designers by highlighting where improved defaults, diagnostics, or guidance could reduce repeated trial-and-error in cache maintenance.

5.2 Approach Our evolution analysis is based on 266 cache-adopted repositories identified in Section 3. Across these repositories, we identify 2,494 cache-related maintenance activities. For each job type, we reconstruct its chronological cache maintenance history from workflow histories, allowing us to trace how caching configurations evolve over time.We represent each maintenance activity as one of the seven cache-related maintenance activity categories defined in Section 2. We model these histories using a first-order Markov model (Gagniuc, 2017). Following the job taxonomy defined in in Section 3, we group jobs by CI/CD job type and construct a separate transition model for each job type. In each model, nodes represent cache-related maintenance activity categories, and edges represent observed transitions between consecutive activities. For each transition, we compute its transition probability and corresponding time-totransition statistics (in days), which are reported in the appendix, and visualize the resulting transition graphs (e.g., Figure 4). To focus on recurring transitions,

16

Kazi Amit Hasan et al.

we prune edges with transition probabilities below 0.05, thereby emphasizing stable and representative transition patterns, following Huizi et al. 2024.

5.3 Results

Fig. 4: Cache-related maintenance activities in build jobs after enabling caching. Edges report transition probability and time-to-transition in days.

In the following paragraphs, we describe how caching configurations evolve over time through cache maintenance activities across CI/CD job types. We begin by describing the overall cache evolution patterns visible in the transition plots, and then report the dominant transition behaviors and timing differences across job types. In Figure 4 to Figure 11, node colors distinguish the functional role of each cache-related maintenance activity. Green denotes cache introduction and extension (Enable cache, Add cache, Add parameter ), yellow denotes maintenance (Cache version update, Parameter update), and red denotes removal (Remove cache, Remove parameter ). Throughout, we

How Developers Adopt, Use, and Evolve CI/CD Caching

17

Fig. 5: Cache-related maintenance activities in test jobs after enabling caching. Edges report transition probability and time-to-transition in days.

refer to the transition probabilities and the median time between consecutive activities. The complete evolution tables for all job types are provided in Appendix A. Observation 2.1: Cache evolution across CI/CD job types follows an iterative pattern of enabling, updating, adding, and occasionally removing cache configurations. Across all job-type-specific transition graphs (Figure 4- 11), cache evolution consistently begins with Enable cache, which serves as the entry point of cache adoption. After this initial step, the evolution is dominated by maintenance-related states, particularly Cache version update and Parameter update, which appear across all job types. In addition to these maintenance activities, cache evolution also includes expansion and adjustment operations. All job types involve Add cache, indicating that caching configurations are incrementally extended after initial adoption. Furthermore, several job types (build, test, integration, release etc.) include removal-related states (e.g., removing caches or parameters), suggesting that developers may

18

Kazi Amit Hasan et al.

Fig. 6: Cache-related maintenance activities in integration jobs after enabling caching. Edges report transition probability and time-to-transition in days.

roll back previous caching decisions. This suggests that cache configuration is an iterative process involving repeated updates and adjustments, rather than a single-pass setup. Observation 2.2: The structural complexity of cache evolution graphs differs across CI/CD job types. To characterize the complexity of cache evolution, we examine the structure of the transition graphs for each job type, including both the number of distinct cache-related maintenance activity states (nodes) and the transitions between them (edges). The transition graphs show clear variation across job types. Among all job types, Test and Lint jobs involve the largest number of distinct nodes after enabling cache. In contrast, Linux jobs show the smallest evolution space with only four nodes (Enable cache, Add cache, Cache version update, Parameter update). Build and Integration jobs each include five different nodes, indicating a broader range of cache-related activities. This indicates that cache evolution differs across job types, with some job types involving a broader set of cache maintenance activities than others.

How Developers Adopt, Use, and Evolve CI/CD Caching

19

Fig. 7: Cache-related maintenance activities in release jobs after enabling caching. Edges report transition probability and time-to-transition in days.

Observation 2.3: Cache maintenance in several CI/CD job types is dominated by repeated self-loop transitions. While Observation 2.1 shows that cache evolution generally involves enabling, updating, adding, and sometimes removing cache configurations, the transition tables further show that the most prevalent maintenance pattern in several job types is repeated self-loop behavior. In particular, across build, test, integration, and release jobs, the dominant transitions are often P up → P up and C up → C up, indicating repeated parameter tuning and repeated cache-version maintenance within the same state. For example, in build jobs, C up → C up accounts for 77.63% of outgoing transitions from C up, with a median time of 0 days (Table 11, Figure 4). This indicates that cache maintenance is rarely a static implementation. Once a developer begins tuning parameters and updating cache versions, they tend to continue doing so in a sequence of related commits. This suggests that maintaining a cache is an iterative process requiring sustained attention rather than a single action.

20

Kazi Amit Hasan et al.

Fig. 8: Cache-related maintenance activities in lint jobs after enabling caching. Edges report transition probability and time-to-transition in days.

Observation 2.4: Parameter updates occur earlier than cache version updates, but the delay differs across job types. Shortly after enabling cache (EC), developers engage in parameter tuning (EC → P up) relatively quickly, with a median of 105.93 days for Build jobs (Table 11, Fig. 4). However, the maintenance activity of updating the cache version (EC → C up) occurs after a significantly longer period. It has a median of 270.37 days in Release jobs (Table 14, Fig. 7) and 384.02 days in Analyze jobs (Table 16, Fig 9). This indicates that once a stable cache version is adopted, developers are reluctant to change it. While they may tweak how the cache is saved through parameters, they often let the core cache version remain unchanged for a much longer period.

How Developers Adopt, Use, and Evolve CI/CD Caching

21

Fig. 9: Cache-related maintenance activities in analyze jobs after enabling caching. Edges report transition probability and time-to-transition in days.

Observation 2.5: Cache removal is often part of an immediate replacement. In build jobs, the transition from removing a cache to immediately adding a new one (C rm → C add) occurs with 21.84% probability and a median of 0.01 days (Table 11, Fig. 4). Similarly, in Test jobs, this occurs with 30.19% probability and a median of 0.00 days (Table 12, Fig. 5). The near-zero median time indicates that developers are not permanently removing caches; they are replacing an old configuration with a new one. This suggests that the initial cache setup was insufficient, prompting a switch to a different strategy rather than abandoning caching altogether. Observation 2.6: In build and test jobs, cache evolution is characterized by frequent and rapid repeated parameter tuning followed by later expansion through additional caching. The P up → P up activity is dominant in both job types. In build jobs, it occurs with 65.53% probability and a median of 4.08 days (Table 11, Figure 4). In test jobs, it occurs with 66.05% probability and a median of 2.93 days (Table 12, Figure 5). In the same job types, cache expansion through EC → C add occurs later, with median

22

Kazi Amit Hasan et al.

Fig. 10: Cache-related maintenance activities in linux jobs after enabling caching. Edges report transition probability and time-to-transition in days.

times of 53.14 days in build jobs and 41.97 days in test jobs. This suggests a two-stage evolution pattern in build and test jobs. Developers first repeatedly refine cache-related parameters soon after adoption, likely to stabilize the initial setup, and only later extend the caching strategy by adding new caches. Observation 2.7: Integration and release jobs exhibit long intervals between cache maintenance activities. In contrast to build and test jobs, the maintenance activity of updating parameters (P up → P up) happens much less often. The median time between these updates is 87.25 days for Integration jobs (Table 13, Fig. 6) and 61.95 days for Release jobs (Table 14, Fig, 7). Furthermore, the time from enabling cache to the first parameter update (EC → P up) in Integration jobs has a median of 203.39 days. This indicates that cache configurations in these job types are occasionally modified. Unlike Build or Test jobs where changes happen frequently, developers do not often update the cache settings for Integration and Release jobs once they are established.

How Developers Adopt, Use, and Evolve CI/CD Caching

23

Fig. 11: Cache-related maintenance activities in sync jobs after enabling caching. Edges report transition probability and time-to-transition in days.

RQ2 Summary: Cache evolution is an iterative and structurally complex process. Across all job types, cache evolution consistently involves maintaining existing configurations through parameter updates and cache version updates, often accompanied by incremental additions and, in some cases, removal of cache-related configurations. Cache configurations are repeatedly refined rather than modified in a single pass. The structural complexity of this evolution varies across job types: Test and Build jobs involve richer transition structures and more frequent maintenance, whereas Integration and Release jobs exhibit simpler structures and substantially longer intervals between maintenance activities.

24

Kazi Amit Hasan et al.

6 RQ3: Why do developers make cache-related modifications in GitHub Actions workflows? 6.1 Motivation RQ2 examines how caching configurations evolve over time across different CI/CD job types by modeling sequences of cache-related maintenance activities as transition graphs. We observed rapid, repeated parameter updates in build and test jobs, whereas maintenance activities in other job types (e.g., release) occur less frequently and over longer intervals. However, these temporal patterns alone do not explain why developers make these modifications. For example, a rapid sequence of parameter updates can reflect deliberate performance refinement, but it can also indicate repeated attempts to resolve cache-related issues. Similarly, the long delay before a cache version update can reflect continued reliance on an existing configuration, or it can indicate that dependency maintenance is postponed until a warning or automated alert triggers action. To move from describing what changes and when they occur to explaining why they occur, RQ3 investigates the underlying drivers of cache-related modifications in GHA workflows, using commit/PR context and PR author information (human vs. bot) to interpret the intent behind the patterns observed in RQ2.

6.2 Approach We answer RQ3 by linking the quantitative cache-related maintenance transition patterns observed in RQ2 to qualitative explanations derived from the developer context. We proceed in two steps. First, we summarize transitions by their target state to identify which types of cache-related modifications are dominant and therefore require explanation (Table 7). This table provides a quantitative basis for RQ3 and is discussed in the Section 6.3 to justify our qualitative focus on particular targets. Second, we perform a qualitative analysis on a statistically significant random sample of transitions for each target state, using a 90% confidence level and a 10% margin of error (AlMulla et al., 2025). Specifically, we manually analyzed 86 (cache version update (C up)), 87 (parameter update (P up)), 77 (adding cache (C add)), 28 (adding parameter (P add)), 79 (removing cache (C rm)), and 32 (removing parameter (P rm)) sampled transitions. For each sampled transition, we examined the workflow file change that introduced the cache-related modification, including the workflow YAML before and after the edit. To infer the rationale behind the change, we examined the associated commit message and, when available, the linked pull request title and description. We identified the initiator of each change from the commit or pull request author and classified it as human or bot using author identity

How Developers Adopt, Use, and Evolve CI/CD Caching

25

Table 7: Statistics of cache-related maintenance transitions, aggregated by target state. For each target cache maintenance activity, we report transition counts from different preceding activities together with the corresponding transition time statistics (in days). From Cache version update Enable cache Parameter update Adding cache Removing cache Adding parameter Removing parameter Parameter update Enable cache Cache version update Adding cache Removing cache Adding parameter Removing parameter Adding cache Removing cache Enable cache Parameter update Cache version update Removing parameter Adding parameter Adding cache Parameter update Cache version update Enable cache Adding parameter Removing cache Removing parameter Removing cache Enable cache Adding cache Parameter update Cache version update Removing parameter Adding parameter Removing cache Parameter update Adding cache Cache version update Adding parameter Enable cache Removing parameter

To

Count

Cache version update

378 168 104 44 18 12 5

Parameter update

497 166 92 66 23 8 2

Adding cache

129 80 72 39 18 14 10

Adding parameter

11 9 6 5 4 3 1

Removing cache

117 111 76 69 45 20 1

Removing parameter

26 10 3 3 2 2 2

Total

Avg

Med

Std

Min

Max

%

745

70.60 543.33 172.52 298.89 210.66 179.04 0.00

0.00 384.13 104.80 231.83 95.11 0.00 0.00

149.79 447.09 241.06 310.83 334.78 392.71 0.00

0.00 0.01 0.00 0.00 0.00 0.00 0.00

713.59 1615.93 1366.00 1294.85 1374.41 1366.00 0.00

51.85 23.05 14.27 6.04 2.47 1.65 0.69

854

75.36 207.31 95.24 106.08 59.31 100.00 117.53

4.29 115.90 0.23 18.14 14.83 3.23 117.53

148.32 260.34 136.98 176.34 132.34 220.22 166.21

0.00 0.00 0.00 0.00 0.00 0.00 0.00

986.95 981.44 592.69 973.79 584.88 628.32 235.06

58.20 19.44 10.77 7.73 2.69 0.94 0.23

362

19.54 49.43 158.53 70.13 54.97 0.00 0.00

0.00 0.00 46.39 10.86 16.83 0.00 0.00

76.00 121.76 212.73 153.92 96.33 0.00 0.00

0.00 0.00 0.00 0.00 0.00 0.00 0.00

463.22 538.67 962.46 903.97 380.01 0.00 0.00

35.64 22.10 19.89 10.77 4.97 3.87 2.76

47

0.04 96.68 2.20 1.01 0.00 57.74 94.21

0.00 4.92 0.00 0.00 0.00 0.00 94.21

0.13 253.28 5.38 2.27 0.00 100.02 0.00

0.00 0.00 0.00 0.00 0.00 0.00 94.21

0.42 769.43 13.18 5.07 0.00 173.23 94.21

28.21 23.08 15.38 12.82 10.26 7.69 2.56

439

21.59 293.62 110.33 168.77 201.85 0.00 875.94

0.00 129.68 4.25 43.29 125.65 0.00 875.94

79.25 362.73 167.29 270.95 205.24 0.00 0.00

0.00 0.00 0.00 0.00 0.00 0.00 875.94

448.01 1514.81 750.21 1154.81 707.67 0.00 875.94

26.65 25.28 17.31 15.72 10.25 4.56 0.23

47

7.36 16.26 1.09 57.31 43.47 432.13 0.00

0.00 7.07 0.00 0.00 43.47 432.13 0.00

37.52 24.39 1.89 99.26 61.48 433.11 0.00

0.00 0.00 0.00 0.00 0.00 125.88 0.00

191.32 70.85 3.27 171.93 86.94 738.39 0.00

54.17 20.83 6.25 6.25 4.17 4.17 4.17

and automation indicators such as Dependabot16 and Renovate17 . Finally, we summarize the driver categories and their frequencies.

6.3 Results We present the findings of RQ3 in a structured manner. We first use Table 7 to identify the most frequently observed target states in the cache-related transition that require explanation. We then interpret these target states using the qualitative driver taxonomy and initiator information in Tables 8, 9, and 10. Table 9 links each target state to its specific modification action and associated 16 17

https://github.com/dependabot https://github.com/renovatebot/renovate

26

Kazi Amit Hasan et al.

Table 8: Taxonomy of drivers for cache-related modifications. Driver

Source

Definition (D) & Example (E)

Freq.

Update dependencies

Human

D: Developers update, upgrade, or downgrade dependencies of caching configurations. E: ’Update Go dependencies’

91

Improve cache

Human

D: Changes to improve caching performance. E: ’improved github workflows’

47

Bot alert

Bots

D: Dependency updates triggered by bot alerts. E: ’Bump actions/cache from 4.0.0 to 4.0.2’

40

Extend cache

Human

D: Adding additional caching configuration to the yaml file (e.g., expand support). E: ’use multiple jobs’

28

Fix caching issues

Human

D: Developers address/fix issues related to caching. E: ’Fix cmake.yml cache issues’

23

Migration

Human

D: Migrate to a different workflow (e.g., pnpm). E: ’migrate to pnpm’

14

Clean up

Human

D: Cleanup, reconfigure, or refactor yaml files. E: ’Simplify pipeline’

14

Remove tests

Human

D: Removal of specific jobs/tests after a period of time. E: ’Remove legacy tests and style guide’

13

Fix security issues

Human/Bot

D: Address security concerns by updating configurations. E: ’ci: Harden GitHub Actions’

6

Disable cache

Human

D: Developers explicitly disable caching. E: ’Removed caching of compiled contracts’

4

Fix deprecated ver.

Human

D: Fixing nearly deprecated version or addressing warnings. E: ’Change GitHub Action version to address deprecation’

2

Other

-

Reason not stated or unclear in commit/PR.

107

drivers, while Table 10 shows how driver categories are distributed across different cache modification targets. The Other category (N=107) in Table 8 is excluded from the discussion because it concerns cases in which the commit/PR text does not provide a clear rationale for cache-related modifications. Observation 3.1: Overall, most cache-related modifications are driven by dependency updates, improving cache, and bot alerts. Table 8 shows

How Developers Adopt, Use, and Evolve CI/CD Caching

27

Table 9: Summary of caching change targets, extended actions, and underlying reasons. Target

Drivers (Count)

Total

Update

Fix security issues (1)

1

Upgrade

Update dependencies (41) Bot alert (40) Fix deprecated versions (2) Other (1)

84

Downgrade

Update dependencies (1)

1

Update

Fix caching issues (9) Update dependencies (7) Improve cache (1) Fix security issues (1) Migration (1) Other (22)

41

Upgrade

Update dependencies (35) Fix caching issues (5) Fix security issues (1) Clean up (1) Other (1)

43

Downgrade

Fix deprecated versions (2) Update dependencies (1)

3

Adding cache

Add

Extend cache (28) Improve cache (17) Fix caching issues (4) Fix security issues (3) Migration (2) Other (23)

77

Removing cache

Remove

Improve cache (14) Remove tests (13) Clean up (10) Migration (4) Disable cache (4) Fix caching issues (1) Other (33)

79

Adding parameter

Add

Improve cache (12) Update dependencies (4) Fix caching issues (2) Migration (2) Clean up (1) Other (7)

28

Removing parameter

Remove

Migration (5) Update dependencies (2) Clean up (2) Improve cache (1) Other (22)

32

Cache version update

Action

Parameter update

that the most common drivers are update dependencies (GHA and associated caching parameters) and improve cache , followed by bot alert, extend cache, and fix caching issues. Lower-frequency drivers include migration, clean up, and remove tests. This indicates that cache modifications are not random edits. Instead, they are largely responses to recurring maintenance and workflow needs.

28

Kazi Amit Hasan et al.

42 (46.2%) 0 (0.0%) 40 (100.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 1 (16.7%) 0 (0.0%) 2 (100.0%) 1 (0.9%)

R

2 (2.2%) 1 (2.1%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 2 (14.3%) 5 (35.7%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 22 (20.6%)

da te p U m

em ov in g

C ng ov i em R

0 (0.0%) 14 (29.8%) 0 (0.0%) 0 (0.0%) 1 (4.3%) 10 (71.4%) 4 (28.6%) 13 (100.0%) 0 (0.0%) 4 (100.0%) 0 (0.0%) 33 (30.8%)

P ar a

ac he

d p . V er ac he

g in

4 (4.4%) 12 (25.5%) 0 (0.0%) 0 (0.0%) 2 (8.7%) 1 (7.1%) 2 (14.3%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 7 (6.5%)

C

0 (0.0%) 17 (36.2%) 0 (0.0%) 28 (100.0%) 4 (17.4%) 0 (0.0%) 2 (14.3%) 0 (0.0%) 3 (50.0%) 0 (0.0%) 0 (0.0%) 23 (21.5%)

A dd

Update dependencies Improve cache Bot alert Extend cache Fix caching issues Clean up Migration Remove tests Fix security issues Disable cache Fix deprecated ver. Other

U

am P ar

ac he C A dd

in g

Categories

P ar am

Table 10: Frequency distribution of drivers across different cache modification targets.

43 (47.3%) 3 (6.4%) 0 (0.0%) 0 (0.0%) 16 (69.6%) 1 (7.1%) 1 (7.1%) 0 (0.0%) 2 (33.3%) 0 (0.0%) 0 (0.0%) 21 (19.6%)

Total 91 47 40 28 23 14 14 13 6 4 2 107

Observation 3.2: Automation explains many cache version upgrades, while most other cache modifications are driven by human workflow needs. Table 8 shows that bot-initiated changes appear primarily under bot alert category. Table 10 shows that all bot alert cases map to cache version update (40 of 40 cases, 100%), and bot alerts do not appear as a driver for other targets in the coded sample. In contrast, Table 8 shows that improve cache, fix caching issues, extend cache, migration, and clean up are human-driven, and Table 10 shows these drivers distribute across parameter changes and add/remove actions. This indicates that bot activity is concentrated on version updates, whereas the broader evolution of caching configurations—parameter tuning, adding/removing caches, and restructuring workflows occurs primarily because developers respond to workflow behavior, CI needs, and caching stability issues. Observation 3.3: Parameter updates are driven by humans. Table 7 shows that transitions targeting parameter update (P up) dominate the transition data. The most frequent repeated behavior includes P up to P up (Table 7). Table 10 shows that when the stated driver is fix caching issues, the target is most commonly parameter update (16 cases, 69.6%). Table 9 further shows that these P up cases are human-initiated and are frequently explained by update dependencies and fix caching issues. This explains the rapid iteration patterns observed in RQ2 (especially in build and test jobs), many frequent parameter updates reflect developers repeatedly adjusting keys/paths and related parameters to stabilize caching behavior and address cache misses or workflow failures, rather than only planned performance refinement. Observation 3.4: Cache version updates occur after longer gaps and changes are driven by bot alerts and human driven dependency maintenance of GHA. Along with fast parameter updates, RQ2 showed that initial version updates (EC → C up) are long-delayed actions. Table 7 confirms this, showing the median time from enable cache to the first cache version update is 384.13 days. Table 9 explains this delay as the cache version update

How Developers Adopt, Use, and Evolve CI/CD Caching

29

action (N=86) is almost exclusively an Upgrade (N=84). The drivers for this upgrade are split between human-led updating dependency and automated bot alert. These evidences indicate that developers tend to stay in their initial cache versions. Also, while bots ensure dependency currency, developers also depend on manual PRs to update caching versions. Although rare, downgrade cases also appear, showing that some upgrades introduce problems and later require recovery.

Observation 3.5: Developers add additional cache to expand and improve their existing workflow. Table 9 shows that for adding cache, the most common drivers are Extend cache and Improve cache, followed by smaller contributions from fix caching issues, fix security issues, and migration. These driver distributions are consistent with the RQ2 observation that EC → C add occurs within weeks in build and test jobs, suggesting incremental adoption. Developers often add caches to expand CI coverage and improve performance beyond the initial configuration. This supports the view that caching usage evolves through gradual refinement after initial enablement.

Observation 3.6: Cache removal appears both as a short-gap reversal after cache addition and as a later maintenance action. Table 7 shows that adding cache → removing cache occurs with a short cadence (median 4.25 days). Table 7 also shows a longer pattern for EC → C rm, with N=111 and a median of 129.68 days. Table 9 shows that improve cache is a common driver for removals that follow additions, while later removals are frequently explained by Remove tests (N=13) and clean up. Table 9 also shows contributions from migration and explicit disable cache. This shows that cache removal is not explained by one reason. Some removals reflect a quick reversal after a cache is added, while others occur months later when workflows change.

Observation 3.7: Security-driven cache modifications are infrequent but involve multiple types of actions. Table 8 shows fix security issues is relatively rare. Table 10 shows that these cases map to adding cache (N=3, 50.0%), parameter update (N=2, 33.3%), and cache version update (N=1, 16.7%). When security concerns appear in commit/PR context, developers respond through different workflow edits (adding caches, adjusting parameters, or updating versions) rather than relying on a single type of cache modification.

30

Kazi Amit Hasan et al.

RQ3 Summary: Parameter updates dominate cache-related modifications and are commonly performed by developers to fix caching issues, leading to repeated edits to cache keys and paths within short gaps. In contrast, cache version updates typically occur long after enabling caching and are mainly explained by updating dependencies, with bot alerts initiating many of these upgrades. Beyond updates, developers add caches mainly to extend cache usage and improve cache behavior, while cache removal reflects two common situations. Some caches are removed soon after being added due to ineffectiveness and others are removed later when workflows change through removing tests, clean up, migration, or disabling caching. Overall, bots are mainly responsible for cache version updates, whereas developers perform most of the remaining cache modifications that shape how caching evolves in practice.

7 Implications Caching is not a one-time optimization and it requires ongoing maintenance. Caching is often presented as a straightforward way to reduce CI runtime (Bouzenia and Pradel, 2024), but our results show that in practice it behaves more like a maintained subsystem than a lightweight optimization flag. RQ2 shows that after enablement, caching evolves through repeated updates, additions, and removals across job types (Figures 4 to 11). In particular, repetitive self-loops such as parameter and cache version updates dominate the transition structure (e.g., build and test in Tables 11 and 12), showing that developers do not simply enable caching once and leave it unchanged. RQ3 reinforces this by showing that the dominant target of maintenance is parameter update with 854 transitions (58.20%) in Table 7. Taken together, these findings imply that caching should not be described merely as a performance optimization. Hence, caching should be treated as an evolving CI configuration component that introduces ongoing maintenance work, corrective edits, and long-term upkeep. Automation support in the current bot ecosystem is largely limited to cache version updates. Our qualitative results show that automation plays a visible but fundamentally narrow role in cache maintenance. Table 8 shows that bot alert accounts for 40 cases, while the remaining drivers are primarily humancentered, such as update dependencies (91), improve cache (47), extend cache (28), and fix caching issues (23). Table 10 further shows that bot alerts map entirely to cache version update (40/40, 100%) and do not appear as a driver for parameter updates, adding cache, removing cache, or cleanup-related actions. This indicates that existing bot ecosystem helps repositories stay current with cache-action versions, but does not reduce the dominant human effort involved in tuning, restructuring, and stabilizing cache configurations. Hence, tool builders should move beyond version bumping and develop

How Developers Adopt, Use, and Evolve CI/CD Caching

31

workflow-aware bots that can assist with broader cache maintenance tasks, especially those involving parameter refinement and corrective reconfiguration. Parameter updates are the central maintenance bottleneck in caching and require intelligent tool support. Among all target states, parameter update is the largest maintenance category, accounting for 854 transitions (58.20%) in Table 7. This is not only frequent, but also highly repetitive. Table 7 shows that P up → P up alone contributes 497 transitions, and RQ2 shows that these repeated updates occur quickly in build and test jobs, with median delays of 4.08 days and 2.93 days, respectively (Tables 11 and 12; Figures 4 and 5). RQ3 further shows that these updates are primarily humandriven and commonly tied to fix caching issues and update dependencies (Tables 8, 9, and 10). This implies that the main engineering burden of caching lies not in enabling cache, but in repeatedly refining keys, paths, and related configuration details. Therefore, future agentic and recommendation-based support should prioritize parameter synthesis, cache-key validation, restore-key design, because this is where the largest share of human maintenance effort is currently spent. Effective cache maintenance support must include helping developers remove unnecessary cache configurations. Our findings show that caching involves not only introducing and updating cache-related settings, but also removing them as workflows evolve. In RQ2, removal-related transitions appear across multiple job types, showing that previously introduced cache-related configurations are sometimes revised through deletion. RQ3 explains that Removing cache is commonly driven by Improve cache, Remove tests, Clean up, Migration, and Disable cache, whereas Removing parameter is associated with Migration, Update dependencies, Clean up, and Improve cache (Table 9). Table 10 further confirms that these removal-related changes arise from multiple human-driven maintenance needs, rather than from a single recurring trigger. Therefore, future caching tools should not assume that more cache-related configuration is always beneficial. Future work should quantify whether frequent cache maintenance actually improves CI execution time. Our findings show that developers engage in highly frequent cache maintenance, particularly repeated parameter updates in build and test jobs that occur every few days (Tables 11 and 12). However, while our study reveals how often these configurations are tuned, it remains unclear whether such repeated adjustments translate into actual performance gains. As the primary goal of caching is to accelerate CI pipelines, future work should evaluate the resulting change in workflow execution time after specific cache-related transitions. By linking maintenance actions to before-and-after workflow run durations, researchers can determine which configuration changes yield measurable performance improvements and which primarily introduce maintenance overhead without corresponding executiontime benefits.

32

Kazi Amit Hasan et al.

8 Threats to Validity Construct validity: Our operationalization of “caching adoption” and cache maintenance activities may not capture all ways developers implement caching in GHA workflows. We identify caching usage through cache-related configuration steps and classify step-level strategies into explicit caching, package manager caching, and Docker layer caching. This approach may miss cases where caching is implemented through custom scripts or actions that do not match our detection rules, and it may also include borderline cases that resemble caching but are used for other purposes. Similarly, our job type categorization relies on developer-defined job names and a keyword-based taxonomy. Because job names are inconsistent across repositories, some jobs may be misclassified, which can affect job-level prevalence results in RQ1 and job-type comparisons in RQ2. Internal validity: We prune low probability transitions below 0.05 to focus on recurring patterns. This design choice may bias the representation of evolutionary patterns by removing rare yet meaningful behaviors. Open coding was the primary methodology employed in our study. This approach introduces subjectivity into our results, as annotators may have different interpretations of the coding scheme. This could introduce annotator bias into our qualitative findings, particularly in the distribution of drivers for cache modifications. In RQ3, we infer drivers from commit messages and, when available, linked PR context. When the rationale is missing or unclear, the transition is coded as Other and excluded from driver-focused interpretation. This introduces a threat that some drivers are underrepresented because they are not explicitly documented in the commit or PR text. Conclusion validity: Our findings rely primarily on descriptive statistics, transition probabilities, and time to transition summaries. Some job types have fewer observations, which can increase variance in estimated probabilities and medians and can make long tail behaviors sensitive to small changes in the dataset. For the qualitative component, we use a statistically significant random sample per target state with a 90% confidence interval and 10% margin of error. While this sampling supports stable estimates for common drivers, rare drivers and rare actions such as downgrades may remain sensitive to sampling noise. We mitigate over interpretation by reporting exact counts and by treating low frequency categories as limited evidence rather than broad trends. External validity: Our dataset consists of GitHub repositories and workflows collected using the GitHub GraphQL API and reconstructed workflow histories, as described in Section 3. The observed adoption rates, job type prevalence, and maintenance patterns may not generalize to private repositories, enterprise environments, or repositories that use different CI platforms. Results may also differ across language ecosystems, workflow templates, and project domains that are not represented in our sample. In addition, our step level strategy taxonomy reflects the caching mechanisms observed in our dataset. Other caching

How Developers Adopt, Use, and Evolve CI/CD Caching

33

approaches may exist and could be more common in different populations. Therefore, we interpret our results as evidence about caching adoption and evolution within the studied GHA sample rather than as universal properties of CI caching. 9 Related Work 9.1 Studies on GitHub Actions GitHub Actions (GHA), introduced in 2019, has rapidly become a major automation platform in open-source software development.18 Early empirical studies established its widespread adoption. For example, Decan et al. analyzed 68K repositories and reported that 43.9% used GHA, while also identifying common workflow triggers and automation practices (Decan et al., 2022). Similarly, Chen et al. found broad adoption of GHA across the projects they studied (Chen et al., 2021). Together, these studies show that GHA has become a core part of continuous integration and deployment (CI/CD) processes. As adoption increased, researchers began to investigate how GHA workflows evolve over time. Valenzuela et al. manually analyzed 222 commits and identified 11 categories of workflow modifications, showing that workflow maintenance is frequent and multifaceted (Valenzuela-Toledo and Bergel, 2022). In subsequent work, they highlighted the hidden costs of workflow maintenance, showing that bug fixing and CI/CD performance improvement are major drivers of workflow changes (Valenzuela-Toledo et al., 2024). Rostami Mazrae et al. conducted a mixed-method study of workflow evolution using 439 manually analyzed modified workflow files and a large-scale quantitative analysis over 49K+ repositories (Rostami Mazrae et al., 2025). Their results show that GHA workflows are continuously maintained, with modifications dominating over additions and removals, and that most changes concentrate on task specification and task configuration, especially within workflow jobs and steps. Complementing this line of work, Zheng et al. conducted a large-scale empirical study of workflow failures and identified recurring failure categories and configuration weaknesses that reduce workflow reliability (Zheng et al., 2025). Huang et al. studied rerun practices in GitHub Actions workflows, focusing on cases where developers rerun workflows or failed jobs without modifying the repository (Huang and Lin, 2026). Their study of 3,320 open-source Java repositories quantified the time and computing waste associated with reruns, analyzed cases where reruns succeed, and investigated root causes of workflow execution flakiness. Their results provide insight into how developers cope with workflow failures and highlight reruns as an important reliability and efficiency concern in GitHub Actions. Other studies have focused on the structural, operational, and security characteristics of GHA workflows. Abrokwah et al. examined workflow heterogeneity, compliance, and complexity, showing substantial structural variation 18

https://github.blog/news-insights/product-news/github-actions-now-support-ci-cd/

34

Kazi Amit Hasan et al.

and deviation from best practices across repositories (Abrokwah and Ghaleb, 2025). Rostami et al. classified workflow changes at scale, revealing systematic patterns of addition, refactoring, and removal over time (Rostami Mazrae et al., 2025). Bouzenia et al. quantified resource usage in GHA and identified opportunities to reduce workflow cost and execution time (Bouzenia and Pradel, 2024). From a security perspective, prior work has shown that many workflows depend on outdated third-party Actions, exposing repositories to known vulnerabilities (Decan et al., 2023; Benedetti et al., 2022). Decan et al. further showed that the reuse of Actions is highly concentrated on a small subset of the ecosystem (Decan et al., 2023). Beyond workflow internals, Wessel et al. demonstrated that adopting GHA can alter pull request dynamics, including review time and communication behavior, and also examined the broader automation and bot ecosystem enabled by GHA (Wessel et al., 2023). In summary, existing research has studied GHA from multiple perspectives, including adoption, maintenance, failures, reruns, complexity, performance, and security. However, caching has not been examined as a distinct topic of empirical study. When discussed, it is usually presented as only one example of workflow optimization and maintenance. In contrast, our work provides a cache-centric and fine-grained view of GHA workflows. Specifically, we examine caching at the job and step levels, analyze where cache-related configurations appear across different CI/CD phases, and study how these configurations evolve over time through concrete cache-related activities. We further investigate the motivations behind these changes and identify who initiates them. This makes our work more granular than prior GHA studies on workflow evolution and also distinct from prior CI caching studies that focus mainly on overall build-level adoption and performance outcomes.

9.2 Studies on CI/CD Caching Strategies Caching is a widely used acceleration technique in software build and CI/CD pipelines, but its effectiveness depends on what is cached, how the cache is maintained, and where it is applied in the pipeline. Prior work has shown that caching can substantially reduce build time (Bouzenia and Pradel, 2024), (Ghaleb et al., 2026b), while also introducing trade-offs related to invalidation, download overhead, and maintenance complexity19 . Gallaba et al. empirically studied environment caching and step-skipping, showing that dependency and environment caching can accelerate builds, but that the benefits may diminish when cache retrieval overhead or invalidation costs are high (Gallaba et al., 2020). In the Bazel ecosystem, Zheng et al. conducted controlled experiments across hundreds of projects, evaluating parallel and incremental builds under popular CI services, including GitHub Actions, and found that remote and incremental caching can improve performance, although the gains are highly workload-dependent (Zheng et al., 2024). Ghaleb et al. studied CI caching in 19

https://www.datadoghq.com/blog/cache-purge-ci-cd/

How Developers Adopt, Use, and Evolve CI/CD Caching

35

Travis CI and showed that caching adoption, maintenance, and performance benefits vary substantially across projects, with many projects applying caching only once and seeing limited or inconsistent build-time improvements (Ghaleb et al., 2026b). These studies provide valuable evidence that caching can improve CI efficiency, but they primarily focus on performance outcomes, cache overhead under specific tools, build systems, and experimental settings. In summary, prior work on caching has mainly evaluated whether particular caching mechanisms improve performance, when they are beneficial, and what overheads they introduce. Our work complements this literature by studying caching as it is configured and maintained in the wild within GHA workflows. Unlike prior work on Travis CI caching, which primarily studies caching at the project and build levels, our work investigates caching in GHA at finer granularity, focusing on job- and step-level configurations. This allows us to examine where caching is introduced across workflow phases, how cache-related configurations evolve through concrete maintenance operations over time, and why and by whom these changes are made. Thus, rather than focusing only on whether caching improves runtime, we study caching as an evolving workflow configuration practice in modern GHA workflows.

10 Conclusion and Future Work This paper presented a comprehensive characterization of caching adoption and evolution in GitHub Actions workflows. We found that repositories adopting caching are typically larger and more active than non-adopters, often managing their CI workloads with a single workflow file. Within these repositories, caching is most prevalent in build and test jobs where developers predominantly rely on explicit caching strategies over package manager caching. Our evolutionary analysis reveals that caching is not a one-time configuration, it requires repetitive maintenance activities. We observed that build and test jobs undergo frequent and rapid parameter tuning as developers iteratively adjust cache keys and paths to address caching issues. in contrast, integration and release jobs remain stable for longer periods. Through our qualitative analysis of drivers, we determined that while bots play a significant role in triggering long-delayed cache version updates, humans are responsible for the majority of maintenance activities. These human-driven modifications focus on extending cache coverage, improving cache performance, and fixing caching issues through repeated parameter updates. In the future, we plan to expand our work by conducting an impact-driven analysis that correlates the evolutionary events we identified with build log data, allowing us to quantify the direct effect of these changes on build duration, success rates, and resource costs. Furthermore, given our finding that humans handle all complex tasks, there is a clear need for better automated support. We therefore plan to design and develop specialized tools for cache maintenance to reduce repetitive trial-and-error edits and help developers keep caching effective over time.

36

Kazi Amit Hasan et al.

Acknowledgements We thank Vu Thanh Loc Mai and Mohammad Sadegh Sheikhaei for their help in this project. Also, we acknowledge the support of the Natural Sciences and Engineering Research Council of Canada (NSERC), [funding reference number: RGPIN-202504723].

Conflict of Interest The authors declare that they have no conflict of interest.

References Abrokwah E, Ghaleb TA (2025) An empirical study of complexity, heterogeneity, and compliance of github actions workflows. arXiv preprint arXiv:250718062 AlMulla B, Assi M, Hassan S (2025) Understanding the challenges and promises of developing generative ai apps: An empirical study. arXiv preprint arXiv:250616453 Baltes S, Diehl S (2019) Usage and attribution of stack overflow code snippets in github projects. Empirical Software Engineering 24(3):1259–1295 Benedetti G, Verderame L, Merlo A (2022) Automatic security assessment of github actions workflows. In: Proceedings of the 2022 ACM Workshop on Software Supply Chain Offensive Research and Ecosystem Defenses, pp 37–45 Bouzenia I, Pradel M (2024) Resource usage and optimization opportunities in workflows of github actions. In: Proceedings of the 46th IEEE/ACM International Conference on Software Engineering, pp 1–12 Chen T, Zhang Y, Chen S, Wang T, Wu Y (2021) Let’s supercharge the workflows: An empirical study of github actions. In: 2021 IEEE 21st International Conference on Software Quality, Reliability and Security Companion (QRS-C), IEEE, pp 01–10 Decan A, Mens T, Mazrae PR, Golzadeh M (2022) On the use of github actions in software development repositories. In: 2022 IEEE International Conference on Software Maintenance and Evolution (ICSME), IEEE, pp 235–245 Decan A, Mens T, Delicheh HO (2023) On the outdatedness of workflows in the github actions ecosystem. Journal of Systems and Software 206:111827 Gagniuc PA (2017) Markov chains: from theory to implementation and experimentation. John Wiley & Sons Gallaba K (2019) Improving the robustness and efficiency of continuous integration and deployment. In: 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME), IEEE, pp 619–623 Gallaba K, Ewart J, Junqueira Y, McIntosh S (2020) Accelerating continuous integration by caching environments and inferring dependencies. IEEE Transactions on Software Engineering 48(6):2040–2052 Ghaleb TA, Abduljalil O, Hassan S (2026a) Ci/cd configuration practices in open source android apps: An empirical study. ACM Transactions on Software Engineering and Methodology 35(2):1–40

How Developers Adopt, Use, and Evolve CI/CD Caching

37

Ghaleb TA, da Costa DA, Zou Y (2026b) The promise and reality of continuous integration caching: An empirical study of travis ci builds. arXiv preprint arXiv:260119146 Hao H, Hasan KA, Qin H, Macedo M, Tian Y, Ding SH, Hassan AE (2024) An empirical study on developers’ shared conversations with chatgpt in github pull requests and issues. Empirical Software Engineering 29(6):150 Huang J, Lin B (2026) On the reruns of github actions workflows. ACM Transactions on Software Engineering and Methodology Rahman N (2023) Exploring the role of continuous integration and continuous deployment (ci/cd) in enhancing automation in modern software development: A study of patterns. Tools, And Outcomes Rostami Mazrae P, Decan A, Mens T, Wessel M (2025) An empirical study of the evolution of github actions workflows. Available at SSRN 5369484 Shahin M, Babar MA, Zhu L (2017) Continuous integration, delivery and deployment: a systematic review on approaches, tools, challenges and practices. IEEE access 5:3909–3943 Valenzuela-Toledo P, Bergel A (2022) Evolution of github action workflows. In: 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), IEEE, pp 123–127 Valenzuela-Toledo P, Bergel A, Kehrer T, Nierstrasz O (2024) The hidden costs of automation: An empirical study on github actions workflow maintenance. In: 2024 IEEE International Conference on Source Code Analysis and Manipulation (SCAM), IEEE, pp 213–223 Wessel M, Vargovich J, Gerosa MA, Treude C (2023) Github actions: the impact on the pull request process. Empirical Software Engineering 28(6):131 Yang S (2025) The impact of continuous integration and continuous delivery on software development efficiency. Journal of Computer, Signal, and System Research 2(3):59–68 Zheng L, Li S, Huang X, Huang J, Lin B, Chen J, Xuan J (2025) Why do github actions workflows fail? an empirical study. ACM Transactions on Software Engineering and Methodology Zheng S, Adams B, Hassan AE (2024) Does using bazel help speed up continuous integration builds? Empirical Software Engineering 29(5):110

A Evolution of caching configurations for different job type

38

Kazi Amit Hasan et al.

Table 11: Evolution of caching configurations for the build job type. For each source state (From), the table details the resulting target states (To), the total occurrences, and the conditional probability (%) of each specific transition. The remaining columns report the descriptive statistics (Avg, Med, Std, Min, Max) for the time in days elapsed before each transition. From

To

Total

Avg

Med

Std

Min

Max

%

C add

C add C rm P up C up

26 19 16 14

75

28.03 112.31 106.70 373.24

0.00 0.00 51.16 309.69

93.91 219.41 132.88 342.69

0.00 0.00 0.05 0.00

462.91 750.21 419.07 1080.84

34.67 25.33 21.33 18.67

C up P up P add P rm

3 2 1 1

7

59.94 3.16 0.00 0.00

0.00 3.16 0.00 0.00

103.82 4.46 – –

0.00 0.00 0.00 0.00

179.82 6.31 0.00 0.00

42.86 28.57 14.29 14.29

C up P up C rm

177 27 18

222

59.56 148.09 270.58

0.00 79.07 328.70

138.32 179.24 225.37

0.00 0.00 0.00

668.08 592.69 707.67

77.63 11.84 7.89

C up EC P up C rm C add

69 62 40 39 21

231

637.14 0.00 173.07 422.58 153.44

587.98 0.00 105.93 343.82 53.14

448.74 0.00 221.14 428.02 177.26

6.35 0.00 0.00 0.00 0.00

1615.93 0.00 874.04 1514.81 504.87

29.49 26.50 17.09 16.67 8.97

C rm C add P up C up

48 19 9 7

83

13.73 47.11 55.79 220.73

0.00 0.01 30.82 0.00

66.32 105.93 101.96 510.42

0.00 0.00 0.00 0.00

390.89 445.13 321.98 1374.41

55.17 21.84 10.34 8.05

P rm

C rm P up

2 2

4

0.00 117.53

0.00 117.53

0.00 166.21

0.00 0.00

0.00 235.06

50.00 50.00

P up

P up C up C rm C add

135 31 24 11

201

45.40 173.03 152.99 25.17

4.08 104.80 43.29 4.73

105.61 277.52 222.58 52.00

0.00 0.00 0.00 1.74

941.90 1366.00 902.66 177.96

65.53 15.05 11.65 5.34

P add

C up

EC

C rm

Count

How Developers Adopt, Use, and Evolve CI/CD Caching

39

Table 12: Evolution of caching configurations for the test job type. For each source state (From), the table details the resulting target states (To), the total occurrences, and the conditional probability (%) of each specific transition. The remaining columns report the descriptive statistics (Avg, Med, Std, Min, Max) for the time in days elapsed before each transition. From

To

Total

Avg

Med

Std

Min

Max

%

P up

P up C up C rm C add

142 31 19 16

208

62.75 111.24 221.16 64.67

2.93 31.50 7.24 43.66

156.46 152.83 393.51 85.87

0.00 0.00 0.00 0.00

986.95 584.03 1154.81 323.27

66.05 14.42 8.84 7.44

C up P up C rm

114 33 12

159

58.67 76.46 150.41

0.00 0.00 94.56

137.94 120.17 148.67

0.00 0.00 0.00

713.59 433.71 435.59

68.26 19.76 7.19

C add C rm P up C up P add

48 42 21 15 7

133

24.83 112.53 105.59 340.95 0.00

0.00 32.49 8.47 231.83 0.00

93.51 152.93 219.12 356.57 0.00

0.00 0.00 0.00 0.01 0.00

463.22 485.66 973.79 1294.85 0.00

36.09 31.58 15.79 11.28 5.26

P up C up EC C rm C add

45 41 38 30 29

183

263.33 515.49 0.00 243.13 172.12

85.03 351.05 0.00 120.77 41.97

347.74 457.35 0.00 304.07 243.45

0.00 0.20 0.00 0.00 0.01

981.44 1549.93 0.00 1271.64 962.46

24.32 22.16 20.54 16.22 15.68

C rm C add P rm P up C up

44 32 10 9 8

103

39.12 21.35 0.00 85.85 265.57

0.00 0.00 0.00 13.21 269.97

105.53 80.79 0.00 189.00 180.03

0.00 0.00 0.00 0.00 0.00

448.01 445.13 0.00 584.88 557.51

41.51 30.19 9.43 8.49 7.55

C add C rm C up

8 3 2

13

0.00 0.00 0.00

0.00 0.00 0.00

0.00 0.00 0.00

0.00 0.00 0.00

0.00 0.00 0.00

61.54 23.08 15.38

C add P up P add C up C rm EC

5 5 3 3 1 1

18

0.00 186.87 0.00 0.00 875.94 0.00

0.00 140.78 0.00 0.00 875.94 0.00

0.00 257.22 0.00 0.00 -

0.00 0.00 0.00 0.00 875.94 0.00

0.00 628.32 0.00 0.00 875.94 0.00

27.78 27.78 16.67 16.67 5.56 5.56

C up

C add

EC

C rm

P rm

P add

Count

40

Kazi Amit Hasan et al.

Table 13: Evolution of caching configurations for the integration job type. For each source state (From), the table details the resulting target states (To), the total occurrences, and the conditional probability (%) of each specific transition. The remaining columns report the descriptive statistics (Avg, Med, Std, Min, Max) for the time in days elapsed before each transition. From

To

Total

Avg

Med

Std

Min

Max

%

P up

P up C up

Count 25 10

35

137.45 235.11

87.25 141.92

124.44 405.25

13.03 0.00

538.19 1366.00

67.57 27.03

C up

P up C rm

9 1

10

78.27 285.92

64.22 285.92

97.34 0.00

0.00 285.92

292.84 285.92

90.00 10.00

EC

P up C up C rm C add

8 3 3 2

16

180.66 474.73 184.33 438.48

203.39 326.13 252.89 438.48

91.09 304.42 119.54 250.70

0.00 273.13 46.30 261.21

300.88 824.91 253.81 615.76

50.00 18.75 18.75 12.50

C rm

C add C rm

4 1

5

19.20 0.00

0.00 0.00

38.39 0.00

0.00 0.00

76.79 0.00

80.00 20.00

C add

C rm C add C up P up

4 2 1 1

8

82.21 5.21 584.03 8.47

1.54 5.21 584.03 8.47

162.37 7.37 0.00 0.00

0.00 0.00 584.03 8.47

325.75 10.42 584.03 8.47

50.00 25.00 12.50 12.50

P rm

C rm

1

1

0.00

0.00

0.00

0.00

0.00

100.00

P add

C up P up

1 1

2

584.03 161.81

584.03 161.81

0.00 0.00

584.03 161.81

584.03 161.81

50.00 50.00

How Developers Adopt, Use, and Evolve CI/CD Caching

41

Table 14: Evolution of caching configurations for the release job type. For each source state (From), the table details the resulting target states (To), the total occurrences, and the conditional probability (%) of each specific transition. The remaining columns report the descriptive statistics (Avg, Med, Std, Min, Max) for the time in days elapsed before each transition. From

To

P up

P up C up C rm

30 11 4

C rm C up P up C add EC

EC

C up

C rm

C add

P add

Count

Total

Avg

Med

Std

Min

Max

%

45

90.86 166.19 91.89

61.95 137.67 85.88

86.46 153.70 31.95

0.00 0.00 59.81

311.97 557.50 135.99

63.83 23.40 8.51

18 13 12 5 5

53

273.29 445.71 203.64 71.33 0.00

85.34 270.37 125.98 11.93 0.00

377.79 473.25 197.30 133.69 0.00

0.00 0.25 0.00 1.08 0.00

1170.03 1298.49 512.00 309.99 0.00

33.96 24.53 22.64 9.43 9.43

C up P up C rm

13 8 4

25

137.12 70.56 371.48

0.00 18.06 429.10

200.97 107.20 238.07

0.00 0.00 34.54

520.81 292.02 593.18

50.00 30.77 15.38

C rm C add C up P up

10 6 1 1

18

14.50 74.19 35.09 36.92

0.00 0.00 35.09 36.92

45.86 181.73 0.00 0.00

0.00 0.00 35.09 36.92

145.03 445.13 35.09 36.92

55.56 33.33 5.56 5.56

C add C up C rm P add P up

4 4 2 1 1

12

0.00 150.73 0.49 0.42 55.04

0.00 117.86 0.49 0.42 55.04

0.00 172.97 0.70 0.00 0.00

0.00 0.04 0.00 0.42 55.04

0.00 367.15 0.99 0.42 55.04

33.33 33.33 16.67 8.33 8.33

C up

1

1

0.00

0.00

0.00

0.00

0.00

100.00

42

Kazi Amit Hasan et al.

Table 15: Evolution of caching configurations for the lint job type. For each source state (From), the table details the resulting target states (To), the total occurrences, and the conditional probability (%) of each specific transition. The remaining columns report the descriptive statistics (Avg, Med, Std, Min, Max) for the time in days elapsed before each transition. From

To

Total

Avg

Med

Std

Min

Max

%

EC

C up C rm P up EC

11 4 4 2

21

476.04 428.71 263.36 0.00

160.59 340.82 99.76 0.00

530.04 382.73 380.50 0.00

0.01 115.27 23.89 0.00

1413.24 917.91 830.04 0.00

47.83 17.39 17.39 8.70

C up P up C rm

8 8 2

18

244.19 116.10 535.55

134.07 103.77 535.55

247.46 83.99 470.44

11.38 27.75 202.90

648.94 263.80 868.20

44.44 44.44 11.11

C up P up C rm P rm

8 7 2 2

19

188.50 60.67 2.04 0.00

190.79 0.03 2.04 0.00

185.58 88.67 2.88 0.00

0.00 0.00 0.00 0.00

508.01 224.30 4.08 0.00

38.10 33.33 9.52 9.52

C rm C add P rm

7 2 2

11

0.00 222.57 0.00

0.00 222.57 0.00

0.00 314.76 0.00

0.00 0.00 0.00

0.00 445.13 0.00

58.33 16.67 16.67

C add C up P add C rm

3 2 1 1

7

0.00 201.12 0.00 0.00

0.00 201.12 0.00 0.00

0.00 234.80 0.00 0.00

0.00 35.09 0.00 0.00

0.00 367.15 0.00 0.00

42.86 28.57 14.29 14.29

C up C add P up

2 1 1

4

814.58 89.77 0.15

814.58 89.77 0.15

779.83 0.00 0.00

263.15 89.77 0.15

1366.00 89.77 0.15

50.00 25.00 25.00

C up C rm

2 2

4

0.00 0.00

0.00 0.00

0.00 0.00

0.00 0.00

0.00 0.00

50.00 50.00

P up

C up

C rm

C add

P add

P rm

Count

How Developers Adopt, Use, and Evolve CI/CD Caching

43

Table 16: Evolution of caching configurations for the lint job type. For each source state (From), the table details the resulting target states (To), the total occurrences, and the conditional probability (%) of each specific transition. The remaining columns report the descriptive statistics (Avg, Med, Std, Min, Max) for the time in days elapsed before each transition. From

To

Total

Avg

Med

Std

Min

Max

%

P up

P up C up

46 3

49

120.01 269.95

26.14 257.62

173.33 276.33

0.20 0.00

611.55 552.24

92.00 6.00

EC

P up C up C rm EC

19 10 3 2

34

151.56 517.14 196.91 0.00

94.53 384.02 3.48 0.00

133.79 542.26 338.04 0.00

0.01 0.01 0.01 0.00

369.06 1549.93 587.24 0.00

54.29 28.57 8.57 5.71

C up C add C rm P up

6 2 1 1

10

199.45 7.10 4.08 0.00

190.42 7.10 4.08 0.00

218.27 10.05 0.00 0.00

0.00 0.00 4.08 0.00

431.84 14.21 4.08 0.00

60.00 20.00 10.00 10.00

C rm C add P rm

5 4 1

10

0.00 56.72 0.00

0.00 3.91 0.00

0.00 108.25 0.00

0.00 0.00 0.00

0.00 219.07 0.00

50.00 40.00 10.00

C rm C up P up

3 2 1

6

251.84 250.48 217.80

250.30 250.48 217.80

132.99 354.23 0.00

119.62 0.00 217.80

385.59 500.96 217.80

50.00 33.33 16.67

C rm

1

1

0.00

0.00

0.00

0.00

0.00

100.00

C up

C rm

C add

P rm

Count

Table 17: Evolution of caching configurations for the linux job type. For each source state (From), the table details the resulting target states (To), the total occurrences, and the conditional probability (%) of each specific transition. The remaining columns report the descriptive statistics (Avg, Med, Std, Min, Max) for the time in days elapsed before each transition. From

To

Total

Avg

Med

Std

Min

Max

%

C up

C up P up

Count 4 1

5

230.93 210.97

196.61 210.97

272.48 0.00

0.00 210.97

530.50 210.97

80.00 20.00

EC

C up C add P up

4 2 1

7

452.34 71.45 133.90

476.09 71.45 133.90

442.51 99.70 0.00

15.77 0.96 133.90

841.42 141.95 133.90

57.14 28.57 14.29

P up

P up C up

3 2

5

191.71 150.85

200.53 150.85

76.88 65.13

110.81 104.80

263.80 196.90

60.00 40.00

C add

C up

1

1

282.87

282.87

0.00

282.87

282.87

100.00

44

Kazi Amit Hasan et al.

Table 18: Evolution of caching configurations for the sync job type. For each source state (From), the table details the resulting target states (To), the total occurrences, and the conditional probability (%) of each specific transition. The remaining columns report the descriptive statistics (Avg, Med, Std, Min, Max) for the time in days elapsed before each transition. From

To

Total

Avg

Med

Std

Min

Max

%

EC

P up C rm

Count 28 2

30

250.33 52.97

72.96 52.97

284.48 69.26

0.00 3.99

693.07 101.94

90.32 6.45

P up

P up C rm P rm C add

18 11 4 2

35

409.30 115.62 23.02 0.00

288.65 13.15 7.07 0.00

220.85 123.56 31.89 0.00

0.69 7.07 7.07 0.00

698.58 244.66 70.85 0.00

48.65 29.73 10.81 5.41

C add

C add P up

11 11

22

0.00 108.01

0.00 8.47

0.00 209.03

0.00 7.78

0.00 685.42

45.83 45.83

P rm

C add C rm

6 5

11

0.00 0.00

0.00 0.00

0.00 0.00

0.00 0.00

0.00 0.00

54.55 45.45

C rm

P rm C add C rm P add

6 5 1 1

13

0.00 0.00 0.00 0.00

0.00 0.00 0.00 0.00

0.00 0.00 0.00 0.00

0.00 0.00 0.00 0.00

0.00 0.01 0.00 0.00

46.15 38.46 7.69 7.69

C up

P up

2

2

0.00

0.00

0.00

0.00

0.00

100.00

P add

C add C up

1 1

2

0.00 0.00

0.00 0.00

0.00 0.00

0.00 0.00

0.00 0.00

50.00 50.00

Related documents

• Requirements After the First Edit: Mining Late Requirement Emergence and Rework in Real-World Coding-Agent Sessions#639748