Caprese: Change Impact Recommendation Using History and Runtime Analysis

arXiv:2611.14937v1 [cs.SE] 11 Nov 2026

Caprese: Change Impact Recommendation Using History and Runtime Analysis Sadjad Tavakoli

Saba Alimadadi

Magellan AI BC, Canada sadjad tavakoli@sfu.ca

Simon Fraser University BC, Canada saba@sfu.ca

Abstract—Change impact recommendation identifies program entities potentially affected by source code changes. Historical co-changes and runtime dependencies provide complementary evidence, particularly in JavaScript, where dynamic and asynchronous behaviour complicates dependency analysis. We present C APRESE, an open-source framework for functionlevel change impact recommendation in Node.js. Given a change set, it produces a ranked impact set of potentially affected functions. It supports repository-based recommendation through confidence-guided mining of recurring, change-set-relevant cochange patterns of varying lengths; dynamic recommendation through execution-derived dependencies involving calls, callbacks, events, timers, and shared state; and hybrid recommendation integrating both analyses. Its modular implementation supports reusable analyses. Evaluation on ten open-source applications and 100 selected change sets shows that, on average across projects, 77.4% of the relevant functions among the top 60 hybrid recommendations are identified exclusively by one analysis, demonstrating substantial complementarity. Dynamic recommendation achieves the highest mean precision among the top five recommendations (0.65), while repository-based recommendation performs best when the ranking is extended to 60 recommendations (0.42). Hybrid recommendation outperforms TARMAQ on average but does not consistently outperform the individual modes. C APRESE is publicly available with source code, documentation, and evaluation artifacts. Index Terms—change impact analysis, change impact recommendation, mining software repositories, dynamic program analysis, software evolution, JavaScript

I. I NTRODUCTION Understanding the downstream effects of source code changes is essential to software maintenance. Given a change set of modified program entities, change impact analysis identifies an impact set of potentially affected entities, while change impact recommendation prioritizes them for inspection, modification, or regression testing [1]–[4]. Repository-based recommendation identifies relationships through historical co-changes, but may include incidental changes or miss dependencies absent from repository history [5], [6]. In JavaScript, dynamic dispatch, callbacks, and asynchronous execution complicate static dependency analysis [4], [7], [8]. Dynamic analysis captures execution-derived dependencies but is limited to observed executions. Integrating historical and runtime evidence can therefore provide complementary coverage of potentially impacted functions.

We present C APRESE, an open-source framework for function-level change impact recommendation in Node.js applications. Given a change set of modified functions, C APRESE produces a ranked impact set, termed the Candidate Inspection Set (CIS), through three modes: repository-based recommendation, using confidence-guided mining of recurring, change-set-relevant co-change patterns of varying lengths; dynamic recommendation, capturing dependencies involving calls, callbacks, events, timers, and shared state; and hybrid recommendation, combining both. Its modular implementation supports reuse of historical and runtime analyses across change sets. We evaluate C APRESE on 10 open-source Node.js applications using 100 selected change sets and manually established reference impact sets. On average across projects, 77.4% of the relevant functions among the top 60 hybrid recommendations are identified exclusively by one analysis, demonstrating substantial complementarity. Dynamic recommendation achieves the highest mean precision at Top-5 and Top-20, while repository-based recommendation leads at Top60. Hybrid recommendation outperforms TARMAQ [6] on average, but does not consistently outperform the individual modes. The main contributions are: • C APRESE , an open-source Node.js impact recommendation framework integrating targeted co-change pattern mining, runtime dependency analysis, and hybrid ranking, with reusable components and documentation. 1 • An empirical evaluation and reusable artifacts characterizing recommendation precision, signal complementarity, and limitations across ten open-source applications. II. C APRESE : D ESIGN AND I MPLEMENTATION A. Architecture and Workflow Figure 1 illustrates C APRESE, an automated framework for function-level change impact recommendation in JavaScript. Given a change set of modified functions, source code, tests, and repository history, C APRESE produces a ranked Candidate Inspection Set (CIS) of functions potentially impacted by the changes. It supports three modes: repository-based recommendation from historical co-change patterns, dynamic 1 https://github.com/SEatSFU/caprese

recommendation from execution-derived dependencies, and hybrid recommendation combining both sources of evidence. B. Repository-Based Recommendation The repository-based mode analyzes evolutionary coupling: functions that repeatedly change together may require joint inspection. Unlike transaction-overlap approaches such as TARMAQ [6], C APRESE independently mines recurring functionlevel co-change patterns associated with the input change set. History extraction. C APRESE represents each commit as a transaction of modified functions. It uses RefDiff’s [9] code structure trees (CSTs) to identify changes between consecutive revisions and extends this analysis to map the detected changes to their enclosing functions. Targeted pattern mining. C APRESE retains transactions overlapping the change set and identifies co-occurring functions within them. Starting with single-function patterns, it recursively extends patterns using observed co-changes, prioritizing extensions that may reveal new inspection candidates. Unlike pairwise co-change analysis, this search discovers patterns of varying lengths that overlap with subsets of the change set, without enumerating all patterns across the repository. For a pattern P , support(P ) is the number of transactions containing all its functions. Confidence measures how frequently the complete pattern occurs relative to its change-set functions: support(P ) , (1) confidence(P ) = support(Pc ) where Pc contains the functions of P belonging to the change set. This normalization directs mining toward patterns associated with the current changes, rather than merely frequent patterns in the repository. Patterns meeting a configurable minimum confidence threshold contribute candidates to the CIS. For each candidate, C APRESE retains the confidence and support of its highest-confidence pattern for ranking. C. Dynamic Dependency Recommendation The dynamic mode instruments the application using NodeProf2 and executes its tests to collect runtime traces. These traces are processed into a directed dependency graph whose vertices represent functions, with special handling for file-level execution and classes. Dependency extraction. C APRESE infers dependencies from function calls and callbacks, event registration and emission, timers, and reads and writes to shared state. Edges represent possible directions of change propagation rather than merely call directions. For example, if function f invokes g, C APRESE adds an edge from g to f , since a change to g may affect its caller. Additional edges capture argument passing, event-related interactions, deferred callbacks, and shared-state dependencies. These relationships capture indirect interactions beyond conventional call graphs. For example, a function emitting an event may be related to another function registering its listener, even without a direct invocation between them. 2 https://www.dag.inf.usi.ch/software/nodeprof

1

2

Inputs

Repository-Based Recommendation

(a) Source Code

3

Dynamic Recommendation

Co-change Pattern Mining

Test Execution & Instrumentation

(c) Repository History

Scoring & Ranking

Dynamic Dependency Extraction

(d) Change Set

Ranked CIS

Scoring & Ranking

(b) Tests

4

Hybrid Recommendation Candidate Union Hybrid Scoring & Ranking

Ranked CIS

Ranked CIS

Fig. 1. C APRESE’s workflow: repository-based and dynamic analyses generate ranked CISs, which the hybrid mode combines.

Candidate generation and ranking. Starting from the changed functions, C APRESE traverses the dependency graph in breadth-first order and collects reachable functions as inspection candidates. Candidates are ranked by their shortest dependency distance from the change set. Dependencies not exercised by the available tests remain undetected. D. Hybrid Recommendation The hybrid mode combines candidates from both analyses, prioritizing functions identified by both. Shared candidates are ranked by increasing runtime dependency distance, breaking ties by decreasing historical support and then confidence. Dynamic-only candidates are ranked by dependency distance, while history-only candidates are ranked by support and then confidence. The two exclusive lists are interleaved after the shared candidates, preserving recommendations unique to either analysis. III. T OOL U SAGE AND R EPRODUCIBILITY Implementation and organization. C APRESE implements three components: frequent pattern detection (FPD), dynamic analysis (DA), and hybrid recommendation. FPD extends RefDiff3 by mapping detected source-code changes to their enclosing functions across Git revisions. It implements changeset-guided evolutionary coupling analysis inspired by sequential pattern mining and CLASP [10], with support from SPMF [11], an open-source data mining library. DA uses NodeProf to instrument Node.js applications and processes execution traces to construct a directed dependency graph. The hybrid component ranks candidates using historical and runtime evidence, while the caprese.js CLI coordinates the analyses and produces ranked recommendations. Historical and runtime analyses can be reused across change sets. Installation and configuration. C APRESE requires Node.js, NodeProf, Graal.js,4 Java, Git,5 and Maven;6 the modified RefDiff component is built with Gradle.7 The repository provides installation instructions and a Dockerfile. Users specify 3 https://github.com/aserg-ufmg/RefDiff 4 https://www.graalvm.org/javascript/docs/ 5 https://git-scm.com 6 https://maven.apache.org 7 https://gradle.org

TABLE I B ENCHMARK CHARACTERISTICS AND CONTRIBUTION OF EACH ANALYSIS TO THE HIGHEST- RANKED RELEVANT RECOMMENDATIONS (%). R EPO /DYN : EXCLUSIVE TO EACH COMPONENT; B OTH : IDENTIFIED BY BOTH . C UTOFF K SELECTS THE TOP -K RECOMMENDATIONS IN THE HYBRID RANKING . Project

Top-5

Top-20

Top-60

KLOC

Commits

Txns Repo

Dyn

Both

Repo

Dyn

Both

Repo

Dyn

Both

bignumber.js session jhipster-uml grant environment cla-assistant assemble nock fastify express

48 4 11 11 33 46 26 36 49 19

220 678 868 1,134 1,165 1,177 1,662 1,756 2,682 5,248

111 257 448 567 661 634 926 882 1,282 3,111

5.0 12.0 14.8 43.8 7.8 44.4 72.2 0.0 14.3 35.7

30.0 11.5 24.1 6.2 24.8 5.6 20.4 22.5 0.0 7.1

65.0 76.5 61.1 50.0 67.4 50.0 7.4 77.5 85.7 57.1

3.8 16.1 32.3 52.6 13.6 54.8 68.7 8.0 29.0 45.2

35.2 29.7 39.0 21.9 27.7 18.8 28.9 38.2 20.8 20.3

61.0 54.2 28.7 25.5 58.7 26.4 2.5 53.8 50.2 34.4

20.1 20.9 31.4 53.3 17.1 60.4 64.3 8.8 22.2 46.6

62.1 44.5 49.3 35.0 51.2 19.9 34.2 53.2 49.8 29.6

17.8 34.6 19.3 11.7 31.7 19.7 1.5 38.0 28.0 23.8

Average

–

–

–

25.0

15.2

59.8

32.4

28.0

39.5

34.5

42.9

22.6

the target repository, branch, test directories, excluded paths, starting commit, and data storage location in constants.js. Execution and output. Users invoke node caprese.js mine to prepare historical co-change information and node caprese.js da to collect runtime dependencies through instrumented application execution. They then run node caprese.js detect "..." to generate ranked recommendations for a specified change set: node caprese.js detect "getAssist-main/assist/getAssists.js-12-18"

Changed functions are identified by name, file path, and source-line range; multiple functions can be specified as space-separated entries. Recommendations are written to capreseResult.json for inspection or further analysis. A. Data and Tool Availability C APRESE is available under the MIT license at https: //github.com/SEatSFU/caprese, with a versioned snapshot archived on Zenodo [12]. The repository includes source code, documentation, analysis components, evaluation scripts, and experimental data to support reproducibility and further experimentation. IV. E VALUATION AND F INDINGS A. Evaluation Setup We evaluated C APRESE on ten open-source Node.js applications spanning 4–49 KLOC, 220–5,248 commits, and 111–3,111 function-level change transactions. We randomly selected ten distinct function-level change sets per project (100 total), using all remaining commits as historical data, including commits that postdate the selected changes. Runtime dependencies were collected by instrumenting application code with NodeProf and executing its tests. Table I summarizes the benchmark characteristics. Two investigators, including one non-author, manually established reference impact sets (RISs) using predefined JavaScript-specific guidelines covering compile-time, runtime, transitive, and artifact-mediated dependencies. Their assessments were jointly confirmed, although the RISs may not be exhaustive.

We compare C APRESE’s repository-based, dynamic, and hybrid modes with a function-level reimplementation of TARMAQ [6], a representative evolutionary coupling-based change recommendation technique. Precision@K measures the fraction of the top-K recommendations included in the RIS, for K ∈ {5, 20, 60}. At the same cutoffs, complementarity measures the proportions of relevant recommendations identified exclusively by repository analysis, exclusively by dynamic analysis, or by both. B. Complementarity of Recommendation Signals At Top-60, averaging across projects, 22.6% of relevant recommendations are identified by both analyses, while 34.5% are exclusive to repository analysis and 42.9% to dynamic analysis (Table I). Thus, 77.4% of relevant recommendations within this cutoff are identified by only one analysis, demonstrating substantial complementarity between historical and runtime evidence. The overlap decreases from 59.8% at Top-5 to 39.5% at Top-20 and 22.6% at Top-60. Although higher overlap at smaller cutoffs partly reflects the hybrid ranking’s prioritization of shared candidates, the low overlap at Top-60 shows that both analyses contribute substantial distinct relevant candidates beyond the highest-ranked functions. Contributions also vary across projects. At Top-60, repository-only candidates account for 64.3% in assemble and 53.3% in grant, whereas dynamic-only candidates account for 62.1% in bignumber.js and 53.2% in nock. This variation further supports retaining both sources of evidence. C. Recommendation Precision As shown in Table II, dynamic recommendation achieves the highest mean Precision@5 (0.65) and Precision@20 (0.47), while repository-based recommendation leads at Precision@60 (0.42). The strong performance of dynamic recommendation at smaller cutoffs is consistent with the usefulness of executionderived dependencies and proximity-based ranking. However, the evaluation does not isolate the contributions of individual dependency types.

TABLE II P RECISION @K OF THE RECOMMENDATION APPROACHES AT THREE INSPECTION BUDGETS . Top-5

Project

Top-20

Top-60

TARMAQ

Repo

Dynamic

Hybrid

TARMAQ

Repo

Dynamic

Hybrid

TARMAQ

Repo

Dynamic

Hybrid

bignumber.js session jhipster-uml grant environment cla-assistant assemble nock fastify express

0.39 0.56 0.22 0.42 0.20 0.30 0.40 0.36 0.10 0.38

0.52 0.69 0.33 0.53 0.36 0.36 0.68 0.53 0.33 0.73

0.92 0.89 0.55 0.37 0.67 0.80 0.65 0.82 0.33 0.49

0.78 0.76 0.34 0.51 0.62 0.48 0.71 0.70 0.38 0.44

0.42 0.38 0.14 0.22 0.26 0.21 0.24 0.33 0.09 0.23

0.46 0.60 0.25 0.47 0.36 0.33 0.64 0.52 0.19 0.52

0.88 0.59 0.26 0.39 0.57 0.43 0.46 0.59 0.23 0.29

0.68 0.61 0.26 0.44 0.43 0.37 0.66 0.61 0.20 0.34

0.34 0.30 0.07 0.18 0.28 0.19 0.22 0.30 0.09 0.18

0.49 0.58 0.25 0.46 0.31 0.31 0.64 0.50 0.16 0.45

0.57 0.30 0.12 0.22 0.37 0.24 0.47 0.31 0.19 0.11

0.60 0.37 0.14 0.32 0.36 0.27 0.64 0.31 0.16 0.25

Average

0.33

0.51

0.65

0.57

0.25

0.43

0.47

0.46

0.22

0.42

0.29

0.34

Repository-based recommendation also exceeds TARMAQ’s mean precision at all three cutoffs. Unlike TARMAQ’s maximum transaction-overlap strategy, C APRESE explores recurring multi-function co-change patterns associated with the change set, using confidence-guided mining and ranking. These results demonstrate improved recommendation precision over the reimplemented baseline, without isolating the contributions of individual algorithmic choices. Hybrid recommendation exceeds TARMAQ’s mean precision at every cutoff (0.57 vs. 0.33 at Top-5, 0.46 vs. 0.25 at Top-20, and 0.34 vs. 0.22 at Top-60), but does not consistently outperform the individual modes. Thus, while integrating historical and runtime evidence retains complementary relevant candidates, the current hybrid ranking does not always prioritize them as effectively as the stronger individual mode. D. Practical Insights and Limitations Manual examination identified relevant coupling captured through history but missed by runtime analysis, including duplicated or semantically similar code, dependencies mediated by external resources, shared identifiers, and unexecuted code. Conversely, historical analysis may introduce false positives through incidental co-changes, obsolete relationships, or bidirectional coupling that obscures asymmetric dependencies. Cochange does not necessarily imply impact propagation. These observations help explain the complementary strengths and limitations of the two analyses. Execution cost. On an Apple M1 system, C APRESE generated ranked recommendations in 3.09 seconds on average, excluding dynamic dependency extraction, compared with 2.66 seconds for TARMAQ. Dynamic dependency extraction, including test execution, required 95.47 seconds on average and can be reused across change sets. Threats to validity. The ten applications and selected change sets limit generalizability. Manually constructed RISs may be subjective or incomplete despite predefined guidelines and assessment by two investigators. Using commits that postdate the selected changes may overestimate performance under prospective deployment. The reported differences are descriptive; statistical significance was not assessed.

V. R ELATED W ORK Dependency-based impact analysis. Prior techniques use static dependencies, program slicing, and runtime relationships [1], [2], [13], [14]. Static analysis faces challenges with JavaScript’s dynamic features, while dynamic analysis is limited by execution coverage. Tochal [4] combines static and dynamic analysis for event-driven, DOM-sensitive impact analysis in client-side JavaScript. C APRESE instead targets Node.js and combines runtime dependencies with historical co-changes. Repository-based and hybrid recommendation. ROSE [5] mines evolutionary coupling, while TARMAQ [6] selects transactions with maximal change-set overlap. Other approaches investigate transitive associations [15], adaptive history selection [16], and sequential co-change patterns [17]. Hybrid approaches integrate history with textual, structural, or execution evidence [3], [18], [19]. CoRec [20] learns JavaScript co-change recommendations from recurring patterns and program features. Unlike TARMAQ, C APRESE mines recurring change-set-relevant function-level patterns of varying lengths and integrates them with runtime dependencies. VI. C ONCLUSION We presented C APRESE, an open-source framework integrating function-level co-change pattern mining and runtime dependency analysis for Node.js change impact recommendation. Evaluation on ten applications demonstrates substantial complementarity between historical and runtime evidence, although hybrid ranking does not consistently outperform the individual modes. The released tool and evaluation artifacts support further research on impact recommendation, including improved hybrid ranking, broader runtime dependency coverage, and additional JavaScript ecosystems. ACKNOWLEDGMENTS ChatGPT (OpenAI) was used to assist with reviewing and revising a manuscript originally drafted by the authors. The authors verified the revised content and remain responsible for its accuracy and conclusions. Generative AI was not used to develop C APRESE, conduct experiments, or collect experimental data.

R EFERENCES [1] R. S. Arnold, Software Change Impact Analysis. Washington, DC, USA: IEEE Computer Society Press, 1996. [2] X. Ren, F. Shah, F. Tip, B. G. Ryder, and O. Chesley, “Chianti: A tool for change impact analysis of java programs,” in Proceedings of the 19th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, ser. OOPSLA ’04. New York, NY, USA: Association for Computing Machinery, 2004, p. 432–448. [Online]. Available: https://doi.org/10.1145/1028976.1029012 [3] M. Gethers, B. Dit, H. Kagdi, and D. Poshyvanyk, “Integrated impact analysis for managing software changes,” Proceedings - 2012 34th International Conference on Software Engineering (ICSE), pp. 430–440, 2012. [4] S. Alimadadi, A. Mesbah, and K. Pattabiraman, “Hybrid dom-sensitive change impact analysis for javascript,” Leibniz International Proceedings in Informatics, LIPIcs, vol. 37, pp. 321–345, 2015. [5] T. Zimmermann, P. Weißgerber, S. Diehl, and A. Zeller, “Mining version histories to guide software changes,” IEEE Transactions on Software Engineering, vol. 31, no. 6, pp. 429–445, jun 2005. [6] T. Rolfsnes, S. Di Alesio, R. Behjati, L. Moonen, and D. W. Binkley, “Generalizing the analysis of evolutionary coupling for software change impact analysis,” in 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), vol. 1. IEEE, 2016, pp. 201–212. [7] S. Alimadadi, S. Sequeira, A. Mesbah, and K. Pattabiraman, “Understanding javascript event-based interactions,” in Proceedings of the 36th International Conference on Software Engineering, 2014, pp. 367–377. [8] A. Feldthaus, M. Schäfer, M. Sridharan, J. Dolby, and F. Tip, “Efficient construction of approximate call graphs for javascript ide services,” in Proceedings of the 2013 International Conference on Software Engineering, ser. ICSE ’13. IEEE Press, 2013, p. 752–761. [9] D. Silva and M. T. Valente, “Refdiff: Detecting refactorings in version histories,” in 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR). IEEE, 2017, pp. 269–279. [10] A. Gomariz, M. Campos, R. Marı́n, and B. Goethals, “Clasp: An efficient algorithm for mining frequent closed sequences,” Pacific-Asia Conference on Knowledge Discovery and Data Mining, vol. 7818 LNAI, pp. 50–61, 04 2013.

[11] P. Fournier-Viger, A. Gomariz, T. Gueniche, A. Soltani, C.-W. Wu, V. S. Tseng et al., “Spmf: a java open-source pattern mining library.” J. Mach. Learn. Res., vol. 15, no. 1, pp. 3389–3393, 2014. [12] S. Tavakoli and S. Alimadadi, “Caprese: Change impact recommendation using history and runtime analysis,” 2026. [Online]. Available: https://doi.org/10.5281/zenodo.24120002 [13] J. Law and G. Rothermel, “Incremental dynamic impact analysis for evolving software systems,” in Proceedings of the 14th International Symposium on Software Reliability Engineering, ser. ISSRE ’03. USA: IEEE Computer Society, 2003, p. 430. [14] M. Acharya and B. Robinson, “Practical change impact analysis based on static program slicing for industrial software systems,” in Proceedings of the 33rd International Conference on Software Engineering, ser. ICSE ’11. New York, NY, USA: Association for Computing Machinery, 2011, p. 746–755. [Online]. Available: https://doi.org/10.1145/1985793.1985898 [15] M. A. Islam, M. M. Islam, M. Mondal, B. Roy, C. K. Roy, and K. A. Schneider, “Detecting evolutionary coupling using transitive association rules,” 2018, pp. 113–122. [16] L. Moonen, D. Binkley, and S. Pugh, “On adaptive change recommendation,” Journal of Systems and Software, vol. 164, p. 110550, 2020. [Online]. Available: https://www.sciencedirect.com/ science/article/pii/S0164121220300327 [17] H. Kagdi, S. Yusuf, and J. I. Maletic, “Mining sequences of changedfiles from version histories,” in Proceedings of the 2006 International Workshop on Mining Software Repositories, ser. MSR ’06. New York, NY, USA: Association for Computing Machinery, 2006, p. 47–53. [Online]. Available: https://doi.org/10.1145/1137983.1137996 [18] H. H. Kagdi, M. Gethers, and D. Poshyvanyk, “Integrating conceptual and logical couplings for change impact analysis in software,” Empir. Softw. Eng., vol. 18, no. 5, pp. 933–969, 2013. [Online]. Available: https://doi.org/10.1007/s10664-012-9233-9 [19] M. Zanjani, G. Swartzendruber, and H. Kagdi, “Impact analysis of change requests on source code based on interaction and commit histories,” 11th Working Conference on Mining Software Repositories, MSR 2014 - Proceedings, 05 2014. [20] Z. Jiang, H. Zhong, and N. Meng, “Investigating and recommending co-changed entities for javascript programs,” Journal of Systems and Software, vol. 180, p. 111027, 2021. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0164121221001242

Related documents

• Caprese: Change Impact Recommendation Using History and Runtime Analysis#1648967
• Turning Interaction History into Execution State: A Runtime Layer for Long-Horizon Coding Agents#424062
• DDB: Source-Level Interactive Debugging for Distributed Applications#346481
• X.js: A Live Benchmark for Node.js Taint-Style Vulnerabilities#1318875
• From Reading Code to Reading Spec: A Verified Layer for LLM-Driven Codebase Maintenance#668138