arXiv:2604.22432v1 [cs.SE] 24 Apr 2026
R2Code: A Self-Reflective LLM Framework for Requirements-to-Code Traceability Yifei Wang
Jacky Keung
Xiaoxue Ma
Dept of Computer Science City University of Hong Kong Hong Kong, China [email protected]
Dept of Computer Science City University of Hong Kong Hong Kong, China [email protected]
Dept of Electronic Engineering and Computer Science Hong Kong Metropolitan University Hong Kong, China [email protected]
Zhenyu Mao
Kehui Chen
Yishu Li*
Dept of Computer Science City University of Hong Kong Hong Kong, China [email protected]
Dept of Computer Science City University of Hong Kong Hong Kong, China [email protected]
Dept of Electronic Engineering and Computer Science Hong Kong Metropolitan University Hong Kong, China [email protected]
Abstract—Accurate requirement-to-code traceability is crucial for software maintenance. However, existing IR- and embeddingbased methods are heavily dependent on lexical similarity, often yielding incomplete or inconsistent links across projects and languages and incurring high cost from long-context retrieval and prompting. This paper presents R2Code, an LLM-based semantic traceability framework designed to improve trace link accuracy while reducing inference cost. R2Code integrates three components: 1) a decomposition-enhanced Bidirectional Alignment Network (BAN) that aligns four-layer requirement semantics with corresponding code structures to support cross-level semantic matching; 2) a Self-Reflective Consistency Verification (SRCV) module that conducts explanation-guided consistency checking to calibrate link reliability; and 3) a Dynamic Context-Adaptive Retrieval (DCAR) mechanism that adjusts retrieval granularity and filters contexts using semantic-overlap weighting for efficient context utilization. Experiments on five public datasets spanning multiple domains and two programming languages demonstrate that R2Code consistently outperforms the strongest baselines, achieving an average F1 gain of 7.4%, while reducing token consumption by up to 41.7% through adaptive context control. Index Terms—Requirements traceability, LLMs, Semantic alignment, Self-reflection, Context-adaptive retrieval.
I. I NTRODUCTION Requirements-to-Code Traceability maintains explicit links among heterogeneous software artifacts such as requirements, design documents, source code, and test cases [1]. It links high-level, free-text documentation (e.g., requirements and design documents) to the source code elements that implement those requirements [2]. Such links support change impact analysis and program comprehension, and they influence the effort required to maintain and evolve software systems [3]. In practice, because traceability links are often missing or were not consistently recorded, an analyst may start from a naturallanguage requirements document and retrieve a ranked list of * Corresponding author: Yishu Li.
code elements that are likely to implement it. However, maintaining traceability in evolving codebases remains challenging: manual tracing is labor-intensive and difficult to scale, and prior studies report that traceability maintenance may consume a substantial portion of overall development effort [4], [5]. Traditional information retrieval (IR) techniques, such as TF-IDF, BM25, VSM, LSI [6], and WMD [7], have long been used for automated trace link recovery. These methods compute similarity from lexical overlap or term statistics, enabling efficient retrieval but limiting their ability to capture deeper semantics [8]. Consequently, they often struggle with (i) surface-level matching without semantic understanding [8], (ii) semantic and structural mismatches between high-level requirement descriptions and low-level code implementations [9], and (iii) missing explicit verification of semantic coherence in inferred links, which can lead to false positives. Recent efforts explored large language models (LLMs) for requirements traceability, leveraging their reasoning and code understanding to go beyond keyword matching and infer requirement-to-code links [10]. Existing studies are preliminary, often relying on direct prompting or lightweight retrieval-augmented pipelines without traceability-specific decision mechanisms [11], [12], while emerging work on engineering LLM-based systems also points to the need for more structured, protocol-driven designs [13]. In real-world settings, traceability requires (i) structured alignment between multilayer requirement semantics and code logic, and (ii) costeffective context construction, as naively expanding context increases token cost and noise [14], which may degrade accuracy, efficiency, and robustness across projects and languages. The core challenge of requirement-to-code traceability is to bridge the semantic and structural gap between humanoriented requirement intent and program-level operational behavior, robustly across domains and programming languages, while controlling the cost of context construction for LLM
inference [15]. This motivates a framework that can explicitly align requirement and code semantics and selectively use the most relevant context for decision-making, enabling more reliable and efficient trace decisions in practice. To address these challenges, we present R2Code, an LLMbased framework for requirements-to-code traceability that targets three core objectives: effectiveness, robustness, and efficiency. R2Code bridges requirement–code mismatches via explicit cross-level semantic alignment, improves link reliability through self-reflective consistency checking, and reduces inference cost by constructing compact evidence with contextadaptive retrieval. We evaluate R2Code on five public datasets spanning multiple domains and two programming languages, demonstrating consistent F1 improvements over strong IR, dense retrieval, and RAG-based baselines while substantially reducing token consumption. The main contributions of this paper include: • Framework. We propose an end-to-end evidence-todecision framework for requirement-to-code traceability with three key designs: bidirectional semantic alignment, explanation-guided confidence calibration, and cost-aware adaptive evidence construction. • Empirical study. We conduct an extensive evaluation on five public datasets across multiple domains and two programming languages, comparing against strong IR, dense retrieval, and RAG-based baselines, and demonstrate consistent F1 gains with substantial reductions in token consumption. II. BACKGROUND AND R ELATEDW ORK A. Requirements-to-Code Traceability: Task Overview Requirements-to-code traceability aims to establish and maintain explicit links between natural-language requirement statements and the source code entities (e.g., files, classes, or methods) that implement them [16], [17]. Such requirement– implementation links provide essential support for software maintenance and evolution tasks, including change impact analysis, program comprehension [18]. In practice, however, trace links are frequently missing, outdated, or inconsistent in evolving codebases [19], which motivates automated techniques for trace link recovery [20]. Formally, given a set of requirements R = {r1 , . . . , rm } and a set of code entities C = {c1 , . . . , cn }, the goal is to recover a trace link set L ⊆ R × C that approximates a ground-truth set L∗ [21]. Prior work formulates the task in two modes: ranking, which returns a ranked list of candidate code entities for each requirement, and classification [22], which predicts whether a requirement–code pair constitutes a valid link under a similarity or confidence threshold [23]. Accordingly, evaluation reports Precision, Recall, and F1 for link prediction [24], and may also report ranking-based metrics (e.g., MAP/MRR) when retrieval quality is emphasized [25]. B. Classical IR-based Methods Classical information retrieval (IR) approaches are widely used as baselines for automated trace link recovery [26]. They
typically follow a common pipeline: (i) represent each requirement and each code entity as a textual artifact (e.g., derived from requirement descriptions and code identifiers/comments); (ii) compute a relevance score s(r, c) from term statistics or vector-space similarity; and (iii) rank candidate code entities c ∈ C for a given requirement r, or convert scores into binary links via thresholding [24]. Representative IR baselines in traceability include TF-IDF and Vector Space Model (VSM) variants, as well as BM25style ranking functions [27], which estimate relevance from term-frequency statistics (e.g., TF, IDF, and length normalization) [27]. Latent Semantic Indexing (LSI) projects documents into a low-dimensional latent space via matrix factorization, capturing co-occurrence patterns beyond surface term overlap [28]. Word Mover’s Distance (WMD) and related distancebased measures compute document similarity using distances in a word-embedding space, enabling softer semantic matching than exact term overlap [29]. Early traceability research instantiated the IR paradigm with a range of relevance models. Marcus and Maletic leveraged LSI to recover documentation-to-code links by ranking candidate code artifacts in a latent semantic space [30]. Sundaram et al. evaluated VSM with alternative term-weighting schemes (e.g., TF-IDF/Okapi) alongside LSI across multiple datasets [31]. Oliveto et al. analyzed the overlap and equivalence of candidate links produced by IR techniques [32]. And Hey et al. explored WMD-based semantic distance for traceability link identification [7]. C. Neural Retrieval and LLM-based Methods Beyond classical IR, a large body of work formulates trace link recovery with neural retrieval models that learn semantic representations of requirements and code artifacts [33], [34]. A common approach is the bi-encoder (dualencoder) architecture, which encodes a requirement r and a code entity c independently into dense vectors and computes similarity (e.g., dot product or cosine similarity) for ranking [35]. This design enables efficient nearest-neighbor search over large candidate sets and is often used as a first-stage retriever. In addition, cross-encoder or interaction-based models are sometimes adopted as rerankers: they jointly encode (r, c) and predict a relevance score, typically trading off higher inference cost for more fine-grained matching [36]. Neural retrieval for traceability often leverages pretrained Transformer encoders to represent requirement text and textualized code artifacts [34]. Requirements are encoded as sentences or short documents, while code entities are represented using identifiers, comments, and related textual signals; code-pretrained models can further improve the capture of programming-language semantics [37], [38]. These encoders can be fine-tuned with supervision from known trace links to improve artifact-level similarity estimation [39]. Similar to IR pipelines, neural approaches support both top-k ranked retrieval and threshold-based link prediction under standard evaluation settings [33], [34].
More recently, LLM-based methods have been explored for traceability by leveraging their reasoning capabilities to model requirement-to-code relationships and generate naturallanguage rationales for predicted links [40], [41]. A common pattern is retrieval-assisted inference: a retriever (IR or neural) first selects a small set of candidate code entities, and the LLM then performs link assessment, sometimes with explanation generation or verification-style prompting, to output the final decision [42], [43]. In this way, LLMs serve as a semantic decision module on top of retrieved evidence, complementing retrieval-centric approaches [44]. D. Research Gap Prior work on requirements-to-code trace link recovery offers strong baselines, yet key gaps remain when jointly optimizing accuracy, robustness, and efficiency [11], [45]. (i) Effectiveness: Cross-level mismatch. Classical IR and neural retrieval approaches largely operationalize traceability as text similarity scoring (lexical matching, latent semantic projection, or dense embeddings), which supports ranking but does not explicitly capture the correspondence between highlevel requirement intent and distributed implementation logic in code. (ii) Robustness: Uncalibrated confidence. Retrievalcentric pipelines typically output similarity scores without an explicit mechanism to verify semantic coherence, limiting interpretability and allowing spurious matches to propagate. (iii) Efficiency: Evidence cost. Recent LLM-based studies often rely on direct prompting or lightweight retrieval-augmented pipelines whose outcomes are sensitive to evidence selection and presentation [10], [42]. Static or overly broad context increases inference cost and introduces irrelevant information, while overly narrow context may omit crucial implementation cues [14]. These limitations motivate traceability designs that integrate structured alignment, calibrated confidence, and costaware evidence construction.
semantic representations, and then uses BAN to perform topdown and bottom-up alignment to compute an initial matching score sBAN (r, c). Self-Reflective Consistency Verification (SRCV). To improve reliability, SRCV generates an explanation for a predicted link and verifies its consistency with the requirement. The verification result calibrates the initial score, yielding the final confidence sfinal (r, c) for link decision-making. Dynamic Context-Adaptive Retrieval (DCAR). DCAR governs evidence construction for LLM inference by adapting retrieval granularity and filtering context based on semantic overlap. It aims to provide compact yet informative context to downstream reasoning, improving efficiency without sacrificing link quality. B. Problem Formulation This work considers the requirements-to-code traceability problem between a set of natural-language requirements and a source code corpus. Let R = {r1 , . . . , rm } denote the requirement set, and C = {c1 , . . . , cn } denote the set of code entities, where each c ∈ C can be a file, class, or method. The goal is to recover a ground-truth link set L∗ ⊆ R × C, where (r, c) ∈ L∗ indicates that c (fully or partially) implements r. Given this definition, a traceability approach defines a scoring function s : R × C → [0, 1] and produces predicted links by thresholding:
III. M ETHODOLOGY L̂ = {(r, c) | r ∈ R, c ∈ C, s(r, c) ≥ θ}, A. Overview Fig. 1 summarizes the R2Code workflow. Given a requirement r and a codebase C, R2Code ranks candidate code entities and outputs trace links with calibrated confidence scores. R2Code adopts a two-stage scoring process: it derives structured semantic representations of requirements and code and computes an alignment score sBAN (r, c) via bidirectional semantic alignment (Sec. III-B), then refines it into a calibrated confidence sfinal (r, c) through self-verification (Sec. III-C). R2Code also generates an explanation E(r, c) to support human validation. In parallel, a context-adaptive retrieval mechanism (Sec. III-D) selects compact and relevant evidence for LLM inference, reducing redundant context while preserving salient signals. R2Code comprises three components: Decomposition-Enhanced BAN (HSD+BAN). R2Code first applies HSD to decompose each requirement and summarize each candidate code entity into comparable multi-layer
where θ ∈ [0, 1] is a decision threshold. We model requirements-to-code traceability as a parameterized evidence-to-decision mapping that assigns each requirement–code pair a matching score and an auxiliary explanation for validation. For each pair (r, c), the framework outputs an alignment score sBAN (r, c), a calibrated confidence sfinal (r, c), and an explanation E(r, c) describing the rationale for accepting or rejecting the link. Overall, R2Code can be viewed as a parameterized mapping fΘ : (R, C) → {(r, c, sBAN (r, c), sfinal (r, c), E(r, c)) | r ∈ R, c ∈ C}, where Θ denotes the framework configuration. sfinal (r, c) is designed to approximate the unknown traceability relation encoded in L∗ , while E(r, c) provides interpretable support for downstream verification.
Fig. 1: R2Code Traceability Workflow TABLE I: Requirement–Code Layer Mapping
C. HSD+BAN: Decomposition and Alignment R2Code begins by converting both requirements and code entities into structured semantic representations, so that matching is performed over aligned semantic dimensions rather than raw text alone. Specifically, the framework applies Hierarchical Semantic Decomposition (HSD) to obtain a fourlayer requirement representation and a four-layer code representation, and then uses a Bidirectional Alignment Network (BAN) to compute an initial alignment score sBAN (r, c) via complementary top-down and bottom-up reasoning. a) Requirement semantic decomposition: For each requirement r, HSD decomposes its semantics into four dimensions: intent, actions, conditions, and outputs Sr = (Ir , Ar , Cr , Or ).
(1)
Here, Ir denotes the intent, Ar = {ai } the required actions, Cr = {ci } the conditions and constraints, and Or = {oi } the expected outputs. Given requirement text Tr , we prompt the LLM to produce Sr in a structured JSON format with the corresponding fields (intent, actions, conditions, outputs). We cap each list at 5 concise phrases and set temperature = 0 for deterministic parsing. Example (simplified): For “The system shall log all requests”, the output is: { "intent": ["Maintain an audit trail for incoming requests"],
Requirement layer Intent Ir Actions Ar Conditions Cr Outputs Or
Code layer Function intent Fc Control flow CFc Variable effects V Ec Return states RSc
What it captures goal / responsibility operations / logic steps constraints / state changes expected results / outputs
"actions": ["record request metadata", " persist log entry"], "conditions": ["for every received request" ], "outputs": ["a stored log entry that can be queried later"] }
b) Code semantic decomposition: For each code entity c (with textual view Tc ), HSD constructs a corresponding fourlayer representation Sc = (Fc , CFc , V Ec , RSc ),
(2)
where Fc captures the function’s intent, CFc = {cfi } represents control-flow structures (e.g., branches and loops), V Ec = {vei } denotes variable effects and state changes, and RSc = {rsi } encodes return values or final states. Table I summarizes the four-layer correspondence between requirement and code semantics used in BAN. c) Bidirectional Alignment Network (BAN): BAN estimates the semantic correspondence between Sr and Sc through two complementary directions. Top-down alignment focuses
on requirement satisfaction by assessing whether the code covers the requirement’s intent and expectations: 1 sTD = sim(Ir , Fc ) + sim(Ar , CFc ) 4 (3) + sim(Cr , V Ec ) + sim(Or , RSc ) .
b) Consistency Reflection: SRCV then evaluates whether the explanation is consistent with the original requirement text by scoring:
Bottom-up alignment evaluates implementation fidelity by verifying that the code’s logic aligns with the requirement: 1 sBU = sim(Fc , Ir ) + sim(CFc , Ar ) 4 (4) + sim(V Ec , Cr ) + sim(RSc , Or ) .
To compute Cons(r, E), the backbone LLM is prompted to assess whether the generated explanation E(r, c) is consistent with the original requirement text Tr . The judge must return JSON in the following schema:
sim(x, y) ∈ [0, 1] is operationalized as a normalized semantic alignment score produced by the backbone LLM under a fixed prompt and a constrained output schema. For each semantic layer pair (x, y) (e.g., (Ir , Fc ), (Ar , CFc )), the LLM performs structured joint reasoning to assess whether y semantically supports x, and returns a normalized confidence score. To ensure consistency and interpretability of the alignment process, the output is constrained to a structured JSON format. Concretely, the LLM must return JSON in the following schema:
Cons(r, E) = LLMjudge (r, E) ∈ [0, 1],
{ "consistent": <"yes" | "no">, "score": <float>, "reasons": [<string>, ...] }
We define Cons(r, E) ∈ [0, 1] as the numeric field “score”, which is used for confidence calibration in Eq. (9). As a lightweight auxiliary signal, SRCV can additionally compute a term-level overlap between the requirement and the explanation: Overlap(r, E) =
{ "score": <float>, "coverage": <"covered" | "partially_covered" | "not_covered">, "evidence": [<string>, ...] }
The numeric field “score” is directly used as sim(x, y). The remaining fields are optional and used only for interpretability and error analysis. The final BAN score combines the two directions: sBAN (r, c) = α sTD + (1 − α) sBU ,
(5)
where α ∈ [0, 1] balances requirement satisfaction (top-down verification) and implementation fidelity (bottom-up validation). The resulting sBAN (r, c) serves as the initial alignment score and is passed to the next stage for confidence calibration via self-verification (Section III-D).
(7)
|Terms(r) ∩ Terms(E)| . |Terms(r) ∪ Terms(E)|
(8)
In our framework, Overlap(r, E) is treated only as supportive evidence rather than a primary decision factor. c) Score Adjustment: Finally, SRCV calibrates the BAN score using the consistency signal: sfinal = Adjust sBAN , Cons(r, E) . (9) High consistency slightly strengthens confidence, moderate consistency applies a mild penalty, and low consistency triggers a stronger penalty. Importantly, SRCV is not a separate predictive model; it is a confidence calibration layer that reduces false positives caused by hallucinated connections while preserving links supported by coherent reasoning. E. DCAR: Dynamic Context-Adaptive Retrieval
(6)
DCAR controls evidence construction for LLM inference by adapting retrieval granularity and filtering noisy context on a per-requirement basis. Rather than using a fixed retrieval window for all requirements, DCAR (i) builds compact, reusable summaries for code units, (ii) allocates a requirement-specific retrieval budget, and (iii) retains only the most relevant items as input evidence for downstream reasoning. a) Code Summarization (Cached): To avoid repeatedly sending full code into the LLM, DCAR generates a compact summary for each code unit c (e.g., file/class/function) with text Tc : Summary(c) = (Fc , Dc , CCc , γc ), (10)
where E(r, c) summarizes the claimed semantic correspondence (e.g., which requirement intents are supported by which code behaviors). This step does not aim at making the output “look reasonable”; rather, it externalizes the rationale into a verifiable object for subsequent checking.
where Fc is the function/name signature, Dc is a short semantic description of the core behavior, CCc is the set of invoked/dependent functions (call links), and γc ∈ [0, 1] is an estimated code complexity. Summaries are cached and reused across requirements.
D. SRCV: Self-Reflective Consistency Verification BAN provides an initial semantic alignment score sBAN (r, c) for each requirement–code pair. However, LLMbased alignment may still exhibit over-confident links when the model forms plausible but unsupported associations. SRCV introduces a lightweight self-verification step that uses the model’s own explanation as evidence and calibrates the confidence of sBAN accordingly, producing the final score sfinal . a) Explanation Generation: Given (r, c) and sBAN (r, c), SRCV first asks the LLM to generate an explicit justification: E(r, c) = LLMgen (r, c, sBAN ),
b) Requirement Complexity: Given a requirement r with text Tr , DCAR computes a semantic complexity score:
To address RQ2, we evaluate R2Code on five public datasets spanning different application domains and two programming languages (Java and C#), including four CoEST Java Γ(r) = w1 ϕlen (Tr )+w2 ϕact (Tr )+w3 ϕcond (Tr )+w4 ϕconn (Tr ), datasets [7] and the RETRO.NET dataset for C# [46]. (11) To address RQ3, we record token consumption, inference where ϕlen , ϕact , ϕcond , ϕconn capture requirement length, aclatency, and estimated cost under different retrieval strategies. tion density, conditional indicators, and logical connectors, We compare fixed-window RAG [47] with the proposed respectively. dynamic context-adaptive retrieval (DCAR) to quantify the c) Adaptive Retrieval Window: Based on Γ(r), DCAR cost–accuracy trade-off. sets a retrieval budget kr using a minimal piecewise rule: B. Settings k0 , Γ(r) < τ1 , (12) kr = 2k0 , τ1 ≤ Γ(r) < τ2 , TABLE II: Dataset Statistics 3k0 , Γ(r) ≥ τ2 , where k0 is the base retrieval size and τ1 , τ2 are complexity thresholds. Simpler requirements retrieve a smaller set of function-level summaries, while more complex requirements may expand retrieval to include call-chain-related context via CCc , so that distributed implementations can be covered. d) Semantic Filtering: After retrieval, DCAR further filters candidate contexts using a lightweight semantic-overlap signal computed between the requirement and each candidate’s summary text: Overlap(r, Dci ) =
|Terms(r) ∩ Terms(Dci )| . |Terms(r) ∪ Terms(Dci )|
(13)
Only items above a threshold are retained, with a final cap on context size: Cfinal = {ci | Overlap(r, Dci ) ≥ θov },
|Cfinal | ≤ kmax . (14) The resulting Cfinal constitutes the compact evidence passed to the downstream LLM decision stage. IV. E XPERIMENT A. Research Questions This study evaluates R2Code through three research questions that reflect the core objectives of the framework: effectiveness, robustness, and efficiency. • RQ1 (Effectiveness): How effectively does R2Code improve traceability accuracy compared with baseline methods, and how do its key components contribute to this improvement? • RQ2 (Robustness): Does R2Code maintain stable performance across different project domains and programming languages? • RQ3 (Efficiency): What efficiency gains does R2Code provide in terms of token usage and inference cost? To address RQ1, we evaluate R2Code against traditional IR baselines (BM25, TF-IDF, VSM, LSI, WMD), a dense retriever, and a standard RAG+LLM pipeline on five benchmark datasets [26], [42]. We report precision, recall, and F1-score, and include ranking metrics ( MRR, and precision/recall@k) to assess retrieval quality [24], [25]. We further examine the contributions of key components by comparing R2Code with its ablated variants.
Dataset
Lang.
Description
Req.
Code
Gold Links
iTrust eTour SMOS eANCI RETRO.NET
Java Java Java Java C#
Medical Tourism Sensor Admin. Library
335 316 154 66 66
1,818 1,337 1,220 316 118
1,156 1,162 552 74 301
1) Dataset: To ensure a robust and reproducible evaluation of R2Code across diverse project domains and programming languages, we conduct experiments on five public requirementto-code traceability datasets. As summarized in Table II, these datasets cover medical systems, tourism platforms, sensor networks, and public administration, and span two programming languages (Java and C#). Each dataset contains naturallanguage requirements, the corresponding source code files, and manually curated ground-truth traceability links. 2) Model: R2Code is implemented with DeepSeek-V3.1Terminus as the backbone LLM for hierarchical decomposition, bidirectional alignment, and consistency verification. We adopt DeepSeek-V3.1-Terminus because DeepSeek’s JSON Output allows us to enforce schema-constrained structured decomposition and normalized alignment scoring, enabling reliable parsing and downstream computation [48]. For the baseline RAG pipeline, dense retrieval is implemented using sentence-transformers/all-mpnet-base-v2. Classical IR baselines include BM25, TF-IDF, VSM, LSI, and WMD, each configured with standard parameter settings. 3) Environment and Parameters: All experiments are conducted on an NVIDIA A100 server using Python 3.9 and PyTorch 2.0, with the random seed fixed to 42. The backbone LLM uses temperature 0.0, top-p 0.95, and max tokens 2048. IR baselines follow commonly used configurations in standard toolkits. Within R2Code, BAN and SRCV use a fixed configuration throughout all experiments. For DCAR, we set kbase = 5 and cap dynamic expansion with kmax = 10 to bound retrieval context and cost. 4) Evaluation Metrics: We evaluate traceability performance from three perspectives: (i) classification quality, reported with Precision, Recall, and F1-score on predicted requirement–code links; (ii) ranking quality, assessed by MRR along with Precision@k and Recall@k (k ∈ {5, 10}) to reflect the quality of top-ranked candidates; and (iii) efficiency, measured by total input/output token consumption, the number
of LLM requests, and end-to-end runtime, with inference cost estimated based on per-token pricing for input and output tokens. All results are averaged over requirements, and we report standard deviations where appropriate. V. R ESULTS In this section, we present experimental results organized by the research questions in Section IV-A. We first report the accuracy of R2Code on the primary benchmark setting (iTrust) and analyze the contribution of key components via ablation (RQ1). We then examine robustness by evaluating the same configuration across multiple datasets spanning different application domains and programming languages (RQ2). Finally, we analyze efficiency in terms of token consumption, runtime, and estimated cost, and quantify the cost–accuracy trade-off under different retrieval strategies (RQ3).
the full configuration achieves the best balance among precision, recall, and F1 on iTrust. Fig. 2 reports BAN’s layer-wise alignment scores on iTrust across four semantic layers. The Intent layer achieves the highest alignment (0.814 in the combined setting), suggesting that high-level goal matching provides a strong traceability signal. The alignment decreases for more fine-grained layers, with Actions (0.781), Outputs (0.803), and Conditions (0.747), indicating increased semantic variability when matching detailed requirement semantics to code. In addition, the topdown (R→Code) direction consistently yields higher alignment scores than bottom-up (Code→R), while the combined strategy offers a more balanced signal across layers.
A. Answer to RQ1: Effectiveness TABLE III: Overall Performance on the iTrust Dataset Method BM25 TF-IDF VSM LSI WMD RAG-LLM R2Code
F1
Recall
Prec.
MRR
P@10
R@10
0.6158 0.6068 0.5928 0.5684 0.5910 0.6949 0.7296
0.6084 0.6014 0.5876 0.5623 0.5944 0.6783 0.7050
0.6234 0.6123 0.5981 0.5746 0.5876 0.7123 0.7560
0.7234 0.7089 0.6924 0.6689 0.6945 0.8123 0.8550
0.5845 0.5712 0.5568 0.5321 0.5523 0.6823 0.7720
0.5234 0.5123 0.4987 0.4715 0.3894 0.6123 0.7060
Fig. 2: Layer-wise alignment scores of BAN on iTrust To answer RQ1, we compare R2Code with classical IR baselines and a standard RAG+LLM pipeline on iTrust. As shown in Table III, R2Code achieves the best overall performance, reaching an F1-score of 0.7296 with 0.7560 precision and 0.7050 recall. Compared with the strongest baseline RAG+LLM (F1=0.6949), R2Code improves F1 by +0.0341, while also achieving higher ranking quality (MRR: 0.8550 vs. 0.8123). Under practical top-k cutoffs, R2Code yields the highest P@10 (0.7720) and R@10 (0.7060), indicating more correct links can be surfaced earlier in the ranked list. TABLE IV: Ablation Study on the iTrust Dataset Ablation
BAN
SRCV
DCAR
F1
Prec.
Recall
Full – BAN – SRCV – DCAR
✓ × ✓ ✓
✓ ✓ × ✓
✓ ✓ ✓ ×
0.7296 0.6949 0.6864 0.7112
0.7560 0.7123 0.7873 0.8117
0.7050 0.6783 0.6084 0.6329
BAN only SRCV only DCAR only
✓ × ×
× ✓ ×
× × ✓
0.6859 0.6854 0.6930
0.7600 0.7234 0.8122
0.6250 0.6512 0.6040
Note. ✓ indicates the component is enabled; × indicates it is removed.
Table IV reports the ablation results of R2Code on iTrust. Removing BAN decreases the F1-score from 0.7296 to 0.6949. Disabling SRCV results in a larger drop (F1=0.6864), accompanied by a reduction in recall to 0.6084. When DCAR is removed, the F1-score remains close to the full model (0.7112), while recall drops from 0.7050 to 0.6329. Overall,
* Alignment score is the mean LLM-derived sim(x, y) per layer pair over evaluated requirement–code pairs in iTrust.
Fig. 3 compares the SRCV consistency score distributions of positive and negative requirement–code pairs on iTrust. The positive pairs exhibit substantially higher consistency scores, while negative pairs are concentrated in the low-score region, indicating that SRCV provides a discriminative validation signal. Using the predefined threshold θconsistency = 0.7, most negative candidates can be suppressed while preserving highconsistency positive links, supporting the effectiveness of SRCV for filtering spurious matches. B. Answer to RQ2: Robustness TABLE V: Cross-Dataset Performance Consistency Dataset
Best Baseline (F1)
R2Code (F1)
Gain
iTrust eTour SMOS eANCI RETRO
0.6949 0.6812 0.6663 0.7058 0.6580
0.7296 0.7342 0.7201 0.7592 0.7134
+5.0% +7.8% +8.1% +7.6% +8.4%
Average Std. Dev.
0.6812 0.0176
0.7313 0.0157
+7.4% –
To answer RQ2, Table V reports the cross-dataset performance of R2Code under a fixed configuration, evaluating its robustness across five traceability datasets with diverse characteristics. For each dataset, we compare R2Code against
expand to 23.4 candidate files. This pattern shows that DCAR increases retrieval context for more complex requirements while keeping the number of files included in the final prompt bounded by kmax=10.
Fig. 3: Discriminative consistency validation of SRCV * SRCV consistency score distributions for positive (ground-truth links) and negative (non-links) requirement–code pairs on iTrust (n = 500 each). The dashed line indicates the filtering threshold θconsistency = 0.7.
the strongest-performing baseline measured by F1-score. As shown in the table, R2Code consistently outperforms the best baseline on all datasets, achieving relative F1-score gains ranging from 5.0% to 8.4%. The improvements are stable across datasets, with an average gain of 7.4% and comparable standard deviations between the baseline and R2Code results, indicating that the proposed approach maintains consistent effectiveness without dataset-specific tuning. C. Answer to RQ3: Efficiency Table VI summarizes the efficiency and cost results on the iTrust dataset. Traditional IR baselines (e.g., BM25 and TF-IDF) incur zero LLM API cost, but achieve substantially lower F1-scores. Compared with the RAG+LLM baseline, R2Code reduces total token consumption from 3.74M to 2.18M (−41.7%), including a 48.0% reduction in input tokens and a 21.6% reduction in output tokens. This results in a lower estimated inference cost ($463.16 to $285.79, −38.3%) and a shorter end-to-end runtime (1,234.5s to 987.3s, −20.0%), while improving F1 from 0.6949 to 0.7296. Table VI further reports the cost per correct traceability link. R2Code achieves $0.2472 per correct link with 1,156 true positives, whereas RAG+LLM requires $0.4027 per correct link for 1,150 true positives. These results indicate that R2Code delivers higher traceability accuracy with lower perlink inference cost under the same evaluation setting. Table VII summarizes token efficiency under fixed-window retrieval and DCAR on the iTrust dataset for the retrieval construction stage. DCAR reduces the average tokens per requirement–code pair from 4,850 to 3,054, corresponding to an approximate 37% reduction. In terms of overall usage, total token consumption decreases from 2,483,200 to 1,563,216, indicating that adaptive context selection can substantially lower token overhead under the same evaluation setting. Table VII also reports how DCAR adjusts retrieval scope based on requirement complexity, measured as the number of retrieved candidate files. Low-complexity requirements retrieve an average of 8.3 candidate files, medium complexity retrieves 15.7 candidate files, and high-complexity requirements
VI. D ISCUSSION A. Finding 1: Effectiveness R2Code’s effectiveness in RQ1 is best understood not as “more retrieval” but as better decision-making under noisy candidates: structured semantic alignment helps separate true implementation evidence from lexical look-alikes, making the top-ranked links more reliable for maintenance use. The ablation results reveal distinct and complementary roles. Removing BAN largely erases the advantage, indicating that cross-layer alignment is the primary source of improvement rather than incidental prompt effects. Notably, removing SRCV can increase precision while substantially hurting recall. This suggests SRCV does more than filter false positives, it calibrates confidence so the system does not become overly conservative and miss valid links when evidence is partial or uneven. DCAR mainly improves evidence coverage. Removing it increases precision but reduces recall, indicating that dynamic evidence construction retrieves additional true links while also introducing harder, borderline candidates that BAN and SRCV help resolve. B. Finding 2: Robustness R2Code shows robust improvements across heterogeneous project structures and transfers to the C# RETRO dataset under the same fixed configuration. This robustness is consistent with decomposition-based alignment: by matching requirements and code at corresponding semantic layers, the framework is less dependent on shared vocabulary that may shift across domains or languages. In addition, SRCV contributes a domainagnostic calibration signal. Fig. 3 shows that consistency scores separate positive from negative pairs and that a single threshold (θ = 0.7) suppresses most negative candidates while preserving high-consistency positives. A remaining boundary case involves highly scattered implementations, where limited context budgets may miss intermediate evidence, suggesting more adaptive evidence expansion. C. Finding 3: Efficiency R2Code improves efficiency not simply by “using fewer tokens,” but by making the evidence budget more selective and task-aware. Compared with a fixed-window RAG+LLM pipeline, it reduces end-to-end runtime and estimated inference cost while still improving F1, indicating that traceability benefits more from compact, high-relevance evidence than from indiscriminately expanding context. Table VII shows that DCAR expands retrieval scope in a structured way by allocating more candidate files to higher-complexity requirements (about 8.3 vs. 23.4 on average), while keeping the final prompt bounded by (kmax=10). This suggests a practical scaling implication: efficiency gains come from removing irrelevant evidence and reserving broader context only for requirements that empirically need it.
TABLE VI: Efficiency and cost summary on the iTrust dataset Model BM25 TF-IDF WMD RAG+LLM R2Code
F1
Input Tokens
Output Tokens
Runtime (s)
Cost (USD)
True Positives
Cost/TP (USD)
0.6158 0.6068 0.5910 0.6949 0.7296
0 0 0 2,847,234 1,479,935
0 0 0 892,156 699,521
0.2 0.5 1.8 1,234.5 987.3
0.00 0.00 0.00 463.16 285.79
704 691 680 1,150 1,156
0.0000 0.0000 0.0000 0.4027 0.2472
TABLE VII: Retrieval-stage efficiency: Fixed RAG vs. DCAR on iTrust Metric
Fixed RAG
DCAR
Tokens per pair Total tokens
4,850 2,483,200
3,054 1,563,216
– – –
8.3 15.7 23.4
Avg. retrieved files (Low complexity) Avg. retrieved files (Medium complexity) Avg. retrieved files (High complexity)
Note. Tokens are counted for retrieval construction only (excluding BAN/SRCV). “Retrieved files” denote pre-cap candidates (code entities).
VII. T HREATS TO VALIDITY Internal validity. The reported results may be influenced by implementation and configuration choices, such as the selected LLM backbone and the prompting protocol, which can affect the precision–recall–cost trade-off in LLM-assisted pipelines. To mitigate this threat, we fix the experimental setup across all datasets (same backbone, prompting protocol, and budgets) with deterministic decoding (e.g., temperature = 0) to reduce run-to-run variance, and we adopt a consistent setting based on a small pilot sensitivity analysis of prompt and configuration variants. In addition, errors in LLM-generated intermediate representations may propagate to alignment and calibration, particularly for ambiguous requirements or highly dispersed implementations. External validity. The evaluation is conducted on public datasets with varying artifact granularity and link sparsity, which may influence absolute effectiveness. Cost estimates are derived from token usage and runtime measured in a specific environment, and may differ under alternative infrastructures or pricing schemes. In evolving repositories, cached code summaries may also become stale, motivating incremental update strategies for continuous integration settings. VIII. C ONCLUSION In this paper, we proposed R2Code for requirements-tocode traceability, aiming to improve effectiveness, enhance efficiency, and maintain robust performance across heterogeneous projects. R2Code combines structured semantic decomposition, bidirectional alignment, confidence calibration, and context-adaptive evidence construction to produce reliable trace links under limited context budgets. Experiments on five public datasets (iTrust, eTour, SMOS, eANCI, and RETRO.NET) show that R2Code consistently outperforms classical IR baselines and LLM-based retrieval pipelines,
achieving an average relative F1 improvement of 7.4% and up to 14.1% on individual datasets. Meanwhile, R2Code reduces total token consumption by up to 41.7%, and improves the cost per correct link by up to 37.0%, demonstrating that better traceability quality can be obtained with substantially lower inference overhead. Future work will extend R2Code to larger-scale industrial repositories and additional software artifacts, such as design documents, issue discussions, and commits, to further evaluate its applicability in more realistic development settings. We also plan to strengthen robustness in continuously evolving codebases by introducing incremental summary refresh and more adaptive evidence expansion for highly scattered implementations. In addition, we will explore more controllable decomposition and verification strategies to improve the reliability and interpretability of trace decisions. Finally, we aim to investigate multilingual traceability settings and broader crossproject scenarios to further enhance the generalization and practical adoption of the proposed framework. ACKNOWLEDGMENT The work described in this paper is substantially supported by Hong Kong Metropolitan University Research Grant (Project Reference No. RD/2025/1.21 and No. RD/2025/1.24), and partially supported by the grant under the Research Grant Council (Project Reference No. UGC/FDS16/E25/25). R EFERENCES [1] B. Ramesh, C. Stubbs, T. Powers, and M. Edwards, “Requirements traceability: Theory and practice,” Annals of software engineering, vol. 3, no. 1, pp. 397–415, 1997. [2] G. Antoniol, G. Canfora, G. Casazza, A. De Lucia, and E. Merlo, “Recovering traceability links between code and documentation: a retrospective,” IEEE Transactions on Software Engineering, 2025. [3] A. Ghabi and A. Egyed, “Code patterns for automatically validating requirements-to-code traces,” in Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering, 2012, pp. 200–209. [4] B. Ramesh, “Factors influencing requirements traceability practice,” Communications of the ACM, vol. 41, no. 12, pp. 37–44, 1998. [5] F. Tian, T. Wang, P. Liang, C. Wang, A. A. Khan, and M. A. Babar, “The impact of traceability on software maintenance and evolution: A mapping study,” Journal of Software: Evolution and Process, vol. 33, no. 10, p. e2374, 2021. [6] Y. Lyu, H. Cho, P. Jung, and S. Lee, “A systematic literature review of issue-based requirement traceability,” Ieee Access, vol. 11, pp. 13 334– 13 348, 2023. [7] T. Hey, F. Chen, S. Weigelt, and W. F. Tichy, “Improving traceability link recovery using fine-grained requirements-to-code relations,” in 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 2021, pp. 12–22.
[8] J. L. Guo, J.-P. Steghöfer, A. Vogelsang, and J. Cleland-Huang, “Natural language processing for requirements traceability,” in Handbook on Natural Language Processing for Requirements Engineering. Springer, 2025, pp. 89–116. [9] T. Merten, D. Krämer, B. Mager, P. Schell, S. Bürsner, and B. Paech, “Do information retrieval algorithms for automated traceability perform effectively on issue tracking system data?” in International Working Conference on Requirements Engineering: Foundation for Software Quality. Springer, 2016, pp. 45–62. [10] M. North, A. Atapour-Abarghouei, and N. Bencomo, “Code gradients: Towards automated traceability of llm-generated code,” in 2024 IEEE 32nd International Requirements Engineering Conference (RE). IEEE, 2024, pp. 321–329. [11] E. Alor, S. Khatoonabadi, and E. Shihab, “Evaluating the use of llms for documentation to code traceability,” arXiv preprint arXiv:2506.16440, 2025. [12] M. A. Zadenoori, J. Dabrowski, W. Alhoshan, L. Zhao, and A. Ferrari, “Large language models (llms) for requirements engineering (re): A systematic literature review,” arXiv preprint arXiv:2509.11446, 2025. [13] Z. Mao, J. Keung, F. Zhang, S. Liu, Y. Wang, and J. Li, “Towards engineering multi-agent llms: A protocol-driven approach,” arXiv preprint arXiv:2510.12120, 2025. [14] Y. Li, J. Keung, Z. Yang, X. Ma, J. Zhang, and S. Liu, “Simac: simulating agile collaboration to generate acceptance criteria in user story elaboration,” Automated Software Engineering, vol. 31, no. 2, p. 55, 2024. [15] B. Wang, Z. Zou, X. Liang, H. Jin, and P. Liang, “Hgnnlink: recovering requirements-code traceability links with text and dependency-aware heterogeneous graph neural networks,” Automated Software Engineering, vol. 32, no. 2, p. 55, 2025. [16] J. Cleland-Huang, O. C. Gotel, J. Huffman Hayes, P. Mäder, and A. Zisman, “Software traceability: trends and future directions,” in Future of software engineering proceedings. ACM, 2014, pp. 55–69. [17] B. Ramesh and M. Jarke, “Toward reference models for requirements traceability,” IEEE transactions on software engineering, vol. 27, no. 1, pp. 58–93, 2002. [18] N. Niu, W. Wang, and A. Gupta, “Gray links in the use of requirements traceability,” in Proceedings of the 2016 24th ACM SIGSOFT international symposium on foundations of software engineering, 2016, pp. 384–395. [19] R. Rasiman, F. Dalpiaz, and S. España, “How effective is automated trace link recovery in model-driven development?” in International Working Conference on Requirements Engineering: Foundation for Software Quality. Springer, 2022, pp. 35–51. [20] T. W. W. Aung, H. Huo, and Y. Sui, “A literature review of automatic traceability links recovery for software change impact analysis,” in Proceedings of the 28th International Conference on Program Comprehension, 2020, pp. 14–24. [21] Y. Zhang, C. Wan, and B. Jin, “An empirical study on recovering requirement-to-code links,” in 2016 17th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD). IEEE, 2016, pp. 121–126. [22] B. Wang, H. Wang, R. Luo, S. Zhang, and Q. Zhu, “A systematic mapping study of information retrieval approaches applied to requirements trace recovery.” in SEKE, 2022, pp. 1–6. [23] T. Hey, J. Keim, and S. Corallo, “Requirements classification for traceability link recovery,” in 2024 IEEE 32nd International Requirements Engineering Conference (RE). IEEE, 2024, pp. 155–167. [24] J. H. Hayes, A. Dekhtyar, and J. Osborne, “Improving requirements tracing via information retrieval,” in Proceedings. 11th IEEE International Requirements Engineering Conference, 2003. IEEE, 2003, pp. 138–147. [25] C. Zhang, Y. Wang, Z. Wei, Y. Xu, J. Wang, H. Li, and R. Ji, “Ealink: An efficient and accurate pre-trained framework for issue-commit link recovery,” in 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2023, pp. 217–229. [26] M. Borg, P. Runeson, and A. Ardö, “Recovering from a decade: a systematic mapping of information retrieval approaches to software traceability,” Empirical Software Engineering, vol. 19, no. 6, pp. 1565– 1616, 2014. [27] S. Robertson, H. Zaragoza et al., “The probabilistic relevance framework: Bm25 and beyond,” Foundations and Trends® in Information Retrieval, vol. 3, no. 4, pp. 333–389, 2009.
[28] S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman, “Indexing by latent semantic analysis,” Journal of the American society for information science, vol. 41, no. 6, pp. 391–407, 1990. [29] M. Kusner, Y. Sun, N. Kolkin, and K. Weinberger, “From word embeddings to document distances,” in Proceedings of the 32nd International Conference on Machine Learning (ICML). PMLR, 2015, pp. 957–966. [Online]. Available: https://proceedings.mlr.press/ v37/kusnerb15.html [30] A. Marcus and J. I. Maletic, “Recovering documentation-to-source-code traceability links using latent semantic indexing,” in 25th International Conference on Software Engineering, 2003. Proceedings. IEEE, 2003, pp. 125–135. [31] S. K. Sundaram, J. H. Hayes, A. Dekhtyar, and E. A. Holbrook, “Assessing traceability of software engineering artifacts,” Requirements engineering, vol. 15, no. 3, pp. 313–335, 2010. [32] R. Oliveto, M. Gethers, D. Poshyvanyk, and A. De Lucia, “On the equivalence of information retrieval methods for automated traceability link recovery,” in 2010 IEEE 18th International Conference on Program Comprehension. IEEE, 2010, pp. 68–71. [33] J. Guo, J. Cheng, and J. Cleland-Huang, “Semantically enhanced software traceability using deep learning techniques,” in 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE). IEEE, 2017, pp. 3–14. [34] J. Lin, Y. Liu, Q. Zeng, M. Jiang, and J. Cleland-Huang, “Traceability transformed: Generating more accurate links with pre-trained bert models,” in 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). IEEE, 2021, pp. 324–335. [35] N. Reimers and I. Gurevych, “Sentence-bert: Sentence embeddings using siamese bert-networks,” arXiv preprint arXiv:1908.10084, 2019. [36] J. Leonhardt, H. Müller, K. Rudra, M. Khosla, A. Anand, and A. Anand, “Efficient neural ranking using forward indexes and lightweight encoders,” ACM Transactions on Information Systems, vol. 42, no. 5, pp. 1–34, 2024. [37] Z. Feng, “Codebert: A pre-trained model for program-ming and natural languages,” arXiv preprint arXiv:2002.08155, 2020. [38] D. Guo, S. Lu, N. Duan, Y. Wang, M. Zhou, and J. Yin, “Unixcoder: Unified cross-modal pre-training for code representation,” arXiv preprint arXiv:2203.03850, 2022. [39] J. Tian, L. Zhang, and X. Lian, “A cross-level requirement trace link update model based on bidirectional encoder representations from transformers,” Mathematics, vol. 11, no. 3, p. 623, 2023. [40] D. Fuchß, T. Hey, J. Keim, H. Liu, N. Ewald, T. Thirolf, and A. Koziolek, “Lissa: toward generic traceability link recovery through retrieval-augmented generation,” in Proceedings of the IEEE/ACM 47th International Conference on Software Engineering. ICSE, vol. 25, 2025. [41] Y. Wang, J. Keung, Z. Mao, J. Zhang, and Y. Cao, “Chart2code-mola: Efficient multi-modal code generation via adaptive expert routing,” arXiv preprint arXiv:2511.23321, 2025. [42] T. Hey, D. Fuchß, J. Keim, and A. Koziolek, “Requirements traceability link recovery via retrieval-augmented generation,” in International Working Conference on Requirements Engineering: Foundation for Software Quality. Springer, 2025, pp. 381–397. [43] S. J. Ali, V. Naganathan, and D. Bork, “Establishing traceability between natural language requirements and software artifacts by combining rag and llms,” in International Conference on Conceptual Modeling. Springer, 2024, pp. 295–314. [44] Y. Li, J. Keung, X. Ma, C. Y. Chong, J. Zhang, and Y. Liao, “Llmbased class diagram derivation from user stories with chain-of-thought promptings,” in 2024 IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC). IEEE, 2024, pp. 45–50. [45] Z. Mao, J. Keung, Y. Sun, Y. Wang, S. Liu, and J. Li, “Towards requirements engineering for genai-enabled software: Bridging responsibility gaps through human oversight requirements,” arXiv preprint arXiv:2511.13069, 2025. [46] J. H. Hayes, A. Dekhtyar, and J. Payne, “The requirements tracing on target (retro). net dataset,” in 2018 IEEE 26th International Requirements Engineering Conference (RE). IEEE, 2018, pp. 424–427. [47] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel et al., “Retrievalaugmented generation for knowledge-intensive nlp tasks,” Advances in neural information processing systems, vol. 33, pp. 9459–9474, 2020. [48] DeepSeek, “Deepseek api guide: Json output,” https://api-docs.deepseek. com/guides/json_mode, accessed: 2026-02-05.