Conceptio › Archive › arXiv CS
arXiv CS open access

An Empirical Study on the Impact of Normalized Use-Case Specifications on Traceability

2026 · arxiv_cs
arXiv CS · Papers · 2026
Open Source ↗Direct PDF ↓
software-architecturesoftware-engineeringtesting
Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

An Empirical Study on the Impact of Normalized Use-Case Specifications on Traceability Luoyuan Shi∗† , Yuanzhao Zhai∗† , Dawei Feng∗† , Jialin Zhao∗† , Zhaoxie Xu∗† , Bo Ding∗† , Huaimin Wang∗† ∗ College of Computer Science and Technology, National University of Defense Technology, Changsha, China † State Key Laboratory of Complex & Critical Software Environment, Changsha, China

arXiv:2608.15726v1 [cs.SE] 16 Aug 2026

shiluoyuan1995@126.com

Abstract—Traceability link recovery between requirements and code is critical for software quality assurance and evolution analysis. Although traceability technologies have made considerable progress, there exists a huge semantic gap between ambiguous natural-language requirements and precise source code, which constitutes the fundamental obstacle to recovering traceability links between requirements and code. Most existing approaches focus on improving traceability techniques for requirements and code while ignoring the quality of requirement descriptions themselves. Such technology-oriented optimization that overlooks source-data quality makes it difficult to fundamentally bridge the semantic gap. To address this issue, this study proposes to start from requirements: adopting controlled-natural-language specifications to unify text structures and filter noise, and leveraging large language models to complement implicit domain context. This enhances the discriminability of semantic representations of requirements in vector space and narrows the semantic gap. We first select four public datasets and employ large language models together with prompt engineering to decompose requirements and transform them into normalized use-case specifications. We then systematically evaluate the impact of normalized use-case specifications on the performance of requirement-to-code traceability link recovery using two representative frameworks. The empirical results demonstrate that normalized use-case specifications can effectively reduce the semantic gap for original requirements with semantic ambiguity and yield consistent performance gains across different traceability frameworks. Meanwhile, this study identifies the applicability boundary of the proposed method: for high-quality requirements whose vocabulary already highly aligns with underlying source code, over-normalization may dilute feature words and degrade performance. Therefore, requirement quality assessment is recommended prior to practical deployment. This research verifies that requirement normalization at the source is an effective strategy to bridge the semantic gap and boost traceability performance, offering new perspectives for traceability link-recovery research. Index Terms—Traceability Link Recovery; Requirements Engineering; Normalized Use-Cases; Large Language Models

I. I NTRODUCTION Traceability between software requirements and code is a core capability for guaranteeing software quality, supporting change-impact analysis and compliance verification [1]. In modern complex software systems, effective requirement traceability not only ensures consistency between system implementations and user intentions but also significantly reduces maintenance costs. Studies show that each additional untraced requirement item raises the average software defect rate by approximately 2.3%. Nevertheless, with the explosive growth of

software scale, manually maintaining traceability links can no longer satisfy high-frequency, low-latency delivery demands under agile iteration. Accordingly, efficient Traceability Link Recovery (TLR) techniques have become a shared goal for both academia and industry. Currently, TLR tasks face a core bottleneck: the semantic gap between ambiguous requirements and precise code. Specifically, original requirements focus on high-level descriptions of business logic, whereas source code targets concrete functional implementations, bringing substantial asymmetry in abstraction levels and vocabulary usage. For instance, a simple requirement such as “payment function” may correspond to multiple complex code logics involving encryption, interfaces and databases. Such one-to-many mappings greatly increase the difficulty of link recovery. Existing research mostly concentrates on traceability-technique optimization, striving to improve requirement-to-code traceability-recovery accuracy via sophisticated deep-learning models or Retrieval-Augmented Generation (RAG). However, such technology-centric optimization that disregards source-data quality cannot fundamentally bridge the semantic gap. Taking the iTrust dataset as an example, when tracing requirement UC10E1 using information-retrieval-based traceability, AddPatientValidator.java contains abundant frequent domain terms such as patient, validate, and invalid that resemble UC10E1. Despite semantic misalignment, lexical co-occurrence leads to false-positive outputs. In contrast, GetUserNameAction.java is semantically compatible yet lacks surface-level lexical overlap with the requirement (e.g., identifier inputMID), and is consequently missed by the system without auxiliary information. Motivated by the above observations, this study aims to narrow the semantic gap starting from the requirement side. Controlled Natural Language (CNL) delivers prominent advantages for processing unstandardized texts owing to its predefined structured templates and constrained vocabulary. By mapping unstructured natural-language texts into uniformly formatted, precise descriptions with rich semantic information, CNL eliminates lexical ambiguity and improves requirement consistency. It provides high-signal-to-noise-ratio inputs for Traceability Link Recovery (TLR), and its normalized features also improve automation efficiency as well as the accuracy of context-semantic understanding by Large Language Mod-

els (LLMs). In this work, four public datasets are adopted. LLMs are utilized to reconstruct original requirements into use-case specifications conforming to verbose-style Controlled Natural Language (CNL-B). Afterwards, we evaluate how normalized use-cases affect traceability performance under two frameworks: the information-retrieval-based Fine-grained Traceability Link Recovery (FTLR) [2], and the RAG-based Linking Software System Artifacts (LISSA) [3]. Our main contributions are summarized as follows: 1) Benchmark dataset construction: We build and release a high-quality triple dataset consisting of original requirements, normalized use-cases and source code to facilitate future research. 2) Automated use-case-specification generation framework: We propose an LLM-driven automated generation approach for CNL-B-compliant normalized use-case specifications, covering requirement decomposition, structured generation and quality validation. 3) Empirical findings: We quantitatively verify the benefits of normalized use-case specifications in narrowing semantic gaps and boosting TLR performance (especially F1-score). 4) Best practices for specification generation: To address coarse-grained and mixed-function original requirements that hinder precise traceability, we introduce a requirement-decomposition mechanism. Two decomposition strategies (business-logic-oriented versus technical-layer-oriented) are compared regarding specification generation and traceability performance. The results indicate that business-logic-oriented decomposition preserves complete business logic while producing fewer normalized use-case specifications. It reduces computational overhead and benefits both retrieval matching and human comprehension. The remainder of this paper is organized as follows. Section 2 reviews related work on requirement-to-code traceability link recovery. Section 3 elaborates our research methodology, including dataset collection, processing and quality assurance. Section 4 validates the effectiveness of normalized use-case specifications via comparative experiments. Section 5 analyzes factors threatening validity. Section 6 provides discussions. Section 7 concludes this paper. II. R ELATED W ORK The domains relevant to this study mainly include requirement-to-code traceability link recovery and standardized representation approaches for use-case specifications. A. Traceability Link Recovery (TLR) TLR aims to identify correspondences between requirements and software artifacts such as source code. Effective requirement traceability is essential for software quality. As mentioned earlier, missing traceability links directly lead to a notable rise in software defect rates [4]. Traceability link recovery techniques can be broadly divided into traditional methods and learning-based methods.

Traditional methods are primarily built upon information retrieval techniques. Candidate links are identified by computing textual similarity across different software artifacts, under the core assumption that semantic similarity among artifacts positively correlates with traceability [1]. Early studies widely adopted techniques including the Vector Space Model (VSM) and Latent Semantic Indexing (LSI) [5] to detect candidate links. To mitigate the prevalent lexical mismatch between requirements and code, researchers have extended basic IR models. For example, Dı́az et al. proposed TYRION, which alleviates lexical mismatch by combining code author information with information retrieval [6]. Hayes et al. improved the vector-space model with external synonym lexicons to boost recall and precision on the NASA MODIS dataset [7]. De Lucia et al. readjusted term weights by introducing smoothing filters to remove “common noise” within homogeneous software artifacts such as code classes [8]. Building upon this, they further presented a hybrid framework that combines orthogonal IR models (e.g., RTM and VSM/JS) via affine transformation, leveraging complementary properties across models to enhance traceability accuracy [9]. Moreover, fine-graining traceability elements constitutes another important direction for improving TLR performance. Methods such as FTLR adopt fastText word embeddings and WMD to compute similarity between sentence-level requirements and code methods, yielding better experimental results than conventional coarse-grained document-level matching [2]. Learning-based methods enhance traceability accuracy by capturing feature patterns. Early work leveraged classic machine-learning algorithms such as SVM and decision trees, as well as deep-learning architectures including CNN and RNN, to learn nonlinear mappings among artifacts [10]. While these approaches achieve promising results on specific datasets, their heavy reliance on high-quality labeled data limits their generalization capability in agile projects with scarce annotations. In recent years, LLM-driven TLR has attracted substantial research attention. In terms of model fine-tuning, specialized models such as NoRBERT are adopted to detect and filter non-functional noise within requirements, supplying higher-quality inputs for traceability algorithms [11]. Chuyan Ge et al. optimized LLaMA variants of different sizes through data augmentation, prompting and fine-tuning. Their approach outperforms traditional IR and deep-learning methods on cross-level requirement traceability tasks, yet exposes limitations such as sensitivity to data augmentation and dependencies on long-text processing [12]. To better unlock the reasoning potential of LLMs, researchers have intensively investigated prompt engineering for traceability tasks. For instance, Alberto D. Rodrı́guez et al. revealed how minor prompt modifications influence traceability outputs, and proposed prompting strategies with chain-of-reasoning and multi-view classification to improve model performance across diverse datasets [13]. On this basis, the LISSA framework further integrates Retrieval-Augmented Generation (RAG). It performs retrieval over candidate links followed by secondary reasoning judgment to further boost traceability performance

[3]. Nevertheless, existing research predominantly focuses on optimizing downstream traceability algorithms while overlooking the root-cause problem of poor original-requirement quality. Specifically, traditional IR methods heavily depend on lexical overlap and struggle to bridge the semantic gap between requirements and code under common agile-development challenges such as terminology mismatch and text sparsity. Learning-based approaches (including LLM-based solutions) intrinsically require high-quality labeled data. When input requirement descriptions are ambiguous and unstandardized, their performance is highly vulnerable to noise and suffers degraded generalization. Accordingly, the effectiveness of both lexicon-dependent IR methods and annotation-reliant learning methods is constrained by input-data ambiguity and inconsistency. Motivated by this observation, this study targets the requirement side. By introducing normalized use-case specifications, we improve the semantic clarity and consistency of requirements and provide higher-quality inputs for subsequent traceability tasks. B. Controlled Natural Language To reduce the semantic gap between unstructured requirements and code implementations, structured requirement representations such as use-case specifications have long been regarded as potential “semantic bridges”. Use-case specifications can deliver richer semantic information than raw requirements, including concrete operational workflows and constraints. Transforming raw requirements into use-case specifications improves requirement precision, analyzability and traceability while preserving readability [11], [14]. Studies have shown that writing use-case specifications under CNL standards effectively reduces ambiguity, enhances requirement consistency, and lays a foundation for automated verification and test generation [15]. Nevertheless, existing CNL-related research and applications still face two major challenges. First, from the research-perspective, most existing work focuses on improving intrinsic requirement quality or downstream tasks such as test generation [16]. Systematical empirical investigations into its effectiveness for requirement-to-code traceability remain scarce. Second, there are constraints imposed by production costs. Manually authoring high-quality use-case specifications incurs substantial overhead, which fundamentally conflicts with the rapid-iteration paradigm advocated by prevalent agile development [17]. Existing use-case generation tools (e.g., the RSL editor) still heavily rely on manual analysis and human annotation [15]. The lack of automatic generation approaches limits their adoption in large-scale projects. Targeting the above challenges, this study adopts the verbose-style Controlled Natural Language (CNL-B) proposed by Alberto Silva [15] as the specification standard for use-case specifications. With predefined structured templates and constrained vocabularies, CNL-B converts loose, ambiguous natural-language requirements into standardized descriptions with uniform formatting, precise wording and complete

information [15]. Three major considerations motivate our selection of CNL-B. First, logical compatibility: CNL-B emphasizes the “subject-action-object” structure, which shares natural structural similarity with the “object-method-parameter” logic in source code. Second, purification and standardization: compared with unstructured requirements, CNL-B filters descriptive noise to a certain extent and guarantees that each use-case step constitutes a distinct functional unit. Third, automation-friendliness: the well-defined syntax of CNL-B can be readily understood by LLMs, enabling automatic transformation from raw requirements to normalized use-cases and substantially reducing human labor. III. R ESEARCH M ETHODOLOGY A. Research Questions To evaluate how CNL-B-compliant use-case-specification datasets affect requirement-to-code traceability performance under the IR-based FTLR framework and the RAG-based LISSA framework, we formulate the following research questions: RQ1: To what extent do automatically generated normalized use-case specifications preserve consistency with the original requirements? This research question validates whether LLMs can accurately and completely inherit the semantic logic of original requirements while complying with CNL-B formatting constraints. At the generation stage, prompt optimization and cross-model validation are adopted to ensure generation stability. At the evaluation stage, a manual auditing scheme is applied to quantitatively score generated specifications across three dimensions: accuracy, completeness, and consistency. Moreover, Cohen’s Kappa coefficient is introduced to assess inter-annotator agreement, providing a reliable scientific benchmark for the quality of the generated dataset. RQ2: Compared with original requirements, to what extent can normalized use-case specifications improve traceability performance (F1-score) under different traceability frameworks (i.e., IR-based framework and RAG-LLM framework)? Under two representative approaches, namely FTLR and LISSA, this research question compares the performance of original requirements and CNL-B-compliant use-case specifications under varying granularities and different similarity computation methods. Evaluated metrics include F1-score, Precision, and Recall. RQ3: What are the differences in the impacts exerted by different requirement-decomposition strategies on use-case-specification generation and traceability performance? By comparing two requirement-decomposition strategies (business-logic-oriented versus technical-layer-oriented), this research question investigates their influences on use-case-specification generation and how such influences propagate to traceability tasks, so as to identify which decomposition strategy is more beneficial for traceability link recovery.

B. Dataset Collection Four open-source requirement-to-code traceability datasets are selected in this study: eANCI, eTour, iTrust and LibEST. These datasets cover diverse domains including government affairs, tourism, healthcare, and cybersecurity, adopt two mainstream programming languages (Java and C), and contain requirement artifacts in template-based use-cases, free-form natural text, and event flows. To guarantee experimental rigor while reducing experimental overhead, we only generate CNL-B-compliant use-case specifications via the LLM-based automatic framework for requirements with ground-truth traceability links in the four datasets. Due to varying granularity of original requirements, one original requirement may yield 1 to 6 relatively independent normalized use-case specifications. Detailed statistics are listed in Table I. C. Evaluation Metrics This paper adopts widely-used evaluation metrics in the requirement traceability community [19], namely Recall, Precision, and F1-score. Recall measures the completeness of traceability links identified by automatic algorithms, i.e., the percentage of correctly detected traceability links among all ground-truth traceability links. The F1-score is the harmonic mean of Precision and Recall. The three metrics are computed from Equation (1) to Equation (3): TP (1) Recall = TP + FN TP (2) Precision = TP + FP 2 · Precision · Recall F1-score = (3) Precision + Recall Where: True Positive (TP): number of samples correctly identified as possessing traceability links; True Negative (TN): number of samples correctly identified as having no traceability links; False Positive (FP): number of negative samples misclassified as having traceability links; False Negative (FN): number of positive samples misclassified as having no traceability links. D. LLM-based Automatic Generation of CNL-B-Compliant Normalized Use-Case Specifications To quantitatively evaluate the potential of CNL-B normalized use-case specifications (see Figure 1) for requirement-to-code TLR, two generation strategies are designed in this study: 1) Requirement-driven approach (detailed in this section): generates normalized use-case specifications solely from original requirements. It simulates real-world industrial scenarios where prior knowledge such as target source code and ground-truth links is unavailable. This serves as our primary generation strategy. 2) Code-augmented approach (Appendix A): leverages ground-truth linked code artifacts to assist generation and serves as a comparison baseline. This variant explores whether injecting low-level implementation details into use-case specifications with access

to linked-code context can further bridge the semantic gap, or instead trigger feature dilution caused by abstraction-level mismatch. This section first introduces the requirement-driven approach: the LLM-based automatic generation method for CNL-B-compliant normalized use-case specifications (see Figure 2). This approach alleviates manual dependency in conventional use-case authoring and mitigates subjective biases over use-case granularity and format consistency. The framework consists of two core phases: 1) requirement decomposition: split original requirements into independent sub-requirements; 2) CNL-B formatting generation: transform sub-requirements into standardized CNL-B representations. The use-case generation task is built upon the Qwen3-Max-Instruct model, whose performance heavily relies on prompt strategy design. Accordingly, Chain-of-Thought (CoT) prompting is adopted to enhance reasoning stability and output standardization. 1) Requirement Decomposition Algorithm: The core task of this phase is to produce sub-requirement decomposition suggestions Aanalysis results for original requirements by exploiting LLM reasoning under predefined decomposition guidelines. We first conduct evaluations over decomposition strategies, and construct prompts based on the optimal strategy to perform requirement decomposition. Furthermore, a requirement decomposition algorithm is designed and implemented, as shown in Algorithm 1. The algorithm adopts an iterative loop mechanism. In each iteration, the system assembles decomposition principles, prompt templates and input requirement files, feeds them into Qwen3-Max-Instruct to generate decomposition suggestions. Outputs are stored after passing format validation. Algorithm 1 Requirement Decomposition Algorithm Require: Original requirement dataset Sdata , Large Language Model Qwen3-Max-Instruct, Initial decomposition principles P Ensure: Requirement decomposition analysis results Aanalysis results 1: for each requirement file reqi ∈ Sdata do 2: P rompt ← AssemblePrompt(reqi , P) ▷ Assemble principles, template and input file 3: Response ← Qwen3-Max-Instruct.Generate(P rompt) 4: if CheckFormat(Response) then ▷ Format validation 5: Resulti , Pnew ← Parse(Response) 6: P ← Update(P, Pnew ) ▷ Principle evolution: absorb decomposition features and update in real-time 7: Aanalysis results [reqi ] ← Resulti 8: end if 9: end for return Aanalysis results This algorithm introduces a principle-evolution mechanism. Each iteration absorbs decomposition characteristics summarized from the current task and updates rules dynamically, improving model adaptability to diverse requirement texts.

TABLE I E XPERIMENTAL DATASETS Dataset

Programming Language

Dataset Source

Requirement Type

Language

Domain

Requirement Files

Traceability Links

Requirements with Traceability Links

CNL-B Use-Case Specifications

eANCI eTour iTrust LibEST

JAVA JAVA JAVA C

CoEST CoEST CoEST Moran et al. [18]

Use-case Template Use-case Template Event-flow Only Natural Text

Italian (automatically translated into English via Google Translate API) English English English

Government System Tourism System Healthcare Cybersecurity

139 58 131 52

567 309 286 204

39 57 105 47

43 66 175 71

CoEST: Center of Excellence for Software & Systems Traceability

Fig. 1. Example of CNL-B-Compliant Use-Case Specification

Fig. 2. LLM-based Automatic Generation of CNL-B-Compliant Normalized Use-Case Specifications

Three core decomposition principles are initialized: Single responsibility: clear business operations initiated by actors; • Clear boundaries: well-defined inputs, processing workflows, and outputs; • Independently testable: testable without relying on other sub-requirements. •

2) CNL-B Use-Case Specification Generation Algorithm: This phase generates CNL-B-compliant use-case specifications from the sub-requirement decomposition suggestions Aanalysis results produced by Algorithm 1. The generation procedure is illustrated in Algorithm 2. First, the system assembles structured prompts from sub-requirement descriptions, original requirements, and the

CNL-B use-case prompt template according to decomposition suggestions. The Qwen3-Max-Instruct model is invoked to produce CNL-B-compliant use-case specifications. To guarantee syntactic and structural compliance of the specifications, the system sequentially performs format normalization and format checking. Core components including Actors, Use Cases, and Use Case Scenarios are primarily validated. Documents passing validation are stored; otherwise, a re-generation instruction is triggered. E. Experimental Setup To systematically evaluate the effectiveness and generality of normalized use-case specifications under different technical architectures, this study selects FTLR [2] and LISSA [3] as baseline experimental frameworks. FTLR represents the

Algorithm 2 CNL-B Use-Case Specification Generation Algorithm Require: Sub-requirement decomposition suggestions Aanalysis results , original requirement dataset Sdata , Large Language Model Qwen3-Max-Instruct, CNL-B prompt template TemplateCNL-B Ensure: Result set of CNL-B use-case specifications Rcnl 1: Rcnl ← ∅ 2: for each sub-requirement sub reqi ∈ Aanalysis results do 3: is passed ← False 4: while not is passed do 5: P rompt ← ConstructPrompt(sub reqi , Sdata , 6: TemplateCNL-B ) 7: Response ← Qwen3-Max-Instruct.Generate( 8: P rompt) ← 9: Standardized Doc FormatNormalize(Response) ▷ Format normalization processing ▷ 10: if CheckModules(Standardized Doc) then Validate modules such as Actors, UseCases, Scenarios 11: is passed ← True 12: Rcnl ← Rcnl ∪ {Standardized Doc} 13: else 14: continue ▷ Validation failed, trigger re-generation 15: end if 16: end while 17: end for return Rcnl

Fig. 3. Comparison between FTLR and LISSA Approaches

traditional information-retrieval-based traceability paradigm, whereas LISSA stands for the novel traceability paradigm built upon retrieval-augmented generation and large language models. Both are open-source projects and have been repeatedly cited in related research, ensuring experimental reproducibility and comparability. Figure 3 illustrates the comparison between the FTLR and LISSA approaches. 1) FTLR Experimental Configuration: FTLR is a fine-grained semantic-matching-based requirement-to-code traceability technique consisting of two phases: precomputation and link processing. In the precomputation phase, requirements and source code are split into sentence-level and method-level units respectively. After

text preprocessing, fastText [20] is adopted to generate embeddings, and the similarity matrix among units is calculated via WMD. In the link-processing phase, a dual-threshold mechanism filters valid links: candidate links are first produced by majority voting over method units for each requirement, and valid links are yielded after final-threshold filtering. This study extends the FTLR framework to support traceability-link establishment from use-case specifications to source code, and enables comparison between two metrics: cosine similarity and word-mover’s distance. 2) LISSA Experimental Configuration: LISSA is a traceability-link-recovery method based on RAG and LLMs. Its workflow mainly comprises three stages: 1) Preprocessing: various software artifacts are structurally decomposed according to artifact types (e.g., splitting requirements into sentences and source code into methods). After textualization, artifact contents are vectorized and stored via an embedding model. 2) Retrieval: for each requirement element, the top-k most similar target units are retrieved as candidate links according to cosine similarity. 3) Mapping: an LLM performs classification judgment on candidate links, supporting two strategies: plain prompting and Chain-of-Thought (CoT) reasoning. To achieve relatively favorable performance, the Chain-of-Thought reasoning strategy is adopted for judgment. Traceability-link-recovery experiments are conducted at two granularities: file level and element level. Furthermore, targeting the changed traceable artifacts for the LISSA framework, three aggregation strategies are reconstructed in our experiments, corresponding to Equation (4), Equation (5), and Equations (6)(7). Since LISSA leverages LLMs for binary classification without outputting similarity scores, all three aggregation strategies adopt coverage ratio for aggregation. Regarding experimental environment configuration, all-MiniLM-L6-v2.onnx is uniformly used for vector embedding, and gpt-4o-2024-05-13 is employed to perform traceability-link-recovery judgments, keeping consistency with the original paper’s experimental settings. Note that LISSA currently only supports Java parsing; hence experiments for the LibEST (C-language) project are excluded. Notation Definitions: F : Code file R : Requirement file U : Use-case specification file (belongs to requirement file R) ER : Set of requirement elements for requirement file R EU : Set of use-case-specification elements for use-case file U EF : Set of code elements for code file F

U (R) : Set of all use-case specification files contained in requirement file   [ L ⊆ ER ∪ Eu × EF : Fine-grained element-level traceabili u∈U (R)

3) File-Level Relative-Coverage Aggregation: This strategy aggregates judgment results between use-case-specification

files and code files into requirement-to-code-file traceabil- are embedded into the use-case-specification generation ity links. A traceability link is established between require- pipeline to enforce compliance with predefined CNL-B ment file R and code file F if more than θ1 fraction of writing patterns. Moreover, the eANCI dataset is selected use-case-specification files contained in R have traceability as the benchmark. All its generated use-case specifications links with F : undergo manual content inspection and format review to  eliminate potential logical deviations and further refine the   U ∈ U (R) Link(U, F ) = 1 ≥ θ1 Trace1 (R, F ) = 1 prompts. |U (R)| (4) IV. E XPERIMENTAL R ESULTS AND A NALYSIS 4) Dual-Direction Relative-Coverage Threshold Judgment: A. RQ1: To what extent do automatically generated norThis strategy directly aggregates judgment results be- malized use-case specifications preserve consistency with the tween requirement elements and code elements into original requirements? requirement-to-code-file traceability links. A traceability link is established between requirement file R and code file F if TABLE II the fraction of requirement elements within R linked to F S TATISTICS OF R ANDOMLY S AMPLED S AMPLES reaches θ2 , or the fraction of code elements within F linked Evaluation Dimension Annotator A Annotator B to R reaches θ2 :  Mean Std. Dev. Mean Std. Dev. |{r ∈ ER | ∃s ∈ EF , (r, s) ∈ L}| ≥ θ2 Trace2 (R, F ) = 1 Accuracy 4.480 0.193 4.488 0.175 |ER | 4.477 0.187 4.423 0.178  Completeness |{s ∈ EF | ∃r ∈ ER , (r, s) ∈ L}| Compliance 4.768 0.423 4.815 0.338 ≥ θ2 ∨ |EF | (5) 5) Fine-Grained Multi-Stage Cascade Aggregation: This strategy performs two-level aggregation. It first aggregates element-level judgments between use-case elements and code elements into use-case-specification-to-code-file traceability links, and further aggregates those results into requirement-to-code-file traceability links. First-stage aggregation: For use-case-specification file U and code file F , a traceability link between U and F is established if the fraction of elements in U linked to F reaches α1 , or the fraction of elements in F linked to U reaches α1 :  |{eu ∈ EU | ∃s ∈ EF , (eu , s) ∈ L}| Link (U, F ) = 1 ≥ α1 |EU |  |{s ∈ EF | ∃eu ∈ EU , (eu , s) ∈ L}| ∨ ≥ α1 |EF | (6) Second-stage aggregation: For requirement file R and code file F , a traceability link is established between R and F if the fraction of use-case specifications within R linked to F reaches α2 :  Fig. 4. Multi-dimensional Quantitative Score Distribution for Randomly  |{U ∈ U (R) | Link(U, F ) = 1}| Sampled Samples ≥ α2 Trace3 (R, F ) = 1 |U (R)| (7) Building upon the aforementioned quality-assurance measures, systematic sampling-based evaluation is conducted to F. Quality Control assess the consistency between generated use-case specificaTo guarantee the syntactic-structural and semantic-logical tions and original requirements. For three datasets including compliance of LLM-generated CNL-B-compliant use-case eTour, iTrust and LibEST, 60 samples are randomly selected specifications, three quality-control strategies are adopted: from a total of 321 generated outputs for quality inspection. prompt optimization, automatic format checking, and Two graduate students independent of the authors are invited manual sampling audit. First, prompts are optimized as separate annotators. They perform 5-point scoring across through multi-round iterations and cross-validated over three dimensions: accuracy (fidelity to original requirements), three models: DeepSeek-V3.2, gemini-3.1-pro-preview completeness (coverage of key logic), and compliance (conand Qwen3-Max-Instruct, to ensure prompt generalization formance to CNL-B format). The inter-annotator agreement capability. Second, format normalization and format validation coefficient is computed to evaluate scoring reliability.

Scoring results over the 60 sampled samples show that Annotator A achieves average scores of 4.480, 4.477 and 4.768 for accuracy, completeness and compliance respectively; Annotator B obtains average scores of 4.488, 4.423 and 4.815 (full score = 5). Variances for all dimensions are below 0.5, indicating favorable semantic stability of generated outputs. Scores are concentrated within the range of 4-5, with no samples scoring lower than 3. The Cohen’s Kappa coefficient for independent annotations from the two annotators reaches 0.8323, corresponding to substantial agreement and demonstrating high reliability of scoring results. Taking REQUIREMENT 48 from the LibEST dataset as an example, Figure 5 illustrates its transformation into a CNL-B-compliant use-case specification. As shown in the comparative analysis in Table III, core elements from the original requirement are accurately mapped into the generated use-case specification: the initiator (“EST client”) corresponds to actor a EstClient; the requested object (“list of CA-desired CSR attributes”) corresponds to data entity e CsrAttributes; the core operation (“requests”) corresponds to use-case name uc 48 RequestCsrAttributes; and operational details (“HTTPS GET...path of ’/csrattrs”’ ”) correspond to concrete steps within scenarios. Moreover, the generated specification provides reasonable refinements and supplements for the original requirement by clarifying preconditions for execution and internal system processing steps. Such supplementary information conforms to the technical background of the EST protocol and does not exceed the intent of the original requirement. The above results demonstrate that generated normalized use-case specifications maintain good consistency with original requirements. Quantitatively, average scores for all three dimensions exceed 4.45, indicating semantic preservation above 89%. Compliance achieves the highest score, which reveals that LLMs can well respect CNL-B formatting constraints. Scoring agreement and stability further validate the credibility of generation quality. From the case-mapping perspective, LLMs can accurately identify core elements within original requirements and map them into structured CNL-B fields without information loss. Collectively, automatically generated normalized use-case specifications effectively preserve the semantic logic of original requirements and deliver high-quality input data for subsequent traceability tasks. B. RQ2: Compared with original requirements, to what extent can normalized use-case specifications improve traceability performance (F1-score) under different traceability frameworks (i.e., IR-based framework and RAG-LLM framework)? To answer this research question, traceability performance of original requirements and CNL-B-compliant use-case specifications are compared under two representative traceability frameworks, FTLR and LISSA, across different granularities and similarity computation methods. The FTLR framework adopts two metrics: cosine similarity and Word-Mover’s Distance, and conducts traceability at both file-level and element-level granularities. Built upon retrieval-augmented generation and LLMs, the LISSA framework evaluates the

effectiveness of normalized use-case specifications at file-level and element-level granularities with different aggregation strategies. Under the FTLR framework, normalized use-case specifications yield performance improvements on most datasets. As shown in Table IV, at both file-level and element-level granularities, whether cosine similarity (Cos) or Word-Mover’s Distance (WMD) is adopted for text similarity calculation, replacing original requirements with CNL-B-compliant use-case specifications increases the F1-score to varying degrees for the iTrust, EanciTrans, and Libset datasets. The most prominent improvement occurs for element-level traceability using WMD on the Libset dataset, where the F1-score rises from 0.065 to 0.579, mitigating WMD metric failure caused by overly concise original requirements. This result indicates that normalization enriches contextual semantic information of requirements and optimizes the representation quality of requirement elements, thus enhancing the recovery capability of traceability links between requirements and source code. Under the LISSA framework, the performance of normalized use-case specifications is strongly coupled with aggregation strategies. Since normalization splits one original requirement into multiple use-case specifications (i.e., one-to-many mapping), directly adopting LISSA’s default “existence-based aggregation” strategy amplifies noise from random spurious associations. Therefore, relative-coverage aggregation and multi-stage cascade aggregation are introduced to mitigate such interference. Experiments demonstrate that combining normalized use-case specifications with properly designed aggregation strategies yields performance gains. Taking the eAnciTrans dataset (Table V) as an example: for file-level traceability, when paired with relative-coverage aggregation, the F1-score increases from 0.212 to 0.324, corresponding to a 52.800% improvement; for element-level traceability combined with multi-stage cascade aggregation, the F1-score rises from 0.211 to 0.416, achieving a 96.869% improvement. Performance degradation is observed for partial datasets under both frameworks. Under the FTLR framework (Table IV), performance drops for the eTour dataset. Under file-level Word-Mover’s Distance metric, the F1-score decreases from 0.422 to 0.387, with a degradation of 8.230%; under element-level cosine-similarity metric, it falls from 0.484 to 0.363, corresponding to a 24.864% degradation. Under the LISSA framework (Table V), for file-level traceability on eTour with relative-coverage aggregation, the F1-score declines from 0.522 to 0.498 (-4.499%); for file-level traceability on iTrust with relative-coverage aggregation, the F1-score drops from 0.075 to 0.057. Analysis reveals that original requirements of eTour possess high lexical overlap with source-code artifacts. When LLMs transform them into lexically constrained normalized use-case specifications, high-weight technical terms are converted into low-weight generic structural words, leading to feature-word dilution and reduced signal-to-noise ratio. For iTrust, original requirements contain limited information and are partially presented in

Fig. 5. Transformation Process from Original Requirement to CNL-B-Compliant Use-Case Specification

TABLE III C OMPARISON BETWEEN O RIGINAL R EQUIREMENT AND C ORRESPONDING U SE -C ASE S PECIFICATION Original Requirement

Corresponding Use-Case Specification

Correspondence Analysis

EST client requests...CSR attributes

③

...from the CA...

②

...list of CA-desired CSR attributes

①

...by sending an HTTPS GET...path of ”/csrattrs”

④

The use-case name directly reflects the core action “request”. The requester is explicitly identified as the EST client. The core requested data “list of CA-desired CSR attributes” is clearly stated. The HTTP method and operation path of the request are precisely matched.

TABLE IV I MPACT OF CNL-B-C OMPLIANT U SE -C ASE S PECIFICATIONS ON T RACEABILITY P ERFORMANCE UNDER THE FTLR F RAMEWORK Dataset

Metric

Req-FileCode-File (Cos)

UC-FileCode-File (Cos)

Req-FileCode-File (WMD)

UC-FileCode-File (WMD)

Req-ElemCode-Method (Cos)

UC-ElemCode-Method (Cos)

Req-ElemCode-Method (WMD)

UC-ElemCode-Method (WMD)

etour

Precision Recall F1-score

0.415 0.380 0.397

0.279 0.263 0.271↓ (-31.695%)

0.496 0.367 0.422

0.327 0.474 0.387↓ (-8.230%)

0.550 0.432 0.484

0.317 0.425 0.363↓ (-24.864%)

0.501 0.597 0.545

0.294 0.588 0.392↓ (-28.115%)

Itrust

Precision Recall F1-score

0.141 0.269 0.185

0.258 0.259 0.258↑ (+39.711%)

0.166 0.311 0.217

0.323 0.255 0.285↑ (+31.408%)

0.333 0.203 0.252

0.396 0.280 0.328↑ (+30.017%)

0.285 0.206 0.239

0.248 0.308 0.275↑ (+14.883%)

EanciTrans

Precision Recall F1-score

0.178 0.208 0.192

0.256 0.924 0.401↑ (+109.363%)

0.185 0.270 0.220

0.362 0.554 0.438↑ (+99.062%)

0.350 0.236 0.282

0.496 0.363 0.420↑ (+48.722%)

0.421 0.192 0.264

0.526 0.358 0.426↑ (+61.372%)

Libset

Precision Recall F1-score

0.288 0.980 0.445

0.315 0.951 0.473↑ (+6.227%)

0.330 0.868 0.478

0.386 0.730 0.505↑ (+5.666%)

0.395 0.824 0.534

0.455 0.819 0.585↑ (+9.551%)

0.700 0.034 0.065

0.450 0.814 0.579↑ (+785.664%)

↑ / ↓ denotes F1-score improvement/degradation of the use-case-based approach compared with the original-requirement-based approach. Bold font marks the better value within each pair; underlined font marks the most prominent improvement. Values in parentheses indicate relative magnitude of change.

TABLE V I MPACT OF CNL-B-C OMPLIANT U SE -C ASE S PECIFICATIONS ON T RACEABILITY P ERFORMANCE UNDER THE LISSA F RAMEWORK UC-FileCode-File (existence-based)

UC-FileCode-File (relative-coverage)

Req-ElemCode-Method (existence-based)

UC-ElemCode-Method (existence-based)

UC-ElemCode-Method (multi-stage cascade)

Dataset

Metric

Req-FileCode-File (existence-based)

EanciTrans

Precision Recall F1-score

0.28 0.171 0.212

0.694 0.212 0.324↑(+52.800%)

0.694 0.212 0.324↑(+52.789%)

0.172 0.273 0.211

0.466 0.376 0.416↑(+96.869%)

0.466 0.376 0.416↑(+96.869%)

etour

Precision Recall F1-score

0.521 0.523 0.522

0.469 0.523 0.495↓(-5.223%)

0.488 0.51 0.498↓(-4.499%)

0.145 0.37 0.208

0.141 0.458 0.216↑(+3.717%)

0.183 0.409 0.253↑(+21.377%)

Itrust

Precision Recall F1-score

0.08 0.07 0.075

0.047 0.066 0.055↓(-26.553%)

0.049 0.066 0.057↓(-24.431%)

0.229 0.381 0.286

0.071 0.612 0.127↓(-55.764%)

0.317 0.287 0.301↑(+5.039%)

Libset

Precision Recall F1-score

0.5 0.88 0.638

0.485 0.951 0.642↑(+0.691%)

0.489 0.946 0.644↑(+1.036%)

— — —

— — —

— — —

Note: ↑ / ↓ denotes F1-score improvement/degradation of the use-case-based approach compared with the original-requirement-based approach. Bold font marks the better value within each pair; underlined font marks the most prominent improvement. Values in parentheses indicate relative magnitude of change.

tabular format, which brings challenges for LLM parsing and expansion. Although generated outputs satisfy specification constraints, they exhibit semantic deviations from source code. These two cases conversely verify that requirement quality constitutes a critical factor determining traceability performance. To further explore the effectiveness boundary of normalizing requirements into use-case specifications, this study introduces a code-augmented approach (see Appendix B for details) as a comparison baseline equipped with prior knowledge. Under the FTLR and LISSA frameworks (as shown in Tables VI to IX), we observe that the code-augmented approach achieves higher traceability performance on most datasets by supplementing code-derived information. Nevertheless, in certain scenarios (e.g., the FTLR cosine metric for eTour and iTrust), the requirement-driven approach outperforms its code-augmented counterpart. To explain this counter-intuitive observation, our analysis reveals that although code details are generally beneficial, excessive injection of low-level code information introduces semantic interference toward high-level requirements. This interference manifests in two aspects. First, the introduction of technical vocabulary increases false positives. When extracting code summaries, the code-augmented approach inevitably imports numerous generic technical terms (e.g., Manager, Context, List). In cosine-similarity retrieval based on lexical co-occurrence, these low-discriminative terms dilute core business features and produce artificially inflated similarity scores between use-case specifications and many irrelevant code files. For instance, the precision drops from 0.258 to 0.196 for the iTrust dataset in Table VI, which stems exactly from increased false positives caused by lexical noise. Second, the coverage-check mechanism embedded in the code-augmented approach (Algorithm 4) enforces use-case specifications to cover and associate all linked code files. Real-world traceability links often span multiple architec-

tural layers. Forcing heterogeneous code logic into a single normalized use-case violates the single-responsibility principle of use cases. Such semantic overload causes misjudgments during matching due to shifted or unfocused semantics. As illustrated in Figure 6, use-case specifications generated by the code-augmented approach contain far more implementation-oriented technical details for the same original requirement. In summary, the experimental comparisons verify the double-edged-sword effect of code context when bridging the semantic gap. Meanwhile, they highlight the core advantage of the proposed requirement-driven approach: without extra computational overhead from code processing or prior ground-truth knowledge, the requirement-driven method maintains pure business logic and a proper level of abstraction, yielding high cost-effectiveness. It achieves near-upper-bound performance in most scenarios while effectively avoiding noise from low-level implementation details, which delivers stronger robustness and practical value for real-world industrial applications. From the above experimental results, the following conclusions can be drawn. Normalized use-case specifications improve traceability performance across most datasets and traceability frameworks, and the improvement is particularly prominent when original requirements are of low quality. The choice of aggregation strategy exerts a considerable impact on traceability performance; appropriate aggregation helps filter noise and boost precision. Requirement-driven normalized use-cases achieve performance close to that of the code-augmented variant under most conditions, demonstrating the high cost-effectiveness of the generation method proposed in this work. Meanwhile, normalization may bring slight performance degradation for requirements with high lexical overlap with source code. This suggests that practitioners should decide whether to apply normalization according to the quality of input requirements in practical deployment.

TABLE VI F ILE -L EVEL T RACEABILITY UNDER FTLR: C OMPARISON BETWEEN U SE -C ASE S PECIFICATIONS AND C ODE -AUGMENTED VARIANT Dataset

Metric

Req-File-Code-File (Cos)

UC-File-Code-File (Cos) Requirement-Driven

Code-Augmented

Req-File-Code-File (WMD)

UC-File-Code-File (WMD) Requirement-Driven

Code-Augmented

etour

Precision Recall F1-score

0.415 0.380 0.397

0.279 0.263 0.271↓(-31.695%)

0.268 0.201 0.23↓(-42.065%)

0.496 0.367 0.422

0.327 0.474 0.387↓(-8.230%)

0.311 0.526 0.391↓(-7.346%)

Itrust

Precision Recall F1-score

0.141 0.269 0.185

0.258 0.259 0.258↑(+39.711%)

0.196 0.213 0.204↑(+10.270%)

0.166 0.311 0.217

0.323 0.255 0.285↑(+31.408%)

0.331 0.479 0.391↑(+80.184%)

EanciTrans

Precision Recall F1-score

0.178 0.208 0.192

0.256 0.924 0.401↑(+109.363%)

0.263 0.915 0.408↑(+112.500%)

0.185 0.270 0.220

0.362 0.554 0.438↑(+99.062%)

0.359 0.603 0.450↑(+104.545%)

Libset

Precision Recall F1-score

0.288 0.980 0.445

0.315 0.951 0.473↑(+6.227%)

0.327 0.912 0.481↑(+8.090%)

0.330 0.868 0.478

0.386 0.730 0.505↑(+5.666%)

0.439 0.819 0.572↑(+19.665%)

Note: ↑ / ↓ denotes F1-score improvement/degradation of the use-case-based approach compared with the original-requirement-based approach. Bold font marks the better value within each pair; underlined font marks the most prominent improvement. Values in parentheses indicate relative magnitude of change.

TABLE VII E LEMENT -L EVEL T RACEABILITY UNDER FTLR: C OMPARISON BETWEEN U SE -C ASE S PECIFICATIONS AND C ODE -AUGMENTED VARIANT Dataset

Metric

Req-Elem-Code-Method (Cos)

UC-Elem-Code-Method (Cos) Requirement-Driven

Code-Augmented

Req-Elem-Code-Method (WMD)

UC-Elem-Code-Method (WMD) Requirement-Driven

Code-Augmented

etour

Precision Recall F1-score

0.550 0.432 0.484

0.317 0.425 0.363↓(-24.864%)

0.252 0.594 0.354↓(-26.796%)

0.501 0.597 0.545

0.294 0.588 0.392↓(-28.115%)

0.238 0.714 0.357↓(-34.469%)

Itrust

Precision Recall F1-score

0.333 0.203 0.252

0.396 0.280 0.328↑(+30.017%)

0.305 0.402 0.347↑(+37.662%)

0.285 0.206 0.239

0.248 0.308 0.275↑(+14.883%)

0.259 0.392 0.312↑(+30.353%)

EanciTrans

Precision Recall F1-score

0.350 0.236 0.282

0.496 0.363 0.420↑(+48.722%)

0.608 0.517 0.559↑(+98.095%)

0.421 0.192 0.264

0.526 0.358 0.426↑(+61.372%)

0.529 0.624 0.573↑(+116.976%)

Libset

Precision Recall F1-score

0.395 0.824 0.534

0.455 0.819 0.585↑(+21.973%)

0.445 0.912 0.598↑(+11.985%)

0.700 0.034 0.065

0.450 0.814 0.579↑(+785.664%)

0.445 0.877 0.591↑(+808.860%)

Note: ↑ / ↓ denotes F1-score improvement/degradation of the use-case-based approach compared with the original-requirement-based approach. Bold font marks the better value within each pair; underlined font marks the most prominent improvement. Values in parentheses indicate relative magnitude of change.

TABLE VIII F ILE -L EVEL T RACEABILITY UNDER LISSA: C OMPARISON BETWEEN U SE -C ASE S PECIFICATIONS AND C ODE -AUGMENTED VARIANT

Dataset

Metric

Req-File-Code-File (existence-based aggregation)

UC-File-Code-File (existence-based aggregation)

UC-File-Code-File (relative-coverage aggregation)

Requirement-Driven

Code-Augmented

Requirement-Driven

Code-Augmented

EanciTrans

Precision Recall F1-score

0.280 0.171 0.212

0.694 0.212 0.324↑(+52.800%)

0.511 0.511 0.511↑(+140.756%)

0.694 0.212 0.324↑(+52.789%)

0.528 0.504 0.516↑(+143.011%)

etour

Precision Recall F1-score

0.521 0.523 0.522

0.469 0.523 0.495↓(-5.223%)

0.183 0.727 0.293↓(-43.930%)

0.488 0.510 0.498↓(-4.499%)

0.441 0.471 0.455↓(-12.758%)

Itrust

Precision Recall F1-score

0.080 0.070 0.075

0.047 0.066 0.055↓(-26.553%)

0.022 0.259 0.041↓(-45.438%)

0.049 0.066 0.057↓(-24.431%)

0.059 0.192 0.091↑(+21.178%)

Libset

Precision Recall F1-score

0.500 0.880 0.638

0.485 0.951 0.642↑(+0.691%)

0.464 0.995 0.633↓(-0.718%)

0.489 0.946 0.644↑(+1.036%)

0.478 0.989 0.644↑(+1.004%)

Note: ↑ / ↓ denotes F1-score improvement/degradation of the use-case-based approach compared with the original-requirement-based approach. Bold font marks the better value within each pair; underlined font marks the most prominent improvement. Values in parentheses indicate relative magnitude of change.

Fig. 6. Comparison of Use-Case Specifications Generated by Requirement-Driven and Code-Augmented Approaches

C. RQ3: What are the differences of requirement-splitting strategies on generated specifications and traceability performance?

different use-case

Original requirement documents contain requirements of diverse granularities. Some requirements have single responsibilities and are easy to comprehend and trace, while others mix multiple functions and increase the difficulty of subsequent processing. To generate high-quality normalized use-case specifications, this study invokes an LLM to judge and split raw requirements in the preprocessing stage: a requirement item will be split if it covers multiple responsibilities. Practical observations reveal two distinct splitting tendencies produced by the LLM. One is business-logic-oriented splitting, which

divides requirements according to complete business flows of user operations. The other is technical-layer-oriented splitting, which partitions requirements from technical-implementation perspectives including front-end interaction, back-end processing, and data manipulation. These two splitting approaches exert different influences on the quantity of generated use-case specifications and subsequent traceability performance. To compare the differences between the two splitting strategies, this study designs two distinct prompts to guide the LLM to perform business-logic-oriented and technical-layer-oriented splitting on the EanciTrans and Libset datasets, which represent different domains and programming languages respectively. We compare the traceability perfor-

TABLE IX E LEMENT -L EVEL T RACEABILITY UNDER LISSA: C OMPARISON BETWEEN U SE -C ASE S PECIFICATIONS AND C ODE -AUGMENTED VARIANT

Dataset

Metric

Req-Elem-Code-Method (existence-based aggregation)

Requirement-Driven

Code-Augmented

Requirement-Driven

Code-Augmented

EanciTrans

Precision Recall F1-score

0.172 0.273 0.211

0.466 0.376 0.416↑(+96.869%)

0.341 0.743 0.468↑(+121.365%)

0.466 0.376 0.416↑(+96.869%)

0.341 0.743 0.468↑(+121.233%)

etour

Precision Recall F1-score

0.145 0.370 0.208

0.141 0.458 0.216↑(+3.717%)

0.088 0.519 0.151↓(-27.612%)

0.183 0.409 0.253↑(+21.377%)

0.273 0.231 0.250↑(+20.175%)

Itrust

Precision Recall F1-score

0.229 0.381 0.286

0.071 0.612 0.127↓(-55.764%)

0.045 0.927 0.086↓(-70.048%)

0.317 0.287 0.301↑(+5.039%)

0.335 0.409 0.369↑(+28.637%)

UC-Elem-Code-Method (existence-based aggregation)

UC-Elem-Code-Method (multi-stage cascade aggregation)

Note: ↑ / ↓ denotes F1-score improvement/degradation of the use-case-based approach compared with the original-requirement-based approach. Bold font marks the better value within each pair; underlined font marks the most prominent improvement. Values in parentheses indicate relative magnitude of change. TABLE X C OMPARISON OF THE NUMBER OF USE - CASES GENERATED BY DIFFERENT SPLITTING STRATEGIES Dataset

Number of traced requirement files

Number of use-case specifications under business-oriented splitting

39 47

43 71

EanciTrans Libset

Number of use-case specifications under techn 119 147

TABLE XI P ERFORMANCE COMPARISON OF DIFFERENT SPLITTING STRATEGIES UNDER FILE - LEVEL COSINE SIMILARITY

Dataset

Metric

FTLR

LISSA

Business-logic-oriented

Technical-layer-oriented

Business-logic-oriented

Technical-layer-oriented

0.256 0.924 0.401

0.257 0.931 0.403

0.694 0.212 0.324

0.657 0.243 0.355

Precision 0.315 0.315 Recall 0.951 0.995 F1-score 0.473 0.478 Note: Bold font denotes the better value within each pair of methods.

0.489 0.946 0.644

0.511 0.913 0.655

EanciTrans

Precision Recall F1-score

Libset

mance of use-case specifications generated by the two splitting strategies under file-level evaluation with cosine similarity for both the FTLR and LISSA traceability frameworks. In terms of the number of generated use-cases (Table X), technical-layer-oriented splitting produces significantly more use-cases than business-logic-oriented splitting. For the EanciTrans dataset, 43 use-cases are generated by business-logic-oriented splitting and 119 use-cases by technical-layer-oriented splitting. For the Libset dataset, 71 use-cases are generated by business-logic-oriented splitting and 147 use-cases by technical-layer-oriented splitting. The number of use-cases generated by technical-layer-oriented splitting is approximately two to three times that of business-logic-oriented splitting. In terms of traceability performance (Table XI), under the FTLR framework, the F1-score of business-logic-oriented splitting is 0.401 and that of technical-layer-oriented splitting is 0.403 for EanciTrans; for Libset, the F1-score

is 0.473 for business-logic-oriented splitting and 0.478 for technical-layer-oriented splitting. Under the LISSA framework, the F1-score of business-logic-oriented splitting is 0.324 and that of technical-layer-oriented splitting is 0.355 for EanciTrans; for Libset, the F1-score is 0.644 for business-logic-oriented splitting and 0.655 for technical-layer-oriented splitting. Overall, technical-layer-oriented splitting yields slightly higher F1-scores, yet the improvement is limited, with differences below 0.02 across both datasets. Apart from quantitative metrics, use-case specifications generated by the two splitting strategies differ in readability. Business-logic-oriented splitting preserves complete user-operation workflows. Its outputs conform better to human reading habits and facilitate subsequent manual comprehension and maintenance. By contrast, technical-layer-oriented splitting decomposes one business workflow into multiple technical-implementation steps. Although it establishes

clearer correspondences with technical-implementation layers, it breaks business integrity and increases the complexity of traceability links. In summary, business-logic-oriented splitting achieves traceability performance comparable to technical-layer-oriented splitting, while generating fewer use-cases with lower computational overhead. Meanwhile, its outputs possess better readability for manual comprehension and maintenance. Therefore, business-logic-oriented splitting is more feasible for generating normalized use-case specifications. V. A NALYSIS OF FACTORS A FFECTING VALIDITY Although this study adopts the high-performance gpt-4o-2024-05-13 model for traceability link judgment in LISSA, large language models are essentially inference systems that perform deep statistical modeling over massive corpora and conduct probabilistic mapping based on high-dimensional vector representations and context-dependent relationships. Lacking genuine semantic understanding, such models still suffer from output uncertainty and potential hallucinations when handling highly specialized software-engineering texts. To mitigate the impact of this factor on experimental results, a rigorous manual quality-check mechanism is introduced. The Cohen’s Kappa coefficient among human reviewers is measured at 0.8323, reaching the standard of “high agreement”, which eliminates systematic errors caused by model output bias as much as possible. While this study covers projects written in Java and C across four different domains, traceability links in industrial-scale ultra-large code repositories can be far more complex. Future work is required to validate the scalability of the proposed approach under ten-million-line code volumes. Experimental results reveal that traceability performance (especially F1-score) is strongly coupled with the choice of aggregation algorithm. Particularly for fine-grained element-level matching, the noise-filtering capability of the aggregation strategy directly determines final outcomes. Aggregation algorithms adopted in this work, such as bidirectional relative-coverage-threshold-based decision-making and mode-mean aggregation, are relatively reasonable solutions determined according to the characteristics of current datasets. Large-scale horizontal ablation experiments over all plausible aggregation algorithms have not been conducted. Accordingly, the performance conclusions drawn in this paper are conditional to some extent, and future work will target more general-purpose aggregation paradigms. VI. D ISCUSSION A. Applicable Boundaries of Normalized Use-Case Specifications This study finds that normalized use-case specifications do not consistently yield positive improvements on traceability performance across all scenarios. The anomalous behavior observed on the eTour dataset delineates the applicable boundaries of our proposed method. This dataset exhibits high

lexical overlap between original requirements and low-level source code. Under such circumstances, normalization converts these high-weight technical terms into generic structural words with low weights, which dilutes discriminative features and degrades signal-to-noise ratio. This observation implies that the effectiveness of normalized use-case specifications is negatively correlated with original-requirement quality: higher-quality original requirements leave smaller room for normalization-driven performance gains; normalization may produce counter-productive effects when original requirements already align well with source code. Motivated by this finding, we recommend pre-evaluation of requirements before applying normalization transformation. For instance, lexical overlap between raw requirements and codebases (e.g., average TF-IDF cosine similarity) can quantify requirement-code alignment. If the alignment is already high (e.g., eTour), raw requirements should be retained or only lightweight format normalization shall be applied. When pre-evaluation indicates vague requirement descriptions and a large semantic gap (e.g., iTrust, LibEST), full CNL-B normalization transformation can deliver tangible performance improvements. B. Bottleneck Analysis and Optimization Directions for Traceability Performance Experimental results demonstrate that normalized use-case specifications improve the baseline of traceability performance, reduce occurrences of extremely low-value samples, and stabilize overall performance. This validates the value of standardizing representations from the requirement side for narrowing the semantic gap. Nevertheless, state-of-the-art performance still has a noticeable gap before practical engineering deployment. Although normalization alleviates semantic ambiguity in requirements, it cannot break the inherent performance upper bound of traceability techniques. Improved input quality guarantees the lower bound of performance, whereas the upper bound remains determined by the matching capability of traceability algorithms. The underlying reason is that bridging the semantic gap requires collaboration between requirement representations and traceability algorithms. Normalized use-case specifications resolve the clarity of requirement descriptions, yet understanding code intent still depends on the intrinsic semantic-matching capacity of traceability algorithms. Traditional lexical-co-occurrence-based frameworks are constrained by shallow lexical overlap. For large-language-model-based frameworks, models still lack sufficient ability to distinguish core information from secondary content within specifications. Therefore, future research shall shift from solely optimizing inputs toward joint optimization of inputs and algorithms. On one hand, key-information filtering mechanisms can be introduced during specification generation to guide traceability algorithms to focus on core semantic units. On the other hand, efficient traceability techniques can be explored upon normalized use-case specifications, attempting to break through current performance ceilings.

C. Benefits and Challenges (Code-Augmentation)

of

Code

Context

Experiments show that use-case specifications generated with code context (code-augmentation) generally outperform counterparts produced purely from requirements, verifying the auxiliary effect of code information for requirement comprehension. However, code-augmentation yields slightly inferior results in a small number of cases. Analysis attributes this phenomenon to how code information is organized: injecting code snippets with low relevance to target requirements may distract models from core semantics. This finding suggests that introducing code context shall be accompanied by effective information-filtering strategies rather than simple information stacking. Prioritizing high-value code information within limited context windows constitutes a key factor for improving generation quality. VII. C ONCLUSION A semantic gap arises between requirements and source code due to divergent abstraction levels. Existing studies mostly focus on algorithm optimization while overlooking quality defects within requirement descriptions themselves. To bridge this gap at the source, this work proposes reconstructing raw requirements automatically into CNL-B-compliant use-case specifications using large language models, so as to unify text structures, filter noise, and supplement implicit domain context. To systematically evaluate the effectiveness of such normalized use-case specifications, empirical experiments are conducted on four public datasets over two representative traceability frameworks: information-retrieval-based FTLR and retrieval-augmented-generation-based LISSA. Experimental results indicate that normalized use-case specifications effectively boost requirement-to-code traceability performance for requirements with ambiguous original descriptions. The main contributions are summarized as follows: 1) A triple-form dataset of “raw requirement-normalized use-case-code” is constructed. Based on four public datasets, LLM-driven automatic generation produces CNL-B-compliant use-case specifications. Multi-round quality evaluations verify the accuracy and reliability of generated data, providing data support for subsequent research within the requirement traceability community. 2) An automatic use-case-specification generation framework is proposed. The pipeline integrates requirement splitting, CNL-B structured generation, and quality validation, offering a reproducible technical solution for automatic construction of normalized use-case specifications. 3) The performance improvement brought by normalized use-case specifications is quantitatively validated. Experiments show that introducing normalized use-case specifications yields consistent performance gains across most datasets. For the LibEST dataset, the F1-score rises from 0.065 to 0.579. This result confirms the

effectiveness of narrowing the semantic gap from the requirement side. 4) Advantages of the business-logic-oriented splitting strategy are revealed. Comparisons between business-logic-oriented and technical-layer-oriented splitting show that the former reduces the volume of normalized use-cases by 58.6% on average. Organized around functional-semantic units, it preserves complete business logic, avoids semantic fragmentation, and achieves stable traceability performance with lower manual and computational costs. This finding delivers practical references for generating normalized use-case specifications. R EFERENCES [1] C. Tao, M. Zhang, H. Guo, and Z. Huang, “A survey of requirement traceability methods for diverse software artifacts,” Chinese Journal of Computers, vol. 45, no. 11, pp. 2393–2419, 2022. [2] T. Hey, F. Chen, S. Weigelt, and W. F. Tichy, “Improving traceability link recovery using fine-grained requirements-to-code relations,” in 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 2021, pp. 12–22. [3] D. Fuchß, T. Hey, J. Keim, H. Liu, N. Ewald, T. Thirolf, and A. Koziolek, “Lissa: toward generic traceability link recovery through retrieval-augmented generation,” in 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). IEEE, 2025, pp. 1396– 1408. [4] P. Rempel and P. Mäder, “Preventing defects: The impact of requirements traceability completeness on software quality,” IEEE Transactions on Software Engineering, vol. 43, no. 8, pp. 777–797, 2016. [5] M. Balları́n, L. Arcega, V. Pelechano, and C. Cetina, “On the influence of architectural languages on requirements traceability,” Software: Practice and Experience, vol. 53, no. 3, pp. 704–728, 2023. [6] D. Diaz, G. Bavota, A. Marcus, R. Oliveto, S. Takahashi, and A. De Lucia, “Using code ownership to improve ir-based traceability link recovery,” in 2013 21st international conference on program comprehension (icpc). IEEE, 2013, pp. 123–132. [7] J. H. Hayes, A. Dekhtyar, and J. Osborne, “Improving requirements tracing via information retrieval,” in Proceedings. 11th IEEE International Requirements Engineering Conference, 2003. IEEE, 2003, pp. 138–147. [8] A. De Lucia, M. Di Penta, R. Oliveto, A. Panichella, and S. Panichella, “Improving ir-based traceability recovery using smoothing filters,” in 2011 ieee 19th international conference on program comprehension. IEEE, 2011, pp. 21–30. [9] M. Gethers, R. Oliveto, D. Poshyvanyk, and A. De Lucia, “On integrating orthogonal information retrieval methods to improve traceability recovery,” in 2011 27th IEEE International Conference on Software Maintenance (ICSM). IEEE, 2011, pp. 133–142. [10] J. Guo, J. Cheng, and J. Cleland-Huang, “Semantically enhanced software traceability using deep learning techniques,” in 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE). IEEE, 2017, pp. 3–14. [11] T. Hey, J. Keim, and S. Corallo, “Requirements classification for traceability link recovery,” in 2024 IEEE 32nd International Requirements Engineering Conference (RE). IEEE, 2024, pp. 155–167. [12] C. Ge, T. Wang, X. Yang, and C. Treude, “Cross-level requirements tracing based on large language models,” IEEE Transactions on Software Engineering, vol. 51, no. 7, pp. 2044–2066, 2025. [13] A. D. Rodriguez, K. R. Dearstyne, and J. Cleland-Huang, “Prompts matter: Insights and strategies for prompt engineering in automated software traceability,” in 2023 IEEE 31st International Requirements Engineering Conference Workshops (REW). IEEE, 2023, pp. 455–464. [14] I. Darif, G. El Boussaidi, S. Kpodjedo, and C. Politowski, “Controlled natural language for requirements specification: A systematic literature review,” ACM Computing Surveys, vol. 58, no. 7, pp. 1–36, 2026. [15] A. R. Da Silva, “Linguistic patterns, styles, and guidelines for writing requirements specifications: Focus on use cases and scenarios,” IEEE Access, vol. 9, pp. 143 506–143 530, 2021.

[16] C. Wang, F. Pastore, A. Goknil, and L. C. Briand, “Automatic generation of acceptance test cases from use case specifications: an nlp-based approach,” IEEE Transactions on Software Engineering, vol. 48, no. 2, pp. 585–616, 2020. [17] S. Maro, J.-P. Steghöfer, P. Bozzelli, and H. Muccini, “Tracimo: a traceability introduction methodology and its evaluation in an agile development team,” Requirements Engineering, vol. 27, no. 1, pp. 53– 81, 2022. [18] K. Moran, D. N. Palacio, C. Bernal-Cárdenas, D. McCrystal, D. Poshyvanyk, C. Shenefiel, and J. Johnson, “Improving the effectiveness of traceability link recovery using hierarchical bayesian networks,” in Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, 2020, pp. 873–885. [19] T. Koboyatshwene and Y. Ayalew, “Requirements traceability: A systematic literature review,” in Proceedings of the 40th ACM/SIGAPP Symposium on Applied Computing, 2025, pp. 1509–1513. [20] E. Grave, P. Bojanowski, P. Gupta, A. Joulin, and T. Mikolov, “Learning word vectors for 157 languages,” in Proceedings of the eleventh international conference on language resources and evaluation (LREC 2018), 2018.

A PPENDIX Appendix A Related Data Tables

Appendix B LLM-based Automatic Generation of CNL-B-Formatted Normalized Use-Case Specifications: Code Augmentation Based on Ground-Truth Links To compare the performance difference between use-case specifications generated from requirements only and those generated by feeding requirements together with code context based on ground-truth traceability links, and to explore the effectiveness of code information for use-case specifications, this study designs a code-augmentation approach (see Figure 7). The framework includes three core stages: 1) Data preprocessing, which enhances the model’s semantic perception through code summarization techniques; 2) Iterative requirement splitting, which decomposes raw requirements into independent sub-requirements via an iterative optimization mechanism; 3) CNL-B formatted generation, which converts sub-requirements into standardized CNL-B format. It should be noted that this framework is solely intended to investigate the behavior of use-case specifications under complete code information and is not applicable in practical industrial deployment. The use-case generation task in this framework also adopts the Qwen3-Max-Instruct model and leverages the CoT prompting strategy to improve the model’s reasoning stability and output standardization. B.1 Data Preprocessing: The preprocessing procedure (Algorithm 3) mainly consists of three steps. First, extract raw requirements R, source code C, and ground-truth traceability links Glinks from each project as the baseline. Second, employ an LLM to generate functional summaries for source code. File summaries Absfile ,j , function signatures Sig j , and function summaries FuncAbs j are assembled to enrich semantic representations. Finally, construct the dictionary Sdata (containing requirements R, traceability-linked code C, and code summaries LinkedCode) based on available traceability

links, which serves as the base input for subsequent processing stages. Algorithm 3 Data Preprocessing and Structuring Require: Raw dataset D, ground-truth traceability links G, large language model LLM Ensure: Structured dataset Sdata 1: Sdata ← ∅ ▷ Initialize empty structure 2: for each dataset d ∈ D do 3: R, C ← LoadRequirementsAndCode(d) ▷ Load requirement and code files 4: Glinks ← ParseTraceabilityLinks(d) ▷ Parse traceability links 5: ▷ Step 1: Process each code file 6: C′ ← ∅ ▷ Store processed code information 7: for each code file cj ∈ C do 8: Sig j ← ExtractSignatures(cj ) ▷ Extract function signatures 9: Abs file,j ← GenerateFileAbstract(LLM , cj ) ▷ Generate file summary 10: FuncAbs j ← GenerateFunctionAbstracts(LLM , cj , Sig j ) ▷ Generate function summaries 11: C ′ [cj ] ← code : content(cj ), code abstract : Abs file,j , functions : {Sig j , FuncAbs j } 12: end for 13: ▷ Step 2: Process each requirement file 14: for each requirement file ri ∈ R do 15: ReqContent ← content(ri ) ▷ Obtain requirement content 16: LinkedCode ← ∅ 17: if ri ∈ domain(Glinks ) then 18: for each code file cj linked to ri in Glinks do 19: if cj ∈ C ′ then 20: LinkedCode[cj ] ← C ′ [cj ] 21: end if 22: end for 23: end if  24: Sdata [ri ] ← req file content : ReqContent, traceability Links : LinkedCode 25: end for 26: end for return Sdata B.2 Iterative Requirement Splitting: The core task of this stage is to generate sub-requirement splitting suggestions Aresults for raw requirements by leveraging the reasoning capability of the LLM under the guidance of decomposition criteria. To this end, this study designs and implements an iterative requirement decomposition algorithm, as illustrated in Algorithm 4. When processing the structured dataset Sdata produced by Algorithm 3, the system first adopts a dynamic code-context selection strategy. According to the scale of the requirement to be split and its associated code, it determines whether full source code or code summaries shall be fed into the prompt,

Fig. 7. Code-Augmentation Approach

so as to balance context-window constraints and information Algorithm 4 Iterative Requirement Decomposition completeness. Require: Structured dataset Sdata (from Algorithm 3), large Subsequently, the algorithm enters an iterative loop. Within language model client LLM , maximum iteration count each iteration, the system constructs a complete prompt by K=3 integrating splitting guidelines, the current requirement, and Ensure: Analysis results Aresults containing use-case splitting the selected code context, and submits it to the LLM. The suggestions generated splitting suggestions undergo coverage validation. 1: P ← InitialSplittingPrinciples ▷ Initialize splitting By comparing splitting outputs against the original traceability principles matrix, the algorithm ensures that all associated code elements 2: Aresults ← ∅ are covered. 3: for each entry (ri , Clinks ) ∈ Sdata do If coverage omissions are detected, the missing code el4: req content ← ri ements together with existing splitting outputs are com5: codes, abstracts ← Clinks posed into a new prompt and sent to the LLM to regener6: if EvaluateContextLimit(req content, codes) then ▷ ate splitting suggestions. This procedure runs at most three Evaluate whether context exceeds limit times. Upon successful validation, an evolution mechanism 7: code ctx ← codes is triggered: project-specific characteristics summarized in the 8: else current round are absorbed to update decomposition crite9: code ctx ← abstracts ria on-the-fly, improving the model’s adaptability to diverse 10: end if project characteristics. 11: iter ← 0, is complete ← False, feedback ← ∅ At initialization, the algorithm sets the following core 12: while iter < K and not is complete do decomposition principles: 13: iter ← iter + 1 • Single responsibility: clear business operations actively 14: P rompt ← AssemblePrompt(req content, code ctx, P, feedback) initiated by actors ▷ Assemble prompt • Well-defined boundaries: explicitly specified inputs, pro15: Response ← LLM .Generate(P rompt) cessing procedures, and outputs 16: Resulti , Pnew ← Parse(Response) ▷ Parse • Independent and testable: testable without relying on response other sub-requirements 17: Missing ← ValidateTraceability(Resulti , codes) B.3 CNL-B Formatted Generation: To standardize the de▷ Check coverage omissions scriptions of use-case specifications, this study adopts CNL-B 18: if Missing = ∅ then as the output format. Built upon the sub-requirement splitting 19: is complete ← True suggestions Aresults produced by Algorithm 4, this generation 20: P ← Update(P, Pnew ) ▷ Update splitting process is implemented via a three-step pipeline, as shown in principles Algorithm 5. 21: else First, guided by splitting suggestions, the system assembles 22: feedback ← GenerateFeedback(Missing) ▷ sub-requirement descriptions and code context into prompts. Generate feedback messages The LLM is employed for information extraction to identify 23: end if key entities including Actors, Core Operations, and Excep24: end while tions. During this procedure, full source code or code sum25: Aresults [ri ] ← Resulti maries are dynamically selected according to current con26: end forreturn Aresults text length to balance information completeness under model context-window constraints. Structured use-case generation

follows, where extracted unstructured information is mapped onto the standard CNL-B JSON template, establishing systematic mappings among requirement elements (actors, use-cases, scenarios). Finally, a format-validation-and-retry mechanism automatically parses generated CNL-B artifacts and inspects key fields (e.g., whether the main scenario is missing). Regeneration is triggered immediately upon validation failure to guarantee syntactic and structural compliance of output files. Algorithm 5 CNL-B Format Conversion Algorithm Require: Analysis results Aresults (from Algorithm 4), dataset structure Sdata , large language model client LLM Ensure: Formatted CNL-B use-case files Fcnl 1: Fcnl ← ∅ 2: Pextract , Pmap ← LoadPromptTemplates() ▷ Load prompt templates 3: for each project entry (rf ile , SubReqs) ∈ Aresults do ▷ 4: Clinks , Abslinks ← Sdata [rf ile ].traceability Links Retrieve traceability links 5: for each (uid , udesc ) ∈ SubReqs do 6: Ctarget ← RetrieveCode(uid , Clinks ) ▷ Retrieve associated code 7: if not EvaluateContextLimit(Pextract , udesc , Ctarget ) then 8: code ctx ← Ctarget 9: else 10: code ctx ← Abslinks 11: end if 12: Entities ← LLM .Generate(Pextract , udesc , code ctx) 13: JSONcnl ← LLM .Generate(Pmap , Entities) 14: Result, isV alid ← ParseAndVerify(JSONcnl ) ▷ Parse and validate 15: if not isV alid then 16: ReGenerate(uid ) ▷ Trigger regeneration 17: else 18: SaveToFile(rf ile , uid , Result) ▷ Save result to file 19: Fcnl ← Fcnl ∪ {Result} 20: end if 21: end for 22: end forreturn Fcnl

TABLE XII M ULTI - DIMENSIONAL Q UALITY Q UANTIFICATION S CORES FOR S AMPLED I NSTANCES

No.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Dataset

eTour eTour eTour eTour eTour eTour eTour eTour eTour eTour eTour eTour eTour eTour eTour eTour eTour eTour eTour eTour iTrust iTrust iTrust iTrust iTrust iTrust iTrust iTrust iTrust iTrust iTrust iTrust iTrust iTrust iTrust iTrust iTrust iTrust iTrust iTrust LibEST LibEST LibEST LibEST LibEST LibEST LibEST LibEST LibEST LibEST LibEST LibEST LibEST LibEST LibEST LibEST LibEST LibEST LibEST LibEST

Use-case Name

UC1 FGUC 001.txt UC4 FGUC 001.txt UC6 FGUC 001.txt UC9 FGUC 001.txt UC12 FGUC 001.txt UC13 FGUC 002.txt UC15 FGUC 001.txt UC19 FGUC 001.txt UC20 FGUC 002.txt UC23 FGUC 002.txt UC28 FGUC 001.txt UC34 FGUC 001.txt UC38 FGUC 001.txt UC41 FGUC 001.txt UC49 FGUC 001.txt UC52 FGUC 001.txt UC53 FGUC 002.txt UC55 FGUC 001.txt UC57 FGUC 001.txt UC58 FGUC 001.txt UC1E1 FGUC 001.txt UC3S3 FGUC 001.txt UC3S5 FGUC 001.txt UC4S3 FGUC 001.txt UC6S2 FGUC 001.txt UC11S1 FGUC 002.txt UC11S1 FGUC 005.txt UC15E1 FGUC 001.txt UC17S2 FGUC 001.txt UC21S1 FGUC 001.txt UC27S2 FGUC 001.txt UC30E1 FGUC 001.txt UC30S2 FGUC 001.txt UC34E1 FGUC 001.txt UC34S5 FGUC 001.txt UC35S1 FGUC 002.txt UC37S1 FGUC 002.txt UC37S3 FGUC 001.txt UC38S1 FGUC 001.txt UC38S2 FGUC 001.txt RQ4 FGUC 001.txt RQ5 FGUC 001.txt RQ8 FGUC 001.txt RQ14 FGUC 001.txt RQ14 FGUC 003.txt RQ16 FGUC 002.txt RQ17 FGUC 006.txt RQ19 FGUC 001.txt RQ20 FGUC 001.txt RQ23 FGUC 001.txt RQ24 FGUC 001.txt RQ26 FGUC 003.txt RQ27 FGUC 001.txt RQ36 FGUC 001.txt RQ39 FGUC 001.txt RQ46 FGUC 002.txt RQ45 FGUC 001.txt RQ49 FGUC 001.txt RQ55 FGUC 001.txt RQ57 FGUC 002.txt

Annotator A

Annotator B

Accuracy (Full score: 5)

Completeness (Full score: 5)

Compliance (Full score: 5)

Accuracy (Full score: 5)

Completeness (Full score: 5)

Compliance (Full score: 5)

4.7 4.4 4.3 4.7 4.4 4.2 4.4 4.3 4.6 4.6 4.8 4.6 4.1 4.5 4.4 4.2 4.5 4.7 4.6 4.7 4.5 4.7 4.5 4.1 4.3 4.5 4.2 4.5 4.4 4.5 4.7 4.4 4.6 4.5 4.6 4.4 4.7 4.3 4.5 4.6 4.5 4.3 4.7 4.4 4.3 4.7 4 4 4.3 4.5 4.8 4.7 4.4 4.6 4.5 4.7 4.7 4.4 4.6 4.5

4.6 4.8 4.5 4.8 4.7 4.7 4.6 4.5 4.7 4.5 4.4 4.3 4.5 4.2 4.3 4.4 4.3 4.5 4.8 4.2 4.7 4.3 4.3 4.5 4.2 4.6 4.4 4.2 4.3 4.5 4.6 4.5 4.3 4.6 4.7 4.6 4.5 4.4 4.7 4.3 4.7 4.6 4.5 4.2 4.5 4.8 4.1 4.2 4.7 4.4 4.3 4.5 4.2 4.7 4.5 4.5 4.4 4.5 4.2 4.6

5 5 5 5 5 5 5 5 4 5 5 5 4 5 5 5 4 5 5 5 5 5 5 5 5 4 4 5 4 5 5 5 4 5 4 4 4 5 5 4 5 5 5 5 5 4 5 5 5 5 4 5 5 5 5 4.1 5 5 5 5

4.5 4.4 4.1 4.5 4.6 4.6 4.4 4.6 4.5 4.8 4.6 4.2 4.8 4.7 4.4 4.6 4.3 4.5 4.7 4.6 4.6 4.7 4.5 4.2 4.4 4.3 4.6 4.6 4.5 4.7 4.4 4.4 4.7 4.5 4.3 4.5 4.6 4.5 4.7 4.5 4.4 4.4 4.3 4.6 4.5 4.6 4 4 4.4 4.7 4.5 4.6 4.3 4.4 4.5 5.6 4.5 4.3 4.4 4.7

4.7 4.5 4.6 4.3 4.3 4.5 4.2 4.4 4.8 4.7 4.7 4.4 4.5 4.6 4.5 4.5 4.2 4.5 4.2 4.5 4.8 4.4 4.6 4.7 4.6 4.5 4.7 4.4 4.6 4.5 4.3 4.5 4.8 4.3 4.2 4.5 4.4 4.8 4.3 4.4 4.7 4.5 4.6 4.5 4.6 4.8 4.2 4.1 4.6 4.4 4.6 4.7 4.5 4.3 4.3 4.7 4.4 4.5 4.6 4.5

5 5 5 5 5 5 5 5 4.2 5 5 5 4.2 5 5 5 4.2 5 5 5 5 5 5 5 5 4.2 4.2 5 4.2 5 5 5 4.2 5 4.2 4.2 4.2 5 5 4.2 5 5 5 5 5 4.2 5 5 5 5 4.3 5 5 5 5 4.2 5 5 5 5

Related documents

Record · ID 459737 · SHA-256 441dc5ccccf710b8
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Bring this data into your agent: Connect via MCP or API · 20 free credits to start · no card required
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive