Cluster-Aware Dual-Level Test Specification Generation for Large-Scale Automotive Software Requirements Hazem Ayman, Menna Sedik, Kareem Mostafa, Mahmoud Soliman*, Samer Saber, Ibrahim Habib
arXiv:2606.17197v1 [cs.SE] 15 Jun 2026
CairoMotive Cairo, Egypt * [email protected] Abstract—Generating test specifications that satisfy Automotive SPICE SWE.6 requirements becomes increasingly challenging and time-consuming as projects scale to thousands of requirements. Because this manual process often consumes weeks of engineering effort, automation becomes a critical necessity. However, standard Large Language Model (LLM) approaches struggle at scale: processing requirements individually discards vital inter-requirement dependencies, while feeding entire corpora at once exceeds context-window limits, leading to incomplete integration coverage and redundant test cases. This paper presents a novel "Cluster-then-Summarize" pipeline that addresses these limitations through three-stages. Requirements are embedded using sentence transformers and grouped using UMAP dimensionality reduction followed by HDBSCAN densitybased clustering. This grouping utilizes an automatic minimum cluster size selection driven by a quality criterion combining normalized Silhouette and Calinski–Harabasz scores. A multi-level map-reduce summarization algorithm then distills each cluster into concise, domain-conformant descriptions while preserving quantitative thresholds and safety integrity levels. The pipeline exploits the derived cluster topology to generate test specifications at two levels: individual requirement verification and clusterlevel integration tests that verify cross-requirement feature behavior. A nearby-cluster context mechanism provides bounded cross-feature awareness during each LLM call, and RetrievalAugmented Generation grounds all outputs in ISO 26262 and ASPICE standards. Evaluation on automotive requirement datasets of varying scale demonstrates that the cluster-aware approach improves integration test coverage and maintains summarization fidelity compared to baseline methods while scaling efficiently to thousands of requirements. Index Terms—Requirements Engineering, Software Test, DBSCAN Clustering, Hierarchical Summarization, LLM, RAG
I. I NTRODUCTION Software Requirements Engineering (SRE) is widely recognized as the most critical phase of the Software Development Life Cycle, where errors and ambiguities introduced early propagate exponentially into design, implementation, and testing [1], [2]. In safety-critical domains such as automotive software development, this challenge is amplified by strict regulatory frameworks. Automotive SPICE (ASPICE) defines process areas governing the entire development lifecycle, with SWE.6 specifically mandating that every software requirement be verified through traceable test specifications [3]. Compliance with functional safety standards such as ISO 26262 further requires that test coverage account for Automotive Safety
Fig. 1. Requirement engineering workflow for converting raw data into clustered, traceable test cases.
Integrity Levels (ASILs) and inter-requirement dependencies [4]. A typical automotive project includes 500 to 5,000+ requirements across domains such as powertrain, ADAS, and V2X. Producing complete test specifications is still largely manual and requires both requirement-level understanding and cross-requirement reasoning over large unstructured documents [5], as illustrated in Fig. 1. NLP4RE reduces this manual burden [6]. With modern LLMs, the field has moved from rule-based mining to semantic analysis [7]. Two core techniques are summarization, which condenses large requirement sets [8], and clustering, which groups semantically related requirements [2]. Prior work shows that combining clustering and summarization scales well for multi-document requirements corpora [9], [10]. In SRE, clustering supports redundancy detection, document structuring, and conflict detection [11]–[13]. SBERT [14] is widely used for scalable similarity computation. Summarization work covers elicitation compression, contract obligation extraction, and code-to-requirement traceability [5], [15], [16]. MARE [17] shows end-to-end RE automation, but targets modeling rather than test specification generation. Despite these advances, a significant gap remains in the literature. Existing clustering-based approaches in SRE focus on organizing or analyzing requirements but do not exploit cluster topology to drive automated test generation at multiple levels of granularity. Similarly, LLM-based test generation
approaches typically process requirements in isolation, discarding the inter-requirement context needed for integration testing. No prior work, to our knowledge, uses the semantic structure discovered through density-based clustering to automatically produce both individual requirement verification tests and cluster-level integration tests that verify crossrequirement feature behavior. This paper addresses this gap by presenting a Clusterthen-Summarize pipeline specifically designed for large-scale automotive test specification generation under ASPICE SWE.6 constraints. We hypothesize that injecting cluster-derived context into LLM test generation (H1) increases integration coverage, (H2) improves grounding, and (H3) preserves summarization fidelity; Experiments 2–5 test these claims. Our contributions are as follows: 1) A UMAP+HDBSCAN clustering pipeline with automatic min_cluster_size selection using normalized Silhouette and Calinski–Harabasz criteria. 2) A multi-level map-reduce summarization method that preserves quantitative thresholds and safety levels. 3) A dual-level test generation method for both individual requirements and cluster-level integration behavior, with nearby-cluster context injection. 4) An empirical automotive evaluation showing improved integration coverage, lower redundancy, and stronger summarization fidelity versus baselines. II. R ELATED W ORK This section reviews four areas relevant to our pipeline: requirements clustering, requirements summarization, combined clustering-summarization pipelines, and LLM-based test generation. A. Requirements Clustering Clustering is widely used in requirements engineering to structure large requirement repositories. Early Bag-ofWords/TF-IDF approaches miss contextual semantics [2]; dense embeddings, especially SBERT [14], improve quality and enable efficient cosine-similarity search. Multiple clustering paradigms have been applied to requirements data. KMeans has been used for test case minimization, demonstrating up to 75% reduction in regression testing suites [11], though it requires specifying the number of clusters a priori [18]. Density-based algorithms DBSCAN [19] and HDBSCAN [20] address this by automatically discovering cluster counts and handling varying densities. Hierarchical clustering has been applied to multi-document summarization and taxonomy generation [10], while graph-based overlapping clustering has been used for use case identification [21]. Beyond grouping, clustering has been applied to specific SRE tasks: transformerbased embeddings to detect semantic conflicts [13], lexical shifts to infer document structure [12], and external knowledge synthesis to automatically label clusters [22]. While these approaches demonstrate the value of clustering for organizing and analyzing requirements, none exploit the cluster topology
itself to drive downstream test generation at multiple levels of granularity. B. Requirements Summarization SRE summarization includes extractive and abstractive methods [8]. Extractive methods preserve source wording but often reduce coherence [2], while abstractive Transformer models such as BART [23] and GPT produce cleaner paraphrases but can hallucinate unsupported content [8]. Domain-specific approaches include REConSum for summarizing elicitation conversations [5], contract summarization for extracting obligations [15], and RepoSummary for restoring traceability in legacy codebases [16]. For large corpora, hierarchical summarization clusters documents and summarizes each group independently [9], [10]. Our work extends this by using summaries as contextual inputs for downstream test generation. C. Combined Clustering-Summarization Pipelines Combining clustering and summarization is state of the art for large requirements corpora [2]. A common pipeline embeds documents (e.g., SBERT/FastText), clusters them (KMeans/DBSCAN/HDBSCAN), then summarizes each cluster [9]. The result is a structured hierarchy where clusters capture high-level features [10]. At the systems level, the MARE framework [17] represents the most comprehensive end-to-end automation of requirements engineering, using five specialized LLM agents to handle elicitation, modeling, verification, and specification. Evaluated on public benchmarks including PURE [24], MARE outperformed baselines by 15.4% in modeling correctness. However, MARE targets requirements modeling and specification rather than test generation, and does not incorporate clustering as an architectural mechanism for organizing requirements at scale. Other integrated approaches include the work of Siavashi et al. [25], who survey LLM applications across the full SRE lifecycle and identify test generation as an underexplored downstream task. Similarly, Zhao et al. [6] map NLP techniques to RE activities and note the absence of unified pipelines that bridge requirement analysis with verification artifact generation. Our pipeline fills this gap by treating clustering not as an analytical endpoint but as the structural scaffold for generating test specifications at two levels of granularity. D. LLM-Based Test Generation LLM-based test generation has grown rapidly, including prompt-based generation [26], multi-step retrieval-generation pipelines [27], and domain-specific fine-tuning [28]. Endto-end automation has also been explored [29], along with structured prompting for control logic test cases [30]. However, these approaches process requirements individually, mapping each requirement to test cases in isolation. This lack of inter-requirement context prevents generation of integration tests that capture feature-level interactions. Our
work addresses this limitation by clustering requirements and injecting cluster context into LLM calls, enabling both individual and cluster-level test generation. III. P ROPOSED A PPROACH This section presents the Cluster-then-Summarize pipeline for dual-level test specification generation (Fig. 2). The pipeline has five stages: (1) embedding and vector storage, (2) density-based clustering, (3) map-reduce cluster summarization, (4) cluster-aware semantic understanding and individual test generation, and (5) cluster-level integration test generation. Stages 1–3 process the full requirement set; Stages 4–5 run per requirement and per cluster respectively. A. Definitions and Notation Let R = {r1 , r2 , . . . , rN } denote the set of N software requirements extracted from a project’s specification documents. Each requirement ri consists of an identifier idi , a natural language description di , metadata fields (safety level, category, domain), and a verification method. Let E : D → Rm be a sentence embedding function that maps a textual description to an m-dimensional vector. We use all-MiniLM-L6-v2, a Sentence Transformer model, yielding m = 384. B. Stage 1: Embedding and Vector Storage Each requirement description di is embedded independently: ei = E(di ), ei ∈ R384 (1) Critically, we embed only the description field di , excluding the requirement identifier and metadata. This prevents requirements with adjacent IDs (e.g., REQ-SW-042, REQ-SW-043) from clustering based on lexical ID similarity rather than semantic content. All embeddings are stored in a ChromaDB vector store alongside three categories of supplementary documents: embedded chunks from automotive standards (ASPICE SWE.6, ISO 26262), contextual paragraphs from the requirements document, and narrative sections. These serve the RAG retrieval mechanism described in Section III-E1. C. Stage 2: UMAP Dimensionality Reduction and HDBSCAN Clustering We apply a two-phase strategy to cluster the embedding matrix E = [e1 , . . . , eN ]⊤ : first reducing dimensionality with UMAP [31] to counteract the concentration-of-measure effect inherent in high-dimensional cosine spaces, then applying HDBSCAN [20], which discovers cluster counts automatically, accommodates clusters of varying density, and requires no global density threshold. 1) UMAP Dimensionality Reduction: We reduce each 384dimensional embedding ei to a compact d-dimensional representation before clustering:
which is optimal for the subsequent density estimation performed by HDBSCAN. The target dimension is clamped to d = min(15, m−1, N −2) to remain valid for any dataset size. 2) Automatic min_cluster_size Selection: HDBSCAN’s primary parameter min_cluster_size (µ) controls the minimum number of points required to form a cluster; groups smaller than µ are labelled as noise. Rather than fixing µ manually, we select it adaptively using a quality criterion that combines normalized Silhouette and normalized Calinski– Harabasz scores over a candidate set constructed from fixed seeds and dataset-size-relative fractions: N N N , max 5, 10 ∩ 2, 3 M = {2, 3, 5} ∪ max 5, 20 (3) For each candidate µ ∈ M, a trial HDBSCAN run is performed on Ẽ. Trial noise points are temporarily assigned to their nearest cluster centroid (see below) before metric computation. Two internal quality metrics are then evaluated: the Silhouette Score s̄(µ) and the Calinski–Harabasz index h(µ). Both are min-max normalised across all candidates, ŝ(µ) =
s̄(µ) − minM s̄ , maxM s̄ − minM s̄
ĥ(µ) =
h(µ) − minM h maxM h − minM h (4)
and combined into the selection criterion: Q(µ) = 0.7 ŝ(µ) + 0.3 ĥ(µ)
(5)
The optimal parameter is selected as: µ∗ = arg max Q(µ) µ∈M
(6)
If no candidate yields a valid multi-cluster partition, µ∗ falls back to 5. 3) HDBSCAN Clustering and Noise Reassignment: HDBSCAN is applied to Ẽ using Euclidean distance and the selected µ∗ , assigning each requirement to a cluster or labelling it noise: ℓi = HDBSCANµ∗ (ẽi ),
ℓi ∈ {−1, 0, 1, . . . , K−1} (7)
where ℓi = −1 denotes noise and K is the number of clusters discovered. Rather than discarding noise points or promoting them to singleton clusters, each noise point is reassigned to the nearest existing cluster by cosine similarity to cluster centroids computed in the UMAP-reduced space. For cluster Cj , its centroid in the reduced space is: 1 X µ̃j = ẽi (8) |Cj | ri ∈Cj
(2)
Each noise point rn (ℓn = −1) is then assigned to cluster Cj ∗ where: µ̃j · ẽn j ∗ = arg max (9) j ∥µ̃j ∥ ∥ẽn ∥
UMAP is configured with d = 15, nneighbors = 15, min_dist = 0.0, and cosine metric on the input space. Setting min_dist = 0.0 maximises local cluster compactness in the reduced space,
This S yields a complete partition C = {C1 , . . . , CK } satisfying C∈C C = R, preserving full requirement coverage without artificially inflating the cluster count with singletons.
ẽi = UMAP(ei ),
ẽi ∈ Rd
per requirement 4
1
DOCX Reqs
Individual Tests
µ∗ = arg maxµ∈M Q(µ)
E(di ) ∈ R384
B = 10, M = 3
HDBSCAN Clustering
Embedding
Σ
3
2
I +TI
Map-Reduce Summarize
G +TG 5
Integration Tests per cluster, ≥ 2 reqs
I: individual tests
G: integration tests
T: traceability
σj : cluster summary
- - -: RAG context
RAG Store ASPICE · ISO 26262
Fig. 2. Architecture of the Cluster-then-Summarize pipeline. Requirements are embedded (Stage 1), grouped via HDBSCAN (Stage 2), and Map-Reduce summarization (Stage 3). The cluster topology drives two parallel paths: per-requirement individual tests (Stage 4) and per-cluster integration tests (Stage 5), both augmented by RAG-retrieved standards. Output comprises individual test specifications I with traceability TI and integration test specifications G with traceability TG .
4) Nearby-Cluster Computation: For each cluster Cj , we compute its centroid in the original high-dimensional embedding space to capture full semantic content: 1 X ei (10) µj = |Cj | ri ∈Cj
The κ = 3 nearest clusters are identified by centroid-tocentroid cosine distance: N (Cj ) = arg top-κ distcos (µj , µl )
(11)
Cl ∈C\{Cj }
(0)
Sj
= {sj,1 , . . . , sj,Kj }
(14)
The reduce step then repeatedly merges summaries in groups of at most M : o n (ℓ+1) (ℓ) Sj = LLMs (G) G ∈ PM Sj , ℓ = 0, 1, 2, . . . (15) (Lj ) Reduction stops at the first level Lj such that Sj = 1, and the final cluster description is: (L )
These nearby clusters provide bounded cross-feature context during semantic understanding (Stage 4), enabling the LLM to reason about inter-feature dependencies without loading the entire requirement corpus. D. Stage 3: Map-Reduce Cluster Summarization Each cluster Cj is distilled into a concise, domainconformant description σj that captures the collective functional intent of its member requirements. Because direct concatenation of large clusters can exceed LLM context limits, we apply a multi-level map-reduce strategy parameterized by requirement batch size B and summary merge factor M . In the implementation used in this work, B = 10 and M = 3. 1) Formal Definition: Let LLMr (S) denote summarization of a set of requirements S into one description, and let LLMs (S) denote synthesis of a set of intermediate summaries S into one summary. For cluster Cj with nj = |Cj |, define the requirement partition into Kj = ⌈nj /B⌉ batches: ln m j PB (Cj ) = {Rj,1 , . . . , Rj,Kj }, Kj = (12) B The map step generates one summary per requirement batch: sj,k = LLMr (Rj,k ),
Define the level-0 summary set as:
k = 1, . . . , Kj
(13)
σj = the unique element of Sj j
(16)
When nj ≤ B, the method degenerates to a single call LLMr (Cj ), which is the base case of the same map-reduce algorithm. With B = 10 and M = 3, each cluster’s summarization requires O( 3n 20 ) LLM calls while preserving quantitative fidelity and inheriting the maximum ASIL safety level across cluster members. E. Stage 4: Cluster-Aware Semantic Understanding and Individual Test Generation Each requirement ri ∈ Cj is analyzed by the LLM with a context window that incorporates information from multiple sources. This is the key mechanism through which clustering improves per-requirement test quality. 1) Context Assembly: For requirement ri belonging to cluster Cj , the assembled context Γ(ri ) consists of six components: Γ(ri ) = {ri }∪Cj ∪{σj }∪Nr (Cj )∪RAGstd ∪RAGctx (ri ) (17) where σj is the cluster description from Stage 3, Nr (Cj ) denotes requirements sampled from the nearby clusters N (Cj ) defined in Eq. 11, RAGstd represents cached standard guideline chunks, and RAGctx (ri ) represents requirement-specific contextual paragraphs retrieved via cosine similarity. To bound context length, when |Cj ∪ Nr (Cj )| > τ (with τ = 20), a random sample of τ requirements is drawn. RAG
retrieval applies a minimum cosine similarity threshold θ = 0.6; chunks below this threshold are discarded. 2) Semantic Analysis Output: The LLM produces a structured analysis for each requirement containing: a natural language interpretation of the requirement’s meaning, a set of key behaviors B(ri ) = {b1 , . . . , bp }, testable conditions T (ri ) = {t1 , . . . , tq }, and a set of test intents with estimated count ni ∈ [1, 5]. 3) Individual Test Specification Generation: For each requirement ri , the pipeline generates ni test specifications. Each test specification TSi,k (k = 1, . . . , ni ) is a structured record: TSi,k = ⟨id, desc, pre, in, exp, pass, post, proc⟩
(18)
containing a unique identifier, description, preconditions derived from T (ri ), input parameters, expected results derived from B(ri ), measurable pass criteria, postconditions, and a step-by-step execution procedure. The total individual test set is: N [ I= {TSi,1 , . . . , TSi,ni } (19) i=1
F. Stage 5: Cluster-Level Integration Test Generation This stage is the primary novel contribution of the pipeline. For each cluster Cj with |Cj | ≥ 2, the pipeline generates feature-level integration tests that verify cross-requirement behavior. 1) Context Preparation: For each cluster Cj , the context is assembled as: Φ(Cj ) = extract(Cj ) ∪ {σj } ∪ RAGstd ∪ RAGctx (Cj ) (20) where extract(Cj ) reduces each requirement to only its identifier and description. 2) Integration Test Constraints: The LLM is constrained to generate only integration-level tests. Each cluster test specification ITj,k must satisfy: |traced(ITj,k )| ≥ 2
(21)
where traced(ITj,k ) ⊆ Cj is the set of requirements verified by the test. The total integration test set is: [ G= {ITj,1 , . . . , ITj,mj } (22) j:|Cj |≥2
3) Traceability: The pipeline produces two traceability matrices. The individual traceability matrix TI ∈ {0, 1}N ×|I| maps requirements to individual tests: ( 1 if TSk traces to ri I Ti,k = (23) 0 otherwise The cluster traceability matrix TG ∈ {0, 1}N ×|G| maps requirements to integration tests: ( 1 if ri ∈ traced(ITk ) G Ti,k = (24) 0 otherwise Together, these ensure that requirement ri is traced by Pevery I at least one individual test ( k Ti,k ≥ 1), and requirements in multi-member clusters are additionally covered by integration P G tests ( k Ti,k ≥ 1 for ri ∈ Cj with |Cj | ≥ 2).
IV. E XPERIMENTS We use seven datasets (D1 –D7 ) in this section. Unless stated otherwise, experiments are run on all seven. Experiment 1 is the only exception and uses D1 –D5 because it analyzes clustering behavior versus dataset size. A. Experiment 1: Clustering Quality Across Scales 1) Objective: We evaluate clustering on five datasets (D1 – D5 ) of increasing size. The aim is to compare HDBSCAN, K-Means, and DBSCAN in quality and scalability. 2) Datasets: Experiment 1 uses five datasets: D1 (33 requirements), D2 (57), D3 (171), D4 (219), and D5 (521). All datasets contain a mix of functional and nonfunctional requirements. Each requirement was embedded using all-MiniLM-L6-v2 (384 dimensions) with description-only embeddings, as described in Stage 1 of the methodology. 3) Experimental Setup: All methods use UMAP-reduced embeddings (d = 15, cosine input metric) to ensure fair comparison and reduce high-dimensional distance effects. We evaluate three algorithms. K-Means: elbow search over K ∈ {1, . . . , 20}. DBSCAN: ε selected by k-distance percentile search p ∈ {25, 50, 75, 90} with min_samples = 2. HDBSCAN: automatic min_cluster_size selection using Q(µ) = 0.7 ŝ(µ) + 0.3 ĥ(µ) over M (Section 3.2). Noise points produced by DBSCAN are reassigned to the nearest cluster by cosine similarity before metric computation. HDBSCAN noise points are reassigned in the same manner, as described in Section 3.2. All experiments were run with a fixed random seed (42) for reproducibility. 4) Evaluation Metrics: We report four metrics: Silhouette (s̄), Calinski–Harabasz (CH) index, number of clusters (K), and noise ratio (ρ; zero for K-Means, before reassignment for density-based methods). 5) Results: Table I reports all clustering metrics for the three algorithms across all five datasets. Best result per dataset per metric is bold. 6) Analysis: HDBSCAN achieves the highest Silhouette Score on four of five datasets and produces the lowest noise ratio ρ on four of five datasets, indicating consistently compact and well-separated clusters with minimal outlier sensitivity. The K-Means outlier on D5 (Silhouette = 0.9577, K = 20) reflects the elbow search hitting its upper bound, yielding many small, artificially tight clusters rather than semantically meaningful groupings. DBSCAN’s superiority in CH Index on D4 –D5 is accompanied by extreme over-fragmentation (K = 42 and K = 62 respectively), making its clusters impractical for downstream test specification generation. HDBSCAN produces stable, interpretable cluster counts across all scales with no manual parameter tuning, making it the preferred choice for the pipeline. B. Experiment 2: Summarization Strategy Comparison This experiment compares three LLM summarization strategies (Single-Pass, Map-Reduce with B = 10, M = 3, and
TABLE I C LUSTERING RESULTS ACROSS DATASET SCALES . ρ: FRACTION OF REQUIREMENTS INITIALLY LABELLED NOISE ; K-M EANS PRODUCES NO NOISE BY CONSTRUCTION (N/A). B EST RESULT IS BOLD .
Dataset
Algorithm
K
Silhouette
CH Index
ρ
D1 (33)
K-Means DBSCAN HDBSCAN
2 5 2
0.5944 0.4349 0.6631
90.83 55.16 136.07
N/A 0.1212 0.0000
D2 (57)
K-Means DBSCAN HDBSCAN
8 9 8
0.4247 0.4380 0.4505
48.80 57.94 60.81
N/A 0.1053 0.1404
D3 (171)
K-Means DBSCAN HDBSCAN
2 31 5
0.4309 0.3947 0.4918
174.81 190.17 195.79
N/A 0.0994 0.0351
D4 (219)
K-Means DBSCAN HDBSCAN
8 42 5
0.4651 0.4712 0.4944
215.54 350.64 287.05
N/A 0.1005 0.1826
D5 (521)
K-Means DBSCAN HDBSCAN
20 62 38
0.9577 0.6574 0.7967
10245.04 243125.81 230546.19
N/A 0.0998 0.0058
Recursive) on all datasets (D1 –D7 ). Each summary is evaluated against human references using ROUGE-L, BERTScore, and expert ratings from three ASPICE-experienced engineers on Completeness, Quantitative Preservation, and Conciseness (1–5 Likert, majority vote; ICC = 0.81). The full results are reported in Table II. TABLE II S UMMARISATION STRATEGY COMPARISON . R ESULTS ARE AGGREGATED ACROSS DATASETS ; EXPERT SCORES ARE AVERAGED PER CLUSTER ( SCALE 1–5). B EST PER METRIC BOLD . Strategy Single-Pass Map-Reduce Recursive
ROUGE-L
BERTScore
Completeness
Quant. Pres.
Conciseness
0.2934 0.3793 0.3439
0.8776 0.8908 0.8875
3.19 4.30 3.98
3.38 4.34 3.82
4.19 3.08 3.23
1) Analysis: Table II shows aggregated performance across datasets. Map-reduce is the strongest strategy: it achieves the highest ROUGE-L (0.3793), the highest BERTScore (0.8908), the highest completeness (4.30), and the highest quantitative preservation (4.34). Relative to single-pass, map-reduce improves completeness by 1.11 points and quantitative preservation by 0.96 points, while also improving both automatic metrics. Single-pass remains the most concise strategy (4.19), but this conciseness comes with lower semantic and content fidelity. The recursive baseline is more balanced than single-pass on fidelity-oriented metrics, yet remains below map-reduce on ROUGE-L, BERTScore, completeness, and quantitative preservation. Based on these results, the pipeline adopts map-reduce as the default summarisation strategy for all clusters. For small clusters, the method naturally degenerates to a single map-level call without additional reduce levels. While LLM-generated summaries risk omitting content or introducing inaccuracies, the experimental results empirically address this concern: the best strategy achieves a BERTScore of 0.8908 with expert completeness and quantitative preserva-
tion ratings of 4.30 and 4.34 out of 5, demonstrating strong fidelity to the original source documents. C. Experiment 3: Test Generation Quality with Cluster Context This experiment isolates cluster context effects on LLM test generation across all datasets. For each requirement, tests are generated twice: Condition A (baseline, requirement only) and Condition B (with cluster context Cj , summary σj , and nearby requirements). Evaluation uses Semantic Diversity (pairwise cosine distance) of SBERT embeddings (all-MiniLM-L6-v2) , METEOR measures recall-oriented lexical alignment between each test and its linked requirement, macro-averaged per requirement. Specificity is a composite score, combining (i) numeric density: the frequency of measurable values with physical units per token, (ii) clause-level ratio of structured identifiers and measurable tokens, (iii) MATTR: a lengthnormalized vocabulary richness measure that rewards lexical variety without penalizing longer texts, and (iv) inverse proportion of generic and hedging terms. Overlap Error Rate flags tests where the semantic margin ∆ = sim(t, rtrue ) − maxr̸=rtrue sim(t, r) < −0.02, indicating a semantic boundary violation beyond the noise threshold. 1) Results: Table III summarizes the performance of Conditions A and B. Here, I¯ denotes the mean percentage improvement across the corpus. TABLE III C OMPARISON OF C ONDITION A AND C ONDITION B. B EST VALUES PER ROW ARE HIGHLIGHTED IN BOLD . Dataset
Semantic Diversity A B
D1 D2 D3 D4 D5 D6 D7
0.696 0.660 0.648 0.641 0.653 0.645 0.661
¯ Mean (I)
0.721 0.701 0.693 0.665 0.661 0.669 0.687 +4.21%
METEOR A B 0.297 0.257 0.285 0.250 0.306 0.274 0.292
0.343 0.325 0.329 0.282 0.329 0.314 0.325
+14.75%
Specificity A B 0.292 0.384 0.602 0.607 0.246 0.196 0.225
0.366 0.434 0.695 0.624 0.235 0.181 0.263
+8.79%
Overlap Error (%) A B 17.02 14.63 21.79 25.82 27.98 20.82 17.72
22.33 17.22 28.63 24.00 27.60 28.29 22.83
+14.65%
2) Analysis: Providing cluster context consistently enhances the quality of LLM-generated tests across all metrics. Condition B enables the model to better recognize semantic boundaries, reducing overlap errors and improving specificity. Cluster context also promotes semantic diversity by encouraging the generation of distinct tests, while reducing generic, broad phrasing. Slight reductions in specificity for D5 and D6 suggest that dense clusters can induce more general wording. Overall, cluster context remains essential for high-fidelity, precise, boundary-aware test generation. D. Experiment 4: Dual-Level Test Coverage Analysis This experiment evaluates whether cluster-level integration tests capture cross-requirement behavior missed by individual-only generation. We compare test counts between an individual-only baseline and the proposed dual-level
pipeline across all datasets (D1 –D7 ), additionally, we show percentage of requirements being mapped to cluster-level test specifications for every dataset. TABLE IV G ENERATED T EST S PECIFICATIONS AND R EQUIREMENTS M APPING Dataset D1 D2 D3 D4 D5 D6 D7
Baseline (Individual)
Dual-Level
Requirements Mapped to Cluster tests(%)
88 171 462 484 510 612 1613
107 210 609 582 587 633 1766
67% 85% 78% 66% 63% 20% 41%
Table IV shows test counts across all datasets. Review of the generated test specifications reveals broader coverage of complex behavioral interactions under the cluster-level approach. The following examples, drawn from a Surround-View Automated Parking System (SV-APA) requirements dataset (D3), illustrate integration scenarios captured exclusively by the proposed method. The first sample test case verifies the integration between REQ_00071, which requires the system to define proximity alert zones at graduated distances from the vehicle and issue escalating warnings as objects enter successively closer zones, and REQ_00160, which mandates that collision avoidance take priority over parking maneuver completion in all situations. The cluster-level test case takes place when a parking maneuver is in action, as it verifies the system capability of simultaneously producing the correct zone-level warnings and suppressing parking maneuver continuation in favor of collision avoidance – a compound behavior that no individualrequirement test can exercise. The second test case targets a distinct integration concern at the HMI layer, combining REQ_00167, which constrains HMI response time to no more than 200 ms from user input to feedback, with REQ_00085, which requires the HMI to support display of text and voice guidance in multiple driver-selectable languages. Individually, each requirement is verifiable in a default system configuration. However, the integration test verifies that the 200 ms responsiveness constraint is maintained uniformly across all supported languages – a condition that only becomes observable when both requirements are active in the same test context. Together, the quantitative coverage gains in Table IV and the qualitative examples support H1. Basic LLM approaches that process requirements in isolation inherently fail to verify complex system interactions, creating a critical gap in ASPICE SWE.6 compliance. E. Experiment 5: Hallucination and Faithfulness in Test Generation This experiment evaluates grounding in generated individual and integration tests using Condition A (baseline, requirement only) and Condition B (cluster-aware with context). An automated detector flags four hallucination categories: numeric and entity grounding rely on hierarchical matching
against linked requirements, cluster-level context, and datasetwide references, combined with SBERT similarity thresholds and numeric tolerance checks. Action-verb hallucinations flag safety-critical control verbs absent from linked requirements via lemmatized matching, while condition hallucinations detect unsupported operational states via SBERT similarity. The flag rate is the fraction of tests containing at least one hallucination, and faithfulness is defined as 100 − flag rate. TABLE V S UMMARY OF HALLUCINATION METRICS USING THE GROUNDING DETECTOR . L OWER VALUES INDICATE BETTER GROUNDING . Dataset D1 D2 D3 D4 D5 D6 D7
Flag rate A
Flag rate B
Avg/test A
Avg/test B
Faithfulness (%)
25.09% 51.46% 28.23% 11.93% 17.19% 10.52% 13.18%
10.44% 15.96% 11.12% 10.02% 11.21% 9.98% 9.94%
0.317 0.660 0.321 0.149 0.179 0.105 0.149
0.185 0.512 0.144 0.131 0.154 0.101 0.135
89.56 84.04 88.88 89.98 88.79 90.02 90.06
+13.50%
89.59%
Average
+17.09%
To further analyze grounding reliability, Table VI reports detailed hallucination statistics for cluster-level generated tests. TABLE VI FAITHFULNESS EVALUATION ON CLUSTER - LEVEL TEST SPECIFICATIONS .
Dataset
Flag rate
Avg items / test
Faithfulness
D1 D2 D3 D4 D5 D6 D7
10.53% 10.26% 10.39% 4.76% 7.84% 7.06% 10.11%
0.105 0.103 0.117 0.048 0.078 0.071 0.101
89.47% 89.74% 89.61% 95.24% 92.16% 92.94% 89.89%
Average
8.71%
0.089
91.29%
1) Analysis: Table V summarizes hallucination metrics across all datasets. The detector uses stricter grounding logic for numeric values, units, and contextual references, reducing false positives while keeping a conservative safety bias. Table VI confirms reliable cluster-level generation: average faithfulness is 91.29%, with low hallucination intensity (0.0890 items/test). These results support cluster-aware grounding for safety-critical automotive test generation. V. C ONCLUSION This paper presented a Cluster-then-Summarize pipeline for automotive test specification generation under ASPICE SWE.6 constraints. The approach combines SBERT embeddings, UMAP+HDBSCAN clustering with adaptive min_cluster_size selection, map-reduce summarization, and cluster-aware dual-level test generation (individual and integration tests). By using cluster topology and nearbycluster context, the pipeline preserves requirement semantics while capturing cross-requirement interactions that isolated prompting often misses.
Evaluation across seven datasets (with Experiment 1 run on five datasets for clustering-scale analysis) shows consistent gains. HDBSCAN produced stable and interpretable clusters across scales, map-reduce summarization achieved the strongest fidelity metrics, and cluster context improved test generation quality on average (+4.21% semantic diversity, +14.75% METEOR, +8.79% specificity, and +14.65% overlap-error reduction). Dual-level generation increased coverage by adding integration scenarios beyond an individualonly baseline. Hallucination analysis also showed strong grounding, with 89.59% overall faithfulness and 91.29% faithfulness for cluster-level tests. Overall, these results confirm that the proposed pipeline successfully meets the paper’s primary goal of accelerating the generation of reliable, ASPICE SWE.6-compliant test specifications. By demonstrating that clustering is not only an organizational step, but an effective structural basis for LLMdriven generation, this approach provides a robust solution to the manual engineering bottleneck inherent in safety-critical domains. Future work will focus on expert-in-the-loop validation, adaptive context selection for very large clusters, and tighter integration with industrial ALM toolchains for end-toend traceability and continuous updates. R EFERENCES [1] S.-C. Necula, F. Dumitriu, and V. Greavu-Şerban, “A systematic literature review on using natural language processing in software requirements engineering,” Electronics, vol. 13, no. 11, p. 2055, 2024. [2] A. M. Rosado da Cruz and E. F. Cruz, “Machine learning techniques for requirements engineering: A comprehensive literature review,” Software, vol. 4, no. 3, p. 14, 2025. [3] VDA QMC Working Group 13, “Automotive SPICE process assessment / reference model, version 3.1,” Verband der Automobilindustrie (VDA), Tech. Rep., 2017. [4] International Organization for Standardization, ISO 26262: Road Vehicles – Functional Safety, Std., 2018, parts 1–12. [5] T. Spijkman, X. de Bondt, F. Dalpiaz, and S. Brinkkemper, “Summarization of elicitation conversations to locate requirements-relevant information,” in International Working Conference on Requirements Engineering: Foundation for Software Quality (REFSQ), ser. Lecture Notes in Computer Science, vol. 13975. Springer, 2023, pp. 127–143. [6] L. Zhao, W. Alhoshan, A. Ferrari, K. J. Letsholo, M. A. Ajagbe, R. T. Batista-Navarro, and M. Sherwood, “Natural language processing for requirements engineering: A systematic mapping study,” vol. 54, no. 3. ACM, 2022, pp. 1–41. [7] N. Nascimento, A. Santos, and C. Lucena, “Large language models (LLMs) for requirements engineering (RE): A systematic literature review,” arXiv preprint arXiv:2509.11446, 2025. [8] H. Abdelaal, M. Kabir, K. N. Phan, and S. R. Fuad, “Abstractive text summarization: State of the art, challenges, and improvements,” arXiv preprint arXiv:2409.02413, 2024. [9] T. Jonsson, “Clustering and summarization of chat dialogues,” Master’s thesis, Linköping University, 2021. [10] J. Christensen, S. Soderland, G. Bansal, and Mausam, “Hierarchical summarization: Scaling up multi-document summarization,” in Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL). Association for Computational Linguistics, 2014, pp. 902–912. [11] S. Sharma et al., “Reducing redundancy in software testing: A K-means clustering approach to test case minimization,” Journal of Information Systems Engineering and Management, vol. 10, no. 3s, 2025. [12] C. Duan and J. Cleland-Huang, “Clustering support for automated tracing,” in Proceedings of the 22nd IEEE/ACM International Conference on Automated Software Engineering (ASE). ACM, 2007, pp. 244–253.
[13] S. Mohamad, A. Cailliau, and R. Darimont, “Identifying the requirement conflicts in SRS documents using sentence transformers and NER,” arXiv preprint arXiv:2206.13690, 2022. [14] N. Reimers and I. Gurevych, “Sentence-BERT: Sentence embeddings using siamese BERT-networks,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong, China: Association for Computational Linguistics, 2019, pp. 3982–3992. [15] C. Jain, S. Raje, and G. Deshpande, “A transformer-based approach for abstractive summarization of requirements from obligations in software engineering contracts,” Automated Software Engineering, vol. 30, no. 2, 2023. [16] J. Zhang, W. Hou, X. Tang, J. Chen, Y. Zhou, and Q. Wei, “RepoSummary: Feature-oriented summarization and documentation generation for code repositories,” arXiv preprint arXiv:2510.11039, 2025. [17] D. Jin, Z. Jin, X. Chen, and C. Wang, “MARE: Multi-agents collaboration framework for requirements engineering,” arXiv preprint arXiv:2405.03256, 2024. [18] A. E. Ezugwu, A. M. Ikotun, O. N. Oyelade, L. Abualigah, J. O. Agushaka, C. I. Eke, and A. A. Akinyelu, “A comprehensive survey of clustering algorithms: State-of-the-art machine learning applications, taxonomy, challenges, and future research prospects,” Engineering Applications of Artificial Intelligence, vol. 110, p. 104743, 2022. [19] M. Ester, H.-P. Kriegel, J. Sander, and X. Xu, “A density-based algorithm for discovering clusters in large spatial databases with noise,” in Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining (KDD). AAAI Press, 1996, pp. 226–231. [20] R. J. G. B. Campello, D. Moulavi, and J. Sander, “Density-based clustering based on hierarchical density estimates,” in Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD), ser. Lecture Notes in Computer Science, vol. 7819. Springer, 2013, pp. 160–172. [21] F. Pudlitz, F. Brokhausen, and A. Vogelsang, “Use case identification of natural language system requirements with graph-based clustering,” Design Science, vol. 9, 2023. [22] J. Mund, H. Femmer, D. Mendez, and J. Eckhardt, “Enhancing software requirements cluster labeling using Wikipedia,” IEEE Access, vol. 7, pp. 145 406–145 419, 2019. [23] M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, “BART: Denoising sequence-tosequence pre-training for natural language generation, translation, and comprehension,” pp. 7871–7880, 2020. [24] A. Ferrari, F. Dell’Orletta, A. Esuli, V. Gervasi, and S. Gnesi, “Natural language requirements processing: A 4d vision,” IEEE Software, vol. 34, no. 6, pp. 28–35, 2017, pURE dataset available at https://zenodo.org/ records/1414117. [25] F. Siavashi, D. Truscan, and O.-C. Granmo, “Research directions for using LLM in software requirement engineering: A systematic review,” Frontiers in Computer Science, vol. 7, 2025. [26] S. Masuda et al., “Generating high-level test cases from requirements using LLM: An industry study,” arXiv preprint arXiv:2510.03641, 2025. [27] S. Adabala et al., “Multi-step generation of test specifications using large language models,” in Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL), Industry Track. Association for Computational Linguistics, 2025. [28] M.-Y. Chow and M. Kitani, “Testing procedure generation based on testing requirements for automotive components with LLM fine-tuning,” in Natural Language Processing and Information Systems (NLDB 2025), ser. Lecture Notes in Computer Science, vol. 15837. Springer, 2026. [29] S. Wang, Y. Yu, R. Feldt, and D. Parthasarathy, “Automating a complete software test process using LLMs: An automotive case study,” arXiv preprint arXiv:2502.04008, 2025. [30] H. Koziolek, V. Ashiwal, S. Bandyopadhyay, and K. R. Chandrika, “Automated control logic test case generation using large language models,” arXiv preprint arXiv:2405.01874, 2024. [31] L. McInnes, J. Healy, and J. Melville, “Umap: Uniform manifold approximation and projection for dimension reduction,” arXiv preprint arXiv:1802.03426, 2018.