Unsupervised Multi-kernel Learning for Automated Algorithm Selection Yihang Lu1,2 , Tome Eftimov3 , and Carola Doerr2 Sorbonne Université, CNRS, INRAE, IRD, iEES, Paris, France 2 LIP6, Sorbonne Université, CNRS, Paris, France 3 Computer Systems Department, Jožef Stefan Institute, Ljubljana, Slovenia
arXiv:2607.19031v1 [cs.LG] 21 Jul 2026
1
Abstract. Automated algorithm selection in black-box optimization typically relies on supervised models that map landscape features to algorithm performance labels. Such models are costly to train, benchmarkdependent, and often fail to generalize to unseen problem classes. We study an unsupervised alternative: multi-kernel clustering over heterogeneous landscape representations, in which problem instances are grouped without using performance labels in the clustering stage, and the resulting clusters are mapped post hoc to solver recommendations through a strictly separated three-stage evaluation protocol. Drawing on two decades of advances in multiple kernel learning, we adopt a multi-kernel k-means formulation that jointly learns cluster assignments and kernel weights over four heterogeneous landscape views: ELA, DeepELA, DoE2Vec, and TransOptAS. On affine BBOB-derived selector tasks for Differential Evolution (DE) and Particle Swarm Optimization (PSO) at a fixed evaluation budget, we report mean±standard-deviation selector profiles over 50 independent random seeds for stochastic configurations. Multi-kernel clustering obtains the strongest mean profile on the DE portfolio and remains competitive with, and nominally ahead of, the leading baselines on the more compressed PSO portfolio, where differences among the best methods are small relative to stochastic variation. In representative median-seed runs used for visualization, the learned kernel weights retain ELA and TransOptAS while assigning zero weight to DeepELA and DoE2Vec, providing a task-specific interpretation of which representations are retained by the multi-kernel model for selectororiented grouping. Keywords: Automated Algorithm Selection · Multi-Kernel Learning · Unsupervised Clustering · Landscape Analysis · Black-Box Optimization
1
Introduction
Automated algorithm selection (AAS) is a central problem in black-box optimization and a key component of modern AutoML systems [11,9]. Given a problem instance, the goal is to identify, from a predefined portfolio, the algorithm most likely to perform well. The dominant approach casts AAS as a supervised learning problem: landscape features extracted from the problem instance are
2
Y. Lu et al.
mapped to algorithm performance labels obtained from extensive benchmarking [13,12]. Despite their empirical success on standard benchmarks, supervised selectors face fundamental limitations. Obtaining reliable performance labels requires exhaustive evaluation across large algorithm–problem combinations, incurring significant computational cost [7,29]. More critically, the resulting models are inherently benchmark-dependent: trained on specific suites such as BBOB [8], they tend to overfit to structural regularities of the training benchmark and fail to generalize to unseen problem classes [4,2]. This lack of generalization has been documented under leave-one-problem-out, problem-split, and cross-benchmark evaluation protocols [23,22], where even state-of-the-art supervised selectors often fail to outperform a simple single-best-solver baseline. Recent studies further show that the difficulty is not confined to the predictive model. Different landscape feature representations impose markedly different structural organizations on the same problem instances, leading to inconsistent similarity notions across views [4]. No single representation is known to robustly capture algorithm suitability across problem classes, which challenges the assumption underlying supervised AAS. Motivated by these observations, we propose an unsupervised alternative. Rather than predicting performance labels, we cluster optimization problem instances based solely on their landscape characteristics, and then map the resulting groups to solver recommendations post hoc. This separates the label-free learning of the partition from the later use of performance data for clusterto-solver routing and evaluation. Unsupervised clustering has previously been applied to analyze problem distributions and benchmark coverage [26,14,1]. Cluster-based algorithm selection is also known from systems such as CSHC and SNNAP [20,5]; our focus differs by studying continuous black-box optimization with unsupervised multi-kernel integration of heterogeneous landscape views. Modern AAS relies on multiple, heterogeneous landscape representations. Classical Exploratory Landscape Analysis (ELA) features [21] capture low-level statistical and geometric properties, while recent deep learning approaches— DoE2Vec [27], DeepELA [25], and TransOptAS [3]—learn transformation-aware embeddings from raw problem samples. These representations are heterogeneous in both dimensionality and semantics, making naive concatenation an unreliable integration strategy. Multiple kernel learning as a principled integration framework. Multiple kernel learning (MKL) provides a natural solution to the challenge of integrating heterogeneous representations[33,31]. Over the past two decades, the MKL literature has produced a rich family of methods that jointly learn cluster assignments and kernel weights over a collection of base kernels, without assuming commensurability across spaces [18,17,19]. State-of-the-art multi-kernel k-means (MKKM) methods offer structured sparsity, hierarchical kernel selection, and monotone descent properties that make them well-suited for integration tasks with heterogeneous input spaces[32,16,15]. We bring this body of work to the AAS setting,
Unsupervised Multi-kernel Learning for AAS
3
treating each landscape view as a separate kernel source and leveraging recent MKKM advances to learn sparse, selector-oriented problem groupings. The central question we address is not merely whether problem instances can be clustered, but whether the resulting structure supports a useful post-hoc cluster-to-solver mapping under EC-relevant evaluation criteria. Our empirical study is therefore organized around selector quality: clustering is learned in a strictly unsupervised manner, and the clusters are converted to solver recommendations only after training, using training-set performance statistics. The resulting selectors are evaluated by gap closure and selector cost. Our contributions are threefold: (i) we introduce multi-kernel learning as a principled framework for unsupervised algorithm selection from heterogeneous landscape views, using a structured kernel integration strategy instead of naive concatenation; (ii) we conduct an extensive selector-oriented evaluation on DE and PSO portfolios over 8,280 affine BBOB-derived instances, reporting mean± standard-deviation profiles over 50 independent random seeds for stochastic configurations; and (iii) we show that the learned kernel weights provide a taskspecific and interpretable view of which representations are retained by the multi-kernel model for selector-oriented grouping.
2
Methodology
2.1
Problem Formulation
Let P = {p1 , . . . , pn } be a set of n optimization problem instances and A = {a1 , . . . , ar } a portfolio of r algorithms. We focus on a strictly unsupervised clustering setting: no algorithm performance information is used to learn the clustering model. Performance labels are used only after clustering, for post-hoc routing and held-out evaluation. We formulate unsupervised AAS as a clustering problem: partition P into k groups, each representing a behavioral regime of problems with similar landscape characteristics. We treat k as a clustering-granularity parameter rather than forcing it to equal the portfolio size. The value k = 8 provides a moderate over-partition of the five-solver portfolios and is not interpreted as a one-to-one match between clusters and solvers. Multiple clusters are allowed to route to the same solver in the post-hoc mapping stage. Formally, we seek a clustering function f : P → {1, . . . , k} based solely on problem landscape features. The resulting cluster assignments are converted post hoc to solver recommendations via a cluster-to-solver mapping defined on the training partition. Each problem instance pi is characterized through m heterogeneous feature views. The v-th view is represented by X(v) ∈ Rn×dv , v = 1, . . . , m, where dv is the view dimensionality. We consider four heterogeneous landscape descriptions: ELA, DeepELA, DoE2Vec, and TransOptAS. All views are standardized independently to zero mean and unit variance. Naive concatenation of all views into a single feature vector implicitly assumes commensurability across heterogeneous spaces and may distort the geometry of
4
Y. Lu et al.
individual representations, suppressing view-specific structure that is informative for selector-relevant problem groupings. A principled alternative is to represent each view as a separate kernel and let the model learn their relative contributions. For each view X(v) , we construct a kernel matrix K(v) ∈ Rn×n encoding pairwise similarities in the v-th feature space (linear kernels unless stated otherwise). This yields a collection of m base kernels {K(v) }m v=1 , one per landscape representation. The multi-kernel formulation preserves the geometric integrity of each view while enabling the model to discover which representations are most informative for solver-relevant grouping. 2.2
Multi-Kernel k -Means for Unsupervised AAS
Multiple kernel k-means (MKKM) [33,31,32] is a mature and well-studied framework that jointly optimizes cluster assignments and kernel weights over a collection of base kernels. Over the past two decades, the MKL community has developed a rich family of MKKM variants with structured sparsity, hierarchical kernel selection, and scalable optimization schemes [17]. Let H ∈ Rn×k be the normalized cluster-indicator matrix, with Hic = −1/2 |Cc | if instance pi belongs to cluster c and 0 otherwise, so that H⊤ H = Ik . Let α = (α1 , . . . , αm ) be a non-negative view-weight vector, and for each view v, let wv = (wv1 , . . . , wvqv ) be a non-negative within-view kernel weight vector. Each view may provide one or more normalized candidate kernels, with qv = 1 recovering the one-kernel-per-view case. The combined kernel is K(α, W) =
m X v=1
αv
qv X
wvℓ K(v,ℓ) .
(1)
ℓ=1
We minimize a kernel k-means objective augmented with structured sparsity penalties: m h X i min Tr K(α, W) I − HH⊤ + λ Ωview (α) + γ αv Ωkernel (wv ),
H,α,W
(2)
v=1
subject to α ≥ 0, 1⊤ α = 1 and wv ≥ 0, 1⊤ wv = 1 for each v, together with cardinality constraints expressed through ∥α∥0 and ∥wv ∥0 to enforce sparse but non-collapsed support. The first term is the standard within-cluster dispersion, while Ωview and Ωkernel are respectively view-level and within-view kernel-level redundancy-control terms. The support constraints prevent degenerate solutions in which the combined kernel collapses to a single view. 2.3
Cluster-to-Solver Mapping and Evaluation
The overall workflow proceeds in three strictly separated stages, which we make explicit here to clarify the role of performance labels and the train/test boundary.
Unsupervised Multi-kernel Learning for AAS
5
Stage 1: Unsupervised clustering. The clustering model is fitted on the training partition using only landscape features. No algorithm performance information is used at this stage. The output is a clustering function f : P → {1, . . . , k} that assigns each problem instance to one of k groups. Stage 2: Post-hoc cluster-to-solver mapping. Once the clustering is fixed, performance labels from the training partition are used to assign a recommended solver to each cluster. For each cluster c, let Ictr denote its training instances and sia the normalized performance cost of solver a on instance i; lower values are better. We assign to cluster c the solver with the best mean training cost: 1 X sia . (3) a∗ (c) = arg min tr a∈A |Ic | tr i∈Ic
Performance labels are used exclusively in this routing step, only after unsupervised training is complete. Stage 3: Selector evaluation. Each test instance is assigned to a cluster by the learned model from Stage 1, then routed to the solver a∗ (c) from Stage 2. All evaluation metrics are computed exclusively on the held-out test partition Dte . Let âi be the solver recommended for test instance i. The selector cost is SC =
1 X siâi . |Dte | te
(4)
i∈D
The mean regret is defined as 1 X siâi − min sia . MR = te a∈A |D | te
(5)
i∈D
which measures the average excess cost of the recommended solver relative to the instance-wise best solver. Lower values indicate better selector quality. The single best solver (SBS) and virtual best solver (VBS) serve as lower and upper reference baselines, both computed on the test partition: 1 X sia , a∈A |D te | te
SBS = min
i∈D
VBS =
1 X min sia . a∈A |Dte | te
(6)
i∈D
Gap closure is defined as GC = (SBS − SC)/(SBS − VBS), with higher values indicating better selector quality. SBS corresponds to GC = 0 (no improvement over the global best fixed solver) and VBS corresponds to GC = 1 (perfect instance-level selection). We report SC and MR alongside GC to keep both raw cost and absolute regret visible.
3
Optimization
The objective in Eq. (2) is non-convex jointly in (H, α, W). We adopt an alternating minimization strategy that iterates between the following two steps.
6
Y. Lu et al.
Step 1: Update cluster assignments H. With (α, W) fixed, the combined kernel K(α, W) is fixed, and the objective reduces to a standard kernel k-means problem: min Tr K(I − HH⊤ ) . (7) H
This is solved by applying kernel k-means on the combined kernel matrix, which makes the objective monotonically non-increasing at each inner iteration [33]. Step 2: Update kernel weights (α, W). With H fixed, define the per-kernel dispersion score dvℓ = Tr K(v,ℓ) (I − HH⊤ ) . (8) The optimization over (α, W) with fixed H is performed under the simplex and hard support constraints. Because the latter can make the feasible set nonconvex, the update is treated as a support-constrained weight step [31,32]. The accepted update leaves the objective unchanged or decreases it. Convergence. The alternating updates therefore generate a monotonically nonincreasing sequence of objective values, which is bounded below because the kernel matrices are positive semi-definite and Tr[K(I − HH⊤ )] ≥ 0. With hard support constraints, the feasible set can be non-convex, so the convergence statement is limited to this monotonicity and termination, under the stated stopping criterion, at a fixed point of the alternating procedure. The complete procedure is summarized in Algorithm 1. Algorithm 1 Multi-Kernel k-Means for Unsupervised AAS Require: Kernel matrices {K(v,ℓ) }, number of clusters k, parameters λ, γ, maximum iterations T Ensure: Cluster assignments H, kernel weights α, W 1: Initialize αv = 1/m for all v; initialize wvℓ uniformly within each view; initialize H randomly 2: for t = 1 to T do 3: Compute K ← K(α, W) 4: Update H by kernel k-means on K 5: Compute dispersion scores dvℓ for all (v, ℓ) 6: Update (α, W) by solving the support-constrained weight sub-problem in Step 2 7: if convergence criterion met then 8: break 9: end if 10: end for 11: return H, α, W
4
Experimental Study
4.1
Experimental Protocol
We study unsupervised algorithm selection on affine BBOB-derived problem sets in dimension d = 10 [8,6,30], following the recombination protocol of recent
Unsupervised Multi-kernel Learning for AAS
7
AAS generalization studies [4,2]. The benchmark comprises 552 problem pairs and 8,280 generated instances. Two algorithm portfolios are evaluated: five DE variants [28] and five PSO variants [10], each treated as a separate selector task and denoted DE1–DE5 and PSO1–PSO5 in the benchmark performance matrix. All Table 1 comparisons are reported at budget B = 100 with k = 8. The value k = 8 provides a moderate over-partition of the five-solver portfolios and is not interpreted as a one-to-one match between clusters and solvers. We additionally include a k-sweep diagnostic over k ∈ {3, 5, 8, 10, 12} in Section 4.4 to assess how sensitive the selector utility is to this granularity choice. Problem instances are represented through four landscape views: ELA [21,24], DeepELA [25], DoE2Vec [27], and TransOptAS [3]. All views are standardized independently. Stochastic configurations are evaluated with the same 50 independent random seeds across views and baselines, and Table 1 reports mean±standard deviation. Agglomerative clustering is deterministic under this protocol and is therefore reported with zero standard deviation. We compare multi-kernel clustering against two families of baselines. Singleview baselines apply each of the four landscape views in isolation with three clustering algorithms: Gaussian Mixture Models (GMM), k-Means, and Agglomerative Clustering, yielding 12 configurations. Concatenation baseline naively merges all four views into a single feature vector and applies the same algorithms. Following the three-stage workflow defined in Section 2.3, clustering is trained on the training partition without labels, the cluster-to-solver mapping is constructed from training-partition performance statistics, and all selector metrics are evaluated on the held-out test partition. The primary evaluation metrics are Gap Closure (GC, ↑), Mean Regret (MR, ↓), and Selector Cost (SC, ↓). SBS and VBS serve as fixed reference bounds: for DE, (SBS, VBS) = (0.1484, 0.0664); for PSO, (SBS, VBS) = (0.1594, 0.1034). 4.2
Main Results
Table 1 reports the full selector profile across all configurations. Each row corresponds to a specific view–clustering combination. The best result within each group is underlined; the best mean per metric is in bold. View heterogeneity has a material impact. The performance of single-view selectors varies substantially, confirming that the choice of landscape representation materially affects unsupervised selector utility. On DE, the strongest singleview mean profile is obtained by ELA+k-Means (GC=0.3253 ± 0.0100), whereas DoE2Vec and TransOptAS remain close to the SBS reference under most clustering choices. On PSO, the single-view differences are much more compressed: the strongest single-view mean GC is DeepELA+k-Means (0.0486 ± 0.0061), followed closely by ELA-based variants. This spread supports the use of view-aware integration rather than assuming that one representation is uniformly reliable. Naive concatenation is competitive but not uniformly decisive. At k = 8, concatenation is a strong baseline on DE: Concat+GMM obtains GC=0.3562 ± 0.0075
8
Y. Lu et al.
Table 1. Selector profile on DE and PSO at the fixed evaluation budget and k = 8. Stochastic configurations report mean±standard deviation over 50 independent random seeds; deterministic agglomerative rows have zero standard deviation. GC ↑: Gap Closure. MR ↓: Mean Regret. SC ↓: Selector Cost. Best mean per metric is in bold; best mean within each group is underlined. Reference bounds—DE: SBS = 0.1484, VBS = 0.0664; PSO: SBS = 0.1594, VBS = 0.1034. DE
PSO
Group
Method
GC ↑
MR ↓
SC ↓
GC ↑
MR ↓
SC ↓
All Views (Concat)
GMM k-Means Agg.
0.3562±0.0075 0.3556±0.0088 0.3239±0.0000
0.0528±0.0006 0.0528±0.0007 0.0554±0.0000
0.1192±0.0006 0.1192±0.0007 0.1218±0.0000
0.0494±0.0020 0.0477±0.0019 0.0518±0.0000
0.0533±0.0001 0.0534±0.0001 0.0531±0.0000
0.1567±0.0001 0.1568±0.0001 0.1565±0.0000
Single View (DeepELA)
GMM k-Means Agg.
0.1879±0.0400 0.2719±0.0131 0.2728±0.0000
0.0666±0.0033 0.0597±0.0011 0.0596±0.0000
0.1330±0.0033 0.1261±0.0011 0.1260±0.0000
0.0481±0.0111 0.0486±0.0061 0.0409±0.0000
0.0533±0.0006 0.0533±0.0003 0.0537±0.0000
0.1567±0.0006 0.1567±0.0003 0.1571±0.0000
Single View (DoE2Vec)
GMM k-Means Agg.
0.0002±0.0009 0.0002±0.0010 0.0000±0.0000
0.0820±0.0001 0.0820±0.0001 0.0820±0.0000
0.1484±0.0001 0.1484±0.0001 0.1484±0.0000
0.0225±0.0062 0.0201±0.0052 0.0077±0.0000
0.0548±0.0003 0.0549±0.0003 0.0556±0.0000
0.1582±0.0003 0.1583±0.0003 0.1590±0.0000
Single View (ELA)
GMM k-Means Agg.
0.2960±0.0320 0.3253±0.0100 0.3066±0.0000
0.0577±0.0026 0.0553±0.0008 0.0569±0.0000
0.1241±0.0026 0.1217±0.0008 0.1232±0.0000
0.0436±0.0028 0.0475±0.0022 0.0482±0.0000
0.0536±0.0002 0.0534±0.0001 0.0533±0.0000
0.1570±0.0002 0.1568±0.0001 0.1567±0.0000
GMM Single View k-Means (TransOptAS) Agg.
0.0093±0.0144 0.0009±0.0026 0.0000±0.0000
0.0812±0.0012 0.0819±0.0002 0.0820±0.0000
0.1476±0.0012 0.1483±0.0002 0.1484±0.0000
0.0316±0.0037 0.0315±0.0037 0.0322±0.0000
0.0543±0.0002 0.0543±0.0002 0.0542±0.0000
0.1577±0.0002 0.1577±0.0002 0.1576±0.0000
Multi-Kernel
MKKM 0.3605±0.0042 0.0524±0.0003 0.1188±0.0003 0.0521±0.0032 0.0531±0.0002 0.1565±0.0002
and Concat+k-Means obtains GC=0.3556±0.0088. These values are higher than the strongest single-view DE baseline but remain slightly below MKKM. On PSO, concatenation also lies in the leading group, with Concat+Agg. reaching GC=0.0518 and Concat+GMM reaching GC=0.0494 ± 0.0020. Thus, the results do not support treating concatenation as a weak baseline; rather, they show that structured multi-kernel integration provides a small but useful improvement over strong concatenation baselines in the k = 8 setting. Multi-kernel clustering gives the strongest mean profile, with different strength on DE and PSO. On DE, MKKM obtains the strongest mean selector profile in Table 1, with GC=0.3605±0.0042, MR=0.0524±0.0003, and SC=0.1188±0.0003. In relative GC terms, this corresponds to about +10.8% over the strongest singleview mean baseline (ELA+k-Means: 0.3253 ± 0.0100) and about +1.2% over the strongest concatenation mean baseline (Concat+GMM: 0.3562 ± 0.0075). The latter gain is only about 0.0043 GC, approximately one standard deviation of the MKKM GC values, so the percentage should be read as a marginal advantage whose interpretation is supported by the paired check below rather than as a large separation. On PSO, MKKM is nominally ahead by mean GC, MR, and SC, reaching GC=0.0521±0.0032, MR=0.0531±0.0002, and SC=0.1565±0.0002. However, the PSO margin over Concat+Agg. (GC=0.0518) is extremely small and lies well within the stochastic standard deviation of MKKM. We therefore interpret the PSO result as competitive and mildly favorable, not as strong dominance.
0.00
0.00
0.54
3
0.28
0.00
0.00
0.72
0.54
0.00
0.00
0.46
0.46
0.00
0.00
0.54
0.22
0.00
0.00
0.78
0.34
0.00
0.00
0.66
ela
deepela
doe2vec
transoptas
6 7
0
2
0.46
1
0.50
0.32
0.5
2
0.00
0.35
0.4
3
0.00
0.48
4
1
0.50
0.58
0.47
0.7 0.6
0.00
0.00
0.42
0.00
0.00
0.68
0.00
0.00
0.65
0.00
0.00
0.52
0.00
0.00
0.53
5
0.46
0.19
0.00
0.00
0.81
6
0.00
0.37
0.00
0.00
0.63
7
0
0.00
5
0.54
4
Unsupervised Multi-kernel Learning for AAS
0.53
0.00
0.00
0.47
ela
deepela
doe2vec
transoptas
0.3 0.2 0.1 0.0
9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0
Fig. 1. Per-cluster normalized view contributions induced by multi-kernel clustering for DE (left) and PSO (right) in representative median-seed runs at k = 8; the representative run is the seed whose test GC is closest to the median over the 50 seeds (DE seed 9, PSO seed 18). DoE2Vec and DeepELA have zero normalized contribution in these runs. ELA and TransOptAS carry the active normalized contribution, with cluster-specific and family-specific balance.
Paired statistical check. Because the stochastic configurations use aligned seeds, we additionally compare GC values with two-sided Wilcoxon signed-rank tests. The direction statements below refer to the observed mean GC differences, while the two-sided p-values assess whether the paired differences are nonzero. On DE, MKKM has higher mean GC than the strongest stochastic concatenation baseline, Concat+GMM, and the paired difference is significant (p = 2.7 × 10−4 ). On PSO, MKKM has higher mean GC than the strongest stochastic concatenation baseline, Concat+GMM, and the paired difference is significant (p = 1.5×10−5 ), but it is not significantly different from the deterministic Concat+Agg. reference when that singleton value is treated as a fixed comparator (p = 0.965). This supports the intended interpretation: a clear DE improvement, but only a competitive and non-dominant PSO result. PSO is harder across the compared methods. All selectors achieve substantially lower GC on PSO than on DE. This is consistent with prior observations that static landscape features have limited discriminative power for PSO-type portfolios [4]. The reported mean±standard-deviation profiles reinforce this interpretation: PSO gains remain compressed even for the strongest methods, and small numerical differences should not be over-interpreted. 4.3
Kernel Weight Analysis
Figure 1 shows the per-cluster normalized view contributions induced by the multi-kernel model in representative median-seed runs at k = 8. We define the representative run as the seed whose test GC is closest to the median over the 50 seeds, yielding seed 9 for DE and seed 18 for PSO. In both algorithm families, DoE2Vec and DeepELA have zero contribution in these representative runs, while ELA and TransOptAS account for the active contribution. This should
0.5
0.559
0.149*
0.718
0.703
0.252
0.4
0.534
0.097*
0.742
0.566
0.192
0.3
0.530
0.080*
0.726
0.669
0.116
0.835
0.298
0.184
0.685
0.102*
DE1
DE2
DE3
DE4
DE5
0.2 0.1
0
0.6
0.229*
0.808
0.150
0.818
0.300
1
0.282
0.631
4
0.678
0.666
5
0.653
0.387
6
0.120*
0.714
7
0.609
0.8
0.143* 0.292*
0.668
0.313
0.667
0.439
0.7
2
0.7
0.352
0.605
0.324*
0.589
0.375
0.6
3
0.8
0.234
0.108*
0.790
0.108
0.799
0.247
0.5
4
0.271
0.614
0.118*
0.810
0.123
0.814
0.272
0.4
5
0.359
0.542
0.051*
0.799
0.053
0.799
0.148
0.3
6
0.418
0.104*
0.140*
0.778
0.157
0.789
0.273
0.2
7
0.113*
0.511
2
0.636
3
0
Y. Lu et al.
1
10
0.153*
0.675
0.163
0.695
0.215
0.1
1
PSO
2
PSO
3
PSO
4
PSO
5
PSO
Fig. 2. Cluster-to-solver routing for DE (left) and PSO (right) in the same representative median-seed runs as Figure 1. Lower cost is better; row-wise best solvers are marked (∗). Patterns are empirical regularities, not intrinsic solver classes.
not be read as evidence that DoE2Vec or DeepELA are generally uninformative for AAS; it only indicates that, under the present benchmark, fixed budget, and regularization regime, they do not add selector-relevant variation beyond the retained views. The relative ELA–TransOptAS contribution is cluster-dependent. On DE, the ELA contribution ranges from 0.22 to 0.54 across the eight clusters; on PSO, it ranges from 0.19 to 0.58. Equivalently, TransOptAS ranges from 0.46 to 0.78 on DE and from 0.42 to 0.81 on PSO. This variation suggests that the learned solution is sparse but not collapsed to a single active representation. The topographic map later in Figure 3 redistributes the cluster-level view-weight information in Figure 1 over problem-pair space and is shown for completeness rather than as an additional quantitative result. Figure 2 provides a post-hoc interpretability check by examining how the unsupervised clusters are routed to solver recommendations. On DE, the routing remains clearly non-degenerate at k = 8: clusters 0, 1, 2, 4, 5, and 6 route to DE2, whereas clusters 3 and 7 route to DE5. The best cluster-level mean costs range from 0.080 to 0.229, and several clusters show clear separation between the best and second-best solver. For example, cluster 2 routes to DE2 with mean cost 0.120, while the next solver among the top three has mean cost 0.282. On PSO, the routing is more compressed but still structured. Clusters 0, 1, 3, 4, 5, 6, and 7 route to PSO1, whereas cluster 2 routes to PSO3. The bestversus-second-best margins are much smaller than on DE; in some clusters, such as clusters 3 and 5, PSO1 and PSO3 are almost tied. This compressed routing is consistent with the small selector gains reported in Table 1. A quantitative contrast reinforces this interpretation. From the row-wise costs in Figure 2, the average gap between the best and second-best solver among the top three solvers is about 0.116 on DE, but only about 0.011 on PSO. We therefore interpret the larger DE gains and the marginal PSO gains as consequences of different solver-response separability under the present benchmark, portfolio, and EC cost criterion.
Unsupervised Multi-kernel Learning for AAS
11
High (1.0) _p8
p19_p4
p12_p10
p17
_p1 p16
p21
p9_p
p20_
22
p18_p1
p18_p16
p1
1
8_p
_p2
15
p1_p1 3
p16_p1 7
p8
p4_p20
p12_p10
5
p6_
_p1
p5
p19
20
p23_
p1_p
p6
p14_p21
p17_p22
p9
3
_p7
_p2
4
p23_p2
p2_p3
p4
Low (0.0) Fig. 3. Problem-pair-level topographic maps for DE (top) and PSO (bottom), using the same representative median-seed MKKM runs as Figure 1. From inner to outer ring, the four views are DeepELA, DoE2Vec, ELA, and TransOptAS. This figure redistributes the cluster-level view-weight information in Figure 1 over problem-pair space and is shown for completeness. The visible contrast is mainly driven by the ELA–TransOptAS balance of the cluster assigned to each problem pair.
12
Y. Lu et al.
4.4
Cluster Granularity Sensitivity
Figure 4 summarizes a diagnostic sweep over k ∈ {3, 5, 8, 10, 12} for MKKM and two simple diagnostic baselines, Concat+k-Means and ELA+k-Means. This sweep assesses the sensitivity of selector utility to the clustering granularity. Table 1 reports the fixed k = 8 comparison, where k = 8 is used as a moderate over-partition of the five-solver portfolios. For k < |A| = 5, the selector can use at most k distinct solvers, so those points are structurally constrained. For k > |A|, multiple clusters may route to the same solver and the extra clusters act as a finer partition of the problem space.
ELA + KMeans
Concat + KMeans
DE
0.06
Gap Closure
0.3
MKKM
PSO
0.05 0.04
0.2
0.03 0.02
0.1
0.01 0.0 4
6
k
8
10
12
0.00
4
6
k
8
10
12
Fig. 4. Cluster-granularity diagnostic over k ∈ {3, 5, 8, 10, 12} for DE and PSO, evaluated by Gap Closure. Curves summarize 50-seed stochastic profiles for the compared configurations. Table 1 reports the fixed k = 8 comparison; the other k values are included only as a granularity-sensitivity diagnostic.
On DE, k = 8 lies on a competitive plateau for MKKM rather than being an isolated optimum. The sweep also shows that Concat+k-Means remains strong for nearby values of k, supporting the cautious interpretation that MKKM improves over a strong concatenation baseline rather than a weak one. On PSO, the selector landscape is compressed at k = 8, and larger granularities may further improve MKKM. We therefore treat the larger-k PSO results as evidence that the role of k as a resolution parameter is family-specific, and as a direction for future tuning rather than as the central claim of this study. Figure 5 reports a compact sensitivity sweep around the sparse, non-collapsed regularization region used in the main experiments at k = 8. The heatmaps summarize median Gap Closure over the stochastic seeds for each (λ, γ) cell. On DE, the median GC values in the explored grid are concentrated around the 0.36 level, with the best cells near 0.365. On PSO, the absolute range is narrower in practical terms, with the leading cells around 0.057 and many cells remaining close to the 0.05 level.
Unsupervised Multi-kernel Learning for AAS
0.25
lambda
0.05 0.35
0.354
0.5
0.025
0.15
0.25
gamma
0.35
0.055
0.15
0.35
0.356
0.5
0.358
0.056
0.054
lambda
0.15
0.362 0.360
0.057
0.25
0.05
0.364
0.5
13
0.053 0.052 0.051 0.050
0.025
0.15
0.25
gamma
0.35
0.5
Fig. 5. Sensitivity of multi-kernel clustering to (λ, γ) on DE (left) and PSO (right) at k = 8, evaluated by Gap Closure. The sweep shows broad competitive regions rather than a single isolated optimum.
The near-flat EC metric surfaces do not imply that the clustering objective itself is insensitive to (λ, γ). Rather, within the explored range, different unsupervised solutions can induce similar post-hoc routing decisions and therefore similar selector-level metrics. The practical implication is that the method does not require a highly precise regularization setting to achieve competitive selector utility in the present benchmark. At the same time, the PSO surface and the k-sweep both caution against overclaiming: small absolute GC differences in this family should be interpreted as weak evidence unless confirmed across broader benchmark and portfolio settings.
5
Conclusion and Discussion
We introduced unsupervised multi-kernel learning as a principled framework for automated algorithm selection from heterogeneous landscape representations. By clustering problem instances directly from landscape characteristics and treating each representation as a separate kernel source, the approach preserves view-specific geometry and avoids forcing heterogeneous views into a single concatenated feature space. The experiments report mean±standard-deviation selector profiles over 50 independent random seeds at the fixed evaluation budget and k = 8. On the affine BBOB-derived selector tasks studied here, MKKM obtains the strongest mean profile among the compared k = 8 configurations on DE, with GC=0.3605 ± 0.0042, MR=0.0524 ± 0.0003, and SC=0.1188 ± 0.0003. This advantage is clear relative to the strongest single-view mean baseline and modest relative to the strongest concatenation mean baseline. On PSO, MKKM is also nominally ahead by mean GC, MR, and SC, but the margin over the strongest
14
Y. Lu et al.
concatenation baseline is very small relative to the stochastic standard deviation and should not be interpreted as strong dominance. We therefore interpret the PSO result as competitive and mildly favorable rather than as strong dominance. The learned kernel weights in representative median-seed runs retain ELA and TransOptAS while assigning zero weight to DeepELA and DoE2Vec. This should be interpreted conditionally on the present benchmark, fixed budget, portfolio, and regularization regime, not as a universal statement about the value of these representations. The routing analysis further indicates markedly stronger solver differentiation on DE than on PSO, with an average best-versussecond-best cluster-level gap of approximately 0.116 on DE versus 0.011 on PSO. Together, these results suggest that multi-kernel clustering is most useful when the learned clusters separate solver-response regimes with sufficiently different cost profiles. Several directions follow naturally from this work. It will be important to test the robustness of these findings across other benchmarks, problem dimensions, budgets, and portfolio compositions, and to examine richer kernel families that may capture nonlinear structure beyond the present setting. At the decision level, softer or uncertainty-aware cluster-to-solver mappings may reduce the gap to the Virtual Best Solver, while evaluations on real-world optimization problems and mixed search spaces will be necessary to assess practical generalization. A particularly promising direction is to combine the present unsupervised grouping stage with lightweight supervised refinement using only a small amount of labeled data, thereby exploring a middle ground between fully unsupervised selection and label-intensive supervised AAS. Overall, these results indicate that multi-kernel learning provides a competitive, interpretable, and label-free foundation for unsupervised algorithm selection, while also motivating further work on evaluation principles that align unsupervised structure discovery with downstream selector utility. Acknowledgments. Y. Lu acknowledges the support from Marie SkłodowskaCurie Grant No. 101081674 (SOUND.AI) of the European Union’s Horizon Europe research and innovation programme. T. Eftimov acknowledges the support of Horizon Europe ERA Chair AutoLearn-SI (101187010), as well as the Slovenian Research Agency through programme grant No. P2-0098 and project grants No. J2-70078 and No. GC-0001. C. Doerr acknowledges funding by the European Union (ERC, “dynaBBO”, grant no. 101125586). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Council Executive Agency. Neither the European Union nor the granting authority can be held responsible for them. This research was also jointly funded by the French National Research Agency (ANR-23-CE23-0035) and the German Research Foundation (DFG; LI 2801/7-1), through project Opt4DAC.
Unsupervised Multi-kernel Learning for AAS
15
References 1. Cenikj, G., Lang, R.D., Engelbrecht, A.P., Doerr, C., Korošec, P., Eftimov, T.: Selector: Selecting a representative benchmark suite for reproducible statistical comparison. In: Proceedings of the Genetic and Evolutionary Computation Conference. pp. 620–629 (2022) 2. Cenikj, G., Petelin, G., Eftimov, T.: A cross-benchmark examination of feature-based algorithm selector generalization in single-objective numerical optimization. Swarm and Evolutionary Computation 87, 101534 (2024) 3. Cenikj, G., Petelin, G., Eftimov, T.: Transoptas: Transformer-based algorithm selection for single-objective optimization. In: Proceedings of the Companion Conference on Genetic and Evolutionary Computation. pp. 403–406 (2024) 4. Cenikj, G., Petelin, G., Seiler, M., Cenikj, N., Eftimov, T.: Landscape features in singleobjective continuous optimization: Have we hit a wall in algorithm selection generalization? Swarm and Evolutionary Computation 94, 101894 (2025) 5. Collautti, M., Malitsky, Y., Mehta, D., O’Sullivan, B.: SNNAP: Solver-based nearest neighbor for algorithm portfolios. In: Machine Learning and Knowledge Discovery in Databases – ECML PKDD 2013. Lecture Notes in Computer Science, vol. 8190, pp. 435–450. Springer (2013) 6. Dietrich, K., Mersmann, O.: Increasing the diversity of benchmark function sets through affine recombination. In: Parallel Problem Solving from Nature – PPSN XVII. pp. 590– 602. Springer (2022) 7. Hansen, N., Auger, A., Ros, R., Mersmann, O., Tušar, T., Brockhoff, D.: Coco: A platform for comparing continuous optimizers in a black-box setting. Optimization Methods and Software 36(1), 114–144 (2021) 8. Hansen, N., Finck, S., Ros, R., Auger, A.: Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions. Tech. Rep. RR-6829, INRIA (2009) 9. Hutter, F., Kotthoff, L., Vanschoren, J. (eds.): Automated Machine Learning - Methods, Systems, Challenges. The Springer Series on Challenges in Machine Learning, Springer (2019) 10. Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks. vol. 4, pp. 1942–1948 (1995) 11. Kerschke, P., Hoos, H.H., Neumann, F., Trautmann, H.: Automated algorithm selection: Survey and perspectives. Evolutionary Computation 27(1), 3–45 (2019) 12. Kerschke, P., Trautmann, H.: Automated algorithm selection on continuous black-box problems by combining exploratory landscape analysis and machine learning. Evol. Comput. 27(1), 99–127 (2019) 13. Kostovska, A., Jankovic, A., Vermetten, D., Džeroski, S., Eftimov, T.: Comparing algorithm selection approaches on black-box optimization problems. In: Proceedings of the Companion Conference on Genetic and Evolutionary Computation. pp. 495–498 (2023) 14. Lang, R.D., Engelbrecht, A.P.: An exploratory landscape analysis-based benchmark suite. Algorithms 14(3), 78 (2021) 15. Lu, J., Lu, Y., Wang, R., Nie, F., Li, X.: Multiple kernel k-means clustering with simultaneous spectral rotation. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 4143–4147. IEEE (2022) 16. Lu, Y., Lu, J., Wang, R., Nie, F.: Discrete multi-kernel k-means with diverse and optimal kernel learning. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 4153–4157 (2022) 17. Lu, Y., Xin, H., Wang, R., Nie, F., Li, X.: Scalable multiple kernel k-means clustering. In: Proceedings of the 31st ACM International Conference on Information and Knowledge Management (CIKM). pp. 4279–4283. ACM (2022) 18. Lu, Y., Zheng, X., Wang, R., Nie, F., Li, X.: A unified framework for discrete multikernel k-means with kernel diversity regularization. In: Proceedings of the International Conference on Pattern Recognition (ICPR). pp. 4934–4940 (2022) 19. Lu, Y., Zheng, X., Lu, J., Wang, R., Nie, F., Li, X.: Self-paced and discrete multiple kernel k -means. In: CIKM. pp. 4284–4288. ACM (2022) 20. Malitsky, Y., Sabharwal, A., Samulowitz, H., Sellmann, M.: Algorithm portfolios based on cost-sensitive hierarchical clustering. In: Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence (IJCAI). pp. 608–614 (2013) 21. Mersmann, O., Bischl, B., Trautmann, H., Preuss, M., Weihs, C., Rudolph, G.: Exploratory landscape analysis. In: Proceedings of the Genetic and Evolutionary Computation Conference. pp. 829–836 (2011) 22. Nikolikj, A., Cenikj, G., Ispirova, G., Vermetten, D., Lang, R., Engelbrecht, A.P., Doerr, C., Korošec, P., Eftimov, T.: Assessing the generalizability of a performance predictive model. In: Proceedings of the Companion Conference on Genetic and Evolutionary Computation. pp. 311–314 (2023) 23. Nikolikj, A., Kostovska, A., Cenikj, G., Doerr, C., Eftimov, T.: Generalization ability of feature-based performance prediction models: A statistical analysis across benchmarks. In: IEEE Congress on Evolutionary Computation (2024)
16
Y. Lu et al.
24. Prager, R.P., Trautmann, H.: Pflacco: Feature-based landscape analysis of continuous and constrained optimization problems in python. Evolutionary Computation 32(3), 211–216 (2024) 25. Seiler, M.V., Kerschke, P., Trautmann, H.: Deep-ela: Deep exploratory landscape analysis with self-supervised pretrained transformers for single- and multi-objective continuous optimization problems. Evolutionary Computation 33(4), 513–540 (2025) 26. Skvorc, U., Eftimov, T., Korošec, P.: Understanding the problem space in single-objective numerical optimization using exploratory landscape analysis. Applied Soft Computing 90, 106138 (2020) 27. van Stein, B., Long, F.X., Frenzel, M., Krause, P., Gitterle, M., Bäck, T.: Doe2vec: Deeplearning based features for exploratory landscape analysis. In: Proceedings of the Companion Conference on Genetic and Evolutionary Computation. pp. 515–518 (2023) 28. Storn, R., Price, K.: Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization 11(4), 341–359 (1997) 29. Tornede, A., Wever, M., Hüllermeier, E.: Extreme algorithm selection with dyadic feature representation. In: Appice, A., Tsoumakas, G., Manolopoulos, Y., Matwin, S. (eds.) Discovery Science - 23rd International Conference, DS 2020, Thessaloniki, Greece, October 19-21, 2020, Proceedings. Lecture Notes in Computer Science, vol. 12323, pp. 309–324. Springer (2020) 30. Vermetten, D., Ye, F., Doerr, C.: Using affine combinations of bbob problems for performance assessment. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO). pp. 873–881 (2023), best Paper Award in ENUM Track 31. Wang, R., Lu, J., Lu, Y., Nie, F., Li, X.: Discrete multiple kernel k-means. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI). pp. 3111–3117 (2021) 32. Wang, R., Lu, J., Lu, Y., Nie, F., Li, X.: Discrete and parameter-free multiple kernel k-means. IEEE Transactions on Image Processing 31, 2796–2808 (2022) 33. Yu, S., Tranchevent, L.C., Liu, X., Glänzel, W., Suykens, J.A.K., De Moor, B., Moreau, Y.: Optimized data fusion for kernel k-means clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence 34(5), 1031–1039 (2012)