WCC: A Learning-Based Framework for Deciding When to Apply Cooperative Coevolution in Large-Scale Optimization

WCC: WHEN TO APPLY COOPERATIVE COEVOLUTION

1

WCC: A Learning-Based Framework for Deciding When to Apply Cooperative Coevolution in Large-Scale Optimization

arXiv:2610.37329v1 [cs.LG] 27 Oct 2026

Wenjie Qiu, Hepei Li, Jianfeng Wang, Jun Zhang, Fellow, IEEE and Yue-Jiao Gong, Senior Member, IEEE

Abstract—Cooperative Coevolution (CC) and NonDecomposition Algorithms (NDAs) are two primary approaches for large-scale global optimization (LSGO). Although both have achieved success, they exhibit markedly different performance across problem types and each faces approachspecific limitations, leaving the question of “when to use CC” unresolved. We address this gap with WCC, a novel learningbased framework that dynamically decides when to deploy CC in LSGO. WCC employs a streamlined design for efficient adaptive learning. We formulate the decision as a Markov decision process, designing a targeted feature set, a robust reward, and a tailored network architecture to provide the agent with rich information about the problem state and optimization progress. Our auto-regressive policy enables WCC to perform fine-grained, dynamic switching between CC and NDAs while conditionally allocating the computational budget without expert specifications. WCC shows strong competitiveness and superior generalization across problems, optimization horizons, and baseline optimizers. It realizes a “train-once and reuse” paradigm under a low budget, significantly curbing LSGO cost. Additionally, in-depth analyses yield insights not captured by existing theory, offering practical guidance on when and how to effectively deploy CC. Index Terms—Large-scale Global Optimization, Cooperative Coevolution, Meta-Black-Box Optimization.

I. I NTRODUCTION Large-scale global optimization (LSGO), typically involving over 1000 variables, represents a challenging class of blackbox optimization problems with widespread real-world applications [1]. The primary challenge of LSGO stems from the “curse of dimensionality,” where the rapidly expanding search space often renders conventional algorithms ineffective. Cooperative Coevolution (CC) based on the “divide-and-conquer” principle has become one of the most commonly adopted frameworks in LSGO [2], [3]. In addition, many classical algorithms have been extended to large-scale variants to tackle LSGO, which are referred to as non-decomposition algorithms (NDAs), in contrast to those developed under the CC. Both CC and NDAs, despite their successes, face notable challenges. For CC, much research focuses on effective problem decomposition, where it often demonstrates competitive performance on conventional separable problems. However, Wenjie Qiu, Hepei Li, and Yue-Jiao Gong are with the School of Computer Science and Engineering, South China University of Technology, Guangzhou 510006, China (e-mail: wukongqwj@gmail.com; csscholarpei@mail.scut.edu.cn; gongyj@scut.edu.cn). Jianfeng Wang is with Taiyuan University of Technology, Taiyuan 030024, China (e-mail: wangjianfeng@tyut.edu.cn). Jun Zhang is with Nankai University, Tianjin 300350, China, and also with Hanyang University ERICA, Ansan 15588, South Korea (e-mail: junzhang@ieee.org).

CC relies on strong problem assumptions (e.g., separability and high-precision evaluations). Existing decomposition strategies have inherent biases, fail to cover all forms of separability, and are prone to decomposition errors [4]. Once ideal decomposition is lost, performance drops sharply, making CC often ill-suited for real-world problems dominated by non-separability [5]. NDAs avoid complex variable decomposition and an increasing number of advanced variants have demonstrated superior performance compared to CCbased algorithms on increasingly complex problem instances. However, NDAs forgo exploiting structural priors that often exist in real problems, inflating the search space and computational cost, which can easily lead to optimization stagnation [6]. As dimensionality keeps growing, a bottleneck becomes inevitable—making decomposition unavoidable [7]. These differences and challenges result in highly divergent performance across problems and, in some cases, optimization stagnation in one approach—yielding outcomes that differ by several orders of magnitude. The distinctly different optimization characteristics of both approaches have made the question of “when to use CC” a critical decision point. Many studies have theoretically analyzed the convergence of CC from perspectives such as Nash equilibrium [5]. However, the black-box nature, heterogeneity, and numerous unknown characteristics of LSGO make such theoretical analysis difficult and often case-specific. Some studies have attempted to combine these two approaches in practice based on expert knowledge. However, these hybrid methods further indicate that the question of when to use CC depends not only on the problem class but also on the optimization stage [8]. Applying CC at an inappropriate stage can severely degrade optimization performance, requiring analysis of phase-specific characteristics and NDA behavior, thereby compounding theoretical challenges [9]. To summarize, existing methods rely on heuristic, problem-specific switching schedules with limited cross-problem generalization, leaving the “when to use CC” decision an open bottleneck. Recognizing that determining “when to use CC” is essentially a meta-level algorithm design problem rather than one concerning the internal mechanisms of CC, we introduce Meta-Black-Box Optimization (MetaBBO) [10], [11] into LSGO. As an emerging research direction in recent years, MetaBBO follows a bi-level structure, consisting of a metalayer and an optimization-layer, and aims to automate the design of BBO algorithms by constructing intelligent agents at the meta-layer as replacements for human experts. Many studies have demonstrated their excellent generalization ability and strong balance between exploration and exploitation [12], [13]. Furthermore, the actions of their learning-based agents

WCC: WHEN TO APPLY COOPERATIVE COEVOLUTION

often provide valuable insights to researchers [14], [15]. Since the decision of “when to use CC” is inherently not a oneshot choice, but a sequential decision-making process and a dynamic control problem under uncertainty, the appropriate decision relies heavily on the problem structure, the optimization stage and the current search status. To address these challenges, we formulate the process as a Markov decision process (MDP) and use deep reinforcement learning (DRL) to adaptively configure efficient algorithms for different LSGO problems. Instead of merely deciding whether to use CC or NDAs, the agent also determines the allocation of function evaluations (FEs) to CC and NDAs at each stage, enabling dynamic switching between the two approaches across optimization stages. To this end, we design an auto-regressive network as the agent architecture. Trained on a diverse set of LSGO problems, the agent learns to make optimal action selections at each optimization step based on a set of welldesigned features. We refer to this architecture as the Whento-use Cooperative Coevolution (WCC) framework. The main contributions of this paper are as follows:

2

The remainder of this paper is organized as follows: Section II presents the methodology, Section III reports the results and insights, and Section IV concludes the paper.

II. M ETHODOLOGY WCC formulates CC/NDA deployment as an MDP, where the agent observes the current optimization state sti ∈ S and selects an action at ∈ A that jointly determines the optimization mode and computational budget. The policy πθ is learned to maximize the expected discounted optimization performance over the target problem distribution P: N

π ∗ ≈ arg max πθ ∈Π

T

1 X X t−1 γ perf sti , πθ (sti ) | fi , N i=1 t=1

(1)

where γ is the discount factor. We optimize πθ using PPO [16].

A. MDP Formulation An intuitive yet effective framework to guide and analyze LSGO: We propose WCC, an innovative framework to learn when to use CC in LSGO. Framed as an MDP and trained via DRL, our agent reduces reliance on scarce expert knowledge and offers a new perspective on this longstanding question. Unlike heuristic designs, WCC offers fine-grained decisions that explicitly determine which optimization stages should employ CC or NDAs. In addition, the framework allows complex problems to be addressed with remarkable clarity, providing the possibility of drawing insightful conclusions. • Streamlined design for efficient adaptive learning: Based on the characteristics of LSGO, we design a robust reward function and targeted feature set for WCC to provide the agent with rich signals about both the problem and the optimization progress. An auto-regressive policy is designed to first choose CC vs. NDA, then allocate FEs conditionally, enabling step-wise switching during optimization. A network architecture is carefully designed for this purpose, enabling the agent to adaptively navigate various scenarios. • Comprehensive generalization performance: WCC demonstrates generalization not only across different LSGO problem groups, but also across optimization horizons and baseline algorithms: WCC can be trained once under low FEs and instantiated with different underlying optimizers, substantially reducing WCC’s training cost. • Insightful analysis for future advancements: Thanks to its interpretable decision process, WCC further reveals clear patterns of when to deploy CC: as variable coupling increases, WCC increasingly favors NDAs, while its temporal preference shifts from early NDA to later CC on separable problems and from early CC to later NDA on non-separable ones. These observations indicate that effective CC deployment should jointly depend on problem structure and optimization stage rather than a fixed switching rule.

•

1) State: The state S comprises four complementary feature groups: problem Spro ∈ R5 , convergence Sconv ∈ R6 , resource Sres ∈ R3 , and mode Smode ∈ R2 . Specifically, Spro captures problem dimensionality and structural information that characterize the suitability of CC and NDAs; Sconv describes the current optimization dynamics to support stagewise switching between their complementary search behaviors [9]; and Sres and Smode characterize resource consumption and historical mode behavior, respectively, providing the contextual information needed for adaptive hybridization. 2) Action: The action A of WCC comprises two components: a mode selector amode (choosing between NDA and CC) and a resource allocator ares (determining the FEs budget and switching timing). Assuming ares has κ levels, the composite action space is defined as: A := amode × ares , amode ∈ {0, 1}, ares ∈ {0, . . . , κ − 1}. (2) 3) Reward: To provide scale-robust and numerically stable feedback across heterogeneous LSGO problems, we adopt a log-ratio reward:   cost∗t−1 + c , c = 10−8 max{1, cost∗t , cost∗t−1 }, rt = ln cost∗t + c (3) where cost∗t−1 and cost∗t denote the best costs at two consecutive steps. The log ratio mitigates sensitivity to objective scales and compresses large improvements, while the adaptive smoothing term c prevents numerical instability when cost∗t approaches zero.

B. Network Design The network of WCC is composed of three components: Feature-Aware State Encoding, the Actor network, and the Critic network. Figure 1 shows the specific architecture.

WCC: WHEN TO APPLY COOPERATIVE COEVOLUTION

3

State Layernorm

s1 s2 s3 s4

...

s15 s16

Probabilities

s1 s2 s3 s4 ... s15 s16

Select

Probabilities

Aggregation

Resource Head

MLP

Baseline for Resource Head

Baseline for Mode Head

Sample

Select

MLP

Mean

MLPh Mode Head

z(0)

Aggregation

MLPg

Probabilities

MLP

z(1)

MLP

MLP

MLP

Actor

Critic

z FeatureSE

Fig. 1. The network design of WCC.

1) Feature-Aware State Encoding: Because the relevance of state information varies across problem types and optimization stages, we introduce Feature-Aware State Encoding (FeatureSE) to adaptively reweight heterogeneous state features. Given the normalized state S̃ = LN(S), FeatureSE learns both group-level gates g over the four state categories and feature-level gates h over individual features. Their effects are combined through residual gating, wj = (1 − λ) + λgk(j) hj ,

zj = sj wj ,

At each optimization step, the current state st is fed to the agent, which selects the optimization mode (CC or NDA) together with its computational budget. The selected optimizer is then executed under this budget, after which the best solution and state are updated. This process repeats until the evaluation budget is exhausted. For each policy head h ∈ {mode, res}, the clipped PPO objective is

(4)

Lclip = −E[min(ρh,t Ah,t , clip(ρh,t , 1 − ϵ, 1 + ϵ)Ah,t )] , (7) h

where k(j) denotes the group of feature j and λ controls the gating strength. This hierarchical reweighting allows WCC to emphasize different information according to the current optimization context while avoiding excessive feature suppression. 2) Actor: We adopt an autoregressive policy that first selects the optimization mode and then allocates resources conditioned on the selected mode:

where ρh,t denotes the policy probability ratio and Ah,t is estimated by GAE using the corresponding critic baseline. Standard value and entropy regularization are further incorporated during training. Methodological details are provided in Appendix A.

π(amode , ares | z) = πθmode (amode | z)πθres (ares | z, amode ), (5) where z is the FeatureSE-enhanced state representation. The mode head outputs a Bernoulli distribution over CC and NDA, while the resource head outputs a categorical distribution over candidate budgets using mode-specific representations to avoid cross-mode information leakage. 3) Critic: To support the autoregressive policy, we adopt a mode-conditioned critic V (z, amode ) with two value heads V0 (z) and V1 (z). For the resource policy, the chosen-mode value V (zt , amode(t) ) provides a baseline independent of ares ; for the mode policy, we use the mode-marginal baseline X V̄ (zt ) = πθmode (m | zt )V (zt , m), (6) m∈{0,1}

which is independent of the realized mode action. These baselines reduce variance while preserving unbiased policygradient estimation. C. Workflow After training, WCC follows a simple decomposition– decision–execution–update loop. The problem is first decomposed into subspaces using an existing decomposition strategy.

III. E XPERIMENTS A. Experimental Setup 1) Comparison algorithms: We instantiate WCC with CMA-ES [17] as the CC optimizer and MMES [7] as the NDA, denoted as WCC-MMES. We compare it with representative CC-based methods, including DG2 [18], CCCMAES [19], ERDG [20], OEDG [21], LCC-CMAES [22]; NDAs including MMES [7], sep-CMA-ES [23], LM-MAES [6], FCMAES [24], and MetaES [25]; and hybrid methods CC-SHADE-ML [26] and HCC-ES [9]. 2) Benchmarks: The primary benchmark is CEC2013LSGO [27], containing 15 problems spanning fully separable, partially separable, overlapping, and nonseparable structures. We treat F12 as non-separable because its chain-like structure induces 999 overlapping variables and subspaces. We further use the 1000-dimensional Auto Overlapping Benchmark (AOB) [9] to evaluate generalization across overlapping and non-separable structures. 3) Settings: WCC-MMES is trained for 90 epochs on representative training problems with MaxFEs = 1E+06 and is directly reused for testing without retraining. Each algorithm is independently evaluated over 30 runs, with comparison

WCC: WHEN TO APPLY COOPERATIVE COEVOLUTION

4

TABLE I C OMPARING WCC-MMES WITH COMPARISON ALGORITHMS ON CEC2013LSGO AT 1.5E6 FE S . problem *1 *4 *5 *8 *9 *12 *13 2 3 6 7 10 11 14 15 +/≈/-

WCC-based Algorithms WCC-MMES 2.16E-20 ± 4.90E-20 2.13E+05 ± 4.72E+04 1.65E+06 ± 3.26E+05 3.97E+09 ± 5.21E+8 1.70E+08 ± 5.78E+07 9.33E+02 ± 8.60E+01 5.56E+06 ± 8.57E+05 2.06E+03 ± 4.53E+02 2.00E+01 ± 2.31E-02 9.95E+05 ± 1.10E+01 4.41E+04 ± 2.37E+04 9.05E+07 ± 5.09E+02 1.45E+07 ± 1.63E+06 2.52E+07 ± 3.74E+06 2.31E+06 ± 1.47E+06 NA

DG2-CMAES 7.36E-20(+) ± 8.97E-21 1.94E+05(-) ± 2.45E+04 1.70E+06(+) ± 3.39E+05 4.63E+09(+) ± 1.40E+09 1.84E+08(+) ± 2.75E+07 1.78E+03(+) ± 4.15E+01 9.05E+10(+) ± 3.45E+09 3.68E+03(+) ± 4.15E+02 2.10E+01(+) ± 3.56E-02 1.06E+06(+) ± 9.01E+02 3.97E+04(-) ± 1.45E+04 9.12E+07(+) ± 2.92E+05 5.74E+07(+) ± 4.03E+07 1.46E+10(+) ± 2.02E+09 6.32E+13(+) ± 4.48E+13 13/0/2

CC-CMAES 9.12E+06(+) ± 2.20E+06 1.18E+11(+) ± 5.01E+11 1.05E+07(+) ± 2.18E+06 5.04E+15(+) ± 7.47E+15 7.62E+08(+) ± 2.18E+08 2.27E+03(+) ± 7.89E+02 6.70E+10(+) ± 3.17E+10 2.16E+03(+) ± 1.39E+02 2.01E+01(≈) ± 2.50E-02 1.03E+06(+) ± 1.58E+04 1.46E+09(+) ± 3.93E+09 9.42E+07(+) ± 4.62E+05 5.06E+11(+) ± 4.85E+11 1.12E+12(+) ± 3.36E+11 3.45E+08(+) ± 3.08E+08 14/1/0

CC-based Algorithms LCC-CMAES ERDG-CMAES 1.13E+05(+) 9.51E-19(+) ± 2.54E+04 ± 6.35E-19 3.48E+10(+) 1.49E+10(+) ± 4.99E+09 ± 7.32E+09 1.05E+07(+) 1.97E+06(+) ± 6.31E+05 ± 3.09E+05 7.50E+14(+) 4.58E+10(+) ± 2.44E+14 ± 1.99E+10 8.95E+08(+) 1.69E+08(≈) ± 8.37E+07 ± 2.94E+07 1.47E+03(+) 9.58E+02(+) ± 1.92E+02 ± 2.94E+00 4.33E+08(+) 1.70E+09(+) ± 2.51E+08 ± 8.68E+08 2.11E+03(+) 1.33E+04(+) ± 1.30E+02 ± 9.59E+02 2.04E+01(+) 2.16E+01(+) ± 4.78E-02 ± 5.01E-03 1.05E+06(+) 1.06E+06(+) ± 2.00E+03 ± 1.21E+03 5.36E+07(+) 5.62E+09(+) ± 2.38E+07 ± 2.08E+09 9.42E+07(+) 9.46E+07(+) ± 4.46E+05 ± 2.21E+05 4.81E+10(+) 1.81E+09(+) ± 3.43E+10 ± 9.20E+08 7.31E+08(+) 2.02E+09(+) ± 5.74E+08 ± 4.69E+08 3.78E+07(+) 4.55E+08(+) ± 4.04E+06 ± 3.74E+07 15/0/0 14/1/0

OEDG-CMAES 1.15E-18(+) ± 3.47E-19 1.87E+10(+) ± 5.44E+09 2.33E+06(+) ± 4.04E+05 3.17E+15(+) ± 3.89E+15 1.57E+08(-) ± 4.02E+07 2.06E+03(+) ± 2.29E+03 1.56E+10(+) ± 5.55E+09 1.35E+04(+) ± 7.05E+02 2.16E+01(+) ± 5.01E-03 1.06E+06(+) ± 1.31E+03 7.86E+09(+) ± 5.61E+09 9.50E+07(+) ± 2.12E+05 1.63E+10(+) ± 1.52E+10 3.10E+10(+) ± 5.07E+10 4.56E+08(+) ± 3.82E+07 14/0/1

MMES 7.98E+05(+) ± 1.37E+05 4.43E+08(+) ± 7.68E+07 4.17E+06(+) ± 9.25E+05 4.21E+11(+) ± 4.35E+10 4.41E+08(+) ± 4.26E+07 9.33E+02(≈) ± 6.74E+01 9.36E+06(+) ± 2.68E+06 1.44E+04(+) ± 9.02E+02 2.01E+01(≈) ± 1.87E-01 9.95E+05(≈) ± 4.52E+01 7.56E+05(+) ± 6.72E+04 9.05E+07(≈) ± 2.04E+04 1.82E+07(+) ± 2.44E+06 3.31E+07(+) ± 5.57E+06 5.89E+06(+) ± 9.36E+05 11/4/0

sep-CMA-ES 3.90E-17(+) ± 7.95E-19 1.38E+11(+) ± 2.55E+10 1.67E+06(+) ± 1.52E+05 1.83E+15(+) ± 8.79E+14 1.41E+08(-) ± 1.70E+07 1.00E+03(+) ± 4.22E+01 2.20E+11(+) ± 1.13E+11 5.51E+03(+) ± 1.59E+02 2.16E+01(+) ± 6.06E-03 1.06E+06(+) ± 1.64E+03 6.79E+09(+) ± 4.62E+09 9.42E+07(+) ± 2.08E+05 1.09E+11(+) ± 2.90E+10 1.22E+11(+) ± 2.17E+11 6.63E+08(+) ± 1.26E+08 14/0/1

NDAs LM-MA-ES 4.40E+06(+) ± 3.27E+05 7.33E+11(+) ± 1.14E+11 1.75E+06(+) ± 1.15E+05 2.90E+15(+) ± 1.47E+15 2.30E+08(+) ± 2.20E+07 8.61E+02(-) ± 3.41E+01 6.76E+07(+) ± 2.15E+06 5.82E+03(+) ± 2.74E+02 2.16E+01(+) ± 2.67E-03 1.06E+06(+) ± 1.25E+03 8.23E+06(+) ± 9.59E+05 9.42E+07(+) ± 2.91E+05 7.61E+08(+) ± 1.39E+08 3.27E+08(+) ± 3.44E+07 2.31E+07(+) ± 6.26E+06 14/0/1

FCMAES 5.90E+06(+) ± 8.20E+05 3.45E+09(+) ± 4.20E+08 4.43E+06(+) ± 8.33E+05 3.61E+12(+) ± 6.78E+11 5.70E+08(+) ± 9.23E+07 8.44E+02(-) ± 1.20E+02 2.71E+07(+) ± 8.49E+06 1.62E+04(+) ± 8.52E+02 2.09E+01(+) ± 1.10E-01 9.96E+05(≈) ± 2.18E+03 4.23E+06(+) ± 5.43E+05 9.07E+07(+) ± 2.83E+05 1.09E+08(+) ± 1.91E+07 1.55E+08(+) ± 7.54E+06 1.75E+07(+) ± 2.31E+06 13/1/1

MetaES 4.49E+11(+) ± 2.35E+10 2.37E+14(+) ± 5.40E+11 1.02E+08(+) ± 5.21E+06 2.49E+18(+) ± 1.38E+16 1.81E+10(+) ± 6.83E+07 9.19E+12(+) ± 3.21E+12 5.74E+18(+) ± 7.59E+17 1.15E+05(+) ± 5.72E+03 2.16E+01(+) ± 7.97E-01 1.06E+06(+) ± 8.23E+03 1.09E+18(+) ± 7.24E+13 9.44E+07(+) ± 3.11E+05 4.88E+19(+) ± 8.79E+18 3.41E+23(+) ± 7.81E+21 1.26E+18(+) ± 6.94E+14 15/0/0

Hybrid Algorithms HCC-ES CC-SHADE-SL 6.96E-20(+) 1.87E+05(+) ± 8.87E-21 ± 3.64E+05 7.18E+06(+) 1.25E+10(+) ± 2.85E+06 ± 1.10E+10 1.81E+06(+) 1.92E+06(+) ± 3.12E+05 ± 7.59E+06 6.60E+09(+) 2.13E+14(+) ± 1.67E+08 ± 2.55E+14 1.51E+08(-) 2.34E+08(+) ± 2.76E+07 ± 1.13E+08 1.72E+03(+) 2.10E+05(+) ± 2.76E+01 ± 6.60E+05 2.88E+07(+) 3.15E+09(+) ± 2.49E+07 ± 1.43E+09 3.72E+03(+) 2.54E+03(+) ± 3.71E+02 ± 1.45E+03 2.10E+01(+) 2.09E+01(+) ± 3.79E-02 ± 8.31E-02 1.06E+06(+) 1.06E+06(+) ± 8.28E+02 ± 2.42E+03 1.69E+05(+) 3.91E+07(+) ± 1.16E+04 ± 4.14E+07 9.11E+07(+) 9.32E+07(+) ± 4.91E+05 ± 3.15E+05 5.10E+07(+) 4.30E+19(+) ± 2.79E+07 ± 6.43E+09 8.78E+07(+) 1.44E+10(+) ± 1.15E+07 ± 2.74E+10 4.48E+13(+) 1.22E+07(+) ± 2.41E+13 ± 5.76E+06 14/0/1 15/0/0

Problems marked with * are representative problems used to form the training set. The symbols ‘+’, ‘−’, and ‘≈’ signify that WCC-MMES is significantly better than, worse than, or comparable to its competitor, respectively.

methods configured following their original settings. The training and unseen test problems may share the same separability category but differ in their underlying base functions. Detailed experimental settings are provided in Appendix B. B. Comparison Analysis 1) Optimization Performance Comparison: Table I reports results on CEC2013LSGO at MaxFEs = 1.5E+06 (including 5E+05 for DG2). Symbols “+”, “−”, and “≈” indicate significantly better, worse, and comparable performance, respectively (Wilcoxon rank-sum test, α = 0.05), with W/T/L counts summarized in the final row. Based on the results from Table I, we can analyze the following outcomes: a) WCC demonstrates superior performance: WCC-MMES consistently performs competitively against all other algorithms. In particular, DG2-CMAES achieves theoretically maximal decomposition accuracy, and MMES is among the most competitive NDAs. WCC’s strong performance thus highlights the power of hybridizing the two approaches. b) Data-driven approaches transcend the limitations of expert heuristics: With expert knowledge scarce, WCC-MMES makes finer-grained action selections in a data-driven manner. Its superior performance over hybrid algorithms further shows that WCC can transcend the limitations of expert heuristics. c) Effectively Learning the Strengths of CC and NDAs: WCC-MMES consistently excels on problems favoring both CC-based algorithms (e.g., F1, F5) and MMES (e.g., F13–F15), demonstrating its ability to effectively integrate the strengths of CC and NDAs. 2) Generalization Experiment: We evaluate WCC-MMES generalization across three settings by directly testing the trained policy without retraining. a) Generalization across Problems: In Table I, 8 of the 15 test problems were unseen during training, yet WCCMMES remained competitive; notably, its strong performance on F15—a complex non-separable problem—underscores its

S15

S10

E_Non

S20

E20 0.8

WCC-MMES DG2-CMAES MMES Sep-CMAES LMMAES FCMAES HCC-ES

0.6

S_Non

E15

0.4 0.2 0.0

A10

E10 R_Non

A15 A20

R20 A_Non

R10

R15

Fig. 2. Comparison of WCC-MMES and baselines on AOB at 1.5E6 FEs. Radial values are min–max normalized scores, with larger values indicating better relative performance.

promising generalization ability. To further validate WCCMMES’s competitiveness on overlapping and non-separable problems and its generalization across problem instances, we generated 16 new AOB problems and tested them under MaxFEs = 1.5E+06. Figure 2 shows that WCC-MMES is competitive on most problems, outperforming both MMES and DG2-CMAES. Its strength on non-separable problems is particularly notable; for example, in the Rastrigin family, WCC-MMES lags on overlapping variants but still excels on non-separable instances, further confirming its ability to tackle complex non-separable problems. b) Generalization across Optimization Horizons: Although trained with only MaxFEs = 1E+06, WCC-MMES remains highly competitive when directly evaluated at 3E+06 FEs without retraining. It achieves +/-/≈ records of 13/0/2, 15/0/0, 12/3/0, and 14/0/1 against DG2-CMAES, LCCCMAES, MMES, and HCC-ES, respectively. The advantage is also substantial on several problems; for example, on F15, WCC-MMES reduces the final cost from the order of 1013 achieved by DG2-CMAES and HCC-ES to 105 , corresponding to nearly eight orders of magnitude improvement. These

11

12

Fig. 3. Generalization experiment of WCC across baseline algorithms.

results demonstrate strong generalization across optimization horizons. Complete results are provided in Appendix B. c) Generalization across Baseline Algorithms: To assess whether WCC generalizes beyond MMES, we directly replace the NDA component in the trained WCC-MMES with SepCMAES, LM-MA-ES, and FCMAES, yielding three WCC variants without retraining. As shown in Figure 3, these variants outperform their corresponding standalone NDAs on most CEC2013LSGO problems at MaxFEs = 1E+06, often by large margins. This transferability stems from WCC’s optimizer-agnostic state and action design, which preserves the same meta-level mode-selection and budget-allocation problem across different NDAs.

0.0 1.0

（b）

State Ablation

（c）

0.8

0.843

0.7

0.8 0.6 0.4 0.2 0.0

0.619

0.4

0.391 0.283

0.3 0.2

（d）

0.8

0.714

0.6

0.564 0.5

Reward Ablation

0.535

0.1 0.0

0.365 0.4 0.2

0.0

Net Ablation

0.750 0.572

0.6

0.398

Fixed(ares = 2E5)

15

Fixed(ares = 1E5)

14

Fixed(ares = 5E4)

13

0.459

0.396

W/O FeatureSE Simple MLP

10

0.817

Simple MLP

9

Fixed(amode = 0)

8

Problem Index

0.817

W/O FeatureSE

6

WCC-MMES

5

Fixed(amode = 1)

4

0.2

0.417

Difference

3

1E2

Sparse Sign

2

1E3

Linear

1

1E3

0.580

Random(ares)

7

0

1E3

WCC-MMES

1E3 1E2 1E1

0.907

0.861

Random(amode)

1E1

0.874

W/O Smode

1E4 1E3 1E1

0.4

Random

1E4

0.6

W/O Sres

5 1E3

Win/Draw/Loss 14/0/1 12/0/3 12/0/3

0.797

0.8

W/O Sconv

Algorithm WCC-Sep-CMAES WCC-LMMAES WCC-FCMAES

10

0.939

WCC-MMES

15

1.0

WCC-MMES

20

Action Ablation

（a）

W/O Spro

WCC-Sep-CMAES vs Sep-CMAES WCC-LMMAES vs LM-MA-ES WCC-FCMAES vs FCMAES Baseline (Performance Ratio = 1) WCC Better Baseline Better

Mean Normalized Performance

25 1E25

5

Mean Normalized Performance

Log Performance Ratio (log (Baseline Algorithm / WCC Variant))

WCC: WHEN TO APPLY COOPERATIVE COEVOLUTION

Fig. 4. Ablation study, including (a) action space ablation, (b) state space ablation, (c) reward function ablation, and (d) network architecture ablation.

when both are removed, validating the joint roles of adaptive state reweighting and mode-conditioned resource allocation. Detailed ablation configurations and further analysis are provided in Appendix B.

C. Ablation Study To further examine where the performance of WCC-MMES comes from, we conduct a comprehensive ablation study from four aspects: action space, state space, reward function, and network architecture. 1) Experimental Protocol: We ablate WCC from four aspects: action, state, reward, and network design. For the action space, we randomize or fix either the mode or resource decision to examine the necessity of jointly learning both components. For the state space, each of the four feature groups {Spro , Sconv , Sres , Smode } is removed individually. For the reward, the proposed log-ratio formulation is replaced with Difference, Sparse Sign, and Linear alternatives. For the network, we remove FeatureSE, replace the autoregressive policy with a simple MLP, or remove both. All variants follow the same training and evaluation protocol, and their final costs are min–max normalized within each category before averaging across problems. 2) Ablation Analysis: Figure 4 confirms the contribution of all four design components. For the action space, randomizing or fixing either decision degrades performance, showing the importance of jointly learning mode selection and resource allocation. For the state space, removing any feature group causes a clear drop, with convergence features having the largest impact, indicating that the four groups provide complementary information. For the reward, all alternative formulations underperform the proposed log-ratio reward, highlighting the importance of scale-robust and numerically stable feedback. For the network, removing FeatureSE or replacing the autoregressive policy with a simple MLP consistently worsens performance, with the largest degradation occurring

D. Further In-depth Analysis Understanding when WCC selects CC is a central focus of this work. We divide each optimization run into four equal intervals and report the mean proportion of NDA selections in Figure 5, revealing clear patterns across both problem structures and optimization stages. Across problem structures, WCC increasingly favors NDAs as variable coupling becomes stronger. CC dominates on fully and partially separable problems by exploiting decomposable structures, whereas NDA usage increases on overlapping problems and becomes dominant on non-separable ones, where decomposition is less effective. Across optimization stages, WCC generally favors NDAs early and gradually shifts toward CC on separable and overlapping problems; in contrast, on non-separable problems, CC is used mainly for limited early exploration, while NDAs dominate the middle and later stages. These results show that the appropriate use of CC depends jointly on problem structure and optimization stage rather than on a fixed switching rule. Individual functions may nevertheless deviate from these category-level trends. For example, although F3 is categorized as fully separable, its exact separability is preserved only under one-dimensional decomposition; the 50-dimensional grouping used here therefore limits the effectiveness of CC, causing WCC to increasingly favor NDAs in later stages, similarly to non-separable problems. A similar pattern appears on F6. This further indicates that WCC adapts its mode selection to the observed optimization context rather than relying solely on nominal separability.

WCC: WHEN TO APPLY COOPERATIVE COEVOLUTION

Fully-separable

Partially Separable I

6

Partially Separable II

Overlapping

Nonseparable

Fig. 5. Proportion of NDA selections across problem types and optimization stages. Each cell reports the mean proportion of amode = NDA, where darker colors indicate more frequent NDA usage. Rows denote four equal intervals divided by MaxFEs, and columns denote test functions grouped by separability.

IV. CONCLUSION AND FUTURE WORK We proposed WCC, a pioneering framework that learns when to use CC in LSGO, concurrently optimizing the resource allocation. WCC enabled fine-grained, dynamic selection and effective hybridization of CC and NDAs without human expertise. Extensively evaluated, WCC exhibited strong competitiveness—especially on non-separable problems, where both CC and NDAs alone typically struggle. Moreover, WCC demonstrated generalization across problems, optimization horizons, and baseline optimizers, enabling trainonce under a low FE budget and reuse with different underlying optimizers to mitigate LSGO’s computational cost. Finally, ablation studies and in-depth analyses yielded insights that go beyond existing theoretical understanding, offering practical guidance on when and how to deploy CC. By shifting the focus from how to decompose to when to deploy CC, WCC opens a new direction for CC research. Future work will explore online separability characterization and adaptive selection among multiple CC and NDA strategies. R EFERENCES [1] M. N. Omidvar, X. Li, and X. Yao, “A review of population-based metaheuristics for large-scale black-box global optimization—part i,” IEEE Transactions on Evolutionary Computation, vol. 26, no. 5, pp. 802–822, 2021. [2] X. Ma, X. Li, Q. Zhang, K. Tang, Z. Liang, W. Xie, and Z. Zhu, “A survey on cooperative co-evolutionary algorithms,” IEEE Transactions on Evolutionary Computation, vol. 23, no. 3, pp. 421–441, 2018. [3] W. Qiu, Z. Weng, Y.-J. Gong, W.-N. Chen, and J. Zhang, “Unicc: A unified coevolutionary architecture with divergence-speedup modeling for large-scale global optimization,” IEEE Transactions on Evolutionary Computation, 2026. [4] M. M. Komarnicki, M. W. Przewozniczek, H. Kwasnicka, and K. Walkowiak, “Incremental recursive ranking grouping for large-scale global optimization,” IEEE Transactions on Evolutionary Computation, vol. 27, no. 5, pp. 1498–1513, 2022. [5] Q. Duan, C. Shao, G. Zhou, H. Yang, Q. Zhao, and Y. Shi, “Cooperative coevolution for non-separable large-scale black-box optimization: Convergence analyses and distributed accelerations,” Applied Soft Computing, vol. 166, p. 112232, 2024. [6] I. Loshchilov, T. Glasmachers, and H.-G. Beyer, “Large scale black-box optimization by limited-memory matrix adaptation,” IEEE Transactions on Evolutionary Computation, vol. 23, no. 2, pp. 353–358, 2018. [7] X. He, Z. Zheng, and Y. Zhou, “Mmes: Mixture model-based evolution strategy for large-scale optimization,” IEEE Transactions on Evolutionary Computation, vol. 25, no. 2, pp. 320–333, 2020.

[8] F. Van den Bergh and A. P. Engelbrecht, “A cooperative approach to particle swarm optimization,” IEEE transactions on evolutionary computation, vol. 8, no. 3, pp. 225–239, 2004. [9] W. Qiu, H. Guo, Z. Ma, and Y.-J. Gong, “A novel two-phase cooperative co-evolution framework for large-scale global optimization with complex overlapping,” arXiv preprint arXiv:2503.21797, 2025. [10] Z. Ma, H. Guo, Y.-J. Gong, J. Zhang, and K. C. Tan, “Toward automated algorithm design: A survey and practical guide to meta-black-boxoptimization,” IEEE Transactions on Evolutionary Computation, 2025. [11] W. Qiu, H. Guo, Z. Ma, J. Zhang, and Y.-J. Gong, “Automated algorithm design for black-box optimization: Progress and challenges,” Chinese Journal of Computers, vol. 49, no. 4, pp. 855–884, 2026. [12] Y. Song, Y. Wu, Y. Guo, R. Yan, P. N. Suganthan, Y. Zhang, W. Pedrycz, S. Das, R. Mallipeddi, O. S. Ajani et al., “Reinforcement learningassisted evolutionary algorithm: A survey and research opportunities,” Swarm and Evolutionary Computation, vol. 86, p. 101517, 2024. [13] W. Qiu, Z. Wang, H. Fang, Z. Ma, and Y.-J. Gong, “A learningbased cooperative coevolution framework for heterogeneous large-scale global optimization,” in Proceedings of the Genetic and Evolutionary Computation Conference, 2026, pp. 938–947. [14] Q. Zhao, T. Liu, B. Yan, Q. Duan, J. Yang, and Y. Shi, “Automated metaheuristic algorithm design with autoregressive learning,” IEEE Transactions on Evolutionary Computation, 2024. [15] W. Yi, R. Qu, L. Jiao, and B. Niu, “Automated design of metaheuristics using reinforcement learning within a novel general search framework,” IEEE Transactions on Evolutionary Computation, vol. 27, no. 4, pp. 1072–1084, 2022. [16] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017. [17] N. Hansen and A. Ostermeier, “Completely derandomized selfadaptation in evolution strategies,” Evolutionary computation, vol. 9, no. 2, pp. 159–195, 2001. [18] M. N. Omidvar, M. Yang, Y. Mei, X. Li, and X. Yao, “Dg2: A faster and more accurate differential grouping for large-scale black-box optimization,” IEEE Transactions on evolutionary computation, vol. 21, no. 6, pp. 929–942, 2017. [19] J. Liu and K. Tang, “Scaling up covariance matrix adaptation evolution strategy using cooperative coevolution,” in International Conference on Intelligent Data Engineering and Automated Learning. Springer, 2013, pp. 350–357. [20] M. Yang, A. Zhou, C. Li, and X. Yao, “An efficient recursive differential grouping for large-scale continuous problems,” IEEE Transactions on Evolutionary Computation, vol. 25, no. 1, pp. 159–171, 2020. [21] M. Tian, M. Chen, W. Du, Y. Tang, and Y. Jin, “An enhanced differential grouping method for large-scale overlapping problems,” IEEE Transactions on Evolutionary Computation, 2024. [22] H. Guo, W. Qiu, Z. Ma, X. Zhang, J. Zhang, and Y.-J. Gong, “Advancing cma-es with learning-based cooperative coevolution for scalable optimization,” arXiv preprint arXiv:2504.17578, 2025. [23] R. Ros and N. Hansen, “A simple modification in cma-es achieving linear time and space complexity,” in International conference on parallel problem solving from nature. Springer, 2008, pp. 296–305.

WCC: WHEN TO APPLY COOPERATIVE COEVOLUTION

[24] Z. Li, Q. Zhang, X. Lin, and H.-L. Zhen, “Fast covariance matrix adaptation for large-scale black-box optimization,” IEEE transactions on cybernetics, vol. 50, no. 5, pp. 2073–2083, 2018. [25] R. Lange, T. Schaul, Y. Chen, T. Zahavy, V. Dalibard, C. Lu, S. Singh, and S. Flennerhag, “Discovering evolution strategies via meta-black-box optimization,” in Proceedings of the companion conference on genetic and evolutionary computation, 2023, pp. 29–30. [26] A. Vakhnin and E. Sopov, “A novel self-adaptive cooperative coevolution algorithm for solving continuous large-scale global optimization problems,” Algorithms, vol. 15, no. 12, p. 451, 2022. [27] X. Li, K. Tang, M. N. Omidvar, Z. Yang, and K. Qin, “Benchmark functions for the CEC’2013 special session and competition on largescale global optimization,” Evolutionary Computation and Machine Learning Group, RMIT University, Melbourne, Australia, Technical Report, 2013.

7

1

Appendix A PPENDIX A M ETHODOLOGY A. State Representation WCC represents the optimization context with 16 features, grouped into problem (s1 –s5 ), convergence (s6 –s11 ), resource (s12 –s14 ), and mode-history (s15 –s16 ) information. Table I lists their complete definitions. Here D is the problem dimension, Θ is the dependency structure matrix (DSM), S is the subspace list, and Ωi is the variable set of subspace i. The bestso-far objective value is denoted by cost∗t , while sumF Es counts consumed function evaluations. The following definitions expand the trajectory-based and mode-history features that require more than a single expression. 1) Low-frequency energy ratio (s7 ): For an observed best−1 cost sequence {cost∗k }Tk=0 , compute the discrete Fourier transform and its normalized amplitude:

4) Robust variation of cost changes (s10 ): Use the most recent window of length L = 128, with τ0 = max(1, t − L), and define ∆costk = cost∗k −cost∗k−1 for k = τ0 , . . . , t−1. Let med be the median of these changes and IQR = Q0.75 −Q0.25 their interquartile range. Then   IQR s10 = tanh λ3 , λ3 = 0.5, ϵ = 10−12 . (7) |med| + ϵ

(1)

When fewer than L changes are available, use all available changes. The feature summarizes fluctuation relative to the typical recent progress. 5) Recent mode-specific reductions (s16 ): Record the fitness reductions obtained by NDA and CC in separate histories RN DA,t and RCC,t before decision step t. For either mode m ∈ {N DA, CC}, define  [−5:]  mean(Rm,t ), |Rm,t | > 5, r̄m = mean(Rm,t ), 0 < |Rm,t | ≤ 5, (8)   0, |Rm,t | = 0.

Retain the positive-frequency indices K = {0, . . . , ⌊T /2⌋−1}. With fk = k/T and cutoff fc = 0.1, define X X Elow = Ak , Etot = Ak ,

The feature s16 = r̄N DA − r̄CC compares the average reductions over the latest five uses of each mode, or over all its uses when fewer than five are available. Positive values indicate larger recent reductions from NDA.

Ĉk =

T −1 X

cost∗u e−i2πku/T ,

Ak =

u=0

k∈K f ≤f

|Ĉk | . T

k∈K

(k c Elow /Etot , Etot > 0, s7 = 0, Etot = 0.

(2)

The ratio summarizes the low-frequency spectral mass of the observed cost trajectory; the historical definition sums amplitudes. 2) First-order convergence trend (s8 ): For the available ∗ trajectory {cost∗k }t−1 k=0 , let gk = ∇costk be its numerical t−1 gradient and ḡ = mean({gk }k=0 ). The feature is s8 = − tanh(λ1 ḡ),

λ1 = 0.5.

(3)

The sign convention makes decreasing costs correspond to positive values, while the hyperbolic tangent limits the scale of the trend statistic. 3) Second-order convergence trend (s9 ): For interior points, use hk = cost∗k+1 − 2cost∗k + cost∗k−1 ,

k = 1, . . . , t − 2. (4)

The two boundary expressions are h0 = 2cost∗0 − 5cost∗1 + 4cost∗2 − cost∗3 , ht−1 = 2cost∗t−1 − 5cost∗t−2 + 4cost∗t−3 − cost∗t−4 .

(5)

With h̄ = mean({hk }t−1 k=0 ), set s9 = − tanh(λ2 h̄),

λ2 = 0.5.

(6)

These boundary formulas require at least four trajectory values.

B. Network Architecture This subsection expands the architecture in Fig. 1 of the main paper. FeatureSE reweights heterogeneous state information; the autoregressive actor conditions resource allocation on the selected mode; and the critic provides a corresponding baseline for each policy head. 1) Feature-aware state encoding: Apply layer normalization to obtain S̃ = LN(S) ∈ R16 . For each of the four feature groups, compute its mean statistic and collect these statistics in S̄ = [S̄pro , S̄conv , S̄res , S̄mode ]. The group and feature gates are
g = σ MLPg (S̄) , g ∈ (0, 1)4 ,
(9) h = σ MLPh (S̃) , h ∈ (0, 1)16 . The two MLPs have dimensions 4 × 8 × 4 and 16 × 8 × 16, respectively, with SiLU nonlinearities and sigmoid outputs. Following Equation (4) of the main paper, residual gating uses wj = (1 − λ) + λgk(j) hj ,

zj = sj wj ,

λ = 0.2. (10)

Thus wj ∈ (0.8, 1), limiting feature suppression. The last-layer biases of both gating MLPs are initialized to 2.0, for which σ(2) ≈ 0.88, to start close to the identity mapping. 2) Autoregressive actor: The policy factorization is π(amode , ares | z) = πθmode (amode | z)πθres (ares | z, amode ). (11) The mode head is an MLP with dimensions 16×128×256×1, ReLU hidden activations, and a Bernoulli output. Mode 0 denotes NDA and mode 1 denotes CC. The resource policy

2

TABLE I C OMPLETE DEFINITIONS OF THE 16 WCC STATE FEATURES . Group Spro

Index 1 2 3 4 5

Sconv

6 7 8

Formula 1000 D P D

i=1

1− {i|

Explanation The relative size of the problem dimensions D. PD

j=1 I(Θ[i,j]=0) D2

PD

The sparsity of DSM Θ characterizes the interactive complexity of the problem.

j=1 Θ[i,j]=1}

D S|S| S|S| i=1

j̸=i

The proportion of completely separable variables.

(Ωi ∩Ωj )

The proportion of overlapping variables, Ωi is the variable set of subspace i. The relative size of the number of subspaces |S|.

D |S| D ∗ cost0 −cost∗ t−1 cost∗ 0 LF ER(cost∗ )

The decrease ratio of optimal cost cost∗t−1 at step t − 1. The proportion of low-frequency energy (LFER) in the optimal cost curve. The normalized average first-order trend of the optimal cost trajectory, reflecting the convergence speed. The normalized second-order trend indicating convergence acceleration.

− tanh(λ1 · mean(∇cost∗ ))

9

− tanh λ2 · mean(∇2 cost∗ )

10

tanh λ3

IQR({∆costk }t−1 ) k=τ

!

0

Median({∆costk }t−1 ) +ϵ k=τ 0

11 Sres

12 13 14

Smode

15 16

dmaxt−1 diameter MaxFEs−sumF Es MaxFEs mean(F EsN DA ) max(ares ) mean(F EsCC ) max(ares ) PT t=1 I(at =N DA) T r̄NDA − r̄CC

uses two separate MLPs, one for each mode, with dimensions 15×160×320×3 and ReLU hidden activations. Each resource head uses a 15-dimensional input excluding the opposite mode’s historical FE statistic: z (0) omits z14 for NDA, while z (1) omits z13 for CC. Each head produces a categorical distribution over the three resource levels in Table II. The resource logits for both modes are computed in parallel; the sampled mode determines which logits are used to sample the resource action. 3) Mode-conditioned critic: The critic uses a backbone of the same style as the actor’s mode head, followed by separate linear value heads V0 (z) and V1 (z). It does not share actor parameters. The chosen-mode value is ( V0 (zt ), amode(t) = 0, V (zt , amode(t) ) = (12) V1 (zt ), amode(t) = 1. For the resource head, this value is independent of the sampled resource action, conditional on the selected mode. For the mode head, the baseline is X V̄ (zt ) = πθmode (m | zt )V (zt , m), (13) m∈{0,1}

which is independent of the realized mode action. This distinction matches the two successive decisions in the actor and reduces policy-gradient variance without introducing actiondependent baselines for either head. C. Training Objectives We train the agent πθ by PPO [1]. At each step t, let dt ∈ {0, 1} be the done flag. The auto-regressive policy has two heads h ∈ {mode, res} with distributions πθh (·). Using the mode-marginal value in Equation (13), the one-step target is

The robust variation coefficient of recent optimal-cost changes, measuring the stability of recent convergence. The ratio of the maximum distance of individuals dmax to the search diameter diameter at step t − 1. The remaining proportion of FEs. The mean of function evaluations consumed by NDA. The mean of function evaluations consumed by CC. The selection ratio of NDA across all optimization steps T . The difference between the average recent fitness reductions achieved by NDA and CC.

Rcond(t) = rt + γ(1 − dt )V̄ (zt+1 ). With value clipping Vclip around the V old , the value loss is: 
 LV = E max (V − Rcond )2 , (Vclip − Rcond )2 . (14) Advantages Ah,t are computed with Generalized Advantage Estimation using the corresponding baseline (standardized per batch). Store old log-probabilities log πθold (·) and recompute h log-probabilities under the current policy log πθnew (·) (no reh sampling) to form ρh,t = exp(log πθnew (ah,t | ·) − log πθold (ah,t | ·)). h h

(15)

The clipped surrogate per head is Lclip = −E [min (ρh Ah , clip (ρh , 1 − ε, 1 + ε) Ah )] . h

(16)

The total loss is clip L = Lclip mode + Lres + cV LV X − βh E [H (πθh )]

(17)

h∈{mode, res}

where cV weights the value loss and βh controls per-head entropy regularization. We optimize with Adam (gradient clipping; exponentially decayed learning rate). Each iteration collects nstep transitions per environment, then performs K epochs over the full segment (masked by (1 − dt )). D. Workflow Algorithm 1 preserves the complete inference procedure. A decomposition strategy obtains the DSM Θ, and RDDSM derives the subspace list S. The trained agent selects the mode and resource budget from the current state. NDA operates in the full space; CC divides the selected budget among the subspaces and updates the incumbent after each subspace.

3

Algorithm 1 WCC Input: Problem instance f and info I, Pre-trained Agent πθ∗ . Output: The best individual Xbest , The best cost Ybest . 1: DSM Θ ← Decomposition Strategy(f, I) 2: The list of Subspaces S ← RDDSM(Θ) 3: Initialize Xbest , Ybest and state vector s0 4: while Termination conditions not met do 5: amode(t) , ares(t) ← πθ∗ (st ) 6: if amode(t) = 0 then 7: NDA(f , I, Xbest , ares(t) ) 8: Xbest , Ybest , st+1 ← Update Xbest , Ybest , st 9: else   10: Each subspace FEs SubF Es = ares(t) /len(S) 11: for sub in S do 12: Asub (f , I, sub, Xbest , SubF Es) 13: Xbest , Ybest ← Update Xbest , Ybest 14: end for 15: Xbest , Ybest , st+1 ← Update Xbest , Ybest , st 16: end if 17: end while 18: return Xbest , Ybest The state is updated after the selected mode finishes, and optimization stops when no FEs remain. A PPENDIX B E XPERIMENTS A. Experimental Setup 1) Benchmarks: CEC2013LSGO [2] contains 15 problems, grouped here as fully separable (F1–F3), partially separable with separable components (F4–F7), partially separable without separable components (F8–F11), overlapping (F13–F14), and non-separable (F12 and F15). As in the main paper, F12 is treated as non-separable because its chain-like dependency structure yields 999 overlapping variables and subspaces. The Auto Overlapping Benchmark (AOB) [3] provides additional 1000-dimensional problems while keeping the dimension fixed as overlap changes. Its base functions are Ackley, Rastrigin, Schwefel, and Elliptic, following the F8– F11 settings of CEC2013LSGO. The 16 instances shown in Fig. 2 of the main paper comprise three overlapping variants per base function, with 10, 15, or 20 overlapping variables per subspace, and one globally rotated non-separable variant. Labels such as A10 indicate the base function and overlap; A Non denotes the non-separable Ackley variant. 2) Comparison algorithms: WCC-MMES uses CMAES [4] for subspace optimization and MMES [5] for fullspace optimization. DG2 and RDDSM provide the decomposition and subspace construction used by its CC component [3], [6]. The CC-based comparisons are DG2-CMAES, CC-CMAES [7], LCC-CMAES [8], ERDG-CMAES [9], and OEDG-CMAES [10]. The NDA comparisons are MMES, sep-CMA-ES [11], LM-MA-ES [12], FCMAES [13], and MetaES [14]. HCC-ES [3] and CC-SHADE-ML [15] are the hybrid comparisons. The original numerical table labels the latter CC-SHADE-SL; that table label is retained below to match Table I of the main paper.

TABLE II R EPORTED WCC-MMES TRAINING AND EVALUATION SETTINGS . Setting Training epochs Training MaxFEs Optimizer Initial learning rate Discount factor γ Rollout length nstep PPO update epochs K Resource budgets FeatureSE strength λ FeatureSE output biases Fully/non-separable grouping Main CEC2013LSGO tests DG2 decomposition allowance AOB tests Horizon tests NDA-transfer tests Independent comparison runs Significance test

Value 90 106 per episode Adam 10−4 ; exponential decay 0.999 5 4 {5×104 , 105 , 2×105 } FEs 0.2 2.0 20 subspaces 1.5×106 FEs 5×105 FEs, included in main budget 1.5×106 FEs 3×106 FEs 106 FEs 30 per algorithm and problem Wilcoxon rank-sum; α = 0.05

3) Training and evaluation: Table II collects the reported settings. The training functions are marked with an asterisk in Table I of the main paper. Training and held-out functions can share a separability category while differing in their base functions and other properties. Decomposition results are precomputed to accelerate training. Comparison methods follow the configurations in their original papers; the NDA implementations use libraries such as PyPop7 [16]. The policy trained at 106 FEs is reused for the generalization tests without retraining. Numerical comparisons report the mean and standard deviation over 30 independent runs. B. Ablation Study To further examine where the performance of WCC-MMES comes from, we conduct a comprehensive ablation study from four aspects: action space, state space, reward function, and network architecture. 1) Experimental Protocol: For the action space ablation, we consider both random and fixed variants. In the random variants, either both action components, only the mode (Random(amode )), or only the resource (Random(ares )) are randomized. In the fixed variants, either the mode is fixed to CC or NDA while the resource is controlled by the agent (Fixed(amode )), or the resource budget is fixed while the mode is controlled by the agent (Fixed(ares )). For the state space ablation, we remove each major state group individually, including problem features (W/O Spro ), convergence features (W/O Sconv ), resource features (W/O Sres ), and mode history features (W/O Smode ). For the reward ablation, the proposed log-ratio reward is replaced by Difference rt = cost∗t−1 − cost∗t , Sparse Sign  1, cost∗t−1 − cost∗t > ϵ rt = ϵ = 1E − 2 0, otherwise cost∗

−cost∗

t−1 t rewards. For the network , and Linear rt = |cost∗t−1 | ablation, we remove FeatureSE (W/O FeatureSE), replace the auto-regressive policy with a simple MLP, or remove both. For fair comparison, the final costs are min–max normalized

4

TABLE III WCC-MMES AND COMPARISON ALGORITHMS ON CEC2013LSGO AT 3 × 106 FE S .

Problem 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 +/≈/-

WCC-based CC-based Algorithms NDAs Hybrid Algorithms Algorithms WCC-MMES DG2-CMAES CC-CMAES LCC-CMAES MMES sep-CMA-ES LM-MA-ES FCMAES MetaES HCC-ES CC-SHADE-SL 2.03E-20 6.68E-20(+) 7.38E+06(+) 1.09E+05(+) 1.18E+04(+) 1.69E-17(+) 1.63E+05(+) 1.95E+05(+) 9.09E+09(+) 6.31E-20(+) 2.58E-05(+) ± 1.78E-20 ± 3.78E-21 ± 3.92E+06 ± 2.71E+04 ± 1.80E+03 ± 7.95E-19 ± 4.33E+04 ± 4.04E+04 ± 7.69E+08 ± 3.45E-21 ± 1.22E-04 1.35E+03 3.47E+03(+) 2.13E+03(+) 2.09E+03(+) 1.26E+04(+) 5.34E+03(+) 5.98E+03(+) 1.38E+04(+) 4.05E+04(+) 3.26E+03(+) 1.98E+03(+) ± 3.55E+02 ± 3.49E+02 ± 1.31E+02 ± 1.82E+02 ± 7.61E+02 ± 1.59E+02 ± 2.74E+02 ± 8.52E+02 ± 5.53E+03 ± 3.25E+02 ± 5.73E+02 2.00E+01 2.10E+01(+) 2.00E+01(≈) 2.03E+01(+) 2.00E+01(≈) 2.14E+01(+) 2.15E+01(+) 2.04E+01(+) 2.16E+01(+) 2.09E+01(+) 2.08E+01(+) ± 6.39E-03 ± 2.10E-02 ± 4.82E-03 ± 3.06E-02 ± 1.85E-03 ± 1.09E-01 ± 1.89E-01 ± 1.55E-01 ± 3.96E-01 ± 3.41E-02 ± 3.91E-02 1.32E+03 9.89E+02(-) 1.18E+11(+) 2.27E+10(+) 6.64E+07(+) 3.73E+10(+) 1.65E+11(+) 4.78E+08(+) 5.08E+12(+) 9.81E+03(+) 6.13E+09(+) ± 8.74E+02 ± 5.73E+00 ± 6.05E+10 ± 1.33E+09 ± 5.45E+06 ± 8.85E+09 ± 3.41E+10 ± 4.98E+07 ± 2.43E+09 ± 3.46E+02 ± 4.68E+09 1.39E+06 1.58E+06(+) 8.65E+06(+) 1.02E+07(+) 4.17E+06(+) 1.64E+06(+) 1.49E+06(+) 3.82E+06(+) 4.89E+07(+) 1.60E+06(+) 2.70E+06(+) ± 1.21E+05 ± 1.62E+05 ± 2.68E+06 ± 1.01E+06 ± 7.67E+05 ± 1.52E+05 ± 4.58E+05 ± 8.33E+05 ± 8.21E+05 ± 1.32E+05 ± 7.29E+05 9.95E+05 1.06E+06(+) 1.02E+06(+) 1.05E+06(+) 9.95E+05(≈) 1.06E+06(+) 1.05E+06(+) 9.96E+05(+) 1.06E+06(+) 1.06E+06(+) 1.05E+06(+) ± 4.08E-01 ± 1.22E+03 ± 1.56E+04 ± 2.08E+03 ± 2.14E-01 ± 1.78E+03 ± 1.35E+03 ± 9.38E+02 ± 1.63E+03 ± 9.98E+02 ± 2.25E+03 9.65E+01 1.38E+02(+) 1.46E+09(+) 4.91E+06(+) 2.45E+04(+) 2.36E+09(+) 3.72E+06(+) 1.65E+05(+) 1.21E+15(+) 9.01E+02(+) 9.23E+05(+) ± 4.97E+00 ± 2.20E+01 ± 2.15E+09 ± 3.11E+06 ± 1.58E+03 ± 1.87E+09 ± 4.13E+05 ± 3.53E+04 ± 7.78E+10 ± 2.44E+01 ± 8.83E+05 1.49E+07 2.42E+06(-) 4.53E+15(+) 3.99E+14(+) 1.84E+11(+) 5.44E+14(+) 1.71E+15(+) 9.49E+11(+) 1.46E+17(+) 3.11E+06(-) 1.21E+14(+) ± 3.70E+06 ± 1.09E+06 ± 2.66E+15 ± 7.17E+13 ± 7.76E+09 ± 6.78E+14 ± 4.20E+14 ± 1.96E+11 ± 6.21E+15 ± 9.40E+05 ± 5.46E+13 1.13E+08 1.20E+08(+) 7.60E+08(+) 8.19E+08(+) 4.41E+08(+) 1.41E+08(+) 2.20E+08(+) 5.61E+08(+) 3.73E+09(+) 1.85E+08(+) 2.27E+08(+) ± 3.12E+07 ± 1.56E+07 ± 1.42E+08 ± 1.24E+08 ± 8.42E+07 ± 1.70E+07 ± 2.20E+07 ± 9.23E+07 ± 1.74E+06 ± 1.24E+07 ± 1.12E+08 9.05E+07 9.09E+07(+) 9.41E+07(+) 9.22E+07(+) 9.05E+07(≈) 9.41E+07(+) 9.40E+07(+) 9.06E+07(≈) 9.40E+07(+) 9.10E+07(+) 9.26E+07(+) ± 6.43E+00 ± 2.77E+05 ± 5.04E+05 ± 2.02E+05 ± 3.47E+02 ± 1.66E+05 ± 2.77E+05 ± 3.98E+04 ± 6.90E+05 ± 2.87E+05 ± 3.44E+05 2.89E+04 2.94E+04(+) 3.76E+11(+) 4.38E+09(+) 2.14E+06(+) 3.85E+11(+) 8.50E+07(+) 1.41E+07(+) 6.47E+16(+) 3.34E+04(+) 1.71E+08(+) ± 2.54E+04 ± 4.36E+04 ± 2.85E+11 ± 4.83E+09 ± 1.87E+05 ± 1.36E+10 ± 1.46E+07 ± 1.96E+06 ± 3.23E+15 ± 3.96E+04 ± 9.20E+07 7.91E+02 1.18E+03(+) 1.29E+03(+) 1.31E+03(+) 8.62E+02(+) 8.66E+02(+) 6.14E+02(-) 5.34E+02(-) 1.04E+08(+) 1.18E+03(+) 1.32E+03(+) ± 1.84E+01 ± 8.26E+01 ± 4.91E+03 ± 1.61E+02 ± 5.76E+02 ± 3.94E+01 ± 1.68E+02 ± 2.29E+02 ± 6.44E+08 ± 8.78E+01 ± 2.89E+02 2.46E+05 1.53E+08(+) 5.79E+10(+) 2.56E+07(+) 2.89E+05(+) 9.63E+10(+) 8.74E+06(+) 1.23E+06(+) 9.44E+16(+) 2.27E+06(+) 9.75E+07(+) ± 1.45E+04 ± 1.06E+08 ± 3.17E+10 ± 4.84E+06 ± 4.27E+04 ± 6.91E+10 ± 9.15E+05 ± 1.24E+05 ± 7.49E+15 ± 1.01E+06 ± 1.19E+08 5.81E+06 3.04E+07(+) 8.53E+11(+) 1.53E+08(+) 7.91E+06(+) 9.01E+09(+) 1.02E+08(+) 2.14E+07(+) 9.47E+17(+) 9.92E+06(+) 9.41E+07(+) ± 4.37E+06 ± 8.08E+06 ± 4.51E+11 ± 1.82E+07 ± 1.22E+06 ± 5.70E+09 ± 6.27E+06 ± 1.32E+06 ± 6.19E+15 ± 1.19E+06 ± 1.11E+08 3.89E+05 3.54E+13(+) 3.06E+08(+) 1.85E+07(+) 1.29E+06(+) 2.66E+08(+) 9.88E+06(+) 6.61E+06(+) 2.35E+15(+) 2.79E+13(+) 2.00E+06(+) ± 1.67E+05 ± 6.35E+12 ± 3.06E+08 ± 3.59E+06 ± 1.35E+05 ± 5.27E+07 ± 2.35E+06 ± 9.06E+05 ± 1.62E+12 ± 1.61E+13 ± 5.73E+05 NA 13/0/2 14/1/0 15/0/0 12/3/0 15/0/0 14/0/1 13/1/1 15/0/0 14/0/1 15/0/0

Entries report means and standard deviations over 30 runs. Symbols +, ≈, and − indicate WCC-MMES is significantly better, comparable, and worse, respectively (Wilcoxon rank-sum, α = 0.05). The final row reports win/tie/loss counts.

Costs are normalized separately for each problem and ablation category before averaging. Higher scores indicate better performance within the corresponding category. 3) Ablation Analysis: The results are shown in Figure 1. a) Action space. The key takeaway from Figure 1(a) is that WCC benefits from jointly learning both the optimization mode and the resource allocation. Fixing the mode leads to the worst performance, indicating that neither CC nor NDA is universally optimal across all problems and stages. Randomizing either the mode selector or the resource allocator

0.7

0.0

0.283

W/O Smode

0.2

0.3 0.2 0.1 0.0

0.535

0.365 0.4 0.2

0.0

Fixed(ares = 2E5)

Net Ablation

0.750 0.572

0.6

0.398

Fixed(ares = 1E5)

Fixed(ares = 5E4)

Fixed(amode = 1)

Fixed(amode = 0)

0.8

0.714

0.6

0.564 0.5 0.4

0.391

W/O Sres

0.4

0.619

WCC-MMES

0.6

（d）

0.459

0.396

W/O FeatureSE Simple MLP

0.843 0.8

Reward Ablation

0.817

Simple MLP

（c）

0.8

0.817

W/O FeatureSE

State Ablation

WCC-MMES

（b）

0.417

Difference

1.0

Sparse Sign

0.0

Linear

0.2

0.580

Random(ares)

0.4

Random

0.6

0.907

0.861

WCC-MMES

0.8

0.874

Random(amode)

0.797

WCC-MMES

Mean Normalized Performance

0.939

W/O Sconv

p∈Pc

1.0

W/O Spro

For the problem set Pc evaluated in category c, the reported mean normalized performance is 1 X (c) qp,a . (19) Q(c) a = |Pc |

Action Ablation

（a）

Mean Normalized Performance

within each category, and the mean normalized performance over all problems is reported. All ablated variants are trained and evaluated under the same protocol as Section III-B1 of the main paper. 2) Normalization and reporting: For category c, let Ac contain WCC-MMES and the variants compared in that category, and let f¯p,a denote the mean final cost of variant a on problem min = mina∈Ac f¯p,a and p. The per-problem extremes are fp,c max ¯ fp,c = maxa∈Ac fp,a . The inverse min–max score is  min ¯  1 − fp,a − fp,c , f max > f min , p,c p,c (c) max min fp,c − fp,c qp,a = (18)  1, max min fp,c = fp,c .

Fig. 1. Ablation study, including (a) action space ablation, (b) state space ablation, (c) reward function ablation, and (d) network architecture ablation.

also degrades performance, showing that the learned decisions cannot be replaced by arbitrary switching. b) State space. The key takeaway from Figure 1(b) is that the four state groups provide complementary information rather than redundant signals. Removing any major state group

5

leads to a clear performance drop: problem features provide structural priors such as dimensionality and decomposability, convergence features capture the current optimization dynamics and cause the largest degradation when removed, while resource and mode-history features describe budget consumption and historical mode preference, improving the timing and stability of the learned switching policy. c) Reward function. The key takeaway from Figure 1(c) is that a stable and scale-aware reward is critical for learning across heterogeneous LSGO problems. Replacing the proposed log-ratio reward with Difference, Sparse Sign, or Linear rewards consistently worsens performance: Difference rewards are sensitive to objective scales, Sparse Sign rewards discard the magnitude of improvement and depend on threshold selection, and Linear rewards may still suffer from numerical instability. d) Network architecture. The key takeaway from Figure 1(d) is that both FeatureSE and the auto-regressive policy contribute to WCC’s performance. Removing FeatureSE degrades performance, confirming the usefulness of adaptively reweighting heterogeneous state features. Replacing the auto-regressive policy with a simple MLP also hurts performance, because WCC naturally involves a hierarchical decision process: the agent should first select CC or NDA and then allocate FEs conditioned on the selected mode. The largest degradation occurs when both FeatureSE and the auto-regressive structure are removed, indicating that heterogeneous state encoding and mode-conditioned resource allocation jointly support effective CC deployment. R EFERENCES [1] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017. [2] X. Li, K. Tang, M. N. Omidvar, Z. Yang, and K. Qin, “Benchmark functions for the CEC’2013 special session and competition on largescale global optimization,” Evolutionary Computation and Machine

Learning Group, RMIT University, Melbourne, Australia, Technical Report, 2013. [3] W. Qiu, H. Guo, Z. Ma, and Y.-J. Gong, “A novel two-phase cooperative co-evolution framework for large-scale global optimization with complex overlapping,” arXiv preprint arXiv:2503.21797, 2025. [4] N. Hansen and A. Ostermeier, “Completely derandomized selfadaptation in evolution strategies,” Evolutionary computation, vol. 9, no. 2, pp. 159–195, 2001. [5] X. He, Z. Zheng, and Y. Zhou, “Mmes: Mixture model-based evolution strategy for large-scale optimization,” IEEE Transactions on Evolutionary Computation, vol. 25, no. 2, pp. 320–333, 2020. [6] M. N. Omidvar, M. Yang, Y. Mei, X. Li, and X. Yao, “Dg2: A faster and more accurate differential grouping for large-scale black-box optimization,” IEEE Transactions on evolutionary computation, vol. 21, no. 6, pp. 929–942, 2017. [7] J. Liu and K. Tang, “Scaling up covariance matrix adaptation evolution strategy using cooperative coevolution,” in International Conference on Intelligent Data Engineering and Automated Learning. Springer, 2013, pp. 350–357. [8] H. Guo, W. Qiu, Z. Ma, X. Zhang, J. Zhang, and Y.-J. Gong, “Advancing cma-es with learning-based cooperative coevolution for scalable optimization,” arXiv preprint arXiv:2504.17578, 2025. [9] M. Yang, A. Zhou, C. Li, and X. Yao, “An efficient recursive differential grouping for large-scale continuous problems,” IEEE Transactions on Evolutionary Computation, vol. 25, no. 1, pp. 159–171, 2020. [10] M. Tian, M. Chen, W. Du, Y. Tang, and Y. Jin, “An enhanced differential grouping method for large-scale overlapping problems,” IEEE Transactions on Evolutionary Computation, 2024. [11] R. Ros and N. Hansen, “A simple modification in cma-es achieving linear time and space complexity,” in International conference on parallel problem solving from nature. Springer, 2008, pp. 296–305. [12] I. Loshchilov, T. Glasmachers, and H.-G. Beyer, “Large scale black-box optimization by limited-memory matrix adaptation,” IEEE Transactions on Evolutionary Computation, vol. 23, no. 2, pp. 353–358, 2018. [13] Z. Li, Q. Zhang, X. Lin, and H.-L. Zhen, “Fast covariance matrix adaptation for large-scale black-box optimization,” IEEE transactions on cybernetics, vol. 50, no. 5, pp. 2073–2083, 2018. [14] R. Lange, T. Schaul, Y. Chen, T. Zahavy, V. Dalibard, C. Lu, S. Singh, and S. Flennerhag, “Discovering evolution strategies via meta-black-box optimization,” in Proceedings of the companion conference on genetic and evolutionary computation, 2023, pp. 29–30. [15] A. Vakhnin and E. Sopov, “A novel self-adaptive cooperative coevolution algorithm for solving continuous large-scale global optimization problems,” Algorithms, vol. 15, no. 12, p. 451, 2022. [16] Q. Duan, G. Zhou, C. Shao, Z. Wang, M. Feng, Y. Huang, Y. Tan, Y. Yang, Q. Zhao, and Y. Shi, “Pypop7: A pure-python library for population-based black-box optimization,” Journal of Machine Learning Research, vol. 25, no. 296, pp. 1–28, 2024.

Related documents

• WCC: A Learning-Based Framework for Deciding When to Apply Cooperative Coevolution in Large-Scale Optimization#1468933
• Toward sustainable and customer-centric reverse logistics: Machine learning-enhanced multi-objective optimization#261950
• AlgoWorlds: Benchmarking Tool Use for Global Optimization in Algorithmic Worlds#794191
• AlgoWorlds: Benchmarking Tool Use for Global Optimization in Algorithmic Worlds#790172
• DMARS_WGO: a deep reinforcement-driven hybrid metaheuristic for intelligent adaptive optimization.#138599