Uncertainty-Aware Probabilistic Constrained Clustering from Entangled Pairwise Supervision

Uncertainty-Aware Probabilistic Constrained Clustering from Entangled Pairwise Supervision Shaojie Zhang Ke Chen Department of Computer Science, The University of Manchester, Manchester M13 9PL, U.K. {shaojie.zhang,ke.chen}@manchester.ac.uk

arXiv:2608.12027v1 [cs.LG] 12 Aug 2026

Abstract Pairwise constrained clustering typically relies on hard must-link/cannot-link labels, whereas realistic pairwise supervision may be real-valued and entangle intrinsic ambiguity, expert judgment, and stochastic corruption. Existing deep constrained clustering (DCC) methods mainly target hard, expert-agnostic constraints, treating soft labels mostly numerically rather than semantically. We formalize this setting as uncertainty-aware probabilistic constrained clustering (UPCC), defining a canonical aleatoric target through a heterogeneous observation process and analyzing its conditional identifiability. We introduce ProbPair, an angular pairwise objective for probabilistic relations, and build ECI-PP, an estimator–corrector–integrator framework that refines imperfect supervision via belief estimation, correction, and reliability-aware integration. Across challenging probabilistic supervision settings, experiments on diverse benchmarks show that ECI-PP outperforms state-of-the-art DCC methods and remains robust with a shared default configuration.

1

Introduction

Clustering is a fundamental tool for uncovering structure in data, yet its unsupervised nature often yields partitions that may not align with domain knowledge [1, 2, 3]; pairwise constrained clustering (CC) addresses this by using weak instance-level pairwise supervision [4, 5]. Traditional CC methods [6, 7, 8, 9, 10, 11], however, often struggle with high-dimensional and complex data, motivating deep constrained clustering (DCC). Existing DCC methods can be broadly organized into two paradigms: end-to-end DCC [12, 13, 14, 15], which reformulates clustering as a pseudo-classification problem with anchors, and deep constraint embedding [16, 17], which instead learns clustering-friendly representations from pairwise supervision and, in more recent advances, further moves pairwise learning from Euclidean to angular space to better reconcile positive and negative relations [3]. Despite these advances, the supervision setting remains largely idealized. Existing work typically assumes hard binary pairwise relations, whereas supervision in practice often lies on a continuum. Moreover, such real-valued pairwise labels may simultaneously reflect intrinsic aleatoric ambiguity, expert-conditioned and pair-dependent epistemic judgment [18], and additional stochastic corruption from annotation or recording pipelines. For example, when comparing two clinical cases, a recorded observation may reflect ambiguity in the cases themselves, systematic clinician judgment bias, and documentation or data-entry errors. These considerations motivate what we term uncertaintyaware probabilistic constrained clustering (UPCC): a more specific and realistic setting in which probabilistic pairwise relations arise from a heterogeneous observation process. In this paper, we formalize the UPCC setting and develop a corresponding solution. We specify an observation model that distinguishes the aleatoric target from expert-conditioned judgment and stochastic corruption, and analyze when this canonical target is identifiable on observable pairs. To learn under UPCC, we first introduce ProbPair, a deep constraint embedding approach linking probabilistic pairwise relations to angular representation learning. Built on it, we further develop a Preprint.

Estimator–Corrector–Integrator ProbPair (ECI-PP) framework, in which Estimators provide model beliefs, Correctors perform structured correction and suppress residual discordance, and the Integrator learns from the resulting surrogate supervision toward the underlying aleatoric relation. Our main contributions are: (i) We formalize UPCC through a heterogeneous observation model and analyze identifiability of the canonical aleatoric target on observable pairs. (ii) We introduce ProbPair, a deep constraint embedding approach linking probabilistic pairwise relations to angular embeddings. (iii) We develop ECI-PP, a practical framework for aleatoric relation learning under entangled supervision. (iv) We empirically show across diverse benchmarks that ECI-PP outperforms state-of-the-art DCC methods and is robust under fallible, heterogeneous, and corrupted supervision.

2

Related work

Beyond binary pairwise constraints. Existing CC methods either extend binary constraints only numerically, without semantically modeling intermediate supervision, or assign intermediate values to the confidence of a pre-defined relation type rather than a unified probability. Advanced deep methods typically employ logistic losses [19, 3] compatible with real-valued constraints in [0, 1]. However, end-to-end DCC methods [12, 14, 20, 21, 22, 15] remain tied to hard anchor semantics, with attendant anchor misalignment [23] and error propagation [24, 25], while deep constraint embedding methods [3, 17, 16] avoid anchors but still require a binary regime for their theoretical geometric guarantees. Likewise, the deep generative approach [2] encodes constraints as a signed prior over discrete assignments, precluding a unified treatment of real-valued relations. Earlier non-deep approaches considered intermediate supervision more explicitly: (i) probabilistic treatments [26, 27] encoded the activation probability of a positive same-cluster tie, but offered neither negative relations nor a unified notion of pairwise affinity; and (ii) fuzzy-constraint formulations attach an intermediate belief [28], degree [29], or penalty [30, 31, 32, 33] only after a discrete relation type has been specified. Taken together, these lines of work do not address uncertain relational judgment expressed by probabilistic labels under a unified semantics. By contrast, our ProbPair formulation, built on deep constraint embedding, directly models such unified probabilistic supervision, accommodating both vague annotations in practice and inherently ambiguous aleatoric relations. Imperfect pairwise supervision. While imperfect annotations in CC have long been recognized [34, 35, 36, 37, 38, 39, 1, 2, 15], more realistic observation settings remain underexplored. Existing studies treat simplified noise through constraint-set robustness [34, 36], coarse prior uncertainty over assignments [2], or expert-level reliability in semi-crowdsourced settings [38, 1]. VolMaxDCC [15] further introduces confusion-based structured annotation modeling beyond unstructured label degradation or flipping. However, it still lacks heterogeneous multi-expert modeling and remains restricted to discrete binary-noise settings, leaving expert-specific, pair-dependent epistemic distortion unmodeled. By contrast, we consider genuinely entangled observations, arising from intrinsic aleatoric relations, expert-conditioned epistemic judgments, and stochastic corruption. Such a setting is reminiscent of learning from crowds, where expert behavior is commonly modeled under a fixed class-label scheme through expert-specific confusion or output layers [40, 41], or through expertise coupled with instance difficulty [42, 43]. Yet these strategies rely on a restricted class-label channel and are therefore not directly transferable to deep constraint embedding, where a highly flexible pairwise learner can readily absorb structured distortion and random corruption into its learned geometry. We therefore develop ECI-PP, a multi-role iterative framework built on ProbPair that uses model-side beliefs to correct structured distortion and filter residual inconsistency, yielding refined surrogate supervision that progressively guides learning toward the underlying aleatoric relation.

3

UPCC formulation and canonical target

We study uncertainty-aware probabilistic constrained clustering (UPCC), where expert-indexed real|X | valued pairwise labels arise from a heterogeneous observation process. Let X = {xj }j=1 be a dataset |C|

to be partitioned into C clusters, and let C = {(ai , bi , ei , yi )}i=1 denote the observed soft constraints, where (ai , bi ) indexes the pair (xai , xbi ), ei ∈ {1, . . . , E} records the expert identity, and yi ∈ [0, 1] is the observed pairwise relation. The formulation targets a canonical pairwise probabilistic relation, while a final hard partition is treated as a downstream summary induced by that relation. 2

3.1

Observation model

We view each observed pairwise relation as arising from three factors: (i) an underlying aleatoric relation in the data (e.g., intrinsic clinical similarities between a common cold and the flu), (ii) an expert-conditioned epistemic judgment built on that relation (e.g., a novice doctor systematically overrating their similarity due to limited knowledge), and (iii) a subsequent stochastic corruption stage accounting for accidental recording noise (e.g., random clerical errors in medical records). Aleatoric relation. Let S be the set of all C-partitions of X , and let S ∈ S be a latent random partition. For any index pair (a, b), define the aleatoric co-membership relation
⋆ Rab := Pr xa and xb belong to the same cluster under S | X ∈ [0, 1].

This quantity serves as the canonical target of the formulation and remains well defined even when the latent partition is intrinsically ambiguous.

Epistemic judgment. Let σ(·) denote the
sigmoid function, and define its clamped inverse logitε (u) := logit min{max{u, ε}, 1 − ε} with ε ≪ 1. We also introduce a deterministic pair descriptor ϕab = ϕ(xa , xb ), which serves as the covariate through which expert-dependent effects are parameterized. For expert e, let me (ϕab ) ∈ R denote a structured mean effect, and let
ue,ab | e, ϕab ∼ N 0, τe2 (ϕab ) , τe (ϕab ) > 0, be a centered residual perturbation. The latent expert judgment is generated as
jud ⋆ ye,ab := σ logitε (Rab ) + me (ϕab ) + ue,ab ∈ (0, 1).

Under this construction, the judgment channel has the logistic-normal density  
1 ⋆ ⋆ φ logit(y); logitε (Rab ) + me (ϕab ), τe2 (ϕab ) pjud y | Rab , e, ϕab = y(1 − y)

(1)

(2)

for y ∈ (0, 1), where φ(·; µ, s2 ) denotes the Gaussian density; see Appendix A.1 for the derivation. The same judgment channel also gives rise to the corresponding mean distortion in probability space, h i jud ⋆ ⋆ ⋆ ϵ̄e,ab (Rab , ϕab ) := E ye,ab − Rab | Rab , e, ϕab . (3) Stochastic corruption. To capture unstructured recording noise beyond the latent judgment, we introduce a corruption indicator ce,ab ∼ Bernoulli(πe ), where πe ∈ [0, 1) is the corruption probability for expert e, and define the final observed relation by ⋆ ye,ab | ce,ab = 0 ∼ pjud (· | Rab , e, ϕab ),

ye,ab | ce,ab = 1 ∼ Uniform(0, 1).

Equivalently, letting 1[0,1] (·) denote the uniform density on [0, 1], the conditional observation law is

⋆ ⋆ p y | Rab , e, ϕab = (1 − πe ) pjud y | Rab , e, ϕab + πe 1[0,1] (y). (4) 3.2

Canonical target and identifiability

We now formalize the canonical target within a structural setup for identifiability. To obtain a finite-dimensional canonical estimand, we parameterize ⋆ Rab = rθ (xa , xb ),

me (ϕab ) = mηe (ϕab ),

τe (ϕab ) = τζe (ϕab ),

E E and collect the parameters into Ξ = (θ, {ηe }E e=1 , {ζe }e=1 , {πe }e=1 ). This parameterization makes the canonical target explicit under the specific observation family. Since ϕab is introduced only as a deterministic pair descriptor without additional representational constraints, the identification of the canonical relation term rθ (xa , xb ) is governed by the structural conditions introduced below.

We first fix the trivial additive ambiguity in the location decomposition: Assumption 3.1 (Centering on observable support). For each expert e with nonzero observation probability, E(a,b)|e [mηe (ϕab )] = 0, where the expectation is taken over the observable-pair distribution conditional on expert e. 3

This centering removes any remaining expert-specific constant shift on the observable support: E Lemma 3.2 (Gauge fixing). Suppose both {mηe }E e=1 and {mηe′ }e=1 satisfy Assumption 3.1. If for each expert e with nonzero observation probability there exists a constant ke ∈ R such that mηe (ϕab ) = mηe′ (ϕab ) − ke for all observable triples (a, b, e), then ke = 0 for every such expert e. The proof is given in Appendix A.2. The logistic-normal clean channel in Eq. (2), combined with the observation law in Eq. (4), yields the following injectivity property: ⋆ Lemma 3.3 (Injectivity of the observation family). Let µ := logitε (Rab ) + me (ϕab ), s := τe (ϕab ), and π := πe . For the observation law rewritten as
1 p(y | µ, s, π) := (1 − π) φ logit(y); µ, s2 + π 1[0,1] (y), y ∈ (0, 1), (5) y(1 − y) the mapping (µ, s, π) 7→ p(· | µ, s, π) is injective on R × (0, ∞) × [0, 1). The proof is given in Appendix A.3. The canonical formulation is
completed by the following structural separability condition on the location term logitε rθ (xa , xb ) + mηe (ϕab ): Assumption 3.4 (Structural separability). Under Assumption 3.1, if for all observable triples (a, b, e),

logitε rθ (xa , xb ) + mηe (ϕab ) = logitε rθ′ (xa , xb ) + mηe′ (ϕab ), then for each expert e there exists a constant ke ∈ R such that for all such observable triples,

logitε rθ (xa , xb ) = logitε rθ′ (xa , xb ) + ke , mηe (ϕab ) = mηe′ (ϕab ) − ke . Assumption 3.4 provides the key structural requirement for the identifiability argument; boundary cases clarifying this condition are discussed in Appendix A.4. Writing the support of observable pairs as Pobs := {(a, b) : ∃e such that (a, b, e) is observable}, we then have the identifiability result: Theorem 3.5 (Conditional canonical identifiability of the aleatoric relation). Let Ξ and Ξ′ satisfy the observation model and Assumption 3.1 and 3.4. Assume further that rθ (xa , xb ), rθ′ (xa , xb ) ∈ [ε, 1−ε] for all (a, b) ∈ Pobs . If the induced conditional laws of the observed response coincide under Ξ and Ξ′ for every observable triple (a, b, e), then rθ (xa , xb ) = rθ′ (xa , xb ) for all (a, b) ∈ Pobs . ⋆ Hence the canonical aleatoric relation Rab = rθ (xa , xb ) is identifiable on observable pairs. The proof is in Appendix A.5. Under the specific observation family, Theorem 3.5 gives conditional identifiability of the canonical aleatoric relation on observable pairs at the law level. In the practical single-observation regime, however, estimation still relies on the shared parametric structure imposed across pairs and experts rather than repeated observations of the same pair–expert combination.

4

Method

We first introduce a probability-aware pairwise learning formulation that links probabilistic pairwise relations to clustering-oriented representations. Building on this formulation and motivated by the theoretical observation model, we develop a practical learning framework in which a surrogate supervision route guides learning from entangled supervision toward the aleatoric relation. 4.1

ProbPair: probability-aware pairwise representation learning

We establish a representation learning formulation under probabilistic pairwise supervision. Given su|C| pervised pairs {(ai , bi , yi )}i=1 , where yi ∈ [0, 1] denotes the probabilistic relation for pair (xai , xbi ), we aim to learn clustering-oriented representations. To this end, we adopt a deep constraint embedding formulation with encoder fψ : X → Z ⊂ RD and latent code zj = fψ (xj ). We work in angular space, where bounded cosine similarity provides a natural geometric quantity for probabilistic calibration. To connect probabilistic pairwise relations with the latent representation space, we introduce the following probabilistic readout:
ŷai bi := σ (cos(zai , zbi ) − m)/T ∈ (0, 1), (6) where m ∈ R and T > 0 are learnable parameters controlling the operating point and sharpness of the mapping. The resulting pairwise objective, referred to as the ProbPair loss, is given by |C| i 1 Xh LPP = − yi log ŷai bi + (1 − yi ) log(1 − ŷai bi ) . |C| i=1

4

(7)

Figure 1: Overview of the ECI-PP pipeline. The Estimators are trained with the ProbPair objective L (Eq. (8)) and produce out-of-fold beliefs ŷioof via the readout in Eq. (6). The Correctors take [ŷioof ; ϕi ] ˆ i via Eq. (10), yielding as input, with ϕi the pair-i descriptor, and learn probability-space corrections ∆ cor corrected relations yi . Reliability-aware screening produces reliability weights wi , and BC fusion forms the refined supervision yiBC via Eqs. (11) and (12). The Integrator learns embeddings from the refined supervision; its Integrator-side relations yiint are fed back to the Estimators for iterative refinement, while the final embeddings Z int are clustered into the final partition. Unlike hard-constraint objectives, Eq. (7) directly accommodates non-binary relation strengths. As a result, the induced geometry need not collapse to a purely binary regime, but can preserve both confident and uncertain pairwise structure. We further include a reconstruction regularizer to avoid degenerate representations and preserve the global structure of the data. Since the representation is learned in angular space, reconstruction is performed from normalized latent embeddings, as in [3]. Specifically, with decoder gψ′ : Z → X and Norm(·) denoting ℓ2 normalization, we decode from x̂j = gψ′ (Norm(zj )) and use reconstrucP|X | tion loss Lrec = |X1 | j=1 ∥xj − x̂j ∥22 . With λrec > 0 balancing pairwise relation fitting and reconstruction regularization, the overall objective is L = LPP + λrec Lrec . (8) |X |

Minimizing Eq. (8) yields optimal normalized embeddings Znorm = {Norm(zj )}j=1 that are shaped by probabilistic pairwise relations while being regularized by reconstruction. These angular embeddings can then be used by a downstream clustering algorithm to obtain the final partition, and can also produce probabilistic relation estimates for instance pairs through Eq. (6). 4.2

Estimator–Corrector–Integrator ProbPair

Building on the above formulation, we develop a practical framework motivated by the observation model. Rather than learning directly from entangled supervision, it progressively refines the supervision signal by correcting the structured observational component, screening the residual inconsistency, and feeding the refined supervision back into learning, thereby yielding an iterative Estimator–Corrector–Integrator ProbPair (ECI-PP) design for learning toward the aleatoric relation. The overall ECI-PP flow is summarized in Fig. 1. Estimator. To enable the subsequent learning of structured observational effects from entangled supervision, ECI-PP first constructs a model-side reference that is less coupled to the raw observations. In flexible pairwise representation learning, beliefs fitted on the same constraints used for training can absorb noisy observations in-sample and are therefore unreliable for this role. We accordingly (k) K partition C into K disjoint folds {C (k) }K }k=1 , k=1 and instantiate K non-shared Estimators {Est (k) ′ where Est = (fωk , gωk , mk , Tk ) denotes the encoder, decoder, and fold-specific readout parameters. Each Est(k) is trained on C \ C (k) by minimizing L in Eq. (8), and then produces holdout beliefs on C (k) through Eq. (6). Aggregating these predictions over all folds yields a full set of out-of-fold |C| beliefs {ŷioof }i=1 , which serves as the model-side reference. To initialize the estimator, we weight each observed pair in ProbPair loss by an information-based decisiveness score, defined as
DKL Bern(yi ) ∥ Bern(ȳ)

∈ [0, 1], κi = (9) (1 − yi ) DKL Bern(0) ∥ Bern(ȳ) + yi DKL Bern(1) ∥ Bern(ȳ) 5

P|C| 1 where ȳ = |C| i=1 yi , DKL (·∥·) denotes the Kullback–Leibler divergence, and Bern(·) denotes the Bernoulli law. This weighting emphasizes more decisive judgments, which are typically expressed with greater expert confidence, and thereby stabilizes the initial out-of-fold beliefs. Corrector. Given the Estimators’ out-of-fold beliefs, the Corrector aims to learn structured expertspecific corrections in the observed supervision. These corrections are motivated by the epistemic component defined in logit space (Section 3.1), whereas for practical stability we work with its mean ⋆ probability-space distortion ϵ̄e,ab (Rab , ϕab ) (see Eq. (3)), because small probability discrepancies near the 0/1 boundaries induce disproportionately large logit residuals. Although approximate and ⋆ not preserving the variance structure encoded by τe (ϕab ), ϵ̄e,ab (Rab , ϕab ) remains a reasonable proxy because it captures the structured mean effect induced by the epistemic channel, making the resulting Corrector a practical surrogate module. We take ŷioof as the current proxy for Ra⋆i bi and introduce expert-specific Correctors {Core }E e=1 in probability space, where each Core is parameterized by a network qνe on Ie := {i : ei = e} with input [ŷioof ; ϕi ]; here we instantiate ϕi = [xai ⊙xbi ; |xai −xbi |] with ⊙ denoting element-wise product, as a simple and stable generic pair feature. Writing ∆i := ŷioof − yi ∈ (−1, 1), the Corrector output is ˆ i := qν ([ŷ oof ; ϕi ]) ∈ (−1, 1). Since ∆i is an empirical target formed from the observed yi and ∆ ei i the proxy ŷioof , it may also contain stochastic corruption and proxy error in addition to structured epistemic effects. We therefore train each small expert-specific Corrector with regularization: i 1 Xh ˆ ˆ i |2 . ρ(∆i − ∆i ) + λcor |∆ (10) L(e) cor = |Ie | i∈Ie

Here, ρ(·) is the Huber loss and λcor > 0 weights the ℓ2 regularizer, which biases the limited-capacity Corrector toward shared learnable structure rather than arbitrary pair-specific fluctuations. Integrator. The Integrator, parameterized as Int = (fω , gω′ , m, T ), is the learner that integrates the refined supervision into a clustering-oriented embedding. To build this refined supervision from the Corrector output, we first form a corrected relation yicor , quantify the remaining inconsistency by gapi , and convert it into a screening weight wi , with γ > 0 controlling the sharpness of down-weighting: ˆ i ), yicor := softclipξ (yi + ∆

gapi := softclipξ (ŷioof ) − yicor ,

wi := (1 − gapi )γ . (11)

Here, softclipξ (u) := ξ −1 softplus(ξu) − ξ −1 softplus(ξ(u − 1)) with ξ > 0 softly maps values into (0, 1) while preserving gradient flow near the boundaries; applying the same map to ŷioof keeps the gap computation in the same bounded space. The residual gapi quantifies the unexplained discrepancy that remains after structured correction. While the observation model associates this discrepancy with stochastic corruption and residual epistemic variation, gapi may also reflect proxy error in ŷioof and imperfect Corrector fitting in practice. We therefore treat it uniformly as a reliability signal for heuristic screening through wi . We then set ni := n0 wi with n0 > 0, and construct the Bayesian-confidence supervision yiBC := (yi + ni yicor )/(1 + ni ),

(12)

which keeps yi as a fixed prior while letting yicor contribute according to its screened reliability. The Integrator then optimizes L (Eq. (8)) using yiBC as the pairwise target, with wi multiplying each per-constraint term in LPP (Eq. (7)) before averaging, yielding Integrator embeddings.

Iterative refinement and deployment. ECI-PP operates by passing supervision through the Estimator–Corrector–Integrator pipeline to produce, via Eq. (6), Integrator-side relations yiint , and then feeding (yiint , wi ) back to the Estimators for iterative refinement. Warm-up initializes this loop by starting the Estimators from (yi , κi ); subsequent rounds replace this with (yiint , wi ). At each round, the refreshed Estimator beliefs are passed again through the Corrector–Integrator pipeline to update yicor , gapi , wi , and yiBC , thereby yielding a practical finite-stage refinement procedure that progressively improves the surrogate supervision toward the aleatoric relation. After refinement, the |X | final normalized Integrator embeddings Z int := {Norm(fω (xj ))}j=1 encode the learned pairwise relation structure; applying an unsupervised clustering algorithm to Z int then yields the final partition. Appendix B provides algorithmic details and complexity analysis. 6

5

Experiments

5.1

Experimental settings

Our experiments evaluate ECI-PP under UPCC along four axes: (i) comparison with DCC and probabilistic pairwise baselines, (ii) robustness to expert quality, corruption, and multi-expert variation, (iii) held-out diagnostics of internal dynamics, and (iv) sensitivity and ablation of key designs. Datasets. We adopt six image benchmarks (CIFAR100-20, CIFAR10 [44], FMNIST [45], ImageNet10 [46], MNIST [47], STL10 [48]) and two text benchmarks (Reuters subset [49], RCV1-10 [3]), covering varying class counts and class imbalance. Dataset details and splits are in Appendix C.1. Compared methods. We compare ECI-PP with six baselines applicable to probabilistic supervision: VanillaDCC [19] as a basic logistic pairwise baseline, VolMaxDCC [15] with explicit noise modeling, CIDEC [14] and SpherePair [3] as state-of-the-art end-to-end DCC and deep constraint embedding methods, respectively, and ProbPair/Weighted ProbPair as direct ablations. ProbPair uses only L in Eq. (8), while Weighted ProbPair additionally uses the information-based weight in Eq. (9). Constraint generation. To instantiate UPCC, we generate constraints with trained expert classifiers rather than deriving binary labels from ground-truth classes. Experts are trained on controlled labeled jud subsets; uniformly sampled constraint pairs assigned to expert e are given a judgment ye,ab = e e ⟨pa , pb ⟩ ∈ [0, 1] from the predictive distributions, and the recorded yi is then produced through the corruption channel in Section 3.1. For single-expert supervision, settings lv0.1/lv0.01/lv0.001 vary the labeled-data fraction used to train the expert. Multi-expert settings multi2/multi3/multi10 use multiple experts with complementary familiar/unfamiliar categories, where unfamiliar categories act as expert blind spots. See Appendix C.3 for the full procedure. Protocol. For a comparative study, we report clustering metrics (ACC/NMI/ARI) over five trials; embedding-based methods use K-means on learned representations, while end-to-end DCC baselines use native outputs. The main comparison uses the default lv0.01 single-expert and multi3 multiexpert regimes, both with corruption probability 0.3 and 9k constraints in total (3k per expert under multi3). To assess robustness, we vary single-expert quality, corruption probability, and multi-expert configuration with matched constraint budgets. To probe ECI-PP beyond final clustering scores, we conduct held-out internal-signal diagnostics and sensitivity/ablation studies. Implementation. For fair comparison, we follow established fully connected architectures and the unsupervised pretraining protocol in [3, 15]: encoder-based methods use a 500–500–2000 backbone with embedding dimension 10 except 20 for CIFAR100-20, while VanillaDCC and VolMaxDCC use their native 512–512 classifier architectures. For baselines, we use reported best settings where available; VolMaxDCC follows its original validation-based search with 1,000 training samples held out. Except in sensitivity/ablation studies, ECI-PP uses one global setting without dataset-specific tuning: 5 estimator folds, Corrector 64–16, λcor = 0.5, γ = 10, n0 = 10, ξ = 20, and λrec = 0.02. Full experimental details are provided in Appendix C to support reproducibility. 5.2

Experimental results

Main comparison. Table 1 compares our ECI-PP against baselines under the lv0.01 single-expert and multi3 multi-expert settings, both with corruption probability 0.3 and 9k constraints; additional constraint-budget results appear in Appendix D.1. Overall, ECI-PP is strongest, ranking first in 41 out of 48 test entries (2 expert regimes×8 datasets×3 metrics) and within the top two in 46/48, while one of the ProbPair-family methods is best in 45/48, supporting the ProbPair formulation under UPCC. Under single-expert supervision, ECI-PP outperforms the strong non-ProbPair baseline, SpherePair, in most entries (4–10% absolute NMI margins); further comparisons without pretraining make this pattern clearer (Appendix D.2). Under multi-expert supervision, the NMI margin over SpherePair widens to 6–32%. Appendix D.3 further compares baselines with an expert-aware ensemble wrapper. As an ablation signal, Weighted ProbPair shows a clearer advantage over ProbPair in the multiexpert setting, suggesting that information-based weighting is more useful when low-decisiveness constraints can indicate expert unfamiliarity. A remaining caveat is the severely imbalanced RCV1-10: SpherePair retains the best ACC, reflecting the advantage of its angular geometry for imbalanced pair structures, while ECI-PP still gives the best NMI/ARI, indicating stronger grouping consistency. 7

Table 1: Comparative performance (%) (ACC, NMI, ARI) across datasets for methods under lv0.01 and multi3 settings, with corruption probability 0.3 and 9k constraints. Blue and black represent training and test results, respectively. Best results are in bold, and second-best are underlined.

ACC NMI ARI ACC NMI ARI ACC NMI ARI ACC NMI ARI ACC NMI ARI ACC NMI ARI ACC NMI ARI ACC NMI ARI ACC NMI ARI ACC NMI ARI ACC NMI ARI ACC NMI ARI ACC NMI ARI ACC NMI ARI ACC NMI ARI ACC NMI ARI

CIFAR100 CIFAR10 FMNIST Singleexpert lv0.01

ImageNet10 MNIST REUTERS STL10 RCV1-10 CIFAR100 CIFAR10 FMNIST

Multiexpert multi3

ImageNet10 MNIST REUTERS STL10

(B) Varying corruption rate

(A) Varying single-expert quality

RCV1-10

Test NMI - CIFAR100-20

Test NMI - CIFAR10

VolMaxDCC 46.1, 45.9 45.6, 46.1 30.4, 30.5 82.5, 82.5 71.9, 72.2 68.3, 68.3 66.5, 65.5 57.2, 56.8 46.4, 45.2 87.4, 88.1 77.8, 79.2 75.2, 76.1 75.6, 76.9 59.3, 61.6 56.5, 58.7 71.7, 76.8 34.3, 43.4 40.9, 50.4 79.0, 80.1 66.7, 69.3 62.8, 64.7 61.3, 61.8 35.7, 36.7 44.7, 45.4 42.8, 42.5 43.4, 43.8 27.1, 27.1 76.5, 76.6 73.1, 73.2 65.8, 65.9 59.4, 59.1 59.2, 59.0 45.0, 44.3 89.9, 90.5 87.8, 88.7 83.9, 84.9 67.1, 67.8 57.5, 58.9 50.9, 52.1 67.5, 68.0 45.4, 46.5 47.0, 47.9 71.0, 71.1 70.2, 69.9 59.5, 59.1 48.3, 49.0 31.4, 32.1 30.2, 30.9

Test NMI - FMNIST

CIDEC

SpherePair

ProbPair

14.3, 13.6 11.0, 11.2 2.8, 2.3 44.3, 44.0 36.1, 37.6 21.1, 21.3 36.1, 35.7 29.7, 30.5 15.7, 15.6 63.5, 68.3 47.2, 57.5 35.5, 42.2 47.0, 46.7 41.4, 43.7 24.7, 24.2 72.9, 79.3 35.7, 48.1 41.8, 54.9 57.8, 63.3 39.3, 48.5 29.8, 37.3 42.5, 44.3 29.5, 30.0 26.2, 28.3 12.8, 12.1 10.0, 10.5 2.1, 2.0 33.2, 31.6 28.7, 29.1 12.3, 12.1 28.6, 28.8 28.3, 30.6 14.5, 15.4 39.4, 38.8 31.0, 34.8 14.3, 15.6 31.8, 31.0 28.8, 30.2 13.2, 13.4 57.3, 59.1 18.3, 25.7 20.4, 23.1 31.4, 31.6 24.9, 31.7 13.6, 16.9 32.3, 32.0 14.1, 13.5 10.8, 10.3

50.2, 50.5 43.0, 44.2 32.1, 32.6 84.9, 85.7 72.4, 74.1 71.2, 72.5 72.0, 71.2 60.4, 60.9 53.0, 52.3 85.1, 89.1 71.0, 79.2 70.7, 78.2 87.8, 89.1 75.4, 78.0 75.4, 77.8 77.9, 81.8 48.7, 56.3 52.9, 61.1 76.1, 79.7 59.5, 66.5 57.2, 63.3 71.2, 71.4 60.5, 61.3 61.0, 61.4 44.3, 44.4 40.5, 41.5 27.1, 27.6 73.8, 74.5 64.7, 66.5 59.0, 60.5 61.0, 61.3 56.6, 57.8 46.8, 47.4 80.7, 85.0 65.6, 73.8 63.4, 71.0 78.6, 80.0 70.8, 74.0 66.3, 69.1 62.1, 69.9 25.6, 37.2 28.0, 40.7 67.8, 71.4 53.2, 59.7 47.8, 53.3 55.7, 55.9 51.9, 52.7 43.6, 44.2

49.4, 49.5 43.2, 44.2 30.6, 30.9 85.9, 86.0 75.0, 75.4 73.0, 73.2 73.6, 72.5 62.9, 62.9 54.8, 53.7 89.2, 90.5 80.1, 83.2 78.5, 80.7 89.0, 90.0 76.9, 79.0 77.4, 79.4 78.2, 80.8 53.2, 58.3 55.5, 61.0 81.6, 83.4 68.2, 72.1 65.8, 68.9 52.7, 53.0 53.8, 54.1 43.6, 43.7 43.3, 43.4 40.0, 40.9 26.0, 26.2 81.3, 81.7 71.5, 72.3 67.1, 67.8 66.6, 66.7 60.7, 61.2 49.9, 50.0 87.4, 89.2 76.1, 80.1 75.0, 78.3 80.5, 81.8 72.1, 74.7 68.7, 71.2 74.5, 79.0 44.6, 52.8 50.2, 58.6 81.1, 83.0 66.1, 70.2 63.7, 67.1 43.4, 43.5 46.6, 47.4 34.4, 34.7

Test NMI - IMAGENET10

0.8 0.4

0.6

Test NMI - MNIST

0.2 lv0.001

lv0.01

lv0.1

0.8

0.5

0.6

0.6

0.4

0.4

0.4

0.3 lv0.001

lv0.01

lv0.1

lv0.01

lv0.1

0.6

0.2

0.2 0.3

0.5

lv0.1

Test NMI - STL10

Test NMI - RCV1-10

0.8

lv0.01

lv0.1

0.6

0.6

0.4

0.4

0.2

0.2

0.0

0.4

lv0.001

lv0.01

lv0.1

0.2

lv0.001

lv0.01

lv0.1

0.1

0.3

0.5

0.1

0.3

0.5

0.0

lv0.001

lv0.01

lv0.1

0.2

0.0

0.0

0.1

0.3

0.5

0.4 0.6

0.6

0.1

0.3

0.5

0.5

0.6

0.4

0.4

multi2

multi3

multi10

0.6

0.6 0.4

0.4

0.2

0.8

0.6

0.6

0.4

0.8

0.2

lv0.001 0.8

0.6 0.4

0.4

0.1

lv0.01

0.8

0.6

0.2

lv0.001

ECI-PP (Ours) 52.7, 52.9 49.2, 49.9 36.2, 36.5 87.8, 87.8 79.0, 79.1 76.7, 76.8 72.4, 71.1 65.7, 65.0 55.5, 53.9 92.8, 93.1 88.1, 88.5 85.8, 86.2 90.7, 91.0 80.7, 81.6 80.8, 81.5 85.1, 86.6 61.4, 64.4 66.6, 70.0 85.4, 85.8 75.8, 76.6 72.8, 73.4 69.0, 68.9 65.5, 65.8 62.0, 62.0 48.6, 48.8 48.1, 48.5 32.9, 32.9 81.6, 82.1 77.8, 77.8 73.1, 73.0 65.3, 64.8 66.0, 65.6 53.4, 52.8 92.4, 92.6 87.7, 88.1 85.4, 85.7 84.7, 85.1 79.9, 80.7 77.1, 77.5 88.3, 89.6 65.4, 68.7 72.9, 76.0 87.9, 87.7 78.3, 78.5 75.9, 75.8 53.4, 53.3 58.9, 59.0 46.2, 46.1

0.8

0.8 0.4

Test NMI - REUTERS

Weighted ProbPair 50.3, 50.8 44.5, 45.7 31.5, 32.1 86.6, 86.7 76.2, 76.8 74.3, 74.5 74.6, 73.6 63.9, 64.0 55.3, 54.3 89.1, 90.3 80.3, 82.7 78.4, 80.2 89.9, 90.8 78.3, 80.2 79.1, 80.9 80.4, 82.7 54.5, 59.2 58.6, 63.4 81.6, 83.1 67.9, 71.6 65.8, 68.4 55.2, 55.1 57.4, 57.5 47.1, 47.0 46.1, 46.5 41.8, 43.0 28.0, 28.5 81.4, 81.7 72.0, 72.6 67.9, 68.4 68.0, 67.6 61.7, 62.0 51.9, 51.8 90.1, 91.3 80.1, 83.0 79.8, 82.0 85.1, 86.1 75.0, 77.2 73.4, 75.5 81.0, 83.9 51.5, 57.9 57.9, 64.2 83.1, 84.3 69.4, 72.0 67.4, 69.4 47.8, 47.9 51.6, 52.0 38.2, 38.4

0.6

0.2 lv0.001

0.8

0.8

0.6

0.4

0.2

0.0

(C) Varying multi-expert config

VanillaDCC 16.8, 17.0 13.5, 14.6 6.0, 6.3 38.3, 38.7 29.9, 31.2 23.2, 24.0 39.8, 40.2 30.1, 31.3 21.5, 22.2 39.0, 41.5 27.6, 33.7 21.1, 25.2 38.9, 40.3 29.5, 32.2 23.1, 24.8 69.7, 77.0 30.5, 43.3 36.8, 50.4 35.5, 37.8 22.9, 27.6 17.1, 19.8 46.2, 46.4 15.5, 15.9 19.8, 20.1 15.4, 15.8 12.8, 13.6 5.8, 5.8 32.4, 32.8 29.1, 30.9 21.2, 22.4 35.2, 35.6 31.4, 32.8 20.4, 21.3 31.8, 34.2 22.0, 26.8 15.8, 18.7 32.2, 33.3 26.5, 29.3 18.4, 20.1 55.3, 59.2 14.7, 22.8 19.2, 26.2 29.2, 30.7 21.5, 26.0 15.2, 17.7 44.3, 44.6 14.3, 14.7 17.8, 18.0

0.2 0.1

0.3

0.5

0.1

0.3

0.4

0.4

0.2

0.2

0.5

0.1

0.3

0.5

0.8

0.8

0.0 0.6

0.6 0.6

0.6

0.4

0.4

0.4

0.4

0.4

0.2

0.2

0.3 0.0

multi2

multi3

multi10

0.2

multi2

multi3

multi10

VanillaDCC

multi2

multi3

VolMaxDCC

multi10

0.2

multi2

CIDEC

multi3

multi10

0.2

multi2

SpherePair

multi3

multi10

ProbPair

0.0

multi2

multi3

multi10

Weighted ProbPair

0.2

multi2

multi3

multi10

0.0

ECI-PP (Ours)

Figure 2: Test NMI performance (mean±std over 5 runs) of all models across datasets under varying (A) expert quality, (B) corruption probability, and (C) multi-expert configuration, with 9k constraints. Robustness across supervision conditions. We further stress-test all methods by varying one supervision factor in UPCC at a time. Around the 9k default supervision in Table 1, Fig. 2 reports test NMI as we vary (A) expert quality, (B) corruption probability, and (C) multi-expert configuration; ACC/ARI results are deferred to Appendix D.4. Across the three sweeps, the trends largely follow the expected supervision-quality ordering: stronger experts, lower corruption, and multi-expert settings with reduced expert blind spots (as detailed in Appendix C.3) generally yield better performance, 8

while the gap between ECI-PP and strong ProbPair-family baselines often narrows in easier regimes. The stability advantage of ECI-PP is broadly visible across datasets, particularly on FMNIST, ImageNet10, and STL10, and is most pronounced under increasing corruption, where most baselines degrade sharply while ECI-PP remains comparatively stable. Overall, these results are consistent with the role of our ECI framework in improving robustness to imperfect and heterogeneous supervision. 



(A) Brier ↓ (multi3) (B) Brier ↓ (multi3) (C) Brier ↓ (multi3) (D) E gap ↓ (multi3) Held-out diagnostics. We ex0.50 0.35 0.40 amine whether ECI-PP’s inter- 0.40 0.30 0.15 0.30 0.30 0.25 nal signals evolve as intended 0.20 0.20 0.10 0.20 0.10 on sample-disjoint held-out con- 0.10 0 100 200 300 400 500 iter. 0 100 200 300 400 500 iter. 0 100 200 300 400 500 iter. 0 100 200 300 400 500 iter. straints. Fig. 3 shows the (E) AUC ↑ (multi3) (F) AP ↑ (multi3) (G) AUC ↑ (oracle) (H) AP ↑ (oracle) MNIST case under multi3 0.70 0.60 0.70 0.50 0.50 with corruption rate 0.3, using 0.60 0.60 0.40 0.40 benchmark labels only to form 0.50 0.30 0.30 0.50 an external hard co-membership 0 100 200 300 400 500 iter. 0 100 200 300 400 500 iter. 0 100 200 300 400 500 iter. 0 100 200 300 400 500 iter. reference for diagnostics, rather than the canonical aleatoric re- Figure 3: Held-out diagnostic evolution over 500 training iteralation R⋆ . Panels (A–C) report tions on MNIST under multi3 with corruption rate 0.3 (mean±std Brier scores of the estimated, over 5 runs). (A–C) Brier scores for estimated, corrected, and corrected, and integrated rela- integrated relations; (D) residual discrepancy between corrected tions, and panel (D) reports the relations and estimator beliefs on uncorrupted constraints; (E,F) residual discrepancy between AUC/AP measuring the alignment between reliability-aware screencorrected relations and estima- ing and injected corruption; (G,H) the same screening diagnostic tor beliefs on uncorrupted con- with oracle-generated pairwise supervision before corruption. straints; lower is better. Panels (E,F) compare reliability-aware screening with the injected corruption indicator using AUC/AP, where higher values indicate better corruption separation. (G,H) repeat the same screening diagnostic with oracle-generated clean supervision before corruption, removing the expert epistemic uncertainty present in (E,F). The curves improve rapidly and then stabilize: relation Brier scores decrease, the post-correction residual discrepancy drops, and the screening signal aligns with corruption above chance, while the oracle-setting curves in (G,H) more closely track corruption as expected. Overall, these held-out trends are consistent with the intended behavior of the ECI framework, from internal relation estimates to reliability-aware screening; full cross-dataset diagnostics are in Appendix D.5. Brierint

APcorrupt baseline = 0.3

AUCcorrupt baseline = 0.5

corrupt

AUCcorrupt baseline = 0.5

i,clean

corrupt

APcorrupt

APcorrupt

corrupt

AUCcorrupt

Brierest

AUCcorrupt

corrupt

int

  E gapi,clean

cor,clean

Briercor,clean

est

APcorrupt baseline = 0.3

Sensitivity and ablation. We vary the main ECI-PP design choices around the default configuration in Section 5.1 to assess sensitivity and ablate key components. The results (see Appendix D.6) show that the default setting is robust over broad ranges: larger K gives only mild gains relative to its cross-fitting cost, information-based warm-up is mildly helpful, small Correctors with moderate regularization are sufficient, and the screening and boundary parameters (γ, n0 , ξ) are generally insensitive. Reconstruction strength allows tuning flexibility, but the shared default remains reliable overall. These results indicate that ECI-PP is robust without dataset-specific hyperparameter tuning.

6

Conclusion

This paper addressed a realistic probabilistic constrained clustering setting, where pairwise supervision entangles aleatoric, epistemic, and stochastic factors. We formalized the constraint-observation process and its conditional identifiability, introduced ProbPair for angular learning from probabilistic relations, and built ECI-PP as a practical framework for such supervision. While existing methods largely rely on idealized constraints, ECI-PP outperforms state-of-the-art alternatives and remains robust under fallible, heterogeneous, and corrupted supervision without dataset-specific tuning. These results come with several qualifications. The identifiability analysis is population-level and conditional; fully factor-specific finite-sample estimation would require stronger assumptions or additional observations beyond the current surrogate residual structure. Accordingly, the Corrector and reliability signal are practical surrogates rather than estimators of individual latent factors; a more theory-aligned extension would model structured correction, residual epistemic variation, and corruption-aware screening separately. Empirically, our experiments use controlled expert simulations, leaving real-world annotator-provided constraints as a natural next step. Finally, richer Corrector pair descriptors and principled stopping criteria may further improve robustness and training adaptivity. 9

References [1] Yucen Luo, Tian Tian, Jiaxin Shi, Jun Zhu, and Bo Zhang. Semi-crowdsourced clustering with deep generative models. Advances in Neural Information Processing Systems, 31, 2018. 1, 2 [2] Laura Manduchi, Kieran Chin-Cheong, Holger Michel, Sven Wellmann, and Julia Vogt. Deep conditional gaussian mixture model for constrained clustering. Advances in Neural Information Processing Systems, 34:11303–11314, 2021. 1, 2, 23 [3] Shaojie Zhang and Ke Chen. Angular constraint embedding via spherepair loss for constrained clustering. In Advances in Neural Information Processing Systems 39, 2025. 1, 2, 5, 7, 17, 18, 19, 23, 24, 25, 38 [4] Ian Davidson and Sugato Basu. A survey of clustering with instance level constraints. ACM Transactions on Knowledge Discovery from data, 1(1-41):2–42, 2007. 1 [5] Germán González-Almagro, Daniel Peralta, Eli De Poorter, José-Ramón Cano, and Salvador García. Semi-supervised constrained clustering: an in-depth overview, ranked taxonomy and future research directions. Artificial Intelligence Review, 58:157, 2025. 1 [6] Kiri Wagstaff and Claire Cardie. Clustering with instance-level constraints. AAAI/IAAI, 1097(577-584):197, 2000. 1 [7] Kiri Wagstaff, Claire Cardie, Seth Rogers, Stefan Schrödl, et al. Constrained k-means clustering with background knowledge. In ICML, volume 1, pages 577–584, 2001. 1 [8] Zhengdong Lu and Todd Leen. Semi-supervised learning with penalized probabilistic clustering. Advances in Neural Information Processing Systems, 17, 2004. 1 [9] Mikhail Bilenko, Sugato Basu, and Raymond J Mooney. Integrating constraints and metric learning in semi-supervised clustering. In Proceedings of the twenty-first International Conference on Machine Learning, page 11, 2004. 1 [10] Sugato Basu, Arindam Banerjee, and Raymond J Mooney. Active semi-supervision for pairwise constrained clustering. In Proceedings of the 2004 SIAM International Conference on Data Mining, pages 333–344. SIAM, 2004. 1 [11] Brian Kulis, Sugato Basu, Inderjit Dhillon, and Raymond Mooney. Semi-supervised graph clustering: a kernel approach. In Proceedings of the 22nd International Conference on Machine Learning, pages 457–464, 2005. 1 [12] Yen-Chang Hsu, Zhaoyang Lv, Joel Schlosser, Phillip Odom, and Zsolt Kira. A probabilistic constrained clustering for transfer learning and image category discovery. In Proceedings of the CVPR 2018 Deep-Vision Workshop, 2018. 1, 2 [13] Yazhou Ren, Kangrong Hu, Xinyi Dai, Lili Pan, Steven CH Hoi, and Zenglin Xu. Semisupervised deep embedded clustering. Neurocomputing, 325:121–130, 2019. 1, 18, 23, 24 [14] Hongjing Zhang, Tianyang Zhan, Sugato Basu, and Ian Davidson. A framework for deep constrained clustering. Data Mining and Knowledge Discovery, 35:593–620, 2021. 1, 2, 7, 18, 23, 24 [15] Tri Nguyen, Shahana Ibrahim, and Xiao Fu. Deep clustering with incomplete noisy pairwise annotations: A geometric regularization approach. In International Conference on Machine Learning, pages 25980–26007. PMLR, 2023. 1, 2, 7, 18, 23, 24 [16] Sharon Fogel, Hadar Averbuch-Elor, Daniel Cohen-Or, and Jacob Goldberger. Clusteringdriven deep embedding with pairwise constraints. IEEE computer graphics and applications, 39(4):16–27, 2019. 1, 2 [17] Abu Quwsar Ohi, Muhammad Firoz Mridha, Farisa Benta Safir, Md Abdul Hamid, and Muhammad Mostafa Monowar. Autoembedder: A semi-supervised dnn embedding system for clustering. Knowledge-Based Systems, 204:106190, 2020. 1, 2 10

[18] Eyke Hüllermeier and Willem Waegeman. Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine Learning, 110(3):457–506, 2021. 1 [19] Yen-Chang Hsu, Zhaoyang Lv, Joel Schlosser, Phillip Odom, and Zsolt Kira. Multi-class classification without multi-class labels. In International Conference on Learning Representations, 2019. 2, 7, 18 [20] Marek Śmieja, Łukasz Struski, and Mário AT Figueiredo. A classification-based approach to semi-supervised clustering with pairwise constraints. Neural Networks, 127:193–203, 2020. 2 [21] Tian Tian, Jie Zhang, Xiang Lin, Zhi Wei, and Hakon Hakonarson. Model-based deep embedding for constrained clustering analysis of single cell rna-seq data. Nature communications, 12(1):1873, 2021. 2 [22] Jann Goschenhofer, Bernd Bischl, and Zsolt Kira. Constraintmatch for semi-constrained clustering. In 2023 International Joint Conference on Neural Networks (IJCNN), pages 1–10. IEEE, 2023. 2 [23] Yi Wen, Suyuan Liu, Xinhang Wan, Siwei Wang, Ke Liang, Xinwang Liu, Xihong Yang, and Pei Zhang. Efficient multi-view graph clustering with local and global structure preservation. In Proceedings of the 31st ACM International Conference on Multimedia, pages 3021–3030, 2023. 2 [24] Jian Dai, Zhenwen Ren, Yunzhi Luo, Hong Song, and Jian Yang. Tensorized anchor graph learning for large-scale multi-view clustering. Cognitive Computation, 15(5):1581–1592, 2023. 2 [25] Suyuan Liu, Qing Liao, Siwei Wang, Xinwang Liu, and En Zhu. Robust and consistent anchor graph learning for multi-view clustering. IEEE Transactions on Knowledge and Data Engineering, 2024. 2 [26] Martin HC Law, Alexander Topchy, and Anil K Jain. Model-based clustering with probabilistic constraints. In Proceedings of the 2005 SIAM International Conference on Data Mining, pages 641–645. SIAM, 2005. 2 [27] Martin HC Law, Alexander Topchy, and Anil K Jain. Clustering with soft and group constraints. In Joint IAPR International Workshops on Statistical Techniques in Pattern Recognition (SPR) and Structural and Syntactic Pattern Recognition (SSPR), pages 662–670. Springer, 2004. 2 [28] Irene Diaz-Valenzuela, M Amparo Vila, and Maria J Martin-Bautista. On the use of fuzzy constraints in semisupervised clustering. IEEE Transactions on Fuzzy Systems, 24(4):992–999, 2015. 2 [29] Zhen Wang, Shan-Shan Wang, Lan Bai, Wen-Si Wang, and Yuan-Hai Shao. Semisupervised fuzzy clustering with fuzzy pairwise constraints. IEEE Transactions on Fuzzy Systems, 30(9):3797–3811, 2021. 2 [30] Jian-Ping Mei, Huajiang Lv, Jiuwen Cao, and Weihua Gong. Pairwise constrained fuzzy clustering: Relation, comparison and parallelization. International Journal of Fuzzy Systems, 21(6):1938–1949, 2019. 2 [31] Jian-Ping Mei and Huajiang Lv. Semi-supervised fuzzy c-means regularized with pairwise constraints. In 2018 14th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), pages 781–786. IEEE, 2018. 2 [32] Violaine Antoine, Benjamin Quost, M-H Masson, and Thierry Denoeux. Cevclus: evidential clustering with instance-level constraints for relational data. Soft Computing, 18(7):1321–1335, 2014. 2 [33] Feng Li, Shoumei Li, and Thierry Denœux. k-cevclus: Constrained evidential clustering of large dissimilarity data. Knowledge-Based Systems, 142:29–44, 2018. 2 11

[34] Dan Pelleg and Dorit Baras. K-means with large and noisy constraint sets. In European Conference on Machine Learning, pages 674–682. Springer, 2007. 2 [35] Thiago F Covoes, Eduardo R Hruschka, and Joydeep Ghosh. A study of k-means-based algorithms for constrained clustering. Intelligent Data Analysis, 17(3):485–505, 2013. 2 [36] Xiatian Zhu, Chen Change Loy, and Shaogang Gong. Constrained clustering with imperfect oracles. IEEE transactions on neural networks and learning systems, 27(6):1345–1357, 2015. 2 [37] Hongfu Liu, Zhiqiang Tao, and Yun Fu. Partition level constrained clustering. IEEE transactions on pattern analysis and machine intelligence, 40(10):2469–2483, 2017. 2 [38] Yale Chang, Junxiang Chen, Michael H Cho, Peter J Castaldi, Edwin K Silverman, and Jennifer G Dy. Multiple clustering views from multiple uncertain experts. In International Conference on Machine Learning, pages 674–683. PMLR, 2017. 2 [39] Hongjing Zhang, Sugato Basu, and Ian Davidson. A framework for deep constrained clusteringalgorithms and advances. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2019, Würzburg, Germany, September 16–20, 2019, Proceedings, Part I, pages 57–72. Springer, 2020. 2 [40] Alexander Philip Dawid and Allan M Skene. Maximum likelihood estimation of observer error-rates using the em algorithm. Journal of the Royal Statistical Society: Series C (Applied Statistics), 28(1):20–28, 1979. 2 [41] Filipe Rodrigues and Francisco Pereira. Deep learning from crowds. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018. 2 [42] Jacob Whitehill, Ting-fan Wu, Jacob Bergsma, Javier Movellan, and Paul Ruvolo. Whose vote should count more: Optimal integration of labels from labelers of unknown expertise. Advances in Neural Information Processing Systems, 22, 2009. 2 [43] Peter Welinder, Steve Branson, Pietro Perona, and Serge Belongie. The multidimensional wisdom of crowds. Advances in Neural Information Processing Systems, 23, 2010. 2 [44] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. 7, 17 [45] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017. 7, 17 [46] Jianlong Chang, Lingfeng Wang, Gaofeng Meng, Shiming Xiang, and Chunhong Pan. Deep adaptive image clustering. In Proceedings of the IEEE international conference on computer vision, pages 5879–5887, 2017. 7, 17 [47] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998. 7, 17 [48] Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 215–223. JMLR Workshop and Conference Proceedings, 2011. 7, 17 [49] Junyuan Xie, Ross Girshick, and Ali Farhadi. Unsupervised deep embedding for clustering analysis. In International Conference on Machine Learning, pages 478–487. PMLR, 2016. 7, 17, 18, 24 [50] Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters. Econometrics Journal, 21:C1–C68, 2018. 15 [51] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A largescale hierarchical image database. In 2009 IEEE conference on Computer Vision and Pattern Recognition, pages 248–255. IEEE, 2009. 17 12

[52] Xifeng Guo, Long Gao, Xinwang Liu, and Jianping Yin. Improved deep embedded clustering with local structure preservation. In IJCAI, volume 17, pages 1753–1759, 2017. 18, 24 [53] Yunfan Li, Peng Hu, Zitao Liu, Dezhong Peng, Joey Tianyi Zhou, and Xi Peng. Contrastive clustering. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 8547–8555, 2021. 23 [54] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 23 [55] Geoffrey E Hinton and Ruslan R Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313(5786):504–507, 2006. 23 [56] Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th International Conference on Machine learning, pages 1096–1103. ACM, 2008. 23

A

Additional theoretical details

A.1

Derivation of the logistic-normal judgment density

Starting from Equation (1), define ⋆ µe,ab := logitε (Rab ) + me (ϕab ). ⋆ Conditionally on (Rab , e, ϕab ), we then have jud
logit ye,ab = µe,ab + ue,ab ,

so that

ue,ab ∼ N 0, τe2 (ϕab ) ,

jud
logit ye,ab ∼ N µe,ab , τe2 (ϕab ) .

For a generic y ∈ (0, 1), write v = logit(y). By the change-of-variables formula,   dv ⋆ pjud (y | Rab , e, ϕab ) = φ v; µe,ab , τe2 (ϕab ) , v = logit(y), y ∈ (0, 1). dy dv 1 Using dy = d logit(y) = y(1−y) , we obtain dy ⋆ pjud (y | Rab , e, ϕab ) =

  1 ⋆ φ logit(y); logitε (Rab ) + me (ϕab ), τe2 (ϕab ) , y(1 − y)

y ∈ (0, 1),

which is exactly Equation (2). A.2

Proof of Lemma 3.2

Proof. Fix an expert e with nonzero observation probability. By assumption, mηe (ϕab ) = mηe′ (ϕab ) − ke for all observable triples (a, b, e). Taking expectation over the observable-pair distribution conditional on e yields E(a,b)|e [mηe (ϕab )] = E(a,b)|e [mηe′ (ϕab )] − ke . Both expectations vanish by Assumption 3.1, hence ke = 0.1 Since e was arbitrary over experts with nonzero observation probability, the conclusion holds for every such expert.

1 The centering is imposed conditionally on each expert because the remaining additive ambiguity after fixing the total location is expert-wise: global centering would only fix the average shift across experts, not each expert-specific shift. Thus, under the present assumptions, global centering would not be sufficient for identifiability unless one further assumes that different experts observe sufficiently overlapping pair regions.

13

A.3

Proof of Lemma 3.3

Proof. By Equations (2) and (4), the observation law takes the form Equation (5). Assume that

p(· | µ, s, π) = p(· | µ′ , s′ , π ′ ) on (0, 1), where p(y | µ, s, π) is the density defined in Equation (5). Since 1[0,1] (y) = 1 for y ∈ (0, 1), equality of the two densities means that (1 − π)

1 1 2
φ logit(y); µ, s2 + π = (1 − π ′ ) φ logit(y); µ′ , s′ + π ′ y(1 − y) y(1 − y)

for all y ∈ (0, 1).

Write v = logit(y), so that y = σ(v) and y(1 − y) = σ(v)(1 − σ(v)). Multiplying both sides by y(1 − y) gives 2

(1 − π) φ(v; µ, s2 ) + π σ(v)(1 − σ(v)) = (1 − π ′ ) φ(v; µ′ , s′ ) + π ′ σ(v)(1 − σ(v))

for all v ∈ R.

As v → +∞, the Gaussian terms decay faster than e−v , whereas σ(v)(1 − σ(v)) = Therefore,

e−v ∼ e−v . (1 + e−v )2

h i lim ev (1 − π) φ(v; µ, s2 ) + π σ(v)(1 − σ(v)) = π,

v→+∞

and similarly the right-hand side tends to π ′ . Hence π = π ′ . Subtracting the common contamination term then yields 2

φ(v; µ, s2 ) = φ(v; µ′ , s′ )

for all v ∈ R.

Injectivity of the Gaussian family implies µ = µ and s = s′ . Therefore (µ, s, π) = (µ′ , s′ , π ′ ), proving the claim. ′

A.4

Boundary cases for structural separability

Assumption 3.4 concerns the decomposition of the identifiable location term logitε (rθ (xa , xb )) + mηe (ϕab ) into a canonical relation component and an expert-specific epistemic component. Two boundary cases clarify this requirement. (i) If the pair descriptor ϕab is uninformative, for example constant over observable pairs for a given expert, then mηe (ϕab ) reduces to an expert-specific constant and cannot capture pair-dependent epistemic effects; the intended epistemic component is then not represented by the chosen model class. (ii) Conversely, if ϕab is overly rich and the function class for mηe (ϕab ) overlaps with that of logitε (rθ (xa , xb )), then shared pair-dependent variation can be moved between the two terms while preserving their sum. For example, a sufficiently small zero-mean shift lying in this overlap can be added to the canonical location term and subtracted from the expert-specific term without violating centering or changing the observation law. The condition therefore rules out such interchangeability, ensuring that canonical aleatoric variation and structured expert-specific variation play distinct explanatory roles on the observable support. A.5

Proof of Theorem 3.5

Proof. Assume that the induced conditional laws coincide under Ξ and Ξ′ for every observable triple (a, b, e). For each such triple, define
µΞ (a, b, e) := logitε rθ (xa , xb ) + mηe (ϕab ), sΞ (a, b, e) := τζe (ϕab ),

and define µΞ′ (a, b, e) and sΞ′ (a, b, e) analogously. By Equations (4) and (5), the conditional law of the observed response for (a, b, e) is exactly
p y | µΞ (a, b, e), sΞ (a, b, e), πe , 14

and similarly under Ξ′ . Since the two conditional laws coincide, Lemma 3.3 implies that for every observable triple (a, b, e), µΞ (a, b, e) = µΞ′ (a, b, e), In particular,

sΞ (a, b, e) = sΞ′ (a, b, e),

πe = πe′ .

logitε rθ (xa , xb ) + mηe (ϕab ) = logitε rθ′ (xa , xb ) + mηe′ (ϕab )

for all observable triples (a, b, e).

Applying Assumption 3.4, for each expert e there exists a constant ke ∈ R such that for all observable triples (a, b, e),

logitε rθ (xa , xb ) = logitε rθ′ (xa , xb ) + ke , mηe (ϕab ) = mηe′ (ϕab ) − ke . By Lemma 3.2, each ke on the observable expert support must be zero. Therefore,

logitε rθ (xa , xb ) = logitε rθ′ (xa , xb ) for all (a, b) ∈ Pobs .

Since rθ (xa , xb ), rθ′ (xa , xb ) ∈ [ε, 1 − ε] on Pobs , the clamped logit satisfies logitε (u) = logit(u) on this interval and is therefore injective there. It follows that rθ (xa , xb ) = rθ′ (xa , xb ) which proves the theorem.

B

Algorithm and complexity analysis

B.1

Algorithm

for all (a, b) ∈ Pobs ,

We provide the detailed ECI-PP procedure introduced in Section 4, summarized in Algorithm 1. B.2

Computational complexity analysis

For an Estimator/Integrator backbone f, g, m, T , let cf and cg denote the per-instance training costs of the encoder f and decoder g, respectively, and let cm,T = O(1) denote the cost of updating the scalar readout parameters (m, T ). We write cpair = O(cf + cm,T ) for the amortized per-constraint cost of the ProbPair-style pairwise term, and crec = O(cf + cg ) for the per-instance reconstruction cost. Thus, one backbone epoch over constraints C and instances X costs O(|C|cpair + |X |crec ). In particular, the pairwise term scales with the number of constraints, while the reconstruction term scales linearly with |X | as the usual autoencoder regularization cost. Notably, each Corrector Core = qνe is a small expert-specific MLP, rather than an encoder–decoder backbone, and its capacity is intentionally kept more limited than that of the Estimator/Integrator networks so that it learns only shared structured expert-specific patterns rather than arbitrary fitting. We write ccor for its amortized per-constraint update cost. Under this notation, with Nwu warm-up epochs
and Nep total epochs, the warm-up stage costs
O Nwu (K + 1) |C|cpair + |X |crec + Jcor |C|ccor , where Jcor is the average number of optimization passes used when fitting the Correctors.
Each of the Tref
:= Nep − Nwu subsequent refinement rounds costs O (K + 1) |C|cpair + |X |crec + Jcor |C|ccor . The total training complexity is



O Nwu + Tref (K + 1) |C|cpair + |X |crec + 1 + Tref Jcor |C|ccor .

The dominant overhead comes from the K-fold cross-fitting
of the Estimators, namely the repeated backbone updates scaled by (K + 1) |C|cpair + |X |crec . This overhead is justified by the need for out-of-fold beliefs: the Estimators are introduced not as a redundant ensemble, but to provide modelside references decoupled from the specific constraints they are later used to assess, thereby reducing self-confirmation and overfitting to noisy supervision. This use of held-out predictions is analogous in spirit to debiased/double machine learning [50]: both reduce in-sample contamination, although ECI-PP uses them to obtain cleaner belief proxies under noisy supervision rather than to pursue semiparametric efficiency. By contrast, the expert-specific correction stage does not scale as O(E|C|), PE because each constraint is routed only to its corresponding expert module and e=1 |Ie | = |C|. Thus, 15

Algorithm 1 ECI-PP for learning under UPCC |X |

|C|

Require: training data X = {xj }j=1 ; constraints C = {(ai , bi , ei , yi )}i=1 ; number of experts E; number of estimator folds K; total training epochs Nep ; warm-up epochs Nwu ; hyperparameters λrec (reconstruction weight), λcor (Corrector regularization), ξ (soft-clipping sharpness), γ (screening sharpness), n0 (Bayesian-confidence scale); clustering routine Clustering(·) 1: Partition C into K disjoint folds {C (k) }K k=1 and define Ie := {i : ei = e} for each expert e (k) E ′ 2: Initialize {Est = (fωk , gωk , mk , Tk )}K k=1 , {Core = qνe }e=1 , and Int = (fω , gω ′ , m, T ) 3: Stage I: Warm-up

|C|

4: Compute decisiveness weights {κi }i=1 by Eq. (9) 5: for n = 1, . . . , Nwu do 6: for k = 1, . . . , K do (k)

7: Update Est on C \C (k) by gradient descent on Eq. (8) with yi and weights κi 8: end for 9: end for (k) oof |C| 10: Aggregate holdout predictions from {Est }K k=1 to obtain {ŷi }i=1 via Eq. (6) 11: for e = 1, . . . , E do 12: Update Core on Ie by gradient descent on Eq. (10) until convergence 13: end for |C| 14: Compute {yicor , gapi , wi }i=1 by Eq. (11) |C|

15: Set ni ← n0 wi and compute Bayesian-confidence supervision {yiBC }i=1 by Eq. (12) 16: for n = 1, . . . , Nwu do 17: Update Int by gradient descent on Eq. (8) with yiBC and weights wi 18: end for 19: Stage II: Iterative refinement 20: for t = Nwu + 1, . . . , Nep do |C|

Read out Integrator-side relations {yiint }i=1 from the Integrator embeddings via Eq. (6) for k = 1, . . . , K do 23: Update Est(k) on C \ C (k) by gradient descent on Eq. (8) with yiint and weights wi 24: end for |C| 25: Aggregate updated holdout predictions to refresh {ŷioof }i=1 26: for e = 1, . . . , E do 27: Update Core on Ie by gradient descent on Eq. (10) until convergence 28: end for |C| 29: Refresh {yicor , gapi , wi }i=1 by Eq. (11) |C| 30: Set ni ← n0 wi and refresh Bayesian-confidence supervision {yiBC }i=1 by Eq. (12) BC 31: Update Int by gradient descent on Eq. (8) with yi and weights wi 32: end for 21: 22:

33: Stage III: Clustering and unseen-instance assignment

|X |

34: Obtain the final normalized Integrator embeddings Z int ← {Norm(fω (xj ))}j=1

b 35: Apply Clustering(Z int ) to obtain the final partition S b 36: Compute cluster centroids {µc } from Z int and S 37: for each unseen instance x̃ do 38: Compute z̃ int ← Norm(fω (x̃)) 39: Assign x̃ to the nearest centroid in {µc } 40: end for

16

the per-expert Correctors add only a linear-in-|C| refinement cost, which is largely unavoidable under heterogeneous expert supervision. More broadly, the practical cost profile of ECI-PP should be interpreted together with its training protocol: it does not require an additional expert-aware wrapper in multi-expert settings (see Appendix C.4), and unlike methods such as VolMaxDCC, its performance is not tightly tied to an extensive hyperparameter-search stage. Meanwhile, Appendix D.2 shows that ECI-PP is less dependent on unsupervised pre-training, which can be advantageous in resource-limited settings, and an overall training-time comparison is reported in Appendix E. A discussion of the choice of K is provided in Appendix D.6.

C

Details of experimental settings

We provide supplementary information for the experimental setup in Section 5.1, including the benchmark datasets, compared methods, constraint-generation procedure, experimental protocol, and implementation details. C.1

Datasets

We use eight benchmark datasets covering image and text clustering tasks, with different numbers of classes and class-balance properties. Their details are as follows. CIFAR100-202 [44]: A 20-class version of CIFAR100 obtained by using the 20 superclasses as clustering targets. The original CIFAR100 dataset contains 60,000 real-world 32 × 32 color images from 100 fine-grained classes, grouped into 20 superclasses. In our experiments, the 20 superclasses are treated as ground-truth clusters, with 3,000 images per superclass. CIFAR103 [44]: A natural-image dataset containing 60,000 32 × 32 color images from 10 object categories, with 6,000 images per category. FMNIST4 [45]: A fashion-product image dataset containing 70,000 grayscale 28 × 28 images from 10 categories. The official split contains 60,000 training images and 10,000 test images. ImageNet105 [46]: A 10-class subset of ImageNet [51], containing 13,000 color images in total, with 1,300 images per class. MNIST6 [47]: A handwritten-digit dataset containing 70,000 grayscale 28 × 28 images from 10 digit classes. The official split contains 60,000 training images and 10,000 test images. Reuters subset7 [49]: A text benchmark derived from the RCV1 corpus8 , using tf–idf features over the 2,000 most frequent words. It contains four root categories, Corporate/Industrial (CCAT), Economics (ECAT), Government/Social (GCAT), and Markets (MCAT), treated as ground-truth clusters. The four categories contain 4,840, 3,470, 2,673, and 1,017 samples, respectively, giving a mildly imbalanced benchmark. The official split contains 10,000 training samples and 2,000 test samples. STL109 [48]: An image dataset containing 13,000 96 × 96 color images from 10 object categories, with 1,300 images per category. RCV1-1010 [3]: An imbalanced 10-category subset of RCV1 constructed from single-label articles in categories C14, C18, C313, C42, E21, E311, GDEF, GODD, GWELF, and M13. It contains 177,669 documents represented by tf–idf features over the 2,000 most frequent word stems. The class distribution is highly skewed, ranging from 903 documents in GWELF to 53,127 documents in M13, making it a benchmark for evaluating clustering under severe class imbalance. 2 CIFAR100 webpage: https://www.cs.toronto.edu/~kriz/cifar.html 3 CIFAR10 webpage: https://www.cs.toronto.edu/~kriz/cifar.html

4 FMNIST repository: https://github.com/zalandoresearch/fashion-mnist 5 ImageNet webpage: https://image-net.org/

6 MNIST webpage: http://yann.lecun.com/exdb/mnist/

7 Preprocessed Reuters subset: https://github.com/piiswrong/dec 8 RCV1 webpage: https://trec.nist.gov/data/reuters/reuters.html 9 STL10 webpage: https://cs.stanford.edu/~acoates/stl10/

10 Preprocessed RCV1-10 subset released with [3]: https://github.com/spherepaircc/SpherePairCC

17

These benchmarks follow common evaluation practice in deep clustering and constrained clustering: MNIST, FMNIST, Reuters subset, CIFAR10, ImageNet10, STL10, and CIFAR100-20 have been widely used in prior deep clustering/DCC studies [49, 52, 13, 14, 15, 3], while RCV1-10 follows the imbalanced text benchmark introduced in [3]. For MNIST, FMNIST, and Reuters subset, we use the official train/test splits, yielding 60,000/10,000, 60,000/10,000, and 10,000/2,000 training/test samples, respectively. For CIFAR100-20, CIFAR10, ImageNet10, STL10, and RCV1-10, we randomly split each dataset into 80% training and 20% testing, yielding 48,000/12,000, 48,000/12,000, 10,400/2,600, 10,400/2,600, and 142,135/35,534 training/test samples, respectively. All probabilistic constraints for training are generated from the training split only. C.2

Compared methods

We summarize the compared methods and their use under probabilistic pairwise supervision. Methods tied to hard binary constraints or discrete signed priors are not included, as they do not natively operate on the real-valued probabilistic supervision considered in UPCC. VanillaDCC. VanillaDCC [19] is an end-to-end DCC method based on the Meta Classification Likelihood (MCL) loss. Given the soft cluster-assignment vector q j = (qj1 , . . . , qjC ) for each instance xj , the pairwise co-clustering probability of a constrained pair is Pico = q ai q ⊤ bi . The MCL loss is |C|

1 X [yi log Pico + (1 − yi ) log(1 − Pico )] . LMCL = − |C| i=1 Although originally used for binary constraints, this logistic/BCE-form objective can directly take real-valued targets yi ∈ [0, 1], so VanillaDCC serves as a simple end-to-end DCC baseline under probabilistic supervision. VolMaxDCC. VolMaxDCC [15] extends MCL by introducing an explicit noise-modeling mechanism through a confusion-adjusted co-clustering probability. Concretely, it replaces Pico with Pi′co = q ai Bq ⊤ bi , where B is derived from a learnable confusion matrix, and adds a volume maximization term: |C|

1 X LVolMax = − [yi log Pi′co + (1 − yi ) log(1 − Pi′co )] − λ log det(Q⊤ Q), |C| i=1 where Q is the assignment matrix. The first term allows the model to account for systematic annotation confusion, while the volume term encourages separated and distinguishable cluster assignments. We therefore include VolMaxDCC as the compared baseline with explicit noise modeling. At the same time, its design is still rooted in noisy binary supervision and assignment-level separation, whereas UPCC requires preserving a continuum of probabilistic pairwise relations rather than only correcting corrupted hard relations. CIDEC. CIDEC [14] extends the DEC/IDEC deep embedding clustering framework [49, 52] by incorporating pairwise constraints through the MCL loss. It uses an autoencoder to obtain latent embeddings, initializes C cluster anchors by K-means, and computes a soft assignment vector q j = (qj1 , . . . , qjC ) for each instance. Its unsupervised clustering term follows the DEC-style KL objective LDEC =

|X | C X X j=1 c=1

pjc log

pjc , qjc

where pj is the sharpened target distribution derived from q j . CIDEC combines this clustering loss with a reconstruction loss and the MCL pairwise constraint loss. In our setting, its MCL component uses the real-valued yi directly through the same BCE-form target as VanillaDCC, making CIDEC a strong end-to-end baseline compatible with probabilistic supervision. 18

SpherePair. SpherePair [3] is a deep constraint embedding method that learns normalized angular representations with an autoencoder. For binary hard constraints, it defines an angular BCE-form loss Lang = −

|C| i 1 Xh − yi log s+ + (1 − y ) log(1 − s ) i ai bi hard,ai bi , |C| i=1

− where s+ ai bi is the must-link similarity and shard,ai bi is the cannot-link transformed similarity:

1 1 s− s+ hard,ai bi = 2 cos(min{ωθz ai ,z bi , π}) + 1 . ai bi = 2 cos(θz ai ,z bi ) + 1 ,

Here ω controls the negative zone of angular size π/ω; the SpherePair theory fixes ω = 2, giving the π/2 negative-zone boundary and the corresponding conflict-free equidistant geometry for hard must-link/cannot-link constraints. Since this original formulation is defined for binary relation types, applying SpherePair to UPCC requires a continuous counterpart for intermediate labels. We therefore keep the same ω = 2 angular scale and use
2 1 s− soft,ai bi = 2 cos(2θz ai ,z bi ) + 1 = cos (θz ai ,z bi ) in the BCE-form loss with each real-valued yi ∈ [0, 1]: Lsoft ang = −

|C| i 1 Xh − yi log s+ + (1 − y ) log(1 − s ) i ai bi soft,ai bi . |C| i=1

Under this soft extension, the attractive endpoint yi = 1 is minimized at θzai ,zbi = 0, while the fully repulsive endpoint yi = 0 is minimized at θzai ,zbi = π/2, matching the orthogonal negative-zone boundary selected by the original ω = 2 SpherePair geometry. For intermediate labels, the preferred pairwise angle varies continuously between these two endpoints; in the single-pair case, the optimum satisfies cos θzai ,zbi = yi /(2 − yi ). SpherePair also uses the normalized-embedding reconstruction P regularizer Lrec = |X1 | j ∥xj − x̂j ∥22 with x̂j = g(Norm(f (xj ))), which is also adopted by our ProbPair, Weighted ProbPair, and ECI-PP. Thus, the SpherePair baseline in our experiments is the natural real-valued angular extension of the original hard-constraint method, while its formal geometric guarantee remains tied to the binary setting. ProbPair and Weighted ProbPair. ProbPair is the direct probabilistic pairwise baseline introduced in Section 4.1. It uses the probabilistic angular readout in Eq. (6) and minimizes the reconstructionregularized objective L in Eq. (8), without our Estimator–Corrector–Integrator refinement. Weighted ProbPair further applies the information-based constraint weight κi in Eq. (9) to the pairwise term: |C|

LWPP = −

1 X κi [yi log ŷai bi + (1 − yi ) log(1 − ŷai bi )] + λrec Lrec . |C| i=1

Thus, ProbPair isolates the base probabilistic angular formulation, while Weighted ProbPair isolates the effect of decisiveness-based weighting. Comparing Weighted ProbPair with ECI-PP further tests the contribution of the Estimator–Corrector–Integrator pipeline beyond static reweighting. C.3

Constraint generation

Here, we detail how probabilistic pairwise constraints are generated in our experiments. We use trained expert classifiers as simulated judgment channels to produce probabilistic pairwise judgments and then apply the corruption channel in Section 3.1, thereby instantiating the UPCC observation process for constraint generation. Expert classifiers. For each dataset, simulated experts are trained on controlled labeled subsets of the training split and then used to generate predictive class distributions for queried pairs. The limited labeled-subset construction makes these classifiers expert-dependent judgment mechanisms, whose soft predictive distributions reflect both systematic bias and residual uncertainty. The subset selection rules for single- and multi-expert settings are described below, while the expert network architecture and optimization details are given in Appendix C.5. 19

Table 2: Unfamiliar-class counts per expert under the multi-expert settings. These counts describe the blind-spot distribution used to construct complementary experts. Dataset type

multi2

multi3

multi10

20-class datasets 10-class datasets 4-class datasets

(10, 10) (5, 5) (2, 2)

(7, 7, 6) (4, 3, 3) (2, 1, 1)

(2, 2, 2, 2, 2, 2, 2, 2, 2, 2) (1, 1, 1, 1, 1, 1, 1, 1, 1, 1) (1, 1, 1, 1, 1, 1, 1, 1, 1, 1)

Single-expert settings. In the single-expert setting, one expert classifier is trained per dataset. We use three expert-quality levels, denoted by lv0.1, lv0.01, and lv0.001. For lvr, the expert is trained using an r fraction of the labeled training samples from each class. The sampling is class-stratified: for every class, the number of selected samples is ⌊rnc ⌋, where nc is the number of available training samples in class c, with at least one sample retained for each class. Thus, smaller values of r produce weaker experts and hence more epistemically distorted probabilistic judgments. Multi-expert settings. In the multi-expert setting, we construct a group of complementary experts. The three settings multi2, multi3, and multi10 use 2, 3, and 10 experts, respectively. Each expert has a set of familiar classes and a set of unfamiliar classes. For all multi-expert settings, familiar classes use a high labeled-data fraction h = 0.1, while unfamiliar classes use a low labeled-data fraction l = 0.0001. The class-level training fraction for expert e is therefore  h, c ∈ Fe , ρe,c = l, c ∈ Ue ,

where Fe and Ue denote the familiar and unfamiliar class sets of expert e.

The unfamiliar classes are assigned to make the experts complementary. Let the dataset contain C classes under the fixed label ordering used in expert generation. When the number of experts E satisfies E ≤ C, the class list is split into E consecutive blocks that are as balanced as possible. Expert e takes the e-th block as its unfamiliar classes and treats all remaining classes as familiar. Equivalently, if C = qE + r with 0 ≤ r < E, then the first r experts have q + 1 unfamiliar classes and the remaining experts have q unfamiliar classes. For example, on a 10-class dataset, multi2 uses unfamiliar-class counts (5, 5), multi3 uses (4, 3, 3), and multi10 gives each expert exactly one unfamiliar class. For CIFAR100-20, the corresponding counts are (10, 10), (7, 7, 6), and (2, . . . , 2). For Reuters subset with four classes, multi2 gives (2, 2) and multi3 gives (2, 1, 1). If E > C, the experts are divided into full groups of size C, where each expert in a group is unfamiliar with one distinct class; any remaining experts form a partial group whose unfamiliar classes are sampled without replacement from the class list. This case occurs for Reuters subset under multi10. The resulting unfamiliar-class counts are summarized in Table 2. Pair sampling and expert assignment. All queried pairs are sampled randomly and uniformly from the training split. In the single-expert setting, every sampled pair is assigned to the single available expert. In the multi-expert setting, constraints are generated separately for each expert, and the total constraint budget is distributed evenly across experts unless otherwise specified. For a queried pair (xa , xb ) assigned to expert e, the trained expert outputs predictive class distributions pea and peb . The clean expert judgment is computed as jud ye,ab = ⟨pea , peb ⟩ =

C X c=1

pea,c peb,c ∈ [0, 1].

This value is high for similar expert-assigned class distributions, low when the expert separates the samples, and intermediate under uncertainty or overlapping class mass. Stochastic corruption. After the clean judgment is obtained, the recorded constraint value is produced through the corruption channel in Section 3.1. For each pair, an independent corruption indicator is drawn as ci ∼ Bernoulli(π), where π is the corruption probability used by the corresponding experiment. If ci = 0, the recorded label is the clean expert judgment: yi = yejud . i ,ai bi 20

If ci = 1, the clean judgment is replaced by an unstructured random value: yi ∼ Uniform(0, 1). Thus, the final constraints combine three factors: data-dependent soft pairwise ambiguity expressed through expert predictive distributions, structured expert-dependent epistemic distortion from classspecific familiarity, and unstructured stochastic corruption from the uniform replacement channel. C.4

Experimental protocol

Given the dataset details in Appendix C.1 and the constraint-generation procedure in Appendix C.3, we specify the experimental protocol for comparative evaluations, expert-aware ensemble references, held-out diagnostics, and sensitivity/ablation studies. Comparative evaluations. For clustering-performance comparisons, we report Accuracy (ACC), Normalized Mutual Information (NMI), and Adjusted Rand Index (ARI) over five trials. For embedding-based methods, including SpherePair, ProbPair, Weighted ProbPair, and ECI-PP, the final partition is obtained by applying K-means to the learned representations, while end-to-end DCC baselines use their native clustering outputs. For the full main comparison, we use lv0.01 and multi3 with corruption probability 0.3 and total constraint budgets 3k/6k/9k; under multi3, these correspond to 1k/2k/3k constraints per expert. Robustness comparisons use a total budget of 9k constraints and vary one supervision factor at a time. For single-expert robustness, we use lv0.01 and corruption probability 0.3 as the reference setting: expert quality is varied over lv0.1/lv0.01/lv0.001 while fixing corruption probability at 0.3, and corruption probability is varied over 0.1/0.3/0.5 while fixing the expert level at lv0.01. For multi-expert robustness, we compare multi2, multi3, and multi10 specified in Appendix C.3; under the 9k total budget, these correspond to 4,500, 3,000, and 900 constraints per expert, respectively. Moving from multi2 to multi10 increases both the number of experts and the average expert accuracy (Table 4), because the complementary blind-spot rule assigns fewer unfamiliar classes to each expert (see Table 2). Expert-aware ensemble references. Under multi-expert supervision, ECI-PP uses expert identities through its expert-specific Correctors, whereas expert-agnostic baselines treat the merged constraints as a single supervision pool. As an additional reference, we also include expert-aware ensemble variants for baselines for multi-expert comparisons. For a baseline trained under an E-expert setting, we instantiate E members, each associated with one expert identity. All members are trained on the |C| full merged constraint set C = {(ai , bi , ei , yi )}i=1 , but member e applies the multiplier  1, ei = e, (e) αi = α ∈ [0, 1], α, ei ̸= e, to the pairwise loss term of each constraint. Thus, α = 0 makes each member use only constraints from its associated expert, while α = 1 recovers the expert-agnostic merged-constraint training for every member. More generally, if the pairwise part of a baseline objective is decomposed into per-constraint terms ℓpair , member e uses i |C|

(e)

LB,pair =

1 X (e) pair α ℓ , |C| i=1 i i

while all non-pairwise components of the baseline objective are kept unchanged. After training, each member produces a hard partition Sb(e) . We aggregate the members through the co-association matrix E i 1 X h b(e) Aens = I S (xa ) = Sb(e) (xb ) , ab E e=1

ens then form Dab = 1−Aens ab and apply hierarchical clustering to obtain the final ensemble partition. The non-target expert weight α is selected separately for each dataset–baseline–multi-expert configuration using the reserved 1,000-sample validation split: we search α ∈ {0, 0.01, 0.05, 0.1, 0.5, 1}, run each candidate three times, and choose the value with the highest mean validation ACC before test evaluation. These ensemble variants give expert-agnostic baselines access to expert identities, but they also alter the training and aggregation structure through multiple expert-weighted members; they are therefore interpreted as auxiliary expert-aware references rather than fully symmetric replacements for the native baselines.

21

Held-out diagnostics. The held-out diagnostics examine whether the internal quantities of ECI-PP show trends consistent with their intended roles: Estimator-side beliefs, corrected relations, and integrated relations are compared with an external pairwise reference, and the post-correction gap is tested as a residual-discrepancy signal. To this end, we construct a diagnostic constraint set C diag where |C diag | = 1000 from the test split. These constraints are sample-disjoint from the training constraints and are used only for diagnosis, not for training, early stopping, or hyperparameter selection. For a held-out pair i = (ai , bi ) with ground-truth class labels tai and tbi , we define the external hard oracle o⋆i = I[tai = tbi ], which serves only as a diagnostic reference and is not identified with the canonical aleatoric relation Ra⋆i bi in Section 3. On C diag , which is not assigned to any cross-fitting fold, the Estimator diagnostic belief is computed by averaging the predictions of all K fold-specific Estimators; we still denote this held-out Estimator belief by ŷioof for notational consistency. We then evaluate ŷioof , the corrected relation yicor , and the Integrator belief yiint against this external oracle using Brier scores. For any held-out score vi ∈ [0, 1] and index set I, we use 1 X Brier(v; I) = (vi − o⋆i )2 . |I| i∈I

The Estimator and Integrator diagnostics are computed on all diagnostic constraints, whereas Corrector-oracle alignment is evaluated on the clean subset Iclean = {i ∈ C diag : ci = 0}. We also compute the clean post-correction discrepancy gapi,clean = softclipξ (ŷioof ) − yicor ,

i ∈ Iclean ,

to measure the residual disagreement between the Estimator belief and the corrected relation on uncorrupted held-out judgments. For corruption-screening diagnostics, the corresponding gap score is computed on all diagnostic constraints and used to rank corrupted records (ci = 1) against clean records (ci = 0), from which we report AUCcorrupt and APcorrupt . Because this gap captures unexplained discrepancy after correction, it may reflect both stochastic corruption and residual epistemic uncertainty, rather than stochastic corruption alone. We therefore also include an oraclesupervision diagnostic, where the clean relation is given by o⋆i before applying the same corruption channel; this provides a reference case in which expert-dependent epistemic distortion is removed. Sensitivity and ablation studies. Sensitivity and ablation studies are conducted for ECI-PP under the default single-expert setting, namely lv0.01 with corruption probability 0.3. Each study is repeated over five trials and varies one factor at a time while keeping the remaining hyperparameters and the constraint-generation protocol fixed. ProbPair and Weighted ProbPair already provide direct ablation-style references for the base probabilistic pairwise objective and the information-based weighting strategy, as described in Appendix C.2. Here, we further examine ECI-PP-specific design and sensitivity factors across its main components: the number of estimator folds K and the use of the information-based warm-up score κi in Eq. (9); the Corrector regularization weight λcor and its capacity; the reliability-screening parameters γ, n0 , and soft-clipping sharpness ξ; and the reconstruction weight λrec . C.5

Implementation

We provide implementation details11 for the experimental pipeline, including the computing environment, data preprocessing, expert classifiers and generated constraints, model backbones, pretraining, optimization, and hyperparameter settings. Software and hardware. All model training code is implemented in Python 3.7 and PyTorch 1.5.112 . For methods whose final prediction is obtained by K-means on learned representations, we use the K-means implementation in scikit-learn13 . For the hierarchical clustering step used by the expertaware ensemble references in Appendix C.4, we use the fastcluster package14 . Each training run is executed on a single NVIDIA A100 40GB or NVIDIA H200 141GB GPU. 11 Our source code is available on GitHub, and the URL is included in the supplementary materials. 12 PyTorch 1.5.1: https://github.com/pytorch/pytorch/releases/tag/v1.5.1

13 Scikit-learn webpage: https://scikit-learn.org/0.19/documentation.html 14 fastcluster library: https://pypi.org/project/fastcluster/

22

Table 3: Test accuracy (%) of single-expert classifiers used for constraint generation. Dataset

lv0.1

lv0.01

lv0.001

CIFAR100-20 CIFAR10 FMNIST ImageNet10 MNIST Reuters STL10 RCV1-10

73.08 91.01 84.45 96.27 93.67 92.90 90.04 94.26

65.23 88.49 74.77 92.62 88.76 83.60 84.31 90.47

46.58 73.83 59.83 87.62 65.20 49.75 72.77 77.62

Table 4: Average test accuracy (%) of multi-expert groups used for constraint generation. Each value is the arithmetic mean over experts in the corresponding group. Dataset

multi2

multi3

multi10

CIFAR100-20 CIFAR10 FMNIST ImageNet10 MNIST Reuters STL10 RCV1-10

41.23 52.72 54.57 73.87 50.45 47.58 53.98 49.06

51.90 65.94 60.68 76.73 65.89 63.02 65.67 64.57

67.36 82.74 77.64 90.80 86.13 71.42 83.22 85.96

Data preprocessing. We follow the same preprocessing pipeline as [3]. For Reuters subset and RCV1-10, we directly use the preprocessed tf–idf representations over the 2,000 most frequent words or word stems, as described in Appendix C.1. For MNIST and FMNIST, the 28 × 28 grayscale images are flattened into 784-dimensional vectors, following the standard fully connected input format used in prior deep clustering/DCC studies [13, 14, 2, 3]. For CIFAR10, CIFAR100-20, STL10, and ImageNet10, we adopt the unsupervised feature extraction strategy in [53]: a ResNet-34 model [54] is trained for 1,000 epochs, and the resulting 512-dimensional representations are used as input features. This image preprocessing follows the setting used in DCC studies [15, 3] and keeps the inputs compatible with fully connected architectures across datasets. The same preprocessed inputs are used for expert classifier training and for all compared clustering methods. Expert classifiers and generated constraints. The expert classifiers used in Appendix C.3 are implemented as MLP classifiers with two hidden layers of size 512–512, ReLU activations, dropout rate 0.2, and a final softmax classification layer. They are trained with cross-entropy loss using Adam, with learning rate 10−3 , batch size 256, maximum 100 epochs, and early stopping patience 10 based on training accuracy. The resulting test accuracies of single experts are reported in Table 3, and the average test accuracies of multi-expert groups are reported in Table 4. These accuracies characterize the effective quality of the simulated experts produced by the subset-selection and blind-spot protocols in Appendix C.3. After the corruption channel in Section 3.1, these expert judgments yield the recorded constraint values used for training. Fig. 4 visualizes their per-dataset distributions for representative lv0.01 single-expert and multi3 multi-expert settings, both with corruption probability 0.3. Model backbones and pretraining. For encoder-based methods, we follow the fully connected autoencoder backbone used in [13, 14, 3]: the encoder has hidden layers 500–500–2000, paired with a symmetric decoder when reconstruction is used. The embedding dimension is set to D = 10 for all datasets except CIFAR100-20, where D = 20, following [3]. VanillaDCC and VolMaxDCC use their native MLP classifier architectures with two hidden layers of size 512–512, and do not use autoencoder pretraining. For pretrain-capable methods, we use the two-stage stacked denoising autoencoder (SDAE) pretraining approach [55, 56], following the protocol used in [13, 14, 3]: hidden layers are first pretrained layer-wise as denoising autoencoders, followed by end-to-end autoencoder fine-tuning on the training split. Unless otherwise specified, all pretrain-capable methods use this 23

Frequency

CIFAR100-20

1.0

CIFAR10

1.0

FMNIST

1.0

IMAGENET10

1.0

MNIST

1.0

REUTERS

1.0

STL10

1.0

0.8

0.8

0.8

0.8

0.8

0.8

0.8

0.8

0.6

0.6

0.6

0.6

0.6

0.6

0.6

0.6

0.4

0.4

0.4

0.4

0.4

0.4

0.4

0.4

0.2

0.2

0.2

0.2

0.2

0.2

0.2

0.0 1.0

Frequency

(A) lv0.01 corruption rate 0.3 (B) multi3 corruption rate 0.3

1.0

0.0 0.2 0.4 0.6 0.8 1.0

0.0 1.0

0.0 0.2 0.4 0.6 0.8 1.0

0.0 1.0

0.0 0.2 0.4 0.6 0.8 1.0

0.0 1.0

0.0 0.2 0.4 0.6 0.8 1.0

0.0 1.0

0.0 0.2 0.4 0.6 0.8 1.0

0.0 1.0

0.0 0.2 0.4 0.6 0.8 1.0

0.0 1.0

0.2 0.0 0.2 0.4 0.6 0.8 1.0

0.0 1.0

0.8

0.8

0.8

0.8

0.8

0.8

0.8

0.8

0.6

0.6

0.6

0.6

0.6

0.6

0.6

0.6

0.4

0.4

0.4

0.4

0.4

0.4

0.4

0.4

0.2

0.2

0.2

0.2

0.2

0.2

0.2

0.0

0.0

0.0

0.0

0.0

0.0

0.0

0.0 0.2 0.4 0.6 0.8 1.0

Constraint value

0.0 0.2 0.4 0.6 0.8 1.0

Constraint value

0.0 0.2 0.4 0.6 0.8 1.0

Constraint value

0.0 0.2 0.4 0.6 0.8 1.0

Constraint value

0.0 0.2 0.4 0.6 0.8 1.0

Constraint value

0.0 0.2 0.4 0.6 0.8 1.0

Constraint value

RCV1-10

0.0 0.2 0.4 0.6 0.8 1.0

0.2 0.0 0.2 0.4 0.6 0.8 1.0

Constraint value

0.0

0.0 0.2 0.4 0.6 0.8 1.0

Constraint value

Figure 4: Distributions of recorded constraint values for representative generated-supervision settings. The top row shows lv0.01 single-expert supervision and the bottom row shows multi3 multi-expert supervision, both with corruption probability 0.3. pretrained initialization. For methods using angular reconstruction (SpherePair, ProbPair, Weighted ProbPair, ECI-PP), decoding is performed from normalized embeddings as in [3]. In ECI-PP, both the K Estimator backbones and the Integrator use the same autoencoder architecture described above. Each Estimator fold has its own encoder–decoder backbone and its own probabilistic readout parameters (mk , Tk ), while the Integrator has its own readout parameters (m, T ) in Eq. (6). These scalar readout parameters are parameterized as m = tanh(m̃) and T = softplus(T̃ ) for numerical stability within the cosine range, with the same parameterization used for each Estimator fold. When pretraining is used, the same pretrained autoencoder weights initialize the Integrator backbone and all Estimator backbones, while the readout parameters and Correctors are initialized separately. Unless otherwise specified, each expert-specific Corrector is an MLP with hidden layers 64–16 and a tanh output, which keeps the predicted probability-space residual within (−1, 1). Optimization and hyperparameters. We use the reported or recommended hyperparameters for the baseline methods whenever available. Below, we summarize the optimization settings and implementation configurations for ECI-PP and all compared baselines. • VanillaDCC. VanillaDCC optimizes the MCL loss LMCL using Adam with learning rate 10−3 and batch size 256. Training runs for at most 500 epochs, with early stopping triggered when the relative change in the soft cluster assignments on the training samples falls below 10−3 for two consecutive checks. This assignment-change stopping rule follows common practice in deep clustering and deep constrained clustering [49, 52, 13, 14]. • VolMaxDCC. For VolMaxDCC, we follow its original optimization and validation-based hyperparameter-search procedure [15]. The learnable matrix B is parameterized elemen−1 ′ ′ ′ twise as Bcc′ = (1 + exp(−Bcc , where Bcc ′ )) ′ is initialized to 1 for c = c and to −1 otherwise. Optimization uses SGD with learning rate 0.5 for the network parameters and 0.1 for B ′ , with batch size 128. The VolMaxDCC geometric regularization weight is selected from {0, 10−1 , 10−2 , 10−3 , 10−4 , 10−5 } using the reserved validation split; for each dataset and supervision setting, we choose the value with the highest mean validation ACC over three validation runs. For multi-expert supervision, a shared selected value is used for the multi-expert settings defined in Appendix C.3, where the familiar and unfamiliar labeled-data fractions are fixed as h = 0.1 and l = 0.0001. The selected values are reported in Table 5. • CIDEC. For CIDEC, we follow the authors’ recommended settings: the reconstruction/clustering trade-off is set to λ1 = 1, and the MCL constraint-balancing parameter is set to λ2 = 0.1. The C cluster anchors are initialized by K-means. Optimization uses Adam with learning rate 10−3 and batch size 256. Training proceeds for at most 500 epochs, with early stopping applied when the relative change in the soft assignments falls below 10−3 over consecutive checks. • SpherePair. For SpherePair, the reconstruction weight is set to 0.02, matching the setting used in [3]. In the original binary formulation, the negative-zone factor is fixed as ω = 2, which yields the theoretically guaranteed π/2 negative-zone boundary. As described in Appendix C.2, we use the real-valued extension of SpherePair by keeping the same ω = 2 24

Table 5: Selected VolMaxDCC geometric regularization weights. Values are selected by validation ACC over three runs for each dataset and supervision setting. Dataset

lv0.1 −3

CIFAR100-20 CIFAR10 FMNIST ImageNet10 MNIST Reuters STL10 RCV1-10

10 10−3 10−3 0 10−2 10−2 10−3 10−4

lv0.01 −3

10 10−3 10−3 10−4 10−3 0 10−3 10−4

lv0.001 −1

10 10−1 10−2 10−1 10−2 10−1 10−2 10−5

multi 10−2 10−2 10−2 10−2 10−2 10−1 10−1 10−3

Table 6: Selected expert-aware ensemble hyperparameter α. Values are chosen by mean validation ACC over three runs and shared across the multi-expert configurations defined in Appendix C.3. Dataset CIFAR100-20 CIFAR10 FMNIST ImageNet10 MNIST Reuters STL10 RCV1-10

SpherePair

Weighted ProbPair

0.05 1.00 1.00 0.00 0.05 0.05 0.01 0.01

0.05 0.05 0.50 0.10 0.50 0.05 0.01 0.05

angular scale and replacing the original clamped cannot-link score with the continuous
1 counterpart s− soft,ai bi = 2 cos(2θz ai ,z bi )+1 . The observed yi ∈ [0, 1] is then used directly in the BCE-form angular loss. Under this implementation, yi = 1 favors θzai ,zbi = 0 and yi = 0 favors θzai ,zbi = π/2, while intermediate labels continuously interpolate between these two angular endpoints through the two BCE terms. Optimization uses Adam with learning rate 10−3 and constraint mini-batch size 256. The instance mini-batch size for reconstruction is set according to the number of constraint mini-batches, following [3]. Training runs for at most 500 epochs, with early stopping after the first 100 epochs if the relative change in the total loss remains below 0.1 for five consecutive checks. For the expert-aware ensemble reference in Appendix C.4, the expert-aware hyperparameter α is selected on the reserved validation split; the selected values are reported in Table 6. • ProbPair variants and ECI-PP (Ours). ProbPair, Weighted ProbPair, and ECI-PP use the ProbPair-style readout in Eq. (6) and the reconstruction weight λrec = 0.02. ProbPair optimizes Eq. (8), while Weighted ProbPair additionally applies the information-based weight κi in Eq. (9). For the expert-aware ensemble reference based on Weighted ProbPair in Appendix C.4, the expert-aware hyperparameter α is selected on the reserved validation split; the selected values are reported in Table 6. The shared base optimizer is Adam with learning rate 10−3 and batch size 256. The probabilistic readout parameters, including (m, T ) and the fold-wise (mk , Tk ) when present, are initialized as (0, 0.1) and optimized with learning rate 10−2 . We use a fixed outer training budget of 500 epochs; principled stopping criteria for iterative refinement remain an open question for future study. For any weighted ProbPair-style objective, including Weighted ProbPair and the weighted Estimator/Integrator updates in ECI-PP, the weight is applied inside the constraint average: i 1 X h wi ỹi log ŷai bi + (1 − ỹi ) log(1 − ŷai bi ) , |C| i=1 |C|

Lw PP = −

where ỹi denotes the corresponding training target, such as yi , yiint , or yiBC . For ECI-PP, we use one global hyperparameter setting unless explicitly varied in sensitivity or ablation studies: K = 5 estimator folds, λcor = 0.5, γ = 10, n0 = 10, soft-clipping sharpness ξ = 20, and information-based warm-up enabled. The default warm-up trains 25

the Estimators and Integrator for 50 epochs before iterative refinement. The initial ŷioof is obtained by strict fold-heldout prediction, and later refinement rounds use refreshed Estimator beliefs under the Integrator-updated targets while retaining the same fold structure. Each Corrector update uses the shared base optimizer setting, with the Huber quadraticto-linear transition in Eq. (10) fixed at 0.1, holds out 10% of its assigned constraints for internal validation, runs for at most 50 epochs, and stops early if the validation loss does not improve for 10 consecutive epochs. Between refinement iterations, Gaussian noise with standard deviation 0.01 is added to the final linear layers of the Estimators, Correctors, and Integrator to mildly perturb the next refinement step. Notably, the validation-based selections of the VolMaxDCC geometric regularization weight and the expert-aware ensemble hyperparameter α rely on ground-truth class labels on the validation split, which are typically unavailable in constrained clustering and are not part of the observable supervision under UPCC. In contrast, ECI-PP uses one global hyperparameter setting across datasets and handles expert identities directly through expert-specific Correctors, without datasetspecific tuning or an additional hyperparameterized ensemble wrapper.

D

Additional experimental results

D.1

Comparison under different constraint budgets

Tables 7 and 8 report the full main-comparison results under the default lv0.01 single-expert and multi3 multi-expert regimes, with corruption probability fixed at 0.3. The total constraint budget is varied over 3k/6k/9k; under multi3, these correspond to 1k/2k/3k constraints per expert. All results in this subsection use pre-trained backbones for methods with a pretraining stage, while VanillaDCC and VolMaxDCC have no pretraining variant. The main-text conclusion remains stable across budgets. Across the 144 test entries in the two tables, ECI-PP ranks first in 122/144 entries and within the top two in 134/144, while one ProbPair-family method is best in 137/144 entries. The budget-wise pattern is not completely uniform, as expected: with only 3k constraints, all methods are less stable and the separation between methods is smaller, whereas at 9k several strong baselines also improve and narrow some ACC gaps. Even under these effects, ECI-PP remains the leading method at every budget, ranking first in 38/48, 43/48, and 41/48 entries under 3k, 6k, and 9k constraints, respectively. The advantage is clearest on structure-sensitive metrics. ECI-PP gives the best NMI in 46/48 entries and the best ARI in 43/48, compared with 33/48 for ACC. Since NMI and ARI better reflect grouping consistency than matched-label accuracy alone, this suggests that ECI-PP most reliably improves the learned clustering structure. For example, on single-expert CIFAR100, ECI-PP raises NMI from roughly 44–46% for the strongest non-ECI competitors to 47–50% across budgets; on multi-expert Reuters, it raises NMI from about 53–58% for Weighted ProbPair to 65–69%. Dataset-wise, ECI-PP is best in all entries on CIFAR100, Reuters, and STL10, and in nearly all entries on CIFAR10 and ImageNet10. FMNIST and MNIST show more competition from Weighted ProbPair or SpherePair, especially on ACC and low-budget MNIST, but ECI-PP remains strongest or near-strongest on NMI/ARI in most cases. The ProbPair variants show a consistent ablation pattern. Weighted ProbPair improves over ProbPair in most entries, and this effect is stronger under multi-expert supervision (67/72 entries) than under single-expert supervision (62/72 entries). This supports the interpretation that information-based weighting is especially useful when low-decisiveness constraints may reflect expert unfamiliarity. ECI-PP further improves over Weighted ProbPair in 129/144 entries, showing that weighting alone does not replace explicit estimation, correction, and integration. Notably, VolMaxDCC also provides a useful noise-aware reference among the non-ProbPair baselines. It improves over VanillaDCC and is competitive on some datasets, but reaches the top two in only 15/144 entries. This suggests that explicit noise modeling is beneficial, yet its hard-noise-oriented formulation is not sufficient for the expert-conditioned probabilistic supervision in UPCC. The main caveat remains the severely imbalanced RCV1-10 benchmark. SpherePair remains the main ACC competitor there, consistent with the advantage of its angular geometry for imbalanced pair structures. Nevertheless, ECI-PP gives the best NMI across all RCV1-10 budgets and regimes, and 26

Table 7: Comparative performance (%) (ACC, NMI, ARI) across datasets for methods under the lv0.01 setting, with corruption probability 0.3 and 3k/6k/9k constraints. Blue and black represent training and test results, respectively. Best results are in bold, and second-best are underlined.

ACC 3k NMI ARI CIFAR100 ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI CIFAR10 ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI FMNIST ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI ImageNet10 ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI MNIST ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI REUTERS ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI STL10 ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI RCV1-10 ACC 6k NMI ARI ACC 9k NMI ARI

VanillaDCC 14.7, 14.7 11.2, 11.8 4.0, 4.1 16.5, 16.6 12.6, 13.5 5.4, 5.6 16.8, 17.0 13.5, 14.6 6.0, 6.3 34.6, 34.6 27.9, 28.1 20.2, 20.3 35.5, 35.7 27.5, 28.3 20.7, 21.2 38.3, 38.7 29.9, 31.2 23.2, 24.0 36.6, 36.5 31.1, 31.1 19.7, 19.5 38.6, 38.5 30.9, 31.3 20.7, 20.9 39.8, 40.2 30.1, 31.3 21.5, 22.2 35.4, 37.3 27.7, 30.8 19.1, 20.8 40.6, 42.0 30.6, 35.2 23.2, 26.0 39.0, 41.5 27.6, 33.7 21.1, 25.2 25.9, 26.1 16.0, 17.0 10.4, 10.8 33.3, 33.9 24.2, 25.8 18.4, 19.4 38.9, 40.3 29.5, 32.2 23.1, 24.8 53.2, 54.0 15.1, 17.9 17.0, 18.0 66.0, 70.6 27.7, 36.3 34.0, 42.6 69.7, 77.0 30.5, 43.3 36.8, 50.4 30.9, 31.5 21.1, 23.4 14.1, 15.0 30.4, 31.6 19.5, 22.2 14.3, 15.7 35.5, 37.8 22.9, 27.6 17.1, 19.8 40.0, 40.1 12.3, 12.4 16.4, 16.5 52.4, 52.6 21.8, 22.2 28.0, 28.2 46.2, 46.4 15.5, 15.9 19.8, 20.1

VolMaxDCC 42.8, 42.5 42.5, 42.9 26.2, 26.0 45.4, 45.3 43.9, 44.5 28.2, 28.2 46.1, 45.9 45.6, 46.1 30.4, 30.5 79.9, 79.8 69.3, 69.4 64.4, 64.4 84.7, 84.6 73.6, 73.7 71.0, 70.9 82.5, 82.5 71.9, 72.2 68.3, 68.3 63.3, 62.5 55.5, 55.2 44.4, 43.6 66.2, 65.1 56.7, 56.3 46.6, 45.5 66.5, 65.5 57.2, 56.8 46.4, 45.2 86.7, 86.9 78.8, 79.4 74.1, 73.9 87.2, 87.5 78.3, 79.3 75.1, 75.6 87.4, 88.1 77.8, 79.2 75.2, 76.1 65.6, 66.4 52.7, 54.1 47.8, 48.9 73.4, 74.4 58.4, 60.5 55.2, 56.8 75.6, 76.9 59.3, 61.6 56.5, 58.7 62.8, 63.2 25.6, 28.0 31.2, 32.4 72.5, 76.8 36.0, 44.9 42.4, 50.8 71.7, 76.8 34.3, 43.4 40.9, 50.4 72.8, 73.3 64.8, 66.5 56.3, 57.0 76.8, 77.5 65.8, 67.7 60.0, 61.3 79.0, 80.1 66.7, 69.3 62.8, 64.7 46.3, 46.4 21.5, 21.9 21.2, 21.4 62.1, 62.4 36.0, 36.7 42.9, 43.3 61.3, 61.8 35.7, 36.7 44.7, 45.4

CIDEC

SpherePair

ProbPair

14.9, 14.6 13.1, 13.6 2.6, 2.1 16.6, 15.6 15.0, 15.1 3.4, 2.9 14.3, 13.6 11.0, 11.2 2.8, 2.3 54.1, 52.7 40.8, 40.7 23.0, 20.5 51.3, 51.0 39.8, 41.1 22.9, 22.8 44.3, 44.0 36.1, 37.6 21.1, 21.3 35.4, 34.4 31.6, 31.3 13.5, 12.4 33.8, 33.3 26.9, 27.1 12.2, 11.9 36.1, 35.7 29.7, 30.5 15.7, 15.6 69.0, 71.2 53.9, 58.7 43.4, 47.1 67.0, 71.2 50.3, 58.7 40.2, 46.3 63.5, 68.3 47.2, 57.5 35.5, 42.2 59.1, 56.5 51.9, 51.0 30.3, 26.7 59.2, 57.0 52.0, 52.1 29.8, 26.4 47.0, 46.7 41.4, 43.7 24.7, 24.2 74.4, 77.0 42.8, 49.8 50.4, 56.7 74.2, 79.3 38.2, 48.8 44.6, 55.6 72.9, 79.3 35.7, 48.1 41.8, 54.9 66.8, 69.4 49.9, 55.1 40.5, 44.8 62.7, 66.7 44.4, 51.4 34.1, 40.0 57.8, 63.3 39.3, 48.5 29.8, 37.3 47.4, 48.7 33.3, 32.5 28.6, 29.1 42.3, 43.5 30.5, 30.5 24.9, 26.7 42.5, 44.3 29.5, 30.0 26.2, 28.3

45.7, 45.7 42.8, 43.3 28.5, 28.5 49.0, 49.2 43.8, 44.9 31.5, 32.0 50.2, 50.5 43.0, 44.2 32.1, 32.6 84.6, 84.9 74.3, 74.7 71.7, 72.0 84.8, 85.3 73.1, 74.2 71.5, 72.3 84.9, 85.7 72.4, 74.1 71.2, 72.5 71.1, 70.0 61.0, 60.7 53.2, 52.0 70.9, 69.8 60.6, 60.5 53.2, 52.1 72.0, 71.2 60.4, 60.9 53.0, 52.3 88.0, 89.2 78.6, 81.4 76.8, 78.9 87.2, 89.3 75.4, 80.4 74.7, 78.7 85.1, 89.1 71.0, 79.2 70.7, 78.2 87.3, 87.9 78.1, 79.3 76.8, 77.8 89.3, 90.1 78.2, 80.2 78.2, 79.9 87.8, 89.1 75.4, 78.0 75.4, 77.8 81.4, 83.4 54.5, 58.9 59.8, 64.4 80.1, 83.2 52.0, 58.5 56.8, 63.2 77.9, 81.8 48.7, 56.3 52.9, 61.1 80.7, 81.7 66.7, 69.1 64.2, 66.1 79.0, 81.7 63.2, 68.2 61.2, 65.5 76.1, 79.7 59.5, 66.5 57.2, 63.3 65.1, 64.9 59.2, 59.6 56.1, 56.2 70.8, 70.7 60.6, 61.1 61.6, 61.8 71.2, 71.4 60.5, 61.3 61.0, 61.4

44.5, 44.4 42.4, 42.9 27.6, 27.6 46.1, 46.4 42.4, 43.4 28.7, 29.0 49.4, 49.5 43.2, 44.2 30.6, 30.9 84.8, 85.0 74.9, 75.3 71.9, 72.1 86.0, 86.0 75.6, 75.9 73.5, 73.5 85.9, 86.0 75.0, 75.4 73.0, 73.2 70.5, 69.5 61.0, 60.6 52.1, 51.0 72.7, 71.5 62.4, 62.1 53.9, 52.6 73.6, 72.5 62.9, 62.9 54.8, 53.7 89.6, 89.8 81.8, 82.3 79.4, 79.6 89.7, 90.5 81.2, 83.3 79.3, 80.7 89.2, 90.5 80.1, 83.2 78.5, 80.7 86.3, 86.6 75.5, 76.5 74.3, 74.9 88.0, 89.2 75.8, 78.1 75.8, 78.1 89.0, 90.0 76.9, 79.0 77.4, 79.4 79.6, 81.3 55.4, 58.5 58.9, 62.3 79.6, 81.5 55.0, 58.9 58.1, 62.0 78.2, 80.8 53.2, 58.3 55.5, 61.0 81.9, 83.0 69.7, 72.0 66.3, 68.2 81.6, 82.9 68.7, 71.6 65.9, 68.2 81.6, 83.4 68.2, 72.1 65.8, 68.9 54.0, 53.8 54.2, 54.4 43.3, 43.1 51.5, 51.3 54.1, 54.2 42.3, 42.1 52.7, 53.0 53.8, 54.1 43.6, 43.7

27

Weighted ProbPair 45.2, 45.2 43.4, 43.7 28.6, 28.5 48.3, 48.5 43.7, 44.5 30.2, 30.5 50.3, 50.8 44.5, 45.7 31.5, 32.1 85.8, 85.7 76.0, 76.0 73.4, 73.2 86.5, 86.5 76.3, 76.5 74.3, 74.2 86.6, 86.7 76.2, 76.8 74.3, 74.5 71.0, 70.0 61.3, 61.0 53.1, 52.1 73.6, 72.5 63.1, 62.9 54.6, 53.5 74.6, 73.6 63.9, 64.0 55.3, 54.3 90.0, 90.1 82.8, 83.3 80.5, 80.5 89.7, 90.2 81.6, 82.8 79.5, 80.0 89.1, 90.3 80.3, 82.7 78.4, 80.2 89.6, 90.2 78.4, 79.7 78.5, 79.7 89.5, 90.1 77.8, 79.2 78.4, 79.7 89.9, 90.8 78.3, 80.2 79.1, 80.9 82.8, 84.4 57.8, 61.6 62.9, 66.1 82.2, 83.9 56.7, 60.4 61.5, 65.3 80.4, 82.7 54.5, 59.2 58.6, 63.4 83.3, 83.9 71.5, 73.0 68.8, 69.8 82.8, 83.9 70.2, 72.5 67.9, 69.6 81.6, 83.1 67.9, 71.6 65.8, 68.4 52.6, 52.4 55.1, 55.3 43.5, 43.4 54.0, 53.9 57.5, 57.8 46.2, 46.1 55.2, 55.1 57.4, 57.5 47.1, 47.0

ECI-PP (Ours) 48.1, 47.8 46.0, 46.5 31.3, 31.4 51.2, 51.1 48.3, 48.8 34.7, 34.7 52.7, 52.9 49.2, 49.9 36.2, 36.5 85.5, 85.4 77.0, 76.9 73.5, 73.3 87.3, 87.2 78.6, 78.7 76.0, 76.0 87.8, 87.8 79.0, 79.1 76.7, 76.8 70.2, 69.2 64.0, 63.4 53.8, 52.6 71.3, 70.2 65.3, 64.5 55.1, 53.8 72.4, 71.1 65.7, 65.0 55.5, 53.9 92.2, 92.5 87.7, 88.1 85.0, 85.4 92.7, 92.9 88.3, 88.5 85.9, 86.1 92.8, 93.1 88.1, 88.5 85.8, 86.2 84.2, 84.6 75.0, 76.1 72.3, 73.1 89.5, 90.3 79.2, 80.3 78.9, 80.1 90.7, 91.0 80.7, 81.6 80.8, 81.5 86.0, 86.8 63.8, 65.6 69.4, 71.0 85.4, 86.4 62.4, 65.0 67.8, 70.3 85.1, 86.6 61.4, 64.4 66.6, 70.0 85.0, 85.3 75.3, 75.9 72.0, 72.5 85.4, 85.5 75.8, 76.3 72.7, 73.0 85.4, 85.8 75.8, 76.6 72.8, 73.4 61.7, 61.7 60.6, 60.8 53.3, 53.5 63.7, 63.7 62.9, 63.0 55.9, 56.1 69.0, 68.9 65.5, 65.8 62.0, 62.0

Table 8: Comparative performance (%) (ACC, NMI, ARI) across datasets for methods under the multi3 setting, with corruption probability 0.3 and 3k/6k/9k total constraints. Blue and black represent training and test results, respectively. Best results are in bold, and second-best are underlined.

ACC 3k NMI ARI CIFAR100 ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI CIFAR10 ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI FMNIST ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI ImageNet10 ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI MNIST ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI REUTERS ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI STL10 ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI RCV1-10 ACC 6k NMI ARI ACC 9k NMI ARI

VanillaDCC 12.2, 12.4 9.5, 9.9 2.6, 2.5 14.2, 14.4 11.3, 12.2 4.6, 4.7 15.4, 15.8 12.8, 13.6 5.8, 5.8 28.8, 28.8 25.2, 25.5 16.5, 16.6 33.6, 33.9 30.9, 32.0 22.4, 23.1 32.4, 32.8 29.1, 30.9 21.2, 22.4 33.7, 33.7 32.9, 33.0 20.7, 20.6 34.2, 34.3 32.1, 32.8 21.0, 21.2 35.2, 35.6 31.4, 32.8 20.4, 21.3 33.9, 34.7 29.0, 31.4 19.4, 20.7 32.7, 34.6 25.3, 30.1 17.9, 20.9 31.8, 34.2 22.0, 26.8 15.8, 18.7 27.0, 27.3 19.2, 20.5 11.5, 12.2 28.1, 29.0 22.0, 23.8 14.7, 15.7 32.2, 33.3 26.5, 29.3 18.4, 20.1 42.8, 41.4 3.2, 3.7 4.7, 4.0 51.4, 52.9 11.8, 15.9 14.0, 16.4 55.3, 59.2 14.7, 22.8 19.2, 26.2 25.6, 26.4 19.7, 21.7 12.3, 13.1 29.8, 31.2 22.3, 26.4 15.5, 17.7 29.2, 30.7 21.5, 26.0 15.2, 17.7 29.2, 29.2 2.5, 2.6 3.2, 3.1 43.2, 43.4 13.7, 14.0 16.6, 16.7 44.3, 44.6 14.3, 14.7 17.8, 18.0

VolMaxDCC 43.7, 43.3 43.4, 43.6 26.8, 26.6 43.1, 42.5 44.3, 44.5 27.7, 27.5 42.8, 42.5 43.4, 43.8 27.1, 27.1 76.0, 75.8 71.0, 70.9 63.3, 63.0 78.0, 78.1 72.6, 72.9 65.7, 65.8 76.5, 76.6 73.1, 73.2 65.8, 65.9 58.1, 57.7 56.7, 56.4 42.7, 42.1 58.2, 57.9 58.3, 58.1 44.4, 43.7 59.4, 59.1 59.2, 59.0 45.0, 44.3 88.8, 88.9 85.9, 85.8 81.9, 81.7 87.6, 88.1 86.5, 87.1 81.9, 82.4 89.9, 90.5 87.8, 88.7 83.9, 84.9 54.6, 55.4 48.5, 49.8 38.6, 39.5 62.5, 63.3 55.1, 56.6 47.1, 48.4 67.1, 67.8 57.5, 58.9 50.9, 52.1 66.9, 67.3 40.8, 42.5 42.8, 43.9 67.0, 68.2 44.3, 46.5 46.5, 48.7 67.5, 68.0 45.4, 46.5 47.0, 47.9 74.6, 74.8 71.5, 72.0 62.3, 62.8 72.7, 72.6 71.1, 71.2 61.1, 60.6 71.0, 71.1 70.2, 69.9 59.5, 59.1 31.7, 31.9 23.0, 23.6 10.8, 11.1 46.3, 46.6 30.2, 30.7 26.3, 26.6 48.3, 49.0 31.4, 32.1 30.2, 30.9

CIDEC

SpherePair

ProbPair

19.5, 18.4 20.5, 20.5 3.9, 3.2 17.4, 16.6 16.2, 16.3 3.2, 2.5 12.8, 12.1 10.0, 10.5 2.1, 2.0 38.4, 36.8 32.3, 31.7 13.4, 11.9 36.5, 34.9 31.9, 31.9 14.2, 14.0 33.2, 31.6 28.7, 29.1 12.3, 12.1 30.1, 29.6 29.8, 30.1 14.1, 14.0 30.4, 30.1 28.1, 29.6 14.2, 14.7 28.6, 28.8 28.3, 30.6 14.5, 15.4 50.7, 49.2 41.2, 43.3 21.7, 20.4 43.0, 41.6 35.4, 38.5 17.3, 17.9 39.4, 38.8 31.0, 34.8 14.3, 15.6 44.9, 41.8 45.6, 44.1 20.4, 17.8 39.8, 38.1 39.2, 39.3 16.8, 15.4 31.8, 31.0 28.8, 30.2 13.2, 13.4 61.4, 62.1 26.3, 32.2 28.1, 28.9 58.7, 60.4 19.8, 27.0 22.6, 25.1 57.3, 59.1 18.3, 25.7 20.4, 23.1 47.4, 47.4 38.3, 41.5 21.8, 22.4 37.6, 37.2 31.2, 37.2 15.8, 18.2 31.4, 31.6 24.9, 31.7 13.6, 16.9 33.5, 34.9 19.1, 18.3 14.3, 14.6 31.8, 32.0 13.5, 13.2 9.8, 9.7 32.3, 32.0 14.1, 13.5 10.8, 10.3

42.0, 42.1 41.5, 42.1 26.1, 26.3 42.4, 42.4 40.0, 40.9 26.2, 26.5 44.3, 44.4 40.5, 41.5 27.1, 27.6 74.8, 75.1 69.5, 70.2 63.0, 63.5 76.5, 76.8 66.9, 68.1 61.7, 62.5 73.8, 74.5 64.7, 66.5 59.0, 60.5 61.3, 61.0 58.1, 58.2 45.9, 45.6 65.4, 65.3 58.9, 59.7 48.7, 48.9 61.0, 61.3 56.6, 57.8 46.8, 47.4 81.0, 81.5 72.7, 74.2 68.2, 69.1 81.8, 83.7 69.2, 73.5 66.3, 70.0 80.7, 85.0 65.6, 73.8 63.4, 71.0 83.6, 84.4 75.2, 76.9 71.5, 72.4 78.8, 79.9 72.8, 75.3 68.2, 70.3 78.6, 80.0 70.8, 74.0 66.3, 69.1 72.8, 75.2 40.2, 44.6 45.2, 50.1 66.7, 72.9 32.8, 43.2 37.2, 48.2 62.1, 69.9 25.6, 37.2 28.0, 40.7 72.7, 73.4 61.7, 64.0 55.0, 56.4 67.7, 69.8 54.8, 58.8 48.6, 51.7 67.8, 71.4 53.2, 59.7 47.8, 53.3 51.1, 51.0 51.2, 51.7 39.5, 39.6 51.2, 51.4 51.1, 51.7 40.9, 41.1 55.7, 55.9 51.9, 52.7 43.6, 44.2

41.5, 41.6 40.2, 41.1 24.8, 25.3 42.0, 42.1 39.8, 40.6 25.2, 25.4 43.3, 43.4 40.0, 40.9 26.0, 26.2 79.8, 79.8 72.0, 72.1 67.0, 66.8 80.7, 81.0 71.2, 71.6 66.2, 66.6 81.3, 81.7 71.5, 72.3 67.1, 67.8 60.9, 60.3 59.2, 58.8 46.4, 45.8 64.1, 63.8 59.6, 59.7 47.8, 47.7 66.6, 66.7 60.7, 61.2 49.9, 50.0 87.8, 88.4 78.8, 80.3 76.7, 77.7 88.6, 89.4 78.3, 80.4 77.2, 78.6 87.4, 89.2 76.1, 80.1 75.0, 78.3 81.2, 82.2 73.0, 74.7 69.4, 70.9 81.0, 82.3 72.2, 74.4 68.7, 70.8 80.5, 81.8 72.1, 74.7 68.7, 71.2 78.6, 79.9 48.0, 50.3 55.0, 57.5 78.6, 81.4 48.3, 54.0 54.5, 60.3 74.5, 79.0 44.6, 52.8 50.2, 58.6 79.2, 79.9 66.5, 68.4 61.6, 62.9 81.0, 82.0 67.4, 69.9 64.0, 65.7 81.1, 83.0 66.1, 70.2 63.7, 67.1 47.9, 47.8 50.0, 50.3 37.8, 37.9 46.6, 46.6 48.1, 48.5 35.7, 35.8 43.4, 43.5 46.6, 47.4 34.4, 34.7

28

Weighted ProbPair 43.1, 43.3 41.9, 42.4 26.7, 26.8 44.1, 44.4 41.1, 41.9 26.4, 26.7 46.1, 46.5 41.8, 43.0 28.0, 28.5 82.0, 82.1 73.2, 73.3 68.7, 68.7 80.9, 81.0 71.9, 72.2 67.8, 67.9 81.4, 81.7 72.0, 72.6 67.9, 68.4 62.6, 62.5 59.1, 59.1 46.4, 46.2 66.1, 66.0 60.1, 60.3 49.6, 49.5 68.0, 67.6 61.7, 62.0 51.9, 51.8 89.5, 89.7 81.6, 82.3 79.7, 79.9 89.6, 90.1 80.2, 81.6 79.3, 80.1 90.1, 91.3 80.1, 83.0 79.8, 82.0 86.5, 87.2 75.2, 76.5 73.9, 74.9 86.5, 87.4 75.3, 76.9 73.9, 75.4 85.1, 86.1 75.0, 77.2 73.4, 75.5 75.7, 78.0 49.8, 53.4 53.8, 58.1 81.0, 83.4 52.9, 57.7 58.8, 63.6 81.0, 83.9 51.5, 57.9 57.9, 64.2 80.4, 80.8 68.9, 70.0 64.7, 65.3 83.1, 84.2 69.5, 72.0 67.4, 69.4 83.1, 84.3 69.4, 72.0 67.4, 69.4 47.0, 46.9 51.4, 51.7 37.3, 37.4 46.9, 46.8 51.6, 51.8 38.0, 37.9 47.8, 47.9 51.6, 52.0 38.2, 38.4

ECI-PP (Ours) 45.6, 45.3 45.0, 45.5 29.7, 29.8 49.1, 49.3 47.1, 47.7 32.1, 32.4 48.6, 48.8 48.1, 48.5 32.9, 32.9 82.8, 82.5 76.0, 75.9 70.8, 70.7 84.0, 84.0 77.0, 77.1 72.6, 72.6 81.6, 82.1 77.8, 77.8 73.1, 73.0 60.1, 59.2 62.7, 61.9 49.5, 48.4 62.0, 61.1 64.3, 63.8 51.0, 51.0 65.3, 64.8 66.0, 65.6 53.4, 52.8 91.5, 91.8 86.9, 87.3 84.1, 84.4 91.9, 92.1 87.2, 87.8 84.6, 85.0 92.4, 92.6 87.7, 88.1 85.4, 85.7 80.9, 81.5 75.7, 77.1 71.9, 73.2 86.1, 86.4 79.8, 80.5 77.8, 78.3 84.7, 85.1 79.9, 80.7 77.1, 77.5 85.0, 86.3 62.0, 65.0 68.3, 71.1 87.3, 88.5 63.8, 67.2 71.3, 74.3 88.3, 89.6 65.4, 68.7 72.9, 76.0 85.7, 85.9 75.7, 76.3 72.2, 72.5 87.4, 87.5 77.5, 78.1 75.1, 75.4 87.9, 87.7 78.3, 78.5 75.9, 75.8 55.3, 55.2 56.8, 56.9 45.6, 45.4 55.2, 55.0 58.4, 58.6 46.8, 46.8 53.4, 53.3 58.9, 59.0 46.2, 46.1

is often strongest in ARI. Thus, ECI-PP corrects much of the weakness of plain ProbPair/Weighted ProbPair on this benchmark, while the ACC results indicate that severe class imbalance remains a limitation for the probabilistic formulation. D.2

Comparison without pretraining

Table 9 repeats the single-expert comparison without the autoencoder pretraining stage for pretraincapable methods, denoted by † in the table: CIDEC† , SpherePair† , ProbPair† , Weighted ProbPair† , and ECI-PP† . These methods are trained from random initialization, while VanillaDCC and VolMaxDCC are unchanged because they have no pretraining variant. All other settings match Table 7: lv0.01 supervision, corruption probability 0.3, and 3k/6k/9k constraints. The main effect of removing pretraining is not a collapse of ECI-PP† , but a sharper separation from the other pretrain-capable methods. ECI-PP† ranks first in 68/72 test entries and within the top two in 71/72, compared with 59/72 and 66/72 in the pre-trained comparison. It is also best in every entry on seven of the eight datasets; the only non-best entries occur on RCV1-10 ACC/ARI at 6k/9k. This indicates that ECI-PP† is less dependent on a favorable autoencoder initialization than the compared deep baselines. The contrast with SpherePair† is also clearer without pretraining. ECI-PP† outperforms SpherePair† in 68/72 entries, compared with 62/72 in the pre-trained setting, and achieves the best NMI in all cases. Its absolute NMI margins over SpherePair† range from about 3–12%, with visible gains on FMNIST, MNIST, and STL10. This suggests that the Estimator–Corrector–Integrator structure helps form a more reliable grouping structure when the representation is learned from scratch. The ProbPair variants show a more mixed response to random initialization. Weighted ProbPair† still improves over ProbPair† in most entries, so information-based weighting remains useful. However, Weighted ProbPair† no longer compares as consistently with SpherePair† : it beats SpherePair† in only 33/72 entries, down from 53/72 with pretraining. By contrast, ECI-PP† improves over Weighted ProbPair† in all entries, showing that weighting alone is not sufficient when the representation starts from a weaker initialization. The remaining caveat is again the severely imbalanced RCV1-10 benchmark. SpherePair† keeps stronger ACC at 9k and stronger ARI at 6k/9k, consistent with the robustness of its angular geometry under severe imbalance. Nevertheless, ECI-PP† gives the best NMI at every budget and substantially improves over ProbPair† /Weighted ProbPair† , indicating stronger grouping consistency despite the remaining imbalance-related ACC/ARI limitation. D.3

Comparison with expert-aware baseline extensions

Table 10 compares ECI-PP with SPEA and WPPEA under the multi3 setting, where SPEA and WPPEA denote our expert-aware ensemble extensions of SpherePair and Weighted ProbPair, respectively. As described in Appendix C.4, these extensions instantiate multiple expert-weighted members for a baseline and aggregate their partitions through ensemble clustering. The expert-weighting parameter α is selected on the reserved validation split, with the selected values reported in Table 6. Thus, the comparison should be read as an auxiliary reference rather than a native baseline comparison: SPEA and WPPEA use expert identities, but they also introduce validation tuning and ensemble aggregation, making them inherently uncomparable with the native ECI-PP protocol in a strict sense. The extensions substantially strengthen the corresponding baselines, confirming that expert identities carry useful information in the multi-expert setting. WPPEA benefits clearly on datasets such as MNIST, FMNIST, and STL10, while SPEA gains competitiveness on Reuters and RCV1-10. However, these gains conflate two effects: using expert identities and ensembling multiple expert-weighted members. For this reason, the results are best interpreted as stronger expert-aware references, not as fully symmetric replacements for the native baselines in Table 1. Even against these stronger references, ECI-PP remains the most reliable method overall. It obtains the best result in 55/72 test entries and ranks within the top two in 68/72. The advantage is clearest in NMI, where ECI-PP is best in 23/24 cases, indicating that its expert-conditioned design most consistently improves grouping structure. Dataset-wise, ECI-PP is best in all entries on CIFAR100, ImageNet10, Reuters, and STL10, and remains strongest in most CIFAR10 entries. The main losses occur on FMNIST and MNIST, mostly on ACC/ARI, where WPPEA is particularly strong, and 29

Table 9: Comparative performance (%) (ACC, NMI, ARI) across datasets for methods under the lv0.01 setting with corruption probability 0.3 and 3k/6k/9k constraints. † indicates models without pretraining. Blue and black represent training and test results, respectively. Best results are in bold, and second-best are underlined.

ACC 3k NMI ARI CIFAR100 ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI CIFAR10 ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI FMNIST ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI ImageNet10 ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI MNIST ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI REUTERS ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI STL10 ACC 6k NMI ARI ACC 9k NMI ARI ACC 3k NMI ARI RCV1-10 ACC 6k NMI ARI ACC 9k NMI ARI

VanillaDCC 14.7, 14.7 11.2, 11.8 4.0, 4.1 16.5, 16.6 12.6, 13.5 5.4, 5.6 16.8, 17.0 13.5, 14.6 6.0, 6.3 34.6, 34.6 27.9, 28.1 20.2, 20.3 35.5, 35.7 27.5, 28.3 20.7, 21.2 38.3, 38.7 29.9, 31.2 23.2, 24.0 36.6, 36.5 31.1, 31.1 19.7, 19.5 38.6, 38.5 30.9, 31.3 20.7, 20.9 39.8, 40.2 30.1, 31.3 21.5, 22.2 35.4, 37.3 27.7, 30.8 19.1, 20.8 40.6, 42.0 30.6, 35.2 23.2, 26.0 39.0, 41.5 27.6, 33.7 21.1, 25.2 25.9, 26.1 16.0, 17.0 10.4, 10.8 33.3, 33.9 24.2, 25.8 18.4, 19.4 38.9, 40.3 29.5, 32.2 23.1, 24.8 53.2, 54.0 15.1, 17.9 17.0, 18.0 66.0, 70.6 27.7, 36.3 34.0, 42.6 69.7, 77.0 30.5, 43.3 36.8, 50.4 30.9, 31.5 21.1, 23.4 14.1, 15.0 30.4, 31.6 19.5, 22.2 14.3, 15.7 35.5, 37.8 22.9, 27.6 17.1, 19.8 40.0, 40.1 12.3, 12.4 16.4, 16.5 52.4, 52.6 21.8, 22.2 28.0, 28.2 46.2, 46.4 15.5, 15.9 19.8, 20.1

VolMaxDCC 42.8, 42.5 42.5, 42.9 26.2, 26.0 45.4, 45.3 43.9, 44.5 28.2, 28.2 46.1, 45.9 45.6, 46.1 30.4, 30.5 79.9, 79.8 69.3, 69.4 64.4, 64.4 84.7, 84.6 73.6, 73.7 71.0, 70.9 82.5, 82.5 71.9, 72.2 68.3, 68.3 63.3, 62.5 55.5, 55.2 44.4, 43.6 66.2, 65.1 56.7, 56.3 46.6, 45.5 66.5, 65.5 57.2, 56.8 46.4, 45.2 86.7, 86.9 78.8, 79.4 74.1, 73.9 87.2, 87.5 78.3, 79.3 75.1, 75.6 87.4, 88.1 77.8, 79.2 75.2, 76.1 65.6, 66.4 52.7, 54.1 47.8, 48.9 73.4, 74.4 58.4, 60.5 55.2, 56.8 75.6, 76.9 59.3, 61.6 56.5, 58.7 62.8, 63.2 25.6, 28.0 31.2, 32.4 72.5, 76.8 36.0, 44.9 42.4, 50.8 71.7, 76.8 34.3, 43.4 40.9, 50.4 72.8, 73.3 64.8, 66.5 56.3, 57.0 76.8, 77.5 65.8, 67.7 60.0, 61.3 79.0, 80.1 66.7, 69.3 62.8, 64.7 46.3, 46.4 21.5, 21.9 21.2, 21.4 62.1, 62.4 36.0, 36.7 42.9, 43.3 61.3, 61.8 35.7, 36.7 44.7, 45.4

CIDEC†

SpherePair† ProbPair†

14.4, 14.2 11.2, 11.5 3.7, 3.5 12.3, 12.2 8.6, 9.4 2.7, 2.7 12.0, 11.9 8.1, 9.0 2.9, 2.9 39.0, 39.0 29.1, 29.6 17.8, 17.7 35.7, 35.9 27.4, 29.1 18.6, 19.0 31.9, 32.1 23.9, 25.8 16.7, 17.3 36.3, 35.6 29.2, 28.9 15.7, 14.9 34.1, 33.6 27.5, 27.8 16.4, 16.1 34.7, 34.3 27.1, 28.0 15.9, 15.7 50.8, 52.0 42.8, 46.5 30.1, 31.5 32.0, 33.6 23.3, 29.4 16.1, 19.4 31.9, 34.5 23.9, 32.4 17.1, 22.1 41.6, 41.0 35.1, 35.6 19.6, 18.8 36.4, 36.3 29.6, 31.0 18.2, 18.0 36.0, 36.8 28.3, 30.9 19.1, 19.8 58.1, 56.0 26.2, 30.4 29.6, 29.7 45.1, 46.1 9.9, 14.1 11.3, 14.3 52.9, 54.5 16.8, 25.5 19.6, 25.5 49.2, 50.3 38.2, 41.9 27.0, 28.5 35.1, 36.8 26.8, 31.7 18.6, 21.3 31.8, 33.4 23.3, 28.8 16.4, 19.3 39.4, 37.4 21.6, 21.4 18.3, 19.6 38.5, 37.0 15.1, 14.9 16.2, 16.1 41.6, 42.0 14.4, 14.9 15.8, 16.4

44.0, 43.8 41.9, 42.4 27.5, 27.5 47.6, 47.8 42.9, 43.9 30.5, 30.9 48.9, 49.0 42.5, 43.6 31.0, 31.4 80.4, 80.7 70.8, 71.1 67.2, 67.4 84.2, 84.5 71.6, 72.5 70.2, 70.8 83.3, 84.0 70.1, 71.7 68.9, 70.2 64.6, 63.6 57.7, 57.2 48.3, 47.3 64.7, 63.7 54.1, 54.1 45.9, 45.4 67.6, 66.9 55.5, 55.9 48.5, 48.0 88.5, 89.8 78.8, 81.6 77.5, 79.7 87.0, 89.4 74.9, 80.2 74.3, 78.7 85.2, 88.9 71.1, 79.1 70.9, 77.9 72.5, 73.2 61.5, 63.0 56.4, 57.7 76.8, 77.9 63.3, 65.2 60.1, 61.7 80.1, 81.3 67.0, 69.6 65.5, 67.7 76.5, 78.8 49.5, 54.7 54.0, 59.2 78.4, 81.2 50.3, 57.1 54.5, 61.1 77.5, 82.2 48.7, 57.0 52.6, 61.4 80.9, 82.9 67.2, 71.0 64.3, 67.8 78.7, 81.7 62.6, 68.4 60.4, 65.6 76.9, 80.8 59.7, 66.9 57.6, 64.1 52.1, 52.2 51.5, 51.7 42.6, 42.7 62.0, 61.9 55.8, 56.2 52.8, 52.9 66.6, 66.8 58.0, 58.8 57.8, 58.4

43.2, 43.0 39.7, 40.4 25.7, 25.8 44.7, 44.9 39.9, 40.8 26.9, 27.3 47.4, 47.5 40.9, 42.0 28.9, 29.2 83.0, 83.1 71.8, 72.1 68.6, 68.7 84.1, 84.4 73.1, 74.0 70.8, 71.2 85.2, 85.6 73.9, 74.8 71.9, 72.5 58.0, 57.3 51.8, 51.4 40.1, 39.3 65.8, 65.0 55.8, 55.7 46.6, 45.8 69.6, 68.8 58.2, 58.6 49.7, 49.2 87.3, 87.6 78.4, 79.3 75.7, 76.0 88.8, 89.8 79.2, 81.5 77.7, 79.4 88.4, 90.1 78.3, 82.2 76.8, 79.9 67.6, 68.8 54.0, 55.9 48.7, 50.4 76.6, 77.5 62.3, 64.2 59.3, 60.9 82.2, 83.6 67.5, 70.2 66.4, 68.9 69.4, 71.5 44.4, 48.8 46.7, 51.3 75.6, 78.2 48.1, 53.5 52.6, 57.9 76.1, 80.1 49.5, 57.5 52.3, 60.0 80.3, 81.1 66.9, 69.2 64.0, 65.5 80.3, 81.9 66.3, 69.6 63.8, 66.4 80.3, 82.8 66.0, 70.8 63.6, 67.7 45.2, 45.1 42.3, 42.5 34.5, 34.5 47.0, 47.1 44.1, 44.5 37.5, 37.6 45.7, 45.5 43.6, 44.0 35.6, 35.6

Weighted ProbPair† 42.0, 41.9 38.9, 39.5 24.9, 25.0 45.2, 45.5 40.2, 41.0 27.1, 27.6 47.8, 48.2 42.1, 43.1 30.0, 30.4 82.8, 82.9 73.3, 73.6 70.3, 70.4 85.1, 85.3 74.3, 74.7 72.4, 72.6 86.1, 86.4 75.0, 75.8 73.5, 73.9 61.9, 61.3 53.2, 53.2 43.0, 42.6 66.8, 66.0 56.1, 56.2 47.3, 46.6 71.0, 69.9 59.2, 59.3 51.0, 50.1 88.9, 89.1 80.6, 81.4 77.9, 78.2 89.6, 90.1 80.9, 82.3 79.3, 79.9 88.9, 90.0 79.4, 82.5 77.8, 79.8 70.1, 70.7 57.2, 58.4 52.1, 53.1 82.4, 83.2 67.5, 69.1 66.5, 68.0 86.0, 87.2 72.2, 74.6 72.3, 74.6 71.6, 73.9 46.7, 51.2 48.7, 53.3 74.5, 77.1 48.1, 53.5 51.2, 56.3 75.3, 79.3 47.7, 54.6 51.3, 58.6 80.8, 81.5 67.5, 69.2 64.7, 66.0 81.5, 82.7 67.9, 70.7 65.7, 67.8 81.3, 82.7 67.1, 70.6 65.3, 67.8 44.6, 44.6 43.9, 44.1 35.0, 35.0 48.2, 48.3 47.4, 47.4 39.0, 38.9 49.8, 49.9 49.7, 49.9 42.9, 43.0

ECI-PP† (Ours) 46.6, 46.5 45.5, 46.2 30.5, 30.8 49.3, 49.1 47.8, 48.3 33.9, 33.9 52.2, 52.5 49.3, 50.1 35.9, 36.3 84.5, 84.4 76.0, 75.9 72.3, 72.1 87.0, 87.0 78.5, 78.5 75.8, 75.7 87.7, 87.7 79.1, 79.0 76.8, 76.7 67.3, 66.6 63.1, 62.4 51.8, 50.7 69.0, 67.7 64.2, 63.4 53.1, 51.6 72.4, 71.2 65.6, 64.7 55.5, 54.0 92.5, 92.6 87.9, 88.2 85.5, 85.6 92.5, 92.8 87.8, 88.4 85.5, 86.0 92.6, 93.1 87.9, 88.5 85.6, 86.2 80.0, 80.6 68.6, 69.6 64.9, 65.9 87.4, 88.0 75.6, 76.8 74.9, 76.1 88.7, 89.2 77.4, 78.5 77.2, 78.2 80.0, 80.7 56.3, 57.9 61.6, 63.2 83.7, 85.2 59.5, 62.7 65.0, 68.1 81.9, 83.1 57.6, 60.2 62.0, 64.7 84.2, 84.7 74.5, 75.1 71.0, 71.6 85.2, 85.7 75.2, 76.2 72.1, 73.0 85.3, 85.7 75.6, 76.3 72.4, 73.0 55.5, 55.7 54.7, 55.1 47.6, 47.9 59.7, 59.9 58.5, 58.8 51.5, 51.7 64.1, 64.1 61.4, 61.6 56.0, 56.0

on RCV1-10 ACC/ARI, where the angular geometry of SPEA remains advantageous under severe imbalance. Overall, expert-aware ensembling makes strong baselines substantially more competitive, 30

Table 10: Comparative performance (%) (ACC, NMI, ARI) across datasets for ECI-PP, SPEA , and WPPEA under the multi3 setting, with corruption probability 0.3 and 3k/6k/9k total constraints. SPEA and WPPEA denote the expert-aware ensemble extensions of SpherePair and Weighted ProbPair, respectively. Blue and black represent training and test results, respectively. Best results are in bold, and second-best are underlined. ACC

3k NMI

ARI

ACC

6k NMI

ARI

ACC

9k NMI

ARI

CIFAR100

SPEA 42.2, 43.2 41.7, 42.5 26.6, 27.0 WPPEA 43.1, 43.4 41.4, 42.2 26.4, 26.5 ECI-PP 45.6, 45.3 45.0, 45.5 29.7, 29.8

41.0, 40.9 39.7, 40.9 25.5, 25.8 41.5, 41.4 40.4, 41.3 25.8, 26.1 49.1, 49.3 47.1, 47.7 32.1, 32.4

41.9, 42.3 39.2, 40.5 26.0, 27.1 46.4, 46.4 42.9, 43.8 29.8, 29.9 48.6, 48.8 48.1, 48.5 32.9, 32.9

CIFAR10

SPEA 76.6, 77.1 70.7, 71.5 64.0, 64.8 WPPEA 82.6, 82.5 74.1, 74.2 69.4, 69.2 ECI-PP 82.8, 82.5 76.0, 75.9 70.8, 70.7

75.5, 75.9 67.9, 69.4 62.4, 63.5 81.2, 81.2 74.6, 74.9 69.6, 69.6 84.0, 84.0 77.0, 77.1 72.6, 72.6

74.1, 75.2 67.0, 69.0 61.3, 62.9 82.5, 82.8 74.2, 74.5 70.0, 70.3 81.6, 82.1 77.8, 77.8 73.1, 73.0

FMNIST

SPEA 63.2, 61.6 60.3, 59.7 48.4, 46.9 WPPEA 64.5, 64.3 59.7, 59.8 47.9, 47.7 ECI-PP 60.1, 59.2 62.7, 61.9 49.5, 48.4

63.5, 62.5 59.4, 59.9 48.5, 48.3 68.0, 68.3 62.1, 62.4 52.0, 52.0 62.0, 61.1 64.3, 63.8 51.0, 51.0

62.5, 62.5 58.6, 60.2 49.0, 49.4 71.8, 71.7 64.6, 64.8 55.2, 55.1 65.3, 64.8 66.0, 65.6 53.4, 52.8

SPEA 81.7, 85.3 78.0, 79.8 71.8, 75.2 ImageNet10 WPPEA 90.5, 90.8 83.8, 84.2 81.8, 82.0 ECI-PP 91.5, 91.8 86.9, 87.3 84.1, 84.4

82.1, 86.6 77.8, 79.6 72.8, 75.8 90.9, 91.6 83.4, 84.8 82.2, 83.4 91.9, 92.1 87.2, 87.8 84.6, 85.0

81.2, 81.4 75.3, 76.7 70.6, 71.3 91.2, 92.1 82.4, 84.7 82.2, 83.8 92.4, 92.6 87.7, 88.1 85.4, 85.7

MNIST

80.5, 81.1 75.2, 76.7 70.7, 71.7 SPEA WPPEA 87.9, 88.7 77.2, 78.5 76.2, 77.4 ECI-PP 80.9, 81.5 75.7, 77.1 71.9, 73.2

78.6, 79.6 72.7, 75.5 67.9, 70.2 85.8, 86.4 77.4, 79.0 75.5, 76.8 86.1, 86.4 79.8, 80.5 77.8, 78.3

77.5, 78.1 70.2, 73.7 65.7, 67.8 89.2, 90.4 78.1, 80.4 78.2, 80.3 84.7, 85.1 79.9, 80.7 77.1, 77.5

REUTERS

SPEA 75.8, 76.1 41.8, 45.7 48.0, 50.2 WPPEA 82.9, 83.3 55.4, 58.3 62.4, 64.5 ECI-PP 85.0, 86.3 62.0, 65.0 68.3, 71.1

72.0, 77.0 35.3, 43.8 40.9, 49.8 81.4, 83.7 53.6, 58.4 59.5, 64.4 87.3, 88.5 63.8, 67.2 71.3, 74.3

68.4, 77.0 30.6, 43.1 34.7, 49.6 77.7, 80.9 50.3, 56.0 54.8, 61.5 88.3, 89.6 65.4, 68.7 72.9, 76.0

STL10

SPEA 74.2, 76.7 66.5, 68.1 59.7, 61.1 WPPEA 84.0, 84.5 72.4, 73.8 69.0, 70.0 ECI-PP 85.7, 85.9 75.7, 76.3 72.2, 72.5

73.0, 75.4 64.0, 67.3 57.4, 60.1 85.2, 85.6 73.7, 75.2 71.3, 72.0 87.4, 87.5 77.5, 78.1 75.1, 75.4

72.7, 72.9 62.5, 66.2 56.7, 58.7 85.1, 86.1 73.3, 75.4 71.1, 72.7 87.9, 87.7 78.3, 78.5 75.9, 75.8

RCV1-10

SPEA 60.2, 60.9 52.6, 53.1 48.9, 49.5 WPPEA 52.5, 47.6 52.6, 52.5 41.9, 38.3 ECI-PP 55.3, 55.2 56.8, 56.9 45.6, 45.4

68.2, 63.5 54.8, 53.8 55.7, 51.4 54.6, 52.6 53.6, 53.7 45.0, 42.8 55.2, 55.0 58.4, 58.6 46.8, 46.8

71.2, 67.7 54.8, 54.4 58.8, 55.9 56.5, 58.4 53.5, 54.2 47.0, 49.2 53.4, 53.3 58.9, 59.0 46.2, 46.1

but ECI-PP still provides a more direct and generally stronger way to use expert identities without relying on a validation-tuned ensemble wrapper. D.4

Robustness across supervision conditions

Figs. 5 to 7 report the full ACC, NMI, and ARI results corresponding to the robustness study in the main text. All three sweeps use a fixed budget of 9k constraints, and the central setting in each sweep, namely lv0.01 expert quality, corruption probability 0.3, and multi3, matches the default setting used in Table 1. Thus, these figures also cover the main-comparison results in Table 1, while additionally reporting the standard deviations omitted from the table for space. For the multi-expert sweep, we further include SPEA and WPPEA , the expert-aware ensemble extensions introduced in Appendix D.3, as stronger references that also exploit expert identities. Effects of supervision factors. Across the three sweeps, the results largely follow the expected supervision-quality ordering. In the single-expert sweep, stronger experts generally yield better clustering quality; in the corruption sweep, increasing corruption leads to clear degradation. In the multi-expert sweep, moving from multi2 to multi10 also tends to help the stronger methods, because the protocol progressively reduces expert blind spots and increases average expert accuracy, as quantified in Table 4. Since the total constraint budget is fixed throughout, the performance trends mainly reflect changes in supervision quality and heterogeneity rather than in the amount of supervision. Baseline behavior. The non-ECI baselines show a clear hierarchy as supervision becomes less reliable. Plain end-to-end methods such as VanillaDCC and CIDEC often nearly collapse under lowquality or heterogeneous supervision, especially in NMI and ARI, suggesting that cluster-assignment training is vulnerable when pairwise supervision is inconsistent. VolMaxDCC is usually more resilient, consistent with the benefit of explicit noise modeling. SpherePair and the ProbPair-family baselines are substantially stronger than the plain end-to-end methods, reflecting the robustness of 31

(A) Varying single-expert quality

Test ACC - CIFAR100-20 Test ACC - CIFAR10

0.8

Test ACC - FMNIST Test ACC - IMAGENET10

0.4

Test ACC - MNIST

Test ACC - REUTERS

0.8

0.8

0.6

0.6

0.4

0.4

lv0.01

lv0.1

0.2

0.4 lv0.001

lv0.01

lv0.1

lv0.001

lv0.01

lv0.1

lv0.001

lv0.01

lv0.1

lv0.001

lv0.01

lv0.1

lv0.001

lv0.01

lv0.1

(B) Varying corruption rate

0.6 0.8 0.6

0.4

0.6 0.4

0.2

0.4

(C) Varying multi-expert config

0.1

0.3

0.5

0.1

0.3

0.5

0.1

0.3

0.6

0.6

0.6

0.4

0.4

0.6

0.6

0.4

0.4 0.2

0.5

0.3

0.5

0.1

0.3

lv0.1

0.7

0.8

0.1

lv0.01

0.8

0.8

0.5

lv0.001 0.8

0.8

0.2

0.2

0.4

0.4

0.4

0.2 lv0.001

0.6

0.6 0.6

0.4

0.2

Test ACC - RCV1-10

0.8 0.8

0.6

0.6

Test ACC - STL10 0.8

0.8

0.5

0.2 0.1

0.3

0.5

lv0.001

lv0.01

lv0.1

0.1

0.3

0.5

multi2

multi3

multi10

0.2 0.1

0.3

0.5

0.8 0.8

0.8

0.8

0.4

0.8

0.6

0.6

0.6

0.6

0.4

0.4

0.6

0.6

0.6 0.2

0.8

0.8

0.4

0.4

0.4

0.4 0.4

multi2

multi3

multi10

multi2

VanillaDCC

multi3

multi10

VolMaxDCC

multi2

multi3

CIDEC

multi10

multi2

SpherePair

multi3

multi10

multi2

SpherePair (EA)

multi3

multi10

multi2

ProbPair

multi3

multi10

multi2

Weighted ProbPair

multi3

multi10

Weighted ProbPair (EA)

ECI-PP (Ours)

(B) Varying corruption rate

(A) Varying single-expert quality

Figure 5: Full robustness results in ACC (mean±std over 5 runs) across datasets under varying (A) expert quality, (B) corruption probability, and (C) multi-expert configuration, with 9k constraints. The multi-expert row additionally includes expert-aware SpherePair and Weighted ProbPair extensions. Test NMI - CIFAR100-20 0.4

Test NMI - FMNIST

0.6

Test NMI - IMAGENET10

Test NMI - MNIST

0.2

0.6 0.5

0.6

0.6

0.4

0.4

0.4

lv0.001

lv0.01

lv0.1

0.3 lv0.001

lv0.01

lv0.1

lv0.01

lv0.1

0.6

0.2

0.2 0.3

0.5

lv0.1

0.1

0.3

0.5

lv0.01

lv0.1

0.6

0.6

0.4

0.4

0.2

0.2

Test NMI - RCV1-10

0.0

0.4

lv0.001

lv0.01

lv0.1

0.2

lv0.001

lv0.01

lv0.1

0.1

0.3

0.5

0.4

0.0

lv0.001

lv0.01

lv0.1

0.2

0.2

0.0

0.0

0.1

0.3

0.5

0.1

0.3

0.5

0.4

0.1

0.3

0.5

0.1

0.3

0.4

0.4

0.2

0.2

0.5

0.1

0.3

0.5

multi2

multi3

multi10

0.6 0.6

0.6

0.4

0.4

0.2

0.2

0.0

0.8 0.6

0.6

0.5

0.2

0.8

0.6

0.4

0.6

0.6 0.4

0.4

0.8

0.6

0.6

0.6

0.4

0.7

0.8

0.2

lv0.001 0.8

0.6 0.4

0.4

0.1

lv0.01

0.8

0.6

0.2

lv0.001

Test NMI - STL10 0.8

0.8

0.8 0.4

Test NMI - REUTERS

0.6

0.2 lv0.001

0.8

0.8

0.8

0.4

0.2

0.0

(C) Varying multi-expert config

Test NMI - CIFAR10

0.8

0.4

0.4 0.4

0.2

0.2

0.3 0.0

multi2

multi3

multi10

0.2

VanillaDCC

multi2

multi3

multi10

VolMaxDCC

multi2

multi3

CIDEC

multi10

multi2

SpherePair

multi3

multi10

multi2

SpherePair (EA)

multi3

multi10

0.0

ProbPair

multi2

multi3

multi10

0.2

Weighted ProbPair

multi2

multi3

multi10

0.0

Weighted ProbPair (EA)

ECI-PP (Ours)

(A) Varying single-expert quality

Figure 6: Full robustness results in NMI (mean±std over 5 runs) across datasets under varying (A) expert quality, (B) corruption probability, and (C) multi-expert configuration, with 9k constraints. The multi-expert row additionally includes expert-aware SpherePair and Weighted ProbPair extensions. Test ARI - CIFAR100-20

(B) Varying corruption rate

Test ARI - FMNIST

0.6 0.4

0.1

0.2 lv0.001

lv0.01

lv0.1

0.2 lv0.001

lv0.01

lv0.1

lv0.001

lv0.01

0.6 0.4

0.1

0.2 0.1

0.3

0.5

0.4

lv0.01

lv0.1

0.3

0.5

0.1

0.3

0.5

lv0.001

lv0.01

lv0.1

0.3

0.6

0.6

0.4

0.4

0.2

0.2

0.0

0.0

0.1

0.3

0.5

0.4 0.4

0.1

0.2

0.2 0.0

multi2

multi3

VanillaDCC

multi10

multi2

multi3

VolMaxDCC

multi10

multi2

CIDEC

multi3

0.6

0.6

0.4

0.4

0.2

0.2

multi10

SpherePair

0.4

0.2

0.2 lv0.001

lv0.01

lv0.1

0.4 0.2

lv0.001

lv0.01

lv0.1

multi2

multi3

multi10

SpherePair (EA)

0.0

0.6

0.6

0.4

0.4

0.4

0.2

0.2

0.2 0.1

0.3

0.5

0.8

0.8

0.6

0.2

0.6

0.4

0.0

Test ARI - RCV1-10 0.6

lv0.001

lv0.01

lv0.1

0.1

0.3

0.5

multi2

multi3

multi10

0.8

0.8 0.6

Test ARI - STL10 0.8

0.8

0.2

0.1

0.6

0.2

0.8

0.2

0.6

0.2 lv0.001

Test ARI - REUTERS 0.8

0.4

lv0.1

Test ARI - MNIST

0.8

0.4

0.8

0.3

0.0

0.6

0.4

0.2

0.0

Test ARI - IMAGENET10

0.6 0.8

0.4

(C) Varying multi-expert config

Test ARI - CIFAR10

0.8 0.3

0.1

0.3

0.5

0.0

0.6

0.1

0.3

0.5

0.0

0.8

0.8

0.8

0.6

0.6

0.6

0.4

0.4

0.4

0.2

0.2 0.2 multi2

multi3

ProbPair

multi10

0.0

multi2

multi3

multi10

Weighted ProbPair

multi2

multi3

multi10

0.0

Weighted ProbPair (EA)

ECI-PP (Ours)

Figure 7: Full robustness results in ARI (mean±std over 5 runs) across datasets under varying (A) expert quality, (B) corruption probability, and (C) multi-expert configuration, with 9k constraints. The multi-expert row additionally includes expert-aware SpherePair and Weighted ProbPair extensions.

32

geometric representation learning under noisy supervision. Nevertheless, corruption still causes visible degradation for the stronger baselines, particularly on FMNIST, ImageNet10, and STL10. ECI-PP robustness. Across the supervision sweeps, ECI-PP better preserves clustering quality than the native baselines, especially as expert quality decreases or corruption increases. This advantage is most consistent on NMI and ARI, suggesting that the correction-and-integration pipeline mainly improves the recovered grouping structure. ACC is more competitive and contains more exceptions, especially on FMNIST, MNIST, and RCV1-10, where Weighted ProbPair, SPEA , or WPPEA can sometimes match or exceed ECI-PP. This is consistent with the main-comparison results (Appendix D.1): matched-label accuracy can favor angular or ensemble-based baselines on some datasets, even when ECI-PP remains stronger or near-stronger on grouping-oriented metrics (NMI/ARI). The expert-aware extensions further clarify the role of expert identities: SPEA and WPPEA often improve over their native counterparts in the multi-expert sweep, confirming that expert identity information is useful under heterogeneous supervision. However, these extensions rely on validation-selected expert weighting and ensemble aggregation, whereas ECI-PP uses expertconditioned correction within the learning pipeline. Overall, the full results support the main-text conclusion that ECI-PP better preserves clustering quality when supervision is fallible, heterogeneous, or corrupted. D.5

Held-out diagnostics

Diagnostic protocol. The held-out diagnostics complement the clustering results by tracking the training dynamics of ECI-PP’s internal quantities. Following Appendix C.4, we construct a sampledisjoint diagnostic constraint set C diag from the test split, used only for diagnosis and never for optimization, early stopping, or hyperparameter selection. Benchmark labels are used only to define an external hard co-membership reference, rather than the canonical aleatoric relation R⋆ . All diagnostics use multi3 supervision with limited 3k training constraints in total (1k per expert) and corruption rate 0.3. To reduce the influence of a strong initial representation or a long warm-up phase, we use random initialization without unsupervised pretraining, give the Estimators and Integrator only one warm-up epoch, and then track the next 500 training iterations, each corresponding to one epoch. We report mean±std over 5 runs, recording at each iteration the Estimator out-of-fold relation ŷioof , the corrected relation yicor , and the Integrator relation yiint on C diag . Diagnostic metrics. For a held-out pair i = (ai , bi ) with benchmark labels tai and tbi , define the external hard co-membership reference o⋆i := I[tai = tbi ]. Let Iclean = {i ∈ C diag : ci = 0} be the uncorrupted held-out subset, and define the residual discrepancy after correction as gapi = softclipξ (ŷioof ) − yicor . For any relation score vi ∈ [0, 1] and index set I, we use 1 X Brier(v; I) = (vi − o⋆i )2 . |I| i∈I

cor We report Brierest = Brier(ŷ oof ; C diag ), Brier Pcor,clean = Brier(y ; Iclean ), Brierint = int diag −1 Brier(y ; C ), and E[gapi,clean ] = |Iclean | i∈Iclean gapi . For reliability-aware screening, gapi is used to rank held-out constraints, and we report AUCcorrupt and APcorrupt against the known corruption indicator ci . These scores measure alignment with corruption rather than pure corruption estimation, since the reliability signal may also reflect difficult clean pairs, residual epistemic variation, and imperfect estimation/correction; nevertheless, corrupted records are expected to rank higher on average. As a reference, we also replace the multi3 expert-generated judgments by o⋆i before applying the same 3k budget and corruption rate 0.3, thereby removing expert epistemic uncertainty from the clean supervision.

Evolution of internal estimates. The full diagnostic curves are shown in Figs. 8 to 15. Panels (A–C) track the Estimator, Corrector, and Integrator relation scores through their Brier errors against 33

(B) Briercor,clean ↓ (multi3)

0.15

0.10

0.20

100 200 300 400 500 iter.

0

100 200 300 400 500 iter.

(F) APcorrupt ↑ (multi3)

AUCcorrupt baseline = 0.5

0.50 0

0.50 0.40 APcorrupt baseline = 0.3

0.30

100 200 300 400 500 iter.

0

0

100 200 300 400 500 iter.

(G) AUCcorrupt ↑ (oracle)

0.80

AUCcorrupt

0.60

0.20

0.10

0.60

APcorrupt

AUCcorrupt

(E) AUCcorrupt ↑ (multi3) 0.70

0.30

0.30

0.10 0.05

0

  E gapi,clean

0.20

0.40

Brierint

Brierest

0.30

  (D) E gapi,clean ↓ (multi3)

(C) Brierint ↓ (multi3)

APcorrupt

(A) Brierest ↓ (multi3) Briercor,clean

0.40

0.70 0.60 AUCcorrupt baseline = 0.5

0.50

100 200 300 400 500 iter.

0

0

100 200 300 400 500 iter.

(H) APcorrupt ↑ (oracle)

0.70 0.60 0.50 0.40

APcorrupt baseline = 0.3

0.30

100 200 300 400 500 iter.

0

100 200 300 400 500 iter.

Figure 8: CIFAR100-20 held-out diagnostic evolution under multi3 with corruption rate 0.3 (mean±std over 5 runs). Same panel definitions as in Fig. 3.

0

0

100 200 300 400 500 iter.

(F) APcorrupt ↑ (multi3) 0.60

APcorrupt

0.60 AUCcorrupt baseline = 0.5

0.50 0

0.20

0.10

(E) AUCcorrupt ↑ (multi3) AUCcorrupt

0.20

0.10

100 200 300 400 500 iter.

0.70

0.25

0.30

0.50 0.40 APcorrupt baseline = 0.3

0.30

100 200 300 400 500 iter.

0

0

0.15 100 200 300 400 500 iter. 0

(G) AUCcorrupt ↑ (oracle)

0.80

APcorrupt

0.10

0.15

  (D) E gapi,clean ↓ (multi3)

(C) Brierint ↓ (multi3)

Brierint

0.20

AUCcorrupt

Brierest

Briercor,clean

0.30

(B) Briercor,clean ↓ (multi3)

  E gapi,clean

(A) Brierest ↓ (multi3)

0.70 0.60 AUCcorrupt baseline = 0.5

0.50

100 200 300 400 500 iter.

0

100 200 300 400 500 iter.

(H) APcorrupt ↑ (oracle)

0.70 0.60 0.50 0.40

APcorrupt baseline = 0.3

0.30

100 200 300 400 500 iter.

0

100 200 300 400 500 iter.

Figure 9: CIFAR10 held-out diagnostic evolution under multi3 with corruption rate 0.3 (mean±std over 5 runs). Same panel definitions as in Fig. 3.

0.20

100 200 300 400 500 iter.

0.70

APcorrupt

AUCcorrupt

0.30

0.60 AUCcorrupt baseline = 0.5

0.50 0

0.10 0

100 200 300 400 500 iter.

(F) APcorrupt ↑ (multi3)

0.50 0.40 APcorrupt baseline = 0.3

0.30 0

0

100 200 300 400 500 iter.

(G) AUCcorrupt ↑ (oracle)

0.70 0.60 AUCcorrupt baseline = 0.5

0.50

100 200 300 400 500 iter.

0

0

100 200 300 400 500 iter.

(H) APcorrupt ↑ (oracle)

0.80

0.60

100 200 300 400 500 iter.

0.20

0.20

AUCcorrupt

0

(E) AUCcorrupt ↑ (multi3)

0.25

0.40

0.15 0.10

0.10

0.50

APcorrupt

0.30

0.20

Brierint

Brierest

0.40

  (D) E gapi,clean ↓ (multi3)

(C) Brierint ↓ (multi3)

  E gapi,clean

(B) Briercor,clean ↓ (multi3) Briercor,clean

0.50

(A) Brierest ↓ (multi3)

0.60 0.50 0.40 APcorrupt baseline = 0.3

0.30

100 200 300 400 500 iter.

0

100 200 300 400 500 iter.

Figure 10: FMNIST held-out diagnostic evolution under multi3 with corruption rate 0.3 (mean±std over 5 runs). Same panel definitions as in Fig. 3.

100 200 300 400 500 iter.

0

100 200 300 400 500 iter.

(F) APcorrupt ↑ (multi3) 0.70

APcorrupt

AUCcorrupt

0.80 0.70 0.60

AUCcorrupt baseline = 0.5

0.50 0

0.60 0.50 0.40

APcorrupt baseline = 0.3

0.30

100 200 300 400 500 iter.

0.15

0.10

0

0

100 200 300 400 500 iter.

(G) AUCcorrupt ↑ (oracle)

0.90 0.80

APcorrupt

0

(E) AUCcorrupt ↑ (multi3)

0.25 0.20

0.20

0.05

  (D) E gapi,clean ↓ (multi3)

(C) Brierint ↓ (multi3)

Brierint

0.10

0.10

AUCcorrupt

Brierest

0.20

0.30

  E gapi,clean

(B) Briercor,clean ↓ (multi3) Briercor,clean

(A) Brierest ↓ (multi3)

0.70 0.60 AUCcorrupt baseline = 0.5

0.50

100 200 300 400 500 iter.

0

0

0.80 0.70 0.60 0.50 0.40 0.30

100 200 300 400 500 iter.

100 200 300 400 500 iter.

(H) APcorrupt ↑ (oracle)

APcorrupt baseline = 0.3

0

100 200 300 400 500 iter.

Figure 11: ImageNet10 held-out diagnostic evolution under multi3 with corruption rate 0.3 (mean±std over 5 runs). Same panel definitions as in Fig. 3. 34

0.30 0.20

0.35 0.30

0.30

0.25

0.20

0.10

  (D) E gapi,clean ↓ (multi3)

(C) Brierint ↓ (multi3)

0.40

0.15

0.10

0.20

0.10

(E) AUCcorrupt ↑ (multi3) APcorrupt AUCcorrupt baseline = 0.5

0.50 0

100 200 300 400 500 iter.

(F) APcorrupt ↑ (multi3)

0.70 0.60

0

0.50 0.40 APcorrupt baseline = 0.3

0.30

100 200 300 400 500 iter.

0

0

100 200 300 400 500 iter.

(G) AUCcorrupt ↑ (oracle)

0.60 AUCcorrupt baseline = 0.5

0.50

100 200 300 400 500 iter.

0

100 200 300 400 500 iter.

(H) APcorrupt ↑ (oracle) 0.60

0.70

APcorrupt

100 200 300 400 500 iter.

AUCcorrupt

0

AUCcorrupt

0.50

Brierint

Brierest

Briercor,clean

0.40

(B) Briercor,clean ↓ (multi3)

  E gapi,clean

(A) Brierest ↓ (multi3)

0

0.50 0.40 APcorrupt baseline = 0.3

0.30

100 200 300 400 500 iter.

0

100 200 300 400 500 iter.

Figure 12: MNIST held-out diagnostic evolution under multi3 with corruption rate 0.3 (mean±std over 5 runs). Same panel definitions as in Fig. 3. (B) Briercor,clean ↓ (multi3)

0.30

0

0.60 0.55 AUCcorrupt baseline = 0.5

0

APcorrupt

AUCcorrupt

0.65

0.50

0.25

0.20 0

100 200 300 400 500 iter.

(F) APcorrupt ↑ (multi3)

0.35 APcorrupt baseline = 0.3

0.30

0

0.70 0.60

100 200 300 400 500 iter.

0

0

100 200 300 400 500 iter.

(H) APcorrupt ↑ (oracle)

0.60 0.50 0.40

APcorrupt baseline = 0.3

0.30

AUCcorrupt baseline = 0.5

0.50 0

100 200 300 400 500 iter.

(G) AUCcorrupt ↑ (oracle)

0.40

100 200 300 400 500 iter.

  E gapi,clean

0.20

100 200 300 400 500 iter.

(E) AUCcorrupt ↑ (multi3)

0.30

APcorrupt

0.20

0.30

Brierint

0.30

  (D) E gapi,clean ↓ (multi3)

(C) Brierint ↓ (multi3) 0.40

0.25

AUCcorrupt

Briercor,clean

Brierest

0.40

(A) Brierest ↓ (multi3)

100 200 300 400 500 iter.

0

100 200 300 400 500 iter.

Figure 13: Reuters held-out diagnostic evolution under multi3 with corruption rate 0.3 (mean±std over 5 runs). Same panel definitions as in Fig. 3. (B) Briercor,clean ↓ (multi3)

0

0.10

APcorrupt

AUCcorrupt

(E) AUCcorrupt ↑ (multi3)

0.60 AUCcorrupt baseline = 0.5

0.50 0

0.20

0.20

0.10

100 200 300 400 500 iter.

0.70

0.25

0.30

0

100 200 300 400 500 iter.

(F) APcorrupt ↑ (multi3) 0.60 0.50 0.40 APcorrupt baseline = 0.3

0.30

100 200 300 400 500 iter.

0

0.15 0

100 200 300 400 500 iter.

(G) AUCcorrupt ↑ (oracle) 0.80

APcorrupt

0.10

  (D) E gapi,clean ↓ (multi3)

(C) Brierint ↓ (multi3)

Brierint

0.20

0.15

AUCcorrupt

Brierest

0.30

0.40

  E gapi,clean

(A) Brierest ↓ (multi3) Briercor,clean

0.40

0.70 0.60 AUCcorrupt baseline = 0.5

0.50

100 200 300 400 500 iter.

0

0

100 200 300 400 500 iter.

(H) APcorrupt ↑ (oracle)

0.70 0.60 0.50 0.40

APcorrupt baseline = 0.3

0.30

100 200 300 400 500 iter.

0

100 200 300 400 500 iter.

Figure 14: STL10 held-out diagnostic evolution under multi3 with corruption rate 0.3 (mean±std over 5 runs). Same panel definitions as in Fig. 3.

0.20 0

AUCcorrupt baseline = 0.5

APcorrupt

AUCcorrupt

0.60

0

0

100 200 300 400 500 iter.

(F) APcorrupt ↑ (multi3)

0.35 APcorrupt baseline = 0.3

0.30

100 200 300 400 500 iter.

0

0.70 0.60

AUCcorrupt baseline = 0.5

0.50 0.40

0

100 200 300 400 500 iter.

(G) AUCcorrupt ↑ (oracle)

0.40

0.25

0.40

0.30

0.20

0.15

0.40 0.35

0.40 0.30

100 200 300 400 500 iter.

(E) AUCcorrupt ↑ (multi3)

0.50

0.50

0.20

  (D) E gapi,clean ↓ (multi3)

(C) Brierint ↓ (multi3)

  E gapi,clean

0.30

0.60

Brierint

0.40

AUCcorrupt

Brierest

Briercor,clean

0.50

(B) Briercor,clean ↓ (multi3)

0.25

100 200 300 400 500 iter.

0

APcorrupt

(A) Brierest ↓ (multi3)

0

100 200 300 400 500 iter.

(H) APcorrupt ↑ (oracle) 0.50 0.40 0.30

0.20 100 200 300 400 500 iter. 0

APcorrupt baseline = 0.3

100 200 300 400 500 iter.

Figure 15: RCV1-10 held-out diagnostic evolution under multi3 with corruption rate 0.3 (mean±std over 5 runs). Same panel definitions as in Fig. 3. 35

the external hard co-membership reference o⋆i . Across datasets, these errors typically decrease quickly in the early stage and then become relatively stable, with some late-stage fluctuation. This pattern is clearer on CIFAR100-20, CIFAR10, ImageNet10, MNIST, and STL10, while FMNIST and Reuters show more rebound after the initial improvement, and RCV1-10 is the most unstable case. Because these Brier scores use o⋆i rather than the latent aleatoric relation Ra⋆i bi , small non-monotone changes should be read as diagnostic behavior rather than as direct evidence about latent-target recovery. Panel (D) gives a complementary correction-side view: the clean residual discrepancy usually drops early, indicating that corrected relations become more aligned with estimator beliefs on uncorrupted held-out constraints. Since the Estimator and Corrector co-evolve, this quantity should be interpreted together with (A–C), as evidence that the correction stage follows the improving estimator signal. Screening against injected corruption. Panels (E,F) evaluate whether the reliability signal aligns with the injected corruption indicator. Under multi3, AUCcorrupt and APcorrupt are generally above their chance baselines, so corrupted records tend to be ranked higher by the screening signal. The alignment is clearer on most image datasets, especially CIFAR100-20, CIFAR10, ImageNet10, MNIST, and STL10, but weaker or noisier on Reuters and particularly RCV1-10. This limitation reflects the heuristic nature of the screening signal: unreliability may arise not only from stochastic corruption, but also from expert-side epistemic variation, imperfect estimation/correction, and partial self-confirmation during training. The oracle-reference curves in panels (G,H) support this interpretation, as they usually track corruption more closely than the multi3 curves when the clean supervision is generated from the hard oracle relation before corruption. Summary. Overall, the diagnostics provide consistent qualitative evidence for the intended behavior of ECI-PP, although the curves are not uniformly monotone across datasets. The internal estimates improve mainly in the early stage, the correction discrepancy is reduced, and the screening signal remains meaningfully associated with injected corruption. RCV1-10 is the most challenging case, consistent with the broader experimental picture under severe class imbalance, where improvements in clustering structure are less uniformly reflected by the held-out diagnostic curves. Early stabilization further suggests that the ECI components often reach useful internal agreement before the fixed training horizon; however, we keep this schedule here and leave principled early stopping for future work. These diagnostics therefore provide practical held-out evidence for ECI-PP’s intended behavior, while remaining proxy measurements rather than exact tests of latent aleatoric recovery or pure corruption detection. D.6

Sensitivity and ablation studies

We further study the main ECI-PP design choices around the default configuration specified in Section 5.1. All sensitivity runs use the default single-expert supervision condition, i.e., lv0.01 expert quality, corruption rate 0.3, and 9k constraints. To expose the effect of each design choice, we use random initialization without unsupervised pretraining and vary one factor while keeping the others fixed. Figs. 16 to 23 report the corresponding test ACC, NMI, and ARI. Estimator choices. Figs. 16 and 17 examine the Estimator-side choices. For the number of folds, increasing K from 2 to 20 gives mild gains on several datasets, such as MNIST, FMNIST, CIFAR10, and ImageNet10, but the curves are mostly flat and the improvement is not proportional to the extra cross-fitting cost. This supports using K = 5 as a practical default: it avoids the weakest cross-fitting setting while keeping the main Estimator cost moderate. The information-based warm-up weight κi provides a small but consistent benefit. Enabling it is usually better or comparable across datasets, with visible gains on MNIST, FMNIST, Reuters, and CIFAR10, indicating that informative constraints are useful for stabilizing the early Estimator signal. This agrees with the main comparison in Table 1, where Weighted ProbPair generally improves over ProbPair by using the same information-based weighting principle. At the same time, the gains remain moderate, suggesting that ECI-PP benefits from this design without being overly dependent on it. Corrector choices. Figs. 18 and 19 study Corrector capacity and regularization. For capacity, we compare a large encoder-sized Corrector (500–500–2000), a medium classifier-sized Corrector (512–512), and the default small Corrector (64–16). The three sizes perform similarly on most datasets, and this supports the default use of a small Corrector: it is cheaper and also limits the risk of fitting arbitrary pair-specific deviations rather than structured expert-conditioned correction. The 36

K-fold K - Test ACC

K-fold K - Test NMI

K-fold K - Test ARI 0.90

0.90

0.80

0.70

Test ARI

0.80

0.80

Test NMI

Test ACC

0.90

0.70

0.40

0.50

0.50 2

5

10

20

2

5

K in K-Fold CIFAR100-20

0.60 0.50

0.60

0.60

0.70

10

0.30

20

2

5

K in K-Fold CIFAR10

FMNIST

10

20

K in K-Fold

IMAGENET10

MNIST

REUTERS

STL10

RCV1-10

Figure 16: Sensitivity to the number of estimator folds K. Each panel reports test ACC, NMI, or ARI across datasets (mean±std over 5 runs).

Information-based κi warm-up - Test ACC

Information-based κi warm-up - Test NMI 0.90

True False

0.90

Information-based κi warm-up - Test ARI 0.90

True False

True False

0.80

0.80

0.70

Test ARI

0.70

Test NMI

Test ACC

0.80 0.70

0.60

0.40

0.50

0.50

0.60 0.50

0.60

0.30 0 S -10 -20 AR10 NIST ET10 NIST ER STL1 V1 M UT FM GEN CIF RC RE IMA

100 AR CIF

100

AR CIF

0 S -10 -20 AR10 NIST ET10 NIST ER STL1 V1 M UT FM GEN CIF RC RE IMA

0 S -10 -20 AR10 NIST ET10 NIST ER STL1 V1 M UT FM GEN CIF RC RE IMA

100

AR CIF

Figure 17: Ablation of the information-based warm-up weight κi . Each panel reports test ACC, NMI, or ARI across datasets (mean±std over 5 runs).

Corrector hidden size - Test ACC

Corrector hidden size - Test NMI large medium small

0.90

Corrector hidden size - Test ARI large medium small

0.90

large medium small

0.90 0.80

0.70

Test ARI

Test NMI

Test ACC

0.80 0.80 0.70

0.40

0.50

0.50 0 0 S -20 AR10 NIST ET10 NIST ER STL1 V1-1 M UT FM GEN CIF RC RE IMA

100 AR CIF

0.60 0.50

0.60

0.60

0.70

0 0 S -20 AR10 NIST ET10 NIST ER STL1 V1-1 M UT FM GEN CIF RC RE IMA

0 0 S -20 AR10 NIST ET10 NIST ER STL1 V1-1 M UT FM GEN CIF RC RE IMA

100

AR CIF

100 AR CIF

Figure 18: Sensitivity to Corrector capacity. Each panel reports test ACC, NMI, or ARI across datasets (mean±std over 5 runs).

Corrector regularization strength λcor - Test ACC

Corrector regularization strength λcor - Test NMI

Corrector regularization strength λcor - Test ARI 0.90

0.90 0.90

0.80 0.80

0.70

Test ARI

0.70

Test NMI

Test ACC

0.80 0.70

0.50

0.60

0.60

0.60

0.40 0.50

0.50 0

0.01

0.05

0.1

0.5

1

5

10

0

λcor CIFAR100-20

0.01

0.05

0.1

0.5

1

5

10

0.30

0

0.01

λcor CIFAR10

FMNIST

IMAGENET10

0.05

0.1

0.5

1

5

10

λcor MNIST

REUTERS

STL10

RCV1-10

Figure 19: Sensitivity to the Corrector regularization strength λcor . Each panel reports test ACC, NMI, or ARI across datasets (mean±std over 5 runs).

37

regularization strength λcor shows the same trade-off more directly. Compared with λcor = 0, mild to moderate regularization can help on datasets such as FMNIST, CIFAR10, CIFAR100-20, and RCV1-10, but overly strong regularization is harmful on some datasets, most clearly Reuters when λcor reaches 5 or 10. Thus, the Corrector should not be left completely unconstrained, but it also should not be forced too strongly toward zero correction. The default λcor = 0.5 lies in the stable middle range. Reliability screening and boundary handling. Figs. 20 to 22 evaluate the parameters used after correction. The screening sharpness γ is robust over the tested range, especially for γ ∈ [5, 20]. Larger values are useful on some difficult datasets, notably RCV1-10 and also Reuters or MNIST in some metrics, indicating that stronger suppression of unreliable constraints can help when supervision is more fragile. The Bayesian-confidence scale n0 is also stable overall. Larger n0 improves RCV110 and slightly helps Reuters, suggesting that these datasets benefit from assigning more evidence to the refined supervision, whereas FMNIST does not benefit from the same increase. Thus, n0 controls a real reliability trade-off, but the default value remains within a safe region. Finally, performance changes little as the soft-clipping sharpness ξ varies: most datasets change little from ξ = 5 to 50, with only mild fluctuations on RCV1-10 and a few small metric-specific changes. This supports soft clipping as a robust boundary-handling design for keeping corrected relations in a valid probability range without relying on a finely tuned clipping sharpness. Reconstruction strength. Fig. 23 shows a noticeable dataset-dependent trade-off for λrec . The default value λrec = 0.02 follows SpherePair [3] and remains a reliable setting, but ECI-PP often benefits from moderately stronger reconstruction. For example, MNIST improves steadily as λrec increases, while Reuters and RCV1-10 prefer a moderate range around 0.05–0.2 before performance drops at larger values. In contrast, FMNIST and CIFAR10 do not benefit from overly large reconstruction weights, and ImageNet10 is almost unchanged. This suggests that reconstruction provides useful supervision-independent structure for out-of-fold estimation and final integration, but too much reconstruction can compete with clustering-oriented pairwise learning on some datasets. When validation-based tuning is available, λrec is therefore a meaningful parameter to adjust; without such tuning, the shared default remains a fair and stable compromise. Screening sharpness γ - Test ACC

Screening sharpness γ - Test NMI

0.90

0.80 0.80

0.80 0.70 0.60

Test ARI

0.70

Test NMI

Test ACC

Screening sharpness γ - Test ARI

0.90

0.90

0.70 0.60

0.50 0.40

0.50

0.50

0.60

0.30 1

2

5

10

20

1

2

5

γ

10

20

1

2

5

γ

CIFAR100-20

CIFAR10

FMNIST

10

20

γ

IMAGENET10

MNIST

REUTERS

STL10

RCV1-10

Figure 20: Sensitivity to the reliability-screening sharpness γ. Each panel reports test ACC, NMI, or ARI across datasets (mean±std over 5 runs). Bayesian-confidence scale n0 - Test ACC

Bayesian-confidence scale n0 - Test NMI

0.90

0.90

0.80

0.70

Test ARI

0.80

0.80

Test NMI

Test ACC

0.90

Bayesian-confidence scale n0 - Test ARI

0.70

0.40

0.50

0.50 5

10

20

50

5

n0 CIFAR100-20

0.60 0.50

0.60

0.60

0.70

10

20

50

5

10

n0 CIFAR10

FMNIST

IMAGENET10

20

50

n0 MNIST

REUTERS

STL10

RCV1-10

Figure 21: Sensitivity to the Bayesian-confidence scale n0 . Each panel reports test ACC, NMI, or ARI across datasets (mean±std over 5 runs).

38

Soft clipping sharpness ξ - Test ACC

Soft clipping sharpness ξ - Test NMI

Soft clipping sharpness ξ - Test ARI 0.90

0.90 0.90

0.80

0.70

Test ARI

Test NMI

Test ACC

0.80 0.80

0.70

0.40

0.50

0.50

0.60 0.50

0.60

0.60

0.70

0.30 5

10

20

50

5

10

20

ξ

50

5

10

20

ξ

CIFAR100-20

CIFAR10

FMNIST

50

ξ

IMAGENET10

MNIST

REUTERS

STL10

RCV1-10

Figure 22: Sensitivity to the soft-clipping sharpness ξ. Each panel reports test ACC, NMI, or ARI across datasets (mean±std over 5 runs). Reconstruction weight λrec - Test ACC

Reconstruction weight λrec - Test NMI

Reconstruction weight λrec - Test ARI 0.90

0.90 0.90

0.80 0.80

0.70

Test ARI

0.70

Test NMI

Test ACC

0.80 0.70

0.60

0.40

0.50

0.50

0.60 0.50

0.60

0.30 0

0.01

0.02

0.05

0.1

0.2

0.5

1

0

0.01

0.02

0.05

λrec

0.1

0.2

0.5

1

0

0.01

0.02

λrec

CIFAR100-20

CIFAR10

FMNIST

IMAGENET10

0.05

0.1

0.2

0.5

1

λrec MNIST

REUTERS

STL10

RCV1-10

Figure 23: Sensitivity to the reconstruction weight λrec . Each panel reports test ACC, NMI, or ARI across datasets (mean±std over 5 runs). Overall, the sensitivity and ablation results support the use of the default ECI-PP configuration in the main experiments. Estimator, Corrector, screening, and soft-clipping choices are stable over broad ranges, while the more dataset-dependent reconstruction weight changes performance gradually rather than causing abrupt failure. The results therefore indicate that ECI-PP is not hyperparameter-fragile, even though dataset-specific validation could still improve individual settings when such tuning is allowed.

E

Learning efficiency

Table 11: Overall wall-clock training time for different methods on eight datasets under the multi3 multi-expert supervision setting with 9k constraints and corruption probability 0.3, measured on a single NVIDIA A100 40GB GPU. Methods marked with ∗ require hyperparameter tuning, and their corresponding times are underlined. CIFAR100-20 VanillaDCC VolMaxDCC∗ CIDEC SpherePair SpherePair (EA)∗ ProbPair Weighted ProbPair Weighted ProbPair (EA)∗ ECI-PP (Ours)

1m12s 10m23s 18m27s 18m27s 3h53m10s 18m19s 18m24s 3h54m42s 29m48s

CIFAR10

FMNIST

ImageNet10

MNIST

1m10s 1m20s 45m21s 1h43m18s 19m26s 24m49s 17m58s 22m25s 3h40m47s 4h30m35s 17m42s 22m16s 17m44s 22m23s 3h34m20s 4h24m15s 29m12s 34m01s

0m45s 6m20s 5m03s 5m09s 1h49m25s 4m57s 5m07s 1h44m36s 15m49s

1m27s 0m37s 0m41s 1m51s 1h35m30s 5m00s 3m06s 2h39m25s 25m06s 5m53s 6m03s 1h07m26s 22m54s 6m05s 5m18s 1h01m54s 4h33m48s 2h12m12s 1h51m31s 10h40m49s 22m41s 5m49s 5m02s 1h01m36s 22m56s 5m52s 5m06s 1h01m37s 4h22m09s 2h01m15s 1h46m31s 10h32m39s 34m39s 17m34s 15m59s 1h15m41s

Reuters

STL10

RCV1-10

Overall training time. Appendix B.2 analyzes the computational complexity of ECI-PP; here, based on the implementation in Appendix C.5, we report empirical wall-clock measurements for the complete training pipelines in Table 11. Each entry is measured under the multi3 setting with 9k constraints and corruption probability 0.3, and covers the full practical training procedure: (i) for methods requiring unsupervised pretraining, the reported time includes the pretraining stage; (ii) for methods marked with ∗ , it also includes three scans over the corresponding hyperparameter list and 39

a final training run with the selected hyperparameter. When both pretraining and hyperparameter tuning are required, the timing includes pretraining on the training split excluding validation samples before tuning, followed by pretraining on the full training split before the final run. Runtime comparison. The runtimes in Table 11 should be interpreted alongside the clustering results in Appendix D.1. VanillaDCC is fastest, but mainly because collapsed assignment learning triggers early stopping; its performance in Table 8 is far below the competitive methods. VolMaxDCC has variable cost, becoming expensive on FMNIST, MNIST, and RCV1-10 due to validation-based hyperparameter search. For methods with autoencoder pretraining, including CIDEC, SpherePair, ProbPair, Weighted ProbPair, and ECI-PP, overall costs remain on the same order, with pretraining as a shared pipeline component. ECI-PP adds moderate overhead from estimator cross-fitting and correction/integration, but this overhead is modest relative to its empirical gains; in more resourcelimited settings, for example when unsupervised pretraining is unavailable, Appendix D.2 shows an even larger performance advantage. The expert-aware SpherePair and Weighted ProbPair extensions are most expensive because ensemble construction and hyperparameter tuning scale with the number of experts. In contrast, Appendix B.2 shows that ECI-PP incorporates expert identities through expert-conditioned correction without replicating the full training pipeline, so its leading cost scales mainly with the constraint budget rather than the expert count.

40

Related documents

• From Local Geometry to Global Pseudo Labeling for Robust Positive Unlabeled Learning under Covariate Shift#670608
• From Local Geometry to Global Pseudo Labeling for Robust Positive Unlabeled Learning under Covariate Shift#668543
• Epistemic Learning from Imprecise Annotation#1119358
• Self-supervised non-dominated sorted model for co-clustering.#13741
• From vanishing pairwise entanglement to global separability:Entanglement structure and measures in the W subspace#609847