arXiv CS open access

Reducing Symmetry Increase in Equivariant Neural Networks

2026 · arxiv_cs

arXiv CS · Papers · 2026

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

Published as a conference paper at ICLR 2026

R EDUCING S YMMETRY I NCREASE IN E QUIVARIANT N EURAL N ETWORKS Ning Lin, Jiacheng Cen, Anyi Li, Wenbing Huang∗, Hao Sun∗ Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China {ninglin00, jiacc.cn, li_anyi}@outlook.com, {hwenbing, haosun}@ruc.edu.cn

arXiv:2608.12010v1 [cs.LG] 12 Aug 2026

A BSTRACT Equivariant Neural Networks (ENNs) have empowered numerous applications in scientific fields. Despite their remarkable capacity for representing geometric structures, ENNs suffer from degraded expressivity when processing symmetric inputs: the output representations are invariant to transformations that extend beyond the input’s symmetries. The mathematical essence of this phenomenon is that a symmetric input, after being processed by an equivariant map, experiences an increase in symmetry. While prior research has documented symmetry increase in specific cases, a rigorous understanding of its underlying causes and general reduction strategies remains lacking. In this paper, we provide a detailed and in-depth characterization of symmetry increase together with a principled framework for its reduction: (i) For any given feature space and input symmetry group, we prove that the increased symmetry admits an infimum determined by the structure of the feature space; (ii) Building on this foundation, we develop a computable algorithm to derive this infimum, and propose practical guidelines for feature design to prevent harmful symmetry increases. (iii) Under standard regularity assumptions, we demonstrate that for most equivariant maps, our guidelines effectively reduce symmetry increase. To complement our theoretical findings, we provide visualizations and experiments on both synthetic datasets and the real-world QM9 dataset. The results validate our theoretical predictions.

I NTRODUCTION

Equivariant Neural Networks (ENNs) have become a cornerstone of modern machine learning, empowering numerous applications in scientific fields ranging from molecular dynamics to materials design (Bronstein et al., 2021; Huang & Cen, 2026). By building in physical symmetries, these models achieve remarkable data efficiency and generalization capabilities when representing complex geometric structures.

C3 σh

σv

(a) 3-fold

σh

σv

σv σv

σv

C2 C2

(b) 4-fold

Figure 1: k-fold structures.

Despite their success, ENNs exhibit a critical vulnerability when processing symmetric inputs: their expressivity can degrade, leading to a loss of information. This phenomenon, which we term symmetry increase, occurs when the output representation becomes invariant to transformations that are not symmetries of the original input itself. A canonical example arises when processing k-fold symmetric structures. These objects, visualized in Fig. 1, possess a specific dihedral symmetry, yet an ENN will map their distinct rotated versions to an identical feature, erasing their orientation. This type of degradation has been documented in previous work. Empirically, it has been observed that the degradation depends on the feature space, particularly for symmetric inputs (Joshi et al., 2023). Theoretically, research on ENNs has shown that for k-fold symmetries, selecting only low-degree ∗

Corresponding authors.

Published as a conference paper at ICLR 2026

features can cause all rotated inputs to collapse into a single representation (Cen et al., 2024). This ENN-specific issue is a modern manifestation of a general phenomenon observed in other fields. It has been linked to physical principles such as Curie’s Principle (Smidt et al., 2021) and described using the concept of orbit types (Kaba & Ravanbakhsh, 2023). However, existing analyses provide an incomplete picture and lack a predictive framework. In our analysis, this degradation of k-fold caused by symmetry increase can be categorized into three distinct types: full degeneration, axial degeneration, and half degeneration (see Fig. 2)1 . The work of Joshi et al. (2023), while empirically important, does not theoretically explore the cause of the degradation. The collapse-to-zero theory proposed by Cen et al. (2024) addresses only full degeneration, which is the most extreme case identified by our analysis. The broader principles discussed by Smidt et al. (2021) and Kaba & Ravanbakhsh (2023) lack a rigorous mathematical description, and the solutions proposed, such as in Kaba & Ravanbakhsh (2023), often involve relaxing the equivariance constraint itself, rather than providing a solution within the equivariant framework. In this paper, we fill this gap by providing a comprehensive mathematical characterization of symmetry increase. Our main contributions are briefly listed as follows: • In § 3, we prove for any given feature space and input symmetry group, that the increased symmetry is bounded from below by a unique symmetry infimum, which is determined entirely by the algebraic structure of the feature space. • In § 4, we develop a computable algorithm to derive this infimum by analyzing the orbit types. This provides practical guidelines for predicting and controlling potential symmetry increases, thereby preventing harmful symmetry increases in feature design. • In § 5, we demonstrate that under regularity conditions, such as the manifold hypothesis for data, our method can fully reduce symmetry increase. Specifically, for most equivariant maps or for ENNs with sufficient approximation capabilities, the output symmetry will be precisely this predictable infimum, preventing orientational information loss. • In § 6, we complement our theoretical findings with empirical evidence. We provide visualizations to illustrate the proposed concepts and present experimental results on both synthetic datasets and the real-world QM9 dataset, which validate our theoretical predictions and demonstrate the practical effectiveness of our framework.

P RELIMINARIES

Group action and representation. Consider the action of a group G on a set X, denoted by ρX . This action is a map that assigns to each element g ∈ G a transformation ρX (g) : X → X, such that ρX (g1 g2 ) = ρX (g1 )ρX (g2 ). We call such a set X a G-set. In particular, if X is a vector space and ρX (g) is a linear transformation for all g ∈ G, we call X a G-representation2 .

Equivariant map. For maps between two G-sets X and Y , an equivariant map is one that respects the group action, meaning that the output transforms accordingly when the input is transformed. Formally, a map f : X → Y is equivariant if for all g ∈ G and x ∈ X: f (ρX (g)(x)) = ρY (g)(f (x)).

(1)

Example 2.1 (Equivariant Encoding of Point Clouds). The symmetry group is G = H × Sn , where H is typically the special orthogonal group SO(3) or the orthogonal group O(3), and Sn is the permutation group. We consider features that are invariant to both permutation and translation. To achieve translation invariance, the input representation X is the space of centered point clouds, where H acts on the coordinate of each point and Sn permutes the points. To achieve permutation invariance, the final output are designed to be invariant with respect to Sn . This means the feature representation of interest is a direct sum of specified irreducible representations of H. The task of equivariant encoding is then to learn an equivariant map f : X → Y . Data symmetry. For a G-set X, the action partitions the space into orbits, G(x) := {g(x) | g ∈ G}. The subgroup that fixes a point is its isotropy subgroup, Gx := {g ∈ G | g(x) = x}. The conjugacy 1

Although we choose k-fold as an illustrative example, our theories are applicable to general cases. Unless otherwise specified, all groups are assumed to be compact Lie groups, and all vector spaces are finite-dimensional real vector spaces. 2

Published as a conference paper at ICLR 2026

class (Gx ) of this subgroup is the orbit type of x. The set of all points with orbit type (H) is X(H) and the set of points fixed by all elements of a subgroup H is the fixed-point set X H . These concepts distinguish between the global symmetry of the space and the intrinsic symmetry of an object. The group G represents the global symmetry, transforming the object between different reference frames. An orbit G(x) thus represents a single physical object in all of its possible orientations. This object possesses its own intrinsic symmetry, mathematically defined by the isotropy subgroup Gx for any point x on its orbit. While the specific subgroup is frame-dependent, its structure up to conjugation is constant. The orbit type (Gx ) therefore serves as a reference-frame independent identifier that precisely describes the physical object’s intrinsic symmetry. The set of all possible orbit types, OG (X), thus catalogs all distinct symmetries that objects in the space can possess. Example 2.2. Now apply the setup from Ex. 2.1. Consider the action of G = O(3) × Sk+1 on centered point cloud space X for k > 2. Let x ∈ X be the set of vertices of a k-fold in the xOy-plane: x = (x0 , x1 , . . . , xk ),

where xi = (cos(2iπ/k), sin(2iπ/k), 0) for i > 0.

(2)

with x0 at the origin. The generators of Gx include:(1) A rotation about the z-axis combined with a cyclic permutation of x1 , . . . , xk . (2) A reflection across the xOz-plane combined with a product of transpositions. (3) A reflection across the xOy-plane combined with the identity. Considering the projection map πX ((g, σ)) = g, where (g, σ) is a pair consisting of a geometric transformation g ∈ O(3) and a permutation σ ∈ Sk+1 , we find that geometric symmetry πX (Gx ) is the dihedral group with horizontal reflection of order 4k, denoted by the Schoenflies symbol Dkh . The symmetry of data can be altered by an equivariant map. The following theorem shows that an equivariant map does not decrease symmetry. Theorem 2.3 (Curie’s principle, Kaba & Ravanbakhsh (2023), Thm. 1). Let f : X → Y be a G-equivariant map. For x ∈ X, the isotropy subgroup of x is contained in that of its image f (x), i.e., Gx ⊆ Gf (x) .

(3)

Such an increase in symmetry becomes unavoidable if the feature space Y cannot support the input’s symmetry. If the orbit type (Gx ) is not present in Y (i.e., (Gx ) ∈ / OG (Y )), then the equality Gx = Gf (x) is impossible (otherwise, Gf (x) becomes an isotropy subgroup, implying (Gx ) ∈ OG (Y ), which leads to a contradiction), and the symmetry must therefore strictly increase. This increase leads to a degeneration, as any transformation g in the larger group Gf (x) that is not in Gx will map the distinct inputs x and g(x) to the same output, since f (g(x)) = g(f (x)) = f (x). Fig. 2 illustrates three possible types of such degenerations for the k-fold structure from Ex. 2.2.

T HE I NFIMUM OF S YMMETRY

Half Degeneration

This section establishes that symmetry increase is governed by an infimum determined by the feature space. We prove the existence of this infimum and show that its coincidence with the input symmetry is a necessary condition for an equivariant map to preserve symmetry.

Full Degeneration

Input Space

3.1

T HE I NFIMUM OF S YMMETRY

Axial Degeneration Output Space

Figure 2: Three types of degeneration of k-fold.

The symmetry of data can increase after being transformed by an equivariant map. This increase can be an intentional design choice, or it can arise from subtle properties of the feature space. For instance, in the task from Ex. 2.1, the requirement for permutation-invariant features means that the permutation group Sn is naturally introduced into the isotropy subgroup of any output. This type of designed, unavoidable symmetry increase is formalized by the kernel of the group action on the feature space. We define the kernel of the action ρX as the set of group elements that fix every point in X: kerρX := {g ∈ G | g(x) = x, ∀x ∈ X} Distinguishing between intentional and unintended symmetry increase is crucial. We begin by assuming that the group action is faithful, i.e., it has a trivial kernel with kerρY = {e}. The case of a nontrivial kernel will be discussed in the next subsection. 3

Published as a conference paper at ICLR 2026

To characterize the behavior of symmetry increase within a representation X, we first establish a partial order to compare different symmetries. An orbit type (H1 ) is considered greater than or equal to another, (H2 ), written as (H1 ) ≥ (H2 ), if H1 contains a conjugate of H2 . This ordering reflects that a larger orbit type corresponds to a higher symmetry. With this framework, we are interested in the lower bound of orbit types that can be reached from a point x with a specific isotropy group H = Gx . The analysis can be framed around any closed subgroup H, as the set of all possible isotropy subgroups is precisely the set of all closed subgroups of G (see, e.g., Field (2007); Mostow (1957)). The fixed-point space X H contains points of all higher orbit types. This leads to the following powerful theorem, which guarantees that the lower bound on symmetry increase is unique, corresponding to an isotropy subgroup that is unique up to conjugation. Theorem 3.1 (Uniqueness of Minimal Type). Let X be a representation of a compact Lie group G. For any closed subgroup H ⊆ G, a unique minimal orbit type exists among the points in the fixed-point subspace X H . In particular, if (H) ∈ OG (X H ), then (H) is the minimal orbit type within that subspace. The uniqueness guaranteed by this theorem allows us to define the symmetry infimum, denoted by IG (X, H), as this unique minimal orbit type. In the context of symmetry increase, we are concerned with the relationship between I(Y, Gx ) and (Gf (x) ). An unexpected symmetry increase occurs if (Gf (x) ) > IG (Y, Gx ). The desired behavior for an equivariant map is captured by the following definition. For a map between G-sets X and Y , we define an isovariant map as one that strictly preserves symmetry for all x ∈ X: Gx = Gf (x) . (4) Using the concept of the symmetry infimum, we can provide a necessary condition for the existence of isovariant maps. In § 5.1, we will see that this condition is in fact not sufficient for equivariant maps between representations, even when we assume a trivial kernel kerρY = {e}. Theorem 3.2. A necessary condition for the existence of an isovariant map between G-sets X and Y is that OG (X) ⊆ OG (Y ). When X and Y are representations of a compact Lie group G, this is equivalent to the condition that IG (Y, H) = (H) for all (H) ∈ OG (X). 3.2

E QUIVARIANCE WITH N ON - TRIVIAL K ERNELS

When the feature space Y is restricted to have a non-trivial kernel, i.e., ker ρY ̸= {e}, the definition of an isovariant map becomes too restrictive. Since every isotropy group Gy in Y contains the kernel, any subgroup H not containing ker ρY cannot occur as an isotropy subgroup. Consequently, for any map f : X → Y , an input isotropy subgroup Gx that does not contain the kernel must increase. To formalize this unavoidable symmetry increase, we introduce an operator pY . In the discussion following Ex. 2.2, the presence of the Sk+1 kernel forces the input isotropy subgroup Gx to become at least Dkh × Sk+1 in the feature space. We generalize this observation. The operator pY (H) is defined as the smallest subgroup containing H that is compatible with the action on Y , given by the projection pY = πY−1 ◦ πY , where πY : G → G/kerρY is the natural projection. This operator is idempotent (p2Y = pY ) and maps any isotropy subgroup in Y to itself. These properties reveal why increasing is unavoidable. For any equivariant map f , the relation Gx ⊆ Gf (x) must hold. Applying the pY operator to this inclusion gives: Gx ⊆ pY (Gx ) ⊆ pY (Gf (x) ) = Gf (x) .

(5)

This unavoidable increasing from Gx to pY (Gx ) means our goal is not to preserve Gx itself, but to ensure no additional symmetry is introduced beyond pY (Gx ). This leads to a more practical definition. We say a map f : X → Y is an isovariant map relative to Y if for all x ∈ X it satisfies pY (Gx ) = Gf (x)

⇐⇒

ρY (Gx ) = ρY (Gf (x) ).

(6)

When the kernel is trivial, this definition reduces to that of a standard isovariant map. With this refined goal, we can state a necessary condition for the existence of such maps. Theorem 3.3. A necessary condition for the existence of an isovariant map relative to Y from a G-set X to a G-set Y is that (pY (H)) ∈ OG (Y ) for every (H) ∈ OG (X). When X and Y are representations, this is equivalent to the condition that IG (Y, H) = (pY (H)) for all (H) ∈ OG (X). 4

Published as a conference paper at ICLR 2026

c R

Change the Rotation Axis

R(θ)

Rotate around the Axis

Figure 3: Visualization of representation spaces. (a) A k-fold structure is reoriented onto multiple planes. (b) Each is further rotated about the perpendicular axis. (c) All structures are embedded and projected into 2D. Marker shapes denote rotation axes, and colors denote rotation rates. Full degeneration appears at l = 0, 1, and axial degeneration at l = 2, 4.

C OMPUTATION OF O RBIT T YPES

The computation of orbit types is a classical problem, with most established results on general representations, often in the context of bifurcation theory. In representation learning tasks, however, we utilize feature spaces containing high multiplicities of these representations, which requires us to supplement the existing computational frameworks. 4.1

O RBIT T YPES OF H IGH -M ULTIPLICITY R EPRESENTATIONS

For a compact Lie group G, any representation space X can be uniquely decomposed into a direct sum of its irreducible components. For G = SO(3), this decomposition is written as ∼ L∞ V ⊕m(X,Vl=l0 ) , X= (7) l0 =0 l=l0 where Vl=l0 is the irreducible representation corresponding to the space of spherical harmonics of degree l0 , and m(X, Vl=l0 ) is its multiplicity. For G = O(3), the decomposition is similar, but the irreducible representations must also be distinguished by parity, denoted Vl=l+ and Vl=l− . 0

We begin with the foundational criterion for identifying isotropy subgroups, first established as a necessary condition by Michel (1980). Theorem 4.1 (Michel’s Criterion, Michel (1980), App. A). Let V be a representation of a group G. A necessary condition for a closed subgroup H to be an isotropy subgroup in V is that for any adjacent closed subgroup H ′ ⊋ H, the dimension of the fixed-point subspace strictly decreases: ′

dim V H < dim V H .

(8)

While this condition is not sufficient for all representations, its sufficiency can be guaranteed under certain common conditions. We define a representation V of a group G as a high-multiplicity representation if for every non-zero isotypic component corresponding to an irreducible representation Vi , its multiplicity m(V, Vi ) is greater than dim G. Proposition 4.2. For a high-multiplicity representation V , the necessary condition stated in Thm. 4.1 is also sufficient. This criterion is particularly powerful for two reasons. First, it offers a computationally convenient method in the form of a chain recursion, which only requires checking adjacent subgroups. The dimensions of the necessary fixed-point spaces can be calculated via the trace formula (Golubitsky 5

Published as a conference paper at ICLR 2026

Algorithm 1: Orbit Type Test for HighMultiplicity Representations Data: Symmetry group G; Closed subgroup H ⊂ G; High-Multiplicity Rep. V of G Result: is_in((H), OG (V )). 1 Let set S = {Hi } ⊂ G to be all adjacent closed supergroups of H in G; 2 O ← ∅;

Algorithm 2: Symmetry Infimum Calculation Data: Symmetry group G; Closed subgroup H ⊂ G; Rep. V of G. Result: Symmetry inf. IG (V, H). 1 if is_in((H), OG (V )) then 2 return (H); 3 end 4 Let set S = {Hi } to be all closed supergroups of H in G; 5 O ← ∅; 6 for Hi in S do 7 if is_in((Hi ), OG (V )) then 8 Add (Hi ) to O; 9 end 10 end 11 return min(O);

dH ← dim V H ; 4 for Hi in S do 5 dHi ← dim V Hi ; 6 if dH − dHi = 0 then 7 return False; 8 end 9 end 10 return True; 3

et al., 1988). Second, the sufficiency condition is frequently met in our applications, as it holds for all finite groups and for feature spaces with a high number of channels. Based on the result of Prop. 4.2, we design an orbit type test Algo. 1 and a symmetry infimum calculation Algo. 2. Using the two algorithms described above, we have characterized all instances of symmetry increase for the closed ⊕r ⊕r subgroups of SO(3) or O(3) in the representations Vl=l and Vl=l ± for r > 3, respectively, see § C.4. 0 0 We now illustrate our algorithms with a simple example. Example 4.3. We illustrate our algorithms by calculating the orbit type and symmetry infimum for the geometric symmetry Dkh (k > 2) of the k-fold from Ex. 2.2, considered as a subgroup of O(3). ⊕r The calculation is performed in the high-multiplicity representation space Y = Vl=l (r > 3, l0 > 0). 0 Here we provide only a sketch of the derivation, the full procedure is provided in § C.3. First, we apply the orbit type test from Algo. 1. This involves comparing the dimension of the fixed-point space of Dkh with that of its adjacent supergroups (e.g., Dpk,h and, for k = 4, Oh ). The analysis shows that (Dkh ) is an orbit type if and only if l0 ≥ k and l0 , k have the same parity. Next, we apply the symmetry infimum calculation from Algo. 2. This requires identifying the minimal orbit type among all supergroups of Dkh , including non-adjacent ones like D∞h and O(3). The final results are summarized in Table 1. ⊕r Table 1: The symmetry infimum IO(3) (Vl=l , Dkh ) for k > 2, r > 3, l0 > 0. 0

k is even k is odd

l0 < k l0 is even l0 is odd

k ≤ l0 < 2k l0 is even l0 is odd

l0 ≥ 2k l0 is even l0 is odd

(D∞h ) (D∞h )

(Dkh ) (D∞h )

(Dkh ) (D2kh )

(O(3)) (O(3))

(O(3)) (Dkh )

The analysis in Ex. 4.3 predicts three types of degeneration for the k-fold inputs from Ex. 2.1: • Half Degeneration: The symmetry infimum of Gx is (D2kh × Sk+1 ). The feature cannot distinguish the k-fold from itself rotated by π/k around the z-axis. • Axial Degeneration: The symmetry infimum of Gx is (D∞h × Sk+1 ). The feature cannot distinguish the k-fold from itself rotated by any angle around the z-axis. • Full Degeneration: The symmetry infimum of Gx is (O(3) × Sk+1 ). The feature cannot distinguish the k-fold from itself rotated by any angle around any axis. We consider encoding k-fold point clouds using equivariant neural networks and visualize the resulting embeddings. The three degenerations are experimentally verified in our visualizations, with Fig. 3 showing full and axial degeneration, and Fig. 4 showing axial and half degeneration. Although derived assuming high multiplicity (r > 3) in the feature representation, these predictions are identical for the single representation case (r = 1), see § C.4. 6

Published as a conference paper at ICLR 2026

a R(π/k)

Axial Deg. Dkh ⊂ D2kh ⊂ D∞h ⊂ O(3) Half Deg.

Full Deg. Symmetry Increase

Figure 4: Visualization of representation spaces. (a) A k-fold (k is odd) structure is rotated π/k about the perpendicular axis. (b) The path of symmetry increase. (c) Half degeneration appears at l = 10. At res = 98 and res = 49, the overall shape is identical, but the yellow data points (i.e. the second half of the rotation) completely cover the blue data points. 4.2

G UIDELINES FOR M ANAGING S YMMETRY I NCREASE

It is a general property of representations that the orbit types of a direct sum are related to those of its components by OG (V1 ) ∪ OG (V2 ) ⊆ OG (V1 ⊕ V2 ), and IG (V1 ⊕ V2 , H) ≤ IG (Vi , H) for i = 1, 2, with equality conditions discussed in § C.5. These properties provide a direct mechanism for controlling the symmetry increase of an equivariant feature, that is to choose components whose symmetry infimum (computed as described in § C.4 for G = SO(3) or O(3)) align with the desired behavior for task-relevant symmetries. Regarding the selection of the feature space Y in equivariant representation learning, this principle translates into two guidelines. For orientation-dependent tasks (e.g., § 6.2), when considering the kernel of feature space, it is crucial to avoid non-trivial symmetry increase (i.e. ensuring the map is relative isovariant) since such increases can lead to the accidental loss of orientational information. Therefore, for a given input symmetry (H), one should select feature components that contain the orbit type (pY (H)). For general tasks (e.g., § 6.3), certain forms of symmetry increase must be avoided, as the output symmetry reflects the dimensionality of the fixed-point subspace where the equivariant features lie (see Remark of Prop. C.2). In this context, one should generally avoid components where the symmetry infimum indicates a severe compression of the fixed-point subspace. Specifically, one must be cautious of components corresponding to non-trivial representations where the symmetry increases to the full group, as this causes the component to be annihilated and lose all discriminative power.

D ENSITY OF (A LMOST ) I SOVARIANT M APS

We now connect the preceding theory to a practical machine learning context by introducing models for the data distribution and for the parameterized map. We show that the necessary conditions for isovariance established previously become sufficient under a relaxed definition of isovariance. 5.1

T HE M ANIFOLD H YPOTHESIS

Motivated by the manifold hypothesis and the broader considerations summarized in § A.3, we modelSthe data distribution as being supported on a finite union of smooth, compact submanifolds M = j Mj embedded in X. When a group G acts on X, this action equips each Mj with a natural G-manifold structure. The central question is: when does an isovariant map f : M → Y exist? The existence is a non-trivial issue. The counterexample in Cex. D.3 demonstrates that the necessary condition of orbit type 7

Published as a conference paper at ICLR 2026

inclusion is not sufficient. Specifically, it shows that an isovariant map can fail to exist precisely because the multiplicities of the irreducible representations in the feature space are insufficient. The non-existence of perfectly isovariant maps motivates a more practical, relaxed definition: a map that is isovariant almost everywhere. To formalize this, we equip M with the d-dimensional Hausdorff measure µM , where d = maxj {dim Mj }. This allows us to identify subsets of measure zero as negligible. Note that M(H) is a finite union of submanifolds, then f is almost isovariant relative to Y if for every orbit type (H) in the data support, the isovariance condition ρY (Gx ) = ρY (Gf (x) ),

(9)

holds for all points x ∈ M(H) except for a subset of µM(H) -measure zero. This ensures that any undesired increase in symmetry occurs only on a negligible portion of the data. 5.2

G ENERICITY OF (A LMOST ) I SOVARIANT M APS

For Ex. 2.1, we select TFN (Thomas et al., 2018), a classic ENN based on tensor products, as our parameterized model. We provide the complete formulation to § D.2. An important property of TFN parameterizations FTFN is that they satisfy a universal approximation theorem (Dym & Maron, 2021). In topology, this is equivalent to FTFN being dense in equivariant function space CG (X, Y ) with respect to the C 0 topology. In fact, we can establish a stronger approximation theorem. Theorem 5.1. In Ex. 2.1, the function families FTFN with smooth non-polynomial activation function ∞ are C ∞ -dense in the space of smooth equivariant maps CG (X, Y ). That is, for any integer r ≥ 0, ∞ any map f ∈ CG (X, Y ), any compact set K ⊂ X, and any ϵ > 0, there exists g ∈ FTFN such that maxx∈K Dk f (x) − Dk g(x) < ϵ, k ≤ r.

(10)

Here, Dk denotes the k-th order total derivative operator. A significant portion of maps within a dense parameterization reflects the generic properties of the mapping space. For equivariant maps, a key generic property, closely related to almost isovariance, is that the dimension of the set of points where the orbit type is increase from (H) to (H ′ ) by a map f is constrained for a generic map. The following theorem shows that for expressive models with C ∞ approximation capabilities, such as the TFN discussed, almost isovariance is a generic property, and full relative isovariance can be achieved by increasing representation multiplicity. As shown in Cex. D.3, this requirement is tight. Theorem 5.2. Let F be a equivariant parametrization with C ∞ approximation capability. If for every (H) ∈ OG (M ) we have (pY (H)) ∈ OG (Y ), then for any finite union of compact, smooth ∞ G-submanifolds M ⊂ X, any f ∈ CG (X, Y ), any integer r ≥ 0, and any ϵ > 0, there exists a map g ∈ F such that maxx∈M ∥Dk f (x) − Dk g(x)∥ < ϵ, k ≤ r, (11) and g|M is almost isovariant relative to Y . Furthermore, if the feature space Y contains a representation Ỹ ⊕r for an integer r > maxj {dim Mj }, where Y satisfies the condition (pY (H)) ∈ OG (Y ), then the approximating map g|M can be chosen to be isovariant relative to Y .

E XPERIMENT

We validate our theoretical analysis through three experiments: representation-space visualizations in § 6.1, a geometric graph discrimination task in § 6.2, and a molecular isotropic polarizability prediction task on QM9 (Ramakrishnan et al., 2014) in § 6.3. Across all experiments, we consider two equivariant architectures, TFN (Thomas et al., 2018) and HEGNN (Cen et al., 2024): TFN is used in the first two experiments, whereas HEGNN is employed in the last two. All detailed experimental settings are provided in § F. 6.1

V ISUALIZATION OF R EPRESENTATION S PACE

To provide a clearer illustration of our theory, we present visualizations of the representation spaces of different degrees, obtained from the 3-fold structure. Dataset. We first construct a k-fold structure lying in the xOy plane (here k = 3), and then apply random rotations to place it on m distinct planes (here m = 6), as illustrated in Fig. 4(a). 8

Published as a conference paper at ICLR 2026

Subsequently, as shown in Fig. 4(b), each structure is further rotated about the axis perpendicular to its plane. The rotation angle θ ∈ [0, 2π/k) is uniformly discretized into res candidate values, defined as {2πi/(k · res)}res−1 i=0 . Unless otherwise stated, we use res = 49 to verify the half-degeneration. We also consider the doubled resolution res = 98. Embeddings. For the resulting m · res candidate structures, we compute the graph-level features via randomly initialized single-layer TFN (Thomas et al., 2018) with detailed setting in § F.1. For l0 = 0 the feature dimension is 1; for visualization we set the second coordinate to zero. For l0 ≥ 1 the feature dimension is 2l0 + 1 > 2; we reduce dimensionality via random projection and then rescale all features to a common range so that visualizations are comparable. Data points are plotted in ascending order of plane index; for structures on the same plane they are ordered by increasing rotation angle about the axis. Results. Detailed experimental results are presented in Figs. 3 and 4. The former shows the input symmetry with l0 ≤ 11 increase to (O(3) × Sk+1 ) (Full degeneration, l0 = 0, 1), (D∞h × Sk+1 ) (Axial degeneration, l0 = 2, 4), or remains non-degenerate. The latter shows the symmetry increase to (D2kh × Sk+1 ) at l0 = 10. These experimental results are consistent with Ex. 4.3. E XPRESSIVITY ON S YMMETRIC G RAPHS 2D Rotation

Dataset. We construct four symmetric k-fold structures (k ∈ 2, 3, 4, 6), each centered at the origin. For each structure G0 we apply a random rotation to obtain G1 , ensuring that G1 does not coincide with the original G0 . The goal is to evaluate whether different ENNs can distinguish G0 from G1 . To probe different aspects of our theory, we treat 2D and 3D rotations separately; in the 3D setting we additionally require that G1 is not coplanar with G0 .

Degree of Models

To experimentally validate our theoretical conclusion established , we design a more comprehensive experiment following Joshi et al. (2023).

0 1 2 3 4 5 6 7 8 9 10 11

3D Rotation

Embedding Difference Norm (Max of All Settings)

6.2

10 3 10 6

2-fold 3-fold 4-fold 6-fold 2-fold 3-fold 4-fold 6-fold

Figure 5: Heatmap of Emb. Diff. Norm.

Embeddings. We employed both TFN (Thomas et al., 2018) and HEGNN (Cen et al., 2024) to compute the norm of the embedding difference across 12 configurations for each, varying by the number of irrep channels (1, 4, 16) and layers (1-4). The extracted l0 -degree embeddings are evaluated via the norm of the difference between the embeddings of G0 and G1 as in Cen et al. (2024) with detailed setting in § F.2.1. When this norm approaches zero, the two embeddings are numerically indistinguishable, and hence the corresponding geometric figures cannot be told apart by the model 3 . Results. The maximum value was selected for each configuration and visualized in a heatmap, as shown in Fig. 5. The results exhibit a clear binary pattern: the values are either greater than 10−3 or less than 10−6 (due to numerical error), with a difference of more than 103 times. This suggests that values exceeding 10−3 indicate distinguishable structures, while those below 10−6 correspond to indistinguishable structures. These findings align precisely with our theoretical predictions. Furthermore, as the maximum value was chosen, with all norms being less than 10−6 , this phenomenon is shown to be independent of the model choice, the number of channels or layers. 6.3

M OLECULE P ROPERTY P REDICTION WITH P RETRAINED E QUIVARIANT F EATURES

To illustrate the guiding significance of the theory presented in this paper for practical applications, we designed experiments on the QM9 dataset (Ramakrishnan et al., 2014) for verification. Dataset. We choose to predict the molecular isotropic polarizability α. It is worth noting that the QM9 dataset (Ramakrishnan et al., 2014) contains many highly symmetric structures spanning 22 3 In Joshi et al. (2023), the embeddings are directly fed to a vanilla classifier. To address issues such as imperfect classifier training and numerical error, we slightly modify this experimental setup. We also reproduce their original experiment in § F.2.2, and the results remain consistent with our theory.

Published as a conference paper at ICLR 2026

MAE loss (log scale)

C1 (111,935 samples) 10

9 10 11

Cs (17,794 samples)

MAE loss (log scale)

C3v (67 samples)

MAE loss (log scale)

9 10 11

MAE loss (log scale)

Model Degree l

9 10 11

Model Degree l

9 10 11

Model Degree l

9 10 11

C v (8 samples)

9 10 11

Td (5 samples)

9 10 11

D3h (15 samples)

D h (8 samples)

D2h (6 samples)

9 10 11

C2v (433 samples)

Ci (15 samples)

C3 (11 samples)

D3d (6 samples)

C2h (48 samples)

D2d (11 samples) 10

C2 (458 samples)

9 10 11

C3h (4 samples)

9 10 11

Model Degree l

9 10 11

Figure 6: MAE loss (in units of a30 ) for isotropic polarizability prediction with degree l = l0 across molecules from the top-16 point groups by molecular count. Each boxplot shows the distribution of errors at a given degree, while diamond markers denote the corresponding mean MAE. molecular symmetry groups 4 , with Fig. 6 reporting sample counts for the 16 most frequent groups; the remaining seven are D3 with 2 samples and C4 , D2 , D6h , Oh , S4 , each appearing only once. Embeddings. We adopt HEGNN (Cen et al., 2024) as our backbone. Specifically, we first pretrain on features of l ≤ 11 to obtain a shared equivariant feature encoder, ensuring that all subsequent configurations operate on the same embedding space. We then consider two fine-tuning strategies: (a) using only features of l = l0 , and (b) using all features of l ≤ l0 , yielding 12 distinct prediction heads for each. The detailed experimental setup is given in § F.3.1. Results. The results are shown in Fig. 6 and § F.3.2, with a summary of all symmetry increases in Table 22. We note that for the majority of samples, different feature components contribute similarly to the prediction. However, as illustrated in Fig. 6, for non-trivial feature components where molecular symmetry increase to O(3), the prediction loss is substantially higher. Furthermore, the results in § F.3.2 show that for symmetries causing full degeneration in 1-degree features, including additional 1-degree features may not provide significant improvement to the model’s prediction performance. This validates the design guidelines in § 4.2. Detailed case studies are provided in § F.3.2.

C ONCLUSION

In this work, we presented a rigorous mathematical framework to address the critical issue of symmetry increase in ENNs. We introduced the concept of the symmetry infimum, a computable lower bound for any increase in symmetry determined by the feature space. Our central contribution is to show that this infimum can be used to precisely predict and control the expressive degradation of ENNs. The framework successfully explains phenomena in settings like those of Joshi et al. (2023), which could not be fully accounted for by prior theories such as the collapse-to-zero model from Cen et al. (2024). Our findings provide both a robust theoretical understanding and practical guidelines for designing more reliable ENNs. 4 We use the QM9 dataset as provided in PyG (Fey & Lenssen, 2019) and apply the PointGroup library (Carreras, 2025) to pre-compute and manually post-process the point groups of all molecules. As a result, our statistics may differ slightly from those reported in previous works like Zeng et al. (2025).

Published as a conference paper at ICLR 2026

ACKNOWLEDGMENTS This work was supported by the National Natural Science Foundation of China (Nos. 62276269, 92270118, and 62376276), the Beijing Natural Science Foundation (No. 1232009), and the Beijing Nova Program (No. 20230484278).

AUTHOR CONTRIBUTIONS Ning Lin organized this project. Ning Lin led the theoretical development in § 2–§ 5 and was responsible for the theoretical proofs of the corresponding part. Ning Lin and Jiacheng Cen jointly led the experimental studies in § 6. Jiacheng Cen was responsible for model implementation and code development. Anyi Li was responsible for data processing. Jiacheng Cen and Anyi Li jointly contributed to figure preparation and visualization. Wenbing Huang and Hao Sun jointly supervised and guided the project. All authors participated in writing and revising the manuscript.

U SAGE OF L ARGE L ANGUAGE M ODELS We only use Large Language Models to polish our writing.

E THICS S TATEMENT This work is a theoretical contribution in the domain of equivariant neural networks, focusing on mathematical properties of symmetry preservation and transformation under equivariant mappings. The research does not involve human subjects, personal data, or real-world deployments, and therefore does not raise concerns related to privacy, fairness, bias, or potential misuse. We affirm that this work adheres to the ICLR Code of Ethics. In particular, we have ensured honesty in representing our contributions, accuracy in reporting our findings, and proper attribution of prior work. As this is a theoretical study, there are no conflicts of interest, sponsorship influences, or applications with foreseeable societal harms to disclose. We support responsible stewardship of machine learning research and believe that foundational advances such as ours contribute positively to the scientific community by enhancing understanding and enabling future trustworthy systems.

R EPRODUCIBILITY S TATEMENT We are committed to ensuring the reproducibility of both the theoretical and experimental components of our work. To support the reproducibility of our theoretical results, we provide complete and self-contained proofs for all main theorems and propositions in the appendix, including detailed derivations and necessary mathematical background. These proofs clarify all assumptions and logical steps required to verify our claims. For the experimental component, we have made our implementation code publicly available at https://github.com/GLAD-RUC/SymInc. The codebase contains clear documentation and scripts that fully reproduce the reported results. All experimental settings, hyperparameters, and data generation procedures are described in detail within the supplementary materials. Together, the comprehensive theoretical appendices and open-sourced code ensure that our findings can be rigorously verified and built upon by the research community.

R EFERENCES V.I. Arnold, S.M. Gusein-Zade, and A.N. Varchenko. Singularities of Differentiable Maps: Classification of Critical Points, Caustics and Wave Fronts, volume 1. Birkhäuser Boston, 2012. doi: 10.1007/978-0-8176-8340-5. URL https://link.springer.com/10.1007/ 978-0-8176-8340-5. M. I. Aroyo. International Tables for Crystallography: Space-group symmetry, volume A. International Union of Crystallography, 2 edition, 2016. ISBN 978-0-470-97423-0. doi: 10.1107/97809553602060000114. URL https://it.iucr.org/Ac/. 11

Published as a conference paper at ICLR 2026

Sarp Aykent and Tian Xia. GotenNet: Rethinking Efficient 3D Equivariant Graph Neural Networks. In The Thirteenth International Conference on LearningRepresentations, 2025. Perla Azzi, Rodrigue Desmorat, Julien Grivaux, and Boris Kolev. Rationality of normal forms of isotropy strata of a representation of a compact lie group. arXiv preprint arXiv:2301.08599v1, 2023. Perla Azzi, Rodrigue Desmorat, Julien Grivaux, and Boris Kolev. On the isotropy stratification of a real representation of a compact lie group. arXiv preprint arXiv:2301.08599v2, 2026. Edward Bierstone. General position of equivariant maps. Transactions of the American Mathematical Society, 234(2):447–466, 1977. Glen E. Bredon. Introduction to compact transformation groups, volume 46. Academic Press, 1972. Michael M. Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges, 2021. Bradley CA Brown, Anthony L Caterini, Brendan Leigh Ross, Jesse C Cresswell, and Gabriel Loaiza-Ganem. Verifying the union of manifolds hypothesis for image data. arXiv preprint arXiv:2207.02862, 2022. Bin Cao, Yang Liu, Longhan Zhang, Yifan Wu, Zhixun Li, Yuyu Luo, Hong Cheng, Yang Ren, and Tongyi ZHANG. Beyond structure: Invariant crystal property prediction with pseudo-particle ray diffraction. In The Fourteenth International Conference on Learning Representations, 2026. Abel Carreras. pointgroup: Python library to determine the point symmetry group of molecular geometries, 2025. URL https://github.com/abelcarreras/pointgroup. Jiacheng Cen, Anyi Li, Ning Lin, Yuxiang Ren, Zihe Wang, and Wenbing Huang. Are high-degree representations really unnecessary in equivariant graph neural networks? In Advances in Neural Information Processing Systems, volume 37, pp. 26238–26266, 2024. Jiacheng Cen, Anyi Li, Ning Lin, Tingyang Xu, Yu Rong, Deli Zhao, Zihe Wang, and Wenbing Huang. Universally invariant learning in equivariant GNNs. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. Alexandre Agm Duval, Victor Schmidt, Alex Hernández-Garcıa, Santiago Miret, Fragkiskos D Malliaros, Yoshua Bengio, and David Rolnick. Faenet: Frame averaging equivariant gnn for materials modeling. In International Conference on Machine Learning, pp. 9013–9033. PMLR, 2023. Nadav Dym and Haggai Maron. On the universality of rotation equivariant point cloud networks. In International Conference on Learning Representations, 2021. Jinjia Feng, Zhewei Wei, Taifeng Wang, and Zongyang Qiu. TetraGT: Tetrahedral geometry-driven explicit token interactions with graph transformer for molecular representation learning. In The Fourteenth International Conference on Learning Representations, 2026. Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with pytorch geometric. arXiv preprint arXiv:1903.02428, 2019. Michael Field. Dynamics and symmetry, volume 3. World Scientific, 2007. Mario Geiger and Tess Smidt. e3nn: Euclidean neural networks. arXiv preprint arXiv:2207.09453, 2022. Martin Golubitsky and Victor Guillemin. Stable mappings and their singularities. Number 14 in Graduate texts in mathematics. Springer, 1973. ISBN 978-0-387-90073-5. doi: 10.1007/ 978-1-4615-7904-5. Martin Golubitsky, Ian Stewart, and David G. Schaeffer. Singularities and Groups in Bifurcation Theory, volume 2, number 69 of Applied Mathematical Sciences. Springer New York, 1988. ISBN 978-1-4612-8929-6 978-1-4612-4574-2. doi: 10.1007/978-1-4612-4574-2. 12

Published as a conference paper at ICLR 2026

Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning, volume 1. MIT Press, 2016. Victor Guillemin and Alan Pollack. Differential topology. Prentice Hall, 1974. ISBN 978-0-13212605-2. Jiaqi Han, Jiacheng Cen, Liming Wu, Zongzhao Li, Xiangzhe Kong, Rui Jiao, Ziyang Yu, Tingyang Xu, Fandi Wu, Zihe Wang, et al. A survey of geometric graph neural networks: Data structures, models and applications. Frontiers of Computer Science, 19(11):1911375, 2025. Morris W. Hirsch. Differential topology. Number 33 in Graduate texts in mathematics. SpringerVerlag, 1976. ISBN 978-1-4684-9451-8. doi: 10.1007/978-1-4684-9449-5. Wenbing Huang and Jiacheng Cen. Geometric graph learning for drug design. Deep Learning in Drug Design, pp. 133–151, 2026. E Ihrig and Martin Golubitsky. Pattern selection with O(3) symmetry. Physica D: Nonlinear Phenomena, 13(1-2):1–33, 1984. Chaitanya K Joshi, Cristian Bodnar, Simon V Mathis, Taco Cohen, and Pietro Lio. On the expressive power of geometric graph neural networks. In International conference on machine learning, pp. 15330–15355. PMLR, 2023. Sékou-Oumar Kaba and Siamak Ravanbakhsh. Symmetry breaking and equivariant neural networks. In NeurIPS 2023 Workshop on Symmetry and Geometry in Neural Representations, 2023. John M. Lee. Introduction to Smooth Manifolds, volume 218 of Graduate Texts in Mathematics. Springer, 2012. doi: 10.1007/978-1-4419-9982-5. Anyi Li, Jiacheng Cen, Songyou Li, Mingze Li, Yang Yu, and Wenbing Huang. Geometric mixture models for electrolyte conductivity prediction. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025a. Qi Li, Rui Jiao, Liming Wu, Tiannian Zhu, Wenbing Huang, Shifeng Jin, Yang Liu, Hongming Weng, and Xiaolong Chen. Powder diffraction crystal structure determination using generative models. Nature Communications, 16(1):7428, 2025b. Zongzhao Li, Jiacheng Cen, Wenbing Huang, Taifeng Wang, and Le Song. Size-generalizable RNA structure evaluation by exploring hierarchical geometries. In The Thirteenth International Conference on Learning Representations, 2025c. M. J. Linehan and G. E. Stedman. Little groups of irreps of O(3), SO(3), and the infinite axial subgroups. Journal of Physics A: Mathematical and General, 34(34):6663–6688, 2001. Eckhard Meinrenken. Group actions on manifolds, 2003. URL https://www.math.toronto. edu/mein/teaching/LectureNotes/action.pdf. Louis Michel. Symmetry defects and broken symmetry. configurations hidden symmetry. Reviews of Modern Physics, 52(3):617–651, 1980. doi: 10.1103/RevModPhys.52.617. John Willard Milnor. Topology from the differentiable viewpoint. Univ. Pr. of Virginia, 8. print edition, 1990. ISBN 978-0-8139-0181-7. G. D. Mostow. Equivariant embeddings in euclidean space. The Annals of Mathematics, 65(3):432, 1957. doi: 10.2307/1970055. Ikumitsu Nagasaki. The weak isovariant borsuk-ulam theorem for compact lie groups. Archiv der Mathematik, 81(3):348–359, 2003. doi: 10.1007/s00013-003-4693-1. Allan Pinkus. Approximation theory of the mlp model in neural networks. Acta numerica, 8:143–195, 1999. Omri Puny, Matan Atzmon, Edward J. Smith, Ishan Misra, Aditya Grover, Heli Ben-Hamu, and Yaron Lipman. Frame averaging for invariant and equivariant network design. In International Conference on Learning Representations, 2022. 13

Published as a conference paper at ICLR 2026

Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and O Anatole Von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data, 1(1):1–7, 2014. Vıctor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural networks. In International Conference on Machine Learning. PMLR, 2021. Tess E Smidt, Mario Geiger, and Benjamin Kurt Miller. Finding symmetry breaking order parameters with euclidean neural networks. Physical Review Research, 3(1):L012002, 2021. Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds. arXiv preprint arXiv:1802.08219, 2018. D. J. A. Trotman. Counterexamples in stratification theory: two discordant horns. Real and complex singularities, Oslo, pp. 679–686, 1976. Stephen Willard. General topology. Addison-Wesley series in mathematics. Addison-Wesley, 1970. ISBN 978-0-201-08707-9. Liming Wu, Zhichao Hou, Jirui Yuan, Yu Rong, and Wenbing Huang. Equivariant spatio-temporal attentive graph networks to simulate physical dynamics. Advances in Neural Information Processing Systems, 36, 2023. Liming Wu, Wenbing Huang, Rui Jiao, Jianxing Huang, Liwei Liu, Yipeng Zhou, Hao Sun, Yang Liu, Fuchun Sun, Yuxiang Ren, et al. Siamese foundation models for crystal structure prediction. arXiv preprint arXiv:2503.10471, 2025. Liming Wu, Rui Jiao, Qi Li, Mingze Li, Songyou Li, Shifeng Jin, and Wenbing Huang. Dmflow: Disordered materials generation by flow matching. arXiv preprint arXiv:2602.04734, 2026. Siyuan Zeng, Kuanping Gong, Yongquan Jiang, and Yan Yang. A method for predicting molecular point group based on graph neural networks. Artificial Intelligence Chemistry, pp. 100097, 2025. Yuelin Zhang, Jiacheng Cen, Jiaqi Han, Zhiqiang Zhang, JUN ZHOU, and Wenbing Huang. Improving equivariant graph neural networks on large geometric graphs via virtual nodes learning. In Forty-first International Conference on Machine Learning, 2024. Yuelin Zhang, Jiacheng Cen, Jiaqi Han, and Wenbing Huang. Fast and distributed equivariant graph neural networks by virtual node learning. arXiv preprint arXiv:2506.19482, 2025.

Published as a conference paper at ICLR 2026

C ONTENTS OF A PPENDIX A Backgrounds

A.1 Equivariant Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

A.2 Closed Subgroups of SO(3) or O(3) . . . . . . . . . . . . . . . . . . . . . . . . .

A.3 Manifolds and Basic Data Assumption . . . . . . . . . . . . . . . . . . . . . . . .

B Proofs of The Infimum of Symmetry (§ 3)

B.1 Proof of Thm. 3.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

B.2 Proof of Thm. 3.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

C Symmetry Increase for G = SO(3) or O(3) in § 4

C.1 General Orbit Type Criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

C.2 Proof of Prop. 4.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

C.3 Detailed Calculation in Ex. 4.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . .

C.4 Calculation of Symmetry Infimum . . . . . . . . . . . . . . . . . . . . . . . . . .

C.5 Composition Property of High-Multiplicity Representation . . . . . . . . . . . . .

D Proofs of Density of (Almost) Isovariant Maps (§ 5)

D.1 Counterexample in § 5.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

D.2 Proof of Thm. 5.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

D.3 Some Results on Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

D.4 Generic Equivariant Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . .

D.5 Proof of Thm. 5.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

E Tables

E.1 Minimal Proper Supergroups in SO(3) or O(3) . . . . . . . . . . . . . . . . . . .

E.2 Dimensions of Fixed-point Subspaces for Subgroups of SO(3) or O(3) . . . . . .

E.3 Symmetry Infimum for Subgroups of SO(3) . . . . . . . . . . . . . . . . . . . . .

E.4 Symmetry Infimum for Subgroups of O(3) . . . . . . . . . . . . . . . . . . . . .

F Detailed Experiment

F.1

Visualization of Representation Space . . . . . . . . . . . . . . . . . . . . . . . .

F.2

Expressivity on Symmetric Graphs . . . . . . . . . . . . . . . . . . . . . . . . . .

F.2.1

Embedding Difference Norm Experiment . . . . . . . . . . . . . . . . . .

F.2.2

Original GWL-Test on Symmetric Graphs . . . . . . . . . . . . . . . . . .

Molecule Property Prediction with Pretrained Equivariant Features . . . . . . . . .

F.3.1

Detailed Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . .

F.3.2

Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

F.3

Published as a conference paper at ICLR 2026

BACKGROUNDS

A.1

E QUIVARIANT N EURAL N ETWORKS

Equivariant Neural Networks (ENNs) have emerged as a cornerstone of modern machine learning, enabling a wide range of applications across the sciences (Han et al., 2025; Wu et al., 2023; Li et al., 2025b; Wu et al., 2025; Li et al., 2025c; Feng et al., 2026; Cao et al., 2026; Wu et al., 2026; Li et al., 2025a). Most mainstream ENNs adopt tensor product operators to design the message passing architecture (Thomas et al., 2018). In particular, since scalarization operations (e.g., norm and inner product) can substantially reduce computational cost, Cartesian vector-based networks (Satorras et al., 2021) and spherical scalarization networks (Cen et al., 2024; Aykent & Xia, 2025) have become particularly popular. More specifically for asymmetric graph structures, Cen et al. (2025) point out that networks employing scalarization operations already possess sufficient expressive power, i.e., universal approximation. However, additional subtleties arise in the presence of input symmetries. The interaction between architectural components and symmetric structures can lead to nontrivial representational effects. For example, techniques such as global virtual nodes (Zhang et al., 2024; 2025) and reference frames (Puny et al., 2022; Duval et al., 2023), though effective in improving model capacity, may exhibit unintended behaviors when applied to symmetric inputs. In particular, their use can potentially induce symmetry increase. In this work, we systematically analyze these phenomena under symmetric structures for general equivariant neural network architectures. A.2

C LOSED S UBGROUPS OF SO(3) OR O(3)

Before proceeding, we briefly review the classification of the closed subgroups of SO(3) or O(3). The closed subgroups of O(3), also known as point groups, are classified up to conjugacy as follows. Throughout this paper, we use the Schoenflies notation to denote these groups. • Finite Subgroups: These are divided into axial and polyhedral groups. The axial groups include the Abelian subgroups (Ck , S2k , Ckh ) and the non-Abelian subgroups (Ckv , Dk , Dkh , Dkd ). The polyhedral groups are all non-Abelian and comprise seven families (T, Td , Th , O, Oh , I, Ih ). Among these, the groups Ck , Dk , T, O, I consist solely of pure rotations and are subgroups of SO(3). • Infinite Subgroups: These include the cylindrical groups (C∞ , C∞h , C∞v , D∞ , D∞h ), which arise as limits of the axial groups, and the spherical groups, which are K = SO(3) and Kh = O(3). To facilitate the identification of input symmetries, we now provide a brief overview of the elemental structure of the key finite subgroups. The identification of cylindrical symmetries, which are infinite, can be achieved by taking the limits of these finite axial groups. A BELIAN A XIAL G ROUPS • Ck (Cyclic Group): Generated by a rotation ck of angle 2π/k, with k elements: (1) Rotations cjk about the principal axis. • S2k (Rotation-Reflection Group): Generated by adding a rotation-reflection element c2k σh to Ck , with 2k elements: (1) Rotations cjk about the principal axis. (2) Rotation-reflections c2j+1 2k σh about the principal axis. • Ckh (Cyclic Groups with Horizontal Reflection): Generated by adding a horizontal reflection plane σh to Ck , with 2k elements: (1) Rotations cjk about the principal axis. (2) Rotation-reflections cjk σh about the principal axis. 16

Published as a conference paper at ICLR 2026

N ON -A BELIAN A XIAL G ROUPS • Ckv (Cyclic Groups with Vertical Reflections): Generated by adding a vertical reflection plane σv to Ck , with 2k elements: (1) Rotations cjk about the principal axis. (2) Reflections cjk σv about the vertical plane. • Dk (Dihedral Groups): Generated by adding a 2-fold rotation axis u2 perpendicular to the principal axis of Ck , with 2k elements: (1) Rotations cjk about the principal axis. (2) 2-fold rotations cjk u2 about horizontal axes. • Dkh (Dihedral Groups with Horizontal Reflection): Generated by adding a horizontal reflection plane σh to Dk , with 4k elements: (1) Rotations cjk about the principal axis. (2) 2-fold rotations cjk u2 about horizontal axes. (3) Rotation-reflections cjk σh about the principal axis. (4) Reflections cjk σv about the vertical plane. • Dkd (Dihedral Groups with Dihedral Reflections): Generated by adding a diagonal reflection plane σd = c2k σv to Dk , with 4k elements: (1) Rotations cjk about the principal axis. (2) 2-fold rotations cjk u2 about horizontal axes. (3) Rotation-reflections c2j+1 2k σh about the principal axis. (4) Reflections c2j+1 2k σv about the diagonal plane. P OLYHEDRAL G ROUPS • T (Tetrahedral Group): The rotational symmetry group of a tetrahedron, with 12 elements. • Td (Full Tetrahedral Group): The full symmetry group of a tetrahedron, with 24 elements. • Th (Pyritohedral Group): Generated by adding an inversion center to T , with 24 elements. • O (Octahedral Group): The rotational symmetry group of a cube or octahedron, with 24 elements. • Oh (Full Octahedral Group): The full symmetry group of a cube or octahedron, generated by adding an inversion center to O, with 48 elements. • I (Icosahedral Group): The rotational symmetry group of an icosahedron, with 60 elements. • Ih (Full Icosahedral Group): The full symmetry group of an icosahedron, generated by adding an inversion center to I, with 120 elements. The subgroup relations among point groups, as well as the dimensions of fixed-point spaces of O(3) representations, can be readily determined. In particular, they can be derived from the minimal subgroup relations provided in § E.1 and from the dimension table for fixed-point spaces of irreducible O(3) representations established in § E.2. A.3

M ANIFOLDS AND BASIC DATA A SSUMPTION

Following the definition in Milnor (1990), a C r manifold M is mathematically defined as a subset of some linear space Rd . For every point x in this subset, there must exist a neighborhood W in Rd such that W ∩ M is C r -diffeomorphic to an open set in another linear space Rl . The integer l is known as the dimension of M . An alternative, intrinsic definition that does not depend on an embedding space can also be found in Hirsch (1976). Machine learning often assumes that the data distribution is supported on a submanifold of the input space. The dimension of this manifold characterizes the directions in which the data can vary within 17

Published as a conference paper at ICLR 2026

the input space. However, the term manifold is often used loosely in this context and does not perfectly align with its mathematical counterpart (Goodfellow et al., 2016). First, the data manifold may have self-intersections, causing the local dimension of data variation to differ at various points. Second, data belonging to different classes or clusters may possess different structures and potentially different dimensions. Lastly, stochastic factors such as observational noise can prevent the data from forming a strict surface in the input space. We will not address this last factor, instead, we assume that the interference from noise is minimal and ignorable. To address the first two issues, we can relax the manifold hypothesis by assuming that the data is supported on a finite union of submanifolds within the input space. Since a manifold with multiple connected components can itself be viewed as a disjoint union of connected manifolds, we can assume, without loss of generality, that each of these submanifolds is connected. For theoretical convenience, we further assume that these manifolds are bounded and closed. By the Weierstrass theorem, this is equivalent to assuming they are compact. We briefly outline the justification for these assumptions below: • Finite Union of Manifolds: This assumption has been partially verified experimentally on computer vision datasets (Brown et al., 2022). Theoretically, it encompasses classic cases of self-intersection. A data manifold may arise as the image of a map. However, the image of an immersion or submersion of a manifold can have self-intersections. The image of such a map from a compact manifold is, however, a finite union of compact manifolds. • Boundedness of Manifolds: This assumption stems from the natural assumption that the data distribution itself is bounded. • Closedness of Manifolds: This assumption covers scenarios where the data is defined by a set of well-behaved constraints. For example, the set of points satisfying n independent, differentiable constraint equations in the input space forms a closed submanifold of codimension n. In equivariant representation learning, the data possesses symmetries, meaning that data corresponding to the same physical object can be transformed by symmetry operations to represent different reference frames. These symmetry transformations typically form a group, and since the transformed data is still valid data, the data manifold must be closed under these group transformations, that is, the data manifold must be invariant under the group action. Considering the group action for any given input, we summarize the preceding discussions into the following assumptions about the data: • Lie Group Assumption: The transformation group G is a compact Lie group. • Manifold Assumption: The input space X is a linear space equipped with a linear action ρX of the group G. That is, there exists a map ρX : G → GL(X) such that ρX (g1 g2 ) = ρX (g1 )ρX (g2 ), and the identity element of the group is mapped to the identity transformation. The data manifold M is a finite union of compact, connected, smooth, and G-invariant submanifolds of X. The smoothness assumption is added for theoretical convenience and the connectedness assumption does not impose additional restrictions, since each compact smooth submanifold admits only finitely many connected components.

Published as a conference paper at ICLR 2026

P ROOFS OF T HE I NFIMUM OF S YMMETRY (§ 3)

B.1

P ROOF OF T HM . 3.1

Lemma B.1 (Azzi et al. (2023), Cor. 2.3). For a compact Lie group G and a representation V , let v ∈ V be any point. There exists a neighborhood U of v such that for any point u ∈ U , we have (Gu ) ≤ (Gv ). Lemma B.2 (Azzi et al. (2023), Cor. 2.20). For a reductive group G and an affine algebraic variety X, let x ∈ X be a point with a closed orbit G(x). There exists a Zariski neighborhood U of x such that for any point u ∈ U , we have (Gu ) ≤ (Gx ).

We use the definition of the complexification from Prop. 3.3 of Azzi et al. (2023). For a real vector space V , its complexification is V C := C ⊗ V . Considering an orthogonal action of G on V , with g being the Lie algebra of G, the complexification of the group is GC := {g exp(iX) | g ∈ G, X ∈ g}. By the above complexification, we obtain the following lemma. Lemma B.3. For a compact Lie group G, consider the inclusion map ι : G ,→ GC into its complexification. Let H and K be closed subgroups of G. Then (H) ≤ (K)

⇐⇒

(H C ) ≤ (K C ),

(12)

where the two partial orders are taken with respect to conjugation in G and G , respectively. C

Proof. The reverse implication follows from Prop. 4.9 of Azzi et al. (2026). For the forward direction, let g0 ∈ G be an element such that g0 Hg0−1 ⊆ K by (H) ≤ (K). Let h and k be the Lie algebras of H and K, respectively. Any element of H C can be expressed as h0 exp(iX0 ) for h0 ∈ H and X0 ∈ h. Its conjugation by g0 is g0 h0 exp(iX0 ) g0−1 = (g0 h0 g0−1 ) exp(iAdg0 X0 ). (13) Since g0 Hg0−1 ⊆ K, the term g0 h0 g0−1 belongs to K. The expression thus belongs to K C provided that Adg0 X0 ∈ k.

This condition on the Lie algebra is obtained by differentiating the subgroup inclusion g0 Hg0−1 ⊆ K at the identity element, which gives Adg0 (h) ⊆ k. As X0 ∈ h, it follows immediately that Adg0 X0 ∈ k. Then, ι(g0 )H C ι(g0 )−1 ⊆ K C , (14) which complete the proof.

Proposition B.4. Let V be a representation of a compact Lie group G. For any closed subgroup H of G, the set of points of minimal orbit type in the fixed-point subspace V H is a dense and open subset of V H . Proof. The proof strategy is similar to that of Prop. 2.10 in Azzi et al. (2023). We first show that the set of points with a minimal orbit type is open, then use complexification to show it is Zariski-open, which implies density. First, to prove openness, let (K) be a minimal orbit type in V H and let x ∈ V H be a point with (Gx ) = (K). By Lem. B.1, there exists a neighborhood U of x such that for any u ∈ U , (Gu ) ≤ (K). Since (K) is minimal in V H , any point in the open set U ∩ V H must have orbit type (K). Thus, openness is established. Next, we consider the complexifications GC and V C . For a point x ∈ V with orbit type (K), by Lem. 3.13 of Azzi et al. (2023), we have (GC )x = (Gx )C = K C . By Prop. 3.14 of Azzi et al. (2023), the orbit GC (x) is closed. Therefore, by Lem. B.2, there exists a Zariski neighborhood U of x in V C such that any point in this neighborhood has an isotropy type less than or equal to (K C ). The C C set U ′ = U ∩ (V C )H is Zariski-open in (V C )H . By Cor. A.7 of Azzi et al. (2023), its real part ′′ ′ U = U ∩ V contains a non-empty real Zariski-open subset of V H . 19

Published as a conference paper at ICLR 2026

We now show that for any y ∈ U ′′ , its orbit type is (Gy ) = (K). Since y ∈ U , Lem. B.2 gives ((GC )y ) ≤ (K C ). By Lem. 3.13 of Azzi et al. (2023), we have (GC )y = (Gy )C . Therefore, the reverse implication of Lem. B.3 yields (Gy ) ≤ (K). Since y ∈ V H , the orbit type (Gy ) occurs among the orbit types in V H . For Lem. B.3, by the minimality of (K), the preceding inequality must be an equality (Gy ) = (K). Since U ′′ contains a nonempty real Zariski-open subset of V H , the set of points of orbit type (K) is dense in V H . Together with the openness proved above, this completes the proof.

Theorem 3.1 (Uniqueness of Minimal Type). Let X be a representation of a compact Lie group G. For any closed subgroup H ⊆ G, a unique minimal orbit type exists among the points in the fixed-point subspace X H . In particular, if (H) ∈ OG (X H ), then (H) is the minimal orbit type within that subspace. Proof of Thm. 3.1. The density and openness of the set of points corresponding to any minimal orbit type is established by Prop. B.4. The uniqueness then follows from the fact that two distinct open dense sets must have a non-empty intersection.

B.2

P ROOF OF T HM . 3.3

Theorem 3.3. A necessary condition for the existence of an isovariant map relative to Y from a G-set X to a G-set Y is that (pY (H)) ∈ OG (Y ) for every (H) ∈ OG (X). When X and Y are representations, this is equivalent to the condition that IG (Y, H) = (pY (H)) for all (H) ∈ OG (X). Proof of Thm. 3.3. The necessity for general G-sets follows directly from the definition of relative isovariance. Now suppose that X and Y are representations. For any closed subgroup H ⊆ G, (pY (H)) ≤ IG (Y, H),

(15)

since every isotropy subgroup occurring in Y H contains both H and ker ρY . If (pY (H)) ∈ OG (Y ), then this orbit type is realized in Y H , because H ⊆ pY (H). Hence, by the minimality of IG (Y, H), IG (Y, H) ≤ (pY (H)).

(16)

Therefore IG (Y, H) = (pY (H)). The converse is immediate since IG (Y, H) is itself an orbit type occurring in Y .

Published as a conference paper at ICLR 2026

S YMMETRY I NCREASE FOR G = SO(3) OR O(3) IN § 4

C.1

G ENERAL O RBIT T YPE C RITERION

To derive Prop. 4.2, we need more general results require criteria for identifying orbit types of general representation. Given the sufficiency of the Ihrig-Golubitsky Criterion (Prop. C.1), we demonstrate that for high-multiplicity representations, the conditions of the Michel Criterion imply the conditions of the Ihrig-Golubitsky Criterion. This, in turn, establishes the sufficiency of the Michel Criterion. Let G be a compact Lie group and X be the input space. The normalizer of a subgroup H ⊆ G is NG (H) := {g ∈ G | gHg −1 = H}. The normalizer of H relative to a supergroup H ′ ⊃ H is NG (H, H ′ ) := {g ∈ G | H ⊂ gH ′ g −1 }. Based on the above definitions, we recall the following general criterion. Proposition C.1 (Ihrig-Golubitsky Criterion, Ihrig & Golubitsky (1984), Prop. 5.3). Let V be a faithful representation of a group G. A sufficient condition for a closed subgroup H to be an isotropy subgroup in V is that for every orbit type (H ′ ) in V with (H ′ ) > (H), the following inequality holds: ′

′

dim V H + αG (H, H ′ ) := dim V H + dim NG (H, H ′ ) − dim NG (H ′ ) < dim V H ,

(17)

where αG (H, H ′ ) is the Ihrig-Golubitsky correction term. Remark. Ihrig & Golubitsky (1984) states that the condition is also necessary for G = SO(3), O(3). Compared to Prop. C.1, the Linehan-Stedman Criterion (Linehan & Stedman, 2001) is more convenient due to its structure, which only requires checking adjacent subgroups. Since the associated theoretical results are not needed in our proofs, we do not elaborate on them here. C.2

P ROOF OF P ROP. 4.2

Proposition 4.2. For a high-multiplicity representation V , the necessary condition stated in Thm. 4.1 is also sufficient. Proof of Prop. 4.2. We show that for representations where each irreducible component has multiplicity r > dim G, the necessary condition from Michel’s Criterion (Thm. 4.1) becomes sufficient by proving it implies the condition in Prop. C.1. We may assume without loss of generality that the representation is faithful. For unfaithful representations, we proceed by factoring out the kernel from G and invoking Prop. C.1, as the procedure remains unchanged. ′

It follows from the condition that for any closed subgroup H ′ of G containing H, we have dim V H < dim V H . The condition implies that for at least one irreducible component Vi , we must have ′ ′ dim ViH < dim ViH . Since dimensions are integers, this is equivalent to dim ViH + 1 ≤ dim ViH . Given that the multiplicity m(Vi , V ) > dim G, it follows that ′

m(Vi , V ) dim ViH + dim G < m(Vi , V ) dim ViH .

(18)

Summing this strict inequality for one such component with the non-strict inequalities for all other components yields ′ dim V H + dim G < dim V H . (19) The term from Prop. C.1 is bounded by 0 ≤ αG (H, H ′ ) = dim NG (H, H ′ ) − dim NG (H ′ ) ≤ dim G.

(20)

The sufficiency of Michel’s condition follows from combining these two results: ′

dim V H < dim V H

=⇒ =⇒

′

dim V H + dim G < dim V H dim V

H′

′

(21) ′

+ dim NG (H, H ) − dim NG (H ) < dim V ,

(22)

where the final implication uses the upper bound from Eq. (20). This shows that Michel’s condition implies the sufficient condition from Prop. C.1. 21

Published as a conference paper at ICLR 2026

C.3

D ETAILED C ALCULATION IN E X . 4.3

We now conduct the orbit-type test and symmetry infimum calculation for the geometric symmetry Dkh (k > 2) of the k-fold from Ex. 2.2. Consider Dkh as a closed subgroups of O(3), the calculation ⊕r is performed in the representation space Y = Vl=l for r > 3 and l0 > 0. We will use the pre0 computed subgroup relations from Table 6, the dimensions of the fixed-point spaces for Dkh and Oh ⊕r in Vl=l0 from Table 8, and the orbit type test results for (D∞h ) and (Oh ) in Vl=l . 0 First, we perform the orbit type test. Consider the case where k ̸= 4. In this situation, the adjacent supergroups of Dkh are of the form Dpk,h , where p is a prime number. We classify the discussion into 12 cases based on the parity of k and l0 , and the ranges l0 < k, k ≤ l0 < 2k, and l0 ≥ 2k, and calculate the dimensions of the fixed-point spaces of Vl=l0 for Dkh and Dpk,h . The calculated dimensions are shown in Table 2. For the case k = 4, we must additionally consider the supergroup D4h Oh Oh . Here, dim Vl=l = dim Vl=l only when l0 is odd, which aligns with the results from Table 2. 0 0 In all other cases, Oh does not affect the orbit type test of (Dkh ). Thus, we find that (Dkh ) is an orbit type when l0 ≥ k and when l0 and k have the same parity. ⊕r Table 2: Dimension of fixed-point space for supergroups of Dkh (k > 2) in Vl=l (r > 3), organized 0 by the parity of k and l0 . l0 < k k ≤ l0 < 2k l0 ≥ 2k l0 is even l0 is odd l0 is even l0 is odd l0 is even l0 is odd k is even Dkh 1 0 2 0 d 0 Dpkh 1 0 1 0 <d 0 k is odd Dkh 1 0 1 1 d d D2kh 1 0 1 0 d 0 Dp∗ kh 1 0 1 0 <d <d

Next, we proceed with the symmetry infimum calculation. Again, we first consider the case where k ̸= 4. In this situation, the supergroups of Dkh are Dpk,h , D∞h , and O(3), where O(3) is always an orbit type. Using the orbit type test results just calculated and those pre-computed, we analyze the cases based on the parity of k and l0 as before, with the results summarized in Table 3. For k = 4, the additional supergroup Oh does not affect the final result. This is because the supergroup Oh is never the minimal isotropy supergroup, as it only becomes an orbit type when (D4h ) already is. This leads us to the symmetry infimums shown in Table 1.

⊕r Table 3: Isotropy conditions for supergroups of Dkh (k > 2) in Vl=l (r > 3), organized by the parity 0 of k and l0 . A ✓ indicates the condition is satisfied, and ✗ that it is not. In the subgroup notation, p denotes a prime number and p∗ denotes an odd prime number. l0 < k k ≤ l0 < 2k l0 ≥ 2k l0 is even l0 is odd l0 is even l0 is odd l0 is even l0 is odd k is even Dkh ✗ ✗ ✓ ✗ ✓ ✗ Dpkh ✗ ✗ ✗ ✗ D∞h ✓ ✗ ✗ ✗ k is odd Dkh ✗ ✗ ✗ ✓ ✗ ✓ D2kh ✗ ✗ ✗ ✓ Dp∗ kh ✗ ✗ ✗ D∞h ✓ ✗ ✓ -

Published as a conference paper at ICLR 2026

C.4

C ALCULATION OF S YMMETRY I NFIMUM

⊕r ⊕r We calculate the orbit types for the SO(3) representation Vl=l and the O(3) representation Vl=l ±. 0 0 As the calculation procedure is highly similar to that in Ex. 4.3, we omit the detailed steps here. For the case of r = 1, the results can be found in Table B.1 and Table B.2 of Linehan & Stedman (2001).

The orbit types are calculated using the procedure in Algo. 1. This calculation requires the dimensions of the fixed-point spaces for the closed subgroups of SO(3) or O(3) in the representations Vl=l0 and Vl=l± , respectively, which are provided in § E.2. 0

According to Prop. 6.2 from Ihrig & Golubitsky (1984), the Ihrig-Golubitsky correction term α(H, H ′ ) = 0 for all subgroups except H = Ck in SO(3), and for all subgroups except H = Ck , S2k , Ckh in O(3). Therefore, for all subgroups other than H = Ck , S2k , Ckh , our results are identical to those calculated by Linehan & Stedman (2001) for r = 1. We present only the results ⊕r ⊕r of whether (Ck ) ∈ OSO(3) (Vl=l ) on Table 4 and whether (Ck ), (S2k ), (Ckh ) ∈ OO(3) (Vl=l − ) on 0 0 Table 5. ⊕r The reason for omitting the discussion of Vl=l + is consistent with the explanation in the header of 0 Table B.2 in Linehan & Stedman (2001). This is because the corresponding conclusions can be ⊕r found in the orbit type table for the SO(3) representation Vl=l via the mappings D∞ → D∞h , 0 C∞ → C∞h , I → Ih , O → Oh , T → Th , as well as Dk → Dkh and Ck → Ckh for even k, and Dk → Dkd and Ck → S2k for odd k. ⊕r Table 4: Modified isotropy subgroups for the multiple irreducible representation Vl=l , r > 3 of 0 SO(3) obtained via the Michel criterion. H Condition of l0

l0 ≥ k

⊕r Table 5: Modified isotropy subgroups for the multiple irreducible representation Vl=l − , r > 3 of 0

O(3) with nontrivial reflection, obtained via the Michel criterion. H Condition of l0 l0 ≥ k l0 ≥ k l0 ≥ k

Ck S2k (k is even) Ckh (k is odd)

The complete set of orbit type calculation results can also be found in the tables for the symmetry infimum. This is because (H) is an orbit type of a representation V if and only if the symmetry infimum of H in V is (H) itself. These non-degenerate cases are highlighted in green in the tables. ⊕r Using the calculated orbit types for the SO(3) representation Vl=l and the O(3) representation 0 ⊕r Vl=l± , we can now compute the symmetry infimum for all subgroups of SO(3) or O(3) in these 0 representations. When calculating the symmetry infimum for all subgroups, the algorithm differs slightly from that in Ex. 4.3. We reduce the need to enumerate all supergroups by leveraging the symmetry infimums of adjacent supergroups. An improved version of Algo. 2 is detailed in Algo. 3.

To reduce the computational load, this algorithm employs a top-down calculation strategy. First, we compute the results for the infinite groups, followed by the polyhedral groups. Finally, we calculate the results for the axial groups in the order Dkh , Dkd , Dk , Ckv , Ckh , S2k , Ck . Due to the exceptional subgroup relations shown in Table 6, special consideration for additional supergroups is required for certain cases where k ∈ {1, 2, 3, 4, 5}. These cases are handled last. The results for SO(3) are presented in § E.3, and the results for O(3) are in § E.4. Here, we provide a general classification for the observed symmetry increases. For a closed subgroup H, an increase 23

Published as a conference paper at ICLR 2026

Algorithm 3: Symmetry Infimum Calculation With Precomputed Results Data: A symmetry group G; a closed subgroup H ⊂ G; a Rep. V of G; a map M of previously computed symmetry infs. Result: Symmetry inf. IG (V, H). 1 if is_in((H), OG (V )) then 2 return (H); 3 end 4 Let S0 = {Hi } ⊂ G to be all adjacent closed supergroups of H in G; 5 O ← ∅; 6 for Hi in S0 do 7 if is_in((Hi ), OG (V )) then 8 Add (Hi ) to the set O; 9 end 10 else if Hi is a key in the map M then 11 Add M [Hi ] = IG (V, Hi ) to the set O; 12 end 13 else 14 Let Si = {Kj } to be all adjacent closed supergroups of Hi in G; 15 for Kj in Si do 16 if (Kj ) ∈ OG (V ) then 17 Add (Kj ) to the set O; 18 end 19 end 20 end 21 end 22 return min(O);

to the full group O(3) or SO(3) is termed full degeneration and marked in red in the tables. An increase to a supergroup of higher dimension than H is termed continuous degeneration and marked in blue . An increase to a supergroup of the same dimension as H is termed discrete degeneration and marked in yellow or light green . No symmetry increase is no degeneration and marked in green . For the subgroup Dkh ⊂ O(3), the classification of degeneration behaviors given after Ex. 4.3 is a special case of this general framework. Specifically, full degeneration is identical to the definition here, axial degeneration corresponds to continuous degeneration, and half degeneration corresponds to discrete degeneration. ⊕r We note that for the representation Y = Vl=l + , the action of G has a non-trivial kernel, and thus 0

symmetry increase is inevitable. Let πY be the natural projection to the quotient G/kerρY In this context, the light green marks an increase to πY−1 (πY (H)), which, as explained in § 3.2, is the lowest possible symmetry that H can be reduced to when a non-trivial kernel is present. This is therefore a predictable behavior. The yellow marks other exceptional cases within discrete degeneration. C.5

C OMPOSITION P ROPERTY OF H IGH -M ULTIPLICITY R EPRESENTATION

In the tables of § E.3 and § E.4, there exists a special class of subgroups H for which any non-trivial symmetry increase always results in an infimum (H0 ) where H0 is an adjacent supergroup. We call (H0 ) the bottleneck of H, denoted by BG (H), and we say that a subgroup H that possesses a bottleneck satisfies the bottleneck condition. These groups that act as bottlenecks satisfy some elegant properties. To demonstrate this, we first prove a structure theorem for the symmetry infimum of high-multiplicity representations. 24

Published as a conference paper at ICLR 2026

Proposition C.2. For a high-multiplicity T representation V , the lowest orbit type in the fixed-point subspace V H is (GV H ), where GV H = x∈V H Gx is the largest subgroup that leaves V H invariant. This shows that IG (V, H) = (GV H ). Proof. Since any element in V H has at least the symmetry GV H , we only need to prove that GV H is an isotropy subgroup. Assume, for the sake of contradiction, that GV H is not an isotropy subgroup. By the sufficiency of the Michel Criterion, there exists a supergroup K such that GV H ⊊ K and dim V H = dim V GV H = dim V K =⇒ V H = V GV H = V K . This means that K leaves all elements of V H fixed. From this, we derive a contradiction: \ K⊂ Gx = GV H .

(23)

(24)

x∈V H

Remark. This result implies that for a high-multiplicity representation, the symmetry increase from (H) to IG (V, H) does not alter the dimension of the fixed-point space. Therefore, for high-multiplicity representations, the dimension of the fixed-point subspace corresponding to the input symmetry group equals that corresponding to the symmetry infimum. Since the fixed-point subspace dimensions for certain closed subgroups exhibit distinct regularities, the behavior of symmetry increase toward these subgroups serves as an indicator of the underlying subspace dimension. For instance, in the cases of SO(3) or O(3), for non-trivial representations, full degeneration corresponds to a 0-dimensional fixed-point subspace, whereas for finite input subgroups, continuous degeneration corresponds to a 1-dimensional subspace. However, we must emphasize that when a quantitative assessment of the equivariant feature’s expressive capacity is required, relying solely on the symmetry infimum is insufficient, as it yields only coarse, qualitative insights. In such cases, one should directly compute the fixed-point subspace dimension. For calculations related to SO(3) or O(3), see § E.2. We can prove that these groups acting as bottlenecks satisfy the property that for any irreducible rep′ resentation V0 , if dim V0H > dim V0H0 , then dim V0H > dim V0H holds for all adjacent supergroups ′ ′ H of H. This is because if we assume there exists an H such that equality holds, then by the Michel Criterion, (H) must undergo a non-trivial symmetry increase to (GV H ) in the high-multiplicity representation V0⊕r . Since these increases always have the bottleneck (H0 ) as their infimum, the inequalities G dim V0H ≥ dim V0H0 ≥ dim V0 V H (25) must in fact collapse to dim V0H = dim V0H0 , which leads to a contradiction.

For any high-multiplicity representation V that satisfies dim V H > dim V H0 , there must exist a component corresponding to an irreducible representation V0 in V such that dim V0H > dim V0H0 . ′ It follows that dim V0H > dim V0H for all adjacent supergroups H ′ of H, which in turn shows that ′ dim V H > dim V H holds for all such H ′ . Therefore, for high-multiplicity representations, H0 controls the dimension gap of the fixed-point spaces between H and its other adjacent supergroups. This property allows us to prove a theorem regarding the direct sum of high-multiplicity representations, which in turn establishes a property for the orbit types of such direct sums. Theorem C.3. Let a subgroup H satisfy the bottleneck condition. For two high-multiplicity representations V1 and V2 , we have (H) ∈ OG (V1 ⊕ V2 ) if and only if (H) ∈ OG (V1 ) or (H) ∈ OG (V2 ). Proof. We have already shown that for any high-multiplicity representation V , if dim V H > ′ dim V H0 , then dim V H > dim V H holds for all adjacent supergroups of H. Therefore, assuming (H) ∈ OG (V1 ⊕ V2 ), the Michel Criterion gives us ′

′

dim V1H + dim V2H > dim V1H + dim V2H .

(26)

By taking H ′ = H0 = BG (H), we see that at least one of the following two conditions must be true: dim ViH > dim ViH0 , 25

i = 1, 2.

(27)

Published as a conference paper at ICLR 2026

This implies that for all adjacent supergroups of H, at least one of the following two conditions must hold: ′ dim ViH > dim ViH , i = 1, 2. (28) Therefore, (H) is an orbit type of either V1 or V2 . The above theorem shows that when we construct a high-multiplicity representation for which (H) is an orbit type using high-multiplicity components, the only way is to find a high-multiplicity component that already contains (H) as an orbit type. For SO(3), all closed subgroups satisfy the bottleneck condition. Consequently, for high-multiplicity representations, we have OG (V1 ) ∪ OG (V2 ) = OG (V1 ⊕ V2 ). This means that IG (V1 ⊕ V2 , H) will ⊕r be the minimum of IG (V1 , H) and IG (V2 , H). Furthermore, the set of representations {Vl=l } is 0 sufficient to generate all closed subgroups as orbit types, because for any closed subgroup, there always exists a high-multiplicity representation for which it is an orbit type. For O(3), the bottleneck condition does not necessarily hold. For example, for H = C∞ , a symmetry increase can result in either D∞ or C∞v , which shows that no bottleneck group exists. The fact that ⊕r C∞ never appears as an orbit type in any representation Vl=l ± demonstrates this point precisely. 0

This introduces a subtle issue when we apply the guideline on § 4.2: for certain orbit types, a representation exhibiting the target orbit type cannot be constructed simply by including the component ⊕r Vl=l ± associated with it. Fortunately, C∞ is the sole instance of this phenomenon encountered when 0

G = O(3). Regarding the construction of a O(3) representation containing C∞ , following Prop. 4.2, ⊕r it suffices to simultaneously select components Vl=l − with both odd and even degrees l0 > 0. 0

Published as a conference paper at ICLR 2026

P ROOFS OF D ENSITY OF (A LMOST ) I SOVARIANT M APS (§ 5)

D.1

C OUNTEREXAMPLE IN § 5.1

Lemma D.1 (Borsuk-Ulam Theorem, Guillemin & Pollack (1974), Chap. 2, Sec. 6). For any continuous odd function g : S n → Rn , there exists a point x ∈ S n such that g(x) = 0. Lemma D.2 (Weak Borsuk-Ulam Theorem, Nagasaki (2003), Thm. A). Let M and N be G-spheres in a representation space for a compact group G. If there exists an equivariant map from M to N , then the following dimensional inequality holds: φG (dim M − dim M G ) ≤ dim N − dim N G ,

(29)

where φG : N → N is a non-decreasing function that diverges to infinity.

Counterexample D.3. For a compact Lie group G, consider a representation Ỹ with no trivial component, i.e. Ỹ G = {0}. Let Y = Ỹ ⊕r and X = Y ⊕(n0 +1) for some r, n0 > dim G. By Prop. 4.2, we have OG (X) = OG (Y ), yet no isovariant map exists from the unit sphere in X to Y for a sufficiently large integer n0 . In particular, for G = Z2 , if Ỹ is the non-trivial irreducible representation, then for any X = Ỹ ⊕r1 and Y = Ỹ ⊕r2 with r1 > r2 , no isovariant map exists from the unit sphere of X to Y . Proof. For any compact Lie group G, consider an equivariant map f from a G-sphere M in a vector space X to a G-representation Y . We assume that the multiplicities of the trivial representation components in both X and Y are zero, i.e., X G = {0} and Y G = {0}. Therefore, only the origin in X and Y has the orbit type (G), and the sphere M does not contain any point of orbit type (G). Consequently, if f is an isovariant map, it must have no zeros. From such a zero-free map f , we can define a map f˜ to the G-sphere N in Y : f˜ : M → N,

f˜(x) = f (x)/∥f (x)∥2 .

(30)

Since scaling in a vector space does not change the orbit type, f˜ is also an isovariant map. Therefore, by Lem. D.2, we obtain the dimensional relation: φG (dim M − dim M G ) = φG (dim X − 1) ≤ dim N − dim N G = dim Y − 1.

(31)

Since φG is a non-decreasing function that diverges to infinity, there must exist an integer n0 > dim G such that φG (n0 ) > dim Y − 1. This implies that φG (dim(Y ⊕(n0 +1) ) − 1) > dim Y − 1.

(32) ⊕(n0 +1)

This shows that no isovariant map can exist from the unit G-sphere M in the space X = Y to the space Y . To complete the counterexample, let Y = Ỹ ⊕r , where r > dim G. By Prop. 4.2, we have OG (Y ) = OG (X), yet no isovariant map exists between them.

For the special case of G = Z2 , let Ỹ be the non-trivial irreducible representation. Let X = Ỹ ⊕r1 and Y = Ỹ ⊕r2 with r1 > r2 . A map f from the unit sphere in X to Y is equivariant if and only if it is an odd function. Since the representations have no trivial components, an isovariant map is equivalent to an odd function that has no zeros. However, by Lem. D.1, any odd map from a sphere in a higher-dimensional space to a lower-dimensional space must have a zero. Thus, no such isovariant map exists. D.2

P ROOF OF T HM . 5.1

TFN is an ENN based on tensor products of hidden features. The TFN architecture is composed of a feature lifting map followed by an equivariant pooling stage. The feature lifting map contains equivariant convolutional layers that update node features by aggregating information from neighbors. These convolutions employ filters built from learnable radial functions parameterized by a Multi-Layer Perceptron (MLP) and real spherical harmonics Ylm . Mathematically, the parameterized map fTFN ∈ FTFN is defined as a sum over feature channels. Each term in the sum is a composition fpool ◦ ffeat , consisting of a feature lifting map followed by 27

Published as a conference paper at ICLR 2026

equivariant linear pooling. Let Z represent the hidden features associated with each node. The feature lifting map ffeat takes the input point cloud X to node features Z ⊗ Rn . This map is constructed as a composition of layers: ffeat = πZ⊗Rn ◦ (f (L) , id) ◦ · · · ◦ (f (1) , id) ◦ ext ◦ C,

(33)

where C is a centering operation, ext is a constant extension map, and each layer f (k) updates the node features v (k−1) to v (k) according to the rule: P P (k) (k−1) (l ,m ) (l ) (k−1) vil3 m3 = θvil3 m3 + j̸=i l1 ,m1 ,l2 ,m2 C(l23,m23),(l1 ,m1 ) Fm22 (xi − xj )vjl1 m1 . (34) (l ,m )

Here, vil3 m3 corresponds to the feature of type l3 at node i, and C(l23,m23),(l1 ,m1 ) are the ClebschGordan coefficients. For G = O(3) × Sn , the parities of l1 and l3 must also be considered. For each degree l and order m, let Ylm denote the corresponding solid spherical harmonic. It satisfies Ylm (x) = ∥x∥l2 Ylm (x/∥x∥2 ) ,

x ̸= 0,

(35)

and extends to x = 0 as a homogeneous harmonic polynomial of degree l. We define the filter function by (l) Fm (x) = hl (∥x∥22 )Ylm (x). (36) This parameterization is a mild modification of the commonly used implementation (l) Fem (x) = e hl (∥x∥2 )Ylm (x/∥x∥2 ) .

(37)

Indeed, away from the origin, the factor ∥x∥l2 can be absorbed into the radial function by setting e hl (r) = rl hl (r2 ), which can in turn be approximated by a radial MLP. We adopt the solid-harmonic form because it yields a globally well-defined filter, including at x = 0. Moreover, this adaptation does not affect the universality result of Dym & Maron (2021), since the polynomial filters used in their construction admit an equivalent representation in the solid-harmonic form above. When the radial function hl : R≥0 → R is parameterized by a Multi-Layer Perceptron (MLP), we denote by FTFN the resulting family of TFN filters under MLP-parameterized radial functions. In contrast, when hl is chosen as polynomial parameterization, we denote the corresponding parameteripoly poly zation by FTFN . Notice that we use FTFN and FTFN denotes the union of these architectures over the number of layers, feature channels, and maximal representation degree. We use L(X, Y ) to denote the vector space of linear maps X → Y , and P (X, Y ) the vector space of polynomial maps X → Y . Given the G-actions on X and Y , we write LG (X, Y ) ⊆ L(X, Y ) and PG (X, Y ) ⊆ P (X, Y ) for the subspaces of G-equivariant maps, i.e., f (g(x)) = g(f (x)) for all g ∈ G and x ∈ X. If Y carries the trivial action, equivariance reduces to invariance. Lemma D.4. Let X, Y, Z be representations of a group G. Consider a subset of G-equivariant polynomial maps S ⊆ PG (X, Z) that satisfies the spanning condition: P (X, Y ) ⊆ span(L(Z, Y ) ◦ S) := span({A ◦ p | A ∈ L(Z, Y ), p ∈ S}).

(38)

Then, it follows that PG (X, Y ) = span(LG (Z, Y ) ◦ S).

In the special case where G = H1 × H2 , the spanning condition can be relaxed to PH2 (X, Y ) ⊆ span(L(Z, Y ) ◦ S).

(39)

Remark. The case for G = SO(3) × Sn is from Thm. 1 of Dym & Maron (2021). In this context, the action of Sn on the representations is faithful, while we are concerned with features that are invariant under Sn . Proof. Let f ∈ PG (X, Y ). Since PG (X, Y ) ⊆ P (X, Y ), by the spanning condition, f can be written as a linear combination of compositions: f=

N X i=1

Ai ◦ p i ,

(40)

Published as a conference paper at ICLR 2026

where pi ∈ S and Ai ∈ L(Z, Y ). Here, we have absorbed the expansion coefficients into the linear maps Ai . Since f is G-equivariant, it is a fixed point of the group averaging operator. Applying this operator to both sides of the equation gives: Z f (x) = ρY (g −1 )f (ρX (g)(x)) dg (41) G ! Z N X −1 = ρY (g ) Ai (ρX (g)(x)) dg (42) G

i=1

N Z X i=1

ρY (g

−1

)Ai ρZ (g) dg pi (x).

(43)

In the last step, we used the fact that the maps pi ∈ S are themselves G-equivariant and moved the integral inward. Let us define the averaged linear maps as Z (AG )i := ρY (g −1 )Ai ρZ (g) dg. (44) G

By construction, each (AG )i is a G-equivariant linear map, i.e., (AG )i ∈ LG (Z, Y ). The expression PN for f (x) can now be written as f (x) = i=1 (AG )i ◦ pi (x). This shows that any map in PG (X, Y ) can be expressed as a linear combination of compositions of maps from S and equivariant linear maps from LG (Z, Y ). Therefore, PG (X, Y ) ⊆ span(LG (Z, Y ) ◦ S).

(45)

The reverse inclusion, span(LG (Z, Y ) ◦ S) ⊆ PG (X, Y ), is true by definition, since the composition of two equivariant maps is equivariant. Thus, the equality PG (X, Y ) = span(LG (Z, Y ) ◦ S) is established. The proof for the relaxed condition when G = H1 × H2 follows the exact same logic, by taking an f ∈ PG (X, Y ) ⊆ PH2 (X, Y ) and averaging over the group H1 . Lemma D.5. Consider the TFN model with polynomial parametric radial function, a input space X, a lifted representation space Z, and a final output space Y . Suppose the final output space Y = Ỹ ⊗ Rn , where Ỹ is a representation of SO(3) or O(3), and the symmetric group Sn acts on Y by permuting the coordinates in the Rn factor. Then the family of polynomial feature maps, poly Ffeat ⊆ PG (X, Z), used in TFN satisfies the relaxed spanning condition: poly PSn (X, Y ) ⊆ span(L(Z, Y ) ◦ Ffeat ).

(46)

Remark. Lem. 4 in Dym & Maron (2021) proves the case for G = SO(3) × Sn . The proof for O(3) is similar to that for SO(3) and is therefore not repeated here. It is worth noting that in the O(3) case, the spherical harmonics map can still be used to construct the component-wise map from representation Vl=1− ∼ = R3 to symmetric algebra Symk (Vl=1− ), which in turn is used to build the lifted representation. poly Lemma D.6. In Ex. 2.1, the function families FTFN contain all equivariant polynomial maps.

Proof. According to Lem. D.4, to prove that a family of equivariant maps contains all equivariant polynomials, it is sufficient to show that it satisfies the (potentially relaxed) spanning condition. We poly verify this for FTFN . poly By Lem. D.5, for Y = Ỹ ⊗ Rn , the family Ffeat satisfies the spanning condition poly PSn (X, Y ) ⊆ span(L(Z, Y ) ◦ Ffeat ).

(47)

We now only need to verify the following spanning condition: poly PSn (X, Ỹ ) ⊆ span(L(Z, Ỹ ) ◦ Ffeat ).

(48)

Published as a conference paper at ICLR 2026

For any g ∈ PSn (X, Ỹ ), we construct an Sn -equivariant polynomial map to Y as g̃ : X → Y,

g̃(x1 , . . . , xn )j = g(x1 , . . . , xn )

for j = 1, . . . , n.

(49)

This function is clearly an Sn -equivariant polynomial because all n components of its output are identical. Thus, g̃ ∈ PSn (X, Y ). By the spanning condition, we can write X poly g̃ = Ai ◦ pi , where Ai ∈ L(Z, Y ) and pi ∈ Ffeat . (50) i

Applying the averaging operator to both sides yields X 1 X (g̃(σ(x)))σ(j) = g(x) = (ASn )i ◦ pi (x), |Sn | i

(51)

σ∈Sn

where (ASn )i =

1 X πỸ ◦ ρY (σ −1 ) ◦ Ai ◦ ρZ (σ) ∈ LG (Z, Ỹ ). |Sn |

(52)

σ∈Sn

This establishes the spanning condition poly PSn (X, Ỹ ) ⊆ span(LG (Z, Ỹ ) ◦ Ffeat ).

(53)

We consider the higher-order approximation theorem of MLPs, and obtain C ∞ -density via higherorder approximations based on polynomial parameterization. Lemma D.7 (Higher-Order Approximation Theorem for MLPs, Pinkus (1999), Thm. 4.1). For any compact set K ⊂ Rn , any function f ∈ C m (Rn ), and any non-polynomial activation function σ ∈ C m (R), let ϵ > 0. Then there exists an MLP parameterized map gθ with activation function σ such that max |∂xk11 . . . ∂xknn f (x) − ∂xk11 . . . ∂xknn gθ (x)| < ϵ (54) x∈K

for all non-negative integers k1 , . . . , kn with k1 + · · · + kn < m.

Theorem 5.1. In Ex. 2.1, the function families FTFN with smooth non-polynomial activation function ∞ are C ∞ -dense in the space of smooth equivariant maps CG (X, Y ). That is, for any integer r ≥ 0, ∞ any map f ∈ CG (X, Y ), any compact set K ⊂ X, and any ϵ > 0, there exists g ∈ FTFN such that maxx∈K Dk f (x) − Dk g(x) < ϵ, k ≤ r.

(10)

Proof of Thm. 5.1. We first extend the polynomial approximation argument of Lem. 1 in Dym & Maron (2021) to the C r topology. Let KG = {ρX (g)x : g ∈ G, x ∈ K}, which is compact. By the C r -density of polynomials on finite-dimensional Euclidean spaces, for any ϵ > 0 there exists a polynomial map p : X → Y such that max ∥Dk f (x) − Dk p(x)∥ < ϵ,

x∈KG

k ≤ r.

(55)

Define its equivariant symmetrization by Z pG (x) :=

ρY (g −1 )p(ρX (g)x) dg.

(56)

Since the actions are linear, pG is an equivariant polynomial. Moreover, differentiation commutes with the integral, and compactness of G implies that pG converges to f in the C r norm on K as ∞ δ → 0. Hence equivariant polynomials are C ∞ -dense in CG (X, Y ). poly It remains to show that every map in FTFN can be approximated in the C r topology by a TFN whose radial functions are parameterized by MLPs. Fix a polynomial TFN pTFN and a compact set K. Only finitely many polynomial radial functions occur in this network. By Lem. D.7, for each such radial function and every ϵ > 0, there exists an MLP radial function whose derivatives up to order r differ by at most ϵ on the compact interval of pairwise distances attained on K.

Published as a conference paper at ICLR 2026

Each TFN layer is obtained from these radial functions and the previous-layer features through finitely many sums, tensor products, Clebsch–Gordan projections, and linear maps. By the Leibniz and chain rules, these operations are continuous in the C r topology on compact sets. Induction over the finitely many layers therefore shows that, for every ϵ > 0, the polynomial TFN can be approximated by an MLP-parameterized TFN g such that max ∥Dk pTFN (x) − Dk g(x)∥ < ϵ, x∈K

k ≤ r.

(57)

Together with Lem. D.6 and the C r -density of equivariant polynomials proved above, this establishes the theorem.

D.3

S OME R ESULTS ON T OPOLOGY

Here we adopt the definition of NG (H) and NG (H, H ′ ) from § C.1, and consider the twisted product between G-sets as defined in Chap. 2, Sec. 2 of Bredon (1972). We denote XH := {x ∈ X | Gx = H}, from which we obtain the following lemma on topology. Lemma D.8. When the action of a group G on a G-manifold M is faithful, the following decompositions hold: M(H ′ ) = MH ′ ×NG (H ′ )/H ′ G/H ′ (58) and H ′ ′ M(H (59) ′ ) = MH ′ ×NG (H ′ )/H ′ NG (H, H )/H . Proof. The first identity is derived from Thm. 1.31 in Meinrenken (2003), which states the existence of a homeomorphism σ : MH ′ ×NG (H ′ )/H ′ G/H ′ → M(H ′ ) ,

given by

σ([x, gH ′ ]) = g(x).

(60)

H We now prove that σ(MH ′ ×NG (H ′ )/H ′ (NG (H, H ′ )/H ′ )) = M(H ′ ) . The second identity then follows by restricting this homeomorphism.

(⊆) Take an element [x, gH ′ ] where x ∈ MH ′ and g ∈ NG (H, H ′ ). Its image under σ is g(x). The isotropy subgroup is Gg(x) = gGx g −1 = gH ′ g −1 . Since g ∈ NG (H, H ′ ), we have H ⊆ gH ′ g −1 . H This implies g(x) ∈ M H , and since its orbit type is (H ′ ), we have g(x) ∈ M(H ′). H ′ (⊇) Take any y ∈ M(H ′ ) . This means H ⊆ Gy and (Gy ) = (H ). The latter implies there exists a ′ −1 ′ −1 g0 ∈ G such that Gy = g0 H g0 . The condition H ⊆ g0 H g0 implies that g0 ∈ NG (H, H ′ ). Let x = g0−1 (y). Then Gx = H ′ , so x ∈ MH ′ . We can then write y as

y = g0 (g0−1 (y)) = g0 (x) = σ([x, g0 H ′ ]).

(61)

′

Since x ∈ MH ′ and g0 ∈ NG (H, H ), this shows that y is in the image of the restricted domain. The inclusion is thus proven. From the following proposition onward, we need to invoke stratification theory. For the definitions of Whitney conditions and stratifications, we refer to Chap. 3, Sec. 9 of Field (2007). Going forward, unless otherwise specified, all function spaces in this paper are equipped with the (weak) C ∞ topology. For the topology on function spaces, we refer to Chap. 2, Sec. 1 of Hirsch (1976). Proposition D.9. Consider smooth manifolds M and N . Let (Sα , Sβ ) be a pair of submanifolds in N that satisfies Whitney’s condition (a). If a map f : M → N is transverse to Sα at a point x ∈ f −1 (Sα ), then there exists a neighborhood U of f in C ∞ (M, N ) and a neighborhood V of x in M , such that for any map g ∈ U and any point y ∈ V , g is transverse to both Sα and Sβ at y. Proof. The proof is adapted from Prop. 1.3 of Trotman (1976). We proceed by contradiction. Assume the conclusion is false. Then for any neighborhood V of x and any neighborhood U of f , there exists a map g ∈ U for which the condition g ⋔ Sα , Sβ on V does not hold. (1)

For a neighborhood V1 of x with radius less than ϵ1 , we can construct a sequence of maps {gn } converging to f such that transversality to Sα or Sβ fails on V1 . Let the sequence of non-transverse 31

Published as a conference paper at ICLR 2026

(1)

points be {yn }. We can similarly construct, for neighborhoods Vt of x with radius less than ϵt , (t) sequences of maps {gn } converging to f and corresponding sequences of non-transverse points (t) {yn }. (n)

Now, consider the diagonal sequence of maps {gn } and the corresponding sequence of non(n) (n) (n) (n) transverse points {yn }. We have gn → f and yn → x. We can partition the sequence {yn } into a subsequence lying in Sα and another lying in Sβ . At least one of these two subsequences must be infinite. It is therefore sufficient to negate the following proposition: There exists a sequence of maps {gn } → f and a sequence of points {yn } → x with either {yn } ⊂ Sα or {yn } ⊂ Sβ , such that for each n, gn is not transverse to Sα or Sβ at yn . This is impossible. Note that for a sufficiently small ϵ1 , the closure of the neighborhood V1 is compact by local compactness of the manifold, and we can consider the control of the neighborhood U over functions on this compact closure in the C ∞ topology of functioin space. Also note that d(gn )yn (Tyn M ) converges to dfx (Tx M ). For the case of Sα , by the smoothness of the manifold, Tgn (yn ) Sα converges to Tf (x) Sα . For the case of Sβ , by Whitney’s condition (a), the limit of Tgn (yn ) Sβ contains Tf (x) Sα . However, f is transverse to Sα at x, meaning dfx (Tx M ) + Tf (x) Sα = Tf (x) N.

(62)

Therefore, due to convergence, the transversality condition for Sα or Sβ must hold for gn at yn for sufficiently large n, which is a contradiction. Corollary D.10. Under the conditions of the previous proposition, assume that the stratification S = {Sα } satisfies Whitney’s condition (a). If a map f is transverse to a stratum Sα at a point x ∈ f −1 (Sα ), then there exists a neighborhood Vx of x and a neighborhood Ux of f in C ∞ (M, N ) such that for any g ∈ Ux and any y ∈ Vx , g is transverse to the entire stratification S in the neighborhood Vx . Proof. Let x be a point where f is transverse to Sα . We consider all strata Sβ that satisfies Sβ ∩ Sα ̸= ∅. Since a stratification is locally finite, there are only a finite number of such adjacent strata. For each such adjacent stratum Sβ , the pair (Sα , Sβ ) satisfies Whitney’s condition (a). By Prop. D.9, (α,β) (α,β) there exist neighborhoods Ux of f and Vx of x where transversality to both Sα and Sβ holds. We can then construct the desired neighborhoods by taking the finite intersection of these individual neighborhoods labelled by β: \ \ Ux := Ux(α,β) and Vx := Vx(α,β) . (63) Sβ ∩Sα ̸=∅

Sβ ∩Sα ̸=∅

Since the intersection is finite, Ux and Vx are still open neighborhoods of f and x, respectively. For any g ∈ Ux and y ∈ Vx , g is transverse to Sα and all its adjacent strata Sβ at y. Therefore, it is transverse to the entire stratification S within the local neighborhood Vx . Lemma D.11. For a G-manifold M and a G-representation Y , the collection of submanifolds {(M(H) × Y(H ′ ) )(H) }(H ′ )≥(H)

(64)

forms a Whitney stratification of (M × Y )(H) . Proof. By Prop. 3.7.1 of Field (2007), we obtain that these point sets in the collection are smooth G-submanifolds. By Lem. D.8, we have (M(H) × Y(H ′ ) )(H) = (M(H) × Y(H ′ ) )H ×NG (H)/H G/H

= (M(H) × Y(H ′ ) )H (H) ×NG (H)/H G/H H H = (M(H) × Y(H ′ ) ) ×NG (H)/H G/H.

(65) (66) (67)

By local trivialization, similar to the proof of Prop. 3.9.2 of Field (2007), it is sufficient to prove that H H the pair (Y(H ′ ) , Y(K) ) satisfies Whitney’s condition (b). By Prop. 3.9.2 of Field (2007), specialized 32

Published as a conference paper at ICLR 2026

to a representation space, the orbit-type stratification of Y is Whitney regular. Hence the pair (Y(H ′ ) , Y(K) ) for (H ′ ) > (K) satisfies Whitney’s condition (b). We show that the condition is preserved after restriction to the fixed-point space Y H . H H H H Let x ∈ Y(H ′ ) ∩ Y(K) , and consider sequences {xn } ⊂ Y(H ′ ) and {yn } ⊂ Y(K) converging to x, such that the secant lines xn yn converge to a line l and H Tyn Y(K) −→ F.

(68)

Passing to a subsequence, we may further assume that Tyn Y(K) −→ E.

(69)

Since (Y(H ′ ) , Y(K) ) satisfies Whitney’s condition (b), we have l ⊆ E. Moreover, since xn , yn ∈ Y H and Y H is a linear subspace of Y , we also have l ⊆ Y H .

H It remains to identify F . Since Y(K) is H-invariant and yn ∈ Y H , we obtain Tyn Y(K) = H Tyn Y(K) . Choose an H-invariant inner product on Y , and let Z PH = ρ(h) dh (70) H H

be the orthogonal projection onto Y . Let Qn and Q denote the orthogonal projections onto Tyn Y(K) and E, respectively. Then Qn → Q. Since Tyn Y(K) is H-invariant, Qn commutes with PH , and H hence Qn PH is the orthogonal projection onto Tyn Y(K) . Therefore, Qn PH −→ QPH , which implies F = E H = E ∩ Y H . Combining l ⊆ E and l ⊆ Y H , we obtain l ⊆ E ∩ Y H = F . Thus H H the pair (Y(H ′ ) , Y(K) ) satisfies Whitney’s condition (b).

Lemma D.12. Let M and N be topological spaces, with M being compact. Consider a continuous map f : M → N . Let C be a closed set in N , and suppose there exists a neighborhood V of the preimage f −1 (C). Then there exists a neighborhood U of f in C(M, N ) with the C 0 topology (compact-open topology) such that for any map g ∈ U , we have g(M \ V ) ∩ C = ∅. Proof. Note that M \ V is a compact set, and N \ C is an open set. By the definition of the compact-open topology, e.g. Definition 43.1 of Willard (1970), the set U := {h ∈ C(M, N ) | h(M \ V ) ⊆ N \ C}

(71)

is an open set. Since V is a neighborhood of f −1 (C), we have f −1 (C) ⊆ V , which implies f (M \ V ) ⊆ N \ C. Therefore, the map f itself is an element of U , meaning U is an open neighborhood of f . For any map g ∈ U , its definition directly implies that g(M \ V ) ⊆ N \ C, which is equivalent to g(M \ V ) ∩ C = ∅. Thus, this set U is the desired neighborhood. D.4

G ENERIC E QUIVARIANT M APPINGS

Proposition D.13 (Equivariant Smooth Extension). Let M be a smooth G-manifold and let S ⊂ M be ∞ a smooth and compact G-submanifold. For any smooth equivariant map f ∈ CG (S, Y ) defined on S ∞ that maps into a representation space Y , there exists a smooth equivariant extension f˜ ∈ CG (M, Y ) ˜ such that its restriction to S is f , i.e., f |S = f . Proof. The proof strategy is the same as that for the Tietze-Gleason Theorem (see Chap. 1, Thm. 2.3 of Bredon (1972)). 33

Published as a conference paper at ICLR 2026

First, we can view the map f as a collection of dim Y smooth real-valued functions defined on S. By the standard smooth extension theorem, e.g., Lem. 5.34 in Lee (2012), there exists a smooth (but not necessarily equivariant) extension φ ∈ C ∞ (M, Y ) such that φ|S = f .

We then construct an equivariant map from φ using the group averaging operator. Define the map ψ : M → Y by Z ρY (g −1 )φ(ρX (g)(x)) dg.

ψ(x) =

(72)

Due to the linearity of the integral and the properties of the Haar measure, this map is equivariant. For any h ∈ G: Z ψ(h(x)) = ρY (g −1 )φ(g(h(x))) dg (73) G Z = ρY ((g ′ h−1 )−1 )φ(g ′ (x)) dg ′ (74) G Z = ρY (h) ρY ((g ′ )−1 )φ(g ′ (x)) dg ′ (75) G

= ρY (h)ψ(x).

(76)

Next, we verify that the restriction of ψ to the submanifold S is equal to the original map f . For any s ∈ S: Z ψ(s) = ρY (g −1 )φ(g(s)) dg (77) G Z = ρY (g −1 )f (g(s)) dg (78) G Z = ρY (g −1 )ρY (g)f (s) dg (79) G Z Z = f (s) dg = f (s) 1 dg = f (s). (80) G

Finally, the smoothness of ψ follows from the smoothness of φ and the properties of integration over a compact group. This can be verified by a local coordinate analysis. Thus, ψ is the desired smooth equivariant extension f˜. Corollary D.14. Consider maps from a G-manifold X to a representation space Y , where X contains ∞ a compact, smooth G-submanifold M . Let S be a subset of CG (M, Y ) that contains an open dense ∞ set. Then the set of maps f ∈ CG (X, Y ) whose restriction to M lies in S (i.e., f |M ∈ S) also ∞ contains an open dense subset of CG (X, Y ). Proof. We recall that a set contains an open dense subset if and only if for any non-empty open set, its ∞ intersection with the set contains a non-empty open subset. Let A := {f ∈ CG (X, Y ) | f |M ∈ S}. ∞ Our goal is to show that for any f ∈ CG (X, Y ) and any of its open neighborhoods U , there exists a non-empty open set V ⊆ U ∩ A. ∞ ∞ Consider the restriction map ResM : CG (X, Y ) → CG (M, Y ), by Prop. D.13, this map is surjective. Furthermore, this map is continuous and open.

Let U be an open neighborhood of an arbitrary map f . Since ResM is an open map, its image, ResM (U ), is an open neighborhood of f |M . By our hypothesis on S, its intersection with the open set ResM (U ) must contain a non-empty open subset. Let us call this non-empty open set V ′ . We have V ′ ⊆ ResM (U ) ∩ S. (81) ′ ′ Now, let’s consider the preimage V := Res−1 M (V ). Since ResM is continuous and V is open, V is an open set. Since ResM is surjective, the preimage V must also be non-empty. Therefore U ∩ V is a non-empty open subset of U ∩ A, which proves the corollary.

For C r manifolds X and Y , C r (X) is the set of C r real-valued functions on X, and C r (X, Y ) is r the set of C r maps from X to Y . For manifolds with a C r G-action, CG (X, Y ) is the set of C r 34

Published as a conference paper at ICLR 2026

equivariant maps from X to Y . We assume these function spaces are endowed with the C r topology; in our proofs, we always consider the C ∞ topology. For a map f ∈ C 1 (X, Y ) and a submanifold A ⊆ Y , f ⋔ A denotes that f is transverse to A. For G-manifolds, f ⋔G A denotes that f is in equivariant general position with respect to the G-submanifold A defined in Bierstone (1977). For f ∈ C r (X, Y ), the jet map j r f : X → J r (X, Y ) maps a point x to the equivalence class of the first r derivatives of f at x. We obtain the following results. Proposition D.15 (Bierstone (1977), Thm. 1.3). Let M and N be smooth G-manifolds. If P is a ∞ closed G-submanifold of N and K is a compact subset of M , then the set of maps f ∈ CG (M, N ) ∞ satisfying f ⋔G P on K forms an open subset of CG (M, N ) (in the Whitney C ∞ topology). Proposition D.16 (Bierstone (1977), Thm. 1.4). Let M, N be smooth G-manifolds and P be a ∞ G-submanifold of N . The set of maps f ∈ CG (M, N ) satisfying f ⋔G P forms a residual subset of ∞ CG (M, N ), i.e., a countable intersection of open dense sets (in the Whitney C ∞ topology). Remark. For compact M , note that the Whitney C ∞ topology coincides with the C ∞ topology. Proposition D.17 (Stratumwise Transversality Theorem). For any orbit type (H) ∈ OG (M ), a map ∞ f ∈ CG (M, N ) with f ⋔G P satisfies the stratumwise transversality property: f |MH : MH 7→ N H , f |MH ⋔ P H .

(82)

Alternatively, this can be expressed in the language of jets as j 0 f |M(H) : M(H) 7→ (M(H) × N )(H) , j 0 f |M(H) ⋔ (M(H) × P )(H)

(83)

Remark. The fact that f ⋔G P implies stratumwise transversality on the fixed-point sets is a fundamental conclusion derived from Prop. 6.4 of Bierstone (1977). Our proof below is a straightforward corollary of this. Proof. From the discussion in Prop. 6.4 of Bierstone (1977), we have j 0 (f |M(H) ) : M(H) 7→ (M × N )(H) ,

j 0 (f |M(H) ) ⋔ (M × P )(H) .

(84)

Note that the image of the map j 0 (f |M(H) ) is contained within M(H) × N . The transversality condition is an equality of tangent spaces: d(j 0 (f |M(H) ))x (Tx M(H) ) + Ty ((M × P )(H) ) = Ty ((M × N )(H) ).

(85)

Intersecting both sides of this equation with Ty ((M(H) × N )(H) ) yields d(j 0 (f |M(H) ))x (Tx M(H) ) + Ty ((M(H) × P )(H) ) = Ty ((M(H) × N )(H) ).

(86)

Therefore, we have j 0 (f |M(H) ) : M(H) → (M(H) × N )(H) ,

j 0 (f |M(H) ) ⋔ (M(H) × P )(H) .

(87)

Proposition D.18 (Bierstone (1977), Sec. 7). Consider smooth G-manifolds M and N . Let P be a smooth G-submanifold of N . If an equivariant map f satisfies f ⋔G P at a point x ∈ f −1 (P ), then ∞ there exists a neighborhood U of f in CG (M, N ) and a G-invariant neighborhood V of the orbit G(x) such that for any map g ∈ U and any point y ∈ V , it holds that g ⋔G P at y. Remark. The proposition above can also be derived from the properties of stratifications given in Prop. D.9 by the definition of equivariant general position in Bierstone (1977). Proposition D.19. Let G be a compact Lie group, M be a compact, smooth G-manifold, and Y ∞ be a G-representation space. The set of smooth equivariant maps f ∈ CG (M, Y ) that satisfy the transversality condition j 0 (f |M(H) ) ⋔ (M(H) × Y(H ′ ) )(H) (88) for all pairs of orbit types (H) and (H ′ ) such that (H) is present in M and (H ′ ) ≥ (H), contains ∞ an open dense subset of CG (M, Y ).

Remark. The proof of density is straightforward. However, the openness of this property does not generally hold. A counterexample can be constructed following Ex. 2.1 of Bierstone (1977). 35

Published as a conference paper at ICLR 2026

Proof. Since M is compact, by Proposition 3.7.2 of Field (2007), the orbit types are finite. For any given symmetry type (H), by Prop. D.16 the set of maps satisfying f ⋔G Y(H) is an intersection of a finite number of residual sets, and is therefore itself a residual set. By Prop. D.17 the set of maps satisfying the transversality condition stated in the theorem is dense. Instead of directly proving openness property, we prove a related proposition: for a map f that satisfies f ⋔G Y(H) for all orbit types (H), there exists a neighborhood U of f such that for any g ∈ U, j 0 (g|M(H) ) ⋔ (M(H) × Y(H ′ ) )(H) for all (H ′ ) ≥ (H). (89) The proof proceeds by induction on the dimension of the strata in the orbit type stratification of Y , ordered from lowest to highest (or equivalently, from highest to lowest symmetry). We discuss the fixed orbit type (H) in M . Then we can construct neighborhoods that hold for all (H) by taking the intersection of the neighborhoods corresponding to each orbit type (H). We start with the lowest-dimensional strata. Let Y(H1 ) be a stratum of minimal dimension. Such a stratum is unique, which corresponds to a maximal symmetry type (G) and is a closed G-submanifold ∞ Y G . By Prop. D.15, the set of maps in general position to Y(H1 ) is open in CG (M, Y ). Thus, there exists a neighborhood U1 of f such that for any g ∈ U1 , g ⋔G Y(H1 ) . By Prop. D.17, this implies that for all (H), j 0 (g|M(H) ) ⋔ (M(H) × Y(H1 ) )(H) . (90) Assume the proposition holds for all strata of dimension up to k − 1. Let Uk−1 be the neighborhood of f found from the inductive hypothesis. For any map g ∈ Uk−1 and for any stratum Y(Hi ) with dim Y(Hi ) ≤ k − 1, we have j 0 (g|M(H) ) ⋔ (M(H) × Y(Hi ) )(H)

for all (Hi ) ≥ (H).

(91)

Note that by Lem. D.11 the collection S(H) = {(M(H) × Y(H ′ ) )(H) }(H ′ )≥(H) is a Whitney stratification and satisfies the frontier condition. Therefore, by Cor. D.10, for any x ∈ M(H) with f (x) ∈ Y(Hi ) , there exists a neighborhood Vx of x and a neighborhood Ux of j 0 (f |M(H) ) such that for any map in Ux , it is transverse to the stratification S(H) . ∞ Next, we need to obtain an open set in CG (M, Y ). We use the following sequence of maps between function spaces: ∞ CG (M, Y )

ResM(H)

∞ CG (M(H) , Y )

(92)

∞ 7→ CG (M(H) , (M(H) × Y )(H) )

(93)

7→

∞

,→ C (M(H) , (M(H) × Y )(H) ).

(94)

∞

The maps between these function spaces are continuous in C topology. For example, the 0-jet map is a restriction of the continuous 0-jet map on the general function space, which can be obtained from the continuity of jet map in the Whitney C ∞ topology established in Chap. 2, Prop. 3.4 of Golubitsky & Guillemin (1973). Since the topology on the equivariant function space is induced from the general function space, the restricted map is also continuous. Similarly, ResM(H) and the inclusion are continuous. From this analysis, we can pull back the neighborhood Ux to obtain a ∞ neighborhood Ux′ of f in CG (M, Y ) where the transversality holds. We deal with the global result. For any i with dim Y(Hi ) ≤ k − 1, the collection of neighborhoods {Vx }x∈f −1 (Y(Hi ) ) for all i with dim Y(Hi ) ≤ k−1 forms an open cover of f −1 (Y(Hi ) ). By the frontier condition, any stratum that intersects the closure Y(Hi ) has strictly smaller dimension than Y(Hi ) . S S Hence i Y(Hi ) is closed. Since M is compact and f is continuous, the preimage f −1 ( i Y(Hi ) ) is compact, so there exists a finite subcover {Vxn }N n=1 of the preimage. We construct the neighborhood SN TN (1) Vk := n=1 Vxn of the preimage in M , and the neighborhood Uk := n=1 Ux′ n of f in the function space. (2)

In the C 0 topology, by Lem. D.12, there exists a neighborhood Uk (2) any g ∈ Uk , [ g(M \ Vk ) ∩ Y(Hi ) = ∅. i

of f in C(M, Y ) such that for (95)

Published as a conference paper at ICLR 2026

∞ By pulling this back through the inclusions CG (M, Y ) ,→ C ∞ (M, Y ) ,→ C(M, Y ), we obtain a (2) ′ ∞ neighborhood (Uk ) in CG (M, Y ). The neighborhood is open in the C 0 topology, so it also open (1) (2) in the C ∞ topology. Let Uk′ = Uk−1 ∩ Uk ∩ (Uk )′ . For any map g ∈ Uk′ , we have 0 j (g|M(H) ) ⋔ S(H) for x ∈ Vk (96) g(x) ∈ / Y(Hi ) for x ∈ M \ Vk

for all i with dim Y(Hi ) ≤ k − 1.

Now we show by contradiction that there exists a neighborhood Uk ⊂ Uk′ of f where the transversality condition holds for strata of dimension k. By the induction hypothesis, the transversality condition holds for all strata of dimension at most k − 1. Therefore, it also holds for all strata of dimension at most k. The proof idea is from the proof of openness of equivariant general position in Sec. 7 of Bierstone (1977), where the closedness of P is replaced by the condition that f does not intersect low-dimensional Y(Hi ) outside of Vk .

Assume openness does not hold. Then there exists a sequence {gn } → f in Uk′ such that each gn fails the stratumwise transversality condition for some stratum of dimension k. Since the stratumwise transversality condition fails, it follows from Prop. D.17 that gn ⋔G Y(Hj ) for all j with dim Y(Hj ) = k also does not S hold. The points of non-transversality for these maps, {yn }, can only occur in M \ Vk and gn (yY ) ∈ j Y(Hj ) . Since M \ Vk is compact, there is a convergent subsequence {yni } with limit x. Then gn (yY ) → f (x). By construction of Uk′ , f (M \ Vk ) does not intersect Y(Hi ) for any dim Y(Hi ) ≤ k − 1. Thus, we must have f (x) ∈ Y(Hj ) for some j. We claim that gni (yni ) ∈ Y(Hj ) for all sufficiently large i. Otherwise, there exists a further subsequence {gniα (yniα )} such that gniα (yniα ) ∈ Y(Hj′ ) for some j ′ ̸= j and all α, while still gniα (yniα ) → f (x) ∈ Y(Hj ) . Hence Y(Hj ) ∩ Y(Hj′ ) ̸= ∅. By the frontier condition of stratification this implies dim Y(Hj ) < dim Y(Hj′ ) . However, Y(Hj ) and Y(Hj′ ) have the same dimension, yielding a contradiction. Therefore, for i sufficiently large, gni (yni ) ∈ Y(Hj ) . We now show that this contradicts the assumption that f ⋔G Y(Hj ) at x. If f were to satisfy f ⋔G Y(Hj ) at x, then by Prop. D.18, there would exist a neighborhood U of f and a G-invariant neighborhood V of G(x) such that any g ∈ U satisfies g ⋔G Y(Hj ) at any y ∈ V . This contradicts the existence of sequence {gn } and points {yn } where transversality fails. This completes the proof. ∞ Proposition D.20. Let F be a C ∞ -dense family of smooth parameterized maps in CG (X, Y ) and {Mj } be a finite collection of compact, connected and smooth G-submanifolds of X. Let

S(H)→(H ′ ) (f ) = {x ∈ X | (Gx ) = (H), (Gf (x) ) = (H ′ )}.

(97)

There is a C ∞ -dense subset G ⊂ F, for g ∈ G, the set S(H)→(H ′ ) (g|Mj ) is a disjoint union of smooth G-submanifolds of X, and its dimension satisfies H dim S(H)→(H ′ ) (g|Mj ) = dim(Mj )(H) − (dim Y H − dim Y(H ′))

(98)

if the right-hand side of the equation is not smaller than dim G − dim H. Otherwise, the set S(H)→(H ′ ) (g|Mj ) is empty. ∞ Proof. By Prop. D.19, for each Mj , there exists an open dense set in CG (Mj , Y ) such that for any map g in this set,

j 0 (g|(Mj )(H) ) ⋔ ((Mj )(H) × Y(H ′ ) )(H)

for all (H ′ ) ≥ (H).

(99)

Therefore, writing codimX Y := dim X − dim Y for the codimension of Y in X, the dimension theorem for transverse maps (see, e.g., Sec. 2.3 of Arnold et al. (2012)) yields codim(Mj )(H) S(H)→(H ′ ) = codim((Mj )(H) ×Y )(H) (Mj )(H) × Y(H ′ ) (H) . (100) Furthermore, from the proof of Lem. D.11, we have H ((Mj )(H) × Y(H ′ ) )(H) = ((Mj )H (H) × Y(H ′ ) ) ×NG (H)/H G/H.

(101)

Published as a conference paper at ICLR 2026

Thus, we obtain

H codim(Mj )(H) S(H)→(H ′ ) = codimY H Y(H ′).

(102)

Moreover, since the dimension of an orbit with orbit type (H) in a G-manifold is dim G − dim H, if the dimension of S(H)→(H ′ ) calculated above is less than dim G − dim H, then S(H)→(H ′ ) is an empty set. ∞ For each Mj , we take the corresponding open dense set Uj ⊂ CG (Mj , Y ) on which the maps ∞ satisfy the dimension theorem. Then, by Cor. D.14, the set of functions f ∈ CG (X, Y ) that satisfy ′ f |Mj T∈ Uj contains an open dense set Uj . Since {Mj } is a finite collection, the intersection U = j Uj′ is also an open dense set. Therefore, the intersection of U with the C ∞ -dense set F is a dense set G. Thus, this intersection is the set of maps in F that we sought to construct, which is dense and whose elements satisfy the dimension theorem.

D.5

P ROOF OF T HM . 5.2

Theorem 5.2. Let F be a equivariant parametrization with C ∞ approximation capability. If for every (H) ∈ OG (M ) we have (pY (H)) ∈ OG (Y ), then for any finite union of compact, smooth ∞ G-submanifolds M ⊂ X, any f ∈ CG (X, Y ), any integer r ≥ 0, and any ϵ > 0, there exists a map g ∈ F such that maxx∈M ∥Dk f (x) − Dk g(x)∥ < ϵ, k ≤ r, (11)

and g|M is almost isovariant relative to Y . Furthermore, if the feature space Y contains a representation Ỹ ⊕r for an integer r > maxj {dim Mj }, where Y satisfies the condition (pY (H)) ∈ OG (Y ), then the approximating map g|M can be chosen to be isovariant relative to Y .

Proof of Thm. 5.2. By Lem. D.8, we consider the relation ′

H H dim NG (H, H ′ ) − dim NG (H ′ ) = α(H, H ′ ) = dim Y(H . ′ ) − dim Y

(103)

Regarding the G-representation as a faithful representation of G/kerρY , for orbit types satisfying (pY (H ′ )) > I(H, Y ), the dense and open property of the minimal orbit type by Prop. B.4 in the fixed-point space implies that for each Mj , ′

H H dim Y H − dim Y(H − dim Y H − αG (H, H ′ ) > 0. ′ ) = dim Y

(104)

By Prop. D.20, it implies dim(Mj )(H) > dim S(H)→(H ′ ) (g|Mj ).

(105)

With respect to the Hausdorff measure Hd , since the dimension of S(H)→(H ′ ) (g|Mj ) is strictly less than the dimension of the manifold it lies in, its measure is zero. Then, by the finiteness of the set of orbit types for a compact manifold and the finiteness of the collection {Mj }, it follows that g|M is an almost isovariant map relative to Y . Furthermore, if we require g to be isovariant relative to Y , we need S(H)→(H ′ ) (g|Mj ) = ∅.

(106)

The condition for this set to be empty is related to its codimension. Since the inequality ′

dim Y H − dim Y H − αG (H, H ′ ) ≥ 1

(107)

scales with multiplicity r to become ′

r(dim Y H − dim Y H ) − αG (H, H ′ ) ≥ r,

(108)

it is sufficient to choose the multiplicity for the representation Y ⊕r such that r > maxj {dim Mj }.

Published as a conference paper at ICLR 2026

TABLES

All closed subgroups of O(3) are denoted using Schoenflies notation, where it should be noted that K = SO(3) and Kh = O(3). When interpreting the tables, care should be taken to recognize the low-dimensional equivalences: Cs = C1h = C1v , D1 = C2 , D1h = C2v , and D1d = C2h . In the tables of symmetry infimum, we list a representative subgroup from each conjugacy class and omit the class notation (e.g., C2 instead of (C2 )) for clarity. E.1

M INIMAL P ROPER S UPERGROUPS IN SO(3) OR O(3)

Minimal proper supergroups table of SO(3) or O(3). Some of the results can be found from Fig. 3.2.1.6 of Aroyo (2016). We only present the results for O(3). The discussion for SO(3) can be obtained by removing the subgroups that are not subgroups of SO(3) (i.e., the subgroups of the first kind), namely Ck , Dk , T, O, I, C∞ , D∞ , and K. Table 6: Table of minimal proper supergroups H of axial closed subgroups of O(3). In the supergroup notation, p denotes a prime number and p∗ denotes an odd prime number. G H for General k H for Special k Ck S2k Ckh Ckv (k > 1) Dk (k > 1) Dkh (k > 1) Dkd (k > 1)

Cpk , S2k , Ckh , Ckv , Dk S2p∗ k , C2k,h , Dkd Cpk,h , Dkh Cpk,v , Dkh , Dkd Dpk , Dkh , Dkd Dpk,h Dp∗ k,d , D2k,h

Dp∗ (k = 2); T (k = 3) Th (k = 3) Cpv (k = 1); Dp∗ d (k = 2) Td (k = 3); Dph (k = 2) T (k = 2); O (k = 3, 4); I (k = 3, 5) Th (k = 2); Oh (k = 4) Td (k = 2); Oh (k = 3); Ih (k = 3, 5)

Table 7: Table of minimal proper supergroups H of other closed subgroups O(3). G H T Td Th O Oh I Ih C∞ C∞h C∞v D∞ D∞h K

Td , Th , O, I Oh Oh , Ih Oh , K Kh Ih , K Kh C∞h , C∞,v , D∞ D∞,h D∞,h D∞,h , K Kh Kh

Published as a conference paper at ICLR 2026

E.2

D IMENSIONS OF F IXED - POINT S UBSPACES FOR S UBGROUPS OF SO(3) OR O(3)

Dimensions table of fixed-point subspaces for subgroups of SO(3) or O(3). Some of the results can be found from Table B.1 and Table B.2 of Linehan & Stedman (2001). We only present the results for O(3). The discussion for SO(3) can be obtained directly from the table. Table 8: Dimensions of fixed-point subspaces for closed subgroups of O(3) acting on the irreducible representations Vl=l± , where ak (l) = ⌊l/k⌋ and bk (l) = ⌊(l + k)/(2k)⌋. 0

Subgroup

l = l0− l0 even l0 odd

Ck S2k Ckh

k even k odd k even k odd

Ckv Dk Dkh Dkd T Th Td O Oh I Ih C∞ C∞h C∞v D∞ D∞h

k even k odd k even k odd

l0 even 2ak (l0 ) + 1

l = l0+ l0 odd

2bk (l0 ) 2a2k (l0 ) + 1 0 2ak (l0 ) + 1 0 2ak (l0 ) + 1 2bk (l0 ) 2a2k (l0 ) + 1 ak (l0 ) ak (l0 ) + 1 ak (l0 ) + 1 ak (l0 ) ak (l0 ) + 1 ak (l0 ) ak (l0 ) + 1 ak (l0 ) 0 ak (l0 ) + 1 ak (l0 ) bk (l0 ) a2k (l0 ) + 1 a2k (l0 ) bk (l0 ) a2k (l0 ) + 1 a2k (l0 ) 0 ak (l0 ) + 1 ak (l0 ) 2a3 (l0 ) + a2 (l0 ) − l0 + 1 0 2a3 (l0 ) + a2 (l0 ) − l0 + 1 a3 (l0 ) + b2 (l0 ) + b1 (l0 ) − l0 a4 (l0 ) + a3 (l0 ) + a2 (l0 ) − l0 + 1 a4 (l0 ) + a3 (l0 ) + a2 (l0 ) − l0 + 1 0 a4 (l0 ) + a3 (l0 ) + a2 (l0 ) − l0 + 1 a5 (l0 ) + a3 (l0 ) + a2 (l0 ) − l0 + 1 0 a5 (l0 ) + a3 (l0 ) + a2 (l0 ) − l0 + 1 1 0 1 0 1 1 0 1 0 1 0 0 1 0

Published as a conference paper at ICLR 2026

E.3

S YMMETRY I NFIMUM FOR S UBGROUPS OF SO(3)

Symmetry infimum table for subgroups of SO(3). In the following tables, we use a color-coding scheme to classify the types of symmetry increase. An increase to the full group is termed full degeneration and is marked in red . An increase to a supergroup of a strictly higher dimension is termed continuous degeneration and is marked in blue . An increase to a supergroup of the same dimension is termed discrete degeneration, and is marked in yellow . No increase is termed no degeneration, and is marked in green . All subgroups increase to to K when l = 0. ⊕r Table 9: Symmetry infimum of general axis subgroup of SO(3) on Vl=l , r > 3, l0 > 0. 0

Ck Dk (k > 2)

l0 < k l0 even l0 odd D∞ C∞ D∞ K

l0 ≥ k Ck Dk

⊕r , r > 3, l0 > 0. Table 10: Symmetry infimum of special axis subgroup of SO(3) on Vl=l 0

l0 = 1 K

l0 = 2 D2

l0 = 3 T

l0 ≥ 4 D2

⊕r Table 11: Symmetry infimum of polyhedral subgroup of SO(3) on Vl=l , r > 3, l0 > 0. The 0 "Caption" in the table takes the value l0 = 6, 10, 12, 15, 16, 18, 20 − 22, 24 − 28. l0 = 6, 9, 10 l0 = 3, 7 l0 = 4, 8 l0 ≥ 12 other T T T O T K l0 = 4, 6, 8, 9, 10 l0 ≥ 12 other O O O K Caption l0 ≥ 30 other I I I K

⊕r Table 12: Symmetry infimum of infinite subgroup of SO(3) on Vl=l , r > 3, l0 > 0. 0

C∞ D∞ K

l0 even D∞ D∞ K

l0 odd C∞ K K

Published as a conference paper at ICLR 2026

E.4

S YMMETRY I NFIMUM FOR S UBGROUPS OF O(3)

Symmetry Infimum table for Subgroups of O(3). The color scheme for full, continuous, and no degeneration is the same as for SO(3). For cases of discrete degeneration of l0 = l+ , we distinguish between (a) light Green , which indicates the predictable increase to π −1 (π(H)) for the projection map π : O(3) → SO(3), and (b) yellow , which indicates all other exceptional cases of discrete degeneration. In the following table, the first type of subgroups degenerate to K and the others to Kh when l0 = 0− . All subgroups increase to to Kh when l0 = 0+ . ⊕r Table 13: Symmetry infimum of general axis subgroup of O(3) on Vl=l − , r > 3, l0 > 0. 0

Subgroup Ck S2k Ckh Ckv (k > 2) Dk (k > 2) Dkh (k > 2) Dkd (k > 2)

k even k odd k even k odd k even k odd k even k odd k even k odd k even k odd k even k odd

0 < l0 < k l0 even l0 odd D∞ C∞v D∞ C∞v Kh Kh Kh Kh Kh Kh Kh Kh Kh C∞v Kh C∞v D∞ Kh D∞ Kh Kh Kh Kh Kh Kh Kh Kh Kh

k ≤ l0 < 2k l0 even l0 odd Ck Ck Ck Ck S2k S2k Kh Kh Kh Kh Ckh Ckh Dkd Ckv Dkh Ckv Dk Dkd Dk Dkh Kh Kh Dkh Dkh Dkd Dkd Kh Kh

l0 ≥ 2k l0 even l0 odd Ck Ck Ck Ck S2k S2k Kh Kh Kh Kh Ckh Ckh Ckv Ckv Ckv Ckv Dk Dk Dk Dk Kh Kh Dkh Dkh Dkd Dkd Kh Kh

⊕r Table 14: Symmetry infimum of general axis subgroup of O(3) on Vl=l + , r > 3, l0 > 0. 0

Subgroup Ck S2k Ckh Ckv (k > 2) Dk (k > 2) Dkh (k > 2) Dkd (k > 2)

k even k odd k even k odd k even k odd k even k odd k even k odd k even k odd k even k odd

0 < l0 < k l0 even l0 odd D∞h C∞h D∞h C∞h D∞h C∞h D∞h C∞h D∞h C∞h D∞h C∞h D∞h Kh D∞h Kh D∞h Kh D∞h Kh D∞h Kh D∞h Kh D∞h Kh D∞h Kh

k ≤ l0 < 2k l0 even l0 odd Ckh Ckh S2k S2k D∞h C∞h S2k S2k Ckh Ckh D∞h C∞h Dkh Dkh Dkd Dkd Dkh Dkh Dkd Dkd Dkh Dkh D∞h Kh D∞h Kh Dkd Dkd

l0 ≥ 2k l0 even l0 odd Ckh Ckh S2k S2k C2kh C2kh S2k S2k Ckh Ckh C2kh C2kh Dkh Dkh Dkd Dkd Dkh Dkh Dkd Dkd Dkh Dkh D2kh D2kh D2kh D2kh Dkd Dkd

Published as a conference paper at ICLR 2026

⊕r Table 15: Symmetry infimum of special axis subgroup of O(3) on Vl=l − , r > 3, l0 > 0. 0

C2v D2 D2h D2d

l0 = 1

l0 = 2

l0 = 3

C∞v Kh Kh Kh

D2d D2 Kh D2d

C2v Td Kh Td

l0 ≥ 4 l0 even l0 odd C2v C2v D2 D2 Kh Kh D2d D2d

⊕r Table 16: Symmetry infimum of special axis subgroup of O(3) on Vl=l + , r > 3, l0 > 0. 0

C2v D2 D2h D2d

l0 = 1

l0 = 2

l0 = 3

Kh Kh Kh Kh

D2h D2h D2h D∞h

Th Th Th Kh

l0 ≥ 4 l0 even l0 odd D2h D2h D2h D2h D2h D2h D4h D4h

⊕r Table 17: Symmetry infimum of polyhedral subgroup of O(3) on Vl=l − , r > 3, l0 > 0. The 0 "Caption" in the table takes the value l0 = 6, 10, 12, 15, 16, 18, 20 − 22, 24 − 28. l0 = 6, 9, 10 l0 = 3, 7 l0 = 4, 8 l0 ≥ 12 other T T Td O T Kh l0 = 3, 6, 7 l0 ≥ 9 other Td Td Td Kh all l0 Th Kh l0 = 4, 6, 8, 9, 10 l0 ≥ 12 other O O O Kh all l0 Oh Kh Caption l0 ≥ 30 other I Ih Ih Kh all l0 Ih Kh

Published as a conference paper at ICLR 2026

⊕r Table 18: Symmetry infimum of polyhedral subgroup of O(3) on Vl=l + , r > 3, l0 > 0.The "Caption" 0 in the table takes the value l0 = 6, 10, 12, 15, 16, 18, 20 − 22, 24 − 28. l0 = 3, 6, 7, 9, 10 l0 = 4, 8 l0 ≥ 12 other T Th Oh Th Kh l0 = 4, 6, 8, 9, 10 l0 ≥ 12 other Td Oh Oh Kh l0 = 3, 6, 7, 9, 10 l0 = 4, 8 l0 ≥ 12 other Th Th Oh Th Kh l0 = 4, 6, 8, 9, 10 l0 ≥ 12 other O Oh Oh Kh l0 = 4, 6, 8, 9, 10 l0 ≥ 12 other Oh Oh Oh Kh Caption l0 ≥ 12 other I Ih Ih Kh Caption l0 ≥ 12 other Ih Ih Ih Kh

⊕r Table 19: Symmetry infimum of infinite subgroup of O(3) on Vl=l − , r > 3, l0 > 0. 0

C∞ C∞h = S2∞ C∞v D∞ D∞h = D∞d K Kh

l0 even D∞ Kh Kh D∞ Kh Kh Kh

l0 odd C∞v Kh C∞v Kh Kh Kh Kh

⊕r Table 20: Symmetry infimum of infinite subgroup of O(3) on Vl=l + , r > 3, l0 > 0. 0

C∞ C∞h = S2∞ C∞v D∞ D∞h =D∞d K Kh

l0 even D∞h D∞h D∞h D∞h D∞h Kh Kh

l0 odd C∞h C∞h Kh Kh Kh Kh Kh

Published as a conference paper at ICLR 2026

D ETAILED E XPERIMENT

Experiments in § 6.1 and 6.2 are conducted with randomly initialized TFN (Thomas et al., 2018) and HEGNN (Cen et al., 2024) architectures as the underlying equivariant models, whereas the experiments in § 6.3 first pretrain a HEGNN encoder to obtain molecular embeddings and then fine-tune separate MLP heads for the final prediction task under different settings. For TFN (Thomas et al., 2018), we use the implementation from GWL-test (Joshi et al., 2023), which relies on irreducible-representation features with alternating parity. For HEGNN, we extend the original design to a multi-channel variant. In particular, the spherical scalarized features computed for nodes j and i are defined as √ (l) (l) (l) zij,c = 1/ C · ⟨ṽi,c , ṽj,c ⟩, (109) (l)

(l)

where ṽi,c and ṽj,c denote the l-th degree steerable features of channel c among a total of C channels. To obtain the degree-l0 graph-level representation, we apply global mean pooling for each graph G(V, E): P (l) (l) ṽG,c = 1/|V| · i ṽi,c . (110) We will detail, in this section, how each experimental task processes these graph-level features. F.1

V ISUALIZATION OF R EPRESENTATION S PACE

We use a TFN with single-layer to calculate the embedding. After that, to extract features of degree l0 , we append an o3.Linear layer from e3nn (Geiger & Smidt, 2022), denoting this setup as TFNl=l0 in our experiments. F.2

E XPRESSIVITY ON S YMMETRIC G RAPHS

This section first introduces the detailed settings of § 6.2 in § F.2.1, and then introduces the reproduction of the original experiment of GWL-test (Joshi et al., 2023) in § F.2.2. F.2.1

E MBEDDING D IFFERENCE N ORM E XPERIMENT

We employed both TFN (Thomas et al., 2018) and HEGNN (Cen et al., 2024) to compute the norm of the embedding difference across 12 configurations for each model, varying the number of channels (1, 4, 16) and layers (1, 2, 3, 4). The degree-l discrepancy between two graphs G0 and G1 is defined as PC (l) (l) ∆(l) = 1/|C| · c=1 ∥ṽG0 ,c − ṽG1 ,c ∥. (111) These choices give rise to 2 × 3 × 4 = 24 distinct ∆(l) . In Fig. 5, we report the maximum norm across all ∆(l) . Since every norm is strictly positive, a maximal value below 10−6 indicates that all corresponding norms fall below 10−6 , meaning none of them can distinguish G0 from G1 . F.2.2

O RIGINAL GWL-T EST ON S YMMETRIC G RAPHS

Dataset. Same as the setting in (Joshi et al., 2023), we construct four symmetric k-fold structures (k ∈ {2, 3, 4, 6}), each centered at the origin. For each structure G0 we apply a random rotation to produce G1 which ensures G1 does not coincide with the original G0 . The goal is to evaluate whether different equivariant neural network architectures can distinguish G0 from G1 . To validate distinct aspects of our theory, we consider rotations separately in 2D and 3D; in the 3D experiments we additionally ensure that G1 is not coplanar with G0 . Embeddings. The extracted l0 -degree embeddings from TFN are fed into a vanilla classifier for the classification task. The model was trained for 300 epochs to ensure the classifier had sufficient capacity to discriminate the classes.

Results. The detailed experimental results are presented in Table 21, which can be observed that the color blocks in this table are completely consistent with Fig. 5. It demonstrates that our theoretical predictions in Table 1 are in complete agreement with the empirical findings obtained from the model. 45

Published as a conference paper at ICLR 2026

Such remarkable consistency not only confirms the correctness of our theoretical analysis, but also highlights the importance of constructing mappings with appropriately chosen features. Table 21: Results of distinguishing k-fold structures rotated in 2D/3D space. GNN Layer

2 fold

TFNl=0 TFNl=1 TFNl=2 TFNl=3 TFNl=4 TFNl=5 TFNl=6 TFNl=7 TFNl=8 TFNl=9 TFNl=10 TFNl=11

50.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0

F.3 F.3.1

2D Rotational Symmetry 3 fold 4 fold 50.0 ± 0.0 50.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 100.0 ± 0.0 100.0 ± 0.0 100.0 ± 0.0 100.0 ± 0.0 100.0 ± 0.0 100.0 ± 0.0

50.0 ± 0.0 50.0 ± 0.0 50.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0

6 fold

2 fold

50.0 ± 0.0 50.0 ± 0.0 50.0 ± 0.0 50.0 ± 0.0 50.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0

50.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0

3D Rotational Symmetry 3 fold 4 fold 50.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 100.0 ± 0.0 100.0 ± 0.0 100.0 ± 0.0 100.0 ± 0.0 100.0 ± 0.0 100.0 ± 0.0 100.0 ± 0.0 100.0 ± 0.0 100.0 ± 0.0

50.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0

6 fold 50.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0 100.0 ± 0.0 50.0 ± 0.0

M OLECULE P ROPERTY P REDICTION WITH P RETRAINED E QUIVARIANT F EATURES D ETAILED E XPERIMENTAL S ETUP

We employ HEGNN as the encoder in our experiments. TFN is computationally prohibitive in this setting: its tensor-product operator has a complexity of O(L6 ), and when the maximum degree is set to L = 11 (the upper limit supported by e3nn), training a four-layer TFN with only four irreducible-representation channels on QM9 requires roughly 10 A100 GPU-hours per epoch. This far exceeds any practical budget. In contrast, HEGNN adopts spherical scalarization, where interactions across different degrees are mediated solely through scalars, reducing the complexity to O(L2 ). For this reason, we choose HEGNN as our encoder. Concretely, we use a four-layer HEGNN with 16 irreducible-representation channels and a hidden dimension of 64. The resulting features are passed through a scalarization layer and then into a two-layer MLP to predict the molecular isotropic polarizability α on QM9. During fine-tuning, we freeze the encoder, apply a mask to selectively remove information, and train a separate two-layer MLP for each setting. Specifically, our designed mask multiplies the features of the unselected degrees by 0, followed by scalarization calculation, that is: ṽc(l) = HEGNN(G),

(112)

(l)

s = vec([⟨ṽc(l) , ṽc(l) ⟩])C×C , l = 1, . . . , 11, 1 2 (0) (1) α̂ = MLPfinetune (ṽ , s , . . . , s(11) ),

(113) (114)

Following the standard protocol, we train on the first 110k molecules for 300 epochs and fine-tune for an additional 30 epochs. Notably, although the final visualizations are produced by running the trained models on the entire QM9 dataset5 , this does not affect our theoretical conclusions, as our analysis relies solely on horizontal comparisons rather than absolute predictive performance. F.3.2

C ASE S TUDIES

To further validate our theory, we analyze three prominent point groups C2h , C3h , and Td (see § F.3.2). Each exhibiting a representative pattern of symmetry increase. These groups display a symmetry increase to the full group O(3) under specific conditions: for C2h when l0 is odd, for C3h when l0 = 1, and for Td when l0 = 1, 2, 5. This increase corresponds to full degeneration, making the features non-discriminative. The impact of this degeneration is evident in our experiments. Due to the full degeneration of 1-degree features, introducing them paradoxically decreases predictive performance for molecules with C2h , C3h , and Td symmetry. Following the introduction of 2-degree features, a marked improvement in performance is observed for C2h and C3h . Conversely, for Td , performance again decreases, a result directly attributable to the fact that its 2-degree features also undergo full degeneration. 5 Minor discrepancies between l = 0 in Fig. 6 and l ≤ 0 in § F.3.2 result from not fixing the random seed and do not affect the conclusions.

MAE loss (log scale)

Published as a conference paper at ICLR 2026

C1 (111,935 samples)

9 10 11

Model Degree 0 ~ l

9 10 11

Model Degree 0 ~ l

9 10 11

Model Degree 0 ~ l

9 10 11

C v (8 samples)

9 10 11

Td (5 samples)

9 10 11

D3h (15 samples)

D h (8 samples)

D2h (6 samples)

9 10 11

C2v (433 samples)

Ci (15 samples)

C3 (11 samples)

D3d (6 samples)

C2 (458 samples)

C2h (48 samples)

D2d (11 samples)

C3v (67 samples)

Cs (17,794 samples)

9 10 11

C3h (4 samples)

9 10 11

Model Degree 0 ~ l

9 10 11

Table 22: Symmetry infimum of point group symmetry on the QM9 dataset. C1 Cs C2 C2v C3v C2h Ci = S2 D3h D2d C3 D∞h C∞v D3d D2h Td C3h

1 C1 Cs C∞v C∞v D∞h Kh Kh Kh Kh C∞v Kh C∞v Kh Kh Kh Kh

2 Cs C2h C2h D2h C∞v C2h Ci D∞h D∞h D∞h D∞h D∞h D∞h D2h Kh D∞h

3 C1 Cs C2 C2v C3v Kh Kh D3h Td C3 Kh C∞v Kh Kh Td C3h

4 Cs C2h C2h D2h D3d C2h Ci D∞h D4h C3h D∞h D∞h D3d D2h Oh D∞h

5 C1 Cs C2 C2v C3v Kh Kh D3h D2d C3 Kh C∞v Kh Kh Kh C3h

6 Cs C2h C2h D2h D3d C2h Ci D6h D4h C3h D∞h D∞h D3d D2h Oh C6h

7 C1 Cs C2 C2v C3v Kh Kh D3h D2d C3 Kh C∞v Kh Kh Td C3h

8 Cs C2h C2h D2h D3d C2h Ci D6h D4h C3h D∞h D∞h D3d D2h Oh C6h

9 C1 Cs C2 C2v C3v Kh Kh D3h D2d C3 Kh C∞v Kh Kh Td C3h

10 Cs C2h C2h D2h D3d C2h Ci D6h D4h C3h D∞h D∞h D3d D2h Oh C6h

11 C1 Cs C2 C2v C3v Kh Kh D3h D2d C3 Kh C∞v Kh Kh Td C3h

The experiment demonstrates a model design principle: not only should one avoid building a model entirely from fully degenerate features, but one should also avoid including individual feature components that undergo full degeneration, as they can be harmful to predictive performance.

Reducing Symmetry Increase in Equivariant Neural Networks

Analyze & explore related research in Conceptio

Related documents