arXiv CS open access

Tensor Field Models

2026 · arxiv_cs

arXiv CS · Papers · 2026

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

Tensor Field Models

arXiv:2608.18808v1 [cs.LG] 19 Aug 2026

Alexander Strunk∗ Evercot AI

Roland Assam Evercot AI 19 June 2026 Abstract

This paper introduces Tensor Field Models (TFMs), realization-level Mathematical Structures in which a learned Operator maps a product of admissible componentsection families to a prescribed family of time-dependent tangent sections on a Generative State Manifold. Analytic and dynamical restrictions are encoded through the choice of admissible families rather than imposed by the root definition. Constructed, component-separable, and Tensor Bundle TFMs provide structured refinements of this common object. In the conditional realizations considered here, a structured condition c = (c1 , . . . , cn ) is mapped componentwise to a reusable (1) (n) collection Hc = (Hc1 , . . . , Hcn ). In the architectures evaluated here, the component representations remain distinct and are combined only by the Field Operator to produce the generated Vector Field. All learned models are trained using Flow Matching. Experiments show that TFMs can improve performance and that amortized sampling enabled by reusable condition representations can accelerate generation.

∗

Corresponding author: astrunk.research@evercot.ai

Introduction

A Tensor Field Model (TFM) is a realization containing its geometric domains, admissible input- and output-section families, parameter space, and parameterized Field Operator: T̂ = (M, I, {(Ωi , Ei , Hi )}ni=1 , V, Φ, F) Here, M denotes the Generative State Manifold, I denotes the time interval, and each πi : Ei → Ωi is a smooth Vector Bundle. Furthermore, Sec(Ei ) represents the space of sections of Ei , while Hi ⊆ Sec(Ei ) denotes a prescribed family of admissible sections.

① ·

& *

Figure 1: A Tensor Field Model can be visualized as a collection of blue component Fields on latent Bundles. Through the Operator Fϕ , each admissible Field collection defines a red time-dependent tangent section on the Generative State Manifold M. This supports batched or parallel evaluation of independent queries while reusing the same Field collection. The prescribed output family satisfies V ⊆ Sec(pr∗M T M), and the parameterized Field Operator has type F:Φ×

n Y

(n)

Hi −→ V

Fϕ [H] := F(ϕ, H)

i=1 (n)

Thus Fϕ [H](x, t) ∈ Tx M. 1

point evaluations query q1 = (x1 , t1 ) vbc (x1 , t1 ) = Fϕ (Hc , x1 , t1 ) structured condition c = (ci )ni=1

component constructors (i) Eθi : Ci → Hi

Field collection (i) Hc = Eθ (c) = (Hci )ni=1

query q2 = (x2 , t2 ) vbc (x2 , t2 ) = Fϕ (Hc , x2 , t2 ) query qNq = (xNq , tNq ) vbc (xNq , tNq ) = Fϕ (Hc , xNq , tNq )

Figure 2: Point-evaluation formulation of a component-separable constructed Tensor Field Model. The component constructors produce Hc = Eθ (c). For Nq queries qj = (xj , tj ) sharing the condition c, the point-evaluation map Fϕ reuses this collection to return vbc (xj , tj ) ∈ Txj M. The state xj remains an evaluation argument. In a dynamical specialization, each trajectory can therefore be evaluated at its current state. Q Conditional Q realizations augment the Operator with a constructor Eθ : C → ni=1 Hi . When C = ni=1 Ci , this constructor may additionally factorize componentwise. For a constructed conditional TFM, the conceptual change is from directly learning c 7→ vc in a high-dimensional section family to learning the composition c 7→ Hc 7→ vbc . When the component Field families have lower effective capacity than the original conditional output family, their collection can provide a reduced representation. This reduction is an architectural assumption, not a theorem following from the existence of the factorization: without restrictions on the intermediate families and Operator, an arbitrary conditional predictor can be written in factorized form. The model therefore does not treat each output-section evaluation as an unrelated prediction. Instead, it reuses the same collection in all evaluations under c. A condition-only construction is computationally useful because continuous-time generative models repeatedly query a state- and time-dependent predictor under fixed conditional information. For Nq such queries, the component Fields are constructed once and the point-evaluation Operator is evaluated Nq times. Whether this is faster than an unfactorized predictor depends on the relative construction and per-query costs analyzed in subsection 3.2.

Mathematical Background

2.1

Tensor Bundles and Tensor Fields

A Tensor Bundle is a smooth Vector Bundle whose Fiber over each point is a Tensor Product space formed from the corresponding Fiber of a specified Vector Bundle and its Dual. More precisely, let M be a smooth d-dimensional Manifold and let πC : C → M be a smooth real Vector Bundle of finite rank q. This carrier Bundle C supplies the Vector Spaces on which the Tensors are formed and need not equal T M . For r, s ∈ N0 , the Tensor Bundle of type (r, s) carried by C is G Trs (C) := C ⊗r ⊗ (C ∗ )⊗s = Trs (C)x x∈M r ⊗r Ts (C)x := Cx ⊗ (Cx∗ )⊗s

Here, C ∗ → M is the Dual Bundle, and a zero-fold Tensor power is understood to be the trivial line Bundle. The induced projection πsr : Trs (C) → M sends every Tensor in Trs (C)x to x.

↑

& Figure 3: A Tensor Bundle induced by a carrier Bundle C → M . The lower region represents the Base Manifold M , and the regions above selected base points represent Fibers Trs (C)x . A smooth section assigns to every x ∈ M one Tensor A(x) in the corresponding Fiber. The carrier Fibers Cx need not be tangent spaces of M . A smooth C-Tensor Field of type (r, s) is a smooth section of this Bundle: A ∈ Γ(Trs (C))

πsr ◦ A = idM 3

Thus A denotes the Field, whereas A(x) ∈ Trs (C)x denotes its value over x. The word “Tensor” refers to the multilinear structure of the carrier Fiber Cx . It does not require Cx to be the tangent space Tx M . The Base Manifold M parametrizes the Fibers on which the Tensorial data live. Let e1 , . . . , eq be a local frame of C over an open set U ⊆ M , and let e1 , . . . , eq be its dual frame. A C-Tensor Field has the local representation A|U = Aa1 ...ar b1 ...bs (x) ea1 ⊗ · · · ⊗ ear ⊗ eb1 ⊗ · · · ⊗ ebs where the Fiber indices range from 1 to q. If another frame is given by eea = ei Gi a , with G : U → GL(q, R), then the components satisfy ea1 ...ar b1 ...bs = (G−1 )a1 i1 · · · (G−1 )ar ir Gj1 b1 · · · Gjs bs Ai1 ...ir j1 ...js A Repeated Fiber indices are summed. This frame-transformation law, rather than the mere presence of several array indices, characterizes a geometric Tensor. In general, G is a transition function of C, not the Jacobian of a coordinate change on M . Moreover, the carrier rank q need not equal the base dimension d. 2.1.1

Ordinary Tensors on the Base Manifold

The familiar Tensor Bundles on M are recovered by choosing the carrier Bundle C = T M : Trs M := Trs (T M )

(Trs M )x = (Tx M )⊗r ⊗ (Tx∗ M )⊗s

For a coordinate chart (U, x1 , . . . , xd ), the coordinate frame ∂i = ∂/∂xi and its dual frame dxj give T |U = T i1 ...ir j1 ...js (x) ∂i1 ⊗ · · · ⊗ ∂ir ⊗ dxj1 ⊗ · · · ⊗ dxjs Under a coordinate change x 7→ x e, the general frame-transformation law specializes to ∂e xa1 ∂e xar ∂xj1 ∂xjs i1 ...ir Tea1 ...ar b1 ...bs = T · · · · · · j1 ...js ∂xi1 ∂xir ∂e xb1 ∂e xbs Hence Scalar Fields f ∈ C ∞ (M ), Vector Fields X ∈ Γ(T M ), One-Forms ω ∈ Γ(T ∗ M ), and Riemannian metrics g ∈ Γ(T02 M ) are the respective Base Manifold examples of types (0, 0), (1, 0), (0, 1), and (0, 2). For contrast, if C = M × V is a trivial carrier Bundle, then a C-Tensor Field is locally a smooth map from M into V ⊗r ⊗ (V ∗ )⊗s . Its indices are internal V -indices and are unaffected by a change of coordinates on M , although they do transform under a change of frame in V . Nontrivial carrier Bundles give the corresponding construction without requiring a global frame.

2.1.2

Mathematical Operations on Tensors

Tensor Operations are defined fiberwise and, when the resulting pointwise assignment is smooth, map Tensor Fields to Tensor Fields. Let C → M be a fixed carrier Bundle. The fundamental algebraic and functorial operations are the following: • Tensor Product: For A ∈ Γ(Trs (C)) and B ∈ Γ(Tuv (C)), their Tensor Product is the type-(r + u, s + v) C-Tensor Field defined by A ⊗ B ∈ Γ Tr+u (A ⊗ B)(x) = A(x) ⊗ B(x) s+v (C) Thus the operation concatenates the contravariant and covariant carrier slots of the two Tensors. • Contraction: Contraction pairs one contravariant slot with one covariant slot by the natural Duality between Cx and Cx∗ . For r, s ≥ 1, 1 ≤ a ≤ r, and 1 ≤ b ≤ s, contracting any selected pair gives Ctra,b : Γ(Trs (C)) −→ Γ Tr−1 s−1 (C) For example, contraction of the first upper and lower indices has the local expression (Ctr1,1 T )i2 ...ir j2 ...js = T ki2 ...ir kj2 ...js The repeated Fiber index k is summed, and the result is independent of the chosen local frame. • Permutation: The contravariant slots and the covariant slots may each be reordered without changing the Tensor type. For σ ∈ Sr and τ ∈ Ss , (Pσ,τ T )i1 ...ir j1 ...js = T iσ(1) ...iσ(r) jτ (1) ...jτ (s) Linear combinations of such permutations yield, for example, symmetrization and antisymmetrization. For a covariant two-Tensor A, Alt(A)ij = 12 (Aij − Aji )

Sym(A)ij = 12 (Aij + Aji )

An upper slot cannot be converted into a lower slot canonically because this would require a chosen identification C ∼ = C ∗ , such as one induced by a Bundle metric. • Pullback of the Carrier Bundle: Let f : L → M be smooth. The pullback carrier Bundle f ∗ C → L has Fiber (f ∗ C)x = Cf (x) , and there is a canonical Bundle isomorphism f ∗ Trs (C) ∼ = Trs (f ∗ C) Consequently, every A ∈ Γ(Trs (C)) induces f ∗ A ∈ Γ(Trs (f ∗ C)) with (f ∗ A)(x) = A(f (x)). This pullback is available for every Tensor type because it retains the pulled-back carrier f ∗ C. It does not by itself identify that carrier with T L.

• Transport by a Bundle Map: Let C → L and D → M be carrier Bundles, and let Φ : C → D be a fiberwise-linear Bundle map covering f : L → M . A covariant D-Tensor Field S ∈ Γ(T0s (D)) pulls back to a C-Tensor Field by (Φ∗ S)x (v1 , . . . , vs ) = Sf (x) Φx v1 , . . . , Φx vs v1 , . . . , vs ∈ Cx A contravariant Tensor Ax ∈ Cx⊗r is pushed forward fiberwise: Φ∗x Ax = (Φx )⊗r Ax ∈ Df⊗r(x) In general this produces a Tensor Field along f . If f is a diffeomorphism and Φ is a Vector Bundle isomorphism, covariant and contravariant slots combine to transport mixed Tensor Fields. For ordinary Tensors on the Base Manifolds, the choice Φ = df : T L → T M recovers the usual pullback of covariant Tensors and pushforward of contravariant Tensors. For example, (f ∗ ω)x (v) = ωf (x) (dfx v),

f∗x Xx = dfx (Xx ).

For detailed treatments of these constructions, see [1, 2, 3, 4, 5, 6].

Tensor Field Models

A Tensor Field Model (TFM) with n components is the following datum: T̂ = (M, I, {(Ωi , Ei , Hi )}ni=1 , V, Φ, F) For any Bundle πE : E → B, the set of sections Sec(E) is defined as Sec(E) := {s : B → E : πE ◦ s = idB } The entries of T̂ satisfy the following typing conditions. The generative state space M is a smooth Manifold, and the time domain I ⊆ R is a nondegenerate interval, viewed as a smooth one-dimensional Manifold, possibly with boundary. For each i = 1, . . . , n, Ωi is a smooth Base Manifold, πi : Ei → Ωi is a finite-rank smooth Vector Bundle, and Hi is a specified nonempty family of admissible component sections: ∅ ̸= Hi ⊆ Sec(Ei ) Define the product of admissible component sections by H

(n)

n Y

H = (H (1) , . . . , H (n) ) ∈ H(n)

Hi ,

i=1

Let prM : M × I → M denote projection onto the state, and let πM : T M → M be the Tangent Bundle projection. The pullback Tangent Bundle has Fiber (pr∗M T M)(x,t) ∼ = Tx M The output datum is a specified nonempty admissible family ∅ ̸= V ⊆ Sec(pr∗M T M) There is a canonical identification Sec(pr∗M T M) ∼ = {v : M × I → T M : πM ◦ v = prM } Under this identification, an output is a time-dependent tangent Field v satisfying v(x, t) ∈ Tx M. Φ is a specified nonempty parameter set, equipped with any topological or smooth structure required by the application. The Field Operator datum F : Φ × H(n) −→ V is a parameterized Field Operator. No linearity, locality, or differentiability is assumed unless it is separately imposed. For fixed ϕ ∈ Φ and H ∈ H(n) , write (n)

Fϕ [H] := F(ϕ, H) Pointwise evaluation is (n) Fϕ (H, x, t) := Fϕ [H] (x, t)

Fϕ (H, x, t) ∈ Tx M 7

3.1

Hierarchy of Tensor Field Models

The realization T̂ is the common root of the hierarchy. C-TFMs and CS-TFMs form the construction branch by adding a constructor and then a componentwise factorization of that constructor. TB-TFMs add Tensor Bundle typing to the component Bundles. This typing can be combined with either construction class when the corresponding Tensor specifications and Bundle isomorphisms are supplied. 3.1.1

Construction Hierarchy

Let C be a specified nonempty condition set and Θ a specified nonempty constructorparameter set, equipped with additional structure when required. A Constructed Tensor Field Model (C-TFM) is the augmented realization T̂C = T̂ , C, Θ, E E : Θ × C −→ H(n) For fixed θ ∈ Θ, write Eθ (c) := E(θ, c) = Hc The requirement that Eθ depend on c but not on the evaluation query (x, t) is part of the C-TFM structure. Its conditional tangent section is the composition (n)

c 7−→ Hc = Eθ (c) 7−→ vbc = Fϕ [Hc ] ∈ V Q Q The constructor may depend jointly on all parts of c. If C = ni=1 Ci , Θ = ni=1 Θi , and c = (c1 , . . . , cn ), a Component-Separable Tensor Field Model (CS-TFM) is the augmented realization T̂CS = T̂C , {Ei }ni=1 with specified component constructors Ei : Θi × Ci −→ Hi

i = 1, . . . , n

For fixed θ = (θ1 , . . . , θn ), define the component Fields by (i)

Hc(i) := Eθi (ci ) := Ei (θi , ci ) ∈ Hi , i

i = 1, . . . , n

such that Eθ (c) = Hc = Hc(1) , . . . , Hc(n) n 1

In the conditional forward map, cross-component interactions can occur only through (n) Fϕ , while the component families may have different geometric types, resolutions, or latent representations. Forgetting the componentwise factorization yields a C-TFM, and forgetting the remaining construction data yields the underlying TFM realization: CS-TFM −→ C-TFM −→ TFM. These arrows schematically denote the corresponding forgetful maps between structured realizations. 8

3.1.2

Tensor-Bundle Tensor Field Models

For a fixed Base Manifold Ω, a Tensor Specification is a finite list S = {(Ca , ra , sa , Va )}m a=1 Here m ≥ 1, each Ca → Ω is a finite-rank carrier Bundle, ra , sa ∈ N0 , and Va is a finitedimensional real Vector Space. Write V a := Ω × Va . The specification determines the Tensor-structured Bundle Ten(S) :=

m M

Trsaa (Ca ) ⊗ V a

a=1

The space Va is a multiplicity space that allows several channels of the same Tensor type. A Tensor-Bundle Tensor Field Model (TB-TFM) is a TFM together with one Tensor specification Si for each component and a Vector Bundle isomorphism ∼ T̂TB := T̂ , {(Si , τi )}ni=1 τi : Ei −→ Ten(Si ) where τi covers idΩi . The admissible family Hi remains a specified subset of Sec(Ei ). i Writing Si = {(Cia , ria , sia , Via )}m a=1 , this definition separates the two roles clearly: Cia supplies the Tensor indices, while Ei is the component Bundle used by the TFM. Ordinary Tensors on Ωi use Cia = T Ωi . Internal Tensors may instead use Cia = Ωi × Wia , or any other prescribed carrier Bundle over Ωi . In each summand the Tensor slots transform under changes of frame of Cia , while the multiplicity coordinates in Via are unchanged. A TB-TFM is a TFM with additional geometric typing data, not necessarily a smaller class of underlying models. The typing alone imposes no symmetry condition on the constructors or Field Operator. 3.1.3

Hierarchy Overview

The hierarchy consists of one construction branch and one compatible Tensor Bundle refinement. Tensor Bundle structure adds Tensor specifications and Bundle identifications and can be combined with a TFM, C-TFM, or CS-TFM. Combined names concatenate only nonredundant prefixes: because every CS-TFM is already a C-TFM, for example, a component-separable Tensor Bundle realization is called a CS-TB-TFM. ( C-TFM ⊃ CS-TFM, TFM ⊃ TB-TFM. Here ⊃ denotes the hierarchy relation “is more general than”: forgetting the additional data and compatibility conditions of a realization on the right yields a realization of the class on the left. It is not meant as literal set inclusion, nor must the relation be proper. The Tensor Bundle refinement can be combined with either construction class when the required specifications and Bundle identifications are supplied. Tangentoutput compatibility and membership in V belong to the root definition, and exact reuse follows from a condition-only constructor, so neither forms a separate refinement. ODEadmissibility and other analytic restrictions are determined by the chosen output family relative to the declared solution concept.

3.2

Amortized Point Evaluation

For a C-TFM, the constructor depends on c but not on the query (x, t). Hence a batch Nq Bc = {qj = (xj , tj )}j=1 sharing the same condition also shares the single Field collection Hc : vbc (xj , tj ) = Fϕ (Hc , xj , tj ) ∈ Txj M

j = 1, . . . , Nq

This is a computational consequence of the factorization, not an additional geometric refinement. It supports batched or parallel evaluation. The following idealized additive cost model assumes query-independent costs. Let Cbuild ≥ 0 be the cost of constructing Hc , CF > 0 the cost of one point evaluation, and CG > 0 the cost of one query to an unfactorized predictor. Then CTFM (Nq ) = Cbuild + Nq CF

Cbase (Nq ) = Nq CG

For Nq ∈ N, reuse is beneficial precisely when Cbuild + Nq CF < Nq CG

equivalently

Cbuild + CF < CG Nq

If CG ≤ CF , this inequality cannot hold. If CG > CF , it is equivalent to the crossover condition Nq >

Cbuild CG − C F

The corresponding idealized speedup is SNq =

Nq CG CG −→ Cbuild + Nq CF CF

as Nq grows. Thus, construction is amortized across the shared queries, while the largefan-out speedup is bounded by the relative per-query costs.

3.3

Training Objectives

Here, E denotes expectation under task-specific probability laws on the relevant measurable spaces. Every displayed random quantity and loss integrand is assumed measurable and integrable whenever the corresponding loss is used. To define the training objectives for constructed Tensor Field Models, equip M with a Riemannian metric g and define ℓx (u, v) = gx (u − v, u − v),

u, v ∈ Tx M

For each condition c, construct Hc = Eθ (c) once and share it across the Nq state–time queries. Given target Vectors vj⋆ ∈ Txj M, direct training minimizes # Nq X 1 (n) Ltask = E ℓx Fϕ (Hc , xj , tj ), vj⋆ Nq j=1 j "

This preserves the model factorization: Hc depends only on the shared condition, whereas Fϕ additionally receives the query (xj , tj ). 10

For post-training conversion, let Gref denote a reference Vector Field satisfying Gref (x, t, c) ∈ Tx M For c ∼ ν and (x, t) ∼ µc , pointwise distillation minimizes (n)

Lfield = E[ℓx (Fϕ (Hc , x, t), Gref (x, t, c))] Because both Vector Fields lie in Tx M, no transport between tangent spaces is required. Optional rollout distillation generates reference and TFM trajectories using the same condition, initial state, time grid, and update noise, together with a Manifold-preserving numerical update of the form Ψℓ : {(x, u) : x ∈ M, u ∈ Tx M} × Ξ × I 2 −→ M Using rollouts presupposes that the selected outputs are admissible for the chosen update rule. A continuous-time ODE interpretation additionally requires the TFM and reference outputs to be ODE-admissible under the declared solution concept. Let t0 < · · · < tK = xref be a time grid in I, fix x0 ∈ M, set xTFM t0 t0 = x0 , and use the same noise variables ξ0 , . . . , ξK−1 ∈ Ξ in both rollouts. For ℓ = 0, . . . , K − 1, define TFM xTFM , Fϕ (Hc , xTFM , tℓ ) , ξℓ , (tℓ , tℓ+1 ) tℓ+1 = Ψℓ xtℓ tℓ ref ref xref tℓ+1 = Ψℓ xtℓ , Gref (xtℓ , tℓ , c) , ξℓ , (tℓ , tℓ+1 ) Using the geodesic distance dg , the rollout objective is " K # X 2 (n) Lroll = E αℓ dg xTFM , xref tℓ tℓ

αℓ ≥ 0

ℓ=1

A Field regularizer (n)

LH = E[R(Hc )] may be used to control the norm, smoothness, or redundancy of the component Fields. The complete objective is (n)

(n)

L(n) = λtask Ltask + λfield Lfield + λroll Lroll + λH LH Terms that are not used are assigned zero weight.

λtask , λfield , λroll , λH ≥ 0

Experiments

Three consequences of the preceding formulation are tested. First, a restricted component factorization can improve generalization when the conditional family has matching structure. Second, a typed Field Operator can impose a known symmetry exactly. Third, a condition-only construction can be reused without changing the output. Every learned model below is trained with the same Flow Matching (FM) or Riemannian Flow Matching (RFM) objective as its control [7, 8]. TFM and direct FM therefore distinguish Vector-Field parameterizations under a common objective. Learned-model results use five paired seeds. Reported values are the seed mean and a twosided 95% Student-t confidence interval with four degrees of freedom. Paired percentage reductions are computed within a seed before aggregation. Timing results instead report medians and interquartile ranges over randomized timing blocks.

4.1

Compositional Conditional Flow Matching in R4

Task. Let c = (i, j), with i, j ∈ {1, . . . , 8}, and p1 (x | i, j) = N (µij , σ 2 I4 )

p0 = N (0, I4 )

σ = 0.35

In the compositional task, the means are generated by a bilinear contraction with latent dimension three in both condition modes: µij,d =

3 X 3 X

uir Adrs wjs

d = 1, . . . , 4

r=1 s=1

Consequently, the resulting 8×8×4 Tensor has row and column multilinear ranks at most three. This construction does not assert CP rank three. The means are then scaled using training combinations only to have coordinate RMS 1.5. A fixed perfect matching of the 8×8 condition table is held out. The remaining 56 pairs form a connected bipartite graph, and every row and column value occurs in seven training pairs. Evaluation therefore tests eight unseen combinations of observed components. Both models use the independent straight CFM coupling x0 ∼ N (0, I4 )

x1 = µij + σϵ

xt = (1 − t)x0 + tx1

ut = x1 − x0

where t ∼ Unif[0, 1] and ϵ ∼ N (0, I4 ). In addition to generated samples, evaluation uses the analytic marginal regression Field vmarg (x, t, i, j) = µij +

σ 2 t − (1 − t) (x − tµij ) (1 − t)2 + σ 2 t2

Models and protocol. The CS-TFM constructors return separate learned representations hi , gj ∈ R3 . Only the Field Operator contracts them: zk (i, j) =

3 X

Ckab hi,a gj,b

k = 1, . . . , 8

a,b=1

q(x, t) = MLP32 (x, t)

vbd = [Bq]d +

X h,k

Tdhk qh zk

The query network has two width-32 SiLU layers and 1,248 parameters. The complete CS-TFM has 2,524 parameters. The direct conditional FM control is a two-hiddenlayer, width-38 SiLU MLP applied to [x, t, onehot(i), onehot(j)]. It has 2,474 parameters, 2.0% fewer. Its first affine layer already contains separate row and column embeddings implicitly, while its subsequent layers are not restricted to a Tensor contraction. Within a seed, the models receive identical minibatches and evaluation noise. Training uses 4,000 AdamW updates, batch size 512, learning rate 2 × 10−3 , weight decay 10−6 , gradient clipping at 5, and cosine learning-rate decay to 2 × 10−4 . Evaluation uses 1,024 velocity queries and 512 generated samples per condition. Generation uses 64 Heun steps, and sliced W2 uses 128 random directions and exact Gaussian target quantiles. Results and falsification controls. On the held-out bilinear combinations, the CSTFM reduces sliced W2 by 82.9% (paired 95% CI 68.6%–97.3%). Every seed favors the CS-TFM. Its train and held-out analytic velocity NMSEs are respectively 0.0183 and 0.0191, whereas the direct model changes from 0.0234 on training combinations to 0.3717 on held-out combinations. This is a compositional-generalization result, not merely a better fit to observed pairs. For a falsification control, the bilinear Tensor is replaced by an IID Gaussian 8 × 8 × 4 mean table, and the complete protocol is repeated. An unseen entry then contains no recoverable row–column information. The CS-TFM no longer improves generation: its sliced W2 is 2.022, versus 1.944 for direct FM, a paired reduction of −4.1% [−15.6%, 7.5%]. It also fits the observed IID table much less accurately (training velocity NMSE 0.325, versus 0.0264), as expected from its latent-dimension restriction. The main model deliberately matches the simulator’s latent dimension so that the controlled test isolates the proposed inductive bias. A latent-dimension sensitivity check makes this dependence explicit. Mild over-specification with model latent dimension four retains a 79.1% sliced-W2 reduction [63.3%, 95.0%], while latent dimension two gives no reliable benefit (−59.0%, [−143.0%, 25.0%]).

4.2

Exactly Equivariant RFM on S2

Task. Conditions are c = (a, b) ∈ S2 × S2 . Training uses the canonical orientation a = e3

b = (sin φ, 0, cos φ)

φ ∼ Unif[35◦ , 85◦ ]

and the endpoint law a + 0.75b + 0.35(a × b) p1 (x | a, b) = vMF , 18 ∥a + 0.75b + 0.35(a × b)∥ For x0 ∼ Unif(S2 ), x1 ∼ p1 , and ϑ = arccos⟨x0 , x1 ⟩, one has ϑ ∈ (0, π) almost surely under these continuous laws. On this event, let d = (x1 − cos ϑ x0 )/ sin ϑ. The shortest-geodesic CFM path and its tangent target are xt = cos(tϑ)x0 + sin(tϑ)d

ut = ϑ[− sin(tϑ)x0 + cos(tϑ)d]

The implementation supplies a stable tangent direction in the probability-zero coincident and antipodal cases.

Models and protocol. Write Px = I − xx⊤ . The CS-TB-TFM constructs typed component Fields from a and b, and its shared Operator multiplies the five tangent basis Fields B(x, a, b) = Px a, Px b, Px (a × b), x × a, x × b by coefficients from a width-96, three-hidden-layer SiLU MLP. That MLP receives only t, ⟨x, a⟩, ⟨x, b⟩, ⟨a, b⟩, ⟨x, a × b⟩ All inputs to the coefficient map are invariant, while the basis Fields are covariant. Hence, for every parameter value and R ∈ SO(3), vb(Rx, t, Ra, Rb) = Rb v (x, t, a, b)

vb(x, t, a, b) ∈ Tx S2

The model has 19,685 parameters. The control is a width-95, three-hidden-layer coordinate MLP on [x, t, a, b], followed by the same projection Px . It has 19,573 parameters (0.57% fewer). Thus tangency is not a confound. The CS-TB-TFM and one coordinate-MLP control are trained on canonical examples without augmentation. A second coordinate-MLP control, initialized identically to the first, receives a fresh Haar rotation applied jointly to (x, a, b, ut ) on every training example. All three models share the underlying minibatch stream. Training uses 3,000 AdamW updates, batch size 512, learning rate 2 × 10−3 , weight decay 10−6 , and gradient clipping at 10. Evaluation uses 8,192 paired canonical examples and jointly Haar-rotated copies. The rotated raw inputs lie outside the sampled canonical training support of both the CS-TB-TFM and the canonical-only coordinate MLP, and they are in distribution for the augmented control. For the CS-TB-TFM, however, rotation leaves the invariant coefficient inputs unchanged and transforms the basis Fields covariantly, so its behavior on the rotated inputs is fixed exactly by equivariance. Generation uses 2,048 paired initial states and 48 embedded Heun steps with normalization retraction. Results. Canonical velocity NMSE is essentially equal for the CS-TB-TFM and canonical-only coordinate MLP: 0.2983 and 0.2988, respectively. After Haar rotation, the TFM mean remains 0.2983, with canonical and rotated NMSE equal within every seed up to numerical precision, while the raw coordinate MLP is 5.9–50.2 times worse seedwise. The geometric-mean error ratio is 12.9 (log-ratio 95% CI 4.0–41.4). The TFM’s normalized equivariance defect is 3.0 × 10−14 . Haar augmentation largely closes the gap. Nevertheless, at the same update budget the CS-TB-TFM has 5.92% lower Haar-rotated velocity NMSE than the augmented control (paired 95% CI 4.94%–6.89%). Their generated distributions are comparable: axial W1 is 0.0060 for the TFM and 0.0083 for the augmented control, whereas it is 0.573 for the rotated canonical-only MLP. The result is therefore interpreted as exact rotational generalization and a fixed-budget inductive-bias benefit, not as proof that augmentation or another equivariant architecture cannot match it.

4.3

Condition-Reuse Audit

The additive law in subsection 3.2 is tested separately with small deterministic scalar Python MLPs on one Apple-arm64 CPU. The TFM has 1,164 parameters, 768 construction multiply–accumulates (MACs), and 300 Operator MACs per query. A parametermatched direct MLP has 1,180 parameters and 1,148 MACs per complete query. For 14

Table 1: Primary held-out results. Model entries are mean [95% Student-t CI] across five seeds. The last column is the mean seedwise percentage reduction of the TFM relative to its named control. The Euclidean control is direct FM. The spherical control is the Haar-augmented projected MLP. Lower is better. Metric sliced W2 sliced W2 velocity NMSE

Structured TFM 0.0549 [0.0480, 0.0617] 2.022 [1.599, 2.446] 0.2983 [0.2946, 0.3020]

(a) Euclidean composition

(b) Sphere rotations

CS-TFM Unfactorized CFM

101

Sliced-W2 ↓

Velocity NMSE ↓

100

10−1

Bilinear structure

IID full-rank control

Control 0.447 [0.116, 0.779] 1.944 [1.610, 2.278] 0.3171 [0.3100, 0.3241]

CS-TB-TFM Projected MLP Haar-aug. MLP

100

Canonical

Haar-rotated

Paired reduction 82.9% [68.6%, 97.3%] −4.1% [−15.6%, 7.5%] 5.92% [4.94%, 6.89%]

Setting Bilinear R4 IID R4 control Haar-rotated S2

Recomputed identical TFM Unfactorized baseline Partially cached exact Generic cached control

2 cached TFM

1 1

256

1024

Queries per condition, Nq

Figure 4: Controlled experimental summary. (a) Seedwise held-out sliced W2 . Gray segments pair the same seed and black bars mark medians. The CS-TFM advantage on the bilinear family disappears for the IID control. (b) Canonical and jointly Haar-rotated velocity NMSE. All models are tangent. Pale lines are individual seeds and heavy lines are medians. The rotated raw inputs lie outside the sampled canonical training support of both the CS-TB-TFM and the unaugmented coordinate MLP. In the CS-TB-TFM, rotation leaves the coefficient inputs invariant and transforms the basis Fields covariantly. The augmented MLP sees such rotations during training. (c) Median serial CPU latency divided by cached-TFM latency. Rebuilding the identical TFM or recomputing the direct predictor is costly, but a generic cached graph and an exactly partially cached MLP match the TFM, showing that reuse is not unique to the terminology. Nq ∈ {1, 2, 4, . . . , 1024}, the timing protocol uses 31 randomized-order repetitions, five warmups, and disabled garbage collection inside timed blocks. At Nq = 1024, building the TFM once is 4.32× faster than rebuilding the identical TFM at every query and 2.97× faster than recomputing the direct MLP. Cached and recomputed outputs agree exactly. These comparisons alone would overstate the consequence, however. A generic conditioner with the identical TFM graph has the same latency, and exactly caching the direct MLP’s first-layer contribution b + Wc c also preserves its output bitwise. Its cost is 896 + 252Nq MACs, versus 768 + 300Nq for the TFM, so the partially cached ordinary predictor is asymptotically cheaper and is 0.989 times the TFM latency at Nq = 1024 in this run. Thus the cost law and exact reuse are verified, but speed is architecture-, implementation-, and workload-dependent and is not unique to TFMs. The scalar CPU audit is not an end-to-end sampler benchmark. Independent-path batching. An additional systems throughput audit evaluates independent conditional trajectories using the preceding equivariant TFM (19,685 parameters) and the direct projected MLP (19,573 parameters) on an Apple M3 CPU (eight

cores, 8 GB RAM) with PyTorch 2.6, four intra-op threads, eager FP32 execution, and fixed, deterministically initialized (untrained) weights. For each B ∈ {1, 8, 32, 128, 512}, both models receive the same seeded initial states under one fixed condition and integrate them either with an outer Python loop over singleton paths or as one tensor batch. Both use 48 Heun steps (96 function evaluations). The integration steps themselves remain sequential. The condition cache is prepared before the timed integration for both the equivariant TFM and an exactly refactored direct MLP, so this audit measures path batching rather than cache construction. Reported values are medians and inclusive interquartile ranges over 15 randomized-order blocks after three warmups. Table 2: CPU latency for independent-path sampling. Times are median [IQR] in milliseconds. Speedup is the ratio of serial and batched medians within each model. B 1 8 32 128 512

TFM serial ms 6.57 [5.63, 8.23] 52.06 [47.37, 54.71] 242.72 [215.22, 311.89] 1082.72 [840.44, 1161.49] 2923.10 [2880.50, 3010.21]

TFM batched ms 5.83 [5.61, 8.01] 7.88 [7.16, 8.93] 9.60 [8.69, 13.43] 16.32 [14.21, 23.34] 25.92 [25.50, 27.38]

TFM speedup 1.13× 6.61× 25.28× 66.33× 112.76×

Direct serial ms 3.99 [3.91, 4.15] 32.20 [31.58, 33.10] 129.13 [128.45, 129.85] 528.81 [520.16, 535.19] 2103.91 [2069.18, 2165.00]

Direct batched ms 4.04 [3.87, 4.37] 5.04 [4.99, 5.27] 6.34 [6.25, 6.56] 10.41 [10.29, 10.57] 20.75 [19.72, 23.60]

Direct speedup 0.99× 6.39× 20.38× 50.82× 101.41×

At B = 512, vectorization is 112.76× faster for the TFM and 101.41× faster for the direct MLP. The direct MLP also has lower absolute batched latency at every reported B. The serial and batched endpoints agree to a maximum absolute discrepancy of 1.2 × 10−7 . These measurements characterize dense execution throughput only and do not compare sample quality.

Related Work

A useful intuition is to regard the components of c as n condition-dependent sources. (i) Component ci determines a typed Field Hci , whose information is available wherever that Field is evaluated. At a query (x, t), the learned Operator combines the complete collection Hc and returns a velocity in Tx M. This retains the “propagated information” picture, but makes two distinctions precise: the components ci need not themselves (i) be points of M, and in the architectures studied here no single Hci is the generated tangent section. The root definition does not forbid the latter degenerate case. The closest connections are therefore to Operator learning, conditional function representations, geometric deep learning, and continuous-time generative models. Operator learning and conditional function representations. Neural Operators learn maps between function spaces rather than only maps between finite-dimensional vectors [9]. DeepONet separates an encoding of an input function from the coordinates at which the output function is queried [10], while Fourier Neural Operators parameterize global integral Operators efficiently in the Fourier domain [11]. TFMs share this function-to-function viewpoint and, in their point-evaluation form, the separation between a condition-dependent representation and an output coordinate. Their emphasis is different: the input is a product of potentially heterogeneous section families, the output belongs to the prescribed family V ⊆ Sec(pr∗M T M), and the component Fields are constructed once for repeated queries. The present formulation does not by itself imply the discretization invariance or resolution transfer often sought in Neural Operator architectures. Those properties depend on the chosen constructors and Field Operator. 16

Continuous-time generative Vector Fields. Neural ordinary differential equations and continuous normalizing flows established learnable dynamics defined by neural Vector Fields [12]. Flow Matching trains such Fields by regression against conditional probability paths [7]. Diffusion and score-based models provide closely related iterative dynamics in discrete, ODE, or SDE form [13, 14, 15]. On Manifolds, Riemannian score models and Riemannian Flow Matching ensure that the learned dynamics respect the state geometry [16, 8]. TFMs are complementary to these training paradigms: they specify how a structured condition parameterizes the Vector Field, not which probability path or stochastic process supplies the regression target. Consequently, the same factorization can be trained directly, distilled from an existing Field, or combined with rollout supervision. Tensor-valued and equivariant networks. Tensor Field Networks process scalar, vector, and higher-order features with rotation- and translation-equivariant operations on point clouds [17]. Despite the similar name, they are not the same construction as Tensor Field Models. Gauge-equivariant convolutional networks extend typed feature Fields to Manifolds while remaining independent of local frame choices [18], and geometric deep learning more broadly organizes architectures around symmetries and domain structure [19]. TFMs adopt the compatible principle that intermediate and output quantities should have explicit geometric types, but focus on factorizing a condition into a Field collection and then a Vector Field. Equivariance is guaranteed only when the component constructors, contractions, and Field Operator are chosen equivariantly, as in the S2 experiment. It is not automatic for every TFM parameterization. Reduced representations and inference acceleration. When explicit capacity restrictions are imposed, the use of the intermediate collection is related in spirit to reducedorder modeling for parametric dynamical systems [20]. It should not, however, be identified with a classical matrix or multilinear low-rank factorization [21, 22]: the component Fields may live in different Bundles, and their learned composition is generally nonlinear. Several complementary approaches reduce the cost of iterative generation. Fast ODE solvers reduce the number of Field evaluations [23]. Progressive distillation and consistency models learn few-step or one-step generators [24, 25]. Parallel sampling reduces sequential dependence [26]. Feature-caching methods instead exploit similarity between neighboring denoising evaluations [27, 28, 29, 30]. TFM reuse is orthogonal to these methods. It caches a condition-only Field collection exactly across sampler steps and independent paths, rather than approximately reusing state-dependent activations across time. It therefore does not reduce the number of causal solver steps, and its benefit emerges only when the construction cost is amortized over enough shared-condition queries. This distinction also allows TFMs to be combined with faster solvers, distillation, temporal caching, or path-level batching. As the cache audit in section 4 demonstrates, the same exact reuse is available to an ordinary predictor whenever its computation graph exposes a condition-only subgraph. The TFM formulation makes this separation explicit but does not monopolize it.

References [1] J.M. Lee. Manifolds and Differential Geometry. Graduate studies in mathematics. American Mathematical Society, 2009. isbn: 9780821848159. url: https://books. google.de/books?id=QqHdHy9WsEoC. [2] L.W. Tu. Differential Geometry: Connections, Curvature, and Characteristic Classes. Graduate Texts in Mathematics. Springer International Publishing, 2017. isbn: 9783319550848. url: https://books.google.de/books?id=bmsmDwAAQBAJ. [3] M. Berger and B. Gostiaux. Differential Geometry: Manifolds, Curves, and Surfaces. Graduate Texts in Mathematics. Springer New York, 1988. isbn: 9783540966265. url: https://books.google.de/books?id=CKWodhkU-Y0C. [4] J.M. Lee. Introduction to Smooth Manifolds. Graduate Texts in Mathematics. Springer New York, 2013. isbn: 9780387217529. url: https://books.google. de/books?id=w4bhBwAAQBAJ. [5] G. Gross and E. Meinrenken. Manifolds, Vector Fields, and Differential Forms: An Introduction to Differential Geometry. Springer Undergraduate Mathematics Series. Springer International Publishing, 2023. isbn: 9783031254093. url: https: //books.google.de/books?id=O6a7EAAAQBAJ. [6] M. Nakahara. Geometry, Topology and Physics, Second Edition. Graduate student series in physics. Taylor & Francis, 2003. isbn: 9780750306065. url: https : / / books.google.de/books?id=cH-XQB0Ex5wC. [7] Yaron Lipman et al. “Flow Matching for Generative Modeling”. In: International Conference on Learning Representations. 2023. url: https://arxiv.org/abs/ 2210.02747. [8] Ricky T. Q. Chen and Yaron Lipman. “Flow Matching on General Geometries”. In: International Conference on Learning Representations. 2024. url: https:// arxiv.org/abs/2302.03660. [9] Nikola Kovachki et al. “Neural Operator: Learning Maps Between Function Spaces with Applications to PDEs”. In: Journal of Machine Learning Research 24.89 (2023), pp. 1–97. url: https://www.jmlr.org/papers/v24/21-1524.html. [10] Lu Lu et al. “Learning Nonlinear Operators via DeepONet Based on the Universal Approximation Theorem of Operators”. In: Nature Machine Intelligence 3 (2021), pp. 218–229. doi: 10.1038/s42256-021-00302-5. [11] Zongyi Li et al. “Fourier Neural Operator for Parametric Partial Differential Equations”. In: International Conference on Learning Representations. 2021. url: https://arxiv.org/abs/2010.08895. [12] Ricky T. Q. Chen et al. “Neural Ordinary Differential Equations”. In: Advances in Neural Information Processing Systems. 2018. url: https://arxiv.org/abs/ 1806.07366. [13] Jonathan Ho, Ajay Jain, and Pieter Abbeel. “Denoising Diffusion Probabilistic Models”. In: Advances in Neural Information Processing Systems. 2020. url: https://arxiv.org/abs/2006.11239.

[14] Jiaming Song, Chenlin Meng, and Stefano Ermon. “Denoising Diffusion Implicit Models”. In: International Conference on Learning Representations. 2021. url: https://arxiv.org/abs/2010.02502. [15] Yang Song et al. “Score-Based Generative Modeling through Stochastic Differential Equations”. In: International Conference on Learning Representations. 2021. url: https://arxiv.org/abs/2011.13456. [16] Valentin De Bortoli et al. “Riemannian Score-Based Generative Modelling”. In: Advances in Neural Information Processing Systems. 2022. url: https://arxiv. org/abs/2202.02763. [17] Nathaniel Thomas et al. “Tensor Field Networks: Rotation- and TranslationEquivariant Neural Networks for 3D Point Clouds”. In: arXiv preprint arXiv:1802.08219 (2018). url: https://arxiv.org/abs/1802.08219. [18] Taco S. Cohen et al. “Gauge Equivariant Convolutional Networks and the Icosahedral CNN”. In: Proceedings of the 36th International Conference on Machine Learning. Vol. 97. Proceedings of Machine Learning Research. 2019, pp. 1321–1330. url: https://proceedings.mlr.press/v97/cohen19d.html. [19] Michael M. Bronstein et al. “Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges”. In: arXiv preprint arXiv:2104.13478 (2021). url: https: //arxiv.org/abs/2104.13478. [20] Peter Benner, Serkan Gugercin, and Karen Willcox. “A Survey of Projection-Based Model Reduction Methods for Parametric Dynamical Systems”. In: SIAM Review 57.4 (2015), pp. 483–531. doi: 10.1137/130932715. [21] Carl Eckart and Gale Young. “The Approximation of One Matrix by Another of Lower Rank”. In: Psychometrika 1 (1936), pp. 211–218. [22] Vin de Silva and Lek-Heng Lim. “Tensor Rank and the Ill-Posedness of the Best Low-Rank Approximation Problem”. In: SIAM Journal on Matrix Analysis and Applications 30.3 (2008), pp. 1084–1127. doi: 10.1137/06066518X. [23] Cheng Lu et al. “DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps”. In: Advances in Neural Information Processing Systems. 2022. url: https://arxiv.org/abs/2206.00927. [24] Tim Salimans and Jonathan Ho. “Progressive Distillation for Fast Sampling of Diffusion Models”. In: International Conference on Learning Representations. 2022. url: https://arxiv.org/abs/2202.00512. [25] Yang Song et al. “Consistency Models”. In: International Conference on Machine Learning. 2023. url: https://arxiv.org/abs/2303.01469. [26] Andy Shih et al. “Parallel Sampling of Diffusion Models”. In: Advances in Neural Information Processing Systems. 2023. url: https : / / arxiv . org / abs / 2305 . 16317. [27] Xinyin Ma, Gongfan Fang, and Xinchao Wang. “DeepCache: Accelerating Diffusion Models for Free”. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024. url: https://arxiv.org/abs/2312.00858. [28] Xinyin Ma et al. “Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching”. In: Advances in Neural Information Processing Systems. 2024. url: https://arxiv.org/abs/2406.01733. 19

[29] Feng Liu, Shiwei Zhang, Xiaofeng Wang, et al. “Timestep Embedding Tells: It’s Time to Cache for Video Diffusion Model”. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2025. url: https://arxiv.org/abs/2411.19108. [30] Huanlin Gao, Ping Chen, Fuyuan Shi, et al. “MeanCache: From Instantaneous to Average Velocity for Accelerating Flow Matching Inference”. In: International Conference on Learning Representations. 2026. url: https://arxiv.org/abs/ 2601.19961. [31] Jordan François and Lucrezia Ravera. “Toward manifest relationality in transformers via symmetry reduction”. In: APS Open Science 1 (2026), p. 000034.

Tensor Field Models

Analyze & explore related research in Conceptio

Related documents