FUSE: Feature-Wise Unified Specialization with Cross-Column Exchange for Mixed-Type Tabular Flow Matching

FUSE: Feature-Wise Unified Specialization with Cross-Column Exchange for Mixed-Type Tabular Flow Matching Suman Cha1 , Seongchan Lee2 , Dohyun Ko1 , Hyunjoong Kim1∗ 1

Department of Statistics and Data Science, Yonsei University 2 Department of Mathematical Sciences, KAIST

arXiv:2608.07294v1 [cs.LG] 7 Aug 2026

Abstract Generating mixed-type tabular data requires jointly modeling diverse feature distributions and their complex cross-column dependencies. Variational flow matching handles distinct endpoints via factorized distributions, yet leaves feature-specific processing and cross-column interactions implicit within a shared backbone. We introduce Feature-wise Unified Specialization with cross-column Exchange (FUSE) to explicitly separate these roles. FUSE applies separate adaptive mixture modules to numerical and categorical features, allowing each feature to combine shared specialized subnetworks, while joint attention preserves information exchange across all columns. We also characterize the excess population risk from restricted conditioning contexts and bound the continuous Wasserstein generation error by endpoint-prediction risk. Comprehensive experiments on eight tabular datasets demonstrate that FUSE achieves strong and consistent performance across distributional fidelity and downstream utility metrics.

1

Introduction

Tabular data is one of the most prevalent data formats in machine learning applications such as healthcare, finance, and public policy (Gardner, Popovic, and Schmidt 2023; Holzmüller, Grinsztajn, and Steinwart 2024; Little, Allmendinger, and Elliot 2025). The ability to generate synthetic tabular data has become essential for data sharing, mitigating data scarcity, and enabling robust downstream analysis (Zhao et al. 2021; Qian, Davis, and van der Schaar 2023; van Breugel et al. 2023). However, generating synthetic samples remains challenging since it requires capturing the complex joint distributions of mixed-type variables. The challenge extends beyond the distinction between numerical and categorical variables. Columns within the same type can also exhibit different statistical structures (Shi et al. 2025). Numerical columns may vary in skewness, tail behavior, and multimodality, while categorical columns differ in cardinality and frequency concentration. Generative models therefore need to capture these column-specific distributions while preserving dependencies between the features. Recent work has adapted flow matching to this setting (Jolicoeur-Martineau, Fatras, and Kachman 2024; GuzmánCordero, Eijkelboom, and Van De Meent 2025). Flow matching learns a time-dependent velocity field whose induced ∗

Corresponding author.

ordinary differential equation transports a tractable source distribution to the data distribution (Liu, Gong, and Liu 2023; Lipman et al. 2023; Albergo and Vanden-Eijnden 2023). Variational Flow Matching (VFM) expresses the velocity through conditional inference over trajectory endpoints, while Mean-Field Variational Flow Matching (MFVFM) makes the endpoint inference tractable by factorizing the variational endpoint distribution across variables (Eijkelboom et al. 2024). For mixed-type data, Exponential Family Variational Flow Matching (EF-VFM) extends this approach by assigning appropriate exponential family distributions to match the statistical properties of each variable type (Guzmán-Cordero, Eijkelboom, and Van De Meent 2025). This factorization decomposes the training objective across columns without restricting the information available to their predictors. Each endpoint factor may condition on all coordinates of the intermediate state. While endpoint factorization aligns the objective with varying data types, the architectural challenge of processing diverse variables remains unresolved. TabbyFlow implements EF-VFM by processing column embeddings through a shared backbone (Guzmán-Cordero, Eijkelboom, and Van De Meent 2025). Although this design supports parameter sharing and encodes feature identity through column embeddings, it provides no explicit mechanism for featuredependent transformations. Motivated by this gap, we propose FUSE, a Feature-wise Unified Specialization with cross-column Exchange architecture for mixed-type tabular flow matching. FUSE applies feature-dependent component recombination with joint selfattention, enabling specialized computation while preserving cross-type information exchange. Within each variable type, adaptive mixture processing forms feature-dependent combinations of shared specialized subnetworks using a differentiable aggregate-transform-recombine operator (Puigcerver et al. 2024a). Joint attention subsequently connects all column representations, preserving information exchange between numerical and categorical variables. FUSE therefore separates feature-specific processing from the conditioning information available to each endpoint predictor. The contributions of this work are summarized as follows. 1. We introduce a mixed-type flow-matching architecture that combines type-specific adaptive mixture processing with joint attention across all columns.

2. We quantify the exact excess population risk induced by restricting the conditioning context. Under regularity conditions, we further bound the continuous-space Wasserstein generation error in terms of endpoint-prediction risk. 3. Comprehensive experiments on eight tabular datasets show that FUSE achieves competitive or superior generation quality across diverse metrics.

2

qθ,t (x1 | x) =

Flow Matching for Mixed-Type Data

We review the flow matching formulation for mixed-type data. Flow matching defines a continuous transport through a time-dependent velocity field, while variational flow matching represents this velocity through conditional endpoint inference. Mean-field and exponential-family extensions yield tractable objectives for numerical and categorical endpoints.

2.1

Flow Matching

Let X1num ∈ Rdnum denote the numerical endpoint and let (k) C1 ∈ {1, . . . , Kk } denote categorical endpoint k for k = 1, . . . , dcat . Using the one-hot map ek : {1, . . . , Kk } → {0, 1}Kk , we write (1) (d )
X1 = X1num , e1 (C1 ), . . . , edcat (C1 cat ) ∈ RD , Pdcat where D = dnum + k=1 Kk . Let X0 ∼ p0 = N (0, ID ) be independent of X1 . The linear conditional path is Xt = (1 − t)X0 + tX1 ,

t ∈ [0, 1).

(1)

Flow matching estimates the marginal velocity of this path and generates samples by integrating the learned ordinary differential equation (Lipman et al. 2023; Albergo, Boffi, and Vanden-Eijnden 2025). For a fixed endpoint x1 , the conditional velocity is ut (x | x1 ) = (x1 −x)/(1−t). The marginal velocity is its conditional expectation given Xt = x ut (x) = E [ut (x | X1 ) | Xt = x] mt (x) − x = , mt (x) = E[X1 | Xt = x]. 1−t Thus, the endpoint posterior determines the marginal velocity only through its mean mt (x).

2.2

Family Variational Flow Matching (Guzmán-Cordero, Eijkelboom, and Van De Meent 2025) extends this construction to mixed-type data by assigning an appropriate exponentialfamily distribution to each endpoint column. Combining these two ingredients, we use Gaussian factors for numerical endpoints and categorical factors for categorical endpoints

Variational Endpoint Inference

Let p1|t (x1 | x) denote the endpoint posterior at time t. Variational Flow Matching (Eijkelboom et al. 2024) approximates this posterior with qθ,t (x1 | x) by minimizing LVFM (θ) = −E [log qθ,t (X1 | Xt )] , where the expectation is taken over t ∼ Unif(0, 1), X0 ∼ p0 , and X1 ∼ p1 . The expectation of the variational endpoint distribution is m b θ,t (x) = Eqθ,t (·|x) [X1 ], which induces the vector field m b θ,t (x) − x vθ,t (x) = . 1−t Mean-Field Variational Flow Matching imposes a product form on the variational endpoint distribution. Exponential

dY num

N xnum 1,j | µθ,j,t (x), νt

j=1

×

d cat Y

(2) (k)

Cat c1 | πθ,k,t (x) .

k=1

where νt > 0 is a predetermined variance. Under this distribution, the conditional expectation of each numerical endpoint is µθ,j,t (x). For a one-hot categorical block, its conditional expectation is the corresponding probability vector πθ,k,t (x). Up to additive terms independent of θ, the negative log-likelihood is then " 1 2 Lend (θ) = E ∥X1num − µθ,t (Xt )∥2 2νt # (3) dcat X − log πθ,k,t (Xt )C (k) . 1

k=1

The mean-field assumption factorizes qθ,t (· | x) across endpoint columns, but places no restriction on the conditioning argument x. Consequently, each column-wise parameter, including µθ,j,t (x) and πθ,k,t (x), may depend on all coordinates of x. The variational objective does not specify how computation should be shared effectively among these column-wise parameter functions.

3

Proposed Architecture

While the VFM-based approaches in Section 2 specify the factorized endpoint distributions, their effective neural parameterization remains open. For mixed-type tabular data, a fully shared backbone lacks feature-specific computation, whereas independent networks sacrifice beneficial parameter sharing. To resolve this tradeoff, we propose FUSE, an architecture interleaving type-specific adaptive mixture processing with joint attention, enabling feature specialization while maintaining cross-column dependencies.

3.1

Adaptive Mixture Processing

Let Xt = (Xtnum , Xtcat ) denote the intermediate state at time (0) t. We project each feature into an initial token hr,j,t ∈ Rdh for type r ∈ {num, cat} via feature-specific linear layers, learnable embeddings, and a time embedding ψ(t). (0) This yields embedding matrices Hnum,t ∈ Rdnum ×dh and (0)

Hcat,t ∈ Rdcat ×dh . At layer ℓ, the adaptive mixture module for type r employs Mr parallel subnetworks to process Sr = Mr Pr latent com(ℓ) ponents. Using the projection Φr ∈ Rdh ×Sr and tokens

Figure 1: Overview of the FUSE architecture. Type-specific adaptive mixture processing transforms numerical and categorical tokens into specialized representations, which then interact via global attention to capture multivariate feature dependencies. (ℓ)

(ℓ)

3.2

H r,t = LN(Hr,t ), we compute the alignment scores as (ℓ) (ℓ) H r,t Φr (ℓ) Rr,t = √ ∈ Rnr ×Sr . dh

(4)

The score matrix is then normalized along two axes. The (ℓ) (ℓ) aggregation weights Dr,t = softmax(Rr,t ) are normalized across feature tokens for each component and determine how the feature representations are aggregated into latent com(ℓ) (ℓ) ponents. The recombination weights Cr,t = softmax(Rr,t ) are normalized across components for each feature and determine how the processed components are combined to update each feature representation. These two normalizations control the aggregation and recombination stages of the module. The aggregation weights first form the latent components (ℓ)

The Sr components are partitioned among the Mr subnetworks, with Pr components assigned to each subnetwork. Let κr (s) = ⌈s/Pr ⌉ denote the subnetwork assigned to component s. Each component is then transformed as   (ℓ) (ℓ) (ℓ) Vr,s,t = gr,κr (s) Ur,s,t , s = 1, . . . , Sr , (ℓ)

and the resulting representations are stacked into Vr,t ∈ RSr ×dh . The recombination weights distribute these transformed components back to the feature tokens, producing the residual update (ℓ) (ℓ) (ℓ) (ℓ) e r,t H = Hr,t + Cr,t Vr,t .

This operation allows each feature token to incorporate information from all numerical and categorical features. The re(ℓ+1) (ℓ+1) sulting sequence is then partitioned into Hnum,t and Hcat,t before entering the next layer. Each layer therefore combines type-specific adaptive processing with joint interaction.

3.3

(ℓ)

Ur,t = (Dr(ℓ) )⊤ H r,t .

(5)

Since both weights depend on the feature representations, each feature can receive a distinct combination of subnetwork transformations. The subnetworks are also shared across all features of type r, allowing type-specific specialization without introducing a separate subnetwork for every feature.

Joint Attention

The adaptive mixture module processes numerical and categorical tokens through separate sets of subnetworks. This separation supports type-specific transformations, but does not allow by itself information exchange across feature types. To enable joint contextualization, we concatenate the type(ℓ) (ℓ) e cat,t e num,t e t(ℓ) = Concat(H ) and apply ,H specific outputs H multi-head self-attention (Vaswani et al. 2017):

(ℓ+1) e t(ℓ) . e t(ℓ) + MHA(ℓ) LN H (6) Ht =H

Endpoint Prediction

After L blocks, feature-specific output heads map the final numerical and categorical tokens to the parameters of their corresponding endpoint factors ⊤

(L)

µθ,j,t (Xt ) = (wµ,j ) hnum,j,t + bµ,j ,   (L) πθ,k,t (Xt ) = softmax Wπ,k hcat,k,t + bπ,k . These outputs parameterize the factorized endpoint distribution in Equation (2) and are optimized using the endpoint objective in Equation (3). Each head operates on a feature-specific representation containing both adaptive transformations and joint context. Concatenating the predicted numerical means and categorical probabilities yields m b θ,t (x), which defines the vector field vθ,t (x) = (m b θ,t (x) − x)/(1 − t).

4

Theoretical Analysis

The mean-field assumption determines only how the endpoint distribution factorizes, leaving unspecified both the

information available to each endpoint function and the representations shared among them. We first use an illustrative construction to examine the approximation error induced by restricting distinct endpoint functions to a single shared representation. We then show that endpoint excess risk controls the Wasserstein error of the generated distribution. All proofs and derivations are provided in the Appendix. Conditioning Penalty. Fix T ∈ (0, 1). Let t be independent of (X0 , X1 ) with density ρ on [0, T ], and let F = σ(t, Xt ) denote the full conditioning σ-algebra, where Xt follows the conditional path in Equation (1). Let G be a restricted conditioning σ-algebra satisfying σ(t) ⊆ G ⊆ F . For any σ-algebra A satisfying σ(t) ⊆ A ⊆ F , define the numerical posterior mean mA = E[X1num | A] and the categor(k) ical posterior probability vector pk,A for C1 . Let R∗T (A) denote the minimum endpoint risk over all A-measurable numerical predictors and categorical probability vectors. Proposition 1. Suppose E∥X1num ∥22 < ∞ with 0 < νt < ∞ almost surely. Then   ∥mF − mG ∥22 R∗T (G) − R∗T (F) = E 2νt dcat X 
 + E KL pk,F ∥ pk,G ≥ 0. k=1

Equality holds if and only if mF = mG and pk,F = pk,G for every k, almost surely. The decomposition vanishes when the restricted context preserves both the numerical posterior mean and every categorical posterior probability. A penalty arises only when the omitted features contain residual endpoint information after conditioning on the retained features.

where a, b ̸= 0. Suppose both endpoint functions are restricted to a single shared scalar representation Hshared = {(θ2 h, θ3 h) : ∥h∥L2 = 1, θ2 , θ3 ∈ R} . The orthogonality of W with V implies i h inf E {m(2) − f2 }2 + {m(3) − f3 }2 (f2 ,f3 )∈Hshared

= min{a2 , b2 } > 0. By contrast, the shared component dictionary represents both endpoint means exactly when the features utilize divergent recombination weights, demonstrated in Appendix Section A. This construction shows that feature-dependent recombination can remove the approximation error induced by the specified single-representation restriction while retaining a shared set of transformations. The two examples concern distinct architectural restrictions. Joint attention allows cross-type information, whereas adaptive mixture processing allows different features to form distinct combinations of shared transformations. Their relevance depends respectively on residual cross-type endpoint information and the adequacy of a shared representation. Distributional Error Bound. Let f ∗ be the unrestricted Bayes predictor and define the excess risk ET (f ) = RT (f )− RT (f ∗ ). The following theorem bounds the Wasserstein error of the generated distribution in terms of ET (f ). Theorem 1. Fix T ∈ (0, 1) and let t have density ρ on [0, T ]. Assume 0 < ρ < ρ(t), 0 < ν ≤ νt ≤ ν < ∞, and E∥X1num ∥22 < ∞. Suppose vθ,t is jointly measurable in (t, x), uniformly L-Lipschitz in x over t, and satisfies Z T ∥vθ,t (0)∥2 dt < ∞. 0

Architectural Bottlenecks. Let S be a Rademacher variable that satisfies P(S = 1) = P(S = −1) = 1/2. Let i.i.d.

ξ ∼ N (0, σ 2 ) and W, V ∼ N (0, 1) be mutually independent and independent of S. Define U = δS + ξ for δ ̸= 0, and suppose νt ≡ ν > 0. (i) Conditioning bottleneck. Suppose a numerical endpoint has conditional mean m(1) (U, S) = αU + cS for c ̸= 0. If its predictor is conditioned on U but not on S, Proposition 1 implies the excess numerical endpoint risk    c2 δU c2 E[Var(S | U )] = E sech2 > 0. 2ν 2ν σ2 The equality follows from E[S | U ] = tanh(δU/σ 2 ). The positive penalty shows that an informative numerical proxy for S need not be sufficient for the endpoint mean when σ 2 > 0. Joint attention allows the categorical representation to enter the numerical feature update and therefore avoids imposing the restricted conditioning used in this example. (ii) Representation bottleneck. Now consider two additional endpoint means m(2) (W, V ) = aW,

m(3) (W, V ) = bV,

Then its time-T distribution pbθT satisfies p Kρ,ν ΓL (T ) p θ ET (θ) W2 (b pT , p1 ) ≤ 1−T + (1 − T ) E∥X0 − X1 ∥22

1/2

,

where Kρ,ν = 2 max{ν, 1}/ρ and ΓL (T ) = (e2LT − 1)/(2L), with Γ0 (T ) = T . The first term converts endpoint excess risk into a Wasserstein error bound, while the second captures truncation at T < 1. Hence, larger T reduces truncation error but increases the risk-dependent coefficient.

5

Experiments

We evaluate the proposed architecture by comparing it against various baseline models across multiple datasets, assessing both distributional fidelity and downstream machine learning utility. We complement this evaluation with qualitative analyses that visually assess the preservation of marginal distributions and pairwise feature correlations. Finally, we conduct ablation studies to examine the respective contributions of joint attention and adaptive mixture processing.

Shape (↑) Method

Adult

Default

Beijing

Shoppers

Magic

News

Diabetes

CTGAN TVAE CoDi TabDDPM TabSyn TabbyFlow TabDiff FUSE

0.821(.001) 0.852(.001) 0.777(.001) 0.990(.001) 0.979(.001) 0.993(.001) 0.992(.000) 0.993(.001)

0.865(.000) 0.897(.001) 0.784(.001) 0.987(.001) 0.954(.001) 0.989(.001) 0.987(.001) 0.992(.001)

0.802(.001) 0.771(.001) 0.801(.001) 0.977(.001) 0.967(.001) 0.991(.001) 0.989(.000) 0.992(.001)

0.777(.001) 0.759(.001) 0.702(.001) 0.971(.001) 0.983(.001) 0.986(.001) 0.987(.002) 0.989(.001)

0.903(.001) 0.921(.002) 0.903(.003) 0.992(.001) 0.989(.001) 0.991(.001) 0.992(.001) 0.992(.001)

0.808(.000) 0.830(.001) 0.687(.001) 0.187(.000) 0.986(.000) 0.975(.000) 0.975(.000) 0.988(.001)

0.695(.005) 0.812(.006) 0.800(.014) 0.930(.088) 0.947(.006) 0.952(.004) 0.952(.005) 0.957(.004)

Trend (↑) Method

Adult

Default

Beijing

Shoppers

Magic

News

Diabetes

CTGAN TVAE CoDi TabDDPM TabSyn TabbyFlow TabDiff FUSE

0.832(.016) 0.857(.012) 0.810(.001) 0.981(.001) 0.954(.001) 0.987(.001) 0.984(.001) 0.988(.001)

0.839(.001) 0.931(.001) 0.640(.088) 0.948(.004) 0.920(.014) 0.988(.001) 0.986(.001) 0.989(.001)

0.889(.003) 0.841(.003) 0.916(.006) 0.977(.001) 0.955(.003) 0.972(.004) 0.967(.003) 0.966(.006)

0.845(.002) 0.900(.002) 0.914(.001) 0.956(.004) 0.982(.002) 0.983(.002) 0.976(.002) 0.988(.002)

0.888(.005) 0.930(.010) 0.941(.004) 0.987(.004) 0.992(.002) 0.993(.001) 0.991(.001) 0.989(.005)

0.884(.001) 0.932(.002) 0.863(.002) 0.602(.008) 0.983(.002) 0.984(.002) 0.941(.002) 0.983(.003)

0.650(.053) 0.695(.027) 0.822(.039) 0.897(.147) 0.928(.010) 0.936(.009) 0.929(.008) 0.966(.005)

C2ST (↑) Method

Adult

Default

Beijing

Shoppers

Magic

News

Diabetes

CTGAN TVAE CoDi TabDDPM TabSyn TabbyFlow TabDiff FUSE

0.568(.004) 0.553(.003) 0.411(.003) 0.970(.004) 0.912(.006) 0.992(.005) 0.981(.005) 0.991(.005)

0.723(.003) 0.561(.003) 0.268(.002) 0.978(.005) 0.847(.004) 0.987(.005) 0.964(.003) 0.989(.010)

0.803(.003) 0.659(.011) 0.632(.002) 0.970(.003) 0.883(.003) 0.991(.005) 0.972(.004) 0.996(.004)

0.644(.009) 0.301(.011) 0.410(.003) 0.886(.006) 0.962(.009) 0.971(.009) 0.982(.008) 0.979(.011)

0.578(.006) 0.729(.004) 0.737(.004) 0.993(.006) 0.991(.007) 0.998(.004) 0.995(.004) 0.997(.004)

0.588(.009) 0.466(.005) 0.081(.001) 0.000(.000) 0.970(.004) 0.922(.004) 0.895(.004) 0.973(.008)

0.143(.007) 0.439(.018) 0.559(.031) 0.952(.179) 0.980(.023) 0.996(.008) 0.983(.022) 0.993(.010)

Adult AUC (↑)

Default AUC (↑)

Beijing RMSE (↓)

Shoppers AUC (↑)

Magic AUC (↑)

News RMSE (↓)

Diabetes AUC (↑)

0.889(.002) 0.878(.002) 0.744(.018) 0.907(.001) 0.908(.001) 0.910(.003) 0.912(.002) 0.909(.001)

0.744(.007) 0.747(.005) 0.483(.024) 0.770(.002) 0.757(.009) 0.764(.007) 0.768(.007) 0.769(.004)

0.865(.017) 0.873(.019) 0.770(.029) 0.608(.018) 0.703(.020) 0.541(.013) 0.573(.013) 0.610(.004)

0.851(.009) 0.872(.006) 0.819(.015) 0.916(.005) 0.909(.005) 0.917(.006) 0.919(.004) 0.921(.003)

0.871(.003) 0.890(.003) 0.926(.002) 0.931(.003) 0.932(.003) 0.931(.003) 0.930(.002) 0.930(.003)

0.894(.013) 0.995(.015) 1.771(.161) 2.510(.890) 0.883(.019) 0.872(.009) 0.888(.026) 0.841(.003)

0.489(.105) 0.808(.026) 0.820(.024) 0.816(.070) 0.823(.034) 0.810(.030) 0.818(.037) 0.810(.023)

MLE Method CTGAN TVAE CoDi TabDDPM TabSyn TabbyFlow TabDiff FUSE

Fault Avg. rank (↓) 0.732(.002) 0.857(.002) 0.777(.040) 0.302(.208) 0.959(.003) 0.970(.002) 0.938(.004) 0.972(.002)

6.88 6.12 7.25 5.12 4.12 2.50 2.75 1.25

Fault Avg. rank (↓) 0.784(.004) 0.892(.004) 0.920(.009) 0.420(.213) 0.969(.004) 0.978(.002) 0.818(.011) 0.985(.001)

7.25 6.25 6.38 5.00 3.75 1.75 3.62 2.00

Fault Avg. rank (↓) 0.175(.010) 0.642(.015) 0.471(.107) 0.008(.009) 0.851(.058) 0.996(.007) 0.635(.027) 0.994(.007)

6.50 6.62 7.00 5.12 4.12 1.75 3.25 1.62

Fault Avg. rank (↓) AUC (↑) 0.772(.034) 0.880(.008) 0.948(.010) 0.511(.030) 0.940(.012) 0.960(.007) 0.929(.006) 0.962(.006)

6.88 6.62 6.00 4.50 3.50 2.75 3.12 2.62

Table 1: Performance comparison across eight datasets. Results are reported as means with standard deviations in parentheses. Bold and underlined values indicate the best and second-best results, respectively, and the proposed method is shaded.

5.1

Experimental Setup

Datasets. We evaluate FUSE on eight tabular datasets: Adult, Default, Beijing, Shoppers, Magic, News, Diabetes, and Fault. Each dataset contains both numerical and categorical features and is associated with either binary classification, multiclass classification, or regression. Dataset sizes range from 691 to 37,581 samples. Detailed dataset profiles are provided in the Appendix. Baselines. We compare FUSE against seven competitive synthetic tabular data generation methods spanning four model families: 1) GAN-based method: CTGAN (Xu et al. 2019); 2) VAE-based method: TVAE (Xu et al. 2019); 3)

Diffusion-based methods: CoDi (Lee, Kim, and Park 2023), TabDDPM (Kotelnikov et al. 2023), TabSyn (Zhang et al. 2024), and TabDiff (Shi et al. 2025); and 4) Flow-based methods: TabbyFlow (Guzmán-Cordero, Eijkelboom, and Van De Meent 2025). Metrics. We evaluate generative fidelity and downstream utility using diverse metrics. Shape quantifies marginal distributional similarity using Kolmogorov-Smirnov statistics for numerical features and total variation distances for categorical features. Trend measures the preservation of pairwise feature dependencies by comparing the correlation structures between real and synthetic data. We also report a Classifier Two-sample Test (C2ST) score, for which a higher value in-

1.6 0.8 0.0

−0.8

0.0

2

2

2

1

1

1

0

0

News: polarity

−0.8

0.0

News: polarity

−0.8

0.0

0

News: polarity

0.4

0.4

0.4

0.4

0.2

0.2

0.2

0.2

0.0

0.0

0.0

0

20

Shoppers: pages 0.6

0.3

0.3

0.2

0.0

0.0

0

3

5

8

Adult: workclass

CoDi

0

20

Shoppers: pages

0

20

NE

TabbyFlow

0

20

Shoppers: pages 0.1

0.2

Beijing: wind

0.0

0.2

0.5 SE

0.0

Shoppers: pages

0.0

−0.8

News: polarity

g

h

0.0

Magic: class

TabDiff

FUSE

0

2

4

6

News: weekday

Real

Figure 3: Real and synthetic marginal distributions for numerical and categorical features.

Figure 2: Absolute differences between pairwise correlations from real and synthetic data. Rows correspond to datasets and columns to methods. Values closer to zero indicate more faithful preservation of feature correlation.

dicates that a discriminator is less able to distinguish real from synthetic samples. To assess downstream utility, Machine Learning Efficiency (MLE) trains a predictive model on synthetic data and evaluates its performance on held-out real data. This section reports Shape, Trend, C2ST, and MLE, while results for α-Precision and β-Recall are provided in the Appendix. Implementation Details. Each main result is averaged over 20 random initialization seeds. For C2ST, we use an XGBoost classifier to discriminate real from synthetic samples. Complete implementation details, including preprocessing procedures and hyperparameter settings, are provided in the Appendix.

5.2

Main Results

Table 1 compares FUSE with seven tabular data generation baselines in terms of fidelity, pairwise dependence, and downstream utility. Shape and Trend. FUSE demonstrates the most consistent marginal fidelity among the evaluated methods. It ranks first on six datasets and second on the remaining two in Shape, resulting in an average rank of 1.25. The next-best method, TabbyFlow, obtains an average rank of 2.50. The improvement is therefore observed across all eight datasets rather than being driven by a small subset of favorable cases. For Trend, FUSE achieves the highest score on five datasets and the highest cross-dataset mean of 0.982, compared with 0.978 for TabbyFlow. The largest improvements over TabbyFlow occur on Diabetes and Fault, where Trend increases from 0.936 to 0.966 and from 0.978 to 0.985, respectively. TabbyFlow nevertheless obtains a slightly better average rank of 1.75, compared with 2.00 for FUSE, re-

flecting its stronger relative rankings on Beijing, Magic, and News. These results show that FUSE provides a clear improvement in marginal fidelity while remaining highly competitive in preserving pairwise feature dependencies. C2ST. FUSE achieves the best C2ST average rank of 1.62, followed by TabbyFlow at 1.75. It ranks first on Default, Beijing, and News and second on each of the remaining five datasets, making it the only method that places among the top two across all eight datasets. Since the C2ST classifier processes all features jointly, this consistency cannot be attributed solely to accurate univariate marginals. Rather, it indicates that FUSE leaves fewer multivariate discrepancies detectable by the selected XGBoost classifier. Machine Learning Efficiency. FUSE also achieves the best average MLE rank of 2.62, outperforming TabbyFlow at 2.75 and TabDiff at 3.12. It obtains the best downstream performance on Shoppers, News, and Fault and the second-best result on Default. The improvement is particularly achieved on News, where FUSE reduces RMSE from 0.872 for TabbyFlow, the strongest competing method on this dataset, to 0.841. This corresponds to a relative reduction of approximately 3.6%. The gains are not uniform across all tasks, as TabbyFlow achieves a lower RMSE on Beijing and TabSyn obtains a higher AUC on Diabetes. Nevertheless, the best average rank shows that the improvements in distributional fidelity are accompanied by competitive downstream utility. Overall, FUSE achieves average ranks of 1.25, 2.00, 1.62, and 2.62 for Shape, Trend, C2ST, and MLE, respectively. The results show a favorable balance across marginal, pairwise, multivariate, and task-oriented evaluations, without relying on uniformly dominant performance under any single metric.

5.3

Synthetic Data Visualization

Figure 3 compares marginal distributions generated by CoDi, TabbyFlow, TabDiff, and FUSE, while Figure 2 visualizes errors in pairwise dependence preservation. For numerical features, kernel density estimates show that FUSE recovers both the global shape and local structure of the empirical distributions. Specifically, FUSE captures

Architecture Variant Dense FFN Restricted attention Shared processing Numerical mixture Categorical mixture FUSE

Numerical mixture

Categorical mixture

– ✓ – ✓ – ✓

– ✓ – – ✓ ✓

Fidelity Joint attention

Shape (↑)

Trend (↑)

C2ST (↑)

– – ✓ ✓ ✓ ✓

0.984 0.984 0.984 0.984 0.983 0.985

0.960 0.960 0.976 0.973 0.970 0.977

0.980 0.990 0.985 0.983 0.973 0.991

MLE α-Prec. (↑)

β-Rec. (↑)

AUC (↑)

RMSE (↓)

0.970 0.972 0.985 0.985 0.982 0.986

0.344 0.371 0.482 0.548 0.507 0.545

0.630 0.644 0.888 0.884 0.882 0.882

0.820 0.818 0.742 0.734 0.725 0.725

Table 2: Component analysis averaged across eight datasets. Check marks indicate active components. MLE reports classification AUC and regression RMSE. Bold and underlined values denote the best and second-best results based on unrounded means. the sequence of local modes in the polarity variable on News and Shoppers. For categorical variables, FUSE closely matches the empirical category frequencies across Adult, Beijing, Magic, and News. It preserves both dominant and low-frequency classes in Adult and accurately reproduces the target balance in Magic. FUSE also generates a tighter representation of less frequent weekday categories. These results demonstrate that FUSE effectively preserves both dominant categorical structures and minority probability masses. Figure 2 reports the absolute difference between the pairwise correlation matrices of the real and synthetic data. The block-structured matrices reveal how well each method preserves cross-type dependencies. FUSE achieves consistently low correlation errors across all blocks, with a pronounced advantage on News and Fault. Specifically on News, FUSE yields near-zero errors, whereas CoDi exhibits widespread discrepancies and TabDiff retains localized high errors. Ultimately, these visualizations confirm that FUSE successfully captures both the intra-type correlations and the complex cross-type dependencies inherent to mixed-type tabular data.

5.4

of withholding cross-feature information. The contribution of adaptive mixture processing is evaluated under joint attention. Compared with shared processing, FUSE improves Shape from 0.984 to 0.985, Trend from 0.976 to 0.977, C2ST from 0.985 to 0.991, and β-Recall from 0.482 to 0.545. It also reduces RMSE from 0.742 to 0.725, although AUC decreases slightly from 0.888 to 0.882. Consequently, FUSE achieves the best result on four of the five fidelity metrics and the second-best β-Recall. Applying the mixture module to a single feature type produces metricspecific trade-offs. The numerical-only configuration attains the highest β-Recall and second-highest AUC, whereas the categorical-only configuration matches the RMSE of FUSE but performs less favorably on the fidelity metrics. Overall, the results support complementary roles for the two components. Joint attention enables effective cross-type conditioning, while adaptive mixture processing further improves fidelity and coverage through type-specific computation. Their combination provides the strongest aggregate fidelity profile, although no configuration uniformly dominates every metric.

Component Analysis

Table 2 evaluates six configurations to separate the contributions of adaptive mixture processing and joint attention. All configurations use identical training objectives, data splits, and five matched random seeds. Dense processing replaces the adaptive mixture modules with feed-forward networks (FFN), while restricted attention limits interactions to features of the same type. The shared-processing configuration combines dense networks with joint attention and serves as the reference for evaluating adaptive mixture processing under unrestricted conditioning. The effect of joint attention is examined through two comparisons. Under dense processing, enabling joint attention preserves Shape at 0.984 while increasing Trend from 0.960 to 0.976 and AUC from 0.630 to 0.888. It also reduces RMSE from 0.820 to 0.742. A similar pattern appears when both adaptive mixture modules are active. Relative to restricted attention, FUSE improves Trend from 0.960 to 0.977, AUC from 0.644 to 0.882, and RMSE from 0.818 to 0.725, while maintaining comparable marginal fidelity. This result shows that cross-type information exchange is particularly important for dependency preservation and downstream utility. It is consistent with Proposition 1, which characterizes the cost

6

Conclusion

In this work, we have introduced FUSE, a network architecture for mixed-type variational flow matching that makes the parameter sharing configuration between mixed-type endpoint factors explicit. FUSE combines type-specific adaptive mixture processing with joint attention. The adaptive mixture modules form feature-dependent combinations of shared specialized subnetworks, while joint attention preserves crosstype information exchange and unrestricted conditioning for every endpoint predictor. Our theoretical analysis characterizes the complementary roles of these components. The conditioning information bound quantifies the excess population risk caused by restricted conditioning contexts, while the architectural bottleneck analysis distinguishes information loss from limitations in shared computation. The distributional error bound further relates endpoint-prediction risk to Wasserstein generation error under the stated assumptions. Comprehensive experiments show that FUSE achieves strong aggregate performance across the evaluated metrics. The component analysis provides evidence for joint attention and shows that adaptive mixture processing improves several fidelity and utility measures.

References Alaa, A.; Van Breugel, B.; Saveliev, E. S.; and van der Schaar, M. 2022. How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models. In Proceedings of the 39th International Conference on Machine Learning, 290–306. Albergo, M.; Boffi, N. M.; and Vanden-Eijnden, E. 2025. Stochastic Interpolants: A Unifying Framework for Flows and Diffusions. Journal of Machine Learning Research, 26(209): 1–80. Albergo, M. S.; and Vanden-Eijnden, E. 2023. Building Normalizing Flows with Stochastic Interpolants. In The Eleventh International Conference on Learning Representations. Ambrosio, L.; Gigli, N.; and Savaré, G. 2008. Gradient Flows: In Metric Spaces and in the Space of Probability Measures. Lectures in Mathematics ETH Zürich. Birkhäuser Basel, 2 edition. ISBN 978-3-7643-8722-8. Bao, H.; Wang, W.; Dong, L.; Liu, Q.; Mohammed, O. K.; Aggarwal, K.; Som, S.; Piao, S.; and Wei, F. 2022. VLMo: Unified Vision-Language Pre-Training with Mixture-OfModality-Experts. In Advances in Neural Information Processing Systems 35, NeurIPS 2022, 32897—-32912. Neural Information Processing Systems Foundation, Inc. (NeurIPS). Choi, J. I. 2026. TabGeoFlow: A Geometric Flow Matching Model for Tabular Data Synthesis. Proceedings of the AAAI Conference on Artificial Intelligence, 40(25): 20562– –20569. Eijkelboom, F.; Bartosh, G.; Naesseth, C.; Welling, M.; and Meent, J.-W. V. D. 2024. Variational Flow Matching for Graph Generation. In Advances in Neural Information Processing Systems 37, NeurIPS 2024, 11735–11764. Neural Information Processing Systems Foundation, Inc. (NeurIPS). Esser, P.; Kulal, S.; Blattmann, A.; Entezari, R.; Müller, J.; Saini, H.; Levi, Y.; Lorenz, D.; Sauer, A.; Boesel, F.; Podell, D.; Dockhorn, T.; English, Z.; and Rombach, R. 2024. Scaling Rectified Flow Transformers for High-Resolution Image Synthesis. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, 12606–12633. PMLR. Gardner, J.; Popovic, Z.; and Schmidt, L. 2023. Benchmarking Distribution Shift in Tabular Data with TableShift. In Advances in Neural Information Processing Systems 36, NeurIPS 2023, 53385–53432. Neural Information Processing Systems Foundation, Inc. (NeurIPS). Guzmán-Cordero, A.; Eijkelboom, F.; and Van De Meent, J.-W. 2025. Exponential Family Variational Flow Matching for Tabular Data Generation. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, 21516–21529. PMLR. Holzmüller, D.; Grinsztajn, L.; and Steinwart, I. 2024. Better by default: Strong pre-tuned MLPs and boosted trees on tabular data. In Advances in Neural Information Processing Systems 37, NeurIPS 2024, 26577–26658. Neural Information Processing Systems Foundation, Inc. (NeurIPS).

Jolicoeur-Martineau, A.; Fatras, K.; and Kachman, T. 2024. Generating and Imputing Tabular Data via Diffusion and Flow-based Gradient-Boosted Trees. In Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, volume 238 of Proceedings of Machine Learning Research, 1288–1296. PMLR. Kotelnikov, A.; Baranchuk, D.; Rubachev, I.; and Babenko, A. 2023. TabDDPM: Modelling Tabular Data with Diffusion Models. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, 17564–17579. PMLR. Lee, C.; Kim, J.; and Park, N. 2023. CoDi: Co-evolving Contrastive Diffusion Models for Mixed-type Tabular Synthesis. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, 18940–18956. PMLR. Lipman, Y.; Chen, R. T. Q.; Ben-Hamu, H.; Nickel, M.; and Le, M. 2023. Flow Matching for Generative Modeling. In The Eleventh International Conference on Learning Representations. Little, C.; Allmendinger, R.; and Elliot, M. 2025. Synthetic census microdata generation: A comparative study of synthesis methods examining the trade-off between disclosure risk and utility. Journal of Official Statistics, 41(1): 255–308. Liu, X.; Gong, C.; and Liu, Q. 2023. Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow. In The Eleventh International Conference on Learning Representations. Mueller, M.; Gruber, K.; and Fok, D. 2025. Continuous Diffusion for Mixed-Type Tabular Data. In International Conference on Learning Representations, volume 2025, 39186– 39225. Mueller, M.; Gruber, K.; and Fok, D. 2026. Cascaded Flow Matching for Heterogeneous Tabular Data with Mixed-Type Features. In Forty-third International Conference on Machine Learning. Nasution, B. I.; Eijkelboom, F.; Elliot, M.; Allmendinger, R.; and Naesseth, C. A. 2026. Flow Matching for Tabular Data Synthesis. Transactions on Machine Learning Research. Puigcerver, J.; Riquelme Ruiz, C.; Mustafa, B.; and Houlsby, N. 2024a. From Sparse to Soft Mixtures of Experts. In International Conference on Learning Representations, volume 2024, 28435–28445. Puigcerver, J.; Riquelme Ruiz, C.; Mustafa, B.; and Houlsby, N. 2024b. From Sparse to Soft Mixtures of Experts. In International Conference on Learning Representations, volume 2024, 28435–28445. Qian, Z.; Davis, R.; and van der Schaar, M. 2023. Synthcity: a benchmark framework for diverse use cases of tabular synthetic data. In Advances in Neural Information Processing Systems 36, NeurIPS 2023, 3173–3188. Neural Information Processing Systems Foundation, Inc. (NeurIPS). Shazeer, N.; Mirhoseini, A.; Maziarz, K.; Davis, A.; Le, Q. V.; Hinton, G. E.; and Dean, J. 2017. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts

Layer. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24–26, 2017, Conference Track Proceedings. OpenReview.net. Shi, J.; Xu, M.; Hua, H.; Zhang, H.; Ermon, S.; and Leskovec, J. 2025. TabDiff: a Mixed-type Diffusion Model for Tabular Data Generation. In International Conference on Learning Representations, volume 2025, 37353–37375. Si, J.; Ou, Z.; Qu, M.; Xiang, Z.; and Li, Y. 2026. TabRep: Training Tabular Diffusion Models with a Simple and Effective Continuous Representation. Transactions on Machine Learning Research.

A.1

Proof of Proposition 1

We use the population version of the endpoint objective in cat Equation (3). For f = (µ, {πk }dk=1 ), write   ∥X1num − µt (Xt )∥22 RT (f ) = E 2νt d cat h i X − E log πk,t (Xt )C (k) , 1

k=1

Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L. u.; and Polosukhin, I. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.

where the expectation includes t, whose density on [0, T ] is ρ. Each πk,t (Xt ) is required to lie in the probability simplex ∆Kk . For a conditioning σ-algebra A, let R∗T (A) be the infimum of RT (f ) over all A-measurable predictors. The assumptions of Proposition 1 do not require νt to be bounded away from zero. Consequently, both Bayes risks can be infinite, in which case the expression +∞ − (+∞) is not defined. We therefore use a common truncation only to define the difference in this exceptional case. Set a(t) = (2νt )−1 and aM (t) = a(t) ∧ M , and let R∗T,M (A) denote the Bayes risk obtained by replacing a with aM in the numerical term. We interpret the risk gap in Proposition 1 as  ∆T (G, F) := sup R∗T,M (G) − R∗T,M (F) . (7)

Xu, L.; Skoularidou, M.; Cuesta-Infante, A.; and Veeramachaneni, K. 2019. Modeling Tabular data using Conditional GAN. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.

The proof below shows that the quantity inside the supremum is nondecreasing in M . Whenever both untruncated Bayes risks are finite, ∆T (G, F) agrees with the ordinary difference R∗T (G) − R∗T (F).

Zhang, H.; Fang, L.; Wu, Q.; and Yu, P. S. 2025. TabNAT: A Continuous-Discrete Joint Generative Framework for Tabular Data. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, 74858–74881. PMLR.

Proof of Proposition 1. For A ∈ {G, F }, define

van Breugel, B.; Seedat, N.; Imrie, F.; and van der Schaar, M. 2023. Can You Rely on Your Model Evaluation? Improving Model Evaluation with Synthetic Test Data. In Advances in Neural Information Processing Systems 36, NeurIPS 2023, 1889–1904. Neural Information Processing Systems Foundation, Inc. (NeurIPS).

Zhang, H.; Zhang, J.; Shen, Z.; Srinivasan, B.; Qin, X.; Faloutsos, C.; Rangwala, H.; and Karypis, G. 2024. MixedType Tabular Data Synthesis with Score-based Diffusion in Latent Space. In International Conference on Learning Representations, volume 2024, 52829–52857. Zhao, Z.; Kunar, A.; Birke, R.; and Chen, L. Y. 2021. CTABGAN: Effective Table Data Synthesizing. In Proceedings of The 13th Asian Conference on Machine Learning, volume 157 of Proceedings of Machine Learning Research, 97–112. PMLR.

Appendix A

Proofs

This section follows the order of the theoretical analysis in the main paper. We first prove Proposition 1, then derive the two architectural bottleneck examples, and finally prove Theorem 1. We conclude by verifying that a fixed FUSE network satisfies the regularity conditions used in Theorem 1.

M >0

mA = E[X1num | A],

(k)

pk,A (r) = P(C1

= r | A).

Because E∥X1num ∥22 < ∞, conditional Jensen’s inequality gives mA ∈ L2 . For any finite A-measurable numerical predictor h, the conditional projection identity gives, in [0, ∞],     E ∥X1num − h∥22 | A = E ∥X1num − mA ∥22 | A + ∥mA − h∥22 . The identity for a predictor without an a priori squareintegrability assumption follows by truncating h and applying monotone convergence. Since aM (t) is σ(t)-measurable and σ(t) ⊆ A, multiplication by aM (t) and integration show that mA minimizes the truncated numerical Prisk. For the categorical term, let H(p) = − r p(r) log p(r). For any A-measurable probability vector qk , h i (k) E − log qk (C1 ) | A = H(pk,A ) + KL (pk,A ∥qk ) . Hence the Bayes categorical predictor is pk,A . Combining the numerical and categorical terms yields dcat   X R∗T,M (A) = E aM (t)∥X1num − mA ∥22 + E[H(pk,A )] . k=1

(8) Because G ⊆ F, the tower property gives mG = E[mF | G] and pk,G = E[pk,F | G] coordinatewise. Applying the

nested projection identity to the numerical term in Equation (8) gives    E aM (t) ∥X1num − mG ∥22 − ∥X1num − mF ∥22   = E aM (t)∥mF − mG ∥22 . For each categorical feature, the cross-entropy decomposition gives E [H(pk,G ) − H(pk,F )] = E [KL (pk,F ∥pk,G )] . R∗T,M (G) − R∗T,M (F) dcat   X = E aM (t)∥mF − mG ∥22 + E[KL (pk,F ∥pk,G )] . k=1

The right-hand side is nondecreasing in M . Taking the supremum in Equation (7) and applying monotone convergence gives   ∥mF − mG ∥22 ∆T (G, F) = E 2νt dcat X

E[KL (pk,F ∥pk,G )] ≥ 0.

k=1

This is the identity stated in Proposition 1, with the commontruncation interpretation used only when ordinary subtraction is undefined. Finally, a(t) > 0 almost surely, and every KL divergence is nonnegative. Thus the gap is zero if and only if mF = mG

and

 δu E[S | U = u] = tanh 2 , σ   δu . Var(S | U = u) = sech2 σ2 

For the numerical endpoint mean m(1) (U, S) = αU + cS, the full and restricted posterior means are (1)

(1)

mG = αU + cE[S | U ],

mF = αU + cS,

Therefore, for every M > 0,

+

Since S ∈ {−1, 1}, it follows that

pk,F = pk,G

for every k,

almost surely. This proves the equality characterization. We briefly record when the common-truncation convention reduces to ordinary subtraction. Since each categorical entropy is bounded by log Kk , both untruncated Bayes risks are finite whenever   ∥X1num ∥22 E < ∞. νt Indeed, the zero numerical predictor and uniform categorical predictors then have finite risk. For the variance schedule used in the experiments, νt = 1 − t2 ≥ 0 on [0, 1], so the assumed second moment of X1num is sufficient. Hence the difference in Proposition 1 is an ordinary finite subtraction for the reported experimental settings.

where F = σ(U, S) and G = σ(U ). Proposition 1 therefore gives " (1) # (1)  {mF − mG }2 c2  E = E {S − E[S | U ]}2 2ν 2ν c2 E[Var(S | U )] 2ν    c2 δU = E sech2 > 0. 2ν σ2 The strict inequality follows because c ̸= 0, ν > 0, and sech2 (z) > 0 for every finite z. =

Representation bottleneck. Let H = L2 (PW,V ). Since W and V are independent standard normal variables, ⟨W, W ⟩H = ⟨V, V ⟩H = 1,

⟨W, V ⟩H = 0.

Fix h ∈ H with ∥h∥H = 1. The least-squares coefficients are θ2∗ = ⟨aW, h⟩H ,

θ3∗ = ⟨bV, h⟩H ,

and hence inf

θ2 ,θ3 ∈R

  E (aW − θ2 h)2 + (bV − θ3 h)2

= a2 + b2 − a2 ⟨W, h⟩2H − b2 ⟨V, h⟩2H . By Bessel’s inequality, a2 ⟨W, h⟩2H + b2 ⟨V, h⟩2H ≤ max{a2 , b2 } ⟨W, h⟩2H + ⟨V, h⟩2H ≤ max{a2 , b2 }. The bound is attained by h = W when a2 ≥ b2 and by h = V otherwise. Therefore, h i inf E {m(2) − f2 }2 + {m(3) − f3 }2 (f2 ,f3 )∈Hshared

= min{a2 , b2 }.

Recall the construction in the

Two shared latent components are sufficient once the features can use different recombination weights. Define 3aW − bV −aW + 3bV g1 = , g2 = , 2 2 c2 = (3/4, 1/4), c3 = (1/4, 3/4).

P(S = 1) = P(S = −1) = 12 , U = δS + ξ, ξ ∼ N (0, σ 2 ),

Both c2 and c3 are probability vectors, and direct calculation gives

A.2

Derivations for the Two Architectural Bottlenecks

Conditioning bottleneck. main paper:

where ξ ⊥ S, δ ̸= 0, and σ 2 > 0. Bayes’ rule gives log

P(S = 1 | U = u) (u + δ)2 − (u − δ)2 2δu = = 2 . P(S = −1 | U = u) 2σ 2 σ

c⊤ 2 (g1 , g2 ) = aW,

c⊤ 3 (g1 , g2 ) = bV.

Thus the same component dictionary represents both endpoint means exactly, while feature-dependent recombination selects a different mixture for each feature.

A.3

Proof of Theorem 1

For t < T , let pt = L(Xt ), where Xt is the interpolation in Equation (1). Define the true and learned embedded endpoint means by mt (x) = E[X1 | Xt = x],
m b θ,t (x) = µθ,t (x), πθ,1,t (x), . . . , πθ,dcat ,t (x) ∈ RD . Each categorical probability vector occupies its corresponding one-hot block. As defined in Section 2, the endpoint predictor induces m b θ,t (x) − x . 1−t Let f ∗ be the unrestricted Bayes predictor and use the shorthand vθ,t (x) =

ET (θ) = RT (fθ ) − RT (f ∗ ), which is the excess risk appearing in Theorem 1. Proof of Theorem 1. Because νt ≥ ν > 0, the zero numerical predictor and uniform categorical predictors give d

0 ≤ RT (f ∗ ) ≤

cat E∥X1num ∥22 X log Kk < ∞. + 2ν

k=1

Thus ET (θ) is well defined in [0, ∞]. If it is infinite, the claimed bound is immediate, so we assume ET (θ) < ∞ below. Moreover, each categorical one-hot block has squared norm one, and therefore E∥X1 ∥22 = E∥X1num ∥22 + dcat < ∞. Hence p0 , p1 ∈ P2 (RD ). For categorical feature k, let (k)

p∗k,t (x) = P(C1

= · | Xt = x).

The categorical blocks of mt (x) are precisely p∗k,t (x). The conditional projection and cross-entropy identities used in the proof of Proposition 1 yield Z 1 T ρ(t) ET (θ) = ∥µθ,t − mnum ∥2L2 (pt ) dt t 2 0 νt Z dcat Z T X
ρ(t) KL p∗k,t (x) πθ,k,t (x) pt (dx) dt. + k=1

0

Pinsker’s inequality implies ∥p∗k,t (x) − πθ,k,t (x)∥22 ≤ ∥p∗k,t (x) − πθ,k,t (x)∥21
≤ 2 KL p∗k,t (x) πθ,k,t (x) . Set b2t = ∥m b θ,t − mt ∥2L2 (pt ) . Jointly measurable versions of the conditional means can be chosen below, so t 7→ b2t is measurable. Combining the numerical projection identity with Pinsker’s inequality gives Z T (9) ρ(t)b2t dt ≤ 2 max{ν, 1}ET (θ). 0

This is the only point at which the numerical Gaussian loss and the categorical cross-entropy loss enter the Wasserstein argument. The assumed uniform L-Lipschitz property gives the linear-growth bound ∥vθ,t (x)∥2 ≤ ∥vθ,t (0)∥2 + L∥x∥2 . RT Together with joint measurability and 0 ∥vθ,t (0)∥2 dt < ∞, this gives a unique global Carathéodory flow Φt . Gronwall’s inequality yields   Z t ∥Φt (x)∥2 ≤ eLt ∥x∥2 + ∥vθ,s (0)∥2 ds . 0 D

Since p0 ∈ P2 (R ), the learned law pbθt = (Φt )# p0 also belongs to P2 (RD ) for every t ≤ T . We next construct an oracle velocity whose marginal law is pt . Let γσ denote the density of N (0, σ 2 ID ). Since T < 1, for t ∈ [0, T ] define Z dt (x) = γ1−t (x − ty) p1 (dy), Z nt (x) = yγ1−t (x − ty) p1 (dy), nt (x) , dt (x) mt (x) − x ut (x) = = E[X1 − X0 | Xt = x]. (10) 1−t The denominator dt (x) is strictly positive. Because E∥X1 ∥2 < ∞ and 1 − t ≥ 1 − T > 0, parameterized dominated convergence shows that (t, x) 7→ (dt (x), nt (x)) is Borel measurable. Thus Equation (10) provides a jointly Borel version of both mt and ut . For every φ ∈ Cc1 (RD ), differentiating the explicit interpolation in Equation (1) gives, for almost every t, Z   d φ(x) pt (dx) = E ∇φ(Xt )⊤ (X1 − X0 ) dt Z = ∇φ(x)⊤ ut (x) pt (dx). mt (x) =

Moreover, conditional Jensen’s inequality gives Z T ∥ut ∥2L2 (pt ) dt ≤ T E∥X1 − X0 ∥22 < ∞, 0

and the interpolation coupling gives W2 (ps , pt ) ≤ |s − t| E∥X1 − X0 ∥22

1/2

.

Therefore (pt , ut ) satisfies the continuity equation with finite kinetic energy. The superposition principle (Ambrosio, Gigli, and Savaré 2008, Theorem 8.2.1) provides a process (Γt )t∈[0,T ] on a common probability space such that Γt ∼ pt ,

Γ̇t = ut (Γt )

Initialize the learned flow with the same random variable Γ0 and set b t = Φt (Γ0 ), b t − Γ t ∥2 , Γ Dt = ∥Γ et = ∥vθ,t (Γt ) − ut (Γt )∥2 .

By the endpoint-induced form of vθ,t and Equation (10), E[e2t ] =

sup ∥m b θ,t (0)∥2 ≤ Bθ,T ,

Equation (9), the lower bound on ρ, and 1 − t ≥ 1 − T imply "Z # T 2 max{ν, 1} 2 et dt ≤ E ET (θ) < ∞. ρ(1 − T )2 0 Thus e· ∈ L1 (0, T ) almost surely. Using the L-Lipschitz property of vθ,t , the chain rule gives, for almost every t, D0 = 0.

The same inequality holds at times for which Dt = 0 by the standard absolute-continuity argument. Gronwall’s inequality therefore gives Z T DT ≤

eL(T −t) et dt.

0

b T , ΓT ), Minkowski’s inequality, Using the coupling (Γ RT Cauchy–Schwarz, and ΓL (T ) = 0 e2L(T −t) dt, we obtain W2 (b pθT , pT ) ≤ ∥DT ∥L2 Z T L(T −t) e bt dt ≤ 1−t 0 p Kρ,ν ΓL (T ) p ≤ ET (θ), 1−T

1/2

.

(12)

Relation to uniform training on [0, 1]. Theorem 1 allows a general time density ρ on [0, T ]. When training uses t ∼ Unif(0, 1) as in Section 2, let rθ (t) ≥ 0 denote the fixedtime excess endpoint risk. The uniform risk restricted to [0, T ] satisfies Z T 0

1 rθ (t) dt ≤ T

(13)

D

for all t ∈ [0, T ], x, y ∈ R . Proof. We use the Euclidean norm for vectors, the Frobenius norm for token matrices, and ∥ · ∥op for the operator norm. Assuming the schema and architecture are finite, all spaces appearing below are finite-dimensional. All constants may depend on the fixed architecture, the parameter setting θ, and T , but not on the input x. We first establish uniform boundedness and Lipschitz continuity of the normalization operators used in the network. We then propagate these properties through the adaptivemixture networks, joint-attention, and endpoint heads. Time-conditioned normalization. Let 1 P = Idh − 11⊤ dh be the centering matrix. For ϵLN > 0, layer normalization has the form Pz + β. LNγ,β (z) = γ ⊙  1/2 −1 dh ∥P z∥22 + ϵLN p ∥P z∥2 ≤ dh ,  −1 1/2 dh ∥P z∥22 + ϵLN we obtain

Theorem 1 concerns the continuous preprocessed Euclidean state in RD , including the one-hot categorical blocks. It therefore bounds the distribution transported by the ODE before categorical argmax and inverse preprocessing.

1 T

∥m b θ,t (x) − m b θ,t (y)∥2 ≤ Kθ,T ∥x − y∥2

(11)

The triangle inequality, together with Equations (11) and (12), proves Theorem 1.

ETunif (θ) =

0≤t≤T

Because

where Kρ,ν = 2 max{ν, 1}/ρ, exactly as in Theorem 1. Finally, coupling XT = (1 − T )X0 + T X1 with X1 gives W2 (pT , p1 ) ≤ (1 − T ) E∥X0 − X1 ∥22

Regularity of the FUSE Vector Field

Lemma 1. Fix T ∈ (0, 1), and let θ be a finite parameter. Then there exist constants Bθ,T , Kθ,T < ∞ such that

b2t . (1 − t)2

Dt′ ≤ LDt + et ,

A.4

Z 1

1 rθ (t) dt = E1unif (θ). T 0

This relation explains how the population objective trained on the full time interval controls the truncated-time risk used in the theorem.

sup ∥LNγ,β (z)∥2 ≤

p

dh ∥γ∥∞ + ∥β∥2 =: BLN .

z∈Rdh

The positive value of ϵLN prevents the normalization denominator from approaching zero. The Jacobian of the normalization map is therefore uniformly bounded, and hence ∥γ∥∞ ∥LNγ,β (z) − LNγ,β (z ′ )∥2 ≤ √ ∥z − z ′ ∥2 ϵLN =: KLN ∥z − z ′ ∥2 . Each processing or attention branch uses its own normalization. We write a generic such operator as Nq (t, z) = aq (t) ⊙ LNγq ,βq (z) + cq (t). For the implementation in Equation (9), aq (t) = 1 + γq (t) and cq (t) = ζq (t). Since the time embedding and all affine maps are continuous in t, compactness of [0, T ] gives Aq := sup ∥aq (t)∥∞ < ∞, 0≤t≤T

Cq := sup ∥cq (t)∥2 < ∞. 0≤t≤T

It follows that sup ∥Nq (t, z)∥2 ≤ Aq BLN,q + Cq =: BN ,q , 0≤t≤T z∈Rdh

∥Nq (t, z) − Nq (t, z ′ )∥2 ≤ Aq KLN,q ∥z − z ′ ∥2 =: KN ,q ∥z − z ′ ∥2 .

Applying the same normalization to a token matrix preserves Lipschitz continuity in the Frobenius norm. Since each schema has finitely many tokens, there are finite constants B N ,q and K N ,q satisfying

If Vr denotes the matrix obtained by stacking the transformed components, the finite number of components implies that there are constants BV,r , KV,r < ∞ satisfying ∥Vr ∥F ≤ BV,r ,

sup ∥Nq (t, H)∥F ≤ B N ,q ,

∥Vr − Vr′ ∥F ≤ KV,r ∥Hr − Hr′ ∥F .

t,H

∥Nq (t, H) − Nq (t, H ′ )∥F ≤ K N ,q ∥H − H ′ ∥F . Adaptive-mixture processing. Consider a mixture of networks for feature type r ∈ {num, cat} at a fixed layer. To simplify notation, omit the layer index and write

The adaptive-mixture residual network is Brmix (t, Hr ) = Cr Vr . Using

H r = Nrmix (t, Hr ),

Cr Vr − Cr′ Vr′ = Cr (Vr − Vr′ ) + (Cr − Cr′ )Vr′ ,

H r Φr Rr = √ , dh Dr = softmaxcol (Rr ),

we obtain Cr = softmaxrow (Rr ).

The two softmax operations correspond to aggregation across feature tokens and recombination across latent components in Equations (4)–(5) of the main paper. The normalization bounds imply B N ,r ∥Φr ∥op √ ∥Rr ∥F ≤ , dh K N ,r ∥Φr ∥op √ ∥Hr − Hr′ ∥F . ∥Rr − Rr′ ∥F ≤ dh Softmax on a fixed finite-dimensional axis is bounded and Lipschitz. Therefore, finite constants BD,r , BC,r , KD,r , and KC,r exist such that ∥Dr ∥F ≤ BD,r ,

∥Cr ∥F ≤ BC,r , ′ ∥Dr − Dr ∥F ≤ KD,r ∥Hr − Hr′ ∥F , ∥Cr − Cr′ ∥F ≤ KC,r ∥Hr − Hr′ ∥F .

∥Brmix (t, Hr )∥F ≤ BC,r BV,r =: bmix r , ∥Brmix (t, Hr )−Brmix (t, Hr′ )∥F ≤ (BC,r KV,r + KC,r BV,r ) ∥Hr − Hr′ ∥F ′ =: ℓmix r ∥Hr − Hr ∥F .

Thus every adaptive-mixture residual network is uniformly bounded and globally Lipschitz. Joint attention. Let H denote the concatenated numerical and categorical tokens entering a joint-attention network, and define Z = N att (t, H). The normalization bounds provide finite constants BZ , KZ such that ∥Z∥F ≤ BZ , ∥Z − Z ′ ∥F ≤ KZ ∥H − H ′ ∥F .

The aggregated components are Ur = Dr⊤ H r . They are uniformly bounded because ∥Ur ∥F ≤ ∥Dr ∥F ∥H r ∥F

For one attention head, write Q = ZWQ + 1b⊤ Q,

V = ZWV + 1b⊤ V.

Because these maps are affine and Z is bounded, there exist finite constants BQ , BK , BV , KQ , KK , KV such that

≤ BD,r B N ,r =: BU,r . For two inputs Hr and Hr′ , we have ′

K = ZWK + 1b⊤ K,

∥Q∥F ≤ BQ , ∥K∥F ≤ BK , ∥V ∥F ≤ BV ,

′

Ur − Ur′ = Dr⊤ (H r − H r ) + (Dr − Dr′ )⊤ H r .

∥Q − Q′ ∥F ≤ KQ ∥H − H ′ ∥F , ∥K − K ′ ∥F ≤ KK ∥H − H ′ ∥F ,

Hence,

∥V − V ′ ∥F ≤ KV ∥H − H ′ ∥F .

∥Ur − Ur′ ∥F ≤ BD,r K N ,r + KD,r B N ,r ∥Hr − Hr′ ∥F

=: KU,r ∥Hr − Hr′ ∥F .

Let

All components of Ur lie in the compact set

 A = softmaxrow

KU,r = {u : ∥u∥2 ≤ BU,r } . Every specialized subnetwork is a fixed finite composition of affine maps and continuously differentiable activations. Since there are finitely many subnetworks, continuity on KU,r gives Bg,r := max sup ∥gr,m (u)∥2 < ∞, m

u∈KU,r

Kg,r := max sup ∥Dgr,m (u)∥op < ∞. m

u∈KU,r

QK ⊤ √ dhead



be the attention-weight matrix. The difference between two attention matrices satisfies QK ⊤ − Q′ K ′⊤ = (Q − Q′ )K ⊤ + Q′ (K − K ′ )⊤ . Consequently, ∥QK ⊤ − Q′ K ′⊤ ∥F ≤ (KQ BK + BQ KK ) ∥H − H ′ ∥F .

Since row-wise softmax is bounded and Lipschitz on a fixed finite-dimensional space, there exist BA , KA < ∞ such that ∥A∥F ≤ BA , ∥A − A′ ∥F ≤ KA ∥H − H ′ ∥F . The head output O = AV therefore satisfies ∥O∥F ≤ BA BV , ∥O − O′ ∥F ≤ (BA KV + KA BV ) ∥H − H ′ ∥F . A finite concatenation of attention heads followed by a finite affine output projection preserves boundedness and global Lipschitz continuity. Thus the complete multi-head attention network in Equation (6) of the main paper satisfies sup ∥B

att

att

(t, H)∥F ≤ b

∥B

(t, H) − B

att

att

for finite constants b

′

att

(t, H )∥F ≤ ℓ att

and ℓ

q=1

Kθ,T = Kout K0

Q Y

(1 + ℓq ).

q=1

< ∞,

t,H att

Endpoint heads. The endpoint heads consist of layer normalization with a positive epsilon, finite affine maps, and categorical softmax maps. They therefore define a Lipschitz output map with a finite value at the origin. Hence, there are finite constants Bout and Kout such that ∥m b θ,t (0)∥2 ≤ Bout + Kout ∥HQ (t, 0)∥2 , ∥m b θ,t (x) − m b θ,t (y)∥2 ≤ Kout ∥HQ (t, x) − HQ (t, y)∥2 . Therefore, we may take ! Q X Bθ,T = Bout + Kout B0 + bq ,

′

∥H − H ∥F

These constants satisfy sup ∥m b θ,t (0)∥2 ≤ Bθ,T , 0≤t≤T

.

Propagation through the residual network. Flatten the finite collection of token matrices into a single Euclidean vector and index all adaptive-mixture and attention residual networks in their computational order by q = 1, . . . , Q. Write Hq (t, x) = Hq−1 (t, x) + Bq (t, Hq−1 (t, x)) . The preceding arguments show that finite constants bq , ℓq exist such that sup ∥Bq (t, H)∥2 ≤ bq , t,H

∥Bq (t, H) − Bq (t, H ′ )∥2 ≤ ℓq ∥H − H ′ ∥2 . The initial tokenization is affine in x and depends on the time embedding. Compactness of [0, T ] therefore gives B0 := sup ∥H0 (t, 0)∥2 < ∞, 0≤t≤T

K0 := sup Lipx H0 (t, ·) < ∞. 0≤t≤T

At the origin, ∥Hq (t, 0)∥2 ≤ ∥Hq−1 (t, 0)∥2 + bq . Induction over the finite residual sequence gives sup ∥HQ (t, 0)∥2 ≤ B0 + 0≤t≤T

Q X

bq .

q=1

For arbitrary x, y ∈ RD , ∥Hq (t, x) − Hq (t, y)∥2 ≤ ∥Hq−1 (t, x) − Hq−1 (t, y)∥2 + ∥Bq (t, Hq−1 (t, x)) − Bq (t, Hq−1 (t, y))∥2 ≤ (1 + ℓq )∥Hq−1 (t, x) − Hq−1 (t, y)∥2 . A second induction yields ∥HQ (t, x) − HQ (t, y)∥2 ≤ K0

Q Y

(1 + ℓq )∥x − y∥2 .

q=1

∥m b θ,t (x) − m b θ,t (y)∥2 ≤ Kθ,T ∥x − y∥2 . Every operation used above is continuous in its finitedimensional arguments, and the time embedding is continuous on [0, T ]. Since FUSE is a finite composition of these operations, (t, x) 7→ m b θ,t (x) is jointly continuous. Finally, the endpoint predictor induces m b θ,t (x) − x vθ,t (x) = . 1−t Since t ≤ T < 1, Kθ,T + 1 ∥vθ,t (x) − vθ,t (y)∥2 ≤ ∥x − y∥2 , 1−T Bθ,T , ∥vθ,t (0)∥2 ≤ 1−T Z T T Bθ,T < ∞. ∥vθ,t (0)∥2 dt ≤ 1−T 0 Joint continuity implies joint Borel measurability. Thus vθ,t is jointly measurable, uniformly Lipschitz in x, and integrable at the origin, as required by Theorem 1.

B

Related Work

Mixed-Type Tabular Generation. Early approaches such as CTGAN and TVAE combine type-specific preprocessing with adversarial or variational objectives (Xu et al. 2019). Diffusion-based methods represent mixed-type variables in different ways. TabDDPM and CoDi couple continuous and categorical diffusion processes, while TabDiff uses a joint continuous-time model with feature-wise learnable noise schedules (Kotelnikov et al. 2023; Lee, Kim, and Park 2023; Shi et al. 2025). TabSyn performs diffusion in a learned continuous latent space, CDTD applies continuous diffusion to both variable types, and TabRep constructs a unified continuous representation (Zhang et al. 2024; Mueller, Gruber, and Fok 2025; Si et al. 2026). TabNAT instead combines diffusion-based numerical generation with masked categorical generation (Zhang et al. 2025). FUSE follows EF-VFM in representing categorical variables as one-hot vectors along a Euclidean interpolation, while revising the network structure used to predict the endpoint-factor parameters.

Variational Flow Matching. Flow matching learns a velocity field that transports a base distribution along a prescribed probability path (Lipman et al. 2023; Albergo, Boffi, and Vanden-Eijnden 2025). Variational flow matching expresses this field through a variational approximation to the posterior distribution over endpoints (Eijkelboom et al. 2024). When the conditional velocity field is linear in the endpoint, the resulting field depends only on the marginal endpoint means. This permits a mean-field factorization without requiring the joint posterior covariance. EF-VFM extends this formulation to mixed-type data using exponential-family endpoint factors and moment matching, with TabbyFlow providing its tabular implementation (Guzmán-Cordero, Eijkelboom, and Van De Meent 2025). Other tabular flow methods replace neural predictors with gradient-boosted trees, impose geometric constraints, adopt cascaded generation, or examine the effects of probability paths and sampling schemes (Jolicoeur-Martineau, Fatras, and Kachman 2024; Choi 2026; Mueller, Gruber, and Fok 2026; Nasution et al. 2026). FUSE retains the probability path and endpoint-factor family of EF-VFM, but focuses on how type-specific processing and full-state conditioning should be organized.

{proc, attn} index the branch. For feature type r ∈ {num, cat} and layer ℓ,
(ℓ) (ℓ) (ℓ) ζr,q (t), γr,q (t) = Wr,q SiLU(ψ(t)) + b(ℓ) r,q , n o (ℓ) (ℓ) (ℓ) Nr,q (H; t) = LN(H) ⊙ 1 + γr,q (t) + ζr,q (t). (15) Layer normalization is applied without learned affine parameters and uses ϵLN = 10−5 . The output maps producing (ℓ) (ℓ) ζr,q (t) and γr,q (t) are zero-initialized. The normalized representation used by adaptive mixture processing is (ℓ) (ℓ)
(ℓ) H r,t = Nr,proc Hr,t ; t , while the attention uses its own normalization parameters.

C.2

Architecture Specification

Adaptive mixture processing. The implementation computes the alignment score matrix as (ℓ) (ℓ) er,t = H r,t Φ(ℓ) R r .

(16)

(ℓ)

Specialized Neural Processing. Mixture architectures process representations through specialized subnetworks whose contributions depend on the input. Sparse mixture models use learned gates to select a subset of subnetworks, whereas Soft MoE replaces discrete assignment with differentiable dispatch and combination weights over weighted token aggregates (Shazeer et al. 2017; Puigcerver et al. 2024b). A related architectural pattern separates modalityspecific feed-forward processing from cross-modal interaction. VLMo combines modality-specific feed-forward modules with shared self-attention, while multimodal diffusion transformers use separate modality-specific parameters while allowing joint attention (Bao et al. 2022; Esser et al. 2024). These architectures show that specialized processing can coexist with unrestricted interaction across input types.

C

Implementation Details

This section specifies the implementation choices omitted from the main paper. It describes time conditioning, the architectural parameterization, and capacity matching.

C.1

Time Conditioning

Let dh ≥ 2 denote the hidden dimension and let dω = ⌊dh /2⌋. The time t is embedded as   k log(10000) ωk = exp − , k = 0, . . . , dω − 1, max{1, dω }
ω −1 ω −1 s(t) = {sin(tωk )}dk=0 , {cos(tωk )}dk=0 , ψ(t) = W2 SiLU(W1 s(t) + b1 ) + b2 .

(14)

When dh is odd, a zero coordinate is appended to s(t) so that its dimension equals dh . The feature-specific affine maps described in the main paper add a type-specific projection of ψ(t) to each initial token. Each processing and attention branch uses independent time-conditioned normalization parameters. Let q ∈

Since Φr is learned, this omission changes its parameterization but not the represented family of alignment scores. Each specialized subnetwork contains two linear layers with a GELU activation. No dropout or learned residual gate is used within the processing module. Joint attention. Joint attention is applied without an attention mask. The restricted-attention variant retains the same parameterization but masks the two cross-type attention blocks, allowing interactions only within each feature type. Every configuration contains L = 4 blocks, each consisting of one processing module followed by one attention module. Attention dropout is set to zero. Endpoint heads. After the final block, numerical and categorical tokens are normalized using separate affine layernormalization modules, h i  (L) (L) hr,j,t = LNr Hr,t , r ∈ {num, cat}, L = 4. j

(L)

Thus, hr,j,t in the endpoint-head equations of the main paper denotes the normalized final token rather than the cor(L) responding row of Hr,t before normalization. The numerical head applies no additional output activation, while the categorical heads retain the feature-wise parameterization defined in the main paper. The final normalization uses ϵLN = 10−5 . For fixed learned parameters, this positive normalization constant ensures bounded final token representations, as required in Lemma 1.

C.3

Capacity Matching

For each feature type, the dense FFN baseline uses a two-layer feed-forward module with hidden width dstandard = 4096, equal to the sum of the widths of the four specialized subnetworks. Across the eight datasets, FUSE contains between 20.264 and 20.317 million trainable parameters, while the dense processing variants contain between 20.242 and

α-Precision (↑) Method

Adult

Default

Beijing

Shoppers

Magic

News

Diabetes

Fault Avg. rank ↓

CTGAN 0.715(.003) 0.683(.002) 0.916(.003) 0.821(.003) 0.789(.004) 0.974(.002) 0.640(.013) 0.656(.012) TVAE 0.923(.003) 0.828(.004) 0.859(.003) 0.567(.004) 0.951(.003) 0.857(.003) 0.774(.018) 0.878(.014) CoDi 0.497(.003) 0.710(.001) 0.949(.003) 0.727(.005) 0.842(.005) 0.523(.002) 0.896(.024) 0.924(.033) TabDDPM 0.971(.004) 0.994(.002) 0.977(.002) 0.938(.004) 0.992(.003) 0.000(.000) 0.876(.206) 0.003(.012) TabSyn 0.974(.003) 0.924(.003) 0.987(.003) 0.978(.006) 0.991(.003) 0.968(.003) 0.955(.025) 0.936(.018) TabbyFlow 0.991(.002) 0.990(.002) 0.994(.002) 0.993(.002) 0.989(.004) 0.918(.003) 0.972(.012) 0.955(.015) TabDiff 0.979(.003) 0.982(.002) 0.981(.003) 0.992(.002) 0.992(.003) 0.965(.003) 0.926(.029) 0.697(.051) FUSE 0.992(.002) 0.993(.002) 0.996(.002) 0.985(.005) 0.994(.003) 0.983(.006) 0.969(.013) 0.976(.012) β-Recall (↑) Method

Adult

Default

Beijing

Shoppers

Magic

News

Diabetes

6.63 6.50 6.38 5.00 3.63 2.63 3.75 1.50

Fault Avg. rank ↓

CTGAN 0.270(.002) 0.169(.002) 0.260(.002) 0.286(.004) 0.082(.003) 0.161(.002) 0.078(.009) 0.007(.002) TVAE 0.340(.003) 0.185(.001) 0.170(.002) 0.173(.003) 0.328(.003) 0.309(.004) 0.346(.010) 0.113(.007) CoDi 0.229(.003) 0.059(.002) 0.300(.003) 0.305(.004) 0.491(.003) 0.112(.001) 0.688(.021) 0.613(.025) TabDDPM 0.479(.002) 0.467(.004) 0.400(.002) 0.527(.006) 0.523(.004) 0.000(.000) 0.713(.173) 0.000(.000) TabSyn 0.428(.003) 0.403(.002) 0.307(.002) 0.494(.006) 0.481(.005) 0.443(.002) 0.510(.022) 0.338(.040) TabbyFlow 0.548(.003) 0.517(.003) 0.552(.003) 0.600(.005) 0.473(.004) 0.578(.002) 0.622(.028) 0.619(.017) TabDiff 0.538(.003) 0.520(.002) 0.472(.002) 0.487(.005) 0.493(.004) 0.387(.003) 0.466(.033) 0.036(.012) FUSE 0.497(.004) 0.452(.004) 0.551(.002) 0.526(.005) 0.468(.005) 0.444(.009) 0.767(.027) 0.703(.028)

7.13 6.50 5.50 4.00 4.38 2.13 3.63 2.75

Table 3: α-Precision and β-Recall across eight datasets. Results are reported as means with standard deviations in parentheses. Bold and underlined values indicate the best and second-best results, respectively, and the proposed method is shaded. 20.295 million. The difference is 22,528 parameters, corresponding to less than 0.12% of the total parameter count. A leading-order multiply-accumulate comparison provides additional context for the processing modules. Let N = dnum + dcat denote the total number of numerical and categorical feature tokens. In the evaluated configuration, Snum = Scat = S = 8,

dff = 1024,

dstd = 4096.

The per-observation counts for one processing layer are MACadap = 4Sdh dff + 3N Sdh , MACstd = 2N dh dstd . At the evaluated widths, the ratio MACstd / MACadap ranges from 2.24 to 11.59. These counts include only the leading matrix products in the processing modules. They exclude attention, normalization, activation functions, and other element-wise operations and therefore are not interpreted as end-to-end runtime measurements.

D

Experimental Details

This section provides additional details for the experiments in the main paper. Datasets. Table 4 summarizes the profiles of the datasets. Adult, Default, Shoppers, Magic, and Diabetes define binary classification tasks, while Fault defines a multiclass classification task. Beijing and News are regression datasets. Training. All FUSE configurations use hidden dimension dh = 256, four layers, and four attention heads. For each feature type, adaptive mixture processing uses Mr = 4 parallel subnetworks, Pr = 2 latent components per subnetwork, and hidden width dff = 1024. Training proceeds using AdamW with learning rate 10−3 . The model parameters are

Dataset

Task

Train

Test Num. Cat.

Adult Binary 32,561 16,281 Default Binary 27,000 3,000 Beijing Regression 37,581 4,176 Shoppers Binary 11,097 1,233 Magic Binary 17,117 1,902 News Regression 35,679 3,965 Diabetes Binary 691 77 Fault Multiclass 1,746 195

6 14 7 10 10 46 8 27

9 10 5 8 1 2 1 1

Table 4: Downstream tasks, processed train and test split sizes, and variable counts. Binary and multiclass denote classification tasks. Num. and Cat. denote numerical and categorical variables, respectively. The prediction target is included in its corresponding variable type.

maintained using an exponential moving average with decay 0.997. Batch sizes are 1024 for Adult, Magic, and News, 672 for Beijing, and 512 for the remaining datasets. The numerical endpoint variance is νt = 1 − t2 for t ∈ [0, 1). This schedule specifies the variance of the numerical endpoint factor and satisfies νt → 0 as t → 1.

b0 ∼ N (0, ID ), Sampling. Generation begins from X where D is the dimension of the transformed row representation. The resulting ODE is integrated to tmax = 0.999 using an adaptive fifth-order Dormand–Prince solver. Both relative and absolute tolerances are set to 10−5 , and the exponentialmoving-average parameters are used for generation. Each generated table contains ntrain observations.

Default: BILL_AMT4 Density Diabetes: Glucose Density

CTGAN

TVAE

CoDi

1.5

3

1.5

3

1.5

80

160

80

160

80

TabDDPM

TabSyn

TabbyFlow

TabDiff

3

1.5

3

1.5

160

80

160

80

CoDi

TabDDPM

TabSyn

FUSE

3

1.5

3

1.5

3

1.5

3

160

80

160

80

160

80

160

TabDiff

FUSE

3.0 1.5 0.0

1.6 0.8 0.0

Real

CTGAN

TVAE

TabbyFlow

Figure 4: Numerical marginal distributions on Default and Diabetes. Gray denotes the real data.

Diabetes: Outcome

Default: PAY_0

CTGAN

TVAE

CoDi

TabDDPM

TabSyn

TabbyFlow

TabDiff

FUSE

0.5 0.2 0.0

-2 1

5

0

1

8

-2 1

5

0

8

-2 1

5

1

0

1

CTGAN

TVAE

8

-2 1

5

0

1

8

-2 1

5

8

0

1

-2 1

5

0

8

-2 1

5

1

0

1

TabDiff

FUSE

8

-2 1

5

0

1

8

0.6 0.3 0.0

Real

CoDi

TabDDPM

TabSyn

TabbyFlow

Figure 5: Categorical marginals on Default and Diabetes. Gray denotes the real data.

E

Additional Experiments

The following experiments extend the main evaluation with support-based metrics, and additional visual comparisons.

E.1

α-Precision and β-Recall

Table 3 reports α-Precision and β-Recall across all eight datasets. The former evaluates whether synthetic observations remain within high-probability regions of the real distribution, while the latter evaluates how well the synthetic distribution covers the support of the real data (Alaa et al. 2022). Reporting both metrics distinguishes fidelity to the data support from support coverage. FUSE achieves the best average rank for α-Precision at 1.50 and the second-best average rank for β-Recall at 2.75, following TabbyFlow at 2.13. It also obtains the highest βRecall on Diabetes and Fault. These results show that the

strong support precision of FUSE is accompanied by competitive coverage of the real distribution.

E.2

Additional Synthetic Data Visualizations

Figures 4 and 5 compare the marginal distributions generated by all eight methods on Default and Diabetes, which are not included in the main marginal visualizations. Each numerical plotting range is determined from the corresponding real training data, while every category observed in the real data is retained in the categorical panels. FUSE closely follows the numerical distribution in Default and the unimodal distribution in Diabetes. In both cases, it reproduces the dominant probability mass while retaining the overall tail behavior. These observations extend the marginal comparisons in the main paper to two additional datasets and are consistent with the aggregate Shape results. For categorical variables, FUSE preserves both the domi-

Num Cat

Num Cat

Num Cat

Num Cat

Num Cat

Num

0.08 0.06

Num

Num

Num

Num

0.02

Cat

Cat

Cat

Cat

0.04

Num

Num Cat

Cat

Num Cat

0.10

Num

Num Cat

FUSE Num Cat

Cat

Num Cat

TabDiff Num Cat

Num

Num Cat

TabbyFlow Num Cat

Cat

Num Cat

TabSyn Num Cat

Diabetes Num

Num Cat

TabDDPM Num Cat

Cat

Num Cat

CoDi Num Cat

Cat

Num

TVAE Num Cat

Cat

Default Cat Num

CTGAN Num Cat

0

nant and lower-frequency levels in Default and closely reproduces the binary class balance in Diabetes. The categorical panels complement the numerical comparisons by illustrating the two components summarized by the Shape metric. Figure 6 extends the dependence analysis to Default and Diabetes and includes the four baselines omitted from the main heatmap. Unlike Trend, which averages similarities over pairs selected according to their dependence in the real data, the heatmaps display every variable pair. Numerical pairs report one half of the absolute difference between the real and synthetic Pearson correlations. Pairs involving a categorical variable report the total variation distance between the corresponding contingency tables. Numerical variables in mixed-type pairs are discretized into ten common bins before the contingency tables are constructed. The shared color scale is capped at 0.1, matching the main heatmap.

E.3

Extended Component Analysis

The component analysis in the main paper separates adaptive mixture processing from joint attention across datasets. This section examines joint attention more directly through a controlled cross-type dependence experiment and matched comparisons on the eight datasets. Controlled Cross-Type Dependence. To isolate the effect of cross-type dependence, we construct a mixed-type distribution in which ρdep controls the association between a binary variable and a numerical variable. For ρdep ∈ {0, 0.2, 0.4, 0.6, 0.8, 0.95}, let C ⋆ ∼ Bernoulli(1/2), ε ∼ N (0, 1), ε ⊥ C ⋆, q Z ⋆ = ρdep (2C ⋆ − 1) + 1 − ρ2dep ε. (17) This construction gives Corr(Z ⋆ , 2C ⋆ − 1) = ρdep . Each observation also contains two independent standard Gaussian variables and one independent categorical variable. The binary variable C ⋆ serves as the downstream target, making its association with Z ⋆ the only cross-type dependence. Joint and restricted attention are compared over five matched seeds at each dependence level. Both configurations use adaptive mixture processing and differ only in their attention scope. Each seed contains 20,000 training and 5,000

Endpoint-risk difference

Figure 6: Absolute differences between pairwise correlations from real and synthetic data (Default and Diabetes). Values closer to zero indicate more faithful preservation of feature correlation.

0.10

Numerical

Categorical

Learned

0.08 0.06 0.04 0.02 0.00 0.00

0.20

0.40 0.60 Dependence ρdep

0.80

0.95

Figure 7: Population conditioning penalty and learned endpoint-risk difference across dependence levels. Stacked bars show the numerical and categorical components of the population penalty. Green markers show the mean learned difference. Positive values favor joint attention. test observations. The population conditioning penalty is estimated using 106 Monte Carlo samples, while each pair of fitted models is evaluated on a separate sample of size 105 . Figure 7 shows that both the population penalty and the learned endpoint-risk reduction are near zero at independence and increase with cross-type dependence. At ρdep = 0.95, they reach 0.1043 and 0.0715, respectively. All five learned reductions are positive at every nonzero dependence level. Although the learned reductions compare fitted predictors rather than estimate the population penalty, their direction is consistent with Proposition 1. Table 5 shows that the MLE gain increases from 0.003 to 0.508 at ρdep = 0.95. In contrast, the effects remain at most 0.003 for Shape and 0.002 for C2ST. Joint attention therefore becomes most consequential for downstream utility as taskrelevant cross-type dependence strengthens. Cross-Dataset Evidence. The controlled experiment is complemented by evaluations on the eight benchmark cat datasets. First, let Rj,d,s denote the categorical endpoint risk on dataset d under seed s and attention configura-

ρdep

Shape

Trend

C2ST

MLE

α-Precision

β-Recall

0.00 0.20 0.40 0.60 0.80 0.95

0.000(.001) 0.000(.001) 0.001(.002) 0.003(.004) 0.000(.001) 0.003(.002)

0.002(.008) 0.002(.007) 0.008(.012) 0.015(.019) 0.028(.018) 0.010(.020)

0.002(.005) 0.000(.007) 0.002(.004) 0.002(.004) 0.002(.004) 0.000(.006)

0.003(.004) 0.104(.004) 0.239(.020) 0.364(.015) 0.482(.035) 0.508(.015)

0.001(.009) -0.003(.007) 0.015(.020) 0.016(.016) 0.004(.015) 0.001(.004)

0.000(.004) 0.000(.004) 0.012(.006) 0.035(.011) 0.102(.005) 0.161(.002)

Table 5: Effects of joint attention across dependence levels. Entries are means with sample standard deviations over five matched seeds. Positive values favor joint attention. Dataset

Shape

Trend

MLE α-Precision

C2ST

β-Recall

Adult 0.001(.001) 0.030(.001) 0.001(.005) 0.034(.001) 0.004(.001) 0.295(.004) Default 0.000(.000) 0.025(.020) 0.007(.011) 0.019(.007) 0.002(.002) 0.139(.004) Beijing 0.000(.001) 0.032(.003) -0.001(.003) 0.164(.004) 0.007(.001) 0.047(.003) Shoppers -0.001(.001) 0.006(.002) 0.004(.026) 0.270(.033) 0.007(.003) 0.092(.005) Magic 0.000(.000) 0.011(.005) 0.007(.006) 0.428(.015) 0.058(.002) 0.069(.007) News 0.000(.001) 0.003(.000) 0.000(.007) 0.007(.001) 0.001(.003) 0.217(.007) Diabetes 0.002(.002) 0.010(.003) 0.003(.014) 0.277(.044) 0.029(.015) 0.084(.025) Fault 0.000(.002) 0.012(.001) 0.007(.020) 0.462(.034) 0.022(.005) 0.303(.021) Table 6: Paired effects of joint attention on the six evaluation metrics. Entries are means with sample standard deviations over five matched seeds after averaging the adaptive and standard processing contrasts. Positive values favor joint attention. Dataset

tently favor joint attention, define  −1, if m is regression MLE, (m) ηd = 1, otherwise,

Relative increase (%) Positive pairs

Adult Default Beijing Shoppers Magic News Diabetes Fault

4.650 (0.051) 15.722 (0.157) 5.009 (0.037) 2.048 (0.179) 9.630 (0.121) 13.562 (0.806) 10.826 (1.689) 51.392 (1.916)

5/5 5/5 5/5 5/5 5/5 5/5 5/5 5/5

and average the paired effect equally over the two processing configurations as (m)  ηd X  (m) (m) Yp,joint,d,s − Yp,restricted,d,s . 2 Joint attention consistently improves Trend, β-Recall, and task-oriented MLE across the matched evaluations. As shown in Table 6, the mean effect on α-Precision is positive for every dataset, whereas the effects on Shape and C2ST remain small and vary in sign. These results indicate that joint attention contributes most consistently to dependence preservation, support coverage, and downstream utility. Together with the controlled experiment, the cross-dataset results show that this benefit persists across heterogeneous benchmark datasets. (m)

Table 7: Relative increase in categorical endpoint risk under restricted attention. Entries are means with sample standard deviations over five random seeds, together with the number of seeds yielding a positive paired difference.

tion j ∈ {joint, restricted}. The relative increase under restricted attention is defined as cat δd,s = 100 ×

cat cat Rrestricted,d,s − Rjoint,d,s . cat Rjoint,d,s

(18)

Both configurations use adaptive mixture processing and are evaluated using common interpolation draws over the same fixed held-out observations. Table 7 reports a positive mean increase on every dataset, with positive paired effects across all evaluations. The endpoint-risk advantage of joint attention therefore holds consistently across the datasets. The second analysis considers all six evaluation met(m) rics under both processing configurations. Let Yp,j,d,s denote metric m under processing configuration p ∈ {adaptive, standard} and attention configuration j ∈ {joint, restricted}. To ensure that positive values consis-

∆d,s =

Related documents

• VTV-FM: Flow Matching through Variational Terminal-Velocity Closure#1155720
• Conditional Flow Matching for Cross-Field MRI Harmonisation#820954
• Zero Flux: Flow-Based Comparison of High-Dimensional Discrete Distributions#1156744
• Prediction--Loss Alignment for Sampler--Robust Flow Matching Training#998105
• LatentFM: A Latent Flow Matching Approach for Generative Medical Image Segmentation#984096