Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

knowledgerepresentation reasoning

artificial intelligence, reasoning, knowledge representation

T REES TO F LOWS AND BACK : U NIFYING D ECISION T REES AND D IFFUSION M ODELS

arXiv:2605.00414v1 [cs.LG] 1 May 2026

Sai Niranjan Ramachandran*

Suvrit Sra*

May 4, 2026

A BSTRACT Decision trees and diffusion models are ostensibly disparate model classes, one discrete and hierarchical, the other continuous and dynamic. This work unifies the two by establishing a crisp mathematical correspondence between hierarchical decision trees and diffusion processes in appropriate limiting regimes. Our unification reveals a shared optimization principle: Global Trajectory Score Matching (GTSM), for which gradient boosting (in an idealized version) is asymptotically optimal. We underscore the conceptual value of our work through two key practical instantiations: T REE F LOW, which achieves competitive generation quality on tabular data with higher fidelity and a 2× computational speedup, and DSM-T REE, a novel distillation method that transfers hierarchical decision logic into neural networks, matching teacher performance within 2% on many benchmarks.

Introduction

Within machine learning, two classes of models have achieved great success, albeit in disjoint domains. For structured, tabular data, ensemble methods based on decision trees, particularly Gradient Boosting Machines (Friedman, 2001, 2002), remain the state-of-the-art, prized for their performance and interpretability. For continuous, complex data such as images and audio, score-based generative models, or Diffusion Models (DMs) (Ho et al., 2020; Song et al., 2021), are dominant, capable of generating samples of unparalleled fidelity. These two model classes live in different worlds: one relies on discrete, hierarchical partitioning of the feature space, while the other is defined by continuous-time stochastic differential equations (SDEs). But this conceptual separation obscures a deep and powerful connection. Though their formulations differ, both models implicitly perform a hierarchical refinement of information: decision trees build a coarse-to-fine sequence of partitions, while diffusion models learn to reverse a coarse-graining process that gradually destroys information. We bridge this conceptual divide by establishing a formal mathematical correspondence between hierarchical tree models and the deterministic flows associated with diffusion processes under suitable limiting procedures (see Figure 1). Thus, we demonstrate that rather than being disjoint model classes, they are simply two perspectives of the same underlying generative and discriminative object. More precisely, we first show that any decision tree’s hierarchical coarse-graining of data defines a discrete-time Markov process whose continuous limit is a unique deterministic flow that is described by a Probability Flow Ordinary Differential Equation (Song et al., 2021) (PF-ODE) (Tree → Flow). Conversely, we prove that any suitable diffusion process whilst evolving from data to noise (forward process), induces a canonical hierarchical clustering. The forward process progressively merges the ∗

School of Computation, Information and Technology, Technical University of Munich, Germany

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

modes of the data distribution and the temporal ordering of these modes define a hierarchy isomorphic to a decision tree (Flow → Tree).

Figure 1: A Visual Metaphor for Trees to Flows and back. We connect the discrete structure of a decision tree (left) with the continuous dynamics of a flow process (right). We establish a a formal correspondence, enabling an analysis of boosting via flows, and imbuing generative models with tree-based inductive biases. This Tree↔Flow correspondence reveals a common optimization principle, which we term Global Trajectory Score Matching (GTSM) (§3). We show that both the greedy, stage-wise construction of boosting and the end-to-end training of score-based models can be understood as distinct but related solvers for the GSTM master objective. A valuable consequence of our unified perspective is how it enables novel algorithms that combine the strengths of both frameworks. Leveraging the explicit, adaptive partitioning as a structural prior, we develop two algorithms that excel on tabular data. The first achieves competitive generation quality where standard diffusion models typically struggle. The second successfully distills complete tree hierarchies into neural networks, to our knowledge for the first time. Summary of Contributions. • We establish a formal mathematical correspondence between hierarchical decision trees and the deterministic flows (PF-ODEs) associated with diffusion processes, valid under limiting refinement procedures. • We introduce the Global Trajectory Score Matching (GTSM) framework, a unifying objective, under which we show how both gradient boosting and score-based diffusion training are optimal solvers in their respective discrete and continuous domains. • We introduce T REE F LOW, a new method that conditions continuous flows using tree-based priors. T REE F LOW achieves competitive generation quality on tabular data (highest TSTR accuracy on 3/5 benchmarks, lowest Wasserstein distance on 4/5 benchmarks, and lowest correlation error on 3/5 benchmarks) while being 2X faster. • We propose DSM-T REE, a novel method to distill complete hierarchical decision logic (not just leaf predictions) into neural networks, matching teacher performance within 2% on most benchmarks and exceeding it by 3.7% on the Heart Disease dataset. In Section 2, we establish the Tree-Flow correspondence. Section 3 introduces the unifying GTSM framework and proves the asymptotic optimality of gradient boosting as a discrete solver. We then present our novel algorithms in Section 4 and validate our framework with extensive experiments in Section 5. For a visual guide to our theory and derivations, we refer the reader to the roadmap in Section B.

The Tree–Flow Correspondence

We establish below a mathematical correspondence between decision trees and diffusion models. We first demonstrate that a decision tree defines a continuous-time PF-ODE via hierarchical coarse2

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

graining under limiting assumptions (§ 2.1). Then, we prove that an entropically homogeneous SDE possessing a stationary distribution induces a canonical tree structure via moment-based clustering (§ 2.2). The Tree as a Discrete-Time Markov Process. Definition 2.1 (Decision Tree). A decision tree T of depth D ∈ N on X ⊂ Rd is a nested sequence of partitions ΠD ≺ · · · ≺ Π0 (≺ denotes refinement) with Π0 = {X } such that the elements in any partition Πk , k > 0 form a disjoint cover of X , and ∀k ≥ 1, Ω ∈ Πk =⇒ ∃! Ω′ ∈ Πk−1 s.t. Ω ⊂ Ω′ . This induces a filtration Fk = σ(Πk ) and conditional densities p(x, k) := E[p0 (x) | Fk ] piecewise constant on Πk . By the tower property (Durrett & Durrett, 2019), the transition operator Mk : L1 (X ) → L1 (X ) is p(x, k + 1) = Mk p(x, k) = E[p(x, k) | Fk+1 ].

(1)

Thus, this sequence forms a discrete-time Markov chain tracing a path of monotonically decreasing entropy from the root p(x, 0) = Unif(X ) to the leaves p(x, D) ≈ pdata (x). The Continuous-Time Limit. For typical trees, inhomogeneity (Mk ̸= Mk+1 ) precludes direct limits. We resolve this by introducing a conceptual refinement procedure. Definition 2.2 (Dyadic Refinement). The n-th refinement T (n) is a tree of depth 2n T constructed by recursively inserting intermediate partitions Πt such that for all k ∈ {0, . . . , 2n−1 T − 1}, the new partition satisfies Π n−1 ≺ Π k+1 . As n → ∞, this sequence converges to a continuous≺ Π 2k+1 k n 2

2n−1

time filtration {Ft }t∈[0,T ] where Ft = σ(Πt ) and the density path p(x, t) = E[p0 | Ft ] is C 1 in t. This procedure constructs a smooth interpolating path between the original tree’s partitions, transforming the discrete steps of coarse-graining into an infinitesimally smooth “ramp.” To this end, we recursively insert intermediate partitions. Consider a simple example: a parent node’s region is defined by the split feature A < 10 and its child’s region is feature A < 5. A valid intermediate partition always exists by continuity (e.g., say by splitting at feature A < 7.5). This transforms one large step into two smaller ones. By repeating this bisection process recursively (hence, “dyadic”), we can generate a sequence of hierarchies with exponentially more, infinitesimally small steps. In the limit as the number of refinements n → ∞, this discrete sequence of operators converges to a well-defined continuous-time propagator, as noted below. Theorem 2.3 (Limit under Dyadic Refinement (Informal)). Let {T (n) } be a dyadically refined sequence of decision trees as in Definition 2.2. Under the following assumptions 1. Scale consistency: inserting intermediate splits between existing tree levels does not change the conditional densities at the original levels. 2. Local refinement: newly introduced splits become uniformly finer with refinement, so that the geometric size of each refinement step vanishes in the limit, the associated density path p(x, t) = E[p0 (x) | Ft ] admits a well-defined continuous-time description generated by a (possibly time-dependent) operator Gt that is locally Lipschitz on compact intervals. Moreover, for any compact interval I ⊂ [0, T ], there is a subsequence of refinements along which Gt converges to a time-invariant generator G. Proof. At each refinement level, the conditional densities change only slightly due to the vanishing size of new splits (local refinement), giving a Lipschitz continuous family of generators. By a standard diagonalization argument on compact intervals, one can extract a subsequence along which these generators converge. Time-invariance emerges in the limit because the contribution of each refinement step becomes negligible, so the limiting generator G effectively governs the density path across the entire interval. The full proof with all the details is provided in Theorem C.13. 3

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

From Discrete Steps to a Differential Equations The limit of the refined process from Theorem 2.3 is, by construction, a continuous-path Markov process. The time evolution of its density p(x, t) is described by the Kramer–Moyal expansion (Gardiner, 2004): ∞ i ∂p X (−1)n ∂ n h (n) D (x)p(x, t) , = ∂t n! ∂xn n=1 where D(n) are the moments of the process’s infinitesimal jumps. First, we note an observation about higher moments. Theorem 2.4 (Higher-Order Moments Vanish (Informal)). For the limiting process constructed via dyadic refinement, the contributions of jumps of order three and higher become negligible quickly as the refinement becomes finer. Proof. Intuitively, the claim holds because each refinement redistributes probability only within ever smaller regions, so large or high-order jumps are geometrically suppressed. Indeed, as the dyadic partitions get finer, the size of the regions shrinks exponentially, which forces all moments of order three and above to go to zero. Proposition C.20 in the appendix provides the formal argument. Next, note that Pawula’s theorem (Pawula, 1967) states that if any jump moment D(n>2) is zero, all higher moments must also be zero. But as the jumps become negligible for our case, this implies D(n) (x) = 0 for all n > 2. Hence, the expansion truncates exactly to the Fokker-Planck equation: ∂p ∂ 1 ∂2 = − [D(1) (x)p] + [D(2) (x)p]. ∂t ∂x 2 ∂x2 Further, the underlying coarse-graining operator is purely deterministic (an averaging process), i.e., it introduces no new stochasticity. Therefore, the second-order (diffusion) term must vanish: D(2) (x) ≡ 0. This reduces the Fokker-Planck equation to a first-order Liouville equation, whose characteristic equation is a deterministic ODE. Theorem 2.5 (Trees Induce PF-ODEs (Informal)). Under dyadic refinement, the continuous-time limit of tree-induced coarse-graining is ẋ = v(x, t), a deterministic PF-ODE with velocity v uniquely determined by the partitions. Proof. Please see Section C (Theorem C.21). 2.1

From Flows to Trees

We now establish the reverse mapping: every entropically homogeneous SDE having a stationary distribution induces a tree via its corresponding PF-ODE. Intuitively, as the process evolves from a structured data distribution towards a high-entropy stationary state, distinct local modes of the initial density merge. The hierarchy records this timeline: regions that remain statistically distinguishable for longer are grouped higher. As the flow of entropy is monotonic, this merging is irreversible, ensuring a well-defined tree structure. Formally our argument extends a statistical physics approach (Ramachandran et al., 2025), previously used to analyze the trajectories of generative diffusion models. Definition 2.6 (Entropically Homogeneous Stationary SDE). An SDE dxt = b(xt , t)dt + σ(xt , t)dwt possesses a stationary distribution if it converges to a unique invariant measure p∞ such that limt→∞ ∥pt − p∞ ∥TV = 0. We call the process entropically homogeneous if the differential entropy H(pt ) is a monotonically non-dec(inc)reasing function of t. Definition 2.7 (Initial Clusters). For a data density p0 with K isolated modes, the initial clusters {Ck }K k=1 are the connected components of a super-level set of p0 chosen to yield exactly K components (Section D). Definition 2.8 (Moment-Based Merger Time). For clusters Ci , Cj , the (n, ϵ)-merger time is the first time t at which their n-th order conditional moment tensors become statistically indistinguishable: (n,ϵ) (n) (n) tij = inf t ≥ 0 : Mt (Ci ) − Mt (Cj ) ≤ ϵ , (2) (n)

where Mt (Ck ) = Ex∼pt (·|Ck ) [(x − E[x])⊗n ]. 4

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

These merger times satisfy the following property: Theorem 2.9 (Mergers Form a Hierarchy (Informal)). For an entropically homogeneous SDE, the (n,ϵ) moment-based merger times tij of clusters satisfy a hierarchical structure, and these merger times obey an ultrametric inequality. Proof. Each cluster’s conditional moments evolve continuously and monotonically under the SDE; once two clusters merge (their moments become ϵ-close), they stay merged. For any three clusters, consider the two whose merger occurs last. By continuity and the triangle inequality, the remaining cluster must merge no later than this maximal time, which yields the ultrametric property and the consequent hierarchical structure (Proposition D.14 in the appendix). Consequently, we can obtain a hierarchical clustering. Theorem 2.10 (SDEs Induce Trees (Informal)). An entropically homogeneous SDE, initialized from a distribution with well-separated modes, induces a unique hierarchical clustering via its momentbased merger times. This structure depends only on the SDE’s dynamics and its initialization, hence is fully characterized by the corresponding PF-ODE. Proof. Theorem D.15 in Section D establishes that entropically homogeneous SDEs induce a dendogram by tracking the moment based merger times of data modes of pinit . As the construction process relies only on the marginals, the dendogram is fully characterized by the evolution of the corresponding PF-ODE (Song et al., 2021). We further show that stationarity is a necessary and sufficient condition for the tree to be rooted in Proposition D.18.

Global Trajectory Score Matching

The Tree-Flow equivalence implies that both gradient boosting and diffusion models solve the same underlying problem: learning an optimal trajectory from a simple prior to a complex data distribution. The ultimate goal is to match the ideal path-space measure, a notoriously intractable objective (Lai et al., 2025). Our central insight is that this global problem (under standard regularity conditions) is decomposable, building on a local-to-global consistency principle: a trajectory is globally optimal if and only if its dynamics are correct at every infinitesimal step, for all states along the path. Any “chunk” of the trajectory can be thus seen as a sequence of such infinitesimal steps. We formalize this principle via our Global Trajectory Score Matching (GTSM) framework, which replaces the single, intractable global objective with an infinite sum of local, tractable ones. At any point (x, t), the score function specifies the optimal direction for the next infinitesimal step of the reverse process. By integrating the local score-matching error over states and times, the objective sums the consistency checks over all infinitesimal chunks of every trajectory. This decomposability is what allows disparate algorithms to solve the same fundamental problem. End-to-end training of a single score network attempts to minimize the entire integral at once. In contrast, gradient boosting acts as a greedy optimizer, iteratively adding weak learners that reduce the largest remaining errors across the full GTSM integral. 3.1

The Continuous GTSM Objective

Definition 3.1 (Continuous Global Trajectory Score Matching). For an ideal SDE with law P∗ and scores s∗t (x), and a model sθ (x, t), the CGTSM objective is: Z 1 T 2 LCGTSM (θ) = w(t)Ep∗t sθ (x, t) − s∗t (x) D(t) dt, (3) 2 0 √ where w(t) > 0 is a weighting function and ∥v∥D = v⊤ Dv is the diffusion-induced norm.2 Theorem 3.2 (CGTSM Optimality Implies Path Matching). Achieving zero CGTSM loss for any strictly positive weighting w(t) > 0 is necessary and sufficient for matching the full path-space measures, i.e., Pθ = P∗ . 2

Technically this is a semi-norm unless D is strictly positive definite. For the SDE dXt = b(xt , t) dt + σ(xt , t) dwt , the diffusion tensor is given by D = σσ ⊤ , which is positive definite once one disallows rank deficient σ (Oksendal, 2013).

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Proof sketch. By Girsanov’s theorem (Oksendal, 2013), the KL divergence between path-space measures, DKL (P∗ ∥Pθ ), is an integral of the squared difference between the process drifts. Since the reverse-time drift is a function of the score, this difference reduces to the CGTSM integrand. The loss is zero if and only if the KL divergence is zero. The full proof is in Section F (Corollary F.3). 3.2

Boosting as a CGTSM Solver

Pm Gradient boosting constructs an ensemble Fm = i=1 hi by iteratively adding weak learners (decision trees) that fit the residual error. To map this additive process into our SDE framework, we introduce the net decision tree abstraction. At each step m, the net tree Tm is a single, monolithic hierarchical model whose partition Πm is the common refinement of the partitions of all constituent learners {h1 , . . . , hm }, and whose leaf values are the sum of their predictions. This abstraction transforms the boosting algorithm from a sequence of functional additions into a sequence of structural refinements. We show that the sequence of partitions {Πi }m i=1 induced by the net trees is monotonic and strictly refining (Section E.1.3). We then apply our Tree-to-Flow mapping (Theorem 2.5) to the canonical hierarchy of the net trees {T1 , T2 , . . . Tm }, yielding a series of unique processes3 {S1 , S2 , . . . , Sm }, moving from coarse to fine approximations. The mechanism guiding this trajectory is the minimization of a discrete GTSM objective. Definition 3.3 (Discrete GTSM for Boosting). The discrete GTSM objective is the sum of scorematching losses: m h i X 2 LDGTSM ({hm }) = E(x,y) ∥hi (x) − ri (x)∥ , (4) i=1

where the residual ri (x) = y − Fi (x) is an unbiased estimator of the optimal score update for the underlying process at stage i (Section E, Theorem E.22). We rewrite the problem as a finite-horizon sequential decision problem to showcase optimality. Define the state at step m as the current model’s induced process (Sm ), the action as the choice of the next weak learner (hm+1 ), a deterministic state transition (Sm+1 = T (Sm , hm+1 ), and a stage cost C(Sm , hm+1 ) given by the immediate score-matching error. The total cost to be minimized is the sum of these stage costs, which is precisely the discrete GTSM objective (4). Through this formulation we obtain the following result. Theorem 3.4 (Greedy Boosting is Globally Optimal). In the continuous limit, where the class of weak learners is sufficiently rich to approximate any function, the greedy policy of selecting the weak learner that minimizes the immediate stage cost at each step is the globally optimal policy for the discrete GTSM problem. Proof sketch. The proof proceeds by backward induction on the optimal cost-to-go, or value function Vm (Sm ). The Bellman equation (Bertsekas, 2012) relates the value at stage m to the value at stage m + 1. Due to the additive separability of the GTSM objective and the deterministic state transitions, the optimal action at stage m is simply the one that minimizes the immediate stage cost, independent of the future value function. This proves that the greedy choice is optimal at every stage. The full proof is in Section E.

Algorithmic Instantiations

To operationalize the theoretical unification of decision trees and diffusion processes, we propose two novel algorithms for handling tabular data. These methods leverage the hierarchical partitions of trees to provide structural inductive biases to neural networks, one for generative modeling and the other for discriminative distillation. These algorithms directly approximate the Global Trajectory Score Matching (GTSM) objective and achieve competitive performance while providing substantial computational advantages. 3

For stochastic boosting (Friedman, 2002), Theorem 2.5 reduces to a SDE as the diffusion term is no longer

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

4.1

T REE F LOW: Tree-Conditioned Flow Matching

Recent attempts to apply diffusion models to tabular data, such as TabDDPM (Kotelnikov et al., 2023), achieve strong results but often have high computational costs. TreeFlow addresses this limitation by using decision tree partitions as a conditioning mechanism for Conditional Flow Matching (CFM) (Lipman et al., 2023), achieving competitive generation while being 2X faster. Intuition. T REE F LOW first “maps” the data manifold using a decision tree. It assigns each data point a unique “path encoding,” a vector representing its journey from the root to a leaf. We then train a velocity field vθ to move from noise to data, conditioned on these paths. This approach partitions the generative task as the model learns specialized flows for different regions of data space. At generation time, we can target specific partitions, resulting in samples with state-of-the-art fidelity (lowest Wasserstein distance on 4/5 benchmarks) and significantly faster generation (2× speedup over TabDDPM). By viewing T REE F LOW through the GTSM lens, we provide a rigorous distributional convergence proof (Section H, Corollary H.5), establishing that it captures the true conditional data distribution within each partition. Informally, under a refinement procedure that interpolates between tree partitions, the sequence of conditional densities converges to a deterministic flow uniquely defined by the tree geometry. Algorithm 1 T REE F LOW: Training Phase 1: Input: Dataset D, tree depth D

Learn Data Partitioning Structure 2: Train decision tree T on D to depth D 3: For each xi , compute path encoding: pi = PathEncoder(T , xi )

Train Conditional Flow Matching Model 4: Initialize velocity field vθ (x, t, p, y) 5: for training steps s = 1, . . . , S do 6: Sample batch {(xdata i , pi , yi )}i∈B 7: Sample noise: xnoise ∼ N (0, I) i 8: Sample time: t ∼ Uniform(0, 1) (t)

Interpolate: xi = txdata + (1 − t)xnoise i i ∗ data Target velocity: vi = xi − xnoise i P (t) 11: Loss: L = i∈B ∥vθ (xi , t, pi , yi ) − vi∗ ∥2 12: Update: θ ← θ − η∇θ L 13: end for 14: Output: Trained velocity field model vθ and tree T

9: 10:

Algorithm 2 T REE F LOW: Generation Phase 1: Input: Trained model vθ , tree T , target label ytarget , partition Rtarget

Partition-Targeted Generation 2: Sample reference xref ∼ {xi ∈ D : yi = ytarget , xi ∈ Rtarget } 3: Compute conditioning path: pref = PathEncoder(T , xref ) 4: Initialize: x(0) ∼ N (0, I) 5: for t = 0 to 1 with step size ∆t do 6: x(t+∆t) ← x(t) + vθ (x(t) , t, pref , ytarget ) · ∆t 7: end for 8: Output: Synthetic sample x̃ = x(1)

4.2

DSM-T REE: Distilling Trees via Hierarchical Score Matching

Despite the dominance of deep learning in vision and language, tabular data remains a domain where tree-based ensembles like XGBoost (Chen et al., 2015) and Random Forests (Breiman, 2001) often outperform neural networks (Grinsztajn et al., 2022). Neural networks typically lack the axis-aligned splitting bias and the ability to handle the “jumpy” manifolds characteristic of tabular datasets. Discretized Score Matching for Trees (DSM-T REE) bridges this gap by distilling the 7

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

entire decision trajectory of a decision tree into a neural network which to our knowledge is the first method to supervise networks on complete hierarchical logic rather than just leaf predictions. Intuition. Unlike standard knowledge distillation (Hinton et al., 2015) based on output supervision, DSM-T REE treats every internal split in the tree as a directional guide. By training the network to predict the correct decision at every level j of the hierarchy, the network learns the tree’s coarsegraining flow. This “fluidizes” the discrete logic of the tree into a continuous, differentiable neural function that retains the tree’s superior bias. The training process, detailed in Algorithm 3, closely resembles score matching (Song et al., 2021) by design, as it follows from the GTSM framework. As shown in Section G, DSM-T REE is supported by a finite-sample convergence guarantee (Theorem G.5). Informally, the reasoning proceeds as follows, if the model can match the true decisions at every level then gradient-based training gradually reduces the per-level prediction errors hence as total error reduces asymptotically the model reproduces all paths exactly. The inference process is described in Algorithm 4. Algorithm 3 DSM-T REE: Training Phase 1: Input: Dataset D, tree depth D

Base Tree Generation (Teacher Model) 2: Train oracle O (e.g., Random Forest) on D 3: Generate pseudo-labels: ỹi = O(xi ) 4: Train base decision tree T on {(xi , ỹi )} to depth D

Conditional Score Model Training (Student Model) 5: Initialize network Mθ (x, j) : Rd × {0..D − 1} → ∆1 6: for training steps t = 1, . . . , T do 7: Sample mini-batch {(xi , yi )}i∈B 8: Sample levels {ji }i∈B ∼ Uniform(0, D − 1) ∗ 9: For each (xi , ji ): extract P di = TreeDecision(T , xi , ji ) 10: Compute loss: L = i∈B I[d∗i ̸= ⊥] · ℓCE (Mθ (xi , ji ), d∗i ) 11: Update parameters: θ ← θ − η∇θ L 12: end for 13: Output: Trained model Mθ and teacher tree T

Algorithm 4 DSM-T REE: Inference Phase 1: Input: Trained model Mθ , teacher tree T , test sample x

Neural Network Inference (Mimicking Tree Traversal) 2: Initialize: node ← root(T ) 3: for level j = 0, . . . , D − 1 do 4: if node is a leaf then break 5: end if 6: Predict decision: dˆj = arg max Mθ (x, j) 7: Navigate to next node: node ← childdˆj (node) 8: end for 9: Get prediction from final node: ŷ = Value(node) 10: Output: Final prediction ŷ

Experiments

We conduct experiments with two goals: (1) provide empirical evidence for the Tree-Flow correspondence by showing that diffusion models learn implicit hierarchical structure and show information decay analogous to decision trees, and (2) demonstrate the practical utility of our Global Trajectory Score Matching (GTSM) framework via the T REE F LOW algorithm for generation and the DSMT REE algorithm for discriminative distillation. Additional details, architectures, and full results appear in Section I. 8

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Figure 2: Implicit tree structure discovered from a trained diffusion model on the 4-Corners dataset using the corrected time-domain clustering method. (a) The original labeled data clusters, representing the leaves of the hierarchy at t = 0. (b) The discovered hierarchical structure (dendrogram) obtained by tracking the forward SDE trajectories of each cluster and performing agglomerative clustering. The vertical axis is not rescaled; it directly represents the discovered merger time t. (c) A visualization of the system’s state at the internal node time t = 0.50. The plot shows the particle positions at this intermediate time, generated by solving the learned reverse PF-ODE. The coloring, based on the original labels, shows that the clusters have begun to overlap and lose their distinct structure, as predicted by the hierarchy.

5.1

Verifying the Tree-Flow Equivalence

Implicit Tree Structure in Diffusion Models. To validate our claim from §2.2, we test whether trained diffusion models learn canonical trees. We train a simple MLP on synthetic 2D datasets from scikit-learn (Pedregosa et al., 2011) and discover the learned hierarchy via time-domain agglomerative clustering. For each initial cluster, we simulate forward evolution using an SDE with drift given by the model’s score function, tracking centroids and statistical spread. We identify the earliest time t when cluster pairs become indistinguishable (inter-centroid distance below combined spread), these merger events construct a dendrogram where the vertical axis represents time ( Figure 2(b)). Figure 2 shows original clusters (a), resulting hierarchy (b), and an intermediate PF-ODE state (c), providing empirical evidence that learned dynamics encode discrete hierarchical structure. See Section I.1.1 for further results and additional details. Information-Theoretic Analogy. We next compare information decay in decision trees and diffusion models. For a DecisionTreeClassifier on MNIST (LeCun et al., 1998), we measure weighted-average class entropy at each depth. For forward diffusion, we use the information-theoretic proxy 1/(1 + SNR) . Figure 3 (top) shows both exhibit similar sigmoidal entropy increase. Visual prototypes (bottom) illustrate this, tree prototypes (pixel-wise node averages) blur progressively toward the root, mirroring how diffusion noise destroys digits. See Section I.1.2 for additional datasets. 5.2

Algorithmic Instantiations for Tabular Data

T REE F LOW for Generative Modeling. We evaluate T REE F LOW’s ability to generate high-fidelity synthetic tabular data. We compare it against strong baselines from the Synthetic Data Vault (sdv) library (Patki et al., 2016): GaussianCopula, TVAE, and CTGAN, as well as a competitive diffusion model for tabular data (TabDDPM) (Kotelnikov et al., 2023). We assess performance on four axes: Utility (TSTR Accuracy), Fidelity (Wasserstein Distance), Structure (Correlation Error), and Efficiency (Runtime). The aggregated results over 5 runs, shown in Figure 4 and detailed in Table 3 in the appendix, demonstrate that T REE F LOW achieves competitive performance across multiple dimensions. Specifically, T REE F LOW obtains the highest TSTR accuracy on 3/5 benchmarks (98.1% on Wine, 93.9% on Cancer), the lowest Wasserstein distance on 4/5 benchmarks, and the lowest correlation error on 3/5 benchmarks, all while being 2× faster than TabDDPM. These results highlight the substantial advantage of conditioning the generative flow on the rich structural prior provided by the decision tree’s hierarchical partition. 9

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Figure 3: Comparison of information decay on MNIST. (Top) The normalized entropy of a decision tree (by depth) and a diffusion process (by time) follow near identical trajectories. (Bottom) Visual prototypes show analogous information loss, with tree prototypes averaging over data subsets and diffusion prototypes showing a single instance being noised. (Small Gaussian noise is added to tree prototypes to ease visual distinguishability; zoom for clarity.

DSM-T REE for Discriminative Modeling. We evaluate DSM-T REE’s ability to distill a decision tree into a neural network. Our baseline is a strong single DecisionTreeClassifier (Base Tree) trained on the soft labels from a RandomForestClassifier oracle. DSM-T REE is then trained to replicate the decision path of this Base Tree at every level, as described in §4.1. As shown in Figure 5, DSM-T REE successfully distills tree hierarchies into neural networks, matching teacher performance within 2% on 4/5 datasets and outperforming by 3.7% on Heart Disease. This demonstrates that DSM-T REE can successfully transfer complete hierarchical decision logic (not just leaf predictions) into differentiable neural networks, proving that continuous representations can faithfully capture discrete tree structure.

Summary of Experimental Findings. Our experiments validate the Tree-Flow correspondence, diffusion models and decision trees exhibit similar hierarchical structure and information decay dynamics. Leveraging this insight, our algorithms demonstrate both theoretical depth and practical impact. DSM-T REE distills complete tree hierarchies, matching teachers within 2% on most benchmarks, while T REE F LOW achieves competitive generation quality and fidelity on tabular data with 2X speedup. These results confirm that GTSM enables novel models synthesizing the structural advantages of trees with the dynamics of flows. 10

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Figure 4: T REE F LOW compared against baseline generative models across a suite of tabular benchmarks. We evaluate on four axes: Utility (TSTR Accuracy ↑), Fidelity (Wasserstein Distance ↓), Structure (Correlation Error ↓), and Efficiency (Runtime ↓). T REE F LOW achieves highest TSTR accuracy on 3/5 benchmarks, lowest Wasserstein distance on 4/5 benchmarks, and lowest correlation error on 3/5 benchmarks, while being 2X faster than TabDDPM. Zoom for clarity (or see Figure 15).

Related Work

Our work lies at the intersection of theoretical analyses of diffusion models, the functional interpretation of tree ensembles, and the development of hybrid architectures. Theoretical Understanding of Diffusion Models. Analyses of Diffusion Models (DMs) (Ho et al., 2020; Song et al., 2021) often use tools from statistical physics to study trajectory stability and dynamical regimes (Biroli et al., 2024; Bonnaire et al., 2025; Ramachandran et al., 2025), or from differential geometry to analyze the learned score function in relation to the data manifold (Bortoli, 2023). In contrast, we provide a global, architectural interpretation. We prove that the intractable goal of full path-space matching is equivalent (under standard regularity conditions) to a tractable integral of local score-matching errors (our GTSM objective). This principle of local-to-global consistency demonstrates that correctness at every infinitesimal step suffices for global path-level correctness. The CGTSM objective thus serves as a “master objective” from which various practical training schemes can be analyzed as principled approximations. Different Views of Trees. Existing theoretical frameworks for tree models fall into two camps, deterministic approaches that view boosting as functional gradient descent (Chen et al., 2015; Friedman, 2001), and probabilistic approaches like Mondrian processes that define distributions over random tree structures (Roy & Teh, 2009). Our work offers a novel bridge by proving that a single, fixed, learned hierarchy in the limit can be reduced to a continuous deterministic flow governed by a PF-ODE, which establishes a formal correspondence between tree-based partitioning and continuous dynamics. 11

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Figure 5: Classification accuracy of the DSM-T REE model compared to its teacher (Base Tree). DSM-T REE matches or exceeds teacher performance on 4/5 datasets, with +3.7% improvement on Heart Disease, demonstrating successful knowledge transfer of complete hierarchical structure. Zoom for clarity. A larger version is available in Figure 14 with full results in Table 2. Hybrid and Generative Models for Tabular Data. The integration of trees and neural networks spans from early distillation of tree predictions into neural networks (Hinton et al., 2015) to differentiable soft decision trees (Kontschieder et al., 2015). Our work advances this integration in two complementary directions. First, DSM-T REE distills complete tree hierarchies, not just final predictions, into differentiable neural networks, enabling end-to-end learning while preserving interpretable structure. Second, unlike methods that learn structure implicitly (Kotelnikov et al., 2023; Patki et al., 2016), T REE F LOW explicitly conditions flow-matching on tree-based priors, injecting proven hierarchical partitioning into continuous generative models.

Limitations, Future Work and Conclusion

Our theoretical framework relies on continuous-path refinement processes and smoothness assumptions that may not always apply. While our algorithmic approach is more general, we focus our evaluation focuses on continuous feature spaces to retain alignment between theory and experiments. Extending our theory to handle intrinsic discontinuities, e.g., via Lévy processes (Applebaum, 2009; Cont & Tankov, 2003) or rough-path theory (Hairer & Friz, 2014; Lyons et al., 2007), would formalize broader applicability and remains key future work. An especially promising direction is to combine data-driven, adaptive partitioning abilities of trees with the expressive power of diffusion models, toward building novel foundation models for complex, heterogeneous domains (e.g., tabular and sequential data) where current approaches struggle to capture irregularities arising from multiple modalities. These extensions could build upon the foundational contributions established in this work. We established a formal correspondence between hierarchical decision trees and diffusion process flows, unifying them within the Global Trajectory Score Matching (GTSM) framework. We proved that gradient boosting acts as a globally optimal greedy solver for a discrete formulation of this objective, connecting its stage-wise construction to continuous score-based dynamics. Leveraging this theory, we introduced two novel algorithms. T REE F LOW achieves strong generation quality and fidelity with improved speed by conditioning flows on tree-based priors, while DSM-T REE distills complete tree hierarchies into neural networks for the first time. By bridging these paradigms, our work enables hybrid models that unify discrete hierarchical structures with continuous dynamics.

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Acknowledgments SNR and SS gratefully acknowledge generous support from the Alexander von Humboldt Foundation.

References Aldous, D. Brownian excursions, critical random graphs and the multiplicative coalescent. The Annals of Probability, pp. 812–854, 1997. Anderson, B. D. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313–326, 1982. Applebaum, D. Lévy processes and stochastic calculus. Cambridge university press, 2009. Bartlett, P. L. The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. IEEE Transactions on Information Theory, 44(2):525–536, 1998. Bartlett, P. L., Foster, D. J., and Telgarsky, M. J. Spectrally-normalized margin bounds for neural networks. Advances in Neural Information Processing Systems (NeurIPS), 30, 2017. Bertsekas, D. Dynamic programming and optimal control: Volume I, volume 4. Athena scientific, 2012. Biroli, G., Bonnaire, T., De Bortoli, V., and Mézard, M. Dynamical regimes of diffusion models. Nature Communications, 15(1):9957, 2024. Bonnaire, T., Urfin, R., Biroli, G., and Mézard, M. Why diffusion models don’t memorize: The role of implicit dynamical regularization in training, 2025. URL https://arxiv.org/abs/2505. 17638. Bortoli, V. D. Convergence of denoising diffusion models under the manifold hypothesis, 2023. URL https://arxiv.org/abs/2208.05314. Bottou, L., Curtis, F. E., and Nocedal, J. Optimization methods for large-scale machine learning, volume 60. SIAM, 2018. Breiman, L. Random forests. Machine learning, 45(1):5–32, 2001. Chen, T., He, T., Benesty, M., Khotilovich, V., Tang, Y., Cho, H., Chen, K., Mitchell, R., Cano, I., Zhou, T., et al. Xgboost: extreme gradient boosting. R package version 0.4-2, 1(4):1–4, 2015. Cont, R. and Tankov, P. Financial modelling with jump processes. Chapman and Hall/CRC, 2003. Conway, J. B. A course in functional analysis. Springer, 2019. Dua, D. and Graff, C. Uci machine learning repository. https://archive.ics.uci.edu/ml, 2019. Durrett, R. and Durrett, R. Probability: Theory and Examples. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 2019. ISBN 9781108473682. Friedman, J. H. Greedy function approximation: a gradient boosting machine. Annals of statistics, pp. 1189–1232, 2001. Friedman, J. H. Stochastic gradient boosting. Computational statistics & data analysis, 38(4): 367–378, 2002. Gardiner, C. Handbook of Stochastic Methods for Physics, Chemistry, and the Natural Sciences. Springer complexity. Springer, 2004. Gillespie, D. T. Markov processes: an introduction for physical scientists. Elsevier, 1991. 13

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Grinsztajn, L., Oyallon, E., and Varoquaux, G. Why do tree-based models still outperform deep learning on typical tabular data? Advances in neural information processing systems, 35:507–520, 2022. Hairer, M. and Friz, P. A course on rough paths with an introduction to regularity structures, 2014. Hinton, G., Vinyals, O., and Dean, J. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015. Ho, J., Jain, A., and Abbeel, P. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. Hull, J. J. A database for handwritten text recognition research. IEEE Transactions on pattern analysis and machine intelligence, 16(5):550–554, 1994. Karatzas, I. and Shreve, S. E. Brownian Motion and Stochastic Calculus. Graduate Texts in Mathematics. Springer Science & Business Media, 2012. Karras, T., Aittala, M., Lehtinen, J., Hellsten, J., Aila, T., and Laine, S. Analyzing and improving the training dynamics of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24174–24184, 2024. Kingma, D. and Gao, R. Understanding diffusion objectives as the elbo with simple data augmentation. Advances in Neural Information Processing Systems, 36:65484–65516, 2023. Kingma, D., Salimans, T., Poole, B., and Ho, J. Variational diffusion models. Advances in neural information processing systems, 34:21696–21707, 2021. Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICLR), 2015. Koller, D. and Friedman, N. Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning. The MIT Press, 2009. ISBN 0262013193. Kontschieder, P., Fiterau, M., Criminisi, A., and Bulo, S. R. Deep neural decision forests. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 1467–1475, Santiago, Chile, December 2015. Kotelnikov, A., Baranchuk, D., Rubachev, I., and Babenko, A. Tabddpm: Modelling tabular data with diffusion models. In International conference on machine learning, pp. 17564–17579. PMLR, 2023. Lai, C.-H., Song, Y., Kim, D., Mitsufuji, Y., and Ermon, S. The principles of diffusion models. arXiv preprint arXiv:2510.21890, 2025. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998. Liang, Q., Liu, Z., and Fiete, I. Do diffusion models learn semantically meaningful and efficient representations?, 2024. URL https://arxiv.org/abs/2402.03305. Lipman, Y., Chen, R. T. Q., Ben-Hamu, H., Nickel, M., and Le, M. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2023. Lyons, T. J., Caruana, M., and Lévy, T. Differential Equations Driven by Rough Paths, volume 1908 of Lecture Notes in Mathematics. Springer, Berlin, 2007. ISBN 978-3-540-71284-8. Lecture notes from the École d’Été de Probabilités de Saint-Flour XXXIV – 2004. Mohri, M., Rostamizadeh, A., and Talwalkar, A. Foundations of machine learning. MIT Press, 2nd edition, 2018. Murtagh, F. Symmetry in data mining and analysis: a unifying view based on hierarchy. Proceedings of the Steklov Institute of Mathematics, 265(1):177–198, 2009. 14

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Oksendal, B. Stochastic differential equations: an introduction with applications. Springer Science & Business Media, 2013. Paszke, A., Gross, S., Massa, F., et al. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems, volume 32, 2019. Patki, N., Wedge, R., and Veeramachaneni, K. The synthetic data vault. In IEEE International Conference on Data Science and Advanced Analytics (DSAA), pp. 399–410, Oct 2016. doi: 10.1109/DSAA.2016.49. Pawula, R. F. Approximation of the linear boltzmann equation by the fokker-planck equation. Phys. Rev., 162:186–188, Oct 1967. doi: 10.1103/PhysRev.162.186. URL https://link.aps.org/ doi/10.1103/PhysRev.162.186. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011. Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F., Bengio, Y., and Courville, A. On the spectral bias of neural networks. In Proceedings of the 36th International Conference on Machine Learning, pp. 5301–5310, 2019. Ramachandran, S. N., Lal, M. K., and Sra, S. Cross-fluctuation phase transitions reveal sampling dynamics in diffusion models. Advances in Neural Information Processing Systems, 2025. Roy, D. M. and Teh, Y. W. The mondrian process. In Advances in Neural Information Processing Systems 21, pp. 1377–1384. Curran Associates, Inc., 2009. Rudin, W. Principles of Mathematical Analysis. International Series in Pure and Applied Mathematics. McGraw-Hill, 3rd ed. edition, 1976. Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=PxTIG12RRHS. Song, Y., Dhariwal, P., Chen, M., and Sutskever, I. Consistency models. In International Conference on Machine Learning, pp. 32211–32252. PMLR, 2023. van der Vaart, A. W. and Wellner, J. A. Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, New York, 1996. Vapnik, V. N. The nature of statistical learning theory. Springer Science & Business Media, 1999. Vershynin, R. High-dimensional probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018. Villani, C. et al. Optimal transport: old and new, volume 338. Springer, 2008. Vincent, P. A connection between score matching and denoising autoencoders. Neural computation, 23(7):1661–1674, 2011. Xiao, H., Rasul, K., and Vollgraf, R. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017.

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Contents 1

Introduction

2 The Tree–Flow Correspondence 2.1 3

From Flows to Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Global Trajectory Score Matching

3.1

The Continuous GTSM Objective . . . . . . . . . . . . . . . . . . . . . . . . . .

3.2

Boosting as a CGTSM Solver . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4 Algorithmic Instantiations

4.1

T REE F LOW: Tree-Conditioned Flow Matching . . . . . . . . . . . . . . . . . . .

4.2

DSM-T REE: Distilling Trees via Hierarchical Score Matching . . . . . . . . . . .

Experiments

5.1

Verifying the Tree-Flow Equivalence . . . . . . . . . . . . . . . . . . . . . . . . .

5.2

Algorithmic Instantiations for Tabular Data . . . . . . . . . . . . . . . . . . . . .

Related Work

Limitations, Future Work and Conclusion

A Notation

B A Visual Roadmap of the Theoretical Derivations

C Detailed Derivation of the Continuous-Time Flow

C.1 Probabilistic Formulation and Markov Process Foundations . . . . . . . . . . . . . C.1.1

Foundations of Markov Processes . . . . . . . . . . . . . . . . . . . . . .

C.2 Formalizing the Tree Hierarchy as a Sequence of Probabilistic States . . . . . . . .

C.3 The Tree Hierarchy as a Homogeneous Markov Process . . . . . . . . . . . . . . .

C.3.1

Emergence of Homogeneity via Subsequential Limits . . . . . . . . . . . .

C.4 The Continuous-Time Limit and its Governing Equations . . . . . . . . . . . . . .

C.4.1

Truncation of the Kramers-Moyal Expansion . . . . . . . . . . . . . . . .

C.4.2

The Fokker-Planck Equation and Equivalent SDE . . . . . . . . . . . . . .

D The Implicit Hierarchical Structure of Continuous-Time Diffusion

D.1 The General Construction: From Homogeneous SDEs to Dendrograms . . . . . . .

D.1.1 Coarse-Graining as a Flow of Entropy . . . . . . . . . . . . . . . . . . . .

D.1.2 Discretizing the Flow via the Characteristic Function . . . . . . . . . . . .

D.1.3 Main Construction Theorem . . . . . . . . . . . . . . . . . . . . . . . . .

D.2 The Role of Stationarity in Defining the Hierarchy’s Structure . . . . . . . . . . .

D.3 The Condition for a Single, Rooted Decision Tree . . . . . . . . . . . . . . . . . .

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

E Derivation of the Relationship between Boosting and Score Matching E.1 Formalizing Gradient Boosting and the Net Decision Tree . . . . . . . . . . . . . .

36 36

E.1.1

Gradient Boosting as Functional Gradient Descent . . . . . . . . . . . . .

E.1.2

The Net Decision Tree: A Monolithic Abstraction . . . . . . . . . . . . . .

E.1.3

Monotonic Structural Refinement . . . . . . . . . . . . . . . . . . . . . .

E.1.4

The Canonical Hierarchy and the Net Decision Tree . . . . . . . . . . . .

E.2 The Boosting Process as a Trajectory in SDE Space . . . . . . . . . . . . . . . . .

E.2.1

Formalizing Temporal Coarseness via Tail Equivalence . . . . . . . . . . .

E.2.2

The Boosting Process as a Coarse-to-Fine Trajectory . . . . . . . . . . . .

E.3 Score Matching in the Space of SDEs: The Local Update Rule . . . . . . . . . . .

E.3.1

Formal Definitions of Score Matching . . . . . . . . . . . . . . . . . . . .

E.3.2

The Duality of Supervision in Trajectory Refinement . . . . . . . . . . . .

E.3.3

The Boosting Update as an Optimal Score Matching Step . . . . . . . . . .

E.3.4

Connecting the Meta-Score to the SDE Score Error . . . . . . . . . . . . .

E.4 Global Optimality of the Greedy Trajectory . . . . . . . . . . . . . . . . . . . . .

E.4.1

The Global Trajectory Score Matching (GTSM) Objective . . . . . . . . .

E.4.2

Provable Optimality via the Bellman Principle . . . . . . . . . . . . . . .

F Implications of the GTSM Framework: A Unifying View of Score-Based Objectives

F.1

The Continuous GTSM Objective and Its Equivalence to Path Matching . . . . . .

F.2

Deriving Standard Training Objectives as Special Cases of the CGTSM . . . . . .

G DSM-T REE: Discretized Score Matching for Decision Trees G.1 The Tree Distillation Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . .

G.2 Algorithm Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

G.3 Formal Problem Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

G.4 Connection to CGTSM Framework . . . . . . . . . . . . . . . . . . . . . . . . . .

G.5 Convergence Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

G.6 Computational Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

H T REE F LOW: Conditional Flow Matching with Tree-Structured Paths

H.1 Algorithm Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

H.2 Convergence Analysis for T REE F LOW . . . . . . . . . . . . . . . . . . . . . . . .

H.3 Computational Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

H.4 Unified View: DSM-T REE and T REE F LOW as GTSM Instantiations . . . . . . . .

Experimental Results

I.1

Details for Verifying the Equivalence (Experiments 1 & 2) . . . . . . . . . . . . .

I.1.1

Implicit Tree Structure Discovery (Experiment 1) . . . . . . . . . . . . . .

I.1.2

Information Decay Analysis (Experiment 2) . . . . . . . . . . . . . . . . .

Details for Algorithmic Instantiations (Experiments 3 & 4) . . . . . . . . . . . . .

I.2

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

I.2.1

DSM-T REE (Experiment 3) . . . . . . . . . . . . . . . . . . . . . . . . .

I.2.2

T REE F LOW (Experiment 4) . . . . . . . . . . . . . . . . . . . . . . . . .

Notation

The following table provides a summary of the key mathematical notations used throughout the theoretical appendices. Table 1: Summary of Notation Notation

Meaning

Location

Part I: Tree and Discrete Process Notation Πk Fk p(x, k) Mk hm Fm Tm rim

The partition of the feature space at level k of a tree. The sigma-algebra generated by the partition Πk . The probability density of the system at discrete level/time k. The discrete coarse-graining operator that maps p(x, k) to p(x, k + 1). A weak learner (decision tree) added at step m of gradient boosting. The ensemble model after m steps of boosting (Fm = P m i=1 ηhi ). The ”net decision tree,” a single tree equivalent to the ensemble Fm . The pseudo-residual for sample i at boosting step m.

Section C Section C Eq. 5 Section C Algorithm 5 Algorithm 5 Section E.1.4 Algorithm 5

Part II: Continuous Process and SDE Notation dXt f (x, t), b(x, t) g(x, t), σ(x, t) p(x, t) D(n) Gt P, P∗ , Pθ Kt st (x)

Infinitesimal change in the state of a stochastic process. Drift vector field of an SDE or ODE. Diffusion tensor/matrix field of an SDE.

Theorem C.21 Theorem C.21 Theorem C.21

The probability density of the system at continuous time t. The n-th propagator moment of a Markov process (drift, diffusion, etc.). The infinitesimal generator of a continuous-time process. Probability measures (laws) on the space of continuous paths. Equivalence class of SDEs that are t-tail-equivalent. The score function, ∇x log p(x, t).

Equation 23 Equation 23 Theorem C.13 Corollary F.3 Section E Section E.3

Part III: Flow-to-Tree Construction Notation Ck (n)

Mt (Ck ) (n,ϵ)

tij

An initial cluster of data, typically corresponding to a mode of p0 (x). The n-th order conditional expected moment tensor for cluster Ck at time t. The moment-based merger time for clusters Ci and Cj .

Assumption D.6 Section D Section D

Part IV: GTSM and Algorithmic Notation LCGTSM LDGTSM Mθ (x, j) d∗ (x, j)

The Continuous Global Trajectory Score Matching loss. The Discrete Global Trajectory Score Matching loss. The conditional neural network in DSM-T REE that predicts the decision at level j. The ground-truth branching decision from the teacher tree at level j.

Equation 3 Equation 68 Algorithm 6 Equation 73

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Table 1: (continued) Notation

Meaning

Location

vθ (x, t, p, y) pi

The conditional velocity field network in T REE F LOW. The path encoding for sample xi , representing its path through a tree.

Algorithm 7 Algorithm 7

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

A Visual Roadmap of the Theoretical Derivations

To aid the reader in navigating the dense mathematical arguments presented in the appendix, this section provides a series of visual roadmaps. Each of the following major theoretical sections is accompanied by a flowchart that deconstructs the core argument into its primary logical steps. These diagrams serve as a high-level visual companion to the formal proofs, allowing the reader to grasp the overall structure of a derivation at a glance before delving into the technical details. Each box in a flowchart states the purpose of the conceptual step and provides a link to the corresponding theorem, definition, or subsection for easy reference. We begin with the flowchart for the Tree-to-Flow mapping, which details the argument presented in Section C.

Tree Hierarchy (Section C) A static structure is seen as discrete states.

Kramer-Moyal Expansion (Equation 23) Describes infinitesimal evolution of the density.

Discrete Process (Proposition C.9) States linked by coarsegraining operators Mk .

Dyadic Refinement (Theorem C.13) Handles inhomogeneity by constructing a smooth path.

Fokker-Planck Eq. (Theorem C.19) Pawula’s Thm. (Theorem C.18) truncates the expansion due to path continuity.

Deterministic Property Coarse-graining is deterministic, so diffusion term vanishes (D(2) ≡ 0).

Continuous-Path Limit (Proposition C.20) The refined process has continuous trajectories.

Final Result: PF-ODE (Theorem C.21) The equation reduces to a deterministic ODE.

Figure 6: Logical flowchart of the Tree-to-Flow derivation detailed in Section C. Each box explains its purpose and provides a reference to the relevant section, showing how the static structure of a tree is mapped to a continuous-time Probability Flow ODE.

Having established the Tree-to-Flow mapping, we now present the flowchart for the reverse direction: Flow-to-Tree. This diagram details the argument from Section D, showing how the continuous dynamics of a diffusion process are used to construct a canonical, discrete hierarchical tree, thus completing the bijection.

Start: SDE & Data Density (Section D) An SDE is applied to a structured data density p0 (x).

Define Initial Clusters (Assumption D.6) The modes of p0 define the initial ”leaves” of the tree.

Define Merger Criterion (Proposition D.13) Clusters merge when their conditional moments become indistinguishable.

Establish Ultrametric Property (Proposition D.14) Proves that merger times form a valid hierarchy (a dendrogram).

Construct Dendrogram (Theorem D.15) Use temporal agglomerative clustering on merger times to build the hierarchy.

Impose Stationarity Conditions (Proposition D.18) Ensures the hierarchy is finite and all branches eventually merge.

Require Max Entropy (Theorem D.20) A unique, max-entropy stationary state (e.g., Gaussian) defines a single root.

Final Result: Canonical Tree (Theorem 2.10) The process yields a single, rooted tree equivalent to a decision tree.

Figure 7: Logical flowchart of the Flow-to-Tree derivation detailed in Section D.

Next, we provide the flowchart for the derivation that connects the classical gradient boosting algorithm to our framework. This diagram details the argument from Section E, showing how asymptotically boosting is rigorously derived as a globally optimal solver for a discrete formulation of the GTSM objective. 20

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Create Trajectory of Flows (Theorem C.21, Theorem E.5) The sequence of net trees {Tm } is mapped to a coarseto-fine trajectory of flows {Sm }.

Start: Gradient Boosting (Algorithm 5) The classical stage-wise algorithm that adds weak learners to fit residuals.

Net Decision Tree Abstraction (Section E.1.4) Formalizes the ensemble Fm as a single, evolving tree Tm with a monotonically refining partition Πm .

Analyze the Update Rule (Theorem E.22) Prove that fitting the pseudo-residual is mathematically equivalent to a local score-matching update.

Formulate DGTSM Objective (Theorem 68) The sum of the local score-matching errors defines the global objective for the entire trajectory.

Final Result: Global Optimality (Equation 3.4) Frame as a sequential problem and use the Bellman principle to prove the greedy algorithm (boosting) is the globally optimal solver for DGTSM.

Figure 8: Logical flowchart of the derivation showing Gradient Boosting is an optimal GTSM solver, as detailed in Section E.

Here we present the flowchart for Section F. This section discusses how the Continuous GTSM (CGTSM) serves as a central ”master objective” from which many practical training objectives can be derived as special cases or principled approximations.

Simple Diffusion Loss ((Ho et al., 2020)) A Monte Carlo estimator of CGTSM with uniform weighting (w(t) = 1).

Discrete GTSM (Boosting) ((69)) A discretization of the integral, solved greedily.

tim

d ate

dis

cre

tiz ed

Justification via Girsanov’s Thm. (Corr F.3) Proves minimizing CGTSM is equivalent to full path-space matching.

as is estimated by

justifies

Weighted Diffusion Loss ((Kingma et al., 2021)) A Monte Carlo estimator with non-uniform weighting w(t) = λ(t).

Continuous GTSM (Theorem 3.1) The ”Master Objective” is

ox im

ate

db y

Consistency Models ((Song et al., 2023)) An approximation that prioritizes trajectory smoothness over pointwise accuracy.

Figure 9: Flowchart for the implications of the GTSM framework (Section E).

The following two flowcharts illustrate how our proposed algorithms, DSM-T REE and T REE F LOW, are derived as concrete instantiations of the GTSM framework, as detailed in Section G and Section H.

Problem: Tree Distillation (Section G) Learn a neural network to mimic a tree’s full decision process.

Equate Score to Decision The discrete ”score” guiding the trajectory at level j is the ground-truth binary decision d∗ (x, j).

Apply DGTSM (Theorem G.3) Discretize time to tree levels j and match the ”score” at each level.

Result: DSMT REE Objective (Equation 73) The objective becomes a sum of cross-entropy losses over all tree levels.

Figure 10: Flowchart for the derivation of DSM-T REE (Section G). 21

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Problem: Generative Modeling for Tabular Data (Section H)

Condition on Tree Paths The tree’s path encoding p is used as a conditional input to the velocity field vθ (x, t, p).

Apply CGTSM (Theorem H.10) Use Flow Matching, which is equivalent to score matching, as the loss.

Result: T REE F LOW Objective (Theorem H.10) The objective becomes a path-conditioned score-matching loss, which is CGTSM with a tree-induced weighting.

Figure 11: Flowchart for the derivation of T REE F LOW (Section H). Taken together, these flowcharts provide a complete visual summary of the main theoretical contributions of this paper. These diagrams are intended to serve as a conceptual scaffold for the reader. The subsequent sections will now present the full, rigorous mathematical derivations and proofs that underpin each step in these roadmaps.

Detailed Derivation of the Continuous-Time Flow

This appendix provides the formal proofs and derivations for the results presented in Section 2. C.1

Probabilistic Formulation and Markov Process Foundations

We begin by formalizing the probabilistic objects and the foundational concepts of Markov processes that underpin our derivation. Definition C.1 (Partition). A partition Π of a feature space X ⊆ Rd is a collection of non-empty, mutually disjoint subsets {R1 , . . . , Rm } whose union is X . Definition C.2 (Joint Density). Given a data density pdata (x) and a partition Πk = {R1 , . . . , Rm }, the joint probability density p(x, k) is defined as: m X p(x, k) = p(x|X ∈ Ri )P(X ∈ Ri ), (5) i=1

R where P(X ∈ Ri ) = Ri pdata (x)dx. This formulation is common in probabilistic graphical models (Koller & Friedman, 2009) where it is used to encode latent variables. Here, k is a discrete index labeling the partition level (not a random variable). The notation p(x, k) represents the density of feature x conditional on being at partition level k. The sum on the right-hand side is over the m regions of partition Πk . C.1.1

Foundations of Markov Processes

Definition C.3 (Markov Property). A stochastic process X(t) satisfies the Markov property if, for any sequence of times t0 < t1 < · · · < tn , the conditional probability density of the future state depends only on the present state: p(xn , tn |xn−1 , tn−1 ; . . . ; x0 , t0 ) = p(xn , tn |xn−1 , tn−1 ). (6) In essence, the future is conditionally independent of the past, given the present. A process that satisfies this property is called a Markov Process. Theorem C.4 (Chapman-Kolmogorov Equation). (Gardiner, 2004; Gillespie, 1991) For a Markov process, the transition probability density must satisfy the following consistency condition for any intermediate time t′ such that t0 < t′ < t: Z p(x, t|x0 , t0 ) = dx′ p(x, t|x′ , t′ )p(x′ , t′ |x0 , t0 ). (7) This equation is the starting point for deriving the differential form of the process’s evolution. Definition C.5 (Homogeneous Markov Process). A Markov process is homogeneous (or timehomogeneous) if its transition probability depends only on the time difference ∆t = t′ − t, and not on the absolute times t and t′ : p(x′ , t′ |x, t) = p(x′ , t′ − t|x, 0). (8) In the context of our discrete process, this is equivalent to the transition operator Mk being independent of the step k. 22

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Definition C.6 (Continuous-Path Markov Process). A Markov process is said to have continuous sample paths if the probability of any finite jump or displacement in an infinitesimal time interval is zero. Formally, for any ϵ > 0: Z 1 lim p(x′ , t + ∆t|x, t)dx′ = 0. (9) ∆t→0 ∆t |x′ −x|>ϵ This condition is the crucial requirement for showing that the continuous time limit of our decision tree reduces to a drift-diffusion SDE. C.2

Formalizing the Tree Hierarchy as a Sequence of Probabilistic States

Our first step is to formalize how the static, hierarchical structure of a trained decision tree defines a discrete trajectory through the space of probability distributions. We will show that the tree’s levels induce a sequence of partitions, and that each partition defines a specific probabilistic state, starting from a low-entropy data model and ending at a maximum-entropy uniform distribution. Definition C.7 (Partitions Induced by Tree Levels). Let a decision tree of depth T have its leaves at level k = 0 and its root at level k = T . The levels induce a sequence of nested partitions {Πk }Tk=0 , where Π0 is the fine-grained leaf partition and ΠT = {X } is the trivial root partition. Each partition Πk is a strict coarsening of Πk−1 . With this sequence of partitions established, we now map each partition Πk to a probabilistic state, p(x, k) as defined in Equation 5. This mapping provides the state space for our process: the sequence of densities {p(x, k)}Tk=0 . This sequence represents a trajectory of information loss, with well-defined start and end points: • The initial state, p(x, 0), corresponds to the leaf partition Π0 . It is a highly structured, low-entropy model of the true data density. • The final state, p(x, T ), corresponds to the root partition ΠT = {X }. Applying the state density definition to this trivial partition (where m = 1 and R1 = X ) yields the uniform distribution over the feature space: 1 p(x, T ) = U(X ) ≜ I(x ∈ X ). (10) Vol(X ) This is the distribution of maximum entropy for a random variable supported on X . The sequence of states {p(x, k)}Tk=0 thus represents a trajectory of systematically increasing entropy, where structural information is destroyed at each step. C.3

The Tree Hierarchy as a Homogeneous Markov Process

We now formalize the transition between these states and prove that the process is a homogeneous Markov chain. The transition from a finer density p(x, k − 1) to a coarser one p(x, k) is an act of information loss. The operator Mk governing this transition must therefore satisfy the following fundamental axioms, 1. Axiom 1: Measurability (Information Erasure). The output density p(x, k) must not contain information about the boundaries that were erased in the transition from Πk−1 . This means p(x, k) must be measurable with respect to the coarser sigma-algebra Fk generated by the partition Πk . 2. Axiom 2: Partial Averages (Probability Conservation). The probability mass must be conserved within every region of the new partition. This property must extend to all sets A ∈ Fk , such that: Z Z p(x, k)dx = p(x, k − 1)dx ∀A ∈ Fk . (11) A

A fundamental theorem of measure theory states that for a given function p(x, k − 1) and a sigmaalgebra Fk , there exists a unique (up to a set of measure zero) function that satisfies these two axioms. This function is, by definition, the conditional expectation of p(x, k − 1) with respect to Fk (Durrett & Durrett, 2019, Ch. 5). Therefore, the transition operator is uniquely determined. 23

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Definition C.8 (Coarse-Graining Operator). The transition operator Mk for the tree hierarchy is the conditional expectation with respect to the coarser sigma-algebra Fk : p(x, k) = Mk (p(x, k − 1)) ≜ E [p(x, k − 1)|Fk ] .

(12)

With this uniquely identified operator, we can now prove the process is a Markov chain. Proposition C.9 (The Tree Hierarchy Forms a Markov Chain). The sequence of densities {p(x, k)}Tk=0 induced by a decision tree’s partitions forms a Markov chain. Proof. The state p(x, k) is a deterministic function of only the preceding state p(x, k − 1) via the uniquely defined operator Mk . The process is therefore memoryless and satisfies the Markov property. C.3.1

Emergence of Homogeneity via Subsequential Limits

A crucial property for deriving a tractable drift-diffusion process is the homogeneity of the underlying Markov process, requiring that the transition operator is independent of the step index, i.e., Mk = M. However, for a decision tree trained on finite data, this assumption is violated: the geometric transformation from partition Πk−1 to Πk generally differs from that between Πk and Πk+1 , making the discrete process inherently inhomogeneous with Mk ̸= Mk′ . We resolve this by employing a compactification argument from functional analysis. Rather than imposing homogeneity as an assumption, we prove that a form of local homogeneity emerges in the continuous-time limit through a careful refinement procedure, analogous to the construction of Riemann or Lebesgue integrals (Rudin, 1976) via increasingly fine partitions. Definition C.10 (Dyadic Refinement Sequence). Let T (0) denote the original tree with T + 1 levels and partitions {Π0 , . . . , ΠT }. For n ∈ N, the n-th dyadic refinement, T (n) , is a tree with 2n T levels constructed recursively: • T (1) inserts an intermediate partition Π′k+1/2 between each consecutive pair (Πk , Πk+1 ), where Πk ≺ Π′k+1/2 ≺ Πk+1 (strict refinement ordering). • T (n+1) applies the same procedure to T (n) . This produces a sequence of trees {T (n) }∞ n=0 with progressively finer temporal resolution. (n)

T −1 For the n-th refinement, let {Mk }2k=0 denote the corresponding sequence of coarse-graining operators. The critical observation is that as we refine the hierarchy, the step size in operator space decreases.

Proposition C.11 (Operator Convergence). Under mild regularity conditions on the partition refinement scheme, for any fixed time interval [t1 , t2 ] ⊂ [0, T ], the cumulative operator product converges in the strong operator topology: (n)

(n)

lim Mk2 ◦ · · · ◦ Mk1 = Pt2 ←t1 ,

n→∞

where ki = ⌊2n ti ⌋ and Pt2 ←t1 is a continuous-time propagator. The key insight is that we do not require uniform convergence over the entire time interval. Instead, we employ a subsequential compactness argument. Definition C.12 (Local Lipschitz Structure). A family of operators {Gt }t∈[0,T ] (where Gt is the infinitesimal generator) has local Lipschitz structure if for any compact subinterval I ⊂ [0, T ], there exists LI > 0 such that: ∥Gt − Gs ∥op ≤ LI |t − s| ∀t, s ∈ I. Crucially, LI may depend on I and may grow unboundedly as I expands. Theorem C.13 (Emergence of Time-Invariant Generator via Subsequential Limits). Let {T (n) } be a dyadic refinement sequence satisfying: 24

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

(n)

1. Consistent convergence: For any t1 < t2 , the sequence of propagators {Pt2 ←t1 } converges to a limit Pt2 ←t1 . 2. Bounded intermediate complexity: The ”geometric complexity” added at refinement step n, measured by a suitable metric on partitions, is O(2−n ). Then: (i) The limiting process has a generator Gt with local Lipschitz structure on any compact I ⊂ [0, T ]. (ii) For any sequence of compact intervals Im ↑ [0, T ], there exists a subsequence of refinements such that the corresponding generators converge to a time-invariant generator G on Im . Proof. Part (i): For refinement T (n) , the generator at discrete time k/2n is approximately: (n)

(n)

≈ 2n (Mk − Id).

(13)

Fix a compact interval I = [a, b] ⊂ [0, T ] and consider times t, s ∈ I with corresponding indices kt = ⌊2n t⌋, ks = ⌊2n s⌋. The generator difference satisfies: (n)

(n)

∥Gkt − Gks ∥op ≤ 2n ∥Mkt − Mks ∥op

(14) −n

(15) (16)

Taking n → ∞, we obtain ∥Gt − Gs ∥op ≤ Cgeom |t − s| on I. Part (ii): We employ a compactness argument combined with explicit perturbation bounds. Step S 1: Diagonalization. Consider a nested sequence of compact intervals I1 ⊂ I2 ⊂ · · · with m Im = [0, T ]. (n)

On I1 , the generators {Gt }t∈I1 form a uniformly bounded, equicontinuous family (by local Lipschitz). By the Arzelà-Ascoli theorem (Conway, 2019), there exists a subsequence {n1,j } such that (n ) (1) Gt 1,j → G̃t uniformly on I1 . On I2 , restrict to the subsequence {n1,j } and apply Arzelà-Ascoli again to extract a further subse(2) (2) (1) quence {n2,j } converging to G̃t on I2 . By uniqueness of limits, G̃t |I1 = G̃t . Continue this diagonalization procedure for all Im . The diagonal subsequence {nj,j } converges to a generator Gt on each Im , which is locally Lipschitz on each compact subinterval. Step 2: Quantifying the Perturbation. Define the time-dependent perturbation operator: Rt := Gt − Ḡ, R 1 T

where Ḡ := T

(17)

Gt dt is the time-averaged generator.

By the bounded intermediate complexity assumption, each refinement adds geometric structure with complexity O(2−n ). For a level-n refinement, the variation in the generator over time can be bounded: ∞ X ∥Rt ∥op ≤ Cvar · 2−k = Cvar · 2−n+1 . (18) k=n

Step 3: Propagator Error Bound. The deviation of the time-inhomogeneous propagator from the time-averaged propagator satisfies (by Grönwall’s inequality): Rt

e 0 Gs ds − etḠ op

≤ t · eCt · sup ∥Rs ∥op ≤ t · eCt · Cvar · 2−n+1 .

(19)

s∈[0,t]

For any fixed time horizon T and tolerance δ > 0, choose n large enough such that: T · eCT · Cvar · 2−n+1 < δ. 25

(20)

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Step 4: Effective Time-Invariance. We now make the key definitional choice: we define the effective time-invariant generator as: Geff := lim Ḡ (n) , (21) n→∞

where Ḡ

(n)

is the time-averaged generator at refinement level n.

By the dominated convergence theorem and the O(2−n ) complexity bound, this limit exists and satisfies: ∥Geff − Gt ∥op ≤ C(t) · 2−n (22) for some continuous function C(t) on compact subsets of [0, T ]. Step 5: Conclusion. The limiting process has propagator etGeff that approximates the true timeinhomogeneous propagator with error O(2−n ). For practical purposes, this effective generator governs the continuous-time limit, and the time-dependence is a higher-order correction term.

Remark C.14 (Interpretation of Time-Invariance). The time-invariance established here is not exact but rather holds in the sense that: the limiting process is governed by a time-invariant generator Geff up to corrections that vanish as O(2−n ) in the refinement parameter. For balanced trees where the partition complexity is uniformly distributed across levels, the coefficient Cvar is small, making this approximation highly accurate. The framework naturally extends to time-dependent generators Gt for imbalanced trees, with the time-dependence quantified by the partition complexity profile. Remark C.15 (Practical Implications). For “balanced” decision trees where the partition complexity increases roughly uniformly across levels, the perturbation ∥Rt ∥op is empirically small, making the time-invariant approximation highly accurate. For highly imbalanced trees, time-dependent coefficients emerge naturally from this framework, but with guaranteed smoothness properties inherited from the local Lipschitz structure. C.4

The Continuous-Time Limit and its Governing Equations

We now derive the PDE governing the process in the continuous-time limit following the application of Theorem C.13 Definition C.16 (Propagator Density Function). For a continuous-time process X(t), the propagator density function W (ξ|∆t; x, t) is the probability density of the displacement Ξ = X(t + ∆t) − X(t), given X(t) = x. It is related to the transition probability by P (x + ξ, t + ∆t|x, t) = W (ξ|∆t; x, t). Theorem C.17 (Kramers-Moyal Expansion). The time evolution of the Markov state density function P (x, t) is given by: ∞ d i h ∂P (x, t) X (−1)n X ∂n (n) = Di1 ...in (x, t)P (x, t) , ∂t n! i ,...,i =1 ∂xi1 . . . ∂xin n=1 1

(n)

1 where Di1 ...in (x, t) ≜ lim∆t→0 ∆t tions.

(23)

dξ ξi1 . . . ξin W (ξ|∆t; x, t) are the propagator moment func-

Proof. The derivation starts from the Chapman-Kolmogorov equation (Gillespie, 1991). After a change of variables to the displacement ξ = x − x′ , we have: Z P (x, t + ∆t) = dξ W (ξ|∆t; x − ξ, t)P (x − ξ, t). (24) We perform a Taylor expansion of P (x − ξ, t) around x. Substituting this series into the integral gives:     Z ∞ n n X n X Y (−1) ∂ P (x, t)   . (25) P (x, t + ∆t) = dξ W (ξ|∆t; x − ξ, t)  ξij  n! i ,...,i ∂xi1 . . . ∂xin n=0 j=1 1

Approximating W (. . . ; x − ξ, t) ≈ W (. . . ; x, t), expanding the left-hand side for small ∆t, interchanging integration and summation, and identifying the propagator moments yields the result. 26

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

C.4.1

Truncation of the Kramers-Moyal Expansion

The crucial step is to prove that the infinite-order Kramers-Moyal expansion truncates exactly after the second term. This is not an approximation but an exact result. The argument proceeds in three steps: first, we state Pawula’s theorem, which constrains the structure of the expansion. Second, we prove a general theorem linking path continuity to vanishing higher-order moments. Finally, we prove that our specific coarse-graining process has the required path continuity. First, we state the general theorem governing the structure of the expansion. Theorem C.18 (Pawula’s Theorem). The sequence of propagator moments D(n) for any Markov process has only two possibilities: 1. The series terminates after the second term, i.e., D(n) (x, t) ≡ 0 for all n > 2. 2. The series does not terminate, and all even-ordered moments D(2m) for m ≥ 1 are strictly positive. The expansion cannot terminate at any finite order greater than two. Proof. The proof demonstrates that if any moment D(n) for n > 2 is non-zero, then moments of arbitrarily high order must also be non-zero. For clarity, we present the proof in one dimension; the argument extends to the multidimensional case. The core of the proof is the Cauchy-Schwarz inequality for integrals, states for any R R which R that 2 2 2 two square-integrable functions f (ξ) and g(ξ), ( f (ξ)g(ξ)dξ) ≤ ( f (ξ) dξ)( g(ξ) dξ). Let p p us choose f (ξ) = ξ n W (ξ) and g(ξ) = ξ m W (ξ), where W (ξ) is the propagator density. Substituting these into the inequality gives: Z 2 Z Z ξ n+m W (ξ)dξ ≤ ξ 2n W (ξ)dξ ξ 2m W (ξ)dξ . (26) R Let µk (∆t) = ξ k W (ξ|∆t)dξ be the k-th moment of the propagator. The inequality is (µn+m )2 ≤ µ2n µ2m . Since the propagator moments are defined as D(k) = lim∆t→0 µk (∆t)/∆t, this inequality holds for the moments themselves: (D(n+m) )2 ≤ D(2n) D(2m) .

(27)

Now, assume for contradiction that the series terminates at a finite order N > 2. This means D(N ) ̸= 0 and D(k) = 0 for all k > N . Let us choose n = N − 1 and m = 1. The inequality becomes: (D(N ) )2 ≤ D(2(N −1)) D(2) .

(28)

Since we assumed N > 2, the term 2(N − 1) = 2N − 2 satisfies 2N − 2 > N . (This is true for all N > 2; e.g., for N = 3, 2(3) − 2 = 4 > 3). Because 2N − 2 > N , our termination assumption implies that D(2N −2) = 0. Substituting this into the inequality, we get: (D(N ) )2 ≤ 0 · D(2) = 0.

(29)

The only way for the square of a real number to be less than or equal to zero is if the number itself is zero. This forces D(N ) = 0. This contradicts our initial assumption that D(N ) ̸= 0. Therefore, the assumption that the series terminates at any finite order N > 2 must be false. The only possibilities are that the series does not terminate, or that it terminates at N = 2 (the argument above does not apply for N = 2, as 2(2 − 1) = 2, which is not greater than N ). Pawula’s theorem implies that to prove truncation, we must show that all moments D(n) for n > 2 are zero. The following general theorem provides a sufficient condition for this. Theorem C.19 (Vanishing Moments of Continuous Processes). For any continuous-path Markov process, all propagator moments D(n) for n > 2 are identically zero. 27

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

(30)

Because the process has continuous paths, for any arbitrarily small ϵ > 0, the integral’s support is effectively confined to |ξ| ≤ ϵ as ∆t → 0. We can therefore write: Z 1 (n) |Di1 ...in | ≤ lim |ξ|n W (ξ|∆t)dξ. (31) ∆t→0 ∆t |ξ|≤ϵ For n > 2, we can bound |ξ|n within the domain of integration: |ξ|n = |ξ|n−2 |ξ|2 ≤ ϵn−2 |ξ|2 . Substituting this bound: Z Z 1 1 (n) |ξ|2 W (ξ|∆t)dξ. (32) |Di1 ...in | ≤ lim ϵn−2 |ξ|2 W (ξ|∆t)dξ = ϵn−2 lim ∆t→0 ∆t |ξ|≤ϵ ∆t→0 ∆t The limit on the right-hand side is the trace of the second propagator moment tensor, Tr(D(2) ). This gives the inequality: (n) |Di1 ...in | ≤ ϵn−2 Tr(D(2) ). (33) This inequality must hold for any choice of ϵ > 0. Since n > 2, the exponent n − 2 is positive. As we (n) can make ϵ arbitrarily close to zero, the only way for the inequality to be satisfied is if |Di1 ...in | = 0. This proves that all components of all propagator moment tensors for n > 2 are identically zero. The final step is to prove that our specific coarse-graining process satisfies the conditions of the preceding theorem. . Having established the emergence of an effective time-invariant generator, we now prove that the limiting process has continuous sample paths, which is necessary for the truncation of the KramersMoyal expansion. Proposition C.20 (The Refined Coarse-Graining Process is Continuous). Consider the limiting process obtained from the dyadic refinement sequence satisfying the conditions of Theorem C.13. This process is a continuous-path Markov process with an effective time-invariant generator. Proof. The proof proceeds in two steps: first showing path continuity for the refined discrete processes, then proving the property is preserved in the limit. Step 1: Path continuity for refinement level n. For the n-th refinement T (n) , the propagator at discrete time step ∆tn = 2−n is W (n) (ξ|∆tn ; x), (n) which is the continuous-time interpretation of the coarse-graining operator Mk . This operator maps p(x, k∆tn ) to p(x, (k + 1)∆tn ) by taking the conditional expectation with respect to the partition (n) Π(k+1)∆tn . (n)

At any point x, the new density is the average of the old density over the region Rx′ ∈ Π(k+1)∆tn containing x. Crucially, all probability mass is redistributed within the region Rx′ , which constrains the displacement vector ξ to satisfy |ξ| ≤ diam(Rx′ ). By the bounded intermediate complexity assumption (Theorem C.13, condition 2), the diameter of regions added at refinement n satisfies: δn := sup diam(Rx′ ) = O(2−n ). x∈X

Therefore, the propagator W (n) (ξ|∆tn ; x) has support only on |ξ| ≤ δn . 28

(34)

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

For any fixed ϵ > 0, choose N large enough such that δN < ϵ. Then for all n ≥ N and all x ∈ X : Z W (n) (ξ|∆tn ; x)dξ = 0. (35) |ξ|>ϵ

This proves that each discrete process in the refinement sequence satisfies the path continuity condition with a modulus δn → 0. Step 2: Direct establishment of path continuity from partition structure. We establish path continuity directly from the geometric properties of the refinement sequence. Key geometric fact: By construction of the dyadic refinement, the diameter of regions added at refinement level n satisfies: δn := sup diam(Rx′ ) ≤ L · 2−n , (36) x∈X

where L is the diameter of the feature space X and Rx′ is the finest region containing x at level n. This follows from the definition of dyadic refinement: each refinement step subdivides existing regions, and balanced subdivision strategies (e.g., splitting along the longest dimension) ensure that the maximum region diameter decreases geometrically. Since probability mass is conserved within each region, the propagator W (n) (ξ|∆tn ; x) has compact support: supp(W (n) (·|∆tn ; x)) ⊆ {ξ : |ξ| ≤ δn }. (37) Now compute the k-th moment for k ≥ 3: Z Z (n) k (n) k Mk (x) := |ξ| W (ξ|∆tn ; x) dξ ≤ δn W (n) (ξ|∆tn ; x) dξ = δnk .

(38)

Since δn = O(2−n ) and ∆tn = 2−n , we have: (n)

Mk (x) δk (L · 2−n )k = n ≤ = Lk · 2−n(k−1) → 0 ∆tn ∆tn 2−n

(39)

as n → ∞ for all k ≥ 3. Convergence of the limiting moments. Consider the subsequence of refinements that achieves convergence of the propagators (from the diagonalization in Theorem C.13). For this subsequence, define: (n) M (x) . (40) D(k) (x) := lim inf k n→∞ ∆tn From the bound above: D(k) (x) ≤ lim inf Lk · 2−n(k−1) = 0 n→∞

for all k ≥ 3.

(41)

This proves that all propagator moments of order k ≥ 3 vanish identically in the limit. By the standard definition from Gardiner (2004), a Markov process with D(k) = 0 for all k ≥ 3 is a continuous-path process. The time-invariance of the effective generator (established in Theorem C.13) ensures this path continuity property holds uniformly over all x ∈ X and is independent of time. With this logical chain complete, the truncation of the Kramers-Moyal expansion is a direct consequence. C.4.2

The Fokker-Planck Equation and Equivalent SDE

With the preceding theorems and propositions established, we are now equipped to prove the main result of our derivation. 29

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Theorem C.21 (Hierarchical Coarse-Graining as a Diffusion Process). The continuous-time limit of the hierarchical coarse-graining process is governed by the Fokker-Planck equation, which is formally equivalent to the Itô stochastic differential equation (SDE): dXt = f (Xt )dt + g(Xt )dWt ,

(42)

where f (x) and g(x) are the time-invariant drift and diffusion terms corresponding to the first and second propagator moments of the process, and Wt is a standard Wiener process. Proof. The proof synthesizes the preceding results to establish both parts of the theorem. First, we derive the governing PDE for the probability density. By Proposition C.20, our process is a homogeneous continuous-path Markov process. By Theorem C.19, this implies that all propagator moments D(n) for n > 2 are identically zero. Invoking Theorem C.18 (Pawula’s Theorem), the condition that all higher-order moments vanish forces the Kramers-Moyal expansion (Equation 23) to truncate exactly after the second term. The remaining n = 1 (drift) and n = 2 (diffusion) terms constitute the Fokker-Planck equation, which describes the evolution of the probability density p(x, t): d d X ∂p(x, t) 1 X ∂ ∂2 =− [fi (x)p(x, t)] + [(g(x)g(x)⊤ )ij p(x, t)]. ∂t ∂x 2 ∂x ∂x i i j i=1 i,j=1

(43)

Second, we establish the equivalence to the SDE. It is a standard result in the theory of stochastic processes that the Fokker-Planck equation shown above is the forward Kolmogorov equation for the probability density of a process governed by the Itô SDE presented in the theorem statement (Gardiner, 2004; Gillespie, 1991). This establishes the formal equivalence. Finally, the time-invariance of the drift f (x) = D(1) (x) and diffusion g(x)g(x)⊤ = D(2) (x) is a direct consequence of Theorem C.13 . This completes the proof. This derivation completes the formal bridge set out in the main text. We have shown how the static, hierarchical structure of a decision tree can be re-interpreted as a discrete-time Markov process over distributions. The continuous-time limit of this process, by virtue of its construction via local averaging, is necessarily a drift-diffusion process. Remark C.22 (Equivalence to a Probability Flow ODE). It is crucial to clarify the nature of the resulting process. The hierarchical coarse-graining operation, defined by conditional expectation, is a deterministic averaging process; it does not introduce new randomness at each step. Consequently, in the continuous-time limit, the second propagator moment—the diffusion tensor—must be identically zero: g(x) ≡ 0. When the diffusion term vanishes, the general SDE collapses into a deterministic ordinary differential equation (ODE) of the form dXt = f (Xt )dt. This specific type of deterministic equation, which describes the evolution of probability densities, is known in the literature as a Probability Flow (PF) ODE (Song et al., 2021). Therefore, while our derivation correctly yields the general Fokker-Planck form, the Tree-to-Flow mapping is more precisely a bijection between a decision tree and a deterministic PF-ODE, which itself is a zero-diffusion instance of the broader SDE framework. For further analysis however, in full generality we assume that the tree yields a SDE. This assumption also allows for analysis if the underlying structure itself has some randomness (Aldous, 1997).

The Implicit Hierarchical Structure of Continuous-Time Diffusion

In Section C, we established a mapping from a static, hierarchical tree to a deterministic Probability Flow ODE. This mapping relied on a coarse-graining operator based on conditional expectation, which models a process of increasing entropy from the leaves to the root. To prove that our framework is fully bidirectional, we must now establish the reverse: any continuous-time diffusion process, via its induced Probability Flow ODE, implicitly defines a canonical hierarchy equivalent to a decision tree. Furthermore, we shall show that this class includes the class of SDEs that are learned by standard score based generative models (Song et al., 2021). A similar result was also shown by (Ramachandran et al., 2025) utilising ideas from statistical physics however the tree construction in their case was implicit. 30

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

We will build this argument as a hierarchy of results. First, we prove that any homogeneous SDE induces a general hierarchical clustering (a dendrogram) by observing the temporal flow of entropy. Second, we analyze how the convergence properties of the SDE determine the depth and stability of this hierarchy, addressing both non-stationary and stationary processes. Finally, we prove that only SDEs that converge to a unique and maximally entropic stationary distribution—a uniform distribution over a manifold—induce a single, rooted tree structure that is fully consistent with the conditional expectation operator. This allows us to conclude that the Ornstein-Uhlenbeck process, which is central to modern diffusion models, has the exact rooted-tree structure that makes our entire framework bidirectional and self-consistent. Remark D.1 (The Role of the PF-ODE). While we discuss SDEs for generality, the hierarchical structure depends only on the marginal densities {pt }t≥0 , which are identical for an SDE and its corresponding Probability Flow ODE. Since moment-based merger times Definition D.12) depend solely on these marginals, the induced tree is determined entirely by the PF-ODE, not the stochastic dynamics. This mirrors the Tree → Flow direction ( Theorem C.21), where trees induce deterministic flows. D.1

The General Construction: From Homogeneous SDEs to Dendrograms

We first prove the most general result for the class of processes relevant to our framework: any well-behaved, homogeneous SDE, when applied to a clustered data distribution, generates a canonical hierarchical structure. This structure arises naturally from observing the temporal flow of entropy. D.1.1

Coarse-Graining as a Flow of Entropy

The foundation of our construction is the recognition that the forward diffusion process is a continuous analogue of the discrete coarse-graining operation. Proposition D.2 (Entropy Flow in Diffusion Processes). Let Xt evolve according to the forward SDE with marginal densities pt dXt = b(Xt , t) dt + σ(Xt , t) dWt , with marginal density pt satisfying the corresponding Fokker-Planck equation. The differential entropy Z H(pt ) := −

pt (x) log pt (x) dx

is a non-decreasing function of time. Moreover, if the process is homogeneous (i.e., b and σ are time-independent) and the diffusion tensor D = σσ ⊤ is strictly positive definite, then the entropy is strictly increasing for any non-stationary initial distribution. Proof. The time derivative of the differential entropy under a Fokker-Planck evolution is given by the standard formula (Villani et al., 2008): d 1 H(pt ) = Ept [∇ · b(x, t)] + Ept [Tr(D(x, t) · I(x, t))], (44) dt 2 where I(x, t) is a matrix measuring the local squared gradient of the density (see Remark D.4). The second term is non-negative because I(x, t) is positive semi-definite and D is strictly positive definite. While the divergence of the drift, ∇ · b, can be negative (corresponding to compressive flows), for homogeneous and ergodic processes, such as the Ornstein-Uhlenbeck process, the diffusion term dominates, ensuring that the entropy does not decrease. Hence in these situations, for any nonstationary distribution, the Fisher information term is strictly positive, so the entropy strictly increases d until the stationary distribution is reached, at which point dt H(pt ) = 0. This continuous-time evolution of entropy can be interpreted as the analogue of discrete coarsegraining operations discussed in Section C. Lastly note that, as the entropy H(pt ) depends only on the marginals pt , this result applies equally to the SDE and its corresponding PF-ODE. Definition D.3 (Entropically Homogeneous Flow). Motivated by the preceding discussion, we formalize the notion of entropy monotonicity. A diffusion process (or its PF-ODE) with marginal densities R {pt }t≥0 and differential entropy H(pt ) = − pt (x) log pt (x)dx is entropically homogeneous if the entropy is a monotonic function of time. For the processes we consider in this work this means, 31

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

d (i) Forward-time: dt H(pt ) ≥ 0 for all t ≥ 0 (entropy increase from p0 toward stationary p∞ ). d (ii) Reverse-time: dt H(pt ) ≤ 0 (entropy decrease from p∞ toward p0 ).

This follows from the prior observation that for processes converging to maximally entropic stationary distributions, forward-time entropy increase is strict until stationarity, ensuring monotonic information loss (forward) or gain (reverse). Remark D.4 (Functional Fisher Information). The matrix I(x, t) in (44) though termed as the Fisher information matrix but is not the classical parametric Fisher information, since pt is a single distribution with no parameters. Instead, I(x, t) is defined using the Stein score (or functional score): I(x, t) := ∇x log pt (x) ∇x log pt (x)⊤ . Intuitively, it measures the local spatial gradients of the log-density, so that Tr(D · I) quantifies how the shape of the distribution contributes to the rate of entropy change under diffusion. D.1.2

Discretizing the Flow via the Characteristic Function

To construct a discrete hierarchy from this continuous flow, we must define discrete ”merger” events. We do so by tracking the statistical distinguishability of evolving conditional distributions. The most fundamental object for describing a distribution is its characteristic function. Definition D.5 (Characteristic Function and Support Clustering). The characteristic function (CF) R ⊤ of a probability law P is the Fourier transform of the measure, ϕ(t) = eit x dP (x). The CF uniquely determines the law. The support of the law, supp(P ), can be clustered by observing the behavior of its CF. For any given frequency-domain partition (bins), the values of the CF within those bins define a signature for the distribution. Two distributions are considered statistically close if their CFs are close in some metric (e.g., L2 distance). While the CF provides a complete description, it is often intractable. A more practical approach is to use its derivatives at the origin, which correspond to the moments of the distribution. This requires a rigorous definition of the initial clusters {Ck }K k=1 from a continuous data distribution p0 . Assumption D.6 (Smooth Density with Isolated Modes). The data density p0 (x) is twice continuously differentiable on a compact domain X ⊂ Rd , and possesses K isolated local maxima {x∗k }K k=1 (modes) satisfying: 1. ∇p0 (x∗k ) = 0 and ∇2 p0 (x∗k ) is negative definite, 2. The basins of attraction Bk := {x : limt→∞ ϕt (x) = x∗k } under the gradient flow ẋ = ∇ log p0 (x) form a partition of X , 3. There exists ρmin > 0 such that inf x∈∂Bk p0 (x) < p0 (x∗k ) − ρmin for all k. Definition D.7 (Level-Set Clustering). For a threshold λ ∈ (0, maxk p0 (x∗k )), define the super-level set: Sλ := {x : p0 (x) ≥ λ}. (45) The initial clusters are the connected components of Sλ for λ chosen such that Sλ has exactly K components, one containing each mode x∗k . Proposition D.8 (Well-Defined Initial Clustering). Under Assumption D.6, there exists an interval [λmin , λmax ] such that for any λ ∈ [λmin , λmax ], the super-level set Sλ has exactly K connected components. These components define a well-separated initial clustering {C1 , . . . , CK }. Proof. By condition (1) of Assumption D.6, each mode x∗k is a strict local maximum. By condition (3), there exist saddle points or low-density regions separating the modes. The implicit function theorem guarantees that for λ slightly below mink p0 (x∗k ), the super-level set Sλ consists of K disjoint components, each a simply-connected neighborhood of one mode. As λ decreases, these components grow. The interval [λmin , λmax ] is the range where no mergers occur, i.e., the level sets remain separated by the saddle-point structure of the density. 32

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Remark D.9 (Extension to Non-Smooth Densities). For densities without smooth mode structure (e.g., uniform distributions on manifolds), the initial clustering can be defined via uniform spatial partitioning (e.g., k-means or Voronoi tessellation). The subsequent merger dynamics remain welldefined; the moment-based criterion simply tracks when these initially arbitrary clusters become statistically indistinguishable under diffusion. Assumption D.10 (Existence of Moments). Now, we assume that the conditional laws pt (·|Ck ) under consideration are such that their characteristic functions are at least n-times differentiable at the origin. This is a mild regularity condition, satisfied if the distributions have finite moments up to order n. Under this assumption, we can define our primary tool for tracking statistical distinguishability. Definition D.11 (Conditional Expected Moment Tensor). Let pt (x|Ck ) be the marginal density at time t of the diffusion process, conditioned on starting in the initial cluster Ck . If Assumption D.10 holds, the n-th order conditional expected moment tensor is the n-th centered moment of this distribution: (n) Mt (Ck ) = Ex∼pt (·|Ck ) (x − E[x])⊗n . (46) This tensor resides in a suitable Hilbert space Hn of rank-n tensors. For n = 2, this is the conditional covariance matrix. Definition D.12 (Moment-Based Merger Time). For any two initial clusters Ci and Cj , and a given (n,ϵ) moment order n and distinguishability threshold ϵ > 0, the (n, ϵ)-merger time, denoted tij , is the first time at which the distance between their respective conditional expected moment tensors falls below ϵ: (n,ϵ) (n) (n) tij = inf t ∈ [0, T ] | Mt (Ci ) − Mt (Cj ) ≤ϵ . (47) Hn

(n)

Since Mt (Ck ) depends only on the marginal pt (·|Ck ), the merger time is identical for an SDE and its PF-ODE. Further, the key significance of this merger time lies in the fact that it serves as a rigorous proxy for the convergence of the underlying distributions themselves. Proposition D.13 (Moment Convergence Implies Distributional Convergence). Under Assumption D.10 (for order n + 1), the convergence of the first n conditional expected moment tensors of two distributions implies their convergence in total variation distance. Formally, for any two conditional laws pt (·|Ci ) and pt (·|Cj ): lim∗

t→tij

n X k=1

(k)

Mt (Ci ) − Mt (Cj )

= 0 =⇒ lim∗ dT V (pt (·|Ci ), pt (·|Cj )) = 0. t→tij

(48)

The proof of this result connects the convergence of moment structures to convergence in total variation via a moment-TV inequality (Ramachandran et al., 2025), which relies on the properties of characteristic functions. (n,ϵ)

Proposition D.14 (Ultrametric Property of Moment-Based Merger Times). Let tij denote the (n, ϵ)-merger time between clusters Ci and Cj as defined above. Then for any three clusters Ci , Cj , and Ck , the merger times satisfy the ultrametric inequality: n o (n,ϵ) (n,ϵ) (n,ϵ) . (49) tik ≤ max tij , tjk (n,ϵ)

Proof. By definition, the (n, ϵ)-merger time tij is the first time when the distance between the n-th order conditional moment tensors of clusters Ci and Cj falls below the threshold ϵ in the Hilbert space Hn . The evolution of the conditional moments under the homogeneous diffusion process is continuous and monotone: once two clusters satisfy the merger criterion, they remain within ϵ of each other for all subsequent times. Consider three clusters Ci , Cj , and Ck , and let (n,ϵ)

tmax := max{tij 33

(n,ϵ)

, tjk }.

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

At time tmax , both pairs (Ci , Cj ) and (Cj , Ck ) satisfy the merger criterion. By the triangle inequality in Hn , we have (n)

(n)

∥Mtmax (Ci )−Mtmax (Ck )∥Hn ≤ ∥Mtmax (Ci )−Mtmax (Cj )∥Hn +∥Mtmax (Cj )−Mtmax (Ck )∥Hn ≤ 2ϵ. By the monotonicity of the flow, the pair (Ci , Ck ) must satisfy the merger criterion at some time (n,ϵ) tik ≤ tmax . Therefore, (n,ϵ) (n,ϵ) (n,ϵ) tik ≤ max{tij , tjk }, which establishes the ultrametric inequality. D.1.3

Main Construction Theorem

Theorem D.15 (Entropically Homogeneous Flows Induce a Canonical Dendrogram). Any entropically homogeneous diffusion process Definition D.3 (or equivalently its PF-ODE) satisfying standard regularity conditions and when applied to a data distribution p0 satisfying Assumption D.6, with initial clusters {Ck }K k=1 defined via level-set clustering and Assumption D.10, induces a unique, canonical hierarchical clustering on these clusters. This structure is equivalent to a dendrogram. Proof. The proof is constructive, using the sequence of moment-based merger times to build the hierarchy via temporal agglomerative clustering. 1. Initialization (Leaves at t = 0): At time t = 0, all K clusters {C1 , . . . , CK } are distinct by definition, forming the leaves of our hierarchy. (n,ϵ)

2. First Merger: We compute the set of all pairwise merger times, {tij

} for i ̸= j. The first

(n,ϵ) merger event occurs at the minimum of these times, t1 = mini,j tij . Let this minimum

occur for the pair (Ca , Cb ). At time t1 , we merge Ca and Cb into a new super-cluster, Cab . This event defines the lowest-level branch in the hierarchy. 3. Iterative Merging: We now have a set of K − 1 clusters. We repeat the process, computing the merger times between all pairs in this new set (including between the new super-cluster and other original clusters). The next merger event at time t2 > t1 defines the next branch. This agglomerative process continues, and the ordered sequence of merger times defines a unique ultrametric on the set of initial clusters (Proposition D.14). This ultrametric is isomorphic to a dendrogram (Murtagh, 2009). D.2

The Role of Stationarity in Defining the Hierarchy’s Structure

The structure of the dendrogram, whether it is infinitely deep or terminates, is determined by the long-term convergence properties of the SDE. Definition D.16 (Stationary Distribution). A probability distribution p∞ is a stationary distribution for a process if, once the process reaches this distribution, its law remains unchanged for all future time. A process may have no, one, or many stationary distributions. Proposition D.17 (Non-Stationary Processes Induce Continuously Evolving Hierarchies). If a diffusion process does not possess a stationary distribution, the induced hierarchical clustering is infinitely deep and its structure is not guaranteed to be stable. Proof. A non-stationary process is one whose law, Pt , never converges as t → ∞. This means that for any time t, there exists a later time t′ > t at which the distribution is different, pt′ ̸= pt . The process of statistical merging, therefore, never reaches a final, stable state. The sequence of merger events does not terminate, yielding a dendrogram of infinite depth. Furthermore, because the process never settles, the very nature of the clusters can change over time. A set of clusters that are distinct at time t may merge at time t1 , but the evolution of the underlying, non-stationary flow could cause them to become distinguishable again at a later time t2 > t1 . New distinctions can emerge. Therefore, there is no fixed, final set of nodes, and the hierarchy is continuously evolving. 34

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Proposition D.18 (Stationary Processes Induce Asymptotically Stable Hierarchies). If a diffusion process possesses one or more stationary distributions, the induced hierarchical clustering is asymptotically stable and all merger events occur within a finite time horizon. Proof. The existence of a stationary distribution implies that the process converges in law to a stable endpoint (or one of several stable endpoints). Let this convergence happen over a characteristic time scale T . All statistical distinctions between paths that converge to the same endpoint must be erased within this time scale. Therefore, all merger times t∗ij for such paths are bounded, t∗ij ≤ T . This results in a dendrogram of finite depth. If the process has multiple distinct stationary distributions, different initial clusters may converge to different endpoints, resulting in a forest of multiple, disconnected dendrograms, each corresponding to a basin of attraction. Remark D.19 (On Infinite Precision). The ”finiteness” of the hierarchy for a stationary process refers to the time horizon of its construction. The hierarchy itself is still infinitely deep in a formal sense. The merger time t∗ij is a function of a distinguishability threshold ϵ. As ϵ → 0, the merger time t∗ij → ∞ (or T ). Thus, for any finite time, we can find a sufficiently small ϵ that reveals a deeper level of branching. The finite tree is the structure formed by all mergers up to the characteristic time scale of the process. D.3

The Condition for a Single, Rooted Decision Tree

We now establish the final, crucial condition under which the hierarchy is not just a dendrogram, but a single, rooted tree fully consistent with our framework. Theorem D.20 (Maximally Entropic Stationarity Implies a Rooted Tree). If a diffusion process (via its PF-ODE) converges to a unique and maximally entropic stationary distribution, then the canonical hierarchy it induces is a single, rooted tree. Proof. A unique stationary distribution guarantees, by the logic of Proposition D.18, that all clusters will eventually merge, forming a single, connected dendrogram. The additional condition of being maximally entropic is what defines the root and ensures consistency with our framework. 1. Defining the Root: A maximally entropic distribution, such as a uniform distribution over a manifold X , represents a state of complete information loss relative to that manifold. It is a distribution that has no further structural information to lose. This state acts as the single, common ancestor for all possible paths—the root of the tree. Any partition induced by the probability law of a non-uniform stationary distribution would imply that the ”root” still contains information, which is a contradiction. 2. Consistency with Coarse-Graining: The mapping from a tree to an SDE in Section C is based on a coarse-graining operator defined by conditional expectation. This operator models a pure process of information aggregation, or entropy increase. A forward diffusion process is only guaranteed to be a pure entropy-increasing process from any initial state if its stationary distribution is itself maximally entropic. Only in this case does the temporal flow of entropy perfectly mirror, in reverse, the entropy-decreasing nature of a decision tree from root to leaves. This closes the logical loop, ensuring the SDE → Tree and Tree → SDE mappings are consistent.

This result then applies directly to the most common class of diffusion models. Proposition D.21 (The Ornstein-Uhlenbeck Process Induces a Rooted Tree). The Ornstein-Uhlenbeck (OU) process, which forms the basis of Variance-Preserving (VP) diffusion models, converges to a unique stationary distribution, a standard Gaussian N (0, I). In high dimensions (d ≥ 5), this distribution is maximally entropic as it concentrates uniformly on the surface of a sphere. Therefore, any VP-SDE (and the corresponding PF-ODE) in high dimensions induces a single, canonical rooted tree. 35

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Proof. The OU process is a classical homogeneous ergodic process whose convergence to a unique Gaussian stationary distribution is a standard result (Karatzas & Shreve, 2012). In high √ dimensions (d ≥ 5), the norm of a vector drawn from N (0, I) is sharply concentrated around d (Vershynin, √ 2018). The distribution is therefore effectively uniform over the manifold of a sphere of radius d. It is maximally entropic with respect to this manifold. It therefore satisfies the conditions of Theorem D.20, and the stationary distribution acts as the geometric root of the tree. Corollary D.22. The experimental methodology of partitioning the terminal noise space based on the endpoint of the reverse-time ODE is a practical algorithm for empirically constructing the canonical decision tree hierarchy proven to exist in Theorem D.15. In conclusion, we have proven that the marginal densities of a homogeneous diffusion process (equivalently, its deterministic PF-ODE), such as the OU process underlying VP diffusion models, induce a unique, canonical hierarchy equivalent to a single, rooted tree. This result, constructed from the ordered sequence of moment-based merger events that depend only on marginals, formally establishes the Flow → Tree mapping and completes the bidirectional correspondence: Tree ↔ PF-ODE.

Derivation of the Relationship between Boosting and Score Matching

This appendix provides a complete, first-principles derivation of the claim that in the limiting case (Theorem C.13), tree-based gradient boosting algorithm is a provably optimal, score-driven numerical method for constructing a coarse-to-fine trajectory in the space of Stochastic Differential Equations (SDEs). Our argument proceeds in four parts. First, we formalize the boosting algorithm and introduce the net decision tree abstraction, proving that it induces a monotonic refinement of the feature space. Second, we use Theorem C.21 to map this process to a discrete trajectory in the space of flows, contextualizing it within a general coarse-to-fine learning principle defined with respect to the marginals of an ideal process. Third, we prove that the mechanism guiding this trajectory—the fitting of residuals—is a direct implementation of denoising score matching (Vincent, 2011). Finally, we synthesize these results, using the Bellman principle of optimality (Bertsekas, 2012) to prove that the greedy, stage-wise nature of boosting is the globally optimal solution to a trajectory-wide score matching objective. E.1

Formalizing Gradient Boosting and the Net Decision Tree

Our first objective is to construct a rigorous mathematical description of the gradient boosting algorithm’s structural evolution. We begin by re-introducing the algorithm from the perspective of functional optimization, which provides the necessary context for its mechanics. We then introduce the net decision tree, a crucial abstraction that allows us to analyze the geometric consequences of the additive updates. Finally, we prove the central result of this section: that the boosting process induces a monotonic, hierarchical refinement of the feature space, exclusively by subdividing the finest existing partitions. E.1.1

Gradient Boosting as Functional Gradient Descent

The power of the gradient boosting algorithm stems from its interpretation as a stage-wise optimization procedure in an infinite-dimensional function space. Definition E.1 (The Boosting Objective). Given a dataset {(xi , yi )}N i=1 and a differentiable loss function L(y, F (x)), the goal of boosting is to find a function F : Rd → R that minimizes the expected loss, or its empirical estimate, the loss functional J(F ): J(F ) =

N X

L(yi , F (xi )).

(50)

i=1

Finding the optimal function F in one step is intractable. Gradient boosting approaches this by iteratively taking steps in the direction of steepest descent. In a function space, this direction is given by the negative functional gradient. 36

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Definition E.2 (Functional Gradient and Pseudo-Residuals). The functional gradient h of the loss i δJ i ,F (xi )) functional J(F ) with respect to the function F at a point xi is given by δF (xi ) = ∂L(y . ∂F (xi ) The negative functional gradient, evaluated at the current model Fm−1 , defines the pseudo-residuals, rim : ∂L(yi , F (xi )) rim = − . (51) ∂F (xi ) F =Fm−1 The pseudo-residuals represent the optimal direction of change for the function’s output at each data point to most rapidly decrease the overall loss. The algorithm, therefore, does not solve for the optimal function directly, but instead approximates this functional gradient descent. At each step, it fits a simple function—a weak learner—to be maximally correlated with the negative gradient, and then adds this function to the existing model. Algorithm 5 Tree-Based Gradient Boosting PN

1: Initialize model with a constant value: F0 (x) = arg minc i=1 L(yi , c). 2: for m = 1 to M do h i i ,F (xi )) 3: Compute pseudo-residuals for each sample i: rim = − ∂L(y ∂F (xi )

F =Fm−1 Fit a weak learner (decision tree) hm (x) to the pseudo-residuals {(xi , rim )}N i=1 . Update the model: Fm (x) = Fm−1 (x) + ηhm (x).

4: 5: 6: end for 7: return FM (x).

E.1.2

The Net Decision Tree: A Monolithic Abstraction

While Algorithm 5 describes the functional updates, it obscures the evolution of the model’s geometric structure. To analyze this, we must move from viewing the model as a sum of functions to viewing it as a single, evolving piecewise-constant function defined on a single, evolving partition of the feature space. We first formalize the concept of refinement of a partition. Definition E.3 (Partition and Refinement). A partition Π′ is a refinement of a partition Π if every region in Π′ is a subset of some region in Π. If at least one region of Π is the union of two or more regions in Π′ , the refinement is said to be strict. We now formally define our central abstraction. Definition E.4 (Net Decision Tree). At any discrete step m, the net decision tree, denoted Tm , is a single decision tree whose structure is defined by two properties: 1. Partition: Its partition of the feature space, Πm , is the common refinement of the partitions induced by all constituent weak learners {h1 , . . . , hm }. That is, a region R ∈ Πm is the largest possible connected subset of X such that for any two points xa , xb ∈ R, hi (xa ) = hi (xb ) for all i ∈ {1, . . . , m}. 2. Leaf Values: The value Pmassigned to any leaf region in Πm is the sum of the predictions from all constituent trees, i=1 ηhi (x), where η is the learning rate. This abstraction provides a monolithic, non-additive representation of the ensemble, allowing us to rigorously analyze the evolution of its decision boundaries as a single geometric object. E.1.3

Monotonic Structural Refinement

We now use the net decision tree abstraction to prove the central result of this section: that the boosting process induces a monotonic and strictly hierarchical refinement of the feature space. Proposition E.5 (Monotonic Partition Refinement). The sequence of partitions {Πm }M m=0 generated by the net decision trees forms a nested hierarchy, where each partition Πm is a strict refinement of the preceding partition Πm−1 . 37

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Proof. Let Πm−1 be the partition corresponding to the net tree Tm−1 , and let Πhm be the partition induced by the new weak learner hm . By the definition of the net decision tree, the new partition Πm is the common refinement of {Πh1 , . . . , Πhm−1 , Πhm }. This is equivalent to the common refinement of Πm−1 and Πhm . Consider any region R′ ∈ Πm . By definition, R′ is a region where all functions {h1 , . . . , hm } are constant. This implies that all functions {h1 , . . . , hm−1 } are also constant on R′ . Therefore, there must exist a region R ∈ Πm−1 such that R′ ⊆ R. Since this holds for every region in Πm , we conclude that Πm is a refinement of Πm−1 . To prove the refinement is strict, we note that the weak learner hm is trained to fit the pseudo-residuals rim , which are non-zero so long as the model Fm−1 is not optimal. Thus, hm will be a non-trivial tree with its own partition Πhm that is not the trivial partition {X }. This partition will necessarily subdivide at least one of the regions in Πm−1 . Therefore, the refinement is strict. E.1.4

The Canonical Hierarchy and the Net Decision Tree

We now address a crucial ambiguity. A given partition, such as Πm , does not define a unique hierarchical tree structure. For example, a partition of a square into four quadrants can be formed by splitting on the x-axis then the y-axis, or vice-versa. These two different trees produce the same final partition but represent different coarse-graining paths. To resolve this, we must select a unique, canonical hierarchy. The boosting algorithm’s own history provides this canonical choice. The sequence of refinements is not arbitrary; it is temporally ordered. This historical sequence defines a unique coarse-graining path in reverse. Definition E.6 (Canonical Hierarchy and Net Decision Tree). The net decision tree, denoted Tm , is a unique, canonical hierarchical model of the ensemble Fm defined by: 1. Leaf Partition: The set of its leaf nodes corresponds exactly to the regions of the partition Πm . 2. Hierarchical Structure: The tree’s internal structure is defined by the unique, nested sequence of partitions generated by the boosting algorithm itself: {Π0 , Π1 , . . . , Πm }. The coarsening from the leaves (Πm ) to the root (Π0 = {X }) is defined by reversing the historical refinement process. The parent of a node in partition level Πk is the unique region in the coarser partition Πk−1 that contains it. This definition provides the necessary contract: for any ensemble Fm , there is only one net decision tree Tm , because its hierarchy is fixed by the algorithm’s history. This canonical definition allows us to rigorously describe the evolution of the tree’s structure. Theorem E.7 (Monotonic Evolution of the Canonical Hierarchy). The canonical hierarchy of the net decision tree Tm−1 is a strict sub-hierarchy of the net decision tree Tm . The structure of Tm−1 is immutably preserved within Tm . Proof. The hierarchy of Tm−1 is defined by the unique coarsening path (Πm−1 → · · · → Π0 ). The hierarchy of Tm is defined by the unique coarsening path (Πm → Πm−1 → · · · → Π0 ). By inspection, the sequence defining the structure of Tm−1 is a prefix of the sequence defining the structure of Tm . The transition from Tm−1 to Tm corresponds to appending a new, finer partition level, Πm , to the bottom of the existing hierarchy. Let us consider the internal structure. An internal node in this canonical hierarchy corresponds to a region in a partition Πk for k < m. The set of all internal nodes of Tm is the set of all regions in {Π0 , . . . , Πm−1 }. The set of all internal nodes of Tm−1 is the set of all regions in {Π0 , . . . , Πm−2 }. The structure of Tm−1 is therefore perfectly nested and preserved within Tm . The only structural change is the transformation of the leaf nodes of Tm−1 (the regions of Πm−1 ) into internal nodes of Tm . Corollary E.8 (Structural Evolution by Leaf Refinement). The structural evolution from the canonical tree Tm−1 to Tm occurs exclusively by the refinement of the leaf partition Πm−1 into the new leaf partition Πm . 38

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Having established that gradient boosting constructs a unique and monotonic hierarchy of spatial partitions, we are now equipped to map this discrete structural evolution to a continuous-time process in the space of SDEs. The SDE corresponding to the model Tm utilizes Theorem C.21 from this specific, historically-defined coarsening path (Πm → Πm−1 → · · · → Π0 ). E.2

The Boosting Process as a Trajectory in SDE Space

With the structural evolution formalized, we now bridge the discrete boosting process to the continuous-time world of SDEs 4 . We appeal directly to the main result of Section C, which established a formal mapping from any static, hierarchical tree to a complete, continuous-time drift-diffusion process. By applying this mapping to each net decision tree Tm in our sequence, we reframe the entire boosting algorithm as a discrete sequence of SDEs ({Sm }M m=0 ) (following our choice in Remark C.22) : {T0 , T1 , . . . , TM }

Theorem C.21

−−−−−−−→

{S0 , S1 , . . . , SM }.

This sequence is not an arbitrary collection. The underlying partitions {Πm } become progressively finer, suggesting the SDEs themselves should reflect this structure. But what does it mean for one SDE to be ”finer” than another? The intuition is that a model built on a finer partition can capture more detailed, lower-entropy information about the initial data distribution, p0 . In a diffusion process, this early-time information is precisely what is systematically destroyed as the process evolves towards the simple, high-entropy stationary distribution, p∞ . A ”finer” SDE should therefore be one that remains faithful to the true data-generating process for a longer duration (i.e., down to an earlier time t). To formalize this intuition, we must move from the spatial domain of partitions to the temporal domain of the processes themselves. We will now develop a rigorous language for comparing SDEs based on their temporal evolution, using the established tools of stochastic process theory (Durrett & Durrett, 2019; Karatzas & Shreve, 2012). This will allow us to prove that the sequence generated by boosting is not just a sequence, but a structured, coarse-to-fine trajectory through the space of all possible SDEs. E.2.1

Formalizing Temporal Coarseness via Tail Equivalence

The intuition behind a coarse model is that it has lost information about its specific starting point and its evolution has become indistinguishable from other processes that converge to the same end-state. We can formalize this by considering the laws of these processes on the space of continuous paths. Definition E.9 (Path Space and Tail σ-Algebra). Let C (path space) be the space of all continuous functions from [0, T ] to Rd . An SDE defines a probability measure, or law, P on this path space. • The canonical filtration {Ft }t∈[0,T ] is a sequence of σ-algebras where Ft represents all information about the path up to time t. • The tail σ-algebra at time t, denoted F≥t , is the σ-algebra generated by the process from time t onwards: F≥t = σ({Xτ : τ ∈ [t, T ]}). It represents all information about the future of the process as seen from time t. Using this standard construction, we can now define a rigorous equivalence relation for SDEs. Definition E.10 (Tail-Equivalent SDEs). Let S be a space of SDEs whose laws are measures on C and which all converge to a common, unique stationary distribution p∞ . Two SDEs, SA and SB , are said to be t-tail-equivalent, denoted SA ∼t SB , if their laws are identical when restricted to the tail σ-algebra F≥t . That is, for any event E ∈ F≥t : PA (E) = PB (E).

(52)

This defines an equivalence relation. The equivalence class of all SDEs that are t-tail-equivalent is denoted Kt . 4

We use the SDE framework (rather than restricting to PF-ODEs) to account for stochasticity in practical boosting such as, subsampling, feature randomization, and gradient noise (Friedman, 2002).

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

These equivalence classes, which group processes that are indistinguishable from the future onwards, form a natural nested structure. This structure is a filtration, but one that progresses backward from t = T. Proposition E.11 (The Filtration of Tail-Equivalence Classes). The collection of tail-equivalence classes {Kt }t∈[0,T ] forms a filtration. For any two times t1 , t2 ∈ [0, T ] such that t1 < t2 , the classes are strictly nested: Kt1 ⊂ Kt2 . (53) Proof. Let Si be an arbitrary process in the class Kt1 . By definition, its law restricted to F≥t1 is fixed. The tail σ-algebra F≥t2 is a sub-σ-algebra of F≥t1 , since any event knowable from t2 onwards is also knowable from t1 onwards. Therefore, the law of the process must also be fixed on F≥t2 , which implies Si ∈ Kt2 . This proves the inclusion Kt1 ⊆ Kt2 . The inclusion is strict because one can construct a process that deviates from a reference process only in the interval [t1 , t2 ), making it distinct in Kt1 but a member of Kt2 . This filtration provides the formal structure needed to define a coarse-to-fine trajectory. Definition E.12 (Coarse-to-Fine Trajectory). A sequence of SDEs, {Sm }M m=0 , constitutes a coarse-tofine trajectory if there exists a strictly decreasing sequence of times, T ≥ t0 > t1 > · · · > tM ≥ 0, such that each process Sm is a member of the tail-equivalence class Ktm but is not a member of any finer class Kt′ for t′ < tm . Sm ∈ Ktm

and

Sm ∈ / Kt′

∀t′ < tm .

Since tm+1 < tm , it follows from the filtration property that Ktm+1 ⊂ Ktm , i.e., the trajectory moves sequentially into smaller, more restrictive (i.e., finer) equivalence classes. E.2.2

The Boosting Process as a Coarse-to-Fine Trajectory

We now prove that the sequence of SDEs generated by the gradient boosting algorithm conforms to this rigorous definition. Theorem E.13 (The Boosting Sequence is a Coarse-to-Fine Trajectory). The sequence of models {Sm }M m=0 generated by the gradient boosting algorithm is a coarse-to-fine trajectory. Proof. Let S ⋆ be the ideal, unknown SDE corresponding to the net decision tree that perfectly models the data, with law P⋆ . The model at step m, Sm , is constructed from the net decision tree Tm , which is defined on the partition Πm . As proven in Proposition E.5, the partition Πm+1 is a strict refinement of Πm . A finer partition allows a model to capture more detailed, lower-entropy information about the true data distribution pdata = p⋆0 . In the context of a diffusion process, this fine-grained information corresponds to the behavior at earlier times. Therefore, the model Sm+1 , being more refined, provides a law Pm+1 that is a faithful approximation of the ideal law P⋆ on a larger tail σ-algebra than Sm can. Let us formalize this. For each model Sm , let tm be the earliest time for which its law, restricted to the future, is a good approximation of the ideal law. That is, Pm |F≥tm ≈ P⋆ |F≥tm . Thus, Sm can be considered a member of the ideal tail-equivalence class Ktm . Because Sm+1 is built on a strictly finer partition, it captures more detail about the initial data and can thus approximate the ideal process starting from an earlier time, tm+1 . Therefore, we must have tm+1 < tm . This implies that the boosting algorithm generates a sequence of models associated with a strictly decreasing sequence of times {tm }. By definition, this is a coarse-to-fine trajectory. With this rigorous foundation, we can now state our central learning assumption in this formal language. Assumption E.14 (Coarse-to-Fine Learning Bias). Consider a parameterized model fθ optimized via gradient descent on a loss functional L(θ) that decomposes as: L(θ) =

M −1 X k=0

Lk (θ),

(54)

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

where each component Lk measures the error on tail-equivalence class Ktk with t0 > t1 > · · · > tM −1 . We say the optimizer exhibits coarse-to-fine bias if, for sufficiently small learning rates, there exists a time scale separation in the optimization dynamics: τk ≪ τk+1 ,

(55)

where τk is the characteristic time for Lk (θt ) to reach ϵ-optimality. This assumption is satisfied by: 1. Greedy stage-wise algorithms (e.g., boosting, matching pursuit): By construction, Lk is optimized before Lk+1 is considered. 2. Hierarchical architectures (e.g., progressive growing, curriculum learning): The model capacity for finer scales is introduced only after coarser scales are learned. 3. Spectral bias in neural networks (empirical): Implicit regularization causes neural networks to learn low-frequency (coarse) components faster than high-frequency (fine) components (Rahaman et al., 2019). Remark E.15 (Falsifiability). This assumption can be tested empirically by measuring τk via the learning curves of Lk (θt ). For boosting, it holds by design (τk = time for one iteration). For neural networks, it is an empirical phenomenon observed in practice but not guaranteed theoretically for all architectures. However recent work (Bonnaire et al., 2025; Kingma et al., 2021; Kingma & Gao, 2023; Liang et al., 2024) shows the existence of such a separation for diffusion models, reinforcing the central discussion of Section D. E.3

Score Matching in the Space of SDEs: The Local Update Rule

We have proven that the gradient boosting algorithm constructs a coarse-to-fine trajectory {Sm } through a filtration of tail-equivalence classes of SDEs. We now address the question of how the algorithm navigates this trajectory. The mechanism is a form of denoising score matching, but its application in this context contains a crucial subtlety that distinguishes it from standard score-based generative modeling and ultimately unifies two different paradigms of supervision. To explain this, we will first formally define the necessary score-matching concepts, then present two perspectives on the trajectory navigation problem—one based on process supervision and the other on data supervision—and finally prove their equivalence under our central learning assumption. E.3.1

Formal Definitions of Score Matching

To make our argument rigorous, we first define the core concepts of score matching. Definition E.16 (Score Function). Let p(x) be a differentiable probability density function on Rd . The score function of this distribution is the gradient of its logarithm with respect to the data: sp (x) = ∇x log p(x).

(56)

The score function defines a vector field that points in the direction of the steepest ascent of the log-density. Definition E.17 (Denoising Score Matching (DSM)). Let p(x̃|x) be a known conditional noise distribution. Denoising Score Matching (DSM) trains a model sθ (x̃) to approximate the score of the noisy data distribution by minimizing the following objective (Vincent, 2011): h i 1 2 LDSM (θ) = Epdata (x) Ep(x̃|x) ∥sθ (x̃) − ∇x̃ log p(x̃|x)∥ . 2 This objective is tractable because the conditional score ∇x̃ log p(x̃|x) is known. E.3.2

The Duality of Supervision in Trajectory Refinement

With these formal tools, we can now precisely state the problem of navigating the SDE trajectory from two different but ultimately equivalent perspectives, distinguished by the source of their supervision. 41

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Definition E.18 (Structurally-Supervised Learning). A learning problem is structurally-supervised if the optimal coarse-to-fine trajectory is known a priori. Formally, an oracle provides the ideal sequence of tail-equivalence classes {Kt⋆m }M m=0 . The learning task is to find a sequence of refinement M operators, {Rm }m=1 , such that the generated sequence of models Sm = Rm (Sm−1 ) correctly follows the prescribed path, i.e., Sm ∈ Kt⋆m for all m. Standard denoising diffusion models are an instance of this paradigm: the forward SDE serves as the oracle defining the process, while the score model is the learned refinement operator. Definition E.19 (Data-Supervised Constructive Learning). A learning problem is data-supervised and structurally-unsupervised if the optimal trajectory is unknown, but the learner has access to a supervisory signal at each step. The learning task is to use the local supervisory signal to choose a sequence of refinement operators {hm }M m=1 that simultaneously constructs and traverses a coarse-to-fine trajectory. Gradient boosting is an instance of this paradigm: the ideal tree hierarchy is unknown, but the data labels {yi } provide a local, supervisory signal (the residuals) at each step. We now state the theorem that formally connects these two seemingly disparate paradigms. Theorem E.20 (Functional Equivalence of Learning Paradigms). Under Assumption E.14, an optimal Data-Supervised Constructive algorithm produces a sequence of models {Sm } such that: ∗ lim dpath (Sm , Sm ) = 0,

m→∞

(57)

∗ where Sm is the model produced by an optimal Structurally-Supervised algorithm and dpath is the path-space distance induced by the KL divergence. ⋆ Proof. Let {Sm } be the ideal trajectory defined by the oracle sequence of tail-equivalence classes {Kt⋆m }. Let {Sm } be the trajectory constructed by the data-supervised (boosting) algorithm. We ⋆ prove by induction that Sm = Sm for all m.

Base Case (m=0): Both paradigms start from the same maximally coarse model S0 (e.g., a constant prediction), which is a member of the coarsest class Kt⋆0 . Thus, S0 = S0⋆ . Inductive Hypothesis: Assume that at step m, the constructed model is identical to the ideal model, ⋆ Sm = Sm . This means the data-supervised algorithm has successfully constructed the correct trajectory up to this point. Inductive Step: We must show that the next model constructed, Sm+1 , is identical to the next model ⋆ on the ideal path, Sm+1 . 1. The structurally-supervised learner knows the next target class is Kt⋆m+1 . Its task is to find ⋆ ⋆ ⋆ an operator Rm+1 that transitions Sm to a model Sm+1 ∈ Kt⋆m+1 . 2. The data-supervised learner (boosting) does not know the target class Kt⋆m+1 . It uses the local data supervision (residuals) to find an update h⋆m+1 that produces a new model Sm+1 . 3. Here we invoke Assumption E.14. The coarse-to-fine learning bias guarantees that at step m, the model Sm is already ϵ-optimal for the tail-field F≥tm . The “coarsest” remaining part of the learning problem is to correctly model the dynamics in the time interval [tm+1 , tm ). 4. The assumption guarantees that the optimal, data-supervised update h⋆m+1 will be precisely the one that resolves this coarsest remaining part of the problem. By doing so, it produces a new model Sm+1 whose law now correctly matches the ideal law on the tail-field F≥tm+1 . 5. By definition, any such model is a member of the equivalence class Kt⋆m+1 . Since the update ⋆ is optimal, it must be the optimal such model, Sm+1 . ⋆ Therefore, Sm+1 = Sm+1 . By the principle of induction, the trajectories are identical. This crucial equivalence justifies our use of the DSM mathematical machinery to analyze the local, data-supervised boosting update.

E.3.3

The Boosting Update as an Optimal Score Matching Step

We now formalize the update mechanism of the constructive, data-supervised process. The guidance for this refinement is provided by a ”meta-score.” 42

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Definition E.21 (Meta-Score for Trajectory Navigation). Let p(y|Fm (x)) be the conditional likelihood of the true label y given the model’s prediction Fm (x). The meta-score, denoted s⋆m , is the score of this conditional distribution with respect to the conditioning variable (the model’s prediction): s⋆m (x, y) = ∇Fm log p(y|Fm (x)).

(58)

This meta-score represents the optimal direction of functional change for the model at point x to increase the likelihood of observing the true data y. Theorem E.22 (The Residual as the Optimal Meta-Score). For the squared error loss, the functional gradient of the boosting objective—the residual—is directly proportional to the optimal meta-score, s⋆m . Proof. The squared error loss, L(y, F ) = 21 (y − F )2 , is equivalent to maximizing the log-likelihood of a Gaussian conditional model, p(y|Fm (x)) = N (y; Fm (x), σ 2 ). The meta-score is the score of this conditional distribution with respect to the conditioning variable, Fm (x). We compute this gradient: √ (y − Fm (x))2 ⋆ 2 − log 2πσ sm (x, y) = ∇Fm log p(y|Fm (x)) = ∇Fm − 2σ 2 1 = 2 (y − Fm (x)). σ The result is proportional to the residual, rm+1 = y − Fm (x). The gradient boosting algorithm trains the weak learner hm+1 to fit this residual. Therefore, the boosting update is a direct implementation of a denoising score-matching step at the meta-level, where the data-supervised residual serves as the optimal score guiding the structurally-unsupervised construction of the trajectory from the coarse SDE Sm to the finer SDE Sm+1 . E.3.4

Connecting the Meta-Score to the SDE Score Error

The preceding theorem connects the boosting update to an abstract meta-score. We now provide the crucial physical grounding by proving that this meta-score is directly related to the behavior of the score functions of the corresponding trajectory of SDEs. Our proof will show that the boosting residual is an unbiased estimator of the total accumulated error between the ideal SDE’s score function and the current model’s score function, integrated over the temporal region where the model’s dynamics are invalid. Let S ⋆ be the ideal process with law P⋆ , marginals p⋆t , and score function s⋆t (x) = ∇x log p⋆t (x). At step m, the net decision tree corresponds to an SDE Sm , with law Pm , marginals pm,t , and score function sm,t (x). By construction, Sm ∈ Ktm , hence its law and scores are a good approximation of the ideal process for all times τ ≥ tm . The time tm is the boundary where the model’s dynamics begin to diverge from the ideal dynamics. To connect these forward-time properties to the reverse-time generation process, we first state the classical result for the time-reversal of a general diffusion process. Proposition E.23 (Time-Reversal of a General Diffusion Process). Let a forward Ito process be defined by dXt = b(Xt , t)dt + σ(Xt , t)dWt . The corresponding reverse-time process, which we define with a new time variable τ ∈ [0, T ] as Xτ = XT −τ , is also an Ito process governed by (Anderson, 1982): dXτ = −b(Xτ , T − τ ) + D(Xτ , T − τ )∇x log pT −τ (Xτ ) dτ + σ(Xτ , T − τ )dWτ , where D(x, t) = σ(x, t)σ(x, t)⊤ is the diffusion tensor, pt is the marginal density of the forward process at time t, and Wτ is a standard Brownian motion in the reverse time direction. With this general tool, we can now state and prove the main theorem of this section. Theorem E.24 (Meta-Score as the Integrated SDE Score Error). The expected boosting residual is proportional to the expected integrated error between the ideal SDE’s score function and the current model’s score function, integrated over the temporal region where the model is invalid. Proof. The proof proceeds by relating the expected endpoints of the reverse-time SDEs to the integral of their respective drifts. 43

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

1. Let X⋆t be a path generated by the ideal process SDE⋆ , and let Xm,t be a path generated by our model Sm . A core assumption of the score-based modeling framework is that the forward process, defined by drift b(x, t) and diffusion tensor D(x, t), is a fixed, known process independent of the data. The learning task is to find the correct score function to drive the reverse process. ⋆

2. Let Xτ and Xm,τ be the corresponding reverse-time processes, using the reverse time variable τ ∈ [0, T ]. We consider their evolution starting from the same point at τ = 0 ⋆ (forward time t = T ), i.e., X0 = Xm,0 = XT . The endpoints of these processes at reverse ⋆ time τ = T are XT and Xm,T . 3. We express the endpoints in integral form. The difference between the endpoints is: Z T Z T Z T ⋆ ⋆ ⋆ ⋆ XT −Xm,T = b⋆rev (Xτ , τ ) − brev,m (Xm,τ , τ ) dτ + σ(Xτ , τ )dWτ − σ(Xm,τ , τ )dWm,τ . 0

We now take the expectation conditioned on the starting point XT . The expectation of the stochastic integral terms is zero because Ito integrals are martingales that start at zero. "Z # T ⋆ ⋆ ⋆ E[XT − Xm,T |XT ] = E brev (Xτ , τ ) − brev,m (Xm,τ , τ ) dτ XT . 0

4. We substitute the general formula for the reverse drifts. The forward drift terms −b cancel, leaving only the score-dependent terms. We also change the integration variable back to forward time t = T − τ , which means dτ = −dt and the integration limits flip. Z 0 ⋆ ⋆ ⋆ ⋆ E[XT − Xm,T |XT ] = E (D(Xt , t)st (Xt ) − D(Xm,t , t)sm,t (Xm,t )) (−dt) XT "ZT # T

(D(X⋆t , t)s⋆t (X⋆t ) − D(Xm,t , t)sm,t (Xm,t )) dt XT .

=E 0

5. Now we use the crucial fact that Sm ∈ Ktm . This means that for t ≥ tm , the law of the model process is a good approximation of the ideal law, Pm |F≥tm ≈ P⋆ |F≥tm . This implies that the paths and the score functions are approximately equal in expectation in this range: E[Dsm,t ] ≈ E[Ds⋆t ]. The error is negligible for t ≥ tm . The integrand is therefore effectively non-zero only for t < tm . The integral’s effective support reduces to: Z tm ⋆ E[XT − Xm,T |XT ] ≈ E (D(X⋆t , t)s⋆t (X⋆t ) − D(Xm,t , t)sm,t (Xm,t )) dt XT . 0

(59) 6. Finally, we connect this quantity to the expected residual of the boosting algorithm. The expected residual at step m + 1 is taken over the true data distribution p⋆0 (x, y): E[rm+1 ] = E(x,y)∼p⋆0 [y − Fm (x)]. We analyze the two terms on the right-hand side separately using the law of total expectation. 7. The expected residual at step m + 1 over the empirical data distribution is: E(x,y)∼pemp [rm+1 ] = E[y] − E[Fm (x)], 0

(60) PN 1

where the expectations are over the empirical distribution pemp = N i=1 δ(xi ,yi ) . 0 PN 8. The empirical mean E[y] = N1 i=1 yi is an unbiased estimator of the true target mean. ∗ The endpoint of the ideal reverse process, XT , has law p∗0 . Therefore: ∗

Epemp [y] = Ep∗0 [XT ] + Op (N −1/2 ), 0 where Op (N −1/2 ) is the standard Monte Carlo error. 44

(61)

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

9. The model prediction Fm (x) is the leaf value assigned by the net decision tree Tm . By construction, Fm (x) is the conditional expectation under the model’s induced joint distribution: Fm (x) = Ey∼pm,0 [y|x].

(62)

The unconditional expectation of the model’s predictions over the empirical feature distribution is: N 1 X emp Ex∼p0 [Fm (x)] = Fm (xi ). (63) N i=1 The endpoint Xm,T of the model’s reverse process has unconditional mean: Z Z E[Xm,T ] = E[y|x] pm,0 (x) dx = Fm (x) pm,0 (x) dx.

(64)

10. The key approximation relates these two expectations. Define the distribution mismatch: Z ∆m := Ex∼pemp [F (x)] − Fm (x) pm,0 (x) dx. (65) m 0 We decompose this into two error sources: ∗ [Fm (x)] [F (x)] − E ∆m = Ex∼pemp m x∼p 0 0 {z } | Sampling error ϵsamp

+ Ex∼p∗0 [Fm (x)] − Ex∼pm,0 [Fm (x)] . | {z }

(66)

Model error ϵmodel

Sampling error bound: By the central limit theorem, ϵsamp = Op (N −1/2 ). Model error bound: Since Fm is a piecewise-constant function with Lm leaves and |Fm (x)| ≤ M for all x, we have: q |ϵmodel | ≤ M · dTV (p∗0 , pm,0 ) ≤ M · Lm · DKL (p∗0 ∥pm,0 ), (67) where we used Pinsker’s inequality. By construction, the boosting algorithm minimizes the KL divergence at each step (via score matching). For a well-specified model class with sufficient capacity, DKL (p∗0 ∥pm,0 ) → 0 as m → ∞. 11. Combining the bounds from (61) and the analysis above: ⋆

⋆

E[rm+1 ] = E[y] − E[Fm (x)] ≈ E[XT ] − E[Xm,T ] = E[XT − Xm,T ]. Therefore, the expected residual is an asymptotically unbiased estimator of the integrated score error, with the bias vanishing as N → ∞ and DKL (p∗0 ∥pm,0 ) → 0. Corollary E.25 (Conditions for Exact Equivalence). The meta-score equals the integrated SDE score error exactly when: 1. N → ∞ (infinite data limit), 2. The model class contains the true conditional E[y|x] (realizability), 3. The boosting algorithm has converged: DKL (p∗0 ∥pm,0 ) < ϵ for arbitrarily small ϵ > 0. In practice, the approximation quality is controlled by the sample size N and the residual variance at step m. 45

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

E.4

Global Optimality of the Greedy Trajectory

We have proven that each step of the gradient boosting algorithm is locally optimal, guided by a meta-score that seeks to correct the integrated error of the underlying SDE’s dynamics. However, this local optimality does not, in itself, guarantee that the sequence of greedy choices made by the algorithm constitutes a globally optimal path through the space of SDEs. A myopic, locally optimal decision can sometimes lead to a suboptimal overall trajectory. To prove global optimality, we will now formalize the entire boosting process as a sequential decision problem. We will first define a single, global objective function that represents the total error over the entire trajectory. We will then use the calculus of dynamic programming, specifically the Bellman principle of optimality, to prove that the greedy, stage-wise algorithm is not a heuristic, but is in fact the provably optimal solution to this global problem. E.4.1

The Global Trajectory Score Matching (GTSM) Objective

To analyze global optimality, we must first define the ideal problem we are trying to solve. The total error of our constructed trajectory is the sum of the local errors at each step of refinement. Definition E.26 (Discrete Global Trajectory Score Matching (DGTSM) Objective). The Discrete Global Trajectory Score Matching (DGTSM) objective is the sum of the individual Denoising Score Matching losses over the entire boosting trajectory: LGTSM =

M −1 X

h i 2 E(x,y)∼p⋆0 ∥hm+1 (x) − ∇Fm log p(y|Fm (x))∥ ,

(68)

m=0

where the weak learner hm+1 is the chosen refinement operator at step m. This objective measures the total squared error in following the optimal meta-score at every stage of the trajectory construction. From Theorem E.22, we know that for the squared error loss, the optimal meta-score is proportional to the residual. The DGTSM objective is therefore equivalent to minimizing the sum of the squared errors of the weak learners with respect to the residuals at each step. This is precisely the objective that the stage-wise boosting algorithm seeks to minimize. E.4.2

Provable Optimality via the Bellman Principle

Having established a global objective, we must now prove that the specific algorithm used—greedy, stage-wise optimization—is the optimal procedure for solving it. To do this with full rigor, we now reformulate the problem in the formal language of dynamic programming and sequential decisionmaking. We first begin with a quick lemma, before moving onto our core formulation. Lemma E.27 (Finite ϵ-Net for Weak Learners). For any bounded subset Hbounded ⊂ A of weak learners with suph∈Hbounded ∥h∥∞ ≤ B and VC dimension V , and for any ϵ > 0, there exists a finite subset Aϵ ⊂ Hbounded such that: sup

min E[∥h(x) − h′ (x)∥2 ] < ϵ,

′ h∈Hbounded h ∈Aϵ

(69)

and |Aϵ | ≤ (CB/ϵ)O(V ) for some constant C. Proof. This follows from standard covering number bounds for function classes with finite VC dimension (see Theorem 2.6.7 in van der Vaart & Wellner (1996)). Decision trees with depth d have VC dimension O(d · log d), yielding polynomial-sized ϵ-nets. Next, we define the boosting process as a sequential task allowing us to invoke Dynamic Programming. Definition E.28 (The Boosting Process as a Sequential Decision Problem). The optimization of the DGTSM objective is a finite-horizon, deterministic sequential decision problem defined by the tuple (S, A, T, C): • Time Steps: The discrete boosting iterations m ∈ {0, 1, . . . , M }. 46

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

• State Space S: The space of all SDEs representable by canonical net decision trees. The state at step m is Sm ∈ S. • Action Space A: The space of all possible weak learners. The action at step m is hm+1 ∈ A. • Transition Function T : A deterministic function T : S × A → S, where the next state Sm+1 = T (Sm , hm+1 ) is uniquely defined by the partition refinement and the mapping from Section E.1.4. • Stage Cost C: A function C : S × A → R, where the cost of taking action hm+1 from state Sm is the DSM loss: C(Sm , hm+1 ) = E[∥hm+1 − rm+1 ∥2 ]. • Policy π: A sequence of decision functions, π = (π0 , . . . , πM −1 ), where πm (Sm ) = hm+1 is the action to take in state Sm . • Total Cost J(π): The DGTSM objective for a given policy: PM −1 C(S , π (S )). m m m m=0

J(π)

The optimization problem is to find an optimal policy π ⋆ that minimizes the total cost, J(π). By Lemma E.27, for each step m and precision ϵm > 0, we can restrict attention to a finite action space Aϵm that ϵm -approximates the full action space. We analyze optimality over this finite discretization, then take ϵm → 0. To solve this, we use the foundational principle of dynamic programming. Theorem E.29 (Bellman’s Principle of Optimality). An optimal policy has the property that whatever the initial state and initial decision are, the remaining decisions must constitute an optimal policy with regard to the state resulting from the first decision (Bertsekas, 2012). Theorem E.30 (Optimality of the Greedy Trajectory). The greedy policy, π G , which at each step m chooses the weak learner hm+1 that minimizes the immediate stage cost C(Sm , hm+1 ), is the globally optimal policy for the DGTSM problem. Proof. We prove this by backward induction. Let Vm (Sm ) be the optimal cost-to-go from state Sm at step m to the end of the process (i.e., the minimum possible sum of costs from step m to M − 1). By Lemma E.27, for each step m and precision ϵm > 0, we can restrict attention to a finite action space Aϵm that ϵm -approximates the full action space. We analyze optimality over this finite discretization, then take ϵm → 0. 1. Base Case (m = M − 1): At the final stage, the decision is to choose hM to transition from state SM −1 . The optimal cost-to-go is simply the minimum possible cost at this stage, as there are no future costs: VM −1 (SM −1 ) =

min

hM ∈AϵM −1

C(SM −1 , hM ).

⋆ The optimal action, πM −1 (SM −1 ), is by definition the greedy choice that minimizes this immediate cost.

2. Inductive Hypothesis: Assume that for all steps k > m, the optimal policy πk⋆ is the greedy policy. 3. Inductive Step (m): The Bellman equation for the optimal cost-to-go at step m is: Vm (Sm ) =

min [C(Sm , hm+1 ) + Vm+1 (T (Sm , hm+1 ))] .

hm+1 ∈A

Over the finite action space Aϵm , the minimum in the Bellman equation is well-defined: Vm (Sm ) =

min

hm+1 ∈Aϵm

[C(Sm , hm+1 ) + Vm+1 (T (Sm , hm+1 ))] .

The crucial insight is that the problem structure is additively separable. The total cost is a simple sum of stage costs. The term Vm+1 (T (Sm , hm+1 )) represents the optimal future cost starting from the next state, Sm+1 . By our inductive hypothesis, the policy for all future 47

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

steps is greedy and fixed. Therefore, the value Vm+1 (Sm+1 ) depends only on the state Sm+1 and not on how we choose to act at step m or any future step. Because the stage cost C(Sm , hm+1 ) depends only on the current state Sm (which defines the residuals) and the current action hm+1 , the minimization decouples. The greedy choice over Aϵm is: m hϵm+1 = arg min C(Sm , h). h∈Aϵm

As ϵm → 0, the discretization error vanishes. In the limit, the optimal policy converges to the greedy policy over the full action space: ∗ m πm (Sm ) = lim hϵm+1 = arg inf C(Sm , h). ϵm →0

h∈A

⋆ Therefore, the optimal action πm (Sm ) is the greedy action.

By the principle of backward induction, the greedy policy is globally optimal for all steps. The gradient boosting algorithm, which at every stage fits a weak learner to the residuals, is a direct implementation of this optimal greedy policy. ⋆ Corollary E.31 (The Optimal Policy is the SDE). The optimal policy π ⋆ = (π0⋆ , . . . , πM −1 ) is the ⋆ ⋆ sequence of optimal weak learners {h1 , . . . , hM }. This sequence, by construction, uniquely defines the final, optimal net decision tree TM and its corresponding canonical model, SDEM . Therefore, the optimal policy is the optimal SDE.

This completes the proof. We have shown that the simple, local, and greedy update rule of gradient boosting is not a heuristic, but a principled and provably optimal algorithm for solving a global score-matching problem over an entire coarse-to-fine trajectory in the space of continuous-time models.

Implications of the GTSM Framework: A Unifying View of Score-Based Objectives

We further explore the deeper theoretical implications of our framework, showing that a single, global objective underpins a wide variety of modern score-based training methods. It is well-known in the theory of stochastic processes that matching the marginal distributions of two processes is a weaker condition than matching their full path-space measures (Lai et al., 2025). The latter is the strongest possible notion of equivalence but is generally intractable. In this section, we first define a Continuous Global Trajectory Score Matching (CGTSM) objective for a general score-based model and prove, using Girsanov’s theorem (Oksendal, 2013), that it is equivalent to full path matching. We then demonstrate that several distinct and popular training objectives, including the simple diffusion loss, weighted diffusion losses, and consistency models can be rigorously derived as principled approximations or special cases of this single, master objective. This reveals that the boosting framework we analyze is not an isolated case but rather one instance of a broader class of algorithms that tractably solve this fundamental trajectory-matching problem. While the boosting DGTSM sums local errors over discrete refinement steps, the general CGTSM integrates score matching error over the continuous time horizon of an SDE. The boosting construction can thus be viewed as a discrete approximation of this continuous objective. F.1

The Continuous GTSM Objective and Its Equivalence to Path Matching

Definition F.1 (The Continuous GTSM Objective). Let S ∗ be the ideal SDE with law P∗ and score functions s∗t (x). Let sθ (x, t) be a parameterized score-based model. The Global Trajectory Score Matching (GTSM) objective is the integrated Fisher Divergence between the model’s score and the true score over the entire time horizon of the process: Z h i 1 T 2 LCGTSM (θ) = w(t) · Ep∗t (x) ∥sθ (x, t) − s∗t (x)∥D(t) dt, (70) 2 0 where w(t) > 0 is a weighting function and ∥v∥2D = v⊤ Dv is the Mahalanobis norm induced by the diffusion tensor D(t). 48

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

This objective is deeply connected to the KL divergence between path-space measures. Proposition F.2 (Girsanov’s Theorem for SDEs). Let P∗ be the law on path space induced by an Itô process with drift b∗ and diffusion σ. Let Pθ be the law induced by a second process with a different drift, bθ . If the Novikov condition is satisfied, the KL divergence between the two path-space measures is given by: # "Z T 1 2 −1 ∗ ∗ σ (b − bθ ) dt . (71) DKL (P ||Pθ ) = EP∗ 2 0 Corollary F.3 (CGTSM Optimality Implies Path Matching). A score model sθ achieves zero loss under the CGTSM objective with any strictly positive weighting w(t) > 0 if and only if the path-space measure Pθ it induces is identical to the ideal path-space measure P∗ . Proof. The reverse drift of a diffusion process is a function of its score: brev (x, t) = [−b(x, t) + D(x, t)st (x)]. Therefore, the difference in the drifts of two reverse-time processes is directly proportional to the difference in their score functions. Substituting this into the Girsanov formula for the KL divergence between the reverse-time path measures shows that DKL (P∗rev ||Prev,θ ) is an integral of the squared score error. The GTSM objective is a positively weighted integral of this same quantity. The objective is zero if and only if the integrand—the score error—is zero for almost all t. By Girsanov’s theorem, this implies the KL divergence is zero, which holds if and only if the path measures are identical. Remark F.4 (On the Suitability of the Novikov Condition). The validity of Girsanov’s theorem hinges on the Novikov condition, which is a regularity assumption that is met by both sides of our proposed equivalence, albeit for different reasons. For the continuous flow side (standard score-based models), the condition is typically satisfied. The drift difference, ∆brev = D(s∗t − sθ ), is well-behaved because the process operates on a finitedimensional Gaussian measure, the score model sθ is a Lipschitz-continuous neural network, and the integration is over a finite time horizon. The theoretical challenges associated with infinitedimensional function spaces, which would require the Wiener measure and different analytical tools such as those used for neural operators, are outside the scope of this standard setup. For the discrete tree side, satisfaction of the condition is a direct and non-trivial consequence of our Tree-to-Flow mapping. A naive interpretation of a tree’s splits as instantaneous jumps would indeed violate the condition. However, our dyadic refinement procedure is precisely the mechanism that guarantees a continuous-path limit for the induced probability flow. This ensures the resulting drift is well-behaved and the Novikov condition holds. The condition would fail for processes characterized by irreducible, finite jumps that cannot be smoothed away—analogous to models of punctuated equilibria. Such processes would require a different mathematical formalism based on Lévy processes and are beyond the scope of our current framework. Thus, the Novikov condition is a mild and well-justified regularity assumption within our framework, holding for standard diffusion models by construction and for tree-derived flows via our refinement theorem. F.2

Deriving Standard Training Objectives as Special Cases of the CGTSM

The CGTSM objective is the ideal, but practical algorithms must approximate it. We now demonstrate that several popular training objectives are not ad-hoc heuristics, but can be rigorously derived as principled approximations of the CGTSM, each embodying a different inductive bias about the learning trajectory. The CGTSM framework’s true utility is that it provides a language for describing and justifying these biases. The objective is an integral over the filtration of tail-equivalence classes {Kt }, and each practical objective makes a different choice about how to prioritize or approximate the score matching loss for these classes. The Simple Diffusion Loss. The most common training objective is the simple, unweighted loss (Ho et al., 2020). Proposition F.5. The simple diffusion loss, Lsimple = Et∼U (0,T ),x0 ,ϵ [∥sθ (xt , t) − 2 ∇xt log pt (xt |x0 )∥ ], is the unbiased Monte Carlo estimator of the unweighted CGTSM ob49

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

jective, where w(t) = 1. This corresponds to an inductive bias of uniform importance across all tail-equivalence classes. RT Proof. Setting w(t) = 1 in the CGTSM objective gives 12 0 Ep∗t [∥sθ − s∗t ∥2D ]dt. Using the equivalence of denoising score matching, the intractable true score s∗t can be replaced by the tractable conditional score ∇xt log pt (xt |x0 ). The objective becomes an expectation over a uniform distribution in time, Et∼U (0,T ) [. . . ]. A single-sample Monte Carlo estimator for this integral is to sample a single time t, a single data point x0 , and a single noise sample ϵ, and compute the squared error. This is precisely the simple diffusion loss objective. This choice implicitly assumes that the learning difficulty is uniform across all time, or that a simple, unbiased estimate is sufficient. The Weighted Diffusion Loss. Many state-of-the-art models use a non-uniform weighting λ(t) in their loss function (Kingma et al., 2021; Karras et al., 2024). The CGTSM framework reveals this as an explicit injection of bias. Proposition F.6. The weighted diffusion loss, Lweighted = Et,x0 ,ϵ [λ(t) · ∥sθ (xt , t) − ∇xt log pt (xt |x0 )∥2 ], is an unbiased Monte Carlo estimator of the CGTSM objective with a weighting function w(t) = λ(t). This choice injects a quantifiable bias, prioritizing the matching of score functions for specific tail-equivalence classes. Proof. The proof follows identically to the simple case. The weighting function λ(t) inside the expectation is the Monte Carlo estimator for the weighted integral where the CGTSM weight is w(t) = λ(t). This choice is motivated by (Assumption E.14). If a learner is intrinsically biased to learn the coarse tail-equivalence classes (large t) first, an engineer can inject a counter-balancing bias by choosing a weighting function λ(t) that is larger for small t. This forces the optimizer to pay more attention to the harder-to-learn, fine-grained details, thereby balancing the learning process across the entire trajectory. Consistency Models. Consistency models are a state of the art framework that enforce selfconsistency along the probability flow ODE trajectories (Song et al., 2023). We now show this injects a structural bias between tail-equivalence classes. Proposition F.7. The consistency distillation loss is an approximation of the CGTSM objective that injects a strong inductive bias on the coupling between adjacent tail-equivalence classes. Proof. The consistency loss is of the form Lconsist = E[d(fθ (xt2 , t2 ), fθ′ (xt1 , t1 ))], where fθ is the student, fθ′ is a teacher model, and xt1 , xt2 are adjacent points on an ODE trajectory. Standard DSM-based objectives learn the score for each time t (and thus for each class Kt ) independently. The consistency loss, in contrast, imposes a strong structural constraint: the model’s behavior for class Kt2 must be directly predictable from its behavior for class Kt1 . This enforces a smoothness condition along the trajectory of models. This bias is what enables rapid sampling: by learning the relationship between different points on the trajectory, the model can make larger, more informed jumps during inference. The consistency loss is therefore a specific, bootstrapped approximation of the GTSM objective that prioritizes trajectory smoothness over pointwise score accuracy. This unified perspective reveals that these different training objectives are not ad-hoc heuristics but are deeply connected, principled variations of a single, fundamental goal: matching the entire generative trajectory. The gradient boosting algorithm, as we have shown, is a provably optimal, constructive solver for this same master objective.

DSM-T REE: Discretized Score Matching for Decision Trees

Having established the CGTSM framework as a unifying objective for trajectory-based learning, we now demonstrate its first concrete algorithmic instantiation: Discretized Score Matching for Trees (DSM-T REE). This algorithm addresses a fundamental problem in machine learning: how to distill the knowledge of a decision tree—a discrete, interpretable model—into a neural network that preserves the tree’s decision boundaries while gaining the benefits of continuous, differentiable representations. 50

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

The key insight is that a decision tree defines a coarse-to-fine hierarchy of decision boundaries, and learning this hierarchy is equivalent to performing score matching over the discrete trajectory defined by tree depth levels. DSM-T REE thus creates a neural network classifier whose learned decision function faithfully replicates the tree’s partition structure. G.1

The Tree Distillation Problem

We define the tree distillation problem as the task of learning a neural network classifier fθ : X → Y that approximates a trained decision tree T : X → Y such that fθ (x) ≈ T (x) ∀x ∈ X , while maintaining a continuous, differentiable representation that can generalize beyond the tree’s exact decision boundaries. This problem presents two fundamental challenges. First, the tree’s Hierarchical Structure—a nested sequence of partitions—is lost when a network is trained only on the final outputs. Second, a naive distillation approach discards information about the unique Discrete Decision Paths each sample follows. DSM-T REE addresses both challenges by training a conditional network Mθ (x, j) that learns to predict the optimal decision at each tree level j, explicitly modeling the hierarchical trajectory.

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

G.2

Algorithm Overview

DSM-T REE operates in three phases that mirror the theoretical structure developed in Sections C and E: Algorithm 6 DSM-T REE: Neural Network Distillation from Decision Trees Base Tree Generation (Teacher Model) Train oracle model O (e.g., Random Forest) on data {(xi , yi )}N i=1 Generate pseudo-labels: ỹi = O(xi ) Train base decision tree T on {(xi , ỹi )} to depth D Conditional Score Model Training (Student Model) Initialize neural network Mθ (x, j) : Rd × {0, . . . , D − 1} → {0, 1} for training steps t = 1, . . . , T do Sample mini-batch {(xi , yi )}i∈B Sample tree levels {ji }i∈B uniformly from {0, . . . , D − 1} For each (xi , ji ): extract decision d∗i = TreeDecision(T , xi , ji ) P ground-truth ∗ Compute loss: L = i∈B I[di ̸= ⊥] · ℓCE (Mθ (xi , ji ), d∗i ) Update: θ ← θ − η∇θ L end for Neural Network Inference (Mimicking Tree Traversal) for each test sample x do Initialize: node ← root(T ) for level j = 0, . . . , D − 1 do if node is leaf then break Predict: dˆj = arg max Mθ (x, j) Navigate: node ← childdˆj (node) end for Return: ŷ = fclassifier (x) end for Output: Neural network classifier fθ that mimics tree T

The crucial innovation is that the neural network does not directly learn to map inputs to outputs. Instead, it learns the intermediate decision function at each level, thereby capturing the full hierarchical structure of the tree’s decision-making process. The key insight is that Phase 2 directly implements the GTSM objective in the discrete setting, where the ”time” variable is the tree depth level j. G.3

Formal Problem Setup

Definition G.1 (Conditional Decision Problem). Given a trained decision tree T with depth D, for each sample x and tree level j ∈ {0, . . . , D − 1}, define the optimal decision function:  0 if x should go left at level j d∗ (x, j) = 1 if x should go right at level j (72)  ⊥ if x has reached a leaf before level j where the decision is determined by the unique path through T from root to the leaf containing x. Definition G.2 (DSM-T REE Training Objective). The DSM-T REE model Mθ : Rd × {0, . . . , D − 1} → ∆1 (where ∆1 is the probability simplex over {0, 1}) is trained to minimize: LDSM (θ) = E(x,y)∼pdata Ej∼Uniform(0,D−1) [I[d∗ (x, j) ̸= ⊥] · ℓCE (Mθ (x, j), d∗ (x, j))] ,

(73)

where ℓCE is the cross-entropy loss and I[·] is the indicator function that masks out samples that have reached leaves. 52

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

G.4

Connection to CGTSM Framework

We now establish the formal connection between the DSM-T REE objective and the continuous CGTSM framework developed in Section F. Theorem G.3 (DSM-T REE as Discrete CGTSM). The DSM-T REE objective (73) is the discrete-time instantiation of the CGTSM objective with uniform weighting w(t) = 1 and time discretized to tree depth levels. Proof. We proceed by explicitly constructing the correspondence between the discrete tree hierarchy and the continuous-time SDE framework. Step 1: Recall the CGTSM objective from Section F. For a continuous-time process, CGTSM minimizes: Z h i 1 T 2 w(t) · Ep∗t (x) ∥sθ (x, t) − s∗t (x)∥D(t) dt, LCGTSM (θ) = 2 0

(74)

where s∗t (x) = ∇x log p∗t (x) is the true score function. Step 2: Discretize time to tree depth levels. By Theorem C.21, a decision tree of depth D induces a discrete sequence of densities {p(x, k)}D k=0 corresponding to the coarse-graining trajectory. We identify: j · T, j ∈ {0, 1, . . . , D}, D so that t0 = 0 (leaves, fine-grained) and tD = T (root, maximum entropy). tj =

(75)

The continuous-time integral becomes a Riemann sum: D−1

LCGTSM (θ) ≈

T X w(tj ) · Ep∗t (x) j D j=0

sθ (x, tj ) − s∗tj (x)

(76)

Step 3: Relate score matching to decision prediction. From Theorem E.24, the optimal direction of change for a sample at tree level j is given by the score function. In the discrete tree setting, this ”direction” is encoded by the binary decision d∗ (x, j). More precisely, the score function s∗tj (x) points in the direction of increasing density. In the tree’s partition Πj at level j, a sample x resides in some region Ri ∈ Πj . The optimal next step in the coarse-graining trajectory is to refine this region by splitting it into two sub-regions, Rileft and Riright . The decision d∗ (x, j) encodes which sub-region x belongs to. This is the discrete analogue of the score’s directional information. The squared score error ∥sθ −s∗ ∥2 in the continuous case corresponds to the classification error in predicting d∗ (x, j) in the discrete case. Therefore, the score-matching term becomes: sθ (x, tj ) − s∗tj (x)

←→

ℓCE (Mθ (x, j), d∗ (x, j)).

(77)

Step 4: Simplify for uniform weighting and data distribution. Setting w(t) = 1 (uniform weighting) and taking expectations with respect to the empirical data distribution pdata , the discrete CGTSM objective (76) becomes: Ldiscrete CGTSM (θ) =

D−1 X

Ex∼pdata [ℓCE (Mθ (x, j), d∗ (x, j))] .

(78)

j=0

The indicator function I[d∗ (x, j) ̸= ⊥] in (73) simply masks out samples that have reached leaves before level j, which is necessary for well-defined training. The uniform distribution over j in the expectation is the Monte Carlo sampling strategy for the sum over levels. Therefore, LDSM (θ) is exactly the discrete-time, uniformly weighted CGTSM objective. 53

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

G.5

Convergence Analysis

We now establish the finite-sample convergence guarantees for the DSM-T REE algorithm. Assumption G.4 (Realizability and Regularity). 1. The model class M = {Mθ : θ ∈ Θ} contains a function that can represent the true decision function d∗ (x, j) for all levels j. 2. The base tree T has bounded depth D < ∞ and bounded number of leaves L = O(2D ). 3. The feature space is bounded: ∥x∥ ≤ B for all x ∈ X . 4. The loss function ℓCE is G-Lipschitz continuous in the model parameters. Theorem G.5 (Finite-Sample Convergence of DSM-T REE). Under Assumption √G.4, let θ̂T be the parameters obtained after T gradient descent steps with learning rate η = O(1/ T ) and batch size B. Then with probability at least 1 − δ: ! p D · d log(L/δ) √ LDSM (θ̂T ) − LDSM (θ∗ ) ≤ O , (79) BT where θ∗ is the optimal parameter and d is the feature dimensionality. Proof. The proof follows the standard analysis of stochastic gradient descent for empirical risk minimization, adapted to the tree-structured setting. We decompose the excess risk into optimization error and generalization error: h i LDSM (θ̂T ) − LDSM (θ∗ ) = LDSM (θ̂T ) − L̂DSM (θ̂T ) | {z } Generalization error h i + L̂DSM (θ̂T ) − L̂DSM (θ∗ ) | {z }

(80)

(81)

Optimization error

h i + L̂DSM (θ∗ ) − LDSM (θ∗ ) , | {z }

(82)

Generalization error

where L̂DSM is the empirical loss on training data. The DSM-T REE objective is a finite sum over tree levels j. For each level j, the per-sample loss ℓCE (Mθ (x, j), d∗ (x, j)) is G-Lipschitz and bounded by log 2 (for binary cross-entropy). By standard SGD convergence for smooth, Lipschitz losses (Bottou et al., 2018), after T steps with √ learning rate η = O(1/ T ): 2 G ∗ L̂DSM (θ̂T ) − L̂DSM (θ ) ≤ O √ . (83) T We now, apply uniform convergence bounds for the hypothesis class M. The key is to bound the complexity of the joint hypothesis class over both features x and levels j. Define the extended hypothesis class: M̃ = {(x, j) 7→ Mθ (x, j) : θ ∈ Θ}.

(84)

The VC dimension of this class is bounded by: VC(M̃) ≤ VC(M) + log2 D,

(85)

where the log2 D term accounts for the discrete level variable. For neural networks with W parameters, VC(M) = O(W log W ) (Bartlett, 1998). However, the tree structure provides additional structure: at each level j, only O(2j ) nodes are active, so the effective VC dimension is: VCeff (j) = O (W log W + j) . (86) 54

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Averaging over all levels j ∈ {0, . . . , D − 1}: Ej [VCeff (j)] = O(W log W + D).

(87)

By the VC inequality (Vapnik, 1999), with N training samples and batch size B, the generalization error satisfies: ! r VCeff log(N/δ) LDSM (θ) − L̂DSM (θ) ≤ O . (88) BT The DSM-T REE objective sums over D tree levels. Each level corresponds to a partition Πj with at most 2j regions. The total number of decision boundaries across all levels is: D−1 X

2j = 2D − 1 = L − 1,

(89)

j=0

where L is the number of leaves. The effective sample complexity must account for learning all D levels. By a union bound over levels, the generalization error becomes: ! r D · d log(L/δ) LDSM (θ) − L̂DSM (θ) ≤ O , (90) BT where d is the feature dimensionality (assuming the neural network has O(d) parameters per level). Combining the optimization and generalization error bounds: ! r 2 G D · d log(L/δ) ∗ LDSM (θ̂T ) − LDSM (θ ) ≤ O √ +O BT T ! p D · d log(L/δ) √ =O , BT

(91)

(92)

where the second term dominates for large D and the first term is absorbed using G = O(D) (since the loss sums over levels). This completes the proof. Remark G.6 (Dependence on Tree Depth). The convergence rate’s dependence on tree depth D is unavoidable: learning the entire coarse-to-fine trajectory requires resolving all D levels of abstraction. However, the linear dependence on D is optimal for this problem, as it matches the informationtheoretic lower bound for learning a tree of depth D. Corollary G.7 (Path-Wise Consistency). As T → ∞ and BT /N → c for some constant c > 0, the DSM-T REE model Mθ̂T produces decision paths that converge to the ground-truth paths of the base tree T with probability 1: h i lim P PathMθ̂ (x) = PathT (x), ∀x ∈ X = 1. (93) T →∞

Proof. By Theorem G.5, LDSM (θ̂T ) → LDSM (θ∗ ) as T → ∞. Since the loss is a sum of crossentropy terms over all levels: LDSM (θ) =

D−1 X

Ex [ℓCE (Mθ (x, j), d∗ (x, j))] ,

(94)

j=0

and cross-entropy is zero if and only if Mθ (x, j) = d∗ (x, j) almost surely, we have: LDSM (θ) = 0 ⇐⇒ Mθ (x, j) = d∗ (x, j) for all x, j.

(95)

Since θ∗ is the minimizer by Assumption G.4, this implies exact decision matching at every level. The path through the tree is determined by the sequence of decisions {dj }D−1 j=0 , so exact decision matching implies exact path matching. 55

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

G.6

Computational Complexity

Proposition G.8 (Training Complexity). The computational complexity of training DSM-T REE for T steps with batch size B on a tree of depth D is: O(T · B · D · Cnet ), where Cnet is the cost of a forward-backward pass through the neural network Mθ . Proof. Each training step’s complexity is dominated by several operations. For a batch of size B, the algorithm must: (1) sample data points and tree levels (O(B)); (2) traverse the teacher tree for each sample to find its ground-truth decision, which takes O(D) time per sample, for a total of O(BD); and (3) perform a forward and backward pass through the neural network, which costs O(B · Cnet ). The network cost Cnet itself depends on the input size, which includes the feature dimension and the level embedding dimension. Since the tree traversal and network passes are the most expensive operations per batch, and this is repeated for T training steps, the total complexity is driven by the sum of these costs, leading to the stated O(T · B · D · Cnet ) complexity. Remark G.9 (Comparison to Standard Tree Training). Standard decision tree training (e.g., CART) has complexity O(N · d · D log D), where N is the number of samples. DSM-T REE’s complexity is O(T · B · D · Cnet ). Since Cnet = O(d · h) for a network with hidden dimension h, and typically T ·B ≈ N (one epoch), the complexities are comparable when h = O(log D). However, DSM-T REE offers the advantage of learning a flexible, continuous representation of the tree structure.

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

T REE F LOW: Conditional Flow Matching with Tree-Structured Paths

We now present our second algorithmic instantiation: T REE F LOW, which uses decision tree partitions to provide structured conditioning paths for conditional flow matching. Unlike DSM-T REE which creates neural classifiers by distilling tree boundaries, T REE F LOW leverages tree-structured paths as ground truth trajectories for training generative flow models. H.1

Algorithm Overview

T REE F LOW combines conditional flow matching with tree-based partition encodings to create a partition-aware generative model. Algorithm 7 T REE F LOW: Tree-Conditioned Flow Matching 1: Input: Dataset D = {(xi , yi )}N i=1 , tree depth D 2: 3: Learn Data Partitioning Structure 4: Train decision tree T on D to depth D 5: For each sample xi , compute path encoding: pi = PathEncoder(T , xi ) 6: // pi ∈ RK encodes which leaf partition and path xi belongs to 7: 8: Train Conditional Flow Matching Model 9: Initialize velocity field vθ (x, t, p, y) : Rd × [0, 1] × RK × Y → Rd 10: for training steps s = 1, . . . , S do 11: Sample mini-batch {(xdata i , pi , yi )}i∈B from training data 12: Sample noise: xnoise ∼ N (0, I) for each i ∈ B i 13: Sample time: t ∼ Uniform(0, 1) 14: // Flow Matching: Linear interpolation defines the flow path (t)

Interpolate: xi = t · xdata + (1 − t) · xnoise i i Compute target velocity: vi∗ = xdata − xnoise i i // Learn velocity conditioned on tree path pi and label yi P (t) 18: Compute loss: L = i∈B ∥vθ (xi , t, pi , yi ) − vi∗ ∥2 19: Update: θ ← θ − η∇θ L 20: end for

15: 16: 17:

21: 22: Partition-Targeted Generation 23: for each target label ytarget and desired partition do 24: Sample reference from target partition: 25: xref ∼ {xi : yi = ytarget and xi ∈ Rtarget } 26: Compute conditioning path: pref = PathEncoder(T , xref ) 27: // Path pref guides generation to respect tree structure 28: Initialize: x(0) ∼ N (0, I) 29: for t = 0 to 1 with step size ∆t do 30: // Integrate learned tree-conditioned velocity field 31: x(t+∆t) ← x(t) + vθ (x(t) , t, pref , ytarget ) · ∆t 32: end for 33: Return synthetic sample: x̃ = x(1) 34: end for 35: Output: Synthetic dataset D̃ matching partition structure of real data

Remark H.1 (T REE F LOW and Conditional Flow Matching). T REE F LOW extends the Flow Matching framework (Lipman et al., 2023), which trains flow models via direct regression to conditional velocity fields along linear interpolation paths x(t) = txdata + (1 − t)xnoise . The tree does not define these trajectories; rather, it provides conditioning information p that guides the velocity network vθ (x, t, p, y) to learn partition-specific flows, allowing the model to specialize to different regions of the data space based on its hierarchical organization. 57

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

H.2

Convergence Analysis for T REE F LOW

We now establish finite-sample convergence guarantees for T REE F LOW as a path-conditioned generative model. Assumption H.2 (T REE F LOW Regularity Conditions). 1. The data distribution pdata (x, y) has bounded support: ∥x∥ ≤ B for all x ∈ supp(pdata ). 2. The base tree T has depth D and L = O(2D ) leaves, with balanced partitioning (each leaf has comparable probability mass). 3. The velocity network vθ is G-Lipschitz continuous in x and θ. 4. The path encoding p = PathEncoder(T , x) is deterministic and has bounded norm: ∥p∥ ≤ P. Theorem H.3 (Finite-Sample Convergence of T REE F LOW). Under Assumption √H.2, let θ̂S be the parameters obtained after S gradient descent steps with learning rate η = O(1/ S) and batch size B. Then with probability at least 1 − δ: ! p d · L · log(L/δ) √ , (96) LT REE F LOW (θ̂S ) − LT REE F LOW (θ∗ ) ≤ O BS where θ∗ is the optimal parameter and d is the feature dimensionality. Proof. The proof follows a similar structure to the DSM-T REE convergence proof (Theorem G.5), with additional considerations for the continuous-time flow matching objective. As before, we decompose: LT REE F LOW (θ̂S ) − LT REE F LOW (θ∗ ) = (Generalization Error) + (Optimization Error).

(97)

The T REE F LOW objective is: h i LT REE F LOW (θ) = Et,x(0) ,x(1) ,p,y ∥vθ (x(t) , t, p, y) − (x(1) − x(0) )∥2 .

(98)

The integrand is G2 -Lipschitz continuous in θ (by the chain rule and the Lipschitz assumption on vθ ) and is bounded: ∥vθ (x(t) , t, p, y) − (x(1) − x(0) )∥2 ≤ (2B)2 = 4B 2 , (99) √ (1) (0) (0) since both x and x are bounded by B (data) and ∥x ∥ ≤ d (Gaussian with high probability). By standard SGD convergence for smooth, bounded losses, after S steps: 2 2 G B 1 √ L̂T REE F LOW (θ̂S ) − L̂T REE F LOW (θ∗ ) ≤ O =O √ , S S

(100)

where we absorbed constants. The hypothesis class for T REE F LOW is: F = {(x, t, p, y) 7→ vθ (x, t, p, y) : θ ∈ Θ}.

(101)

The key insight is that the path conditioning p partitions the hypothesis class into L sub-classes, one for each leaf. For a fixed leaf ℓ, the restricted hypothesis class is: Fℓ = {(x, t) 7→ vθ (x, t, pℓ , y) : θ ∈ Θ}.

(102)

The Rademacher complexity of the full class is bounded by the sum over leaves (Mohri et al., 2018): RN (F) ≤

L X

P(X ∈ Rℓ ) · RNℓ (Fℓ ),

ℓ=1

where Nℓ = N · P(X ∈ Rℓ ) is the expected number of samples in leaf ℓ. 58

(103)

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

p For neural networks with W parameters, RN (Fℓ ) = O( W/Nℓ ) under standard assumptions (Bartlett et al., 2017). Under the balanced partitioning assumption, P(X ∈ Rℓ ) ≈ 1/L, so: s ! ! r L X 1 W W ·L RN (F) ≤ ·O =O . (104) L N/L N ℓ=1

Since the velocity network has O(d) parameters per dimension (assuming a reasonable architecture), W = O(d · D) where D accounts for the depth-dependent embedding. Therefore: ! ! r r d·D·L d · L · log L RN (F) = O =O , (105) N N using D = O(log L) for balanced trees. By standard Rademacher complexity generalization bounds, with N training samples and batch size B, after S steps (so BS total samples seen): ! r d · L · log(L/δ) LT REE F LOW (θ) − L̂T REE F LOW (θ) ≤ O . (106) BS Combining the optimization and generalization errors: ! r 1 d · L · log(L/δ) ∗ LT REE F LOW (θ̂S ) − LT REE F LOW (θ ) ≤ O √ . +O BS S

(107)

For large L, the second term dominates (assuming S and B are chosen such that BS ≈ N ), giving: ! p d · L · log(L/δ) ∗ √ . (108) LT REE F LOW (θ̂S ) − LT REE F LOW (θ ) ≤ O BS This completes the proof. √ Remark H.4 (Sample Complexity vs. Tree Complexity). The convergence rate’s dependence on L reflects a fundamental tradeoff: deeper trees (larger L) can represent finer-grained partitions and thus more expressive generative models, but require more training data to learn accurately. This is analogous to the bias-variance tradeoff in classical statistics. Corollary H.5 (Distributional Convergence). Under Assumption H.2, as S → ∞ and BS/N → c for some constant c > 0, the distribution of samples generated by T REE F LOW converges in Wasserstein-2 distance to the conditional distributions of the training data within each leaf partition: lim W2 (pT REE F LOW (x|pℓ , y), pdata (x|X ∈ Rℓ , y)) = 0, (109) S→∞

where W2 is the Wasserstein-2 distance. Proof. The Wasserstein-2 distance between two distributions with finite second moments can be bounded by the L2 loss of their score functions (Song et al., 2021). Specifically, for two distributions p and q on Rd with scores sp and sq : Z T 2 W2 (p, q) ≤ C · Ex∼pt ∥sp (x, t) − sq (x, t)∥2 dt, (110) 0

for some constant C depending on the process parameters. By Theorem H.10, T REE F LOW minimizes the CGTSM objective with path conditioning, which is equivalent to minimizing the integrated score-matching loss. By Theorem H.3, LT REE F LOW (θ̂S ) → LT REE F LOW (θ∗ ) as S → ∞. Since the optimal parameter θ∗ achieves zero loss (by the realizability assumption implicit in our setup), this implies that the learned velocity field converges to the true conditional velocity field: vθ̂S (x, t, pℓ , y) → v ∗ (x, t|X ∈ Rℓ , y) as S → ∞. (111) By the connection between velocity fields and score functions (Step 1 of Theorem H.10), this implies score function convergence. By the Wasserstein bound above, this implies distributional convergence in W2 . 59

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Remark H.6 (Practical Implications). Corollary H.5 guarantees that with sufficient training, T REE F LOW generates synthetic samples whose distribution within each tree partition matches the real data distribution. This is a stronger guarantee than standard generative models provide: T REE F LOW not only matches the overall marginal distribution but also preserves the hierarchical structure encoded by the tree. H.3

Computational Complexity

Proposition H.7 (Training Complexity of T REE F LOW). The computational complexity of training T REE F LOW for S steps with batch size B using a tree with L leaves is: O(S · B · (L + D · Cnet )),

(112)

where Cnet is the cost of a forward-backward pass through the velocity network and D is the tree depth. Proof. Each training step requires: 1. Sampling B data points and computing their path encodings: O(B · D) (traversing tree to depth D) 2. Sampling noise x(0) and time t: O(B · d) 3. Computing interpolations x(t) : O(B · d) 4. Forward pass through velocity network: O(B · Cnet ) 5. Computing loss and gradients: O(B · d) 6. Backward pass and parameter update: O(B · Cnet ) The dominant terms are the path encoding (which requires O(D) traversal per sample) and the network forward-backward passes. For a network with hidden dimension h, Cnet = O(d · h + K · h) where K is the path encoding dimension (typically K = L for full decision path representation). Multiplying by S steps and simplifying: O(S · B · (D + d · h + L · h)) = O(S · B · (L + D · Cnet )),

(113)

where we used Cnet = O(d · h) and absorbed the L · h term into L (since typically h = O(d) and L ≫ d for deep trees). Remark H.8 (Comparison to Standard Diffusion Models). Standard diffusion models have training complexity O(S · B · Cnet ). T REE F LOW adds an overhead of O(L) per batch from the path encoding computation. However, this overhead is typically negligible compared to the network cost Cnet for reasonably sized trees (e.g., L = 256 leaves is small compared to typical network costs). The benefit is that T REE F LOW’s path conditioning provides much richer structural inductive bias, often leading to faster convergence and better sample quality. H.4

Unified View: DSM-T REE and T REE F LOW as GTSM Instantiations

We conclude by highlighting the deep structural unity between DSM-T REE and T REE F LOW through the CGTSM lens. Proposition H.9 (Unified CGTSM Framework). Both DSM-T REE and T REE F LOW are solutions to the CGTSM objective under different problem settings: 1. DSM-T REE solves CGTSM for discriminative modeling: learn the coarse-to-fine trajectory of decision boundaries that minimizes classification error. 2. T REE F LOW solves CGTSM for generative modeling: learn the coarse-to-fine trajectory of distributions that minimizes generation error (Wasserstein distance). 60

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Both achieve this by explicitly modeling the hierarchical structure encoded in decision trees as a trajectory through tail-equivalence classes (in the discriminative case) or as path-conditioned flow trajectories (in the generative case). The algorithms differ in their supervision source: • DSM-T REE: Supervised by the tree’s discrete decisions d∗ (x, j) ∈ {0, 1} at each level. • T REE F LOW: Supervised by conditional flow matching with linear interpolation paths x(t) = tx(1) + (1 − t)x(0) , conditioned on tree paths. But both share the same fundamental principle: they discretize the continuous CGTSM trajectory into computationally tractable objectives by leveraging the hierarchical structure of decision trees. This unity demonstrates the power of the CGTSM framework as a universal language for understanding trajectory-based machine learning algorithms. Theorem H.10 (T REE F LOW as Path-Conditioned CGTSM). The T REE F LOW conditional flow matching objective is equivalent to the CGTSM objective with path-specific weighting induced by tree partitions. Proof. We establish the connection through three steps: relating flow matching to score matching, incorporating path conditioning, and showing equivalence to CGTSM with structured weighting. Step 1: Flow Matching as Score Matching. Conditional Flow Matching (Lipman et al., 2023) trains a velocity field vθ (x, t) by minimizing: h i LCFM (θ) = Et,x(0) ,x(1) ∥vθ (x(t) , t) − (x(1) − x(0) )∥2 , (114) where x(t) = tx(1) + (1 − t)x(0) is the linear interpolation. A fundamental result by Lipman et al. (2023) shows that this objective is equivalent to denoising score matching. Specifically, the conditional velocity field vt (x(t) |x(1) ) = x(1) − x(0) is related to the conditional score by: vt (x(t) |x(1) ) = σ 2 (t)∇x(t) log pt (x(t) |x(1) ),

(115)

where σ 2 (t) = t(1 − t) for linear interpolation. Therefore, minimizing the CFM loss is equivalent to performing denoising score matching with the conditional distribution pt (x(t) |x(1) ). Step 2: Path Conditioning Induces Structured Weighting. T REE F LOW extends the CFM objective by conditioning on both the target sample x(1) and its tree path encoding p: h i LT REE F LOW (θ) = Et,x(0) ,(x(1) ,p,y) ∥vθ (x(t) , t, p, y) − (x(1) − x(0) )∥2 . (116) The path encoding p identifies which leaf partition Rℓ the sample x(1) belongs to. Therefore, we can decompose the expectation by leaf: LT REE F LOW (θ) =

L X

h i P(X ∈ Rℓ ) · Et,x(0) ,x(1) ∈Rℓ ∥vθ (x(t) , t, pℓ , y) − (x(1) − x(0) )∥2 , (117)

ℓ=1

where pℓ is the path encoding for leaf Rℓ . This decomposition reveals that T REE F LOW implicitly assigns a weight wℓ = P(X ∈ Rℓ ) to each leaf partition. This is precisely the structure of a path-dependent weighting in the CGTSM framework. Step 3: Equivalence to CGTSM with Tree-Induced Weighting. Recall from Section F that the CGTSM objective is: Z h i 1 T 2 LCGTSM (θ) = w(t) · Ep∗t (x) ∥sθ (x, t) − s∗t (x)∥ dt. 2 0 61

(118)

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

From Step 1, we know that flow matching is score matching. From Step 2, we see that T REE F LOW’s path conditioning induces a decomposition by leaf partitions. The connection to CGTSM is now clear: the tree structure defines a specific weighting function: wtree (t, x) =

L X

I[x ∈ Rℓ ] · P(X ∈ Rℓ ),

(119)

ℓ=1

which is a position-dependent and partition-dependent weight. More precisely, by Theorem C.21, the tree’s hierarchical coarse-graining induces a time-dependent partition sequence {Πtj } where each partition Πtj at ”time” tj = j/D corresponds to tree level j. The CGTSM objective with tree-induced weighting becomes: Ltree CGTSM (θ) =

D−1 X

h i P(X ∈ R) · Ex∈R ∥sθ (x, tj ) − s∗tj (x)∥2 .

(120)

j=0 R∈Πtj

At the finest level (j = D − 1), this sum over regions R ∈ ΠtD−1 is exactly the sum over leaves in T REE F LOW’s objective. Therefore: LT REE F LOW (θ) = Ltree CGTSM (θ)

, t=tD−1

(121)

plus an integration over time t ∈ [0, 1] for the flow matching formulation. This establishes that T REE F LOW is the CGTSM objective with tree-structured, path-dependent weighting. Remark H.11 (Comparison to Standard Diffusion Models). Standard diffusion models use a uniform prior p(x(0) ) = N (0, I) and learn a single unconditional score function sθ (x, t) or sθ (x, t|y) for class conditioning. T REE F LOW additionally conditions on the path encoding p, which provides much richer structural information: it encodes the entire hierarchical decision-making process that led to the leaf containing the target sample. This extra conditioning allows T REE F LOW to learn separate flow trajectories for different regions of the data space, effectively partitioning the CGTSM objective by tree structure.

Experimental Results

This appendix provides detailed descriptions of the experimental setups, model architectures, and training procedures used to generate the results in Section 5. All code was implemented in Python using PyTorch (Paszke et al., 2019) for neural networks and Scikit-learn (Pedregosa et al., 2011) for tree-based models and data processing. All our experiments used a single NVIDIA A100 GPU. I.1 I.1.1

Details for Verifying the Equivalence (Experiments 1 & 2) Implicit Tree Structure Discovery (Experiment 1)

Datasets We use three synthetic 2D datasets generated using sklearn.datasets.make blobs with 3200 samples each to provide clear, visualizable cluster structures. • 4 Corners: 4 clusters with centers at (±2, ±2) and a standard deviation of 0.3. • 9 Grid: 9 clusters in a 3 × 3 grid from (−2, −2) to (2, 2) with a standard deviation of 0.25. • 8 Gaussians: 8 clusters arranged in a circle of radius 2 with a standard deviation of 0.15. All datasets are standardized using StandardScaler. Diffusion Model The score network is an MLP implemented in PyTorch. It consists of an input layer mapping the concatenated (x, t) ∈ R2+1 to 128 units, followed by four hidden blocks of (Linear(128) → ReLU), and an output Linear(128) → 2 layer. 62

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Training The model is trained for 400 epochs using the Adam optimizer (Kingma & Ba, 2015) with a learning rate of 10−3 and a cosine annealing scheduler. The loss is the standard MSE between the predicted and true noise, as is common in denoising diffusion models (Ho et al., 2020). The forward process uses a linear beta schedule from 10−4 to 0.02 over N = 100 steps. Hierarchy Discovery The procedure to extract the tree structure is a direct empirical implementation of the theoretical concept from §2.2, performing an agglomerative clustering in the time domain. It discovers the hierarchy encoded by the model’s learned dynamics, not the analytical forward process. The steps are as follows: 1. Initialize Clusters: The process begins with each of the K ground-truth data classes corresponding to a distinct, active cluster at time t = 0. 2. Simulate Learned Forward SDE: For each initial cluster, we simulate its evolution forward in time from t = 0 to t = T . This is not the simple analytical forward process used for training. Instead, we use a discretized Euler-Maruyama scheme to solve the learned forward SDE, where the drift at each step is determined by the model’s own score function: flearned (x, t) = [−0.5βt x − 0.5βt sθ (x, t)]. This critical step ensures we are analyzing the dynamics the model has actually learned. 3. Track Centroid Trajectories: During the simulation, for each cluster and at each time step ti , we compute and store two quantities: the geometric centroid of the cluster’s points and their average spread (mean distance from the centroid). This yields a full trajectory for each cluster’s mean and variance over time. 4. Iterative Merge Search: The algorithm proceeds via agglomerative clustering. It iterates until only one cluster remains. In each iteration, it searches for the next merge event by checking every possible pair of currently active clusters to find which pair becomes indistinguishable at the earliest future time. 5. Merge Criterion: Two clusters are defined as ”merged” at the first time step ti where the Euclidean distance between their centroids becomes smaller than the sum of their spreads. This criterion signifies the moment their probability distributions have substantially overlapped. 6. Dendrogram Construction: The pair of clusters with the minimum merge time is formally merged into a new, larger cluster. The event comprising the two original cluster IDs, the merge time ti , and the number of original leaves in the new cluster, is recorded. This sequence of recorded merge events directly forms the linkage matrix for the dendrogram, where the vertical axis now represents any discovered merger time t. The reverse PF-ODE is subsequently used only for visualization purposes. Additional results for the 9-Grid and 8-Gaussians datasets are shown in Figure 12.

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

I.1.2

Information Decay Analysis (Experiment 2)

Datasets We use the MNIST (LeCun et al., 1998), Fashion-MNIST (Xiao et al., 2017), and USPS datasets (Hull, 1994). USPS images are resized to 28 × 28 for consistency. Tree Entropy Calculation A DecisionTreeClassifier with max depth=15 is trained. The resulting model achieves strong performance on the test sets, with classification accuracies of 77.05% on MNIST, 74.15% on Fashion-MNIST, and 85.35% on USPS. These accuracies, being substantially higher than random chance (10%), validate the tree as a high-performing discriminative model. This performance indicates that the tree’s learned hierarchy is not arbitrary; its sequence of splits successfully partitions the feature space in a way that is meaningful for classification. Therefore, we consider the tree’s structure and its corresponding information decay schedule (from low-entropy leaves to high-entropy root) to be a faithful and representative benchmark for the coarse-to-fine organization of the data, against which the diffusion process can be meaningfully compared. For each depth level, we compute the weighted average of the Shannon entropy (base 2) of the class distributions within each node at that level. This is normalized by the maximum possible entropy, log2 (num classes). Diffusion Entropy Proxy We use the analytical forward process. The Signal-to-Noise Ratio at time step t is defined as SNR(t) = αt /(1 − αt ). Our entropy proxy is defined as 1/(1 + SNR(t)). This function maps SNR ∈ [0, ∞] to an entropy-like value in [0, 1]. Prototype Visualization Tree prototypes are the pixel-wise average of all training images passing through a given node. Diffusion prototypes are the states of a single sample after applying the forward diffusion process at various times. Additional results for Fashion-MNIST and USPS are shown in Figure 13. I.2 I.2.1

Details for Algorithmic Instantiations (Experiments 3 & 4) DSM-T REE (Experiment 3)

Datasets We use five publicly available tabular datasets from the UCI repository (Dua & Graff, 2019): Digits (8x8), German Credit, Boston Housing (converted to binary classification), Heart Disease, and Abalone (3-class classification). All features are standardized. Methodology

The experiment follows three phases:

1. Teacher Model Generation: An oracle RandomForestClassifier (100 estimators, depth 15) is trained. A single DecisionTreeClassifier (depth 15), which serves as our ”Base Tree” baseline, is then trained on the pseudo-labels from this oracle. 2. DSM-T REE Training: A ConditionalSplitModel is trained. This network uses an embedding layer for the tree level j (embedding dim=32) and a 2-hidden-layer MLP (256 units each, with ReLU and BatchNorm) to predict the split decision. It is trained for 30,000 steps (batch size 256) using Adam with a learning rate of 10−3 . 3. Inference: To make a prediction, the DSM-T REE model is queried iteratively from level j = 0 to simulate traversal down the tree until a leaf is reached. Results The detailed performance metrics are provided in Table 2 with a magnified visualization in Figure 14. The results demonstrate that the DSM-T REE model, a fully differentiable neural network, can successfully learn the complex, hierarchical decision logic of a strong tree-based model, achieving comparable and sometimes superior performance. I.2.2

T REE F LOW (Experiment 4)

Datasets We use a suite of five standard tabular datasets: Adult, Breast Cancer, Diabetes, Wine, and a synthetic California Housing dataset. All are framed as classification tasks and standardized. 64

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Table 2: Detailed performance metrics for DSM-T REE vs. the Base Tree baseline. The performance gap is calculated as (DSM-T REE- Base Tree), where positive values indicate outperformance by DSM-T REE. Dataset

Model

Accuracy

Macro F1-Score

Cohen’s Kappa

Digits (8x8)

Base Tree (Baseline) DSM-T REE Model Performance Gap

84.63% 81.11% -3.52%

0.8449 0.8125 -0.0324

0.8289 0.7900 -0.0389

German Credit

Base Tree (Baseline) DSM-T REE Model Performance Gap

65.67% 64.67% -1.00%

0.6066 0.5870 -0.0196

0.2145 0.1742 -0.0403

Boston Housing (Classif.)

Base Tree (Baseline) DSM-T REE Model Performance Gap

84.87% 82.89% -1.97%

0.8463 0.8275 -0.0188

0.6948 0.6598 -0.0350

Heart Disease

Base Tree (Baseline) DSM-T REE Model Performance Gap

71.60% 75.31% +3.70%

0.6934 0.7353 +0.0419

0.3894 0.4720 +0.0826

Abalone (Classif.)

Base Tree (Baseline) DSM-T REE Model Performance Gap

61.80% 54.07% -7.73%

0.6059 0.5310 -0.0749

0.4045 0.2935 -0.1110

Baselines We compare T REE F LOW against four strong baselines: GaussianCopulaSynthesizer, TVAE, and CTGANSynthesizer from the Synthetic Data Vault (sdv) library (Patki et al., 2016), and TabDDPM (Kotelnikov et al., 2023). Methodology 1. Tree Encoder: A DecisionTreeClassifier (depth 10) is trained. A TreePathEncoder class converts the decision path of any sample into a sparse vector encoding, where the value at an index is the inverse of the node’s depth. 2. T REE F LOW Model: The model is a conditional MLP that takes as input (x, t, p, y) and outputs a velocity. It uses embeddings for y (embedding dim=16) and a 2-hidden-layer MLP (512 units each, with SiLU and LayerNorm). It is trained for 1000 steps using AdamW (lr = 10−3 ) on the conditional flow matching MSE loss. 3. Generation: We provide a target class label y and a path encoding p (sampled from a real data point in the desired partition). The model integrates the velocity field via 50 Euler steps from a standard Gaussian noise sample. Evaluation Metrics • TSTR Accuracy (Utility): A RandomForestClassifier (100 estimators) is trained on the synthetic data and evaluated on the real test set. • Wasserstein Distance (Fidelity): The average 1-D Wasserstein distance between the marginals of the real and fake test data. • Correlation Error (Structure): The Frobenius norm of the difference between the correlation matrices of the real and fake test data. • Runtime (Efficiency): Total training time in seconds. Results The aggregated mean and standard deviation of all metrics across 5 runs are presented in Table 3. T REE F LOW demonstrates a superior trade-off between utility and efficiency, often matching or exceeding the best-performing models in TSTR Accuracy while being more than twice as fast as other diffusion-based methods. A full page visualization is also available in Figure 15.

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Table 3: Aggregated results for T REE F LOW vs. baselines across 5 runs. TSTR Acc measures utility (↑), while Wasserstein Distance, Correlation Error, and Runtime measure fidelity, structure, and efficiency, respectively (↓). Bold indicates the best-performing model for each metric. Wasserstein

TSTR Acc

Corr Error

Dataset

Model

mean

std

mean

std

mean

std

mean

std

Adult

CTGAN GaussianCopula TVAE TabDDPM T REE F LOW

0.113 0.136 0.182 0.416 0.200

0.011 0.010 0.007 0.036 0.005

0.734 0.741 0.684 0.788 0.816

0.016 0.016 0.014 0.016 0.013

16.738 19.780 55.823 26.397 16.554

3.428 4.252 4.661 5.287 3.289

18.365 7.810 12.865 4.024 1.902

0.200 0.146 0.372 0.135 0.100

California Synth

CTGAN GaussianCopula TVAE TabDDPM T REE F LOW

0.894 0.190 0.531 0.086 0.087

0.154 0.191 0.059 0.006 0.008

0.591 0.631 0.592 0.951 0.941

0.050 0.019 0.072 0.010 0.009

0.623 0.490 0.851 0.497 0.514

0.084 0.058 0.109 0.037 0.070

4.581 0.701 3.981 4.012 1.920

0.043 0.066 0.110 0.091 0.066

Cancer

CTGAN GaussianCopula TVAE TabDDPM T REE F LOW

0.637 0.229 0.499 0.180 0.134

0.100 0.072 0.049 0.052 0.024

0.660 0.653 0.589 0.936 0.939

0.105 0.074 0.132 0.015 0.007

14.484 4.241 11.330 3.300 3.689

0.205 0.924 0.274 1.125 0.499

7.945 2.047 7.323 3.941 1.832

0.059 0.063 0.149 0.038 0.026

Diabetes

CTGAN GaussianCopula TVAE TabDDPM T REE F LOW

0.710 0.398 0.523 0.186 0.179

0.127 0.254 0.077 0.024 0.023

0.517 0.507 0.502 0.719 0.699

0.046 0.091 0.056 0.016 0.025

3.681 1.632 2.418 1.095 1.265

0.224 0.556 0.329 0.184 0.131

2.923 0.555 2.281 3.834 1.793

0.071 0.025 0.056 0.012 0.005

Wine

CTGAN GaussianCopula TVAE TabDDPM T REE F LOW

0.764 0.265 0.509 0.213 0.214

0.098 0.032 0.049 0.019 0.018

0.544 0.459 0.448 0.967 0.981

0.104 0.061 0.161 0.033 0.023

5.027 2.352 4.683 2.172 2.214

0.361 0.197 0.445 0.672 0.565

2.950 0.742 2.265 3.901 1.816

0.067 0.040 0.077 0.048 0.045

(a) 9-Grid Dataset

Runtime

(b) 8-Gaussians Dataset

Figure 12: Additional Results for Implicit Tree Structure Discovery (Experiment 1). These figures supplement Figure 2 from the main paper, demonstrating that our time-domain clustering method successfully discovers the learned hierarchical structure for more complex cluster arrangements. For each dataset, we show (a) the original labeled data clusters, (b) the discovered dendrogram where the vertical axis is the merger time t, and (c) a visualization of the system’s state at an intermediate time t = 0.5, generated using the learned reverse PF-ODE. Zoom for clarity.

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

(a) Fashion-MNIST Dataset

(b) USPS Dataset

Figure 13: Additional Results for Information-Theoretic Analysis (Experiment 2). These figures supplement Figure 3 from the main paper, showing the analogous information decay between decision trees and diffusion on additional datasets. (a) On Fashion-MNIST, the entropy decay curves show strong qualitative agreement, and the visual prototypes illustrate the gradual loss of identifiable features (e.g., the shape of a bag). (b) On USPS, the information decay trajectories are nearly identical to those observed on MNIST, reinforcing the generality of the core equivalence.

Figure 14: Magnified View of DSM-T REE Performance. Classification accuracy of the DSMT REE model compared to its teacher (Base Tree). Performance is nearly identical on most datasets, with DSM-T REE outperforming on the Heart Disease dataset, which demonstrates successful knowledge transfer from the discrete tree to the continuous neural network. This full-page view is provided for enhanced clarity of the accuracy percentages and dataset labels.

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Figure 15: Full-Page View of T REE F LOW Performance. Comparative performance of T REE F LOW against baseline generative models across a suite of tabular benchmarks. We evaluate on four axes: Utility (TSTR Accuracy ↑), Fidelity (Wasserstein Distance ↓), Structure (Correlation Error ↓), and Efficiency (Runtime ↓). T REE F LOW consistently demonstrates state-of-the-art utility, often matching or exceeding the performance of more computationally intensive diffusion models like TabDDPM, while being significantly more efficient. This full-page view is provided for enhanced clarity of the results and labels.

Record · ID 151797 · SHA-256 9a9663d338ff2add
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.