Direct Product Flow Matching: Decoupling Radial and Angular Dynamics for Few-Shot Adaptation
arXiv:2605.05054v1 [cs.CV] 6 May 2026
Hongxu Chen1 , Yanghao Wang1 , Bowei Zhu2 , Hongxiang Li1 , Zhen Wang1 , Ziqi Jiang1 , Lin Li1 , Rui Liu3 , Long Chen1∗ 1 HKUST, 2 USTC, 3 Huawei Research
Abstract Recent flow matching (FM) methods improve the few-shot adaptation of visionlanguage models, by modeling cross-modal alignment as a continuous multi-step flow. In this paper, we argue that existing FM methods are inherently constrained by incompatible geometric priors on pre-trained cross-modal features, resulting in suboptimal adaptation performance. We first analyze these methods from a polar decomposition perspective (i.e., radial and angular sub-manifolds). Under this new geometric view, we identify three overlooked limitations in them: 1) Angular dynamics distortion: The radial-angular coupling induces non-uniform speed on the angular sub-manifold, leading to regression training difficulty and extra truncation errors. 2) Radial dynamics neglect: Feature normalization discards modality confidence, failing to distinguish out-of-distribution and in-distribution data, and abandoning crucial radial dynamics. 3) Context-agnostic unconditional flow: Dataset-specific information loss during pre-trained cross-modal feature extraction remains unrecovered. To resolve these issues, we propose warped product flow matching (WP-FM), a unified Riemannian framework that reformulates alignment on a warped product manifold. Within this framework, we derive direct product flow matching (DP-FM) by introducing a constant-warping metric, which yields a decoupled cylindrical manifold (i.e., direct product manifold). DP-FM enables independent radial evolution and constant-speed angular geodesic transport, effectively eliminating angular dynamics distortion while preserving radial consistency. Meanwhile, we incorporate classifier-free guidance by conditioning the flow on the pre-trained VLMs’ hidden states to inject missing dataset-specific information. Extensive results across 11 benchmarks have demonstrated that DP-FM achieves a new state-of-the-art for multi-step few-shot adaptation.
1
Introduction
The success of vision-language models (VLMs) [1, 2, 3, 4, 5], such as CLIP [4], fundamentally relies on cross-modal alignment. This capability embeds semantically corresponding visual and textual signals into a shared, expressive latent space, where their alignment is primarily characterized by angular similarity. This precise alignment serves as the core mechanism enabling robust generalization and semantic understanding across diverse downstream tasks. However, when transferring these pre-trained VLMs to specific target domains with limited data, the pre-trained alignment often suffers from domain shift, requiring domain-specific fine-tuning (i.e., few-shot adaptation). Traditionally, such adaptation relies on parameter-efficient fine-tuning (PEFT) methods [6, 7, 8, 9], which can enhance the cross-modal alignment capability of the pretrained VLMs. However, PEFT methods typically perform alignment in a single-step adjustment, where cross-modal features align utilizing only one forward pass of the PEFT module (cf., Figure 1(a)), lacking an explicit mechanism to model continuous and multi-step alignment dynamics. As a result, they often struggle ∗ Corresponding author.
Preprint.
Rabbit Dog Panda (b). Multi-Step FM
non-uniform
𝝎(𝒕) t
𝝎(𝒕)
0.0
𝒙𝒕
ഥ𝟎 𝒙
Cat
1.0
chord
Velocity Magnitude
Cat
Velocity Magnitude
(a). Single-Step PEFT
ഥ𝟏 𝒙
uniform
𝝎(𝒕)
t
𝝎(𝒕)
0.0
1.0
𝒙𝒕
𝒙𝟎
𝒙𝟏 Dog
Dog
FMA
DP-FM
HFM
Angular Sub-manifold DP-FM
Rabbit
Dog Panda
FMA
O
(c) FMA
Dog
(d) DP-FM
Figure 1: (a). Single-step parameter-efficient fine-tuning (PEFT) mostly performs cross-modal alignment in a single-step manner. (b). Multi-step flow matching (FM) methods model continuous and multi-step alignment dynamics. During the training stage, (c). FMA undergoes a non-uniform angular speed induced by radial–angular coupling. However, (d). DP-FM follows a constant-speed angular geodesic due to decoupled radial and angular dynamics. WP-FM
to resolve the alignment of highly entangled cross-modal distributions (i.e., image features and text features) in challenging datasets [10]. To address this limitation, flow matching for few-shot adaptation (FMA) [10] introduces a paradigm shift by conceptualizing cross-modal alignment as an unconditional21.0 continuous probability flow. Angular Velocity Magnitude
WP-FM (Hyperbolic)
WP-FM (Hyperbolic) HFM FMA FMA
DP-FM
DP-FM
Accuracy
0.1
FMA
1.0 Specifically, FMA first normalizes the pre1.6 45.0 trained cross-modal features (e.g., CLIP image and text features x0 , x1 ) into unit-length pairs (x̄0 , x̄1 ), where x̄i = xi /∥xiTime ∥2 for i ∈t {0, 1}. It then trains a veStep locity network to regress the target velocity 34.5 0.8 x̄1 − x̄0 induced by the linear interpolation trajectory xt = (1 − t)x̄0 + tx̄1 on a 0.1 1.0 1.0 0.2 Time step t Time step t 1.0 flat manifold (i.e., the chord as shown in Figure 2: Comparison between FMA [10], WP-FM Figure 1(c)). By multi-step integration over (Hyperbolic) [11], and DP-FM on Aircraft dataset at the predicted velocity, FMA progressively 100 epochs. DP-FM shows enhanced accuracy and more transports the image features to the text uniform angular speed across time step t at inference. features (Figure 1(b)), thereby achieving improved alignment compared to previous PEFT methods.
To analyze the inherent mechanics of cross-modal alignment, it is natural to apply a polar decomposition [12] to the interpolated features xt , disentangling them into a radial component (rt = ∥xt ∥2 , i.e., radial sub-manifold Mr , reflecting modality confidence) and an angular component (θt = xt /∥xt ∥2 , i.e., angular sub-manifold Mθ , encoding semantic direction). Under this geometric and probabilistic view, we argue that existing continuous flow matching (FM) methods [10, 11] for few-shot adaptation remain bottlenecked, failing to exploit the underlying capabilities of FM with suboptimal adaptation performance mostly due to three main limitations: 1) Angular Dynamics Distortion. They impose an incompatible geometric prior that distorts the intrinsic alignment dynamics. In FMA, the linear interpolation trajectory on a flat manifold induces severe radial–angular coupling. As shown in Figure 1(c), when the interpolated feature xt traverses this chord, the speed ω(t)3 on the angular sub-manifold (i.e., purple arc) is projected from velocity x̄1 − x̄0 . It undergoes angular dynamics distortion, characterized by non-uniform speed with non-zero acceleration. This induces extra curvature in the target velocity field, which increases the difficulty of regression training and leads to additional truncation errors, resulting in suboptimal performance during multi-step alignment. The following work HFM [11] implicitly bypasses this distortion by forcefully adjusting pre-trained features into a hyperbolic space [13] before applying FM. However, it introduces unnecessary geometric complexity and destroys the inherent angular sub-manifold inherent to these pre-trained features, leading to incompatibility with standard, off-the-shelf adaptation pipelines widely used in current cross-modal tasks. 2 An unconditional flow indicates that the learned velocity field v(x , t) depends solely on the current state x and time t. t t It models the marginal probability path between two distributions without relying on any external conditions. 3We use speed as a scalar to denote the velocity magnitude, and velocity represents both magnitude and direction.
2
2) Radial Dynamics Neglect. Radial information intrinsically reflects modality confidence, acting as an explicit signal to distinguish out-of-distribution (OOD) and in-distribution (ID) data [14] (Refer to Appendix B). This confidence is helpful for the velocity network in few-shot adaptation, as it enables the alignment process to remain sensitive to uncertain OOD samples and apply focused corrections. However, due to the inherent radial–angular coupling in existing methods, explicitly retaining this radial information exacerbates the distortion in angular dynamics, leading to degraded performance. 3) Context-Agnostic Unconditional Flow. In few-shot adaptation scenarios, cross-modal features extracted by pre-trained VLMs are highly generalized and lack dataset-specific details for the target domain. Existing FM methods model cross-modal alignment as an unconditional probability flow, relying solely on these generalized features. The problem is that an unconditional flow purely aligns pair-wise features, making it inherently context-agnostic. Since it is blind to the whole dataset context, it cannot well adapt these generalized features into domain-specific ones. Therefore, there is no additional conditional guidance to introduce the target domain context into the alignment process, i.e., the adaptation performance inevitably degrades. Unified WP-FM Framework. To systematically resolve the geometric Limitation 1) and 2), we propose warped product flow matching (WP-FM) framework, which elevates the aforementioned polar decomposition into the Riemannian framework [15]. Rather than describing cross-modal alignment on a flat manifold, WP-FM structures the features as a warped product manifold M = Mr ×gϕ Mθ . However, merely decoupling the features topologically does not resolve radial–angular coupling, as coupling behavior is dictated by the underlying Riemannian metric gϕ . To capture this, we equip M with a unified metric tensor gϕ : ds2 = dr2 + ϕ(r)2 dθ 2 . Here, ds, dr, and dθ denote the infinitesimal distance elements on the full manifold M, the radial sub-manifold Mr , and the angular sub-manifold Mθ , respectively, while ϕ(r) acts as a radial warping function that controls the coupling behavior. Under this general framework, we can define different metrics to instantiate different variants covering previous FM methods for adaptation: • When ϕ(r) = r, it degrades to FM on the flat manifold, as in FMA [10]. • When ϕ(r) = sinh(r), it degrades to FM on the Hyperbolic manifold, as in HFM [11]. It is worth noting that these non-constant warping functions (in above-mentioned variants) intrinsically couple the radial and angular dynamics, which are “warped”, directly causing the distortion discussed previously. To ensure cross-modal features naturally evolve along dual, independent geodesics γ(t) = (γr (t), γθ (t)) with a constant-speed angular geodesic, we introduce direct product flow matching (DP-FM) under the unified WP-FM framework: by deriving the metric for unraveling the radial–angular coupling, we acquire the constant warping function (i.e., ϕ(r) = CONSTANT), yielding FM on a decoupled cylindrical manifold (i.e., direct product manifold). In DP-FM, both Limitation 1) and 2) are resolved: the radial trajectory γr (t) preserves the dynamics of modality confidence, while the angular trajectory γθ (t) independently traverses a constant-speed geodesic (cf., Figure 1(d)), eliminating angular dynamics distortion for enhanced performance (cf., Figure 2). Furthermore, to resolve the context-agnostic Limitation 3) caused by information loss, DP-FM can incorporate classifier-free guidance (CFG) [16]. Instead of depending exclusively on the pre-trained generalized cross-modal features, we condition the flow on the pre-trained model’s hidden states [17]. This explicitly injects the missing dataset-specific information into the alignment, allowing DP-FM to recover fine-grained details in the target domain and guarantee enhanced few-shot adaptation. Extensive experiments across 11 diverse few-shot benchmarks demonstrate that DP-FM achieves enhanced adaptation accuracy compared to previous state-of-the-art FM methods for few-shot adaptation. In summary, our main contributions are threefold: 1). Theoretical Insight: We identify fundamental geometric limitations in existing continuous FM methods for few-shot adaptation, namely angular dynamics distortion and radial dynamics neglect. 2). Unified WP-FM Framework: We propose warped product FM to reformulate cross-modal alignment on a unified Riemannian manifold. 3). DP-FM with CFG: We derive DP-FM to strictly decouple radial and angular dynamics. Additionally, we integrate CFG to inject dataset-specific information back into the alignment process.
2
Related Work
Few-shot Adaptation in VLMs. Few-shot adaptation of vision-language models (VLMs) [4, 18, 19] aims to transfer pre-trained cross-modal alignment to downstream domains using limited data. To 3
achieve this, prior work predominantly adopts parameter-efficient fine-tuning (PEFT) methods [20], which adapt models without updating the entire model. Among these approaches, prompt tuning methods [7, 6] optimize learnable context vectors, typically within the text encoder, while adapterbased methods [21, 22, 9] introduce lightweight trainable modules into the model. These methods have demonstrated effective empirical performance in few-shot adaptation. Recent works [10, 11] explore continuous formulations based on flow matching, where cross-modal alignment is modeled as a multi-step unconditional probability flow. While they provide an expressive multi-step framework for cross-modal alignment, they fail to exploit the underlying capabilities of FM with suboptimal adaptation performance. As analyzed in Limitation 1) and 2), these issues are closely related to the underlying geometric limitations inherent in existing methods. Flow Matching (FM) and Riemannian FM. FM [23, 24] has emerged as an efficient framework for generative modeling [25, 26, 27, 28]. By directly regressing target velocity fields induced by simple and tractable probability paths, FM enables a streamlined training pipeline and improved numerical stability compared to prior approaches (e.g., traditional diffusion models [29, 30]). To accommodate non-Euclidean data, Riemannian FM (RFM) [15] constructs optimal transport trajectories under the Riemannian metric. While theoretically principled, RFM lacks a task-specific metric design. Recent FM method for few-shot adaptation [10] defaults to incompatible geometries, ignoring the intrinsic radial and angular structure of pre-trained cross-modal features. To bridge this gap, our proposed warped product FM (WP-FM) tailors RFM for cross-modal alignment by introducing a unified metric with a radial warping function. Crucially, by adopting an optimal constant-warping metric, we derive direct product FM (DP-FM), which decouples radial and angular dynamics. This efficiently resolves Limitations 1) and 2), establishing a robust geometric foundation for multi-step adaptation.
3
Methodology
3.1
Unified Warped Product Flow Matching (WP-FM) Framework
Preliminaries: Warped Product Manifolds. In differential geometry, a warped product manifold [31] provides a generalized structure for the product of two Riemannian manifolds. Given a base manifold (B, gB ) and a fiber manifold (F, gF ), their warped product manifold M = B ×f F is constructed by equipping the product topological space B × F with a metric tensor characterized by the following line element: ds2 = ds2B + f (x)2 ds2F , where f : B → R>0 is a smooth scalar function called the warping function. Under this structure, the intrinsic geometry of the fiber manifold F is modified (i.e., “warped”) by f (x), depending solely on the point x ∈ B. This framework provides a foundation for pre-trained cross-modal feature representation, allowing us to map the modality confidence to the base manifold and the semantic direction to the fiber manifold. Riemannian FM on Warped Product Manifolds. We begin by formalizing cross-modal alignment as a continuous probability flow on a Riemannian manifold. Given features x ∈ Rd (i.e., any possible interpolated features between cross-modal features) in the cross-modal alignment, we apply a polar decomposition to disentangle the feature: x = r · θ, r = ∥x∥2 , θ ∈ Sd−1 . This induces a warped product manifold M = Mr ×gϕ Mθ , where Mr = R>0 can be interpreted as reflecting modality confidence (i.e., base radial sub-manifold), and Mθ = Sd−1 encodes the semantic direction (i.e., fiber angular sub-manifold). To characterize the dynamics and radial-angular coupling behavior, we establish a unified Riemannian manifold (M, gϕ ), where the metric tensor gϕ is defined by a family of rotationally symmetric line elements: ds2 = dr2 + ϕ(r)2 dθ 2 , where ds, dr, and dθ denote the infinitesimal distance elements on the full manifold M, the radial sub-manifold Mr , and the angular sub-manifold Mθ , respectively, ϕ(r) is a radial warping function. Warped Product Flow Matching. This formulation provides a unified geometric framework that subsumes a broad class of FM methods based on different choices of gϕ (e.g., ϕ(r) = r degrades to the flat manifold in FMA [10], ϕ(r) = sinh(r) degrades to the Hyperbolic manifold in HFM [11]). We define Riemannian FM on this warped product manifold as warped product flow matching (WP-FM). Given paired pretrained cross-modal features (x0 , x1 ) (i.e., x0 is image feature, and x1 is text feature), the optimal transport trajectory corresponds to a geodesic γ(t) under the chosen metric d gϕ . The target velocity fields regressed by the network are then defined as v ∗ (xt , t) = dt γ(t). 4
3.2
Analyses: Angular Dynamics Distortion
FMA [10] acts as a suboptimal instantiation of WP-FM that explicitly discards radial dynamics via L2 normalization. By implicitly setting the warping function to ϕ(r) = r, FMA degrades the metric to a flat space. Under this incompatible geometric prior, the optimal transport geodesic defaults to a linear interpolation: γ(t) = (1 − t)x̄0 + tx̄1 , where x̄i = xi /∥xi ∥2 . While this trajectory maintains a constant velocity in the ambient space, it inherently introduces radial–angular coupling and induces severe angular dynamics distortion. Specifically, let γr (t) = ∥γ(t)∥2 denote the radial magnitude and ω(t) = ∥γ̇θ (t)∥2 denote the angular speed, where γθ (t) := γ(t)/∥γ(t)∥2 . Under this interpolation, the trajectory behaves as a linear chord, causing the magnitude γr (t) to sag (γ˙r (t) ̸= 0). This geometric sagging amplifies the angular speed near the midpoint (t = 0.5). Taking the time derivative of the angular speed yields a non-zero variation in angular velocity: ω̇(t) ∝ −γ˙r (t)/γr (t)3 ̸= 0.
(1)
Eq. (1) shows that the angular dynamics are actually affected by radial dimension in the flat space (i.e., radial–angular coupling). This incompatible geometry forces the trajectory to exhibit non-uniform angular speed with non-zero angular acceleration (cf., Figure 1(c)). It inherently introduces additional curvature into the effective velocity field on the angular sub-manifold, exacerbating the difficulty of regression training and leading to extra truncation errors4 , which causes suboptimal alignment performance during multi-step alignment for adaptation. If FMA retains the radial information without any normalization, the angular speed will be even more non-uniform. The following work HFM [11] forcefully adjusts the pre-trained cross-modal features into a hyperbolic space [13] before applying FM via centripetal hyperbolic alignment, which can implicitly bypass this distortion. Although adjusting features toward a hyperbolic boundary can artificially boost class separability to achieve competitive few-shot performance, this extra mapping step introduces an inefficient two-stage training paradigm and unnecessary geometric complexity, which disrupts the inherent angular sub-manifold of pre-trained cross-modal features, leading to incompatibility with standard, off-the-shelf adaptation pipelines widely used in current cross-modal tasks. 3.3
Direct Product Flow Matching (DP-FM)
To eliminate this distortion while preserving the inherent manifold of pre-trained cross-modal representations, the angular trajectory should independently follow a constant-speed geodesic on Sd−1 . In Riemannian geometry, this requires the covariant derivative of the angular velocity to D strictly vanish: dt γ̇θ (t) = 0. Under the generalized metric, expanding the angular acceleration via ′
(r) Christoffel symbols [31] (Γθrθ = ϕϕ(r) ) reveal the coupling:
D ϕ′ (γr (t)) γ̇θ (t) = −2 γ̇r (t)γ̇θ (t). dt ϕ(γr (t))
(2)
This mandates ϕ′ (γr (t)) = 0, leading to the metric: ϕ(r) = CONSTANT. This constant-warping metric establishes a theoretically optimal product manifold, i.e., a decoupled cylindrical manifold R>0 × Sd−1 . Driven by this finding, we propose direct product flow matching (DP-FM). By constructing the continuous flow exclusively under this optimal metric, the target velocity field d naturally decomposes into two independent dynamic objectives: v ∗ (xt , t) = dt γ(t) = (γ̇r (t), γ̇θ (t)). Specifically, given an image feature x0 = r0 θ0 and a text feature x1 = r1 θ1 , the decoupled groundtruth geodesics are formulated as: sin((1 − t)α) sin(tα) θ0 + θ1 , (3) sin α sin α where α = arccos(⟨θ0 , θ1 ⟩) ∈ [0, π]. This explicit dual-geodesic dynamics cleanly resolves the two geometric limitations of previous methods: 1). Mitigating Angular Dynamics Distortion: As formulated in Eq. (3), the angular geodesic γθ (t) naturally evolves via exact spherical linear interD polation (slerp) on Sd−1 . This guarantees a constant-speed geodesic ( dt γ̇θ (t) = 0), fundamentally eliminating the angular dynamics distortion, thereby preventing extra curvature in the target velocity field. 2). Addressing Radial Dynamics Neglect: Rather than discarding radial information, the radial geodesic γr (t) is explicitly retained and modeled as an independent radial trajectory, which preserves modality confidence to distinguish OOD from ID samples for the velocity network. γr (t) = (1 − t)r0 + tr1 ,
γθ (t) =
4 Derivations are in the Appendix A, showing that any non-constant warping metric induces an extra truncation error.
5
Algorithm 1: Warped Product Flow Matching (WP-FM) Phase 1: Training Require: Image and text features P0 , P1 , velocity field vψ , steps N , dropout pdrop . 2 while not converged do 3 Sample x0 ∼ P0 , x1 ∼ P1 , t ∼ U (0, 1), extract hidden states condition c; s·t 4 Time-shift schedule t ← 1+(s−1)t , set c ← ∅ with probability pdrop ; 5 Polar decompose ri ← ∥xi ∥2 , θi ← xi /ri (i ∈ {0, 1}); 6 Compute coupled geodesics (γr (t), γθ (t)) and target velocities (γ̇r (t), γ̇θ (t)); 7 Assemble feature xt ← γr (t) · γθ (t), predict velocity vψ ← vψ (xt , t|c); 8 Project to tangent spaces: vrad ← ⟨vψ , γθ (t)⟩, vang ← (vψ − vrad γθ (t))/γr (t); 9 Compute metric-aware loss: L ← ∥vrad − γ̇r (t)∥22 + ϕ(γr (t))2 ∥vang − γ̇θ (t)∥22 ; 10 Update network parameters ψ via ∇ψ L; 1
Phase 2: Inference Require: Image feature xtest ∼ P0 , hidden states condition c, CFG scale w. 12 Initialize r0 ← ∥xtest ∥2 , θ0 ← xtest /r0 , set step size ∆t ← 1/N ; 13 for k = 0 to N − 1 do 14 State xk ← rk · θk at time t ← k∆t; 15 Predict condition-guided velocity: ṽψ ← vψ (xk , t|∅) + w · vψ (xk , t|c) − vψ (xk , t|∅) ; 16 Project to tangent spaces: vrad ← ⟨ṽψ , θk ⟩, vang ← (ṽψ − vrad θk )/rk ; 17 Integration on the manifold: (rk+1 , θk+1 ) ← ExpM (rk ,θk ) ([vrad , vang ]∆t); 11
Output: xN ← rN · θN ;
3.4
Algorithmic Design & Implementation of WP-FM
We introduce several critical designs for both the training and inference stages for WP-FM. The detailed algorithm is shown in Algorithm 1. Classifier-Free Guidance in Adaptation. As identified in Limitation 3), the pre-trained VLMs often yield generalized features that discard dataset-specific information in few-shot scenarios. To explicitly inject the missing information back into the alignment process, we equip WP-FM with classifier-free guidance (CFG) [16], where the conditioning signal c is derived from the pre-trained model’s hidden states given the input x0 . During training, we parameterize conditional vector fields vθ (xt , t|c). We jointly train an unconditional field by randomly replacing c with a learnable null token ∅ at a fixed dropout probability pdrop . During inference, we construct a semantic-guided velocity field ṽψ : ṽψ (xt , t|c) = vψ (xt , t|∅) + w · vψ (xt , t|c) − vψ (xt , t|∅) , (4) where w ≥ 1 is the guidance scale. By integrating over ṽψ , DP-FM amplifies the target-domain information. This effectively steers the flow away from context-agnostic trajectories, recovering the fine-grained information discarded during pre-trained feature extraction for enhanced adaptation. Velocity Decomposition. Standard neural networks 𝒗𝒓𝒂𝒅 Angular: typically output velocity predictions in the ambient 𝒗𝝍 d 𝒙𝟏 Radius: 𝒙𝒕 Euclidean space, denoted as vψ (xt , t) ∈ R . To adhere to our warped product manifold geometry, this Dog output is explicitly projected onto the tangent spaces 𝒗𝒂𝒏𝒈 T Mr and T Mθ . Specifically, as shown in Figure 3, 𝒙𝟎 the radial velocity is extracted via the inner product: Hidden States vrad = ⟨vψ , γθ (t)⟩. For the angular component, we Condition isolate the orthogonal projection and fundamentally divide it by the radius rt : 𝒕 Velocity O 𝒗𝝍 Network 1 vang = (vψ − vrad γθ (t)) , (5) γr (t) Figure 3: Illustration of DP-FM. which converts the predicted velocity in ambient Euclidean space into the angular velocity measured in radians, ensuring strict dimensionality alignment with the ground-truth geodesic. 6
Table 1: Comparison with Pure FM methods for few-shot adaptation. Aircraft SAT DTD SUN Cars AVG
Shots Methods
Pets UCF Flowers Caltech Food Net AVG
0
CLIP [4]
24.8
47.8 43.8
62.6 65.5 48.9
89.1 66.8
71.2
92.9
86.1 66.7 78.8
1
WP-FM (Euclidean) [10] WP-FM (Hyperbolic) [11] DP-FM
26.8 26.3 26.3
66.7 53.5 66.5 53.5 68.6 53.3
66.3 67.0 56.1+7.2 65.7 67.4 55.9+7.0 66.3 67.2 56.3+7.4
90.2 71.2 89.8 71.3 90.5 72.2
82.2 81.7 83.2
93.9 94.1 94.0
86.1 67.5 81.9+3.1 86.1 67.5 81.8+3.0 86.1 67.4 82.2+3.4
4
WP-FM (Euclidean) [10] WP-FM (Hyperbolic) [11] DP-FM
30.7 32.4 32.4
77.9 61.4 78.5 61.1 79.9 61.9
69.9 71.0 62.2+13.3 91.8 77.8 70.1 72.6 62.9+14.0 91.9 78.5 70.6 71.9 63.3+14.4 92.1 78.1
92.3 94.3 93.6
95.2 94.4 95.5
86.4 68.9 85.4+6.6 86.5 68.8 85.7+6.9 86.5 69.0 85.8+7.0
16
WP-FM (Euclidean) [10] WP-FM (Hyperbolic) [11] DP-FM
44.6 44.2 46.0
87.0 70.6 86.9 71.8 87.8 71.2
74.4 82.7 71.9+23.0 92.9 82.7 73.5 83.2 71.9+23.0 93.1 83.9 73.8 83.6 72.5+23.6 93.2 83.4
98.3 97.8 97.8
95.7 95.8 96.2
86.8 71.1 87.9+9.1 86.9 71.0 88.1+9.3 87.0 71.1 88.1+9.3
(a) Aircraft
(b) DTD
DP-FM
WP-FM (Hyperbolic)
FMA
DP-FM
WP-FM (Hyperbolic)
(C) UCF
FMA
DP-FM
WP-FM (Hyperbolic)
FMA
DP-FM
WP-FM (Hyperbolic)
FMA
DP-FM
WP-FM (Hyperbolic)
FMA
DP-FM
WP-FM (Hyperbolic)
Angular Velocity Magnitude
FMA 1.6
0.8
0.1
1.0
0.40
0.1
1.0
0.1
1.0
0.85
Accuracy
0.70
0.22
0.68
0.45
0.1
Time Step t
1.0
0.1
Time Step t
1.0
0.1
Time Step t
1.0
Figure 4: Comparison between WP-FM (Euclidean [10], Hyperbolic [11]) and DP-FM on (a). Aircraft, (b). DTD, and (c). UCF dataset at 20 epochs. Metric-Aware Loss Objective. Unlike standard FM, which minimizes mean squared error (MSE) uniformly across Euclidean dimensions, WP-FM utilizes a specific optimization objective function corresponding to the Riemannian manifold. Guided by the metric tensor ds2 = dr2 + ϕ(r)2 dθ 2 , the regression loss for WP-FM is formulated as a metric-aware objective: LWP-FM = Et,x0 ,x1 ∥vrad − γ̇r (t)∥22 + ϕ(γr (t))2 ∥vang − γ̇θ (t)∥22 . (6) This ensures that the gradient updates strictly follow the geometric structure of the manifold. Inference via Exponential Map. During the inference phase5 of WP-FM, discrete multi-step integration (e.g., Euler method) in ambient space easily drifts off the manifold, leading to severe error accumulation. To guarantee mathematical closure, our method utilizes the formal Riemannian exponential map ExpM x [15] for the ordinary differential equation (ODE) integration step ∆t: (rt+∆t , θt+∆t ) ← ExpM (rt ,θt ) ([vrad , vang ]∆t).
(7)
Time-shift Schedule. Standard uniform time sampling allocates insufficient capacity to the complex initial phase of cross-modal alignment. To address this, we apply a non-linear time-shift schedule, s·t which is widely used in high-resolution generative modeling [32, 33, 34]: t′ = 1+(s−1)t , where s is a time-shift parameter. Setting s < 1 monotonically biases the sampling density toward t → 0, forcing the network to prioritize early velocity estimation and accelerating overall convergence.
4
Experiments
Datasets and Baselines. We evaluated DP-FM on few-shot image classification tasks. Specifically, we conducted experiments on 11 benchmarks, including Aircraft [35], EuroSAT [36], DTD [37], SUN397 [38], StanfordCars [39], OxfordPets [40], UCF101 [41], Flowers102 [42], Caltech101 [43], Food101 [44], and ImageNet [45]. Following the FMA setting, we partitioned them into two subsets: five datasets form the difficult group, and the remaining six constitute the easy group. For each dataset, we adopted the standard train/validation/test splits. Under the K-shot setting, we constructed the training set by randomly sampling K labeled images per class, with the rest used for validation and testing. For PEFT methods, we adopted CoOp [7], CoCoOp [6], TIP-Adapter [46], CLIPAdapter [22], PLOT++ [47], KgCoOp [48], ProGrad [49], and CLIP-LoRA [8] as our baselines. For multi-step FM methods, we adopted FMA [10] and HFM [11] as our baselines. 5 In the inference stage, we utilize r and θ to replace γ (t) and γ (t). t t r θ
7
Table 2: Comparison with state-of-the-art FM methods for adaptation building upon CLIPLoRA and CLIP-Adapter. † denotes that the PEFT module is trained using centripetal hyperbolic alignment [13]. ∆ represents the average performance improvement by FM over the baseline. Difficult Datasets Shot Methods
Easy Datasets
Aircraft SAT DTD SUN Cars AVG
∆
Pets UCF Flowers Caltech Food Net AVG
∆
1
CLIP-LoRA† [13] +HFM [11] CLIP-LoRA [8] +FMA [10] +DP-FM
29.2 30.5 28.0 28.3 29.6
73.1 76.8 71.9 73.0 81.7
56.4 57.4 54.1 55.1 56.9
69.9 71.2 70.3 70.6 71.0
68.7 69.8 69.4 69.8 70.8
59.5 92.1 75.8 61.1 +1.7 92.2 76.6 58.7 91.9 75.4 59.4 +0.6 92.1 75.9 62.0 +3.3 93.1 76.8
81.9 85.8 81.4 84.9 89.0
94.5 95.1 93.8 94.5 94.4
86.8 85.9 85.1 85.2 85.0
69.7 70.0 70.3 70.2 70.3
83.5 84.3 +0.8 83.0 83.8 +0.8 84.8 +1.8
4
CLIP-LoRA† [13] +HFM [11] CLIP-LoRA [8] +FMA [10] +DP-FM
40.3 43.8 38.8 40.3 41.7
89.0 90.7 83.5 85.0 89.3
66.5 67.6 64.0 67.0 69.2
74.1 75.5 72.8 73.7 73.4
77.4 79.9 77.4 78.9 80.5
69.5 93.4 81.0 71.5 +2.0 93.7 83.4 67.3 90.6 81.1 69.0 +1.7 90.8 82.4 70.8 +3.5 92.2 82.3
93.7 95.5 92.9 95.0 96.8
95.7 96.1 95.0 95.8 96.0
86.8 86.8 82.6 83.2 83.7
71.4 72.1 71.4 72.0 72.1
87.0 87.9 +0.9 85.6 86.5 +0.9 87.2 +1.6
16
CLIP-LoRA† [13] +HFM [11] CLIP-LoRA [8] +FMA [10] +DP-FM
57.6 61.4 54.7 57.8 59.2
91.3 93.1 90.7 91.0 93.1
73.7 75.2 73.0 75.4 77.1
77.1 77.6 76.0 77.2 77.6
86.5 88.6 86.0 87.7 88.8
77.2 93.6 87.3 79.2 +1.9 94.0 88.9 76.1 91.6 86.2 77.8 +1.7 91.6 87.1 79.2 +3.1 92.8 88.0
98.7 98.7 97.9 99.1 99.0
96.4 96.8 96.1 96.5 96.6
87.3 87.3 84.2 85.1 85.8
73.6 73.7 73.4 73.5 74.4
89.5 89.9 +0.4 88.2 88.8 +0.6 89.4 +1.2
16
CLIP-Adapter† [13] +HFM [11] CLIP-Adapter [22] +FMA [10] +DP-FM
34.2 43.8 33.8 35.8 45.1
72.3 85.7 70.4 85.6 86.4
64.1 71.4 59.3 69.2 71.6
74.2 76.1 74.3 74.4 75.8
73.6 80.6 74.2 74.7 82.7
63.7 92.7 82.4 71.5 +7.8 93.3 84.6 62.4 92.4 80.1 67.9 +5.5 92.9 81.5 72.3 +9.9 92.9 84.6
93.3 97.2 93.6 95.6 97.9
95.1 96.2 94.9 96.0 96.0
87.0 87.1 87.1 87.2 87.1
71.2 72.1 71.6 71.3 72.4
87.0 88.4 +1.5 86.6 87.4 +0.8 88.5 +1.9
Settings. We implemented DP-FM based on the pre-trained CLIP, CLIP-LoRA [8], and CLIPAdapter [22] model. Specifically, we first extract cross-modal features from the pre-trained models, then utilize a velocity network to regress velocity fields that transport image features to text features. Following the FMA [10] setting, we utilize a lightweight MAR [50] network as our velocity network. In the DP-FM algorithm, we set CONSTANT = 25, time-shift parameter s = 0.1, dropout probability pdrop = 0.1, and CFG scale w = 5.0 across all the experiments. We trained DP-FM with AdamW [51] using a learning rate of 2 × 10−4 and a weight decay of 0.01. Performance against FM Baselines. We evaluate DP-FM against recent FM methods for few-shot adaptation, including WP-FM (in Euclidean6 [10], Hyperbolic7 [11] space). For a fair comparison, all the methods operate on the same pre-trained CLIP model and are equipped with the time-shift schedule and CFG. As shown in Table 1, DP-FM consistently achieves higher average accuracy across the 1, 4, and 16-shot settings. By isolating the geometric design on the pre-trained cross-modal features, these results explicitly confirm that the decoupled cylindrical manifold provides an enhanced foundation for FM compared to flat Euclidean or Hyperbolic spaces. Furthermore, as illustrated in Figure 4, DP-FM exhibits a more uniform angular speed and enhanced adaptation performance. Furthermore, we extend our evaluation to compare DP-FM against state-of-the-art FM methods when integrated with established PEFT baselines, specifically CLIP-LoRA and CLIP-Adapter, as shown in Table 2. DP-FM consistently delivers the most substantial performance improvements (∆) over the baselines across different datasets. For instance, under the 16-shot setting with CLIP-Adapter, DP-FM achieves a remarkable average improvement of +9.9% on the difficult datasets, which significantly surpasses the gains provided by FMA (+5.5%) and HFM (+7.8%). Similarly, when built upon the CLIP-LoRA architecture, DP-FM outperforms competing FM methods in most cases, demonstrating its enhanced capability to refine feature alignment effectively. Comparison with state-of-the-art PEFT. As presented in Table 3, we evaluate the performance of our DP-FM against a comprehensive set of single-step PEFT methods. Built upon CLIP-LoRA, DP-FM consistently establishes new state-of-the-art results across 1, 4, and 16-shot settings for both difficult and easy dataset groups. Most notably, under the 16-shot setting, DP-FM achieves an average accuracy of 79.2% on the difficult datasets and 89.4% on the easy datasets. This performance significantly exceeds strong single-step baselines such as PLOT++ (74.1% and 88.7%) and the standalone CLIP-LoRA (76.1% and 88.2%). These results confirm that modeling the adaptation process as a continuous flow on a decoupled cylindrical manifold yields superior generalization compared to conventional static, single-step prompting or adaptation strategies. 6WP-FM (Euclidean) is the same as FMA [10], operating on pre-trained cross-modal features 7 To isolate the impact of different geometries, we omit HFM’s centripetal hyperbolic alignment and diameter-based
stopping phases, ensuring FMA, HFM, and DP-FM operate on identical pre-trained cross-modal features in 1-stage training.
8
Table 3: Comparison with other state-of-the-art PEFT methods. Building upon CLIP-LoRA, adaptation performance can be further improved by our DP-FM. Difficult Datasets 0
1
4
16
Easy Datasets
Aircraft SAT DTD SUN Cars AVG
Shot Method
Pets UCF Flowers Caltech Food Net AVG
CLIP [4]
24.8
47.8 43.8
62.6 65.5 48.9
89.1 66.8
71.2
92.9
86.1 66.7 78.8
CoOp [7] CoCoOp [6] TIP-Adapter [46] CLIP-Adapter [22] PLOT++ [47] KgCoOp [48] ProGrad [49] CLIP-LoRA [8] +DP-FM
20.8 28.1 28.8 25.2 28.6 26.8 28.9 28.0 29.6
56.4 55.4 67.8 49.3 65.4 61.9 57.0 71.9 81.7
50.1 52.6 51.6 44.2 54.6 52.7 52.8 54.1 56.9
67.0 68.7 67.2 65.4 66.8 68.4 67.0 70.3 71.0
67.5 67.6 67.1 65.7 68.8 66.7 68.2 69.4 70.8
52.4 54.5 56.5 50.0 56.8 55.3 54.8 58.7 62.0+3.3
90.2 91.9 90.6 89.0 91.9 92.1 91.4 91.9 93.1
71.2 70.4 73.4 66.9 74.3 72.8 73.3 75.4 76.8
78.3 73.4 83.8 71.3 80.5 74.7 80.9 81.4 89.0
92.5 94.1 94.0 92.0 94.3 94.2 93.5 93.8 94.4
84.3 84.9 85.8 86.1 86.2 86.4 84.9 85.1 85.0
65.7 69.4 69.4 67.9 66.5 68.9 67.0 70.3 70.3
80.4 80.7 82.8 78.9 82.3 81.5 81.8 83.0 84.8+1.8
CoOp [7] CoCoOp [6] TIP-Adapter [46] CLIP-Adapter [22] PLOT++ [47] KgCoOp [48] ProGrad [49] CLIP-LoRA [8] +DP-FM
30.9 30.6 35.7 27.9 35.3 32.2 34.1 38.8 41.7
69.7 61.7 76.8 51.2 83.2 71.8 69.6 83.5 89.3
59.5 55.7 59.8 46.1 62.4 58.7 59.7 64.0 69.2
69.7 70.4 70.8 68.0 71.7 71.5 71.7 72.8 73.4
74.4 69.5 74.1 67.5 76.3 69.5 75.0 77.4 80.5
60.8 57.6 63.4 52.1 65.8 60.7 62.0 67.3 70.8+3.5
92.5 92.7 91.9 90.8 92.7 92.6 92.1 90.6 92.2
77.6 75.3 78.1 70.6 79.8 77.6 77.9 81.1 82.3
92.2 81.5 92.1 73.1 92.9 87.0 91.1 92.9 96.8
94.5 94.8 94.8 94.0 95.1 95.0 94.4 95.0 96.0
84.5 86.3 86.5 86.5 86.5 86.9 85.4 82.6 83.7
68.8 70.6 70.7 68.6 70.4 69.9 70.2 71.4 72.1
85.0 83.5 85.7 80.6 86.2 84.8 85.2 85.6 87.2+1.6
CoOp [7] CoCoOp [6] TIP-Adapter [46] CLIP-Adapter [22] PLOT++ [47] KgCoOp [48] ProGrad [49] CLIP-LoRA [8] +DP-FM
43.3 33.8 44.6 34.2 46.7 36.5 43.0 54.7 59.2
86.0 75.5 85.9 71.4 92.0 76.2 83.6 90.7 93.1
70.0 65.8 70.8 59.4 71.4 68.7 68.8 73.0 77.1
74.9 72.8 76.0 74.2 76.0 73.3 75.1 76.0 77.6
83.1 72.4 82.3 74.0 84.6 74.8 82.9 86.0 88.8
71.5 64.1 71.9 62.6 74.1 65.9 70.7 76.1 79.2+3.1
91.1 93.2 92.6 92.3 93.6 93.2 92.8 91.6 92.8
83.1 76.0 83.9 80.2 85.3 81.7 82.7 86.2 88.0
97.2 87.1 96.2 92.9 97.6 93.4 96.6 97.9 99.0
95.5 95.2 95.7 94.9 96.0 95.2 95.9 96.1 96.6
84.4 87.4 86.8 87.1 87.1 87.2 85.8 84.2 85.8
71.4 71.1 73.4 69.8 72.6 70.4 72.1 73.4 74.4
87.1 85.0 88.1 86.2 88.7 86.9 87.7 88.2 89.4+1.2
Table 4: Ablation study on Time-shift Schedule (T), Radius component (R), and Classifier-free guidance (C) under the 16-shot setting. The numbers in parentheses denote CFG scale w. Method
Aircraft SAT DTD SUN Cars AVG
Pets UCF Flowers Caltech Food Net AVG
T R
C
CLIP [4] × ×
×
24.8
47.8 43.8
62.6 65.5 48.9
89.1 66.8
71.2
92.9
86.1 66.7 78.8
× × ✓ +DP-FM ✓ ✓ ✓
× ✓(5) ✓(5) × ✓(1) ✓(5)
42.7 41.3 44.2 44.6 45.8 46.0
85.5 86.7 86.8 87.0 87.8 87.8
73.7 72.4 73.6 73.8 73.1 73.8
92.9 92.1 93.0 92.7 93.1 93.2
97.3 97.3 97.8 97.7 97.7 97.8
95.9 95.2 96.1 95.9 96.2 96.2
86.8 86.4 86.9 87.1 87.0 87.0
× ✓ × ✓ ✓ ✓
70.8 69.8 69.6 71.1 71.2 71.2
82.9 79.8 83.6 82.6 82.6 83.6
71.1+22.2 70.0+21.1 71.6+22.7 71.8+22.9 72.1+23.2 72.5+23.6
82.3 82.6 82.6 82.6 82.6 83.4
70.8 71.0 71.0 71.1 71.1 71.1
87.7+8.9 87.4+8.6 87.9+9.1 87.9+9.1 88.0+9.2 88.1+9.3
Ablation Study and Component Analysis. To analyze the contribution of each proposed component, we conduct ablation experiments under the 16-shot setting (Table 4). First, we examine the effect of the Time-shift Schedule (T) on training convergence. Notably, adding the Radius component (R) and classifier-free guidance (C) without T (Row 3) yields a lower performance (70.0%) than the baseline lacking all three components (Row 2, 71.1%). This is likely because introducing extra radial and conditional information increases optimization complexity, which delays the training process. The time-shift schedule addresses this by prioritizing training of early-stage velocity estimation. With T stabilizing the optimization, the empirical benefits of R and C become clearly evident, as integrating the radial component and CFG both lead to consistent performance improvements. Ultimately, combining all three components achieves the highest average accuracy of 72.5% on difficult datasets, validating the integrated design of our method.
5
Conclusion
In this paper, we investigated the geometric limitations of existing FM methods for few-shot VLM adaptation, demonstrating that they inherently suffer from angular dynamics distortion, radial dynamics neglect, and context-agnostic flows. To address this, we introduced warped product flow matching (WP-FM), a unified Riemannian framework. By adopting a constant-warping metric, we derived DP-FM to strictly decouple radial and angular dynamics, ensuring constant-speed geodesic transport and preserving modality confidence. Additionally, we integrated hidden-state conditioned classifierfree guidance to recover dataset-specific context. Extensive experiments across 11 vision-language benchmarks confirm our method achieves enhanced adaptation accuracy during multi-step alignment. 9
References [1] Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. NeurIPS, 35:23716–23736, 2022. [2] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping languageimage pre-training with frozen image encoders and large language models. In ICML, pages 19730–19742. PMLR, 2023. [3] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. NeurIPS, 36:34892–34916, 2023. [4] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, pages 8748–8763. PMLR, 2021. [5] Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In ICML, pages 12888– 12900. PMLR, 2022. [6] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language models. In CVPR, pages 16816–16825, 2022. [7] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. IJCV, 130(9):2337–2348, 2022. [8] Maxime Zanella and Ismail Ben Ayed. Low-rank few-shot adaptation of vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1593–1603, 2024. [9] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022. [10] Ziqi Jiang, Yanghao Wang, and Long Chen. Exploring cross-modal flows for few-shot learning. In ICLR, 2026. [11] Lin Li, Ziqi Jiang, Gefan Ye, Zhenqi He, Jiahui Li, Jun Xiao, Kwang-Ting Cheng, and Long Chen. Path-decoupled hyperbolic flow matching for few-shot adaptation. arXiv preprint, 2026. [12] Minsoo Jo, Dongyoon Yang, and Taesup Kim. Angular gradient sign method: Uncovering vulnerabilities in hyperbolic networks. In AAAI, volume 40, pages 5566–5574, 2026. [13] Karan Desai, Maximilian Nickel, Tanmay Rajpurohit, Justin Johnson, and Shanmukha Ramakrishna Vedantam. Hyperbolic image-text representations. In ICML, pages 7694–7731. PMLR, 2023. [14] Jaewoo Park, Jacky Chen Long Chai, Jaeho Yoon, and Andrew Beng Jin Teoh. Understanding the feature norm for out-of-distribution detection. In ICCV, pages 1557–1567, 2023. [15] Ricky TQ Chen and Yaron Lipman. Flow matching on general geometries. arXiv preprint, 2023. [16] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint, 2022. [17] Yongming Rao, Wenliang Zhao, Guangyi Chen, Yansong Tang, Zheng Zhu, Guan Huang, Jie Zhou, and Jiwen Lu. Denseclip: Language-guided dense prediction with context-aware prompting. In CVPR, pages 18082–18091, 2022. [18] Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In ICCV, pages 11975–11986, 2023. [19] Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In ICML, pages 4904–4916. PMLR, 2021. 10
[20] Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint, 2024. [21] Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In ICML, pages 2790–2799. PMLR, 2019. [22] Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with feature adapters. IJCV, 132(2):581–595, 2024. [23] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint, 2022. [24] Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint, 2022. [25] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. Highresolution image synthesis with latent diffusion models. In CVPR, pages 10684–10695, 2022. [26] Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijnden, and Saining Xie. Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers. In ECCV, pages 23–40. Springer, 2024. [27] William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, pages 4195–4205, 2023. [28] Hongxu Chen, Hongxiang Li, Zhen Wang, and Long Chen. Bi-anchor interpolation solver for accelerating generative modeling. arXiv preprint, 2026. [29] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NeurIPS, 33:6840–6851, 2020. [30] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint, 2020. [31] Barrett O’neill. Semi-Riemannian geometry with applications to relativity, volume 103. Academic press, 1983. [32] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In ICML, 2024. [33] Jiayan Teng, Wendi Zheng, Ming Ding, Wenyi Hong, Jianqiao Wangni, Zhuoyi Yang, and Jie Tang. Relay diffusion: Unifying diffusion process across resolutions for image synthesis. arXiv preprint, 2023. [34] Emiel Hoogeboom, Jonathan Heek, and Tim Salimans. simple diffusion: End-to-end diffusion for high resolution images. In ICML, pages 13213–13232. PMLR, 2023. [35] Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Finegrained visual classification of aircraft. arXiv preprint, 2013. [36] Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. J-STARS, 12(7):2217–2226, 2019. [37] Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In CVPR, pages 3606–3613, 2014. [38] Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In CVPR, pages 3485–3492. IEEE, 2010. 11
[39] Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In ICCV, pages 554–561, 2013. [40] Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In CVPR, pages 3498–3505. IEEE, 2012. [41] Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint, 2012. [42] Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In ICVGIP, pages 722–729. IEEE, 2008. [43] Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In CVPR, pages 178–178. IEEE, 2004. [44] Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101–mining discriminative components with random forests. In ECCV, pages 446–461. Springer, 2014. [45] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248–255. Ieee, 2009. [46] Renrui Zhang, Wei Zhang, Rongyao Fang, Peng Gao, Kunchang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li. Tip-adapter: Training-free adaption of clip for few-shot classification. In ECCV, pages 493–510. Springer, 2022. [47] Guangyi Chen, Weiran Yao, Xiangchen Song, Xinyue Li, Yongming Rao, and Kun Zhang. Plot: Prompt learning with optimal transport for vision-language models. arXiv preprint, 2022. [48] Hantao Yao, Rui Zhang, and Changsheng Xu. Visual-language prompt tuning with knowledgeguided context optimization. In CVPR, pages 6757–6767, 2023. [49] Beier Zhu, Yulei Niu, Yucheng Han, Yue Wu, and Hanwang Zhang. Prompt-aligned gradient for prompt tuning. In ICCV, pages 15659–15669, 2023. [50] Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. NeurIPS, 37:56424–56445, 2024. [51] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint, 2017.
12
A
Angular Truncation Error of WP-FM
In this section, we provide a detailed derivation of the angular truncation error for first-order Riemannian ODE solvers under the warped product geometry introduced in Sec. 3. Proposition 1 (Angular Truncation Error). Let (M, gϕ ) be the warped product manifold defined in Sec. 3.1: M = Mr ×gϕ Mθ , gϕ = dr2 + ϕ(r)2 dθ 2 , (8) where Mr = R>0 , Mθ = Sd−1 , ϕ ∈ C 2 (R>0 ) and ϕ(r) > 0. Let γ(t) = (γr (t), γθ (t)) be a unit-speed geodesic defined on t ∈ [0, T ]. Denote the angular velocity magnitude (speed) as ω(t) := ∥γ̇θ (t)∥2 . Consider the first-order Riemannian integrator applied to the angular component: θ γ̂θ (t + ∆t) = ExpM (9) γθ (t) ∆t γ̇θ (t) . Then there exists a constant C > 0, depending only on bounded trajectory quantities, such that the angular truncation error ϵθ satisfies: ϕ′ (γr (t)) ϵθ := dMθ γ̂θ (t + ∆t), γθ (t + ∆t) ≤ C ∆t2 γ̇r (t) ∥γ̇θ (t)∥2 + O(∆t3 ). ϕ(γr (t))
(10)
Moreover, under the non-degeneracy condition γ̇r (t) ̸= 0 and γ̇θ (t) ̸= 0, the leading-order term vanishes if and only if: ϕ′ (r) ≡ 0. (11) Proof. Local truncation error on Riemannian manifolds. Lemma 1 (Exponential Euler local error). Let (N , h) be a Riemannian manifold and x(t) a C 2 curve. Then: 1 dN Expx(t) (∆t ẋ(t)), x(t + ∆t) = ∆t2 ∥∇ẋ ẋ∥h + O(∆t3 ), 2 where the remainder is uniform on compact intervals.
(12)
Levi-Civita connection under warped product geometry. For γ(t) = (γr (t), γθ (t)), the LeviCivita connection under metric gϕ satisfies: ϕ′ (γr ) D γ̇θ + 2 γ̇r γ̇θ . ∇γ̇ γ̇ = γ̈r − ϕ(γr )ϕ′ (γr )∥γ̇θ ∥22 , (13) dt ϕ(γr ) Since γ is a geodesic on M, we have: ∇γ̇ γ̇ = 0,
(14)
which directly implies the angular acceleration component is strictly zero: ϕ′ (γr ) D γ̇θ = −2 γ̇r γ̇θ . dt ϕ(γr )
(2)
Angular truncation error. Applying Lemma 1 on the angular sub-manifold Mθ gives: ϵθ =
1 2 D ∆t γ̇θ + O(∆t3 ). 2 dt 2
(15)
Substituting Eq. (2), we obtain the magnitude of the angular acceleration: D ϕ′ (γr ) γ̇θ = 2 γ̇r ∥γ̇θ ∥2 . dt ϕ(γr ) 2
(16)
Therefore, the truncation error is: ϵθ = ∆t2
ϕ′ (γr ) γ̇r ∥γ̇θ ∥2 + O(∆t3 ). ϕ(γr ) 13
(17)
Characterization of vanishing error. The leading-order term vanishes iff: ϕ′ (γr ) γ̇r ∥γ̇θ ∥2 = 0. ϕ(γr )
(18)
Under the non-degeneracy condition γ̇r ̸= 0 and γ̇θ ̸= 0, this holds iff: ϕ′ (r) ≡ 0,
(19)
i.e., the warping function ϕ is constant. This result characterizes the source of angular dynamics distortion in WP-FM. Any non-constant ′ (r) warping function introduces a coupling term proportional to ϕϕ(r) γ̇r , which induces angular acceleration (Eq. (2)) and leads to a second-order truncation error on the angular sub-manifold of pre-trained cross-modal features. In the Euclidean case ϕ(r) = r, this coefficient reduces to γ̇γrr , which becomes large when γr → 0, explaining the instability of angular updates observed in Sec. 3.
B
Radial Information Reflects Modality Confidence
Density
As illustrated in Figure 5, we visualize the statistical distribution of the Cars radial magnitudes (i.e., the L2 norm) Aircraft Pets of image features extracted by the pretrained CLIP image encoder across three different datasets. The overall magnitude distribution exhibits a clear hierarchical trend, ranking as OxfordPets > StanfordCars > FGVCAircraft. Notably, this trend aligns perfectly with the zero-shot classification accuracy of the pre-trained model on these respective datasets: 89.1% Radial Magnitude for OxfordPets, 65.5% for StanfordFigure 5: Radial Magnitude Distribution across Datasets. Cars, and 24.8% for FGVCAircraft. This positive correlation provides compelling empirical evidence that the radial dimension intrinsically encodes the model’s modality confidence. Specifically, when the model processes images from a domain it is highly confident in (such as OxfordPets), it produces features with larger radial magnitudes. Conversely, for more challenging or out-of-distribution domains (such as FGVCAircraft), the resulting feature magnitudes are significantly smaller. Consequently, these findings emphasize that radial information serves as a crucial explicit signal for distinguishing in-distribution data from uncertain samples, and discarding it inherently blinds the velocity network, degrading the performance observed in the ablation study.
C
Limitations
To resolve the alignment of highly entangled cross-modal distributions, this method (DP-FM) models the alignment process as a continuous multi-step probability flow. Therefore, during the inference phase, the model relies on multi-step integration to progressively transport features, which differs from traditional parameter-efficient fine-tuning (PEFT) methods that only require a single-step adjustment. Objectively speaking, this does mean that multiple steps must be executed during runtime. However, this multi-step mechanism is precisely the core foundation that enables the Flow Matching framework to improve alignment and achieve enhanced performance.
14