COMPLEX INTERPOLATION OF MATRICES WITH AN APPLICATION TO MULTI-MANIFOLD LEARNING
arXiv:2604.14118v1 [cs.LG] 15 Apr 2026
ADI ARBEL∗ , STEFAN STEINERBERGER† , AND RONEN TALMON‡ Abstract. Given two symmetric positive-definite matrices A, B ∈ Rn×n , we study the spectral properties of the interpolation A1−x B x for 0 ≤ x ≤ 1. The presence of ‘common structures’ in A and B, eigenvectors pointing in a similar direction, can be investigated using this interpolation perspective. Generically, exact log-linearity of the operator norm ∥A1−x B x ∥ is equivalent to the existence of a shared eigenvector in the original matrices; stability bounds show that approximate log-linearity forces principal singular vectors to align with leading eigenvectors of both matrices. These results give rise to and provide theoretical justification for a multi-manifold learning framework that identifies common and distinct latent structures in multiview data. Key words. matrix interpolation, spectral analysis, singular values, eigenvector alignment, positive definite matrices, manifold learning, multimodal data MSC codes. 15A18, 47A56, 65F35, 68T10
1. Introduction and Results. 1.1. The problem. Let A, B ∈ Rn×n be two symmetric and positive definite matrices. We assume that A has eigenvalues σ(A) = {λ1 , . . . , λn } and B has eigenvalues σ(B) = {µ1 , . . . , µn }. We make no additional assumptions on A and B and are motivated by the question whether the eigenvectors of A could, in some natural way, be matched with the eigenvectors of B (the underlying motivation comes from a concrete application discussed in section 2). This question is sufficiently vague that many solutions are possible: for example, one could think of the eigenvectors as two sets of n vectors on Sn−1 and then match them by minimizing over some notion of distance. A particular way of matching spectra was proposed in the multimodal manifold learning literature [14] (see section 2 and section 4 for details); its effectiveness in concrete applications motivated our interest in the underlying theory. Since A and B are symmetric and positive definite, their powers A1−z and B z are well-defined for any z ∈ C and, in particular, for real 0 ≤ x ≤ 1. A1−x and B x are also symmetric and positive definite. Their product A1−x B x is not necessarily diagonalizable; however, it is a square matrix that has at least n singular values. One could now try to understand the singular values of A1−x B x for 0 ≤ x ≤ 1. The main purpose of our paper is to show that the singular values, i.e. the n real-valued functions x → σk (A1−x B x ), for 0 ≤ x ≤ 1, 1. are an interesting object with an interesting underlying mathematical structure (see subsection 1.2 and subsection 1.3) 2. which prove to be useful in specific applications; we discuss the case of multimanifold learning in section 2 and section 4. A very rough motivation is as follows: one sometimes measures the same object in different ways which may end up resulting in two different symmetric positivedefinite (kernel) matrices; however, these two matrices should correspond to the same ∗ Viterbi Faculty of Electrical and Computer Engineering, Technion – Israel Institute of Technology, Haifa, Israel ([email protected]). † Department of Mathematics and Department of Applied Mathematics, University of Washington, Seattle, WA 98195, USA ([email protected]). ‡ Viterbi Faculty of Electrical and Computer Engineering, Technion – Israel Institute of Technology, Haifa, Israel ([email protected]).
1
2
A. ARBEL, S. STEINERBERGER, AND R. TALMON
underlying ‘ground truth’, and this similarity should be reflected in their spectrum, where there should be a natural ‘bijection’ between eigenvectors. An example is shown in Figure 1: the same overall geometry is captured by similar eigenvectors (used here to color the point clouds).
Fig. 1: Two 2D projections of a 3D point cloud of a cow on two different view angles: given only the two two-dimensional point sets (discretized via a kernel into two symmetric positive-definite matrices A, B ∈ Rn×n ), can one automatically discover that the eigenstructure captures the same underlying ground truth? (Details in section 2 and section 4).
1.2. Identifying Common Eigenvectors. We are now able to motivate the first basic result: the largest singular value or, equivalently, the operator norm σ1 (A1−x B x ) = ∥A1−x B x ∥ has the property that ∥A1−x B x ∥ ≤ ∥A∥1−x ∥B∥x . Moreover, we will argue that for generic pairs of matrices A, B, we have equality if and only if their operator norm is realized by a shared eigenvector v ∈ Rn normalized to ∥v∥ℓ2 = 1 which satisfies ∥A∥ = ∥Av∥ = ∥Bv∥ = ∥B∥. One direction is simple: log ∥A1−x B x ∥ ≤ log ∥A1−x ∥∥B x ∥ = log ∥A∥1−x ∥B∥x = log ∥A∥1−x + log (∥B∥x ) = (1 − x) log ∥A∥ + x log ∥B∥. If A and B have a common eigenvector v ∈ Rn , say Av = λv and Bv = µv, then ∥A1−x B x v∥ = µx ∥A1−x v∥ = λ1−x µx , for which the logarithm is linear. One could now wonder about the inverse result: does the linearity of log ∥A1−x B x v∥ imply that v is an eigenvector of both An and 1−x x B? If, for example, B = 2A, then log ∥A B v∥ is linear for each v ∈ R , we therefore have to ensure that A and B are ‘different’. Our assumption will be that the ratio of two eigenvalues λi /µj uniquely identifies λi and µj (a property satisfied by generic pairs of matrices).
COMPLEX INTERPOLATION OF MATRICES
3
Theorem 1 (Identifiability). Let A, B ∈ Rn×n be two symmetric and positive definite matrices. Suppose the map d : σ(A) × σ(B) → R>0 defined by d(λ, µ) = λ/µ is injective and suppose there exists 0 ̸= v ∈ Rn such that log A1−x B x v
is linear,
then v is an eigenvector of both A and B. For example, if A, B ∈ Rn×n are two diagonal matrices with entries that are chosen uniformly at random from [1, 2] that are then sorted in decreasing order. The n functions log σk (A1−x B x ) then form, almost surely, n lines. 1.3. A stability version. While Theorem 1 is an encouraging fact, it is only applicable if the leading eigenvectors of A and B are exactly the same; this is rarely the case in practical applications. Luckily, Theorem 1 remains ‘morally’ true in the case when A and B ‘almost’ share an eigenvector. To simplify exposition, we remove the scaling symmetry and assume without loss of generality that λ1 (A) = ∥A∥ = 1 = ∥B∥ = λ1 (B). Then ∥A1−x B x ∥ ≤ 1 and Theorem 1 states that subject to the genericity assumption, the case of equality ∥A1−x B x ∥ = 1 for some 0 < x < 1 implies that A and B have an eigenvector in common. We can now state the main stability result: if ∥A1−x B x ∥ is very close to 1, then the left principal singular vector u ∈ Rn of A1−x B x has to point in nearly the same direction as the leading eigenvector Aa1 = ∥A∥a1 = a1 . Moreover, the right principal singular vector v ∈ Rn of A1−x B x has to point in nearly the same direction as the leading eigenvector Bb1 = ∥B∥b1 = b1 . Theorem 2 (Stability). Let A, B ∈ Rn×n be symmetric, positive definite, normalized to ∥A∥ = 1 = ∥B∥. We assume a1 and b1 , satisfying Aa1 = a1 and Bb1 = b1 , are ℓ2 -normalized eigenvectors corresponding to the eigenspace associated with eigenvalue 1, which has multiplicity 1. Let λ2 (A) < 1 and λ2 (B) < 1 denote the second largest eigenvalues of A and B, respectively. Let 0 ≤ x ≤ 1 and let u ∈ Rn and v ∈ Rn be the principal left and right singular vectors of A1−x B x , respectively. Then (1.1)
2
|⟨u, a1 ⟩| ≥ 1 −
1 − ∥A1−x B x ∥2 , (1 − x) (1 − λ2 (A))
and (1.2)
2
|⟨v, b1 ⟩| ≥ 1 −
1 − ∥A1−x B x ∥2 . x (1 − µ2 (B))
Remarks. Several remarks are in order. 1. Theorem 2 states that if log ∥A1−x B x ∥ is close to a line (i.e. ∥A1−x B x ∥ is close to 1), then the left principal singular vector of A1−x B x is close (in inner product) to the eigenvector a1 of A corresponding to the largest eigenvalue of A and the right principal singular vector of A1−x B x is close to the eigenvector b1 of B corresponding to the largest eigenvalue of B. 2. The factor (1 − x) in (1.1) and x in (1.2) are natural: one cannot hope to get too much information about A from Aε B 1−ε when 0 < ε ≪ 1. Likewise, one would not expect A1−ε B ε to provide high quality information about B uniformly as ε → 0+ . 3. The factors controlling the spectral gap 1 − λ2 (A) and 1 − µ2 (B) are also natural since we are making a pointwise statement about the eigenvectors a1 , b1 . If the spectral gap is small, then ∥Aa2 ∥ = λ2 (A) ∼ 1 may almost realize the operator norm. We note that the bound presented has a tighter version using deeper spectral components (see subsection 3.3).
4
A. ARBEL, S. STEINERBERGER, AND R. TALMON
1.4. Related results. We are not aware of any such results in the literature; however, there are some philosophically related ideas. Our main motivation is a matrix inequality of Alan McIntosh [17], which generalizes a number of older inequalities: if A, B ∈ Rn×n is symmetric and positive-definite and X ∈ Rn×n is arbitrary, then for any 0 < r < 1 ∥Ar XB 1−r ∥ ≤ ∥AX∥r ∥XB∥1−r . This is known to imply the Löwner-Heinz inequality [16], the Heinz-Kato inequality [11, 13], the Cordes inequality [6], and several other such results. The approach of McIntosh is to consider complex interpolation of operators z → Az XB 1−z v in combination with the maximum principle; it was pointed out by one of the authors [23] that such an argument comes, automatically, with stability estimates: for the maximum principle to be sharp, there cannot be too much oscillation; this argument was then carried out in [23]. Our arguments follow the same philosophical line of reasoning to obtain a similar structural result in our setting. Our work, when seen as an application to multi-manifold learning, is closely related to a kernel-based approaches. A key method in this direction is alternating diffusion [15, 24]: given two matrices A, B ∈ Rn×n constructed from different modalities, alternating diffusion considers their (unweighted) product AB to form a composite diffusion operator. It was shown that the leading singular vectors of the product operator are associated with the geometry of the common latent variables. Several extensions of this idea have been proposed: these include composite diffusion operators [21], using geodesic interpolation under the affine-invariant Riemannian metric [22, 14], compositions of diffusion-type operators across time [9, 10]. Another related line of research seeks functions that are jointly smooth with respect to multiple kernels [7, 5]. A classical and conceptually related notion of commonality is provided by canonical correlation analysis (CCA) [12] and the extension to Kernel CCA [1, 2] and nonparametric CCA [18]. We are not aware of a fine analysis of complex interpolation of operators having been previously used in the context of multi-manifold learning. 2. Application to Multi-Manifold Learning. We briefly describe how the interpolation framework introduced above arises in multimodal manifold learning (more details can be found in Section 4). We consider two datasets consisting of aligned point clouds (1) (2) {si }ni=1 ⊂ M1 ⊂ Rd1 , {si }ni=1 ⊂ M2 ⊂ Rd2 , (1)
(2)
where each pair (si , si ) corresponds to two observations of two manifolds M1 and M2 embedded in Euclidean spaces. This setting naturally arises in multimodal data analysis, where different sensing mechanisms capture complementary views of a common phenomenon of interest. From each point cloud, we construct a symmetric and positive-definite kernel matrix using pairwise affinities via (1) (1) (2) (2) Aij = exp −∥si − sj ∥2 /ε(1) , Bij = exp −∥si − sj ∥2 /ε(2) , followed by standard normalization (see Section 4 for details) resulting in symmetric positive-definite A, B ∈ Rn×n . The central object of interest is the interpolated family γ(x) = A1−x B x ,
x ∈ [0, 1],
whose spectral properties encode relationships between the two point clouds. To analyze γ(x), we consider the singular values of A1−x B x as functions of x. This
COMPLEX INTERPOLATION OF MATRICES
5
Fig. 2: Two aligned point clouds sampled from two 2D cylinders embedded in R3 . The height (red) axis is shared among the point clouds, whereas the azimuthal angle is distinct.
leads to the construction of a singular value flow diagram (SVFD), which tracks the evolution of the leading singular values along the interpolation path.
Fig. 3: The SVFD of the point clouds depicted in Figure 2. The empirical singular values of the interpolated kernels connecting the second common eigenvalues (shown left and right) are highlighted in yellow.
In practice, this is done by sampling a discrete set of points xk ∈ [0, 1], computing the leading singular values of A1−xk B xk at each point, and plotting their logarithms as functions of xk . The resulting curves provide a compact representation of how spectral components evolve between the two matrices. Our theoretical results in section 1 provide a rigorous interpretation of these diagrams: approximately log-linear trajectories correspond to spectral components shared between A and B, while curved trajectories indicate distinct components. We illustrate the approach on a synthetic example consisting of two cylindrical manifolds with a shared latent variable. The construction is described in detail in section 4. Figure 2 shows the sampled point clouds, where the vertical coordinate represents a common latent variable, while the angular coordinates differ between the two
6
A. ARBEL, S. STEINERBERGER, AND R. TALMON
datasets. The resulting SVFD is shown in Figure 3. In this figure, several singular value trajectories exhibit near log-linear behavior across the interpolation parameter x, indicating spectral components that are shared between the two point clouds. In particular, the highlighted trajectory (shown in yellow) closely follows a straight line in the logarithmic scale, consistent with the theoretical characterization of common eigenvectors. The insets in Figure 3 further illustrate this phenomenon by coloring the two cylindrical point clouds according to the corresponding left and right singular vectors at an intermediate interpolation point (here x = 0.5). The coloring reveals a coherent structure across both point clouds, with the variation aligned along the vertical axis, confirming that this spectral component captures the common latent variable. In contrast, in Figure 4, we highlight a trajectory associated with a distinct
Fig. 4: The same as Figure 3, but highlighting the empirical singular values originating at x = 0 from the fourth largest eigenvalue of C1 . The point clouds on the left and right are colored by the corresponding eigenvector. The curve is far from a line, the eigenvectors have little in common.
component. In the SVFD, this trajectory deviates significantly from log-linearity, exhibiting pronounced curvature. This behavior reflects the lack of a shared eigenstructure between the corresponding components of A and B. The insets in Figure 4 show the cylinders colored using the singular vector associated with this trajectory. Unlike the previous case, the coloring patterns are not consistent across the two point clouds: a structured harmonic pattern visible on one cylinder does not transfer coherently to the other. This lack of geometric alignment indicates that the corresponding spectral component does not represent a shared structure. This example demonstrates that the geometry of the singular value trajectories provides a direct and interpretable signature of common versus distinct spectral components. 3. Proofs. 3.1. Proof of Theorem 1. Proof of Theorem 1. Let us assume that log A1−x B x v, A1−x B x v
is linear.
This means that there exist c1 , c2 ∈ R such that A1−x B x v, A1−x B x v = c1 ec2 x . We
7
COMPLEX INTERPOLATION OF MATRICES
first start by simplifying the expression: assuming that n X
Av =
λk ⟨v, ak ⟩ ak
and
Bv =
k=1
n X
µk ⟨v, bk ⟩ bk ,
k=1
we have Bxv =
n X
µxl ⟨v, bl ⟩ bl
l=1
and furthermore A1−x B x v =
n X
λ1−x ⟨B x v, ak ⟩ ak k
k=1
and therefore A
1−x
x
1−x
B v, A
x
B v =
* n X
λ1−x ⟨B x v, ak ⟩ ak , k
k=1
=
n X
n X
+ λ1−x ⟨B x v, ak ⟩ ak k
k=1 2
λ2−2x ⟨B x v, ak ⟩ . k
k=1
Altogether, this implies A
1−x
x
1−x
B v, A
x
B v =
n X
n X
λ2−2x k
k=1
!2 µxl ⟨v, bl ⟩ ⟨bl , ak ⟩
= c1 ec2 x .
l=1
For the remainder of the argument, we will exploit the algebraic structure: writing αk = log (λk ), βl = log (µl ),
and
ck,l = λk ⟨v, bl ⟩ ⟨bl , ak ⟩
allows to notationally simplify the equation to n X
e−2αk x
k=1
n X
!2 ck,l eβl x
= c1 ec2 x .
l=1
We note that v ̸= 0 and both A, B are positive definite, their eigenvectors form a basis of Rn and therefore, there exists at least one ck,l ̸= 0. Let us now first assume that A and B are both simple: their eigenvalues have multiplicity 1. We then define the quantities σ = min {2βl − 2αk : ck,l ̸= 0}
σ = max {2βl − 2αk : ck,l ̸= 0} .
and
These numbers give the smallest and largest occurring frequencies: since the spectrum is assumed to be simple, for each k there exists at most one l such that 2βl − 2αk is maximized or minimized. Therefore, !2 n n n X X X X c2l,k + dj eγj x e−2αk x ck,l eβl x = eσx k=1
j
k,l=1 2βl −2αk =σ
l=1
+ eσx
n X k,l=1 2βl −2αk =σ
c2l,k ,
8
A. ARBEL, S. STEINERBERGER, AND R. TALMON
where the dj , γj could be explicitly computed and the arising frequencies satisfy σ < γj < σ. Note that the cross terms of the form βl + βm − 2αk for l ̸= m do not affect the extreme frequencies, and therefore, absorbed into the intermediate terms dj eγj x . In addition, by construction, n X
n X
c2l,k ̸= 0 ̸=
k,l=1 2βl −2αk =σ
c2l,k .
k,l=1 2βl −2αk =σ
However, in order for this expression to be c1 ec2 x , we have to have σ = c2 = σ. This implies that whenever ck,l ̸= 0, then 2βl − 2αk = c2 . This equation, in turn, has a unique solution (due to the assumption of an injective d(µ, λ) = µλ ) from which we deduce that there exists a single pair (k, l) such that ck,l ̸= 0. This means that there exists a single pair (k, l) for which ⟨v, bl ⟩ ⟨bl , ak ⟩ ̸= 0. We note that for each l there exists at least one k for which ⟨bl , ak ⟩ ̸= 0. This means there exists exactly one l such that ⟨v, bl ⟩ ̸= 0 which implies that v = bl ∥v∥ and therefore v is an eigenvector of B. Then, however, fixing this value of l, there can exist at most one k such that ⟨bl , ak ⟩ ̸= 0, which implies that v is also an eigenvector of A. It remains to deal with the general case. If the eigenvalues of both matrices can have multiplicities, then, arguing in exactly the same way as above, we see that we can write !2 n n X X X βl x −2αk x ck,l e = eσx A1 + e dj eγj x + eσx A2 j
l=1
k=1
where the expressions for A1 and A2 are now slightly more involved. Since 2βl −2αk = σ has a unique solution, we can call the corresponding eigenvalues α and β (keeping in mind that they might have a nontrivial multiplicity). A short computation shows that 2 2 X X X X A1 = ck,l = λk ⟨v, bl ⟩ ⟨bl , ak ⟩ αk =α
βl =β
αk =α
X
X
= e2α
αk =α
βl =β
2
⟨v, bl ⟩ ⟨bl , ak ⟩ .
βl =β
We recall an elementary fact for Hilbert spaces: if {x1 , . . . , xn } is an orthonormal basis of a subspace S of some Hilbert space, then for all x ∈ S and all y ∈ H, * n + n X X ⟨x, y⟩ = ⟨x, xi ⟩ xi , y = ⟨x, xi ⟩ ⟨xi , y⟩ . i=1 n
i=1
n
Therefore, using πβ : R → R to denote the orthogonal projection onto the eigenspace corresponding to eigenvalue eβ , we have X 2 A1 = e2α ⟨πβ v, ak ⟩ . αk =α
9
COMPLEX INTERPOLATION OF MATRICES
Using the Pythagorean theorem and using πα : Rn → Rn to denote the orthogonal projection onto the eigenspace corresponding to eigenvalue eα , we have X 2 2 A1 = e2α ⟨πβ v, ak ⟩ = e2α ∥πα πβ v∥ . αk =α
We note that if, for any eigenvalue β, the vector πβ v ̸= 0, then there exists at least one other eigenvalue α for which πα πβ ̸= 0. This means that v has to be an eigenvector of B. Simultaneously, for any vector v ̸= 0 there exists at least one eigenvalue α such that πα v ̸= 0. This proves the desired statement. 3.2. Proof of Theorem 2. 3.2.1. Preliminaries. We first recall that if A ∈ Rn×n is a symmetric and positive definite matrix with eigenvalues and eigenvectors given by Aak = λk ak , then the complex power Az for z ∈ C is defined by z
A v=
n X
λzk ⟨v, ak ⟩ ak .
k=1
The purpose of this section is to recall some basic facts regarding complex powers of linear operators. Lemma 1. If A ∈ Rn×n is a symmetric and positive definite matrix, then Ait is unitary for all t ∈ R. Proof. Note that, for λ ≥ 0, λit = cos ((log λ) t) + i sin ((log λ) t) = ei(log λ)t . The result then follows by explicit computation since n X
2
λit k ⟨v, ak ⟩ ak
= ℜ
n X
λit k ⟨v, ak ⟩ ak + ℑ
k=1
k=1
=
n X
n X
2
λit k ⟨v, ak ⟩ ak
k=1 2
cos (log (λk ) t) ⟨v, ak ⟩ ak
k=1
+ = =
n X
2
sin ((log λk ) t) ⟨v, ak ⟩ ak
k=1 n X
2 2 cos ((log λk ) t) + sin2 ((log λk ) t) |⟨v, ak ⟩|
k=1 n X
2
|⟨v, ak ⟩| = ∥v∥2 .
k=1
Lemma 2. If A, B ∈ Rn×n are two symmetric and positive definite matrices normalized to ∥A∥ = 1 = ∥B∥. Then, for all 0 ≤ ℜ(z) ≤ 1, we have ∥A1−z B z ∥ ≤ 1.
10
A. ARBEL, S. STEINERBERGER, AND R. TALMON
Proof. The Cauchy-Schwarz inequality is still valid in the holomorphic case since |⟨v, w⟩R | =
n X
vi wi ≤
i=1
n X
|vi wi | ≤
i=1
n X
! 21 |vi |
2
i=1
n X
! 12 |wi |
2
= ∥v∥∥w∥.
i=1
Now, consider z = 0 + it for t ∈ R. Using Cauchy-Schwarz, we have that A1−it B it v, A1−it B it v R ≤ A1−it B it v
2
.
A−it and B it are unitary matrices and ∥v∥ = 1, therefore A1−it B it v = AB it v ≤ ∥A∥ B it v = 1. By the same reasoning, we have that for z = 1 + it A−it B 1+it v = B 1−it v ≤ ∥B∥ B −it v = 1. Using the trivial estimate ∥A1−z B z v∥ ≤ ∥A1−z ∥∥B z ∥ ≤ ∥A∥1−ℜ(z) ∥B∥ℜ(z) ≤ max(∥A∥, 1) max(∥B∥, 1) Under the assumption ∥A∥ = ∥B∥ = 1 we obtain the desired bound. We conclude with a short Lemma for a harmonic function defined on the strip D = {z ∈ C : 0 ≤ ℜ(z) ≤ 1} . Lemma 3. Let F : D → R be a harmonic function satisfying ∥F ∥L∞ (D) ≤ 1. If, for some 0 ≤ x ≤ 1, we have F (x, 0) ≥ 1 − ε, then we have max F (0, y) ≥ 1 − y∈R
ε 1−x
and
max F (1, y) ≥ 1 − y∈R
ε . x
Proof. Let (Zt )t≥0 be a standard two-dimensional Brownian motion starting at Z0 = (x, 0), and let τ = inf{t > 0 : Zt ∈ / D} be the first exit time from the domain. Zt is an Itô process by definition. Since F is harmonic it is a twice continuously differentiable function, so by Itô’s formula [19] F (Zt ) is also an Itô process, whose evolution is given by: dF (Zt ) =
∂F 1 (Zt )dt + ∇F (Zt ) · dZt + ∆F (Zt ) d2 Zt . ∂t 2
Since F is harmonic and time invariant, ∆F = ∂F ∂t = 0, , the drift term dt as well as the second order term vanish. Consequently, the process Mt = F (Zt ) is a local martingale (a drift-less process). Furthermore, since F is bounded (∥F ∥L∞ ≤ 1) and the Brownian motion exits the strip D almost surely, the conditions for the Optional Stopping Theorem are satisfied. This allows us to equate the function’s value at the starting point to its expected value at the exit time F (x, 0) = E[M0 ] = E[Mτ ] = E [F (Zτ )] . The boundary ∂D consists of the left line L = {0} × R and the right line R = {1} × R. The probability of the Brownian motion exiting through the right boundary corresponds to the initial location along the x axis: P(Zτ ∈ L) = 1 − x,
and
P(Zτ ∈ R) = x.
COMPLEX INTERPOLATION OF MATRICES
11
We decompose the expectation over these two exit events: F (x, 0) = (1 − x) · E[F (Zτ ) | Zτ ∈ L] + x · E[F (Zτ ) | Zτ ∈ R]. Using the assumption F (x, 0) ≥ 1 − ε and the global bound supz∈∂D F (z) ≤ 1, 1 − ε ≤ (1 − x) sup F (0, y) + x sup F (1, y) y∈R
y∈R
≤ (1 − x) sup F (0, y) + x · 1. y∈R
Rearranging the inequality yields: (1 − x) sup F (0, y) ≥ 1 − x − ε, y∈R
which implies sup F (0, y) ≥ 1 − y∈R
ε . 1−x
The bound for the right boundary follows by symmetry. 3.2.2. Sketch of the proof. We follow a similar approach as in [23]. We will be working on the fundamental strip (3.1)
D = {x + iy ∈ C : x ∈ [0, 1] , y ∈ R} .
y
.. .
D x=0
x x=1
.. . Fig. 5: A sketch of the domain D.
Instead of analyzing the norm of the interpolated operator directly, we will, for any arbitrary v ∈ Rn , study the behavior of the expression x → A1−x B x v, A1−x B x v as a function of 0 ≤ x ≤ 1. We note that if v happens to be the principal singular vector of A1−x B x , then this expression is merely the square of the operator norm. Such quantities are often easier to analyze in the complex plane, and we will generalize the interpolation scheme from the real interval to D to the fundamental strip by instead considering the functions z → A1−z B z v, A1−z B z v R as well as the
12
A. ARBEL, S. STEINERBERGER, AND R. TALMON
dual object z → u∗ A1−z B z , u∗ A1−z B z R both of which are holomorphic by defining Pn ⟨v, w⟩R ≜ k=1 vi wi . We note that this reduces to the earlier expression whenever z = t + 0i for 0 ≤ t ≤ 1. We can now make use of the fact that any holomorphic function inside a domain is uniquely defined by its values on the boundary. Moreover, since the fundamental strip D is geometrically rather simple, this is completely explicit (see e.g. [25]). Every analytic complex-valued function f : D 7→ C can be represented as follows Z ∞ 1 P (x, t − y) · f (0 + it) dt f (x + iy) = 2π −∞ Z ∞ 1 + P (1 − x, t − y) · f (1 + it) dt, 2π −∞ where P is a Poisson kernel given by (3.2)
P (x, y) =
π sin (πx) . cosh (πy) − cos (πx)
This allows us to reduce the analysis of the special case A1−z B z for 0 ≤ ℜ(z) ≤ 1 to that of A1−it B it as well as A−it B 1+it . 3.2.3. Proof. Proof. We fix 0 ≤ x ≤ 1 and define ε via the relationship A1−x B x
2
= 1 − ε.
This means that for the principal right singular vector v with ∥v∥ = 1, we have A1−x B x v, A1−x B x v R ≥ 1 − ε. Applying Lemma 3, we deduce the existence of t ∈ R such that ℜF (1 + it) = ℜ A−it B 1+it v, A−it B 1+it v R ≥ 1 −
ε x
Using the definition of complex powers, we have A−it B 1+it v =
n X
λ−it B 1+it v, ak ak k
k=1
allowing us to rewrite ℜF (1 + it) as ℜF (1 + it) = ℜ A−it B 1+it v, A−it B 1+it v R * n + n X X −it −it 1+it 1+it =ℜ λk B v, ak ak , λk B v, ak ak =ℜ
k=1 n n XX
k=1 −it λ−it B 1+it v, ak k λk ′
k=1 k′ =1
=
n X k=1
2 ℜ λ−2it B 1+it v, ak k
R
B 1+it v, ak′ ⟨ak , ak′ ⟩R
13
COMPLEX INTERPOLATION OF MATRICES
This implies the existence of t ∈ R such that 1−
n
n
k=1
k=1
X X ε 2 2 1+it ≤ ℜ λ−2it B v, a ≤ λ−2it B 1+it v, ak . k k k x
We first observe that λit = cos ((log λ) t) + i sin ((log λ) t) = ei(log λ)t and therefore λ−2it has a magnitude of 1. Therefore k n
1−
X ε ≤ B 1+it v, ak x
2
k=1
n
For any w ∈ C , since the eigenvectors ak form a basis of Rn , n X
2
|⟨ℜw + iℑw, ak ⟩| =
k=1
n X
2
2
|⟨ℜw, ak ⟩| + |⟨ℑw, ak ⟩|
k=1 2
2
2
= ∥ℜw∥ + ∥ℑw∥ = ⟨w, w⟩ = ∥w∥ . Therefore, recalling that purely imaginary powers are unitary, ε 2 2 1 − ≤ B 1+it v = ∥B v∥ . x Using the spectral theorem combined with the fact that the largest eigenvalue of B is 1 (and simple) and the second largest eigenvalue is µ2 < 1, we have 1−
ε 2 2 ≤ ∥B v∥ ≤ ⟨v, b1 ⟩ + µ2 ∥πb⊥ v∥2 1 x 2
2
= ⟨v, b1 ⟩ + µ2 1 − ⟨v, b1 ⟩ = µ2 + (1 − µ2 ) ⟨v, b1 ⟩
2
This implies the desired inequality for B. The exact same analysis can be applied to the boundary z = 0 + it, yielding the second part of the statement 3.3. Refinements. The proof implies a slightly stronger statement. It is easily seen that the argument shows, for example, that for any vector v ∈ Rn for which ∥A1−x B x v∥ = ∥A1−x B x ∥, we automatically have that 2
∥Bv∥ ≥ 1 −
1 − ∥A1−x B x ∥2 x
which forces ∥Bv∥ to be large. ∥Bv∥ being large in combination with a spectral gap automatically forces that v has a large inner product with the leading eigenvector. However, this is also the worst case; in practice, one would perhaps expect that the vector v ∈ Rn for which ∥A1−x B x v∥ = ∥A1−x B x ∥ has a nontrivial inner product also with other (smaller) eigenvectors of B which then implies an even stronger concentration for the leading eigenvectors. Following the proof of Theorem 2, we derive a tighter bound by relaxing the reliance on the spectral gap 1 − µ22 . Resuming from ∥Bv∥2 ≥ 1 − ε/x and expanding v in the eigenbasis {bm } of B (where µ1 = 1), we obtain: n X ε (1 − µ2m )|⟨v, bm ⟩|2 ≤ . x m=2
14
A. ARBEL, S. STEINERBERGER, AND R. TALMON
We normalize this inequality by the tail mass 1−|⟨v, b1 ⟩|2 = the second moment of the tail spectrum as Pn 2 2 m=2 µm |⟨v, bm ⟩| , ρb ≜ P n 2 m=2 |⟨v, bm ⟩|
Pn
2 m=2 |⟨v, bm ⟩| . Defining
the inequality simplifies to (1 − |⟨v, b1 ⟩|2 )(1 − ρb ) ≤ xε . Rearranging terms yields the improved bound: ε |⟨v, b1 ⟩|2 ≥ 1 − . x(1 − ρb ) This tightens the bound since ρb ≤ µ22 , with ρb becoming smaller if the error aligns with high-frequency modes (small µm ). By symmetry, an analogous bound holds for the left singular vector using A. 4. Application to Multi-Manifold Learning. Multimodal manifold learning deals with the fundamental challenge of representing data from diverse sources and modalities. This task is crucial for data analysis, as it helps describe relationships between different data modalities, a core challenge, and a shared goal across many domains and applications. 4.1. Setting. Consider three hidden manifolds M1 , M2 , and M3 , which are observed through two observation functions g : M1 × M2 × M3 → S1 h : M1 × M2 × M3 → S2 , where S1 and S2 are subsets of (possibly different) Euclidean spaces. We can think of the triplet (M1 , M2 , M3 ) as the underlying global structure and of the functions g and h as two different ways of extracting information (e.g., these functions may represent samples captured by two different sensors). Following [15, 24], we assume that g is a smooth isometric embedding of M1 × M3 into S1 , ignoring M2 , and h is a smooth isometric embedding of M2 × M3 into S2 , ignoring M1 . This assumption implies that M3 represents the common component of the observed data (which is often the desired piece of information), while M1 and M2 represent observationspecific perspectives (often associated with interferences). The problem at hand is to obtain a representation of the common component given observations through g and h (see Figure 6). Consider n inaccessible samples {(xi , yi , zi )}ni=1 from some joint distribution supported on the product of the hidden manifolds M1 × M2 × M3 , which give rise to (1) (2) (1) n pairs of accessible data points {(si , si )}ni=1 such that si = g(xi , yi , zi ) and (2) si = h(xi , yi , zi ). The two sets of samples are viewed as a discretization of the respective underlying manifolds. We compute two kernels, one for each set, consisting of pairwise local affinities between the observed samples. Typically, the positive Gaussian kernel is used to measure local affinities, i.e., (1) (1) (4.1) Ai,j = exp −∥si − sj ∥22 /ε(1) (2) (2) (4.2) Bi,j = exp −∥si − sj ∥22 /ε(2) for i, j = 1, . . . , n, where ε(1) , ε(2) > 0 are two scale parameters. After applying the conventional normalizations to the kernels (see [4]), we obtain two symmetric positive n×n definite matrices A, B ∈ Sym+ , where ∥A∥ = ∥B∥ = 1. n ⊂R
15
COMPLEX INTERPOLATION OF MATRICES
M3 M1
S1
M2
S2
Fig. 6: A sketch of the multimanifold learning setup: three hidden manifolds and two observables, where at each observation only one manifold (M3 ) is common.
4.2. Algorithm. Here, following [14], we take an approach that relies on kernel interpolation. We apply the eigenvalue decomposition to the normalized kernels A and B to obtain their eigenvalues, denoted as λk and µm , respectively. To analyze the relationship between the two sets of measurements, we interpolate between A and B according to the continuous map γ : [0, 1] → Rn×n given by (4.3)
γ(x) = A1−x B x .
For the interpolation, we use a regular grid with M + 1 points, xi = Mi for 0 ≤ i ≤ M , yielding M + 1 symmetric positive-definite matrices γ(xi ). To each matrix γ(xi ), we apply the singular value decomposition and obtain the top K singular values {σxki }K k=1 . We then generate a diagram depicting the variation of the top K singular values across the interpolated points by plotting them along the interpolation axis x, where for each xi we have K singular values (see subsection 4.3). For the special cases xi = 0 and xi = 1, the singular values coincide with the eigenvalues: σ0k = λk and σ1k = µk for k = 1, . . . , K. We summarize this in Algorithm 4.1. Algorithm 4.1 Singular Values Flow Diagram Generation (1)
(2)
Input: Two sets of aligned measurements, {si }ni=1 ⊂ S1 , {si }ni=1 ⊂ S2 Output: Singular values diagram Parameters: • M – The number of points on the interpolation axis • K – The number of singular values at each interpolation point Build two kernels for the two input sets of measurements: a: Construct two SPD affinity kernels A and B using (4.1) and (4.2). b: Normalize the kernels. Consider a regular grid of M + 1 points {xi = Mi }M i=0 in [0, 1]. For each xi : a: Compute the matrix A1−xi B xi k K b: Apply SVD and obtain the largest K singular values {σx } i k=1 c: Scatter plot the logarithm of the obtained singular values as a function of xi .
16
A. ARBEL, S. STEINERBERGER, AND R. TALMON
4.3. An example. We demonstrate our method on a pair of cylindrical surfaces, denoted by C1 and C2 and illustrated in Figure 2. We sample n = 1000 tuples n {(xi , yi , zi )}i=1 uniformly from a product of three hidden 1D manifolds S 1 × S 1 × [0, 2π], where xi ∈ S 1 , yi ∈ S 1 , zi ∈ [0, 2π], and S 1 denotes the 1D sphere. These samples are then mapped onto two cylindrical surfaces using the functions g, h P cos(xi ) P2 cos(yi ) 1 1 1 (2) (1) P2 sin(yi ) , P1 sin(xi ) , si = h(xi , yi , zi ) = si = g(xi , yi , zi ) = 2π 2π L1 · zi L2 · zi where the parameters are set to L1 = 2, P1 = 1.25, L2 = 2, P2 = 3. The mapped samples are viewed as observations on 2D cylinders embedded in R3 , where the common variable zi ∈ [0, 2π] represents the height coordinate, and xi ∈ S 1 and yi ∈ S 1 are distinct and represent azimuthal angles. A 2D cylindrical surface with Neumann boundary conditions has a spectrum that is analytically tractable. Specifically, the eigenvalues of C1 and C2 are given respectively by the following closed-form expressions: 2 2 2 2 πkz πkz 2π kx 2π ky (ky ,kz ) (kx ,kz ) λ1 = , λ2 = , + + L1 P1 2 L2 P2 2 where kx , ky , kz = 0, 1, 2, . . . are indices. We see in these expressions that the “degree of commonality” is determined by the ratio between the shared height, L1 or L2 , and the distinct perimeter, P1 or P2 , respectively: a small ratio pushes common eigenvalues deeper in the spectrum, whereas a large ratio does so for distinct ones. We apply Algorithm 4.1 with M = 51 to the two sets of samples on the two cylinders. In Figure 3, we plot the resulting singular values diagram (gray). On the boundaries, at x = 0 and x = 1, using the following relation [8, equation (7)]: 2 ε (4.4) λ̃ = exp − λ , 4 we overlay three common analytical eigenvalues of the two cylinders (setting kx = ky = 0) on the empirical eigenvalues of C1 and C2 and mark them by blue squares. Dashed lines show the log-linear interpolation between corresponding analytical eigenvalues (with the same kz index). We see that the resulting empirical singular values at any interpolated point x ∈ (0, 1) nearly coincide with the log-linear interpolation between the analytical spectrum. In Figure 4, we show the same SVFD as in Figure 3, but now highlight the empirical singular values corresponding to two non-common spectral components. Specifically in Figure 4, we examine the fourth-largest eigenvector of C1 , which corresponds to azimuthal oscillations. The SVFD presents common and noncommon spectral components differently: curves associated with eigenpairs that share the common height variable are approximately straight curves that closely follow the dashed interpolations, while eigenpairs dominated by the distinct azimuthal variables give rise to curved trajectories. 4.4. Concluding remarks. The proposed interpolation γ(x) = A1−x B x in (4.3) enables a separation between common and non-common spectral components. It is efficient and mathematically tractable, and we show both theoretically and empirically that it conveys not only dichotomous information, but also the degree of commonality of the components. However, the considered interpolation is not unique and raises several questions. For example, does the order of the matrices in the product affect the result? For instance, symmetric interpolations such as B x/2 A1−x B x/2 ,
COMPLEX INTERPOLATION OF MATRICES
17
A1−x B 2x A1−x , or B x A2(1−x) B x can be considered. In [14], another symmetric interpolation scheme based on the geodesic between two symmetric positive-definite matrices under the affine-invariant metric [20, 3] was presented. Similarly, one could consider geodesics, or other trajectories on the symmetric positive-definite manifold, induced by different Riemannian metrics. We have established a theoretical framework and provided tools for such an approach to multimodal manifold learning via kernel interpolation; we believe that the theorems presented here can serve as a blueprint for what is possible more generally. Acknowledgments. This work was funded by the European Union’s Horizon 2020 research and innovation programme under Grant 802735-ERC-DIFFOP. REFERENCES [1] S. Akaho, A kernel method for canonical correlation analysis, arXiv preprint cs/0609071, (2006). [2] F. R. Bach and M. I. Jordan, Kernel independent component analysis, Journal of Machine Learning Research, 3 (2002), pp. 1–48. [3] R. Bhatia, Positive definite matrices, Princeton university press, 2009. [4] R. R. Coifman and S. Lafon, Diffusion maps, Applied and computational harmonic analysis, 21 (2006), pp. 5–30. [5] R. R. Coifman, N. F. Marshall, and S. Steinerberger, A common variable minimax theorem for graphs, Foundations of Computational Mathematics, 23 (2023), pp. 493–517. [6] H. O. Cordes, Spectral theory of linear differential operators and comparison algebras, vol. 76, Cambridge University Press, 1987. [7] F. Dietrich, O. Yair, R. Mulayoff, R. Talmon, and I. G. Kevrekidis, Spectral discovery of jointly smooth features for multimodal data, SIAM Journal on Mathematics of Data Science, 4 (2022), pp. 410–430. [8] C. Dsilva, R. Talmon, R. Coifman, and I. Kevrekidis, Parsimonious representation of nonlinear dynamical systems through manifold learning: A chemotaxis case study, Applied and Computational Harmonic Analysis, 44 (2018), pp. 759–773. [9] G. Froyland, Dynamic isoperimetry and the geometry of lagrangian coherent structures, Nonlinearity, 28 (2015), pp. 3587–3622. [10] G. Froyland and E. Kwok, A dynamic laplacian for identifying lagrangian coherent structures on weighted riemannian manifolds, Journal of Nonlinear Science, 30 (2020), pp. 1889– 1971. [11] E. Heinz, Beiträge zur störungstheorie der spektralzerleung, Mathematische Annalen, 123 (1951), pp. 415–438. [12] H. Hotelling, Relations between two sets of variates, Biometrika, 28 (1936), pp. 321–377. [13] T. Kato, Notes on some inequalities for linear operators, Mathematische Annalen, 125 (1952), pp. 208–212. [14] O. Katz, R. R. Lederman, and R. Talmon, Multimodal manifold learning using kernel interpolation along geodesic paths, Information Fusion, 114 (2025), p. 102637. [15] R. R. Lederman and R. Talmon, Learning the geometry of common latent variables using alternating-diffusion, Applied and Computational Harmonic Analysis, 44 (2018), pp. 509 – 536. [16] K. Löwner, Über monotone matrixfunktionen, Mathematische Zeitschrift, 38 (1934), pp. 177– 216. [17] A. McIntosh, Heinz inequalities and perturbation of spectral families, Macquarie Mathematics Reports, (1979). [18] T. Michaeli, W. Wang, and K. Livescu, Nonparametric canonical correlation analysis, in International conference on machine learning, PMLR, 2016, pp. 1967–1976. [19] B. Øksendal, The ito formula and the martingale representation theorem, in Stochastic Differential Equations: An Introduction with Applications, Springer, 2003, pp. 43–63. [20] X. Pennec, P. Fillard, and N. Ayache, A riemannian framework for tensor computing, International Journal of computer vision, 66 (2006), pp. 41–66. [21] T. Shnitzer, M. Ben-Chen, L. Guibas, R. Talmon, and H.-T. Wu, Recovering hidden components in multimodal data with composite diffusion operators, SIAM Journal on Mathematics of Data Science, 1 (2019), pp. 588–616.
18
A. ARBEL, S. STEINERBERGER, AND R. TALMON
[22] T. Shnitzer, H.-T. Wu, and R. Talmon, Spatiotemporal analysis using riemannian composition of diffusion operators, Applied and Computational Harmonic Analysis, 68 (2024), p. 101583. [23] S. Steinerberger, Refined heinz-kato-löwner inequalities, J. Spectr. Theory, 9 (2019), pp. 1– 20. [24] R. Talmon and H.-T. Wu, Latent common manifold learning with alternating diffusion: Analysis and applications, Applied and Computational Harmonic Analysis, 47 (2019), pp. 848–892. [25] D. V. Widder, Functions harmonic in a strip, Proceedings of the American Mathematical Society, 12 (1961), pp. 67–72.