KL Divergence Between Gaussians: A Step-by-Step Derivation for the Variational Autoencoder Objective arXiv:2604.11744v1 [cs.LG] 13 Apr 2026
Andrés Muñoz Rodrigo Ramele Instituto Tecnológico de Buenos Aires
Abstract Kullback-Leibler (KL) divergence is a fundamental concept in information theory that quantifies the discrepancy between two probability distributions. In the context of Variational Autoencoders (VAEs), it serves as a central regularization term, imposing structure on the latent space and thereby enabling the model to exhibit generative capabilities. In this work, we present a detailed derivation of the closed-form expression for the KL divergence between Gaussian distributions—a case of particular importance in practical VAE implementations. Starting from the general definition for continuous random variables, we derive the expression for the univariate case and extend it to the multivariate setting under the assumption of diagonal covariance. Finally, we discuss the interpretation of each term in the resulting expression and its impact on the training dynamics of the model.
Keywords: Kullback–Leibler divergence, multivariate Gaussian, closed-form solution, Variational Autoencoders (VAE).
1
Introduction
Probabilistic modeling has become a cornerstone of modern machine learning, particularly in the development of generative models capable of capturing complex data distributions. Among these, Variational Autoencoders (VAEs), introduced by Kingma and Welling (2013) have emerged as a powerful framework that integrates principles from deep learning and variational inference to learn meaningful latent representations.
1
A key component of the VAE formulation is the incorporation of a structured latent space, typically enforced by assuming a prior distribution over the latent variables. In practice, this prior is commonly chosen as a standard normal distribution. To ensure that the learned representations adhere to this prior, VAEs incorporate a regularization term based on the Kullback–Leibler (KL) divergence, which measures the discrepancy between the approximate posterior distribution produced by the encoder and the prior distribution. Despite its widespread use, the KL divergence is often introduced in an abstract manner, and its closed-form expression for Gaussian distributions is frequently presented without a detailed derivation. However, understanding this derivation is essential for gaining deeper insight into the behavior of VAEs, particularly regarding how the latent space is shaped during training. In this work, we provide a step-by-step derivation of the KL divergence between Gaussian distributions, starting from its general definition for continuous random variables. We first consider the univariate case and then extend the result to the multivariate setting under the assumption of diagonal covariance. Finally, we discuss the interpretation of the resulting expression and its implications for the training dynamics and regularization properties of VAEs.
2
Kullback–Leibler Divergence
The Kullback-Leibler (KL) divergence, originally introduced in Kullback and Leibler (1951), is a fundamental measure in information theory that quantifies the discrepancy between two probability distributions. Let P and Q be two probability distributions defined over the same space X , with corresponding densities p and q. The KL divergence of P with respect to Q is defined as:
Z DKL (P ∥Q) =
p(x) log X
p(x) q(x)
dx
provided that p(x) = 0 whenever q(x) = 0, ensuring that the expression is well-defined. Equivalently, the KL divergence can be expressed as an expectation with respect to P : DKL (P ∥Q) = EP
p(x) log q(x)
The KL divergence satisfies the following properties: • DKL (P ∥Q) ≥ 0
(non-negativity)
• DKL (P ∥Q) = 0 if and only if P = Q (almost everywhere) 2
(1)
• DKL (P ∥Q) ̸= DKL (Q∥P ) in general (asymmetry) Despite being commonly referred to as a “distance”, the KL divergence is not a true metric, as it does not satisfy symmetry nor the triangle inequality. Instead, it can be interpreted as the expected information loss incurred when using Q to approximate P .
3
Derivation for Gaussian Distributions
Let P and Q be two probability distributions such that P ∼ N (µ1 , Σ1 ),
Q ∼ N (µ2 , Σ2 ),
with µ1 , µ2 ∈ Rk and Σ1 , Σ2 ∈ Rk×k and let p and q denote their corresponding probability density functions 1 1 ⊤ −1 p(x) = p exp − (x − µ1 ) Σ1 (x − µ1 ) 2 (2π)k |Σ1 | (2) 1 1 q(x) = p exp − (x − µ2 )⊤ Σ−1 2 (x − µ2 ) k 2 (2π) |Σ2 | By a slight abuse of notation, we will use p and q interchangeably to refer to the corresponding distributions when no ambiguity arises. In the following, we assume that X = Rk . For notational simplicity, the domain of integration will be omitted. Starting from Definition (1) and applying basic properties of the logarithm, the Kullback Leibler divergence can be rewritten as: Z DKL (P ∥Q) = p(x) log p(x) − log q(x) dx (3) The logarithms of each density (2) are: k 1 1 log p(x) = − log(2π) − log |Σ1 | − (x − µ1 )⊤ Σ−1 1 (x − µ1 ) 2 2 2 k 1 1 log q(x) = − log(2π) − log |Σ2 | − (x − µ2 )⊤ Σ−1 2 (x − µ2 ) 2 2 2 Substituting into (3) the KL divergence (noting that the constant terms cancel), we obtain:
3
Z DKL (P ∥Q) =
"
# 1 1 |Σ2 | 1 ⊤ −1 ⊤ −1 p(x) log + (x − µ2 ) Σ2 (x − µ2 ) − (x − µ1 ) Σ1 (x − µ1 ) dx 2 |Σ1 | 2 2
Using the linearity of the integral, we decompose the expression into three terms, which we denote by H1 , H2 , and H3 . 1 |Σ2 | p(x) log dx 2 |Σ1 | Z 1 + p(x) (x − µ2 )⊤ Σ−1 2 (x − µ2 ) dx 2 Z 1 − p(x) (x − µ1 )⊤ Σ−1 1 (x − µ1 ) dx 2 Z
DKL (P ∥Q) =
= H1 + H2 − H3 We now compute each term separately. Term H1 . Since the logarithmic term is constant with respect to x, and recalling that Z p(x) dx = 1, we obtain: H1 =
|Σ2 | 1 log . 2 |Σ1 |
Term H2 . We consider the term H2 = EP (x − µ2 )⊤ Σ−1 (x − µ ) . 2 2 Rewriting the difference as x − µ2 = (x − µ1 ) + (µ1 − µ2 ), we expand the quadratic form:
4
(4)
⊤ −1 (x − µ2 )⊤ Σ−1 Σ2 (x − µ1 ) + (µ1 − µ2 ) 2 (x − µ2 ) = (x − µ1 ) + (µ1 − µ2 ) = (x − µ1 )⊤ Σ−1 2 (x − µ1 ) + 2(µ1 − µ2 )⊤ Σ−1 2 (x − µ1 ) + (µ1 − µ2 )⊤ Σ−1 2 (µ1 − µ2 ) Taking expectation with respect to P , we analyze each term separately. First term. Using the identity E[x⊤ Ax] = tr(A E[xx⊤ ]), applied to the centered variable (x − µ1 ), and recalling that the covariance matrix is defined as Σ1 = E (x − µ1 )(x − µ1 )⊤ , we obtain: −1 ⊤ EP (x − µ1 )⊤ Σ−1 2 (x − µ1 ) = tr Σ2 EP [(x − µ1 )(x − µ1 ) ] = tr Σ−1 2 Σ1 Second term. Since EP [x] = µ1 and (µ1 − µ2 )⊤ Σ−1 2 is constant with respect to x, we obtain: ⊤ −1 EP (µ1 − µ2 )⊤ Σ−1 2 (x − µ1 ) = (µ1 − µ2 ) Σ2 EP [x − µ1 ] = 0
Third term. Since this term is constant with respect to x, we have: ⊤ −1 EP (µ1 − µ2 )⊤ Σ−1 2 (µ1 − µ2 ) = (µ1 − µ2 ) Σ2 (µ1 − µ2 ) Combining the three terms, we obtain: H2 = tr Σ−1 Σ + (µ1 − µ2 )⊤ Σ−1 1 2 2 (µ1 − µ2 ) Term H3 . We consider the term H3 = EP
1 ⊤ −1 (x − µ1 ) Σ1 (x − µ1 ) 2
Using the identity E[x⊤ Ax] = tr(A E[xx⊤ ]), and recalling that the covariance matrix is defined as Σ1 = E (x − µ1 )(x − µ1 )⊤ , 5
we obtain: 1 H3 = tr Σ−1 1 Σ1 2 1 = tr(Ik ) 2 1 = k 2 Final expression. Substituting H1 , H2 , and H3 into the expression (4) for the KL divergence, we obtain: DKL (P ∥Q) = H1 + H2 − H3 =
1 1 |Σ2 | 1 ⊤ −1 log + tr(Σ−1 Σ ) + (µ − µ ) Σ (µ − µ ) − k 1 1 2 1 2 2 2 2 |Σ1 | 2 2
Rearranging terms we obtein: 1 DKL (P ∥Q) = 2
|Σ2 | −1 ⊤ −1 tr(Σ2 Σ1 ) + (µ1 − µ2 ) Σ2 (µ1 − µ2 ) − k + log |Σ1 |
Proposition 1 Let P ∼ N (µ1 , Σ1 ) and Q ∼ N (µ2 , Σ2 ) be two multivariate Gaussian distributions in Rk , where µ1 , µ2 ∈ Rk are the mean vectors, and Σ1 , Σ2 ∈ Rk×k are the corresponding covariance matrices, assumed to be symmetric and positive definite. Then, the Kullback–Leibler divergence between P and Q is given by: |Σ2 | 1 −1 ⊤ −1 tr(Σ2 Σ1 ) + (µ1 − µ2 ) Σ2 (µ1 − µ2 ) − k + log . DKL (P ∥Q) = 2 |Σ1 | Proof. See the derivation above. Special case (VAE). This particular case is of special interest in the context of Variational Autoencoders (VAEs), where the latent variables are typically modeled using a Gaussian approximate posterior and a standard normal prior. Under these assumptions, the KL divergence admits a closed-form expression, making it computationally efficient to evaluate and differentiate during training. This property is crucial for enabling scalable optimization via gradient-based methods and for enforcing a well-structured latent space.
6
In this setting, the approximate posterior is given by q(z|x) ∼ N (µ(x), Σ(x)), while the prior is defined as p(z) ∼ N (0, Ik ). Therefore, the KL divergence simplifies to: DKL (q(z|x)∥p(z)) =
1 tr(Σ(x)) + µ(x)⊤ µ(x) − k − log |Σ(x)| 2
This expression is commonly used as the regularization term in the VAE objective function.
4
Algebraic Identities
In this appendix, we summarize several algebraic identities used throughout the derivation. Trace properties. Let A, B be matrices of compatible dimensions. Then: • tr(A) = tr(A⊤ ) • tr(AB) = tr(BA) • tr(ABC) = tr(BCA) = tr(CAB) • tr(aA + bB) = a tr(A) + b tr(B) for scalars a, b Quadratic forms. Let x ∈ Rk and A ∈ Rk×k . Then: • x⊤ Ax = tr(x⊤ Ax) = tr(Axx⊤ ) Expectation and trace. For a random vector x ∈ Rk : • E[tr(X)] = tr(E[X]) • E[x⊤ Ax] = tr(A E[xx⊤ ]) Covariance matrix. For a random vector x with mean µ: Σ = E (x − µ)(x − µ)⊤ 7
Gaussian expectations. If x ∼ N (µ, Σ), then: • E[x] = µ • E[(x − µ)(x − µ)⊤ ] = Σ • E[xx⊤ ] = Σ + µµ⊤
References Kingma, D. P. and Welling, M. (2013). Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114. Kullback, S. and Leibler, R. A. (1951). On information and sufficiency. The Annals of Mathematical Statistics, 22(1):79–86.
8