IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. AA, NO. BB, MMMM YYYY
1
Information Plane Analysis of Binary Neural Networks
arXiv:2605.03636v1 [cs.LG] 5 May 2026
Maximilian Nothnagel and Bernhard C. Geiger, Senior Member, IEEE
Abstract—Information plane (IP) analysis has been suggested to study the training dynamics of deep neural networks through mutual information (MI) between inputs, representations, and targets. However, its statistical validity is often compromised by the difficulty of estimating MI from samples of high-dimensional, deterministic representations. In this work, we perform IP analyses on binary neural networks (BNNs) where activations are discrete and MI is finite. We characterise the finite-sample behaviour of the plug-in entropy estimator and identify regimes for sample size N and representation dimensionality D under which MI estimates are reliable. Outside these regimes, we show that empirical MI estimates saturate to log2 N , rendering IP trajectories uninformative. Restricting attention to the reliable regime, we train 375 BNNs to investigate the existence of latestage compression phases and the relationship between compressed representations and generalisation performance. Our results show that while late-stage compression is frequently observed, compressed latent representations do not consistently correlate with improved generalization performance. Instead, the relationship between compression and generalisation is highly dependent on task, architecture, and regularisation. Index Terms—information plane analysis, deep learning, binary neural networks, information bottleneck theory, compression and generalization
I. I NTRODUCTION
B
ASED on the information bottleneck (IB) theory, the authors of [1] introduce the concept of information plane (IP) analysis of deep neural networks (DNNs). Given the latent representations Tℓ of a neuron layer ℓ, the mutual information (MI) I(X; Tℓ ) between input and representation and the MI I(Tℓ ; Y ) between target and representation are plotted for each epoch on respective axes. The authors claim that IPs can visually explain the inner training dynamics of DNNs, by analysing compression and fitting phases—i.e., the decrease of I(X; Tℓ ) and increase of I(Tℓ ; Y ), respectively [1], [2]. The concept of IP analysis was met with a broad number of subsequent analyses by several authors; however, the evidence regarding the importance or even existence of the compression phase has been inconclusive and topic of discussion [2], [3]. While initially the existence of a compression phase has been linked to the usage of certain activation functions (e.g., [4]), more recent work has argued that the qualitative picture in the IP is heavily influenced by the properties of the method of M. Nothnagel ([email protected]) is with Graz University of Technology, Graz, Austria and was with Know Center Research GmbH, Graz, Austria. B. C. Geiger ([email protected]) is with the Signal Processing and Speech Communication Laboratory, Graz University of Technology, Graz, Austria, with Know Center Research GmbH, Graz, Austria, and with the Graz Center for Machine Learning (GraML), Graz, Austria.
estimating MI. Specifically, for deterministic DNNs, where MI is infinite for a continuous-valued input [5, Th. 1], the IP is now claimed to show a geometric, rather than informationtheoretic picture due to the inherent properties of many estimators [2], including the popular binning estimator [6]. Furthermore, even if true MI was bounded, the performance of many estimators depends on the dimensionality of the latent representation Tℓ and the number N of samples available for estimation. Taken together, these considerations render many of previous IP analyses unreliable, even for questions as simple as regarding the existence of a compression phase. In this work, we present IP analyses free from these shortcomings. First, we focus on binary neural networks (BNNs) [7], [8], in which the neuron activations are inherently discrete values and where, hence, MI terms are finite. For these discrete activations, the plug-in estimate [9] is an appropriate choice, but still affected by the dimensionality of Tℓ and the availability of samples. Presenting previous theoretical results together with experiments on synthetic data with known entropy, we show in Section III-B that entropy estimates outside a suggested data regime strongly diverge from the truth, as estimates start saturating for increased dimensionality. Having thus established sufficient conditions for a reliable IP analysis, our main contribution is an in-depth investigation of the interplay between information-theoretic compression and generalisation performance on well-chosen toy examples involving BNNs. We are interested in the following questions: 1) Do BNNs exhibit a compression phase in the IP in the sense that the MI I(X; Tℓ ) decreases during a late period in training (Section V)? 2) Do compressed representations, i.e., latent Tℓ with small I(X; Tℓ ), correlate with improved generalisation performance (Section VI)? While the first question is connected to the ongoing debate about whether training DNNs proceeds in two phases, the second question contributes to the practically relevant hypothesis that information-theoretic compression is a possible approach to prevent overfitting. To answer above questions, we conduct 125 different experiments and three runs per experiment, totalling to 375 trained BNNs, distributed among four datasets. We estimate the MI quantities of the binary activations of model layers for which estimation can be done reliably. We conclude that compression is not necessarily indicating better generalisation. However, the movement and behaviour in the IP can provide insight into the performance of the model. We discuss these and other implications of our work as well as its limitations in Section VII.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. AA, NO. BB, MMMM YYYY
N
II. R ELATED W ORK Several authors with varying approaches and focus have conducted IP analyses on binarised or quantised neural networks. The authors of [10] focused on stochastic BNNs with four hidden layers, estimating the MI using Monte Carlo samples. Experiments on the synthetic SZT dataset provided in [1] and MNIST reveal no compression phase. In [11], fully connected binarised networks were trained on SZT and MNIST, applying the plug-in estimate and binning the real valued input X and softmax output Ŷ into 30 bins. The authors observed compression for the binary activations for MNIST, but no ‘explicit’ compression phase on SZT. Quantisationaware training (QAT) [12] was applied in [3] on different models. On SZT, ReLU and tanh as activation function with 4, 8-, and 32-bit quantisation is applied, exhibiting compression in hidden layers only for tanh. On MNIST, ReLU activation with 8-bit quantisation showed small compression. Thus, the question regarding information-theoretic compression in BNNs remains inconclusive, similar to general DNNs. In addition, previous studies focus on the SZT and MNIST dataset, with a lack of experiments on more complex datasets and more varying architectures. Furthermore, previous work often analysed layer widths that lie outside a suggest samples-to-dimensionality regime (Section III-B), resulting in unreliable entropy and MI estimation on either axis of the IP plot. III. M ETHODOLOGY A. Binary Neural Networks In this work, we apply the sign function ( 1 if x > 0, sign x = 0 else
2
(1)
to the activations of hidden fully connected (FC) layers with batch normalisation in between. Input data and output of nonFC layers are kept as full-precision real values. To circumvent the non-differentiability of the sign function, a straight-through estimator (STE) [13], [14] is applied during the backward pass. Let t = sign z be the quantisation of the previous (pre-)activation z and assume that the gradient ∂C ∂t has been obtained. Then, the saturation-aware STE [8] is defined as: ( 1 if − 1 ≤ z ≤ 1, ∂C ∂C = · 1|z|≤1 , with 1|z|≤1 = ∂z ∂t 0 else (2) The indicator function 1|z|≤1 serves as a (hard) threshold regarding the magnitude of the non-quantised activation, cancelling the gradient if |z| is too large [8]. B. Entropy Estimation We utilise the discrete binary activations of our models and apply the empirical or plug-in estimate [9] for entropy. Let X be a discrete random variable (RV) with an (unknown) probability mass function pX on an alphabet X of cardinality k. Given a set of N independent and identically distributed
(iid) observations D = {xi }i=1 , where each xi is a realisation of X, the empirical distribution is computed as nx p̂X (x) = , (3a) N where nx is the number of occurrences of x in D. ‘Plugging in’ p̂X into the definition of the entropy H(X) of X, and defining 0 log 0 = 0 by continuous extension, yields the plugin estimate of entropy: X Ĥ(X) = − p̂X (x) log2 p̂X (x) (3b) x∈X
The estimate is shown to be consistent for N → ∞ [9]. However, in the more practical ‘large N ’ regime, assuming N to be a function of k, the estimate is suboptimal if the alphabet size is increased but the number of samples is not increased accordingly. More specifically, the estimate is suboptimal in k2 the samples-to-dimensionality regime of N = o log2 k [15]. 2 Considering the datasets used in previous and our work, the N ≈ 800 samples in the validation set of the synthetic SZT data would suffice for alphabet sizes of up to k ≈ 27 ; for N = 10,000, that is MNIST and similar, the alphabet size can be as large as k ≈ 210 . Assuming binary activations thus means that datasets of this size can reliable estimate entropy only in layers with up to seven (SZT) or ten (MNIST or similar) neurons. To illustrate this, Fig. 1 shows an exemplary evaluation of the plug-in estimate on synthetic data. We generate 20 experiments with N = 1,000 samples of Ddimensional Bernoulli vectors X = (X1 , . . . , XD ), with D ∈ {1, . . . , 20}, where each vector dimension is drawn independently from a Bernoulli distribution with success probability p ∈ {0.5, 0.7, 0.9}. The true entropy per experiment is computed as: H(X) =
D X
H(Xd ) = D · h2 (p)
(4)
d=1
= −D · (p log2 (p) + q log2 (q)) ,
q =1−p
It is trivial to see that H grows linearly with D. In addition, Ĥ starts underestimating at D ≥ 7 for all p, adhering to the required regime proposition mentioned before. Beyond this limit, the estimate starts saturating to log2 N , as the alphabet size k = 2D grows so large that each of the N samples assumes a different value of the alphabet. C. Information Plane Analyses and Estimation of Mutual Information Treating X and Y as RVs, the latent representation Tℓ obtained by the binary activations of the ℓ-th hidden layer is an RV with an alphabet Tℓ of cardinality kℓ = 2dℓ , where dℓ is the number of neurons in this layer. For IP analyses, estimates of the MI terms I(X; Tℓ ) and I(Tℓ ; Y ) are computed for each (or a relevant subset) of the training epochs. These estimates are computed from a validation set D = {(xi , yi )}N i=1 of size N . More specifically, for a given epoch and a given layer index ℓ, Dℓ = {(ti , yi )}N i=1 is the resulting set of realisations of the N joint RV (Tℓ , Y ), and Dℓ,t = {ti }i=1 , where ti ∈ {0, 1}dℓ .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. AA, NO. BB, MMMM YYYY
p = 0.5
p = 0.7
10 5 4
8 12 16 RV vector size D
20
8 Entropy Estimate H
15
0
p = 0.9
15 Entropy Estimate H
Entropy Estimate H
20
3
10 5 0
4
8 12 16 RV vector size D
20
6 4 2 0
4
8 12 16 RV vector size D
20
Fig. 1: Evaluation of the plug-in estimate Ĥ with N = 1,000 samples of D-dimensional Bernoulli vectors of varying success probability p. Data is averaged over 20 experiments. The dotted line represents the true entropy H. It can be seen that while the true entropy increases linearly with dimensionality D, the estimate Ĥ starts diverging at D ≈ 8 and eventually saturates at log2 N . Since in the BNNs in our study the latent representations Tℓ are deterministic functions of X, we obtain [4] I(X; Tℓ ) = H(Tℓ ) − H(Tℓ | X) = H(Tℓ ).
(5)
ˆ Hence the estimate I(X; Tℓ ) of I(X; Tℓ ) is obtained by setting ˆ I(X; Tℓ ) = Ĥ(Tℓ )
(6)
and by applying the plug-in estimator (3) to the dataset Dℓ,t . For the term I(Tℓ ; Y ) we recognise that X pTℓ ,Y (t, y) (7) I(Tℓ ; Y ) = pTℓ ,Y (t, y) log2 pTℓ (t) · pY (y) t∈Tℓ ,y∈Y
and hence obtain ˆ ℓ; Y ) = I(T
X
p̂Tℓ ,Y (t, y) log2
t∈Tℓ ,y∈Y
p̂Tℓ ,Y (t, y) , p̂Tℓ (t) · p̂Y (y)
(8)
where the empirical distributions p̂Tℓ , p̂Y and p̂Tℓ ,Y (t, y) are estimated from Dℓ using (3a). For example, the empirical joint distribution p̂Tℓ ,Y is given as: N
p̂Tℓ ,Y (t, y) =
1 X 1(t ,y )=(t,y) N i=1 i i
(9)
IV. E XPERIMENTAL S ETUPS We conduct IP analyses of BNNs on four datasets. For each dataset, different neural networks—varying in architecture and regularisation in the form of weight decay—are trained over 3,000 epochs. Each experiment is performed on three consecutive runs.1 In total, we analyse 375 trained models, of which select results are presented.2 The MIs of the latent representations of our models are estimated on the validation subset of the respective dataset. The output layer is not considered for MI estimation and IP analysis. 1 The behaviour in the IPs is quite similar over all runs. Thus, we only show the first run of experiments for all IP plots. 2 The code is available online on https://github.com/ InformationPlanesDecompositions/entropy-estimation.
A. Datasets We employ the synthetic SZT dataset introduced in [1], as well as MNIST, FashionMNIST, and CIFAR-10. The SZT dataset represents a binary classification problem from a 12dimensional binary input and contains 4,096 unique samples. We follow previous work [3], [11] and use 20 % of the samples as validation split D. MNIST and FashionMNIST contain 60,000 training and 10,000 test samples, each being a 28 × 28 px greyscale image representing a handwritten digit or clothing article, respectively, associated with one label from ten classes. CIFAR-10 consists of 60,000 samples of 32 × 32 px colour images assigned to one of ten classes representing objects. The dataset is split into 50,000 training and 10,000 test samples, the latter containing exactly 1,000 samples per class. For MNIST, FashionMNIST, and CIFAR-10, we use the designated test split as the dataset D from which informationtheoretic quantities are estimated. B. Model Architecture and Regularisation In each experiment, all FC layers of the neural networks are binary, apart from the output layer. The weights are kept at full precision. As activation function, the sign function in (1) is used, while applying the saturation-aware STE in (2) as backward approximation. For the experiments without regularisation, the Adam optimiser is used with a learning rate of 1 × 10−4 for SZT and 1 × 10−5 otherwise. The batch size is set to 64 for SZT and 256 otherwise. The models used vary in the number of hidden layers and neurons. Table I gives an overview of the applied model structures and on which datasets they are used. Between each hidden layer, batch normalisation is applied. For the hourglass and bottleneck variants, the variable neuron width is set to A ∈ {2, 4, 6, 8, 10}. LeNet5 models use the conventional form of CV(6)−MP−CV(16)−MP for the convolutional part. Each convolution (CV) uses a kernel size of 5, batch normalisation,
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. AA, NO. BB, MMMM YYYY
4
TABLE I: Overview of the model architectures used in our experiments and on which datasets they are applied. A and B refer to architecture variations (see text). Name SZT* Raj-like† Hourglass Bottleneck Small BNN LeNet5‡ variant LeNet5 small LeNet5
Architecture
Datasets
10 − 8 − 6 − 4 1024 − 20 − 20 − 20 − 10 1024 − 20 − 10 − A − 10 − 20 − 10 1024 − 20 − 10 − A − 10 50 − 10 − 10 LeNet5 − 120 − 84 − 10 LeNet5 − A − B − 10 LeNet5 − 50 − 50 − 10
SZT, MNIST MNIST, FashionMNIST MNIST, FashionMNIST MNIST, FashionMNIST MNIST, FashionMNIST MNIST, FashionMNIST, CIFAR-10 CIFAR-10 CIFAR-10
* Used in [11] in accordance with [1]. † Used in [11] without the last layer with width 10. ‡ The convolutional activations are not binary.
and rectified linear unit (ReLU) activation, followed by a 2 × 2 max pooling (MP) with stride 2. Except for CIFAR10, the first CV adds a padding of 2 to its input. The variant LeNet5 model varies the neuron widths of the FC layers with A ∈ {50, 70, 120} and B ∈ {20, 50}. As an experimental variant, regularisation is added in the form of the weight decay coefficient λ ∈ {0.1, 0.2, 0.5, 0.7, 1, 1.1, 1.2, 1.5, 1.7, 2}, resulting in a total number of ten applied coefficients. For these variants, the setups remain identical apart from using the AdamW optimiser. In the IP plots, the notation λ = 0 means there is no weight decay applied and the Adam optimiser is used. C. Metrics As performance metric, we compute the classification accuracy on the validation set as percentage. In order to quantify compression, we define the compression factor ϱ=
ˆ ¯ I(X; Tℓ )max − I(X; Tℓ )50 , ˆ I(X; Tℓ )max
(10)
ˆ where I(X; Tℓ )max is the maximum value over all epochs and ¯ ˆ I(X; Tℓ )50 is the mean of I(X; Tℓ ) over the last 50 epochs. This factor dismisses any intermediate, temporary compression and focuses on the end-of-training behaviour of the models. Lastly, in order to quantify the correlation between the end-ofˆ training I(X; Tℓ ) and performance, we compute Spearman’s rank correlation coefficient rs over the mean values of MI and accuracy over the last 50 epochs. V. L ATE -S TAGE C OMPRESSION IN BNN S Late-stage compression, as experimentally observed by [1] and others, is characterised by the MI I(X; Tℓ ) or its estimate ˆ I(X; Tℓ ) reducing during a second phase3 of training. The compression factor ϱ is positive if such late-stage compression occurs, thus acts as a proxy for this phenomenon. The compression factors for the weight decay experiments, obtained using (10), are shown in Fig. 2. The factors are 3 We are aware that this second phase of training, during which I(X; T ) ℓ is purported to reduce, can assume a large portion of the total number of epochs. We abuse terminology and use the term “late-stage” to refer to this phase.
computed per run and layer and arranged per experiment and, if necessary, per layer. For all datasets and experiments, we observe compression to a certain degree, with a large majority of the experiments exhibiting a compression factor of ϱ ≥ 25 %. Furthermore, regularisation in the form of weight decay can drive models to a stronger compression. This effect is not as clear for MNIST and FashionMNIST as it is for SZT and CIFAR-10. Later layers appear to compress less than earlier layers (as shown for the Small BNN on MNIST and FashionMNIST), and compression in the last hidden layer seems to be more pronounced if the preceding part of the network has larger expressive power (LeNet5 vs. Small BNN). To highlight some aspects of IP analysis, additional IPs for select experiments are shown in Fig. 3. The Raj-like model trained on MNIST in Fig. 3a exhibits strong and clear compressional movement within the first hundred epochs of ¯ training. The end-of-training value I(X; Tℓ )50 is only slightly larger than log2 (10), which indicates that the latent representations Tℓ have mostly collapsed to ten binary patterns, corresponding to the ten classes of MNIST. The small regularised BNN on FashionMNIST in Fig. 3b still shows compression with a positive ϱ, but the qualitative picture is different: The last layer (left curve) only partially or temporarily compresses and, towards the end of training, ˆ ˆ increases I(X; T3 ) towards the value I(X; T2 ) of its preceding layer (right curve) in the IP. This renders the third layer obsolete, as it becomes a bijective map of the previous activations4 , e.g., by merely permuting the preceding activation bits. While we have to look at the concrete activations to confirm this suspicion, it is interesting that this assumption can be deduced from the IP. Fig. 3c shows yet another qualitatively different compression phase for the LeNet5 model on CIFAR-10: Here, compression happens early during training, with the reduction ˆ ˆ ℓ ; Y ). of I(X; Tℓ ) happening faster than the increase of I(T Considering the complexity of the underlying dataset and the model’s mediocre performance, this might be caused ˆ ℓ ; Y ) only starting to increase once I(X; ˆ by I(T Tℓ ) stops decreasing. This stands in contrast to previous work [1], [3], [4], [11], among others, and might hint towards IP analysis 4 Since T is a function of T , by the data processing inequality we can 3 2 only have H(T3 ) = H(T2 ) if the map is bijective.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. AA, NO. BB, MMMM YYYY
SZT
MNIST
0.6
0.8 0.7
5
2.00
0.5
0.6 0.5
0.4
0.4
0.3
1.75
0.3
1.50
FashionMNIST
S Lamall ye BN r -1 N
S Lamall ye BN r -2 N
et5 Le N
La WD ye r -1
La WD ye r -2
0.2
Ra j-li ke
0.2
CIFAR-10
0.5
0.45
0.4
0.40
0.3
1.00
0.75
0.35
0.2
1.25
0.50
0.30
0.1
0.25
Sm
all
Le N
et5
et5 Le N
S Lamall ye BN r -1 N
S Lamall ye BN r -2 N
like Ra j-
Le N
et5
0.25
Fig. 2: Compression factor ϱ computed per experiment run and layer, grouped by dataset and experiment group. Each dot is coloured according to the weight decay coefficient λ used in its experiment. Layer indices are given as negative offset starting from the output layer, i.e. layer -1 represents the last hidden layer.
being able to explain mediocre performance on hard problems in general. All selected experiments in Fig. 3 have in common that ˆ the maximum value of I(X; Tℓ ) is achieved at the first epoch. This behaviour was also visible for the BNNs studied in [11]. In contrast, applying binning together with the plug-in estimator to deterministic neural networks with realˆ valued activations shows an initial increase of I(X; Tℓ ), i.e., ˆ I(X; Tℓ )max is assumed at a later epoch. This discrepancy between BNNs and real-valued neural networks is noteworthy and may be explained by the fact that random, small-weight initialisations result in random binary patterns in BNNs, but in low-magnitude activations in classical neural networks, which populate only a small number of the available bins and hence lead to small entropy estimates. VI. G ENERALISATION OF C OMPRESSED R EPRESENTATIONS It was hypothesised that the late-stage compression exhibited by neural networks and, as a consequence, information-
theoretically compressed latent representations, are an explanation for the excellent generalisation capabilities of deep learning. While intuitive, the scientific literature has shown mixed results regarding the connection between compression and generalisation performance [2, Table 1]. To contribute to this hypothesis, we analyse the relation between the MI ˆ I(X; Tℓ ) and the validation accuracy at the end of training. ˆ To this end, we aggregate I(X; Tℓ ) and accuracy of the last 50 training epochs to compute Spearman’s rank correlation coefficient rs over all random initialisations and weight decay coefficients for a given model architecture. Table II shows the resulting coefficients and p-values per experiment group and layer, where negative coefficients indicate that information-theoretically compressed representations lead to improved generalisation performance. As it can be seen, both positive and negative rank correlations are significant, with substantial effect sizes (|rs | > 0.5). This suggests that the effect of information-theoretic compression depends on the architecture and dataset, among other things. To shed light into these varying interplays, we plot the
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. AA, NO. BB, MMMM YYYY
2.75
3.0
1.6
2500
2.50
2.0
1.4
2.25
I(T ; Y)
I(T ; Y)
2.5 I(T ; Y)
6
2.00
2000 1500
1.2 1000
1.75 1.5
1.0
500
1.50 4
5
6 I(X; T )
7
(a) MNIST, Raj-like, λ = 0
5
6
7 I(X; T )
8
7
(b) FashionMNIST, small BNN, λ = 1
8 I(X; T )
9
Epoch0
(c) CIFAR-10, LeNet5, λ = 0
Fig. 3: Select IPs of trained models. All figures show the last (and second to last in (b)) hidden FC layer, all with dℓ = 10. The achieved mean validation accuracies are 96.8 %, 83.0 %, and 53.3 %, respectively. Compressional movement is observable in all figures, with (a) exhibiting prototypical compression and (b) making the redundancy of the last hidden layer visually ˆ observable. (c) shows a qualitatively different compression, as the reduction of I(X; Tℓ ) happens faster than the increase of ˆ I(Tℓ ; Y ). aggregated values over the last 50 epochs for select experiments in Fig. 4. The prototypical expected behaviour is seen in Fig. 4c for the small LeNet5 model trained on CIFAR10: The MI reduces with increasing weight decay parameter λ, while accuracy first increases and then decreases as λ becomes larger. This suggests a transition from overfitting through a well-trained model to underfitting due to strong regularisation. The result is a weakly negative, but insignificant rank correlation. On FashionMNIST, the small BNN shows a similar behaviour regarding accuracy, see Fig. 4b. Indeed, accuracy and MI initially increase, but start to drop once λ reaches a certain threshold. As a consequence, the rank correlation between these two quantities is positive and significant, indicating that compressed representations generalise worse than uncompressed latent representations. Finally, the Raj-like model trained on MNIST, shown in Fig. 4a, exhibits a significant negative correlation with monotonic tendency. Increasing the weight decay parameter both increases MI and decreases accuracy, suggesting a transition into underfitting. Furthermore, the models’ accuracies vary more towards end of training for large values of λ. In summary, it appears that whether compressed representations generalise better depends at least partly on how well the regularised architecture is capable of solving the dataset-dependent task. Taking Fig. 4c as the prototypical behaviour, one can see that an appropriate selection of weight decay parameters can yield both positive and negative rank correlation values. The fact that in some architectures, stronger ˆ regularisation leads to larger values of I(X; Tℓ ) is surprising and shall be the object of future study. VII. D ISCUSSION AND C ONCLUSIONS In this work, we studied MI in BNNs, with the purpose of determining whether compression occurs during training, and whether compressed representations generalise better. While our experiments show that some form of compression during
TABLE II: Spearman’s rank correlation coefficients over the ¯ mean I(X; Tℓ )50 and the mean validation accuracy of the last 50 epochs. Positive values indicate that informationtheoretic compression hurts classification performance. Each value is computed per layer over each experiment group, including weight decays. Layers are given as negative offset indices from the output layer. All layers have a width of ten neurons, except for SZT, consider Table I for more detail. Dataset
Experiment Group
Layer
rs
p-value
SZT SZT SZT SZT
WD WD WD WD
-4§ -3§ -2 -1
0.602 0.548 0.495 0.546
< 0.001 0.001 0.003 0.001
MNIST MNIST MNIST MNIST MNIST MNIST
Raj-like Small BNN Small BNN LeNet5 Bottleneck Hourglass
-1 -2 -1 -1 -1 -1
−0.761 −0.077 0.378 −0.573 0.682 0.782
< 0.001 0.672 0.030 < 0.001 0.005 0.001
FashionMNIST FashionMNIST FashionMNIST FashionMNIST FashionMNIST FashionMNIST
Raj-like Small BNN Small BNN LeNet5 Bottleneck Hourglass
-1 -2 -1 -1 -1 -1
−0.360 0.569 0.768 −0.286 0.857 0.604
0.040 0.001 < 0.001 0.106 < 0.001 0.017
CIFAR-10 CIFAR-10 CIFAR-10
LeNet5 Variant LeNet5 Small LeNet5
-1 -1 -1
−0.642 0.490 −0.262
< 0.001 0.015 0.141
§ Note that the layers with eight/ten neurons are outside the suggested
regime for N ≈ 800 for reliable entropy estimation.
training is a prevalent phenomenon (see Fig. 3), we conclude that compressed representations are, in general, not performing better than uncompressed representations (Table II). An intuitive explanation for this inconsistent picture is that a representation can be compressed not only by being cleaned of irrelevant information, but also by becoming uninformative for the task, as in underfitting models. While we believe this explanation to be valid in many settings, a closer look
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. AA, NO. BB, MMMM YYYY
0.84
Accuracy
Accuracy
Accuracy
0.82
0.85
0.80 0.78
0.80 4.0
4.5 5.0 I(X; T )
(a) MNIST, Raj-like
5.5
2.00
0.625
0.95 0.90
7
5.0
5.5 6.0 I(X; T )
6.5
(b) FashionMNIST, small BNN
1.75
0.600
1.50
0.575
1.25 1.00
0.550
0.75
0.525
0.50
0.500
0.25
5.0
5.5
6.0 6.5 I(X; T )
7.0
0.00
(c) CIFAR-10, small LeNet5
Fig. 4: Comparison of the MI of the penultimate layer w.r.t. the input data X and the validation accuracy for varying weight decay λ. Points are averaged values over the last 50 epochs, with minimum-maximum range shown within this time frame. Each figure highlights a different relation: In (a), negative correlation and a suggested transition to underfitting caused by an increased weight decay can be seen. For (b), accuracy and MI initially increase, but start to drop after a certain threshold λ. ˆ (c) exhibits prototypical behaviour, as I(X; Tℓ ) increases with λ, but accuracy initially increases and then decreases again as λ grows larger, hinting towards a transition from a well-trained model to underfitting.
at Fig. 4 shows that the interplay between compression and generalisation is more nuanced even in small BNNs. One obvious limitation of our study is that it is restricted to comparably small BNNs, and then only to their smallest layers. While this is an immediate consequence of the properties of the plug-in estimator and the availability of validation data (cf. Section III-B), we believe that slightly wider layers can be studied with more sophisticated estimators, such as the best upper bounds estimator of [16]. Even this estimator, however, does not yield reliable estimates for analysing the IP of contemporary networks using common datasets, suggesting that future research should focus on estimating informationtheoretic quantities from high-dimensional data. A second limitation of our work is the fact that we summarise any reduction ˆ of I(X; Tℓ ) during training via a single number ϱ, which is of course insufficient to capture the wide range of qualitative behaviour shown in IPs. Specifically, while originally compression was linked to a second phase during training [1], our ˆ compression factor ϱ is positive whenever I(X; Tℓ ) does not ˆ achieve its maximum at the end of training, even if I(X; Tℓ ) increases during the last epochs (e.g., Fig. 3b). Our theoretical and experimental discussion of MI estimator properties in Section III-B sheds new light on previous IP analyses. While previous work acknowledged that deterministic networks suffer from infinite I(X; Tℓ ) and hence mostly display geometric properties in the IP, our work suggests that even for BNNs, for which I(X; Tℓ ) is finite, estimation may be unreliable. For example, [11, Fig. 3] plots the IP for a BNN with architecture 1024 − 20 − 20 − 20. Even for the later layers, the alphabet size k = 220 ≈ 106 is too large to admit reliable estimation from only N = 60, 000 samples from the MNIST dataset (training and test set combined). Similarly [3] investigated IPs for neural networks with activations quantised to 8 bit. For a layer of width dℓ = 8, the corresponding alphabet size is k = 2dℓ ·8 ≈ 1019 , requiring N ≈ 1035 samples for reliable estimation. While small true
values of I(X; Tℓ ) result in a substantially smaller effective alphabet size, i) the true value of I(X; Tℓ ) is neither known a priori nor can it be reasonably bounded, and ii) the fact ˆ that I(X; Tℓ ) ≈ log N at least for early epochs and layers indicates that MI estimation saturated (see Fig. 1) and that the resulting estimates are not reliable. Therefore, while restricted to only small architectures and to the specific setting of BNNs, our work is among the first that presents reliable IP analyses and experimental evidence regarding the connection between information-theoretic compression and generalisation. ACKNOWLEDGMENTS This research was funded by the Austrian Science Fund (FWF) under grant 10.55776/PAT7753623. Know Center is a COMET competence center that is financed by the Austrian Federal Ministry of Innovation, Mobility and Infrastructure (BMIMI), the Austrian Federal Ministry of Economy, Energy and Tourism (BMWET), the Province of Styria, the Steirische Wirtschaftsförderungsgesellschaft m.b.H. (SFG), the Vienna business agency and the Standortagentur Tirol. The COMET programme is managed by the Austrian Research Promotion Agency FFG. R EFERENCES [1] R. Shwartz-Ziv and N. Tishby, “Opening the black box of deep neural networks via information,” arXiv:1703.00810 [cs.LG], Mar. 2017. [2] B. C. Geiger, “On Information Plane Analyses of Neural Network Classifiers—A Review,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 12, pp. 7039–7051, Dec. 2022. [3] S. S. Lorenzen, C. Igel, and M. Nielsen, “Information bottleneck: Exact analysis of (quantized) neural networks,” arXiv:2106.12912 [cs.LG], Jun. 2021. [4] A. M. Saxe, Y. Bansal, J. Dapello, M. Advani, A. Kolchinsky, B. D. Tracey, and D. D. Cox, “On the information bottleneck theory of deep learning,” Journal of Statistical Mechanics: Theory and Experiment, vol. 2019, no. 12, p. 124020, Dec. 2019.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. AA, NO. BB, MMMM YYYY
[5] R. A. Amjad and B. C. Geiger, “Learning representations for neural network-based classification using the information bottleneck principle,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 9, pp. 2225–2239, Sep. 2020, open-access: arXiv:1802.09766 [cs.LG]. [6] Z. Goldfeld, E. Van Den Berg, K. Greenewald, I. Melnyk, N. Nguyen, B. Kingsbury, and Y. Polyanskiy, “Estimating information flow in deep neural networks,” pp. 2299–2308, Jun. 2019. [7] M. Courbariaux, Y. Bengio, and J.-P. David, “BinaryConnect: Training Deep Neural Networks with binary weights during propagations,” in Proc. Advances in Neural Information Processing Systems, 2015. [8] I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio, “Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations,” Journal of Machine Learning Research, vol. 18, no. 187, pp. 1–30, 2018. [9] A. Antos and I. Kontoyiannis, “Convergence properties of functional estimates for discrete distributions,” Random Structures & Algorithms, vol. 19, no. 3-4, pp. 163–193, Oct. 2001. [10] T. Nguyen-Tang and J. Choi, “Markov Information Bottleneck to Improve Information Flow in Stochastic Neural Networks,” Entropy, vol. 21, no. 10, p. 976, Oct. 2019. [11] V. Raj, N. Nayak, and S. Kalyani, “Understanding learning dynamics of binary neural networks via information bottleneck,” arXiv:2006.07522 [cs.LG], Jun. 2020. [12] B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko, “Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Jun. 2018, pp. 2704–2713. [13] G. Hinton, “Neural Networks for Machine Learning,” University of Toronto, 2012. [14] Y. Bengio, N. Léonard, and A. Courville, “Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation,” Aug. 2013, arXiv:1308.3432 [cs]. [15] Y. Wu and P. Yang, “Minimax rates of entropy estimation on large alphabets via best polynomial approximation,” IEEE Trans. Inf. Theory, vol. 62, no. 6, pp. 3702–3720, 2016. [16] L. Paninski, “Estimation of Entropy and Mutual Information,” Neural Computation, vol. 15, no. 6, pp. 1191–1253, Jun. 2003.
8