arXiv CSopen access

Associative Emotional Learning in Convolutional Neural Networks

Associative Emotional Learning in Convolutional Neural Networks Seowung Leem 1, Andreas Keil 6, Mingzhou Ding 1 *, Ruogu Fang 1,2,3,4,5,* 1

J. Crayton Pruitt Family Dept. of Biomedical Engineering, University of Florida, Gainesville,

FL 32611, USA 2

Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL

32611, USA 3

Department of Computer and Information Science and Engineering, University of Florida,

Gainesville, FL. 32611, USA 4

Department of Radiology, University of Florida, Gainesville, FL. 32611

5

Center for Cognitive Aging and Memory, University of Florida, Gainesville, FL 32611, USA

6

Center for the Study of Emotion and Attention, University of Florida, Gainesville, Florida,

USA

*

Corresponding Author:

Ruogu Fang, Ph.D. J. Crayton Pruitt Family Department of Biomedical Engineering Herbert Wertheim College of Engineering University of Florida PO Box 116131, 1275 Center Drive

Gainesville, FL 32611-6131 Phone: (352) 294-1375 Email: [email protected]

Mingzhou Ding, Ph.D. J. Crayton Pruitt Family Department of Biomedical Engineering Herbert Wertheim College of Engineering University of Florida Building J285, 1275 Center Drive Gainesville, FL 32611-6131 Phone: (352) 273-9332 Email: [email protected] Abstract Associative emotional learning enables organisms to adaptively link pleasant or unpleasant outcomes to the presence of predictive stimuli. Whereas computational models such as the Rescorla-Wagner model have shed light on this important function, the limitations of these models are also known, especially when they are applied to neural data. The advent of deep neural networks has opened another avenue for modeling associative emotional learning. In this work we proposed a deep neural network model of visual valence processing, consisting of a visual module that encodes complex natural scenes and a module that recognizes their emotional

2

significance in terms of valence, a key dimension of emotion, and tested a novel Pavlovian learning paradigm on the model. The results showed that with learning, the model reproduced several observations from human associative learning studies, including association formation and generalization, and that the neural representations of the conditioned and the unconditioned stimuli became increasingly aligned both at the single unit and at the neural population level. Comparison between the model and human experimental data provided further validation of our approach. This study thus suggests that deep neural network models, when combined with appropriate learning algorithms, can be used to model behavioral and neural signatures of associative emotion/valence learning. Keywords. Computational Neuroscience, Associative Valence Learning, Deep Learning, Pavlovian Conditioning

1.

Introduction The formation of associative emotional memories, where discrete, non-emotional events

become linked to specific emotional responses such as fear or pleasure, is a fundamental building block of human behavior. Impaired emotional learning is characteristic of mental health problems such as anxiety, post-traumatic stress disorder (PTSD), and substance use disorder (Fullana et al., 2020). Therefore, understanding and modeling the mechanisms of associative emotional learning has both basic and clinical science significance. In human experimental research, Pavlovian (classical) conditioning is one of the key paradigms for probing how individuals form associations between stimuli (Beckers et al., 2023). In this framework, a neutral stimulus, referred to as the conditioned stimulus or CS+, is repeatedly paired with an unconditioned stimulus (US) that carries inherent emotional value, such as aversive electric

3

shock or a rewarding outcome. Over time, the CS+ acquires the capacity to evoke emotional and physiological responses similar to those triggered by the US, measurable through subjective ratings and physiological indices (e.g., skin conductance, heart rate, etc.) (Dunning & Hajcak, 2007; Freund & Keil, 2021; Lonsdorf et al., 2017; Löw et al., 2008). Classic models of Pavlovian conditioning, such as the Rescorla-Wagner model (Rescorla & Wagner, 1972; Mackintosh, 1975; Pearce & Hall, 1980; Sutton & Barto, 1990), capture many key behavioral observations but do not yield much insight into the underlying neural underpinnings. Neural models based on biophysically grounded neurons can account for fear acquisition, generalization, extinction, and contextual modulations in emotion-processing circuits such as the amygdala (Armony et al., 1995; G. Li et al., 2009; Schmajuk, 2012; Vlachos et al., 2011), but are limited in modeling large-scale changes of neuronal representations through associative learning. More recently, Artificial Neural Networks (ANNs) from the field of deep learning have shown markedly improved predictive power and flexibility (J. Liu et al., 2019, 2021; Shuvaev et al., 2021; Park et al., 2025). These models, while incorporating learning principles into the training process, the training procedures for these models often diverge from the experimental paradigm in empirical Pavlovian conditioning. How to achieve classical conditioning effects in contemporary ANNs under experimental paradigms typically used in empirical neuroscientific settings remains to be addressed. Neurophysiologically, anterior brain structures such as amygdala, hippocampus, anterior cingulate and medial frontal cortices have traditionally been viewed as the neural substrate of aversive Pavlovian conditioning (Davis, 1997; LeDoux, 1995; Maren & Fanselow, 1996; Rolls, 2023), recent studies have begun to emphasize the crucial role of sensory cortices in emotional processing (Bo et al., 2021) and in the acquisition and maintenance of conditioned emotional

4

responses (Friedl & Keil, 2021a; W. Li & Keil, 2023; You et al., 2021). Convolutional Neural Networks (CNNs), architecturally and computationally inspired by the primate visual system, offer new opportunities to simulate, test, and refine hypotheses about emotional learning, especially concerning the role of the sensory cortex. In particular, representations of CNNs have been shown to replicate stimulus representations in key visual areas such as V1, V4, and the inferior temporal (IT) cortex (Yamins et al., 2014; Khaligh-Razavi & Kriegeskorte, 2014; Güçlü & van Gerven, 2015; Seeliger et al., 2018; Cadena et al., 2019). Moreover, CNNs can be trained to recognize emotions in natural scenes, highlighting the sensory cortex’s role in emotional processing (Kragel et al., 2019; P. Liu et al., 2024). Presently, CNN-based models are trained on supervised learning, in which the difference between the model output and the ground truth label is minimized. To what extent these types of models can be trained using Pavlovian learning paradigms and how to evaluate the outcomes of such training remains to be understood. In this study, we designed an ANN model consisting of a CNN as the visual processing module, which sends the output to an emotion processing module, inspired by the function of the anterior valence-processing structures such as the amygdala and orbitofrontal cortex, to extract the valence of the input. A shortcut connection, motivated by the multi-pathway hypothesis, was further added to link early visual responses from the CNN and the valence-extracting module (Armony et al., 1995; LeDoux, 1996; Rudrauf et al., 2008; Schmajuk, 2012; P. Liu, 2021). The model was first trained to recognize the valence of intrinsically emotional natural images (e.g., those from the International Affective Picture System). A learning paradigm, paralleling the Pavlovian conditioning paradigms used in a large body of empirical psychological and neuroscientific studies (Miskovic & Keil, 2012; Rhodes et al., 2018), was then implemented. Specifically, two Gabor patches, 45 degrees and 135 degrees in orientation, were used as the

5

CS+s and presented together with unpleasant and pleasant images from the International Affective Picture System (IAPS) (Bradley & Lang, 2007) as USs, respectively. During learning, the model was trained to recognize the valence of the compound stimulus (Gabor patch + IAPS picture). The model’s capability of associative valence learning was assessed by examining its responses to a CS+ Gabor patch when it was presented alone. Additional questions considered included: (1) neural mechanisms of associative emotional learning, (2) generalization of the learning effects to new contexts, and (3) alignment between the model and human experimental data.

6

2. Materials and Methods Visual Cortex Module (VGG16)

28x28x512

14x14x512 7x7x512

Valence Module

56x56x256 112x112x128 224x224x64

1 2 FC FC

4096x1

) (5, 3

Skip Connection

t tc u or Sh FC1

2x2x128

18x18x128 (9 , 5

(1

Valence 1x1

t tc u or Sh FC2

) 2x2x128

10x10x128 3, 7)

1024x1 Sigmoid

2x2x128

7x7x128

512x1

(

2x2x128 4) 29, 1

512x1

512x1 512x1

Flatten 1024x1

2x2x128

Shortcut Pathway Convolution MaxPooling

Fully Connected Adaptive Maxpool

Adaptive Avgpool Sigmoid Weight Vector

Output

Frozen

Element-wise Multiplication Concatenation

Learnable

Fig.1. The Visual-Valence Model for associative emotional learning. The network model builds on VGG-16, initialized with weights pretrained on the ImageNet object recognition corpus as a model of the visual cortex, to provide a high-capacity visual feature extractor resembling the human ventral visual stream. A parallel Shortcut pathway was added to approximate rapid visual routes from early visual processing to emotion-related circuitry, motivated by evidence of both cortical and subcortical projections connecting early visual processing with higher-order brain structures. Outputs from the Visual Cortex and Shortcut pathways were concatenated and passed to two fully connected layers, the Valence Module, which emulates the anterior emotion processing structures and produces a scalar hedonic valence prediction.

7

2.1. Model Architecture for Conditioning Paradigm The Visual-Valence Model was designed to receive an input image (224x224) and output the single-digit valence decoding of the input image (See Fig. 1). It consists of 3 major components: a convolutional neural network (CNN) based Visual Cortex module, a Valence Module consisting of two fully connected layers + one-unit output layer, and a Shortcut Pathway module linking the early layers of the Visual Cortex module and the Valence Module. Prior work has shown that CNNs exhibit functional similarities to the human and the nonhuman primate ventral visual stream (Yamins et al., 2014; Kriegeskorte, 2015; Dobs et al., 2022) and that the convolutional window (kernel/filter size) mimics the local receptive fields of visual neurons and multiple convolutional layers model the hierarchical processing of visual features. The particular CNN model considered here is VGG-16 pretrained on the ImageNet dataset (Simonyan & Zisserman, 2015). For the Valence Module, the 2 fully connected layers plus the one-unit output layer was inspired by the three anatomical structures crucial for emotional processing: lateral nucleus (LA) and central nucleus (CE) of the amygdala, and orbitofrontal cortex (OFC), with the LA being the gateway to the amygdala, integrating the information from incoming sensory pathways (Erlich et al., 2012), the CE being the key structure which is in charge of emotional expressions from visual input and generates output to other parts of the brain to coordinate autonomic and behavioral responses appropriate for the goal of the organism (Kalin et al., 2004), and the OFC being a final decision making unit of emotional behavior based on processed sensory and hedonic stimuli (Carmichael & Price, 1995; Hornak et al., 2003; Kringelbach et al., 2003; Rempel-Clower, 2007; Rolls, 2023). In addition to receiving fully processed visual features from the final layer of the Visual Cortex Module, the Valence Module further receives partially processed visual features through the Shortcut Pathway Module, bypassing the

8

intermediate- and high-level visual processing stages. The Shortcut Pathway was inspired by the multi-pathway hypothesis in emotion processing (LeDoux, 1996; Rudrauf et al., 2008; Catani et al., 2003; Zemmoura et al., 2021) and implemented by (i) extracting the comprehensive features from the early layers of VGG-16 and (ii) processing the computed information using a channelwise attention mechanism. Pooling layers with different sizes and strides were used to obtain comprehensive features from early visual features from the Visual Cortex module. The channelwise attention mechanism was implemented by aggregating the features from the pooling layers and processing them using efficient channel attention (Kim & Kwon, 2025; Ramirez-Quintana et al., 2025; Wang et al., 2020). It is worth noting that the attention mechanism considered here refers to a computational mechanism and is not directly related to biological attention. 2.2. Model Development for Valence Decoding from Natural Scenes The model in Figure 1 was first trained to recognize the valence of the visual input. As indicated above, the weights in VGG 16 were held frozen, and the weights in the Shortcut pathway and in the connections between VGG16 and the valence module were trained so that the model acquires the ability to recognize the motivational significance of the visual input, i.e., the valence (on a scale of 1 to 9, with 1 corresponding to extreme displeasure, 9 to extreme pleasure, and 5 to the midpoint of neutral content). The Videoframe dataset (see Section 2.3.1 for image sampling from videoframes) was used by splitting the data into training, validation, and test splits at a split ratio of 8:1:1 (Cowen & Keltner, 2017); see Fig. 2A. The distribution of the valence ratings across the splits was made approximately equal to prevent any emotional bias (Appendix A; Table S1). The learning objective of the model was to decrease the mean squared error between the ground truth valence rating and the predicted valence rating of the image. Additional pictures from the International Affective Picture System (IAPS) were also used. The

9

original valence ratings from Videoframe (Likert scale) and IAPS (Self-Assessment Manikin scale; SAM scale) (Bradley & Lang, 1994) dataset were used as a valence label across this study. Across all training, image augmentations were applied to training and validation sets using a series of methods with corresponding probabilities: rotation (50%), color jittering (20%), Gaussian blurring (20%), and horizontal flipping (50%). The augmentations were not applied to the test set. Using the training and validation data, the model was optimized using the Stochastic Gradient Descent (SGD) algorithm in 50 epochs with a learning rate of 2e-5 and a batch size of 128. Specifically, the momentum of 0.9 and the weight decay of 1e-4 were applied in the SGD. In addition, the learning rate decayed with a multiplicative factor of 0.1 was applied for every 20 epochs. During the training, the best model was collected, defined as the one achieving the smallest loss on the validation set. The unconditioned stimuli used in the conditioning paradigm were images from the IAPS (see below). Therefore, the model trained on the Videoframe dataset was further finetuned with the IAPS dataset to minimize the effect of stimulus variations between the datasets. 10% of the IAPS data were used for training, 10% for validation, and the remaining 80% for testing. In this way, the vast majority of samples were reserved to evaluate the finetuned model. The model was optimized with a learning rate of 2e-4 and a batch size of 10, with 100 epochs. Given that in the conditioning paradigm, the CS+ and the US are presented in the second and the fourth quadrants, respectively, additional tuning was performed by reducing the field of view by placing the same IAPS dataset on 4th quadrant. The tuning was performed with 100 epochs, with a learning rate of 1e-5 and a batch size of 16. The final model, after finetuning on quadrants, was used in each conditioning scenario.

10

To evaluate the model’s performance on emotional decoding, the Pearson correlation coefficient between the model’s predicted valence and human-labeled valence of input stimuli was acquired to provide the behavioral level performance of the model. In each training step (after training with Videoframe and finetuning with IAPS), the model was evaluated with the test split corresponding to the source training data. 2.3. Affective Datasets More details of the two affective image datasets considered here: (1) the Videoframe dataset and (2) the IAPS dataset are provided below. 2.3.1 Cowen & Keltner Videoframe Dataset The Videoframe dataset (Cowen & Keltner, 2017) consists of 2,185 short videos designed to evoke emotional experiences. Each video has associated normative valence and arousal ratings, collected from 853 (403 female) English-speaking US participants using Amazon Mechanical Turk. The ratings were based on a nine-point Likert scale with 5 as neutral, 9 as most pleasant, and 1 as most unpleasant. In this study, the Videoframe dataset was used to develop the model’s emotion prediction capability. Because the model used in this study only accepted static pictures, to convert the videos into a set of static images, for a given video, every 10th frame was extracted and assigned the video’s valence. The dataset had a total of 21,876 affective images. 2.3.2 International Affective Picture System. The International Affective Picture System (IAPS) dataset consists of 1182 images, and each image has a normative valence rating collected from human observers (Bradley & Lang, 2007). The ratings of the image were based on the SAM scale, from 1 as most unpleasant to 9 as most pleasant. Unconditioned stimuli (appetitive/pleasant and aversive/unpleasant) for the 11

conditioning experiment were taken from this dataset. Specifically, 22 images with the most aversive valence (valence = 8.09±0.12), and 22 images with the most appetitive valence (valence = 1.55±0.10) were used as unconditioned stimuli. The purpose for using=>+/'/'? these extreme images was to achieve the most robust conditioning effects. The rest of the 1,138 images were used to fine-tune the model used in this study.

:

=>+/'/'? 9=)'3'3*&,+9

: >)1'7)9'63&,+9

!"#$%%&'()*+,

9+,9&,+9

!"#$%$&'()*+,+'-

-#!""&'()*+,

/',0)12/)1+34+&567+1 =>+/'/'? ./,0+)123#-/#'

4#5*) 2"#3'(.*4(.#)'*+#5678

!!<&'()*+,

!

23#-/#'+) .+)*'6*$@8B

23#-/#'+)

Testing .+)*'6*$@8B

!

A ? !

@#'5/-/#'*5 45o ./,0+)123#-/#' 4#5*)

!

!

!

!

!

1

!

!

45o & Unpleasant

./,0+)123#-/#' 23#-/#'+) 4#5*) .+)*'6*$#7$&8

135o

Emotional Valence of US

Visual-Valence Model

A

?637'9'63+7 9@#'5/-/#'*5 /','632/)1+34+&567+1

A

9

!

!

!

9;"#

!

4#5*) !"#

!

!

!

!

./,0+)123#-/#' 4#5*) !"#

@#'5/-/#'*5 =*,-/'? ./,0+)123#-/#'

Training

135o & Pleasant

:

!

!

/',0)12/)1+34+&567+1 !

!"#$%$&'()*+,+'-

B

9"#:/(-'*'/('(;#7&%&-';1

!

!

--&8(6,9:&;1+),)39&'()*+, --&8(6,9:&03;1+),)39&'()*+, 9;"#

!

.!-&'()*+,

23#-/#'+) .+)*'6*$#7$&8

23#-/#'+) .+)*'6*$#7$&8

9;"#

9;"#$%$<)*+,+'-

9

9 =*,-/'?

=*,-/'?

!!<&'()*+,

!"#$ %&'&()'

!

!

!

-#!".&'()*+,

!"#$%$&'()*+,+'-

./,0+)123#-/#' 4#5*)

!

9;"#$%$<)*+,+'-

*+,)-./&0) %&'&()'

!

!

!"#$%&'(#)'*+#,'-./0%&1.

!

A

!

9;"#$%$<)*+,+'-

Conditioned Visual-Valence Model

Emotional Valence CS+

23#-/#'+) .+)*'6*$@8B

!"#

Fig. 2. Model Training and Pavlovian conditioning paradigm. (A) Training and fine-tuning of the Visual-Valence model for achieving valence decoding. The model was first trained on the Videoframe dataset, which was split into training, validation, and test sets with 8:1:1 ratio. The trained model was finetuned with the IAPS dataset, with 10% of the images for training, 10% for

12

validation, and 80% for testing. From the IAPS dataset, the 22 most pleasant and 22 most unpleasant images based on the valence were selected as unconditioned stimuli (US) for the conditioning paradigm. The final model after quadrant-specific finetuning was used in the following conditioning experiment. (B) Schematic overview of the training and test of the conditioning paradigm. Each trial presents a compound stimulus consisting of a Gabor patch (an oriented grating) as the conditioned stimulus (CS+) and an affective natural image from IAPS as an US. The 45° Gabor patch was always paired with unpleasant images, whereas the 135° Gabor patch was always paired with pleasant images. The unconditional stimulus (i.e., an IAPS picture) was placed in the 4th quadrant. The conditional stimulus (CS+) was presented in the 2nd quadrant. These locations were fixed throughout the learning process. The task of the model was to predict the valence rating of the US for the compound stimulus. Bottom (testing): During the testing phase, Gabor patches of different orientations, spatial frequencies, and contrasts, as well as appearing in different quadrants, were presented, and the valence prediction from the model was obtained. (Note: The US images shown in this figure are shown for illustrative purposes only and were not part of the experimental stimulus set. The “family portrait” image was obtained from StockSnap (https://stocksnap.io/photo/family-portrait-6G9JL1XVYS), and the “gun” image was obtained from Rawpixel (https://www.rawpixel.com/image/5964101); both are released under the CC0 1.0 Universal license.)

2.4. Conditioning Paradigm for Deep Neural Networks After the model in Figure 1 was trained and finetuned to acquire the ability to recognize the valence of the visual input, the following learning paradigm was then implemented to test associative emotional learning.

13

2.4.1 Overview Fig. 2(B) shows the Pavlovian conditioning paradigm. Gabor patch gratings were used as CS+s. In particular, 45° and 135° Gabors were CS+s, associated with unpleasant and pleasant USs, respectively. The pairings of CS+s and US were performed for 100 epochs, with randomized pairings in every epoch. To understand how generalizable our paradigm is, we also tested frequency conditioning and contrast conditioning; for frequency conditioning, 0.1 and 0.3 cycles per pixel were CS+s associated with unpleasant and pleasant USs, respectively, and for contrast conditioning, 20% and 80% contrasts were CS+s associated with unpleasant and pleasant USs, respectively, and included the results in the Supplementary Information (Fig. S3). As the orientation conditioning depicted in Figure 2 is the most commonly used paradigm in empirical settings, the focus of this paper was primarily placed on orientation conditioning. Each conditioning scenario was repeated 10 independent times with different random seeds to account for the variability of valence prediction. 2.4.2 Paradigm (Training & Testing) The association of affective stimuli to neutral cues was implemented by presenting natural affective images (unconditional stimulus; US) and Gabor patches (conditional stimulus; CS+) simultaneously in two different quadrants: The US was always placed in the 4th quadrant, and CS+ was always located in the 2nd quadrant during both the learning and testing stages. During the learning phase, the model’s task was to predict the valence of the US for the CS+-US compound stimulus. The optimization method of the model was the same as the method used in the training & fine-tuning step, except that the learning rate was chosen to be 1e-4 and batch size 10. In the testing phase, only the CS+ (i.e., Gabor patches) was shown to the model, placed in the 2nd quadrant. The valence predictions by the model were obtained to assess whether, after

14

associative emotional learning, the neutral stimuli acquired the ability to evoke emotional responses. For generalization testing, the valence ratings of 13 different orientations (from 0° to 180° with 15° differences) were acquired, which included the two CS+s and 11 other Gabor gratings never seen by the model during the learning stage. In addition, the valence ratings of CS+s placed in different quadrants were obtained to test generalization across spatial locations. 2.5. Analysis of Neural Activation and Representation in the Model 2.5.1 Individual Neuron Level Correlation Analysis To characterize how associative learning reshapes the neural activation to stimuli, unitlevel responses to the CS+ and US were analyzed across the artificial neurons in the four trainable network layers (Shortcut FC1, Shortcut FC2, FC1, FC2) pre- and post-conditioning (see Fig. 1.). Co-selectivity was indexed by the Jaccard index, measuring the degree of overlap between units activated by the US and by the CS+. Within each layer, Pearson’s correlation was then computed between the CS+ and US evoked response neurons, reported both for all units and separately, for the subset co-active to both stimuli. The significance of correlation was obtained by a two-sided t-test under Student’s t distribution. This analysis was to examine whether the neurons responsive to US also became responsive to CS+ through associative learning. 2.5.2. Population Level Decoding Analysis via Support Vector Machine Extensive evidence has shown that population-level neural activity patterns encode information not possible at the single neuron level. The decoding analysis framework was set to read out (1) emotion (pleasant vs unpleasant) and (2) orientation (45° vs 135°) from network population features. The support vector machines (SVM) with a Gaussian kernel (𝛾=1/number of features) were used as a decoder to perform the analysis. The features were neural activity across

15

a layer standardized within a preprocessing pipeline using the mean and variance computed on the training split only to prevent data leakage. Unless noted, decoding was performed separately on features taken from each of four fully connected layers (Shortcut FC1, Shortcut FC2, FC1, FC2). Each decoding analysis was performed 100 times, and statistical significance was computed by the one-sample t-test. To compare the representations of the US and CS+ after associative emotional learning, the post-conditioning model was used as a feature extractor. Using the feature extractor, US (n=44; 22 unpleasant and 22 pleasant) and CS+ (n=2; 45° and 135°) representations were obtained from 4 layers. Then, the US representation was split into training/test sets into half (22 images per set) with a stratified split to maintain the same number of examples for each emotion. The decoder was fit with a training set. The fitted decoders were tested with US representations in the test set and CS+ representations. For CS+, 45° was considered unpleasant, and 135° was considered pleasant in this decoding analysis. To test whether the encoding of the physical stimulus property (e.g., orientation) associated with the CS+ (grating orientation) was preserved after associative learning, an orientation decoder was fitted independently of outcome valence. Two orientations (45° and 135°) were augmented with spatial frequencies (0.02~0.4 cycles per pixel, 0.02 interval) and spatial contrast (0~100%, with 10% interval), yielding 220 images per orientation (440 in total). For each image, feature vectors were extracted from the four fully connected layers pre- and post-conditioning. A decoder fit to pre-conditioning features was then evaluated on features from both the pre-conditioning and post-conditioning networks, providing a measure of orientation encoding and its persistence following learning.

16

2.

Results

3.1. The Visual-Valence Model The Visual-Valence Model used in this study consisted of a Visual Cortex module modeled by VGG-16 pretrained on ImageNet (Deng et al., 2009), a Valence module modeled by two fully connected layers plus a one-unit output layer, and a Shortcut Pathway connecting the early layers of the Visual Cortex module to the Valence module. The Valence module thus receives both low- and high-level visual representations of the input for mapping into emotional representations (Fig. 1). To endow the Visual-Valence model with the ability to assess the emotional significance of visual input, we first trained the model to predict the hedonic valence of natural scenes on a scale from 1 to 9, with 1 corresponding to extreme displeasure, 9 to extreme pleasure, and 5 to the midpoint of neutral content (Fig. S1). The decoding performance model for each test split (Videoframe=0.4007; IAPS=0.4395; result of train and validation split available in Appendix A). These results indicate the model had acquired the ability to reliably capture the hedonic valence of natural visual input.

17

A Pleasant Predicted Valence

9

135o

8 7 6 5 4

45o

3 2 1

Unpleasant

0

25

50 Epoch

B Pleasant

100

75

Pre

Predicted Valence

9

Post

8 7 6 5 4 3 2 1 0o

Unpleasant

C

15o 30o 45o 60o 75o 90o 105o 120o 135o 150o 165o 180o Orientation

D

Pleasant

Pleasant

9 8

****

9

**** ****

Predicted Valence

Predicted Valence

****

7 6 5 4 3 2 1

Unpleasant

quadrant1 45o

quadrant2

quadrant3

quadrant4

45o

**** ****

8 7 6 5 4 3 2 1

Unpleasant

quadrant1 135o

45o

45o

quadrant2

quadrant3

quadrant4

135o 135o

135o

Fig. 3. Model performance during and after associative emotional learning. (A) Model predicted valence of the conditioned stimuli (CS+) as a function of training epochs. The result of 10 independent associative learning experiments with different random seeds is shown in the figure, with the mean value displayed as a solid line and the standard error represented by shaded regions. 18

The two different conditioned stimuli (CS+) used in the associative learning process are displayed in different colors (violet for unpleasant, cyan for pleasant). Through associative learning, the model was able to differentiate the opposite emotional significance of the two CS+ gratings, consistent with findings from differential Pavlovian conditioning. (B) The predicted valence across 13 different orientations, including 11 orientations not used during training, demonstrates generalization by the trained model. It is noteworthy that before associative learning, the model assigned neutral valence ratings to all 13 stimuli (~5). Bar plot shows the valence prediction result of (C) 45°orientation and (D) 135° orientation placed in different quadrants after associative learning, demonstrating generalization across spatial locations. Here, the result was obtained by performing 10 different iterations. Error bars indicate the standard error of 10 different iterations, and statistical significance was calculated using a corrected t-test. N=10, ****P<0.0001, Bonferroni corrected. (Note: The y-axis in the figures is labeled with symbolic face markers to show the meaning of valence intuitively.)

3.2. Associative Emotional Learning in the Visual-Valence Model The Pavlovian conditioning paradigm is shown in Fig. 2B. There were two phases. In the training phase, the US and CS+ were shown simultaneously in different quadrants as a compound stimulus to the model. For the US, we used 22 unpleasant images (valence mean ± std; 1.31±0.10) for aversive conditioning and 22 pleasant images (valence mean ± std; 8.09±0.12) for appetitive conditioning, where all images were taken from IAPS. For the CS+, we used gratings oriented at 45 and 135 degrees relative to the vertical axis with 45 degrees always paired with the unpleasant images and 135 degrees with the pleasant images. On each trial, the CS+ and the US were presented in the 2nd quadrant and the 4th quadrant, respectively, and the

19

network was trained to predict the valence of the US. In the testing phase, CS+ was presented alone in the 2nd quadrant, and the predicted valence by the model was recorded and analyzed to examine whether the CS+ had acquired appetitive or aversive properties through associative learning and conditioning (see Section 2.5 for details). Fig. 3A shows the predicted valence of the two CS+ as associative learning progressed; here, the learning progression was indexed by the epoch number. At the start of the learning process (epoch 0), the valence of both CS+ was predicted by the model to be in the neutral range (for 45° and 135°orientations, the model predicted valence =5.37 & 5.48, respectively). While this result was somewhat expected because, at Epoch 0, CS+ has not been associated with emotional outcomes, it is nevertheless noteworthy as the model at this point has never been exposed to Gabor patches (i.e., there were no Gabor patches in the ImageNet database). After aversive and appetitive associative learning, the model predicted emotion significance of the 45° CS+ transitioned from neutral to unpleasant (valence=2.31±0.15) and the 135° CS+ from neutral to pleasant (valence=8.01±0.06). A similar pattern was found when CS+ was parametrized along the dimension of spatial frequency, but not when it was parametrized along the dimension of spatial contrast (Fig. S3; Appendix B.2). The association was not achieved when the valence of the US was randomized (Fig. S4A; Appendix B.3) and the learning was present with CS-US pairing was reversed (Fig. S4B; Appendix B.3). Having demonstrated that our model is capable of associative emotional learning, we next asked how the model responds to novel stimuli that bear resemblance to the CS+ but have never been shown to the model during learning. More specifically, we tested whether the model is capable of generalization, a well-established property in experimental emotional conditioning

20

for both mammals and humans (Dunsmoor et al., 2009; Hull, 1943; Lissek et al., 2008; Pavlov (1927), 2010). Two tests were performed. First, we tested the generalizability of our model to Gabor patches of different orientations (Fig. 3B). Specifically, Gabor patches of 13 different orientations (spanning 0° to 180° at 15° intervals) were presented to the Visual-Valence model pre- and post-conditioning. As expected, the valence for all 13 gratings was rated to be neutral (mean±s.d.; 5.08±1.62) before learning (pre-conditioning), implying that the deep neural network viewed the artificial stimuli as neutral despite the fact that it was never exposed to such stimuli during its training process. After conditioning (post-conditioning), the predicted valence across the 13 Gabor patches exhibited the classic generalization pattern, in which aversiveness and appetitiveness responses evoked by the stimuli in the model gradually diminished as the orientation moved away from 45° & 135°, respectively; in other words, the effect of associative learning was stronger when stimuli bore greater similarity to a given CS+. The monotonic generalization was observed in patterns across different contrast conditions in the orientation conditioned model (Fig. S5; Appendix B.4). Second, we analyzed the model’s spatial generalizability by presenting the CS+ in all 4 quadrants of the input image, including those where CS+ had never appeared during the learning phase, i.e., first, third, and fourth quadrants. The bar plots in Fig. 3C and Fig. 3D show the valence prediction result of two CS+ (45° and 135°). The emotional learning effects were most significant when CS+ was placed in the 2nd quadrant (mean±s.d.; 45°=2.31±0.15, 135°= 8.01±0.06), as expected, because the 2nd quadrant was where the CS+ appeared in the learning process, and became weaker but still significantly compared to the neutral valence in preassociative learning when it was placed in the other three quadrants (1st quadrant;

21

45°=3.95±0.26 t(9)=28.6, p<0.0001, 135°= 7.10±0.19 t(9)=19.4, p<0.0001, 3rd quadrant; 45°=3.03±0.19 t(9)=15.4, p<0.0001, 135°= 6.79±0.12 t(9)=32.6, p<0.0001, 4th quadrant; 45°=3.80±0.23 t(9)=19.8, p<0.0001, 135°= 6.07±0.12 t(9)=57.8, p<0.0001). These results demonstrate that emotional learning effects generalize across spatial locations. This is similar to what is observed in human experiments (Friedl & Keil, 2021b). To show that the conditionings effects were not specific to the particular type of CS+ used in the paradigm, using bar-shaped stimuli as a CS+, we found comparable conditioning outcomes (Fig. S6; Appendix B.5). In addition, to determine whether the conditioning effects reflect actual effects of learning rather than effects driven by low-level visual properties of the CS+ (e.g., orientation or contrast), we conducted a partial regression analysis controlling for preconditioning responses and stimulus features (Appendix C). The results indicated that the network’s predictions were driven by higher-level semantic structures, which is valence from the US in our study, rather than memorization of pixel-level features associated with the CS+ stimulus.

22

Fig. 4. Unit-level responses to CS+ and US. The violet (unpleasant) and cyan (pleasant) colors indicate the single neuron activation evoked by the unconditioned stimulus and the conditioned stimulus in layer (FC2) pre- (top row) and post- (bottom row) conditioning. The points were minmax scaled to 0 and 1, and the solid line represents a regression fit with a 95% confidence interval (shade). The metrics are (1) Pearson r (all): correlation computed across all neuron activation, (2) Pearson r (active): correlation computed across neurons active for both US and CS+, (3) Jaccard: co-activation index for neurons active for both CS+ and US. Statistical significance was calculated by the two-sided t-test. Metrics are derived from the original activations; scaling is for visualization purposes only.

23

3.3. Neural Mechanisms of Associative Emotional Learning It is expected that the neural responses to US and CS+ would become more similar as learning progressed. To determine whether our model conforms to this pattern, we quantified the impact of associative learning on the representation of the stimuli. First, unit-level responses across all trainable layers were analyzed. Specifically for each layer, the unit activations in response to CS+ (45° and 135°) and US (unpleasant and pleasant) were extracted. The activations evoked by US stimuli were averaged to yield a single US activation. Then, Pearson correlation between CS+ and US across all neurons within each layer before and after associative learning was computed. The correlation measures the degree of the learned representation of CS+ converging toward US, providing an interpretable change induced by the associative learning. Before associative learning, the last layer immediately preceding the valence output (FC2) exhibited the greatest co-activation (Jaccard index; unpleasant (Pre-Cond) =0.49, pleasant (Pre-Cond) =0.50, unpleasant (Pre-Cond) =0.55, pleasant (Pre-Cond) =0.49; Fig. 4) relative to earlier layers (Fig. S7; Appendix B.6). These findings indicate that learning-related changes should be most apparent in the last layer. Following associative learning, unit-wise responses to CS+ and US became more similar, and correlations increased by 0.33 (unpleasant) and 0.18 (pleasant). This convergence strengthened with layer depth (Shortcut FC1: Δr = 0, 0; Shortcut FC2: Δr = -0.06, 0.03; FC1: Δr = 0.15, 0.08). Consistent with the partial regression analysis of model behavior, we performed a similar analysis on neuronal activations while controlling for preconditioning responses and low-level visual properties of the Gabor patch stimuli (Appendix C). The results indicated that the representational structure of the CS+ was modulated by

24

associative learning. Thus, at the single neuron level, associative learning increased the alignment of the activation evoked by the CS+ and the US.

A

B

C

800

Pre-Cond

Post-Cond

135o 105o 120o 150o 165o

40 Unpleasant US Pleasant US

400 200 90 1800o 105o60o15o o 120o75oo165 o 135 150 o 45o30 o

0

20 t-SNE 2

t-SNE 2

600

o

90o 0o 180o

0 60o 15o 75o 45o o 30

-20

Unpleasant US

-200

-40 -200 -100

0

100 200 t-SNE 1

300

Pleasant US -100

400

-50

0 t-SNE 1

50

100

Fig. 5. Population-level representations of CS+ and US. (A) Classification accuracy of the support vector machine (SVM) emotion decoder fitted on half of the US feature representations and tested on the remaining US (yellow) and CS+ (teal) representations across 4 learnable layers.

25

The dashed line marks the 50% chance level. 100 different iterations were performed to derive statistical significance (****P<0.0001, one-sample t test). (B) Classification accuracy of the SVM orientation decoder fitted on the orientation representation of pre-conditioning representations of 45 and 135 degree gratings. The dashed line marks the 50% chance level reference, and the result of 100 different iterations is shown. The representation of orientation gratings from the model before (Pre-Cond) and after conditioning (Post-Cond) is tested. All results were statistically significant (P<0.0001, one-sample t test). (C) t-distributed stochastic neighborhood embedding (t-SNE) results show how the embedding of CS+ and US is modified by appetitive and aversive conditioning. The population-level representations of CS+ and US were obtained from the FC2 layer of the model. The colors of the data points indicate the emotion class of the US. The embeddings of 13 different orientations were labeled inside each data point. The embeddings of pre- and post-conditioning features were performed separately; therefore, the distance between the data points in the figure is not comparable.

To further demonstrate the population-level or pattern-level characteristic of the neural process under associative learning, we trained a series of support vector machine (SVM) decoders. The first set of decoders was trained on US images to discriminate unpleasant versus pleasant images, using layer-wise activation patterns as feature vectors (Fig. 5A). This process led to 4 different SVM classifiers: each SVM classifier for each of the four fully connected layers (see Section 2.5.2 for details). The objective of the SVM was to classify the two CS+s and the remaining US into two emotional classes. When evaluated on US images, decoding accuracy increased with depth (mean ± s.d.; chance level=0.5000, Shortcut pathway FC1 = 0.5068±0.1042, Shortcut pathway FC2 = 0.5073±0.0861, FC1=0.7027±0.0862,

26

FC2=0.8295±0.0836). This indicates that the affective information is concentrated in the deeper layers. Applying the same decoders to CS+ yielded a parallel depth-dependent pattern (Shortcut pathway FC1 = 0.5075±0.0321, Shortcut pathway FC2 = 0.5008±0.0050, FC1=0.5500±0.2072, FC2=0.7450±0.2512), suggesting that at the neural population level, the CS+ representation became more similar to US representations after associative learning. A second set of decoders was trained to discriminate the orientations of the two CS+s (45° and 135°) using layer-wise activation patterns as features to probe representational remapping (Fig. 5B). One decoder was fit per learnable fully connected layer with preconditioning activations and used to classify pre- and post-conditioning activations of two orientations. On pre-conditioning activations, all decoders performed at ceiling (chance level = 0.5000; accuracy = 1.0000 across layers), confirming robust orientation encoding. When evaluated on post-conditioning activations, performance collapsed to chance level (accuracy = 0.5000 across layers), indicating that the original orientation information in the CS+ representation was effectively overwritten by a new emotional code driven by associative learning. In Fig. 5C, the t-distributed stochastic neighborhood embedding (t-SNE) offers an additional intuitive visualization of the disposition of different Gabor patches and the unconditioned stimuli within the feature space of the last fully connected layer. Pre-conditioning, Gabors of different orientations were clustered in the middle between the unpleasant and pleasant images, whereas in post-conditioning, Gabors of different orientations were divided into three groups clustered together with the emotional images. Thus, the high classification accuracy of the SVM trained with the features from the last fully connected layer is visually explained by the t-SNE embeddings. Furthermore, embeddings post-conditioning suggests the learning

27

process induced convergence between feature representations of the conditioned and unconditioned stimuli, rather than merely aligning the orientation embeddings with unchanged emotional embeddings.

28

3.

Discussion In the present study, we set out to examine the extent to which deep neural networks can

be used as a model for associative emotional learning. We first developed a Visual-Valence model, a deep neural network consisting of a visual cortex module, an emotion evaluation module and a shortcut pathway connecting early visual processing with the emotion module, and trained the model to recognize the hedonic valence (pleasure versus displeasure) associated with intrinsically natural scenes. We then introduced a novel learning paradigm, in which neutral stimuli (Gabor patches) differing along a fundamental feature dimension, such as orientation, were paired with pleasant or unpleasant pictures, and the network was asked to assign the same valence as the emotional pictures to the compound stimuli. The valence ratings were then obtained for Gabor patches presented without the emotional pictures. We found that the model learned, within dozens of paired presentations, to assign pleasant or unpleasant valence to previously neutral visual stimuli that were consistently paired with pleasant or unpleasant outcomes, and that conditioned emotional evaluations generalized to similar stimuli in a fashion that resembled generalization in humans. Generalization was most pronounced for orientation conditioning, compared to contrast or spatial frequency conditioning, paralleling what has been observed with human participants (Friedl & Keil, 2021a). Unit-level and population-level neural representation analysis revealed that learning prompted by associative emotional conditioning mainly took place in deeper structures of the DNN model. Interestingly, as shown in Supplementary Materials (give details), as learning progressed, the neural activation patterns in these structures increasingly resembled those of human electrophysiological recordings. In the field of affective neuroscience, diverse neurophysiological models of emotional stimulus processing have been proposed based on brain imaging research (Pessoa & Adolphs,

29

2010; LeDoux, 1996). Our model’s architecture is based on having a sequential connection from visual input to anterior brain structures modulated by emotion, such as the amygdala and orbitofrontal cortex (OFC), and a shortcut pathway from early visual cortex to these structures, to simulate faster but less refined visual information (Kawasaki et al., 2001; Streit et al., 2003). Although the biological circuitry (e.g., cortico-visual pathway as well as subcortical visualemotion system) is far more complex than our simplified architecture, our experiments show that the model, when combined with a novel associative emotional learning paradigm, was able to produce the well-known conditioning effects where previously neutral stimuli (e.g., Gabor patches) were recognized as emotionally significant. In addition, neural mechanism analysis supports the core hypothesis of classical conditioning, which is that learning is done by increasing the representation similarity between conditioned and unconditioned stimuli. In particular, our model performs the learning by adjusting the hierarchical feature representations so that the conditioned stimulus comes to resemble the unconditioned stimulus. More broadly, the results underscore the utility of CNNs for computational studies of associative learning, owing to the fact that CNN-based architectures provide rich, layered sensory encodings of conditioned cues, enabling models to process complex visual inputs in a manner analogous to mammalian perception. This capacity makes them well-suited for probing how stimulus features are transformed during learning and for building models that bridge perception and affective processing. Additionally, in Supplementary Materials, it is shown the Visual-Valence model without the shortcut pathway was not able to demonstrate emotional associative learning (Fig. S2; Appendix B.1), lending support to the multi-pathway hypothesis of emotional processing (LeDoux, 1996; Rudrauf et al., 2008; Catani et al., 2003; Zemmoura et al., 2021).

30

In the context of recent neurocomputational theories, the amygdala is thought to play a role in affective processing and automatic responses, and signals from the orbitofrontal cortex encode the reward value of a given stimulus (e.g., Rolls, 2023). In this framework, appetitive and aversive dispositions arise from the brain systems that represent reward and punishment, and stimulus-reinforcer associations are learned through OFC and other prefrontal regions. In our Visual-Valence model, the associative learning between sensory stimuli and valence signals was implemented similarly. Moreover, conditioned valence representations mainly occurred in deeper layers, mirroring the formation of expected value representations. Although our approach does not explicitly simulate the recurrent processing, neuromodulator signals, or decision competition between reward representations, our model, by focusing on representational geometry in a high-dimensional sensory network trained under Pavlovian constrains, can be viewed as a complementary study of value learning that emphasizes how both shallow and deep sensory hierarchies together can support the emergence of conditioned representation. Our associative learning paradigm was grounded in prior human affective studies on classical conditioning. A substantial body of work has leveraged Gabor patches as conditioned stimuli (Song & Keil, 2014; Laurent et al., 2015; Nakashima & Sugita, 2017; Rhodes et al., 2018; Friedl & Keil, 2022). Compared to other types of stimuli, Gabor patches are advantageous in terms of their simple parametric features, allowing precise control over low-level visual properties with minimal confounds. In addition, Gabor filters have been widely used in visual neuroscience to probe early visual cortical processing (Schumacher & Olman, 2010; Bang et al., 2018; Fang et al., 2023), making them a suitable choice for CS+ in our conditioning paradigm for a deep learning model. The selection of two orthogonal orientations serves to maximize the perceptual discriminability and experimental control while enabling concurrent aversive and

31

appetitive conditioning along the same stimulus dimension (orientation). It is important to note that the conditioning effect we report in our study is not limited to a specific stimulus class. Using oriented bars as conditioned stimuli, our model achieved similar conditioning outcomes, in line with successful associative learning from recent rodent studies (Goltstein et al., 2018). One notable observation is that Gabor patches exhibited neutral valence prior to associative learning (Fig. 3A and 3B). Whereas biological organisms possess the intrinsic ability to recognize the emotional significance of visual input, in the Visual-Valence model, the model’s ability to predict the valence of unconditioned stimuli (e.g., IAPS images) is acquired after explicit training. After being trained to recognize the valence of the intrinsically positive or negative images, the model requires features or patterns in the input to drive the valence-related labels. The absence of these features in the Gabor patches will make them to receive a valence assignment around 5 (neutral). During valence learning, the model was assigned positive or negative valence to input images based on features or patterns driving affective responses. The neutrality of low-level visual features defined by the Gabors suggest that features capable of driving affective responses such as valence, likely depend on higher-level visual representations that are semantic- or object-dependent. In deep learning, reinforcement learning (RL; Sutton & Barto, 2018) provides an established account of how agents adapt their behavior to maximize cumulative reward by reward-prediction-error (RPE) driven learning. As a computational framework, RL has been used to formalize associative learning in human behavior (Gershman, 2015; Law & Gold, 2009). Within this view, operant conditioning is naturally captured as a similar type of learning where policies are shaped by the experienced consequence of choices. By contrast, Pavlovian conditioning is stimulus-centric and typically indexed by autonomic or reflexive responses to

32

conditioned cues. In Pavlovian conditioning, the brain learns to predict the occurrence of an unconditioned stimulus based on the presence of a conditioned stimulus, with the minimization of prediction error. Computationally, this has been modeled as reducing the discrepancy between a conditioned stimulus and an expected unconditioned outcome, closely aligned with temporal difference (TD) learning. Existing studies have leveraged an RL framework (Montague et al., 1996) to show that TD learning can capture key features of classical conditioning. Supervised learning is also closely aligned with Pavlovian conditioning in terms of minimization of prediction error. Our experimental results support the notion of similarity between naturalistic Pavlovian and supervised machine learning by reproducing behavioral and representational hallmarks of empirical observations from human observers (see Appendix D). The decoding analyses based on neural activation patterns in different layers of the model highlight that associative learning prompts changes in the representations of both the CS+ and the US. The findings regarding the two final fully connected layers in our model, intended to represent anterior emotion-modulated brain regions, support the presence of specific units that are selective for features of both CS+ and US. This is consistent with evidence supporting the role of structures such as the amygdala and orbitofrontal cortex as the locus in which the association between the CS+ and US is represented (Barot et al., 2008; Paton et al., 2006; Romanski et al., 1993). The decoding analysis of classifiers discriminating CS+ and US representations after associative learning (Figs. 5A and 5B) and projection of CS+ and US representations into two-dimensional space (Fig. 5C) suggest that the association of emotional context to neutral cues is established through the co-evolution of the representation patterns of CS+ and US. Specifically, patterns associated with the CS+ and US become more proximate to each other as learning progresses. This suggests that, in a computational architecture, associative

33

learning affects not only the neural representation of the CS+ but also that of the US. This was found in the deeper layers of our neural network, which were built to mimic the role of the amygdaloid complex and other anterior structures linked to emotional processing (Johansen et al., 2010). Based on the behavioral (Baltissen & Boucsein, 1986; Rust, 1976) and fMRI (Dunsmoor et al., 2008) studies of learning related changes of US, an in-depth analysis of how CS+ and US representations influence each other during the learning process is required. Thus, the present study lends support to the hypothesis that a similar process of representational change of CS and US might occur in brain regions where sensory information and emotional outcomes are integrated. In addition to the findings discussed above, this study contributes to bringing machine learning/AI and computational neuroscience together in a novel way. While our Pavlovian conditioning paradigm falls under the broader category of supervised learning, it diverges significantly from conventional supervised learning in its training and testing methodologies. In our paradigm, supervised learning is first performed by pairing unconditioned stimuli (US) with their associated valence scores (labels). This is followed by supervised training that assigns the compound stimulus consisting of the US and CS+ presently simultaneously, but in different locations, the label of the US. Finally, inference is performed on CS+ only. It is important to note that the present associative learning paradigm differs from associative rule learning (Agrawal et al., 1993), a data mining technique used to identify correlations and co-occurrences between datasets. By integrating associative learning principles from neuroscience into a computational machine learning approach, this study highlights the functional and representational similarities between artificial neurons in deep neural networks (DNN) and biological neurons in the neural system.

34

Several limitations in this study need to be acknowledged. First, the visual cortex module was kept frozen during the associative learning process, and as such, we were not able to examine the neural plasticity of visual neurons and their influence on the behavioral outcomes in associative learning (McTeague et al., 2015; Miskovic & Keil, 2012; Stolarova et al., 2006). Second, our model lacks sufficient biological realism. Conventional CNNs are pure feedforward models without any temporal processing capability. The absence of recurrence and feedback connections in both visual and emotional networks (Pessoa, 2017; Wyatte et al., 2014) limits the model’s capacity to explain the impact of top-down selective attention to cues in associative learning (Mackintosh, 1975; Pearce & Hall, 1980). In addition, our model does not have the ability to account for the effects of neuromodulators such as dopamine (Smith & Torregrossa, 2021; van der Schaaf et al., 2013), which is known to shape and configure aversive and appetitive representation by dopaminergic prediction error. Third, our learning paradigm only has the acquisition phase, whereas the typical associative learning paradigms also contain an extinction phase. This limits us from implementing different phases of conditioning, as well as how the temporal relationship between stimuli is established through learning (Schultz et al., 1997; Sutton, 1988; Sutton & Barto, 1990). Efforts are currently underway to enrich the paradigm so that additional associative learning processes, such as extinction learning, can be tested within the deep learning framework. Acknowledgements This research was supported by the National Science Foundation grant 1908299 and National Science Foundation grant 2318984, and National Institutes of Mental Health grants R01 MH112558 and R01 MH125615. References

35

Agrawal, R., Imieliński, T., & Swami, A. (1993). Mining association rules between sets of items in large databases. SIGMOD Rec., 22(2), 207–216. https://doi.org/10.1145/170036.170072 Armony, J. L., Servan-Schreiber, D., Cohen, J. D., & LeDoux, J. E. (1995). An anatomically constrained neural network model of fear conditioning. Behavioral Neuroscience, 109(2), 246–257. https://doi.org/10.1037//0735-7044.109.2.246 Baltissen, R., & Boucsein, W. (1986). Effects of a warning signal on reactions to aversive white noise stimulation: Does warning “short-circuit” habituation? Psychophysiology, 23(2), 224–231. https://doi.org/10.1111/j.1469-8986.1986.tb00623.x Bang, J. W., Sasaki, Y., Watanabe, T., & Rahnev, D. (2018). Feature-Specific Awake Reactivation in Human V1 after Visual Training. The Journal of Neuroscience, 38(45), 9648–9657. https://doi.org/10.1523/JNEUROSCI.0884-18.2018 Barot, S. K., Kyono, Y., Clark, E. W., & Bernstein, I. L. (2008). Visualizing stimulus convergence in amygdala neurons during associative learning. Proceedings of the National Academy of Sciences of the United States of America, 105(52), 20959–20963. https://doi.org/10.1073/pnas.0808996106 Beckers, T., Hermans, D., Lange, I., Luyten, L., Scheveneels, S., & Vervliet, B. (2023). Understanding clinical fear and anxiety through the lens of human fear conditioning. Nature Reviews Psychology, 2(4), 233–245. https://doi.org/10.1038/s44159-023-00156-1 Bo, K., Yin, S., Liu, Y., Hu, Z., Meyyappan, S., Kim, S., Keil, A., & Ding, M. (2021). Decoding Neural Representations of Affective Scenes in Retinotopic Visual Cortex. Cerebral Cortex, 31(6), 3047–3063. https://doi.org/10.1093/cercor/bhaa411

36

Bradley, M. M., & Lang, P. J. (1994). Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 25(1), 49–59. https://doi.org/10.1016/0005-7916(94)90063-9 Bradley, M. M., & Lang, P. J. (2007). The International Affective Picture System (IAPS) in the study of emotion and attention. In Handbook of emotion elicitation and assessment (pp. 29–46). Oxford University Press. Cadena, S. A., Denfield, G. H., Walker, E. Y., Gatys, L. A., Tolias, A. S., Bethge, M., & Ecker, A. S. (2019). Deep convolutional models improve predictions of macaque V1 responses to natural images. PLOS Computational Biology, 15(4), e1006897. https://doi.org/10.1371/journal.pcbi.1006897 Carmichael, S. T., & Price, J. L. (1995). Limbic connections of the orbital and medial prefrontal cortex in macaque monkeys. Journal of Comparative Neurology, 363(4), 615–641. https://doi.org/10.1002/cne.903630408 Catani, M., Jones, D. K., Donato, R., & ffytche, D. H. (2003). Occipito‐temporal connections in the human brain. Brain, 126(9), 2093–2107. https://doi.org/10.1093/brain/awg203 Cowen, A. S., & Keltner, D. (2017). Self-report captures 27 distinct categories of emotion bridged by continuous gradients. Proceedings of the National Academy of Sciences, 114(38), E7900–E7909. https://doi.org/10.1073/pnas.1702247114 Davis, M. (1997). Neurobiology of fear responses: The role of the amygdala. The Journal of Neuropsychiatry and Clinical Neurosciences, 9(3), 382–402. https://doi.org/10.1176/jnp.9.3.382

37

Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009). ImageNet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition, 248–255. https://doi.org/10.1109/CVPR.2009.5206848 Dobs, K., Martinez, J., Kell, A. J. E., & Kanwisher, N. (2022). Brain-like functional specialization emerges spontaneously in deep neural networks. Science Advances, 8(11), eabl8913. https://doi.org/10.1126/sciadv.abl8913 Dunning, J. P., & Hajcak, G. (2007). Error-related negativities elicited by monetary loss and cues that predict loss. Neuroreport, 18(17), 1875–1878. https://doi.org/10.1097/WNR.0b013e3282f0d50b Dunsmoor, J. E., Bandettini, P. A., & Knight, D. C. (2008). Neural correlates of unconditioned response diminution during Pavlovian conditioning. NeuroImage, 40(2), 811–817. https://doi.org/10.1016/j.neuroimage.2007.11.042 Dunsmoor, J. E., Mitroff, S. R., & LaBar, K. S. (2009). Generalization of conditioned fear along a dimension of increasing fear intensity. Learning & Memory, 16(7), 460–469. https://doi.org/10.1101/lm.1431609 Erlich, J. C., Bush, D. E. A., & Ledoux, J. E. (2012). The role of the lateral amygdala in the retrieval and maintenance of fear-memories formed by repeated probabilistic reinforcement. Frontiers in Behavioral Neuroscience, 6, 16. https://doi.org/10.3389/fnbeh.2012.00016 Fang, Z., Bloem, I. M., Olsson, C., Ma, W. J., & Winawer, J. (2023). Normalization by orientation-tuned surround in human V1-V3. PLOS Computational Biology, 19(12), e1011704. https://doi.org/10.1371/journal.pcbi.1011704

38

Freund, A. M., & Keil, A. (2021). Adult age-related differences in appetitive and aversive associative learning. Emotion, 21(6), 1239–1251. https://doi.org/10.1037/emo0000860 Friedl, W. M., & Keil, A. (2021a). Aversive Conditioning of Spatial Position Sharpens Neural Population-Level Tuning in Visual Cortex and Selectively Alters Alpha-Band Activity. Journal of Neuroscience, 41(26), 5723–5733. https://doi.org/10.1523/JNEUROSCI.288920.2021 Friedl, W. M., & Keil, A. (2021b). Aversive Conditioning of Spatial Position Sharpens Neural Population-Level Tuning in Visual Cortex and Selectively Alters Alpha-Band Activity. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 41(26), 5723–5733. https://doi.org/10.1523/JNEUROSCI.2889-20.2021 Friedl, W. M., & Keil, A. (2022). Effects of Aversive Classical Conditioning on Pupil Dilation and Microsaccades (p. 2022.06.09.495508). bioRxiv. https://doi.org/10.1101/2022.06.09.495508 Fullana, M. A., Dunsmoor, J. E., Schruers, K. R. J., Savage, H. S., Bach, D. R., & Harrison, B. J. (2020). Human fear conditioning: From neuroscience to the clinic. Behaviour Research and Therapy, 124, 103528. https://doi.org/10.1016/j.brat.2019.103528 Gershman, S. J. (2015). A Unifying Probabilistic View of Associative Learning. PLOS Computational Biology, 11(11), e1004567. https://doi.org/10.1371/journal.pcbi.1004567 Goltstein, P. M., Meijer, G. T., & Pennartz, C. M. (2018). Conditioning sharpens the spatial representation of rewarded stimuli in mouse primary visual cortex. eLife, 7, e37683. https://doi.org/10.7554/eLife.37683

39

Güçlü, U., & van Gerven, M. A. J. (2015). Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream. The Journal of Neuroscience, 35(27), 10005. https://doi.org/10.1523/JNEUROSCI.5023-14.2015 Hornak, J., Bramham, J., Rolls, E. T., Morris, R. G., O’Doherty, J., Bullock, P. R., & Polkey, C. E. (2003). Changes in emotion after circumscribed surgical lesions of the orbitofrontal and cingulate cortices. Brain, 126(7), 1691–1712. https://doi.org/10.1093/brain/awg168 Hull, C. L. (1943). Principles of behavior: An introduction to behavior theory (pp. x, 422). Appleton-Century. Johansen, J. P., Tarpley, J. W., LeDoux, J. E., & Blair, H. T. (2010). Neural substrates for expectation-modulated fear learning in the amygdala and periaqueductal gray. Nature Neuroscience, 13(8), 979–986. https://doi.org/10.1038/nn.2594 Kalin, N. H., Shelton, S. E., & Davidson, R. J. (2004). The Role of the Central Nucleus of the Amygdala in Mediating Fear and Anxiety in the Primate. The Journal of Neuroscience, 24(24), 5506–5515. https://doi.org/10.1523/JNEUROSCI.0292-04.2004 Kawasaki, H., Adolphs, R., Kaufman, O., Damasio, H., Damasio, A. R., Granner, M., Bakken, H., Hori, T., & Howard, M. A. (2001). Single-neuron responses to emotional visual stimuli recorded in human ventral prefrontal cortex. Nature Neuroscience, 4(1), Article 1. https://doi.org/10.1038/82850 Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014). Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical Representation. PLOS Computational Biology, 10(11), e1003915. https://doi.org/10.1371/journal.pcbi.1003915

40

Kim, B., & Kwon, Y. (2025). Searching for effective preprocessing method and CNN based architecture with efficient channel attention on speech emotion recognition. Scientific Reports, 15(1), 32689. https://doi.org/10.1038/s41598-025-19887-7 Kragel, P. A., Reddan, M. C., LaBar, K. S., & Wager, T. D. (2019). Emotion schemas are embedded in the human visual system. Science Advances, 5(7), eaaw4358. https://doi.org/10.1126/sciadv.aaw4358 Kriegeskorte, N. (2015). Deep Neural Networks: A New Framework for Modeling Biological Vision and Brain Information Processing. Annual Review of Vision Science, 1(1), 417– 446. https://doi.org/10.1146/annurev-vision-082114-035447 Kringelbach, M. L., O’Doherty, J., Rolls, E. T., & Andrews, C. (2003). Activation of the Human Orbitofrontal Cortex to a Liquid Food Stimulus is Correlated with its Subjective Pleasantness. Cerebral Cortex, 13(10), 1064–1071. https://doi.org/10.1093/cercor/13.10.1064 Laurent, P. A., Hall, M. G., Anderson, B. A., & Yantis, S. (2015). Valuable Orientations Capture Attention. Visual Cognition, 23(1–2), 133–146. https://doi.org/10.1080/13506285.2014.965242 Law, C.-T., & Gold, J. I. (2009). Reinforcement learning can account for associative and perceptual learning on a visual-decision task. Nature Neuroscience, 12(5), 655–663. https://doi.org/10.1038/nn.2304 LeDoux, J. E. (1995). Emotion: Clues from the brain. Annual Review of Psychology, 46, 209– 235. https://doi.org/10.1146/annurev.ps.46.020195.001233 LeDoux, J. E. (1996). The emotional brain: The mysterious underpinnings of emotional life (p. 384). Simon & Schuster.

41

Li, G., Nair, S. S., & Quirk, G. J. (2009). A biologically realistic network model of acquisition and extinction of conditioned fear associations in lateral amygdala neurons. Journal of Neurophysiology, 101(3), 1629–1646. https://doi.org/10.1152/jn.90765.2008 Li, W., & Keil, A. (2023). Sensing fear: Fast and precise threat evaluation in human sensory cortex. Trends in Cognitive Sciences, 27(4), 341–352. https://doi.org/10.1016/j.tics.2023.01.001 Lissek, S., Biggs, A. L., Rabin, S. J., Cornwell, B. R., Alvarez, R. P., Pine, D. S., & Grillon, C. (2008). Generalization of conditioned fear-potentiated startle in humans: Experimental validation and clinical relevance. Behaviour Research and Therapy, 46(5), 678–687. https://doi.org/10.1016/j.brat.2008.02.005 Liu, J., Gong, M., & He, H. (2019). Deep associative neural network for associative memory based on unsupervised representation learning. Neural Networks, 113, 41–53. https://doi.org/10.1016/j.neunet.2019.01.004 Liu, J., Zhang, W., Liu, F., & Xiao, L. (2021). Deep associative learning for neural networks. Neurocomputing, 443, 222–234. https://doi.org/10.1016/j.neucom.2021.03.012 Liu, P. (2021). Biology and Neuroscience Inspired Deep Learning. https://ufdc.ufl.edu/UFE0058130/00001/citation Liu, P., Bo, K., Ding, M., & Fang, R. (2024). Emergence of Emotion Selectivity in Deep Neural Networks Trained to Recognize Visual Objects. bioRxiv, 2023.04.16.537079. https://doi.org/10.1101/2023.04.16.537079 Lonsdorf, T. B., Menz, M. M., Andreatta, M., Fullana, M. A., Golkar, A., Haaker, J., Heitland, I., Hermann, A., Kuhn, M., Kruse, O., Meir Drexler, S., Meulders, A., Nees, F., Pittig, A., Richter, J., Römer, S., Shiban, Y., Schmitz, A., Straube, B., … Merz, C. J. (2017). Don’t

42

fear “fear conditioning”: Methodological considerations for the design and analysis of studies on human fear acquisition, extinction, and return of fear. Neuroscience and Biobehavioral Reviews, 77, 247–285. https://doi.org/10.1016/j.neubiorev.2017.02.026 Löw, A., Lang, P. J., Smith, J. C., & Bradley, M. M. (2008). Both predator and prey: Emotional arousal in threat and reward. Psychological Science, 19(9), 865–873. https://doi.org/10.1111/j.1467-9280.2008.02170.x Mackintosh, N. J. (1975). A theory of attention: Variations in the associability of stimuli with reinforcement. Psychological Review, 82(4), 276–298. https://doi.org/10.1037/h0076778 Maren, S., & Fanselow, M. S. (1996). The amygdala and fear conditioning: Has the nut been cracked? Neuron, 16(2), 237–240. https://doi.org/10.1016/s0896-6273(00)80041-0 McTeague, L. M., Gruss, L. F., & Keil, A. (2015). Aversive learning shapes neuronal orientation tuning in human visual cortex. Nature Communications, 6, 7823–7823. PubMed (26215466). https://doi.org/10.1038/ncomms8823 Miskovic, V., & Keil, A. (2012). Acquired fears reflected in cortical sensory processing: A review of electrophysiological studies of human classical conditioning. Psychophysiology, 49(9), 1230–1241. https://doi.org/10.1111/j.1469-8986.2012.01398.x Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 16(5), 1936–1947. https://doi.org/10.1523/JNEUROSCI.16-05-01936.1996 Nakashima, Y., & Sugita, Y. (2017). The reference frame of the tilt aftereffect measured by differential Pavlovian conditioning. Scientific Reports, 7(1), 40525. https://doi.org/10.1038/srep40525

43

Park, W., Kim, J., Jeong, I., & Lee, K. J. (2025). Temporal pavlovian conditioning of a model spiking neural network for discrimination sequences of short time intervals. Journal of Computational Neuroscience, 53(1), 163–179. https://doi.org/10.1007/s10827-02500896-4 Paton, J. J., Belova, M. A., Morrison, S. E., & Salzman, C. D. (2006). The primate amygdala represents the positive and negative value of visual stimuli during learning. Nature, 439(7078), 865–870. https://doi.org/10.1038/nature04490 Pavlov (1927), P. I. (2010). Conditioned reflexes: An investigation of the physiological activity of the cerebral cortex. Annals of Neurosciences, 17(3), 136–141. https://doi.org/10.5214/ans.0972-7531.1017309 Pearce, J. M., & Hall, G. (1980). A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review, 87(6), 532–552. Pessoa, L. (2017). A Network Model of the Emotional Brain. Trends in Cognitive Sciences, 21(5), 357–371. https://doi.org/10.1016/j.tics.2017.03.002 Pessoa, L., & Adolphs, R. (2010). Emotion processing and the amygdala: From a “low road” to “many roads” of evaluating biological significance. Nature Reviews. Neuroscience, 11(11), 773–783. PubMed (20959860). https://doi.org/10.1038/nrn2920 Ramirez-Quintana, J. A., Muñoz-Pacheco, J. J., Ramirez-Alonso, G., Medrano-Hermosillo, J. A., & Corral-Saenz, A. D. (2025). Lightweight Convolutional Neural Network with Efficient Channel Attention Mechanism for Real-Time Facial Emotion Recognition in Embedded Systems. Sensors (Basel, Switzerland), 25(23), 7264. https://doi.org/10.3390/s25237264

44

Rempel-Clower, N. L. (2007). Role of Orbitofrontal Cortex Connections in Emotion. Annals of the New York Academy of Sciences, 1121(1), 72–86. https://doi.org/10.1196/annals.1401.026 Rescorla, R., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. https://www.semanticscholar.org/paper/A-theory-of-Pavlovian-conditioning-%3AVariations-in-Rescorla-Wagner/afaf65883ff75cc19926f61f181a687927789ad1#citingpapers Rhodes, L. J., Ruiz, A., Ríos, M., Nguyen, T., & Miskovic, V. (2018). Differential aversive learning enhances orientation discrimination. Cognition & Emotion, 32(4), 885–891. https://doi.org/10.1080/02699931.2017.1347084 Rolls, E. T. (2023). Emotion, motivation, decision-making, the orbitofrontal cortex, anterior cingulate cortex, and the amygdala. Brain Structure & Function, 228(5), 1201–1257. https://doi.org/10.1007/s00429-023-02644-9 Romanski, L. M., Clugnet, M. C., Bordi, F., & LeDoux, J. E. (1993). Somatosensory and auditory convergence in the lateral nucleus of the amygdala. Behavioral Neuroscience, 107(3), 444–450. https://doi.org/10.1037//0735-7044.107.3.444 Rudrauf, D., David, O., Lachaux, J.-P., Kovach, C. K., Martinerie, J., Renault, B., & Damasio, A. (2008). Rapid Interactions between the Ventral Visual Stream and Emotion-Related Structures Rely on a Two-Pathway Architecture. Journal of Neuroscience, 28(11), 2793– 2803. https://doi.org/10.1523/JNEUROSCI.3476-07.2008

45

Rust, J. (1976). Unconditioned response diminution in the skin resistance response. The Journal of General Psychology, 95(1st Half), 77–84. https://doi.org/10.1080/00221309.1976.9710867 Schmajuk, N. A. (2012). Computational Models of Classical Conditioning. In N. M. Seel (Ed.), Encyclopedia of the Sciences of Learning (pp. 700–707). Springer US. https://doi.org/10.1007/978-1-4419-1428-6_528 Schultz, W., Dayan, P., & Montague, P. R. (1997). A Neural Substrate of Prediction and Reward. Science, 275(5306), 1593–1599. https://doi.org/10.1126/science.275.5306.1593 Schumacher, J. F., & Olman, C. A. (2010). High-resolution BOLD fMRI measurements of local orientation-dependent contextual modulation show a mismatch between predicted V1 output and local BOLD response. Vision Research, 50(13), 1214–1224. https://doi.org/10.1016/j.visres.2010.04.005 Seeliger, K., Fritsche, M., Güçlü, U., Schoenmakers, S., Schoffelen, J.-M., Bosch, S. E., & van Gerven, M. A. J. (2018). Convolutional neural network-based encoding and decoding of visual object recognition in space and time. NeuroImage, New Advances in Encoding and Decoding of Brain Signals, 180, 253–266. https://doi.org/10.1016/j.neuroimage.2017.07.018 Shuvaev, S. A., Tran, N. B., Stephenson-Jones, M., Li, B., & Koulakov, A. A. (2021). Neural Networks With Motivation. Frontiers in Systems Neuroscience, 14. https://doi.org/10.3389/fnsys.2020.609316 Simonyan, K., & Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition (arXiv:1409.1556). arXiv. https://doi.org/10.48550/arXiv.1409.1556

46

Smith, D. M., & Torregrossa, M. M. (2021). Valence encoding in the amygdala influences motivated behavior. Behavioural Brain Research, 411, 113370. https://doi.org/10.1016/j.bbr.2021.113370 Song, I., & Keil, A. (2014). Differential Classical Conditioning Selectively Heightens Response Gain of Neural Population Activity in Human Visual Cortex. Psychophysiology, 51(11), 1185–1194. https://doi.org/10.1111/psyp.12260 Stolarova, M., Keil, A., & Moratti, S. (2006). Modulation of the C1 visual event-related component by conditioned stimuli: Evidence for sensory plasticity in early affective perception. Cerebral Cortex (New York, N.Y.: 1991), 16(6), 876–887. https://doi.org/10.1093/cercor/bhj031 Streit, M., Dammers, J., Simsek-Kraues, S., Brinkmeyer, J., Wölwer, W., & Ioannides, A. (2003). Time course of regional brain activations during facial emotion recognition in humans. Neuroscience Letters, 342(1), 101–104. https://doi.org/10.1016/S03043940(03)00274-X Sutton, R. S. (1988). Learning to predict by the methods of temporal differences. Machine Learning, 3(1), 9–44. https://doi.org/10.1007/BF00115009 Sutton, R. S., & Barto, A. G. (1990). Time-derivative models of Pavlovian reinforcement. In Learning and computational neuroscience: Foundations of adaptive networks (pp. 497– 537). The MIT Press. Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning: An Introduction (2nd ed.). MIT Press. https://mitpress.mit.edu/9780262039246/reinforcement-learning/ van der Schaaf, M. E., Zwiers, M. P., van Schouwenburg, M. R., Geurts, D. E. M., Schellekens, A. F. A., Buitelaar, J. K., Verkes, R. J., & Cools, R. (2013). Dopaminergic drug effects

47

during reversal learning depend on anatomical connections between the orbitofrontal cortex and the amygdala. Frontiers in Neuroscience, 7. https://doi.org/10.3389/fnins.2013.00142 Vlachos, I., Herry, C., Lüthi, A., Aertsen, A., & Kumar, A. (2011). Context-Dependent Encoding of Fear and Extinction Memories in a Large-Scale Network Model of the Basal Amygdala. PLOS Computational Biology, 7(3), e1001104. https://doi.org/10.1371/journal.pcbi.1001104 Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., & Hu, Q. (2020). ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 11531–11539. https://doi.org/10.1109/CVPR42600.2020.01155 Wyatte, D., Jilk, D. J., & O’Reilly, R. C. (2014). Early recurrent feedback facilitates visual object recognition under challenging conditions. Frontiers in Psychology, 5. https://doi.org/10.3389/fpsyg.2014.00674 Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014). Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proceedings of the National Academy of Sciences, 111(23), 8619–8624. https://doi.org/10.1073/pnas.1403112111 You, Y., Brown, J., & Li, W. (2021). Human Sensory Cortex Contributes to the Long-Term Storage of Aversive Conditioning. The Journal of Neuroscience, 41(14), 3222–3233. https://doi.org/10.1523/JNEUROSCI.2325-20.2021

48

Zemmoura, I., Burkhardt, E., & Herbet, G. (2021). The inferior longitudinal fasciculus: Anatomy, function and surgical considerations. Journal of Neurosurgical Sciences, 65(6), 590–604. https://doi.org/10.23736/S0390-5616.21.05391-1

Author Statement* Author Contributions: SL: Conceptualization, Methodology, Investigation, Visualization, Writing—original draft, Writing—review & editing. AK: Conceptualization, Methodology, Investigation, Supervision, review & editing. RF: Conceptualization, Methodology, Investigation, Supervision, review & editing. MD: Conceptualization, Methodology, Investigation, Supervision, review & editing.

49