Curr Biol︱ Novelty detection and surprise in relation to recency in the primate brain

Written by ︱ Kaining Zhang ︱ Sizhen Wang Humans and other animals learn by exploring objects, and novel objects often play an important role in our daily lives
.

Behavioral experiments in primates have shown that novel visual objects can attract attention, turn the head and eyes, and promote the formation of new memories [1-6]
.

Novelty detection, on the other hand, determines whether a given object has been seen before
.

It is a topic of research in both neuroscience and machine learning
.

There have been several algorithms in the field of machine learning to achieve novelty detection [7, 8]
.

It has also been reported in the field of neuroscience that neurons in many brain regions respond differently to novel and familiar stimuli [9-11]
.

However, how the primate brain achieves novelty detection is still not fully understood
.

 In April 2022, Ilya Monosov's research team at Washington University in St.
Louis published an article entitled "Surprise and recency in novelty detection in the primate brain" in Current Biology
.

Zhang Kaining is the first author, and Ilya Monosov is the corresponding author
.

This paper illustrates that the brain circuits for novel object detection in the primate brain are intertwined and underpinned by the computational circuits of recency (how long ago a similar stimulus was experienced) and surprise (the unpredictability of the stimulus)
.

and found that neurons in the brain have multiple timescales of learning rates and forgetting in the process of learning new objects
.

Novel stimuli differ from familiar stimuli in many ways
.

For example, novel stimuli are often unexpected or different from recently experienced stimuli [12, 13]
.

Previous studies have proposed several theories and models of novelty detection in the brain, each of which makes different predictions about the nature of neurons in the brain that respond to novel objects (Fig.
1A)
.

The first category of models conceptualizes novelty as a form of sensory surprise [14]
.

Sensory surprise comes from the mismatch between incoming sensory information and predictions [6]
.

Under such models, neurons that respond to novel objects should also be sensitive to sensory surprise
.

A second class of models conceptualize novelty as a recency or repetition effect, usually operationally defined as the ability of a neuron to distinguish how long ago the last identical stimulus occurred [13, 15, 16]
.

These two classes of models may also be interdependent and cooperative [17, 18], especially when the brain contains circuits with multiple memory timescales
.

Therefore, a third class of models predicts that novelty detection may be related to both sensory surprise and recency computations
.

Finally, a fourth class of models predicts that novelty detection may be independent of sensory surprise or recency calculations, purely to distinguish whether the same stimulus has been seen before [19, 20]
.

 In this study, the researchers gave two Monkeys implanted with electrode arrays to record neurons in the temporal cortex, amygdala, hippocampus, basal ganglia, and prefrontal cortex
.

While recording neuronal signals, the two monkeys participated in an unsupervised learning behavioral task (Fig.
1B)
.

 The behavioral task generally involves showing the animal a series of fractal pictures
.

These pictures include novel or familiar pictures, predictable or unpredictable familiar pictures, and recent or not new familiar pictures
.

Figure 1.
Model prediction and behavioral experimental tasks for novelty detection (Source: Zhang K, et al.
, Curr Biol, 2022) In the recorded neuron data, the researchers found that neurons are sensitive to novelty, surprise, and recency.
The relationship between the responses was consistent with that predicted by the third type of model
.

Figure 2 shows an example neuron in the amygdala
.

The neuron showed stronger excitation to new stimuli than to familiar stimuli
.

At the same time, this neuron is also sensitive to surprise: it is more excited to unpredictable familiar stimuli than to predictable familiar stimuli
.

In addition, this neuron is also sensitive to recency: it is more excited to familiar stimuli that have not been presented recently
.

Thus, cells respond to all three types of objects—novel, surprising, and recently unseen objects
.

Figure 2 Example neuron (Source: Zhang K, et al.
, Curr Biol, 2022) In the population encoding of neurons, the researchers found the same result, neurons that responded to novelty as a group simultaneously Sensory surprise and recency showed significant sensitivity (Fig.
3A)
.

We also used classifiers trained to detect whether pictures were surprising or recent to detect whether pictures were novel (Fig.
3B)
.

The results show that these classifiers are all able to judge the novelty of images significantly higher than chance
.

This suggests that novelty and surprise, recency have similar neuronal population encodings
.

And, in neurons excited to novelty, the degree of neuronal sensitivity to novelty was significantly positively correlated with the degree of sensitivity to sensory surprise and recency (Fig.
3C)
.

In addition, the researchers have controlled experiments showing that neuronal responses to novelty and recency and surprise are correlated not due to some global variable such as arousal or attention
.

Figure 3 Novelty-sensitive neurons are also sensitive to sensory surprise and recency (Source: Zhang K, et al.
, Curr Biol, 2022) Distribution of novelty-sensitive neurons in primate brains not uniform (Fig.
4 left)
.

Some brain regions, notably the anterior ventromedial temporal cortex (AVMTC), and areas connected to it, such as area 46v, the basal forebrain, and the amygdala, are rich in novel objects responsive to neuron
.

The researchers found a cross-regional relationship between novelty, recency, and sensory surprise—regions that were broadly rich in neurons that responded to novel objects were also rich in neurons that responded to recency and sensory surprise , which is consistent with the results of neuronal population encoding
.

 This finding raises another question, whether the relationship between novelty and sensory surprise and recency is only associated with a few brain regions rich in neurons that respond to novelty? Or is this relationship a feature shared by brain regions involved in novelty processing? The data suggest that the latter is the case: there is a remarkably consistent positive correlation between novelty and sensory surprise and recency in the vast majority of recorded brain regions (Fig.
4 right)
.

 In conclusion, neuronal novelty sensitivity was strongly correlated with sensory surprise and recency, and the researchers observed this relationship across and within brain regions
.

Figure 4 The distribution of neurons sensitive to novelty, sensory surprise and recency in different brain regions (Source: Zhang K, et al.
, Curr Biol, 2022) After studying the factors related to the processing of novel stimuli by neurons in the brain , the researchers also measured the time scale of novelty detection
.

The novelty of an object is inseparable from the learning process, and new objects are gradually familiarized after repeated experience
.

This learning can have multiple timescales; it can be fast or slow, and it can be mixed with forgetting
.

To investigate the timescale of this learning, the behavioral task (Fig.
1) also included repeated novel images, which were generated before the experiment began, and then repeated in the experiment for up to 5 days
.

The researchers used these "repeating novel" pictures to measure how neurons responded at different stages of learning
.

 The researchers found that neuronal populations that responded positively to novelty had a progressive learning process for repeated novel images, characterized by rapid learning in each experiment, followed by massive forgetting the next day (Fig.
5).
)
.

It is worth noting that, in order to rule out sensory adaptation, the researchers used normalized neuronal activity relative to the same number of occurrences of completely novel and completely familiar objects
.

 On the first day that the repeated novel pictures appeared, neuronal excitation to them rapidly diminished, suggesting a learned behavior in the animals and their neurons
.

On the second day and after, at the beginning of the experiment, the neurons' responses to the same repeated novel pictures rebounded, but were still lower than their responses to the full novel pictures, suggesting that neurons retained some learning As a result there is also a great deal of forgetting
.

This is contrary to some theoretical expectations that neural memory is enhanced after a night's rest [21], and these findings are more similar to the "spontaneous recovery" observed in sensory, motor, and motivational learning [22, 23]
.

Figure 5 Neuron learning and forgetting of repeated novel pictures (Source: Zhang K, et al.
, Curr Biol, 2022) In life, some novel things only appear in the short term and then disappear, while others have a long-term importance
.

Does the brain contain neural networks with different learning rates or time scales to deal with the diversity in time scales of new things in life? The researchers also explored this, and they found that novel response neurons have different learning patterns
.

Even in the same brain region, some neurons are "fast learning, fast forgetting" while others are "slow learning, slow forgetting" (Fig.
6A)
.

 We used two indices for each neuron to quantify the neuron's learning within one day and forgetting the next day, and found that the two indices were significantly positively correlated in neurons that responded positively to novelty (Fig.
7C)
.

This suggests that neurons that learn more one day also tend to forget more the next day
.

Furthermore, although within brain regions these types of neurons are mixed (Fig.
6A), there are differences between regions, with regions that learn more in one day also tend to forget the next day more (Figure 6C)
.

This result supports the theory that there are different time scales of learning in the brain [24-26]
.

Figure 6 Different neurons and brain regions have diverse learning and forgetting (Source: Zhang K, et al.
, Curr Biol, 2022) Conclusions and discussions, inspiration and prospects There are different models for novelty detection, and their mechanisms can be related to Judging recency correlation can also be turned into a form of sensory surprise
.

In this article, the researchers found that novelty detection in the primate brain is closely related to both recency and surprise calculations
.

This suggests that novelty detection is associated with and influenced by multiple mechanisms, including those related to predictive coding
.

This provides directions for building novelty detection models in the brain in the future
.

However, its specific loop and computational details remain to be studied
.

 The diversity of neuronal learning and forgetting further suggests that there may be multiple systems in the brain that process novelty information
.

This diverse system can control the flow of information and adapt to new things with various timescales
.

A possible hypothesis is that the information of short-term relevant things mainly resides in the system of fast time scale, while the information of long-term important things will gradually transfer from the fast time scale system to the slow time scale system, and then for a long time.
Save
.

 In conclusion, this study discovers and explores the relationship between novelty detection and the computation of surprise and recency, as well as the diversity of novelty detection and memory-encoding mechanisms on timescales
.

Link to the original text: https://doi.
org/10.
1016/j.
cub.
2022.
03.
064 The first author, Kaining Zhang (left), the corresponding author Ilya Monosov (right) (photo courtesy of Ilya Monosov's lab) Students who are interested in neuroscience and computational neuroscience https://neuroscience.
wustl.
edu/people/ilya-monosov-phd/ 
.

Selected Previous Articles【1】Neurosci Bull︱Qian Lingjia's group reveals that homocysteine ​​affects cognitive function by regulating DNA methylation during chronic stress【2】Front Aging Neurosci︱Ma Tao's group reveals The mechanism of traditional Chinese medicine compound multi-channel and multi-target improving energy metabolism in Alzheimer's disease【3】Aging Cell︱Gao Xu's team found that good sleep quality can delay the accelerated aging caused by air pollution【4】Autophagy︱Shen Hanming's research group revealed autophagy The new mechanism of phagocytosis-related protein WIPI2 in regulating mitochondrial outer membrane protein degradation and mitophagy【5】Review by Neuron︱The team of Sheng Zuhang focuses on the importance of axonal mitochondria maintenance and energy supply in neurodegenerative diseases and repair after nerve injury Role 【6】Cell Death Dis︱Kong Hui et al.
Reveal the role of P2X7/NLRP3 inflammasome pathway in early diabetic retinopathy 【7】Sci Adv︱Liu Xingguo/Tian Mei team found new mitochondrial clearance in drug-induced Parkinson’s syndrome Mechanism【8】Front Aging Neurosci︱Bowel preparation can affect postoperative delirium by changing the composition of microflora【9】Mol Psychiatry︱Keqiang Ye’s group revealed that C/EBPβ/AEP signaling pathway mediates atherosclerosis and its induced Alzheimer’s disease Zheimer's disease【10】Neurosci Bull viewpoint article︱The mechanism of flexible allocation of limited working memory resources under multitasking High-quality scientific research training course recommendation【1】Patch clamp and optogenetic and calcium imaging technology seminar May 14-15 Tencent Conference [2] Scientific Research Skills︱The 4th Near Infrared Brain Function Data Analysis Class (Online: 2022.
4.
18~4.
30) References (swipe up and down to read) 1.
Jaegle, A.
, V.
Mehrpour, and N.
Rust , Visual novelty, curiosity, and intrinsic reward in machine learning and the brain.
Curr Opin Neurobiol, 2019.
58: p.
167-174.
2.
Zhang, K.
, CD Chen, and IE Monosov, Novelty, Salience,and Surprise Timing Are Signaled by Neurons in the Basal Forebrain.
Curr Biol, 2019.
29(1): p.
134-142 e3.
3.
Tiitinen, H.
, et al.
, Attentive novelty detection in humans is governed by pre- attentive sensory memory.
Nature, 1994.
372: p.
90.
4.
Tapper, AR and S.
Molas, Midbrain circuits of novelty processing.
Neurobiology of Learning and Memory, 2020: p.
107323.
5.
Ghazizadeh, A.
, W.
Griggs, and O.
Hikosaka, Ecological Origins of Object Salience: Reward, Uncertainty, Aversiveness, and Novelty.
Front Neurosci, 2016.
10: p.
378.
6.
Barto, A.
, M.
Mirolli, and G.
Baldassarre, Novelty or surprise? Front Psychol , 2013.
4: p.
907.
7.
Markou, M.
and S.
Singh, Novelty detection: a review - part 2: neural network based approaches.
Signal Processing, 2003.
83(12): p.
2499-2521.
8.
Pimentel, MAF, et al.
, A review of novelty detection.
Signal Processing, 2014.
99: p.
215-249.
9.
Ranganath, C.
and G.
Rainer, Neural mechanisms for detecting and remembering novel events.
Nat Rev Neurosci, 2003.
4(3): p.
193-202.
10.
Petrides, M.
, B.
Alivisatos, and S .
Frey, Differential activation of the human orbital, mid-ventrolateral, and mid-dorsolateral prefrontal cortex during the processing of visual stimuli.
Proc Natl Acad Sci USA, 2002.
99(8): p.
5649-54.
11.
Ogasawara, T.
, et al.
, A primate temporal cortex–zona incerta pathway for novelty seeking.
Nature neuroscience, 2022.
25(1): p.
50-60.
12.
Berlyne, DE, Novelty and curiosity as determinants of exploratory behaviour.
British Journal of Psychology , 1950.
41(1): p.
68.
13.
Brown, MW and JZ Xiang, Recognition memory: neuronal substrates of the judgement of prior occurrence.
Prog Neurobiol, 1998.
55(2): p.
149-89.
14.
Kumaran, D .
and EA Maguire,Which computational mechanisms operate in the hippocampus during novelty detection? Hippocampus, 2007.
17(9): p.
735-48.
15.
Xiang, J.
-Z.
and M.
Brown, Differential neuronal encoding of novelty, familiarity and recency in regions of the anterior temporal lobe.
Neuropharmacology, 1998.
37(4-5): p.
657-676.
16.
Vogels, R.
, Sources of adaptation of inferior temporal cortical responses.
Cortex, 2016.
80: p.
185-95.
17.
Bogacz, R.
, MW Brown, and C.
Giraud-Carrier, Model of co-operation between recency, familiarity and novelty neurons in the perirhinal cortex.
Neurocomputing, 2001.
38: p.
1121-1126.
18.
Hart, EW and J.
Jacoby.
Novelty, recency, and scarcity as predictors of perceived newness.
in Proceedings of the Annual Convention of the American Psychological Association.
1973.
American Psychological Association.
19.
Bogacz, R.
, MW Brown, and C.
Giraud-Carrier, Model of familiarity discrimination in the perirhinal cortex.
J Comput Neurosci, 2001.
10(1): p.
5-23.
20.
Bogacz, R.
and MW Brown, Comparison of computational models of familiarity discrimination in the perirhinal cortex.
Hippocampus, 2003.
13(4): p.
494-524.
21.
Stickgold, R.
, Sleep-dependent memory consolidation.
Nature, 2005.
437(7063): p.
1272-1278.
22.
Kording, KP, JB Tenenbaum, and R .
Shadmehr, The dynamics of memory as a consequence of optimal adaptation to a changing body.
Nature neuroscience, 2007.
10(6): p.
779-786.
23.
Smith, MA, A.
Ghazizadeh, and R.
Shadmehr, Interacting adaptive processes with different timescales underlie short-term motor learning.
PLoS biology, 2006.
4(6): p.
e179.
24.
Spitmaan, M.
, et al.
, Multiple timescales of neural dynamics and integration of task-relevant signals across cortex.
Proceedings of the National Academy of Sciences, 2020.
117(36): p.
22522-22531.
25.
Murray, JD, et al.
, A hierarchy of intrinsic timescales across primate cortex.
Nature neuroscience, 2014.
17(12): p.
1661-1663.
26.
Bromberg-Martin, ES, et al.
, Multiple timescales of memory in lateral habenula and dopamine neurons.
Neuron, 2010.
67(3): p.
499-510.
Plate making︱Sizhen Wang End of this paper