Nat Neurosci︱VTA dopaminergic neurons are involved in encoding social prediction error and social reinforcement learning

Written by ︱ Nie Lutunan Editor ︱ Sizhen Wang Whether it is a rodent or a human, social behavior means cooperation among the same species, and cooperation is beneficial to avoid predators
.

Most social behavior is beneficial to survival, so social behavior is considered a rewarding behavior
.

Most of the previous studies on reward have used food as a reward signal [1,2], while fewer studies have used socialization as a reward signal
.

Much remains unknown about the mechanics of social reward
.

 Classical dopamine projections, ventral tegmental area (VTA) → nucleus accumbens (NAc) and VTA → prefrontal cortex (PFC), are recognized as the projections that regulate reward and motivation [3], then the regulation of social reward signals It is very likely related to the dopaminergic neurons of the VTA
.

Human imaging studies have shown that visual social stimuli increase human VTA excitability [4]
.

The study of odontomas found that the projection of social behavior excited VTA→NAc, social experience or social related cues will strengthen social motivation [4]
.

Previous research by Camilla Bellone's group also found that inhibiting VTA dopaminergic neurons can reduce the social behavior of odontomas [5]
.

 Although previous studies have shown that VTA dopaminergic is involved in the regulation of social behavior, there is great heterogeneity in the response of VTA dopaminergic neurons to social behavior [6,7], and previous behavioral tests cannot reflect VTA dopaminergic neurons.
Which stage of social behavior is regulated
.

VTA dopaminergic neurons encode reward prediction error (RPE), and through RPE modulates animal motivational behavior and then affects reinforcement learning (reinforcement learning), which is used to evaluate the reward to be obtained and used for decision-making [8,9]
.

So, how do VTA dopaminergic neurons encode social behavior? Does it also affect social learning through RPE? In December 2021, Professor Camilla Bellone's research group from the Neuroscience Research Center of the University of Geneva, Switzerland published an article entitled "VTA dopamine neuron activity encodes social interaction and promotes reinforcement learning through social prediction error" in Nature Neuroscience, finding that VTA dopamine Neurons encode different kinds of social behaviors, explaining why VTA dopaminergic neurons show great heterogeneity in the firing frequency of social behaviors; on this basis, it is found that VTA dopamine neurons encode social prediction error (social prediction error, SPE) and regulate social learning through SPE
.

In order to study the role of VTA dopaminergic neurons in social behaviors, we first explored whether VTA dopaminergic neurons are involved in encoding odontogenic social behaviors
.

By recording the firing frequency of dopaminergic neurons in the odontoid VTA by in vivo electrophysiological techniques, it was found that the firing frequency of VTA dopaminergic neurons increased during social activities in experimental mice compared to when they were alone (Fig.
1ac)
.

For further analysis of behavioral data, a two-dimensional coordinate system was established with the experimental mouse as the origin (Figure 1d).
There is a correlation with the firing frequency of VTA dopaminergic neurons (Fig.
1 f, h) (DeepLabCut: a deep learning-based label-free pose estimation for custom body positions [10])
.

These results suggest that VTA dopaminergic neuron activity is involved in encoding social behavior, and VTA dopaminergic neuron activity is enhanced in free social interaction behavior
.

Figure 1 Increased excitability of VTA dopaminergic neurons during social activities (Source: Clément Solié, et al.
, Nat Neurosci, 2021) Social behaviors were classified based on the relative position of the rats to their social counterparts
.

As shown in Figure 2a, the first two social positional relationships are classified as active social interaction, and the third is classified as passive social interaction
.

Among them, the frequency of active socialization is significantly greater than that of passive socialization (Figure 2b); the authors analyzed each social activity and dopaminergic neuron activity separately, and found that for the three social activities, 53%, 42% and 7% of dopaminergic neurons were Meta-response (Fig.
2d, g, j), different neurons are inhibited or activated in different social activities, and some neurons are involved in coding all social activities (Fig.
2 i, m)
.

From the above analysis, the authors concluded that dopaminergic neurons in different regions of the VTA are involved in encoding different kinds of social activities
.

Figure 2.
The excitability of VTA dopaminergic neurons is heterogeneous in social activities (Source: Clément Solié, et al.
, Nat Neurosci, 2021) Because some VTA dopaminergic neurons have increased firing frequency in response to novel stimulation
.

So, do VTA dopaminergic neurons respond differently to different social activities? Therefore, we next repeated three social trials using the same mouse (Fig.
3a, b) and recorded the firing frequency of VTA dopaminergic neurons during social activity
.

It was found that the firing frequency of VTA dopaminergic neurons gradually decreased as the number of social interactions increased during three repetitions of social interaction (Fig.
3 cf)
.

This suggests that VTA dopaminergic neurons exhibit adaptations in repetitive social activities
.

Figure 3.
VTA dopaminergic neurons exhibit adaptive excitability during multiple repetitions of social activity (Credit: Clément Solié, et al.
, Nat Neurosci, 2021) However, when the authors also classified social behaviors as previously , which is divided into active socialization and passive socialization.
After experiments, it was found that in repeated social behaviors, the neurons responsible for encoding different social behaviors can also show a strong increase in the frequency of firing (Fig.
4b,e,h)
.

Figure 4 Dopaminergic neurons in different regions of the VTA are involved in encoding different social types (Source: Clément Solié, et al.
, Nat Neurosci, 2021) So far, the article found that dopaminergic neurons in the VTA are involved in encoding social behavior, and the Dopaminergic neurons encode different types of social behaviors, which explains the heterogeneity of firing frequency of dopaminergic neurons in social behaviors
.

So, do VTA dopaminergic neurons play a role in social reinforcement learning
.

To investigate this issue, the authors designed the SIT test (social instrumental task)
.

In a two-chamber box with a partition in the middle, the left laboratory rat can press the plate to open the middle partition.
After the partition is opened, the laboratory mouse cannot enter the other side of the box, but can touch the other side of the small box.
rat (Fig.
5a)
.

After training (1-10 days), the experimental mice were able to establish the connection between the platen and socialization; this was followed by the experimental period (11-25 days) (Fig.
5b)
.

The results showed that in the two stages, the number of pressings of the experimental mice gradually increased (Fig.
5e, f), the number of successful social activities after pressing the plate also gradually increased (Fig.
5 g, h), and the speed of the experimental mice after pressing also increased.
increased (in order to reduce the number of social failures) (Fig.
5ik), and overall, the frequency of socialization via platens increased in the experimental phase compared to the training phase (Fig.
5l)
.

The above data show that, similar to ordinary reward behavior, social behavior also has the characteristics of reinforcement learning
.

Figure 5 Social task experiment (source: Clément Solié, et al.
, Nat Neurosci, 2021) During the SIT training phase and the experimental phase, the authors also recorded the firing frequency of VTA dopaminergic neurons in different phases
.

It was found that both the pressing operation and social behavior in the training stage can cause the VTA dopaminergic neuron firing frequency to increase, and the increase in the firing frequency is very similar (Figure 6 ae).
gradually increased, while the firing amplitude of VTA dopaminergic neurons activated during social activities gradually decreased (Fig.
6 fj); the authors also found that the firing frequency of VTA dopaminergic neurons activated when the experimental mice were pressed was related to the time of social activities.
showing a positive correlation
.

This suggests that VTA dopaminergic neuron activity encodes the timing of social contact primarily through phasing signals that emerge gradually during learning
.

Figure 6 VTA dopaminergic neurons involved in coding social task experiments (Source: Clément Solié, et al.
, Nat Neurosci, 2021) After two experimental stages of SIT, the authors designed a memory erasure stage, which The experimental mice can also perform the bar pressing, but after the bar pressing, the social contact is randomly opened, and the probability of opening is 50% (Fig.
7a,b)
.

VTA dopaminergic neuron activity was inhibited when experimental mice performed rod pressing but could not engage in social activities (Fig.
7c, d); VTA dopaminergic neurons could also fire when rod pressing and social contact were possible.
frequency increases (Fig.
7c,d)
.

As the experiment progressed, the firing activity of VTA dopaminergic neurons tended to stabilize when the mice performed pressure-sensitive manipulations (Fig.
7l)
.

These results suggest that VTA dopaminergic neurons encode social prediction error (SPE) and regulate social learning through SPE
.

Figure 7 VTA dopaminergic neurons are involved in encoding SPR (Source: Clément Solié, et al.
, Nat Neurosci, 2021) At the end of the article, the author injected virus (AAV-EF1α-DIO-eYFP) into the VTA brain region of DAT-Cre mice, Interventions in VTA brain DA neurons were performed using optogenetics (Fig.
8c,d) and photoinhibition during the experimental phase of the social task (Fig.
8a,b)
.

It was found that the number of presses in the mice was significantly reduced (Fig.
8 ei)
.

This suggests that inhibition of VTA dopaminergic neurons is sufficient to inhibit social reinforcement learning in mice
.

Figure 8 Inhibition of VTA dopaminergic neurons reduces the frequency of social activities in mice (Source: Clément Solié, et al.
, Nat Neurosci, 2021) Conclusion and discussion, inspiration and prospect In conclusion, social behavior is a motivation Behavior, and motivation can promote learning, so how does social behavior promote learning? That is, how is reinforcement learning of social behavior encoded? This is the scientific question posed by the article
.

The article first explains that VTA dopaminergic neurons are involved in encoding social behavior, and then the authors set up a social task experiment to prove that VTA dopaminergic neurons encode social prediction error (SPE) to promote social reinforcement learning
.

 The article studies socialization as a rewarding behavior, showing that socialization as a rewarding behavior is consistent with previous research on reward theory
.

In traditional reward behaviors, such as feeding studies, VTA dopaminergic neurons encode food reward cues, not feeding behavior itself [3,10], while this study found that VTA dopaminergic neurons temporally respond to social contact more sensitive
.

In the coding of social behavior, VTA dopaminergic neurons show different characteristics, which may be due to the more complex regulatory mechanism of social behavior
.

 Socialization is undoubtedly more complex than feeding behavior
.

Therefore, the article classifies social behaviors into active socialization and passive socialization.
The definitions of these two social behaviors are not perfect, but the article can correspond to the discharge signals of VTA dopaminergic neurons and the classified social behaviors
.

However, it does not explain why different VTA dopaminergic neurons respond to different social behaviors, which may be the content of the authors' later research
.

Link to the original text: https:// Selected articles from previous issues [1] Nature︱ New discovery! Inflammatory lymphocytes or new targets mediating CNS inflammation? 【2】Neurosci Bull︱Hu Bo’s research group reveals that neurons in the deep cerebellar nucleus projecting to the ventromedial thalamus are specifically involved in the regulation of combined sensory-motor learning behavior 【3】Nat Neurosci︱Wu Longjun’s research group reveals key pathological proteins in ALS A new ligand for TDP-43——TREM2【4】Cereb Cortex︱Luo Yuejia’s team revealed the cognitive control mechanism under uncertain expectations of anxious individuals【5】PNAS︱Xiao Bo’s research group revealed a new mechanism for regulating the development of myelin in the central nervous system Mechanism 【6】Science︱Neuronal mechanism controlling behavioral motivation 【7】Sci Adv︱Chen Zhong’s team proposes a new idea for epilepsy drug treatment: electro-responsive polydopamine nano-drug delivery system 【8】Nat Commun︱Xing Dajun’s research group reveals brightness The brain mechanism of information processing: the visual system's strategy for encoding the brightness of the surface of objects [9] STAR Protocols︱ Zhao Min's research group proposed a new plan for physical intervention in the psychological craving of methamphetamine use disorder [10] Cereb Cortex︱ Li Qiang / Mingdong research group Recommendations for high-quality scientific research training courses to jointly reveal key white matter structural lesions in patients with mild cognitive impairment [1] Seminar on Single-Cell Sequencing and Spatial Transcriptomic Data Analysis References (swipe up and down to view) 1, Berridge, KC & Kringelbach, ML Afective neuroscience of pleasure: reward in humans and animals.
Psychopharmacology 199, 457–480 (2008).
2, Alhadef, AL et al.
Natural and drug rewards engage distinct pathways that converge on coordinated hypothalamic and reward circuits.
Neuron 103, 891– 908 (2019).
3, Gunaydin, LA