Nat Commun︱New progress in monitoring technology of cerebral cortex activity in free state of small animals: intelligent optical fiber two-photon microscope

Written by ︱ Li Dawei in charge of editing ︱ Wang Sizhen When creatures communicate with each other, are attracted by delicious food, walk and jump, how does the brain work? Opening a window on the head of a small animal model and using a microscope with subcellular resolution to observe neuronal activity in real time may help us understand this correlation
.

Two-photon fluorescence microscope observation of GCaMP6 (a genetically encoded calcium indicator, GECI) transgenic mouse cerebral cortex neurons is an effective means to study this correlation [1]
.

To perform these kinds of experiments, small animal models often need to be immobilized under a microscope lens, and this immobilization and the physical stress it entails inevitably affects neuronal activity
.

At the same time, there are many studies in the field of neuroscience that hope to observe freely moving animals, such as observing the activity of the cerebral cortex during animal behaviors such as climbing and social interaction [2], so it is necessary to "wear" a small animal A microscope that "follows the animal" on the head
.

At present, several R&D teams have successively developed miniaturized two-photon endoscopic microscopes [3-5], which provide the possibility to study the correlation between animal behavior and neuronal activity in the cerebral cortex in a free state [6]
.

Among them, the Johns Hopkins University team (https://bit.
bme.
jhu.
edu) developed a two-photon fiber microscope with simple structure, small size (2-3mm), ultra-light weight (0.
6 g), and high stability [ 7, 8]
.

However, the extreme pursuit of its structure and weight limits the choice of opto-mechanical devices, which affects the imaging frame rate (<5 fps), making it difficult to track neurons and dendrites at high speed [9]
.

On March 22, 2022, Professor Li Xingde's research group from Johns Hopkins University, Professor Lu Hui's research group from George Washington University, and Dr.
Li Mingjun from Corning Incorporated jointly published a paper entitled "Nature Communications".
The research paper "Deep-learning two-photon fiberscopy for video-rate brain imaging in freely-behaving mice" proposes a high-speed fiber-optic two-photon microscope enhanced by artificial intelligence, which can be used for brain imaging of small animals in a freely moving state Real-time observation of cortical activity
.

Guan Honghua and Li Dawei are the co-first authors of the paper, and Professor Li Xingde is the corresponding author of the paper
.

In this research, the research group developed a high-speed sparse sampling fiber-scanning two-photon microscope, the imaging frame rate was increased by more than 10 times to 26fps, and the deep learning architecture was used for image enhancement to provide better image quality
.

This work provides a protocol for monitoring neuronal activity under prolonged free movement in small animal models
.

The core components of the ultralight two-photon fiber microscope developed by the Johns Hopkins team include: a single double-clad fiber for emitting excitation signals and receiving fluorescence signals, a single double-clad fiber for generating scanning patterns Miniature piezo tube driver and an achromatic micro-objective
.

When observing the cerebral cortex of free animals, a fiber-optic microscope was first attached to the animal's head, allowing the animal to move with the microscope
.

When the animal is moving, the femtosecond laser is guided through the optical fiber and focused on the target brain area through the objective lens, and the entire field of view (FOV) is scanned under the control of the piezoelectric ceramic driver.
The scanned area produces fluorescence under the excitation of the laser.
Collected by the objective lens, it is transmitted to the photoelectric conversion module through the inner cladding of the fiber, and then reconstructed into an image (Fig.
1a)
.

Figure 1 (a) Use of fiber-optic two-photon microscope; (b) high scan density scan and reconstructed image; (c) sparse scan, reconstructed image and image enhanced by deep neural network
.

  The frame rate of an image is determined by the scan speed (line scan rate) and the scan density (line density)
.

In order to obtain the image quality that meets the observation requirements, the project team used a low scanning speed and high scanning density fiber microscope in the previous work for signal acquisition [4] (Fig.
1b)
.

In order to improve the frame rate, the project team redesigned the high-speed sparse scanning fiber microscope [10]
.

However, the increase in scan speed reduces the pixel dwell time and thus sacrifices signal-to-noise ratio (SNR), while the lower scan density sacrifices imaging resolution
.

The project team uses deep neural network (DNN) to improve the signal-to-noise ratio and restore resolution of the image, which can enhance the low-signal-to-noise ratio and low-resolution images collected at high frame rates into high-signal-to-noise ratio and high-resolution images.
image
.

The network can be understood as a combination of: (1) a denoising filter customized for the two-photon system; (2) a high- and low-resolution neural structure correspondence established for the mouse cerebral cortex
.

In addition, the DNN-enhanced images are less affected by motion artifacts, and the image quality can even be better than the image quality acquired under the condition of low scan speed and high scan density (Fig.
1c)
.

 The project team used supervised learning to train the DNN
.

The supervised learning training scheme requires pairs of images (including input/input and ground truth/ground truth) as a training set
.

The advantage of this scheme is that it can reduce the number of images in the training set and achieve pixel-level accuracy
.

The weakness of this scheme is that DNN performance is highly dependent on the quality of the training set, especially the quality of the ground truth
.

In other words, in order to enhance the image acquired under high-speed (low signal-to-noise ratio) sparse scanning (low-resolution) to a high-signal-to-noise ratio high-resolution image through DNN, DNN needs to be composed of pre-enhancement (as input) and Augmented (as ground truth) images for training (Fig.
2a)
.

However, the ground truth of images acquired by high-speed sparse-fiber microscopy on free in vivo is not available (otherwise this established image enhancement scheme would be fine)
.

To train the neural network without the final ground truth, we employ a two-step stage to train the neural network (Fig.
2c)
.

The first stage uses ex vivo imaging data to train a neural network 1 (DNN-1) to learn the noise distribution of a fiber-optic two-photon microscope system, giving it the ability to improve the signal-to-noise ratio
.

The second stage focuses on learning the correspondence between low-resolution neural structures and high-resolution neural structures: collecting high-density living data, and using the living data to synthesize a training set (1) Use DNN-1 to denoise the living data Augmentation as ground truth; (2) Simulate sparse sampling scan pattern to downsample this data as training input
.

This training set is then used to train a neural network 2 (DNN-2) capable of improving both signal-to-noise ratio and resolution
.

The trained DNN-2 can enhance the free-living data collected at high frame rate into high-resolution data with high signal-to-noise ratio
.

Figure 2 (a) training of neural network; (b) image enhancement using trained neural network; (c) training process of neural network
.

(Image source: Guan H H, et al.
, Nat Commun, 2022) Conclusions and discussions, inspiration and prospects Under the given imaging conditions, improving imaging quality is the eternal pursuit of optical imaging system design
.

Improving the optical design and exploiting the potential of the hardware are the main considerations for the optical imaging system
.

The rapid development of advanced imaging algorithms, such as deep learning algorithms, provides more ideas for improving the performance of optical imaging systems
.

This paper describes an ultralight two-photon fiber-optic microscope system that utilizes neural networks to achieve significant improvements in imaging quality with minimal improvements to the system, providing high-speed (~26 fps) observation of cerebral cortical activity in free small animals.
solution
.

In the process of training the neural network, the construction of the training set is one of the core issues.
This paper provides a scheme to construct the training set in the absence of reference standards, thereby realizing the effective training of the neural network
.

The evaluation of image enhancement effect in this paper relies on peak signal-to-noise ratio, structural similarity and human eye evaluation, and more effective evaluation criteria need to be developed
.

In addition, although this project has traversed all the data in the test set, it remains to be further explored whether corner cases will produce artifacts
.

Link to the original text: https://doi.
org/10.
1038/s41467-022-29236-1 Corresponding author Prof.
Li Xingde (first from right), co-first author Guan Honghua (second from right), and co-first author Li Dawei (right 3) (Photo provided from: Johns Hopkins University Li Xingde Laboratory) Selected articles from previous issues [1] J Neuroinflammation︱An Jing/Luan Guoming team collaborated to reveal a new pathogenesis of Rasmussen encephalitis [2] JCI︱Zhang Liang/Wang Zhanxiang The team discovered that the autocrine pathway regulates oligodendrocyte differentiation and promotes remyelination【3】Front Cell Neurosci Review︱The role and research progress of microglial membrane proteins or receptors in neuroinflammation and degeneration【4】Nat Biomed Eng︱Using infrared light through the brain to modulate deep brain neural activity【5】Review of Neurosci Bull︱Research progress, problems and prospects of humoral biomarkers in Alzheimer's disease【6】Current Biology︱Chen Zhong's team in histamine New achievements in regulating feeding mechanism: H2 receptor-dependent medial septal nucleus histaminergic circuit【7】Nat Commun︱Guo Ming’s team discovered a new mechanism of mitochondrial fission and a new target for the prevention and treatment of Parkinson’s disease【8】Front Cell Neurosci︱ Shi Peng/Liu Zhen's group collaborated to reveal the shared molecular mechanism of sensorineural hearing loss caused by multiple factors【9】Cell Death Dis︱Li Xian's group revealed the role of ferroptosis in oligodendrocyte precursor cells in white matter damage after intracerebral hemorrhage 【10】Front Mol Neurosci︱Gao Shangbang's research group analyzes the composition and molecular mechanism of motor neuron oscillators.
Recommended high-quality scientific research training courses 【1】Symposium on patch clamp and optogenetics and calcium imaging technology May 14-15 Tencent meeting [2] Scientific research skills︱The 4th near-infrared brain function data analysis class (online: 2022.
4.
18~4.
30) [3] Scientific research skills︱Introduction to magnetic resonance brain network analysis class (online: 2022.
4.
6~4.
16) Reference Literature (swipe up and down to read) 1.
JND Kerr and W.
Denk, "Imaging in vivo: watching the brain in action," Nature Reviews Neuroscience 9(3):195-205 (2008).
2.
P.
Jercog, T.
Rogerson, and MJ Schnitzer, "Large-Scale Fluorescence Calcium-Imaging Methods for Studies of Long-Term Memory in Behaving Mammals," Cold Spring Harb Perspect Biol 8(5) (2016).
3.
YY Zhang , ML Akins, K.
Murari, JF Xi, M.
-J.
Li, K.
Luby-Phelps, M.
Mahendroo, and XD Li, "A compact fiber-optic SHG scanning endomicroscope and its application to visualize cervical remodeling during pregnancy ," Proceedings of the National Academy of Sciences 109(32):12878-12883 (2012).
4.
WX Liang, G.
Hall, B.
Messerschmidt, M.
-J.
Li, and XD Li, "Nonlinear optical endomicroscopy for label-free functional histology in vivo," Light: Science & Applications 6(-):e17082 (2017).
5.
A.
Lombardini, V.
Mytskaniuk, S.
Sivankutty, ER Andresen, X.
Chen, J.
Wenger, M .
Fabert, N.
Joly, F.
Louradour, A.
Kudlinski, and H.
Rigneault, "High-resolution multimodal flexible coherent Raman endoscope," Light: Science & Applications 7(1):10 (2018).
6.
W.
Zong, R.
Wu, M.
Li, Y.
Hu, Y.
Li, J.
Li, H.
Rong, H.
Wu, Y.
Xu, Y.
Lu, H.
Jia, M.
Fan, Z.
Zhou, Y.
Zhang, A.
Wang, L.
Chen, and H.
Cheng, " Fast high-resolution miniature two-photon microscopy for brain imaging in freely behaving mice," Nature Methods 14(7):713-719 (2017).
7.
H.
Guan, W.
Liang, A.
Li, Y.
-TA Gau, D.
Chen, M.
-J.
Li, DE Bergles, and XD Li, "Multicolor fiber-optic two-photon endomicroscopy for brain imaging," Optics Letters 46(5):1093-1096 (2021).
8.
A.
Li, H.
Guan, HC Park, Y.
Yue, D.
Chen, W.
Liang, M.
-J.
Li, H.
Lu, and XD Li, "Twist-free ultralight two-photon fiberscope enabling neuroimaging on freely rotating/walking mice," Optica 8(6):870-879 (2021).
9.
C.
C.
Chen, J.
Lu, and Y.
Zuo, "Spatiotemporal dynamics of dendritic spines in the living brain," Front Neuroanat 8(28 (2014).
10.
H.
-C.
Park, H.
Guan, A.
Li , Y.
Yue, M.
-J.
Li, H.
Lu, and XD Li, "High-speed fiber-optic scanning nonlinear endomicroscopy for imaging neuron dynamics in vivo," Optics Letters 45(13):3605-3608 (2020 ).
Plate making︱Wang Sizhen End of this article