Current Issue Cover
跨模态脑图谱数据融合研究进展

罗娜, 宋明, 杨正宜, 蒋田仔(中国科学院自动化研究所, 北京 100190)

摘 要
脑图谱是研究脑结构和功能及脑疾病的基础,不同类型的脑图谱从不同角度提供了脑的组织模式或连接信息。随着图像采集和生物检测技术的发展,不同模态的脑影像和生物组学数据迅速增长。相较于单模态,多模态融合数据能够同时考察不同模态数据间的多元化信息,挖掘蕴含的未知新信息。因此,开展跨模态脑图谱数据融合研究有助于更全面地理解大脑的结构和功能,并辅助加深对脑发育、老化和病变机理的理解。本文根据参与融合的模态是否具有空间信息,将近年来有代表性的跨模态脑图谱融合技术分为脑影像融合和脑数据融合两大类。脑影像融合是指对宏观脑影像(磁共振等)和组织学脑影像(胞体染色、轴突染色等)等具有空间信息的数据进行融合,构建涵盖脑结构和功能信息的跨模态多尺度脑图谱,为研究宏观特征的介观机制以及介观特征的宏观表征提供了重要途径。脑数据融合是指对缺乏脑空间信息的生物大数据,包括基因组、电生理、认知和行为等,利用脑图谱提供精细空间信息,挖掘高维、异构生物大数据蕴含的信息,明确脑图谱的生理意义,并提升其应用价值。本文将针对这两类融合类型阐述国内外有代表性的研究进展,并对比国内外研究现状的差异。此外,为促进跨模态脑图谱数据融合领域的交流和发展,总结了部分有代表性的大样本公开数据集。最后讨论了当前该领域待解决的问题以及未来的发展趋势。
关键词
Survey on cross-modal fusion based on brain atlas data

Luo Na, Song Ming, Yang Zhengyi, Jiang Tianzi(Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China)

Abstract
The human brain is the most complex network system in the world. Different types of brain atlas provide brain tissue or connection information from different aspects. With the development of image acquisition and biological detection technology,researchers reveal that combing multi-modal brain data is able to provide more information through exploiting the rich multimodal information that exits,for example it can inform us about how brain structure shapes brain function,in which way they are impacted by psychopathology and which functional or structural aspects of physiology could drive human behavior and cognition. Through summarizing the representative multi-modal brain fusion technologies in recent years,we first divide them into two categories in this review,brain-imaging fusion and brain-data fusion. 1)Brain-imaging fusion refers to the fusion of data with fine spatial information such as brain images (magnetic resonance imaging,etc.) and histological images (cell body staining,axon staining,etc.),through which could help construct a multi-modal multi-scale brain map covering both structure and function information. For example, the Julich-Brain Atlas,a three-dimensional atlas containing cytoarchitectonic maps of cortical areas and subcortical nuclei,is built through the fusion of magnetic resonance imaging and histological slices,allowing comparison of functional activations, networks, genetic expression patterns, anatomical structures,and other data obtained across different studies in a common stereotaxic reference space. The multiscale brain atlas brings together data from these different levels of nervous system organization to form a better understanding of between-scale relationships of brain structure,function,and behavior in health and disease. 2)Brain-data fusion refers to applying brain atlas with spatial information to fuse biological big data without spatial information (genome,electrophysiology,cognition and behavior,etc.),through which could help make up the shortcomings from single modality and capitalize on joint information among modalities to dig out new information. For example,combing brain atlas with electroencephalogram data can simultaneously extract signals with high spatial and temporal resolution. For each fusion type discussed above,we elaborate the representative research progress at home and abroad in recent years,as well as their comparison in the current review. Then,in order to promote the development of this filed,we introduce several multi-modal datasets in the community,i.e.,UK Biobank, Human Connectome Project(HCP),Chinese Imaging Genetics(CHIMGEN),Adolescent Brain Cognitive Development(ABCD),Philadelphia Neurodevelopmental Cohort(PNC),IMAGEN,Pediatric Imaging, Neurocognition, and Genetics(PING),Autism Brain Imaging Data Exchang(ABIDE),Alzheimer's Disease Neuroimaging Initiative(ADNI),ADHD-200,PRIMatE Data Exchange (PRIME-DE),National Chimpanzee Brain Resource(NCBR),the Allen Atlas and EBRAINS Reference Atlas. Finally,we discuss the important problems to be solved and the future direction in this field. The common challenge of this filed is how to integrate the information from different scales and modalities to build a multi-modal and cross-scale brain atlas, and how to analyze the neural mechanism of major brain diseases based on the new brain atlas, and forming a new paradigm for brain disorder diagnosis and treatment paradigm at last. To achieve this major goal,1)the acquisition of high-quality big data is the important foundation, which rely on the development of acquisition equipments. 2)The high resolution of data (i.e.,cell and neuron level)raises large storaging and quick processing demands. 3)Developing new fusion methods based on artificial intelligence is also one of the urgent issues in multi-modal fusion research.
Keywords

订阅号|日报