浏览全部资源
扫码关注微信
Received：11 June 2020，
Revised：2020-7-3，
Accepted：10 July 2020，
Published：16 October 2020
移动端阅览
精神分裂症是一种大脑存在结构和功能异常的严重精神疾病，目前尚无十分有效的诊疗手段。许多研究者尝试采用基于磁共振影像的机器学习方法辅助诊断精神分裂症。深度学习由于其强大的特征表示能力，在医学影像等领域得到了广泛的应用，表现出比传统机器学习方法更优异的性能。目前已有论文对深度学习在医学影像领域内的应用进行了详细的总结和分析，却很少有论文对其在具体某一疾病（例如精神分裂症）诊断中的应用进行系统的梳理和总结。因此，本文主要关注深度学习在基于磁共振影像的精神分裂症诊断中的应用。首先介绍了基于磁共振影像的精神分裂症诊断中常用的几种深度学习模型；其次按照数据的不同模态将相关研究划分为：基于结构磁共振成像的方法、基于功能磁共振成像的方法以及基于多模态数据的方法，并对各类方法的具体应用进行了总结；最后针对目前的研究现状，进一步分析和总结出样本量偏小、样本类别不均衡、模型可解释性低、多模态分析中缺失模态数据以及缺少有效融合策略等，是目前该领域研究中存在的主要问题。多中心及纵向数据分析以及针对不同症状亚型的个体化精准诊断模型的构建，是未来该领域研究发展的趋势。本文系统地总结了深度学习在基于磁共振影像的精神分裂症诊断中的应用，分析了该领域面临的问题和下一步的研究方向，对该领域的研究具有一定的参考和指导意义。
Schizophrenia is a severe psychiatric disorder with abnormalities in brain structure and function. The early diagnosis and timely intervention can significantly improve the quality of life. However
no effective methods are available for the diagnosis and treatment of schizophrenia yet due to the complex pathology of this disease. Recently
deep learning methods have been widely used in medical imaging and have shown their great feature representation capability. The performance of deep learning methods has been demonstrated to be superior to that of traditional machine-learning methods. Researchers have attempted to use deep learning to assist the diagnosis of schizophrenia on the basis of magnetic resonance imaging (MRI) data. However
few surveys have systematically summarized the application of deep learning in the MRI-based diagnosis of schizophrenia. Therefore
MRI-based deep learning methods for the classification of schizophrenia patients and healthy controls were reviewed in this paper. First
commonly used deep learning models in the field of MRI-based diagnosis of schizophrenia were introduced. These models were classified into five categories: feed-forward neural network
recurrent neural network
convolutional neural network
unsupervised feature-learning models
and other deep models. The basic idea of the model in each category was introduced. Specifically
perceptron and its deeper expanding model
called multilayer perceptron
were briefly introduced in the review of feed-forward neural network. The most famous recurrent neural network model
named long short-term memory
was described. For the convolutional neural network
the common model components were reviewed. Unsupervised feature-learning models were further divided into two subcategories: stacked auto-encoder and deep belief network. The basic components of the model in each subcategory were described. We also reviewed two other deep models that cannot be classified into the categories described above
namely
capsule network and multi-grained cascade forest. Examples of deep learning studies used in the diagnosis of schizophrenia in each category were included to show the effectiveness of these models. Second
the deep learning methods for schizophrenia classification were divided on the basis of the data modality of MRI and reviewed in accordance with the following categories: structural MRI (sMRI)-
functional MRI (fMRI)-
and multimodal-based methods. Papers using MRI-based deep learning models for schizophrenia diagnosis were also summarized in a table. For sMRI-based methods
studies using 2D and 3D models and different methods for solving the requirement of large sample size were discussed. Resting-state fMRI- and task-fMRI-based methods were included in the fMRI-based category. In addition to different methods for solving limited sample size
studies using regional- or voxel-based functional activity features and functional connectivity-based features were introduced in this category. Some multimodal-based methods
which show better performance than most of unimodal-based approaches
were also described. Lastly
on the basis of existing studies
we concluded the main challenges and obstacles in the real application of these models in schizophrenia diagnosis. The first one refers to sample-related problems
including limited sample size and imbalanced sample class distribution. The second one is the lack of interpretability in deep learning models
which is a critical limitation for the application of deep learning models in real clinical condition. The last one is related to problems in multimodality analyses. Missing modality and the lack of effective multimodal feature fusion strategies would be the key problems to be solved. Future research directions lie in multisite and longitudinal data analyses and the construction of personalized precise models for the diagnosis of different subtypes of schizophrenia. Multisite data analyses would lead to a robust model and enable effective evaluation for deep learning models before the translation to clinical applications. Longitudinal data would provide the dynamically developmental patterns of the disease
leading to enhanced model performance. Personalized precise models are urgently needed to improve the accuracy of computer-aided diagnostic systems in consideration of the broad symptoms of schizophrenia and existing biased methods. In summary
this article reviewed the application of MRI-based deep learning models in the diagnosis of schizophrenia and provided guidance for future studies in this field.
Aguiar-Pulido V, Seoane J A, Rabunal J R, Dorado J, Pazos A and Munteanu C R. 2010. Machine learning techniques for single nucleotide polymorphism-disease classification models in schizophrenia. Molecules, 15(7):4875-4889[DOI:10.3390/molecules-15074875]
Anderson D and McNeill G. 1992. Artificial Neural Networks Technology. New York: Kaman Sciences Corporation: 1-83
Cai L, Wang Z Y, Gao H Y, Shen D G and Ji S W. 2018. Deep adversarial learning for multi-modality missing data completion//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. London: ACM: 1158-1166[ DOI: 10.1145/3219819.3219963 http://dx.doi.org/10.1145/3219819.3219963 ]
Campese S, Lauriola I, Scarpazza C, Sartori G and Aiolli F. 2019. Psychiatric disorders classification with 3D convolutional neural networks//Oneto L, Navarin N, Sperduti A and Anguita D, eds. Recent Advances in Big Data and Deep Learning. Italy: Springer: 48-57[ DOI: 10.1007/978-3-030-16841-4_6 http://dx.doi.org/10.1007/978-3-030-16841-4_6 ]
Correll C U, Cañas F, Larmo I, Levy P, Montes J M,Fagiolini A, Papageorgiou G, Rossi A, Sturlason R and Zink M. 2011. Individualizing antipsychotic treatment selection in schizophrenia:characteristics of empirically derived patient subgroups. European Psychiatry, 26(1S):3-16[DOI:10.1016/S0924-9338(11)71709-6]
Dakka J, Bashivan P, Gheiratmand M, Rish I, Jha S and Greiner R. 2017. Learning neural markers of schizophrenia disorder using recurrent neural networks[EB/OL].[2020-05-15] . https: //arxiv.org/pdf/1712.00512.pdf https: //arxiv.org/pdf/1712.00512.pdf
Doshi-Velez F and Kim B. 2017. Towards a rigorous science of interpretable machine learning[EB/OL].[2020-05-15] . https://arxiv.org/pdf/1702.08608.pdf https://arxiv.org/pdf/1702.08608.pdf
Feczko E, Miranda-Dominguez O, Marr M, Graham A M, Nigg J T and Fair D A. 2019. The heterogeneity problem:approaches to identify psychiatric subtypes. Trends in Cognitive Sciences, 23(7):584-601[DOI:10.1016/j.tics.2019.03.009]
Fischl B. 2012. FreeSurfer. NeuroImage, 62(2):774-781[DOI:10.1016/j.neuroimage.2012.01.021]
Fornito A and Bullmore E T. 2015. Reconciling abnormalities of brain network structure and function in schizophrenia. Current Opinion in Neurobiology, 30:44-50[DOI:10.1016/j.conb.2014.08.006]
Friston K J. 2011. Functional and effective connectivity:a review. Brain Connectivity, 1(1):13-36[DOI:10.1089/brain.2011.0008]
GBD 2017 Disease and Injury Incidence and Prevalence Collaborators. 2018. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990-2017: a systematic analysis for the global burden of disease study 2017. The Lancet, 392(10159): 1789-1858[ DOI: 10.1016/S0140-6736(18)32279-7 http://dx.doi.org/10.1016/S0140-6736(18)32279-7 ]
Geng X J, Song P L, Chen E H Y, Wang Y P and Lin J X. 2020. Neural effect induced by exercise intervention can be categorized by altered functional connectivity in early psychotic patients//Proceedings of the SPIE 11313, Medical Imaging 2020: Image Processing. Houston: SPIE: #113130U[ DOI: 10.1117/12.2549908 http://dx.doi.org/10.1117/12.2549908 ]
Ghorbani A, Abid A and Zou J. 2017. Interpretation of neural networks is fragile[EB/OL].[2020-05-15] . https://arxiv.org/pdf/1710.10547.pdf https://arxiv.org/pdf/1710.10547.pdf
Gunning D, Stefik M, Choi J, Miller T, Stumpf S and Yang G Z. 2019. XAI-Explainable artificial intelligence. Science Robotics, 4(37):#7120[DOI:10.1126/scirobotics.aay7120]
Han S Q, Huang W, Zhang Y, Zhao J P and Chen H F. 2017. Recognition of early-onset schizophrenia using deep-learning method. Applied Informatics, 4(1):#16[DOI:10.1186/s40535-017-0044-3]
Hatcher W G and Yu W. 2018. A survey of deep learning:platforms, applications and emerging research trends. IEEE Access, 6:24411-24432[DOI:10.1109/ACCESS.2018.2830661]
Hinton G E, Krizhevsky A and Wang S D. 2011. Transforming auto-encoders//Proceedings of the 21st International Conference on Artificial Neural Networks. Finland: Springer: 44-51[ DOI: 10.1007/978-3-642-21735-7_6 http://dx.doi.org/10.1007/978-3-642-21735-7_6 ]
Hochreiter S and Schmidhuber J. 1997. Long short-term memory. Neural Computation, 9(8):1735-1780[DOI:10.1162/neco.1997.9.8.1735]
Hu M J, Sim K, Zhou J H, Jiang X D and Guan C T. 2020. Brain MRI-based 3D convolutional neural networks for classification of schizophrenia and controls[EB/OL].[2020-05-15] . https://arxiv.org/pdf/2003.08818.pdf https://arxiv.org/pdf/2003.08818.pdf
Kang B Y, Xie S N, Rohrbach M, Yan Z C, Gordo A, Feng J S and Kalantidis Y. 2019. Decoupling representation and classifier for long-tailed recognition[EB/OL].[2020-05-15] . https://arxiv.org/pdf/1910.09217.pdf https://arxiv.org/pdf/1910.09217.pdf
Kim J, Calhoun V D, Shim E and Lee J H. 2016. Deep neural network with weight sparsity control and pre-training extracts hierarchical features and enhances classification performance:evidence from whole-brain resting-state functional connectivity patterns of schizophrenia. NeuroImage, 124:127-146[DOI:10.1016/j.neuroimage.2015.05.018]
Latha M and Kavitha G. 2019. Detection of schizophrenia in brain MR images based on segmented ventricle region and deep belief networks. Neural Computing and Applications, 31(9):5195-5206[DOI:10.1007/s00521-018-3360-1]
LeCun Y, Bengio Y and Hinton G. 2015. Deep learning. Nature, 521(7553):436-444[DOI:10.1038/nature14539]
Li A, Zalesky A, Yue W H, Howes O, Yan H, Liu Y, Fan L Z, Whitaker K J, Xu K B, Rao G X, Li J, Liu S, Wang M, Sun Y Q, Song M, Li P, Chen J, Chen Y C, Wang H N, Liu W M, Li Z G, Yang Y F, Guo H, Wan P, Lv L X, Lu L, Yan J, Song Y Q, Wang H L, Zhang H X, Wu H W, Ning Y P, Du Y H, Cheng Y Q, Xu J, Xu X F, Zhang D, Wang X Q, Jiang T Z and Liu B. 2020a. A neuroimaging biomarker for striatal dysfunction in schizophrenia. Nature Medicine, 26(4):558-565[DOI:10.1038/s41591-020-0793-8]
Li G, Han D P, Wang C, Hu W X, Calhoun V D and Wang Y P. 2020b. Application of deep canonically correlated sparse autoencoder for the classification of schizophrenia. Computer Methods and Programs in Biomedicine, 183:#105073[DOI:10.1016/j.cmpb.2019.105073]
Lieberman J A, Small S A and Girgis R R. 2019. Early detection and preventive intervention in schizophrenia:from fantasy to reality. American Journal of Psychiatry, 176(10):794-810[DOI:10.1176/appi.ajp.2019.19080865]
Lipton Z C, Berkowitz J and Elkan C. 2015. A critical review of recurrent neural networks for sequence learning[EB/OL].[2020-05-15] . https://arxiv.org/pdf/1506.00019.pdf https://arxiv.org/pdf/1506.00019.pdf
Litjens G, Kooi T, Bejnordi B E, Setio A A A, Ciompi F, Ghafoorian M, van der Laak J A W M, van Ginneken B and Sánchez C I. 2017. A survey on deep learning in medical image analysis. Medical Image Analysis, 42:60-88[DOI:10.1016/j.media.2017.07.005]
Liu F Y, Wang S H and Zhang Y D. 2018. Survey on deep belief network model and its applications. Computer Engineering and Applications, 54(1):11-18, 47 [DOI:10.3778/j.issn.1002-8331.1711-0028]
刘方园, 王水花, 张煜东. 2018.深度置信网络模型及应用研究综述.计算机工程与应用, 54(1):11-18, 47[DOI:10.3778/j.issn.1002-8331.1711-0028]
Liu M H, Cheng D N, Wang K D, Wang Y P and the Alzheimer's Disease Neuroimaging Initiative. 2018. Multi-modality cascaded convolutional neural networks for Alzheimer's disease diagnosis. Neuroinformatics, 16(3/4): 295-308[ DOI: 10.1007/s12021-018-9370-4 http://dx.doi.org/10.1007/s12021-018-9370-4 ]
Matsubara T, Tashiro T and Uehara K. 2019. Deep neural generative model of functional MRI images for psychiatric disorder diagnosis. IEEE Transactions on Biomedical Engineering, 66(10):2768-2779[DOI:10.1109/TBME.2019.2895663]
Niu Y W, Lin Q H, Qiu Y, Kuang L D and Calhoun V D. 2019. Sample augmentation for classification of schizophrenia patients and healthy controls using ICA of fMRI data and convolutional neural networks//Proceedings of the 10th International Conference on Intelligent Control and Information Processing. Morocco: IEEE: 297-302[ DOI: 10.1109/ICICIP47338.2019.9012169 http://dx.doi.org/10.1109/ICICIP47338.2019.9012169 ]
Oh J, Oh B L, Lee K U, Chae J H and Yun K. 2020. Identifying schizophrenia using structural MRI with a deep learning algorithm. Frontiers in Psychiatry, 11:#16[DOI:10.3389/fpsyt.2020.00016]
Oh K, Kim W, Shen G F, Piao Y H, Kang N I, Oh I S and Chung Y C. 2019. Classification of schizophrenia and normal controls using 3D convolutional neural network and outcome visualization. Schizophrenia Research, 212: 186-195[ DOI: 10.1016/j.schres.2019.07.034 http://dx.doi.org/10.1016/j.schres.2019.07.034 ]
Patel P, Aggarwal P and Gupta A. 2016. Classification of Schizophrenia versus normal subjects using deep learning//Proceedings of the 10th Indian Conference on Computer Vision, Graphics and Image Processing. India: ACM: 1-6[ DOI: 10.1145/3009977.3010050 http://dx.doi.org/10.1145/3009977.3010050 ]
Pinaya W H L, Gadelha A, Doyle O M, Noto C, Zugman A, Cordeiro Q, Jackowski A P, Bressan R A and Sato J R. 2016. Using deep belief network modelling to characterize differences in brain morphometry in schizophrenia. Scientific Reports, 6(1): #38897[ DOI: 10.1038/srep38897 http://dx.doi.org/10.1038/srep38897 ]
Pinaya W H L, Mechelli A and Sato J R. 2019. Using deep autoencoders to identify abnormal brain structural patterns in neuropsychiatric disorders:a large-scale multi-sample study. Human Brain Mapping, 40(3):944-954[DOI:10.1002/hbm.24423]
Qi J and Tejedor J. 2016. Deep multi-view representation learning for multi-modal features of the schizophrenia and schizo-affective disorder//Proceedings of 2016 IEEE International Conference on Acoustics, Speech, and Signal Processing. Shanghai: IEEE: 952-956[ DOI: 10.1109/ICASSP.2016.7471816 http://dx.doi.org/10.1109/ICASSP.2016.7471816 ]
Qiu Y, Lin Q H, Kuang L D, Zhao W D, Gong X F, Cong F Y and Calhoun V D. 2019. Classification of schizophrenia patients and healthy controls using ICA of complex-valued fMRI data and convolutional neural networks//Proceedings of the 16th International Symposium on Neural Networks. Russia: Springer: 540-547[ DOI: 10.1007/978-3-030-22808-8_53 http://dx.doi.org/10.1007/978-3-030-22808-8_53 ]
Qureshi M N I, Oh J and Lee B. 2019.3D-CNN based discrimination of schizophrenia using resting-state fMRI. Artificial Intelligence in Medicine, 98:10-17[DOI:10.1016/j.artmed.2019.06.003]
Ren S Q, He K M, Girshick R and Sun J. 2017. Faster R-CNN:towards real-timeobject detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6):1137-1149[DOI:10.1109/TPAMI.2016.2577031]
Sabour S, Frosst N and Hinton G E. 2017. Dynamic routing between capsules//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: Curran Associates Inc.: 3856-3866
Singh A, Sengupta S and Lakshminarayanan V. 2020. Explainable deep learning models in medical image analysis[EB/OL].[2020-05-15] . https://arxiv.org/pdf/2005.13799.pdf https://arxiv.org/pdf/2005.13799.pdf
Srinivasagopalan S, Barry J, Gurupur V and Thankachan S. 2019. A deep learning approach for diagnosing schizophrenic patients. Journal of Experimental and Theoretical Artificial Intelligence, 31(6):803-816[DOI:10.1080/0952813X.2018.1563636]
Ulloa A, Plis S, Erhardt E and Calhoun V. 2015. Synthetic structural magnetic resonance image generator improves deep learning prediction of schizophrenia//Proceedings of the 25th IEEE International Workshop on Machine Learning for Signal Processing (MLSP). Boston: IEEE: 1-6[ DOI: 10.1109/MLSP.2015.7324379 http://dx.doi.org/10.1109/MLSP.2015.7324379 ]
Ulloa A E, Plis S and Calhoun V D. 2018. Improving classification rate of schizophrenia using a multimodal multi-layer perceptron model with structural and functional MRI[EB/OL].[2020-05-15] . https://arxiv.org/pdf/1804.04591.pdf https://arxiv.org/pdf/1804.04591.pdf
Wang T, Bezerianos A, Cichocki A and Li J H. 2019a. Multi-kernel capsule network for schizophrenia identification[EB/OL].[2020-05-15] . https://arxiv.org/pdf/1907.12827.pdf https://arxiv.org/pdf/1907.12827.pdf
Wang Z J, Sun Y R, Shen Q Z and Cao L. 2019b. Dilated 3D convolutional neural networks for brain MRI data classification. IEEE Access, 7:134388-134398[DOI:10.1109/ACCESS.2019.2941912]
Yamashita R, Nishio M, Do R K G and Togashi K. 2018. Convolutional neural networks:an overview and application in radiology. Insights into Imaging, 9(4):611-629[DOI:10.1007/s13244-018-0639-9]
Yan W Z, Calhoun V, Song M, Cui Y, Yan H, Liu S F, Fan L Z, Zuo N M, Yang Z Y, Xu K B, Yan J, Lv L X, Chen J, Chen Y C, Guo H, Li P, Lu L, Wan P, Wang H N, Wang H L, Yang Y F, Zhang H X, Zhang D, Jiang T Z and Sui J. 2019. Discriminating schizophrenia using recurrent neural network applied on time courses of multi-site FMRI data. EBioMedicine, 47:543-552[DOI:10.1016/j.ebiom.2019.08.023]
Yan W Z, Plis S, Calhoun V D, Liu S F, Jiang R T, Jiang T Z and Sui J. 2017. Discriminating schizophrenia from normal controls using resting state functional network connectivity: a deep neural network and layer-wise relevance propagation method//Proceedings of the 27th IEEE International Workshop on Machine Learning for Signal Processing (MLSP). Tokyo: IEEE: 1-6[ DOI: 10.1109/MLSP.2017.8168179 http://dx.doi.org/10.1109/MLSP.2017.8168179 ]
Yang B, Chen Y, Shao Q M, Yu R, Li W B, Guo G Q, Jiang J Q and Pan L. 2019. Schizophrenia classification using fMRI data based on a multiple feature image capsule network ensemble. IEEE Access, 7:109956-109968[DOI:10.1109/ACCESS.2019.2933550]
Yang H H, Liu J Y, Sui J, Pearlson G and Calhoun V D. 2010. A hybrid machine learning method for fusing fMRI and genetic data:combining both improves classification of schizophrenia. Frontiers in Human Neuroscience, 4:#192[DOI:10.3389/fnhum.2010.00192]
Yang J C, Han S J, Mao L, Dai X Z and Chen Y R. 2019. Review of capsule network. Journal of Shandong University (Engineering Science), [DOI:10.6040/j.issn.1672-3961.0.2019.312]
杨巨成, 韩书杰, 毛磊, 代翔子, 陈亚瑞. 2019.胶囊网络模型综述.山东大学学报(工学版), 49(6):1-10)[DOI:10.6040/j.issn.1672-3961.0.2019.312]
Yang L, Wu Y X, Wang J L and Liu Y L. 2018. Research on recurrent neural network. Journal of Computer Applications, 38(S2):1-6, 26
杨丽, 吴雨茜, 王俊丽, 刘义理. 2018.循环神经网络研究综述.计算机应用, 38(S2):1-6, 26
Zeng L L, Wang H N, Hu P P, Yang B, Pu W D, Shen H, Chen X G, Liu Z N, Yin H, Tan Q R, Wang K and Hu D W. 2018. Multi-site diagnostic classification of schizophrenia using discriminant deep learning with functional connectivity MRI. EBioMedicine, 30:74-85[DOI:10.1016/j.ebiom.2018.03.017]
Zhang L, Wang M L, Liu M X and Zhang D Q. 2020. A survey on deep learning for neuroimaging-based brain disorder analysis[EB/OL].[2020-05-15] . https://arxiv.org/pdf/2005.04573.pdf https://arxiv.org/pdf/2005.04573.pdf
Zhang Q C, Yang L T, Chen Z K and Li P. 2018. A survey on deep learning for big data. Information Fusion, 42:146-157[DOI:10.1016/j.inffus.2017.10.006]
Zhou A J, Cui Y and Jiang T Z. 2018. Multisite schizophrenia classification based on Brainnetome atlas by deep learning//Proceedings of the 5th IEEE International Conference on Cloud Computing and Intelligence Systems (CCIS). Nanjing: IEEE: 451-455[ DOI: 10.1109/CCIS.2018.8691336 http://dx.doi.org/10.1109/CCIS.2018.8691336 ]
Zhou T, Thung K H, Zhu X F and Shen D G. 2019. Effective feature learning and fusion of multimodality data using stage-wise deep neural network for dementia diagnosis. Human Brain Mapping, 40(3):1001-1016[DOI:10.1002/hbm.24428]
Zhou Z H and Feng J. 2017. Deep forest: towards an alternative to deep neural networks//Proceedings of the 26th International Joint Conference on Artificial Intelligence. Australia: Morgan Kaufmann: 3553-3559[ DOI: 10.24963/ijcai.2017/497 http://dx.doi.org/10.24963/ijcai.2017/497 ]
相关作者
相关机构
京公网安备11010802024621
