浏览全部资源
扫码关注微信
收稿：2020-06-11，
修回：2020-7-3，
录用：2020-7-10，
纸质出版：2020-10-16
移动端阅览
磁共振（magnetic resonance，MR）成像作为一种安全非侵入式的成像技术，可以提供高分辨率且具有不同对比度的大脑图像，被越来越多地应用于婴儿大脑研究中。将婴儿脑MR图像准确地分割为灰质、白质和脑脊液，是研究早期大脑发育模式不可或缺的基础处理环节。由于在等强度阶段（6~9月龄）婴儿脑MR图像中，灰质和白质信号强度基本一致，组织对比度极低，导致此阶段的脑组织分割非常具有挑战性。基于深度学习的等强度婴儿脑MR图像分割方法，由于其卓越的性能受到研究人员的广泛关注，但目前尚未有文献对该领域的方法进行系统总结和分析。因此本文对目前基于深度学习的等强度婴儿脑MR图像分割方法进行了系统总结，从基本思想、网络架构、性能及优缺点4个方面进行了介绍。并针对其中的典型算法在iSeg-2017数据集上的分割结果进行了对比分析，最后对等强度婴儿脑MR图像分割中存在的问题及未来研究方向进行展望。本文通过对目前基于深度学习的等强度婴儿脑MR图像分割方法进行总结，可以看出深度学习方法已经在等强度期婴儿脑分割中展现出巨大优势，相比传统方法在分割精度和效率上均有较大提升，将进一步促进人类人脑早期发育研究。
Emerging interest has been shown in the study of infant brain development by using magnetic resonance (MR) imaging because it provides a safe and noninvasive way of examining cross-sectional views of the brain in multiple contrasts. Quantitative analysis of brain MR images is a conventional routine for many neurological diseases and conditions
which relies on the accurate segmentation of structures of interest. Accurate segmentation of infant brain MR images into white matter (WM)
gray matter (GM)
and cerebrospinal fluid is of great importance in studying and measuring normal and abnormal early brain development. However
in the isointense phase (6-9 months of age)
WM and GM exhibit similar levels of intensity in T1-weighted and T2-weighted MR images due to the inherent myelination and maturation
posing significant challenges for automated segmentation. Compared with traditional methods
deep learning-based methods have greatly improved the accuracy and efficiency of isointense infant brain segmentation. Thus
deep learning-based segmentation methods have been increasingly used by researchers due to their excellent performance. Nevertheless
no literature has systematically summarized and analyzed the methods in this field. The current study aims to review existing deep learning-based approaches for isointense infant brain MR image segmentation. With the extensive research in the literature
we systematically summarized the current deep learning-based methods for isointense infant brain MR image segmentation. We first introduced an authoritative isointense infant brain segmentation dataset
which was used in the iSeg-2017 challenge
hosted as a part of the Medical Image Computing and Computer Assisted Intervention Society Conference 2017. Afterward
several evaluation metrics
including Dice coefficient
95th-percentile Hausdorff distance
and average symmetric surface distance
were briefly described. We classified the existing deep learning-based methods for isointense infant brain MR image segmentation into two categories: 2D and fully convolutional neural network-based methods. Fully convolutional neural network-based methods can be further divided into two subcategories: 2D and 3D network-based approaches. On the basis of the two categories
we comprehensively described and analyzed the basic ideas
network architecture
improvement schemes
and segmentation performance of each method. In addition
we compared the performance of some existing deep learning-based methods and summarized the analysis results of typical methods on the iSeg-2017 dataset. Lastly
three possible research directions of isointense infant brain MR image segmentation methods based on deep learning were discussed. We drew three main conclusions by reviewing the main work in this field
thereby providing a good overview of existing deep learning methods for isointense infant brain MR image segmentation. First
using multimodality data is beneficial for the network to obtain rich feature information
which can improve the segmentation accuracy. Second
compared with 2D fully convolutional network-based methods
3D-based methods for MR image brain segmentation can integrate richer spatial context information
resulting in higher accuracy and efficiency. Third
adopting complicated network architecture and dense network connection could make the network achieve superior accuracy performance. Three possible future directions
namely
embedding powerful feature representation modules (e.g.
attention mechanism)
adding prior knowledge of the infant brain to the network model (e.g.
the cortical thickness of infant brain is within a certain range)
and constructing a semisupervised or weakly supervised network model trained with a small amount of labeled data
were recommended. Precise segmentation of infant brain tissues is an essential step towards comprehensive volumetric studies and quantitative analysis of early brain development. Deep learning has shown great advantages in isointense infant brain segmentation
and its accuracy and efficiency have been greatly improved compared with traditional methods. With the development of deep learning
it will bring further improvement in the research field of isointense infant brain segmentation and promote the research of early human brain development.
Bui T D, Shin J and Moon T. 2017.3D densely convolutional networks for volumetric segmentation[EB/OL].[2020-05-15] . https://arxiv.org/pdf/1709.03199.pdf https://arxiv.org/pdf/1709.03199.pdf
Cardoso M J, Melbourne A, Kendall G S, Modat M, Robertson N J, Marlow N and Ourselin S. 2013. AdaPT:an adaptive preterm segmentation algorithm for neonatal brain MRI. NeuroImage, 65:97-108[DOI:10.1016/j.neuroimage.2012.08.009]
ÇiçekÖ, Abdulkadir A, Lienkamp S S, Brox T and Ronneberger O. 2016.3D U-Net: learning dense volumetric segmentation from sparse annotation//Proceedings of the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention. Athens: Springer: 424-432[ DOI:10.1007/978-3-319-46723-8_49 http://dx.doi.org/10.1007/978-3-319-46723-8_49 ]
Dai Y K, Shi F, Wang L, Wu G R and Shen D G. 2013. iBEAT:a toolbox for infant brain magnetic resonance image processing. Neuroinformatics, 11(2):211-225[DOI:10.1007/s12021-012-9164-z]
Dolz J, Desrosiers C, Wang L, Yuan J, Shen D G and Ayed I B. 2020. Deep CNN ensembles and suggestive annotations for infant brain MRI segmentation. Computerized Medical Imaging and Graphics, 79:#101660[DOI:10.1016/j.compmedimag.2019.101660]
Dolz J, Gopinath K, Yuan J, Lombaert H, Desrosiers C and Ayed I B. 2019. HyperDense-Net:a hyper-densely connected CNN for multi-modal image segmentation. IEEE Transactions on Medical Imaging, 38(5):1116-1126[DOI:10.1109/TMI.2018.2878669]
Fan Y, Shi F, Smith J K, Lin W L, Gilmore J H and Shen D G. 2011. Brain anatomical networks in early human brain development. NeuroImage, 54(3):1862-1871[DOI:10.1016/j.neuroimage.2010.07.025]
Fonov V S, Doyle A, Evans A C and Collins D L. 2018. NeuroMTL iSEG challenge methods[EB/OL].[2020-05-15] . https://www.biorxiv.org/content/10.1101/278465v1 https://www.biorxiv.org/content/10.1101/278465v1
Gilmore J H, Shi F, Woolson S L, Knickmeyer R C, Short S J, Lin W L, Zhu H T, Hamer R M, Styner M and Shen D G. 2012. Longitudinal development of cortical and subcortical gray matter from birth to 2 years. Cerebral Cortex, 22(11):2478-2485[DOI:10.1093/cercor/bhr327]
Glorot X, Bordes A and Bengio Y. 2011. Deep sparse rectifier neural networks//Proceedings of the 14th International Conference on Artificial Intelligence and Statistics. Florida: PMLR: 315-323
Goodfellow I J, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A and Bengio Y. 2014. Generative adversarial nets//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal: Curran Associates Inc.: 2672-2680
Gui L, Lisowski R, Faundez T, Hüppi P S, Lazeyras F and Kocher M. 2012. Morphology-driven automatic segmentation of MR images of the neonatal brain. Medical Image Analysis, 16(8):1565-1579[DOI:10.1016/j.media.2012.07.006]
Hazlett H C, Gu H B, Munsell B C, Kim S H, Styner M, Wolff J J, Elison J T, Swanson M R, Zhu H T, Botteron K N, Collins D L, Constantino J N, Dager S R, Estes A M, Evans A C, Fonov V S, Gerig G, Kostopoulos P, McKinstry R C, Pandey J, Paterson S, Pruett J R, Schultz R T, Shaw D W, Zwaigenbaum L, Piven J and The IBIS Network. 2017. Early brain development in infants at high risk for autism spectrum disorder. Nature, 542(7641):348-351[DOI:10.1038/nature21369]
He K M, Zhang X Y, Ren S Q and Sun J. 2015. Delving deep into rectifiers: surpassing human-level performance on ImageNet classification//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago: IEEE: 1026-1034[ DOI:10.1109/ICCV.2015.123 http://dx.doi.org/10.1109/ICCV.2015.123 ]
Hu J, Shen L and Sun G. 2018. Squeeze-and-excitation networks//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 7132-7141[ DOI:10.1109/CVPR.2018.00745 http://dx.doi.org/10.1109/CVPR.2018.00745 ]
Huang G, Liu Z, Van Der Maaten L and Weinberger K Q. 2017. Densely connected convolutional networks//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 2261-2269[ DOI:10.1109/CVPR.2017.243 http://dx.doi.org/10.1109/CVPR.2017.243 ]
Ioffe S and Szegedy C. 2015. Batch normalization: accelerating deep network training byreducing internal covariate shift[EB/OL].[2020-05-15] . https://arxiv.org/pdf/1502.03167.pdf https://arxiv.org/pdf/1502.03167.pdf
Kim S H, Fonov V S, Dietrich C, Vachet C, Hazlett H C, Smith R G, Graves M M, Piven J, Gilmore J H, Dager S R, McKinstry R C, Paterson S, Evans A C, Collins D L, Gerig G, Styner M and The IBIS network. 2013. Adaptive prior probability and spatial temporal intensity change estimation for segmentation of the one-year-old human brain. Journal of Neuroscience Methods, 212(1):43-55[DOI:10.1016/j.jneumeth.2012.09.018]
Knickmeyer R C, Gouttard S, Kang C, Evans D, Wilber K, Smith J K, Hamer R M, Lin W L, Gerig G and Gilmore J H. 2008. A structural MRI study of human brain development from birth to 2 years. The Journal of Neuroscience, 28(47):12176-12182[DOI:10.1523/JNEUROSCI.3479-08.2008]
Krizhevsky A, Sutskever I and Hinton G E. 2012. ImageNet classification with deep convolutional neural networks//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe: Curran Associates Inc: 1097-1105
Li G, Nie J X, Wang L, Shi F, Lin W L, Gilmore J H and Shen D G. 2013. Mapping region-specific longitudinal cortical surface expansion from birth to 2 years of age. Cerebral Cortex, 23(11):2724-2733[DOI:10.1093/cercor/bhs265]
Li G, Wang L, Shi F, Lin W L and Shen D G. 2014. Constructing 4D infant cortical surface atlases based on dynamic developmental trajectories of the cortex//Proceedings of the 17th International Conference on Medical Image Computing and Computer-Assisted Intervention. Boston: Springer: 89-96[ DOI:10.1007/978-3-319-10443-0_12 http://dx.doi.org/10.1007/978-3-319-10443-0_12 ]
Li X J, Chen J, Xia J, Wang M M, Li M X, Wang L, Li G, Shen D G and Yang J. 2019. Quantitative evaluation of the cortical development on neonates based on segmentation of 3D T1 WI images using transfer learning. Chinese Journal of Magnetic Resonance Imaging, 10(10):736-742
李贤军, 陈健, 夏菁, 王苗苗, 李梦轩, 王利, 李刚, 沈定刚, 杨健. 2019.基于迁移学习算法对新生儿大脑3D T1 WI的灰白质分割及其发育量化研究.磁共振成像, 10(10):736-742 [DOI:10.12015/issn.1674-8034.2019.10.004]
Lin Y and Tian J. 2002. A survey on medical image segmentation methods. Pattern Recognition and Artificial Intelligence, 15(2):192-204
林瑶, 田捷. 2002.医学图像分割方法综述.模式识别与人工智能, 15(2):192-204 [DOI:10.3969/j.issn.1003-6059.2002.02.012]
Litjens G, Kooi T, Bejnordi B E, Setio A A A, Ciompi F, Ghafoorian M, van Der Laak J A W M, van Ginneken B and Sánchez C I. 2017. A survey on deep learning in medical image analysis. Medical Image Analysis, 42:60-88[DOI:10.1016/j.media.2017.07.005]
Long J, Shelhamer E and Darrell T. 2015. Fully convolutional networks for semantic segmentation//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE: 3431-3440[ DOI:10.1109/CVPR.2015.7298965 http://dx.doi.org/10.1109/CVPR.2015.7298965 ]
Moeskops P and Pluim J P W. 2017. Isointense infant brain MRI segmentation with a dilated convolutional neural network[EB/OL].[2020-05-15] . https://arxiv.org/pdf/1-708.02757.pdf https://arxiv.org/pdf/1-708.02757.pdf
Moeskops P, Viergever M A, Mendrik A M, De Vries L S, Benders M J N L and Išgum I. 2016. Automatic segmentation of MR brain images with a convolutional neural network. IEEE Transactions on Medical Imaging, 35(5):1252-1261[DOI:10.1109/TMI.2016.2548501]
Nie D, Wang L, Gao Y Z and Shen D G. 2016. Fully convolutional networks for multi-modality isointense infant brain image segmentation//Proceedings of 13th International Symposium on Biomedical Imaging. Prague: IEEE: 638-642[ DOI:10.1109/ISBI.2016.7493515 http://dx.doi.org/10.1109/ISBI.2016.7493515 ]
Oquab M, Bottou L, Laptev I, Laptev I and Sivic J. 2015. Is object localization for free?-Weakly-supervised learning with convolutional neural networks//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE: 685-694[ DOI:10.1109/CVPR.2015.7298668 http://dx.doi.org/10.1109/CVPR.2015.7298668 ]
Paus T, Collins D L, Evans A C, Leonard G, Pike B and Zijdenbos A. 2001. Maturation of white matter in the human brain:a review of magnetic resonance studies. Brain Research Bulletin, 54(3):255-266[DOI:10.1016/S0361-9230(00)00434-2]
Ren S Q, He K M, Girshick R and Sun J. 2017. Faster R-CNN:towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6):1137-1149[DOI:10.1109/TPAMI.2016.2577031]
Ronneberger O, Fischer P and Brox T. 2015. U-Net: convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: Springer: 234-241[ DOI:10.1007/978-3-319-24574-4_28 http://dx.doi.org/10.1007/978-3-319-24574-4_28 ]
Shi F, Yap P T, Gilmore J H, Lin W L and Shen D G. 2010. Spatial-temporal constraint for segmentation of serial infant brain MR images//Proceedings of the 5th International Workshop on Medical Imaging and Augmented Reality. Beijing: Springer: 42-50[ DOI:10.1007/978-3-642-15699-1_5 http://dx.doi.org/10.1007/978-3-642-15699-1_5 ]
Srivastava N, Hinton G, Krizhevsky A, Sutskever I and Salakhutdinov R. 2014. Dropout:a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1):1929-1958
Wang L, Nie D, Li G N, Puybareau É, Dolz J, Zhang Q, Wang F, Xia J, Wu Z W, Chen J W, Thung K H, Bui T D, Shin J, Zeng G D, Zheng G Y, Fonov V S, Doyle A, Xu Y C, Moeskops P, Pluim J P W, Desrosiers C, Ayed I B, Sanroma G, Benkarim O M, Casamitjana A, Vilaplana V, Lin W L, Li G and Shen D G. 2019. Benchmark on automatic six-month-old infant brain segmentation algorithms:the iSeg-2017 Challenge. IEEE Transactions on Medical Imaging, 38(9):2219-2230[DOI:10.1109/TMI.2019.2901712]
Wang L, Shi F, Gao Y Z, Li G, Gilmore J H, Lin W L and Shen D G. 2014. Integration of sparse multi-modality representation and anatomical constraintfor isointense infant brain MR image segmentation. NeuroImage, 89:152-164[DOI:10.1016/j.neuroimage.2013.11.040]
Wang L, Shi F, Lin W L, Gilmore J H and Shen D G. 2011. Automatic segmentation of neonatal images using convex optimization and coupled level sets. NeuroImage, 58(3):805-817[DOI:10.1016/j.neuroimage.2011.06.064]
Wang L, Shi F, Yap P T, Lin W L, Gilmore J H and Shen D G. 2013. Longitudinally guided level sets for consistent tissue segmentation of neonates. Human Brain Mapping, 34(4):956-972[DOI:10.1002/hbm.21486]
Wei Y, Zhang K and Han F. 2017. Improved ASFCM-based algorithm for infant brain MRI segmentation. Journal of Northeastern University (Natural Science), 38(8):1079-1084
魏颖, 张开, 韩枫. 2017.改进的基于ASFCM婴幼儿脑部MRI分割算法.东北大学学报(自然科学版), 38(8):1079-1084 [DOI:10.12068/j.issn.1005-3026.2017.08.004]
Xiao T J, Xu Y C, Yang K Y, Zhang J X, Peng Y X and Zhang Z. 2015. The application of two-level attention models in deep convolutional neural network for fine-grained image classification//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE: 842-850[ DOI:10.1109/CVPR.2015.7298685 http://dx.doi.org/10.1109/CVPR.2015.7298685 ]
Xu Y C, Géraud T and Bloch I. 2017. From neonatal to adult brain MR image segmentation in a few seconds using 3D-like fully convolutional network and transfer learning//Proceedings of 2017 IEEE International Conference on Image Processing. Beijing: IEEE: 4417-4421[ DOI:10.1109/icip.2017.8297117 http://dx.doi.org/10.1109/icip.2017.8297117 ]
Xue H, Srinivasan L, Jiang S Z, Rutherford M, Edwards A D, Rueckert D and Hajnal J V. 2007. Automatic segmentation and reconstruction of the cortex from neonatal MRI. NeuroImage, 38(3):461-477[DOI:10.1016/j.neuroimage.2007.07.030]
Yushkevich P A, Piven J, Hazlett H C, Smith R G, Ho S, Gee J C and Gerig G. 2006. User-guided 3D active contour segmentation of anatomical structures:significantly improved efficiency and reliability. NeuroImage, 31(3):1116-1128[DOI:10.1016/j.neuroimage.2006.01.015]
Zeng G D and Zheng G Y. 2018. Multi-stream 3D FCN with multi-scale deep supervision for multi-modality isointense infant brain MR image segmentation//Proceedings of the 15th International Symposium on Biomedical Imaging. Washington: IEEE: 136-140[ DOI:10.1109/isbi.2018.8363540 http://dx.doi.org/10.1109/isbi.2018.8363540 ]
Zhang W L, Li R J, Deng H T, Wang L, Lin W L, Ji S W and Shen D G. 2015. Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. NeuroImage, 108:214-224[DOI:10.1016/j.neuroimage.2014.12.061]
相关作者
相关机构
京公网安备11010802024621
