融合跨阶段深度学习的脑肿瘤MRI图像分割

夏峰; 邵海见; 邓星

doi:10.11834/jig.210330

磁共振图像 | 浏览量 : 0 下载量: 346 CSCD: 5

PDF
导出
分享
收藏
专辑

融合跨阶段深度学习的脑肿瘤MRI图像分割
Cross-stage deep-learning-based MRI fused images of human brain tumor segmentation
2022年27卷第3期页码：873-884
收稿：2021-05-18，

修回：2021-10-29，

录用：2021-11-5，

纸质出版：2022-03-16
DOI： 10.11834/jig.210330
稿件说明：

移动端阅览

夏峰, 邵海见, 邓星. 融合跨阶段深度学习的脑肿瘤MRI图像分割[J]. 中国图象图形学报, 2022,27(3):873-884. DOI： 10.11834/jig.210330.

Feng Xia, Haijian Shao, Xing Deng. Cross-stage deep-learning-based MRI fused images of human brain tumor segmentation[J]. Journal of Image and Graphics, 2022, 27(3): 873-884. DOI： 10.11834/jig.210330.

摘要

目的

磁共振成像(magnetic resonance imaging，MRI)作为一种非侵入性的软组织对比成像方式，可以提供有关脑肿瘤的形状、大小和位置等有价值的信息，是用于脑肿瘤患者检查的主要方法，在脑肿瘤分割任务中发挥着重要作用。由于脑肿瘤本身复杂多变的形态、模糊的边界、低对比度以及样本梯度复杂等问题，导致高精度脑肿瘤MRI图像分割非常具有挑战性，目前主要依靠专业医师手动分割，费时且可重复性差。对此，本文提出一种基于U-Net的改进模型，即CSPU-Net(cross stage partial U-Net)脑肿瘤分割网络，以实现高精度的脑肿瘤MRI图像分割。

方法

CSPU-Net在U-Net结构的上下采样中分别加入两种跨阶段局部网络结构(cross stage partial module，${\rm{CSP}}$)提取图像特征，结合${\rm{GDL}}$(general Dice loss)和${\rm{WCE}}$(weighted cross entropy)两种损失函数解决训练样本类别不平衡问题。

结果

在BraTS (brain tumor segmentation) 2018和BraTS 2019两个数据集上进行实验，在BraTS 2018数据集中的整体肿瘤分割精度、核心肿瘤分割精度和增强肿瘤分割精度分别为87.9%、80.6%和77.3%，相比于传统U-Net的改进模型(ResU-Net)分别提升了0.80%、1.60%和2.20%。在BraTS 2019数据集中的整体肿瘤分割精度、核心肿瘤分割精度和增强肿瘤分割精度分别为87.8%、77.9%和70.7%，相比于ResU-Net模型提升了0.70%、1.30%和1.40%。

结论

本文提出的跨阶段局部网络结构，通过增加梯度路径、减少信息损失，可以有效提高脑肿瘤分割精度，实验结果证明了该模块对脑肿瘤分割任务的有效性。

Abstract

Objective

Human brain tumors are a group of mutant cells in the brain or skull. These benign or malignant brain tumors can be classified based on their growth characteristics and influence on the human body. Gliomas are one of the most frequent forms of malignant brain tumors

accounting for approximately 40% to 50% of all brain tumors. Glioma is classified as high-grade glioma (HGG) or low-grade glioma (LGG) depending on the degree of invasion. Low-grade glioma (LGG) is a well-differentiated glioma with a prompt prognosis. High-grade glioma (HGG) is a poorly differentiated glioma with a in qualified prognosis. Gliomas with varying degrees of differentiation are appeared following the varied degrees of peritumoral edema

edema types

and necrosis. the boundary of gliomas and normal tissues is often blurred. It is difficult to identify the scope of lesions and surgical area

which has a significant impact on surgical quality and patient prognosis. As a non-invasive and clear soft tissue contrast imaging tool

magnetic resonance imaging (MRI) can provide vital information on the shape

size

and location of brain tumors. High-precision brain tumor MRI image segmentation is challenged due to the complicated and variable morphology

fuzzy borders

low contrast

and complicated sample gradients of brain tumors. Manual segmentation is time-consuming and inconsistent. The International Association for Medical Image Computing and Computer-Aided Intervention (MICCAI)'s Brain Tumor Segmentation (BraTS) is a global medical image segmentation challenge concentrating on the evaluation of automatic segmentation methods for human brain tumors. There are four types of automatic brain tumor segmentation algorithms as mentioned below: supervised learning

semi-supervised learning

unsupervised learning

and hybrid learning. Supervised-learning-based algorithm is currently the effective method. Various depth neural network models for computer vision problems

such as Visual Geometry Group Network (VGGNet)

GoogLeNet

ResNet

and DenseNet

have been presented in recent years. The above deep neural network model proposes a novel approach to the problem of MRI brain image segmentation

and it significantly advances the development of deep learning-based brain tumor diagnosis methods. As a result

deep learning method is to develop the task of automatic segmentation of brain tumor MRI images.

Method

Our research integrates low resolution information and high resolution information via the U-Net structure of the all convolutional neural network. An improved cross stage partial U-Net(CSPU-Net) brain tumor segmentation network derived from the U-Net network achieves high-precision brain tumor MRI image segmentation. The basic goal of the cross stage partial (CSP) module is to segment the grass-roots feature mapping into two sections as following: 1)Dividing the gradient flow to extend distinct network paths

and then 2) fusing the two portions based on horizontal hierarchy. The conveyed gradient information can have huge correlation discrepancies by alternating series and transition procedures. To extract image features

CSPU-Net adds two types of cross stage partial network structures to the up and down sampling of the U-Net network. The number of gradient routes is increased using the splitting and merging technique. The drawbacks of utilizing explicit feature map replication for connection are mitigated

enhancing the model's feature learning capabilities. To overcome the imbalance issue of sample class

two loss functions

general dice loss and weighted cross-entropy are combined. The cross stage partial structure is final compared to ResU-Net

which adds a residual block

to in identify the effectiveness of cross stage partial structure as ResU-Net in the brain tumor segmentation test.

Result

Our experimental results of the CSPU-Net model in the context of the BraTS 2018 and BraTS 2019 datasets has its priority. The BraTS 2018 dataset yielded 87.9% accuracy in whole tumor segmentation

80.6% accuracy in core tumor segmentation

and 77.3% accuracy in enhanced tumor segmentation

respectively. This method enhances the segmentation accuracy of brain tumor MRI images by 0.80%

1.60%

and 2.20% each. In the BraTS 2019 dataset

the whole tumor segmentation accuracy is 87.8%

the core tumor segmentation accuracy is 77.9%

and the improved tumor segmentation accuracy is 70.7%

respectively. This method enhances the segmentation accuracy of brain tumor MRI images by 0.70%

1.30%

and 1.40%

respectively in comparison of the traditional improved ResU-Net.

Conclusion

This research provides a cross-stage deep learning-based 2D segmentation network for human brain tumor MRI images. Using cross stage partial network structure in U-Net up and down sampling

the model enhances the accuracy of brain tumor segmentation via gradient path expansion and information loss deduction. The demonstrated results illustrate that our model has its potentials on BraTS datasets on 2D segmentation models development and demonstrates the module's efficiency in the brain tumor segmentation task.

关键词

Keywords

references

Alom M Z, Yakopcic C, Hasan M, Taha T M and Asari V K. 2019. Recurrent residual U-Net for medical image segmentation. Journal of Medical Imaging, 6(1): #014006[DOI:10.1117/1.JMI.6.1.014006]

Amian M and Soltaninejad M. 2020. Multi-resolution 3D CNN for MRI brain tumor segmentation and survival prediction//Proceedings of the 5th International MICCAI Brainlesion Workshop. Shenzhen, China: Springer: 221-230[ DOI: 10.1007/978-3-030-46640-4_21 http://dx.doi.org/10.1007/978-3-030-46640-4_21 ]

Baid U, Shah N A and Talbar S. 2020. Brain tumor segmentation with cascaded deep convolutional neural network//Proceedings of the 5th International MICCAI Brainlesion Workshop. Shenzhen, China: Springer: 90-98[ DOI: 10.1007/978-3-030-46643-5_9 http://dx.doi.org/10.1007/978-3-030-46643-5_9 ]

Bhalerao M and Thakur S. 2020. Brain tumor segmentation based on 3D residual U-Net//Proceedings of the 5th International MICCAI Brainlesion Workshop. Shenzhen, China: Springer: 218-225[ DOI: 10.1007/978-3-030-46643-5_21 http://dx.doi.org/10.1007/978-3-030-46643-5_21 ]

Bochkovskiy A, Wang C Y and Liao H Y M. 2020. YOLOv4: optimal speed and accuracy of object detection[EB/OL]. [2021-05-03] . https://arxiv.org/pdf/2004.10934.pdf https://arxiv.org/pdf/2004.10934.pdf

Chen W, Liu B Q, Peng S T, Sun J W and Qiao X. 2019. S3D-UNet: separable 3DU-Net for brain tumor segmentation//Proceedings of the 4th International MICCAI Brainlesion Workshop. Granada, Spain: Springer: 358-368[ DOI: 10.1007/978-3-030-11726-9_32 http://dx.doi.org/10.1007/978-3-030-11726-9_32 ]

Di Ieva A, Russo C, Liu S D, Jian A N, Bai M Y, Qian Y and Magnussen J S. 2021. Application of deep learning for automatic segmentation of brain tumors on magnetic resonance imaging: a heuristic approach in the clinical scenario. Neuroradiology, 63(8): 1253-1262[DOI: 10.1007/s00234-021-02649-3]

Ding Y, Gong L P, Zhang M F, Li C and Qin Z G. 2020. A multi-path adaptive fusion network for multimodal brain tumor segmentation. Neurocomputing, 412: 19-30[DOI: 10.1016/j.neucom.2020.06.078]

Dong H, Yang G, Liu F D, Mo Y H and Guo Y K. 2017. Automatic brain tumor detection and segmentation using U-Net based fully convolutional networks//Proceedings of the 21st Annual Conference on Medical Image Understanding and Analysis. Edinburgh, UK: Springer: 506-517[ DOI: 10.1007/978-3-319-60964-5_44 http://dx.doi.org/10.1007/978-3-319-60964-5_44 ]

Feng B W. 2020. Research on multi-scale feature segmentation of Brain Tumors MRI based on parallel CNN

冯博文. 2020. 脑肿瘤MRI的并行CNN多尺度特征分割技术研究. 内蒙古: 内蒙古科技大学[ DOI:10.27724/d.cnki.gnmgk.2020.000421 http://dx.doi.org/10.27724/d.cnki.gnmgk.2020.000421 ]

Gondal A H and Khan M N A. 2013. A review of fully automated techniques for brain tumor detection from MR images. International Journal of Modern Education and Computer Science, 5(2): 55-61[DOI: 10.5815/ijmecs.2013.02.08]

Gordillo N, Montseny E and Sobrevilla P. 2013. State of the art survey on MRI brain tumor segmentation. Magnetic Resonance Imaging, 31(8): 1426-1438[DOI: 10.1016/j.mri.2013.05.002]

Guo X T, Yang C S, Ma T, Zhou P Z, Lu S F, Ji N, Li D L, Wang T and Lyu H Y. 2020. Brain tumor segmentation based on attention mechanism and multi-model fusion//Proceedings of the 5th International MICCAI Brainlesion Workshop. Shenzhen, China: Springer: 50-60[ DOI: 10.1007/978-3-030-46643-5_5 http://dx.doi.org/10.1007/978-3-030-46643-5_5 ]

Haris M, Gupta R K, Singh A, Husain N, Husain M, Pandey C M, Srivastava C, Behari S and Rathore R K S. 2008. Differentiation of infective from neoplastic brain lesions by dynamic contrast-enhanced MRI. Neuroradiology, 50(6): 531-540[DOI: 10.1007/s00234-008-0378-6]

Havaei M, Davy A, Warde-Farley D, Biard A, Courville A, Bengio Y, Pal C, Jodoin P M and Larochelle H. 2017. Brain tumor segmentation with deep neural networks. Medical Image Analysis, 35: 18-31[DOI: 10.1016/j.media.2016.05.004]

He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE: 770-778[ DOI: 10.1109/CVPR.2016.90 http://dx.doi.org/10.1109/CVPR.2016.90 ]

Hu K, Gan Q H, Zhang Y, Deng S H, Xiao F, Huang W, Cao C H and Gao X P. 2019. Brain tumor segmentation using multi-cascaded convolutional neural networks and conditional random field. IEEE Access, 7: 92615-92629[DOI:10.1109/ACCESS.2019.2927433]

Huang G, Liu Z, Van Der Maaten Land Weinberger K Q. 2017. Densely connected convolutional networks//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 2261-2269[ DOI: 10.1109/CVPR.2017.243 http://dx.doi.org/10.1109/CVPR.2017.243 ]

Isensee F, Kickingereder P, Wick W, Bendszus M and Maier-Hein K H. 2019. No New-Net//Proceedings of the 4th International MICCAI Brainlesion Workshop. Granada, Spain: Springer: 234-244[ DOI: 10.1007/978-3-030-11726-9_21 http://dx.doi.org/10.1007/978-3-030-11726-9_21 ]

Işın A, Direkoǧlu C and Şah M. 2016. Review of MRI-based brain tumor image segmentation using deep learning methods. Procedia Computer Science, 102: 317-324[DOI: 10.1016/j.procs.2016.09.407]

Jiang Z Y, Ding C X, Liu M F and Tao D C. 2020. Two-stage cascaded U-Net: 1st place solution to brats challenge 2019 segmentation task//Proceedings of the 5th International MICCAI Brainlesion Workshop. Shenzhen, China: Springer: 231-241[ DOI: 10.1007/978-3-030-46640-4_22 http://dx.doi.org/10.1007/978-3-030-46640-4_22 ]

Kong X M, Sun G X, Wu Q, Liu J and Lin F M. 2018. Hybrid pyramid U-Net model for brain tumor segmentation//Proceedings of the 10th International Conference on Intelligent Information Processing. Nanning, China: Springer: 346-355[ DOI: 10.1007/978-3-030-00828-4_35 http://dx.doi.org/10.1007/978-3-030-00828-4_35 ]

Li X M, Chen H, Qi X J, Dou Q, Fu C W and Heng P A. 2018. H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Transactions on Medical Imaging, 37(12): 2663-2674[DOI: 10.1109/TMI.2018.2845918]

Litjens G, Kooi T, Bejnordi B E, Setio A A A, Ciompi F, Ghafoorian M, Van Der Laak J A W M, Van Ginneken B and Sánchez C I. 2017. A survey on deep learning in medical image analysis. Medical Image Analysis, 42: 60-88[DOI: 10.1016/j.media.2017.07.005]

Ma F. 2013. 3DVolume Enhanced Scanning Techniques of Magnetic Resonance Imaging Used in Intracranial Tumors. Yinchuan: Ningxia Medical University

马凤. 2013. 磁共振三维容积增强扫描技术在颅内肿瘤中的应用. 银川: 宁夏医科大学[DOI: 10.7666/d.D421508]

Ma J and Yang X P. 2019. Automatic brain tumor segmentation by exploring the multi-modality complementary information and cascaded 3D lightweight CNNs//Proceedings of the 4th International MICCAI Brainlesion Workshop. Granada, Spain: Springer: 25-36[ DOI: 10.1007/978-3-030-11726-9_3 http://dx.doi.org/10.1007/978-3-030-11726-9_3 ]

Marcinkiewicz M, Nalepa J, Lorenzo P R, Dudzik W and Mrukwa G. 2019. Segmenting brain tumors from MRI using cascaded multi-modal U-Nets//Proceedings of the 4th International MICCAI Brainlesion Workshop. Granada, Spain: Springer: 13-24[ DOI: 10.1007/978-3-030-11726-9_2 http://dx.doi.org/10.1007/978-3-030-11726-9_2 ]

Moeskops P, Viergever M A, Mendrik A M, de Vries L S, Benders M J N L and Išgum I. 2016. Automatic segmentation of MR brain images with a convolutional neural network. IEEE Transactions on Medical Imaging, 35(5): 1252-1261[DOI: 10.1109/TMI.2016.2548501]

Myronenko A. 2019. 3D MRI brain tumor segmentation using autoencoder regularization//4th International MICCAI Brainlesion Workshop. Granada, Spain: Springer: 311-320[ DOI: 10.1007/978-3-030-11726-9_28 http://dx.doi.org/10.1007/978-3-030-11726-9_28 ]

Pereira S, Pinto A, Alves V and Silva C A. 2016. Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Transactions on Medical Imaging, 35(5): 1240-1251[DOI: 10.1109/TMI.2016.2538465]

Ren X X, Xing Z C, Xia X, Lo D, Wang X Y and Grundy J. 2019. Neural network-based detection of self-admitted technical debt: from performance to explainability. ACM transactions on software engineering and methodology, 28(3): #15[DOI: 10.1145/3324916]

Ronneberger O, Fischer P and Brox T. 2015. U-Net: convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: Springer: 234-241[ DOI: 10.1007/978-3-319-24574-4_28 http://dx.doi.org/10.1007/978-3-319-24574-4_28 ]

Shaikh M, Anand G, Acharya G, Amrutkar A, Alex V and Krishnamurthi G. 2018. Brain tumor segmentation using dense fully convolutional neural network//Proceedings of the 3rd International MICCAI Brainlesion Workshop. Quebec City, Canada: Springer: 309-319[ DOI: 10.1007/978-3-319-75238-9_27 http://dx.doi.org/10.1007/978-3-319-75238-9_27 ]

Shelhamer E, Long J and Darrell T. 2017. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4): 640-651[DOI: 10.1109/TPAMI.2016.2572683]

Shen H C, Wang R X, Zhang J G and McKenna S. 2017. Multi-task fully convolutional network for brain tumour segmentation//Proceedings of the 21st Annual Conference on Medical Image Understanding and Analysis. Edinburgh, UK: Springer: 239-248[ DOI: 10.1007/978-3-319-60964-5_21 http://dx.doi.org/10.1007/978-3-319-60964-5_21 ]

Simonyan K and Zisserman A. 2015. Very deep convolutional networks for large-scale image recognition[EB/OL]. [2021-05-03] . https://arxiv.org/pdf/1409.1556.pdf https://arxiv.org/pdf/1409.1556.pdf

Sudre C H, Li W Q, Vercauteren T, Ourselin S and Cardoso M J. 2017. Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations//Proceedings of the 3rd International Workshop on Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Québec City, Canada: Springer: 240-248[ DOI: 10.1007/978-3-319-67558-9_28 http://dx.doi.org/10.1007/978-3-319-67558-9_28 ]

Sun H. 2020. Research on Classification and Segmentation of MRI of Brain Tumor based on Deep Learning. Hangzhou: Zhejiang University

孙浩. 2020. 基于深度学习的脑肿瘤MRI分类与分割技术研究. 杭州: 浙江大学[DOI: 10.27461/d.cnki.gzjdx.2020.000444]

Sun J W. 2020. Application of Deep Learning Methods in Brain Tumors. Ji'nan: Shandong University

孙佳伟. 2020. 深度学习在脑肿瘤中的应用. 济南: 山东大学[DOI: 10.27272/d.cnki.gshdu.2020.004996]

Sun L, Zhang S T, Chen H and Luo L. 2019. Brain tumor segmentation and survival prediction using multimodal MRI scans with deep learning. Frontiers in Neuroscience, 13: #810[DOI: 10.3389/fnins.2019.00810]

Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V and Rabinovich A. 2015. Going deeper with convolutions//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, USA: IEEE: 1-9[ DOI: 10.1109/CVPR.2015.7298594 http://dx.doi.org/10.1109/CVPR.2015.7298594 ]

Tian W Q. 2020. The Study of Medical Image Segmentation based on Convolution Neural Network. Baoding: Hebei University

田伟倩. 2020. 基于卷积神经网络的医学图像分割算法研究. 保定: 河北大学[DOI: 10.27103/d.cnki.ghebu.2020.001097]

Wang C Y, Liao H Y M, Wu Y H, Chen P Y, Hsieh J W and Yeh I H. 2020. CSPNet: a new backbone that can enhance learning capability of CNN//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Seattle, USA: IEEE: 1571-1580[ DOI: 10.1109/CVPRW50498.2020.00203 http://dx.doi.org/10.1109/CVPRW50498.2020.00203 ]

Wang G T, Li W Q, Ourselin S and Vercauteren T. 2019a. Automatic brain tumor segmentation using convolutional neural networks with test-time augmentation//Proceedings of the 4th International MICCAI Brainlesion Workshop. Granada, Spain: Springer: 61-72[ DOI: 10.1007/978-3-030-11726-9_6 http://dx.doi.org/10.1007/978-3-030-11726-9_6 ]

Wang W, Yu K C, Hugonot J, Fua P and Salzmann M. 2019b. Recurrent U-Net for resource-constrained segmentation//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Korea (South): IEEE: 2142-2151[ DOI: 10.1109/ICCV.2019.00223 http://dx.doi.org/10.1109/ICCV.2019.00223 ]

Wong K P. 2005. Medical image segmentation: methods and applications in functional imaging//Handbook of Biomedical Image Analysis. Boston, USA: Springer: 111-182[ DOI: 10.1007/0-306-48606-7_3 http://dx.doi.org/10.1007/0-306-48606-7_3 ]

Zhao L Y and Jia K B. 2016. Multiscale CNNs for brain tumor segmentation and diagnosis. Computational and Mathematical Methods in Medicine, 2016: #8356294[DOI: 10.1155/2016/8356294]

Zhou Z X, He Z S, ShiM F, Du J L and Chen D D. 2020. 3D dense connectivity network with atrous convolutional feature pyramid for brain tumor segmentation in magnetic resonance imaging of human heads. Computers in Biology and Medicine, 121: #103766[DOI: 10.1016/j.compbiomed.2020.103766]