浏览全部资源
扫码关注微信
收稿：2020-09-30，
修回：2021-1-6，
录用：2021-1-13，
纸质出版：2021-09-16
移动端阅览
目的
2
脑肿瘤核磁共振（magnetic resonance，MR）图像分割对评估病情和治疗患者具有重要意义。虽然深度卷积网络在医学图像分割中取得了良好表现，但由于脑胶质瘤的恶性程度与外观表现有巨大差异，脑肿瘤MR图像分割仍是一项巨大挑战。图像语义分割的精度取决于图像特征的提取和处理效果。传统的U-Net网络以一种低效的拼接方式集成高层次特征和低层次特征，从而导致图像有效信息丢失，此外还存在未能充分利用上下文信息和空间信息的问题。对此，本文提出一种基于注意力机制和多视角融合U-Net算法，实现脑肿瘤MR图像的分割。
方法
2
在U-Net的解码和编码模块之间用多尺度特征融合模块代替传统的卷积层，进行多尺度特征映射的提取与融合；在解码模块的级联结构中添加注意力机制，增加有效信息的权重，避免信息冗余；通过融合多个视角训练的模型引入3维图像的空间信息。
结果
2
提出的模型在BraTS18（Multimodal Brain Tumor Segmentation Challenge 2018）提供的脑肿瘤MR图像数据集上进行验证，在肿瘤整体区域、肿瘤核心区域和肿瘤增强区域的Dice score分别为0.907、0.838和0.819，与其他方法进行对比，较次优方法分别提升了0.9%、1.3%和0.6%。
结论
2
本文方法改进了传统U-Net网络提取和利用图像语义特征不足的问题，并引入了3维MR图像的空间信息，使得肿瘤分割结果更加准确，具有良好的研究和应用价值。
Objective
2
Magnetic resonance (MR) image segmentation of brain tumors is crucial for patient evaluation and treatment. In recent years
as the feature extraction capability of convolutional neural networks improved
deep learning technology has been applied to medical image segmentation and achieved better results than those of traditional segmentation methods. The accuracy of image semantic segmentation depends on the effect of semantic feature extraction and processing. Traditional U-Net integrates high-level and low-level features inefficiently
leading to a loss of effective image information. In addition
U-Net does not make full use of context information. This study proposes a segmentation method for brain tumor MR images based on the attention mechanism and multi-view fusion U-Net algorithm.
Method
2
The model is modified as follows. First
in order to improve the network structure
residual structure is added to the U-Net to enhance the compensation of the network low-level information to the high-level information. On the one hand
the application of the residual idea
removes the same main part of the output of each convolution layer
highlights the small changes
makes each layer of the network more sensitive to the changes of the output
and makes the training easier
on the other hand
it alleviates the problem of gradient disappearance. Second
attention mechanism is added to the cascade structure
and the weight of tumor region is increased adaptively to enhance the utilization of effective information and improve the segmentation accuracy. Third
multi-scale feature fusion module is used to replace the traditional convolution layer between the down sampling structure and up sampling structure
and hole convolution with different sampling rates is used to extract and fuse the multi-scale feature information of the image in parallel. The above is the adjustment of the network structure
and the influence of loss function and 3D structure of Brain tumor MR data set on model training is also considered in the experiment. There is a class imbalance problem in brain tumor segmentation
tumor area is smaller than normal brain tissue
network training is easily guided by a large number of irrelevant pixels
and linear combination of generalized dice loss and cross entropy loss is used to solve class imbalance problem and accelerate convergence. At the same time
Considering the influence of different view slices on the segmentation performance of the model
the three view slices of 3D MR image are trained respectively
and the multi view training model is fused in the segmentation prediction to improve the segmentation performance of the model.
Result
2
The proposed model is validated using the brain tumor MR image data set provided by Multi- modal Brain Tumor Segmentation Challenge 2018 (BraTS18)
which includes four kinds of MR images of 210 high-grade gliomas(HGG) patients and 75 low-grade gliomas(LGG) patients and their real segmentation labels. The Dice similarity coefficient and Hausdorff distance95 are selected as technical indicators to further evaluate the accuracy of the brain tumor segmentation results. The effectiveness of the proposed module is proved by ablation experiments. After adding all modules
the performance of the model is optimal
the Dice scores of the entire tumor area
the core region
and the enhanced region reach 0. 883
0. 812
and 0.774. Compared with the traditional U-Net
it is improved by 3.9%
5.1% and 3.3% respectively
especially in the core region. After the fusion of three perspective slice training
comprehensive experiments show that the Dice scores of the entire tumor area
the core region
and the enhanced region reach 0. 907
0. 838
and 0. 819 respectively
this algorithm exhibits better performance in terms of Dice score and Hausdorff 95 distance than others. At the same time
the slices showthe slice shows the segmentation comparison with other classical models such as FCN. In the HGG sample
the proposed method has more delicate segmentation effect in the tumor boundary area. However
the segmentation results of the LGG sample is not as good as the HGG sample
this is due to the small number of LGG training samples
which is only one third of HGG samples
proposed method is still performs better than other methods.
Conclusion
2
The segmentation method of brain tumor MR images proposed in this study improves the short- comings of the traditional U-Net network in extracting and using image semantic features
and introduces attention mechanism and multi-scale feature fusion module. The weighted mixed loss function is used to solve the class imbalance problem in brain tumor segmentation
and considering the spatial characteristics of the dataset
the multi-view model is fused to improve the segmentation performance. Experimental results show that the algorithm has good performance in the segmentation of different tumor regions and has stronger robustness
thus providing a useful reference for the clinical application of brain tumor MR image segmentation.
Ahmed S F, Rahman F S, Tabassum T and Bhuiyan T I. 2019. 3D U-Net: fully convolutional neural network for automatic brain tumor segmentation//Proceedings of the 22nd International Conference on Computer and Information Technology (ICCIT). Dhaka, Bangladesh: IEEE: 1-6[ DOI:10.1109/ICCIT48885.2019.9038237 http://dx.doi.org/10.1109/ICCIT48885.2019.9038237 ]
Bakas S, Akbari H, Sotiras A, Bilello M, Rozycki M, Kirby J S, Freymann J B, Farahani K and Davatzikos C. 2017. Advancing the cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features. Scientific Data, 4: #170117[DOI:10.1038/sdata.2017.117]
Banerjee S, Mitra S and Shankar B U. 2019. Multi-planar spatial-ConvNet for segmentation and survival prediction in brain cancer//Proceedings of the 4th International Workshop on Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Granada, Spain: Springer: 94-104[ DOI:10.1007/978-3-030-11726-9_9 http://dx.doi.org/10.1007/978-3-030-11726-9_9 ]
Chen L C, Papandreou G, Kokkinos I, Murphy K and Yuille A L. 2018. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4): 834-848[DOI:10.1109/TPAMI.2017.2699184]
Chen W, Liu B Q, Peng S T, Sun J W and Qiao X. 2019. S3D-UNet: separable 3D U-Net for brain tumor segmentation//Proceedings of the 4th International Workshop on Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Granada, Spain: Springer: 358-368[ DOI:10.1007/978-3-030-11726-9_32 http://dx.doi.org/10.1007/978-3-030-11726-9_32 ]
Hamaguchi R, Fujita A, Nemoto K, Imaizumi T and Hikosaka S. 2018. Effective use of dilated convolutions for segmenting small object instances in remote sensing imagery//Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). Lake Tahoe, USA: IEEE: 1442-1450[ DOI:10.1109/WACV.2018.00162 http://dx.doi.org/10.1109/WACV.2018.00162 ]
He K M, Zhang X Y, Ren S Q and Sun J. 2015. Delving deep into rectifiers: surpassing human-level performance on ImageNet classification//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE: 1026-1034[ DOI:10.1109/ICCV.2015.123 http://dx.doi.org/10.1109/ICCV.2015.123 ]
Huang F Q, Chen C X and Wu W J. 2007. Application of PSO algorithm to brain tumor image division. Journal of Henan University of Science and Technology (Natural Science), 28(6): 97-99
黄峰茜, 陈春晓, 吴文佳. 2007. 粒子群优化算法在脑部肿瘤图像分割中的应用. 河南科技大学学报(自然科学版), 28(6): 97-99)[DOI:10.3969/j.issn.1672-6871.2007.06.028]
Iftekharuddin K M, Zheng J, Islam M A and Ogg R J. 2009. Fractal-based brain tumor detection in multimodal MRI. Applied Mathematics and Computation, 207(1): 23-41[DOI:10.1016/j.amc.2007.10.063]
Islam M, Jose V J M and Ren H L. 2019. Glioma prognosis: segmentation of the tumor and survival prediction using shape, geometric and clinical information//Proceedings of the 4th International Workshop on Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Granada, Spain: Springer: 142-153[ DOI:10.1007/978-3-030-11726-9_13 http://dx.doi.org/10.1007/978-3-030-11726-9_13 ]
Liu Y, Wang X P, Wang J Q and Yu H. 2015. Watershed algorithm for brain tumor segmentation based on morphological reconstruction and gradient layered modification. Application Research of Computers, 32(8): 2487-2491
刘岳, 王小鹏, 王金全, 于挥. 2015. 基于形态学重建和梯度分层修正的分水岭脑肿瘤分割. 计算机应用研究, 32(8): 2487-2491)[DOI:10.3969/j.issn.1001-3695.2015.08.060]
Ma J and Yang X P. 2019. Automatic brain tumor segmentation by exploring the multi-modality complementary information and cascaded 3D lightweight CNNs//Proceedings of the 4th International Workshop on Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Granada, Spain: Springer: 25-36[ DOI:10.1007/978-3-030-11726-9_3 http://dx.doi.org/10.1007/978-3-030-11726-9_3 ]
Menze B H, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby J, Burren Y, Porz N, Slotboom J, Wiest R, Lanczi L, Gerstner E, Weber M A, Arbel T, Avants B B, Ayache N, Buendia P, Collins D L, Cordier N, Corso J J, Criminisi A, Das T, Delingette H, Demiralp ç, Durst C R, Dojat M, Doyle S, Festa J, Forbes F, Geremia E, Glocker B, Golland P, Guo X T, Hamamci A, Iftekharuddin K M, Jena R, John N M, Konukoglu E, Lashkari D, Mariz J A, Meier R, Pereira S, Precup D, Price S J, Raviv T R, Reza S M S, Ryan M, Sarikaya D, Schwartz L, Shin H C, Shotton J, Silva C A, Sousa N, Subbanna N K, Szekely G, Taylor T J, Thomas O M, Tustison N J, Unal G, Vasseur F, Wintermark M, Ye D H, Zhao L, Zhao B S, Zikic D, Prastawa M, Reyes M and Van Leemput K. 2015. The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Transactions on Medical Imaging, 34(10): 1993-2024[DOI:10.1109/TMI.2014.2377694]
Milletari F, Navab N and Ahmadi S A. 2016. V-Net: fully convolutional neural networks for volumetric medical image segmentation//Proceedings of the 4th International Conference on 3D Vision. Stanford, USA: IEEE: 565-571[ DOI:10.1109/3DV.2016.79 http://dx.doi.org/10.1109/3DV.2016.79 ]
Noori M, Bahri A and Mohammadi K. 2019. Attention-guided version of 2D UNet for automatic brain tumor segmentation//Proceedings of the 9th International Conference on Computer and Knowledge Engineering (ICCKE). Mashhad, Iran: IEEE: 269-275[ DOI:10.1109/ICCKE48569.2019.8964956 http://dx.doi.org/10.1109/ICCKE48569.2019.8964956 ]
Ronneberger O, Fischer P and Brox T. 2015. U-Net: convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: Springer: 234-241[ DOI:10.1007/978-3-319-24574-4_28 http://dx.doi.org/10.1007/978-3-319-24574-4_28 ]
Sang L Q, Qiu M G, Wang L, Zhang J N and Zhang Y. 2010. Brain tumor MRI image segmentation based on statistical thresholding method. Journal of Biomedical Engineering Research, 29(4): 237-239
桑林琼, 邱明国, 王莉, 张静娜, 张晔. 2010. 基于统计阈值的脑肿瘤MRI图像的分割方法. 生物医学工程研究, 29(4): 237-239)[DOI:10.3969/j.issn.1672-6278.2010.04.004]
Shelhamer E, Long J and Darrell T. 2017. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4): 640-651[DOI:10.1109/TPAMI.2016.2572683]
Shen H C, Wang R X, Zhang J G and Mckenna S. 2017. Multi-task fully convolutional network for brain tumour segmentation//Proceedings of the 21st Annual Conference on Medical Image Understanding and Analysis. Edinburgh, UK: Springer: 239-248[ DOI:10.1007/978-3-319-60964-5_21 http://dx.doi.org/10.1007/978-3-319-60964-5_21 ]
Sudre C H, Li W Q, Vercauteren T, Ourselin S and Cardoso M J. 2017. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations//Proceedings of the 3rd MICCAI International Workshop on Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Québec City, Canada: Springer: 240-248[ DOI:10.1007/978-3-319-67558-9_28 http://dx.doi.org/10.1007/978-3-319-67558-9_28 ]
Tustison N J, Avants B B, Cook P A, Zheng Y J, Egan A, Yushkevich P A and Gee J C. 2010. N4ITK: improved N3 bias correction. IEEE Transactions on Medical Imaging, 29(6): 1310-1320[DOI:10.1109/TMI.2010.2046908]
Wang W, Yu K C, Hugonot J, Fua P and Salzmann M. 2019. Recurrent U-Net for resource-constrained segmentation//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Korea (South): IEEE: 2142-2151[ DOI:10.1109/ICCV.2019.00223 http://dx.doi.org/10.1109/ICCV.2019.00223 ]
Zhou B L, Khosla A, Lapedriza A, Oliva A and Torralba A. 2016. Learning deep features for discriminative localization//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE: 2921-2929[ DOI:10.1109/CVPR.2016.319 http://dx.doi.org/10.1109/CVPR.2016.319 ]
相关作者
相关机构
京公网安备11010802024621
