浏览全部资源
扫码关注微信
纸质出版日期： 2021-02-16 ，
录用日期： 2020-05-07
移动端阅览
刘畅, 林楠, 曹仰杰, 杨聪. Seg-CapNet：心脏MRI图像分割神经网络模型[J]. 中国图象图形学报, 2021,26(2):452-463.
Chang Liu, Nan Lin, Yangjie Cao, Cong Yang. Seg-CapNet: neural network model for the cardiac MRI segmentation[J]. Journal of Image and Graphics, 2021,26(2):452-463.
目的
2
针对现有神经网络模型需要对左心室心肌内膜和外膜单独建模的问题，本文提出了一种基于胶囊结构的心脏磁共振图像（magnetic resonance imaging，MRI）分割模型Seg-CapNet，旨在同时提取心肌内膜和外膜，并保证两者的空间位置关系。
方法
2
首先利用胶囊网络将待分割目标转换成包含目标相对位置、颜色以及大小等信息的向量，然后使用全连接将这些向量的空间关系进行重组，最后采用反卷积对特征图进行上采样，将分割图还原为输入图像尺寸。在上采样过程中将每层特征图与卷积层的特征图进行连接，有助于图像细节还原以及模型的反向传播，加快训练过程。Seg-CapNet的输出向量不仅有图像的灰度、纹理等底层图像特征，还包含目标的位置、大小等语义特征，有效提升了目标图像的分割精度。为了进一步提高分割质量，还提出了一种新的损失函数用于约束分割结果以保持多目标区域间的相对位置关系。
结果
2
在ACDC（automated cardiac diagnosis challenge）2017、MICCAI（medical image computing and computer-assisted intervention）2013和MICCAI2009等3个心脏MRI分割竞赛的公开数据集上对Seg-CapNet模型进行训练和验证，并与神经网络分割模型U-net和SegNet进行对比。实验结果表明，相对于U-Net和SegNet，Seg-CapNet同时分割目标重叠区域的平均Dice系数提升了3.5%，平均豪斯多夫距离（Hausdorff distance，HD）降低了18%。并且Seg-CapNet的参数量仅为U-Net的54%、SegNet的40%，在提升分割精度的同时，降低了训练时间和复杂度。
结论
2
本文提出的Seg-CapNet模型在保证同时分割重叠区域目标的同时，降低了参数量，提升了训练速度，并保持了较好的左心室心肌内膜和外膜分割精度。
Objective
2
Image segmentation tasks suffer from the problem in which multiple overlapping regions are required to be extracted
such as the division of the endocardium and epicardium of the heart's left ventricle. Existing neural network segmentation models typically segment the target based on pixel classification due to the overlapping of pixels in the two regions and then convert the segmentation problem into a classification problem. However
the overlapping area of pixels may not be simultaneously classified well. In general
existing neural networks must train model parameters for each target to obtain accurate segmentation results
reducing segmentation efficiency. To address these issues
we propose a segmentation model
called Seg-CapNet
which is based on a capsule network structure.
Method
2
Current segmentation models based on convolutional neural networks control the size of feature maps through operations
such as maximum or average pool
and transmit image feature information from the upper layer to the next layer. Such pooling operations lose the spatial information of components in the process of information transmission. Therefore
the proposed Seg-CapNet model uses a capsule network structure to extract vectors that contain spatial
color
size
and other target information. Compared with current network structures
the output of a capsule network is in vector form
and the information of the target is included in the entity vector through routing iteration. Seg-CapNet utilizes this feature to strip overlapping objects from the image space and convert them into noninterference feature vectors
separating objects with overlapping regions. Then
the spatial position relation of multiple target vectors are reconstructed using fully connected layers. Lastly
the reconstructed image is up-sampled and the segmented image is restored to the same size as the input image. During up-sampling
the feature graph of the up-sampling layer and that of the convolutional layer are skip-connected. This process is conducive to restoring image details and accelerating the training process while the model is backpropagating. To improve segmentation results
we also design a new loss function for constraining segmentation results to ensure that they can maintain a relative position relationship among multiple target areas to follow cardiac morphology. In the loss function based on the Dice coefficient
the ratio constraint of the area beyond the epicardium boundary to the area of the endocardium is added
and thus
the area of the endocardium is divided as far as possible within the outer membrane. To prevent the ratio from becoming too small to influence parameter updating in the backpropagation process
we control its value within an appropriate range through exponential transformation and keep it synchronized with the loss function based on the Dice coefficient. This method is implemented using Python 3.6 and TensorFlow on Nvidia Tesla K80 GPU
Intel E5-2650 CPU
and 10 G main memory. The learning rate is 0.001. Image sizes from different devices are inconsistent because data sources are collected from different imaging devices. However in cardiac magnetic resonance imaging (MRI)
the heart is typically located near the center. Therefore
the 128×128 pixel region centered on an image is extracted as the size of the input model image
and image size can be unified
including the image of the whole heart.
Result
2
We train and verify the Seg-CapNet model on the automated cardiac diagnosis challenge(ACDC)2017
medical image computing and computer-assisted intervention(MICCAI)2013
and MICCAI2009 datasets
and then compare the results with those of the neural network segmentation models
U-Net and SegNet. Experimental results show that the average Dice coefficient of our model increased by 4.7% and the average Hausdorff distance decreased by 22% compared with those of U-Net and SegNet. Moreover
the number of Seg-CapNet parameters was only 54% of that of U-Net and 40% of that of SegNet. Our results illustrate that the proposed model improves segmentation accuracy and reduces training time and complexity. In addition
we validate the performance of the proposed loss function on the ACDC2017 dataset. By comparing the segmentation results of the model before and after random selection and adding the constraint loss function
the new loss function avoids the internal region located outside the epicardium
violating the anatomical structure of the heart. Simultaneously
we calculate the mean Dice value of the segmentation results before and after adding the constraint to the loss function. The experimental results show that Dice value of the segmentation results of the left ventricular endocardium and epicardium with the new loss function increases by an average of 0.6%.
Conclusion
2
We propose a Seg-CapNet model that ensures the simultaneous segmentation of multiple overlapping targets
reduces the number of participants
and accelerates the training process. The results show that our model can maintain good segmentation accuracy while segmenting two overlapping regions of the heart's left ventricle in MRI.
神经网络胶囊网络图像分割重叠区域目标心脏磁共振图像
neural networkcapsule networkimage segmentationoverlapping-area targetcardiac MRI
Badrinarayanan V, Handa V and Cipolla R. 2015. SegNet: a deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling[EB/OL].[2019-11-13].https://arxiv.org/pdf/1505.07293.pdfhttps://arxiv.org/pdf/1505.07293.pdf
Badrinarayanan V, Kendall A and Cipolla R. 2017. SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12): 2481-2495[DOI: 10.1109/TPAMI.2016.2644615]
Cootes T F, Edwards G J and Taylor C J. 2001. Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6): 681-685[DOI: 10.1109/34.927467]
Cootes T F, Taylor C J, Cooper D H and Graham J. 1995. Active shape models-their training and application. Computer Vision and Image Understanding, 61(1): 38-59[DOI: 10.1006/cviu.1995.1004]
Duong C N, Luu K, Quach K G and Bui T D. 2019. Deep appearance models: a deep boltzmann machine approach for face modeling. International Journal of Computer Vision, 127(5): 437-455[DOI: 10.1007/s11263-018-1113-3]
Hesamian M H, Jia W J, He X J and Kennedy P. 2019. Deep learning techniques for medical image segmentation: achievements and challenges. Journal of Digital Imaging, 32(4): 582-596[DOI: 10.1007/s10278-019-00227-x]
Hu H F, Liu H H, Gao Z Y and Huang L. 2013. Hybrid segmentation of left ventricle in cardiac MRI using Gaussian-mixture model and region restricted dynamic programming. Magnetic Resonance Imaging, 31(4): 575-584[DOI: 10.1016/j.mri.2012.10.004]
Ioffe S and Szegedy C. 2015. Batch normalization: accelerating deep network training by reducing internal covariate shift[EB/OL].[2019-11-15].https://arxiv.org/pdf/1502.03167v3.pdfhttps://arxiv.org/pdf/1502.03167v3.pdf
Isensee F, Jaeger P F, Full P M, Wolf I, Engelhardt S and Maier-Hein K H. 2017. Automatic cardiac disease assessment on cine-MRI via time-series segmentation and domain specific features//Proceedings of the 8th International Workshop on Statistical Atlases and Computational Models of the Heart. Quebec City, Canada: Springer: 120-129[DOI:10.1007/978-3-319-75541-0_13http://dx.doi.org/10.1007/978-3-319-75541-0_13]
Jiang Z K, Lyu X G, Zhang J X, Zhang Q and Wei X P. 2020. Review of deep learning methods for MRI brain tumor image segmentation. Journal of Image and Graphics, 25(2): 215-228
江宗康, 吕晓钢, 张建新, 张强, 魏小鹏. 2020. MRI脑肿瘤图像分割的深度学习方法综述.中国图象图形学报, 25(2): 215-228)[DOI: 10.11834/jig.190173]
Krizhevsky A, Sutskever I and Hinton G E. 2012. ImageNet classification with deep convolutional neural networks//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada, USA: ACM: 1097-1105
LeCun Y, Bengio Y and Hinton G. 2015. Deep learning. Nature, 521(7553): 436-444[DOI: 10.1038/nature14539]
Liu F C, Xu L Y, Sun Q S and Xia D S. 2010. Segmentation of left ventricle from Tagged MR images based on ASM and feature fusion strategy. Computer Engineering and Applications, 46(10): 160-164
刘复昌, 徐丽燕, 孙权森, 夏德深. 2010.结合ASM及特征融合策略的Tagged MR左心室分割.计算机工程与应用, 46(10): 160-164)[DOI: 10.3778/j.issn.1002-8331.2010.10.051]
Liu H, Hu H F, Xu X Y and Song E M. 2012. Automatic leftventricle segmentation in cardiac MRI using topological stable-state thresholding and region restricted dynamic programming. Academic Radiology, 19(6): 723-731[DOI: 10.1016/j.acra.2012.02.011]
Livne M, Rieger J, Aydin O U, Taha A A, Akay E M, Kossen T, Sobesky J, Kelleher J D, Hildebrand K, Frey D and Madai V I. 2019. A U-Net deep learning framework for high performance vessel segmentation in patients with cerebrovascular disease. Frontiers in Neuroscience, 13: #97[DOI: 10.3389/fnins.2019.00097]
Long J, Shelhamer E and Darrell T. 2015. Fully convolutional networks for semantic segmentation//Proceedings of 2015 IEEE conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE: 3431-3440[DOI:10.1109/CVPR.2015.7298965http://dx.doi.org/10.1109/CVPR.2015.7298965]
Noh H, Hong S and Han B. 2015. Learning deconvolution network for semantic segmentation//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE: 1520-1528[DOI:10.1109/ICCV.2015.178http://dx.doi.org/10.1109/ICCV.2015.178]
Patravali J, Jain S and Chilamkurthy S. 2017. 2D-3D Fully convolutional neural networks for cardiac MR segmentation//Proceedings of the 8th International Workshop on Statistical Atlases and Computational Models of the Heart. Quebec City, Canada: Springer: 130-139[DOI:10.1007/978-3-319-75541-0_14http://dx.doi.org/10.1007/978-3-319-75541-0_14]
Queiros S, Barbosa D, Heyde B, Morais P, Vilaça J L, Friboulet D, Bernard O and D'hooge J. 2014. Fast automatic myocardial segmentation in 4D cine CMR datasets. Medical Image Analysis, 18(7): 1115-1131[DOI: 10.1016/j.media.2014.06.001]
Ronneberger O, Fischer P and Brox T. 2015. U-Net: convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: Springer: 234-241[DOI:10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28]
Sabour S, Frosst N and Hinton G E. 2017. Dynamic routing between capsules//Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems. Long Beach, USA: [s.n.]: 3856-3866
Sedai S, Garnavi R, Roy P and Liang X. 2015. Multi-atlas label fusion using hybrid of discriminative and generative classifiers for segmentation of cardiac MR images//Proceedings of the 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Milan, Italy: IEEE: 2977-2980[DOI:10.1109/EMBC.2015.7319017http://dx.doi.org/10.1109/EMBC.2015.7319017]
Soliman A, Khalifa F, Elnakib A, Abou El-Ghar M, Dunlap N, Wang B, Gimel'farb G, Keynton R and El-Baz A. 2017. Accurate lungs segmentation on CT chest images by adaptive appearance-guided shape modeling. IEEE Transactions on Medical Imaging, 36(1): 263-276[DOI: 10.1109/TMI.2016.2606370]
Viergever M A, Maintz J B A, Klein S, Murphy K, Staring M and Pluim J P W. 2016. A survey of medical image registration-under review. Medical Image Analysis, 33: 140-144[DOI: 10.1016/j.media.2016.06.030]
Wachinger C, Fritscher K, Sharp G and Golland P. 2015. Contour-driven atlas-based segmentation. IEEE Transactions on Medical Imaging, 34(12): 2492-2505[DOI: 10.1109/TMI.2015.2442753]
Wang X J, Wang M H, Li A, Min J, Dong L N and Feng H Q. 2013. Left ventricle segmentation based on improved multi-scale ASM and non-rigid registration from 4D-CT dataset. Journal of University of Science and Technology of China, 43(4): 319-325
王兴家, 王明会, 李骜, 闵捷, 董利娜, 冯焕清. 2013.基于改进多尺度ASM和非刚性配准的4D-CT左心室分割.中国科学技术大学学报, 43(4): 319-325)[DOI: 10.3969/j.issn.0253-2778.2013.04.010]
Wolterink J M, Leiner T, Viergever M A and Išgum I. 2017. Automatic segmentation and disease classification using cardiac cine MR images//Proceedings of the 8th International Workshop on Statistical Atlases and Computational Models of the Heart. Quebec City, Canada: Springer: 101-110[DOI:10.1007/978-3-319-75541-0_1http://dx.doi.org/10.1007/978-3-319-75541-0_1]
Zhan S, Chang H, Jiang J G and Ando S. 2008. Improved 3D AAMs for facial recognition based CIS 3D facial imaging. Journal of Image and Graphics, 13(10): 2059-2062
詹曙, 常虹, 蒋建国, Ando S. 2008.基于相关型图像传感器3维人脸成像的3维AAMs人脸识别方法的研究.中国图象图形学报, 13(10): 2059-2062)[DOI: 10.11834/jig.20081060]
Zhu K, Fu Z L and Chen X Q. 2019. Left ventricular segmentation method of ultrasound image based on convolutional neural network. Journal of Computer Applications, 39(7): 2121-2124
朱锴, 付忠良, 陈晓清. 2019.基于卷积神经网络的超声图像左心室分割方法.计算机应用, 39(7): 2121-2124)[DOI: 10.11772/j.issn.1001-9081.2018112321]
相关作者
相关机构
京公网安备11010802024621