浏览全部资源
扫码关注微信
收稿：2020-05-22，
修回：2020-6-29，
录用：2020-7-5，
纸质出版：2020-10-16
移动端阅览
目的
2
从医学影像中进行肝脏与肿瘤分割是计算机辅助诊断和治疗的重要前提。常见的胸部和腹部扫描成像效果中，图像对比度偏低，边界模糊，需要医生丰富的临床解剖学知识才能准确地分割，所以精确的自动分割是一个极大的挑战。本文结合深度学习与医学影像组学，提出一种肝脏肿瘤CT(computed tomography)分割方法。
方法
2
首先建立一个级联的2D图像端到端分割模型对肝脏和肿瘤同时进行分割，分割模型采用U-Net深度网络框架，在编码器与解码器内部模块以及编码器与解码器层次之间进行密集连接，这种多样化的特征融合可以获取更准确的全局位置特征和更丰富的局部细节纹理特征；同时融入子像素卷积与注意力机制，有利于分割出更加微小的肿瘤区域；接着生成两个用于后处理的学习模型，一个基于影像组学的分类模型用于假阳性肿瘤的去除；另一个基于3D体素块的分类模型用于分割边缘的细化。
结果
2
实验数据来自某医院影像科300个肝癌病例CT，每个序列中的肝脏与肿瘤都是由10年以上的医学专家进行分割标注。对数据进行5倍交叉验证，敏感度(sensitivity)、命中率(positive predicted value)和戴斯系数(Dice coefficient)在验证结果中的平均值分别达到0.87±0.03、0.91±0.03和0.86±0.05，相比于性能第2的模型分别提高了0.03、0.02和0.04。
结论
2
肝脏肿瘤CT的精确分割可以形成有价值的术前预判、术中监测和术后评价，有助于制定完善的手术治疗方案，提高肝脏肿瘤手术的成功率，且该方法不局限于肝脏肿瘤的分割，同样也适用于其他医学影像组织器官与肿瘤的分割。
Objective
2
In clinical diagnosis
the manual segmentation of liver tumor needs to have anatomical knowledge and experience
which is highly subjective. Automatic segmentation of liver tumor based on computed tomography(CT) medical image research is important in guiding clinical diagnosis and treatment. It can form accurate preoperative prediction
intraoperative monitoring
and postoperative evaluation
which develop a perfect surgical treatment plan and improve the success rate of liver tumor surgery. However
a typical medical image is complex
which may contain many organs and tissues. In the imaging process
it often produces more or less interference information due to the influence of imaging equipment and human factors. According to the imaging principle and characteristics of medical imaging
not all information may be helpful for medical diagnosis. Whether CT or magnetic resonance imaging(MRI)
as the common chest and abdomen scanning
in the imaging effect
the image contrast is low; the boundary is fuzzy; most of the time
accurate segmentation of interested organs and corresponding tumors requires clinical anatomy knowledge of doctors. In general
many boundaries are achieved through consultation between medical personnel. Notably
the automatic and accurate segmentation of liver tumor in medical images is a great challenge. In this paper
an effective segmentation method is proposed by combining multiple deep learning techniques with radiomics.
Method
2
First
a cascaded 2D image end-to-end segmentation model is established to segment the liver and tumor simultaneously. The segmentation model adopts the U-Net deep network framework
which connects the internal modules of the encoder and decoder and the layers of the encoder and decoder intensively. This diversified feature fusion can obtain accurate global position features and abundant local details or physical characteristics. Sub-pixel convolution is added during upsampling by pixel shuffling multiple low-resolution feature images to output the high-resolution image and make up for the loss of some image texture details. Given the small area and volume of the small tumor area
feature disappearance or weakening can easily occur in the convolutional neural network. A new attention gate model for medical imaging is used here to improve the segmentation performance of the tumor. This model can automatically learn the target with different volumes and shapes
which is suitable for liver tumor with the distribution characteristics of volume and shape polymorphism. Morphological expansion and corrosion may lead to the loss of some small areas
which are not continuous. Considering that CT scan is continuous in a short time
the lost areas can be recovered by using the continuity of this image in time. If there are segmentation regions in the former and latter image
but the current image does not
then the average value of the segmentation results before and after the current image is used. Given the heterogeneity of malignant tumors
the radiomic characteristics can be an effective
non-invasive imaging biomarker. In this paper
the radiomic features are divided into eight groups: histogram statistical features
gradient features
run-length texture features
gray-level co-occurrence matrix texture features
shape-based features
second-order moment features
boundary features
and three-dimensional wavelet features. According to a known research scope
considerable domestic and foreign research literature of radiomics show that the comprehensive performance of the random forest model is the best
and the random forest classifier is also selected to remove the false-positive tumor. Some segmentation areas are more accurate after morphological operation
but some segmentation areas will be larger than the real results. Therefore
the edge contour must be further refined. Here
the classification based on 3D image blocks does not need complex convolutional neural network(CNN)
and simple CNN can achieve a good classification effect. The AlexNet model framework is selected and adjusted accordingly to refine the contour.
Result
2
The experimental data are from 300 cases of liver cancer in the imaging department of the cooperative hospital. The liver and tumor in each sequence are segmented and labeled by medical experts for more than 10 years. The average accuracy of sensitivity
positive predicted value
and Dice coefficient in the validation results are 0.87 ±0.03
0.91 ±0.03
and 0.86 ±0.05
respectively
which are improved by 0.03
0.02
and 0.04
respectively
compared with the model with the second performance. The training process of the method proposed in this paper is not end-to-end but is completed by three learning models
which are generally carried out on the GPU server under the Linux system. After the model training
the image can be arranged in the server or the general PC client. If the image is in the server
the image needs to request and transmit the sequence image through the network and then return to the client after segmentation. A typical way is to upload the image to the server and perform offline batch segmentation. After processing
only the segmentation result data will be returned.
Conclusion
2
In clinical diagnosis
the automatic or semi-automatic segmentation of the liver and tumor in CT medical image by using artificial intelligence and computer vision technology gradually replaces the pure artificial operation mode. The method proposed in this paper is not limited to the segmentation of liver tumors but also applicable to the segmentation of other medical imaging organs and tumors. In the future research
we will integrate the generative adversarial network technology to improve the segmentation performance.
Aerts H J W L, Velazquez E R, Leijenaar R T H, Parmar C, Grossmann P, Carvalho S, Bussink J, Monshouwer R, Haibe-Kains B, Rietveld D, Hoebers F, Rietbergen M M, Leemans C R, Dekker A, Quackenbush J, Gillies R J and Lambin P. 2014. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nature Communications, 5(1):1-9[DOI:10.1038/ncomms5006]
Afshar P, Mohammadi A, Plataniotis K N, Oikonomou A and Benali H. 2019. From handcrafted to deep-learning-based cancer radiomics:challenges and opportunities. IEEE Signal Processing Magazine, 36(4):132-160[DOI:10.1109/MSP.2019.2900993]
Bilic P, Christ P F, Vorontsov E, Chlebus G, Chen H, Dou Q, Fu C W, Han X, Heng P A, Hesser J, Kadoury S, Konopczynski T, Le M, Li C M, Li X M, LipkovàJ, Lowengrub J, Meine H, Moltz J H, Pal C, Piraud M, Qi X J, Qi J, Rempfler M, Roth K, Schenk A, Sekuboyina A, Vorontsov E, Zhou P, Hülsemeyer C, Beetz M, Ettlinger F, Gruen F, Kaissis G, Lohöfer F, Braren R, Holch J, Hofmann F, Sommer W, Heinemann V, Jacobs C, Mamani G E H, van Ginneken B, Chartrand G, Tang A, Drozdzal M, Ben-Cohen A, Klang E, Amitai M M, Konen E, Greenspan H, Moreau J, Hostettler A, Soler L, Vivanti R, Szeskin A, Lev-Cohain N, Sosna J, Joskowicz L and Menze B H. 2019. The liver tumor segmentation benchmark (LiTS)[EB/OL].[2020-05-22] . https://arxiv.org/pdf/1901.04056.pdf https://arxiv.org/pdf/1901.04056.pdf
Chen L, Song H, Wang C, Cui Y T, Yang J, Hu X H and Zhang L. 2019. Liver tumor segmentation in CT volumes using an adversarial densely connected network. BMC Bioinformatics, 20(S16):#587[DOI:10.1186/s12859-019-3069-x]
Choudhary A, Moretto N, Ferrarese F P and Zamboni G A. 2008. An entropy based multi-thresholding method for semi-automatic segmentation of liver tumors. MICCAI workshop, 41(43):43-49
ÇiçekÖ, Abdulkadir A, Lienkamp S S, Brox T and Ronneberger O. 2016.3D U-Net: learning dense volumetric segmentation from sparse annotation//Proceedings of the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention. Athens: Springer: 424-432[ DOI: 10.1007/978-3-319-46723-8_49 http://dx.doi.org/10.1007/978-3-319-46723-8_49 ]
Cirillo M D, Abramian D and Eklund A. 2020. Vox2Vox: 3D-GAN for brain tumour segmentation[EB/OL].[2020-05-22] . https://arxiv.org/pdf/2003.13653.pdf https://arxiv.org/pdf/2003.13653.pdf
Conze P H, Noblet V, Rousseau F, Heitz F, de Blasi V, Memeo R and Pessaux P. 2017. Scale-adaptive supervoxel-based random forests for liver tumor segmentation in dynamic contrast-enhanced CT scans. International Journal of Computer Assisted Radiology and Surgery, 12(2):223-233[DOI:10.1007/s11548-016-1493-1]
Das A and Sabut S K. 2016. Kernelized fuzzy C-means clustering with adaptive thresholding for segmenting liver tumors. Procedia Computer Science, 92:389-395[DOI:10.1016/j.procs.2016.07.395]
Fang R G, Zabih R, Raj A and Chen T. 2011. Segmentation of liver tumor using efficient global optimal tree metrics graph cuts//Proceedings of the 3rd International MICCAI Workshop on Computational and Clinical Challenges in Abdominal Imaging. Toronto: Springer: 51-59[ DOI: 10.1007/978-3-642-28557-8_7 http://dx.doi.org/10.1007/978-3-642-28557-8_7 ]
Häme Y and Pollari M. 2012. Semi-automatic liver tumor segmentation with hidden Markov measure field model and non-parametric distribution estimation. Medical Image Analysis, 16(1):140-149[DOI:10.1016/j.media.2011.06.006]
He K M, Gkioxari G, Dollár P and Girshick R. 2020. Mask R-CNN. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2):386-397[DOI:10.1109/TPAMI.2018.2844175]
Hu R H, Dollár P, He K M, Darrell T and Girshick R. 2017. Learning to segment every thing//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 4233-4241[ DOI: 10.1109/cvpr.2018.00445 http://dx.doi.org/10.1109/cvpr.2018.00445 ]
Huang G, Liu Z, van der Maaten L and Weinberger K O. 2017. Densely connected convolutional networks//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 2261-2269[ DOI: 10.1109/cvpr.2017.243 http://dx.doi.org/10.1109/cvpr.2017.243 ]
Huang W M, Yang Y Z, Lin Z P, Huang G B, Zhou J Y, Duan Y P and Xiong W. 2014. Random feature subspace ensemble based extreme learning machine for liver tumor detection and segmentation//Proceedings of the 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Chicago: IEEE: 4675-4678[ DOI: 10.1109/embc.2014.6944667 http://dx.doi.org/10.1109/embc.2014.6944667 ]
Isola P, Zhu J Y, Zhou T H and Efros A A. 2017. Image-to-image translation with conditional adversarial networks//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 5967-5976[ DOI: 10.1109/cvpr.2017.632 http://dx.doi.org/10.1109/cvpr.2017.632 ]
Kadoury S, Vorontsov E and Tang A. 2015. Metastatic liver tumour segmentation from discriminant Grassmannian manifolds. Physics in Medicine and Biology, 60(16):6459-6478[DOI:10.1088/0031-9155/60/16/6459]
Korez R, Likar B, PernušF and Vrtovec T. 2016. Model-based segmentation of vertebral bodies from MR images with 3D CNNs//Proceedings of the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention. Athens: Springer: 433-441[ DOI: 10.1007/978-3-319-46723-8_50 http://dx.doi.org/10.1007/978-3-319-46723-8_50 ]
Krishan A and Mittal D. 2019. Effective segmentation and classification of tumor on liver MRI and CT images using multi-kernel K-means clustering. Biomedical Engineering/Biomedizinische Technik, 65(3):301-313[DOI:10.1515/bmt-2018-0175]
Lambin P, Leijenaar R T H, Deist T M, Peerlings J, de Jong E E C, van Timmeren J, Sanduleanu S, Larue R T H M, Even A J G, Jochems A, van Wijk Y, Woodruff H, van Soest J, Lustberg T, Roelofs E, van Elmpt W, Dekker A, Mottaghy F M, Wildberger J E and Walsh S. 2017. Radiomics:the bridge between medical imaging and personalized medicine. Nature Reviews Clinical Oncology, 14(12):749-762[DOI:10.1038/nrclinonc.2017.141]
Li B N, Chui C K, Chang S and Ong S H. 2012. A new unified level set method for semi-automatic liver tumor segmentation on contrast-enhanced CT images. Expert Systems with Applications, 39(10):9661-9668[DOI:10.1016/j.eswa.2012.02.095]
Li X M, Chen H, Qi X J, Dou Q, Fu C W and Heng P A. 2018. H-DenseUNet:hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Transactions on Medical Imaging, 37(12):2663-2674[DOI:10.1109/tmi.2018.2845918]
Li Y Z, Hara S, Ito W and Shimura K. 2007. A machine learning approach for interactive lesion segmentation//Proceedings of SPIE 6512, Medical Imaging 2007: Image Processing. San Diego: SPIE: 651246[ DOI: 10.1117/12.708910 http://dx.doi.org/10.1117/12.708910 ]
Liang D, Lin L F, Chen X, HU H J, Zhang Q W, Chen Q Q, Iwamoto Y, Han Y H, Chen Y W, Tong R F and Wu J. 2019. Multi-stream scale-insensitive convolutional and recurrent neural networks for liver tumor detection in dynamic Ct images//Proceedings of 2019 IEEE International Conference on Image Processing. Taipei: IEEE: 794-798[ DOI: 10.1109/icip.2019.8803730 http://dx.doi.org/10.1109/icip.2019.8803730 ]
Liao M, Liu Y Z, Ouyang J L, Yu J Y, Zhao Y Q and Zhang B Z. 2019. Automatic segmentation of liver tumor in CT volumes using nonlinear enhancement and graph cuts. Journal of Computer-Aided Design and Computer Graphics, 31(6):1030-1038
廖苗, 刘毅志, 欧阳军林, 余建勇, 赵于前, 张宝泽. 2019.基于非线性增强和图割的CT序列肝脏肿瘤自动分割.计算机辅助设计与图形学学报, 31(6):1030-1038[DOI:10.3724/sp.j.1089.2019.17258]
Lu R, Marziliano P and Thng C H. 2005. Liver tumor volume estimation by semi-automatic segmentation method//Proceedings of 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference. Shanghai: IEEE: 3296-3299[ DOI: 10.1109/iembs.2005.1617181 http://dx.doi.org/10.1109/iembs.2005.1617181 ]
Milletari F, Navab N and Ahmadi S A. 2016. V-Net: fully convolutional neural networks for volumetric medical image segmentation//Proceedings of the 4th International Conference on 3D Vision. Stanford: IEEE: 565-571[ DOI: 10.1109/3DV.2016.79 http://dx.doi.org/10.1109/3DV.2016.79 ]
Oktay O, Schlemper J, Le Folgoc L L, Lee M, Heinrich M, Misawa K, Mori K, McDonagh S, Hammerla N Y, Kainz B, Glocker B and Rueckert D. 2018. Attention u-net: learning where to look for the pancreas[EB/OL].[2020-05-22] . https://arxiv.org/pdf/1804.03999.pdf https://arxiv.org/pdf/1804.03999.pdf
Pang Y W, Hu D and Sun M. 2019. A modified scheme for liver tumor segmentation based on cascaded FCNs//Proceedings of 2019 International Conference on Artificial Intelligence, Information Processing and Cloud Computing. Sanya: ACM: 1-6[ DOI: 10.1145/3371425.3371451 http://dx.doi.org/10.1145/3371425.3371451 ]
Parekh V S and Jacobs M A. 2019. Radiomic synthesis using deep convolutional neural networks//Proceedings of 2019 IEEE International Symposium on Biomedical Imaging. Venice: IEEE: 1114-1117[ DOI: 10.1109/ISBI.2019.8759491 http://dx.doi.org/10.1109/ISBI.2019.8759491 ]
Pinheiro P O, Lin T Y and Collobert R Dollár P. 2016. Learning to refine object segments//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands: Springer: 75-91[ DOI: 10.1007/978-3-319-46448-0_5 http://dx.doi.org/10.1007/978-3-319-46448-0_5 ]
Rezaei M, Yang H J and Meinel C. 2018. Instance tumor segmentation using multitask convolutional neural network//Proceedings of 2018 International Joint Conference on Neural Networks. Rio de Janeiro: IEEE: 1-8[ DOI: 10.1109/ijcnn.2018.8489105 http://dx.doi.org/10.1109/ijcnn.2018.8489105 ]
Rezaei M, Yang H J and Meinel C. 2020. Recurrent generative adversarial network for learning imbalanced medical image semantic segmentation. Multimedia Tools and Applications, 79(21):15329-15348[DOI:10.1007/s11042-019-7305-1]
Ronneberger O, Fischer P and Brox T. 2015. U-net: convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: Springer: 234-241[ DOI: 10.1007/978-3-319-24574-4_28 http://dx.doi.org/10.1007/978-3-319-24574-4_28 ]
Shelhamer E, Long J and Darrell T. 2017. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4):640-651[DOI:10.1109/TPAMI.2016.2572683]
Shi W Z, Caballero J, Huszár F, Totz J, Aitken A P, Bishop R, Rueckert D and Wang Z H. 2016. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE: 1874-1883[ DOI: 10.1109/cvpr.2016.207 http://dx.doi.org/10.1109/cvpr.2016.207 ]
Vorontsov E, Abi-Jaoudeh N and Kadoury S. 2014. Metastatic liver tumor segmentation using texture-based omni-directional deformable surface models//Proceedings of the 6th International MICCAI Workshop on Computational and Clinical Challenges in Abdominal Imaging. Cambridge: Springer: 74-83[ DOI: 10.1007/978-3-319-13692-9_7 http://dx.doi.org/10.1007/978-3-319-13692-9_7 ]
Vorontsov E, Tang A, Pal C and Kadoury S. 2018. Liver lesion segmentation informed by joint liver segmentation//Proceedings of the 15th IEEE International Symposium on Biomedical Imaging. Washington: IEEE: 1332-1335[ DOI: 10.1109/isbi.2018.8363817 http://dx.doi.org/10.1109/isbi.2018.8363817 ]
Wu W W, Wu S C, Zhou Z H, Zhang R and Zhang Y H. 2017.3D liver tumor segmentation in CT images using improved fuzzy C -means and graph cuts. BioMed Research International, 2017:#5207685[DOI:10.1155/2017/5207685]
Xu F, Ma H Y, Sun J X, Wu R, Liu X and Kong Y Y. 2019. LSTM multi-modal UNet for brain tumor segmentation//Proceedings of the 2019 IEEE International Conference on Image, Vision and Computing. Xiamen: IEEE: 236-240[ DOI: 10.1109/icivc47709.2019.8981027 http://dx.doi.org/10.1109/icivc47709.2019.8981027 ]
Yang D, Xiong T, Xu D G and Zhou K. 2020. Segmentation using adversarial image-to-image networks//Zhou S K, Rueckert D and Fichtinger G. Handbook of Medical Image Computing and Computer Assisted Intervention.[s.l.]: Academic Press: 165-182[ DOI: 10.1016/b978-0-12-816176-0.00012-0 http://dx.doi.org/10.1016/b978-0-12-816176-0.00012-0 ]
Yang J C, Fang R Y, Ni B B, Li Y M, Xu Y and Li L G. 2019. Probabilistic radiomics: ambiguous diagnosis with controllable shape analysis//Proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen: Springer: 658-666[ DOI: 10.1007/978-3-030-32226-7_73 http://dx.doi.org/10.1007/978-3-030-32226-7_73 ]
Zhou J Y, Xiong W, Tian Q, Qi Y Y, Liu J, Leow W K, Han T, Venkatesh S K and Wang S C. 2008. Semi-automatic segmentation of 3D liver tumors from CT scans using voxel classification and propagational learning//Proceedings of Learning, Workshop on 3D Segmentation in the Clinic.[s.l.]: [s.n.]: 1-10
相关作者
相关机构
京公网安备11010802024621
