浏览全部资源
扫码关注微信
Published： 16 November 2023 ，
移动端阅览
尹静， 刘哲， 宋余庆， 邱成健. 2023. 基于3D路径聚合高分辨率网络的胰腺分割. 中国图象图形学报， 28(11):3602-3617
Yin Jing， Liu Zhe， Song Yuqing， Qiu Chengjian. 2023. Pancreas segmentation based on 3D path aggregation high-resolution network. Journal of Image and Graphics， 28(11):3602-3617
目的
2
胰腺的准确分割是胰腺癌识别和分析的重要前提。现有基于深度学习的主流胰腺分割网络大多是编码—解码结构，对特征图采用先降低再增加分辨率的方式，严重丢失了胰腺位置和细节信息，导致分割效果不佳。 针对上述问题，提出了基于3D路径聚合高分辨率网络的胰腺分割方法。
方法
2
首先，为了捕获更多3D特征上下文信息，将高分辨率网络中的2D运算拓展为3D运算；其次，提出全分辨特征路径聚合模块，利用连续非线性变换缩小全分辨率输入图像与分割头网络输出特征语义差异的同时，减少茎网络下采样丢失的位置和细节信息对分割结果的影响；最后，提出多尺度特征路径聚合模块，利用渐进自适应特征压缩融合方式，避免低分辨率特征通道过度压缩导致的信息内容损失。
结果
2
在公开胰腺数据集上，提出方法在Dice系数（Dice similarity coefficient，DSC）、Jaccard系数（Jaccard index，JI）、精确率（precision）和召回率（recall）上相比3D高分辨率网络（3D high-resolution net， 3DHRNet）分别提升了1.41%、2.09%、2.35%和0.49%，相比具有代表性编码—解码结构的胰腺分割方法，取得了更高的分割精度。
结论
2
本文提出的3D路径聚合高分辨率网络（3D pathaggregation high-resolution network， 3D PAHRNet）具有更强的特征位置和细节信息的保留能力，能够显著改善在腹部CT （computed tomography）图像中所占比例较小的胰腺器官的分割结果。开源代码可在
https://github.com/qiuchengjian/PAHRNet3D
https://github.com/qiuchengjian/PAHRNet3D
获得。
Objective
2
Accurate pancreas segmentation is an important prerequisite for the detection， identification， and analysis of pancreatic cancer. However， due to the small proportion of the pancreas in the input CT volume and the large variations in its position and shape， accurate pancreas segmentation has always been a challenging task. Most of the existing mainstream deep learning pancreas segmentation networks are based on the encoding-decoding structure， which initially reduces the resolution of the input image through continuous down-sampling in the encoder to capture strong semantics on a large receptive field， identify the complete pancreas， and gradually restore the lowest-resolution encoder features to obtain the predicted segmentation results. However， the continuous down-sampling in the encoder leads to the loss of location and details of features.
Method
2
To alleviate the above problem， this paper proposes a 3D path aggregation high-resolution network （3DPAHRNet） for pancreas segmentation. First， to capture additional 3D feature context information， the 2D convolution operation in the high-resolution network is extended to the 3D convolution operation. Second， this paper proposes a full-resolution path aggregation module that utilizes five consecutive nonlinear transformations to reduce the semantic difference between the full-resolution input and the output of the segmentation head network while reducing the impact of location and detail information loss due to the continuous down-sampling of the stem network on the segmentation results. Finally， this paper proposes a multi-scale feature path aggregation module that leverages the progressive feature channel compression and fusion strategy in order for the multi-scale features outputted by the high-resolution network to adaptively adjust the features in the network and avoid the problem of information content loss caused by the excessive compression of multi-scale low-resolution feature channels.
Result
2
To verify the effectiveness of the proposed method， extensive experiments are conducted on a public pancreas dataset. First， the segmentation results are compared with those of mainstream pancreas segmentation networks， including 3D U-Net， AttentionUNet， VNet， and 3D HRNet. Compared with the state-of-the-art segmentation results， the proposed method improves the Dice similarity coefficient， Jaccard index， precision， and recall by 1.41%， 2.09%， 2.35%， and 0.49%， respectively. Second， the effectiveness of the proposed module is verified by conducting three ablation studies. Experimental results show that when the number of down-sampling times in the stem subnetwork of 3DHRNet is reduced， either the full-resolution or multi-scale feature path aggregation module is added， and the average segmentation accuracy is significantly improved. Finally， the proposed method is compared with representative pancreas segmentation methods. Comparison results show that the proposed method improves the state-of-the-art segmentation accuracy by 1.1%.
Conclusion
2
This paper proposes the 3DPAHRNet for pancreas segmentation. Unlike the use of high-resolution net on natural images， the proposed method not only keeps the high-resolution features in the network but also enables the network to retain additional location and detail features of the full-resolution input， thus significantly improving the performance of existing pancreas segmentation networks. The open-source code is available at
https://github.com/qiuchengjian/PAHRNet3D
https://github.com/qiuchengjian/PAHRNet3D
.
胰腺分割卷积网络3D路径聚合高分辨率网络全分辨率特征多尺度特征
pancreas segmentationconvolutional networks3D path aggregation high-resolution networkfull resolution featuremulti-scale feature
Asaturyan H， Gligorievski A and Villarini B. 2019. Morphological and multi-level geometrical descriptor analysis in CT and MRI volumes for automatic pancreas segmentation. Computerized Medical Imaging and Graphics， 75： 1-13 ［DOI： 10.1016/j.compmedimag.2019.04.004http://dx.doi.org/10.1016/j.compmedimag.2019.04.004］
Badrinarayanan V， Kendall A and Cipolla R. 2017. SegNet： a deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence， 39（12）： 2481-2495 ［DOI： 10.1109/TPAMI.2016.2644615http://dx.doi.org/10.1109/TPAMI.2016.2644615］
Bi X L， Lu M， Xiao B and Li W S. 2022. Pancreas segmentation based on dual-decoding U-Net. Journal of Software， 33（5）： 1947-1958
毕秀丽， 陆猛， 肖斌， 李伟生. 2022. 基于双解码U型卷积神经网络的胰腺分割. 软件学报， 33（5）： 1947-1958 ［DOI： 10.13328/j.cnki.jos.006198http://dx.doi.org/10.13328/j.cnki.jos.006198］
Cai J Z， Lu L， Xie Y P， Xing F Y and Yang L. 2017. Improving deep pancreas segmentation in CT and MRI images via recurrent neural contextual learning and direct loss function ［EB/OL］. ［2022-09-22］. https://arxiv.org/pdf/1707.04912.pdfhttps://arxiv.org/pdf/1707.04912.pdf
Çiçek Ö， Abdulkadir A， Lienkamp S S， Brox T and Ronneberger O. 2016. 3D U-Net： learning dense volumetric segmentation from sparse annotation//Proceedings of the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention. Athens， Greece： Springer： 424-432 ［DOI： 10.1007/978-3-319-46723-8_49http://dx.doi.org/10.1007/978-3-319-46723-8_49］
Fang C W， Li G B， Pan C W， Li Y M and Yu Y Z. 2019. Globally guided progressive fusion network for 3D pancreas segmentation//Proceedings of the 22nd International conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen， China： Springer： 13-17 ［DOI： 10.1007/978-3-030-32245-8_24http://dx.doi.org/10.1007/978-3-030-32245-8_24］
Farag A， Lu L， Roth H R， Liu J M， Turkbey E and Summers R M. 2017. A bottom-up approach for pancreas segmentation using cascaded superpixels and （Deep） image patch labeling. IEEE Transactions on Image Processing， 26（1）： 386-399 ［DOI： 10.1109/TIP.2016.2624198http://dx.doi.org/10.1109/TIP.2016.2624198］
Hammon M， Cavallaro A， Erdt M， Dankerl P， Kirschner M， Drechsler K， Wesarg S， Uder M and Janka R. 2013. Model-based pancreas segmentation in portal venous phase contrast-enhanced CT images. Journal of Digital Imaging， 26（6）： 1082-1090 ［DOI： 10.1007/s10278-013-9586-7http://dx.doi.org/10.1007/s10278-013-9586-7］
Hu J， Shen L and Sun G. 2018. Squeeze-and-excitation networks//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Utah， USA： IEEE： 7132-7141 ［DOI： 10.1109/CVPR.2018.00745http://dx.doi.org/10.1109/CVPR.2018.00745］
Huang M L and Wu Y Z. 2022. Semantic segmentation of pancreatic medical images by using convolutional neural network. Biomedical Signal Processing and Control， 73： #103458 ［DOI： 10.1016/j.bspc.2021.103458http://dx.doi.org/10.1016/j.bspc.2021.103458］
Isensee F， Jaeger P F， Kohl S A A， Petersen J and Maier-Hein K H. 2021. nnU-Net： a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods， 18（2）： 203-211 ［DOI： 10.1038/s41592-020-01008-zhttp://dx.doi.org/10.1038/s41592-020-01008-z］
Jain S， Gupta S and Gulati A. 2015. An adaptive hybrid technique for pancreas segmentation using CT image sequences//Proceedings of 2015 International Conference on Signal Processing， Computing and Control. Waknaghat， India： IEEE： 272-276 ［DOI： 10.1109/ISPCC.2015.7375039http://dx.doi.org/10.1109/ISPCC.2015.7375039］
Karasawa K， Oda M， Kitasaka T， Misawa K， Fujiwara M， Chu C W， Zheng G Y， Rueckert D and Mori K. 2017. Multi-atlas pancreas segmentation： atlas selection based on vessel structure. Medical Image Analysis， 39： 18-28 ［DOI： 10.1016/j.media.2017.03.006http://dx.doi.org/10.1016/j.media.2017.03.006］
Krizhevsky A， Sutskever I and Hinton G E. 2012. ImageNet classification with deep convolutional neural networks//Proceedings of the 26th International Conference on Neural Information Processing Systems. Lake Tahoe， USA： MIT： 1106-1114
Li J， Lin X Z， Che H， Li H and Qian X H. 2021. Pancreas segmentation with probabilistic map guided bi-directional recurrent U-Net. Physics in Medicine and Biology， 66（11）： #115010 ［DOI： 10.1088/1361-6560/abfce3http://dx.doi.org/10.1088/1361-6560/abfce3］
Liu Z， Su J， Wang R H， Jiang R， Song Y Q， Zhang D Y， Zhu Y， Yuan D Q， Gan Q S and Sheng V S. 2022. Pancreas Co-segmentation based on dynamic ROI extraction and VGGU-Net. Expert Systems with Applications， 192： #116444 ［DOI： 10.1016/j.eswa.2021.116444http://dx.doi.org/10.1016/j.eswa.2021.116444］
Long J， Shelhamer E and Darrell T. 2015. Fully convolutional networks for semantic segmentation// Proceedings of 2015 IEEE conference on computer vision and pattern recognition. Massachusetts， USA： IEEE： 3431-3440 ［DOI： 10.1109/CVPR.2015.7298965http://dx.doi.org/10.1109/CVPR.2015.7298965］
Milletari F， Navab N and Ahmadi S A. 2016. V-Net： fully convolutional neural networks for volumetric medical image segmentation//Proceedings of the 4th International Conference on 3D Vision. Stanford， USA： IEEE： 565-571 ［DOI： 10.1109/3DV.2016.79http://dx.doi.org/10.1109/3DV.2016.79］
Mo J， Zhang L， Wang Y and Huang H Y. 2020. Iterative 3D feature enhancement network for pancreas segmentation from CT images. Neural Computing and Applications， 32（16）： 12535-12546 ［DOI： 10.1007/s00521-020-04710-3http://dx.doi.org/10.1007/s00521-020-04710-3］
Oda M， Shimizu N， Karasawa K， Nimura Y， Kitasaka T， Misawa K， Fujiwara M， Rueckert D and Mori K. 2016. Regression forest-based atlas localization and direction specific atlas generation for pancreas segmentation//Proceedings of the 19th International conference on Medical Image Computing and Computer-Assisted Intervention. Athens， Greece： Springer： 556-563 ［DOI： 10.1007/978-3-319-46723-8_64http://dx.doi.org/10.1007/978-3-319-46723-8_64］
Ronneberger O， Fischer P and Brox T. 2015. U-Net： convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich， Germany： Springer： 234-241 ［DOI： 10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28］
Roth H R， Lu L， Farag A， Shin H C， Liu J M， Turkbey E B and Summers R M. 2015. DeepOrgan： multi-level deep convolutional networks for automated pancreas segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich， Germany： Springer： 556-564 ［DOI： 10.1007/978-3-319-24553-9_68http://dx.doi.org/10.1007/978-3-319-24553-9_68］
Roth H R， Lu L， Lay N， Harrison A P， Farag A， Sohn A and Summers R M. 2018. Spatial aggregation of holistically-nested convolutional neural networks for automated pancreas localization and segmentation. Medical Image Analysis， 45： 94-107 ［DOI： 10.1016/j.media.2018.01.006http://dx.doi.org/10.1016/j.media.2018.01.006］
Schlemper J， Oktay O， Schaap M， Heinrich M， Kainz B， Glocker B and Rueckert D. 2019. Attention gated networks： learning to leverage salient regions in medical images. Medical Image Analysis， 53： 197-207 ［DOI： 10.1016/j.media.2019.01.012http://dx.doi.org/10.1016/j.media.2019.01.012］
Siegel R L， Miller K D， Fuchs H E and Jemal A. 2021. Cancer statistics， 2021. CA： A Cancer Journal for Clinicians， 71（1）： 7-33 ［DOI： 10.3322/caac.21654http://dx.doi.org/10.3322/caac.21654］
Simonyan K and Zisserman A. 2014. Very deep convolutional networks for large-scale image recognition ［EB/OL］. ［2022-09-22］. https://arxiv.org/pdf/1409.1556.pdfhttps://arxiv.org/pdf/1409.1556.pdf
Sun K， Xiao B， Liu D and Wang J D. 2019. Deep high-resolution representation learning for human pose estimation// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach， USA： IEEE： 5686-5696 ［DOI： 10.1109/CVPR.2019.00584http://dx.doi.org/10.1109/CVPR.2019.00584］
Szegedy C， Liu W， Jia Y Q， Sermanet P， Reed S， Anguelov D， Erhan D， Vanhoucke V and Rabinovich A. 2015. Going deeper with convolutions//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston， USA： IEEE： 1-9 ［DOI： 10.1109/CVPR.2015.7298594http://dx.doi.org/10.1109/CVPR.2015.7298594］
Szegedy C， Vanhoucke V， Ioffe S， Shlens J and Wojna Z. 2016. Rethinking the inception architecture for computer vision//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 2818-2826 ［DOI： 10.1109/CVPR.2016.308http://dx.doi.org/10.1109/CVPR.2016.308］
Tam T D and Binh N T. 2014. Efficient pancreas segmentation in computed tomography based on region-growing//Proceedings of the 17th International Conference on Nature of Computation and Communication. Ho Chi Minh City， Vietnam： Springer： 332-340 ［DOI： 10.1007/978-3-319-15392-6_31http://dx.doi.org/10.1007/978-3-319-15392-6_31］
Wang J D， Sun K， Cheng T H， Jiang B R， Deng C R， Zhao Y， Liu D， Mu Y D， Tan M K， Wang X G， Liu W Y and Xiao B. 2021. Deep high-resolution representation learning for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence， 43（10）： 3349-3364 ［DOI： 10.1109/TPAMI.2020.2983686http://dx.doi.org/10.1109/TPAMI.2020.2983686］
Wang R H， Liu Z and Song Y Q. 2021. Multi-stage pancreas localization and segmentation combined with slices context information. Acta Electronica Sinica， 49（4）： 706-715
王瑞豪， 刘哲， 宋余庆. 2021. 结合切片上下文信息的多阶段胰腺定位与分割. 电子学报， 49（4）： 706-715 ［DOI： 10.12263/DZXB.20200101http://dx.doi.org/10.12263/DZXB.20200101］
Xie L X， Yu Q H， Zhou Y Y， Wang Y， Fishman E K and Yuille A L. 2020. Recurrent saliency transformation network for tiny target segmentation in abdominal CT scans. IEEE Transactions on Medical Imaging， 39（2）： 514-525 ［DOI： 10.1109/TMI.2019.2930679http://dx.doi.org/10.1109/TMI.2019.2930679］
Yu Q H， Xie L X， Wang Y， Zhou Y Y， Fishman E K and Yuille A L. 2018. Recurrent saliency transformation network： incorporating multi-stage visual cues for small organ segmentation//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City， USA： IEEE： 8280-8289 ［DOI： 10.1109/CVPR.2018.00864http://dx.doi.org/10.1109/CVPR.2018.00864］
Zhang D W， Zhang J J， Zhang Q， Han J G， Zhang S and Han J W. 2021a. Automatic pancreas segmentation based on lightweight DCNN modules and spatial prior propagation. Pattern Recognition， 114： #107762 ［DOI： 10.1016/j.patcog.2020.107762http://dx.doi.org/10.1016/j.patcog.2020.107762］
Zhang Y， Wu J， Liu Y L， Chen Y F， Chen W， Wu E， Li C M and Tang X Y. 2021b. A deep learning framework for pancreas segmentation with multi-atlas registration and 3D level-set. Medical Image Analysis， 68： #101884 ［DOI： 10.1016/j.media.2020.101884http://dx.doi.org/10.1016/j.media.2020.101884］
Zhou T， Dong L Y， Huo B Q， Liu S and Ma Z J. 2021. U-Net and its applications in medical image segmentation： a review. Journal of Image and Graphics， 26（9）： 2058-2077
周涛， 董雅丽， 霍兵强， 刘珊， 马宗军. 2021. U-Net网络医学图像分割应用综述. 中国图象图形学报， 26（9）： 2058-2077 ［DOI： 10.11834/jig.200704http://dx.doi.org/10.11834/jig.200704］
Zhou Y Y， Xie L X， Shen W， Wang Y， Fishman E K and Yuille A L. 2017. A fixed-point model for pancreas segmentation in abdominal CT scans//Proceedings of the 20th International conference on Medical Image Computing and Computer-Assisted Intervention. Quebec City， Canada： Springer： 693-701 ［DOI： 10.1007/978-3-319-66182-7_79http://dx.doi.org/10.1007/978-3-319-66182-7_79］
Zhou Z W， Siddiquee M M R， Tajbakhsh N and Liang J M. 2020. UNet++： redesigning skip connections to exploit multiscale features in image segmentation. IEEE Transactions on Medical Imaging， 39（6）： 1856-1867 ［DOI： 10.1109/TMI.2019.2959609http://dx.doi.org/10.1109/TMI.2019.2959609］
Zhu Z T， Xia Y D， Shen W， Fishman E and Yuille A. 2018. A 3D coarse-to-fine framework for volumetric medical image segmentation//Proceedings of the 6th International conference on 3D vision. Verona， Italy： IEEE： 682-690 ［DOI： 10.1109/3DV.2018.00083http://dx.doi.org/10.1109/3DV.2018.00083］
相关作者
相关机构
京公网安备11010802024621