浏览全部资源
扫码关注微信
收稿：2022-09-27，
修回：2023-02-14，
纸质出版：2023-11-16
移动端阅览
目的
2
胰腺的准确分割是胰腺癌识别和分析的重要前提。现有基于深度学习的主流胰腺分割网络大多是编码—解码结构，对特征图采用先降低再增加分辨率的方式，严重丢失了胰腺位置和细节信息，导致分割效果不佳。 针对上述问题，提出了基于3D路径聚合高分辨率网络的胰腺分割方法。
方法
2
首先，为了捕获更多3D特征上下文信息，将高分辨率网络中的2D运算拓展为3D运算；其次，提出全分辨特征路径聚合模块，利用连续非线性变换缩小全分辨率输入图像与分割头网络输出特征语义差异的同时，减少茎网络下采样丢失的位置和细节信息对分割结果的影响；最后，提出多尺度特征路径聚合模块，利用渐进自适应特征压缩融合方式，避免低分辨率特征通道过度压缩导致的信息内容损失。
结果
2
在公开胰腺数据集上，提出方法在Dice系数（Dice similarity coefficient，DSC）、Jaccard系数（Jaccard index，JI）、精确率（precision）和召回率（recall）上相比3D高分辨率网络（3D high-resolution net， 3DHRNet）分别提升了1.41%、2.09%、2.35%和0.49%，相比具有代表性编码—解码结构的胰腺分割方法，取得了更高的分割精度。
结论
2
本文提出的3D路径聚合高分辨率网络（3D pathaggregation high-resolution network， 3D PAHRNet）具有更强的特征位置和细节信息的保留能力，能够显著改善在腹部CT （computed tomography）图像中所占比例较小的胰腺器官的分割结果。开源代码可在
https://github.com/qiuchengjian/PAHRNet3D
https://github.com/qiuchengjian/PAHRNet3D
获得。
Objective
2
Accurate pancreas segmentation is an important prerequisite for the detection， identification， and analysis of pancreatic cancer. However， due to the small proportion of the pancreas in the input CT volume and the large variations in its position and shape， accurate pancreas segmentation has always been a challenging task. Most of the existing mainstream deep learning pancreas segmentation networks are based on the encoding-decoding structure， which initially reduces the resolution of the input image through continuous down-sampling in the encoder to capture strong semantics on a large receptive field， identify the complete pancreas， and gradually restore the lowest-resolution encoder features to obtain the predicted segmentation results. However， the continuous down-sampling in the encoder leads to the loss of location and details of features.
Method
2
To alleviate the above problem， this paper proposes a 3D path aggregation high-resolution network （3DPAHRNet） for pancreas segmentation. First， to capture additional 3D feature context information， the 2D convolution operation in the high-resolution network is extended to the 3D convolution operation. Second， this paper proposes a full-resolution path aggregation module that utilizes five consecutive nonlinear transformations to reduce the semantic difference between the full-resolution input and the output of the segmentation head network while reducing the impact of location and detail information loss due to the continuous down-sampling of the stem network on the segmentation results. Finally， this paper proposes a multi-scale feature path aggregation module that leverages the progressive feature channel compression and fusion strategy in order for the multi-scale features outputted by the high-resolution network to adaptively adjust the features in the network and avoid the problem of information content loss caused by the excessive compression of multi-scale low-resolution feature channels.
Result
2
To verify the effectiveness of the proposed method， extensive experiments are conducted on a public pancreas dataset. First， the segmentation results are compared with those of mainstream pancreas segmentation networks， including 3D U-Net， AttentionUNet， VNet， and 3D HRNet. Compared with the state-of-the-art segmentation results， the proposed method improves the Dice similarity coefficient， Jaccard index， precision， and recall by 1.41%， 2.09%， 2.35%， and 0.49%， respectively. Second， the effectiveness of the proposed module is verified by conducting three ablation studies. Experimental results show that when the number of down-sampling times in the stem subnetwork of 3DHRNet is reduced， either the full-resolution or multi-scale feature path aggregation module is added， and the average segmentation accuracy is significantly improved. Finally， the proposed method is compared with representative pancreas segmentation methods. Comparison results show that the proposed method improves the state-of-the-art segmentation accuracy by 1.1%.
Conclusion
2
This paper proposes the 3DPAHRNet for pancreas segmentation. Unlike the use of high-resolution net on natural images， the proposed method not only keeps the high-resolution features in the network but also enables the network to retain additional location and detail features of the full-resolution input， thus significantly improving the performance of existing pancreas segmentation networks. The open-source code is available at
https://github.com/qiuchengjian/PAHRNet3D
https://github.com/qiuchengjian/PAHRNet3D
.
Asaturyan H ， Gligorievski A and Villarini B . 2019 . Morphological and multi-level geometrical descriptor analysis in CT and MRI volumes for automatic pancreas segmentation . Computerized Medical Imaging and Graphics ， 75 ： 1 - 13 ［ DOI： 10.1016/j.compmedimag.2019.04.004 http://dx.doi.org/10.1016/j.compmedimag.2019.04.004 ］
Badrinarayanan V ， Kendall A and Cipolla R . 2017 . SegNet： a deep convolutional encoder-decoder architecture for image segmentation . IEEE Transactions on Pattern Analysis and Machine Intelligence ， 39 （ 12 ）： 2481 - 2495 ［ DOI： 10.1109/TPAMI.2016.2644615 http://dx.doi.org/10.1109/TPAMI.2016.2644615 ］
Bi X L ， Lu M ， Xiao B and Li W S . 2022 . Pancreas segmentation based on dual-decoding U-Net . Journal of Software ， 33 （ 5 ）： 1947 - 1958
毕秀丽 ， 陆猛 ， 肖斌 ， 李伟生 . 2022 . 基于双解码U型卷积神经网络的胰腺分割 . 软件学报 ， 33 （ 5 ）： 1947 - 1958 ［ DOI： 10.13328/j.cnki.jos.006198 http://dx.doi.org/10.13328/j.cnki.jos.006198 ］
Cai J Z ， Lu L ， Xie Y P ， Xing F Y and Yang L . 2017 . Improving deep pancreas segmentation in CT and MRI images via recurrent neural contextual learning and direct loss function ［EB/OL］. ［ 2022-09-22 ］. https://arxiv.org/pdf/1707.04912.pdf https://arxiv.org/pdf/1707.04912.pdf
Çiçek Ö ， Abdulkadir A ， Lienkamp S S ， Brox T and Ronneberger O . 2016 . 3D U-Net： learning dense volumetric segmentation from sparse annotation // Proceedings of the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention . Athens， Greece ： Springer： 424 - 432 ［ DOI： 10.1007/978-3-319-46723-8_49 http://dx.doi.org/10.1007/978-3-319-46723-8_49 ］
Fang C W ， Li G B ， Pan C W ， Li Y M and Yu Y Z . 2019 . Globally guided progressive fusion network for 3D pancreas segmentation // Proceedings of the 22nd International conference on Medical Image Computing and Computer-Assisted Intervention . Shenzhen， China ： Springer： 13 - 17 ［ DOI： 10.1007/978-3-030-32245-8_24 http://dx.doi.org/10.1007/978-3-030-32245-8_24 ］
Farag A ， Lu L ， Roth H R ， Liu J M ， Turkbey E and Summers R M . 2017 . A bottom-up approach for pancreas segmentation using cascaded superpixels and （Deep） image patch labeling . IEEE Transactions on Image Processing ， 26 （ 1 ）： 386 - 399 ［ DOI： 10.1109/TIP.2016.2624198 http://dx.doi.org/10.1109/TIP.2016.2624198 ］
Hammon M ， Cavallaro A ， Erdt M ， Dankerl P ， Kirschner M ， Drechsler K ， Wesarg S ， Uder M and Janka R . 2013 . Model-based pancreas segmentation in portal venous phase contrast-enhanced CT images . Journal of Digital Imaging ， 26 （ 6 ）： 1082 - 1090 ［ DOI： 10.1007/s10278-013-9586-7 http://dx.doi.org/10.1007/s10278-013-9586-7 ］
Hu J ， Shen L and Sun G . 2018 . Squeeze-and-excitation networks // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Utah， USA ： IEEE： 7132 - 7141 ［ DOI： 10.1109/CVPR.2018.00745 http://dx.doi.org/10.1109/CVPR.2018.00745 ］
Huang M L and Wu Y Z . 2022 . Semantic segmentation of pancreatic medical images by using convolutional neural network . Biomedical Signal Processing and Control ， 73 ： # 103458 ［ DOI： 10.1016/j.bspc.2021.103458 http://dx.doi.org/10.1016/j.bspc.2021.103458 ］
Isensee F ， Jaeger P F ， Kohl S A A ， Petersen J and Maier-Hein K H . 2021 . nnU-Net： a self-configuring method for deep learning-based biomedical image segmentation . Nature Methods ， 18 （ 2 ）： 203 - 211 ［ DOI： 10.1038/s41592-020-01008-z http://dx.doi.org/10.1038/s41592-020-01008-z ］
Jain S ， Gupta S and Gulati A . 2015 . An adaptive hybrid technique for pancreas segmentation using CT image sequences // Proceedings of 2015 International Conference on Signal Processing， Computing and Control . Waknaghat， India ： IEEE： 272 - 276 ［ DOI： 10.1109/ISPCC.2015.7375039 http://dx.doi.org/10.1109/ISPCC.2015.7375039 ］
Karasawa K ， Oda M ， Kitasaka T ， Misawa K ， Fujiwara M ， Chu C W ， Zheng G Y ， Rueckert D and Mori K . 2017 . Multi-atlas pancreas segmentation： atlas selection based on vessel structure . Medical Image Analysis ， 39 ： 18 - 28 ［ DOI： 10.1016/j.media.2017.03.006 http://dx.doi.org/10.1016/j.media.2017.03.006 ］
Krizhevsky A ， Sutskever I and Hinton G E . 2012 . ImageNet classification with deep convolutional neural networks // Proceedings of the 26th International Conference on Neural Information Processing Systems . Lake Tahoe， USA ： MIT： 1106 - 1114
Li J ， Lin X Z ， Che H ， Li H and Qian X H . 2021 . Pancreas segmentation with probabilistic map guided bi-directional recurrent U-Net . Physics in Medicine and Biology ， 66 （ 11 ）： # 115010 ［ DOI： 10.1088/1361-6560/abfce3 http://dx.doi.org/10.1088/1361-6560/abfce3 ］
Liu Z ， Su J ， Wang R H ， Jiang R ， Song Y Q ， Zhang D Y ， Zhu Y ， Yuan D Q ， Gan Q S and Sheng V S . 2022 . Pancreas Co-segmentation based on dynamic ROI extraction and VGGU-Net . Expert Systems with Applications ， 192 ： # 116444 ［ DOI： 10.1016/j.eswa.2021.116444 http://dx.doi.org/10.1016/j.eswa.2021.116444 ］
Long J ， Shelhamer E and Darrell T . 2015 . Fully convolutional networks for semantic segmentation // Proceedings of 2015 IEEE conference on computer vision and pattern recognition . Massachusetts， USA ： IEEE： 3431 - 3440 ［ DOI： 10.1109/CVPR.2015.7298965 http://dx.doi.org/10.1109/CVPR.2015.7298965 ］
Milletari F ， Navab N and Ahmadi S A . 2016 . V-Net： fully convolutional neural networks for volumetric medical image segmentation // Proceedings of the 4th International Conference on 3D Vision . Stanford， USA ： IEEE： 565 - 571 ［ DOI： 10.1109/3DV.2016.79 http://dx.doi.org/10.1109/3DV.2016.79 ］
Mo J ， Zhang L ， Wang Y and Huang H Y . 2020 . Iterative 3D feature enhancement network for pancreas segmentation from CT images . Neural Computing and Applications ， 32 （ 16 ）： 12535 - 12546 ［ DOI： 10.1007/s00521-020-04710-3 http://dx.doi.org/10.1007/s00521-020-04710-3 ］
Oda M ， Shimizu N ， Karasawa K ， Nimura Y ， Kitasaka T ， Misawa K ， Fujiwara M ， Rueckert D and Mori K . 2016 . Regression forest-based atlas localization and direction specific atlas generation for pancreas segmentation // Proceedings of the 19th International conference on Medical Image Computing and Computer-Assisted Intervention . Athens， Greece ： Springer： 556 - 563 ［ DOI： 10.1007/978-3-319-46723-8_64 http://dx.doi.org/10.1007/978-3-319-46723-8_64 ］
Ronneberger O ， Fischer P and Brox T . 2015 . U-Net： convolutional networks for biomedical image segmentation // Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention . Munich， Germany ： Springer： 234 - 241 ［ DOI： 10.1007/978-3-319-24574-4_28 http://dx.doi.org/10.1007/978-3-319-24574-4_28 ］
Roth H R ， Lu L ， Farag A ， Shin H C ， Liu J M ， Turkbey E B and Summers R M . 2015 . DeepOrgan： multi-level deep convolutional networks for automated pancreas segmentation // Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention . Munich， Germany ： Springer： 556 - 564 ［ DOI： 10.1007/978-3-319-24553-9_68 http://dx.doi.org/10.1007/978-3-319-24553-9_68 ］
Roth H R ， Lu L ， Lay N ， Harrison A P ， Farag A ， Sohn A and Summers R M . 2018 . Spatial aggregation of holistically-nested convolutional neural networks for automated pancreas localization and segmentation . Medical Image Analysis ， 45 ： 94 - 107 ［ DOI： 10.1016/j.media.2018.01.006 http://dx.doi.org/10.1016/j.media.2018.01.006 ］
Schlemper J ， Oktay O ， Schaap M ， Heinrich M ， Kainz B ， Glocker B and Rueckert D . 2019 . Attention gated networks： learning to leverage salient regions in medical images . Medical Image Analysis ， 53 ： 197 - 207 ［ DOI： 10.1016/j.media.2019.01.012 http://dx.doi.org/10.1016/j.media.2019.01.012 ］
Siegel R L ， Miller K D ， Fuchs H E and Jemal A . 2021 . Cancer statistics， 2021 . CA： A Cancer Journal for Clinicians ， 71 （ 1 ）： 7 - 33 ［ DOI： 10.3322/caac.21654 http://dx.doi.org/10.3322/caac.21654 ］
Simonyan K and Zisserman A . 2014 . Very deep convolutional networks for large-scale image recognition ［EB/OL］. ［ 2022-09-22 ］. https://arxiv.org/pdf/1409.1556.pdf https://arxiv.org/pdf/1409.1556.pdf
Sun K ， Xiao B ， Liu D and Wang J D . 2019 . Deep high-resolution representation learning for human pose estimation // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach， USA ： IEEE： 5686 - 5696 ［ DOI： 10.1109/CVPR.2019.00584 http://dx.doi.org/10.1109/CVPR.2019.00584 ］
Szegedy C ， Liu W ， Jia Y Q ， Sermanet P ， Reed S ， Anguelov D ， Erhan D ， Vanhoucke V and Rabinovich A . 2015 . Going deeper with convolutions // Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition . Boston， USA ： IEEE： 1 - 9 ［ DOI： 10.1109/CVPR.2015.7298594 http://dx.doi.org/10.1109/CVPR.2015.7298594 ］
Szegedy C ， Vanhoucke V ， Ioffe S ， Shlens J and Wojna Z . 2016 . Rethinking the inception architecture for computer vision // Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas， USA ： IEEE： 2818 - 2826 ［ DOI： 10.1109/CVPR.2016.308 http://dx.doi.org/10.1109/CVPR.2016.308 ］
Tam T D and Binh N T . 2014 . Efficient pancreas segmentation in computed tomography based on region-growing // Proceedings of the 17th International Conference on Nature of Computation and Communication . Ho Chi Minh City， Vietnam ： Springer： 332 - 340 ［ DOI： 10.1007/978-3-319-15392-6_31 http://dx.doi.org/10.1007/978-3-319-15392-6_31 ］
Wang J D ， Sun K ， Cheng T H ， Jiang B R ， Deng C R ， Zhao Y ， Liu D ， Mu Y D ， Tan M K ， Wang X G ， Liu W Y and Xiao B . 2021 . Deep high-resolution representation learning for visual recognition . IEEE Transactions on Pattern Analysis and Machine Intelligence ， 43 （ 10 ）： 3349 - 3364 ［ DOI： 10.1109/TPAMI.2020.2983686 http://dx.doi.org/10.1109/TPAMI.2020.2983686 ］
Wang R H ， Liu Z and Song Y Q . 2021 . Multi-stage pancreas localization and segmentation combined with slices context information . Acta Electronica Sinica ， 49 （ 4 ）： 706 - 715
王瑞豪 ， 刘哲 ， 宋余庆 . 2021 . 结合切片上下文信息的多阶段胰腺定位与分割 . 电子学报 ， 49 （ 4 ）： 706 - 715 ［ DOI： 10.12263/DZXB.20200101 http://dx.doi.org/10.12263/DZXB.20200101 ］
Xie L X ， Yu Q H ， Zhou Y Y ， Wang Y ， Fishman E K and Yuille A L . 2020 . Recurrent saliency transformation network for tiny target segmentation in abdominal CT scans . IEEE Transactions on Medical Imaging ， 39 （ 2 ）： 514 - 525 ［ DOI： 10.1109/TMI.2019.2930679 http://dx.doi.org/10.1109/TMI.2019.2930679 ］
Yu Q H ， Xie L X ， Wang Y ， Zhou Y Y ， Fishman E K and Yuille A L . 2018 . Recurrent saliency transformation network： incorporating multi-stage visual cues for small organ segmentation // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City， USA ： IEEE： 8280 - 8289 ［ DOI： 10.1109/CVPR.2018.00864 http://dx.doi.org/10.1109/CVPR.2018.00864 ］
Zhang D W ， Zhang J J ， Zhang Q ， Han J G ， Zhang S and Han J W . 2021a . Automatic pancreas segmentation based on lightweight DCNN modules and spatial prior propagation . Pattern Recognition ， 114 ： # 107762 ［ DOI： 10.1016/j.patcog.2020.107762 http://dx.doi.org/10.1016/j.patcog.2020.107762 ］
Zhang Y ， Wu J ， Liu Y L ， Chen Y F ， Chen W ， Wu E ， Li C M and Tang X Y . 2021b . A deep learning framework for pancreas segmentation with multi-atlas registration and 3D level-set . Medical Image Analysis ， 68 ： # 101884 ［ DOI： 10.1016/j.media.2020.101884 http://dx.doi.org/10.1016/j.media.2020.101884 ］
Zhou T ， Dong L Y ， Huo B Q ， Liu S and Ma Z J . 2021 . U-Net and its applications in medical image segmentation： a review . Journal of Image and Graphics ， 26 （ 9 ）： 2058 - 2077
周涛 ， 董雅丽 ， 霍兵强 ， 刘珊 ， 马宗军 . 2021 . U-Net网络医学图像分割应用综述 . 中国图象图形学报 ， 26 （ 9 ）： 2058 - 2077 ［ DOI： 10.11834/jig.200704 http://dx.doi.org/10.11834/jig.200704 ］
Zhou Y Y ， Xie L X ， Shen W ， Wang Y ， Fishman E K and Yuille A L . 2017 . A fixed-point model for pancreas segmentation in abdominal CT scans // Proceedings of the 20th International conference on Medical Image Computing and Computer-Assisted Intervention . Quebec City， Canada ： Springer： 693 - 701 ［ DOI： 10.1007/978-3-319-66182-7_79 http://dx.doi.org/10.1007/978-3-319-66182-7_79 ］
Zhou Z W ， Siddiquee M M R ， Tajbakhsh N and Liang J M . 2020 . UNet++： redesigning skip connections to exploit multiscale features in image segmentation . IEEE Transactions on Medical Imaging ， 39 （ 6 ）： 1856 - 1867 ［ DOI： 10.1109/TMI.2019.2959609 http://dx.doi.org/10.1109/TMI.2019.2959609 ］
Zhu Z T ， Xia Y D ， Shen W ， Fishman E and Yuille A . 2018 . A 3D coarse-to-fine framework for volumetric medical image segmentation // Proceedings of the 6th International conference on 3D vision . Verona， Italy ： IEEE： 682 - 690 ［ DOI： 10.1109/3DV.2018.00083 http://dx.doi.org/10.1109/3DV.2018.00083 ］
相关作者
相关机构
京公网安备11010802024621
