浏览全部资源
扫码关注微信
Published： 16 July 2021 ，
Accepted： 23 January 2021
移动端阅览
Huacong Sun, Yanjun Peng, Yanfei Guo, Xiaoqing Zhang. 3D multi-scale deep convolutional neural networks in pulmonary nodule detection. [J]. Journal of Image and Graphics 26(7):1716-1725(2021)
目的
2
肺结节是肺癌的早期存在形式。低剂量CT（computed tomogragphy）扫描作为肺癌筛查的重要检查手段，已经大规模应用于健康体检，但巨大的CT数据带来了大量工作，随着人工智能技术的快速发展，基于深度学习的计算机辅助肺结节检测引起了关注。由于肺结节尺寸差别较大，在多个尺度上表示特征对结节检测任务至关重要。针对结节尺寸差别较大导致的结节检测困难问题，提出一种基于深度卷积神经网络的胸部CT序列图像3D多尺度肺结节检测方法。
方法
2
包括两阶段：1）尽可能提高敏感度的结节初检网络；2）尽可能减少假阳性结节数量的假阳性降低网络。在结节初检网络中，以组合了压缩激励单元的Res2Net网络为骨干结构，使同一层卷积具有多种感受野，提取肺结节的多尺度特征信息，并使用引入了上下文增强模块和空间注意力模块的区域推荐网络结构，确定候选区域；在由Res2Net网络模块和压缩激励单元组成的假阳性降低网络中对候选结节进一步分类，以降低假阳性，获得最终结果。
结果
2
在公共数据集LUNA16（lung nodule analysis 16）上进行实验，实验结果表明，对于结节初检网络阶段，当平均每例假阳性个数为22时，敏感度可达到0.983，相比基准ResNet + FPN（feature pyramid network）方法，平均敏感度和最高敏感度分别提高了2.6%和0.8%；对于整个3D多尺度肺结节检测网络，当平均每例假阳性个数为1时，敏感度为0.924。
结论
2
与现有主流方案相比，该检测方法不但提高了肺结节检测的敏感度，还有效地控制了假阳性，取得了更优的性能。
Objective
2
Pulmonary nodules are early forms of lung cancer
one of the most threatening malignancies for human health and life. As an important means of lung cancer screening
low-dose computerized tomographic scanning has been widely used in health examinations. However
a large amount of computed tomography(CT) data brings a heavy workload to doctors and radiologists
and high-intensity work can result in misdiagnosis. With the rapid development of artificial intelligence technology
computer-aided lung-nodule detection based on deep learning has attracted much attention. As the size of pulmonary nodules varies greatly
representing features on multiple scales is critical for nodule detection tasks. To solve the problem of difficulty in detection caused by the large difference in size of nodules
this paper proposes a 3D multi-scale pulmonary nodule detection method in chest CT sequence images based on deep convolutional neural network.
Method
2
The method mainly consists of two stages: 1) nodule candidate detection stage that maximizes system sensitivity
and 2) false positive reduction stage that minimizes the number of false positive nodules. Specifically
a series of preprocessing operations is performed on the original CT images first
and the regions of interest (ROIs) of lung nodules are obtained by cropping. In the training phase of the nodule candidate detection network
after the preprocessing steps
data augmentation is performed by randomly rotating
flipping
and scaling. Then
nodule cubes and non-nodule cubes with a size of 128×128×128 are randomly cropped out and input to the network. The nodule candidate detection network uses the combination of the squeeze-and-excitation units and the Res2Net modules as the backbone structure
so that the convolutions of the same layer have a variety of receptive fields. Thus
the network can extract the multi-scale feature information of pulmonary nodules. In addition
the nodule candidate detection network also uses the region proposal network structure that introduces context enhancement module and spatial attention module to identify region candidates. In the test phase of the nodule candidate detection network
the preprocessed CT image is divided into several small patches of size 208×208×208
which are used as the inputs of the network
and adjacent small patches overlap 32 pixels. For each CT image
the nodule candidates obtained from all small patches are summarized
and the nodules with higher overlap are merged through non-maximum suppression with an intersection over union(IOU) threshold of 0.1 to obtain the detection results. In the training phase of the false positive reduction network
because the average number of false positive nodules per scan is 22 obtained through experiments in the nodule candidate detection network
the positive samples are augmented by 22 times to balance the number of positive and negative samples. The augmentation methods are consistent with the methods in the training phase of the nodule candidate detection network. The false positive reduction network mainly consisting of Res2Net modules and squeeze-and-excitation units further classifies nodule candidates to reduce the number of false positives. The testing phase of the false positive reduction network takes the nodule candidate coordinates obtained by the nodule candidate detection network as the centers
and crops cubes of size 48×48×48 as the inputs of the false-positive reduction network. The outputs of the false-positive reduction network are the confidences of nodule candidate cubes. Among them
the squeeze-and-excitation unit can capture the channel dependence comprehensively
which makes the channel weight that contains abundant nodule information significant
and makes the channel weight without nodule information small. Res2Net module increases the receptive field of each output feature map without increasing the computational load
which causes the network to have stronger multi-scale representation ability. The region proposal network can take images of any scale as input and output a series of region candidates with scores
which are robust. Context enhancement module can fuse high-level semantic information and low-level position information. Its structure is simple
the implementation is easy
and the calculation cost is low
but it has good performance. The spatial attention module enables the network to pay more attention to the ROIs
which can reduce the difficulty of accurately distinguishing because of the visual similarity between pulmonary nodules and the structures such as blood vessels and shadows around the pulmonary nodules. The effectiveness of this method is validated on the publicly available dataset LUNA16(lung nodule analysis 16) and extensive ablation validation experiments are conducted to demonstrate the contribution of each key component to our proposed framework. The LUNA16 dataset is a subset of LIDC-IDRI(lung image database consortium and image database resource initiative)
the largest public dataset of lung nodules. The LUNA16 dataset excludes CT images with slice thickness greater than 2.5 mm from the LIDC-IDRI dataset. A total of 888 CT images remain
with slice thickness of 0.62.5 mm
spatial resolution of 0.460.98 mm
and average diameter of 8.3 mm. The criteria for judging a nodule in the LUNA16 dataset is that at least three of the four radiologists believe that the diameter of the nodule is greater than 3 mm. Therefore
a total of 1 186 positive nodules are annotated in the dataset. The evaluation metric
FROC(free-response receiver operating characteristic curves)
is the average recall rate at the average number of false positive nodules at 0.125
0.25
0.5
1
2
4
and 8 per scan
which is the official evaluation metric for the LUNA16 dataset.
Result
2
The experimental results show that in the nodule candidate detection stage
the sensitivity can reach 0.983 when the average number of false positives per scan is 22. Compared with the benchmark ResNet + FPN(feature pyramid network) method
the average sensitivity and the maximum sensitivity are increased by 2.6%and 0.8%
respectively. For the entire 3D multi-scale pulmonary nodule detection network
when the average number of false positives per scan is 1
the sensitivity is 0.924.
Conclusion
2
Compared with the state-of-the-art methods
our method not only improves the sensitivity of pulmonary nodule detection but also effectively controls the number of false positives and achieves better performance. As this method can only output the position information of nodules
in actual lung cancer screening
the growth position
edge shape
and internal structure of the nodules are all significant for clinical diagnosis. Analysis of the characteristics of the nodules can make this method more practical.
肺结节检测卷积神经网络(CNN)多尺度区域推荐网络上下文增强空间注意力假阳性降低
pulmonary nodule detectionconvolutional neural network(CNN)multi-scaleregion proposal networkcontext enhancementspatial attentionfalse positive reduction
Armato Ⅲ S G, McLennan G, Bidaut L, McNitt-Gray M F, Meyer C R, Reeves A P, Zhao B S, Aberle D R, Henschke C I, Hoffman E A, Kazerooni E A, MacMahon H, van Beek E J R, Yankelevitz D, Biancardi A M, Bland P H, Brown M S, Engelmann R M, Laderach G E, Max D, Pais R C, Qing D P Y, Roberts R Y, Smith A R, Starkey A, Batra P, Caligiuri P, Farooqi A, Gladish G W, Jude C M, Munden R F, Petkovska I, Quint L E, Schwartz L H, Sundaram B, Dodd L E, Fenimore C, Gur D, Petrick N, Freymann J, Kirby J, Hughes B, Casteele A V, Gupte S, Sallam M, Heath M D, Kuhn M H, Dharaiya E, Burns R, Fryd D S, Salganicoff M, Anand V, Shreter U, Vastagh S, Croft B Y and Clarke L P. 2011. The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Medical Physics, 38(2): 915-931[DOI: 10.1118/1.3528204]
Bray F, Ferlay J, Soerjomataram I, Siegel R L, Torre L A and Jemal A. 2018. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: A Cancer Journal for Clinicians, 68(6): 394-424[DOI: 10.3322/caac.21492]
Ding J, Li A X, Hu Z Q and Wang L W. 2017. Accurate pulmonary nodule detection in computed tomography images using deep convolutional neural networks//Proceedings of the 20th International Conference on Medical Image Computing and Computer Assisted Intervention. Quebec City, Canada: Springer: 559-567[DOI: 10.1007/978-3-319-66179-7_64http://dx.doi.org/10.1007/978-3-319-66179-7_64]
Dou Q, Chen H, Jin Y M, Lin H J, Qin J and Heng P A. 2017a. Automated pulmonary nodule detection via 3D ConvNets with online sample filtering and hybrid-loss residual learning//Proceedings of the 20th International Conference on Medical Image Computing and Computer Assisted Intervention. Quebec City, Canada: Springer: 630-638[DOI: 10.1007/978-3-319-66179-7_72http://dx.doi.org/10.1007/978-3-319-66179-7_72]
Dou Q, Chen H, Yu L Q, Qin J and Heng P A. 2017b. Multilevel contextual 3-D CNNs for false positive reduction in pulmonary nodule detection. IEEE Transactions on Biomedical Engineering, 64(7): 1558-1567[DOI: 10.1109/TBME.2016.2613502]
Gao S H, Cheng M M, Zhao K, Zhang X Y, Yang M H and Torr P. 2021. Res2Net: a new multi-scale backbone architecture. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(2): 652-662[DOI: 10.1109/TPAMI.2019.2938758]
He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 770-778[DOI: 10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90]
Hu J, Shen L, Albanie S, Sun G and Wu E H. 2020. Squeeze-and-excitation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(8): 2011-2023[DOI: 10.1109/TPAMI.2019.2913372]
Khosravan N and Bagci U. 2018. S4ND: single-shot single-scale lung nodule detection//Proceedings of the 21st International Conference on Medical Image Computing and Computer Assisted Intervention. Granada, Spain: Springer: 794-802[DOI: 10.1007/978-3-030-00934-2_88http://dx.doi.org/10.1007/978-3-030-00934-2_88]
Li F, Huang H Y, Wu Y W, Cai C B, Huang Y and Ding X H. 2019. Lung nodule detection with a 3D ConvNet via IoU self-normalization and maxout unit//Proceedings of 2019 International Conference on Acoustics, Speech and Signal Processing. Brighton, UK: IEEE: 1214-1218[DOI: 10.1109/ICASSP.2019.8683537http://dx.doi.org/10.1109/ICASSP.2019.8683537]
Lim H I. 2021. A study on dropout techniques to reduce overfitting in deep neural networks. Lecture Notes in Electrical Engineering, 716: 133-139[DOI: 10.1007/978-981-15-9309-3_20]
Lin T Y, Dollár P, Girshick R, He K M, Hariharan B and Belongie S. 2017. Feature pyramid networks for object detection//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 936-944[DOI: 10.1109/CVPR.2017.106http://dx.doi.org/10.1109/CVPR.2017.106]
Murphy A, Skalski M and Gaillard F. 2018. The utilisation of convolutional neural networks in detecting pulmonary nodules: a review. The British Journal of Radiology, 91(1090): #20180028[DOI: 10.1259/bjr.20180028]
Pezeshk A, Hamidian S, Petrick N and Sahiner B. 2019. 3-D convolutional neural networks for automatic detection of pulmonary nodules in chest CT. IEEE Journal of Biomedical and Health Informatics, 23(5): 2080-2090[DOI: 10.1109/JBHI.2018.2879449]
Qin Z, Li Z M, Zhang Z N, Bao Y P, Yu G, Peng Y X and Sun J. 2019. ThunderNet: towards real-time generic object detection on mobile devices//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul, Korea (South): IEEE: 6717-6726[DOI: 10.1109/ICCV.2019.00682http://dx.doi.org/10.1109/ICCV.2019.00682]
Ren S Q, He K M, Girshick R and Sun J. 2017. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6): 1137-1149[DOI: 10.1109/TPAMI.2016.2577031]
Rothe R, Guillaumin M and Van Gool L. 2015. Non-maximum suppression for object detection by passing messages between windows//Proceedings of the 12th Asian Conference on Computer Vision. Singapore, Singapore: Springer: 290-306[DOI: 10.1007/978-3-319-16865-4_19http://dx.doi.org/10.1007/978-3-319-16865-4_19]
Setio A A A, Traverso A, de Bel T, Berens M S N, van den Bogaard C, Cerello P, Chen H, Dou Q, Fantacci M E, Geurts B, van der Gugten R, Heng P A, Jansen B, de Kaste M M, Kotov V, Lin J Y H, Manders J T M C, Sóñora-Mengana A, García-Naranjo J, Papavasileiou E, Prokop M, Saletta M, Schaefer-Prokop C M, Scholten E T, Scholten L, Snoeren M M, Torres E L, Vandemeulebroucke J, Walasek N, Zuidhof G C A, van Ginneken B and Jacobs C. 2017. Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge. Medical Image Analysis, 42: 1-13[DOI: 10.1016/j.media.2017.06.015]
Shelhamer E, Long J and Darrell T. 2017. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4): 640-651[DOI: 10.1109/TPAMI.2016.2572683]
Szegedy C, Ioffe S, Vanhoucke V and Alemi A A. 2017. Inception-v4, inception-ResNet and the impact of residual connections on learning//Proceedings of the 31st AAAI Conference on Artificial Intelligence. San Francisco, USA: AAAI: 4278-4284
Tong G F, Chen H R, Li Y, Du X C and Zhang Q C. 2019. Object detection for panoramic images based on MS-RPN structure in traffic road scenes. IET Computer Vision, 13(5): 500-506[DOI: 10.1049/iet-cvi.2018.5304]
Wang J, Wang J W, Wen Y F, Lu H B, Niu T Y, Pan J F and Qian D H. 2019. Pulmonary nodule detection in volumetric chest CT scans using CNNS-based nodule-size-adaptive detection and classification. IEEE Access, 7: 46033-46044[DOI: 10.1109/ACCESS.2019.2908195]
Wood D E, Kazerooni E A, Baum S L, Eapen G A, Ettinger D S, Hou L F, Jackman D M, Klippenstein D, Kumar R, Lackner R P, Leard L E, Lennes I T, Leung A N C, Makani S S, Massion P P, Mazzone P, Merritt R E, Meyers B F, Midthun D E, Pipavath S, Pratt C, Reddy C, Reid M E, Rotter A J, Sachs P B, Schabath M B, Schiebler M L, Tong B C, Travis W D, Wei B, Yang S C, Gregory K M and Hughes M. 2018. Lung cancer screening, version 3. 2018, NCCN clinical practice guidelines in oncology. Journal of the National Comprehensive Cancer Network, 16(4): 412-441[DOI: 10.6004/jnccn.2018.0020]
Xie H T, Yang D B, Sun N N, Chen Z N and Zhang Y D. 2019. Automated pulmonary nodule detection in CT images using deep convolutional neural networks. Pattern Recognition, 85: 109-119[DOI: 10.1016/j.patcog.2018.07.031]
Yang D M, Zou Y X, Zhang J and Li G. 2019. C-RPNs: promoting object detection in real world via a cascade structure of region proposal networks. Neurocomputing, 367: 20-30[DOI: 10.1016/j.neucom.2019.08.016]
Yuan J R, Xue B, Zhang W S, Xu L, Sun H Y and Zhou J H. 2019. RPN-FCN based rust detection on power equipment. Procedia Computer Science, 147: 349-353[DOI: 10.1016/j.procs.2019.01.236]
Zhu W T, Liu C C, Fan W and Xie X H. 2018. DeepLung: deep 3D dual path nets for automated pulmonary nodule detection and classification//Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). Lake Tahoe, USA: IEEE: 673-681[DOI: 10.1109/WACV.2018.00079http://dx.doi.org/10.1109/WACV.2018.00079]
相关文章
相关作者
相关机构
京公网安备11010802024621