浏览全部资源
扫码关注微信
Published： 16 September 2021 ，
Accepted： 20 April 2021
移动端阅览
Fuling Zhang, Shaomin Zhang, Lijia Zhi, Tao Zhou. Detection of pulmonary nodules in CT images by combining an attention mechanism and a feature pyramid network. [J]. Journal of Image and Graphics 26(9):2156-2170(2021)
目的
2
针对现有肺结节检测算法存在的因肺部计算机断层扫描（computed tomography，CT）图像肺结节与周边组织复杂性导致结节本身结构差异性不明显的问题，以及特征提取网络多次下采样造成图像分辨率降低进而导致检测结果差、仅使用网络顶层特征图进行预测造成图像空间信息丢失进而导致小结节漏检等问题，提出了一种基于注意力机制和特征金字塔的肺结节检测算法。
方法
2
根据语义与空间特征补偿机制以及卷积神经网络中网络深度所提取特征的信息量不同，在以ResNet为骨干网络的特征提取网络中设计通道—空间注意力机制，尽可能同时获取含有较多上下文语义以及空间位置信息的特征信息。在网络预测部分设计特征金字塔网络，将高维带有丰富语义信息的特征图与低维带有位置信息的特征图融合进行多尺度预测，增强网络对于小结节以及近血管结节等非显著性目标的检测性能。
结果
2
在LUNA16（lung nodule analysis 16）数据集上进行十折交叉验证显示，当平均假阳性个数为25.99时敏感度达到了97.13%，与基准方法相比，敏感度提高了2.53%，平均假阳性降低了28.54，实现了高敏感度低假阳性；在0.125、0.25、0.5、1、2、4、8这7个假阳率点的敏感度平均值为0.854，其中在每个扫描4次和8次假阳性时敏感度分别达到了0.940和0.951，其效果优于主流的结节检测方法。
结论
2
提出的结节检测模型，可以提高对3~10 mm小结节、近血管结节等非显著性目标的检测性能，并具有较低的假阳率。
Objective
2
The different structure of the nodules have not been clear in terms of the lung computer tomography (CT) images interpretation level for lung nodules and surrounding tissues. The detected image resolution has been reduced due to multiple downsampling of the feature extraction network. The feature map of the network top-layer has been predicted to the losses of spatial information and missing nodules. A algorithm called attention_FPN_RPN(attention_feature pyramid networks-region proposal network) for lung nodule detection have been demonstrated via attention mechanism and feature pyramid network.
Method
2
The detection performance has been improved for an end-to-end lung nodule detection model training via integrating nodule candidate detection and false positive reduction into a model for joint training. At the beginning
the lung parenchyma has been segmented to form the image dataset for training and testing via threshold and morphology. The designated nodule detection model has been based on the region proposal network (RPN) of the faster R-CNN(region convolutional neural network) network. The network model has been illustrated based on the U-Net network structure using low-dose chest CT thin-slice plain scan images as input and outputs candidate nodules position and probability information. Next
the channel-spatial attention mechanism in multi-layers network has been designed in the feature extraction network via the backbone network-ResNet in terms of the semantic and spatial feature compensation mechanism and the amount of extracted feature information. Multi-layers features have been refined to retain feature information each. The integration of lower and higher dimensional features have been spatially interpreted to obtain feature information with more contextual semantics and spatial location information. The semantic information has been qualified via the top-layer of feature map for prediction though the losses of spatial information extracted. The 3D spatial information in CT sequence has not been be fully utilized. The feature pyramid network has been fused high-dimensional feature maps via rich semantic information and low-dimensional feature maps with location information. Multi-scale information network-based detection for insignificant targets like small nodules and proximal vascular nodules has been enhanced. Data enhancement and hard negative data mining has been adopted to optimize the dataset. The sliding window analysis has been used to crop 208×208×208 patch in the preprocessed CT image-based lung parenchymal area. The step size has been consistent with the size of the nodule probability map based on network analysis. Based on convolution calculation
an overlap of 32 pixels amongst the divided blocks have been cropped to eliminate the undesired boundary. At the end
the probability map of the obtained small image block has been refilled into the CT image space to obtain the final probability based on the same size of the graph and the input CT image. Each voxel in the probability graph has illustrated the intended area of lung nodule. For the output result of each small block
the 100 nodules with the highest probability have been retained. Based on summing all patches to obtain candidate nodules
the candidate nodules with high overlap have merged via non-maximum suppression (NMS) method with a threshold of 0.5 for each image. The nodule based on the highest probability has been opted for each group of candidate nodules in the context of a high degree of overlap.
Result
2
10-fold cross-validation on the lung nodule analysis 16(LUNA16) dataset has shown that the sensitivity of the proposed model can be reached 97.13% under the average of 25.99 false positives per scan. In the benchmark
the sensitivity has been increased by 2.53%. The false positive average value has been reduced simultaneously to achieve high sensitivity low false positive
the average sensitivity value of the seven false positive rate points has acquired 0.125
0.25
0.5
1
2
4
and 8 respectively. The sensitivity of the proposed model has reached 0. 940 and 0.951 respectively under the average of 4
8 false positives per scan
which is better than mainstream nodule detection methods.
Conclusion
2
In the context of fewer false positives
the demonstrated nodule detection model has improved the detection results of insignificant targets like 3~10 mm small nodules and proximal vessel nodules based on non-false positive reduction measures. A precise detection for tiny lung nodules has prior to the existing pulmonary nodule detection. The setup of an automatic screening system for small lung nodules and a clinical auxiliary diagnosis system for lung nodules have been benefited from the research.
肺结节检测注意力机制特征金字塔网络(FPN)非显著性目标检测十折交叉验证
pulmonary nodule detectionattention mechanismfeature pyramid network(FPN)insignificant target detection10-fold cross-validation
Berens M, Van Der Gugten R, De Kaste M, Manders J and Zuidhof G. 2016. ZNET-lung nodule detection[EB/OL]. [2021-03-05].https://www.scribd.com/document/429578776/ZNET-NDET-160831https://www.scribd.com/document/429578776/ZNET-NDET-160831
Cao Y, Xing S X, Pang J L, Wang X Y, Wang Y, Pan Z Y and Shen N. 2020. Detection of pulmonary nodules based on improved VGG-16 convolution neural network. Chinese Journal of Medical Physics, 37(7): 940-944
曹宇, 邢素霞, 逄键梁, 王孝义, 王瑜, 潘子妍, 申楠. 2020. 基于改进的VGG-16卷积神经网络的肺结节检测. 中国医学物理学杂志, 37(7): 940-944[DOI:10.3969/j.issn.1005-202X.2020.07.026]
Ding J, Li A X, Hu Z Q and Wang L W. 2017. Accurate pulmonary nodule detection in computed tomography images using deep convolutional neural networks//Proceedings of the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention. Quebec City, Canada: Springer, Cham: 559-567[DOI:10.1007/978-3-319-66179-7_64http://dx.doi.org/10.1007/978-3-319-66179-7_64]
Dou Q, Chen H, Jin Y M, Li H J, Qin J and Heng P A. 2017. Automated pulmonary nodule detection via 3D convnets with online sample filtering and hybrid-loss residual learning//Proceedings of the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention. Quebec City, Canada: Springer: 630-638[DOI:10.1007/978-3-319-66179-7_72http://dx.doi.org/10.1007/978-3-319-66179-7_72]
Firmino M, Angelo G, Morais H, Dantas M and Valentim R. 2016. Computer-aided detection (CADe) and diagnosis (CADx) system for lung cancer with likelihood of malignancy. BioMedical Engineering OnLine, 15, 1(2016-01-06), 2016, 15(1): #2[DOI:10.1186/s12938-015-0120-7]
Fu R, Chai Y F, Zhang J T, Zhang T, Chen X K, Dong S, Yan H H, Yang X N, Huang M P, Wu Y L, Zhuang J and Zhong W Z. 2020. Three-dimensional printed navigational template for localizing small pulmonary nodules: a case-controlled study. Thoracic Cancer, 11(9): 2690-2697[DOI:10.1111/1759-7714.13550]
Gao H M. 2019. Research on Detection and Benign and Malignant Classification Method of Pulmonary Nodules Based on Convolutional Neural Network. Taiyuan: Taiyuan University of Technology
高慧明. 2019. 基于卷积神经网络的肺结节检测及良恶性分类方法研究. 太原: 太原理工大学
Gao Z Y, Huang J Z and Du C G. 2020. Pulmonary nodule detection based on feature pyramid networks. Journal of Computer Applications, 40(9): 2571-2576
高智勇, 黄金镇, 杜程刚. 2020. 基于特征金字塔网络的肺结节检测. 计算机应用, 40(9): 2571-2576[DOI:10.11772/j.issn.1001-9081.2019122122]
Guo H C, Wei Y X, Han Y, Hong L J, Lin Y H, Hai Q X and Yang D Z. 2017. Deep convolution neural networks for pulmonary nodule detection in CT imaging[EB/OL]. [2021-8-13].https://grand-challenge-public-prod.s3.amazonaws.com/f/challenge/71/101b150c-88c5-4f9d-a374-d8d3fc166aff/20171222_073722_JianpeiCAD_NDET.pdfhttps://grand-challenge-public-prod.s3.amazonaws.com/f/challenge/71/101b150c-88c5-4f9d-a374-d8d3fc166aff/20171222_073722_JianpeiCAD_NDET.pdf
Jacobs C, van Rikxoort E M, Twellmann T, Scholten E T, de Jong P A, Kuhnigk J M, Oudkerk M, de Koning H J, Prokop M, Schaefer-Prokop C and van Ginneken B. 2014. Automatic detection of subsolid pulmonary nodules in thoracic computed tomography images. Medical Image Analysis, 18(2): 374-384[DOI:10.1016/j.media.2013.12.001]
Jiang H Y, Ma H, Qian W, Gao M D and Li Y. 2018. An automatic detection system of lung nodule based on multigroup patch-based deep learning network. IEEE Journal of Biomedical and Health Informatics, 22(4): 1227-1237[DOI:10.1109/JBHI.2017.2725903]
Li L, Qiao L and Zhang H Y. 2020. Automatic detection algrithm of pulmonary nodules based on deep learning. Computer Applications and Software, 37(7): 95-100
李莉, 乔璐, 张浩洋. 2020. 基于深度学习的肺结节自动检测算法. 计算机应用与软件, 37(7): 95-100[DOI:10.3969/j.issn.1000-386x.2020.07.015]
Liao F Z, Liang M, Li Z, Hu X L and Song S. 2019. Evaluate the malignancy of pulmonary nodules using the 3-D deep leaky noisy-OR network. IEEE Transactions on Neural Networks and Learning Systems, 30(11): 3484-3495[DOI:10.1109/TNNLS.2019.2892409]
Liu X L, Qi S S, Xiong P, Liu J, Wang H R and Yang J L. 2020. An automatic pulmonary nodules detection algorithm with multi-scale information fusion. Journal of Biomedical Engineering, 37(3): 434-441
刘秀玲, 戚帅帅, 熊鹏, 刘京, 王洪瑞, 杨建利. 2020. 融合多尺度信息的肺结节自动检测算法. 生物医学工程学杂志, 37(3): 434-441[DOI:10.7507/1001-5515.201910047]
Lung Nodule Analysis 2016. 2016. LUNA16-Results[EB/OL]. [2021-03-05].https://luna16.grand-challenge.org/Results/https://luna16.grand-challenge.org/Results/
Murphy K, van Ginneken B, Schilham A M R, de Hoop B J, Gietema H A and Prokop M. 2009. A large-scale evaluation of automatic pulmonary nodule detection in chest CT using local image features and k-nearest-neighbour classification. Medical Image Analysis, 13(5): 757-770[DOI:10.1016/j.media.2009.07.001]
Qi Y J, Gu J H, Zhang Y J, Wang F and Tian Z P. 2020. Deep mixed convolution model for pulmonary nodule detection. Journal of Computer Applications, 40(10): 2904-2909
戚永军, 顾军华, 张亚娟, 王锋, 田泽培. 2020. 基于深度混合卷积模型的肺结节检测方法. 计算机应用, 40(10): 2904-2909[DOI:10.11772/j.issn.1001-9081.2020020192]
Ramachandran S S, George J, Skaria S and Varun V V. 2018. Using YOLO based deep learning network for real time detection and localization of lung nodules from low dose CT scans//Proceedings of SPIE 10575, Medical Imaging 2018: Computer-Aided Diagnosis. Houston, USA: SPIE: #10575[DOI:10.1117/12.2293699http://dx.doi.org/10.1117/12.2293699]
Setio A A A, Ciompi F, Litjens G, Gerke P, Jacobs C, van Riel S J, Wille M M W, Naqibullah M, Sánchez C I and van Ginneken B. 2016. Pulmonary noduledetection in CT images: false positive reduction using multi-view convolutional networks. IEEE Transactions on Medical Imaging, 35(5): 1160-1169[DOI:10.1109/TMI.2016.2536809]
Setio A A A, Traverso A, de Bel T, Berens M S N, van den Bogaard C, Cerello P, Chen H, Dou Q, Fantacci M E, Geurts B, van der Gugten R, Heng P A, Jansen B, de Kaste M M J, Kotov V, Lin J Y H, Manders J T M C, Só?ora-Mengana A, García-Naranjo J C, Papavasileiou E, Prokop M, Saletta M, Schaefer-Prokop C M, Scholten E T, Scholten L, Snoeren M M, Torres E L, Vandemeulebroucke J, Walasek N, Zuidhof G C A, van Ginneken B and Jacobs C. 2017. Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 Challenge. Medical Image Analysis, 42: 1-13[DOI:10.1016/j.media.2017.06.015]
Shi L K, Ma H Q, Zhang C Z and Fan S Y. 2020. Detection method of pulmonary nodules based on improve residual structure. Journal of Computer Applications, 40(7): 2110-2116
石陆魁, 马红祺, 张朝宗, 樊世燕. 2020. 基于改进残差结构的肺结节检测方法. 计算机应用, 40(7): 2110-2116[DOI:10.11772/j.issn.1001-9081.2019122095]
Tan M, Deklerck R, Jansen B, Bister M and Cornelis J. 2011. A novel computer-aided lung nodule detection system for CT images. Medical Physics, 38(10): 5630-5645[DOI:10.1118/1.3633941]
Tang S Y, Yang M and Bai J N. 2020. Detection of pulmonary nodules based on a multiscale feature 3D U-Net convolutional neural network of transfer learning. PLoS One, 15(8): #0235672[DOI:10.1371/journal.pone.0235672]
Torres E L, Fiorina E, Pennazio F, Peroni C, Saletta M, Camarlinghi N, Fantacci M E and Cerello P. 2015. Large scale validation of the M5L lung CAD on heterogeneous CT datasets. Medical Physics, 42(4): 1477-1489[DOI:10.1118/1.4907970]
Wang D, Zhang Y, Zhang K X and Wang L W. 2020. FocalMix: semi-supervised learning for 3D medical image detection//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE: 3951-3960[DOI:10.1109/CVPR42600.2020.00401http://dx.doi.org/10.1109/CVPR42600.2020.00401]
Xie H T, Yang D B, Sun N N, Chen Z N and Zhang Y D. 2019. Automated pulmonary nodule detection in CT images using deep convolutional neural networks. Pattern Recognition, 85: 109-119[DOI:10.1016/j.patcog.2018.07.031]
Zhang F L and Zhang S M. 2020. Review of deep learning methods applied to lung nodule detection in CT images. Computer Engineering and Applications, 56(13): 20-32
张福玲, 张少敏. 2020. 应用于CT图像肺结节检测的深度学习方法综述. 计算机工程与应用, 56(13): 20-32[DOI:10.3778/j.issn.1002-8331.2002-0051]
Zhang J J, Xia Y, Zeng H Y and Zhang Y N. 2018. NODULe: combining constrained multi-scale LoG filters with densely dilated 3D deep convolutional neural network for pulmonary nodule detection. Neurocomputing, 317: 159-167[DOI:10.1016/j.neucom.2018.08.022]
ZhaoP F, Zhao J J, Qiang Y, Wang F Z and Zhao W T. 2018. Study on detection method of pulmonary nodules with multiple input convolution neural network. Computer Science, 45(1): 162-166
赵鹏飞, 赵涓涓, 强彦, 王峰智, 赵文婷. 2018. 多输入卷积神经网络肺结节检测方法研究. 计算机科学, 45(1): 162-166[DOI:10.11896/j.issn.1002-137X.2018.01.028]
Zheng S Y, Guo J P, Cui X N, Veldhuis R N J, Oudkerk M and van Ooijen P M. 2020. Automatic pulmonary nodule detection in CT scans using convolutional neural networks based on maximum intensity projection. IEEE Transactions on Medical Imaging, 39(3): 797-805[DOI:10.1109/TMI.2019.2935553]
Zhu W T, Liu C C, Fan W and Xie X H. 2018a. DeepLung: deep 3D dual path nets for automated pulmonary nodule detection and classification//Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision. Lake Tahoe, USA: IEEE: 673-681[DOI:10.1109/WACV.2018.00079http://dx.doi.org/10.1109/WACV.2018.00079]
Zhu W T, Vang Y S, Huang Y F and Xie X H. 2018b. DeepEM: deep 3D convnets with EM for weakly supervised pulmonary nodule detection//Proceedings of the 21 st International Conference on Medical Image Computing and Computer-Assisted Intervention. Granada, Spain: Spring: 812-820[DOI:10.1007/978-3-030-00934-2_90http://dx.doi.org/10.1007/978-3-030-00934-2_90]
相关文章
相关作者
相关机构
京公网安备11010802024621