浏览全部资源
扫码关注微信
收稿：2021-03-24，
修回：2021-4-19，
录用：2021-4-26，
纸质出版：2022-10-16
移动端阅览
目的
2
超声弹性成像技术已逐步应用于支气管淋巴结良恶性的诊断，帮助确定肺癌分期。在支气管超声弹性图像中，淋巴结区域的精确定位对诊断准确度具有重要影响，但通常依赖专业医师的手动分割，费时费力。为此，本文设计了一种注意力上下文编码器网络(attention context encoder network，ACE-Net)。
方法
2
本文网络模型包括编码器、上下文提取器和解码器3部分。使用在ImageNet数据集上预训练且去掉平均池化层和全连接层的34层残差网络ResNet-34作为编码器提取特征，上下文提取器从编码器的输出中进一步提取高级语义信息，同时保留尽可能多的空间信息，基于AG(attention gate)的解码器可以抑制输入图像中的不相关区域，同时突出对当前任务更关键的特征。
结果
2
实验在本文收集的包含支气管超声弹性图像及对应分割标签的数据集上进行，与6种典型的U-Net结构深度网络模型的分割性能进行对比，数据集中的每幅图像中的淋巴结都由专业医师手动分割标注。基础U-Net网络得到淋巴结分割结果的Dice系数、敏感度和特异度分别为0.820 7、85.08%和96.82%，其他改进网络的分割性能在此基础上均有一定提高，本文方法的Dice系数、敏感度和特异度分别为0.845 1、87.92%和97.04%，Dice系数和敏感度在所有方法中取得了最优值，特异度取得了次优值。
结论
2
以U-Net为代表的深度学习模型在支气管超声弹性图像淋巴结分割问题中具有很大潜力，将上下文提取器和注意力机制融入U-Net网络可以一定程度提升分割精度。本文收集的数据集将有助于推动支气管超声弹性图像淋巴结分割问题的研究。
Objective
2
Lung cancer threatens human health severely. It is one of the malignant tumors with the fastest increase in morbidity and mortality and the greatest threat to the life of the population. In the past 50 years
many countries have addressed that the incidence and mortality of lung cancer have increased significantly. The incidence and mortality of lung cancer rank the first among all the malignant tumors. Recent ultrasound elastography technology has been gradually applied to diagnose the benign and malignant bronchial lymph nodes to aid the degree analysis of lung cancer. Ultrasonic elastography provides more information than conventional two-dimensional ultrasound via the evaluation of lesion toughness. Color Doppler energy imaging superimposes the color coding system on the conventional ultrasound image. In general
the hardness of the diseased lymph node is relatively large
and the degree of deformation is small after being squeezed
which is represented as blue color in the elastic image. The normal lymph node is relatively soft
which is represented as red or green colors in the elastic image. Bronchial ultrasound elastography is generated through the squeezing deformation issues of the lymph nodes in related to the record of heartbeat
breathing movement and the pulsation of blood vessels around the lungs. In bronchial ultrasound elastic images
the precise positioning of the lymph node area is of great significance to the diagnosis accuracy of the disease. However
this kind of task is time-consuming and laborious due to its manual segmentation in clinical. We carried out the deep learning based automatic segmentation method of mediastinal lymph nodes in bronchial ultrasound elastic images via U-Net-type architectures.
Method
2
A dataset consisting of 205 bronchial ultrasound elastic images and corresponding segmentation labels is collected. The lymph nodes of each image are manually segmented and labeled. Based on this dataset
six classic deep network models based on U-Net are tested. The U-Net has an encoder-decoder structure. The encoder aims to capture more advanced semantic features and reduce the spatial dimension of the feature map gradually
while the decoder is used to restore spatial details and dimensions. We design a new U-Net-based bronchial ultrasound elastic image segmentation method based on the integration of context extractor and attention mechanism. To avoid gradient explosion and disappearance
the encoder is the ResNet-34 pretrained on ImageNet with no average pooling layer and the fully connected layer. The context extractor is used to extract high-level semantic information further from the output of the encoder while preserving as much spatial information as possible. The attention mechanism aims to select features that are more important to the current task. The prediction result of the segmentation network is the probability value of the pixel classification
so a binarization operation is performed by setting the threshold to 0.5. That is
the pixels are assigned to 0 if the probability value is less than 0.5 and otherwise is 1. In this way
the segmented binary image is obtained.
Result
2
To verify the performance of different networks
a five-fold cross validation evaluation is conducted on the dataset. That is
we divide the dataset into five equal parts in random
and four of them is as the training set each time and the remaining one is as the testing set. The preprocessing operations are related to data cropping
data augmentation and normalized operation on the training set. The input images and the ground-truth segmentation maps are resized to 320×320 pixels. Data augmentation approaches include random vertical flip and random angle rotation. The Adam optimizer is selected and the learning rate is set to 0.000 1. The batch size is set to 8. The number of epoch is 150. The GPU used in the experiment is GeForce RTX 2080Ti. The segmentation task is implemented using python3.7 under the Ubuntu16.04.1 operating system and the core framework is pytorch1.7.1. The results of Dice coefficient
sensitivity and specificity of U-Net network lymph node segmentation are 0.820 7
85.08% and 96.82%
respectively. On this basis
the segmentation performances of other modified versions of U-Net are all improved to a certain extent. Among them
our Dice coefficient
sensitivity and specificity are 0.845 1
87.92% and 97.04% of each
which are 0.024 4
2.84% and 0.22% higher than the baseline U-Net
respectively. Compared to the other methods
the Dice coefficient and the sensitivity achieve the first place
while the specificity ranks the second.
Conclusion
2
Our analyses demonstrate that deep learning models represented by U-Net have great potential in the segmentation of mediastinal lymph nodes in bronchial ultrasound elastic images. Fused by the context extractor and attention mechanism
the integrated U-Net network can improve the segmentation accuracy to a certain extent. In addition
the illustrated dataset can promote the research of lymph node segmentation in bronchial ultrasound elastic images. Our method can be used for the segmentation of lymph nodes in bronchial ultrasound elastography images. It has potentials for the segmentation of more medical imaging organs and tumors as well. However
due to the relatively small scale of the dataset
there is still large room for further improvement on the segmentation performance
although the data augmentation approaches have been performed. To improve the segmentation accuracy further
it is required to increase the scale of the dataset in consistency.
Abedalla A, Abdullah M, Al-Ayyoub M and Benkhelifa E. 2020. 2ST-UNet: 2-stage training model using U-Net for pneumothorax segmentation in chest X-rays//Proceedings of 2020 International Joint Conference on Neural Networks (IJCNN). Glasgow, UK: IEEE: 1-6 [ DOI:10.1109/IJCNN48605.2020.9207268 http://dx.doi.org/10.1109/IJCNN48605.2020.9207268 ]
Gao Y Q, Guo S, Li N, Wang K, Kang H and Li T. 2020. Arteriovenous classification method in fundus images based on semantic fusion. Journal of Image and Graphics, 25(10): 2259-2270
高颖琪, 郭松, 李宁, 王恺, 康宏, 李涛. 2020. 语义融合眼底图像动静脉分类方法. 中国图象图形学报, 25(10): 2259-2270 [DOI: 10.11834/jig.200187]
Gu Z W, Cheng J, Fu H Z, Zhou K, Hao H Y, Zhao Y T, Zhang T Y, Gao S H and Liu J. 2019. CE-Net: context encoder network for 2D medical image segmentation. IEEE Transactions on Medical Imaging, 38(10): 2281-2292 [DOI: 10.1109/TMI.2019.2903562]
Guo X D, Zhang N, Guo J F, Zhang H H, Hao Y G and Hang J Q. 2019. Automated polyp segmentation for colonoscopy images: a method based on convolutional neural networks and ensemble learning. Medical Physics, 46(12): 5666-5676 [DOI: 10.1002/mp.13865]
Hao X Y, Xiong J F, Xue X D, Shi J, Wen K, Han W T, Li X Y, Zhao J and Fu X L. 2020. 3D U-Net with dual attention mechanism for lung tumor segmentation. Journal of Image and Graphics, 25(10): 2119-2127
郝晓宇, 熊俊峰, 薛旭东, 石军, 文可, 韩文廷, 李骁扬, 赵俊, 傅小龙. 2020. 融合双注意力机制3D U-Net的肺肿瘤分割. 中国图象图形学报, 25(10): 2119-2127 [DOI: 10.11834/jig.200282]
He H Y, Lyu X D, Ma H, Zhu J and Huang M. 2016. Value of endobronchial ultrasound elastography in the diagnosis of mediastinal and hilar lymph node metastasis in lung cancer. Journal of Central South University (Medical Science), 41(1): 30-36
何海艳, 吕学东, 马航, 朱杰, 黄茂. 2016. 气道内超声弹性技术对肺癌患者肺门纵膈淋巴结转移的诊断价值. 中南大学学报(医学版), 41(1): 30-36 [DOI: 10.11817/j.issn.1672-7347.2016.01.005]
He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE: 770-778 [ DOI:10.1109/CVPR.2016.90 http://dx.doi.org/10.1109/CVPR.2016.90 ]
Huang G, Liu Z, Van Der Maaten L and Weinberger K Q. 2017a. Densely connected convolutional networks//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE: 2261-2269 [ DOI:10.1109/CVPR.2017.243 http://dx.doi.org/10.1109/CVPR.2017.243 ]
Huang H D, Huang Z A, Wang Q, Wang X N, Dong Y C, Zhang W, Zarogoulidis P, Man Y G, Schmidt W H and Bai C. 2017b. Effectiveness of the benign and malignant diagnosis of mediastinal and hilar lymph nodes by endobronchial ultrasound elastography. Journal of Cancer, 8(10): 1843-1848 [DOI: 10.7150/jca.19819]
Jiang Z K, Lyu X G, Zhang J X, Zhang Q and Wei X P. 2020. Review of deep learning methods for MRI brain tumor image segmentation. Journal of Image and Graphics, 25(2): 215-228
江宗康, 吕晓钢, 张建新, 张强, 魏小鹏. 2020. MRI脑肿瘤图像分割的深度学习方法综述. 中国图象图形学报, 25(2): 215-228 [DOI: 10.11834/jig.190173]
Li L Z, Gu W, Tang Y, Su N J, Shu Y and Fang S R. 2018. Endobronchial ultrasound elastography in diagnosis of mediastinal lymph nodes. China Journal of Endoscopy, 24(5): 23-30
李灵芝, 谷伟, 唐云, 苏宁静, 舒雅, 方苏榕. 2018. 气道内超声弹性成像技术在纵隔淋巴结性质判定中的价值. 中国内镜杂志, 24(5): 23-30 [DOI: 10.3969/j.issn.1007-1989.2018.05.005]
Li X M, Chen H, Qi X J, Dou Q, Fu C W and Heng P A. 2018. H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Transactions on Medical Imaging, 37(12): 2663-2674 [DOI: 10.1109/TMI.2018.2845918]
Liu Y P, Liu G P, Wang R F, Jin R, Sun D C, Qiu H, Dong C, Li J and Hong G B. 2020. Accurate segmentation method of liver tumor CT based on the combination of deep learning and radiomics. Journal of Image and Graphics, 25(10): 2128-2141
刘云鹏, 刘光品, 王仁芳, 金冉, 孙德超, 邱虹, 董晨, 李瑾, 洪国斌. 2020. 深度学习结合影像组学的肝脏肿瘤CT分割. 中国图象图形学报, 25(10): 2128-2141 [DOI: 10.11834/jig.200198]
Oktay O, Schlemper J, Le Folgoc L, Lee M, Heinrich M, Misawa K, Mori K, McDonagh S, Hammerla N Y, Kainz B, Glocker B and Rueckert D. 2018. Attention U-Net: learning where to look for the pancreas [EB/OL]. [2021-03-22] . https://arxiv.org/pdf/1804.03999.pdf https://arxiv.org/pdf/1804.03999.pdf
Ronneberger O, Fischer P and Brox T. 2015. U-Net: convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: Springer: 234-241 [ DOI:10.1007/978-3-319-24574-4_28 http://dx.doi.org/10.1007/978-3-319-24574-4_28 ]
Shelhamer E, Long J and Darrell T. 2017. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4): 640-651 [DOI: 10.1109/TPAMI.2016.2572683]
Simonyan K and Zisserman A. 2014. Very deep convolutional networks for large-scale image recognition [EB/OL]. [2021-03-22] . https://arxiv.org/pdf/1409.1556.pdf https://arxiv.org/pdf/1409.1556.pdf
Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V and Rabinovich A. 2015. Going deeper with convolutions//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, USA: IEEE: 1-9 [ DOI:10.1109/CVPR.2015.7298594 http://dx.doi.org/10.1109/CVPR.2015.7298594 ]
Wang H, Wan Y X, Zhang L, Tao H Y and Huang H R. 2018. Clinical value of bronchial ultrasound elastography in the differential diagnosis of benign and malignant hilar and mediastinal lymph nodes. Chinese Journal of Clinical Oncology, 45(14): 721-725
王虹, 万毅新, 张丽, 陶红艳, 黄晖蓉. 2018. 支气管超声弹性成像技术对肺门及纵隔淋巴结良恶性鉴别诊断的临床价值. 中国肿瘤临床, 45(14): 721-725 [DOI: 10.3969/j.issn.1000-8179.2018.14.358]
Woo S, Park J, Lee J Y and Kweon I S. 2018. CBAM: convolutional block attention module//Proceedings of the 15th European Conference on Computer Vision. Munich, Germany: Springer: 3-19 [ DOI:10.1007/978-3-030-01234-2_1 http://dx.doi.org/10.1007/978-3-030-01234-2_1 ]
相关作者
相关机构
京公网安备11010802024621
