浏览全部资源
扫码关注微信
Published： 16 July 2023 ，
移动端阅览
于典， 彭延军， 郭燕飞. 2023. 面向甲状腺结节超声图像分割的多尺度特征融合“h”形网络. 中国图象图形学报， 28(07):2195-2207
Yu Dian， Peng Yanjun， Guo Yanfei. 2023. Ultrasonic image segmentation of thyroid nodules-relevant multi-scale feature based h-shape network. Journal of Image and Graphics， 28(07):2195-2207
目的
2
准确定位超声甲状腺结节对甲状腺癌早期诊断具有重要意义，但患者结节大小、形状以及位置的不确定性极大影响了结节分割的准确率和模型的泛化能力。为了提高超声甲状腺结节分割的精度，增强泛化性能并降低模型的参数量，辅助医生诊断疾病，减少误诊，提出一种面向甲状腺结节超声图像分割的多尺度特征融合“h”形网络。
方法
2
首先提出一种网络框架，形状与字母h相似，由一个编码器和两个解码器组成，引入深度可分离卷积缩小网络尺寸。编码器用于提取图像特征，且构建增强下采样模块来减少下采样时造成的信息损失，增强解码器特征提取的能力。第1个解码器负责获取图像的初步分割信息；第2个解码器通过融合第1个解码器预先学习到的信息来增强结节的特征表达，提升分割精度，并设计了融合卷积池化金字塔实现多尺度特征融合，增强模型的泛化能力。
结果
2
该网络在内部数据集上的Dice相似系数（Dice similarity coefficients， DSC）、豪斯多夫距离（Hausdorff distance，HD）、灵敏度（sensitivity，SEN）和特异度（specificity，SPE）分别为0.872 1、0.935 6、0.879 7和0.997 3，在公开数据集DDTI（digital database thyroid image）上，DSC和SPE分别为0.758 0和0.977 3，在数据集TN3K（thyroid nodule 3 thousand）上的重要指标DSC和HD分别为0.781 5和4.472 6，皆优于其他模型。
结论
2
该网络模型以较低的参数量提升了甲状腺超声图像结节的分割效果，增强了泛化性能。
Objective
2
Early diagnosis of thyroid cancer-beneficial lesions of ultra-sound thyroid nodules are required to be located accurately. Ultra-sound imaging technique is potential for the diagnosis of thyroid diseases and it is cost-effective and simplified to a certain extent. Thyroid imaging reporting and data system （TI-RADS） is focused on benign and malignant nodules-relevant evaluation recently. The probability of the nodule will be much more distorted when the level is higher. The first step of ultrasound evaluation is oriented to segment thyroid nodule. At present， the commonly-used segmentation method is focused on manual segmentation， which is still labor intensive and experience behavioral. Computer technology-based medical imaging technique is focused on realizing automatic segmentation of ultrasonic nodules and the speed and accuracy of diagnosis can be improved. Current deep learning technique has its potentials for several of visual recognition tasks in recent years. Compared to traditional contour-shape and region based methods， deep learning technology is preferred to improve the accuracy of tasks. Fully convolutional neural network （FCN） and convolutional neural network （CNN） based multiple models are oriented to achieve specific segmentation tasks. However， the speckle noise of ultrasound images and the uncertainty of the size， shape and location of the patient’s nodules have affected the accuracy of nodule greatly.
Method
2
First， the h-shape network framework is proposed in terms of an encoder and two decoders. The shape of the framework is similar to the letter “h”， and the depth separable convolution is introduced to shrink the network size. The second convolution of each layer in the network is replaced by the depth separable convolution to lower the number of parameters of the model. The encoder is used to extract image features， and the enhanced down-sampling module is constructed to alleviate down-sampling-led information loss. The module is composed of a connection of maximum pool and average pool， batch normalization and average pool， which is used to enhance the feature extraction capability of the decoder. The first decoder is in supporting of the preliminary segmentation information of the image， and the second decoder can be used to enhance the feature expression of the nodules， and the segmentation accuracy can be improved via the first decoder-related fusion of the learned information. Finally， the fusion convolutional pyramid pooling module is designed as well， in which atrous spatial pyramid pooling module and deep separable convolution can be integrated together to realize multi-scale feature fusion while the network size and the generalization ability of the model are optimized. The four sorts of decoder blocks of the second decoder can be operated through its fusion convolutional pyramid pooling module for each of them， and final prediction result is generated after concat operation. Three kinds of datasets are provided to verify the model well， which consists of 3 622 ultrasound images-within internal dataset， 637 ultrasound images-involved digital database thyroid image dataset， and 3 493 ultrasound images-included TN3K public dataset. The internal dataset and TN3K dataset are divided into training set， validation set， and test set in a ratio of 8∶1∶1 to train the model. Due to the small number of thyroid nodule samples are linked to the DDTI dataset， partitioning of the DDTI dataset is prone to be over fitted， and the weight information of the internal dataset can be used to test the DDTI dataset straightforward. The experiment is built on the Pytorch framework and Nvidia RTX 2080 TI is used to train the model. Using Adam as the optimizer， the initial learning rate is 0.000 1. There are 200 rounds of training， and the learning rate is lower to half every 20 rounds. The batch size is set to 8. To perform well in the segmentation of small nodules on the basis of stable overall segmentation， DiceBCELoss is regarded as the loss function， which can combine BCE loss function with Dice loss function. The segmentation results are analyzed in quantitative in terms of the Dice similarity coefficient （DSC）， Hausdorff distance （HD）， sensitivity （SEN）， and specificity （SPE）.
Result
2
To validate the ability of the proposed method， comparative analysis is carried out， which is in comparison with AttentionUNet， marker-guided U-Net （MG-UNet）， fully convolutional dense dilated Net （FCdDN）， DeepLab V3+， segmentation network （segNet） and context encoder network （CE_NET）. For the internal dataset， the DSC， HD， SEN and SPE indexes of the proposed model are reached to 0.872 1， 0.935 6， 0.879 7 and 0.997 3 each. The DSC is 15.53% improved than the worst model， and 1.2% improved than the second best one； The HD is 2.583 6 decreased than the worst model， and 0.034 1 decreased than the second best model； The SEN and SPE are increased by 0.32% and 1.17% than the second best model， which are 7.57% and 9.96% higher than the worst model. For the digital database thyroid image dataset， the DSC and SPE are 0.758 0 and 0.977 3 each， which are 9.83% and 15.25% improved than the worst model， 1.02% and 0.71% increased than the best model. The DSC of 0.781 5 and the HD of 4.472 6 are obtained on TN3K dataset， which are 1.27% higher and 0.634 5 lower than the model with the second best performance. Furthermore， a series of ablation experiments are conducted on the proposed model as well to demonstrate the effectiveness of the different steps of the fusion algorithm.
Conclusion
2
This proposed network can improve the segmentation accuracy of thyroid nodules， and its computing cost and generalization ability of the model can be optimized further.
深度学习甲状腺结节超声分割h网络增强下采样多尺度
deep learningthyroid noduleultrasound segmentationh-networkenhanced down-samplingmulti-scale
Amiri M， Brooks R and Rivaz H. 2020. Fine-tuning U-Net for ultrasound image segmentation： different layers， different outcomes. IEEE Transactions on Ultrasonics， Ferroelectrics， and Frequency Control， 67（12）： 2510-2518 ［DOI： 10.1109/TUFFC.2020.3015081http://dx.doi.org/10.1109/TUFFC.2020.3015081］
Avola D， Cinque L， Fagioli A， Foresti G and Mecca A. 2021. Ultrasound medical imaging techniques： a survey. ACM Computing Surveys （CSUR）， 54（3）： #67 ［DOI： 10.1145/3447243http://dx.doi.org/10.1145/3447243］
Badrinarayanan V， Kendall A and Cipolla R. 2017. SegNet： a deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence， 39（12）： 2481-2495 ［DOI： 10.1109/TPAMI.2016.2644615http://dx.doi.org/10.1109/TPAMI.2016.2644615］
Buda M， Wildman-Tobriner B， Castor K， Hoang J K and Mazurowski M A. 2020. Deep learning-based segmentation of nodules in thyroid ultrasound： improving performance by utilizing markers present in the images. Ultrasound in Medicine and Biology， 46（2）： 415-421 ［DOI： 10.1016/j.ultrasmedbio.2019.10.003http://dx.doi.org/10.1016/j.ultrasmedbio.2019.10.003］
Chen J Y， You H J and Li K. 2020a. A review of thyroid gland segmentation and thyroid nodule segmentation methods for medical ultrasound images. Computer Methods and Programs in Biomedicine， 185： #105329 ［DOI： 10.1016/j.cmpb.2020.105329http://dx.doi.org/10.1016/j.cmpb.2020.105329］
Chen L C， Papandreou G， Kokkinos I， Murphy K and Yuille A L. 2018a. DeepLab： semantic image segmentation with deep convolutional nets， atrous convolution， and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence， 40（4）： 834-848 ［DOI： 10.1109/TPAMI.2017.2699184http://dx.doi.org/10.1109/TPAMI.2017.2699184］
Chen L C， Zhu Y K， Papandreou G， Schroff F and Adam H. 2018b. Encoder-decoder with atrous separable convolution for semantic image segmentation//Proceedings of the 15th European Conference on Computer Vision （ECCV）. Munich， Germany： Springer： 833-851 ［DOI： 10.1007/978-3-030-01234-2_49http://dx.doi.org/10.1007/978-3-030-01234-2_49］
Chen T Y， Cai Z H， Zhao X， Chen C， Liang X F， Zou T R and Wang P. 2020b. Pavement crack detection and recognition using the architecture of segNet. Journal of Industrial Information Integration， 18： #100144 ［DOI： 10.1016/j.jii.2020.100144http://dx.doi.org/10.1016/j.jii.2020.100144］
Chi J N， Yu X S and Zhang Y F. 2018. Thyroid nodule malignantrisk detection in ultrasound image by fusing deep and texture features. Journal of Image and Graphics， 23（10）： 1582-1593
迟剑宁， 于晓升， 张艺菲. 2018. 融合深度网络和浅层纹理特征的甲状腺结节癌变超声图像诊断. 中国图象图形学报， 23（10）： 1582-1593 ［DOI： 10.11834/jig.180232http://dx.doi.org/10.11834/jig.180232］
Chu C， Zheng J H and Zhou Y. 2021. Ultrasonic thyroid nodule detection method based on U-Net network. Computer Methods and Programs in Biomedicine， 199： #105906 ［DOI： 10.1016/j.cmpb.2020.105906http://dx.doi.org/10.1016/j.cmpb.2020.105906］
De Macedo B M， Izquierdo R F， Golbert L and Meyer E L S. 2018. Reliability of thyroid imaging reporting and data system （TI-RADS）， and ultrasonographic classification of the American Thyroid Association （ATA） in differentiating benign from malignant thyroid nodules. Archives of Endocrinology and Metabolism， 62： 131-138 ［DOI： 10.20945/2359-3997000000018http://dx.doi.org/10.20945/2359-3997000000018］
Gong H F， Chen G Q， Wang R R， Xie X， Mao M Z， Yu Y Z， Chen F and Li G B. 2021. Multi-task learning for thyroid nodule segmentation with thyroid region prior//Proceedings of the 18th IEEE International Symposium on Biomedical Imaging （ISBI）. Nice， France： IEEE： 257-261 ［DOI： 10.1109/ISBI48211.2021.9434087http://dx.doi.org/10.1109/ISBI48211.2021.9434087］
Gu Z W， Cheng J， Fu H Z， Zhou K， Hao H Y， Zhao Y T， Zhang T Y， Gao S H and Liu J. 2019. CE-Net： context encoder network for 2D medical image segmentation. IEEE Transactions on Medical Imaging， 38（10）： 2281-2292 ［DOI： 10.1109/TMI.2019.2903562http://dx.doi.org/10.1109/TMI.2019.2903562］
Hu Y S， Qin P L， Zeng J C， Chai R and Wang L F. 2020. Ultrasound thyroid segmentation based on segmented frequency domain and local attention. Journal of Image and Graphics， 25（10）： 2195-2205
胡屹杉， 秦品乐， 曾建潮， 柴锐， 王丽芳. 2020. 结合分段频域和局部注意力的超声甲状腺分割. 中国图象图形学报， 25（10）： 2195-2205 ［DOI： 10.11834/jig.200230http://dx.doi.org/10.11834/jig.200230］
Kamal K C， Yin Z D， Wu M Y and Wu Z L. 2019. Depthwise separable convolution architectures for plant disease classification. Computers and Electronics in Agriculture， 165： #104948 ［DOI： 10.1016/j.compag.2019.104948http://dx.doi.org/10.1016/j.compag.2019.104948］
Khanna A， Londhe N D， Gupta S and Semwal A. 2020. A deep Residual U-Net convolutional neural network for automated lung segmentation in computed tomography images. Biocybernetics and Biomedical Engineering， 40（3）： 1314-1327 ［DOI： 10.1016/j.bbe.2020.07.007http://dx.doi.org/10.1016/j.bbe.2020.07.007］
Liu R Y， Zhou S C， Guo Y， Wang Y Y and Chang C. 2021. U2F-GAN： weakly supervised super-pixel segmentation in thyroid ultrasound images. Cognitive Computation， 13（5）： 1099-1113 ［DOI： 10.1007/s12559-021-09909-7http://dx.doi.org/10.1007/s12559-021-09909-7］
Long J， Shelhamer E and Darrell T. 2015. Fully convolutional networks for semantic segmentation//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Boston， USA： IEEE： 3431-3440 ［DOI： 10.1109/CVPR.2015.7298965http://dx.doi.org/10.1109/CVPR.2015.7298965］
Ma J L， Wu F， Jiang T， Zhao Q Y and Kong D X. 2017. Ultrasound image-based thyroid nodule automatic segmentation using convolutional neural networks. International Journal of Computer Assisted Radiology and Surgery， 12（11）： 1895-1910 ［DOI： 10.1007/s11548-017-1649-7http://dx.doi.org/10.1007/s11548-017-1649-7］
Oktay O， Schlemper J， Le Folgoc L， Lee M， Heinrich M， Misawa K， Mori K， McDonagh S， Hammerla N Y， Kainz B， Glocker B and Rueckert D. 2018. Attention U-Net： learning where to look for the pancreas ［EB/OL］. ［2021-12-08］. https://arxiv.org/pdf/1804.03999.pdfhttps://arxiv.org/pdf/1804.03999.pdf
Ouahabi A and Taleb-Ahmed A. 2021. RETRACTED： deep learning for real-time semantic segmentation： application in ultrasound imaging. Pattern Recognition Letters， 144： 27-34 ［DOI： 10.1016/j.patrec.2021.01.010http://dx.doi.org/10.1016/j.patrec.2021.01.010］
Pedraza L， Vargas C， Narváez F， Durán O， Muñoz E and Romero E. 2015. An open access thyroid ultrasound image database//Proceedings of the 10th International Symposium on Medical Information Processing and Analysis. Cartagena de Indias， Colombia： SPIE： #2073532 ［DOI： 10.1117/12.2073532http://dx.doi.org/10.1117/12.2073532］
Ronneberger O， Fischer P and Brox T. 2015. U-Net： convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich， Germany： Springer： 234-241 ［DOI： 10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28］
Sharifi Y， Bakhshali M A， Dehghani T， DanaiAshgzari M， Sargolzaei M and Eslami S. 2021. Deep learning on ultrasound images of thyroid nodules. Biocybernetics and Biomedical Engineering， 41（2）： 636-655 ［DOI： 10.1016/j.bbe.2021.02.008http://dx.doi.org/10.1016/j.bbe.2021.02.008］
Shin H C， Roth H R， Gao M C， Lu L， Xu Z Y， Nogues I， Yao J H， Mollura D and Summers R M. 2016. Deep convolutional neural networks for computer-aided detection： CNN architectures， dataset characteristics and transfer learning. IEEE Transactions on Medical Imaging， 35（5）： 1285-1298 ［DOI： 10.1109/TMI.2016.2528162http://dx.doi.org/10.1109/TMI.2016.2528162］
Sun J D， Peng Y J， Guo Y F and Li D P. 2021. Segmentation of the multimodal brain tumor image used the multi-pathway architecture method based on 3D FCN. Neurocomputing， 423： 34-45 ［DOI： 10.1016/j.neucom.2020.10.031http://dx.doi.org/10.1016/j.neucom.2020.10.031］
Wang B， Li M X and Liu X. 2022. Ultrasound image segmentation method of thyroid nodules based on the improved U-net network. Journal of Electronics and Information Technology， 44（2）： 514-522
王波， 李梦翔， 刘侠. 2022. 基于改进U-Net网络的甲状腺结节超声图像分割方法. 电子与信息学报， 44（2）： 514-522 ［DOI： 10.11999/JEIT210015http://dx.doi.org/10.11999/JEIT210015］
Wang J and Liu X P. 2021. Medical image recognition and segmentation of pathological slices of gastric cancer based on Deeplab v3+ neural network. Computer Methods and Programs in Biomedicine， 207： #106210 ［DOI： 10.1016/j.cmpb.2021.106210http://dx.doi.org/10.1016/j.cmpb.2021.106210］
Welker M J and Orlov D. 2003. Thyroid nodules. American Family Physician， 67（3）： 559-566
Yang S， Lin G S， Jiang Q P and Lin W S. 2020. A dilated inception network for visual saliency prediction. IEEE Transactions on Multimedia， 22（8）： 2163-2176 ［DOI： 10.1109/TMM.2019.2947352http://dx.doi.org/10.1109/TMM.2019.2947352］
Ying X， Yu Z H， Yu R G， Li X W， Yu M， Zhao M K and Liu K. 2018. Thyroid nodule segmentation in ultrasound images based on cascaded convolutional neural network//Proceedings of the 25th International Conference on Neural Information Processing. Siem Reap， Cambodia： Springer： 373-384 ［DOI： 10.1007/978-3-030-04224-0_32http://dx.doi.org/10.1007/978-3-030-04224-0_32］
相关文章
相关作者
相关机构
京公网安备11010802024621