Thyroid nodule malignantrisk detection in ultrasound image by fusing deep and texture features

Jianning Chi; Xiaosheng Yu; Yifei Zhang

doi:10.11834/jig.180232

Medical Image Processing | Views : 0 下载量: 4 CSCD: 11

PDF
Export
Share
Collection
Album

Thyroid nodule malignantrisk detection in ultrasound image by fusing deep and texture features
Vol. 23, Issue 10, Pages: 1582-1593(2018)
Received：03 April 2018，

Revised：2018-5-3，

Published：16 October 2018
DOI： 10.11834/jig.180232
稿件说明：

移动端阅览

DOI：

Jianning Chi, Xiaosheng Yu, Yifei Zhang. Thyroid nodule malignantrisk detection in ultrasound image by fusing deep and texture features[J]. Journal of Image and Graphics, 2018, 23(10): 1582-1593. DOI： 10.11834/jig.180232.

摘要

目的

在甲状腺结节图像中对甲状腺结节进行良恶性分析，对于甲状腺癌的早期诊断有着重要的意义。随着医疗影像学的发展，大部分的早期甲状腺结节可以在超声图像中准确地检测出来，但对于结节的性质仍然缺乏准确的判断。因此，为实现更为准确的早期甲状腺结节良恶性超声图像诊断，避免不必要的针刺或其他病理活检手术、减轻病患生理痛苦和心理压力及其医疗费用，提出一种基于深度网络和浅层纹理特征融合的甲状腺结节良恶性分类新算法。

方法

本文提出的甲状腺结节分类算法由4步组成。首先对超声图像进行尺度配准、人工标记以及图像复原去除以增强图像质量。然后，对增强的图像进行数据扩展，并作为训练集对预训练过的GoogLeNet卷积神经网络进行迁移学习以提取图像中的深度特征。同时，提取图像的旋转不变性局部二值模式（LBP）特征作为图像的纹理特征。最后，将深度特征与图像的纹理特征相融合并输入至代价敏感随机森林分类器中对图像进行良恶性分类。

结果

本文方法在标准的甲状腺结节癌变数据集上对甲状腺结节图像取得了正确率99.15%，敏感性99.73%，特异性95.85%以及ROC曲线下面积0.997 0的的好成绩，优于现有的甲状腺结节图像分类方法。

结论

实验结果表明，图像的深度特征可以描述医疗超声图像中病灶的整体感官特征，而浅层次纹理特征则可以描述超声图像的边缘、灰度分布等特征，将二者统一的融合特征则可以更为全面地描述图像中病灶区域与非病灶区域之间的差异以及不同病灶性质之间的差异。因此，本文方法可以准确地对甲状腺结节进行分类从而避免不必要手术、减轻病患痛苦和压力。

Abstract

Objective

Detection and analysis of thyroid nodules play vital roles in diagnosing thyroid cancer.With the development of theories and technologies in medical imaging

most of the thyroid nodules can be incidentally detected in the early stage.However

the nature of nodule lacks accurate judgement

leading to that many patients with benign nodules still need Fine Needle Aspiration (FNA) biopsies or surgeries

increasing the physical pain and mental pressure of patients as well as unnecessary medical health care costs.Therefore

we present an image-based computer-aided diagnosis (CAD) system for classifying the thyroid nodules in ultrasound images in this paper

which novelly applies the image features fused by both high-level features from deep learning network and low-level features from texture descriptor.

Method

Our proposed thyroid nodule classification method consists of four steps.Firstly

we pre-process the ultrasound image to enhance the image quality

including searching scale ticks

calibrating images so that the pixel-distance in each image represents the same real-world distance

removing the artifacts and restoring images so that the lesion regions in the images are not interrupted.Secondly

the enhanced images are augmented to enlarge the size of training data set and used to fine-tune the parameters of the pre-trained GoogLeNet convolutional neural network.Meanwhile the rotation invariant uniform local binary pattern (ULBP) features are extracted from each of the images as the low-level texture features.Thirdly

the high-level deep features extracted by the fine-tuned GoogLeNet neural network and the low-level ULBP features are normalized and cascaded as one fusion feature that can represent both the semantic context and the texture patterns distributed in the image.Finally

the fusion features of the images are sent to the Cost-sensitive Random Forest classifier to classify the images into "malignant" and "benign" cases.

Result

The proposed classification method is applied to a standard open access thyroid nodule database to evaluate the effectiveness

achieving excellent classification performance where the accuracy

sensitivity

specificity and area of the ROC curve are 99.15%

99.73%

95.85% and 0.9970 respectively.

Conclusion

Experimental results indicate that the high-level features extracted by the deep neural network from the medical ultrasound image can reflect the visual features of the lesion region

while the low-level texture features can describe the edges

direction and distribution of intensities.The combination of the two types of features can describe the differences between the lesion regions and other regions

and the differences between lesions regions of malignant and benign thyroid nodules.Therefore

the proposed method can classify the thyroid nodules accurately

and provides the superior performance over most of the state-of-the-art thyroid nodule classification approaches

especially in reducing the false positive rate of the diagnosis.

关键词

Keywords

references

Gaitini D, Evans R M, Ivanac G.Thyroid ultrasound[M]//Dietrich C F.EFSUMB-European Course Book.Germany: EFSUMB, 2011.

Kwak J Y, Han K H, Yoon J H, et al.Thyroid imaging reporting and data system for us features of nodules:a step in establishing better stratification of cancer risk[J].Radiology, 2011, 260(3):892-899.[DOI:10.1148/radiol.11110206]

Lingam R K, Qarib M H, Tolley N S.Evaluating thyroid nodules:predicting and selectingmalignant nodules for fine-needle aspiration (FNA) cytology[J].Insights into Imaging, 2013, 4(5):617-624.[DOI:10.1007/s13244-013-0256-6]

Chen S J, Chang C Y, Chang K Y, et al.Classification of the thyroid nodules based on characteristic sonographic textural feature and correlated histopathology using hierarchical support vector machines[J].Ultrasound in Medicine and Biology, 2010, 36(12):2018-2026.[DOI:10.1016/j.ultrasmedbio.2010.08.019]

Chang C Y, Liu H Y, Tseng C H, et al.Computer-aided diagnosis for thyroid graves' disease in ultrasound images[J].Biomedical Engineering:Applications, Basis and Communications, 2010, 22(2):91-99.[DOI:10.4015/S1016237210001815]

Katsigiannis S, Keramidas E G, Maroulis D.A contourlet transform feature extraction scheme for ultrasound thyroid texture classification[J].International Journal of Engineering Intelligent Systems for Electrical Engineering and Communications, 2010, 18(3-4):171.

Acharya U R, Sree S V, Swapna G, et al.Effect of complex wavelet transform filter on thyroid tumor classification in three-dimensional ultrasound[J].Proceedings of the Institution of Mechanical Engineers, Part H:Journal of Engineering in Medicine, 2013, 227(3):284-292.[DOI:10.1177/0954411912472422]

Savelonas M A, Iakovidis D K, Dimitropoulos N, et al.Computational characterization of thyroid tissue in the radon domain[C]//The Twentieth IEEE International Symposium on Computer-Based Medical Systems.Maribor, Slovenia: IEEE, 2007: 189-192.[ DOI: 10.1109/CBMS.2007.33 http://dx.doi.org/10.1109/CBMS.2007.33 ]

Iakovidis D K, Keramidas E G, Maroulis D.Fuzzy local binary patterns for ultrasound texture characterization[C]//Proceedings of the 5th International Conference on Image Analysis and Recognition.Póvoa de Varzim, Portugal: Springer, 2008: 750-759.[ DOI 10.1007/978-3-540-69812-8_74 http://dx.doi.org/10.1007/978-3-540-69812-8_74 ]

Chang C Y, Chen S J, Tsai M F.Application of support-vector-machine-based method for feature selection and classification of thyroid nodules in ultrasound images[J].Pattern Recognition, 2010, 43(10):3494-3506.[DOI:10.1016/j.patcog.2010.04.023]

Acharya U R, Swapna G, Sree S V, et al.A review on ultrasound-based thyroid cancer tissue characterization and automated classification[J].Technology in Cancer Research&Treatment, 2014, 13(4):289-301.[DOI:10.7785/tcrt.2012.500381]

Krizhevsky A, Sutskever I, Hinton G E.ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems.Lake Tahoe, Nevada: Curran Associates Inc., 2012: 1097-1105.

Hijazi S, Kumar R, Rowen C.Using convolutional neural networks for image recognition[R].Cadence Design Systems, Inc., San Jose, CA, USA,, 2015.

Spanhol F A, Oliveira L S, Petitjean C, et al.Breast cancer histopathological image classification using convolutional neural networks[C]//Proceedings of 2016 International Joint Conference on Neural Networks.Vancouver, BC, Canada: IEEE, 2016: 2560-2567.[ DOI: 10.1109/IJCNN.2016.7727519 http://dx.doi.org/10.1109/IJCNN.2016.7727519 ]

Li W, Cao P, Zhao D Z, et al.Pulmonary nodule classification with deep convolutional neural networks on computed tomography images[J].Computational and Mathematical Methods in Medicine, 2016, 2016:#6215085.[DOI:10.1155/2016/6215085]

Roth R H, Lu L, Seff A, et al.A new 2.5D representation for lymph node detection using random sets of deep convolutional neural network observations[C] //Proceedings of the 17th International Conference on Medical Image Computing and Computer-Assisted Intervention.Boston, MA, USA: Springer, 2014: 520-527.[ DOI: 10.1007/978-3-319-10404-1_65 http://dx.doi.org/10.1007/978-3-319-10404-1_65 ]

Szegedy C, Liu W, Jia Y Q, et al.Going deeper with convolutions[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston, MA, USA: IEEE, 2015: 1-9.[ DOI: 10.1109/CVPR.2015.7298594 http://dx.doi.org/10.1109/CVPR.2015.7298594 ]

Dunn P F.Measurement and Data Analysis for Engineering and Science[M].New York:McGraw-Hill, 2005.

Lloyd S P.Least squares quantization in PCM[J].IEEE Transactions on Information Theory, 1982, 28(2):129-137.[DOI:10.1109/TIT.1982.1056489]

Osher S, Burger M, Goldfarb D, et al.An iterative regularization method for total variation-based image restoration[J].Multiscale Modelling&Simulation, 2005, 4(2):460-489.[DOI:10.1137/040605412].

Gando G, Yamada T, Sato H, et al.Fine-tuning deep convolutional neural networks for distinguishing illustrations from photographs[J].Expert Systems with Applications, 2016, 66:295-301.[DOI:10.1016/j.eswa.2016.08.057]

Jia Y Q, Shelhamer E, Donahue J, et al.Caffe: convolutional architecture for fast feature embedding[C]//Proceedings of the 22nd ACM International Conference on Multimedia.Orlando, Florida, USA: ACM, 2014: 675-678.[ DOI: 10.1145/2647868.2654889 http://dx.doi.org/10.1145/2647868.2654889 ]

Ojala T, Pietikainen M, Maenpaa T.Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7):971-987.[DOI:10.1109/TPAMI.2002.1017623]

Zadrozny B, Langford J, Abe N.Cost-sensitive learning by cost-proportionate example weighting[C]//Proceedings of the 3rd IEEE International Conference on Data Mining.Melbourne, FL, USA: IEEE, 2003: 435-442.[ DOI: 10.1109/ICDM.2003.1250950 http://dx.doi.org/10.1109/ICDM.2003.1250950 ]

Witten I H, Frank E.Weka:Practical machine learning tools and techniques with Java implementations[J].ACM Sigmod Record, 1999, 31(1):76-77.

Pedraza L, Vargas C, Narváez F, et al.An open access thyroid ultrasound image database[C]//Proceedings of SPIE Volume 9287, 10th International Symposium on Medical Information Processing and Analysis.Cartagena de Indias, Colombia: SPIE, 2015: #92870W.[ DOI: 10.1117/12.2073532 http://dx.doi.org/10.1117/12.2073532 ]

Bhullar E W, Babyn P, Groot G, et al.A deep learning approach to thyroid nodule classification in sonographic images[C]//Proceedings of 2016 Scientific Conference on Machine Intelligence in Medical Imaging.Alexandria, VA, USA: C-MIMI, 2016.

Lim K J, Choi C S, Yoon D Y, et al.Computer-aided diagnosis for the differentiation of malignant from benign thyroid nodules on ultrasonography[J].Academic Radiology, 2008, 15(7):853-858.[DOI:10.1016/j.acra.2007.12.022]

Savelonas M, Maroulis D, Sangriotis M.A computer-aided system for malignancy risk assessment of nodules in thyroid US images based on boundary features[J].Computer Methods and Programs in Biomedicine, 2009, 96(1):25-32.[DOI:10.1016/j.cmpb.2009.04.001]

Iakovidis D K, Keramidas E G, Maroulis D.Fusion of fuzzy statistical distributions for classification of thyroid ultrasound patterns[J].Artificial Intelligence in Medicine, 2010, 50(1):33-41.[DOI:10.1016/j.artmed.2010.04.004]

Legakis I, Savelonas M A, Maroulis D, et al.Computer-based nodule malignancy risk assessment in thyroid ultrasound images[J].International Journal of Computers and Applications, 2011, 33(1):29-35.[DOI:10.2316/Journal.202.2011.1.202-2955]

Luo S, Kim E H, Dighe M, et al.Thyroid nodule classification using ultrasound elastography via linear discriminant analysis[J].Ultrasonics, 2011, 51(4):425-431.[DOI:10.1016/j.ultras.2010.11.008]

Alert me when the article has been cited

提交

Survey on the applications of deep learning to histopathology

Classification modeling and recognition for cross modal and multi-label biomedical image

Correlation subdomain alignment network based cross-domain hyperspectral image classification method

Ultrasonic image segmentation of thyroid nodules-relevant multi-scale feature based h-shape network

Model distillation for high-level semantic understanding： a survey