浏览全部资源
扫码关注微信
收稿：2021-08-13，
修回：2021-11-18，
录用：2021-11-25，
纸质出版：2022-03-16
移动端阅览
目的
2
为了提升基于单模态B型超声($\rm B$超)的乳腺癌计算机辅助诊断(computer-aided diagnosis，CAD)模型性能，提出一种基于两阶段深度迁移学习(two-stage deep transfer learning，TSDTL)的乳腺超声CAD算法，将超声弹性图像中的有效信息迁移至基于$\rm B$超的乳腺癌CAD模型之中，进一步提升该CAD模型的性能。
方法
2
在第1阶段的深度迁移学习中，提出将双模态超声图像重建任务作为一种自监督学习任务，训练一个关联多模态深度卷积神经网络模型，实现$\rm B$超图像和超声弹性图像之间的信息交互迁移；在第2阶段的深度迁移学习中，基于隐式的特权信息学习(learning using privilaged information
LUPI)范式，进行基于双模态超声图像的乳腺肿瘤分类任务，通过标签信息引导下的分类进一步加强两个模态之间的特征融合与信息交互；采用单模态$\rm B$超数据对所对应通道的分类网络进行微调，实现最终的乳腺癌$\rm B$超图像分类模型。
结果
2
实验在一个乳腺肿瘤双模超声数据集上进行算法性能验证。实验结果表明，通过迁移超声弹性图像的信息，TSDTL在基于$\rm B$超的乳腺癌诊断任务中取得的平均分类准确率为87.84±2.08%、平均敏感度为88.89±3.70%、平均特异度为86.71±2.21%、平均约登指数为75.60±4.07%，优于直接基于单模态$\rm B$超训练的分类模型以及多种典型迁移学习算法。
结论
2
提出的TSDTL算法通过两阶段的深度迁移学习，将超声弹性图像的信息有效迁移至基于$\rm B$超的乳腺癌CAD模型，提升了模型的诊断性能，具备潜在的应用可行性。
Objective
2
B-mode ultrasound (BUS) provides information about the structure and morphology information of human breast lesions
while elastography ultrasound (EUS) can provide additional bio-mechanical information. Dual-modal ultrasound imaging can effectively improve the accuracy of the human breast cancer diagnosis. The single-modal ultrasound-based computer-aided diagnosis (CAD) model has its potential applications. Deep transfer learning is a significant branch of transfer learning analysis. This technique can be utilized to guide the information transfer between EUS images and BUS images. However
clinical image samples are limited based on training deep learning models due to the high cost of data collection and annotation. Self-supervised learning (SSL) is an effective solution to demonstrate its potential in a variety of medical image analysis tasks. In respect of the SSL pipeline
the backbone network is trained based on a pretext task
where the supervision information is generated from the train samples without manual annotation. Based on the weight parameters of the trained backbone network
the obtained results are then transferred to the downstream network for further fine-tuning with small size annotated samples. A step-based correlation multi-modal deep convolution neural network (CorrMCNN) has been facilitated to conduct a self-supervised image reconstruction task currently. In the training process
the model transfers the effective information between two modalities to optimize the correlation loss through SSL-based deep transfer learning. Since each BUS and EUS scan the same lesion area for the targeted patient simultaneously
the analyzed results are demonstrated in pairs and share labels. Learning using privileged information (LUPI) is a supervised transfer learning paradigm for paired source domain (privileged information) and target domain data based on shared labels. It can exploit the intrinsic knowledge correlation between the paired data in the source domain and target domain with shared labels
which guides knowledge transfer to improve model capability. Since the label information is used to guide transfer learning in classifiers
the current LUPI algorithm focus on the classifier. Feature representation is also the key step for a qualified CAD system. A two-stage deep transfer learning (TSDTL) algorithm is demonstrated for human breast ultrasound CAD
which transfers the clear information from EUS images to the CAD model of BUS-based human breast cancer and further improves the performance of the CAD model.
Method
2
In the first stage of deep transfer learning
an SSL task is first designed based on dual-modal ultrasonic image reconstruction
which trains the CorrMCNN model to conduct the interactive transfer of information between the two modal images of BUS and EUS images. The bi-channel encoder networks are adopted to learn the feature representation derived from the dual-modal images
respectively. The high-level learned features are used following for concatenation to obtain the joint representation. The original BUS and EUS images are reconstructed from the joint feature representation through the bi-channel decoder networks. In the training process
the network implicitly implements deep transfer learning via the correlation loss optimization amongst high-level features derived from two channels. In the second stage of deep transfer learning
the pre-training backbone network is reused followed by a sub-network for classification. The BUS and EUS images are input into this new network for targeted breast cancer classification based on dual-modal ultrasound images. In this training process
the data of the source domain and target domain can be applied to supervised transfer learning with the shared labels
and this strategy belongs to the general LUPI paradigm. Consequently
it can be considered that this deep transfer learning stage implicitly conducts knowledge transfer under the LUPI paradigm
which is based on the dual-modal ultrasound breast cancer classification task. In the final stage
the corresponding channel sub-network is fine-tuned with single-modal ultrasound data
which obtains an accurate breast cancer B-mode image classification model. The obtained single-channel network is the final network model of BUS-based breast cancer CAD
which can be directly applied to the diagnostic tasks of the emerging BUS images.
Result
2
The performance of the algorithm is demonstrated on a breast tumor dual-modal ultrasound dataset. Our illustrated TSDTL achieves the mean classification accuracy of 87.84±2.08%
sensitivity of 88.89±3.70%
specificity of 86.71±2.21%
and Youden index of 75.60±4.07% respectively
which develops the classification model trained on single-modal BUS images and a variety of typical deep transfer learning algorithms.
Conclusion
2
Our TSDTL algorithm analysis can effectively transfer the information of EUS to the BUS-based human breast cancer CAD model through our illustrated two-stage deep transfer learning.
Bhatt G, Jha P and Raman B. 2019. Representation learning using step-based deep multi-modal autoencoders. Pattern Recognition, 95: 12-23[DOI: 10.1016/j.patcog.2019.05.032]
Carlsen J F, Ewertsen C, Lönn L and Nielsen M B. 2013. Strain elastography ultrasound: an overview with emphasis on breast cancer diagnosis. Diagnostics, 3(1): 117-125[DOI: 10.3390/diagnostics3010117]
Chen L, Bentley P, Mori K, Misawa K, Fujiwara M and Rueckert D. 2019. Self-supervised learning for medical image analysis using image context restoration. Medical Image Analysis, 58: #101539[DOI: 10.1016/j.media.2019.101539]
Du Z J, Gong X, Luo J, Zhang Z M and Yang F. 2020. Classification method for samples that are easy to be confused in breast ultrasound images. Journal of Image and Graphics, 25(7): 1490-1500
杜章锦, 龚勋, 罗俊, 章哲敏, 杨菲. 2020. 乳腺超声图像中易混淆困难样本的分类方法. 中国图象图形学报, 25(7): 1490-1500[DOI: 10.11834/jig.190442]
Duan L X, Xu Y W, Li W, Chen L, Wong D W K, Wong T Y and Liu J. 2014. Incorporating privileged genetic information for fundus image based glaucoma detection//Proceedings of the 17th International Conference on Medical Image Computing and Computer-Assisted Intervention. Boston, USA: Springer: 204-211[ DOI: 10.1007/978-3-319-10470-6_26 http://dx.doi.org/10.1007/978-3-319-10470-6_26 ]
He K M, Zhang X Y, Ren S Q and Sun J. 2015. Delving deep into rectifiers: surpassing human-level performance on ImageNet classification//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE: 1026-1034[ DOI: 10.1109/ICCV.2015.123 http://dx.doi.org/10.1109/ICCV.2015.123 ]
He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 770-778[ DOI: 10.1109/CVPR.2016.90 http://dx.doi.org/10.1109/CVPR.2016.90 ]
Jing L L and Tian Y L. 2021. Self-supervised visual feature learning with deep neural networks: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(11): 4037-4058[DOI: 10.1109/TPAMI.2020.2992393]
Li X, Du B, Xu C, Zhang Y P, Zhang L F and Tao D C. 2018. R-SVM+: robust learning with privileged information//Proceedings of the 27th International Joint Conference on Artificial Intelligence. Stockholm, Sweden: IJCAI: 2411-2417[ DOI: 10.24963/ijcai.2018/334 http://dx.doi.org/10.24963/ijcai.2018/334 ]
Liu X, Zhang F J, Hou Z Y, Mian L, Wang Z Y, Zhang J and Tang J. 2020. Self-supervised learning: generative or contrastive[EB/OL]. [2021-07-23] . https://arxiv.org/pdf/2006.08218.pdf https://arxiv.org/pdf/2006.08218.pdf
Long M S, Cao Y, Wang J M and Jordan M I. 2015. Learning transferable features with deep adaptation networks//Proceedings of the 32nd International Conference on Machine Learning. Lille, France: JMLR. org: 97-105
Mahmood T, Li J Q, Pei Y, Akhtar F, Imran A and Rehman K U. 2020. A brief survey on breast cancer diagnostic with deep learning schemes using multi-image modalities. IEEE Access, 8: 165779-165809[DOI: 10.1109/ACCESS.2020.3021343]
Pechyony D and Vapnik V. 2012. Fast optimization algorithms for solving SVM+//Statistical Learning and Data Science. London: Chapman and Hall/CRC: 27-42[ DOI: 10.1201/b11429-7 http://dx.doi.org/10.1201/b11429-7 ]
Sahiner B, Chan H P and Hadjiiski L. 2008. Classifier performance estimation under the constraint of a finite sample size: resampling schemes applied to neural network classifiers. Neural Networks, 21(2/3): 476-483[DOI: 10.1016/j.neunet.2007.12.012]
Shi X Q, Li J, Qian L, Xue X, Li J and Wan W. 2018. Correlation between elastic parameters and collagen fibre features in breast lesions. Clinical Radiology, 73(6): 595. e1-595. e7[DOI: 10.1016/j.crad.2018.01.019]
Siegel R L, Miller K D and Jemal A. 2020. Cancer statistics, 2020. CA: A Cancer Journal for Clinicians, 70(1): 7-30[DOI: 10.3322/caac.21590]
Sigrist R M S, Liau J, Kaffas A E, Chammas M C and Willmann J K. 2017. Ultrasound elastography: review of techniques and clinical applications. Theranostics, 7(5): 1303-1329[DOI: 10.7150/thno.18650]
Sun B C, Feng J S and Saenko K. 2016. Return of frustratingly easy domain adaptation//Proceedings of the 30th AAAI Conference on Artificial Intelligence. Phoenix, Arizona, USA: AAAI Press: 2058-2065
Sun B C and Saenko K. 2016. Deep CORAL: correlation alignment for deep domain adaptation//Proceedings of European Conference on Computer Vision. Amsterdam, the Netherlands: Springer: 443-450[ DOI: 10.1007/978-3-319-49409-8_35 http://dx.doi.org/10.1007/978-3-319-49409-8_35 ]
Sutton G C. 1989. Computer-aided diagnosis: a review. British Journal of Surgery, 76(1): 82-85[DOI: 10.1002/bjs.1800760126]
Tzeng E, Hoffman J, Zhang N, Saenko K and Darrell T. 2014. Deep domain confusion: maximizing for domain invariance[EB/OL]. [2021-07-23] . https://arxiv.org/pdf/1412.3474.pdf https://arxiv.org/pdf/1412.3474.pdf
Vapnik V and Vashist A. 2009. A new learning paradigm: learning using privileged information. Neural Networks, 22(5/6): 544-557[DOI: 10.1016/j.neunet.2009.06.042]
Wang Y, Chen H D, Li N, Ren J S, Zhang K, Dai M and He J. 2019. Ultrasound for breast cancer screening in high-risk women: results from a population-based cancer screening program in China. Frontiers in Oncology, 9: #286[DOI: 10.3389/fonc.2019.00286]
Weiss K, Khoshgoftaar T M and Wang D D. 2016. A survey of transfer learning. Journal ofBig Data, 3(1): #9[DOI: 10.1186/s40537-016-0043-6]
Yu C H, Wang J D, Chen Y Q and Huang M Y. 2019. Transfer learning with dynamic adversarial adaptation network//Proceedings of 2019 IEEE International Conference on Data Mining. Beijing, China: IEEE: 778-786[ DOI: 10.1109/ICDM.2019.00088 http://dx.doi.org/10.1109/ICDM.2019.00088 ]
Yu Y H, Lin H F, Meng J N, Wei X C, Guo H and Zhao Z H. 2017. Deep transfer learning for modality classification of medical images. Information, 8(3): #91[DOI: 10.3390/info8030091]
Zhang H L, Guo L H, Wang D, Wang J, Bao L L, Ying S H, Xu H X and Shi J. 2021. Multi-source transfer learning via multi-kernel support vector machine plus for B-mode ultrasound-based computer-aided diagnosis of liver cancers. IEEE Journal of Biomedical and Health Informatics, 25(10): 3874-3885[DOI: 10.1109/JBHI.2021.3073812]
Zhu Y C, Zhuang F Z, Wang J D, Ke G L, Chen J W, Bian J, Xiong H and He Q. 2021. Deep subdomain adaptation network for image classification. IEEE Transactions on Neural Networks and Learning Systems, 32(4): 1713-1722[DOI: 10.1109/TNNLS.2020.2988928]
相关文章
相关作者
相关机构
京公网安备11010802024621
