浏览全部资源
扫码关注微信
纸质出版日期： 2023-02-16 ，
录用日期： 2022-01-26
移动端阅览
管安娜, 刘骊, 付晓东, 刘利军, 黄青松. 面向大规模胸片图像的深度哈希检索[J]. 中国图象图形学报, 2023,28(2):601-612.
Anna Guan, Li Liu, Xiaodong Fu, Lijun Liu, Qingsong Huang. A deep hash retrieval for large-scale chest radiography images[J]. Journal of Image and Graphics, 2023,28(2):601-612.
目的
2
医学图像检索在疾病诊断、医疗教学和辅助症状参考中发挥了重要作用，但由于医学图像类间相似度高、病灶易遗漏以及数据量较大等问题，使得现有哈希方法对病灶区域特征的关注较少，图像检索准确率较低。对此，本文以胸部X-ray图像为例，提出一种面向大规模胸片图像的深度哈希检索网络。
方法
2
在特征学习部分，首先采用ResNet-50作为主干网络对输入图像进行特征提取得到初步特征，将该特征进行细化后获得全局特征；同时将初步特征输入构建的空间注意模块，该注意模块结合了3个描述符用于聚焦胸片图像中的显著区域，将该模块的输出进行细化得到局部特征；最后融合全局特征与局部特征用于后续哈希码优化。在哈希码优化部分，使用定义的二值交叉熵损失、对比损失和正则化损失的联合函数进行优化学习，生成高质量的哈希码用于图像检索。
结果
2
为了验证方法的有效性，在公开的ChestX-ray8和CheXpert数据集上进行对比实验。结果显示，构建空间注意模块有助于关注病灶区域，定义特征融合模块有效避免了信息的遗漏，联合3个损失函数进行优化可以获得高质量哈希码。与当前先进的医学图像检索方法比较，本文方法能够有效提高医学图像检索的准确率，在两个数据集上的检索平均精度分别提高了约6%和5%。
结论
2
在大规模胸片图像检索中，本文提出的深度哈希检索方法能够有效关注病灶区域，提高胸片图像检索的准确率。
Objective
2
Big medical data is mainly concerned of data storage-related like electronic healthy profiles
medical image and genetic information. It is essential to process large-scale medical image data efficiently. For large-scale retrieval tasks
deep hashing methods can be used to optimize traditional retrieval methods. To improve its retrieval efficiency
the potential ability is developed to map the high-dimensional features of an image into the binary space
generate low-dimensional binary encoded features
and avoid dimensional catastrophe problem. Hash-depth methods are divided into two data categories of those are independent data and dependent data. Although the deep hashing method has great advantages in large-scale image retrieval
the challenges are still to be resolved for the features loss issues of key areas of the lesions like redundant lesions
high noise
and small targets. So
we develop a deep hash retrieval network for large-scale human chest-related X-ray images.
Method
2
For the feature learning part: to obtain their initial features
the ResNet-50 is used as the backbone network and the input image is subjected to feature extraction. To obtain global features
a feature-refined block is followed. Here
the feature-refined block is structured via the residual block and the average pooling layer. To obtain the detailed focal regions
we design a spatial attention module in related to three descriptors: 1) maximum element along the channel axis
2) average element
and 3) maximum pooling. In addition
to obtain a feature focusing on the prominent region
the key features are input into the spatial attention module
and then local features are obtained in terms of feature-refined block. First
the resulting global and local features are integrated seamlessly by dimension. Next
to optimize hash codes
the cascade layer is connected to the fully-connected layer. For the part of hash code optimization: in order to obtain high quality hash codes and improve the quality of sorting results
a joint loss function is used to define the target error. To generate a more discriminative hash code
we leverage the label information and semantic features of the image in related to the losses of contrast
regularization and cross entropy. Finally
the searching results are calculated in terms of the similarity metric.
Result
2
The comparative experiments are carried out on two different datasets of those are ChestX-ray8 and CheXpert. Our analysis is compared to other five classical generic hashing methods for the same task
including deep hashing methods and shallow hashing methods. Among them
the deep hashing methods are based on deep hashing (DH)
deep supervised hashing (DSH)
and attention-based triplet hashing (ATH)
and the shallow hashing methods are based on semi-supervised hashing (SSH) and iterative quantization (ITQ). The normalized discounted cumulative gain (nDCG@100) and mean average precision (mAP) are used as evaluation metrics. The experimental results show that the retrieval performance of our method has some optimal value in comparison with deep learning-relevant methods. For the ChestX-ray8 dataset
the mAP is increased by about 6% and the nDCG@100 is improved by 4%. For the CheXpert database
the mAP is higher by about 5% and the nDCG@100 is improved by 3%.
Conclusion
2
To deal with the problem that the existing hashing methods pay less attention to salient region features
we demonstrate a deep hash retrieval network for large-scale chest X-ray images for large-scale human chest-relevant radiographic image retrieval. To improve the accuracy and ranking quality of image retrieval
this deep hash retrieval method is proposed and be focused on the lesion region effectively. It is beneficial to clarify focal area information and reveal the attention-less problem to salient areas in terms of a spatial attention module-constructed. The feature fusion-defined module can be used resolve the problem of information loss effectively. We use three loss functions to make the real value output more similar to the binary hash code
which can optimize the sorting quality problem of the retrieval results. It is possible to adjust the order of network composition for the regions of interest (RoI)-concerned. The loss function can be optimized the existing hashing method. It is potential to distinguish small sample images further.
医学图像检索注意力机制特征融合深度哈希(DH)
medical image retrievalattention mechanismfeature fusiondeep hashing (DH)
Ahmad J, Muhammad K and Baik S W. 2018. Medical image retrieval with compact binary codes generated in frequency domain using highly reactive convolutional features. Journal of Medical Systems, 42(2): #24[DOI: 10.1007/s10916-017-0875-4]
Chen Z X, Cai R J, Lu J W, Feng J J and Zhou J. 2018. Order-sensitive deep hashing for multimorbidity medical image retrieval//Proceedings of the 21st International Conference on Medical Image Computing and Computer Assisted Intervention. Granada, Spain: Springer: 620-628[DOI: 10.1007/978-3-030-00928-1_70http://dx.doi.org/10.1007/978-3-030-00928-1_70]
Conjeti S, Roy A G, Katouzian A and Navab N. 2017. Hashing with residual networks for image retrieval//Proceedings of the 20th International Conference on Medical Image Computing and Computer Assisted Intervention. Quebec City, Canada: Springer: 541-549[DOI: 10.1007/978-3-319-66179-7_62http://dx.doi.org/10.1007/978-3-319-66179-7_62]
Fang J S, Fu H Z and Liu J. 2021. Deep triplet hashing network for case-based medical image retrieval. Medical Image Analysis, 69: #101981[DOI: 10.1016/j.media.2021.101981]
Gong Y C, Lazebnik S, Gordo A and Perronnin F. 2013. Iterative quantization: a procrustean approach to learning binary codes for large-scale image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(12): 2916-2929[DOI: 10.1109/TPAMI.2012.193]
Guan Q J, Huang Y P, Zhong Z, Zheng Z D, Zheng L and Yang Y. 2020. Thorax disease classification with attention guided convolutional neural network. Pattern Recognition Letters, 131: 38-45[DOI: 10.1016/j.patrec.2019.11.040]
Haq N F, Moradi M and Wang Z J. 2020. A deep community based approach for large scale content based X-ray image retrieval. Medical Image Analysis, 68(2): #101847[DOI: 10.1016/j.media.2020.101847]
Hwang K H, Lee H and Choi D. 2012. Medical image retrieval: past and present. Healthcare Informatics Research, 18(1): 3-9[DOI: 10.4258/hir.2012.18.1.3]
Indyk P and Motwani R. 1998. Approximate nearest neighbors: towards removing the curse of dimensionality//Proceedings of the 30th Annual ACM Symposium on Theory of Computing. Dallas, USA: Association for Computing Machinery: 604-613[DOI: 10.1145/276698.276876http://dx.doi.org/10.1145/276698.276876]
Irvin J, Rajpurkar P, Ko M, Yu Y F, Ciurea-Ilcus S, Chute C, Marklund H, Haghgoo B, Ball R, Shpanskaya K, Seekins J, Mong D A, Halabi S S, Sandberg J K, Jones R, Larson D B, Langlotz C P, Patel B N, Lungren M P and Ng A Y. 2019. CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison. Proceedings of the AAAI Conference on Artificial Intelligence, 33(1): 590-597[DOI: 10.1609/aaai.v33i01.3301590]
Jiang M L, Zhang S T, Li H S and Metaxas D N. 2015. Computer-aided diagnosis of mammographic masses using scalable image retrieval. IEEE Transactions on Biomedical Engineering, 62(2): 783-792[DOI: 10.1109/tbme.2014.2365494]
Kashif M, Raja G and Shaukat F. 2020. An efficient content-based image retrieval system for the diagnosis of lung diseases. Journal of Digital Imaging, 33(4): 971-987[DOI: 10.1007/s10278-020-00338-w]
Kong W H and Li W J. 2012. Isotropic hashing//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, USA: Curran Associates Inc. : 1646-1654
Li L, Xu M, Wang X F, Jiang L and Liu H R. 2019. Attention based glaucoma detection: a large-scale database and CNN model//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, USA: IEEE: 10563-10572[DOI: 10.1109/CVPR.2019.01082http://dx.doi.org/10.1109/CVPR.2019.01082]
Li Q, Sun Z N, He R and Tan T N. 2017. Deep supervised discrete hashing//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA: Curran Associates Inc. : 2479-2488
Li Z Y, Zhang X F, Müller H and Zhang S T. 2018. Large-scale retrieval for medical image analytics: a comprehensive review. Medical Image Analysis, 43: 66-84[DOI: 10.1016/j.media.2017.09.007]
Liong V E, Lu J W, Wang G, Moulin P and Zhou J. 2015. Deep hashing for compact binary codes learning//Proceedings of 2015 Conference on Computer Vision and Pattern Recognition (CVPR). Boston, USA: IEEE: 2475-2483[DOI: 0.1109/CVPR.2015.7298862http://dx.doi.org/0.1109/CVPR.2015.7298862]
Liu H M, Wang R P, Shan S G and Chen X L. 2016. Deep supervised hashing for fast image retrieval//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE: 2064-2072[DOI: 10.1109/CVPR.2016.227http://dx.doi.org/10.1109/CVPR.2016.227]
Liu J J, Zhang S T, Liu W, Zhang X F and Metaxas D N. 2014. Scalable mammogram retrieval using Anchor Graph Hashing//Proceedings of the 11th IEEE International Symposium on Biomedical Imaging (ISBI). Beijing, China: IEEE: 898-901[DOI: 10.1109/ISBI.2014.6868016http://dx.doi.org/10.1109/ISBI.2014.6868016]
Liu Y, Cheng M, Wang F P, Li D X, Liu W and Fan J L. 2020. Deep Hashing image retrieval methods. Chinese Journal of Image and Graphics, 25(7): 1296-1317
刘颖, 程美, 王富平, 李大湘, 刘伟, 范九伦. 2020. 深度哈希图像检索方法综述. 中国图形图象学报, 25(7): 1296-1317[DOI: 10.11834/jig.190518]
Nie D, Gao Y Z, Wang L and Shen D G. 2018. ASDNet: attention based semi-supervised deep networks for medical image segmentation//Proceedings of the 21st International Conference on Medical Image Computing and Computer Assisted Intervention. Granada, Spain: Springer: 370-378[DOI: 10.1007/978-3-030-00937-3_43http://dx.doi.org/10.1007/978-3-030-00937-3_43]
Qin P L, Li Q, Zeng J C, Liu H Y and Cui Y H. 2019. Fully convolutional-based dense network for lung nodule image retrieval algorithm. International Journal of Performability Engineering, 15(1): 326-336[DOI: 10.23940/ijpe.19.01.p33.326336]
Renita D B and Christopher C S. 2020. Novel real time content based medical image retrieval scheme with GWO-SVM. Multimedia Tools and Applications, 79(23): 17227-17243[DOI: 10.1007/s11042-019-07777-w]
Shi X S, Sapkota M, Xing F Y, Liu F J, Cui L and Yang L. 2018. Pairwise based deep ranking hashing for histopathology image classification and retrieval. Pattern Recognition, 81: 14-22[DOI: 10.1016/j.patcog.2018.03.015]
Silva W, Poellinger A, Cardoso J S and Reyes M. 2020. Interpretability-guided content-based medical image retrieval//Proceedings of the 23rd International Conference on Medical Image Computing and Computer Assisted Intervention. Lima, Peru: Springer: 305-314[DOI: 10.1007/978-3-030-59710-8_30http://dx.doi.org/10.1007/978-3-030-59710-8_30]
Wang J, Kumar S and Chang S F. 2012. Semi-supervised hashing for large-scale search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(12): 2393-2406[DOI: 10.1109/TPAMI.2012.48]
Wang X S, Peng Y F, Lu L, Lu Z Y, Bagheri M and Summers R M. 2017. ChestX-Ray8: hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE: 3462-3471[DOI: 10.1109/cvpr.2017.369http://dx.doi.org/10.1109/cvpr.2017.369]
Woo S, Park J, Lee J Y and Kweon I S. 2018. CBAM: convolutional block attention module//Proceedings of the 15th European Conference on Computer Vision. Munich, Germany: Springer: 3-19[DOI: 10.1007/978-3-030-01234-2_1http://dx.doi.org/10.1007/978-3-030-01234-2_1]
Wu M L, Chen Q and SunQ S. 2013. Medical image retrieval by graph-based semi-supervised learning. Journal of Computer-Aided Design and Computer Graphics, 25(9): 1354-1360
吴梦麟, 陈强, 孙权森. 2013. 基于图半监督学习的医学图像检索. 计算机辅助设计与图形学学报, 25(9): 1354-1360[DOI:10.3969/j.issn.1003-9775.2013.09.012]
相关作者
相关机构
京公网安备11010802024621