浏览全部资源
扫码关注微信
Published： 16 April 2023 ，
移动端阅览
赵樱莉， 丁维龙， 游庆华， 朱峰龙， 朱筱婕， 郑魁， 刘丹丹. 2023. 融合空间相关性特征的乳腺组织病理全切片分类. 中国图象图形学报， 28(04):1134-1145
Zhao Yingli， Ding Weilong， You Qinghua， Zhu Fenglong， Zhu Xiaojie， Zheng Kui， Liu Dandan. 2023. Classification of whole slide images of breast histopathology based on spatial correlation characteristics. Journal of Image and Graphics， 28(04):1134-1145
目的
2
病理学检查是明确乳腺癌诊断及肿瘤类型的金标准。深度神经网络广泛应用于乳腺病理全切片的诊断工作并取得了明显进展，但是现有大多数工作只是将全切片切割成小图像块，对每个图像块进行单独处理，没有考虑它们之间的空间信息。为此，提出了一种融合空间相关性特征的乳腺组织病理全切片分类方法。
方法
2
首先基于卷积神经网络对病理图像块进行预测，并提取每个图像块有代表性的深层特征，然后利用特征融合将图像块及其周围图像的特征进行聚合，以形成具有空间相关性的块描述符，最后将全切片图像中最可疑的块描述符传递给循环神经网络，以预测最终的全切片级别的分类。
结果
2
本文构建了一个经过详细标注的乳腺病理全切片数据集，并在此数据集上进行良性/恶性二分类实验。在自建的数据集中与3种全切片分类方法进行了比较。结果表明，本文方法的分类精度达到96.3%，比未考虑空间相关性的方法高出了1.9%，与基于热力图特征和基于空间性和随机森林的方法相比，分类精度分别高出8.8%和1.3%。
结论
2
本文提出的乳腺组织病理全切片识别方法将空间相关性特征融合和RNN分类集成到一个统一模型，有助于提高图像识别准确率，为病理图像诊断工作提供了高效的辅助诊断工具。
Objective
2
Pathological examination can be as the “gold standard” to interpret breast cancer diagnosis and its tumor types. To improve the treatment effect of breast cancer， accurate interpretation is beneficial to clarify the type of the disease early and effectively. However， it is time-consuming to check the pathological whole slide images （WSIs） manually， and the diagnosis result is easily affected by personal experience. Due to computer vision-based convolutional neural networks （CNNs） are applied to the classification task of histopathological images computer-aided diagnostic （CAD） techniques for digital images of histopathology has been developing intensively. However， it is still challenged to split large-sized WSIs into small patches and each patch is processed individually without the spatial information between them. Also， the information-involved of the patches learned is not utilized in the small size patch feature aggregation process. To resolve these problems， we develop a spatial correlation-based classification method for breast histopathology images， which can re-identify the WSIs classification problem through deep learning feature fusion and recurrent neural network （RNN） based classification.
Method
2
Our demonstration consists of four aspects： 1） pre-processing operation of WSIs， 2） CNN-based breast patch prediction， 3） image patch feature fusion， and 4） RNN-based WSIs classification. First， it is focused on a preprocessing operation on whole slide images of breast pathology， using a sliding window to cut the WSIs into patches of a suitable size in terms of a CNN. Second， CNN-based patch prediction is used to predict the possibility of cancer in each patch and each patch is encoded as a fixed-length feature. To reduce unpredictable class of cancerous patch being classified as non-cancerous patch， the ResNet34 is used as the patch classification model and penalty factors is added in Focal loss. Third， the feature fusion of image blocks can be recognized as a feature fusion niche that takes the entire grid of patches as input and the spatial correlation of patches and their surrounding patches are as feature fusion of all patches within the grid， thus block descriptors are formed. Feature fusion methods are involved in， including 1） Weight， 2） Max， 3） Avg， 4） Norm3， and 5） WeightNorm3. Finally， the RNN-based WSIs classification is used to pass the block feature descriptors with high probability of cancer in each WSIs to the RNN， and the RNN model is used to learn the sequence relationship between the image block feature sequences. It can expand the processing field of view of the classification model to multiple Blocks and optimize the classification accuracy of breast pathology WSIs. Additionally， we design and develop an online recognition system for assistance.
Result
2
A detailed annotated whole slide images dataset of breast pathology is constructed， and benign/malignant dichotomous classification experiments are carried out on this dataset. The results are compared to three WSIs classification methods in the self-constructed dataset， and it shows that the classification accuracy of the method can be reached to 96.3%， which is 1.9% higher than non-spatial correlation method. The classification accuracy of each is 8.8% and 1.3% higher compared to heat map features-based methods and spatial correlation and random forest-related methods.
Conclusion
2
The proposed breast pathology recognition method can improve image recognition accuracy and provide an efficient diagnostic aid for pathology image diagnosis work， which integrates feature fusion and RNN classification into a synthesized model.
乳腺癌病理组织全切片分类卷积神经网络（CNN）特征融合循环神经网络（RNN）
breast cancerpathological whole slide imagesclassificationconvolutional neural network（CNN）feature fusionrecurrent neural network（RNN）
Araújo T， Aresta G， Castro E， Rouco J， Aguiar P， Eloy C， Polónia A and Campilho A. 2017. Classification of breast cancer histology images using convolutional neural networks. PLoS One， 12（6）： #e0177544 ［DOI： 10.1371/journal.pone.0177544http://dx.doi.org/10.1371/journal.pone.0177544］
Bian X W and Ping Y F. 2019. Pathology in China： challenges and opportunities. Journal of Third Military Medical University， 41（19）： 1815-1817
卞修武， 平轶芳. 2019. 我国病理学科发展面临的挑战和机遇. 陆军军医大学学报， 41（19）： 1815-1817 ［DOI： 10.16016/j.1000-5404.201909212http://dx.doi.org/10.16016/j.1000-5404.201909212］
Campanella G， Hanna M G， Geneslaw L， Miraflor A， Silva V W K， Busam K J， Brogi E， Reuter V E， Klimstra D S and Fuchs T J. 2019. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature Medicine， 25（8）： 1301-1309 ［DOI： 10.1038/s41591-019-0508-1http://dx.doi.org/10.1038/s41591-019-0508-1］
Cruz-Roa A， Basavanhally A， González F， Gilmore H， Feldman M， Ganesan S， Shih N， Tomaszewski J and Madabhushi A. 2014. Automatic detection of invasive ductal carcinoma in whole slide images with convolutional neural networks//Proceedings of SPIE 9041， Medical Imaging 2014： Digital Pathology. San Diego， USA： SPIE： #904103 ［DOI： 10.1117/12.2043872http://dx.doi.org/10.1117/12.2043872］
He K M， Zhang X Y， Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 770-778 ［DOI： 10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90］
Hinton G. 1986. Learning distributed representations of concepts//Proceedings of the 8th Annual Conference of the Cognitive Science Society. Hillsdale， Canada： Lawrence Erlbaum Associates： 2999-3007
Hochreiter S and Schmidhuber J. 1997. Long short-term memory. Neural Computation， 9（8）： 1735-1780 ［DOI： 10.1162/neco.1997.9.8.1735http://dx.doi.org/10.1162/neco.1997.9.8.1735］
Hou L， Samaras D， Kurc T M， Gao Y， Davis J E and Saltz J H. 2016. Patch-based convolutional neural network for whole slide tissue image classification//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 2424-2433 ［DOI： 10.1109/CVPR.2016.266http://dx.doi.org/10.1109/CVPR.2016.266］
Jin X， Wen K， Lyu G F， Shi J， Chi M X， Wu Z and An H. 2020. Survey on the applications of deep learning to histopathology. Journal of Image and Graphics， 25（10）： 1982-1993
金旭， 文可， 吕国锋， 石军， 迟孟贤， 武铮， 安虹. 2020. 深度学习在组织病理学中的应用综述. 中国图象图形学报， 25（10）： 1982-1993 ［DOI： 10.11834/jig.200460http://dx.doi.org/10.11834/jig.200460］
Kong B， Wang X， Li Z Y， Song Q and Zhang S T. 2017. Cancer metastasis detection via spatially structured deep network//Proceedings of the 25th International Conference on Information Processing in Medical Imaging. Boone， USA： Springer： 236-248 ［DOI： 10.1007/978-3-319-59050-9_19http://dx.doi.org/10.1007/978-3-319-59050-9_19］
Li H， Yang J N， Liu F， Nan F Z and Qian Y R. 2020. Survey of breast cancer histopathology image classification based on deep learning. Computer Engineering and Applications， 56（13）： 1-11
李华， 杨嘉能， 刘凤， 南方哲， 钱育蓉. 2020. 基于深度学习的乳腺癌病理图像分类研究综述. 计算机工程与应用， 56（13）： 1-11 ［DOI： 10.3778/j.issn.1002-8331.2001-0220http://dx.doi.org/10.3778/j.issn.1002-8331.2001-0220］
Li Y and Ping W. 2018. Cancer metastasis detection with neural conditional random field ［EB/OL］. ［2021-11-19］. https://arxiv.org/pdf/1806.07064.pdfhttps://arxiv.org/pdf/1806.07064.pdf
Liang Z Y， Wang Q Y， Liao H W， Zhao M， Lee J Y， Yang C， Li F Y and Ling D S. 2021. Artificially engineered antiferromagnetic nanoprobes for ultra-sensitive histopathological level magnetic resonance imaging. Nature Communications， 12（1）： #3840 ［DOI： 10.1038/s41467-021-24055-2http://dx.doi.org/10.1038/s41467-021-24055-2］
Lin T Y， Goyal P， Girshick R， He K M and Dollár P. 2017. Focal loss for dense object detection//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice， Italy： IEEE： 2999-3007 ［DOI： 10.1109/ICCV.2017.324http://dx.doi.org/10.1109/ICCV.2017.324］
Liu Y， Gadepalli K， Norouzi M， Dahl G E， Kohlberger T， Boyko A， Venugopalan S， Timofeev A， Nelson P Q， Corrado G S， Hipp J D， Peng L and Stumpe M C. 2017. Detecting cancer metastases on gigapixel pathology images ［EB/OL］. ［2021-11-19］. http：//arxiv.org/pdf/1703.02442.pdfhttp://arxiv.org/pdf/1703.02442.pdf
Otsu N. 1979. A threshold selection method from gray-level histograms. IEEE Transactions on Systems， Man， and Cybernetics， 9（1）： 62-66 ［DOI： 10.1109/TSMC.1979.4310076http://dx.doi.org/10.1109/TSMC.1979.4310076］
Reinhard E， Adhikhmin M， Gooch B and Shirley P. 2001. Color transfer between images. IEEE Computer Graphics and Applications， 21（5）： 34-41 ［DOI： 10.1109/38.946629http://dx.doi.org/10.1109/38.946629］
Wang D Y， Khosla A， Gargeya R， Irshad H and Beck A H. 2016. Deep learning for identifying metastatic breast cancer ［EB/OL］. ［2021-11-19］. https://arxiv.org/pdf/1606.05718.pdfhttps://arxiv.org/pdf/1606.05718.pdf
Wang X， Chen H， Gan C X， Lin H J， Dou Q， Tsougenis E， Huang Q T， Cai M Y and Heng P A. 2020. Weakly supervised deep learning for whole slide lung cancer image analysis. IEEE Transactions on Cybernetics， 50（9）： 3950-3962 ［DOI： 10.1109/TCYB.2019.2935141http://dx.doi.org/10.1109/TCYB.2019.2935141］
Xiao H Y and Hao X S. 2021. Research and application of key technologies for early diagnosis and treatment of breast cancer. Science and Technology Review， 39（7）： 75-78
肖怀远， 郝希山. 2021. 乳腺癌早诊早治关键技术的研究与应用. 科技导报， 39（7）： 75-78 ［DOI： 10.3981/j.issn.1000-7857.2021.07.007http://dx.doi.org/10.3981/j.issn.1000-7857.2021.07.007］
Xu Y， Jia Z P， Ai Y Q， Zhang F， Lai M D and Chang E I C. 2015. Deep convolutional activation features for large scale brain tumor histopathology image classification and segmentation//Proceedings of 2015 IEEE International Conference on Acoustics， Speech and Signal Processing. South Brisbane， Australia： IEEE： 947-951 ［DOI： 10.1109/ICASSP.2015.7178109http://dx.doi.org/10.1109/ICASSP.2015.7178109］
Xu Y， Mo T， Feng Q W， Zhong P L， Lai M D and Chang I E C. 2014. Deep learning of feature representation with multiple instance learning for medical image analysis//Proceedings of 2014 IEEE International Conference on Acoustics， Speech and Signal Processing. Florence， Italy： IEEE： 1626-1630 ［DOI： 10.1109/ICASSP.2014.6853873http://dx.doi.org/10.1109/ICASSP.2014.6853873］
Zheng Y， Wu C X and Zhang M L. 2013. The epidemic and characteristics of female breast cancer in China. China Oncology， 23（8）： 561-569
郑莹， 吴春晓， 张敏璐. 2013. 乳腺癌在中国的流行状况和疾病特征. 中国癌症杂志， 23（8）： 561-569 ［DOI： 10.3969/j.issn.1007-3969.2013.08.001http://dx.doi.org/10.3969/j.issn.1007-3969.2013.08.001］
Zheng Y S， Jiang Z G， Zhang H P， Xie F Y， Ma Y B， Shi H Q and Zhao Y. 2018. Histopathological whole slide image analysis using context-based CBIR. IEEE Transactions on Medical Imaging， 37（7）： 1641-1652 ［DOI： 10.1109/TMI.2018.2796130http://dx.doi.org/10.1109/TMI.2018.2796130］
相关作者
相关机构
京公网安备11010802024621