浏览全部资源
扫码关注微信
纸质出版日期： 2020-10-16 ，
录用日期： 2020-08-27
移动端阅览
金旭, 文可, 吕国锋, 石军, 迟孟贤, 武铮, 安虹. 深度学习在组织病理学中的应用综述[J]. 中国图象图形学报, 2020,25(10):1982-1993.
Xu Jin, Ke Wen, Guofeng Lyu, Jun Shi, Mengxian Chi, Zheng Wu, Hong An. Survey on the applications of deep learning to histopathology[J]. Journal of Image and Graphics, 2020,25(10):1982-1993.
组织病理学是临床上肿瘤诊断的金标准，直接关系到治疗的开展与预后的评估。来自临床的需求为组织病理诊断提出了质量与效率两个方面的挑战。组织病理诊断涉及大量繁重的病理切片判读任务，高度依赖医生的经验，但病理医生的培养周期长，人才储备缺口巨大，病理科室普遍超负荷工作。近年来出现的基于深度学习的组织病理辅助诊断方法可以帮助医生提高诊断工作的精度与速度，缓解病理诊断资源不足的问题，引起了研究人员的广泛关注。本文初步综述深度学习方法在组织病理学中的相关研究工作。介绍了组织病理诊断的医学背景，整理了组织病理学领域的主要数据集，重点介绍倍受关注的乳腺癌、淋巴结转移癌、结肠癌的病理数据及其分析任务。本文归纳了数据的存储与处理、模型的设计与优化以及小样本与弱标注学习这3项需要解决的技术问题。围绕这些问题，本文介绍了包括数据存储、数据预处理、分类模型、分割模型、迁移学习和多示例学习等相关研究工作。最后总结了面向组织病理学诊断的深度学习方法研究现状，并指出当下研究工作可能的改进方向。
Histopathology is the gold standard for the clinical diagnosis of tumors and directly related to clinical treatment and prognosis. Its application inclinics has presented challenges in terms of the accuracy and efficiency of histopathological diagnosis
and pathological diagnosis is time consuming and requires pathologists to examine slides with a microscope for them to make reliable decisions. Moreover
the training period of a pathologist is long. In many parts of China
pathology departments are generally overworked due to the insufficient number of pathologists. Recently
deep learning has achieved great success in computer vision. The utilization of whole slide scanners enables the application of deep learning-based classification and segmentation methods to histopathological diagnosis
thereby improving efficiency and accuracy. In this paper
we first introduce the medical background of histopathology diagnosis. Then
we provide an overview of the primary datasets of histopathological diagnosis. We focus on introducing the datasets of three types of malignant tumors along with corresponding computer vision tasks. Breast cancer forms in the cells of breasts and is one of the most common cancer diagnosed in women. Early diagnosis can significantly improve survival rate and quality of life. Sentinel lymph node metastasis are visible when a cancer spreads. The diagnosis of lymph node metastasis is directly related to cancer staging and surgical plan decision. Colon cancer can be detected by colonoscopy biopsy
and early diagnosis requires pathologists to examine slides thoroughly for small malignancies.Computer-aided diagnosis can increase the efficiency of pathologists. Moreover
we propose three key technical problems: data storage and processing
model design and improvement
and learning with small amount or weakly labeled data. Then we review research progress related to tasks
including data preprocessing
classification
and segmentation and transfer learning and multiple instance learning. Pathology datasets are usually stored in a pyramidal tiled image format for fast loading and rescaling. The OpenSlide library provides high-performance pathology data reading
and the open-source softwareautomated slide analysis platform (ASAP) can be used in viewing and labeling these data. Trimming white backgrounds can reduce storage and calculation overhead to 82% on mainstream datasets. A stain normalization technology can eliminate color difference caused by slide production and scanning process. The classification of pathological image patches is the basic structure of whole slide classification and the backbone network of segmentation. Mainstream convolutional neural network models in the field of computer vision
including AlexNet
visual geometry group(VGG)
GoogLeNet
and residual neural networks can reach satisfying results for pathological image patches. The patch sampling method can divide a whole slide image into smaller patches that can be processed by the mainstream convolutional neural network models. By aggregating the features of sampled patches through random forest or voting
a patch sampling method can be used in classifying or segmenting arbitrarily sized images. A migration learning technology based on neural network models pretrained on an ImageNet dataset is effective in alleviating the problem introduced by the small number of training samples in histopathological data. Fully convolutional network (FCN) represented by U-Net is a network designed for medical image segmentation tasks and are faster than convolutional neural networks with patch sampling methods. To utilize weakly labeled data
multiple instance learning (MIL) treats whole slide image as a bag of unlabeled pathological image patches. With bags labeled
MIL can be used for weakly supervised learning. Finally
this paper summarizes the main works surveyed and identifies challenges for future research. To make deep learning-based computer aided diagnosis clinically practical
researchers have to improve model accuracy
expand clinical application scenarios
and improve the interpretability of results.
组织病理学深度学习卷积神经网络迁移学习多示例学习
histopathologydeep learningconvolutional neural networktransfer learningmultiple instance learning
Altunbay D, Cigir C, Sokmensuer C and Gunduz-Demir C. 2010. Color graphs for automated cancer diagnosis and grading. IEEE Transactions on Biomedical Engineering, 57(3):665-674[DOI:10.1109/TBME.2009.2033804]
Araújo T, Aresta G, Castro E, Rouco J, Aguiar P, Eloy C, Polónia A and Campilho A. 2017. Classification of breast cancer histology images using convolutional neural networks. PLoS One, 12(6): #0177544[DOI:10.1371/journal.pone.0177544http://dx.doi.org/10.1371/journal.pone.0177544]
Araújo Teresa. 2015. BIOIMAGING 2015 Breast Histology Dataset[EB/OL].[2020-5-14].http://www.bioimaging2015.ineb.up.pt/dataset.htmlhttp://www.bioimaging2015.ineb.up.pt/dataset.html
Aresta G, Araújo T, Kwok S, Chennamsetty S S, Safwan M, Alex V, Marami B, Prastawa M, Chan M, Donovan M, Fernandez G, Zeineh J, Kohl M, Walz C, Ludwig F, Braunewell S, Baust M, Vu Q D, To M N N, Kim E, Kwak J T, Galal S, Sanchez-Freire V, Brancati N, Frucci M, Riccio D, Wang Y Q, Sun L L, Ma K Q, Fang J N, KoneI, Boulmane L, Campilho A, Eloy C, Polónia A and Aguiar P. 2019. BACH:grand challenge on breast cancer histology images. Medical Image Analysis, 56:122-139[DOI:10.1016/j.media.2019.05.010]
Bejnordi B E, Veta M, van Diest P J, van Ginneken B, Karssemeijer N, Litjens G, van der Laak J A W M and the CAMELYON16 Consortium. 2016. ISBI Challenge on cancer metastasis detection in lymph node[EB/OL].[2020-05-14].http://www.diagnijmegen.nl/index.php/Pathology_Project_Camelyon16http://www.diagnijmegen.nl/index.php/Pathology_Project_Camelyon16
Bejnordi B E, Veta M, van Diest P J, van Ginneken B, Karssemeijer N, Litjens G, van der Laak J A W M and the CAMELYON16 Consortium. 2017. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA, 318(22): 2199-2210[DOI:10.1001/jama.2017.14585http://dx.doi.org/10.1001/jama.2017.14585]
Bian X W and Ping Y F. 2019. Pathology in China:challenges and opportunities. Journal of Third Military Medical University, 41(19):1815-1817 [DOI:10.16016/j.1000-5404.201909212]
卞修武, 平轶芳. 2019.我国病理学科发展面临的挑战和机遇.第三军医大学学报, 41(19):1815-1817[DOI:10.16016/j.1000-5404.201909212]
Cruz-Roa A, Basavanhally A, González F, Gilmore H, Feldman M, Ganesan S, Shih N, Tomaszewski J and Madabhushi A. 2014. Automatic detection of invasive ductal carcinoma in whole slide images with convolutional neural networks//Proceedings of SPIE 9041, Medical Imaging 2014: Digital Pathology. San Diego: SPIE: #904103[DOI:10.1117/12.2043872http://dx.doi.org/10.1117/12.2043872]
Deng J, Dong W, Socher R, Li L J, Li K and Li F F. 2009. ImageNet: a large-scale hierarchical image database//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami: IEEE: 248-255[DOI:10.1109/CVPR.2009.5206848http://dx.doi.org/10.1109/CVPR.2009.5206848]
Goode A, Gilbert B, Harkes J, Jukic D and Satyanarayanan M. 2013. OpenSlide:a vendor-neutral software foundation for digital pathology. Journal of Pathology Informatics, 4:#27[DOI:10.4103/2153-3539.119005]
He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE: 770-778[DOI:10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90]
Hou L, Samaras D, Kurc T M, Gao Y, Davis J E and Saltz J H. 2016. Patch-based convolutional neural network for whole slide tissue image classification//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE: 2424-2433[DOI:10.1109/CVPR.2016.266http://dx.doi.org/10.1109/CVPR.2016.266]
Kather J N, Halama N and Marx A. 2018.100, 000 histological images of human colorectal cancer and healthy tissue (Version v0.1)[EB/OL].[2020-05-14].http://doi.org/10.5281/zenodo.1214456http://doi.org/10.5281/zenodo.1214456
Krizhevsky A, Sutskever I and Hinton G E. 2012. Imagenet classification with deep convolutional neural networks//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe: NIPS: 1097-1105
Li J H, Yang S, Huang X D, Da Q, Yang X Q, Hu Z Q, Duan Q, Wang C F and Li H S. 2019. Signet ring cell detection with a semi-supervised learning framework//Proceedings of the 26th International Conference on Information Processing in Medical Imaging. Hong Kong, China: Springer: 842-854[DOI:10.1007/978-3-030-20351-1_66http://dx.doi.org/10.1007/978-3-030-20351-1_66]
Li Y and Ping W. 2018. Cancer metastasis detection with neural conditional random field[EB/OL].[2020-05-14].https://arxiv.org/pdf/1806.07064.pdfhttps://arxiv.org/pdf/1806.07064.pdf
Litjens G. 2017. Automated slide analysis platform[EB/OL].[2020-05-14].https://computationalpathologygroup.github.io/ASAP/https://computationalpathologygroup.github.io/ASAP/
Liu Y, Gadepalli K, Norouzi M, Dahl G E, Kohlberger T, Boyko A, Venugopalan S, Timofeev A, Nelson P Q, Corrado G S, Hipp J D, Peng L and Stumpe M C. 2017. Detecting cancer metastases on gigapixel pathology images[EB/OL].[2020-05-14].https://arxiv.org/pdf/1703.02442.pdfhttps://arxiv.org/pdf/1703.02442.pdf
Macenko M, Niethammer M, Marron J S, Borland D, Woosley J T, Guan X J, Schmitt C and Thomas N E. 2009. A method for normalizing histology slides for quantitative analysis//Proceedings of 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro. Boston: IEEE: 1107-1110[DOI:10.1109/ISBI.2009.5193250http://dx.doi.org/10.1109/ISBI.2009.5193250]
MICCAI. 2014. Brain tumor digital pathology challenge[EB/OL].[2020-05-14].https://wiki.cancerimagingarchive.net/display/Public/MICCAI+2014+Grand+Challengeshttps://wiki.cancerimagingarchive.net/display/Public/MICCAI+2014+Grand+Challenges
Naik S, Doyle S, Agner S, Madabhushi A, Feldman M and Tomaszewski J. 2008. Automated gland and nuclei segmentation for grading of prostate and breast cancer histopathology//Proceedings of the 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro. Paris: IEEE: 284-287[DOI:10.1109/ISBI.2008.4540988http://dx.doi.org/10.1109/ISBI.2008.4540988]
Nature E. 2017. Histopathology is ripe for automation. Nature Biomedical Engineering, 1(12):#925[DOI:10.1038/s41551-017-0179-5]
Oskal K R J, Risdal M, Janssen E A M, Undersrud E S and Gulsrud T O. 2019. A U-net based approach to epidermal tissue segmentation in whole slide histopathological images. SN Applied Sciences, 1(7):1-12[DOI:10.1007/s42452-019-0694-y]
Otsu N. 1979. A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 9(1):62-66[DOI:10.1109/TSMC.1979.4310076]
Petushi S, Garcia F U, Haber M M, Katsinis C and Tozeren A. 2006. Large-scale computations on histology images reveal grade-differentiating parameters for breast cancer. BMC Medical Imaging, 6:#14[DOI:10.1186/1471-2342-6-14]
Qaiser T, Mukherjee A, Reddy P B C, Munugoti S D, Tallam V, Pitkäaho T, Lehtimäki T, Naughton T, Berseth M, Pedraza A, Mukundan R, Smith M, Bhalerao A, Rodner E, Simon M, Denzler J, Huang C H, Bueno G, Snead D, Ellis I O, Ilyas M and Rajpoot N. 2018. Her2 challenge contest:a detailed assessment ofautomated Her2 scoring algorithms in whole slide images of breast cancer tissues. Histopathology, 720(2):227-238[DOI:10.1111/his.13333]
Reinhard E, Adhikhmin M, Gooch B and Shirley P. 2001. Color transfer between images. IEEE Computer Graphics and Applications, 21(5):34-41[DOI:10.1109/38.946629]
Ronneberger O, Fischer P and Brox T. 2015. U-net: convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: Springer: 234-241[DOI:10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28]
Ruderman D L, Cronin T W and Chiao C C. 1998. Statistics of cone responses to natural images:implications for visual coding. Journal of the Optical Society of America A, 15(8):2036-2045[DOI:10.1364/JOSAA.15.002036]
Simonyan K and Zisserman A. 2014. Very deep convolutional networks for large-scale image recognition//Proceedings of the 3rd International Conference on Learning Representations. San Diego: [s.n.]: 1-14
Sirinukunwattana K, Pluim J P W, Chen H, Qi X J, Heng P A, Guo Y B, Wang L Y, Matuszewski B J, Bruni E, Sanchez U, Böhm A, Ronneberger O, Cheikh B B, Racoceanu D, Kainz P, Pfeiffer M, Urschler M, Snead D R J and Rajpoot N M. 2017. Gland segmentation in colon histology images:the glas challenge contest. Medical Image Analysis, 35:489-502[DOI:10.1016/j.media.2016.08.008]
Spanhol F A, Oliveira L S, Petitjean C and Heutte L. 2016. A dataset for breast cancer histopathological image classification. IEEE Transactions on Biomedical Engineering, 63(7):1455-1462[DOI:10.1109/TBME.2015.2496264]
Sun S J, Jiang B N, Zheng Y S and Xie F Y. 2019. A comparative study of CNN and FCN for histopathology whole slide image analysis//Proceedings of the 10th International Conference on Image and Graphics. Beijing, China: Springer: 558-567[DOI:10.1007/978-3-030-34110-7_47http://dx.doi.org/10.1007/978-3-030-34110-7_47]
Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V and Rabinovich A. 2015. Going deeper with convolutions//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE: 1-9[DOI:10.1109/CVPR.2015.7298594http://dx.doi.org/10.1109/CVPR.2015.7298594]
TCGA. 2006. The cancer genome atlas program[EB/OL].[2020-05-14].https://www.cancer.gov/tcgahttps://www.cancer.gov/tcga
Vahadane A, Peng T Y, Albarqouni S, Baust M,Steiger K, Schlitter A M, Sethi A, Esposito I and Navab N. 2015. Structure-preserved color normalization for histological images//The IEEE 12th International Symposium on Biomedical Imaging. New York: IEEE: 1012-1015[DOI:10.1109/ISBI.2015.7164042http://dx.doi.org/10.1109/ISBI.2015.7164042]
Veeling B S, Linmans J, Winkens J, Cohen T and Welling X. 2018. Rotation equivariant CNNs for digital pathology//Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention. Granada: Springer: 210-218[DOI:10.1007/978-3-030-00934-2_24http://dx.doi.org/10.1007/978-3-030-00934-2_24]
Wang D Y, Khosla A, Gargeya R, Irshad H and Beck A H. 2016. Deep learning for identifying metastatic breast cancer[EB/OL].[2020-05-14].https://arxiv.org/pdf/1606.05718.pdfhttps://arxiv.org/pdf/1606.05718.pdf
Wang X, Chen H, Gan C X, Lin H J, Dou Q, Tsougenis E, Huang Q T, Cai M Y and Heng P A. 2020. Weakly supervised deep learning for whole slide lung cancer image analysis. IEEE Transactions on Cybernetics, 50(9):3950-3962[DOI:10.1109/TCYB.2019.2935141]
Xu Y, Jia Z P, Wang L B, Ai Y Q, Zhang F, Lai M D and Chang E I C. 2017. Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features. BMC Bioinformatics, 18:#281[DOI:10.1186/s12859-017-1685-x]
Xu Y, Mo T, Feng Q W, Zhong P L, Lai M D and Chang E I C. 2014. Deep learning of feature representation with multiple instance learning for medical image analysis//Proceedings of 2014 IEEE international conference on acoustics, speech and signal processing. Florence: IEEE: 1626-1630[DOI:10.1109/ICASSP.2014.6853873http://dx.doi.org/10.1109/ICASSP.2014.6853873]
Zhang A, Lipton Z C, Li M and Smola A J. 2020. Dive into deep learning[EB/OL].[2020-05-14].https://d2l.ai/https://d2l.ai/
相关作者
相关机构
京公网安备11010802024621