多尺度信息交互与融合的乳腺病理图像分类方法
丁维龙1, 朱峰龙2, 郑魁3, 贾秀鹏4(1.浙江工业大学计算机科学技术学院;2.浙江工业大学计算机科学与技术学院;3.上海派影医疗科技有限公司;4.浙江省宁波市临床病理诊断中心) 摘 要
目的 基于深度学习方法进行乳腺癌识别是一项具有挑战的任务,目前较多研究使用单一倍率下的乳腺组织病理图像作为模型的输入,忽略了乳腺组织病理图像固有的多倍率特点,而少数将不同倍率下的图像作为模型输入的研究,存在特征利用率较低以及不同倍率的图像之间缺乏信息交互等问题。方法 针对上述问题,本文提出一种基于多尺度和分组注意力机制的卷积神经网络改进策略。该策略主要包括信息交互模块和特征融合模块。前者通过空间注意力加强不同倍率的图像之间的相关性,然后将加权累加的结果反馈给原始分支进行动态选择实现特征流通;后者则利用一种分组注意力来提升特征的利用率,同时基于特征金字塔来消除图像之间的感受野差异。结果 本文将上述策略应用到多种卷积网络中,并与最新的方法进行比较。在Camelyon16公开数据集上进行五折交叉验证实验,并对每一项评价指标计算均值和标准差。相比于单一尺度图像作为输入的卷积网络,本文改进的方法在准确率上提升0.9%~1.1%,F1分数提升1.1%~1.2%;相比于性能最好的TransPath网络,本文改进的DenseNet201在准确率上提升0.6%,精确率提升0.8%,F1分数提升0.6%,并且各项指标的标准差低于Transpath,表明加入策略的网络具有更好的稳定性。结论 本文所提出的策略能弥补一般多尺度网络的缺陷,并具备一定的通用性,可获得更好的乳腺癌分类性能。
关键词
A classification method of breast pathological images based on multi-scale information interaction and fusion
(1.College of Computer Science and Technology,Zhejiang University of Technology;2.Shanghai Paiyin Medical Technology Co., Ltd;3.Ningbo Clinical Pathology Diagnostic Center) Abstract
Objective Breast cancer recognition based on deep learning methods is a challenging task due to the large size of breast histopathology images (single image size is about 1GB), which needs to be cut and then recognized due to the current computational power limitations. Current research on breast cancer recognition tends to focus on single-scale networks, ignoring the characteristics of multiple magnifications and pyramidal structure storage of breast histopathology images. Several studies on multi-scale networks only input images of different magnifications into the network model and concatenate or sum different features after multi-layer convolutional layer operations. The feature fusion is simple and ignores the correlation between images of different scales, and does not consider the guidance between images of different scales when extracting the texture features of images in the shallow part of the network model. Therefore, there are problems such as low feature utilization and lack of information interaction between images of different magnifications.Method To address the above problems, this paper proposes a convolutional neural network improvement strategy based on multi-scale and group attention mechanisms. The strategy mainly includes two modules, which are the information interaction module and the feature fusion module. The first module extracts clearer cell morphological structure information from high-magnification images and global context information from low-magnification images through a spatial attention mechanism, and the feature information with high relevance to the classification target of the main branch will be given more weight. Finally, these features are weighted and accumulated, and the results are fed back to the original branch for dynamic selection to achieve feature interaction and circulation. The second module considers that the number of channels on the feature map will multiply as the depth of the network increases, and the general channel attention has the problems of large computation and low feature activation rate, this paper proposes group attention based on group convolution and combines it into the feature fusion module. In addition, since there is a difference in the perceptual domain of the images at different magnifications, i.e., the actual length of each pixel is different, this paper uses a feature pyramid to eliminate the perceptual domain difference in the feature fusion process. Result In this paper, the above strategy is applied to a variety of convolutional neural networks and compared with the latest methods. A five-fold cross-validation experiment is conducted on the Camelyon16 public dataset, and the mean and standard deviation are calculated for each evaluation metric. Compared with the single-scale convolutional network, the improved method in this paper improves 0.9%-1.1% in accuracy and 1.1%-1.2% in F1 score; compared with the best-performing TransPath network in the single-scale network, the improved DenseNet201 in this paper improves 0.6% in accuracy, 0.8% in precision, 0.6% in F1 score, and the standard deviation of the indicators is lower than that of Transpath, indicating that the network incorporating the strategy has a better stability. Conclusion In summary, the strategy proposed in this paper can make up for the shortcomings of general multi-scale networks and has certain generality to obtain better performance in breast cancer image classification. It is useful for future multi-scale research and feature extraction for downstream tasks.
Keywords
|