目的 胆管癌是一种致死率较高的癌症,癌症早期发现和治疗可显著降低发病率,实现病理切片的数字化诊断,能有效提升癌症临床诊断的精度和效率。病理切片的显微高光谱图像相对于彩色图像包含更丰富的光谱信息,为癌细胞和正常细胞的分类提供巨大潜力。多数病理高光谱图像分类算法的性能高度依赖于高质量标注数据集,病理高光谱图像需要由经验丰富的病理学家手工标记,该过程耗时、费力。基于自监督的特征提取算法可以缓解数据标注难题。然而传统对比自监督学习提取高级语义信息能力有限,且目前没有针对病理高光谱图像的图像增强方法。因此,本文提出了自监督提取光谱序列和语义信息的胆管癌高光谱图像分类方法,提升自监督方法的特征提取能力及分类精度。方法 首先,设计能有效捕获病理高光谱中序列信息的Transformer编码器进行高光谱图像增强,随后通过新颖的原型对比学习网络捕获图像中高级语义信息,网络提取特征的过程使用未标记数据,最后,通过少量标记数据微调下游分类任务网络得到分类结果。结果 在多维胆管癌病理高光谱数据集的8个场景上实验,实验结果表明,相比于现有12种广泛使用的高光谱特征提取方法,本文方法提取的特征在下游分类任务中能达到更高的分类精度,平均总体分类精度达到96.63%。结论 本文方法能从未标记的胆管癌高光谱图像中提取有效特征,特征应用于分类任务中达到较高分类精度,缓解了病理高光谱图像数据标注难题,对胆管癌的医学诊断具有一定的研究价值和现实意义。

Objective Cholangiocarcinoma is a kind of cancer with high fatality rate. Early detection and treatment of cancer can significantly reduce the incidence. Digital diagnosis of pathological sections can effectively improve the accuracy and efficiency of clinical diagnosis of cancer. Microscopic hyperspectral images of pathological sections contain richer spectral information than color images. Due to the specific spectral response of biological tissues, pathological tissues have different spectral characteristics from normal tissues, and meaningful and rich spectral information provides great potential for the classification of cancer cells and healthy cells. The performance of most pathologic hyperspectral image classification algorithms is highly dependent on high-quality labeled data sets, however pathologic hyperspectral images need to be manually labeled by experienced pathologists, which is time-consuming and laborious. The feature extraction algorithm based on self-supervision first extracts features from unlabeled image data in an unsupervised way by designing pretext tasks, and then transfers them to downstream tasks. After fine-tuning the downstream task network with a limited number of labeled samples, the performance can achieve the supervised learning performance, and the method alleviates the data annotation problem. However, traditional contrast self-supervised learning has a limited ability to extract high-level semantic information, and there is no image enhancement method for pathological hyperspectral images. Therefore, this paper proposes a self-supervised method to extract sequential spectral data and semantic information from hyperspectral images of cholangiocarcinoma to improve the feature extraction capability and classification accuracy of the self-supervised method. Method The pathological hyperspectral data of cholangiocarcinoma are characterized by high spectral dimension, strong correlation and nonlinearity of spectral response curves of similar cells. The feature map extracted by CNN network corresponds to the local receptive field, which will ignore the global information of spectral dimension. In view of the limited ability of CNN to characterize sequential spectral data, this paper first designs transformer encoder structure for image enhancement to fully retain the sequence details in the original image. Using the powerful sequence information modeling capability of Transformer architecture from natural language processing, the spectral curve reflected by each pixel of hyperspectral images is regarded as a spectral sequence. Transformer uses position embedding and attention module to pay attention to the differences between spectral sequences, so as to better learn sequential spectral data. Secondly, after the image is enhanced with transformer encoder structure to obtain positive samples, the convolutional autoencoder can be used as another set of image enhancement to obtain negative samples required for contrast learning. Then, in view of the limited ability of traditional contrastive learning to extract advanced semantic information, this paper applies prototypical contrastive learning to feature extraction of pathological hyperspectral images. Positive and negative samples are trained through the clustering and instance discrimination tasks of prototypical contrastive learning network to learn advanced semantic information in images. The above process of extracting features from network structures uses unlabeled data. Finally, the classification results are obtained by fine-tuning the downstream classification task network with a few labeled features. Result Experiments were conducted on 8 scenes in the hyperspectral data set of multidimensional cholangiocarcinoma pathology. 8 scenes were selected from 8 patients. In order to ensure the representativeness of the scenes, cancer cell morphology, cancer cell proportion and spectral response curve were different in each scene. Among them, the proportion of cancer region in scenes 2, 3 and 8 was only 1/8 of the whole picture. Experiments were conducted on each scenario, 5% of the data was labeled for training, and 95% of the data was used for testing. To verify the effectiveness of the self-supervised method proposed in this paper on pathological hyperspectral datasets, we compared it with 12 widely used algorithms and networks, including six supervised feature extraction methods and six unsupervised feature extraction algorithms. In the process of the experiment, the effective features are extracted from the original data by various methods. Then, the fully connected layer is set as the downstream task network, and the effectiveness of the feature extraction method is evaluated by the classification results of the downstream task network. The experimental results show that the features extracted by this method can achieve higher classification accuracy in downstream classification tasks, with the average overall classification accuracy reaching 96.63%. At the same time, abundant ablation experiments and feature dimension reduction experiments were carried out to verify the necessity of the network structure of each part of the proposed method. The tsne dimensional reduction visualization diagram reflected that the semantic features extracted by the proposed method were linearly separable. Conclusion The method in this paper can extract effective features from unlabeled hyperspectral images of cholangiocarcinoma, and the features can be applied to classification tasks to achieve high classification accuracy, which alleviates the problem of pathological hyperspectral image data labeling, and has certain research value and practical significance for the medical diagnosis of cholangiocarcinoma.