Fourier transform channel attention network for cholangiocarcinoma hyperspectral image segmentation

Shaojia Zheng; Song Qiu; Qingli Li; Mei Zhou; Menghan Hu; Guanzhen Yu

doi:10.11834/jig.210207

Medical Hyperspectral Imagery | Views : 0 下载量: 79 CSCD: 5

PDF
Export
Share
Collection
Album

Fourier transform channel attention network for cholangiocarcinoma hyperspectral image segmentation
Vol. 26, Issue 8, Pages: 1836-1846(2021)
Received：06 April 2021，

Revised：2021-5-12，

Accepted：19 May 2021，

Published：16 August 2021
DOI： 10.11834/jig.210207
稿件说明：

移动端阅览

DOI：

Shaojia Zheng, Song Qiu, Qingli Li, Mei Zhou, Menghan Hu, Guanzhen Yu. Fourier transform channel attention network for cholangiocarcinoma hyperspectral image segmentation[J]. Journal of Image and Graphics, 2021, 26(8): 1836-1846. DOI： 10.11834/jig.210207.

摘要

目的

胆管癌高光谱图像的光谱波段丰富但存在冗余，造成基于深度神经网络高光谱图像分割方法的分割精度下降，虽然一些基于通道注意力机制的网络能够关注重要通道，但在处理通道特征时存在信息表示不足问题，因此本文研究构建一种新的通道注意力机制深度网络，以提高分割准确性。

方法

提出了傅里叶变换多频率通道注意力机制(frequency selecting channel attention，FSCA)。FSCA对输入特征进行2维傅里叶变换，提取部分频率特征，再通过两层全连接层得到通道权重向量，将通道权重与对应通道特征相乘，获得了融合通道注意力信息的输出。针对患癌区域和无癌区域数据不平衡问题引入了Focal损失，结合Inception模块，构建基于Inception-FSCA的胆管癌高光谱图像分割网络。

结果

在采集的胆管癌高光谱数据集上进行实验，Inception-FSCA网络的准确率(accuracy)、精度(precision)、敏感性(sensitivity)、特异性(specificity)、Kappa系数分别为0.978 0、0.965 4、0.958 6、0.985 2、0.945 6，优于另外5种对比方法。与合成的假彩色图像的分割结果相比，高光谱图像上的实验指标分别提高了0.058 4、0.105 8、0.087 5、0.039 0、0.149 3。

结论

本文所提出的傅里叶变换多频率通道注意力机制能够更有效地利用通道信息，基于Inception-FSCA的胆管癌高光谱图像分割网络能够提升分割效果，在胆管癌医学辅助诊断方面具有研究和应用价值。

Abstract

Objective

Cholangiocarcinoma is a rare but highly malignant tumor. Hyperspectral imaging (HSI)

which originated from remote sensing

is an emerging image modality for diagnosis and image-guided surgery. HSI takes the advantage of acquiring 2D images across a wide range of electromagnetic spectrum. HSI can obtain spectral and optical properties of tissue and provide more information than RGB images. Redundant information will persist even though HSI contains tens the amount of data compared with RGB images with the same spatial dimension. Traditional dimensionality reduction methods

such as principal component analysis and kernel method

reduce the data by converting the original spectral space to a low-dimensional one

which is not suitable in end-to-end models. Recently

convolutional neural networks have demonstrated excellent performance on computer vision tasks

including classification

segmentation

and detection. Attention mechanism is used in convolutional neural network(CNN) to improve the representation of feature maps. Typical channel attention modules

such as squeeze-and-excitation net (SENet)

squeezes the input features by global average pooling to produce a channel descriptor. However

different channels could have the same mean value. We proposed frequency selecting channel attention (FSCA) mechanism to address this issue. An Inception-FSCA network is also proposed for the segmentation of a hyperspectral image of cholangiocarcinoma tissues.

Method

FSCA can exploit the information from different frequency components. This method consists of three steps. First

the input feature map is transformed in the frequency domain by Fourier transform. Second

a representative frequency amplitude is selected to efficiently use the obtained frequencies. These selected frequencies are arranged in a column of vectors. Third

these vectors are sent to two consecutive fully connected layers to obtain a channel weight vector. Then

a sigmoid function is used to scale each channel weight between zero and one. Every element in the channel weight vector is multiplied with the corresponding channel feature. FSCA can adjust the channel information

strengthen the important channels

and suppress the unimportant. This work uses a microscopic hyperspectral imaging system to obtain hyperspectral images of cholangiocarcinoma tissues. These images have a spectral bandwidth from 550 nm to 1 000 nm in 7.5 nm increments

producing a hypercube with 60 spectral bands. Spatial resolution of each image is 1 280×1 024 pixels. The ground truth label is manually annotated by experts. The method is implemented using Python3.6 and TensorFlow1.14.0 on NVDIA TITAN X GPU

Intel i7-9700KF CPU. The learning rate is 0.000 5

the batch size is 256

and the optimization strategy is Adam. Cancerous areas have different sizes

resulting in unbalanced positive and negative samples. Focal loss is chosen as a loss function.

Result

We conducted comparative and ablation experiments on our dataset. We use several evaluation metrics to evaluate the performance of the inception-FSCA. The accuracy

precision

sensitivity

specificity

and Kappa are 0.978 0

0.965 4

0.958 6

0.985 2

and 0.945 6

respectively.

Conclusion

In this study

we proposed a Fourier transform frequency selecting channel attention mechanism. The proposed channel attention module can be conveniently inserted in CNN. An Inception-FSCA network is built for the segmentation of hyperspectral images of cholangiocarcinoma tissues. Quantitative results show that our method has excellent performance. Inception-FSCA can be applied in the outer image segmentation and classification tasks.

关键词

Keywords

references

Camps-Valls G and Bruzzone L. 2005. Kernel-based methods for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 43(6): 1351-1362[DOI: 10.1109/TGRS.2005.846154]

Fang B, Li Y, Zhang H K and Chan J C W. 2019. Hyperspectral images classification based on dense convolutional networks with spectral-wise attention mechanism. Remote Sensing, 11(2): #159[DOI: 10.3390/rs11020159]

Guan S H, Yang G, Lu S and Fu Y Y. 2020. Multi-objective optimization of hyperspectral band selection based on attention mechanism. Acta Optica Sinica, 40(21): #2128002

关世豪, 杨桄, 卢珊, 付严宇. 2020. 基于注意力机制的多目标优化高光谱波段选择. 光学学报, 40(21): #2128002 [DOI: DOI:10.3788/AOS202040.2128002]

Halicek M, Lu G L, Little J V, Wang X, Patel M, Griffith C C, Ei-Deiry M W, Chen A Y and Fei B W. 2017. Deep convolutional neural networks for classifying head and neck cancer using hyperspectral imaging. Journal of Biomedical Optics, 22(6): #060503[DOI: 10.1117/1.JBO.22.6.060503]

He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 770-778[ DOI:10.1109/CVPR.2016.90 http://dx.doi.org/10.1109/CVPR.2016.90 ]

Hu J, Shen L and Sun G. 2018. Squeeze-and-excitation networks//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. SaltLake City, USA: IEEE: 7132-7141[ DOI:10.1109/CVPR.2018.00745 http://dx.doi.org/10.1109/CVPR.2018.00745 ]

Huang G, Liu Z, van der Maaten L and Weinberger K Q. 2017. Densely connected convolutional networks//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 2261-2269[ DOI:10.1109/CVPR.2017.243 http://dx.doi.org/10.1109/CVPR.2017.243 ]

Kingma D and Ba J. 2015. Adam: a method for stochastic optimization[EB/OL]. [2020-07-29] . https://arxiv.org/pdf/1412.6980.pdf https://arxiv.org/pdf/1412.6980.pdf

Krizhevsky A, Sutskever I and Hinton G E. 2012. ImageNet classification with deep convolutional neural networks. Adavances in neural information processing systems, 25(2): 1097-1105[DOI: 10.1145/3065386]

Li X, Wang W H, Hu X L and Yang J. 2019. Selective kernel networks//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE: 510-519[ DOI:10.1109/CVPR.2019.00060 http://dx.doi.org/10.1109/CVPR.2019.00060 ]

Lin T Y, Goyal P, Girshick R, He K M and Dollár P. 2017. Focal loss for dense object detection//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE: 2999-3007[ DOI:10.1109/ICCV.2017.324 http://dx.doi.org/10.1109/ICCV.2017.324 ]

Lu G L and Fei B W. 2014. Medical hyperspectral imaging: a review. Journal of Biomedical Optics, 19(1): #010901[DOI: 10.1117/1.JBO.19.1.010901]

Ma X J, Liu R, Li C X, Chen W L and Xu K X. 2020. Hyperspectral imaging of in vivo tissues: a review. Laser and Optoelectronics Progress, 57(8): #080002

马雪洁, 刘蓉, 李晨曦, 陈文亮, 徐可欣. 2020. 高光谱在体组织成像方法的研究进展. 激光与光电子学进展, 57(8): #080002 [DOI: 10.3788/LOP57.080002]

Melgani F and Bruzzone L. 2004. Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience and Remote Sensing, 42(8): 1778-1790[DOI: 10.1109/TGRS.2004.831865]

Ni Q H and Wang J. 2015. Interpretation and consideration of guidelines for diagnosis and treatment of hilar cholangiocarcinoma (2013 edition). Journal of Hepatopancreatobiliary Surgery, 27(6): 450-454

倪其泓, 王坚. 2015. 肝门部胆管癌诊断和治疗指南(2013版)的解读与思考. 肝胆胰外科杂志, 27(6): 450-454 [DOI: 10.11952/j.issn.1007-1954.2015.06.003]

Roy A G, Navab N and Wachinger C. 2019. Recalibrating fully convolutional networks with spatial and channel "squeeze and excitation" blocks. IEEE Transactions on Medical Imaging, 38(2): 540-549[DOI: 10.1109/TMI.2018.2867261]

Song N, Guo H Z, Shen C Y, Sun C, Yang J and Zhang J N. 2020. Research on detection technology of brain glioma based on hyperspectral imaging. Spectroscopy and Spectral Analysis, 40(12): 3784-3788

宋楠, 郭汉洲, 沈春洋, 孙慈, 杨晋, 张金男. 2020. 高光谱成像的脑胶质瘤检测. 光谱学与光谱分析, 40(12): 3784-3788 [DOI: 10.3964/j.issn.1000-0593(2020)12-3784-05]

Szegedy C, Ioffe S, Vanhoucke V and Alemi A A. 2017. Inception-v4, inception-ResNet and the impact of residual connections on learning//Proceedings of the 31st AAAI Conference on Artificial Intelligence. San Francisco, USA: AAAI: 4278-4284

Wang F, Jiang M Q, Qian C, Yang S, Li C, Zhang H G, Wang X G and Tang X O. 2017. Residual attention network for image classification//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 6450-6458[ DOI:10.1109/CVPR.2017.683 http://dx.doi.org/10.1109/CVPR.2017.683 ]

Wang Q L, Wu B G, Zhu P F, Li P H, Zou W M and Hu Q H. 2020. ECA-Net: efficient channel attention for deep convolutional neural networks//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE: 11531-11539[ DOI:10.1109/CVPR42600.2020.01155 http://dx.doi.org/10.1109/CVPR42600.2020.01155 ]

Yu H L, Huo J Y and Zhang Y Z. 2020. Hyperspectral image terrain classification based on multi-layer feature SENet and multi-scale wide residual. Research and Exploration in Laboratory, 39(7): 28-34, 44

于慧伶, 霍镜宇, 张怡卓. 2020. 融合多层特征SENet和多尺度宽残差的高光谱图像地物分类. 实验室研究与探索, 39(7): 28-34, 44 [DOI: 10.3969/j.issn.1006-7167.2020.07.007]

Zheng X. 2019. Study on the Recognition of Breast Tumor Tissue Microarray based on Microscopic Hyperspectral Imaging. Shanghai: East China Normal University

郑欣. 2019. 基于显微高光谱成像的乳腺肿瘤组织微阵列识别分析方法研究. 上海: 华东师范大学

Zhong L H, Hu L N and Zhou H. 2019. Deep learning based multi-temporal crop classification. Remote Sensing of Environment, 221: 430-443[DOI: 10.1016/j.rse.2018.11.032]

Zhong Z L, Li J, Luo Z M and Chapman M. 2018. Spectral-spatial residual network for hyperspectral image classification: a 3-D deep learning framework. IEEE Transactions on Geoscience and Remote Sensing, 56(2): 847-858[DOI: 10.1109/TGRS.2017.2755542]

Alert me when the article has been cited

提交

LFSCA-UNet: liver fibrosis region segmentation network based on spatial and channel attention mechanisms

Remote sensing image segmentation model based on an adaptive receptive field mechanism

Bidirectional feedback iterative optimization network for joint reconstruction and segmentation of photoacoustic tomographic images

Cross-scale Transformer image super-resolution reconstruction with fusion channel attention

Potential and prospects of segment anything model： a survey