浏览全部资源
扫码关注微信
Received：01 September 2022，
Revised：2022-11-23，
Published：16 October 2023
移动端阅览
目的
2
胸腔积液肿瘤细胞团块的分割对肺癌的筛查有着积极作用。胸腔积液肿瘤细胞团块显微图像存在细胞聚集、对比度低和边界模糊等问题，现有网络模型进行细胞分割时无法达到较高精度。提出一种基于UNet网络框架，融合过参数卷积与注意力机制的端到端语义分割模型DOCUNet（depthwise over-parameterized CBAM UNet）。
方法
2
将UNet网络中的卷积层替换为过参数卷积层。过参数卷积层结合了深度卷积和传统卷积两种卷积，保证网络深度不变的同时，提高模型对图像特征的提取能力。在网络底端的过渡区域，引入结合了通道注意力与空间注意力机制的注意力模块CBAM（convolutional block attention module），对编码器提取的特征权重进行再分配，增强模型的分割能力。
结果
2
在包含117幅显微图像的胸腔积液肿瘤细胞团块数据集上进行5折交叉实验。平均IoU（intersection over union）、Dice系数、精确率、召回率和豪斯多夫距离分别为0.858 0、0.920 4、0.928 2、0.920 3和18.17。并且与UNet等多种已存在的分割网络模型进行对比，IoU、Dice系数和精确率、召回率相较于UNet提高了2.80%、1.65%、1.47%和1.36%，豪斯多夫距离下降了41.16%。通过消融实验与类激活热力图，证明加入CBAM注意力机制与过参数卷积后能够提高网络分割精度，并能使网络更加专注于细胞的内部特征。
结论
2
本文提出的DOCUNet将过参数卷积和注意力机制与UNet相融合，实现了胸水肿瘤细胞团块的有效分割。经过对比实验证明所提方法提高了细胞分割的精度。
Objective
2
Lung cancer-related early detection and intervention is beneficial for lowering mortality rates. Pleural effusion symptoms， and tumor cells and tumor cell masses can be sorted out in relevant to pleural effusion and its metastatic contexts. The detection of tumor cells in pleural effusion can be recognized as an emerging screening tool for lung cancer for early-stage intervention. One of the key preprocessing steps is focused on the precise segmentation of tumor cell masses in related to pleural fluid tumor cells. However， due to severe tumor cell masses-between overlapping and adhesion， unclear cell-to-cell spacing， and unstable staining results of tumor cells in pleural effusion are challenged to be resolved using conventional staining methods， manual micrographs of unstained pleural fluid tumor clumps for cell clump segmentation derived of experienced and well-trained pathologists. But， it still has such problems of inefficiency and the inevitable miss segmentation due to its labor-intensive work. In recent years， computer vision techniques have been developing intensively for optimizing the speed and accuracy of image analysis. Traditional methods for segmenting cellular microscopic images are carried out， including thresholding and such algorithms of clustering-based， graph-based， and active contouring. However， these methods are required for image downscaling， and they have the limitations of undeveloped graphical features. Convolutional neural network （CNN） based deep learning can be used to automatically find suitable features for image segmentation tasks nowadays. The UNet is derived from the end-to-end full convolutional network （FCN） structure， and it is widely used in medical image segmentation tasks due to its unique symmetric encoder and decoder network structure to get the segmentation result relevant to location information of the segmented target， in which arbitrary size image input and equal size output image can be yielded for arbitrary size image input and equal size output image. We develop a new CNN-based UNet network structure （DOCUNet） to perform tumor cell segmentation in pleural effusion， which can be focused on the integrated depthwise over-parameterized convolution （DO-conv） and channel and spatial attention convolutional block attention module （CBAM）.
Method
2
The network is developed and divided into three sections： encoder， feature enhancer， and decoder. The encoder consists of a convolution operation and a down-sampler， and a hybrid of depthwise convolution and vanilla convolution is demonstrated in terms of depthwise over-parameterized convolution （DO-conv） based convolution operation rather than vanilla convolution. In practice， to get the final image features extraction， the first stage of feature extraction is obtained by depthwise convolution along the feature dimension of the input image and followed by a vanilla convolution operation. This design improves the network’s ability to extract features from cell clumps while keeping the output image size constant， and it addresses the issue of unclear features caused by severe intercellular adhesion. For the transition to the decoder， the CBAM attention module is inserted as a feature enhancer in the last layer of the encoder. The CBAM attention module is based on the channel attention mechanism， and the spatial attention mechanism is used to redistribute the weights of the encoder’s high-dimensional features through suppressing other related cells-interfered background features and enhancing the network’s utilization of the tumor cells’ internal features. For the purpose of feature redistribution， channel and spatial-generated attention maps are pointed multiplied with the input features. The use of jump connections in the decoder allows the network to learn features at multiple scales contextual information. Our research is affiliated to Tianjin Medical University’s microscopic images of tumor cell masses in pleural effusion. For the training sample set， 80 percent of the 117 images collection with completed labeling is chosen in random. After data enhancement， 20% of the images collection is chosen as the test sample set. The framework for building the model is chosen as Pytorch. The training is carried out on an NVIDIA RTX 3090 GPU. Each of loss function is binary cross entropy （BCE）， the batch size is 4， and Adam is used as the optimizer based on an initial learning rate of 0.003， 1， and 2 of 0.9 and 0.999.
Result
2
To validate the proposed method’s effectiveness， the network models of all the five sort of semantic segmentation networks UNet， UNet++， ResUNet， Attention-UNet， UNet3+ and U2Net are involved for its comparative experiments using the same test sample set， and five sort of measurements of intersection over union （IoU）， Dice coefficient， precision， recall and Hausdorff distance， as evaluation metrics， are used to evaluate its segmentation results. For each of the five evaluation metrics， the proposed network is valued and yielded of 0.858 0，0.920 4，0.928 2， 0.920 3 and 18.17. Compared to UNet， first of four measures are improved by 2.80， 1.65， 1.47 and 1.36 percent each. The Hausdorff distance is decreased by 41.16%. The proposed network’s segmentation results are visually closer to ground truth further， and the segmentation is clearer than other cell boundary-related models to a certain extent. The SEG-GradCAM-like activation heat maps-relevant ablation experiments is demonstrated that the proposed method can improve the network’s feature extraction ability， for which the network is allowed to focus on more on the internal features of tumor cells while suppressing irrelevant feature information in the image background.
Conclusion
2
To achieve effective tumor cell cluster segmentation in pleural effusion， our DOCUNet is developed in terms of an attention mechanism and a UNet-integrated depthwise over-parameterized convolution. Comparative experiments demonstrate that the proposed method can be used to improve cell segmentation accuracy and its contexts further.
Badrinarayanan V ， Handa A and Cipolla R . 2015 . SegNet： a deep convolutional encoder-decoder architecture for robust semantic pixel-wise labeling ［EB/OL］. ［ 2022-7-22 ］. https://arxiv.org/pdf/1505.07293.pdf https://arxiv.org/pdf/1505.07293.pdf
Caicedo J C ， Roth J ， Goodman A ， Becker T ， Karhohs K W ， Broisin M ， Molnar C ， McQuin C ， Singh S ， Theis F J and Carpenter A E . 2019 . Evaluation of deep learning strategies for nucleus segmentation in fluorescence images . Cytometry Part A ， 95 （ 9 ）： 952 - 965 ［ DOI： 10.1002/cyto.a.23863 http://dx.doi.org/10.1002/cyto.a.23863 ］
Cao J M ， Li Y Y ， Sun M C ， Chen Y ， Lischinski D ， Cohen-Or D ， Chen B Q and Tu C H . 2022 . DO-Conv： depthwise over-parameterized convolutional layer . IEEE Transactions on Image Processing ， 31 ： 3726 - 3736 ［ DOI： 10.1109/TIP.2022.3175432 http://dx.doi.org/10.1109/TIP.2022.3175432 ］
Chaurasia A and Culurciello E . 2017 . LinkNet： exploiting encoder representations for efficient semantic segmentation // Proceedings of 2017 IEEE Visual Communications and Image Processing （VCIP） . St. Petersburg， USA ： IEEE： 1 - 4 ［ DOI： 10.1109/VCIP.2017.8305148 http://dx.doi.org/10.1109/VCIP.2017.8305148 ］
Chen Y S ， Li H ， Zhou X T and Wan C . 2021 . The fusing of dilated convolution and attention for segmentation of gastric cancer tissue sections . Journal of Image and Graphics ， 26 （ 9 ）： 2281 - 2292
陈颍锶 ， 李晗 ， 周雪婷 ， 万程 . 2021 . 融合空洞卷积与注意力的胃癌组织切片分割 . 中国图象图形学报 ， 26 （ 9 ）： 281 - 2292 ［ DOI： 10.11834/jig.200765 http://dx.doi.org/10.11834/jig.200765 ］
Dice L R . 1945 . Measures of the amount of ecologic association between species . Ecology ， 26 （ 3 ）： 297 - 302 ［ DOI： 10.2307/1932409 http://dx.doi.org/10.2307/1932409 ］
Echle A ， Rindtorff T ， Brinker T J ， Luedde T ， Pearson A T and Kather J N . 2021 . Deep learning in cancer pathology： a new generation of clinical biomarkers . British Journal of Cancer ， 124 （ 4 ）： 686 - 696 ［ DOI： 10.1038/s41416-020-01122-x http://dx.doi.org/10.1038/s41416-020-01122-x ］
Guo F ， Li W Q ， Zhao X and Zou B J . 2021 . Glaucoma screening method based on semantic feature map guidance . Journal of Computer-Aided Design and Computer Graphics ， 33（3） 363 - 375
郭璠 ， 李伟清 ， 赵鑫 ， 邹北骥 . 2021 . 语义特征图引导的青光眼筛查方法 . 计算机辅助设计与图形学学报 ， 33 （ 3 ）： 363 - 375 ［ DOI： 10.3724/SP.J.1089.2021.18474 http://dx.doi.org/10.3724/SP.J.1089.2021.18474 ］
Huang H M ， Lin L F ， Tong R F ， Hu H J ， Zhang Q W ， Iwamoto Y ， Han X H ， Chen Y W and Wu J . 2020 . UNet 3+： a full-scale connected UNet for medical image segmentation // Proceedings of ICASSP 2020-2020 IEEE International Conference on Acoustics， Speech and Signal Processing （ICASSP） . Barcelona， Spain ： IEEE： 1055 - 1059 ［ DOI： 10.1109/icassp40776.2020.9053405 http://dx.doi.org/10.1109/icassp40776.2020.9053405 ］
Ioffe S and Szegedy C . 2015 . Batch normalization： accelerating deep network training by reducing internal covariate shift ［EB/OL］. ［ 2022-7-22 ］. https://arxiv.org/pdf/1502.03167.pdf https://arxiv.org/pdf/1502.03167.pdf
Kingma D P and Ba J L . 2017 . Adam： a method for stochastic optimization ［EB/OL］. ［ 2022-7-22 ］. https://arxiv.org/pdf/1412.6980.pdf https://arxiv.org/pdf/1412.6980.pdf
Ma S K ， Zhao M ， Wang H ， Shi F ， Sun X G ， Chen S Y and Dai H N . 2021 . Fused 3-stage image segmentation for pleural effusion cell clusters // Proceedings of the 25th International Conference on Pattern Recognition （ICPR） . Milan， Italy ： IEEE： 1934 - 1941 ［ DOI： 10.1109/ICPR48806.2021.9412567 http://dx.doi.org/10.1109/ICPR48806.2021.9412567 ］
Ma S K ， Zhao M ， Shi F ， Sun X G and Chen S Y . 2022 . Attention driven nuclei segmentation method for cell clusters . Journal of Xidian University ， 49 （ 2 ）： 198 - 206
马思珂 ， 赵萌 ， 石凡 ， 孙续国 ， 陈胜勇 . 2022 . 注意力驱动的细胞团簇细胞核分割算法 . 西安电子科技大学学报 ， 49 （ 2 ）： 198 - 206 ［ DOI： 10.19665/j.issn1001-2400.2022.02.023 http://dx.doi.org/10.19665/j.issn1001-2400.2022.02.023 ］
McGrath J ， Jimenez M and Bridle H . 2014 . Deterministic lateral displacement for particle separation： a review . Lab on a Chip ， 14 （ 21 ）： 4139 - 4158 ［ DOI： 10.1039/c4lc00939h http://dx.doi.org/10.1039/c4lc00939h ］
Nagrath S ， Sequist L V ， Maheswaran S ， Bell D W ， Irimia D ， Ulkus L ， Smith M R ， Kwak E L ， Digumarthy S ， Muzikansky A ， Ryan P ， Balis U J ， Tompkins R G ， Haber D A and Toner M . 2007 . Isolation of rare circulating tumour cells in cancer patients by microchip technology . Nature ， 450 （ 7173 ）： 1235 - 1239 ［ DOI： 10.1038/nature06385 http://dx.doi.org/10.1038/nature06385 ］
Oktay O ， Schlemper J ， Le Folgoc L ， Lee M ， Heinrich M ， Misawa K ， Mori K ， McDonagh S ， Hammerla N Y ， Kainz B ， Glocker B and Rueckert D . 2018 . Attention U-Net： learning where to look for the pancreas ［EB/OL］. ［ 2022-7-22 ］. https://arxiv.org/pdf/1804.03999.pdf https://arxiv.org/pdf/1804.03999.pdf
Paszke A ， Chaurasia A ， Kim S and Culurciello E . 2016 . ENet： a deep neural network architecture for real-time semantic segmentation ［EB/OL］. ［ 2022-7-22 ］. https://arxiv.org/pdf/1606.02147.pdf https://arxiv.org/pdf/1606.02147.pdf
Qin X B ， Zhang Z C ， Huang C Y ， Dehghan M ， Zaiane O R and Jagersand M . 2020 . U2-Net： going deeper with nested U-structure for salient object detection . Pattern Recognition ， 106 ： # 107404 ［ DOI： 10.1016/j.patcog.2020.107404 http://dx.doi.org/10.1016/j.patcog.2020.107404 ］
Ronneberger O ， Fischer P and Brox T . 2015 . U-net： convolutional networks for biomedical image segmentation // Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention . Munich， Germany ： Springer： 234 - 241 ［ DOI： 10.1007/978-3-319-24574-4_28 http://dx.doi.org/10.1007/978-3-319-24574-4_28 ］
Sarioglu A F ， Aceto N ， Kojic N ， Donaldson M C ， Zeinali M ， Hamza B ， Engstrom A ， Zhu H L ， Sundaresan T K ， Miyamoto D T ， Luo X ， Bardia A ， Wittner B S ， Ramaswamy S ， Shioda T ， Ting D T ， Stott S L ， Kapur R ， Maheswaran S ， Haber D A and Toner M . 2015 . A microfluidic device for label-free， physical capture of circulating tumor cell clusters . Nature Methods ， 12 （ 7 ）： 685 - 691 ［ DOI： 10.1038/nmeth.3404 http://dx.doi.org/10.1038/nmeth.3404 ］
Selvaraju R R ， Cogswell M ， Das A ， Vedantam R ， Parikh D and Batra D . 2020 . Grad-CAM： visual explanations from deep networks via gradient-based localization . International Journal of Computer Vision ， 128 （ 2 ）： 336 - 359 ［ DOI： 10.1007/s11263-019-01228-7 http://dx.doi.org/10.1007/s11263-019-01228-7 ］
Sung H ， Ferlay J ， Siegel R L ， Laversanne M ， Soerjomataram I ， Jemal A and Bray F . 2021 . Global cancer statistics 2020： GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries . CA： A Cancer Journal for Clinicians ， 71 （ 3 ）： 209 - 249 ［ DOI： 10.3322/caac.21660 http://dx.doi.org/10.3322/caac.21660 ］
Tian Y T and Kang W Z . 2021 . New progress in research on global cancer incidence . China Medicine ， 16 （ 10 ）： 1446 - 1447
田艳涛 ， 康文哲 . 2021 . 全球癌症发病情况研究新进展 . 中国医药 ， 16 （ 10 ）： 1446 - 1447 ［ DOI： 10.3760/j.issn.1673-4777.2021.10.002 http://dx.doi.org/10.3760/j.issn.1673-4777.2021.10.002 ］
Vinogradova K ， Dibrov A and Myers G . 2020 . Towards interpretable semantic segmentation via gradient-weighted class activation mapping （student abstract） . Proceedings of the AAAI Conference on Artificial Intelligence ， 34 （ 10 ）： 13943 - 13944 ［ DOI： 10.1609/aaai.v34i10.7244 http://dx.doi.org/10.1609/aaai.v34i10.7244 ］
Wang J J ， Yu Z S ， Luan Z Y ， Ren J W ， Zhao Y H and Yu G . 2021 . DA-ResUNet： a novel method for brain tumor segmentation based on U-Net with residual block and CBAM // Proceedings of 2021 International Conference on Image， Video Processing， and Artificial Intelligence . Shanghai， China ： SPIE： 86 - 91 ［ DOI： 10.1117/12.2611751 http://dx.doi.org/10.1117/12.2611751 ］
Woo S ， Park J ， Lee J Y and Kweon I S . 2018 . CBAM： convolutional block attention module // Proceedings of the 15th European Conference on Computer Vision （ECCV） . Munich， Germany ： Springer： 3 - 19 ［ DOI： 10.1007/978-3-030-01234-2_1 http://dx.doi.org/10.1007/978-3-030-01234-2_1 ］
Xiao X ， Lian S ， Luo Z M and Li S Z . 2018 . Weighted Res-UNet for high-quality retina vessel segmentation // Proceedings of the 9th International Conference on Information Technology in Medicine and Education . Hangzhou， China ： IEEE： 327 - 331 ［ DOI： 10.1109/itme.2018.00080 http://dx.doi.org/10.1109/itme.2018.00080 ］
Zhai T T ， Ye D K ， Zhang Q W ， Wu Z Q and Xia X H . 2017 . Highly efficient capture and electrochemical release of circulating tumor cells by using aptamers modified gold nanowire arrays . ACS Applied Materials and Interfaces ， 9 （ 40 ）： 34706 - 34714 ［ DOI： 10.1021/acsami.7b11107 http://dx.doi.org/10.1021/acsami.7b11107 ］
Zhao L L . 2020 . Study on Microfluidic Chip Detection of Pleural Effusion Tumor Cell Methodology . Tianjin ： Tianjin Medical University
赵莉逦 . 2020 . 基于微流控芯片检测胸水肿瘤细胞团块的方法学研究 . 天津 ： 天津医科大学
Zhou Z W ， Rahman Siddiquee M M ， Tajbakhsh N and Liang J M . 2018 . UNet++： a nested U-Net architecture for medical image segmentation // Proceedings of the 4th International Workshop and the 8th International Workshop on Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support . Granada， Spain ： Springer： 3 - 11 ［ DOI： 10.1007/978-3-030-00889-5_1 http://dx.doi.org/10.1007/978-3-030-00889-5_1 ］
相关文章
相关作者
相关机构
京公网安备11010802024621
