浏览全部资源
扫码关注微信
Published： 16 November 2023 ，
移动端阅览
花勇， 李珍珍， 潘建宏， 杨烜. 2023. 边界信息保持的全染色肾脏切片多粒度分割. 中国图象图形学报， 28(11):3575-3589
Hua Yong， Li Zhenzhen， Pan Jianhong， Yang Xuan. 2023. Boundary-preserving multi-scale glomerulus segmentation for full-stained kidney slice. Journal of Image and Graphics， 28(11):3575-3589
目的
2
肾小球图像的准确分割对肾脏病理学的疾病诊断和定量分析起到关键作用，然而全染色肾脏切片图像存在由肾小球个体差异大导致的空间尺度和上下文形状变化大，以及图像分辨率过高的问题，给高精度、高性能分割任务带来挑战。为此，提出一种边界信息保持的全染色肾脏切片多粒度分割方法。
方法
2
使用一种多粒度上下文的空间注意力机制生成多粒度和多形状变化的空间注意力图，以限制上下文特征，减弱背景对目标的影响，强化网络对目标的感知能力，使网络更多地关注小目标特征；将原图像切分为若干小图来解决全染色图像分辨率高的问题，使用增广路径边界补零策略处理卷积核存在的贡献偏移效应，解决了肾小球目标处于图像边界所导致的分割困难问题，保证图像块的信息无损失地向高层传递，提高处于图像块边界的肾小球目标的分割精度；进一步地，针对图像块拼接带来的边缘肾小球容易漏检、计算开销大的问题，采用特征复用的概率累积滑窗策略，同时提高了分割精度和效率。
结果
2
在小鼠肾脏细胞切片和HuBMAP（human biomolecular atlas program）人体肾脏数据上，本文方法提高了分割精度，并使预测速度提高50%左右。
结论
2
对于全染色肾脏切片的肾小球分割问题，多粒度上下文特征和增广路径边界补零策略解决了边界区域肾小球目标分割困难、分割精度低的问题，并通过概率累积滑窗策略提高分割速度，相较传统的分割方法有更优秀的性能。
Objective
2
Medical image segmentation is a key issue in determining whether medical images can provide reliable information in treatment and clinical diagnosis. The accurate segmentation of glomeruli plays a key role in diagnosing and quantitatively analyzing diseases in renal pathology. Traditional methods used in glomerular image segmentation include traditional pattern recognition and machine learning-based recognition. However， these methods required hand-crafted features. Segmentation methods based on convolutional neural networks （CNNs） have shown strong generalization performance with features learned by networks. Early diagnosis is conducive to treating kidney disease. However， a full-stained kidney slice suffers from significant variations in the scale， shape， and texture of objects. Moreover， the high image resolution brings challenges to prediction efficiency. Therefore， CNN-based glomerulus segmentation plays an important role in clinical applications.
Method
2
This paper proposes a method for glomerular segmentation in full-stained kidney slices. A multi-granularity spatial attention mechanism is designed to deal with the diverse appearances of the glomerulus. This mechanism generates multiple scales and shape-changing feature maps for each pixel to focus on its context area instead of a fixed rectangular area as in traditional networks. For glomerulus with different sizes， the features of these feature maps should be fused at different scales， and the spatial information of features should be extracted by networks. Multi-granularities context feature maps are generated to pay attention to small objects using the context-based spatial attention mechanism， which can control the receptive field to obtain multi-granularities information and reduce background interference. The problem of high image resolution is addressed by cutting the original image into image patches. To detect the glomerulus located on the edge of two image patches， a padding strategy is formulated based on an augmented path. The disadvantages of the zero-padding strategy in standard convolution operation are then analyzed， and the contribution shifting effect is highlighted. The proposed padding strategy ensures that the boundary information of image patches is transferred to high levels of the network without information loss. Furthermore， given the very large resolution of the complete stained kidney slice， a window should be sliced along the image to predict objects. However， small objects are sensitive to the positions， thereby leading to different predictive probabilities. Sliding a window in an image also involves high computation complexity. To address these problems， this paper proposes a sliding window strategy that uses probability accumulation to fill those objects that are missing in stitching image patches. This strategy has high computation efficiency and can improve the detection accuracy of small objects in full-stained kidney slices.
Result
2
The proposed method achieves a higher segmentation accuracy on the mouse kidney cell and human biomolecular atlas program（HuBMAP） human kidney datasets compared with state-of-art methods. Specifically， the segmentation accuracy increases by 1% in Dice compared with U-Net after using a multi-granularities context-based spatial attention mechanism， and the number of missing and false objects is also reduced. The padding strategy based on an augmented path improves the predictive accuracy with only a few additional FLOPs（floating-point aperations per second）. The probabilistic cumulative strategy is also compared with the non-probabilistic cumulative sliding window strategy.
Result
2
show that the probabilistic cumulative sliding window strategy saves 52.83% of the time in the first layer of the network and 49.98% in the second layer compared with the non-probabilistic sliding window strategy. Overall， the proposed method increases the prediction speed by about 50%.
Conclusion
2
The probabilistic accumulation sliding window strategy improves prediction efficiency and glomerulus segmentation accuracy compared with the state-of-the-art methods. The proposed multi-granularity context spatial attention mechanism fuses the information of multiple scales through multi-granularity receptive fields to enhance the relevant features and suppress the irrelevant features of the glomerulus. The proposed padding strategy based on an augmented path can deal with information attenuation and contribution-shifting issues in traditional zero padding and effectively preserve information when objects are located on the boundary of patches. Combining multi-grained context features with the proposed padding strategy also improves object segmentation in fully stained kidney images. During network inference， the proposed sliding window with probability accumulation reuses features to significantly increase the prediction efficiency. This method is also beneficial in detecting small objects that are sensitive to the position. Experimental results on different datasets show that the proposed method outperforms the state-of-the-art methods and is both stable and robust. Meanwhile， the sliding window with probability accumulation improves segmentation accuracy and greatly reduces the calculation time. The role of local window size learning in the multi-granularity spatial attention mechanism will be explored in future work. In addition， given that some objects in a glomerulus slice are too small to predict， additional training samples must be generated using the generative adversarial network to further improve prediction accuracy.
卷积神经网络（CNN）医学图像分割全染色图像多粒度上下文特征补零
convolution neural network （CNN）medical image segmentationglomerular imagemulti-scale contextual featurepadding
Altini N， Cascarano G D， Brunetti A， Marino F， Rocchetti M T， Matino S， Venere U， Rossini M， Pesce F， Gesualdo L and Bevilacqua V. 2020. Semantic segmentation framework for glomeruli detection and classification in kidney histological sections. Electronics， 9（3）： #503 ［DOI： 10.3390/electronics9030503http://dx.doi.org/10.3390/electronics9030503］
Badrinarayanan V， Kendall A and Cipolla R. 2017. SegNet： a deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence， 39（12）： 2481-2495 ［DOI： 10.1109/TPAMI.2016.2644615http://dx.doi.org/10.1109/TPAMI.2016.2644615］
Barker J， Hoogi A， Depeursinge A and Rubin D L. 2016. Automated classification of brain tumor type in whole-slide digital pathology images using local representative tiles. Medical Image Analysis， 30： 60-71 ［DOI： 10.1016/j.media.2015.12.002http://dx.doi.org/10.1016/j.media.2015.12.002］
Chen L C， Papandreou G， Kokkinos I， Murphy K and Yuille A L. 2018a. DeepLab： semantic image segmentation with deep convolutional nets， atrous convolution， and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence， 40（4）： 834-848 ［DOI： 10.1109/TPAMI.2017.2699184http://dx.doi.org/10.1109/TPAMI.2017.2699184］
Chen L C， Zhu Y K， Papandreou G， Schroff F and Adam H. 2018b. Encoder-decoder with atrous separable convolution for semantic image segmentation//Proceedings of the 15th European Conference on Computer Vision. Munich， Germany： Springer： 833-851 ［DOI： 10.1007/978-3-030-01234-2_49http://dx.doi.org/10.1007/978-3-030-01234-2_49］
Ding H H， Jiang X D， Shuai B， Liu A Q and Wang G. 2019. Semantic correlation promoted shape-variant context for segmentation//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach， USA： IEEE： 8877-8886 ［DOI： 10.1109/CVPR.2019.00909http://dx.doi.org/10.1109/CVPR.2019.00909］
Feng S L， Zhao H M， Shi F， Cheng X N， Wang M， Ma Y H， Xiang D H， Zhu W F and Chen X J. 2020. CPFNet： context pyramid fusion network for medical image segmentation. IEEE Transactions on Medical Imaging， 39（10）： 3008-3018 ［DOI： 10.1109/TMI.2020.2983721http://dx.doi.org/10.1109/TMI.2020.2983721］
Gadermayr M， Eschweiler D， Jeevanesan A， Klinkhammer B M， Boor P and Merhof D. 2017. Segmenting renal whole slide images virtually without training data. Computers in Biology and Medicine， 90： 88-97 ［DOI： 10.1016/j.compbiomed.2017.09.014http://dx.doi.org/10.1016/j.compbiomed.2017.09.014］
Gadermayr M， Strauch M， Klinkhammer B M， Djudjaj S， Boor P and Merhof D. 2016. Domain adaptive classification for compensating variability in histopathological whole slide images//Proceedings of the 13th International Conference on Image Analysis and Recognition. Póvoa de Varzim， Portugal： Springer： 616-622 ［DOI： 10.1007/978-3-319-41501-7_69http://dx.doi.org/10.1007/978-3-319-41501-7_69］
Gu R， Wang G T， Song T， Huang T， Aertsen M， Deprest J， Ourselin S， Vercauteren T and Zhang S T. 2021. CA-Net： comprehensive attention convolutional neural networks for explainable medical image segmentation. IEEE Transactions on Medical Imaging， 40（2）： 699-711 ［DOI： 10.1109/TMI.2020.3035253http://dx.doi.org/10.1109/TMI.2020.3035253］
Han K， Wang Y H， Tian Q， Guo J Y， Xu C J and Xu C. 2020. GhostNet： more features from cheap operations//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle， USA： IEEE： 1577-1586 ［DOI： 10.1109/CVPR42600.2020.00165http://dx.doi.org/10.1109/CVPR42600.2020.00165］
Hervé N， Servais A， Thervet E， Olivo-Marin J C and Meas-Yedid V. 2011. Statistical color texture descriptors for histological images analysis//2011 IEEE International Symposium on Biomedical Imaging： From Nano to Macro. Chicago， USA： IEEE： 724-727 ［DOI： 10.1109/ISBI.2011.5872508http://dx.doi.org/10.1109/ISBI.2011.5872508］
Hou L， Samaras D， Kurc T M， Gao Y， Davis J E and Saltz J H. 2016. Patch-based convolutional neural network for whole slide tissue image classification//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 2424-2433 ［DOI： 10.1109/CVPR.2016.266http://dx.doi.org/10.1109/CVPR.2016.266］
Hu J， Shen L and Sun G. 2020. Squeeze-and-excitation networks//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City， USA： IEEE： 7132-7141 ［DOI： 10.1109/CVPR.2018.00745http://dx.doi.org/10.1109/CVPR.2018.00745］
HuBMAP Consortium. 2019. The human body at cellular resolution： the NIH human biomolecular atlas program. Nature， 574（7777）： 187-192 ［DOI： 10.1038/s41586-019-1629-xhttp://dx.doi.org/10.1038/s41586-019-1629-x］
Jha A， Yang H C， Deng R N， Kapp M E， Fogo A B and Huo Y K. 2021. Instance segmentation for whole slide imaging： end-to-end or detect-then-segment. Journal of Medical Imaging， 8（1）： #014001 ［DOI： 10.1117/1.JMI.8.1.014001http://dx.doi.org/10.1117/1.JMI.8.1.014001］
Kamnitsas K， Ledig C， Newcombe V F J， Simpson J P， Kane A D， Menon D K， Rueckert D and Glocker B. 2017. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Medical Image Analysis， 36： 61-78 ［DOI： 10.1016/j.media.2016.10.004http://dx.doi.org/10.1016/j.media.2016.10.004］
Kannan S， Morgan L A， Liang B， Cheung M G， Lin C Q， Mun D， Nader R G， Belghasem M E， Henderson J M， Francis J M， Chitalia V C and Kolachalama V B. 2019. Segmentation of glomeruli within trichrome images using deep learning. Kidney International Reports， 4（7）： 955-962 ［DOI： 10.1016/j.ekir.2019.04.008http://dx.doi.org/10.1016/j.ekir.2019.04.008］
Kingma D P and Ba J. 2017. Adam： a method for stochastic optimization ［EB/OL］. ［2022-10-25］. https://arxiv.org/pdf/1412.6980.pdfhttps://arxiv.org/pdf/1412.6980.pdf
Li Z Z， Poon K W and Yang X. 2021. RIAP： a method for effective receptive field rectification//Proceedings of the 30th International Conference on Artificial Neural Networks. Bratislava， Slovakia： Springer： 226-278 ［DOI： 10.1007/978-3-030-86380-7_22http://dx.doi.org/10.1007/978-3-030-86380-7_22］
Liu J J， Hou Q B， Cheng M M， Feng J S and Jiang J M. 2019. A simple pooling-based design for real-time salient object detection//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach， USA： IEEE： 3912-3921 ［DOI： 10.1109/CVPR.2019.00404http://dx.doi.org/10.1109/CVPR.2019.00404］
Long J， Shelhamer E and Darrell T. 2015. Fully convolutional networks for semantic segmentation//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston， USA： IEEE： 3431-3440 ［DOI： 10.1109/CVPR.2015.7298965http://dx.doi.org/10.1109/CVPR.2015.7298965］
Luo H L and Zhang Y. 2019. Semantic segmentation method with combined context features with CNN multi-layer features. Journal of Image and Graphics， 24（12）： 2200-2209
罗会兰， 张云. 2019. 结合上下文特征与CNN多层特征融合的语义分割. 中国图象图形学报， 24（12）： 2200-2209 ［DOI： 10.11834/jig.190087http://dx.doi.org/10.11834/jig.190087］
Nguyen T C， Nguyen T P， Diep G H， Tran-Dinh A H， Nguyen T V and Tran M T. 2021. CCBANet： cascading context and balancing attention for polyp segmentation//Proceedings of the 24th International Conference on Medical Image Computing and Computer-Assisted Intervention. Strasbourg， France： Springer： 633-643 ［DOI： 10.1007/978-3-030-87193-2_60http://dx.doi.org/10.1007/978-3-030-87193-2_60］
Pedraza A， Gallego J， Lopez S， Gonzalez L， Laurinavicius A and Bueno G. 2017. Glomerulus classification with convolutional neural networks//Proceedings of the 21st Annual Conference on Medical Image Understanding and Analysis. Edinburgh， UK： Springer： 839-849 ［DOI： 10.1007/978-3-319-60964-5_73http://dx.doi.org/10.1007/978-3-319-60964-5_73］
Ronneberger O， Fischer P and Brox T. 2015. U-Net： convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich， Germany： Springer： 234-241 ［DOI： 10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28］
Sarker M K， Rashwan H A， Akram F， Banu S F， Saleh A， Singh V K， Chowdhury F U H， Abdulwahab S， Romani S， Radeva P and Puig D. 2018. SLSDeep： skin lesion segmentation based on dilated residual and pyramid pooling networks//Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention. Granada， Spain： Springer： 21-29 ［DOI： 10.1007/978-3-030-00934-2_3http://dx.doi.org/10.1007/978-3-030-00934-2_3］
Sertel O， Kong J， Shimada H， Catalyurek U V， Saltz J H and Gurcan M N. 2009. Computer-aided prognosis of neuroblastoma on whole-slide images： classification of stromal development. Pattern Recognition， 42（6）： 1093-1103 ［DOI： 10.1016/j.patcog.2008.08.027http://dx.doi.org/10.1016/j.patcog.2008.08.027］
Shi Y G， Qian M Y and Liu Z W. 2017. Renal cortex segmentation with fully convolutional network and GrowCut. Journal of Image and Graphics， 22（10）： 1418-1427
时永刚， 钱梦瑶， 刘志文. 2017. 结合全卷积网络和GrowCut的肾皮质分割算法. 中国图象图形学报， 22（10）： 1418-1427 ［DOI： 10.11834/jig.170190http://dx.doi.org/10.11834/jig.170190］
Simonyan K and Zisserman K. 2015. Very deep convolutional networks for large-scale image recognition//Proceedings of the 3rd International Conference on Learning Representations. San Diego， USA： ICLR： 1-14 ［DOI： 10.48550/arxiv.1409.1556http://dx.doi.org/10.48550/arxiv.1409.1556］
Szegedy C， Ioffe S， Vanhoucke V and Alemi A. 2017. Inception-v4， inception-ResNet and the impact of residual connections on learning//Proceedings of the 31st AAAI Conference on Artificial Intelligence. San Francisco， USA： AAAI： 4278-4284 ［DOI： 10.1609/aaai.v31i1.11231http://dx.doi.org/10.1609/aaai.v31i1.11231］
Wang K， Liang S J and Zhang Y. 2021. Residual feedback network for breast lesion segmentation in ultrasound image//Proceedings of the 24th International Conference on Medical Image Computing and Computer-Assisted Intervention. Strasbourg， France： Springer： 471-481 ［DOI： 10.1007/978-3-030-87193-2_45http://dx.doi.org/10.1007/978-3-030-87193-2_45］
Zhang C， Shu H Z， Yang G Y， Li F Q， Wen Y G， Zhang Q， Dillenseger J L and Coatrieux J L. 2020. HIFUNet： multi-class segmentation of uterine regions from MR images using global convolutional networks for HIFU surgery planning. IEEE Transactions on Medical Imaging， 39（11）： 3309-3320 ［DOI： 10.1109/TMI.2020.2991266http://dx.doi.org/10.1109/TMI.2020.2991266］
Zhang Q， Cui Z P， Niu X G， Geng S J and Qiao Y. 2017. Image segmentation with pyramid dilated convolution based on ResNet and U-Net//Proceedings of the 24th International Conference on Neural Information Processing. Guangzhou， China： Springer： 364-372 ［DOI： 10.1007/978-3-319-70096-0_38http://dx.doi.org/10.1007/978-3-319-70096-0_38］
Zhao H S， Shi J P， Qi X J， Wang X G and Jia J Y. 2017. Pyramid scene parsing network//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu， USA： IEEE： 6230-6239 ［DOI： 10.1109/CVPR.2017.660http://dx.doi.org/10.1109/CVPR.2017.660］
相关文章
相关作者
相关机构
京公网安备11010802024621