AS-PANet: a chromosome instance segmentation method based on improved path aggregation network architecture

Chengchuang Lin; Gansen Zhao; Aihua Yin; Bichao Ding; Li Guo; Hanbiao Chen

doi:10.11834/jig.200236

Computerd Tomography Image | Views : 0 下载量: 112 CSCD: 9

PDF
Export
Share
Collection
Album

AS-PANet: a chromosome instance segmentation method based on improved path aggregation network architecture
Vol. 25, Issue 10, Pages: 2271-2280(2020)
Received：31 May 2020，

Revised：2020-7-6，

Accepted：13 July 2020，

Published：2020-10
DOI： 10.11834/jig.200236
稿件说明：

移动端阅览

DOI：

Chengchuang Lin, Gansen Zhao, Aihua Yin, Bichao Ding, Li Guo, Hanbiao Chen. AS-PANet: a chromosome instance segmentation method based on improved path aggregation network architecture[J]. Journal of Image and Graphics, 2020, 25(10): 2271-2280. DOI： 10.11834/jig.200236.

摘要

目的

染色体是遗传信息的重要载体，健康的人体细胞中包含46条染色体，包括22对常染色体和1对性染色体。染色体核型化分析是产前诊断和遗产疾病诊断的重要且常用方法。染色体核型化分析是指从分裂中期的细胞显微镜图像中，分割出染色体并根据染色体的条带进行分组排列的过程。染色体核型化分析通常由细胞学家手工完成，但是这个过程非常费时、繁琐且容易出错。由于染色体的非刚性特质，多条染色体之间存在重叠及交叉现象，致使染色体实例分割非常困难。染色体分割是染色体核型化分析过程中最重要且最困难的一步，因此本文旨在解决重叠、交叉染色体实例分割问题。

方法

本文基于路径增强网络（PANet）模型，提出AS-PANet（amount segmentation PANet）模型用于解决重叠染色体实例分割问题。在路径增强网络的基础上引入染色体计数领域知识预测作为模型的一个预测分支，并改进了路径增强网络的模型结构和损失函数，使图像分类、目标检测、实例分割和染色体计数4个子任务共享卷积特征，进行联合训练。在临床染色体图像数据上进行标注并构建训练集和测试集，同时提出有效的数据增广方法用以扩充染色体标注训练数据集，提升模型的训练效果。

结果

在临床染色体数据集中开展实证研究实验。实验结果表明，本文方法在临床染色体数据集中，平均分割精度mAP（mean average precision）为90.63%。该结果比PANet提升了1.18%，比基线模型Mask R-CNN提升了2.85%。分割准确率为85%，相比PANet提升了2%，相比Mask R-CNN（region with convolutional neural network）提升3.75%。

结论

本文染色体实例分割方法能够更有效地解决临床染色体分割问题，相比现有的方法，分割效果更好。

Abstract

Objective

Twenty-three pairs of chromosomes

including 22 pairs of autosomes and a pair of sex chromosomes

are found in the cells of a healthy human body. As those chromosomes are vital genetic information carriers

karyotyping chromosomes is important for medical diagnostics

drug development

and biomedical research. Chromosome karyotyping analysis refers to segmenting chromosome instances from a stained cell microphotograph and arranging them into karyotype in accordance with their band criterion. However

chromosome karyotyping analysis is usually performed by skilled cytologists manually

which requires extensive experience

domain expertise

and considerable manual efforts. In this study

we focus on the segmentation of chromosome instances because it is a crucial and challenging obstacle of chromosome karyotyping.

Method

In this study

we propose the AS-PANet(amount segmentation PANet) method by improving the PANet instance segmentation model for the segmentation task of chromosome instances. We add a chromosome counting task into AS-PANet for joint training

thereby enabling the classification

detection

segmentation

and counting tasks to share latent features. We propose a chromosome label augmentation algorithm for augmenting the training dataset.

Result

We collect clinical metaphase cell microphotograph images from Guangdong Women and Children Hospital. Then

we build a clinical dataset by labeling 802 chromosome clusters. The clinical dataset is split into training and testing datasets with a ratio of 8 :2. The proposed augmentation algorithm augments the training dataset from 802 samples into 13 482 samples. Our solution achieves a more competitive result on the clinical chromosome dataset with 90.63% mean average precision (mAP)

whereas PANet yields 89.45% mAP and Mask R-CNN(regirn with convolutional neural network) obtains 87.78% mAP at the equal experimental condition. The AS-PANet method yields 85% instance segmentation accuracy

which is 2% higher than the result of PANet and 3.75% higher than the result of Mask R-CNN.

Conclusion

Experimental results demonstrate that the proposed method is an effective and promising solution for solving the segmentation problem of clinical chromosome instances. The contributions and highlights of this study are summarized as follows: 1) we proposed a novel chromosome instance segmentation method by improving the path aggregation network architecture. 2) We built a clinical dataset for training and verifying the proposed method. 3) We demonstrated the effectiveness of the proposed method for tackling the chromosome instance segmentation task. Experimental results showed that the proposed method is more promising than previous studies. 4) The highlights of this work are high chromosome segmentation certainty and accuracy with a small amount of manual labeling cost.

关键词

Keywords

references

Arora T and Dhir R. 2019. A novel approach for segmentation of human metaphase chromosome images using region based active contours. The International Arab Journal of Information Technology, 16(1):132-137

Feng T, Chen B and Zhang Y F. 2020. Chromosome segmentation framework based on improved mask region-based convolutional neural network[EB/OL].[2020-05-01] . https://www.cnki.net/KCMS/detail/51.1307.TP.20200609.0949.002.html https://www.cnki.net/KCMS/detail/51.1307.TP.20200609.0949.002.html

冯涛, 陈斌, 张跃飞. 2020.基于改进的Mask R-CNN的染色体图像分割框架[EB/OL].[2020-06-09] . https://www.cnki.net/KCMS/detail/51.1307.TP.20200609.0949.002.html https://www.cnki.net/KCMS/detail/51.1307.TP.20200609.0949.002.html

Girshick R. 2015. Fast R-CNN//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE: 1440-1448[ DOI:10.1109/ICCV.2015.169 http://dx.doi.org/10.1109/ICCV.2015.169 ]

Grisan E, Poletti E and Ruggeri A. 2009. Automatic segmentation and disentangling of chromosomes in Q-band prometaphase images. IEEE Transactions on Information Technology in Biomedicine, 13(4):575-581[DOI:10.1109/TITB.2009.20144 64]

He K M, Gkioxari G, Dollár P and Girshick R. 2017. Mask R-CNN//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE: 2980-2988[ DOI:10.1109/ICCV.2017.322 http://dx.doi.org/10.1109/ICCV.2017.322 ]

He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE: 770-778[ DOI:10.1109/CVPR.2016.90 http://dx.doi.org/10.1109/CVPR.2016.90 ]

Hu R L, Karnowski J, Fadely R and Pommier J P. 2017. Image segmentation to distinguish between overlapping human chromosomes[EB/OL].[2020-05-01] . https://arxiv.org/pdf/1712.07639v1.pdf https://arxiv.org/pdf/1712.07639v1.pdf

Kirillov A, Wu Y X, He K M and Girshick R. 2019. PointRend: image segmentation as rendering[EB/OL].[2020-05-01] . https://arxiv.org/pdf/1912.08193.pdf https://arxiv.org/pdf/1912.08193.pdf

Kong T, Yao A B, Chen Y R and Sun F C. 2016. HyperNet: towards accurate region proposal generation and joint object detection//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE: 845-853[ DOI:10.1109/CVPR.2016.98 http://dx.doi.org/10.1109/CVPR.2016.98 ]

Li K, Xie N, Li X and Tan K. 2020. Statistical karyotype analysis using CNN and geometric optimization. Journal of Nanjing University (Natural Science), 56(1):116-124

李康, 谢宁, 李旭, 谭凯. 2020.基于卷积神经网络和几何优化的统计染色体核型分析方法.南京大学学报(自然科学), 56(1):116-124)[DOI:10.13232/j.cnki.jnju.2020.01.013]

Lin C C, Zhao G S, Yang Z R, Yin A H, Wang X M, Guo L, Chen H B, Ma Z H, Zhao L, Luo H Y, Wang T X, Ding B C, Pang X W and Chen Q R. 2020. CIR-Net: automatic classification of human chromosome based on inception-ResNet architecture.[EB/OL].[2020-05-01] . https://ieeexplore.ieee.org/abstract/docurment/9120197 https://ieeexplore.ieee.org/abstract/docurment/9120197 [ DOI:10.1109/TCBB.2020.3003445 http://dx.doi.org/10.1109/TCBB.2020.3003445 ].

Lin T Y, Dollár P, Girshick R, He K M, Hariharan B and Belongie S. 2017. Feature pyramid networks for object detection//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 936-944[ DOI:10.1109/CVPR.2017.106 http://dx.doi.org/10.1109/CVPR.2017.106 ]

Liu S, Qi L, Qin H F, Shi J P and Jia J Y. 2018. Path aggregation network for instance segmentation//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 8759-8768[ DOI:10.1109/CVPR.2018.00913 http://dx.doi.org/10.1109/CVPR.2018.00913 ]

Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C Y and Berg A C. 2016. SSD: single shot MultiBox detector//Proceedings of the 14th European Conference on Computer Vision. Amsterdam: Springer: 21-37[ DOI:10.1007/978-3-319-46448-0_2 http://dx.doi.org/10.1007/978-3-319-46448-0_2 ]

Minaee S, Fotouhi M and Khalaj B H. 2014. A geometric approach to fully automatic chromosome segmentation//Proceedings of 2014 IEEE Signal Processing in Medicine and Biology Symposium. Philadelphia: IEEE: 1-6[ DOI:10.1109/SPMB.2014.7163174 http://dx.doi.org/10.1109/SPMB.2014.7163174 ]

Neubeck A and Van Gool L. 2006. Efficient non-maximum suppression//Proceedings of the 18th International Conference on Pattern Recognition. Hong Kong, China: IEEE: 850-855[ DOI:10.1109/ICPR.2006.479 http://dx.doi.org/10.1109/ICPR.2006.479 ]

Ronneberger O, Fischer P and Brox T. 2015. U-Net: convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: Springer: 234-241[ DOI:10.1007/978-3-319-24574-4_28 http://dx.doi.org/10.1007/978-3-319-24574-4_28 ]

Shaffer L G, McGowan-Jordan J and Schmid M. 2013. ISCN 2013: An International System for Human Cytogenetic Nomenclature. Basel: Karger Medical and Scientific Publishers

You Y, Gitman I and Ginsburg B. 2017. Large batch training of convolutional networks[EB/OL].[2020-05-01] . https://arxiv.org/pdf/1708.03888.pdf https://arxiv.org/pdf/1708.03888.pdf

Alert me when the article has been cited

提交

Enhanced attention-based joint semantic instance segmentation network for point clouds

Multi-scale fusion-enhanced ultrasound elastic images segmentation for mediastinal lymph node

Review of deep learning methods for karyotype analysis

Semantic segmentation and model matching-integrated indoor scenario-relevant reconstruction method

The improved atrous spatial pyramid pooling and polarized self-attention based bottom-up panoptic segmentation