浏览全部资源
扫码关注微信
Received：03 June 2020，
Revised：2020-7-19，
Accepted：26 July 2020，
Published：16 May 2021
移动端阅览
目的
2
青光眼会对人的视力造成不可逆的损伤，从眼底图像中精确地分割视盘和视杯是青光眼诊治中的一项重要工作，为有效提升视盘和视杯的分割精度，本文提出了融合上下文和注意力的视盘视杯分割方法（context attention U-Net，CA-Net）。
方法
2
进行极坐标转换，在极坐标系下进行分割可以平衡数据分布。使用修改的预训练ResNet作为特征提取网络，增强特征提取能力。采用上下文聚合模块（context aggregation module，CAM）多层次聚合图像上下文信息，使用注意力指导模块（attention guidance module，AGM）对融合后的特征图进行特征重标定，增强有用特征；使用深度监督思想同时对浅层网络权重进行训练，同时在视杯分割网络中引入了先验知识，约束对视杯的分割。
结果
2
在3个数据集上与其他方法进行对比实验，在Drishti-GS1数据集中，分割视盘的Dice（dice coefficient）和IOU（intersection-over-union）分别为0.981 4和0.963 5，分割视杯的Dice和IOU分别为0.926 6和0.863 3；在RIM-ONE（retinal image database for optic nerve evaluation）-v3数据集中，分割视盘的Dice和IOU分别为0.976 8和0.954 6，分割视杯的Dice和IOU分别为0.864 2和0.760 9；在Refuge数据集中，分割视盘的Dice和IOU分别为0.975 8和0.952 7，分割视杯的Dice和IOU分别为0.887 1和0.797 2，均优于对比算法。同时，消融实验验证了各模块的有效性，跨数据集实验进一步表明了CA-Net的泛化性，可视化图像也表明CA-Net能够分割出更接近标注的分割结果。
结论
2
在Drishti-GS1、RIM-ONE-v3和Refuge三个数据集的测试结果表明，CA-Net均能取得最优的视盘和视杯分割结果，跨数据集测试结果也更加表明了CA-Net具有良好的泛化性能。
Objective
2
Glaucoma can cause irreversible damage to vision. Glaucoma is often diagnosed on the basis of the cup-to-disc ratio (CDR). A CDR greater than 0.65 is considered to be glaucoma. Therefore
segmenting the optic disc (OD) and optic cup (OC) accurately from fundus images is an important task. The traditional methods used to segment the OD and OC are mainly based on deformable model
graph-cut
edge detection
and super pixel classification. Traditional methods need to manually extract image features
which are easily affected by light and contrast
and the segmentation accuracy is often low. In addition
these methods require careful adjustment of model parameters to achieve performance improvements and are not suitable for large-scale promotion. In recent years
with the development of deep learning
the segmentation methods of OD and OC based on the convolutional neural network (CNN)
which can automatically extract image features
have become the main research direction and have achieved better segmentation performance than traditional methods. OC segmentation is more difficult than OD segmentation because the OC boundary is not obvious. The existing methods based on CNN can be divided into two categories. One is joint segmentation; that is
the OD and OC can be segmented simultaneously using the same segmentation network. The other is two-stage segmentation; that is
the OD is segmented first
and then the OC is segmented. Previous studies have shown that the accuracy of joint segmentation is often inferior to that of two-stage segmentation
and joint segmentation can lead to biased optimization result to OD or OC. However
the connection between OD and OC is often ignored in two-stage segmentation. In this study
U-Net is improved
and a two-stage segmentation network called context attention U-Net (CA-Net) is proposed to segment the OD and OC sequentially. The prior knowledge is introduced into the OC segmentation network to further improve the OC segmentation accuracy.
Method
2
First
we locate the OD center and crop the region of interest (ROI) from the whole fundus image according to the OD center
which can reduce irrelevant regions. The size of the ROI image is 512×512. Then
the cropped ROI image is transferred by polar transformation from the Cartesian coordinate into the polar coordinate
which can balance disc and cup proportion. Because the OC region always accounts for a low proportion
it easily leads to overfitting and bias in training the deep model. Finally
the transferred images are fed into CA-Net to predict the final OD or OC segmentation maps. The OC is inside the OD
which means that the area that belongs to the OC also belongs to the OD. Specifically
we train two segmentation networks with the same structure to segment the OD and OC
respectively. To segment the OC more accurately
we utilize the connection between the OD and the OC as prior information. The modified pre-trained ResNet34 is used as the feature extraction network to enhance the feature extraction capability. Concretely
the first max pooling layer
the last average pooling layer
and the full connectivity layer are removed from the original ResNet34. Compared with training the deep learning model from scratch
loading pre-trained parameters on ImageNet (ImageNet Large-Scale Visual Recognition Challenge) helps prevent overfitting. Moreover
a context aggregation module (CAM) is proposed to aggregate the context information of images from multiple scales
which exploits the different sizes of atrous convolution to encode the rich semantic information. Because there will be a lot of irrelevant information during the fusion of shallow and deep feature maps
an attention guidance module (AGM) is proposed to recalibrate the feature maps after fusion of shallow and deep feature maps to enhance the useful feature information. In addition
the idea of deep supervision is also used to train the weights of shallow network. Finally
CA-Net outputs the probability map
and the largest connected region is selected as the final segmentation result to remove noise. We do not use any post-processing techniques such as ellipse fitting. DiceLoss is used as loss function to train CA-Net. We use a NVIDIA GeForce GTX 1080 Ti device to train and test the proposed CA-Net.
Result
2
We conducted experiments on three commonly used public datasets (Drishti-GS1
RIM-ONE-v3
and Refuge) to verify the effectiveness and generalization of CA-Net. We trained the model on the training set and reported the model performance on the test set. The Drishti-GS1 dataset was split into 50 training images and 51 test images. The RIM-ONE-v3 dataset was randomly split into 99 training images and 60 test images. The Refuge dataset was randomly split into 320 training images and 80 test images. Two measures were used to evaluate the results
namely
Dice coefficient (Dice) and intersection-over-union (IOU). For OD segmentation
the Dice and IOU obtained by CA-Net are 0.981 4 and 0.963 5 on the Drishti-GS1 dataset and 0.976 8 and 0.954 6 on the retinal image database for optic nerve evaluation(RIM-ONE)-v3 dataset
respectively. For OC segmentation
the Dice and IOU obtained by CA-Net are 0.926 6 and 0.863 3 on the Drishti-GS1 dataset and 0.864 2 and 0.760 9 on the RIM-ONE-v3 dataset
respectively. Moreover
CA-Net achieved a Dice of 0.975 8 and IOU of 0.952 7 in the case of OD segmentation and a Dice of 0.887 1 and IOU of 0.797 2 in the case of OC segmentation on the Refuge dataset
which further demonstrated the effectiveness and generalization of CA-Net. We also used the Refuge training dataset to train CA-Net and directly evaluated it on the Drishti-GS1 and RIM-ONE-v3 testing datasets. In addition
ablation experiments on the three datasets also showed the effectiveness of each module in the network
such as AGM
CAM
polar transformation
and deep supervision. The experiments also showed that CA-Net could achieve higher segmentation accuracy in the case of Dice and IOU when compared with U-Net
M-Net
and DeepLab v3+. The visual segmentation results also proved that CA-Net could achieve segmentation results more similar to ground truth.
Conclusion
2
This study presents a new two-stage segmentation method based on U-Net for OD and OC segmentation
which is proved to be effective. The experiments showed that CA-Net can obtain better results than other methods on the Drishti-GS1
RIM-ONE-v3
and Refuge datasets. In the future
we will focus on the problem of domain adaptation and solve the problem of OD and OC segmentation when the training samples are insufficient.
Al-Bander B, Williams B W, Al-Nuaimy W, Al-Taee M A, Pratt H and Zheng Y L. 2018. Dense fully convolutional segmentation of the optic disc and cup in color fundus for glaucoma diagnosis. Symmetry, 10(4): #87[DOI:10.3390/sym10040087]
Aquino A, Gegúndez-Arias M E and Marín D. 2010. Detecting the optic disc boundary in digital fundus images using morphological, edge detection, and feature extraction techniques. IEEE Transactions on Medical Imaging, 29(11): 1860-1869[DOI:10.1109/TMI.2010.2053042]
Biswal B, Vyshnavi E, Sairam M V S and Rout P K. 2020. Robust retinal optic disc and optic cup segmentation via stationary wavelet transform and maximum vessel pixel sum. IET Image Processing, 14(4): 592-602[DOI:10.1049/iet-ipr.2019.0845]
Chen L C, Zhu Y K, Papandreou G, Schroff F and Adam H. 2018. Encoder-decoder with atrous separable convolution for semantic image segmentation[EB/OL]. [2020-06-03] . https://arxiv.org/pdf/1802.02611.pdf https://arxiv.org/pdf/1802.02611.pdf
Cheng J, Liu J, Xu Y W, Yin F S, Wong D W K, Tan N M, Tao D C, Cheng C Y, Aung T and Wong T Y. 2013. Superpixel classification based optic disc and optic cup segmentation for glaucoma screening. IEEE Transactions on Medical Imaging, 32(6): 1019-1032[DOI:10.1109/TMI.2013.2247770]
Edupuganti V G, Chawla A and Kale A. 2018. Automatic optic disk and cup segmentation of fundus images using deep learning//Proceedings of the 25th IEEE International Conference on Image Processing. Athens, Greece: IEEE: 2227-2231[ DOI: 10.1109/ICIP.2018.8451753 http://dx.doi.org/10.1109/ICIP.2018.8451753 ]
Fu H Z, Cheng J, Xu Y W, Wong D W K, Liu J and Cao X C. 2018. Joint optic disc and cup segmentation based on multi-label deep network and polar transformation. IEEE Transactions on Medical Imaging, 37(7): 1597-1605[DOI:10.1109/TMI.2018.2791488]
Fumero F, Alayon S, Sanchez J L, Sigut J and Gonzalez-Hernandez M. 2011. RIM-ONE: an open retinal image database for optic nerve evaluation//The 24th International Symposium on Computer-Based Medical Systems (CBMS). Bristol, UK: IEEE: 1-6[ DOI: 10.1109/CBMS.2011.5999143 http://dx.doi.org/10.1109/CBMS.2011.5999143 ]
Gu Z W, Cheng J, Fu H Z, Zhou K, Hao H Y, Zhao Y T, Zhang T Y, Gao S H and Liu J. 2019. CE-Net: context encoder network for 2D medical image segmentation. IEEE Transactions on Medical Imaging, 38(10): 2281-2292[DOI:10.1109/TMI.2019.2903562]
He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 770-778[ DOI: 10.1109/CVPR.2016.90 http://dx.doi.org/10.1109/CVPR.2016.90 ]
Jiang Y, Tan N and Peng T T. 2019a. Optic disc and cup segmentation based on deep convolutional generative adversarial networks. IEEE Access, 7: 64483-64493[DOI:10.1109/ACCESS.2019.2917508]
Jiang Y M, Duan L X, Cheng Jun, Gu Z W, Xia H, Fu H Z, Li C S and Liu J. 2019b. JointRCNN: a region-based convolutional neural network for optic disc and cup segmentation. IEEE Transactions on Biomedical Engineering, 67(2): 335-343[DOI:10.1109/TBME.2019.2913211]
Jiang Y M, Xia H, Xu Y W, Cheng J, Fu H Z, Duan L X, Meng Z G and Liu J. 2018. Optic disc and cup segmentation with blood vessel removal from fundus images for glaucoma detection//Proceedings of the 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Honolulu, USA: IEEE: 862-865[ DOI: 10.1109/EMBC.2018.8512400 http://dx.doi.org/10.1109/EMBC.2018.8512400 ]
Joshua A O, Nelwamondo F V and Mabuza-Hocquet G. 2019. Segmentation of optic cup and disc for diagnosis of glaucoma on retinal fundus images//Proceedings of 2019 Southern African Universities Power Engineering Conference/Robotics and Mechatronics/Pattern Recognition Association of South Africa. Bloemfontein, South Africa: IEEE: 183-187[ DOI: 10.1109/RoboMech.2019.8704727 http://dx.doi.org/10.1109/RoboMech.2019.8704727 ]
Kim J, Tran L, Chew E Y and Antani S. 2019. Optic disc and cup segmentation for glaucoma characterization using deep learning//The 32nd IEEE International Symposium on Computer-Based Medical Systems. Cordoba, Spain: IEEE: 489-494[ DOI: 10.1109/CBMS.2019.00100 http://dx.doi.org/10.1109/CBMS.2019.00100 ]
Liu Q, Hong X P, Li S, Chen Z L, Zhao G Y and Zou B J. 2019. A spatial-aware joint optic disc and cup segmentation method. Neurocomputing, 359: 285-297[DOI:10.1016/j.neucom.2019.05.039]
Long J, Shelhamer E and Darrell T. 2015. Fully convolutional networks for semantic segmentation//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE: 3431-3440[ DOI: 10.1109/CVPR.2015.7298965 http://dx.doi.org/10.1109/CVPR.2015.7298965 ]
Lowell J, Hunter A, Steel D, Basu A, Ryder R, Fletcher E and Kennedy L. 2004. Optic nerve head segmentation. IEEE Transactions on Medical Imaging, 23(2): 256-264[DOI:10.1109/TMI.2003.823261]
Lu S J. 2011. Accurate and efficient optic disc detection and segmentation by a circular transformation. IEEE Transactions on Medical Imaging, 30(2): 2126-2133[DOI:10.1109/TMI.2011.2164261]
Lu Z, Chen D L and Xue D Y. 2019. Algorithm for size-constrained weakly supervised optic disc segmentation of fundus images. Journal of Image and Graphics, 24(5): 827-835
鲁正, 陈大力, 薛定宇. 2019. 带尺寸约束的弱监督眼底图像视盘分割. 中国图象图形学报, 24(5): 827-835[DOI:10.11834/jig.180487]
Maninis K K, Pont-Tuset J, Arbeláez P and Van Gool L. 2016. Deep retinal image understanding//Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Athens, Greece: Springer: 140-148[ DOI: 10.1007/978-3-319-46723-8_17 http://dx.doi.org/10.1007/978-3-319-46723-8_17 ]
Milletari F, Navab N and Ahmadi S A. 2016. V-Net: fully convolutional neural networks for volumetric medical image segmentation//Proceedings of the 4th International Conference on 3D Vision. Stanford, USA: IEEE: 565-571[ DOI: 10.1109/3DV.2016.79 http://dx.doi.org/10.1109/3DV.2016.79 ]
Mohan D, Kumar J R H and Seelamantula C S. 2019. Optic disc segmentation using cascaded multiresolution convolutional neural networks//Proceedings of 2019 IEEE International Conference on Image Processing. Taipei, China: IEEE: 834-838[ DOI: 10.1109/ICIP.2019.8804267 http://dx.doi.org/10.1109/ICIP.2019.8804267 ]
Murugesan B, Sarveswaran K, Shankaranarayana S M, Ram K, Joseph J and Sivaprakasam M. 2019. Psi-Net: shape and boundary aware joint multi-task deep network for medical image segmentation//Proceedings of the 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Berlin, Germany: IEEE: 7223-7226[ DOI: 10.1109/EMBC.2019.8857339 http://dx.doi.org/10.1109/EMBC.2019.8857339 ]
Orlando J I, Fu H Z, Breda J B, van Keer K, Bathula D R, Diaz-Pinto A, Fang R G, Heng P A, Kim J, Lee J, Lee J, Li X X, Liu P, Lu S, Murugesan B, Naranjo V, Phaye S S R, Shankaranarayana S M, Sikka A, Son J, van den Hengel A, Wang S J, Wu J Y, Wu Z F, Xu G H, Xu Y L, Yin P S, Li F, Zhang Z L, Xu Y W and Bogunović H. 2020. REFUGE Challenge: a unified framework for evaluating automated methods for glaucoma assessment from fundus photographs. Medical Image Analysis, 59: #101570[DOI:10.1016/j.media.2019.101570]
Ronneberger O, Fischer P and Brox T. 2015. U-Net: convolutional networks for biomedical image segmentation//Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: Springer: 234-241[ DOI: 10.1007/978-3-319-24574-4_28 http://dx.doi.org/10.1007/978-3-319-24574-4_28 ]
Sevastopolsky A. 2017. Optic disc and cup segmentation methods for glaucoma detection with modification of U-Net convolutional neural network. Pattern Recognition and Image Analysis, 27(3): 618-624[DOI:10.1134/S1054661817030269]
Shankaranarayana S M, Ram K, Mitra K and Sivaprakasam M. 2017. Joint optic disc and cup segmentation using fully convolutional and adversarial networks//Proceedings of the International Workshop on Ophthalmic Medical Image Analysis. Québec City, Canada: Springer: 168-176[ DOI: 10.1007/978-3-319-67561-9_19 http://dx.doi.org/10.1007/978-3-319-67561-9_19 ]
Shankaranarayana S M, Ram K, Mitra K and Sivaprakasam M. 2019. Fully convolutional networks for monocular retinal depth estimation and optic disc-cup segmentation. IEEE Journal of Biomedical and Health Informatics, 23(4): 1417-1426[DOI:10.1109/JBHI.2019.2899403]
Sivaswamy J, Krishnadas S, Chakravarty A and Ujjwal J G D. 2015. A comprehensive retinal image dataset for the assessment of glaucoma from the optic nerve head analysis. JSM Biomedical Imaging Data Papers, 2(1): 1004-1012[DOI:10.1109/ISBI.2014.6867807]
Wang S J, Yu, L Q, Yang X, Fu C W and Heng P A. 2019a. Patch-based output space adversarial learning for joint optic disc and cup segmentation. IEEE Transactions on Medical Imaging, 38(11): 2485-2495[DOI:10.1109/TMI.2019.2899910]
Wang Z Y, Dong N Q, Rosario S D, Xu M, Xie P T and Xing E P. 2019b. Ellipse detection of optic disc-and-cup boundary in fundus images//The 16th IEEE International Symposium on Biomedical Imaging. Venice, Italy: IEEE: 601-604[ DOI: 10.1109/ISBI.2019.8759173 http://dx.doi.org/10.1109/ISBI.2019.8759173 ]
Wong D W K, Liu J, Lim J H, Jia X, Yin F, Li H and Wong T Y. 2008. Level-set based automatic cup-to-disc ratio determination using retinal fundus images in ARGALI//Proceedings of the 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Vancouver, Canada: IEEE: 2266-2269[ DOI: 10.1109/IEMBS.2008.4649648 http://dx.doi.org/10.1109/IEMBS.2008.4649648 ]
Wong D W K, Liu J, Lim J H, Li H and Wong T Y. 2009. Automated detection of kinks from blood vessels for optic cup segmentation in retinal images//Proceedings Volume 7260, Medical Imaging 2009: Computer-Aided Diagnosis. Florida, USA: SPIE: 7260-7268[ DOI: 10.1117/12.810784 http://dx.doi.org/10.1117/12.810784 ]
Xie Z, Ling T H, Yang Y Y, Shu R and Liu B J. 2020. Optic disc and cup image segmentation utilizing contour-based transformation and sequence labeling networks. Journal of Medical Systems, 44(5): #96[DOI:10.1007/s10916-020-01561-2]
Xu Y L, Lu S, Li H X and Li R R. 2019. Mixed maximum loss design for optic disc and optic cup segmentation with deep learning from imbalanced samples. Sensors (Basel), 19(20): #4401[DOI:10.3390/s19204401]
Yu F and Koltun V. 2016. Multi-scale context aggregation by dilated convolutions[EB/OL]. [2020-06-03] . https://arxiv.org/pdf/1511.07122.pdf https://arxiv.org/pdf/1511.07122.pdf
Yu S, Xiao D, Frost S and Kanagasingam Y. 2019. Robust optic disc and cup segmentation with deep learning for glaucoma detection. Computerized Medical Imaging and Graphics, 74: 61-71[DOI:10.1016/j.compmedimag.2019.02.005]
Zahoor M N and Fraz M M. 2017. Fast optic disc segmentation in retina using polar transform. IEEE Access, 5: 12293-12300[DOI:10.1109/ACCESS.2017.2723320]
Zhang Z J, Fu H Z, Dai H, Shen J B, Pang Y W and Shao L. 2019. ET-Net: a generic edge-aTtention guidance network for medical image segmentation//Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen, China: Springer: 442-450[ DOI: 10.1007/978-3-030-32239-7_49 http://dx.doi.org/10.1007/978-3-030-32239-7_49 ]
相关作者
相关机构
京公网安备11010802024621
