浏览全部资源
扫码关注微信
Received：19 May 2020，
Revised：2020-7-6，
Accepted：13 July 2020，
Published：2020-10
移动端阅览
目的
2
眼底图像中的动静脉分类是许多系统性疾病风险评估的基础步骤。基于传统机器学习的方法操作复杂，且往往依赖于血管提取的结果，不能实现端到端的动静脉分类，而深度语义分割技术的发展使得端到端的动静脉分类成为可能。本文结合深度学习强大的特征提取能力，以提升动静脉分类精度为目的，提出了一种基于语义融合的动静脉分割模型SFU-Net（semantic fusion based U-Net）。
方法
2
针对动静脉分类任务的特殊性，本文采用多标签学习的策略来处理该问题，以降低优化难度。针对动静脉特征的高度相似性，本文以DenseNet-121作为SFU-Net的特征提取器，并提出了语义融合模块以增强特征的判别能力。语义融合模块包含特征融合和通道注意力机制两个操作：1）融合不同尺度的语义特征从而得到更具有判别能力的特征；2）自动筛选出对目标任务更加重要的特征，从而提升性能。针对眼底图像中血管与背景像素之间分布不均衡的问题，本文以focal loss作为目标函数，在解决类别不均衡问题的同时重点优化困难样本。
结果
2
实验结果表明，本文方法的动静脉分类的性能优于现有绝大多数方法。本文方法在DRIVE（digital retinal images for vessel extraction）数据集上的灵敏性（sensitivity）与目前最优方法相比仅有0.61%的差距，特异性（specificity）、准确率（accuracy）和平衡准确率（balanced-accuracy）与目前最优方法相比分别提高了4.25%，2.68%和1.82%；在WIDE数据集上的准确率与目前最优方法相比提升了6.18%。
结论
2
语义融合模块能够有效利用多尺度特征并自动做出特征选择，从而提升性能。本文提出的SFU-Net在动静脉分类任务中表现优异，性能超越了现有绝大多数方法。
Objective
2
Arteriovenous (A/V) classification in fundus images is a fundamental step for the risk assessment of many systemic diseases. A/V classification methods based on traditional machine learning require complicated feature engineering
consistently rely on the results of blood vessel extraction
and cannot achieve end-to-end A/V classification. The development of deep semantic segmentation technology makes end-to-end A/V classification possible
and has been commonly used in fundus image analysis. In this paper
a segmentation model semantic fusion based U-Net (SFU-Net) is proposed combined with the powerful feature extraction capabilities of deep learning to improve the accuracy of A/V classification.
Method
2
First
the arteries and veins in the fundus image belong to blood vessels and are highly similar in structure. Existing deep learning-based A/V classification methods frequently treat this problem as a multiclassification problem. This paper proposes a multilabel learning strategy to address this problem for reducing the difficulty of optimization and deal with the situation where the arteries and veins in the fundus image cross. The lower layers of the network are mainly responsible for extracting the common features of the two structures. The upper layers of the network learn two binary classifiers and extract the arteries and veins independently. Second
considering the high similarity of the description features of arteries and veins in color and structure
this paper improves the U-Net architecture in two aspects. 1) The original simple feature extractor of U-Net is replaced by DenseNet-121. The original U-Net encoder is composed of 10 convolutional layers and four maximum pooling layers
and the feature extraction capability is extremely limited. By contrast
DenseNet-121 has many convolutional layers
and the introduction of dense connections makes the feature utilization rate high
the transmission efficiency of features and gradients in the network is high
and the feature extraction ability is strong. This paper reduces four downsampling operations of U-Net to three
and the input image is downsampled by eight times to avoid the loss of detailed information. 2) A semantic fusion module is proposed. The semantic fusion module includes two operations
namely
feature fusion and channelwise attention mechanism. Low-level features have high resolution and contain many location and detail information
but few semantic features and many noises. High-level features have strong semantic information
but their resolution is extremely low and the detail information is few. The features from different layers are first fused to enhance their distinguishing ability. For the fused features
the channelwise attention mechanism is used to select the features. The convolution filter can only capture local information. The global average pooling operation is performed on the input features in the channel dimension to capture global context information. Each element of the resulting vector is a concentrated representation of its corresponding channel. Two nonlinear transformations are then performed on the vector to model the correlation between channels and reduce the amount of parameters and calculations. The vector is restored to its original dimension and normalized to 0-1 through the sigmoid gate. Each element in the obtained vector is regarded as the importance of each channel in the input feature
and each feature channel of the input feature is weighted through a multiplication operation. Through the channel attention mechanism
the network can automatically focus on the feature channels that are important to the task while suppressing the features that are unimportant
thereby improving the performance of the model during the training process. Third
considering the problem of uneven distribution between blood vessels and background pixels in the fundus image
this paper takes focal loss as the loss function to solve the problem of class imbalance and focus on difficult samples at the same time. Focal loss introduces parameters
α
and
γ
in the cross-entropy loss function. Parameter
α
is used to balance the difference between positive and negative samples. Parameter
γ
adjusts the degree where the loss of simple samples is reduced
thereby amplifying the difference between the loss values of difficult and simple samples. The values of the two parameters are determined through cross-validation. The overall optimization goal is the sum of the focal loss of arteries and veins
thereby optimizing the arteries and veins during training.
Result
2
The proposed method is verified on two public datasets
namely
digital retinal images for vessel extraction(DRIVE) and WIDE
and its performance is evaluated from two perspectives
namely
segmentation and classification. Experimental results demonstrate that the proposed method shows better performance than most existing methods. The proposed method achieves an area under the curve of 0.968 6 and 0.973 6 for segmenting arteries and veins on the DRIVE dataset
and the sensitivity
specificity
accuracy and balanced-accuracy of A/V classification are 88.39%
94.25%
91.68%
and 91.32%
respectively. Compared with state-of-the-art method
the sensitivity of the proposed method only decreases by 0.61%
and specificity
accuracy
and balaned-auuracy have absolute improvements of 4.25%
2.68%
and 1.82%
respectively. The proposed method achieves an accuracy of 92.38%
which is 6.18% higher than the state-of-the-art method.
Conclusion
2
The fusion module can effectively use multi-scale features and automatically select many important features
thereby improving performance. The proposed method performs well in A/V classification
exceeding most existing methods.
Abràmoff M D, Garvin M K and Sonka M. 2010. Retinal imaging and image analysis. IEEE Reviews in Biomedical Engineering, 3:169-208[DOI:10.1109/RBME.2010.2084567]
Chen L C, Zhu Y K, Papandreou G, Schroff F and Adam H. 2018. Encoder-decoder with atrous separable convolution for semantic image segmentation//Proceedings of the 15th European Conference on Computer Vision (ECCV). Munich: Springer: 801-818[ DOI:10.1007/978-3-030-01234-2_49 http://dx.doi.org/10.1007/978-3-030-01234-2_49 ]
Cheung C Y L, Ikram M K, Chen C and Wong T Y. 2017. Imaging retina to study dementia and stroke. Progress in Retinal and Eye Research, 57:89-107[DOI:10.1016/j.preteyeres.2017.01.001]
Dashtbozorg B, Mendonça A M and Campilho A. 2014. An automatic graph-based approach for artery/vein classification in retinal images. IEEE Transactions on Image Processing, 23(3):1073-1083[DOI:10.1109/TIP.2013.2263809]
Estrada R, Allingham M J, Mettu P S, Cousins S W, Tomasi C and Farsiu S. 2015a. Retinal artery-vein classification via topology estimation. IEEE Transactions on Medical Imaging, 34(12):2518-2534[DOI:10.1109/TMI.2015.2443117]
Estrada R, Tomasi C, Schmidler S C and Farsiu S. 2015b. Tree topology estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(8):1688-1701[DOI:10.1109/TPAMI.2014.2382116]
Galdran A, Meyer M, Costa P, Mendonça and Campilho A. 2019. Uncertainty-aware artery/vein classification on retinal images//2019 IEEE 16th International Symposium on Biomedical Imaging. Venice, Italy: IEEE: 556-560[ DOI:10.1109/ISBI.2019.8759380 http://dx.doi.org/10.1109/ISBI.2019.8759380 ]
Girard F, Kavalec C and Cheriet F. 2019. Joint segmentation and classification of retinal arteries/veins from fundus images. Artificial Intelligence in Medicine, 94:96-109[DOI:10.1016/j.artmed.2019.02.004]
Grisan E and Ruggeri A. 2003. A divide et impera strategy for automatic classification of retinal vessels into arteries and veins//Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Cancun, Mexico: IEEE: 890-893[ DOI:10.1109/IEMBS.2003.1279908 http://dx.doi.org/10.1109/IEMBS.2003.1279908 ]
Hemelings R, Elen B, Stalmans I, Van Keer K, De Boever P and Blaschko M B. 2019. Artery-vein segmentation in fundus images using a fully convolutional network. Computerized Medical Imaging and Graphics, 76:#101636[DOI:10.1016/j.compmedimag.2019.05.004]
Hu J, Shen L and Sun G. 2018. Squeeze-and-excitation networks//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE: 7132-7141[ DOI:10.1109/CVPR.2018.00745 http://dx.doi.org/10.1109/CVPR.2018.00745 ]
Hu Q, Abràmoff M D and Garvin M K. 2015. Automated construction of arterial and venous trees in retinal images. Journal of Medical Imaging, 2(4):#044001[DOI:10.1117/1.JMI.2.4.044001]
Huang F, Dashtbozorg B, Tan T and ter Haar Romeny B M. 2018. Retinal artery/vein classification using genetic-search feature selection. Computer Methods and Programs in Biomedicine, 161:197-207[DOI:10.1016/j.cmpb.2018.04.016]
Huang G, Liu Z, Van Der Maaten L and Weinberger K Q. 2017. Densely connected convolutional networks//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE: 4700-4708[ DOI:10.1109/CVPR.2017.243 http://dx.doi.org/10.1109/CVPR.2017.243 ]
Ikram M K, de Jong F J, Vingerling J R, Witteman J C, Hofman A, Breteler M M B and de Jong P T V M. 2004. Are retinal arteriolar or venular diameters associated with markers for cardiovascular disorders? The rotterdam study. Investigative Ophthalmology and Visual Science, 45(7):2129-2134[DOI:10.1167/iovs.03-1390]
Joshi V S, Reinhardt J M, Garvin M K and Abramoff M D. 2014. Automated method for identification and artery-venous classification of vessel trees in retinal vessel networks. PLoS One, 9(2):e88061[DOI:10.1371/journal.pone.0088061]
LeCun Y, Bengio Y and Hinton G. 2015. Deep learning. Nature, 521(7553):436-444[DOI:10.1038/nature14539]
Liew G and Wang J J. 2011. Retinal vascular signs:a window to the heartñ. Revista Española de Cardiología (English Edition), 64(6):515-521[DOI:10.1016/j.rec.2011.02.017]
Ma Z Y. 2015. An Automatic Method for the Artery/Vein Classification in Retinal Images. Beijing: Beijing Institute of Technology
马志扬. 2015.基于彩色眼底图像的视网膜血管动静脉分类研究.北京: 北京理工大学
Mirsharif Q, Tajeripour F and Pourreza H. 2013. Automated characterization of blood vessels as arteries and veins in retinal images. Computerized Medical Imaging and Graphics, 37(7/8):607-617[DOI:10.1016/j.compmedimag.2013.06.003]
Muramatsu C, Hatanaka Y, Iwase T, Hara T and Fujita H. 2011. Automated selection of major arteries and veins for measurement of arteriolar-to-venular diameter ratio on retinal fundus images. Computerized Medical Imaging and Graphics, 35(6):472-480[DOI:10.1016/j.compmedimag.2011.03.002]
Niemeijer M, Xu X Y, Dumitrescu A V, Gupta P, Van Ginneken B, Folk J C and Abramoff M D. 2011. Automated measurement of the arteriolar-to-venular width ratio in digital color fundus photographs. IEEE Transactions on Medical Imaging, 30(11):1941-1950[DOI:10.1109/TMI.2011.2159619]
Pellegrini E, Robertson G, MacGillivray T, van Hemert J, Houston G and Trucco E. 2018. A graph cut approach to artery/vein classification in ultra-widefield scanning laser ophthalmoscopy. IEEE Transactions on Medical Imaging, 37(2):516-526[DOI:10.1109/TMI.2017.2762963]
Ronneberger O, Fischer P and Brox T. 2015. U-net: convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: Springer: 234-241[ DOI:10.1007/978-3-319-24574-4_28 http://dx.doi.org/10.1007/978-3-319-24574-4_28 ]
Rothaus K, Jiang X Y and Rhiem P. 2009. Separation of the retinal vascular graph in arteries and veins based upon structural knowledge. Image and Vision Computing, 27(7):864-875[DOI:10.1016/j.imavis.2008.02.013]
Srinidhi C L, Aparna P and Rajan J. 2019. Automated method for retinal artery/vein separation via graph search metaheuristic approach. IEEE Transactions on Image Processing, 28(6):2705-2718[DOI:10.1109/TIP.2018.2889534]
Staal J, Abràmoff M D, Niemeijer M, Viergever M A and Van Ginneken B. 2004. Ridge-based vessel segmentation in color images of the retina. IEEE Transactions on Medical Imaging, 23(4):501-509[DOI:10.1109/tmi.2004.825627]
Vázquez S G, Cancela B, Barreira N, Penedo M G, Rodríguez-Blanco M, Seijo M P, de Tuero G C, Barceló M A and Saez M. 2013. Improving retinal artery and vein classification by means of a minimal path approach. Machine Vision and Applications, 24(5):919-930[DOI:10.1007/s00138-012-0442-4]
Welikala R A, Foster P J, Whincup P H, Rudnicka A R, Owen C G, Strachan D P, Barman S A and the UK Biobank Eye and Vision Consortium. 2017. Automated arteriole and venule classification using deep learning for retinal images from the UK Biobank cohort. Computers in Biology and Medicine, 90:23-32[DOI:10.1016/j.compbiomed.2017.09.005]
Wong T Y, Klein R, Klein B E K, Tielsch J M, Hubbard L and Nieto F J. 2001. Retinal microvascular abnormalities and their relationship with hypertension, cardiovascular disease, and mortality. Survey of Ophthalmology, 46(1):59-80[DOI:10.1016/S0039-6257(01)00234-X]
Xu X Y, Ding W X, Abràmoff M D and Cao R F. 2017. An improved arteriovenous classification method for the early diagnostics of various diseases in retinal image. Computer Methods and Programs in Biomedicine, 141:3-9.[DOI:10.1016/j.cmpb.2017.01.007]
Xu X Y, Wang R D, Lv P L, Gao B, Li C, Tian Z Q, Tan T and Xu F. 2018. Simultaneous arteriole and venule segmentation with domain-specific loss function on a new public database. Biomedical Optics Express, 9(7):3153-3166[DOI:10.1364/BOE.9.003153]
Xue L Y, Cao X R, Lin J W, Zheng S H and Yu L. 2017. Artery/vein automatic classification in retinal images and vessel diameter Measurement. Chinese Journal of Scientific Instrument, 38(9):2307-2316
薛岚燕, 曹新容, 林嘉雯, 郑绍华, 余轮. 2017.动静脉血管自动分类方法及其管径测量.仪器仪表学报, 38(9):2307-2316)[DOI:10.3969/j.issn.0254-3087.2017.09.027]
Zhao Y T, Xie J Y, Zhang H Z, Zheng Y L, Zhao Y F, Qi H, Zhao Y C, Su P, Liu J and Liu Y H. 2020. Retinal vascular network topology reconstruction and artery/vein classification via dominant set clustering. IEEE Transactions on Medical Imaging, 39(2):341-356[DOI:10.1109/TMI.2019.2926492]
相关文章
相关作者
相关机构
京公网安备11010802024621
