浏览全部资源
扫码关注微信
纸质出版日期： 2023-09-16 ，
移动端阅览
许聪， 郝华颖， 王阳， 马煜辉， 阎岐峰， 陈浜， 马韶东， 王效贵， 赵一天. 2023. 融合隐向量对齐和Swin Transformer的OCTA血管分割. 中国图象图形学报， 28(09):2927-2939
Xu Cong， Hao Huaying， Wang Yang， Ma Yuhui， Yan Qifeng， Chen Bang， Ma Shaodong， Wang Xiaogui， Zhao Yitian. 2023. Vessel segmentation of OCTA images based on latent vector alignment and swin Transformer. Journal of Image and Graphics， 28(09):2927-2939
目的
2
光学相干断层扫描血管造影（optical coherence tomography angiography，OCTA）是一种非侵入式的新兴技术，越来越多地应用于视网膜血管成像。与传统眼底彩照相比，OCTA技术能够显示黄斑周围的微血管信息，在视网膜血管成像邻域具有显著优势。临床实践中，医生可以通过OCTA图像观察不同层的血管结构，并通过分析血管结构的变化来判断是否存在相关疾病。大量研究表明，血管结构的任何异常变化通常都意味着存在某种眼科疾病。因此，对OCTA图像中的视网膜血管结构进行自动分割提取，对众多眼部相关疾病量化分析和临床决策具有重大意义。然而，OCTA图像存在视网膜血管结构复杂、图像整体对比度低等问题，给自动分割带来极大挑战。为此，提出了一种新颖的融合隐向量对齐和Swin Transformer的视网膜血管结构的分割方法，能够实现血管结构的精准分割。
方法
2
以ResU-Net为主干网络，通过Swin Transformer编码器获取丰富的血管特征信息。此外，设计了一种基于隐向量的特征对齐损失函数，能够在隐空间层次对网络进行优化，提升分割性能。
结果
2
在3个OCTA图像数据集上的实验结果表明，本文方法的AUC（area under curce）分别为94.15%，94.87%和97.63%，ACC（accuracy）分别为91.57%，90.03%和91.06%，领先其他对比方法，并且整体分割性能达到最佳。
结论
2
本文提出的视网膜血管分割网络，在3个OCTA图像数据集上均取得了最佳的分割性能，优于对比方法。
Objective
2
Optical coherence tomography angiography （OCTA） is a noninvasive， emerging technique that has been increasingly used for images of the retinal vasculature at the capillary-level resolution. OCTA technology can demonstrate the microvascular information around the macula and has significant remarkable advantages in retinal vascular imaging. Fundus fluorescence angiography can visualize the retinal vascular system， including capillaries. However， the technique requires intravenous injection of contrast. This process is relatively time-consuming and may have serious side effects. In clinical practice， doctors can look at different layers of vascular structures through OCTA images and analyze changes in vascular structures to determine the presence of related diseases. In particular， any abnormality in the microvasculature distributed in the macula often indicates the presence of some diseases， such as early-stage glaucomatous optic neuropathy， diabetic retinopathy， and age-related macular degeneration. Therefore， the automatic segmentation and extraction of retinal vascular structure in OCTA are vital for the quantitative analysis and clinical decision-making of many ocular diseases. However， the OCTA imaging process usually produces images with a low signal-to-noise ratio， thereby posing a great challenge for the automatic segmentation of vascular structures. Moreover， variations in vessel appearance， motion， and shadowing artifacts in different depth layers and underlying pathological structures significantly remarkably increase the difficulty in accurately segmenting retinal vessels. Therefore， this study proposes a novel segmentation method of retinal vascular structures by fusing hidden vector alignment and Swin Transformer to achieve the accurate segmentation of vascular structures.
Method
2
In this study， the ResU-Net network is used as the base network （the encoder and decoder layers consist of residual blocks and pooling layers）， and the Swin Transformer is introduced into ResU-Net to form a new encoder structure. The encoding step of the feature encoder consists of four stages. Each stage comprises two layers： the Transformer layer consisting of several Swin Transformer blocks stacked together and the residual structure. The Swin Transformer encoder can acquire rich feature information， whereas the feature maps output from each Swin Transformer layer is combined with the feature maps sampled on the decoder via a jump connection. A feature alignment loss function based on hidden vectors is also designed in this study. This feature alignment loss function is different from the classical pixel-level loss function. Feature alignment loss can optimize segmentation results in terms of feature dimensions. It can also enhance the encoder’s ability to extract the structural features of OCTA image vessels and optimize the network at the hidden space level by constraining the consistency of labels and images in the hidden space to improve the segmentation performance.
Result
2
Experimental results on three OCTA datasets （including two public datasets and one private dataset） show that our method is ahead of other comparative methods and has the best overall segmentation performance. In particular， the area under the curves （AUCs） of this method reaches 94.15%， 94.87%， and 97.63%， whereas the accuracy （ACCs） reaches 91.57%， 90.03%， and 91.06%， respectively. Compared with the classical medical image segmentation network U-Net， the proposed method improves the AUC， Kappa， false discovery rate （FDR）， and Dice by approximately 4.06%， 10.18%， 23.16%， and 7.87%， respectively， on the OCTA-O dataset. In addition， ablation experiments are conducted for each component in this study to verify the validity of each component of the proposed model. The results show that each component can play a positive role.
Conclusion
2
An end-to-end vascular segmentation network is proposed in this study to address the challenges of complex retinal vascular structures and low overall image contrast present in OCTA. In this study， ResU-Net is used as the backbone network to mitigate the interference of scattering noise and artifacts on segmentation through image multifusion input. Moreover， the Swin Transformer module is used as the coding structure to obtain rich features. A novel hidden vector alignment loss function that can optimize the network at the hidden space level is also designed in this study. Thus， the gap between segmentation results and labels is reduced， and the segmentation performance is improved. The experimental results demonstrate that the method in this study achieves the best segmentation performance on all three OCTA datasets， and it outperforms other comparative methods.
血管分割光学相干断层扫描血管造影（OCTA）深度学习疾病量化分析隐向量
vessel segmentationoptical coherence tomography angiography （OCTA）deep learningquantitative analysis of diseaselatent vector
Alam M， Toslak D， Lim J I and Yao X C. 2018. Color fundus image guided artery-vein differentiation in optical coherence tomography angiography. Investigative Ophthalmology and Visual Science， 59（12）： 4953-4962 ［DOI： 10.1167/iovs.18-24831http://dx.doi.org/10.1167/iovs.18-24831］
Azzopardi G， Strisciuglio N， Vento M and Petkov N. 2015. Trainable COSFIRE filters for vessel delineation with application to retinal images. Medical Image Analysis， 19（1）： 46-57 ［DOI： 10.1016/j.media.2014.08.002http://dx.doi.org/10.1016/j.media.2014.08.002］
Camino A， Zhang M， Liu L， Wang J， Jia Y L and Huang D. 2018. Enhanced quantification of retinal perfusion by improved discrimination of blood flow from bulk motion signal in OCTA. Translational Vision Science and Technology， 7（6）： #20 ［DOI： 10.1167/tvst.7.6.20http://dx.doi.org/10.1167/tvst.7.6.20］
Cao H， Wang Y Y， Chen J， Jiang D S， Zhang X P， Tian Q and Wang M N. 2021. Swin-unet： unet-like pure Transformer for medical image segmentation//Proceedings of European Conference on Computer Vision. Tel Aviv， Israel： Springer ［DOI： 10.1007/978-3-031-25066-8_9http://dx.doi.org/10.1007/978-3-031-25066-8_9］
Chen J N， Lu Y Y， Yu Q H， Luo X D， Adeli E， Wang Y， Lu L， Yuille A L and Zhou Y Y. 2021. Transunet： Transformers make strong encoders for medical image segmentation ［EB/OL］. ［2022-02-23］. https://arxiv.org/pdf/2102.04306.pdfhttps://arxiv.org/pdf/2102.04306.pdf
Dai Y， Gao Y F and Liu F Y. 2021. Transmed： Transformers advance multi-modal medical image classification. Diagnostics， 11（8）： #1384 ［DOI： 10.3390/diagnostics11081384http://dx.doi.org/10.3390/diagnostics11081384］
Deng K Z， Meng Y D， Gao D X， Bridge J， Shen Y C， Lip G， Zhao Y T and Zheng Y L. 2021. TransBridge： a lightweight Transformer for left ventricle segmentation in echocardiography//Proceedings of the 2nd International Workshop on Advances in Simplifying Medical Ultrasound. Strasbourg， France： Springer： 63-72 ［DOI： 10.1007/978-3-030-87583-1_7http://dx.doi.org/10.1007/978-3-030-87583-1_7］
Eladawi N， Elmogy M， Helmy O， Aboelfetouh A， Riad A， Sandhu H， Schaal S and El-Baz A. 2017. Automatic blood vessels segmentation based on different retinal maps from OCTA scans. Computers in Biology and Medicine， 89： 150-161 ［DOI： 10.1016/j.compbiomed.2017.08.008http://dx.doi.org/10.1016/j.compbiomed.2017.08.008］
Giarratano Y， Bianchi E， Gray C， Morris A， MacGillivray T， Dhillon B and Bernabeu M O. 2020. Automated segmentation of optical coherence tomography angiography images： benchmark data and clinically relevant metrics. Translational Vision Science and Technology， 9（13）： #5 ［DOI： 10.1167/tvst.9.13.5http://dx.doi.org/10.1167/tvst.9.13.5］
Gu Z W， Cheng J， Fu H Z， Zhou K， Hao H Y， Zhao Y T， Zhang T Y， Gao S H and Liu J. 2019. CE-Net： context encoder network for 2D medical image segmentation. IEEE Transactions on Medical Imaging， 38（10）： 2281-2292 ［DOI： 10.1109/TMI.2019.2903562http://dx.doi.org/10.1109/TMI.2019.2903562］
Hatamizadeh A， Tang Y C， Nath V， Yang D， Myronenko A， Landman B， Roth H R and Xu D G. 2022. UNETR： Transformers for 3D medical image segmentation//Proceedings of 2022 IEEE/CVF Winter Conference on Applications of Computer Vision. Waikoloa， USA： IEEE： 574-584 ［DOI： 10.1109/wacv51458.2022.00181http://dx.doi.org/10.1109/wacv51458.2022.00181］
Hormel T T， Hwang T S， Bailey S T， Wilson D J， Huang D and Jia Y L. 2021. Artificial intelligence in OCT angiography. Progress in Retinal and Eye Research， 85： #100965 ［DOI： 10.1016/j.preteyeres.2021.100965http://dx.doi.org/10.1016/j.preteyeres.2021.100965］
Huang X H， Deng Z F， Li D D and Yuan X G. 2021. MISSFormer： an effective medical image segmentation Transformer ［EB/OL］. ［2022-02-12］. https://arxiv.org/pdf/2109.07162.pdfhttps://arxiv.org/pdf/2109.07162.pdf
Jin Q G， Meng Z P， Pham T D， Chen Q， Wei L Y and Su R. 2019. DUNet： a deformable network for retinal vessel segmentation. Knowledge-Based Systems， 178： 149-162 ［DOI： 10.1016/j.knosys.2019.04.025http://dx.doi.org/10.1016/j.knosys.2019.04.025］
Leitgeb R A. 2019. En face optical coherence tomography： a technology review ［Invited］. Biomedical Optics Express， 10（5）： 2177-2201 ［DOI： 10.1364/BOE.10.002177http://dx.doi.org/10.1364/BOE.10.002177］
Li M C， Chen Y R， Ji Z X， Xie K R， Yuan S T， Chen Q and Li S. 2020. Image projection network： 3D to 2D image segmentation in OCTA images. IEEE Transactions on Medical Imaging， 39（11）： 3343-3354 ［DOI： 10.1109/TMI.2020.2992244http://dx.doi.org/10.1109/TMI.2020.2992244］
Liu H C， Ren W Q， Wang R and Cao X C. 2022. A super-resolution Transformer fusion network for single blurred image. Journal of Image and Graphics， 27（5）： 1616-1631
刘花成， 任文琦， 王蕊， 操晓春. 2022. 用于单幅模糊图像超分辨的Transformer融合网络. 中国图象图形学报， 27（5）： 1616-1631 ［DOI： 10.11834/jig.210847http://dx.doi.org/10.11834/jig.210847］
Ma Y H， Hao H Y， Xie J Y， Fu H Z， Zhang J， Yang J L， Wang Z， Liu J， Zheng Y L and Zhao Y T. 2021. ROSE： a retinal OCT-angiography vessel segmentation dataset and new model. IEEE Transactions on Medical Imaging， 40（3）： 928-939 ［DOI： 10.1109/TMI.2020.3042802http://dx.doi.org/10.1109/TMI.2020.3042802］
McCollough C H， Bartley A C， Carter R E， Chen B Y， Drees T A， Edwards P， Holmes D R， Huang A E， Khan F， Leng S， McMillan K L， Michalak G J， Nunez K M， Yu L F and Fletcher J G. 2017. Low‐dose CT for the detection and classification of metastatic liver lesions： results of the 2016 low dose CT grand challenge. Medical Physics， 44（10）： e339-e352 ［DOI： 10.1002/mp.12345http://dx.doi.org/10.1002/mp.12345］
Mou L， Zhao Y T， Chen L， Cheng J， Gu Z W， Hao H Y， Qi H， Zheng Y L， Frangi A and Liu J. 2019. CS-Net： channel and spatial attention network for curvilinear structure segmentation//Proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen， China： Springer： 721-730 ［DOI： 10.1007/978-3-030-32239-7_80http://dx.doi.org/10.1007/978-3-030-32239-7_80］
Ronneberger O， Fischer P and Brox T. 2015. U-Net： convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich， Germany： Springer： 234-241 ［DOI： 10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28］
Shen Z Q， Fu R D， Lin C N and Zheng S H. 2021. COTR： convolution in Transformer network for end to end polyp detection//Proceedings of the 7th International Conference on Computer and Communications （ICCC）. Chengdu， China： IEEE： 1757-1761 ［DOI： 10.1109/ICCC54389.2021.9674267http://dx.doi.org/10.1109/ICCC54389.2021.9674267］
Szkulmowski M， Gorczynska I， Szlag D， Sylwestrzak M， Kowalczyk A and Wojtkowski M. 2012. Efficient reduction of speckle noise in optical coherence tomography. Optics Express， 20（2）： 1337-1359 ［DOI： 10.1364/OE.20.001337http://dx.doi.org/10.1364/OE.20.001337］
Valanarasu J M J， Oza P， Hacihaliloglu I and Patel V M. 2021. Medical Transformer： gated axial-attention for medical image segmentation//Proceedings of the 24th International Conference on Medical Image Computing and Computer-Assisted Intervention. Strasbourg， France： Springer： 36-46 ［DOI： 10.1007/978-3-030-87193-2_4http://dx.doi.org/10.1007/978-3-030-87193-2_4］
Wang C， Shang K， Zhang H M， Li Q， Hui Y and Zhou S K. 2021. DuDoTrans： dual-domain Transformer provides more attention for sinogram restoration in sparse-view CT reconstruction ［EB/OL］. ［2022-02-12］. https://arxiv.org/pdf/2111.10790.pdfhttps://arxiv.org/pdf/2111.10790.pdf
Wang S L， Yin Y L， Cao G B， Wei B Z， Zheng Y J and Yang G P. 2015. Hierarchical retinal blood vessel segmentation based on feature and ensemble learning. Neurocomputing， 149： 708-717 ［DOI： 10.1016/j.neucom.2014.07.059http://dx.doi.org/10.1016/j.neucom.2014.07.059］
Witmer M T， Parlitsis G， Patel S and Kiss S. 2013. Comparison of ultra-widefield fluorescein angiography with the Heidelberg spectralis® noncontact ultra-widefield module versus the optos® optomap®. Clinical Ophthalmology， 7： 389-394 ［DOI： 10.2147/OPTH.S41731http://dx.doi.org/10.2147/OPTH.S41731］
Yan Z Q， Yang X and Cheng K T. 2018. A three-stage deep learning model for accurate retinal vessel segmentation. IEEE Journal of Biomedical and Health Informatics， 23（4）： 1427-1436 ［DOI： 10.1109/JBHI.2018.2872813http://dx.doi.org/10.1109/JBHI.2018.2872813］
Yoon S P， Grewal D S， Thompson A C， Polascik B W， Dunn C， Burke J R and Fekrat S. 2019. Retinal microvascular and neurodegenerative changes in Alzheimer’s disease and mild cognitive impairment compared with control participants. Ophthalmology Retina， 3（6）： 489-499 ［DOI： 10.1016/j.oret.2019.02.002http://dx.doi.org/10.1016/j.oret.2019.02.002］
Yousefi S， Liu T and Wang R K. 2015. Segmentation and quantification of blood vessels for OCT-based micro-angiograms using hybrid shape/intensity compounding. Microvascular Research， 97： 37-46 ［DOI： 10.1016/j.mvr.2014.09.007http://dx.doi.org/10.1016/j.mvr.2014.09.007］
Zhang J， Chen Y， Bekkers E， Wang M L， Dashtbozorg B and Romeny B M T H. 2017. Retinal vessel delineation using a brain-inspired wavelet transform and random forest. Pattern Recognition， 69： 107-123 ［DOI： 10.1016/j.patcog.2017.04.008http://dx.doi.org/10.1016/j.patcog.2017.04.008］
Zhang J， Dashtbozorg B， Bekkers E， Pluim J P W， Duits R and Romeny B M T H. 2016. Robust retinal vessel segmentation via locally adaptive derivative frames in orientation scores. IEEE Transactions on Medical Imaging， 35（12）： 2631-2644 ［DOI： 10.1109/TMI.2016.2587062http://dx.doi.org/10.1109/TMI.2016.2587062］
Zhang J， Qiao Y C， Sarabi M S， Khansari M M， Gahm J K， Kashani A H and Shi Y G. 2020. 3D shape modeling and analysis of retinal microvasculature in OCT-angiography images. IEEE Transactions on Medical Imaging， 39（5）： 1335-1346 ［DOI： 10.1109/TMI.2019.2948867http://dx.doi.org/10.1109/TMI.2019.2948867］
Zhang Y D， Liu H Y and Hu Q. 2021a. TransFuse： fusing Transformers and CNNs for medical image segmentation//Proceedings of the 24th International Conference on Medical Image Computing and Computer-Assisted Intervention. Strasbourg， France： Springer： 14-24 ［DOI： 10.1007/978-3-030-87193-2_2http://dx.doi.org/10.1007/978-3-030-87193-2_2］
Zhang Y L， Higashita R， Fu H Z， Xu Y W， Zhang Y， Liu H F， Zhang J and Liu J. 2021b. A multi-branch hybrid Transformer network for corneal endothelial cell segmentation//Proceedings of the 24th International Conference on Medical Image Computing and Computer-Assisted Intervention. Strasbourg， France： Springer： 99-108 ［DOI： 10.1007/978-3-030-87193-2_10http://dx.doi.org/10.1007/978-3-030-87193-2_10］
Zhang Z J， Fu H Z， Dai H， Shen J B， Pang Y W and Shao L. 2019. ET-Net： a generic edge-attention guidance network for medical image segmentation//Proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen， Chian： Springer： 442-450 ［DOI： 10.1007/978-3-030-32239-7_49http://dx.doi.org/10.1007/978-3-030-32239-7_49］
Zhao C Q， Wang H H， Zhao J J， Ji L W， Wang Q D， Li H Z and Zhao Z J. 2022. Cerebral stroke detection algorithm for visual Transformer and multi-feature fusion. Journal of Image and Graphics， 27（3）： 923-934
赵琛琦， 王华虎， 赵涓涓， 冀伦文， 王麒达， 李慧芝， 赵紫娟. 2022. 视觉Transformer与多特征融合的脑卒中检测算法. 中国图象图形学报， 27（3）： 923-934 ［DOI： 10.11834/jig.210745http://dx.doi.org/10.11834/jig.210745］
Zhao Y T， Rada L， Chen K， Harding S P and Zheng Y L. 2015. Automated vessel segmentation using infinite perimeter active contour model with hybrid region information with application to retinal images. IEEE Transactions on Medical Imaging， 34（9）： 1797-1807 ［DOI： 10.1109/TMI.2015.2409024http://dx.doi.org/10.1109/TMI.2015.2409024］
Zhao Y T， Zheng Y L， Liu Y H， Yang J， Zhao Y F， Chen D D and Wang Y T. 2017. Intensity and compactness enabled saliency estimation for leakage detection in diabetic and malarial retinopathy. IEEE Transactions on Medical Imaging， 36（1）： 51-63 ［DOI： 10.1109/TMI.2016.2593725http://dx.doi.org/10.1109/TMI.2016.2593725］
Zhao Y T， Zheng Y L， Liu Y H， Zhao Y F， Luo L L， Yang S Y， Na T， Wang Y T and Liu J. 2018. Automatic 2-D/3-D vessel enhancement in multiple modality images using a weighted symmetry filter. IEEE Transactions on Medical Imaging， 37（2）： 438-450 ［DOI： 10.1109/TMI.2017.2756073http://dx.doi.org/10.1109/TMI.2017.2756073］
相关作者
相关机构
京公网安备11010802024621