Research status of cardiac image segmentation based on deep learning

Zeng Jiatao; Zhang Heye; Liu Huafeng

doi:10.11834/jig.230027

Pattern Recognition and Intelligent Visualization | Views : 0 下载量: 4 CSCD: 0

PDF
Export
Share
Collection
Album

Research status of cardiac image segmentation based on deep learning
Vol. 28, Issue 6, Pages: 1811-1828(2023)
Published： 16 June 2023 ，
DOI： 10.11834/jig.230027
稿件说明：

移动端阅览

曾嘉涛，张贺晔，刘华锋. 2023. 基于深度学习的心脏图像分割研究现状. 中国图象图形学报， 28(06):1811-1828

Zeng Jiatao， Zhang Heye， Liu Huafeng. 2023. Research status of cardiac image segmentation based on deep learning. Journal of Image and Graphics， 28(06):1811-1828
曾嘉涛，张贺晔，刘华锋. 2023. 基于深度学习的心脏图像分割研究现状. 中国图象图形学报， 28(06):1811-1828 DOI： 10.11834/jig.230027.

Zeng Jiatao， Zhang Heye， Liu Huafeng. 2023. Research status of cardiac image segmentation based on deep learning. Journal of Image and Graphics， 28(06):1811-1828 DOI： 10.11834/jig.230027.

摘要

面对严重的医学影像分析缺口，深度学习的发展能够满足国内医疗行业的需求。心脏图像的处理方法可大致分为传统的图像处理技术、基于图谱的方法（atlas-based methods）、基于模型的方法（model-based methods）以及目前热门的采用机器学习和深度学习的方法。在深度学习兴起之前，传统的机器学习技术如模型法和图集法在心脏图像分割中有良好表现，但通常需要大量的特征工程知识或先验知识才能获得令人满意的精度。而基于深度学习的算法能从数据中自动发现复杂的特征以进行对象检测和分割。得益于先进的计算机硬件以及更多可用于训练的数据集，基于深度学习的分割算法已超越了以往的传统方法。本文回顾了2012—2022年有关心室、心外膜和心包脂肪的图像处理的各项方法、衡量指标及其目前的研究现状，并结合分割技术的发展，讨论了心脏分割的发展趋势。

Abstract

Human-cardiovascular disease is challenged for its high morbidity and severe sequelae nowadays. To meet the need of the medical industry， current medical image analysis is facilitated via the development of deep learning. Conventional image processing technology is processed basically in terms of thresholding. The emerging deep learning technique can be focused on reality-oriented function in terms of specific eigenvalues. Such deep residual network and generative confrontation network have its potentials for its effectiveness and robust originated from good learning ability and data-driven factors. Our critical analysis is based on 1） characteristics of representative methods， 2） resources and scale of cardiac images， 3） comparative study of the performance evaluation and application conclusions of different methods through popular evaluation indicators， and 4） clinical domains are discussed as well. The literature review is originated from IEEE， SPIE， and China National Knowledge Network， with image processing and heart as search keywords. The difference between image processing methods is evaluated in terms of Dice coefficient and Hausdorff distance， and the performance is evaluated in quantitative further. For chamber segmentation， several approaches for right ventricle segmentation are reviewed and analyzed. As far as the principle of the segmentation method is concerned， the single of threshold is still challenged for segmentation unless it is integrated into other related methods， so it cannot be used as a single zone in segmentation technique. A brief theoretical introduction is mentioned for each method. Then， its methodology， prior datasets， and the effectiveness of segmentation process are involved in evaluation. Finally， the pros and cons of each method are analyzed as well. For the domain of epicardium and pericardium tissue， we will briefly introduce the popular image processing techniques for segmenting epicardium and pericardium tissue. Four category of key methods are analyzed in relevance to its： traditional image processing methods， atlas-based methods， machine learning， and deep learning. Traditional image processing methods are composed of such techniques of thresholding， region growing， and active contouring. Finally， Dice coefficient-derived capabilities of each algorithm are compared horizontally. For the segmentation method of the epicardium， it is easier to segment the epicardium into pericardium-illustrated coordination. Epicardial and pericardial fatty tissue are unevenly distributed around the heart， resulting in large sections-between variability and the images-between for its computed tomography（CT） and magnetic resonance imaging（MRI）. The heterogeneity in shape is required to demonstrate further. However， the pericardium is featured of more smoother， thinner and oval in CT and MRI images. Such methods of active contours or ellipse fitting are suitable for segmenting such shapes naturally. Once the pericardium is divided， the epicardium is more easily divided into all pericardium-within fatty tissue. The great challenge is focused on epicardium-thinner segmentation. The slice of thickness can be set at 2～3 mm when CT scans are collected for coronary artery calcification（CAC） scoring. The pericardium is usually less than 2 mm thick， and it will often appear blurred or blurred on CT images in accordance with partial volume averaging， especially for heart organ-moving consistency. Some methods are purely pericardial delineation methods， while others are part of a method to segment and quantify the epicardium. For the epicardium part， we will mainly introduce the method of epicardium segmentation by the first pericardium-segmented. Pericardial fat segmentation methods typically rely on traditional image processing methods， such as 1） thresholding and region growing， and 2） various preprogrammed heuristics can be used to identify common structures and segment pericardial fat. Recent atlas-based segmentation approaches are employed but its clinical ability is relatively weakened. After current situation of segmentation is introduced， we will introduce some real scenarios applied in clinical practice. We can see that cardiac image processing has a large number of clinical problems are required to be solved. At the same time， we will briefly introduce the market situation of image processing in domestic market， integration of industry， education and research， and the main relevant policy trends. the development of the main related industries is introduced and involve in like 1） the establishment of related imaging databases in China， 2） the development of related imaging technologies in China， and 3） the development of related hardware equipment in China. At the end， it is discussed that the development of cardiac image segmentation processing is increasingly inseparable in related to the development of deep learning. However， because deep learning itself is difficult to be explained， we called on medical knowledge-interpreted method models， and deep learning based constraints are called to be resolved further， such homogeneity data sets and its related of higher accuracy.

关键词

图像分割全心分割心外膜心包脂肪辅助检测

Keywords

image segementationwhole heart segmentationepicardiumpericardial fatauxiliary detection

references

Avendi M R， Kheradvar A and Jafarkhani H. 2017. Automatic segmentation of the right ventricle from cardiac MRI using a learning-based approach. Magnetic Resonance in Medicine， 78（6）： 2439-2448 ［DOI： 10.1002/MRM.26631http://dx.doi.org/10.1002/MRM.26631］

Baessler B， Mannil M， Oebel S， Maintz D， Alkadhi H and Manka R. 2018. Subacute and chronic left ventricular myocardial scar： accuracy of texture analysis on nonenhanced cine MR images. Radiology， 286（1）： 103-112 ［DOI： 10.1148/RADIOL.2017170213http://dx.doi.org/10.1148/RADIOL.2017170213］

Bai W J， Shi W Z， Wang H Y， Peters N S and Rueckert D. 2012. Multi-atlas based segmentation with local label fusion for right ventricle MR images. Image， 6： #9

Bandekar A N， Naghavi M and Kakadiaris I A. 2006. Automated pericardial fat quantification in CT data//Proceedings of 2006 International Conference of the IEEE Engineering in Medicine and Biology Society. New York， USA： IEEE： 932-935 ［DOI： 10.1109/IEMBS.2006.259259http://dx.doi.org/10.1109/IEMBS.2006.259259］

Benčević M， Habijan M and Galić I. 2021. Epicardial adipose tissue segmentation from CT images with a Semi-3D neural network//Proceedings of 2021 International Symposium ELMAR. Zadar， Croatia： IEEE： 87-90 ［DOI： 10.1109/ELMAR52657.2021.9550936http://dx.doi.org/10.1109/ELMAR52657.2021.9550936］

Chen J， Yang G， Khan H， Zhang H Y， Zhang Y P， Zhao S， Mohiaddin R， Wong T， Firmin D and Keegan J. 2022. JAS-GAN： generative adversarial network based joint atrium and scar segmentations on unbalanced atrial targets. IEEE Journal of Biomedical and Health Informatics， 26（1）： 103-114 ［DOI： 10.1109/JBHI.2021.3077469http://dx.doi.org/10.1109/JBHI.2021.3077469］

Commandeur F， Goeller M， Betancur J， Cadet S， Doris M， Chen X， Berman D S， Slomka P J， Tamarappoo B K and Dey D. 2018. Deep learning for quantification of epicardial and thoracic adipose tissue from non-contrast CT. IEEE Transactions on Medical Imaging， 37（8）： 1835-1846 ［DOI： 10.1109/TMI.2018.2804799http://dx.doi.org/10.1109/TMI.2018.2804799］

Coppini G， Favilla R， Marraccini P， Moroni D and Pieri G. 2010. Quantification of epicardial fat by cardiac CT imaging. The Open Medical Informatics Journal， 4（1）： 126-135 ［DOI： 10.2174/1874431101004010126http://dx.doi.org/10.2174/1874431101004010126］

Dey D， Suzuki Y， Suzuki S， Ohba M， Slomka P J， Polk D， Shaw L J and Berman D S. 2008. Automated quantitation of pericardiac fat from noncontrast CT. Investigative Radiology， 43（2）： 145-153 ［DOI： 10.1097/RLI.0b013e31815a054ahttp://dx.doi.org/10.1097/RLI.0b013e31815a054a］

Ding X W， Pang J N， Ren Z， Diaz-Zamudio M， Jiang C， Fan Z Y， Berman D S， Li D B， Terzopoulos D， Slomka P J and Dey D. 2016. Automated pericardial fat quantification from coronary magnetic resonance angiography： feasibility study. Journal of Medical Imaging， 3（1）： #014002 ［DOI： 10.1117/1.JMI.3.1.014002http://dx.doi.org/10.1117/1.JMI.3.1.014002］

Ding X W， Terzopoulos D， Diaz-Zamudio M， Berman D S， Slomka P J and Dey D. 2014. Automated epicardial fat volume quantification from non-contrast CT//Proceedings Volume 9034， Medical Imaging 2014： Image Processing. San Diego， USA： SPIE： 124-129 ［DOI： 10.1117/12.2043326http://dx.doi.org/10.1117/12.2043326］

El-Rewaidy H and Fahmy A S. 2015. Segmentation of the right ventricle in MR images using dual active shape model in the Bookstein coordinates//The 12th IEEE International Symposium on Biomedical Imaging （ISBI）. Brooklyn， USA： IEEE： 1320-1323 ［DOI： 10.1109/ISBI.2015.7164118http://dx.doi.org/10.1109/ISBI.2015.7164118］

He X X， Guo B J， Lei Y， Wang T H， Liu T， Curran W J， Zhang L J and Yang X F. 2020. Automatic epicardial fat segmentation in cardiac CT imaging using 3D deep attention U-Net//Proceedings Volume 11313， Medical Imaging 2020： Image Processing. Houston， USA： SPIE： 589-595 ［DOI： 10.1117/12.2550383http://dx.doi.org/10.1117/12.2550383］

Hu C P and Zhao Y X.2017. The relationship between epicardial adipose tissue and coronary heart disease. Chinese Journal of Cardiovascular Diseases， 45（5）： 447-450

胡成平，赵迎新. 2017. 心外膜脂肪组织与冠心病的关系.中华心血管病杂志， 45（5）： 447-450

Išgum I， Rutten A， Prokop M and van Ginneken B. 2007. Detection of coronary calcifications from computed tomography scans for automated risk assessment of coronary artery disease. Medical Physics， 34（4）： 1450-1461 ［DOI： 10.1118/1.2710548http://dx.doi.org/10.1118/1.2710548］

Isgum I， van Ginneken B and Olree M. 2004. Automatic detection of calcifications in the aorta from CT scans of the abdomen： 3D computer-aided diagnosis. Academic Radiology， 11（3）： 247-257 ［DOI： 10.1016/s1076-6332（03）00673-1http://dx.doi.org/10.1016/s1076-6332（03）00673-1］

Kazemi A， Keshtkar A， Rashidi S， Aslanabadi N， Khodadad B and Esmaeili M. 2020. Segmentation of cardiac fats based on Gabor filters and relationship of adipose volume with coronary artery disease using FP-Growth algorithm in CT scans. Biomedical Physics and Engineering Express， 6（5）： #055009 ［DOI： 10.1088/2057-1976/ABA441http://dx.doi.org/10.1088/2057-1976/ABA441］

Kolossv􀅡ry M， Kar􀅡dy J， Szilveszter B， Kitslaar P， Hoffmann U， Merkely B and Maurovich-Horvat P. 2017. Radiomic features are superior to conventional quantitative computed tomographic metrics to identify coronary plaques with Napkin-ring sign. Circulation： Cardiovascular Imaging， 10（12）： #e006843 ［DOI： 10.1161/CIRCIMAGING.117.006843http://dx.doi.org/10.1161/CIRCIMAGING.117.006843］

Li L， Zimmer V A， Schnabel J A and Zhuang X H. 2022. AtrialJSQnet： a new framework for joint segmentation and quantification of left atrium and scars incorporating spatial and shape information. Medical Image Analysis， 76： #102303 ［DOI： 10.1016/j.media.2021.102303http://dx.doi.org/10.1016/j.media.2021.102303］

Li Q， Huangfu Y B， Li J Y， Yang Z F， Chen P and Wang Z H. 2022. UConvTrans： a dual-flow cardiac image segmentation network by global and local information integration. Journal of Shanghai Jiao Tong University

李擎，皇甫玉彬，李江昀，杨志方，陈鹏，王子涵. 2022. UConvTrans：全局和局部信息交互的双分支心脏图像分割. 上海交通大学学报［DOI： 10.16183/j.cnki.jsjtu.2022.088http://dx.doi.org/10.16183/j.cnki.jsjtu.2022.088］

Li Z ， Zou L and Yang R. 2019. A neural network-based method for automatic pericardium segmentation//Proceedings of the 2nd International Conference on Computer Science and Software Engineering. Xi’an，China： ACM： 45-49［DOI：10.1145/3339363.3339372http://dx.doi.org/10.1145/3339363.3339372］

Lin T Y， Song L， Gao Z F and Zhang H Y. 2022. Evaluation of a deep learning-based model for 2-D echocardiography segmentation on small datasets. Journal of Jinan University （Natural Science and Medicine Edition）， 43（2）： 191-198

林天予，宋亮，高智凡，张贺晔. 2022. 基于深度学习的2维心脏超声图像分割模型在小规模数据集上的性能评估. 暨南大学学报（自然科学与医学版）， 43（2）： 191-198 ［DOI： 10.11778/j.jdxb.2022.02.010http://dx.doi.org/10.11778/j.jdxb.2022.02.010］

Liu C J， Xu J C， Xiao Z Y and Chai Z L. 2022. An automatic cardiac magnetic resonance image segmentation algorithm based on deep learning. Computer Engineering and Science， 44（9）： 1646-1654

刘从军，徐佳陈，肖志勇，柴志雷. 2022. 基于深度学习的心脏核磁共振图像自动分割算法. 计算机工程与科学， 44（9）： 1646-1654 ［DOI： 10.3969/j.issn.1007-130X.2022.09.015http://dx.doi.org/10.3969/j.issn.1007-130X.2022.09.015］

Luo G N， An R， Wang K Q， Dong S Y and Zhang H G. 2016. A deep learning network for right ventricle segmentation in short-axis MRI//Proceedings of 2016 Computing in Cardiology Conference （CinC）. Vancouver， Canada： IEEE： 485-488

Mahapatra D. 2013. Cardiac image segmentation from cine cardiac MRI using graph cuts and shape priors. Journal of Digital Imaging， 26（4）： 721-730 ［DOI： 10.1007/s10278-012-9548-5http://dx.doi.org/10.1007/s10278-012-9548-5］

Mahapatra D. 2014. Automatic cardiac segmentation using semantic information from random forests. Journal of Digital Imaging， 27（6）： 794-804 ［DOI： 10.1007/s10278-014-9705-0http://dx.doi.org/10.1007/s10278-014-9705-0］

Maier O M O， Jiménez D， Santos A and Ledesma-Carbayo M J. 2012. Segmentation of RV in 4D cardiac MR volumes using region-merging graph cuts//Proceedings of 2012 Computing in Cardiology. Krakow， Poland： IEEE： 697-700

Mannil M， von Spiczak J， Manka R and Alkadhi H. 2018. Texture analysis and machine learning for detecting myocardial infarction in noncontrast low-dose computed tomography： unveiling the invisible. Investigative Radiology， 53（6）： 338-343 ［DOI： 10.1097/RLI.0000000000000448http://dx.doi.org/10.1097/RLI.0000000000000448］

Militello C， Rundo L， Toia P， Conti V， Russo G， Filorizzo C， Maffei E， Cademartiri F， La Grutta L， Midiri M and Vitabile S. 2019. A semi-automatic approach for epicardial adipose tissue segmentation and quantification on cardiac CT scans. Computers in Biology and Medicine， 114： #103424 ［DOI： 10.1016/j.compbiomed.2019.103424http://dx.doi.org/10.1016/j.compbiomed.2019.103424］

Norlén A， Alvén J， Molnar D， Enqvist O， Norrlund R R， Brandberg J， Bergström G and Kahl F. 2016. Automatic pericardium segmentation and quantification of epicardial fat from computed tomography angiography. Journal of Medical Imaging， 3（3）： #034003 ［DOI： 10.1117/1.JMI.3.3.034003http://dx.doi.org/10.1117/1.JMI.3.3.034003］

Ou Y M， Doshi J， Erus G and Davatzikos C. 2012. Multi-atlas segmentation of the cardiac MR right ventricle//Proceedings of 2012 Conference on 3D Cardiovascular Imaging： A MICCAI Segmentation Challenge， 1-6

Priya C and Sudha S. 2019. Adaptive fruitfly based modified region growing algorithm for cardiac fat segmentation using optimal neural network. Journal of medical systems， 43（5）： #104［DOI：10.1007/s10916-019-1227-3http://dx.doi.org/10.1007/s10916-019-1227-3］

Punithakumar K， Noga M， Ben Ayed I and Boulanger P. 2015. Right ventricular segmentation in cardiac MRI with moving mesh correspondences. Computerized Medical Imaging and Graphics， 43： 15-25 ［DOI： 10.1016/j.compmedimag.2015.01.004http://dx.doi.org/10.1016/j.compmedimag.2015.01.004］

Rodrigues É O， Morais F F C， Morais N A O S， Conci L S， Neto L V and Conci A. 2016. A novel approach for the automated segmentation and volume quantification of cardiac fats on computed tomography. Computer Methods and Programs in Biomedicine， 123： 109-128 ［DOI： 10.1016/j.cmpb.2015.09.017http://dx.doi.org/10.1016/j.cmpb.2015.09.017］

Rodrigues É O， Rodrigues L O， Oliveira L S N， Conci A and Liatsis P. 2017. Automated recognition of the pericardium contour on processed CT images using genetic algorithms. Computers in Biology and Medicine， 87： 38-45 ［DOI： 10.48550/arXiv.2208.14375http://dx.doi.org/10.48550/arXiv.2208.14375］

Shahzad R， Bos D， Metz C， Rossi A， Kirişli H， van der Lugt A， Klein S， Witteman J， de Feyter P， Niessen W， van Vliet L and van Walsum T. 2013. Automatic quantification of epicardial fat volume on non-enhanced cardiac CT scans using a multi-atlas segmentation approach. Medical Physics， 40（9）： #091910 ［DOI： 10.1118/1.4817577http://dx.doi.org/10.1118/1.4817577］

Tran P V. 2017. A fully convolutional neural network for cardiac segmentation in short-axis MRI ［EB/OL］. ［2023-01-04］. https://arxiv.org/pdf/1604.00494.pdfhttps://arxiv.org/pdf/1604.00494.pdf

Vladimir Z ， Lidija K ， Lazar V ， Branislav P ， Marko J and Ratko O . 2017. Semiautomatic epicardial fat segmentation based on fuzzy c-means clustering and geometric ellipse fitting. Journal of Healthcare Engineering： 1-12 ［DOI：10.1155/2017/5817970http://dx.doi.org/10.1155/2017/5817970］

Wang K M， Ma Y R， Lei R M， Yang Z and Ma Y D. 2017. Automatic right ventricle segmentation in cardiac MRI via anisotropic diffusion and SPCNN//Proceedings Volume 10225， the 8th International Conference on Graphic and Image Processing. Tokyo， Japan： SPIE： 444-449 ［DOI： 10.1117/12.2266118http://dx.doi.org/10.1117/12.2266118］

Wu X M， Liu S， Huo L， Zhao X H and Shang F. 2021. Cardiac PET/CT and MR image registration based on unsupervised deep learning. Chinese Journal of Medical Imaging， 29（11）： 1158-1164

毋晓萌，刘帅，霍力，赵锡海，尚斐. 2021. 基于无监督深度学习的心脏PET/CT和MRI图像配准. 中国医学影像学杂志， 29（11）： 1158-1164 ［DOI： 10.3969/j.issn.1005-5185.2021.11.023http://dx.doi.org/10.3969/j.issn.1005-5185.2021.11.023］

Xu J C and Xiao Z Y. 2022. A spatio-temporal multi-scale network for cardiac dynamic MRI image segmentation. Journal of Image and Graphics， 27（3）： 862-872

徐佳陈，肖志勇. 2022. 心脏动态MRI图像分割的时空多尺度网络. 中国图象图形学报， 27（3）： 862-872 ［DOI： 10.11834/jig.200783http://dx.doi.org/10.11834/jig.200783］

Zhang L， Karanikolas G V， Akçakaya M and Giannakis G B. 2018. Fully automatic segmentation of the right ventricle via multi-task deep neural networks//Proceedings of 2018 IEEE International Conference on Acoustics， Speech and Signal Processing. Calgary， Canada： IEEE： 6677-6681 ［DOI： 10.1109/ICASSP.2018.8461556http://dx.doi.org/10.1109/ICASSP.2018.8461556］

Zhang N， Yang G， Gao Z F， Xu C C， Zhang Y P， Shi R， Keegan J， Xu L， Zhang H Y， Fan Z M and Firmin D. 2019. Deep learning for diagnosis of chronic myocardial infarction on nonenhanced cardiac cine MRI. Radiology， 291（3）： 606-617 ［DOI： 10.1148/radiol.2019182304http://dx.doi.org/10.1148/radiol.2019182304］

Zhang Q， Zhou J H， Zhang B， Jia W J and Wu E H. 2020. Automatic epicardial fat segmentation and quantification of CT scans using dual U-Nets with a morphological processing layer. IEEE Access， 8： 128032-128041 ［DOI： 10.1109/access.2020.3008190http://dx.doi.org/10.1109/access.2020.3008190］

Zotti C， Luo Z M， Humbert O， Lalande A and Jodoin P M. 2018. GridNet with automatic shape prior registration for automatic MRI cardiac segmentation//Proceedings of the 8th International Workshop on Statistical Atlases and Computational Models of the Heart. Quebec City， Canada： Springer： 73-81 ［DOI： 10.1007/978-3-319-75541-0_8http://dx.doi.org/10.1007/978-3-319-75541-0_8］

Zreik M， van Hamersvelt R W， Wolterink J M， Leiner T， Viergever M A and Išgum I. 2019. A recurrent CNN for automatic detection and classification of coronary artery plaque and stenosis in coronary CT angiography. IEEE Transactions on Medical Imaging， 38（7）： 1588-1598 ［DOI： 10.1109/TMI.2018.2883807http://dx.doi.org/10.1109/TMI.2018.2883807］

Alert me when the article has been cited

提交

暂无数据