浏览全部资源
扫码关注微信
Published： 16 August 2023 ，
移动端阅览
徐光柱， 钱奕凡， 王阳， 刘蓉， 周军， 雷帮军. 2023. 基于两级分割的胎儿四腔心超声切面质量评测. 中国图象图形学报， 28(08):2476-2490
Xu Guangzhu， Qian Yifan， Wang Yang， Liu Rong， Zhou Jun， Lei Bangjun. 2023. Quality assessment for fetal four-chamber ultrasound views based on two-stage segmentation. Journal of Image and Graphics， 28(08):2476-2490
目的
2
为解决基于深度学习算法在执行胎儿四腔心超声切面图像质量评测时无法准确反映心脏区域中瓣膜与房室间隔及心室心房区域的可见程度问题，提出一种目标检测与两级分割相结合的胎儿四腔心超声切面图像质量评测方法。
方法
2
首先利用自行构建的胎儿超声切面数据集训练主流的YOLOv5x（you only look once v5x）模型，实现四腔心区域与胸腔区域的有效定位。当检测到四腔心区域在胸腔区域内时，将其视为感兴趣区域送入训练好的U
2
-Net模型，进一步分割出包含心房室及瓣膜的部分。然后利用形态学算子去除其外围可能存在的少许心脏外膜区域得到四腔心内区域后，通过直方图修正与最大类间方差法（OTSU）相结合的方法分割出瓣膜连同房室间隔区域，并通过减法操作得到心室心房区域的分割图。最后通过联合胎儿四腔心超声切面图像中瓣膜连同房室间隔与心室心房区域的面积之比、瓣膜与房室间隔区域以及心室心房区域的平均灰度构建评分公式与评分标准，实现胎儿四腔心超声切面图像质量的有效评测。
结果
2
在胸腔和四腔心区域的检测任务上的mAP@0.5、mAP@0.5-0.95和召回率分别为99.5%、84.6%和99.9%；在四腔心内部区域分割任务上的灵敏度、特异度和准确度分别为95.0%、95.1%和94.9%；所提质量评测方法在所构建的A、B、C三类评测数据集上分别取得了93.7%、90.3%和99.1%的准确率。
结论
2
所提方法的评测结果与医生主观评测结果相近，具有较好的可解释性，拥有良好的实际应用价值。
Objective
2
To diagnose the fetal congenital heart disease （CHD） during screening， clinical-related ultrasound technique is adopted and focused on the captured images in terms of different critical cardiac scanning planes. Ultrasound scanning image quality assessment （QA） is indispensable for its efficiency and effectiveness. Four-chamber （4C） view-related multiple fetal cardiac ultrasound scan planes are commonly-used for CHD. The emerging artificial intelligence （AI） technique is beneficial for automatic fetal 4C view QA algorithm research further. In recent years， deep convolutional neural network （DCNN） based AI technique has been widely developing in the context of medical image processing and analysis. However， duo to the lack of relevant data-sets and the 4C region only occupying a small part in the whole fetal 4C view， the confidence of detection bounding-box from general purpose object detection network is challenged to reflect the visibility and clarity of four chambers of the heart and its related crux area well， which includes mitral valve， tricuspid valve， interatrial septum and interventricular septum in cardiac region. In addition， current fetal 4C view QA methods are still challenged for reasonable explainability based on pure deep learning （DL） techniques. To resolve this problem， we proposed a novel fetal 4C view QA algorithm through the integration of object detection and two-stage segmentation operations， which is mutual-benefited for both of DL and traditional image processing techniques to get better accuracy and interpretability.
Method
2
A self-built medical data-set of 1 696 images is used for fetal 4C view QA research. The data-set offers common objects in context （COCO） format labels for 4C and thorax regions， semantic segmentation labels for 4C inner regions， and also contains QA labels annotated manually. First， object detection network of you only look once v5x （YOLOv5x） is trained to realize effective 4C region detection and extraction. It illustrates that the 4C region’s location is normal and can be treated as the region of interest （ROI） when the detected 4C region locates inner the thorax region. And the ROI will be fed into the semantic segmentation network U
2
-Net which is well trained based the 4C inner region data-set which is a sub-set of the self-built data-set. The U
2
-Net considers four chambers of the heart and crux areas as foreground and implements the initial segmentation. And， the U
2
-Net output is a gray-scale image， in which values of background pixels are restrained effectively and foreground pixels are highlighted as well. Then， the maximum inter-class variance method （OTSU’s method） is adopted to binarize the U
2
-Net output. The morphological erosion operation is employed to optimize the binary segmentation result further， and a binary mask is produced to isolate the gray-scale 4C region. Next， OTSU-integrated histogram adjustment is used to separate the crux area leveraged from the isolated 4C region. And， the rest part of it can be considered as the four chambers of the heart area. After that， three QA indices are designed， and they can be used to represent area ration of the crux and four chambers of the heart， average gray scale of the crux and four chambers of the heart. Finally， to achieve effective fetal 4C view QA， evaluation formulations and standards are developed based on the three QA indices above.
Result
2
The experimental results show that the well trained YOLOv5x model based on the self-built data-set can achieve 99.5% mAP@0.5 and 84.6% mAP@0.5-0.95 in the object detection task for thorax and 4C regions respectively， and the recall rate is as high as 99.9%； and the well trained U
2
-Net model can achieve 95.0% sensitivity， 95.1% specificity and 94.9% accuracy in the segmentation task for 4C inner region. The proposed fetal 4C view QA method can get 93.7%， 90.3% and 99.1% accuracy on each evaluation data-set of class A， B and C.
Conclusion
2
To solve the problem that DL based image classification network cannot consider the location relationships of anatomy parts， and the object detection network cannot reflect their visibility and clarity， which leads to unreliable evaluation results， a fetal 4C view QA algorithm is proposed based on an object detection incorporated with two-stage segmentation. Evaluation results show that the trained object detection network has its potentials for 4C and thorax region detection， and the trained semantic segmentation network is optimized for 4C inner region extraction as well. The adopted two-stage segmentation strategy which combines DL and traditional image processing can not only shrink the costs of data annotation and network training greatly， but also strengthen optimal explainable results as well. The designed QA standards can be developed farther in terms of the three key indices.
深度学习（DL）卷积神经网络（CNN）超声图像质量评测目标检测两级图像分割
deep learning（DL）convolutional neural network （CNN）ultrasound image quality assessmentobject detectiontwo-stage image segmentation
Abdi A H， Luong C， Tsang T， Allan G， Nouranian S， Jue J， Hawley D， Fleming S， Gin K， Swift J， Rohling R and Abolmaesumi P. 2017. Automatic quality assessment of echocardiograms using convolutional neural networks： feasibility on the apical four-chamber view. IEEE Transactions on Medical Imaging， 36（6）： 1221-1230 ［DOI： 10.1109/TMI.2017.2690836http://dx.doi.org/10.1109/TMI.2017.2690836］
Baumgartner C F， Kamnitsas K， Matthew J， Fletcher T P， Smith S， Koch L M， Kainz B and Rueckert D. 2017. Sononet： real-time detection and localisation of fetal standard scan planes in freehand ultrasound. IEEE Transactions on Medical Imaging， 36（11）： 2204-2215 ［DOI： 10.1109/TMI.2017.2712367http://dx.doi.org/10.1109/TMI.2017.2712367］
Bochkovskiy A， Wang C Y and Liao H Y M. 2020. YOLOv4： optimal speed and accuracy of object detection ［EB/OL］. ［2022-03-28］. https://arxiv.org/pdf/2004.10934.pdfhttps://arxiv.org/pdf/2004.10934.pdf
Chen K， Wang J Q， Pang J M， Cao Y H， Xiong Y， Li X X， Sun S Y， Feng W S， Liu Z W， Xu J R， Zhang Z， Cheng D Z， Zhu C C， Cheng T H， Zhao Q J， Li B Y， Lu X， Zhu R， Wu Y， Dai J F， Wang J D， Shi J P， Ouyang W L， Loy C C and Lin D H. 2019. MMDetection： open MMLab detection toolbox and benchmark ［EB/OL］. ［2022-03-28］. https://arxiv.org/pdf/1906.07155.pdfhttps://arxiv.org/pdf/1906.07155.pdf
Chen Q， Wang Y M， Yang T， Zhang X Y， Cheng J and Sun J. 2021. You only look one-level feature//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Nashville， USA： IEEE. ［DOI： 10.1109/CVPR46437.2021.01284http://dx.doi.org/10.1109/CVPR46437.2021.01284］
Del Bianco A， Russo S， Lacerenza N， Rinaldi M， Rinaldi G， Nappi L and Greco P. 2006. Four chamber view plus three-vessel and trachea view for a complete evaluation of the fetal heart during the second trimester. Journal of Perinatal Medicine， 34（4）： 309-312 ［DOI： 10.1515/JPM.2006.059http://dx.doi.org/10.1515/JPM.2006.059］
Dong J B， Liu S F， Liao Y M， Wen H X， Lei B Y， Li S L and Wang T F. 2020. A generic quality control framework for fetal ultrasound cardiac four-chamber planes. IEEE Journal of Biomedical and Health Informatics， 24（4）： 931-942 ［DOI： 10.1109/JBHI.2019.2948316http://dx.doi.org/10.1109/JBHI.2019.2948316］
Dudley N J and Chapman E. 2002. The importance of quality management in fetal measurement. Ultrasound in Obstetrics and Gynecology， 19（2）： 190-196 ［DOI： 10.1046/j.0960-7692.2001.00549.xhttp://dx.doi.org/10.1046/j.0960-7692.2001.00549.x］
Ge Q Q， Zhang Z J， Yuan L， Li X M and Sun J M. 2021. Safety helmet wearing detection method of fusing environmental features and improved YOLOv4. Journal of Image and Graphics， 26（12）： 2904-2917
葛青青， 张智杰， 袁珑， 李秀梅， 孙军梅. 2021. 融合环境特征与改进YOLOv4的安全帽佩戴检测. 中国图象图形学报， 26（12）： 2904-2917 ［DOI： 10.11834/jig.200606http://dx.doi.org/10.11834/jig.200606］
Ge Z， Liu S T， Wang F， Li Z M and Sun J. 2021. YOLOX： exceeding YOLO series in 2021 ［EB/OL］. ［2022-03-28］. https://arxiv.org/pdf/2107.08430.pdfhttps://arxiv.org/pdf/2107.08430.pdf
Jeanty P， Chaoui R， Tihonenko I and Grochal F. 2007. A review of findings in fetal cardiac section drawings： part 1： the 4-chamber view. Journal of Ultrasound in Medicine， 26（11）： 1601-1610 ［DOI： 10.7863/jum.2007.26.11.1601http://dx.doi.org/10.7863/jum.2007.26.11.1601］
Lange L W， Sahn D J， Allen H D， Goldberg S J， Anderson C and Giles H. 1980. Qualitative real-time cross-sectional echocardiographic imaging of the human fetus during the second half of pregnancy. Circulation， 62（4）： 799-806 ［DOI： 10.1161/01.CIR.62.4.799http://dx.doi.org/10.1161/01.CIR.62.4.799］
Lin Z H， Le M H， Ni D， Chen S P， Li S L， Wang T F and Lei B Y. 2018. Quality assessment of fetal head ultrasound images based on Faster R-CNN//Proceedings of International Workshops on Simulation， Image Processing， and Ultrasound Systems for Assisted Diagnosis and Navigation. Granada， Spain： Springer， 11042： 38-46 ［DOI： 10.1007/978-3-030-01045-4_5http://dx.doi.org/10.1007/978-3-030-01045-4_5］
Otsu N. 1979. A threshold selection method from gray-level histograms. IEEE Transactions on Systems， Man， and Cybernetics， 9（1）： 62-66 ［DOI： 10.1109/TSMC.1979.4310076http://dx.doi.org/10.1109/TSMC.1979.4310076］
Qin X B， Zhang Z C， Huang C Y， Dehghan M， Zaiane O R and Jagersand M. 2020. U2-Net： going deeper with nested u-structure for salient object detection. Pattern Recognition， 106： #107404 ［DOI： 10.1016/j.patcog.2020.107404http://dx.doi.org/10.1016/j.patcog.2020.107404］
Rahmatullah B， Sarris I， Papageorghiou A and Noble J A. 2011. Quality control of fetal ultrasound images： detection of abdomen anatomical landmarks using adaboost//Proceedings of 2011 IEEE International Symposium on Biomedical Imaging： From Nano to Macro. Chicago， USA： IEEE： 6-9 ［DOI： 10.1109/ISBI.2011.5872342http://dx.doi.org/10.1109/ISBI.2011.5872342］
Redmon J， Divvala S， Girshick R and Farhadi A. 2016. You only look once： unified， real-time object detection//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Las Vegas， USA： IEEE： #91 ［DOI： 10.1109/CVPR.2016.91http://dx.doi.org/10.1109/CVPR.2016.91］
Redmon J and Farhadi A. 2017. YOLO9000： better， faster， stronger//Proceeding of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu， USA： IEEE： 6517-6525 ［DOI： 10.1109/CVPR.2017.690http://dx.doi.org/10.1109/CVPR.2017.690］
Redmon J and Farhadi A. 2018. YOLOv3： an incremental improvement ［EB/OL］. ［2022-03-28］. https://arxiv.org/pdf/1804.02767.pdfhttps://arxiv.org/pdf/1804.02767.pdf
Ren S Q， He K M， Girshick R and Sun J. 2017. Faster R-CNN： towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence， 39（6）： 1137-1149 ［DOI： 10.1109/TPAMI.2016.2577031http://dx.doi.org/10.1109/TPAMI.2016.2577031］
Ronneberger O， Fischer P and Brox T. 2015. U-Net： convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich， Germany： Springer： ［DOI： 10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28］
Salomon L J， Bernard J P， Duyme M， Doris B， Mas N and Ville Y. 2006. Feasibility and reproducibility of an image-scoring method for quality control of fetal biometry in the second trimester. Ultrasound in Obstetrics and Gynecology， 27（1）： 34-40 ［DOI： 10.1002/uog.2665http://dx.doi.org/10.1002/uog.2665］
Salomon L J and Ville Y. 2005. Quality control of prenatal ultrasound. The Ultrasound Review of Obstetrics and Gynecology， 5（4）： 297-303 ［DOI： 10.3109/14722240500415419http://dx.doi.org/10.3109/14722240500415419］
Shelhamer E， Long J and Darrell T. 2017. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence， 39（4）： 640-651 ［DOI： 10.1109/TPAMI.2016.2572683http://dx.doi.org/10.1109/TPAMI.2016.2572683］
Ultralytics. 2020. YOLOV5 ［EB/OL］. ［2022-03-28］. https://github.com/ultralytics/yolov5https://github.com/ultralytics/yolov5
Vullings R. 2019. Fetal electrocardiography and deep learning for prenatal detection of congenital heart disease//Proceedings of 2019 Computing in Cardiology. Singapore， Singapore： IEEE： 1-4 ［DOI： 10.22489/CinC.2019.072http://dx.doi.org/10.22489/CinC.2019.072］
Wang Y， Ge X K， Ma H， Qi S L， Zhang G J and Yao Y D. 2021. Deep learning in medical ultrasound image analysis： a review. IEEE Access， 9： 54310-54324 ［DOI： 10.1109/ACCESS.2021.3071301http://dx.doi.org/10.1109/ACCESS.2021.3071301］
Wu L Y， Cheng J Z， Li S L， Lei B Y， Wang T F and Ni D. 2017. FUIQA： fetal ultrasound image quality assessment with deep convolutional networks. IEEE Transactions on Cybernetics， 47（5）： 1336-1349 ［DOI： 10.1109/TCYB.2017.2671898http://dx.doi.org/10.1109/TCYB.2017.2671898］
Wu Y D， Gong M， Gu X Y， Zhang H J and He Y H. 2021. Advances in machine learning in the field of cardiac ultrasound. Journal of Clinical Ultrasound in Medicine， 23（9）： 692-694
武玉多， 贡鸣， 谷孝艳， 张宏家， 何怡华. 2021. 机器学习在心脏超声领域中的研究进展. 临床超声医学杂志， 23（9）： 692-694 ［DOI： 10.16245/j.cnki.issn1008-6978.2021.09.017http://dx.doi.org/10.16245/j.cnki.issn1008-6978.2021.09.017］
Yagel S， Cohen S M and Achiron R. 2001. Examination of the fetal heart by five short-axis views： a proposed screening method for comprehensive cardiac evaluation. Ultrasound in Obstetrics and Gynecology， 17（5）： 367-369 ［DOI： 10.1046/j.1469-0705.2001.00414.xhttp://dx.doi.org/10.1046/j.1469-0705.2001.00414.x］
Zhang B， Liu H， Luo H and Li K J. 2021a. Automatic quality assessment for 2D fetal sonographic standard plane based on multitask learning. Medicine， 100（4）： #24427 ［DOI： 10.1097/MD.0000000000024427http://dx.doi.org/10.1097/MD.0000000000024427］
Zhang S Y， Wang Y F， Jiang J Y， Dong J X， Yi W W and Hou W G. 2021b. CNN-based medical ultrasound image quality assessment. Complexity， 2021： #9938367 ［DOI： 10.1155/2021/9938367http://dx.doi.org/10.1155/2021/9938367］
Zhou Z W， Siddiquee M M R， Tajbakhsh N and Liang J M. 2018. Unet++： a nested U-Net architecture for medical image segmentation//Proceedings of the 4th International Workshop Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Granada， Spain： Springer. ［DOI： 10.1007/978-3-030-00889-5_1http://dx.doi.org/10.1007/978-3-030-00889-5_1］
Zhu J W. 2020. Unet-zoo ［EB/OL］. ［2022-03-28］. https://github.com/Andy-zhujunwen/UNET-ZOOhttps://github.com/Andy-zhujunwen/UNET-ZOO
相关文章
相关作者
相关机构
京公网安备11010802024621