浏览全部资源
扫码关注微信
纸质出版日期： 2023-04-16 ，
移动端阅览
苏逸欣， 肖志勇. 2023. 面向3维心脏MRI分割的半监督双任务交叉一致性约束网络. 中国图象图形学报， 28(04):1198-1211
Su Yixin， Xiao Zhiyong. 2023. Semi-supervised dual-task cross-consistency constraint network for 3D cardiac MRI segmentation. Journal of Image and Graphics， 28(04):1198-1211
目的
2
针对人体组织器官及病灶区域的3维图像分割是计算机辅助医疗诊断的重要前提，是医学影像3维可视化的重要技术基础。深度学习方法在医学图像分割任务中的成功通常取决于大量有标注数据。半监督学习利用未标注数据容易获取的优点，在模型训练过程中使用少量标注数据和大量未标注数据进行学习，缓解了数据标注昂贵耗时的问题，在医学图像分割中受到了广泛关注。为更好地利用无标注数据，提升医学图像分割效果，提出一种新的一致性正则方法用于半监督3维医学图像分割。
方法
2
模型以V-Net为基础架构，通过扩展网络结构，在均带有分割任务及回归任务属性的双任务主副解码器之间添加了用于正则化约束的交叉损失，构建了具有形状感知的基于双任务的交叉一致性正则网络SACC-Net（shape-aware cross-consistency regular network based on dual tasks），实现将数据层面和模型层面的扰动融合进多任务机制的一致性正则方法，使模型能够更好地利用未标注数据的有效先验信息，并且具有更好的泛化性能。
结果
2
在MICCAI 2018（Medical Image Computing and Computer Assisted Intervention Society）心房分割挑战赛公布的数据集中的3维左心房核磁共振成像上验证本文算法，在仅使用训练集中10% 的有标注数据的实验组中，Dice系数、Jaccard指数、HD（Hausdorff distance）距离和平均对称表面距离分别为88.01%、78.89%、8.19和2.09。在另一组仅使用20%的有标注数据的实验中Dice系数、Jaccard指数、HD距离和平均对称表面距离分别达到90.11%、82.11%、6.57和1.78。
结论
2
本文提出的半监督分割模型性能显著，综合了数据、模型和任务层面的一致性正则约束，与其他半监督方法相比分割效果更好且具有更佳的泛化性能。
Objective
2
Three-dimensional （3D） image segmentation of human tissues， organs and lesion areas is projected for computer-aided diagnosis and medical images-related 3D visualization. Thanks to the emerging deep learning technique， fully-supervised network models have been developing intensively in relevant to medical image segmentation tasks. However， it is challenged for a large amount of annotated data and 3D image segmentation data-labeled is costly and inefficient. Semi-supervised learning is focused on a small size of data-labeled and sufficient data-unlabeled in terms of easy acquisition of unlabeled data， which can alleviate the cost and time-consuming problem of data labeling. Our research is focused on new consistency regular method for semi-supervised 3D medical image segmentation model. To improve the medical image segmentation effect， our model can use unlabeled data through the fusion of different consistency methods.
Method
2
The network model is demonstrated on the V-Net， which can remove the residual structure of encoding and decoding. To get efficient features of unlabeled data， the proposed shape-aware cross-consistency regular network is introduced via the V-Net network structure extension on the basis of an encoder and two independent decoders-involved dual tasks （shape-aware cross-consistency regular network based on dual tasks （SACC-Net））， which are divided into a main decoder a and an auxiliary decoder b. The output of encoder-shared is transmitted to the two decoders after noise disturbance. At the same time， the two decoders can output the prediction results after each iteration. To increase the generalization and anti-noise ability of the model， it can minimize the difference between the two parts of the results during the training process. Additionally， the proportion of labeled samples in the training samples is extremely small because the feature distributions between the pre-processed medical image samples are relatively similar. To improve the learning ability of the model to segmented samples further， geometric prior information constraints are melted into the segmentation target. A shape-aware regression layer is added at the end of each decoder as well. During the training phase， each decoder can output two parts of the prediction results at the same time. That is， the total output of each iteration-after network consists of four parts. It can be used to decode the segmentation map
SA
and the signed distance map output by the decoder
A
， and the segmentation map
SB
and the signed distance map output by the decoder
B
through the dual-task consistency of each decoding part. To enhance the model’s ability and learn the effective features of segmentation targets to a greater extent， constraints and cross constraints can be used to realize a consistent regular method that combines data-level and model-level disturbances with multi-task mechanisms， and make better use of unlabeled data.
Result
2
Our algorithm is validated on the MRI data set published in Atrium Segmentation Challenge held by MICCAI （Medical Image Computing and Computer Assisted Intervention Society） in 2018. The experiment is divided into two test groups based on the amounts proportion of labeled data. In the training set， 10% annotated data is used only in the experimental group， the Dice coefficient， Jaccard index， HD （Hausdorff distance） distance， and average symmetric surface distance is reached to 88.01%， 78.89%， 8.19， and 2.09 of each. In the other group， 20% annotated data of the experiments are used only. The median Dice coefficient， Jaccard index， HD distance and average symmetric surface distance can be reached to 90.14%， 82.11%， 6.57， and 1.78 each as well. Furthermore， In respect of the shape perception method using the level set function for regression tasks， the Dice evaluation index can be improved by 0.69 and 0.60 in comparison with shape-aware semi-supervised net（SASS Net） in 10% and 20% of the marked training results. Each improvement is reached to 1.44% and 0.72% in terms of comparative results of dual-task consistency（DTC） trained with 10% and 20% labeled data.
Conclusion
2
The semi-supervised segmentation model （SACC-Net） is illustrated for the criteria optimization for both of the region and boundary-based segmentation， which can incorporate the level-consistency of its data， model and task. The constrained method has its potential segmentation effect and generalization performance for semi-supervised methods.
医学影像半监督学习3维图像分割一致性正则核磁共振图像（MRI）
medical imagingsemi-supervised learningthree-dimensional image segmentationconsistency regularizationmagnetic resonance imaging （MRI）
Bachman P， Alsharif O and Precup D. 2014. Learning with pseudo-ensembles//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal， Canada： MIT Press： 3365-3373
Bai W J， Oktay O， Sinclair M， Suzuki H， Rajchl M， Tarroni G， Glocker B， King A， Matthews P M and Rueckert D. 2017. Semi-supervised learning for network-based cardiac MR image segmentation//Proceedings of the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention. Quebec City， Canada： Springer： 253-260 ［DOI： 10.1007/978-3-319-66185-8_29http://dx.doi.org/10.1007/978-3-319-66185-8_29］
Bortsova G， Dubost F， Hogeweg L， Katramados I and De Bruijne M. 2019. Semi-supervised medical image segmentation via learning consistency under transformations//Proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen， China： Springer： 810-818 ［DOI： 10.1007/978-3-030-32226-7_90http://dx.doi.org/10.1007/978-3-030-32226-7_90］
Chen X K， Yuan Y H， Zeng G and Wang J D. 2021. Semi-supervised semantic segmentation with cross pseudo supervision//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville， USA： IEEE： 2613-2622 ［DOI： 10.1109/CVPR46437.2021.00264http://dx.doi.org/10.1109/CVPR46437.2021.00264］
Çiçek Ö， Abdulkadir A， Lienkamp S S， Brox T and Ronneberger O. 2016. 3D U-Net： learning dense volumetric segmentation from sparse annotation//Proceedings of the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention. Athens， Greece： Springer： 424-432 ［DOI： 10.1007/978-3-319-46723-8_49http://dx.doi.org/10.1007/978-3-319-46723-8_49］
Drozdzal M， Vorontsov E， Chartrand G， Kadoury S and Pal C. 2016. The importance of skip connections in biomedical image segmentation//Proceedings of the 1st International Workshop on Deep Learning in Medical Image Analysis. Athens， Greece： Springer： 179-187 ［DOI： 10.1007/978-3-319-46976-8_19http://dx.doi.org/10.1007/978-3-319-46976-8_19］
Fotedar G， Tajbakhsh N， Ananth S and Ding X W. 2020. Extreme consistency： overcoming annotation scarcity and domain shifts//Proceedings of the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention. Lima， Peru： Springer： 699-709 ［DOI： 10.1007/978-3-030-59710-8_68http://dx.doi.org/10.1007/978-3-030-59710-8_68］
Hang W L， Feng W， Liang S， Yu L Q， Wang Q， Choi K S and Qin J. 2020. Local and global structure-aware entropy regularized mean teacher model for 3D left atrium segmentation//Proceedings of the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention. Lima， Peru： Springer： 562-571 ［DOI： 10.1007/978-3-030-59710-8_55http://dx.doi.org/10.1007/978-3-030-59710-8_55］
He K H and Xiao Z Y. 2021. LRUNet： a lightweight rapid semantic segmentation network for brain tumors. Journal of Image and Graphics， 26（9）： 2233-2242
何康辉， 肖志勇. 2021. LRUNet： 轻量级脑肿瘤快速语义分割网络. 中国图象图形学报， 26（9）： 2233-2242 ［DOI： 10.11834/jig.200436http://dx.doi.org/10.11834/jig.200436］
He Y T， Yang G Y， Chen Y， Kong Y Y， Wu J S， Tang L J， Zhu X M， Dillenseger J L， Shao P F， Zhang S B， Shu H Z， Coatrieux J L and Li S. 2019. DPA-DenseBiasNet： semi-supervised 3D fine renal artery segmentation with dense biased network and deep priori anatomy//Proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen， China： Springer： 139-147 ［DOI： 10.1007/978-3-030-32226-7_16http://dx.doi.org/10.1007/978-3-030-32226-7_16］
Karamitsos T D， Francis J M， Myerson S， Selvanayagam J B and Neubauer S. 2009. The role of cardiovascular magnetic resonance imaging in heart failure. Journal of the American College of Cardiology， 54（15）： 1407-1424 ［DOI： 10.1016/j.jacc.2009.04.094http://dx.doi.org/10.1016/j.jacc.2009.04.094］
Laradji I， Rodriguez P， Mañas O， Lensink K， Law M， Kurzman L， Parker W， Vázquez D and Nowrouzezahrai D. 2021. A weakly supervised consistency-based learning method for covid-19 segmentation in CT images//Proceedings of 2021 IEEE Winter Conference on Applications of Computer Vision. Waikoloa， USA： IEEE： 2452-2461 ［DOI： 10.1109/WACV48630.2021.00250http://dx.doi.org/10.1109/WACV48630.2021.00250］
Li S L， Zhang C Y and He X M. 2020. Shape-aware semi-supervised 3D semantic segmentation for medical images//Proceedings of the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention. Lima， Peru： Springer： 552-561 ［DOI： 10.1007/978-3-030-59710-8_54http://dx.doi.org/10.1007/978-3-030-59710-8_54］
Li S M， Zhao Z Y， Xu K X， Zeng Z and Guan C T. 2021. Hierarchical consistency regularized mean teacher for semi-supervised 3D left atrium segmentation//Proceedings of the 43rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Guadalajara， Mexico： IEEE： 3395-3398 ［DOI： 10.1109/EMBC46164.2021.9629941http://dx.doi.org/10.1109/EMBC46164.2021.9629941］
Li X M， Yu L Q， Chen H， Fu C W and Heng P A. 2018. Semi-supervised skin lesion segmentation via transformation consistent self-ensembling model//Proceedings of 2018 British Machine Vision Conference. Newcastle， UK： BMVC： #63
Long J， Shelhamer E and Darrell T. 2015. Fully convolutional networks for semantic segmentation//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston， USA： IEEE： 3431-3440 ［DOI： 10.1109/CVPR.2015.7298965http://dx.doi.org/10.1109/CVPR.2015.7298965］
Luo X D， Chen J N， Song T and Wang G T. 2021a. Semi-supervised medical image segmentation through dual-task consistency. Proceedings of the AAAI Conference on Artificial Intelligence， 35（10）： 8801-8809 ［DOI： 10.1609/aaai.v35i10.17066http://dx.doi.org/10.1609/aaai.v35i10.17066］
Luo X D， Liao W J， Chen J N， Song T， Chen Y N， Zhang S C， Chen N Y， Wang G T and Zhang S T. 2021b. Efficient semi-supervised gross target volume of nasopharyngeal carcinoma segmentation via uncertainty rectified pyramid consistency//Proceedings of the 24th International Conference on Medical Image Computing and Computer-Assisted Intervention. Strasbourg， France： Springer： 318-329 ［DOI： 10.1007/978-3-030-87196-3_30http://dx.doi.org/10.1007/978-3-030-87196-3_30］
Mao X J， Shen C H and Yang Y B. 2016. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona， Spain： Curran Associates Inc.： 2810-2818
Milletari F， Navab N and Ahmadi S A. 2016. V-Net： fully convolutional neural networks for volumetric medical image segmentation//Proceedings of the 4th International Conference on 3D Vision （3DV）. Stanford， USA： IEEE： 565-571 ［DOI： 10.1109/3DV.2016.79http://dx.doi.org/10.1109/3DV.2016.79］
Mittal S， Tatarchenko M and Brox T. 2021. Semi-supervised semantic segmentation with high-and low-level consistency. IEEE Transactions on Pattern Analysis and Machine Intelligence， 43（4）： 1369-1379 ［DOI： 10.1109/TPAMI.2019.2960224http://dx.doi.org/10.1109/TPAMI.2019.2960224］
Miyato T， Maeda S I， Koyama M and Ishii S. 2019. Virtual adversarial training： a regularization method for supervised and semi-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence， 41（8）： 1979-1993 ［DOI： 10.1109/tpami.2018.2858821http://dx.doi.org/10.1109/tpami.2018.2858821］
Nie D， Gao Y， Wang L and Shen D. 2018. ASDNet： attention based semi-supervised deep networks for medical image segmentation//Proceedings of 2018 International Conference on Medical Image Computing and Computer-Assisted Intervention. Granada， Spain： Springer： 370-378［DOI： 10.1007/978-3-030-00937-3_43http://dx.doi.org/10.1007/978-3-030-00937-3_43］
Orhan A E and Pitkow X. 2018. Skip connections eliminate singularities//Proceedings of the 6th International Conference on Learning Representations. Vancouver， Canada： ICLR
Ouali Y， Hudelot C and Tami M. 2020. Semi-supervised semantic segmentation with cross-consistency training//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle， USA： IEEE： 12671-12681 ［DOI： 10.1109/CVPR42600.2020.01269http://dx.doi.org/10.1109/CVPR42600.2020.01269］
Peng J Z， Pedersoli M and C. Desrosiers. 2020. Mutual information deep regularization for semi-supervised segmentation//Proceedings of 2020 International Conference on Medical Imaging with Deep Learning. Montréal， Canada： MIDL： 601-613
Pop M， Sermesant M， Zhao J， Li S， McLeod K， Young A A， Rhode K and Mansi T. 2019. Statistical atlases and computational models of the heart. Atrial segmentation and LV quantification challenges//Proceedings of the 9th International Workshop. Granada， Spain： Springer： #11395 ［DOI： 10.1007/978-3-030-12029-0http://dx.doi.org/10.1007/978-3-030-12029-0］
Rizve M N， Duarte K， Rawat Y S and Shah M. 2021. In defense of pseudo-labeling： an uncertainty-aware pseudo-label selection framework for semi-supervised learning ［EB/OL］. ［2021-10-13］. https://arxiv.org/pdf/2101.06329.pdfhttps://arxiv.org/pdf/2101.06329.pdf
Ronneberger O， Fischer P and Brox T. 2015. U-net： convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich， Germany： Springer： 234-241 ［DOI： 10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28］
Sajjadi M， Javanmardi M and Tasdizen T. 2016. Regularization with stochastic transformations and perturbations for deep semi-supervised learning//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona， Spain： Curran Associates Inc.： 1171-1179
Sohn K， Berthelot D， Li C L， Zhang Z Z， Carlini N， Cubuk E D， Kurakin A， Zhang H and Raffel C. 2020. FixMatch： simplifying semi-supervised learning with consistency and confidence ［EB/OL］. ［2021-10-13］. https://arxiv.org/pdf/2001.07685.pdfhttps://arxiv.org/pdf/2001.07685.pdf
Suinesiaputra A， Cowan B R， Finn J P， Fonseca C G， Kadish A H， Lee D C， Medrano-Gracia P， Warfield S K， Tao W C and Young A A. 2011. Left ventricular segmentation challenge from cardiac MRI： a collation study//Proceedings of the 2nd International Workshop on Statistical Atlases and Computational Models of the Heart. Toronto， Canada： Springer： 88-97 ［DOI： 10.1007/978-3-642-28326-0_9http://dx.doi.org/10.1007/978-3-642-28326-0_9］
Tarvainen A and Valpola H. 2017. Mean teachers are better role models： weight-averaged consistency targets improve semi-supervised deep learning results//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach， USA： Curran Associates Inc.： 1195-1204
Tobon-Gomez C， Geers A J， Peters J， Weese J， Pinto K， Karim R， Ammar M， Daoudi A， Margeta J， Sandoval Z， Stender B， Zheng Y F， Zuluaga M A， Betancur J， Ayache N， Chikh M A， Dillenseger J L， Kelm B M， Mahmoudi S， Ourselin S， Schlaefer A， Schaeffter T， Razavi R and Rhode K S. 2015. Benchmark for algorithms segmenting the left atrium from 3D CT and MRI datasets. IEEE Transactions on Medical Imaging， 34（7）： 1460-1473 ［DOI： 10.1109/TMI.2015.2398818http://dx.doi.org/10.1109/TMI.2015.2398818］
Wang Y L， Huang G， Song S J， Pan X R， Xia Y T and Wu C. 2022. Regularizing deep networks with semantic data augmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence， 44（7）： 3733-3748 ［DOI： 10.1109/TPAMI.2021.3052951http://dx.doi.org/10.1109/TPAMI.2021.3052951］
Wang Y X， Zhang Y， Tian J， Zhong C， Shi Z C， Zhang Y and He Z Q. 2020. Double-uncertainty weighted method for semi-supervised learning//Proceedings of the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention. Lima， Peru： Springer： 542-551 ［DOI： 10.1007/978-3-030-59710-8_53http://dx.doi.org/10.1007/978-3-030-59710-8_53］
Xiong Z H， Fedorov V V， Fu X H， Cheng E， Macleod R and Zhao J C. 2019. Fully automatic left atrium segmentation from late gadolinium enhanced magnetic resonance imaging using a dual fully convolutional neural network. IEEE Transactions on Medical Imaging， 38（2）： 515-524 ［DOI： 10.1109/TMI.2018.2866845http://dx.doi.org/10.1109/TMI.2018.2866845］
Xiong Z H， Xia Q， Hu Z Q， Huang N， Bian C， Zheng Y F， Vesal S， Ravikumar N， Maier A， Yang X， Heng P A， Ni D， Li C Z， Tong Q Q， Si W X， Puybareau E， Khoudli Y， Géraud T， Chen C， Bai W J， Rueckert D， Xu L C， Zhuang X H， Luo X Z， Jia S M， Sermesant M， Liu Y S， Wang K Q， Borra D， Masci A， Corsi C， De Vente C， Veta M， Karim R， Preetha C J， Engelhardt S， Qiao M Y， Wang Y Y， Tao Q， Nuñez-Garcia M， Camara O， Savioli N， Lamata P and Zhao J C. 2021. A global benchmark of algorithms for segmenting the left atrium from late gadolinium-enhanced cardiac magnetic resonance imaging. Medical Image Analysis， 67： #101832 ［DOI： 10.1016/j.media.2020.101832http://dx.doi.org/10.1016/j.media.2020.101832］
You C Y， Zhao R H， Staib L H and Duncan J S. 2022. Momentum contrastive voxel-wise representation learning for semi-supervised volumetric medical image segmentation//Proceedings of the 25th International Conference on Medical Image Computing and Computer-Assisted Intervention. Singapore， Singapore： Springer： 639-652 ［DOI： 10.1007/978-3-031-16440-8_61http://dx.doi.org/10.1007/978-3-031-16440-8_61］
Yu L Q， Wang S J， Li X M， Fu C W and Heng P A. 2019. Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation//Proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen， China： Springer： 605-613 ［DOI： 10.1007/978-3-030-32245-8_67http://dx.doi.org/10.1007/978-3-030-32245-8_67］
Zhang Y Z， Yang L， Chen J X， Fredericksen M， Hughes D P and Chen D Z. 2017. Deep adversarial networks for biomedical image segmentation utilizing unannotated images//Proceedings of the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention. Quebec City， Canada： Springer： 408-416 ［DOI： 10.1007/978-3-319-66179-7_47http://dx.doi.org/10.1007/978-3-319-66179-7_47］
Zheng H， Lin L F， Hu H J， Zhang Q W， Chen Q Q， Iwamoto Y， Han X H， Chen Y W， Tong R F and Wu J. 2019. Semi-supervised segmentation of liver using adversarial learning with deep atlas prior//Proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen， China： Springer： 148-156 ［DOI： 10.1007/978-3-030-32226-7_17http://dx.doi.org/10.1007/978-3-030-32226-7_17］
Zhou Y N， Chen H， Lin H J and Heng P A. 2020. Deep semi-supervised knowledge distillation for overlapping cervical cell instance segmentation//Proceedings of the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention. Lima， Peru： Springer： 521-531 ［DOI： 10.1007/978-3-030-59710-8_51http://dx.doi.org/10.1007/978-3-030-59710-8_51］
相关作者
相关机构
京公网安备11010802024621