深度医学图像配准研究进展：迈向无监督学习

马露凡; 罗凤; 严江鹏; 徐哲; 罗捷; 李秀

doi:10.11834/jig.200361

综述 | 浏览量 : 0 下载量: 430 CSCD: 6

PDF
导出
分享
收藏
专辑

深度医学图像配准研究进展：迈向无监督学习
Deep-learning based medical image registration pathway: towards unsupervised learning
2021年26卷第9期页码：2037-2057
收稿：2020-07-09，

修回：2020-9-22，

录用：2020-9-29，

纸质出版：2021-09-16
DOI： 10.11834/jig.200361
稿件说明：

移动端阅览

马露凡, 罗凤, 严江鹏, 徐哲, 罗捷, 李秀. 深度医学图像配准研究进展：迈向无监督学习[J]. 中国图象图形学报, 2021,26(9):2037-2057. DOI： 10.11834/jig.200361.

Lufan Ma, Feng Luo, Jiangpeng Yan, Zhe Xu, Jie Luo, Xiu Li. Deep-learning based medical image registration pathway: towards unsupervised learning[J]. Journal of Image and Graphics, 2021, 26(9): 2037-2057. DOI： 10.11834/jig.200361.

摘要

在疾病诊断、手术引导及放射性治疗等图像辅助诊疗场景中，将不同时间、不同模态或不同设备的图像通过合理的空间变换进行配准是必要的处理流程之一。随着深度学习的快速发展，基于深度学习的医学图像配准研究以其耗时短、精度高的优势吸引了研究者的广泛关注。本文全面整理了2015—2019年深度医学图像配准方向的论文，系统地分析了深度医学图像配准领域的最新研究进展，展现了深度配准算法研究从迭代优化到一步预测、从有监督学习到无监督学习的总体发展趋势。具体来说，本文在界定深度医学图像配准问题和介绍配准研究分类方法的基础上，以相关算法的网络训练过程中所使用的监督信息多少作为分类标准，将深度医学图像配准划分为全监督、双监督与弱监督、无监督医学图像配准方法。全监督配准方法通过采用随机变换、传统算法和模型生成等方式获取近似的金标准作为监督信息；双监督、无监督配准方法通过引入图像相似度损失、标签相似度损失等其他监督信息以降低对金标准的依赖；无监督配准方法则完全消除对标注数据的需要，仅使用图像相似度损失和正则化损失监督网络训练。目前，无监督医学图像算法已经成为医学图像配准领域的研究重点，在无需获得代价高昂的标注信息下就能够取得与有监督和传统方法相当甚至更高的配准精度。在此基础上，本文进一步讨论了医学图像配准研究后续可能的4个未来挑战，希望能够为更高精度、更高效率的深度医学图像配准算法的研究提供方向，并推动深度医学图像配准技术在临床诊疗中落地应用。

Abstract

Medical image registration (MIR) has aimed to implement the optimal transformation via aligning anatomical structures of a pair of medical images spatially. The crucial clinical applications like disease diagnosis

surgical guidance and radiation therapy have been envolved. Scholors have categorized MIR into inter-/intra-patient registration

uni-/multi-modal registration and rigid/non-rigid registration. Image classification has been developing deep learning-based (DL-based) MIR methods. The DL-based MIR has demonstrated substantial improvement in computational efficiency and task-specific registration accuracy over traditional iterative registration approaches. A sophisticated literature review of DL-based MIR have benefited to the disciplines. The current MIR has been analysed based on iterative optimization to one-step prediction and supervised learning to unsupervised learning. The DL-based MIR has been classified into fully supervised

dual supervised

weakly supervised and unsupervised approaches to train the DL network via the amount of supervision. Each category has been systematically reviewed. At the beginning

fully supervised methods have been reviewed in terms of the initial exploration to remove the time-consuming with low inference speed issues of deep iterative registration algorithms (deep similarity-based registration

reinforcement learning-based registration). One-step fully supervised registration has predicted the final transformation. The lack of training datasets with ground-truth transformations have barriered to train a fully supervised registration network. Most scholors have generated synthesized transformations with the following three approaches as below: 1) random augmentation-based generation; 2) traditional registration-based generation; 3) model-based generation. Next

the integration of dual-supervised and weak supervised registration have alleviated the reliance on ground truth compared with fully supervised approaches via the transition technologies between fully supervised and unsupervised methods. Dual-supervised registration frameworks have integrated image similarity metric to supervise the training. Weak supervised registration in the context of anatomical labels of interest (solid organs

vessels

ducts

structure boundaries and other subject-specific ad hoc landmarks) has replaced ground truth. The label similarity using label-driven supervised registration has facilitated the network to directly estimate the transformation for paired fixed image and moving image. The end-to-end unsupervision has been used to indicate the DL-based medical image registration evolved into the unsupervised field gradually. The unsupervision has avoided the acquisition of ground-truth transformations and segmentation labels for the supervised methods. Unsupervised registration frameworks have performed spatial data based on spatial transformer network (STN) to flat image similarity loss calculation during the training process with unknown transformations further. The latest developments and applications of DL-based unsupervised registration methods have been summarized from the aspects of loss functions and network architectures. DL-based unsupervised registration algorithms on liver CT(computed tomography) scan datasets have also been re-implemented. The demonstrated analyses have the priority to baseline model. At the end

the potentials and possibilities have been illustrated as following: 1) constructing more robust similarity metrics and more effective regularization terms to deal with multi-modality MIR; 2) quantifying registration result confidence of various DL-based models or integrating domain knowledge into current data-driven networks; 3) designing more qualified networks with fewer parameters (e.g.

3D convolution factorization

capsule network architecture).

关键词

Keywords

references

Andrade N, Faria F A and Cappabianco F A M. 2018. A practical review on medical image registration: from rigid to deep learning based approaches//Proceedings of the 31st SIBGRAPI Conference on Graphics, Patterns and Images. Parana, Brazil: IEEE: 463-470[ DOI: 10.1109/sibgrapi.2018.00066 http://dx.doi.org/10.1109/sibgrapi.2018.00066 ]

Arsigny V, Commowick O, Pennec X and Ayache N. 2006. A log-Euclidean framework for statistics on diffeomorphisms//Proceedings of the 9th International Conference on Medical Image Computing and Computer-Assisted Intervention. Copenhagen, Denmark: Springer: 924-931[ DOI: 10.1007/11866565_113 http://dx.doi.org/10.1007/11866565_113 ]

Ashburner J. 2007. A fast diffeomorphic image registration algorithm. NeuroImage, 38(1): 95-113[DOI: 10.1016/j.neuroimage.2007.07.007]

Auer M, Regitnig P and Holzapfel G A. 2005. An automatic nonrigid registration for stained histological sections. IEEE Transactions on Image Processing, 14(4): 475-486[DOI: 10.1109/TIP.2005.843756]

Avants B B, Epstein C L, Grossman M and Gee J C. 2008. Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Medical Image Analysis, 12(1): 26-41[DOI: 10.1016/j.media.2007.06.004]

Avants B B, Tustison N J, Song G, Cook P A, Klein A and Gee J C. 2011. A reproducible evaluation of ANTs similarity metric performance in brain image registration. NeuroImage, 54(3): 2033-2044[DOI: 10.1016/j.neuroimage.2010.09.025]

Balakrishnan G, Zhao A, Sabuncu M R, Dalca A V and Guttag J V. 2018. An unsupervised learning model for deformable medical image registration//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE: 9252-9260[ DOI: 10.1109/cvpr.2018.00964 http://dx.doi.org/10.1109/cvpr.2018.00964 ]

Balakrishnan G, Zhao A, Sabuncu M R, Guttag J and Dalca A V. 2019. VoxelMorph: a learning framework for deformable medical image registration. IEEE Transactions on Medical Imaging, 38(8): 1788-1801[DOI: 10.1109/TMI.2019.2897538]

Barnea D I and Silverman H F. 1972. A class of algorithms for fast digital image registration. IEEE Transactions on Computers, C-21(2): 179-186[DOI: 10.1109/TC.1972.5008923]

Berendsen F F, Kotte A N T J, Viergever M A and Pluim J P W. 2014. Registration of organs with sliding interfaces and changing topologies//Proceedings of SPIE 9034, Medical Imaging 2014: Image Processing. San Diego, USA: SPIE: 3877-2877[ DOI: 10.1117/12.2043447 http://dx.doi.org/10.1117/12.2043447 ]

Boveiri H R, Khayami R, Javidan R and Mehdizadeh A R. 2020. Medical image registration using deep neural networks: a comprehensive review[EB/OL]. [2020-07-04] . https://arxiv.org/pdf/2002.03401.pdf https://arxiv.org/pdf/2002.03401.pdf

Cao X H, Yang J H, Wang L, Xue Z, Wang Q and Shen D G. 2018b. Deep learning based inter-modality image registration supervised by intra-modality similarity//Proceedings of the 9th International Workshop on Machine Learning in Medical Imaging. Granada, Spain: Springer: 55-63[ DOI: 10.1007/978-3-030-00919-9_7 http://dx.doi.org/10.1007/978-3-030-00919-9_7 ]

Cao X H, Yang J H, Zhang J, Nie D, Kim M, Wang Q and Shen D G. 2017. Deformable image registration based on similarity-steered CNN regression//Proceedings of the 20th Medical Image Computing and Computer-Assisted Intervention. Quebec City, Canada: Springer: 300-308[ DOI: 10.1007/978-3-319-66182-7_35 http://dx.doi.org/10.1007/978-3-319-66182-7_35 ]

Cao X H, Yang J H, Zhang J, Wang Q, Yap P T and Shen D G. 2018a. Deformable image registration using a cue-aware deep regression network. IEEE Transactions on Biomedical Engineering, 65(9): 1900-1911[DOI: 10.1109/TBME.2018.2822826]

Chee E and Wu Z Z. 2018. AIRNet: self-supervised affine registration for 3D medical images using neural networks[EB/OL]. [2020-07-04] . https://arxiv.org/pdf/1810.02583.pdf https://arxiv.org/pdf/1810.02583.pdf

Cheng X, Zhang L and Zheng Y F. 2018. Deep similarity learning for multimodal medical images. Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization, 6(3): 248-252[DOI: 10.1080/21681163.2015.1135299]

Choi Y, Choi M, Kim M, Ha J W, Kim S and Choo J. 2018. StarGAN: unified generative adversarial networks for multi-domain image-to-image translation//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE: 8789-8797[ DOI: 10.1109/cvpr.2018.00916 http://dx.doi.org/10.1109/cvpr.2018.00916 ]

Chollet F. 2017. Xception: deep learning with depthwise separable convolutions//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 1800-1807[ DOI: 10.1109/CVPR.2017.195 http://dx.doi.org/10.1109/CVPR.2017.195 ]

Çiçek O, Abdulkadir A, Lienkamp S S, Brox T and Ronneberger O. 2016. 3D U-Net: learning dense volumetric segmentation from sparse annotation//Proceedings of the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention. Athens, Greece: Springer: 424-432[ DOI: 10.1007/978-3-319-46723-8_49 http://dx.doi.org/10.1007/978-3-319-46723-8_49 ]

de Vos B D, Berendsen F F, Viergever M A, Sokooti H, Staring M and Išgum I. 2019. A deep learning framework forunsupervised affine and deformable image registration. Medical Image Analysis, 52: 128-143[DOI: 10.1016/j.media.2018.11.010]

de Vos B D, Berendsen F F, Viergever M A, Staring M and Išgum I. 2017. End-to-end unsupervised deformable image registration with a convolutional neural network//Proceedings of the 3rd International Workshop on Multimodal Learning for Clinical Decision Support. Québec City, Carada: Springer: 204-212[ DOI: 10.1007/978-3-319-67558-9_24 http://dx.doi.org/10.1007/978-3-319-67558-9_24 ]

Dosovitskiy A, Fischer P, Ilg E, Häusser P, Hazirbas C, Golkov V, van der Smagt P, Cremers D and Brox T. 2015. FlowNet: learning optical flow with convolutional networks//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE: 2758-2766[ DOI: 10.1109/ICCV.2015.316 http://dx.doi.org/10.1109/ICCV.2015.316 ]

Elmahdy M S, Wolterink J M, Sokooti H, Išgum I and Staring M. 2019. Adversarial optimization for joint registration and segmentation in prostate CT radiotherapy//Proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen, China: Springer: 366-374[ DOI: 10.1007/978-3-030-32226-7_41 http://dx.doi.org/10.1007/978-3-030-32226-7_41 ]

Eppenhof K A J, Lafarge M W, Moeskops P, Veta M and Pluim J P W. 2018. Deformable image registration using convolutional neural networks//Proceedings of SPIE 10574, Medical Imaging 2018: Image Processing. Houston, USA: SPIE: #105740S[ DOI: 10.1117/12.2292443 http://dx.doi.org/10.1117/12.2292443 ]

Eppenhof K A J and Pluim J P W. 2018. Error estimation of deformable image registration of pulmonary CT scans using convolutional neural networks. Journal of Medical Imaging, 5(2): #024003[DOI: 10.1117/1.JMI.5.2.024003]

Eppenhof K A J and Pluim J P W. 2019. Pulmonary CT registration through supervised learning with convolutional neural networks. IEEE Transactions on Medical Imaging, 38(5): 1097-1105[DOI: 10.1109/TMI.2018.2878316]

Fan J F, Cao X H, Wang Q, Yap P T and Shen D G. 2019b. Adversarial learning for mono- or multi-modal registration. Medical Image Analysis, 58: #101545[DOI: 10.1016/j.media.2019.101545]

Fan J F, Cao X H, Xue Z, Yap P T and Shen D G. 2018. Adversarial similarity network for evaluating image alignment in deep learning based registration//Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention. Granada, Spain: Springer: 739-747[ DOI: 10.1007/978-3-030-00928-1_83 http://dx.doi.org/10.1007/978-3-030-00928-1_83 ]

Fan J F, Cao X H, Yap P T and Shen D G. 2019a. BIRNet: brain image registration using dual-supervised fully convolutional networks. Medical Image Analysis, 54: 193-206[DOI: 10.1016/j.media.2019.03.006]

Fu Y B, Lei Y, Wang T H, Curran W J, Liu T and Yang X F. 2020a. Deep learning in medical image registration: a review. Physics in Medicine and Biology, 62(20):#20TR01[DOI: 10.1088/1361-6560/ab843e]

Fu Y B, Lei Y, Wang T H, Higgins K, Bradley J D, Curran W J, Liu T and Yang X. 2020b. LungRegNet: an unsupervised deformable image registration method for 4D-CT lung. Medical Physics, 47(4): 1763-1774[DOI: 10.1002/mp.14065]

Gendrin C, Furtado H, Weber C, Bloch C, Figl M, Pawiro S A, Bergmann H, Stock M, Fichtinger G, Georg D and Birkfellner W. 2012. Monitoring tumor motion by real time 2D/3D registration during radiotherapy. Radiotherapy and Oncology, 102(2): 274-280[DOI: 10.1016/j.radonc.2011.07.031]

Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A and Bengio Y. 2014. Generative adversarial nets//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montréal, Canada: NIPS: 2672-2680[ DOI: 10.5555/2969033.2969125 http://dx.doi.org/10.5555/2969033.2969125 ]

Hatt C R, Speidel M A and Raval A N. 2015. Robust 5DOF transesophageal echo probe tracking at fluoroscopic frame rates//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: Springer: 290-297[ DOI: 10.1007/978-3-319-24553-9_36 http://dx.doi.org/10.1007/978-3-319-24553-9_36 ]

Heinrich M P, Jenkinson M, Bhushan M, Matin T, Gleeson F V, Brady S M and Schnabel J A. 2012. MIND: modality independent neighbourhood descriptor for multi-modal deformable registration. Medical Image Analysis, 16(7): 1423-1435[DOI: 10.1016/j.media.2012.05.008]

Hering A, Kuckertz S, Heldmann S and Heinrich M P. 2019. Enhancing label-driven deep deformable image registration with local distance metrics for state-of-the-art cardiac motion tracking//Proceedings of Bildverarbeitung für die Medizin. Lubeck, Germany: Springer: 309-314[ DOI: 10.1007/978-3-658-25326-4_69 http://dx.doi.org/10.1007/978-3-658-25326-4_69 ]

Hu Y P, Gibson E, Ghavami N, Bonmati E, Moore C M, Emberton M, Vercauteren T, Noble J A and Barratt D C. 2018b. Adversarial deformation regularization for training image registration neural networks//Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention. Granada, Spain: Springer: 774-782[ DOI: 10.1007/978-3-030-00928-1_87 http://dx.doi.org/10.1007/978-3-030-00928-1_87 ]

Hu Y P, Modat M, Gibson E, Ghavami N, Bonmati E, Moore C M, Emberton M, Noble J A, Barratt D C and Vercauteren T. 2018a. Label-driven weakly-supervised learning for multimodal deformable image registration//The 15th IEEE International Symposium on Biomedical Imaging. Washington, USA: IEEE: 1070-1074[ DOI: 10.1109/ISBI.2018.8363756 http://dx.doi.org/10.1109/ISBI.2018.8363756 ]

Hu Y P, Modat M, Gibson E, Li W Q, Ghavami N, Bonmati E, Wang G T, Bandula S, Moore C M, Emberton M, Ourselin S, N J A, Barratt D C and Vercauteren T. 2018c. Weakly-supervised convolutional neural networks for multimodal image registration. Medical Image Analysis, 49: 1-13[DOI: 10.1016/j.media.2018.07.002]

Iandola F, Moskewicz M, Karayev S, Girshick R, Darrell T and Keutzer K. 2014. DenseNet: implementing efficient convnet descriptor pyramids[EB/OL]. [2020-07-04] . https://arxiv.org/pdf/1404.1869.pdf https://arxiv.org/pdf/1404.1869.pdf

Isola P, Zhu J Y, Zhou T H and Efros A A. 2017. Image-to-image translation with conditional adversarial networks//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 5967-5976[ DOI: 10.1109/CVPR.2017.632 http://dx.doi.org/10.1109/CVPR.2017.632 ]

Jaderberg M, Simonyan K, Zisserman A and Kavukcuoglu K. 2015. Spatial transformer networks//Proceedings of the 28th International Conference on Neural Information Processing Systems. Montreal, Canada: NIPS: 2017-2025[ DOI: 10.5555/2969442.2969465 http://dx.doi.org/10.5555/2969442.2969465 ]

Jiang Z R, Yin F F, Ge Y and Ren L. 2020. A multi-scale framework with unsupervised joint training of convolutional neural networks for pulmonary deformable image registration. Physics in Medicine and Biology, 65(1): #015011[DOI: 10.1088/1361-6560/ab5da0]

Kabus S, Netsch T, Fischer B and Modersitzki J. 2004. B-spline registration of 3D images with Levenberg-Marquardt optimization//Proceedings of SPIE 5370, Medical Imaging 2004: Image Processing. San Diego, USA: SPIE:304-313[ DOI: 10.1117/12.533976 http://dx.doi.org/10.1117/12.533976 ]

Kim B, Kim J, Lee J G, Kim D H, Park S H and Ye J C. 2019. Unsupervised deformable image registration using cycle-consistent CNN//Proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen, China: Springer: 166-174[ DOI: 10.1007/978-3-030-32226-7_19 http://dx.doi.org/10.1007/978-3-030-32226-7_19 ]

Kori A and Krishnamurthi G. 2019. Zero shot learning for multi-modal real time image registration[EB/OL]. [2020-07-04] . https://arxiv.org/pdf/1908.06213.pdf https://arxiv.org/pdf/1908.06213.pdf

Krebs J, Mansi T, Delingette H, Zhang L, Ghesu F C, Miao S, Maier A K, Ayache N, Liao R and Kamen A. 2017. Robust non-rigid registration through agent-based action learning//Proceedings of the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention. Quebec City, Canada: Springer: 344-352[ DOI: 10.1007/978-3-319-66182-7_40 http://dx.doi.org/10.1007/978-3-319-66182-7_40 ]

Krizhevsky A, Sutskever I and Hinton G E. 2012. ImageNet classification with deep convolutional neural networks//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, USA: NIPS: 1097-1105[ DOI: 10.5555/2999134.2999257 http://dx.doi.org/10.5555/2999134.2999257 ]

Kuang D Y. 2019. On reducing negative Jacobian determinant of the deformation predicted by deep registration networks[EB/OL]. [2020-07-04] . https://arxiv.org/pdf/1907.00068.pdf https://arxiv.org/pdf/1907.00068.pdf

Lei Y, Fu Y B, Harms J, Wang T H, Curran J W, Liu T, Higgins X and Yang X F. 2019. 4D-CT deformable image registration using an unsupervised deep convolutional neural network//Proceedings of the 1st Workshop on Artificial Intelligence in Radiation Therapy. Shenzhen, China: Springer: 26-33[ DOI: 10.1007/978-3-030-32486-5_4 http://dx.doi.org/10.1007/978-3-030-32486-5_4 ]

Li H M and Fan Y. 2018. Non-rigid image registration using self-supervised fully convolutional networks without training data//The 15th IEEE International Symposium on Biomedical Imaging. Washington, USA: IEEE: 1075-1078[ DOI: 10.1109/isbi.2018.8363757 http://dx.doi.org/10.1109/isbi.2018.8363757 ]

Li X F, Zhang C L, Li H P and Zang X B. 2010. Development of medical image registration technology. Computer Science, 37(7): 27-33

李雄飞, 张存利, 李鸿鹏, 臧雪柏. 2010. 医学图像配准技术进展. 计算机科学, 37(7): 27-33[DOI: 10.3969/j.issn.1002-137X.2010.07.006]

Liao R, Miao S, De Tournemire P, Grbic S, Kamen A, Mansi T and Comaniciu D. 2016. An artificial agent for robust image registration[EB/OL]. [2020-07-04] . https://arxiv.org/pdf/1611.10336.pdf https://arxiv.org/pdf/1611.10336.pdf

Lin J Q, Han J, Yuan Z M and Peng J L. 2020. Orientation correction for CT images via multitask deep convolutional network. Journal of Huaqiao University (Natural Science), 41(3): 366-373

林家庆, 韩娟, 袁直敏, 彭佳林. 2020. 多任务深度卷积网络的CT图像方向校正. 华侨大学学报(自然科学版), 41(3): 366-373[DOI: 10.11830/ISSN.1000-5013.201911039]

Liu C, Ma L H, Lu Z M, Jin X C and Xu J Y. 2019. Multimodal medical image registration via common representations learning and differentiable geometric constraints. Electronics Letters, 55(6): 316-318[DOI: 10.1049/el.2018.6713]

Liu M Y, Breuel T and Kautz J. 2017. Unsupervised image-to-image translation networks//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA: NIPS: 700-708[ DOI: 10.5555/3294771.3294838 http://dx.doi.org/10.5555/3294771.3294838 ]

Liu Q G and Leung H. 2017. Tensor-based descriptor for image registration via unsupervised network//Proceedings of the 20th IEEE International Conference on Information Fusion. Xi'an, China: IEEE: 1-7[ DOI: 10.23919/ICIF.2017.8009689 http://dx.doi.org/10.23919/ICIF.2017.8009689 ]

Long J, Shelhamer E and Darrell T. 2015. Fully convolutional networks for semantic segmentation//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE: 3431-3440[ DOI: 10.1109/CVPR.2015.7298965 http://dx.doi.org/10.1109/CVPR.2015.7298965 ]

Luo J, Frisken S, WangD, Golby A J, Sugiyama M and Wells Ⅲ W. 2020a. Are registration uncertainty and error monotonically associated?//Proceedings of the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention. Lima, Peru: Springer: 264-274[ DOI: 10.1007/978-3-030-59716-0_26 http://dx.doi.org/10.1007/978-3-030-59716-0_26 ]

Luo J, Ma G S, Frisken S, Juvekar P, Haouchine N, Xu Z, Xiao Y M, Golby A, Codd P, Sugiyama M and Wells Ⅲ W. 2020b. Do public datasets assure unbiased comparisons for registration evaluation?[EB/OL]. [2020-07-04] . https://arxiv.org/pdf/2003.09483.pdf https://arxiv.org/pdf/2003.09483.pdf

Luo J, Sedghi A, Popuri K, Cobzas D, Zhang M M, Preiswerk F, Toews M, Golby A, Sugiyama M, Wells Ⅲ W M and Frisken S. 2019. On the applicability of registration uncertainty//Proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen, China: Springer: 410-419[ DOI: 10.1007/978-3-030-32245-8_46 http://dx.doi.org/10.1007/978-3-030-32245-8_46 ]

Luo J, Toews M, Machado I, Frisken S F, Zhang M M, Preiswerk F, Sedghi A, Ding H Y, Pieper S, Golland P, Golby A, Sugiyama M and Wells Ⅲ W M. 2018. A feature-driven active framework for ultrasound-based brain shift compensation//Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention. Granada, Spain: Springer: 30-38[ DOI: 10.1007/978-3-030-00937-3_4 http://dx.doi.org/10.1007/978-3-030-00937-3_4 ]

Luo S Q and Lyu W X. 1999. Medical image registration technology. Foreign medical science. International Journal of Biomedical Engineering, (1): 1-8

罗述谦, 吕维雪. 1999. 医学图像配准技术: 国外医学. 生物医学工程分册, (1): 1-8

Mahapatra D. 2018. Elastic registration of medical images with GANs[EB/OL]. [2020-07-04] . https://arxiv.org/pdf/1805.02369v1.pdf https://arxiv.org/pdf/1805.02369v1.pdf

Mahapatra D, Antony B, Sedai S and Garnavi R. 2018b. Deformable medical image registration using generative adversarial networks//The 15th IEEE InternationalSymposium on Biomedical Imaging. Washington, USA: IEEE: 1449-1453[ DOI: 10.1109/isbi.2018.8363845 http://dx.doi.org/10.1109/isbi.2018.8363845 ]

Mahapatra D, Ge Z Y, Sedai S and Chakravorty R. 2018a. Joint registration and segmentation of xray images using generative adversarial networks//Proceedings of the 9th International Workshop on Machine Learning in Medical Imaging. Granada, Spain: Springer: 73-80[ DOI: 10.1007/978-3-030-00919-9_9 http://dx.doi.org/10.1007/978-3-030-00919-9_9 ]

Miao S, Wang Z J and Liao R. 2016a. A CNN regression approach for real-time 2D/3D registration. IEEE Transactions on Medical Imaging, 35(5): 1352-1363[DOI: 10.1109/TMI.2016.2521800]

Miao S, Wang Z J, Zheng Y F and Liao R. 2016b. Real-time 2D/3D registration via CNN regression//The 13th IEEE International Symposium on Biomedical Imaging. Prague, Czech: IEEE: 1430-1434[ DOI: 10.1109/ISBI.2016.7493536 http://dx.doi.org/10.1109/ISBI.2016.7493536 ]

Öfverstedt J, Lindblad J and Sladoje N. 2019. Fast and robust symmetric image registration based on distances combining intensity and spatial information. IEEE Transactions on Image Processing, 28(7): 3584-3597[DOI: 10.1109/TIP.2019.2899947]

Oliveira F P M and Tavares J M R S. 2014. Medical image registration: a review. Computer Methods in Biomechanics and Biomedical Engineering, 17(2): 73-93[DOI: 10.1080/10255842.2012.670855]

Onieva J O, Marti-Fuster B, de la Puente M P and Estépar R S J. 2018. Diffeomorphic lung registration using deep cnns and reinforced learning//Proceedings of the 3rd International Workshop on Reconstruction and Analysis of Moving Body Organs. Granada, Spain: Springer: 284-294[ DOI: 10.1007/978-3-030-00946-5_28 http://dx.doi.org/10.1007/978-3-030-00946-5_28 ]

Qin C, Bai W J, Schlemper J, Petersen S E, Piechnik S K, Neubauer S and Rueckert D. 2018. Joint learning of motion estimation and segmentation for cardiac MR image sequences//Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention. Granada, Spain: Springer: 472-480[ DOI: 10.1007/978-3-030-00934-2_53 http://dx.doi.org/10.1007/978-3-030-00934-2_53 ]

Qin C, Shi B B, Liao R, Mansi T, Rueckert D and Kamen A. 2019. Unsupervised deformable registration for multi-modal images via disentangled representations//Proceedings of the 29th International Conference on Information Processing in Medical Imaging. Hong Kong, China: Springer: 249-261[ DOI: 10.1007/978-3-030-20351-1_19 http://dx.doi.org/10.1007/978-3-030-20351-1_19 ]

Qiu Z F, Yao T and Mei T. 2017. Learning spatio-temporal representation with pseudo-3D residual networks//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE: 5534-5542[ DOI: 10.1109/iccv.2017.590 http://dx.doi.org/10.1109/iccv.2017.590 ]

RohéM M, Datar M, Heimann T, Sermesant M and Pennec X. 2017. SVF-Net: learning deformable image registration using shape matching//Proceedings of the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention. Quebec City, Canada: Springer: 266-274[ DOI: 10.1007/978-3-319-66182-7_31 http://dx.doi.org/10.1007/978-3-319-66182-7_31 ]

Ronneberger O, Fischer P and Brox T. 2015. U-Net: convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: Springer: 234-241[ DOI: 10.1007/978-3-319-24574-4_28 http://dx.doi.org/10.1007/978-3-319-24574-4_28 ]

Ruhaak J, Heldmann S, Kipshagen T and Fischer B. 2013. Highly accurate fast lung CT registration//Proceedings of SPIE 8669, Medical Imaging 2013: Image Processing. Lake Buena Vista, USA: SPIE: #86690Y[ DOI: 10.1117/12.2006035 http://dx.doi.org/10.1117/12.2006035 ]

Sabour S, Frosst N and Hinton G E. 2017. Dynamic routing between capsules[EB/OL]. [2020-07-04] . https://arxiv.org/pdf/1710.09829.pdf https://arxiv.org/pdf/1710.09829.pdf

Salehi S S M, Khan S, Erdogmus D and Gholipour A. 2018. Real-time deep registration with geodesic loss[EB/OL]. [2020-07-04] . https://arxiv.org/pdf/1803.05982v1.pdf https://arxiv.org/pdf/1803.05982v1.pdf

Schmid J and Chênes C. 2014. Segmentation of X-ray images by 3D-2Dregistration based on multibody physics//Proceedings of the 12th Asian Conference on Computer Vision. Singapore, Singapore: Springer: 674-687[ DOI: 10.1007/978-3-319-16808-1_45 http://dx.doi.org/10.1007/978-3-319-16808-1_45 ]

Sedghi A, Luo J, Mehrtash A, Pieper S, Tempany C M, Kapur T, Mousavi P and Wells Ⅲ W M. 2019. Semi-supervised image registration using deep learning//Proceedings of SPIE 10951, Medical Imaging 2019: Image-Guided Procedures, Robotic Interventions, and Modeling. San Diego, USA: SPIE: #109511G[ DOI: 10.1117/12.2513020 http://dx.doi.org/10.1117/12.2513020 ]

Sentker T, Madesta F and Werner R. 2018. GDL-FIRE 4D : deep learning-based fast 4D CT image registration//Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention. Granada, Spain: Springer: 765-773[ DOI: 10.1007/978-3-030-00928-1_86 http://dx.doi.org/10.1007/978-3-030-00928-1_86 ].

Shan S Y, Yan W, Guo X Q, Chang E I C, Fan Y B and Xu Y. 2017. Unsupervised end-to-end learning for deformable medical image registration[EB/OL]. [2020-07-04] . https://arxiv.org/pdf/1711.08608.pdf https://arxiv.org/pdf/1711.08608.pdf

Sheikhjafari A, Noga M, Punithakumar K and Ray N. 2019. Unsupervised deformable image registration with fully connected generative neural network//Proceedings of the Medical Imaging with Deep Learning. Amsterdam, the Netherlands: [s. n.]

Shekhar R and Zagrodsky V. 2002. Mutual information-based rigid and nonrigid registration of ultrasound volumes. IEEE Transactions on Medical Imaging, 21(1): 9-22[DOI: 10.1109/42.981230]

Shen Y Y and Feng H S. 2020. 2D-3D double X-ray image registration method based on neural network. Chinese Journal of Medical Physics, 37(3): 293-298

沈延延, 冯汉升. 2020. 基于神经网络的双X射线影像2D-3D配准算法. 中国医学物理学杂志, 37(3): 293-298[DOI: 10.3969/j.issn.1005-202X.2020.03.007]

Simonovsky M, Gutiérrez-Becker B, Mateus D, Navab N and Komodakis N. 2016. A deep metric for multimodal registration//Proceedings of the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention. Athens, Greece: Springer: 10-18[ DOI: 10.1007/978-3-319-46726-9_2 http://dx.doi.org/10.1007/978-3-319-46726-9_2 ]

Singh N K and Raza K. 2020. Medical image generation using generative adversarial networks[EB/OL]. [2020-07-04] . https://arxiv.org/pdf/2005.10687.pdf https://arxiv.org/pdf/2005.10687.pdf

Sloan J M, Goatman K A and Siebert J P. 2018. Learning rigid image registration-utilizing convolutional neural networks for medical image registration//Proceedings of the 11th Biomedical Engineering Systems and Technologies. Madeira, Portugal: Springer: 89-99[ DOI: 10.5220/0006543700890099 http://dx.doi.org/10.5220/0006543700890099 ]

Sokooti H, de Vos B, Berendsen F, Ghafoorian M, Yousefi S, Lelieveldt B P, Isgum I and Staring M. 2019. 3D convolutional neural networks image registration based on efficient supervised learningfrom artificial deformations[EB/OL]. [2020-07-04] . https://arxiv.org/pdf/1908.10235.pdf https://arxiv.org/pdf/1908.10235.pdf

Sokooti H, de Vos B, Berendsen F, Lelieveldt B P F, Išgum I and Staring M. 2017. Nonrigid image registration using multi-scale 3D convolutional neural networks//Proceedings of the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention. Quebec City, Canada: Springer: 232-239[ DOI: 10.1007/978-3-319-66182-7_27 http://dx.doi.org/10.1007/978-3-319-66182-7_27 ]

Sun L and Zhang S T. 2018. Deformable mri-ultrasound registration using 3D convolutional neural network//Proceedings of the 1st International Workshop on Computational Precision Medicine. Granada, Spain: Springer: 152-158[ DOI: 10.1007/978-3-030-01045-4_18 http://dx.doi.org/10.1007/978-3-030-01045-4_18 ]

Sun Y Y, Moelker A, Niessen W J and van Walsum T. 2018. Towards robust CT-ultrasound registration using deep learning methods//Proceedings of the 1st International Workshop on Interpretability of Machine Intelligence in Medical Image Computing. Granada, Spain: Springer: 43-51[ DOI: 10.1007/978-3-030-02628-8_5 http://dx.doi.org/10.1007/978-3-030-02628-8_5 ]

Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V and Rabinovich A. 2015. Going deeper with convolutions//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE: 1-9[ DOI: 10.1109/CVPR.2015.7298594 http://dx.doi.org/10.1109/CVPR.2015.7298594 ]

Tang L S, Hamarneh G and Celler A. 2006. Co-registration of bone CT and SPECT images using mutual information//IEEE International Symposium on Signal Processing and Information Technology. Vancouver, Canada: IEEE: 116-121[ DOI: 10.1109/ISSPIT.2006.270781 http://dx.doi.org/10.1109/ISSPIT.2006.270781 ]

Uzunova H, Wilms M, Handels H and Ehrhardt J. 2017. Training CNNs for image registration from few samples with model-based data augmentation//Proceedings of the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention. Quebec City, Canada: Springer: 223-231[ DOI: 10.1007/978-3-319-66182-7_26 http://dx.doi.org/10.1007/978-3-319-66182-7_26 ]

Vercauteren T, Pennec X, Perchant A and Ayache N. 2009. Diffeomorphic demons: efficient non-parametric image registration. NeuroImage, 45(1): 61-72[DOI: 10.1016/j.neuroimage.2008.10.040]

Wang C F and Jiang M. 2006. Review of image registration methods for medical images. CT Theory and Application, 15(2): 74-80

王彩芳, 姜明. 2006. 医学图像配准综述. CT理论与应用研究, 15(2): 74-80[DOI: 10.3969/j.issn.1004-4140.2006.02.015]

Wang H N, He C Y, Lei F Y and Zhang X Y. 2005. A survey of non-rigid medical image registration. Computer Engineering and Applications, 41(11): 180-184

王海南, 郝重阳, 雷方元, 张先勇. 2005. 非刚性医学图像配准研究综述. 计算机工程与应用, 41(11): 180-184[DOI: 10.3321/j.issn:1002-8331.2005.11.059]

Wang L F, Zhang C C, Qin P L, Lin S Z, Gao Y and Dou J L. 2020. Image registration method with residual dense relativistic average CGAN. Journal of Image and Graphics, 25(4): 745-758

王丽芳, 张程程, 秦品乐, 蔺素珍, 高媛, 窦杰亮. 2020a. 残差密集相对平均CGAN的脑部图像配准. 中国图象图形学报, 25(4): 745-758)[ DOI: 10.11834/jig.190116 http://dx.doi.org/10.11834/jig.190116 ]

Wang S W, Zhao Q Y, Wang X Y and Rao X P. 2020. Region division method of brain slice image based on deep learning. Journal of Computer Applications, 40(4): 1202-1208

王松伟, 赵秋阳, 王宇航, 饶小平. 2020. 基于深度学习的脑片图像区域划分方法. 计算机应用, 40(4): 1202-1208[DOI: 10.11772/j.issn.1001-9081.2019091521]

Wilms M, Handels H and Ehrhardt J. 2017. Multi-resolution multi-object statistical shape models based on the locality assumption. Medical Image Analysis, 38: 17-29[DOI: 10.1016/j.media.2017.02.003]

Wu G R, Kim M, Wang Q, Gao Y Z, Liao S and Shen D G. 2013. Unsupervised deep feature learning for deformable registration of MR brain images//Proceedings of the 16th International Conference on Medical Image Computing and Computer-Assisted Intervention. Nagoya, Japan: Springer: 649-656[ DOI: 10.1007/978-3-642-40763-5_80 http://dx.doi.org/10.1007/978-3-642-40763-5_80 ]

Xu Z, Luo J, Yan J P, Pulya R, Li X, Wells Ⅲ W and Jagadeesan J. 2020. Adversarial uni- and multi-modal stream networks for multimodal image registration//Proceedings of the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention. Lima, Peru: Springer: 222-232

Yan J P, Chen S, Zhang Y B and Li X. 2020. Neural architecture search for compressed sensing magnetic resonance image reconstruction. Computerized Medical Imaging and Graphics, 85: #101784[DOI: 10.1016/j.compmedimag.2020.101784]

Yan P K, Xu S, Rastinehad A R and Wood B J. 2018. Adversarial image registration with application for MR and TRUS image fusion//Proceedings of the 9th International Workshop on Machine Learning in Medical Imaging. Granada, Spain: Springer: 197-204[ DOI: 10.1007/978-3-030-00919-9_23 http://dx.doi.org/10.1007/978-3-030-00919-9_23 ]

Yang X, Kwitt R and Niethammer M. 2016. Fast predictive image registration//Proceedings of the 1st International Workshop onDeep Learning in Medical Image Analysis. Athens, Greece: Springer: 48-57[ DOI: 10.1007/978-3-319-46976-8_6 http://dx.doi.org/10.1007/978-3-319-46976-8_6 ]

Yang X, Kwitt R, Styner M and Niethammer M. 2017. Quicksilver: fast predictive image registration-a deep learning approach. NeuroImage, 158: 378-396[DOI: 10.1016/j.neuroimage.2017.07.008]

Yi Z L, Zhang H, Tan P and Gong M L. 2017. DualGAN: unsupervised dual learning for image-to-image translation//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE: 2868-2876[ DOI: 10.1109/ICCV.2017.310 http://dx.doi.org/10.1109/ICCV.2017.310 ]

Yoo I, Hildebrand D G C, Tobin W F, Lee W C A and Jeong W K. 2017. ssEMnet: serial-section electron microscopy image registration using a spatial transformer network with learned features//Proceedings of the 3rd International Workshop on Multimodal Learning for Clinical Decision Support. Québec City, Canada: Springer: 249-257[ DOI: 10.1007/978-3-319-67558-9_29 http://dx.doi.org/10.1007/978-3-319-67558-9_29 ]

Yu H J, Zhou X R, Jiang H Y, Kang H J, Wang Z G, Hara T and Fujita H. 2019. Learning 3D non-rigid deformation based on an unsupervised deep learning for PET/CT image registration//Proceedings of SPIE 10953, Medical Imaging 2019: Biomedical Applications in Molecular, Structural, and Functional Imaging. San Diego, USA: SPIE: #109531X[ DOI: 10.1117/12.2512698 http://dx.doi.org/10.1117/12.2512698 ]

Zhang J. 2018. Inverse-consistent deep networks for unsupervised deformable image registration[EB/OL]. [2020-07-04] . https://arxiv.org/pdf/1809.03443.pdf https://arxiv.org/pdf/1809.03443.pdf

Zhao S Y, Dong Y, Chang E and Xu Y. 2019. Recursive cascaded networks for unsupervised medical image registration//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul, Korea(South): IEEE: 10599-10609[ DOI: 10.1109/ICCV.2019.01070 http://dx.doi.org/10.1109/ICCV.2019.01070 ]

Zhao S Y, Lau T, Luo J, Chang E I C and Xu Y. 2020. Unsupervised 3 d end-to-end medical image registration with volume tweening network. IEEE Journal of Biomedical and Health Informatics, 24(5): 1394-1404[DOI: 10.1109/JBHI.2019.2951024]

Zheng J N, Miao S and Liao R. 2017. Learning CNNs with pairwise domain adaption for real-time 6 DoF ultrasound transducer detection and tracking from x-ray images//Proceedings of the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention. Quebec City, Canada: Springer: 646-654[ DOI: 10.1007/978-3-319-66185-8_73 http://dx.doi.org/10.1007/978-3-319-66185-8_73 ]

Zheng J N, Miao S, Wang Z J and Liao R. 2018. Pairwise domain adaptation module for CNN-based 2-D/3-D registration. Journal of Medical Imaging, 5(2): #021204[DOI: 10.1117/1.JMI.5.2.021204]

Zhu J Y, Park T, Isola P and Efros A A. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE: 2242-2251[ DOI: 10.1109/ICCV.2017.244 http://dx.doi.org/10.1109/ICCV.2017.244 ]

Zou M Y, Yang H, Pang G H and Zhong Y. 2019. Research progress and challenges of deep learning in medical image registration. Journal of Biomedical Engineering, 36(4): 677-683

邹茂扬, 杨昊, 潘光晖, 钟勇. 2019. 深度学习在医学图像配准上的研究进展与挑战. 生物医学工程学杂志, 36(4): 677-683[DOI: 10.7507/1001-5515.201810004]