Non-rigid medical image registration based on residual-in-residual dense block and GAN

Guimei Zhang; Qiang Hu; Lei Gong

doi:10.11834/jig.200130

Magnetic Resonance Image | Views : 0 下载量: 31 CSCD: 0

PDF
Export
Share
Collection
Album

Non-rigid medical image registration based on residual-in-residual dense block and GAN
Vol. 25, Issue 10, Pages: 2182-2194(2020)
Received：14 April 2020，

Revised：2020-7-10，

Accepted：17 July 2020，

Published：16 October 2020
DOI： 10.11834/jig.200130
稿件说明：

移动端阅览

DOI：

Guimei Zhang, Qiang Hu, Lei Gong. Non-rigid medical image registration based on residual-in-residual dense block and GAN[J]. Journal of Image and Graphics, 2020, 25(10): 2182-2194. DOI： 10.11834/jig.200130.

摘要

目的

现有的医学图像配准算法在处理较大非刚性形变的医学图像时，存在配准精度低和泛化能力不足的问题。针对该问题，提出结合密集残差块和生成对抗网络(generative adversarial network，GAN)的图像配准方法，用于多模态医学图像的非刚性配准。

方法

将密集残差块引入到生成器，提取待配准图像对的更多高层特征，从而提高配准精度；在GAN中采用最小二乘损失代替逻辑回归构造的交叉熵损失，由于最小二乘损失的收敛条件更严格，同时能缓解梯度消失和过拟合，从而提高配准模型的稳定性；在判别器网络中引入相对平均GAN(relative average GAN，RaGAN)，即在标准判别器的基础上增加一项梯度惩罚因子，降低判别器的判别误差，从而相对减少配准模型的配准误差，有助于稳定配准精度。

结果

在DRIVE(digital retinal images for vessel extraction)数据集上进行训练和验证，在Sunybrook Cardiac数据集和Brain MRI数据集上进行泛化性能测试，并与几种主流医学图像非刚性配准算法进行对比实验，表明，本文配准模型在精度和泛化能力上均有一定程度的提升，相比其他方法，视网膜图像、心脏图像和脑部图像的配准Dice值分别提升了3.3%、3.0%、1.5%。

结论

提出的配准方法能够获取更多高层特征信息，从而提升配准精度；同时基于最小二乘法构建损失函数并对判别器进行加强，能够使得配准模型在训练阶段快速收敛，提高了配准模型的稳定性和泛化能力，适合存在较大非刚性形变的医学图像的配准。

Abstract

Objective

Medical image registration is an important research task in the field of medical image analysis

which is the basis of medical image fusion and medical image reconstruction. Conventional registration methods that build loss function based on normalized mutual information by using iterative gradient descent to achieve registration tend to be time consuming. The existing deep learning-based medical image registration methods have limitation in registering medical images with large non-rigid deformation

which cannot achieve high registration accuracy and have poor generalization ability. Thus

this paper proposes a method to register multi-modal medical images by combining residual-in-residual dense block (RRDB) with generative adversarial network (GAN).

Method

First

RRDB are introduced to the standard generator network to extract high-level feature information from unpaired image pairs; thus

registration accuracy is improved. Then

a least-squares loss is used to substitute cross-entropy loss constructed by the logistic regression objective. The convergence condition of least-squares loss is strict and promotes the model to reach convergence at the optimal parameters

which can alleviate gradient disappearance and overfitting; thus

the robustness of model training is enhanced. In addition

relative average GAN (RaGAN) is embedded into the standard discriminator network

namely

adding a gradient penalty in the discriminator network

which reduce the error of discriminator to estimate the relative true and false probability between the fixed image and moving image. The enhanced discriminator can help the generator to learn clearer and texture information; therefore

the registration error can be decreased

and the registration accuracy can be stabilized.

Result

This registration model is trained and validated on DRIVE(digital retinal images for vessel extraction) dataset. Generalization performance tests are performed on Sunybrook Cardiac Dataset and Brain MRI Dataset. Compared with state-of-the-art registration methods

extensive experiments demonstrate that the proposed model achieves good registration results; both registration accuracy and generalization ability have been improved. Compared with the basic literature

the registration Dice values of retinal images

cardiac images

and brain images are improved by 3.3%

3.0%

and 1.5%

respectively. According to the stability verification experiment of the registration model

as the number of iterations augments

the Dice value of this paper gradually increases

and the change is more stable than that of baseline literature. The number of iterations in this paper is 80 000

whereas that of baseline literature is 10 000. This verification experiment shows that this registration model has been effectively improved in the training phase; not only the convergence speed is accelerated but also stability is higher compared with that of baseline literature.

Conclusion

The proposed registration method can obtain high-level feature information; thus

the registration accuracy is improved. Simultaneously

the loss function is built on the basis of the least-squares method

and the discriminator is also strengthened

which can enable the registration model to quickly converge during the training phase and then improve model stability and generalization ability. This method is suitable for medical image registration with large non-rigid deformation.

关键词

Keywords

references

Balakrishnan G, Zhao A, Sabuncu M R, Guttag J and Dalca A V. 2019. VoxelMorph:a learning framework for deformable medical image registration. IEEE Transactions on Medical Imaging, 38(8):1788-1800[DOI:10.1109/TMI.2019.2897538]

Chen M, Chen W F, Feng Q J and Yang F. 2003. Medical imageregistration based on mutual information and fuzzy gradient similarity. Acta Electronica Sinica, 31(12):1835-1838

陈明, 陈武凡, 冯前进, 杨丰. 2003.基于互信息量和模糊梯度相似性的医学图像配准.电子学报, 31(12):1835-1838[DOI:10.3321/j.issn:0372-2112.2003.12.018]

De Vos B D, Berendsen F F, Viergever M A, Staring M and Išgum I. 2017. End-to-end unsupervised deformable image registration with a convolutional neural network//Proceedings of the 3rd International Workshop on Deep Learning in Medical Image Analysis. Québec City: Springer: 204-212[ DOI: 10.1007/978-3-319-67558-9_24 http://dx.doi.org/10.1007/978-3-319-67558-9_24 ]

Ferrante E, Dokania P K, Silva R M and Paragios N. 2019. Weakly supervised learning of metric aggregations for deformable image registration. IEEE Journal of Biomedical and Health Informatics, 23(4):1374-1384[DOI:10.1109/JBHI.2018.2869700]

Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A and Bengio Y. 2014. Generative adversarial networks//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal: MIT Press: 2672-2680

Hu Y P, Modat M, Gibson E, Ghavami, N, Bonmati E, Moore C M, Emberton M, Noble J A, Barratt D C and Vercauteren T. 2018a. Label-driven weakly-supervised learning for multimodal deformable image registration//The 15th International Symposium on Biomedical Imaging. Washington: IEEE: 1070-1074[ DOI: 10.1109/ISBI.2018.8363756 http://dx.doi.org/10.1109/ISBI.2018.8363756 ]

Hu Y P, Modat M, Gibson E, Li W Q, Ghavami N, Bonmati E, Wang G T, Bandula S, Moore C M, Emberton M, Ourselin S, Noble J A, Barratt D C and Vercauteren T. 2018b. Weakly-supervised convolutional neural networks for multimodal image registration. Medical Image Analysis, 49:1-13[DOI:10.1016/j.media.2018.07.002]

Jolicoeur-Martineau A. 2018. The relativistic discriminator: a key element missing from standard GAN//Proceedings of the IEEE Conference on Computer Vision & Pattern Recognition, 86(43): 540-554[ DOI: 10.1109/CVPR.2018.294 http://dx.doi.org/10.1109/CVPR.2018.294 ]

Klein S, Staring M, Murphy K, Viergever M A and Pluim J P W. 2010. elastix:a toolbox for intensity-based medical image registration. IEEE Transactions on Medical Imaging, 29(1):196-205[DOI:10.1109/TMI.2009.2035616]

Krizhevsky A, Sutskever I and Hinton G. 2012. ImageNet classification with deep convolutional neural networks//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe: ACM: 1097-1105

Liang W, Bao X D and Luo L M. 2004.2D-3D medical image registration based on mutual information. Journal of Biomedical Engineering Research, 23(1):14-16

梁玮, 鲍旭东, 罗立民. 2004.基于互信息的2D-3D医学图像配准.生物医学工程研究, 23(1):14-16[DOI:10.3969/j.issn.1672-6278.2004.01.005]

Lu Z T and Chen W F. 2007. Medical image registration using co-occurrence mutual information. Chinese Journal of Computers, 30(6):1022-1027

卢振泰, 陈武凡. 2007.基于共生互信息量的医学图像配准.计算机学报, 30(6):1022-1027[DOI:10.3321/j.issn:0254-4164.2007.06.018]

Mahapatra D, Sedai S and Garnavi R. 2018. Elastic registration of medical images with GANs[EB/OL].[2020-04-14] . https://amds123.github.io/2018/05/23/Elastic-Registration-of-Medical-Images-With-GANs/ https://amds123.github.io/2018/05/23/Elastic-Registration-of-Medical-Images-With-GANs/

Mao X D, Li Q, Xie H R, Lau R Y K, Wang Z and Smolley S P. 2017. Least squares generative adversarial networks//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice: IEEE: 2813-2821[ DOI: 10.1109/ICCV.2017.304 http://dx.doi.org/10.1109/ICCV.2017.304 ]

Miao S, Wang Z J and Liao R. 2016. A CNN regression approach for real-time 2D/3D registration. IEEE Transactions on Medical Imaging, 35(5):1352-1363[DOI:10.1109/TMI.2016.2521800]

Sokooti H, De Vos B, Berendsen F, Lelieveldt B P F, Išgum I and Staring M. 2017. Nonrigid image registration using multi-scale 3D convolutional neural networks//Proceedings of the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention. Quebec City: Springer: 232-239[ DOI: 10.1007/978-3-319-66182-7_27 http://dx.doi.org/10.1007/978-3-319-66182-7_27 ]

Wang H N, Hao C Y, Lei F Y and Zhang X Y. 2005. A survey of non-rigid medical image registration. Computer Engineering and applications, 41(11):180-184

王海南, 郝重阳, 雷方元, 张先勇. 2005.非刚性医学图像配准研究综述.计算机工程与应用, 41(11):180-184[DOI:10.3321/j.issn:1002-8331.2005.11.059]

Wang X G and Xu W B. 2008. Research of medical image registration based on max mutual information and quantum-behaved particle swarm algorithm. Application Research of Computers, 25(12):3685-3687

王小根, 须文波. 2008.基于最大互信息和量子粒子群优化算法的医学图像配准研究.计算机应用研究, 25(12):3685-3687[DOI:10.3969/j.issn.1001-3695.2008.12.047]

Wang X T, Yu K, Wu S X, Gu J J, Liu Y H, Dong C, Qiao Y and Loy C C. 2019. ESRGAN: enhanced super-resolution generative adversarial networks//Proceedings of 2019 European Conference on Computer Vision. Munich: Springer: 63-79[ DOI: 10.1007/978-3-030-11021-5_5 http://dx.doi.org/10.1007/978-3-030-11021-5_5 ]

Wu G R, Kim M, Wang Q, Gao Y Z, Liao S and Shen D G. 2013. Unsupervised deep feature learning for deformable registration of MR brain images//Proceedings of the 16th International Conference on Medical Image Computing and Computer-Assisted Intervention. Nagoya: Springer: 649-656[ DOI: 10.1007/978-3-642-40763-5_80 http://dx.doi.org/10.1007/978-3-642-40763-5_80 ]

Zhu J Y, Park T, Isola P and Efros A A. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice: IEEE: 2242-2251[ DOI: 10.1109/ICCV.2017.244 http://dx.doi.org/10.1109/ICCV.2017.244 ]

Alert me when the article has been cited

提交

Non-rigid Mammogram Registration Based on Improved Demons Algorithm

Single vertebra 2D/3D registration with fusion of local and global features

Large-capacity data hiding and authentication method in encryted domain for medical image

Meta-optimized multi-adversarial domain adaptation for thyroid ultrasound image

Deep learning image inpainting combining semantic segmentation reconstruction and edge reconstruction