浏览全部资源
扫码关注微信
收稿：2020-04-14，
修回：2020-7-10，
录用：2020-7-17，
纸质出版：2020-10-16
移动端阅览
目的
2
现有的医学图像配准算法在处理较大非刚性形变的医学图像时，存在配准精度低和泛化能力不足的问题。针对该问题，提出结合密集残差块和生成对抗网络(generative adversarial network，GAN)的图像配准方法，用于多模态医学图像的非刚性配准。
方法
2
将密集残差块引入到生成器，提取待配准图像对的更多高层特征，从而提高配准精度；在GAN中采用最小二乘损失代替逻辑回归构造的交叉熵损失，由于最小二乘损失的收敛条件更严格，同时能缓解梯度消失和过拟合，从而提高配准模型的稳定性；在判别器网络中引入相对平均GAN(relative average GAN，RaGAN)，即在标准判别器的基础上增加一项梯度惩罚因子，降低判别器的判别误差，从而相对减少配准模型的配准误差，有助于稳定配准精度。
结果
2
在DRIVE(digital retinal images for vessel extraction)数据集上进行训练和验证，在Sunybrook Cardiac数据集和Brain MRI数据集上进行泛化性能测试，并与几种主流医学图像非刚性配准算法进行对比实验，表明，本文配准模型在精度和泛化能力上均有一定程度的提升，相比其他方法，视网膜图像、心脏图像和脑部图像的配准Dice值分别提升了3.3%、3.0%、1.5%。
结论
2
提出的配准方法能够获取更多高层特征信息，从而提升配准精度；同时基于最小二乘法构建损失函数并对判别器进行加强，能够使得配准模型在训练阶段快速收敛，提高了配准模型的稳定性和泛化能力，适合存在较大非刚性形变的医学图像的配准。
Objective
2
Medical image registration is an important research task in the field of medical image analysis
which is the basis of medical image fusion and medical image reconstruction. Conventional registration methods that build loss function based on normalized mutual information by using iterative gradient descent to achieve registration tend to be time consuming. The existing deep learning-based medical image registration methods have limitation in registering medical images with large non-rigid deformation
which cannot achieve high registration accuracy and have poor generalization ability. Thus
this paper proposes a method to register multi-modal medical images by combining residual-in-residual dense block (RRDB) with generative adversarial network (GAN).
Method
2
First
RRDB are introduced to the standard generator network to extract high-level feature information from unpaired image pairs; thus
registration accuracy is improved. Then
a least-squares loss is used to substitute cross-entropy loss constructed by the logistic regression objective. The convergence condition of least-squares loss is strict and promotes the model to reach convergence at the optimal parameters
which can alleviate gradient disappearance and overfitting; thus
the robustness of model training is enhanced. In addition
relative average GAN (RaGAN) is embedded into the standard discriminator network
namely
adding a gradient penalty in the discriminator network
which reduce the error of discriminator to estimate the relative true and false probability between the fixed image and moving image. The enhanced discriminator can help the generator to learn clearer and texture information; therefore
the registration error can be decreased
and the registration accuracy can be stabilized.
Result
2
This registration model is trained and validated on DRIVE(digital retinal images for vessel extraction) dataset. Generalization performance tests are performed on Sunybrook Cardiac Dataset and Brain MRI Dataset. Compared with state-of-the-art registration methods
extensive experiments demonstrate that the proposed model achieves good registration results; both registration accuracy and generalization ability have been improved. Compared with the basic literature
the registration Dice values of retinal images
cardiac images
and brain images are improved by 3.3%
3.0%
and 1.5%
respectively. According to the stability verification experiment of the registration model
as the number of iterations augments
the Dice value of this paper gradually increases
and the change is more stable than that of baseline literature. The number of iterations in this paper is 80 000
whereas that of baseline literature is 10 000. This verification experiment shows that this registration model has been effectively improved in the training phase; not only the convergence speed is accelerated but also stability is higher compared with that of baseline literature.
Conclusion
2
The proposed registration method can obtain high-level feature information; thus
the registration accuracy is improved. Simultaneously
the loss function is built on the basis of the least-squares method
and the discriminator is also strengthened
which can enable the registration model to quickly converge during the training phase and then improve model stability and generalization ability. This method is suitable for medical image registration with large non-rigid deformation.
Balakrishnan G, Zhao A, Sabuncu M R, Guttag J and Dalca A V. 2019. VoxelMorph:a learning framework for deformable medical image registration. IEEE Transactions on Medical Imaging, 38(8):1788-1800[DOI:10.1109/TMI.2019.2897538]
Chen M, Chen W F, Feng Q J and Yang F. 2003. Medical imageregistration based on mutual information and fuzzy gradient similarity. Acta Electronica Sinica, 31(12):1835-1838
陈明, 陈武凡, 冯前进, 杨丰. 2003.基于互信息量和模糊梯度相似性的医学图像配准.电子学报, 31(12):1835-1838[DOI:10.3321/j.issn:0372-2112.2003.12.018]
De Vos B D, Berendsen F F, Viergever M A, Staring M and Išgum I. 2017. End-to-end unsupervised deformable image registration with a convolutional neural network//Proceedings of the 3rd International Workshop on Deep Learning in Medical Image Analysis. Québec City: Springer: 204-212[ DOI: 10.1007/978-3-319-67558-9_24 http://dx.doi.org/10.1007/978-3-319-67558-9_24 ]
Ferrante E, Dokania P K, Silva R M and Paragios N. 2019. Weakly supervised learning of metric aggregations for deformable image registration. IEEE Journal of Biomedical and Health Informatics, 23(4):1374-1384[DOI:10.1109/JBHI.2018.2869700]
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A and Bengio Y. 2014. Generative adversarial networks//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal: MIT Press: 2672-2680
Hu Y P, Modat M, Gibson E, Ghavami, N, Bonmati E, Moore C M, Emberton M, Noble J A, Barratt D C and Vercauteren T. 2018a. Label-driven weakly-supervised learning for multimodal deformable image registration//The 15th International Symposium on Biomedical Imaging. Washington: IEEE: 1070-1074[ DOI: 10.1109/ISBI.2018.8363756 http://dx.doi.org/10.1109/ISBI.2018.8363756 ]
Hu Y P, Modat M, Gibson E, Li W Q, Ghavami N, Bonmati E, Wang G T, Bandula S, Moore C M, Emberton M, Ourselin S, Noble J A, Barratt D C and Vercauteren T. 2018b. Weakly-supervised convolutional neural networks for multimodal image registration. Medical Image Analysis, 49:1-13[DOI:10.1016/j.media.2018.07.002]
Jolicoeur-Martineau A. 2018. The relativistic discriminator: a key element missing from standard GAN//Proceedings of the IEEE Conference on Computer Vision & Pattern Recognition, 86(43): 540-554[ DOI: 10.1109/CVPR.2018.294 http://dx.doi.org/10.1109/CVPR.2018.294 ]
Klein S, Staring M, Murphy K, Viergever M A and Pluim J P W. 2010. elastix:a toolbox for intensity-based medical image registration. IEEE Transactions on Medical Imaging, 29(1):196-205[DOI:10.1109/TMI.2009.2035616]
Krizhevsky A, Sutskever I and Hinton G. 2012. ImageNet classification with deep convolutional neural networks//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe: ACM: 1097-1105
Liang W, Bao X D and Luo L M. 2004.2D-3D medical image registration based on mutual information. Journal of Biomedical Engineering Research, 23(1):14-16
梁玮, 鲍旭东, 罗立民. 2004.基于互信息的2D-3D医学图像配准.生物医学工程研究, 23(1):14-16[DOI:10.3969/j.issn.1672-6278.2004.01.005]
Lu Z T and Chen W F. 2007. Medical image registration using co-occurrence mutual information. Chinese Journal of Computers, 30(6):1022-1027
卢振泰, 陈武凡. 2007.基于共生互信息量的医学图像配准.计算机学报, 30(6):1022-1027[DOI:10.3321/j.issn:0254-4164.2007.06.018]
Mahapatra D, Sedai S and Garnavi R. 2018. Elastic registration of medical images with GANs[EB/OL].[2020-04-14] . https://amds123.github.io/2018/05/23/Elastic-Registration-of-Medical-Images-With-GANs/ https://amds123.github.io/2018/05/23/Elastic-Registration-of-Medical-Images-With-GANs/
Mao X D, Li Q, Xie H R, Lau R Y K, Wang Z and Smolley S P. 2017. Least squares generative adversarial networks//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice: IEEE: 2813-2821[ DOI: 10.1109/ICCV.2017.304 http://dx.doi.org/10.1109/ICCV.2017.304 ]
Miao S, Wang Z J and Liao R. 2016. A CNN regression approach for real-time 2D/3D registration. IEEE Transactions on Medical Imaging, 35(5):1352-1363[DOI:10.1109/TMI.2016.2521800]
Sokooti H, De Vos B, Berendsen F, Lelieveldt B P F, Išgum I and Staring M. 2017. Nonrigid image registration using multi-scale 3D convolutional neural networks//Proceedings of the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention. Quebec City: Springer: 232-239[ DOI: 10.1007/978-3-319-66182-7_27 http://dx.doi.org/10.1007/978-3-319-66182-7_27 ]
Wang H N, Hao C Y, Lei F Y and Zhang X Y. 2005. A survey of non-rigid medical image registration. Computer Engineering and applications, 41(11):180-184
王海南, 郝重阳, 雷方元, 张先勇. 2005.非刚性医学图像配准研究综述.计算机工程与应用, 41(11):180-184[DOI:10.3321/j.issn:1002-8331.2005.11.059]
Wang X G and Xu W B. 2008. Research of medical image registration based on max mutual information and quantum-behaved particle swarm algorithm. Application Research of Computers, 25(12):3685-3687
王小根, 须文波. 2008.基于最大互信息和量子粒子群优化算法的医学图像配准研究.计算机应用研究, 25(12):3685-3687[DOI:10.3969/j.issn.1001-3695.2008.12.047]
Wang X T, Yu K, Wu S X, Gu J J, Liu Y H, Dong C, Qiao Y and Loy C C. 2019. ESRGAN: enhanced super-resolution generative adversarial networks//Proceedings of 2019 European Conference on Computer Vision. Munich: Springer: 63-79[ DOI: 10.1007/978-3-030-11021-5_5 http://dx.doi.org/10.1007/978-3-030-11021-5_5 ]
Wu G R, Kim M, Wang Q, Gao Y Z, Liao S and Shen D G. 2013. Unsupervised deep feature learning for deformable registration of MR brain images//Proceedings of the 16th International Conference on Medical Image Computing and Computer-Assisted Intervention. Nagoya: Springer: 649-656[ DOI: 10.1007/978-3-642-40763-5_80 http://dx.doi.org/10.1007/978-3-642-40763-5_80 ]
Zhu J Y, Park T, Isola P and Efros A A. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice: IEEE: 2242-2251[ DOI: 10.1109/ICCV.2017.244 http://dx.doi.org/10.1109/ICCV.2017.244 ]
相关作者
相关机构
京公网安备11010802024621
