浏览全部资源
扫码关注微信
Published： 16 April 2023 ，
移动端阅览
张孟希， 魏然， 刘博， 徐寿平， 白相志， 周付根. 2023. CG-DRR:数字重建放射影像循环一致性生成对抗算法. 中国图象图形学报， 28(04):1212-1222
Zhang Mengxi， Wei Ran， Liu Bo， Xu Shouping， Bai Xiangzhi， Zhou Fugen. 2023. CG-DRR：digital reconstructed radiograph generation algorithm based on Cycle-GAN. Journal of Image and Graphics， 28(04):1212-1222
目的
2
当前数字重建放射影像（digitally reconstructed radiograph，DRR）生成算法难以同时保证影像生成效率和质量。为此，提出一种基于循环一致性生成对抗网络（generative adversarial network， GAN）的新型DRR生成算法（cycle-GAN-DRR， CG-DRR），在保证生成效率的同时，兼顾生成影像与真实X射线影像的灰度和结构一致性。
方法
2
CG-DRR算法包含数据预处理、网络训练和网络应用3个阶段。数据预处理阶段准备后续网络训练需要的图像数据；网络训练阶段使用预处理后的数据训练循环一致性生成对抗网络；在应用阶段，训练后网络输入光线跟踪法生成的DRR图像实现DRR图像灰度校正。
结果
2
使用4个常用图像相似性指标和2个梯度相似性指标分别评估原始DRR图像与灰度校正后的DRR图像的灰度和几何结构一致性。与传统光线跟踪法相比，盆腔和胸腔数据平均峰值信噪比分别提高18.22 dB和8.82 dB，平均绝对误差分别减少0.18和0.07，归一化均方根误差分别减少0.23和0.10，结构相似度指数分别提高23.5%和13.5%，与当前最新RealDRR算法结果指标相差无几。与RealDRR算法结果相比，CG-DRR算法在DRR图像灰度校正前后人体组织结构保持更好的一致性，盆腔和胸腔数据图像特征相似性指数分别提高0.02和0.03，梯度相似性偏差分别减少0.18和0.03。CG-DRR算法生成一幅DRR图像平均耗时0.31 s。
结论
2
本文创新性地将循环一致性生成对抗机制应用于DRR生成，所提算法可以很好地应对实际临床中DRR图像与X光影像存在结构偏差的问题，可在保证生成效率的前提下兼顾灰度相似性及结构一致性，相比于现有算法更具优势。
Objective
2
Real-time tumor localization is essential for tumor tracking radiotherapy. Conventional tumor localization methods commonly estimate the tumor motion by measuring the similarity between the X-ray images and digitally reconstructed radiography （DRR） computed from computed tomography （CT）. However， due to the scatter， beam hardening and quantum noise， there exists intensity inconsistency between the X-ray projections and DRR which may compromise the accuracy of tumor localization. Thus， it is crucial to calculate DRR similar with the X-ray images for precise tumor localization. Current DRR-relevant methods can be segmented into two categories： 1） statistical-based Monte Carlo （MC） simulation and 2） analysis-based ray tracing （RT）. The MC methods simulate the interaction process between photons and human tissue and can generate DRRs of high similarity with the X-ray projections. But it suffers from low computational efficiency which hinders its clinical application. The RT methods calculate DRR by simulating the absorption and attenuation process of X-ray penetrating human tissue. Compared to MC methods， the RT methods have higher computational efficiency， but there is a big intensity gap between their results and the real X-ray images. To address the problems mentioned above， we develop an improved cycle consistency generative adversarial network （Cycle-GAN） based DRR generation algorithm （CG-DRR）， which can efficiently generate DRR with high similarity to the X-ray images.
Method
2
CG-DRR consists of two mapping functions
G
y
，
G
x
and associated adversarial discriminators
D
y
，
D
x
. The
G
y
is trained to generate DRRs indistinguishable from X-ray images based on DRR calculated by RT （DRR
RT
）， while
D
y
aims to distinguish between generated DRRs and real X-ray images， and vice versa for
G
x
， D
x
. The training loss for CG-DRR is composed of three elements： 1） adversarial loss for matching the distribution of generated DRR to the X-ray distribution in the target domain； 2） a cycle consistency L1-norm loss to prevent the learned mappings
G
y
，
G
x
from contradicting each other； and 3） gradient penalty configuration to stabilize network training. For validation， planning CT and CBCT （cone beam computed tomography） projections （X-ray images） for radiotherapy of 3 pelvic and 3 chest patients are collected. For the pelvic/chest data， the CG-DRR is trained on 1 077/588 CBCT projections randomly selected from two patients and tested on 100/50 unseen CBCT projections from the same patients， and 100/50 CBCT projections randomly selected from the third patient. The third patient data was only used for testing to evaluate the inter-patient generalization performance of the CG-DRR. The overall framework is composed of three stages. In the data preprocessing stage， FDK （Feldkamp-Davis-Kress） algorithm is first used to reconstruct the 3D CBCT image based on the CBCT projections. Rigid registration is then performed to align the CT with CBCT. The DRR
RT
can be generated according to the geometric parameters of CBCT projections. The CBCT projections and DRR
RT
are rescaled to 256 × 256 pixels and their intensity is normalized into ［0， 1］. In the training stage， the parameters of CG-DRR are optimized using mini-batch （size = 4） stochastic gradient descent （SGD） and Adam solver （
β
1
= 0.5） through alternating gradient descent steps on the discriminators and generators. The learning rate is fixed at 0.001 in the first 100 epochs， and then decreased to 0 in the next 100 epochs linearly. In the application stage， the input of
G
y
is DRR
RT
， which is rescaled to 256 × 256 pixels and normalized into ［0， 1］. The output of
G
y
is up-sampled and de-normalized to obtain a DRR with the same size and intensity range as the CBCT projections.
Result
2
Evaluation is performed by comparing the generated DRR to ground-truth CBCT projections in terms of the peak signal-to-noise ratio （PSNR）， the mean absolute error （MAE）， the normalized root-mean-square error （NRMSE）， and the structural similarity index （SSIM）. To further evaluate the structural consistency， two additional indicators， feature similarity index measure （FSIM） and gradient magnitude similarity deviation （GMSD）， are also evaluated. For RT， RealDRR and CG-DRR， 1）the average PSNR are 11.6 dB， 32.9 dB， 29.6 dB for pelvic data， 16.4 dB， 31.3 dB， 25.2 dB for chest data； 2） the average MAE are 0.21， 0.02， 0.03 for pelvic data and 0.12， 0.03， 0.05 for chest data； and 3） the average NRMSE are 0.27， 0.03， 0.04 for pelvic data and 0.16， 0.04， 0.06 for chest data； 4） the average SSIM are 0.745， 0.985， 0.980 for pelvic data and 0.840， 0.985， 0.975 for chest data. But， the results of RealDRR contain noticeable structural distortions， with faked or missed tissue structures compared with DRR
RT
. Especially for chest data， the position of the thoracic diaphragm is significantly shifted. Compared with the results of RealDRR， CG-DRR can keep a good consistency of tissue structure. For the average FSIM， the pelvic results are increased from 0.855 to 0.870， an improvement of 2%； the chest results are increased from 0.91 to 0.93， an improvement of 2.2%. For the average GMSD， the pelvic results decreased from 0.175 to 0.17， a reduction of 2.9%； the chest results decreased from 0.135 to 0.115， a reduction of 14.8%. For computational efficiency， the CG-DRR can render a highly realistic DRR in 0.31 s.
Conclusion
2
The cycle-consistent generative adversarial mechanism is applied to DRR generation. The proposed algorithm can efficiently generate DRR that has good intensity similarity and structural consistency with X-ray projections.
数字重建放射影像（DRR）灰度校正深度学习循环一致性对抗网络梯度惩罚
digital reconstructed radiograph （DRR）intensity correctiondeep learningcycle-consistent generative adversarial networksgradient penalty
Alsaati L and Hashim S Z M. 2020. Cycle generative adversarial network for unpaired sketch-to-character translation//Saeed F， Mohammed F and Gazem N， eds. Emerging Trends in Intelligent Computing and Informatics. Cham， Germany： Springer： 321-329 ［DOI： 10.1007/978-3-030-33582-3_31http://dx.doi.org/10.1007/978-3-030-33582-3_31］
Arjovsky M， Chintala S and Bottou L. 2017. Wasserstein generative adversarial networks//Proceedings of the 34th International Conference on Machine Learning. Sydney， Australia： PMLR： 214-223
Cui Z， Zhang L Z and Zhu M L. 2001. Digitally reconstructed radiographs based on ray tracing. Mini-Micro Systems， 22（10）： 1258-1260
崔智， 张良震， 朱淼良. 2001. 基于光线跟踪的数字影像重建技术. 小型微型计算机系统， 22（10）： 1258-1260 ［DOI： 10.3969/j.issn.1000-1220.2001.10.028http://dx.doi.org/10.3969/j.issn.1000-1220.2001.10.028］
Dhont J， Verellen D， Mollaert I， Vanreusel V and Vandemeulebroucke J. 2020. RealDRR-Rendering of realistic digitally reconstructed radiographs using locally trained image-to-image translation. Radiotherapy and Oncology， 153： 213-219 ［DOI： 10.1016/j.radonc.2020.10.004http://dx.doi.org/10.1016/j.radonc.2020.10.004］
Feldkamp L A， Davis L C and Kress J W. 1984. Practical cone-beam algorithm. Journal of the Optical Society of America A， 1（6）： 612-619 ［DOI： 10.1364/JOSAA.1.000612http://dx.doi.org/10.1364/JOSAA.1.000612］
Gulrajani I， Ahmed F， Arjovsky M， Dumoulin V and Courville A. 2017. Improved training of wasserstein GANs//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach， USA： Curran Associates Inc.： 5769-5779
Isola P， Zhu J Y， Zhou T H and Efros A A. 2017. Image-to-image translation with conditional adversarial networks//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu， USA： IEEE： 1125-1134 ［DOI： 10.1109/CVPR.2017.632http://dx.doi.org/10.1109/CVPR.2017.632］
Joseph P M. 1982. An improved algorithm for reprojecting rays through pixel images. IEEE Transactions on Medical Imaging， 1（3）： 192-196 ［DOI： 10.1109/TMI.1982.4307572http://dx.doi.org/10.1109/TMI.1982.4307572］
Li R J， Lewis J H， Jia X， Gu X J， Folkerts M， Men C H， Song W Y and Jiang S B. 2011. 3D tumor localization through real-time volumetric x-ray imaging for lung cancer radiotherapy. Medical Physics， 38（5）： 2783-2794 ［DOI： 10.1118/1.3582693http://dx.doi.org/10.1118/1.3582693］
Liu K， Lyu X Q， Gu Y， Yu H F， Ren G Y and Zhang M. 2016. The 2D/3D medical image registration algorithm based on rapid digital image reconstruction. Journal of Image and Graphics， 21（1）： 69-77
刘坤， 吕晓琪， 谷宇， 于荷峰， 任国印， 张明. 2016. 快速数字影像重建的2维/3维医学图像配准. 中国图象图形学报， 21（1）： 69-77 ［DOI： 10.11834/jig.20160109http://dx.doi.org/10.11834/jig.20160109］
Moore C S， Wood T J， Saunderson J R and Beavis A W. 2017. A method to incorporate the effect of beam quality on image noise in a digitally reconstructed radiograph （DRR） based computer simulation for optimisation of digital radiography. Physics in Medicine and Biology， 62（18）： 7379-7393 ［DOI： 10.1088/1361-6560/aa81fbhttp://dx.doi.org/10.1088/1361-6560/aa81fb］
Oulbacha R and Kadoury S. 2020. MRI to CT synthesis of the lumbar spine from a pseudo-3D cycle GAN//Proceedings of the 17th IEEE International Symposium on Biomedical Imaging. Iowa City， USA： IEEE： 1784-1787 ［DOI： 10.1109/ISBI45749.2020.9098421http://dx.doi.org/10.1109/ISBI45749.2020.9098421］
Shieh C C， Keall P J， Kuncic Z， Huang C Y and Feain I. 2015. Markerless tumor tracking using short kilovoltage imaging arcs for lung image-guided radiotherapy. Physics in Medicine and Biology， 60（24）： 9437-9454 ［DOI： 10.1088/0031-9155/60/24/9437http://dx.doi.org/10.1088/0031-9155/60/24/9437］
Tien H J， Yang H C， Shueng P W and Chen J C. 2021. Cone-beam CT image quality improvement using Cycle-Deblur consistent adversarial networks （Cycle-Deblur GAN） for chest CT imaging in breast cancer patients. Scientific Reports， 11（1）： #1133 ［DOI： 10.1038/s41598-020-80803-2http://dx.doi.org/10.1038/s41598-020-80803-2］
Unberath M， Zaech J N， Lee S C， Bier B， Fotouhi J， Armand M and Navab N. 2018. DeepDRR——A catalyst for machine learning in fluoroscopy-guided procedures//Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention. Granada， Spain： Springer： 98-106 ［DOI： 10.1007/978-3-030-00937-3_12http://dx.doi.org/10.1007/978-3-030-00937-3_12］
Wei R， Liu B， Zhou F G， Bai X Z， Fu D S， Liang B and Wu Q W. 2020a. A patient-independent CT intensity matching method using conditional generative adversarial networks （cGAN） for single x-ray projection-based tumor localization. Physics in Medicine and Biology， 65（14）： #145009 ［DOI： 10.1088/1361-6560/ab8bf2http://dx.doi.org/10.1088/1361-6560/ab8bf2］
Wei R， Zhou F G， Liu B， Bai X Z， Fu D S， Liang B and Wu Q W. 2020b. Real-time tumor localization with single x-ray projection at arbitrary gantry angles using a convolutional neural network （CNN）. Physics in Medicine and Biology， 65（6）： #065012 ［DOI： 10.1088/1361-6560/ab66e4http://dx.doi.org/10.1088/1361-6560/ab66e4］
Xu Y， Yan H， Ouyang L， Wang J， Zhou L H， Cervino L， Jiang S B and Jia X. 2015. A method for volumetric imaging in radiotherapy using single x-ray projection. Medical Physics， 42（5）： 2498-2509 ［DOI： 10.1118/1.4918577http://dx.doi.org/10.1118/1.4918577］
Xue W F， Zhang L， Mou X Q and Bovik A C. 2014. Gradient magnitude similarity deviation： a highly efficient perceptual image quality index. IEEE Transactions on Image Processing， 23（2）： 684-695 ［DOI： 10.1109/TIP.2013.2293423http://dx.doi.org/10.1109/TIP.2013.2293423］
Yan F. 2010. Research on Digitally Reconstructed Radiograph Rendering Method and its Application. Hefei， China： Hefei University of Technology
闫锋. 2010. 数字重建放射影像生成方法及其应用研究. 合肥： 合肥工业大学
Zhang L， Zhang L， Mou X Q and Zhang D. 2011. FSIM： a feature similarity index for image quality assessment. IEEE Transactions on Image Processing， 20（8）： 2378-2386 ［DOI： 10.1109/TIP.2011.2109730http://dx.doi.org/10.1109/TIP.2011.2109730］
Zhu J Y， Park T， Isola P and Efros A A. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice， Italy： IEEE： 2242-2251 ［DOI： 10.1109/ICCV.2017.244http://dx.doi.org/10.1109/ICCV.2017.244］
相关文章
相关作者
相关机构
京公网安备11010802024621