浏览全部资源
扫码关注微信
Received：17 October 2023，
Revised：2023-11-29，
Published：16 October 2024
移动端阅览
在大型高分辨显示器和头戴式显式设备中实现实时、逼真的渲染仍然是计算机图形学面临的主要挑战之一。注视点渲染（foveated rendering）利用人类视觉系统的局限性，根据注视点调整图像渲染质量，从而在不损失用户感知质量的前提下大大提高渲染速度。随着深度学习方法在渲染领域的广泛应用，涌现出大量基于深度学习的注视点渲染新方法。本文从深度学习的角度对注视点渲染领域的最新方法进行综述。首先，概述了人类视觉感知的背景知识。接着，简要介绍了注视点渲染中最具代表性的非深度学习方法，包括自适应分辨率、几何简化、着色简化和硬件实现，并总结了这些方法的优缺点。随后，描述了文中用于评估深度学习不同方法所使用的评估准则，包括常用的注视点渲染图像的评估指标和注视点预测评估指标。接下来，将注视点渲染中的深度学习方法细分为超分辨率、降噪、补全、图像合成、注视点预测和图像应用，对它们进行详细概述和总结。最后，提出了深度学习方法目前面临的问题和挑战。通过对注视点渲染领域的深度学习方法的讨论，可以更详细地展示深度学习在注视点渲染中的研究前景和发展方向，对后续研究人员在选择研究方向和设计网络架构等方面都有一定的参考价值。
The widespread adoption of virtual reality （VR） and augmented reality technologies across various sectors， including healthcare， education， military， and entertainment， has propelled head-mounted displays with high resolution and wide fields of view into the forefront of display devices. However， attaining a satisfactory level of immersion and interactivity poses a primary challenge in the realm of VR， with latency potentially leading to user discomfort in the form of dizziness and nausea. Multiple studies have underscored the necessity of achieving a highly realistic VR experience while maintaining user comfort， entailing the elevation of the screen’s image refresh rate to 1 800 Hz and keeping latency below 3～40 ms. Achieving real-time， photorealistic rendering at high resolution and low latency represents a formidable objective. Foveated rendering is an effective approach to address these issues by adjusting the rendering quality across the image based on gaze position， maintaining high quality in the fovea area while reducing quality in the periphery. This technique leads to substantial computational savings and improved rendering speed without a perceptible loss in visual quality. While previous reviews have examined technical approaches to foveated rendering， they focused more on categorizing the implementation techniques. A comprehensive review within the domain of machine learning still needs to be explored. With the ongoing advancements in machine learning within the rendering field， combining machine learning and foveated rendering is considered a promising research area， especially in postprocessing， where machine learning methods have great potential. Nonmachine learning methods inevitably introduce artifacts. By contrast， machine learning methods have a wide range of applications in the postprocessing domain of rendering to optimize and improve foveated rendering results and enhance the realism and immersion of foveated images in a manner unattainable through nonmachine learning approaches. Therefore， this work presents a comprehensive overview of foveated rendering from a machine-learning perspective. In this paper， we first provide an overview of the background knowledge of human visual perception， including aspects of the human visual system， contrast sensitivity functions， visual acuity models， and visual crowding. Subsequently， this paper briefly describes the most representative nonmachine learning methods for point-of-attention rendering， including adaptive resolution， geometric simplification， shading simplification， and hardware implementation， and summarizes these methods’ features， advantages， and disadvantages. Additionally， we describe the criteria employed for method evaluation in this review， including evaluation metrics for foveated images and gaze-point prediction. Next， we subdivide machine learning methods into super-resolution， denoise， image reconstruction， image synthesis， gaze prediction， and image application. We provide a detailed summary of them in terms of four aspects： results quality， network speed， user experience， and the ability to handle objects. Among them， super-resolution methods commonly use more neural blocks in the foveal region while fewer neural blocks in the periphery region， resulting in variable regional super-resolution quality. Similarly， foveated denoising usually performs fine denoising in the fovea and coarse denoising in the peripheral， but the denoising aspect has yet to receive extensive attention. The initial attempt to integrate image reconstruction with gaze utilized generative adversarial networks （GANs）， yielding promising outcomes. Then， some researchers combined direct prediction and kernel prediction for image reconstruction， which is also the state of the art in this field. Gaze prediction is a key development direction for future VR rendering， which is mostly combined with saliency detection to predict the location of the viewpoint. Substantial work remains in the field， but unfortunately， only a tiny portion of the work can be achieved in real time. Finally， we present the current problems and challenges machine learning methods face. Our review of machine learning approaches in foveated rendering not only elucidates the research prospects and developmental direction but also provides insights for future researchers in choosing research direction and designing network architectures.
Arabadzhiyska E ， Tursun O T ， Myszkowski K ， Seidel H P and Didyk P . 2017 . Saccade landing position prediction for gaze-contingent rendering . ACM Transactions on Graphics ， 36 （ 4 ）： # 50 ［ DOI： 10.1145/3072959.3073642 http://dx.doi.org/10.1145/3072959.3073642 ］
Bauer D ， Wu Q and Ma K L . 2023 . FoVolNet： fast volume rendering using foveated deep neural networks . IEEE Transactions on Visualization and Computer Graphics ， 29 （ 1 ）： 515 - 525 ［ DOI： 10.1109/TVCG.2022.3209498 http://dx.doi.org/10.1109/TVCG.2022.3209498 ］
Chen M X ， Webb R and Bovik A C . 2022 . Foveated MOVI-Codec： foveation-based deep video compression without motion // 2022 IEEE 14th Image， Video， and Multidimensional Signal Processing Workshop （IVMSP） . Nafplio， Greece ： IEEE： 1 - 5 ［ DOI： 10.1109/IVMSP54334.2022.9816221 http://dx.doi.org/10.1109/IVMSP54334.2022.9816221 ］
Cornia M ， Baraldi L ， Serra G and Cucchiara R . 2018 . Predicting human eye fixations via an LSTM-based saliency attentive model . IEEE Transactions on Image Processing ， 27 （ 10 ）： 5142 - 5154 ［ DOI： 10.1109/TIP.2018.2851672 http://dx.doi.org/10.1109/TIP.2018.2851672 ］
Cuervo E ， Chintalapudi K and Kotaru M . 2018 . Creating the perfect illusion： what will it take to create life-like virtual reality headsets? // Proceedings of the 19th International Workshop on Mobile Computing Systems and Applications . Tempe， USA ： ACM： 7 - 12 ［ DOI： 10.1145/3177102.3177115 http://dx.doi.org/10.1145/3177102.3177115 ］
Frank L H ， Casali J G and Wierwille W W . 1988 . Effects of visual display and motion system delays on operator performance and uneasiness in a driving simulator . Human Factors ， 30 （ 2 ）： 201 - 217 ［ DOI： 10.1177/001872088803000207 http://dx.doi.org/10.1177/001872088803000207 ］
Franke L ， Fink L ， Martschinke J ， Selgrad K and Stamminger M . 2021 . Time-warped foveated rendering for virtual reality headsets . Computer Graphics Forum ， 40 （ 1 ）： 110 - 123 ［ DOI： 10.1111/cgf.14176 http://dx.doi.org/10.1111/cgf.14176 ］
Fridman L ， Jenik B ， Keshvari S ， Reimer B ， Zetzsche C and Rosenholtz R . 2017 . SideEye： a generative neural network based simulator of human peripheral vision ［EB/OL］. ［ 2023-09-25 ］. https://arxiv.org/pdf/1706.04568.pdf https://arxiv.org/pdf/1706.04568.pdf
Friess F ， Braun M ， Bruder V ， Frey S ， Reina G and Ertl T . 2021 . Foveated encoding for large high-resolution displays . IEEE Transactions on Visualization and Computer Graphics ， 27 （ 2 ）： 1850 - 1859 ［ DOI： 10.1109/TVCG.2020.3030445 http://dx.doi.org/10.1109/TVCG.2020.3030445 ］
Guenter B ， Finch M ， Drucker S ， Tan D and Snyder J . 2012 . Foveated 3D graphics . ACM Transactions on Graphics ， 31 （ 6 ）： # 164 ［ DOI： 10.1145/2366145.2366183 http://dx.doi.org/10.1145/2366145.2366183 ］
Guo F ， Shen J B and Li X L . 2014 . Learning to detect stereo saliency // Proceedings of 2014 IEEE International Conference on Multimedia and Expo （ICME） . Chengdu， China ： IEEE： 1 - 6 ［ DOI： 10.1109/ICME.2014.6890321 http://dx.doi.org/10.1109/ICME.2014.6890321 ］
Horvitz E J and Lengyel J . 2013 . Perception， attention， and resources： a decision-theoretic approach to graphics rendering ［EB/OL］. ［ 2023-09-25 ］. https://arxiv.org/pdf/1302.1547.pdf https://arxiv.org/pdf/1302.1547.pdf
Hsu C F ， Chen A ， Hsu C H ， Huang C Y ， Lei C L and Chen K T . 2017 . Is foveated rendering perceivable in virtual reality？： Exploring the efficiency and consistency of quality assessment methods // Proceedings of the 25th ACM International Conference on Multimedia . Mountain View， USA ： ACM： 55 - 63 ［ DOI： 10.1145/3123266.3123434 http://dx.doi.org/10.1145/3123266.3123434 ］
Huang X ， Shen C Y ， Boix X and Zhao Q . 2015 . SALICON： reducing the semantic gap in saliency prediction by adapting deep neural networks // Proceedings of 2015 IEEE International Conference on Computer Vision （ICCV） . Santiago， Chile ： IEEE： 262 - 270 ［ DOI： 10.1109/ICCV.2015.38 http://dx.doi.org/10.1109/ICCV.2015.38 ］
Huong T T ， Ha D T ， Tran H T T ， Viet N D ， Tien B D ， Thanh N H ， Thang T C and Nam P N . 2022 . An effective foveated 360° image assessment based on graph convolution network . IEEE Access ， 10 ： 98165 - 98178 ［ DOI： 10.1109/access.2022.3204766 http://dx.doi.org/10.1109/access.2022.3204766 ］
Illahi G ， Siekkinen M and Masala E . 2017 . Foveated video streaming for cloud gaming // Proceedings of the 19th IEEE International Workshop on Multimedia Signal Processing （MMSP） . Luton， UK ： IEEE： 1 - 6 ［ DOI： 10.1109/MMSP.2017.8122235 http://dx.doi.org/10.1109/MMSP.2017.8122235 ］
Itti L ， Koch C and Niebur E . 1998 . A model of saliency-based visual attention for rapid scene analysis . IEEE Transactions on Pattern Analysis and Machine Intelligence ， 20 （ 11 ）： 1254 - 1259 ［ DOI： 10.1109/34.730558 http://dx.doi.org/10.1109/34.730558 ］
Jaramillo-Avila U and Anderson S R . 2019 . Foveated image processing for faster object detection and recognition in embedded systems using deep convolutional neural networks //Martinez-Hernandez U， Vouloutsi V， Mura A， Mangan M， Asada M， Prescott T J and Verschure P F M J， eds. 8th International Conference on Biomimetic and Biohybrid Systems . Nara， Japan ： Springer： 193 - 204 ［ DOI： 10.1007/978-3-030-24741-6_17 http://dx.doi.org/10.1007/978-3-030-24741-6_17 ］
Jiang J J ， Cheng H ， Li Z Y ， Liu X M and Wang Z Y . 2023 . Deep learning based video-related super-resolution technique： a survey . Journal of Image and Graphics ， 28 （ 7 ）： 1927 - 1964
江俊君 ， 程豪 ， 李震宇 ， 刘贤明 ， 王中元 . 2023 . 深度学习视频超分辨率技术综述 . 中国图象图形学报 ， 28 （ 7 ）： 1927 - 1964 ［ DOI： 10.11834/jig.220130 http://dx.doi.org/10.11834/jig.220130 ］
Ju Y G and Park J H . 2019 . Foveated computer-generated hologram and its progressive update using triangular mesh scene model for near-eye displays . Optics Express ， 27 （ 17 ）： 23725 - 23738 ［ DOI： 10.1364/OE.27.023725 http://dx.doi.org/10.1364/OE.27.023725 ］
Kaplanyan A S ， Sochenov A ， Leimkühler T ， Okunev M ， Goodall T and Rufo G . 2019 . DeepFovea： neural reconstruction for foveated rendering and video compression using learned statistics of natural videos . ACM Transactions on Graphics ， 38 （ 6 ）： # 212 ［ DOI： 10.1145/3355089.3356557 http://dx.doi.org/10.1145/3355089.3356557 ］
Kim J ， Jeong Y ， Stengel M ， Akşit K ， Albert R ， Boudaoud B ， Greer T ， Kim J ， Lopes W ， Majercik Z ， Shirley P ， Spjut J ， McGuire M and Luebke D . 2019 . Foveated AR： dynamically-foveated augmented reality display . ACM Transactions on Graphics ， 38 （ 4 ）： # 99 ［ DOI： 10.1145/3306346.3322987 http://dx.doi.org/10.1145/3306346.3322987 ］
Kim S ， Seo M W ， Lee S J and Kang S J . 2018 . Object tracking-based foveated super-resolution convolutional neural network for head mounted display // SIGGRAPH Asia 2018 Posters . Tokyo， Japan ： ACM： #60 ［ DOI： 10.1145/3283289.3283325 http://dx.doi.org/10.1145/3283289.3283325 ］
Koskela M ， Viitanen T ， Jääskeläinen P and Takala J . 2016 . Foveated path tracing： a literature review and a performance gain analysis //Bebis G， Boyle R， Parvin B， Koracin D， Porikli F， Skaff S， Entezari A， Min J Y， Iwai D， Sadagic A， Scheidegger C and Isenberg T， eds. 12th International Symposium on Advances in Visual Computing . Las Vegas， USA ： Springer International Publishing： 723 - 732 ［ DOI： 10.1007/978-3-319-50835-1_65 http://dx.doi.org/10.1007/978-3-319-50835-1_65 ］
Krajancich B ， Kellnhofer P and Wetzstein G . 2021 . A perceptual model for eccentricity-dependent spatio-temporal flicker fusion and its applications to foveated graphics . ACM Transactions on Graphics ， 40 （ 4 ）： # 47 ［ DOI： 10.1145/3450626.3459784 http://dx.doi.org/10.1145/3450626.3459784 ］
Kroner A ， Senden M ， Driessens K and Goebel R . 2020 . Contextual encoder-decoder network for visual saliency prediction . Neural Networks ， 129 ： 261 - 270 ［ DOI： 10.1016/j.neunet.2020.05.004 http://dx.doi.org/10.1016/j.neunet.2020.05.004 ］
Kruthiventi S S S ， Ayush K and Babu R V . 2017 . DeepFix： a fully convolutional neural network for predicting human eye fixations . IEEE Transactions on Image Processing ， 26 （ 9 ）： 4446 - 4456 ［ DOI： 10.1109/TIP.2017.2710620 http://dx.doi.org/10.1109/TIP.2017.2710620 ］
Kümmerer M ， Bethge M and Wallis T S A . 2022 . DeepGaze III： modeling free-viewing human scanpaths with deep learning . Journal of Vision ， 22 （ 5 ）： # 7 ［ DOI： 10.1167/jov.22.5.7 http://dx.doi.org/10.1167/jov.22.5.7 ］
Kümmerer M ， Theis L and Bethge M . 2015 . Deep gaze I： boosting saliency prediction with feature maps trained on ImageNet ［EB/OL］. ［ 2023-09-25 ］. https://arxiv.org/pdf/1411.1045.pdf https://arxiv.org/pdf/1411.1045.pdf
Kümmerer M ， Wallis T S A and Bethge M . 2016 . DeepGaze II： reading fixations from deep features trained on object recognition ［EB/OL］. ［ 2023-09-25 ］. https://arxiv.org/pdf/1610.01563.pdf https://arxiv.org/pdf/1610.01563.pdf
Linardos A ， Kummerer M ， Press O and Bethge M . 2021 . DeepGaze IIE： calibrated prediction in and out-of-domain for state-of-the-art saliency modeling // Proceedings of 2021 IEEE/CVF International Conference on Computer Vision （ICCV） . Montreal， Canada ： IEEE： 12899 - 12908 ［ DOI： 10.1109/ICCV48922.2021.01268 http://dx.doi.org/10.1109/ICCV48922.2021.01268 ］
Liu N ， Han J W ， Zhang D W ， Wen S F and Liu T M . 2015 . Predicting eye fixations using convolutional neural networks // Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition （CVPR） . Boston， USA ： IEEE： 362 - 370 ［ DOI： 10.1109/CVPR.2015.7298633 http://dx.doi.org/10.1109/CVPR.2015.7298633 ］
Mantiuk R K ， Ashraf M and Chapiro A . 2022 . stelaCSF： a unified model of contrast sensitivity as the function of spatio-temporal frequency， eccentricity， luminance and area . ACM Transactions on Graphics ， 41 （ 4 ）： # 145 ［ DOI： 10.1145/3528223.3530115 http://dx.doi.org/10.1145/3528223.3530115 ］
Mantiuk R K ， Denes G ， Chapiro A ， Kaplanyan A ， Rufo G ， Bachy R ， Lian T and Patney A . 2021 . FovVideoVDP： a visible difference predictor for wide field-of-view video . ACM Transactions on Graphics ， 40 （ 4 ）： # 49 ［ DOI： 10.1145/3450626.3459831 http://dx.doi.org/10.1145/3450626.3459831 ］
Marianos N X . 2018 . Foveated Rendering Algorithms Using Eye-Tracking Technology in Virtual Reality . Chania， Greece ： Technical University of Crete ： 1 - 113
Milef N and Kalantari N . 2021 . Foveated monte-carlo denoising // Proceedings of 2021 Special Interest Group on Computer Graphics and Interactive Techniques Conference Posters . Virtual Event， USA ： ACM： #33 ［ DOI： 10.1145/3450618.3469140 http://dx.doi.org/10.1145/3450618.3469140 ］
Mohanto B ， Islam A T ， Gobbetti E and Staadt O . 2022 . An integrative view of foveated rendering . Computers and Graphics ， 102 ： 474 - 501 ［DO 18 . Towa I： 10.1016/j.cag.2021.10.010 http://dx.doi.org/10.1016/j.cag.2021.10.010 ］
Molenaar E N . 20rds real-time ray tracing through foveated rendering ［EB/OL］. ［ 2023-09-25 ］. https://studenttheses.uu.nl/bitstream/handle/20.500.12932/28691/Thesis%20Erik%20Molenaar.pdf?sequence=2andisAllowed=y https://studenttheses.uu.nl/bitstream/handle/20.500.12932/28691/Thesis%20Erik%20Molenaar.pdf?sequence=2andisAllowed=y
Nakashima R ， Fang Y ， Hatori Y ， Hiratani A ， Matsumiya K ， Kuriki I and Shioiri S . 2015 . Saliency-based gaze prediction based on head direction . Vision Research ， 117 ： 59 - 66 ［ DOI： 10.1016/j.visres.2015.10.001 http://dx.doi.org/10.1016/j.visres.2015.10.001
Nam H ， Kang H and Cho H . 2022 . 65 - 4 ： foveated super resolution network for virtual reality head-mounted displays . SID Symposium Digest of Technical Papers ， 53 （ 1 ）： 869 - 872 ［ DOI： 10.1002/sdtp.15631 http://dx.doi.org/10.1002/sdtp.15631 ］
Pan X Y ， Jia N X ， Mu Y Z and Gao X R . 2023 . Survey of small object detection . Journal of Image and Graphics ， 28 （ 9 ）： 2587 - 2615
潘晓英 ， 贾凝心 ， 穆元震 ， 高炫蓉 . 2023 . 小目标检测研究综述 . 中国图象图形学报 ， 28 （ 9 ）： 2587 - 2615 ［ DOI： 10.11834/jig.220455 http://dx.doi.org/10.11834/jig.220455 ］
Patney A ， Salvi M ， Kim J ， Kaplanyan A ， Wyman C ， Benty N ， Luebke D and Lefohn A . 2016 . Towards foveated rendering for gaze-tracked virtual reality . ACM Transactions on Graphics ， 35 （ 6 ）： # 179 ［ DOI： 10.1145/2980179.2980246 http://dx.doi.org/10.1145/2980179.2980246 ］
Peuhkurinen A and Mikkonen T . 2021 . Real-time human eye resolution ray tracing in mixed reality // Proceedings of the 16th International Joint Conference on Computer Vision， Imaging and Computer Graphics Theory and Applications . Virtual Event ： SciTePress： 169 - 176 ［ DOI： 10.5220/0010205701690176 http://dx.doi.org/10.5220/0010205701690176 ］
Reed N . 2015 . VR direct： how NVIDIA technology is improving the VR experience ［EB/OL］. ［ 2023-09-25 ］. https://developer.nvidia.com/sites/default/files/akamai/gameworks/vr/GameWorks_VR_2015_Final_handouts.pdf https://developer.nvidia.com/sites/default/files/akamai/gameworks/vr/GameWorks_VR_2015_Final_handouts.pdf
Reich C ， Memmel M and Grebe J H . 2020 . DeepFovea++： reconstruction and super-resolution for natural foveated rendered videos ［EB/OL］. ［ 2023-09-25 ］. https://github.com/ChristophReich1996/DeepFoveaPP_for_Video_Reconstruction_and_Super_Resolution https://github.com/ChristophReich1996/DeepFoveaPP_for_Video_Reconstruction_and_Super_Resolution
Rimac-Drlje S ， Martinović G and Zovko-Cihlar B . 2011 . Foveation-based content Adaptive Structural Similarity index // Proceedings of the 18th International Conference on Systems， Signals and Image Processing . Sarajevo， Bosnia and Herzegovina ： IEEE： 1 - 4
Rimac-Drlje S ， Vranješ M and Žagar D . 2010 . Foveated mean squared error—a novel video quality metric . Multimedia Tools and Applications ， 49 （ 3 ）： 425 - 445 ［ DOI： 10.1007/s11042-009-0442-1 http://dx.doi.org/10.1007/s11042-009-0442-1 ］
Rosenholtz R ， Huang J and Ehinger K A . 2012 . Rethinking the role of top-down attention in vision： effects attributable to a lossy representation in peripheral vision . Frontiers in Psychology ， 3 ： # 13 ［ DOI： 10.3389/fpsyg.2012.00013 http://dx.doi.org/10.3389/fpsyg.2012.00013 ］
Stengel M ， Grogorick S ， Eisemann M and Magnor M . 2016 . Adaptive image-space sampling for gaze-contingent real-time rendering . Computer Graphics Forum ， 35 （ 4 ）： 129 - 139 ［ DOI： 10.1111/cgf.12956 http://dx.doi.org/10.1111/cgf.12956 ］
Surace L ， Wernikowski M ， Tursun C ， Myszkowski K ， Mantiuk R and Didyk P . 2023 . Learning GAN-based foveated reconstruction to recover perceptually important image features . ACM Transactions on Applied Perception ， 20 （ 2 ）： # 7 ［ DOI： 10.1145/3583072 http://dx.doi.org/10.1145/3583072 ］
Swafford N T ， Cosker D and Mitchell K . 2015 . Latency aware foveated rendering in unreal engine 4//Proceedings of the 12th European Conference on Visual Media Production. London， United Kingdom： ACM： # 17 ［ DOI： 10.1145/2824840.2824863 http://dx.doi.org/10.1145/2824840.2824863 ］
Swafford N T ， Iglesias-Guitian J A ， Koniaris C ， Moon B ， Cosker D and Mitchell K . 2016 . User ， metric， and computational evaluation of foveated rendering methods //Proceedings of the ACM Symposium on Applied Perception. Anaheim， California， USA ： ACM： 7 - 14 ［ DOI： 10.1145/2931002.2931011 http://dx.doi.org/10.1145/2931002.2931011 ］
Tariq T ， Tursun C and Didyk P . 2022 . Noise-based enhancement for foveated rendering . ACM Transactions on Graphics ， 41 （ 4 ）： # 143 ［ DOI： 10.1145/3528223.3530101 http://dx.doi.org/10.1145/3528223.3530101 ］
Thumuluri V and Sharma M . 2020 . A unified deep learning approach for foveated rendering and novel view synthesis from sparse RGB-D light fields // Proceedings of 2020 International Conference on 3D Immersion （IC3D） . Brussels， Belgium ： IEEE： 1 - 8 ［ DOI： 10.1109/IC3D51119.2020.9376340 http://dx.doi.org/10.1109/IC3D51119.2020.9376340 ］
Tursun O T ， Arabadzhiyska-Koleva E ， Wernikowski M ， Mantiuk R ， Seidel H P ， Myszkowski K and Didyk P . 2019 . Luminance-contrast-aware foveated rendering . ACM Transactions on Graphics ， 38 （ 4 ）： # 98 ［ DOI： 10.1145/3306346.3322985 http://dx.doi.org/10.1145/3306346.3322985 ］
Vaidyanathan K ， Salvi M ， Toth R ， Foley T ， Akenine-Möller T ， Nilsson J ， Munkberg J ， Hasselgren J ， Sugihara M ， Clarberg P ， Janczak T and Lefohn A . 2014 . Coarse pixel shading ［EB/OL］. ［ 2023-09-25 ］. https://www.intel.cn/content/www/cn/zh/developer/articles/technical/coarse-pixel-shading.html https://www.intel.cn/content/www/cn/zh/developer/articles/technical/coarse-pixel-shading.html
Wang L D ， Hajiesmaili M and Sitaraman R K . 2021 . FOCAS： practical video super resolution using foveated rendering // Proceedings of the 29th ACM International Conference on Multimedia . Virtual Event， China ： ACM： 5454 - 5462 ［ DOI： 10.1145/3474085.3475673 http://dx.doi.org/10.1145/3474085.3475673 ］
Wang L L ， Shi X H and Liu Y . 2023 . Foveated rendering： a state-of-the-art survey . Computational Visual Media ， 9 （ 2 ）： 195 - 228 ［ DOI： 10.1007/s41095-022-0306-4 http://dx.doi.org/10.1007/s41095-022-0306-4 ］
Wang W G ， Shen J B and Shao L . 2015 . Consistent video saliency using local gradient flow optimization and global refinement . IEEE Transactions on Image Processing ， 24 （ 11 ）： 4185 - 4196 ［ DOI： 10.1109/TIP.2015.2460013 http://dx.doi.org/10.1109/TIP.2015.2460013 ］
Wang Z ， Bovik A C ， Lu L G and Kouloheris J L . 2001 . Foveated wavelet image quality index // Proceedings of Volume 4472 ， Applications of Digital Image Processing XXIV. San Diego， United States ： SPIE： 42 - 52 ［ DOI： 10.1117/12.449797 http://dx.doi.org/10.1117/12.449797 ］
Wang Z Q ， Zhang Y S ， Yu Y ， Min J and Tian H . 2022 . Review of deep learning based salient object detection . Journal of Image and Graphics ， 27 （ 7 ）： 2112 - 2128
王自全 ， 张永生 ， 于英 ， 闵杰 ， 田浩 . 2022 . 深度学习背景下视觉显著性物体检测综述 . 中国图象图形学报 ， 27 （ 7 ）： 2112 - 2128 ［ DOI： 10.11834/jig.200649 http://dx.doi.org/10.11834/jig.200649 ］
Weier M ， Stengel M ， Roth T ， Didyk P ， Eisemann E ， Eisemann M ， Grogorick S ， Hinkenjann A ， Kruijff E ， Magnor M ， Myszkowski K and Slusallek P . 2017 . Perception-driven accelerated rendering . Computer Graphics Forum ， 36 （ 2 ）： 611 - 643 ［ DOI： 10.1111/cgf.13150 http://dx.doi.org/10.1111/cgf.13150 ］
Weymouth F W . 1958 . Visual sensory units and the minimal angle of resolution . American Journal of Ophthalmology ， 46 （ 1 ）： 102 - 113 ［ DOI： 10.1016/0002-9394（58）90042-4 http://dx.doi.org/10.1016/0002-9394（58）90042-4 ］
White S ， ClAndrew ， Lucas E D ， Mabee D ， Jenks A ， Sherer T and gongwaner . 2023 . Variable-rate shading （VRS） ［EB/OL］. ［ 2023-09-25 ］. https://learn.microsoft.com/zh-cn/windows/win32/direct3d12/vrs https://learn.microsoft.com/zh-cn/windows/win32/direct3d12/vrs
Wu J X ， Zhong S H ， Ma Z ， Heinen S J and Jiang J M . 2018 . Foveated convolutional neural networks for video summarization . Multimedia Tools and Applications ， 77 （ 22 ）： 29245 - 29267 ［ DOI： 10.1007/s11042-018-5953-1 http://dx.doi.org/10.1007/s11042-018-5953-1 ］
Xiao K ， Liktor G and Vaidyanathan K . 2018 . Coarse pixel shading with temporal supersampling // Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games . Montreal， Canada ： ACM： #1 ［ DOI： 10.1145/3190834.3190850 http://dx.doi.org/10.1145/3190834.3190850 ］
Xu Y Y ， Dong Y B ， Wu J R ， Sun Z Z ， Shi Z R ， Yu J Y and Gao S H . 2018 . Gaze prediction in dynamic 360° immersive videos // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City， USA ： IEEE： 5333 - 5342 ［ DOI： 10.1109/CVPR.2018.00559 http://dx.doi.org/10.1109/CVPR.2018.00559 ］
Yang H ， Chen R ， An S P ， Wei H and Zhang H . 2023 . The growth of image-related three dimensional reconstruction techniques in deep learning-driven era： a critical summary . Journal of Image and Graphics ， 28 （ 8 ）： 2396 - 2409
杨航 ， 陈瑞 ， 安仕鹏 ， 魏豪 ， 张衡 . 2023 . 深度学习背景下的图像三维重建技术进展综述 . 中国图象图形学报 ， 28 （ 8 ）： 2396 - 2409 ［ DOI： 10.11834/jig.220376 http://dx.doi.org/10.11834/jig.220376 ］
Yoo C ， Xiong J H ， Moon S ， Yoo D ， Lee C K ， Wu S T and Lee B . 2020 . Foveated display system based on a doublet geometric phase lens . Optics Express ， 28 （ 16 ）： 23690 - 23702 ［ DOI： 10.1364/OE.399808 http://dx.doi.org/10.1364/OE.399808 ］
Zare A ， Aminlou A ， Hannuksela M M and Gabbouj M . 2016 . HEVC-compliant tile-based streaming of panoramic video for virtual reality applications // Proceedings of the 24th ACM International Conference on Multimedia . Amsterdam， the Netherlands ： ACM： 601 - 605 ［ DOI： 10.1145/2964284.2967292 http://dx.doi.org/10.1145/2964284.2967292 ］
Zhang X C ， Sugano Y ， Fritz M and Bulling A . 2015 . Appearance-based gaze estimation in the wild // Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition （CVPR） . Boston， USA ： IEEE： 4511 - 4520 ［ DOI： 10.1109/CVPR.2015.7299081 http://dx.doi.org/10.1109/CVPR.2015.7299081 ］
Zhang X C ， Sugano Y ， Fritz M and Bulling A . 2017 . It’s written all over your face： Full-face appearance-based gaze estimation // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops （CVPRW） . Honolulu， USA ： IEEE： 2299 - 2308 ［ DOI： 10.1109/CVPRW.2017.284 http://dx.doi.org/10.1109/CVPRW.2017.284 ］
Zhao Y Q ， Rao Y ， Dong S P and Zhang J Y . 2020 . Survey on deep learning object detection . Journal of Image and Graphics ， 25 （ 4 ）： 629 - 654
赵永强 ， 饶元 ， 董世鹏 ， 张君毅 . 2020 . 深度学习目标检测方法综述 . 中国图象图形学报 ， 25 （ 4 ）： 629 - 654 ［ DOI： 10.11834/jig.190307 http://dx.doi.org/10.11834/jig.190307 ］
Zheng Z P ， Cao W J ， Zhou W J ， Yang Z and Zhan Y W . 2020 . Perceptual model based surface tessellation for VR foveated rendering // Proceedings of the 6th International Conference on Robotics and Artificial Intelligence . Singapore， Singapore ： ACM： 64 - 68 ［ DOI： 10.1145/3449301.3449313 http://dx.doi.org/10.1145/3449301.3449313 ］
Zhu F ， Lu P ， Li P ， Sheng B and Mao L J . 2020 . Gaze-contingent rendering in virtual reality //Magnenat-Thalmann N， Stephanidis C， Wu E H， Thalmann D， Sheng B， Kim J， Papagiannakis G and Gavrilova M， eds. 37th Computer Graphics International Conference on Advances in Computer Graphics . Geneva， Switzerland ： Springer International Publishing： 16 - 23 ［ DOI： 10.1007/978-3-030-61864-3_2 http://dx.doi.org/10.1007/978-3-030-61864-3_2 ］
相关文章
相关作者
相关机构
京公网安备11010802024621
