浏览全部资源
扫码关注微信
收稿：2023-05-09，
修回：2023-08-18，
纸质出版：2024-04-16
移动端阅览
目的
2
传统降噪方法通常忽视人眼感知因素，对不同区域的图像块都进行同等处理。当使用传统降噪算法对全景画面滤波处理时，全景画面两极区域容易产生模糊问题，尤其是通过视口观察时，该问题更加明显。针对此问题，提出一种视觉显著性驱动的蒙特卡洛渲染生成全景图非局部均值（visual saliency driven non-local means，VSD-NLM）滤波降噪算法。
方法
2
在VSD-NLM算法中首先使用全景图显著区域检测算法获取全景画面的显著区域；然后使用梯度幅值相似性偏差辅助的非局部均值（gradient magnitude similarity deviation assisted non-local means，GMSDA-NLM）滤波算法，降低显著区域的噪声；同时设计并行非局部均值（parallel non-local means，P-NLM）滤波算法，加快降噪处理速度，降低非显著区域噪声；最后利用改进的Canny算法提取梯度特征，同时结合各向异性扩散引导滤波来优化降噪结果。
结果
2
采用结构相似度（structural similarity，SSIM）和FLIP作为评价指标，来对比VSD-NLM算法与非局部均值滤波算法、多特征非局部均值滤波算法以及渐进式去噪算法等其他算法的性能。实验结果表明，VSD-NLM算法的降噪结果在客观评价指标上均优于对比算法，SSIM值比其他算法平均提高14.7%，FLIP值比其他算法平均降低15.2%。在视觉效果方面，VSD-NLM算法能够减轻全景画面模糊，提升视觉感知质量。本文对GMSDA-NLM和P-NLM算法的有效性进行了实验验证，相较于非局部均值滤波算法，GMSDA-NLM算法能够有效去除噪声并保持图像细节的完整性。P-NLM算法在运行速度方面相较对比算法平均提高约6倍，与串行算法生成的图像之间的SSIM值可达到0.996。
结论
2
本文算法能够更好地用于全景图降噪，滤波效果更佳，对全景电影制作应用有重要的理论和实际意义。
Objective
2
Panoramic movie technology has experienced notable advancements to enrich the audiovisual experience for viewers， resulting in a heightened sense of immersion within the visual environment. Nevertheless， the production of high-quality images poses a challenge for conventional rasterization techniques， necessitating the exploration of alternative approaches. Monte Carlo path tracing algorithms have been proven effective in generating high-quality images， offering exceptional visual fidelity in various rendering applications. However， the computational overhead associated with this algorithm remains challenging. Thus， reducing the number of pixels sampled in Monte Carlo path tracing is a common approach to optimize computation. However， this reduction often introduces noticeable noise in the resulting images， compromising their overall quality. This paper aims to address the issue of image noise in Monte Carlo path tracing by exploring and proposing advanced techniques for denoising. Two main denoising approaches are commonly used in the domain of Monte Carlo rendering. The first approach utilizes traditional filtering methods with artificially designed filters to remove image noise. This approach is versatile， but its effectiveness in noise removal may be limited， often resulting in residual noise. The second approach involves deep learning-based denoising methods， which can effectively eliminate noise but may exhibit performance limitations on specific image types. Most existing image denoising algorithms are currently developed and studied for ordinary flat images， with limited research dedicated to denoising algorithms specifically designed for panoramic images. Panoramic images possess unique characteristics， including a 360° field of view in the horizontal direction， a 180° field of view in the vertical direction， distorted edges， and varying prominence of equatorial and polar pixels as perceived by human observers. Conventional flat image denoising methods often fail to fully account for these panoramic image characteristics， leading to excessive blurring or residual noise in the equatorial， polar， and distorted edge regions after the denoising process. Therefore， this paper proposes a visual saliency-driven non-local means （VSD-NLM） filtering denoising algorithm explicitly tailored for Monte Carlo rendering of panoramic images. The algorithm aims to leverage the distinctive characteristics of panoramic images， such as the 360° field of view， distorted edges， and varying pixel prominence， to effectively reduce noise while preserving the essential features of panoramic images. Through comprehensive experimentation and evaluation， the proposed algorithm demonstrates its efficacy in enhancing the image quality of Monte Carlo-rendered panoramic images， providing a valuable contribution to the field of panoramic image denoising.
Method
2
This paper presents the design and optimization of the VSD-NLM filtering algorithm for denoising Monte Carlo-rendered panoramic images. The proposed algorithm comprises two key components aimed at effectively removing noise and enhancing image quality in panoramic scenes. The first component focuses on enhancing the non-local means filtering process specifically tailored for panoramic images. Initially， a panoramic image saliency detection model is utilized to generate a saliency image， incorporating an equatorial bias to improve saliency accuracy. Subsequently， the saliency image is employed to delineate saliency and non-saliency regions within the panoramic image. In the saliency region， the deviation value of the gradient magnitude similarity between image blocks is calculated to refine the weights used in non-local means filtering. For the non-saliency region， parallel algorithms for non-local means filtering are devised to accelerate the filter reconstruction process. Finally， denoising results from the saliency and non-saliency regions are combined to produce the final denoised panoramic image. The second component of the algorithm focuses on optimized noise reduction， specifically addressing the distorted edge regions of the panoramic image. Improvements are made to the Canny algorithm to obtain a highly accurate edge gradient image. Such improvements involve optimizing the weights for the 45° and 135° directions of the image， generating adaptive high and low thresholds using an improved Otsu method， and enhancing the local thresholds to optimize the performance of the Canny operator. Subsequently， anisotropic diffusion filtering is combined with guided filtering by utilizing the gradient image as a guide to filter and enhance the combined images. The optimizations of the proposed algorithm collectively contribute to effective noise reduction in the distorted edge regions of panoramic images， resulting in enhanced image quality and reduced noise artifacts.
Result
2
This paper presents a comprehensive performance evaluation of the proposed denoising algorithm for panoramic images and utilizes structural similarity （SSIM） and FLIP metrics as objective evaluation indicators. The performance of the VSD-NLM algorithm is compared with other algorithms such as non-local means filtering， multifeature non-local means filtering， and progressive denoising algorithms to assess its effectiveness in reducing noise and improving the visual quality of panoramic images. Experimental results reveal that the proposed algorithm outperforms the comparison algorithms in terms of objective evaluation indicators. The average FLIP value achieved by the proposed algorithm is 15.2% lower compared with other algorithms. Similarly， the average SSIM value attained by the proposed algorithm is 14.7% higher than other algorithms， indicating its enhanced SSIM preservation. Furthermore， the visual effects of the algorithm are assessed， demonstrating its capability to mitigate blurring artifacts in panoramic images and enhance visual perception quality. This paper also presents an experimental verification of the effectiveness of two denoising algorithms： gradient magnitude similarity deviation assisted non-local means （GMSDA-NLM） and parallel non-local means （P-NLM）. The GMSDA-NLM algorithm combines the strengths of non-local mean filtering and gradient magnitude similarity deviation to achieve superior noise reduction capabilities while maintaining the integrity of image details. This algorithm effectively identifies and suppresses noise while preserving the essential image features. The P-NLM algorithm exhibits a notable average speed increase of approximately six times compared to the nonparallel algorithm， facilitating real-time or near-real-time noise reduction applications. The SSIM value between P-NLM and the image generated by the nonparallel algorithm can reach 0.996.
Conclusion
2
This paper introduces a specialized denoising algorithm tailored for panoramic images， specifically addressing the unique challenges associated with denoising in this domain. From a practical perspective， the proposed algorithm holds substantial value for panoramic film production. The algorithm enhances the visual quality and fidelity of panoramic films by markedly reducing noise in panoramic images. The remarkable results obtained through the proposed algorithm contribute to immersive visual storytelling， elevating the overall cinematic experience and capturing the attention of audiences. Overall， the exceptional results achieved through the algorithm present valuable theoretical advancements and provide practical implications for panoramic film production， enhancing the quality and impact of visual narratives in the realm of immersive cinematography.
Andersson P ， Nilsson J ， Akenine-Möller T ， Oskarsson M ， Åström K and Fairchild M D . 2020 . FLIP： a difference evaluator for alternating images // Proceedings of ACM on Computer Graphics and Interactive Techniques ， 3 （ 2 ）： #15 ［ DOI： 10.1145/3406183 http://dx.doi.org/10.1145/3406183 ］
Balint M ， Wolski K ， Myszkowski K ， Seidel H P and Mantiuk R . 2023 . Neural partitioning pyramids for denoising Monte Carlo renderings // Proceedings of ACM SIGGRAPH 2023 Conference . Los Angeles， USA ： ACM： #60 ［ DOI： 10.1145/3588432.3591562 http://dx.doi.org/10.1145/3588432.3591562 ］
Buades A ， Coll B and Morel J M . 2005 . A non-local algorithm for image denoising // Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition . San Diego， USA ： IEEE： 60 - 65 ［ DOI： 10.1109/CVPR.2005.38 http://dx.doi.org/10.1109/CVPR.2005.38 ］
Canny J . 1986 . A computational approach to edge detection . IEEE Transactions on Pattern Analysis and Machine Intelligence， PAMI-8 （ 6 ）： 679 - 698 ［ DOI： 10.1109/TPAMI.1986.4767851 http://dx.doi.org/10.1109/TPAMI.1986.4767851 ］
Cao F D ， Zhao H C ， Liu P F and Li P X . 2021 . Local filtering framework using sorting clustering . Journal of Computer-Aided Design and Computer Graphics ， 33 （ 10 ）： 1532 - 1540
曹飞道 ， 赵怀慈 ， 刘鹏飞 ， 李培玄 . 2021 . 利用排序聚类的局部滤波框架 . 计算机辅助设计与图形学学报 ， 33 （ 10 ）： 1532 - 1540 ［ DOI： 10.3724/SP.J.1089.2021.18624 http://dx.doi.org/10.3724/SP.J.1089.2021.18624 ］
Chen L L ， Zhou X D ， Xie J C and Liu Q . 2021 . Fast affine non-local means image denosing . Journal of Graphics ， 42 （ 5 ）： 762 - 766
陈玲玲 ， 周旭东 ， 谢傢成 ， 刘乾 . 2021 . 快速仿射非局部均值图像去噪 . 图学学报 ， 42 （ 5 ）： 762 - 766 ［ DOI： 10.11996/JG.j.2095-302X.2021050762 http://dx.doi.org/10.11996/JG.j.2095-302X.2021050762 ］
Chen Y K ， Lu Y F ， Zhang X H and Xie N . 2023 . Interactive neural cascade denoising for 1-spp Monte Carlo images . The Visual Computer ， 39 （ 8 ）： 3197 - 3210 ［ DOI： 10.1007/s00371-023-02951-6 http://dx.doi.org/10.1007/s00371-023-02951-6 ］
Fan H M ， Wang R ， Huo Y C and Bao H J . 2021 . Real-time Monte Carlo Denoising with weight sharing kernel prediction network . Computer Graphics Forum ， 40 （ 4 ）： 15 - 27 ［ DOI： 10.1111/cgf.14338 http://dx.doi.org/10.1111/cgf.14338 ］
Firmino A ， Frisvad J R and Jensen H W . 2022 . Progressive Denoising of Monte Carlo rendered images . Computer Graphics Forum ， 41 （ 2 ）： 1 - 11 ［ DOI： 10.1111/cgf.14454 http://dx.doi.org/10.1111/cgf.14454 ］
Froment J . 2014 . Parameter-free fast pixelwise non-local means denoising . Image Processing on Line ， 4 ： 300 - 326 ［ DOI： 10.5201/ipol.2014.120 http://dx.doi.org/10.5201/ipol.2014.120 ］
Han K B ， Odenthal O G ， Kim W J and Yoon S E . 2023 . Pixel-wise guidance for utilizing auxiliary features in Monte Carlo denoising // Proceedings of 2023 ACM on Computer Graphics and Interactive Techniques . New York， USA ： ACM： #11 ［ DOI： 10.1145/3585505 http://dx.doi.org/10.1145/3585505 ］
Harris M ， Sengupta and Owens J D . 2007 . Parallel prefix sum （scan） with CUDA //Nguyen H， ed. GPU Gems 3 . Upper Saddle River， USA ： Addison-Wesley
He K M ， Sun J and Tang X O . 2013 . Guided image filtering . IEEE Transactions on Pattern Analysis and Machine Intelligence ， 35 （ 6 ）： 1397 - 1409 ［ DOI： 10.1109/TPAMI.2012.213 http://dx.doi.org/10.1109/TPAMI.2012.213 ］
Huo Y C and Yoon S E . 2021 . A survey on deep learning-based Monte Carlo Denoising . Computational Visual Media ， 7 （ 2 ）： 169 - 185 ［ DOI： 10.1007/s41095-021-0209-9 http://dx.doi.org/10.1007/s41095-021-0209-9 ］
Işık M ， Mullia K ， Fisher M ， Eisenmann J and Gharbi M . 2021 . Interactive Monte Carlo Denoising using affinity of neural features . ACM Transactions on Graphics ， 40 （ 4 ）： # 37 ［ DOI： 10.1145/3450626.3459793 http://dx.doi.org/10.1145/3450626.3459793 ］
Izadi S ， Sutton D and Hamarneh G . 2023 . Image denoising in the deep learning era . Artificial Intelligence Review ， 56 （ 7 ）： 5929 - 5974 ［ DOI： 10.1007/s10462-022-10305-2 http://dx.doi.org/10.1007/s10462-022-10305-2 ］
Lebreton P and Raake A . 2018 . GBVS360， BMS360， ProSal： extending existing saliency prediction models from 2D to omnidirectional images . Signal Processing： Image Communication ， 69 ： 69 - 78 ［ DOI： 10.1016/j.image.2018.03.006 http://dx.doi.org/10.1016/j.image.2018.03.006 ］
Li X Y ， Wang L H ， Zhou Y C and Zhang J . 2022 . Color image denoising using adaptive non-local 3D total variation . Journal of Image and Graphics ， 27 （ 12 ）： 3450 - 3460
李潇瑶 ， 王炼红 ， 周怡聪 ， 章兢 . 2022 . 自适应非局部3维全变分彩色图像去噪 . 中国图象图形学报 ， 27 （ 12 ）： 3450 - 3460 ［ DOI： 10.11834/jig.210579 http://dx.doi.org/10.11834/jig.210579 ］
Otsu N . 1979 . A threshold selection method from gray-level histograms . IEEE Transactions on Systems， Man， and Cybernetics ， 9 （ 1 ）： 62 - 66 ［ DOI： 10.1109/TSMC.1979.4310076 http://dx.doi.org/10.1109/TSMC.1979.4310076 ］
Perona P and Malik J . 1990 . Scale-Space and edge detection using anisotropic diffusion . IEEE Transactions on Pattern Analysis and Machine Intelligence ， 12 （ 7 ）： 629 - 639 ［ DOI： 10.1109/34.56205 http://dx.doi.org/10.1109/34.56205 ］
Thakur R K and Maji S K . 2023 . Multi scale pixel attention and feature extraction based neural network for image denoising . Pattern Recognition ， 141 ： # 109603 ［ DOI： 10.1016/j.patcog.2023.109603 http://dx.doi.org/10.1016/j.patcog.2023.109603 ］
Thanh D N H ， Hien N N ， Kalavathi P and Prasath V S S . 2020 . Adaptive switching weight mean filter for salt and pepper image denoising . Procedia Computer Science ， 171 ： 292 - 301 ［ DOI： 10.1016/j.procs.2020.04.031 http://dx.doi.org/10.1016/j.procs.2020.04.031 ］
Tian C W ， Zheng M H ， Zuo W M ， Zhang B ， Zhang Y N and Zhang D . 2023 . Multi-stage image denoising with the wavelet transform . Pattern Recognition ， 134 ： # 109050 ［ DOI： 10.1016/j.patcog.2022.109050 http://dx.doi.org/10.1016/j.patcog.2022.109050 ］
Wang Z ， Bovik A C ， Sheikh H R and Simoncelli E P . 2004 . Image quality assessment： from error visibility to structural similarity . IEEE Transactions on Image Processing ， 13 （ 4 ）： 600 - 612 ［ DOI： 10.1109/TIP.2003.819861 http://dx.doi.org/10.1109/TIP.2003.819861 ］
Xing Q W ， Chen C Y and Li Z H . 2021 . Progressive path tracing with bilateral-filtering-based denoising . Multimedia Tools and Applications ， 80 （ 1 ）： 1529 - 1544 ［ DOI： 10.1007/s11042-020-09650-7 http://dx.doi.org/10.1007/s11042-020-09650-7 ］
Xue W F ， Zhang L ， Mou X Q and B C Alan . 2014 . Gradient magnitude similarity deviation： a highly efficient perceptual image quality index . IEEE Transactions on Image Processing ， 23 （ 2 ）： 684 - 695 ［ DOI： 10.1109/TIP.2013.2293423 http://dx.doi.org/10.1109/TIP.2013.2293423 ］
Yang K ， Chen C Y ， Hu X J and Yu H Y . 2022 . Denoising algorithm based on multi-feature non-local mean filtering for Monte Carlo rendered images . Journal of System Simulation ， 34 （ 6 ）： 1259 - 1266
杨凯 ， 陈纯毅 ， 胡小娟 ， 于海洋 . 2022 . 蒙卡渲染画面多特征非局部均值滤波降噪算法 . 系统仿真学报 ， 34 （ 6 ）： 1259 - 1266 ［ DOI： 10.16182/j.issn1004731x.joss.20-1037 http://dx.doi.org/10.16182/j.issn1004731x.joss.20-1037 ］
Zhang G D ， Zhang W L and Duan J H . 2023 . Monte Carlo medical volume rendering denoising via auxiliary feature guided self-attention and convolution integrated // Proceedings of the 16th International Conference on Knowledge Science， Engineering and Management . Guangzhou， China ： Springer： 228 - 236 ［ DOI： 10.1007/978-3-031-40286-9_19 http://dx.doi.org/10.1007/978-3-031-40286-9_19 ］
Zhang Q ， Xiao J Y ， Tian C W ， Lin J W C and Zhang S C . 2023 . A robust deformed convolutional neural network （CNN） for image denoising . CAAI Transactions on Intelligence Technology ， 8 （ 2 ）： 331 - 342 ［ DOI： 10.1049/cit2.12110 http://dx.doi.org/10.1049/cit2.12110 ］
Zhang X B . 2022 . Two-step non-local means method for image denoising . Multidimensional Systems and Signal Processing ， 33 （ 2 ）： 341 - 366 ［ DOI： 10.1007/s11045-021-00802-y http://dx.doi.org/10.1007/s11045-021-00802-y ］
相关作者
相关机构
京公网安备11010802024621
