视觉显著性驱动的全景渲染图非局部降噪
韩鲁光, 陈纯毅, 申忠业, 胡小娟, 于海洋(长春理工大学) 摘 要
目的 传统降噪方法通常忽视人眼感知因素,对不同区域的图像块都进行同等处理。当使用传统降噪算法对全景画面滤波处理时,全景画面两极区域容易产生模糊问题,尤其是通过视口观察时,该问题更加明显。针对上述问题,本文提出一种视觉显著性驱动的蒙特卡洛渲染生成全景图非局部均值(visual saliency driven non-local means,VSD-NLM)滤波降噪算法。方法 在VSD-NLM算法中首先使用全景图显著区域检测算法获取全景画面的显著区域;然后使用梯度幅值相似性偏差辅助的非局部均值(gradient magnitude similarity deviation assisted non-local means,GMSDA-NLM)滤波算法,降低显著区域的噪声;同时设计并行非局部均值(parallel non-local means,P-NLM)滤波算法,加快降噪处理速度,降低非显著区域噪声;最后利用改进的Canny算法提取梯度特征,同时结合各向异性扩散引导滤波来优化降噪结果。结果 采用结构相似度(structural similarity,SSIM)和FLIP作为评价指标,来对比VSD-NLM算法与非局部均值滤波算法、多特征非局部均值滤波算法以及渐进式去噪算法等其他算法的性能。实验结果表明,VSD-NLM算法的降噪结果在客观评价指标上均优于对比算法,SSIM值比其他算法平均提高14.7%,FLIP值比其他算法平均降低15.2%。在视觉效果方面,VSD-NLM算法能够减轻全景画面模糊,提升视觉感知质量。本文对GMSDA-NLM和P-NLM算法的有效性进行了实验验证,相较于非局部均值滤波算法,GMSDA-NLM算法能够有效去除噪声并保持图像细节的完整性。P-NLM算法在运行速度方面平均提高约6倍,与串行算法生成的图像之间的SSIM值可达到0.996。结论 相比于其他算法,本文算法能够更好地用于全景图降噪,滤波效果更佳,对全景电影制作应用有重要的理论和实际意义。
关键词
Visual-Saliency-Driven Non-Local Denoising of Rendered Panoramic Images
hanluguang, chenchunyi, shenzhongye, huxiaojuan, yuhaiyang(Changchun University of Science and Technology) Abstract
Objective Panoramic movie technology has experienced notable advancements in order to enrich the audio-visual experience for viewers, resulting in a heightened sense of immersion within the visual environment. Nevertheless, the production of high-quality images poses a challenge for conventional rasterization techniques, necessitating the exploration of alternative approaches. Monte Carlo path tracing algorithms have proven to be effective in generating high-quality images, offering exceptional visual fidelity in various rendering applications. However, the computational overhead associated with this algorithm remains a significant challenge. To optimize computation, reducing the number of pixels sampled in Monte Carlo path tracing is a common approach. Unfortunately, this reduction often leads to the introduction of noticeable noise in the resulting images, compromising their overall quality. This paper aims to address the issue of image noise in Monte Carlo path tracing by exploring and proposing advanced techniques for denoising. In the domain of Monte Carlo rendering, two main approaches to denoising are commonly used. The first approach utilizes traditional filtering methods with artificially designed filters to remove image noise. While this approach is versatile, its effectiveness in noise removal may be limited, often resulting in residual noise. The second approach involves deep learning-based denoising methods, which can effectively eliminate noise but may exhibit performance limitations on specific image types. Currently, most existing image denoising algorithms are developed and studied for ordinary flat images, with limited research dedicated to denoising algorithms specifically designed for panoramic images. Panoramic images possess unique characteristics, including a 360° field of view in the horizontal direction, a 180° field of view in the vertical direction, distorted edges, and varying prominence of equatorial and polar pixels as perceived by human observers. Conventional flat image denoising methods often fail to fully account for these panoramic image characteristics, leading to excessive blurring or residual noise in the equatorial, polar, and distorted edge regions after the denoising process. Therefore, this paper proposes a visual saliency driven non-local mean filtering (VSD-NLM) denoising algorithm explicitly tailored for Monte Carlo rendering of panoramic images. The algorithm aims to leverage the distinctive characteristics of panoramic images, such as the 360° field of view, distorted edges, and varying pixel prominence, to effectively reduce noise while preserving the essential features of panoramic images. Through comprehensive experimentation and evaluation, the proposed algorithm demonstrates its efficacy in enhancing the image quality of Monte Carlo-rendered panoramic images, providing a valuable contribution to the field of panoramic image denoising. Method This paper presents the design and optimization of a visual saliency-driven non-local mean filtering algorithm for denoising Monte Carlo rendered panoramic images. The proposed algorithm comprises two key components aimed at effectively removing noise and enhancing image quality in panoramic scenes. The first component focuses on enhancing the non-local mean filtering process specifically tailored for panoramic images. Initially, a panoramic image saliency detection model is utilized to generate a saliency image, incorporating an equatorial bias to improve saliency accuracy. Subsequently, the saliency image is employed to delineate saliency and non-saliency regions within the panoramic image. In the saliency region, the deviation value of the gradient magnitude similarity between image blocks is calculated to refine the weights used in non-local mean filtering. For the non-saliency region, parallel algorithms for non-local mean filtering are devised to accelerate the filter reconstruction process. Finally, denoising results from the saliency and non-saliency regions are combined to produce the final denoised panoramic image. The second component of the algorithm focuses on optimized noise reduction, specifically addressing the distorted edge regions of the panoramic image. Improvements are made to the Canny algorithm to obtain a more accurate edge gradient image. This involves optimizing the weights for the 45° and 135° directions of the image, generating adaptive high and low thresholds using an enhanced Otsu method, and optimizing the local thresholds to improve the performance of the Canny operator. Subsequently, anisotropic diffusion filtering is combined with guided filtering, utilizing the gradient image as a guide, to filter and enhance the combined images. The proposed algorithm"s optimizations collectively contribute to effective noise reduction in the distorted edge regions of panoramic images, resulting in enhanced image quality and reduced noise artifacts. Result This paper presents a comprehensive performance evaluation of the proposed denoising algorithm for panoramic images, utilizing structural similarity (SSIM) and FLIP metrics as objective evaluation indicators. The performance of VSD-NLM algorithm is compared with other algorithms such as non-local means filtering algorithm, multi-feature non-local means filtering algorithm and progressive denoising algorithm, aiming to assess its effectiveness in reducing noise and improving the visual quality of panoramic images. Experimental results reveal that the proposed algorithm outperforms the comparison algorithms in terms of objective evaluation indicators. The average FLIP value achieved by the proposed algorithm is 15.2% lower compared to other algorithms. Similarly, the average SSIM value attained by the proposed algorithm is 14.7% higher than other algorithms, signifying its enhanced structural similarity preservation. Furthermore, the algorithm"s visual effects are assessed, demonstrating its ability to mitigate blurring artifacts in panoramic images and enhance visual perception quality. This article presents an experimental verification of the effectiveness of two denoising algorithms: gradient magnitude similarity deviation assisted non-local means (GMSDA-NLM) and parallel non-local means (P-NLM). The GMSDA-NLM algorithm combines the strengths of non-local mean filtering and gradient magnitude similarity deviation to achieve superior noise reduction capabilities while maintaining the integrity of image details. the algorithm effectively identifies and suppresses noise while preserving the essential image features. the P-NLM algorithm exhibits a notable average speed increase of approximately 6 times compared to the non-parallel algorithm, facilitating real-time or near-real-time noise reduction applications. The SSIM value between P-NLM and the image generated by the non-parallel algorithm can reach 0.996. Conclusion This paper introduces a specialized denoising algorithm tailored for panoramic images, specifically addressing the unique challenges associated with denoising in this domain. From a practical perspective, our algorithm holds substantial value for panoramic film production. By significantly reducing noise in panoramic images, our algorithm enhances the visual quality and fidelity of panoramic films. The remarkable results obtained through our algorithm contribute to immersive visual storytelling, elevating the overall cinematic experience and capturing the attention of audiences. In conclusion, The exceptional results achieved through our algorithm present valuable theoretical advancements and have practical implications for panoramic film production, enhancing the quality and impact of visual narratives in the realm of immersive cinematography.
Keywords
Panoramic Images Non Local Mean Filter Gradient Magnitude Similarity Deviation Guided Filtering Image Denoising
|