Current Issue Cover
三维重建场景的纹理优化算法研究综述

于柳, 吴晓群(北京工商大学)

摘 要
三维重建场景的纹理优化是计算机图形学和计算机视觉等领域的基础任务之一,其目的在于优化纹理映射,减小重建几何体和纹理之间的对齐误差,提升重建场景的细节表现。为了对三维重建场景纹理优化算法的现状进行全面研究,本文从传统优化算法和基于深度学习的优化算法两个方面对现有的三维重建场景的纹理优化算法进行综述,传统的纹理优化算法一般通过减小相机姿态累积的误差以及重建几何体的精度对纹理映射质量的影响来达到三维场景纹理优化的目的,按照优化方式的不同,主要包括基于图像融合的优化算法,基于图像拼接的优化算法以及纹理与几何联合优化算法,而基于深度学习的优化算法则利用神经网络优化三维场景纹理。同时,本文汇总了常用的三维重建场景纹理优化的数据集与评价指标,并重点讨论了不同数据集和评价指标的特点和用法。此外,本文对现有的各类纹理优化算法进行了定性分析和定量对比,重点阐述了这些算法的原理及优缺点,最后探讨了三维重建场景的纹理优化面临的挑战和发展方向。
关键词
A Survey of Texture Optimization Algorithms for 3D Reconstructed Scenes

YuLiu, Wu Xiaoqun(Beijing Technology and Business University)

Abstract
Texture optimization is an important task in the field of computer graphics andScomputer vision, and it plays an essential role in creating realistic 3D reconstructed scenes, which aims to enhance the quality of texture mapping for 3D reconstructed scenes. It has a wide range of applications in digital entertainment, heritage restoration, smart cities, virtual/augmented reality and other fields. To achieve completeStexture mapping, a single image is often insufficient, and multiple angle color images are required. By combining these images and projecting them onto theS3D scene, high-quality texture mapping can be achieved, ideally with consistent luminosity across all images. However, the accuracy of 3D reconstruction can be impacted by errors inScamera pose estimationSand geometry, resulting in misaligned projected images. This limitation restricts the use of 3D scenes in various fields, making 3D scene texture optimization a crucial task. TheStexture mapping processSinvolves projecting a 2D image onto a 3D surface to create a realistic representation of the scene. However, this process can be challenging due to the complexity of the 3D data and the inherent inaccuracies in theS3D reconstruction process. Texture optimization algorithms attempt to reduce the errors inStexture mappingSand improve the visual quality of the resulting scene.There are two main types ofStexture optimization algorithms: traditional optimization algorithms and deep learning-basedSoptimization algorithms. Traditional optimization algorithms typically aim to reduce the accumulated error of camera pose and the impact ofSreconstruction geometry accuracySon texture mapping quality. These algorithms often involve techniques such asSimage fusion,Simage stitching, and joint texture and geometry optimization. Image fusion-based optimization algorithms attempt to reduce blurring and artifacts in texture mapping by optimizing the camera pose and addingSdeformation functionsSto the images before weighting theScolor samplesSof multiple texture images from different views using a mixed weight function. This approach can help to improve the visual quality of the scene and reduce the impact ofSreconstruction errorsSon texture mapping. Image stitching-based optimization algorithms select anSoptimal texture imageSfor each triangular slice on the model and then deal with the seams caused by stitching multiple images. This approach can help to reduce the impact of reconstruction errors on texture mapping and improve the visual quality of the resulting scene. Joint texture andSgeometry optimization algorithmsSfurther improve the quality of texture mapping by jointly optimizing the camera pose,Sgeometric vertexSposition, and texture image color. This approach can help to reduce the impact of reconstruction errors on texture mapping and achieve more realistic texture mapping. However, traditional optimization algorithms can be computationally expensive due to the complexity of 3D data. To address this challenge, researchers have developed deep learning-based optimization algorithms, which useSneural networksSto optimize 3D scene textures. These algorithms can be further classified intoSconvolutional neural network-based optimization algorithms, GAN-based optimization algorithms,Sneural textures, and text-driven diffusion model optimization algorithms. Convolutional neural network-based optimization algorithms useSdeep neural networksSto learn the texture features of theSinput imagesSand generate high-quality texture maps forS3D reconstructed scenes. GAN-based optimization algorithms use aSgenerative adversarial networkS(GAN) to generate high-quality texture maps that are visually indistinguishable from real images. Neural textures use aSneural networkSto synthesize new textures that can be applied to 3D reconstructed scenes. Text-driven diffusion model optimization algorithms use a diffusion model to optimize the texture of large missing 3D scenes based on a given text description. Deep learning-based optimization algorithms have shown promising results in improving the visual quality of texture mapping and reducing computational costs. They can effectively combine prior knowledge with neural networks to infer the texture ofSlarge missing 3D scenes, which not only reduces computation but also ensures texture details and greatly enhancesSvisual realism. In addition to texture optimization algorithms, various datasets and evaluation metrics have been developed to evaluate the performance of these algorithms. The commonly used datasets include theSbundlefusion dataset, theSscannet dataset,et al. TheSevaluation metricsSused to evaluate the performance of texture optimization algorithms include peak signal-to-noise ratio (PSNR),Sstructural similarity indexS(SSIM), andSmean squared errorS(MSE). In conclusion, texture optimization of 3D reconstructed scenes is a challenging research topic in the fields of computer graphics and computer vision. Traditional optimization algorithms and deep learning-based optimization algorithms have been developed to address this challenge. The future trend of texture optimization is likely to be deep learning-based optimization algorithms, which can effectively reduce computational costs and enhance visual realism. However, there are still many challenges and opportunities in this field, and researchers need to continue exploring new approaches to improve the quality of texture mapping for 3D reconstructed scenes.
Keywords

订阅号|日报