A Survey of Texture Optimization Algorithms for 3D Reconstructed Scenes
YuLiu, Wu Xiaoqun(Beijing Technology and Business University)
Texture optimization is an important task in the field of computer graphics andScomputer vision, and it plays an essential role in creating realistic 3D reconstructed scenes, which aims to enhance the quality of texture mapping for 3D reconstructed scenes. It has a wide range of applications in digital entertainment, heritage restoration, smart cities, virtual/augmented reality and other fields. To achieve completeStexture mapping, a single image is often insufficient, and multiple angle color images are required. By combining these images and projecting them onto theS3D scene, high-quality texture mapping can be achieved, ideally with consistent luminosity across all images. However, the accuracy of 3D reconstruction can be impacted by errors inScamera pose estimationSand geometry, resulting in misaligned projected images. This limitation restricts the use of 3D scenes in various fields, making 3D scene texture optimization a crucial task. TheStexture mapping processSinvolves projecting a 2D image onto a 3D surface to create a realistic representation of the scene. However, this process can be challenging due to the complexity of the 3D data and the inherent inaccuracies in theS3D reconstruction process. Texture optimization algorithms attempt to reduce the errors inStexture mappingSand improve the visual quality of the resulting scene.There are two main types ofStexture optimization algorithms: traditional optimization algorithms and deep learning-basedSoptimization algorithms. Traditional optimization algorithms typically aim to reduce the accumulated error of camera pose and the impact ofSreconstruction geometry accuracySon texture mapping quality. These algorithms often involve techniques such asSimage fusion,Simage stitching, and joint texture and geometry optimization. Image fusion-based optimization algorithms attempt to reduce blurring and artifacts in texture mapping by optimizing the camera pose and addingSdeformation functionsSto the images before weighting theScolor samplesSof multiple texture images from different views using a mixed weight function. This approach can help to improve the visual quality of the scene and reduce the impact ofSreconstruction errorsSon texture mapping. Image stitching-based optimization algorithms select anSoptimal texture imageSfor each triangular slice on the model and then deal with the seams caused by stitching multiple images. This approach can help to reduce the impact of reconstruction errors on texture mapping and improve the visual quality of the resulting scene. Joint texture andSgeometry optimization algorithmsSfurther improve the quality of texture mapping by jointly optimizing the camera pose,Sgeometric vertexSposition, and texture image color. This approach can help to reduce the impact of reconstruction errors on texture mapping and achieve more realistic texture mapping. However, traditional optimization algorithms can be computationally expensive due to the complexity of 3D data. To address this challenge, researchers have developed deep learning-based optimization algorithms, which useSneural networksSto optimize 3D scene textures. These algorithms can be further classified intoSconvolutional neural network-based optimization algorithms, GAN-based optimization algorithms,Sneural textures, and text-driven diffusion model optimization algorithms. Convolutional neural network-based optimization algorithms useSdeep neural networksSto learn the texture features of theSinput imagesSand generate high-quality texture maps forS3D reconstructed scenes. GAN-based optimization algorithms use aSgenerative adversarial networkS(GAN) to generate high-quality texture maps that are visually indistinguishable from real images. Neural textures use aSneural networkSto synthesize new textures that can be applied to 3D reconstructed scenes. Text-driven diffusion model optimization algorithms use a diffusion model to optimize the texture of large missing 3D scenes based on a given text description. Deep learning-based optimization algorithms have shown promising results in improving the visual quality of texture mapping and reducing computational costs. They can effectively combine prior knowledge with neural networks to infer the texture ofSlarge missing 3D scenes, which not only reduces computation but also ensures texture details and greatly enhancesSvisual realism. In addition to texture optimization algorithms, various datasets and evaluation metrics have been developed to evaluate the performance of these algorithms. The commonly used datasets include theSbundlefusion dataset, theSscannet dataset,et al. TheSevaluation metricsSused to evaluate the performance of texture optimization algorithms include peak signal-to-noise ratio (PSNR),Sstructural similarity indexS(SSIM), andSmean squared errorS(MSE). In conclusion, texture optimization of 3D reconstructed scenes is a challenging research topic in the fields of computer graphics and computer vision. Traditional optimization algorithms and deep learning-based optimization algorithms have been developed to address this challenge. The future trend of texture optimization is likely to be deep learning-based optimization algorithms, which can effectively reduce computational costs and enhance visual realism. However, there are still many challenges and opportunities in this field, and researchers need to continue exploring new approaches to improve the quality of texture mapping for 3D reconstructed scenes.