Wang Hui, Peng Zongju, Jiao Renzhi, Chen Fen, Yu Mei, Jiang Gangyi. Fast 3D-warping algorithm for depth image based rendering[J]. Journal of Image and Graphics, 2016, 21(6): 805-814.DOI: 10.11834/jig.20160613.
Fast 3D-warping algorithm for depth image based rendering
is currently gaining popularity. As its primary advantage
FVV allows audiences to enjoy the video service of an arbitrary visual angle and feel a vivid stereoscopic visual sense as though they are part of the scene they are watching. Depth image based rendering (DIBR)
which can synthesize virtual views with the referenced views and associated depth information that are randomly located on terminal display devices
is a key FVV technology to generate 3D views. However
this technology cannot satisfy real-time needs
and the noise that appears in the synthesized images can lead to visual discomfort because of the imperfect depth maps. Given the poor objective and subjective qualities of the virtual views
this paper proposes a novel 3D-warping algorithm for DIBR to increase the rendering speed as much as possible. This study makes three main contributions. First
to reduce the computation time in 3D-warping
the mapped table for converting depth to parallax is introduced before the rendering. During the rendering process
the programs search the mapped table to obtain the parallax and to avoid repetitive operations. Second
the depth maps of the referenced views are divided into similarly sized square blocks. We adopt block-based 3D-warping
which requires only one instance of warping
if the pixels in the same block have the same depth. Otherwise
we employ the pixel-based 3D-warping
which must conduct warping one by one. The block-based 3D-warping saves much time in decreasing the mapping times of pixels. Third
we propose a corresponding modified interpolation method for block- and pixel-based 3D-warping. The improved interpolation method combines nearest interpolation with splatting interpolation along the horizontal direction. Nearest interpolation is performed if the mapped pixels are near the pixels to be interpolated. Otherwise
splatting interpolation is performed. The proposed interpolation method also betters the Z-Buffer technology in depth direction
which discards the mapped pixels if they are located too far from the cameras and calculates the accumulative average value based on depth value otherwise. Extensive experiments show that the proposed algorithm not only saves 57.81% time on the average compared with the integer-pixel synthesized scheme of VSRS3.5 but also enhances the PSNR and SSIM-two indexes by 0.355 dB and 0.001 15 dB for evaluating the objective quality of the virtual views. The proposed algorithm is highly suitable for DIBR's integer-pixel rendering in a parallel camera system and is especially effective for video sequences that have many flat areas in associated depth maps. This algorithm not only accelerates the 3D-warping process but also improves the objective quality of the virtual views.