浏览全部资源
扫码关注微信
Received：17 August 2017，
Revised：2017-10-25，
Published：16 April 2018
移动端阅览
目的
2
针对高效3维视频编码标准（3D-HEVC）深度视频编码复杂度高和获取不准确的两个问题，现有算法单独进行处理，并没有进行联合优化。为了同时提升深度视频编码速度和编码效率，提出一种联合深度视频增强处理和帧内快速编码的方法。
方法
2
首先，引入深度视频空域增强处理，消除深度视频中的虚假纹理信息，增强其空域相关性，为编码单元（CU）划分和预测模式选择提供进一步优化的空间；然后，针对增强处理过的深度视频的空域特征，利用纹理复杂度将CU进行分类，提前终止平坦CU的分割过程，减少了CU分割次数；最后，利用边缘强度对预测单元（
$${\rm PU}$$
）进行分类，跳过低边缘强度
$$ {\rm PU}$$
的深度模型模式。
结果
2
实验结果表明，与原始3D-HEVC的算法相比，本文算法平均节省62.91%深度视频编码时间，并且在相同虚拟视点质量情况下节省4.63%的码率。与当前代表性的帧内低复杂度编码算法相比，本文算法深度视频编码时间进一步减少26.10%，相同虚拟视点质量情况下，编码码率节省5.20%。
结论
2
该方法通过深度视频增强处理，保证了虚拟视点质量，提升了编码效率。对深度视频帧内编码过程中复杂度较高的CU划分和预测模式选择分别进行优化，减少了率失真代价计算次数，有效地降低了帧内编码复杂度。
Objective
2
With the development of 3D content acquisition and display technologies in recent years
three-dimensional (3D) video has received increasing attention. The multi-view video plus depth format is the main representation of a 3D scene. In the 3D extension of high-efficiency video coding (3D-HEVC)
the main framework for depth video is similar to that of HEVC. Each coding unit (CU) is recursively divided into four sub-CUs. Each CU depth level enables 37 types of intra modes in intra frames. Unlike conventional texture videos
depth videos are not used for watching
but for virtual view rendering. The preservation of depth sharp object edges is important for depth video compression. Several new techniques
such as depth modeling mode (DMM) and view synthesis optimization
are introduced into the current 3D-HEVC test model to improve the efficiency of depth video intra coding and the quality of synthesized views. These techniques improve the coding efficiency of depth videos. However
they greatly increase the computational complexity of depth intra coding
thereby hindering the real-time applications of 3D-HEVC. Depth videos are also inaccurate and inconsistent because of the limitations of mainstream capture technologies. The inaccuracy of depth videos further increases the computational complexity of intra coding. Previous research on low-complexity depth video intra coding and depth video enhancement has been conducted separately. Thus
a joint depth video enhancement and fast intra-coding algorithm is proposed in this study.
Method
2
An enhancement method is applied before encoding to remove inaccurate textures in a depth video and enhance the spatial correlation of the depth video. The edge region is preserved for rendering performance. For non-edge regions
Gaussian and adaptive window smoothing filters are used. The enhanced depth video is mainly characterized by sharp object edges and large areas of nearly constant regions. We can skip some prediction modes and CU depth levels rarely used in homogeneity regions by fully exploiting such features. CUs are classified according to texture complexity
and the partition process of CUs with low texture complexity is terminated early. Prediction units (PUs) are classified according to edge intensity. The proposed algorithm selectively omits unnecessary DMM in the mode decision process on the basis of the
$${\rm PU}$$
classification results. The algorithm is implemented on the reference software
HTM-16.0
of the 3D-HEVC standard and tested under common test conditions required by the Joint Collaborative Team on 3D Video Coding to evaluate its performance. The proposed algorithm is specially designed for depth videos that are estimated using stereo matching; thus
sequences synthesized by a computer are not tested. The proposed scheme aims at depth video intra coding
and all test sequences are coded with intra-only structure and three-view configuration. The rate distortion performance of the proposed algorithm is evaluated by using the Bjontegaard delta bitrate
which is calculated by the peak signal-to-noise ratio of the synthesis view quality and the total bitrate
including color and depth videos.
Result
2
Experimental results show that the proposed algorithm significantly saves the encoding time of the depth video and reduces bitrate under the same synthesized virtual view quality. The coding time reduction obtained by the proposed algorithm compared with that of the original 3D-HEVC encoder ranges from 61.35% to 65.73% and is 62.91% on average. In terms of coding efficiency
our proposed algorithm can reduce bitrate by 4.63% under the condition of the same synthesized virtual view quality
in which the maximal and minimal reductions are 8.10% and 2.60%
respectively. The subjective quality of the proposed algorithm is significantly improved compared with that of the original 3D-HEVC encoder. The significant performance improvement of depth video coding contributes to the depth video enhancement and the fast algorithm. The proposed algorithm is superior to the state-of-the-art fast depth intra-coding algorithm. The encoding time saving of depth video is greatly increased by 26.10%
and the bitrate is further reduced by 5.20% under the condition of the same synthesized virtual view quality.
Conclusion
2
A joint depth video enhancement and fast intra-coding algorithm is proposed to solve the problems in 3D-HEVC
the high computational complexity of depth video intra coding
and the inaccuracy of depth videos. The proposed enhancement method improves the spatial correlation of depth videos. The fast intra-coding scheme can significantly reduce the encoding time of depth videos. Therefore
the proposed method not only reduces encoding time but also improves the compression performance of depth video intra coding.
Chen Y, Vetro A. Next-generation 3D formats with depth map support[J]. IEEE Multimedia, 2014, 21(2):90-94.[DOI:10.1109/MMUL.2014.31]
Huszák Á. Advancedfree viewpoint video streaming techniques[J]. Multimedia Tools and Applications, 2017, 76(1):373-396.[DOI:10.1007/s11042-015-3048-9]
Fehn C. Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV[C]//Proceedings of the SPIE 5291, Stereoscopic Displays and Virtual Reality Systems XI. San Jose, California, United States: SPIE, 2004, 5291: 93-104. [ DOI:10.1117/12.524762 http://dx.doi.org/10.1117/12.524762 ]
Zhang S, Wang C, Chan S C. A new high resolution depth map estimation system using stereo vision and kinect depth sensing[J]. Journal of Signal Processing Systems, 2015, 79(1):19-31.[DOI10.1007/s11265-013-0821-8]
Honnungar S, Holloway J, Pediredla A K, et al. Focal-sweep for large aperture time-of-flight cameras[C]//Proceedings of 2016 IEEE International Conference on Image Processing (IPIC). Phoenix: IEEE, 2016:953-957. [ DOI:10.1109/ICIP.2016.7532498 http://dx.doi.org/10.1109/ICIP.2016.7532498 ]
Peng Z J, Han H M, Chen F, et al. Joint processing and fast encoding algorithm for multi-view depth video[J]. EURASIP Journal on Image and Video Processing, 2016, 2016:#24.[DOI:10.1186/s13640-016-0128-3]
Sullivan G J, Ohm J, Han W J, et al. Overview of the high efficiency video coding (HEVC) standard[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2012, 22(12):1649-1668.[DOI:10.1109/TCSVT.2012.2221191]
Tech G, Chen Y, Müller K, et al. Overview of the multiview and 3D extensions of high efficiency video coding[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2016, 26(1):35-49.[DOI:10.1109/TCSVT.2015.2477935]
Zhang H B, Fu C H, Su W M, et al. Fast coding unit decision algorithm for depth intra coding in 3D-HEVC[J]. Journal of Electronics&Information Technology, 2016, 38(10):2523-2530.
张洪彬, 伏长虹, 苏卫民, 等. 3D-HEVC深度图像帧内编码单元划分快速算法[J].电子与信息学报, 2016, 38(10):2523-2530. [DOI:10.11999/JEIT151426]
Zhang H B, Fu C H, Chan Y L, et al. Probability-based depth intra mode skipping strategy and novel VSO metric for DMM decision in 3D-HEVC[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2016, 28(2):513-517.[DOI:10.1109/TCSVT.2016.2612693]
Lei J J, Duan J H, Wu F, et al. Fast mode decision based on grayscale similarity and inter-view correlation for depth map coding in 3D-HEVC[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2016, 99:1.[DOI:10.1109/TCSVT.2016.2617332]
Park C S. Edge-based intramode selection for depth-map coding in 3D-HEVC[J]. IEEE Transactions on Image Processing, 2015, 24(1):155-162.[DOI:10.1109/TIP.2014.2375653]
Zheng H J, Zhu J Q, Zeng H Q, et al. Low complexity depth intra coding in 3D-HEVC based on depth classification[C]//Proceedings of 2016 Visual Communications and Image Processing. Chengdu: IEEE, 2016: 1-4. [ DOI:10.1109/VCIP.2016.7805527 http://dx.doi.org/10.1109/VCIP.2016.7805527 ]
Peng Z J, Guo M S, Chen F, et al. A depth video processing algorithm based on cluster dependent and corner-ware filtering[J]. Neurocomputing, 2016, 215:90-99.[DOI:10.1016/J.NEUCOM.2015.07.154]
Zhang Y, Zhu L W, Liu X K, et al. Allowable depth distortion based depth filtering for 3D high efficiency video coding[C]//2016 IEEE International Symposium on Circuits and Systems (ISCAS). Montreal: IEEE, 2016: 2559-2562. [ DOI:10.1109/ISCAS.2016.7539115 http://dx.doi.org/10.1109/ISCAS.2016.7539115 ]
Yoon S M, Yoon J. Depth map enhancement using adaptive moving least squares method with a total variation minimization[J]. Multimedia Tools and Applications, 2016, 75(23):15929-15938.[DOI:10.1007/S11042-015-2905-X]
Boseen F. 3D-HEVC software HTM16. 0[Online]. [2017-08-17] http://hevc.hhi.fraunhofer.de/svn/svn3DVCSoft/tags/HTM16.0 http://hevc.hhi.fraunhofer.de/svn/svn3DVCSoft/tags/HTM16.0 , 2015.
Peng Z J, Zhou H, Jiang G Y, et al. Depth video preprocessing algorithm based on adaptive window[J]. Journal of Optoelectronics·Laser, 2013, 24(4):769-776.
彭宗举, 周浩, 蒋刚毅, 等.基于自适应窗口的深度视频预处理算法[J].光电子·激光, 2013, 24(4):769-776.]
Huynh-The T, Banos O, Lee S, et al. NIC:a robust background extraction algorithm for foreground detection in dynamic scenes[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2017, 27(7):1478-1490.[DOI:10.1109/TCSVT.2016.2543118]
Bjøntegaard G. Calculation of average PSNR differences between RD-curves. VCEG-M33[R]. Austin:Video Coding Experts Group, 2001.
相关作者
相关机构
京公网安备11010802024621
