树木结构层次细化的立体树木图像构建
Construction of 3D tree image with hierarchical tree structures
- 2018年23卷第9期 页码:1382-1392
收稿:2018-01-30,
修回:2018-4-3,
纸质出版:2018-09-16
DOI: 10.11834/jig.180035
移动端阅览

浏览全部资源
扫码关注微信
收稿:2018-01-30,
修回:2018-4-3,
纸质出版:2018-09-16
移动端阅览
目的
2
2维转3维技术可以将现有的丰富2维图像资源快速有效地转为立体图像,但是现有方法只能对树木的整体进行深度估计,所生成的图像无法表现出树木的立体结构。为此,提出一种树木结构层次细化的立体树木图像构建方法。
方法
2
首先利用Lab颜色模型下的像素色差区别将2维树木图像的树干区域和树冠区域分割开来,并对树冠区域进行再分割;然后,在深度梯度假设思想基础上建立多种类型的深度模板,结合深度模板和树冠的区域信息为典型树木对象构建初始深度图,并通过基础深度梯度图组合的方式为非典型树木进行个性化深度构建;最后,根据应用场景对树木深度信息进行自适应调整与优化,将树木图像合成到背景图像中,并构建立体图像。
结果
2
对5组不同的树木图像及背景图像进行了立体树木图像的构建与合成。结果表明,不同形态的树木图像都能生成具有层次感的深度图并自适应地合成到立体背景图像中,构建树木图像深度图的时间与原始树木图像的尺寸成正比,而构建立体树木图像并合成到背景中所需时间在2~4 s之间。对立体图像质量的主观评价测试中,这些图像的评分均达到良好级别以上,部分立体图像达到了优秀级别。
结论
2
该方法充分利用了树木的形态结构特征,能同时适用于典型和非典型树木,所构建的立体树木图像质量较高,具有丰富的层次感,并具有舒适的立体观看效果。
Objective
2
2D to 3D technology can quickly and effectively transfer the existing rich resource of 2D images into 3D images. However
existing methods only consider depth estimation on the entire object of a tree in the process of depth map generation. In this way
the tree in the final 3D images generated would normally lack a sense of depth and look like a piece of paper stuck on the background. The depth map generated by these methods cannot show the natural three-dimensional structural features of trees and has no sufficient rich stereoscopic levels to show the layering on the trees. To this end
this paper presents a 3D tree image construction method on the basis of depth template.
Method
2
In our proposed method
we first utilize the color difference of pixels under the laboratory color model to divide the 2D tree image into trunk and canopy areas and then divide the canopy area again into several small areas by using multiscale spectral-based image segmentation method. Then
considering the assumption of depth gradient hypothesis
we create various types of basic depth templates. In addition to the six commonly used basic depth templates
two new basic depth templates are added on the basis of the structural features of the trees. In accordance with the complexity of the morphology of trees
this paper divides the tree objects into two major categories:typical and atypical trees. The typical trees are those with regular shapes and can basically belong to the four typical tree models of spherical
conical
cylindrical
and wide spread shapes. The atypical trees are inconsistent with the basic characteristics of the existing four tree models. The initial depth maps of typical tree objects are constructed by combining the basic depth template and the area information of canopy. For the atypical tree object
we first select some basic depth templates to generate a personal depth temple
and then a personalized initial depth map is constructed through the combination of the personal depth temple and the tree's canopy area information. Finally
the tree depth information is adjusted and optimized adaptively in accordance with the application scene. 3D images of trees are adaptively adjusted in accordance with the depth information of the corresponding position of the background to obtain a depth map consistent with the background depth information. In addition
different objects in the scene are located at different depths
so when the tree images are synthesized into the background image
the occlusion between the objects in the scene needs to be adjusted. After the abovementioned process
the tree image is synthesized into the background image and the 3D image is constructed.
Result
2
To verify the effectiveness and practicability of our method
we used different background and tree image materials to create 3D image pairs. We show the five generated 3D images from the experiment in this paper. The experimental results show that all the tree images of different sizes
namely
typical and atypical
can produce layered depth images and can be adaptively synthesized into different 3D background images. The operating efficiency of the system is stable
and the time required to construct the stereoscopic image is linearly related to the size of the original tree image
which means that the run-time growth would not be explosively increased due to the increase in tree image size. In the subjective evaluation test on the stereoscopic image quality
we conducted tests and statistics on three aspects:3D image pair quality
depth map quality
and 3D comfort situation. Five different ratings are available for each aspect
covering all aspects related to the quality of the 3D image. In accordance with the statistics of each item
we obtain the total score under the percentile and then divide the total score into five grades:excellent
good
medium
normal
and poor. In the test
the ratings for these images all reach good levels
and some of them even achieve excellent levels.
Conclusion
2
To naturally blend 3D trees and 3D background images and enhance the 3D display of trees
by using the morphological characteristics of trees
this paper presents a 3D tree image construction method on the basis of depth template. The method takes full advantage of the morphological characteristics of trees and can be applied to both typical and atypical trees. The constructed 3D trees have a high image quality and comfortable stereoscopic effect. The depth template used in this paper presents a greater improvement in the depth sense of trees in 3D images than the existing methods.
Cheng C C, Li C T, Chen L G. A novel 2Dd-to-3D conversion system using edge information[J]. IEEE Transactions on Consumer Electronics, 2010, 56(3):1739-1745.[DOI:10.1109/TCE.2010.5606320]
Wang P, An P, Wang K, et al. Depth extraction of single-view image using region merging[J]. Video Engineering, 2011, 35(19):11-13.
王平, 安平, 王奎, 等.基于区域融合的单视点图像深度信息提取[J].电视技术, 2011, 35(19):11-13. [DOI:10.3969/j.issn.1002-8692.2011.19.004]
Lim H, Park H W. Depth map rasterization using triangulation and color consistency for various sampling structures[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2015, 25(7):1081-1098.[DOI:10.1109/TCSVT.2014.2376313]
Saxena A, Sun M, Ng A Y. Make3D:learning 3D scene structure from a single stillimage[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(5):824-840.[DOI:10.1109/TPAMI.2008.132]
Jung J I, Ho Y S. Depth map estimation from single-view image using object classification based on Bayesian learning[C]//Proceedings of 20103DTV-Conference: the True Vision-Capture, Transmission and Display of 3D Video. Tampere, Finland: IEEE, 2010: 1-4. [ DOI:10.1109/3DTV.2010.5506603 http://dx.doi.org/10.1109/3DTV.2010.5506603 ]
Fang S, Jin R, Cao Y. Fast depth estimation from single image using structured forest[C]//Proceedings of 2016 IEEE International Conference on Image Processing. Phoenix, AZ, USA: IEEE, 2016: 4022-4026. [ DOI:10.1109/ICIP.2016.7533115 http://dx.doi.org/10.1109/ICIP.2016.7533115 ]
Wang M, Zhang F L, Hu S M. Data-driven image analysis and editing:a survey[J]. Journal of Computer-Aided Design&Computer Graphics, 2015, 27(11):2015-2024.
汪淼, 张方略, 胡事民.数据驱动的图像智能分析和处理综述[J].计算机辅助设计与图形学学报, 2015, 27(11):2015-2024.
Konrad J, Brown G, Wang M, et al. Automatic 2D-to-3D image conversion using 3D examples from the internet[C]//Proceedings of SPIE Stereoscopic Displays and Applications XXⅢ. Burlingame, California, USA: SPIE, 2012: 12.
Yuan H X, Wu S Q, Yu H Q, et al. 2D-to-3D method via semantic depth transfer[J]. Journal of Computer-Aided Design&Computer Graphics, 2014, 26(1):72-80.
袁红星, 吴少群, 余辉晴, 等.语义级深度迁移的2D转3D算法[J].计算机辅助设计与图形学学报, 2014, 26(1):72-80.
Cao Y, Kang K, Zhang S J, et al. Automatic tag saliency ranking for stereo images[J]. Neurocomputing, 2016, 172:9-18.[DOI:10.1016/j.neucom.2014.09.097]
Kanungo T, Mount D M, Netanyahu N S, et al. An efficient k-means clustering algorithm:analysis and implementation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7):881-892.[DOI:10.1109/TPAMI.2002.1017616]
Soille P. Morphological Image Analysis: Principles and Applications[M]. Berlin, Heidelberg: Springer Science&Business Media, 2013.
Shi J B, Malik J. Normalized cuts and image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(8):888-905.[DOI:10.1109/34.868688]
Kim J, Li H, Yue J J, et al. Shape matching using a Self similar affine invariant descriptor[C]//Proceedings of 2016 IEEE International Conference on Image Processing. Phoenix, AZ, USA: IEEE, 2016: 2470-2474. [ DOI:10.1109/ICIP.2016.7532803 http://dx.doi.org/10.1109/ICIP.2016.7532803 ]
Liu F, Deng Z R. Denoising algorithm based on adaptive wavelet threshold and bilateral filter[J]. Journal of System Simulation, 2014, 26(12):2935-2938.
刘芳, 邓志仁.基于自适应小波阈值和双边滤波器的去噪算法[J].系统仿真学报, 2014, 26(12):2935-2938. [DOI:10.16182/j.cnki.joss.2014.12.019]
Smith B M, Zhang L, Jin H L. Stereo matching with nonparametric smoothness priors in feature space[C]//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, FL, USA: IEEE, 2009: 485-492. [ DOI:10.1109/CVPR.2009.5206793 http://dx.doi.org/10.1109/CVPR.2009.5206793 ]
Fácil J M, Concha A, Montesano L, et al. Single-view and multi-view depth fusion[J]. IEEE Robotics and Automation Letters, 2017, 2(4):1994-2001.[DOI:10.1109/LRA.2017.2715400]
Zhang Y J, Li S M, Wei J J, et al. Subjective quality evaluation method of stereo image[J]. Acta Photonica Sinica, 2012, 41(5):602-607.
张英静, 李素梅, 卫津津, 等.立体图像质量的主观评价方案[J].光子学报, 2012, 41(5):602-607.[DOI:10.3788/gzxb20124105.0602]
相关作者
相关机构
京公网安备11010802024621