目的 深度图像作为一种普遍的3维场景信息表达方式在立体视觉领域有着广泛的应用。Kinect深度相机能够实时获取场景的深度图像，但由于内部硬件的限制和外界因素的干扰，获取的深度图像存在分辨率低、边缘不准确的问题，无法满足实际应用的需要。为此提出了一种基于彩色图像边缘引导的Kinect深度图像超分辨率重建算法。方法 首先对深度图像进行初始化上采样，并提取初始化深度图像的边缘；进一步利用高分辨率彩色图像和深度图像的相似性，采用基于结构化学习的边缘检测方法提取深度图的正确边缘；最后找出初始化深度图的错误边缘和深度图正确边缘之间的不可靠区域，采用边缘对齐的策略对不可靠区域进行插值填充。结果 在NYU2数据集上进行实验，与8种最新的深度图像超分辨率重建算法作比较，用重建之后的深度图像和3维重建的点云效果进行验证。实验结果表明本文算法在提高深度图像的分辨率的同时，能有效修正上采样后深度图像的边缘，使深度边缘与纹理边缘对齐，也能抑制上采样算法带来的边缘模糊现象；3维点云效果显示，本文算法能准确区分场景中的前景和背景，应用于3维重建等应用能取得较其他算法更好的效果。结论 本文算法普遍适用于Kinect深度图像的超分辨率重建问题，该算法结合同场景彩色图像与深度图像的相似性，利用纹理边缘引导深度图像的超分辨率重建，可以得到较好的重建结果。

Objective Depth map plays an increasingly important role in many computer vision applications,such as 3D reconstruction,augmented reality,and gesture recognition.A new generation of active 3D range sensors,such as Microsoft Kinect camera,enables the acquisition of a real-time and affordable depth map.Incidentally,unlike natural images captured by RGB sensors,the depth maps captured by range sensors typically have low resolution (LR) and inaccurate edges due to their intrinsic physical constraints.Given that an accurate and high-resolution (HR) depth map is required and preferable in many applications,excellent depth map super-resolution (SR) techniques are desirable.Depth map SR can be generally addressed by two different types of approaches that depend on the use of input data.For single depth map SR,the resolution of the input depth map can be enhanced based on the information learned with from a pre-collected training database.Meanwhile,depth map SR algorithms that use RGB-D data can be further classified into MRF and filtering-based approaches.MRF-based methods view depth map SR as an optimization problem.Filtering-based methods obtain the weighted average of local depth map pixels for SR purposes.These methods aim to obtain a smooth HR depth map for regions belonging to the same object.However,these methods have two main issues:1) the inaccurate edges of the depth map cannot be fully refined and 2) the edges of the HR depth map suffer from blurring.In this paper,a novel texture edge-guided depth reconstruction approach is proposed to address the issue of existing methods.We pay more attention to the depth edge refinement,which is usually ignored by existing methods.Method In the first stage,an initial HR depth map is obtained by general up-sampling methods,such as interpolation and filters.Then,initial depth edges are extracted from the initial HR depth map by using many edge detectors for edge detection,such as Sobel and Canny.The edges extracted directly from the initial HR depth map are not the true edges because the misalignment between the LR depth map edges and the texture edges and the up-sampling operation can cause further edge errors.Subsequently,the texture edges are extracted from the color image.Traditional approaches for edge detection do not consider the visually salient edges;the texture edges and illusory contours are all taken as image edges.Moreover,many edges of the color image do not correspond to depth edges,such as the edges inside the object.Inspired by the advanced positive result of the vision field,we propose a depth map edge detection method based on the structured forest.The edge map of the color image is initially extracted by using the recently structured learning approach.By incorporating the 3D space information provided by the initial HR depth map,the texture edges of the objects inside are removed.Then,we obtain a clear and true depth edge map.Finally,the depth values on each side of the depth edge are refined to align the depth edges and correct the depth errors in the initial HR depth map.We detect the incorrect depth regions between the initial depth edges and the corresponding true depth edges and then fill the incorrect regions until the depth edges are consistent with the corresponding color image.The incorrect regions of initial HR depth map are refined by the joint bilateral filter in an outside-inward refining order that is regularized by the detected true depth edges.Result We perform experiments on the NYU dataset,which offers real-world color-depth image pairs that were captured by a Kinect camera.To evaluate the performance of our proposed method,we compare our results with two method categories:1) state-of-the-art single depth image super resolution methods (ScSR,PB,and E.G.) and 2) state-of-the-art color-guided depth map super resolution approaches (JBU,GIU,MRF,WMF,and JTU).We implement most of these methods by using the same parameter settings as provided in the corresponding papers.We down-sample the original depth maps into LR ones and perform SR.We evaluate our proposed method with the recovered HR depth map and the reconstructed point clouds.The recovered HR depth maps indicate that our proposed methods generate more visually appealing results than the compared approaches.The boundaries in our results are generally sharper and smoother along the edge direction,whereas the compared methods suffer from blurred artifacts around the boundaries.To demonstrate further the effectiveness of our proposed approach,we provide the 3D point clouds constructed from the up-scaled depth map with different methods.Results indicate that our proposed method yields a relatively clear foreground and background,while the competing results suffer from obvious flying pixels and aliased planes.Conclusion We present a novel method for depth map SR for Kinect depth.Experimental results demonstrate that the proposed method provides sharp and clear edges for the Kinect depth,and the depth edges are aligned with the texture edges.The proposed framework synthesizes an HR depth map given its LR depth map and corresponding HR color image.Our proposed method first estimates the initial HR depth map via traditional up-sampling approaches,then extracts the true edges of the RGB-D data and the fake edges of the initial HR depth map to identify the incorrect regions between the two edges.The incorrect regions of the initial HR depth maps are further refined by joint bilateral filter in an outside-inward refining order to align the edges of color image and depth map.The key to our success is the use of RGB-D depth edge detection,which is inspired by the structured forests-based edge detection.Besides,unlike most depth enhancement methods that use raster-scan order to fill incorrect regions,our method can determine the filing order by considering the true edges.Thus,our HR depth map output exhibits better quality with clear and aligned depth edges compared with the existing depth map SR.However,texture-based guidance may result in incorrect depth value due to the smooth object surface with rich color texture.Thus,the suppression of texture copying artifacts may be our next research goal.