Monocular depth measurement using motion cues

Wei Wang; Fengmei Liang; Linlin Wang

doi:10.11834/jig.190223

Image Analysis and Recognition | Views : 0 下载量: 44 CSCD: 1

PDF
Export
Share
Collection
Album

Monocular depth measurement using motion cues
Vol. 25, Issue 3, Pages: 468-475(2020)
Received：27 May 2019，

Revised：2019-7-28，

Accepted：04 August 2019，

Published：16 March 2020
DOI： 10.11834/jig.190223
稿件说明：

移动端阅览

DOI：

Wei Wang, Fengmei Liang, Linlin Wang. Monocular depth measurement using motion cues[J]. Journal of Image and Graphics, 2020, 25(3): 468-475. DOI： 10.11834/jig.190223.

摘要

目的

传统的单目视觉深度测量方法具有设备简单、价格低廉、运算速度快等优点，但需要对相机进行复杂标定，并且只在特定的场景条件下适用。为此，提出基于运动视差线索的物体深度测量方法，从图像中提取特征点，利用特征点与图像深度的关系得到测量结果。

方法

对两幅图像进行分割，获取被测量物体所在区域；然后采用本文提出的改进的尺度不变特征变换SIFT（scale-invariant feature transtorm）算法对两幅图像进行匹配，结合图像匹配和图像分割的结果获取被测量物体的匹配结果；用Graham扫描法求得匹配后特征点的凸包，获取凸包上最长线段的长度；最后利用相机成像的基本原理和三角几何知识求出图像深度。

结果

实验结果表明，本文方法在测量精度和实时性两方面都有所提升。当图像中的物体不被遮挡时，实际距离与测量距离之间的误差为2.60%，测量距离的时间消耗为1.577 s；当图像中的物体存在部分遮挡时，该方法也获得了较好的测量结果，实际距离与测量距离之间的误差为3.19%，测量距离所需时间为1.689 s。

结论

利用两幅图像上的特征点来估计图像深度，对图像中物体存在部分遮挡情况具有良好的鲁棒性，同时避免了复杂的摄像机标定过程，具有实际应用价值。

Abstract

Objective

The process of shooting a normal camera involves forming a 2D image after projecting a 3D scene onto the imaging plane. This process will lose the information of the scene

so the related research based on depth information cannot be developed. For example

target detection and tracking

3D model reconstruction

and intelligent robots in industrial automation need to obtain depth information for the scene. The depth information of the scene is the basic problem in the field of machine vision. With the development of machine vision

the use of visualization methods to solve the deep extraction problem became an important topic in computer vision research. Among the methods

image depth measurement based on monocular vision has the advantages of simple equipment

low price

and fast calculation speed and could become a research hot spot today. The traditional measurement method based on monocular visual depth requires complex calibration of the camera

so the operability is not strong and it is difficult to apply in practice. Most of the traditional methods are only suitable for specific scene conditions

such as occlusion relations or defocusing in the scene

which limits the application of traditional methods. Aiming at the limitations of traditional methods

this paper proposes an object depth measurement method based on motion parallax cues

extracts feature points from images

analyzes the relationship between feature points and image depth

and obtains image depth results based on the relationship between the two.

Method

The method uses the parallax cues generated by the camera motion to obtain the image depth

so the method requires two images. The camera is mounted on a movable rail

and after the first image is taken

the camera is moved along the optical axis. The second image is taken at a distance when no adjustment to any parameter of the camera is required. First

the two images acquired by camera movement are segmented

and the region of interest (ROI) is segmented. Second

the improved scale-invariant feature transform algorithm proposed in this paper is used to perform two images. The results of image segmentation and image matching are combined to obtain the matching result of the object to be measured. Then

Graham scanning method is used to obtain the convex hull of the feature points after the two images are matched

thereby obtaining the length of the longest line segment on the convex hull. Finally

the basic principles of camera imaging and triangulation knowledge are used to calculate image depth.

Result

This proposed method is compared with another method

and results are provided in a table. The experiment is divided into two groups

mainly comparing the two methods from two aspects:measurement time and precision. The first set of experimental results show that the proposed method achieves good measurement results in a simple background environment. The error between the actual distance and the measured distance is 2.60%

and the time consumption of the measured distance is 1.577 s. The second set of experiments shows that when partial occlusion occurs in the scene

the error between the actual distance and the measured distance is 3.19%

and the time required to measure the distance is 1.689 s. By comparing the two sets of experimental data

we found that the method has improved the measurement accuracy and measurement time compared with the previous method

especially in reducing the image depth measurement time.

Conclusion

Through research

the method estimates the image depth by moving the camera to obtain the corresponding line segment length on the two images

which avoids the complicated camera calibration process. This method improves the image matching algorithm and reduces the computational complexity

which is fast and accurate. The method of obtaining image depth information has certain research value and only needs to process the two pictures

where the hardware requirement is simple. The measurement process does not require a large amount of scene information

so the scope of application is wide. The method utilizes feature points on the image to perform image depth measurement

so the method is not constrained by partial occlusion of the measured object

and still has good robustness. However

as the method uses an image segmentation algorithm

the result of image segmentation greatly influences the accuracy of the measurement. If the captured image contains a complex background environment

obtaining accurate image segmentation results and ideal depth measurement results is difficult. Therefore

the directions of optimization and improvement of this method are adaptable to complex background environments.

关键词

Keywords

references

Bay H, Ess A, Tuytelaars T and Van Gool L. 2008. Speeded-up robust features (SURF). Computer Vision and Image Understanding, 110(3):346-359[DOI:10.1016/j.cviu.2007.09.014]

Feng F, Ma J, Yue Z H and Shen L. 2017. Extraction of monocular depth information based on image intensity clue. Journal of Image and Graphics, 22(12):1701-1708

冯帆, 马杰, 岳子涵, 沈亮. 2017.图像亮度线索下的单目深度信息提取.中国图象图形学报, 22(12):1701-1708)[DOI:10.11834/jig.170001]

Harris C and Stephens M. 1988. A combined corner and edge detector//Proceedings of Alvey Vision Conference. Manchester: AVC University of Manchester: 147-151[ DOI:10.5244/C.2.23 http://dx.doi.org/10.5244/C.2.23 ]

He L X. 2018. Research on Depth Measurement Methods of Monocular Image. Hefei:University of Science and Technology of China

何立新. 2018.单目视觉图像深度测量方法研究.合肥:中国科学技术大学

Juan L and Gwun O. 2009. A comparison of SIFT, PCA-SIFT and SURF. International Journal of Image Processing, 3(4):143-152

Ke Y and Sukthankar R. 2004. PCA-SIFT: a more distinctive representation for local image descriptors//Proceedings of 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, DC, USA: IEEE: 506-513[ DOI:10.1109/CVPR.2004.1315206 http://dx.doi.org/10.1109/CVPR.2004.1315206 ]

Liu K, Xia M and Yang X M. 2017. An effective 2D convex hull algorithm. Advanced Engineering Sciences, 49(5):109-116

刘凯, 夏苗, 杨晓梅. 2017.一种平面点集的高效凸包算法.工程科学与技术, 49(5):109-116)[DOI:10.15961/j.jsuese.201601149]

Liu M, Salzmann M and He X. 2014. Discrete-continuous depth estimation from a single image//2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus: OH: 716-723[ DOI:10.1109/CVPR.2014.97 http://dx.doi.org/10.1109/CVPR.2014.97 ]

Lowe D G. 2004. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91-110[DOI:10.1023/b:visi.0000029664.99615.94]

Ming A L, Wu T F, Ma J X, Sun F and Zhou Y. 2016. Monocular depth-ordering reasoning with occlusion edge detection and couple layers inference. IEEE Intelligent Systems, 31(2):54-65[DOI:10.1109/mis.2015.94]

Moravec H P. 1977. Toward automatic visual obstacle avoidance//Proceedings of the 5th International Joint Conference on Artificial Inteuigence. 2.584

Qi L. 2016. Research on Method of 3D Video Reconstruction Based on 2D Video. Chongqing:Chongqing University

亓林. 2016.基于2D视频的3D视频构建方法研究.重庆:重庆大学

Rajasekhar D, Jayachandra P T and Soundararajan K. 2018. Implementation and performance analysis of SIFT and ASIFT image matching algorithms//Dash S S, Naidu P C B, Bayindir R and Das S, eds. Artificial Intelligence and Evolutionary Computations in Engineering Systems. Singapore: Springer: 441-453[ DOI:10.1007/978-981-10-7868-2_43 http://dx.doi.org/10.1007/978-981-10-7868-2_43 ]

Schmid C, Mohr R and Bauckhage C. 2000. Evaluation of interest point detectors. International Journal of Computer Vision, 37(2):151-172[DOI:10.1023/A:1008199403446]

Smith S M and Brady J M. 1997. SUSAN——a new approach to low level image processing. International Journal of Computer Vision, 23(1):45-78[DOI:10.1023/A:1007963824710]

Tafti A P, Baghaie A, Kirkpatrick A B, Holz J D, Owen H A, D'Souza R M and Yu Z Y. 2018. A comparative study on the application of SIFT, SURF, BRIEF and ORB for 3D surface reconstruction of electron microscopy images. Computer Methods in Biomechanics and Biomedical Engineering:Imaging and Visualization, 6(1):17-30[DOI:10.1080/21681163.2016.1152201]

Toussaint N, Stoeck C T, Schaeffter T, Kozerke S, Sermesant M and Batchelor P G. 2013. In vivo human cardiac fibre architecture estimation using shape-based diffusion tensor processing. Medical Image Analysis, 17(8):1243-1255[DOI:10.1016/j.media.2013.02.008]

Tsai Y M, Chang Y L and Chen L G. 2006. Block-based vanishing line and vanishing point detection for 3D scene reconstruction//Proceedings of 2006 International Symposium on Intelligent Signal Processing and Communications. Tottori, Japan: IEEE: 586-589[ DOI:10.1109/ISPACS.2006.364726 http://dx.doi.org/10.1109/ISPACS.2006.364726 ]

Yang F J and Zheng L Y. 2019. New feature extraction matching algorithm based on SIFT. Applied Science and Technology, 46(2):94-97, 103

杨福嘉, 郑丽颖. 2019.基于SIFT的新特征提取匹配算法.应用科技, 46(2):94-97, 103)[DOI:10.11991/yykj.201806011]

Zhu Y and Yu Q. 2017. Non-parametric sampling based depth estimation from single image. Application Research of Computers, 34(6):1876-1880

朱尧, 喻秋. 2017.基于非参数化采样的单幅图像深度估计.计算机应用研究, 34(6):1876-1880)[DOI:10.3969/j.issn.1001-3695.2017.06.063]

Zhuo W, Salzmann M, He X M and Liu M M. 2015. Indoor scene structure analysis for single image depth estimation//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA: IEEE: 614-622[ DOI:10.1109/CVPR.2015.7298660 http://dx.doi.org/10.1109/CVPR.2015.7298660 ]

Zhuo S J and Sim T. 2011. Defocus map estimation from a single image. Pattern Recognition, 44(9):1852-1858[DOI:10.1016/j.patcog.2011.03.009]