Semi-global stereo matching with adaptive window based on grayscale value

Chao Huang; Huazhi Zhao

doi:10.11834/jig.180574

Image Understanding and Computer Vision | Views : 0 下载量: 4 CSCD: 10

PDF
Export
Share
Collection
Album

Semi-global stereo matching with adaptive window based on grayscale value
Vol. 24, Issue 8, Pages: 1381-1390(2019)
Received：24 September 2018，

Revised：2019-2-17，

Published：16 August 2019
DOI： 10.11834/jig.180574
稿件说明：

移动端阅览

DOI：

Chao Huang, Huazhi Zhao. Semi-global stereo matching with adaptive window based on grayscale value[J]. Journal of Image and Graphics, 2019, 24(8): 1381-1390. DOI： 10.11834/jig.180574.

摘要

目的

立体匹配算法是立体视觉研究的关键点，算法的匹配精度和速度直接影响3维重建的效果。对于传统立体匹配算法来说，弱纹理区域、视差深度不连续区域和被遮挡区域的匹配精度依旧不理想，为此选择具有全局匹配算法和局部匹配算法部分优点、性能介于两种算法之间、且鲁棒性强的半全局立体匹配算法作为研究内容，提出自适应窗口与半全局立体匹配算法相结合的改进方向。

方法

以通过AD（absolute difference）算法求匹配代价的半全局立体匹配算法为基础，首先改变算法匹配代价的计算方式，研究窗口大小对算法性能的影响，然后加入自适应窗口算法，研究自适应窗口对算法性能的影响，最后对改进算法进行算法性能评价与比较。

结果

实验结果表明，匹配窗口的选择能够影响匹配算法性能、提高算法的适用范围，自适应窗口的加入能够提高算法匹配精度特别是深度不连续区域的匹配精度，并有效降低算法运行时间，对Cones测试图像集，改进的算法较改进前误匹配率在3个测试区域平均减少2.29%；对于所有测试图像集，算法运行时间较加入自适应窗口前平均减少28.5%。

结论

加入自适应窗口的半全局立体匹配算法具有更优的算法性能，能够根据应用场景调节算法匹配精度和匹配速度。

Abstract

Objective

Stereovision is the current research focus in the field of computer vision

and its main research content is to reconstruct a 3D scene through two or more 2D images of the same scene. Stereovision has been widely used in the fields of military

aerospace

and unmanned aerial vehicle

which require 3D reconstruction and speed measurement. The stereovision system generally consists of four basic processes:image acquisition

camera calibration

stereo matching

and 3D reconstruction. Research on the stereo matching algorithm can be considered the key point of stereovision research because the matching accuracy and speed of the stereo algorithm directly affect the result of 3D reconstruction. Therefore

research on stereo matching algorithm has great practical value and theoretical significance. However

the traditional stereo matching algorithm suffers from problems of weak matching in regions with weak textures

deep discontinuities

and non-occlusion. Therefore

we select the semi-global stereo matching algorithm

which has strong robustness and some advantages of global and local matching algorithms. Furthermore

we propose an improved method that combines adaptive window and semi-global stereo matching algorithms.

Method

Our algorithm improvement is based on the adaptive window and semi-global stereo matching algorithms

and it uses the absolute difference (AD) algorithm to calculate the matching cost. First

we changed the original AD algorithm to the sum of absolute differences (SAD) algorithm to obtain the matching cost

which provides the possibility of implementing the adaptive window. Thereafter

we analyzed the necessity and rationality of assuming an adaptive window by studying the effect of window size on the performance of the SAD and semi-global stereo matching algorithms. Furthermore

we added the adaptive window algorithm to the SAD and semi-global stereo matching algorithms to study the effects of the adaptive window on the performance of the SAD and semi-global stereo matching algorithms. In this part

we proposed a new parameter adaptive window judgment threshold. We tested the influences of this judgment threshold on the matching algorithm. Next

we evaluated the performance of the algorithms and compared them in terms of the optimal matching precision and matching speed by using the standard test image pairs provided by the test platform. Finally

we used a binocular camera to obtain left and right views in a real indoor scene. We further compared the performance of the above stereo matching algorithms by using a disparity map and by analyzing the algorithms' runtime.

Result

The experimental results show that the selection of the size of the matching window can affect the performance of the matching algorithm and improve the applicable range of the algorithm. The addition of the adaptive window could improve the matching accuracy of the algorithm

especially in the depth discontinuous region

and effectively reduce the runtime of the algorithm. After adding the adaptive window algorithm

a large preset maximum window corresponds to more evident optimization of the algorithm runtime. However

the change in matching accuracy is uncertain

which may be improved or decreased. As for the effects of window size judgment threshold

the optimal number of judgment thresholds varies in different standard test image pairs

and the judgment thresholds have different effects on the SAD and the semi-global stereo matching algorithms. The window size judgment threshold has minimal influence on the performance of the semi-global stereo matching algorithm. Thus

the choice of the number of window size judgment threshold is more flexible. The optimal window size of the semi-global stereo matching algorithm is small due to the influence of other parameters (penalty coefficient and threshold) on the performance of the algorithm

and the adaptive window performs limited optimization of the runtime of the algorithm. For the test image pair cones

the improved semi-global stereo matching algorithm mismatch rate is reduced by 2.29% on average in three test areas

and for all test image pairs

the runtime of the algorithm is reduced by 28.5%.

Conclusion

In this paper

we present an improved algorithm that combines the adaptive window and semi-global stereo matching algorithms. This improved algorithm was evaluated on standard image pairs

and its performance of our algorithm was compared with that of conventional algorithms. The improved algorithm showed competitive processing time results and accuracy in cones and teddy image pairs

which have a rich texture and a large disparity range. Such approach could optimize the matching accuracy and runtime despite being evaluated on image pairs with a weak texture and small disparity range. This paper contains detailed experimental results of the mismatch rate and runtime of different matching algorithms in four standard test image pairs and three image test areas. We conclude that our algorithm has the advantages of improving matching accuracy in depth discontinuity regions

effectively reducing the runtime

and adjusting the matching accuracy and speed according to the application scene.

关键词

Keywords

references

Xu L F, Au O C, Sun W X, et al. Stereo matching by adaptive weighting selection based cost aggregation[C]//2013 IEEE International Symposium on Circuits and Systems. Beijing: IEEE, 2013: 1420-1423.[ DOI: 10.1109/ISCAS.2013.6572122 http://dx.doi.org/10.1109/ISCAS.2013.6572122 ]

Lin C H, Liu C W. Accurate stereo matching algorithm based on cost aggregation with adaptive support weight[J]. The Imaging Science Journal, 2015, 63(8):423-432.[DOI:10.1179/1743131X15Y.0000000024]

Hosni A, Bleyer M, Gelautz M. Secrets of adaptive support weight techniques for local stereo matching[J]. Computer Vision and Image Understanding, 2013, 117(6):620-632.[DOI:10.1016/j.cviu.2013.01.007]

Veksler O. Fast variable window for stereo correspondence using integral images[C]//Proceedings of 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Madison, WI, USA: IEEE, 2003: I-I.[ DOI: 10.1109/CVPR.2003.1211403 http://dx.doi.org/10.1109/CVPR.2003.1211403 ]

Shi H, Zhu H. Stereo matching based on adaptive matching windows and multi-feature fusion[J]. Pattern Recognition and Artificial Intelligence, 2016, 29(3):193-202.

时华, 朱虹.基于自适应匹配窗及多特征融合的立体匹配[J].模式识别与人工智能, 2016, 29(3):193-202. [DOI:10.16451/j.cnki.issn1003-6059.201603001]

Liang Q, Yang Y Y, Liu B. Efficient adaptive window matching algorithm based on cross search[C]//Proceedings of the 3rd International Conference on Multimedia Technology (ICMT 2013). Berlin, Heidelberg: Springer, 2014: 257-265.[ DOI: 10.1007/978-3-642-41407-7_25 http://dx.doi.org/10.1007/978-3-642-41407-7_25 ]

Tang Y Y, Wang H B, Liu B. A new stereo matching algorithm based on adaptive window[C]//Proceedings of International Conference on Systems & Informatice. Yantai, China: IEEE, 2012.[ DOI: 10.1109/ICSAI.2012.6223397 http://dx.doi.org/10.1109/ICSAI.2012.6223397 ]

Zhang X X, Liu Z G. A survey on stereo vision matching algorithms[C]//Proceedings of the 11th World Congress on Intelligent Control and Automation. Shenyang: IEEE, 2014: 2026-2031.[ DOI: 10.1109/WCICA.2014.7053033 http://dx.doi.org/10.1109/WCICA.2014.7053033 ]

Zhai Z G. Research on stereo matching algorithm[D]: Beijing: Beijing Institute of Technology, 2010.

翟振刚.立体匹配算法研究[D].北京: 北京理工大学, 2010.

Bradski G, Kaehler A. Learning OpenCV[M]. Yu S Q, Liu R Z, trans. Beijing: Tsinghua University Press, 2009.

Bradski G, Kaehler A.学习OpenCV(中文版).于仕琪, 刘瑞祯, 译.北京: 清华大学出版社, 2009.

Hermann S, Morales S, Klette R. Half-resolution semi-global stereo matching[C]//Proceedings of 2011 IEEE Intelligent Vehicles Symposium. Baden-Baden, Germany: IEEE, 2011: 201-206.[ DOI: 10.1109/IVS.2011.5940427 http://dx.doi.org/10.1109/IVS.2011.5940427 ]

Hirschmuller H. Accurate and efficient stereo processing by semi-global matching and mutual information[C]//Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, CA, USA: IEEE, 2005: 807-814.[ DOI: 10.1109/CVPR.2005.56 http://dx.doi.org/10.1109/CVPR.2005.56 ]

Birchfield S, Tomasi C. Depth discontinuities by pixel-to-pixel stereo[J]. International Journal of Computer Vision, 1999, 35(3):269-293.[DOI:10.1023/A:1008160311296]

Alert me when the article has been cited

提交

Transformer-based multi-style information transfer in image processing

3D object detection for autonomous driving from image：a survey ——benchmarks，constraints and error analysis

The review of distortion-related image quality assessment

Players-aware tracking algorithm in soccer video

Tracking multiple moving objects based on OPTICS and object probability model