浏览全部资源
扫码关注微信
Received：01 November 2018，
Revised：2019-1-8，
Published：16 June 2019
移动端阅览
目的
2
像对稠密匹配是3维重建和SLAM（simultaneous localization and mapping）等高级图像处理的基础，而摄影基线过宽、重复纹理、非刚性形变和时空效率低下等问题是影响这类方法实用性的主要因素，为了更好地解决这类问题，本文提出一种面向重复纹理及非刚性形变的高效稠密匹配方法。
方法
2
首先，采用DeepMatching算法获得降采样后像对的匹配点集，并采用随机抽样一致算法剔除其中外点。其次，利用上一步得到的匹配结果估计相机位姿及缩放比例，以确定每个点对稠密化过程中的邻域，再对相应点对的邻域提取HOG描述符并进行卷积操作得到分数矩阵。最后，根据归一化后分数矩阵的数值以及下标距离的方差确定新的匹配点对以实现稠密化。
结果
2
在多个公共数据集上采用相同大小且宽高比为4：3的像对进行实验，实验结果表明，本文方法具备一定的抗旋转、尺度变化与形变的能力，能够较好地完成宽基线条件下具有重复纹理及非刚性形变像对的匹配。与DeepMatching算法进行对比实验，本文方法在查准率、空间效率和时间效率上分别提高了近10%、25%和30%。
结论
2
本文提出的稠密匹配方法具有较高的查准率和时空效率，其结果可以运用于3维重建和超分辨率重建等高级图像处理技术中。
Objective
2
Dense matching between images is the basis of 3D reconstruction
SLAM (simultaneous localization and mapping)
and other advanced image processing methods. However
the problems of excessive baseline
repeated texture
non-rigid deformation
and time-space efficiency largely affect the practicability of such methods. To solve such problems
this study proposes an efficient dense matching method for repeated textures and non-rigid deformation.
Method
2
First
the source and target images are scaled
$$ \alpha $$
via linear-bilinear interpolation. A series of matching points are obtained via DeepMatching (DM)
which constitutes the set
S
and the outer points are eliminated by random sample consensus. Second
the matching set
S
obtained in the previous step is used to estimate the camera pose
x
and scaling
$$ \alpha $$
to determine the neighborhood of each point during densification. Third
the fractional matrix
Sim
is obtained by convoluting the HOG (histogram of gradient) descriptors extracted from the corresponding neighborhood. The fractional matrix
Sim
which is composed of similarity scores between all points in the neighborhood
is the most important concept in our method because it connects two major steps:selecting the appropriate convolution region and determining the new matching point. The size and position of the convolution area
which are respectively decided by scaling factor
$$ \alpha $$
and camera position
x
determine the appropriate neighborhood. The selection of the above convolution neighborhood is still stable under conditions of rotation and scaling. Finally
new matching points are determined according to the values and variance of the subscript distance of the normalized fractional matrix
Sim
to achieve densification. This condition also means that the relative coordinates of the maximum values in each group of
Sim
are restored to the absolute coordinates of the input image.
Result
2
The code is implemented in VS2013 with Intel MKL2015 and Opencv3. Image pairs with the same size and an aspect ratio of 4:3 on Mikolajczyk
MPI-Sintel
and Kitti datasets are used for the experiment in an environment with a 3.8 GHz CPU and 8 GB RAM. To evaluate our method comprehensively and objectively
we select multiple sets of images with different sizes to compare the time and memory usage and precision of the proposed method with those of DeepMatching. To illustrate the problem solved by the proposed method
the method is applied to the matching of image pairs under repeated texture and non-rigid deformation conditions. Under the condition of repeated texture
the method can not only solve the matching problem under rotation and scaling conditions but also realize the matching problem of repeated texture under a wide baseline; the method also performs well in non-rigid deformation. To evaluate the time and space efficiency of the method
the same size and aspect ratio 4:3 pairs were tested on the Mikolajczyk
MPI-Sintel
and Kitti datasets
respectively. From the experiment
the proposed algorithm outperformed the DM in terms of time and space efficiency
especially in processing certain large-size images. For the convenience of comparison of the processing time
the experiment was performed on the Kitti dataset and the median of theresultswas taken as the data
when
$$ \alpha $$
was seted 0.5
the execution time of the algorithm and the memory usage rate were both low and the density in the unit pixel is similar to the original image (
$$ \alpha $$
=1). To evaluate the accuracy assessment of this method
a pixel was considered correct if its pixel match in the second image was closer than 8 pixels to the ground-truth
while allowing some tolerance in the blurred areas that were difficult to match exactly. Since our method used camera pose to eliminate some outer points in the process of determining the centre of the neighbourhood
so the accuracy of our method is better than the DM when the image size selected between 16 and 512
but as the image size increased to 5121 024
the proportion of DM outer points is less and less due to the increase of the number of DM inner points. The accuracy of DM and ours was basically the same. In summary
by combining the calculationresultson precision in the above datasets
the precision of the experimentalresultsof this method is determined to be better than that of the direct use of the DeepMatching algorithm (average increase of about 10%). Moreover
as the image size increases
memory and time usage increase by nearly 25% and 30%
respectively.
Conclusion
2
To verify the effectiveness of the proposed method
the time and memory usage and precision of this method are compared with those of DeepMatching in multiple public datasets. Precision and time and memory usage increase by 10%
25%
and 30%
respectively. The effect of wide baseline
repeated texture
and non-rigid deformation on the robustness and efficiency of matchingresultsis solved. We code rotation and scaling to achieve algorithm versatility. For high versatility and practicality
we will integrate this method into advanced image processing
such as 3D reconstruction and SLAM.
Liu J, Hu Y N, Yang J, et al. 3D feature constrained reconstruction for low-dose CT imaging[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2018, 28(5):1232-1247.[DOI:10.1109/TCSVT.2016.2643009]
Yi C, Zhao Y Q, Chan J C W. Hyperspectral image super-resolution based on spatial and spectral correlation fusion[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(7):4165-4177.[DOI:10.1109/TGRS.2018.2828042]
Song J W, Wang J, Zhao L, et al. MIS-SLAM:real-time large-scale dense deformable SLAM system in minimal invasive surgery based on heterogeneous computing[J]. IEEE Robotics and Automation Letters, 2018, 3(4):4068-4075.[DOI:10.1109/LRA.2018.2856519]
Ma Y B, Jiang Z G, Zhang H P, et al. Breast histopathological image retrieval based on latent dirichlet allocation[J]. IEEE Journal of Biomedical and Health Informatics, 2017, 21(4):1114-1123.[DOI:10.1109/JBHI.2016.2611615]
Zhu H, Song W D, Yang D, et al. Dense matching method of inserting point into the Delaunay triangulation for close-range image[J]. Science of Surveying and Mapping, 2016, 41(4):19-23.
朱红, 宋伟东, 杨冬, 等.近景影像三角网内插点密集匹配方法[J].测绘科学, 2016, 41(4):19-23.[DOI:10.16251/j.cnki.1009-2307.2016.04.005]
Wu W, Shen Z F, Wang X W, et al. An affine invariant-based match propagation method for quasi-dense image registration[J]. Geomatics and Information Science of Wuhan University, 2018, 43(6):930-936.
吴炜, 沈占锋, 王显威, 等.一种仿射不变量支持的准稠密控制点匹配算法[J].武汉大学学报:信息科学版, 2018, 43(6):930-936.[DOI:10.13203/j.whugis2016]
Chuang T Y, Ting H W, Jaw J J. Dense stereo matching with edge-constrained penalty tuning[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(5):664-668.[DOI:10.1109/LGRS.2018.2805916]
Barnes C, Shechtman E, Goldman D B, et al. The generalized PatchMatch correspondence algorithm[C]//Proceedings of the 11th European Conference on Computer Vision. Heraklion, Crete, Greece: Springer, 2010: 29-43.[ Doi: 10.1007/978-3-642-15558-1_3 http://dx.doi.org/10.1007/978-3-642-15558-1_3 ]
HaCohen Y, Shechtman E, Goldman D B, et al. Non-rigid dense correspondence with applications for image enhancement[C]//Proceedings of the SIGGRAPH'11 ACM SIGGRAPH 2011 Papers. Vancouver, British Columbia, Canada: ACM, 2011: #70.[ Doi: 10.1145/1964921.1964965 http://dx.doi.org/10.1145/1964921.1964965 ]
Yang H S, Lin W Y, Lu J B. DAISY filter flow: a generalized discrete approach to dense correspondences[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA: IEEE, 2014: 3406-3413.[ Doi: 10.1109/CVPR.2014.435 http://dx.doi.org/10.1109/CVPR.2014.435 ]
Kim J, Liu C, Sha F, et al. Deformable spatial pyramid matching for fast dense correspondences[C]//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland, OR, USA: IEEE, 2013: 2307-2314.[ Doi: 10.1109/CVPR.2013.299 http://dx.doi.org/10.1109/CVPR.2013.299 ]
Braux-Zin J, Dupont R, Bartoli A. A general dense image matching framework combining direct and feature-based costs[C]//Proceedings of 2013 IEEE International Conference on Computer Vision. Sydney, NSW, Australia: IEEE, 2013: 185-192.[ Doi: 10.1109/ICCV.2013.30 http://dx.doi.org/10.1109/ICCV.2013.30 ]
Revaud J, Weinzaepfel P, Harchaoui Z, et al. DeepMatching:hierarchical deformable dense matching[J]. International Journal of Computer Vision, 2016, 120(3):300-323.[DOI:10.1007/s11263-016-0908-3]
Keysers D, Deselaers T, Gollan C, et al. Deformation models for image recognition[J]. IEEE Transactions on Pattern Analysis and MachineIntelligence, 2007, 29(8):1422-1435.[DOI:10.1109/TPAMI.2007.1153]
Mikolajczyk K, Tuytelaars T, Schmid C, et al. A comparison of affine region detectors[J]. International Journal of Computer Vision, 2005, 65(1-2):43-72.[DOI:10.1007/s11263-005-3848-x]
Butler D J, Wulff J, Stanley G B, et al. A naturalistic open source movie for optical flow evaluation[C]//Proceedings of the 12th European Conference on Computer Vision. Florence, Italy: Springer, 2012: 611-625.[ Doi: 10.1007/978-3-642-33783-3_44 http://dx.doi.org/10.1007/978-3-642-33783-3_44 ]
Geiger A, Lenz P, Stiller C, et al. Vision meets robotics:the KITTI dataset[J]. The International Journal of Robotics Research, 2013, 32(11):1231-1237.[DOI:10.1177/0278364913491297]
相关作者
相关机构
京公网安备11010802024621
