Objective

2

Nowadays， navigation and positioning technology has become an indispensable part of People's Daily life， and the satellite positioning system has been successfully built and widely used. However， in the environment where buildings are dense or satellites are blocked indoors， satellite positioning is inaccurate due to signal interference. Therefore， in order to overcome this difficulty， visual navigation and positioning technology came into being， which determines the position and attitude of the camera in three-dimensional space through image processing and computer vision technology， is an important means to solve the navigation and positioning problem in the satellite navigation rejection scene. Usually， visual matching navigation requires the pre-construction of three-dimensional point cloud information of the scene， and the acquisition of three-dimensional point cloud models can be mainly divided into three types： Manual model construction by mathematical modeling software， mapping by professional instruments and crowdsourcing mapping by consumer terminals are time-consuming and laborious in constructing large-scale feature point cloud databases， while visual modeling of video stream data based on consumer terminals has advantages such as low cost， convenient data update and wide spatial coverage. However， due to the large number of video frames and the image redundancy， the 3D model reconstruction calculation cost is high， the cumulative error is large and even the reconstruction failure is caused， so this paper proposes a 3D reconstruction key frame extraction method based on mutual check weighted optical flow.

Method

2

First of all， the image scene is pre-classified. In the process of video shooting， the camera passes through multiple scenes， and the video frame changes in different scenes. The pixel changes of the straight road are mainly distributed at the edge of the image， while the pixels of the whole image have changed at the turning point， so the video needs to be pre-classified. The system receives the self-collected video stream data， and combines the gyroscope data obtained by the mobile terminal to divide the scene into two types： straight road and turning road， which provides a basis for subsequent targeted optical flow aggregation adjustment and adjacent frame similarity calculation. Then， cross-check adjacent frame matching is carried out， the SIFT（Scale Invariant Feature Transform） algorithm is used to detect feature points and their descriptors in the previous frame image， the matching points in the second frame are calculated by FLANN（Fast Library for Approximate Nearest Neighbors） matching and pyramid LK（Lucas-Kanade） optical flow method respectively， and the feature points successfully matched by both methods are detected， and the wrong matching points are eliminated by calculating two-dimensional Euclidian distance. Strong matching point pairs were obtained to capture the dynamic changes between frames， so as to ensure the accuracy and effectiveness of the subsequent optical flow calculation. Finally， the total optical flow field is aggregated and the similarity of adjacent frames is calculated. Since there are differences in the variation of the video frames of the straight road and the turning road， the contribution of matching feature points of adjacent frames is also different in the similarity calculation， so it is necessary to carry out optical flow aggregation by weighting. After detecting the vanishing point of the image and taking it as the center， different weights are assigned to the strong matching point pairs near the vanishing point according to the scene classification， and then the aggregate information of the total optical flow field of adjacent frames is weighted to obtain the optical flow changes between frames. Then the images with significant motion changes are judged as key frames according to the set threshold， and the key frames of video stream are finally extracted.

Result

2

In the experiment， the self-developed video acquisition app of the research group in the consumer terminal was used to conduct data self-acquisition of the target environment. In different scenes with different lighting and ground feature distribution， four groups of data were obtained by using different traveling routes and speeds， and each group recorded the scene video information and gyroscope data synchronously. The proposed algorithm is compared with the traditional key frame extraction algorithm， and different algorithms are used to screen video key frames， count the number of extracted key frames， and use the structural similarity index （Structure Similarity Index Measure， SSIM） to calculate the number of highly similar frames in key frames. High similarity frames are the images with high visual similarity between the two frames in the extracted video key frames. The larger the number， the larger the redundancy of the extracted key frames. Then， the results of key frame extraction in the straight line and turning road are compared with the original video frame respectively， and the proportion of the number of key frames extracted in the original video frame is calculated in different scenes， so as to evaluate the scene adaptability of the algorithm. Finally， the key frames extracted by the algorithm are used for 3D model reconstruction experiment， and the road map is drawn with GNSS data， so as to evaluate the integrity of the reconstructed model. Experimental results show that the proposed algorithm can reduce the total number of video frames to about 10%， and the minimum can reach 4.56%. Meanwhile， the proportion of high similarity frames in key frames is less than 3%， and the minimum is 1.91%， which is significantly less than other algorithms. In addition， the number of key frames extracted by the algorithm in this paper is much larger than that in the straight road， which is better than other algorithms， and meets the expected demand for the number of images in different scenes under the application of 3D reconstruction. The integrity of the final model reconstruction is 100%， 100%， 97.46% and 96.54% on the 4 groups of data， which is obviously better than other algorithms.

Conclusion

2

This paper proposes a key frame extraction method for 3D reconstruction based on mutual check weighted optical flow， which can effectively reduce the number of video frames and improve the quality of key frame screening in diversified scenes. At the same time， the extracted key frames can improve the matching accuracy and stability of adjacent frames and enhance the robustness of 3D reconstruction.