Wang Feng, Zhang Lei, Huang Hua. Marine video deshaking using steady flow estimation[J]. Journal of Image and Graphics, 2016, 21(3): 365-372. DOI: 10.11834/jig.20160311.
Marine video deshaking using steady flow estimation
A marine video exhibits large-area textureless regions
and traditional video deshaking methods based on feature point detection and tracking for motion estimation usually have poor effects in processing such kinds of videos. In most marine videos captured in a ship
salient feature points are difficult to detect in regions of dominant water and sky
and the involved wave motions makes it difficult to track the detected few feature points. Thus
desirable deshaking results cannot be obtained by applying traditional video deshaking methods. Alternatively
this paper proposes a marine video deshaking method based on the estimation of steady optical flow or the SteadyFlow. The proposed algorithm is based on hierarchical block matching and integrating some smoothness constraints to compute the flow motion of the corresponding hierarchical blocks
thus facilitating rapid and accurate computation of the approximate optical flow field that exists in the marine video. The hierarchical blocks are typically organized into a pyramid with a few levels of blocks; in each level
the most matched blocks are searched in the neighborhood with the local smoothness constraints. The displacements of the blocks in the finest level form the optical flow motion of the marine video. Such motion estimation scheme is more suitable for a marine video with large regions of water and sky regions. Then
the estimated optical flow is smoothed in a spatially and temporally consistent manner to obtain a visually steady motion
where an energy functional optimization is applied to realize the efficient deshaking operation on the marine videos by using the steady optical flow. The SteadyFlow based marine video is implemented using the proposed deshaking algorithm; this is then tested on many marine video examples captured in a ship
which have obvious shakiness caused by wave motions. For the optical flow estimation
different methods on some public databases that have ground truth optical flow are compared for quantitative comparison. For the visual deshaking effect
a set of shaky marine videos are collected. Then
using such data
the proposed method is run along with other comparative methods and sophisticated software for further comparison. The user study is conducted to qualitatively evaluate the performances of all evaluated methods in terms of marine video deshaking. The experiment results on the running time statistics and visual quality comparison demonstrate that the proposed algorithm does not only efficiently realize marine video deshaking effects
but also accelerate the running speed of the process
thus reducing the timing by up to 70% of the traditional methods. A video deshaking algorithm based on the steady optical flow estimation is proposed to deal with the particular set of marine videos. Compared with some traditional methods and software
the method proposed in this paper is more suitable for processing marine video shakiness because it can obtain more accurate motion estimation by using hierarchical block matching with smoothness constraints to compute the motion field of the marine videos.