邵绪强,杨艳,刘艺林(华北电力大学控制与计算机工程学院, 保定 071003)
Review of optical flow algorithms in fluid motion estimation
Shao Xuqiang,Yang Yan,Liu Yilin(School of Control and Computer Engineering, North China Electric Power University, Baoding 071003, China)
The motion analysis of fluid image sequences has been an important research topic in the fields of fluid mechanics, medicine, and computer vision. The dense and accurate velocity vector field extracted from image pairs can provide valuable information for these fields. For example, in the field of fluid mechanics, the velocity vector field can be used to calculate the divergence and curl fields of fluid; in the field of meteorology, the analysis of the velocity vector field can be used to provide weather forecast; in the field of medicine, the velocity vector field is applied to match medical images. In recent years, fluid motion estimation technology based on an optical flow method has become a promising direction in this subject due to its unique advantages. Compared with particle image velocimetry based on a correlation method, an optical flow method can obtain a denser velocity field and can estimate the motion of a scalar image and not just a particle image. In addition, an optical flow method can easily introduce various physical constraints in accordance with the motion characteristics of the fluid and obtain more accurate motion estimation results. In accordance with the basic principles of an optical flow method, this paper reviews a fluid motion estimation algorithm based on an optical flow method. Referring to a large number of domestic and foreign studies, existing algorithms are classified in accordance with outstanding problems to be solved: combining the energy minimization function with the knowledge of fluid mechanics, improving robustness to illumination changes, estimating large displacements, and eliminating outliers. Combining the minimization function with the knowledge of fluid mechanics introduces various physical constraints for improving the energy minimization function, providing physically meaningful data items and regularization terms, and improving the accuracy of fluid motion estimation results. Algorithms for improving robustness to illumination changes can be classified into four types: using a high-order constancy assumption to expand data items that depend on the constant brightness assumption, extracting illumination-invariant features in the image for data items, using structure-texture decomposition methods, and establishing a mathematical model for light changes. Various methods are applicable to different light change conditions. For the large displacement estimation problem, the pyramid-based multi-resolution optical flow method is first used; however, this method cannot estimate the large displacement of fine structures. To solve this problem, a hybrid motion estimation method that combines the cross-correlation method with a wavelet-based optical flow method is proposed in recent research. This hybrid method uses the cross-correlation method to calculate the large displacement of a fine structure and then uses an optical flow method to refine and redetermine the flow field, combining the advantages of the two methods. The optical flow estimation method based on wavelet transform provides a good mathematical framework for the multi-resolution estimation algorithm and avoids the linear problem that exists in the “coarse-to-fine” multi-resolution framework when estimating large displacements. Methods for eliminating outliers can be divided into three basic categories: methods that use a robust penalty function, median filtering, and forward-backward optical flow consistency check. In this paper, each kind of method is introduced from the perspective of the problem solving process, and the characteristics and limitations of existing algorithms are analyzed in various outstanding problems. Finally, the major research problems are summarized and discussed, and several possible research directions for the future are proposed. First, an optical flow method introduces various physical constraints into the objective function to conform to fluid motion characteristics. Hence, although accurate estimation results can be obtained, the resulting optical flow equation is too complex to solve, and no good numerical solution is obtained. Second, several methods based on an optical flow method exhibit different advantages under varying light change conditions; they also have corresponding shortcomings. Therefore, further research on how to combine the advantages of various methods to cope with different light changing conditions is particularly important. Third, although the hybrid method that combines the cross-correlation and optical flow methods can utilize the advantages of the two methods to obtain high-resolution motion results for the large displacement problem, this method can only be successfully applied to the motion estimation of particle images at present. Thus, exploring this method for other types of fluid motion images is worthwhile. Finally, an optical flow method requires complex variational optimization and its computational efficiency is low. Although some graphics processing unit(GPU) parallel algorithms proposed in recent years have effectively improved computational efficiency, they still cannot achieve real-time estimation. Therefore, improving the computational efficiency of fluid motion estimation algorithms and realizing real-time estimation are among the directions that are worth studying in the future.