发布时间: 2018-09-16 摘要点击次数: 全文下载次数: DOI: 10.11834/jig.180029 2018 | Volume 23 | Number 9 图像理解和计算机视觉

 收稿日期: 2018-01-22; 修回日期: 2018-04-03 基金项目: 国家自然科学基金项目（61272523） 第一作者简介: 翟丁丁, 1993年生, 女, 大连理工大学计算机科学与技术专业硕士研究生, 主要研究方向为图像处理。E-mail:zhaidingding@mail.dlut.edu.cn;王琦, 男, 博士, 主要研究方向为图像处理。E-mail:wangqi@mail.dlut.edu.cn;杨燕, 女, 讲师, 主要研究方向为计算机视觉、图像处理。E-mail:xuemeng62038216@163.com;王凡, 女, 副教授, 主要研究方向为图像处理。E-mail:wangfan@dlut.edu.cn. 中图法分类号: TP391.41 文献标识码: A 文章编号: 1006-8961(2018)09-1393-10

关键词

Image compensation for object detection under rotating camera
Zhai Dingding, Wang Qi, Yang Yan, Wang Fan, Hu Xiaopeng
School of Computer Science and Technology, Dalian University of Technology, Dalian 116024, China
Supported by: National Natural Science Foundation of China(61272523)

Abstract

Objective In the field of moving object detection, detecting fixed cameras has gradually matured. In numerous practical applications, camera motion, such as rotating scan, is required to increase the monitoring range and achieve gaze monitoring. In comparison with moving object detection under fixed camera conditions, the camera motion causes further difficulty in moving object detection. Image compensation is needed to eliminate the effect of image transformation caused by the camera motion. However, the traditional linear model cannot solve the nonlinear transform generated by the rotating scan movement of cameras. Under the condition of camera rotating scan, the key step of image compensation is to find an accurate motion model to describe image transformations between image frames, including rotation, translation, and scaling. The existing methods are not able to meet simultaneously the application requirements in terms of calculation time and accuracy. To solve this problem, a robust image compensation method under the condition of camera rotating scan is proposed, which can simultaneously achieve background motion and image nonlinear transform compensations. Method Our method includes four steps to achieve the goal of image compensation for camera rotating scan. First, corresponding point pairs are obtained through image matching. Feature points in the current frame are extracted through Features from Accelerated Segment Test (FAST) corner detection method and then matched with those in the previous frame. Subsequently, the global displacement of the background is computed through the matching points. On this basis, the Kalman filter updates its state and predicts the global displacement of the next frame and the positions of the current feature points appearing on the next image. Consequently, the feature points in the next frame matched with the current feature point are searched in the estimated image area. As the matched image area is reduced, feature matching accuracy can be improved. Second, a global transformation model between adjacent frames is established. In accordance with the analysis of the camera imaging mechanism for rotating scan, a nonlinear motion model is proposed. On the basis of the nonlinear motion model, a camera equation is established, which is further transformed into a linear problem by parameter space conversion. Third, Hough transform is utilized to estimate the parameters of the global motion model by using the matched point pairs. The global motion model is then mapped into the image to obtain the coordinate transformation relationship between adjacent images. Through the coordinate transformation relationship, the image is normalized to a unified coordinate system. This step leads to the implementation of background motion and nonlinear transform compensations. Finally, foreground objects are segmented from the image. The block-based inter-frame difference method is used to detect moving objects. To extract the foreground objects completely, the mathematical morphological opening is operated to eliminate isolated pixel points and small line segments. Then, a closing operation is performed to fill the holes in the object regions to maintain the completeness of the object. Result To prove the validity of the proposed method, different experiments are tested on several videos, including grass, traffic section, and indoor and other real scenes. All the experiments run on the Windows platform and the algorithm is implemented in C++. The adopted camera is Hikvision's DS-2DF230IW-A with resolution of 1 280×720. To evaluate the performance of this method, we compare it with other global motion models, including affine transformation and local linear models. The experimental results can be summarized as follows. When the frame interval is small, the affine transformation model produces a large error. The local model and method presented in this paper can achieve improved results. As the rotation angle of the camera increases, nonlinear transformation becomes increasingly significant. The compensation result generates isolated pixels and small segments due to the occurrence of edge effect for the local linear model method. However, the method proposed in this paper can remove 90% of the isolated pixels and small segments, which can solve the problem of nonlinear transformation for camera rotating scan. In addition, the proposed method can be quickly solved by camera equations and Hough transform with processing speed of 50 frames per second (fps), which can meets real-time requirements. This method also has limitations in that it is only suitable for a low-pitch angle of the camera. The influence of pitch angle on the results of our method requires further analysis and research. Conclusion Detecting moving objects on a rotating scan camera is a difficult issue because the motion of the camera leads to the movement of the background and deformation of the image. Image compensation is required to remove background motion and image deformation. The quality of the image compensation method directly affects the final result of moving object detection. For traditional methods, nonlinear transformation is not considered thoroughly. The camera imaging mechanism under the condition of camera rotating scan is analyzed in this paper. Then, a nonlinear transformation model and the corresponding calculation method are presented. Results prove that compared with existing methods, our proposed method can achieve real-time performance and smaller compensation errors under the condition of camera rotating scan. On the basis of this method, the object detection problem in the dynamic background is converted into the object detection problem in the static background. Then, the reliable detection of the moving object can be achieved by using frame difference. As pan-tilt-zoom monitoring technology is increasingly widely used for scanning and monitoring large-scale scenes, the proposed method has practical values for object detection.

Key words

rotating-scan; nonlinear transform; Hough transform; frame difference; image compensation

1 摄像机旋转扫描模型

Table 1 Error distribution

 帧间间隔 摄相机旋转角度 $x$坐标范围(误差 < 0.5像素) 1帧 0.18° [-640, 640] 5帧 0.9° [-525, 420] 10帧 1.8° [-455, 240] 15帧 2.7° [-475, 150] 20帧 3.6° [-530, 100]

1.1 1维扫描条件下摄像机方程

1) $x$坐标。为方便观察成像点横坐标之间的关系，将旋转扫描运动模型进行水平方向上的投影，如图 3所示。

$CD$$EF是前后两帧的像平面, 那么横坐标的变换则由{N_1}{{A'}_1}变换成了{N_2}{{A'}_2}，令{N_1}{{A'}_1} = x$${N_2}{{A'}_2} = x'$，通过几何关系推导，能够得到两个成像点的横坐标之间关系式为

 $x' = \frac{{x + f \cdot {\rm{tan}}\left( \theta \right)}}{{1 - \frac{x}{f}{\rm{tan}}\left( \theta \right)}}$ (1)

2) $y$坐标。如图 2所示，${A_1}$, ${A_2}$点的纵坐标分别是${A_1}{{A'}_1}$${A_2}{{A'}_2}，令{A_1}{{A'}_1} = y$${A_2}{{A'}_2} = y'$通过几何关系推导，能够得到两个成像点的纵坐标之间的关系式为

 $y' = \frac{{y \cdot \left( {{\rm{1 + ta}}{{\rm{n}}^2}\left( \theta \right)} \right)\cos \left( \theta \right)}}{{1 - \frac{x}{f} \cdot {\rm{tan}}\left( \theta \right)}}$ (2)

$x$坐标变换关系如图 1所示，可以看出，摄像机左右旋转扫描时，非线性变换主要体现在$x$坐标上。

1.2 方程线性化及参数求解

 $x' = \frac{{x + A}}{{1 - x \times B}}$ (3)

 $A = - xx'B + \left( {x' - x} \right)$ (4)

参考文献

• [1] Li J, Lan J H, Li J. A novel fast moving target detection method[J]. Journal of Central South University:Science and Technology, 2013, 44(3): 978–984. [李建, 蓝金辉, 李杰. 一种新型快速运动目标检测算法[J]. 中南大学学报:自然科学版, 2013, 44(3): 978–984. ]
• [2] Tian Y M, Wan B, Dong W T. Object detection algorithm based on moving background in MPEG-4 video[J]. Acta Optica Sinica, 2009, 29(5): 1227–1231. [田玉敏, 万波, 董文涛. MPEG-4视频中运动背景下的目标检测算法[J]. 光学学报, 2009, 29(5): 1227–1231. ] [DOI:10.3788/AOS20092905.1227]
• [3] Zhang S, Wang C, Chan S C, et al. New object detection, tracking, and recognition approaches for video surveillance over camera network[J]. IEEE Sensors Journal, 2015, 15(5): 2679–2691. [DOI:10.1109/JSEN.2014.2382174]
• [4] Tao T F, Han C Z, Wu Y Q, et al. Motion estimation based on an improved block matching technique[J]. Chinese Optics Letters, 2006, 4(4): 208–210.
• [5] Turetken E, Alatan A A. Temporally consistent layer depth ordering via pixel voting for pseudo 3D representation[C]//Proceedings of 20093DTV Conference: the True Vision-Capture, Transmission and Display of 3D Video. Potsdam, Germany: IEEE, 2009: 1-4. [DOI:10.1109/3DTV.2009.5069679]
• [6] Hagege R, Francos J M. Parametric estimation of affine transformations:an exact linear solution[J]. Journal of Mathematical Imaging and Vision, 2010, 37(1): 1–16. [DOI:10.1007/s10851-009-0188-4]
• [7] Wang H B, Zhao Z Y, Gu B, et al. Research on multi-moving object detection algorithm based on global motion compensation[J]. Application of Electronic Technique, 2011, 37(1): 110–113, 116. [王洪斌, 赵振涌, 顾彪, 等. 基于全局运动补偿的多运动目标检测方法研究[J]. 电子技术应用, 2011, 37(1): 110–113, 116. ] [DOI:10.3969/j.issn.0258-7998.2011.01.037]
• [8] Zheng S Y, Fei S M, Liu H, et al. A novel approach to moving object detection in image sequence acquired by a mobile camera[J]. Journal of Image and Graphics, 2007, 12(9): 1590–1597. [郑世友, 费树岷, 刘怀, 等. 动态场景图像序列中运动目标检测新方法[J]. 中国图象图形学报, 2007, 12(9): 1590–1597. ] [DOI:10.3969/j.issn.1006-8961.2007.09.015]
• [9] Wang S J, Zhao J, Han X Z. Fast global motion estimation algorithm based on affine transformation[J]. Chinese Journal of Liquid Crystals and Displays, 2012, 27(2): 263–266. [王思珺, 赵建, 韩希珍. 基于仿射变换的快速全局运动估计算法[J]. 液晶与显示, 2012, 27(2): 263–266. ] [DOI:10.3788/YJYXS20122702.0263]
• [10] Jung B, Sukhatme G S. Real-time motion tracking from a mobile robot[J]. International Journal of Social Robotics, 2010, 2(1): 63–78. [DOI:10.1007/s12369-009-0038-y]
• [11] Lin C C, Wolf M. Detecting moving objects using a camera on a moving platform[C]//Proceedings of the 201020th International Conference on Pattern Recognition. Istanbul, Turkey: IEEE, 2010: 460-463. [DOI:10.1109/ICPR.2010.121]
• [12] Yu H F, Liu W, Yuan H, et al. Moving object detection based on sub-block motion compensation[J]. Acta Electronica Sinica, 2017, 45(1): 173–180. [于红绯, 刘威, 袁淮, 等. 基于子块运动补偿的运动目标检测[J]. 电子学报, 2017, 45(1): 173–180. ] [DOI:10.3969/j.issn.0372-2112.2017.01.024]
• [13] Xu P L, Shimada S. Least squares parameter estimation in multiplicative noise models[J]. Communications in Statistics-Simulation and Computation, 2000, 29(1): 83–96. [DOI:10.1080/03610910008813603]
• [14] Dahyot R. Statistical Hough transform[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(8): 1502–1509. [DOI:10.1109/TPAMI.2008.288]
• [15] Guerreiro R F C, Aguiar P M Q. Incremental local Hough Transform for line segment extraction[C]//Proceedings of the 8th IEEE International Conference on Image Processing. Brussels, Belgium: IEEE, 2011: 2841-2844. [DOI:10.1109/ICIP.2011.6116139]
• [16] von Gioi R G, Jakubowicz J, Morel J M, et al. LSD:a line segment detector[J]. Image Processing on Line, 2012, 2: 35–55. [DOI:10.5201/ipol.2012.gjmr-lsd]
• [17] Wang M, Dai Y P. Local robust feature based on FAST corner detection[J]. Transactions of Beijing Institute of Technology, 2013, 33(10): 1045–1050. [王蒙, 戴亚平. 基于FAST角点检测的局部鲁棒特征[J]. 北京理工大学学报, 2013, 33(10): 1045–1050. ] [DOI:10.3969/j.issn.1001-0645.2013.10.010]
• [18] Gao W G, Yang Y X, Zhang S C. Adaptive robust kalman filtering based on the current statistical model[J]. Acta Geodaetica et Cartographica Sinica, 2006, 35(1): 15–18, 29. [高为广, 杨元喜, 张双成. 基于当前加速度模型的抗差自适应Kalman滤波[J]. 测绘学报, 2006, 35(1): 15–18, 29. ] [DOI:10.3321/j.issn:1001-1595.2006.01.004]
• [19] Hu W M, Tan T N, Wang L, et al. A survey on visual surveillance of object motion and behaviors[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 2004, 34(3): 334–352. [DOI:10.1109/TSMCC.2004.829274]
• [20] Hu J H, Yang F, Xu J J. Motion detection based on blocks frame difference and background subtraction[J]. Electronic Measurement Technology, 2007, 30(10): 49–53. [胡建华, 杨帆, 徐健健. 基于分块差分与背景减除的运动检测[J]. 电子测量技术, 2007, 30(10): 49–53. ] [DOI:10.3969/j.issn.1002-7300.2007.10.014]
• [21] Serra J. Introduction to mathematical morphology[J]. Computer Vision Graphics and Image Processing, 1986, 35(3): 283–305. [DOI:10.1016/0734-189X(86)90002-2]
• [22] Yuan C, Medioni G, Kang J M, et al. Detection and tracking of moving objects from a moving platform in presence of strong parallax: IEEE Computer Society, USA, 8073196[P]. 2011-12-06.