|
|
|
发布时间: 2019-11-16 |
图像理解和计算机视觉 |
|
|
|
黄明益, 吴军 |
收稿日期: 2019-03-13; 修回日期: 2019-04-30; 预印本日期: 2019-05-07
基金项目: 国家自然科学基金项目（41761087）；广西自然科学基金项目（2015GXNSFDA139030，2017GXNSFAA198162）；桂林电子科技大学研究生教育创新计划资助项目（2018YJCXB62）
第一作者简介:
黄明益, 1989年生, 男, 博士研究生, 主要研究方向为摄像机标定、影像地理配准、视频融合。E-mail:1240409615@qq.com.
中图法分类号: TP391
文献标识码: A
文章编号: 1006-8961(2019)11-1972-13
|
摘要
目的
鱼眼镜头是发展轻、小型全方位视觉系统的理想光学传感器，但由于镜头焦距短、视场大及光学原理约束，鱼眼图像存在严重畸变，为此提出一种高精度、应用方式灵活的鱼眼镜头内部参数标定方法，以期将鱼眼图像转换成符合人眼视觉习惯的平面透视投影图像。方法
从球面透视投影模型出发，首先分析给出空间直线在水平面上的理想投影椭圆约束，进而结合椭圆严格几何特性建立误差方程对鱼眼相机等效焦距
关键词
鱼眼镜头标定; 鱼眼相机标定; 鱼眼图像矫正; 球面透视投影; 理想投影椭圆约束
Huang Mingyi, Wu Jun Abstract
Objective
Fish-eye lens is an ideal optical sensor to develop light and small omnidirectional vision systems. Due to the characteristics of large field of view and low cost, it is widely used in various places such as security monitoring. Based on a large number of original fisheye video materials, it has the potential for in-depth research and full exploitation. Therefore, the calibration of fisheye cameras needs to obtain the internal parameters of the images. How to improve the indoor and outdoor aspects of the city is imperative. The calibration efficiency of model fisheye camera is a valuable and challenging research work. However, due to the limitation of short focal length, large field of view and special optical principle, fish-eye images will produce serious barrel distortion, which is not conducive to the subsequent development and application of video images. Constraints of optical principle it is expected to be transformed into plane perspective projection, which conforms to human visual habits, by a set of high-precision parameters associated with the optical imaging model of the fish-eye lens. To this end, a high-precision and flexible method for calibrating the internal parameters of fish-eye lens is proposed in this study.
Method
The calibration is achieved through the following steps: Firstly, we need to obtain the initial internal parameters of fisheye image. According to the principle that the spatial line is imaged as an elliptic curve on the image, we extract the image elliptic curve. The specific methods are as follows: a) obtain the coordinates of the curve points on the image by image segmentation, b) obtain the general curve equation of the ellipse by curve fitting, than decompose the general equation into the ellipse long and short axis length and image principal point used as initial value. Secondly, ideal projection ellipse constraints (IPECs) for any space line on the horizontal plane under spherical perspective projection are mathematically set. The constraints are as follows: a) the half length of the long axis of projection ellipse of different space straight lines is constantly equal to the radius of the projection sphere; and b) the length ratio of long axis to short axis of the projection ellipse is constant for any one space line when the radius of the projection sphere is changed. Thirdly, a nonlinear function is built on the basis of the proposed IPECs and the strict geometric properties of ellipses to conduct an iterative least square estimation for the uncalibrated fish-eye lens parameters, namely, focal length f, aspect ratio
Key words
fish-eye lens calibration; fish-eye camera calibration; fish-eye image correction; spherical perspective projection; ideal projection ellipse constraint
0 引言
鱼眼镜头具有视角广阔(接近甚至超过180°)、体积小等优点，是发展轻、小型全方位视觉系统的理想光学传感器，在安全监控、机器人导航、全景泊车等众多领域有极其广泛的应用前景[1-3]。然而，由于镜头焦距短、视场大等特点以及光学原理约束，鱼眼图像存在严重畸变，使用前需转换成符合人眼视觉习惯的平面透视投影图像，前提是获得与鱼眼镜头光学成像模型相联系的一套高精度参数[4-6]。鱼眼镜头通常由10组以上、甚至多达几十组的光学镜片复杂组合而成，光路计算十分困难[7]，目前主要利用球面投影模型来表征其光学成像过程，从这一角度出发，现有鱼眼镜头参数标定方法可概略分为球面投影成像和非球面投影成像两类。
球面投影成像类标定方法主要利用空间直线在球面上投影为一个大圆(或在鱼眼图像上的投影为椭圆弧)的几何特性来求解模型参数。黄有度等人[8]利用圆锥曲线方程描述空间直线在鱼眼图像上的投影椭圆弧，从方程系数中可分解出球面投影模型参数—球面直径(椭圆长轴)和图像光学中心(椭圆中心)，但实际应用中水平面平行条件难以满足且参数标定精度有限。Huang等人[9]在椭圆拟合基础上引入高斯曲面拟合，后续标定计算可给出更为精确的图像光学中心位置，但要求标定影像获取时鱼眼相机主光轴垂直于平面标定板且需在不同深度位置多次摄影；文献[10-11]引用单位视球(viewing sphere)概念并以平面棋盘格为2维靶标，先根据棋盘格角点在世界坐标系、球面坐标系及鱼眼图像坐标系下的坐标映射关系获得参数初值，再综合利用空间直线位置特性及其投影特性进行非线性优化，模型参数求解精度高，但由于全局优化中外部参数数量多，不准确的外部参数初始值极易使优化陷入局部最优。皮英冬等人[12]利用DLT模型迭代求解球面变换半径，鱼眼图像上较多的标定控制点使得参数空间关系恢复计算更稳健，但高精度3维标定场建造较复杂且需不断维护，同时对包括DLT系数在内的全部相机参数进行全局优化将更为合理。Kanatani[13]建立统一的特征值最小化计算框架以综合利用平面网格直线特性精确估计鱼眼相机内部参数，大大降低对标定参照物的摄影要求及标定影像特征提取难度，但对网格直线的共线、平行、正交特性联合利用存在较高的限制且初值的合理选取有待解决；Zhang等人[14]以投影球面某一外切平面(垂直于主光轴)为纠正平面，基于直线透视保持特性建立标定方程并通过能量最小化求解鱼眼相机内部参数，但超大视角(≥180°)鱼眼图像并不能透视纠正到单个平面且未考虑光学畸变参数影响；Aghayari等人[15]以单位视球上设定网格(点)的经、纬度角表征鱼眼相机内部参数，利用共线方程同时求解鱼眼相机内、外部参数并进行集束捆绑调整(bundle adjusting)，潜在的问题在于如何优化网格设置以使网格(点)与3D控制点相对应，从而避免计算矩阵的降秩退化。
非球面投影成像类标定方法的主要目的是利用标定参数对鱼眼图像进行平面透视几何纠正，隐含利用了空间直线平面透视投影仍为直线这一几何限制。文献[16-17]均采用通用光学图像畸变模型对鱼眼图像进行矫正，区别在于文献[16]借鉴了空间直线球面透视投影特性，通过最小化目标函数对应于同一条空间直线的球面点到相应拟合大圆的球面距离的平方和来获得鱼眼图像径向、切向畸变参数，文献[17]则建立了光线跟踪下的鱼眼镜头成像畸变模型并通过曲线拟合获得所需的径向、切向畸变函数，两种算法均采用高阶畸变模型，计算复杂度高。廖士中等人[18]利用多项式表示矫正前后像素点坐标之间的关系，并通过最小二乘法求得多项式参数以实现鱼眼图像矫正。杨玲等人[19]建立经纬映射图像关系将扭曲的半球鱼眼图像投射为普通照片的四方形状。魏利胜等人[20]针对传统经纬矫正模型水平方向畸变大的问题，通过正交投影策略将鱼眼图像映射到球面上，从而投射为以横向、纵向双经度坐标为基础的正方形平面图像，这类方法的优点在于标定参数计算过程简单、复杂度低，但精度不高且无法获得鱼眼相机等效焦距等参数，不利于量测信息的获取。此外，涂波等人[21]利用扩展小孔成像模型对鱼眼相机90°视场范围先进行矫正, 再结合直线拟合及多视图将矫正视场范围扩展到180°，不足之处在于要使用特殊的、非等间距的点阵模板，且需移动模板位置来获得鱼眼图像不同区域的矫正结果。贾云得等人[22]在小孔成像模型基础上引入鱼眼镜头径向、偏心及薄棱镜光学畸变, 建立了一种精确标定鱼眼镜头成像立体视觉系统的方法，但其参数达19个，存在过参数化问题。Ramalingam等人[23]基于3视张量建立了适用于单(多)摄影中心、平面(球面)透视成像相机的统一标定模型，但仅限于相机外部参数求解。
1 鱼眼成像模型
透视成像主要有两种方式：一是利用位于投影中心附近(但不过中心)的平面与这些通过投影中心O的射线相交(平面透视投影)，见图 1(a)；二是利用球心在O点的球面与这些通过投影中心的射线相交(球面透视投影), 见图 1(b)。由图 1可看出，平面透视投影模型仅限于OP与主轴OZ夹角小于90°的射线(当夹角为90°时，空间点在平面上投影为无穷远点)，而球面透视投影模型则对夹角大于或等于90°的射线仍起作用，鱼眼镜头多采用球面透视投影模型来获得大于或等于180°的视场。
本文鱼眼成像采用单位球面透视投影[10-11]，成像过程如图 2所示，从空间点经过4个阶段变换到鱼眼图像像素坐标。
1) 空间坐标变换，即将世界坐标系下任一空间点
| $ {\mathit{\boldsymbol{P}}_{\rm{C}}} = \mathit{\boldsymbol{R}}{\mathit{\boldsymbol{P}}_{\rm{W}}} + \mathit{\boldsymbol{T}} $ | (1) |
式中，
2) 单位球面映射，即将点
| $ \begin{array}{*{20}{c}} {{X_{\rm{S}}} = \frac{{{X_{\rm{C}}}}}{{\sqrt {X_{\rm{C}}^2 + Y_{\rm{C}}^2 + Z_{\rm{C}}^2} }},{Y_{\rm{S}}} = \frac{{{Y_{\rm{C}}}}}{{\sqrt {X_{\rm{C}}^2 + Y_{\rm{C}}^2 + Z_{\rm{C}}^2} }},}\\ {{Z_{\rm{S}}} = \frac{{{Z_{\rm{C}}}}}{{\sqrt {X_{\rm{C}}^2 + Y_{\rm{C}}^2 + Z_{\rm{C}}^2} }}} \end{array} $ | (2) |
3) 球面投影，即按选定模型将点
| $ {x_m} = {X_{\rm{S}}},{y_m} = {Y_{\rm{S}}} $ | (3) |
4) 像素坐标变换，即利用相机内部参数将理想投影点坐标变换到鱼眼图像像素坐标，相机内部参数通常表示为矩阵
| $ \left\{ {\begin{array}{*{20}{l}} {\left( {u - {u_0}} \right) \times \left( {1 + {k_1}{r^2} + {k_2}{r^4}} \right) = Af \times {x_n}}\\ {\left( {v - {v_0}} \right) \times \left( {1 + {k_1}{r^2} + {k_2}{r^4}} \right) = f \times {y_m}}\\ {{r^2} = {{\left( {u - {u_0}} \right)}^2} + {{\left( {v - {v_0}} \right)}^2}} \end{array}} \right. $ | (4) |
式中, (
上述成像过程中，参数(
2 水平面理想投影椭圆约束
本文将结合椭圆自身几何特性以及空间直线在球面透视投影下的投影椭圆特性给出关于上述鱼眼镜头内部参数的标定方程进行参数估计。令
| $ \begin{array}{*{20}{c}} {\sqrt {{{\left( {x - {x_0} + c \times \cos \theta } \right)}^2} + {{\left( {y - {y_0} + c \times \sin \theta } \right)}^2}} + }\\ {\sqrt {{{\left( {x - {x_0} - c \times \cos \theta } \right)}^2} + {{\left( {y - {y_0} - c \times \sin \theta } \right)}^2}} - }\\ {2a = 0} \end{array} $ | (5) |
式中，(
式(5)为非线性方程，由椭圆轮廓点直接计算几何参数(
| $ {x^2} + Axy + B{y^2} + Cx + Dy + E = 0 $ | (6) |
式中,
| $ \begin{array}{*{20}{c}} {\min F:F\left( {A,B,C,D,E} \right) = }\\ {\sum\limits_{i = 1}^n {{{\left( {x_i^2 + A{x_i}{y_i} + By_i^2 + C{x_i} + D{y_i} + E} \right)}^2}} } \end{array} $ | (7) |
可用最小二乘求解方程系数
| $ \left\{ \begin{array}{l} {x_0} = \frac{{2BC - AD}}{{{A^2} - 4B}},{y_0} = \frac{{2D - AC}}{{{A^2} - 4B}}\\ \theta = {\tan ^{ - 1}}\sqrt {\frac{{{a^2} - {b^2}B}}{{{a^2}B - {b^2}}}} \\ a = \sqrt {\frac{{2\left( {ACD - B{C^2} - {D^2} + 4BE - {A^2}E} \right)}}{{\left( {{A^2} - 4B} \right)\left( {B + 1 - \sqrt {{A^2} + {{\left( {1 - B} \right)}^2}} } \right)}}} = \\ \;\;\;\;\;\;\sqrt {\frac{{2\left( {x_0^2 + By_0^2 + A{x_0}{y_0} - E} \right)}}{{B + 1 - \sqrt {{A^2} + {{\left( {1 - B} \right)}^2}} }}} \\ b = \sqrt {\frac{{2\left( {ACD - B{C^2} - {D^2} + 4BE - {A^2}E} \right)}}{{\left( {{A^2} - 4B} \right)\left( {B + 1 + \sqrt {{A^2} + {{\left( {1 - B} \right)}^2}} } \right)}}} = \\ \;\;\;\;\;\;\sqrt {\frac{{2\left( {x_0^2 + By_0^2 + A{x_0}{y_0} - E} \right)}}{{B + 1 + \sqrt {{A^2} + {{\left( {1 - B} \right)}^2}} }}} \end{array} \right. $ | (8) |
如图 1(b)所示，球面透视投影下的空间直线
| $ \left\{ \begin{array}{l} {x^2} + {y^2} + {z^2} = {R^2}\\ Px + Qy + Sz = 0 \end{array} \right. $ | (9) |
式中，
| $ \begin{array}{*{20}{c}} {\frac{{\left( {1 + {P^2}/{S^2}} \right)}}{{{R^2}}}{x^2} + \frac{{\left( {2P \cdot Q/{S^2}} \right)}}{{{R^2}}}xy + }\\ {\frac{{\left( {1 + {Q^2}/{S^2}} \right)}}{{{R^2}}}{y^2} - 1 = 0} \end{array} $ | (10) |
式(10)可视为球面大圆在水平面的投影，若令
| $ {L_1} = \frac{{\left( {1 + {P^2}/{S^2}} \right)}}{{{R^2}}},{L_2} = \frac{{\left( {2P \cdot Q/{S^2}} \right)}}{{{R^2}}},{L_3} = \frac{{\left( {1 + {Q^2}/{S^2}} \right)}}{{{R^2}}} $ |
将式(10)左右两边同除
| $ {x^2} + \frac{{{L_2}}}{{{L_1}}}xy + \frac{{{L_3}}}{{{L_1}}}{y^2} + Cx + Dy - \frac{1}{{{L_1}}} = 0 $ | (11) |
则可导出该椭圆中心(
| $ \left\{ \begin{array}{l} {a^2} = \frac{{ - 2E}}{{B + 1 - \sqrt {{A^2} + {{\left( {1 - B} \right)}^2}} }} = \\ \;\;\;\;\;\;\;\frac{{ - 2\left( {1/{L_1}} \right)}}{{\left( {{L_3}/{L_1}} \right) + 1 - \sqrt {{{\left( {{L_2}/{L_1}} \right)}^2} + {{\left( {1 - {L_3}/{L_1}} \right)}^2}} }} = \\ \;\;\;\;\;\;\;\frac{2}{{{L_3} + {L_1} - \sqrt {L_2^2 + {{\left( {{L_1} - {L_3}} \right)}^2}} }} = {R^2}\\ {b^2} = \frac{{ - 2E}}{{B + 1 + \sqrt {{A^2} + {{\left( {1 - B} \right)}^2}} }} = \\ \;\;\;\;\;\;\;\frac{{ - 2\left( {1/{L_1}} \right)}}{{\left( {{L_3}/{L_1}} \right) + 1 + \sqrt {{{\left( {{L_2}/{L_1}} \right)}^2} + {{\left( {1 - {L_3}/{L_1}} \right)}^2}} }} = \\ \;\;\;\;\;\;\;\frac{2}{{{L_3} + {L_1} + \sqrt {L_2^2 + {{\left( {{L_1} - {L_3}} \right)}^2}} }} = \frac{{{R^2}}}{{1 + \left( {{P^2} + {Q^2}} \right)/{S^2}}} \end{array} \right. $ | (12) |
式(12)表明:1)不同空间直线在球面透视投影下水平面投影椭圆，其椭圆中心为投影球心，长半轴
| $ \begin{array}{l} \sqrt {{{\left( {{x_m} + c \times \cos \theta } \right)}^2} + {{\left( {{y_m} + c \times \sin \theta } \right)}^2}} + \\ \sqrt {{{\left( {{x_m} - c \times \cos \theta } \right)}^2} + {{\left( {{y_m} - c \times \sin \theta } \right)}^2}} - 2 = 0 \end{array} $ | (13) |
将式(13)左右同乘以等效焦距
| $ \begin{array}{*{20}{c}} {\sqrt {{{\left( {f \times {x_m} + c \times \cos \theta } \right)}^2} + {{\left( {f \times {y_m} + c \times \sin \theta } \right)}^2}} + }\\ {\sqrt {{{\left( {f \times {x_m} - c \times \cos \theta } \right)}^2} + {{\left( {f \times {y_m} - c \times \sin \theta } \right)}^2}} - }\\ {2f = 0} \end{array} $ | (14) |
式中，
| $ \begin{array}{*{20}{c}} {\sqrt {\begin{array}{l} {\left( {\left( {u - {u_0}} \right) \times \left( {1 + {k_1}{r^2} + {k_2}{r^4}} \right)/A + c \times \cos \theta } \right)^2} + \\ {\left( {\left( {v - {v_0}} \right) \times \left( {1 + {k_1}{r^2} + {k_2}{r^4}} \right) + c \times \sin \theta } \right)^2} \end{array} }}\\ {\sqrt {\begin{array}{l} {\left( {\left( {u - {u_0}} \right) \times \left( {1 + {k_1}{r^2} + {k_2}{r^4}} \right)/A - c \times \cos \theta } \right){^2}} - \\ {\left( {\left( {v - {v_0}} \right) \times \left( {1 + {k_1}{r^2} + {k_2}{r^4}} \right) - c \times \sin \theta } \right)^2} \end{array} }}\\ {2f = 0} \end{array} $ | (15) |
式中，(
| $ {E_r}\left( {{k_1},{k_2},A,f,b,\theta } \right) = \sqrt {{A_t}} + \sqrt {{B_t}} - 2f = 0 $ | (16) |
| $ \left\{ \begin{array}{l} {A_t} = A_1^2 + A_2^2,Bt = B_1^2 + B_2^2\\ {A_1} = \left( {u - {u_0}} \right) \times \left( {1 + {k_1}{r^2} + {k_2}{r^4}} \right)/A + c \times \cos \theta \\ {A_2} = \left( {v - {v_0}} \right) \times \left( {1 + {k_1}{r^2} + {k_2}{r^4}} \right) + c \times \sin \theta \\ {B_1} = \left( {u - {u_0}} \right) \times \left( {1 + {k_1}{r^2} + {k_2}{r^4}} \right)/A - c \times \cos \theta \\ {B_2} = \left( {v - {v_0}} \right) \times \left( {1 + {k_1}{r^2} + {k_2}{r^4}} \right) - c \times \sin \theta \\ c = \sqrt {{f^2} - {b^2}} \\ {r^2} = {\left( {u - {u_0}} \right)^2} + {\left( {v - {v_0}} \right)^2} \end{array} \right. $ |
式(16)为非线性方程，按泰勒级数展开并取一次项，得误差方程为
| $ \begin{array}{*{20}{c}} {V = {E_r}\left( {k_1^0,k_2^0,{A^0},{f^0},{b^0},{\theta ^0}} \right) + }\\ {\frac{{\partial {E_r}}}{{\partial {k_1}}} + \frac{{\partial {E_r}}}{{\partial {k_2}}} + \frac{{\partial {E_r}}}{{\partial A}} + \frac{{\partial {E_r}}}{{\partial f}} + \frac{{\partial {E_r}}}{{\partial b}} + \frac{{\partial {E_r}}}{{\partial \theta }}} \end{array} $ | (17) |
| $ \left\{ \begin{array}{l} \frac{{\partial {E_r}}}{{\partial {k_1}}} = \frac{{\left( {u - {u_0}} \right){r^2}}}{A}\left( {\frac{{{A_1}}}{{\sqrt {At} }} + \frac{{{B_1}}}{{\sqrt {Bt} }}} \right) + \\ \;\;\;\;\;\;\;\;\;{r^2}\left( {v - {v_0}} \right)\left( {\frac{{{A_2}}}{{\sqrt {At} }} + \frac{{{B_2}}}{{\sqrt {Bt} }}} \right)\\ \frac{{\partial {E_r}}}{{\partial {k_2}}} = \frac{{\left( {u - {u_0}} \right){r^4}}}{A}\left( {\frac{{{A_1}}}{{\sqrt {At} }} + \frac{{{B_1}}}{{\sqrt {Bt} }}} \right) + \\ \;\;\;\;\;\;\;\;\;{r^2}\left( {v - {v_0}} \right)\left( {\frac{{{A_2}}}{{\sqrt {At} }} + \frac{{{B_2}}}{{\sqrt {Bt} }}} \right)\\ \frac{{\partial {E_r}}}{{\partial A}} = \frac{{u + \left( {u - {u_0}} \right)\left( {{k_1}{r^2} + {k_2}{r^4}} \right)}}{{ - {A^2}}}\left( {\frac{{{A_1}}}{{\sqrt {At} }} + \frac{{{B_1}}}{{\sqrt {Bt} }}} \right)\\ \frac{{\partial {E_r}}}{{\partial f}} = \frac{f}{{c\sqrt {At} }}\left( {{A_1}\cos \theta + {A_2}\sin \theta } \right) - \\ \;\;\;\;\;\;\;\;\;\frac{f}{{c\sqrt {Bt} }}\left( {{B_1}\cos \theta + {B_2}\sin \theta } \right) - 2\\ \frac{{\partial {E_r}}}{{\partial b}} = \frac{f}{{c\sqrt {Bt} }}\left( {{B_1}\cos \theta + {B_2}\sin \theta } \right) - \\ \;\;\;\;\;\;\;\;\;\frac{f}{{c\sqrt {At} }}\left( {{A_1}\cos \theta + {A_2}\sin \theta } \right)\\ \frac{{\partial {E_r}}}{{\partial \theta }} = \frac{c}{{\sqrt {At} }}\left( {{A_2}\cos \theta - {A_1}\sin \theta } \right) + \\ \;\;\;\;\;\;\;\;\;\frac{c}{{\sqrt {Bt} }}\left( {{B_1}\sin \theta - {B_2}\cos \theta } \right) \end{array} \right. $ |
| $ \begin{array}{*{20}{c}} {{E_r}\left( {k_1^0,k_2^0,{A^0},{f^0},{b^0},{\theta ^0}} \right) = \sqrt {{{\left( {A_1^0} \right)}^2} + {{\left( {A_2^0} \right)}^2}} + }\\ {\sqrt {{{\left( {B_1^0} \right)}^2} + {{\left( {B_2^0} \right)}^2}} - 2{f^0}} \end{array} $ |
| $ A_1^0 = \frac{{\left( {u - {u_0}} \right)\left( {1 + k_1^0{r^2} + k_2^0{r^4}} \right)}}{{{A^0}}} + {c^0}\cos {\theta ^0} $ |
| $ A_2^0 = \left( {v - {v_0}} \right)\left( {1 + k_1^0{r^2} + k_2^0{r^4}} \right) + {c^0}\sin {\theta ^0} $ |
| $ B_1^0 = \frac{{\left( {u - {u_0}} \right)\left( {1 + k_1^0{r^2} + k_2^0{r^4}} \right)}}{{{A^0}}} - {c^0}\cos {\theta ^0} $ |
| $ B_2^0 = \left( {v - {v_0}} \right)\left( {1 + k_1^0{r^2} + k_2^0{r^4}} \right) - {c^0}\sin {\theta ^0} $ |
| $ {c^0} = \sqrt {{{\left( {{f^0}} \right)}^2} - {{\left( {{b^0}} \right)}^2}} $ |
令
3 实验分析
本文在英特尔E5-1620处理器，win10操作系统，VS2010编译环境PC机下实现上述算法。算法验证采用两种方式：基于平面参照物的多视标定和基于自然场景的单视自标定。多视标定选用海康威视定焦鱼眼相机DS-2CD2942F-I，见图 4(a)，该相机视野满足180°，传感器尺寸1/1.8英寸，影像分辨率1 280×1 280像素。现有鱼眼镜头标定方法在几何成像过程、光学畸变模型选取及参数数量上存在差异，本文利用标定参数对鱼眼图像进行立方盒展开(平面透视纠正)，以纠正图像上的直线拟合精度作为算法性能评价依据，并与文献[4]方法结果进行对比，该文献方法实现由其网上发布的标定工具箱[26]给出，通过最小化标定影像上的棋盘格角点重投影误差获得相机内、外部参数，这里算法所需空间直线统一由绘制在LCD(liquid crystal display)上的棋盘格给出(格网大小为
考虑到模型估计稳健性及距影像中心远的区域受光学畸变影响大，本文多视标定步骤如下：
1) 以中间影像为对象，首先提取影像中的棋盘格角点并利用圆锥曲线方程拟合单个投影椭圆，再以棋盘格角点为观测值，根据第2节建立误差方程并设定参数初值进行最小二乘估计，获得鱼眼镜头内部参数值(
2) 逐个提取边缘影像中的棋盘格角点并利用圆锥曲线方程拟合单个投影椭圆，因径向畸变参数
3) 以步骤1)中求出的鱼眼镜头内部参数值为初值，以步骤1)和步骤2)中求出的(
由图 5可以看出，约18次迭代计算即可收敛，收敛时的均方根误差(RMSE)稳定在0.12像素，表明具有很高的模型估计精度。表 1列出了本文方法得到鱼眼镜头内部参数值, 其中, (
表 1
多视标定参数值对比
Table 1
Comparison of fish-eye cameras parameters through multi-view calibration process
| ( |
( |
( |
RMSE | |
| 本文 | (443.38, 454.75) | (640.0, 640.0) | (-7.71E-7, 1.898 E-13, -, -) | 0.12 |
| 工具箱 | (449.37, 471.40) | (638.9, 646.9) | (0.000 44, -0.055 85, 0.035 82, -0.009 33) | 0.19 |
表 2
多视标定鱼眼影像纠正直线拟合误差RMSE对比
Table 2
Comparison of linear fitting error RMSE in corrected fisheye image with multi-view calibration parameters
| 待纠正影像 | ||||||
| 中 | 左 | 右 | 顶 | 底 | 平均 | |
| 本文 | 0.136 | 0.186 | 0.196 | 0.18 | 0.212 | 0.183 |
| 工具箱 | 0.288 | 0.246 | 0.172 | 0.101 | 0.156 | 0.223 |
本文棋盘格由两组正交平行直线组成，图 6给出了标定前后中间影像上的棋盘格投影椭圆参数变化及绘制结果。因棋盘格中间2条横线、1条竖线的投影椭圆过于扁平、几何条件相对较差，文中建立标定误差方程时不予考虑，故此时单张标定影像中的椭圆数量为22-3 = 19个。由图 6(a)可明显看出，标定前棋盘格直线投影椭圆的长半轴并不相等，其原因在于鱼眼成像过程中的像素变换阶段存在径向畸变且相机纵横比不等于1，而标定后的投影椭圆长半轴则完全遵循了空间直线水平面投影椭圆约束，在估计出(
针对同一组多视标定影像，本文在表 1和表 2中给出了工具箱方法计算得到的文献方法标定参数及其立方盒展开后的棋盘格角点直线拟合误差。由表 1和表 2可以看出，本文方法和工具箱方法在纠正影像上的直线拟合误差相接近，总体精度上本文方法略优。图 7给出了两种方法参数对标定鱼眼图像的立方盒展开(约170°视角)结果，两者视觉质量几乎一致，包括棋盘格在内、不同区域中的直线特征均具有较好的线性保持效果，但文献方法在模型表征、估计计算方面更为复杂，不仅额外增加了两个径向畸变参数
单视自标定以网上具有丰富直线特征的鱼眼图像为对象，见图 8的第1行。因拍摄相机无法获得，基于平面棋盘格参照物的标定方式已无法使用，但本文方法仍可通过Canny边缘检测、随机霍夫椭圆提取[27]操作获得单张鱼眼图像上的椭圆弧, 见图 8的第2行, 进而迭代计算出鱼眼镜头内部参数，见图 9，其参数初值选取与多视标定计算相同，两者迭代计算收敛速度也相接近，表 3列出了最终标定参数。结合图 8和表 3可以看出，本文方法单视标定参数估计精度与提取椭圆(弧)在鱼眼图像上的空间分布有关，图像边缘处直线特征给出的水平面投影椭圆约束对模型估计精度改善形成重要作用，如图像
表 3
本文方法单视自标定参数值
Table 3
Single-view calibration parameters and its re-projection error RMSE in this paper
| 图像 | ( |
( |
( |
RMSE |
| (261.90, 260.08) | (463.0, 463.0) | (-2.993E-6, 4.503 3E-12) | 0.292 | |
| (579.99, 583.15) | (960.0, 960.0) | (-5.609E-7, 1.442 8E-13) | 0.318 | |
| (379.95, 370.26) | (645.0, 641.0) | (-1.729E-6, 1.914 9E-12) | 0.542 |
本文研究结果表明:
1) 基于空间直线在球面透视投影下的水平面理想投影椭圆约束及椭圆内在严格几何特性，建立标定方程对包括等效焦距
2) 本文方法标定参数对鱼眼图像不同区域的平面透视纠正效果总体稳健、精度高，多视标定参数误差RMSE约0.1像素，纠正影像上直线拟合误差RMSE约0.2像素，略优于对比文献方法；单视标定参数误差RMSE约0.3像素，纠正影像范围、直线透视特性保持均优于对比文献方法及商业软件DXO。
3) 本文方法适用于鱼眼相机视野内任一空间直线并与直线方向、位置、长度无关，对标定参照物要求不高，对于具有大量平行直线的人工场景理论上可实现自标定，具有较好的应用价值。
4 结论
鱼眼镜头根据非相似成像原理设计，在获得大视角的同时也会发生不符合人眼视觉习惯的严重成像畸变。本文充分利用空间直线在球面透视投影下的水平面理想投影椭圆约束及椭圆内在几何特性，建立严格标定方程直接求解鱼眼镜头内部参数，标定过程简单、通用性好、鱼眼图像纠正精度高，具有较好的应用前景和价值。为简化标定模型的复杂性，本文假定相机主点位于图像中心，但在给定主点初值条件下，本文给出的标定方程经简单拓展后理论上同样适用，下一阶段将结合不同类型、视野鱼眼镜头及鱼眼图像视觉测量、3维重建等实际任务进一步完善本文方法并开展相关研究工作。
参考文献
-
[1] Feng W J, Zhang B F, Cao Z L. Omni-directional vision parameter calibration and rectification based on fish-eye lens[J]. Journal of Tianjin University, 2011, 44(5): 417–424. [冯为嘉, 张宝峰, 曹作良. 基于鱼眼镜头的全方位视觉参数标定与畸变矫正[J]. 天津大学学报, 2011, 44(5): 417–424. ] [DOI:10.3969/j.issn.0493-2137.2011.05.008]
-
[2] Zhang H B, Yu Y, Li L, et al. Scene roaming method of fisheye image based on viewpoint correction[J]. Journal of Graphics, 2014, 35(3): 435–441. [张海彬, 余烨, 李琳, 等. 基于视点纠正的鱼眼图像场景化漫游方法[J]. 图学学报, 2014, 35(3): 435–441. ] [DOI:10.3969/j.issn.2095-302X.2014.03.018]
-
[3] Li H B, Chu G Y, Zhang Q, et al. Space point positioning based on optimization of fisheye lens imaging model[J]. Acta Optica Sinica, 2015, 35(7): 0715003. [李海滨, 褚光宇, 张强, 等. 基于优化的鱼眼镜头成像模型的空间点定位[J]. 光学学报, 2015, 35(7): 0715003. ] [DOI:10.3788/aos201535.0715003]
-
[4] Kannala J, Brandt S S. A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(8): 1335–1340. [DOI:10.1109/TPAMI.2006.153]
-
[5] Schneider D, Schwalbe E, Maas H G. Validation of geometric models for fisheye lenses[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2009, 64(3): 259–266. [DOI:10.1016/j.isprsjprs.2009.01.001]
-
[6] Wei J, Li C F, Hu S M, et al. Fisheye video correction[J]. IEEE Transactions on Visualization and Computer Graphics, 2012, 18(10): 1771–1783. [DOI:10.1109/TVCG.2011.130]
-
[7] Wang Y Z. Fish-eye Lens Optics[M]. Beijing: Science Press, 2006: 5-15. [ 王永仲. 鱼眼镜头光学[M]. 北京: 科学出版社, 2006: 5-15.]
-
[8] Huang Y D, Su H M. A simple transforming model from fisheye image to perspective projection image[J]. Journal of System Simulation, 2005, 17(1): 29–32, 52. [黄有度, 苏化明. 一种鱼眼图象到透视投影图象的变换模型[J]. 系统仿真学报, 2005, 17(1): 29–32, 52. ] [DOI:10.3969/j.issn.1004-731X.2005.01.007]
-
[9] Huang F Y, Wang Y Z, Shen X J, et al. Method for calibrating the fisheye distortion center[J]. Applied Optics, 2012, 51(34): 8169–8176. [DOI:10.1364/AO.51.008169]
-
[10] Lin Y, Gong X J, Liu J L. Calibration of fisheye cameras based on the viewing sphere[J]. Journal of Zhejiang University:Engineering Science, 2013, 47(8): 1500–1507. [林颖, 龚小谨, 刘济林. 基于单位视球的鱼眼相机标定方法[J]. 浙江大学学报:工学版, 2013, 47(8): 1500–1507. ]
-
[11] Wu Z J, Wu Q Y, Zhang B C. A new calibration method for fisheye lens based on spherical model[J]. Chinese Journal of Lasers, 2015, 42(5): 0508006. [吴泽俊, 吴庆阳, 张佰春. 一种新的基于球面模型的鱼眼镜头标定方法[J]. 中国激光, 2015, 42(5): 0508006. ] [DOI:10.3788/cjl201542.0508006]
-
[12] Pi Y D, Li X, Chen Z Y, et al. Calibration and rectification of fisheye images based on three-dimensional control field[J]. Acta Optica Sinica, 2017, 37(1): 0115001. [皮英冬, 李欣, 陈智勇, 等. 基于三维控制场的鱼眼影像检校和纠正方法[J]. 光学学报, 2017, 37(1): 0115001. ] [DOI:10.3788/aos201737.0115001]
-
[13] Kanatani K. Calibration of ultrawide fisheye lens cameras by eigenvalue minimization[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(4): 813–822. [DOI:10.1109/TPAMI.2012.146]
-
[14] Zhang M, Yao J, Xia M H, et al. Line-based multi-label energy optimization for fisheye image rectification and calibration[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA: IEEE, 2015: 4137-4145.[DOI: 10.1109/CVPR.2015.7299041]
-
[15] Aghayari S, Saadatseresht M, Omidalizarandi M, et al. Geometric calibration of full spherical panoramic Ricoh-theta camera[J]. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2017, 4-1/W1: 237–245. [DOI:10.5194/isprs-annals-Ⅳ-1-W1-237-2017]
-
[16] Ying X H, Hu Z Y. Fisheye lense distortion correction using spherical perspective projection constraint[J]. Chinese Journal of Computers, 2003, 26(12): 1702–1708. [英向华, 胡占义. 一种基于球面透视投影约束的鱼眼镜头校正方法[J]. 计算机学报, 2003, 26(12): 1702–1708. ] [DOI:10.3321/j.issn:0254-4164.2003.12.013]
-
[17] Zheng L P, Xu G Q, Li L, et al. Imaging modeling and correction of nonlinear distortion distribution ellipse fish-eye lens[J]. Chinese Journal of Scientific Instrument, 2012, 33(6): 1331–1337. [郑利平, 徐刚强, 李琳, 等. 非线性畸变分布椭圆鱼眼镜头成像建模和校正[J]. 仪器仪表学报, 2012, 33(6): 1331–1337. ] [DOI:10.3969/j.issn.0254-3087.2012.06.019]
-
[18] Liao S Z, Gao P H, Su Y, et al. A geometric rectification method for lens camera[J]. Journal of Image and Graphics, 2000, 5(7): 593–596. [廖士中, 高培焕, 苏艺, 等. 一种光学镜头摄像机图像几何畸变的修正方法[J]. 中国图像图形学报, 2000, 5(7): 593–596. ] [DOI:10.11834/jig.20000711]
-
[19] Yang L, Cheng Y. The designing methods of fish-eye distortion correction using latitude-longitude projection[J]. Journal of Engineering Graphics, 2010, 31(6): 19–22. [杨玲, 成运. 应用经纬映射的鱼眼图像校正设计方法[J]. 工程图学学报, 2010, 31(6): 19–22. ] [DOI:10.3969/j.issn.1003-0158.2010.06.004]
-
[20] Wei L S, Zhou S W, Zhang P G, et al. Double longitude model based correction method for fish-eye image distortion[J]. Chinese Journal of Scientific Instrument, 2015, 36(2): 377–385. [魏利胜, 周圣文, 张平改, 等. 基于双经度模型的鱼眼图像畸变矫正方法[J]. 仪器仪表学报, 2015, 36(2): 377–385. ] [DOI:10.19650/j.cnki.cjsi.2015.02.016]
-
[21] Xu B, Liu L, Liu Y H, et al. A calibration method for fish-eye cameras based on pinhole model[J]. Acta Automatica Sinica, 2014, 40(4): 653–659. [涂波, 刘璐, 刘一会, 等. 一种扩展小孔成像模型的鱼眼相机矫正与标定方法[J]. 自动化学报, 2014, 40(4): 653–659. ] [DOI:10.3724/SP.J.1004.2014.00653]
-
[22] Jia Y D, Lü H J, Xu A, et al. Fish-eye lens camera calibration for stereo vision system[J]. Chinese Journal of Computers, 2000, 23(11): 1215–1219. [贾云得, 吕宏静, 徐岸, 等. 一种鱼眼镜头成像立体视觉系统的标定方法[J]. 计算机学报, 2000, 23(11): 1215–1219. ] [DOI:10.3321/j.issn:0254-4164.2000.11.016]
-
[23] Ramalingam S, Sturm P. A unifying model for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(7): 1309–1319. [DOI:10.1109/TPAMI.2016.2592904]
-
[24] Hughes C, Denny P, Jones E, et al. Accuracy of fish-eye lens models[J]. Applied Optics, 2010, 49(17): 3338–3347. [DOI:10.1364/AO.49.003338]
-
[25] Ray A, Srivastava D C. Non-linear least squares ellipse fitting using the genetic algorithm with applications to strain analysis[J]. Journal of Structural Geology, 2008, 30(12): 1593–1602. [DOI:10.1016/j.jsg.2008.09.003]
-
[26] Bouguet J Y. Camera calibration toolbox for Matlab[EB/OL].[2019-02-28]2015-08-14. http://www.vision.caltech.edu/bouguetj/calib_doc.
-
[27] Lu W, Tan J L. Detection of incomplete ellipse in images with strong noise by iterative randomized Hough transform (IRHT)[J]. Pattern Recognition, 2008, 41(4): 1268–1279. [DOI:10.1016/j.patcog.2007.09.006]
