发布时间: 2019-11-16 摘要点击次数: 全文下载次数: DOI: 10.11834/jig.190061 2019 | Volume 24 | Number 11 图像理解和计算机视觉

 收稿日期: 2019-03-13; 修回日期: 2019-04-30; 预印本日期: 2019-05-07 基金项目: 国家自然科学基金项目（41761087）；广西自然科学基金项目（2015GXNSFDA139030，2017GXNSFAA198162）；桂林电子科技大学研究生教育创新计划资助项目（2018YJCXB62） 第一作者简介: 黄明益, 1989年生, 男, 博士研究生, 主要研究方向为摄像机标定、影像地理配准、视频融合。E-mail:1240409615@qq.com. 中图法分类号: TP391 文献标识码: A 文章编号: 1006-8961(2019)11-1972-13

# 关键词

Fish-eye lens calibration by ideal projection ellipse constraints
Huang Mingyi, Wu Jun
School of Electronic Engineering and Automation, Guilin University of Electronic Technology, Guilin 541004, China
Supported by: National Natural Science Foundation of China (41761087)

# Abstract

Objective Fish-eye lens is an ideal optical sensor to develop light and small omnidirectional vision systems. Due to the characteristics of large field of view and low cost, it is widely used in various places such as security monitoring. Based on a large number of original fisheye video materials, it has the potential for in-depth research and full exploitation. Therefore, the calibration of fisheye cameras needs to obtain the internal parameters of the images. How to improve the indoor and outdoor aspects of the city is imperative. The calibration efficiency of model fisheye camera is a valuable and challenging research work. However, due to the limitation of short focal length, large field of view and special optical principle, fish-eye images will produce serious barrel distortion, which is not conducive to the subsequent development and application of video images. Constraints of optical principle it is expected to be transformed into plane perspective projection, which conforms to human visual habits, by a set of high-precision parameters associated with the optical imaging model of the fish-eye lens. To this end, a high-precision and flexible method for calibrating the internal parameters of fish-eye lens is proposed in this study. Method The calibration is achieved through the following steps: Firstly, we need to obtain the initial internal parameters of fisheye image. According to the principle that the spatial line is imaged as an elliptic curve on the image, we extract the image elliptic curve. The specific methods are as follows: a) obtain the coordinates of the curve points on the image by image segmentation, b) obtain the general curve equation of the ellipse by curve fitting, than decompose the general equation into the ellipse long and short axis length and image principal point used as initial value. Secondly, ideal projection ellipse constraints (IPECs) for any space line on the horizontal plane under spherical perspective projection are mathematically set. The constraints are as follows: a) the half length of the long axis of projection ellipse of different space straight lines is constantly equal to the radius of the projection sphere; and b) the length ratio of long axis to short axis of the projection ellipse is constant for any one space line when the radius of the projection sphere is changed. Thirdly, a nonlinear function is built on the basis of the proposed IPECs and the strict geometric properties of ellipses to conduct an iterative least square estimation for the uncalibrated fish-eye lens parameters, namely, focal length f, aspect ratio $A$, and distortion parameters $k$1, $k$2. Finally, the distorted fish-eye images are corrected by using the estimated lens parameters and cube-box expansion. Result One focus-fixed fish-eye camera is selected to test the proposed approach under multiple-view condition. In addition, several parameter-free fish-eye images downloaded from the Internet are selected to test the proposed approach under single-view condition. Experimental results show that stable and high-quality correction is achieved in different areas of fish-eye images by using the estimated calibration parameters. The root-mean-square error (RMSE) in multiple-view calibration for the selected fish-eye camera is approximately 0.1 pixel, and the straight-line fitness RMSE in the corrected fish-eye image is only approximately 0.2 pixel. These results are slightly better than the results produced by an online calibration toolbox. Compared with our method in which only a small number of lens internal parameters needed to be solved directly, the online calibration toolbox is more complex in model characterization and estimation calculation, in which two additional radial distortion parameters $k$3 and $k$4 are added to characterize the internal parameters of fish-eye lens. The external parameters of the camera are also required for simultaneous estimation. Although our method uses the straight line features on a chessboard, no requirement is set for its spatial (physical) accuracy (position and direction). By contrast, the online calibration toolbox depends fundamentally on the interposition accuracy of a chessboard's corner points; multiple photographs of a small-sized chessboard at a specific angle are often required for ideal control conditions because photographs of a large-sized chessboard with high accuracy are difficult to obtain. The single-view calibration RMSE is approximately 0.3 pixel, and its straight-line geometry preservation on corrected fish-eye images is obviously better than the results produced by popular commercial software DXO toolbox. Conclusion The proposed calibration can be realized with few calibration parameters and a simple calculation that allows it to be implemented via self-calibration for artificial scenes with a large number of lines. This characteristic makes the calibration useful in applications such as panorama surveillance, 3D reconstruction, and robot navigation.

# Key words

fish-eye lens calibration; fish-eye camera calibration; fish-eye image correction; spherical perspective projection; ideal projection ellipse constraint

# 1 鱼眼成像模型

1) 空间坐标变换，即将世界坐标系下任一空间点$\mathit{\boldsymbol{P}}_{\rm{W}}$=[$X_{\rm{W}}$, $Y_{\rm{W}}$, $Z_{\rm{W}}$]T，经旋转和平移空间坐标变换，转换为相机坐标系下的点$\mathit{\boldsymbol{P}}_{\rm{C}}$=[$X_{\rm{C}}$, $Y_{\rm{C}}$, $Z_{\rm{C}}$]T，两者满足以下关系

 ${\mathit{\boldsymbol{P}}_{\rm{C}}} = \mathit{\boldsymbol{R}}{\mathit{\boldsymbol{P}}_{\rm{W}}} + \mathit{\boldsymbol{T}}$ (1)

2) 单位球面映射，即将点$\mathit{\boldsymbol{P}}_{\rm{C}}$沿射线$O_{\rm{C}}$$P_{\rm{C}}$方向映射为单位球面上的点$\mathit{\boldsymbol{P}}_{\rm{S}}$($X_{\rm{S}}$, $Y_{\rm{S}}$, $Z_{\rm{S}}$), 即

 $\begin{array}{*{20}{c}} {{X_{\rm{S}}} = \frac{{{X_{\rm{C}}}}}{{\sqrt {X_{\rm{C}}^2 + Y_{\rm{C}}^2 + Z_{\rm{C}}^2} }},{Y_{\rm{S}}} = \frac{{{Y_{\rm{C}}}}}{{\sqrt {X_{\rm{C}}^2 + Y_{\rm{C}}^2 + Z_{\rm{C}}^2} }},}\\ {{Z_{\rm{S}}} = \frac{{{Z_{\rm{C}}}}}{{\sqrt {X_{\rm{C}}^2 + Y_{\rm{C}}^2 + Z_{\rm{C}}^2} }}} \end{array}$ (2)

3) 球面投影，即按选定模型将点$P_{\rm{S}}$投影至鱼眼图像平面。现有的球面投影模型分为4种[24]：等距、等立体角、体视、正交。其中，正交投影模型计算简单且可建立空间点与鱼眼图像点的可逆变换关系，本文选用该投影模型，如图 3所示，将点$\mathit{\boldsymbol{P}}_{\rm{S}}$正投影到一个与$Z$轴(鱼眼镜头主光轴)垂直的固定平面(像平面)，从而获得投影点$\mathit{\boldsymbol{P}}_m$($x_m$, $y_m$)，两者坐标变换关系如下

 ${x_m} = {X_{\rm{S}}},{y_m} = {Y_{\rm{S}}}$ (3)

4) 像素坐标变换，即利用相机内部参数将理想投影点坐标变换到鱼眼图像像素坐标，相机内部参数通常表示为矩阵$\boldsymbol{k}=\left[\begin{array}{ccc}{A f} & {0} & {u_{0}} \\ {0} & {f} & {v_{0}} \\ {0} & {0} & {1}\end{array}\right]$，考虑到鱼眼镜头光学畸变并主要受径向畸变误差影响，则从理想投影点到像素的坐标变换关系为

 $\left\{ {\begin{array}{*{20}{l}} {\left( {u - {u_0}} \right) \times \left( {1 + {k_1}{r^2} + {k_2}{r^4}} \right) = Af \times {x_n}}\\ {\left( {v - {v_0}} \right) \times \left( {1 + {k_1}{r^2} + {k_2}{r^4}} \right) = f \times {y_m}}\\ {{r^2} = {{\left( {u - {u_0}} \right)}^2} + {{\left( {v - {v_0}} \right)}^2}} \end{array}} \right.$ (4)

# 2 水平面理想投影椭圆约束

 $\begin{array}{*{20}{c}} {\sqrt {{{\left( {x - {x_0} + c \times \cos \theta } \right)}^2} + {{\left( {y - {y_0} + c \times \sin \theta } \right)}^2}} + }\\ {\sqrt {{{\left( {x - {x_0} - c \times \cos \theta } \right)}^2} + {{\left( {y - {y_0} - c \times \sin \theta } \right)}^2}} - }\\ {2a = 0} \end{array}$ (5)

 ${x^2} + Axy + B{y^2} + Cx + Dy + E = 0$ (6)

 $\begin{array}{*{20}{c}} {\min F:F\left( {A,B,C,D,E} \right) = }\\ {\sum\limits_{i = 1}^n {{{\left( {x_i^2 + A{x_i}{y_i} + By_i^2 + C{x_i} + D{y_i} + E} \right)}^2}} } \end{array}$ (7)

 $\left\{ \begin{array}{l} {x_0} = \frac{{2BC - AD}}{{{A^2} - 4B}},{y_0} = \frac{{2D - AC}}{{{A^2} - 4B}}\\ \theta = {\tan ^{ - 1}}\sqrt {\frac{{{a^2} - {b^2}B}}{{{a^2}B - {b^2}}}} \\ a = \sqrt {\frac{{2\left( {ACD - B{C^2} - {D^2} + 4BE - {A^2}E} \right)}}{{\left( {{A^2} - 4B} \right)\left( {B + 1 - \sqrt {{A^2} + {{\left( {1 - B} \right)}^2}} } \right)}}} = \\ \;\;\;\;\;\;\sqrt {\frac{{2\left( {x_0^2 + By_0^2 + A{x_0}{y_0} - E} \right)}}{{B + 1 - \sqrt {{A^2} + {{\left( {1 - B} \right)}^2}} }}} \\ b = \sqrt {\frac{{2\left( {ACD - B{C^2} - {D^2} + 4BE - {A^2}E} \right)}}{{\left( {{A^2} - 4B} \right)\left( {B + 1 + \sqrt {{A^2} + {{\left( {1 - B} \right)}^2}} } \right)}}} = \\ \;\;\;\;\;\;\sqrt {\frac{{2\left( {x_0^2 + By_0^2 + A{x_0}{y_0} - E} \right)}}{{B + 1 + \sqrt {{A^2} + {{\left( {1 - B} \right)}^2}} }}} \end{array} \right.$ (8)

 $\left\{ \begin{array}{l} {x^2} + {y^2} + {z^2} = {R^2}\\ Px + Qy + Sz = 0 \end{array} \right.$ (9)

 $\begin{array}{*{20}{c}} {\frac{{\left( {1 + {P^2}/{S^2}} \right)}}{{{R^2}}}{x^2} + \frac{{\left( {2P \cdot Q/{S^2}} \right)}}{{{R^2}}}xy + }\\ {\frac{{\left( {1 + {Q^2}/{S^2}} \right)}}{{{R^2}}}{y^2} - 1 = 0} \end{array}$ (10)

 ${L_1} = \frac{{\left( {1 + {P^2}/{S^2}} \right)}}{{{R^2}}},{L_2} = \frac{{\left( {2P \cdot Q/{S^2}} \right)}}{{{R^2}}},{L_3} = \frac{{\left( {1 + {Q^2}/{S^2}} \right)}}{{{R^2}}}$

 ${x^2} + \frac{{{L_2}}}{{{L_1}}}xy + \frac{{{L_3}}}{{{L_1}}}{y^2} + Cx + Dy - \frac{1}{{{L_1}}} = 0$ (11)

 $\left\{ \begin{array}{l} {a^2} = \frac{{ - 2E}}{{B + 1 - \sqrt {{A^2} + {{\left( {1 - B} \right)}^2}} }} = \\ \;\;\;\;\;\;\;\frac{{ - 2\left( {1/{L_1}} \right)}}{{\left( {{L_3}/{L_1}} \right) + 1 - \sqrt {{{\left( {{L_2}/{L_1}} \right)}^2} + {{\left( {1 - {L_3}/{L_1}} \right)}^2}} }} = \\ \;\;\;\;\;\;\;\frac{2}{{{L_3} + {L_1} - \sqrt {L_2^2 + {{\left( {{L_1} - {L_3}} \right)}^2}} }} = {R^2}\\ {b^2} = \frac{{ - 2E}}{{B + 1 + \sqrt {{A^2} + {{\left( {1 - B} \right)}^2}} }} = \\ \;\;\;\;\;\;\;\frac{{ - 2\left( {1/{L_1}} \right)}}{{\left( {{L_3}/{L_1}} \right) + 1 + \sqrt {{{\left( {{L_2}/{L_1}} \right)}^2} + {{\left( {1 - {L_3}/{L_1}} \right)}^2}} }} = \\ \;\;\;\;\;\;\;\frac{2}{{{L_3} + {L_1} + \sqrt {L_2^2 + {{\left( {{L_1} - {L_3}} \right)}^2}} }} = \frac{{{R^2}}}{{1 + \left( {{P^2} + {Q^2}} \right)/{S^2}}} \end{array} \right.$ (12)

 $\begin{array}{l} \sqrt {{{\left( {{x_m} + c \times \cos \theta } \right)}^2} + {{\left( {{y_m} + c \times \sin \theta } \right)}^2}} + \\ \sqrt {{{\left( {{x_m} - c \times \cos \theta } \right)}^2} + {{\left( {{y_m} - c \times \sin \theta } \right)}^2}} - 2 = 0 \end{array}$ (13)

 $\begin{array}{*{20}{c}} {\sqrt {{{\left( {f \times {x_m} + c \times \cos \theta } \right)}^2} + {{\left( {f \times {y_m} + c \times \sin \theta } \right)}^2}} + }\\ {\sqrt {{{\left( {f \times {x_m} - c \times \cos \theta } \right)}^2} + {{\left( {f \times {y_m} - c \times \sin \theta } \right)}^2}} - }\\ {2f = 0} \end{array}$ (14)

 $\begin{array}{*{20}{c}} {\sqrt {\begin{array}{l} {\left( {\left( {u - {u_0}} \right) \times \left( {1 + {k_1}{r^2} + {k_2}{r^4}} \right)/A + c \times \cos \theta } \right)^2} + \\ {\left( {\left( {v - {v_0}} \right) \times \left( {1 + {k_1}{r^2} + {k_2}{r^4}} \right) + c \times \sin \theta } \right)^2} \end{array} }}\\ {\sqrt {\begin{array}{l} {\left( {\left( {u - {u_0}} \right) \times \left( {1 + {k_1}{r^2} + {k_2}{r^4}} \right)/A - c \times \cos \theta } \right){^2}} - \\ {\left( {\left( {v - {v_0}} \right) \times \left( {1 + {k_1}{r^2} + {k_2}{r^4}} \right) - c \times \sin \theta } \right)^2} \end{array} }}\\ {2f = 0} \end{array}$ (15)

 ${E_r}\left( {{k_1},{k_2},A,f,b,\theta } \right) = \sqrt {{A_t}} + \sqrt {{B_t}} - 2f = 0$ (16)

 $\left\{ \begin{array}{l} {A_t} = A_1^2 + A_2^2,Bt = B_1^2 + B_2^2\\ {A_1} = \left( {u - {u_0}} \right) \times \left( {1 + {k_1}{r^2} + {k_2}{r^4}} \right)/A + c \times \cos \theta \\ {A_2} = \left( {v - {v_0}} \right) \times \left( {1 + {k_1}{r^2} + {k_2}{r^4}} \right) + c \times \sin \theta \\ {B_1} = \left( {u - {u_0}} \right) \times \left( {1 + {k_1}{r^2} + {k_2}{r^4}} \right)/A - c \times \cos \theta \\ {B_2} = \left( {v - {v_0}} \right) \times \left( {1 + {k_1}{r^2} + {k_2}{r^4}} \right) - c \times \sin \theta \\ c = \sqrt {{f^2} - {b^2}} \\ {r^2} = {\left( {u - {u_0}} \right)^2} + {\left( {v - {v_0}} \right)^2} \end{array} \right.$

 $\begin{array}{*{20}{c}} {V = {E_r}\left( {k_1^0,k_2^0,{A^0},{f^0},{b^0},{\theta ^0}} \right) + }\\ {\frac{{\partial {E_r}}}{{\partial {k_1}}} + \frac{{\partial {E_r}}}{{\partial {k_2}}} + \frac{{\partial {E_r}}}{{\partial A}} + \frac{{\partial {E_r}}}{{\partial f}} + \frac{{\partial {E_r}}}{{\partial b}} + \frac{{\partial {E_r}}}{{\partial \theta }}} \end{array}$ (17)

 $\left\{ \begin{array}{l} \frac{{\partial {E_r}}}{{\partial {k_1}}} = \frac{{\left( {u - {u_0}} \right){r^2}}}{A}\left( {\frac{{{A_1}}}{{\sqrt {At} }} + \frac{{{B_1}}}{{\sqrt {Bt} }}} \right) + \\ \;\;\;\;\;\;\;\;\;{r^2}\left( {v - {v_0}} \right)\left( {\frac{{{A_2}}}{{\sqrt {At} }} + \frac{{{B_2}}}{{\sqrt {Bt} }}} \right)\\ \frac{{\partial {E_r}}}{{\partial {k_2}}} = \frac{{\left( {u - {u_0}} \right){r^4}}}{A}\left( {\frac{{{A_1}}}{{\sqrt {At} }} + \frac{{{B_1}}}{{\sqrt {Bt} }}} \right) + \\ \;\;\;\;\;\;\;\;\;{r^2}\left( {v - {v_0}} \right)\left( {\frac{{{A_2}}}{{\sqrt {At} }} + \frac{{{B_2}}}{{\sqrt {Bt} }}} \right)\\ \frac{{\partial {E_r}}}{{\partial A}} = \frac{{u + \left( {u - {u_0}} \right)\left( {{k_1}{r^2} + {k_2}{r^4}} \right)}}{{ - {A^2}}}\left( {\frac{{{A_1}}}{{\sqrt {At} }} + \frac{{{B_1}}}{{\sqrt {Bt} }}} \right)\\ \frac{{\partial {E_r}}}{{\partial f}} = \frac{f}{{c\sqrt {At} }}\left( {{A_1}\cos \theta + {A_2}\sin \theta } \right) - \\ \;\;\;\;\;\;\;\;\;\frac{f}{{c\sqrt {Bt} }}\left( {{B_1}\cos \theta + {B_2}\sin \theta } \right) - 2\\ \frac{{\partial {E_r}}}{{\partial b}} = \frac{f}{{c\sqrt {Bt} }}\left( {{B_1}\cos \theta + {B_2}\sin \theta } \right) - \\ \;\;\;\;\;\;\;\;\;\frac{f}{{c\sqrt {At} }}\left( {{A_1}\cos \theta + {A_2}\sin \theta } \right)\\ \frac{{\partial {E_r}}}{{\partial \theta }} = \frac{c}{{\sqrt {At} }}\left( {{A_2}\cos \theta - {A_1}\sin \theta } \right) + \\ \;\;\;\;\;\;\;\;\;\frac{c}{{\sqrt {Bt} }}\left( {{B_1}\sin \theta - {B_2}\cos \theta } \right) \end{array} \right.$

 $\begin{array}{*{20}{c}} {{E_r}\left( {k_1^0,k_2^0,{A^0},{f^0},{b^0},{\theta ^0}} \right) = \sqrt {{{\left( {A_1^0} \right)}^2} + {{\left( {A_2^0} \right)}^2}} + }\\ {\sqrt {{{\left( {B_1^0} \right)}^2} + {{\left( {B_2^0} \right)}^2}} - 2{f^0}} \end{array}$

 $A_1^0 = \frac{{\left( {u - {u_0}} \right)\left( {1 + k_1^0{r^2} + k_2^0{r^4}} \right)}}{{{A^0}}} + {c^0}\cos {\theta ^0}$

 $A_2^0 = \left( {v - {v_0}} \right)\left( {1 + k_1^0{r^2} + k_2^0{r^4}} \right) + {c^0}\sin {\theta ^0}$

 $B_1^0 = \frac{{\left( {u - {u_0}} \right)\left( {1 + k_1^0{r^2} + k_2^0{r^4}} \right)}}{{{A^0}}} - {c^0}\cos {\theta ^0}$

 $B_2^0 = \left( {v - {v_0}} \right)\left( {1 + k_1^0{r^2} + k_2^0{r^4}} \right) - {c^0}\sin {\theta ^0}$

 ${c^0} = \sqrt {{{\left( {{f^0}} \right)}^2} - {{\left( {{b^0}} \right)}^2}}$

$e^{i}\left(u_{0}, v_{0}, a, b^{i}, \theta^{i}\right)$, $i$=0, 1, …, $N$-1, 表示不同空间直线在鱼眼图像上的投影椭圆，$p_j^i = \left({u_j^i} \right., \left. {v_j^i} \right)$, $j$=0, 1, …, $M$-1, 为投影椭圆$e_i$上的像素点，则当$N$≥2, $M$≥5时，根据式(17)以投影椭圆像点为观测值建立误差方程组进行最小二乘估计，待估计的未知数总数为4+2$N$。方程(15)非线性，需在给定参数初值$\left(k_{1}^{0}, k_{2}^{0}, A^{0}, f^{0}, b^{i 0}, \theta^{i 0}\right)$, $i$=0, 1, …, $N$-1条件下迭代求解。需要指出的是，上述误差方程对同一定焦相机而言，适用于其目标场景内任一空间直线并与空间直线方向、位置、长度无关，计算形式统一、过程简单，故可灵活应用于单视、多视条件。

# 3 实验分析

1) 以中间影像为对象，首先提取影像中的棋盘格角点并利用圆锥曲线方程拟合单个投影椭圆，再以棋盘格角点为观测值，根据第2节建立误差方程并设定参数初值进行最小二乘估计，获得鱼眼镜头内部参数值($f$, $A$, $k$1, $k$2)及其余椭圆几何参数值($b$, $θ$)；

2) 逐个提取边缘影像中的棋盘格角点并利用圆锥曲线方程拟合单个投影椭圆，因径向畸变参数$k$1, $k$2已于步骤1)中求出，故此时的棋盘格角点可预先进行畸变改正，从而使拟合得到的投影椭圆几何参数($b$, $θ$)初值更为合理；

3) 以步骤1)中求出的鱼眼镜头内部参数值为初值，以步骤1)和步骤2)中求出的($b$, $θ$)为相应的椭圆几何参数初值，以全部标定影像上的棋盘格角点为观测值建立误差方程进行全局优化求解获得最后的鱼眼镜头内部参数值。图 5给出了鱼眼镜头内部参数值($f$, $A$, $k$1, $k$2)的迭代计算过程，其中：径向畸变参数$k$1, $k$2初值取0，纵横比参数$A$初值取1.0，各投影椭圆$e_i$几何参数利用式(6)拟合相应的棋盘格角点得到，直接取其($b$, $θ$)作为相应参数初值，$f$初值则由各投影椭圆长半轴均值给出。

Table 1 Comparison of fish-eye cameras parameters through multi-view calibration process

 ($f_x$, $f_y$)/像素 ($u$0, $v$0)/像素 ($k$1, $k$2, $k$3, $k$4) RMSE 本文 (443.38, 454.75) (640.0, 640.0) (-7.71E-7, 1.898 E-13, -, -) 0.12 工具箱 (449.37, 471.40) (638.9, 646.9) (0.000 44, -0.055 85, 0.035 82, -0.009 33) 0.19

Table 2 Comparison of linear fitting error RMSE in corrected fisheye image with multi-view calibration parameters

 待纠正影像 中 左 右 顶 底 平均 本文 0.136 0.186 0.196 0.18 0.212 0.183 工具箱 0.288 0.246 0.172 0.101 0.156 0.223

Table 3 Single-view calibration parameters and its re-projection error RMSE in this paper

 图像 ($f_x$, $f_y$)/像素 ($u$0, $v$0)/像素 ($k$1，$k$2) RMSE $A$ (261.90, 260.08) (463.0, 463.0) (-2.993E-6, 4.503 3E-12) 0.292 $B$ (579.99, 583.15) (960.0, 960.0) (-5.609E-7, 1.442 8E-13) 0.318 $C$ (379.95, 370.26) (645.0, 641.0) (-1.729E-6, 1.914 9E-12) 0.542

1) 基于空间直线在球面透视投影下的水平面理想投影椭圆约束及椭圆内在严格几何特性，建立标定方程对包括等效焦距$f$、纵横比$A$及径向畸变$k$1, $k$2在内的鱼眼镜头内部参数直接进行最小二乘最优估计是可行的，计算形式统一、过程简单，可灵活应用于单视、多视标定条件。

2) 本文方法标定参数对鱼眼图像不同区域的平面透视纠正效果总体稳健、精度高，多视标定参数误差RMSE约0.1像素，纠正影像上直线拟合误差RMSE约0.2像素，略优于对比文献方法；单视标定参数误差RMSE约0.3像素，纠正影像范围、直线透视特性保持均优于对比文献方法及商业软件DXO。

3) 本文方法适用于鱼眼相机视野内任一空间直线并与直线方向、位置、长度无关，对标定参照物要求不高，对于具有大量平行直线的人工场景理论上可实现自标定，具有较好的应用价值。

# 参考文献

• [1] Feng W J, Zhang B F, Cao Z L. Omni-directional vision parameter calibration and rectification based on fish-eye lens[J]. Journal of Tianjin University, 2011, 44(5): 417–424. [冯为嘉, 张宝峰, 曹作良. 基于鱼眼镜头的全方位视觉参数标定与畸变矫正[J]. 天津大学学报, 2011, 44(5): 417–424. ] [DOI:10.3969/j.issn.0493-2137.2011.05.008]
• [2] Zhang H B, Yu Y, Li L, et al. Scene roaming method of fisheye image based on viewpoint correction[J]. Journal of Graphics, 2014, 35(3): 435–441. [张海彬, 余烨, 李琳, 等. 基于视点纠正的鱼眼图像场景化漫游方法[J]. 图学学报, 2014, 35(3): 435–441. ] [DOI:10.3969/j.issn.2095-302X.2014.03.018]
• [3] Li H B, Chu G Y, Zhang Q, et al. Space point positioning based on optimization of fisheye lens imaging model[J]. Acta Optica Sinica, 2015, 35(7): 0715003. [李海滨, 褚光宇, 张强, 等. 基于优化的鱼眼镜头成像模型的空间点定位[J]. 光学学报, 2015, 35(7): 0715003. ] [DOI:10.3788/aos201535.0715003]
• [4] Kannala J, Brandt S S. A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(8): 1335–1340. [DOI:10.1109/TPAMI.2006.153]
• [5] Schneider D, Schwalbe E, Maas H G. Validation of geometric models for fisheye lenses[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2009, 64(3): 259–266. [DOI:10.1016/j.isprsjprs.2009.01.001]
• [6] Wei J, Li C F, Hu S M, et al. Fisheye video correction[J]. IEEE Transactions on Visualization and Computer Graphics, 2012, 18(10): 1771–1783. [DOI:10.1109/TVCG.2011.130]
• [7] Wang Y Z. Fish-eye Lens Optics[M]. Beijing: Science Press, 2006: 5-15. [ 王永仲. 鱼眼镜头光学[M]. 北京: 科学出版社, 2006: 5-15.]
• [8] Huang Y D, Su H M. A simple transforming model from fisheye image to perspective projection image[J]. Journal of System Simulation, 2005, 17(1): 29–32, 52. [黄有度, 苏化明. 一种鱼眼图象到透视投影图象的变换模型[J]. 系统仿真学报, 2005, 17(1): 29–32, 52. ] [DOI:10.3969/j.issn.1004-731X.2005.01.007]
• [9] Huang F Y, Wang Y Z, Shen X J, et al. Method for calibrating the fisheye distortion center[J]. Applied Optics, 2012, 51(34): 8169–8176. [DOI:10.1364/AO.51.008169]
• [10] Lin Y, Gong X J, Liu J L. Calibration of fisheye cameras based on the viewing sphere[J]. Journal of Zhejiang University:Engineering Science, 2013, 47(8): 1500–1507. [林颖, 龚小谨, 刘济林. 基于单位视球的鱼眼相机标定方法[J]. 浙江大学学报:工学版, 2013, 47(8): 1500–1507. ]
• [11] Wu Z J, Wu Q Y, Zhang B C. A new calibration method for fisheye lens based on spherical model[J]. Chinese Journal of Lasers, 2015, 42(5): 0508006. [吴泽俊, 吴庆阳, 张佰春. 一种新的基于球面模型的鱼眼镜头标定方法[J]. 中国激光, 2015, 42(5): 0508006. ] [DOI:10.3788/cjl201542.0508006]
• [12] Pi Y D, Li X, Chen Z Y, et al. Calibration and rectification of fisheye images based on three-dimensional control field[J]. Acta Optica Sinica, 2017, 37(1): 0115001. [皮英冬, 李欣, 陈智勇, 等. 基于三维控制场的鱼眼影像检校和纠正方法[J]. 光学学报, 2017, 37(1): 0115001. ] [DOI:10.3788/aos201737.0115001]
• [13] Kanatani K. Calibration of ultrawide fisheye lens cameras by eigenvalue minimization[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(4): 813–822. [DOI:10.1109/TPAMI.2012.146]
• [14] Zhang M, Yao J, Xia M H, et al. Line-based multi-label energy optimization for fisheye image rectification and calibration[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA: IEEE, 2015: 4137-4145.[DOI: 10.1109/CVPR.2015.7299041]
• [15] Aghayari S, Saadatseresht M, Omidalizarandi M, et al. Geometric calibration of full spherical panoramic Ricoh-theta camera[J]. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2017, 4-1/W1: 237–245. [DOI:10.5194/isprs-annals-Ⅳ-1-W1-237-2017]
• [16] Ying X H, Hu Z Y. Fisheye lense distortion correction using spherical perspective projection constraint[J]. Chinese Journal of Computers, 2003, 26(12): 1702–1708. [英向华, 胡占义. 一种基于球面透视投影约束的鱼眼镜头校正方法[J]. 计算机学报, 2003, 26(12): 1702–1708. ] [DOI:10.3321/j.issn:0254-4164.2003.12.013]
• [17] Zheng L P, Xu G Q, Li L, et al. Imaging modeling and correction of nonlinear distortion distribution ellipse fish-eye lens[J]. Chinese Journal of Scientific Instrument, 2012, 33(6): 1331–1337. [郑利平, 徐刚强, 李琳, 等. 非线性畸变分布椭圆鱼眼镜头成像建模和校正[J]. 仪器仪表学报, 2012, 33(6): 1331–1337. ] [DOI:10.3969/j.issn.0254-3087.2012.06.019]
• [18] Liao S Z, Gao P H, Su Y, et al. A geometric rectification method for lens camera[J]. Journal of Image and Graphics, 2000, 5(7): 593–596. [廖士中, 高培焕, 苏艺, 等. 一种光学镜头摄像机图像几何畸变的修正方法[J]. 中国图像图形学报, 2000, 5(7): 593–596. ] [DOI:10.11834/jig.20000711]
• [19] Yang L, Cheng Y. The designing methods of fish-eye distortion correction using latitude-longitude projection[J]. Journal of Engineering Graphics, 2010, 31(6): 19–22. [杨玲, 成运. 应用经纬映射的鱼眼图像校正设计方法[J]. 工程图学学报, 2010, 31(6): 19–22. ] [DOI:10.3969/j.issn.1003-0158.2010.06.004]
• [20] Wei L S, Zhou S W, Zhang P G, et al. Double longitude model based correction method for fish-eye image distortion[J]. Chinese Journal of Scientific Instrument, 2015, 36(2): 377–385. [魏利胜, 周圣文, 张平改, 等. 基于双经度模型的鱼眼图像畸变矫正方法[J]. 仪器仪表学报, 2015, 36(2): 377–385. ] [DOI:10.19650/j.cnki.cjsi.2015.02.016]
• [21] Xu B, Liu L, Liu Y H, et al. A calibration method for fish-eye cameras based on pinhole model[J]. Acta Automatica Sinica, 2014, 40(4): 653–659. [涂波, 刘璐, 刘一会, 等. 一种扩展小孔成像模型的鱼眼相机矫正与标定方法[J]. 自动化学报, 2014, 40(4): 653–659. ] [DOI:10.3724/SP.J.1004.2014.00653]
• [22] Jia Y D, Lü H J, Xu A, et al. Fish-eye lens camera calibration for stereo vision system[J]. Chinese Journal of Computers, 2000, 23(11): 1215–1219. [贾云得, 吕宏静, 徐岸, 等. 一种鱼眼镜头成像立体视觉系统的标定方法[J]. 计算机学报, 2000, 23(11): 1215–1219. ] [DOI:10.3321/j.issn:0254-4164.2000.11.016]
• [23] Ramalingam S, Sturm P. A unifying model for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(7): 1309–1319. [DOI:10.1109/TPAMI.2016.2592904]
• [24] Hughes C, Denny P, Jones E, et al. Accuracy of fish-eye lens models[J]. Applied Optics, 2010, 49(17): 3338–3347. [DOI:10.1364/AO.49.003338]
• [25] Ray A, Srivastava D C. Non-linear least squares ellipse fitting using the genetic algorithm with applications to strain analysis[J]. Journal of Structural Geology, 2008, 30(12): 1593–1602. [DOI:10.1016/j.jsg.2008.09.003]
• [26] Bouguet J Y. Camera calibration toolbox for Matlab[EB/OL].[2019-02-28]2015-08-14. http://www.vision.caltech.edu/bouguetj/calib_doc.
• [27] Lu W, Tan J L. Detection of incomplete ellipse in images with strong noise by iterative randomized Hough transform (IRHT)[J]. Pattern Recognition, 2008, 41(4): 1268–1279. [DOI:10.1016/j.patcog.2007.09.006]