浏览全部资源
扫码关注微信
收稿：2017-09-08，
修回：2017-12-16，
纸质出版：2018-06-16
移动端阅览
目的
2
受成像距离、光照条件、动态模糊等因素影响，监控系统拍摄的车牌图像往往并不具备较高的可辨识度。为改善成像质量，提升对车牌的识别能力，提出一种基于亮度与梯度联合约束的车牌图像超分辨率重建方法。
方法
2
首先充分结合亮度约束和梯度约束的优势，实现对运动位移和模糊函数的精确估计；为抑制重建图像中的噪声与伪影，基于车牌图像的文字化特征，进一步确定了亮度与梯度联合约束的图像先验模型。
结果
2
为验证该方法的有效性，利用监控系统获得4组车牌图像，分别进行模拟和真实的超分辨率重建实验。在模拟实验中将联合约束图像先验重建结果与拉普拉斯、Huber-Markov（HMRF）以及总变分（TV）先验的处理结果进行对比，联合约束先验对车牌纹理信息的恢复效果优于其他3种常见图像先验；同时，在模拟和真实实验中，将本文算法与双三次插值、传统最大后验概率、非线性扩散正则化和自适应范数正则化方法的超分辨率重建结果进行比较，模拟实验的结果表明，在不添加噪声情况下，该算法峰值信噪比（PSNR）和结构相似性（SSIM）指标分别为35.326 dB和0.958，优于其他4种算法；该算法在真实实验中，能够有效增强车牌图像纹理信息，获得较优的视觉效果，通过对重建车牌图像的字符识别精度比较，本文算法重建结果的识别精度远高于其他3种算法，平均字符差距为1.3。
结论
2
模拟和真实图像序列的实验结果证明，基于亮度—梯度联合约束的超分辨率重建方法，能够降低运动和模糊等参数的估计误差，有效减少图像中存在的模糊和噪声，提高车牌的识别精度。该算法广泛适用于因光照变化、相对运动等因素影响下的低质量车牌图像超分辨率重建。
Objective
2
License plate images captured by monitoring systems often have relatively low spatial resolution and are thus difficult to identify due to the large distances between vehicles and cameras
the low resolution of the imaging devices used for video images
and factors such as atmospheric disturbances
lighting conditions
and motion blur. The high-resolution reconstruction of low-resolution license plate images is crucial in enhancing license plate image resolution and thus increasing the accuracy of license plate recognition. Multiframe super-resolution techniques are particularly well suited for this application because they facilitate recovery of valid results from low-quality images by gathering information not only from the image itself but also from the constraints that must be satisfied. In this work
a super-resolution reconstruction algorithm based on intensity-gradient prior combinations is proposed to improve the quality and detectability of license plate images.
Method
2
The proposed algorithm includes three steps. First
the motion displacement between the multiframe images is estimated with a novel optical flow estimation method under a robust data function. The data fidelity model that adds gradient constancy constraints to the color constancy constraint is more accurate in terms of modeling the confidence of pixel correspondence than that using only one out of the two terms
which is not appropriate because the color constancy constraint is often violated when illumination or exposure changes. In the optical flow estimation model
the selective combination of the color and gradient constraints in defining the data term enables the recovery of many motion details that are robust to outliers. The blur function of the reference license plate image is then estimated with a blind deblurring method that is based on regularized intensity and gradient prior in the second step of the algorithm. The proposed intensity and gradient prior
which is based on distinctive properties such that text and background regions usually have nearly uniform intensity values in clear images without blurs
is effective for cases with rich text
which can be modeled by two-tone distributions. We can reliably estimate the blur kernel with an efficient optimization algorithm. By combining the advantages of intensity and gradient constraints fully combined
we can realize the high accurate estimation of accurately estimate the motion displacement and the blur function. Meanwhile
an intensity-gradient image prior combination model based on the characterization of license plate images is also further utilized in the super-resolution algorithm to suppress the noise and artifacts in the reconstructed images.
Result
2
To verify the effectiveness of the proposed algorithm
experiments with simulated and real license plate images are implemented. The proposed algorithm is utilized to reconstruct a low-quality license plate image and compare its results with those obtained by bicubic
traditional maximum a posterior
nonlinear diffusion regularization
and adaptive norm regularized methods
which are used as benchmarks. Qualitative and quantitative analyses are conducted to evaluate the proposed algorithm. Experimental results show that the proposed technique can remove image noise and blur and produce the best reconstructed image among all compared methods. The peak signal-to-noise ratio (PSNR) and structure similarity image measure (SSIM) value of the proposed method are higher than those of the four other algorithms. The PSNR and SSIM values obtained by the proposed method without Gaussian noise in the simulated license plate image experiments are 35.326 dB and 0.958
respectively. The proposed method can also effectively enhance the detailed information of license plate images and obtain superior visual effect in the real experiments. In addition
the license plate recognition accuracy of the proposed algorithm is higher than that of the three other algorithms.
Conclusion
2
The proposed method can effectively eliminate artifacts and compensate for the texture information of low-quality license plate images. The effectiveness of the proposed method is validated by simulation and real image reconstruction experiments. The results of these experiments demonstrate that the proposed method can significantly reduce the error of motion and blur estimation
effectively decrease image blur and noise
and achieve promising PSNRs. Notably
the proposed method performs better than existing methods in terms of accuracy and visual improvement results
which are natural and consistent with those of the human visual system. The accuracy of the license plate image detection results shows that the super-resolution reconstruction method proposed in this study significantly improves the identification of license plate characters. Consequently
this method can also enhance license plate image resolution and effectively increase the accuracy of license plate recognition. The proposed method can be widely applied to the super-resolution reconstruction of license plate images
which are seriously affected by illumination variation and motion blur.
Wang Y M. Research on super-resolution reconstruction algorithm of vehicle plate images[D]. Dalian: Dalian Maritime University, 2016. http://cdmd.cnki.com.cn/Article/CDMD-10151-1017039283.htm .
王艳梅. 车牌图像超分辨率重建算法研究[D]. 大连: 大连海事大学, 2016.
Park S C, Park M K, Kang M G. Super-resolution image reconstruction:a technical overview[J]. IEEE Signal Processing Magazine, 2003, 20(3):21-36.[DOI:10.1109/MSP.2003.1203207]
Su B H, Jin W Q, Niu L H, et al. Super-resolution image restoration and progress[J]. Optical Technique, 2001, 27(1):6-9.
苏秉华, 金伟其, 牛丽红, 等.超分辨率图像复原及其进展[J].光学技术, 2001, 27(1):6-9. [DOI:10.3321/j.issn:1002-1582.2001.01.018]
Tsai R Y, Huang T S. Multiframe image restoration and registration[M]//Huang T S. Advances in Computer Vision and Image Processing. Greenwich: JAI Press, 1984: 317-339.
Tekalp A M, Ozkan MK, Sezan M I. High-resolution image reconstruction from lower-resolution image sequences and space-varying image restoration[C]//Proceedings of 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing. San Francisco: IEEE, 1992: 169-172. [ DOI:10.1109/ICASSP.1992.226249 http://dx.doi.org/10.1109/ICASSP.1992.226249 ]
Davila C E. Recursive total least squares algorithms for adaptive filtering[C]//Proceedings of 1991 International Conference on Acoustics, Speech, and Signal Processing. Toronto: IEEE, 1991: 1853-1856. [ DOI:10.1109/ICASSP.1991.150722 http://dx.doi.org/10.1109/ICASSP.1991.150722 ]
Kaltenbacher E, Hardie R C. High resolution infrared image reconstruction using multiple, low resolution, aliased frames[C]//Proceedings of IEEE 1996 National Aerospace and Electronics Conference. Dayton: IEEE, 1996: 702-709. [ DOI:10.1109/NAECON.1996.517726 http://dx.doi.org/10.1109/NAECON.1996.517726 ]
Shah N R, Zakhor A. Resolution enhancement of color video sequences[J]. IEEE Transactions on Image Processing, 1999, 8(6):879-885.[DOI:10.1109/83.766865]
Irani M, Peleg S. Improving resolution by image registration[J]. CVGIP:Graphical Models and Image Processing, 1991, 53(3):231-239.[DOI:10.1016/1049-9652(91)90045-L]
Liang F M, Xu Y J, Zhang M X, et al. A POCS algorithm based on text features for the reconstruction of document images at super-resolution[J]. Symmetry, 2016, 8(10):#102.[DOI:10.3390/sym8100102]
Schultz R R, Stevenson R L. Extraction of high-resolution frames from video sequences[J]. IEEE Transactions on Image Processing, 1996, 5(6):996-1011.[DOI:10.1109/83.503915]
Shen H F, Zhang L P, Huang B, et al. A MAP approach for joint motion estimation, segmentation, andsuper resolution[J]. IEEE Transactions on Image Processing, 2007, 16(2):479-490.[DOI:10.1109/TIP.2006.888334]
Akgun T, Altunbasak Y, Mersereau R M. Super-resolution reconstruction of hyperspectral images[J]. IEEE Transactions on Image Processing, 2005, 14(11):1860-1875.[DOI:10.1109/TIP.2005.854479]
Zhang L P, Zhang H Y, Shen H F, et al. A super-resolution reconstruction algorithm for surveillance images[J]. Signal Processing, 2010, 90(3):848-859.[DOI:10.1016/j.sigpro.2009.09.002]
Isaac J S, Kulkarni R. Super resolution techniques for medical image processing[C]//Proceedings of 2015 International Conference on Technologies for Sustainable Development. Mumbai: IEEE, 2015: 1-6. [ DOI:10.1109/ICTSD.2015.7095900 http://dx.doi.org/10.1109/ICTSD.2015.7095900 ]
Suresh K V, Kumar G M, Rajagopalan A N.Superresolution of license plates in real traffic videos[J]. IEEE Transactions on Intelligent Transportation Systems, 2007, 8(2):321-331.[DOI:10.1109/TITS.2007.895291]
Zeng W L, Lu X B.A generalized DAMRF image modeling for superresolution of license plates[J]. IEEE Transactions on Intelligent Transportation Systems, 2012, 13(2):828-837.[DOI:10.1109/TITS.2011.2180714]
Wei L N, Liu Y. A license plate super-resolution reconstruction algorithm based on manifold learning[C]//Proceedings of the 17th IEEE International Conference on Computational Science and Engineering. Chengdu: IEEE, 2014: 1855-1859. [ DOI:10.1109/CSE.2014.340 http://dx.doi.org/10.1109/CSE.2014.340 ]
Chuang C H, Tsai L W, Deng M S, et al. Vehicle licence plate recognition using super-resolution technique[C]//Proceedings of the 201411th IEEE International Conference on Advanced Video and Signal Based Surveillance. Seoul: IEEE, 2014: 411-416. [ DOI:10.1109/AVSS.2014.6918703 http://dx.doi.org/10.1109/AVSS.2014.6918703 ]
Marsi S, Carrato S, Ramponi G. Arobust tracking algorithm for super-resolution reconstruction of vehicle license plates[M]//De Gloria A. Applications in Electronics Pervading Industry, Environment and Society. Cham: Springer, 2016: 65-73. [ DOI:10.1007/978-3-319-20227-3_9 http://dx.doi.org/10.1007/978-3-319-20227-3_9 ]
Jin R C, Zhao S R, Xu X Y, et al.Multiframe superresolution of vehicle license plates based on distribution estimation approach[J]. Journal of Control Science and Engineering, 2016, 2016:#4194309.[DOI:10.1155/2016/4194309]
Wu W, Yang X M, Qing L B, et al. Low-resolution license plate imagesrestoration based on MRF[J]. Application Research of Computers, 2010, 27(3):1170-1172, 1186.
吴炜, 杨晓敏, 卿粼波, 等.基于马尔可夫随机场的低分辨率车牌图像复原算法[J].计算机应用研究, 2010, 27(3):1170-1172, 1186. [DOI:10.3969/j.issn.1001-3695.2010.03.102]
Zhou L, Lu X B, Yang L.A local structure adaptive super-resolution reconstruction method based on BTV regularization[J]. Multimedia Tools and Applications, 2014, 71(3):1879-1892.[DOI:10.1007/s11042-012-1311-x]
Zhang D, He J Z. Super-resolution reconstruction of low-resolution vehicle plates: a comparative study and a new algorithm[C]//Proceedings of the 20147th International Congress on Image and Signal Processing. Dalian: IEEE, 2015: 359-364. [ DOI:10.1109/CISP.2014.7003806 http://dx.doi.org/10.1109/CISP.2014.7003806 ]
Lou Y F, Bertozzi A L, Soatto S. Direct sparse deblurring[J]. Journal of Mathematical Imaging and Vision, 2011, 39(1):1-12.[DOI:10.1007/s10851-010-0220-8]
Protter M, Elad M. Super resolution with probabilistic motion estimation[J]. IEEE Transactions on Image Processing, 2009, 18(8):1899-1904.[DOI:10.1109/TIP.2009.2022440]
Yuan X H, Ouyang X L, Xia D S. An overview on super resolution image restoration[J]. Geography and Geo-Information Science, 2006, 22(3):43-47.
袁小华, 欧阳晓丽, 夏德深.超分辨率图像恢复研究综述[J].地理与地理信息科学, 2006, 22(3):43-47. [DOI:10.3969/j.issn.1672-0504.2006.03.010]
Xu L, Jia J Y, Matsushita Y. Motion detail preserving optical flow estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(9):1744-1757.[DOI:10.1109/TPAMI.2011.236]
Pan J S, Hu Z, Su Z X, et al. L 0 -regularized intensity and gradient prior for deblurring text images and beyond[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(2):342-355.[DOI:10.1109/TPAMI.2016.2551244] .
Shen D F, Chiu C W. Fundamental techniques for resolution enhancement of average subsampled images[J]. Journal of Electronic Imaging, 2012, 21(3):#033027.[DOI:10.1117/1.JEI.21.3.033027]
Gleich D.Markov random field models for non-quadratic regularization of complex SAR images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2012, 5(3):952-961.[DOI:10.1109/JSTARS.2011.2179524]
Boyd S, Parikh N, Chu E, et al. Distributed optimization and statistical learning via the alternating direction method of multipliers[J]. Foundations and Trends ® in Machine Learning, 2011, 3(1):1-122.[DOI:10.1561/2200000016]
Maiseli B J, Ally N, Gao H J. A noise-suppressing and edge-preserving multiframe super-resolution image reconstruction method[J].Signal Processing:Image Communication, 2015, 34:1-13.[DOI:10.1016/j.image.2015.03.001]
Shen H F, Peng L, Yue L W, et al. Adaptive norm selection for regularized image restoration and super-resolution[J]. IEEE Transactions on Cybernetics, 2016, 46(6):1388-1399.[DOI:10.1109/TCYB.2015.2446755]
EasyPR software[CP/OL]. [2017-11-01] . https://github.com/liuruoze/EasyPR https://github.com/liuruoze/EasyPR .
相关作者
相关机构
京公网安备11010802024621
