浏览全部资源
扫码关注微信
Published： 16 November 2024 ，
移动端阅览
罗杰， 林森. 2024. 基于透射率修正和分层优化的夜间图像去雾. 中国图象图形学报， 29(11):3357-3370
Luo Jie， Lin Sen. 2024. Nighttime image dehazing based on transmittance correction and layered optimization. Journal of Image and Graphics， 29(11):3357-3370
目的
2
大气中颗粒对光线的吸收和散射以及人造光源的影响，导致夜间获取的图像存在雾化、照度低和颜色偏差等问题，但传统的夜间去雾方法常局限于处理特定情况，未能综合考虑夜间图像中的各种影响因素。针对上述问题，提出一种基于透射率补偿与归一化和结构纹理优化的两阶段夜间图像去雾方法。
方法
2
首先，提出融合透射率补偿与归一化的修正方法获取透射率图，同时使用二次高斯滤波方法获取大气光图，并根据夜间成像模型实现图像去雾；其次，使用改进的基于Retinex的结构纹理分层模型（structure and texture aware retinex model based on the YUV color space，STAR-YUV）将图像分为结构层和纹理层，对结构层进行照明补偿和颜色校正，对纹理层采用拉普拉斯高斯滤波器以丰富细节信息；最后，采用两阶段融合方法将图像分步融合得到增强后的图像。
结果
2
理论分析和实验结果表明，经本文算法处理过的测试集图像，其峰值信噪比（peak signal-to-noise ratio，PSNR）、结构相似性（structural similarity index measure，SSIM）、平均梯度（average gradient，AG）、信息熵（information entropy，IE）和自然图像质量评估器（natural image quality evaluator，NIQE）指标平均值分别达到了17.024 dB、0.765、7.604、7.528和2.693，在对比的传统和深度学习算法中均位于前列，表明本文算法能够很好地实现夜间图像去雾，对细节和图像自然度的恢复也取得了较好结果。
结论
2
所提出的方法将透射率修正与结构纹理优化有效结合在一起，对含有整体色偏问题的夜间图像有更好的效果，能够提高场景亮度、校正色偏并丰富细节信息，具有普适性。
Objective
2
Adverse weather conditions， such as the presence of haze， along with the absorption and scattering of light by atmospheric particles， as well as insufficient and colored artificial light sources， pose a series of challenges for nighttime image dehazing. These challenges include hazing， low illumination， and color deviations， resulting in substantial degradation of images obtained by intelligent image capture systems. Consequently， the demand for advanced nighttime image processing technology continues to rise. At present， traditional nighttime dehazing methods are generally tailored for specific situations involving phenomena such as glow or low illuminance without considering various factors found in nighttime images， including low-illumination conditions， color discrepancies， and image blurring. This limitation leads to subpar performance when handling complex nighttime scenes. Meanwhile， most nighttime dehazing methods build upon daytime dehazing techniques as their foundation. However， daytime dehazing methods often overlook the impact of artificial light sources. Consequently， these methods can result in color distortions and insufficient brightness when applied directly to nighttime images. Furthermore， deep learning-based methods for nighttime image dehazing demand a substantial volume of training data and high-performance computing resources to achieve visually pleasing results. However， neglecting the fundamental principles of image formation in hazy conditions results in diminished interpretability of their results. A nighttime image dehazing method based on transmittance correction and layered optimization is presented to address the aforementioned challenge. On the one hand， the atmospheric light function is used to model nighttime scenes， while the non-uniform distribution of nighttime atmospheric light is considered to correct the transmittance. On the other hand， the structure and texture are optimized in layers to solve problems such as color deviation and low illumination.
Method
2
First， a novel transmittance correction method is introduced. This method establishes clear initial transmittance maps by setting maximum and minimum boundary values through boundary constraints. Subsequently， compensation and normalization are conducted based on whether the initial transmittance map corresponds to the light source area of the image. This process ensures the relevance and effectiveness of dehazing， resulting in the final transmittance map. Haze is concentrated on the haze line within the RGB color space and can be precisely mapped onto the gray component of the Y channel in the YUV color space. This relationship between the Y channel and haze is leveraged to obtain the final atmospheric light map. This approach is achieved by employing the quadratic Gaussian filtering method， which involves conducting an initial Gaussian filter on the Y channel to extract relevant information and applying a second Gaussian filter to all channels. Afterward， image dehazing is accomplished by utilizing the two resulting maps obtained via the nighttime imaging model. Second， a STAR model based on the YUV color space （STAR-YUV） is introduced. This model decomposes the Y channels， which can reflect the lighting information of each component， into structural and texture layers and enhances the image features accordingly. For structural layers with rich color information， gamma correction is applied for illumination compensation， and MSRCR color correction is used to effectively correct color deviations， obtaining natural color output. For texture layers with rich detail information， Laplacian Gaussian filters are employed to enrich and sharpen complex details while preserving image edges and texture information， resulting in detail enhancement output. Finally， a novel two-stage nighttime image fusion method is introduced to achieve a superior visual result and integrate the previous two steps. In the first stage， a nonlinear fusion approach is employed on the structural and texture layers to preserve structural details and fine features， respectively， based on the principles of the Retinex theory. In the second stage， a linear fusion method is applied to merge the dehazing results with the outcomes of the first stage， thereby leveraging the advantages of dehazing and image enhancement to generate a high-quality and visually appealing final image. Considering the distinctive characteristics of nighttime images and addressing the limitations of existing approaches， this method consolidates the transmittance correction method for transmittance compensation with structural and textural optimization， thereby offering a highly effective solution for enhancing nighttime images.
Result
2
Based on theoretical analysis and extensive experiments conducted on two distinct test datasets， the proposed algorithm is shown to achieve dehazing of nighttime images. On the ZS330 test dataset images， the visual quality of the processed images was improved， with the average values of AG， IE， and NIQE indicators reaching 7.836， 7.461， and 2.683， respectively. On the HC770 test dataset images， the processed images also exhibited better performance， with PSNR， SSIM， AG， IE， and NIQE indicators reaching 17.024 dB， 0.765， 7.371， 7.595， and 2.702， respectively. Across the entire test dataset， the average PSNR， SSIM， AG， IE， and NIQE indicators reached 17.024 dB， 0.765， 7.604， 7.528， and 2.693， respectively. Compared to traditional and deep learning-based algorithms used in the paper， this approach consistently ranks among the top performers across various indicators. The proposed algorithm displays strong performance in restoring image details， preserving naturalness， and effectively addressing severe color deviations commonly observed in nighttime images. Additionally， the algorithm demonstrates robustness by effectively enhancing images in scenarios with or without artificial light sources.
Conclusion
2
This approach combines a novel transmittance correction method with structural and texture layer optimization. Leveraging the strengths of the two methods has resulted in advancements in restoring the clarity of nighttime images， improving lighting conditions， correcting color deviations， and preserving intricate details.
夜间图像去雾透射率修正方法结构纹理优化两阶段图像融合
nighttime image dehazingtransmittance correction methodstructure and texture optimizationtwo stagesimage fusion
Cai B L， Xu X M， Jia K， Qing C M and Tao D C. 2016. DehazeNet： an end-to-end system for single image haze removal. IEEE Transactions on Image Processing， 25（11）： 5187-5198 ［DOI： 10.1109/TIP.2016.2598681http://dx.doi.org/10.1109/TIP.2016.2598681］
Cui Z T， Li K C， Gu L， Su S H， Gao P， Jiang Z K， Qiao Y and Harada T. 2022. Illumination adaptive transformer［EB/OL］. ［2023-11-23］. https://arxiv.org/pdf/2205.14871v1.pdfhttps://arxiv.org/pdf/2205.14871v1.pdf
He K M， Sun J and Tang X O. 2011. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence， 33（12）： 2341-2353 ［DOI： 10.1109/TPAMI.2010.168http://dx.doi.org/10.1109/TPAMI.2010.168］
Jobson D J， Rahman Z and Woodell G A. 1997. A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Transactions on Image Processing， 6（7）： 965-976 ［DOI： 10.1109/83.597272http://dx.doi.org/10.1109/83.597272］
Kuanar S， Rao K R， Mahapatra D and Bilas M. 2019. Night time haze and glow removal using deep dilated convolutional network［EB/OL］. ［2023-11-23］. https://arxiv.org/pdf/1902.00855.pdfhttps://arxiv.org/pdf/1902.00855.pdf
Li B Y， Ren W Q， Fu D P， Tao D C， Feng D， Zeng W J and Wang Z Y. 2019. Benchmarking single-image dehazing and beyond. IEEE Transactions on Image Processing， 28（1）： 492-505 ［DOI： 10.1109/TIP.2018.2867951http://dx.doi.org/10.1109/TIP.2018.2867951］
Li Y， Tan R T and Brown M S. 2015. Nighttime haze removal with glow and multiple light colors//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago， Chile： IEEE： 226-234 ［DOI： 10.1109/ICCV.2015.34http://dx.doi.org/10.1109/ICCV.2015.34］
Li Z L， Che W W， Qian M H and Xu X L. 2021. An improved image defogging algorithm based on Histogram equalization. Henan Science， 39（1）： 1-6
李竹林， 车雯雯， 钱梦杭， 徐雪丽. 2021. 一种改进的直方图均衡化图像去雾算法. 河南科学， 39（1）： 1-6 ［DOI： 10.3969/j.issn.1004-3918.2021.01.001http://dx.doi.org/10.3969/j.issn.1004-3918.2021.01.001］
Liao Y H， Su Z， Liang X G and Qiu B. 2018. HDP-Net： haze density prediction network for nighttime dehazing// Proceedings of the 19th Pacific-Rim Conference on Multimedia. Hefei， China： Springer： 469-480 ［DOI： 10.1007/978-3-030-00776-8_43http://dx.doi.org/10.1007/978-3-030-00776-8_43］
Liu X N， Li H and Zhu C. 2022a. Joint contrast enhancement and exposure fusion for real-world image dehazing. IEEE Transactions on Multimedia， 24： 3934-3946 ［DOI： 10.1109/TMM.2021.3110483http://dx.doi.org/10.1109/TMM.2021.3110483］
Liu Y， Yan Z S， Tan J G and Li Y C. 2023. Multi-purpose oriented single nighttime image haze removal based on unified variational retinex model. IEEE Transactions on Circuits and Systems for Video Technology， 33（4）： 1643-1657 ［DOI： 10.1109/TCSVT.2022.3214430http://dx.doi.org/10.1109/TCSVT.2022.3214430］
Liu Y， Yan Z S， Ye T， Wu A M and Li Y C. 2022b. Single nighttime image dehazing based on unified variational decomposition model and multi-scale contrast enhancement. Engineering Applications of Artificial Intelligence， 116： #105373 ［DOI： 10.1016/j.engappai.2022.105373http://dx.doi.org/10.1016/j.engappai.2022.105373］
Lu W， Gao T， Wang C C， Chen B H and Zhang S. 2019. Low illumination color image enhancement algorithm based on fusion idea under Retinex theory. Science Technology and Engineering， 19（13）： 151-157
卢玮， 高涛， 王翠翠， 陈本豪， 张赛. 2019. Retinex理论下基于融合思想的低照度彩色图像增强算法. 科学技术与工程， 19（13）： 151-157 ［DOI： 10.3969/j.issn.1671-1815.2019.13.024http://dx.doi.org/10.3969/j.issn.1671-1815.2019.13.024］
McCartney E J and Hall F F. 1977. Optics of the atmosphere： scattering by molecules and particles. Physics Today， 30（5）： 76-77 ［DOI： 10.1063/1.3037551http://dx.doi.org/10.1063/1.3037551］
Meng G F， Wang Y， Duan J Y， Xiang S M and Pan C H. 2013. Efficient image dehazing with boundary constraint and contextual regularization//Proceedings of 2013 IEEE International Conference on Computer Vision. Sydney， Australia： IEEE： 617-624 ［DOI： 10.1109/ICCV.2013.82http://dx.doi.org/10.1109/ICCV.2013.82］
Mittal A， Soundararajan R and Bovik A C. 2013. Making a “completely blind” image quality analyzer. IEEE Signal Processing Letters， 20（3）： 209-212 ［DOI： 10.1109/LSP.2012.2227726http://dx.doi.org/10.1109/LSP.2012.2227726］
Sahu G， Seal A， Bhattacharjee D， Frischer R and Krejcar O. 2023. A novel parameter adaptive dual channel MSPCNN based single image dehazing for intelligent transportation systems. IEEE Transactions on Intelligent Transportation Systems， 24（3）： 3027-3047 ［DOI： 10.1109/TITS.2022.3225797http://dx.doi.org/10.1109/TITS.2022.3225797］
Sahu G， Seal A， Bhattacharjee D， Nasipuri M， Brida P and Krejcar O. 2022. Trends and prospects of techniques for haze removal from degraded images： a survey. IEEE Transactions on Emerging Topics in Computational Intelligence， 6（4）： 762-782 ［DOI： 10.1109/TETCI.2022.3173443http://dx.doi.org/10.1109/TETCI.2022.3173443］
Sharma G， Wu W C and Dalal E N. 2005. The CIEDE2000 color‐difference formula： implementation notes， supplementary test data， and mathematical observations. Color Research & Application， 30（1）： 21-30 ［DOI： 10.1002/col.20070http://dx.doi.org/10.1002/col.20070］
Tang Y Q， Xiang Y and Chen G F. 2023. A nighttime and daytime single-image dehazing method. Applied Sciences， 13（1）： #255 ［DOI： 10.3390/app13010255http://dx.doi.org/10.3390/app13010255］
Wang W H， Wang A N and Liu C. 2022. Variational single nighttime image haze removal with a gray haze-line prior. IEEE Transactions on Image Processing， 31： 1349-1363 ［DOI： 10.1109/TIP.2022.3141252http://dx.doi.org/10.1109/TIP.2022.3141252］
Wang Z， Bovik A C， Sheikh H R and Simoncelli E P. 2004. Image quality assessment： from error visibility to structural similarity. IEEE Transactions on Image Processing， 13（4）： 600-612 ［DOI： 10.1109/TIP.2003.819861http://dx.doi.org/10.1109/TIP.2003.819861］
Wu Y Q， Tao D P， Zhan Y B and Zhang C Y. 2022. BiN-flow： bidirectional normalizing flow for robust image dehazing. IEEE Transactions on Image Processing， 31： 6635-6648 ［DOI： 10.1109/TIP.2022.3214093http://dx.doi.org/10.1109/TIP.2022.3214093］
Xiang Y， Chen G F and Li X. 2021. Nighttime dehazing based on mixed filtering and transmission optimization. Acta Photonica Sinica， 50（12）： #1210003
项胤， 陈广锋， 李侠. 2021. 基于混合滤波和透射率优化的夜间去雾. 光子学报， 50（12）： #1210003 ［DOI： 10.3788/gzxb20215012.1210003http://dx.doi.org/10.3788/gzxb20215012.1210003］
Xu J， Hou Y K， Ren D W， Liu L， Zhu F， Yu M Y， Wang H Q and Shao L. 2020. STAR： a structure and texture aware retinex model. IEEE Transactions on Image Processing， 29： 5022-5037 ［DOI： 10.1109/TIP.2020.2974060http://dx.doi.org/10.1109/TIP.2020.2974060］
Yang Y and Zhang W B. 2023. Dehazing algorithm with atmospheric light veil estimation based on haze distribution. Chinese Journal of Liquid Crystals and Displays， 38（4）： 534-542
杨燕， 张雯波. 2023. 基于雾气分布的大气光幕估计去雾算法. 液晶与显示， 38（4）： 534-542 ［DOI： 10.37188/CJLCD.2022-0237http://dx.doi.org/10.37188/CJLCD.2022-0237］
Yu J， Xu D B and Liao Q M. 2011. Image defogging： a survey. Journal of Image and Graphics， 16（9）： 1561-1576
禹晶， 徐东彬， 廖庆敏. 2011. 图像去雾技术研究进展. 中国图象图形学报， 16（9）： 1561-1576 ［DOI： 10.11834/jig.20110920http://dx.doi.org/10.11834/jig.20110920］
Zhang J， Cao Y and Wang Z F. 2014. Nighttime haze removal based on a new imaging model//Proceedings of 2014 International Conference on Image Processing. Paris， France： IEEE： 4557-4561 ［DOI： 10.1109/ICIP.2014.7025924http://dx.doi.org/10.1109/ICIP.2014.7025924］
Zhang J， Cao Y， Zha Z J and Tao D C. 2020. Nighttime dehazing with a synthetic benchmark//Proceedings of the 28th ACM International Conference on Multimedia. Seattle， USA： Association for Computing Machinery： 2355-2363 ［DOI： 10.1145/3394171.3413763http://dx.doi.org/10.1145/3394171.3413763］
Zhou M L， Leng H Y， Fang B， Xiang T， Wei X K and Jia W J. 2023. Low-light image enhancement via a frequency-based model with structure and texture decomposition. ACM Transactions on Multimedia Computing， Communications and Applications， 19（6）： #187 ［DOI： 10.1145/3590965http://dx.doi.org/10.1145/3590965］
Zhuang P X and Ding X H. 2020. Underwater image enhancement using an edge-preserving filtering Retinex algorithm. Multimedia Tools and Applications， 79（25/26）： 17257-17277 ［DOI： 10.1007/s11042-019-08404-4http://dx.doi.org/10.1007/s11042-019-08404-4］
Zhuang P X， Li C Y and Wu J M. 2021. Bayesian retinex underwater image enhancement. Engineering Applications of Artificial Intelligence， 101： #104171 ［DOI： 10.1016/j.engappai.2021.104171http://dx.doi.org/10.1016/j.engappai.2021.104171］
相关文章
相关作者
相关机构
京公网安备11010802024621