浏览全部资源
扫码关注微信
纸质出版日期： 2022-05-16 ，
录用日期： 2021-12-22
移动端阅览
赵明华, 程丹妮, 都双丽, 胡静, 石程, 石争浩. 改进融合策略下透明度引导的逆光图像增强[J]. 中国图象图形学报, 2022,27(5):1554-1564.
Minghua Zhao, Danni Cheng, Shuangli Du, Jing Hu, Cheng Shi, Zhenghao Shi. An improved fusion strategy based on transparency-guided backlit image enhancement[J]. Journal of Image and Graphics, 2022,27(5):1554-1564.
目的
2
针对传统的逆光图像增强算法存在的曝光正常区域与逆光区域间阈值计算复杂、分割精度不足、过度曝光以及增强不足等问题，提出一种改进融合策略下透明度引导的逆光图像增强算法。
方法
2
对逆光图像在HSV(hue
saturation
value)空间中的亮度分量进行亮度提升和对比度增强，然后通过金字塔融合策略对改进的亮度分量进行分解和重构，恢复逆光区域的细节和颜色信息。此外，利用深度抠图网络计算透明度蒙版，对增强的逆光区域与源图像进行融合处理，维持非逆光区域亮度不变。通过改进融合策略增强的图像在透明度引导下既有效恢复了逆光区域又避免了曝光过度的问题。
结果
2
实验在多幅逆光图像上与直方图均衡算法、MSR (multi-scale Retinex)、Zero-DEC (zero-reference deep curve estimation)、AGLLNet (attention guided low-light image enhancement)和LBR (learning-based restoration) 5种方法进行了比较，在信息熵(information entropy
IE)和盲图像质量指标(blind image quality indicators
BIQI)上，比AGLLNet分别提高了1.9%和10.2%；在自然图像质量评价(natural image quality evaluation
NIQE)方面，比Zero-DCE(zero-reference deep curve estimation)提高了3.5%。从主观评估上看，本文算法增强的图像在亮度、对比度、颜色及细节上恢复得更加自然，达到了较好的视觉效果。
结论
2
本文方法通过结合金字塔融合技术与抠图技术，解决了其他方法存在的色彩失真和曝光过度问题，具有更好的增强效果。
Objective
2
The backlit image is a kind of redundant reflection derived of the light straightforward to the camera
resulting in dramatic reduced visibility of region of interest (ROI) in the captured image. Different from ordinary low-light images
the backlit image has a wider grayscale range due to the extremely dark and bright parts. Traditional enhancement algorithms restore brightness and details of backlit parts in terms of overexposure and color distortion. Fusion technology or threshold segmentation is difficult to implement sufficient enhancement or adequate segmentation accuracy due to uneven images gray distribution. A transparency-guided backlit image enhancement method is demonstrated based on an improved fusion strategy.
Method
2
The backlit image enhancing challenge is to segment and restore the backlit region
which is regarded as the foreground and the rest as the background. First
the deep image matting model like encoder-decoder network and refinement network is illustrated. The backlit image and its related trimap are input into the encoder-decoder network to get the preliminary transparency value matrix. The output is melted into the refinement network to calculate the transparency value of each pixel
which constitutes the same scale alpha matte as the original image. The range of transparency value is between 0 and 1
0 and 1 indicates pixels in the normal exposure region and the backlit region
respectively. The value between 0 and 1 is targeted to the overlapped regions. The alpha matte can be used to substitute the traditional weight map for subsequent fusion processing in terms of processed non-zero pixels. Next
the backlit image is converted into HSV(hue
saturation
value) space to extract the luminance component and the adaptive logarithmic transformation is conducted to enhance brightness in terms of the base value obtained from the number of low-gray image pixels. Contrast-limited adaptive histogram equalization is also adopted to enhance the contrast of the luminance component while logarithmic transformation can only be stretched or compressed gray values. Subsequently
Laplacian pyramid fusion is illustrated on the two improved luminance components. The obtained result was integrated into the original hue component and saturation component and it is converted to RGB space to obtain the global enhanced image. Finally
the alpha matte is used to linearly fuse the source image and the global enhanced image to maintain the brightness of non-backlit area.
Result
2
Our demonstration is compared to existing methods
including histogram equalization (HE)
multi-scale Retinex (MSR)
zero-reference deep curve estimation(Zero-DCE)
attention guided low-light image enhancement(AGLLNet) and learning-based restoration (LBR). Information entropy (IE)
blind image quality indicator (BIQI) and natural image quality evaluation (NIQE) are utilized to evaluate the restored image quality. IE is used to measure the richness of image information. The larger the value is
the richer the information and the higher the image quality are. BIQI performs distortion recognition is based on the calculated degradation rate of the image
and a small value represents a high quality image. NIQE compares the image with the designed natural image model
and the lower the value is
the higher the image sharpness is. Both subjective visual effects and objective image quality evaluation indicators are analyzed further. Our qualitatiave analysis can demonstrate better backlit image
no artifacts and natural visual effect
while HE causes color distortion and serious halo phenomenon in non-backlit region. MSR processes the three color channels each to cause the color information loss
Zero-DCE lacks color saturation and the enhancement effect of AGLLNet is not clarified. LBR is limited to segmentation accuracy
leading to color distortion and edge artifacts. Our quantitative algorithm illustrates its priorities in IE
BIQI and NIQE
improving by 0.137 and 3.153 compared with AGLLNet in IE and BIQI
respectively
and 3.5% in NIQE compared with Zero-DCE.
Conclusion
2
We introduced the deep image matting with precise segmentation capability to detect and identify the backlit region. An enhanced image brightness component is obtained via the improved Laplacian pyramid based fusion strategy. Our demonstration improves the brightness and contrast of the backlit image and restores the details and color information. The over-exposure and insufficient enhancement can be resolved further. Artifacts and distortion issues are not involved in our algorithm
and the improvement of objective quality evaluation index validates the backlight image enhancement.
图像处理逆光图像增强深度抠图灰度拉伸金字塔融合
image processingbacklight image enhancementdeep image mattinggray stretchingpyramid fusion
Ancuti C O, Ancuti C, De Vleeschouwer C and Bekaert P. 2018. Color balance and fusion for underwater image enhancement. IEEE Transactions on Image Processing, 27(1): 379-393 [DOI: 10.1109/TIP.2017.2759252]
Boda J and Pandya D. 2018. A survey on image matting techniques//Proceedings of 2018 International Conference on Communication and Signal Processing. Chennai, India: IEEE: 765-770 [DOI: 10.1109/ICCSP.2018.8523834http://dx.doi.org/10.1109/ICCSP.2018.8523834]
Buades A, Lisani J L, Petro A B and Sbert C. 2020. Backlit images enhancement using global tone mappings and image fusion. IET Image Processing, 14(2): 211-219 [DOI: 10.1049/iet-ipr.2019.0814]
Chang Y K, Jung C, Ke P, Song H and Hwang J. 2018. Automatic contrast limited adaptive histogram equalization with dual gamma correction. IEEE Access, 6: 11782-11792 [DOI: 10.1109/ACCESS.2018.2797872]
Chen Q F, Li D Z Y and Tang C K. 2013. KNN matting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(9): 2175-2188 [DOI: 10.1109/TPAMI.2013.18]
Chuang Y Y, Curless B, Salesin D H and Szeliski R. 2001. A Bayesian approach to digital matting//Proceedings of 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Kauai, USA: IEEE: 264-271 [DOI: 10.1109/CVPR.2001.990970http://dx.doi.org/10.1109/CVPR.2001.990970]
Guo C L, Li C Y, Guo J C, Loy C C, Hou J H, Kwong S and Cong R M. 2020. Zero-reference deep curve estimation for low-light image enhancement//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE: 1777-1786 [DOI: 10.1109/CVPR42600.2020.00185http://dx.doi.org/10.1109/CVPR42600.2020.00185]
Hou Q Q and Liu F. 2019. Context-aware image matting for simultaneous foreground and alpha estimation//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul, Korea (South): IEEE: 4129-4138 [DOI: 10.1109/ICCV.2019.00423http://dx.doi.org/10.1109/ICCV.2019.00423]
Jobson D J, Rahman Z and Woodell G A. 1997a. Properties and performance of a center/surround Retinex. IEEE Transactions on Image Processing, 6(3): 451-462 [DOI: 10.1109/83.557356]
Jobson D J, Rahman Z and Woodell G A. 1997b. A multiscale Retinex for bridging the gap between color images and the human observation of scenes. IEEE Transactions on Image Processing, 6(7): 965-976 [DOI: 10.1109/83.597272]
Ju M Y, Ding C, Ren W Q, Yang Y, Zhang D Y and Guo Y J. 2021. IDE: image Dehazing and exposure using an enhanced atmospheric scattering model. IEEE Transactions on Image Processing, 30: 2180-2192 [DOI: 10.1109/TIP.2021.3050643]
Li Z H, Cheng K and Wu X L. 2015. Soft binary segmentation-based backlit image enhancement//Proceedings of the 17th IEEE International Workshop on Multimedia Signal Processing. Xiamen, China: IEEE: 1-5 [DOI: 10.1109/MMSP.2015.7340808http://dx.doi.org/10.1109/MMSP.2015.7340808]
Li Z H and Wu X L. 2018. Learning-based restoration of backlit images. IEEE Transactions on Image Processing, 27(2): 976-986 [DOI: 10.1109/TIP.2017.2771142]
Lisani J L. 2018. An analysis and implementation of the shape preserving local histogram modification algorithm. Image Processing on Line, 8: 408-434 [DOI: 10.5201/ipol.2018.236]
Lyu F F, Li Y and Lu F. 2021. Attention guided low-light image enhancement with a large scale low-light simulation dataset. International Journal of Computer Vision, 129(7): 2175-2193 [DOI: 10.1007/s11263-021-01466-8]
Man L, Zhao Y and Wang H X. 2017. Improved nonlinear brightness-lifting model for restoring backlight images. Journal of Computer Applications, 37(2): 564-568
满乐, 赵钰, 王好贤. 2017. 改进非线性亮度提升模型的逆光图像恢复. 计算机应用, 37(2): 564-568 [DOI: 10.11772/j.issn.1001-9081.2017.02.0564]
Mittal A, Soundararajan R and Bovik A C. 2013. Making a "Completely Blind" image quality analyzer. IEEE Signal Processing Letters, 20(3): 209-212 [DOI: 10.1109/LSP.2012.2227726]
Moorthy A K and Bovik A C. 2010. A two-step framework for constructing blind image quality indices. IEEE Signal Processing Letters, 17(5): 513-516 [DOI: 10.1109/LSP.2010.2043888]
Roberts J W, van Aardt J A and Ahmed F B. 2008. Assessment of image fusion procedures using entropy, image quality, and multispectral classification. Journal of Applied Remote Sensing, 2(1): #023522 [DOI: 10.1117/1.2945910]
Trongtirakul T, Chiracharit W and Agaian S S. 2020. Single backlit image enhancement. IEEE Access, 8: 71940-71950 [DOI: 10.1109/ACCESS.2020.2987256]
Wang Q H, Fu X Y, Zhang X P and Ding X H. 2016. A fusion-based method for single backlit image enhancement//Proceedings of 2016 IEEE International Conference on Image Processing. Phoenix, USA: IEEE: 4077-4081 [DOI: 10.1109/ICIP.2016.7533126http://dx.doi.org/10.1109/ICIP.2016.7533126]
Wang R X, Zhang Q, Fu C W, Shen X Y, Zheng W S and Jia J Y. 2019. Underexposed photo enhancement using deep illumination estimation//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE: 6842-6850 [DOI: 10.1109/CVPR.2019.00701http://dx.doi.org/10.1109/CVPR.2019.00701]
Wei C, Wang W J, Yang W H and Liu J Y. 2018. Deep Retinex decomposition for low-light enhancement[EB/OL]. [2021-08-24].https://arxiv.org/pdf/1808.04560.pdfhttps://arxiv.org/pdf/1808.04560.pdf
Xu K, Yang X, Yin B C and Lau R W H. 2020. Learning to restore low-light images via decomposition-and-enhancement//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE: 2278-2287 [DOI: 10.1109/CVPR42600.2020.00235http://dx.doi.org/10.1109/CVPR42600.2020.00235]
Xu N, Price B, Cohen S and Huang T. 2017. Deep image matting//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 311-320 [DOI: 10.1109/CVPR.2017.41http://dx.doi.org/10.1109/CVPR.2017.41]
Yang W H, Wang W J, Huang H F, Wang S Q and Liu J Y. 2021. Sparse gradient regularized deep Retinex network for robust low-light image enhancement. IEEE Transactions on Image Processing, 30: 2072-2086 [DOI: 10.1109/TIP.2021.3050850]
Yuan F N, Li Z Q, Shi J T, Xia X and Li Y. 2021. Image defogging algorithm using a two-phase feature extraction strategy. Journal of Image and Graphics, 26(3): 568-580
袁非牛, 李志强, 史劲亭, 夏雪, 李雅. 2021. 两阶段特征提取策略的图像去雾. 中国图象图形学报, 26(3): 568-580 [DOI: 10.11834/jig.200057]
Zhu R, Zhu L and Li D N. 2015. Study of color heritage image enhancement algorithms based on histogram equalization. Optik, 126(24): 5665-5667 [DOI: 10.1016/j.ijleo.2015.08.169]
相关作者
相关机构
京公网安备11010802024621