浏览全部资源
扫码关注微信
Published： 16 November 2023 ，
移动端阅览
闫晓阳， 王华珂， 侯兴松， 顿玉洁. 2023. YCbCr空间分治的双分支低照度图像增强网络. 中国图象图形学报， 28(11):3415-3427
Yan Xiaoyang， Wang Huake， Hou Xingsong， Dun Yujie. 2023. Dual-branch low-light image enhancement network via YCbCr space divide-and-conquer. Journal of Image and Graphics， 28(11):3415-3427
目的
2
现有的低照度图像增强算法通常在RGB颜色空间采用先增强后去噪的方式提升对比度并抑制噪声，由于亮度失真和噪声在RGB颜色空间存在复杂的耦合关系，往往导致增强结果不理想。先增强后去噪的方式也放大了原本隐藏在黑暗中的噪声，使去噪变得困难。为有效处理亮度失真并抑制噪声，提出了一个基于YCbCr颜色空间的双分支低照度图像增强网络，以获得正常亮度和具有低噪声水平的增强图像。
方法
2
由于YCbCr颜色空间可以分离亮度信息与色度信息，实现亮度失真和噪声的解耦，首先将低照度图像由RGB颜色空间变换至YCbCr颜色空间，然后设计一个双分支增强网络，该网络包含亮度增强模块和噪声去除模块，分别对亮度信息和色度信息进行对比度增强和噪声去除，最后使用亮度监督模块和色度监督模块强化亮度增强模块和噪声去除模块的功能，确保有效地提升对比度和去除噪声。
结果
2
在多个公开可用的低照度图像增强数据集上测试本文方法的有效性，对比经典的低照度图像增强算法，本文方法生成的增强图像细节更加丰富、颜色更加真实，并且含有更少噪声，在LOL（low-light dataset）数据集上，相比经典的KinD++（kindling the darkness），峰值信噪比（peak signal-to-noise ratio，PSNR）提高了3.09 dB，相比URetinex（Retinex-based deep unfolding network），PSNR提高了2.74 dB。
结论
2
本文提出的空间解耦方法能够有效地分离亮度失真与噪声，设计的双分支网络分别用于增强亮度和去除噪声，能够有效地解决低照度图像中亮度与噪声的复杂耦合问题，获取低噪声水平的亮度增强图像。
Objective
2
The images acquired at night or backlight conditions always have poor visibility and have details that are hidden in the dark. Moreover， due to insufficient lighting and limited exposure time， the number of incident photons on these images decreases， thereby resulting in a large amount of non-negligible noise. Therefore， improving the contrast and removing noise from low-light images present a challenge. The existing low-light image enhancement algorithms usually enhance the contrast and suppress the noise in the RGB color space by way of enhancing and then denoising. However， due to the complex coupling relationship between brightness distortion and noise in the RGB space， enhancing-and-then-denoising methods usually amplify the noise that is originally hidden in the dark， thus increasing the difficulty of the denoising task， affecting the aesthetic quality of images， and constraining subsequent image processing tasks on the aspects of image classification， object detection， and recognition. To effectively deal with brightness distortion and noise， a dual-branch low-light image enhancement network based on YCbCr space is proposed in this paper to yield enhanced images with minimal color distortion and less noise.
Method
2
The YCbCr space can separate luminance information from chrominance information. Experiments show that brightness distortion is mostly observed in luminance information and that chrominance information is heavily polluted with noise. This paper designs a corresponding network structure based on the divide-and-conquer method to deal with different degradation modes， where different modules can effectively deal with a specific distortion and reduce the difficulty of network learning. A novel dual-branch network in YCbCr space is then developed to realize the decoupling of luminance distortion and noise， which can enhance the luminance information and denoise the chrominance information. First， the low-light images are transformed from the RGB space to the YCbCr space. Second， the images in the YCbCr space are inputted into the proposed dual-branch enhancement network for low-light image enhancement. This network comprises a luminance enhancement module， a noise removal module， and a fusion module. The luminance enhancement module performs contrast enhancement on the luminance information， the noise removal module denoises the chrominance information， and the fusion module combines the features of luminance and chrominance to obtain the final enhanced results. The luminance channel Y and chrominance channels Cb and Cr serve as inputs of the luminance enhancement and noise removal modules， respectively. Given that the luminance distortion is often non-local， the luminance enhancement module adopts a U-shaped network that can obtain the rich brightness context information of the image via an encoder and decoder. Specifically， in the U-Net network， the encoder expands the receptive fields of convolutions through a pooling operation， and at the bottleneck layer， the large receptive field can extract non-local luminance information for contrast enhancement. This non-local information can then be extended to the global scale by up-sampling at the decoder layer by layer. Given that noise damages the texture of low-light images， the noise removal module uses a multi-scale denoising network to enrich the details of the image and effectively learn its multi-scale features. The noise removal module consists of a multi-residual channel attention block， pixel detection， and pixel shuffle. The residual channel attention block， which is commonly found in several networks， is suitable for feature extraction. Instead of up-sampling and down-sampling， the pixel unshuffled and shuffle operations are used to obtain multi-scale input images， remove noise at multi-scale features， fuse multiple features， and eventually obtain valuable features for denoising while avoiding information loss in the images. The brightness supervision and chrominance supervision modules are also used to strengthen the functions of the brightness enhancement and noise removal modules， which can help boost contrast and remove noise effectively. The supervision modules ensure the features of brightness enhancement and noise removal. Several simple convolution operations are then applied to concatenate the outputs of the brightness enhancement and noise removal modules and obtain the final results in the YCbCr space. An additional transform operation is applied to convert the images from the YCbCr space to the RGB space.
Result
2
The effectiveness of the proposed algorithm is tested on several paired and unpaired public low-light image enhancement datasets， such as the low-light dataset （LOL）， vision enhancement in the low-light condition dataset， multi-exposure image fusion dataset， and naturalness preserved enhancement dataset. The proposed method is also compared with several classical low-light enhancement algorithms that have achieved excellent results in low-light image enhancement. These algorithms are implemented using public codes， and their parameters are set by referring to their original papers. These methods are then objectively evaluated based on their visual results and by using several quality metrics， including peak signal-to-noise ratio （PSNR）， structure similarity index， natural image quality evaluator， and learned perceptual image patch similarity. The images generated by the proposed method have richer details， more realistic color， and less noise compared with those generated by the other algorithms. Moreover， on the LOL dataset， compared with kindling the darkness （KinD++）， the PSNR of the proposed method increases by 3.09 dB， while compared with the Retinex-based deep unfolding network that uses Transformer as its basic module， the PSNR of the proposed method increases by 2.74 dB.
Conclusion
2
The proposed spatial decoupling method effectively decouples the brightness distortion and noise of low-light images. The designed dual branch network can enhance the brightness and remove noise， thus effectively solving the complex coupling problem of brightness and noise in low-light images and obtaining noise-free brightness-enhanced images. Extensive experiments demonstrate the effectiveness and generalization of this method， which outperforms the state-of-the-art both qualitatively and quantitatively.
低照度增强YCbCr颜色空间双分支网络噪声去除分治策略
low-light image enhancementYCbCr color spacedual-branch networknoise removaldivide and conquer
Celik T and Tjahjadi T. 2011. Contextual and variational contrast enhancement. IEEE Transactions on Image Processing， 20（12）： 3431-3441 ［DOI： 10.1109/TIP.2011.2157513http://dx.doi.org/10.1109/TIP.2011.2157513］
Dang-Nguyen D T， Pasquini C， Conotter V and Boato G. 2015. RAISE： a raw images dataset for digital image forensics//Proceedings of the 6th ACM Multimedia Systems Conference. Portland， Oregon， USA： ACM： 219-224 ［DOI： 10.1145/2713168.2713194http://dx.doi.org/10.1145/2713168.2713194］
Guo C L， Li C Y， Guo J C， Loy C C， Hou J H， Kwong S and Cong R M. 2020. Zero-reference deep curve estimation for low-light image enhancement//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle， USA： IEEE： 1777-1786 ［DOI： 10.1109/CVPR42600.2020.00185http://dx.doi.org/10.1109/CVPR42600.2020.00185］
Guo X J， Li Y and Ling H B. 2017. LIME： low-light image enhancement via illumination map estimation. IEEE Transactions on Image Processing， 26（2）： 982-993 ［DOI： 10.1109/TIP.2016.2639450http://dx.doi.org/10.1109/TIP.2016.2639450］
Jiang Y F， Gong X Y， Liu D， Cheng Y， Fang C， Shen X H， Yang J C， Zhou P and Wang Z Y. 2021. EnlightenGAN： deep light enhancement without paired supervision. IEEE Transactions on Image Processing， 30： 2340-2349 ［DOI： 10.1109/TIP.2021.3051462http://dx.doi.org/10.1109/TIP.2021.3051462］
Lai W S， Huang J B， Ahuja N and Yang M H. 2019. Fast and accurate image super-resolution with deep Laplacian pyramid networks. IEEE Transactions on Pattern Analysis and Machine Intelligence， 41（11）： 2599-2613 ［DOI： 10.1109/TPAMI.2018.2865304http://dx.doi.org/10.1109/TPAMI.2018.2865304］
Land E H and McCann J J. 1971. Lightness and retinex theory. Journal of the Optical Society of America， 61（1）： 1-11 ［DOI： 10.1364/JOSA.61.000001http://dx.doi.org/10.1364/JOSA.61.000001］
Lee C， Lee C and Kim C S. 2013. Contrast enhancement based on layered difference representation of 2D histograms. IEEE Transactions on Image Processing， 22（12）： 5372-5384 ［DOI： 10.1109/TIP.2013.2284059http://dx.doi.org/10.1109/TIP.2013.2284059］
Li M D， Liu J Y， Yang W H， Sun X Y and Guo Z M. 2018. Structure-revealing low-light image enhancement via robust retinex model. IEEE Transactions on Image Processing， 27（6）： 2828-2841 ［DOI： 10.1109/TIP.2018.2810539http://dx.doi.org/10.1109/TIP.2018.2810539］
Liu J Y， Xu D J， Yang W H， Fan M H and Huang H F. 2021. Benchmarking low-light image enhancement and beyond. International Journal of Computer Vision， 129（4）： 1153-1184 ［DOI： 10.1007/s11263-020-01418-8http://dx.doi.org/10.1007/s11263-020-01418-8］
Lore K G， Akintayo A and Sarkar S. 2017. LLNet： a deep autoencoder approach to natural low-light image enhancement. Pattern Recognition， 61： 650-662 ［DOI： 10.1016/j.patcog.2016.06.008http://dx.doi.org/10.1016/j.patcog.2016.06.008］
Lyu F F， Lu F， Wu J H and Lim C. 2018. MBLLEN： low-light image/video enhancement using CNNs//Proceedings of 2018 British Machine Vision Conference. Newcastle， UK： IEEE： #220
Ma L， Ma T Y and Liu R S. 2022. The review of low-light image enhancement. Journal of Image and Graphics， 27（5）： 1392-1409
马龙， 马腾宇， 刘日升. 2022. 低光照图像增强算法综述. 中国图象图形学报， 27（5）： 1392-1409 ［DOI： 10.11834/jig.210852http://dx.doi.org/10.11834/jig.210852］
Mittal A， Soundararajan R and Bovik A C. 2013. Making a “completely blind” image quality analyzer. IEEE Signal Processing Letters， 20（3）： 209-212 ［DOI： 10.1109/LSP.2012.2227726http://dx.doi.org/10.1109/LSP.2012.2227726］
Rahman S， Rahman M M， Abdullah-Al-Wadud M， Al-Quaderi G D and Shoyaib M. 2016. An adaptive gamma correction for image enhancement. EURASIP Journal on Image and Video Processing， 2016： #35 ［DOI： 10.1186/s13640-016-0138-1http://dx.doi.org/10.1186/s13640-016-0138-1］
Ronneberger O， Fischer P and Brox T. 2015. U-Net： convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich， Germany： Springer： 234-241 ［DOI： 10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28］
Wang W， Chen X， Yang C， Li X， Hu X M and Yue T. 2019. Enhancing low light videos by exploring high sensitivity camera noise//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul， Korea （South）： IEEE： 4110-4118 ［DOI： 10.1109/ICCV.2019.00421http://dx.doi.org/10.1109/ICCV.2019.00421］
Wang W J， Wei C， Yang W H and Liu J Y. 2018. GLADNet： low-light enhancement network with global awareness//Proceedings of the 13th IEEE International Conference on Automatic Face and Gesture Recognition. Xi’an， China： IEEE： 751-755 ［DOI： 10.1109/FG.2018.00118http://dx.doi.org/10.1109/FG.2018.00118］
Wang Z， Bovik A C， Sheikh H R and Simoncelli E P. 2004. Image quality assessment： from error visibility to structural similarity. IEEE Transactions on Image Processing， 13（4）： 600-612 ［DOI： 10.1109/TIP.2003.819861http://dx.doi.org/10.1109/TIP.2003.819861］
Wang Z Y， Zhang E H， Shi Z H and Duan J H. 2022. Deep learning based backlight image enhancement method derived of zero-reference samples. Journal of Image and Graphics， 27（5）： 1589-1603
王知音， 张二虎， 石争浩， 段敬红. 2022. 零参考样本下的逆光图像深度学习增强方法. 中国图象图形学报， 27（5）： 1589-1603 ［DOI： 10.11834/jig.210783http://dx.doi.org/10.11834/jig.210783］
Wei C， Wang W J， Yang W H and Liu J Y. 2018. Deep retinex decomposition for low-light enhancement ［EB/OL］. ［2022-10-31］. https://arxiv.org/pdf/1808.04560.pdfhttps://arxiv.org/pdf/1808.04560.pdf
Wu W H， Weng J， Zhang P P， Wang X， Yang W H and Jiang J M. 2022. URetinex-Net： retinex-based deep unfolding network for low-light image enhancement//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans， USA： IEEE： 5891-5900 ［DOI： 10.1109/CVPR52688.2022.00581http://dx.doi.org/10.1109/CVPR52688.2022.00581］
Ying Z Q， Li G and Gao W. 2017. A bio-inspired multi-exposure fusion framework for low-light image enhancement ［EB/OL］. ［2022-10-31］. https://arxiv.org/pdf/1711.00591.pdfhttps://arxiv.org/pdf/1711.00591.pdf
Zamir S W， Arora A， Khan S， Hayat M， Khan F S， Yang M H and Shao L. 2021. Multi-stage progressive image restoration//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville， USA： IEEE： 14816-14826 ［DOI： 10.1109/CVPR46437.2021.01458http://dx.doi.org/10.1109/CVPR46437.2021.01458］
Zhang R， Isola P， Efros A A， Shechtman E and Wang O. 2018. The unreasonable effectiveness of deep features as a perceptual metric//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City， USA： IEEE： 586-595 ［DOI： 10.1109/CVPR.2018.00068http://dx.doi.org/10.1109/CVPR.2018.00068］
Zhang Y H， Guo X J， Ma J Y， Liu W and Zhang J W. 2021. Beyond brightening low-light images. International Journal of Computer Vision， 129（4）： 1013-1037 ［DOI： 10.1007/s11263-020-01407-xhttp://dx.doi.org/10.1007/s11263-020-01407-x］
Zhang Y H， Zhang J W and Guo X J. 2019. Kindling the darkness： a practical low-light image enhancer//Proceedings of the 27th ACM International Conference on Multimedia. Nice， France： ACM： 1632-1640 ［DOI： 10.1145/3343031.3350926http://dx.doi.org/10.1145/3343031.3350926］
Zhao M H， Cheng D N， Du S L， Hu J， Shi C and Shi Z H. 2022. An improved fusion strategy based on transparency-guided backlit image enhancement. Journal of Image and Graphics， 27（5）： 1554-1564
赵明华， 程丹妮， 都双丽， 胡静， 石程， 石争浩. 2022. 改进融合策略下透明度引导的逆光图像增强. 中国图象图形学报， 27（5）： 1554-1564 ［DOI： 10.11834/jig.210739http://dx.doi.org/10.11834/jig.210739］
Zhu A Q， Zhang L， Shen Y， Ma Y， Zhao S J and Zhou Y C. 2020. Zero-shot restoration of underexposed images via robust retinex decomposition//Proceedings of 2020 IEEE International Conference on Multimedia and Expo （ICME）. London， UK： IEEE： 1-6 ［DOI： 10.1109/ICME46284.2020.9102962http://dx.doi.org/10.1109/ICME46284.2020.9102962］
相关文章
相关作者
相关机构
京公网安备11010802024621