浏览全部资源
扫码关注微信
纸质出版日期： 2023-09-16 ，
移动端阅览
朱仲杰， 崔伟锋， 白永强， 井维一， 金敏红. 2023. 宏微观信息增强与色彩校正的高效色调映射. 中国图象图形学报， 28(09):2833-2843
Zhu Zhongjie， Cui Weifeng， Bai Yongqiang， Jing Weiyi， Jin Minhong. 2023. Efficient tone mapping via macro and micro information enhancement and color correction. Journal of Image and Graphics， 28(09):2833-2843
目的
2
色调映射是一种在保持视觉效果基本不变的前提下将高动态范围图像映射到常规低动态显示设备上进行显示的技术。针对现有方法存在细节模糊、边缘光晕及色彩失真等不足，提出一种宏微观信息增强与色彩校正的色调映射新方法。
方法
2
将给定的高动态范围图像映射到HSV（hue， saturation， value）颜色空间，分离亮度信息与色彩信息。基于人类视觉感知机制，在亮度通道构建宏观一致性和微观显著性的亮度感知压缩模型，并进一步通过调节模型缩放因子消除边缘光晕现象。基于颜色恒常性原理，在色度通道构建自适应饱和度偏移模型，融合亮度压缩信息调整图像的饱和度信息，解决色调映射所造成的主观色彩失真问题。
结果
2
实验结果表明，所提算法在结构保真度、自然度和色调映射质量指数等客观评价方面均优于对比色调映射算法，同时主观平均意见值也取得了最高的4.3分（即好—非常好）。
结论
2
宏微观信息增强的亮度感知压缩模型，在确保场景亮度信息不变的情况下，可以有效增强图像纹理细节的完整性和保真性。融合亮度压缩的饱和度偏移模型可以有效解决亮度压缩导致的图像色彩失真等问题。该色调映射算法效率高、通用性强，可广泛应用于图像压缩、生物医学和视频编码等领域。
Objective
2
The traditional 8-bit images cannot accurately store and represent the real natural scene because the brightness variations in reality are very wide， ranging from faint starlight to direct sunlight with more than nine orders of magnitude. High dynamic range （HDR） imaging technology adopts floating-point numbers to address this deficiency. This technology can accurately represent the fidelity of a real scene with abundant brightness and chroma information. However， HDR images cannot be rendered directly on conventional display devices. Tone mapping （TM） technology aims to convert HDR images into traditional images while preserving the natural scene without losing information.Many excellent TM operators have emerged and have been widely used in business. However， the scene information is inevitably lost in different degrees because of the large-scale transformation and compression of the brightness range. In particular， even the state-of-art TM operators for complex scenes still have some problems， such as blurred details， edge halation， brightness imbalance， and color distortion， which seriously affect the subjective feeling of human eyes. Hence， a novel TM algorithm is proposed in this study via macro and micro information enhancement and color correction.
Method
2
Targeted algorithm structures with different strategies for the brightness and chroma domains are constructed in this study based on the human visual perception mechanism. First， an HDR image is converted from RGB color space to HSV color space， and the independent luminance information and chrominance information can be separated effectively. Thus， the subsequent processing can be performed smoothly without mutual interference. Second， different processing and optimization strategies are adopted for the brightness and chroma channels， respectively. For the former， the brightness range is greatly compressed to meet the demand of low dynamic range images while enhancing the detailed information perceived by human eyes from the macro and micro points of view. In particular， the brightness channel is divided into the basic and detail layers through the weighted guidance filter. The basic layer is compressed and combined with the macro statistical information to reduce the brightness contrast of the image and ensure the authenticity and integrity of the image background information and the overall structure. Subsequently， the salient region of the real scene is extracted by the gray-level co-occurrence matrix based on the human eye attention mechanism. According to the saliency information distribution， the texture information of the detail layer is enhanced. The edge halation is further eliminated by adjusting the scaling factor. Finally， the compressed base layer and enhanced detail layer are linearly fused to the targeted brightness channel while ensuring macro consistency and micro significance with the HDR image. For the chroma channel， a saturation migration model is designed with integrating brightness compression. This model can adaptively adjust the information saturation with brightness variety while keeping the hue information unchanged. According to the principle of color constancy， people’s perception of the color of the object’s surface remains unchanged when the color light irradiating the object’s surface changes. Moreover， experience shows that different saturation levels directly affect people’s subjective perception of color， even if the hue information remains unchanged. Therefore， a median shift model is constructed to adjust the image chromaticity saturation adaptively by combining the changes in the statistical information of brightness compression. Thus， the constancy of object surface color can be ensured， and the subjective color distortion caused by information compression of the luminance channel can be effectively avoided. The main experiments include the establishment of a database containing nearly 200 HDR images with different brightness dynamic ranges， light and dark area distribution， and detail richness. This database is used to verify the feasibility and generalization of the proposed algorithm.
Result
2
Experimental results show that the proposed algorithm is superior to the existing TM algorithms in subjective and objective evaluations. In terms of objective evaluation， the images are scored using the TM quality index （TMQI）. The comprehensive evaluation score is obtained by calculating the naturalness and structural fidelity. Compared with the algorithms in the reviewed studies， the proposed algorithm exhibits a comprehensive TMQI score reaching the highest score of 0.862 9. The proposed algorithm is also superior to most of the existing methods in terms of naturalness and structural fidelity. For the subjective evaluation， we refer to the international mean opinion score standard， with scores ranging from 1 to 5， indicating the worst to the best. The subjects score the test images according to their personal preferences， which are combined with the images’ texture details， edge halation， brightness imbalance， and color distortion. The scores of 20 subjects， including 10 men and 10 women， are counted. Results show that the subjects give four points to most images mapped by the proposed algorithm， and a few images achieve the best five points. In particular， the average score of the proposed algorithm reaches the highest 4.3 points.
Conclusion
2
In this study， a brightness perception compression model with macro consistency and micro significance is constructed in the brightness channel. Thus， the drawbacks of the existing TM algorithms， such as the loss of detail texture information， edge halation， and brightness imbalance， can be effectively solved. Moreover， a saturation migration model is designed by integrating brightness compression in the chroma channel， effectively solving the color distortion caused by brightness compression. The experimental comparison results indicate that the TM algorithm via macro and micro information enhancement and color correction proposed in this study is better than the existing TM algorithms. Moreover， the proposed algorithm provides a beneficial condition for us to conduct high dynamic image generation and high dynamic video coding.
高动态范围图像色调映射（TM）宏微观信息增强显著性图像色彩校正
high dynamic range imagetone mapping （TM）macro and micro information enhancementsalient imagecolor correction
Agarwal V， Abidi B， Koschan A and Abidi M A. 2006. An overview of color constancy algorithms. Journal of Pattern Recognition Research， 1（1）： 42-54 ［DOI： 10.13176/11.9http://dx.doi.org/10.13176/11.9］
Barnard K， Cardei V and Funt B. 2002. A comparison of computational color constancy algorithms. I： methodology and experiments with synthesized data. IEEE Transactions on Image Processing， 11（9）： 972-984 ［DOI： 10.1109/TIP.2002.802531http://dx.doi.org/10.1109/TIP.2002.802531］
Cheng R Q， Yu Y， Shi D Z and Cai W. 2022. The critical review of image and video quality assessment methods. Journal of Image and Graphics， 27（5）： 1410-1429
程茹秋， 余烨， 石岱宗， 蔡文. 2022. 图像与视频质量评价综述. 中国图象图形学报， 27（5）： 1410-1429 ［DOI： 10.11834/jig.210314http://dx.doi.org/10.11834/jig.210314］
Eilertsen G， Mantiuk R K and Unger J. 2016. Real-time noise-aware tone-mapping and its use in luminance retargeting//Proceedings of 2016 IEEE International Conference on Image Processing. Phoenix， USA： IEEE： 894-898 ［DOI： 10.1109/ICIP.2016.7532486http://dx.doi.org/10.1109/ICIP.2016.7532486］
Fahim M A N I and Jung H Y. 2020. Fast single-image HDR tone-mapping by avoiding base layer extraction. Sensors， 20（16）： #4378 ［DOI： 10.3390/s20164378http://dx.doi.org/10.3390/s20164378］
Farbman Z， Fattal R， Lischinski D and Szeliski R. 2008. Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Transactions on Graphics， 27（3）： 1-10 ［DOI： 10.1145/1360612.1360666http://dx.doi.org/10.1145/1360612.1360666］
Gao X H， Brooks S and Arnold D V. 2016. Automatic blended tone mapping through evolutionary optimization//Proceedings of 2016 IEEE Congress on Evolutionary Computation. Vancouver， Canada： IEEE： 3855-3862 ［DOI： 10.1109/CEC.2016.7744278http://dx.doi.org/10.1109/CEC.2016.7744278］
Gijsenij A， Gevers T and van de Weijer J. 2011. Computational color constancy： survey and experiments. IEEE Transactions on Image Processing， 20（9）： 2475-2489 ［DOI： 10.1109/TIP.2011.2118224http://dx.doi.org/10.1109/TIP.2011.2118224］
Gu B， Li W J， Zhu M Y and Wang M H. 2013. Local edge-preserving multiscale decomposition for high dynamic range image tone mapping. IEEE Transactions on Image Processing， 22（1）： 70-79 ［DOI： 10.1109/TIP.2012.2214047http://dx.doi.org/10.1109/TIP.2012.2214047］
Guo J C， Yue H H， Zhang Y， Liu D， Liu X W and Zheng S D. 2022. The analysis of image enhancement on salient object detection. Journal of Image and Graphics， 27（7）： 2129-2147
郭继昌， 岳惠惠， 张怡， 刘迪， 刘晓雯， 郑司达. 2022. 图像增强对显著性目标检测的影响研究. 中国图象图形学报， 27（7）： 2129-2147 ［DOI： 10.11834/jig.200735http://dx.doi.org/10.11834/jig.200735］
Hana F M and Maulida I D. 2021. Analysis of contrast limited adaptive histogram equalization （CLAHE） parameters on finger knuckle print identification. Journal of Physics： Conference Series， 1764（1）： #012049 ［DOI： 10.1088/1742-6596/1764/1/012049http://dx.doi.org/10.1088/1742-6596/1764/1/012049］
He K M， Sun J and Tang X O. 2013. Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence， 35（6）： 1397-1409 ［DOI： 10.1109/TPAMI.2012.213http://dx.doi.org/10.1109/TPAMI.2012.213］
He W and Pan C. 2022. The salient object detection based on attention-guided network. Journal of Image and Graphics， 27（4）： 1176-1190
何伟， 潘晨. 2022. 注意力引导网络的显著性目标检测. 中国图象图形学报， 27（4）： 1176-1190 ［DOI： 10.11834/jig.200658http://dx.doi.org/10.11834/jig.200658］
Iizuka S， Simo-Serra E and Ishikawa H. 2017. Globally and locally consistent image completion. ACM Transactions on Graphics， 36（4）： #107 ［DOI： 10.1145/3072959.3073659http://dx.doi.org/10.1145/3072959.3073659］
Jung C and Xu K Q. 2017. Naturalness-preserved tone mapping in images based on perceptual quantization//Proceedings of 2017 IEEE International Conference on Image Processing. Beijing， China： IEEE： 2403-2407 ［DOI： 10.1109/ICIP.2017.8296713http://dx.doi.org/10.1109/ICIP.2017.8296713］
Khan I R， Rahardja S， Khan M M， Movania M M and Abed F. 2018. A tone-mapping technique based on histogram using a sensitivity model of the human visual system. IEEE Transactions on Industrial Electronics， 65（4）： 3469-3479 ［DOI： 10.1109/TIE.2017.2760247http://dx.doi.org/10.1109/TIE.2017.2760247］
Larson G W， Rushmeier H and Piatko C. 1997. A visibility matching tone reproduction operator for high dynamic range scenes. IEEE Transactions on Visualization and Computer Graphics， 3（4）： 291-306 ［DOI： 10.1109/2945.646233http://dx.doi.org/10.1109/2945.646233］
Lee D H， Fan M， Kim S W， Kang M C and Ko S J. 2018. High dynamic range image tone mapping based on asymmetric model of retinal adaptation. Signal Processing： Image Communication， 68： 120-128 ［DOI： 10.1016/j.image.2018.07.008http://dx.doi.org/10.1016/j.image.2018.07.008］
Li H， Jia X X and Zhang L. 2018. Clustering based content and color adaptive tone mapping. Computer Vision and Image Understanding， 168： 37-49 ［DOI： 10.1016/j.cviu.2017.11.001http://dx.doi.org/10.1016/j.cviu.2017.11.001］
Li J， Tian Y H， Huang T J and Gao W. 2010. Probabilistic multi-task learning for visual saliency estimation in video. International Journal of Computer Vision， 90（2）： 150-165 ［DOI： 10.1007/s11263-010-0354-6http://dx.doi.org/10.1007/s11263-010-0354-6］
Li M X and Zhai D H. 2020. Significance detection method with irregular pixel clusters. Journal of Image and Graphics， 25（9）： 1837-1847
李明旭， 翟东海. 2020. 不规则像素簇显著性检测算法. 中国图象图形学报， 25（9）： 1837-1847 ［DOI： 10.11834/jig.190587http://dx.doi.org/10.11834/jig.190587］
Liang Z T， Xu J， Zhang D， Cao Z S and Zhang L. 2018. A hybrid 11-10 layer decomposition model for tone mapping//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City， USA： IEEE： 4758-4766 ［DOI： 10.1109/CVPR.2018.00500http://dx.doi.org/10.1109/CVPR.2018.00500］
Liu J P， Liu H， Gong S B， Tang Z H， Xie Y F， Yin H Z and Niyoyita J P. 2021. Automated cardiac segmentation of cross-modal medical images using unsupervised multi-domain adaptation and spatial neural attention structure. Medical Image Analysis， 72： #102135 ［DOI： 10.1016/J.MEDIA.2021.102135http://dx.doi.org/10.1016/J.MEDIA.2021.102135］
Lu S J， Tan C and Lim J H. 2014. Robust and efficient saliency modeling from image Co-occurrence histograms. IEEE Transactions on Pattern Analysis and Machine Intelligence， 36（1）： 195-201 ［DOI： 10.1109/TPAMI.2013.158http://dx.doi.org/10.1109/TPAMI.2013.158］
Lu S and Lim J H. 2012. Saliency modeling from image histograms//Proceedings of the 12th Europeain Conference on Computer Vision. Florence， Italy：［s.l.］：321-332. ［DOI：10.1007/978-3-642-33786-4_24http://dx.doi.org/10.1007/978-3-642-33786-4_24］
Ma L and Zhang X H. 2014. Relationship between saturation and brightness value in HSV color space. Journal of Computer-Aided Design and Computer Graphics， 26（8）： 1272-1278
马玲， 张晓辉. 2014. HSV颜色空间的饱和度与明度关系模型. 计算机辅助设计与图形学学报， 26（8）： 1272-1278
Manchanda M and Sharma R. 2017. Fusion of visible and infrared images in HSV color space//Proceedings of the 3rd International Conference on Computational Intelligence and Communication Technology. Ghaziabad， India： IEEE： 1-6 ［DOI： 10.1109/CIACT.2017.7977384http://dx.doi.org/10.1109/CIACT.2017.7977384］
Mantiuk R， Mantiuk R， Tomaszewska A and Heidrich W. 2009. Color correction for tone mapping. Computer Graphics Forum， 28（2）： 193-202 ［DOI： 10.1111/j.1467-8659.2009.01358.xhttp://dx.doi.org/10.1111/j.1467-8659.2009.01358.x］
Marnerides D， Bashford-Rogers T and Debattista K. 2021. Deep HDR hallucination for inverse tone mapping. Sensors， 21（12）： #4032 ［DOI： 10.3390/s21124032http://dx.doi.org/10.3390/s21124032］
Miao D S， Zhu Z J， Bai Y Q， Jiang G Y and Duan Z Y. 2019. Novel tone mapping method via macro-micro modeling of human visual system. IEEE Access， 7： 118359-118369 ［DOI： 10.1109/ACCESS.2019.2937211http://dx.doi.org/10.1109/ACCESS.2019.2937211］
Ou Y F， Ambalathankandy P， Takamaeda S， Motomura M， Asai T and Ikebe M. 2022. Real-time tone mapping： a survey and cross-implementation hardware benchmark. IEEE Transactions on Circuits and Systems for Video Technology， 32（5）： 2666-2686 ［DOI： 10.1109/TCSVT.2021.3060143http://dx.doi.org/10.1109/TCSVT.2021.3060143］
Shao H and Yu M. 2017. A perception-inspired tone mapping algorithm based on components decomposition. Optical Technique， 43（5）： 415-422
邵华， 郁梅. 2017. 一种人眼感知驱动的成分分解色调映射算法. 光学技术， 43（5）： 415-422 ［DOI： 10.13741/j.cnki.11-1879/o4.2017.05.006http://dx.doi.org/10.13741/j.cnki.11-1879/o4.2017.05.006］
Su C C， Wang R， Lin H J， Liu Y L， Chen C P， Chang Y L and Pei S C. 2021. Explorable tone mapping operators//Proceedings of the 25th International Conference on Pattern Recognition. Milan， Italy： IEEE： 10320-10326 ［DOI： 10.1109/ICPR48806.2021.9412070http://dx.doi.org/10.1109/ICPR48806.2021.9412070］
Yang J， Lin M C， Liu Z Y， Shahnovich U and Yadid-Pecht O. 2021. Mobile-end tone mapping based on integral image and integral histogram ［EB/OL］. ［2022-05-31］. https：//arxiv.org/pdf/2102.01289.pdfhttps://arxiv.org/pdf/2102.01289.pdf
Yeganeh H and Wang Z. 2013. Objective quality assessment of tone-mapped images. IEEE Transactions on Image Processing， 22（2）： 657-667 ［DOI： 10.1109/TIP.2012.2221725http://dx.doi.org/10.1109/TIP.2012.2221725］
Zhao X Y and Sun B Y. 2022. An attention mechanism and contextual information based low-light image enhancement method. Journal of Image and Graphics， 27（5）： 1565-1576
赵兴运， 孙帮勇. 2022. 融合注意力机制和上下文信息的微光图像增强. 中国图象图形学报， 27（5）： 1565-1576 ［DOI： 10.11834/jig.210583http://dx.doi.org/10.11834/jig.210583］
相关作者
相关机构
京公网安备11010802024621