浏览全部资源
扫码关注微信
纸质出版日期： 2022-11-16 ，
录用日期： 2021-12-08
移动端阅览
郭嘉骏, 姜求平, 邵枫. 分形理论引导的图像临界差异感知阈值估计[J]. 中国图象图形学报, 2022,27(11):3303-3315.
Jiajun Guo, Qiuping Jiang, Feng Shao. Fractal-guided JND threshold estimation for natural images[J]. Journal of Image and Graphics, 2022,27(11):3303-3315.
目的
2
图像的临界差异（just noticeable difference，JND）阈值估计对提升图像压缩比以及信息隐藏效率具有重要意义。亮度适应性和空域掩蔽效应是决定JND阈值大小的两大核心因素。现有的空域掩蔽模型主要考虑对比度掩蔽和纹理掩蔽两方面。然而，当前采用的纹理掩蔽模型不能有效地描述与纹理粗糙度相关的掩蔽效应对图像JND阈值的影响。对此，本文提出一种基于分形理论的JND阈值估计模型。
方法
2
首先，考虑到人眼视觉系统对具有粗糙表面的图像内容变化具有较低的分辨能力，通过经典的分形理论来计算图像局部区域的分形维数，并以此作为对纹理粗糙度的度量，并在此基础上提出一种新的基于纹理粗糙度的纹理掩蔽模型。然后，将提出的纹理掩蔽模型与传统的亮度适应性相结合估计得到初步的JND阈值。最后，考虑到人眼的视觉注意机制，进一步考虑图像内容的视觉显著性，对JND阈值进行感知一致性修正，估计得到最终的JND阈值。
结果
2
选取4种相关方法进行对比，结果表明，在注入相同甚至更多噪声的情况下，相较于对比方法中的最优结果，本文方法的平均VSI（visual saliency-induced index）和平均MOS（mean opinion score）在LIVE（Laboratory for Image & Video Engineering）图像库上分别提高了0.001 7和50%，在TID 2013（tampere image database 2013）图像库上分别提高了0.001 9和40%，在CSIQ（categorical subjective image quality）图像库上分别提高了0.001 3和9.1%，在基于VVC（versatile video coding）的JND图像库上分别提高了0.000 3和54.5%。此外，作为另一典型应用，开展了感知冗余去除实验。实验结果表明，在保持视觉质量的前提下，经过本文JND模型平滑处理后的图像，其JPEG压缩图像相比于原图直接JPEG压缩得到的图像能节省12.5%的字节数。
结论
2
本文提出的基于分形维数的纹理粗糙度能够有效刻画纹理掩蔽效应，构建的纹理掩蔽效应与传统的空域掩蔽效应相结合能够大幅提升图像JND阈值估计的准确性和可靠性。
Objective
2
The perception of visual information of human visual system has distorted multi-sensitivities due to multiple categories. Current researches have shown that the human visual system is restricted of the changes or distortions in image perception under some threshold circumstances. This kind of threshold is referred to the just noticeable difference (JND) threshold. JND threshold estimation of images is of great significance for many perceptual image processing applications like the optimized image compression and information hiding. Generally
luminance adaptation and spatial masking effect can be as the key factors to identify the JND threshold. The existing spatial masking models have mainly been evolved on two aspects of contrast masking and texture masking. However
the current texture masking model cannot effectively clarify the influence of JND-threshold-oriented texture roughness. Meanwhile
human visual system has lower sensitivities for perceiving the rough-surfaced changes or differences. First
our research is focused on fractal dimension based (FD-based) description using quantitative-oriented image texture roughness in terms of classic fractal theory. A new FD-based texture masking model is demonstrated as well. Regular higher texture-roughness-based targeted areas have stronger texture masking effect due to the lower texture-roughness-derived weaker texture masking effect. Next
our newly texture is developed an improved spatial masking estimation function based on the integration of a masking model and traditional contrast masking. Finally
the improved spatial mask estimation function is combined with luminance adaptability function for the final JND profile. Considering the visual attention mechanism of the human visual system
we also refine the JND threshold using the classical graph based visual saliency (GBVS) method in visual saliency map.
Method
2
First
we get the luminance contrast masking of the original image. Then
we calculate the FD of each block with size 8×8 by the widely-adopted differential box-counting (DBC) method and get a FD map of the original image. Third
the new spatial masking map is obtained via combining the luminance contrast map with the proposed FD map. Forth
we also apply the luminance adaptation map to interpret the influence of luminance on JND. Based on the luminance adaptation map and the new spatial masking map
a direct-multiplication-derived coarse JND map can be obtained by fusing the two maps together. Fifth
considering the visual attention mechanism of human visual system
we refine the coarse JND map using visual saliency. To characterize the influence of visual saliency on JND
we compute the visual saliency map based on the GBVS method and a sigmoid-like control function is formulated. Finally
we can obtain the final JND map by fusing the coarse JND map and the multiplication-based sigmoid-like control function.
Result
2
Our proposed JND profile has the highest visual saliency-induced index(VSI) and the highest mean opinion score(MOS) when injecting the more equivalent noises. Specifically
our model has a 0.001 7 higher average VSI score than the second-best model on Laboratory for Image & Video Engineering(LIVE)
and the MOS is 50% higher. On TID2013
the average VSI of our model is 0.001 9 higher than the second-best model
and the MOS is also 40% higher. On categorical subjective image quality(CSIQ)
the average VSI of our model is 0.001 3 higher than the second-best model
and the MOS is 9.1% higher as well. In addition
we also conduct the perceptual redundancy removal experiments to demonstrate the application capability of the proposed JND profile. The experimental results shown that our JND model can save 12.5% bytes on JPEG compression.
Conclusion
2
Our FD-based JND model can estimate the JND threshold effectively on the texture region. The fractal-based roughness on local areas
we can propose a new texture masking effect to estimate the JND threshold on these areas. To tackle more distorted issues
our model can inject more noise in the areas with high roughness and less noise in the smooth areas.
临界差异(JND)分形维数纹理粗糙度空域掩蔽纹理掩蔽
just noticeable difference (JND)fractal dimensiontexture coarsenessspatial maskingtexture masking
Ahumada A J Jr and Peterson H A. 1997. Luminance-model-based DCT quantization for color image compression. Proceedings of SPIE-The International Society for Optical Engineering, 1666: 365-374[DOI: 10.1117/12.135982]
Bae S H and Kim M. 2017. A DCT-based total JND profile for spatiotemporaland foveated masking effects. IEEE Transactions on Circuits and Systems for Video Technology, 27(6): 1196-1207[DOI: 10.1109/TCSVT.2016.2539862]
Chen Z Z and Guillemot C. 2010. Perceptually-friendly H. 264/AVC video coding based on foveated just-noticeable-distortion model. IEEE Transactions on Circuits and Systems for Video Technology, 20(6): 806-819[DOI: 10.1109/TCSVT.2010.2045912]
Chen Z Z and Wu W. 2020. Asymmetric foveated just-noticeable-difference model for images with visual field inhomogeneities. IEEE Transactions on Circuits and Systems for Video Technology, 30(11): 4064-4074[DOI: 10.1109/TCSVT.2019.2952675]
Chou C H and Li Y C. 1995. A perceptually tuned subband image coder based on the measure of just-noticeable-distortion profile. IEEE Transactions on Circuits and Systems for Video Technology, 5(6): 467-476[DOI: 10.1109/76.475889]
Foley J M. 1994. Human luminance pattern-vision mechanisms: masking experiments require a new model. Journal of the Optical Society of America A, 11(6): 1710-1719[DOI: 10.1364/JOSAA.11.001710]
Hadizadeh H. 2016. A saliency-modulated just-noticeable-distortion model with non-linear saliency modulation functions. Pattern Recognition Letters, 84: 49-55[DOI: 10.1016/j.patrec.2016.08.011]
International Telecommunication Union. 2002. ITU-R BT. 500-11. Methodology for the subjective assessment of the quality of television pictures. Geneva, Switzerland
Larson E C and Chandler D M. 2010. Most apparent distortion: full-reference image quality assessment and the role of strategy. Journal of Electronic Imaging, 19(1): #011006[DOI: 10.1117/1.3267105]
Legge G E and Foley J M. 1980. Contrast masking in human vision. Journal of the Optical Society of America, 70(12): 1458-1471[DOI: 10.1364/JOSA.70.001458]
Liu A M, Lin W S, Paul M, Deng C W and Zhang F. 2010. Just noticeable difference for images with decomposition model for separating edge and textured regions. IEEE Transactions on Circuits and Systems for Video Technology, 20(11): 1648-1652[DOI: 10.1109/TCSVT.2010.2087432]
Macknik S L and Livingstone M S. 1998. Neuronal correlates of visibility and invisibility in the primate visual system. Nature Neuroscience, 1(2): 144-149[DOI: 10.1038/393]
Mandelbrot B B. 1977. Fractals: Form, Chance, and Dimension. San Francisco, USA: W. H. Freeman and Company
Mandelbrot B B. 1982. The Fractal Geometry of Nature. San Francisco, USA: W. H. Freeman and Company
Niu Y Q, Kyan M, Ma L, Beghdadi A andKrishnan S. 2013. Visual saliency's modulatory effect on just noticeable distortion profile and its application in image watermarking. Signal Processing: Image Communication, 28(8): 917-928[DOI: 10.1016/j.image.2012.07.009]
Pentland A P. 1984. Fractal-based description of natural scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-6(6): 661-674[DOI: 10.1109/TPAMI.1984.4767591]
Ponomarenko N, Ieremeiev O, Lukin V, Egiazarian K, Jin L, Astola J, Vozel B, Chehdi K, Carli M, Battisti F and Kuo C C J. 2013. Color image database TID2013: peculiarities and preliminary results//Proceedings of European Workshop on Visual Information Processing. Paris, France: IEEE: 106-111
Sarkar N and Chaudhuri B B. 1994. An efficient differential box-counting approach to compute fractal dimension of image. IEEE Transactions on Systems, Man, and Cybernetics, 24(1): 115-120[DOI: 10.1109/21.259692]
Schölkopf B, Platt J and Hofmann T. 2007. Graph-based visual saliency//Advances in Neural Information Processing Systems 19: Proceedings of 2006 Conference. Vancouver, Canada: MIT Press: 545-552
Sheikh H R, Bovik A C, Cormack L and Wang Z. 2005. LIVE image quality assessment database release 2[DB/OL]. [2021-04-16].http://live.ece.utexas.edu/research/qualityhttp://live.ece.utexas.edu/research/quality
Shen X L, Ni Z K, Yang W H, Zhang X F, Wang S Q and Kwong S. 2021. Just noticeable distortion profile inference: a patch-level structural visibility learning approach. IEEE Transactions on Image Processing, 30: 26-38[DOI: 10.1109/TIP.2020.3029428]
Wang H K, Yu L, Yin H B, Li T S and Wang S W. 2020. An improved DCT-based JND estimation model considering multiple masking effects. Journal of Visual Communication and Image Representation, 71: #102850[DOI: 10.1016/j.jvcir.2020.102850]
Watson A B. 1993. DCTune: a technique for visual optimization of DCT quantization matrices for individual images. Society for Information Display Digest of Technical Papers: 946-949[DOI: 10.2514/6.1993-4512]
Watson A B and Solomon J A. 1997. Model of visual contrast gain control and pattern masking. Journal of the Optical Society of America, 14(9): 2379-2391[DOI: 10.1364/JOSAA.14.002379]
Wei Z Y and Ngan K N. 2009. Spatio-temporal just noticeable distortion profile for grey scale image/video in DCT domain. IEEE Transactions on Circuits and Systems for Video Technology, 19(3): 337-346[DOI: 10.1109/TCSVT.2009.2013518]
Wu J J, Li L D, Dong W S, Shi G N, Lin W S and Kuo C C J. 2017. Enhanced just noticeable difference model for images with pattern complexity. IEEE Transactions on Image Processing, 26(6): 2682-2693[DOI: 10.1109/TIP.2017.2685682]
Wu J J, Lin W S, Shi G M, Wang X T and Li F. 2013a. Pattern masking estimation in image with structural uncertainty. IEEE Transactions on Image Processing, 22(12): 4892-4904[DOI: 10.1109/TIP.2013.2279934]
Wu J J, Shi G M, Lin W S, Liu A M and Qi F. 2013b. Just noticeable difference estimation for images with free-energy principle. IEEE Transactions on Multimedia, 15(7): 1705-1710[DOI: 10.1109/TMM.2013.2268053]
Xu C, Luo T, Jiang G Y, Yu M, Jiang Q P and Xu H Y. 2019. Just distortion threshold estimation on natural images using fusion of structured and unstructured information. Journal of Image and Graphics, 24(9): 1546-1557
许辰, 骆挺, 蒋刚毅, 郁梅, 姜求平, 徐海勇. 2019. 融合结构与非结构信息的自然图像恰可察觉失真阈值估计. 中国图象图形学报, 24(9): 1546-1557[DOI: 10.11834/jig.180631]
Yang X K, Ling W S, Lu Z K, Ong E P and Yao S S. 2005. Just noticeable distortion model and its applications in video coding. Signal Processing: Image Communication, 20(7): 662-680[DOI: 10.1016/j.image.2005.04.001]
Zhang L, Shen Y and Li H Y. 2014. VSI: a visual saliency-induced index for perceptual image quality assessment. IEEE Transactions on Image Processing, 23(10): 4270-4281[DOI: 10.1109/TIP.2014.2346028]
Zhang X H, Lin W S and Xue P. 2005. Improved estimation for just-noticeable visual distortion. Signal Processing, 85(4): 795-808[DOI:10.1016/j.sigpro.2004.12.002]
相关作者
相关机构
京公网安备11010802024621