|
发布时间: 2017-10-16 |
图像处理和编码 |
|
|
收稿日期: 2017-05-09; 修回日期: 2017-06-09
基金项目: 浙江省自然科学基金项目(LY14F020004);国家自然科学基金项目(61003188,61379075);国家科技支撑计划项目(2014BAK14B01);浙江省公益性技术应用研究计划项目(2015C33071);浙江工商大学青年人才基金项目(QZ13-9);浙江省智能交通工程技术研究中心开放课题(2015ERCITZJ-KF1)
第一作者简介:
邵欢(1993-), 男, 现为浙江工商大学计算机技术专业硕士研究生, 主要研究方向为图像处理与模式识别。E-mail:shaohuan93@outlook.com.
中图法分类号: TP391.4
文献标识码: A
文章编号: 1006-8961(2017)10-1364-09
|
摘要
目的 针对目前已有的纹理平滑方法难以在抑制强梯度和尺度变化纹理的同时保持完整结构的问题,提出一种结构识别引导下的纹理抑制图像平滑算法。方法 首先,结构与纹理的根本区别在于重复模式,结构应该是稀疏的,而纹理应该是一个有重复模式的区域,因此,通过对结构/纹理的多尺度分析,提取了对于结构/纹理具有辨别力的多尺度内变差特征;然后,借助支持向量机,对提取的特征样本点训练出一个结构/纹理分类器;就分类结果中存在的结构较粗、毛刺等问题,进一步对分类结果进行细化和剔除毛刺与孤立点的后处理操作,以获得最终的更为精细的结构识别结果;最后,提出结构引导下的自适应双边图像滤波算法,达到既能抑制强梯度和尺度变化的纹理又能保持结构完整性的图像平滑效果。结果 本文提出的多尺度内变差特征在支持向量机训练中达到了96.12%的正确率,结构引导下的图像滤波能够在保持结构的同时,有效地抑制强梯度和尺度变化的纹理细节。结论 本文算法在兼顾结构的保持和强梯度以及尺度变化纹理的抑制方面超越了已有的方法,对于结构提取、细节增强、图像分割、色调映射、图像融合和目标识别等众多技术领域的发展将具有较强的促进作用,体现了潜在的实际应用价值。
关键词
纹理抑制; 结构识别; 多尺度内变差; 强梯度纹理; 尺度变化
Abstract
Objective Natural scenes generally contain different scale objects and textures, which carry rich information in regard to human perception.Texture usually signifies pixel values, which change with high frequency.Generally, images are composed of many important structures, texture, edges, etc.Therefore, mining the meaningful structure from textures or complex background images is a critical task in vision processing.The core of image smoothing lies in the separation of structure and texture.Effective preservation of the structure while suppressing the texture with strong gradient or varying scales is a challenging problem.Most of the existing image smoothing methods tends to deal with weak gradient texture images; if the texture gradient is strong, then these methods will fail.To solve the abovementioned problem, a structure recognition guided texture smoothing algorithm is proposed, which deals with the structure and the texture separately and detect structure before image smoothing. Method First, this paper argues that the fundamental difference between structure and texture is the repetition pattern.Particularly, the structure should be sparse and the texture should be a region with a repeating pattern.According to this characteristic, the discriminative features for distinguishing between structure and texture are designed and extracted based on the multi-scale analysis of inherent variation.At least two reasons are available for presenting the multi-scale approach.One reason is that structure and texture are relative.When the scale is small, the texture may not show up, and thus the scale needs to be enlarged and the essence of the texture is released.The other reason is that the texture in the image is diverse, and the adaptive scale in different regions is difficult.Furthermore, textures with various attributes may exist in the same image, a single scale can only solve the partial texture with the default scale parameter and the recognition of other textures will lose.Therefore, multi-scale analysis of inherent variation is proposed to ensure that different textures can display their own repetitive pattern attributes.Second, the core part in the field of pattern recognition is feature extraction.Therefore, the feature extracted must be more robust to guarantee the discrimination ability is strong enough and the stability is good enough.To obtain more accurate features, we need to consider the multi-scale inherent variation in the macroscopic view and grasp its general rules.After we analyze the trend of multi-scale inherent variation curves at different pixel locations, several discriminative features are extracted.Then, these features can be used for subsequent structural recognition.We regard the separation of texture and structure as a typical two-class issue, and the support vector machine is a classical two-class classification method.Compared with many existent machine learning methods, it is a relatively lightweight classifier, which can obtain desirable classification results without a large sample.Consequently, this paper prefers to use the support vector machine to distinguish the texture and structure, with the help of support vector machine, a classifier is trained with the extracted feature pixels, and utilized to classify structure and non-structure pixels efficiently.However, due to the block effect in edge compression and the computational mechanism of inherent variation, pixels nearby the structure will always be affected by the real structure and its multi-scale inherent variation curve is similar in structure.Hence, the support vector machine classification results cannot reach a single pixel.We observed large amounts of data and find that the non-structured pixel appeared symmetrically on both sides of the window.Although the support vector machine classification results are coarser, in the middle of the skeleton should be considered as the real structure.In this paper, a morphological thinning method is adopted directly to get a thinner structure, but the results of thinning still have some weakness.To dispose the shortcomings of the support vector machine classification results after thinning operation, we design two steps of post-processing work, including outlier rejection and deburring, which solve the burr and mistaken isolate.As such, the finer structure recognition maps can be obtained.Finally, based on the fine structure obtained in the previous step, a structure guided bilateral image smoothing method is put forward to remove texture while preserving structure. Result The multi-scale inherent variation features proposed in this paper achieve a correct rate of 96.12% with support vector machine, and our structure guided image smoothing results can effectively suppress the texture details with strong gradient or varying scales while preserving the structure.These excellent experimental results are compared to some results of previous methods, which reveal that the proposed methodology yields better image smoothing. Conclusion In view of the limitations of existing similar methods, this paper analyzes the characteristics of inherent variation deeply and proposes an algorithm to distinguish the structure and texture by means of multi-scale inherent variations.Based on the support vector machine classification results, a post-processing is used to obtain a finer structure recognition map.Then, a structure guided bilateral image smoothing method is applied to remove texture while preserving structure.Our algorithm outperforms the state-of-the-art image smoothing methods, especially for those images containing texture with strong gradient or varying scales, which could strongly promote such technical fields as structure extraction, detail enhancement, image segmentation, tone mapping, image fusion, and object recognition, which reflect the potential practical application values.
Key words
texture suppressing; structure recognition; multi-scale inherent variation; strong texture; varying scales
0 引言
在人类视觉感知中,图像通常是由结构、纹理和平滑区域三大部分构成。实际上,绝大多数的纹理只是起到丰富细节的作用,结构和平滑区域就足以清晰地表达图像的主要内容,因此保持结构的平滑图像有着很大的潜在应用价值。
图像平滑基本上可分为局部平滑和全局平滑两类。高斯滤波[1]是常用的局部平滑方法,它对去除服从正态分布的噪声很有效,但是不利于保持结构。Tomasi等人[2-4]在此基础上提出了双边滤波,在原有空间核的基础上加入颜色核,同时考虑空域信息和颜色相似性,这一操作使得平滑结果有了很大改善,结构保持较好,但是纹理信息依然无法抑制。Cho等人[5]在双边滤波的框架上提出块偏移的思想,称之为双边纹理滤波,可以抑制部分纹理,但是梯度较强的纹理会被误认为结构保持下来。Subr等人[6]提出了用平均的极值包络线来达到平滑的目的,对纹理抑制的效果有了改善但是整体并不平滑,而且容易产生偏色。Karacan等人[7]提出用区域协方差来区分结构纹理,但是可能会过度平滑结构。Zhang等人[8]通过尺度的深度剖析提出了一种迭代的双边滤波,采用先平滑再恢复结构的方法,但是随着迭代次数增多,结构会出现钝化和严重的偏色。
上述平滑方法基本属于局部的平滑方式,还有一类基于全局优化的角度来实现,要求平滑结果与输入图像差异较小,再加一些约束项保证平滑,两者相互权衡得到最佳结果。Rudin等人[9-10]提出了一种基于全变分的图像平滑方法,它以简单的梯度函数作为约束,对梯度较小的纹理和噪声有一定平滑效果。Farbman等人[11]提出了基于加权最小平方的图像平滑方法,其以梯度的二范式加权和来实现平滑功能,结果依然受限于梯度。Xu等人[12]提出一种基于图像梯度零范数(L0)的全局平滑方法,通过图像梯度的零范数最小化来实现图像的平滑处理,但其过分依赖图像梯度,导致对噪声缺乏鲁棒性,以及平滑的结果存在阶梯效应。Xu等人[13]又提出一种相对全变分(RTV)方法,该方法改进了原来的全变分(TV),考虑了纹理和结构在重复模式方面的差别,可以抑制强梯度的纹理,但无法抑制尺度变化的纹理以及保持结构完整。
综上所述,很多方法无法在抑制纹理的同时保持结构,为了解决该难题,提出了一种新的基于结构识别的图像平滑方法,通过结构的引导进行双边图像滤波,实现既能抑制强梯度和尺度变化的纹理,又能保持结构的效果。
1 本文算法
本文算法首先通过观察多尺度下内变差值
1.1 基于多尺度内变差的特征提取
借用内变差[13]来反映像素点
$ L\left( p \right) = \frac{1}{{{n_c}}}\sum\limits_{c = 1}^{{n_c}} {\sum\limits_{d \in \left\{ {x, y} \right\}} {\frac{{\left| {\sum\limits_{q \in R\left( p \right)} {{g_{p, q}} \cdot {{\left( {{\partial _d}I} \right)}_q}} } \right|}}{{\sum\limits_{q \in R\left( p \right)} {{g_{p, q}}} }}} } $ | (1) |
式中,
$ {g_{p, q}} \propto {\rm{exp}}\left( { - \frac{{{{\left\| {p - q} \right\|}^2}}}{{2{\sigma ^2}}}} \right) $ | (2) |
式中,
$ \sigma = \alpha \cdot {r_l} $ | (3) |
式中,
$ S\left( p \right) = \left\{ {{L_1}\left( p \right), {L_2}\left( p \right), \cdots, {L_N}\left( p \right)} \right\} $ | (4) |
式中,
测试大量的图像后发现,处在不同位置的像素
$ \mathit{\boldsymbol{f}}\left( p \right)\left[ 1 \right] = \left\{ \begin{array}{l} 1\;\;\;\;\;\forall {L_i}\left( p \right) \ge {L_j}\left( p \right)\& \& 0 < i < j \le N\\ 0\;\;\;\;\;其他 \end{array} \right.\\ \;\;\;\;\;\;\mathit{\boldsymbol{f}}\left( p \right)\left[ 2 \right] = {\rm{min}}\left( {{L_i}\left( p \right),i = 1,2, \cdots ,N} \right) $ | (5) |
1.2 基于SVM和后处理的结构识别
支持向量机(SVM)[14]是机器学习领域里的一个经典有效的分类工具,本文用它来进行结构和非结构(包括纹理和平滑区域)的分类。事先人为地挑选出样本,特别是一些具有挑战性的像素点。图 4显示了两幅样本图像。为了便于观察,在梯度图上做了标记,结构和非结构像素分别标记为黄色和绿色。实验中挑选多幅图像的共5 000个正样本点和5 000个负样本点,提取它们的特征向量
由于内变差
1.3 结构引导下的双边图像滤波
双边滤波器[5]是一种用于平滑图像和减少噪声的边缘保持滤波器。以此为基础,本文提出了一种结构引导下的双边图像滤波算法。首先,对于一幅输入图像
$ {G_p} = \frac{1}{k}\sum\limits_{q \in \Omega \left( p \right)} {{\rm{exp}}\left\{ { - \frac{1}{2}\left( {\frac{{{{\left( {{p_x} - {q_x}} \right)}^2} + {{\left( {{p_y} - {q_y}} \right)}^2}}}{{K_p^2}}} \right)} \right\}} {I_q} $ | (6) |
式中,
在获得引导图
$ {S_p} = \frac{1}{k}\sum\limits_{q \in N\left( p \right)} {{g_{{\sigma _s}}}\left( {p, q} \right){g_{{\sigma _r}}}\left( {{G_p}, {G_q}} \right)} {I_q} $ | (7) |
式中,
2 实验结果与讨论
本文算法采用MATLAB R2012a实现,PC处理器为Intel® CoreTM i5-3.2 GHz,内存4 GB,操作系统为64位Windows 7。
图 7和图 8展示了本文算法与L0[12]、RTV[13]、BTF[5]以及Zhang等人[8]方法的运行结果。已有方法的运行结果使用了相应作者给出的代码,并通过调节参数使其达到最优。从给出的4个实例可以看出,对于强梯度以及尺度变化的纹理(如实例1的背景,实例2的花纹,实例3的木板和实例4的叶子和背景),L0对于强梯度纹理几乎全部保留下来;RTV可以抑制部分密集规则的纹理,但强梯度纹理会被当作结构保持下来;BTF多次迭代后对强梯度纹理的平滑有一定效果,但是同时会过度平滑结构;Zhang等人[8]的方法中,结构被模糊并且出现严重的偏色。本文算法在平滑具有强梯度和尺度变化的纹理图像时,既可以有效平滑纹理,又能保持结构信息,显示出了更好的效果。图 9展示了更多不同类型的图像平滑结果,也表明本文算法具有很强的适用性。
3 结论
针对同类方法的局限性,本文深度剖析内变差的特点,提出基于多尺度的内变差来提取特征以区分结构和纹理,然后借助SVM并设计了后处理手段,得到精细的结构识别图,最后提出结构引导下的双边图像滤波算法,取得了良好的平滑结果。本文算法在大量不同类型的测试图像上的实验结果均展示了良好的图像平滑效果,显示出具有较强的鲁棒性和普适性。该算法能够在抑制纹理的同时兼顾结构的保持,尤其在强梯度和尺度变化的纹理抑制方面显示了较高的优越性。它可以适用于结构提取、细节增强、图像分割、色调映射、图像融合和目标识别等众多图像处理技术领域中,将对这些技术的发展起到较强的促进作用,体现了潜在的实际应用价值。
然而,本文算法也有不足之处。由于SVM分类结果具有一定的宽度,这对于区分尖锐区域的结构像素带来了一定的难度。另外,多尺度内变差值特征的提取过程比较耗时,导致算法运行时间较长。接下来,希望进一步加入新的有效特征以识别出尖锐区域的精细结构像素,并进一步优化算法效率使其尽量达到实时响应速度。
参考文献
-
[1] Baek J, Jacobs D E. Accelerating spatially varying Gaussian filters[J]. ACM Transactions on Graphics, 2010, 29(6): #169. [DOI:10.1145/1882261.1866191]
-
[2] Tomasi C, Manduchi R.Bilateral filtering for gray and color images[C]//Proceedings of IEEE 6th International Conference on Computer Vision.Bombay, India:IEEE, 1998:839-846.[DOI:10.1109/ICCV.1998.710815]
-
[3] Weiss B. Fast median and bilateral filtering[J]. ACM Transactions on Graphics, 2006, 25(3): 519–526. [DOI:10.1145/1141911.1141918]
-
[4] Paris S, Durand F. A fast approximation of the bilateral filter using a signal processing approach[J]. International Journal of Computer Vision, 2009, 81(1): 24–52. [DOI:10.1007/s11263-007-0110-8]
-
[5] Cho H, Lee H, Kang H, et al. Bilateral texture filtering[J]. ACM Transactions on Graphics, 2014, 33(4): #128. [DOI:10.1145/2601097.2601188]
-
[6] Subr K, Soler C, Durand F. Edge-preserving multiscale image decomposition based on local extrema[J]. ACM Transactions on Graphics, 2009, 28(5): #147. [DOI:10.1145/1618452.1618493]
-
[7] Karacan L, Erdem E, Erdem A. Structure-preserving image smoothing via region covariances[J]. ACM Transactions on Graphics, 2013, 32(6): #176. [DOI:10.1145/2508363.2508403]
-
[8] Zhang Q, Shen X Y, Xu L, et al.Rolling guidance filter[C]//Proceedings of 13th European Conference on Computer Vision.Zurich, Switzerland:Springer, 2014:815-830.[DOI:10.1007/978-3-319-10578-9_53]
-
[9] Yu L H, Feng Y Q, Chen W F. Adaptive regularization method based total variational de-noising algorithm[J]. Journal of Image and Graphics, 2009, 14(10): 1950–1954. [余丽红, 冯衍秋, 陈武凡. 基于自适应正则化的全变分去噪算法[J]. 中国图象图形学报, 2009, 14(10): 1950–1954. ] [DOI:10.11834/jig.20091004]
-
[10] Rudin L I, Osher S, Fatemi E. Nonlinear total variation based noise removal algorithms[J]. Physica D:Nonlinear Phenomena, 1992, 60(1-4): 259–268. [DOI:10.1016/0167-2789(92)90242-F]
-
[11] Farbman Z, Fattal R, Lischinski D, et al. Edge-preserving decompositions for multi-scale tone and detail manipulation[J]. ACM Transactions on Graphics, 2008, 27(3): #67. [DOI:10.1145/1360612.1360666]
-
[12] Xu L, Lu C W, Xu Y, et al. Image smoothing via L0 gradient minimization[J]. ACM Transactions on Graphics, 2011, 30(5): #174. [DOI:10.1145/2024156.2024208]
-
[13] Xu L, Yan Q, Xia Y, et al. Structure extraction from texture via relative total variation[J]. ACM Transactions on Graphics, 2012, 31(6): #139. [DOI:10.1145/2366145.2366158]
-
[14] Chang C C, Lin C J. LIBSVM:A library for support vector machines[J]. ACM Transactions on Intelligent Systems and Technology, 2011, 2(3): #27. [DOI:10.1145/1961189.1961199]
-
[15] Boser B E, Guyon I M, Vapnik V N.A training algorithm for optimal margin classifiers[C]//Proceedings of the 5th Annual Workshop on Computational Learning Theory.Pittsburgh, Pennsylvania:ACM, 1992:144-152.[DOI:10.1145/130385.130401]
-
[16] Cortes C, Vapnik V. Support-vector networks[J]. Machine Learning, 1995, 20(3): 273–297. [DOI:10.1007/BF00994018]
-
[17] Gonzalez R C, Woods R E.Digital Image Processing[M].3rd ed.Beijing:Publishing House of Electronics Industry, 2010:402-431.Beijing:Electronics Industry, 2012:402-431.