发布时间: 2018-06-16 摘要点击次数: 全文下载次数: DOI: 10.11834/jig.170488 2018 | Volume 23 | Number 6 图像分析和识别

 收稿日期: 2017-09-08; 修回日期: 2018-01-02 基金项目: 国家自然科学基金项目（61172144）；辽宁省教育厅基金项目（L2012113） 第一作者简介: 崔丽群(1969-), 女, 副教授, 硕士生导师, 2004年于辽宁工程技术大学获计算机应用技术专业硕士学位, 主要研究方向为图像与视觉计算。E-mail:373604814@qq.com. 中图法分类号: TP391.9 文献标识码: A 文章编号: 1006-8961(2018)06-0846-11

# 关键词

Saliency object detection method based on complex domains
Cui Liqun, Zhao Yue, Hu Zhiyi, Zhao Yukang
School of Software, Liaoning Technical University, Huludao 125105, China
Supported by: National Natural Science Foundation of China(61172144)

# Abstract

Objective Saliency object detection with development of human visual attention mechanism has been widely studied by computer vision researchers. Visual significance is an important mechanism of human visual system. It simulates the human visual attention mechanism, extracts the most interesting areas of the scene quickly and accurately, and ignores redundant information. Saliency object detection has been widely used in image compression, segmentation, redirection, video coding, target detection, recognition, and many other tasks. Although numerous significant target detection methods are available, problems remain. For example, the detection results look well when the background is simple, but when the background is complex, the results may have some uncertainty as regards the environment, cluttered background in the area around the target, or influence of selection on the significant target detection method. The problem of cluttered background and inaccurate detection area often occurs when the salient object detection method generates significant graphs. To solve these problems, saliency object detection method is proposed based on complex domain. The complex domain combines frequency, spatial, and wavelet domains; takes advantage of the complex domain to combine the advantages on three domains; and suppresses the background to obtain an accurate and clear salient target area. Method Environmental conditions are one of the key factors that influence saliency object detection; for example, weak light or foggy days can cause unclear images and lead to poor results of significant target detection. Multi-scale retinex is an image enhancement algorithm based on color theory. By introducing multi-scale retinex algorithm, the image restoration is realized by linear weighting in the process of dynamically scaling a picture. First, multi-scale retinex enhancement algorithm is used to preliminarily process the original image in spatial domain and exclude environmental impacts. After image processing, the brightness becomes more appropriate to the real scene brightness, and the foreground and background contrast is also significantly improved. In addition to the environmental impact, the background areas of the non-significant target often occupy most of the image space in the saliency object detection images. These background areas increase the error detection problem and reduce the accuracy rate. Experiments found that most background areas are the sky, trees, grasslands, and buildings, which are beyond the scope of this study. The characteristics of the background areas with repeatability can be suppressed by hyper-complex Fourier transform. Then, undirected graph is established and node features on the images are extracted preliminarily. The hyper-complex Fourier transform in the frequency domain is reconstructed to acquire the smoothing amplitude spectrum, phase spectrum, and Euler spectrum. Then, background suppression graphs are obtained through the smoothness of multi-scale Gaussian kernel. At the same time, the multi-level feature of wavelet transform in the wavelet domain is utilized to extract multiple features in terms of images, and the saliency graph of multiple features is calculated. The saliency graph effectively preserves the details of the image because of the unique localization characteristics of the wavelet domain. Finally, the proposed adaptive threshold selection method is used to fuse the background suppression diagram with the saliency graph of multiple features and the final saliency graph is selected and obtained. The final saliency figure suppresses the background while preserving the details of the image. Result To make the experimental effect persuasive, saliency object detection experiments in the standard test dataset images MSRA10K and THUR15k are conducted. MSRA10K datasets consist of 10 000 images of hand-annotated and accurate to pixel-level salient target annotations, including images of natural scenery, biology, architecture, and transportation. THUR15K datasets consist of 15 000 web images with five keywords, namely, butterflies, airplanes, giraffes, cups, and dogs, representing significant targets with pixel precision as the former datasets. The two datasets are public standard image databases and are widely used in salient target detection and image segmentation. A total of 300 background-complex pictures are selected from each dataset, under the same experimental conditions, and compared with six popular significant target detection methods. Results show that the problems presented by our method had a good solution. Even in a complex environment, the accuracy and recall rate of the algorithm are higher than those of state-of-the-art contrast algorithms. In MSRA10k datasets, the mean absolute error (MAE) value is 0.106; in THUR15K datasets, the mean absolute error value was reduced to 0.068, and the average structure (s) measure value was 0.844 9. The result of the MAE evaluation reflects the advantage of a saliency object detection method based on complex domain in terms of overall performance, and the s-measure indicates that the detected target is highly similar to the structure of the target of the ground truth graph. Conclusion Saliency object detection is a promising preprocessing operation in image processing and analysis. In this study, a new saliency object detection method based on complex domain is proposed. Multi-scale retinex algorithm in spatial domain can be used for pretreatment of images; it enhances contrast and prevents images from being affected by environmental factors. Hyper-complex Fourier transform in the frequency domain can suppress complex repetitive background regions, and the significant target detection method in the wavelet domain can completely describe the details of the target. Moreover, the proposed algorithm integrates the advantages of multiple domains and improves the accuracy while suppressing background clutter. Thus, the proposed algorithm is suitable for detecting significant target images, such as natural scenery, biology, architecture, and transportation. To improve the algorithm speed, our next research project aims to reduce the complexity of the algorithm by using the influence of wavelet transform function on time complexity.

# Key words

salient objects detection; multi-scale Retinex enhancement algorithm; hyper complex Fourier transformation; wavelet transform; adaptive threshold selection method

# 1.1 空域MSR增强目标表示

MSR是一种图像增强算法，在Retinex的基础上提出的。Retinex是一种基于色彩理论的图像增强算法，人眼看到的图像是由自然界中的光照射在物体上，再由物体反射到人眼中所形成的景象。数学表达为

 $\mathit{\boldsymbol{I}} = \mathit{\boldsymbol{L}} * \mathit{\boldsymbol{R}}$ (1)

$\mathit{\boldsymbol{I}}$表示人眼观察到的图像，$\mathit{\boldsymbol{L}}$表示环境中光的照射分量，$\mathit{\boldsymbol{R}}$表示场景中物体的反射分量。

MSR算法在Retinex的基础上，引入多尺度概念，在动态缩放图片的过程中进行线性加权实现图像的恢复。由于光照的影响，场景中显著性目标与背景的边界模糊，对比度减弱。为了克服光照的影响，增强暗区域的图像细节，采用MSR算法实现图像色彩的恢复。物体的反射分量是其在不受环境影响的物体自然的状态，其数学表达式为

 $\begin{array}{*{20}{c}} {{{\log }_2}\left[ {R\left( {x,y} \right)} \right] = {{\log }_2}\left[ {R\left( {x,y} \right)} \right] + }\\ {w\left( i \right) \times \left( {{{\log }_2}\left[ {{I_i}\left( {x,y} \right)} \right] - {{\log }_2}\left[ {{L_i}\left( {x,y} \right)} \right]} \right)} \end{array}$ (2)

$w\left( i \right)$代表各个尺度下的权重值，权重值一般取等权重，权重和为1。

MSR算法对图像去雾、老照片、非均匀光照下的图片都有明显的效果。图 2为非均匀光照下MSR算法对图片的处理效果。由图 2可知，经过处理后的图像，亮度会更加贴切真实场景的亮度，前景与背景的对比度也会有明显的提高。

# 1.2 频域HFT抑制背景

 $\begin{array}{*{20}{c}} {\mathit{\boldsymbol{f}}\left( {n,m} \right) = {\varepsilon _1}{\mathit{\boldsymbol{F}}_1} + {\varepsilon _2}{\mathit{\boldsymbol{F}}_2}a + }\\ {{\varepsilon _3}{\mathit{\boldsymbol{F}}_3}b + {\varepsilon _4}{\mathit{\boldsymbol{F}}_4}c} \end{array}$ (3)

 ${\mathit{\boldsymbol{F}}_H}\left[ {u,v} \right] = \frac{1}{{\sqrt {MN} }}\sum\limits_{m = 0}^{M - 1} {\sum\limits_{n = 0}^{N - 1} {{{\rm{e}}^{ - \mu 2{\rm{ \mathsf{ π} }}\left( {\left( {\frac{{mv}}{M}} \right) + \left( {\frac{{mu}}{N}} \right)} \right)}}\mathit{\boldsymbol{f}}\left( {n,m} \right)} }$ (5)

 ${\mathit{\boldsymbol{F}}_H}\left[ {u,v} \right] = \left\| {{\mathit{\boldsymbol{F}}_H}\left[ {u,v} \right]} \right\|{{\rm{e}}^{\mu \varphi \left( {u,v} \right)}}$ (6)

 $\left\{ \begin{array}{l} \mathit{\boldsymbol{A}}\left( {u,v} \right) = \left\| {{\mathit{\boldsymbol{F}}_H}\left( {u,v} \right)} \right\|\\ \mathit{\boldsymbol{P}}\left( {u,v} \right) = \varphi \left( {u,v} \right) = {\tan ^{ - 1}}\frac{{\left\| {V\left( {{\mathit{\boldsymbol{F}}_H}\left( {u,v} \right)} \right)} \right\|}}{{S\left( {\mathit{\boldsymbol{F}}\left( {u,v} \right)} \right)}}\\ \mathit{\boldsymbol{X}}\left( {u,v} \right) = \mu \left( {u,v} \right) = \frac{{V\left( {{\mathit{\boldsymbol{F}}_H}\left( {u,v} \right)} \right)}}{{\left\| {V\left( {{\mathit{\boldsymbol{F}}_H}\left( {u,v} \right)} \right)} \right\|}} \end{array} \right.$ (7)

 $\mathit{\boldsymbol{H}}_i^c\left( x \right) = \mathit{\boldsymbol{W}}\left( {\mathit{\boldsymbol{I}}_i^c\left( x \right)} \right)$ (10)

$i$表示特征数目，$c$表示LAB 3个通道，$W$表示贝多西小波函数，$\mathit{\boldsymbol{H}}\left( x \right)$表示小波分解得到的多个特征。

 ${\mathit{\boldsymbol{f}}_i} = {\mathit{\boldsymbol{g}}_{m \times n}} * {\mathit{\boldsymbol{W}}^{ - 1}}\left[ {\mathit{\boldsymbol{H}}_i^c\left( x \right)} \right]$ (11)

 ${\mathit{\boldsymbol{f}}_{\rm{w}}} = \sum\limits_{i = 1}^n {{\omega _i} \times {\mathit{\boldsymbol{f}}_i} - \tau }$ (12)

${\omega _i}$表示特征对应的权重值，$\tau$表示调节系数，一般取值为0.20.5[19]

# 1.4 自适应阈值修正

 ${\mathit{\boldsymbol{f}}_{{\rm{fin}}}} = {\mathit{\boldsymbol{f}}_{\rm{w}}} * T\left( t \right)$ (13)

# 2.2 评价体系

Table 1 Status values

 检测结果 目标区域 背景区域 显著性区域 TP FP 非显著性区域 FN TN

 $\begin{array}{l} P = \frac{{TP}}{{TP + FP}}\\ R = \frac{{TP}}{{TP + FN}}\\ MAE = \frac{{FP + FN}}{M} \end{array}$ (14)

# 2.4.2 客观比较与分析

 $\begin{array}{l} S = \sigma \times {S_o} + \left( {1 - \sigma } \right) \times {S_r}\\ {S_r} = \sum\limits_{k = 1}^K {{w_k} \times s\left( k \right)} \\ {S_o} = \partial \times {O_{{\rm{FG}}}} + \left( {1 - \partial } \right) \times {O_{{\rm{BG}}}} \end{array}$ (15)

${S_r}$表示物体部分的相似性，${w_k}$表示每个块的权重值，$s\left( k \right)$表示结构相似性；${S_o}$表示面向物体的结构度量，${O_{{\rm{FG}}}}$表示显著图与标注图之间物体级别的相似性，${O_{{\rm{BG}}}}$表示背景的相似性，$\partial$表示标注图中前景区域与图像区域(宽×高)比值；$S$为结构相似性指标，$\sigma$属于0到1，本实验中取0.5。对每个数据集中选择的300幅图片计算所有方法的S-measure平均值，作S-measure的折线图如图 9(a)(b)所示。

# 参考文献

• [1] Du H. Visual saliency detection research based on multiple domains and features[D]. Changchun: Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, 2016. [杜慧. 基于视觉显著性的空频域多特征的目标检测方法研究[D]. 长春: 中国科学院长春光学精密机械与物理研究所, 2016.] http://cdmd.cnki.com.cn/Article/CDMD-80139-1017013963.htm
• [2] Qian K, Li F, Wen L M, et al. Color and space distance based salient region detection using fixed threshold segmentation[J]. Computer Science, 2016, 43(1): 103–106, 144. [钱堃, 李芳, 文益民, 等. 基于颜色和空间距离的显著性区域固定阈值分割算法[J]. 计算机科学, 2016, 43(1): 103–106, 144. ] [DOI:10.11896/j.issn.1002-137x.2016.01.024]
• [3] Xu W, Tang Z M. Exploiting hierarchical prior estimation for salient object detection[J]. Acta Automatica Sinica, 2015, 41(4): 799–812. [徐威, 唐振民. 利用层次先验估计的显著性目标检测[J]. 自动化学报, 2015, 41(4): 799–812. ] [DOI:10.16383/j.aas.2015.c140281]
• [4] Liang Y, Yu J, Lang C Y, et al. Research on salient region extraction technology[J]. Computer Science, 2016, 43(3): 27–32. [梁晔, 于剑, 郎丛妍, 等. 显著区域检测技术研究[J]. 计算机科学, 2016, 43(3): 27–32. ] [DOI:10.11896/j.issn.1002-137X.2016.03.005]
• [5] Lee G, Tai Y W, Kim J. ELD-Net:An efficient deep learning architecture for accurate saliency detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017. [DOI:10.1109/TPAMI.2017.2737631]
• [6] Li Y Y, Xu Y L, Ma S P, et al. Saliency detection based on deep convolutional neural network[J]. Journal of Image and Graphics, 2016, 21(1): 53–59. [李岳云, 许悦雷, 马时平, 等. 深度卷积神经网络的显著性检测[J]. 中国图象图形学报, 2016, 21(1): 53–59. ] [DOI:10.11834/jig.20160107]
• [7] Du Y L, Li J Z, Zhang Y, et al. Saliency detection based on deep cross CNN and non-interaction GrabCut[J]. Computer Engineering and Applications, 2017, 53(3): 32–40. [杜玉龙, 李建增, 张岩, 等. 基于深度交叉CNN和免交互GrabCut的显著性检测[J]. 计算机工程与应用, 2017, 53(3): 32–40. ] [DOI:10.3778/j.issn.1002-8331.1607-0134]
• [8] Yao Z J, Tan T Z. Saliency detection combining background and foreground prior[J]. Journal of Image and Graphics, 2017, 22(10): 1381–1391. [姚钊健, 谭台哲. 结合背景和前景先验的显著性检测[J]. 中国图象图形学报, 2017, 22(10): 1381–1391. ] [DOI:10.11834/jig.170114]
• [9] Itti L, Koch C, Niebur E. A model of saliency-based visual attention for rapid scene analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, 20(11): 1254–1259. [DOI:10.1109/34.730558]
• [10] Harel J, Koch C, Perona P. Graph-based visual saliency[C]//Proceedings of 19th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2006: 545-552.
• [11] Hou X D, Zhang L Q. Saliency detection: a spectral residual approach[C]//Proceedings of 2007 IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, USA: IEEE, 2007: 1-8. [DOI:10.1109/CVPR.2007.383267]
• [12] Achanta R, Hemami S, Estrada F, et al. Frequency-tuned salient region detection[C]//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, USA: IEEE, 2009: 1597-1604. [DOI:10.1109/CVPR.2009.5206596]
• [13] Cheng M M, Zhang G X, Mitra N J, et al. Global contrast based salient region detection[C]//Proceedings of 2011 IEEE Conference on Computer Vision and Pattern Recognition. Colorado Springs, USA: IEEE, 2011: 409-416. [DOI:10.1109/CVPR.2011.5995344]
• [14] LI J, Levine M D, AN X J, et al. Visual saliency based on scale-space analysis in the frequency domain[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(4): 996–1010. [DOI:10.1109/TPAMI.2012.147]
• [15] Imamoglu N, Lin W, Fang Y. A saliency detection model using low-level features based on wavelet transform[J]. IEEE Transactions on Multimedia, 2013, 15(1): 96–105. [DOI:10.1109/TMM.2012.2225034]
• [16] Peng H W, Li B, Ling H B, et al. Salient object detection via structured matrix decomposition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 818–832. [DOI:10.1109/TPAMI.2016.2562626]
• [17] Petro A B, Sbert C, Morel J M. Multiscale retinex[J]. Image Processing on Line, 2014, 4: 71–88. [DOI:10.5201/ipol.2014.107]
• [18] Guo Y C, Feng Y H, Yan G, et al. Image saliency detection in wavelet domain based on the contrast sensitivity function[J]. Journal on Communications, 2015, 36(10): 47–55. [郭迎春, 冯艳红, 阎刚, 等. 基于对比敏感度的小波域图像显著性检测[J]. 通信学报, 2015, 36(10): 47–55. ] [DOI:10.11959/j.issn.1000-436x.2015262]
• [19] Chen Z X, He C, Liu C Y. Image saliency target detection based on global features and local features[J]. Control and Decision, 2016, 31(10): 1899–1902. [陈振学, 贺超, 刘成云. 基于局部特征与全局特征的图像显著性目标检测[J]. 控制与决策, 2016, 31(10): 1899–1902. ] [DOI:10.13195/j.kzyjc.2015.1017]
• [20] Fan D P, Cheng M M, Liu Y, et al. Structure-measure: A new way to evaluate foreground maps[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017: 4558-4567. [DOI:10.1109/ICCV.2017.487]