发布时间: 2018-12-16 摘要点击次数: 全文下载次数: DOI: 10.11834/jig.180323 2018 | Volume 23 | Number 12 遥感图像处理

 收稿日期: 2018-05-18; 修回日期: 2018-07-16 基金项目: 国家自然科学基金项目（41601507） 第一作者简介: 余东行, 1993年生, 男, 硕士研究生, 主要研究方向为深度学习、遥感影像目标检测与识别。E-mail:dong_hang@aliyun.com;张保明, 男, 教授, 主要研究方向为数字摄影测量和遥感影像处理。E-mail:zbm1961@163.com;郭海涛, 男, 副教授, 主要研究方向为数字摄影测量和变化检测。E-mail:ghtgip2002@163.com;赵传, 男, 博士研究生, 主要研究方向为深度学习与点云数据处理。E-mail:zc_mail163@163.com;徐俊峰, 男, 博士研究生, 主要研究方向为遥感影像变化检测。E-mail:xjf4606@foxmail.com. 中图法分类号: TP391 文献标识码: A 文章编号: 1006-8961(2018)12-1947-12

# 关键词

Joint salient feature and convolutional neural network for ship detection in remote sensing images
Yu Donghang, Zhang Baoming, Guo Haitao, Zhao Chuan, Xu Junfeng
Information Engineering University, Zhengzhou 450001, China
Supported by: National Natural Science Foundation of China (417601507)

# Abstract

Objective Ships and warships are important sea-based transportation carriers and military targets. Thus, detecting and recognizing these targets in high resolution remote sensing images are of substantial practical significance to. However, satellite imaging can be affected by weather, illumination, cloud, and atmosphere scattering. In addition, the targets in the image can be disturbed by sea clutters and other objects, which render the ship detection and recognition increasingly difficult. The majority of ship recognition algorithms typically adopt low-level features, such as shape, invariant moment, and histogram of gradient (HOG), which are simple but not robust to disturbances, such as waves, clouds, and islands. In general, handcraft features can only be used to distinguish ships from other interferences on the sea surface and have weak ability to differentiate various types of ship. In view of the above mentioned problems, this study proposes a new method that combines salient features and a convolutional neural network to recognize ships in remote sensing images. Method The proposed method consists of three parts, namely, image pre-processing, ship pre-detection, and ship recognition. First, the image can be enhanced by a homomorphic filter to improve the texture clarity and contrast in the pre-processing phase, which is helpful for the detection and recognition of the subsequent phase. In the ship detection stage, the saliency map of images can be calculated by phase spectrum of Fourier transform (PFT), which is a technique based on the analysis of the frequency domain. To take account of different resolutions, the multi-scale saliency maps are fused. The PFT method can effectively suppress the interference of cloud and sea wave, but the distinction between background and ship is barely notable. To solve this problem, logarithmic transformation is utilized to enhance the saliency map. Then, the gray morphological operation of close is adapted to eliminate the noise areas and fill holes, and the image segmentation algorithm of Otsu is used to extract all salient areas as areas of interest. In the stage of recognition, a deep convolutional neural network (CNN) can be well trained with a small number of ship samples based on the concept of transfer learning. All areas of interest can be finally classified and recognized by the CNN. Result To verify the effectiveness of the proposed algorithm, experiments were conducted on remote sensing images with varying backgrounds. The experiments were conducted in three aspects, namely, visual saliency detection, ship detection, and ship recognition. Three kinds of indicators, namely, detection, false alarm, and recognition rates were used to quantify experimental results. The qualitative results indicate that saliency detection based on PFT can effectively restrain the disturbance of sea surface and clutter, in which logarithmic transformation substantially improves the integrity of the ships' contour. Quantitative analysis shows that the three indexes of the proposed method are 93.63%, 3.01%, and 90.09%, respectively, which are extensively better than the compared algorithms. Conclusion Visual saliency detection is one of the commonly used and effective methods for ship detection. This paper combined the advantages of visual salient features and convolutional neural network for ship recognition in remote sensing images. The method can realize the rapid detection of ship targets with high accuracy of classification in complex backgrounds.

# Key words

ship detection; remote sensing image; visual saliency detection on frequency domain; convolutional neural network; transfer learning

# 1.2.1 显著性检测

 $\mathit{\boldsymbol{I}} = {\mathit{\boldsymbol{I}}_1} + {\mathit{\boldsymbol{I}}_2}$ (1)

 $E\left\{ {\mathit{\boldsymbol{A}}\left( f \right)} \right\} \propto 1/f$ (2)

 ${\mathit{\boldsymbol{h}}_n}\left( f \right) = \frac{1}{{n \times n}}{\left[ {\begin{array}{*{20}{c}} 1& \cdots &1\\ \vdots &{}& \vdots \\ 1& \cdots &1 \end{array}} \right]_{n \times n}}$ (4)

 $\mathit{\boldsymbol{P}}\left( f \right) = {\mathop{\rm Im}\nolimits} \left( {\mathit{\Gamma }\left( {\mathit{\boldsymbol{I}}\left( {x,y} \right)} \right)} \right)$ (5)

 $\mathit{\boldsymbol{A}}\left( f \right) = {\mathop{\rm Re}\nolimits} \left( {\mathit{\Gamma }\left( {\mathit{\boldsymbol{I}}\left( {x,y} \right)} \right)} \right)$ (6)

 $\mathit{\boldsymbol{L}}\left( f \right) = \log \left( {\mathit{\boldsymbol{A}}\left( f \right)} \right)$ (7)

 $\mathit{\boldsymbol{R}}\left( f \right) = \mathit{\boldsymbol{L}}\left( f \right) - {\mathit{\boldsymbol{h}}_n}\left( f \right) * \mathit{\boldsymbol{L}}\left( f \right)$ (8)

 $\mathit{\boldsymbol{S}}\left( x \right) = \mathit{\boldsymbol{G}}\left( x \right) * {\mathit{\Gamma }^{ - 1}}{\left[ {\exp \left( {\mathit{\boldsymbol{R}}\left( f \right) + \mathit{\boldsymbol{P}}\left( f \right)} \right)} \right]^2}$ (9)

 ${S_{{\rm{PFT}}}}\left( x \right) = \mathit{\boldsymbol{G}}\left( x \right) * {\mathit{\Gamma }^{ - 1}}{\left[ {\exp \left( {\mathit{\boldsymbol{P}}\left( f \right)} \right)} \right]^2}$ (10)

 $g\left( {x,y} \right) = \frac{{{{\log }_2}\left[ {\sigma \mathit{\boldsymbol{I}}\left( {x,y} \right) + 1} \right]}}{{{{\log }_2}\left[ {\sigma + 1} \right]}}$ (13)

# 1.2.3 兴趣区域提取

1) 灰度形态学闭运算。对显著图进行闭运算处理能够有效填充舰船目标的细小孔洞，平滑噪声及轮廓边界。显著性检测极大地抑制了云雾和海面杂波的干扰，为了尽可能获取船体完整轮廓，在形态学闭运算中采取较大的结构元素$\mathit{\boldsymbol{S}}$(本文结构元素为大小10×10像素)对经过对数变换后的显著图$\mathit{\boldsymbol{I}}$进行闭运算处理

 $\mathit{\boldsymbol{I}} \odot \mathit{\boldsymbol{S = }}\left( {\mathit{\boldsymbol{I}} \oplus \mathit{\boldsymbol{S}}} \right) \odot \mathit{\boldsymbol{S}}$ (14)

2) 显著图分割。经过以上步骤，显著图中背景与目标区分度较为明显，采用大津法进行分割，即可取得较好的分割效果

 $\mathit{\boldsymbol{I}}\left( {x,y} \right) = \left\{ {\begin{array}{*{20}{c}} \begin{array}{l} 0\\ 1 \end{array}&\begin{array}{l} \mathit{\boldsymbol{I}}\left( {x,y} \right) < T\\ \mathit{\boldsymbol{I}}\left( {x,y} \right) > T \end{array} \end{array}} \right.$ (15)

3) 兴趣区域选取。对阈值化的显著图进行八邻域连通区标记，分离所有独立目标区域，计算这些区域的外接矩形，获取其对应在原始影像上的位置。为了保证船体的完整性，分别计算每个兴趣区域的最大边长和质心，以其质心为中心，构建长宽均为最大边长的外接矩形，并在此区域基础上再扩大$m$个像素作为最终用于目标识别的兴趣区域(本文$m$取10)。

# 1.3.1 卷积神经网络

 $J\left( \theta \right) = - \frac{1}{N}\sum\limits_{i = 1}^N {\sum\limits_{c = 1}^C {\left[ \begin{array}{l} \delta \left( {{y_i} = c} \right) \cdot \\ \log P\left( {{y_i} = c\left| {{x_i}} \right.,\theta } \right) \end{array} \right]} }$ (16)

# 1.3.2 迁移训练

Table 1 Parameters for training

 数据集 迭代次数 批尺寸 学习率 权重衰减 动量 CIFAR-10 200 128 0.01 0.005 0.9 Ships 50 128 0.001 0.005 0.9

# 2.2 舰船粗检测

 ${F_{{\rm{DR}}}} = \frac{{TP}}{{PN}}$ (17)

 ${F_{{\rm{FAR}}}} = \frac{{FP}}{{DN}}$ (18)

Table 2 Results of ship detection with different methods

 方法 $PN$ $DN$ $FP$ $TP$ DR/% FAR/% SR 503 627 319 308 61.23 50.88 PQFT 503 634 304 330 65.61 47.95 文献[8] 503 582 141 441 87.67 24.23 文献[10] 503 611 158 453 90.06 25.86 本文(无监督) 503 605 122 483 96.24 20.16 本文(CNN) 503 498 15 471 93.63 3.01 注：加粗数字表示本列最优结果，加横线数字表示本列次优值。

# 2.3 舰船类型识别

 $RR = \frac{{NT}}{{NP}}$ (19)

Table 3 Results of ship classification

 方法 小船 货船 运砂船 海面 陆地 平均识别率/% HOG+SVM 11 274 14 35 43 59.86 LeNet-5 17 382 22 39 58 78.14 本文方法 21 415 27 45 66 90.90 注：加粗数字表示最优识别率。

Table 4 Running times of each phase

 /s 方法 显著性检测 兴趣区域提取 目标识别 本文 0.184 0.685 ＜0.01 文献[10] 0.104 1.727 0.674

# 参考文献

• [1] Wang Y Q, Ma L, Tian Y. State-of-the-art of ship detection and recognition in optical remotely sensed imagery[J]. Acta Automatica Sinica, 2011, 37(9): 1029–1039. [王彦情, 马雷, 田原. 光学遥感图像舰船目标检测与识别综述[J]. 自动化学报, 2011, 37(9): 1029–1039. ] [DOI:10.3724/SP.J.1004.2011.01029]
• [2] Tang M E, Lin T Q, Wen G J. Overview of ship detection methods in remote sensing image[J]. Application Research of Computers, 2011, 28(1): 29–36. [唐沐恩, 林挺强, 文贡坚. 遥感图像中舰船检测方法综述[J]. 计算机应用研究, 2011, 28(1): 29–36. ] [DOI:10.3969/j.issn.1001-3695.2011.01.007]
• [3] Zhou J, Tian J W. Method of detecting small target in port-sea background[J]. Infrared and Laser Engineering, 2005, 34(4): 486–489. [周静, 田金文. 基于海岸线背景的海面小目标检测方法[J]. 红外与激光工程, 2005, 34(4): 486–489. ] [DOI:10.3969/j.issn.1007-2276.2005.04.026]
• [4] Sun Y J, Lei W H, Hu Y H, et al. Ship target detection method based on multi-source remote sensing satellite[J]. Laser & Infrared, 2018, 48(2): 267–272. [孙越娇, 雷武虎, 胡以华, 等. 基于多源遥感卫星的海面舰船目标检测方法[J]. 激光与红外, 2018, 48(2): 267–272. ] [DOI:10.3969/j.issn.1001-5078.2018.02.023]
• [5] Wang B Y, Zhang R, Yuan Y, et al. A new multi-level threshold segmentation method for ship targets detection in optical remote sensing images[J]. Journal of University of Science and Technology of China, 2011, 41(4): 293–298. [王保云, 张荣, 袁圆, 等. 可见光遥感图像中舰船目标检测的多阶阈值分割方法[J]. 中国科学技术大学学报, 2011, 41(4): 293–298. ] [DOI:10.3969/j.issn.0253-2778.2011.04.002]
• [6] Wang F C, Zhang M, Gong L M, et al. Fast detection algorithm for ships under the background of ocean[J]. Laser & Infrared, 2016, 46(5): 602–606. [王方超, 张旻, 宫丽美, 等. 海洋背景下的海面舰船快速检测算法[J]. 激光与红外, 2016, 46(5): 602–606. ] [DOI:10.3969/j.issn.1001-5078.2016.05.018]
• [7] Zhang X D, He S H, Yang S Q. Ship targets detection method based on multi-scale fractal feature[J]. Laser & Infrared, 2009, 39(3): 315–318. [张东晓, 何四华, 杨绍清. 一种多尺度分形的舰船目标检测方法[J]. 激光与红外, 2009, 39(3): 315–318. ] [DOI:10.3969/j.issn.1001-5078.2009.03.023]
• [8] Ding P, Zhang Y, Jia P, et al. Ship detection on sea surface based on multi-feature and multi-scale visual attention[J]. Optics and Precision Engineering, 2017, 25(9): 2461–2468. [丁鹏, 张叶, 贾平, 等. 基于多尺度多特征视觉显著性的海面舰船检测[J]. 光学精密工程, 2017, 25(9): 2461–2468. ] [DOI:10.3788/ope.20172509.2461]
• [9] Cheng H, Liu S T, Sun W B, et al. Fine rapid detection of ship targets in remote sensing images[J]. Opto-Electronic Engineering, 2016, 43(4): 25–32. [程红, 刘思彤, 孙文邦, 等. 遥感图像中舰船目标的快速精细检测[J]. 光电工程, 2016, 43(4): 25–32. ] [DOI:10.3969/j.issn.1003-501X.2016.04.005]
• [10] Xu F, Liu J H, Zeng D D, et al. Detection and identification of unsupervised ships and warships on sea surface based on visual saliency[J]. Optics and Precision Engineering, 2017, 25(5): 1300–1311. [徐芳, 刘晶红, 曾冬冬, 等. 基于视觉显著性的无监督海面舰船检测与识别[J]. 光学精密工程, 2017, 25(5): 1300–1311. ] [DOI:10.3788/OPE.20172505.1300]
• [11] Wang H L. Research on detection of typical man-made objects in high resolution optical remote sensing images[D]. Changchun: Changchun Institute of Optics, Fine Mehcanics and Physics, Chinese Academy of Sciences, 2017. [王慧利.高分辨率光学遥感图像中典型人造目标检测技术研究[D].长春: 中国科学院长春光学精密机械与物理研究所, 2017.]
• [12] Hou X D, Zhang L Q. Saliency detection: a spectral residual approach[C]//Proceedings of 2007 IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, MN, USA: IEEE, 2007: 1-8.[DOI: 10.1109/CVPR.2007.383267]
• [13] Chen H Y, Xu S, Liu K, et al. Surface defect detection of steel strip based on spectral residual visual saliency[J]. Optics and Precision Engineering, 2016, 24(10): 2572–2580. [陈海永, 徐森, 刘坤, 等. 基于谱残差视觉显著性的带钢表面缺陷检测[J]. 光学精密工程, 2016, 24(10): 2572–2580. ] [DOI:10.3788/OPE.20162410.2572]
• [14] Guo C L, Ma Q, Zhang L M. Spatio-temporal Saliency detection using phase spectrum of quaternion Fourier transform[C]//Proceedings of 2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, AK, USA: IEEE, 2008: 1-8.[DOI: 10.1109/CVPR.2008.4587715]
• [15] Deng J, Dong W, Socher R, et al. ImageNet: a large-scale hierarchical image database[C]//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, FL, USA: IEEE, 2009: 248-255.[DOI: 10.1109/CVPR.2009.5206848]
• [16] Krizhevsky A. Learning multiple layers of features from tiny images[R]. Toronto: Science Department, University of Toronto, 2009.
• [17] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE, 2016: 770-778.[DOI: 10.1109/CVPR.2016.90]
• [18] Achanta R, Hemami S, Estrada F, et al. Frequency-tuned salient region detection[C]//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, FL, USA: IEEE, 2009: 1597-1604.[DOI: 10.1109/CVPR.2009.5206596]