Island shoreline segmentation in remote sensing image based on improved Deeplab network

Zhenhua Wang; Yuanfu Zhong; Wanwen He; Nianyi Qu; Lizhi Xu; Wenping Zhang; Zhixiang Liu

doi:10.11834/jig.190330

Remote Sensing Image Processing | Views : 0 下载量: 5 CSCD: 4

PDF
Export
Share
Collection
Album

Island shoreline segmentation in remote sensing image based on improved Deeplab network
Vol. 25, Issue 4, Pages: 768-778(2020)
Received：11 July 2019，

Revised：2019-9-29，

Accepted：06 October 2019，

Published：16 April 2020
DOI： 10.11834/jig.190330
稿件说明：

移动端阅览

DOI：

Zhenhua Wang, Yuanfu Zhong, Wanwen He, Nianyi Qu, Lizhi Xu, Wenping Zhang, Zhixiang Liu. Island shoreline segmentation in remote sensing image based on improved Deeplab network[J]. Journal of Image and Graphics, 2020, 25(4): 768-778. DOI： 10.11834/jig.190330.

摘要

目的

海岛作为一项特殊资源，在海洋开发和利用方面发挥着重要的作用；遥感作为一种非接触式远距离探测手段，为海岛研究提供了重要的数据来源；而深度学习因其对图像特征的提取能力和对复杂问题的拟合能力广泛应用于各个领域。本文结合深度学习的计算优势，兼顾遥感影像的波段数量多和覆盖范围大的特征，以海岛岸线的快速分割为目的，提出了一种改进的海岛岸线遥感影像分割模型。

方法

该分割模型包括3方面：1）针对遥感影像的多波段特征，提出基于最佳指数的遥感影像波段组合选择，将选择后的波段组合作为海岛岸线分割模型的输入数据；2）针对遥感影像大范围覆盖的特征，提出基于Deeplab神经网络结构的海岛岸线粗分割，将粗分割结果作为海岛岸线优化的初始边界；3）利用全连接条件随机场优化海岛岸线，实现海岛岸线的细分割提取。

结果

以大小不等的4个海岛的岸线提取为例，分别采用改进的海岛岸线分割模型、全卷积神经网络模型（fully convolutional networks，FCN）、Deeplab模型和目视解译法从遥感影像数据中分割海岛岸线。同时，引入平均交并比（mean intersection over union，MIoU）和相对误差，对各模型分割的海岛岸线结果进行精度比较。结果表明，本文改进的海岛岸线分割模型克服了FCN模型海岛岸线的不连续性问题，降低了海岛岸线的误分割现象；从MIoU值的比较可以看出，本文改进模型与目视解译的海岛岸线结果具有更高的吻合度，较FCN模型提高了17.7%，较Deeplab模型提高了5.2%；从海岸岸线的周长和面积的相对误差可以看出，本文改进模型的相对误差均低于FCN模型和Deeplab模型。

结论

本文改进模型包含了面向遥感影像的波段选择、利用神经网络训练的海岛岸线粗分割和基于全连接条件随机场的海岛岸线优化，在保证岸线连续性的前提下，提高了海岛岸线的分割精度。

Abstract

Objective

Island is naturally formed around the ocean and exposed the sea level at high tide. As a special resource

islands play an important role in ocean development and utilization. Most of the island shorelines were extracted based on the compared and analyzed historical charts

topographic maps

and field survey results

which had several shortcomings

such as difficult observation

high cost

and long period. As a noncontact detection method

remote sensing has become the most important data source for island research. In recent years

deep learning has been widely used in various fields because of its ability to extract image features and to fit complex problems. In this study

a novel segmentation method for island shoreline was proposed

which was designed by the optimum index factor (OIF)

convolutional neural network model (CNN)

and a fully connected conditional random field (CRF).

Method

The proposed method included three aspects:1) According to the multiband characteristics of the remote sensing

the optimal band combination was selected based on the OIF. OIF is defined as the ratio of standard deviation of the combined bands to the correlation coefficients of each band. In this study

taking Landsat-8 remote sensing image as data source

the spectral characteristic information of each band of the image was analyzed based on OIF

and then 1

and 6 bands were selected as the optimal band combination

which is considered as the input data for the island shoreline segmentation.2) According to the large coverage characteristics of the remote sensing

a deep convolution neural network model

Deeplab

was selected to obtain essential features of the input image. In general

segmentation methods extracted the characteristics of training samples relying on the artificial experiences

and a single-layer feature

without a hierarchical structure

was obtained by learning. These methods could supply shallow structures and simple features. This condition is the inadequacy of island shoreline segmentation. Thus

given that Deep CNN architecture has the ability to effectively encode spectral and spatial information

it had quickly become the prominent tool in remote sensing applications. Here

the representative deep CNN architecture and Deeplab neural network were selected. Deeplab neural network consists of three parts:(1) ResNet-101 residual network

which is used as the backbone network to the high-dimensional characteristics of remote sensing images. Residual learning can reduce the burden of deep network training

reduce the phenomenon of gradient disappearance caused by several layers

and improve the training accuracy. (2) Dilated convolution in Deeplab neural network

which can aggregate multiscale contextual information without losing resolution or analyzing rescaled images

increase the performance of dense prediction architectures by aggregating multiscale contextual information. (3) Deeplab neural network

which includes aurous spatial pyramid pooling (ASPP) module. It uses dilated convolution with different sampling rates and can effectively capture multiscale information of island images. 3) The island shoreline was optimized by the fully connected conditional random field. As an excellent representative of a probabilistic graphical model

the fully connected conditional random field can incorporate the spatial contextual information in the aspects of labels and observed data. The uniqueness of this model is that it can be flexible to modeling posterior distribution directly. We pre-classify the entire remote sending image into certain island shoreline types via the Deeplab model by using the results of class membership probabilities as the unary potential in the CRF model. The pairwise potential of CRF is defined by a linear combination of Gaussian kernels

which form a fully connected neighbor structure instead of the common four-neighbor or eight-neighbor structure. We use the mean field approximation method to obtain super pixels and correct the classification results by calculating their average posterior probabilities.

Result

Taking four different islands as example

the island shorelines were segmented by the proposed method

full convolutional neural network (FCN) model

Deeplab model

and the visual interpretation method. Taking the relative error of area and perimeter of the island and the mean intersection over union (MIoU) as the accuracy parameters

the shoreline segmentation results of the four islands were compared. In addition

the relative error of area and perimeter of the proposed method (perimeter) are reduced by 4.6% and 17.7%

respectively. MIoU of the proposed method is increased by 5.2%.

Conclusion

The comparison results showed that the proposed method overcomes the shortcomings of the traditional segmentation methods for island shoreline segmentation

and it takes full use of the band information of the remote sensing images.

关键词

Keywords

references

Cao M, Shi Z L and Shen Q F. 2008. On selection of ALOS image optimum band combination in land cover classification. Bulletin of Surveying and Mapping, (9):16-18, 27

曹敏, 史照良, 沈泉飞. 2008. ALOS影像在土地覆被分类中最佳波段选取的研究.测绘通报, (9):16-18, 27

Chavez P S, Berlin G L and Sowers L B. 1982. Statistical method for selecting landsat MSS. Journal of Applied Photographic Engineering, 8(1):23-30

Chavez P S.1984. Image Processing techniques for Thermatic Mapper data//Proceedings of ASPRS-ACSM Technical Papers: 50th Annual Meeting of ASPRS. Washington DC, USA, [s.n.], 2: 728-743

Chen L C, Papandreou G, Kokkinos I, Murphy K and Yuille A L. 2014. Semantic image segmentation with deep convolutional nets and fully connected CRFs[EB/OL].[2019-07-01] . https://arxiv.org/pdf/1412.7062.pdf https://arxiv.org/pdf/1412.7062.pdf

Chen L C, Papandreou G, Schroff F and Adam H. 2017. Rethinking atrous convolution for semantic image segmentation[EB/OL].[2019-07-01] . https://arxiv.org/pdf/1706.05587.pdf https://arxiv.org/pdf/1706.05587.pdf

Cheng D C, Meng G F, Xiang S M and Pan C H. 2016. Efficient sea-land segmentation using seeds learning and edge directed graph cut. Neurocomputing, 207:36-47[DOI:10.1016/j.neucom.2016.04.020]

Fang L, Fu C Q, Fang T and Hong J G. 2016. Rapid abstraction of island or reef shoreline based on mean shift method. Journal of PLA University of Science and Technology (Natural Science Edition), 17(5):488-492

方亮, 付成群, 方涛, 洪建国. 2016.基于均值漂移的海岛(礁)岸线快速提取.解放军理工大学学报(自然科学版), 17(5):488-492[DOI:10.12018/j.issn.1009-3443.20151219001]

Feng Y J and Han Z. 2012. Cellular automata approach to extract shoreline from remote sensing imageries and its application. Journal of Image and Graphics, 17(3):441-446

冯永玖, 韩震. 2012.海岸线遥感信息提取的元胞自动机方法及其应用.中国图象图形学报, 17(3):441-446[DOI:10.11834/jig.20120320]

Grangier D, Bottou L and Collobert R. 2009. Deep convolutional networks for scene parsing//Proceedings of ICML 2009 Workshop on Learning Feature Hierarchies. Montreal: [s.n.]: 109

Guo H T, Sun L, Shen J S, Chen X W and Zhang H W. 2016. An island and coastal image segmentation method based on quadtree and GAC model. Acta Geodaetica et Cartographica Sinica, 45(1):65-72

郭海涛, 孙磊, 申家双, 陈小卫, 张宏伟. 2016.一种四叉树和测地线活动轮廓模型相结合的海陆影像分割方法.测绘学报, 45(1):65-72[DOI:10.11947/j.AGCS.2016.20150240]

He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE: 770-778[ DOI: 10.1109/CVPR.2016.90 http://dx.doi.org/10.1109/CVPR.2016.90 ]

Huang J F, Zhang X C, Xin Q C, Sun Y and Zhang P C. 2019. Automatic building extraction from high-resolution aerial images and LiDAR data using gated residual refinement network. ISPRS Journal of Photogrammetry and Remote Sensing, 151:91-105[DOI:10.1016/j.isprsjprs.2019.02.019]

Krähenbühl P and Koltun V. 2011. Efficient inference in fully connected CRFs with Gaussian edge potentials//Advances in Neural Information Processing Systems. Granada: [s.n.]: 109-117

Li J, Chen Y H, Liu Z G, Zhou G H, Meng J M, MaY and Deng L. 2010. Remote Sensing of Island and Coast Environment. Beijing: Science Press

李京, 陈云浩, 刘志刚, 周冠华, 孟俊敏, 马毅, 邓磊. 2010.海岛与海岸带环境遥感.北京: 科学出版社

Li Z W, Shen H F, Cheng Q, Liu Y H, You S C and He Z Y. 2019. Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors. ISPRS Journal of Photogrammetry and Remote Sensing, 150:197-212[DOI:10.1016/j.isprsjprs.2019.02.017]

Liao X, Miao J, Chu J and Zhang G M. 2019. Multi-image object semantic segmentation by fusing segmentation priors. Journal of Image and Graphics, 24(6):890-901

廖旋, 缪君, 储珺, 张桂梅. 2019.融合分割先验的多图像目标语义分割.中国图象图形学报, 24(6):890-901[DOI:10.11834/jig.180568]

Liu D E, Yu H X, Lan X J and Chen Y Z. 2013. Land cover information classification based on the optimal band combination of TM image. Metal Mine, 42(10):80-83

刘德儿, 于海霞, 兰小机, 陈元增. 2013.基于最优波段组合的TM影像土地覆盖信息分类.金属矿山, 42(10):80-83

Long J, Shelhamer E and Darrell T. 2017. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4):640-651[DOI:10.1109/TPAMI.2016.2572683]

Ma L, SoomroN Q, Shen J J, Chen L, Mai Z H and Wang G Q. 2017. Hierarchical sea-land segmentation for panchromatic remote sensing imagery. Mathematical Problems in Engineering, 2017:1-8[DOI:10.1155/2017/4728425]

Nandi S, Ghosh M, Kundu A, Dutta D and Baksi M. 2016. Shoreline shifting and its prediction using remote sensing and GIS techniques:a case study of Sagar Island, West Bengal (India). Journal of Coastal Conservation, 20(1):61-80[DOI:10.1007/s11852-015-0418-4]

Tong Q X, Zhang B and Zhang L F. 2016. Current progress of hyperspectral remote sensing in China. Journal of Remote Sensing, 20(5):689-707

童庆禧, 张兵, 张立福. 2016.中国高光谱遥感的前沿进展.遥感学报, 20(5):689-707[DOI:10.11834/jrs.20166264]

Wang Z H, He W W, Sun J Q, Qu N Y and Huang D M. 2019. Research on improved level set model for island boundary rapid segmentation. Journal of Frontiers of Computer Science and Technology, 13(7):1206-1216

王振华, 何婉雯, 孙婧琦, 曲念毅, 黄冬梅. 2019.改进水平集模型的海岛边界快速分割方法研究.计算机科学与探索, 13(7):1206-1216[DOI:10.3778/j.issn.1673-9418.1807051.]

Wu Z H, Gao Y M, Li L, Xue J and Li Y T. 2019. Semantic segmentation of high-resolution remote sensing images using fully convolutional network with adaptive threshold. Connection Science, 31(2):169-184[DOI:10.1080/09540091.2018.1510902]

Zeng R and Chen D S. 2018. Concatenating two deep learned spectral-spatial features for hyperspectral remote sensing imagery classification. Journal of Chinese Computer Systems, 39(2):396-400

曾锐, 陈锻生. 2018.结合双深度学习特征的高光谱遥感图像分类.小型微型计算机系统, 39(2):396-400[DOI:10.3969/j.issn.1000-1220.2018.02.038]

Zhao E W, Wen J M, Yang H M, Wu X J and Zhang Z. 2015. A fast method for automatic extraction of island in remote sensing imagery. Remote Sensing Information, 30(6):63-66, 70

赵恩伟, 温金苗, 杨鹤猛, 伍小洁, 张增. 2015.一种快速遥感影像海岛自动提取方法.遥感信息, 30(6):63-66, 70[DOI:10.3969/j.issn.1000-3177.2015.06.012]