Objective

2

Remote sensing image semantic segmentation

in which each pixel in an image is classified according to the land cover type

presents an important research direction in the field of remote sensing image processing. However

accurately segmenting and extracting features from remote sensing images is difficult due to the wide coverage of these images and the large-scale difference and complex boundaries among these features. Meanwhile

the traditional remote sensing image processing methods are inefficient

inaccurate

and require much expertise. Convolutional neural networks are deep learning networks that are suitable for processing data with grid structures

such as 1D data with time series features (e.g.

speech) and image data with 2D pixel matrix grids. Given its multi-layer structure

a convolutional neural network can automatically learn features at different levels. This network also has two features that facilitate image processing. First

a convolutional neural network uses the 2D characteristics of an image in feature extraction. Given the high correlation among adjacent pixels in an image

the neuron nodes in the network do not need to connect all pixels; only a local connection is required to extract features. Second

convolution kernel parameters are shared when the convolutional neural network performs convolution operations

and features at different positions of an image use the same convolution kernel to calculate their values

there by greatly reducing the model parameters. In this paper

a full convolutional neural network based on a residual dense spatial pyramid is applied in urban remote sensing image segmentation to achieve an accurate semantic segmentation of high-resolution remote sensing images.

Method

2

To improve the semantic segmentation precision of high-resolution urban remote sensing images

we first take a 101-layer residual convolutional network as our backbone in extracting remote sensing image feature maps. When extracting features by using classic convolutional neural networks

the repeated concatenation of max-pooling and striding at consecutive layers significantly reduces the spatial resolution of the feature maps

typically by a factor of 32 across each direction in general deep convolutional neural networks(DCNNs)

thereby leading to spatial information loss. Semantic segmentation is a pixel-to-pixel mapping task whose class intensity reaches the pixel level. Reducing the spatial resolution of feature maps can lead to spatial information loss

which is not conducive to the semantic segmentation of remote sensing images. To avoid such loss

the proposed model introduces atrous convolution into the residual convolutional neural network. Compared with ordinary convolution

atrous convolution uses the parameter

r

to control the receptive field of the convolution kernel during the calculation. The convolutional neural network with atrous convolution can expand the receptive field of the feature map while keeping the feature map size unchanged

thereby significantly improving the remote sensing image semantic segmentation performance of the proposed model. Objects in remote sensing images often demonstrate large-scale variations and complex texture features

both of which challenge the accurate encoding of multi-scale advanced features. To accurately extract multi-scale features in these images

the proposed model cascades each branch of aspatial pyramid structure based on a dense connection mechanism

which allows each branch to output highly dense receptive field information. In these mantic segmentation of remote sensing images

not only the high-level semantic features extracted by the convolutional neural network are required to correctly determine the category of each pixel; low-level texture features are also required to determine the edges of the target. Low-level texture features can benefit the reconstruction of object edges during semantic segmentation. Our proposed model uses a simple encoder to effectively use high-level semantic features and low-level texture features in a network. A decoder also uses skip connection to fuse cross-layer network information and to combine high-level semantic features with the underlying texture features. After fusing high- and low-level information

we use two 3×3 convolutions to integrate the information among channels and to recover spatial information. We eventually input the extracted feature map to a softmax classifier for pixel-level classification and obtain the remote sensing image semantic segmentation results.

Result

2

Full experiments are performed by using the ISPRS(International Society for Phtogrammetry and Remote Sensing) remote sensing dataset of the Vaihingen area. WE use intersection over union (IoU) and F

1

as our indicators for evaluating the segmentation performance of the proposed model. We also build and train our models based on the NVIDIA Tesla P100 platform and the Tensorflow deep learning framework. The complexity of tasks in the experiment increases at each stage. Experimental results show that the proposed model obtains mean IoU (MIoU) and F

1

values of 69.88% and 81.39% over six types of surface features

respectively

thereby demonstrating vast improvements compared with a residual convolutional network without atrous convolution. Our proposed method also outperforms SegNet

Res-shuffling-Net and SDFCN (symmetrical dense-shortcut fully convolutional network) in terms of mathematics and outperforms pix2pix in terms of visual effects

thereby cementing its validity. We then apply this model on the remote sensing image data of Potsdam area and obtain MIoU and F

1

values of 74.02% and 83.86%

respectively

thereby proving the robustness of our model.

Conclusion

2

We build an end-to-end deep learning model for the semantic segmentation of remote sensing images of high-resolution urban areas. By applying an improved spatial pyramid pooling network based on atrous convolution and dense connections

our proposed model effectively extracts multi-scale features from remote sensing images and fuse high-level semantic information and low-level texture information of the network

which in turn can improve the accuracy of the model in the remote sensing image segmentation of urban areas. Experimental results prove that the proposed model achieves an excellent performance in terms of mathematical and visual effects and has high application value in the semantic segmentation of high-resolution remote sensing images.