Objective

2

While dealing with high-resolution remote sensing image scene recognition

classical supervised machine learning algorithms are considered effective on two conditions

namely

1) test samples should be in the same feature space with training samples

and 2) adequate labeled samples should be provided to train the model fully. Deep learning algorithms

which achieve remarkable results in image classification and object detection for the past few years

generally require a large number of labeled samples to learn the accurate parameters.The main image classification methods select raining and test samples randomly from the same dataset

and adopt cross validation to testify the effectiveness of the model. However

obtaining scene labels is time consuming and expensive for remote sensing images. To deal with the insufficiency of labeled samples in remote sensing image scene recognition and the problem that labeled samples cannot be shared between different datasets due to different sensors and complex light conditions

deep learning architecture and adversarial learning are investigated. A feature transfer method based on adversarial variational autoencoder is proposed.

Method

2

Feature transfer architecture can be divided into three parts. The first part is the pretrain model.Given the limited samples with scene labels

the unsupervised learning model

variational autoencoder(VAE)

is adopted. The VAE is unsupervised trained on the source dataset

and the encoder part in the VAE is finetuned together with classifier network using labeled samples in the source dataset. The second part is adversarial learning module. In most of the research

adversarial learning is adopted to generate new samples

while the idea is used to transfer the features from source domain to target domain in this paper.Parameters of the finetuned encoder network for the source dataset are then used to initialize the target encoder. Using the idea of adversarial training in generative adversarial networks (GAN)

a discrimination network is introduced into the training of the target encoder. The goal of the target encoder is to extract features in the target domain to have as much affinity to those of the source domain as possible

such that the discrimination network cannot distinguish the features are from either the source domain or target domain. The goal of the discrimination network is to optimize the parameters for better distinction. It is called adversarial learning because of the contradiction between the purpose of encoder and discrimination network. The features extracted by the target encoder increasingly resemble those by the source encoder by training and updating the parameters of the target encoder and the discrimination network alternately. In this manner

by the time the discrimination network can no longer differentiate between source features and target features

we can assume that the target encoder can extract similar features to the source samples

and remote sensing feature transfer between the source domain and target domain is accomplished. The third part is target finetuning and test module. A small number of labeled samples in target domain is employed to finetune the target encoder and source classifier

and the other samples are used for evaluation.

Result

2

Two remote sensing scene recognition datasets

UCMerced-21 and NWPU-RESISC45

are adopted to prove the effectiveness of the proposed feature transfer method. SUN397

a natural scene recognition dataset is employed as an attempt for the cross-view feature transfer. Eight common scene types between the three datasets

namely

baseball field

beach

farmland

forest

harbor

industrial area

overpass

and river/lake

are selected for the feature transfer task.Correlation alignment (CORAL) and balanced distribution adaptation (BDA) are used as comparisons. In the experiments of adversarial learning between two remote sensing scene recognition datasets

the proposed method boosts the recognition accuracy by about 10% compared with the network trained only by the samples in the source domain.

Results

2

improve more substantially when few samples in the target domain are involved. Compared with CORAL and BDA

the proposed method improves scene recognition accuracy by more than 3% when using a few samples in the target domain and between 10%~40% without samples in the target domain. When using the information of a natural scene image

the improvement is not as much as that of a remote sensing image

but the scene recognition accuracy using the proposed feature transfer method is still increased by approximately 6% after unsupervised feature transfer and 36% after a small number of samples in the target domain are involved in finetuning.

Conclusion

2

In this paper

an adversarial transfer learning network is proposed. The experimental results show that the proposed adversarial learning method can make the most of sample information of other dataset when the labeled samples are insufficient in the target domain. The proposed method can achieve the feature transfer between different datasets and scene recognition effectively

and remarkably improve the scene recognition accuracy.