谭琨,王雪,杜培军(华东师范大学地理信息科学教育部重点实验室, 上海 200241;中国矿业大学自然资源部国土环境与灾害监测重点实验室, 徐州 221116;南京大学自然资源部卫星测绘技术与应用重点实验室, 南京 210023)
本文以结合深度学习的遥感影像特征提取和不充足样本下地物识别与分类作为出发点，对2017—2019年用于遥感图像处理中小样本训练的深度学习方法进行归类总结，介绍如何结合深度学习技术解决遥感影像在样本不充分情况下的有效训练问题，从深度生成模型、迁移学习以及一些高效特征提取网络3个方面进行全面剖析。首先，探讨了以GAN（generative adversarial network）和VAE（variational autoencoder）及其衍生结构在遥感技术中分类、变化检测上的应用；然后，在基于知识复用的辅助训练策略——迁移学习中主要从基于网络的迁移和基于数据结构的迁移两大类应用展开讨论；最后探讨了结合半监督学习和主动学习等思想的深度学习算法以及一些新颖的网络结构的应用。虽然深度学习在遥感技术领域发挥了极大的优势，性能也普遍超过了浅层的学习器，但结合物理模型的分析和高性能的实用性遥感应用仍需进一步发展与研究。
Research progress of the remote sensing classification combining deep learning and semi-supervised learning
Tan Kun,Wang Xue,Du Peijun(Key Laboratory of Geographic Information Science, Ministry of Education, East China Normal University, Shanghai 200241, China;Key Laboratory for Land Environment and Disaster Monitoring of Ministry of Natural Resources, China University of Mining and Technology, Xuzhou 221116, China;Key Laboratory for Satellite Mapping Technology and Applications of Ministry of Natural Resources, Nanjing University, Nanjing 210023, China)
Remote sensing image, which is an important data source in spatial analysis, records both spectral and spatial information of the scene. Therefore, it is widely utilized in areas such as terrain classification, change detection and object identification etc.. Classification is the most primary problem of remote sensing applications while the issues of large date redundancy and small training set are still the barrier of its widespread application and development. Deep learning is one kind of representational learning, which has led to significant advances in imaging technology. Traditional pattern recognition algorithm always be the thought of a strategy of “divide and conquer”, which tends to be divided into feature extraction and feature selection, classifier design processes. Although this solution idea can decompose the problem into several controllable subproblems, at the same time, the optimum solution of these subproblems cannot converge to the global optimum, and even the best feature extraction methods cannot make sure the classification of boundary is prefect. Comparing with the hand-crafted feature extraction methods, the end-to-end optimization pattern of deep learning has brought a superior performance for the remote sensing classification. Unfortunately, deep learning usually requires big data, with respect to both volume and variety, while most remote sensing applications only have limited training data, of which a small subset is labeled. Herein, we provide the most comprehensive survey of state-of-the-art approaches in deep learning to combat this challenge over recent one or two years, and to enable researchers to explore its theory and development. This paper summarizes three kinds of methods to train the deep model under limited training data. The first topic is deep generative model, in which we explore the applications of the generative adversarial networks (GAN), variation autoencoders (VAE) and their derived structures in remote sensing classification and change detection, and the application fields, applicable data and characteristics of the generated model are summarized. The next is transfer learning, in which we review the approaches that network structures or the data features are transferred from one domain to anther domain. Although transfer learning provides a possible solution to make full use of the existing tag data, the ability of transfer learning is also limited. When the task and data distribution between two domains are very different, negative migration is easy to occur. Therefore, the mobility measurement standards in the field of remote sensing technology should be further studied. The last is the novel deep neural networks which is trained with semi-supervised learning or active learning strategies towards remote sensing classification. At the same time, the author enumerates some attempts made by scholars based on novel network structures in recent years. There are two main solutions to handle the full training of deep learning under the limited training data. One is to enhance the prior knowledge. Before the deep generative model appears, data enhancement often relies on simple image transformation and data augmentation. For multispectral data, simple data transformations, such as translation, rotation, scaling, shearing, or any combination of these, were carried out to expand the training samples. For hyperspectral data, data simulation based on physical models, such as spectrum simulation under different illumination of the same ground scene, label propagation driven by data or additional Gaussian white noise is adopted. These solutions rely heavily on the data itself or the assumptions of the physical environment. Different from traditional methods, deep generative methods and transfer learning have strong capability in the learning of prior knowledges. And in the most case, the combination of these two kinds approaches are used.The other is to extract the more effective time-spatio-spectral features towards classification using novel deep neural networks on the limited labelled data. Such methods build or select advanced network structures, which perform well in computer vision fields, and the training process combine such training strategies as semi-supervised learning, ensemble learning and active learning strategies. In the practical applications, these mentioned solutions are jointed to obtain the optimum performance. Due to the exploration of various models, deep learning has shown its superiority in remote sensing technology and the performance exceeds the shallow model in general. Nonetheless, the physics-based deep learning approach and high-powered practicality are still worth studying.