浏览全部资源
扫码关注微信
Published： 16 August 2021 ，
Accepted： 31 May 2021
移动端阅览
Yanxin Shi, Jinrong He, Zhaokui Li, Zhigao Zeng. Hyperspectral image classification model based on 3D convolutional auto-encoder. [J]. Journal of Image and Graphics 26(8):2021-2036(2021)
目的
2
高光谱图像分类是遥感领域的基础问题，高光谱图像同时包含丰富的光谱信息和空间信息，传统模型难以充分利用两种信息之间的关联性，而以卷积神经网络为主的有监督深度学习模型需要大量标注数据，但标注数据难度大且成本高。针对现有模型的不足，本文提出了一种无监督范式下的高光谱图像空谱融合方法，建立了3D卷积自编码器（3D convolutional auto-encoder，3D-CAE）高光谱图像分类模型。
方法
2
3D卷积自编码器由编码器、解码器和分类器构成。将高光谱数据预处理后，输入到编码器中进行无监督特征提取，得到一组特征图。编码器的网络结构为3个卷积块构成的3D卷积神经网络，卷积块中加入批归一化技术防止过拟合。解码器为逆向的编码器，将提取到的特征图重构为原始数据，用均方误差函数作为损失函数判断重构误差并使用Adam算法进行参数优化。分类器由3层全连接层组成，用于判别编码器提取到的特征。以3D-CNN（three dimensional convolutional neural network）为自编码器的主干网络可以充分利用高光谱图像的空间信息和光谱信息，做到空谱融合。以端到端的方式对模型进行训练可以省去复杂的特征工程和数据预处理，模型的鲁棒性和稳定性更强。
结果
2
在Indian Pines、Salinas、Pavia University和Botswana等4个数据集上与7种传统单特征方法及深度学习方法进行了比较，本文方法均取得最优结果，总体分类精度分别为0.948 7、0.986 6、0.986 2和0.964 9。对比实验结果表明了空谱融合和无监督学习对于高光谱遥感图像分类的有效性。
结论
2
本文模型充分利用了高光谱图像的光谱特征和空间特征，可以做到无监督特征提取，无需大量标注数据的同时分类精度高，是一种有效的高光谱图像分类方法。
Objective
2
Hyperspectral image classification is a basic problem in the field of remote sensing
and it has been one of the research hotspots of numerous scholars. Hyperspectral images contain rich spectral and spatial information
and the classification accuracy of remote sensing images can be improved by using spectral and spatial features. Early traditional models
such as support vector machine and decision trees
could not fully utilize both information. With the development of deep learning technology
an increasing number of scholars use convolutional neural network as a model to extract the features of hyperspectral images. However
two dimensional convolutional neural network(2D-CNN) can only extract the spatial features of hyperspectral images and cannot fully use the band information of remote sensing data. 3D-CNN can efficiently simultaneously extract spectral and spatial features. The recurrent neural network cannot complete the task of hyperspectral image classification because of the difficulty of finding the optimal sequence length and over-fitting. At present
scholars focus on supervised deep learning model
which needs a substantial amount of labeled data to be effectively trained. However
labeled data are difficult and costly in reality. Therefore
the model must have good performance in the unknown world. An unsupervised normal form classification method for spatial-spectral fusion of hyperspectral images is proposed to address the problem that the existing models cannot fully use the spatial and spectral information and require a large amount of data for training. An unsupervised hyperspectral image classification model based on 3D convolution self-encoder is also established.
Method
2
The 3D convolution auto-encoder(3D-CAE) proposed in this work is composed of an encoder
a decoder
and a classifier. The hyperspectral image is inputted into an encoder after data pre-processing for unsupervised feature extraction to produce a set of feature maps. The network structure of the encoder is a 3D convolutional neural network of three convolution blocks
each of which is made up of two convolution layers and two global max-pooling layers. Batch normalization technique is added to the convolution blocks to prevent over-fitting. The decoder is an inverted encoder
which reconstructs the extracted feature graph into original data
and uses the mean square error function as the loss function to judge the reconstruction error and optimizes the parameters with the Adam algorithm. The classifier consists of three fully connected layers and uses ReLU as the activation function of the fully connected layer to classify the features extracted by the encoder. The backbone network with 3D-CNN as auto-encoder can fully use the spatial and spectral information of hyperspectral images to achieve spatial spectral fusion. The model is also trained end to end
eliminating the need for complex feature engineering and data pre-processing
making it more robust and stable.
Result
2
The seven methods on Indian Pines
Salinas
Pavia University
and Botswana datasets achieve the best results compared with the traditional single feature and deep learning methods. The overall classification accuracies are 0.948 7
0.986 6
0.986 2
and 0.964 9
the average classification accuracies are 0.936 0
0.992 4
0.982 9
and 0.965 9
and the Kappa values are 0.941 5
0.985 1
0.981 7
and 0.962 0
respectively. Comparative experimental results show that the spatial-spectral fusion and unsupervised learning are effective for hyperspectral remote sensing image classification. The ablation experiment is added because 3D-CAE is composed of a self-encoder and a classifier. Under the condition of the same self-encoder
four classifiers with different structures are used for classification. The experimental results are stable
and the validity of the self-encoder is proved. Five different proportions of datasets are used to prove the generalization of 3D-CAE. The training set proportions are 5%
8%
10%
15%
and 20%. The loss of the auto-encoder and the classifier on the four datasets remained stable and low
and no oscillation was observed
indicating the better generalization of 3D-CAE. Finally
we analyze and discuss the parameters of each deep learning model. 3D-CAE has less parameters and the best classification performance
which proves its high efficiency.
Conclusion
2
The 3D-CAE model proposed in this work fully uses the spectral and spatial features of hyperspectral images. This model also achieves unsupervised feature extraction without substantial pre-processing and high classification accuracy without a large amount of labeled data. Thus
this model is an effective method for hyperspectral image classification.
遥感图像分类空谱特征融合3D-CNN自编码器卷积神经网络(CNN)深度学习
remote sensing image classificationspatial spectral feature fusion3D-CNNauto-encoderconvolutional neural network(CNN)deep learning
Byeon W, Breuel T M, Raue F and Liwicki M. 2015. Scene labeling with LSTM recurrent neural networks//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE: 3547-3555[DOI: 10.1109/CVPR.2015.7298977http://dx.doi.org/10.1109/CVPR.2015.7298977]
Chang C I. 2003. Hyperspectral Imaging: Techniques for Spectral Detection and Classification. New York: Springer US: 15-34
Chen X, Fang T, Huo H and Li D R. 2011. Graph-based feature selection for object-oriented classification in VHR airborne imagery. IEEE Transactions on Geoscience and Remote Sensing, 49(1): 353-365[DOI:10.1109/TGRS.2010.2054832]
Chen Y S, Jiang H L, Li C Y, Jia X P and Ghamisi P. 2016. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 54(10): 6232-6251[DOI:10.1109/TGRS.2016.2584107]
Chen Y S, Zhao X and Jia X P. 2015. Spectral-spatial classification of hyperspectral data based on deep belief network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 8(6): 2381-2392[DOI:10.1109/JSTARS.2015.2388577]
Du P J, Xia J S, Xue Z H, Tan K, Su H J and Bao R. 2016. Review of hyperspectral remote sensing image classification. Journal of Remote Sensing, 20(2): 236-256
杜培军, 夏俊士, 薛朝辉, 谭琨, 苏红军, 鲍蕊. 2016. 高光谱遥感影像分类研究进展. 遥感学报, 20(2): 236-256[DOI:10.11834/jrs.20165022]
Fauvel M, Tarabalka Y, Benediktsson J A, Chanussot J and Tilton J C. 2013. Advances in spectral-spatial classification of hyperspectral images. Proceedings of the IEEE, 101(3): 652-675[DOI:10.1109/JPROC.2012.2197589]
Hara K, Kataoka H and Satoh Y. 2018. Can spatiotemporal 3D CNNs retrace the history of 2D CNNs and ImageNet?//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE: 6546-6555[DOI: 10.1109/CVPR.2018.00685http://dx.doi.org/10.1109/CVPR.2018.00685]
Hochreiter S and Schmidhuber J. 1997. Long short-term memory. Neural Computation, 9(8): 1735-1780[DOI:10.1162/neco.1997.9.8.1735]
Hu W, Huang Y Y, Wei L, Zhang F and Li H C. 2015. Deep convolutional neural networks for hyperspectral image classification. Journal of Sensors, 2015: #258619[DOI:10.1155/2015/258619]
Kang X D, Li S T and Benediktsson J A. 2014. Feature extraction of hyperspectral images with image fusion and recursive filtering. IEEE Transactions on Geoscience and Remote Sensing, 52(6): 3742-3752[DOI:10.1109/TGRS.2013.2275613]
Krizhevsky A, Sutskever I and Hinton G E. 2017. ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6): 84-90[DOI:10.1145/3065386]
Li G D, Zhang C J, Gao F and Zhang X Y. 2019. Doubleconvpool-structured 3D-CNN for hyperspectral remote sensing image classification. Journal of Image and Graphics, 24(4): 639-654
李冠东, 张春菊, 高飞, 张雪英. 2019. 双卷积池化结构的3D-CNN高光谱遥感影像分类方法. 中国图象图形学报, 24(4): 639-654[DOI:10.11834/jig.180422]
Li W J, Fu H H, Yu L, Gong P, Feng D L, Li C C and Clinton N. 2016. Stacked autoencoder-based deep learning for remote-sensing image classification: a case study of African land-cover mapping. International Journal of Remote Sensing, 37(23): 5632-5646[DOI:10.1080/01431161.2016.1246775]
Li S and Zhang E X. 2003. The decision tree classification and its application in land cover. Areal Research and Development, 22(1): 17-21
李爽, 张二勋. 2003. 基于决策树的遥感影像分类方法研究. 地域研究与发展, 22(1): 17-21[DOI:10.3969/j.issn.1003-2363.2003.01.005]
Liang J, Zhou J, Qian Y T, Wen L, Bai X and Gao Y S. 2017. On the sampling strategy for evaluation of spectral-spatial methods in hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(2): 862-880[DOI:10.1109/TGRS.2016.2616489]
Liang P, Shi W Z and Zhang X K. 2018. Remote sensing image classification based on stacked denoising autoencoder. Remote Sensing, 10(2): #16[DOI:10.3390/rs10010016]
Lu X Q, Zheng X T and Yuan Y. 2017. Remote sensing scene classification by unsupervised representation learning. IEEE Transactions on Geoscience and Remote Sensing, 55(9): 5148-5157[DOI:10.1109/TGRS.2017.2702596]
Martin G and Plaza A. 2012. Spatial-spectral preprocessing prior to endmember identification and unmixing of remotely sensed hyperspectral data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 5(2): 380-395[DOI:10.1109/JSTARS.2012.2192472]
Masci J, Meier U, Cireşan D and Schmidhuber J. 2011. Stacked convolutional auto-encoders for hierarchical feature extraction//Honkela T, Duch W, Girolami M and Kaski S, eds. Artificial Neural Networks and Machine Learning-ICANN. Berlin, Germany: Springer: 52-59[DOI: 10.1007/978-3-642-21735-7_7http://dx.doi.org/10.1007/978-3-642-21735-7_7]
Mei S H, Ji L Y, Geng Y H, Zhang Z, Xu L and Du Q. 2019a. Unsupervised spatial-spectral feature learning by 3D convolutional autoencoder for hyperspectral classification. IEEE Transactions on Geoscience and Remote Sensing, 57(9): 6808-6820[DOI:10.1109/TGRS.2019.2908756]
Mei X G, Pan E T, Ma Y, Dai X B, Huang J, Fan F, Du Q L, Zheng H and Ma J Y. 2019b. Spectral-spatial attention networks for hyperspectral image classification. Remote Sensing, 11(8): #963[DOI:10.3390/rs11080963]
Melgani F and Bruzzone L. 2004. Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience and Remote Sensing, 42(8): 1778-1790[DOI:10.1109/TGRS.2004.831865]
Mou L C, Ghamisi P and Zhu X X. 2017. Deep recurrent neural networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(7): 3639-3655[DOI:10.1109/TGRS.2016.2636241]
Paoletti M E, Haut J M, Plaza J and Plaza A. 2019. Deep learning classifiers for hyperspectral imaging: a review. ISPRS Journal of Photogrammetry and Remote Sensing, 158: 279-317[DOI:10.1016/j.isprsjprs.2019.09.006]
Paoletti M E, Haut J M, Plaza J and Plaza A. 2020. Scalable recurrent neural network for hyperspectral image classification. The Journal of Supercomputing, 76(11): 8866-8882[DOI:10.1007/s11227-020-03187-0]
Romero A, Gatta C and Camps-Valls G. 2016. Unsupervised deep feature extraction for remote sensing image classification. IEEE Transactions on Geoscience and Remote Sensing, 54(3): 1349-1362[DOI:10.1109/TGRS.2015.2478379]
Rußwurm M and Körner M. 2017. Temporal vegetation modelling using long short-term memory networks for crop identification from medium-resolution multi-spectral satellite images//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Honolulu, USA: IEEE: 1496-1504[DOI: 10.1109/CVPRW.2017.193http://dx.doi.org/10.1109/CVPRW.2017.193]
Song B Q, Li J, Mauro D M, Li P J, Plaza A, Bioucas-Dias J M, Benediktsson J A and Chanussot J. 2014. Remotely sensed image classification using sparse representations of morphological attribute profiles. IEEE Transactions on Geoscience and Remote Sensing, 52(8): 5122-5136[DOI:10.1109/TGRS.2013.2286953]
Swain P H and Hauska H. 1977. The decision tree classifier: design and potential. IEEE Transactions on Geoscience Electronics, 15(3): 142-147[DOI:10.1109/TGE.1977.6498972]
Tong Q X, Zhang B and Zheng L F. 2006. Hyperspectral Remote Sensing. Beijing: Higher Education Press
童庆禧, 张兵, 郑兰芬. 2006. 高光谱遥感: 原理、技术与应用. 北京: 高等教育出版社
Villa A, Benediktsson J A, Chanussot J and Jutten C. 2011. Hyperspectral image classification with independent component discriminant analysis. IEEE Transactions on Geoscience and Remote Sensing, 49(12): 4865-4876[DOI:10.1109/TGRS.2011.2153861]
Wan Y L, Zhong X W, Liu H and Qian Y R. 2021. Survey of application of convolutional neural network in classification of hyperspectral images[J/OL]. Computer Engineering and Applications. [2021-01-08]
万亚玲, 钟锡武, 刘慧, 钱育蓉. 2021. 卷积神经网络在高光谱图像分类中的应用综述[J/OL]. 计算机工程与应用. [2021-01-08].https://kns.cnki.net/kcms/detail/11.2127.TP.20210107.0841.002.htmlhttps://kns.cnki.net/kcms/detail/11.2127.TP.20210107.0841.002.html
Wang Z W, Sun J J, Yu Z Y and Bu Y Y. 2016. Review of remote sensing image classification based on support vector machine. Computer Science, 43(9): 11-17, 31
王振武, 孙佳骏, 于忠义, 卜异亚. 2016. 基于支持向量机的遥感图像分类研究综述. 计算机科学, 43(9): 11-17, 31[DOI:10.11896/j.issn.1002-137X.2016.9.002]
Zhang H K, Li Y and Jiang Y N. 2018. Deep learning for hyperspectral imagery classification: the state of the art and prospects. Acta Automatica Sinica, 44(6): 961-977
张号逵, 李映, 姜晔楠. 2018. 深度学习在高光谱图像分类领域的研究现状与展望. 自动化学报, 44(6): 961-977[DOI:10.16383/j.aas.2018.c170190]
Zhang L P, Zhang L F and Du B. 2016. Deep learning for remote sensing data: a technical tutorial on the state of the art. IEEE Geoscience and Remote Sensing Magazine, 4(2): 22-40[DOI:10.1109/MGRS.2016.2540798]
Zhao J, Zhong Y F, Shu H and Zhang L P. 2016. High-resolution image classification integrating spectral-spatial-location cues by conditional random fields. IEEE Transactions on Image Processing, 25(9): 4033-4045[DOI:10.1109/TIP.2016.2577886]
Zhao W Z and Du S H. 2016. Spectral-spatial feature extraction for hyperspectral image classification: a dimension reduction and deep learning approach. IEEE Transactions on Geoscience and Remote Sensing, 54(8): 4544-4554[DOI:10.1109/TGRS.2016.2543748]
Zhu J Z, Shi Q, Chen F E, Shi X D, Dong Z M and Qin Q Q. 2016. Research status and development trends of remote sensing big data. Journal of Image and Graphics, 21(11): 1425-1439
朱建章, 石强, 陈风娥, 史晓丹, 董泽民, 秦前清. 2016. 遥感大数据研究现状与发展趋势. 中国图象图形学报, 21(11): 1425-1439[DOI:10.11834/jig.20161102]
相关文章
相关作者
相关机构
京公网安备11010802024621