浏览全部资源
扫码关注微信
收稿：2020-12-29，
修回：2021-4-12，
录用：2021-4-19，
纸质出版：2021-08-16
移动端阅览
目的
2
场景分类是遥感领域一项重要的研究课题，但大都面向高分辨率遥感影像。高分辨率影像光谱信息少，故场景鉴别能力受限。而高光谱影像包含更丰富的光谱信息，具有强大的地物鉴别能力，但目前仍缺少针对场景级图像分类的高光谱数据集。为了给高光谱场景理解提供数据支撑，本文构建了面向场景分类的高光谱遥感图像数据集(hyperspectral remote sensing dataset for scene classification，HSRS-SC)。
方法
2
HSRS-SC来自黑河生态水文遥感试验航空数据，是目前已知最大的高光谱场景分类数据集，经由定标系数校正、大气校正等处理形成。HSRS-SC分为5个类别，共1 385幅图像，且空间分辨率较高(1 m)，波长范围广(380~1 050 nm)，同时蕴含地物丰富的空间和光谱信息。
结果
2
为提供基准结果，使用AlexNet、VGGNet-16、GoogLeNet在3种方案下组织实验。方案1仅利用可见光波段提取场景特征。方案2和方案3分别以加和、级联的形式融合可见光与近红外波段信息。结果表明有效利用高光谱影像不同波段信息有利于提高分类性能，最高分类精度达到93.20%。为进一步探索高光谱场景的优势，开展了图像全谱段场景分类实验。在两种训练样本下，高光谱场景相比RGB图像均取得较高的精度优势。
结论
2
HSRS-SC可以反映详实的地物信息，能够为场景语义理解提供良好的数据支持。本文仅利用可见光和近红外部分波段信息，高光谱场景丰富的光谱信息尚未得到充分挖掘。后续可在HSRS-SC开展高光谱场景特征学习及分类研究。
Objective
2
Remote sensing scene classification is an important research topic in remote sensing community
and it has provided important data or decision support for land resource planning
coverage mapping
ecological environment monitoring
and other real-world applications. In scene classification
extracting scene-level discriminative features is a key factor to bridge the "semantic gap" between low-level visual attributes and high-level understanding of images. Deep learning models are currently showing excellent performance in remote sensing image analysis
and many convolutional neural network (CNN)-based methods have been widely proposed in feature extraction and classification of remote sensing scene images. Although the aforementioned methods have achieved good results
they are all designed for scene images of high spatial resolution
such as University of California(UC) Merced Land-Use
WHU-RS19
scene image dataset designed by RS_IDEA Group in Wuhan University(SIRI-WHU)
RSSCN7
aerial image dataset(AID)
a publicly available benchmark for remote sensing image scene classification created by Northwestern Polytechnical University(NWPU-RESISC45)
and optical imagery analysis and learning(OPTIMAL-31) datasets. Remote sensing data of high spatial resolution can present spatial details of ground objects. However
they contain less spectral information. As a result
their discriminative ability is relatively limited in scene classification. Hyperspectral images have abundant spectral information
and they have strong discriminative ability for ground objects. However
the existing datasets of hyperspectral images (e.g.
Indian Pines
Pavia University
Washington DC Mall
Salinas
and Xiongan New Area) are mostly oriented toward pixel-level classification and are difficult to directly apply on research of scene-level image classification. Tiangong-1 hyperspectral remote sensing scene classification dataset (TG1HRSSC) is produced for scene-level image interpretation. However
the TG1HRSSC dataset is small (204 scene images) and has inconsistent image bands. A hyperspectral remote sensing dataset is constructed for scene classification (HSRS-SC) in this study to overcome the aforementioned disadvantages. The dataset can provide a good benchmark platform for evaluating intelligent algorithms of hyperspectral scene classification.
Method
2
The HSRS-SC is derived from the aerial data of the Heihe Watershed Allied Telemetry Experimental Research (HiWATER)
and raw data can be downloaded from the National Tibetan Plateau/Third Pole Environment Data Center. A large-scale dataset is finally formed after calibration coefficient correction
atmospheric correction
image cropping
and manual visual annotation. To the best of our knowledge
the HSRS-SC is currently the largest hyperspectral scene dataset
and it contains 1 385 hyperspectral scene images which have been resized to 256×256 pixels. The dataset is divided into 5 categories
and the number of samples in each category ranges from 154 to 485. In the HSRS-SC dataset
each hyperspectral scene image has a high spatial resolution (1 m) and a wide wavelength range (from visible light to near-infrared
380~1 050 nm
48 bands)
which can reflect the detailed spatial and spectral information of ground objects
including cars
roadway
buildings
and vegetation. Specifically
the blue band (450~520 nm) has a certain penetration ability to water bodies; the green band (520~600 nm) is more sensitive to the reflection of vegetation; the red band (630~690 nm) is the main absorption band of chlorophyll; the near-infrared band (760900 nm) reflects the strong reflection of vegetation
and it is also the absorption band of water bodies. The dataset will be publicly available in the near future
and it can be used for non-commercial academic research.
Result
2
This study uses three classic deep models (i.e.
AlexNet
VGGNet-16
and GoogLeNet) to organize experiments under three different schemes for providing benchmark results of HSRS-SC dataset. In the first scheme
false color images are synthesized from the 19th
13th
and 7th bands of visible light range
and then
they are fed into deep models to extract global scene features. In the second and third schemes
information of the visible light (19th
13th
and 7th) and near-infrared (46th
47th
and 48th) bands are comprehensively utilized by fusion approaches of addition and concatenation
respectively. In the experiments
10 samples per class are randomly selected to finetune pre-trained CNN models
and the rest are used for test set. The experimental results on the HSRS-SC dataset show that the effective utilization of information from different bands of hyperspectral images improves the classification performance
and concatenation fusion achieves better results than addition fusion. Comparing the three CNN models shows that the VGGNet-16 model is more suitable for the HSRS-SC dataset
and the highest overall classification accuracy reaches 93.20%. Furthermore
this study shows confusion matrices of different methods. Effective use of spectral information can reduce the confusion of semantic categories given that vegetation
buildings
roads
water bodies
and rocks have great differences in absorption and reflection at different bands. This study also organizes hyperspectral scene classification experiments to further explore the advantages of hyperspectral scenes. Hyperspectral scenes have a higher accuracy advantage than RGB images under the two training samples.
Conclusion
2
The abovementioned experimental results show that the HSRS-SC dataset can reflect detailed information of ground objects
and it can provide effective data support for semantic understanding of remote sensing scenes. Although experiments in this study adopt three different schemes to utilize the information of the visible light (19th
13th
and 7th bands) and near-infrared (46th
47th
and 48th bands) of the hyperspectral scenes
the rich spectral information has not been fully explored. For the future work
suitable models will be designed for feature extraction and classification of hyperspectral remote sensing scenes. We will also further expand the HSRS-SC dataset to ensure its practicality by supplementing more semantic categories and the total number of samples and increasing the diversity of data.
Cen Y, Zhang L F, Zhang X, Wang Y M, Qi W C, Tang S L and Zhang P. 2020. Aerial hyperspectral remote sensing classification dataset of Xiongan New Area (Matiwan Village). Journal of Remote Sensing, 24(11): 1299-1306
岑奕, 张立福, 张霞, 王跃明, 戚文超, 汤森林, 张鹏. 2020. 雄安新区马蹄湾村航空高光谱遥感影像分类数据集. 遥感学报, 24(11): 1299-1306 [DOI: 10.11834/jrs.20209065]
Chaib S, Liu H, Gu Y F and Yao H X. 2017. Deep feature fusion for VHR remote sensing scene classification. IEEE Transactions on Geoscience and Remote Sensing, 55(8): 4775-4784[DOI: 10.1109/TGRS.2017.2700322]
Chen Y S, Huang L B, Zhu L, Yokoya N and Jia X P. 2019. Fine-grained classification of hyperspectral imagery based on deep learning. Remote Sensing, 11(22): #2690[DOI: 10.3390/rs11222690]
Cheng G, Yang C Y, Yao X W, Guo L and Han J W. 2018. When deep learning meets metric learning: remote sensing image scene classification via learning discriminative CNNs. IEEE Transactions on Geoscience and Remote Sensing, 56(5): 2811-2821[DOI: 10.1109/TGRS.2017.2783902]
Fang J, Yuan Y, Lu X Q and Feng Y C. 2019. Robust space-frequency joint representation for remote sensing image scene classification. IEEE Transactions on Geoscience and Remote Sensing, 57(10): 7492-7502[DOI: 10.1109/TGRS.2019.2913816]
He M Y, Chang W J and Mei S H. 2013. Advance in feature mining from hyperspectral remote sensing data. Spacecraft Recovery and Remote Sensing, 34(1): 1-12
何明一, 畅文娟, 梅少辉. 2013. 高光谱遥感数据特征挖掘技术研究进展. 航天返回与遥感, 34(1): 1-12 [DOI: 10.3969/j.issn.1009-8518.2013.01.001]
He N J, Fang L Y, Li S T, Plaza J and Plaza A. 2020. Skip-connected covariance network for remote sensing scene classification. IEEE Transactions on Neural Networks and Learning Systems, 31(5): 1461-1474[DOI: 10.1109/TNNLS.2019.2920374]
Huang H and Xu K J. 2019. Combing triple-part features of convolutional neural networks for scene classification in remote sensing. Remote Sensing, 11(14): #1687[DOI: 10.3390/rs11141687]
Huang H, Xu K J and Shi G Y. 2020. Scene classification of high-resolution remote sensing image by multi-scale and multi-feature fusion. Acta Electronica Sinica, 48(9): 1824-1833
黄鸿, 徐科杰, 石光耀. 2020. 联合多尺度多特征的高分遥感图像场景分类. 电子学报, 48(9): 1824-1833 [DOI: 10.3969/j.issn.0372-2112.2020.09.021]
Huang H, Shi G Y, He H B, Duan Y L and Luo F L. 2020. Dimensionality reduction of hyperspectral imagerybased on spatial-spectral manifold learning. IEEE Transactions on Cybernetics, 50(6): 2604-2616[DOI: 10.1109/TCYB.2019.2905793]
Krizhevsky A, Sutskever I and Hinton G E. 2017. ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6): 84-90[DOI: 10.1145/3065386]
Li D R. 2019. The intelligent processing and service of spatiotemporal big data. Journal of Geo-information Science, 21(12): 1825-1831
李德仁. 2019. 论时空大数据的智能处理与服务. 地球信息科学学报, 21(12): 1825-1831 [DOI: 10.12082/dqxxkx.2019.190694]
Li X, Liu S M, Xiao Q, Ma M G, Jin R, Che T, Wang W Z, Hu X L, Xu Z W, Wen J G and Wang L X. 2017. A multiscale dataset for understanding complex eco-hydrological processes in a heterogeneous oasis system. Scientific Data, 4: #170083[DOI: 10.1038/sdata.2017.83]
Liu K, Zhou Z, Li S Y, Liu Y F, Wan X, Liu Z W, Tan H and Zhang W F. 2020. Scene classification dataset using the Tiangong-1 hyperspectral remote sensing imagery and its applications. Journal of Remote Sensing, 24(9): 1077-1087
刘康, 周壮, 李盛阳, 刘云飞, 万雪, 刘志文, 谭洪, 张万峰. 2020. 天宫一号高光谱遥感场景分类数据集及应用. 遥感学报, 24(9): 1077-1087 [DOI: 10.11834/jrs.20209323]
Liu Q C, Xiao L, Yang J X and Chan J C W. 2020. Content-guided convolutional neural network for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 58(9): 6124-6137[DOI: 10.1109/TGRS.2020.2974134]
Luo F L. 2017. Sparse manifold learning for hyperspectral imagery. Acta Geodaetica et Cartographica Sinica, 46(3): #400
罗甫林. 2017. 高光谱图像稀疏流形学习方法研究. 测绘学报, 46(3): #400 [DOI: 10.11947/j.AGCS.2017.20160621]
Luo F L, Zhang L P, Du B and Zhang L F. 2020. Dimensionality reduction with enhanced hybrid-graph discriminant learning for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 58(8): 5336-5353[DOI: 10.1109/TGRS.2020.2963848]
Penatti O A B, Nogueira K and dos Santos J A. 2015. Do deep features generalize from everyday objects to remote sensing and aerial scenes domains?//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Boston, USA: IEEE: 44-51[ DOI:10.1109/CVPRW.2015.7301382 http://dx.doi.org/10.1109/CVPRW.2015.7301382 ]
Rasti B, Hong D F, Hang R L, Ghamisi P, Kang X D, Chanussot J and Benediktsson J A. 2020. Feature extraction for hyperspectral imagery: the evolution from shallow to deep: overview and toolbox. IEEE Geoscience and Remote Sensing Magazine, 8(4): 60-88[DOI: 10.1109/MGRS.2020.2979764]
Sheng G F, Yang W, Xu T and Sun H. 2012. High-resolution satellite scene classification using a sparse coding based multiple feature combination. International Journal of Remote Sensing, 33(8): 2395-2412[DOI: 10.1080/01431161.2011.608740]
Simonyan K and Zisserman A. 2015. Very deep convolutional networks for large-scale image recognition[EB/OL]. [2020-12-01] . https://arxiv.org/pdf/1409.1556v4.pdf https://arxiv.org/pdf/1409.1556v4.pdf
Sun W W, YangG, Chen C, Chang M H, Huang K, Meng X Z and Liu L Y. 2020. Development status and literature analysis of China's earth observation remote sensing satellites. Journal of Remote Sensing, 24(5): 479-510
孙伟伟, 杨刚, 陈超, 常明会, 黄可, 孟祥珍, 刘良云. 2020. 中国地球观测遥感卫星发展现状及文献分析. 遥感学报, 24(5): 479-510 [DOI: 10.11834/jrs.20209464]
Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V and Rabinovich A. 2015. Going deeper with convolutions//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE: 1-9[ DOI:10.1109/CVPR.2015.7298594 http://dx.doi.org/10.1109/CVPR.2015.7298594 ]
Tan K, Wang X and Du P J. 2019. Research progress of the remote sensing classification combining deep learning and semi-supervised learning. Journal of Image and Graphics, 24(11): 1823-1841
谭琨, 王雪, 杜培军. 2019. 结合深度学习和半监督学习的遥感影像分类进展. 中国图象图形学报, 24(11): 1823-1841 [DOI: 10.11834/jig.190348]
Tong Q X, Zhang B and Zhang L F. 2016. Current progress of hyperspectral remote sensing in China. Journal of Remote Sensing, 20(5): 689-707
童庆禧, 张兵, 张立福. 2016. 中国高光谱遥感的前沿进展. 遥感学报, 20(5): 689-707 [DOI: 10.11834/jrs.20166264]
Wang Q, Liu S T, Chanussot J and Li X L. 2019. Scene classification with recurrent attention of VHR remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 57(2): 1155-1167[DOI: 10.1109/TGRS.2018.2864987]
Wang Q, Sun L, Wang Y, Zhou M, Hu M H, Chen J G, Wen Y and Li Q L. 2021. Identification of melanoma from hyperspectral pathology image using 3D convolutional networks. IEEE Transactions on Medical Imaging, 40(1): 218-227[DOI: 10.1109/TMI.2020.3024923]
Xia G S, Hu J W, Hu F, Shi B G, Bai X, Zhong Y F, Zhang L P and Lu X Q. 2017. AID: a benchmark data set for performance evaluation of aerial scene classification. IEEE Transactions on Geoscience and Remote Sensing, 55(7): 3965-3981[DOI: 10.1109/TGRS.2017.2685945]
Xiao L, Liu P F and Li H. 2020. Progress and challenges in the fusion of multisource spatial-spectral remote sensing images. Journal of Image and Graphics, 25(5): 851-863
肖亮, 刘鹏飞, 李恒. 2020. 多源空-谱遥感图像融合方法进展与挑战. 中国图象图形学报, 25(5): 851-863 [DOI: 10.11834/jig.190620]
Xiao Q and Wen J G. 2017. HiWATER: visible and near-infrared hyperspectral radiometer (Jun. 29, 2012)[EB/OL]. National Tibetan Plateau Data Center. [2020-12-01]
肖青, 闻建光. 2017. 黑河生态水文遥感试验: 可见光近红外高光谱航空遥感(2012年6月29日)[EB/OL]. 国家青藏高原科学数据中心. [2020-12-01] . http://www.tpdc.ac.cn/zh-hans/data/1e7e8a06-1e10-4fd3-a94e-d83e463a835e/ http://www.tpdc.ac.cn/zh-hans/data/1e7e8a06-1e10-4fd3-a94e-d83e463a835e/ )[ DOI:10.3972/hiwater.012.2013.db http://dx.doi.org/10.3972/hiwater.012.2013.db .
Xu K J, Huang H, Deng P F and Shi G Y. 2020a. Two-stream feature aggregation deep neural network for scene classification of remote sensing images. Information Sciences, 539: 250-268[DOI: 10.1016/j.ins.2020.06.011]
Xu K J, Huang H, Li Y and Shi G Y. 2020b. Multilayer feature fusion network for scene classification in remote sensing. IEEE Geoscience and Remote Sensing Letters, 17(11): 1894-1898[DOI: 10.1109/LGRS.2019.2960026]
Yang Y and Newsam S. 2010. Bag-of-visual-words and spatial extensions for land-use classification//Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems. San Jose, USA: ACM: 270-279[ DOI:10.1145/1869790.1869829 http://dx.doi.org/10.1145/1869790.1869829 ]
Yu H Y, Gao L R, Liao W Z, Zhang B, Zhuang L N, Song M P and Chanussot J. 2020. Global spatial and local spectral similarity-based manifold learning group sparse representation for hyperspectral imagery classification. IEEE Transactions on Geoscience and Remote Sensing, 58(5): 3043-3056[DOI: 10.1109/TGRS.2019.2947032]
Yuan J W, Wu C, Du B, Zhang L P and Wang S G. 2020. Analysis of landscape pattern on urban land use based on GF-5 hyperspectral data. Journal of Remote Sensing, 24(4): 465-478
袁静文, 武辰, 杜博, 张良培, 王树根. 2020. 高分五号高光谱遥感影像的城市土地利用景观格局分析. 遥感学报, 24(4): 465-478 [DOI: 10.11834/jrs.20209252]
Zhang F, Du B and Zhang L P. 2015. Saliency-guided unsupervised feature learning for scene classification. IEEE Transactions on Geoscience and Remote Sensing, 53(4): 2175-2184[DOI: 10.1109/TGRS.2014.2357078]
Zhang M M, Li W and Du Q. 2018. Diverse region-based CNN for hyperspectral image classification. IEEE Transactions on Image Processing, 27(6): 2623-2634[DOI: 10.1109/TIP.2018.2809606]
Zhao B, Zhong Y F, Xia G S and Zhang L P. 2016. Dirichlet-derived multiple topic scene classification model for high spatial resolution remote sensing imagery. IEEE Transactions on Geoscience and Remote Sensing, 54(4): 2108-2123[DOI: 10.1109/TGRS.2015.2496185]
Zhao Z M, Gao L R, Chen D, Yue A Z, Chen J B, Liu D S, Yang J and Meng Y. 2019. Development of satellite remote sensing and image processing platform. Journal of Image and Graphics, 24(12): 2098-2110
赵忠明, 高连如, 陈东, 岳安志, 陈静波, 刘东升, 杨健, 孟瑜. 2019. 卫星遥感及图像处理平台发展. 中国图象图形学报, 24(12): 2098-2110 [DOI: 10.11834/jig.190450]
Zhu Q Q, Zhong Y F, Zhao B, Xia G S and Zhang L P. 2016. Bag-of-visual-words scene classifier with local and global features for high spatial resolution remote sensing imagery. IEEE Geoscience and Remote Sensing Letters, 13(6): 747-751[DOI: 10.1109/LGRS.2015.2513443]
Zou Q, Ni L H, Zhang T and Wang Q. 2015. Deep learning based feature selection for remote sensing scene classification. IEEE Geoscience and Remote Sensing Letters, 12(11): 2321-2325[DOI:10.1109/LGRS.2015.2475299]
相关作者
相关机构
京公网安备11010802024621
