浏览全部资源
扫码关注微信
Received：10 January 2024，
Revised：2024-02-23，
Published：16 June 2024
移动端阅览
遥感对地观测中普遍存在多平台、多传感器和多角度的多源数据，为遥感场景解译提供协同互补信息。然而，现有的场景解译方法需要根据不同遥感场景数据训练模型，或者对测试数据标准化以适应现有模型，训练成本高、响应周期长，已无法适应多源数据协同解译的新阶段。跨域遥感场景解译将已训练的老模型迁移到新的应用场景，通过模型复用以适应不同场景变化，利用已有领域的知识来解决未知领域问题。本文以跨域遥感场景解译为主线，综合分析国内外文献，结合场景识别和目标识别两个典型任务，论述国内外研究现状、前沿热点和未来趋势，梳理总结跨域遥感场景解译的常用数据集和统一的实验设置。本文实验数据集及检测结果的公开链接为：
https://github.com/XiangtaoZheng/CDRSSI
https://github.com/XiangtaoZheng/CDRSSI
。
In remote sensing of Earth observation， multi-source data can be captured by multiple platforms， multiple sensors， and multiple perspectives. These data provide complementary information for interpreting remote sensing scenes. Although these data offer richer information， they also increase the demand for model depth and complexity. Deep learning plays a pivotal role in unlocking the potential of remote sensing data by delving deep into the semantic layers of scenes and extracting intricate features from images. Recent advancements in artificial intelligence have greatly enhanced this process. However， deep learning networks have limitations when applied to remote sensing images. 1） The huge number of parameters and the difficulty in training， as well as the over-reliance on labeled training data， can affect these images. Remote sensing images are characterized by “data miscellaneous marking difficulty”， which makes manual labeling insufficient for meeting the training needs of deep learning. 2） Variations in remote sensing platforms， sensors， shooting angles， resolution， time， location， and weather can all impact remote sensing images. Thus， the interpreted images and training samples cannot have the same distribution. This inconsistency results in weak generalization ability in existing models， especially when dealing with data from different distributions. To address this issue， cross-domain remote sensing scene interpretation aims to train a model on labeled remote sensing scene data （source domain） and apply it to new， unlabeled scene data （target domain） in an appropriate way. This approach reduces the dependence on target domain data and relaxes the assumption of the same distribution in existing deep learning tasks. The shallow layers of convolutional neural networks can be used as general-purpose feature extractors， but deeper layers are more task-specific and may introduce bias when applied to other tasks. Therefore， the migration model must be modified to accomplish the task of interpreting the target domain. Cross-domain interpretation tasks aim to establish a model that can adapt to various scene changes by utilizing migration learning， domain adaptation and other techniques for reducing model prediction inaccuracy caused by changes in the data domain. This approach improves the robustness and generalization ability of the model. Interpreting cross-domain remote sensing scenes typically requires using data from multiple remote sensing sources， including radar， aerial and satellite imagery. These images may have varying views， resolutions， wavelength bands， lighting conditions and noise levels. They may also originate from different locations or sensors. As the Global Earth Observation Systems continues to advance， remote sensing images now include cross-platform， cross-sensor， cross-resolution， and cross-region， which results in enormous distributional variances. Therefore， the study of cross-domain remote sensing scene interpretation is essential for the commercial use of remote sensing data and has theoretical and practical importance. This report categorizes scene decoding tasks into four main types based on the labeled set of data： methods based on closed-set domain adaptation， partial-domain adaptation， open-set domain adaptation and generalized domain adaptation. Approaches based on closed-set domain adaptation focus on tasks where the label set of the target domain is the same as that of the source domain. Partial domain adaptation focuses on tasks where the label set of the target domain is a subset of the source domain. Open-set domain adaptation aims to research tasks where the label set of the source domain is a subset of the label set of the target domain， and it does not apply restrictions in the approach of generalized domain adaptation. This study provides an in-depth investigation of two typical tasks in cross-domain remote sensing interpretation： scene recognition and target knowledge. The first part of the study utilizes domestic and international literature to provide a comprehensive assessment of the current research status of the four types of methods. Within the target recognition task， cross-domain tasks are further subdivided into cross-domain for visible light data and cross-domain from visible light to Synthetic Aperture Radar images. After a quantitative analysis of the sample distribution characteristics of different datasets， a unified experimental setup for cross-domain tasks is proposed. In the scene classification task， the dataset is explored by classifying it according to the label set categorization， and specific examples are given to provide the corresponding experimental setup for the readers’ reference. The fourth part of the study discusses the research trends in cross-domain remote sensing interpretation， which highlights four challenging research directions： few-shot learning， source domain data selection， multi-source domain interpretation， and cross-modal interpretation. These areas will be important directions for the future development of remote sensing scene interpretation， which offers potential choices for readers’ subsequent research directions.
Al Rahhal M M ， Bazi Y ， Al-Hwiti H ， Alhichri H and Alajlan N . 2021 . Adversarial learning for knowledge adaptation from multiple remote sensing sources . IEEE Geoscience and Remote Sensing Letters ， 18 （ 8 ）： 1451 - 1455 ［ DOI： 10.1109/LGRS.2020.3003566 http://dx.doi.org/10.1109/LGRS.2020.3003566 ］
Chen S Q ， Zhan R H ， Wang W and Zhang J . 2022 . Domain adaptation for semi-supervised ship detection in SAR images . IEEE Geoscience and Remote Sensing Letters ， 19 ： # 4507405 ［ DOI： 10.1109/lgrs.2022.3171789 http://dx.doi.org/10.1109/lgrs.2022.3171789 ］
Chen Y ， Wang H R ， Li W ， Sakaridis C ， Dai D X and Van Gool L . 2021 . Scale-aware domain adaptive faster R-CNN . International Journal of Computer Vision ， 129 （ 7 ）： 2223 - 2243 ［ DOI： 10.1007/s11263-021-01447-x http://dx.doi.org/10.1007/s11263-021-01447-x ］
Cheng G ， Han J W ， Zhou P C and Guo L M . 2014 . Multi-class geospatial object detection and geographic image classification based on collection of part detectors . ISPRS Journal of Photogrammetry and Remote Sensing ， 98 ： 119 - 132 ［ DOI： 10.1016/J.ISPRSJPRS.2014.10.002 http://dx.doi.org/10.1016/J.ISPRSJPRS.2014.10.002 ］
Cheng G ， Han J W and Lu X Q . 2017 . Remote sensing image scene classification： Benchmark and State of the Art . Proceedings of the IEEE ， 105 （ 10 ）： 1865 - 1883
Du Y D ， Feng L ， Tao P ， Gong X and Wang J . 2023 . Meta-transfer learning in cross-domain image classification with few-shot learning . Journal of Image and Graphics ， 28 （ 9 ）： 2899 - 2912
杜彦东 ， 冯林 ， 陶鹏 ， 龚勋 ， 王俊 . 2023 . 元迁移学习在少样本跨域图像分类中的研究 . 中国图象图形学报 ， 28 （ 9 ）： 2899 - 2912 ［ DOI： 10.11834/jig.220664 http://dx.doi.org/10.11834/jig.220664 ］
Fu B ， Cao Z J ， Long M S and Wang J M . 2020 . Learning to detect open classes for universal domain adaptation // Proceedings of the 16th European Conference on Computer Vision . Glasgow， UK ： Springer： 567 - 583 ［ DOI： 10.1007/978-3-030-58555-6_34 http://dx.doi.org/10.1007/978-3-030-58555-6_34 ］
Gong T F ， Zheng X T and Lu X Q . 2021 . Cross-domain scene classification by integrating multiple incomplete sources . IEEE Transactions on Geoscience and Remote Sensing ， 59 （ 12 ）： 10035 - 10046 ［ DOI： 10.1109/TGRS.2020.3034344 http://dx.doi.org/10.1109/TGRS.2020.3034344 ］
Guo Q ， Pu S L ， Zhang S F and Li B . 2022 . Cross-domain object detection based foggy image enhancement . Journal of Image and Graphics ， 27 （ 5 ）： 1481 - 1492
郭强 ， 浦世亮 ， 张世峰 ， 李波 . 2022 . 适合跨域目标检测的雾霾图像增强 . 中国图象图形学报 ， 27 （ 5 ）： 1481 - 1492 ［ DOI： 10.11834/jig.210994 http://dx.doi.org/10.11834/jig.210994 ］
He J J ， Wu L ， Tao C F and Lv F M . 2023 . Source-free domain adaptation with unrestricted source hypothesis . Pattern Recognition ， 149 ： #110246 . ［ DOI： 10.1016/j.patcog.2023.110246 http://dx.doi.org/10.1016/j.patcog.2023.110246 ］
Huang W ， Shi Y L ， Xiong Z T ， Wang Q and Zhu X X . 2023 . Semi-supervised bidirectional alignment for remote sensing cross-domain scene classification . ISPRS Journal of Photogrammetry and Remote Sensing ， 195 ： 192 - 203 ［ DOI： 10.1016/j.isprsjprs.2022.11.013 http://dx.doi.org/10.1016/j.isprsjprs.2022.11.013 ］
Isobe T ， Jia X ， Chen S J ， He J Z ， Shi Y J ， Liu J Z ， Lu H C and Wang S J . 2021 . Multi-target domain adaptation with collaborative consistency learning // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Nashville， USA ： IEEE ［ DOI： 10.1109/cvpr46437.2021.00809 http://dx.doi.org/10.1109/cvpr46437.2021.00809 ］
Li H F ， Jiang H ， Gu X ， Peng J ， Li W B ， Hong L and Tao C . 2020a . CLRS： continual learning benchmark for remote sensing image scene classification . Sensors ， 20 （ 4 ）： # 1226 ［ DOI： 10.3390/s20041226 http://dx.doi.org/10.3390/s20041226 ］
Li J W ， Qu C W and Shao J Q . 2017 . Ship detection in SAR images based on an improved faster R-CNN // Proceedings of 2017 SAR in Big Data Era： Models， Methods and Applications . Beijing， China ： IEEE： 1 - 6 ［ DOI： 10.1109/BIGSARDATA.2017.8124934 http://dx.doi.org/10.1109/BIGSARDATA.2017.8124934 ］
Li K ， Wan G ， Cheng G ， Meng L Q and Han J W . 2020b . Object detection in optical remote sensing images： a survey and a new benchmark . ISPRS Journal of Photogrammetry and Remote Sensing ， 159 ： 296 - 307 ［ DOI： 10.1016/J.ISPRSJPRS.2019.11.023 http://dx.doi.org/10.1016/J.ISPRSJPRS.2019.11.023 ］
Liu B ， Li L G ， Xiao Q ， Ni W and Yang Z . 2022 . Remote sensing fine-grained ship data augmentation pipeline with local-aware progressive image-to-image translation . IEEE Transactions on Geoscience and Remote Sensing ， 60 ： # 5631716 ［ DOI： 10.1109/TGRS.2022.3211517 http://dx.doi.org/10.1109/TGRS.2022.3211517 ］
Liu Z K ， Yuan L ， Weng L B and Yang Y P . 2017 . A high resolution optical satellite image dataset for ship recognition and some new baselines // Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods . Porto， Portugal ： SciTePress： 324 - 331 ［ DOI： 10.5220/0006120603240331 http://dx.doi.org/10.5220/0006120603240331 ］
Lu X Q ， Gong T F and Zheng X T . 2020 . Multisource compensation network for remote sensing cross-domain scene classification . IEEE Transactions on Geoscience and Remote Sensing ， 58 （ 4 ）： 2504 - 2515 ［ DOI： 10.1109/TGRS.2019.2951779 http://dx.doi.org/10.1109/TGRS.2019.2951779 ］
Ma X H ， Gao J Y and Xu C S . 2021 . Active universal domain adaptation // Proceedings of 2021 IEEE/CVF International Conference on Computer Vision . Montreal， Canada ： IEEE ［ DOI： 10.1109/iccv48922.2021.00884 http://dx.doi.org/10.1109/iccv48922.2021.00884 ］
Ngo B H ， Chae Y J ， Park J H ， Kim J H and Cho S I . 2023 . Easy-to-hard structure for remote sensing scene classification in multitarget domain adaptation . IEEE Transactions on Geoscience and Remote Sensing ， 61 ： # 4700215 ［ DOI： 10.1109/TGRS.2023.3235886 http://dx.doi.org/10.1109/TGRS.2023.3235886 ］
Ngo B H ， Kim J H ， Park S J and Cho S I . 2022 . Collaboration between multiple experts for knowledge adaptation on multiple remote sensing sources . IEEE Transactions on Geoscience and Remote Sensing ， 60 ： # 4707815 ［ DOI： 10.1109/TGRS.2022.3190476 http://dx.doi.org/10.1109/TGRS.2022.3190476 ］
Nguyen-Meidine L T ， Belal A ， Kiran M ， Dolz J ， Blais-Morin L A and Granger E . 2021 . Unsupervised multi-target domain adaptation through knowledge distillation // Proceedings of 2021 IEEE Winter Conference on Applications of Computer Vision . Waikoloa，USA ： IEEE： 1339 - 1347 ［ DOI： 10.1109/wacv48630.2021.00138 http://dx.doi.org/10.1109/wacv48630.2021.00138 ］
Niu B ， Pan Z X ， Chen K Y ， Hu Y X and Lei B . 2023 . Open set domain adaptation via instance affinity metric and fine-grained alignment for remote sensing scene classification . IEEE Geoscience and Remote Sensing Letters ， 20 ： # 6005805 ［ DOI： 10.1109/LGRS.2023.3276968 http://dx.doi.org/10.1109/LGRS.2023.3276968 ］
Niu B ， Pan Z X ， Wu J X ， Hu Y X and Lei B . 2022 . Multi-representation dynamic adaptation network for remote sensing scene classification . IEEE Transactions on Geoscience and Remote Sensing ， 60 ： # 5633119 ［ DOI： 10.1109/TGRS.2022.3217180 http://dx.doi.org/10.1109/TGRS.2022.3217180 ］
Saha S ， Zhao S and Zhu X X . 2022 . Multitarget domain adaptation for remote sensing classification using graph neural network . IEEE Geoscience and Remote Sensing Letters ， 19 ： # 6506505 ［ DOI： 10.1109/LGRS.2022.3149950 http://dx.doi.org/10.1109/LGRS.2022.3149950 ］
Shi ， R. Zhao ， B. Pan ， Z. Zou and Z. Shi . 2023 . Unsupervised multimodal remote sensing image registration via domain adaptation . IEEE Transactions on Geoscience and Remote Sensing ， 61 ， 1 - 11 ［DOI］： 10.1109/TGRS.2023.3333889.
Sun X ， Wang Z R ， Sun Y R ， Diao W H ， Zhang Y ， Fu K . 2019 . AIR-SARShip-1.0： High resolution SAR ship detection dataset . Journal of Radars ， 8 （ 6 ）： 852 - 862
孙显 ， 王智睿 ， 孙元睿 ， 刁文辉 ， 张跃 ， 付琨 . 2019 . AIR-SARShip-1.0：高分辨率SAR舰船检测数据集 . 雷达学报 ， 8 （ 6 ）： 852 - 862 ［ DOI： 10.12000/JR19097 http://dx.doi.org/10.12000/JR19097 ］
Tasar O ， Happy S L ， Tarabalka Y and Alliez P . 2020 . SEMI2 I： semantically consistent image-to-image translation for domain adaptation of remote sensing data //2020 IEEE International Geoscience and Remote Sensing Symposium. Waikoloa， USA ： IEEE： 1837 - 1840 ［ DOI： 10.1109/IGARSS39084.2020.9323711 http://dx.doi.org/10.1109/IGARSS39084.2020.9323711 ］
Wang K ， Zhang G and Leung H . 2019a . SAR target recognition based on cross-domain and cross-task transfer learning . IEEE Access ， 7 ： 153391 - 153399 ［ DOI： 10.1109/ACCESS.2019.2948618 http://dx.doi.org/10.1109/ACCESS.2019.2948618 ］
Wang Q ， Liu S T ， Chanussot J and Li X L . 2019b . Scene classification with recurrent attention of VHR remote sensing images . IEEE Transactions on Geoscience and Remote Sensing ， 57 （ 2 ）： 1155 - 1167 ［ DOI： 10.1109/TGRS.2018.2864987 http://dx.doi.org/10.1109/TGRS.2018.2864987 ］
Wang S Y ， Hou D Y and Xing H Q . 2023 . A self-supervised-driven open-set unsupervised domain adaptation method for optical remote sensing image scene classification and retrieval . IEEE Transactions on Geoscience and Remote Sensing ， 61 ： # 5605515 ［ DOI： 10.1109/TGRS.2023.3260873 http://dx.doi.org/10.1109/TGRS.2023.3260873 ］
Wang Y ， Xu C F ， Liu C W and Li Z K . 2022 . Context information refinement for few-shot object detection in remote sensing images . Remote Sensing ， 14 （ 14 ）： # 3255 ［ DOI： 10.3390/rs14143255 http://dx.doi.org/10.3390/rs14143255 ］
Wang Y Y ， Wang C ， Zhang H ， Dong Y B and Wei S S . 2019c . A SAR dataset of ship detection for deep learning under complex backgrounds . Remote Sensing ， 11 （ 7 ）： # 765 ［ DOI： 10.3390/rs11070765 http://dx.doi.org/10.3390/rs11070765 ］
Wei S J ， Zeng X F ， Qu Q Z ， Wang M ， Su H and Shi J . 2020 . HRSID： a high-resolution SAR images dataset for ship detection and instance segmentation . IEEE Access ， 8 ： 120234 - 120254 ［ DOI： 10.1109/ACCESS.2020.3005861 http://dx.doi.org/10.1109/ACCESS.2020.3005861 ］
Xia G S ， Bai X ， Ding J ， Zhu Z ， Belongie S ， Luo J B ， Datcu M ， Pelillo M and Zhang L P . 2018 . DOTA： a large-scale dataset for object detection in aerial images // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City， USA ： IEEE： 3974 - 3983 ［ DOI： 10.1109/CVPR.2018.00418 http://dx.doi.org/10.1109/CVPR.2018.00418 ］
Xiao Z F ， Long Y ， Li D R ， Wei C S ， Tang G F and Liu J Y . 2017 . High-resolution remote sensing image retrieval based on CNNs from a dimensional perspective . Remote Sensing ， 9 （ 7 ）： # 725 ［ DOI： 10.3390/rs9070725 http://dx.doi.org/10.3390/rs9070725 ］
Xu C J ， Zheng X T and Lu X Q . 2022a . Multi-level alignment network for cross-domain ship detection . Remote Sensing ， 14 （ 10 ）： # 2389 ［ DOI： 10.3390/rs14102389 http://dx.doi.org/10.3390/rs14102389 ］
Xu Q S ， Shi Y L and Zhu X X . 2022b . Universal domain adaptation without source data for remote sensing image scene classification // Proceedings of 2022 IEEE International Geoscience and Remote Sensing Symposium . Kuala Lumpur， Malaysia ： IEEE： 5341 - 5344 ［ DOI： 10.1109/IGARSS46834.2022.9884889 http://dx.doi.org/10.1109/IGARSS46834.2022.9884889 ］
Xu Q S ， Shi Y L ， Yuan X and Zhu X X . 2023 . Universal domain adaptation for remote sensing image scene classification . IEEE Transactions on Geoscience and Remote Sensing ， 61 ： # 4700515 ［ DOI： 10.1109/TGRS.2023.3235988 http://dx.doi.org/10.1109/TGRS.2023.3235988 ］
Yang Y and Newsam S . 2010 . Bag-of-visual-words and spatial extensions for land-use classification // Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems . San Jose， USA ： Association for Computing Machinery： 270 - 279 ［ DOI： 10.1145/1869790.1869829 http://dx.doi.org/10.1145/1869790.1869829 ］
You K C ， Long M S ， Cao Z J ， Wang J M and Jordan M I . 2019 . Universal domain adaptation // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach ： IEEE： #283 ［ DOI： 10.1109/cvpr.2019.00283 http://dx.doi.org/10.1109/cvpr.2019.00283 ］
Zhang Y L ， Yuan Y ， Feng Y C and Lu X Q . 2019 . Hierarchical and robust convolutional neural network for very high-resolution remote sensing object detection . IEEE Transactions on Geoscience and Remote Sensing ， 57 （ 8 ）： 5535 - 5548 ［ DOI： 10.1109/TGRS.2019.2900302 http://dx.doi.org/10.1109/TGRS.2019.2900302 ］
Zhao X Y ， Hu J L ， Mou L C ， Xiong Z T and Zhu X X . 2023 . Cross-city Landuse classification of remote sensing images via deep transfer learning . International Journal of Applied Earth Observation and Geoinformation ， 122 ： # 103358 ［ DOI： 10.1016/j.jag.2023.103358 http://dx.doi.org/10.1016/j.jag.2023.103358 ］
Zheng J P ， Wu W Z ， Fu H H ， Li W J ， Dong R M ， Zhang L X and Yuan S . 2020 . Unsupervised mixed multi-target domain adaptation for remote sensing images classification // Proceedings of 2020 IEEE International Geoscience and Remote Sensing Symposium . Waikoloa， USA ： IEEE： 1381 - 1384 ［ DOI： 10.1109/IGARSS39084.2020.9323602 http://dx.doi.org/10.1109/IGARSS39084.2020.9323602 ］
Zheng J P ， Wu W Z ， Yuan S ， Zhao Y ， Li W J ， Zhang L X ， Dong R M and Fu H H . 2022a . A two-stage adaptation network （TSAN） for remote sensing scene classification in single-source-mixed-multiple-target domain adaptation （S²M²T DA） scenarios . IEEE Transactions on Geoscience and Remote Sensing ， 60 ： # 5609213 ［ DOI： 10.1109/TGRS.2021.3105302 http://dx.doi.org/10.1109/TGRS.2021.3105302 ］
Zheng J P ， Zhao Y ， Wu W Z ， Chen M X ， Li W J and Fu H H . 2023a . Partial domain adaptation for scene classification from remote sensing imagery . IEEE Transactions on Geoscience and Remote Sensing ， 61 ： # 5601317 ［ DOI： 10.1109/TGRS.2022.3229039 http://dx.doi.org/10.1109/TGRS.2022.3229039 ］
Zheng X T ， Cui H W ， Xu C J and Lu X Q . 2023b . Dual teacher： a semisupervised cotraining framework for cross-domain ship detection . IEEE Transactions on Geoscience and Remote Sensing ， 61 ： # 5613312 ［ DOI： 10.1109/TGRS.2023.3287863 http://dx.doi.org/10.1109/TGRS.2023.3287863 ］
Zheng Z D ， Zhong Y F ， Su Y and Ma A L . 2022b . Domain adaptation via a task-specific classifier framework for remote sensing cross-scene classification . IEEE Transactions on Geoscience and Remote Sensing ， 60 ： # 5620513 ［ DOI： 10.1109/TGRS.2022.3151689 http://dx.doi.org/10.1109/TGRS.2022.3151689 ］
Zhou W X ， Newsam S ， Li C M and Shao Z F . 2018 . PatternNet： a benchmark dataset for performance evaluation of remote sensing image retrieval . ISPRS Journal of Photogrammetry and Remote Sensing ， 145 ： 197 - 209 ［ DOI： 10.1016/j.isprsjprs.2018.01.004 http://dx.doi.org/10.1016/j.isprsjprs.2018.01.004 ］
Zhu S H ， Du B ， Zhang L P and Li X . 2022 . Attention-based multiscale residual adaptation network for cross-scene classification . IEEE Transactions on Geoscience and Remote Sensing ， 60 ： # 5400715 ［ DOI： 10.1109/TGRS.2021.3056624 http://dx.doi.org/10.1109/TGRS.2021.3056624 ］
Zhu S H ， Wu C ， Du B and Zhang L P . 2023 . Adversarial divergence training for universal cross-scene classification . IEEE Transactions on Geoscience and Remote Sensing ， 61 ： # 5609712 ［ DOI： 10.1109/TGRS.2023.3274781 http://dx.doi.org/10.1109/TGRS.2023.3274781 ］
Zou Q ， Ni L H ， Zhang T and Wang Q . 2015 . Deep learning based feature selection for remote sensing scene classification . IEEE Geoscience and Remote Sensing Letters ， 12 （ 11 ）： 2321 - 2325 ［ DOI： 10.1109/LGRS.2015.2475299 http://dx.doi.org/10.1109/LGRS.2015.2475299 ］
相关作者
相关机构
京公网安备11010802024621
