浏览全部资源
扫码关注微信
Published： 16 October 2023 ，
移动端阅览
王浩宇， 程玉虎， 王雪松. 2023. 关联子域对齐网络的跨域高光谱图像分类. 中国图象图形学报， 28(10):3255-3266
Wang Haoyu， Cheng Yuhu， Wang Xuesong. 2023. Correlation subdomain alignment network based cross-domain hyperspectral image classification method. Journal of Image and Graphics， 28(10):3255-3266
目的
2
近年来，深度网络成功应用于高光谱图像分类。然而，难以获取充足的标记数据大大限制了深度网络的充分训练，进而导致网络对高光谱图像的分类能力下降。为解决以上困难，提出一种关联子域对齐网络的高光谱图像迁移分类方法。
方法
2
基于深度迁移学习方法，通过对两域分布进行多角度、全面领域适应的同时将两域分类器进行差异适配。一方面，利用关联对齐从整体上对齐了两域的二阶统计量信息，适配了两域的全局分布；另一方面，利用局部最大均值差异对齐了相关子域的一阶统计量信息，适配了两域的局部分布。另外，构造一种分类器适配模块并将其加入所提网络中，通过对两域分类器差异进行适配，进一步增强网络的领域适应效果。
结果
2
从4组真实高光谱数据集上的实验结果可看出：在分别采集于不同区域的高光谱图像数据对上，所提方法的精度比排名第2的分类方法高出1.01%、0.42%、0.73%和0.64%。本文方法的Kappa系数也取得最优结果。
结论
2
与现有主流算法相比较，所提网络能够在整体和局部、一阶和二阶统计量上分别对两域进行有效对齐，进而充分利用在源域上训练好的分类器完成对目标域高光谱数据的跨域分类。
Objective
2
Hyperspectral image （HSI） classification method is focused on spectral and spatial information to recognize the category of each ground object pixel. Due to the large number of spectral bands and redundant information between spectral bands in hyperspectral data， it is challenged to extract the identifiable features. Deep learning technique has been widely used in HSI classification because of its potential feature extraction and generalization abilities. However， the classification performance of it often based on much more labeled training samples. Due to the relative high sensor and labor costs generated from collection to calibration of HSI， deep learning techniques are challenged for sufficient labeled his normally. It is feasible to classify a HSI （target domain） using another similar but not identical HSI （source domain） with rich labeling information to some extent. To complete the target-domain HSI classification， transfer learning technique can be used to transfer the domain-invariant knowledge learned from the labeled source domain to the target domain with a similar but different distribution. However， due to the lighting and sensor-derived constraints， the collected cross-domain HSIs have a large distribution difference frequently， and their distributions are challenged to be fully adapted. In addition， most transfer learning methods are rarely melted into in related to two-domain classifiers-between mismatched problems. Therefore， we develop a simple and effective deep transfer HSI classification method， called correlation subdomain alignment network （CSADN）， which can be focused on distribution and classifier adaptations and the labeled source-domain knowledge can be transferred to an unlabeled target domain. The proposed method can use the labeled source-domain samples to complete the target domain HSI classification further.
Method
2
The CSADN is mainly composed of four aspects as mentioned below： 1） Data preprocessing： band selection is used to lower the dimension of the original HSIs. 2） Depth network-based feature learning： deep neural network is developed and used for feature learning. 3） Feature distribution adaptation： a covariance adaptation term is added to the loss function through minimizing the two-domain covariance to complete the global distribution adaptation. 4） Subdomain adaptation： a subdomain adaptation term is added to the loss function further， and the local features are aligned via minimizing the subdomain difference. 5） Classifier adaptation： the classifier difference is captured based on the classifier adaptation module， and the low-density separation criterion is utilized to yield the source-domain classifier to adapt the target-domain data better. To improve the classification accuracy of CSADN for the target-domain data further， joint domain adaptation is carried out between the feature level and classifier level.
Result
2
The classification performance and domain adaptation ability of CSADN are evaluated in experiments. For classification， ten sorts of cross-domain classification methods are selected for comparative experiments in the context of traditional transfer learning methods and deep transfer learning methods. For CSADN validation， four sets of real HSI data pairs are selected， including Botswana5-6 （BOT5-6）， BOT6-7， BOT7-5， and Houston bright-shallow. It demonstrates that each of the accuracy of CSADN is optimized by 1.01%， 0.42%， 0.73%， and 0.64% for BOT5-6， BOT6-7， BOT7-5， and Houston bright-shallow data pairs. The Kappa coefficient of CSADN has its potentials apparently. For domain adaptation ability， the t-distributed stochastic neighbor embedding can be used to verify the effectiveness of CSADN via high-dimensional data visualization as well. The original features of HSIs and the domain adaptation after features of CSADN are visualized on four experimental data sets. Compared with the original features of HSIs， the CSADN-extracted features have their higher inter-class difference and lower intra-class difference simultaneously. Moreover， the feature distributions-between covariance differences can be reduced via the domain adaptation. It illustrates that the proposed method has its potential domain adaptation ability to a certain extent.
Conclusion
2
The proposed CSADN is feasible to integrate the feature distribution adaptation and classifier adaptation into transfer knowledge from the source domain to the target domain. It can classify unlabeled samples in the target domain using labeled samples in the source domain only. Specifically， a domain adaptation layer is designed and embedded into as well. To complete the feature distribution adaptation， the difference between the first-order and second-order statistics of both domains is adapted through aligning the relevant subdomains and covariance. The classifier adaptation module is constructed and added to CSADN. The domain adaptation ability can be enhanced in terms of adapting the difference between classifiers， and the classification accuracy of the target domain data can be improved further. The target-domain pseudo label is used in CSADN， and the quality of the pseudo label can affect the domain adaptation effect of CSADN. To get better performance in cross-domain HSI classification tasks， future research direction is predicted that the pseudo label optimization technology can be introduced into CSADN further.
高光谱图像（HSI）分类迁移学习深度学习跨域
hyperspectral image （HSI）classificationtransfer learningdeep learningcross-domain
Chen Q， Wang W， Wu F Y， De S， Wang R L， Zhang B L and Huang X. 2019. A survey on an emerging area： deep learning for smart city data. IEEE Transactions on Emerging Topics in Computational Intelligence， 3（5）： 392-410 ［DOI： 10.1109/TETCI.2019.2907718http://dx.doi.org/10.1109/TETCI.2019.2907718］
Chen Y S， Zhao X and Jia X P. 2015. Spectral-spatial classification of hyperspectral data based on deep belief network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing， 8（6）： 2381-2392 ［DOI： 10.1109/JSTARS.2015.2388577http://dx.doi.org/10.1109/JSTARS.2015.2388577］
Delalieux S， Somers B， Haest B， Spanhove T， Vanden Borre J and Mücher C A. 2012. Heathland conservation status mapping through integration of hyperspectral mixture analysis and decision tree classifiers. Remote Sensing of Environment， 126： 222-231 ［DOI： 10.1016/j.rse.2012.08.029http://dx.doi.org/10.1016/j.rse.2012.08.029］
Deng B， Jia S and Shi D M. 2020. Deep metric learning-based feature embedding for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing， 58（2）： 1422-1435 ［DOI： 10.1109/TGRS.2019.2946318http://dx.doi.org/10.1109/TGRS.2019.2946318］
Ganin Y and Lempotsky V. 2015. Unsupervised domain adaptation by backpropagation//Proceedings of the 32nd International Conference on Machine Learning. Lille， France： JMLR.org： 1180-1189
Gong B Q， Shi Y， Sha F and Grauman K. 2012. Geodesic flow kernel for unsupervised domain adaptation//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Rhode Island， USA： IEEE： 2066-2073 ［DOI： 10.1109/CVPR.2012.6247911http://dx.doi.org/10.1109/CVPR.2012.6247911］
Hong D F， Gao L R， Yao J， Zhang B， Plaza A and Chanussot J. 2021. Graph convolutional networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing， 59（7）： 5966-5978 ［DOI： 10.1109/TGRS.2020.3015157http://dx.doi.org/10.1109/TGRS.2020.3015157］
Li W， Wei W， Zhang L， Wang C and Zhang Y N. 2019. Unsupervised deep domain adaptation for hyperspectral image classification//Proceedings of 2019 IEEE International Geoscience and Remote Sensing Symposium. Yokohama， Japan： IEEE： 1-4 ［DOI： 10.1109/IGARSS40859.2019.8976372http://dx.doi.org/10.1109/IGARSS40859.2019.8976372］
Li W， Wu G D， Zhang F and Du Q. 2017. Hyperspectral image classification using deep pixel-pair features. IEEE Transactions on Geoscience and Remote Sensing， 55（2）： 844-853 ［DOI： 10.1109/TGRS.2016.2616355http://dx.doi.org/10.1109/TGRS.2016.2616355］
Liu Y， Gao L R， Xiao C C， Qu Y， Zheng K and Marinoni A. 2020. Hyperspectral image classification based on a shuffled group convolutional neural network with transfer learning. Remote Sensing， 12（11）： #1780 ［DOI： 10.3390/rs12111780http://dx.doi.org/10.3390/rs12111780］
Liu Z X， Ma L and Du Q. 2021. Class-wise distribution adaptation for unsupervised classification of hyperspectral remote sensing images. IEEE Transactions on Geoscience and Remote Sensing， 59（1）： 508-521 ［DOI： 10.1109/TGRS.2020.2997863http://dx.doi.org/10.1109/TGRS.2020.2997863］
Long M S， Cao Y， Cao Z J， Wang J M and Jordan M I. 2019. Transferable representation learning with deep adaptation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence， 41（12）： 3071-3085 ［DOI： 10.1109/TPAMI.2018.2868685http://dx.doi.org/10.1109/TPAMI.2018.2868685］
Long M S， Wang J M， Ding G G， Sun J G and Yu P S. 2013. Transfer feature learning with joint distribution adaptation//Proceedings of 2013 IEEE International Conference on Computer Vision. Sydney， Australia： IEEE： 2200-2207 ［DOI： 10.1109/ICCV.2013.274http://dx.doi.org/10.1109/ICCV.2013.274］
Long M S， Zhu H， Wang J M and Jordan M I. 2016. Unsupervised domain adaptation with residual transfer networks//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona， Spain： Curran Associates Inc.： 136-144
Ma X R， Mou X R， Wang J， Liu X K， Geng J and Wang H Y. 2021. Cross-dataset hyperspectral image classification based on adversarial domain adaptation. IEEE Transactions on Geoscience and Remote Sensing， 59（5）： 4179-4190 ［DOI： 10.1109/TGRS.2020.3015357http://dx.doi.org/10.1109/TGRS.2020.3015357］
Ma X R， Mou X R， Wang J， Liu X K， Wang H Y and Yin B C. 2019. Cross-data set hyperspectral image classification based on deep domain adaptation. IEEE Transactions on Geoscience and Remote Sensing， 57（12）： 10164-10174 ［DOI： 10.1109/TGRS.2019.2931730http://dx.doi.org/10.1109/TGRS.2019.2931730］
Mei S H， Zhang B W， Ma M Y and Jia S. 2021. Predicting near-infrared hyperspectral images from visible hyperspectral images. Journal of Image and Graphics， 26（8）： 1786-1795
梅少辉， 张博威， 马明阳， 贾森. 2021. 近红外高光谱图像数据预测技术. 中国图象图形学报， 26（8）： 1786-1795 ［DOI： 10.11834/jig.210184http://dx.doi.org/10.11834/jig.210184］
Melgani F and Bruzzone L. 2004. Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience and Remote Sensing， 42（8）： 1778-1790 ［DOI： 10.1109/TGRS.2004.831865http://dx.doi.org/10.1109/TGRS.2004.831865］
Sun B C and Saenko K. 2016. Deep CORAL： correlation alignment for deep domain adaptation//Proceedings of 2016 European Conference on Computer Vision. Amsterdam， the Netherlands： Springer： 443-450 ［DOI： 10.1007/978-3-319-49409-8_35http://dx.doi.org/10.1007/978-3-319-49409-8_35］
Sun T， Lu C and Ling H B. 2022. Prior knowledge guided unsupervised domain adaptation//Proceedings of the 17th European Conference on Computer Vision. Tel Aviv， Israel： IEEE： 639-655 ［DOI： 10.1007/978-3-031-19827-4_37http://dx.doi.org/10.1007/978-3-031-19827-4_37］
Sun Y B， Li J， Wang W， Plaza A and Chen Z Q. 2016. Active learning based autoencoder for hyperspectral imagery classification//Proceedings of 2016 IEEE International Geoscience and Remote Sensing Symposium. Beijing， China： IEEE： 469-472 ［DOI： 10.1109/IGARSS.2016.7729116http://dx.doi.org/10.1109/IGARSS.2016.7729116］
Tuia D， Persello C and Bruzzone L. 2016. Domain adaptation for the classification of remote sensing data： an overview of recent advances. IEEE Geoscience and Remote Sensing Magazine， 4（2）： 41-57 ［DOI： 10.1109/MGRS.2016.2548504http://dx.doi.org/10.1109/MGRS.2016.2548504］
Wang C， Zhang L， Wei W and Zhang Y N. 2020. Hyperspectral image classification with data augmentation and classifier fusion. IEEE Geoscience and Remote Sensing Letters， 17（8）： 1420-1424 ［DOI： 10.1109/LGRS.2019.2945848http://dx.doi.org/10.1109/LGRS.2019.2945848］
Wang F， Guo Q and Ge X Q. 2021. Pan-sharpening by deep recursive residual network. Journal of Remote Sensing， 25（6）： 1244-1256
王芬， 郭擎， 葛小青. 2021. 深度递归残差网络的遥感图像空谱融合. 遥感学报， 25（6）： 1244-1256 ［DOI： 10.11834/jrs.20219250http://dx.doi.org/10.11834/jrs.20219250］
Wang H Y， Cheng Y H， Chen C L P and Wang X S. 2022a. Hyperspectral image classification based on domain adversarial broad adaptation network. IEEE Transactions on Geoscience and Remote Sensing， 60： #5517813 ［DOI： 10.1109/TGRS.2021.3128162http://dx.doi.org/10.1109/TGRS.2021.3128162］
Wang Q， Lin J Z and Yuan Y. 2016. Salient band selection for hyperspectral image classification via manifold ranking. IEEE Transactions on Neural Networks and Learning Systems， 27（6）： 1279-1289 ［DOI： 10.1109/TNNLS.2015.2477537http://dx.doi.org/10.1109/TNNLS.2015.2477537］
Wang R， Wu Z X， Weng Z J， Chen J J， Qi G J and Jiang Y G. 2022b. Cross-domain contrastive learning for unsupervised domain adaptation. IEEE Transactions on Multimedia ［DOI： 10.1109/TMM.2022.3146744http://dx.doi.org/10.1109/TMM.2022.3146744］
Wang Z M， Du B， Shi Q and Tu W P. 2019. Domain adaptation with discriminative distribution and manifold embedding for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters， 16（7）： 1155-1159 ［DOI： 10.1109/LGRS.2018.2889967http://dx.doi.org/10.1109/LGRS.2018.2889967］
Ye Z， Bai L and He M Y. 2021. Review of spatial-spectral feature extraction for hyperspectral image. Journal of Image and Graphics， 26（8）： 1737-1763
叶珍， 白璘， 何明一. 2021. 高光谱图像空谱特征提取综述. 中国图象图形学报， 26（8）： 1737-1763 ［DOI： 10.11834/jig.210198http://dx.doi.org/10.11834/jig.210198］
Zhang H K， Li Y， Zhang Y Z and Shen Q. 2017. Spectral-spatial classification of hyperspectral imagery using a dual-channel convolutional neural network. Remote Sensing Letters， 8（5）： 438-447 ［DOI： 10.1080/2150704X.2017.1280200http://dx.doi.org/10.1080/2150704X.2017.1280200］
Zhang J， Wang Y B and Fang S. 2020. Multi-label hyperspectral image classification. Journal of Image and Graphics， 25（3）： 568-578
张晶， 王亦斌， 方帅. 2020. 多标签高光谱图像地物分类. 中国图象图形学报， 25（3）： 568-578 ［DOI： 10.11834/jig.190285http://dx.doi.org/10.11834/jig.190285］
Zhang M M， Li W and Du Q. 2018. Diverse region-based CNN for hyperspectral image classification. IEEE Transactions on Image Processing， 27（6）： 2623-2634 ［DOI： 10.1109/TIP.2018.2809606http://dx.doi.org/10.1109/TIP.2018.2809606］
Zhou X and Prasad S. 2017. Domain adaptation for robust classification of disparate hyperspectral images. IEEE Transactions on Computational Imaging， 3（4）： 822-836 ［DOI： 10.1109/TCI.2017.2752150http://dx.doi.org/10.1109/TCI.2017.2752150］
Zhu Y C， Zhuang F Z， Wang J D， Ke G L， Chen J W， Bian J， Xiong H and He Q. 2021. Deep subdomain adaptation network for image classification. IEEE Transactions on Neural Networks and Learning Systems， 32（4）： 1713-1722 ［DOI： 10.1109/TNNLS.2020.2988928http://dx.doi.org/10.1109/TNNLS.2020.2988928］
相关文章
相关作者
相关机构
京公网安备11010802024621