浏览全部资源
扫码关注微信
Published： 16 August 2024 ，
移动端阅览
贾萌， 赵秦， 鲁晓锋. 2024. 双边对抗自编码网络的无监督异质遥感图像变化检测. 中国图象图形学报， 29(08):2188-2204
Jia Meng， Zhao Qin， Lu Xiaofeng. 2024. Bipartite adversarial autoencoder network for unsupervised detection of changes in heterogeneous remote sensing images. Journal of Image and Graphics， 29(08):2188-2204
目的
2
异质遥感图像由不同类型的传感器所获取，在数据结构、分辨率及辐射特性上均存在巨大差异。变化检测任务旨在通过分析在不同时间获取的同一目标区域的图像来检测地表覆盖物变化，然而异质遥感图像的数据异构特性会使得变化检测过程更加困难。针对这个问题，提出了一种嵌入聚类分析的双边对抗自编码网络来实现异质遥感图像地物变化的精确检测。
方法
2
构造双边对抗自编码网络对异质遥感图像进行重构和风格转换，通过结构一致性损失和对抗损失对网络训练进行约束，迫使网络将异质图像转换到公共数据域。考虑到变化区域像素对于对抗损失函数在网络优化中的不利影响，对映射到公共数据域的两对同质图像进行聚类分析，基于此提出一种新的语义信息约束的对抗损失函数，迫使网络生成具有更加一致风格的图像。
结果
2
在4组典型的异质遥感图像数据集上对提出的变化检测网络性能进行测试，在Italy数据集、California数据集、Tianhe数据集以及Shuguang数据集上的总体检测精度分别达到0.970 5、0.938 2、0.994 7以及0.982 6。与现有的传统以及深度学习方法对比，提出算法在视觉及定量分析结果上均取得了较好的检测性能。
结论
2
针对异质遥感图像变化检测所要面临的由环境、数据异构等因素造成的检测困难、错检率高的问题，提出的基于双边对抗自编码网络的无监督异质遥感图像变化检测方法，既可以实现变化检测过程完全无监督，又充分利用网络特性和语义信息，提高了变化检测性能。
Objective
2
Heterogeneous remote sensing images from different sensors are quite different in imaging mechanism， radiation characteristics， and geometric characteristics. Thus， they reflect the physical properties of the ground target at different levels. Therefore， no relationship exists between the observed values of the same object， which usually leads to “pseudo changes”. As a result， the change detection task has more difficulty obtaining accurate change information of the observed ground objects. Efforts have been made toward unsupervised detection of changes in heterogeneous remote sensing images by designing various methods to obtain change information. However， traditional image difference operators based on the difference or ratio of radiation measurement is no longer applicable. Therefore， transferring the bitemporal images in a common space is a convenient way to calculate differences. Considering the excellent and flexible feature learning capability of deep neural networks， they have been widely applied in change detection tasks for heterogeneous images to effectively alleviate the influence of “pseudo changes”. Moreover， by fully utilizing the characteristics of deep neural networks， they can be designed to transform heterogeneous remote sensing images into the same feature domain. Then， the change information can be accurately represented. Inspired by the paradigm of image translation， heterogeneous images are transformed into a common domain with consistent feature representations to enable direct comparisons of data. The key point of this task is learning a suitable one-to-one mapping to build a relationship between distinct appearances of images and exclude the effect of interference factors. Therefore， this study proposes a bipartite adversarial autoencoder network with clustering （BAACL） to detect changes between heterogeneous remote sensing images.
Method
2
A bipartite adversarial autoencoder network is constructed to reconstruct the bitemporal images and achieve the transformation of heterogeneous images to common domain. Appropriate reconstruction loss regularization should be applied to obtain good data mapping from the constructed neural network optimization. Next， the bipartite convolutional autoencoders can be jointly trained to encode the input images and reconstruct them with high fidelity in output. Meanwhile， for the image-to-image translation task， additional structural consistency loss and adversarial loss terms are designed to constrain the network training process for converting heterogeneous images to the common data domain. The structural consistency loss term is designed to describe the internal structural relationships of images before and after translation. For this purpose， affinity matrixes are adopted to express the structural self-similarity within an image. The heterogeneous images can be conveniently compared in a new affinity space. Notably， based on the paradigm of image translation， our model can be viewed as learning two “transformation functions”， which are adopted to transform each of the heterogeneous images to the opposite data space. Furthermore， such a setup can regarded as a special case of an “adversarial mechanism”， which is formulated by an adversarial loss to train autoencoders for matching the opposite image style. Considering the disadvantage of these changed pixels to the adversarial loss term in network optimization， the pseudo-difference image generated by two pairs of homogeneous images mapped in the common data domain is analyzed by clustering in an unsupervised manner. Thereafter， the obtained semantic information is adopted to further constrain the adversarial loss term. The overall performance of the BAACL network is illustrated on four sets of publicly available datasets of heterogeneous remote sensing images. Five popular traditional and deep learning-based detection methods for changes in heterogeneous remote sensing images are selected and compared with this method to verify its effectiveness.
Result
2
Results obtained from the Italy， California， Tianhe， and Shuguang datasets can also illustrate the performance of the proposed BAACL， with the overall detection accuracies of up to 0.970 5， 0.938 2， 0.994 7， and 0.982 6， respectively. Meanwhile， the proposed method is superior to the five compared methods in terms of visual results of the final change map. A set of ablation experiments is designed to verify the influence of the improved adversarial loss term on network optimization performance. the performance of the BAACL network does not change dramatically with various proportions of sample changing regions due to the semantic regularization. It verifies the effectiveness of the proposed semantic information-based adversarial loss term for network optimization.
Conclusion
2
The semantic information-based adversarial loss term is designed to narrow the distance of the unchanged regions of the bitemporal images for image style consistency. The reason is that， for adversarial loss term without semantic regularization， a larger proportion of the change region of the current training sample will result in a worse trained network effect. Therefore， an accurate definition of the network constraint term will result in good network optimization and further affect the change detection performance. In view of the difficulty and high false alarm rate of detection methods for changes in heterogeneous remote sensing images caused by factors such as seasons and data heterogeneity， the proposed bipartite adversarial autoencoder network for the detection of changes in heterogeneous remote sensing images not only can fully utilize network characteristics and semantic information to improve image style consistency but also can realize completely unsupervised change detection process. The change detection performance can be greatly improved as well.
域转换聚类分析语义信息无监督变化检测异质遥感图像
domain transformationcluster analysissemantic informationunsupervised change detectionheterogeneous remote sensing images
Bovolo F and Bruzzone L. 2005. A detail-preserving scale-driven approach to change detection in multitemporal SAR images. IEEE Transactions on Geoscience and Remote Sensing， 43（12）： 2963-2972 ［DOI： 10.1109/TGRS.2005.857987http://dx.doi.org/10.1109/TGRS.2005.857987］
Brennan R L and Prediger D J. 1981. Coefficient kappa： some uses， misuses， and alternatives. Educational and Psychological Measurement， 41（3）： 687-699 ［DOI： 10.1177/001316448104100307http://dx.doi.org/10.1177/001316448104100307］
Brunner D， Lemoine G and Bruzzone L. 2010. Earthquake damage assessment of buildings using VHR optical and SAR imagery. IEEE Transactions on Geoscience and Remote Sensing， 48（5）： 2403-2420 ［DOI： 10.1109/TGRS.2009.2038274http://dx.doi.org/10.1109/TGRS.2009.2038274］
Fan W， Zhou M and Huang R. 2020. Multiscale deep features fusion for change detection. Journal of Image and Graphics， 25（4）： 669-678
樊玮， 周末， 黄睿. 2020. 多尺度深度特征融合的变化检测. 中国图象图形学报， 25（4）： 669-678 ［DOI： 10.11834/jig.190312http://dx.doi.org/10.11834/jig.190312］
Gong M G， Su L Z， Li H and Liu J. 2016. A survey on change detection in synthetic aperture radar imagery. Journal of Computer Research and Development， 53（1）： 123-137
公茂果， 苏临之， 李豪， 刘嘉. 2016. 合成孔径雷达影像变化检测研究进展. 计算机研究与发展， 53（1）： 123-137 ［DOI： 10.7544/issn1000-1239.2016.20150662http://dx.doi.org/10.7544/issn1000-1239.2016.20150662］
Griffiths P， Hostert P， Gruebner O and van der Linden S. 2010. Mapping megacity growth with multi-sensor data. Remote sensing of Environment， 114（2）： 426-439 ［DOI： /j.rse.2009.09.012http://dx.doi.org//j.rse.2009.09.012］
Inglada J and Mercier G. 2007. A new statistical similarity measure for change detection in multitemporal SAR images and its extension to multiscale change analysis. IEEE Transactions on Geoscience and Remote Sensing， 45（5）： 1432-1445 ［DOI： 10.1109/TGRS.2007.893568http://dx.doi.org/10.1109/TGRS.2007.893568］
Isola P， Zhu J Y， Zhou T H and Efros A A. 2017. Image-to-image translation with conditional adversarial networks//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Honolulu， USA： 5967-5976 ［DOI： 10.1109/CVPR.2017.632http://dx.doi.org/10.1109/CVPR.2017.632］
Jia M， Zhang C， Lyu Z Y， Zhao Z Q and Wang L. 2022. Bipartite adversarial autoencoders with structural self-similarity for unsupervised heterogeneous remote sensing image change detection. IEEE Geoscience and Remote Sensing Letters， 19： #6515705 ［DOI： 10.1109/LGRS.2022.3201925http://dx.doi.org/10.1109/LGRS.2022.3201925］
Khan S H， He X M， Porikli F and Bennamoun M. 2017. Forest change detection in incomplete satellite images with deep neural networks. IEEE Transactions on Geoscience and Remote Sensing， 55（9）： 5407-5423 ［DOI： 10.1109/TGRS.2017.2707528http://dx.doi.org/10.1109/TGRS.2017.2707528］
LeCun Y A， Bottou L， Orr G B and Müller K R. 2012. Efficient BackProp//Montavon G， Orr G B and Müller K R， eds. Neural Networks： Tricks of the Trade. 2nd ed. Berlin， Heidelberg： Springer： 9-48 ［DOI： 10.1007/978-3-642-35289-8_3http://dx.doi.org/10.1007/978-3-642-35289-8_3］
Li L， Wang C， Zhang H， Zhang B and Wu F. 2019. Urban building change detection in SAR images using combined differential image and residual U-Net network. Remote Sensing， 11（9）： #1091 ［DOI： 10.3390/rs11091091http://dx.doi.org/10.3390/rs11091091］
Li X H， Du Z S， Huang Y Y and Tan Z Y. 2021. A deep translation （GAN） based change detection network for optical and SAR remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing， 179： 14-34 ［DOI： 10.1016/j.isprsjprs.2021.07.007http://dx.doi.org/10.1016/j.isprsjprs.2021.07.007］
Liu J， Gong M G， Qin K and Zhang P Z. 2018a. A deep convolutional coupling network for change detection based on heterogeneous optical and radar images. IEEE Transactions on Neural Networks and Learning Systems， 29（3）： 545-559 ［DOI： 10.1109/TNNLS.2016.2636227http://dx.doi.org/10.1109/TNNLS.2016.2636227］
Liu J， Zhang W H， Liu F and Xiao L. 2022. A probabilistic model based on bipartite convolutional neural network for unsupervised change detection. IEEE Transactions on Geoscience and Remote Sensing， 60： #4701514 ［DOI： 10.1109/TGRS.2021.3071347http://dx.doi.org/10.1109/TGRS.2021.3071347］
Liu Z G， Li G， Mercier G， He Y and Pan Q. 2018b. Change detection in heterogenous remote sensing images via homogeneous pixel transformation. IEEE Transactions on Image Processing， 27（4）： 1822-1834 ［DOI： 10.1109/TIP.2017.2784560http://dx.doi.org/10.1109/TIP.2017.2784560］
Luppino L T， Hansen M A， Kampffmeyer M， Bianchi F M， Moser G， Jenssen R and Anfinsen S N. 2024. Code-aligned autoencoders for unsupervised change detection in multimodal remote sensing images. IEEE Transactions on Neural Networks and Learning Systems， 35（1）： 60-72 ［DOI： 10.1109/TNNLS.2022.3172183http://dx.doi.org/10.1109/TNNLS.2022.3172183］
Luppino L T， Kampffmeyer M， Bianchi F M， Moser G， Serpico S B， Jenssen R and Anfinsen S N. 2022. Deep image translation with an affinity-based change prior for unsupervised multimodal change detection. IEEE Transactions on Geoscience and Remote Sensing， 60： #4700422 ［DOI： 10.1109/TGRS.2021.3056196http://dx.doi.org/10.1109/TGRS.2021.3056196］
Lyu Z Y， Huang H T， Li X H， Zhao M H， Benediktsson J A， Sun W W and Falco N. 2022. Land cover change detection with heterogeneous remote sensing images： review， progress， and perspective. Proceedings of the IEEE， 110（12）： 1976-1991 ［DOI： 10.1109/JPROC.2022.3219376http://dx.doi.org/10.1109/JPROC.2022.3219376］
Mercier G， Moser G and Serpico S B. 2008. Conditional copulas for change detection in heterogeneous remote sensing images. IEEE Transactions on Geoscience and Remote Sensing， 46（5）： 1428-1441 ［DOI： 10.1109/TGRS.2008.916476http://dx.doi.org/10.1109/TGRS.2008.916476］
Muchoney D M and Haack B N. 1994. Change detection for monitoring forest defoliation. Photogrammetric Engineering and Remote Sensing， 60（10）： 1243-1251
Niu X D， Gong M G， Zhan T and Yang Y L. 2019. A conditional adversarial network for change detection in heterogeneous images. IEEE Geoscience and Remote Sensing Letters， 16（1）： 45-49 ［DOI： 10.1109/LGRS.2018.2868704http://dx.doi.org/10.1109/LGRS.2018.2868704］
Pantze A， Fransson J E S and Santoro M. 2010. Forest change detection from L-band satellite SAR images using iterative histogram matching and thresholding together with data fusion//IEEE International Geoscience and Remote Sensing Symposium. Honolulu， USA： IEEE： 1226-1229 ［DOI： 10.1109/IGARSS.2010.5650677http://dx.doi.org/10.1109/IGARSS.2010.5650677］
Radke R J， Andra S， Al-Kofahi O and Roysam B. 2005. Image change detection algorithms： a systematic survey. IEEE Transactions on Image Processing， 14（3）： 294-307 ［DOI： 10.1109/TIP.2004.838698http://dx.doi.org/10.1109/TIP.2004.838698］
Rignot E J M and van Zyl J J. 1993. Change detection techniques for ERS-1 SAR data. IEEE Transactions on Geoscience and Remote Sensing， 31（4）： 896-906 ［DOI： 10.1109/36.239913http://dx.doi.org/10.1109/36.239913］
Royer A， Bousmalis K， Gouws S， Bertsch F， Mosseri I， Cole F and Murphy K. 2020. XGAN： unsupervised image-to-image translation for many-to-many mappings//Singh R， Vatsa M， Patel V M and Ratha N， eds. Domain Adaptation for Visual Understanding. Switzerland： Springer： 33-49 ［DOI： 10.1007/978-3-030-30671-7_3http://dx.doi.org/10.1007/978-3-030-30671-7_3］
Sun Y L， Lei L， Li X， Sun H and Kuang G Y. 2021. Nonlocal patch similarity based heterogeneous remote sensing change detection. Pattern Recognition， 109： #107598 ［DOI： /j.patcog.2020.107598http://dx.doi.org//j.patcog.2020.107598］
Sun Y L， Lei L， Li X， Tan X and Kuang G Y. 2022. Structure consistency-based graph for unsupervised change detection with homogeneous and heterogeneous remote sensing images. IEEE Transactions on Geoscience and Remote Sensing， 60： #4700221 ［DOI： 10.1109/TGRS.2021.3053571http://dx.doi.org/10.1109/TGRS.2021.3053571］
Taigman Y， Polyak A and Wolf L. 2016. Unsupervised cross-domain image generation. ［EB/OL］. ［2023-07-20］. http://arxiv.org/pdf/1611.02200.pdfhttp://arxiv.org/pdf/1611.02200.pdf
Wang X L， Girshick R， Gupta A and He K M. 2018. Non-local neural networks//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City， USA： IEEE： 7794-7803 ［DOI： 10.1109/CVPR.2018.00813http://dx.doi.org/10.1109/CVPR.2018.00813］
Wang Y H， Gao L R， Chen Z C and Zhang B. 2020. Deep learning and superpixel-based method for high-resolution remote sensing image change detection. Journal of Image and Graphics， 25（6）： 1271-1282
王艳恒， 高连如， 陈正超， 张兵. 2020. 结合深度学习和超像元的高分遥感影像变化检测. 中国图象图形学报， 25（6）： 1271-1282 ［DOI： 10.11834/jig.190319http://dx.doi.org/10.11834/jig.190319］
Yu H X， Wu A and Zheng W S. 2020. Unsupervised person re-identification by deep asymmetric metric embedding. IEEE Transactions on Pattern Analysis and Machine Intelligence， 42（4）： 956-973 ［DOI： 10.1109/TPAMI.2018.2886878http://dx.doi.org/10.1109/TPAMI.2018.2886878］
相关文章
相关作者
相关机构
京公网安备11010802024621