浏览全部资源
扫码关注微信
Published： 16 September 2023 ，
移动端阅览
杜彦东， 冯林， 陶鹏， 龚勋， 王俊. 2023. 元迁移学习在少样本跨域图像分类中的研究. 中国图象图形学报， 28(09):2899-2912
Du Yandong， Feng Lin， Tao Peng， Gong Xun， Wang Jun. 2023. Meta-transfer learning in cross-domain image classification with few-shot learning. Journal of Image and Graphics， 28(09):2899-2912
目的
2
现有基于元学习的主流少样本学习方法假设训练任务和测试任务服从相同或相似的分布，然而在分布差异较大的跨域任务上，这些方法面临泛化能力弱、分类精度差等挑战。同时，基于迁移学习的少样本学习方法没有考虑到训练和测试阶段样本类别不一致的情况，在训练阶段未能留下足够的特征嵌入空间。为了提升模型在有限标注样本困境下的跨域图像分类能力，提出简洁的元迁移学习（compressed meta transfer learning，CMTL）方法。
方法
2
基于元学习，对目标域中的支持集使用数据增强策略，构建新的辅助任务微调元训练参数，促使分类模型更加适用于域差异较大的目标任务。基于迁移学习，使用自压缩损失函数训练分类模型，以压缩源域中基类数据所占据的特征嵌入空间，微调阶段引导与源域分布差异较大的新类数据有更合适的特征表示。最后，将以上两种策略的分类预测融合视为最终的分类结果。
结果
2
使用mini-ImageNet作为源域数据集进行训练，分别在EuroSAT（European Satellite）、ISIC（International Skin Imaging Collaboration）、CropDiseas（Crop Diseases）和Chest-X（Chest X-Ray）数据集上测试CMTL模型的跨域图像分类能力，在5-way 1-shot和5-way 5-shot跨域任务中，准确率分别达到68.87%和87.74%、34.47%和49.71%、74.92%和93.37%、22.22%和25.40%。与当前主流少样本跨域图像分类方法比较，提出的CMTL方法在各个数据集中都有较好的跨域图像分类能力。
结论
2
提出的CMTL方法综合了迁移学习和元学习方法各自在跨域任务上的优势，能有效解决少样本跨域图像分类问题。
Objective
2
Few-shot learning image classification aims to recognize images with limited labeled samples. At present， few-shot learning methods are roughly divided into three categories： gradient optimization， metric learning， and transfer learning. Gradient optimization methods usually consist of two loop stages. In the inner loop stage， the base model quickly adapts to new tasks with limited labeled samples； in the outer loop stage， the meta-model learns cross-task knowledge for good generalization performance. The metric learning method first maps the samples to the high-dimensional embedding space. Then， the correct classification of the unknown samples is completed according to the similarity measure. The transfer learning method first pretrains a high-quality feature extractor with a large amount of annotated data and migrates to fine-tune the classifier. Thus， the model is suitable for the current task. The existing few-shot learning methods based on meta-learning assume that the training and test tasks are the same or have a similar distribution. However， these methods face cross-domain classification challenges， such as weak generalization ability and poor classification accuracy. In the training and testing stages， the few-shot learning method based on transfer learning does not consider the inconsistency of sample categories. It also fails to leave enough feature embedding space for new category samples. On the basis of the idea of integrating transfer learning and meta-learning， we propose a model of compressed meta-transfer learning （CMTL） to improve the cross-domain ability of few-shot learning.
Method
2
The method is mainly composed of two aspects. On the one hand， for meta-learning， the prior knowledge generated by meta-training is used to complete the classification of the target task. When the source and target tasks have different data distributions， the base class data used for training comes from the source domain natural dataset mini-ImageNet， whereas the novel class data used for testing comes from the target domain medical dataset Chest-X. The meta-knowledge acquired by the source task training cannot be quickly generalized to the target task because of its lack of universality， which further leads to poor cross-domain classification effects. In this study， new auxiliary tasks with strategies， such as random cropping and gamma transformation， were constructed on the support set of the target domain during the testing process. These auxiliary tasks fine-tune the meta-training parameters to improve task adaptability. On the other hand， for transfer learning， the sample categories are consistent during training and testing. Thus， the feature embedding space available for the novel class samples is small if the deep learning model is optimized with the traditional softmax loss function. This scenario further leads to the unsatisfactory feature extraction ability of the model and poor classification accuracy. Given the above problems， this study proposes using a self-compression loss function in the pretraining stage. This loss function adjusts the distribution position between prototypes of the base classes to make the base class samples concentrated in the embedding space and reserve part of the embedding space for the novel class. In the fine-tuning stage， the novel class with large domain distribution is guided to obtain expressive features. The existing studies on cross-domain few-shot learning show that meta-learning methods perform well when the data distributions of the target and source tasks are similar. Conversely， transfer learning methods perform effectively. The ensemble of prediction scores of the above two strategies is regarded as the final classification result to take full advantage of the above two methods.
Result
2
This study compares the proposed model with several state-of-the-art cross-domain few-shot image classification models， such as graph convolutional network （GCN）， adversarial task augmentation （ATA）， self-training to adapt representations to unseen problems （STARTUP）， and other classic methods. Compared with the current state-of-the-art cross-domain few-shot methods， CMTL has advantages， as shown in the experimental results. In the testing phase， extensive experiments are performed on the 5-way 1-shot and 5-way 5-shot settings to complete the validation of the model and ensure a fair comparison with advanced methods. The experimental results show that on the 5-way 1-shot and 5-way 5-shot settings， the mini-ImageNet is used as the source domain dataset for training. Moreover， the effectiveness of the CMTL on the EuroSAT， ISIC， CropDisease， and Chest-X datasets is tested， and the accuracy rates reach 68.87%/87.74%， 34.47%/49.71%， 74.92%/93.37%， and 22.22%/25.40%， respectively. Compared with meta-learning and transfer learning models， our model achieves competitive results on the 5-way 1-shot and 5-way 5-shot settings on all cross-domain tasks.
Conclusion
2
This study proposes a cross-domain few-shot image classification model based on meta-transfer learning. The proposed model improves the generalization ability of few-shot learning. The experimental results show that the CMTL proposed in this study combines the advantages of meta-learning and transfer learning methods. It also has significant effects on cross-domain few-shot tasks.
图像分类少样本跨域元学习迁移学习少样本学习（FSL）
image classificationfew-shot cross-domainmeta learningtransfer learningfew-shot learning （FSL）
Bertinetto L， Henriques J F， Valmadre J， Torr P H S and Vedaldi A. 2016. Learning feed-forward one-shot learners//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona， Spain： Curran Associates Inc.： 523-531
Chen S， Zhong H C， Li Y Z， Wang S Z and Yang J G. 2021. Haze image recognition based on multi-scale feature and multi-adversarial networks. Journal of Image and Graphics， 26（11）： 2680-2690
陈硕， 钟汇才， 李勇周， 王师峥， 杨建刚. 2021. 基于多尺度特征多对抗网络的雾天图像识别. 中国图象图形学报， 26（11）： 2680-2690 ［DOI： 10.11834/jig.200491http://dx.doi.org/10.11834/jig.200491］
Chen W Y， Liu Y C， Kira Z， Wang Y C F and Huang J B. 2019. A closer look at few-shot classification//Proceedings of the 7th International Conference on Learning Representations. New Orleans， USA： OpenReview.net： 1-17
Codella N， Rotemberg V， Tschandl P， Cele M E， Dusza S， Gutman D， Helba B， Kalloo A， Liopyris K， Marchetti M， Kittler H and Halpern A. 2019. Skin lesion analysis toward melanoma detection 2018： a challenge hosted by the international skin imaging collaboration （ISIC） ［EB/OL］. ［2022-06-15］. https://arxiv.org/pdf/1902.03368.pdfhttps://arxiv.org/pdf/1902.03368.pdf
Finn C， Abbeel P and Levine S. 2017. Model-agnostic meta-learning for fast adaptation of deep networks//Proceedings of the 34th International Conference on Machine Learning. Sydney， Australia： PMLR： 1126-1135
Fu Y Q， Fu Y W and Jiang Y G. 2021. Meta-FDMixup： cross-domain few-shot learning guided by labeled target data//Proceedings of the 29th ACM International Conference on Multimedia. Chengdu， China： ACM： 5326-5334 ［DOI： 10.1145/3474085.3475655http://dx.doi.org/10.1145/3474085.3475655］
Ganin Y， Ustinova E， Ajakan H， Germain P， Larochelle H， Laviolette F， Marchand M and Lempitsky V. 2016. Domain-adversarial training of neural networks. The Journal of Machine Learning Research， 17（1）： 2096-2030
Garcia V and Bruna J. 2018. Few-shot learning with graph neural networks//Proceedings of the 6th International Conference on Learning Representations. Vancouver， Canada： OpenReview.net： 1-13
Guo Y H， Codella N C， Karlinsky L， Codella J V， Smith J R， Saenko K， Rosing T and Feris R. 2020. A broader study of cross-domain few-shot learning//Proceedings of the 16th European Conference on Computer Vision. Glasgow， UK： Springer： 124-141 ［DOI： 10.1007/978-3-030-58583-9_8http://dx.doi.org/10.1007/978-3-030-58583-9_8］
He K M， Zhang X Y， Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 770-778 ［DOI： 10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90］
Helber P， Bischke B， Dengel A and Borth D. 2019. EuroSAT： a novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing， 12（7）： 2217-2226 ［DOI： 10.1109/JSTARS.2019.2918242http://dx.doi.org/10.1109/JSTARS.2019.2918242］
Hoffman J， Tzeng E， Park T， Zhu J Y， Isola P， Saenko K， Efros A A and Darrell T. 2018. CyCADA： cycle-consistent adversarial domain adaptation//Proceedings of the 35th International Conference on Machine Learning. Stockholm， Sweden： PMLR： 1994-2003
Kang G L， Jiang L， Yang Y and Hauptmann A G. 2019. Contrastive adaptation network for unsupervised domain adaptation//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach， USA： IEEE： 4888-4897 ［DOI： 10.1109/CVPR.2019.00503http://dx.doi.org/10.1109/CVPR.2019.00503］
Kumar A， Sattigeri P， Wadhawan K， Karlinsky L， Feris R， Freeman W T and Wornell G. 2018. Co-regularized alignment for unsupervised domain adaptation//Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montreal， Canada： Curran Associates Inc.： 9367-9378
Li W H， Liu X L and Bilen H. 2021. Universal representation learning from multiple domains for few-shot classification//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal， Canada： IEEE： 9506-9515 ［DOI： 10.1109/ICCV48922.2021.00939http://dx.doi.org/10.1109/ICCV48922.2021.00939］
Long M S， Zhu H， Wang J M and Jordan M I. 2017. Deep transfer learning with joint adaptation networks//Proceedings of the 34th International Conference on Machine Learning. Sydney， Australia： JMLR.org： 2208-2217
Mohanty S P， Hughes D P and Salathé M. 2016. Using deep learning for image-based plant disease detection. Frontiers in Plant Science， 7： #1419 ［DOI： 10.3389/fpls.2016.01419http://dx.doi.org/10.3389/fpls.2016.01419］
Nichol A， Achiam J and Schulman J. 2018. On first-order meta-learning algorithms ［EB/OL］. ［2022-06-15］. https://arxiv.org/pdf/1803.02999v3.pdfhttps://arxiv.org/pdf/1803.02999v3.pdf
Phoo C P and Hariharan B. 2021. Self-training for few-shot transfer across extreme task differences//Proceedings of the 9th International Conference on Learning Representations. Vienna， Austria： OpenReview.net： 1-19
Rajasegaran J， Khan S， Hayat M， Khan F S and Shah M. 2021. Self-supervised knowledge distillation for few-shot learning//Proceedings of the 32nd British Machine Vision Conference. ［s.l.］： BMVA Press： #179
Ravi S and Larochelle H. 2017. Optimization as a model for few-shot learning//Proceedings of the 5th International Conference on Learning Representations. Toulon， France： OpenReview.net： 1-11
Ren H and Lu H. 2022. Compositional coding capsule network with k-means routing for text classification. Pattern Recognition Letters， 160： 1-8 ［DOI： 10.1016/j.patrec.2022.05.028http://dx.doi.org/10.1016/j.patrec.2022.05.028］
Russakovsky O， Deng J， Su H， Krause J， Satheesh S， Ma S A， Huang Z H， Karpathy A， Khosla A， Bernstein M， Berg A C and Li F F. 2015. ImageNet large scale visual recognition challenge. International Journal of Computer Vision， 115（3）： 211-252 ［DOI： 10.1007/s11263-015-0816-yhttp://dx.doi.org/10.1007/s11263-015-0816-y］
Snell J， Swersky K and Zemel R. 2017. Prototypical networks for few-shot learning//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach， USA： Curran Associates Inc.： 4080-4090
Sun B C， Feng J S and Saenko K. 2016. Return of frustratingly easy domain adaptation//Proceedings of the 30th AAAI Conference on Artificial Intelligence. Phoenix， USA： AAAI： 2058-2065 ［DOI： 10.1609/aaai.v30i1.10306http://dx.doi.org/10.1609/aaai.v30i1.10306］
Sun J M， Lapuschkin S， Samek W， Zhao Y Q， Cheung N M and Binder A. 2021. Explanation-guided training for cross-domain few-shot classification//Proceedings of the 25th International Conference on Pattern Recognition. Milan， Italy： IEEE： 7609-7616 ［DOI： 10.1109/icpr48806.2021.9412941http://dx.doi.org/10.1109/icpr48806.2021.9412941］
Sung F， Yang Y X， Zhang L， Xiang T， Torr P H S and Hospedales T M. 2018. Learning to compare： relation network for few-shot learning//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City， USA： IEEE： 1199-1208 ［DOI： 10.1109/CVPR.2018.00131http://dx.doi.org/10.1109/CVPR.2018.00131］
Tseng H Y， Lee H Y， Huang J B and Yang M H. 2020. Cross-domain few-shot classification via learned feature-wise transformation//Proceedings of the 8th International Conference on Learning Representations. Addis Ababa， Ethiopia： OpenReview.net：1-18
Tzeng E， Hoffman J， Saenko K and Darrell T. 2017. Adversarial discriminative domain adaptation//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu， USA： IEEE： 2962-2971 ［DOI： 10.1109/CVPR.2017.316http://dx.doi.org/10.1109/CVPR.2017.316］
Vinyals O， Blundell C， Lillicrap T， Kavukcuoglu K and Wierstra D. 2016. Matching networks for one shot learning//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona， Spain： Curran Associates Inc.： 3637-3645
Wang C， Liu R Z， Ye H J and Yu Y. 2020. Novelty-prepared few-shot classification ［EB/OL］. ［2022-06-15］. https://arxiv.org/pdf/2003.00497v1.pdfhttps://arxiv.org/pdf/2003.00497v1.pdf
Wang H Q and Deng Z H. 2021. Cross-domain few-shot classification via adversarial task augmentation//Proceedings of the 30th International Joint Conference on Artificial Intelligence. Montreal， Canada： ijcai.org. 1075-1081 ［DOI： 10.24963/ijcai.2021/149http://dx.doi.org/10.24963/ijcai.2021/149］
Wang M M， Yang X Q and Liu Y. 2022. A spatio-temporal encoded network for single object tracking. Journal of Image and Graphics， 27（9）： 2733-2748
王蒙蒙， 杨小倩， 刘勇. 2022. 利用时空特征编码的单目标跟踪网络. 中国图象图形学报， 27（9）： 2733-2748 ［DOI： 10.11834/jig.211157http://dx.doi.org/10.11834/jig.211157］
Wang X S， Peng Y F， Lu L， Lu Z Y， Bagheri M and Summers RM. 2017. ChestX-ray8： hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu， USA： IEEE： 3462-3471 ［DOI： 10.1109/CVPR.2017.369http://dx.doi.org/10.1109/CVPR.2017.369］
Yeh J F， Lee H Y， Tsai B C， Chen Y R， Huang P C and Hsu W H. 2020. Large margin mechanism and pseudo query set on cross-domain few-shot learning ［EB/OL］. ［2022-06-15］. https://arxiv.org/pdf/2005.09218.pdfhttps://arxiv.org/pdf/2005.09218.pdf
Yu Y， Feng L， Wang G G and Xu Q F. 2019. A few-shot learning model based on semi-supervised with pseudo label. Acta Electronica Sinica， 47（11）： 2284-2291
余游， 冯林， 王格格， 徐其凤. 2019. 一种基于伪标签的半监督少样本学习模型. 电子学报， 47（11）： 2284-2291 ［DOI： 10.3969/j.issn.0372-2112.2019.11.007http://dx.doi.org/10.3969/j.issn.0372-2112.2019.11.007］
Zhang L L， Chen Y W， Wu W J， Wei B F， Luo X， Chang X J and Liu J. 2021. Interpretable few-shot learning with contrastive constraint. Journal of Computer Research and Development， 58（12）： 2573-2584
张玲玲， 陈一苇， 吴文俊， 魏笔凡， 罗炫， 常晓军， 刘均. 2021. 基于对比约束的可解释小样本学习. 计算机研究与发展， 58（12）： 2573-2584 ［DOI： 10.7544/issn1000-1239.2021.20210999http://dx.doi.org/10.7544/issn1000-1239.2021.20210999］
Zhao K L， Jin X L and Wang Y Z. 2021. Survey on few-shot learning. Journal of Software， 32（2）： 349-369
赵凯琳， 靳小龙， 王元卓. 2021. 小样本学习研究综述. 软件学报， 32（2）： 349-369 ［DOI： 10.13328/j.cnki.jos.006138http://dx.doi.org/10.13328/j.cnki.jos.006138］
Zhu J Y， Park T， Isola P and Efros A A. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice， Italy： IEEE： 2242-2251 ［DOI： 10.1109/ICCV.2017.244http://dx.doi.org/10.1109/ICCV.2017.244］
相关文章
相关作者
相关机构
京公网安备11010802024621