浏览全部资源
扫码关注微信
Published： 16 April 2023 ，
移动端阅览
吕佳， 巫若愚. 2023. 多层自适应聚合的自监督小样本图像分类. 中国图象图形学报， 28(04):1056-1068
Lyu Jia， Wu Ruoyu. 2023. Multi-layer adaptive aggregation self-supervised few-shot learning image classification. Journal of Image and Graphics， 28(04):1056-1068
目的
2
在图像分类领域，小样本学习旨在利用从大规模数据集中训练到的知识来处理仅包含少量有标记训练样本的下游分类任务。通常情况下，下游任务只涉及新类样本，由于元训练阶段会构造大量任务随机抽取训练集中不同类别的样本且训练集与测试集类别间存在领域间隙，因此模型训练周期长且可能对训练集过拟合，以致元知识无法迁移到测试集，进而导致模型泛化性差。针对以上问题，提出一种多层自适应聚合的自监督小样本图像分类模型。
方法
2
首先使用分组卷积对残差块进行改进，减少神经网络参数量，降低训练难度，缩短训练时间；然后采用多层自适应聚合的方法改进骨干网络，对网络各层语义信息加以提炼聚合，自适应分配各层权重，将聚合后的特征图作为后续分类的依据；最后加入自监督对比学习结合有监督学习挖掘样本自身潜在的信息，从而提升样本特征表达能力。
结果
2
在mini-ImageNet 数据集和CUB（Caltech-UCSD birds-200-2011）数据集上与当前主流模型进行分类效果对比实验，与baseline相比，所提模型的准确率在mini-ImageNet数据集的5-way 1-shot与5-way 5-shot实验上分别提升了6.31%和6.04%，在CUB数据集的5-way 1-shot与5-way 5-shot实验上分别提升了8.95%和8.77%。
结论
2
本文模型能在一定程度上缩短训练时间、增强样本特征表达能力和优化数据分布，并缓解领域间隙所带来的问题，从而提高模型泛化性与分类效果。
Objective
2
The emerging deep learning technique has facilitated such artificial intelligence （AI）-related domains like image classification， natural language processing， speech recognition， and reinforcement learning. However， it is being challenged for the over-fitting problem of the models. Effective data is required to be obtained from mass， especially in the tasks of image classification. To tackle this problem， the concept of few-shot learning is developed， which aims at well-generalized knowledge learned from a large-scale dataset to handle downstream classification tasks with little training samples. Currently， most popular methods for few-shot image classification are based on meta-learning， which can be learnt to deal with few-shot tasks via similar classification tasks. The process of meta-learning is divided into two steps： 1） meta training， and 2） meta testing. For the meta training， an embedding network is trained by the meta-training set， and it is used to tackle a few training data-constructed downstream classification tasks from the meta-testing set. There is no intersection between meta-training set and meta-testing set， which means that no reliable prior knowledge is obtained by meta-learner in the meta-training process. Due to category differences between meta-training and meta-testing， some new challenges to meta-learning models are required to be resolved. If it focuses on training tasks only， the effectiveness of models will be affected when the meta-learner meets with the few-shot tasks with brand-new categories. To tackle this challenge with metric-based methods， we develop a multi-layer adaptive aggregation self-supervised few-shot classification model.
Method
2
First， to reduce the parameters of the backbone and lower the training difficulty， a group of convolution blocks are used to replace the original convolution. Next， to improve the backbone， the multi-layer adaptive aggregation module is illustrated， which can refine the information of each network layer dynamically and balance the weights of each layer adaptively via the aggregated feature maps of those are the basis for subsequent downstream few-shot classification. Finally， to enhance the transferability of the learned model， the self-supervised contrastive learning is introduced to assist supervised learning to mine the potential information of the data themselves. It does not suffer from over-fitting because contrastive learning is not required to be supervised. It can be as an additional source of regularization as well， which is beneficial for the construction of feature space. The embedding networks can be paid more attention to learn the well-generalized knowledge in terms of the proposed self-supervised contrastive learning method， which makes the distribution of embedding feature maps smoother and the classification model is more suitable for the domain of downstream tasks.
Result
2
To validate the effectiveness of the proposed model， comparative analysis is carried out with some popular models， including 1） prototype network， 2） relation network， 3） cosine classifier， as well as the 4） mini-ImageNet dataset and 5） Caltech-UCSD birds-200-2011（CUB） dataset. For the mini-ImageNet dataset， each accuracy of the proposed model can be reached to 63.13% on 5-way 1-shot and 78.14% on 5-way 5-shot， it can be optimized by 13.71% and 9.94% each for the original Prototype network. For the fine-grained CUB dataset， the accuracies of the proposed model can be reached to 75.93% on 5-way 1-shot and 87.56% on 5-way 5-shot， which are 24.48% and 13.05% higher than the original Prototype network of each. Compared to the baseline on 5-way 1-shot and 5-way 5-shot， each of model accuracy is increased by 6.31%， 6.04% on mini-ImageNet， and they are increased by 8.95%， 8.77% of each as well on CUB. The comparative experiments demonstrate that the parameters of our backbone are optimized in comparison with the parameters of 5 backbones on Prototype network. A couple of ablation experiments are also conducted to verify the proposed model. Additionally， the heat maps-related contrastive experiment between baseline and the proposed model verifies that our model can prevent the embedding network from more background information of images and alleviate the interference of downstream classification tasks. Furthermore， the t-SNE method is used for visualization to sort the distribution of samples out in the feature space. The obtained feature distribution of t-SNE visualization experiment on CUB dataset can demonstrate that our model is capable of differentiating samples well from different categories， which can make the meta-testing set linearly separable.
Conclusion
2
To resolve some problems in the field of few-shot learning， we develop a multi-layer adaptive aggregation self-supervised few-shot classification model. To alleviate the problem of training difficulty， the improved group convolution can be used to reduce the parameters of backbone. To optimize over-fitting and domain gap， the multi-layer adaptive aggregation method and the self-supervised contrastive learning method are used to adjust the distribution of embedding feature maps. In particular， the embedding networks are not be affected by the background-redundant of images based on our self-supervised contrastive learning method.
小样本学习图像分类自适应聚合自监督学习对比学习
few-shot learningimage classificationadaptive aggregationself-supervised learningcontrastive learning
Chen T， Kornblith S， Norouzi M and Hinton G E. 2020. A simple framework for contrastive learning of visual representations//Proceedings of the 37th International Conference on Machine Learning. Vienna， Austria： PMLR： 1597-1607
Chen W Y， Liu Y C， Kira Z， Wang Y C F and Huang J B. 2019a. A closer look at few-shot classification//Proceedings of the 7th International Conference on Learning Representations. New Orleans， USA： OpenReview.net
Chen Z T， Fu Y W， Zhang Y D， Jiang Y G， Xue X Y and Sigal L. 2019b. Multi-level semantic feature augmentation for one-shot learning. IEEE Transactions on Image Processing， 28（9）： 4594-4605 ［DOI： 10.1109/TIP.2019.2910052http://dx.doi.org/10.1109/TIP.2019.2910052］
Finn C， Abbeel P and Levine S. 2017. Model-agnostic meta-learning for fast adaptation of deep networks//Proceedings of the 34th International Conference on Machine Learning. Sydney， Australia： JMLR.org： 1126-1135
He K M， Fan H Q， Wu Y X， Xie S N and Girshick R. 2020. Momentum contrast for unsupervised visual representation learning//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle， USA： IEEE： 9729-9738 ［DOI： 10.1109/CVPR42600.2020.00975http://dx.doi.org/10.1109/CVPR42600.2020.00975］
He K M， Zhang X Y， Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 770-778 ［DOI： 10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90］
Hu Z P， Li Z J， Wang X Y and Zheng S Y. 2022. Unsupervised descriptor selection based meta-learning networks for few-shot classification. Pattern Recognition， 122： #108304 ［DOI： 10.1016/j.patcog.2021.108304http://dx.doi.org/10.1016/j.patcog.2021.108304］
Jamal M A and Qi G J. 2019. Task agnostic meta-learning for few-shot learning//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach， USA： IEEE： 11719-11727 ［DOI： 10.1109/CVPR.2019.01199http://dx.doi.org/10.1109/CVPR.2019.01199］
Lake B M， Salakhutdinov R and Tenenbaum J B. 2013. One-shot learning by inverting a compositional causal process//Proceedings of the 26th International Conference on Neural Information Processing Systems. Lake Tahoe， USA： Curran Associates Inc： 2526-2534
Li F Z， Liu Y， Wu P X， Dong F， Cai Q and Wang Z. 2021. A survey on recent advances in meta-learning. Chinese Journal of Computers， 44（2）： 422-446
李凡长， 刘洋， 吴鹏翔， 董方， 蔡奇， 王哲. 2021. 元学习研究综述. 计算机学报， 44（2）： 422-446 ［DOI： 10.11897/SP.J.1016.2021.00422http://dx.doi.org/10.11897/SP.J.1016.2021.00422］
Li K， Zhang Y L， Li K P and Fu Y. 2020a. Adversarial feature hallucination networks for few-shot learning//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle， USA： IEEE： 13467-13476 ［DOI： 10.1109/CVPR42600.2020.01348http://dx.doi.org/10.1109/CVPR42600.2020.01348］
Li X M， Yu L Q， Fu C W， Fang M and Heng P A. 2020b. Revisiting metric learning for few-shot image classification. Neurocomputing， 406： 49-58 ［DOI： 10.1016/j.neucom.2020.04.040http://dx.doi.org/10.1016/j.neucom.2020.04.040］
Lin T Y， Dollár P， Girshick R， He K M， Hariharan B and Belongie S. 2017. Feature pyramid networks for object detection//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu， USA： IEEE： 936-944 ［DOI： 10.1109/CVPR.2017.106http://dx.doi.org/10.1109/CVPR.2017.106］
Luo X， Chen Y X， Wen L J， Pan L L and Xu Z L. 2021. Boosting few-shot classification with view-learnable contrastive learning//Proceedings of 2021 IEEE International Conference on Multimedia and Expo. Shenzhen， China： IEEE： #9428444 ［DOI： 10.1109/ICME51207.2021.9428444http://dx.doi.org/10.1109/ICME51207.2021.9428444］
Munkhdalai T， Yuan X D， Mehri S and Trischler A. 2018. Rapid adaptation with conditionally shifted neurons//Proceedings of the 35th International Conference on Machine Learning. Stockholm， Sweden： PMLR： 3664-3673
Snell J， Swersky K and Zemel R. 2017. Prototypical networks for few-shot learning//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach， USA： Curran Associates Inc： 1-11
Sun Q R， Liu Y Y， Chua T S and Schiele B. 2019. Meta-transfer learning for few-shot learning//Proceedings of 2019 IEEE Conference on Computer Vision and Pattern Recognition. Long Beach， USA： IEEE： 403-412 ［DOI： 10.1109/CVPR.2019.00049http://dx.doi.org/10.1109/CVPR.2019.00049］
Sung F， Yang Y X， Zhang L， Xiang T， Torr P H S and Hospedales T M. 2018. Learning to compare： relation network for few-shot learning//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City， USA： IEEE： 1199-1208 ［DOI： 10.1109/CVPR.2018.00131http://dx.doi.org/10.1109/CVPR.2018.00131］
Szegedy C， Liu W， Jia Y Q， Sermanet P， Reed S， Anguelov D， Erhan D， Vanhoucke V and Rabinovich A. 2015. Going deeper with convolutions//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston， USA： IEEE： #7298594 ［DOI： 10.1109/CVPR.2015.7298594http://dx.doi.org/10.1109/CVPR.2015.7298594］
Vinyals O， Blundell C， Lillicrap T， Kavukcuoglu K and Wierstra D. 2016. Matching networks for one shot learning//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona， Spain： Curran Associates Inc： 3630-3638
Wang Y X， Girshick R， Hebert M and Hariharan B. 2018. Low-shot learning from imaginary data//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City， USA： IEEE： 7278-7286 ［DOI： 10.1109/CVPR.2018.00760http://dx.doi.org/10.1109/CVPR.2018.00760］
Xie S N， Girshick R， Dollár P， Tu Z W and He K M. 2017. Aggregated residual transformations for deep neural networks//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu， USA： IEEE： 5987-5995 ［DOI： 10.1109/CVPR.2017.634http://dx.doi.org/10.1109/CVPR.2017.634］
Ye H J， Hu H X， Zhan D C and Sha F. 2020. Few-shot learning via embedding adaptation with set-to-set functions//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle， USA： IEEE： 8805-8814 ［DOI： 10.1109/CVPR42600.2020.00883http://dx.doi.org/10.1109/CVPR42600.2020.00883］
Zhang C， Cai Y J， Lin G S and Shen C H. 2020. DeepEMD： few-shot image classification with differentiable earth mover’s distance and structured classifiers//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle， USA： IEEE： 12200-12210 ［DOI： 10.1109/CVPR42600.2020.01222http://dx.doi.org/10.1109/CVPR42600.2020.01222］
Zhao K L， Jin X L and Wang Y Z. 2021. Survey on few-shot learning. Journal of Software， 32（2）： 349-369
赵凯琳， 靳小龙， 王元卓. 2021. 小样本学习研究综述. 软件学报， 32（2）： 349-369 ［DOI： 10.13328/j.cnki.jos.006138http://dx.doi.org/10.13328/j.cnki.jos.006138］
Zhao P， Wang C Y， Zhang S Y and Liu Z Y. 2021. A zero-shot image classification method based on subspace learning with the fusion of reconstruction. Chinese Journal of Computers， 44（2）： 409-421
赵鹏， 汪纯燕， 张思颖， 刘政怡. 2021. 一种基于融合重构的子空间学习的零样本图像分类方法. 计算机学报， 44（2）： 409-421 ［DOI： 10.11897/SP.J.1016.2021.00409http://dx.doi.org/10.11897/SP.J.1016.2021.00409］
相关文章
相关作者
相关机构
京公网安备11010802024621