浏览全部资源
扫码关注微信
Published： 16 May 2023 ，
移动端阅览
吴禹航， 桑农. 2023. 基于一致性约束和标签优化的无监督域适应行人重识别. 中国图象图形学报， 28(05):1372-1383
Wu Yuhang， Sang Nong. 2023. Consistency constraints and label optimization-relevant domain-unsupervised adaptive pedestrians’ re-identification. Journal of Image and Graphics， 28(05):1372-1383
目的
2
无监督域适应行人重识别（unsupervised domain adaptive pedestrians’ re-identification，UDA Re-ID）旨在通过已有标记的应用场景（即源域）数据和新的无标记应用场景（即目标域）数据，训练一个可以在目标域泛化性能好的行人重识别模型。现有方法没有考虑实例特征在训练过程中的不稳定问题，也没有显式考虑由于相机变化所导致的行人类内距离变大、类间距离变小的问题，以及无标注目标域数据聚类误差带来的伪标签噪声问题。针对这些问题，提出了一种具有一致性约束和标签优化的方法。
方法
2
首先提出了实例一致性以约束同一实例在不同增广下的特征距离，提升行人实例特征稳定性；然后提出相机一致性以约束跨相机正实例特征对之间的距离，提升对相机变化的鲁棒性；最后提出了基于标签集成的标签优化，将one-hot编码的伪标签转换为更可靠的软标签，提升了监督信号的鲁棒性。
结果
2
本文方法在Duke→Market，Market→Duke，Duke→MSMT，Market→MSMT等常用的UDA Re-ID任务上的平均精度均值（mean average precision，mAP）分别为85.0%，73.5%，41.3%， 39.3%；Rank-1分别为94.0%，85.6%，71.6%，69.5%。通过消融实验验证了本文提出的3个模块的有效性。
结论
2
提出的实例一致性约束和相机一致性约束可以使模型学习到更鲁棒的行人特征表达，提升行人特征的稳定性，提出的基于标签集成的标签优化可以减少伪标签噪声的过拟合风险。
Objective
2
Pedestrians’ re-identification （Re-ID） can be as one of key techniques for multi-camera pedestrians’ retrieval in the video surveillance system. To achieve good performance， current Re-ID model is often trained on a scene based on a large number of annotations （source domain）， but the performance will be dropped significantly when a new scene （target domain） is applied straightforward. However， re-labeling is time-consuming and labor-intensive for the new scene， which is beneficial for Re-ID-related optimization. The unsupervised domain adaptive pedestrians’ re-identification （UDA Re-ID） method is focused on a model training， which can be generalized on the target domain well using the existing source domain data-labeled and target domain data-unlabeled. But， these methods are still challenged for the instability of instance features and image heterogeneity of the intra-class distance-wider and the inter-class distance-narrowed. Furthermore， current cluster unlabeled target domain data can be melted into multiple clusters， and the encoded pseudo labels can be assigned for each cluster. However， due to the limited representation ability of the model， the clustering results are incredible， especially in the early stage of training. One pedestrian-related image is grouped into different clusters while some images of different pedestrian are merged into a cluster， called pseudo label noises. However， it is still challenged for the problem of pseudo label noises-over-fitted and the performance of the model-suppressed although pseudo labels are recognized as a supervision signal for the feature learning process （e.g.， contrastive learning）. To resolve these problems， we develop a multi-centroid representation network with consistency constraints method （MCRNCC） in terms of the popular multi-centroid representation network method （MCRN）.
Method
2
The MCRNCC is designed on the basis of three MCRN-related modules to improve the stability of instance features and the robustness of pedestrian features， and the overfitting risk of the pseudo-label noise can be reduced. First， to optimize the instance feature stability and semantic information， an instance-consistent is demonstrated to suppress the feature distance of the same instance under different augmentation. The exponential moving average model is illustrated to output additional features based on recent self-supervised learning works. For each image of the training batch， it can be augmented twice in random， the features can be extracted in relevance to original model and exponential moving average model， and cosine distance is used to constrain the feature pairs. Second， to improve robustness of multiple variations-captured， its homogeneity is concerned for suppressing the distance between feature pairs of positive instances. Specifically， two instances are opted to construct a positive pair in the context of same labels-without identity label， and two instances is opted as well to construct a negative pair in related to same label-within multiple identity labels， and a triplet is built up to optimize the network as well. Finally， the label-ensemble-based optimization is carried out to convert one-hot encoded pseudo labels into more reliable soft labels， which improves the robustness of supervision signals. In detail， we add a target domain classifier to generate additional label predictions， followed by linearly weighting the predictions and one-hot encoded pseudo labels into refined soft labels.
Result
2
To verify the effectiveness of our method， adequate experiments are carried out on 4 popular UDA Re-ID tasks like 1） Duke→Market， 2） Market→Duke， 3） Duke→MSMT， and 4） Market→MSMT tasks. At the beginning， the ablation studies are carried out about the modules in MCRNCC. The four tasks-derived instance consistency constraints can be reached to 0.6%， 0.2%， 0.7% and 0.8% of each mean average precision （mAP）， which demonstrates the effectiveness of the instance consistency constraint. The camera consistency constraint yields a general improvement for all of 4 tasks. For example， mAP/Rank-1 is increased by 3.5%/3.2% and 5.3%/4.7% on Duke→MSMT and Market→MSMT. In addition， we visualize the feature space of adding camera consistency constraint before and after. Furthermore， we compare the feature space of some pedestrian-focused close to the camera consistency constraint， and the visualization results show that the camera consistency can make the feature space more compactible. The label-ensemble-optimized can be improved to 0.6%， 0.6%， 1.4% and 0.4% of the mAP for each 4 tasks. Second， our proposed MCRNCC is compared to the existing methods. The comparative analysis shows that the MCRNCC can be reached to 85.0%/94.0%， 73.5%/85.6%， 41.3%/71.6% and 39.3%/69.5% for the optimization of mAP/Rank-1 performance， and the MCRN is surpassed by 1.2%/0.2%， 2.0%/1.1%， 5.6%/4.1%， and 6.5%/5.1% as well.
Conclusion
2
we develop a method MCRNCC to resolve the UDA Re-ID problem further. The instance consistency constraint and camera consistency constraint proposed in MCRNCC can enable the model to learn more robust pedestrian-related feature representations， while the proposed label ensemble-based optimization can reduce the overfitting risks of pseudo label noises. Experiments show that the effectiveness of three-module based MCRNCC has its potentials for future works.
计算机视觉行人重识别（Re-ID）无监督域适应（UDA）一致性约束标签优化
computer visionpedestrians’ re-identification（Re-ID）unsupervised domain adaption（UDA）consistency constraintlabel refinery
Chang W G， You T， Seo S， Kwak S and Han B. 2019. Domain-specific batch normalization for unsupervised domain adaptation//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach， USA： IEEE： 7346-7354 ［DOI： 10.1109/CVPR.2019.00753http://dx.doi.org/10.1109/CVPR.2019.00753］
Chen H， Wang Y H， Lagadec B， Dantcheva A and Bremond F. 2021. Joint generative and contrastive learning for unsupervised person re-identification//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Virtual， USA： IEEE： 2004-2013 ［DOI： 10.1109/CVPR46437.2021.00204http://dx.doi.org/10.1109/CVPR46437.2021.00204］
Chen T， Kornblith S， Norouzi M and Hinton G. 2020. A simple framework for contrastive learning of visual representations//Proceedings of the 37th International Conference on Machine Learning. Virtual： PMLR.org： 1597-1607
Dai Y X， Liu J， Sun Y F， Tong Z K， Zhang C and Duan L Y. 2021. IDM： an intermediate domain module for domain adaptive person Re-ID//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal， Canada： IEEE： 11844-11854 ［DOI： 10.1109/ICCV48922.2021.01165http://dx.doi.org/10.1109/ICCV48922.2021.01165］
Deng J， Dong W， Socher R， Li L J， Li K and Fei-Fei L. 2009. ImageNet： a large-scale hierarchical image database//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami， USA： IEEE： 248-255 ［DOI： 10.1109/CVPR.2009.5206848http://dx.doi.org/10.1109/CVPR.2009.5206848］
Deng W J， Zheng L， Ye Q X， Kang G L， Yang Y and Jiao J B. 2018. Image-image domain adaptation with preserved self-similarity and domain-dissimilarity for person re-identification//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City， USA： IEEE： 994-1003 ［DOI： 10.1109/CVPR.2018.00110http://dx.doi.org/10.1109/CVPR.2018.00110］
Ester M， Kriegel H P， Sander J and Xu X W. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise//Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining. Portland， USA： AAAI Press： 226-231
Ge Y X， Chen D P and Li H S. 2020a. Mutual mean-teaching： pseudo label refinery for unsupervised domain adaptation on person re-identification ［EB/OL］. ［2022-06-24］. https：//arxiv.org/pdf/2001.01526/pdfhttps://arxiv.org/pdf/2001.01526/pdf
Ge Y X， Zhu F， Chen D P， Zhao R and Li H S. 2020b. Self-paced contrastive learning with hybrid memory for domain adaptive object Re-ID//Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver， Canada： Curran Associates Inc.： 11309-11321
Goodfellow I J， Pouget-Abadie J， Mirza M， Xu B， Warde-Farley D， Ozair S， Courville A and Bengio Y. 2014. Generative adversarial nets//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal， Canada： MIT Press： 2672-2680
Grill J B， Strub F， Altché F， Tallec C， Richemond P H， Buchatskaya E， Doersch C， Pires B A， Guo Z D， Azar M G， Piot B， Kavukcuoglu K， Munos R and Valko M. 2020. Bootstrap your own latent a new approach to self-supervised learning//Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver， Canada： Curran Associates Inc.： 21271-21284
He K M， Fan H Q， Wu Y X， Xie S N and Girshick R. 2020. Momentum contrast for unsupervised visual representation learning//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle， USA： IEEE： 9726-9735 ［DOI： 10.1109/CVPR42600.2020.00975http://dx.doi.org/10.1109/CVPR42600.2020.00975］
He K M， Zhang X Y， Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas， USA： IEEE： 770-778 ［DOI： 10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90］
Isobe T， Li D， Tian L， Chen W H， Shan Y and Wang S J. 2021. Towards discriminative representation learning for unsupervised person re-identification//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal， Canada： IEEE： 8506-8516 ［DOI： 10.1109/ICCV48922.2021.00841http://dx.doi.org/10.1109/ICCV48922.2021.00841］
Ren S Q， He K M， Girshick R and Sun J. 2015. Faster R-CNN： towards real-time object detection with region proposal networks//Proceedings of the 28th International Conference on Neural Information Processing Systems. Montreal， Canada： MIT Press： 91-99
Ristani E， Solera F， Zou R， Cucchiara R and Tomasi C. 2016. Performance measures and a data set for multi-target， multi-camera tracking//Proceedings of 2016 European Conference on Computer Vision. Amsterdam， the Netherlands： Springer： 17-35 ［DOI： 10.1007/978-3-319-48881-3_2http://dx.doi.org/10.1007/978-3-319-48881-3_2］
Shi W D， Zhang Y Z， Liu S W， Zhu S D and Bao J N. 2020. Person re-identification based on deformation and occlusion mechanisms. Journal of Image and Graphics， 25（12）： 2530-2540
史维东， 张云洲， 刘双伟， 朱尚栋， 暴吉宁. 2020. 针对形变与遮挡问题的行人再识别. 中国图象图形学报， 25（12）： 2530-2540 ［DOI： 10.11834/jig.200016http://dx.doi.org/10.11834/jig.200016］
Tian Z， Shen C H， Chen H and He T. 2019. FCOS： fully convolutional one-stage object detection//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul， Korea （South）： IEEE： 9626-9635 ［DOI： 10.1109/ICCV.2019.00972http://dx.doi.org/10.1109/ICCV.2019.00972］
Wei L H， Zhang S L， Gao W and Tian Q. 2018. Person transfer GAN to bridge domain gap for person re-identification//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City， USA： IEEE： 79-88 ［DOI： 10.1109/CVPR.2018.00016http://dx.doi.org/10.1109/CVPR.2018.00016］
Wu Y H， Huang T T， Yao H T， Zhang C， Shao Y J， Han C C， Gao C X and Sang N. 2022. Multi-centroid representation network for domain adaptive person Re-ID. Proceedings of the AAAI Conference on Artificial Intelligence， 36（3）： 2750-2758 ［DOI： 10.1609/aaai.v36i3.20178http://dx.doi.org/10.1609/aaai.v36i3.20178］
Zhai Y P， Ye Q X， Lu S J， Jia M X， Ji R R and Tian Y H. 2020. Multiple expert brainstorming for domain adaptive person re-identification//Proceedings of the 16th European Conference on Computer Vision. Glasgow， UK： Springer： 594-611 ［DOI： 10.1007/978-3-030-58571-6_35http://dx.doi.org/10.1007/978-3-030-58571-6_35］
Zheng K C， Lan C L， Zeng W J， Zhang Z Z and Zha Z J. 2021a. Exploiting sample uncertainty for domain adaptive person re-Identification. Proceedings of the AAAI Conference on Artificial Intelligence， 35（4）： 3538-3546 ［DOI： 10.1609/aaai.v35i4.16468http://dx.doi.org/10.1609/aaai.v35i4.16468］
Zheng K C， Liu W， He L X， Mei T， Luo J B and Zha Z J. 2021b. Group-aware label transfer for domain adaptive person re-identification//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville， USA： IEEE： 5306-5315 ［DOI： 10.1109/CVPR46437.2021.00527http://dx.doi.org/10.1109/CVPR46437.2021.00527］
Zheng L， Shen L Y， Tian L， Wang S J， Wang J D and Tian Q. 2015. Scalable person re-identification： a benchmark//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago， Chile： IEEE： 1116-1124 ［DOI： 10.1109/ICCV.2015.133http://dx.doi.org/10.1109/ICCV.2015.133］
Zheng X， Lin L， Ye M， Wang L and He C L. 2020. Improving person re-identification by attention and multi-attributes. Journal of Image and Graphics， 25（5）： 936-945
郑鑫， 林兰， 叶茂， 王丽， 贺春林. 2020. 结合注意力机制和多属性分类的行人再识别. 中国图象图形学报， 25（5）： 936-945 ［DOI： 10.11834/jig.190185http://dx.doi.org/10.11834/jig.190185］
Zheng Y， Tang S X， Teng G L， Ge Y X， Liu K J， Qin J， Qi D L and Chen D P. 2021c. Online pseudo label generation by hierarchical cluster dynamics for adaptive person re-identification//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal， Canada： IEEE： 8351-8361 ［DOI： 10.1109/ICCV48922.2021.00826http://dx.doi.org/10.1109/ICCV48922.2021.00826］
Zhong Z， Zheng L， Kang G L， Li S Z and Yang Y. 2020. Random erasing data augmentation. Proceedings of the AAAI Conference on Artificial Intelligence， 34（7）： 13001-13008 ［DOI： 10.1609/aaai.v34i07.7000http://dx.doi.org/10.1609/aaai.v34i07.7000］
相关文章
相关作者
相关机构
京公网安备11010802024621