浏览全部资源
扫码关注微信
Published： 16 May 2023 ，
移动端阅览
刘宇奇， 马丙鹏. 2023. 素描图像指导的换装行人重识别. 中国图象图形学报， 28(05):1396-1408
Liu Yuqi， Ma Bingpeng. 2023. Sketch images-guided clothes-changing person re-identification. Journal of Image and Graphics， 28(05):1396-1408
目的
2
传统行人重识别方法提取到的特征中包含大量的衣物信息，在换装行人重识别问题中，依靠衣物相关的信息难以准确判别行人身份，使模型性能显著下降；虽然一些方法从轮廓图像中提取行人的体型信息以增强行人特征，但轮廓图像的质量参差不齐，鲁棒性差。针对这些问题，本文提出一种素描图像指导的换装行人重识别方法。
方法
2
首先，本文认为相对于轮廓图像，素描图像能够提供更鲁棒、更精准的行人体型信息，因此本文使用素描图像提取行人的体型信息，并将其融入表观特征以获取完备的行人特征。然后，提出一个基于素描图像的衣物无关权重指导模块，进一步使用素描图像中的衣物位置信息指导表观特征的提取过程，从而减少表观特征中的衣物信息，增强表观特征的判别力。
结果
2
在LTCC（long-term cloth changing）和PRCC（person re-identification under moderate clothing change）两个常用换装行人数据集上，本文方法与最先进的方法进行了对比。相较于先进方法，在LTCC和PRCC数据集上，本文方法在Rank-1性能指标上分别提高了6.5%和3.9%。实验结果表明，素描图像在鲁棒性和准确性上均优于轮廓图像，能够更好地获取行人体型信息，并且能够为表观特征提供更多体型互补信息。
结论
2
提出的衣物无关权重指导模块能有效减少行人表观特征中衣物信息的含量；提出的素描图像指导的换装行人重识别方法有效获取了包含衣物无关表观特征和体型特征在内的完备行人特征。
Objective
2
Video surveillance systems have been widely used for public security such as tracking the suspect and looking for missing person. It is really expensive and time-consuming to analyze videos manually. Person re-identification （ReID） aims to match the same person appearing at different times and places under non-overlapping cameras. With the development of deep learning， ReID has gained significant performance increment on the benchmark. It is known that in ReID， the retrieval mainly depends on apparent cues such as clothes information. However， if the surveillance video is captured in a long-time span， people may change their clothes when appearing in the surveillance system. Besides， criminals may also change their clothes to evade surveillance cameras. In such cases， existing methods are likely to fail because they extract unreliable clothes-relevant features. Clothes-changing problem is inevitable in the real-scene application of ReID. Recently， clothes-changing ReID receives a lot of attention. In clothes-changing ReID， every person wears multiple outfits. The key point to clothes-changing ReID is to extract discriminative clothes-irrelevant features from images with different clothes. Usually， body shape can be used to identify people. Some existing methods extract body shape information from contour images but suffer from low image quality and poor robustness. To resolve these problems， we propose a sketch images-guided method for clothes-changing person re-identification method. There are two main approaches in existing methods： 1） extracting clothes-irrelevant information such as key points， pose and gait and fuse clothes-irrelevant information into person features. 2） Decoupling clothes-irrelevant and clothes-relevant feature using an encoder-decoder architecture.
Method
2
First， to improve the accuracy and robustness of body shape information， we propose to obtain more accurate and robust shape information from sketch images rather than contour images. Then we use an extra independently-trained network to extract the shape features of the person. Additionally， to reduce the clothes information in visual features and improve the discrimination of visual features， we propose a clothes-irrelevant weight guidance module based on sketch images. The module further uses the clothes position information in sketch images to guide the extraction process of visual features. With the guidance， the model can extract features with less clothes information. We use a two-stream network to fuse the shape feature and clothes-irrelevant apparent feature to get the complete person feature. We implement our method by using python and PyTorch. We train our network with one NVIDIA 3090 GPU device. We perform random horizontal flips and random erasing for augmentation. We use Adam optimizer and the learning rate is set to 0.000 35. The learning rate will decay per 20 batches.
Result
2
The performance of the proposed method is evaluated on two public clothes-changing dataset： long-term cloth changing （LTCC） and person re-identification under moderate clothing change （PRCC）. Our method overperforms the state-of-the-art methods on both two datasets. The proposed method obtains 38.0% Rank-1 and 15.9% mean average precision （mAP） on LTCC dataset； 55.5% Rank-1 and 52.6% mAP on PRCC dataset. The results of the ablation experiment demonstrate that the sketch images have their priority in robustness and accuracy compared with contour images. Visible results show that our proposed method can effectively weaken the model’s attention on the clothes area.
Conclusion
2
We propose a better way to extract body shape information and propose a sketch-based guidance module， which utilizes clothes-irrelevant information to wipe out clothing information in visual features. Experiments show that sketch images are superior to contour images in robustness and accuracy. Sketch images can provide more body shape information as a complement to visual features than contour images. The proposed clothes-irrelevant weight guidance module can effectively reduce clothing information in visual features. Our proposed sketch images-guided clothes-changing person re-identification method effectively extracts complete person features， which include clothing-irrelevant visual features and body shape features.
计算机视觉换装行人重识别素描图像表观特征体型特征双流网络
computer visionclothes-changing person re-identificationsketch imagesappearance featureshape featuretwo-stream network
Bhattarai M， Oyen D， Castorena J， Yang L P and Wohlberg B. 2020. Diagram image retrieval using sketch-based deep learning and transfer learning//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle， USA： IEEE： 174-175 ［DOI： 10.1109/CVPRW50498.2020.00095http://dx.doi.org/10.1109/CVPRW50498.2020.00095］
Chen J X， Jiang X Y， Wang F D， Zhang J， Zheng F， Sun X and Zheng W S. 2021. Learning 3D shape feature for texture-insensitive person re-identification//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville， USA： IEEE： 8142-8151 ［DOI： 10.1109/CVPR46437.2021.00805http://dx.doi.org/10.1109/CVPR46437.2021.00805］
Deng J， Dong W， Socher R， Li L J， Li K and Li F F. 2009. ImageNet： a large-scale hierarchical image database//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami， USA： IEEE： 248-255 ［DOI： 10.1109/CVPR.2009.5206848http://dx.doi.org/10.1109/CVPR.2009.5206848］
Eitz M， Hays J and Alexa M. 2012. How do humans sketch objects？ ACM Transactions on Graphics， 31（4）： #44 ［DOI： 10.1145/2185520.2185540http://dx.doi.org/10.1145/2185520.2185540］
Geirhos R， Rubisch P， Michaelis C， Bethge M， Wichmann F A and Brendel W. 2018. ImageNet-trained CNNs are biased towards texture； increasing shape bias improves accuracy and robustness//Proceedings of the 7th International Conference on Learning Representations （ICLR）. New Orleans， USA： OpenReview.net.： 1-10 ［DOI： 10.48550/arXiv.1811.12231http://dx.doi.org/10.48550/arXiv.1811.12231］
Gu X Q， Chang H， Ma B P， Bai S T， Shan S G and Chen X L. 2022. Clothes-changing person re-identification with RGB modality only//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans， USA： IEEE： 1060-1069 ［DOI： 10.1109/CVPR52688.2022.00113http://dx.doi.org/10.1109/CVPR52688.2022.00113］
Hertzmann A. 2020. Why do line drawings work？ A realism hypothesis. Perception， 49（4）： 439-451 ［DOI： 10.1177/030100662090-8207http://dx.doi.org/10.1177/030100662090-8207］
Hertzmann A. 2021. The role of edges in line drawing perception. Perception， 50（3）： 266-275 ［DOI： 10.1177/0301006621994407http://dx.doi.org/10.1177/0301006621994407］
Hong P X， Wu T， Wu A C， Han X T and Zheng W S. 2021. Fine-grained shape-appearance mutual learning for cloth-changing person re-identification//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville， USA： IEEE： 10508-10517 ［DOI： 10.1109/CVPR46437.2021.01037http://dx.doi.org/10.1109/CVPR46437.2021.01037］
Hou R B， Ma B P， Chang H， Gu X Q， Shan S G and Chen X L. 2019. Interaction-and-aggregation network for person re-identification//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach， USA： IEEE： 9309-9318 ［DOI： 10.1109/CVPR.2019.00954http://dx.doi.org/10.1109/CVPR.2019.00954］
Jia X M， Zhong X， Ye M， Liu W X and Huang W X. 2022. Complementary data augmentation for cloth-changing person re-identification. IEEE Transactions on Image Processing， 31： 4227-4239 ［DOI： 10.1109/TIP.2022.3183469http://dx.doi.org/10.1109/TIP.2022.3183469］
Jin X， He T Y， Zheng K C， Yin Z H， Xu S， Zhen H， Feng R Y， Huang J Q， Chen Z B， Hua X S. 2022. Cloth-changing person re-identification from a single image with gait prediction and regularization//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans， USA： IEEE： 14278-14287 ［DOI： 10.1109/CVPR52688.2022.01388http://dx.doi.org/10.1109/CVPR52688.2022.01388］
Li P K， Xu Y Q， Wei Y C and Yang Y. 2019. Self-correction for human parsing. IEEE Transactions on Pattern Analysis and Machine Intelligence， 44（6）： 3260-3271 ［DOI： 10.1109/TPAMI.2020.3048039http://dx.doi.org/10.1109/TPAMI.2020.3048039］
Li Y J， Weng X S and Kitani K M. 2021. Learning shape representations for person re-identification under clothing change//Proceedings of 2021 IEEE/CVF Winter Conference on Applications of Computer Vision. Waikoloa， USA： IEEE： 2431-2440 ［DOI： 10.1109/WACV48630.2021.00248http://dx.doi.org/10.1109/WACV48630.2021.00248］
Lorenz D， Bereska L， Milbich T and Ommer B. 2019. Unsupervised part-based disentangling of object shape and appearance//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach， USA： IEEE： 10947-10956 ［DOI： 10.1109/CVPR.2019.01121http://dx.doi.org/10.1109/CVPR.2019.01121］
Qian X L， Wang W X， Zhang L， Zhu F R， Fu Y W， Xiang T， Jiang Y G and Xue X Y. 2020. Long-term cloth-changing person re-identification//Proceedings of the 15th Asian Conference on Computer Vision. Kyoto， Japan： Springer： 71-88 ［DOI： 10.1007/978-3-030-69535-4_5http://dx.doi.org/10.1007/978-3-030-69535-4_5］
Shi W D， Zhang Y Z， Liu S W， Zhu S D and Pu J N. 2020. Person re-identification based on deformation and occlusion mechanisms. Journal of Image and Graphics， 25（12）： 2530-2540
史维东， 张云洲， 刘双伟， 朱尚栋， 暴吉宁. 2020. 针对形变与遮挡问题的行人再识别. 中国图象图形学报， 25（12）： 2530-2540 ［DOI： 10.11834/jig.200016http://dx.doi.org/10.11834/jig.200016］
Su Z， Liu W Z， Yu Z T， Hu D W， Liao Q， Tian Q， Pietikäinen M and Liu L. 2021. Pixel difference networks for efficient edge detection//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal， Canada： IEEE： 5117-5127 ［DOI： 10.1109/ICCV48922.2021.00507http://dx.doi.org/10.1109/ICCV48922.2021.00507］
Sun Y F， Zheng L， Yang Y， Tian Q and Wang S J. 2018. Beyond part models： person retrieval with refined part pooling （and a strong convolutional baseline）//Proceedings of the 15th European Conference on Computer Vision. Munich， Germany： Springer： 501-518 ［DOI： 10.1007/978-3-030-01225-0_30http://dx.doi.org/10.1007/978-3-030-01225-0_30］
Wang X N， Liu C H， Qi G Q and Zhang S Q. 2020. Person re-identification based on top-view depth head and shoulder sequence. Journal of Image and Graphics， 25（7）： 1393-1407
王新年， 刘春华， 齐国清， 张世强. 2020. 俯视深度头肩序列行人再识别. 中国图象图形学报， 25（7）： 1393-1407 ［DOI： 10.11834/jig.190608http://dx.doi.org/10.11834/jig.190608］
Wu A C， Lin C Z and Zheng W S. 2022. Single-modality self-supervised information mining for cross-modality person re-identification. Journal of Image and Graphics， 27（10）： 2843-2859
吴岸聪， 林城梽， 郑伟诗. 2022. 面向跨模态行人重识别的单模态自监督信息挖掘. 中国图象图形学报， 27（10）： 2843-2859 ［DOI： 10.11834/jig.211050http://dx.doi.org/10.11834/jig.211050］
Yan Y M， Yu H M， Li S Z， Lu Z H， He J F， Zhang H Z and Wang R F. 2022. Weakening the influence of clothing： universal clothing attribute disentanglement for person re-identification//Proceedings of the 31st International Joint Conference on Artificial Intelligence. Vienna， Austria： Morgan Kaufmann： 1523-1529 ［DOI： 10.24963/ijcai.2022/212http://dx.doi.org/10.24963/ijcai.2022/212］
Yang Q Z， Wu A C and Zheng W S. 2019. Person re-identification by contour sketch under moderate clothing change. IEEE Transactions on Pattern Analysis and Machine Intelligence， 43（6）： 2029-2046 ［DOI： 10.1109/TPAMI.2019.2960509http://dx.doi.org/10.1109/TPAMI.2019.2960509］
Yang Z W， Zhong X， Liu H， Zhong Z and Wang Z. 2022. Attentive decoupling network for cloth-changing re-identification//Proceedings of 2022 IEEE International Conference on Multimedia and Expo. Taipei， China： IEEE： 1-6 ［DOI： 10.1109/ICME52920.2022.9859851http://dx.doi.org/10.1109/ICME52920.2022.9859851］
Ye M， Shen J B， Lin G J， Xiang T， Shao L and Hoi S C H. 2022. Deep learning for person re-identification： a survey and outlook. IEEE Transactions on Pattern Analysis and Machine Intelligence， 44（6）： 2872-2893 ［DOI： 10.1109/TPAMI.2021.3054775http://dx.doi.org/10.1109/TPAMI.2021.3054775］
Yu Q， Liu F， Song Y Z， Xiang T， Hospedales T M and Loy C C. 2016. Sketch me that shoe//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Las Vegas， USA： IEEE： 799-807 ［DOI： 10.1109/CVPR.2016.93http://dx.doi.org/10.1109/CVPR.2016.93］
Yu Z X， Zhao Y L， Hong B， Jin Z M， Huang J Q， Cai D， He X F and Hua X S. 2020b. Apparel-invariant feature learning for apparel-changed person re-identification［EB/OL］. ［2020-08-17］. https：//arxiv.org/pdf/2008.06181.pdfhttps://arxiv.org/pdf/2008.06181.pdf
Zhang X， Luo H， Fan X， Xiang W L， Sun Y X， Xiao Q Q， Jiang W， Zhang C and Sun J. 2018. AlignedReID： surpassing human-level performance in person re-identification ［EB/OL］. ［2018-01-31］. https：//arxiv.org/pdf/1711.08184.pdfhttps://arxiv.org/pdf/1711.08184.pdf
Zheng Z D， Yang X D， Yu Z D， Zheng L， Yang Y and Kautz J. 2019. Joint discriminative and generative learning for person re-identification//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach， USA： IEEE： 2138-2147 ［DOI： 10.1109/CVPR.2019.00224http://dx.doi.org/10.1109/CVPR.2019.00224］
相关文章
相关作者
相关机构
京公网安备11010802024621