浏览全部资源
扫码关注微信
Published： 16 May 2023 ，
移动端阅览
张红颖， 王徐泳， 彭晓雯. 2023. 结合前景分割的多特征融合行人重识别. 中国图象图形学报， 28(05):1360-1371
Zhang Hongying， Wang Xuyong， Peng Xiaowen. 2023. Foreground segmentation-relevant multi-feature fusion person re-identification. Journal of Image and Graphics， 28(05):1360-1371
目的
2
行人重识别任务中，同一行人在不同图像中的背景差异会导致识别准确率下降，出现误识别的现象。针对此问题，提出了一种结合前景分割的多特征融合行人重识别方法。
方法
2
首先构建前景分割模块，提取图像的前景，并通过前景分割损失，保持前景图像的平滑性和完整性；然后提出了注意力共享策略和多尺度非局部运算方法，将图像中的全局特征与局部特征、高维特征与低维特征结合，实现不同特征之间的优势互补；最后通过多损失函数对网络模型进行训练优化。
结果
2
在3个公开数据集Market-1501、DukeMTMC-reID（Duke multi-tracking multi-camera re-identification）和MSMT17（multi-scene multi-time person ReID dataset）上进行了消融实验和对比实验，并以首位命中率（rank-1 accuracy，Rank-1）和平均精度均值（mean average precision，mAP）作为评价指标。实验结果显示，在引入前景分割和多特征融合方法时，网络的识别准确率均有一定提升。本文方法在Market-1501、DukeMTMC-reID和MSMT17数据集上Rank-1和mAP分别为96.8%和91.5%、91.5%和82.3%以及83.9%和63.8%，相比于对比算法，本文方法具有较大优势。
结论
2
本文提出的结合前景分割的多特征融合方法，在提取前景的同时，综合了不同尺度和不同粒度图像特征，相较于已有模型，提高了识别准确率。同时，前景分割模块消除了无用背景，缓解了背景差异导致的误识别现象，使行人重识别模型的实用性得到加强，在面对实际背景情况时，也能有较好的识别效果。
Objective
2
Person re-identification （ReID） is a computer vision-based cross camera recognition technology to target a pedestrian-specific in an image or video sequence. To obtain more discriminative features and achieve high accuracy， the deep learning based method of ReID is focused on processing personal features in recent years. The whole pedestrian image is often as the sample for the ReID model and each pixel feature is as the basis for recognition in the image. However， ReID， as a cross-camera recognition task， is required to deploy a wide range and number of camera locations， which will inevitably lead to background variations in pedestrian images. In other words， the heterogeneity of images-captured is challenged for the interrelations between background similarity and its identity for both of single and multiple cameras. Therefore， it is necessary to rich background information to the pedestrian similarity metric in the ReID model. To resolve this problem， we develop a foreground segmentation-based multi-branch joint person re-identification method in terms of the residual network 50 （ResNet50）.
Method
2
An integration of foreground segmentation and ReID method are employed. First， as the input for feature extraction， the foreground area of the pedestrian images is extracted by the foreground segmentation module. Then， to achieve mutual benefits between different features， the global features in the image are combined with local features and high-dimensional features with low-dimensional features using a multi-grain feature-guided branch and multi-scale feature fusion branch. For the foreground segmentation module， an attention mechanism is used to improve mask region-based convolutional neural network （Mask R-CNN）. And， the foreground segmentation loss function is adopted to optimize the feature information loss derived of rough segmentation of the foreground. For multi-grain feature branch， the convolutional block attention module （CBAM） is improved initially in terms of a three-branch attention structure. The information-interacted between two dimensions is based on adding new branches between the spatial and channel dimensions. Furthermore， an attention-sharing strategy is implemented. To improve the effectiveness of feature extraction and avoid feature chunking-derived missing information， the attention information is shared in coarse-grained branches with fine-grained branches， which can yield global features to guide the extraction of local features simply. For multi-scale feature fusion branch， the features at different stages backbone network feature extraction are used as the input of multi-scale fusion straightforward. Additionally， the pyramid attention structure is used as well to get the feature information before fusion. Next， for the fusion module， to synthesize the global information and alleviate the loss of feature information， a non-local algorithm is illustrated in multiscale for feature fusion. Finally， as a joint loss function， the loss-incorporated is in relevance with foreground segmentation， TriHard， and Softmax. And， it is used to train the network for optimization further.
Result
2
The comparative analysis is based on 3 publicly available datasets （Market-1501， Duke multi-tracking multi-camera re-identification（DukeMTMC-reID） and multi-scene multi-time person ReID dataset（MSMT17））. The metrics-related evaluations consist of rank-
n
accuracy （Rank-
n
） and mean average precision （mAP）. For the Market-1501 dataset， the Rank-1 and mAP can be reached 96.8% and 91.5% of each， which is a 0.6% improvement in Rank-1 and a 1% improvement in mAP compared to attention pyramid network （APNet-C）； For the DukeMTMC-reID dataset， the Rank-1 and mAP can be reached to 91.5% and 82.3% of each， as well as an improvement of 1.1% in Rank-1 and an improvement of 0.8% in mAP compared to the model APNet-C； For the MSMT17 dataset， the Rank-1 and mAP can be reached to 83.9% and 63.8% of each， which is increased by 0.2% in Rank-1 and 0.3% in mAP compared to APNet-C；
Conclusion
2
We facilitate a foreground segmentation based multi-branch joint model. It can be focused on foreground extraction-based in accordance with an integration of multiple scales and grains image features. At the same time， the foreground segmentation module can wipe out ineffective background and alleviate the background-differentiated false recognition more.
前景分割语义分割行人重识别（ReID）特征融合注意力机制
foreground segmentationsemantic segmentationperson re-identification（ReID）feature fusionattention mechanism
Benzine A， El Amine Seddik M and Desmarais J. 2021. Deep miner： a deep and multi-branch network which mines rich and diverse features for person re-identification ［EB/OL］. ［2022-07-07］. https： //arxiv.org/pdf/2102.09321.pdfhttps://arxiv.org/pdf/2102.09321.pdf
Chen G Y， Gu T P， Lu J W， Bao J A and Zhou J. 2021. Person re-identification via attention pyramid. IEEE Transactions on Image Processing， 30： 7663-7676 ［DOI： 10.1109/TIP.2021.3107211http://dx.doi.org/10.1109/TIP.2021.3107211］
Chen T L， Ding S J， Xie J Y， Yuan Y， Chen W Y， Yang Y， Ren Z and Wang Z Y. 2019. ABD-Net： attentive but diverse person re-identification//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision （ICCV）. Seoul， Korea （South）： IEEE： 8350-8360 ［DOI： 10.1109/ICCV.2019.00844http://dx.doi.org/10.1109/ICCV.2019.00844］
Chong Y W， Zhang C， Feng W Q and Pan S M. 2022. Person re-identification based on multi-level and generated alignment network. Journal of Huazhong University of Science and Technology （Natural Science Edition）， 50（4）： 64-70
种衍文， 章郴， 冯文强， 潘少明. 2022. 基于多粒度生成对齐网络的行人重识别. 华中科技大学学报（自然科学版）， 50（4）： 64-70 ［DOI： 10.13245/j.hust.220411http://dx.doi.org/10.13245/j.hust.220411］
Deng J H， Hao Y， Khokhar M S， Kumar R， Cai J Y， Kumar J and Aftab M U. 2021. Trends in vehicle re-identification past， present， and future： a comprehensive review. Mathematics， 9（24）： #3162 ［DOI： 10.3390/math9243162http://dx.doi.org/10.3390/math9243162］
Ding C X， Wang K， Wang P F and Tao D C. 2022. Multi-task learning with coarse priors for robust part-aware person re-identification. IEEE Transactions on Pattern Analysis and Machine Intelligence， 44（3）： 1474-1488 ［DOI： 10.1109/TPAMI.2020.3024900http://dx.doi.org/10.1109/TPAMI.2020.3024900］
Gong Y X. 2020. Research on Person Re-identification Method Based on Foreground Segmentation and Multi-loss Ensemble. Hefei： Hefei University of Technology
龚毓秀. 2020. 基于前景分割与多损失融合的行人重识别方法研究. 合肥： 合肥工业大学
Gong Y X， Wang R G， Yang J， Xue L X and Hu M. 2021. Person re-identification with global-local background_bias net. Journal of Visual Communication and Image Representation， 74： #102961 ［DOI： 10.1016/j.jvcir.2020.102961http://dx.doi.org/10.1016/j.jvcir.2020.102961］
He K M， Gkioxari G， Dollár P and Girshick R. 2017. Mask R-CNN//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice， Italy： IEEE： 2980-2988 ［DOI： 10.1109/ICCV.2017.322http://dx.doi.org/10.1109/ICCV.2017.322］
Huang Y， Wu Q， Xu J S and Zhong Y. 2019. SBSGAN： suppression of inter-domain background shift for person re-identification//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul， Korea （South）： IEEE： 9526-9535 ［DOI： 10.1109/ICCV.2019.00962http://dx.doi.org/10.1109/ICCV.2019.00962］
Li Q， Hu W Y， Li J Y， Liu Y and Li M X. 2022. A survey of person re-identification based on deep learning. Chinese Journal of Engineering， 44（5）： 920-932
李擎， 胡伟阳， 李江昀， 刘艳， 李梦璇. 2022. 基于深度学习的行人重识别方法综述. 工程科学学报， 44（5）： 920-932 ［DOI： 10.13374/j.issn2095-9389.2020.12.22.004http://dx.doi.org/10.13374/j.issn2095-9389.2020.12.22.004］
Li Y F， Zhang B， Sun J， Chen H J and Zhu J L. 2020. Cross-dataset person re-identification method based on multi-pool fusion and background elimination network. Journal on Communications， 41（10）： 70-79
李艳凤， 张斌， 孙嘉， 陈后金， 朱锦雷. 2020. 基于多池化融合与背景消除网络的跨数据集行人再识别方法. 通信学报， 41（10）： 70-79 ［DOI： 10.11959/j.issn.1000-436x.2020181http://dx.doi.org/10.11959/j.issn.1000-436x.2020181］
Liu M J， Zhao J Q， Zhou Y， Zhu H C， Yao R and Chen Y. 2022. Survey for person re-identification based on coarse-to-fine feature learning. Multimedia Tools and Applications， 81（15）： 21939-21973 ［DOI： 10.1007/s11042-022-12510-1http://dx.doi.org/10.1007/s11042-022-12510-1］
Liu Z G， Huang Z， Xie D J， Tian F and Li T Y. 2022. Person re-identification for suppressing background interference. Journal of Computer-Aided Design and Computer Graphics， 34（4）： 563-569
刘志刚， 黄朝， 谢东军， 田枫， 李婷玉. 2022. 抑制背景干扰的行人重识别方法. 计算机辅助设计与图形学学报， 34（4）： 563-569
Martinel N， Foresti G L and Micheloni C. 2019. Aggregating deep pyramidal representations for person re-identification//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）. Long Beach， USA： IEEE： 1544-1554 ［DOI： 10.1109/CVPRW.2019.00196http://dx.doi.org/10.1109/CVPRW.2019.00196］
Misra D， Nalamada T， Arasanipalai A U and Hou Q B. 2021. Rotate to attend： convolutional triplet attention module//Proceedings of 2021 IEEE Winter Conference on Applications of Computer Vision. Waikoloa， USA： IEEE： 3138-3147 ［DOI： 10.1109/WACV48630.2021.00318http://dx.doi.org/10.1109/WACV48630.2021.00318］
Quispe R and Pedrini H. 2019. Improved person re-identification based on saliency and semantic parsing with deep neural network models. Image and Vision Computing， 92： #103809 ［DOI： 10.1016/j.imavis.2019.07.009http://dx.doi.org/10.1016/j.imavis.2019.07.009］
Rao Y M， Chen G Y， Lu J W and Zhou J. 2021. Counterfactual attention learning for fine-grained visual categorization and re-identification//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal， Canada： IEEE： 1005-1014 ［DOI： 10.1109/ICCV48922.2021.00106http://dx.doi.org/10.1109/ICCV48922.2021.00106］
Ristani E， Solera F， Zou R， Cucchiara R and Tomasi C. 2016. Performance measures and a data set for multi-target， multi-camera tracking//Proceedings of 2016 European Conference on Computer Vision. Amsterdam， the Netherlands： Springer： 17-35 ［DOI： 10.1007/978-3-319-48881-3_2http://dx.doi.org/10.1007/978-3-319-48881-3_2］
Shen Q， Tian C， Wang J B， Jiao S S and Du L. 2020. Multi-resolution feature attention fusion method for person re-identification. Journal of Image and Graphics， 25（5）： 946-955
沈庆， 田畅， 王家宝， 焦珊珊， 杜麟. 2020. 多分辨率特征注意力融合行人再识别. 中国图象图形学报， 25（5）： 946-955 ［DOI： 10.11834/jig.19023http://dx.doi.org/10.11834/jig.19023］
Song X R， Yang J， Gao S， Chen C B and Song S. 2022. Person re-identification method based on attention mechanism and multi-scale feature fusion. Science Technology and Engineering， 22（4）： 1526-1533
宋晓茹， 杨佳， 高嵩， 陈超波， 宋爽. 2022. 基于注意力机制与多尺度特征融合的行人重识别方法. 科学技术与工程， 22（4）： 1526-1533 ［DOI： 10.3969/j.issn.1671-1815.2022.04.029http://dx.doi.org/10.3969/j.issn.1671-1815.2022.04.029］
Tian M Q， Yi S， Li H S， Li S H， Zhang X S， Shi J P， Yan J J and Wang X G. 2018. Eliminating background-bias for robust person re-identification//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City， USA： IEEE： 5794-5803 ［DOI： 10.1109/CVPR.2018.00607http://dx.doi.org/10.1109/CVPR.2018.00607］
Wang G S， Yuan Y F， Chen X， Li J W and Zhou X. 2018. Learning discriminative features with multiple granularities for person re-identification//Proceedings of the 26th ACM International Conference on Multimedia. Seoul， Korea （South）： ACM： 274-282 ［DOI： 10.1145/3240508.3240552http://dx.doi.org/10.1145/3240508.3240552］
Wei L H， Zhang S L， Gao W and Tian Q. 2018. Person transfer GAN to bridge domain gap for person re-identification//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City， USA： IEEE： 79-88 ［DOI： 10.1109/CVPR.2018.00016http://dx.doi.org/10.1109/CVPR.2018.00016］
Zhang H Y and Bao W J. 2022. The cross-view gait recognition analysis based on generative adversarial networks derived of self-attention mechanism. Journal of Image and Graphics， 27（4）： 1097-1109
张红颖， 包雯静. 2022. 融合自注意力机制的生成对抗网络跨视角步态识别. 中国图象图形学报， 27（4）： 1097-1109 ［DOI： 10.11834/jig.200482http://dx.doi.org/10.11834/jig.200482］
Zhang S F， Yin Z R， Wu X， Wang K， Zhou Q and Kang B. 2021. FPB： feature pyramid branch for person re-identification［EB/OL］. ［2022-07-07］. https： //arxiv.org/pdf/2108.01901.pdfhttps://arxiv.org/pdf/2108.01901.pdf
Zheng L， Shen L Y， Tian L， Wang S J， Wang J D and Tian Q. 2015. Scalable person re-identification： a benchmark//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago， Chile： IEEE： 1116-1124 ［DOI： 10.1109/ICCV.2015.133http://dx.doi.org/10.1109/ICCV.2015.133］
Zheng X， Lin L， Ye M， Wang L and He C L. 2020. Improving person re-identification by attention and multi-attributes. Journal of Image and Graphics， 25（5）： 936-945
郑鑫， 林兰， 叶茂， 王丽， 贺春林. 2020. 结合注意力机制和多属性分类的行人再识别. 中国图象图形学报， 25（5）： 936-945 ［DOI： 10.11834/jig.19018http://dx.doi.org/10.11834/jig.19018］
相关文章
相关作者
相关机构
京公网安备11010802024621