浏览全部资源
扫码关注微信
纸质出版日期： 2023-02-16 ，
录用日期： 2021-12-23
移动端阅览
乐应英, 徐丹, 贺康建, 张浩. 低视点下遮挡自适应感知的多目标跟踪算法[J]. 中国图象图形学报, 2023,28(2):441-457.
Yingying Yue, Dan Xu, Kangjian He, Hao Zhang. An adaptive occlusion-aware multiple targets tracking algorithm for low viewpoint[J]. Journal of Image and Graphics, 2023,28(2):441-457.
目的
2
针对低视点多目标跟踪场景的遮挡问题，提出一种能够遮挡自适应感知的多目标跟踪算法。
方法
2
首先根据每帧图像的全局遮挡状态，提出了“自适应抗遮挡特征”，增强目标特征对遮挡的感知和调整能力。同时，采用“级联筛查机制”，减少由遮挡带来的目标特征剧烈变化而认定为“虚新入目标”的错误跟踪现象。最后，考虑到历史模板库中存在遮挡的模板对跟踪性能的影响，根据每一帧中目标的局部遮挡状态，提出自适应干扰模板更新机制，进一步提高对遮挡的应变和适应能力。
结果
2
实验结果表明，本文算法在MOTA（multiple object tracking accuracy）、MOTP（multiple object tracking precision）、FN（false negatives）、Rcll（recall）、ML（mostly lost tracklets）等指标上明显优于STAM（spatial-temporal attention mechanism）、ATAF（aggregate tracklet appearance features）、STRN（spatial-temporal relation network）、BLSTM_MTP_O（bilinear long short-term memory with multi-track pooling）、IADMR（instance-aware tracker and dynamic model refreshment）等典型算法。消融实验表明，自适应抗遮挡特征在MOTA指标上，相比混合特征、外观特征和运动特征分别提升了1.9%、1.8%和13.6%。去干扰模板更新策略在MOTA指标上，相比带权更新策略和常规更新策略分别提升了10.7%和17.7%。
结论
2
本文算法在低视点跟踪场景下，能够减弱部分遮挡、短时全遮挡和长时全遮挡对跟踪性能的影响，跟踪鲁棒性得到了提升。
Objective
2
Multi-target tracking technique is essential for the computer vision-relevant applications like video surveillance
smart cities
and intelligent public transportation. The task of multi-target tracking is required to better location for multiple targets of each frame through the context information of the video sequence. To generate the motion trajectory of each target
its identity information (ID) is required to keep in consistency. So
we focus on low viewpoint-based multi-target tracking with no high viewpoint involved. For low viewpoint tracking scenes
the occlusion can be as a key factor to optimize tracking performance. The occlusion-completed is restricted by the target-captured issues temporarily
which is challenged for target tracking. The partial-occluded target is still challenged to be captured because the visual information of the occluded target is contaminated and the extracted target features are incomplete
and it will cause tracking drift as well.
Method
2
To resolve occlusion problem
we develop a low viewpoint-based adaptive occlusion-relevant multiple targets tracking algorithm. The proposed algorithm is composed of three main aspects as following: 1) An adaptive anti-occlusion feature is illustrated in terms of the occlusion degree of each frame. To enhance its adaptability for occlusion
global occlusion information is used to adjust feature-related structure dynamically. 2) When the occlusion occurs
the target will disappear temporarily. When it reappears again after occlusion
it is often transferred to a new target and the tracking ID switch occurs. Therefore
a cascade screening mechanism is melted into for new target problem-identified. Due to the intensive change of occlusion-based target features
high-level and low-level features are employed both to prevent the virtual phenomenon for new target. 3) A large amount of target-occluded noise will be introduced into the template library if they are updated into the template library with no clarification. Therefore
an adaptive anti-interference template update mechanism is proposed for that. Multiple weights are given to the target templates-profiled of different occlusion states based on the local occlusion information of all targets
and the weights-based adaptive template-updated is then performed
which can alleviate the interference of severe-occluded targets to the template library.
Result
2
Our algorithm is experimented on the low viewpoint tracking videos-selected of MOT16
which includes special tracking scenes like 1) partial occlusion
2) short-term full occlusion
and 3) long-term full occlusion. The experimental results show that the tracking performance of our algorithm has been improved
achieve improvement of 3.67%
1.57%
2.77%
5.71%
and 3.07% on MOTA (multiple object tracking accuracy) respectively than STAM (spatial-temporal attention mechanism)
ATAF (aggregate tracklet appearance features)
STRN (spatial-temporal relation network)
BLSTM_MTP_O (bilinear long short-term memory with multi-track pooling) and IADMR (instance-aware tracker and dynamic model refreshment). Furthermore
the ablation experiment shows that our anti-occlusion feature proposed can achieve an increase of 1.9% compared to the hybrid feature
an improvement of 1.8% compared to the appearance feature
and an optimization of 13.6% compared to the motion feature on MOTA. Compared with the weighted update strategy
the adaptive anti-interference update strategy proposed has achieved an improvement of 10.7% on MOTA
and an improvement of 17.7% compared with the conventional update strategy. Moreover
compared with the weighted update strategy
the number of ID switching times is significantly reduced from 244 to 119
which shows that our anti-interference update strategy can optimize the cleanliness of the template library and the accuracy of data association. Additionally
to validate the effectiveness of the update strategy we proposed
more indicators are improved obviously
such as Rcll (recall)
FN (false negatives)
MT (mostly lost tracklets)
ML (mostly lost tracklets)
and Frag (fragments).
Conclusion
2
The low viewpoint-based adaptive occlusion-relevant multiple targets tracking algorithm can be used to enhance the perception and balancing capabilities of the features-used in data association
reduces the impact of severe-occluded target templates beyond template library-profiled on the multi-tracking performance. Limitation and recommendation our proposed algorithm have no motion and speed-related estimation-specific mechanism for the rigid motion of the camera. Our data association-based algorithm is still cohesive to target detection algorithm severely. Therefore
the trajectory has to be disturbed and crossed when the target is missed or falsely detected. The future work can be focused on improving the tracking adaptability to actual tracking scenarios and the immunity of detection errors further.
多目标跟踪低视点遮挡抗遮挡特征数据关联模板更新
multiple targets trackinglow viewpointocclusionanti-occlusion featuredata associationtemplate update
Bae S H and Yoon K J. 2014. Robust online multi-object tracking based on tracklet confidence and online discriminative appearance learning//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE: 1218-1225 [DOI: 10.1109/CVPR.2014.159http://dx.doi.org/10.1109/CVPR.2014.159]
Berclaz J, Fleuret F, Türetken E and Fua P. 2011. Multiple object tracking using k-shortest paths optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(9): 1806-1819 [DOI: 10.1109/TPAMI.2011.21]
Bewley A, Ge Z Y, Ott L, Ramos F and Upcroft B. 2016. Simple online and realtime tracking//Proceedings of 2016 IEEE International Conference on Image Processing (ICIP). Phoenix, USA: IEEE: 3464-3468 [DOI: 10.1109/ICIP.2016.7533003http://dx.doi.org/10.1109/ICIP.2016.7533003]
Brendel W, Amer M and Todorovic S. 2011. Multiobject tracking as maximum weight independent set//Proceedings of 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Colorado Springs, USA: IEEE: 1273-1280 [DOI: 10.1109/CVPR.2011.5995395http://dx.doi.org/10.1109/CVPR.2011.5995395]
Chen L, Ai H Z, Chen R and Zhuang Z J. 2019. Aggregate tracklet appearance features for multi-object tracking. IEEE Signal Processing Letters, 26(11): 1613-1617 [DOI: 10.1109/LSP.2019.2940922]
Chu P, Fan H, Tan C C and Ling H B. 2019. Online multi-object tracking with instance-aware tracker and dynamic model refreshment//Proceedings of 2019 IEEE Winter Conference on Applications of Computer Vision. Waikoloa, USA: IEEE: 161-170 [DOI: 10.1109/WACV.2019.00023http://dx.doi.org/10.1109/WACV.2019.00023]
Chu Q, Ouyang W L, Li H S, Wang X G, Liu B and Yu N H. 2017. Online multi-object tracking using CNN-based single object tracker with spatial-temporal attention mechanism//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE: 4846-4855 [DOI: 10.1109/ICCV.2017.518http://dx.doi.org/10.1109/ICCV.2017.518]
Dehghan A, Assari S M and Shah M. 2015. GMMCP tracker: Globally optimal Generalized Maximum Multi Clique problem for multiple object tracking//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE: 4091-4099 [DOI: 10.1109/CVPR.2015.7299036http://dx.doi.org/10.1109/CVPR.2015.7299036]
Dendorfer P, Ošep A, Milan A, Schindler K, Cremers D, Reid I, Roth S and Leal-Taixé L. 2021. MOTChallenge: a benchmark for single-camera multiple target tracking. International Journal of Computer Vision, 129(4): 845-881 [DOI: 10.1007/s11263-020-01393-0]
Fang L and Yu F Q. 2020. Multi-object tracking based on adaptive online discriminative appearance learning and hierarchical association. Journal of Image and Graphics, 25(4): 708-720
方岚, 于凤芹. 2020. 自适应在线判别外观学习的分层关联多目标跟踪. 中国图象图形学报, 25(4): 708-720 [DOI: 10.11834/jig.190320]
He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 770-778 [DOI: 10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90]
Izadinia H, Saleemi I, Li W H and Shah M. 2012. (MP)2T: multiple people multiple parts tracker//Proceedings of the 12th European Conference on Computer Vision. Florence, Italy: Springer: 100-114 [DOI: 10.1007/978-3-642-33783-3_8http://dx.doi.org/10.1007/978-3-642-33783-3_8].
Kim C, Li F X, Alotaibi M and Rehg J M. 2021. Discriminative appearance modeling with multi-track pooling for real-time multi-object tracking//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, USA: IEEE: 9548-9557 [DOI: 10.1109/cvpr46437.2021.00943http://dx.doi.org/10.1109/cvpr46437.2021.00943]
Li M Y. 2020. Research on Key Technologies of Real-Time Multiple Object Tracking Based on Deep Learning. Changchun: Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences
李沐雨. 2020. 基于深度学习的实时多目标跟踪关键技术的研究. 长春: 中国科学院长春光学精密机械与物理研究所
Liu P X. 2020. Research on Key Technologies of Video Multiple Object Tracking Based on Data Association. Chengdu: University of Electronic Science and Technology of China
刘沛鑫. 2020. 基于数据关联的视频多目标跟踪关键技术研究. 成都: 电子科技大学
Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C Y and Berg A C. 2016. SSD: single shot MultiBox detector//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, the Netherlands: Springer: 21-37 [DOI: 10.1007/978-3-319-46448-0_2http://dx.doi.org/10.1007/978-3-319-46448-0_2]
Milan A, Roth S and Schindler K. 2014. Continuous energy minimization for multitarget tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(1): 58-72 [DOI: 10.1109/TPAMI.2013.103]
Pirsiavash H, Ramanan D and Fowlkes C C. 2011. Globally-optimal greedy algorithms for tracking a variable number of objects//Proceedings of 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Colorado Springs, USA: IEEE: 1201-1208 [DOI: 10.1109/CVPR.2011.5995604http://dx.doi.org/10.1109/CVPR.2011.5995604]
Redmon J, Divvala S, Girshick R and Farhadi A. 2016. You only look once: unified, real-time object detection//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 779-788 [DOI: 10.1109/CVPR.2016.91http://dx.doi.org/10.1109/CVPR.2016.91]
Redmon J and Farhadi A. 2018. YOLOv3: an incremental improvement. [EB/OL]. [2018-04-08].https://arxiv.org/pdf/1804.02767.pdfhttps://arxiv.org/pdf/1804.02767.pdf
Ren S Q, He K N, Girshick R and Sun J. 2017. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6): 1137-1149 [DOI: 10.1109/TPAMI.2016.2577031]
Roshan Z A, Dehghan A and Shah M. 2012. GMCP-tracker: global multi-object tracking using generalized minimum clique graphs//Proceedings of the 12th European Conference on Computer Vision. Florence, Italy: Springer: 343-356 [DOI: 10.1007/978-3-642-33709-3_25http://dx.doi.org/10.1007/978-3-642-33709-3_25]
Shu G, Dehghan A, Oreifej O, Hand E and Shah M. 2012. Part-based multiple-person tracking with partial occlusion handling//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, USA: IEEE: 1815-1821 [DOI: 10.1109/CVPR.2012.6247879http://dx.doi.org/10.1109/CVPR.2012.6247879]
Tang S Y, Andriluka M and Schiele B. 2014. Detection and tracking of occluded people. International Journal of Computer Vision, 110(1): 58-69 [DOI: 10.1007/s11263-013-0664-6]
Wang X Q, Jiang J G and Qi M B. 2017. Hierarchical multi-object tracking algorithm based on globally multiple maximum clique graphs. Journal of Image and Graphics, 22(10): 1401-1408
王雪琴, 蒋建国, 齐美彬. 2017. 全局多极团的分层关联多目标跟踪. 中国图象图形学报, 22(10): 1401-1408 [DOI: 10.11834/jig.160527]
Wojke N, Bewley A and Paulus D. 2017. Simple online and realtime tracking with a deep association metric//Proceedings of 2017 IEEE International Conference on Image Processing. Beijing, China: IEEE: 3645-3649 [DOI: 10.1109/ICIP.2017.8296962http://dx.doi.org/10.1109/ICIP.2017.8296962]
Xu J R, Cao Y, Zhang Z and Hu H. 2019. Spatial-temporal relation networks for multi-object tracking//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul, Korea (South): IEEE: 3987-3997 [DOI: 10.1109/ICCV.2019.00409http://dx.doi.org/10.1109/ICCV.2019.00409]
Zhang L, Li Y and Nevatia R. 2008. Global data association for multi-object tracking using network flows//Proceedings of 2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, USA: IEEE: 1-8 [DOI: 10.1109/CVPR.2008.4587584http://dx.doi.org/10.1109/CVPR.2008.4587584]
相关作者
相关机构
京公网安备11010802024621