浏览全部资源
扫码关注微信
Received：03 July 2019，
Revised：2019-9-22，
Accepted：29 September 2019，
Published：16 April 2020
移动端阅览
目的
2
复杂场景下目标频繁且长时间的遮挡、跟踪目标外观相似引起身份转换等问题给多目标跟踪带来许多挑战。针对多目标跟踪在复杂场景中因长时间遮挡引起身份转换和轨迹分段的问题，提出一种基于自适应在线判别外观学习的分层关联多目标跟踪算法。
方法
2
利用轨迹置信度将多目标跟踪分为局部关联和全局关联两个层次。在局部关联中，置信度高的可靠轨迹利用外观、位置-大小相似度与当前帧检测点进行关联；在全局关联中，置信度低的不可靠轨迹引入运动模型和有效关联范围进一步关联分段的轨迹。在提取目标外观特征时引入增量线性可判别分析方法以解决身份转换问题，依据新增样本与目标样本均值的外观特征差异自适应地更新目标外观模型。
结果
2
在公开数据集2D MOT2015中的PETS09-S2L1、TUD-Stadmitte、Town-Center 3个数据集中与当前10种多目标跟踪算法进行比较，该方法对各个数据集身份转换和轨迹分段都有减少，其中在Town-Center数据集中，身份转换减少了60个，轨迹分段减少了84个，跟踪准确度提高了5.2%以上。
结论
2
本文多目标跟踪方法，能够在复杂场景中稳定有效地实现多目标跟踪，减少轨迹分段现象，其中引入的在线线性可判别外观学习对遮挡产生的身份转换具有良好的解决效果。
Objective
2
Multi-object tracking is an important research topic in computer vision. Although several previous studies have dealt with varieties of particular problems in multi-object tracking
many challenges are still observed
such as object detection errors
missed detection
frequent and long-term occlusion of objects in complex scenes
and identity switches of tracking objects with similar appearance. All of which are easy to lead to trajectory drift or tracking interruption. With the improvement of object detection
the object tracking method based on detection shows good performance. The key of tracking-by-detection algorithm is the data association between detection points
which mainly consists of two types
namely
frame-by-frame association and multi-frame association. Frame-by-frame data association refers to the association between detection points in the two consecutive frames
which is carried out according to the properties of detection points
such as appearance
location
and size. Tracking drift or failure is likely to occur when object is blocked
misdetected or similar appearance exist due to that the frame-by-frame data association only contains the information of the previous two frames. Multi-frame data association establishes a relational model by using object detection information of multiple frames rather than only previous two frames. This condition can effectively reduce the object error association and deal with occlusion. However
if the occlusion time is longer than the time segment needed for multi-frame data association
the detection points before and after still cannot be successfully associated
and the tracking will also be interrupted. Moreover
this method needs all detection information before tracking
which cannot meet the real-time requirement. Aiming at the problems of ID switches and trajectory fragmentation caused by long-term occlusion
an online multi-object tracking algorithm based on adaptive online discriminative appearance learning and hierarchical association is proposed for multi-object tracking in complex scenes. This process combines the low-level appearance
position-size characteristics used in local association
and high-level motion model established in global association and can meet the real-time tracking requirement.
Method
2
In this study
multi-object tracking is divided into two stages according to track confidence:local association and global association. The establishment of the object robust appearance model is the key to local association and global association. An online incremental linear discriminant analysis(ILDA) method was introduced to discriminate the appearances of objects and adaptively update the object appearance models based on the difference value between the new sample and the mean of object samples to address the problem of identity switches. The reliable tracklet with high confidence in the local association stage is associated with the current frame detections by low-level properties of detection points:appearance and position-size similarity
which allows reliable trajectories to grow constantly. The unreliable tracklet with low confidence in the global association stage resulted from long-term occlusion is further associated. In this stage
the candidate object consists of two kinds. One is the detection points that are not associated in local association
and the other one is continuous trajectory with high confidence meeting the time condition. The end time of trajectory is before the current time. When we associate detection points that reappear after long-term occlusion
only appearance similarity is utilized within a validation range without the position-size property due to the unreliable motion dynamics of unreliable objects. At the same time
introducing a valid association range is related to the trajectory confidence. Once the track confidence is reduced
the valid association range is increased because the distance between a drifting track and the corresponding object can grow large if the track drift persists. This condition allows us to reassign drifting tracks to detections of reappearing objects
which is even distant from the corresponding tracks. When two track fragments are associated
a motion model is introduced to determine whether the two trajectories belong to the same object. In this condition
the average velocity vector angle of the two track fragments is larger than a threshold
indicating that it may include unreliable tracks. Thus
we only consider appearance similarity between the pair. Otherwise
we combine the appearance
position size
and motion similarity to make an association between the pair. If two track fragments are associated successfully
the linear interpolation is used to fill the lost interval of this object. Thus
the two trajectory fragments can be connected effectively.
Result
2
We compared our method with 10 state-of-the-art multi-object tracking algorithms
including five offline tracking approaches and five online tracking methods on three public datasets
namely
PETS09-S2L1
TUD-Stadmitte
and Town-Center. The quantitative evaluation metrics contained multi-object tracking accuracy (MOTA)
multi-object tracking precision (MOTP)
the number of identity switches (IDS)
the ratio of mostly tracked trajectories (MT)
the ratio of mostly lost trajectories (ML)
and the number of track fragmentation (Frag). The experiment results illustrate that our tracking method outperforms in MOTA and MOTP compared with selected online multi-object tracking methods
which include two tracking approaches based on hierarchical association. In addition
the proposed approach performs almost the same or even better when compared with offline tracking methods. In the PETS09-S2L1 data set
the proposed approaches are superior to other comparators in MOTP
IDS
and Frag. MOTP increased by 6.1%
IDS reduced by 5
and Frag reduced by 21. In TUD-Stadmitte dataset
IDS reduced by 4. Compared with online tracking approaches
the MOTP and MOTA increased by 36.3% and 11.1%
respectively. In Town-Center dataset
MOTA and MT increased by 5.2% and 16.9%
respectively. IDS and Frag reduced by 60 and 84
respectively
and ML decreased by 1.5%.
Conclusion
2
In this study
we take the idea of hierarchical data association
proposing a multi-object tracking based on adaptive online discriminative appearance learning and hierarchical association. The experiment results indicate that our method has a good solution to the problems of ID switches and trajectory fragmentation caused by long-term occlusion in complex scenes.
Bernardin K and Stiefelhagen R. 2008. Evaluating multiple object tracking performance:the clear mot metrics. EURASIP Journal on Image and Video Processing, 2008(1):1-10[DOI:10.1155/2008/246309]
Dollár P, Appel R, Belongie S and Perona P. 2014. Fast feature pyramids for object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(8):1532-1545[DOI:10.1109/TPAMI.2014.2300479]
Huang C, Li Y and Nevatia R. 2013. Multiple target tracking by learning-based hierarchical association of detection responses. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(4):898-910[DOI:10.1109/TPAMI.2012.159]
Kalman R E. 1960. A New approach to linear filtering and prediction problems. Journal of Basic Engineering Transactions, 82(1):35-45[DOI:10.1115/1.3662552]
Khan Z, Balch T and Dellaert F. 2005. MCMC-based particle filtering for tracking a variable number of interacting targets. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(11):1805-1819[DOI:10.1109/TPAMI.2005.223]
Kuhn H W. 1955. The Hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2(1/2):83-97[DOI:10.1002/nav.3800020109]
Li Y, Huang C and Nevatia R. 2009. Learning to associate: hybrid boosted multi-target tracker for crowded scene//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, FL, USA: IEEE: 2953-2960[ DOI: 10.1109/CVPR.2009.5206735 http://dx.doi.org/10.1109/CVPR.2009.5206735 ]
McLaughlin N, Del Rincon J M and Miller P. 2013. Online multiperson tracking with occlusion reasoning and unsupervised track motion model//Proceedings of the 10th IEEE International Conference on Advanced Video and Signal Based Surveillance. Krakow, Poland: IEEE: 37-42[ DOI: 10.1109/AVSS.2013.6636613 http://dx.doi.org/10.1109/AVSS.2013.6636613 ]
Milan A, Roth S and Schindler K. 2014. Continuous energy minimization for multitarget tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(1):58-72[DOI:10.1109/tpami.2013.103]
Pang S N, Ozawa S and Kasabov N. 2005. Incremental linear discriminant analysis for classification of data streams. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 35(5):905-914[DOI:10.1109/TSMCB.2005.847744]
Pirsiavash H, Ramanan D and Fowlkes C C. 2011. Globally-optimal greedy algorithms for tracking a variable number of objects//Proceedings of 2011 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI, USA: IEEE: 1201-1208[ DOI: 10.1109/CVPR.2011.5995604 http://dx.doi.org/10.1109/CVPR.2011.5995604 ]
Possegger H, Mauthner T, Roth P M and Bischof H. 2014. Occlusion geodesics for online multi-object tracking//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA: IEEE: 1306-1313[ DOI: 10.1109/CVPR.2014.170 http://dx.doi.org/10.1109/CVPR.2014.170 ]
Qi M B, Yue Z L, Shu K and Jiang J G. 2017. Multi-object tracking using hierarchical data association based on generalized correlation clustering graphs. Acta Automatica Sinica, 43(1):152-160
齐美彬, 岳周龙, 疏坤, 蒋建国. 2017.基于广义关联聚类图的分层关联多目标跟踪.自动化学报, 43(1):152-160[DOI:10.16383/j.aas.2017.c150519]
Shu G, Dehghan A, Oreifej O, Hand E and Shah M. 2012. Part-based multiple-person tracking with partial occlusion handling//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI, USA: IEEE: 1815-1821[ DOI: 10.1109/CVPR.2012.6247879 http://dx.doi.org/10.1109/CVPR.2012.6247879 ]
Tesfaye Y T, Zemene E, Pelillo M and Prati A. 2016. Multi-object tracking using dominant sets. IET Computer Vision, 10(4):289-297[DOI:10.1049/iet-cvi.2015.0297]
Wang B, Wang L, Shuai B, Zuo Z, Liu T, Chan K L and Wang G. 2016. Joint learning of convolutional neural networks and temporally constrained metrics for tracklet association//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Las Vegas, NV, USA: IEEE: 386-393[ DOI: 10.1109/CVPRW.2016.55 http://dx.doi.org/10.1109/CVPRW.2016.55 ]
Wu B and Nevatia R. 2006. Tracking of multiple, partially occluded humans based on static body part detection//Proceedings of 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York, USA: IEEE: 951-958[ DOI: 10.1109/CVPR.2006.312 http://dx.doi.org/10.1109/CVPR.2006.312 ]
Yamaguchi K, Berg A C, Ortiz L E and Berg T L. 2011. Who are you with and where are you going?//Proceedings of 2011 IEEE Conference on Computer Vision and Pattern Recognition. Colorado Providence, RI, USA: IEEE: 1345-1352[ DOI: 10.1109/CVPR.2011.5995468 http://dx.doi.org/10.1109/CVPR.2011.5995468 ]
Yang B and Nevatia R. 2012. An online learned CRF model for multi-target tracking//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI, USA: IEEE: 2034-2041[ DOI: 10.1109/CVPR.2012.6247907 http://dx.doi.org/10.1109/CVPR.2012.6247907 ]
Zamir A R, Dehghan A and Shah M. 2012. GMCP-Tracker: global multi-object tracking using generalized minimum clique graphs//Proceedings of the 12th European Conference on Computer Vision. Florence, Italy: Springer: 343-356[ DOI: 10.1007/978-3-642-33709-3_25 http://dx.doi.org/10.1007/978-3-642-33709-3_25 ]
Zhang L, Li Y and Nevatia R. 2008. Global data association for multi-object tracking using network flows//Proceedings of 2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, AK, USA: IEEE: 1-8[ DOI: 10.1109/CVPR.2008.4587584 http://dx.doi.org/10.1109/CVPR.2008.4587584 ]
Zhang L J and Zhou Z P. 2018. Multiple target tracking using hierarchical data association based on network flows. Journal of Computer-Aided Design and Computer Graphics, 30(9):1670-1677
张丽娟, 周治平. 2018.基于网络流的分层关联多目标跟踪.计算机辅助设计与图形学学报, 30(9):1670-1677[DOI:10.3724/SP.J.1089.2018.16906]
Zhu S H, Sun C J and Shi Z. 2016. Multi-target tracking via hierarchical association learning. Neurocomputing, 208:365-372[DOI:10.1016/j.neucom.2016.02.071]
相关作者
相关机构
京公网安备11010802024621
