浏览全部资源
扫码关注微信
收稿：2020-04-03，
修回：2020-12-17，
录用：2020-12-24，
纸质出版：2021-11-16
移动端阅览
目的
2
卫星视频作为新兴遥感数据，可以提供观测区域高分辨率的空间细节信息与丰富的时序变化信息，为交通监测与特定车辆目标跟踪等应用提供了不同于传统视频视角的信息。相较于传统视频数据，卫星视频中的车辆目标分辨率低、尺度小、包含的信息有限。因此，当目标边界不明、存在部分遮挡或者周边环境表观模糊时，现有的目标跟踪器往往存在严重的目标丢失问题。对此，本文提出一种基于特征融合的卫星视频车辆核相关跟踪方法。
方法
2
对车辆目标使用原始像素和方向梯度直方图（histogram of oriented gradient，HOG）方法提取包含互补判别能力的特征，利用核相关目标跟踪器分别得到具备不变性和判别性的响应图；通过响应图融合的方式结合两种特征的互补信息，得到目标位置；使用响应分布指标（response distribution criterion，RDC）判断当前目标特征的稳定性，决定是否更新跟踪器的表征模型。本文使用的相关滤波方法具有计算量小且运算速度快的特点，具备跟踪多个车辆目标的拓展能力。
结果
2
在8个卫星视频序列上与主流的6种相关滤波跟踪器进行比较，实验数据涵盖光照变化、快速转弯、部分遮挡、阴影干扰、道路颜色变化和相似目标临近等情况，使用准确率曲线和成功率曲线的曲线下面积（area under curve，AUC）对车辆跟踪的精度进行评价。结果表明，本文方法较好地均衡了使用不同特征的基础跟踪器（性能排名第2）的判别能力，准确率曲线AUC提高了2.9%，成功率曲线AUC下降了4.1%，成功跟踪车辆目标，不发生丢失，证明了本文方法的先进性和有效性。
结论
2
本文提出的特征融合的卫星视频车辆核相关跟踪方法，均衡了不同特征提取器的互补信息，较好解决了卫星视频中车辆目标信息不足导致的目标丢失问题，提升了精度。
Objective
2
Satellite video is a new type of remote sensing system
which is capable of dynamic video and conventional image capturing. Compared with conventional very-high-resolution (VHR) remote sensing systems
a video satellite observes the Earth with a real-time temporal resolution
which has led to studies in the field of traffic density estimation
object detection
and 3D reconstruction. Satellite video has a strong potential in monitoring traffic
animal migration
and ships entering and leaving ports due to its high temporal resolution. Despite much research in the field of conventional video
relatively minimal work has been performed in object tracking for satellite video. Existing object tracking methods primarily emphasize relatively large objects
such as trains and planes. Several researchers have explored replacing or fusing the motion feature for a more accurate prediction of object position. However
few studies have focused on solving the problem caused by the insufficient amount of information of smaller objects
such as vehicles. Tracking vehicles in satellite video has three main challenges. The main challenge is the small size of the target. While the size of a single frame can be as large as 12 000×4 000 pixels
moving targets
such as cars
can be very small and only occupy 10~30 pixels. The second challenge is the lack of clear texture because the vehicle targets contain limited and/or confusing information. The third challenge is that unlike aircraft and ships
vehicles are more likely to appear in situations where the background is complex
which makes tracking the vehicle more challenging. For instance
a vehicle may make quick turns
appear partially to the vehicle
or be marked by instant changes in illumination. Selecting or constructing a single image feature that can handle all the situations mentioned above is difficult. Using multiple complementary image features is proposed by merging them into a unified framework based on a lightweight kernelized correlation filter to tackle these challenges.
Method
2
First
two complementary features with certain invariance and discriminative ability
histogram of gradients (HOG) and raw pixels
are used as descriptors of the target image patch. HOG is tied to edge information of vehicles
such as orientations
offering some discriminative ability. A HOG-based tracker can distinguish targets even when partial occlusion occurs or when illumination or road color changes. However
it would be unable to correctly classify the target from similar shapes in its surroundings
suffering from the problems caused by insufficient information. However
the raw pixel feature describes all contents in the image patch without processing
and more information can be kept without post-processing considering the smaller size of vehicles. It is invariant to the plane motion of a rigid object under low-texture information and to tracking vehicles in terms of orientation changes. However
it fails to track vehicles that are partially occluded or in changes of road color and illumination. A response map merging strategy is proposed to fuse the complementary image features by maintaining two trackers
one using the HOG feature to discriminate the target and the other using the raw pixel feature to improve invariance. In this manner
a peak response may arise at a new position
representing invariance and discriminative ability. Finally
restricted by the insufficient information of the target and the discriminative ability of the observation model
responses usually show a multipeak pattern when a disturbance exists. A response distribution criterion-based model updater is exploited to measure the distribution of merged responses. Using a correlation filter facilitates multiple vehicle tracking due to its calculation speed and online training mechanism.
Result
2
Our model is compared with six state-of-the-art correlation filter-based models. Experiments are performed on eight satellite videos captured in different locations worldwide under challenging situations
such as illumination variance
quick turn
partial occlusion
and road color change. Precision plot and success plot are adopted for evaluation. Ablation experiments are performed to demonstrate the efficiency of the method proposed
and quantitative assessments show that our method leads to an effective balance between two trackers. Moreover
visualization results of three videos show how our method achieves a balance between the two trackers. Our method outperforms all the six state-of-the-art methods and achieves a balance between the base trackers.
Conclusion
2
In this paper
a new tracker fused with complementary image features for vehicle tracking in satellite videos is proposed. To overcome the difficulties posed by the small size of the target and the lack of texture and complex background in satellite video tracking
combining the use of HOG and raw pixel features is proposed by merging the response maps of the two trackers to increase their discriminative and invariance abilities. Experiments on eight satellite videos under challenging circumstances demonstrate that our method outperforms other state-of-the-art algorithms in precision plots and success plots.
Bertinetto L, Valmadre J, Golodetz S, Miksik O and Torr P H S. 2016a. Staple: complementary learners for real-time tracking//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 1401-1409[ DOI: 10.1109/CVPR.2016.156 http://dx.doi.org/10.1109/CVPR.2016.156 ]
Bertinetto L, Valmadre J, Henriques J F, Vedaldi A and Torr P H S. 2016b. Fully-convolutional siamese networks for object tracking//Proceedings of the European Conference on Computer Vision. Amsterdam, the Netherlands: Springer Verlag: 850-865[ DOI: 10.1007/978-3-319-48881-3_56 http://dx.doi.org/10.1007/978-3-319-48881-3_56 ]
Bolme D S, Beveridge J R, Draper B A and Lui Y M. 2010. Visual object tracking using adaptive correlation filters//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, USA: IEEE: 2544-2550[ DOI: 10.1109/CVPR.2010.5539960 http://dx.doi.org/10.1109/CVPR.2010.5539960 ]
Chen T T, Wang R L, Dai B, Liu D X and Song J Z. 2016. Likelihood-field-model-based dynamic vehicle detection and tracking for self-driving. IEEE Transactions on Intelligent Transportation Systems, 17(11): 3142-3158[DOI: 10.1109/tits.2016.2542258]
Danelljan M, Bhat G, Khan F S and Felsberg M. 2017. ECO: efficient convolution operators for tracking//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 6931-6939[ DOI: 10.1109/CVPR.2017.733 http://dx.doi.org/10.1109/CVPR.2017.733 ]
Danelljan M, Robinson A, Khan F S and Felsberg M. 2016. Beyond correlation filters: learning continuous convolution operators for visual tracking//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, the Netherlands: Springer Verlag: 472-488[ DOI: 10.1007/978-3-319-46454-1_29 http://dx.doi.org/10.1007/978-3-319-46454-1_29 ]
Danelljan M. 2018. Learning Convolution Operators for Visual Tracking. Linköping: Linköping University Electronic Press: 49-51[EB/OL]. [2020-04-03] . https://www.google.com.au/books/edition/Learning_Convolution_Operators_for_Visua/nSFdDwAAQBAJ?hl=en&gbpv=0&kptab=publisherseries https://www.google.com.au/books/edition/Learning_Convolution_Operators_for_Visua/nSFdDwAAQBAJ?hl=en&gbpv=0&kptab=publisherseries
Du B, Sun Y J, Cai S H, Wu C and Du Q. 2018. Object tracking in satellite videos by fusing the kernel correlation filter and the three-frame-difference algorithm. IEEE Geoscience and Remote Sensing Letters, 15(2): 168-172[DOI: 10.1109/LGRS.2017.2776899]
Henriques J F, Caseiro R, Martins P and Batista J. 2012. Exploiting the circulant structure of tracking-by-detection with kernels//Proceedings of the 12th European Conference on Computer Vision. Florence, Italy: Springer Verlag: 702-715[ DOI: 10.1007/978-3-642-33765-9_50 http://dx.doi.org/10.1007/978-3-642-33765-9_50 ]
Henriques J F, Caseiro R, Martins P and Batista J. 2015. High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(3): 583-596[DOI: 10.1109/TPAMI.2014.2345390]
Kalal Z, Mikolajczyk K and Matas J. 2012. Tracking-learning-detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(7): 1409-1422[DOI: 10.1109/TPAMI.2011.239]
Li H C and Man Y Y. 2016. Moving ship detection based on visual saliency for video satellite//Proceedings of 2016 IEEE International Geoscience and Remote Sensing Symposium. Beijing, China: IEEE: 1248-1250[ DOI: 10.1109/IGARSS.2016.7729316 http://dx.doi.org/10.1109/IGARSS.2016.7729316 ]
Ma C, Huang J B, Yang X K and Yang M H. 2015. Hierarchical convolutional features for visual tracking//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE: 3074-3082[ DOI: 10.1109/ICCV.2015.352 http://dx.doi.org/10.1109/ICCV.2015.352 ]
Piciarelli C, Micheloni C and Foresti G L. 2008. Trajectory-based anomalous event detection. IEEE Transactions on Circuits and Systems for Video Technology, 18(11): 1544-1554[DOI: 10.1109/TCSVT.2008.2005599]
Qi Y K, Zhang S P, Qin L, Huang Q M, Yao H X, Lim J and Yang M H. 2019. Hedging deep features for visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(5): 1116-1130[DOI: 10.1109/TPAMI.2018.2828817]
Ríos H, Falcón R, González O A and Dzul A. 2019. Continuous sliding-mode control strategies for quadrotor robust tracking: real-time application. IEEE Transactions on Industrial Electronics, 66(2): 1264-1272[DOI: 10.1109/TIE.2018.2831191]
Shao J, Du B, Wu C, Wu J, Hu R M and Li X L. 2018. VCF: velocity correlation filter, towards space-borne satellite video tracking//Proceedings of 2018 IEEE International Conference on Multimedia and Expo. San Diego, USA: IEEE: 1-6[ DOI: 10.1109/ICME.2018.8486451 http://dx.doi.org/10.1109/ICME.2018.8486451 ]
Tao R, Gavves E and Smeulders A W M. 2016. Siamese instance search for tracking//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 1420-1429[ DOI: 10.1109/CVPR.2016.158 http://dx.doi.org/10.1109/CVPR.2016.158 ]
Uzkent B, Rangnekar A and Hoffman M J. 2017. Aerial vehicle tracking by adaptive fusion of hyperspectral likelihood maps//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu, USA: IEEE: 233-242[ DOI: 10.1109/CVPRW.2017.35 http://dx.doi.org/10.1109/CVPRW.2017.35 ]
Valmadre J, Bertinetto L, Henriques J, Vedaldi A and Torr P H S. 2017. End-to-end representation learning for correlation filter based tracking//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE: 5000-5008[ DOI: 10.1109/CVPR.2017.531 http://dx.doi.org/10.1109/CVPR.2017.531 ]
Wan X, Liu J G, Yan H S, Morgan G L K and Sun T. 2016. 3D super resolution scene depth reconstruction based on SkySat video image sequences//2016 IEEE International Geoscience and Remote Sensing Symposium. Beijing, China: IEEE: 6653-6656[ DOI: 10.1109/IGARSS.2016.7730737 http://dx.doi.org/10.1109/IGARSS.2016.7730737 ]
Wang N Y and Yeung D Y. 2014. Ensemble-based tracking: aggregating crowdsourced structured time series data//Proceedings of the 31st International Conference on International Conference on Machine Learning. Beijing, China: JMLR: 2807-2817
Wu Y, Lim J and Yang M H. 2013. Online object tracking: a benchmark//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland, USA: IEEE: 2411-2418[ DOI: 10.1109/CVPR.2013.312 http://dx.doi.org/10.1109/CVPR.2013.312 ]
Wu Y, Lim J and Yang M H. 2015. Object tracking benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9): 1834-1848[DOI: 10.1109/TPAMI.2014.2388226]
Xuan S Y, Li S Y, Han M F, Wan X and Xia G S. 2020. Object tracking in satellite videos by improved correlation filters with motion estimations. IEEE Transactions on Geoscience and Remote Sensing, 58(2): 1074-1086[DOI: 10.1109/TGRS.2019.2943366]
Yang T, Wang X W, Yao B W, Li J, Zhang Y N, He Z N and Duan W C. 2016. Small moving vehicle detection in a satellite video of an urban area. Sensors, 16(9): #1528[DOI: 10.3390/s16091528]
Yao Y J, Wu X H, Zhang L, Shan S G and Zuo W M. 2018. Joint representation and truncated inference learning for correlation filter based tracking//Proceedings of the 15th European Conference on Computer Vision. Munich, Germany: Springer Verlag: 560-575[ DOI: 10.1007/978-3-030-01240-3_34 http://dx.doi.org/10.1007/978-3-030-01240-3_34 ]
Zhou J L, Wang R and Ding J W. 2018. Online learning of spatial-temporal convolution response for robust real-time tracking//Proceedings of the 24th International Conference on Pattern Recognition. Beijing, China: IEEE: 1821-1826[ DOI: 10.1109/ICPR.2018.8545048 http://dx.doi.org/10.1109/ICPR.2018.8545048 ]
Zhu Z, Wang Q, Li B, Wu W, Yan J J and Hu W M. 2018. Distractor-aware siamese networks for visual object tracking//Proceedings of the 15th European Conference on Computer Vision. Munich, Germany: Springer Verlag: 103-119[ DOI:10.1007/978-3-030-01240-3_7 http://dx.doi.org/10.1007/978-3-030-01240-3_7 ]
相关作者
相关机构
京公网安备11010802024621
