Current Issue Cover
面向智慧城市的交通视频结构化分析前沿进展

赵耀1, 田永鸿2, 党建武3, 付树军4, 王恒友5, 万军6, 安高云1, 杜卓然1, 廖理心1, 韦世奎1(1.北京交通大学信息科学研究所, 北京 100044;2.北京大学信息科学技术学院, 北京 100871;3.兰州交通大学电子与信息工程学院, 兰州 730070;4.山东大学数学学院, 济南 250100;5.北京建筑大学理学院, 北京 102616;6.中国科学院自动化研究所模式识别国家重点实验室, 北京 100190)

摘 要
随着智慧城市建设的不断深入,大量的传感器设备铺置在城市公路和轨道等交通场景,为多维度全方位感知城市交通状态构建了广泛的感知网络,产生了海量的交通视频数据。海量交通视频数据是城市管理的数据宝藏,理解与分析这些数据是智慧城市建设的关键。面对高度冗余的交通视频数据,如何高效准确地挖掘和提取结构化信息,实现对重点目标(如人、车、物)的快速检测、识别与检索,是交通视频处理的核心问题——交通视频结构化分析。交通视频结构化分析包括车辆视频结构化分析、人员结构化分析及其行为分析。其中,车辆结构化作为一个复杂的多步骤任务,主要由车辆的检测、车辆的属性(车牌、车型和颜色)识别以及车辆的检索和重识别等子任务构成。人脸结构化和行人结构化是交通视频中行人结构化智能分析中的两个重要研究方向,主要分析人脸或者行人的一些表观属性。行人行为分析是指对行人在复杂交通环境下做出的动作进行识别和预测。本文从交通视频中的车辆、行人及其行为分析等方面,阐述交通视频结构化分析领域的研究热点及前沿进展,汇总比较国内外的相关成果,并对交通视频结构化分析领域的研究进行总结分析与展望。
关键词
Frontiers of transportation video structural analysis in the smart city

Zhao Yao1, Tian Yonghong2, Dang Jianwu3, Fu Shujun4, Wang Hengyou5, Wan Jun6, An Gaoyun1, Du Zhuoran1, Liao Lixin1, Wei Shikui1(1.Institute of Information Science, Beijing Jiaotong University, Beijing 100044, China;2.School of Electronics Engineering and Computer Science, Peking University, Beijing 100871, China;3.School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China;4.School of Mathematics, Shandong University, Jinan 250100, China;5.School of Science, Beijing University of Civil Engineering and Architecture, Beijing 102616, China;6.National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China)

Abstract
As the construction of smart cities continues to deepen, our country gradually builds multidimensional and omnidirectional sensor systems in roads, railways, and urban rails and other ground transportation fields to build strong data support for smart transportation. Faced with all-weather traffic data collected by sensors, analyzing the data by relying solely on human resources is no longer possible. Therefore, studying the structural analysis technology of traffic video and establishing a safe, flexible, and efficient intelligent transportation system has significant social benefits and application value. Traffic video structural analysis is the core technology in smart transportation. It aims to use artificial intelligence algorithms to parse unstructured traffic video data into structured semantic information that is easy for workers and computers to understand and provide basic technical support for subsequent related tasks. The structural analysis of traffic video is a key technology for smart city construction. It can help the police in quickly locating criminal vehicles and travel routes, greatly improve the police’s efficiency in solving crimes, and maintain city safety; it can also automatically identify illegal vehicles and types of violation, constrains people to abide by the traffic order, and realize a smooth urban traffic environment. With the advent of the 5G internet of things era, ultrahigh network bandwidth and transmission speed further improves the quality and efficiency of vehicle video transmission. Efficiently and accurately conducting traffic video structure analysis will be the focus of research in the next few years. Traffic video structural analysis includes vehicle video structural analysis, personnel structural analysis, and behavior analysis. Among them, as a complex, multistep task, vehicle structuring is mainly composed of three subtasks, namely, vehicle detection, vehicle attributes (license plate, type, and color) recognition, and vehicle retrieval and reidentification. Human face structuring and pedestrian structuring are two important research directions in the intelligent analysis of traffic videos. They mainly analyze some apparent attributes of human faces or pedestrians, such as age, gender, mask, backpack, clothing color, and length. Pedestrian behavior analysis refers to the identification and prediction of pedestrian actions. For example, the speed at which pedestrians currently head and in which direction, whether they are answering calls, and whether they have to cross the road. For the task of vehicle structure analysis, first, the object detection technology must be used to quickly and accurately locate the vehicle. Second, on the basis of positioning the vehicle, it fully excavates the visual characteristics of the vehicle, realizes the identificati?瑮愠獯武猠??剥愠灩楮摨?牲敥瑮牴椠敡癴慴汲?潢牵?牥敳椠摯敦渠瑶楥晨楩捣慬瑥椬漠湡?潤映?瑥慮牥杲敡瑴?癳攠桳楴捲汵散獴?楲湥?挠潴浡灧汳攠硡?慯湵摴?捴桨慥渠杶敥慨扩汣敬?献挠敆湩敮獡?楬獹?挠牯畮挠楴慨汥???瑳?牳攠煯畦椠牳整獲?捣潴湵瑲楥湤甠潴畡獧?椬渠癴敨獥琠浲敥湴瑲?慥湶摡?愠?浥畣捨桮?楬湯湧潹瘠慡瑮楤瘠敲?物敤獥敮慴物捦桩?扡祴?獯据椠整湥瑣楨普楯捬?牧敹猠敡慲牥挠桦敵牲獴??呲栠散?湭敢敩摮?晤漠牴?猠瑲牥畡捬瑩畺牥攠摴?灥攠摲敥獴瑲物楥慶湡?愠湡慮汤礠獲楥獩?桥慮獴?杦物慣摡畴慩汯汮礠?敦洠敡爠杳数摥?睩楦瑩档?晶略牨瑩档敬牥?楩浮瀠牴潨癥攠浭敡湳瑳?潶晥?當物扤慥湯?浤慡湴慡朮攠浐敥湲瑳??偮敥摬攠獳瑴牲極慣湴?獲瑡牬甠捡瑮畡牬楹湳杩?洠慡楮湤氠祢?慨湡慶汩祯穲攠獡?獡潬浹敳?慳瀠灣慡牮攠湤瑥?慥瑣瑴爠楡扮畤琠敩獤?潮晴?晦慹挠数獥?潥牳?灲敩摡敮獳琠物楮愠湴獲??獦畩捣栠?慩獤?慯杳攠??杤攠湣摯敮牤??慴渠摳?捲汵潣瑴桵楲湥杤?獤瑡祴污攠??慴湲摡?灴物潯癮椠摡敮獤?浢潥牨敡?摩敯瑲愠楡汮敡摬?摳慩瑳愠?獦甠灤灥潴牥瑣?晥潤爠?獥畲扳獯敮煮略敬渮琠?牮攠汴慨瑥攠摡?瑡慬獹歳獩??偯敦搠数獥瑲牳楯慮湮?獬琠牳畴捲瑵畣牴敵摲?愬渠慡氠祰獥楲獳?瑮攠捩桳渠潥汸潴杲祡?桴慥獤?慡汳猠潡?畤獥桳散牲敩摰?楩湶?愠?灮敤物楶潩摤?潡晬?爠慉灮椠摴?摲敭癳攠汯潦瀠浦敡湣瑥?睳楴瑲桵?瑴桵敲?搬攠癩整氠潩灮浣敬湵瑤?潳映?摣散敵灲?汴敥愠牦湡楣湩条???潯睳敩癴敩牯??瑮桧攬?獦瑡牣畩捡瑬甠牦敥摡?慵湲慥氠祥獸楴獲?潣晴?灯敮搬攠獡瑮牤椠慦湡獣?晡潬爠?獥灡整捵楲晥椠捣?獭捰敡湲慩牳楯潮献??獮甠捴桥?慭獳?慯捦挠異牥慤瑥敳汴祲?楡摮攠湳瑴楲晵祣楴湵杲?琬栠敩?愠杩敮?慬湵摤?杳攠湧摥敮牤?潲昬?慡?灥攬爠獡潮湤?楡湧?愠湯?甠湴捨潥渠獰瑥牲慳楯湮攮搠?敡湲癩楯牵潳渠浤敥湳瑣??楰浴灩汶敥洠敩湮瑦楯湲杭?瑴桩敯?搠敩灮汣潬祵浤敥湳琠?潥晩?桨楴本栠?灡物敲挠楡獣楣潥湳?浯潲摩敥汳猬?楣湬?瑴敨物浮楧測愠汣?獲祲獹瑩敮浧猠?睴楥瑭桳?氠楡浮楤琠敷摡?牫敩獮潧甠牰捡整獴??慮湳搮?楐湥瑤敥杳牴慲瑩楡湮朠?浥畨污瑶楩浯潲搠慡汮?楬湹晳潩牳洠慩瑳椠潣湡?瑲潩?晤甠牯瑵桴攠牯?椠浴灨牥漠癢敡?瑩桳攠?慦挠捰略牲慳捯祮?潥晬?灳整摲敵獣瑴牵楲慥渠?慮瑡瑬特楳扩畳琮攠?牥敨捡潶杩湯楲琠楡潮湡??湳敩敳搠獲?晦略牲瑳栠整牯?牴敨獥攠慲牥捣桯??偩整摩敯獮琬爠楣慯湭?扲敥桨慥癮楳潩牯?愬渠慡汮祤猠楰獲?楤獩?慴?浯潮爠敯?愠摰癥慤湥捳整摲?瑡慮猠歡?楴湩?瑮牳愮映晉楮挠?癨楥搠敡潲?扡椠杯?搠慢瑩慧?灤牡潴捡攠獰獲楯湣来?慳湩摮?愠湡慮汤礠獡楮獡???瑩?椠獯?洠潴牲敡?捦桩慣氠汶敩湤来楯測朠?摥畳敥?瑲潣?映慯据琠潶牥獨??獬略挠桳?慲獵?獴桵潲潩瑮楧渠杳?獡捲整湥敤猠??浲潬癩楥湲朠?据慤洠敲牥慬獡??癤椠整睥楣湨杮?慬湯杧汩敥獳??慡湶摥?污楬杳桯琠楤湥杶?捬桯慰湥杤攠獲???畤摬杹椬渠杢?晴爠潩浴?瑣桡敮?扳整桩慬癬椠潢牥?牲敥捭潡杲湫楡瑢楬潹渠?敥晶晥敬捯瑰?潤昮?瑔桨敥?浰慲楥湭獩瑳牥攠慯浦?湶敥畨物慣汬?渠敳瑴睲潵牣歴?慲物据桧椠瑩敳挠瑶略牨敩??瑥栠敤?捴略牣牴敩湯瑮?洠潷摨敩汣?搠潩敳猠?湦潦瑥?慴捥桤椠敢癹攠?瑨桥攠?摨敯獯楴物敮摧?敳晣晥敮捥琠?潮湤?瑴桨敥?汭慯牶杩敮?猠捳慰汥敥?戠敯桦愠癴楨潥爠?摥慨瑩慣?獥攮琠??楣湵敲瑡楴捥?批攠捬慯畣獡整?瑮桧攠?敨硥椠獶瑥楨湩杣?浥漠摩敮氠?獨瑥椠汣污?晥愠楯汦猠?瑯潷?晬畩汧汨祴?污敮慤爠湴?慥渠摦?浳潴搠敶汥?瑩档敬?戠敳桰慥癥楤漠物慳氠?瑴楩浬楬渠条?牰敲汯慢瑬楥潭渠獴桯椠灢???湯?瑶桥敤?映楍敡汮摹?潴晹?扥敳栠慯癦椠潶牥?物散捬潥杳渠楡瑲楥漠湦??普畤琠畩牮攠?牨敥猠敭慡牲捫桥?挬愠湡?獤琠楴汨汥?晤潩捦畦獥?潥湮?牥敳挠潢来湴楷瑥楥潮渠?浯潤摥敬汳猠?晦漠牳?摭敩獬楡杲渠楢湲条?汤潳渠条?瑥椠浳敭?摬敬瀮攠湌摩散湥瑮?湥攠瑰睬潡牴步?慲牥捣桯楧瑮敩捴瑩畯牮攠獨??愠摢慥灣瑯業湥朠?汯慲牥朠敩?獰捯慲汴敡?摴愮琠慉?猠散瑯獭??慥湸搠?慮捤栠楣敨癡楮湧来?汢楬来栠瑳督敥楮来桳琬?扴敨桥愠癧楥潮牥??坬楩瑺桡?瑩桯敮?摡敮癤攠污潣灣浵敲湡瑣?漠景??湴瑨敥爠湰敯瑳?潴晩?呮桩楮湧朠獡?慤渠摲????瑮敩捴桩湯潮氠潡杬楧敯獲??瑨桭攠?灨牯潵浬潤琠楢潥渠?潵晲?湨敥睲?瑩敭捰桲湯潶汥潤朮椠敔獨?栠慥獸?慥汮獳潩?灥氠慤祥数摬?慹湭?楮浴瀠潯牦琠慴湲瑡?牦潩汣攠?楯湮?瑴桯敲?獮瑧爠略捱瑵畩牰慭汥?慴渠慲汥祡獬楩獺?潳映?瑬牬愭晷晥楡捴?癥楲搠敭潯??呴潯?扩敮?攠煯畦椠灲灥敬摥?睡楮瑴栠??潡呤?摳敹癳楴捥敭獳?桡慮獤?扦敵捲潴浨敥?愠湩?楣湲敥癡楳瑥慳戠汴敨?琠牤敩湦摦?晣潵牬?浹漠摯敦爠湶?捨慩牣獬??噲敥桴楲捩汥敶獡?挠慡湮?戠敲?捩潤湥湮整捩瑦敩摣?瑴潩?扮愀猀椀挀?琀爀愀渀猀瀀漀爀琀愀琀椀漀渀?昀愀挀椀氀椀琀椀攀猀??瘀攀栀椀挀氀攀?琀漀?椀渀昀爀愀猀琀爀甀挀琀甀爀攀??嘀????漀爀?琀漀?猀甀爀爀漀甀渀搀椀渀最?瘀攀栀椀挀氀攀猀??瘀攀栀椀挀氀攀?琀漀?瘀攀栀椀挀氀攀??嘀?嘀???吀栀攀?搀攀瘀攀氀漀瀀洀攀渀琀?漀昀?琀栀攀猀攀?琀攀挀栀渀漀氀漀最椀攀猀?搀攀瀀攀渀搀猀?漀渀?琀栀攀?挀漀洀洀漀渀?瀀爀漀最爀攀猀猀?漀昀?瘀攀栀椀挀氀攀?瘀椀搀攀漀?猀琀爀甀挀琀甀爀椀渀最?愀渀搀?椀渀琀攀爀渀攀琀?漀昀?琀栀椀渀最猀?琀攀挀栀渀漀氀漀最礀??圀椀琀栀?琀栀攀?最氀漀戀愀氀?瀀漀瀀甀氀愀爀椀琀礀?漀昀????琀攀挀栀渀漀氀漀最礀??爀愀瀀椀搀?琀爀愀渀猀洀椀猀猀椀漀渀?漀昀?栀椀最栀?焀甀愀氀椀琀礀?瘀椀搀攀漀?搀愀琀愀?栀愀猀?戀攀挀漀洀攀?愀?爀攀愀氀椀琀礀???砀琀爀愀挀琀椀渀最?猀琀爀甀挀琀甀爀攀搀?椀渀昀漀爀洀愀琀椀漀渀?洀漀爀攀?攀昀昀椀挀椀攀渀琀氀礀?昀爀漀洀?琀爀愀昀昀椀挀?瘀椀搀攀漀猀??猀甀挀栀?愀猀?瘀攀栀椀挀氀攀?椀渀昀漀爀洀愀琀椀漀渀??瀀攀搀攀猀琀爀椀愀渀?椀渀昀漀爀洀愀琀椀漀渀??愀渀搀?戀攀栀愀瘀椀漀爀?瀀爀攀搀椀挀琀椀漀渀??栀愀猀?戀攀挀漀洀攀?洀漀爀攀?甀爀最攀渀琀??刀攀猀攀愀爀挀栀攀爀猀?猀栀漀甀氀搀?猀琀甀搀礀?漀渀?椀洀瀀爀漀瘀椀渀最?琀栀攀?瀀攀爀昀漀爀洀愀渀挀攀?漀昀?爀攀氀愀琀攀搀?愀氀最漀爀椀琀栀洀猀??猀栀漀甀氀搀?搀攀猀椀最渀?洀漀爀攀?攀昀昀椀挀椀攀渀琀?栀愀爀搀眀愀爀攀?猀礀猀琀攀洀猀??愀渀搀?戀甀椀氀搀?洀漀爀攀?攀昀昀椀挀椀攀渀琀?琀爀愀昀昀椀挀?瘀椀搀攀漀?猀琀爀甀挀琀甀爀攀搀?愀渀愀氀礀猀椀猀?猀礀猀琀攀洀猀?琀栀爀漀甀最栀?猀漀昀琀眀愀爀攀?愀渀搀?栀愀爀搀眀愀爀攀?挀漀氀氀愀戀漀爀愀琀椀漀渀猀??圀攀?搀椀猀挀甀猀猀?琀栀攀?爀攀氀愀琀攀搀?眀漀爀欀?漀渀?琀爀愀昀昀椀挀?瘀椀搀攀漀?猀琀爀甀挀琀甀爀愀氀?愀渀愀氀礀猀椀猀?椀渀?搀攀琀愀椀氀?昀爀漀洀?琀栀爀攀攀?愀猀瀀攀挀琀猀??愀猀?昀漀氀氀漀眀猀??瘀攀栀椀挀氀攀??瀀攀爀猀漀渀渀攀氀??愀渀搀?戀攀栀愀瘀椀漀爀?愀渀愀氀礀猀椀猀???漀爀攀漀瘀攀爀??眀攀?猀甀洀洀愀爀椀稀攀?琀栀攀猀攀?爀攀猀攀愀爀挀栀?眀漀爀欀猀?愀渀搀?瀀爀漀瘀椀搀攀?猀漀洀攀?爀攀愀猀漀渀愀戀氀攀?搀椀爀攀挀琀椀漀渀猀?昀漀爀?昀甀琀甀爀攀?眀漀爀欀?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
Keywords

订阅号|日报