浏览全部资源
扫码关注微信
纸质出版日期： 2021-10-16 ，
录用日期： 2020-11-09
移动端阅览
王新年, 于丹, 张涛. 穿鞋足迹序列的足迹能量图组表达与识别[J]. 中国图象图形学报, 2021,26(10):2357-2375.
Xinnian Wang, Dan Yu, Tao Zhang. Shoeprint sequence representation and recognition using shoeprint energy map set[J]. Journal of Image and Graphics, 2021,26(10):2357-2375.
目的
2
现有的足迹研究主要针对赤足和穿袜足迹，取得了较高的识别精度，但需要进行脱鞋配合；而单枚穿鞋足迹由于受到鞋底花纹的影响，识别精度低，主要用于检索。由于穿鞋足迹序列不仅包含人足的结构特征还包含人行走的运动特征，将其用于人身识别会比基于单枚穿鞋足迹的识别精度高。基于此，本文对基于穿鞋足迹序列的身份识别方法进行了研究，提出了穿鞋足迹序列的足迹能量图组表达与识别算法。
方法
2
构建反映人足结构和走路行为特性的足迹能量图组来表达足迹序列，从而进行身份识别。足迹能量图组由步态能量图、步幅能量图和步宽能量图构成。步态能量图反映的是足底各个部位与承痕体相互作用形成的效果以及脚的解剖结构特征；步幅能量图和步宽能量图反映的是行走过程中双脚的空间搭配关系以及运动特征，体现人的行为信息。足迹序列之间的匹配得分由各能量图之间的相似度加权计算，其中加权系数采用铰链损失函数训练而得，各能量图之间的相似度采用归一化互相关函数计算而得。将匹配得分最高的足迹序列对应的标签作为最终的识别结果。
结果
2
根据采集方式、鞋的新旧程度和鞋底花纹种类构建了3个数据集，分别为采用光学成像仪采集的穿日常鞋的穿鞋足迹序列数据集MUSSRO-SR、采用光学成像仪采集的穿同花纹新鞋的穿鞋足迹序列数据集MUSSRO-SS和采用墨拓扫描方式采集的穿新鞋的穿鞋足迹序列数据集MUSSRS-SS。分别在上述3个数据集上进行了识别模式和验证模式实验，识别率分别达到100%、97.65%和83%，等错误率分别为0.36%、1.17%和6.99%。
结论
2
在3种类型不同的数据集上的实验结果表明，本文提出的足迹能量图组能够实现对穿鞋足迹序列的有效表达，并实际验证了基于穿鞋足迹序列的身份识别的可行性。
Objective
2
Shoeprints are impressions created when footwear is pressed or stamped against a surface during a human's walking
in which the characteristics of the shoe and feet are transferred to the surface. Shoeprints can be divided into two categories: 2D and 3D shoeprints. In this paper
we focus on 2D shoeprints. The shoeprint sequence is defined as the sequential shoeprints in time order
which conveys many important human characteristics
such as foot morphology
walking habits
and identity
and plays a vital role in crime investigations. Existing research has mainly focused on using one single footprint or footprint sequence to recognize a person and has achieved a promising performance. However
unlike shoeprints
footprint sequences seldom appear in most scenarios. Although a single shoeprint is not sufficient to represent a person for the influence of shoe patterns
a shoeprint sequence could be possible because it can additionally provide walking characteristics. Therefore
our goal is to try to identify a person using his/her shoeprint sequence.
Method
2
A shoeprint sequence is a time series that 2D shoeprints from one person repeat at a stable frequency
and we propose a spatial representation named shoeprint energy map set (SEMS) to represent a shoeprint sequence and use it to identify a person in collaboration with the proposed matching score method. An SEMS consists of left/right tread energy maps
left/right step energy maps
and left/right step width energy maps. The tread energy map (TEM) is defined as the average image of all the aligned left or right shoeprint images from a shoeprint sequence
which includes LTEM and RTEM. The LTEM is constructed from left shoeprints only
and the RTEM is computed from right shoeprints only. The TEM carries information about one's personal features such as foot morphology
walking habits
and step angles. The step energy map (SEM) is computed by averaging all aligned step images of multiple walking cycles from a shoeprint sequence. The step image refers to the region cropped from a shoeprint sequence according to the bounding box that only encloses two successive shoeprints. According to which foot is ahead in one walking cycle
step images can be divided into right step images and left step images. Hence
LSEM and RSEM are produced. Compared with TEM
one's SEM carries additional step information such as step length and step width. The step width energy map (SWEM) is constructed by averaging all step width images of multiple walking cycles from a shoeprint sequence
which includes LSWEM and RSWEM. The step width image refers to the image removing the blank region representing the step length from the step image. Different from SEM
the SWEM does not carry one's step length information. The matching score between two shoeprint sequences is defined as the weighted average of the element wise similarity scores of two SEMSs. The element wise similarity score is computed by max pooling of the normalized 2D cross-correlation response map of two corresponding elements such as LTEMs. The weights are learned from the training sets by maximizing the proposed hinge loss function.
Result
2
According to the imaging methods
the status of shoes and the kinds of shoe sole patterns
three datasets
namely MUSSRO-SR
MUSSRO-SS
and MUSSRS-SS
are constructed. Volunteers are young college students whose heights range from 155 cm to 185 cm and weights are between 43 kg and 85 kg. MUSSRO-SR consists of 875 shoeprint sequences from 125 volunteers. The shoeprint sequences are captured by the way that each person walks normally on the footprint sequence scanner wearing his/her daily shoes. MUSSRO-SS is composed of 595 shoeprint sequences from 85 persons
and the capturing method is that each volunteer walks normally on the footprint sequence scanner wearing new shoes of same patterns. MUSSRS-SS is constructed by scanning papers
where each of 100 persons walks normally wearing new shoes after stepping in a tray full of black ink. The proposed method is evaluated in identification mode and verification mode. To the best of our knowledge
we have not found shoeprint sequence-based person recognition methods in the literature. Therefore
our method is compared with gait measurement (GM)-based method used in forensic practice. The correct recognition rates (higher is better) in identification mode on three datasets are 100%
97.65%
and 83%. Compared with GM
the correct recognition rates of the proposed method are increased by 57.6%
61.18%
and 48.35%. The performances in verification mode are measured by equal error rate (ERR). The lower the EER is
the higher the performance is. The ERR on MUSSRO-SR is 0.36%. Compared with GM
it is decreased by 14.1 percentage points. ERR on MUSSRO-SS is 1.17%
which is decreased by 10.43 percentage points. The ERR on MUSSRS-SS is 6.99%
which is decreased by 10.8 percentage points. Performances on MUSSRO-SR are higher than those on other datasets for the following reasons: 1) Shoeprints left by wearing daily shoes carry unique personal wear characteristics. 2) Shoe patterns are not all the same. Performances on MUSSRS-SS are lower than others for the following reasons: 1) Shoeprints scanned from inked paper carry less personal information than those captured by specified acquisition devices. 2) The amount of ink attached to the sole decreases while a person is walking
which degrades the image quality of shoeprints. In addition
a series of ablation studies is conducted to show the effectiveness of the proposed three kinds of shoeprint energy maps and the matching score computing method.
Conclusion
2
In this study
a shoeprint sequence is used to recognize a person
and constructing an SEMS is proposed to represent a shoeprint sequence
which carries the psychological and behavioral characteristics of humans. Experimental results show the promising
competitive performance of the proposed method. How the sole pattern or substrate material with a large difference affects the performance is not studied
and cross-pattern/substrate material shoeprint sequence recognition is our next work.Moreover
a larger volume dataset is under preparation.
身份识别穿鞋足迹序列识别足迹能量图组(SEMS)步态能量图(TEM)步幅能量图(SEM)步宽能量图(SWEM)
person recognitionshoeprint sequence recognitionshoeprint energy map set(SEMS)tread energy map(TEM)step energy map(SEM)step width energy map(SWEM)
Bao W X, Qu J J, Wang N, Tang J and Lu X L. 2020a. Force-tactile footprint recognition based on spatial aggregation weighted convolutional neural network. Journal of Southeast University (Natural Science Edition), 50(5): 959-964
鲍文霞, 瞿金杰, 王年, 唐俊, 鲁玺龙. 2020a. 基于空间聚合加权卷积神经网络的力触觉足迹识别. 东南大学学报(自然科学版), 50(5): 959-964 [DOI: 10.3969/j.issn.1001-0505.2020.05.023]
Bao W X, Wang Y F, Wang N and Tang J. 2020b. Identification algorithm of optical footprint image based on metric learning kernel function. Journal of Huazhong University of Science and Technology (Natural Science Edition), 48(11): 11-16
鲍文霞, 王云飞, 王年, 唐俊. 2020b. 基于度量学习核函数的光学足迹图像识别算法. 华中科技大学学报, 48(11): 11-16 [DOI: 10.13245/j.hust.201103]
Bhowmick P, Banerjee A, Dey D and Munshi S, 2018. Cross Teager-Kaiser energy operator based feature extraction method for gait recognition from cumulative foot pressure images//Proceedings of 2018 IEEE Applied Signal Processing Conference (ASPCON), Kolkata, India: IEEE: 371-374[DOI: 10.1109/ASPCON.2018.8748726http://dx.doi.org/10.1109/ASPCON.2018.8748726]
Bodziak W J. 1999. Footwear Impression Evidence: Detection, Recovery, and Examination. 2nd ed. Boca Raton: CRC Press[DOI: 10.1201/9780203755587].
Costilla-Reyes O, Vera-Rodriguez R, Scully P and Ozanyan K B. 2016. Spatial footstep recognition by convolutional neural networks for biometrie applications//Proceedings of 2016 IEEE SENSORS. Orlando, USA: IEEE: 1-3[DOI: 10.1109/ICSENS.2016.7808890http://dx.doi.org/10.1109/ICSENS.2016.7808890]
Costilla-Reyes O, Vera-Rodriguez R, Scully P and Ozanyan K B. 2019. Analysis of spatio-temporal representations for robust footstep recognition with deep residual neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(2): 285-296[DOI: 10.1109/TPAMI.2018.2799847]
Jung J W, Bien Z, Lee S W and Sato T. 2003. Dynamic-footprint based person identification using mat-type pressure sensor//Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Cancun, Mexico: IEEE: 3: 2937-2940. [DOI: 10.1109/IEMBS.2003.1280533http://dx.doi.org/10.1109/IEMBS.2003.1280533]
Jung J W, Lee S W, Bien Z and Sato T. 2013. Person recognition method using sequential walking footprints via overlapped foot shape and center-of-pressure trajectory. Journal of Systemics. 11(4): 34-39
Kennedy R B. 1996. Uniqueness of bare feet and its use as a possible means of identification. Forensic Science International, 82(1): 81-87[DOI: 10.1016/0379-0738(96)01969-X]
Li C Z, Wang Q J, Han J L and Wang J Y. 1982. Application of U test in footprint identification. Journal of Applied Statistics and Management, (2): 17-21
李从珠, 王清举, 韩均良, 王锦义. 1982. U检验法在步法鉴定中的应用. 数理统计与管理, (2): 17-21 [DOI: 10.13860/j.cnki.sltj.1982.02.005]
Li Y, Pan N, Liu F and Liu Y. 2014. Study of cluster footprint analysis testing algorithm and system. Machinery and Electronics, (5): 48-50, 51
李岩, 潘楠, 刘凤, 刘益. 成趟足迹分析检验算法及系统研究. 机械与电子, (5): 48-50, 51 [DOI: 10.3969/j.issn.1001-2257.2014.05.012]
McNutt E J, Zipfel B and DeSilva J M. 2018. The evolution of the human foot. Evolutionary Anthropology, 27(5): 197-217[DOI: 10.1002/evan.21713]
Peng J. 2016. Study of Cluster and 3D Footprint Analysis Testing System. Kunming: Kunming University of Science and Technology
彭进. 2016. 成趟与立体足迹分析检验系统的研究. 昆明: 昆明理工大学
Qian G, Zhang J Q and Kidané A. 2010. People identification using floor pressure sensing and analysis. IEEE Sensors Journal, 10(9): 1447-1460[DOI: 10.1109/JSEN.2010.2045158]
Robbins L M. 1978. The individuality of human footprints. Journal of Forensic Sciences, 23(4): 778-785
Shi L M and Ma J P. 2014. Footprint. Beijing: Chinese People's Public Security University Press
史力民, 马建平. 2014. 足迹学. 北京: 中国人民公安大学出版社
Takeda T, Kuramoto K, Kobashi S and Hata Y. 2013. On optimal operator for combining left and right sole pressure data in biometrics security. Advances in Fuzzy Systems, 2013: #620312[DOI: 10.1155/2013/620312]
Tao Y N, Su F, Yuan P J, Wang T M, Zhong T and Hao J R. 2020. Intelligent criminal investigation system based on both footprint recognition and surveillance video analysis. Journalof Beijing University of Aeronautics and Astronautics, 46(9): 1730-1738
陶一宁, 苏峰, 袁培江, 王田苗, 钟涛, 郝静如. 2020. 联合足迹识别与监控视频分析的智能刑侦系统. 北京航空航天大学学报, 46(9): 1730-1738 [DOI: 10.13700/j.bh.1001-5965.2020.0062]
Keatsamarn T and Pintavirooj C. 2018. Footprint identification using deep learning//Proceedings of the 11th Biomedical Engineering International Conference(BMEiCON). Chiang Mai, Thailand: IEEE: 1-4[DOI: 10.1109/BMEiCON.2018.8609926http://dx.doi.org/10.1109/BMEiCON.2018.8609926]
Vera-Rodriguez R, Mason J S D, Fierrez J and Ortega-Garcia J. 2013. Comparative analysis and fusion of spatiotemporal information for footstep recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(4): 823-834[DOI: 10.1109/TPAMI.2012.164]
Wang X N, Wang H Y, Cheng Q, Nankabirwa N L and Zhang T. 2017. Single 2D pressure footprint based person identification//Proceedings of 2017 IEEE International Joint Conference on Biometrics. Denver, USA: IEEE: 413-419[DOI: 10.1109/BTAS.2017.8272725http://dx.doi.org/10.1109/BTAS.2017.8272725]
Zhou B, Singh M S, Doda S, Yildirim M, Cheng J Y and Lukowicz P. 2017. The carpet knows: identifying people in a smart environment from a single step//Proceedings of 2017 IEEE International Conference on Pervasive Computing and Communications Workshops. Kona, USA: IEEE: 527-532[DOI: 10.1109/PERCOMW.2017.7917618http://dx.doi.org/10.1109/PERCOMW.2017.7917618]
相关文章
相关作者
相关机构
京公网安备11010802024621