Current Issue Cover
穿鞋足迹序列的足迹能量图组表达与识别

王新年1, 于丹1, 张涛2(1.大连海事大学, 大连 116026;2.辽宁师范大学, 大连 116029)

摘 要
目的 现有的足迹研究主要针对赤足和穿袜足迹,取得了较高的识别精度,但需要进行脱鞋配合;而单枚穿鞋足迹由于受到鞋底花纹的影响,识别精度低,主要用于检索。由于穿鞋足迹序列不仅包含人足的结构特征还包含人行走的运动特征,将其用于人身识别会比基于单枚穿鞋足迹的识别精度高。基于此,本文对基于穿鞋足迹序列的身份识别方法进行了研究,提出了穿鞋足迹序列的足迹能量图组表达与识别算法。方法 构建反映人足结构和走路行为特性的足迹能量图组来表达足迹序列,从而进行身份识别。足迹能量图组由步态能量图、步幅能量图和步宽能量图构成。步态能量图反映的是足底各个部位与承痕体相互作用形成的效果以及脚的解剖结构特征;步幅能量图和步宽能量图反映的是行走过程中双脚的空间搭配关系以及运动特征,体现人的行为信息。足迹序列之间的匹配得分由各能量图之间的相似度加权计算,其中加权系数采用铰链损失函数训练而得,各能量图之间的相似度采用归一化互相关函数计算而得。将匹配得分最高的足迹序列对应的标签作为最终的识别结果。结果 根据采集方式、鞋的新旧程度和鞋底花纹种类构建了3个数据集,分别为采用光学成像仪采集的穿日常鞋的穿鞋足迹序列数据集MUSSRO-SR、采用光学成像仪采集的穿同花纹新鞋的穿鞋足迹序列数据集MUSSRO-SS和采用墨拓扫描方式采集的穿新鞋的穿鞋足迹序列数据集MUSSRS-SS。分别在上述3个数据集上进行了识别模式和验证模式实验,识别率分别达到100%、97.65%和83%,等错误率分别为0.36%、1.17%和6.99%。结论 在3种类型不同的数据集上的实验结果表明,本文提出的足迹能量图组能够实现对穿鞋足迹序列的有效表达,并实际验证了基于穿鞋足迹序列的身份识别的可行性。
关键词
Shoeprint sequence representation and recognition using shoeprint energy map set

Wang Xinnian1, Yu Dan1, Zhang Tao2(1.Dalian Maritime University, Dalian 116026, China;2.Liaoning Normal University, Dalian 116029, China)

Abstract
Objective Shoeprints are impressions created when footwear is pressed or stamped against a surface during a human's walking, in which the characteristics of the shoe and feet are transferred to the surface. Shoeprints can be divided into two categories:2D and 3D shoeprints. In this paper, we focus on 2D shoeprints. The shoeprint sequence is defined as the sequential shoeprints in time order, which conveys many important human characteristics, such as foot morphology, walking habits, and identity, and plays a vital role in crime investigations. Existing research has mainly focused on using one single footprint or footprint sequence to recognize a person and has achieved a promising performance. However, unlike shoeprints, footprint sequences seldom appear in most scenarios. Although a single shoeprint is not sufficient to represent a person for the influence of shoe patterns, a shoeprint sequence could be possible because it can additionally provide walking characteristics. Therefore, our goal is to try to identify a person using his/her shoeprint sequence. Method A shoeprint sequence is a time series that 2D shoeprints from one person repeat at a stable frequency, and we propose a spatial representation named shoeprint energy map set (SEMS) to represent a shoeprint sequence and use it to identify a person in collaboration with the proposed matching score method. An SEMS consists of left/right tread energy maps, left/right step energy maps, and left/right step width energy maps. The tread energy map (TEM) is defined as the average image of all the aligned left or right shoeprint images from a shoeprint sequence, which includes LTEM and RTEM. The LTEM is constructed from left shoeprints only, and the RTEM is computed from right shoeprints only. The TEM carries information about one's personal features such as foot morphology, walking habits, and step angles. The step energy map (SEM) is computed by averaging all aligned step images of multiple walking cycles from a shoeprint sequence. The step image refers to the region cropped from a shoeprint sequence according to the bounding box that only encloses two successive shoeprints. According to which foot is ahead in one walking cycle, step images can be divided into right step images and left step images. Hence, LSEM and RSEM are produced. Compared with TEM, one's SEM carries additional step information such as step length and step width. The step width energy map (SWEM) is constructed by averaging all step width images of multiple walking cycles from a shoeprint sequence, which includes LSWEM and RSWEM. The step width image refers to the image removing the blank region representing the step length from the step image. Different from SEM, the SWEM does not carry one's step length information. The matching score between two shoeprint sequences is defined as the weighted average of the element wise similarity scores of two SEMSs. The element wise similarity score is computed by max pooling of the normalized 2D cross-correlation response map of two corresponding elements such as LTEMs. The weights are learned from the training sets by maximizing the proposed hinge loss function. Result According to the imaging methods, the status of shoes and the kinds of shoe sole patterns, three datasets, namely MUSSRO-SR, MUSSRO-SS, and MUSSRS-SS, are constructed. Volunteers are young college students whose heights range from 155 cm to 185 cm and weights are between 43 kg and 85 kg. MUSSRO-SR consists of 875 shoeprint sequences from 125 volunteers. The shoeprint sequences are captured by the way that each person walks normally on the footprint sequence scanner wearing his/her daily shoes. MUSSRO-SS is composed of 595 shoeprint sequences from 85 persons, and the capturing method is that each volunteer walks normally on the footprint sequence scanner wearing new shoes of same patterns. MUSSRS-SS is constructed by scanning papers, where each of 100 persons walks normally wearing new shoes after stepping in a tray full of black ink. The proposed method is evaluated in identification mode and verification mode. To the best of our knowledge, we have not found shoeprint sequence-based person recognition methods in the literature. Therefore, our method is compared with gait measurement (GM)-based method used in forensic practice. The correct recognition rates (higher is better) in identification mode on three datasets are 100%, 97.65%, and 83%. Compared with GM, the correct recognition rates of the proposed method are increased by 57.6%,61.18%, and 48.35%. The performances in verification mode are measured by equal error rate (ERR). The lower the EER is, the higher the performance is. The ERR on MUSSRO-SR is 0.36%. Compared with GM, it is decreased by 14.1 percentage points. ERR on MUSSRO-SS is 1.17%, which is decreased by 10.43 percentage points. The ERR on MUSSRS-SS is 6.99%, which is decreased by 10.8 percentage points. Performances on MUSSRO-SR are higher than those on other datasets for the following reasons:1) Shoeprints left by wearing daily shoes carry unique personal wear characteristics. 2) Shoe patterns are not all the same. Performances on MUSSRS-SS are lower than others for the following reasons:1) Shoeprints scanned from inked paper carry less personal information than those captured by specified acquisition devices. 2) The amount of ink attached to the sole decreases while a person is walking, which degrades the image quality of shoeprints. In addition, a series of ablation studies is conducted to show the effectiveness of the proposed three kinds of shoeprint energy maps and the matching score computing method. Conclusion In this study, a shoeprint sequence is used to recognize a person, and constructing an SEMS is proposed to represent a shoeprint sequence, which carries the psychological and behavioral characteristics of humans. Experimental results show the promising, competitive performance of the proposed method. How the sole pattern or substrate material with a large difference affects the performance is not studied, and cross-pattern/substrate material shoeprint sequence recognition is our next work.Moreover,a larger volume dataset is under preparation.
Keywords

订阅号|日报