0 引言

1 行人再识别的相关工作

2 多形状局部区域神经网络结构

2.1 网络结构

多形状局部区域神经网络结构如图 1所示。本文网络所用的骨干网络是resnet50，它在很多行人再识别任务中都有着较好的表现^{[3-4, 21]}。基于resnet50，将conv4_X后的网络分成4个独立的分支，这4个分支的网络部分与原始resnet50结构的conv5_X相似。并在网络的conv3_X和conv4_X的输出位置上加入DropBlock。表 1展示了本文网络的结构设置。

图 1 多形状局部区域神经网络结构的示意图

Fig. 1 Illustration of multi-shape part network architecture

表 1 本文网络结构设置对比
Table 1 Comparison of the settings for three branches in our network

下载CSV

分支名	分支号	特征大小	特征维度	特征名
Global	1	12×4	256	$\mathit{\boldsymbol{g}}$
Part-H-2	2	24×8	256×2+256	$1{\mathit{\boldsymbol{p}}_i}|_1^2$
Part-H-3	3	24×8	256×3+256	$2{\mathit{\boldsymbol{p}}_i}|_1^3$
Part-Ⅴ-2	4	24×8	256×2+256	$3{\mathit{\boldsymbol{p}}_i}|_1^2$

表 1中Global分支在conv5_1 block中采用stride-2卷积进行下采样，然后进行全局平均池化(GAP)得到2 048维的特征${\mathit{\boldsymbol{f}}_1}$，再对${\mathit{\boldsymbol{f}}_1}$采用1×1卷积进行降维，得到256维的特征${\mathit{\boldsymbol{h}}_1}$。然后进行批归一化处理(BN)和ReLU，最终进入全连接层和交叉熵损失。该分支学习具有区分性信息的全局特征$\mathit{\boldsymbol{g}}$表示，因此将该分支命名为全局分支。

其余3个分支与全局分支共享相同的网络结构。本文conv5_1 block中不采用下采样操作，并且每个分支的输出特征在水平或竖直方向上均匀分割成若干条状特征作为局部特征$\left\{ {1{\mathit{\boldsymbol{p}}_i}|_1^2, 2{\mathit{\boldsymbol{p}}_i}|_1^3, 3{\mathit{\boldsymbol{p}}_i}|_1^2} \right\}$，局部特征的后面部分操作与全局特征分支相似，这样便可提取局部特征。本文对切分得到的条状局部特征采用全局最大池化(GMP)操作。将网络的这些分支称为Part-H(Ⅴ)-$N$分支，其中H表示水平方向切分特征图，Ⅴ表示竖直方向切分特征图，$N$表示切分的数量。如表 1中的第2个分支命名为Part-H-2, 最后一个分支命名为Part-Ⅴ-2。

2.2 局部特征的横向和竖向切分

如图 1所示，用了两个分支{Part-H-2，Part-H-3}做横向切分，分别将本分支最后1个卷积层的特征图切分成两份$\left\{ {1{\mathit{\boldsymbol{p}}_1}, 1{\mathit{\boldsymbol{p}}_2}} \right\}$和3份$\left\{ {2{\mathit{\boldsymbol{p}}_1}, 2{\mathit{\boldsymbol{p}}_2}, 2{\mathit{\boldsymbol{p}}_3}} \right\}$。多种横向切分方式能够得到多粒度的局部特征。由于输入的行人图像较窄，只采用一个将特征图切分成两份$\left\{ {3{\mathit{\boldsymbol{p}}_1}, 3{\mathit{\boldsymbol{p}}_2}} \right\}$的竖向分支。图 2展示了被切分的特征图所对应的图像切分。

图 2 Tensor $\boldsymbol{T}$的3种切分方式

Fig. 2 Three partition methods of Tensor $\boldsymbol{T}$

然后，在切分后的条状局部特征图$\left\{ {1{\mathit{\boldsymbol{p}}_i}|_1^2, 2{\mathit{\boldsymbol{p}}_i}|_1^3, 3{\mathit{\boldsymbol{p}}_i}|_1^2} \right\}$上进行全局池化操作，同一个行人不同的两幅图像只要在对应的局部特征上具有相同的部位特征，即便部位特征在对应局部特征图上的位置不一致，而在全局池化后依然能得到相差不多的结果。因此，横向切分分支和竖向切分分支相当于增强了网络提取特征的横向和竖向不对齐的容错率。

在图 3中，蓝色线条表示错误的切分方向，黄色线条表示正确的切分方向。图 3(a)中左右两张图包含的是同一行人，但两张图中的行人图像因为无法对齐，容易被判断成不同的两个行人。左图行人在图正中，右图的行人在图中偏下的位置，若采用黄线表示的横向切分，则上端切分出的大部分是背景和很少部分人头，而左图上端切分出的是人的上半身。若图 3(a)采用蓝线表示的竖向切分，则左右两图对应的切分后的局部特征包含相差不多的身体部位，再将局部特征做全局池化，忽略掉不重要的信息，左右两图便更容易匹配。图 3(b)左右两图包含的也是同一个行人。根据蓝线和黄线切分结果可以看出，采用黄线表示的横向切分能让两图的局部特征更好地匹配。因此，根据不同的图像，对应的合理的切分方式不同。

图 3 多形状局部区域

Fig. 3 Multi-shape part

((a) vertical strip; (b) horizontal strip)

采用softmax激活函数作为全局特征和每个局部特征的分类器，softmax损失表达为

$ L_{\mathrm{softmax}}=-\sum\limits_{i=1}^{N} \log \frac{\exp \left(W_{y_{i}}^{T} \boldsymbol{h}_{i}\right)}{\sum\limits_{k=1}^{c} \exp \left(W_{k}^{T} \boldsymbol{h}_{i}\right)} $

(1)

式中，$N$是mini-batch的大小，$C$是训练数据集的种类数, ${\mathit{\boldsymbol{h}}_i}$是输入的图像特征, $W_{y_{i}}^{T}$和$W_{k}^{T}$都表示全连接层的权重。

在训练阶段，每个分类器预测输入图像行人的身份标签，并用交叉熵损失作为监督；在测试阶段，将特征${\mathit{\boldsymbol{p}}_i}(i = 1, 2, \cdots, 8)$拼接起来作为输入图像的最终表示。

2.3 DropBlock

3 实验

3.1 实验数据集

3.2 实验设置及评价标准

3.3 在Market-1501数据集上的实验结果

Market-1501数据集的实验结果如表 2。MSPN w/o drop表示MSPN不带DropBlock的实验，MSPN w/o part-Ⅴ-2表示MSPN去除竖向切分的分支part-Ⅴ-2后的实验。

表 2 在数据集Market-1501上的比较结果
Table 2 Comparison results on Market-1501 dataset

下载CSV

/%
方法		mAP	CMC
			rank 1	rank 5	rank 10
	MSPN	84.57	94.51	97.68	98.40
本文	MSPN w/o drop	84.37	94.26	97.82	98.34
	MSPN w/o	83.61	93.81	97.34	98.43
	PCB+RPP[3]	81.6	93.8	97.5	98.5
	PCB[3]	77.4	92.3	97.2	98.2
	GLAD*[30]	73.9	89.9	-	-
	MultiScale[31]	84.1	88.9	-	-
局部特征的深度学习	PartLoss[8]	69.3	88.2	-	-
	PDC*[6]	63.4	84.4	92.7	94.9
	MultiLoss[2]	64.4	83.9	-	-
	PAR[12]	84.9	88.2	90.5	94.7
	HydraPlus[11]	85.1	76.9	91.3	94.5
	MultiRegion[33]	41.2	66.4	85.0	90.2
	DML[34]	68.8	87.7	-	-
	Transfer[35]	65.5	83.7	-	-
全局特征的深度学习	Triplet Loss[36]	69.1	84.9	94.2	-
	PAN[37]	63.4	82.8	-	-
	SVDNet[38]	62.1	82.3	92.3	95.2
	SOMAnet[39]	47.9	73.9	-	-
注：*表示需要辅助部件标签的方法，加粗数值表示最优结果，“-”代表原论文没有给出结果，w/o表示没有采用。

实验结果表明基于局部的方法通常得到更好的评估分数。本文提出的网络MSPN依靠合理的局部区域划分方式取得最好的结果。不带part-Ⅴ-2分支的MSPN与MSPN相比mAP低0.96%，rank-1低0.5%，MSPN相比没有DropBlock的MSPN带来0.2%的mAP提升，0.25%的rank-1提升，这表明DropBlock和part-Ⅴ-2分支对模型有正向提升，part-Ⅴ-2分支能够提高1.26%的mAP精度，具有较明显作用。

MSPN w/o part-Ⅴ-2与只用到横向切分的方法PCB+RPP进行对比，mAP有2.01%的提升，rank-1相差不多，性能好于PCB+RPP。

3.4 在数据集DukeMTMC-ReID上的实验结果

数据集DukeMTMC-ReID实验如表 3所示。

表 3 在数据集DukeMTMC-ReID上比较结果
Table 3 Comparison results on DukeMTMC-ReID dataset

下载CSV

/%
方法	mAP	rank-1
MSPN(本文)	75.96	87.12
MSPN w/o drop	75.30	86.98
MSPN w/o part-Ⅴ-2	74.52	86.06
SVDNet[38]	56.8	76.7
AOS[7]	62.1	79.2
HA-CNN[9]	63.8	80.5
GSRW[40]	66.4	80.7
DuATM[2]	64.6	81.8
PCB+RPP[3]	69.2	83.3
PSE+ECN[23]	75.7	84.5
DNN-CRF[21]	69.5	84.9
GP-reid[41]	72.8	85.2
注：加粗字体表示最优结果。

从表 3可以看出MSPN相比其他方法取得最好的实验结果。没有part-Ⅴ-2分支的MSPN与MSPN相比mAP低1.44%，rank-1低1.06%，MSPN相比没有DropBlock的MSPN带来mAP 0.78%的提升，0.14%的rank-1提升，这表明了竖向切分分支part-Ⅴ-2给网络带来了整体提升，DropBlock也展示出一定的效果。

MSPN w/o part-Ⅴ-2相比同样只有横向切分成局部区域的PCB mAP有5.32%的提升，rank-1有2.76%的提升。可以看到在数据集DukeMTMC-ReID上本文方法优势明显。

3.5 在数据集CUHK03上的实验结果

数据集CUHK03实验如表 4所示，MSPN相比其他方法取得最好的实验结果。

表 4 在数据集CUHK03上的比较结果
Table 4 Comparison results on CUHK03 dataset

下载CSV

/%
方法	mAP	rank-1
MSPN(本文)	66.52	68.73
MSPN w/o drop	66.12	68.35
MSPN w/o part-Ⅴ-2	65.85	67.95
PCB+RPP[3]	57.5	63.7
MLFN[42]	47.8	52.8
HA-CNN[9]	48.6	41.7
SVDNet[38]	37.3	41.5
PAN[20]	34	36.3
IDE[16]	19.7	21.3
注：加粗字体表示最优结果。

没有part-Ⅴ-2分支的MSPN与MSPN相比mAP低0.97%，rank-1低0.78%，MSPN相比没有DropBlock的MSPN带来mAP 0.4%的提升，0.38%的rank-1提升。

相比对特征图进行横向切分的PCB+RPP，MSPN在mAP上高出9.02%，rank-1高出5.03%，具有明显的优势。

图 4展示了在Market-1501数据集中通过几幅检索图像用MSPN检索得到的排序前10的结果。排序前10的图像中绿色边缘框的图像和检索图像属于同一人，红色边缘框则不为同一人。图 4中的行人图像检索结果展示了多形状局部区域网络(MSPN)很强的鲁棒性。无论被检索图像中的行人姿态、位置和视角如何变化都得到了很好的排序结果。

图 4 数据集Market-1501实验结果示例

Fig. 4 Example of experimental results on Market-1501 datasets by MSPN

4 结论

参考文献

[1] Cheng D, Gong Y H, Zhou S P, et al. Person re-identification by multi-channel parts-based CNN with improved triplet loss function[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE, 2016: 1335-1344.[DOI: 10.1109/CVPR.2016.149]
[2] Li W, Zhu X T, Gong S G. Person re-identification by deep joint learning of multi-loss classification[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence. Melbourne, Australia, 2017: 2194-2200.[DOI: 10.24963/ijcai.2017/305]
[3] Sun Y F, Zheng L, Yang Y, et al. Beyond part models: person retrieval with refined part pooling (and a strong convolutional baseline)[C]//Proceedings of the 15th European Conference on Computer Vision. Munich, Germany: Springer, 2018: 501-518.[DOI: 10.1007/978-3-030-01225-0_30]
[4] Zhang X, Luo H, Fan X, et al. AlignedReID: surpassing human-level performance in person re-identification[EB/OL].2018-11-22[2019-06-02].https://arxiv.org/pdf/1711.08184.pdf.
[5] Zhao H Y, Tian M Q, Sun S Y, et al. Spindle net: person reidentification with human body region guided feature decomposition and fusion[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE, 2017: 907-915.[DOI: 10.1109/CVPR.2017.103]
[6] Su C, Li J N, Zhang S L, et al. Pose-driven deep convolutional model for person re-identification[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017: 3980-3989.[DOI: 10.1109/ICCV.2017.427]
[7] Huang H J, Li D W, Zhang Z, et al. Adversarially occluded samples for person re-identification[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018: 5098-5107.[DOI: 10.1109/CVPR.2018.00535]
[8] Yao H T, Zhang S L, Zhang Y D, et al. Deep representation learning with part loss for person re-identification[EB/OL].2017-11-16[2019-06-02].https://arxiv.org/pdf/1707.00798.pdf.
[9] Li W, Zhu X T, Gong S G. Harmonious attention network for person re-identification[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018: 2285-2294.[DOI: 10.1109/CVPR.2018.00243]
[10] Liu H, Feng J S, Qi M B, et al. End-to-end comparative attention networks for person re-identification[J]. IEEE Transactions on Image Processing, 2017, 26(7): 3492–3506. [DOI:10.1109/TIP.2017.2700762]
[11] Liu X H, Zhao H Y, Tian M Q, et al. Hydraplus-net: attentive deep features for pedestrian analysis[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017: 350-359.[DOI: 10.1109/ICCV.2017.46]
[12] Zhao L M, Xi L, Zhuang Y T, et al. Deeply-learned part-aligned representations for person re-identification[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017: 3219-3228.[DOI: 10.1109/ICCV.2017.349]
[13] Ghiasi G, Lin T Y, Le Q V. DropBlock: a regularization method for convolutional networks[C]//Proceedings of Advances in Neural Information Processing Systems. 2018: 10727-10737.
[14] Li W, Zhao R, Xiao T, et al. DeepReID: deep filter pairing neural network for person re-identification[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA: IEEE, 2014: 152-159.[DOI: 10.1109/CVPR.2014.27]
[15] Yi D, Lei Z, Liao S C, et al. Deep metric learning for person reidentification[C]//Proceedings of the 22nd International Conference on Pattern Recognition. Stockholm, Sweden: IEEE, 2014: 34-39.[DOI: 10.1109/ICPR.2014.16]
[16] Zheng L, Yang Y, Hauptmann A G. Person re-identification: past, present and future[EB/OL].2016-10-10[2019-06-02].https://arxiv.org/pdf/1610.02984.pdf.
[17] Ahmed E, Jones M, Marks T K. An improved deep learning architecture for person re-identification[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA: IEEE, 2015: 3908-3916.[DOI: 10.1109/CVPR.2015.7299016]
[18] Varior R R, Haloi M, Wang G. Gated siamese convolutional neural network architecture for human re-identification[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands: Springer, 2016: 791-808.[DOI: 10.1007/978-3-319-46484-8_48]
[19] Xiao T, Li H S, Ouyang W L, et al. Learning deep feature representations with domain guided dropout for person re-identification[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE, 2016: 1249-1258.[DOI: 10.1109/CVPR.2016.140]
[20] Zhong Z, Zheng L, Cao D L, et al. Re-ranking person re-identification with k-reciprocal encoding[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE, 2017: 3652-3661.[DOI: 10.1109/CVPR.2017.389]
[21] Bai X, Yang M K, Huang T T, et al. Deep-person: learning discriminative deep features for person re-identification[EB/OL].2018-07-04[2019-06-02].https://arxiv.org/pdf/1711.10658.pdf.
[22] Sun Y, Wang X G, Tang X O. Deeply learned face representations are sparse, selective, and robust[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA: IEEE, 2015: 2892-2900.[DOI: 10.1109/CVPR.2015.7298907]
[23] Li D W, Chen X T, Zhang Z, et al. Learning deep context-aware features over body and latent parts for person re-identification[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE, 2017: 7398-7407.[DOI: 10.1109/CVPR.2017.782]
[24] Girshick R. Fast R-CNN[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE, 2015: 1440-1448.[DOI: 10.1109/ICCV.2015.169]
[25] Jaderberg M, Simonyan K, Zisserman A, et al. Spatial transformer networks[C]//Proceedings of Advances in Neural Information Processing Systems[EB/OL].2016-02-04[2019-06-02].https://arxiv.org/pdf/1506.02025.pdf.
[26] Zheng L, Shen L Y, Tian L, et al. Scalable person re-identification: a benchmark[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE, 2015: 1116-1124.[DOI: 10.1109/ICCV.2015.133]
[27] Zheng Z D, Zheng L, Yang Y. Unlabeled samples generated by GAN improve the person re-identification baseline in vitro[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017: 3774-3782.[DOI: 10.1109/ICCV.2017.405]
[28] Felzenszwalb P, McAllester D, Ramanan D. A discriminatively trained, multiscale, deformable part model[C]//Proceedings of 2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, AK, USA: IEEE, 2008: 1-8.[DOI: 10.1109/CVPR.2008.4587597]
[29] Ristani E, Solera F, Zou R, et al. Performance measures and a data set for multi-target, multi-camera tracking[C]//Proceedings of European Conference on Computer Vision. Amsterdam, The Netherlands: Springer, 2016: 17-35.[DOI: 10.1007/978-3-319-48881-3_2]
[30] Wei L H, Zhang S L, Yao H T, et al. GLAD: global-local-alignment descriptor for pedestrian retrieval[C]//Proceedings of the 25th ACM International Conference on Multimedia. Mountain View, California, USA: ACM, 2017: 420-428.[DOI: 10.1145/3123266.3123279]
[31] Chen Y B, Zhu X T, Gong S G. Person re-identification by deep learning multi-scale representations[C]//Proceedings of 2017 IEEE International Conference on Computer Vision Workshops. Venice, Italy: IEEE, 2017: 2590-2600.[DOI: 10.1109/ICCVW.2017.304]
[32] Su C, Li J N, Zhang S L, et al. Pose-driven deep convolutional model for person re-identification[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017: 3980-3989.[DOI: 10.1109/ICCV.2017.427]
[33] Ustinova E, Ganin Y, Lempitsky V. Multi-region bilinear convolutional neural networks for person re-identification[C]//Proceedings of the 14th IEEE International Conference on Advanced Video and Signal Based Surveillance. Lecce, Italy: IEEE, 2017: 1-6.[DOI: 10.1109/AVSS.2017.8078460]
[34] Zhang Y, Xiang T, Hospedales T M, et al. Deep mutual learning[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018: 4320-4328.[DOI: 10.1109/CVPR.2018.00454]
[35] Geng M Y, Wang Y W, Xiang T, et al. Deep transfer learning for person re-identification.[EB/OL].2016-11-16[2019-06-02].https: //arxiv.org/pdf/1611.05244.pdf.
[36] Hermans A, Beyer L, Leibe B. In defense of the triplet loss for person re-identification[EB/OL].2017-11-21[2019-06-02].https://arxiv.org/pdf/1611.05244.pdf.
[37] Zheng Z D, Zheng L, Yang Y. Pedestrian alignment network for large-scale person re-identification[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2018. [DOI:10.1109/TCSVT.2018.2873599]
[38] Sun Y F, Zheng L, Deng W J, et al. SVDNet for pedestrian retrieval[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017: 3820-3828.[DOI: 10.1109/ICCV.2017.410]
[39] Barbosa I B, Cristani M, Caputo B, et al. Looking beyond appearances:synthetic training data for deep CNNs in re-identification[J]. Computer Vision and Image Understanding, 2018, 167: 50–62. [DOI:10.1016/j.cviu.2017.12.002]
[40] Shen Y T, Li H S, Xiao T, et al. Deep group-shuffling random walk for person re-identification[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018: 2265-2274.[DOI: 10.1109/CVPR.2018.00241]
[41] Si J L, Zhang H G, Li C G, et al. Dual attention matching network for context-aware feature sequence based person re-identification[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018: 5363-5372.[DOI: 10.1109/CVPR.2018.00562]
[42] Chang X B, Hospedales T M, Xiang T. Multi-level factorisation net for person re-identification[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018: 2109-2118.[DOI: 10.1109/CVPR.2018.00225]