Current Issue Cover
三元组约束的类通用扰动人脸图像去识别方法

王慧娇, 熊卓, 管军霖, 蔡鼎, 王丽(桂林电子科技大学)

摘 要
目的 人脸图像去识别是保护人脸隐私的一种手段,类通用扰动作为人脸图像去识别的一种方法,为每个用户生成专属扰动来抵御深度人脸识别系统的恶意分析行为。针对现有类通用扰动方法存在用户训练数据不足的问题以及进一步提升扰动保护效果的需要,提出基于三元组损失约束的类通用扰动生成方法,同时引入一种基于特征子空间方法扩充训练数据构建三元组所需的负样本。方法 首先将深度神经网络提取的用户人脸图像特征作为正样本,然后对单个用户所有正样本进行仿射组合构建特征子空间,再结合凸优化方法训练样本远离特征子空间,生成负样本扩充训练数据。之后对原始图像叠加随机扰动,提取特征得到待训样本。利用三元组函数约束扰动训练过程,使待训样本远离正样本的同时靠近负样本,并以余弦距离作为指标计算损失值。对训练生成的扰动施加一个缩放变换,得到用户的类通用扰动。结果 针对具有不同损失函数(ArcFace、SFace和CosFace)和网络架构(SENet、MobileNet和IResNet)的6个人脸识别模型在2个数据集上进行实验,与相关的4种方法进行比较均取得了最优效果。在Privacy-Commons和Privacy-Celebrities数据集上,相比已知最优的方法,扰动训练效率平均提升了66.5%,保护成功率平均提升了5.76%。结论 本文提出的三元组约束扰动生成方法,在兼顾扰动生成效率的同时,既缓解了训练样本不足的问题,又使类通用扰动综合了梯度攻击信息和特征攻击信息,提升了人脸隐私保护效果。 关键词:类通用扰动;三元组约束;人脸图像去识别;数据扩充;人脸隐私保护
关键词
Face image de-identification with class universal perturbations based on triplet constraints

wanghiujiao, xiongzhuo, guanjunlin, caiding, wangli(Guilin University of Electronic Technology)

Abstract
Objective With the development of face recognition technology, face images have been used as identity verification in many fields. Face images, as an important biometric feature, usually involve personal identity information. If the face images are illegally obtained and used by attackers, it may cause serious losses and harm to individuals. Protecting face privacy and security has always been an urgent problem. The de-identification of face image is conducted in this paper and the convenient and efficient face privacy protection method by using class universal perturbation is studied. The class universal perturbation method generates exclusive perturbation information for each user. Then the exclusive perturbation is superimposed on the face image to de-identify it, resisting the behavior of deep face recognizer maliciously analyzing user information. In view of the limited face images provided by users when using class universal perturbation to de-identify users, it usually faces the problem of insufficient samples. In addition, it is hard to extract face image features due to different shooting angles, which increases the difficulty of learning user features through class universal perturbation. At the same time, the protection scenario faced by class universal perturbation is complex. Class universal perturbation is generated from local proxy model and needs to have the ability to resist different face recognition models. Different face recognition models use different data sets, loss functions and network architectures, which increases the difficulty of generating class universal perturbation with transferability. In view of the problem of insufficient user training data and the need to further improve the protection effect of perturbation in the field of the class universal perturbation, a generation method of class universal perturbation constrained by the triplet loss function is proposed in this paper, which is named as FDUP-TC(face image de-identification with class universal perturbations based on triplet constraints). The negative samples are constructed based on feature subspace to augment the training data and obtain samples in triplets. Method Res-Net50 deep neural network is adopted to extract the features of user face images. The extracted features are used as positive samples for training. Then, the feature subspace is constructed by using three different affine combination methods (affine hull, convex hull and class center) of positive samples. The maximum distance between samples and feature subspace is solved by convex optimization method. The training samples are optimized along the direction away from the feature subspace and label the optimized samples as negative samples. Perturbations are randomly generated which as initial values for class general perturbations and added to the original image. Features are extracted from the perturbed images to obtain training samples. Positive samples, negative samples and training samples constitute the triplet required for training. The cosine distance as the measurement when training perturbations. The distance between training samples and positive samples is maximized, while the distance between training samples and negative samples is minimized. The training sample moves closer to the negative sample when the training sample is equidistant from the positive sample so that the perturbations can learn more adversarial information within a limited range. A scaling transformation is applied to the generated perturbation. The parts of the perturbation value greater than 0 are set to the upper limit value of the perturbation threshold, and the parts less than 0 are set to the lower limit value of the perturbation threshold. The class universal perturbation is obtained. Result The data required for the experiment are from the data sets of MegaFace challenge, MSCeleb-1M and LFW. Privacy-Common public data set representing ordinary users is constructed, Privacy-Celebrities celebrity data set representing celebrity users is constructed and test sets corresponding to both from the above three public data sets are constructed. Black box tests are conducted on the two data sets for face recognition models with different loss functions and network architectures. Three of the black box models use different loss functions, namely CosFace, ArcFace and SFace. The other three black box models use different network architectures, namely SENet, MobileNet and IResNet variants. The proposed FDUP-TC is compared with GD-UAP(generalizable data-free objective for crafting universal adversarial perturbations), GAP(generative adversarial perturbations), UAP(universal adversarial perturbations) and OPOM(one person one mask). In Privacy-Commons data set, the highest Top-1 protection success rate of each method in the face of different face recognition models is 8.7% (GD-UAP), 59.7% (GAP), 64.2% (UAP), 86.5% (OPOM) and 90.6% (FDUP-TC) respectively. The highest protection success rate of Top-5 is 3.5% (GD-UAP), 46.7% (GAP), 51.7% (UAP), 80.1% (OPOM) and 85.8% (FDUP-TC) respectively. Compared with the well-known OPOM method, the protection success rate increased by an average of 5.74%. In Privacy-Celebrities data set, the highest Top-1 protection success rate of each method in the face of different face recognition models is 10.7% (GD-UAP), 53.3% (GAP), 59% (UAP), 69.6% (OPOM) and 75.9% (FDUP-TC) respectively. The highest protection success rate of Top-5 is 4.2% (GD-UAP), 42.7% (GAP), 47.8% (UAP), 60.6% (OPOM) and 67.9% (FDUP-TC) respectively. Compared with the well-known OPOM method, the protection success rate increased by an average of 5.81%. The time spent to generate perturbations for 500 users is used as the indicator to measure the efficiency of each method. The time consumption of each method is 19.44 minutes (OPOM), 10.41 minutes (UAP), 6.52 minutes (FDUP-TC), 4.51 minutes (GAP), and 1.12 minutes (GD-UAP) respectively. The above experimental results verify the superiority of FDUP-TC method in face de-identification and its transferability on different models. The FDUP-TC with perturbation scaling transformation has an average protection success rate of Top-1 of 80% and 64.6% on Privacy-Commons and Privacy-Celebrities data sets respectively, and the FDUP-TC without perturbation scaling transformation has an average protection success rate of Top-1 of 78.1% and 62.5.1% on the two data sets respectively. The FDUP-TC method with perturbation scaling transformation increased the protection success rate by about 2% on average, proving the effectiveness of the perturbation scaling method. In addition to using convex hull to model user feature subspace and generate negative samples, the ways of constructing negative samples by using universal adversarial perturbation method FI-UAP(feature iterative universal adversarial perturbations) and FI-UAP+(feature iterative universal adversarial perturbations enhanced by incorporating intra-class interactions)and Gauss random perturbation are compared. The highest protection success rate of Top-1 of each method on Privacy-Commons and Privacy-Celebrities data sets is 85.6% (FI-UAP), 86% (FI-UAP+), 44.8% (Gauss) and 90.6% (convex hull) respectively. The average protection success rate of using convex hull method to construct negative samples on the two data sets is 4.9% higher than that of the suboptimal FI-UAP+ method, which verifies the rationality of the negative sample construction in this paper. Conclusion The proposed method in this paper based on the triple of positive samples, negative samples and training samples as constraints to obtain the class universal perturbation for face image de-identification. The negative samples are constructed from the original training data, which alleviates the problem of insufficient training samples. The class universal perturbation trained by the triple provides feature attack information. At the same time, the introduction of perturbation scaling increases the strength of class universal perturbation and makes the face image de-identification effect better. By comparing with GD-UAP, GAP, UAP and OPOM, four related methods, the superiority of this method in face de-identification is verified. Keywords: class universal perturbation; triplet constraint; face image de-identification; data augmentation; face privacy protection
Keywords

订阅号|日报