瞿左珉, 殷琪林, 盛紫琦, 吴俊彦, 张博林, 余尚戎, 卢伟(中山大学计算机学院)
An overview of Deepfake proactive defense techniques
Qu Zuomin, Yin Qilin, Sheng Ziqi, Wu Junyan, Zhang Bolin, Yu Shangrong, Lu Wei(School of computer science and engineering, Sun Yat-Sen University)
In recent years, with the development of generative adversarial network technology, facial manipulation technology has advanced significantly in both academia and industry. In particular, the deep face forgery model represented by Deepfake has been widely used on the Internet. The word "Deepfake" is a combination of "deep learning" and "fake". It refers to a face modification technology based on deep learning that is able to modify faces in videos and images, including face swapping, face expression editing and face attribute editing, etc. Deepfake can be roughly divided into two categories: identity-agnostic manipulation and identity-related manipulation. The method called face swapping is classified as identity-related manipulation, which aims to replace the target face area with the original face. While face expression editing and face attribute editing are classified as identity-agnostic manipulation, they attempt to modify the attributes of the face, such as expression, hair color, age and gender, etc., without transforming identity. On the one hand, Deepfake technology has been widely used in film special effects, advertising and entertainment apps, etc. For example, some films achieve more realistic and low-cost special effects through such technology. For the customer, one can personalize the model on screen according to one’s body dimension, color, and hair type before buying products. At the same time, Deepfake inspires an increasing number of entertainment applications that have greatly lowered the threshold of using this technology, such as ZAO, Meitu Xiuxiu, FaceApp, etc. Through these applications, users can easily replace the faces of actors in movies or television dramas with their own faces, or change their hair color or makeup at will. Deepfake forgery, On the other hand, is currently being applied to some scenarios that may cause adverse effects. For example, one of the most notorious Deepfake applications, DeepNude, attempts to replace the face of a porn actor with one of a star, causing serious damage to individual privacy and even the personal reputation of citizens. In addition, Deepfake with target attributes may pass the verification of commercial applications, threatening application security and harming the property of the person who has been impersonated. To date, some fake news in which the politician speaks in a speech that does not belong to him/her, also poses a serious threat to social stability and national security. Based on this, some defense methods of Deepfake forgery are proposed. Existing defense technologies can be toughly divided into two categories of passive defense and proactive defense. Passive defense is mainly based on detection. Despite their considerable accuracy, these detectors are simply passive measures against Deepfake attacks because they cannot eliminate the negative effects of fake content that has been generated and widely disseminated. In a nutshell, it is difficult to achieve prior defense and cannot intervene in the generation of Deepfake faces. Therefore, current mainstream thinking believes that proactive defense techniques are more defensive and practical. In contrast to passive defense, proactive defense disrupts Deepfake proactively by adding special adversarial perturbations or watermarks into source images or videos before they are shared online. When a malicious user attempts to use them for Deepfake forgery, the output of the Deepfake forgery model will be seriously damaged in visual quality and cannot be successfully forged, or even if the indistinguishable fake images are obtained, we can also trace the source through forged images to find the malicious user. This paper principally reviews the current Deepfake proactive defense techniques. Our overview is focused on the three perspectives as following: 1) A brief introduction to Deepfake forgery technologies and their impact; 2) A systematic summary of current proactive defense algorithms for Deepfake forgery, including technical principles, classification, performance, datasets and evaluation methods; 3) A description of the challenges faced by Deepfake proactive defense, and a discussion of its future directions. From the perspective of defense target, the Deepfake forgery proactive defense can be divided into proactive disruption defense technology and proactive forensics defense technology. For proactive disruption defense technology, it can be subdivided from the point of view of technical implementation into data poisoning defense methods, adversarial attack defense methods and latent space defense methods. The data poisoning defense method destroys Deepfake forgery in the training stage, which requires the faker to use the poisoned images as training data to train the Deepfake forgery model. While forgery destruction of the adversarial attack defense method works in the test stage. When the faker uses the well-trained Deepfake forgery model to manipulate face images with adversarial perturbations, the output image will be destroyed. This idea of defense based on adversarial attack is the most widely used in existing studies. When implementing latent space defense methods, perturbations are not added directly to the image. In contrast, the image will be first mapped into latent space, and this mapping is implemented with an elaborate transformation so that the image is protected from the threat of Deepfake forgery. It is noteworthy that this method relies heavily on the effect of GAN Inversion technology. We then give a brief introduction to evaluation methods and datasets used in proactive defense. The evaluation of defense technology is usually carried out from two aspects: the effect of disrupting the output of the Deepfake forgery model and the effect of maintaining the visual quality of disturbed images. They are generally evaluated in terms of pixel distance, feature distance, attack success rate. At the same time, some commonly used facial indicators such as structural similarity index measure (SSIM), frechet inception distance (FID) and normalization mean error (NME) are taken into consideration during evaluation. Finally, we expound the challenges faced by Deepfake proactive defense, mainly including the circumvention of proactive defense, the improvement of performance in black-box scenarios and practicality issues. In addition, we look forward to the future directions of proactive defense. More robust performance and better visual quality are identified as the two main concerns. In conclusion, our survey summarizes the principal concept and classification of Deepfake proactive defense, as well as the detailed explanation of various methods, evaluation metrics, commonly used datasets, main challenges, and prospects. We hope that it will serve as an introduction and guide for Deepfake proactive defense research.