Current Issue Cover


摘 要
目的 跨年龄素描-照片转换旨在根据面部素描图像合成同一人物不同年龄阶段的面部照片图像。该任务在公共安全和数字娱乐等领域具有广泛的应用价值,然而由于配对样本难以收集和人脸老化机制复杂等原因,目前鲜有文献研究。针对此情况,提出一种基于双重对偶生成对抗网络(Double Dual Generative Adversarial Networks,D-DualGANs)的跨年龄素描-照片转换方法。方法 该网络通过设置四个生成器和四个判别器,以对抗训练的方式,分别学习素描到照片、源年龄组到目标年龄组的正向及反向映射。使素描图像、照片图像的生成过程相结合,老化图像、退龄图像的生成过程相结合,分别实现图像风格属性和年龄属性上的对偶。并增加重构身份损失和完全重构损失以约束图像生成。最终使输入的来自不同年龄组的素描图像和照片图像,分别转换成对方年龄组下的照片和素描。结果 为香港中文大学面部素描数据集CUFS和香港中文大学面部素描人脸识别技术数据集CUFSF的图像制作对应的年龄标签,并依据标签将图像分成3个年龄组,共训练6个D-DualGANs模型以实现3个年龄组图像之间的两两转换。同非端到端的方法相比,本文方法生成图像的变形和噪声更小,且年龄平均绝对误差(MAE)更低,与原图像相似度的投票对比表明11-30素描与31-50照片的转换效果最好。结论 双重对偶生成对抗网络可以同时转换输入图像的年龄和风格属性,且生成的图像有效保留了原图像的身份特征,有效解决图像跨风格且跨年龄的转换问题。
Double dual generative adversarial networks for cross-age sketch-to-Photo translation

Wu Liuwei,Sun Rui,Kan Junsong,Gao Jun(School of Computer and Information,Hefei University of Technology,Hefei)

Objective Sketch-to-Photo translation has a wide range of applications in the public safety and digital entertainment arena. For example, it can help the police find fugitives and missing children or generate a avatar of social account. The existing algorithm of sketch-to-photo translation can only translate sketches into photos under the same age group, but does not solve the problem of Cross-age Sketch-to-Photo Translation. Cross-age sketch-to-photo translation characters also have a wide range of applications. For example, when the sketch image of the police at hand is out of date after a long time, the task can generate an aging photo based on outdated sketches to help the police find the suspect. Since paired cross-age sketches and photo images are difficult to obtain, no data sets are available. In order to solve the above problem, this paper combines Dual Generative Adversarial Networks (DualGANs) and Identity-Preserved Conditional Generative Adversarial Networks (IPCGANs) to propose Double Dual Generative Adversarial Networks (D-DualGANs). Method DualGANs have the advantage of two-way conversion without the need to pair samples. But it can only achieve a two-way conversion of an attribute, and can not achieve the conversion of two attributes at the same time. IPCGANs can complete the aging or rejuvenation of the face while retaining the personalized features of the person"s face, but it cannot complete the two-way change between different age groups. This article considers the span of age as a domain conversion problem. And considers the cross-age sketch-to-photo translation task as a problem of style and age conversion. Combine the characteristics of the above network to build Double Dual Generative Adversarial Networks. By setting up four generators and four discriminators to combat training. The method not only learns the mapping of the sketch domain to the photo domain and the mapping of the photo domain to the sketch domain,also learns the mapping of the source age group to the target age group and the mapping of the target age group to the original age group .In D-DualGANs ,the original sketch image or the original photo image is successively completed by four generators to achieve four-domain conversion to obtain cross-age photo images or cross-age sketch images and reconstructed same-age sketch images or reconstructed same-age photo images. The generator is optimized by measuring the distance between the generated cross-age image and the reconstructed image of the same age by full reconstruction loss. And using the identity retention module to introduce reconstructed identity loss to maintain the personalized features of the face. Eventually, the input sketch images and photo images from different age groups are converted into photos and sketches of the other age group. And this method does not require paired samples, overcoming the problem of paired samples of cross-age sketches and photos that do not currently. Result Experiments combine the images of the CUFS and CUSFS sketch photo datasets and produces corresponding age labels for each image based on the results of the age estimation software. According to the age label, the sketch and photo images in the datasets are divided into three groups of 11-30,31-50 and 50+,and each age group is evenly distributed. A total of six D-DualGANs models were trained to realize the two-two conversion between sketches and photographic images of the three age groups. That is,the 11-30 sketch and the 31-50 photo,the 11-30 sketch and the 50+ photo, the 31-50 sketch and the 11-30 photo, the 31-50 sketch and the 50+ photo, the 50+ sketch and the 31-50 photo. As there is little research on cross-age sketch-to-photo translation. In order to illustrate the effectiveness of the method, the generated image obtained by this method is compared with the generated image obtained by DualGANs and then by IPCGANs. Our images are of better quality with less distortion and noise. Using an age estimate CNN to judge the age accuracy of the generated image, the average absolute age error (MAE) of our method is lower than the direct addition of DualGANs and IPCGANs. In order to evaluate the similarity between the generated image and the original image. We invite volunteers unrelated to this study to determine whether the generated image is the same as the original image. The results show that the resulting aging image is more similar and the resulting younger image is worse. Among them, the 31-50 photos generated by 11-30 sketches are the same as the original image. Conclusion Double Dual Generative Adversarial Networks proposed in this paper learns mapping and inverse mapping between the sketch domain and the photo domain, and the mapping and inverse mapping between different age groups. It also converts both the age and style properties of the input image. Photo images of different ages can be generated from a given sketch image. Through the introduced reconstructed identity loss and complete identity loss, the generated image effectively retains the identity features of the original image, effectively solving the problem of image cross-style and cross-age translation. Double Dual Generative Adversarial Networks can be used as a general framework to solve other computer vision tasks that need to complete two attribute conversions at the same time. However, there are still some shortcomings in this method. For example, conversion between different age groups requires training different models, such as to achieve 11-30 sketches to 31-50 photos and 11-30 sketches to 50+ photos. It is necessary to train two D-DualGANs models separately. This is somewhat cumbersome in practical applications and can be used as an improvement direction in the future, so that training a network model can achieve conversion between all age groups.