Current Issue Cover
联合局部约束的邻域嵌入人脸超分辨率重建

黄福珍, 周晨旭, 何林巍(上海电力学院自动化工程学院, 上海 200090)

摘 要
目的 现有的基于邻域嵌入的人脸超分辨率重建算法只利用了低分辨率图像流形空间的几何结构,而忽略了原始高分辨率图像的流形几何结构,不能很好的反映高低分辨率图像流形几何结构的关系。此外,其对同一幅图像中的不同图像块选取固定数目的最近邻域图像块,从而导致重建质量的下降。为了充分利用原始高分辨率图像空间的几何结构信息,提出基于联合局部约束和自适应邻域选择的邻域嵌入人脸超分辨率重建算法。方法 该方法结合待重构图像与低分辨率图像样本库的相似性约束与初始高分辨图像与高分辨率图像样本库的相似性约束,形成约束低分辨率图像块的重构权重,并利用该重构权重估计出高分辨率的人脸图像,同时引入自适应邻域选择的方法。结果 在CAS-PEAL-R1人脸库上的实验结果表明,相较于传统的基于邻域嵌入的人脸超分辨率重建方法,本文算法在PSNR和SSIM上分别提升了0.39 dB和0.02。相较于LSR重建方法,在PSNR和SSIM上分别提升了0.63 dB和0.01;相较于LcR重建方法,在PSNR和SSIM上分别提升了0.36 dB和0.003 2;相较于TRNR重建方法,在PSNR和SSIM上分别提升了0.33 dB和0.001 1。结论 本文所提的重建方法在现有人脸数据库上进行实验,在主观视觉和客观评价指标上均取得了较好的结果,可进一步适用于现实监控视频中人脸图像的高分辨率重建。
关键词
Face super-resolution reconstruction method based on joint local constraint neighbor embedding

Huang Fuzhen, Zhou Chenxu, He Linwei(College of Automation Engineering, Shanghai University of Electric Power, Shanghai 200090, China)

Abstract
Objective Face super-resolution reconstruction is a technique based on an observed low-quality, low-resolution face image. Combined with prior knowledge, which is acquired from learning high- and low-resolution training sample pairs, face super-resolution reconstruction can estimate clear high-resolution face images and increase the resolution of the image to effectively improve the image visual effects. The existing face super-resolution reconstruction algorithm mainly adopts the method of neighborhood embedding, but the traditional reconstruction algorithm based on neighborhood embedding has drawbacks. For example, the algorithm only utilizes the geometric structure of the low-resolution image manifold space while ignoring the manifold geometry of the original high-resolution image, so it cannot effectively reflect the relationship between the high- and low-resolution image manifold geometry. This situation also causes a significant loss in available information of the original high-resolution images and eventually leads to a reconstructed face image with incomplete local details, image blur, and other issues. In addition, the method is based on the image super-resolution reconstruction, where a fixed number of nearest neighborhood image blocks are selected for different image blocks in the same image. Both methods are based on the assumption that the high- and low-resolution image block manifolds have a consistent geometric structure. However, a “one-to-many” correspondence exists between the high-and low-resolution images, and the assumptions of the proposed manifold geometry consistency do not conform to the actual situation. The neighborhood relationship of the low-resolution image obtained after the original image is degraded and cannot truly reflect the neighborhood relationship of the original image, resulting in a decline in the quality of reconstruction. To fully utilize the geometric structure information of the original high-resolution image space, this work proposes a neighborhood embedded face super-resolution reconstruction algorithm based on joint local constraints and adaptive neighborhood selection. Method To solve the problem caused by the algorithm presented, this work proposes a face super-resolution reconstruction method based on joint local constraint neighbor embedding. The algorithm mainly includes two parts, namely, sample library and match reconstruction. Establishing the sample library mainly involves construction on high-and low-resolution image blocks. When matching the reconstruction process, to better characterize the neighborhood relation of the original image block, this study introduces the following constraints: the distance between the input low-resolution image and the low-resolution sample is taken as one of the constraints, and the distance between the initial high-resolution image obtained by the input low-resolution image interpolation and the high-resolution sample is assumed as another constraint. The neighborhood of the low-resolution image block in the low-resolution image block sample and the initial high-resolution image block in the high-resolution image block sample are used as local constraints to solve the optimal reconstruction weight coefficients. The two constraints are combined to effectively maintain the geometric structure of the image, forming a reconstructed weight coefficient that constrains the low-resolution image block. For each image block obtained from the input low-resolution face image, the optimal reconstruction weight is solved by minimizing the reconstruction error. After determining the optimal weight, the reconstructed high-resolution image block can be obtained by weighted summing and using the weight and the corresponding high-resolution image sample block. Then, the high-resolution face image is estimated by using the reconstructed weight. At the same time, the method of adaptive neighborhood selection is introduced. Result In this study, the experiment is conducted on the CAS-PEAL-R1 face database, and the proposed reconstruction algorithm is compared with the relatively advanced face super-resolution reconstruction algorithm to verify the effectiveness of the algorithm. Furthermore, the subjective image of the experimental results is given, and the image quality of the reconstructed results of different methods is evaluated by the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). The experimental results show that, compared with the traditional face super-resolution reconstruction method based on neighborhood embedding, the subject image obtained by the proposed algorithm has more details on facial features and is more similar to the original high-resolution face image. Moreover, the contrast of the objective indicators, which were raised to 0.39 and 0.02 dB in the PSNR and SSIM, respectively, shows the superiority of the algorithm. Compared with the indicators for the LSR reconstruction method, the PSNR and SSIM were improved by 0.63 dB and 0.01, respectively. Compared with the indicators for the LCR reconstruction method, the PSNR and SSIM were improved by 0.36 dB and 0.003 2, respectively. Compared with the indicators for the TRNR reconstruction method, the PSNR and SSIM were improved by 0.33 dB and 0.001 1, respectively. Conclusion The reconstruction method described in this study is conducted on the existing face database. The facial image is unified and segmented through the experiment to determine the optimal experimental parameters, while ignoring the location information of the facial features. Compared with the indicators of other algorithms, the subjective visual and objective evaluation indicators of the proposed algorithm achieved good results, which can be further applied to a real surveillance video with high-resolution image reconstruction. Combined with the location information of facial features and a reasonable segmentation of human face images, the full use of the prior structure of face images for super-resolution reconstruction is a future research direction.
Keywords

订阅号|日报