胡张颖1, 周全1, 陈明举2, 崔景程1, 吴晓富1, 郑宝玉1(1.南京邮电大学;2.四川轻化工大学人工智能四川省重点实验室)
A survey of image deblurring
(Nanjing University of Posts and Telecommunications)
Image blurring refers to the loss of clarity and detail in an image during its capture or transmission due to factors such as motion of the lens or camera, lighting conditions, and other environmental variables. This loss of quality and usability can significantly influence the overall visual impact of the image. To mitigate such effects, the technique of image deblurring has emerged. Its purpose is to automatically predict the clear version of an image by constructing computer mathematical models that measure the blurriness of the image. The research and development of image deblurring algorithms have not only provided convenience for other tasks in the field of computer vision, such as object detection, but have offered assurance in various aspects of life, including security monitoring. Depending on the cause of the blurring, it can be mainly divided into motion blur, out-of-focus blur and Gaussian blur. Out-of-focus and Gaussian blurs are less prevalent and relatively easier to handle, whereas motion blur is more likely to occur in situations such as road traffic cameras, pedestrian movement, and fast-moving vehicles, making it a more critical issue to be addressed. After deblurring an image, evaluating the quality of the results becomes essential, which is carried out using methods for image quality assessment (IQA) , categorized as either subjective or objective. Objective evaluation methods can be divided into three types: full-reference (FR) , reduced reference (RR) , and no reference (NR) . Due to constraints in resources, objective evaluation methods make up the majority of IQA approaches. The process of image blurring can be represented as the convolution of a clear image with a blur kernel, accompanied by greater or lesser degrees of noise. Therefore, there are two types of image deblurring: non-blind image deblurring (NBID) and blind image deblurring (BID). Non-blind deblurring involves the restoration of an image with a known blur kernel, requiring prior knowledge of the blur kernel"s parameters. On the other hand, blind deblurring aims to restore images with unknown blur kernels or unknown clear images, posing a more challenging problem to solve due to the increased number of unknown factors. In light of these considerations, we have embarked on a systematic and critical review to explore the recent advancements in image deblurring. First, a comprehensive and systematic introduction of the image deblurring is presented from the following two aspects: 1) the evolution of traditional image deblurring, 2) the development of deep learning-based image deblurring. From the perspective of traditional image deblurring, the existing image deblurring methods can be divided into two categories: non-blind deblurring and blind deblurring. Specifically, traditional NBID algorithms rely on prior knowledge of the blur kernel for the restoration process. Common methods include denoising-based methods and iteration-based methods. Traditional BID methods primarily involve estimating the blur kernel first and then transforming into a NBID problem. On the other hand, the kernel and clear image are often estimated iteratively until satisfactory results are obtained. The emerging deep learning methods extract blur image features through training neural network and employing logistic regression to update the models. Unlike traditional methods that require prior knowledge of the degree of image blur, deep learning-based methods are capable of directly processing blurry images without the need for prior estimation of the blur degree. From the perspective of network architecture, deep learning-based image deblurring algorithms can be classified into convolutional neural network based (CNN-based), recurrent neural network based (RNN-based), generative adversarial network based (GAN-based) and Transformer-based network. The CNN-based methods can learn the mapping between blurry and clear images by training on a large number of image pairs, which enables them to perform blind deblurring. These algorithms take advantage of parameter sharing and local receptive fields, reducing the number of model parameters and improving the accuracy of image feature extraction. Image deblurring based on RNN is a type of neural network model that can handle sequential data through learning the relationship between sequential data. The GAN-based deblurring approaches define image deblur problems as an advers-arial game between the generators and discriminators. The Transformer-based methods employ self-attention mechanism to encode global dependencies between different spatial positions, thereby better capturing the global information of the entire image. Our critical review is focused on the main concepts and discussions of the characteristics of each method for image deblurring from the perspective of the network architecture. Especially, we summarize the limitations of different deblurring algorithms. Secondly, we briefly introduce the popular public datasets. Then, we review some image deblurring literatures from two aspects, including traditional methods and deep learning-based methods, respectively. The capability of representative algorithms is analyzed derived from peak signal to noise ratio (PSNR) and structural similarity (SSIM) evaluation indexes in terms of laboratory for GoPro, HIDE and other datasets. Finally, this review has critically analyzed the conclusion, highlight the challenges in the image deblurring.