Current Issue Cover
Bayer阵列图像去马赛克算法综述

魏凌云1, 孙帮勇1,2(1.西安理工大学印刷包装与数字媒体学院, 西安 710048;2.中国科学院西安光学精密机械研究所光谱成像技术重点实验室, 西安 710119)

摘 要
Bayer阵列图像去马赛克技术是对稀疏采样的Bayer阵列图像进行RGB信息重建,图像重建质量是成像设备评价的重要因素之一,同时也对其他计算机视觉任务(如图像分割、人脸识别)产生影响。随着深度学习方法的快速发展,图像去马赛克领域提出了多种高性能算法。为了便于研究者更全面了解图像去马赛克算法的原理和研究进展,本文对该领域的经典算法和深度学习算法进行综述。首先对Bayer采样阵列原理和图像去马赛克技术进行概述。然后将现有方法分为传统方法和基于深度学习方法两类进行总结,同时根据去马赛克任务是否具有独立性,将深度学习方法分为独立去马赛克任务和联合去马赛克任务两类,分析不同方法的原理和优缺点,重点阐述基于深度学习的去马赛克方法的网络结构和重建机理,介绍去马赛克领域常用的公共数据集和性能评价指标,并对图像去马赛克相关实验进行分析对比。最后,围绕网络深度、运算效率和实用性等方面分析了现阶段图像去马赛克技术面临的挑战及未来发展方向。目前,基于深度学习的图像去马赛克方法已成为主流发展方向,但仍然存在计算成本较高、实际应用性不强等问题。因此,如何开发出重建精度高、处理时间短以及实用性强的图像去马赛克方法,是该领域未来重要的研究方向。
关键词
The review of demosaicing methods for Bayer color filter array image

Wei Lingyun1, Sun Bangyong1,2(1.College of Printing, Packaging and Digital Media, Xi'an University of Technology, Xi'an 710048, China;2.Key Laboratory of Spectral Imaging Technology, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an 710119, China)

Abstract
The Bayer array image demosaicing technology is one of the essential parts in the digital image signal processing process mechanism,which reconstructs the sparsely sampled Bayer array image into a completed RGB image.In terms of a single-sensor image acquisition of Bayer color filter array,single color channel value is just recorded by the position of each pixel.The other two color channels information needs to be reconstructed by the Bayer color filter array demosaicking technology.The reconstruction effect of qualified RGB information determines the accuracy of computer vision analysis like image segmentation and face recognition.Deep learning based Bayer array image demosaicing technology has promoted more high-performance algorithms nowadays.To recognize the principle and development of image demosaicing algorithms,we review some classic algorithms and analyze the key concepts.First,an overview is illustrated for the Bayer sampling array and image demosaicing technology.Next,we divide the existing methods into two categories in accordance with traditional methods and deep learning methods.The pros and cons of different methods are analyzed and the information transmission issue of the learning-based demosaicing network is focused on.We introduce traditional demosaicing methods in brief based on the basic principle,growth path and applications of the residual interpolation algorithms.Specifically,we divide the learning-based methods into single demosaic algorithms and joint multi-task demosaic algorithms derived of the independence of the demosaicing task.Single demosaicing algorithms can be divided into three categories of those are two-stage demosaicing network,three-stage demosaicing network,and end-to-end demosaicing network.Joint multi-task demosaic algorithms are mostly based on end-to-end multiple networks structure like full convolution,residual,densely connected,U-Net,feature pyramid,and generative adversarial network (GAN).At the same time,we compare seven representative methods to the two benched datasets of Kodak and McMaster,including three traditional methods and four deep learning methods.In addition,we introduce common public datasets and performance evaluation metrics peak signal-to-noise ratio (PSNR) and structural similarity in the field of demosaicing.Finally,we analyze the technical problems faced by the current image demosaicing technology and the future development direction from the perspectives of its network depth,computing efficiency,and feasibility.Our experimental results show that the deep learning based method surpasses the traditional methods in terms of Objective quality evaluation indicators and subjective vision.We clarified that the highest PSNR value of the learning-based methods on the Kodak dataset surpassed the traditional methods by 12.73 dB,and the highest PSNR value of the learning-based methods on the McMaster dataset surpassed the traditional methods by 5.10 dB.The traditional method obtains the estimated value of the color image via the degradation model of the Bayer array image and some prior information.However,some distortion issues appear in the clear edges and multi-textured areas of the reconstructed image,such as color artifacts and checkerboard effects.Current deep learning methods have become the main development prospects in the field of demosaicing.Although the reconstructed image of learning-based methods is more similar to the original image than the traditional method on the visual effect and accuracy,it also has the problem of high computational cost and poor practical applicability.It is challenged to guarantee the accuracy of network algorithms and reduce the computational cost of the network both.Therefore,a high-performance and effective image demosaicing method has to be developed further in terms of datasets option,network structure optimization,and improvement of computing efficiency.
Keywords

订阅号|日报