浏览全部资源
扫码关注微信
收稿：2017-12-22，
修回：2018-4-7，
纸质出版：2018-10-16
移动端阅览
目的
2
针对传统基于样本块的图像修复算法中仅利用图像的梯度信息和颜色信息来修复破损区域时，容易产生错误填充块的问题，本文在Criminisi算法的基础上，利用结构张量特性，提出了一种改进的基于结构张量的彩色图像修复算法。
方法
2
首先利用结构张量的特征值定义新的数据项，以确保图像的结构信息能够更加准确地传播；然后利用该数据项构成新的优先权函数，使得图像的填充顺序更加精准；最后利用结构张量的平均相干性来自适应选择样本块大小，以克服结构不连续和错误延伸的缺点；同时在匹配准则中，利用结构张量特征值来增加约束条件，以减少错误匹配率。
结果
2
实验结果表明，改进算法的修复效果较理想，在主观视觉上有明显的提升，其修复结果的峰值信噪比（PSNR）和结构相似度（SSIM）都有所提高；与传统Criminisi算法相比，其峰值信噪比提高了1~3 dB。
结论
2
本文算法利用结构张量的特性实现了对不同结构特征的彩色破损图像的修复，对复杂的线性结构和纹理区域都有较理想的修复，有效地保持了图像边缘结构的平滑性，而且对大物体的移除和文字去除也有较好的修复效果。
Objective
2
The rapid development of multimedia information technology has led to images that have become the main carrier of information in people's lives.People communicate information through various means
such as voice
images
text
and video.Consequently
digital image inpainting technology has gradually attracted increasing attention
and its application fields are extensive.Digital image inpainting refers to the process of repairing or rebuilding missing information in damaged images by using a specific image inpainting algorithm
such that the observer cannot easily detect that the image has been repaired or damaged.Image inpainting technology has been used in many research areas
such as the restoration of old photos
the removal of image text
and the preservation of cultural relics.The traditional exemplar-based image inpainting algorithm only uses gradient information and the color information of the image to repair damaged areas
which can easily generate incorrect filling patches.In addition
the definition of the priority function is unreasonable
and thus causes the wrong filling order during inpainting
which affects the overall restoration effect.To solve these problems
an improved color image inpainting algorithm that is based on structure tensor is presented in this paper.
Method
2
Structure tensor is often used to analyze the local geometry of an image
which not only contains the intensity information of the local region
but also the main directions of the neighborhood gradient of a pixel and the degree of coherence of these directions.Its two eigenvalues can distinguish the edge
texture
and flat areas of an image.First
the proposed algorithm uses the structure tensor to define the data items to ensure that the structure information of the image can be transmitted accurately
and then uses the data items to form a new priority function for a more precise filling order.Second
different sizes of sample patches can be used to search for the best matching patch because an image has different structural features in different regions.Therefore
the size of the sample patch is adaptively selected according to the average coherence of the structure tensor.In other words
when the average coherence of the patch to be repaired is large
this patch is at the edge of the image and a small sample patch should be used; when the average coherence is small
it is in the flat region of the image and a large sample patch should be used.In this manner
when repairing complex damaged images
the continuity of the edge structure can be maintained; the flat area of the image can be effectively repaired.Finally
the traditional inpainting algorithm only uses the color information of the image to find the best matching patch
which renders the matching patch sub optimal.In this study
the eigenvalues of the structure tensor are added to the matching criteria to reduce the false matching rate.
Result
2
Experimental results show that the improved algorithm is more effective in subjective vision than the other related algorithms.Moreover
the improved algorithm can achieve good results for different types of damaged images and effectively maintain the smoothness of the edge structure of the image.Compared with the traditional Criminisi algorithm
the power signal-to-noise ratio of the result has improved by approximately 1~3 dB and enhanced structure similarity.In addition
the proposed algorithm has a higher running time than other algorithms because in the inpainting process
the proposed algorithm uses the adaptive sample patch size to search for the best matching patch
and when analyzing the local structural features of the image
it needs to calculate the coherence factor of the pixels.These steps consequently increase the running time and reduce the efficiency of image inpainting.
Conclusion
2
When the traditional algorithm repairs the strong damaged area of the edge
the structural integrity and the good visual effect of the image are difficult to balance.In this study
we use the structure tensor of the color image to analyze the structure and texture area of the image.This paper discusses a color image inpainting algorithm based on the structure tensor.The proposed algorithm first uses the eigenvalue of the structure tensor instead of the isophote line in the traditional algorithm to improve the data items
which can spread the structure information of the image more accurately.Then
the average coherence factor of the structure tensor is used to analyze the texture and structural features of the image to repair the different image structural features.Finally
the matching rate used to select the best matching patch is improved with the addition of the constraint of the structural tensor to the traditional matching criteria.The proposed algorithm can obtain a better visual effect for damaged images with different structural features.The proposed algorithm can also effectively maintain the structural integrity of the image
and the complex texture area does not exhibit a wrong filled patch.Moreover
the large object and text removals by the proposed algorithm also have a good restoration effect.Compared with related Criminisi algorithms
the proposed algorithm has a better repair effect on complex linear structure and texture regions and effectively improves the overall quality of image restoration.
Bertalmio M, Sapiro G, Caselles V, et al.Image inpainting[C]//Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques.New York, USA: ACM, 2000: 417-424.[ DOI: 10.1145/344779.344972 http://dx.doi.org/10.1145/344779.344972 ]
Shen J H, Chan T F.Mathematical models for local nontexture inpaintings[J].SIAM Journal on Applied Mathematics, 2002, 62(3):1019-1043.[DOI:10.1137/S0036139900368844]
Shen J H, Kang S H, Chan T F.Euler's elastica and curvature-based inpainting[J].SIAM Journal on Applied Mathematics, 2002, 63(2):564-592.[DOI:10.1137/S0036139901390088]
Tsai A, Yezzi A, Willsky A S.Curve evolution implementation of the Mumford-Shah functional for image segmentation, denoising, interpolation, and magnification[J].IEEE Transactions on Image Processing, 2001, 10(8):1169-1186.[DOI:10.1109/83.935033]
Criminisi A, Perez P, Toyama K.Region filling and object removal by exemplar-based image inpainting[J].IEEE Transactions on Image Processing, 2004, 13(9):1200-1212.[DOI:10.1109/TIP.2004.833105]
Ren S, Tang X H, Kang J L.Improved Criminisi algorithm with the texture and edge features[J].Journal of Image and Graphics, 2012, 17(9):1086-1091.
任澍, 唐向宏, 康佳伦.利用纹理和边缘特征的Criminisi改进算法[J].中国图象图形学报, 2012, 17(9):1085-1091. [DOI:10.11834/jig.20120906]
Zhou Y T, Li L, XiaK W.Research on weighted priority of exemplar-based image inpainting[J].Journal of Electronics (China), 2012, 29(1-2):166-170.[DOI:10.1007/s11767-012-0843-6]
Xu Z B, Sun J.Image inpainting by patch propagation using patch sparsity[J].IEEE Transactions on Image Processing, 2010, 19(5):1153-1165.[DOI:10.1109/TIP.2010.2042098]
Hou Y T, Peng J Y, Han D C.Research of improved Criminisi image resoration algorithm[J].Computer Engineering and Applications, 2015, 51(11):135-138, 144.
侯玉婷, 彭进业, 韩东辰.改进的Criminisi图像修复算法研究[J].计算机工程与应用, 2015, 51(11):135-138, 144. [DOI:10.3778/j.issn.1002-8331.1306-0276]
Choi J H, Hahm C H.An exemplar-based image inpainting method with search region prior[C]//Proceedings of the 2nd Global Conference on Consumer Electronics.Tokyo: IEEE, 2013: 68-71.[ DOI:10.1109/GCCE.2013.6664927 http://dx.doi.org/10.1109/GCCE.2013.6664927 ]
Patel K R, Jain L.A novel approach to exemplar based image inpainting[C]//Proceedings of 2016 International Conference on Communication and Signal Processing.Melmaruvathur, India: IEEE, 2016: 0810-0814.[ DOI: 10.1109/ICCSP.2016.7754257 http://dx.doi.org/10.1109/ICCSP.2016.7754257 ]
Ren S, Tang X H, Kang J L.An image inpainting algorithm combined with texture and edge features[J].Journal of Computer-Aided Design&Computer Graphics, 2013, 25(11):1682-1693.
任澍, 唐向宏, 康佳伦.纹理和边缘特征相结合的图像修复算法[J].计算机辅助设计与图形学学报, 2013, 25(11):1682-1693.
Liu Y, Liu C J, Zou H L, et al.A novel exemplar-based image inpainting algorithm[C]//Proceedings of International Conference on Intelligent Networking and Collaborative Systems.Taipei, Taiwan, China: IEEE, 2015: 86-90.[ DOI: 10.1109/INCoS.2015.15 http://dx.doi.org/10.1109/INCoS.2015.15 ]
Siadati S Z, Yaghmaee F, Mahdavi P.A new exemplar-based image inpainting algorithm using image structure tensors[C]//Proceeding of the 24th Iranian Conference on Electrical Engineering.Shiraz, Iran: IEEE, 2016: 995-1001.[ DOI: 10.1109/IranianCEE.2016.7585666 http://dx.doi.org/10.1109/IranianCEE.2016.7585666 ]
Wang J, Lu K, Pan D R, et al.Robust object removal with an exemplar-based image inpainting approach[J].Neurocomputing, 2014, 123:150-155.[DOI:10.1016/j.Neucom.2013.06.022]
Akl A, Saad E, Yaacoub C.Structure-based image inpainting[C]//Proceedings of 2017 International Conference on Image Processing Theory, TOOLS and Applications.Oulu, Finland: IEEE, 2017.[ DOI: 10.1109/IPTA.2016.7820976 http://dx.doi.org/10.1109/IPTA.2016.7820976 ]
Akl A, Yaacoub C, Donias M, et al.Texture synthesis using the structure tensor[J].IEEE Transactions on Image Processing, 2015, 24(11):4082-4095.[DOI:10.1109/TIP.2015.2458701]
Meur O L, Gautier J, Guillemot C.Examplar-based inpainting based on local geometry[C]//Proceedings of the 18th IEEE International Conference on Image Processing.Brussels, Belgium: IEEE, 2011: 3401-3404.[ DOI: 10.1109/ICIP.2011.6116441 http://dx.doi.org/10.1109/ICIP.2011.6116441 ]
Wang Z, Bovik A C, Sheikh H R, et al.Image quality assessment:from error visibility to structural similarity[J].IEEE Transactions on Image Processing, 2004, 13(4):600-612.[DOI:10.1109/TIP.2003.819861]
相关作者
相关机构
京公网安备11010802024621
