浏览全部资源
扫码关注微信
Received：18 January 2018，
Revised：2018-4-4，
Published：16 September 2018
移动端阅览
目的
2
目前在图像补全领域研究的重点和难点是补全具有复杂结构信息和丰富纹理信息的大破损区域的图像。传统的基于样本块的图像补全算法主要采用规则的模板块和匹配块来进行补全，补全过程中不能充分利用图像的结构或纹理的不规则信息，从而影响算法修复的精度和效率。针对这一问题，本研究提出一种基于不规则块的图像补全算法。
方法
2
在该算法中，首先利用结构稀疏度来区分图像的结构信息和纹理信息并基于结构稀疏度和置信度计算破损区域边界点的优先级，然后选择优先级最高的点构造规则模板块。对处于复杂结构区域的模板块，如果其邻域含有已知的结构信息，则膨胀该规则模板并利用其周围的结构信息来辅助构造不规则模板块。接下来，在图像完好区域内搜索与该模板块对应的匹配块，如果该匹配块的邻域包含有效的结构信息，则膨胀该匹配块并补充其周围的结构信息来完善该不规则匹配块。最后，利用该不规则匹配块补全破损区域。对于补全过程中块间接缝造成的视觉不连通问题，本研究利用图像的纹理信息来进行修饰。
结果
2
将本文算法与4种修复效果较好的算法（3种基于规则块的算法和1种基于局部敏感哈希的修复算法）进行对比，通过8组经典图像进行实例验证，采用客观评价指标峰值信噪比PSNR和主观视觉连通性进行评价，结果表明本文提出的算法峰值信噪比相较4种对比算法均有04 dB的提高，且在补全的精细度和视觉连通性方面有更佳的效果。
结论
2
本文算法在补全含有较复杂结构和丰富纹理的破损自然图像、壁画图像和目标物体移除上有较好的修复效果，普适性较强。
Objective
2
The highlight and difficult point of the current research in terms of image completion is to complete the image of a large damaged region with complex structure and rich texture information. Traditional exemplar-based completion algorithms mainly adapt the regular sample and matching patches to complete the damaged region. During the completion process
the irregular information
especially the local irregular information
of the image structure and texture cannot be fully used. This condition affects the accuracy and efficiency of the algorithm. To solve this problem
an image completion method with irregular patch is proposed in this paper.
Method
2
In our approach
the structure sparsity is used to distinguish the structure information and texture information of the input image first. Thus
a patch with a structure sparsity higher than a threshold is marked as a structural patch
and a patch with a structure sparsity lower than a threshold is regarded as a texture patch. Moreover
the patch priority of pixels on the boundary of the damaged region is calculated on the basis of the structure sparsity and confidence terms. Then
the pixel of the highest priority is selected as the center of the sample patch. If the neighborhood of a sample patch
which is in the complex structure area of the image
contains the known structure information
then this regular patch is dilated to construct its corresponding irregular sample patch by using structure information in its neighborhood. Then
the matching patch is being searched in the known part of the image. If the neighborhood of this matching patch contains a valid structure information
then the matching patch is dilated
and the structure information in its neighborhood is supplemented to construct its corresponding irregular matching patch. Finally
this irregular matching patch is used to complete its corresponding damaged region. The steps of constructing irregular patches specifically include transforming the inspection patch into the grayscale patch
using Gaussian blurring to denoise
extracting the structural information by using the Canny operator
integrating the structural information and original image patch to form an irregular mask
and picking the mask into the source image to obtain the irregular patch. In terms of the problem of visual disconnectivity
which is caused by seams during the completion process
the texture information of the image is used for modification in the proposed algorithm. The specific method aims to map the overlapping region of the texture patches into a network
and then
the graph cut algorithm is used to cut the network to generate a new optimal seam.
Result
2
Our approach and four other completion algorithms
three of which are based on the regular patch and one is based on sensitive hash
are used to complete eight classic image benchmarks. The efforts of these algorithms are evaluated by objective and subjective evaluation
which are the peak signal-to-noise ratio (PSNR) and visual connectivity. In comparison with these four completion algorithms
the results show that the proposed algorithm can achieve a better result in the fineness of the completion as well as the visual connectivity. Moreover
the PSNR of the proposed algorithm is improved by 0 dB to 4 dB.
Conclusion
2
The proposed algorithm improves the traditional exemplar-based completion algorithms
which cannot fully use the image irregularity information
and enhances the accuracy. Simultaneously
the texture information of the image is used to modify the visual disconnectivity so the visual effect of the image is improved. The proposed algorithm has a good application in the restoration of damaged natural images and cultural relics with both geometric structure and rich texture as well as the removal of target objects. Thus
the proposed algorithm is generalizable.
Zhang H Y, Peng Q C. A survey on digital image inpainting[J]. Journal of Image and Graphics, 2007, 12(1):1-10.
张红英, 彭启琮.数字图像修复技术综述[J].中国图象图形学报, 2007, 12(1):1-10. [DOI:10.3969/j.issn.1006-8961.2007.01.001]
Bertalmio M, Sapiro G, Caselles V, et al. Image inpainting[C]//Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM Press, 2000: 417-424. [ DOI:10.1145/344779.344972 http://dx.doi.org/10.1145/344779.344972 ]
Chan T F, Shen J H. Mathematical models for local nontexture inpaintings[J]. SIAM Journal on Applied Mathematics, 2002, 62(3):1019-1043. [DOI:10.1137/S0036139900368844]
Chan T F, Shen J H. Nontexture inpainting by curvature-driven diffusions[J]. Journal of Visual Communication and Image Representation, 2001, 12(4):436-449. [DOI:10.1006/jvci.2001.0487]
Tsai A, Yezzi A, Willsky A S. Curve evolution implementation of the Mumford-Shah functional for image segmentation, denoising, interpolation, and magnification[J]. IEEE Transactions on Image Processing, 2001, 10(8):1169-1186. [DOI:10.1109/83.935033]
Drori I, Cohen-Or D, Yeshurun H. Fragment-based image completion[J]. ACM Transactions on Graphics, 2003, 22(3):303-312. [DOI:10.1145/1201775.882267]
Criminisi A, Pérez P, Toyama K. Region filling and object removal by exemplar-based image inpainting[J]. IEEE Transactions on Image Processing, 2004, 13(9):1200-1212. [DOI:10.1109/tip.2004.833105]
Kwatra V, Essa I, Bobick A, et al. Texture optimization for example-based synthesis[J]. ACM Transactions on Graphics, 2005, 24(3):795-802. [DOI:10.1145/1186822.1073263]
Kuo T Y, Su P C, Kuan Y P. SIFT-guided multi-resolution video inpainting with innovative scheduling mechanism and irregular patch matching[J]. Information Sciences, 2016, 373:95-109. [DOI:10.1016/j.ins.2016.08.091]
Sun J, Yuan L, Jia J Y, et al. Image completion with structure propagation[C]//Proceeding SIGGRAPH '05 ACM SIGGRAPH 2005 Papers. Los Angeles, California: ACM Press, 2005: 861-868. [ DOI:10.1145/1073204.1073274 http://dx.doi.org/10.1145/1073204.1073274 ]
Tang C W, Hu X, Chen L, et al. Sample-based image completion using structure synthesis[C]//Proceedings of the 20th IEEE International Conference on Image Processing. Melbourne, VIC, Australia: IEEE, 2013: 1336-1340. [ DOI:10.1109/icip.2013.6738275 http://dx.doi.org/10.1109/icip.2013.6738275 ]
Wang M, Zhai D H, Nie H Y, et al. Image inpainting with weight variation of neighborhood window[J]. Journal of Image and Graphics, 2015, 20(8):1000-1007.
王猛, 翟东海, 聂洪玉, 等.邻域窗口权重变分的图像修复[J].中国图象图形学报, 2015, 20(8):1000-1007. [DOI:10.11834/jig.20150802]
Zeng J X, Wang C. Image completion based on redefined priority and image division[J]. Journal of Image and Graphics, 2017, 22(9):1183-1193.
曾接贤, 王璨.基于优先权改进和块划分的图像修复[J].中国图象图形学报, 2017, 22(9):1183-1193. [DOI:10.11834/jig.170054]
Mairal J, Elad M, Sapiro G. Sparse representation for color image restoration[J]. IEEE Transactions on Image Processing, 2008, 17(1):53-69. [DOI:10.1109/TIP.2007.911828]
Shen B, Hu W, Zhang Y M, et al. Image inpainting via sparse representation[C]//IEEE International Conference on Acoustics, Speech and Signal Processing. Taipei, Taiwan: IEEE, 2009: 697-700. [ DOI:10.1109/icassp.2009.4959679 http://dx.doi.org/10.1109/icassp.2009.4959679 ]
Xu Z B, Sun J. Image inpainting by patch propagation using patch sparsity[J]. IEEE Transactions on Image Processing, 2010, 19(5):1153-1165. [DOI:10.1109/tip.2010.2042098]
Canny J. A computational approach to edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986, PAMI-8(6):679-698. [DOI:10.1109/TPAMI.1986.4767851]
Kwatra V, Essa I, Turk G, et al. Graphcut textures: image and video synthesis using graph cuts[C]//Proceedings SIGGRAPH'03. San Diego, California: ACM, 2003: 277-286. [ DOI:10.1145/1201775.882264 http://dx.doi.org/10.1145/1201775.882264 ]
Greig D M, Porteous B T, Seheult A H. Exact Maximum A Posteriori Estimation for Binary Images[J]. Journal of the Royal Statistical Society, 1989, 51(2):271-279.
Boykov Y, Kolmogorov V. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(9):1124-1137. [DOI:10.1109/TPAMI.2004.60]
Barnes C, Shechtman E, Finkelstein A, et al. PatchMatch:a randomized correspondence algorithm for structural image editing[J]. ACM Transactions on Graphics, 2009, 28(3):24. [DOI:10.1145/1531326.1531330]
Shih T K, Chang R C, Lu L C, et al. Adaptive digital image inpainting[C]//Proceedings of the 18th International Conference on Advanced Information Networking and Applications. Fukuoka, Japan: IEEE Computer Society, 2004: 71-76. [ DOI:10.1109/AINA.2004.1283890 http://dx.doi.org/10.1109/AINA.2004.1283890 ]
Patil B C, Mahajan P K J. Image inpainting using coherent sensitive hashing[J]. International Journal of Advance Engineering and Research Development, 2015, 2(10):47-51. [DOI:10.21090/ijaerd.02109]
相关作者
相关机构
京公网安备11010802024621
