目的 针对低质量浅浮雕表面的噪声现象,提出一种二次联合局部自适应稀疏表示和非局部组低秩近似的浅浮雕优化算法。方法 本文算法分为两个阶段。在第一阶段,将浅浮雕灰度图划分成大小相同的数据块,提取边界块并进行去噪。然后,分别对数据块进行稀疏表示和低秩近似处理。一方面,通过字典学习获得过完备字典和稀疏编码；另一方面,利用K-means聚类算法将事先构建的外部字典库划分成k类,从k个簇中心匹配每个数据块的相似块并组成相似矩阵,依次进行低秩近似和特征增强处理。最后通过最小二乘法求解,重建并聚合新建数据块以得到新的高度场。第二阶段与第一阶段的结构相似,主要区别在于改用重建高度场的非局部自身相似性来实现块匹配。结果 在不同图像压缩率下(70%,50%,30%),对比本文算法与BM3D、WNNM、STROLLR、TWSC四个方法的浅浮雕重建结果,发现BM3D和STROLLR方法的特征保持虽好但平滑效果最差,WNNM方法出现模型破损现象,TWSC方法的平滑效果比BM3D和STROLLR方法更好,但特征也同时被光顺化。相比之下,本文算法在浅浮雕的特征增强、平滑和去噪等方面都展现出更好的性能。结论 本文综合数据块的局部稀疏性和数据块之间的非局部相似性对粗糙的浅浮雕模型进行二次高度场重建。实验结果表明,本文算法有效改善了现有浅浮雕模型的质量,提高了模型的整体视觉效果,为浅浮雕的优化提供了新的方法。
Bas-relief optimization based on twice-joint sparse representation and low-rank approximation
Zhang Qiankan,Ji Zhongping,Fu Xiaofeng(Institute of Graphics and Image,Hangzhou Dianzi University)
Abstract: Objective The bas-relief is a semi-stereoscopic sculpture between 2D and 3D that is usually attached to a plane. It, often used for decoration of buildings, coins, badges, ceramics and utensils, takes up less space and enhances stereo feeling mainly through lines. The research on bas-relief is mainly focused on two modeling techniques of 3D models and 2D images. It is carried out in the following three aspects: enhancing the continuity of the height field, preserving the original structure and details, and avoiding the attenuation of the stereoscopic effect. The bas-relief modeling technique based on a 3D model starts from the aspects of spatial domain, gradient domain or normal domain, and compresses the depth values of the 3D model in a given visual direction to generate the bas-relief model. The bas-relief modeling technique based on a 2D image extracts the gray information from the image and converts it into depth information, thus completing the reconstruction from the image to the bas-relief model. However, little attention has been paid to the repair and optimization of the existing bas-reliefs. In practice, there are many low-quality bas-relief models. Sometimes, in order to reduce the file size or protect the work, bas-relief models often appear as grayscale images during storage or transmission, such as lossy compressed JPG images. The quality of the bas-reliefs directly transformed from these lossy compressed images is quite rough. And the model surfaces present obvious block distribution and boundary noise, which seriously reduce the overall visual effect. Moreover, the greater the degree of image compression is, the more obvious the model noise will be. It can be seen that the lossy compression of the grayscale image really reduces the quality and the overall visual effect of the bas-relief model. In the face of a large number of bas-relief grayscale images on the network, how to recover high-quality bas-relief models is a problem that is worthy of further study. In this paper, a novel algorithm for bas-relief optimization is proposed，which is based on both local adaptive sparse representation and non-local group low-rank approximation. Method The algorithm has two stages. In the first stage, the initial gray image is first divided into patches of the same size, and the edge patches are extracted for denoising. Then, patches are processed by sparse representation and low-rank approximation respectively. On the one hand, the edge-denoised patches are trained by the K-SVD dictionary learning algorithm to obtain an overcomplete dictionary, which is used to sparsely decompose the noisy patches to obtain the corresponding sparse coding. On the other hand, the K-Means Clustering algorithm is used to classify the external dictionary library constructed in advance into K classes. Similar matrix for low-rank approximation is formed by similar patches selected from the K cluster centers. And further feature enhancement processing is performed on the low-rank approximation results. Combined with the results of edge denoising, sparse code reconstruction and feature enhancement, new patches are generated by a least squares solution to reconstruct a new height field. The second stage is similar to the structure of the first stage, and the main difference is the block matching operation. In the first stage, although the noisy patches are smoothed and denoised by external clean data, it constrains the consistency between patches and similar patches in the external dictionary library, resulting in insufficient smoothness between the patches of the height field. Therefore, in the second stage, block matching is realized by non-local similarity of the reconstructed height field to ensure the consistency between patches, thereby improving smoothing effect of bas-relief. Result By comparing bas-relief reconstruction results of the proposed method with BM3D (Block-Matching and 3D filtering), WNNM (Weighted Nuclear Norm Minimization), STROLLR (Sparsifying Transform Learning and Low-Rank), TWSC (Trilateral Weighted Sparse Coding) at different image compression rates (70%, 50%, 30%, respectively), we found that BM3D and STROLLR have better feature retention, while their smoothing effect behaves worst. WNNM shows a model breakage. The smoothing effect of TWSC is better than BM3D and STROLLR, but the features are also smoothed at the same time, resulting in inconspicuous details. The proposed method is further compared with Shape from Shading (SFS), which is an image-based 3D model reconstruction method for transforming 2D images into 3D models. Under different illumination conditions, SFS calculates the depth information by the shadow and brightness of the image, and then combines with the reflected illumination model to realize the reconstruction of 3D model. The overall structure shape of 3D model generated by SFS is reasonable, but the following shortcomings exist. First, the reconstruction results of SFS are not fine enough to restore lines and features. Second, model produced by SFS is influenced by the light source, position and direction. Third, there are many bump grooves in the reconstruction model of SFS. With the influence of light and shadow, local areas are either bright or dark. And the model looks complicated because of the lines generated by bump grooves. In contrast, the proposed method is more clear and intuitive, and exhibits better performance in terms of smoothing, feature enhancement and edge denoising of bas-relief. Experiment compares the operating efficiencies of different methods. The time taken by the four smoothing methods is in ascending order of BM3D, WNNM, TWSC, STROLLR. The proposed method takes longer than the above four methods. For a grayscale image of , the average computing time of the proposed method is 1000s, which is about 1.5 times of STROLLR. And the time spent by SFS is only about 40% ~ 60% of the proposed method. Although the computing time of the proposed method is higher, its optimization ability for 3D models is stronger and the visual effect is more significant. Conclusion The sparse representation contains the differences between patches, effectively retaining the details of bas-relief. The low-rank approximation contains the correlation between patches and has a good smoothing effect. The proposed method achieves the complementary of sparse representation and low rank approximation, and comprehensively considers the local sparsity of patches and the non-local similarity between patches. Experiment shows that the proposed method effectively improves the overall visual effect of bas-relief models and provides a new method for bas-relief optimization.