Current Issue Cover
联合深度学习和宽度学习的纹理样图自动提取(China VR推荐)

吴惠思, 梁崇鑫, 颜威, 文振焜(深圳大学)

摘 要
目的 纹理样图是指一张用于描述纹理特征的图像,在基于样图的纹理合成任务之中有着举足轻重的作用,决定了纹理合成任务的效果。纹理样图多样性在纹理合成任务中是至关重要的,它可以为合成的纹理带来更丰富、多样和逼真的外观,同时为艺术家和设计师提供了更大的创作灵感和自由度。当前,纹理样图的提取主要通过手工剪裁和算法自动提取的方式,从大量的图片中手工剪裁提取出高质量的纹理样图十分耗费纹理艺术家的精力和时间,并且该方式易受主观驱动且多样性受限。而目前最先进的纹理样图自动提取算法为基于卷积神经网络的 Trimmed T-CNN 模型,该模型则存在推理速度慢的问题。基于此,本文致力于利用互联网上丰富的图片资源,自动快速地从各种图片中裁剪出理想且多样的纹理样图,让用户有更多的选择。方法 本文提出一个结合深度学习和宽度学习的从原始图像中自动提取纹理样图的方法。为了获取理想的纹理样图,本文首先通过残差特征金字塔网络提取特征图旨在有效地从输入图像中识别样图候选者,然后采用区域候选网络快速自动地获取大量的纹理样图候选区域。接下来,本文利用宽度学习系统对纹理样图的候选区域进行分类。最后,本文使用评分准则对宽度学习系统的分类结果进行评分从而筛选出理想的纹理样图。结果 为了验证本文所提出的方法,本文收集大量理想纹理样图并将它们分成六个类进行实验验证,本文模型的准确度达到了94.66%。与最先进的方法Trimmed T-CNN相比,本文模型准确度提高了0.22%且速度得到了提升,具体来说,对于分辨率为512×512、1024×1024 和 2048×2048的图片,本文算法速度分别提快了1.3938s、1.8643s和2.3687s。结论 本文所提出的纹理样图自动提取算法,综合了深度学习和宽度学习的优点,使纹理样图的提取结果更加准确且高效。
关键词
Automatic texture exemplar extraction with jointed deep and broad learning models

Wu Huisi, Liang Chongxin, Yan Wei, Wen Zhenkun(Shenzhen University)

Abstract
Objective Texture exemplar refers to the input samples or templates for texture synthesis or texture generation, which contain the desired texture features and structures. Texture synthesis refers to the generation of new texture images by combining or duplicating one or more texture samples. In the texture synthesis task based on the texture exemplar, the diversity and texture structure of the texture exemplar play a decisive role, which determines the effect of the texture synthesis task. In the field of computer vision, texture sample diversity is crucial in texture synthesis tasks, which can bring richer, diverse and realistic appearance to synthesized textures. At the same time, it can also provide greater creative inspiration and design ideas for artists and designers. Currently, texture exemplars can be extracted from multiple sources, such as public texture datasets, Internet picture clips or photography. In other words, texture exemplars are mainly extracted by manual cutting and automatic algorithm extraction. However, not everyone is an artist after all, and it is difficult for ordinary people to extract a good texture sample or cut out a small texture exemplar from an existing image. In addition, manually cropping and extracting high-quality texture samples from a large number of images consumes a lot of energy and time for texture artists, and this method is easily driven by subjectivity and limited in diversity. With the development of deep learning algorithms, the current state-of-the-art automatic texture exemplar extraction algorithm is the Trimmed T-CNN model based on the convolutional neural network, which can effectively extract a variety of texture exemplars from the input image. However, since the model uses a selective search algorithm to generate candidate region, this process is time-consuming and computationally complex, so the model suffers from slow inference speed. Based on the above reasons, this paper is committed to using the rich image resources on the Internet to automatically, quickly and accurately cut out ideal and diverse texture exemplar from various images, so that users have more choices, so as to better meet the needs of texture synthesis task requirements. Method In this paper, based on the algorithm idea of object detection, we propose an automatic texture exemplar extraction algorithm combining deep learning and broad learning, which generates candidate texture exemplar regions through convolutional neural network and then uses broad learning for classification. To be specific, in order to obtain the ideal texture exemplar, this paper first uses the residual feature pyramid network to extract feature maps from the original image, aiming to effectively identify and generate texture exemplar candidates from the input image, and then uses the region candidate network to automatically and quickly obtain a large number of multi-scale texture exemplar candidate regions. Subsequently, we leverage a broad learning system to classify the candidate regions of texture exemplar extracted in the previous step. Finally, in order to obtain the ideal texture exemplar, we designed a scoring criterion based on the classification accuracy, distribution characteristics and size, aiming to use the scoring criterion to score the classification results of the broad learning system to screen out the ideal texture exemplar. Result To verify the effectiveness of the proposed method, we first collect a large number of ideal texture exemplar with distinguishable and representative features as a training dataset, and divides them into six classes based on size and regularity for experimental verification. A large number of qualitative and quantitative experiments have been carried out in this paper. The experimental results show that the accuracy of the model in this paper has reached 94.66%. Compared with the state-of-the-art method Trimmed T-CNN, the accuracy of the model in this paper has increased by 0.22% and the speed has been improved. Specifically, for images with resolutions of 512×512, 1024×1024 and 2048×2048, the speed of the algorithm in this paper is increased by 1.3938s, 1.8643s and 2.3687s respectively. Conclusion In this study, we propose an automatic texture exemplar extraction algorithm based on deep learning and broad learning, which effectively combines the advantages of convolutional neural networks and broad learning classification systems. Experimental results show that our model outperforms several state-of-the-art texture exemplar extraction methods, making texture exemplar extraction results more accurate and efficient.
Keywords

订阅号|日报