Objective

2

Texture filtering is a low-level task in image processing and computer vision

which aims to filter the image through the essential image structure preservation and other texture smoothing details. Current texture filtering algorithms are mainly divided into two categories like local-based and global-based orientation. Traditional methods are challenged to distinguish image structure and strong gradient textures in common. Due to the lack of reliable training set

recent deep learning algorithms often use the results of existing traditional methods as ground truth

so they are unable to refill the gaps of the existing traditional algorithms. For example

texture and structure aware filtering network (TSAFN) performs data synthesis by filling the whole image with the same texture

but textures should be object-dependent. Such a synthesis way will lead to a large gap between synthetic images and real-world images. In order to solve these problems

our novel dataset is generated for texture filtering training

and the image smoothing algorithm is proposed based on directional filtering scales-predicting model.

Method

2

First

a texture filtering dataset for deep learning is generated by filling texture images per object structure based on the existing structure images. At the same time

we processed the image structure via smoothing and compression. Hence

the dataset we generated can not only enhance the ability of the algorithm to distinguish strong gradient texture and structure

but also reduce the domain gap between synthetic images and real images. Then

the image smoothing algorithm based on directional filtering scales-predicting model is designed

which includes a scale-aware sub-network and an image smoothing sub-network. The scale-aware sub-network is used to predict directional texture filtering scales map. It not only reflects whether a pixel and its surrounding pixels are in the same texture

but also implies information about whether the pixel is a structural pixel or not. The image smoothing sub-network takes the stack of scales map predicted by the scale-aware sub-network and original image as input

and gets the filtered image through a small amount of convolution layer. It can complete the smoothing and correct the imperfection of the result of scale-aware sub-network quickly. In edge-aware sub-network

we applied the classic U-Net because of its excellent ability to easy use low-level features straightforward

and we change its input and output dimensions. The input of the scale-aware sub-network is the stack of RGB image and gradient map

the output of the scale-aware sub-network is a six-dimensional scales map. The image smoothing sub-network consists of seven convolutional layers

the first six layers are followed by ReLU and batch normalization

while the last layer is followed by sigmoid for preventing the pixel value out of bounds

the input of the image smoothing sub-network is the stack of an image and six-dimensional scales map

the output of the image smoothing sub-network is the filtered image. The number of images related to our training set

test set and verification set are 10 000

1 500

1 000

respectively

they were selected from our dataset randomly

and they did not overlap. Our network is implemented in Pytorch toolbox. The input images and ground truth images are clipped to 224×224 pixels for training

the momentum parameter is set to 0.9

the learning rate is set to 1E-2

and the weight decay is 0.000 2. We use an adaptive method

that is

if the loss does not decrease by more than 0.003 for 5 epochs

then the learning rate will be halved. The stochastic gradient descent(SGD) learning procedure is accelerated using a NVIDIA RTX 2080 GPU device.

Result

2

We compared our algorithm to the five traditional algorithms and two deep learning algorithms on our dataset and other real-world image datasets. The quantitative evaluation metrics used in our dataset contain the peak signal to noise ratio (PSNR)

the structural similarity (SSIM)

the mean square error (MSE) and the running time. In comparison with the results of different filtering algorithms from our dataset

our PSNR is 2.79 higher (higher is better) than the second-best

our SSIM is 0.013 3 higher (higher is better) than the second-best

our MSE is 6.863 8 lower (less is better) than the second-best

the running time of our method is 0.002 s faster than the second-best. All deep learning algorithms have been re-trained from our dataset

it is sorted out that our algorithm keeps the leading effect in the discrimination of structure and strong gradient texture based on the comparative results of the texture filtering results of real-world images. The results are trained by different datasets are compared in terms of same model

and it is proved that our dataset can make the model have better generalization ability and stronger ability of distinguishing the strong gradient texture and structure.

Conclusion

2

Our dataset contains a variety of textures and structures

which can develop the model to distinguish strong gradient texture and object structure better. Our data synthesis method can make the model have better generalization potential ability. Additionally

the designed image smoothing algorithm surpasses the existing methods in performance and speed based on directional filtering scales-predicting model.