Objective

2

Infrared (IR) images are based on the thermal radiation of the scene

and they are not susceptible to illumination and weather conditions. IR images are insensitive to the change of the brightness of the scene

and they usually have poor image quality and lack detailed information of the scene. By contrast

visible (VIS) images are sensitive to the optical information of the scene and contain a large amount of texture details. However

in low light and nighttime conditions

VIS images cannot capture the target clearly. IR and VIS images can provide complementary and redundancy information of a scene in the fusion image. Thus

image fusion is an important technique for image processing and computer vision applications such as feature extraction and target recognition. Multi-scale decomposition (MSD) has the advantage of extracting features at different scales

which is one of the most widely used image fusion methods. Many traditional multi-scale transform method signore the different image features of IR and VIS images. Therefore

traditional IR and VIS image fusion methods always lead to problems of missing the edge detail information and suppressing less halo. In this study

an IR and VIS image fusion algorithm based on anisotropic guide filter and phase congruency (PC) is proposed

which preserves edge details and suppresses artifacts effectively.

Method

2

The proposed scheme can not only preserve the details of source IR and VIS images

but also suppress the halo and artifacts effectively by combining the advantages of edge-preserving filter and PC. First

the input images are decomposed into a base layer and a series of detail layers. The base layer contains large scale variations in intensity

and the detail layers capture enough texture details by anisotropic guided filtering. Second

the saliency maps of the source images are calculated on the PC and Gaussian filter

and then

the binary weight maps are optimized by anisotropic guided filters of different scales

which can reduce noise and suppress halo. Finally

the fusion result is reconstructed by the base and detail layers by reconstruction rules. The main contributions of the proposed algorithm are as follows: 1) An edge-preserving filter based on MSD is employed to extract the image features at different scales. The anisotropic guided filtering weights are optimized based on the local neighborhood variances to achieve strong anisotropic filtering. Thus

this operation cannot only extract the image's texture details and preserve its edge features

but also prevent the halo phenomenon at the edges. 2) A novel weight optimization based on space consistency is proposed

which can reduce the noise and make the surface smooth. The anisotropic guided filtering is used to optimize the weighting maps of each layer

which is obtained by multi-scale edge-preserving decomposition. Compared with the original guide filter

the anisotropic guided filtering addresses the disadvantages of detail halos and the handling of inconsistent structures existing in previous variants of the guided filter. The experimental results show that the proposed scheme cannot only make the detail information more prominent

but also suppress the artifacts effectively. 3) A PC operator is used instead of the Laplace operator to generate the saliency maps from source images because the PC operator is insensitive to variations of contrast and brightness.

Result

2

We test our method on the TNO image fusion dataset

which contains different military-relevant scenarios that are registered with different multiband camera systems (including Athena

DHV

FEL

and TRICLOBS). Fifteen typical image pairs are chosen to assess the performance of the proposed method and four classical methods. Four representative fusion methods

namely

convolutional neural network (CNN)-based method

dual tree complex wavelet transform (DTCWT)-based method

weighted average fusion algorithm based on guided filtering (GFF)

and methods based on anisotropic diffusion and anisotropic diffusion (ADF)

are used in this study. CNN is a representative of deep learning (DL)-based methods. DTCWT is a representative of wavelet-based methods. GFF and ADF are representative of edge-preserving filter-based methods. The experimental results demonstrate that the proposed method could effectively extract the target feature information and preserve the background information from source images. The subjective evaluation results show that the proposed method is superior to the other four methods in detail

background

and object representation. The proposed method shows clear advantages not only in subjective evaluation

but also in several objective evaluation metrics. In objective evaluation

four indices are selected

including mutual information (MI)

degree of edge information (

Q

AB/F

)

entropy (EN)

and gradient-based mutual information (FMI_gradient).The objective evaluation results show that the proposed algorithm shows obvious advantages on the four metrics. The proposed algorithm has the largest MI and

Q

AB/F

values compared with the other four algorithms. This means that the proposed algorithm extracts much more edge and detail information than the other methods. In addition

our method has the best performance on EN and FMI_gradient values.

Conclusion

2

We proposed a new IR and VIS image fusion scheme by combining multi-scale edge-preserving decomposition with anisotropic guided filtering. The multi-scale edge-preserving decomposition can effectively extract the meaningful information from source images

and the anisotropic guided filtering can eliminate artifacts and detail "halos". In addition

to improve the performance of our method

we employ PC operator to obtain saliency maps. The proposed fusion algorithm can effectively suppress halo in fused results and can better retain the edge details and background information of the source image. The experimental results show that the proposed algorithm is more effective in preserving details that exist in VIS images and highlighting target information that exists in IR images compared with the other algorithms. The proposed method can be further improved by combining it with DL-based methods.