Objective

2

Image fusion technology is of great significance for image recognition and comprehension. Infrared and visible image fusion has been widely applied in computer vision

target detection

video surveillance

military and many other areas. The weakened target

unclear background details

blurred edges and low fusion efficiency have been existing due to high algorithm complexity in fusion. The dual-scale methods can reduce the complexity of the algorithm and obtain satisfying results in the first level of decomposition itself compared to most multi-scale methods that require more than two decomposition levels

with utilizing the large difference of information on the two scales. However

insufficient extraction of salient features and neglect of the influence of noise which may lead to unexpected fusion effect. Dual-scale decomposition has been combined to the saliency analysis and spatial consistency for acquiring high-quality fusion of infrared and visible images.

Method

2

The visual saliency has been used to integrate the important and valuable information of the source images into the fused image. The spatial consistency has been fully considered to prevent the influence of noise on the fusion results. First

the mean filter has been used to filter the source image

to separate the high-frequency and low-frequency information in the image: the base image containing low-frequency information has been obtained first. The detail image containing high-frequency information has been acquired second via subtracting from the source image. Next

a simple weighted average fusion rule

that is

the arithmetic average rule

has been used to fuse the base image via the different sensitivity of the human visual system to the information of base image and detail image. The common features of the source images can be preserved and the redundant information of the fused base image can be reduced; For the detail image

the fusion weight based on visual saliency has been selected to guide the weighting. The saliency information of the image can be extracted using the difference between the mean and the median filter output. The saliency map of the source images can be obtained via Gaussian filter on the output difference. Therefore

the initial weight map has been constructed via the visual saliency. Furthermore

combined with the principle of spatial consistency

the initial weight map has been optimized based on guided filtering for the purpose of reducing noise and keeping the boundary aligned. The detail image can be fused under the guidance of the final weight map obtained. Therefore

the target

background details and edge information can be enhanced and the noise can be released. At last

the dual-scale reconstruction has been performed to obtain the final fused image of the fused base image and detail image.

Result

2

Based on the different characteristics of traditional and deep learning methods

two groups of different gray images from TNO and other public datasets have been opted for comparison experiments. The subjective and objective evaluations have been conducted with other methods to verify the effectiveness and superiority performance of the proposed method on the experimental platform MATLAB R2018a.The key prominent areas have been marked with white boxes in the results to fit the subjective analysis for illustrating the differences of the fused images in detail. The subjective analyzing method can comprehensively and accurately extract the information to obtain clear visual effect based on the source images and the fused image. First

the first group of experimental images and the effectiveness of the proposed method in improving the fusion effect can be verified on the aspect of objective evaluation. Next

the qualified average precision of average gradient

edge intensity

spatial frequency

feature mutual information and cross-entropy have been presented quantitatively

which are 3.990 7

41.793 7

10.536 6

0.446 0 and 1.489 7

respectively. At last

the proposed method has shown obvious advantages in the second group of experimental images compared with a deep learning method. The highest entropy has been obtained both. An average increase of 91.28%

91.45%

85.10%

0.18% and 45.45% in the above five metrics have been acquired respectively.

Conclusion

2

Due to the complexity of salient feature extraction and the uncertainty of noise in the fusion process

the extensive experiments have demonstrated that some existing fusion methods are inevitably limited

and the fusion effect cannot meet high-quality requirements of image processing. By contrast

the proposed method combining the dual-scale decomposition and the fusion weight based on visual saliency has achieved good results. The enhancement effect of the target

background details and edge information are particularly significant including anti-noise performance. High-quality fusion of multiple groups of images can be achieved quickly and effectively for providing the possibility of real-time fusion of infrared and visible images. The actual effect of this method has been more qualified in comparison with a fusion method based on deep learning framework. The further research method has been more universal and can be used to fuse multi-source and other multi-source and multi-mode images.