Objective

2

Image fusion is the process of using multiple image information of the same scene according to certain rules to obtain better fusion image. The fusion image contains the outstanding features of the image to be fused

which can improve the utilization of image information and provide more accurate help for later decision-making based on the image. Multi-scale analysis method

sparse representation method

and saliency method are three kinds of image representation methods that can be used in image fusion. Multi-scale analysis method is an active field in image fusion

but only the appropriate transformation method can improve the performance of the fusion image. The sparse representation method has good performance for image representation

but multi-value representation of the image easily leads to the loss of details. The significance method is unique due to its ability to capture the outstanding target in the image. However

visual saliency is a subjective image description index

and the proper construction of saliency map is an urgent problem to be solved. To address the problem of insufficient information and fuzzy edge details in image fusion

an image fusion method of convolution sparsity and detail saliency map analysis is proposed

which combines the advantages of multi-scale analysis

sparse representation method

and saliency method and at the same time avoids their disadvantages as much as possible.

Method

2

First

to address the insufficient image information after fusion

a multi-directional method is proposed to construct the adaptive training sample set. Then through dictionary training

a more abundant dictionary filter bank suitable for the image to be fused is obtained. Second

low-and high-frequency subgraphs are obtained by multi-scale analysis. The low-frequency subgraph contains a lot of basic information of source image

and it is represented by convolution sparsity using the trained adaptive dictionary filter bank. In this way

the sparse matrix of global single-value representation is obtained. The activity of each pixel in the image can be represented by the L

1

norm of this multidimensional sparse representation coefficient matrix. The more prominent its feature is

the more active the image is

so the weight can be measured by measuring the activity of the image to be fused. Through weight analysis

a weighted fusion rule is constructed to obtain the low-frequency subgraph with more abundant information. At the same time

to solve the problem of fuzzy edge details in the process of fusion

the fusion of high-frequency subgraphs is processed as follows. Because the high-frequency subgraph reflects the singularity of the image

a high-frequency detail saliency graph is constructed to highlight this feature. This detail saliency map is constructed by cross reconstruction of high-and low-frequency subgraphs. According to the distance difference between the high-frequency subgraph and the detail saliency map

similarity analysis is carried out

and a high-frequency fusion rule is established to obtain the high-frequency subgraph with more prominent edge details. Finally

the final fusion image is obtained by inverting the high-frequency and low-frequency subgraphs after fusion.

Result

2

In the experiment

three sets of gray image sets (including Mfi image set

Irvis image set

Medical image set) and four sets of color image sets (including Cmfi image set

Cirvis image set

Cmedical image set

Crsi image set) are randomly selected to verify the subjective visual and objective values of the proposed method NSST-Self-adaption-CSR-MAP (NSaCM). The results are compared with seven typical fusion methods

including convolutional sparse representation

convolutional sparsity-based morphological component analysis

parameter-adaptive pulse coupled-neural network

convolutional neural network

double-two direction sparse representation

wave-average-max

and non-subsampled contourlet transform and fusion. The experimental results show that the subjective visual effect of NSaCM is obviously better than that of other fusion methods. In the comparison of the average gradient

the objective numerical results of the seven methods mentioned above increased by 39.3%

32.1%

34.7%

28.3%

35.8%

28%

and 30.4% on average

respectively. In the comparison of information entropy

the objective numerical results of the seven methods mentioned above increased by 6.2%

4.5%

1.9%

0.4%

1.5%

2.4%

and 2.9% on average

respectively. In the comparison of spatial frequencies

the objective numerical results of the seven methods mentioned above increased by 31.8%

25.8%

29.7%

22.2%

28.6%

22.9%

and 25.3% on average

respectively. In the comparison of edge strength

the objective numerical results of the seven methods mentioned above increased by 39.5%

32.1%

35.1%

28.8%

36.6%

28.7%

and 31.3% on average

respectively.

Conclusion

2

NSaCM is suitable for gray and color images. The experimental results show that the fusion image obtained by the proposed method achieves better results in terms of both subjective and objective indicators. As seen from the numerical promotion of information entropy

the fusion image obtained by NSaCM contains more information

inherits more basic information of the source image

and solves the problem of insufficient information to a certain extent. From the numerical enhancement of average gradient

spatial frequency

and edge strength

NSaCM has a better expression of detail contrast. The overall activity of the fused image is higher

and the image content that makes image singularity more obvious is preserved in the fusion process

which solves the problem of edge detail blur in image fusion. However

the real-time performance of this method in terms of time consumption is still lacking

and further research is needed in the future.