目的 红外与可见光图像融合是图像处理领域的一个热门问题,一般的红外与可见光图像融合算法都能达到认知场景的目的,但是无法对场景中的细节特征进行更加细致的刻画,进一步提高场景辨识度。针对此类问题,提出了一种基于Tetrolet变换的多尺度几何变换图像融合算法,以改善现有算法的不足。方法 首先,将红外与可见光图像映射到Tetrolet变换域,将其二者分解为低频系数和高频系数；然后,对低频系数,将区域能量理论与传统的加权法相结合,利用区域能量的多变性和区域像素的相关性,自适应地选择加权系数进行融合；对高频系数,利用改进的多方向拉普拉斯算子方法计算拉普拉斯能量和,再引入区域平滑度为阈值设定高频系数融合规则；最后,将融合所得新的低频和高频系数进行图像重建得到融合结果。结果 在三组红外与可见光图像上与3种方法的融合结果进行比较,主观评判上,本文算法融合结果在背景、目标物以及细节体现方面均优于其他三种方法；客观指标上,本文算法相较于其他3种方法,运行时间较之NSCT方法提升了0.37秒,AvG值和SF值均有大幅提高,提高幅度最大为5.42和2.75,在PSNR值、IE值和SSIM值均有小幅度提高,提升幅度分别为0.25、0.12和0.19。结论 本文所提出的红外与可见光图像融合算法改善了融合图像对细节的刻画,使观察者对场景的理解能力有所提升。
Image fusion based on multi-directional SML and Tetrolet transform
SHEN Yu,CHEN Xiaopeng,YANG Qian(Lanzhou Jiaotong University School of Electronics and Information Engineering)
Objective Image fusion is an important form of information fusion, which is widely used in image understanding and computer vision. It combines multiple images that are described the same scene in different forms to obtain more accurate and comprehensive information processing process. The fused image can provide more effective information for subsequent image processing to some extent. Among them, infrared and visible image fusion is a hot issue in the field of image fusion. By combining the background information in the visible light image with the target features in the infrared image, the information of the two images can be fully fused, which can describe more comprehensively and accurately , improve the target features and background recognition in the scene and enhance people"s perception and understanding of the image. General infrared and visible image fusion algorithms can achieve the purpose of cognitive scenes, but they cannot reflect the detailed features of the scene in a more detailed way to further improve the scene identification, so as to provide more effective information for subsequent image processing. Aiming at such problems, this paper proposes a Tetrolet-based multi-scale geometric transformation fusion algorithm to improve the shortcomings of existing algorithms. The Tetrolet transform divides the source image into several image blocks and transforms each image block to obtain low-frequency coefficients and high-frequency coefficients. The low frequency and high frequency coefficients of all image blocks are arranged and integrated into an image matrix to obtain the low frequency and high frequency coefficients of the source image. Method Firstly, the infrared and visible light images are mapped to the Tetrolet transform domain, and the two images are respectively subjected to Tetrolet transformation. According to the four-lattice patchwork filling theory, the best filling method is selected based on the criterion of the maxmum first-order norm among 117 filling methods. In this way, the respective low-frequency coefficients and high-frequency coefficients of the infrared and visible images are calculated. Then, the low-frequency coefficients of the two are combined with the theory of regional energy and the traditional weghting method. By taking advantage of the variability of regional energy and the correlation of regional pixels, according to the constant change of the central pixel, the weighting coefficients are adaptively selected for fusion to obtain the fused low-frequency coefficients; For the high-frequency coefficients of the two images, the traditional Laplace energy only according to the up, down, left and right four The Laplace operator of the direction is calculated. Considering that the pixel points in the diagonal direction also contribute to the calculation of the Laplace energy sum, this paper uses the improved eight-direction Laplace operator calculation method to calculate the Laplace energy and to introduce regional smoothness as the threshold value; If the Laplace energy sum is above the threshold value, the weighted coefficient is calculated according to smoothness and threshold value to carry out weighted fusion; otherwise, the fusion rule is set according to the maximum and minimum values of Laplace energy sum of two high-frequency components to obtain the high-frequency coefficient after fusion; finally, the low-frequency and high-frequency coefficients obtained after the fusion are reconstructed to get the fused image. Result The fusion results of three sets of infrared and visible images are compared with the CL method, DWT method and NSCT method. From the perspective of visual effect, the fusion image of the algorithm in this paper is superior to the other three methods in image background, scene object and detail embodiment. In terms of objective indicators, compared with the other three methods, the running time required by the algorithm in this paper is 0.37 seconds shorter than that of the NSCT method, and the AvG value and SF value of the fused images are greatly improved, with the maximum increases of 5.42 and 2.75, and the PSNR value, IE value and SSIM value are slightly increased, with the improvement ranges of 0.25, 0.12 and 0.19 respectively. The experimental results show that the proposed algorithm in this paper improves the fusion image of effect and quality to a certain extent. Conclusion This paper proposes an infrared and visible image fusion method based on regional energy and improved multi-directional Laplace energy. The infrared image and visible light image are mapped into the transform domain by Tetrolet transformation, which is decomposed into low frequency coefficient and high frequency. The fusion of the low-frequency coefficients is carried out based on the regional energy theory and the adaptive weighted fusion criterion; According to the improved Laplace energy and the regional smoothness ,the high-frequency coefficients of the infrared and visible images are selected to achieve the fusion of the high-frequency coefficients; The fusion results of low frequency and high frequency coefficients are obtained by inverse transformation. Compared with the fusion results of the other three transform domain algorithms, the fused images not only enhance the background information, but also remarkably enhance the embodiment of the details in the scene, and have certain advantages in objective evaluation indexes such as average gradient and peak signal-to-noise ratio. The observer"s ability to understand the scene has been improving.