Multi-sensor image fusion algorithm considering neighborhood consistency in the nonsubsampled Contourlet transform domain

For the fusion problem of the multi-sensor images of the same scene, a new algorithm is proposed based on neighbor energy and variance in the nonsubsampled Contourlet transform (NSCT) domain. Source images are firstly decomposed in the NSCT domain. For low frequency sub-band coefficients selection, the decision value of variance and energy based on neighbor variance and average neighbor energy is constructed for each pixel, and the rule based on the maximum of the decision value is adopted, so as to keep both image luminance and image details. For band-pass directional sub-band coefficients selection, the rule of maximum neighbor energy is used to keep more edge information. Finally the fused image is obtained through inverse transform. The algorithm has been used to merge multi-focus images and also infrared and visible light images. Experimental results indicate that the proposed method avoids the introduction of artifacts, with better edge details and luminance information, so that the fused image has a better subjective visual effect and objective evaluation criteria.