Effects of Aliasing and Mis-Registration on Pan-Sharpening Methods Based on Either Component Substitution or Multi-Resolution Analysis

Pan-sharpening is a branch of data fusion, more specifically of image fusion, that is receiving an ever increasing attention from the remote sensing community. New-generation space-borne imaging sensors operating in a variety of ground scales and spectral bands provide huge volumes of data having complementary spatial and spectral resolutions. Constraints on the signal to noise ratio (SNR) impose that the spatial resolution must be lower, if the desired spectral resolution is larger. Conversely, the highest spatial resolution is obtained whenever no spectral diversity is required. The trade-off of spectral and spatial resolution makes it desirable to perform a spatial enhancement of the lower resolution multi-spectral (MS) data or, equivalently, to increase the spectral resolution of the data-set having a higher ground resolution, but a lower spectral resolution; as a limit case, constituted by a unique panchromatic image (Pan) bearing no spectral information.

To pursue this goal, an extensive number of methods have been proposed in the literature over the last two decades. Most of them follow a general protocol, that can be summarized in the following two key points:
1) extract high-resolution geometrical information of the scene, not present in the MS image, from the Pan image;
2) incorporate such spatial details into the low-resolution MS bands, interpolated to the spatial scale of the Pan image, by properly modelling the relationships between the MS bands and the Pan image.

In general, the image fusion methods described by this protocol can be divided into two main classes depending on how the spatial details are extracted from the Pan image: component substitution (CS) techniques that are based on a spectral transformation of the MS data followed by replacement of the first transformed component with the Pan image and reverse transformation to yield back the sharpened MS bands; techniques that employ multi-resolution analysis (MRA) to extract the geometrical information that will be added to the MS bands, from the Pan image.

CS-based pan-sharpening is a typology of simple and fast techniques based on a spectral transformation of the original bands in a new vector space. Most widely used transformations are intensity-hue-saturation (IHS), principal components analysis (PCA) and Gram-Schmidt orthogonalisation procedure. IHS fusion technique, originally defined for three bands only, has been extended to an arbitrary number of spectral bands. The rationale of CS fusion is that one of the transformed components (usually the first component or intensity I(L)) is substituted by the high-resolution Pan image, P, before the inverse transformation is applied. To ensure global preservation of radiometry, P is histogram-matched to I(L), in such a way that the histogram-matched sharpening Pan, once degraded to the spatial resolution of I(L), exhibits same global mean and variance as I(L). However, since the histogram-matched Pan image and I(L) may not have the same local radiometry, spectral distortion, appearing as local colour changes in a composition of three bands at a time, may occur in pan-sharpened products. To mitigate local spectral distortion, I(L) may be taken as a linear combination of the MS bands with weighting coefficients adjusted to the extents of overlap between the spectral response of each MS channel and that of the Pan image. In principle, if the low-pass approximation of the Pan image synthesized by combining the spectral channels exactly matches the low-resolution version of P, spectral distortion does not occur.

MRA-based techniques substantially split the spatial information of the MS bands and of the Pan image into a series of band-pass spatial frequency channels. The high frequency channels are inserted into the corresponding channels of the interpolated MS bands. The sharpened MS bands are synthesized from their new sets of spatial frequency channels. The “à trous” wavelet transform and the Laplacian pyramid are most widely used to perform the MRA. In such cases, the zero-mean high-frequency spatial details are simply given as the difference between, P, and its low-pass filter version PL. Recent studies have demonstrated that if the low-pass filter is designed in such a way that it matches the modulation transfer function (MTF) of the spectral channel in which details will be injected, the spatial enhancement provided by MRA techniques becomes comparable to that of CS techniques.

Regardless of how spatial details have been obtained, their injection into the interpolated MS bands may be weighed by suitable gains, different for each band, possibly space-varying, i.e. a different gain at each pixel. Algorithms based on context-adaptive, i.e. local, models generally perform better than models fitting each band globally.

CS-based and MRA-based fusion techniques exhibit complementary spectral-spatial quality trade-off. The former provide fused images with high geometrical quality of spatial details, but with possible spectral impairments. The latter are spectrally accurate in general, but may be unsatisfactory in terms of spatial enhancement. However, once CS is optimized for spectral quality of pan-sharpened products and MRA is optimized for spatial quality, the two categories of methods yield very similar results in terms of overall quality.

In this paper, the characteristics of CS-based and MRA-based fusion method will be investigated, both theoretically and experimentally on QuickBird data sets. The conclusion is that MRA-based fusion is far more sensitive than CS-based fusion to:
a) registration errors, i.e. spatial misalignments between MS and Pan images, possibly originated by cartographic re-sampling of individual data sets;
b) aliasing occurring in MS bands and stemming from an MTF excessively broad relatively to the sampling interval.

Conversely, fusion methods relying on MRA are particularly sensitive to temporal misalignments, i.e. MS and Pan acquired not at the same time, that may introduce severe spectral distortions.