بازیابی داده های طیفی از صحنه های طبیعی با یک دوربین دیجیتال RGB و فیلترهای رنگی Recovering Spectral Data from Natural Scenes with an RGB Digital Camera and Colored Filters

INTRODUCTION The goal of multispectral imaging is to recover radiance or reflectance spectra at each pixel in a scene of interest.1–۴ Typically, a multispectral system consists of a digital camera coupled to a range of spectrally broad-band or narrow-band filters. If the number of filters is sufficiently large and their bandwidths are sufficiently small, as with a hyperspectral imaging system,2,5,6 spectral data can be recovered exactly.5–۹ But with just a few broad-band filters, spectral recovery presents an ill-posed problem. Many multispectral-imaging methods exploit the underlying smoothness of signal spectra,10 with illuminants11–۱۳ and spectral reflectances14,15 represented by low-dimensional models based on principal component analysis (PCA) or independent component analysis (ICA).16–۲۰ Thus, given a linear model,10,21 if the number of PCA or ICA coefficients of a particular set of spectra is the same as the number of camera responses (three in the simplest trichromatic case), then the spectra can be derived by an inverse transformation of the set of camera responses, with the forward transformation being estimated from a representative (‘‘training’’) data set. If the number of coefficients is more than the number of response values, then the latter may need to be increased by imaging the scene under different illuminants or by introducing colored fil- ters one at a time in front of the camera to modify the sensor spectra.18,22,23 Rather than an initial PCA or ICA being performed, however, the set of signal spectra may instead be estimated directly from the set of camera responses. These responses may be obtained from the camera itself or, for the present purposes, calculated from a set of known camera spectral sensitivities (Camera Spectral Sensitivities and Colored Filters section). For a conventional RGB digital camera, the set of camera responses comprises a matrix of three values for each pixel over all the pixels in the scene. If colored filters are introduced, then the set of camera responses comprises several of these matrices (Computations section). The ‘‘direct-mapping’’ method24 is described in more detail in the next section. CONCLUSIONS Natural scenes with complex variations in spatial structure and uncontrolled illumination present particular problems for recovering radiance and reflectance spectra. The present work has shown, however, that a combination of the direct-mapping method and a conventional RGB digital camera with a limited number of colored filters can provide acceptably accurate estimates, complementing related work on recovering illuminant spectra in natural scenes.3,34,37,38 Of the different combinations of colored filters, best performance was obtained with three filters. In terms of the three measures, the recovered signal had a goodnessof-fit coefficient better than 99.0%, a root-mean-square error less than 5.0%, and a CIELAB color difference less than 0.27. Using a conventional RGB digital camera in combination with a limited number of colored filters offers a less time-consuming and more economical approach to the multispectral capture of natural and artificial scenes than traditional methods, and the direct-mapping method provides acceptable performance without requiring more computationally expensive optimization procedures for error minimization. This approach allows a larger range of spectra to be sampled in the construction of the recovery matrix than with more constrained reflectance sets such as the Macbeth ColorChecker chart. Yet some data grouping is useful. Although good recovery is possible with undifferentiated training sets from urban and rural scenes, recovery was better for rural scenes with just rural scenes as the training set, and for urban scenes with just urban scenes as the training set. This work was computational and responses were calculated for a particular RGB digital camera and set of colored filters, but its practical application to other cameras and filters, after basic preprocessing operations such as noise removal and correction of inhomogeneities and response nonlinearities,39 should be straightforward.