Project topics and materials
Get Complete Project Material File(s) Now! »
Demosaicing quality assessment on CAVE image set
We objectively compare the demosaicing performances reached by PPI-based methods (see Section 2.6) with those provided by the existing methods described in Section 2.4. The experimental procedure is presented in Section 3.2.1 and the results in terms of PSNR, color difference, and computation time, are shown in Section 3.2.2. Images obtained by either selecting one spectral channel or color simulation by projection in sRGB color space are visually compared in Section 3.2.3.
Experimental procedure
Let us consider the multispectral image I = {Ik}Kk =1 made of K fully-defined channels associated to the K bands of a single-sensor MSFA camera. Although this image cannot be provided by such cameras, it is called the reference image because it is often used as a reference to assess the demosaicing quality. To get reference images, we simulate the 32 multispectral images of size 512 × 512 pixels from the CAVE database [124] with the CIE D65 illuminant and IMEC16 SSFs as described in Section 1.6.1. To obtain raw images, we spectrally sample reference images according to the IMEC16 MSFA whose pattern is shown in Fig. 2.4b. The fully-defined image is then estimated from the raw image using any demosaicing method and compared
Acquisition properties and demosaicing performances
We here study the impact of the illumination and camera properties on demosaicing performances. In Section 3.3.1 we show that demosaicing performances are fairly affected by illumination changes. According to the image formation model (see Section 1.3.1), illumination as well as camera SSFs have an influence on the values of pixels. By studying the demosaicing performances with respect to each channel, we show in Section 3.3.2 that the channels that receive little energy, i.e., that have low values on average with respect to other channels, are difficult to demosaic. Finally, we show in Section 3.3.3 that such variation of values between channels has a great influence on spectral correlation.
PSNR assessment with respect to illumination
In order to study the influence of illumination changes on the demosaicing performances, we compute the PSNR provided by each method on the 32 IMEC16 images simulated with the various illuminations of Fig. 1.2. The results displayed in Table 3.4 show that the performances of all methods are affected by illumination changes. Images simulated under E and D65 are fairly well demosaiced since these illuminants uniformly illuminate the whole spectrum. Using A and HA illuminations reduces the performances because the spectral power distribution (SPD) of these illuminations increases with respect to the wavelength. At last, using F12 and LD illuminations whose SPDs present 3 peaks in the visible domain severely reduces demosaicing performances. Table 3.4 also shows that WB, BTES, and PPBTES methods are fairly robust to illumination variations because they are mainly based on spatial correlation. Other methods use spectral correlation assumption that is weakened by illumination changes.
Effect of illumination and SSFs on spectral correlation
To highlight the effect of spectrally non-uniform illumination or SSFs areas on spectral correlation, we compute the correlation coefficient between the high-frequency information of each channel pair [29, 59]. For this purpose, we apply a circular high-pass filter with a cut-off spatial frequency of 0.25 cycle/pixel on the 2D Fourier transform of each channel. For each illumination and camera, we compute the average Pearson correlation coefficient μC (see Eq. (1.8)) over all possible high frequency channel pairs and the standard deviation σC of the correlation coefficient. Table 3.5 shows the correlation and its dispersion on average over all 32 IMEC16 and IC images simulated with each illumination. These results show that the illuminationswhose SPD is uniform(E) over W or can be considered as such (D65) provide channels with the highest and less scattered spectral correlations. The illuminations A and HA, for which E(λ) increases with respect to λ over W, provide channels with lower and more scattered spectral correlations. The illuminations F12 and LD, for which E(λ) ≈ 0 except for three marked peaks, provide channels with the lowest and most scattered spectral correlations. By comparing Table 3.5a with Table 3.5b we see that SSF areas over W strongly affect spectral correlation, and that channels are more correlated when SSFs have similar areas. To conclude, the illumination SPDand camera SSF areas strongly affect values of pixels in different channels. Such variation of values froma channel to anotherweakens spectral correlation, which affects the performance of demosaicing procedures that rely on this property. In the next section we propose three ways to overcome this issue.
Robust demosaicing for various acquisition properties
We propose pre- and post-normalization steps for demosaicing in Section 3.4.1 that adjust the values of channels before demosaicing and restore themafterwards. These stepsmake demosaicing robust to acquisition properties by using the normalization factors presented in Section 3.4.2. Such normalization factors depend on acquisition properties or on raw image statistics. In Section 3.4.3, we finally assess the demosaicing methods presented in Sections 2.4 and 2.6 when the proposed normalization steps are performed.
Table of contents :
Introduction
1 Multispectral images
1.1 Introduction
1.2 From illumination to multispectral image
1.2.1 Illuminations
1.2.2 Reflected radiance
1.2.3 Multispectral image
1.3 Multispectral image acquisition
1.3.1 Multispectral image formation model
1.3.2 Multispectral image acquisition systems
1.3.3 Databases of acquired radiance
1.4 Databases of estimated reflectance
1.4.1 Reflectance estimation and existing databases
1.4.2 Our proposed database: HyTexiLa
1.4.3 Database acquisition and reflectance estimation
1.5 Multispectral image simulation
1.5.1 Image simulation model
1.5.2 IMEC16 multispectral filter array (MSFA) camera
1.5.3 Simulation validation with IMEC16 camera
1.6 Properties of multispectral images
1.6.1 Two simulated radiance image sets
1.6.2 Spatial properties
1.6.3 Spectral properties
1.7 Conclusion
2 MSFA raw image demosaicing
2.1 Introduction
2.2 Multispectral filter array technology
2.2.1 MSFA-based acquisition pipeline
2.2.2 MSFA design
2.2.3 MSFA basic patterns
2.3 MSFA demosaicing
2.3.1 MSFA demosaicing problem
2.3.2 VIS5 MSFA demosaicing
2.3.3 Data-driven demosaicing
2.4 Demosaicing methods for IMEC16 MSFA
2.4.1 Generic demosaicing methods
2.4.2 Spectral difference-based methods
2.4.3 Binary tree-based methods
2.5 From raw to pseudo-panchromatic image (PPI)
2.5.1 Limitations of existing methods
2.5.2 PPI definition and properties
2.5.3 PPI estimation
2.6 PPI-based demosaicing
2.6.1 Using PPI in DWT (PPDWT)
2.6.2 Using PPI in BTES (PPBTES)
2.6.3 Proposed PPI difference (PPID)
2.7 Conclusion
3 Demosaicing assessment and robustness to acquisition properties
3.1 Introduction
3.2 Demosaicing quality assessment on CAVE image set
3.2.1 Experimental procedure
3.2.2 Objective assessment
3.2.3 Subjective assessment
3.3 Acquisition properties and demosaicing performances
3.3.1 PSNR assessment with respect to illumination
3.3.2 PSNR with respect to spectral sensitivity function (SSF)
3.3.3 Effect of illumination and SSFs on spectral correlation
3.4 Robust demosaicing for various acquisition properties
3.4.1 Raw value scale adjustment
3.4.2 Normalization factors
3.4.3 Normalization assessment
3.5 Demosaicing HyTexiLa images with various cameras
3.5.1 Considered cameras and demosaicing methods
3.5.2 Extension ofWB and PPID methods to the four MSFAs
3.5.3 PSNR comparison
3.6 Conclusion
4 MSFA raw image classification
4.1 Introduction
4.2 Classification scheme
4.2.1 Classification of MSFA raw texture images
4.2.2 Local binary patterns (LBPs)
4.2.3 Decision algorithm and similarity measure
4.3 LBP-based Spectral texture features
4.3.1 Moment LBPs
4.3.2 Map-based LBPs
4.3.3 Luminance–spectral LBPs
4.3.4 Opponent band LBPs
4.4 LBP-based MSFA texture feature
4.4.1 MSFA neighborhoods
4.4.2 MSFA-based LBPs
4.4.3 Relation between MSFA-based and opponent band LBPs
4.4.4 Neighborhoods in MSFA-based LBPs
4.5 Experimental results
4.5.1 Feature extraction
4.5.2 Accuracy vs. computation cost
4.5.3 Classification results and discussion
4.6 Conclusion
Conclusion
A Conversions from multispectral to XYZ, sRGB and L*a*b* spaces
A.1 From XYZ to sRGB color space
A.2 From XYZ to L*a*b* color space
B Spectral sensitivity functions
C Weight computation for demosaicing 101
C.1 Weight computation in BTES
C.2 Weight computation in MLDI
C.3 Weight computation in PPBTES
Bibliography
