Project topics and materials
Get Complete Project Material File(s) Now! »
Signal Representations and Metrics for Recognition
This section reviews some of the existing tools for signal representation and discrimina-tion in recognition tasks. Invariance and stability are formulated in terms of Lipschitz continuity conditions, and are then studied on a variety of representations.
Local translation invariance, Deformation and Additive Stability
In recognition tasks, the action of small geometric deformations and small additive per-turbations produce small changes in the appearance of objects and textures. This mo-tivates the study of signal representations defining an Euclidean metric stable to those perturbations. These stability properties can be expressed mathematically as Lipschitz continuity properties. Stability to additive noise is guaranteed by imposing Φ to be non-expansive: ∀ x, x˜ ∈ L2(Rd) , Φ(x) − Φ(˜x) ≤ x − x˜ . (2.1).
Indeed, it results that the metric defined by Φ is Lipschitz continuous with respect to the Euclidean norm of an additive perturbation: d(x + h, x) = Φ(x + h) − Φ(x) ≤ h .
On the other hand, stability to deformations is achieved by controlling the behavior of Φ under the action of diffeomorphisms u u − τ (u), where τ : Rd → Rd is an invertible displacement field. The amount of deformation can be measured with a metric on the space of diffeomorphisms. If |τ (u)| denotes the Euclidean norm in Rd, |∇τ (u)| denotes the operator norm of ∇τ (u) and |Hτ (u)| is the sup norm of the Hessian tensor, then the norm of the space of C2 diffeomorphisms measures the amount of deformation over any compact subset Ω ⊂ Rd as τ = sup |τ (u)| + sup |∇τ (u)| + sup |Hτ (u)| . u∈Ω u∈Ω u∈Ω This deformation metric penalizes displacement fields by its maximum amplitude supu∈Ω |τ (u)| and maximum elasticity supu∈Ω |∇τ (u)| . In most contexts, however, rigid displacement fields, corresponding to translations, do not affect recognition to the same extent as non-rigid, elastic deformations. This motivates the notion of locally translation invariant representations. We say that Φ is Lipschitz continuous to the action of C2 diffeomorphisms and locally translation invariant at scale 2J if for any compact Ω ⊂ Rd there exists C such that, for all x ∈ L2(Rd) supported in Ω and all τ ∈ C2, d(L[τ ]x, x) = Φ(L[τ ]x) − Φ(x) ≤ C x −J sup τ (u) + sup |∇ τ (u) + sup Hτ (u) . 2 u∈Ω || u∈Ω | u∈Ω | |
The reference scale 2J controls the amount of translation invariance required on the representation, by diminishing the influence of the amplitude of τ in the deformation metric. If τ is a displacement field with maximum amplitude supu∈Ω |τ (u)| ≪ 2J , then (2.2) shows that the representation stability is controlled by the amount of elastic defor- mation applied to x. On the other hand, the scale of local invariance also controls the amount of delocalization of the representation. Pattern recognition tasks often require signal representations which keep spatial information up to a certain resolution, whereas we will ask stationary texture representations to be fully translation invariant, by letting the local invariance scale J go to infinity.
Fourier Modulus, Autocorrelation and Registration Invariants
Translation invariant representations can be obtained from registration, auto-correlation or Fourier modulus operators. However, the resulting representations are not Lipschitz continuous to deformations. A representation Φ(x) is translation invariant if it maps global translations xc(u) = x(u − c) by c ∈ Rd of any function x ∈ L2(Rd) to the same image: ∀ x ∈ L2(Rd) , ∀ c ∈ Rd , Φ(xc) = Φ(x) . (2.4)The Fourier transform modulus is an example of a translation invariant representa-tion. Let xˆ(ω) be the Fourier transform of x(u) ∈ L2(Rd). Since xc(ω) = e−ic.ω xˆ(ω), it results that |xc| = |xˆ| does not depend upon c. A Fourier modulus is translation invariant and stable to additive noise, but unstable to small deformations at high frequencies [Mal12], as illustrated with the following dila-tion example. Let τ (x) = sx denote a linear displacement field where |s| is small, and let x(u) = eiξuθ(u) be a modulated version of a lowpass window θ(u). Then the dilation xτ (u) = L[τ ]x(u) = x((1 + s)u) moves the central frequency of xˆ from ξ to (1 + s)ξ. If 2 = 2 ˆ 2 σθ |ω| |θ(ω)| dω measures the frequency spread of θ, then σx2 = |ω − ξ|2|xˆ(ω)|2dω = σθ2 ,
Table of contents :
1 Introduction
1.2 The Scattering Representation
1.3 Image and Pattern Classification
1.4 Texture Discrimination and Reconstruction from Scattering
1.5 Multifractal Scattering
2 Invariant Scattering Representations
2.1 Introduction
2.2 Signal Representations and Metrics for Recognition
2.2.1 Local translation invariance, Deformation and Additive Stability .
2.2.2 Kernel Methods
2.2.3 Deformable Templates
2.2.4 Fourier Modulus, Autocorrelation and Registration Invariants .
2.2.5 SIFT and HoG
2.2.6 Convolutional Networks
2.3 Scattering Review
2.3.1 Windowed Scattering transform
2.3.2 Scattering metric and Energy Conservation
2.3.3 Local Translation Invariance and Lipschitz Continuity to Deformations
2.3.4 Integral Scattering transform
2.3.5 Expected Scattering for Processes with stationary increments
2.4 Characterization of Non-linearities
2.5 On the L1 continuity of Integral Scattering
2.6 Scattering Networks for Image Processing
2.6.1 Scattering Wavelets
2.6.2 Scattering Convolution Network
2.6.3 Analysis of Scattering Properties
2.6.4 Fast Scattering Computations
2.6.5 Analysis of stationary textures with scattering
3 Image and Pattern Classification with Scattering
3.1 Introduction
3.2 Support Vector Machines
3.3 Compression with Cosine Scattering
3.4 Generative Classification with Affine models
3.4.1 Linear Generative Classifier
3.4.2 Renormalization
3.4.3 Comparison with Discriminative Classification
3.5 Handwritten Digit Classification
3.6 Towards an Object Recognition Architecture
4 Texture Discrimination and Synthesis with Scattering
4.1 Introduction
4.2 Texture Representations for Recognition
4.2.1 Spectral Representation of Stationary Processes
4.2.2 High Order Spectral Analysis
4.2.3 Markov Random Fields
4.2.4 Wavelet based texture analysis
4.2.5 Maximum Entropy Distributions
4.2.6 Exemplar based texture synthesis
4.2.7 Modulation models for Audio
4.3 Image texture discrimination with Scattering representations
4.4 Auditory texture discrimination
4.5 Texture synthesis with Scattering
4.5.1 Scattering Reconstruction Algorithm
4.5.2 Auditory texture reconstruction
4.6 Scattering of Gaussian Processes
4.7 Stochastic Modulation Models
4.7.1 Stochastic Modulations in Scattering
5 Multifractal Scattering
5.1 Introduction
5.2 Review of Fractal Theory
5.2.1 Fractals and Singularitites
5.2.2 Fractal Processes
5.2.3 Multifractal Formalism and Wavelets
5.2.4 Multifractal Processes and Wavelets
5.2.5 Cantor sets and Dirac Measure
5.2.6 Fractional Brownian Motions
5.2.7 α-stable L´evy Processes
5.2.8 Multifractal Random Cascades
5.2.9 Estimation of Fractal Scaling Exponents
5.3 Scattering Transfer
5.3.1 Scattering transfer for Processes with stationary increments
5.3.2 Scattering transfer for non-stationary processes
5.3.3 Estimation of Scattering transfer
5.3.4 Asymptotic Markov Scattering
5.4 Scattering Analysis of Monofractal Processes
5.4.1 Gaussian White Noise
5.4.2 Fractional Brownian Motion and FGN
5.4.3 L´evy Processes
5.5 Scattering of Multifractal Processes
5.5.1 Multifractal Scattering transfer
5.5.2 Energy Markov Property
5.5.3 Intermittency characterization from Scattering transfer
5.5.4 Analysis of Scattering transfer for Multifractals
5.5.5 Intermittency Estimation for Multifractals
5.6 Scattering of Turbulence Energy Dissipation
5.7 Scattering of Deterministic Multifractal Measures
5.7.1 Scattering transfer for Deterministic Fractals
5.7.2 Dirac measure
5.7.3 Cantor Measures
A Wavelet Modulation Operators
A.1 Wavelet and Modulation commutation
A.2 Wavelet Near Diagonalisation Property
A.3 Local Scattering Analysis of Wavelet Modulation Operators
B Proof of Theorem 5.4.1
B.1 Proof of Lemma 5.4.2
B.2 Proof of Lemma 5.4.3
B.3 Proof of Lemma 5.4.4
References
