Unit-Economy via the Two Parent Wavelets

It is quite evident that, by itself, the introduction of the translation-generated basis elements (the $P$ 's and the $Q$ 's) constitutes a proliferation of concepts: their sheer number prevents them from being automatically accessible for further study; one's mind run's the danger of being subjected to information overload. Such a state of affairs motivates an inquiry as to the applicability of the principle of unit-economy²². Can one, by introducing simplifying concepts, reduce this number by consolidating these $P$ 's and the $Q$ 's into one or two concepts?

An affirmative answer to this question is based on the introduction of two ``mother wavelet'' for all the $P$ 's and a ``father wavelet'' for all the $Q$ 's.

Recall that the $Q$ 's have already been consolidated by Eqs.(2.79), (2.80), and (2.88) into the single scaling function, the ``father wavelet''

$$\phi(t)=\frac{\sin \pi t}{\pi t}~.$$ (294)

Thus by applying to this wavelet a translation, a compression, and an amplification, one obtains

$$\sqrt{2^{-(k+1)}}\,\phi(2^{-(k+1)}t-\ell)= Q^{\varepsilon'}_{0\ell}(t) \quad \textrm{with }\varepsilon'=2\pi\,2^{-(k+1)}~.$$

In other words,

$$\mathbf{V}_{k+1}=span\{\sqrt{2^{-(k+1)}}\,\phi(2^{-(k+1)}t-\ell):~\ell=0,\pm1,\pm2,\cdots\}~.$$

The successful application of the principle of unit-economy to the basis of $\mathbf{V}_{k+1}$ can be extended to the bases of $\mathbf{O}_{k+1}^\pm$ in an analogous manner. First of all, one observes that the basis elements of $\mathbf{O}_{k+1}^+$ , Eq.(2.89), are precisely the positive frequency wavelets $W_{k+1\,\ell}^+(t)$ , Eq.(2.65) with $k\rightarrow k+1$ , provided one sets $\varepsilon_0=\pi$ :

$$P_{1\ell}^{\varepsilon'/2}(t)= W_{k+1\,\ell}^+(t)\left.\right\ver... ...varepsilon_0=2\pi}\stackrel{2}{=} \sqrt{2^{-(k+2)}}\,\psi^+(2^{-(k+2)}t-\ell)~.$$ (295)

Similarly one finds that the basis elements of $\mathbf{O}_{k+1}^-$ , Eq.(2.90), are precisely the negative frequency wavelets $W_{k+1\,\ell}^-(t)$ , Eq.(2.70) with $k\rightarrow k+1$ ,

$$P_{-2\,\ell}^{\varepsilon'/2}(t)= W_{k+1\,\ell}^-(t)\left.\right\... ...varepsilon_0=2\pi}\stackrel{2}{=} \sqrt{2^{-(k+2)}}\,\psi^-(2^{-(k+2)}t-\ell)~.$$ (296)

Equality 1 follows from Eq.(2.71), while 2 follows from Eq.(2.72), and the positive (resp. negative) frequency mother wavelets $\psi^+$ (resp. $\psi^-$ ) are given by

$$\psi^\pm(t)=e^{\pm 3i\pi t} ~\frac{\sin\pi t}{\pi t}.$$ (297)

They are merely complex conjugates of each other. Substitute Eqs.(2.95) and (2.96) into Eqs.(2.91) and (2.93). The result,

$$\mathbf{O}^+_{k+1} =span\{ \sqrt{2^{-(k+2)}}\,\psi^+(2^{-(k+2)}t-\ell):~\ell=0,\pm1,\pm2,\cdots \}$$

$$\mathbf{O}^-_{k+1} =span\{ \sqrt{2^{-(k+2)}}\,\psi^-(2^{-(k+2)}t-\ell):~\ell=0,\pm1,\pm2,\cdots \}~,$$

highlights the fact that the two orthogonal subspaces $\mathbf{O}^+_{k+1}$ and $\mathbf{O}^-_{k+1}$ are spanned by basis vectors which are generated by the positive frequency mother wavelet and its complex conjugate respectively.

Thus one has the result that, for every integer $k$ , each of the wavelets $\psi^+, \psi^-$ and $\phi$ procreates its respective vector space $\mathbf{O}^+_{k+1}$ , $\mathbf{O}^-_{k+1}$ , and $\mathbf{V}_{k+1}$ . The application of this fact to their direct sum

$$\mathbf V_{k}=\mathbf V_{k+1}\oplus\mathbf O_{k+1}^+\oplus\mathbf O_{k+1}^-~,$$ (298)

is as follows: Let $f$ be any square-integrable function, and let $P_{\mathbf V_{k}}f$ , Eq.(2.85), be its $\mathbf V_{k}$ -least squares approximation. Then Eq.(2.98) expresses the fact that $P_{\mathbf V_{k}}f$ decomposes uniquely into three parts,

$$P_{\mathbf V_{k}}f=P_{\mathbf V_{k+1}}f+P_{\mathbf O^+_{k+1}}f+P_{\mathbf O^-_{k+1}}f~.$$ (299)

They are

$$P_{\mathbf V_{k+1}}f(t) =\sum_\ell 2^{-(k+1)}\phi(2^{-(k+1)} t-\ell) \langle \phi(2^{-(k+1)} u-\ell),f(u)\rangle$$ (2100)

$$P_{\mathbf O^+_{k+1}}f(t) =\sum_\ell 2^{-(k+2)}\psi^+(2^{-(k+2)} t-\ell) \langle \psi^+(2^{-(k+1)} u-\ell),f(u)\rangle$$

$$P_{\mathbf O^-_{k+1}}f(t) =\sum_\ell 2^{-(k+2)}\psi^-(2^{-(k+2)} t-\ell) \langle \psi^-(2^{-(k+1)} u-\ell),f(u)\rangle~.$$

They crystalize, within the context of resolution $2^{-k}$ ,

• the essential degrees of freedom of the space of square-integrable functions (a.k.a. signals) and
• the detail degrees of freedom relative to the next (lower) resolution $2^{-(k+1)}$ .

The representations of $f$ at resolutions $2^{-k}$ and $2^{-(k+1)}$ are given by

$$P_{\mathbf V_{k}}f(t)=\sum_\ell 2^{-k}\phi(2^{-k} t-\ell) \langle \phi(2^{-k} u-\ell),f(u)\rangle$$ (2101)

and Eq.(2.100) respectively. The basis elements

$$\phi(2^{-k} t-\ell)\quad \ell=0,\pm 1,\cdots$$
and

$$\phi(2^{-(k+1)} t-\ell)\quad \ell=0,\pm 1,\cdots$$

are the essential degrees of freedom of $f$ within the context of resolutions $2^{-k}$ and $2^{-(k+1)}$ respectively. The associated coefficients $\langle\cdots\, ,\, \cdots\rangle$ are the corresponding amplitudes. We say that these degrees of freedom are independent - there is no redundancy - because the basis elements for each resolution space $\mathbf{V}_k$ are mutually orthogonal.

When one compares a function $f$ at resolutions $2^{-k}$ and $2^{-(k+1)}$ , then the difference

$$P_{\mathbf V_{k}}f-P_{\mathbf V_{k+1}}f= P_{\mathbf O^+_{k+1}}f+P_{\mathbf O^-_{k+1}}f~,$$

is called the detail of $f$ relative to the next resolution $2^{-(k+1)}$ . The basis elements

$$\psi(2^{-(k+1)} t-\ell)\quad \ell=0,\pm 1,\cdots$$

are the corresponding detail degrees of freedom. They are independent of the essential degrees of freedom at resolution $2^{-(k+1)}$ but not so at resolution $2^{-k}$ . These detail degrees of freedom span the vector space

$$\mathbf O_{k+1}=\mathbf O_{k+1}^+\oplus\mathbf O_{k+1}^- ~,$$

which is the orthogonal complement of $\mathbf V_{k+1}$ in $\mathbf V_{k}$ . The relation between these subspaces is depicted in Figure 2.23 below.

$$\begin{array}{ccclclc} \cdots&\mathbf{V}_{k-1}&\rightarrow&... ...plus&\\ ~ &~ &~ &\mathbf O_k& ~ &\mathbf O_{k+1}&~ \end{array}$$

Figure 2.23: Hierarchical relation between the resolution subspaces and their orthogonal detail subspaces. The arrows are orthogonal projections onto the subspaces.

It is difficult to overstate the importance of the principle of unit-economy. Its application is implicit and is taken for granted through out any theoretical development, ours in particular. However, there are situations where it is instructive to highlight particularly significant instances of its application. A case in point is the introduction of the scaling function, the father wavelet $\phi(t)$ . By this process an entire aggregate of concepts has been condensed into a single new concept, a multiscale analysis (MSA), with a scaling function $\phi(t)$ residing at its core. The economy in the number of concepts achieved by this condensation is a tribute to this principle. It demands that any new concept be defined in terms of essential properties.

The gist of the last two pages consisted of the task of establishing the two alternative bases of the resolution space, Eq.(2.98), in terms of a single scaling function, Eq.(2.94), and the two ``mother wavelets'', Eq.(2.97). Furthermore, the development was based on a scaling function having a rather specialized form, the Shannon wavelet $\sin \pi t /\pi t$ . One therefore wonders whether the benefits to be gained from such a highly specialized activity are really worth the effort expended. That the answer is ``yes'' is due to the fact that the development identifies a wider principle constructively: For every MSA there is a scaling function $\phi(t)$ , and for every scaling function there exists a MSA. The assumed specialized form, Eq.(2.97), is non-essential. The identification, MSA  $\leftrightarrow~\phi(t)$ , is captured by means of the following definition:

Definition (Multiscale Resolution Analysis)

An increasing sequence of Hilbert spaces

$$\{ \mathbf V_{k}:~\{0\}\subset\cdots\subset\mathbf V_{2}\subset\m... ...mathbf V_{-1} \subset\mathbf V_{-2}\subset\cdots\subset L^2(-\infty,\infty)\}~,$$ (2102)

is said to be a multiscale analysis of the space of square integrable functions $L^2(-\infty ,\infty )$ if

1. (the Cauchy completion of) their union is that space of square integrable functions:

   $$\overline{ \bigcup_{k=-\infty}^\infty \mathbf V_{k} }=L^2(-\infty,\infty)~,$$ (2103)

   
   

2. their intersection is the zero function:

   $$\bigcap_{k=-\infty}^\infty \mathbf V_{k}=\{0\}~,$$ (2104)

   
   

3. every resolution space $\mathbf V_{k}$ is related to a central (i.e. reference) space $\mathbf V_{0}$ by a dilation of its elements:

   $$f(t)\in \mathbf V_{k}\Longleftrightarrow f(2^kt)\in \mathbf V_{0}~,$$ (2105)

   
   

4. there exists a square-integrable function $\phi$ such that its integer translates form an orthonormal basis for the central approximation space $\mathbf V_{0}$ :

   $$\mathbf V_{0} =span\{\phi(t-\ell):~\ell=0,\pm 1,\cdots\}$$ (2106)
   with

   $$\int\limits_{-\infty}^\infty \overline\phi (u-\ell)\phi(u-\ell')du=\delta_{\ell\ell'} ~.$$ (2107)

   
   

Thus a multiscale analysis (MSA) is a type of hierarchy having properties 1-4. There are many other hierarchies that have only properties 1-2. But only MSA's are characterized by property 3.

This property is the essential (distinguishing) characteristic of a MSA. It says that, in order for a hierarchy of linear spaces to be a MSA, each one of these spaces must be a scaled version of a reference space $\mathbf V_{0}$ , the central approximation space. By starting with a function in $\mathbf V_{0}$ , and applying iteratively scaling operations, compression ($\times2$ ) or dilation ( $\times2^{-1}$ ), to its argument, one moves up or down this hierarchy of approximation spaces.

The purpose of Property 4 is not to define what a MSA is. Instead, its role is to have the scaling function $\phi(t)$ serve as a unique identifier of the central approximation space $\mathbf V_{0}$ , and hence, by Property 3, of a particular MSA. Thus Property 4 establishes a unique correspondence between the set of MSA's and the set of scaling functions.

The unique identification of $\mathbf V_{0}$ is achieved by having the discrete translates of $\phi(t)$ form an orthonormal basis of $\mathbf V_{0}$ . That translation process is depicted in Figure 2.12 on page . The ablity of $\phi(t)$ to serve as a unique identifier for the whole MSA becomes evident when one applies Property 3 to this functions. One finds that the set of translated and scaled functions

$$\{ \sqrt{2^{-k}}\,\phi(2^{-k}t-\ell):~l=0,\pm 1,\cdots \}$$

form o.n. bases for the respective approximation spaces $\mathbf V_{k}$ , and hence form a basis for the whole MSA. This means that every MSA is distinguished from every other MSA by means of its scaling function $\phi(t)$ .

Consequently, the definition of a MSA by properties 1-4 not only defines what a MSA is, but also establishes a one-to-one correspondence between the set of MSA's and the set of scaling functions.

The translates of the scaling function need not be orthonormal. In that case the orthonormality condition, Eq.(2.107), gets replaced by the condition that $\{\phi(t-\ell)\}$ form a Riesz basis, i.e. that

$$A\sum_{\ell=-\infty}^\infty\vert c_\ell\vert^2\le \Vert\sum_{\ell... ...c_\ell \phi(t-\ell)\Vert^2\le B\sum_{\ell=-\infty}^\infty\vert c_\ell\vert^2~.$$

Here $A,B>0$ are positive constants, $\{c_\ell\}$ is a square-summable sequence, and $\Vert\cdots\Vert$ is the $L^2$ -norm. In that circumstance there exists a theorem due to Mallat which guarantees that the Riesz basis can be orthonormalized by an appropriate renormalization procedure in the Fourier domain of $\phi(t)$ .

Exercise 26.1 (IMPROVED FIDELITY BY AUGMENTATION)
SHOW that

$$\overline{ \bigcup_{k=-\infty}^\infty \mathbf V_{k} }=L^2(-\infty... ...y) \Longleftrightarrow \lim_{k\to -\infty}\Vert P_{ \mathbf V_{k}}f-f\Vert=0~,$$

where $P_{\mathbf V_{k}}$ is the orthogonal projection onto $\mathbf V_{k}$ and $\Vert\cdots\Vert$ is the $L^2$ -norm.

Exercise 26.2 (LOSS OF FIDELITY BY CURTAILMENT)
SHOW that

$$\bigcap_{k=-\infty}^\infty \mathbf V_{k}=\{0\} \Longleftrightarrow \lim_{k\to \infty}\Vert P_{ \mathbf V_{k}}f\Vert=0~.$$

Exercise 26.3 (TRANSLATION INVARIANT FUNCTION SPACES)
(a) SHOW that $\mathbf V_{0}$ is discrete translation invariant, i.e. that

$$f(t)\in \mathbf V_{0} \Longleftrightarrow f(t-\ell) \in \mathbf V_{0}~~~~\textrm{where $\ell$\ is an integer}.$$

(b) SHOW that $\mathbf V_{k}$ is $2^k$ -shift invariant, in particular that

$$f(t)\in \mathbf V_{k} \Longleftrightarrow f(t-2^{k}\ell) \in \mathbf V_{k}~.$$

Footnotes

... unit-economy²²

As identified in the footnote on Page .