[figure: “JMMBook” 25/9/2004 page i]

Contents

I Linear Image Analysis 1

1 The Heat Equation 3 1.1 Linear smoothing and the Laplacian . . . . . . . . . . . . . . . . 3 1.2 Existence and uniqueness of solutions of the heat equation …. 6 1.3 Comments and references . . . . . . . . . . . . . . . . . . . . . . 17 1.4 Applications of the Heat Equation to Image Analysis . . . . . . . 19

1.4.1 Convergence of iterated filters to the heat equation …. 19 1.5 Directional averages and directional heat equations . . . . . . . . 24 1.6 Edge detection and linear scale space . . . . . . . . . . . . . . . . 26

1.6.1 The edge detection doctrine . . . . . . . . . . . . . . . . . 26 1.6.2 Discussion and critique . . . . . . . . . . . . . . . . . . . 27

1.7 Dynamic shape . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 1.8 Curve evolution using the heat equation . . . . . . . . . . . . . . 32 1.9 Restoring locality and causality . . . . . . . . . . . . . . . . . . . 34

1.9.1 Localizing the dynamic shape method . . . . . . . . . . . 34 1.9.2 Renormalized heat equation for curves . . . . . . . . . . . 35

1.10 Comments and references . . . . . . . . . . . . . . . . . . . . . . 36

II Contrast-Invariant Image Analysis 41

2 Contrast-Invariant Functions and Their Level Sets 43

2.1 From an image to its level sets and back . . . . . . . . . . . . . . 43

2.1.1 Functions and level sets defined almost everywhere …. 47 2.2 Contrast changes and level sets . . . . . . . . . . . . . . . . . . . 50 2.3 Semicontinuous contrast changes . . . . . . . . . . . . . . . . . . 55 2.4 Comments and references . . . . . . . . . . . . . . . . . . . . . . 57

3 Level Lines and Level Surfaces 59 3.1 Curves: tangents, normals, and curvature . . . . . . . . . . . . . 59 3.2 The level-line structure (or topographic map) of an image …. 63 3.3 The topographic map as a complete image representation …. 65 3.4 Generalized level lines and topographic map . . . . . . . . . . . . 69 3.5 Curvature of the level lines . . . . . . . . . . . . . . . . . . . . . 69 3.6 The principal curvatures of a level surface . . . . . . . . . . . . . 72 3.7 Comments and references . . . . . . . . . . . . . . . . . . . . . . 76

i

[figure: “JMMBook” 25/9/2004 page ii]

ii CONTENTS

4 The Main Contrast-Invariant Equations 77 4.1 The definition of a shape and how it is recognized . . . . . . . . 77 4.2 Multiscale features and scale space . . . . . . . . . . . . . . . . . 78 4.3 From image motion to curve motion . . . . . . . . . . . . . . . . 80

4.3.1 Curvature equations for curves and real-valued functions . 80 4.3.2 Introduction to the affine curve and function equations . . 83 4.3.3 The affine scale space as an intrinsic heat equation …. 85

4.4 Curvature motion in N dimensions . . . . . . . . . . . . . . . . . 86 4.5 Comments and references . . . . . . . . . . . . . . . . . . . . . . 87

5 Contrast-Invariant Monotone Operators 89 5.1 Contrast-invariant function operators . . . . . . . . . . . . . . . . 89 5.2 From contrast-invariant function operators to set operators … 92 5.3 From set operators to contrast-invariant function operators … 95 5.4 The extrema killer: An application . . . . . . . . . . . . . . . . . 101 5.5 Comments and references . . . . . . . . . . . . . . . . . . . . . . 106

6 Contrast-Invariant Monotone Operators as Sup-Inf Operators 107 6.1 Translation-invariant monotone set operators . . . . . . . . . . . 107 6.2 Sup-inf operators . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 6.3 The Evans–Spruck extension . . . . . . . . . . . . . . . . . . . . 114 6.4 The extension to upper semicontinuous functions . . . . . . . . . 116 6.5 A Who’s Who of monotone operators . . . . . . . . . . . . . . . 119 6.6 Comments and references . . . . . . . . . . . . . . . . . . . . . . 121

7 Erosions and Dilations 123

7.1 Erosions and dilations as multiscale contrast-invariant monotone

operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 7.2 The PDEs associated with erosions and dilations . . . . . . . . . 131 7.3 Comments and references . . . . . . . . . . . . . . . . . . . . . . 133

8 Median Filters and Mathematical Morphology 135 8.1 The median operators Med_{k} and Med_{k} . . . . . . . . . . . . . . 135 8.2 The median operators Med_{k}^{−} and Med_{k}^{−} . . . . . . . . . . . . . . 137 8.3 Discrete median filters and the ”usual” median value . . . . . . . 140 8.4 Comments and references . . . . . . . . . . . . . . . . . . . . . . 143

III Local Asymptotic Analysis of Operators 145

9 Asymptotic Behavior of Contrast-Invariant Operators in ℝ^{2} 147

9.1 Asymptotic behavior theorem in ℝ^{2} . . . . . . . . . . . . . . . . 148 9.1.1 Proof strategy and types of results . . . . . . . . . . . . . 149 9.1.2 The asymptotic behavior of T_{h} when T [x](0) = 0 . . . . . 151

9.2 Median filters and curvature motion in ℝ^{2} . . . . . . . . . . . . . 155 9.3 Comments and references . . . . . . . . . . . . . . . . . . . . . . 159

[figure: “JMMBook” 25/9/2004 page iii]

CONTENTS iii

10 Asymptotic Behavior of Contrast-Invariant Operators in ℝ^{N} 165 10.1 Asymptotic behavior theorem in ℝ^{N} . . . . . . . . . . . . . . . . 165 10.2 Asymptotic behavior of median filters in ℝ^{N} . . . . . . . . . . . 167 10.3 Applications in three dimensions . . . . . . . . . . . . . . . . . . 171 10.4 Comments and references . . . . . . . . . . . . . . . . . . . . . . 173

11 Affine-Invariant Mathematical Morphology 175 11.1 Notes on geometry . . . . . . . . . . . . . . . . . . . . . . . . . . 175 11.2 Affine-invariant erosions and dilations . . . . . . . . . . . . . . . 176 11.3 Comments and references . . . . . . . . . . . . . . . . . . . . . . 182

12 Localizability and the Maximum Principle 185 12.1 Localizable sets of structuring elements . . . . . . . . . . . . . . 186 12.2 The local maximum principle . . . . . . . . . . . . . . . . . . . . 187 12.3 B_{aff} is 1-localizable . . . . . . . . . . . . . . . . . . . . . . . . . . 190 12.4 Comments and references . . . . . . . . . . . . . . . . . . . . . . 194

13 Asymptotic Behavior of Affine-Invariant Filters 195 13.1 The analysis of IS_{h} . . . . . . . . . . . . . . . . . . . . . . . . . . 196 13.2 Alternate schemes . . . . . . . . . . . . . . . . . . . . . . . . . . 203 13.3 Comments and references . . . . . . . . . . . . . . . . . . . . . . 207

IV Viscosity Solutions and Convergence of Iterated Filters 209

14 Viscosity Solutions 211 14.1 Definition and main properties . . . . . . . . . . . . . . . . . . . 211 14.2 An application to mathematical morphology . . . . . . . . . . . . 221 14.3 Approximation theory of viscosity solutions . . . . . . . . . . . . 222 14.4 A uniqueness result for viscosity solutions . . . . . . . . . . . . . 226 14.5 Comments and references . . . . . . . . . . . . . . . . . . . . . . 226

15 Curvature Equations and Iterated Contrast-Invariant Operators 227 15.1 The main curvature equations used for image processing . . . . . 227 15.2 Contrast invariance and viscosity solutions . . . . . . . . . . . . . 228 15.3 Uniform continuity of approximate solutions . . . . . . . . . . . . 231 15.4 Convergence of iterated median filters . . . . . . . . . . . . . . . 234 15.5 Convergence of iterated affine-invariant operators . . . . . . . . . 235 15.6 Comments and references . . . . . . . . . . . . . . . . . . . . . . 239

V An Axiomatic Theory of Scale Spaces 241

16 Scale Spaces and Partial Differential Equations 243 16.1 Basic assumptions about scale spaces . . . . . . . . . . . . . . . . 243 16.2 Causal scale spaces are governed by PDEs . . . . . . . . . . . . . 247 16.3 Scale spaces yield viscosity solutions . . . . . . . . . . . . . . . . 249 16.4 Scale space invariants and implications for F . . . . . . . . . . . 250

16.4.1 Translation, rotation, and reflection . . . . . . . . . . . . 250

[figure: “JMMBook” 25/9/2004 page iv]

iv CONTENTS

16.4.2 Contrast invariance . . . . . . . . . . . . . . . . . . . . . . 252 16.4.3 Scale and affine invariance . . . . . . . . . . . . . . . . . . 254

16.5 Axiomatic approach to linear scale space . . . . . . . . . . . . . . 260 16.6 Comments and references . . . . . . . . . . . . . . . . . . . . . . 263

17 The Contrast-Invariant and Affine-Invariant Scale Spaces 269 17.1 The two-dimensional case . . . . . . . . . . . . . . . . . . . . . . 269 17.2 Contrast-invariant scale space equations in N dimensions …. 274 17.3 Affine-invariant scale spaces for N ≥ 2 . . . . . . . . . . . . . . . 277 17.4 Comments and references . . . . . . . . . . . . . . . . . . . . . . 281

A Jordan Curves 283

[figure: “JMMBook” 25/9/2004 page 1]

Part I

Linear Image Analysis

1

Chapter 1

The Heat Equation

The heat equation is the prototype of all the PDEs used in image analysis. As such, its mathematics forms an essential background that is used throughout the book. We begin the analysis of the heat equation in this chapter by proving the existence and uniqueness of its solutions, which incidentally reveals the connection between the Gaussian and the heat equation. A simple example of linear smoothing is used to illustrate the relation between linear smoothing and the Laplacian and to pave the way for a more detailed analysis in the next chapter.

1.1 Linear smoothing and the Laplacian

Consider a function u_{0}, which we interpret to be an image, defined on a domain Ω of ℝ^{2}, and assume that u_{0} ∈ C^{2}(Ω). If we wish to smooth u_{0}, then the simplest way to do so without favoring a particular direction is to replace u_{0}(x) with the unweighted average of the values of u_{0} in a disk D(x, h) of radius h centered at x. This means that we replace u_{0}(x) with

M_{h}u_{0}(x) = \frac{1}{πh^{2}} ∫_{D(x,h)} u_{0}(y) dy = \frac{1}{πh^{2}} ∫_{D(0,h)} u_{0}(x + y) dy.  (1.1)

Although the operator M_{h} is quite simple, it exhibits important characteristics of a general linear isotropic smoothing operator. For example, it is localizable: As h becomes small, M_{h} becomes more localized, that is, M_{h}u_{0}(x) depends only on the values of u_{0}(x) in a small neighborhood of x. Other characteristic properties will be mentioned as we proceed. Smoothing an image by averaging over a small symmetric area is illustrated in Figure 1.1.

Our objective is to show the relation between the action of M_{h} and the action of the Laplacian, or the heat equation. We begin by expanding u_{0} around the point x using Taylor’s formula:

u_{0}(x + y) = u_{0}(x) + Du_{0}(x) · y + \frac{1}{2}D^{2}u_{0}(x)(y, y) + o(|y|^{2}).  (1.2)

Now apply M_{h} to both sides of this expansion. On the left-hand side we have, by definition

\frac{1}{πh^{2}} ∫_{D(0,h)} u_{0}(x + y) dy = M_{h}u_{0}(x).

3 4 CHAPTER 1. THE HEAT EQUATION

Figure 1.1: Local averaging algorithm. Left to right: original image; result of replacing the grey level at each pixel by the average of the grey levels over the neighboring pixels. The shape of the neighborhood is shown by the black spot displayed in the upper right-hand corner.

The first term on the right-hand side is

\frac{1}{πh^{2}} ∫_{D(0,h)} u_{0}(x) dy = u_{0}(x).

The second (linear) term on the right-hand side vanishes because

∫_{D(0,h)} x dy = ∫_{D(0,h)} y dy = 0.  (1.3)

Similarly, the cross term in the third (quadratic) term on the right-hand side vanishes:

∫_{D(0,h)} xy dy = 0.  (1.4)

This leaves the problem of computing the other quadratic terms, but since

∫_{D(0,h)} x^{2} dy = ∫_{D(0,h)} y^{2} dy,  (1.5)

we have

∫_{D(0,h)} x^{2} dy = \frac{1}{2} ∫_{D(0,h)}(x^{2} + y^{2}) dy = \frac{1}{2} ∫_{0}^{h} ∫_{0}^{2πr^{3}} dr dθ = \frac{πh^{4}}{4} .

The last term on the right-hand side of (1.2) is the error term. As long as the point x = 0 is fixed, we can say that ∫_{D(0,h)} o(|y|^{2}) dy = πh^{2}o(h^{2}) = h^{4}ε(h), where ε is the generic error function. “Generic” means that different appearances of ε may, and probably do, represent different functions. We will sometimes write ε(h) = ε(x, h) to remind the reader of the dependence on x.

Putting this all together, we have

M_{h}u_{0}(x) = u_{0}(x) + \frac{h^{2}}{8} Δu_{0}(x) + h^{2}ε(x, h).  (1.6)

1.1. LINEAR SMOOTHING AND THE LAPLACIAN 5

Equation (1.6) says that averaging the image u_{0} over a small circular area reproduces u_{0} (which we expected) plus a small amount of the Laplacian of u_{0} and an error term. Equation (1.6) provides the theoretical basis for deblurring an image by subtracting a small amount of its Laplacian.

Notice that there are no linear derivatives and no mixed derivatives on the right-hand side of equation (1.6); only the Laplacian remains. This happened because of the relations (1.3), (1.4), and (1.5). These three properties of M_{h} will appear as assumptions in the next chapter when we treat general linear smoothing operators.

Next, assume that we have established the existence and uniqueness of the solution of the heat equation

\frac{∂u}{∂t}(t, x) = \frac{1}{8}Δu(t, x)  (1.7)

with the initial condition u_{0}(x) = u(0, x) and that u has continuous second derivatives. Then, for fixed t_{0} > 0 and x, we can apply M_{h} to u, and by (1.6) we have

M_{h}u(t_{0}, x) = u(t_{0}, x) + \frac{h^{2}}{8} Δu(t_{0}, x) + h^{2}ε(t_{0}, x, h).  (1.8)

Since u is a solution of (1.7), we can also write

u(t_{0} + h^{2}, x) = u(t_{0}, x) + \frac{h^{2}}{8} Δu(t_{0}, x) + h^{2}ε(t_{0}, x, h).  (1.9)

Subtracting (1.9) from (1.8) shows that

M_{h}u(t_{0}, x) = u(t_{0} + h^{2}, x) + h^{2}ε(t_{0}, x, h).  (1.10)

This last equation says that applying M_{h} to a solution of the heat equation (1.7) advances time by h^{2}, plus an error term.

We would like to iterate M_{h}, but to do so we need to control the error terms ε(t_{0}, x, h), and for this we assume that these terms are bounded uniformly for x ∈ Ω and for t_{0} ≤ t ≤ t_{1} by δ(h) > 0, where δ(h) → 0 as h → 0. (This assumption, and all the other details, will be justified in section 2.1.) This assumption implies that when we apply M_{h} to any of the ε error functions, we get another error term bounded by the same bound, that is,

|M_{h}ε(t, x, h)| ≤ \frac{1}{πh^{2}} ∫ |ε(t, y, h)| dy ≤ \frac{1}{πh^{2}} ∫ δ(h) dy = δ(h).

Having assumed a uniform bound on the error functions, we are ostensibly free to apply M_{h} successively to (1.10). Doing so yields the equations

M_{h}^{n}u(t_{0}, x) = u(t_{0} + nh^{2}, x) + nh^{2}ε(h),  (1.11)

where n = 1, 2, 3, …. The obvious problem with this is that for a fixed h and fixed t_{0}, t_{0} +nh^{2} will soon exceed t_{1}, and we no longer necessarily have a bound on the error terms.

It is here that the asymptotic idea enters the discussion. If we let n → +∞ and at the same time require that nh^{2} → t, t ≤ t_{1}, then

M_{h}^{n}u(t_{0}, x) → u(t_{0} + t, x).  (1.12)

6 CHAPTER 1. THE HEAT EQUATION

Figure 1.2: The Gaussian in two dimensions.

If we take t_{0} = 0, which has not been justified, then we would have

lim_{{n→+∞ \atop nh^{2}→t}} M_{h}^{n}u_{0}(x) = u(t, x).  (1.13)

This says that the asymptotic limits of M_{h}^{n} operate on u_{0} exactly as the heat equation (1.7) does; it thus establishes a relation between smoothing with M_{h} and smoothing with the Gaussian. This “result” also tells us something about the relation between iterating M_{h} and localizing M_{h}. Section 2.1 is devoted to establishing a rigorous form of this result under general conditions. Our purpose here has been to use a simple example to introduce all of the essential elements (including (1.3) and (1.4), and (1.5) ) that appear in the general result, without the burden of technical details. We begin these details by proving that the heat equation has a unique solution for a given initial condition u_{0}, and that this solution at time t is equal to the convolution G_{t} ∗ u_{0}, where G_{t} is the Gaussian (Figure 1.2). The effect on level lines of smoothing with the Gaussian is shown in Figure 1.5.

1.2 Existence and uniqueness of solutions of the heat equation

Until now, we have been vague about the domain of definition of an image. On one hand, a real digital image is defined on a finite grid, and its continuous representations are defined on a finite domain of ℝ^{2}. On the other hand, it is convenient for technical reasons to have images defined on all of ℝ^{2}, but it is not convenient to extend them by making them zero outside their original domains of definition. We start by showing how to resolve this problem. Since it costs little extra, we will work in ℝ^{N}.

Assume that the real-valued image u is originally defined on the hypercube [0, 1]^{N}. We first extend u to the hypercube C = [−1, 1]^{N} by symmetry across 1.2. EXISTENCE AND UNIQUENESS OF SOLUTIONS OF THE HEAT EQUATION7

Figure 1.3: Image extension by symmetry, followed by periodization.

the coordinate hyperplanes: For each point x = (x_{1}, x_{2}, …, x_{N}) in C, define u(x) by

u(x) = u(|x_{1}|, |x_{2}|, …, |x_{N}|).

This extension satisfies the following relation: For each x = (x_{1}, x_{2}, …, x_{N}) in C,

u(x_{1}, x_{2}, …, x_{N}) = u(ε_{1}x_{1}, ε_{1}x_{2}, …, ε_{N}x_{N}),  (1.14)

where (ε_{1}, ε_{2}, …, ε_{N}) is any vector in {−1, 1}^{N}. Next, we extend u periodically to all of ℝ^{N}. Since the period must be two in each of the variables x_{i},

u(x_{1}, x_{2}, …, x_{N}) = u(x_{1} + 2n_{1}, x_{2} + 2n_{2}, …, x_{N} + 2n_{N}),  (1.15)

for all (x_{1}, x_{2}, …, x_{N}) ∈ ℝ^{N} and all (n_{1}, n_{2}, …, n_{N}) ∈ ℤ^{N}. Figure 1.3 illustrates this kind of extension.

The purpose of these successive extensions of u to all of ℝ^{N} is to facilitate the definition of certain operations on u, such as convolution with smoothing kernels, and, at the same time, to preserve the continuity of u if u is continuous on [0, 1]^{N}. This method of extending a function is widely used in image processing; in particular, it is used in most compression and transmission standards. It is easy to check that the extension of u is continuous on ℝ^{N} if the original u is continuous on [0, 1]^{N}.

Definition 1.1. If a function u defined on ℝ^{N} satisfies the periodicity condition (1.15), then we say that u belongs to L_{C}. We also say that u is C-periodic. If, 8 CHAPTER 1. THE HEAT EQUATION

in addition, u is bounded, then we say that it belongs to L_{C}^{∞}. If u is integrable on C, that is, if ∫_{C} |u(x)| dx < ∞, then we say that u belongs to L_{C}^{1}. We endow these spaces with the following norms:

The L_{C}^{∞} norm is ∥u∥_{L^{∞}(C)} = sup_{x∈C} |u(x)|.

The L_{C}^{1} norm is ∥u∥_{L^{1}(C)} = ∫_{C} |u(x)| dx.

If, in addition, u satisfies the symmetry conditions (1.14), then we say that u is symmetric.

Convergence in the norm of L_{C}^{∞} is called uniform convergence, and convergence in the norm of L_{C}^{1} is called L_{C}^{1} convergence. When we assume that a function u ∈ L_{C}^{∞} is C^{m}, or has other smoothness properties, we mean that u has these properties on all of ℝ^{N} and not just on C. If u ∈ L_{C}^{∞} is continuous, then it is uniformly continuous. Indeed, since u is periodic, sup_{x∈ℝ^{N}} |u(x+h)−u(x)| = sup_{x∈C} |u(x + h) − u(x)|, and since C is compact, sup_{x∈C} |u(x + h) − u(x)| = ε(|h|). In the same way, if u is C^{1}, then sup_{x∈ℝ^{N}} |Du|(x) = sup_{x∈C} |Du|(x), and so on. Here is a particularly relevant example: If the function u_{0} in equation (1.2) belongs to L_{C}^{∞} in addition to being C^{2}, then the error term o(|y|^{2}), which depends on both x and y, satisfies

sup_{x∈ℝ^{2}} |o(x, y)| ≤ |y|^{2}ε(|y|)

for some ε function.

The fact that the continuous functions with compact support are dense in L^{1}(ℝ^{N}) and in L_{C}^{1} can be used to prove that translation is continuous in these two spaces. This is the content of the following proposition.

Proposition 1.2. If u ∈ L_{C}^{1}, then

∫_{C} |u(x − y) − u(x)| dx = ε(|y|).

If u ∈ L^{1}(ℝ^{N}), then

∫_{ℝ^{N}} |u(x − y) − u(x)| dx = ε(|y|).

Proof. Fix ε > 0. There exists a continuous function v ∈ L_{C}^{∞} such that ∥u − v∥_{L^{1}(C)} ≤ ε, Since v is uniformly continuous, we have sup_{x∈C} |v(x−y)−v(x)| ≤ ε if |y| is sufficiently small. Then we have

∫_{C} |u(x − y) − u(x)| dx ≤ ∫_{C} |u(x − y) − v(x − y)| dx

+ ∫_{C} |v(x − y) − v(x)| dx + ∫_{C} |v(x) − u(x)| dx ≤ 3ε.

The proof of the second statement is an easy adaptation of this argument.

1.2. EXISTENCE AND UNIQUENESS OF SOLUTIONS OF THE HEAT EQUATION9

Definition 1.3. We say that a function g defined on ℝ^{N} belongs to the Schwartz class S if g ∈ C^{∞}(ℝ^{N}) and if for each pair of multi-indices α, β there is a constant C such that

|x|^{β}|∂^{α}g(x)| ≤ C.

Proposition 1.4. If g ∈ S, then g ∈ L^{1}(ℝ^{N}). For each pair of multi-indices α, β, the function x^{β}∂^{α}g also belongs to S, and ∂^{α}g is uniformly continuous on ℝ^{N}.

Proof. The second statement follows from the Leibnitz rule for differentiating a product. (A complete proof by induction is tedious but not profound.) By the definition of S, there is a constant C such that |x|^{N+2}|g(x)| ≤ C. Thus there is another C such that |g(x)| ≤ C/(1 + |x|^{N+2}); since C/(1 + |x|^{N+2}) ∈ L^{1}(ℝ^{N}), g ∈ L^{1}(ℝ^{N}). Finally, note that |∂^{α}g(x)| → 0 as |x| → ∞. But any continuous function on ℝ^{N} that tends to zero at infinity is uniformly continuous.

Proposition 1.5 (The Gaussian and the heat equation). For all t > 0, the function x ↦ G_{t}(x) = (1/(4πt)^{N/2})e^{−|x|^{2}/4t} belongs to S and satisfies the heat equation

\frac{∂G_{t}}{∂t} − ΔG_{t} = 0.

Proof. It is sufficient to prove the first statement for the function g(x) = e^{−|x|^{2}}. An induction argument shows that ∂^{α}g(x) = P_{α}(x)e^{−|x|^{2}}, where P_{α}(x) is a polynomial of degree |α| in the variables x_{1}, x_{2}, …, x_{N}. The fact that, for every k ∈ ℕ, x^{k}e^{−x^{2}} → 0 as |x| → +∞ finishes the proof. Differentiation shows that G_{t} satisfies the heat equation.

Linear image filtering is mainly done by convolving an image u with a positive integrable kernel g. This means that the smoothed image is given by the function g ∗ u defined as

g ∗ u(x) = ∫_{ℝ^{N}} g(x − y)u(y) dy = ∫_{ℝ^{N}} g(y)u(x − y) dy.

Note that the convolution, when it makes sense, is translation invariant. This means that g ∗ u(x − z) = g_{z} ∗ u(x), where g_{z}(x) = g(x − z). (Linear filtering with the Gaussian at several scales is illustrated in Figure 1.4.) The next result establishes properties of the convolution that we need for our treatment of the heat equation.

Proposition 1.6. Assume that u ∈ L_{C}^{1} and that g ∈ L^{1}(ℝ^{N}). Then g ∗ u(x) exists for almost every x ∈ ℝ^{N}, the function g ∗ u belongs to L_{C}^{1}, and it satisfies the inequality

∥g ∗ u∥_{L^{1}(C)} ≤ ∥g∥_{L^{1}(ℝ^{N})}∥u∥_{L^{1}(C)}.  (1.16)

If u is bounded, then g ∗ u(x) exists for every x ∈ ℝ^{N} and

∥g ∗ u∥_{L^{∞}(C)} ≤ ∥g∥_{L^{1}(ℝ^{N})}∥u∥_{L^{∞}(C)}.  (1.17)

10 CHAPTER 1. THE HEAT EQUATION

Figure 1.4: Convolution with Gaussian kernels (heat equation). Displayed from top-left to bottom-right are the original image and the results of convolutions with Gaussians of increasing variance. A grey level representation of the convolution kernel is put on the right of each convolved image to give an idea of the size of the involved neighborhood.

Proof. We start by proving that g ∗ u is well defined, that it belongs to L_{C}^{1}, and that (1.16) holds. For any x ∈ ℝ^{N}, we can always write

|g ∗ u(x)| ≤ ∫_{ℝ^{N}} |g(y)||u(x − y)| dy

and integrate both sides over C. Thus,

∫_{C} |g ∗ u(x)| dx ≤ ∫_{C} ∫_{ℝ^{N}} |g(y)||u(x − y)| dy dx.

Since everything in sight is positive, we can interchange the order of integration by Fubini’s theorem and have

∫_{C} |g∗u(x)| dx ≤ ∫_{ℝ^{N}} ∫_{C} |g(y)||u(x−y)| dx dy = ∫_{ℝ^{N}} |g(y)| ∫_{C} |u(x−y)| dx dy.

The last term on the right is equal to ∥g∥_{L^{1}(ℝ^{N})}∥u∥_{L^{1}(C)}. To see this we use the fact that u us C-periodic, which implies that ∫_{C} |u(x−y)| dx = ∫_{C} |u(x)| dx for any y ∈ ℝ^{N}. Thus the integral on the left-hand side is finite and g ∗ u(x) exists for almost every x. It is left as an exercise to show that g ∗ u is C-periodic, and thus complete the proof of the first statement. If u is bounded, then

|g ∗u(x)| ≤ ∫_{ℝ^{N}} |g(x−y)||u(y)| dy ≤ ∥u∥_{L_{C}^{∞}} ∫_{ℝ^{N}} |g(x−y)| dy = ∥u∥_{L_{C}^{∞}}∥g∥_{L_{ℝ^{N}}^{1}} ,

and we have ∥g ∗ u∥_{L_{C}^{∞}} ≤ ∥g∥_{L_{ℝ^{N}}^{1}} ∥u∥_{L_{C}^{∞}}.

Exercise 1.1. Verify that g ∗ u is C-periodic. Show that if, in addition to the assumptions of Proposition 1.6, g is even (g(−x) = g(x)) and u is symmetric (u satisfies (1.14)), then g ∗ u is also symmetric.

Inequality (1.17) provides a uniform bound on g ∗ u when u ∈ L_{C}^{∞} and g ∈ L^{1}(ℝ^{N}). For what follows, we wish to establish an estimate of ∥g ∗ u∥_{L_{C}^{∞}} 1.2. EXISTENCE AND UNIQUENESS OF SOLUTIONS OF THE HEAT EQUATION11

when we assume only that u ∈ L_{C}^{1}. We can do this by taking advantage of the C-periodicity of u and by an additional assumption about g.

Let C_{n} = {x | x = y + 2n, y ∈ C, n ∈ ℤ^{N}}. C_{n} is just the hypercube C translated so that it is centered at 2n. Using this notation, we have the inequalities

|g ∗ u(x)| ≤ ∫_{ℝ^{N}} |u(x − y)||g(y)| dy = ∑_{n∈ℤ^{N}} ∫_{C_{n}} |u(x − y)||g(y)| dy

≤ ∑_{n∈ℤ^{N}} sup_{y∈C_{n}} |g(y)| ∫_{C_{n}} |u(x − y)| dy = ∥u∥_{L^{1}(C)} ∑_{n∈ℤ^{N}} sup_{y∈C_{n}} |g(y)|.

If the last sum on the right-hand side of these inequalities is finite, we have a uniform estimate for |g ∗ u(x)|. We state this as a lemma.

Lemma 1.7. Assume that g ∈ L^{1}(ℝ^{N}) is such that

∑_{n∈ℤ^{N}} sup_{y∈C_{n}} |g(y)| < +∞.  (1.18)

Then for any u ∈ L_{C}^{1},

∥g ∗ u∥_{L^{∞}(C)} ≤ ( ∑_{n∈ℤ^{N}} sup_{y∈C_{n}} |g(y)|) ∥u∥_{L^{1}(C)}.  (1.19)

Condition (1.18) is true for the Gaussian and, in fact, for all functions g that decay rapidly at infinity, that is, functions for which, given any multi-index α, there exists a constant C = C(α) such that

|x|^{α}|g(x)| < C  (1.20)

for all x ∈ ℝ^{N}.

12 CHAPTER 1. THE HEAT EQUATION

Exercise 1.2.

(i) Show that if g decays rapidly at infinity, then for some C > 0,

∑_{n∈ℤ^{N}} sup_{x∈C_{n}} |g(x)| ≤ C ∑_{n∈ℤ^{N}} ∏_{i=1}^{N} \frac{1}{1 + |n_{i}|^{2}} < +∞.

(ii) Show that for each t_{0} > 0, there exists a C(t_{0}) > 0 such that

∑_{n∈ℤ^{N}} sup_{x∈C_{n}} |G_{t}(x)| ≤ C(t_{0})

for all t ≥ t_{0} and that C(t_{0}) ≤ ( \frac{1}{2\sqrt{πt_{0}}} + 1)^{N}.

This exercise shows, in particular, that condition (1.18) is true for functions g ∈ S. We are now going to focus on kernels that, like the Gaussian, belong to S.

Proposition 1.8. If u ∈ L_{C}^{1} and g ∈ S, then g ∗ u ∈ C^{∞}(ℝ^{N}) ∩ L^{∞}(C) and

∂^{α}(g ∗ u) = (∂^{α}g) ∗ u  (1.21)

for every multi-index α.

Proof. Since g ∈ S, g ∈ L^{1}(ℝ^{N}), as is ∂^{α}g for any multi-index α (Proposition 1.4). Thus by Proposition 1.6, g ∗ u belongs to L_{C}^{1}. To prove (1.21), it is sufficient to prove it for α = (1, 0, …, 0). Indeed, we know that ∂^{α}g is in S if g is in S, so the general case follows from the case α = (1, 0, …, 0) by induction. Letting e_{1} = (1, 0, …, 0) and using Taylor’s formula with Lagrange’s form for the remainder, we can write

\begin{aligned}g ∗ u(x + he_{1}) − g ∗ u(x) = & ∫_{ℝ^{N}} (g(x + he_{1} − y) − g(x − y))u(y) dy \\ = & ∫_{ℝ^{N}} (g(y + he_{1}) − g(y))u(x − y) dy \\ = & h∫_{ℝ^{N}} \frac{∂g}{∂x_{1}} (y)u(x − y) dy \\ & + \frac{h^{2}}{2} ∫_{ℝ^{N}} \frac{∂^{2}g}{∂x_{1}^{2}} (y + θ(y)he_{1})u(x − y) dy, \end{aligned} (1.22)

where 0 ≤ θ(y) ≤ 1. To complete the proof, we wish to have a bound on the last integral that is independent of x ∈ C. This last integral is of the form f ∗ u, where f is defined by f(y) = (∂^{2}g/∂x_{1}^{2})(y + θ(y)he_{1}). Since g ∈ S, ∂g/∂x_{1} ∈ S, and from this it is a simple computation to show that f decays rapidly at infinity. Having done this, Lemma 1.7 applies, and we deduce that g ∗ u is differentiable in x_{1} and that ∂(g ∗ u)/∂x_{1} = (∂g/∂x_{1}) ∗ u.

Proposition 1.9. Assume that g decreases rapidly at infinity, that g(x) ≥ 0 for all x ∈ ℝ^{N}, and that ∫_{ℝ^{N}} g(x) dx = 1, and define g_{t} for t > 0 as g_{t}(x) = (1/t^{N})g(x/t). Then we have the following results:

1.2. EXISTENCE AND UNIQUENESS OF SOLUTIONS OF THE HEAT EQUATION13

(i) If u_{0} ∈ L_{C}^{∞} is continuous, g_{t} ∗ u_{0} converges to u_{0} uniformly as t → 0. In addition, we have a maximum principle :

inf_{x∈C} u_{0}(x) ≤ g_{t} ∗ u_{0}(x) ≤ sup_{x∈C} u_{0}(x).  (1.23)

(ii) If we only know that u_{0} ∈ L_{C}^{1}, then

∫_{C} |g_{t} ∗ u_{0}(x) − u_{0}(x)| dx = ∥g_{t} ∗ u_{0} − u_{0}∥_{L^{1}(C)} → 0 as t → 0.

Proof. We begin by proving (i). Note first that g_{t} is normalized so that

∫_{ℝ^{N}} g_{t}(y) dy = 1.  (1.24)

Next, since g decreases rapidly at infinity, a quick computation shows that, for any η > 0,

∫_{|y|≥η} g_{t}(y) dy → 0 as t → 0.  (1.25)

Using (1.24), we have

g_{t} ∗ u_{0}(x) − u_{0}(x) = ∫_{ℝ^{N}} g_{t}(y)(u_{0}(x − y) − u_{0}(x)) dy.  (1.26)

As already mentioned, since u_{0} is continuous on C and C-periodic, it is uniformly continuous on ℝ^{N}. Thus, for any ε > 0, there is an η = η(ε) > 0 such that |u_{0}(x − y) − u_{0}(x)| ≤ ε when |y| ≤ η. Using this inequality, we have

|g_{t} ∗ u_{0}(x) − u_{0}(x)| ≤ ∫_{|y|<η} g_{t}(y)|u_{0}(x − y) − u_{0}(x)| dy
+ ∫_{|y|≥η} g_{t}(y)|u_{0}(x − y) − u_{0}(x)| dy
≤ε ∫_{|y|<η} g_{t}(y) dy + 2∥u∥_{L^{∞}(C)} ∫_{|y|≥η} g_{t}(y) dy.

Since ∫_{|y|<η} g_{t}(y) dy ≤ 1 and ∫_{|y|≥η} g_{t}(y) dy → 0 as t → 0, we conclude that g_{t} ∗ u tends to u_{0} uniformly in x as t → 0. Relation (1.23) is an immediate consequence of the assumption that g_{t}(x) ≥ 0 and equation (1.24). This completes the proof of (i).

To prove (ii), we must estimate the L_{C}^{1} norm of g ∗ u_{0} − u_{0}. By using (1.26), taking absolute values, and integrating over C, we see that

∫_{C} |g_{t} ∗ u_{0}(x) − u_{0}(x)| dx ≤ ∫_{C} ∫_{ℝ^{N}} g_{t}(y)|u_{0}(x − y) − u_{0}(x)| dy dx.

Once again everything in sight is positive, so we can interchange the order of integration to see that

∥g_{t} ∗ u_{0} − u_{0}∥_{L^{1}(C)} ≤ ∫_{ℝ^{N}} g_{t}(y) ∫_{C} |u_{0}(x − y) − u_{0}(x)| dx dy.

14 CHAPTER 1. THE HEAT EQUATION

As before, we break the integral over ℝ^{N} into two parts:

\begin{aligned}∥g_{t} ∗ u_{0} − u_{0}∥_{L^{1}(C)} ≤ & ∫_{|y|<η} g_{t}(y) ∫_{C} |u_{0}(x − y) − u_{0}(x)| dx dy \\ & + ∫_{|y|≥η} g_{t}(y) ∫_{C} |u_{0}(x − y) − u_{0}(x)| dx dy, \end{aligned} (1.27)

which is true for any η > 0. We now invoke Proposition 1.2: Given ε > 0, there is an η = η(ε) > 0 such that

∫_{C} |u_{0}(x − y) − u_{0}(x)| dx ≤ ε for |y| ≤ η.

Then for this η, we have the following estimate for the first integral on the right-hand side of (1.27):

∫_{|y|<η} g_{t}(y) ∫_{C} |u_{0}(x − y) − u_{0}(x)| dx dy ≤ ε.

For the second integral in (1.27), we have

∫_{|y|≥η} g_{t}(y) ∫_{C} |u_{0}(x − y) − u_{0}(x)| dx dy ≤ 2∥u∥_{L^{1}(C)} ∫_{|y|≥η} g_{t}(y) dy.

The right-hand side of this inequality tends to zero as t → 0. Thus, for sufficiently small t > 0, ∥g_{t} ∗u_{0} −u_{0}∥_{L^{1}(C)} ≤ ε. Since ε was arbitrary, this completes the proof of (ii).

Except for a small detail, all of the tools are in place to state and prove the main theorem of this chapter. It remains to be shown that (∂G_{t}/∂t) ∗ u_{0} = ∂(G_{t} ∗ u_{0})/∂t, where G_{t} is the Gaussian and u_{0} ∈ L_{C}^{1}. Proposition 1.8 does not apply directly. It applies to the spatial partial derivatives of G_{t} but not to the derivative with respect to t. Perhaps the simplest way to prove the result for t in the context of what we have done, is to observe that a slight modification of the proof of Proposition 1.8 does the job: Replace g with G_{t} and x_{1} with t. Then the crux of the matter is to notice that, given an interval 0 < t_{0} < t_{1}, there is a rapidly decreasing function f such that |(∂^{2}G_{t}/∂t^{2})(t + θ(t)h, y)| ≤ f(y) uniformly for t ∈ [t_{0}, t_{1}], where f depends on t_{0} and t_{1} but not on h. Then Lemma 1.7 applies, and the last integral in equation (1.22) is uniformly bounded.

Theorem 1.10 (Existence and uniqueness of solutions of the heat equation). Assume that u_{0} ∈ L_{C}^{1}. Then we have the following two results:

(i) For all t > 0 and x ∈ ℝ^{N}, u(t, ·) = G_{t} ∗ u_{0} satisfies the heat equation with initial value u_{0} in the sense that

\frac{∂u}{∂t} = Δu and ∫_{C} |u(t, x) − u_{0}(x)| dx → 0 as t → 0.  (1.28)

Furthermore, for each t > 0, u(t, ·) ∈ C^{∞}(ℝ^{N}) ∩ L_{C}^{∞}, and for any t_{0} > 0 it is uniformly bounded for t ∈ [t_{0}, +∞), specifically,

sup_{x∈ℝ^{N} ,t≥t_{0}} |u(t, x)| ≤ C(t_{0})∥u_{0}∥_{L^{1}(C)}.  (1.29)

1.2. EXISTENCE AND UNIQUENESS OF SOLUTIONS OF THE HEAT EQUATION15

(ii) Conversely, given u_{0} ∈ L_{C}^{1}, G_{t} ∗ u_{0} is the unique solution u of (1.28) that belongs to L_{C}^{∞} for each t > 0, that is uniformly bounded on [t_{0}, +∞) for each t_{0} > 0, and that is C^{2} on (0, +∞) × ℝ^{N}.

Proof. We first prove (i). For each t > 0, G_{t} ∈ S, so by Proposition 1.8 and the remarks before the statement of the theorem concerning the interchange of integration and differentiation, we know that we can write

\frac{∂u}{∂t} − Δu = u ∗ (\frac{∂G_{t}}{∂t} − ΔG_{t}).  (1.30)

Proposition 1.8 also tells us that u(t, ·) ∈ C^{∞}(ℝ^{N}) ∩ L_{C}^{∞} for each t > 0. The right-hand side of (1.30) is zero by Proposition 1.5, and the fact that ∫_{C} |u(t, x)− u_{0}(x)| dx → 0 as t → 0 follows from Proposition 1.9(ii). The inequality (1.29) will follows directly from Lemma 1.7 and Exercise 1.2(ii). This completes the proof of (i).

We prove (ii) by assuming that there are two solutions of the heat equation (1.28) and reaching a contradiction. If both v and w are solutions of the heat equation with the same initial condition u_{0} ∈ L_{C}^{1}, then u = v − w is in L_{C}^{1} and satisfies (1.28) with the initial condition u_{0}(x) = 0 for all x ∈ ℝ^{N}. In particular, ∫_{C} |u(t, y)| dy → 0 as t → 0. Also, by the assumptions of (ii), u is uniformly bounded on [t_{0}, +∞)×ℝ^{N} for each t_{0} > 0 and is C^{2} on (0, +∞)×ℝ^{N}. This last condition ensures that it makes sense to say that u satisfies the heat equation; it is also used to justify the interchange of integration and differentiation. We wish to show that u(t, x) = 0 for all t > 0 and all x ∈ ℝ^{N}. Assume that this is not the case. Then there is some point (t, x) where u(t, x) ≠ 0. Assume that u(t, x) > 0, by changing u to −u if necessary. For reasons that become clear as the argument progresses, we introduce the function u_{h}, h > 0, defined by

u_{h}(t, x) = ∫_{ℝ^{N}} G_{h}(y)u(t, y − x) dy  (1.31)

for all t > 0 and x ∈ ℝ^{N}. This new function is in L_{C}^{∞}, and by Lemma 1.7,

∥u_{h}∥_{L^{∞}L(C)} ≤ C(G_{h})∥u∥_{L^{1}(C)} → 0 as t → 0.  (1.32)

Take h sufficiently small so that u_{h}(t, x) > 0 for some (t, x) ∈ (0, +∞)×ℝ^{N} and fix h. This is possible by Proposition 1.9. The function u_{h} is also a solution of the heat equation. This is clear as soon as we justify interchanging integration and differentiation to write

\frac{∂u_{h}}{∂t} (t, x) = ∫_{ℝ^{N}} G_{h}(y)\frac{∂u}{∂t}(t, y−x) dy and Δu_{h}(t, x) = ∫_{ℝ^{N}} G_{h}(y)Δu(t, y−x) dy.

The justification follow from previous arguments and the fact that the derivatives of u ∈ L_{C}^{∞} are assumed to be continuous on (0, +∞) × ℝ^{N}.

Here, in review, are the important facts about u_{h}: It satisfies the heat equation with the initial condition u_{0}(x) = 0; u_{h}(t, x) → 0 uniformly in x as t → 0; u_{h} is bounded on [0, +∞) × ℝ^{N}; and u_{h}(t, x) > 0 for some (t, x) ∈ (0, +∞) × ℝ^{N}.

We now consider yet another function u^{ε} defined by u^{ε}(t, x) = e^{−εt}u_{h}(t, x). This function is continuous on [0, +∞) × ℝ^{N}, it is C-periodic in x, and it tends 16 CHAPTER 1. THE HEAT EQUATION

Figure 1.5: Level lines and the heat equation. Top, left to right: original 410×270 grey level image; level lines of original image for levels at multiples of 12. Bottom, left to right: original image smoothed by the heat equation (convolution with the Gaussian). The standard deviation of the Gaussian is 4, which means that its spatial range is comparable to a disk of radius 4. The image gets blurred by the convolution, which averages grey level values and removes all sharp edges. This can be appreciated on the right, where we have displayed all level lines for levels at multiples of 12. Note how some level lines on the boundaries of the image have split into parallel level lines that have drifted away from each other. The image has become smooth, but it is losing its structure.

to zero uniformly in x as t → 0 and as t → +∞. These conditions imply that u^{ε} attains its supremum at some point (t_{0}, x_{0}) ∈ (0, +∞) × C, and this means that Δu^{ε}(t_{0}, x_{0}) = e^{−εt}Δu_{h}(t_{0}, x_{0}) ≤ 0 and (∂u^{ε}/∂t)(t_{0}, x_{0}) = 0. Here is the payoff: Using the fact that u_{h} is a solution of the heat equation, we have the following relations:

0 = \frac{∂u^{ε}}{∂t} (t_{0}, x_{0}) = −εu^{ε}(t_{0}, x_{0}) + e^{−εt} \frac{∂u_{h}}{∂t} (t_{0}, x_{0})
= −εu^{ε}(t_{0}, x_{0}) + e^{−εt}Δu_{h}(t_{0}, x_{0}) ≤ −εu^{ε}(t_{0}, x_{0}) < 0.

This contradiction completes the uniqueness proof.

If the original data had been v_{0} ∈ L^{1}([0, 1]^{N}) and v_{0} had been extended to a symmetric C-periodic function u_{0}, then we know from Exercise 1.1 that u(t, ·) = G_{t} ∗ u_{0} is symmetric for all t > 0. Theorem 1.10 implies that ∥u(t, ·) − v_{0}∥_{L^{1}([0,1]^{N})} → 0 as t → 0. Note also that ∥u_{0}∥_{L^{1}(C)} can be replaced with 2^{N}∥v_{0}∥_{L^{1}([0,1]^{N})} in (1.29).

1.3. COMMENTS AND REFERENCES 17

1.3 Comments and references

The heat equation. One should not conclude from Theorem 1.10 that the solutions of the heat equation are always unique. The assumption in (ii) that the solution was bounded is crucial. In fact, without this assumption, there are solutions u that grow so fast that gu is not in L^{1}(ℝ^{N}) for g ∈ S (see, for example, [254, page 217]). Also, observe that the role of G_{h} in the proof was only that of a convenient positive kernel satisfying the conditions of Proposition 1.9. It had nothing to do with the fact that G_{t} is the fundamental solution of the heat equation. The existence and uniqueness proof of Theorem 1.10 is classic and can found in most textbooks on partial differential equations, such as Evans [94], Taylor [254], or Brezis [47].

Convolution. The heat equation—its solutions and their uniqueness—has been the main topic in this chapter, but to approach this, we have studied several aspects of the convolution, such as the continuity properties (1.16) and (1.17). We also noted that the convolution commutes with translation. Conversely, under quite certain conditions, a linear operator T defined on a function space that is continuous and translation invariant is a convolution, that is, T u = g ∗ u for some kernel g. Also, if u ≥ 0 (u(x) ≥ 0 for all x) implies that T u ≥ 0, then g ≥ 0. This is one of the fundamental theorems of both mechanics and signal processing, and it has many generalizations that depend on the domain, range, and continuity properties of T. For instance, if T is translation invariant and continuous from L^{2}(ℝ^{N}) into L^{∞}(ℝ^{N}) ∩ C(ℝ^{N}), then the convolution kernel g is in L^{2}(ℝ^{N}). This is a direct consequence of the fact that every bounded linear functional on L^{2}(ℝ^{N}) is of the form f ↦ ∫_{ℝ^{N}} f(x)g(x) dx for some g ∈ L^{2}(ℝ^{N}). This latter result was discovered independently by F. Riesz and M. Fréchet in 1907 (see [228, page 61]). Since we want smoothing to be translation invariant and continuous in some topology, this means that linear smoothing operators—which are called filters in the context of signal and image processing—are described by their convolution kernels. The Gaussian serves as a model for linear filters because it is the only one whose shape is stable under iteration. Other positive filters change their shape when iterated. Furthermore, we will show in the next chapter that a large class of linear filters tend asymptotically to the Gaussian.

1.4. APPLICATIONS OF THE HEAT EQUATION TO IMAGE ANALYSIS19

1.4 Applications of the Heat Equation to Image Analysis

1.4.1 Convergence of iterated filters to the heat equation

We wish to prove that iterated linear smoothing is asymptotic to applying the heat equation, or equivalently, to convolution with the Gaussian at different scales.

Definition 1.11. We say that a real-valued kernel g ∈ L^{1}(ℝ^{N}) is Laplacian consistent if it satisfies the following moment conditions:

1. ∫_{ℝ^{N}} g(x) dx = 1.

2. For i = 1, 2, …, N, ∫_{ℝ^{N}} x_{i}g(x) dx = 0.

3. For each pair i, j = 1, 2, …, N, i ≠ j, ∫_{ℝ^{N}} x_{i}x_{j}g(x) dx = 0.

4. For i = 1, 2, …, N, ∫_{ℝ^{N}} x_{i}^{2}g(x) dx = σ, where σ > 0.

5. ∫_{ℝ^{N}} |x|^{3}|g(x)| dx < +∞.

Note that we do not assume that g ≥ 0; in fact, many important filters used in signal and image processing are not positive. However, condition (i) implies that g is “on average” positive. More precisely, if we consider g ∗ u(x) to be a weighted average of the values of u(x), then condition (i) says that the sum of the weights is one.

We say that a function g is radial if g(x) = g(|x|), x ∈ ℝ^{N}. This is equivalent to saying that g is invariant under all rotations around the origin in ℝ^{N}.

Exercise 1.3. Assume g ∈ L^{1}(ℝ^{N}) is radial with finite second moments, ∫_{ℝ^{N}} |x|^{2}|g(x)| dx < +∞, such that ∫_{ℝ^{N}} x_{i}^{2}g(x) dx > 0. Show that g satisfies conditions (ii) and (iii) and that, for suitably chosen a, b ∈ ℝ, the rescaled function x ↦ ag(x/b) satisfies conditions (i) and (iv), where σ can be taken to be an arbitrary positive number.

Exercise 2.1 says that any radial function g ∈ L^{1}(ℝ^{N}) can be rescaled to be Laplacian consistent if it decays fast enough at infinity and if ∫_{ℝ^{N}} x_{i}^{2}g(x) dx and ∫_{ℝ^{N}} g(x) dx have the same sign. (The significance of this last condition will become clear from the proof of Theorem 2.2.)

We consider rescalings of a kernel g defined by

g_{h}(x) = \frac{1}{h^{N/2}} g(\frac{x}{h^{1/2}} )  (1.33)

for h > 0 (see Figure 1.6). Notice that this rescaling differs slightly from the one used in section 1.2. We have used the factor h^{1/2} here because it agrees with the factor t^{1/2} in the Gaussian. We denote the convolution of g with itself n times by g^{n∗}. The main result of this section concerns the behavior of g_{h}^{n∗} as n → +∞ and h → 0.

Exercise 1.4. Prove the following two statements:

(i) g_{h} is Laplacian consistent if and only it g is Laplacian consistent. (ii) If g ∈ L^{1}(ℝ^{N}), then (g_{h})^{n∗} = (g^{n∗})_{h}.

20 CHAPTER 1. THE HEAT EQUATION

Figure 1.6: The rescalings g_{t}(x) = (1/t^{2})g(x/t) of a kernel for t=4, 3, and 2.

Our first result concerns the behavior of g_{h} as h → 0. This will establish a more general and precise form of equation (1.6).

Theorem 1.12. If g is Laplacian consistent, then for every u ∈ L_{C}^{∞} ∩ C^{3},

g_{h} ∗ u(x) − u(x) = h\frac{σ}{2} Δu(x) + ε(h, x)  (1.34)

where |ε(h, x)| ≤ Ch^{3/2}.

Proof. We use condition (i), the definition of g_{h}, and rescaling inside the integral to see that

g_{h} ∗ u(x) − u(x) = ∫_{ℝ^{N}} \frac{1}{h^{N/2}} g(\frac{x}{h^{1/2}} )(u(x − y) − u(x)) dy
= ∫_{ℝ^{N}} g(z)(u(x − h^{1/2}z) − u(x)) dz.

Using Taylor’s formula with the Lagrange remainder, we have

u(x−h^{1/2}z)−u(x) = −h^{1/2}Du(x)·z+\frac{h}{2}D^{2}u(x)(z, z)−\frac{1}{6}h^{3/2}D^{3}u(x−h^{1/2}θz)(z, z, z),

where θ = θ(x, z, h) ∈ [0, 1]. By condition (ii), ∫_{ℝ^{N}} g(z)Du(x) · z dz = 0; by conditions (iii) and (iv), ∫_{ℝ^{N}} g(z)D^{2}u(x)(z, z) dz = σΔu(x). Thus,

g_{h} ∗ u(x) − u(x) = h\frac{σ}{2} Δu(x) − \frac{1}{6}h^{3/2} ∫_{ℝ^{N}} g(z)D^{3}u(x − h^{1/2}θz)(z, z, z) dz.

We denote the error term by ε(h, x). Then we have the following estimate:

|ε(h, x)| ≤\frac{1}{6}h^{2/3} ∫_{ℝ^{N}} |g(z)D^{3}u(x − h^{1/2}θz)(z, z, z)| dz
≤\frac{1}{6}h^{3/2}N^{3/2} sup_{α,x} |∂^{α}u(x)| ∫_{ℝ^{N}} |z|^{3}|g(z)| dz,

where the supremum is taken over all vectors α = (α_{1}, α_{2}, ⋯, α_{N}), α_{j} ∈ {1, 2, 3}, such that |α| = 3 and over all x ∈ C. This proves the theorem with

C = \frac{1}{6}N^{3/2} sup_{α,x} |∂^{α}u(x)| ∫_{ℝ^{N}} |z|^{3}|g(z)| dz.

1.4. APPLICATIONS OF THE HEAT EQUATION TO IMAGE ANALYSIS21

Figure 1.7: Iterated linear smoothing converges to the heat equation. In this experiment with one-dimensional functions, it can be appreciated how fast an iterated convolution of a positive kernel converges to a Gaussian. On the left are displayed nine iterations of the convolution of the characteristic function of an interval with itself, with appropriate rescalings. On the right, the same experiment is repeated with a much more irregular kernel. The convergence is almost as fast as the first case.

Theorem 2.2 shows a direct relation between smoothing with a Laplacianconsistent kernel and the heat equation. It also shows why we require σ to be positive: If it is not positive, the kernel is associated with the inverse heat equation. The next theorem is the asymptotic result that generalizes the example in section 1.1. This result is illustrated in Figure 1.7.

Theorem 1.13. Let g be a nonnegative Laplacian-consistent kernel with σ = 2 and define g_{h} by (1.33). Write T_{h}u_{0} = g_{h} ∗ u_{0} for u_{0} ∈ L_{C}^{1}, and let u(t, ·) = G_{t} ∗ u_{0} be the solution of the heat equation (1.28). Then, for each t > 0,

T_{h}^{n}u_{0} → u(t, ·) in L^{1}(C) as n → +∞ and nh → t.  (1.35)

Proof. The first order of business is to say precisely what is meant by the asymptotic limit (1.35): Given t > 0 and given ε > 0, there exists an n_{0} = n_{0}(t, ε) and a δ = δ(t, ε) such that ∥T_{h}^{n}u_{0} − u(t, ·)∥_{L^{1}(C)} ≤ ε if n > n_{0} and |nh − t| ≤ δ. This is what we must prove. It says that T_{h}^{n}u_{0} → u(t, ·) in L^{1}(C) as n → +∞ if, at the same time, h → 0 in a certain way, which is that h must be close to the asymptote t/n. We will first prove the result by requiring that h is actually on the asymptote, which means that h = t/n. We will then show that the result is true when h is suitably close to t/n.

We begin with comments about the notation. By Exercise 2.1, (T_{h})^{n} = (T^{n})_{h}, so there is no ambiguity in writing T_{h}^{n}. We will be applying T_{h}^{n} to the solution u of the heat equation, which is C-periodic and C^{∞} on (0, +∞) × ℝ^{N}. In this situation, t is considered to be a parameter, and we write T_{h}^{n}u(t, x) as shorthand for T_{h}^{n}u(t, ·)(x). Throughout the proof, we will be dealing with error terms that we write as O(h^{r}). These terms invariably depend on h, t, and x. However, in all cases, given a closed interval [t_{1}, t_{2}] ⊂ (0, +∞), there will be a constant C such that |O(h^{r})| ≤ Ch^{r} uniformly for t ∈ [t_{1}, t_{2}] and x ∈ C. Finally, keep in mind that everything in sight is C-periodic.

We wish to fix the interval [t_{1}, t_{2}], but since this depends on the point t in (1.35) and on ε, we must first choose these numbers. Thus, choose τ > 0 and keep it fixed. This will be the “t” in (1.35). Next, choose ε > 0. Here are the conditions we wish t_{1} and t_{2} to satisfy:

22 CHAPTER 1. THE HEAT EQUATION

(1) t_{1} is small enough so ∥u(t_{1}, ·) − u_{0}∥_{L^{1}(C)} < ε. (This is possible by Theorem 1.10.)

(2) t_{1} is small enough so ∥u(t_{1} + τ, ·) − u(τ, ·)∥_{L^{1}(C)} < ε. (Again, by Theorem 1.10.)

(3) t_{2} is large enough so t_{1} + τ < t_{2}.

There is no problem meeting these conditions, so we fix the interval [t_{1}, t_{2}] ⊂ (0, +∞) and use Theorem 2.2 to write

T_{h}u(t, x) − u(t, x) = hΔu(t, x) + O(h^{3/2}),  (1.36)

where t ∈ [t_{1}, t_{2}]. That the error function is bounded uniformly by Ch^{3/2} on the compact set [t_{1}, t_{2}]×C follows from the fact that sup_{α,t,x} |∂^{α}u(t, x)| is finite for (t, x) ∈ [t_{1}, t_{2}] × C (see the proof of Theorem 2.2). Since u is a solution of the heat equation, we also have

u(t + h, x) − u(t, x) = hΔu(t, x) + O(h^{2}).  (1.37)

This time the error term is bounded uniformly by Ch^{2} on [t_{1}, t_{2}] × C because u is C^{∞} on (0, +∞) × ℝ^{N}. By subtracting (1.37) from (1.36) we see that

T_{h}u(t, x) = u(t + h, x) + O(h^{3/2}).  (1.38)

This shows that applying T_{h} to a solution of the heat equation at time t advances the solution to time t + h, plus an error term.

So far we have not used the assumption that g is nonnegative. Thus, (1.38) is true for any Laplacian-consistent kernel g with σ = 2. However, we now wish to apply the linear operator T_{h} to both sides of equation (1.38), and in doing so we do not want the error term to increase. Since g ≥ 0, this is not a problem:

|T_{h}O(h^{3/2})| ≤ ∫_{ℝ^{N}} |O(h^{3/2})|g_{h}(x − y) dy ≤ ∫_{ℝ^{N}} Ch^{3/2}g_{h}(x − y) dy = Ch^{3/2}.

With this in hand, we can apply T_{h} to both sides of (1.38) and obtain

T_{h}^{2}u(t, x) = T_{h}u(t + h, x) + O(h^{3/2}).  (1.39)

If we write equation (1.38) with t + h in place of t and substitute the expression for T_{h}u(t + h, x) in equation (1.39), we have

T_{h}^{2}u(t, x) = u(t + 2h, x) + 2O(h^{3/2}).  (1.40)

We can iterate this process and get the relation

T_{h}^{n}u(t, x) = u(t + nh, x) + nO(h^{3/2})  (1.41)

with the same constant C in the estimate |O(h^{3/2})| ≤ Ch^{3/2} as long as t + nh ∈ [t_{1}, t_{2}]. To ensure that this happens, we take t = t_{1} and h = τ/n. Then

T_{h}^{n}u(t_{1}, x) = u(t_{1} + τ, x) + O((\frac{τ}{n})^{1/2})  (1.42)

and

lim_{{n→+∞ \atop nh=τ}} T_{h}^{n}u(t_{1}, x) = u(t_{1} + τ, x),  (1.43)

1.4. APPLICATIONS OF THE HEAT EQUATION TO IMAGE ANALYSIS23

where the convergence is uniform for x ∈ C. If we could take t_{1} = 0, this would end the proof. This is not possible because all of the O terms were based on a fixed interval [t_{1}, t_{2}]. However, we have taken t_{1} small enough to finish the proof .

By Proposition 1.6,

∥g_{h} ∗ v∥_{L^{1}(C)} ≤ ∥g_{h}∥_{L^{1}(ℝ^{N})}∥v∥_{L^{1}(C)}

for v ∈ L_{C}^{1}. Since g ≥ 0 and ∫_{ℝ^{N}} g(x) dx = 1, ∥g_{h}∥_{L^{1}(ℝ^{N})} = 1, and thus

∥g_{h}^{n∗} ∗ v∥_{L^{1}(C)} ≤ ∥v∥_{L^{1}(C)}.

If we take v = u(t_{1}, ·) − u_{0}, then this inequality and condition (1) imply that

∥T_{h}^{n}u(t_{1}, ·) − T_{h}^{n}u_{0}∥_{L^{1}(C)} < ε.  (1.44)

If we write (1.42) as

T_{h}^{n}u(t_{1}, x) − u(t_{1} + τ, x) = O((\frac{τ}{n})^{1/2})

and integrate both sides over C, then we also have

∥T_{h}^{n}u(t_{1}, ·) − u(t_{1} + τ, ·)∥_{L^{1}(C)} = O((\frac{τ}{n})^{1/2}) < ε  (1.45)

for all sufficiently large n, say n > n_{0}. Inequalities (1.44) and (1.45) imply that

∥T_{h}^{n}u_{0} − u(t_{1} + τ, ·)∥_{L^{1}(C)} < 2ε.  (1.46)

This inequality and condition (2) show that

∥T_{h}^{n}u_{0} − u(τ, ·)∥_{L^{1}(C)} < 3ε  (1.47)

for n > n_{0} and h = τ/n. This proves the theorem in the case h = τ/n.

It is a simple matter to obtain the more general result. Again, by Theorem 1.10, there is a δ = δ(τ, ε) such that |nh − τ| < δ implies that ∥u(nh, ·) − u(τ, ·)∥_{L^{1}(C)} < ε and that nh ∈ [t_{1}, t_{2}] (by condition (3)). Combining this with (1.47) shows that

∥T_{h}^{n}u_{0} − u(nh, ·)∥_{L^{1}(C)} < 4ε

if n > n_{0} and |nh − τ| < δ, and this completes the proof.

We will denote the characteristic function of a set A ⊂ ℝ^{N} by 1_{A}. Thus, 1_{A}(x) = 1 if x ∈ A and 1_{A}(x) = 0 otherwise.

Exercise 1.5. The kernel in the example in section 1.1 was g = (1/π)1_{D(0,1)}, where D(0, 1) is the disk of radius one centered at zero. In this case, g is a radial function and it is clearly Laplacian consistent. For N = 2, let A = [−1/2, 1/2] × [−1/2, 1/2]. Then g = 1_{A} is not radial. Show that it is, however, Laplacian consistent. If we take B = [−1, 1] × [−1/2, 1/2], then g = (1/2)1_{B} is no longer Laplacian consistent because it does not satisfy condition (iv). Show that this kernel does, however, satisfy a relation similar to (1.34).

24 CHAPTER 1. THE HEAT EQUATION

1.5 Directional averages and directional heat equations

The purpose of this section is to show how to approximate directional second derivatives using local averages. Consider a C^{2} function from ℝ^{N} into ℝ and a vector z ∈ ℝ^{N} with |z| = 1. We wish to compute the mean value of u along a segment of the line through x parallel to the vector z. To do this, we define the operator T_{h}^{z}, h ∈ [−1, 1], by

T_{h}^{z}u(x) = \frac{1}{2h} ∫_{−h}^{h} u(x + sz) ds.

This operator is the directional counterpart of the isotropic operator M_{h} defined by equation (1.1). We use Taylor’s formula to expand u at the point x along the line through x parallel to the vector z:

u(x + sz) = u(x) + sDu(x) · z + \frac{s^{2}}{2} D^{2}u(x)(z, z) + o(s^{2}).  (1.48)

By averaging both sides of (1.48) over s ∈ [−h, h], we obtain the next result.

Proposition 1.14.

T_{h}^{z}u(x) = u(x) + \frac{h^{2}}{6} D^{2}u(x)(z, z) + o(h^{2}).

Proposition 2.4 is similar to equation (1.6) and to Theorem 2.2, and it suggests that iterations of the operator T_{h}^{z} are associated with the directional heat equation

\frac{∂u}{∂t}(t, x) = \frac{1}{6}D^{2}u(t, x)(z, z)  (1.49)

in the same way that the iterations of the operator T_{h} in Theorem 2.3 are associated with the ordinary heat equation. If z is fixed, then the operator T_{h}^{z} and equation (1.49) act on u along each line in ℝ^{N} parallel to z separately; there is no “cross talk” between lines. Exercise 2.4 formalizes and clarifies these comments.

Exercise 1.6. Let z be a fixed vector in ℝ^{N} with |z| = 1 and let u_{0} be in L_{C}^{1}. Define the kernel g by g(s) = \frac{1}{2} 1_{[−1,1]}(s).

(i) Show that g satisfies the conditions of Definition 2.1 with N = 1. Compute the variance σ of g.

(ii) Show that

u(t, x) = ∫_{ℝ} u_{0}(x + sz)G_{t}(s) ds

is a solution of the directional heat equation

\frac{∂u}{∂t} (t, x) = D^{2}u(t, x)(z, z),  (1.50)

in the sense that u satisfies (1.50) and

∫_{C} |u(t, x) − u_{0}(x)| dx → 0 as t → 0.

Give an example to show that u(t, ·) is not necessarily C^{2}. This being the case, how does one interpret the right-hand side of (1.50)?

1.5. DIRECTIONAL AVERAGES AND DIRECTIONAL HEAT EQUATIONS25

(iii) Let g_{h}(s) = (6h)^{−1/2}g(s/(6h)^{−1/2}) and T_{h}u(x) = ∫_{ℝ} u(x + sz)g_{h}(s) ds. By applying Theorem 2.3 for N = 1, show that, for each t > 0,

T_{h}^{n}u_{0} → u(t, ·) in L^{1}(C) as n → +∞ and nh → t.  (1.51)

Exercise 2.4 has been established under the assumption that z is fixed. However, Proposition 2.4 is true when z is a function of x. This means that we are able to approximate the directional second derivative by taking directional averages when z varies from point to point. The main choices considered in the book are z = Du/|Du| and z = Du^{⊥}/|Du|, where Du = (u_{x}, u_{y}) and Du^{⊥} = (−u_{y}, u_{x}). Then by Proposition 2.4 we have the following limiting relations:

• Average in the direction of the gradient. By choosing z = Du/|Du|,

\frac{1}{|Du|^{2}} D^{2}u(Du, Du) = 6 lim_{h→0} \frac{T_{h}^{Du/|Du|}u − u}{h^{2}}.

We will interpret this differential operator as Haralick’s edge detector in section 2.3.

• Average in the direction orthogonal to the gradient. By choosing z = Du/|Du^{⊥}|,

\frac{1}{|Du|^{2}} D^{2}u(Du^{⊥}, Du^{⊥}) = 6 lim_{h→0} \frac{T_{h}^{Du^{⊥}/|Du|}u − u}{h^{2}}.

This differential operator appears as the second term of the curvature equation.

Although we have not written them as such, the limits are pointwise in both cases.

This section has been about approximating directional second derivatives by computing directional averages. Exercise 2.5 provides another method for computing second derivatives.

Exercise 1.7. Suppose that u ∈ C^{2}(ℝ). Assuming that u′(x) ≠ 0, show that

u′′(x) = lim_{h→0} \frac{1}{h^{2}} ( max_{s∈[−h,h]} u(x + s) + min_{s∈[−h,h]} u(x + s) − 2u(x)).  (1.52)

What is the value of the right-hand side of (1.52) if u′(x) = 0?

Now consider u ∈ C^{2}(ℝ^{2}). We wish to establish an algorithm similar to (1.52) to compute the second derivative of u in the direction of the gradient Du = (u_{x}, u_{y}). For this to make sense, we must assume that Du(x) ≠ 0. With these assumptions, we know from (1.52) that

u_{ξξ}(x) = \frac{∂^{2}v}{∂ξ^{2}} (x, 0) = lim_{h→0} \frac{1}{h^{2}} ( max_{s∈[−h,h]} u(x+sz)+ min_{s∈[−h,h]} u(x+sz)−2u(x)), (1.53)

where v(x, ξ) = u(x + ξz) and z = Du/|Du|. The second part of the exercise is to prove that, in fact,

u_{ξξ}(x) = lim_{h→0} \frac{1}{h^{2}} ( max_{y∈D(0,h)} u(x + y) + min_{y∈D(0,h)} u(x + y) − 2u(x)),  (1.54)

where D(0, h) is the disk of radius h centered at the origin. Intuitively, (1.54) follows from (1.53) because the gradient indicates the direction of maximal change in u(x), so in the limit as h → 0, taking max and min in the direction of the gradient is equivalent to taking max and min in the disk. The point of the exercise is to formalize this.

26 CHAPTER 1. THE HEAT EQUATION

1.6 Edge detection and linear scale space

The general analysis framework in which an image is associated with smoothed versions of itself at several scales is called scale space. We are first going to discuss some of the ways linear smoothing has been used in image analysis. We will then indicate the problems associated with the linear theory.

1.6.1 The edge detection doctrine

One of the uses of linear theory in two dimensions is edge detection. The assumption of the edge detection doctrine is that relevant information is contained in the traces produced in an image by the apparent contours of physical objects. If a black object is photographed against a white background, then one expects the silhouette of the object in the image to be bounded by a closed curve across which the light intensity u_{0} varies strongly. We call this curve an edge. At first glance, it would seem that this edge could be detected by computing the gradient Du_{0}, since at a point x on the edge, |Du_{0}(x)| should be large and Du(x) should point in a direction normal to the boundary of the silhouette. It would therefore appear that finding edges amounts to computing the gradient of u_{0} and determining the points where the gradient is large. This conclusion is unrealistic for two reasons:

(a) There may be many points where the gradient is large due to small oscillations in the image that are not related to real objects. Recall that digital images are always noisy, and thus there is no reason to assume the existence or computability of a gradient.

(b) The points where the gradient exceeds a given threshold are likely to form regions and not curves.

As we emphasized in the Introduction, objection (a) is dealt with by smoothing the image. We associate with the image u_{0} smoothed versions u(t, ·), where the scale parameter t indicates the amount of smoothing. In the classical linear theory, this smoothing is done by convolving u_{0} with the Gaussian G_{t}.

One way that objection (b) has been approached is by redefining edge points. Instead of just saying an edge point is a point x where |Du_{0}(x)| exceeds a threshold, one requires the gradient to satisfy a maximal property. We illustrate this in one dimension. Suppose that u ∈ C^{2}(ℝ) and consider the points where |u′(x)| attains a local maximum. At some of these points, the second derivative u′′ changes sign, that is, sign(u′′(x − h)) ≠ sign(u′′(x + h)) for sufficiently small h. These are the points where u′′ crosses zero, and they are taken to be the edge points. Note that this criterion avoids classifying a point x as an edge point if the gradient is constant in an interval around x. Marr and Hildreth generalized this idea to two dimensions by replacing u′′ with the Laplacian Δu, which is the only isotropic linear differential operator of order two that generalizes u′′ [191]. Haralick’s edge detector is different but in the same spirit [129]. Haralick gives up linearity and define edge points as those points where the gradient has a local maximum in the direction of the gradient. In other words, an edge point x satisfies g′(0) = 0, where g(t) = |Du(x + tDu(x)|/|Du(x)|. This implies that D^{2}u(x)(Du(x), Du(x)) = 0.

Exercise 1.8. Check that g′(0) = 0 implies that D^{2}u(x)(Du(x), Du(x)) = 0.

1.6. EDGE DETECTION AND LINEAR SCALE SPACE 27

We are now going to state these two algorithms formally. They are illustrated in Figures 1.9 and 1.10, respectively.

Algorithm 1.1 (Edge detection: Marr–Hildreth zero-crossings).

(1) Create the multiscale images u(t, ·) = G_{t} ∗ u_{0} for increasing values of t.

(2) At each scale t, compute all the points where Δu changes sign and Du ≠ 0. These points are called zero-crossings of the Laplacian, or simply zerocrossings.

(3) (Optional) Eliminate the zero-crossings where the gradient is small.

Algorithm 1.2 (Edge detection: The Haralick–Canny edge detector).

(1) As before, create the multiscale images u(t, ·) = G_{t} ∗u_{0} for increasing values of t.

(2) At each scale t, find all points x where Du(x) ≠ 0 and D^{2}u(x)(z, z) crosses zero, z = Du/|Du|. At such points, the function s ↦ u(x + sz) changes from concave to convex, or conversely, as s passes through zero.

(3) At each scale t, fix a threshold θ(t) and retain as edge points at scale t only those points found above that satisfy |Du(x)| > θ(t).

In practice, edges are computed for a finite number of dyadic scales, t = 2^{n}, n ∈ ℤ.

1.6.2 Discussion and critique

The Haralick–Canny edge detector is generally preferred for its accuracy to the Marr–Hildreth algorithm. Their use and characteristics are, however, essentially the same. There are also many variations—attempted improvements—of the algorithms we have described, and the following discussion adapts easily to these related edge detection schemes. The first thing to notice is that, by Theorem 1.10, u(t, ·) = G_{t} ∗ u_{0} is a C^{∞} function for each t > 0 if u_{0} ∈ L_{C}^{1}. Thus we can indeed compute second order differential operators applied to u(t, ·) = G_{t} ∗ u_{0}, t > 0. In the case of linear operators like the Laplacian or the gradient, the task is facilitated by the formula proved in Proposition 1.8. For example, we have Δu(t, x) = Δ(G_{t} ∗ u_{0})(x) = (ΔG_{t}) ∗ u_{0}(x), where in dimension two (Figure 1.8),

ΔG_{t}(x) = \frac{|x|^{2} − 4t}{16πt^{3}}e^{−|x|^{2}/4t}.

In the same way, Haralick’s edge detector makes sense because u is C^{∞} at all points where Du(x) ≠ 0. If Du(x) = 0, then x cannot be an edge point, since u is “flat” there. Thus, thanks to the filtering, there is no theoretical problem with computing edge points. There are, however, practical objections to these methods, which we will now discuss.

28 CHAPTER 1. THE HEAT EQUATION

Figure 1.8: A three-dimensional representation of the Laplacian of the Gaussian. This convolution kernel, which is a wavelet, is used to estimate the Laplacian of an image at different scales of linear smoothing.

Figure 1.9: Zero-crossings of the Laplacian at different scales. This figure illustrates the original scale space theory as developed by David Marr [190]. To extract more global structure, the image is convolved with Gaussians whose variances are powers of two. One computes the Laplacian of the smoothed image and displays the lines along which this Laplacian changes sign: the zerocrossings of the Laplacian. According to Marr, these zero-crossings represent the “raw primal sketch” of the image, or the essential information on which further vision algorithms should be based. Above, left to right: the results of smoothing and the associated Gaussian kernels at scales 1, 2, and 4. Below, left to right: the zero-crossings of the Laplacian and the corresponding kernels, which are the Laplacians of the Gaussians used above.

Linear scale space

The first serious problems are associated with the addition of an extra dimension: Having many images u(t, ·) at different scales t confounds our understanding of the image and adds to the cost of computation. We no longer have an absolute definition of an edge. We can only speak of edges at a certain scale. Conceivably, a way around this problem would be to track edges across scales. In fact, it has been observed in experiments that the “main edges” persist under convolution as t increases, but they lose much of their spatial accuracy. On the other hand, filtering with a sharp low-pass filter, that is, with t small, keeps these edges in their proper positions, but eventually, as t becomes very small, even these main edges can be lost in the crowd of spurious edge signals due to noise and texture. The scale space theory of Witkin proposes to identify the main edges at some scale t and then to track them backward as t decreases [274].

1.6. EDGE DETECTION AND LINEAR SCALE SPACE 29

Figure 1.10: Canny’s edge detector. These images illustrate the Canny edge detector. Left column: result of the Canny filter without the threshold on the gradient. Middle column: result with a visually “optimal” scale and an imagedependent threshold (from top to bottom: 15, 0.5, 0.6). Right column: result with a fixed gradient threshold equal to 0.5. Note that this edge detection “theory” depends on no fewer than three parameters that must be fixed by the user.

In theory, it would seem that this method could give an accurate location of the main edges. In practice, any implementation of these ideas is computationally costly due to the problems involved with multiple thresholdings and following edges across scales. In fact, tracking edges across scales is incompatible with having thresholds for the gradients, since such thresholds may remove edges at certain scales and not at others. The conclusion is that one should trace all zero-crossings across scales without considering whether they are true edges or not. This makes matching edges across scales very difficult. For example, experiments show that zero-crossings of sharp edges that are sparse at small scales are no longer sparse at large scales. (Figure 1.11 shows how zero-crossings can be created by linear smoothing.) The Haralick–Canny detector suffers from the same problems, as is well demonstrated by experiments.

Other problems with linear scale space are illustrated in Figures 1.12 and 1.13. Figure 1.12 illustrates how linear smoothing can create new gray levels and new extrema. Figure 1.13 shows that linear scale space does not maintain the inclusion between objects. The shape inclusion principal will be discussed in Chapter 21.

We must conclude that the work on linear edge detection has been an attempt to build a theory that has not succeeded. After more than thirty years of activity, it has become clear that no robust technology can be based on these ideas. Since edge detection algorithms depend on multiple thresholds on the gradient, followed by “filling-the-holes” algorithms, there can be no scientific agreement on the identification of edge points in a given image. In short, the 30 CHAPTER 1. THE HEAT EQUATION

Figure 1.11: Zero-crossings of the Laplacian of a synthetic image. Left to right: the original image; the image linearly smoothed by convolution with a Gaussian; the sign of the Laplacian of the filtered image (the gray color corresponds to values close to 0, black to clear-cut negative values, white to clear-cut positive values); the zero-crossings of the Laplacian. This experiment clearly shows a drawback of the Laplacian as edge detector.

problems associated with linear smoothing followed by edge detection have not been resolved by the idea of chasing edges across scales.

Figure 1.12: The heat equation creates structure. This experiment shows that linear scale space can create new structures and thus increase the complexity of an image. Left to right: The original synthetic image (a) contains three gray levels. The black disk is a regional and absolute minimum. The “white” ring around the black disk is a regional and absolute maximum. The outer gray ring has a gray value between the other two and is a regional minimum. The second image (b) shows what happens when (a) is smoothed with the heat equation: New extrema have appeared. Image (c) illustrates the action on (a) of a contrast-invariant local filter, the iterated median filter, which is introduced in Chapter 9.

Contrast invariance

As already mentioned in the Introduction, a central theme of the book is that the use of contrast-invariant operators will solve most of the technical problems associated with linear smoothing and other linear image operators. The development of these ideas starts in Chapter 3.

Recall from section I.3 that an (image) operator u ↦ T u is contrast invariant if T commutes with all nondecreasing functions g, that is, if

g(T u) = T (g(u)).  (1.55)

1.6. EDGE DETECTION AND LINEAR SCALE SPACE 31

Figure 1.13: Violation of the inclusion by the linear scale space. Top, left: an image that contains a black disk enclosed by a white disk. Top, right: At a certain scale, the black and white circles mix together. Bottom, left: The boundaries of the two circles. Bottom, right: After smoothing with a certain value of t, the inclusion that existed for very small t in no longer preserved. The topology of the situation has changed.

If image analysis is to be robust, it must be invariant under changes in lighting that produce contrast changes. It must also be invariant under the nonlinear response of the sensors used to capture an image. These, and perhaps other, contrast changes are modeled by g. If g is strictly increasing, then relation (1.55) ensures that the filtered image T u = g^{−1}(T (g(u))) does not depend on g. A problem with linear theory is that linear smoothing, that is, convolution, is not generally contrast invariant:

g(k ∗ u) ≠ k ∗ (g(u)).

In the same way, the operator T_{t} that maps u_{0} into the solution of the heat equation, u(t, ·) is not generally contrast invariant. In fact, if g is C^{2}, then

\frac{∂(g(u))}{∂t} = g′(u)\frac{∂u}{∂t}

and

Δ(g(u)) = g′(u)Δu + g′′(u)|Du|^{2}.

32 CHAPTER 1. THE HEAT EQUATION

Figure 1.14: Nonlocal interactions in the dynamic shape method. Left to right: Two close disks interact as the scale increases. This creates a new, qualitatively different, shape. The change of topology, at the scale where the two disks merge into one shape, also entails the appearance of a singularity (a cusp) on the shape(s) boundaries.

Of course, if g is linear, then g(u) satisfies the heat equation if u does.

Exercise 1.9. Construct simple functions u, g, and k such that g(k ∗ u) ≠ k ∗ (g(u)).

1.7 Dynamic shape

In 1986, Koendernik and van Doorn defined a shape in ℝ^{N} to be a closed subset X of ℝ^{N} [167]. They then proposed to smooth the shape by applying the heat equation ∂u/∂t − Δu = 0 directly to 1_{X}, the characteristic function of X. Of course, the solution G_{t} ∗1_{X} is not a characteristic function. The authors defined the evolved shape at scale t to be

X_{t} = {x | u(t, x) ≥ 1/2}.

The value 1/2 is chosen so the following simple requirement is satisfied: Suppose that X is the half-plane X = {(x, y) | (x, y) ∈ ℝ^{2}, x ≥ 0}. The requirement is that

X = X_{t} = {(x, y) | G_{t} ∗ 1_{X}(x, y) ≥ λ},

and this is true only if λ = 1/2. There are at least two problems with dynamic shape evolution for image analysis. The first concerns nonlocal interactions, as illustrated in Figure 1.14. Here we have two disks that are near one another. The evolution of the union of both disks, considered as a single shape, is quite different from the evolution of the disks separately. A related problem, also illustrated in Figure 1.14, is the creation of singularities. Note how a singularity in orientation and the curvature of the boundary of the shape develops at the point where the two disks touch. Figure 1.15 further illustrates the problems associated with the dynamic shape method.

1.8 Curve evolution using the heat equation

We consider shapes in ℝ^{2} whose boundaries can be represented by a finite number of simple closed rectifiable Jordan curves. Thus, each curve we consider can be represented by a continuous mapping f : [0, 1] → ℝ^{2} such that f is one-to-one on (0, 1) and f(0) = f(1), and each curve has a finite length. We also assume that these curves do not intersect each other. We will focus on smoothing one 1.8. CURVE EVOLUTION USING THE HEAT EQUATION 33

Figure 1.15: Nonlocal behavior of shapes with the dynamic shape method. This image displays the smoothing of two irregular shapes by the dynamic shape method (Koenderink–van Doorn). Top left: initial image, made of two irregular shapes. From left to right, top to bottom: dynamic shape smoothing with increasing Gaussian variance. Notice how the shapes merge more and more. We do not have a separate analysis of each shape but rather a “joint analysis” of the two shapes. The way the shapes merge is of course sensitive to the initial distance between the shapes. Compare with Figure 1.17.

of these Jordan curves, which we call C_{0}. We assume that C_{0} is parameterized by s ∈ [0, L], where L is the length of the curve. Thus, C_{0} is represented as x_{0}(s) = (x(s), y(s)), where s is the length of the curve between x_{0}(0) and x_{0}(s).

At first glance, it might seem reasonable to smooth C_{0} by smoothing the coordinate functions x and y separately. If this is done linearly, we have seen from Theorem 2.3 that the process is asymptotic to smoothing with the heat equation. Thus, one is led naturally to consider the vector heat equation

\frac{∂x}{∂t}(t, s) = \frac{∂^{2}x}{∂s^{2}}(t, s)  (1.56)

with initial condition x(0, s) = x_{0}(s). If x(t, s) = (x(t, s), y(t, s)) is the solution of (1.56), then we know from Proposition 1.9 that

inf_{s∈[0,L]} x_{0}(s) ≤ x(t, s) ≤ sup_{s∈[0,L]} x_{0}(s),
inf_{s∈[0,L]} y_{0}(s) ≤ y(t, s) ≤ sup_{s∈[0,L]} y_{0}(s),

for s ∈ [0, L] and t ∈ [0, +∞). Thus, the evolved curves C_{t} remain in the rectangle that held C_{0}. Also, we know from Theorem 1.10 that the coordinate functions x(t, ·) and y(t, ·) are C^{∞} for t > 0. There are, however, at least two reasons that argue against smoothing curves this way:

34 CHAPTER 1. THE HEAT EQUATION

Figure 1.16: Curve evolution by the heat equation. The coordinates of the curves are parameterized by the arc length and then smoothed as real functions of the length using the heat equation. From A to D: the coordinates are smoothed with an increasing scale. Each coordinate function therefore is C^{∞}; the evolving curve can, however, develop self-crossings (as in C) or singularities (as in D).

(1) When t > 0, s is no longer a length parameter for the evolved curve C_{t}.

(2) Although x(t, ·) and y(t, ·) are C^{∞} for t > 0, this does not imply that the curves C_{t} have similar smoothness properties. In fact, it can be seen from Figure 1.16 that it is possible for an evolved curve to cross itself and it is possible for it to develop singularities.

In image processing, we say that a process that introduces new features, such as described in item (2) above, is not causal. (This informal definition should not be confused with the use of “causality,” as it is used, for example, when speaking about filters: A filter F is said to be causal, or realizable, if the equality of two signals s_{0} and s_{1} up to time t_{0} implies that F s_{0}(t) = F s_{1}(t) for the same period.)

Exercise 1.10. Construct a C^{∞} mapping f : [0, 1] → ℝ^{2} such that the image of [0, 1] is a square. This shows that a curve can have a C^{∞} parameterization without being smooth.

1.9 Restoring locality and causality

The main objective of this book is to redefine the smoothing processes so they are local and do not create new singularities. This can be done, as we prove in the following chapters, by alternating a small-scale linear convolution with a natural renormalization process. We end this section by introducing two algorithms as a preview to these later developments.

1.9. RESTORING LOCALITY AND CAUSALITY 35

Figure 1.17: The Bence–Merriman–Osher shape smoothing method is a localized and iterated version of the dynamic shape method. A convolution of the binary image with small-sized Gaussians is alternated with mid-level thresholding. It uses the same initial data (top, left) as in Figure 1.15. From left to right, top to bottom: smoothing with increasing scales. Notice that the shapes remain separate. In fact, their is no interaction between the evolving shapes. Each one evolves as if the other did not exist.

1.9.1 Localizing the dynamic shape method

In the case of dynamic shape analysis, we define an alternate dynamic shape algorithm as follows:

Algorithm 1.3 (The Merriman–Bence–Osher algorithm).

(1) Convolve the characteristic function of the initial shape X_{0} with G_{h}, where h is small.

(2) Define X_{1} = {x | G_{h} ∗ 1_{X_{0}} ≥ 1/2}.

(3) Set X_{0} = X_{1} and go back to (1).

This is an iterated dynamic shape algorithm. The dynamic shape method itself is an example of a median filter, which will be defined in Chapter 9. The Merriman–Bence–Osher algorithm is thus an iterated median filter (see Figure 1.17). We will see in Chapters 10 and 11 that median filters have asymptotic properties that are similar to those expressed in Theorem 2.3. In the case of median filters, the associated partial differential equation will be a curvature motion equation (defined in Chapter 5).

36 CHAPTER 1. THE HEAT EQUATION

1.9.2 Renormalized heat equation for curves

In 1992, Mackworth and Mokhtarian noticed the loss of causality when the heat equation was applied to curves [181]. Their method to restore causality looks, at least formally, like the remedy given for the nonlocalization of the dynamic shape method. Instead of applying the heat equation for relatively long times (or, equivalently, convolving the curve x with the Gaussian G_{t} for large t), they use the following algorithm:

Algorithm 1.4 (Renormalized heat equation for curves).

(1) Convolve the initial curve x_{0}, parameterized by its length parameter s_{0} ∈ [0, L_{0}], with the Gaussian G_{h}, where h is small.

(2) Let L_{n} denote the length of the curve x_{n} obtained after n iterations and let s_{n} denote its length parameter. For n ≥ 1, write x̃_{n+1}(s_{n}) = G_{h} ∗ x_{n}(s_{n}). Then reparameterize x̃_{n+1} by its length parameter s_{n+1} ∈ [0, L_{n+1}], and denote it by x_{n+1}.

(3) Iterate.

This algorithm is illustrated in Figure 1.18; it should be compared with Figure 1.16.

Theorem 1.15. Let x be a C^{2} curve parameterized by its length parameter s ∈ [0, L]. Then for small h,

G_{h} ∗ x(s) − x(s) = h\frac{∂^{2}x}{∂s^{2}} + o(h).  (1.57)

Exercise 1.11. Prove Theorem 2.5. If x is a C^{3} function of s, then the result follows directly from Theorem 2.2. The result holds, however, for a C^{2} curve.

In view of (1.57) and what we have seen regarding asymptotic limits in Theorem 2.3 and Exercise 2.4, it is reasonable to conjecture that, in the asymptotic limit, Algorithm 2.4 will yield the solution of following evolution equation:

\frac{∂x}{∂t} = \frac{∂^{2}x}{∂s^{2}},  (1.58)

where x_{0} = x(0, ·). It is important to note that (1.58) is not the heat equation (1.56). Indeed, from Algorithm 2.4 we see that s must denote the length parameter of the evolved curve x(t, ·) at time t. Since ∂^{2}x/∂s^{2} is the curvature vector Curv(x), this is easier to remember if we write (1.58) as

\frac{∂x}{∂t} = Curv(x).

We will study this nonlinear curve evolution equation in Chapter 5.

1.10. COMMENTS AND REFERENCES 37

Figure 1.18: Curve evolution by the renormalized heat equation (Mackworth– Mokhtarian). After each smoothing step, the coordinates of the curve are reparameterized by the arc length of the smoothed curve. From A to D: the curve is smoothed with an increasing scale. Note that, in contrast with the linear heat equation (Figure 1.16), the evolving curve shows no singularities and does not cross itself.

1.10 Comments and references

Smoothing and the Laplacian. One of the first tools proposed in the early days of image processing in the 1960s came, not surprisingly, directly from signal processing. The idea was to restore an image by averaging the gray levels locally (see, for example, [122] and [134]). The observation that the difference between an image and its local average is proportional to the Laplacian of the image has proved to be one of the most fruitful contributions to image processing. As noted in the Introduction, this method for deblurring an image was introduced by Kovasznay and Joseph in 1955 [168], and it was studied and optimized by Gabor in 1965 [113] (information taken from [176]). (See also [145] and [146].) Burt and Adelson based their Laplacian pyramid algorithm on this idea, and this was one of the results that led to multiresolution analysis and wavelets [50].

Asymptotics. Our proof that iterated and rescaled convolutions of a Laplacianconsistent kernel tend asymptotically to the Gaussian is a version of the De Moivre–Laplace formula, or the central limit theorem, adapted to image processing [46]. This result is particularly relevant to image analysis, since it implies that iterated linear smoothing leads inevitably to convolution with the Gaussian, or equivalently, to the application of the heat equation. We will prove this again in Chapter 19 in the context of the axiomatic development of scale space. We do not wish to imply, however, that the Gaussian is the only important kernel for image processing. The Gaussian plays a significant role in our form of image analysis, but there are other kernels that, because of their spectral and 38 CHAPTER 1. THE HEAT EQUATION

algebraic properties, have equally important roles in other aspects of signal and image processing. This is particularly true for wavelet theory.

Directional diffusion. Directional diffusion has a long history that began when Hubel and Wiesel showed the existence of direction-sensitive cells in the visual areas of the neocortex [140]. There has been an explosion of publication on directional linear filters, beginning, for example, with influential papers such as that by Daugman [82]. We note again that Gabor’s contribution to directional filtering is described in [176].

Gaussian smoothing and edge detection. The use of Gaussian filtering in image analysis is so pervasive that it is impossible to point to a “first paper.” It is, however, safe to say that David Marr’s famous book, Vision [190], and the original paper by Hildreth and Marr [191] have had an immeasurable impact on edge detection and image processing in general. The term “edge detection” appeared as early as 1959 in connection with television transmission [155]. The idea that the computation of derivatives of an image necessitates a previous smoothing has been aggressively developed by the Dutch school of image analysis [43, 111]. See also the books by Florack [106], Lindeberg [175], and Romeny [256], and the paper [99]. Haralick’s edge detector [129], as implemented by Canny [52], is probably the best known image analysis operator. A year after Canny’s 1986 paper, Deriche published a recursive implementation of Canny’s criteria for edge detection [84].

Scale space. The term “scale space” was introduced by Witkin in 1983. He suggested tracking the zero-crossings of the Laplacian of the smoothed image across scales [274]. Yuille and Poggio proved that these zero-crossings can be tracked for one-dimensional signals [278]. Hummel and Moniot [143, 147] and Yuille and Poggio [279] analyzed the conjectures of Marr and Witkin according to which an image is completely recoverable from its zero-crossings at different scales. Mallat formulated Marr’s conjecture as an algorithm in the context of wavelet analysis. He replaced the Gaussian with a two-dimensional cubic spline, and he used both the zero-crossings of the smoothed images and the nonzero values of the gradients at these points to reconstruct the image. This algorithm works well in practice, and the conjecture was that these zero-crossings and the values of the gradients determined the image. A counterexample given by Meyer shows that this is not the case. Perfect reconstruction is possible in the one-dimensional case for signals with compact support if the smoothing kernel is the Tukey window, k(x) = 1 + cos x for |x| ≤ π and zero elsewhere. An account of the Mallat conjecture and these examples can be found in [153]. Koenderink presents a general and insightful theory of image scale space in [164].

Dynamic shape, curve evolution, and restoring causality. Our account of the dynamic shape method is based on the well-known paper by Koenderink and van Doorn in which they introduced this notion [167]. The curve evolution by the heat equation if from the first 1986 version of curve analysis proposed by Mackworth and Mokhtarian [180]. See also the paper by Horn and Weldon [137]. The algorithm that restores causality and locality to the dynamic shape 1.10. COMMENTS AND REFERENCES 39

method was discovered by Merriman, Bence, and Osher, who devised this algorithm for a totally different reason: They were looking for a cleaver numerical implementation of the mean curvature equation [195]. There were model errors in the 1986 paper [180] that were corrected by the authors in their 1992 paper [181]. There, they also proposed the correct intrinsic equation. However, this 1992 paper contains several inexact statements about the properties of the intrinsic equation. The correct theorems and proofs can be found in a paper by Grayson written in 1987 [123].

Topological change under smoothing. We have included several figures that illustrate how essential topological properties of an image change when the image is smoothed with the Gaussian. Damon has made a complete analysis of the topological behavior of critical points of an image under Gaussian smoothing [80].

Part II

Contrast-Invariant Image Analysis

41

Chapter 2

Contrast-Invariant Functions and Their Level Sets

We have already argued that for image analysis to be robust, the operators must be contrast invariant. The formal development of these operators begins in this chapter and continues through Chapter 9. The overall theme is to develop a duality between contrast-invariant function operators and monotone set operators. This duality is based on one of the fundamentals of mathematical morphology: A real-valued function is completely described by its level sets. This theory will allow one to process an image in either of two equivalent ways: If the image is represented as a suitable real-valued function u, then one can form T u, where T is a contrast-invariant function operator; if the image is represented by a collection of level set, then one can operate on these level sets with the set operator T that is paired with T. Sometimes T appears as the fundamental operator and in other cases T is the fundamental operator. The theory allows us to move from one setting to the other.

2.1 From an image to its level sets and back

We will be working with real-valued functions, and for reasons that will become clear, we wish to extend the range of the functions to include −∞ and +∞. Thus we add these ideal points to ℝ and denote this set by \overline{ℝ} = ℝ∪\{−∞, +∞\}. \overline{ℝ} has the usual topology, so it is homeomorphic to, say, [−1, 1].

Definition 2.1. Let u be a function defined on ℝ^{N} with values in \overline{ℝ}. The level set of u at level λ is denoted by X_{λ}u and defined by

X_{λ}u = {x | u(x) ≥ λ}.

Technically, what we have defined is an upper level set, which is sometimes denoted by X_{λ}^{+}u. For most of our development, level sets are upper level sets, although lower level sets do appear in later chapters.

43 44CHAPTER 2. CONTRAST-INVARIANT FUNCTIONS AND THEIR LEVEL SETS

Several level sets of a digital image are illustrated in Figure 2.1 and all of the level sets of a synthetic image are illustrated in Figure 2.2. The reconstruction of an image from its level sets is illustrated in Figure 2.3. Two important properties of the level sets of a function follow directly from the definition. The first is that the level sets provide a complete description of the function. Indeed, we can reconstruct u from its level sets X_{λ}u by the formula

u(x) = sup\{λ | x ∈ X_{λ}u, λ ∈ \overline{ℝ}\}.

Exercise 2.1. Prove this last formula.

The second important property is that level sets of a function are globally invariant under certain contrast changes (see Definition 3.9). We say that two functions u and v have the same level sets globally if for every λ ∈ \overline{ℝ} there is a μ ∈ \overline{ℝ} such that X_{μ}v = X_{λ}u, and conversely. Now suppose that the contrast change g : \overline{ℝ} → \overline{ℝ} is continuous and increasing. Then it is not difficult to show that v = g(u) and u have the same level sets globally, that is, every level set of u is a level set of v, and conversely.

Exercise 2.2. Check this last statement for any function u : ℝ^{N} → \overline{ℝ} and any continuous increasing contrast change g : \overline{ℝ} → \overline{ℝ}.

Later we will be interested in more general contrast changes, which will be neither continuous nor strictly increasing. This general setting is justified by current technology where contrast changes appear implicitly in many aspects of image science. In this general setting, we will see that the level sets of g(u) are either level sets of u or strict level sets of u, which are sets of the form {x | u(x) > λ}. We will also prove a converse result: If the level sets of a function v are either level sets of u or strict level sets of u, then there is a contrast change g such the v = g(u).

The following proposition describes structural properties of the family of level sets X_{λ}u of a function u. Conversely, given a family of sets (X_{λ})_{λ∈\overline{ℝ}} that satisfies the same structural properties, it is possible to construct a function u such that X_{λ}u = X_{λ}, λ ∈ \overline{ℝ}. This illustrates one of the main techniques of mathematical morphology [194].

Proposition 2.2. Let X_{λ}, λ ∈ \overline{ℝ}, denote the level sets X_{λ}u of the function u : ℝ^{N} → \overline{ℝ}. Then the sets X_{λ} satisfy the following two structural properties:

(i) If λ > μ, then X_{λ} ⊂ X_{μ}, and X_{−∞} = ℝ^{N}.

(ii) X_{λ} = ⋂_{μ<λ} X_{μ} for every λ > −∞.

Conversely, if (X_{λ})_{λ∈\overline{ℝ}} is a family of subsets of ℝ^{N} that satisfies (i), then the level sets of the function u defined by

u(x) = sup\{λ | x ∈ X_{μ}, λ ∈ \overline{ℝ}\}  (2.1)

satisfy

X_{λ} ⊂ X_{λ}u = ⋂_{μ<λ} X_{μ}  (2.2)

for all λ > −∞. If (X_{λ})_{λ∈\overline{ℝ}} also satisfies (ii), then X_{λ}u = X_{λ} for all λ ∈ \overline{ℝ}.

2.1. FROM AN IMAGE TO ITS LEVEL SETS AND BACK 45

Figure 2.1: Level sets of a digital image. Left to right, top to bottom: We first show an image with range of gray levels from 0 to 255. Then we show eight level sets in decreasing order from λ = 225 to λ = 50, where the grayscale step is 25. Notice how essential features of the shapes are contained in the boundaries of level sets, the level lines. Each level set (which appears as white) is contained in the next one, as guaranteed by Proposition 3.2.

Proof. Relation (i) follows directly from the definition, and (ii) follows from the fact that u(x) ≥ λ if and only if u(x) ≥ μ for every μ < λ.

Now assume that the family of subsets (X_{λ})_{λ∈\overline{ℝ}} satisfies (i). We wish to prove that X_{λ} ⊂ X_{λ}u = ⋂_{μ<λ} X_{μ} for every λ ∈ \overline{ℝ}, where u is defined by (2.1). (Note that the assumption X_{−∞} = \overline{ℝ} in (i) ensures that u is well defined on all of ℝ^{N}.)

Suppose that x ∈ X_{λ}. Then it follows from the definition of u that u(x) ≥ λ, and hence x ∈ X_{λ}u. Thus, X_{λ} ⊂ X_{λ}u. Now, assume that x ∈ X_{λ}u, with λ > −∞. Then u(x) = sup{ν | x ∈ X_{ν}} ≥ λ. Consider any μ < λ. Then there exists a μ′ such that μ < μ′ ≤ sup{ν | x ∈ X_{ν}} and x ∈ X_{μ′}. It follows from (i) that x ∈ X_{μ}. Since μ was any number less that λ, we conclude that x ∈ ⋂_{μ<λ} X_{μ}. Conversely, assume that x ∈ ⋂_{μ<λ} X_{μ}. Then x ∈ X_{μ} for all μ < λ, and this implies by the definition of u that u(x) ≥ λ. Hence x ∈ X_{λ}u, and we conclude that X_{λ}u = ⋂_{μ<λ} X_{μ}. This proves that X_{λ} ⊂ X_{λ}u = ⋂_{μ<λ} X_{μ} in case λ > −∞. If (X_{λ})_{λ∈\overline{ℝ}} satisfies (ii), then clearly X_{λ}u = X_{λ} for λ > −∞. In case λ = −∞, 46CHAPTER 2. CONTRAST-INVARIANT FUNCTIONS AND THEIR LEVEL SETS

Figure 2.2: A simple synthetic image and all of its level sets (in white) with decreasing levels, from left to right and from top to bottom.

we have X_{−∞}u ⊂ X_{−∞} from the assumption that X_{−∞} = ℝ^{N}.

Families of sets that satisfy conditions (i) and (ii) in Proposition 2.2 appear so often that it is convenient to introduce some nomenclature.

Definition 2.3. A family of subsets (X_{λ})_{λ∈\overline{ℝ}} of ℝ^{N} that satisfies condition (i) in Proposition 2.2 is called a descending chain; it is called a closed descending chain if it also satisfies condition (ii). If the sets X_{λ} all belong to some family L of subsets of ℝ^{N}, then we say that (X_{λ})_{λ∈\overline{ℝ}} is a descending chain (or closed descending chain) in L. We use the same terms when the index set is ℝ, but we will call (X_{n})_{n∈ℕ} a sequence.

The content of Proposition 2.2 is that there is a correspondence between extended real-valued function on ℝ^{N} and closed descending chains of subsets of ℝ^{N}. This correspondence is the fundamental link between function operators and set operators. We will be considering chains of sets that belong to special classes of subsets of ℝ^{N}, which we generally denote by L. For example, L might consist of all measurable subsets of ℝ^{N} or of all compact subsets of ℝ^{N}. Given any descending chain (X_{λ})_{λ∈\overline{ℝ}} in L, closed or not, it is possible to define a function u by equation (2.1). Going from u to its level sets, we see from Proposition 2.2 that the descending chain (X_{λ}u)_{λ∈\overline{ℝ}} is closed. This means that the correspondence between chains and functions is not necessarily one-to-one. The next result and remarks fix this.

Corollary 2.4. Suppose that (X_{λ})_{λ∈\overline{ℝ}} is a descending chain. Define the new closed chain (Y_{λ})_{λ∈\overline{ℝ}} by Y_{λ} = ⋂_{μ<λ} X_{μ} for λ > −∞ and Y_{−∞} = ℝ^{N}. Let u(x) = sup{λ | x ∈ X_{λ}} and let v(x) = sup{λ | x ∈ Y_{λ}}. Then u = v.

Proof. From Proposition 3.2, X_{λ}u = Y_{λ} for all λ ∈ \overline{ℝ}. Being level sets of the function u, the Y_{λ} satisfy conditions (i) and (ii). Thus, by Proposition 3.2, X_{λ}v = Y_{λ}. Since u and v have the same level sets, u(x) = v(x) for all x ∈ ℝ^{N}.

2.1. FROM AN IMAGE TO ITS LEVEL SETS AND BACK 47

This result shows that although a descending chain (X_{λ})_{λ∈\overline{ℝ}} may contain “gaps” where X_{λ} ≠ ⋂_{μ<λ} X_{μ}, the closed descending chain (Y_{λ})_{λ∈\overline{ℝ}} that is generated by “closing the gaps” produces the same function as does (X_{λ})_{λ∈\overline{ℝ}}. Here is another way to look at this result: If we say that two descending chains (X_{λ})_{λ∈\overline{ℝ}} and (Y_{λ})_{λ∈\overline{ℝ}}, X_{−∞} = Y_{−∞} = ℝ^{N} are equivalent if and only if ⋂_{μ<λ} X_{μ} = ⋂_{μ<λ} Y_{μ} for all λ ∈ \overline{ℝ}, then the set of all descending chains is partitioned into equivalence classes, and these equivalence classes are in one-to-one correspondence with the functions u : ℝ^{N} → \overline{ℝ}. Again, the requirement that X_{−∞} = ℝ^{N} is necessary to ensure that the corresponding function is defined on all of ℝ^{N}.

Notice that if u(x) < +∞ for all x ∈ ℝ^{N}, then X_{+∞}u = ∅. Conversely, if X_{+∞} = ∅ and u is defined by (2.1), then u(x) < +∞ for all x ∈ ℝ^{N}. In the same way, u(x) > −∞ for all x if and only if

⋃_{λ>−∞} X_{λ} = ℝ^{N}.

Exercise 2.3. Prove the following statements:

(i) X_{λ} = ℝ^{N} for λ < λ_{0} implies that u(x) ≥ λ_{0}, x ∈ ℝ^{N}. (ii) X_{λ} = ∅ for λ > λ_{0} implies that u(x) ≤ λ_{0}, x ∈ ℝ^{N}.

Proposition 2.5. Let u be a real-valued function with level sets X_{λ}u = {x | u(x) ≥ λ} for λ ∈ ℝ ∪ {+∞}. If (μ_{n})_{n∈ℕ} is an increasing sequence that tends to λ, then

X_{λ}u = ⋂_{n∈ℕ} X_{μ_{n}}u.  (2.3)

Also,

{x | u(x) > λ} = ⋃_{μ>λ} X_{μ}u.  (2.4)

Proof. If x ∈ X_{μ_{n}}u for all n ∈ ℕ, then u(x) ≥ μ_{n} for all n ∈ ℕ. Since μ_{n} ↑ λ, this implies that u(x) ≥ λ, which means that x ∈ X_{λ}u. This proves (2.3). Relation (2.4) follows from the simple observation that u(x) > λ if and only if there is a μ > λ such that u(x) ≥ μ.

The results proved so far remain true if ℝ^{N}, the domain of the functions, is replaced by an arbitrary set D. Although we think of the functions as being images, the results are true for any function with values in \overline{ℝ}. On the other hand, properties of the extended reals \overline{ℝ}, which serves as the range of the functions considered and as the index set for the chains (X_{λ})_{λ∈\overline{ℝ}} from which functions are constructed, are critical for what has been done. We will make additional assumptions about the functions in section 3.1.1, and properties of their domain of definition ℝ^{N} will also come into play.

2.1.1 Functions and level sets defined almost everywhere

We are going to recast Proposition 3.2 in a measure-theoretic setting and thereby obtain a simpler result. The functions and sets considered will be measurable and defined almost everywhere, and the various statements will hold almost everywhere, that is, modulo sets of measure zero. We say that a set X is 48CHAPTER 2. CONTRAST-INVARIANT FUNCTIONS AND THEIR LEVEL SETS

Figure 2.3: Reconstruction of an image from its level sets: an illustration of Proposition 3.2. We use four different subsets of the image’s level sets to give four reconstructions. Top, left: all level sets; top, right: all level sets whose gray level is a multiple of 8; bottom, left: multiples of 16; bottom, right: multiples of

32. Notice the relative stability of the image shape content under these drastic quantizations of the gray levels.

contained in a set Y almost everywhere (and write X ⊂ Y a.e.) if meas (X\Y) = 0, where “meas (X)” denotes the usual Lebesgue measure of the set X ⊂ ℝ^{N}. We say that X = Y a.e. if X ⊂ Y a.e. and Y ⊂ X a.e. We say that two functions u and v are equal almost everywhere if meas ({x | u(x) ≠ v(x)}) = 0. More generally, we say that a property P (x), x ∈ ℝ^{N}, holds almost everywhere, or for almost every x, if it is true for every x except for a set in ℝ^{N} with Lebesgue measure zero.

Lemma 2.6. Assume that (X_{λ})_{λ∈ℝ} is a descending chain of measurable sets. Then,

X_{λ} = ⋂_{μ<λ} X_{μ}  (2.5)

almost everywhere, with the possible exception of at most a countable number of λ′s,

Proof. Let h be any strictly positive continuous function in L^{1}(ℝ^{N}). (For example, one could take the Gaussian G_{1}.) Write m(X) = ∫_{X} h(x) dx, and 2.1. FROM AN IMAGE TO ITS LEVEL SETS AND BACK 49

note that m(X) = 0 if and only if meas (X) = 0 (Exercise 3.4). The function λ ↦ m(X_{λ}) is nonincreasing. Thus, it is continuous except possibly for a countable number of points where it “jumps.” Call this exceptional set Λ. Then lim_{μ→λ} m(X_{μ}) = m(X_{λ}) for λ ∈ ℝ \ Λ. For any λ and for any increasing sequence (μ_{n})_{n∈ℕ} that tends to λ, we know that m(X_{μ_{n}}) ↓ m(⋂_{n∈ℕ} X_{μ_{n}}) as n → +∞ (for example, by the countable additivity of Lebesgue measure). This is equivalent to saying that m(X_{μ}) ↓ m(⋂_{μ<λ} X_{μ}) as μ ↑ λ. Thus, for every λ ∈ ℝ \ Λ, m(⋂_{μ<λ} X_{μ}) = m(X_{λ}). Since X_{λ} ⊂ ⋂_{μ<λ} X_{μ}, this means that m(⋂_{μ<λ} X_{μ} \ X_{λ}) = 0, which implies (2.5).

Exercise 2.4. Fill in a couple of the details in the proof of Lemma 2.6: Prove that m(X) = 0 if and only if meas (X) = 0 and that m(X_{μ_{n}}) ↓ m(⋂_{n∈ℕ} X_{μ_{n}}) as n → +∞.

The (possible) exceptional set Λ is going to appear often in the following pages, and thus it is useful to note that it always arrises in the same way: It appears as the points of discontinuity, which we call jumps, in a function λ ↦ m(X_{λ}), where (X_{λ})_{λ∈\overline{ℝ}} is some descending chain of subset of ℝ^{N}. To avoid confusion, one should note that the jumps in the values of m(X_{λ}) have nothing to do with the gaps (points where X_{λ} ≠ ⋂_{μ<λ} X_{μ}) in the chain (X_{λ})_{λ∈\overline{ℝ}}. There can be only a countable number of jumps, but there can be an uncountable number of gaps. The content of Lemma 3.6 is that the gaps have measure zero except when they coincide with a jump. If the X_{λ} are the level sets of a measurable function u : ℝ^{N} → \overline{ℝ}, then we know from Proposition 3.2 that (X_{λ})_{λ∈\overline{ℝ}} contains no gaps. However, it is possible that the function λ ↦ m(X_{λ}) has up to a countable number of jumps. The next exercise sheds more light on these possible jumps.

Exercise 2.5. Assume that (X_{λ})_{λ∈\overline{ℝ}} is the family of level sets of a measurable function u : ℝ^{N} → \overline{ℝ}.

(i) Show that lim_{μ→λ,μ<λ} m(X_{μ}) = m(X_{λ}) for all λ ∈ ℝ ∪ {+∞}. (ii) Suppose that λ ∈ ℝ ∪ {−∞}. Show that m({x | u(x) = λ}) = m(X_{λ}) −

lim_{μ→λ,μ>λ} m(X_{μ}). (iii) Prove that the Lebesgue measure of {x | u(x) = λ} is greater than zero for at

most a countable number of λ ∈ \overline{ℝ}.

This exercise shows that there is a one-to-one correspondence between the points λ ∈ \overline{ℝ} where the function λ ↦ m(X_{λ}) is discontinuous and the sets {x | u(x) = λ} that have positive measure.

The next result is the “almost everywhere” version of Proposition 2.2. We note in advance that all of the sets and functions considered are assumed to be measurable.

Corollary 2.7. Let (X_{λ})_{λ∈\overline{ℝ}} be a descending chain in ℝ^{N} and define u : ℝ^{N} → \overline{ℝ} by

u(x) = sup{λ | x ∈ X_{λ}}.

Then X_{λ} = X_{λ}u almost everywhere, except possibly for at most a countable number of λ′s.

Proof. By Proposition 2.2, X_{λ} ⊂ X_{λ}u = ⋂_{μ<λ} X_{μ} for all λ > −∞. By Lemma 3.6, X_{λ} = ⋂_{μ<λ} X_{μ} almost everywhere, except possibly for at most a countable number of λ’s. This proves the result.

50CHAPTER 2. CONTRAST-INVARIANT FUNCTIONS AND THEIR LEVEL SETS

We end this section with a useful lemma that ensures the reconstruction of a function up to a set of measure zero given that we know its level sets almost everywhere at almost every level.

Lemma 2.8. Let u be a function u : ℝ^{N} → ℝ and (Y_{λ})_{λ∈ℝ} a family of sets such that for a.e. λ ∈ ℝ, X_{λ}u = Y_{λ} almost everywhere. Then u(x) = sup{λ | x ∈ Y_{λ}} for a.e. x ∈ ℝ^{N}.

Proof. Let N ⊂ ℝ be the set of measure zero for which X_{λ}u ≠ Y_{λ} almost everywhere. Then for all λ ∈ ℝ\N, X_{λ}u = Y_{λ} almost everywhere. Let Λ ⊂ ℝ\N be a countable dense subset of ℝ. Then u(x) = sup{λ | λ ∈ ℝ, x ∈ X_{λ}u} = sup{λ | λ ∈ Λ, x ∈ X_{λ}u}. For λ ∈ Λ, let N_{λ} = (X_{λ}u\Y_{λ})∪(Y_{λ} \X_{λ}u), and write M = ⋃_{λ∈Λ} N_{λ}. Then meas (M) = 0, and u(x) = sup{λ | λ ∈ Λ, x ∈ X_{λ}u} = sup{λ | x ∈ Y_{λ}} for x ∈ ℝ^{N} \ M.

Since this argument will be used again, it is worth emphasizing two points: Given any descending chain (X_{λ})_{λ∈ℝ}, the function u(x) = sup{λ | x ∈ X_{λ}} is completely determined by the chain (X_{λ})_{λ∈Λ}, where Λ is any dense subset of ℝ, in particular, any countable dense subset of ℝ. By choosing a countable dense set Λ that avoided the exceptional set N, it was possible to find a single set M of measure zero such that u(x) = sup{λ | x ∈ Y_{λ}} for x ∈ ℝ^{N} \M. Here is another point of view: Since X_{λ}u \ M = Y_{λ} \ M for λ ∈ Λ, the two chains (X_{λ}u \ M)_{λ∈Λ} and (Y_{λ} \ M)_{λ∈Λ} generate the same function on the domain ℝ^{N} \ M, and that function is u restricted to ℝ^{N} \ M.

Exercise 2.6. Construct a simple family of sets X_{λ} ⊂ ℝ^{2}, λ ∈ \overline{ℝ}, that have the structural properties given in Proposition 3.2 and such that the function u defined by (2.1) is +∞ and −∞ on subsets of ℝ^{2} with positive measure.

2.2 Contrast changes and level sets

We have already seen that the level sets of a function are globally preserved under a continuous increasing contrast change. Although we have not mentioned it, it is clear that the structural properties of the level sets are also preserved under such a transformation. We are now going to consider the more realistic situation where we relax the assumptions about the contrast change. Practical aspects of contrast changes are illustrated in Figures 3.4, 3.5, 3.6, and 3.7.

Definition 2.9. Any nondecreasing function g : ℝ → ℝ will be called a contrast change. We also consider its natural extension to \overline{ℝ} by defining

g(−∞) = inf_{x∈ℝ} g(x) and g(+∞) = sup_{x∈ℝ} g(x).

In case g is continuous and increasing, g has a well-defined inverse g^{−1}, but this is not true for a general contrast change. On the other hand, we need to have a pseudoinverse for g that is compatible with the structural properties of level sets.

Definition 2.10. The pseudoinverse of any contrast change g : \overline{ℝ} → \overline{ℝ} is defined by

g^{(−1)}(λ) = inf{r | g(r) ≥ λ}.

2.2. CONTRAST CHANGES AND LEVEL SETS 51

Figure 2.4: The histogram of the image Bird. For each i ∈ {0, 1, …, 255}, we display (above, right) the function h(i) = Card {x | u(x) = i}. The function below is the is given by g(i) = Card {x | u(x) ≤ i}, an integral of h. It provides an indication about the overall contrast of the image and about the contrast change imposed by the sensors. The inverse function g^{−1} can be used as a contrast change to create an image g^{−1}(u) with a flat histogram.

For consistency, throughout the book we use the following conventions regarding the infimum and supremum of the null set ∅ ⊂ \overline{ℝ}:

inf ∅ = +∞ and sup ∅ = −∞.

Exercise 2.7. Compute g^{(−1)} for the following functions.

(i) g(s) = max(0, s).

(ii) g(s) = 1 if s ≥ 0, and g(s) = 0 if s < 0.

Lemma 2.11. Suppose that g : \overline{ℝ} → \overline{ℝ} is a contrast change. Then for λ ∈ \overline{ℝ}, we have the following two properties:

If λ ≤ g(g^{(−1)}(λ)), then g(s) ≥ λ if and only if s ≥ g^{(−1)}(λ).  (2.6)
If λ > g(g^{(−1)}(λ)), then g(s) ≥ λ if and only if s > g^{(−1)}(λ).  (2.7)

52CHAPTER 2. CONTRAST-INVARIANT FUNCTIONS AND THEIR LEVEL SETS

Proof. First assume that λ ≤ g(g^{(−1)}(λ)). If g(s) ≥ λ, then s ≥ g^{(−1)}(λ) by the definition of g^{(−1)}(λ). Conversely, if s ≥ g^{(−1)}(λ), g(s) ≥ g(g^{(−1)}(λ)) ≥ λ and thus g(s) ≥ λ. This yields (2.6). Now assume that λ > g(g^{(−1)}(λ)). Then g(s) ≥ λ implies g(s) > g(g^{(−1)}(λ)). Thus, g being nondecreasing, s > g^{(−1)}(λ). Conversely, if s > g^{(−1)}(λ), then there exists by the definition of g^{(−1)} some r < s such that g(r) ≥ λ. Thus, g(s) ≥ g(r) ≥ λ.

The next result explains how the level sets of a function g(u) are related to the level sets of u, where g is a contrast change.

Theorem 2.12. Let u be a real-valued function defined on ℝ^{N} and let g be a contrast change. Then for λ ∈ \overline{ℝ}, the level set X_{λ}g(u) satisfies one of the following relations: Either

X_{λ}g(u) = {x | u(x) ≥ g^{(−1)}(λ)} = X_{g^{(−1)}(λ)}u,  (2.8)

or

X_{λ}g(u) = {x | u(x) > g^{(−1)}(λ)} = ⋃_{μ>g^{(−1)}(λ)} X_{μ}u.  (2.9)

Proof. The proof is read directly from Lemma 3.11, which says that for any λ ∈ \overline{ℝ}, either

g(s) ≥ λ if and only if s ≥ g^{(−1)}(λ),

or

g(s) ≥ λ if and only if s > g^{(−1)}(λ).

By letting s = u(x), this alternative implies the following one: Either

X_{λ}g(u) = {x | g(u(x)) ≥ λ} = {x | u(x) ≥ g^{(−1)}(λ)} = X_{g^{(−1)}(λ)}u,  (2.10)

or

X_{λ}g(u) = {x | g(u(x)) ≥ λ} = {x | u(x) > g^{(−1)}(λ)} = ⋃_{μ>g^{(−1)}(λ)} X_{μ}u. (2.11)

Note that the last equality in (3.11) comes from Proposition 3.5.

Exercise 2.8. Let u : ℝ^{N} → ℝ be measurable and let g be a contrast change. Show that for λ ∈ ℝ and ε > 0, there exists a μ ∈ ℝ such that X_{μ}u ⊂ X_{λ}g(u) and meas (X_{λ}g(u) \ X_{μ}u) < ε.

Both alternatives, (3.8) and (3.9), can occur in the same image. Here is a simple example: Let u(x) = x for x ∈ ℝ and let g(x) = x if x ≤ 0 and g(x) = x + 1 if x > 0. Then the level set X_{1}g(u) is not a level set of u, but X_{1}g(u) = (0, +∞) = {x | u(x) > 0}.

Exercise 2.9. Compute all of the level sets of u and g(u) in the example and compare them. Exercise 2.10. Give a simple example of a function u such that the functions g(u) have the same level sets as u for many contrast changes g.

The next result is the converse of Theorem 3.12. Roughly, it states that if the level sets of v are level sets of u, then v = g(u) for some contrast change g. In addition, Theorem 3.13 gives an algorithm for computing a contrast change that maps u onto v. We know from Exercise 3.10 that this contrast change is not necessarily unique.

2.2. CONTRAST CHANGES AND LEVEL SETS 53

Figure 2.5: Contrast changes and an equivalence class of images. The three images have exactly the same level sets and level lines, but their level sets are mapped onto three different gray-level scales. The graphs on the right are the graphs of the contrast changes u ↦ g(u) that have been applied to the initial gray levels. The first one is concave; it enhances the darker parts of the image. The second one is the identity; it leaves the image unaltered. The third one is convex; it enhances the brighter parts of the image. Software allows one to manipulate the contrast of an image to obtain the best visualization. From the image analysis viewpoint, image data should be considered as an equivalence class under all possible contrast changes.

Theorem 2.13. Suppose that u and v are two real-valued functions defined on ℝ^{N} such that for every λ ∈ \overline{ℝ} the following alternative holds: Either there exists a μ ∈ \overline{ℝ} such that

X_{λ}v = {x | u(x) ≥ μ} = X_{μ}u,  (2.12)

or there exists a μ ∈ \overline{ℝ} such that

X_{λ}v = {x | u(x) > μ} = ⋃_{ν>μ} X_{ν}u.  (2.13)

Then the contrast change g : \overline{ℝ} → \overline{ℝ} defined by

g(μ) = sup{λ | X_{μ}u ⊂ X_{λ}v}  (2.14)

maps u onto v, that is, v = g(u).

Proof. The first thing to note is that g is defined for every μ ∈ \overline{ℝ} and that ν > μ implies that g(ν) ≥ g(μ). This follows from Proposition 3.2(i). The proof that v = g(u) will be presented in two steps.

54CHAPTER 2. CONTRAST-INVARIANT FUNCTIONS AND THEIR LEVEL SETS

Figure 2.6: The two images (left) have the same set of level sets. The contrast change that maps the upper image onto the lower image is displayed on the right. It corresponds to one of the possible g functions whose existence is stated in Corollary 3.14. The function g may be locally constant on intervals where the histogram of the upper image is zero (see top, middle graph). Indeed, on such intervals, the level sets are invariant.

Step 1: proof that v(x) ≥ g(u(x)). Consider Λ = {λ | X_{μ}u ⊂ X_{λ}v}. We wish to show that g(μ) = sup Λ ∈ Λ. If sup Λ = −∞, then Λ = {−∞}, so we may assume that sup Λ > −∞. If λ_{0} ∈ Λ, then λ ∈ Λ for all λ < λ_{0} by Proposition 3.2(i). This means that λ ∈ Λ for all λ < sup Λ. But by Proposition 3.2(ii), X_{sup Λ}v = ⋂_{λ<sup Λ} X_{λ}v, and this implies by the definition of Λ that g(μ) = sup Λ ∈ Λ. Thus,

X_{μ}u ⊂ X_{g(μ)}v.  (2.15)

Given x ∈ ℝ^{N}, let μ = u(x) in (2.15). Then,

X_{u(x)}u ⊂ X_{g(u(x))}v.

Since x ∈ X_{u(x)}u, we conclude that x ∈ X_{g(u(x))}v = {y | v(y) ≥ g(u(x))}.

Step 2: proof that v(x) ≤ g(u(x)). Given x ∈ ℝ^{N}, we write the assumed alternative with λ = v(x) as follows: Either there exists a μ(x) ∈ \overline{ℝ} such that

X_{v(x)}v = {y | u(y) ≥ μ(x)} = X_{μ(x)}u,  (2.16)

2.3. SEMICONTINUOUS CONTRAST CHANGES 55

or there exists a μ(x) ∈ \overline{ℝ} such that

X_{v(x)}v = {y | u(y) > μ(x)} = ⋃_{ν>μ(x)} X_{ν}u.  (2.17)

First assume that (2.16) holds. Since x ∈ X_{v(x)}v, we know that x ∈ X_{μ(x)}u. Thus, u(x) ≥ μ(x), and X_{u(x)}u ⊂ X_{μ(x)}u = X_{v(x)}v. This last relation implies by the definition of g that v(x) ≤ g(u(x)).

Now assume that (2.17) holds. Since x ∈ X_{v(x)}v, we deduce from (2.17) the existence of some ν > μ(x) such that x ∈ X_{ν}u. Thus, u(x) ≥ ν, and we have the inclusions

X_{u(x)}u ⊂ X_{ν}u ⊂ X_{v(x)}v.

Again, by the definition of g, v(x) ≥ g(u(x)).

The contrast change g defined by (3.14) is, in general, not the only nondecreasing function that maps u into v (see Exercise 3.10). However, g does have a unique property: If one considers the family of functions H = {h | h is a contrast change, v(x) = h(u(x))}, then g(x) = sup_{h∈H} h(x).

Exercise 2.11. With the assumptions of Theorem 3.13, prove that f(x) = sup_{h∈H} h(x) is a contrast change and that f = g, where g is defined by (2.14).

We end this section with a statement that is easy to remember: If all level sets of v are level sets of u, then there exists a contrast change g such that v = g(u). Here is the precise statement:

Corollary 2.14. If u and v are two real-valued functions such that all the level sets of v are level sets of u, then the contrast change g : \overline{ℝ} → \overline{ℝ} defined by

g(μ) = sup{λ | X_{μ}u ⊂ X_{λ}v}  (2.18)

maps u onto v.

2.3 Semicontinuous contrast changes

We are going to consider contrast changes that are upper semicontinuous (u.s.c.). (There are parallel results for lower semicontinuous (l.s.c.) contrast changes.) This subject is relevant for image processing because threshold functions and quantization functions are typically either u.s.c. or l.s.c.

Definition 2.15. A nondecreasing function (a contrast change) g : \overline{ℝ} → \overline{ℝ} is upper semicontinuous if, for all r < +∞,

g(r) = lim_{{ρ→r \atop ρ>r}} g(ρ) = lim_{ρ→r^{+}} g(ρ).

Corollary 2.16. Assume that g is an upper semicontinuous contrast change. Then for every λ ∈ \overline{ℝ} for which g^{(−1)}(λ) < +∞,

g(s) ≥ λ if and only if s ≥ g^{(−1)}(λ).  (2.19)

56CHAPTER 2. CONTRAST-INVARIANT FUNCTIONS AND THEIR LEVEL SETS

Proof. Since g^{(−1)}(λ) < +∞, there is, by the definition of g^{(−1)}(λ), a sequence (r_{n})_{n∈ℕ} such that r_{n} ↓ g^{(−1)}(λ) and g(r_{n}) ≥ λ. Since g is u.s.c., g(r_{n}) ↓ g(g^{(−1)}(λ)) as n → +∞. Hence, g(g^{(−1)}(λ)) ≥ λ, and (2.19) follows from Lemma 3.11.

Exercise 2.12. Show that the result may be false if g^{(−1)}(λ) = +∞. Take g(s) = 0 for all s. Then g^{(−1)}(1) = +∞, and g(s) ≥ 1 is impossible.

Corollary 2.17. Let g be an u.s.c. contrast change and let u be a function from ℝ^{N} to ℝ. Then every level set X_{λ}g(u) of g(u) is a level set of u, and, in fact,

X_{λ}g(u) = X_{μ}u,  (2.20)

where μ = g^{(−1)}(λ).

Proof. Assume that g^{(−1)}(λ) < +∞. By Corollary 3.16, g(u(x)) ≥ λ if and only if u(x) ≥ g^{(−1)}(λ), which proves (2.20). If g^{(−1)}(λ) = +∞, (2.20) is also true, but both sets are empty. Indeed, if g^{(−1)}(λ) = +∞, this implies that either \{r | g(r) ≥ λ\} contains the single point +∞ or g(r) < λ for all r ∈ \overline{ℝ}. In either case, g(r) < λ for all r ∈ ℝ, so g(u(x)) < λ for all x ∈ ℝ^{N}. Hence, both X_{λ}g(u) and X_{+∞}u are empty.

Properties of g^{(−1)} in case g is continuous will be used in Chapter 6, so with this in mind, we are going to investigate these properties here. Assume that g is a continuous contrast change and write g(−∞) = λ^{−} and g(+∞) = λ^{+}. The case where g is a constant is left as an exercise, so assume that λ^{+} > λ^{−}. We wish to show that g(g^{(−1)}(λ)) = λ if λ ∈ (λ^{−}, λ^{+}), which is not necessarily true if g is only u.s.c. Our approach is to show that

g^{(−1)}(λ) = inf{r | g(r) = λ}.  (2.21)

The first thing to notice is that {r | g(r) = λ} is not empty, since g is continuous and nondecreasing on (λ^{−}, λ^{+}). Let inf{r | g(r) ≥ λ} = r_{λ}^{−} and let inf{r | g(r) = λ} = r_{λ}^{+}. Both infima are attained, r_{λ}^{−} ≤ r_{λ}^{+}, g(r_{λ}^{+}) = λ, and g(r_{λ}^{−}) ≥ λ. Since g is nondecreasing, g(r_{λ}^{−}) ≤ g(r_{λ}^{+}) = λ. Thus, g(r_{λ}^{−}) = λ. But r_{λ}^{+} was the smallest r such that g(r) = λ, so r_{λ}^{−} = r_{λ}^{+}. Since r_{λ}^{−} = g^{(−1)}(λ) by definition, g(g^{(−1)}(λ)) = λ.

Another property of g^{(−1)} is that it is strictly increasing: If λ < μ, λ, μ ∈ (λ^{−}, λ^{+}), then from what we have just shown, g(g^{(−1)}(λ)) < g(g^{(−1)}(μ)). Since g is nondecreasing, we must conclude that g^{(−1)}(λ) < g^{(−1)}(μ).

The final property we wish to highlight is that g^{(−1)} is lower semicontinuous. In fact, it is l.s.c. on all of \overline{ℝ}. We collect these results in the following lemma.

Lemma 2.18. Let g be a nonconstant continuous contrast change. Then, on the interval (λ^{−}, λ^{+}), g(g^{(−1)}(λ)) = λ and g^{(−1)} is strictly increasing. Furthermore, g^{(−1)} is lower semicontinuous on \overline{ℝ}.

Proof. It remains to be shown that g^{(−1)} is l.s.c. If λ ≤ λ^{−}, then g^{(−1)}(λ) = −∞, and it is clear that g^{(−1)} is l.c.s. for all of these values. The same is true for λ > λ^{+}. Now assume that λ ∈ (λ^{−}, λ^{+}), and let (μ_{n}) be an increasing sequence that tends to λ. Since g^{(−1)} and g are nondecreasing and since g is continuous, we see that g(g^{(−1)}(μ_{n})) = μ_{n} ↑ λ and g^{(−1)}(μ_{n}) ↑ s_{0}, for some 2.4. COMMENTS AND REFERENCES 57

Figure 2.7: The original image (top, left) has a strictly positive histogram (all gray levels between 0 and 255 are represented). Therefore, if any contrast change g that is not strictly increasing is applied, then some data will be lost. Every level set of the transformed image g(u) is a level set of the original image; however, the original image has more level sets than the transformed image.

value s_{0} ≤ g^{(−1)}(λ). Since g(s_{0}) = λ, we see from the definition of g^{(−1)}(λ) that g^{(−1)}(λ) ≤ s_{0}. Thus, s_{0} = g^{(−1)}(λ), which proves that g^{(−1)} is lower semicontinuous at λ. The only point that remains to be investigated in λ = λ^{+}, and we leave this as the next exercise.

Exercise 2.13. Prove that g^{(−1)} is l.s.c. at λ = λ^{+}. Hint: There are two cases: Either there is an r ∈ ℝ such that g(r) = λ^{+}, or g(r) < λ^{+} for all r ∈ ℝ. The techniques of Lemma 3.18 can be used in both cases.

2.4 Comments and references

Contrast invariance and level sets. It was Wertheimer who noticed that the actual local values of the gray levels in an image could not be relevant information for the human visual system [272]. Contrast invariance is one of the 58CHAPTER 2. CONTRAST-INVARIANT FUNCTIONS AND THEIR LEVEL SETS

fundamental model assumptions in mathematical morphology. The two basic books on this subject are Matheron [194] and Serra [242, 244]. See also the fundamental paper by Serra [243]. Ballester et al. defined an “image intersection” whose principle is to keep all pieces of bilevel sets common to two images [33]. (A bilevel set is of the form {x | λ ≤ u(x) ≤ μ}.) Monasse and Guichard recently developed a fast level set transform (FLST) to associate with every image the inclusion tree of connected components of level sets [199]. They show that the inclusion trees of connected upper and lower level sets can be fused into a single inclusion tree; among other applications, this tree can be used for image registration. See Monasse [198].

Contrast changes. The ability to vary the contrast (to apply a contrast change) of a digital image is a very useful tool for improving image visualization. Professional image processing software has this capability, and it is also found in popular software for manipulating digital images. For more about contrast changes that preserve level sets, see [62]. Many reference on contrast-invariant operators are given at the end of Chapter 6.

Chapter 3

Level Lines and Level Surfaces

This chapter contains the fundamentals of differential geometry that are used in Chapter 5 and elsewhere in the book. The development here and in Chapter 5 is a diversion from the continuous development in Chapters 3, 6, 7, 8, and 9. However, the material in this chapter and Chapter 5 is essential for the overall development of the theory, and we decided that the best place for it was between Chapters 3 and 6.

3.1 Curves: tangents, normals, and curvature

We summerize in this section the concepts and results about curves that are needed in this chapter and elsewhere in the book. The curves we considered will always be plane curves. A plane curve Γ is defined as the image of a continuous function z : [0, 1] → ℝ^{2}, z(t) = (x(t), y(t)). This definition is much too general and includes objects such as Peano’s space filling curve. Thus, we have two more-restrictive definitions: Γ is a simple arc or Jordan arc if the mapping z : [0, 1] → ℝ^{2} is one-to-one. We say that Γ is a simple closed curve or Jordan curve if the mapping restricted to (0, 1) is one-to-one and if z(0) = z(1). In other words, a Jordan arc is topologically equivalent to a closed interval, and a Jordan curve is topologically equivalent to a circle.

Recall that a Jordan arc (or curve) Γ is said to be rectifiable if the sums

∑_{i=1}^{n} |z(t_{i}) − z(t_{i−1})|

taken over all partitions 0 = t_{0} < t_{1} < ⋯ < t_{n} = 1 are bounded. If this is the case, then by definition the length of Γ is the supremum of these sums. Saying that Γ is rectifiable is equivalent to saying that the function z is of bounded variation. However, being rectifiable is an intrinsic property of Γ and does not depend on the particular representation z. If z is continuously differentiable on [0, 1], then it is of bounded variation and Γ is rectifiable. Furthermore, we have the familiar integral that measures the arc length of the segment of the curve

59 60 CHAPTER 3. LEVEL LINES AND LEVEL SURFACES

between z(s_{0}) and z(s_{1}):

L(z, s_{0}, s_{1}) = ∫_{s_{0}}^{s_{1}} |z′(t)| dt = ∫_{s_{0}}^{s_{1}} \sqrt{z′(t) · z′(t)} dt.  (3.1)

The curves we deal with will always be rectifiable, and we generally want them to have some degree of smoothness. Furthermore, we want the definition of “smoothness” to describe an intrinsic property of Γ rather than a property of some representation of Γ. If a function z representing Γ is C^{1}, then the function L in equation (3.1) has a derivative

\frac{dL}{ds} = |z′(s)|

that is continuous. Nevertheless, the curve itself may not conform to our idea of being smooth, which at a minimum requires a tangent at every point y ∈ Γ. For example, the motion of a point on the boundary of a unit disk as it rolls along the x-axis is described by z(θ) = (θ − sin θ, 1 − cos θ), which is a C^{∞} function. Nevertheless, the curve has cusps at all multiples of 2π. The problem is that z′(2kπ) = 0. It is this sort of phenomena that motivates the definition of smoothness for curves.

Definition 3.1. We say that a curve Γ admits an arc-length parameterization s ↦ z(s) if the function z is C^{1} and dL/ds = |z′(s)| = 1 for all s. We say that Γ is C^{m}, m ∈ ℕ, m ≥ 1, if the arc-length parameterization z is a C^{m} function.

An arc-length parameterization is also called a Euclidian parameterization. If a Jordan curve has an arc-length parameterization z, then the domain of definition of z on the real line must be an interval [a, b], where b − a is the length of Γ, which we denote by l(Γ). In this case, we will always take [0, l(Γ)] as the domain of definition of z. We identify [0, l(Γ)] algebraically with the circle group by adding elements of [0, l(Γ)] modulo l(Γ).

Definition 3.2. Assume that Γ is C^{2} and let s ↦ z(s) be an arc-length parameterization. The tangent vector τ is defined as τ = dz/ds. The curvature vector of the curve Γ is defined by κ = d^{2}z/ds^{2}. The normal vector n is defined by n = τ^{⊥}, where (x, y)^{⊥} = (−y, x).

Exercise 3.1. Let Γ by a Jordan arc parameterized by x : [0, 1] → Γ and by y : [0, 1] → Γ. Show that x = y(f) or x = y(1 − f), where f is a continuous, strictly increasing function that maps [0, 1] onto [0, 1]. Hint: x and y are one-to-one, and since [0, 1] is compact, they are homeomorphisms. Thus, y^{−1}(x) = f is a one-to-one continuous mapping of [0, 1] onto itself.

If a Jordan arc Γ is C^{1}, then the result of Exercise 1 implies that there are only two arc-length parameterizations of Γ: If x : [0, l(Γ)] → Γ is an arclength parameterization of Γ and y is another one, then either y(s) = x(s) or y(s) = x(l(Γ) − s), s ∈ [0, l(Γ)]. Here is a proof: ¿From Exercise 1 we know that a continuous, strictly increasing function f exists such that x = y(f) or x = y(l(Γ) − f). Assume that x = y(f). Then x(0) = y(0). By the definition of arc-length parameterization, the distance along Γ from x(0) to x(s) is s, and the distance from y(0) to y(f(s)) is f(s). If x = y(f), then we must have f(s) = s. A similar argument works if we assume that x = y(l(Γ) − f).

3.1. CURVES: TANGENTS, NORMALS, AND CURVATURE 61

The argument is slightly more involved in case Γ is a Jordan curve, since in this case we are dealing with the topology of the circle rather the closed interval. We state this as a proposition.

Proposition 3.3. Suppose that Γ is a C^{1} Jordan curve with arc-length parameterization x : [0, l(Γ)] → Γ. Then any other arc-length parameterization y : [0, l(Γ)] → Γ is of the form y(s) = x(s + σ) or y(s) = x(−s + σ) for some σ ∈ [0, l(Γ)].

Proof. Recall that addition is modulo l(Γ). The functions are one-to-one, so there is a unique σ in [0, l(Γ)) such that y(0) = x(σ). Define x_{σ} by x_{σ}(s) = x(s + σ). Take 0 < a < l(Γ))/2 and consider the two arcs γ_{x_{σ}} = x_{σ}([−a, a]) and γ_{y} = y([−a, a]). These are Jordan arcs in Γ. In fact, since x_{σ} and y are both arc-length parameterizations and since x_{σ}(0) = y(0), it follows that γ_{x_{σ}} = γ_{y}. This being the case, the arguments given above for Jordan arcs apply, and we conclude that y(s) = x_{σ}(s) or y(s) = x_{σ}(−s), for s ∈ [−a, a]. Since the functions are continuous and l(Γ)-periodic, and since we can take a as near to l(Γ))/2 as we wish, the result holds for all s ∈ ℝ.

Proposition 3.4. Let Γ be a C^{2} Jordan arc or curve, and let x and y by any two arc-length parameterizations of Γ.

(i) If x(s) = y(t), then x′(s) = ±y′(t).

(ii) The vector κ is independent of the choice of arc-length parameterizations and it is orthogonal to τ = x′.

Proof. In case Γ is a Jordan curve, y(s) = x(±s + σ) by Proposition 4.3, and (i) follows by differentiation. This is also geometrically obvious: x′(s) and y′(t) are unit vectors tangent to Γ at the same point. Thus, they either point in the same direction they point in opposite directions. In the case of a Jordan arc, we have y(s) = x(s) or y(s) = x(l(Γ) − s), and differentiation gives the result. Of course, in this case, we have one-sided derivatives at the end points.

Using any of the above representations and differentiating twice shows that x′′ = y′′. Since x′ ·x′ = 1, differentiating this expression shows that x′′ ·x′ = 0. Thus, x′′ and x′ are orthogonal and x′′ and x^{′⊥} are collinear.

The curvature vector has been defined in terms of the arc length. Curves, however, are often naturally defined in terms of other parameters. The next two exercises develop the differential relations between an arc-length parameterization and an other parameterization.

Exercise 3.2. Assume that Γ is a C^{2} Jordan arc or curve. Let s ↦ x(s) be an arc-length parameterization and let t ↦ y(t) be any other parameterization with the property that y′(t) ≠ 0. Since x and y are one-to-one, we can consider the function y^{−1}(x) = φ. Then x(s) = y(φ(s)), where φ(s) = t. The inverse function φ^{−1} is given by

s = φ^{−1}(t) = ∫_{t_{0}}^{t} \sqrt{y′(r) · y′(r)} dr,

so we know immediately that φ^{−1} is absolutely continuous with continuous derivative equal to \sqrt{y′(t) · y′(t)}. Thus, we also know that φ′(s) = |y′(φ(s))|^{−1}. Note that we made a choice above by taking \sqrt{y′(r) · y′(r)} to be positive. This is equivalent to assuming that x′(s) and y′(φ(s)) point in the same direction or that φ′(s) > 0.

62 CHAPTER 3. LEVEL LINES AND LEVEL SURFACES

(i) Show that κ(s) = x′′(s) = y′′(φ(s))[φ′(s)]^{2} + y′(φ(s))φ′′(s) and deduce that

φ′′(s) = − \frac{y′′(φ(s))φ′(s) · y′(φ(s))}{|y′(φ(s))|^{3}} = − \frac{y′′(φ(s)) · y′(φ(s))}{|y′(φ(s))|^{4}}.

(ii) Use the results of (i) to show that

κ(s) = x′′(s) = \frac{1}{|y′(t)|^{2}} [y′′(t) − (y′′(t) · \frac{y′(t)}{|y′(t)|}) \frac{y′(t)}{|y′(t)|}] ,  (3.2)

where φ(s) = t. Show that we get the same expression for the right-hand side of (3.2) with the assumption that φ′(s) < 0. This shows that the curvature vector κ does not depend on the choice of parameter.

(iii) Consider the scalar function κ(y) defined by κ(y)(s) = κ(s) · x′(s)^{⊥}. Use equation (3.2) to show that

κ(y)(t) = \frac{y′′(t) · [y′(t)]^{⊥}}{|y′(t)|^{3}}

Note that κ(y) is determined up to a sign that depends on the sign of φ′(s); however, |κ(y)| = |κ| is uniquely determined.

Exercise 3.3. Assume that Γ is a Jordan arc or curve that is represented by a C^{1} function t ↦ x(t) with the property that x′(t) ≠ 0. Prove that Γ is C^{1}.

Exercise 3.4.

(i) Consider the arc-length parameterization of the circle with radius r centered at the origin given by x(s) = (r cos(s/r), r sin(s/r)). Show that the length of the curvature vector is 1/r.

(ii) Compute the scalar curvature of the graph of y = (a/2)x^{2} at x = 0.

The generalization of this last exercise is essential for work that follows, particularly for studying the properties of curves that are presented to us as level lines of functions u : ℝ^{2} → ℝ. Assume we are given a C^{1} Jordan arc Γ and an arc-length parameterization c in a neighborhood of c(s_{0}) ∈ Γ. We assume, without loss of generality, that s_{0} = 0 and that c(0) = 0. Then we can establish a local coordinate system based on the two unit vectors c′(0) and c′(0)^{⊥}, where the x-axis is positive in the direction of c′(0). If we write c(s) = (x(s), y(s)) in this coordinate system, then

x(s) = c(s) · c′(0),
y(s) = c(s) · c′(0)^{⊥}.

Since dx/ds(s) = c′(s) · c′(0), dx/ds(0) = 1. Then the inverse function theorem implies the existence of a C^{1} function g and a δ > 0 such that s = g(x) for |x| < δ. This means that, for |x| < δ, Γ is represented locally by the graph of the C^{1} function f, where f(x) = y(g(x)) = c(g(x)) · c′(0)^{⊥}. To be slightly more precise, denote the graph of f for |x| < δ by Γ_{f}. Since g is one-to-one, Γ_{f} is a homeomorphic image of the open interval (−δ, δ) and Γ_{f} ⊂ Γ. If Γ is C^{2}, then f is C^{2} and f′′(0) = c′′(0) · c′(0)^{⊥}.

Conversely, given a C^{1} function f, we can consider the graph Γ_{f} of f in a neighborhood of the origin. Then Γ_{f} is represented by c, where c(x) = (x, f(x)).

3.2. THE LEVEL-LINE STRUCTURE (OR TOPOGRAPHIC MAP) OF AN IMAGE63

We may assume that f(0) = 0 and f′(0) = 0 (by a translation and rotation if necessary). Then arc-length along Γ is measured by

s(x) = ∫_{0}^{x} \sqrt{1 + [f′(t)]^{2}} dt,

and ds/dx = \sqrt{1 + [f′(x)]^{2}}. This time there is a C^{1} function h such that h(s) = x and h′(s) = (1 + [f′(h(s))]^{2})^{1/2}. Then Γ is represented by c̃(s) = (h(s), f(h(s))). Short computations show that |c̃′(s)| = 1. If f is C^{2}, then Γ is C^{2} and c̃′′(0) · c̃′(0)^{⊥} = f′′(0). Thus we have the following result: A C^{1} Jordan arc Γ can be represented locally as the graph of a C^{1} scalar function f. Conversely, the graph of a C^{1} function is a C^{1} Jordan arc.

This is all we need to say about Jordan arcs and curves for this and the next chapter. Further results about Jordan curves, which are needed for the axiomatic theory of shape scale spaces (Chapter 19), are given in Appendix A.

3.2 The level-line structure (or topographic map) of an image

We saw in Chapter 3 how an image can be represented by its level sets. The next step, with a view toward shape analysis, is the representation of an image in terms of its level lines. We rely heavily on the implicit function theorem to develop this representation. We begin with a two-dimensional version. The statement here is just a slight variation on the implicit function theorem quoted in section I.4 of the Introduction.

Theorem 3.5. Let u be a real-valued C^{1} function defined on an open set Ω ⊂ ℝ^{2}. Assume that Du(x_{0}) ≠ 0 at x_{0} = (x_{0}, y_{0}) in Ω. Let i denote the unit vector in the direction (u_{x}, u_{y}), let j denote the unit vector in the orthogonal direction (−u_{y}, u_{x}), and write x = x_{0} + xi + yj. Then there is a disk D(x_{0}, r) and a unique C^{1} function φ, φ : [−r, r] → ℝ, such that if x ∈ D(x_{0}, r), then

u(x, y) = 0 ⟺ x = φ(y).

If Du(x_{0}) ≠ 0, then Theorem 4.5 ensures that there is a disk D(x_{0}, r) such that the set {x | u(x) = u(x_{0}), x ∈ D(x_{0}, r)} is a C^{1} Jordan arc that is the graph of a unique C^{1} function. The following corollary is a global version of this local result.

Corollary 3.6. Assume that u ∈ C^{1}(ℝ^{2}) and let u^{−1}(λ) = {x | u(x) = λ} for λ ∈ ℝ. If u^{−1}(λ)(≠ ∅) is compact and if Du(x) ≠ 0 for all x ∈ u^{−1}(λ), then u^{−1}(λ) is a finite union of disjoint Jordan curves.

Proof. From Theorem 4.5 we know that for each point x ∈ u^{−1}(λ) there is a neighborhood D(x, r(x)) such that D(x, r(x)) ∩ u^{−1}(λ) is a C^{1} Jordan arc. Since u^{−1}(λ) is compact, there are a finite number of point x_{i}, i = 1, …, m, such that u^{−1}(λ) ⊂ ⋃_{i=1}^{m} D(x_{i}, r(x_{i})). This implies that u^{−1}(λ) is a finite union of Jordan arcs and hence has a finite number of connected components. Let Γ be one of these components. We wish to prove that Γ is a Jordan curve, and here are the ingredients we have to do this.

64 CHAPTER 3. LEVEL LINES AND LEVEL SURFACES

There is a finite number of C^{1} Jordan arcs Γ_{j}, j = 1, …, n, such that Γ = ⋃_{j=1}^{n} Γ_{j}. Each arc Γ_{j} can be represented by a C^{1} arc-length parameterization c_{j}. The functions c_{j} are homeomorphisms, and the domain of definition of c_{j} may be chosen to be any interval I_{j} = (a_{j}, b_{j}) such that b_{j} − a_{j} = l(Γ_{j}), the length of Γ_{j}. The connected component Γ is closed (hence compact), so the points c_{j}(a_{j}) and c_{j}(b_{j}) are also in Γ. The Γ_{j} were originally given as the graphs of C^{1} functions φ_{j} defined on a local coordinate system, so we are assured that c_{j}(a_{j}) ≠ c_{j}(b_{j}) and hence that c_{j} : [a_{j}, b_{j}] → \overline{Γ}_{j} is also a homeomorphism. By partially ordering the Γ_{j} and denoting the maximal elements by Γ_{m(j)}, we see that Γ = ⋃_{j=1}^{n} Γ_{m(j)}. Thus, we may assume that the Γ_{j} are all maximal and, hence, that Γ_{i} ⊄ Γ_{j} if i ≠ j. We are now ready to prove the result.

Note first that n ≥ 2: Γ is compact and Γ_{1} is not. We begin by assuming that n > 2. Since Γ is connected, Γ_{1} must intersect one of the other arcs, which, by renumbering if necessary, we call Γ_{2}. Let x be any point in Γ_{1} ∩ Γ_{2}. Then there are two (unique) points s ∈ I_{1} and t ∈ I_{2} such that c_{1}(s) = c_{2}(t). By Proposition 4.3, we may assume that c′_{1}(s) = c′_{2}(t). Since we can change the domain I_{2} as long as we retain the length of the interval b_{2} −a_{2}, we may assume that s = t and that the two intervals I_{1} = (a_{1}, b_{1}) and I_{2} = (a_{2}, b_{2}) and s are such that

a_{1} < a_{2} < s < b_{1} < b_{2}.

Now focus on the fact that x ∈ Γ_{1} ∩ Γ_{2}. The curves Γ_{1} and Γ_{2} that pass through x are the graphs of functions φ_{1} and φ_{2} that satisfy φ_{j}(y) = x, x = (x, y), and u(φ_{j}(y), y) = λ, j = 1, 2. Theorem 4.2 states that, in some neighborhood of x, these functions are unique. This means that the two curves Γ_{1} and Γ_{2} must agree is some neighborhood of x, and this in turn implies that c_{1}(t) = c_{2}(t) for t is some open interval containing s. Now let t_{0} = sup{t | a_{2} < t < b_{1}, c_{1}(t) = c_{2}(t)}. Then the argument just given shows that t_{0} = b_{1}: If t_{0} < b_{1}, then there is a neighborhood of c_{1}(t_{0}) = c_{2}(t_{0}) in which Γ_{1} and Γ_{2} agree, which implies that there are t > t_{0} where c_{1}(t) = c_{2}(t). The same argument proves that inf{t | a_{2} < t < b_{1}, c_{1}(t) = c_{2}(t)} = a_{2}. This proves that c_{1}(t) = c_{2}(t) for t ∈ [a_{2}, b_{1}] and allows us to define the function c_{12} on (a_{1}, b_{2}) by c_{12}(t) = c_{1}(t) for t ∈ (a_{1}, b_{1}) and c_{12}(t) = c_{2}(t) for t ∈ (a_{2}, b_{2}). The function c_{12} is one-to-one on (a_{1}, b_{1}) and it is one-to-one on (a_{2}, b_{2}). If c_{12} is one-to-one on all of (a_{1}, b_{2}), then Γ_{1} ∪ Γ_{2} = Γ_{12} is a Jordan arc and Γ = Γ_{12} ∪ ( ⋃_{j=3}^{n} Γ_{j}).

Here is what we have shown so far: If n > 2 and if c_{12} is one-to-one on (a_{1}, b_{2}), then Γ is the union of n − 1 Jordan arcs. Now repeat the process, beginning by partially ordering the remaining Jordan arcs and then retaining only those that are maximal. After at most n − 2 steps, there will be only two arcs remaining. (Note that repeating the process cannot reduce n to one, since the remaining arc would not be compact.)

We arrive at the following situation: Γ = Γ_{1} ∪ Γ_{2}, Γ_{1} ⊄ Γ_{2}, and Γ_{2} ⊄ Γ_{1}. The arcs are represented by arc-length parameterizations c_{1} : (a_{1}, b_{1}) → Γ_{1} and c_{2} : (a_{2}, b_{2}) → Γ_{2} with a_{1} < a_{2} < b_{1} < b_{2}. This time, the function c_{12}, which agrees with c_{1} on (a_{1}, b_{1}) and c_{2} on (a_{2}, b_{2}), cannot be one-to-one on all of (a_{1}, b_{2}) because Γ is compact. Consequently, there are points s and t such that c_{12}(s) = c_{12}(t) and a_{1} < s ≤ a_{2} < b_{1} ≤ t < b_{2}. An argument given above using the uniqueness of level curves passing through a point, which here is y = c_{12}(s) = c_{12}(t), proves that the arcs Γ_{1} and Γ_{2} must coincide on the sets [a_{1}, s] and [t, b_{2}]. We are going to remove the overlapping parts of the curves that 3.3. THE TOPOGRAPHIC MAP AS A COMPLETE IMAGE REPRESENTATION65

are not needed for representing Γ. Thus, define Γ̃_{1} = c_{1}[s, a_{2}] and Γ̃_{2} = c_{2}[a_{2}, t]. Then Γ = Γ̃_{1} ∪ Γ̃_{2}, c_{12} is one-to-one on (s, a_{2}) and one-to-one on (a_{2}, t), and c_{12}(s) = c_{12}(t). The function c_{12} is a C^{1} arc-length parameterization of Γ with |c′_{12}(x)| = 1 for all x ∈ [s, t]. It remains only to show that c_{12} is one-to-one on (s, t). If this is not the case, there are two points p and q such that p < q and c_{12}(p) = c_{12}(q). We have argued above, and this is the key to the proof, that in such a situation the curves passing through the point y = c_{12}(p) = c_{12}(q) must coincide “as far as possible,” which in this case means on the interval [p, q]. This, however, implies that there is a discontinuity in c′ at the point (p + q)/2, so we must conclude that c_{12} is one-to-one on (s, t), and, hence, that Γ is a Jordan curve.

Exercise 3.5. For those readers well versed in the theory of differentiable manifolds, the last proof my seem naive. For those who are not well versed, we suggest filling in the omitted details.

The next theorem is one of the results that we are going to quote rather than prove, as we have done with the theory of integration and the implicit function theorem, and as we will do with the Jordan curve theorem in the next section.

Theorem 3.7 (Sard’s theorem). Assume that u : ℝ^{N} → ℝ is C^{1} and has compact support. Then for almost every λ in the range of u, the set u^{−1}(λ) is nonsingular, which means that for all x ∈ u^{−1}(λ), Du(x) ≠ 0.

Recall that in section 1.2 we began with a function u defined on [0, 1]^{N} and extended it first by symmetry to C = [−1, 1]^{N} and then by periodicity to all of ℝ^{N}. The resulting function was called C-periodic and symmetric. We are concerned in the next result with functions defined on ℝ^{2} that are C^{1} and C-periodic, where C = [−1, 1]^{2}.

We assume that u : ℝ^{2} → ℝ is C-periodic and C^{1}. To apply Sard’s we need to have the support of u compact, but this is the case, since the “C-periodic topology” is just the topology of the two-dimensional torus 𝕋^{2}. Thus, u^{−1}(λ) is nonsingular for almost every λ in the range of u. If λ is such that u^{−1}(λ) is nonsingular, then we can interpret Corollary 4.6 in the 𝕋^{2}-context and conclude that u^{−1}(λ) is a finite set of simple closed curves on 𝕋^{2}. We state this result as a corollary.

Corollary 3.8. Let u : ℝ^{2} → ℝ be C^{1} and C-periodic. Then for almost every λ in the range of u, the set u^{−1}(λ) is the union of C^{1} curves. Viewed on 𝕋^{2}, u^{−1}(λ) is a finite set of disjoint simple closed curves.

When viewed in ℝ^{2}, there can be two kinds of C^{1} curves: Jordan curves and curves that extend to infinity. These sets are, of course, C-periodic.

3.3 The topographic map as a complete image representation

The sole purpose of this section is to convince the reader that the level lines of a function provide a faithful representation of the function, in the sense that it is possible, at least theoretically, to reconstruct the function from its level lines.

66 CHAPTER 3. LEVEL LINES AND LEVEL SURFACES

We will argue this for the case u is a C^{1} function on ℝ^{2} with compact support, and we will comment on other cases. The result that allows us to prove this special case is the Jordan curve theorem, which we quote.

Theorem 3.9 (Jordan curve theorem). Let Γ be a Jordan curve, that is, Γ is the image in ℝ^{2} of a continuous mapping z : [0, 1] → ℝ^{2} such that z is oneto-one on (0, 1) and z(0) = z(1). Then ℝ^{2} \ Γ consists of two open, connected components, one bounded and the other unbounded.

We denote the bounded component of ℝ^{2} \ Γ by γ and the unbounded component by g, and we refer to γ as the interior of Γ and to g as the exterior. Similarly, if we have a number of curves Γ_{i}, i = 1, …, n, then γ_{i} and g_{i} denote the bounded and unbounded components of ℝ^{2} \ Γ_{i} respectively.

Here is another result that is essential for our argument; it can be considered a generalization of the Jordan curve theorem.

Proposition 3.10. If Ω is an open, connected set in ℝ^{2} and if Γ ⊂ Ω is a Jordan curve, then Ω ∩ γ and Ω ∩ g are also open and connected.

In particular, this says that removing Jordan curves and their interiors does not disconnect an open, connected set. This will allow us to assign an orientation to the level lines of a C^{1} function.

Suppose that u is a C^{1} function on ℝ^{2} with compact support. We know from Sard’s theorem and Corollary 4.6 that for almost every λ in the range of u, u^{−1}(λ) is a finite union of disjoint Jordan curves, which we call Γ_{i}, i = 1, …, n. The plan is to show that the level sets of u, X_{λ}u, can be represented in terms of the Γ_{i}, γ_{i}, and g_{i} for a dense set of λ’s, that is, for the nonsingular λ’s in the range of u. It then follows from Lemma 3.8 that u(x) = sup{λ | x ∈ X_{λ}u} for almost every x ∈ ℝ^{2}.

Note that the γ_{i} are partially ordered by inclusion. This in turn partially orders the curves Γ_{i}: Γ_{i} ≺ Γ_{j} if and only if γ_{i} ⊂ γ_{j}. Thus we can think of the Γ_{i} as forming a finite number of disjoint trees. For convenience of discussion, we will say that the maximal elements of these trees are at level 1, that the curves just below are at level 2, and so on. Given a curve Γ_{i} at, say, level k, there may or may not be any elements inside Γ_{i} at level k + 1. If there are curves Γ_{j} inside Γ_{i} at level k + 1, then we know from Proposition 4.10 that γ_{i} \ ⋃_{j}(Γ_{j} ∪ γ_{j}) = γ_{i} \ ⋃_{j} \overline{γ}_{j} is open and connected. This implies that either u(x) > λ for x ∈ γ_{i} \ ⋃_{j} \overline{γ}_{j} or u(x) < λ for x ∈ γ_{i} \ ⋃_{j} \overline{γ}_{j}. (Recall that u^{−1}(λ) = ∪_{i}Γ_{i}.) In the first case we say that Γ_{i} has positive orientation and in the second case that Γ_{i} has negative orientation.

Observe that the maximal elements, the curves Γ_{m} at the top of the trees, all have the same orientation. This is because ℝ^{2} \ ⋃_{m} \overline{γ}_{m} is open and connected. Thus, either u(x) > λ on ℝ^{2} \ ⋃_{m} \overline{γ}_{m}, in which case the Γ_{m} have negative orientation, or u(x) < λ on ℝ^{2} \ ⋃_{m} \overline{γ}_{m}, and the Γ_{m} have positive orientation.

To represent X_{λ}u, we need to consider two cases: The maximal curves all have positive orientation, or the maximal curves all have negative orientation. In the first case, we have the following representation for X_{λ}u:

X_{λ}u = ⋃_{p} (\overline{γ}_{p} \ ( ∪_{j} γ_{j})),  (3.3)

3.3. THE TOPOGRAPHIC MAP AS A COMPLETE IMAGE REPRESENTATION67

Figure 3.1: Level lines as a complete representation of the shapes present in an image. All level lines of the image of a sea bird for levels that are multiples of 12 are displayed. Notice that we do not need a previous smoothing to visualize the shape structures in an image: It is sufficient to quantize the displayed levels.

where the first union is taken over all p such that Γ_{p} has positive orientation and the second union is taken over all j such that γ_{j} ⊂ γ_{p} and level(γ_{j}) = level(γ_{p}) + 1.

IIf the maximal curves all have negative orientation, then the set ⋂_{m} \overline{g}_{m} must be added to the right-hand side of (3.3):

X_{λ}u = ( ⋃_{p} (\overline{γ}_{p} \ ( ∪_{j} γ_{j}))) ∪ ( ⋂_{m} \overline{g}_{m}),  (3.4)

where the union indexed by m is taken over all maximal curves.

We hope that this discussion has persuaded the reader of the truth of the following result.

Proposition 3.11. Let u be a C^{1} function with compact support defined on ℝ^{2}. Then u can be recovered (a.e.) from its level lines.

In fact, we have shown that u can be recovered from the level lines u^{−1}(λ) for which λ is nonsingular in the sense of Sard’s theorem. We define the topographic map of u to be the mapping λ : ℝ ↦ u^{−1}(λ), where λ is nonsingular. Thus another way to state Proposition 4.11 is that u can be recovered from its topological map. Note also that Proposition 4.11 holds under the weaker assumption that u(x) tends to some limit as |x| → +∞.

In practice, we are faced with a digital image defined on a rectangular grid, and as we have argued elsewhere, these digital images can be interpolated several ways to produce a good rendition of the “original” image. The properties of the level lines of the interpolated function clearly depend on the kind of interpolation. To avoid discussing the different cases and topological discussions that take us too far afield, we are going to make rather specific assumptions about the level lines in the next proposition.

Proposition 3.12. Let u be a real-valued function defined on a closed rectangle R ∈ ℝ^{2}. Assume that, for a dense set of λ’s in the range of u, the set 68 CHAPTER 3. LEVEL LINES AND LEVEL SURFACES

u^{−1}(λ) consists of a finite number of disjoint Jordan arcs and curves. Assume further that the Jordan curves are in the interior of R and that the Jordan arcs terminate on the boundary of R. Then u can be recovered from its topographic map.

Proof. We will, in fact, give only an outline of a proof: Many details will be omitted.

Assume that there are k, k ≥ 0, Jordan arcs that terminate on the boundary of R. It follows from the Jordan curve theorem that the these arcs partition the interior of R into k + 1 simply connected open sets. Denote these sets by R_{i}, i = 1, …, k + 1.

We treat each R_{i} separately. Here, a given R_{i} plays essentially the same role that ℝ^{2} played in the argument given for the case u had compact support. Let Γ_{i,j} denote the Jordan curves in R_{i}. Then the set R_{i} \ ⋃_{m}(Γ_{i,m} ∪ γ_{i,m}), where the Γ_{i,m} are the maximal elements in R_{i}, is open and connected. Thus, u(x) > λ for all x ∈ R_{i} \ ⋃_{m}(Γ_{i,m} ∪ γ_{i,m}), or u(x) < λ for all x ∈ R_{i} \ ⋃_{m}(Γ_{i,m} ∪ γ_{i,m}). Continuing this way, with arguments parallel to those given above, we arrive at expressions like (3.3) and (3.4) for X_{λ}u_{i}, where u_{i} is the restriction of u to R_{i}. Finally, X_{λ}u = ⋃_{i} X_{λ}u_{i} ∪ ⋃_{j} Λ_{j}, where the Λ_{j} are the k Jordan arcs that terminate on the boundary of R. Of course, the sets g_{m} in (3.4) must be replaced with g_{i,m} ∩ R_{i}.

3.4 Generalized level lines and topographic map

We have made assumptions about the function u so the level lines of u are, for a dense set of λ, disjoint Jordan curves and arcs. This is nice for illustrating the ideas concerning the reconstruction of u from its level lines, but it is too restrictive for practical digital processing. We wish to move to the situation where the level sets of u are “smooth enough” so their boundaries are the unions of Jordan curves. Here is an example: Suppose Γ is a Jordan curve and consider the function u(x) = 1 if x is inside Γ and u(x) = 0 otherwise. The implicit function theorem does not apply, and the set {x | u(x) = 1} is not a level line. This leads to the following definition, which covers this particular case and others.

Definition 3.13. Let u be upper semicontinuous, which is equivalent to saying that its level sets are closed. Let X be a level set of u and consider the connected components of X^{c}, the complement of X. Assume that the boundary of each of these components is a finite union of Jordan curves that do not cross and that meet in at most a finite number of points. Under these conditions, we say that these Jordan curves are level lines of u.

This definition may seem rather restrictive and it is. It has, however, the advantage of applying to one of the main situation in image processing and one we will be considering later in the book: upper semicontinuous piecewise constant images, in particular, images that are constant on the pixels. In this case, each level set is a union of closed squares on a grid, and it is easy to check that our definition applies and uniquely defines a set of Jordan curves that we 3.4. GENERALIZED LEVEL LINES AND TOPOGRAPHIC MAP 69

Figure 3.2: Level lines as representatives of the shapes present in an image. Left: noisy binary image with two apparent shapes; right: the two longest level lines.

Figure 3.3: Intrinsic coordinates. Note that φ′′(0) > 0, so b < 0.

call the level lines of the image. Conversely, these level lines uniquely define the associated level sets, and these level sets can be used to reconstruct the image.

As an example, consider a checkerboard image u, where the pixels have alternate values 255 (white) and 0 (black). If we decide that u is u.s.c., then the white pixels include their boundaries and are connected. The black pixels are then disconnected. The level lines of u are simply all the boundaries of the black pixels.

70 CHAPTER 3. LEVEL LINES AND LEVEL SURFACES

3.5 Curvature of the level lines

We continue to work in ℝ^{2}. Consider a real-valued function u that is twice continuously differentiable in a neighborhood of x_{0} ∈ ℝ^{2}. Without loss of generality, we let x_{0} = 0 and assume that u(0) = 0. (To simplify the notation, we will often write Du rather than Du(0), and so on.) If the gradient Du = (u_{x}, u_{y}) ≠ 0, then we establish a local coordinate system by letting i = Du/|Du| and j = Du^{⊥}/|Du|, where Du^{⊥} = (−u_{y}, u_{x}). Thus, for a point x near 0, we write x = xi + yj. (See Figure 3.3.)

Since u is C^{2}, we can use Taylor’s formula to express u in this coordinate system in a neighborhood of 0:

u(x) = px + ax^{2} + by^{2} + cxy + O(|x|^{3}),  (3.5)

where p = |Du(0)| > 0 and

\begin{aligned}a & = \frac{1}{2} \frac{∂^{2}u}{∂x^{2}}(0) = & \frac{1}{2}D^{2}u(\frac{Du}{|Du|}, \frac{Du}{|Du|}), \\ b & = \frac{1}{2} \frac{∂^{2}u}{∂y^{2}}(0) = & \frac{1}{2}D^{2}u(\frac{Du^{⊥}}{|Du|} , \frac{Du^{⊥}}{|Du|} ), \\ c & = \frac{∂^{2}u}{∂x∂y} (0) = & D^{2}u(\frac{Du^{⊥}}{|Du|} , \frac{Du}{|Du|}). \end{aligned}  (3.6)

The implicit function theorem (Theorem 4.5) ensures that in a neighborhood of 0, the set {x | u(x) = 0} is a C^{2} graph whose equation can be written x = φ(y), where φ is a C^{2} function in a neighborhood N of y = 0. In this neighborhood, we have u(φ(y), y) = 0. Differentiating this shows that u_{x}φ′ + u_{y} = 0 for y ∈ N. Since Du(0) = u_{x}(0) in the coordinate system defined in terms of Du(0)/|Du(0)|, we conclude that u_{y}(0) = 0 and, hence, that φ′(0) = 0. A second differentiation show that

(u_{xx}φ′ + u_{xy})φ′ + u_{x}φ′′ + u_{yx}φ′ + u_{yy} = 0.

Since φ′(0) = 0, we have φ′′(0) = −u_{yy}(0)/u_{x}(0). Then using the notation of (3.6), we see that

φ(y) = −\frac{b}{p}y^{2} + o(y^{2}).  (3.7)

Equation (3.7) is the representation of the level line {x | u(x) = u(0)} in the intrinsic coordinates at 0. It is easy to show that |2b/p| = 1/R is the inverse of the radius of the circle that has “three points of contact” at y = 0 with the curve {φ(y), y}. This circle is called the osculating circle.

We are now going to do another simple computation to determine the curvature vector of the Jordan arc c defined by c(y) = (φ(y), y) near y = 0. (From here on, we will generally denote the curvature of a curve c by Curv(c) and the value of this function at a point y by Curv(c)(y).) Since c′(y) = (φ′(y), 1) and c′′(y) = (φ′′(y), 0), at y = 0, we have c′(0) = (0, 1) and c′′(0) = (φ′′(0), 0), so c′′(0) · c′(0) = 0. Using this and equation (3.2) we see that Curv(c)(0) = (φ′′(0), 0). We now use (3.7) and (3.6) to write the last expression as

Curv(c)(0) = −\frac{1}{|Du|}D^{2}u(\frac{Du^{⊥}}{|Du|} , \frac{Du^{⊥}}{|Du|} )\frac{Du}{|Du|}(0),  (3.8)

3.5. CURVATURE OF THE LEVEL LINES 71

where all of the expressions are evaluated at y = 0. This tells us that the vectors Curv(c)(0) and Du(0) are collinear. Equation (3.8) also leads to the following definition and lemma.

Definition 3.14. Let u be a real-valued function that is C^{2} in a neighborhood of a point x ∈ ℝ^{2} and assume that Du(x) ≠ 0. The curvature of u at x, denoted by curv(u)(x), is the real number defined by

\frac{1}{|Du|^{3}} D^{2}u(Du^{⊥}, Du^{⊥})(x) = \frac{u_{xx}u_{y}^{2} − 2u_{xy}u_{x}u_{y} + u_{yy}u_{x}^{2}}{(u_{x}^{2} + u_{y}^{2})^{3/2}} (x).  (3.9)

Lemma 3.15. Assume that u : ℝ^{2} → ℝ is C^{2} in a neighborhood of a point x_{0} and assume that Du(x_{0}) ≠ 0. Let N = N(x_{0}) be a neighborhood of x_{0} in which the level set of u {x | u(x) = u(x_{0})} is a simple C^{2} arc, which we denote by z. Then

Curv(z)(x) = −curv(u)(x)\frac{Du}{|Du|}(x),  (3.10)

where the relation holds for x ∈ N ∩ {x | u(x) = u(x_{0})}.

Proof. We essentially proved the lemma when we derived equation (3.8) for x_{0} = 0. We need only remark that, given the hypotheses of the lemma, there is a neighborhood N of x_{0} such that Du(x) ≠ 0 for x ∈ N and such that {x | u(x) = u(x_{0})} is a simple C^{2} arc z for x ∈ N. Then the argument we made to derive (3.8) holds for any point x ∈ N ∩ {x | u(x) = u(x_{0})}.

Exercise 3.6. Use equation (3.9) to show that

curv(u) = div( \frac{Du}{|Du|} ).  (3.11)

Use this equation to show that curv(g(u)) = curv(u) if g is any C^{2} function g : ℝ → ℝ such that g′(x) > 0 for all x ∈ ℝ. What happens if g′(x) < 0 for all x ∈ ℝ. Show that curv(U) = curv(u), where U(s, t) = u(x, y) and x = s cos θ−t sin θ, y = s sin θ+t cos θ.

Exercise 3.7. The kinds of techniques used in this exercise are important for work in later chapters. The exercise demonstrates that it is possible to bracket a C^{2} function locally with two functions that are radial and either increasing or decreasing. We say that a function f is radial and increasing if there exists an increasing function g : ℝ^{+} → ℝ such that f(x) = g(|x_{c} − x|^{2}), x_{c} ∈ ℝ^{2}. We say that f is radial and decreasing if g is decreasing. Let u : ℝ^{2} → ℝ be C^{2} and assume that Du(x_{0}) ≠ 0. We wish to show that for every ε > 0 there exist two C^{2} radial and functions f_{ε}^{−} and f_{ε}^{+} (increasing or decreasing, depending on the situation) that satisfy the following four conditions:

f_{ε}^{−}(x_{0}) = u(x_{0}) = f_{ε}^{+}(x_{0}),  (3.12)
Df_{ε}^{−}(x_{0}) = Du(x_{0}) = Df_{ε}^{+}(x_{0}),  (3.13)
curv(f_{ε}^{−})(x_{0}) + \frac{2ε}{p} = curv(u)(x_{0}) = curvf_{ε}^{+}(x_{0}) − \frac{2ε}{p} ,  (3.14)
f_{ε}^{−}(x) + o(|x_{0} − x|^{2}) ≤ u(x) ≤ f_{ε}^{+}(x) + o(|x_{0} − x|^{2}).  (3.15)

1. Without loss of generality, take x_{0} = (0, 0), u(0, 0) = 0, and Du(x_{0}) = (p, 0), p > 0. Then we have the Taylor expansion

u(x) = px + ax^{2} + by^{2} + cxy + o(x^{2} + y^{2}),

72 CHAPTER 3. LEVEL LINES AND LEVEL SURFACES

where a, b, and c are given in (3.6). Show that for every ε > 0,

px+(− \frac{c^{2}}{ε} +a)x^{2}+(b−ε)y^{2}+o(x^{2}+y^{2}) ≤ u(x, y) ≤ px+( \frac{c^{2}}{ε} +a)x^{2}+(b+ε)y^{2}+o(x^{2}+y^{2}).

2. Let f be a radial function defined by f(x, y) = g((x − x_{c})^{2} + y^{2}), where g : ℝ^{+} → ℝ is C^{2} and either increasing or decreasing. Show by expanding f at (0, 0) that

f(x, y) = g(x_{c}^{2}) − 2x_{c}g′(x_{c}^{2})x + (2x_{c}^{2}g′′(x_{c}^{2}) + g′(x_{c}^{2}))x^{2} + g′(x_{c}^{2})y^{2} + o(x^{2} + y^{2}).

3. The idea is to construct f_{ε}^{+} and f_{ε}^{−} by matching the coefficients of the expansion of f with the coefficients of the functions px + (±(c^{2}/ε) + a)x^{2} + (b ± ε)y^{2}. There are three cases to consider: b < 0, b = 0, and b > 0. Show that in each case it is possible to find values of x_{c} and functions g so the functions f_{ε}^{+} and f_{ε}^{−} satisfy the four condition. Note that both x_{c} and g depend on ε. Discuss the geometry for each case.

Before leaving this section, we wish to emphasize geometric aspects of the functions we have introduced. Perhaps the most important fact is that the curvature of a C^{2} Jordan arc Γ is an intrinsic property of Γ; it does not depend on the parameterization. If x is a point on Γ, then the curvature vector Curv(Γ)(x) points toward the center of the osculating circle, which is the circle that has three points of contact with Γ at x. Furthermore, 1/|Curv(Γ)(x)| is the radius of this circle, so when |Curv(Γ)(x)| is large, the osculating circle is small, and the curve is ”turning a sharp corner.”

If Du(x) ≠ 0, then the vector Du(x) points in the direction of greatest increase, or steepest ascent, of u at x: Following the gradient leads “up hill.” The function curv(u) does not have such a clear geometric interpretation, and it is perhaps best thought of in terms of equation (3.10): curv(u)(x) is the coefficient of −Du(x)/|Du(x)| that yields the curvature vector Curv(z)(x), where z is the level curve through the point x at level u(x). We cannot over emphasize the importance of the two operators curv and Curv for the theories that follow. In addition to (3.10), a further relation between these operators is shown in Proposition 5.8, and it is this result that connects function smoothing with curve smoothing.

3.6 The principal curvatures of a level surface

We saw in Exercise 4.5 that curv(u) was contrast invariant. This idea will be generalized to ℝ^{N} by introducing other differential operators that are contrast invariant. These operators will be functions of the principal curvatures of the level surfaces of u. For z ∈ ℝ^{N}, z^{⊥} will denote the hyperplane {y | z · y = 0} that is orthogonal to z. (There should be no confusion with this notation and the same notation for z ∈ ℝ^{2}. In ℝ^{2}, z^{⊥} is a vector orthogonal to z, and the corresponding “hyperplane” is the line {tz^{⊥} | t ∈ ℝ}.)

Proposition 3.16. Assume that u : ℝ^{N} → ℝ is C^{2} in a neighborhood of a point x_{0} and assume that Du(x_{0}) ≠ 0. Let g : ℝ → ℝ be a C^{2} contrast change such that g′(s) > 0 for all s ∈ ℝ. Then Dg(u(x_{0})) = g′(u(x_{0}))Du(x_{0}), and D̃^{2}g(u(x_{0})) = g′(u(x_{0}))D̃^{2}u(x_{0}), where D̃^{2}u(x_{0}) denotes the restriction of the symmetric matrix D^{2}u(x_{0}) to the hyperplane Du(x_{0})^{⊥}. This means, in particular, that (1/|Du(x_{0})|)D̃^{2}u(x_{0}) is invariant under such a contrast change.

3.6. THE PRINCIPAL CURVATURES OF A LEVEL SURFACE 73

Proof. To simplify the notation, we will suppress the argument x_{0}; thus, we write Du for Du(x_{0}), and so on. We use the notation y ⊗ y, y ∈ ℝ^{N}, to denote the linear mapping y ⊗ y : ℝ^{N} → ℝ^{N} defined by (y ⊗ y)(x) = (x · y)y. The range of y ⊗ y is the one-dimensional space ℝy.

An application of the chain rule shows that Dg(u) = g′(u)Du. This implies that Du^{⊥} = Dg(u)^{⊥}. (Recall that g′(s) > 0 for all s ∈ ℝ.) A second differentiation shows that

D^{2}g(u) = g′′(u)Du ⊗ Du + g′(u)D^{2}u.

If y ∈ Du^{⊥}, then (Du⊗Du)(y) = 0, and we see that D^{2}g(u)(y, y) = g′(u)D^{2}u(y, y). This means that D^{2}g(u) = g′(u)D^{2}u on Du^{⊥} = Dg(u)^{⊥}, which proves the result.

Exercise 3.8. By computing explicitly the terms ∂g(u)/∂x_{i}, verify that Dg(u) = g′(u)Du. Similarly, verify that D^{2}(g(u)) = g′′(u)Du ⊗ Du + g′(u)D^{2}u by computing the second-order terms ∂^{2}g(u)/∂x_{i}∂x_{j}.

We are now going to define locally the level surface of a smooth function u, and for this we quote one more version of the implicit function theorem.

Theorem 3.17 (Implicit function theorem). Assume that u : ℝ^{N} → ℝ is C^{m} in the neighborhood of x_{0} and assume that Du(x_{0}) ≠ 0. Write x = x_{0} +y +zi, where i = Du(x_{0})/|Du(x_{0})| and y ∈ Du(x_{0})^{⊥}. Then there exists a ball B(x_{0}, ρ) and a unique real-valued C^{m} function φ defined on B(x_{0}, ρ) ∩ {x | x = x_{0} + y, i · y = 0} such that for every x ∈ B(x_{0}, ρ)

u(x) = u(x_{0}) ⟺ φ(y) = z.

In other words, the equation φ(y) = z describes the set {x | u(x) = u(x_{0})} near x_{0} as the graph of a C^{m} function φ. Thus, locally we have a surface passing through x_{0} that we call the level surface of u around x_{0}.

We are going to use Proposition 4.16 and Theorem 4.16, first, to give a simple intrinsic representation for the level surface of a function u around a point x_{0} and, second, to relate the eigenvalues introduced in Proposition 4.16 to the curvatures of lines drawn on the level surface of u.

Proposition 3.18. Assume that u : ℝ^{N} → ℝ is C^{2} in a neighborhood of x_{0} ∈ ℝ^{N} and that p = Du(x_{0}) ≠ 0. Denote the eigenvalues of the restriction of the symmetric matrix D^{2}u(x_{0}) to the hyperplane Du(x_{0})^{⊥} by κ_{1}, …, κ_{N−1}. Let i_{N} = Du(x_{0})/|Du(x_{0})| and select i_{1}, …, i_{N−1} so they form an orthonormal basis of eigenvectors of the restriction of D^{2}u(x_{0}) to Du(x_{0})^{⊥}. Write x = x_{0} + z, where z = x_{1}i_{1} + ⋯ + x_{N}i_{N} = y + x_{N}i_{N}. Then if |z| is sufficiently small, the function φ(y) = x_{N} that solves that equation u(y, φ(y)) = u(x_{0}) can be expressed locally as

x_{N} = \frac{−1}{2p} ∑_{i=1}^{N−1} κ_{i}x_{i}^{2} + o(|y|^{2}).

Proof. Assume, without loss of generality, that x_{0} = 0 and that u(0) = 0. Using the notation of Theorem 4.17, we know that, for x ∈ B(0, ρ), u(y, x_{N}) = 74 CHAPTER 3. LEVEL LINES AND LEVEL SURFACES

0 if and only if φ(y) = x_{N}, and that φ is C^{2} in B(0, ρ). Furthermore, by differentiating the expression u(y, φ(y)) = 0, we see that u_{x_{i}} + u_{x_{N}} φ_{x_{i}} = 0, i = 1, …, N − 1 for |x| < ρ. In particular, u_{x_{i}}(0) + u_{x_{N}} (0)φ_{x_{i}}(0) = 0. In the local coordinate system we have chosen, |Du(0)| = |u_{x_{N}} (0)|, and since Du(0) ≠ 0, we conclude that u_{x_{i}}(0) = 0 for i = 1, …, N − 1 and hence that φ_{x_{i}}(0) = 0 for i = 1, …, N − 1. This means that the local expansion of φ has the form

φ(y) = \frac{1}{2}D^{2}φ(0)(y, y) + o(|y|^{2}).

Now differentiate the relation u_{x_{i}} + u_{x_{N}} φ_{x_{i}} = 0 again to obtain

u_{x_{i}x_{j}} + u_{x_{i}x_{N}} φ_{x_{j}} + (u_{x_{N} x_{j}} + u_{x_{N} x_{N}} φ_{x_{j}})φ_{x_{i}} + u_{x_{N}} φ_{x_{i}x_{j}} = 0.

Since we have just shown that φ_{x_{i}}(0) = 0 for i = 1, …, N − 1, we see from this last expression that D^{2}u(0) + pD^{2}φ(0) = 0, where p = u_{x_{N}} (0). Thus we have

x_{N} = \frac{−1}{2p}D^{2}u(0)(y, y) + o(|y|^{2}).  (3.16)

Recall that y ∈ Du(0)^{⊥} and that y = x_{1}i_{1} + ⋯ + x_{N−1}i_{N−1}, where the i_{i} are an orthonormal basis of eigenvectors of D^{2}(0) restricted to Du(0)^{⊥}. Thus,

x_{N} = \frac{−1}{2p} ∑_{i=1}^{N−1} κ_{i}x_{i}^{2} + o(|y|^{2}),

which is what we wished to prove.

This formula reads

x_{2} = \frac{−1}{2p}κ_{1}x_{1}^{2} + o(|x_{1}|^{2})

if N = 2, which is just equation (3.7) with different notation. Thus, κ_{1} = |Du|curv(u), confirming that κ_{1} = ∂^{2}u/∂x_{1}^{2}.

We are now going to use our two-dimensional analysis to give a further interpretation of the eigenvalues κ_{i} for N > 2. We begin by considering the curve Γ_{ν} defined by the two equations x = x_{0} + tν + x_{N}i_{N} and φ(tν) = x_{N}, where ν is a unit vector in Du(x_{0})^{⊥}. Their solution in the local coordinates is φ(tν) = x_{N}, whenever t ∈ ℝ is small. From the proof of Proposition 4.18, we know that

φ(tν) = \frac{−1}{2|Du(x_{0})|}D^{2}u(x_{0})(ν, ν)t^{2} + o(t^{2}),

and a computation like the one we did to obtain equation (3.8) leads to the relation

Curv(Γ_{ν})(x_{0}) = \frac{−1}{|Du(x_{0})|}D^{2}u(x_{0})(ν, ν)\frac{Du(x_{0})}{|Du(x_{0})|}.

By defining κ_{ν} = |Du(x_{0})|^{−1}D^{2}u(x_{0})(ν, ν), we have

Curv(Γ_{ν})(x_{0}) = −κ_{ν} \frac{Du(x_{0})}{|Du(x_{0})|},

which has the same form as equation (3.10). This suggest calling κ_{ν} a curvature, and we will do this but only in case ν is one of the eigenvectors of D^{2}u(x_{0}) restricted to Du^{⊥}.

3.6. THE PRINCIPAL CURVATURES OF A LEVEL SURFACE 75

Definition 3.19. The principal curvatures of the level surface of u at x_{0} are by definition the numbers κ_{ν} obtained by letting ν = i_{j}, j = 1, …, N − 1, where the unit vectors i_{j} are the eigenvectors of D^{2}u(x_{0}) restricted to Du(x_{0})^{⊥}.

Thus, the principal curvatures of u at x_{0} are the real numbers

κ_{i_{j}} = \frac{κ_{j}}{|Du(x_{0})|},

where the κ_{j} are the eigenvalues of D^{2}u(x_{0}) restricted to Du(x_{0})^{⊥}. It follows from Proposition 4.16 that the principal curvatures are invariant under a C^{2} contrast change g such that g′(s) > 0 for all s ∈ ℝ.

Definition 3.20. The mean curvature of a C^{2} function u : ℝ^{N} → ℝ at x_{0} ∈ ℝ^{N} is the sum of the principal curvatures at x_{0}. It is denoted by curv(u)(x_{0}).

Note that this definition agrees with the Definition 4.14 when N = 2. The next result provides another representation for curv(u).

Proposition 3.21. The mean curvature of u is given by

curv(u) = div(\frac{Du}{|Du|}).

Proof. Represent the matrix D^{2}u in the coordinate system i_{j}, j = 1, …, N −1, and i_{N} = Du(x_{0})/|Du(x_{0})|, where the i_{j}, j = 1, …, N −1, form a complete set of eigenvectors of the linear mapping D^{2}u(x_{0}) restricted to Du^{⊥}(x_{0}). Then in this coordinates system, D^{2}u(x_{0}) has the following form (illustrated for N = 5):

D^{2}u(x_{0}) = [\begin{matrix}u_{11} & 0 & 0 & 0 & u_{15} \\ 0 & u_{22} & 0 & 0 & u_{25} \\ 0 & 0 & u_{33} & 0 & u_{35} \\ 0 & 0 & 0 & u_{44} & u_{45} \\ u_{51} & u_{52} & u_{53} & u_{54} & u_{55} \end{matrix}] ,

where u_{jk} = u_{x_{j}x_{k}}(x_{0}), and u_{jj} = κ_{j} is the eigenvalue associated with i_{j}. Thus, by definition, we see that

curv(u) = \frac{Δu}{|Du|} − \frac{1}{|Du|}D^{2}u(\frac{Du}{|Du|}, \frac{Du}{|Du|}).

We also have

div(\frac{Du}{|Du|}) = ∑_{j=1}^{N} \frac{∂}{∂x_{j}} (\frac{u_{x_{j}}}{|Du|})

= \frac{1}{|Du|} ∑_{j=1}^{N} u_{x_{j}x_{j}} − \frac{1}{|Du|^{3}} ∑_{j,k=1}^{N} u_{x_{j}x_{k}}u_{x_{j}}u_{x_{k}}

= \frac{Δu}{|Du|} − \frac{1}{|Du|}D^{2}u(\frac{Du}{|Du|}, \frac{Du}{|Du|}).

With this representation, it is clear that the mean curvature has the same invariant properties as the curvature of a C^{2} function defined on ℝ^{2}. (See Exercise 4.5.)

76 CHAPTER 3. LEVEL LINES AND LEVEL SURFACES

3.7 Comments and references

Calculus and differential geometry. The differential calculus of curves and surfaces used in this chapter can be found in many books, and no doubt most readers are familiar with this material. Nevertheless, a few references to specific results may be useful. As a general reference on calculus, and as a specific reference for the implicit function theorem, we suggest the text by Courant and John [76]. (The implicit function theorem can be found on page 221 of volume II.) Elementary results about classical differential geometry can be found in [250]. A statement and proof of Sard’s theorem can be found in [170].

Topology of plane sets. All of the material presented is quite straightforward, except the Jordan curve theorem. We believe that one of the best proofs of this result is still to be found in the book by M. H. A. Newman [206]. Newman’s treatment is elementary, it describes the problems associated with proving this “obvious” result, and it introduces the nonspecialist to some simple combinatorial topology. Another interesting approach to prove the Jordan curve theorem can be found in [87].

Level lines. An introduction to the use of level lines in computer vision can be found in [57]. We relied directly on the Jordan curve theorem to show that the level sets of a function, and hence the function, could be reconstructed from its level lines (Propositions 4.11 and 4.12). In Proposition 4.12 we made some (unnecessary) assumptions to ensure that we were dealing with a finite number of curves in the rectangle. A more general reconstruction assumes that the set to be reconstructed from its boundary is a set with finite perimeter. Then it can be shown that the set’s boundary can be defined uniquely as a countable union of Jordan arcs and curves, and the reconstruction is still valid. A complete discussion of this general reconstruction is given in [16].

Curvature. It is by now a well-known mathematical technique to define a set implicitly as the zero set of its distance function. In case the set is a curve, one can compute its curvature at a point x by computing the curvature curv(u)(x), where u is a signed distance function of the curve. This yields an intrinsic formula for the curvature that is not dependent on a parameterization of the curve. The same technique has been applied in recent years as a useful numerical tool. This started with Barles report on flame propagation and was extended by Sethian [245] and by Osher and Sethian [214] in a series of papers on the numerical simulation of the motion of a surface by its mean curvature.

Chapter 4

The Main Contrast-Invariant Equations

The purpose of this chapter is to introduce those contrast-invariant PDEs that are essential for contrast-invariant image analysis. These are the differential equations that will appear later in asymptotic results similar to the results in Theorem 2.3. There, we related the heat equation to a linear integral operator. In the later results, the PDEs presented in this chapter will replace the heat equation, and nonlinear contrast-invariant operators will replace the linear integral operator. The differential geometry used in this chapter was developed in Chapter 4, and in that sense this chapter follows Chapter 4 directly. In fact, Chapters 4 and 5 form a logical unit.

4.1 The definition of a shape and how it is recognized

Recall that a shape was defined by Koendernik and van Doorn as a closed subset of ℝ^{N} (section 2.4); recall further the problems that arose when such shapes were smoothed using the Gaussian (Figures 2.9 and 2.10). This highlights the fact that there are two interacting ideas here: the definition of shape and the means of smoothing. In our approach, smoothing is more fundamental, and thus we need to define shape so it is compatible with our concepts of contrastinvariant smoothing and shape recognition. We wish to reduce image smoothing to smoothing the level sets of an image and, further, to smoothing the boundaries of these level sets. Thus, based on what we have done in Chapter 4, it would be natural to define a shape as a compact set in ℝ^{2} whose boundary is a finite set of Jordan curves. If all of these Jordan curves are disjoint, we have an unambiguous picture, but if the curves touch or coincide in places, problems, both theoretical and practical, arise. These are important problems, but they are outside the main focus of the book. For these reason, we are going to make the following very restrictive definition of shape.

77 78 CHAPTER 4. THE MAIN CONTRAST-INVARIANT EQUATIONS

Definition 4.1. An elementary shape is a compact set in ℝ^{2} whose interior is the interior of a rectifiable Jordan curve.

The implication of this definition is that real shapes are made up of these elementary shape, but by considering only these elementary shapes we avoid the problems hinted at above. Thus, is what follows, we will be smoothing and recognizing individual Jordan curves.

In images of a natural environment, most observed objects are partially hidden (occluded) by other objects and often deformed by perspective. When we observe a Jordan curve we cannot be sure that it belongs to a single object; it may be composed of pieces of the boundaries of several objects that are occluding each other. Shape recognition technology has therefore focused on local methods, that is, methods that work even if a shape is not in full view and if the visible part is distorted. As a consequence, image analysis adopts the following principle: Shape recognition must be based on local features of the shape’s boundary, in this case, local features of the Jordan curve and not on its global features. If the boundary has some degree of smoothness, then these local features are based on the derivatives of the curve, which can include the tangent vector, curvature, and so on.

Before beginning the technical aspects of our version of shape recognition, we note that most local recognition methods involve the “salient” points of a shape, which are the points where the curvature is zero (inflection points) and points where the curvature has a maximum or minimum (the “corners” of the shape). These methods reduce a shape to a finite code that consists of the coordinates of a set of characteristic points, which are mainly corners and inflection points. Recognition then amounts to comparing these sets of numbers. This is done statistically.

4.2 Multiscale features and scale space

The methods we have just outlined—in fact, all nonglobal computational shape recognition methods—make the following two basic assumptions, neither of which is true in practice for the analyzed shapes:

• The shape has a smooth boundary (a smooth Jordan curve).

• The boundary has a finite number of inflexion points and points where the curvature has a local maximum or local minimum.

The fact that these conditions can be obtained by properly smoothing a Jordan curve was proven in 1986 by Gage and Hamilton [116] and in 1987 by Grayson [123]. They showed that it is possible to transform a rectifiable Jordan curve into a smooth Jordan curve by using the so-called intrinsic heat equation.

Before proceeding, we wish to inject a comment about notation. For convenience, and unless it would cause ambiguity, we will not make a distinction between a Jordan curve Γ (the topological object) and a function x ↦ x(x) such that Γ = {x(x)}. As we have already done, we will speak of the Jordan curve x. Since we will be speaking of families of Jordan curves dependent on a parameter t > 0, we will most often denote these families by x(t, y), where the second variable is the parameterization of the Jordan curve. Thus, x(t, y) has 4.2. MULTISCALE FEATURES AND SCALE SPACE 79

three meanings: a family of Jordan curves, a family of functions that represent these curves, and a particular point on one of these curves. We will usually use s to denote an arc-length parameter. Finally, everything we do in this chapter is local—it takes place in some neighborhood of a given point. This means we are generally speaking of Jordan arcs.

Definition 4.2. Let x(t, s), t > 0, be a family of C^{2} Jordan curves and assume that for each t, s is an arc length parameterization of x(t, s). We say that x(t, s) satisfies the intrinsic heat equation if

\frac{∂x}{∂t}(t, s) = \frac{∂^{2}x}{∂s^{2}}(t, s) = Curv(x)(t, s).  (4.1)

Theorem 4.3 (Grayson). Let x_{0} be a Jordan curve with finite length. By using the intrinsic heat equation, it is possible to evolve x_{0} into a family of Jordan curves x(t, s) such that x(0, s) = x_{0}(s) and such that for every t > 0, x(t, s) is C^{∞} and satisfies equation (4.1). Furthermore, x(t, s) has only a finite number of inflection points and curvature extrema, and the number of these points does not increase with t. For every initial curve, there is a scale t_{0} such that the curve x(t, s) is convex for t ≥ t_{0} and there is a scale t_{1} such that the curve x(t, s) is a single point for t ≥ t_{1}.

We have assumed that the initial curve x_{0} has finite length. This means that x_{0} is rectifiable and, consequently, is differentiable almost everywhere. That is about all one can say about x_{0}. However, the finite-length assumption is enough to ensure that, for each t > 0, x(t, s) is C^{∞} and the scalar function n(t, s) · Curv(x)(t, s) has finitely many zero-crossings and extrema. (Here we have defined the normal vector n as the tangent vector ∂x/∂s rotated to the left by π/2.) What is even better is that the number of these characteristic points can only decrease as the “time parameter” t increase. This means that the scale space is causal in the terminology of vision theory. (Recall that in Chapter 2 we said that a process is causal if it did not introduce new features.) These results of Gage and Hamilton and of Grayson were quickly embraced by the image processing community, where several attempts to define scale space for shapes had already come very close to the equation studied by these investigators.

It is time to say what we mean by “scale space.” We will refer to any process that smooths a shape (which is represented here by a Jordan curve) and that depends on a real parameter t (which measures the degree of smoothing) a shape scale space. Thus a shape scale space associates with an initial Jordan curve x(0, s) = x_{0}(s) a family of smooth curves x(t, s). For example, the intrinsic heat equation eliminates spurious details of the initial shape and retains simpler, more reliable versions of the shape, and these smoothed shapes have finite codes. Suppose that we wish to compare two original versions of a shape x_{0} and x_{1} that have been captured under different conditions of noise and distortions. Comparing these two shapes is simply impossible. If, however, they are smoothed to the shapes x_{0}(t, ·) and x_{1}(t, ·), then it is possible to compare the codes of x_{0}(t, ·) and x_{1}(t, ·).

80 CHAPTER 4. THE MAIN CONTRAST-INVARIANT EQUATIONS

4.3 From image motion to curve motion

The intrinsic heat equation is only one example from a large family of nonlinear equations that move curves with a curvature-dependent speed, that is, ∂x/∂t is a function of the curvature of the curve x. The only requirement for our purposes is that the speed is a nondecreasing function of the magnitude of the curvature |Curv(x)|.

Definition 4.4. We say that a C^{2} function u : ℝ^{+} × ℝ^{2} → ℝ satisfies a curvature equation if for some real-valued function g(κ, t), which is nondecreasing in κ,

\frac{∂u}{∂t}(t, x) = g(curv(u)(t, x), t)|Du|(t, x).  (4.2)

Definition 4.5. Let x(t, s) be a family of C^{2} Jordan curves such that for every t > 0, s ↦ x(t, s) is an arc-length parameterization. We say that the functions x(t, s) satisfy a curvature equation if for some real-valued function g(κ, t), which is nondecreasing in κ, they satisfy

\frac{∂x}{∂t}(t, s) = g(|Curv(x)(t, s)|, t)n(t, s),  (4.3)

where n is a unit vector in the direction of Curv(x).

The fact that this last equation does not make sense if Curv(x) = 0 is not a problem, since it is easily fixed in the cases we consider. We will only be interested in g that are symmetric in their first variable: g(−κ, t) = −g(κ, t) for all t > 0. With this assumption, we proceed as follows: The Jordan curves x(t, s) all have C^{1} unit tangents τ = ∂x/∂s = x′, and hence C^{1} normals n = τ^{⊥}. In the notation of Proposition 4.4, Curv(x) = κ = τ′. The vectors n and κ are collinear so we can write κ = κn, where κ = κ · n. Then (4.3) becomes

\frac{∂x}{∂t}(t, s) = g(κ(t, s), t)n(t, s).  (4.4)

All of these equations are candidates to be curve or image scale spaces, and one of the main objectives of this book is to identify precisely those equations that are relevant for image analysis. The above definitions are quite restrictive because they require the curves or image to be C^{2}. A more generally applicable definition of solutions for these equations will be given in Chapter 15 with the introduction of viscosity solutions. Our immediate objective is to establish the link between the motion of an image and the motion of its level lines. This will establish the relation between equations (4.2) and (4.3).

4.3.1 Curvature equations for curves and real-valued functions

Lemma 4.6. Suppose that (t, x) ↦ u(t, x) is C^{2} in a neighborhood T × U of the point (t_{0}, x_{0}) ∈ ℝ × ℝ^{2}, and assume that Du(t_{0}, x_{0}) ≠ 0. Then there exists an open interval J centered at t_{0}, an open disk V centered at x_{0}, and a unique C^{1} function x : J × V → ℝ^{2} that satisfy the following properties:

(i) u(t, x(t, y)) = u(t_{0}, y) and x(t_{0}, y) = y for all (t, y) ∈ J × V.

4.3. FROM IMAGE MOTION TO CURVE MOTION 81

(ii) The vectors (∂x/∂t)(t, y) and Du(t, x(t, y)) are collinear.

(iii) The function x satisfies the following differential equation:

\frac{∂x}{∂t}(t, y) = −(\frac{1}{|Du|} \frac{∂u}{∂t})\frac{Du}{|Du|}(t, x(t, y)).  (4.5)

Proof. We begin be assuming that a C^{1} function x exists and satisfies (i). By differentiating the expression u(t, x(t, y)) = u(t_{0}, y) with respect to t, we see that

\frac{∂u}{∂t}(t, x(t, y)) + Du(t, x(t, y) · \frac{∂x}{∂t}(t, y) = 0.

The vectors (∂x/∂t)(t, y) and Du(t, x(t, y)) are collinear if and only if x satisfies equation (4.5). Our proof focuses on the solutions of equation (4.5), and to make things more transparent, we write equation (4.5) as follows:

\frac{dx}{dt}(t) = f(t, x(t)),  (4.6)

where

f(t, x(t)) = −(\frac{1}{|Du|} \frac{∂u}{∂t})\frac{Du}{|Du|}(t, x(t)).

We have suppressed the parameter y, which will be the initial value of the solution of equation (4.6), to clarify the properties of f and the properties of the solutions of (4.6).

Since u is C^{2} in T × U, f is continuously differentiable at any point (t, y) ∈ T ×U where Du(t, y) ≠ 0. The assumption that Du(t_{0}, x_{0}) ≠ 0 implies that this is true in a neighborhood of (t_{0}, x_{0}), which, to avoid more notation, we assume is T × U. Thus, f is continuously differentiable on T × U, and consequently f satisfies a Lipschitz condition at each point of T × U. We can now apply a version of Cauchy–Lipschitz theorem that includes dependence of the solutions on initial conditions:

(1) There is an open interval J ⊂ T centered at t_{0} and an open disk V ⊂ U centered at x_{0} such that, for every point (t′, y) ∈ J × V, there is a unique solution t ↦ x(t, t′, y) of equation (4.6), whose values are in U and such that x(t′, t′, y) = y.

(2) The mapping (t, t′, y) ↦ x(t, t′, y) is uniformally continuous on J × J × V.

(3) There is an open disk W ⊂ V centered at x_{0} such that, for any point (t, t′, y) in J × J × W, the equation y = x(t′, t, z) has the unique solution z = x(t, t′, y) in V.

This is clearly more than we need to prove the lemma. We will complete the proof of the lemma and then comment on other properties of the solutions.

The proof is essentially complete; we need only to interpret (1). For the lemma, we take the J and V of item (1). The function t ↦ x(t, t′, y) is the unique solution of equation (4.5), and by taking t′ = t_{0} in (1) we have the boundary condition that appears in (i). Condition (ii) is clearly satisfied because x satisfies (4.5). Finally, the solution (t, t_{0}, y) ↦ x(t, t_{0}, y), which we write x(t, y), satisfies u(t, x(t, y)) = u(t_{0}, y): The derivative with respect to t is zero, so u(t, x(t, y)) is a constant, and we know that u(t_{0}, x(t_{0}, y)) = u(t_{0}, y).

82 CHAPTER 4. THE MAIN CONTRAST-INVARIANT EQUATIONS

We wish to relate this result directly to the movement of the level lines of u. Thus, let Γ(t_{0}, x_{0}) denote the level line {y | u(t_{0}, y) = u(t_{0}, x_{0})} and parameterize Γ(t_{0}, x_{0}) by an arc length s ∈ (−ε, ε) such that y(0) = x_{0}. For a fixed t ∈ J, consider the function s ↦ x(t, t_{0}, y(s)). This is a continuous function of s (by (2)), and it is one-to-one for y(s) ∈ W (by (3)). Furthermore, u(t, x(t, y(s)) = u(t, y(s)) = u(t_{0}, x_{0}) by (i) and the fact that y(s) ∈ Γ(t_{0}, x_{0}). This means that s ↦ x(t, t_{0}, y(s)) is a Jordan arc that agrees with the level line of u that passes through x(t, y(0)), which means, by the uniqueness of this level line, that s ↦ x(t, t_{0}, y(s)) is the level line of u at level u(t_{0}, x_{0}) is some neighborhood of x(t, y(0)). The point of this is that the solutions of equation (4.5) describes more than the motion of points y: If the initial values y are restricted to the level curve Γ(t_{0}, x_{0}), then the equation describes the evolution of this curve, and the evolved curves are again level curves of u at the same level as the original curve.

What should emerge from this lemma is the following picture: There is a disk W centered at x_{0} and it is filled with two sets of curves: the level curves of u and the trajectories t ↦ x(t, y). Through each point y ∈ W, there is a unique trajectory x(t, y) and a unique level line of u; furthermore, the vectors (∂x/∂t)(t, y) and Du(t, x(t, y)) are collinear. This means that (∂x/∂t)(t, y) is normal to the level line of u at (t, y). This geometry explains the next definition.

Definition 4.7. Given the conditions of Lemma 5.6, the trajectory t ↦ x(t, y) is called the normal flow starting from (t_{0}, y).

The following results are local, in the sense that we will always be looking at what happens in a neighborhood of a point (t_{0}, x_{0}) where the gradient of a function u does not vanish. Also, the function x will mean the function (t, y) ↦ x(t, y), where y is restricted to be on the level curve of u that passes through x(t, x_{0}). In other words, x(t, y) is the family of level curves described above. We will also write this function as x(t, s), where s is an arc-length parameter for the level curve of u. The next few results relate a given evolution rule for u with the corresponding evolution rule for its level curves x.

Proposition 4.8. Assume that the function (t, x) ↦ u(t, x) is C^{2} in a neighborhood of (t_{0}, x_{0}) and that Du(t_{0}, x_{0}) ≠ 0. Let g : ℝ × ℝ^{+} → ℝ be continuous and nondecreasing with respect to κ and such that g(−κ, t) = −g(κ, t). Then u satisfies the curvature motion equation

\frac{∂u}{∂t}(t, x) = g(curv(u)(t, x), t)|Du|(t, x)  (4.7)

in a neighborhood of (t_{0}, x_{0}) if and only if the normal flow t ↦ x(t, ·) satisfies the curvature equation

\frac{∂x}{∂t}(t, s) = g(κ(t, s), t)n(t, s).  (4.8)

Proof. We know from Lemmas 4.15 and 5.6 that there is a neighborhood of N of (t_{0}, x_{0}) such that

Curv(z)(t, s) = −curv(u)(t, x)\frac{Du}{|Du|}(t, x)  (4.9)

4.3. FROM IMAGE MOTION TO CURVE MOTION 83

and

\frac{∂x}{∂t}(t, s) = −(\frac{1}{|Du|} \frac{∂u}{∂t})\frac{Du}{|Du|}(t, x(t, s))  (4.10)

for (t, x) ∈ N. An explanation of the notation is in order before proceeding with the proof, which is a direct consequence of (4.9) and (4.10): The level curve of u at scale t and level u(t_{0}, x_{0}) is denoted by z(t, ·). For a given t > 0, s denotes an arc-length parameter for the curve z(t, ·). The point x appearing in the right-hand side of equation (4.9) is the point on the curve z(t, ·) that corresponds to the value of s appearing in the left-hand side of (4.9). The s in equation (4.10) represents the initial value of the normal flow, as previously explained following Lemma 5.6.

The first step is to choose the arc-length parameterization of z(t, ·) so the unit normal n = τ^{⊥} is equal to Du/|Du|. Then, Curv(z) · n = −curv(u) and κ = −curv(u). Assume that

\frac{∂u}{∂t} = g(curv(u), t)|Du|

and substitute the right-hand side in (4.10) to see that

\frac{∂x}{∂t} = −g(curv(u), t)n = −g(−κ, t)n = g(κ, t)n,

which is equation (4.8). For the other direction, assuming (4.8) and substituting it in (4.10) shows that

−g(curv(u), t)n = −(\frac{1}{|Du|} \frac{∂u}{∂t})\frac{Du}{|Du|} = −\frac{1}{|Du|} \frac{∂u}{∂t}n,

which implies the scalar equation (4.7).

Corollary 4.9. Assume that the function (t, x) ↦ u(t, x) is C^{2} in a neighborhood of (t_{0}, x_{0}) and that Du(t_{0}, x_{0}) ≠ 0. Then u satisfies the curvature motion equation

\frac{∂u}{∂t}(t, x) = curv(u)(t, x)|Du|(t, x)  (4.11)

in a neighborhood of (t_{0}, x_{0}) if and only if the normal flow t ↦ x(t, ·) satisfies the intrinsic heat equation (4.1),

\frac{∂x}{∂t}(t, s) = \frac{∂^{2}x}{∂s^{2}}(t, s) = Curv(x)(t, s).

Exercise 4.1. Check that all of the curvature equations mentioned in this section are contrast invariant. That is, assuming that g is a real-valued C^{2} function defined on ℝ, show that the function v defined by v(t, x) = g(u(t, x)) satisfies one of the equations E if and only if u satisfies E.

4.3.2 Introduction to the affine curve and function equations

There are two curvature equations that are affine invariant and are therefore particularly well suited for use in shape recognition.

84 CHAPTER 4. THE MAIN CONTRAST-INVARIANT EQUATIONS

Definition 4.10. The image evolution equation

\frac{∂u}{∂t}(t, x) = (curv(u)(t, x))^{1/3}|Du(t, x)|  (4.12)

is called affine morphological scale space (AMSS). The curve evolution equation

\frac{∂x}{∂t}(t, s) = |Curv(x)(t, s)|^{1/3}n(t, s) ( = \frac{Curv(x)(t, s)}{|Curv(x)(t, s)|^{2/3}} )  (4.13)

is called affine scale space (ASS). (For x ∈ ℝ, x^{1/3} will be defined to be sign(x)|x|^{1/3}.)

It is clear that AMSS and ASS are equivalent in the sense of Proposition 5.8. As one would expect from the names of these equations, they both have some sort of affine invariance. This is the subject of the next definition, the next two exercises, and the next section.

Definition 4.11. We say that a curvature equation E (image evolution equation) is affine invariant, if for every linear map A with positive determinant, there is a positive constant c = c(A) such that (t, x) ↦ u(t, x) is a solution of E if and only if (ct, Ax) ↦ u(ct, Ax) is a solution of E.

Exercise 4.2. Assume that (t, x) ↦ u(t, x) is a C^{2} function and that A is a 2 × 2 matrix with positive determinant, which we denote by |A|. Define the function v by v(t, x) = u(ct, Ax), where c = |A|^{−2/3}.

(i) Prove that for each point x such that Du(x) ≠ 0 one has the relation

curv(v)(x)|Dv(x)|^{3} = |A|^{2}curv(u)(Ax)|Du(Ax)|^{3}.

(ii) Use (i) to deduce that the AMSS equation (4.12) is affine invariant, that is, (t, x) ↦ u(t, x) is a solution of AMSS if and only (t, x) ↦ v(t, x) is a solution of (4.12).

Exercise 4.3. This exercise is to show that the affine scale space (equation (4.13)) is affine invariant. It relies directly on results from Exercise 4.2. Let σ ↦ c(σ) be a C^{2} curve, and assume that |c′(σ)| > 0. Then we know from Exercise 4.2 that

Curv(c)(σ) = \frac{1}{|c′(σ)|^{2}} [c′′(σ) − (c′′(σ) · \frac{c′(σ)}{|c′(σ)|}) \frac{c′(σ)}{|c′(σ)|}] .  (4.14)

Now assume that we have a family of C^{2} Jordan arcs (t, σ) ↦ c(t, σ). By projecting both sides of the intrinsic heat equation onto the unit vector c^{′⊥}/|c′| and by using (4.14), we have the following equation:

\frac{∂c}{∂t} · \frac{c^{′⊥}}{|c′|} = \frac{c′′ · c^{′⊥}}{|c′|^{3}}  (4.15)

We say that c satisfies a curvature equation if it satisfies equation (4.15). In the same spirit, we say that c satisfies an affine equation if for some constant γ > 0

\frac{∂c}{∂t} · c^{′⊥} = γ(c′′ · c^{′⊥})^{1/3}.  (4.16)

4.3. FROM IMAGE MOTION TO CURVE MOTION 85

(i) Suppose that σ = s, an arc-length parameterization of c. Show that equation

(4.15) can be written as

\frac{∂c}{∂t} = Curv(c) + λτ,

where λ is a real-valued function and τ is the unit tangent vector ∂c/∂s. (See the remark following equation (4.23).)

(ii) Let A be a 2 × 2 matrix with positive determinant, and define the curve y by y(t, σ) = Ac(t, σ). We wish to show that if c satisfies an affine motion, then so does y. As a first step, show that Ax · (Ay)^{⊥} = |A|x · y and hence that A(x^{⊥}) · (Ax)^{⊥} = |A||x|^{2} for any x, y ∈ ℝ^{2}.

(iii) Show that if c satisfies equation (4.16), then y satisfies

\frac{∂y}{∂t} · y^{′⊥} = γ|A|^{2/3}(y′′ · y^{′⊥})^{1/3}.

4.3.3 The affine scale space as an intrinsic heat equation

Suppose that for each scale t, σ ↦ x(t, σ) is a Jordan arc (or curve) parameterized by σ, which is not in general an arc length. As in Chapter 4, we will denote the curvature of x by κ. We wish to demonstrate a formal equivalence between the affine scale space,

\frac{∂x}{∂t} = |κ|^{1/3}n(x),  (4.17)

and an “intrinsic heat equation”

\frac{∂x}{∂t} = \frac{∂^{2}x}{∂σ^{2}},  (4.18)

where σ is a special parameterization called affine length. We define an affine length parameter of a Jordan curve (or arc) to be any parameterization σ ↦ x(σ) such that

[x_{σ}, x_{σσ}] = 1,  (4.19)

where [x, y] = x^{⊥} · y. If s is an arc-length parameterization, then we have (Definition 4.2)

τ = x_{s} n = |κ|^{−1}x_{ss} ( = \frac{Curv(x)}{|Curv(x)|}).  (4.20)

We also have

x_{σ} = x_{s} \frac{∂s}{∂σ} and x_{σσ} = x_{ss}(\frac{∂s}{∂σ} )^{2} + x_{s} \frac{∂^{2}s}{∂σ^{2}} .  (4.21)

Thus,

[x_{σ}, x_{σσ}] = [x_{s}, x_{ss}](\frac{∂s}{∂σ} )^{3},

and if (4.19) holds, then

[x_{s}, x_{ss}](\frac{∂s}{∂σ} )^{3} = 1.

Since by (4.20) [x_{s}, x_{ss}] = sign([x_{s}, x_{ss}])|κ|, we conclude that

\frac{∂s}{∂σ} = (sign([x_{s}, x_{ss}])|κ|)^{−1/3}.  (4.22)

86 CHAPTER 4. THE MAIN CONTRAST-INVARIANT EQUATIONS

Substituting this result in the expression for x_{σσ} shown in (4.21) and writing x_{s} = τ, we see that

x_{σσ} = |κ|^{1/3}n + (\frac{∂^{2}s}{∂σ^{2}} )τ .

This tells us that equation (4.18) is equivalent to the following equation:

\frac{∂x}{∂t} = |κ|^{1/3}n + (\frac{∂^{2}s}{∂σ^{2}} )τ .  (4.23)

Now it turns out that the graphs of the functions x that you get from one time to another do not depend on the term involving τ; you could drop this term and get the same graphs. More precisely, Epstein and Gage [90] have shown that the tangential component of an equation like (4.23) dose not matter as far as the geometric evolution of the curve is concerned. In fact, the tangential term just moves points along the curve itself, and the total curve evolution is determined by the normal term. As a consequence, equation (4.17) is equivalent to equation (4.18) in any neighborhood that avoids an inflection point, that is, in any neighborhood where n(x) ≠ 0. At an inflection point, κ = 0, and the two equations give the same result.

4.4 Curvature motion in N dimensions

We consider an evolution (t, x) ↦ u(t, x), where x ∈ ℝ^{N} and u(0, ·) = u_{0} is an initial N–dimensional image. Let κ_{i}(u)(t, x), i = 1, …, N − 1, denote the i^{th} principal curvature at the point (t, x). (This is slightly different from the notation in section 4.6.) The mean curvature is by definition curv(u) = ∑_{i=1}^{N−1} κ_{i}. We will now define three curvature motion flow equations in N dimensions.

Mean curvature motion. This equation is a direct translation of equation (4.11):

\frac{∂u}{∂t} = |Du|curv(u).

This says that the motion of a level hypersurface of u in the normal direction is proportional to its mean curvature.

Gaussian curvature motion for convex functions. We say that a function is convex if all of its principal curvatures have the same sign. An example of such a function is the signed distance function to a regular convex shape. The equation is

\frac{∂u}{∂t} = |Du| ∏_{i=1}^{N−1} κ_{i}.

The motion of a level hypersurface is proportional to the product of its principal curvatures, which is the Gaussian curvature. As we will see in Chapter 20, this must be modified before it can be applied to a nonconvex function.

4.5. COMMENTS AND REFERENCES 87

Affine-invariant curvature motion. The equation is

\frac{∂u}{∂t} = |Du|| ∏_{i=1}^{N−1} κ_{i}|^{1/(N+1)}H( ∑_{i=1}^{N−1} sign(κ_{i})),

where H(N − 1) = 1, H(−N + 1) = −1, and H(n) = 0 otherwise. The motion is similar to Gaussian curvature motion, but the affine invariance requires that the Gaussian curvature be raised to the power 1/(N + 1). There is no motion at a point where the principal curvatures have mixed signs.

4.5 Comments and references

Our definition of shape. We have elected to define a shape as a compact set whose interior consists of the interiors of a finite number of rectifiable Jordan curves whose interiors are disjoint because it presents an unambiguous picture relating a shape to its boundary curves and, conversely, the Jordan curves that are the boundary to the shape. There are, however, other ways to do this: If we were to define a shape as a compact set whose boundary is a finite set of Jordan curves that do not cross (whose interiors do not meet), then there are other ways to make the decomposition of the boundary into Jordan curves unambiguous, from both the mathematical and numerical points of view.

The Italian mathematician Renato Caccioppoli proposed a theory of sets whose boundaries have finite length (finite Hausdorff measure). From his theory, it can be deduced that the boundary of a Caccioppoli set is composed of a countable number of Jordan curves, up to a set with zero length. This decomposition can even be made unambiguous. In other words, the set of Jordan curves associated with a given Caccioppoli set is unique and gives enough information to reconstruct the set [15]. This result justifies our focus on Jordan curves as the representatives of shapes. (For an account of the fascinating life of Caccioppoli we suggest a visit to http://www-gap.dcs.st-and.ac.uk/ history/Mathematicians/Caccioppoli.html.)

The role of curvature in shape analysis. We mention the thesis by G. J. Agin [2] as being one of the first references dealing with the use of curvature for the representation and recognition of objects in computer vision. The nowclassic paper by Asada and Brady [28] entitled “The curvature primal sketch” introduced the notion of computing a “multiscale curvature” as a tool for object recognition. (The title is an allusion to David Marr’s famous “raw primal sketch,” which is a set of geometric primitives extracted from and representing an image.) The Asada–Brady paper led to a long series of increasingly sophisticated attempts to represent shape from curvature [88, 89] and to compute curvature correctly [200].

Affine-invariant curve shortening. Affine-invariant geometry seems to have been founded by W. Blaschke. His three-volume work “Vorlesungen über Differentialgeometrie” (1921–1929) contains definitions of affine length and affine curvature. Curves with constant affine curvature are discussed in [182]. The term “affine shortening” and the corresponding curve evolution equation were introduced by Sapiro and Tannenbaum in [238]. Some mathematical properties 88 CHAPTER 4. THE MAIN CONTRAST-INVARIANT EQUATIONS

were developed by the same authors in [239] and [240]. Angenent, Sapiro, and Tannenbaum gave the first existence and uniqueness proof of affine shortening in [24].

Curve shortening. The mathematical study of the intrinsic heat equation (or curvature motion in two dimensions) was done is a series of brilliant papers in differential geometry between 1983 and 1987. We repeat a few of the titles, which indicate the progress: There was Gage [114] and Gage [115]: “Curve shortening makes convex curves circular.” Then there was Gage and Hamilton [116]: “The heat equation shrinking convex plane curves.” In this paper the authors showed that a plane convex curve became asymptotically close to a shrinking circle. In 1987 there was the paper by Epstein and Gage [90], and, in the same year, Grayson removed the convexity condition and finished the job [123]: “The heat equation shrinks embedded plane curves to round points.” As the reviewer, U. Pinkall, wrote, “This paper contains the final solution of the long-standing curve-shortening problem for plane curves.”

The first papers that brought curve shortening (and some variations) to image analysis were by Kimia, Tannenbaum, and Zucker [160] and by Mackworth and Mokhtarian [181]. Curve shortening was introduced as a way to do a multiscale analysis of curves, which were considered as shapes extracted from an image. In the latter paper, curve shortening was proposed as an efficient numerical tool for multiscale shape analysis.

Mean curvature motion. In his famous paper entitled “Shapes of worn stones,” Firey proposed a model for the natural erosion of stones on a beach [104]. He suggested that the rate of erosion of the surface of a stone was proportional to the Gaussian curvature of the surface, so that areas with high Gaussian curvature eroded faster than areas with lower curvature, and he conjectured that the final shape was a sphere. The first attempt at a mathematical definition of the mean curvature motion is found in Brakke [45]. Later in the book, we will discuss the Sethian’s clever numerical implementation of the same equation [246]. Almgren, Taylor, Wang proposed a more general formulation of mean curvature motion that is applicable to crystal growth and, in general, to the evolution of anisotropic solids [3].

Chapter 5

Contrast-Invariant Monotone Operators. The Threshold Decomposition Principle

This chapter is a direct continuation of Chapter 3. We are going to establish the relations between monotone image operators that are contrast invariant and monotone set operators. We first show how to define a set operator from a function operator using the function’s level sets, and then we show how to construct a contrast-invariant monotone function operator from a monotone set operator. This development culminates in Theorem 6.12. As a first application of these ideas, we introduce the successful denoising operator due to Vincent and Serra called the extrema killer.

5.1 Contrast-invariant function operators

We will be dealing with operators T defined on a set F of real-valued functions on ℝ^{N}. We will also be considering set operators T defined on a collection of subsets of ℝ^{N}, which we denote by L. We are primarily interested in the duality between a function operator T and a set operator T that is established either by beginning with T and defining T on the level sets of functions in F or by starting with T and defining T on functions whose level sets are in L. With this in mind, when we begin with T, we will usually (but not always) assume that 1_{X_{λ}u} ∈ F whenever u ∈ F. On the other hand, if we begin with T , then we will assume that L is closed under descending chains, which means that ⋂_{μ<λ} X_{μ} ∈ L whenever (X_{λ})_{λ∈ℝ} is a descending chain in L. For example, if we begin with T defined on a set F of continuous real-valued functions on ℝ^{N}, then L is a collection of closed subsets of ℝ^{N}. If F is a set of Lebesgue-measurable functions on ℝ^{N}, then L is a set of Lebesgue-measurable subsets of ℝ^{N}. If we begin with a T defined on all of the compact subsets of ℝ^{N}, then the functions in F will have compact level sets. There are other examples, but the main point is that the duality between a function operator T and a set operator T will

89 90 CHAPTER 5. CONTRAST-INVARIANT MONOTONE OPERATORS

depend on a relation between F and L.

We are mainly interested in monotone operators, since they are nonlinear generalizations of linear smoothing using a nonnegative convolution kernel.

Definition 5.1. We say that a function operator T is monotone if

u ≥ v ⟹ T u ≥ T v.  (5.1)

We say that a set operator T is monotone if

X ⊂ Y ⟹ T X ⊂ T Y.  (5.2)

Recall from Chapter 3, Definition 3.9, that any nondecreasing function g : ℝ → ℝ is called a contrast change. In case g is continuous, we call it a continuous contrast change.

Definition 5.2. A function operator T defined on F is said to be contrast invariant with respect to a set G of contrast changes if g ∈ G and u ∈ F imply that g(u) ∈ F and that

g(T u) = T g(u).  (5.3)

If G consists of all continuous contrast changes, we simply say that T is contrast invariant.

It is important to note that saying that T is contrast invariant with respect to some G carries with it the information that the domain of T is invariant with respect to G. We denote the set of contrast changes that satisfy a Lipschitz condition by G_{Lip}.

We saw in Figures 2.6, 2.7, and 2.8 that linear smoothing can introduce new gray levels in an image. One of the main properties of contrast-invariant operators is that they do not create new grey levels in an image. This means in practice that the contrast in images filtered by contrast-invariant operators is not degraded (see Figure 2.7(c)). This is made precise by the next result. The range Ru of a function u : ℝ^{N} → ℝ is the set {s | u(x) = s for some x ∈ ℝ^{N}}.

Proposition 5.3. Let T be an operator on a function space F and assume that T is contrast invariant with respect to G_{Lip}. Then for every u ∈ F, R(T u) ⊂ \overline{Ru}. In particular, if Ru is finite, then the range of T u is a finite subset of Ru.

Proof. Given u ∈ F, we wish to have a continuous contrast change g such that g(s) = s for s ∈ \overline{Ru} and g(s) > s otherwise. One such function is given by

g(s) = s + \frac{1}{2}d(s, \overline{Ru}),

where d(s, X) denotes the distance from s to X, that is, d(s, X) = inf_{x∈X} |s−x|. The function s ↦ d(s, X) is continuous in any complete metric space, and the fact that g is nondecreasing follows directly from the definition of g and the triangle inequality. Thus g is a continuous contrast change. It is not difficult to show that g ∈ G_{Lip}, and we leave this as an exercise. Furthermore, g(s) = s if 5.1. CONTRAST-INVARIANT FUNCTION OPERATORS 91

and only if s ∈ \overline{Ru}, and g(s) > s if s ∉ \overline{Ru}. In particular, g(u) = u. This and the contrast invariance of T imply that

T u = T g(u) = g(T u).

Hence, for every x ∈ ℝ^{N}, T u(x) is a fixed point of g. This means that T u(x) ∈ \overline{Ru}, which proves the result.

The following exercise is more general than is needed for the proof of Proposition 6.3, however, the result will be used in Chapter 7.

Exercise 5.1. Let X be a closed subset of ℝ^{N} and consider the distance function

d(y) = d(y, X) = inf_{x∈X} |x − y|.

Show that d satisfies a Lipschitz condition. Outline: Let Y_{λ} = {y | d(y) ≤ λ}. First, show that Y_{μ} = {y | d(y, Y_{λ}) ≤ μ−λ} for μ > λ. Next, assume that d(y_{0}) < d(y_{1}) and show that y_{0} ∈ Y_{d(y_{0})} but that y_{1} ∉ Y_{(d(y_{0})+d(y_{1}))/2}. Conclude that d(y_{1}, Y_{d(y_{0})}) ≥ (d(y_{1}) − d(y_{0}))/2.

We have indicated several times the importance of image operators being contrast invariant. In practice, image operators are also translation invariant. For x ∈ ℝ^{N} we are going to use the notation τ_{x} to denote the translation operator for both sets and functions: For X ∈ L, τ_{x}X = {x + y | y ∈ X}, and for u ∈ F, τ_{x}u is defined by τ_{x}u(y) = u(y − x). Although τ_{x} denotes two different operators, there should be no confusion because the domains are completely different. We can now define the notion of translation invariance.

Definition 5.4. A set operator T is said to be translation invariant if X ∈ L and x ∈ ℝ^{N} imply that τ_{x}X ∈ L and that

τ_{x}T X = T τ_{x}X.

A function operator T is said to be translation invariant if u ∈ F and x ∈ ℝ^{N} imply that τ_{x}u ∈ F and that

τ_{x}T u = T τ_{x}u.

Translation-invariant operators will play a larger role in Chapter 7 than they do here. We have introduced them now so we can prove Lemma 6.5, but before doing so, we need to define one more invariance. We say that a function operator T commutes with the addition of constants if u ∈ F and c ∈ ℝ imply that u + c ∈ F and that T (u + c) = T u + c. Contrast-invariant operators clearly commute with the addition of constants: Consider the contrast change defined by g(s) = s + c.

Lemma 5.5. Let T be a translation-invariant monotone function operator defined on F that commutes with the addition of constants. If u ∈ F satisfies the Lipschitz condition |u(x) − u(y)| ≤ K|x − y|, then |T u(x) − T u(y)| ≤ K′|x − y| and K′ ≤ K.

Proof. For any x, y, and z, we can write

u(y + z) − K|x − y| ≤ u(x + z) ≤ u(y + z) + K|x − y|.  (5.4)

92 CHAPTER 5. CONTRAST-INVARIANT MONOTONE OPERATORS

Consider x and y fixed and the terms in (5.4) to be functions of z. Since T is monotone, we can apply T to the functions in (5.4) and preserve the inequalities. If we do this and evaluate the result at z = 0, we have

T (u(y + ·) − K|x − y|)(0) ≤ T u(x + ·)(0) ≤ T (u(y + ·) + K|x − y|)(0).

Now use the fact that T commutes with the addition of constants to obtain

T (u(y + ·)(0) − K|x − y| ≤ T u(x + ·)(0) ≤ T (u(y + ·)(0) + K|x − y|.

Next, use the translation invariance of T to write

T u(y) − K|x − y| ≤ T u(x) ≤ T u(y) + K|x − y|,

which is the announced result.

The regularity-preserving properties of monotone operators are further developed in the following exercises.

Exercise 5.2. Assume that the function u : ℝ^{N} → ℝ is uniformly continuous and write

|u(x + y) − u(x)| ≤ ε(|y|),

where x, y ∈ ℝ^{N} are arbitrary and ε is the modulus of continuity of u. Assume that T is a translation-invariant monotone operator that commutes with the addition of constants. Prove that T u is uniformly continuous and that the modulus of continuity of T u is dominated by ε. (Hint: Adapt the proof of Lemma 6.5.) Exercise 5.3. Let T be a contrast-invariant monotone operator. Prove the following statements:

(i) T u = c for every constant function u : ℝ^{N} → c. (ii) u ≥ c implies T u ≥ c, and u ≤ c implies T u ≤ c. (iii) sup |T u(x) − T v(x)| ≤ sup |u(x) − v(x)|, where the supremum is taken over

x ∈ ℝ^{N}. (Hint: Write − sup |u(x) − v(x)| ≤ u(x) − v(x) ≤ sup |u(x) − v(x)|.) (iv) Show that (i), (ii), and (iii) remain true under the weaker assumption that T is

monotone and commutes with the addition of constants.

5.2 From contrast-invariant function operators to set operators: The threshold superposition principle

We begin with a function operator T defined on a class of functions F, and we assume that F contains the characteristic functions 1_{X_{λ}u} for all u ∈ F. Then we can associate with T a set operator T defined on L = {X | X = X_{λ}u, u ∈ F, λ ∈ ℝ} as follows:

T X = X_{1}T 1_{X}.  (5.5)

The first thing to notice is that T is well defined in the sense that the righthand side of (5.5) is a subset of ℝ^{N}. Without further assumptions about T, we cannot say much more. If T is a monotone function operator, then T is a monotone set operator. Indeed, X ⊂ Y if and only if 1_{X} ≤ 1_{Y}, in which case T 1_{X} ≤ T 1_{Y} and X_{1}T 1_{X} ⊂ X_{1}T 1_{Y}.

5.2. FROM CONTRAST-INVARIANT FUNCTION OPERATORS TO SET OPERATORS93

Exercise 5.4. If T is contrast invariant with respect to G_{Lip}, then we know from Proposition 6.3 that RT 1_{X} ⊂ R1_{X} ⊂ {0, 1}. Thus, in this case, T 1_{X} is the characteristic function of some set Y ⊂ ℝ^{N}. Prove that Y = T X, that is, that T 1_{X} = 1_{T X}, where T is defined by (5.5).

Much of what follows in this chapter and in Chapter 7 is based on the next result. For this reason, it is important to pay close attention to the assumptions, which are the standard assumptions we will make when proving results about monotone function operators. Our immediate goal is to reconstruct the function operator T from the set operator T .

Lemma 5.6. Let T be a monotone function operator defined on F and assume that T is contrast invariant with respect to G_{Lip}. Consider the threshold function γ_{λ} defined by γ_{λ}(s) = 1 if s ≥ λ and γ_{λ}(s) = 0 otherwise, and assume that γ_{λ}(u) ∈ F for u ∈ F and λ ∈ ℝ. Then, for u ∈ F,

T γ_{λ}(u) = γ_{λ}(T u)

almost everywhere, except possibly for at most a countable number of λ’s. The inequality T γ_{λ}(u) ≤ γ_{λ}(T u) holds for all λ ∈ ℝ.

Proof. Consider the two functions γ_{λ}^{+ε} and γ_{λ}^{−ε}, illustrated in Figure 6.1, that are defined for ε > 0 as follows:

γ_{λ}^{+ε}(s) = \begin{cases}0 & if s ≤ λ − ε; \\ \frac{s − (λ − ε)}{ε} & if λ − ε ≤ s ≤ λ; \\ 1 & if s ≥ λ. \end{cases}

γ_{λ}^{−ε}(s) = γ_{λ}^{+ε}(s − ε).

Then γ_{λ}^{+ε} and γ_{λ}^{−ε} are Lipschitz-continuous contrast changes, and γ_{λ}^{−ε} ≤ γ_{λ} ≤ γ_{λ}^{+ε}. Since T is monotone and since T and F are invariant under Lipschitzcontinuous contrast changes, we have the following relations for any u ∈ F:

γ_{λ}^{−ε}(T u) = T γ_{λ}^{−ε}(u) ≤ T γ_{λ}(u) ≤ T γ_{λ}^{+ε}(u) = γ_{λ}^{+ε}(T u).

As ε → 0, γ_{λ}^{−ε}(s) ↑ γ_{λ}^{−}(s) and γ_{λ}^{+ε}(s) ↓ γ_{λ}(s) for all s ∈ ℝ. Thus, in the limit we have

γ_{λ}^{−}(T u) ≤ T γ_{λ}(u) ≤ γ_{λ}(T u),

where γ_{λ}^{−}(s) = 1 if s > λ and γ_{λ}^{−}(s) = 0 otherwise. The crux of the proof is to show that

γ_{λ}^{−}(T u(x)) = γ_{λ}(T u(x))

almost everywhere, except possibly for a countable number of λ’s. The only x for which γ_{λ}^{−}(T u(x)) ≠ γ_{λ}(T u(x)) are the points where T u(x) = λ. Let Λ = Λ(T u) be the set of λ for which meas ({x | T u(x) = λ}) > 0. We know from Exercise 3.5 that Λ is at most countable. (Λ may be the empty set.) If λ ∉ Λ, then γ_{λ}^{−}(T u(x)) = γ_{λ}(T u(x)) almost everywhere.

We will usually write “1_{X_{λ}u} ∈ F” rather than “γ_{λ}(u) ∈ F,” but of course they mean the same thing. This is an important assumption about the domain of T, and if 1_{X_{λ}u} is not in F, we may wish to extend T to these functions, as is done in section 7.3.

94 CHAPTER 5. CONTRAST-INVARIANT MONOTONE OPERATORS

Figure 5.1: Approximation of the threshold function γ_{λ} from above and from below

Definition 5.7. We say that the function operator T is obtained from the set operator T by the threshold superposition principle almost everywhere if

T u(x) = sup{λ | x ∈ T X_{λ}u}  (5.6)

for almost every x ∈ ℝ^{N}. If (5.6) is true for all x, then we say that the threshold superposition principle holds everywhere.

Proposition 5.8. Let T be a monotone function operator defined on F and assume that T is contrast invariant with respect to G_{Lip}. Assume that F contains 1_{X_{λ}u} for u ∈ F, and let L be the set of all level sets X_{λ}u, u ∈ F. Define the associated set operator T on L by T X = X_{1}T 1_{X}. Then we have the following results:

(i) If u ∈ F, then

T X_{λ}u = X_{λ}T u  (5.7)

almost everywhere, except possibly for at most a countable number of λ’s.

(ii) The threshold superposition principle holds almost everywhere:

T u(x) = sup{λ | x ∈ T X_{λ}u} for a.e. x ∈ ℝ^{N}.  (5.8)

5.3. FROM SET OPERATORS TO CONTRAST-INVARIANT FUNCTION OPERATORS95

(iii) T is monotone, T ∅ = ∅, and T ℝ^{N} = ℝ^{N}.

Proof. By definition, T X_{λ}u = X_{1}T 1_{X_{λ}u}. We have T X_{λ}u = X_{1}T γ_{λ}(u) everywhere, since γ_{λ}(u) = 1_{X_{λ}u}. By Lemma 6.6, we can interchange the order of T and γ_{λ}, with the possible exception of a countable number of λ. Call this possible exceptional set Λ = Λ(T u). Then for λ ∈ ℝ \ Λ, T X_{λ}u = X_{1}γ_{λ}(T u) almost everywhere. But X_{1}γ_{λ}(v) = X_{λ}v everywhere, and we conclude that for λ ∈ ℝ \ Λ, T X_{λ}u = X_{λ}T u almost everywhere, which proves (i).

By Exercise 3.1, we have T u(x) = sup{λ | x ∈ X_{λ}T u}. Using (i) and Lemma 3.8, we can replace X_{λ}T u with T X_{λ}u to obtain the almost-everywhere result (ii).

We have already shown that T is monotone. The other statements in (iii) follow directly from the definition of T and the fact that T is contrast invariant: T ∅ = X_{1}T 1_{∅}. But 1_{∅}(x) = 0 for all x, and by Exercise 6.2(i) T 1_{∅}(x) = 0 for all x. This means that X_{1}T 1_{∅} = {x | T 1_{∅}(x) ≥ 1} = ∅. A similar argument shows that T ℝ^{N} = ℝ^{N}.

The relation (5.8) has practical implications. It means that T u can be computed by applying T separately to each characteristic function of the level sets X_{λ}u. This leads to the following stack filter algorithm:

u X_{λ}u → T X_{λ}u ; ↗  ↘ ; ⋮ ; ↘  ↗ ; X_{μ}u → T X_{μ}u T u(x) = sup{λ | x ∈ T X_{λ}u}.

The stack-filter method will be developed in Chapter 23 to maintain accurate contrast invariance in numerical schemes for several contrast-invariant PDEs.

5.3 From set operators to contrast-invariant function operators

We have seen in the last section how a contrast-invariant monotone function operator T can be used to define a monotone set operator T and how the function operator can be recovered from T using the threshold superposition principal. A natural question is, Can a monotone set operator be used to define a contrast-invariant monotone function operator? The positive answer is given by Proposition 6.10, but first we will establish the setting.

We are given a monotone set operator T defined on a family of measurable subsets of ℝ^{N} denoted by L, and we wish to define a function operator T by superposition on some set of real-valued functions F. The first question to answer is, How is F defined? There are at least two ways to proceed. Since we are given L, we may be aware of some functions F all of whose level sets belong to L. This means that given u ∈ F, there is a closed descending chain (X_{λ}u)_{λ∈\overline{ℝ}} in L identified with u. There may, however, be descending chains in L that are not the level sets of some u ∈ F. Since the focus in on T and L, we prefer to proceed as follows: Let F be the set of all functions u defined by

u(x) = sup{λ | x ∈ X_{λ}},  (5.9)

96 CHAPTER 5. CONTRAST-INVARIANT MONOTONE OPERATORS

where (X_{λ})_{λ∈\overline{ℝ}} is a closed descending chain in L. Then F contains all of the characteristic function 1_{X} for X ∈ L. (To see this, let (X_{λ})_{λ∈\overline{ℝ}} be defined by X_{λ} = ℝ^{N} for λ ≤ 0, X_{λ} = X for 0 < λ ≤ 1, and X_{λ} = ∅ for λ > 1.)

At this point, one might ask, What about the functions u that are generated by descending chains that are not closed? If u is such a function, we know from Proposition 3.2 that X_{λ} ⊂ X_{λ}u = ⋂_{μ<λ} X_{μ}. Since we plan to define T on u by superposition,

T u(x) = sup{λ | x ∈ T X_{λ}},  (5.10)

where (X_{λ})_{λ∈\overline{ℝ}} is the chain in L that generated u, we want to be assured that X_{λ} is indeed the level set X_{λ}u. It is for this reason that we make the further assumption about L: If (X_{λ})_{λ∈\overline{ℝ}} is a descending chain in L, then ⋂_{μ<λ} X_{μ} ∈ L. This implies that the functions in F are in one-to-one correspondence with the equivalence classes of descending chains in L.

If we wish to have T contrast invariant with respect to some set G of continuous contrast changes, then it would appear that we need to make some assumptions about T and L that will lead to this result. Clearly, the required assumption should be equivalent to saying that for g ∈ G, u ∈ F implies that g(u) ∈ F. By Corollary 3.16, this is equivalent to saying that (X_{g^{−1}(λ)})_{λ∈\overline{ℝ}} is a closed descending chain in L whenever (X_{λ})_{λ∈\overline{ℝ}} is a closed descending chain in L. It is one of the advantages of starting with T and L that no further assumptions are needed. Roughly speaking, monotone set operators are contrast invariant.

Lemma 5.9. If (X_{λ})_{λ∈\overline{ℝ}} is closed descending chain in L, then (X_{g^{(−1)}(λ)})_{λ∈\overline{ℝ}} is a closed descending chain in L for any continuous contrast change g.

Proof. We wish to show that X_{g^{(−1)}(λ)} = ⋂_{μ<λ} X_{g^{(−1)}(μ)} for λ > −∞. (X_{g^{(−1)}(−∞)} = ℝ^{N}, since X_{−∞} = ℝ^{N}.) We are going to use Lemma 3.18, so we will first consider λ ∈ (λ^{−}, λ^{+}), where λ^{−} = g(−∞) and λ^{+} = g(+∞). We also assume that λ^{−} < λ^{+}. Let (μ_{n})_{n∈ℕ} be an increasing sequence that tends to λ as n → +∞. By Lemma 3.18, we know that s_{n} = g^{−1}(μ_{n}) is a strictly increasing sequence such that s_{n} ↑ g^{(−1)}(λ). Since (X_{λ})_{λ∈\overline{ℝ}} is a closed descending chain in L,

X_{g^{(−1)}(λ)} = ⋂_{s<g^{(−1)}(λ)} X_{s} = ⋂_{s_{n}} X_{s_{n}}.

Similarly, ⋂_{μ<λ} X_{g^{(−1)}(μ)} = ⋂_{μ_{n}} X_{g^{(−1)}(μ_{n})}, and since ⋂_{μ_{n}} X_{g^{(−1)}(μ_{n})} = ⋂_{s_{n}} X_{s_{n}}, we have the result for λ ∈ (λ^{−}, λ^{+}). The points λ ≤ λ^{−} and λ ≥ λ^{+} are easily checked using Lemma 3.18 and Exercise 3.13.

Proposition 5.10. Assume that T is a monotone operator defined on a family of sets L with the property that ⋂_{μ<λ} X_{μ} ∈ L whenever (X_{λ})_{λ∈\overline{ℝ}} is a descending chain in L. Assume further that ∅, ℝ^{N} ∈ L and that T ∅ = ∅ and T ℝ^{N} = ℝ^{N}. Let F be the set of all functions u defined by

u(x) = sup{λ | x ∈ X_{λ}},

where (X_{λ})_{λ∈\overline{ℝ}} is a closed descending chain in L. Define the operator T on F by

T u(x) = sup{λ | x ∈ T X_{λ}},

5.3. FROM SET OPERATORS TO CONTRAST-INVARIANT FUNCTION OPERATORS97

where (X_{λ})_{λ∈\overline{ℝ}} is the chain that generated u. Then the T defined on F is monotone and contrast invariant. Furthermore, for u ∈ F,

X_{λ}T u = T X_{λ}u  (5.11)

almost everywhere, except possibly for at most a countable number of λ’s.

Proof. The assumptions about L allow us to identify the level sets of functions in F directly with the sets (chains) in L that were used to generate them. Thus, we have a choice: We can write either X_{λ}u or X_{λ}, where X_{λ} is in the chain that generated u.

The proof of (5.11) is read from Corollary 3.7 by taking X_{λ} = T X_{λ}u and checking that X_{−∞} = ℝ^{N}. The latter follows from the assumption that T ℝ^{N} = ℝ^{N} and the fact that X_{−∞}u = ℝ^{N} (recall that u : ℝ^{N} → ℝ).

If u ≥ v, then X_{λ}v ⊂ X_{λ}u; thus T X_{λ}v ⊂ T X_{λ}u, since T is monotone. If, for some x ∈ ℝ^{N}, T u(x) < T v(x), then there is a λ such that T u(x) < λ ≤ T v(x) and x ∈ T X_{λ}v but x ∉ T X_{λ}u. This is a contradiction. Thus, u ≥ v implies that T u ≥ T v.

To prove that T is contrast invariant, we are going to show that T g(u) and g(T u) have the same level sets whenever g is any continuous contrast change. This follows from the monotonicity of T and the properties of g^{(−1)} given in Lemmas 3.18 and 6.9. As before, we use the notation λ^{+} = g(+∞) and λ^{−} = g(−∞), and we assume that λ^{+} > λ^{−}. (The case λ^{+} = λ^{−} is easily checked.) Thus, assume that λ ∈ (λ^{−}, λ^{+}). For this λ, g^{(−1)}(λ) < +∞, so we know from Corollary 3.16 that g(s) ≥ λ if and only if s ≥ g^{(−1)}(λ). Thus,

X_{λ}g(T u) = {x | g(T u(x)) ≥ λ} = {x | T u(x) ≥ g^{(−1)}(λ)} = X_{g^{(−1)}(λ)}T u.

(5.12)

The function T u is generated by the descending chain (T X_{s})_{s∈\overline{ℝ}}, and although we may not be able to write X_{g^{(−1)}(λ)}T u = T X_{g^{(−1)}(λ)}, we can always write

X_{λ}g(T u) = X_{g^{(−1)}(λ)}T u = ⋂_{s<g^{(−1)}(λ)} T X_{s}.

Now consider g(u). The function u was generated by the closed descending chain (X_{s})_{s∈\overline{ℝ}}, and the same reasoning used to produce (5.12) tells us that the closed descending chain (X_{g^{(−1)}(λ)})_{λ∈\overline{ℝ}} yields g(u). Thus the descending chain (T X_{g^{(−1)}(λ)})_{λ∈\overline{ℝ}} defines T g(u), and we can write

X_{λ}T g(u) = ⋂_{μ<λ} T X_{g^{(−1)}(μ)}.

We have thus reduced the proof to showing that

⋂_{s<g^{(−1)}(λ)} T X_{s} = ⋂_{μ<λ} T X_{g^{(−1)}(μ)}.  (5.13)

The argument should be familiar by now, since it is essentially the one used to prove Lemma 6.9. The intersection on the left-hand side of (5.13) contains all of the sets T X_{s} for s < g^{(−1)}(λ), while the intersection on the right-hand side contains only those T X_{s} for which s = g^{(−1)}(μ) for some μ < λ. This is 98 CHAPTER 5. CONTRAST-INVARIANT MONOTONE OPERATORS

enough, however, since by Lemma 3.17 every neighborhood of g^{(−1)}(λ) contains infinitely many points s = g^{(−1)}(μ), μ < λ. We leave it as an exercise to check that (5.13) is true for λ ≤ λ^{−} and λ ≥ λ^{+}.

This completes the proof, but perhaps we should mention that the monotonicity was used when we used T to map a closed descending chain like (X_{s})_{s∈\overline{ℝ}} into a descending chain (T X_{s})_{s∈\overline{ℝ}}. The latter is not necessarily closed, and in fact, the condition on T that ensures that (T X_{s})_{s∈\overline{ℝ}} is closed is introduced in Definition 6.11.

Exercise 5.5. Show that (5.13) is true for λ ≤ λ^{−} and λ ≥ λ^{+}. Exercise 5.6. The proof of Proposition 6.10 is simpler if g is assumed to be strictly increasing.

(i) Prove Proposition 6.10 for this case. (ii) Given a continuous contrast change g, show that there exist functions f_{ε} and h_{ε},

ε > 0, that are strictly increasing contrast changes such that f_{ε} ≤ g ≤ h_{ε} and such that f_{ε}(x) ↑ g(x) and h_{ε}(x) ↓ g(x) for each x ∈ ℝ as ε → 0.

(iii) Use (i) and (ii), along with the fact that T is monotone, to prove Proposition

6.10.

The assumptions that T ∅ = ∅ and T ℝ^{N} = ℝ^{N} were used at least twice in the proof of Proposition 6.9. The next exercise illustrates why these assumptions are necessary to cover all cases.

Exercise 5.7.

(i) Assume that g(x) = λ_{0} for all x ∈ ℝ. Prove that T g(u) = g(T u) in this case. Notice where that assumptions T ∅ = ∅ and T ℝ^{N} = ℝ^{N} are used.

(ii) Let X_{0} be a fixed subset of ℝ^{N} and consider the mapping T X = X_{0} for all X ∈ L. Check that T u(x) = +∞ if x ∈ X_{0} and that T u(x) = −∞ otherwise. Let g be any bounded, continuous contrast change. Show that T g(u) ≠ g(T u).

(iii) Take X_{1} ⊂ X_{0}, where X_{1} ≠ ∅ and X_{0} ≠ ℝ^{N}. Define the set operator T by T ℝ^{N} = X_{0}, T X = X_{1} if X ≠ ℝ^{N} and X ≠ ∅, and T ∅ = ∅. Show that T u(x) = −∞ if x ∉ X_{0} and deduce again that T g(u) ≠ g(T u) if g is bounded.

(iv) Let T be a monotone set operator, without further assumptions. Show that the corresponding function operator T commutes with any continuous contrast change for which g(−∞) = −∞ and g(+∞) = +∞.

The next exercise is a converse of Proposition 6.8. It states what the previous exercise suggested: T monotone implies T contrast invariant if and only if T ∅ = ∅ and T ℝ^{N} = ℝ^{N}.

Exercise 5.8. Assume that the set operator T is monotone and that the associated function operator T is contrast invariant. Prove that T ∅ = ∅ and that T ℝ^{N} = ℝ^{N}. Hint: Assume that T ℝ^{N} ≠ ℝ^{N} and compute T 1_{ℝ^{N}} using its definition (6.9) and separately using the assumption that T is contrast invariant.

So far, we have considered set operators T that are monotone and such that T ∅ = ∅ and T ℝ^{N} = ℝ^{N}. These conditions are “compatible” with the structural properties of level sets announced in Proposition 3.2(i), but we have made no assumption that is related to the property in Proposition 3.2(ii). This lack of a continuity assumption meant that relation (6.11) is not completely satisfying, since it is an almost-everywhere statement. We will use the condition in the following definition to correct this. We assume, as usual, that L is closed under the intersections ⋂_{n∈ℕ} indexed by ℕ, which is equivalent to being closed under the intersections ⋂_{λ∈\overline{ℝ}} indexed by \overline{ℝ}.

5.3. FROM SET OPERATORS TO CONTRAST-INVARIANT FUNCTION OPERATORS99

Definition 5.11. A set operator T defined on L is said to be upper semicontinuous (u.s.c.) on L if for every descending sequence (X_{n})_{n∈ℕ} in L,

T ( ⋂_{n∈ℕ} X_{n}) = ⋂_{n∈ℕ} T X_{n}.  (5.14)

We also say that T is continuous on descending sequences in L.

The first thing to notice is that if T is u.s.c., then it is monotone. To see this assume that X ⊂ Y and consider the sequence defined by X_{1} = X, X_{n} = Y for n > 1. Then ⋂_{n∈ℕ} X_{n} = X and ⋂_{n∈ℕ} T X_{n} = T X ∩ T Y, and we conclude that T X ⊂ T Y. The next two exercises develop equivalent properties for an operator T to be upper semicontinuous.

Exercise 5.9. Show that a set operator T is upper semicontinuous if and only if

T ( ⋂_{μ<λ} X_{μ}) = ⋂_{μ<λ} T (X_{μ})

for every descending chain (X_{λ})_{λ∈\overline{ℝ}} in L.

We introduce one more notion of continuity for set operators: Given a closed descending chain (X_{λ})_{λ∈\overline{ℝ}} in L, we say that T is upper semicontinuous on closed descending chains in L if T X_{λ} = ⋂_{μ<λ} T X_{μ} whenever (X_{λ})_{λ∈\overline{ℝ}} is a closed chain in L.

It is clear that T is upper semicontinuous on closed chains if it is upper semicontinuous. The content of the next exercise is to show that the converse is true.

Exercise 5.10. Prove the following: If the set operator T is upper semicontinuous on closed chains, then it is upper semicontinuous. Hint: First show that T is monotone. Then, for any descending chain (X_{λ})_{λ∈\overline{ℝ}}, consider the closed descending chain defined by Y_{λ} = ⋂_{μ<λ} X_{μ}.

The next result is Proposition 6.10 with the additional assumption that T is continuous on closed descending chains. For convenience, we restate all of the hypotheses.

Theorem 5.12. Assume that T is a set operator defined on a family of sets L with the property that ⋂_{μ<λ} X_{μ} ∈ L whenever (X_{λ})_{λ∈\overline{ℝ}} is a descending chain in L. Assume further that ∅, ℝ^{N} ∈ L and that T ∅ = ∅ and T ℝ^{N} = ℝ^{N}, and assume also that T is u.s.c. on L. Let F be the set of all functions u defined by

u(x) = sup{λ | x ∈ X_{λ}},

where (X_{λ})_{λ∈\overline{ℝ}} is a closed descending chain in L. Define the operator T on F by

T u(x) = sup{λ | x ∈ T X_{λ}},

where (X_{λ})_{λ∈\overline{ℝ}} is the chain that generated u. Then the T defined on F is monotone and contrast invariant. Furthermore, for u ∈ F,

X_{λ}T u = T X_{λ}u  (5.15)

for every λ ∈ \overline{ℝ}.

100 CHAPTER 5. CONTRAST-INVARIANT MONOTONE OPERATORS

Proof. We have seen that if T is u.s.c., then it is monotone. That T is monotone and contrast invariant is part of the content of Proposition 6.10. Thus, we need only to prove equation (5.15). This, however, is read directly from Proposition 3.2. With the notation of Proposition 3.2, let T X_{λ}u = X_{λ}. The first part of condition 3.2(i) is satisfied because T is monotone. The second part is also satisfied: X_{−∞}u = ℝ^{N}, and thus X_{−∞} = T X_{−∞}u = ℝ^{N}. Condition 3.2(ii) follows from the assumption that T is upper semicontinuous: ⋂_{μ<λ} T X_{μ}u = T ⋂_{μ<λ} X_{μ}u (by upper semicontinuity of T ), and X_{λ}u = ⋂_{μ<λ} X_{μ}u (by 3.2(ii) applied to u). Thus, ⋂_{μ<λ} T X_{μ}u = T X_{λ}u. Since 3.2(ii) is thus satisfied, we conclude that X_{λ}T u = T X_{λ}u for all λ ∈ ℝ.

Before ending this section with an example, we summerize what has been proven: Starting with a monotone set operator T defined on L, and assuming that L satisfies certain conditions, we were able to define a set of functions F and operator T on F. The assumptions on L implied that 1_{X_{λ}}u ∈ F whenever u ∈ F. The monotonicity of T implied that F was closed under continuous contrast changes and that T was monotone and contrast invariant. And, finally, the assumption that T is continuous on closed descending chains in L, implies that

X_{λ}T u = X_{1}T 1_{X_{λ}u}  (5.16)

for all λ ∈ \overline{ℝ}.

There is, of course, a dual theory that starts with a contrast-invariant monotone function operator T defined on a set of functions F that contains 1_{X_{λ}}u for all u ∈ F and all λ ∈ \overline{ℝ}. Much of this dual theory has already been developed, mainly in Proposition 6.8. If we know that (5.16) holds for T and F, then it is easy to show that superposition (equation (5.8)) holds everywhere and that T is continuous on the closed chains (X_{λ}u)_{λ∈\overline{ℝ}}, where u ∈ F. There is so far, however, an obvious gap in the duality between (T , L) and (T, F): Starting with T, we have T defined on the level sets X_{λ}u of functions u ∈ F. Thus L consists of level sets of F, and we can recover u and T u by superposition. But what happens when we consider v(x) = sup{λ | x ∈ X_{λ}} and T v(x) = sup\{λ | x ∈ T X_{λ}\}, where (X_{λ})_{λ∈\overline{ℝ}} is an arbitrary descending chain of sets in L? This creates an enlargement of F to, say, F′ and an extension of T to T′ on F′. The question is, How are properties preserved as one goes from (T, F) to (T , L) and back? We are not going to treat the general issue, but we do deal with a special case—where L consists of the compact subsets of ℝ^{N}—in Chapter 7.

Exercise 5.11. Prove that if (5.16) holds, then (5.8)) holds everywhere and T is continuous on the closed chains (X_{λ}u)_{λ∈\overline{ℝ}}, where u ∈ F.

Theorem 6.12 and the converse, Exercise 6.11, show that a monotone set operator T must be upper semicontinuous on L to ensure that it and its associated function operator satisfy the commutation relation (5.15) for every λ. Here is a simple example that illustrates this necessity. Define T on measurable subsets of ℝ as follows: For a > 0, let T X = X if meas (X) > a, and T X = ∅ otherwise. Let u : ℝ → ℝ be defined by u(x) = −|x|. Then

T u(x) = sup{λ | x ∈ T X_{λ}u} = min{−|x|, −a/2}.

Thus X_{−a/2}T u = [−a/2, a/2], and X_{−a/2}u = [−a/2, a/2]. Since meas (X_{−a/2}u) = a, T X_{−a/2}u = ∅ and X_{−a/2}T u ≠ T X_{−a/2}u.

5.4. THE EXTREMA KILLER: AN APPLICATION 101

Here is another way to look at the example. Let the nonincreasing family of sets (X_{λ})_{λ∈\overline{ℝ}} be defined by X_{λ} = [−|λ|, |λ|] for λ ≤ 0 and X_{λ} = ∅ for λ > 0. Then, for all λ, X_{λ} = ⋂_{μ<λ} X_{μ}, but T X_{−a/2} ≠ ⋂_{μ<−a/2} T X_{μ}. Note that λ = −a/2 is the only place where T is discontinuous.

5.4 The extrema killer: An application

This section is devoted to the study of operators that remove “peaks,” or extreme values, from an image. Such peaks are often created by impulse noise. The operators we study are called extrema killer operators, and they have been shown to be very effective at removing this kind of noise. The action of these operators is illustrated in Figures 6.2 and 6.3.

We apply the methods developed in section 6.3 that allow us to define operators on sets and then to extend them to functions, which in this case are images. We will be working with closed, connected sets and connected components of closed sets. The definitions are standard, but we include them here for completeness.

Definition 5.13. Consider a closed subset X of ℝ^{N}. X is disconnected if, when it is considered to be a topological space with the topology inherited from ℝ^{N}, it can be represented as two disjoint open sets. X is connected if it is not disconnected. The connected component of x in X, denoted by cc(x, X), is the maximal connected subset of X that contains x.

Here is an equivalent definition: X is connected if it cannot be written as

X = (A ∩ X) ∪ (B ∩ X),

where A and B are disjoint open sets and both A ∩ X and B ∩ X are not empty.

We wish to define a denoising operator on the set L of all compact subsets of ℝ^{N}. We begin by fixing a scale parameter a > 0. If X ∈ L, then its connected components X_{i} are uniquely determined, and thus X can be represented uniquely as X = ⋃_{i} X_{i}. We want the operator T to remove all of the connected components of X that have measure strictly less than a. We therefore define the small component killer by

T_{a}X = ⋃_{meas (X_{i})≥a} X_{i}.  (5.17)

(Theoretically, X can have an uncountable number of components; take, for example, the Cantor set. However, X can have only a countable number of components with positive measure, and in practice, everything is finite.)

We are going to prove that the small component killer is upper semicontinuous on descending sequences of compact subsets, and for this we need to consider an arbitrary descending sequence of nonempty compact sets (Y_{n})_{n∈ℕ} and its limit Y = ⋂_{n∈ℕ} Y_{n}. The most important comment here is that Y is not empty. Indeed, if Y were empty then the opens sets X_{n} = ℝ^{N} \Y_{n}, n > 1, cover Y_{1}, but there is no finite collection of the X_{n} that covers Y_{1}. This contradicts the assumption that Y_{1} is compact. Thus, Y is nonempty, and of course it is compact. The next most important observation is that the convergence Y_{n} ↓ Y 102 CHAPTER 5. CONTRAST-INVARIANT MONOTONE OPERATORS

Figure 5.2: Extrema killer: maxima killer followed by minima killer. The extrema killer removes all connected components of upper and lower level sets with area less than some threshold, which here equals 20 pixels.

is continuous in the following sense: Given any open set Z that covers Y, there is an index n_{0} such that Y_{n} ⊂ Z for all n ≥ n_{0}. If this were not the case, then Y_{n} ∩ (ℝ^{N} \ Z) ≠ ∅ infinitely often. This implies that (Y_{n} ∩ (ℝ^{N} \ Z))_{n∈ℕ} is a descending sequence of nonempty compact subsets of Y_{1}. But this means that Y ∩ (ℝ^{N} \ Z) ≠ ∅, which is a contradiction.

Lemma 5.14. Let (Y_{n})_{n∈ℕ} be a descending sequence of nonempty compact subsets of ℝ^{N}, and consider the intersection Y = ⋂_{n∈ℕ} Y_{n}. If the Y_{n} are connected, then Y is connected.

Proof. We know that Y is not empty and compact. We will prove that it is connected by contradiction. Thus suppose that Y is not connected. Then we can represent Y by Y = (Y ∩ Z_{1}) ∪ (Y ∩ Z_{2}), where Z_{1} and Z_{2} are disjoint open sets, Y ∩ Z_{1} ≠ ∅, and Y ∩ Z_{1} ≠ ∅. Since Y ⊂ Z_{1} ∪ Z_{2}, there exists an n_{0} such that Y_{n} ⊂ Z_{1} ∪ Z_{2} for all n ≥ n_{0}, and for these n we have

Y_{n} = Y_{n} ∩ (Z_{1} ∪ Z_{2}) = (Y_{n} ∩ Z_{1}) ∪ (Y_{n} ∩ Z_{2}).

Furthermore, Y_{n} ∩ Z_{1} ≠ ∅ and Y_{n} ∩ Z_{1} ≠ ∅. This contradicts the fact that the Y_{n} are connected, and we conclude that Y must be connected.

With the extrema killer we have a prime example of a theory that begins with a set operator T_{a} defined on a collection of subsets L of ℝ^{N}. What we have done in section 6.3 requires that ℝ^{N} and ∅ belong L and that T_{a}ℝ^{N} = ℝ^{N} and T_{a}∅ = ∅. This presents no problem: We add ℝ^{N} to L and extend T_{a} to ℝ^{N} by T_{a}ℝ^{N} = ℝ^{N}. Technically, this is equivalent to extending T_{a} to the one-point 5.4. THE EXTREMA KILLER: AN APPLICATION 103

compactification of ℝ^{N}. The other property of L that was always assumed was that ⋂_{μ<λ} X_{μ} was in L for all descending chains (X_{λ})_{λ∈\overline{ℝ}} in L. Our discussion shows that this is true if L is the family of all compact subsets of ℝ^{N}.

Based on what we have already done, it is now quite easy to show that T_{a} is continuous on descending chains of compact sets.

Lemma 5.15. The small component killer T_{a} is upper semicontinuous on L, the set of all compact subsets of ℝ^{N}.

Proof. We first prove that T_{a} is monotone. Thus, assume X ⊂ Y. Then for every x ∈ X, cc(x, X) ⊂ cc(x, Y). If meas (cc(x, X)) ≥ a, then meas (cc(x, Y)) ≥ a, and we conclude that T_{a}X ⊂ T_{a}Y. Another important property of T_{a} is that it is idempotent: T_{a}^{2}X = T_{a}X. And finally, T_{a} is a contraction mapping: T_{a}X ⊂ X.

Now let (X_{λ})_{λ∈\overline{ℝ}} be any closed descending chain of nonempty compact sets. We wish to show that T_{a}X_{λ} = ⋂_{μ<λ} T_{a}X_{μ}. For any λ > −∞ we have T_{a}X_{λ} ⊂ X_{λ}, and hence

T_{a}X_{λ} ⊂ ⋂_{μ<λ} T_{a}X_{μ} ⊂ ⋂_{μ<λ} X_{μ} = X_{λ}.  (5.18)

By applying T_{a} to (5.18) we see that

T_{a}^{2}X_{λ} ⊂ T_{a}( ⋂_{μ<λ} T_{a}X_{μ}) ⊂ T_{a} ⋂_{μ<λ} X_{μ} = T_{a}X_{λ}.  (5.19)

Since T_{a}^{2}X_{λ} = T_{a}X_{λ}, we see from (5.19) that T_{a}( ⋂_{μ<λ} T_{a}X_{μ}) = T_{a}X_{λ}, and thus we have reduced the proof to showing thatT_{a}( ⋂_{μ<λ} T_{a}X_{μ}) = ⋂_{μ<λ} T_{a}X_{μ}. This last equality can only be false if ⋂_{μ<λ} T_{a}X_{μ} contains connected components with measure less than a. Intuitively, this cannot happen because any such component has already been killed, and here is the proof. Assume that ⋂_{μ<λ} T_{a}X_{μ} is not empty and let Y denote one of its connected components. (If ⋂_{μ<λ} T_{a}X_{μ} is empty the result is clearly true.) Then Y ⊂ T_{a}X_{μ} for every μ < λ. Let Y_{μ} denote the unique connected component of T_{a}X_{μ} that contains Y. If ν < μ, then Y_{μ} is contained in a unique connected component of T_{a}X_{ν}. This component must be Y_{ν}, since both Y_{ν} and Y_{μ} contain Y. Thus, ν < μ implies that Y_{μ} ⊂ Y_{ν}, so by Lemma 6.14 (which clearly is true for chains as well as sequences), ⋂_{μ<λ} Y_{μ} is a connected compact set that contains Y. Since Y is a maximal connected subset of ⋂_{μ<λ} T_{a}X_{μ}, it must be that Y = ⋂_{μ<λ} Y_{μ}, and since meas (Y_{μ}) ≥ a, meas (Y) ≥ a. Thus, as per intuition, all of the components of ⋂_{μ<λ} T_{a}X_{μ} have measure greater than or equal to a, and hence T_{a}( ⋂_{μ<λ} T_{a}X_{μ}) = ⋂_{μ<λ} T_{a}X_{μ}, and T_{a}X_{λ} = ⋂_{μ<λ} T_{a}X_{μ}.

We wish to use T_{a} to define a function operator T_{a} on a family of functions F. If we apply Theorem 6.12 directly, we obtain a function operator T_{a} defined on the set of functions F that is characterized by the fact that u ∈ F if and only if X_{λ}u is compact for all λ ∈ ℝ. In particular, F contains all the continuous functions whose level sets are compact. Furthermore, T_{a} is monotone and contrast invariant and T_{a}X_{λ}u = X_{λ}T_{a}u for all λ. The operators T_{a} and T_{a} are also translation invariant.

There is, however, another setting that is more appropriate for image analysis. This is the situation described in Chapter 1, where we started with a 104 CHAPTER 5. CONTRAST-INVARIANT MONOTONE OPERATORS

function u (an image) defined on [0, 1]^{N} and extended u by symmetry and periodicity to a symmetric C-periodic function u defined on all of ℝ^{N}. We denote these functions by L_{Cs}, where the “s” is to remind us that the functions are symmetric as well as C-periodic. Of course, L_{Cs} ⊂ L_{C}, the set of all C-periodic functions. We could focus on functions defined on [0, 1]^{N} without extending them, but this would have the disadvantage that the space of functions is not translation invariant. For this reason, we prefer to work with L_{C}. To do this, we replace ℝ^{N} with the N-dimensional torus, 𝕋^{N}. All of the results developed in Chapter 3 and this chapter remain true if ℝ^{N} is replaced with the compact space 𝕋^{N}, but the results must be interpreted in this new setting. For example, if we replace ℝ^{N} with 𝕋^{N} in Theorem 6.12, then we have the same results, but the functions are defined on 𝕋^{N}. The functions defined on 𝕋^{N} are in one-to-one correspondence with the C-periodic functions on ℝ^{N}, and thus one can consider either C-periodic functions on ℝ^{N} or functions on 𝕋^{N}. We prefer to interpret these as C-periodic functions on ℝ^{N}. On the other hand, it is perhaps easier to think of the level sets of these functions as subsets of 𝕋^{N} rather than “periodized” subsets of C = [−1, 1]^{N}. With this as background, the next definition makes sense.

Definition 5.16. The function operator T_{a} defined by

T_{a}u(x) = sup{λ | x ∈ T_{a}X_{λ}u}

on the set F of functions in L_{C} whose level sets are compact (as subsets of 𝕋^{N}) is called a maxima killer.

Exercise 5.12. Show that T_{a} maps L_{Cs} into L_{Cs}.

Theorem 5.17. The maxima killer T_{a} is monotone, contrast invariant, and translation invariant, and

X_{λ}T_{a}u = T_{a}X_{λ}u  (5.20)

for all λ. As a consequence of (5.20), no connected component of a level set of T u has measure less than a. Furthermore, T_{a} maps the continuous C-periodic functions into continuous C-periodic functions.

Proof. The proof that T_{a} is monotone, contrast invariant, and satisfies (5.20) is read directly from Theorem 6.12 and Lemma 6.15. Clearly, T_{a} and T_{a} are translation invariant. Knowing that T_{a} satisfies these condition, we see from Exercise 6.2(iii) that T_{a} is a contraction mapping:

sup_{x∈ℝ^{N}} |T_{a}u(x) − T_{a}v(x)| ≤ sup_{x∈ℝ^{N}} |u(x) − v(x)|  (5.21)

for u, v ∈ F. If u ∈ L_{C} is continuous, then it is uniformly continuous and we can write |u(x + y) − u(x)| ≤ ε(|y|), where ε is the modulus of continuity of u. Then from Exercise 6.2 and (5.21) we have

|T_{a}u(x + y) − T_{a}u(x)| ≤ |u(x + y) − u(x)| ≤ ε(|y|).

A maxima killer T_{a} cuts off the peaks of continuous functions, but it does nothing for the “valleys.” We could define a minima killer to fill the valleys by 5.4. THE EXTREMA KILLER: AN APPLICATION 105

Figure 5.3: Extrema killer: maxima killer followed by minima killer. Above, left: original image. Above, right: image after extrema killer removes connected components of 20 pixels or less. Below: level lines (levels of multiples of 4) of the image before and after the application of the extrema killer.

developing a dual theory based on lower level sets X_{λ}^{−}u = {x | u(x) ≤ λ}. This would lead to a different set of functions F^{−} and a different function operator T_{a}^{−}, but the set operator T_{a} would be the same. It is, however, simpler to define a minima killer T_{a}^{−} in terms of T_{a} as

T_{a}^{−}u = −T_{a}(−u).

It is clear that both of these operators have the same continuity properties. A good denoising process is to alternate T_{a} and T_{a}^{−}, as illustrated in Figures 6.2 and 6.3 . We note, however, that T_{a} and T_{a}^{−} do not necessarily commute, as is shown in the next exercise.

Exercise 5.13. Let N = 1 and take u(x) = sin x. Compute T_{a}u and T_{a}^{−}u and show that they commute on u if a ≤ π and do not commute if a > π. Following the same idea, construct function u defined on [−π, π]^{2} such that T_{a}T_{a}^{−}u ≠ T_{a}^{−}T_{a}u.

Exercise 5.14 (A major project). Outline the dual theory based on lower level sets. Begin with Proposition 3.2. Pay particular attention to the definitions and key results that must be recast. Give a reformulation of Theorem 6.12, and finally, show that the resulting minima killer is the operator defined above as T^{−} .

106 CHAPTER 5. CONTRAST-INVARIANT MONOTONE OPERATORS

5.5 Comments and references

Contrast invariance and stack filters. Image operators that commute with thresholds have been popular because, among other reasons, they are easily implemented in hardware (VLSI). This led to very simple patents being awarded in signal and image processing as late as 1987 [77]. These operators have been given four different names, although operators are equivalent: stack filters [51, 131, 271]; threshold decomposition [135]; rank filters [68, 159, 273]; and order filters [253]. The best known of these are the sup, inf, and median operators. The implementation of the last named has received much attention because of its remarkable denoising properties [105, 208, 276].

Maragos and Shafer [187, 188] and Maragos and Ziff [189] introduced the functional notation and established the link between stack filters and the Matheron formalism in “flat” mathematical morphology. The complete equivalence between contrast-invariant operators and stack filters, as developed in this chapter, does not seem to have appeared elsewhere; at least we do not know of other references. A related classification of rank filters with elegant and useful generalizations to the so-called neighborhood filters can be found in [159].

The extrema killer. The extrema killer is probably the most efficient denoising filter for images degraded by impulse noise, which is manifest by small spots. In spite of its simplicity, this filter has only recently seen much use. This is undoubtedly due to the nontrivial computations involved in searching for the connected components of upper and lower level sets. The first reference to the extrema killer that we know is [71]. The filter in its generality was defined by Vincent in [263]. This definition fits into the general theory of connected filters developed by Salembier and Serra [235]. Masnou defined a variant called the grain filter that is both contrast invariant and invariant under reverse contrast changes [193]. Monasse and Guichard developed a fast implementation of this filter based on the so-called fast level set transform [199].

We will develop in Chapter 19 a theory of scale space that is based on a family of image smoothing operators T_{t}, where t is a scale parameter. We note here that the family (T_{a})_{a∈ℝ^{+}} of extrema killers does not constitute a scale space because it does not satisfy one of the conditions, namely, what we call the local comparison principle. That this is so, is the content of Exercise 19.1.

Chapter 6

Contrast-Invariant Monotone Operators as Sup-Inf Operators

The main contents of this chapter are two representation theorems: one for translation-invariant monotone set operators and one for functions operators that are monotone, contrast invariant, and translation invariant. If T is a function operator satisfying these three conditions, then it has a “sup-inf” representation of the form

T u(x) = sup_{B∈B} inf_{y∈B} u(x + y),

where B is a family of subsets of P(ℝ^{N}), the set of all subsets of ℝ^{N}. This theorem is a nonlinear version of the Riesz theorem that states that a continuous linear translation-invariant operator can be represented as a convolution

T u(x) = ∫_{ℝ^{N}} u(x − y)k(y) dy.

In this case, the kernel k is called the impulse response. In the same way, B is an impulse response for the nonlinear operator.

6.1 Translation-invariant monotone set operators

The translation operators τ_{x} were defined for sets and functions in section 6.1 and translation invariance for set operators and function operators was defined in Definition 6.4.

Exercise 6.1. Let u : ℝ^{N} → ℝ be any real-valued function. Show that τ_{x}X_{λ}u = X_{λ}τ_{x}u, x ∈ ℝ^{N}. Exercise 6.2. Prove that T ℝ^{N} = ℝ^{N} if T is translation invariant and T ℝ^{N} ≠ ∅.

We consider a translation-invariant monotone set operator T defined on a family of sets L ⊂ P(ℝ^{N}). We will always assume that L is closed under translation and that ℝ^{N} ∈ L, and to avoid the trivial case, that T ℝ^{N} ≠ ∅.

107 108CHAPTER 6. CONTRAST-INVARIANT MONOTONE OPERATORS AS SUP-INF OPERATORS

Theorem 6.1 (Matheron). Let T be a translation-invariant monotone set operator acting on a family of sets L that contains ℝ^{N}, and assume that T ℝ^{N} ≠ ∅. Then T can be represented by

T X = ⋃_{B∈B} ⋂_{y∈B}(X − y) = {x | x + B ⊂ X for some B ∈ B},  (6.1)

where B = {X | 0 ∈ T X}. Conversely, if B is any nonempty subset of P(ℝ^{N}), then formula (6.1) defines a translation-invariant monotone set operator on P(ℝ^{N}).

Proof. We first check that ⋃_{B∈B} ⋂_{y∈B}(X −y) = {x | x+B ⊂ X for some B ∈ B}, where B is any subset of P(ℝ^{N}) and X is any subset of ℝ^{N}. If x ∈ ⋃_{B∈B} ⋂_{y∈B}(X −y), then for some B ∈ B, x ∈ X −y for all y ∈ B. This means that x + B ⊂ X, and hence x ∈ {x | x + B ⊂ X for some B ∈ B}. Now assume that x + B ⊂ X for some B ∈ B. Then x ∈ X − y for all y ∈ B, which proves that x ∈ ⋃_{B∈B} ⋂_{y∈B}(X − y).

Now let B = {X | 0 ∈ T X}. (Note that B ≠ ∅.) The proof of the main part of the theorem is given by a chain of equivalence relations, which we number to indicate where the assumptions are used:

x ∈ T X ⟺^{(1)} 0 ∈ T X − x ⟺^{(2)} 0 ∈ T (X − x) ⟺^{(3)} X − x ∈ B

⟺^{(4)} X − x = B for some B ∈ B ⟺^{(5)} x + B ⊂ X for some B ∈ B.

The equivalence (2) follows from the translation invariance of T X; (3) is just the definition of B; and (4) is a restatement of (3). The implication from left to right in (5) is obvious. The implication from right to left in (5) is the point where the monotonicity of T is used: Since B ⊂ X − x, it follows from the monotonicity of T that X − x ∈ B. This proves (6.1).

Given any family of sets B, it is possible to define a set operator T : P(ℝ^{N}) → P(ℝ^{N}) by (6.1). Using the representation T X = {x | x + B ⊂ X for some B ∈ B} it is easy to check that T is monotone and translation invariant.

The next exercise shows that B = {X | 0 ∈ T X} is not the only set that can be used to represent T .

Exercise 6.3. Let the set operator T^{∘} be defined on all elements of P(ℝ^{N}) by T^{∘}X = X^{∘}, where X^{∘} denotes the interior of X.

(i) Show that T^{∘} is monotone and translation invariant. (ii) Show that there is a countable set B of open sets that generates T^{∘}. (iii) Show that there are many families B of open sets that generate T^{∘}. Exercise 6.4. Given a nonempty family of sets B ⊂ P(ℝ^{N}), we know from Theorem 7.1 that the set operator T defined by T X = ⋃_{B∈B} ⋂_{y∈B}(X − y) = {x | x + B ⊂ X for some B ∈ B} is a translation-invariant monotone operator defined on all of P(ℝ^{N}). Assume that B ⊂ L ⊂ P(ℝ^{N}), where L is translation invariant but otherwise arbitrary. Let B_{L} = {X | X ∈ L, 0 ∈ T X} and let T_{L} be the operator generated by B_{L}. Show that T = T_{L} Hint: Show that B_{L} ∈ B_{L} implies that there is a B ∈ B such that B ⊂ B_{L}. Use the fact that T and T_{L} agree on L. Prove the general result: If B and B′ are two families of sets such that (i) B ⊂ B′ and (ii) given B′ ∈ B′ there is a B ∈ B such that B ⊂ B′, then B and B′ generate the same set operator T .

6.2. SUP-INF OPERATORS 109

The main point of both of these exercises is that T is generated by the small sets in B. Thus if B ⊂ B′, then you generate the same operator as long as you do not add sets to B′ that are strictly smaller than sets in B. This is the point of condition (ii) in Exercise 7.4.

We will be defining both set operators and function operators in terms of sets B, which are called sets of structuring elements in the language of mathematical morphology. In view of the important role commutation plays in image analysis, it is useful to have a test for B to determine whether or not the operator T can be expected to be upper semicontinuous on a family of sets L. Not surprisingly, the condition we are going to formulate involves both B and L. As usual, we assume that L is closed under translations and that it contains ⋂_{μ<λ} X_{μ} whenever (X_{λ})_{λ∈ℝ} is a decreasing chain of sets in L.

Proposition 6.2. The translation-invariant monotone operator in Theorem 7.1 defined by a given set B is upper semicontinuous on L if and only if the following condition holds: If ⋂_{n∈ℕ} T X_{n} ≠ ∅, then there is a B ∈ B such that x + B ⊂ ⋂_{n∈ℕ} X_{n}, where x ∈ ⋂_{n∈ℕ} T X_{n} and (X_{n})_{n∈ℕ} is any descending sequence in L.

Proof. The proof in one direction is just the definition of T . Assume T is u.s.c. and that ⋂_{n∈ℕ} T X_{n} ≠ ∅. Then T (⋂_{n∈ℕ} X_{n}) = ⋂_{n∈ℕ} T X_{n} is not empty. Thus by the definition of T , for x ∈ T (⋂_{n∈ℕ} X_{n}) there is a set B ∈ B such that x + B ⊂ ⋂_{n∈ℕ} X_{n}.

Now assume that ⋂_{n∈ℕ} T X_{n} ≠ ∅ and take x ∈ ⋂_{n∈ℕ} T X_{n}. Since it is always true that T (⋂_{n∈ℕ} X_{n}) ⊂ ⋂_{n∈ℕ} T X_{n}, we must show that x ∈ T (⋂_{n∈ℕ} X_{n}). By hypothesis, there is a B ∈ B such that x + B ⊂ ⋂_{n∈ℕ} X_{n}. Since T is monotone and translation invariant, x + T B ⊂ T (⋂_{n∈ℕ} X_{n}). The result follows from that fact that 0 ∈ T B.

This result is probably more useful in a negative rather than a positive sense. For example, suppose that B contains a descending sequence (B_{n})_{n∈ℕ}. (This is not always the case. See Exercise 7.4.) Then 0 ∈ ⋂_{n∈ℕ} T B_{n}, but there may not be a B such that B ⊂ ⋂_{n∈ℕ} B_{n}. In this situation, there is no possibility of T being u.s.c. on some set L. A case in point is the operator T^{∘} in Exercise 7.3.

Exercise 6.5. Suppose that B contains exactly one set. Show that T is u.s.c. on P(ℝ^{N}). Generalize this to the case B contains a finite number of sets.

6.2 Sup-inf operators

We are now going to shift the focus from set operators to function operators. The next theorem is the dual to Theorem 7.1.

Theorem 6.3. Let T be a monotone function operator defined on F and assume T is contrast invariant with respect to G_{Lip}. Assume that F contains 1_{X_{λ}u} for u ∈ F, and let L be the set of all level sets X_{λ}u, u ∈ F. Let T be the set operator associated with T: T X = X_{1}T 1_{X}. For x ∈ ℝ^{N} and X ∈ L, let B_{x} = {X | x ∈ T X}. Then for u ∈ F,

T u(x) = sup_{B∈B_{x}} inf_{y∈B} u(y) for a.e. x ∈ ℝ^{N}.  (6.2)

110CHAPTER 6. CONTRAST-INVARIANT MONOTONE OPERATORS AS SUP-INF OPERATORS

If T is also translation invariant, then

T u(x) = sup_{B∈x+B_{0}} inf_{y∈B} u(y) for a.e. x ∈ ℝ^{N}.  (6.3)

Conversely, suppose that for every x ∈ ℝ^{N} there corresponds a nonempty family of sets B_{x}. Then equation (6.2) defines a contrast-invariant monotone function operator, and equation (6.3) defines a contrast-invariant monotone function operator that is also translation invariant. These operators are defined on all functions u : ℝ^{N} → ℝ.

Proof. The first thing to observe is that definitions (6.2) and (6.3) are not vacuous: By Proposition 6.8(iii), T ℝ^{N} = ℝ^{N}, so ℝ^{N} ∈ B_{x} for all x ∈ ℝ^{N}.

For u ∈ F, let T̃u(x) = sup_{B∈B_{x}} inf_{y∈B} u(y), where B_{x} = {X | x ∈ T X}. We are going to prove that T̃u(x) = T u(x) for a.e. x ∈ ℝ^{N}. We know from Proposition 6.8 that X_{μ}T u = T X_{μ}u almost everywhere, except possibly for a countable number of μ. We denote this possible exceptional set by M. Thus, as in the proof of Lemma 3.8, we can choose a countable dense set Λ such that Λ ∩ M = ∅ and X_{λ}T u = T X_{λ}u almost everywhere for λ ∈ Λ. For each λ ∈ Λ, let N_{λ} = (X_{λ}T u \ T X_{λ}u) ∪ (T X_{λ}u \ X_{λ}T u). Then N_{λ} has measure zero as does the countable union N = ⋃_{λ∈Λ} N_{λ}. We are going to prove that T̃u(x) = T u(x) for a.e. x ∈ ℝ^{N} by showing that

T̃u(x) ≥ λ ⟺ T u(x) ≥ λ  (6.4)

for λ ∈ Λ and x ∈ ℝ^{N} \ N. This means that X_{λ}T̃u = X_{λ}T u almost everywhere for all λ ∈ Λ, and it follows from Lemma 3.8 that T̃u(x) = T u(x) for a.e. x ∈ ℝ^{N}.

The proof of (6.4) consists of a sequence of equivalence relations that are justified in the text following the display. In the following, μ and λ are in Λ and x ∈ ℝ^{N} \ N.

T u(x) ≥ λ ⟺^{(1)} T u(x) ≥ μ for all μ < λ ⟺^{(2)} x ∈ X_{μ}T u for all μ < λ

⟺^{(3)} x ∈ T X_{μ}u for all μ < λ ⟺^{(4)} X_{μ}u ∈ B_{x} for all μ < λ

⟺^{(5)} There is a B ∈ B_{x} such that inf_{y∈B} u(y) ≥ μ for all μ < λ

⟺^{(6)} sup_{B∈B_{x}} inf_{y∈B} u(y) ≥ λ ⟺^{(7)} T̃u(x) ≥ λ.

Equivalence (1) is just a statement about real numbers, and (2) is just the definition of level set. It is at (3) that we replace X_{μ}T u with T X_{μ}u while avoiding the possible exceptional sets M and N. Equivalence (4) is just the definition of B_{x}. The implication (5) from left to right is true by taking B = X_{μ}u. The implication from right to left is where the monotonicity of T is used: Since T is monotone, T is monotone, and thus it follows directly from the definition of B that B ⊂ X and B ∈ B_{x} imply that X ∈ B_{x}. Assuming the right-hand side of (5), we have B ⊂ X_{μ}u for all μ < λ. Hence, from what we hve just said, X_{μ}u ∈ B_{x} for all μ < λ, which is the left-hand side of (5). Equivalence

(6) is another statement about real numbers, and (7) is the definition of T̃. This completes the proof of (6.2).

6.2. SUP-INF OPERATORS 111

If T is also translation invariant, then it is clear from Definition 6.4 and the definition of T that T is translation invariant. This implies that B_{x} = x + B_{0} for all x ∈ ℝ^{N}, and (6.3) follows from (6.2).

To prove the converse, define T on {u | u : ℝ^{N} → ℝ} by

T u(x) = sup_{B∈B_{x}} inf_{y∈B} u(y).

The operator T is well defined, since we have assumed that B_{x} is not empty. Clearly, T is monotone: If u ≥ v, then inf_{y∈B} u(y) ≥ inf_{y∈B} v(y), and it follows that T u ≥ T v. If g is any continuous contrast change, then for any nonempty set I ⊂ ℝ, inf_{x∈I} g(x) = g(inf_{x∈I} x) and sup_{x∈I} g(x) = g(sup_{x∈I} x). This implies that T commutes with g. If we replace B_{x} with x + B_{0}, then it is easy to check that T is also translation invariant.

Theorem 7.2 is a result about T that is based on Proposition 6.8. The set operator T enters as a convenience, in the sense that the results can be stated completely in terms of T. For example, the proof began with the fact that X_{λ}T u = T X_{λ}u almost everywhere, with the possible exception of at most a countable number of λ’s. This is exactly the content of Lemma 6.6, where it is written as T γ_{λ}(u) = γ_{λ}(T u). Lemma 6.6 is based on the facts that T is monotone and invariant under Lipschitz-continuous contrast changes. If we bypass the contrast-invariant assumption and assume that T is monotone and that T γ_{λ}(u) = γ_{λ}(T u) for all λ and all u ∈ F, then a review of the proof of Theorem 7.2 reveals that (6.2) is true for all x ∈ ℝ^{N}. Similarly, (6.3) is true for all x ∈ ℝ^{N} if we add the hypothesis that T is translation invariant. We state this as a corollary.

Corollary 6.4. Let T be a monotone operator defined on a family of functions F that includes the functions 1_{X_{λ}u}, u ∈ F, λ ∈ ℝ, and assume that T γ_{λ}(u) = γ_{λ}(T u) for all λ ∈ ℝ and all u ∈ F. Then T can be represented by T u(x) = sup_{B∈B_{x}} inf_{y∈B} u(y) for all x ∈ ℝ^{N}. If T is also translation invariant, then T u(x) = sup_{B∈x+B_{0}} inf_{y∈B} u(y) for all x ∈ ℝ^{N}.

One may ask, What happened to the contrast-invariance assumption? The answer is that it is implied by the very strong assumption that T commutes with all (upper) thresholds: T γ_{λ}(u) = γ_{λ}(T u). In fact, once we have a sup-inf representation, it is clear from Theorem 7.2 that the operator T is contrast invariant. Furthermore, it can be extended as a contrast-invariant monotone operator to all functions u : ℝ^{N} → \overline{ℝ}. The problem with this is that there is no guarantee that this extension preserves commutation with thresholds. But without some form of commutation with thresholds, the theory is not very interesting or useful for applications. Fortunately, there is a more conservative extension that preserves commutation. Here is how it works:

Let F_{g} denote the set of all functions of the form g(u) where u ∈ F and g is a continuous contrast change. Then F ⊂ F_{g} and F_{g} is closed under continuous contrast changes. Extend T to F_{g} using the sup-inf representation given by Corollary 7.3. We know from the proof of Theorem 7.2 that g(T u) = T g(u). Furthermore, the condition T γ_{λ}(u) = γ_{λ}(T u) is preserved with this extension:

X_{λ}T g(u) = X_{λ}g(T u) = X_{g_{λ}^{(−1)}}T u = T X_{g_{λ}^{(−1)}}u = T X_{λ}g(u).

112CHAPTER 6. CONTRAST-INVARIANT MONOTONE OPERATORS AS SUP-INF OPERATORS

Corollary 6.5. With the assumptions of Corollary 7.3, the extension of T to F_{g} preserves commutation with thresholds.

Theorem 7.2 has a useful semi-converse. It states that given a translationinvariant sup-inf operator T represented with the set of structuring elements B, the associated set operator T can also be represented with B.

Proposition 6.6. Let T be a function operator defined on the set of functions F and assume that 1_{X_{λ}u} ∈ F, u ∈ F, λ ∈ ℝ. Let T be the associated set operator defined on L, the set of all level sets of functions in F, by T X = X_{1}T 1_{X}. If T is represented by

T u(x) = sup_{B∈x+B} inf_{y∈B} u(y),  (6.5)

where B is a nonempty set of structuring elements, then T can be represents by

T X = ⋃_{B∈B} ⋂_{y∈B}(X − y) = {x | x + B ⊂ X for some B ∈ B}.  (6.6)

Conversely, if we are given a set operator T that is represented by (6.6) and if we know that T maps measurable sets into measurable sets, then the same set B can be used to represent the function operator T u(x) = sup{λ | x ∈ T X_{λ}u} in the sense that

T u(x) = sup_{B∈x+B} inf_{y∈B} u(y)  (6.7)

for almost every x ∈ ℝ^{N}.

Proof. We have assumed that 1_{X_{λ}u} ∈ F for u ∈ F. If we had not made this assumption, we could have extended T to these functions using the representation (6.5). Thus, for X ∈ L, consider T X = X_{1}T 1_{X}. We see that

inf_{y∈x+B} 1_{X}(y) = \begin{cases}0 if x + B ⊄ X, \\ 1 if x + B ⊂ X. \end{cases}

Thus, sup_{B∈B} inf_{y∈x+B} 1_{X}(y) = 1 if and only if there is a B ∈ B such that x + B ⊂ X, and the representation (6.6) follows directly.

For the converse, assume that we are given the set operator T defined by the nonempty family B. By assuming that T maps measurable sets to measurable sets, we can apply Proposition 6.10 and conclude that X_{λ}T u = T X_{λ}u almost everywhere, with the possible exception of at most a countable set of λ’s. Here we assume that the operator T is defined by T u(x) = sup{λ | x ∈ T X_{λ}u}. With these conditions, we can repeat the proof of Theorem 7.2 and conclude that

T u(x) = sup_{B′∈x+B′} inf_{y∈B′} u(y)  (6.8)

for a.e. x ∈ ℝ^{N}, where B′ = {X | 0 ∈ T X}. We know from Exercise 7.4 that B and B′ generate the same set operator T and hence the same function operator T.

6.2. SUP-INF OPERATORS 113

With the second half of Proposition 7.5, our point of view shifted to that of “starting with a set operator T .” In this setting, the assumption that a function operator commutes with thresholds is replaced with the assumption that T is upper semicontinuous on L, its domain of definition. Thus, if we assume that a set operator T is represented by (6.6) and if we assume that T satisfies the hypotheses of Theorem 6.12, then we can conclude that the function operator T associated with T can be represented by

T u(x) = sup_{B∈x+B} inf_{y∈B} u(y)

for all x ∈ ℝ^{N}, where B is the same family of sets used to represent T .

Exercise 6.6. Use these last results to show that the extrema killer T_{a} can be represented as a sup-inf function operator with the structuring elements

B = {B | B is compact, connected, meas (B) = a, and 0 ∈ B}.

The sup-inf representation for a function operator T and the “∪∩” representation for a set operator T are so general that once an operator is represented in one of these forms on a set of functions F or a set of sets L it can immediately be extended to all functions on ℝ^{N} or all sets X ⊂ ℝ^{N}. This can be a bit dangerous, since, as mentioned above, these extensions cannot be expected to preserve the commutation relation T X_{λ}u = X_{λ}T u, and it is important to maintain this relation for image operators. We saw in the case of the extrema killer that there is commutation in its strongest form, that is, for all λ when F is the set of continuous functions on 𝕋^{N} and L is the set of compact subsets of 𝕋^{N}. In this case, we began with the set operator; in other cases we will be presented with the function operator first and then define the set operator. In all of these cases, we wish to preserve commutation, but to do so it will be necessary to restrict either the family B or the domains of definition of T and T . The next exercise illustrates this problem.

Exercise 6.7. Let B = {{x} | x ∈ D(0, 1)}, where D(0, 1) = {x | |x| ≤ 1}. Show that T u(x) = sup_{y∈x+D} u(y). Let (q_{n})_{n∈ℕ} be a countable dense set in ℝ^{N} and consider u defined by u(x) = 1−1/n if x = q_{n} and u(x) = 0 otherwise. Show that T X_{1}u ≠ X_{1}T u.

The operator T in Exercise 7.7 is one of the classic image operators called a dilation. We will see in section 7.3 that T will commute with all thresholds when its domain of definition is restricted to upper semicontinuous functions.

We end this section by showing that sup-inf operators can be represented as inf-sup operators.

Proposition 6.7. Assume that T can be represented by T u(x) = sup_{B∈B} inf_{y∈x+B} u(y). Then T also has an inf-sup representation

T u(x) = inf_{B∈B′} sup_{y∈x+B} u(y),  (6.9)

where B′ is a new structuring set.

Proof. The result follows from the axiom of choice, Boolean algebra, and properties of real numbers. A map ϕ : B → ⋃_{B∈B} B with the property that ϕ(B) ∈ B is called a selection function. (In general, one must invoke the axiom of choice 114CHAPTER 6. CONTRAST-INVARIANT MONOTONE OPERATORS AS SUP-INF OPERATORS

to guarantee the existence of ϕ.) Let S_{B} denote the set of all selection functions. We then have

sup_{B∈B} inf_{y∈B} u(y) = inf_{ϕ∈S_{B}} sup_{B∈B} u(ϕ(B)) = inf_{ϕ∈S_{B}} sup_{y∈ϕ(B)} u(y).

For ϕ ∈ S_{B} write ϕ(B) = ⋃_{B∈B} ϕ(B) and let D = {ϕ(B) | ϕ ∈ S_{B}}. Then

sup_{B∈B} inf_{y∈B} u(y) = inf_{ϕ(B)∈D} sup_{y∈ϕ(B)} u(y).

The structuring set D thus obtained for the inf-sup representation is not the same one used for the sup-inf representation.

6.3 The Evans–Spruck extension

We are going to consider a monotone function operator T defined on a family of continuous functions F that does not necessarily contain the functions 1_{X_{λ}u} for u ∈ F. We assume that F contains a subset of functions F_{c} all of whose level sets are compact. Our goal is to use T to define its associated set operator on all of the compact sets X = X_{λ}u, where u ∈ F_{c}. To be able to write T X = X_{1}T 1_{X_{λ}u}, we must extend T to the functions 1_{X_{λ}u} = γ_{λ}(u), u ∈ F_{c}. We are going to extend T to these functions by defining T γ_{λ}(u) = γ_{λ}(T u). This means that to have a proper extension (definition) we must show that γ_{λ}(u) = γ_{μ}(v) implies that T γ_{λ}(u) = T γ_{μ}(v) for u, v ∈ F_{c}. This is equivalent to showing that X_{λ}u = X_{μ}v implies that X_{λ}T u = X_{μ}T v for u, v ∈ F_{c}.

Until now, when we began with a monotone function operator T on a set of functions F, we have assumed that F contained the functions 1_{X_{λ}u} for u ∈ F. This assumption and the assumption that F and T were invariant under Lipschitz-continuous contrast changes were sufficient for developing the theory. To compensate for not assuming that F contains the functions 1_{X_{λ}u}, we are going to assume that F and T are invariant under all continuous contrast changes. Clearly, the set F_{c} is invariant under these contrast changes.

We prepare to prove that X_{λ}u = X_{μ}v implies X_{λ}T u = X_{μ}T v for u, v ∈ F_{c} by making two simplifications. Since g(s) = s + λ is an admissible contrast change, u ∈ F_{c} if and only if u + λ ∈ F_{c} for any constant λ, and thus we may assume that X_{0}u = X_{0}v. Similarly, we can replace u and v with the functions g(u) and g(v), where g(s) = min(s, 0), and have X_{0}g(u) = X_{0}u and X_{0}g(v) = X_{0}v. These simplifications justify the assumptions of the next lemma, which is the key to proving our version of the Evans–Spruck extension.^{1}

Lemma 6.8. Let u and v be real-valued functions bounded above by zero and assume that X_{0}u = X_{0}v (≠ ∅). If u is continuous and if the level sets of v are compact for all levels λ ≤ 0, then there is a continuous contrast change h such that h(0) = 0 and u ≥ h(v).

Proof. Define h̃(r) = min{u(x) | x ∈ X_{r}v} for r ≤ 0 and h̃(r) = 0 for r > 0. Notice that h̃(0) = 0 and that h̃ is nondecreasing. The following relation holds 6.3. THE EVANS–SPRUCK EXTENSION 115

1What we are doing here is related to the scheme originally introduced by Osher and Sethian as a numerical method for front propagation [214]. We briefly described their work in the Introduction (see page ??).

for all x ∈ ℝ^{N} by the definition of h̃ and because u(x) ∈ {u(y) | v(y) ≥ v(x)}:

u(x) ≥ min{u(y) | v(y) ≥ v(x)} = h̃(v(x)).

We now use the compactness of the level sets of v to show that h̃ is continuous at zero. Let (r_{k})_{k∈ℕ} be an arbitrary increasing sequence tending to zero. Choose x_{k} ∈ X_{r_{k}}v such that h̃(r_{k}) = u(x_{k}). This is possible because u is continuous and the X_{r_{k}}v are compact and nonempty. Since h̃ is nondecreasing, h̃(r_{k}) → h̃^{−}(0).

Let x be any accumulation point of the set {x_{k}}_{k∈ℕ}. Since the X_{r_{k}}v are compact, all the accumulation points of the set {x_{k}}_{k∈ℕ} are contained in X_{0}v = ⋂_{k∈ℕ} X_{r_{k}}v. This means that u(x) = 0. But lim sup u(x_{k}) = u(x) by the continuity of u, and we conclude that h̃^{−}(0) = 0. At this point h̃ satisfies the announced requirements of h, except that it is not always continuous for all r < 0. This is easily fixed by choosing a continuous nondecreasing function h such that h̃ ≥ h and h(0) = 0. One way to do this is to take h(r) = (1/|r|) ∫_{2r}^{r} h̃(s) ds for r < 0. Then u(x) ≥ h̃(v(x)) ≥ h(v(x)) as announced.

Exercise 6.8. Prove that h(r) = (1/|r|) ∫_{2r}^{r} h̃(s) ds is indeed continuous for r ≤ 0 and that h̃ ≥ h. Find examples of functions u and v defined on ℝ for which h̃ is not continuous.

Everything is now in place to state and prove our version of the Evans– Spruck extension theorem.

Theorem 6.9 (Evans–Spruck). Let T be a contrast-invariant monotone function operator defined on a set F of continuous functions that is invariant under continuous contrast changes. With these assumptions, it is possible to extend T to the characteristic functions 1_{X_{λ}u} by the formula T 1_{X_{λ}u} = 1_{X_{λ}T u} whenever the level sets of u are compact.

Proof. Our preliminary comments show that it is sufficient to prove that X_{0}u = X_{0}v implies X_{0}T u = X_{0}T v when u and v are continuous and bounded by zero and the level sets of u and v are all compact. We assume that X_{0}u = X_{0}v is not empty.

The proof follows directly from Lemma 7.7: Let h be a continuous contrast change such that h(0) = 0 and u ≥ h(v). Since T is monotone and contrast invariant, T u ≤ 0 and T u ≥ T h(v) = h(T v). Using the fact that h(0) = 0, we see that T v(x) = 0 implies that T u(x) = 0. By interchanging the roles of u and v we see that T u(x) = 0 implies that T v(x) = 0 and conclude that X_{0}T u = X_{0}T v.

Corollary 7.9 follows directly from Theorem 7.8.

Corollary 6.10. Let T be a contrast-invariant monotone operator on a set F of continuous functions u : ℝ^{N} → ℝ. Assume that F is invariant under continuous contrast changes, and let F_{c} ⊂ F be the functions in F whose level sets are compact. Then the set operator T defined on the level sets of functions in F_{c} by T X_{λ}u = X_{1}T 1_{X_{λ}u} satisfies the commutation T X_{λ}u = X_{λ}T u for all λ ∈ ℝ.

116CHAPTER 6. CONTRAST-INVARIANT MONOTONE OPERATORS AS SUP-INF OPERATORS

The next result is based on our discussion in section 6.5 regarding the relation between the continuous functions in L_{C}, which we call continuous C-periodic functions, and C(𝕋^{N}), the continuous functions on the N-dimensional torus. The main thing to keep in mind is that, although there is a complete correspondence between C(𝕋^{N}) and the continuous C-periodic functions on ℝ^{N}, the level sets of continuous C-periodic functions do not include all of the compact subsets of C = [−1, 1]^{N}, but only the “C-periodic” compact subsets of C = [−1, 1]^{N}. These, of course, are in one-to-one correspondence with the compact subsets of 𝕋^{N}. On the other hand, every compact subset of [0, 1]^{N} appears in its “symmetrized form” as a compact subset of 𝕋^{N}, or equivalently, as a “C-periodic” compact subset of C = [−1, 1]^{N}.

Corollary 6.11. Let F be a set of real-valued continuous C-periodic functions on ℝ^{N} that is invariant under continuous contrast changes and let T be a contrast-invariant monotone operator defined on F. Then for all λ ∈ ℝ, the set operator T defined on the level sets of the elements of F by the Evans–Spruck extension satisfies the commutation relation

T X_{λ}u = X_{λ}T u.  (6.10)

6.4 The extension to upper semicontinuous functions

We argued in the Introduction that technology is such that digital images can be interpolated to provide a good continuous representations of the original images, and hence that we have at hand continuous images. In practice, the gray levels of these images are quantized and we are left to deal with a finite number of level sets (about 255 with current technology). If we process each level separately, as we wish to do, then we are, in fact, dealing with images that are discontinuous along their quantized level lines. For this reason, it is useful to consider upper semicontinuous functions. These functions arise naturally in our theory if we extend T to a larger class of functions using T and superposition (Proposition 6.10). We will see that this extension is necessary in practice; for example, we will need to use it to extend curvature motion to upper semicontinuous functions.

Definition 6.12. A function u : ℝ^{N} → ℝ is said to be upper semicontinuous (u.s.c.) if either of the following equivalent conditions is true:

(i) The level sets X_{λ}u = {x | u(x) ≥ λ} are closed.

(ii) At every x ∈ ℝ^{N}, u(x) ≥ lim sup_{y→x} u(y) = lim_{r→0} ( lim sup_{y∈D(x,r)} u(y)).

A function u is said to be lower semicontinuous (l.s.c.) if −u is u.s.c.

Exercise 6.9. Show that conditions (i) and (ii) in Definition 7.11 are equivalents and that u is continuous if and only if it is both u.s.c and l.s.c. Prove the following result: If u is u.s.c. and X is compact, then there is a point x ∈ X such that u(x) = sup_{y∈X} u(y). What is the corresponding result if u is l.s.c.?

6.4. THE EXTENSION TO UPPER SEMICONTINUOUS FUNCTIONS 117

Exercise 6.10. Show that Definition 7.11 agrees with Definition 3.15 in case g is a nondecreasing real-valued function defined on ℝ.

Exercise 6.11. Let F be the set of all u.s.c. functions on ℝ^{N} and let T_{r}, r > 0, be the operator on F defined by T_{r}u(x) = sup_{y∈D(x,r)} u(y). Show that T maps F into F and that the corresponding set operator T_{r} is defined by T_{r}X = {x | d(x, X) ≤ r}. T_{r} is a classic image operator called a dilation.

The next step in the program is to extend T to a family of upper semicontinuous functions whose level sets are compact. Here is the setting as it has been developed: The contrast-invariant operator T is defined on a set of continuous functions F. F_{c} is the subset of F all of whose level sets are compact. We have extended T to the functions 1_{X_{λ}u} for u ∈ F_{c} by defining T 1_{X_{λ}u} = 1_{X_{λ}T u}, and thus we have preserved the commutation T X_{λ}u = X_{λ}T u for all u ∈ F_{c}. We would like to use Proposition 6.10 to extend T to all of the functions u defined by u(x) = sup{λ | x ∈ X_{λ}}, where (X_{λ})_{λ∈ℝ} is a descending chain in L, the domain of T . The first problem with this plan is that ⋂_{μ<λ} X_{λ} may not be in the domain of T , and thus Proposition 6.10 cannot be used. Another way to look at this is that there may not be enough functions in F_{c} to ensure that ⋂_{μ<λ} X_{λ} ∈ L. The second problem encountered in trying to apply Proposition 6.10 is that ℝ^{N} is not in L. The way around both of these obstacles is to replace ℝ^{N} with a compact space, and we are going to do this two ways. The first case we have encountered before: We replace ℝ^{N} with 𝕋^{N}, which is equivalent to considering C-periodic functions on ℝ^{N}. In this case, to have all of the compact subsets of 𝕋^{N}, we let F_{c} be all of the continuous functions on 𝕋^{N}. Then the hypotheses of Proposition 6.10 are fulfilled.

For the other case, we replace ℝ^{N} with its one-point compactification ℝ^{N∗}, and we let F_{c} be all of the continuous functions on ℝ^{N∗}. Then ℝ^{N} is compact in the topology of ℝ^{N∗}, and Proposition 6.10 applies. What we have done here is completely equivalent to letting F_{c} be the set of all continuous functions on ℝ^{N} that tend to a finite limit as |x| → +∞. In both cases, F_{c} is invariant under continuous contrast changes. The next theorem includes both of these cases. We will prove the theorem for the case the functions are thought of as defined on ℝ^{N∗}. The proof for the other case follows word for word.

Theorem 6.13. Assume that F_{c} is either the set of all continuous functions on ℝ^{N} that tend to a finite limit as |x| → +∞ or the set of all continuous C-periodic functions on ℝ^{N}. Let T be a contrast-invariant monotone operator defined on F_{c} and let T be its Evans–Spruck extension to L, the set of all compact subsets of ℝ^{N∗} (or 𝕋^{N}). Then T can be extended to a contrast-invariant monotone operator T^{+} on F_{c}^{+}, the set of all upper semicontinuous functions on ℝ^{N} (or upper semicontinuous C-periodic functions on ℝ^{N}) whose level sets are compact. Furthermore, for u ∈ F_{c}^{+} and for all λ ∈ ℝ,

X_{λ}T^{+}u = T X_{λ}u.  (6.11)

Proof. We have argued that Proposition 6.10 applies. This means that there is a contrast-invariant monotone operator T^{+} defined on F_{c}^{+}, since it is clear that the functions u defined by u(x) = sup{λ | x ∈ X_{λ}}, where (X_{λ})_{λℝ} is a closed descending chain in L, are exactly F_{c}^{+}, the upper semicontinuous functions on ℝ^{N∗} whose level sets are compact. If u ∈ F_{c}, then we know from Corollaries 7.9 118CHAPTER 6. CONTRAST-INVARIANT MONOTONE OPERATORS AS SUP-INF OPERATORS

(and 7.10) that T X_{λ}u = X_{λ}T u, and we have

T^{+}u(x) = sup{λ | x ∈ T X_{λ}u} = sup{λ | x ∈ X_{λ}T u} = T u(x).

Hence, T^{+} is indeed a contrast-invariant monotone extension of T from F_{c} to F_{c}^{+}.

We must now prove (6.11). The simplifications used to prove Theorem 7.8 may be used here, so it is sufficient to prove (6.11) for u ≤ 0 and λ = 0. Thus we must show that X_{0}T^{+}u = T X_{0}u, where u ∈ F_{c}^{+} and u ≤ 0. In different notation, this is equivalent to proving that T^{+}γ_{0}(u) = γ_{0}(T^{+}u). From Lemma 6.6, T^{+}γ_{λ}(u) ≤ γ_{λ}(T^{+}u) for all λ ∈ ℝ, or equivalently, T X_{λ}u ⊂ X_{λ}T^{+}u for all λ ∈ ℝ. This reduces the proof to showing that

X_{0}T^{+}u ⊂ T X_{0}u.  (6.12)

The problem here is that we do not know much about the left-hand side of (6.12). To get a handle on this, we introduce the distance function d defined by

d(x) = −d(x, X_{0}u) = − inf_{y∈X_{0}u} |x − y|,

and to avoid the trivial case, we assume that the compact set X_{0}u is not empty. Clearly, X_{0}u = X_{0}d. The functions d and u satisfy the hypotheses of Lemma 7.7, so we know that there is a continuous contrast change h such that h(0) = 0 and d ≥ h(u). Then h(u) ∈ F_{c}^{+}, and since T^{+} is monotone and contrast invariant, it is not difficult to see that

X_{0}T^{+}u = X_{0}h(T^{+}u).

Thus we have

X_{0}T^{+}u = X_{0}h(T^{+}u) = X_{0}T^{+}(h(u)) ⊂ X_{0}T^{+}d = X_{0}T d.  (6.13)

On the other hand, we have

T X_{0}u = T X_{0}d = X_{0}T d,  (6.14)

and together (6.13) and (6.14) prove (6.12).

Recall that in section 6.4 we mentioned in connection with the maxima killer T_{a} a theory that is dual to the one we have developed. This dual theory is based on lower level sets X_{λ}^{−}u = {x | u(x) ≤ λ}, whereas the theory we have developed is based on upper level sets X_{λ}^{+}u = X_{λ}u = {x | u(x) ≥ λ}. The operator T in Theorem 7.12 is defined on all continuous function on ℝ^{N∗} or on 𝕋^{N}. This led to a set operator T defined on all compact subsets of ℝ^{N∗} or 𝕋^{N}. We used T to extend T to T^{+} defined on all functions u of the form u(x) = sup{λ | x ∈ X_{λ}}, where (X_{λ})_{λ∈ℝ} is a closed descending chain in the domain of T , that is, a closed descending chain of compact sets. We can also use T to extend T to T^{−} defined on all functions u of the form

u(x) = inf{λ | x ∈ X_{λ}},  (6.15)

where (X_{−λ})_{λ∈ℝ} is the closed descending chain of compact sets that generated the u.s.c. function −u. These are the l.s.c. functions whose lower level sets are compact. Then T^{−} is defined on the l.s.c. function u by

T^{−}u(x) = inf{λ | x ∈ T X_{λ}},  (6.16)

6.5. A WHO’S WHO OF MONOTONE OPERATORS 119

where (X_{−λ})_{λ∈ℝ} generated −u. We leave it as an exercise to show that T^{−}u = −T (−u).

Exercise 6.12. Show that defining T^{−} on l.s.c. functions of the form (6.15) using (6.16) is equivalent to defining T^{−} by T^{−}u = −T (−u).

The duality between upper and lower semicontinuous functions that have, respectively, upper and lower compact level sets, leads immediately to the dual of Theorem 7.12, which we state as a corollary.

Corollary 6.14. With the same hypotheses as Theorem 7.12, T can be extended to a contrast-invariant monotone operator T^{−} on F_{c}^{−}, the set of all lower semicontinuous functions on ℝ^{N∗} (or C-periodic functions on ℝ^{N}) whose lower level sets are compact. Furthermore, for u ∈ F_{c}^{−} and all λ ∈ ℝ

X_{λ}^{−}T^{−}u = T X_{λ}^{−}u.

Exercise 6.13. Let T be a contrast-invariant monotone operator defined on F. Assume that u ∈ F implies that 1_{X_{λ}u} ∈ F and that F is invariant under upper semicontinuous contrast changes. Prove that if T commutes with all thresholds, that is, if T γ_{λ}(u) = γ_{λ}(T u) for all λ ∈ ℝ, then T g(u) = g(T u) for all upper semicontinuous contrast changes g. Hint: Show that T g(u) and g(T u) have the same level sets. For this, it is a notational convenience to introduce the associated set operator T .

Figure 7.1 illustrates a common problem in vision. Looking at the chessboard we ask, Are there black squares on a white background or are there white squares on a black background? Saying that the background is white means that the white area is connected and closed. If black is coded with zero and white is coded with 255, then this means that the image is being interpreted as being upper semicontinuous. If we interpret the image as being white squares on a black background, then we are interpreting it as a lower semicontinuous image. If we interpret it as being u.s.c. and apply T^{+}, we get one result. We get a quite different result if we make the other interpretation and apply T^{−}.

6.5 A Who’s Who of monotone operators

We wish to summarize the main results about monotone function and set operators that have been developed beginning in Chapter 3 and continuing through Chapter 6 and the present chapter, and we would like to do this in a compact graphic style that can be used as a reference. With this in mind, we abbreviate the often-appearing phrase “almost everywhere, except possibly for at most a countable number of λ’s” by writing “aeecλ’s.” All of the function and set operators are assumed to be monotone. The domain of T is denoted by F and the domain of T is denoted by L. Recall also that an assumption about T or T entails the related assumption about its domain of definition.

As we have tried to do in elsewhere, we focus either on a function operator T or on a set operator T , which is another way of saying that we begin with T or we begin with T . This reflects what happens in practice. It is often easier and more intuitive to define an image operator as a set operator. We have already encountered this in the extrema killer, and this will be the case when we introduce the median filter; there will be other examples. Other image operators appear first as differential operators on a set of continuous functions.

120CHAPTER 6. CONTRAST-INVARIANT MONOTONE OPERATORS AS SUP-INF OPERATORS

Figure 6.1: The chessboard dilemma. Left: a chessboard image. Next: result with the finite difference scheme for the curvature motion, applied up to a fixed scale (Chapter 23). The creation of a new gray level proves that the scheme is not fully contrast invariant. Indeed, by Proposition 6.3, a contrast-invariant operator does not create new levels. The observed new gray level corresponds to an average of the original ones. The next two images are obtained by the Evans– Spruck extension of the curvature motion, first under the assumption that the image is u.s.c. and second under the assumption that it is l.s.c. The schemes are fully contrast invariant in both cases and are extensions of the curvature motion as specified in Theorem 7.12 and Corollary 7.13.

This is the case, for example, of the curvature motion. In this setting, we go from a function operator T on continuous functions to a set operator T on compact sets. The theory we have developed gives us the freedom to move between (T, F) and (T , L).

Case 1: Given a set operator T defined on L.

A_{T} (Basic assumptions): ℝ^{N}, ∅ ∈ L. (X_{λ})_{λ∈ℝ} ∈ L implies ⋂_{μ<λ} X_{μ} ∈ L. T ∅ = ∅, T ℝ^{N} = ℝ^{N}.

⟹^{Prop. 6.10} T u(x) := sup{λ | x ∈ T X_{λ}},

where (X_{λ})_{λ∈ℝ} generates u. T is contrast invariant. T X_{λ}u = X_{λ}T u

aeecλ’s.
A_{T} plus T u.s.c. on L.  ⟹^{Thm. 6.12} T X_{λ}u = X_{λ}T u for all λ ∈ ℝ.
A_{T} plus T translation invariant. ⟹^{Thm. 7.1} {T X = \{x | ∃B ∈ B : x + B ⊂ X\}, \atop B := \{X | 0 ∈ T X, X ∈ L\}.}

Case 2: Given a function operator T defined on F.

A_{T} (Basic assumptions): u ∈ F implies 1_{X_{λ}}u ∈ F. T contrast invariant with respect to G_{Lip}.

⟹^{Prop 6.8} T u(x) = sup{λ | x ∈ T X_{λ}u} a.e.,

T X_{λ}u := X_{1}T 1_{X_{λ}u}, L := {X_{λ}u}.
T X_{λ}u = X_{λ}T u aeecλ’s.
T satisfies A_{T}.

A_{T} plus T 1_{X_{λ}u} = 1_{X_{λ}T u}.

⟹^{Thm. 6.12}

The results above hold for all x ∈ ℝ^{N} and all λ ∈ ℝ.

A_{T} ⟹^{Thm. 7.3} T u(x) = sup_{B∈B_{x}} inf_{y∈B} u(y)

a.e., B_{x} := {X | x ∈ T X, X ∈ L}.

6.6. COMMENTS AND REFERENCES 121

A_{T} plus T translation invariant.

⟹^{Thm. 7.3}
T u(x) = sup_{B∈B_{0}} inf_{y∈B} u(y) a.e.,
B_{0} := {X | 0 ∈ T X, X ∈ L}.

Case 3: Given a set B ⊂ P(ℝ^{N}).

D_{T} (Definition): T X := {x | ∃B ∈ B : x + B ⊂ X}

⟹^{Thm. 7.1}

T monotone and translation

invariant on P(ℝ^{N}).

D_{T} (Definition): T u(x) = sup_{B∈x+B} inf_{y∈B} u(y)

⟹^{Thm. 7.3}

T monotone, translation and contrast invariant on u : ℝ^{N} → ℝ.

D_{T} plus condition in Prop. 7.2. ⟹^{Prop. 7.2} T u.s.c. on L.

Case 4: Given T defined on the continuous functions on 𝕋^{N} or ℝ^{N∗}.

T can be extended uniquely to the functions 1_{X} where X is any compact subset of 𝕋^{N} or ℝ^{N∗} (Theorem 7.8, Evans–Spruck). The set operator T defined by T X = X_{1}T 1_{X} can be used to extend T to all upper semicontinuous functions on 𝕋^{N} or ℝ^{N∗} that have compact level sets. Each of these extensions preserves commutation.

6.6 Comments and references

The formalism presented in this chapter is due to Matheron [194] in the case of set operators and to Serra [242] and Maragos [184] in the case of function operators. Serra’s formalism is actually more general than the one presented here; it will be developed in Chapter 15, which is about “nonflat” morphology. Our presentation relating the sup-inf form of the operator directly to contrast invariance and establishing the full equivalence between sup-inf operators and contrast-invariant monotone operators is original. The mysterious “set of structuring elements” has received a great deal of attention in the literature. Here are a few references: on finding the right set of structuring elements [234, 252]; on simplifying them [241]; on decomposing them into simpler ones as one does with linear filters [215, 281, 282]; on reducing the number [227].

Chapter 7

Erosions and Dilations

We are going to study in detail two of the simplest operators of mathematical morphology. These are the erosions and dilations. In fact, there will be essentially four operators: two set operators and the two related function operators. These operators will depend on a scale parameter t. We will also study the underlying PDEs ∂u/∂t = c|Du|, where c = 1 for dilations and c = −1 for erosions.

7.1 Erosions and dilations as multiscale contrastinvariant monotone operators

We saw in the last chapter that every contrast-invariant monotone function operator has a sup-inf and an inf-sup representation. We also saw that it was possible to define a contrast-invariant function operator in terms of a given set of structuring elements. This is the point of view we take here, and furthermore, we assume that the set of structuring elements B has the simplest possible form, namely, B = {B}. We then consider the two operators

T_{d}u(x) = sup_{y∈x+B} u(y) and T_{e}u(x) = inf_{y∈x+B} u(y).

The operator T_{d} is called a dilation, and T_{e} is called an erosion. These function operators have associated set operators, and in fact we are going to define the set operators first.

Definition 7.1. Let B be a subset of ℝ^{N} and let t ≥ 0 be a scale parameter. The set operators D_{t} and E_{t} are defined on subsets X ∈ P(ℝ^{N}) by

D_{t}X = X + tB = {z | z = x + y, x ∈ X, y ∈ tB} = ⋃_{y∈tB}(X + y), (7.1)

and

E_{t}X = {x | x + tB ⊂ X} = ⋂_{y∈−tB}(X + y).  (7.2)

D_{t}X is called the dilation of X with structuring element B at scale t. E_{t}X is called the erosion of X with structuring element B at scale t.

123 124 CHAPTER 7. EROSIONS AND DILATIONS

Figure 7.1: Dilation of a set. Left to right: A set; its dilation by a ball of radius 20; the difference set.

The operators D_{t} and E_{t} are in a certain sense the inverse of each other. This is clearly the case, for example, if B = {x_{0}}. Then D_{t} is just translation by tx_{0}, and E_{t} = D_{t}^{−1} is translation by −tx_{0}. If B is the open ball centered at zero with radius one, then D_{t}X is the set of all points whose distance from X is less than t, or the t-neighborhood of X. When B is symmetric with respect to zero, the operator D_{t}E_{t} is called an opening at scale t and E_{t}D_{t} is called a closing at scale t. These names have a topological origin. If B is the open ball centered at zero with radius one, then the opening at scale t of a set X is the union of all balls with radius t contained in X. The interior of X is the union of all open balls contained in X; it is also the largest open set contained in X. If we call the interior map T^{∘}X = X^{∘} (see Exercise 7.3) the opening, then an opening at scale t appears as a qualified opening. The topological statement “the closure of the complement of X is the complement of the interior of X” has its counter part for openings and closings at scale t, as shown by the next exercise. The actions of erosions and dilations are illustrated in Figures 8.1, 8.2, and 8.5; actions of openings and closings are illustrated in Figures 8.3, 8.4, 8.6, 8.7, and 8.8.

Exercise 7.1.

(i) Let B = {x | |x| < 1}. Show that D_{t}E_{t}X is the union of all open balls with radius t contained in X.

(ii) Let B be any structuring element that is symmetric with respect to zero. Write

7.1. EROSIONS AND DILATIONS AS MULTISCALE CONTRAST-INVARIANT MONOTONE OPERATORS125

X^{c} = ℝ^{N} \ X. Show that D_{t}X^{c} = (E_{t}X)^{c}. Use this to show that E_{t}D_{t}X^{c} = (D_{t}E_{t}X)^{c}.

We are going to define the function operators that correspond to D_{t} and E_{t}, but first we represent these set operators in the form given by equation (6.1). This will give us more direct access to results in Chapter 7. For this, we define B′ = {{−x} | x ∈ B}. Then,

D_{t}X = ⋃_{C∈tB′} ⋂_{y∈C}(X − y) = ⋃_{y∈tB}(X + y),  (7.3)

and

E_{t}X = ⋃_{C∈tB} ⋂_{y∈C}(X − y) = ⋂_{y∈−tB}(X + y).  (7.4)

We now have at hand some of the machinery of Chapter 7. In particular, we can apply Proposition 7.2 and Exercise 7.5 to E_{t} and conclude that E_{t} is upper semicontinuous on P(ℝ^{N}). On the other hand, we have evidence from Exercises 7.7 that D_{t} cannot be u.s.c. without restrictions on B and L. The first restriction we make is that both D_{t} and E_{t} map measurable sets into measurable sets. Without this assumption, or an assumption that implies this, much of the theory developed in Chapters 3, 6, and 7 is not available to us. Thus, we are going to assume that D_{t} and E_{t} are defined on a family of measurable sets L that is closed under descending chains. We further assume that B is such that D_{t}X and E_{t}X are measurable when X ∈ L. Now let F be the functions u defined by u(x) = sup{λ | x ∈ X_{λ}}, where (X_{λ})_{λ∈ℝ} is a closed descending chain in L.

At this point there are two ways to proceed: We can define the corresponding functions operators directly as T_{d} and T_{e}, or we can define them by superposition. We take the former definitions and then show their connection with superposition in the next theorem.

Definition 7.2. For u ∈ F, define D_{t}u by

D_{t}u(x) = sup_{y∈tB} u(x − y)  (7.5)

and define E_{t}u by

E_{t}u(x) = inf_{y∈−tB} u(x − y).  (7.6)

Theorem 7.3. For u ∈ F,

D_{t}u(x) = sup{λ | x ∈ D_{t}X_{λ}u} for a.e. x ∈ ℝ^{N},  (7.7)

and

D_{t}X_{λ}u = X_{λ}D_{t}u  (7.8)

almost everywhere, except possibly for at most a countable number of λ’s. Similarly,

E_{t}u(x) = sup{λ | x ∈ E_{t}X_{λ}u} for all x ∈ ℝ^{N},  (7.9)

and

E_{t}X_{λ}u = X_{λ}E_{t}u  (7.10)

for all λ ∈ ℝ.

126 CHAPTER 7. EROSIONS AND DILATIONS

Figure 7.2: Erosion of a set. Left to right: A set; its erosion by a ball of radius 20; the difference set.

Proof. Beginning with D_{t}, let D̃_{t}u(x) = sup{λ | x ∈ D_{t}X_{λ}u}. Then we know from equation (7.3) and Proposition 7.6 that

D̃_{t}u(x) = sup_{C∈x+tB′} inf_{y∈C} u(x)  (7.11)

for a.e. x ∈ ℝ^{N}. The right-hand side of (7.11) is easily seen to be the definition of D_{t}, so we have D̃_{t}u(x) = D_{t}u(x) for a.e. x ∈ ℝ^{N}. This proves (7.7). This also implies that X_{λ}D̃_{t}u = X_{λ}D_{t}u almost everywhere. By Proposition 6.10, X_{λ}D̃_{t}u = D_{t}X_{λ}u almost everywhere, except possibly for at most a countable number of λ’s, and this proves (7.8).

The proofs of equations (7.9) and (7.10) follow the same arguments, except in this case the operator E_{t} is u.s.c. and we invoke Theorem 6.12 and Corollary 7.4.

Exercise 7.2. Show that E_{t}(−u) = −D_{t}(u) if B is symmetric with respect to zero.

Exercise 7.3. Combine the results of Exercise 8.1 with the fact that E_{t} is u.s.c. to reveal a continuity property of D_{t}. Exercise 7.4. Let u be a continuous real-valued function defined on ℝ^{N} and consider the dilation D_{t} associated with B = {x | |x| < 1}. Check to be sure that D_{t} and E_{t} map measurable sets to measurable sets. Then Theorem 8.3 applies, and we know that

7.1. EROSIONS AND DILATIONS AS MULTISCALE CONTRAST-INVARIANT MONOTONE OPERATORS127

Figure 7.3: Opening of a set as curvature threshold from above. Left to right: A set X; its opening by a ball of radius 20; the difference set. This opening transforms X into the union of all balls of radius 20 contained in it. The resulting operation can be understood as a threshold from above of the curvature of the set boundary.

D_{t}X_{λ}u = X_{λ}D_{t}u almost everywhere, except possibly for at most a countable number of λ’s. Show that D_{t}X_{λ}u is open and that X_{λ}D_{t}u is closed. Deduce from this that the relation can hold for all x ∈ ℝ^{N} only if both sets are equal to ℝ^{N} or are empty.

We have by now seen plenty of evidence that D_{t} is not in general u.s.c., or equivalently, that D_{t} does not commute with thresholds. There are, however, two simple restrictions on B and L that ensure the upper semicontinuity of D_{t} on L.

Proposition 7.4. If either of the two conditions hold, then D_{t} is u.s.c. on L.

(i) B is compact and L is the set of closed subsets of ℝ^{N}.

(ii) B is closed and L is the set of compact subsets of ℝ^{N}.

Proof. We are going to use Proposition 7.2. Thus assume that (i) holds and that (X_{n})_{n∈ℕ} is a decreasing sequence in L such that ⋂_{n∈ℕ} D_{t}X_{n} ≠ ∅. Let x be any element in ⋂_{n∈ℕ} D_{t}X_{n}. Then x ∈ D_{t}X_{n} for all n ∈ ℕ, and so by definition there is a b_{n} ∈ B such that x − tb_{n} ∈ X_{n}. This means that (x − tB) ∩ X_{n} is not empty for all n. Since (x − tB) ∩ X_{n} is a decreasing sequence of compact 128 CHAPTER 7. EROSIONS AND DILATIONS

Figure 7.4: Closing of a set as a curvature threshold from below. Left to right: A set X; its closing by a ball of radius 20; the difference set. The closing of X is just the opening of X^{c}. It can be viewed as a threshold from below of the curvature of the set boundary.

sets, the intersection ⋂_{n∈ℕ}((x − tB) ∩ X_{n}) = (x − tB) ∩ ⋂_{n∈ℕ} X_{n} is not empty. Thus there is a b ∈ B such that x − tb ∈ ⋂_{n∈ℕ} X_{n}, and this, by Proposition 7.2, proves the result for (i). The same argument proves the result for (ii).

The two conditions (i) and (ii) above are of course that same in case we replace ℝ^{N} with 𝕋^{N} or ℝ^{N∗}, which are the most interesting cases for image analysis.

If B is not symmetric with respect to zero, then there are in general two other function operators related naturally to D_{t} and E_{t}. These are the operators D_{t}^{−} and E_{t}^{−} defined by

D_{t}^{−}u(x) = −D_{t}(−u)(x) = inf_{y∈tB} u(x − y)  (7.12)

and

E_{t}^{−}u(x) = −E_{t}(−u)(x) = sup_{y∈−tB} u(x − y).  (7.13)

Note that D_{t}^{−} is an erosion and E_{t}^{−} is a dilation. When B is symmetric with respect to zero, we have D_{t}^{−} = E_{t} and E_{t}^{−} = D_{t}.

7.1. EROSIONS AND DILATIONS AS MULTISCALE CONTRAST-INVARIANT MONOTONE OPERATORS129

Proposition 7.5. Assume that B is a compact subset of ℝ^{N} and letL be the family of closed subsets of ℝ^{N}. Let F^{+} denote the set of upper semicontinuous function on ℝ^{N} and let F^{−} denote the set of lower semicontinuous function on ℝ^{N}. Then for u ∈ F^{+}, D_{t}X_{λ}u = X_{λ}D_{t}u and E_{t}X_{λ}u = X_{λ}E_{t}u. If u ∈ F^{−}, then D_{t}X_{μ}^{−}u = X_{μ}^{−}D_{t}^{−}u and E_{t}X_{μ}^{−}u = X_{μ}^{−}E_{t}^{−}u.

Proof. We know from Proposition 8.4 that D_{t} is u.s.c. on L. (That E_{t} is u.s.c. on L follows from Proposition 7.2.) The functions u defined by u(x) = sup{λ | x ∈ X_{λ}} where (X_{λ})_{λ∈ℝ} is a closed descending chain in L are exactly the u.s.c. functions F^{+}, and the results for u ∈ F^{+} follow directly from Theorem 6.12. The results for u ∈ F^{−} are quickly obtained by applying the first results to −u ∈ F^{+}.

We say that the family of dilations {D_{t} | t > 0} associated with a structuring element B is recursive if D_{t}D_{s} = D_{t+s} for all s, t > 0, and similarly for {E_{t} | t > 0}. (A recursive family is also called a semigroup.) Being recursive is a very desirable property for any family of scaled operators used for image analysis. For example, we have already seen in Chapters 2 the interest in studying that asymptotic behavior of operators T_{h} that satisfy (T_{h})^{n} = T_{hn}. Having T_{t} = (T_{t/n})^{n} is also useful for practical computations. {D_{t} | t > 0} and {E_{t} | t > 0} will be recursive if and only if B is convex, but before proving this result we need the condition for B to be convex given in the next lemma.

Lemma 7.6. B is convex if and only if (s + t)B = sB + tB for all s, t ≥ 0.

Proof. Assume that B is convex and take x, y ∈ B. Then (s/(s+t))x+(t/(s+ t))y ∈ B, and this implies that sB + tB ⊂ (s + t)B. The inclusion in the other direction is obvious. Now assume that (s + t)B = sB + tB and that x, y ∈ B. Then one can find z ∈ B such that (s + t)z = sx + ty. This implies that (s/(s + t))x + (t/(s + t))y ∈ B. Since s, t ≥ 0 are arbitrary, it follows that B is convex.

Proposition 7.7. The dilations D_{t} and D_{t} and the erosions E_{t} and E_{t} are recursive if and only the structuring element B is convex.

Proof. We begin with the dilations D_{t}. Using the definition (7.1), we have

D_{t}D_{s}X = (X + sB) + tB = X + sB + tB

and

D_{s+t}X = X + (s + t)B.

If (t + s)B = tB + sB, then clearly D_{t}D_{s}X = D_{s+t}X. Conversely, if D_{t}D_{s}X = D_{s+t}X, then by taking X = {0} we see that (t + s)B = tB + sB. This implies immediately that the dilations D̃_{t} defined by superposition are recursive. However, since the D_{t} are not necessarily u.s.c., we can only conclude that D_{t}D_{s}u(x) = D_{s+t}u(x) for a.e. x ∈ ℝ^{N}. Thus, we are going to prove that D_{t}D_{s} = D_{s+t} directly from Definition 8.2.

D_{t}D_{s}u(x) = sup_{z∈tB} D_{s}u(x−z) = sup_{z∈tB}( sup_{y∈sB} u(x−z−y)) = sup_{a∈B,b∈B} u(x−(sa+tb)),

130 CHAPTER 7. EROSIONS AND DILATIONS

Figure 7.5: Erosion and dilation of a natural image. First row: a sea bird image and its level lines for all levels multiple of 12. Second row: an erosion with radius 4 has been applied. On the right, the resulting level lines where the circular shape of the structuring element (a disk with radius 4) appears around each local minimum of the original image. Erosion removes local maxima (in particular, all small white spots) but expands minima. Thus, all dark spots, like the eye of the bird, are expanded. Third row: the effect of a dilation with radius 4 and the resulting level lines. We see how local minima are removed (for example, the eye of the bird) and how white spots on the tail expand. Here, in turn, circular level lines appear around all local maxima of the original image.

and

D_{s+t}u(x) = sup_{y∈(s+t)B} u(x − y) = sup_{c∈B} u(x − (s + t)c).

If (t + s)B = tB + sB, then D_{t}D_{s}u(x) = D_{s+t}u(x) for all x ∈ ℝ. Conversely, if D_{t}D_{s}u(x) = D_{s+t}u(x), then by taking u = 1_{{0}} we have see that (t + s)B = tB + sB.

A similar argument, this time taking u = −1_{{0}}, shows that E_{t}E_{s} = E_{s+t} if and only if (t + s)B = tB + sB. Since E_{t} is u.s.c., it follows directly from the relation E_{t}X_{λ}u = X_{λ}E_{t}u that E_{t}E_{s} = E_{s+t} if and only if (t + s)B = tB + sB.

7.2. THE PDES ASSOCIATED WITH EROSIONS AND DILATIONS 131

Figure 7.6: Openings and closings of a natural image. First row: the original image and its level lines for all levels multiple of 12. Second row: an opening with radius 4 has been applied. Third row: a closing with radius has been applied. We can recognize the circular shape of the structuring element in the level lines displayed on the right.

7.2 The PDEs associated with erosions and dilations

As indicated in the introduction to the chapter, scaled dilations and erosions are associated with the equations ∂u/∂t = ±|Du|. To explain this connection, we begin with a bounded convex set B that contains the origin, and we define the gauge ∥ · ∥_{B} on ℝ^{N} associated with B by ∥x∥_{B} = sup_{y∈B}(x · y). If B is a ball centered at the origin with radius one, then ∥ · ∥_{B} is the usual Euclidean norm, which we write simply as | · |.

Proposition 7.8 (Hopf–Lax formula [94, 172]). Assume that B is a bounded convex set in ℝ^{N} that contains the origin. Given u_{0} : ℝ^{N} → ℝ, define u : ℝ^{+} × ℝ^{N} → ℝ by u(t, x) = D_{t}u_{0}(x). Then u satisfies the equation

\frac{∂u}{∂t} = ∥Du∥_{−B}

at each point (t, x) where u has continuous derivatives in t and x. The same result hold when D_{t} is replaced with E_{t} and the equation is replaced with ∂u/∂t = 132 CHAPTER 7. EROSIONS AND DILATIONS

Figure 7.7: Denoising based on openings and closings. First row: scanned picture of the word “operator” with black dots and a black line added; a dilation with a 2 × 5 rectangle; an erosion with the same structuring element applied to the middle image. The resulting operator is a closing. Small black structures are removed by such a process. Second row: the word “operator” with a white line and white dots inside the letters; erosion with a rectangle 2 × 5; a dilation with the same structuring element applied to the middle image. The resulting operator is an opening. This time, small white structures are removed.

−∥Du∥_{−B}.

Proof. We begin by proving the result for D_{t} at t = 0. Thus assume that u_{0} is C^{1} at x. Then

u_{0}(x − y) − u_{0}(x) = −Du_{0}(x) · y + o(|y|),

and we have

u(h, x) − u(0, x) = sup_{y∈hB}(−Du_{0}(x) · y + o(|y|)).

Since B is bounded, the term o(|y|) is o(|h|) for y ∈ hB, and we have

u(h, x) − u(0, x) = h sup_{z∈B}((−Du_{0}(x) · z) + o(|h|).

We can divide both sides by h and pass to the limit as |h| → 0 to obtain

\frac{∂u}{∂t}(0, x) = ∥Du_{0}(x)∥_{−B},

7.3. COMMENTS AND REFERENCES 133

Figure 7.8: Examples of denoising based on opening or closing, as in Figure 7.7. Perturbations made with both black and white lines or dots have been added to the “operator” image. First column, top to bottom: original perturbed image; erosion with a 1 × 3 rectangle; then dilation with the same structuring element. (In other words, opening with this rectangle.) Then a dilation is applied with a rectangle 3 × 1, and finally an erosion with the same rectangle. Second column: The same process is applied, but with erosions and dilations exchanging their roles. It does not work so well because closing expands white perturbations and opening expands black perturbations. These operators do not commute. See Figure ??, where an application of the median filter is more successful.

which is the result for t = 0. For an arbitrary t > 0, we have D_{t+h} = D_{t}D_{h} = D_{h}D_{t}, and we can write

u(t + h, x) − u(t, x) = D_{h}u(t, ·)(x) − u(t, x).

By repeating the argument made for t = 0 with u_{0} replaced with u(t, ·), we arrive at the general result. The proof for E_{t} is similar.

7.3 Comments and references

Erosions and dilations. Matheron introduced dilations and erosions as useful tools for set and shape analysis in his fundamental book [194]. A full account 134 CHAPTER 7. EROSIONS AND DILATIONS

of the properties of dilations, erosions, openings, and closings, both as set operators and function operators, can be found in Serra’s books [242, 244]. We also suggest the introductory paper by Haralick, Sternberg, and Zhuang [130] and an earlier paper by Nakagawa and Rosenfeld [204]. An axiomatic algebraic approach to erosions, dilations, openings, and closings has been developed by Heijmans and Ronse [133, 229]. We did not develop this algebraic point of view here. The obvious relations among the dilations and erosions of a set and the distance function have been exploited numerically in [138], [162], and [247]. The skeleton of a shape can be defined as the set of points where the distance function to the shape is singular. A numerical procedure for computing the skeleton this way is proposed in [163].

The PDEs. The connection between the PDEs ∂u/∂t = ±|Du| and multiscale dilations and erosions comes from the work of Lax, where it is used to give stable and efficient numerical schemes for solving the equations [172]. Rouy and Tourin have shown that the distance function to a shape is a viscosity solution of 1 − |Du| = 0 with the null boundary condition (Dirichlet condition) on the boundary of the shape. To define efficient numerical schemes for computing the distance function, they actually implement the evolution equation ∂u/∂t = 1 − |Du| starting from zero and with the null boundary condition on the boundary of the shape. The fact that the multiscale dilations and erosions can be computed using the PDEs ∂u/∂t = ±|Du| has been rediscovered or revived, thirty years after Lax’s work, by several authors: Alvarez et. al. [10], van den Boomgaard and Smeulders [260], Maragos [185, 186]. For an implementation using curve evolution, see [237]. Curiously, the link between erosions, dilations, and their PDEs seems to have remained unknown or unexploited until 1992. The erosion and dilation PDEs can be used for shape thinning, which is a popular way to compute the skeleton. Pasquignon developed an erosion PDE with adaptive stopping time that allows one to compute directly a skeleton that does not look like barbed wire [216].

Chapter 8

Median Filters and Mathematical Morphology

This entire chapter is devoted to median filters. They are among the most characteristic and numerically efficient of the contrast-invariant monotone operators. The denoising effects of median filters are illustrated in Figures 9.1 and 9.2; the smoothing effect of a median filter is illustrated in Figure 9.3.

As usual, there will be two associated operators, a set operator and a function operator. The only restrictions we make are the measurability assumptions, which are by now familiar: The domain of definition of the set operators will always be a family of measurable sets L that is closed under descending chains. Similarly, we require any domain F of the function operators to consist of measurable functions.

All of the median operators (or filters) will be defined in terms of a nonnegative measurable weight function k : ℝ^{N} → [0, +∞) that is normalized:

∫_{ℝ^{N}} k(y) dy = 1.

The k-measure of a subset B ⊂ ℝ^{N} is denoted by |B|_{k} and defined by

|B|_{k} = ∫_{B} k(y) dy = ∫_{ℝ^{N}} k(y)1_{B}(y) dy.

Clearly, 0 ≤ |B|_{k} ≤ 1. We have already encountered weight functions, both specifically and generally: The operator M_{h} in section 1.1 and the Gaussian G_{t} are specific examples, while the operators introduced in Definition 2.1 (although they were not required to be positive) are general examples. By far the simplest example is the generalization of M_{h} to N dimensions. This is given by the function k = c_{N}^{−1}(r)1_{B(0,r)}, where B(0, r) denotes the ball of radius r centered at the origin and c_{N}(r) is the Lebesgue measure of B(0, r).

8.1 The median operators Med_{k} and Med_{k}

As we did for the erosions and dilations, we first define the set operators.

135 136CHAPTER 8. MEDIAN FILTERS AND MATHEMATICAL MORPHOLOGY

Definition 8.1. Let X be a measurable subset of ℝ^{N} and let k be a weight function. The median set of X weighted by k, which we denote by Med_{k}X, is defined by

Med_{k}X = \{x | |X − x|_{k} ≥ \frac{1}{2}\}.  (8.1)

Note that we have already encountered this idea in section 2.4. Koendernik and van Doorn defined the dynamic shape of X at scale t to be the set of x such that G_{t} ∗ 1_{X}(x) ≥ 1/2. Their dynamic shape is, in our terms, a Gaussianweighted median filter.

To gain some intuition about median filters, we suggest considering the weight k defined on ℝ^{2} by k = (1/πr^{2})1_{B(0,r)}. Then x belongs to Med_{k}X if and only if the Lebesgue measure of X ∩ B(x, r) is greater than or equal to half the measure of B(0, r). Intuitively, x ∈ Med_{k}X if points of X are in the majority around x.

We are going to define the function operator Med_{k} associated with the set operator Med_{k}, but as preparation we will show that Med_{k} is upper semicontinuous on the set of measurable sets L.

Lemma 8.2. The set operator Med_{k} is upper semicontinuous on the measurable sets L.

Proof. Given a descending sequence (X_{n})_{n∈ℕ} in L, we must show that

Med_{k} ⋂_{n∈ℕ} X_{n} = ⋂_{n∈ℕ} Med_{k}X_{n}.

Since Med_{k} is clearly monotone, it is always true that Med_{k} ⋂_{n∈ℕ} X_{n} ⊂ ⋂_{n∈ℕ} Med_{k}X_{n}. To prove the other inclusion, assume that x ∈ ⋂_{n∈ℕ} Med_{k}X_{n}. Then by the definition of Med_{k}, |X_{n} − x| ≥ 1/2 for all n ∈ ℕ. Since X_{n} − x ↓ ⋂_{n∈ℕ}(X_{n} − x), we know from the countable additivity of Lebesgue measure that |X_{n} − x|_{k} ↓ | ⋂_{n∈ℕ}(X_{n} − x)|_{k}. This means that | ⋂_{n∈ℕ}(X_{n} − x)|_{k} ≥ 1/2, and hence that x ∈ Med_{k} ⋂_{n∈ℕ}(X_{n} − x).

Definition 8.3. The function operator Med_{k} is defined on the set of all measurable functions F by the superposition principle:

Med_{k}u(x) = sup{λ | x ∈ Med_{k}X_{λ}u}.

It is important to note that F, the set of all measurable functions u : ℝ^{N} → \overline{ℝ}, is exactly the set of all functions u defined by u(x) = sup\{λ | x ∈ X_{λ}\}, where (X_{λ})_{λ∈ℝ} is a closed descending chain in L, the set of all measurable subsets of ℝ^{N}. Expressed another way, the measurable functions are in one-to-one correspondence with the closed descending chains of measurable sets. Here is another useful observation, which we set as an exercise.

Exercise 8.1. If we represent Med_{k} by

Med_{k}X = {x | x + B ⊂ X, for some B ∈ B},  (8.2)

then B can be taken to be the set of all measurable sets B such that |B|_{k} = 1/2. (Note the similarity with Exercise 7.6.)

8.2. THE MEDIAN OPERATORS MED_{K}^{−} AND MED_{K}^{−} 137

Proposition 8.4. The median operator Med_{k} has the sup-inf representation

Med_{k}u(x) = sup_{B∈B} inf_{y∈x+B} u(y),  (8.3)

where B = {B | B ∈ L, |B|_{k} = 1/2}. Furthermore,

X_{λ}Med_{k}u = Med_{k}X_{λ}u  (8.4)

for all λ ∈ \overline{ℝ}.

Proof. We mention again that Med_{k} is clearly monotone. It is also clearly translation invariant. The proof of (8.4) is read directly from Theorem 6.12. Equation (8.4) implies that Med_{k} commutes with all thresholds γ_{λ}, that is, Med_{k}γ_{λ}(u) = γ_{λ}(Med_{k}u) for all λ ∈ ℝ. This implies by Corollary 7.4 that Med_{k} has the sup-inf representation

Med_{k}u(x) = sup_{B∈x+B_{0}} inf_{y∈B} u(y),

where B_{0} = {B | 0 ∈ Med_{k}B}. It is easy to see that B_{0} and B are equivalent sets of structuring elements (see Exercise 7.4), and thus we have (8.3).

We are going to consider three simple examples to gain some understanding about the action of Med_{k} on functions that have “flat” parts and jumps. The functions will be defined on ℝ, and k = (1/2)1_{[−1,1]}. This implies that |B|_{k} = 1/2 if and only if meas ([−1, 1] ∩ B) = 1. This is useful for computing the value of Med_{k}u(0) in the three examples.

Example (1): u(x) = x if x ≤ 0, and u(x) = 0 if x > 0. Then Med_{k}u(0) = 0, and the median value coincides with the supremum.

Example (2): u(x) = −1 if x ≤ 0, and u(x) = 1 if x > 0. Then Med_{k}u(0) = 1, and we notice that the operator Med_{k} as we have defined it favors the value +1, although −1 and +1 would seem to be equally good candidates.

Example (3): u(x) = 0 if x ≤ −1/4, u(x) = 1 if −1/4 < x < 1/3, and u(x) = 2 if x ≥ 1/3. Then Med_{k}u(0) = 1.

Notice that the median value in all three examples is different from the mean value on the neighborhood [−1, 1].

8.2 The median operators Med_{k}^{−} and Med_{k}^{−}

The median operator Med_{k}, as defined, is not invariant under “reverse contrast,” that is, it does not satisfy −Med_{k}u = Med_{k}(−u) for all measurable u. This is clear from Example (2) above, where Med_{k}u(0) = 1 and Med_{k}(−u)(0) = 1. As we have done elsewhere for other operators (the extrema killer, the Evans– Spruck extension, dilations, and erosions), we are going to define the “lower” forms of Med_{k} and Med_{k}. We define Med_{k}^{−} by

Med_{k}^{−}u(x) = inf_{|B|_{k}≥ \frac{1}{2}} sup_{y∈x+B} u(y).  (8.5)

138CHAPTER 8. MEDIAN FILTERS AND MATHEMATICAL MORPHOLOGY

Figure 8.1: Example of denoising with a median filter. Left to right: scanned picture of the word “operator” with perturbations and noise made with black or white lines and dots; the image after one application of a median filter with a circular neighborhood of radius 2; the image after a second application of the same filter. Compare with the denoising using openings and closings (Figure 7.8).

Then we clearly have the relation

Med_{k}^{−}u = −Med_{k}(−u),  (8.6)

which we have used previously as the definition for the lower form of a function operator. If we now define Med_{k}^{−} in terms of Med_{k}^{−} by Med_{k}^{−}X = X_{1}Med_{k}^{−}1_{X}, then using (8.6) and the relation 1_{X} + 1_{X^{c}} = 1 it is easily seen that

Med_{k}^{−}X = (Med_{k}X^{c})^{c}.  (8.7)

Since ∫_{X−x} k(y) dy + ∫_{X^{c}−x} k(y) dy = 1 for all x, we infer from (8.7) that

Med_{k}^{−}X = \{x | |X − x|_{k} > \frac{1}{2}\}.  (8.8)

Although Med_{k} and Med_{k}^{−} both map measurable sets into measurable sets, remarkably Med_{k} maps measurable sets into closed sets and Med_{k}^{−} maps measurable sets into open sets.

Lemma 8.5. If X is a measurable subset of ℝ^{N}, then Med_{k}X is closed and Med_{k}^{−}X is open.

8.2. THE MEDIAN OPERATORS MED_{K}^{−} AND MED_{K}^{−} 139

Proof. If x_{n} ∈ Med_{k}X, then |X−x_{n}|_{k} ≥ 1/2, which by definition of k-measure means that ∫_{X} k(y − x_{n}) dy ≥ 1/2. If x_{n} → x, then ∫_{X} k(y − x_{n}) dy → ∫_{X} k(y − x) dy by Proposition 1.2. Thus |X − x|_{k} ≥ 1/2, and x ∈ Med_{k}X. That Med_{k}^{−}X is open follows from (8.7).

Theorem 8.6. If u is a measurable function, then Med_{k}u is upper semicontinuous and Med_{k}^{−}u is lower semicontinuous.

Proof. If u is a measurable function on ℝ^{N}, then Med_{k}u(x) = sup{λ | x ∈ Med_{k}X_{λ}u} by definition. Since Med_{k} is upper semicontinuous, we know that the upper level sets of Med_{k}u are the sets Med_{k}X_{λ}u, which by Lemma 9.5 are closed. Thus Med_{k}u is upper semicontinuous. The relation (8.6) implies that Med_{k}^{−}u is lower semicontinuous.

Knowing this result, it is reasonable to ask, Are there conditions on k that ensure Med_{k} maps continuous functions into continuous functions? The next result shows that a quite general condition on k is sufficient to guarantee that not only is Med_{k}u continuous when u is continuous, but also that Med_{k} and Med_{k}^{−} agree on continuous functions.

We say that k is not separable if |B|_{k} ≥ 1/2 and |B′|_{k} ≥ 1/2 imply that \overline{B} ∩ \overline{B′} ≠ ∅.

Proposition 8.7.

(i) For every measurable function u, Med_{k}u ≥ Med_{k}^{−}u.

(ii) Assume that k is not separable. If u is continuous, then Med_{k}u = Med_{k}^{−}u and Med_{k}u is continuous.

Proof. Both operators are translation invariant, so without loss of generality we may assume that x = 0. To prove (i), let λ = Med_{k}u(0) = sup_{|B|_{k}≥1/2} inf_{y∈B} u(y). Take ε > 0 and consider the level set X_{λ+ε}u. Then inf_{y∈X_{λ+ε}} u(y) ≥ λ+ε. Thus |X_{λ+ε}u|_{k} < 1/2, since inf_{y∈B} < λ + ε for any set B such that |B| ≥ 1/2. Then |(X_{λ+ε}u)^{c}|_{k} ≥ 1/2, and by the definition of level sets, sup_{y∈(X_{λ+ε}u)^{c}} u(y) ≤ λ+ε. This implies that

inf_{|B|_{k}≥ \frac{1}{2}} sup_{y∈B} u(y) ≤ λ + ε.

Since ε > 0 was arbitrary, this proves (i).

The assumption that k is not separable implies that for all B and B′ having k-measure greater than or equal to 1/2, we have inf_{y∈\overline{B}} u(y) ≤ sup_{y∈\overline{B′}} u(y). If u is continuous, then we have inf_{y∈B} u(y) ≤ sup_{y∈B′} u(y). Since B and B′ were arbitrary except for the conditions |B|_{k} ≥ 1/2 and |B′|_{k} ≥ 1/2, the last inequality implies that

sup_{|B|_{k}≥ \frac{1}{2}} inf_{y∈B} u(y) ≤ inf_{|B′|_{k}≥ \frac{1}{2}} sup_{y∈B′} u(y).

From this last inequality and (i), we conclude that Med_{k}u = Med_{k}^{−}u when u is continuous. This means by Theorem 9.6 that Med_{k}u is both upper and lower continuous, which means that it is continuous.

140CHAPTER 8. MEDIAN FILTERS AND MATHEMATICAL MORPHOLOGY

Figure 8.2: Denoising based on a median filter. Left: an image altered on 40% of its pixels with salt and pepper noise. Right: the same image after three iterations of a median filter with a 3 × 3 square mask.

Exercise 8.2. Saying that k is not separable is a fairly weak assumption. It corresponds roughly to saying that the support of k cannot be split into two disjoint connected components each having k-measure 1/2. Show that if k is continuous and if its support is connected, then it is not separable. Exercise 8.3. Prove the following inequalities:

sup_{|B|_{k}≥ \frac{1}{2}} inf_{y∈x+B} u(y) ≥ sup_{|B|_{k}> \frac{1}{2}} inf_{y∈x+B} u(y) ≥ inf_{|B|_{k}≥ \frac{1}{2}} sup_{y∈x+B} u(y),

sup_{|B|_{k}≥ \frac{1}{2}} inf_{y∈x+B} u(y) ≥ inf_{|B|_{k}> \frac{1}{2}} sup_{y∈x+B} u(y) ≥ inf_{|B|_{k}≥ \frac{1}{2}} sup_{y∈x+B} u(y).

8.3 Discrete median filters and the ”usual” median value

We define a discrete median filter by considering a uniform discrete measure k defined on a finite number of points {x_{i} | i = 1, …, N}. This means that k is defined by |{x_{i}}|_{k} = 1, which is to say that Dirac masses δ_{i} exist at each point x_{i}, i ∈ {1, …, N}. We could normalize k, but this is not necessary, as will become clear. Translates of the points x_{i} create the discrete neighborhood that is used to compute the median value of a function u at a point x. We denote the 8.3. DISCRETE MEDIAN FILTERS AND THE ”USUAL” MEDIAN VALUE141

set of subsets of {1, …, N} by P(N) and the number of elements in P ∈ P(N) by card(P). Since card(P) = |P |_{k}, we will suppress the k-notation is favor of the more transparent “card(P),” but one should remember that the k measure is still there. Here are the definitions of the two discrete median filters:

Medu(x) = sup_{{P ∈P(N) \atop card(P)≥N/2}} inf_{i∈P} u(x − x_{i}),

Med^{−}u(x) = inf_{{P ∈P(N) \atop card(P)≥N/2}} sup_{i∈P} u(x − x_{i}).

When k was continuous, we could replace “|B|_{k} ≥ 1/2” with , “|B|_{k} = 1/2,” but this is not directly possible in the discrete case, since N/2 is not an integer if N is odd. To fix this, we define the function M by M(N) = N/2 if N is even and M(N) = (N/2) + (1/2) if N is odd. Now we have

Medu(x) = sup_{{P ∈P(N) \atop card(P)=M(N)}} inf_{i∈P} u(x − x_{i}),

Med^{−}u(x) = inf_{{P ∈P(N) \atop card(P)=M(N)}} sup_{i∈P} u(x − x_{i}).

The fact that we can replace “card(P) ≥ N/2” with “card(P) = M(N)” has been argued elsewhere for the continuous case; for the discrete case, it is a matter of simple combinatorics. Given any x, let y_{i} = u(x − x_{i}). After a suitable permutation of the i’s, we can order the y_{i} as follows: y_{1} ≤, ⋯, ≤ y_{M} ≤, ⋯ ≤ y_{N}. Then for N even,

{inf_{i∈P} y_{i} | card(P) ≥ N/2} = {inf_{i∈P} y_{i} | card(P) = M} = {y_{1}, …, y_{M+1}},
{sup_{i∈P} y_{i} | card(P) ≥ N/2} = {sup_{i∈P} y_{i} | card(P) = M} = {y_{M}, …, y_{N}},

and Medu(x) = y_{M+1} ≥ y_{M} = Med^{−}u(x). If N is odd, we have

{inf_{i∈P} y_{i} | card(P) ≥ N/2} = {inf_{i∈P} y_{i} | card(P) = M} = {y_{1}, …, y_{M}},
{sup_{i∈P} y_{i} | card(P) ≥ N/2} = {sup_{i∈P} y_{i} | card(P) = M} = {y_{M}, …, y_{N}},

and Medu(x) = Med^{−}u(x) = y_{M}. This shows that Med = Med^{−} if and only if N is odd. What we see here is the discrete version of Proposition 9.7. When N is odd, the measure is not separable, since two sets P and P′ with card(P) ≥ N/2 and card(P′) ≥ N/2 always have a nonempty intersection. In general, a median filter with an odd number of pixels is preferred, since Med = Med^{−} in this case.

This discussion shows that the definition of the discrete median filter Med corresponds to the usual statistical definition of the median of a set of data: If the given data consists of the numbers y_{1} ≤ y_{2} ≤ ⋯ ≤ y_{N} and N = 2n+1, them by definition, the median is y_{n+1}. In case N = 2n, the median is (y_{n} + y_{n+1})/2. In both cases, half of the terms are greater than or equal to the median and half of the terms are less than or equal to the median. The usual median minimizes 142CHAPTER 8. MEDIAN FILTERS AND MATHEMATICAL MORPHOLOGY

the functional ∑_{i=1}^{N} |y_{i} − y|. Exercise 9.5 shows how Med and Med^{−} relate to this functional.

The discrete median filters can also be defined in terms of a nonuniform measure k that places different weights on the points x_{i}. To see what this does, assume that the weights are integers k_{i}, so |{x_{i}}|_{k} = k_{i}. Then k has total mass ∑_{i=1}^{N} k_{i} = K, and the condition card(P) ≥ N/2 is replaced with |P |_{k} ≥ K/2. As before, let y_{i} = u(x − x_{i}) and display the data set as y_{1} ≤ y_{2} ≤ ⋯ ≤ y_{N}. Then Med_{k}u(x) = y_{j}, where j is the largest index such that k_{j} +⋯+k_{N} ≥ N/2. To see this, transform the original ordered sequence into the expanded ordered sequence

\underbrace{y_{1} = ⋯ = y_{1}}_{k_{1} terms} ≤ ⋯ ≤ \underbrace{y_{i} = ⋯ = y_{i}}_{k_{i} terms} ≤ ⋯ ≤ \underbrace{y_{N} = ⋯ = y_{N}}_{k_{N} terms} .  (8.9)

Then by the definition of j, y_{j} ∈ {inf_{i∈P} y_{i} | |P |_{k} ≥ K/2}, but y_{i} for i > j is not in this set. Thus, Med_{k}u(x) = y_{j}. Conversely, if Med_{k}u(x) = y_{j}, then y_{j} is the largest member of the set {inf_{i∈P} y_{i} | |P |_{k} ≥ K/2}. This implies that k_{j} + ⋯ + k_{N} ≥ N/2, but that k_{i} + ⋯ + k_{N} < K/2 for i > j.

If K is odd, what we have just done implies that Med_{k}^{−} = Med_{k}, and that Med_{k}u(x) is equal to the ordinary median of the ordered set (8.9). The following exercise completes this part of the theory.

Exercise 8.4. It is clear that Med_{k}^{−}u ≤ Med_{k}u, either by Proposition 9.7, or by the above discussion. Prove that Med_{k}^{−}u = Med_{k}u if and only if there is no subset of the numbers k_{1}, …, k_{N} whose sum is K/2. In particular, if K is odd, then Med_{k}^{−}u = Med_{k}u. Exercise 8.5. Let arginf_{m}g(m) denote the value of m, if it exists, at which g attains its infimum. Consider the N real numbers {x_{i} | i = 1, 2, …, N}.

(i) Show that

\frac{1}{N} ∑_{i=1}^{N} x_{i} = arginf_{m} ∑_{i=1}^{N}(x_{i} − m)^{2}.

(ii) Show that

Med^{−}(x_{i}) ≤ arginf_{m} ∑_{i=1}^{N} |x_{i} − m| ≤ Med(x_{i}).

(iii) Let k = 1_{B}, where B is set with Lebesgue measurable equal to one. Let Med_{B}u denote the median value of u in B, defined by Med_{B}u = Med_{k}u(0). Consider a bounded measurable function u defined on B. Show that

∫_{B} u(x) dx = arginf_{m} ∫_{B}(u(x) − m)^{2} dx

and that

Med_{B}^{−}u ≤ arginf_{m} ∫_{B} |u(x) − m| dx = \frac{Med_{B}^{−}u + Med_{B}u}{2} ≤ Med_{B}u.

(iv) Deduce from the above that the mean value is the best constant approximation in the L^{2} norm and that the median is the best constant approximation in the L^{1} norm.

Finally, we wish to show that the discrete median filter Med is a cyclic operator on discrete images. To be precise, we assume that the digital image 8.4. COMMENTS AND REFERENCES 143

u has been periodized so that we have a C-periodic digital image. Assume that the original image defined on he hypercube [0, 1]^{N} contained n pixels. Then the total number of gray levels is bounded by n. Since a median filter Med is contrast invariant, the grey levels of Medu are among the original ones. The same is true for the kth iteration Med^{k}u. Since that maximum number of different realizations is n^{n}, after n^{n} iterations, there will be two identical images among the images Med^{k}u, k ∈ {1, 2, …, n^{n}}. Thus, Med is cyclic. As a simple example, consider the chessboard image, where u(i, j) = 255 if i + j is even and u(i, j) = 0 otherwise. When we apply the median filter that takes the median of the four values surrounding a pixel and the pixel value, it is clear that the filter “reverses” the chessboard pattern. Indeed, any white pixel (value 255) is surrounded by four black pixels (value zero), so the median filter transforms the white pixel into a black pixel. In the same way, a black pixel is transformed into a white pixel.

8.4 Comments and references

The remarkable denoising properties and numerical efficiency of median filters for the removal of all kinds of impulse noise in digital images, movies, and video signals are well known and acclaimed [83, 150, 205, 220, 225]. The last reference cited as well as the next three all propose simple and efficient implementations of the median filter [29, 81, 139]. An introduction to weighted median filters can be found in [48, 276], and information about some generalizations (conditional median filters, for example) can be found in [27, 173, 251]. The min, max, and median filters are particular instances of rank order filters; see [78] for a general presentation of these filters. There are few studies on iterated median filters. The use of iterated median filters as a scale space is, however, proposed in [34]. The extension of median filtering to multichannel (color) images is problematic, although there have been some interesting attempts [64, 226].

144CHAPTER 8. MEDIAN FILTERS AND MATHEMATICAL MORPHOLOGY

Figure 8.3: Smoothing effect of a median filter on level lines. Above, left to right: original image; all of its level lines (boundaries of level sets) with levels multiple of 12; level lines at level 100. Below,left to right: result of two iterations of a median filter with a disk with radius 2; corresponding level lines (levels multiple of 12); level lines at level 100.

Part III

Local Asymptotic Analysis of Operators

145

Chapter 9

Asymptotic Behavior of Contrast-Invariant Isotropic Operators, Dimension Two

We are going to study the inf-sup function operators introduced in section 7.2 defined by

T u(x) = inf_{B∈B} sup_{y∈x+B} u(y),  (9.1)

where B is a given set of structuring elements. Theorem 7.3 states that these operators are contrast and translation invariant. Theorem 7.3 also states that function operators that are contrast and translation invariant on a suitable domain F can be represented (for almost all x ∈ ℝ^{N}) as inf-sup(or sup-inf) operators. Corollary 7.4 states that these representations hold for all x ∈ ℝ^{N} if T commutes with the threshold operator γ_{λ} for all λ ∈ ℝ. This means that our asymptotic analysis of contrast-invariant operators using their inf-sup representation is completely general (at least up to a set of measure zero).

In this chapter and the next, we assume that the set of structuring elements B is bounded and isotropic. This means that there is an M > 0 such that B ⊂ D(0, M) for all B ∈ B and that B ∈ B implies that RB ∈ B for all linear isometries R of ℝ^{2}. It may seem that these assumptions leave room for a large collection of operators, since they clearly leave room for a large collection of structuring elements. We will see, however, that the operators associated with these structuring elements fall into several large classes. We introduce a scale parameter 0 < h ≤ 1 and define the scaled operators T_{h} by

T_{h}u(x) = inf_{B∈B} sup_{y∈x+hB} u(y).

We will prove that in the limit, as h tends to zero, the action of T_{h} on smooth functions is not as varied as one might expect given the possible sets of structuring elements. In other words, the process of defining the operators T_{h} and letting h tend to zero maps the class of bounded isotropic structuring elements into equivalence classes that are simply characterized. This result is analogous to Theorem 2.2 for linear filters, which mapped certain Laplacian-consistent kernels into classes based on the value of their second moments. As an example,

147 148CHAPTER 9. ASYMPTOTIC BEHAVIOR OF CONTRAST-INVARIANT OPERATORS IN ℝ^{2}

we will show that if T_{h} is a scaled median operator, then

T_{h}u(x) − u(x) = h^{2}C|Du(x)|curv(u)(x) + o(h^{2}),

where the constant C depends only on the function k used to define the median operator. Thus, the operator |Du|curv(u) plays the same role for a class of contrast-invariant operators as the Laplacian Δu does for linear operators.

Exercise 9.1. The purpose of this exercise is to relate invariant properties of the set of structuring elements B to properties of T. Assume that B is invariant under a group G of transformations of ℝ^{2} onto ℝ^{2}. This means that, for all g ∈ G, B ∈ B if and only if gB ∈ B. Define the operator I_{g} on functions u : ℝ^{2} → ℝ by I_{g}u(x) = u(gx). Show that T I_{g} = I_{g}T. If G is the group of isometries of ℝ^{2} that leave the origin fixed, then T is by definition an isometric operator.

9.1 Asymptotic behavior theorem in ℝ^{2}

Let T be an inf-sup operator generated by a set of structuring elements B that is bounded and isotropic, and define the function H(s), s ∈ ℝ, by

H(s) = T [x + sy^{2}](0),  (9.2)

where T [x + sy^{2}] denotes “T u with u(x, y) = x + sy^{2}.” Since T is monotone, H is a nondecreasing function; it is also a Lipschitz function. To see this, let B ∈ B be one of the structuring elements that define T and write x + s_{1}y^{2} = x + s_{2}y^{2} + (s_{1} − s_{2})y^{2}. Then

sup_{(x,y)∈B}(x + s_{1}y^{2}) ≤ sup_{(x,y)∈B}(x + s_{2}y^{2}) + |s_{2} − s_{1}|M^{2},

since B is contained in D(0, M). By taking the infimum over B ∈ B of both sides and using the definition of H, we see that

H(s_{1}) − H(s_{2}) ≤ |s_{1} − s_{2}|M^{2}.

By interchanging s_{1} and s_{2} in this last inequality, we deduce the Lipschitz relation

|H(s_{1}) − H(s_{2})| ≤ |s_{1} − s_{2}|M^{2}.  (9.3)

We also note the following relations for 0 < h ≤ 1:

T_{h}[x + sy^{2}](0) = hT [x + hsy^{2}](0) = hH(hs),  (9.4)
T_{h}[x](0) = hT [x](0) = hH(0)  (9.5)

It is important to keep in mind that T [x](0) is not a function of x; it is the result of applying T to the “test” function u(x, y) = x and evaluating it at the origin. Thus, T [x](0) is a constant that depends only on B.

Theorem 9.1. Let B be a set of structuring elements that is invariant under the group of linear isometries of ℝ^{2} and assume that B is bounded by M. Define the operator T by T u(x) = inf_{B∈B} sup_{y∈x+B} u(y), and define the scaled operators T_{h}, 0 < h ≤ 1, by T_{h}u(x) = inf_{B∈B} sup_{y∈x+hB} u(y). Then, for any C^{2} function u : ℝ^{2} → ℝ,

T_{h}u(x) − u(x) = hT [x](0)|Du(x)| + O(x, h^{2}).

Furthermore, |O(x, h^{2})| ≤ C_{x}h^{2}, where C_{x} depends only on u and x.

9.1. ASYMPTOTIC BEHAVIOR THEOREM IN ℝ^{2} 149

Proof. The invariance of B under the action of linear isometries implies by Exercise 10.1 that all of the T_{h} are invariant under rotations. Now suppose that p = |Du(x)| > 0. Then by a suitable rotation, we may assume that Du(x) = (|Du(x)|, 0), and the first-order expansion of u in a neighborhood of x can be written as

u(x + y) = u(x) + px + O(x, |y|^{2}),  (9.6)

where y = (x, y). Define C = sup_{|y|≤M} |u(x + y) − u(x) − px|/|y|^{2} and deduce from (9.6) that

|u(x + y) − u(x) − px| ≤ C|y|^{2}

for y ∈ D(0, M). Hence,

u(x + hy) − u(x) ≤ phx + Ch^{2}|y|^{2} and phx ≤ u(x + hy) − u(x) + Ch^{2}|y|^{2}

(9.7)

for all y ∈ D(0, M). Since hB ⊂ D(0, hM), we see from the first inequality of (9.7) that

sup_{y∈B} u(x + hy) − u(x) ≤ sup_{y∈B}[phx] + sup_{y∈B} Ch^{2}|y|^{2} = hp sup_{y∈B}[x] + CM^{2}h^{2}.

This implies that

T_{h}u(x) − u(x) ≤ hp inf_{B∈B} sup_{y∈B}[x] + CM^{2}h^{2}

for 0 < h ≤ 1, and since inf_{B∈B} sup_{y∈B}[x] = T [x](0), we see that

T_{h}u(x) + u(x) ≤ hpT [x](0) + CM^{2}h^{2}.

The same argument applied to the second inequality of (9.7) shows that

hpT [x](0) ≤ T_{h}u(x) − u(x) + CM^{2}h^{2},

so |T_{h}u(x) − u(x) − hpT [x](0)| ≤ CM^{2}h^{2}. Since p = |Du(x)|, we see that

T_{h}u(x) − u(x) = hT [x](0)|Du(x)| + O(x, h^{2}),

which proves the result in case p ≠ 0. If p = |Du(x)| = 0, equation (9.6) remains true, as does the following argument, so the result is true for this case as well. Finally, to finish the proof, take C_{x} = CM^{2}.

9.1.1 Proof strategy and types of results

The statement and proof of Theorem 10.1 are models for similar results and arguments presented in this and this following five chapters, including the next theorem. Since it is so typical, it is useful to review and comment on the steps of the proof and the results. The purpose is to demystify the more complex results and arguments that follow.

Invariably, the operators T_{h} are inf-sup (or sup-inf) operators and are thus always translation invariant and commute with the addition of constants. (In abstract cases, we will assume these properties.) This means that it is always possible to assume, without loss of generality, that the point of interest is x = 0 150CHAPTER 9. ASYMPTOTIC BEHAVIOR OF CONTRAST-INVARIANT OPERATORS IN ℝ^{2}

and that u(x) = 0. (We chose not to do this in the proofs of Theorems 10.2 and 10.2, since to do so may have obscured the argument.)

The next most important invariant in isotropy. Without exception, the sets of structuring elements we consider will be isotropic, and so, by Exercise 10.1, the associated operators are isotropic. It is this invariant that allows us, without loss of generality, to rotate the coordinate system so Du(x) = (|Du(x), 0). This, in turn, allows us to expand u at x (or at the origin after a translation) as

u(x + y) = u(x) + P (x, y) + R(x, y).

P is a polynomial (usually of degree two) whose linear term is |Du(x)|x. The variable y = (x, y_{1}, …, y_{N−1}), and R is an error term for which we have estimates in terms of the derivatives of u. In this chapter and the next, we assume that the structuring elements are bounded by a constant M. This allows us to make the following kind of estimate when P is of order two:

sup_{y∈B} u(x+hy)−u(x) ≤ sup_{y∈B} P (x, hy)+ sup_{|y|≤M} |R(x, y)| ≤ sup_{y∈B} P (x, hy)+CM^{3}h^{3},

for all x is some set and all y ∈ D(0, M). This inequality and the one obtained by interchanging u(x + y) − u(x) and P (x, y) imply a relation of the form

T_{h}u(x) − u(x) = T_{h}[P (x, hy)](0) + O(x, |y|^{3}).  (9.8)

Once a formula like this is established, further analysis focuses on investigating T_{h}[P (x, hy)](0) using more specific properties of B. We will see this in the proof of Theorem 10.2. There we consider B for which T [x](0) = 0. Theorem 10.1 is also a case in point. It tells us that the behavior of the inf-sup operators T_{h} generated by bounded isotropic structuring elements depends, for small h, completely on the action of T on the test function u(x, y) = x. If T [x](0) ≠ 0, then, informally, we see that

\frac{T_{h}u(x) − u(x)}{h} = hT [x]|Du(x)| + O(h),

so in the limit as h → 0, T acts like a dilation if T [x] > 0 and like an erosion if T [x] < 0 (see Proposition 8.8). Thus, if T [x](0) ≠ 0, there is no need to define T with a complicated set of structuring elements. Asymptotically these operators are either dilations or erosions, and these can be defined with a single structuring element, namely, a disk.

Equations like (9.8) relate the operators T_{h} to a PDE. On the other hand, there are results like Theorem 10.2(ii) that provide estimates on the size of |T_{h}u(x) − u(x)|, and these result are used to relate the viscosity solutions of the PDEs to functions generated by the T_{h} called approximate solutions (Chapters 15 and 16). Other kinds of results are developed to support the analysis of T_{h}[P (x, hy)](0) for special classes of structuring elements B, including median filters and affine-invariant sets of structuring elements.

We have stated and demonstrated Theorem 10.1 for inf-sup operators. The result is also true for sup-inf operators, and this is the subject of the next exercise.

Exercise 9.2. Assume that the set of structuring elements B is isometric and bounded, as in Theorem 10.1. Define T by T u(x) = inf_{B∈B} sup_{y∈x+B} u(y). Prove that T [−x](0) = T [x](0). Use this and the relation T^{−}u = −T (−u) to show that the results of Theorem 10.1 hold for sup-inf operators.

9.1. ASYMPTOTIC BEHAVIOR THEOREM IN ℝ^{2} 151

Exercise 9.3. Assume that T is defined as an inf-sup operator and that the structuring elements are bounded but not necessarily isotropic. Show that

T_{h}u(x) = u(x) + hT [Du(x) · x](0) + O(h^{2}).

9.1.2 The asymptotic behavior of T_{h} when T [x](0) = 0

If T [x](0) = 0, then Theorem 10.1 is true but not very interesting. On the other hand, operators for which T [x](0) = 0 are interesting. If we consider T u(x) to be a kind of average of the values of u in a neighborhood of x, then assuming that T [x](0) = 0 makes sense and leads to interesting results. This means, however, that we must consider the next term in the expansion of T_{h}u; to do so we need to assume that u is C^{3}. This is the content of the next theorem, which is the main theoretical result of the chapter. The proof is more involved than that of Theorem 10.1, but at the macro level, they are similar.

Theorem 9.2. Let B be a set of structuring elements that is isometric and bounded, and let M denote the bound. Define the operator T by T u(x) = inf_{B∈B} sup_{y∈x+B} u(y), and define the scaled operator T_{h}, 0 < h ≤ 1, by T_{h}u(x) = inf_{B∈B} sup_{y∈x+hB} u(y). Also assume that T [x](0) = 0. Then for any C^{3} function u : ℝ^{2} → ℝ we have the following results:

(i) On every compact set K ⊂ {x | Du(x) ≠ 0},

T_{h}u(x) − u(x) = h|Du(x)|H(\frac{1}{2}h curv(u)(x)) + O(x, h^{3}),

where |O(x, h^{3})| ≤ C_{K}h^{3} for some constant C_{K} that depends only on u and K.

(ii) On every compact set K in ℝ^{2},

|T_{h}u(x) − u(x)| ≤ \frac{h^{2}M^{2}}{2} ∥D^{2}u(x)∥ + C_{K}h^{3}

where the constant C_{K} depends only on u and K.

Proof. For the meanings of p, a, b, and c see section 4.5. In particular, p = |Du(x)|. We are not going to set x = 0 and u(x) = 0, but we will define the local coordinate system by taking Du(x) = (p, 0). Using this notation, we expand u for y = (x, y) in a neighborhood of x:

u(x + y) = u(x) + px + ax^{2} + by^{2} + cxy + R(x, y).  (9.9)

The remainder term R is going to follow us throughout the proof (and indeed throughout the next six chapters), so it is useful to deal with it now. R can be written as

R(x, y) = ( ∫_{0}^{1}(1 − t)^{2}D^{3}u(x + ty) dt)y^{(3)},

which gives the estimate

|R(x, y)| ≤ ( sup_{t∈[0,1]} ∑_{(i_{1},i_{2},i_{3})} |\frac{∂^{3}u}{∂x_{i_{1}}∂x_{i_{2}}∂x_{i_{3}}} (x + ty)|)|y|^{3}.

152CHAPTER 9. ASYMPTOTIC BEHAVIOR OF CONTRAST-INVARIANT OPERATORS IN ℝ^{2}

We will write

∥D^{3}u(x)∥ = ∑_{(i_{1},i_{2},i_{3})} |\frac{∂^{3}u}{∂x_{i_{1}}∂x_{i_{2}}∂x_{i_{3}}} (x)|

and ∥D^{2}u(x)∥ = ∑_{i,j} |u_{ij}(x)|. Since u is assumed to be C^{3}, the two functions

x ↦ ∥D^{2}u(x)∥ and x ↦ ∥D^{3}u(x)∥

are continuous and thus bounded on any compact set K. With this notation, we have

|R(x, hy)| ≤ sup_{x∈K,|y|≤M} ∥D^{3}u(x + hy)∥h^{3}M^{3}.

If we define K_{M} = {x | d(x, K) ≤ M}, then we can write this inequality as

|R(x, hy)| ≤ sup_{z∈K_{M}} ∥D^{3}u(z)∥h^{3}M^{3}

for all x ∈ K and y ∈ D(0, M), and we have a uniform bound on K × D(0, M).

Now return to equation (9.9). For 0 < h ≤ 1,

u(x + hy) = u(x) + h(px + ahx^{2} + bhy^{2} + chxy) + R(x, hy),

and so for any B ∈ B,

sup_{y∈B} u(x + hy) ≤ u(x) + h sup_{y∈B}[u_{h}(x, y)] + sup_{y∈B} |R(x, hy)|.

Thus,

T_{h}u(x) ≤ u(x) + hT [u_{h}(x, y)](0) + inf_{B∈B} sup_{y∈B} |R(x, hy)|,  (9.10)

where u_{h}(x, y) = px + ahx^{2} + bhy^{2} + chxy and y = (x, y). Now let K be an arbitrary compact set. From our analysis of R, we deduce that

T_{h}u(x) ≤ u(x) + hT [u_{h}(x, y)](0) + sup_{z∈K_{M}} ∥D^{3}u(z)∥h^{3}M^{3}  (9.11)

for all x ∈ K. The same analysis shows that

u(x) ≤ T_{h}u(x) + hT [u_{h}(x, y)](0) + sup_{z∈K_{M}} ∥D^{3}u(z)∥h^{3}M^{3},  (9.12)

and we conclude that

T_{h}u(x) − u(x) = hT [u_{h}(x, y)](0) + O(x, h^{3})  (9.13)

for all x ∈ K. Furthermore, |O(x, h^{3})| ≤ C_{K}h^{3}, with C_{K} = sup_{z∈K_{M}} ∥D^{3}u(z)∥M^{3}, and C_{K} depends only on K and u. (Of course, it depends also on B, but that is understood.) At this point, we have now reduced the proof to an analysis of T u_{h}(0).

We are now going to show that a and c have no role in the asymptotic behavior of T u_{h}(0). To make the proof as clear as possible, we break it into several steps.

9.1. ASYMPTOTIC BEHAVIOR THEOREM IN ℝ^{2} 153

Step 1: Estimating T u_{h}(0). If x ∈ K and y = (x, y) ∈ B, then |y| ≤ M and

px − h(|a| + |b| + |c|)M^{2} ≤ u_{h}(x, y) ≤ px + h(|a| + |b| + |c|)M^{2}.

We write this as

p(x)x − \frac{hM^{2}}{2} ∥D^{2}u(x)∥ ≤ u_{h}(x, y) ≤ p(x)x + \frac{hM^{2}}{2} ∥D^{2}u(x)∥

to highlight the dependence on x, even though x is fixed. By assumption T [x](0) = 0 (hence T [px](0) = 0), so after applying T to the inequalities, we see that

|T [u_{h}(x, y)](0)| ≤ \frac{hM^{2}}{2} ∥D^{2}u(x)∥.  (9.14)

This and equation (9.13) show that

|T_{h}u(x) − u(x)| ≤ \frac{h^{2}M^{2}}{2} ∥D^{2}u(x)∥ + C_{K}h^{3}.  (9.15)

We can take C_{K} = sup_{z∈K_{M}} ∥D^{3}u(z)∥M^{3}. Thus, (9.15) holds uniformly for x ∈ K and for all h, 0 < h ≤ 1. This proves part (ii).

Step 2: First reduction. We now focus on proving (i), and for this we assume that p = |Du(x)| ≠ 0. Define C = (|a| + |b| + |c|)M^{2} = (1/2)∥D^{2}u(x)∥M^{2}. By Step 1, for every B ∈ B, we see that

sup_{y∈B} u_{h}(x, y) ≥ inf_{B∈B} sup_{y∈B} u_{h}(x, y) = T [u_{h}(x, y)](0) ≥ −Ch.

If y = (x, y) ∈ B and x < −2Ch/p, then

u_{h}(x, y) = px + ahx^{2} + bhy^{2} + chxy < −2Ch + h(|a| + |b| + |c|)M^{2} = −Ch.

Thus, if we let C′ = 2C/p, then for any B ∈ B we have

sup_{y∈B} u_{h}(x, y) = sup_{y∈B∩{(x,y)|x≥−C′h}} u_{h}(x, y)

Step 3: Second reduction. Since T [u_{h}(x, y)](0) ≤ Ch (Step 1), it is not necessary to consider sets B for which sup_{y∈B} u_{h}(x, y) ≥ Ch. If sup_{y∈B} u_{h}(x, y) ≤ Ch, then for all (x, y) ∈ B

px + ahx^{2} + bhy^{2} + chxy ≤ Ch,

and hence

x ≤ \frac{1}{p}(Ch + (|a| + |b| + |c|)M^{2}h) ≤ \frac{2Ch}{p} = C′h.

This means that we can write

T [u_{h}(x, y)](0) = inf_{B∈B,B⊂{(x,y)|x≤C′h}} sup_{y∈B} u_{h}(x, y),  (9.16)

154CHAPTER 9. ASYMPTOTIC BEHAVIOR OF CONTRAST-INVARIANT OPERATORS IN ℝ^{2}

and by the result of Step 2,

T [u_{h}(x, y)](0) = inf_{B∈B,B⊂{(x,y)|x≤C′h}} sup_{y∈B∩{(x,y)|x≥−C′h}} u_{h}(x, y).  (9.17)

This relation is true if we replace u_{h}(x, y) with px + bhy^{2} and leads directly to the inequality

T [u_{h}(x, y)](0) ≤ T [px+bhy^{2}](0)+h inf_{B∈B,B⊂{(x,y)|x≤C′h}} sup_{y∈B∩{(x,y)|x≥−C′h}} |ax^{2}+cxy|

and, by interchanging u_{h}(x, y) and px + bhy^{2}, to the equation

T [u_{h}(x, y)](0) = T [px + bhy^{2}](0) + ε(x, y).  (9.18)

The estimated for the error term is

|ε(x, y)| ≤ h^{3}|a|C^{′2} + h^{2}|c|C′M.

Step 4: Conclusion. We now return to equation (9.13),

T_{h}u(x) − u(x) = hT [u_{h}(x, y)](0) + O(x, h^{3}),

and replace T [u_{h}(x, y)](0) with T [px + bhy^{2}](0) + ε(x, y) to obtain

T_{h}u(x) − u(x) = hT [px + bhy^{2}](0) + hε(x, y) + O(x, h^{3}).

By definition H(s) = T [x + sy^{2}](0), so the last equation can be written as

T_{h}u(x) − u(x) = hpH(bh/p) + hε(x, y) + O(x, h^{3}),

or, by replacing p and b with |Du(x)| and (1/2)curv(u)(x)|Du(x)|, as

T_{h}u(x) − u(x) = h|Du(x)|H(h\frac{1}{2}curv(u)(x)) + hε(x, y) + O(x, h^{3}). (9.19)

To finish the proof, we must examine the error term ε to establish a uniform approximation on compact sets where Du(x) ≠ 0. Thus, let K be any compact subset of ℝ^{2} such that K ⊂ {x | Du(x) ≠ 0}. For y ∈ D(0, M) (hence for y ∈ B ∈ B ∈ B), we have

|ε(x, y)| ≤ h^{3}|a|C^{′2} + h^{2}|c||C′|M

= \frac{h^{2}}{|Du(x)|^{2}} [\frac{h}{2}|u_{xx}(x)|∥D^{2}u(x)∥^{2} + |u_{xy}(x)||Du(x)|∥D^{2}u(x)∥M].

Since u is C^{3}, all of the functions on the right-hand side of this relation are continuous on K; in particular, |Du(x)| is bounded away from zero on K. Thus there is a constant C′_{K} that depends only on u and K such that |ε(x, y)| ≤ h^{2}C′_{K}. By combining and renaming the constants C_{K} and C′_{K}, this completes the proof of (i) and the proof of the theorem.

We apply these results to median filters in the next section, but before doing this, we pause to check that the results of Theorem 10.2 hold for operators represented in their sup-inf form. This is the content of the next exercise.

Exercise 9.4. Let B be a set of structuring elements that is bounded and invariant under linear isometries. Define T^{−}u(x) = sup_{B∈B} inf_{y∈x+B} u(y), and define H^{−}(s) = T^{−}[x + sy^{2}](0).

(i) Show that H^{−}(s) = −H(−s). (ii) Use (i) and the relation T^{−}u = −T (−u) to show that the results of Theorem

10.2 are true for sup-inf operators.

9.2. MEDIAN FILTERS AND CURVATURE MOTION IN ℝ^{2} 155

9.2 Median filters and curvature motion in ℝ^{2}

Recall that the median filter, Med_{k}, defined in Chapter 9 can be written as

Med_{k}u(x) = sup_{B∈B} inf_{y∈x+B} u(y),  (9.20)

where B = {B | B is measurable, |B|_{k} = 1/2} (Proposition 9.4). The first example we examine is the case k = 1_{D(0,1)}/π, that is, k is the uniform measure on the unit disk. We know by Proposition 9.7 that Med_{k}u = Med_{k}^{−}u if u is continuous, so by restricting the domain of Med_{k} to continuous functions u : ℝ^{2} → ℝ it can be represented by the inf-sup form

Med_{k}u(x) = inf_{B∈B} sup_{y∈x+B} u(y),  (9.21)

where B = {B | B is measurable, |B|_{k} = 1/2}. By using the results of Exercise 7.4, it is easy to show that the set of structuring elements B′ = {B′ | B′ = B ∩ D(0, 1), B ∈ B} is equivalent to B, in the sense that they both generate the same median filter. Thus we assume in what follows that B ⊂ D(0, 1). There is one more point that needs to be clarified, and we relegate it to the next exercise.

Exercise 9.5. The scaled median filter (Med_{k})_{h}, h < 1, is defined by

(Med_{k})_{h}u(x) = inf_{B∈hB} sup_{y∈x+B} u(y).  (9.22)

At first glance, it is not clear that this is a median filter, but, in fact, it is: Show that (Med_{k})_{h} = Med_{k_{h}}, where k_{h} = 1_{D(0,h)}/πh^{2}.

The actions of median filters and comparisons of these filters with other simple filters are illustrated in Figures 9.2, 9.3, 9.4, 9.5, and 9.6.

Everything is now in place to investigate the asymptotic behavior of the scaled median filter Med_{k_{h}}, which is represented by

Med_{k_{h}}u(x) = inf_{B∈hB} sup_{y∈x+B} u(y),

where hB = {B | |B|_{k_{h}} = 1/2, B ⊂ D(0, h)}. The main result of this section, Theorem 10.4, gives an infinitesimal interpretation of this filter. This result will be extended in the next chapter to a general weighted median filter in ℝ^{N}. Theorem 10.4 can be deduced directly from Theorem 10.2 once we have the following lemma. This lemma is an example of the kind of detailed analysis of T_{h}[P (x, hy)](0) that we spoke about in section 10.1.1, and characteristically, the proof is quite special, having no immediate generalization to ℝ^{N}.

Lemma 9.3.

Med_{k}[x + sy^{2}](0) = \frac{s}{3} + O(|s|^{3}).

Proof. Represent Med_{k} by Med_{k}u(x) = sup{λ | x ∈ Med_{k}X_{λ}u}. Then

Med_{k}[x + sy^{2}](0) = sup{λ | 0 ∈ Med_{k}X_{λ}[x + sy^{2}] }.

By definition, 0 ∈ Med_{k}X_{λ}[x + sy^{2}] if and only if |X_{λ}[x + sy^{2}]|_{k} ≥ 1/2. This implies that Med_{k}[x+sy^{2}](0) = m(s), where |X_{m(s)}[x+sy^{2}]|_{k} = 1/2, and this is 156CHAPTER 9. ASYMPTOTIC BEHAVIOR OF CONTRAST-INVARIANT OPERATORS IN ℝ^{2}

true if and only if the graph of x + sy^{2} = m(s) divides D(0, 1) into two sets that have equal area. Of course, we are only considering small s, say |s| ≤ 1/2. The geometry of this situation is illustrated in Figure 9.1. The signed area between the y-axis and the parabola P (s) for |y| ≤ 1 is

∫_{−1}^{1}(m(s) − sy^{2}) dy = 2m(s) − \frac{2s}{3}.

Thus, m(s) is the proper value if and only if

m(s) − \frac{s}{3} = Area(ABE),  (9.23)

where ABE denotes the curved triangle bounded by the parabola, the circle, and the line y = −1. This area could be computed, but it is sufficient to bound it by Area(ABCD). The length of base AB is |m(s)−s|, and an easy computation shows that the length of the hight BC is less than (m(s) − s)^{2}. This and (9.23) imply that

|m(s) − \frac{s}{3}| ≤ |m(s) − s|^{3}.

From this we conclude that m(s) = s/3 + O(|s|^{3}), which proves the lemma.

9.2. MEDIAN FILTERS AND CURVATURE MOTION IN ℝ^{2} 157

Theorem 9.4. If u : ℝ^{2} → ℝ is C^{2}, then we have the following expansions:

(i) On every compact set K ⊂ {x | Du(x) ≠ 0},

Med_{k_{h}}u(x) = u(x) + \frac{1}{6}|Du(x)|curv(u)(x)h^{2} + O(x, h^{3}),

where |O(x, h^{3})| ≤ C_{K}h^{3} for some constant C_{K} that depends only on u and K.

(ii) On every compact set K in ℝ^{2},

|Med_{k_{h}}u(x) − u(x)| ≤ \frac{1}{2}∥D^{2}u(x)∥h^{2} + C_{K}h^{3}

where the constant C_{K} depends only on u and K.

Proof. We have shown (or it is immediate) that the operator T_{h} = Med_{k_{h}} satisfies all of the hypotheses of Theorem 10.2. In particular, H(0) = Med_{k}[x + sy^{2}](0) = 0 by Lemma 10.3. Also by Lemma 10.3, H(s) = s/3 + O(|s|^{3}). This means that we have

H(\frac{1}{2}h curv(u)(x)) = \frac{1}{6}h curv(u)(x) + O(h^{3} |curv(u)(x)|^{3}).

The first result is now read directly from Theorem 10.2(i). Relation (ii) follows immediately from Theorem 10.2(ii).

Our second example is called the Catté–Dibos scheme. It involves another application of Theorem 10.2.

Theorem 9.5. Let B be the set of all line segments of length 2 centered at the origin of ℝ^{2}. Define the operators SI_{h} and IS_{h} by

SI_{h}u(x) = sup_{B∈hB} inf_{y∈x+B} u(y)  and  IS_{h}u(x) = inf_{B∈hB} sup_{y∈x+B} u(y).

If u : ℝ^{2} → ℝ is C^{2} and |Du(x)| ≠ 0, then

\frac{1}{2}(IS_{h} + SI_{h})u(x) = u(x) + h^{2} \frac{1}{4}curv(u)(x)|Du(x)| + O(h^{3}).

Proof. The first step is to compute the action of the operators on u(x, y) = x + sy^{2}. Define H(s) = IS[x + sy^{2}](0) and write (x, y) = (r cos θ, r sin θ). Then

H(s) = inf_{− \frac{π}{2} ≤θ≤ \frac{π}{2}} sup_{−1≤r≤1}(r cos θ + sr^{2} sin^{2} θ).

For s ≥ 0 and r ≥ 0, the function r ↦ r cos θ + sr^{2} sin^{2} θ is increasing. Hence,

H(s) = inf_{− \frac{π}{2} ≤θ≤ \frac{π}{2}} (cos θ + s sin^{2} θ) = s

for sufficiently small s, say, s < 1/2. If s ≤ 0, then H(0) = 0, since

0 ≤ sup_{−1≤r≤1}(r cos θ + sr^{2} sin^{2} θ) ≤ cos θ.

158CHAPTER 9. ASYMPTOTIC BEHAVIOR OF CONTRAST-INVARIANT OPERATORS IN ℝ^{2}

Figure 9.1: When s is small, the parabola P (s) with equation x + sy^{2} = m divides D(0, 1) into two components. The median value m(s) of x + sy^{2} on D(0, 1) simply is the value m for which these two components have equal area.

If H^{−}(s) = SI[x + sy^{2}](0), then we know from Exercise 10.4 that H^{−}(s) = −H(−s). Thus we have

H(s) = \begin{cases}s, & if s ≥ 0; \\ 0, & if s < 0; \end{cases}  and  H^{−}(s) = \begin{cases}0, & if s ≥ 0; \\ s, & if s < 0. \end{cases}

Thus, H(s) + H^{−}(s) = s for all small s. Since H(0) = H^{−}(0) = 0, the conclusions of Theorem 10.2 apply. By applying Theorem 10.2(i) to IS_{h} and SI_{h} and adding, we have

(IS_{h} + SI_{h})u(x) = 2u(x) + h(H + H^{−})(\frac{h}{2}curv(u)(x)) + O(h^{3})

= 2u(x) + \frac{h^{2}}{2} curv(u)(x) + O(h^{3}).

Dividing both sides by two gives the result.

Exercise 9.6. Let B be the set of all rectangles in the plane with length two, width δ < 1, and centered at the origin. Define the operators IS_{h} and SI_{h} by

IS_{h}u(x) = inf_{B∈hB} sup_{y∈x+B} u(y)  and  SI_{h}u(x) = sup_{B∈hB} inf_{y∈x+B} u(y).

9.3. COMMENTS AND REFERENCES 159

Figure 9.2: The result of smoothing with an erosion is independent of the curvature of the level lines. Left: image of a simple shape. Right: difference of this image and its eroded image. Note that the width of the difference is constant.

(i) Let u : ℝ^{2} → ℝ be C^{2}. Compute the expansions of IS_{h}u(x), SI_{h}u(x), and (1/2)(IS_{h} + SI_{h})u(x) in terms of small h > 0.

(ii) Take δ = h and compute the same expansions. (iii) Take δ = h^{α} and interpret the expansions for α > 0 and for α < 0.

9.3 Comments and references

Merriman, Bence, and Osher [195] discovered, and gave some heuristic arguments to prove, that a convolution of a shape with a Gaussian followed by a threshold at 1/2 simulated the mean-curvature motion given by ∂u/∂t = |Du|curv(u). The consistency of their arguments was checked by Mascarenhas [192]. Barles and Georgelin [37] and Evans [93] also gave consistency proofs; in addition, they showed that iterated weighted Gaussian median filtering converges to the mean curvature motion. An extension of this result to any iterated weighted median filter was given by Ishii in [136]. An interesting attempt to generalize this result to vector median filters was made Caselles, Sapiro, and Chung in [64]. Catté, Dibos, and Koepfler [66] related mean curvature motion to the classic morphological filters whose structuring elements are one-dimensional sets oriented in all directions (see [203] and [249] regarding these filters).

The importance of the function H in the main expansion theorem raises the following question: Given an increasing continuous function H, are there structuring elements B such that H(s) = inf_{B∈B} sup_{(x,y)∈B}(x + sy^{2})? As we have seen is this chapter, the function H(s) = s is attained by a median filter. Pasquignon [217] has studied this question extensively and shown that all of the 160CHAPTER 9. ASYMPTOTIC BEHAVIOR OF CONTRAST-INVARIANT OPERATORS IN ℝ^{2}

Figure 9.3: Median filter and the curvature of level lines. Smoothing with a median filter is related to the curvature of the level lines. Left: image of a simple shape. Right: difference of this image with itself after it has been smoothed by one iteration of the median filter. We see, in black, the points which have changed. The width of the difference is proportional to the curvature, as indicated by Theorem 10.4.

functions of the form H(s) = s^{α} are possible using sets of simple structuring elements.

The presentation of the main results of this chapter is mainly original and was announced in the tutorials [127] and [128]. An early version of this work appeared in [125].

9.3. COMMENTS AND REFERENCES 161

Figure 9.4: Fixed point property of the median filter, showing its griddependence. Left: original image. Right: result of 46 iterations of the median filter with a radius of 2. The resulting image turns out to be a fixed point of this median filter. This is not in agreement with Theorem 10.4, which shows that median filters move images by their curvature: The image on the right clearly has nonzero curvatures! However, the discrete median filter that we have applied here operating on a discrete image is grid-dependent and blind to small curvatures.

162CHAPTER 9. ASYMPTOTIC BEHAVIOR OF CONTRAST-INVARIANT OPERATORS IN ℝ^{2}

Figure 9.5: Comparing an iterated median filter and a median filter. Top-left: original image. Top-middle: 16 iterations of the median filter with a radius 2, Top-right: one iteration of the same median filter with a radius 8. Below each image are the level-lines for grey levels equal to multiples of 16. This shows that iterating a small size median filter provides more accuracy and less shape mixing than applying a large size median filter. Compare this with the Koenderink– Van Doorn shape smoothing and the Bence–Merriman–Osher iterated filter in Chapter 2, in particular Figures 2.9, 2.10, and 2.12.

9.3. COMMENTS AND REFERENCES 163

Figure 9.6: Consistency of the median filter and of the Catté–Dibos. Top row: the sea bird image and its level lines for all levels equal to multiples of 12. Second row: a median filter on a disk with radius 2 has been iterated twice. Third row: an inf-sup and then a sup-inf filter based on segments have been applied. On the right: the corresponding level lines of the results, which, according to the theoretical results (Theorems 10.4 and 10.5), must have moved at a speed proportional to their curvature. The results are very close. This yields a cross validation of two very different numerical schemes that implement curvaturemotion based smoothing.

164CHAPTER 9. ASYMPTOTIC BEHAVIOR OF CONTRAST-INVARIANT OPERATORS IN ℝ^{2}

Chapter 10

Asymptotic Behavior of Contrast-Invariant Isotropic Operators, Dimension N

We are going to generalize to N dimensions the asymptotic results of Chapter 10. Our aim is to show that the action of any isometric inf-sup filter, when properly scaled, is a motion of the N-dimensional image that is controlled by its principal curvatures. Once we have established the main asymptotic result, we will, as in Chapter 10, analyze several multidimensional filters. In particular, we will relate the median filter to the mean curvature of the level surface. The action of median filtering in three dimensions is illustrated in Figures 11.1 and 11.2.

10.1 Asymptotic behavior theorem in ℝ^{N}

The main theorem, Theorem 11.1, is a straightforward generalization of Theorems 10.1 and 10.2; the proof is basically the same, once the notation is established. For x ∈ ℝ^{N}, we write x = (x, y_{2}, …, y_{N}) = (x, y), so y ∈ ℝ^{N−1}. We let s = (s_{2}, …, s_{N}) and define

H(s) = T [x + s_{2}y_{2}^{2} + ⋯ + s_{N}y_{N}^{2}](0),  (10.1)

where the operator T is defined by

T u(x) = inf_{B∈B} sup_{y∈x+B} u(y).

Throughout the chapter, the set of structuring elements B will be bounded and isotropic. Thus, we assume that the elements of B are contained in the ball D(0, M) and that, for any linear isometry R, RB ∈ B if and only if B ∈ B. Arguments similar to those given in section 10.1 show that H is a nondecreasing function is each variable s_{i}, and that it satisfies the Lipschitz condition

|H(s_{1}) − H(s_{2})| ≤ M^{2}|s_{1} − s_{2}|.  (10.2)

165 166CHAPTER 10. ASYMPTOTIC BEHAVIOR OF CONTRAST-INVARIANT OPERATORS IN ℝ^{N}

Let u : ℝ^{N} → ℝ be C^{3} and assume that Du(x) ≠ 0. Then we denote the vector whose terms are the N − 1 principal curvatures of the level surface {y | u(y) = u(x)} that passes through x by κ(u)(x) = κ(u) = (κ_{2}, …, κ_{N}). The terms κ_{i}(u)(x)|Du(x)| are then the eigenvalues of the restriction of D^{2}u(x) to Du(x)^{⊥}. (We refer to section 4.6 for information about principal curvature.)

As in Chapter 10, we will state and prove the main results, given in Theorem 11.1, for operators written in their inf-sup form. We leave it as exercise to show that the relation T^{−}u = −T (−u) implies the result for sup-inf operators.

Theorem 10.1. Let B be a family of structuring elements belonging to ℝ^{N} and assume that B is bounded and isotropic. Define the operator T and its scaled versions T_{h}, 0 < h ≤ 1, by

T u(x) = inf_{B∈B} sup_{y∈x+B} u(y)  and  T_{h}u(x) = inf_{B∈hB} sup_{y∈x+B} u(y),

respectively, and write H(s) = T [x + s_{2}y_{2}^{2} + ⋯ + s_{N}y_{N}^{2}](0). Then for every C^{3} function u : ℝ^{N} → ℝ, we have the following results:

(i) It is always true that

T_{h}u(x) = u(x) + hH(0)|Du(x)| + O(x, h^{2});

(ii) If H(0) = 0, then on every compact set K contained in {x | Du(x) ≠ 0}

T_{h}u(x) = u(x) + hH(h\frac{1}{2}κ(u)(x))|Du(x)| + O(x, h^{3})

where |O(x, h^{3})| ≤ C_{K}h^{3} for some constant C_{K} that depends only on u and K;

(iii) If H(0) = 0, then on every compact set K ⊂ ℝ^{N},

|T_{h}u(x) − u(x)| ≤ \frac{h^{2}M^{2}}{2} ∥D^{2}u(x)∥ + O(x, h^{3}),

where |O(x, h^{3})| ≤ C_{K}h^{3} for some constant C_{K} that depends only on u

and K.

Proof. The proof follows exactly the proofs of Theorems 10.1 and 10.2; we simply have to relate the notation used for the N-dimensional case to that used in the two-dimensional case. We begin by assuming that Du(x) ≠ 0. We then establish the local coordinate system at x defined by i_{1} = Du(x)/|Du(x) and i_{2}, …, i_{N}, where i_{2}, …, i_{N} are the eigenvectors of the restriction of D^{2}u(x) to the hyperplane Du(x)^{⊥}. Then in a neighborhood of x we can expand u as follows:

u(x + y) = u(x) + px + ax^{2} + b_{2}y_{2}^{2} + ⋯ + b_{N}y_{N}^{2} + (c · y)x + R(x, y), (10.3)

where y = xi_{1} + y_{2}i_{2} + ⋯ + y_{N}i_{N}, p = |Du(x)| > 0, and for j = 2, …, N,

\begin{aligned}a & = \frac{1}{2} \frac{∂^{2}u}{∂x^{2}}(x) = \frac{1}{2}D^{2}u(x)(i_{1}, i_{1}), \\ b_{j} & = \frac{1}{2} \frac{∂^{2}u}{∂y_{j}^{2}}(x) = \frac{1}{2}D^{2}u(x)(i_{j}, i_{j}), \\ c_{j} & = \frac{∂^{2}u}{∂x∂y_{j}} (x) = D^{2}u(x)(i_{1}, i_{j}). \end{aligned}  (10.4)

10.2. ASYMPTOTIC BEHAVIOR OF MEDIAN FILTERS IN ℝ^{N} 167

Figure 10.1: Three-dimensional median filter. The original three-dimensional image (not shown) is of 20 slices of a vertebra. Three successive slices are displayed in the left column. The next column shows their level lines (multiples of 20). The third column shows these three slices after one iteration of the median filter based on the three-dimensional ball of radius two. The resulting level lines are shown in the last column.

We can also write b_{j} as

b_{j} = \frac{1}{2}|Du(x)|κ_{j}(u)(x).  (10.5)

For the proof of (i), we write u(x + y) = u(x) + px + O(x, |y|^{2}) and just follow the steps of the proof of Theorem 11.1. The proof of (ii) and (iii) follows, step by step, the proof of Theorem 10.2. We need only make the following identifications: cxy ↔ (c·y)x, by^{2} ↔ b_{2}y_{2}^{2}+⋯+b_{N}y_{N}^{2}, and curv(u) ↔ κ(u).

10.2 Asymptotic behavior of median filters in ℝ^{N}

The median filters we consider will be defined in terms of a continuous weight function k : ℝ^{N} → [0, +∞) that is radial, k(x) = k(|x|), and that is normalized, 168CHAPTER 10. ASYMPTOTIC BEHAVIOR OF CONTRAST-INVARIANT OPERATORS IN ℝ^{N}

∫_{ℝ^{N}} k(x) dx = 1. Recall that, by definition,

|B|_{k} = ∫_{B} k(x) dx

and that the corresponding median operator is defined by

Med_{k}u(x) = sup_{|B|_{k}≥1/2} inf_{y∈x+B} u(y).  (10.6)

We also assume that k is nonseparable, which is the case if {x | k(x) > 0} is connected. Then by Proposition 9.7, Med_{k}u = Med_{k}^{−}u if u is continuous. This means that we can represent Med_{k} as an inf-sup operator using the same structuring elements, namely, {B | |B|_{k} ≥ 1/2}. In fact, we need consider only these B for which |B|_{k} = 1/2.

Define the scaled weight function k_{h}, 0 < h ≤ 1, by k_{h}(x) = h^{−N}k(x/h). Then a change of variable shows that |B|_{k} = 1/2 if and only if |hB|_{k_{h}} = 1/2, and this implies that (Med_{k})_{h} = Med_{k_{h}} (see Exercise 10.5). Since we consider only one weight function at a time, there should be no confusion if we write Med_{h} for the scaled operator.

We analyzed the asymptotic behavior of a median filter in ℝ^{2} whose weight function was the characteristic function of the unit disk in Chapter 10. This can be generalized to ℝ^{N} by taking k to be the normalized characteristic function of the unit ball, but we not do that here. We will go in a different direction by taking smooth weight functions. On the other hand, our analysis will not be as general as possible because this would be needlessly complicated. The k we consider will be smooth (C^{∞}) and have compact support. This means that the structuring elements are bounded and the results of Theorem 11.1 apply. On the other hand, k that do not have compact support are important; consider, for example, the Gaussian.

As in the case of ℝ^{2}, it is that action of Med_{k} on a “test function” that controls the first term of the expansion of (Med_{k})_{h}, and as before, we deal with this in a lemma.

Lemma 10.2. Let k be a nonnegative radial function belonging to the Schwartz class S. Assume that ∫_{ℝ^{N}} k(x) dx = 1 and that the support of k is connected. Then the function H(hb) = Med_{k}[x + h(b_{2}y_{2}^{2} + ⋯ + b_{N}y_{N}^{2})](0) can be expressed as

H(hb) = hc_{k}( ∑_{j=2}^{N} b_{j}) + O(h^{2}),

where

c_{k} = \frac{∫_{ℝ^{N−1}} y_{2}^{2}k(y) dy}{∫_{ℝ^{N−1}} k(y) dy} ,

y = (y_{2}, …, y_{N}), and b = (b_{2}, …, b_{N}).

Proof. Before beginning the proof, note that we have not assumed that k has compact support, so the result applies to the Gaussian, for example.

We will use the abbreviation b(y, y) = b_{2}y_{2}^{2} + ⋯ + b_{N}y_{N}^{2}, since b is, in fact, a diagonal matrix. Our proof is based on an analysis of the function f(λ, h) = 10.2. ASYMPTOTIC BEHAVIOR OF MEDIAN FILTERS IN ℝ^{N} 169

|X_{λ}(x + hb(y, y))|_{k}. Since X_{λ}(x + hb(y, y)) = {(x, y) | x + hb(y, y) ≥ λ}, we can express f as an integral,

f(λ, h) = ∫_{ℝ^{N−1}} ∫_{λ−hb(y,y)}^{∞} k(x, y) dx dy.

It follows from the assumption that k is in the Schwartz class that f : ℝ^{2} → ℝ is bounded and C^{∞}. Also, for every h ∈ ℝ, lim_{λ→−∞} f(λ, h) = 1 and lim_{λ→+∞} f(λ, h) = 0. Thus, for every h ∈ ℝ, there is at least one λ such that f(λ, h) = 1/2. In fact, there is only one such λ; this is a consequence of the assumption that the k is continuous and that its support is connected, which implies that it is nonseparable (see Exercise 9.2). To see that λ is unique, assume that there are λ < λ′ such that f(λ, h) = 1/2 and f(λ′, h) = 1/2. Then the two sets {(x, y) | x + hb(y, y) ≥ λ′} and {(x, y) | x + hb(y, y) ≤ λ} both have k-measure 1/2, but their intersection is empty. This contradicts the fact that k is nonseparable. This means that the relation f(λ, h) = 1/2 defines implicitly a well-defined function h ↦ λ(h).

Recall that Med_{k} was originally defined in terms of the superposition formula

Med_{k}u(x) = sup{λ | x ∈ Med_{k}X_{λ}u}.

This translates for our case into the relation

Med_{k}[x + hb(y, y)](0) = sup{λ | 0 ∈ Med_{k}X_{λ}[x + hb(y, y)]} = λ(h)

because 0 ∈ Med_{k}X_{λ}[x + hb(y, y)] if and only if |X_{λ}[x + hb(y, y)]|_{k} ≥ 1/2.

We are interested in the behavior of h ↦ λ(h) near the origin. The first thing to note is that λ(0) = 0. To see this, write

f(λ(0), 0) = ∫_{ℝ^{N−1}} ∫_{λ(0)}^{∞} k(x, y) dx dy = \frac{1}{2}.

Since k is radial, the value λ = 0 solves the equation ∫_{ℝ^{N−1}} ∫_{λ}^{∞} k(x, y) dx dy = 1/2. We have just shown that this equation has a unique solution, so λ(0) = 0.

Now consider the first partial derivatives of f:

\frac{∂f}{∂λ}(λ, h) = − ∫_{ℝ^{N−1}} k(((λ − hb(y, y))^{2} + y · y)^{1/2}) dy.  (10.7)
\frac{∂f}{∂h} (λ, h) = ∫_{ℝ^{N−1}} b(y, y)k(((λ − hb(y, y))^{2} + y · y)^{1/2}) dy.  (10.8)

These functions are C^{∞} because k is in the Schwartz class; also, (∂f/∂λ)(0, 0) ≠ 0. Then by the implicit function theorem, we know that the function h ↦ λ(h) that satisfies f(λ(h), h) = 1/2 is also C^{∞} and that

λ′(h)\frac{∂f}{∂λ}(λ(h), h) + \frac{∂f}{∂h} (λ(h), h) = 0.

Thus, for small h,

λ′(h) = − \frac{\frac{∂f}{∂h}(λ(h), h)}{\frac{∂f}{∂λ}(λ(h), h)},

170CHAPTER 10. ASYMPTOTIC BEHAVIOR OF CONTRAST-INVARIANT OPERATORS IN ℝ^{N}

Figure 10.2: Median filtering of a three-dimensional image. The first image is a representation of the horizontal slices of a three-dimensional level surface of the three-dimensional image of a vertebra. Right to left, top to bottom: 1, 2, 5, 10, 20, 30, 60, 100 iterations of a three-dimensional median filter based on a ball with radius three. This scheme is a possible implementation of the mean curvature motion, originally proposed as such by Bence, Merriman, and Osher.

and, using equations (10.7) and(10.8), we see that

λ′(0) = \frac{∫_{ℝ^{N−1}} b(y, y)k((y · y)^{1/2}) dy}{∫_{ℝ^{N−1}} k((y · y)^{1/2}) dy} .

Now expand λ for small h:

λ(h) = λ(0) + λ′(0)h + O(h^{2}).

Since ∫_{ℝ^{N−1}} b(y, y)k((y·y)^{1/2}) dy = ( ∑_{j=2}^{N−1} b_{j}) ∫_{ℝ^{N−1}} y_{2}^{2}k((y·y)^{1/2}) dy, H(hb) = λ(h), and λ(0) = 0, this proves the lemma.

Theorem 10.3. Let k be a nonnegative radial function belonging to the Schwartz class S. Assume that ∫_{ℝ^{N}} k(y) dy = 1 and that the support of k is compact and connected. Then for every C^{3} function u : ℝ^{N} → ℝ, we have the following results:

10.3. APPLICATIONS IN THREE DIMENSIONS 171

(i) On every compact set K ⊂ {x | Du(x) ≠ 0},

Med_{h}u(x) = u(x) + h^{2} \frac{1}{2}c_{k}( ∑_{i=2}^{N} κ_{i}(u)(x))|Du(x)| + O(x, h^{3}),

where |O(x, h^{3})| ≤ C_{K}h^{3} for some constant that depends only on u and K.

(ii) On every compact set K ⊂ ℝ^{N},

|Med_{h}u(x) − u(x)| ≤ \frac{h^{2}M^{2}}{2} ∥D^{2}u(x)∥ + O(x, h^{3}),

where |O(x, h^{3})| ≤ C_{K}h^{3} for some constant C_{K} that depends only on u

and K.

Proof. Recall that Med_{h} = (Med_{k})_{h} = Med_{k_{h}}. Since k is not separable and since we are restricting the domain of Med to continuous functions, we can represent it as an inf-sup operator (Proposition 9.7). Furthermore, the set of structuring elements may be taken to be B = {B | |B|_{k} = 1/2, B ⊂ support of k}. Since the support of k is compact, B is bounded. The set B is invariant under linear isometries of ℝ^{N} because k is radial. Thus, Theorem 11.1 is applicable. We know from Lemma 11.2 that H(0) = 0, so we can read (ii) directly from Theorem 11.1(iii).

By Lemma 11.2,

H(hκ(u)) = hc_{k}( ∑_{i=2}^{N} κ_{i}(u)|Du|) + O(h^{2}).

From this and Theorem 11.1(ii), we have

Med_{h}u(x) = u(x) + h^{2} \frac{1}{2}c_{k}( ∑_{i=2}^{N} κ_{i}(u)(x))|Du(x)| + O(x, h^{3}),

and we know that the estimate is uniform on any compact set K ⊂ {x | Du(x) ≠ 0}.

As mentioned above, this is by no means the most general result. The following exercise generalizes the result in one direction.

Exercise 10.1. Prove Lemma 11.2 and Theorem 11.3 assuming that k has compact support and is C^{2}, radial, and nonseparable.

10.3 Applications in three dimensions

This section contains several applications of Theorem 11.1 in three dimensions. A level surface of a C^{3} function in three dimensions has two principal curvatures, and this provides an extra degree of freedom for constructing contrast-invariant operators base on curvature motion. We develop the applications in three exercises. For each case, we will assume that the principal curvatures κ_{1} and κ_{2} are ordered such that κ_{1} ≤ κ_{2}. In each example, the set of structuring element B 172CHAPTER 10. ASYMPTOTIC BEHAVIOR OF CONTRAST-INVARIANT OPERATORS IN ℝ^{N}

is constructed from a single set B in ℝ^{2} by rotating B is all possible ways, that is, B = {RB | B ∈ ℝ^{2}, R ∈ SO(3)}. For each example we write

SI_{h}u(x) = sup_{B∈B} inf_{y∈x+hB} u(y)

and

IS_{h}u(x) = inf_{B∈B} sup_{y∈x+hB} u(y),

where 0 < h ≤ 1.

Exercise 10.2. Let B be a segment of length 2 centered at the origin. Our aim is to show that

IS_{h}u = u + h^{2} \frac{1}{2} κ_{1}^{+}(u)|Du| + O(h^{3}),
SI_{h}u = u + h^{2} \frac{1}{2} κ_{2}^{−}(u)|Du| + O(h^{3}),

and that

IS_{h}(SI_{h})u = u + h^{2} \frac{1}{2} (sign(κ_{1}(u)) + sign(κ_{2}(u))) min(|κ_{1}(u)|, |κ_{2}(u)|) + O(h^{3}).

(i) The first step is to compute H(hb). One way to do this is to write x = r sin ϕ, y_{2} = r cos ϕ cos θ, y_{3} = r cos ϕ cos θ, and use an argument similar to that given in the proof of Theorem 10.5 to show that, for a fixed θ and small h, the “inf-sup” of

r sin ϕ + hb_{2}r^{2} cos^{2} ϕ cos^{2} θ + hb_{3}r^{2} cos^{2} ϕ cos^{2} θ

always occurs at ϕ = 0. Then H(hb) = hH(b) and

H(b) = inf_{B∈B} sup_{y∈B} (b_{2}y_{2}^{2} + b_{3}y_{3}^{2}) = inf_{θ} sup_{0≤r≤1} r^{2}(b_{2} cos^{2} θ + b_{3} sin^{2} θ).

Deduce that b_{2} < 0 or b_{3} < 0 implies H(b) = 0 and that 0 ≤ b_{2} ≤ b_{3} implies H(b) = b_{2}.

(ii) Since H(0) = 0, deduce from Theorem 11.1 that

IS_{h}u(x) = u(x) + h^{2} \frac{1}{2} κ_{1}^{+}(u)(x)|Du(x)| + O(x, h^{3}).  (10.9)

Exercise 10.3. Let B be the two symmetric points (1, 0, 0) and (−1, 0, 0). Use the techniques of Exercise 11.2 to show that

IS_{h}u = u + h^{2} \frac{1}{2} min\{κ_{1}(u), κ_{2}(u)\}|Du| + O(h^{3});
SI_{h}u = u + h^{2} \frac{1}{2} max\{κ_{1}(u), κ_{2}(u)\}|Du| + O(h^{3});
IS_{h}(SI_{h})u = u + h^{2} \frac{1}{2} (κ_{1}(u) + κ_{2}(u))|Du| + O(h^{3}).

The last formula shows that the operator IS_{h}(SI_{h}) involves the mean curvature of u at x.

Exercise 10.4. Let B consist of two orthogonal segments of length two centered at the origin.

10.4. COMMENTS AND REFERENCES 173

(i) Show that

IS_{h}u = u + h^{2} \frac{1}{2} ( \frac{κ_{1}(u) + κ_{2}(u)}{2} )^{+}|Du| + O(h^{3});
SI_{h}u = u + h^{2} \frac{1}{2} ( \frac{κ_{1}(u) + κ_{2}(u)}{2} )^{−}|Du| + O(h^{3}).

(ii) Show that you can get the mean curvature by simply taking B to be the four endpoints of the orthogonal segments. Check that another possibility for obtaining the mean curvature is to alternate these operators or to add them.

10.4 Comments and references

The references for this chapter are essentially the same as those for Chapter 10. The main theorem on the asymptotic behavior of morphological filters was first stated and proved in [127] and [128]. The examples developed in Exercises 11.2, 11.3, and 11.4 have not been published elsewhere. The consistency of Gaussian smoothing followed by thresholding and mean-curvature motion was proved in increasing mathematical sophistication and generality by Merriman, Bence, and Osher [195], Mascarenhas [192], Barles and Georgelin [37], and Evans [93]. Our presentation is slightly more general than the ones cited because we allow any nonnegative weight function in the Schwartz class. The most general result was given by Ishii in [136].

174CHAPTER 10. ASYMPTOTIC BEHAVIOR OF CONTRAST-INVARIANT OPERATORS IN ℝ^{N}

Chapter 11

Affine-Invariant Mathematical Morphology

In Chapter 8, we introduced a class of simple set and function operators called erosions and dilations. These operators were defined by a single structuring set. They were contrast invariant and translation invariant, but they are not affine invariant. In this chapter, we introduce set operators, also called erosions and dilations, that are affine invariant.

Our interest in affine-invariant smoothing, like our interest in contrast- and translation-invariant smoothing, is based on practical considerations. When we take a photograph of a plane image, say, a painting, the image is projected onto the focal plane of the camera. If the camera is an ideal pin-hole device, then this is a projective transformation where the center of projection is the pin hole. In any case, it approximates a projective transformation. If we are far removed from the plane of the painting, then the focal plane of the camera approximates the plane at infinity, and the transformation looks like an affine transformation. For a more common example, we note that most copy machines, fax machines, and scanners introduce a slight affine distortion. Thus, we would like the smoothing to be affine invariant so it is “blind” to any deformations introduced by these processes. It would be nice to have smoothing that is invariant under the full projective group, but we will see later (Chapter 20) that this is not possible.

11.1 Notes on geometry

Isometries, by definition, preserve the distance between points, and hence, preserve the angle between vectors. In a finite dimensional space ℝ^{N}, any isometry can be represented by x ↦ Ax + a, where A is an orthogonal matrix and a is a fixed vector. These transformations include all of the rigid motions of ℝ^{N} plus reflections. Classical Euclidean geometry in ℝ^{2} is concerned with the objects that are invariant under these transformations. If we loosen the requirement that the matrix A be orthogonal and assume only that it is nonsingular, then we have generalized Euclidean motions to affine motions, and the distance between points is no longer an invariant. However, there are affine invariants, and the most important from our point of view are that parallel lines are mapped

175 176CHAPTER 11. AFFINE-INVARIANT MATHEMATICAL MORPHOLOGY

into parallel lines and finite points are mapped into finite points. Furthermore, if the determinant of A is one, |A| = 1, then the transformation preserves area: A unit square is mapped into a parallelogram whose area is one. Note, however, that this parallelogram can be arbitrarily long.

If we jump to projective geometry and the projective plane, then parallel lines are not necessarily preserved. If, however, the point of projection (the central point) is a point on the line at infinity (or, said another way, the point at infinity in a given direction), then the mapping reduces to an affine mapping. Thus, affine transformations are a special class of projective transformations. This means that affine geometry can be considered a generalization of Euclidian geometry or a specialization of projective geometry. In any case, the most we can ask is the our transforms be affine invariant and thus preserve parallel lines and area. Incidentally, this is the view taken in classical Chinese drawing, which tends always to display scenes as seen from a distance.

11.2 Affine-invariant erosions and dilations

Everything in this chapter will take place in ℝ^{2}, so the following definition is given for ℝ^{2}. Recall that the group of all linear affine transformations A : ℝ^{2} → ℝ^{2} is called the full linear group, F L(ℝ^{2}), and is represented by all 2 × 2 nonsingular matrices. The subgroup of those transformations represented by A with determinant one, |A| = 1, is called the special linear group; it is often denoted by SL(ℝ^{2}).

Our goal is to define erosions and dilations that are invariant under SL(ℝ^{2}). Any attempt to do so using Euclidean distance is doomed to failure, since the distances between points in any given direction can be arbitrarily augmented. It is thus necessary to base the definition of affine-invariant erosions and dilations on some affine invariant, and the most obvious one to use is area. This leads to the notion of the affine-invariant distance between a point and a set. We begin with the definition of a chord-arc set.

Definition 11.1. Let X be a subset of ℝ^{2} and let Δ be a straight line in ℝ^{2}. Any connected component of X \ Δ = X ∩ Δ^{c} is called a chord-arc set defined by X and Δ.

Exercise 11.1. Chord-arc sets take their name from the case where X is a disk, in which case a chord-arc set is called a segment. What are the chord-arc sets if (i) X is an open disk and (ii) X is a closed disk? What is the situation when X is an arc of a circle or a segment of a disk? Exercise 11.2. Suppose that x ∈ (\overline{X})^{c} ∩ Δ. Show that there are two, and only two, chord-arc sets defined by (\overline{X})^{c} = (X^{c})^{∘} and Δ that contain x in their boundary. (Note that we are assuming that (\overline{X})^{c} ≠ ∅.)

We are interested first is some special chord-arc sets that will be used to define the affine distance from a point x to a set X. From Exercise 12.2, we know that there are only two chord-arc sets defined by (\overline{X})^{c} and Δ that contain x in their boundary, if \overline{X} ≠ ℝ^{2}. In this case, we call these two sets CA_{1}(x, Δ, X) and CA_{2}(x, Δ, X), and we order them so

area(CA_{1}(x, Δ, X)) ≤ area(CA_{2}(x, Δ, X)).

11.2. AFFINE-INVARIANT EROSIONS AND DILATIONS 177

Figure 11.1: Affine distance to a set.

Definition 11.2. Let X be an arbitrary subset of ℝ^{2} and let x be an arbitrary point in ℝ^{2}. We define the affine distance from x to X to be δ(x, X) = inf_{Δ}[area(CA_{1}(x, Δ, X))]^{1/2} if x ∈ (\overline{X})^{c}(≠ ∅) and δ(x, X) = 0 otherwise. (See Figure 11.1.)

The power 1/2 is taken so δ has the “dimension” of a distance. Notice that δ is well defined because CA_{1}(x, Δ, X) and CA_{2}(x, Δ, X) are Borel sets and thus measurable. And notice that δ(x, X) can be infinite: Take X to be convex and compact and x ∉ X. Then all chord-arc sets defined by a straight line Δ through x have infinite area. Notice also that δ(x, ∅) = +∞ and δ(x, ℝ^{2}) = 0.

Definition 11.3. The affine a-dilation D̃_{a} and the affine a-erosion Ẽ_{a} are set operators defined on X ⊂ ℝ^{2} by D̃_{a}X = {x | δ(x, X) ≤ a^{1/2}} and Ẽ_{a}X = {x | δ(x, X^{c}) > a^{1/2}}.

Exercise 11.3. Check that D̃_{a}ℝ^{2} = Ẽ_{a}ℝ^{2} = ℝ^{2}. Show that Ẽ_{a}X = (D̃_{a}X^{c})^{c}. (Recall Exercise 8.1(ii).) This relation shows that eroding a set and dilating its complement yield complementary sets. This is a useful symmetry, since the same shape can appear as an upper level set or as the complement of an upper level set, depending on whether it is darker or lighter then the background.

The names we have used for the operators Ẽ_{a} and D̃_{a} are not standard nomenclature in mathematical morphology. Indeed, in mathematical morphology, a dilation must commute with set union and an erosion is expected to commute with set intersection. It is easy to check that the Ẽ_{a} and D̃_{a} do not satisfy these properties.^{1} Nevertheless, we will use these names. There should be no confusion, and these operators are clearly sound generalizations of the corresponding Euclidean erosions and dilations discussed in Chapter 8.

1We thank Michael Schmidtt for pointing this out to us.

178CHAPTER 11. AFFINE-INVARIANT MATHEMATICAL MORPHOLOGY

Proposition 11.4. Let A be a linear affine transformation such that |A| > 0. Then Ẽ_{a} and D̃_{a} are affine-invariant monotone operators in the sense that Ẽ_{a}A = AẼ_{|A|^{−1}a} and D̃_{a}A = AD̃_{|A|^{−1}a}. In particular, Ẽ_{a}A = AẼ_{a} and D̃_{a}A = AD̃_{a} if |A| = 1.

Proof. If X ⊂ Y, then it is easily seen from the definitions that δ(x, Y) ≤ δ(x, X) for every x ∈ ℝ^{2}. It then follows from the definition of D̃_{a} that D̃_{a}X ⊂ D̃_{a}Y. Hence, D̃_{a} is monotone. The monotonicity of Ẽ_{a} follows directly from the relation Ẽ_{a}X = (D̃_{a}X^{c})^{c}.

The transformation A preserves the topological properties of the configuration determined by x, X, and Δ. In particular CA_{i}(Ax, AX, AΔ) = A(CA_{i}(x, X, Δ) for i = 1 and 2. Furthermore, area(CA_{i}(Ax, AX, AΔ)) = |A|area(CA_{i}(x, X, Δ)), if |A| > 0. This means that δ(Ax, AX) = |A|^{1/2}δ(x, X). It follows from the definition of D̃_{a} that D̃_{a}AX = AD̃_{|A|^{−1}a}X, or that D̃_{a}A = AD̃_{|A|^{−1}a}. The result for Ẽ_{a} follows from the result for D̃_{a} and the relations AX^{c} = (AX)^{c} and Ẽ_{a}X = (D̃_{a}X^{c})^{c}.

We have defined the operators D̃_{a} and Ẽ_{a} on all subsets of ℝ^{2}, but it turns out that Ẽ_{a} maps many sets to ∅ and D̃_{a} maps many sets to ℝ^{2}. This is shown by the next result.

Proposition 11.5. For any X ⊂ ℝ^{2}, Ẽ_{a}X = Ẽ_{a}X^{∘} and D̃_{a}X = D̃_{a}\overline{X}, where X^{∘} denotes the interior of X and \overline{X} denotes the closure of X.

Proof. We will first prove the result about Ẽ_{a}. Since Ẽ_{a} is monotone, Ẽ_{a}X^{∘} ⊂ Ẽ_{a}X. Now, x ∈ Ẽ_{a}X if and only if δ(x, X^{c}) > a^{1/2}. By the definition, δ(x, Y) = δ(x, \overline{Y}), so x ∈ Ẽ_{a}X if and only if δ(x, \overline{X^{c}}) > a^{1/2}. Since \overline{X^{c}} = (X^{∘})^{c}, this means that δ(x, (X^{∘})^{c}) > a^{1/2}, which proves that x ∈ Ẽ_{a}X^{∘}. That D̃_{a}X = D̃_{a}\overline{X} follows from the two identities Ẽ_{a}X = (D̃_{a}X^{c})^{c} and ((X^{c})^{∘})^{c} = \overline{X}.

This result means that affine erosion erases all boundary points. It also means that Ẽ_{a}X = ∅ if X^{∘} = ∅ and that D̃_{a}X = ℝ^{2} if X is dense in ℝ^{2}. If we let X_{i}^{∘}, i ∈ I, denote the connected components of X^{∘}, then the Ẽ_{a}X_{i}^{∘} are disjoint Ẽ_{a}X^{∘} = ⋃_{i∈I} Ẽ_{a}X_{i}^{∘}.

We have not said so explicitly, but it should be clear that Ẽ_{a}(X+b) = Ẽ_{a}X+b and D̃_{a}(X+b) = D̃_{a}X+b for any fixed b ∈ ℝ^{2}, which means that these operators are translation invariant. Since these operators are also monotone, and since D̃_{a}ℝ^{2} = Ẽ_{a}ℝ^{2} = ℝ^{2}, we can use Theorem 7.1 (Matheron) to represent them in the form

T X = ⋃_{B∈B} ⋂_{y∈B}(X − y) = {x | x + B ⊂ X for some B ∈ B},

where B = {X | 0 ∈ T X}. The task reduces to characterizing the set of structuring elements B. We know that 0 ∈ Ẽ_{a}X if and only if Ẽ_{a}X ≠ ∅ and δ(0, X^{c}) > a^{1/2}, and this, and the results stated above, lead to the following definition.

Definition 11.6. A set B is an affine structuring element if B is open and connected, 0 ∈ B, and δ(0, B^{c}) > 1. We denote the set of all affine structuring elements by B_{aff}.

11.2. AFFINE-INVARIANT EROSIONS AND DILATIONS 179

If A is a linear affine transformation and B is an affine structuring element, then δ(0, (AB)^{c}) = δ(0, AB^{c}) = |A|^{1/2}δ(0, B^{c}), and AB is a structuring element if |A|^{1/2} ≥ 1. In particular, AB_{aff} = B_{aff} for all A ∈ SL(ℝ^{2}).

Proposition 11.7. For every set X ⊂ ℝ^{2},

Ẽ_{a}X = ⋃_{B∈B_{aff}} ⋂_{y∈a^{1/2}B}(X − y) = {x | x + a^{1/2}B ⊂ X for some B ∈ B_{aff}}.

Proof. By Proposition 12.5 and the remark following it, we know that

Ẽ_{a}X = Ẽ_{a}X^{∘} = ⋃_{i∈I} Ẽ_{a}X_{i}^{∘}.

This means that we may assume that X is open and connected. The result then follows almost directly from Matheron’s theorem. We need only check the scaling. Take a = 1. Then 0 ∈ Ẽ_{1}X if and only if Ẽ_{1}X ≠ ∅ and δ(0, X^{c}) > 1, so by definition the set of structuring elements for Ẽ_{1} is B_{aff}. The structuring elements for Ẽ_{a} are the sets {X | 0 ∈ Ẽ_{a}X}. These are just the sets for which δ(0, X^{c}) > a^{1/2}, which are exactly the sets a^{1/2}B, B ∈ B_{aff}.

There is another way to characterize an affine structuring element B that is perhaps more geometric. The condition δ(0, B^{c}) > 1, implies, by definition of δ, that for any straight line Δ through the origin, the two connected components of ℝ^{2} \ (B^{c} ∪ Δ)(= B \ Δ) that contain the origin in their boundary, CA_{1}(0, Δ, B^{c}) and CA_{2}(0, Δ, B^{c}), satisfy the relation 1 < δ(0, B^{c}) ≤ CA_{1}(0, Δ, B^{c}) ≤ CA_{1}(0, Δ, B^{c}). This means that all chord-arc sets defined by B and Δ, where Δ contains the origin, have area greater than some b > 1.

Conversely, if we assume that, for any Δ through the origin, the two connected components of B \ Δ that have the origin in their boundaries have areas greater than some b > 1, then δ(0, B^{c}) > 1.

This gives us an equivalent definition for the affine structuring elements: A set B is an affine structuring element if it is open, connected, and contains the origin, and if, for every straight line Δ through the origin, the two connected components of B \ Δ that contain the origin in their bondary each have area greater than some number b > 1, where b depends only on B. (See Figure 11.2.)

It might seem that Ẽ_{a}X could be obtained from X by removing all chord-arc sets C defined by X and Δ that have area less than or equal to a, but this is not quite the case. Nevertheless, from the practical numerical point of view, this is a way to perform an affine erosion. To have this idea mathematically correct, it is necessary to make some assumptions. Recall that affine erosion removes all of the boundary points of a set X, and this is not necessarily because these points are in a chord-arc set with small area, but because a boundary point is in \overline{X} and thus, by the definition of affine distance, δ(x, X^{c}) = 0. This leads us to assume that X is open, and from the results above, we may as well assume that X is also connected. We will also assume that X is bounded, which is surely true in practice. Even with these assumptions, removing small chord-arc sets defined by X and lines Δ will not complete the erosion. A simple example is given by taking X to be an open disk, say, D(0, 1). Removing all chord-arc sets with area less than or equal to a, for a small a, will leave a closed disk D(0, r), whereas Ẽ_{a}D(0, 1) is the open disk D(0, r). The point is that removing small 180CHAPTER 11. AFFINE-INVARIANT MATHEMATICAL MORPHOLOGY

Figure 11.2: An affine structuring element: All lines through 0 divide B into several connected components. The two of them which contain 0 in their boundary have area larger or equal to b.

chord-arc sets can miss some points, but since these points are always in Δ, we need only make a small modification. Thus, if C is a chord-arc set defined by X and Δ, define C^{∗} by C^{∗} = C ∪ (\overline{C} ∩ Δ). Then we have the following result.

Proposition 11.8. Assume that X is open, connected, and bounded. Then Ẽ_{a}X can be obtained from X by removing all of the sets C^{∗}, where C is a chord-arc set defined by X and Δ that has area less then or equal to a.

Proof. Let C be any chord-arc set defined by X and a line Δ such that area(C) ≤ a. Then we claim that x ∈ C^{∗} implies that δ(x, X^{c}) ≤ a^{1/2}. To see this, let Δ′ be the line parallel to Δ that contains x. The lines Δ and Δ′ each define two open half-planes, which we denote by R and L and R′ and L′. Without loss of generality, we may assume that Δ′ ⊂ R, or equivalently, that Δ ⊂ L′. (See Figure 11.3.)

Consider the sets CA_{i}(x, Δ′, X^{c}), i = 1, 2. These are the connected components of ℝ^{2} \(\overline{X^{c}} ∪Δ′) = (\overline{X^{c}})^{c} ∩(Δ′)^{c} = X^{∘} \Δ′ that contain x in their boundaries. One of these sets, say, CA_{1}(x, Δ′, X^{c}) is in R∩R′. Now \overline{CA_{1}(x, Δ′, X^{c})}∩ C ≠ ∅, since both sets contain x. This means that CA_{1}(x, Δ′, X^{c}) ∪ C is connected, and since C was a maximal connected set in X \ Δ we conclude that CA_{1}(x, Δ′, X^{c}) ⊂ C. Thus the area of CA_{1}(x, Δ′, X^{c}) is less than or equal to a, so δ(x, X^{c}) ≤ a^{1/2}, and x ∉ Ẽ_{a}X.

For the converse, we must show that if a point was eroded, then it had to have been in a set C^{∗}, where C is a chord-arc set with area less than or equal to a. Our proof uses a result that we prove in the next exercise, namely, that if δ(x, X^{c}) ≤ a^{1/2}, then there is a line Δ that contains x such that δ(x, X^{c}) = [area(C)]^{1/2}, where C is a chord-arc set defined by X and Δ. Then for this C, x ∈ C^{∗} and area(C) ≤ a.

11.2. AFFINE-INVARIANT EROSIONS AND DILATIONS 181

Figure 11.3: Illustration for Proposition 12.8.

Exercise 11.4. Assume that X is open and bounded, and, since the argument involves only one component of X, assume that X is connected. We refer to Figure 11.4 below for the definitions of the various objects. Thus, Δ(0) is an arbitrary line that contains x and C(0) is the connected component of X \ Δ(0) on the arrow-side of Δ(0) whose boundary contains x. C(φ) is the connected component of X \Δ(φ) on the arrow-side of Δ(φ) whose boundary contains x. Since X is open, there is an r > 0 such that the disk D(x, r) is contained in X; since X is assumed to be bounded, there is an R > 0 such that X ⊂ D(x, R).

(i) Show that C(0, φ) ⊂ C(0) and C(0, φ) ⊂ C(φ).

(ii) Show that area(C(0) ∪ C(0, φ)^{c}) → 0 as φ → 0 and similarly that area(C(φ) ∪ C(0, φ)^{c}) → 0 as φ → 0.

(iii) Deduce that area(C(φ)) → area(C(0)) as φ → 0.

This shows that area(C(φ)) is a continuous function of φ. Thus, the inf_{Δ(φ)} area(C(φ)) is attained, which means that for every x ∈ X there is some φ such that δ(x, X^{c}) = [area(C(φ))]^{1/2}.

Exercise 11.5. The aim of this exercise is to prove that a one-to-one mapping Ã : ℝ^{2} → ℝ^{2} that preserves parallelism must be of the form Ã(x) = Ax + b, where A is a linear mapping and b is a fixed vector. The preservation of parallelism is defined as follows: If any four points x_{1}, x_{2}, x_{3}, and x_{4} satisfy x_{1} − x_{2} = λ(x_{3} − x_{4}) for some λ ∈ ℝ, then there exists a μ ∈ ℝ such that Ãx_{1} − Ãx_{2} = μ(Ãx_{3} − Ãx_{4}).

(i) Let i and j be the usual orthonormal basis for ℝ^{2} and write x = xi + yj. Define A by Ax = Ãx − Ã0. Show that there are two real function μ_{i} : ℝ → ℝ, i = 1, 2, such that A(xi) = μ_{1}(x)Ai and A(yj) = μ_{2}(y)Aj.

(ii) Notice that A preserves parallelism and that A0 = 0.

(iii) Show that Ax = μ_{1}(x)Ai + μ_{2}(y)Aj.

(iv) Show that μ_{1}(λ) = μ_{2}(λ).

(v) We wish to show that μ_{1}(x) = x. One way to do this is to prove that μ_{1} : ℝ → ℝ is an isomorphism. This can be done using the fact that x_{1} − x_{2} = λ(x_{3} − x_{4}) implies that Ax_{1} −Ax_{2} = μ(Ax_{3} −Ax_{4}). Once you have shown that μ_{1} : ℝ → ℝ is an isomorphism, you can quote the result that says any isomorphism of ℝ onto itself must be x ↦ x, or you can prove this result.

182CHAPTER 11. AFFINE-INVARIANT MATHEMATICAL MORPHOLOGY

Figure 11.4: Definition of C(0, φ). The set C(0, φ) is the connected component of X \ (Δ(0) ∪ Δ(φ)) that lies in the direction of the arrows.

11.3 Comments and references

Shape-recognition algorithms in the plane are clearly more robust if they are affine invariant, if only because most optical devices that copy plane images (photocopiers) or that convert plane images to digital information (scanners, faxes) create a slight affine distortion. Also, all diffeomorphisms are locally affine. Affine-invariant techniques for matching shapes are described in [148]; discussions of the role of affine and projective invariance for object recognition can be found in [44], [270], and [157]. Corners and T-junctions can appear in images with arbitrary angles, and the detection of angles between straight lines should be affine invariant. Algorithms for affine-detection of angles are proposed in [42], [13], [262], and [85]. See Merriman, Bence, and Osher [196] for a very original numerical view for filtering multiple junctions. Because of the relevance to computer vision, there has been considerable research devoted to looking for affine-invariant definitions of classical concepts in geometric measure theory and integral geometry. An interesting attempt to define an “affine-invariant length” and an “affine-invariant dimension” analogous to Hausdorff lengths and dimensions is given in [86]. The diameters of the sets of a Hausdorff covering are simply replaced by their areas. Several attempts to define affine-invariant analyses of discrete sets of points are described in [117] and [236]. An affineinvariant symmetry set (skeleton) for shapes is defined in [119]; the 1/3 power 11.3. COMMENTS AND REFERENCES 183

law of planar motion perception and generation is related to affine invariance in [223]. Some of the techniques on affine erosions and dilations presented in this chapter were announced in [178]. We have made liberal use of the Matheron formalism for monotone set operators [194].

184CHAPTER 11. AFFINE-INVARIANT MATHEMATICAL MORPHOLOGY

Chapter 12

Localizable Structuring Elements and the Local Maximum Principle

Given a set of structuring elements B, the scaled operators IS_{h} defined by

IS_{h}u(x) = inf_{B∈B} sup_{y∈x+hB} u(y)

are immediately translation invariant and contrast invariant. Furthermore, if the elements of B are uniformly bounded, then the operators satisfy an important local property that we have not yet emphasized: If two functions u and v are such that u(y) ≤ v(y) for all y in some disk D(x, r), then for sufficiently small h, hB ∈ D(0, r), and IS_{h}u(x) ≤ IS_{h}v(x). This is a special case of the local maximum principle, which for bounded structuring elements goes almost un-noticed. It might seem at first glance that we would not have a local maximum principle if the structuring elements were not bounded. It turns out, however, that for a large class of unbounded structuring elements, the operators IS_{h} behave as if they were local operators—in the sense that they satisfy a local maximum principle. For example, if the operators IS_{h} are affine invariant, then the affine-invariant structuring elements B cannot be bounded. Indeed, affine invariance allows an element B ∈ B to be stretched arbitrarily far in any direction: The matrix A = ( \begin{matrix}ε & 0 \\ 0 & 1/ε \end{matrix} ), where ε is small, followed by a rotation, does the job. Nevertheless, there are affine-invariant operators that satisfy a local maximum principle. In particular, we will show that this is the case for B = B_{aff}. In general, the application of operators that satisfy this property involves an error term:

IS_{h}u(x) ≤ IS_{h}v(x) + o(h^{β}).

We are going to define a property of structuring elements B called localizability, and even though B may contain arbitrarily large elements, or even unbounded elements, if it is localizable, then the inf-sup operators defined by B will satisfy a local maximum principle. The importance of the local maximum principle for our program will become clear in the chapter on viscosity solutions. Since our focus is on affine-invariant operators, we will apply the concept

185 186CHAPTER 12. LOCALIZABILITY AND THE MAXIMUM PRINCIPLE

of localizability only to families of affine-invariant structuring elements, but the reader should keep in mind that concept is applicable to other situations. For example, the structuring elements associated with a median filter defined by a k that does not have compact support, are unbounded, but they may be localizable. A case in point is the Gaussian: It does not have compact support, but it can be shown that the structuring elements are localizable.

12.1 Localizable sets of structuring elements

Recall that the Euclidean distance d between a point x and a set Y is defined by

d(x, Y) = inf_{y∈Y} |x − y|.

It will be convenient to use the following notation: D(0, ρ) will denote the open disk (or ball) {x | |x| < ρ}, and D_{a} will denote the dilation operator defined by

D_{a}(X) = {x | d(x, X) < a}.

In the notation of Chapter 8, this means that D_{a} = D_{a}, where the structuring element for D_{a} is D(0, 1). Note that if X is open and connected, then D_{a}(X) is open and connected. We will write ∂X to denote the boundary of X. These definitions and notation are used to define the concept of a set of structuring elements being localizable.

For convenience, we introduce two set operators:

• C_{0}[X] = the connected component of X that contains the origin.

• C_{0}^{∂}[X] = the connected component of X that contains the origin in its boundary.

The set X will always be open and either contain the origin or contain the origin in its boundary. Note that these operators commute with scaling: For example, C_{0}[hX] = hC_{0}[X]. Note also that these operators are monotone.

Definition 12.1. Let α > 0 be a positive constant. Assume that B is a set of structuring elements whose members are open and contain the origin. B is said to be α-localizable if there are two constants c > 0 and R > 0, where c and R depend on B, such that for every ρ > R the following conditions holds: For each B ∈ B, there is a B′ ∈ B such that

(i) B′ ⊂ D(0, ρ);

(ii) B′ ⊂ D_{c/ρ^{α}}(C_{0}[B_{ρ}]), where B_{ρ} = B ∩ D(0, ρ).

The constant α is called the exponent of localizability.

We wish to emphasize that our definition of localizable includes the assumption that the elements of B are open and contain the origin. Note also that a localizable set B contains bounded members. Also, it may happen that C_{0}[D_{c/ρ^{α}}(C_{0}[B_{ρ}]) ∩ D(0, ρ)] is itself a member of B for all B ∈ B; indeed, the proof of Proposition 13.4 shows that this is the case if B = B_{aff}. The next result shows how the concept of localizability scales.

12.2. THE LOCAL MAXIMUM PRINCIPLE 187

Proposition 12.2. Let h, 0 < h ≤ 1, be a scaling factor and assume the notation of Definition 13.1. A set of structuring elements B is α-localizable if and only if there are constants c > 0 and R > 0 such that for all r > 0 and all h < r/R the following conditions holds: For each B ∈ hB, there is a B′ ∈ hB such that

(i′) B′ ⊂ D(0, r);

(ii′) B′ ⊂ D_{ch^{α+1}/r^{α}}(C_{0}[B_{r}]), where B_{r} = B ∩ D(0, r).

Proof. Assume the conditions of the proposition. Then r/R > 1 for all r > R. Since h can be any number in the range 0 < h < r/R, the conditions of the proposition are true for h = 1. By letting ρ = r, we have the conditions of Definition 13.1.

To prove that the conditions of Definition 13.1 imply the conditions of the proposition, let ρ = r/h. Then the statement “all ρ > R” is equivalent to the statement “all r > 0 and all h < r/R.” Next, we need to see how D_{c/ρ^{α}}(X) scales:

hD_{c/ρ^{α}}(X) = {hx | d(x, X) < c/ρ^{α}}
= {hx | d(hx, hX) < ch/ρ^{α}}
= {y | d(y, hX) < ch/ρ^{α}}
= D_{ch/ρ^{α}}(hX).

If X = C_{0}[B_{ρ}], then hD_{c/ρ^{α}}(C_{0}[B_{ρ}]) = D_{ch^{α+1}/r^{α}}(C_{0}[hB_{ρ}]), where hB_{ρ} = B_{r}. In other words, B′ ∈ D_{c/ρ^{α}}(C_{0}[B_{ρ}]) implies that hB′ ∈ D_{ch^{α+1}/r^{α}}(C_{0}[B_{r}]), which shows that (ii) implies (ii′).

We will use Definition 13.1 and its scaled version, Proposition 13.1, to prove two results: The first is that if B is localizable, then the IS_{h} satisfy the local maximum principle; the second is that B_{aff} is 1-localizable.

12.2 The local maximum principle

While the notion of α-localizability has an important role in mathematical morphology, we are concerned in this book only with the 1-localizabable families of structuring elements. Thus, from this point, we assume that α = 1 and leave the general cases as exercises.

Lemma 12.3 (local maximum principle). Let B be a 1-localizable set of structuring elements with the associated constants c > 0 and R > 0. Assume that the functions u and v satisfy a Lipschitz condition on a disk D(x, r) with Lipschitz constant L. If u(y) ≤ v(y) for y ∈ D(x, r) and if h < r/R, then

IS_{h}u(x) ≤ IS_{h}v(x) + Lc\frac{h^{2}}{r} and SI_{h}u(x) ≤ SI_{h}v(x) + Lc\frac{h^{2}}{r} .

Proof. For notational convenience, we take x = 0. Then

IS_{h}v(0) = inf_{B∈hB} sup_{y∈B} v(y) ≥ inf_{B∈hB} sup_{y∈B∩D(0,r)} v(y) ≥ inf_{B∈hB} sup_{y∈B∩D(0,r)} u(y).

(12.1)

188CHAPTER 12. LOCALIZABILITY AND THE MAXIMUM PRINCIPLE

By Proposition 13.2, for all B ∈ hB, there exists B′ ∈ hB such that B′ ⊂ D(0, r) and B′ ⊂ D_{ch^{2}/r}(C_{0}[B ∩ D(0, r)]). Thus, since u is Lipschitz with constant L and since for every point b′ ∈ B′ there is a point b ∈ B ∩ D(0, r) such that |b′ − b| < ch^{2}/r, we have

sup_{y∈B∩D(0,r)} u(y) ≥ sup_{y∈B′} u(y) − Lc\frac{h^{2}}{r} .  (12.2)

These last two inequalities imply that

IS_{h}v(0) + Lc\frac{h^{2}}{r} ≥ inf_{B∈hB} sup_{y∈B′} u(y),

where B′ is any set such that B′ ∈ hB, B′ ⊂ D(0, r), and B′ ⊂ D_{ch^{2}/r}(C_{0}[B ∩ D(0, r)]). If we denote the family of all such sets B′ associated with B by B′ = B′(B), then a more precise statement is that

IS_{h}v(0) + Lc\frac{h^{2}}{r} ≥ inf_{B∈hB} ( sup_{B′∈B′} sup_{y∈B′} u(y)).

Let inf_{B∈hB} ( sup_{B′∈B′} sup_{y∈B′} u(y)) = λ. Then we claim that inf_{B∈hB} sup_{y∈B} u(y) ≤ λ. To see this, let ε > 0 be arbitrary. By the definition of λ, there is some set B ∈ hB such that sup_{y∈B} u(y) ≤ λ + ε. Thus, inf_{B∈hB} sup_{y∈B} u(y) ≤ λ + ε, which, since ε is arbitrary, implies that inf_{B∈hB} sup_{y∈B} u(y) ≤ λ. This yields the result:

IS_{h}u(x) = inf_{B∈hB} sup_{y∈x+B} u(y) ≤ IS_{h}v(x) + Lc\frac{h^{2}}{r} .

The result for SI_{h} follows from the relation −IS_{h}(−u) = SI_{h}(u).

Note that the proof for IS_{h} does not use the fact that v is locally Lipschitz. In fact, the proof works for any v. The problem with this is that we would not have the result for SI_{h} if we did not assume that v is locally Lipschitz. Also, taking x = 0 in the proof is indeed only a notational convenience; knowing that the assumptions hold at x does not imply they hold elsewhere. Thus, as stated, the result of Lemma 13.3 is strictly local.

Lemma 13.3 and the next lemma provide the links between the localizability of structuring elements and the local properties of the associated operators. As such, they lie near the heart of our program. Their use is the key to demonstrating the asymptotic behavior of inf-sup operators defined by 1-localizable families of affine-invariant structuring elements. The local maximum principle is also used in the proof of the Barles–Souganidis theorem, Proposition 15.13, which is essential for relating the inf-sup operators to their associated PDEs via viscosity solutions of the PDEs.

Exercise 12.1. Prove the general form of Lemma 13.3: Replace the hypothesis that B is 1-localizable with the hypothesis that it is α-localizable and conclude that

IS_{h}u(x) ≤ IS_{h}v(x) + Lc \frac{h^{α+1}}{r^{α}} .

Lemma 13.3 compares the action of IS_{h} on two functions u and v. In the next lemma, we consider an operator IS_{h}^{r} that approximates IS_{h} and examine 12.2. THE LOCAL MAXIMUM PRINCIPLE 189

its action on a single function u. The approximate operator IS_{h}^{r} is defined by truncating the structuring elements B: We replace the family B with the family B_{r} = {B_{r} | B_{r} = B ∩ D(0, r), B ∈ B}. Thus,

IS_{h}^{r}u(x) = inf_{B_{r}∈hB_{r}} sup_{y∈x+B_{r}} u(y) = inf_{B∈hB} sup_{y∈(x+B)∩D(x,r)} u(y).

The local properties of IS_{h}^{r} have been imposed by definition. Later, when we apply this result, we will take r = h^{1/2}, so the error term will be Lch^{3/2}.

Lemma 12.4 (localization lemma). Let B be a 1-localizable set of structuring elements with constants c > 0 and R > 0 and assume that h < r/R. If u satisfies a Lipschitz condition in D(x, r) with constant L, then

(i) IS_{h}^{r}u(x) ≤ IS_{h}u(x) ≤ IS_{h}^{r}u(x) + Lch^{2}/r;

(ii) |IS_{h}^{r}u(x) − IS_{h}u(x)| ≤ Lch^{2}/r;

(iii) |SI_{h}^{r}u(x) − SI_{h}u(x)| ≤ Lch^{2}/r.

(iv) |SI_{h}^{r}IS_{h}^{r}u(x) − SI_{h}IS_{h}u(x)| ≤ 2Lch^{2}/r, if u is L-Lipschitz on ℝ^{N}

Proof. By taking u = v in inequality (12.1), we see that IS_{h}u(x) ≥ IS_{h}^{r}u(x). This half of (i) does not depend on u being Lipschitz on D(x, r), but the other half of (i) does depend on u being Lipschitz on D(x, r). To prove the other half of (i), we are going to follow the proof of Lemma 13.3, including the notational convenience that x = 0. In particular, we use the 1-localizability of B to establish the inequality

sup_{y∈B∩D(0,r)} u(y) ≥ sup_{y∈B′} u(y) + Lc\frac{h^{2}}{r} ,

which is (12.2). The remainder of the proof shows that

IS_{h}^{r}u(x) + Lc\frac{h^{2}}{r} ≥ IS_{h}u(x),

and this proves the other half of (i).

Inequality (ii) is just a restatement of (i). Inequality (iii) is deduced from (ii) by using the relation IS_{h}(−u) = −SI_{h}u. To prove (iv), first recall from Lemma 6.5 that if u is Lipschitz with constant L, then IS_{h}u and IS_{h}^{r}u are Lipschitz with constants no greater than L. By (ii) and (iii) we have

IS_{h}^{r}u(x) ≤ IS_{h}u(x) ≤ IS_{h}^{r}u(x) + Lch^{2}/r;  (12.3)
SI_{h}^{r}u(x) ≤ SI_{h}u(x) ≤ SI_{h}^{r}u(x) + Lch^{2}/r.  (12.4)

Replacing u with IS_{h}u in (12.4) and applying SI_{h}^{r} to (12.3) shows that

SI_{h}^{r}IS_{h}^{r}u(x) ≤ SI_{h}IS_{h}u(x) ≤ SI_{h}^{r}IS_{h}^{r}u(x) + 2Lch^{2}/r,

which proves (iv).

The statements and proofs of Lemmas 13.3 and 13.4 are strictly local. There are, however, immediate global generalizations, and since these more general results are important for later applications, we give them a precise statement for future reference.

190CHAPTER 12. LOCALIZABILITY AND THE MAXIMUM PRINCIPLE

Lemma 12.5. Let K be an arbitrary set and assume that the function u and v in Lemmas 13.3, 13.4(i), 13.4(ii), and 13.4(iii) are L-Lipschitz in D(x, r) for every x ∈ K. Then the results of these lemmas are true uniformly for x ∈ K.

These uniform results need no special proofs. One merely rereads the proofs of the local lemmas and notes that, if the same hypotheses hold at each point x ∈ K, then the results are true for each x with exactly the same error term. When we apply Lemma 13.5, K will be compact, but clearly this is not a necessary condition for the lemma.

The next result is a direct consequence of Lemma 13.3. It allows us to fix an optimal relation between the localization scale r and the operator scale h. Again, it is a local result that can easily be made uniform.

Lemma 12.6. Let B be a 1-localizable set of structuring elements with constants c > 0 and R > 0. Let u and v be two continuous functions that satisfy Lipschitz conditions with the same constant L on a disk D(0, r). If

|u(x) − v(x)| ≤ C|x|^{3}

for x ∈ D(0, r), and if h ≤ r^{2} and h < 1/R^{2}, then

|IS_{h}u(0) − IS_{h}v(0)| ≤ (C + Lc)h^{3/2}.

Proof. The relation v(x)−Cr^{3} ≤ u(x) ≤ v(x)+Cr^{3} is true for all x ∈ D(0, r), so we can apply Lemma 13.3 and conclude that

IS_{h}v(0) − Cr^{3} − Lc\frac{h^{2}}{r} ≤ IS_{h}u(0) ≤ IS_{h}v(0) + Cr^{3} + Lc\frac{h^{2}}{r}

for h < r/R. This argument is also true for 0 < s ≤ r, if we have h < s/R. So, in particular, if we take s = h^{1/2} ≤ r and h < s/R, that is, h < 1/R^{2}, we have

IS_{h}v(0) − Ch^{3/2} − Lch^{3/2} ≤ IS_{h}u(0) ≤ IS_{h}v(0) + Ch^{3/2} + Lch^{3/2},

which proves the result.

Here, we have taken the point of view that r is given, and we ask that h = r^{2}. In other situation, we may take the opposite view and ask that r be determined by h. This is the case, for example, if we are able to choose the size of r for the localized operator IS_{h}^{r}.

The main application of Lemma 13.4 is to reduce the asymptotic analysis of the operator IS_{h} as h → 0 to the case where it is applied to quadratic polynomials. (We have seen in Chapters 10 and 11 how this kind of analysis works in the case of structuring elements that are bounded and isotropic.)

12.3 B_{aff} is 1-localizable

We are going to prove that B_{aff} is 1-localizable, but to make things as transparent as possible, we first do some geometry. Thus, consider Figure 12.1.

We are interested in the area of D_{c/ρ}([0, z]) ∩ D(0, ρ) ∩ Δ^{c}. This is the area of the figure ABCD. In what follows, c > 0 is a constant. For the figure to 12.3. B_{AFF} IS 1-LOCALIZABLE 191

Figure 12.1: Dilation of [0, z].

make sense, we must assume that ρ^{2} ≥ c. We want a lower bound for the area of the set ABCD that will hold for any β ∈ [0, π/2]. (In the limit case β = π/2, and z ∈ Δ.) We also wish to compare this area with that of another set, and the way we do this limits us to using the set A0zD. (The reason for this will become clear.) Denote the area of A0zD by A = A(α, β). Thus, assuming ρ^{2} ≥ c, we have

A(α, β) = \frac{c}{2}(β sin α + \frac{α}{sin α} + cos α).

This is written in terms of α and β because it is easy to see what happens as α ranges from zero to π/2. This is equivalent to ρ going from +∞ to \sqrt{c}, which is just the range of interest. The smallest value of the function

f(α, β) = β sin α + \frac{α}{sin α} + cos α

for 0 ≤ α ≤ π/2, 0 ≤ β ≤ π/2 occurs at α = π/2, β = 0, and A(π/2, 0) = (π/4)c for these values.

To avoid repeating it, we assume that the set D(0, ρ) ∩ Δ^{c} always denotes the same open half-disk, and all of the sets we consider are understood to lie on the same side of Δ as D(0, ρ) ∩ Δ^{c}.

Parameterize the segment [0, z] so the points are represented by tz, t ∈ [0, 1]. Let L(t) denote the line orthogonal to [0, z] at tz. Let y_{t} be any point on L(t) such that y_{t} ∈ D(0, ρ)∩Δ^{c}, and consider the set D(y_{t}, c/ρ)∩D(0, ρ)∩Δ^{c}. Then the open segment L(t) ∩ D(y_{t}, c/ρ) ∩ D(0, ρ) ∩ Δ^{c} is always at least as long as the segment on the same line defined by D(tz, c/ρ) ∩ L(t) ∩ (A0zD). Note that this is true for all t ∈ [0, 1], even for those t close to one.

Lemma 12.7. Assume the geometry and notation of Figure 12.1. Let Γ be any Jordan arc connecting the origin and z and lying completely in the set D(0, ρ) ∩ Δ^{c}, except for the end points. Then

area(D_{c/ρ}(Γ) ∩ D(0, ρ) ∩ Δ^{c}) ≥ area(A0zD) ≥ \frac{π}{4} c

192CHAPTER 12. LOCALIZABILITY AND THE MAXIMUM PRINCIPLE

whenever ρ^{2} ≥ c.

Proof. Let f denote the characteristic function of D_{c/ρ}(Γ) ∩ D(0, ρ) ∩ Δ^{c} = G. Then

area(G) = ∫_{ℝ^{2}} f(x, y) dx dy.

By Fubini’s theorem,

area(G) = ∫_{ℝ} ( ∫_{ℝ} f(x, y) dy) dx,

and

area(G) ≥ \frac{c}{2}β sin α + ∫_{0}^{1} ( ∫_{ℝ} f(tz, y) dy) dt.

For fixed t,

∫_{ℝ} f(tz, y) dy = ∫_{l(t)} f(tz, y) dy,

where l(t) = G ∩ L(t). We know that l(t) contains a line segment at least as long as (A0zD) ∩ L(t), so

area(G) ≥ \frac{c}{2}β sin α + area(0zDE) = area(A0zD).

Note that in adding the term (c/2)β sin α to the area of G, we use the fact that the origin is a point of Γ.

Ostensibly, this lemma has little to do with B_{aff}. The lemma only compares the area of D_{c/ρ}(Γ) ∩ D(0, ρ) ∩ Δ^{c}, where Γ is a Jordan arc that connects the origin to z ∈ ∂D(0, ρ), with the area of D_{c/ρ}([0, z]) ∩ D(0, ρ) ∩ Δ^{c}. This is a purely geometric result, however, the application to B_{aff} is direct.

Before stating and proving the theorem, we note that all of the connected sets involved in the proof are open and thus arcwise connected. As usual, Δ denotes a straight line through the origin, and Δ^{c} always denotes the same open half-plane. If A is an open set that contains the origin, then the set A ∩ Δ^{c} will contain a half-neighborhood D(0, ε) ∩ Δ^{c} for some ε > 0.

Proposition 12.8. B_{aff} is 1-localizable.

Proof. We must exhibit a c > and an R > 0 such that the conditions of Definition 13.1 hold. Taking a clue from Lemma 13.6, we wish to have c > 4/π, so we take c = 2 and R = \sqrt{2}. These are not the “best” constants; we only claim that they work.

Let B be any element of B_{aff}. Then B is open and connected, B contains the origin, and δ(0, B^{c}) > 1, or equivalently, given any Δ through the origin, the two connected components of B ∩Δ^{c} that contain the origin in their boundaries always have areas greater than or equal to b = δ(0, B^{c}).

12.3. B_{AFF} IS 1-LOCALIZABLE 193

Three open connected sets enter the proof:

B′ = C_{0}[D_{c/ρ}(C_{0}[B ∩ D(0, ρ)]) ∩ D(0, ρ)].
C′ = C_{0}^{∂}[B′ ∩ Δ^{c}].
C = C_{0}^{∂}[B ∩ Δ^{c}].

By our convention regarding the use of Δ^{c}, the sets C and C′ lie on the same side of Δ.

The plan is to show that B′ is in B_{aff}. Since B′ is open and connected, it remains to exhibit a c′ > 1 that does not depend on Δ such that area(C′) ≥ c′ > 1.

There are two cases: C ⊂ D(0, ρ) and C ⊄ D(0, ρ). If C ⊂ D(0, ρ), then, by the definition of B_{aff}, area(C) ≥ b. (This is the only place where the definition of B_{aff} is used.) In this case, C ⊂ C′, and area(C′) ≥ b > 1. Thus, c′ = b works for this case, and we are done. (The proof that C ⊂ C′ is left as an exercise.)

If C ⊄ D(0, ρ), then there is a point a ∈ C such that |a| > ρ. There is a Jordan arc γ ∈ C that connects the origin to a. (In fact, we may assume that this Jordan arc is piecewise linear.) Let t : [0, 1] → γ be a parameterization such that γ(0) = 0 and γ(1) = a. Then there is a smallest t = t_{0} such that γ(t_{0}) ∈ ∂D(0, ρ). Call this point z and let Γ denote the part of γ defined by 0 ≤ t ≤ t_{0}. The arc Γ lies in C ∩ D(0, ρ) ∩ Δ^{c}, and in particular, Γ ⊂ B ∩ D(0, ρ) ∩ Δ^{c}. It follows that D_{c/ρ}(Γ) ∩ D(0, ρ) ∩ Δ^{c} ⊂ C′.

By Lemma 13.6, area(D_{c/ρ}(Γ) ∩ D(0, ρ) ∩ Δ^{c}) ≥ (π/4)c, so area(C′) > π/2, by the definition of c. Thus, by taking c′ = min{π/2, b}, it is always true that area(C′) ≥ c′ > 1, and c′ does not depend on Δ. It follows from Definition 13.1 that B_{aff} is 1-localizable.

Exercise 12.2. We skipped over two points in the proof that the reader should check. The first was when we stated that C ⊂ C′ (the case C ⊂ D(0, ρ)), and the second was when we claimed that D_{c/ρ}(Γ) ∩ D(0, ρ) ∩ Δ^{c} ⊂ C′ (the case C ⊄ D(0, ρ)). (Hint: All of the sets in sight are open and connected, so they are arcwise connected.)

The next two exercises show that there are other affine-invariant families of structuring elements that are 1-localizable.

Exercise 12.3. Let B be an affine-invariant family of open convex sets, each of which contains the origin and has area less than one. The goal is to show that B is 1localizable. Here is one way to do this. Suppose ρ > R, where R > 0 is a constant to be determined, and that B is an element of B. If B ⊂ D(0, ρ), take B′ = B, and we are done. If not, let x be a vector such that |x| = ξ = sup_{y∈\overline{B}} |y|. Establish the coordinate system based on i = x/|x| and j = i^{⊥}. Consider the affine transformation defined by A = ( \begin{matrix}ρ/ξ & 0 \\ 0 & ξ/ρ \end{matrix} ), and show that if ρ is greater than some constant, then AB ⊂ D(ρ, 0). Let B′ = AB, which belongs to B by assumption. (Hint: Let η be the longest perpendicular distance from the x-axis to B^{c}. Relate the product ηξ to the area of B and determine a value for R that works.) Having found an R such that B′ ⊂ D(0, ρ) for ρ > R, look for a c > 0 such that B′ ⊂ D_{c/ρ}(B).

Exercise 12.4. Let B be a bounded, open, and connected set that contains the origin. Define the affine-invariant family B by B = {AB | A ∈ SL(ℝ^{2})}. Use the methods of Exercise 3.2 to show that B is 1-localizable.

194CHAPTER 12. LOCALIZABILITY AND THE MAXIMUM PRINCIPLE

12.4 Comments and references

We were vague about the domains of the various functions that appear in section 13.2. In fact, the results are true in ℝ^{N} even though we spoke of “disks” rather than “balls.” The results in section 13.3 are, however, strictly limited to ℝ^{2}.

The mathematical techniques for localizing a set of structuring elements developed in this chapter were first explained in [128], [127], and [125]. The version presented here is much simpler. The doctoral dissertations by Frédéric Cao and Denis Pasquignon contain related techniques [53, 54, 217].

Chapter 13

Asymptotic Behavior of Affine-Invariant Filters

We are going to analyze the asymptotic behavior of affine-invariant operators in much the same way we analyzed contrast-invariant isometric operators in Chapters 10 and 11. The analysis in this chapter will be in ℝ^{2}. Recall that when we say an operator T is affine invariant, we mean that T commutes with all elements of the special linear group SL(ℝ^{2}). Thus, for A ∈ SL(ℝ^{2}), we have AT u = T Au for all functions u in the domain of T, where Au is defined by Au(x) = u(Ax), x ∈ ℝ^{2}. At this point, there are two possible scenarios: Assume we are given an affine-invariant operator T that is also contrast and translation invariant and then use Theorem 7.3 to conclude that T can be represented as

T u(x) = inf_{B∈B} sup_{y∈x+B} u(y),  (13.1)

where the set of structuring elements B may be taken to be {X | 0 ∈ T X}, T X = X_{1}T 1_{X}, and where (14.1) holds almost everywhere for u in the domain of T. The other approach, which is the one we take, is to assume the set of structuring elements B is given and to define T by (14.1). This places the focus on B. With this approach, we know immediately that T is contrast and translation invariant and that it is defined on all u : ℝ^{2} → ℝ. We are, however, left with the task of proving that T is affine invariant if and only if B is affine invariant. (This is the content of Exercise 14.1.). Again, it is understood that, in our context, “affine invariant” always means “invariant with respect to SL(ℝ^{2}).”

We assume that B is a set of affine-invariant structuring elements, and we define for every u : ℝ^{2} → ℝ,

SI_{h}u(x) = sup_{B∈B} inf_{y∈x+hB} u(y);
IS_{h}u(x) = inf_{B∈B} sup_{y∈x+hB} u(y). (13.2)

SI_{h}u is considered to be an affine erosion of u, and IS_{h}u is considered to be an affine dilation of u. (Note that this nomenclature is consistent with the definitions of Ẽ_{a} and D̃_{a} in Chapter 12.) Since we have the relation SI_{h}u = −IS_{h}(−u), it suffices to study just one of these operators, and we choose to

195 196CHAPTER 13. ASYMPTOTIC BEHAVIOR OF AFFINE-INVARIANT FILTERS

investigate IS_{h}. Our main concern is the behavior of IS_{h}u(x) as h → 0 for u ∈ C^{3}(ℝ^{2}). We will prove that, if B is affine invariant and 1-localizable, then

lim_{h→0} \frac{IS_{h}u(x) − u(x)}{h^{4/3}} = c_{B}|Du(x)|(\frac{1}{2}curv(u)(x)^{+})^{1/3},

where c_{B} is a suitable constant. (As before, r^{1/3} means (r/|r|)|r|^{1/3}.)

Exercise 13.1. Show that T is affine invariant if and only if B is affine invariant.

13.1 The analysis of IS_{h}

The analysis of affine-invariant operators IS_{h} will follow the general plan outlined in section 10.1.1 and exemplified by Theorem 10.2, with the important difference that the structuring elements B are not bounded. They are, however, isometric, since the group of isometries is a subgroup of the SL(ℝ^{2}). This means that given a C^{3} function u, we can expand it in the form

u(x + y) = u(x) + px + ax^{2} + by^{2} + cxy + R(x, y),

where y = (x, y) and the linear term is px. (We use the notation and conventions of section 4.5.) If we assume that this expansion holds for y ∈ D(0, r), then the analysis of the error term R given in the proof of Theorem 10.2 implies that

|u(x + y) − u(x) − (px + ax^{2} + by^{2} + cxy)| ≤ sup_{|y|≤r} ∥D^{3}(x + y)∥|y|^{3}.

Define v for y ∈ D(0, r) by v(y) = u(x) + px + ax^{2} + by^{2} + cxy. If B is 1-localizable, and if h ≤ r^{2} and h < 1/R^{2}, then we know from Lemma 13.6 that

|IS_{h}u(x) − IS_{h}v(x)| ≤ (C + Kc)h^{3/2}.

(Here and elsewhere we use the fact that, if u is locally C^{3}, then it is locally Lipschitz. Also, refer to sections 13.1 and 13.2 for the meaning of the constants.) This implies that the analysis of IS_{h} can be reduced to analyzing the action of IS_{h} on polynomials of degree two. This analysis will be done in Theorem 14.4, but before we get there, we need to consider the action of IS_{h} on two specific polynomials. Because the cases a = c = 0 and b = 1 or b = −1 play key roles, we introduce special notation:

c_{B}^{+} = inf_{B∈B} sup_{y∈B}(x + y^{2}) and c_{B}^{−} = inf_{B∈B} sup_{y∈B}(x − y^{2}),

where “IS_{h}(px+ax^{2} +by^{2} +cxy)” always means “IS_{h}(px+ax^{2} +by^{2} +cxy)(0).” Since our main results use these constants, it is worth examining some examples.

Lemma 13.1. (i) Let B be an affine-invariant family of open convex sets that have area one and that are symmetric with respect to the origin. Then c_{B}^{+} > 0 and c_{B}^{−} = 0. (ii) If B = B_{aff}, then c_{B}^{+} > 0 and c_{B}^{−} = 0. (iii) If a set of structuring elements B is affine invariant and contains one bounded element that is open and contains the origin, then c_{B}^{−} = 0. In particular, this is the case if B is affine invariant and 1-localizable.

13.1. THE ANALYSIS OF IS_{H} 197

Proof. We prove (i) first. Let B be an element of B, and let l(α) denote the line segment defined in polar coordinates by (α, ρ), 0 ≤ ρ < +∞. Both B and l(α) are convex, so their intersection B ∩ l(α) is convex. Since B is open, this set has the form [0, d(α)), where d(α) is the distance from the origin to the boundary of B in the direction α. The function α ↦ d(α) is continuous (Exercise 14.2). Since B has area one, this function can be a constant only if d(α) = 1/\sqrt{π}. In all other cases, d(α) takes values greater than 1/\sqrt{π} and less than 1/\sqrt{π}, and since it is continuous, it must take the value 1/\sqrt{π}. In fact, since B is symmetric, d must assume the value 1/\sqrt{π} four times. In any case, there is a point (x, y) ∈ ∂B such that x > 0 and x^{2} + y^{2} = 1/π.

Now consider the disk D = D(0, 1/\sqrt{π}). We have just seen that there is a point (x, y) ∈ ∂B ∩ ∂D such that x > 0. If (x, y) ∈ D and x > 0, then x > x^{2}, and we have the following inequalities:

sup_{(x,y)∈B}(x + y^{2}) ≥ sup_{(x,y)∈B∩D,x>0}(x + y^{2}) ≥ sup_{(x,y)∈B∩D,x>0}(x^{2} + y^{2}) = \frac{1}{π} .

The right-hand term does not depend on B, so we have c_{B}^{+} ≥ 1/π. The value 1/π is not significant for our purposes; we just wish to show that c_{B}^{+} > 0.

To prove that c_{B}^{−} = 0, first note that since B is open and contains the origin, there are points (x, y) ∈ B with x > 0 and y = 0. Thus, c_{B}^{−} ≥ 0. Fix B ∈ B and consider the sets obtained by “squeezing” B onto the line x = 0:

B_{ε} = {(x′, y′) | x′ = εx, y′ = y/ε, (x, y) ∈ B}.

Then B_{ε} is an affine transform of B, so B_{ε} ∈ B. Therefore,

c_{B}^{−} ≤ sup_{(x,y)∈B_{ε}}(x − y^{2}) ≤ sup_{(x,y)∈B_{ε}}(x) ≤ Cε.

Thus, c_{B}^{−} = 0.

We turn now to the proof of (ii). Assume that B is in B_{aff}. Then by definition, B is open, connected, and contains the origin, and the connected components of B ∩ Δ^{c} that contain the origin in their boundaries have areas greater than or equal to b = δ(0, B^{c}) > 1. If we let Δ be the y-axis and H be the open half-plane defined by x > 0, then the definition implies that area(B ∩ H) > 1. This implies that

( sup_{(x,y)∈B∩H}(x))( sup_{(x,y)∈B∩H}(|y|)) ≥ \frac{1}{2}.  (13.3)

We wish to find a lower bound for sup_{(x,y)∈B}(x + y^{2}), so we may assume that B is bounded, and define μ = sup_{(x,y)∈B∩H}(x) and ν = sup_{(x,y)∈B∩H}(y^{2}). Then

sup_{(x,y)∈B}(x + y^{2}) ≥ sup_{(x,y)∈B∩H}(x + y^{2}) ≥ inf{μ, ν}.

Thus, c_{B_{aff}}^{+} ≥ inf{μ, ν}. Using the constraint (13.3), we conclude that

c_{B_{aff}}^{+} ≥ inf_{μ≥0} \{μ, \frac{1}{4μ^{2}} \} = 2^{−2/3} > 0.

198CHAPTER 13. ASYMPTOTIC BEHAVIOR OF AFFINE-INVARIANT FILTERS

Finally, we must show that c_{B_{aff}}^{−} = 0. If B ∈ B_{aff} and if B is bounded, then, as in the proof of (i),

c_{B_{aff}}^{−} ≤ sup_{(x,y)∈B_{ε}}(x − y^{2}) ≤ sup_{(x,y)∈B_{ε}}(x) ≤ Cε,

and c_{B_{aff}}^{−} = 0. The proof of (iii) is exactly the same as showing that c_{B_{aff}}^{−} = 0.

Exercise 13.2. The purpose of this exercise is to show that the function α ↦ d(α) in Lemma 14.1 is continuous. (Hint: The assumption that d is not continuous leads to a contradiction of the fact that the line segment l(α) intersects ∂B in one and only one point.) Exercise 13.3. This exercise is to show that it is possible to have c_{B}^{−} < 0 for a simple set of structuring elements. Let B = {AC | A ∈ SL(ℝ^{2})}, where C is a square with one side missing defined as follows: C = {(x, y) | x = −2, −2 ≤ y ≤ 2; y = +2, −2 ≤ x ≤ 2; y = −2, −2 ≤ x ≤ 2}. Show that c_{B}^{+} > 0 and c_{B}^{−} < 0.

As one can imagine, the polynomial px + by^{2}, with p > 0, is particularly important in the affine-invariant theory. Fortunately, an invariance argument allows us to compute explicitly the action of IS_{h} on px + by^{2}.

Lemma 13.2. Let B be an affine-invariant set of structuring elements and assume that at least one B ∈ B is bounded. Let IS_{h} be the associate inf-sup operator and assume that p > 0. Then

IS_{h}(px + by^{2}) = c_{B}^{+}(\frac{b}{p})^{1/3}ph^{4/3} if b > 0;
IS_{h}(px + by^{2}) = c_{B}^{−}(\frac{−b}{p})^{1/3}ph^{4/3} if b ≤ 0.

Proof. The existence of a bounded structuring element ensures that sup_{(x,y)∈B}(px+ by^{2}) is not always infinite. If b ≠ 0, then

B ∈ B ⟺ h (\begin{matrix}h^{1/3}|b|^{1/3} & 0 \\ 0 & h^{−1/3}|b|^{−1/3} \end{matrix}) B ∈ hB.

Thus,

inf_{B∈hB} sup_{(x,y)∈B}(x + by^{2}) = inf_{B∈B} sup_{(x,y)∈B}(|b|^{1/3}h^{4/3}x + b(|b|^{−2/3}h^{4/3}y^{2}))

= |b|^{1/3}h^{4/3} inf_{B∈B} sup_{(x,y)∈B}(x + (b/|b|)y^{2}).

Then we have

IS_{h}(x + by^{2}) = \begin{cases}c_{B}^{+} b^{1/3}h^{4/3} if b > 0; \\ c_{B}^{−}(−b)^{1/3}h^{4/3} if b < 0. \end{cases}

Since p > 0, IS_{h}(px + by^{2}) = pIS_{h}(x + (b/p)y^{2}), and we deduce that

IS_{h}(px + by^{2}) = \begin{cases}c_{B}^{+}(b/p)^{1/3}ph^{4/3} if b > 0; \\ c_{B}^{−}(−b/p)^{1/3}ph^{4/3} if b < 0. \end{cases}

13.1. THE ANALYSIS OF IS_{H} 199

Finally, we must deal with the case b = 0. Let B be a bounded element of B and assume it is contained in the square [−R, R] × [−R, R]. Then the element h( \begin{matrix}hε & 0 \\ 0 & h^{−1}ε^{−1} \end{matrix} )B belongs to hB and is contained in the rectangle R =

[−Rεh^{2}, Rεh^{2}] × [−R/ε, R/ε]. Hence,

0 ≤ IS_{h}(px) ≤ sup_{(x,y)∈R}(px) ≤ pRh^{2}ε.

Since we can take ε > 0 arbitrarily small, IS_{h}(px) = 0.

When we studied the asymptotic behavior of an operator T applied to a smooth function u in Chapters 10 and 11, we usually assumed that Du(x) ≠ 0; Du(x) = 0 was a special case. This is also true for affine-invariant inf-sup operators, and the next lemma deals with this case.

Lemma 13.3. Let B be an affine-invariant set of structuring elements, one of which is bounded and all of which contain the origin, and let K be a compact subset of ℝ^{2}. Then for x ∈ K the following inequality holds for every C^{3} function u:

|IS_{h}u(x) − u(x)| ≤ C(∥D^{2}u(x)∥ + |Du(x)|)h^{4/3} + C_{K}h^{2},

where 0 < h ≤ 1, C > 0 is a constant that depends only on B, and the constant C_{K} depends only on B, u, and K. If Du(x) = 0, then

|IS_{h}u(x) − u(x)| ≤ C′∥D^{2}u(x)∥h^{2} + C′_{K}h^{3},

where C′ depends only on B and C′_{K} depends only on B, u, and K.

Proof. We use the notation of sections 4.5 and 10.1. Let B be an arbitrary element of B. Since B contains the origin, sup_{y∈x+hB} u(y) ≥ u(x), which implies that IS_{h}u(x) ≥ u(x).

Now expand u in the familiar local coordinate system in a neighborhood of x:

u(x + hy) = u(x) + phx + ah^{2}x^{2} + bh^{2}y^{2} + ch^{2}xy + R(x, hy),

where y = (x, y). Then for any B ∈ B,

sup_{y∈B} u(x + hy) ≤ u(x) + h sup_{y∈B}(px + ahx^{2} + bhy^{2} + chxy) + sup_{y∈B} |R(x, hy)|.

Now assume that B^{∗} is bounded. Then all of the suprema are finite, and sup_{y∈B^{∗}} u(x + hy) is a finite upper bound for inf_{B∈B} sup_{y∈B} u(x + hy) = IS_{h}u(x). Thus we have

0 ≤ IS_{h}u(x)−u(x) ≤ h sup_{y∈B^{∗}}(px+ahx^{2}+bhy^{2}+chxy)+ sup_{y∈B^{∗}} |R(x, hy)|, (13.4)

which is true for any bounded set B^{∗} ∈ B. We are now going to use the affine invariance of B to manipulate B^{∗} and thereby obtain a good estimate for the terms on the right-hand side of (13.4). Since B^{∗} is bounded, it is contained in the square [−S, S] × [−S, S], where S = sup_{y∈B^{∗}} |y|. By the affine invariance of

B, the set B′ = ( \begin{matrix}h^{1/3} & 0 \\ 0 & h^{−1/3} \end{matrix} )B^{∗} belongs to B and is contained in the rectangle

R = [−Sh^{1/3}, Sh^{1/3}] × [−Sh^{−1/3}, Sh^{−1/3}].

200CHAPTER 13. ASYMPTOTIC BEHAVIOR OF AFFINE-INVARIANT FILTERS

We replace B^{∗} with B′ in (13.4) and proceed to estimate the terms on the right-hand side.

sup_{y∈B′}(px + ahx^{2} + bhy^{2} + chxy) ≤ sup_{y∈R}(px + ahx^{2} + bhy^{2} + chxy)

≤ pSh^{1/3} + |a|S^{2}h^{5/3} + |b|S^{2}h^{1/3} + |c|S^{2}h ≤ pSh^{1/3} + (|a| + |b| + |c|)S^{2}h^{1/3}

= |Du(x)|Sh^{1/3} + (1/2)∥D^{2}u(x)∥S^{2}h^{1/3} ≤ C(∥D^{2}u(x)∥ + |Du(x)|)h^{1/3},

where C = max{S, S^{2}/2}. Note that C depends only on B; in particular, it does not depend on x. Note also that this holds for all h, 0 < h ≤ 1. We now turn to the other term:

sup_{y∈B′} |R(x, hy)| ≤ sup_{y∈R} |R(x, hy)| ≤ sup_{y∈R} ∥D^{3}u(x + hy)∥h^{3}|y|^{3}

≤ sup_{y∈R} ∥D^{3}u(x + hy)∥h^{3}(S^{2}h^{2/3} + S^{2}h^{−2/3})^{3/2}

≤ sup_{y∈R} ∥D^{3}u(x + hy)∥2^{3/2}S^{3}h^{2}.

If K is an arbitrary compact set, then sup_{x∈K} sup_{y∈R} ∥D^{3}u(x + hy)∥2^{3/2}S^{3} ≤ C_{K} for some constant that depends only on K, u, and B. This proves that

|IS_{h}u(x) − u(x)| ≤ C(|Du(x)| + ∥D^{2}u(x)∥)h^{4/3} + C_{K}h^{2}.

If Du(x) = 0, we do the same computation, but we treat both axes the same: R is replaced with the square [−Sh, Sh] × [−Sh, Sh], and

sup_{y∈R}(ax^{2} + by^{2} + cxy) + sup_{y∈R} |R(x, hy)| ≤ C′∥D^{2}u(x)∥h^{2} + C′_{K}h^{3}.

Since SI_{h}u = −IS_{h}(−u), it is clear that the lemma is true for SI_{h}. Everything is now in place to state and prove the first major result of this chapter.

Theorem 13.4. Let B be a 1-localizable affine-invariant family of structuring elements. Assume that u : ℝ^{2} → ℝ is a Lipschitz function that is C^{3} in a neighborhood of x. Then

IS_{h}u(x) − u(x) = h^{4/3}c_{B}|Du(x)|(\frac{1}{2}curv(u)(x)^{+})^{1/3} + o(x, h^{4/3}),

where c_{B} = c_{B}^{+}. If u is C^{3} in a neighborhood of a compact set K and Du(x) ≠ 0 on K, then the result holds for all x ∈ K and o(x, h^{4/3})/h^{4/3} → 0 as h → 0 uniformly for x ∈ K.

Proof. The proof is rather involved, and it is difficult to motivate the various steps. The overall object is to “eliminate” the x^{2}-term and the xy-term and then to apply Lemma 14.2. This is done using the fact that IS_{h} is contrast invariant.

Since IS_{h} is invariant under translation, we assume that x = 0, and since IS_{h} commutes with the addition of constants, we assume that u(0) = 0. Since B is isometric, we can represent u in the disk D(0, 1) as

u(y) = px + ax^{2} + by^{2} + cxy + R(y),  (13.5)

13.1. THE ANALYSIS OF IS_{H} 201

where |R(y)| ≤ C_{R}r^{3} for y = (x, y) ∈ D(0, r), 0 ≤ r ≤ 1. Then

px + ax^{2} + by^{2} + cxy − C_{R}r^{3} ≤ u(y) ≤ px + ax^{2} + by^{2} + cxy + C_{R}(r^{3}).

The operators IS_{h}^{r} satisfy the local maximum principle in the sense that v(y) ≤ w(y) for y ∈ D(0, r) implies that IS_{h}^{r}v(0) ≤ IS_{h}^{r}w(0) for 0 < h ≤ 1. This implies that for all h, 0 < h ≤ 1,

IS_{h}^{r}u(0) = IS_{h}^{r}[px + ax^{2} + by^{2} + cxy](0) + O(r^{3}),  (13.6)

where |O(r^{3})| ≤ C_{R}r^{3}. The proof now branches into two cases, the first being simple and the second being more complicated.

Case (1): p = |Du(0)| = 0.

From Lemma 14.3, IS_{h}u(0) = O(h^{2}). Hence, IS_{h}u(0)/h^{4/3} = O(h^{2/3}), which proves the assertion.

Case (2): p = |Du(0)| ≠ 0.

Eliminating the xy-term. For any ε > 0,

−|c|εy^{2} − \frac{|c|}{ε}x^{2} ≤ cxy ≤ |c|εy^{2} + \frac{|c|}{ε}x^{2}.

Then using (13.6) and the fact that the IS_{h}^{r} are localized, we deduce the following inequalities:

IS_{h}^{r}u(0) ≤ IS_{h}^{r}[px + (a + |c|ε^{−1})x^{2} + (b + |c|ε)y^{2}](0) + C_{R}r^{3};  (13.7)
IS_{h}^{r}u(0) ≥ IS_{h}^{r}[px + (a − |c|ε^{−1})x^{2} + (b − |c|ε)y^{2}](0) + C_{R}r^{3}.  (13.8)

These inequalities hold for all ε > 0, 0 < h ≤ 1, and 0 < r ≤ 1. This leads us to study the expressions

IS_{h}^{r}[px + (a ± |c|ε^{−1})x^{2} + (b ± |c|ε)y^{2}](0),

which do not contain xy-terms. To keep the notation simple, we will write A = a ± |c|ε^{−1} and B = b ± |c|ε. The important point for making the estimates is that A = O(ε^{−1}) and B = O(1). For typographical reasons, we will write IS_{h}[P (y)] for IS_{h}[P (y)](0) for polynomials P. We will also assume that b ≠ 0; the case b = 0 leads to similar computations and the same conclusion.

Eliminating the x^{2}-term. We are going to use the fact that the operators are contrast invariant, and the contrast change that we choose is defined by g_{ε}(s) = s − (A/p^{2})s^{2}. For |s| sufficiently small, g_{ε} is increasing, and we wish to say that

IS_{h}^{r}[px+Ax^{2}+By^{2}] = g_{ε}^{−1}(IS_{h}^{r}[px+By^{2}−\frac{A}{p^{2}} ((Ax^{2}+By^{2})^{2}+2px(Ax^{2}+By^{2}))]),

but to do so, we must be sure that the argument of g_{ε}^{−1} is in a neighborhood of zero where g_{ε} is increasing. If we let t denote the argument of g_{ε}^{−1}, this means that |t| must be less than Cε for some constant C > 0. (Note that there are a couple of cases involving the values of a and |c| that must be investigated, but the controlling condition is that |t| is less than Cε for some constant C > 0.)

202CHAPTER 13. ASYMPTOTIC BEHAVIOR OF AFFINE-INVARIANT FILTERS

The next step is then to estimate the term (A/p^{2})((Ax^{2}+By^{2})^{2}+2px(Ax^{2}+ By^{2})). Using the fact that A = O(ε^{−1}) and B = O(1), a straightforward computation shows that

\frac{A}{p^{2}} ((Ax^{2} + By^{2})^{2} + 2px(Ax^{2} + By^{2})) = O(ε^{−2}r^{3})

for (x, y) ∈ D(0, r). At this point, it is necessary to relate the three parameters ε, r, and h. Our point of view is that ε and r are functions of h, and we let r = h^{1/2} and ε = h^{θ}, where θ > 0 is a small number that must meet several conditions. The first of these is that |IS_{h}^{r}[px + By^{2} − O(ε^{−2}r^{3})]| < Cε = Ch^{θ}.

We know from Lemma 14.2 that IS_{h}[px + By^{2}] = O(h^{4/3}) + O(ε^{1/3}h^{4/3}) = O(h^{4/3}), so we know from Lemma 13.4(i) that

IS_{h}^{r}[px + By^{2}] = IS_{h}[px + By^{2}] + O(h^{3/2}) = O(h^{4/3}).

This implies that IS_{h}^{r}[px + By^{2} − O(ε^{−2}r^{3})] = O(h^{4/3}) + O(h^{(3/2)−2θ}), which means that we must choose θ such that O(h^{4/3}) + O(h^{(3/2)−2θ}) < Ch^{θ} to be certain that

IS_{h}^{r}[px + By^{2} − O(ε^{−2}r^{3})]

is in the domain of g_{ε}^{−1}. Clearly, any value of θ less than 1/2 will work.

Now that we have IS_{h}^{r}[px + By^{2} − O(ε^{−2}r^{3})] in the domain of g_{ε}^{−1}, we use that fact that g_{ε}^{−1}(t) = t + O(ε^{−1}t^{2}) to deduce that

IS_{h}^{r}[px + Ax^{2} + By^{2}] = g_{ε}^{−1}(IS_{h}^{r}[px + By^{2} − O(ε^{−2}r^{3})])
= IS_{h}^{r}[px + By^{2} − O(ε^{−2}r^{3})] + O(ε^{−1}h^{8/3})
= IS_{h}^{r}[px + By^{2}] + O(ε^{−2}h^{3/2}) + O(ε^{−1}h^{8/3})
= IS_{h}^{r}[px + By^{2}] + O(ε^{−2}h^{3/2}),

where for these estimates we must have θ ≤ 1/12, which is the second condition on θ.

Final estimates. Use Lemma 13.4(i) to replace IS_{h}^{r}[px+By^{2}] with IS_{h}[px+ By^{2}]:

IS_{h}^{r}[px + Ax^{2} + By^{2}] = IS_{h}[px + By^{2}] + O(ε^{−2}h^{3/2}).

Using inequalities (13.7) and (13.8) and another call on Lemma 13.4(i) shows that

IS_{h}[px+(b−|c|ε)y^{2}]−O(ε^{−2}h^{3/2}) ≤ IS_{h}u(0) ≤ IS_{h}[px+(b+|c|ε)y^{2}]+O(ε^{−2}h^{3/2}).

We now apply Lemma 14.2 to the terms IS_{h}(px + (b ± |c|ε)y^{2}) in this last expression and use the inequalities (α + β)^{1/3} ≤ α^{1/3} + β^{1/3} and α^{1/3} − β^{1/3} ≤ (α − β)^{1/3}, for α, β ≥ 0, to deduce that

IS_{h}u(0) = c_{B}^{+}(\frac{b}{p})^{1/3}ph^{4/3} + O(h^{(3/2)−2θ}) + O(h^{(4+θ)/3}) if b > 0.

This essentially concludes the pointwise part of the proof. We simply replace b with (1/2)|Du(0)|curv(u)(0) and p with |Du(0)| in these last equations and, once again, choose a suitable θ. If we choose θ so that (3/2) − 2θ = (4 + θ)/3, 13.2. ALTERNATE SCHEMES 203

that is θ = 1/14, then it satisfies the two conditions on already encountered and gives an error term O(h^{19/14}). Recall that when we apply Lemma 13.4(i), there is the restriction that h < r/R. Since r = h^{1/2}, this means that we must have h < 1/R^{2}. To avoid repeating this restriction, we write the error term as o(h^{4/3}) rather than as O(h^{19/14}) with the caveat “for small h.” This is completely consistent with the fact that we are only interested in the behavior of (IS_{h}u(x) − u(x))/h^{4/3} as h → 0.

Uniform convergence. It remains to argue that everything we have done is “uniform on compact sets on which Du(x) ≠ 0.” Based on previous work, particularly in the proof of Theorem 10.2, this should be familiar territory: Given a compact set K on which Du(x) ≠ 0, the functions a, b and c are bounded on K, p is bounded away from zero, and the various estimates can be made to hold uniformly on K. The new element in the current proof is the family of functions g_{ε}: It must be shown that IS_{h}^{r}[px+By^{2} −O(ε^{−2}r^{3})] is in the domain of g_{ε} uniformly for (x, y) ∈ K. The problem is not to have the estimate IS_{h}^{r}[px + By^{2} − O(ε^{−2}r^{3})] ≤ Ch^{θ} uniform, but rather the issue is to establish a uniform domain for the functions g_{ε}^{−1}. Thus, it is necessary to show that the domain of g_{ε}^{−1} is {t | |t| < Ch^{θ}}, where the constant C > 0 does not depend on the point (x, y) ∈ K. Once stated, this is easy to demonstrate.

Write g_{ε}(s) = s(1 − ks), where k = (a ± |c|ε^{−1})/p^{2}. Since a and c are continuous on K and since p is bounded away from zero on K, there is a constant C > 0 such that

|k| ≤ C(1 + \frac{1}{ε})

uniformly on K. The domain of g_{ε}^{−1} is the set {t | |t| < 1/4|k|}, so the set {t | |t| < (8C)^{−1}ε} forms a “uniform” domain for g_{ε}^{−1}, that is, IS_{h}^{r}[px + By^{2} − O(ε^{−2}r^{3})] ≤ (8C)^{−1}h^{θ} implies that IS_{h}^{r}[px + By^{2} − O(ε^{−2}r^{3})] is in the domain of g_{ε}^{−1}.

Exercise 13.4. Lemma 13.4(i) was used twice in the proof. Check to be sure the functions involved were locally Lipschitz.

13.2 Alternate schemes

One of the main difficulties in mathematical morphology is that erosions and dilations do not commute, and this means that openings and closings do not commute. Here is the reason this is a problem: If we perform an opening of a shape in ℝ^{2} using a disk of radius h, then the external peaks are removed and their radii of curvature become greater than h. A closing with a disk of radius h removes the peaks pointing inward making their radii of curvature greater than h. Thus, the aim of opening and closing is to smooth the peaks in exactly the same way, and it would be quite desirable to do both operations simultaneously. In other words, it would be nice if openings and closings commuted so we could speak of a “curvature thresholding” operator. This kind of commutation is roughly attained asymptotically by alternating openings and closings [242]. The idea is to choose a small scale h and to alternate the openings O and the closings F as follows:

O_{h}F_{h}O_{h/2}F_{h/2} ⋯ O_{h/2^{n}}F_{h/2^{n}}

204CHAPTER 13. ASYMPTOTIC BEHAVIOR OF AFFINE-INVARIANT FILTERS

Since O_{h} and F_{h} are idempotent, the growth in scale is necessary to perform a progressive smoothing up to scale h. In the same way, affine erosions and dilations do not commute and should be used alternately. Since these operators are not idempotent, the situation is simpler. We need only choose a scale small enough and alternate Ẽ_{h} and D̃_{h}, that is, compute (Ẽ_{h}D̃_{h})^{n}. In Chapter 16, we will prove that the iterated filter (Ẽ_{h}D̃_{h})^{n} converges to the affine shortening equations when h → 0 and n → ∞ appropriately.

We are now going to extend the results of section 14.1, particularly Theorem 14.4, to alternate schemes, that is, to products like IS_{h}SI_{h}. The alternate schemes are easier to implement and numerically more efficient if we want to have the property T (−u) = −T u. It was precisely because this property is not satisfied by erosions and dilations that it has been proposed, with some experimental success, to build alternating operators like T = IS_{h}SI_{h}. In this case, it is not true that T (−u) = −T u. We will show, however, that (IS_{h}SI_{h})^{n} tends to an operator that does satisfy this condition if n → ∞ and h → 0 appropriately.

It is not possible to obtain an asymptotic result for IS_{h}SI_{h} by applying Theorem 14.4 twice. Indeed, there is no guarantee (and it is generally false) that IS_{h}u is C^{3} if u is C^{3}. The next lemma shows, however, how it is possible to extend to alternate operators convergence results like the ones given in Theorem 14.4.

Here and in the next chapter, we will be dealing with real-valued functions of several variables F : (A, p, x) ↦ F (A, p, x), where A is a symmetric matrix, p is a vector, and x ∈ ℝ^{N}. We will say more about these functions in the next chapter, but, in general, they will be continuous with respect to all arguments, except where p = 0.

Lemma 13.5. Let S_{h} and T_{h} be scaled versions of two inf-sup operators S and T. Let F_{i} : (A, p, x) ↦ F_{i}(A, p, x), i = 1, 2, be two functions that are continuous on every compact set on which p ≠ 0. Assume that for some exponent α the operators S_{h} and T_{h} are such that

\begin{aligned}S_{h}u(x) − u(x) & = h^{α}F_{1}(D^{2}u, Du, x) + o_{1}(x, h^{α}), \\ T_{h}u(x) − u(x) & = h^{α}F_{2}(D^{2}u, Du, x) + o_{2}(x, h^{α}), \end{aligned}  (13.9)

for any u that is C^{3} in a neighborhood of x. Assume that S_{h} is localizable in the sense that there is an exponent γ such that

|S_{h}u(x) − S_{h}^{h^{γ}} u(x)| ≤ o_{3}(x, h^{α})  (13.10)

if u is a Lipschitz function. These relations are assumed to hold uniformly on any compact set K on which Du(x) ≠ 0, that is, o_{j}(x, h^{α})/h^{α} → 0, j ∈ {1, 2, 3}, uniformly for x ∈ K. Then the alternate operator S_{h}T_{h} satisfies

S_{h}T_{h}u(x) − u(x) = h^{α}(F_{1}(D^{2}u, Du, x) + F_{2}(D^{2}u, Du, x)) + o(x, h^{α}).

for any u : ℝ^{2} → ℝ that is C^{3} in a neighborhood of x and is Lipschitz on ℝ^{2}. If, in addition, u is C^{3} in a neighborhood of a compact set K on which Du(x) ≠ 0, then this relation is true uniformly for x ∈ K, that is, o(x, h^{α})/h^{α} → 0 uniformly for x ∈ K.

13.2. ALTERNATE SCHEMES 205

Proof. Assume that u is C^{3} in a neighborhood of a compact set K on which Du(x) ≠ 0. (K may be a single point.) Let K_{r} = \overline{D}_{r}(K) be the closure of the r-dilation of K, where r is chosen so that |Du(x)| is bounded away from zero for x ∈ K_{r}. For x ∈ K, define

f^{+}(x, r) = sup_{y∈\overline{D}(x,r)} F_{2}(D^{2}u, Du, y)
f^{−}(x, r) = inf_{y∈\overline{D}(x,r)} F_{2}(D^{2}u, Du, y),

and define o_{2}(x, r, h^{α}) = sup_{y∈\overline{D}(x,r)} |o_{2}(y, h^{α})|. Then we have

u(y) + h^{α}f^{−}(x, r) − o_{2}(x, r, h^{α}) ≤ T_{h}u(y) ≤ u(y) + h^{α}f^{+}(x, r) + o_{2}(x, r, h^{α})

for y ∈ \overline{D}(x, r). Applying S_{h}^{h^{γ}} to this relation with h^{γ} = r and using the fact that S_{h}^{h^{γ}} is monotone and commutes with constants, and more significantly, that S_{h}^{h^{γ}} satisfies the local maximum principle with zero error, we have

S_{h}^{h^{γ}} u(x)+h^{α}f^{−}(x, r)−o_{2}(x, r, h^{α}) ≤ S_{h}^{h^{γ}} T_{h}u(x) ≤ S_{h}^{h^{γ}} u(x)+h^{α}f^{+}(x, r)+o_{2}(x, r, h^{α}).

By Lemma 6.5, the function T_{h}u is Lipschitz. Thus, we can use (13.10) to replace S_{h}^{h^{γ}} T_{h}u(x) with S_{h}T_{h}u(x) and S_{h}^{h^{γ}} u(x) with S_{h}u(x) to obtain

S_{h}u(x) − o′_{3}(x, h^{α}) + h^{α}f^{−}(x, r) − o_{2}(x, r, h^{α}) ≤ S_{h}T_{h}u(x)
≤ S_{h}u(x) + o′_{3}(x, h^{α}) + h^{α}f^{+}(x, r) + o_{2}(x, r, h^{α}),

where o′_{3}(x, h^{α}) is a nonnegative error term that incorporates all of the errors accumulated in making these replacements. Written slightly differently, we have

S_{h}u(x) − S_{h}T_{h}u(x) ≤ o′_{3}(x, h^{α}) − h^{α}f^{−}(x, r) + o_{2}(x, r, h^{α}),
S_{h}T_{h}u(x) − S_{h}u(x) ≤ o′_{3}(x, h^{α}) + h^{α}f^{+}(x, r) + o_{2}(x, r, h^{α}).

Now use (13.9) to replace S_{h}u(x) with u(x) + h^{α}F_{1}(D^{2}u, Du, x) + o_{1}(x, h^{α}). This shows that

u(x) − S_{h}T_{h}u(x) ≤ −h^{α}F_{1}(D^{2}u, Du, x) − h^{α}f^{−}(x, r) + o^{−}(x, r, h^{α}),
S_{h}T_{h}u(x) − u(x) ≤ h^{α}F_{1}(D^{2}u, Du, x) + h^{α}f^{+}(x, r) + o^{+}(x, r, h^{α}),

where o^{±}(x, r, h^{α}) = ± o_{1}(x, h^{α})+o_{2}(x, r, h^{α})+o′_{3}(x, h^{α}). As a last step, we replace f^{−}(x, r) with F_{2}(D^{2}u, Du, y)+o_{4}^{−}(x, r) and f^{+}(x, r) with F_{2}(D^{2}u, Du, y)+ o_{4}^{+}(x, r), where o_{4}^{±}(x, r) → 0 as r → 0 because F_{2} is continuous on K_{r}. Thus,

S_{h}T_{h}u(x) − u(x) = h^{α}(F_{1}(D^{2}u, Du, x) + F_{2}(D^{2}u, Du, x)) + E(x, r, h, α, γ),

and it remains to show that the error term is o(x, h^{α}) and that this error is uniform on K. Having followed all of the individual error terms, we have the estimate

|E(x, r, h, α, γ)| ≤ |o_{1}(x, h^{α})| + o_{2}(x, r, h^{α}) + o′_{3}(x, h^{α}) + h^{α}|o_{4}^{±}(x, r)|

The first term, |o_{1}(x, h^{α})|, satisfies the conditions by hypothesis. By definition,

o_{2}(x, r, h^{α}) = sup_{y∈\overline{D}(x,r)} |o_{2}(y, h^{α})| ≤ sup_{y∈K_{r}} |o_{2}(y, h^{α})|,

206CHAPTER 13. ASYMPTOTIC BEHAVIOR OF AFFINE-INVARIANT FILTERS

and, again, sup_{y∈K_{r}} |o_{2}(y, h^{α})|/h^{α} → 0 as h → 0 by hypothesis. Similarly, the third term is uniform. The last term h^{α}|o_{4}^{±}(x, r)| = h^{α}|o_{4}^{±}(x, h^{γ})| requires a slightly different argument: |o_{4}^{±}(x, h^{γ})| → 0 uniformly on K, for γ > 0, because F_{1} is uniformly continuous on K_{r}. Thus, E(x, r, h, α, γ) = o(x, h^{α}) and o(x, h^{α})/h^{α} → 0 uniformly on K.

We were very careful in this proof to show in detail how the various assumptions were used to establish the properties of the final error term. In contrast, we only mention that the proof of the next result uses the same techniques. In fact, it is slightly simpler.

Lemma 13.6. Assume all of the hypotheses of Lemma 14.5, except for these changes: The functions G_{i}, i = 1, 2, satisfy the same conditions as the F_{i}. Equations (13.9) are replaced with the following inequalities:

\begin{matrix}0 ≥ S_{h}u(x) − u(x) ≥ h^{α}G_{1}(D^{2}u, Du, x) + o(x, h^{α}), \\ 0 ≤ T_{h}u(x) − u(x) ≤ h^{α}G_{2}(D^{2}u, Du, x) + o(x, h^{α}). \end{matrix}  (13.11)

Then the alternate operators S_{h}T_{h} satisfy

h^{α}G_{1}(D^{2}u, Du, x)+o(x, h^{α}) ≤ S_{h}T_{h}u(x)−u(x) ≤ h^{α}G_{2}(D^{2}u, Du, x)+o(x, h^{α})

for any Lipschitz function u that is C^{3} in a neighborhood of x. If u is C^{3} in the neighborhood of a compact set K on which Du does not vanish, then o(x, h^{α})/h^{α} → 0 uniformly on K as h → 0.

We are now in position to state and prove an asymptotic behavior theorem for the main affine-invariant alternate filter, namely, the filter obtained by alternating the affine erosions and dilations generated with B_{aff}.

Theorem 13.7. Let B = B_{aff}, and let SI_{h} and IS_{h} be the affine erosions and affine dilations defined by equations (13.2). Then for every Lipschitz function u that is C^{3} in a neighborhood of x and Du(x) ≠ 0,

lim_{h→0} \frac{T_{h}u(x) − u(x)}{h^{4/3}} = c_{B}|Du(x)|g(curv(u)(x)),  (13.12)

where

g(r) = \begin{cases}(r^{+}/2)^{1/3} if T_{h} = IS_{h}, \\ (r^{−}/2)^{1/3} if T_{h} = SI_{h}, \\ (r/2)^{1/3} if T_{h} = SI_{h}IS_{h}. \end{cases}

If u is C^{3} in the neighborhood of a compact set K on which Du(x) ≠ 0, then the convergence is uniform on K. Also, there are continuous functions G^{±} such that G^{±}(0, 0, x) = 0 and

h^{4/3}G^{−}(D^{2}u, Du, x)+o(x, h^{4/3}) ≤ T_{h}u(x)−u(x) ≤ h^{4/3}G^{+}(D^{2}u, Du, x)+o(x, h^{4/3}).

(13.13)

Furthermore, o(x, h^{4/3})/h^{4/3} → 0 uniformly on every compact set K.

13.3. COMMENTS AND REFERENCES 207

Proof. We know from Theorem 14.4 that (13.12) is true for T_{h} = IS_{h} and T_{h} = SI_{h}, since B_{aff} satisfies the hypothesis, namely, B_{aff} is 1-localizable. We know from Lemma 14.1 that c_{B_{aff}}^{+} > 0 and c_{B_{aff}}^{−} = 0, so c_{B} = c_{B_{aff}}^{+} in (13.12). We obtain (13.12) for the alternate operator by applying Lemma 14.5 with γ = 1/2 and α = 4/3.

The two inequalities (13.13) are read directly from Lemma 14.3 for T_{h} = IS_{h} and T_{h} = SI_{h}. This is possible since the elements of B_{aff} are open, contain the origin, and there is a B ∈ B_{aff} that is bounded. Once we have the inequalities for these two operators, we appeal to Lemma 14.6 to have the result for the alternate operators. This is possible since both IS_{h} and SI_{h} are localizable by Lemma 13.4 with γ = 1/2.

As a general rule, affine-invariant families of structuring whose members are open, connected, contain the origin, and some of which are bounded with areas bounded above, are localizable. In these cases, it is possible to define alternate schemes that have the properties that we have described in this chapter.

Exercise 13.5. Check that the last proof and the results apply to the affine-invariant, localizable families considered in Exercises 13.3 and 13.4.

13.3 Comments and references

Early versions of the results and proofs contained in this chapter were given in [128, 127, 125]. The version presented here is shorter and simpler. Serra provides a good introduction to alternate sequential filters in [243]. A general axiomatic theory of self-dual morphological operators is given by Heijmans in [132]. Applications of alternate sequential filters are discussed in [219].

208CHAPTER 13. ASYMPTOTIC BEHAVIOR OF AFFINE-INVARIANT FILTERS

Part IV

Viscosity Solutions and Convergence of Iterated

Filters

209

Chapter 14

Viscosity Solutions

This book is about image analysis and PDEs, and one of the main ideas we wish to convey is that the operators we have studied are closely associated with certain differential equations. What is more, we wish to show that whole groups of operators can be associated with essentially the same equation. For example, in Theorem 2.3 we proved that there were many linear operators that converged asymptotically to the heat equation, and in section 8.2 we showed how dilations and erosions are associated with the equations ∂u/∂t = ±|Du|. There is, however, a gap to fill. As Figures 15.1 and 15.2 illustrate, applying dilations and erosions to smooth functions can produce functions that are no longer C^{1} and, hence, cannot satisfy ∂u/∂t = ±|Du| in the usual, or classical, sense. The notion of viscosity solution, which is a late twentieth-century development, provides a way to close this gap. In this chapter, we introduce viscosity solutions, show that classical solutions are viscosity solutions, and quote a uniqueness theorem. We close the gap in the next chapter by showing that the approximate solutions generated by the iterated filters we have studied do indeed converge to viscosity solutions of their associated PDEs.

14.1 Definition and main properties

The functions u : [0, ∞) × ℝ^{N} will always be continuous. If u is C^{2}, then Du and D^{2}u denote the first and second derivatives of u with respect to x ∈ ℝ^{N}; Du is an N-dimensional vector and D^{2}u is an N × N matrix. If x ∈ ℝ^{N} is represented in an orthonormal basis by x = (x_{1}, x_{2}, …, x_{N}), then the elements of D^{2}u are u_{ij} = ∂^{2}u/∂x_{i}∂x_{j}, and D^{2}u is symmetric. Symmetric matrices and their associated quadratic forms play a central role in this chapter because they can be ordered. A real symmetric matrix A = (a_{ij})_{1≤i,j≤N} is said to be nonnegative if its associated quadratic form is nonnegative, that is, if

A(z, z) = ∑_{i,j=1}^{N} a_{ij}z_{i}z_{j} ≥ 0

for all z ∈ ℝ^{N}. This is equivalent to saying that all of the eigenvalues of A are greater than or equal to zero. We say that A ≥ B if A − B ≥ 0. The parabolic PDEs that we consider are of the form ∂u/∂t = F (D^{2}u, Du, x, t),

211 212 CHAPTER 14. VISCOSITY SOLUTIONS

where (A, p, x, t) ↦ F (A, p, x, t) is real-valued and continuous on its domain of definition. We also assume that F is nondecreasing with respect to its first argument. What we mean by this is that, for all A, B, p, x, and t in the domain of definition,

A ≥ B ⟹ F (A, p, x, t) ≥ F (B, p, x, t).  (14.1)

The functions F we consider fall into two classes that are characterized by the domains of definition. In one case, F is defined for all real-valued N × N symmetric matrices A, all p ∈ ℝ^{N}, all x ∈ ℝ, and all t ∈ [0, ∞). We denote this domain by 𝔻^{N}. If we denote the N(N − 1)/2-dimensional space of symmetric N × N matrices by 𝕊^{N}, then

𝔻^{N} = 𝕊^{N} × ℝ^{N} × ℝ × [0, ∞).

In the other case, F is defined on

𝔻̃^{N} = 𝕊^{N} × (ℝ^{N} \ {0}) × ℝ × [0, ∞),

where ℝ^{N} \ {0} is ℝ^{N} with the origin removed. It is important to deal with functions F that are defined on 𝔻̃^{N} rather than 𝔻^{N} because this is the case for one of the most relevant equations in the book, namely, the curvature equation ∂u/∂t = |Du|curv(u). (Note that we have shifted the notation: In this chapter, p = Du rather than p = |Du| as before.) It is this dichotomy of domains that motivates the following definition.

Definition 14.1. We say that a function F is admissible if it is defined and continuous on 𝔻^{N} or 𝔻̃^{N}, if it satisfies (14.1), and if it satisfies the following conditions: There exists two continuous functions G^{+} and G^{−} defined on 𝔻^{N} such that

G^{+}(0, 0, x, t) = G^{−}(0, 0, x, t) = 0,
G^{+}(A, 0, x, t) ≥ 0 and G^{−}(−A, 0, x, t) ≤ 0 for all A ≥ 0,
G^{−}(A, p, x, t) ≤ F (A, p, x, t) ≤ G^{+}(A, p, x, t),

(14.2)

for all (A, p, x, t) in the domain of F.

If F is defined and continuous on 𝔻^{N}, if F satisfies (14.1), and if F (0, 0, x, t) = 0, then F is admissible: Just let G^{±} = F. Conversely, if F is admissible and defined on 𝔻^{N}, then F is continuous at (0, 0, x, t) by definition, and the last line of (14.2) implies that F (0, 0, x, t) = 0.

We are going to illustrate this definition by describing the functions F that are associated with the PDEs that we have studied and that are of particular interest for image processing. This list will also serve as a reference for this and the next chapter. The analysis will rely on fundamental facts about real symmetric matrices, most of which are listed in section I.4 of the Introduction.

• F (A, p) = |p| or F (A, p) = −|p|. These functions are related to dilations and erosions and are associated with the equations ∂u/∂t = |Du| and ∂u/∂t = −|Du|. They are clearly admissible.

• F (A, p) = trace(A) = Tr(A). This is associated with the heat equation ∂u/∂t = Δu. F is defined and continuous on 𝔻^{N}. Since Tr(A + B) = Tr(A) + Tr(B) and Tr(A) ≥ 0 for A ≥ 0, it follows that F satisfies (14.1). It satisfies (14.2) by taking G^{±} = F.

14.1. DEFINITION AND MAIN PROPERTIES 213

Figure 14.1: Erosions and dilations can create singularities. Top-left: original C^{∞} image. Below: representation of the intensity along the horizontal axis. Middle: dilation with a circle of 30 pixels. Below: at the central point, the image is no longer C^{∞} or even C^{1}. Right: erosion with the same circle. Below: some loss of regularity; the solution is no longer C^{2}.

• F (A, p) = A(p^{⊥}/|p|, p^{⊥}/|p|), which in two dimensions is

F (A, p) = \frac{p_{1}^{2}a_{22} − 2p_{1}p_{2}a_{12} + p_{2}^{2}a_{11}}{|p|^{2}} .

This function is related to the curvature equation ∂u/∂t = |Du|curv(u). It is defined and continuous on 𝔻̃^{N}. It clearly satisfies (14.1). To see that it satisfies (14.2), let G^{+}(A) = largest eigenvalue of A and G^{−}(A) = smallest eigenvalue of A.

• F (A, p) = (A(p^{⊥}, p^{⊥}))^{1/3}. This function is related to affine-invariant smoothing and is associated with the equation

\frac{∂u}{∂t} = |Du|(curv(u))^{1/3}.  (14.3)

This function is defined and continuous on 𝔻^{N} and clearly satisfies (14.1). It satisfies (14.2) by taking G^{±} = F.

• F (A, p) = ((A(p^{⊥}, p^{⊥}))^{+})^{1/3} is associated with ∂u/∂t = |Du|(curv(u)^{+})^{1/3}. Once we note that the operation f ↦ f^{+} preserves continuity and order, the argument in the last item shows that F is admissible.

214 CHAPTER 14. VISCOSITY SOLUTIONS

Figure 14.2: Erosion and dilation can generate singularities within the level lines. Left: original image where the only level-line, corresponding to the boundary of the black shape, is C^{2}. Right: erosion with a circle of radius 10; the resulting level lines have sharp angles.

• F (A, p) = Tr(A) − A(p/|p|, p/|p|) is associated with

\frac{∂u}{∂t} = |Du|( ∑_{i=1}^{N−1} κ_{i}(u)),

where the κ_{i}(u) are the principal curvatures of u (see Proposition 4.21). In other words, this is the equation for mean curvature motion. F is defined and continuous on 𝔻̃^{N}, and it is easy to see that F (A, p) ≥ 0 for A ≥ 0. It follows from this that F satisfies (14.1). To satisfy (14.2), we can take G^{+}(A) = (N − 1)(maximum eigenvalue of A) and G^{−}(A) = (N − 1)(minimum eigenvalue of A).

• F (A, p) = min_{p} A(p^{⊥}/|p|, p^{⊥}/|p|) is associated with ∂u/∂t = |Du| min{κ_{1}, κ_{2}} in three dimensions.

We are going to define viscosity solution for these and other PDEs, but before doing so it is useful to say a few words about solutions in general. It would be convenient to define solutions to the equations listed above by saying that any u that is C^{2} on ℝ^{N} and C^{1} on [0, ∞) and satisfies ∂u/∂t = F (D^{2}u, Du, x, t) at all points (t, x) is a solution. Indeed, we call such a u a classical solution. This definition works well, for example, for the heat equation, where we have shown the existence of classical solutions. There are, however, among the equations listed above, those for which a C^{2} solution, or even a C^{1} solution, cannot be expected. Here is an example where an ostensibly classical solution is not the correct one and violates the comparison principle, which says that v_{0} ≥ u_{0} 14.1. DEFINITION AND MAIN PROPERTIES 215

implies that v ≥ u. Let u_{0}(x) = −|x| be the initial condition. Then one is tempted to take as a solution for the equation ∂u/∂t = |Du| the function ũ(t, x) = t − |x|. Indeed, for all t ≥ 0 and all x ≠ 0, ∂ũ/∂t = 1 = |Dũ|. However, this “solution” violates the comparison principle. This can be checked by comparing it with the C^{∞} solution of the same equation with initial condition v_{0}(x) ≡ 0, which is clearly v(t, x) ≡ 0. The correct comparison-preserving solution is, in fact, u(t, x) = min{0, t − |x|}. One way to work around this difficulty is to use contrast invariance: Let g : ℝ → ℝ be a nondecreasing continuous function that is not C^{1}, and let u_{0}(x), x ∈ ℝ^{N}, be the initial data. Assume that we have been able to define a classical solution u for a curvature equation like (14.3) (or, in ℝ^{N}, a mean curvature motion equation) that satisfies the initial condition u(0, x) = u_{0}(x). Then it is reasonable to expect that g(u) will be a solution of the same equation with initial condition g(u_{0}). Since g is not C^{1}, we cannot expect this solution will be C^{1}. The most we can expect is that it is continuous.

Another way to define an irregular function u as a solution of a PDE is to compare it with classical solutions of the PDE. This idea is made precise in the next definition.

Definition 14.2 (viscosity solution). F is assumed to be an admissible function, and u : [0, ∞) × ℝ^{N} → ℝ is assumed to be continuous. We say that u is a viscosity subsolution of the equation

\frac{∂u}{∂t}(t, x) − F (D^{2}u(t, x), Du(t, x), x, t) = 0  (14.4)

at the point x_{0} and scale t_{0} if, for all φ ∈ C^{∞}([0, ∞) × ℝ^{N}) for which (t_{0}, x_{0}) is a strict local maximum of u − φ, the following conditions are true:

If Dφ(t_{0}, x_{0}) ≠ 0, then

\frac{∂φ}{∂t}(t_{0}, x_{0}) − F (D^{2}φ(t_{0}, x_{0}), Dφ(t_{0}, x_{0}), x_{0}, t_{0}) ≤ 0.  (14.5)

If Dφ(t_{0}, x_{0}) = 0 and D^{2}φ(t_{0}, x_{0}) = 0, then

\frac{∂φ}{∂t}(t_{0}, x_{0}) ≤ 0.  (14.6)

Similarly, u is a viscosity supersolution at (t_{0}, x_{0}) if, for all φ ∈ C^{∞}([0, ∞)× ℝ^{N}) for which (t_{0}, x_{0}) is a strict local minimum of u−φ, the following conditions are true:

If Dφ(t_{0}, x_{0}) ≠ 0, then

\frac{∂φ}{∂t}(t_{0}, x_{0}) − F (D^{2}φ(t_{0}, x_{0}), Dφ(t_{0}, x_{0}), x_{0}, t_{0}) ≥ 0.  (14.7)

If Dφ(t_{0}, x_{0}) = 0 and D^{2}φ(t_{0}, x_{0}) = 0, then

\frac{∂φ}{∂t}(t_{0}, x_{0}) ≥ 0.  (14.8)

The function u is said to be a viscosity solution at the point x_{0} and scale t_{0} if it is both a viscosity subsolution and supersolution. If u is a viscosity solution of (14.4) at every point of [0, ∞) × ℝ^{N} and if u(0, x) = u_{0}(x), we say that u is a viscosity solution of (14.4) with initial condition u_{0}.

216 CHAPTER 14. VISCOSITY SOLUTIONS

One of the first things we must do is show that a classical solution of (14.4) is a viscosity solution. This will be done in Proposition 15.10, but before tackling this issue, we present several lemmas that significantly simplify the task of showing that a function is a viscosity solution. We will prove these lemmas for subsolutions; all of the statements and proofs regarding subsolutions are true for supersolutions with the obvious modifications. The first step is to drop the requirement that the extrema be strict.

Lemma 14.3. A continuous function u : [0, ∞) × ℝ^{N} → ℝ is a viscosity subsolution of (14.4) at (t_{0}, x_{0}) if and only if the following condition is true. If φ is in C^{∞}([0, ∞) × ℝ^{N}) and u − φ has a local maximum (not necessarily strict) at (t_{0}, x_{0}), then (14.5) and (14.6) hold. A similar statement is true for viscosity supersolutions.

Proof. Assume that u is a viscosity subsolution and that u − φ has a local maximum at (t_{0}, x_{0}). Replace φ with ψ, where ψ(t, x) = φ(t, x) + (t − t_{0})^{4} + |x − x_{0}|^{4}. Then u − ψ has a strict local maximum at (t_{0}, x_{0}), and ψ satisfies (14.5) and (14.6) by definition. Since the first and second derivatives of φ and ψ agree at (t_{0}, x_{0}), φ satisfies (14.5) and (14.6). The implication in the other direction is just a matter of noting that “local maximum” includes “strict local maximum.” A dual argument proves the statement for supersolutions.

The relations (14.5)-(14.8) do not provide a bound for ∂φ/∂t in case Dφ = 0 and D^{2}φ ≠ 0. This due to the possibility that F is defined on 𝔻̃^{N} and cannot be continued continuously at p = 0. The next lemma provides an equivalent definition of viscosity solution that cover this case by using the fact that F is admissible.

Lemma 14.4. Assume that F is admissible and that G^{+} and G^{−} are any two continuous functions that satisfy the conditions of (14.2). A continuous function u is a viscosity subsolution of (14.4) at a point (t_{0}, x_{0}) if and only if, for all φ ∈ C^{∞}([0, ∞) × ℝ^{N}) for which (t_{0}, x_{0}) is a strict local maximum of u − φ, φ satisfies (14.5) and

\frac{∂φ}{∂t}(t_{0}, x_{0}) − G^{+}(D^{2}φ(t_{0}, x_{0}), Dφ(t_{0}, x_{0}), x_{0}, t_{0}) ≤ 0.  (14.9)

Similarly, u is a viscosity supersolution of (14.4) at a point (t_{0}, x_{0}) if and only if, for all φ for which (t_{0}, x_{0}) is a strict local minimum of u − φ, φ satisfies (14.7) and

\frac{∂φ}{∂t}(t_{0}, x_{0}) − G^{−}(D^{2}φ(t_{0}, x_{0}), Dφ(t_{0}, x_{0}), x_{0}, t_{0}) ≥ 0.  (14.10)

In particular, if F is continuous, then (14.5) and (14.7) are true even if Dφ(t_{0}, x_{0}) = 0.

Proof. The last statement is an immediate consequence of the other parts of the lemma.

We will prove only the “subsolution” half of the lemma. In one direction, we assume (14.5) and (14.9). Then to show that the lemma implies the definition, 14.1. DEFINITION AND MAIN PROPERTIES 217

we need only show that (14.6) holds. But this follows immediately from (14.9) and the fact that G^{+}(0, 0, x, t) = 0.

To prove the converse, assume that u is a viscosity subsolution in the sense of Definition 17.2, that is, it “satisfies” (14.5) and (14.6), and assume that φ is a C^{∞} function such that u − φ has a strict maximum at (t_{0}, x_{0}). We must prove that (14.9) is true for any continuous G^{+} that satisfies conditions (14.2). We assume that Dφ(t_{0}, x_{0}) = 0 and D^{2}(t_{0}, x_{0}) ≠ 0, since the other cases follow immediately.

Consider the function ψ_{ε} defined by

ψ_{ε}(t, x, y) = u(t, x) − φ(t, y) − \frac{|x − y|^{4}}{ε} , ε > 0.

Since u − φ has a strict maximum at (t_{0}, x_{0}), it is possible to show that, for every sufficiently small ε, there is a (t_{ε}, x_{ε}, y_{ε}) where the function ψ_{ε} has a local maximum (Exercise 15.1). Furthermore, (t_{ε}, x_{ε}, y_{ε}) → (t_{0}, x_{0}, x_{0}) as ε → 0. (The points (t_{ε}, x_{ε}, y_{ε}) may not be unique, so assume, using the axiom of choice, that we have selected one for each sufficiently small ε.)

If we fix x = x_{ε}, then the function ψ_{ε,x_{ε}} : (t, x_{ε}, y) ↦ ψ_{ε}(t, x_{ε}, y) is C^{∞} with respect to y and has a local maximum at the point (t_{ε}, y_{ε}).

This implies that Dψ_{ε,x_{ε}}(t_{ε}, x_{ε}, y_{ε}) = 0 and that D^{2}ψ_{ε,x_{ε}}(t_{ε}, x_{ε}, y_{ε}) ≤ 0. By computing Dψ_{ε,x_{ε}} and D^{2}ψ_{ε,x_{ε}} and using these relations, we see that

Dφ(t_{ε}, y_{ε}) = \frac{4(x_{ε} − y_{ε})|x_{ε} − y_{ε}|^{2}}{ε}

and

D^{2}φ(t_{ε}, y_{ε}) ≥ −\frac{1}{ε}(4|x_{ε} − y_{ε}|^{2}I + 8A),

where I is the identity matrix and A = (x_{ε} − y_{ε}) ⊗ (x_{ε} − y_{ε}). (If we write A = (a_{ij}), then a_{ij} = (x_{εi} − y_{εi})(x_{εj} − y_{εj}).) We now consider two cases.

Case (1): Dφ(t_{ε}, y_{ε}) = 0.

This means that x_{ε} = y_{ε}, which in turn implies that D^{2}φ(t_{ε}, y_{ε}) ≥ 0. Thus, for any G^{+} that satisfies (14.2), G^{+}(D^{2}φ(t_{ε}, y_{ε}), 0, y_{ε}, t_{ε}) ≥ 0.

We now take another point of view: Fix y = y_{ε} and consider the function

ψ_{ε,y_{ε}}(t, x, y_{ε}) = u(t, x) − φ(t, y_{ε}) − \frac{|x − y_{ε}|^{4}}{ε} = u(t, x) − φ_{ε}(t, x).

This function has a local maximum at (t_{ε}, x_{ε}). Furthermore, Dφ_{ε}(t_{ε}, x_{ε}) = Dφ(t_{ε}, x_{ε}) = 0, and D^{2}φ_{ε}(t_{ε}, x_{ε}) = 0. Thus, Lemma 15.3 applies, and (14.6) holds:

\frac{∂φ}{∂t}(t_{ε}, x_{ε}) ≤ 0.

Since x_{ε} = y_{ε} and G^{+}(D^{2}φ(t_{ε}, y_{ε}), 0, y_{ε}, t_{ε}) ≥ 0, it follows that

\frac{∂φ}{∂t}(t_{ε}, y_{ε}) ≤ G^{+}(D^{2}φ(t_{ε}, y_{ε}), 0, y_{ε}, t_{ε}).

Both ∂φ/∂t and G^{+} are continuous in all of their variables, so by letting ε tend to zero we arrive at the inequality (14.9).

218 CHAPTER 14. VISCOSITY SOLUTIONS

Case (2): Dφ(t_{ε}, y_{ε}) ≠ 0.

Note that the function

(t, x) ↦ u(t, x) − φ(t, x − (x_{ε} − y_{ε})) − \frac{|x − y_{ε}|^{4}}{ε} = u(t, x) − ϑ_{ε}(t, x)

has a local maximum at (t_{ε}, x_{ε}). This means that Lemma 15.3 applies to ϑ_{ε}. Since Dφ_{ε}(t_{ε}, y_{ε}) = Dϑ_{ε}(t_{ε}, y_{ε}) ≠ 0 and D^{2}φ(t_{ε}, y_{ε}) = D^{2}ϑ_{ε}(t_{ε}, y_{ε}), we have

\frac{∂φ}{∂t}(t_{ε}, y_{ε}) ≤ F (D^{2}φ(t_{ε}, y_{ε}), Dφ(t_{ε}, y_{ε}), y_{ε}, t_{ε}) ≤ G^{+}(D^{2}φ(t_{ε}, y_{ε}), Dφ(t_{ε}, y_{ε}), y_{ε}, t_{ε}),

using (14.5) and (14.2). Again, letting ε tend to zero gives (14.9).

Exercise 14.1. The exercise is to prove the existence of the points (t_{ε}, x_{ε}, y_{ε}) where ψ_{ε} has local maxima and that tend to (t_{0}, x_{0}, y_{0}) as ε → 0. Here is one way to proceed: Since u − φ has a local maximum at (t_{0}, x_{0}), we can choose an r such that u(t, x) − φ(t, x) ≤ u(t_{0}, x_{0}) − φ(t_{0}, x_{0}) = λ for (t, x) ∈ \overline{D}((t_{0}, x_{0}), r) = \overline{D}. For any (t, x), (t, y) ∈ \overline{D} we have

u(t, x) − φ(t, y) − \frac{|x − y|^{4}}{ε} ≤ sup_{(t,x),(t,y)∈\overline{D}} (u(t, x) − φ(t, y) − \frac{|x − y|^{4}}{ε} ).

Let x = y = x_{0} on the left-hand side, so

λ ≤ sup_{(t,x),(t,y)∈\overline{D}} (u(t, x) − φ(t, y) − \frac{|x − y|^{4}}{ε} ).

For each ε > 0, there is some point (t_{ε}, x_{ε}, y_{ε}) where the supremum is attained. Now argue that these points must tend to (t_{0}, x_{0}, y_{0}) as ε → 0. (This is where the fact that (t_{0}, x_{0}, y_{0}) is “strict” is used.) Hence, for small enough ε the points (t_{ε}, x_{ε}, y_{ε}) must all be in the interior of \overline{D} and are therefore local maximum points.

The next lemma provides another simplification for verifying that a function u is a viscosity solution of (14.4). It shows that it is possible to use test functions for which the variables are separated.

Lemma 14.5. To show that u is a viscosity subsolution (or supersolution), it suffices to use test functions φ ∈ C^{∞}([0, ∞) × ℝ^{N}) of the form φ(t, x) = f(x) + g(t).

Proof. This is the assumption: If a function of the form φ(t, x) = f(x)+g(t) is such that u−φ attains a local maximum at (t_{0}, x_{0}), then (14.5) and (14.9) follow. From this assumption, we must prove: If φ is any function in C^{∞}([0, ∞) × ℝ^{N}) such that u − φ attains a local maximum, then (14.5) and (14.9) follow. The technique for doing this is to develop φ is a Taylor series and separate the variables. To keep the notation manageable, we will assume that (t_{0}, x_{0}) = (0, 0) = 0. (Of course, t = 0 is topologically a special point, but there will be no loss of generality in our presentation.) With this assumption, the Taylor expansion of φ is

φ(t, x) = a + bt + ⟨p, x⟩ + ct^{2} + ⟨Qx, x⟩ + t⟨q, x⟩ + o(|x|^{2} + t^{2}),

where a = φ(0), b = ∂φ/∂t(0), c = (1/2)∂^{2}φ/∂t^{2}(0), Q = (1/2)D^{2}φ(0), and

q = (\frac{∂^{2}φ}{∂x_{1}∂t}(0), …, \frac{∂^{2}φ}{∂x_{N}∂t}(0)).

14.1. DEFINITION AND MAIN PROPERTIES 219

For ε > 0, we define

f(x) = a + ⟨p, x⟩ + ⟨Qx, x⟩ + ε|x|^{2} + ε|q||x|^{2}

and

g(t) = bt + \frac{|q|}{ε}t^{2} + εt^{2} + ct^{2}.

This means that

φ(t, x) = f(x) + g(t) − (ε|q||x|^{2} + \frac{|q|}{ε}t^{2} − t⟨q, x⟩ + ε(|x|^{2} + t^{2})) + o(|x|^{2} + t^{2}).

Since, by the Cauchy–Schwartz inequality, ε|q||x|^{2} + (|q|/ε)t^{2} − t⟨q, x⟩ ≥ 0, we have φ(t, x) ≤ f(x) + g(t) for all sufficiently small (t, x). Thus, in some neighborhood of (t_{0}, x_{0}), u(t, x)−φ(t, x) ≥ u(t, x)−f(x)−g(t). The assumption is that u − φ has a local maximum at (0, 0), so this last inequality implies that u−f −g has a local maximum at (0, 0). Thus, by assumption, (14.5) and (14.9) hold for f + g. More precisely, we have the following two cases.

Case (1): D(f + g)(0) ≠ 0.

From (14.5),

\frac{∂(f + g)}{∂t}(0) ≤ F (D^{2}(f + g)(0), D(f + g)(0), 0, 0).

It is easy to see that Dφ(0) = D(f + g)(0) ≠ 0 and (∂φ/∂t)(0) = (∂(f + g)/∂t)(0), and a short computation shows that D^{2}(f + g)(0) = D^{2}φ(0) + 2ε(1 + |q|)I). Substituting these values in the expression above shows that

\frac{∂φ}{∂t}(0) ≤ F (D^{2}φ(0) + 2ε(1 + |q|)I, Dφ(0), 0, 0).

We let ε → 0 and use the continuity of F to see that (14.5) holds for φ.

Case (2): D(f + g)(0) = 0.

In this case, (14.9) is true for f + g:

\frac{∂(f + g)}{∂t}(0) ≤ G^{+}(D^{2}(f + g)(0), D(f + g)(0), 0, 0).

Letting ε → 0 and using the continuity of G^{+} yields (14.9) for φ.

Exercise 14.2. We have just shown that the test functions in Definition 15.2 can be replaced with functions of the form φ(t, x) = f(x) + g(t). This exercise is to show that the requirement can be weakened further by only requiring that f belongs to any class C of C^{2} functions that has the following property: For any x ∈ ℝ^{N}, any a ∈ ℝ, any p ∈ ℝ^{N}, and any symmetric N × N matrix A, there exists f ∈ C such that

f(x − y) = a + ⟨p, x − y⟩ + \frac{1}{2} ⟨A(x − y), (x − y)⟩ + o(|x − y|^{2}).

The same techniques used to prove Lemma 15.5 can be used to prove this result. One assumes the following: If a function of the form ϑ(t, x) = f(x) + g(t), f ∈ C, is such that u − ϑ attains a local maximum at (t_{0}, x_{0}), then (14.5) and (14.9) follow. From this assumption, one must prove: If φ = f + g, f ∈ C^{∞}(ℝ^{N}), is such that u − φ attains a local maximum at (t_{0}, x_{0}), then (14.5) and (14.9) follow. Another part of the exercise is to loosen the requirement on g.

220 CHAPTER 14. VISCOSITY SOLUTIONS

We are now in position to see how classical and viscosity solutions are related. The next two propositions show that the notion of viscosity solution is indeed a generalization of that of classical solution.

Proposition 14.6. Let F be an admissible function that is continuous everywhere, and assume u : [0, ∞) × ℝ^{N} → ℝ is C^{2} with respect to x and C^{1} with respect to t. If u is a classical solution of

\frac{∂u}{∂t}(t, x) = F (D^{2}u, Du, x, t)

at (t_{0}, x_{0}), then u is a viscosity solution at (t_{0}, x_{0}).

Proof. We prove this for the case Du(t_{0}, x_{0}) ≠ 0. (The other cases follow immediately.) Thus, let φ ∈ C^{∞}([0, ∞) × ℝ^{N}) be such that u − φ has a local maximum at (t_{0}, x_{0}). This implies that (∂u/∂t, Du)(t_{0}, x_{0}) = (∂φ/∂t, Dφ)(t_{0}, x_{0}) and that D^{2}(u − φ)(t_{0}, x_{0}) ≤ 0, so

D^{2}u(t_{0}, x_{0}) ≤ D^{2}φ(t_{0}, x_{0}).

Hence,

\frac{∂φ}{∂t}(t_{0}, x_{0}) = \frac{∂u}{∂t}(t_{0}, x_{0}) = F (D^{2}u(t_{0}, x_{0}), Du(t_{0}, x_{0}), x_{0}, t_{0})
≤ F (D^{2}φ(t_{0}, x_{0}), Dφ(t_{0}, x_{0}), x_{0}, t_{0}).

This proves that u is a viscosity supersolution. A similar argument shows that it is a viscosity subsolution.

Proposition 14.7. Assume that F is admissible and continuous everywhere. Let u be a C^{2}(ℝ^{+} × ℝ^{N}) viscosity solution of ∂u/∂t = F (D^{2}u, Du, x, t). Then u is a classical solution of the same equation.

Proof. Assume that u is a viscosity solution at the point (t_{0}, x_{0}). We write the second-order Taylor expansion of u in the N + 1 variables near (t_{0}, x_{0}) as

u(t, x) = u(t_{0}, x_{0}) + ⟨D̃u(t_{0}, x_{0}), (t − t_{0}, x − x_{0})⟩

+ ⟨D̃^{2}u(t_{0}, x_{0})(t − t_{0}, x − x_{0}), (t − t_{0}, x − x_{0})⟩ + o(|t − t_{0}|^{2} + |x − x_{0}|^{2}),

where the operators D̃ and D̃^{2} involve all N + 1 variables. For ε > 0, define φ_{ε} by

φ_{ε}(t, x) = u(t_{0}, x_{0}) + ⟨D̃u(t_{0}, x_{0}), (t − t_{0}, x − x_{0})⟩
+ ⟨(D̃^{2}u(t_{0}, x_{0}) + εI)(t − t_{0}, x − x_{0}), (t − t_{0}, x − x_{0})⟩.

Thus,

u(t, x) − φ_{ε}(t, x) = −ε(|t − t_{0}|^{2} + |x − x_{0}|^{2}) + o(|t − t_{0}|^{2} + |x − x_{0}|^{2})

and the point (t_{0}, x_{0}) is a local maximum of u − φ_{ε} for all ε > 0. Similarly, (t_{0}, x_{0}) is a local minimum of u − φ_{−ε}. The test functions φ_{ε} and φ_{−ε} are C^{∞}, so we can apply Definition 15.2 directly. Thus,

\frac{∂u}{∂t}(t_{0}, x_{0}) = \frac{∂φ_{ε}}{∂t} (t_{0}, x_{0}) ≤ F (D^{2}φ_{ε}(t_{0}, x_{0}), Dφ_{ε}(t_{0}, x_{0}), x_{0}, t_{0})
= F ((D^{2}u + εI)(t_{0}, x_{0}), Du(t_{0}, x_{0}), x_{0}, t_{0}),

14.2. AN APPLICATION TO MATHEMATICAL MORPHOLOGY 221

and

\frac{∂u}{∂t}(t_{0}, x_{0}) = \frac{∂φ_{−ε}}{∂t} (t_{0}, x_{0}) ≥ F (D^{2}φ_{−ε}(t_{0}, x_{0}), Dφ_{ε}(t_{0}, x_{0}), x_{0}, t_{0})
= F ((D^{2}u − εI)(t_{0}, x_{0}), Du(t_{0}, x_{0}), x_{0}, t_{0}).

Letting ε → 0 and using the continuity of F shows that

\frac{∂u}{∂t}(t_{0}, x_{0}) = F (D^{2}u(t_{0}, x_{0}), Du(t_{0}, x_{0}), x_{0}, t_{0}).

14.2 An application to mathematical morphology

In Proposition 8.8, we showed that u(t, x) = D_{t}u_{0}(x) is a solution of ∂u/∂t = |Du| at each point (t_{0}, x_{0}) where u is C^{1}. We are now going to prove that u is a viscosity solution of this equation at all points.

Theorem 14.8. Assume that u_{0} ∈ C(ℝ^{N}). Let D(0, 1) be the unit ball in ℝ^{N} and let u be defined by

u(t, x) = D_{t}u_{0}(x) = sup_{y∈tD(0,1)} u_{0}(x + y).

Then u is a viscosity solution of

\frac{∂u}{∂t} = |Du|, u(0, x) = u_{0}(x).  (14.11)

Proof. We will use that fact that the dilation D_{t} is recursive, that is, D_{s+t} = D_{s}D_{t} (see Proposition 8.7). In particular, for t > 0, D_{t} = D_{h}D_{t−h}, so

u(t, x) = sup_{|y|<h} u(t − h, x + y).  (14.12)

Let φ be a C^{∞} test function and assume that u − φ has a local maximum at (t_{0}, x_{0}). To prove that u is a viscosity subsolution of (14.11), we must show that

\frac{∂u}{∂t}(t_{0}, x_{0}) − |Du(t_{0}, x_{0})| ≤ 0.

Since u − φ has a local maximum at (t_{0}, x_{0}), we have for sufficiently small h and |y|,

u(t_{0} − h, x_{0} + y) − φ(t_{0} − h, x_{0} + y) ≤ u(t_{0}, x_{0}) − φ(t_{0}, x_{0})

It follow that

sup_{|y|<h} u(t_{0} − h, x_{0} + y) ≤ u(t_{0}, x_{0}) − φ(t_{0}, x_{0}) + sup_{|y|<h} φ(t_{0} − h, x_{0} + y),

and using (14.12) shows that

u(t_{0}, x_{0}) ≤ u(t_{0}, x_{0}) − φ(t_{0}, x_{0}) + sup_{|y|<h} φ(t_{0} − h, x_{0} + y).

222 CHAPTER 14. VISCOSITY SOLUTIONS

Thus,

φ(t_{0}, x_{0}) ≤ sup_{|y|<h} φ(t_{0} − h, x_{0} + y).

Subtracting φ(t_{0} − h, x_{0}) from both sides shows that

φ(t_{0}, x_{0}) − φ(t_{0} − h, x_{0}) ≤ sup_{|y|<h} (φ(t_{0} − h, x_{0} + y) − φ(t_{0} − h, x_{0})).

By writing φ(t_{0} − h, x_{0} + y) − φ(t_{0} − h, x_{0}) = ⟨D̃φ(t_{0}, x_{0}), (0, y)⟩ + o(h + |y|), we see that

φ(t_{0}, x_{0}) − φ(t_{0} − h, x_{0}) ≤ |Dφ(t_{0}, x_{0})|h + o(h).

Dividing both sides by h and letting h tend to zero shows that

\frac{∂u}{∂t}(t_{0}, x_{0}) − |Du(t_{0}, x_{0})| ≤ 0.

We have proven this under the assumption that t_{0} > 0 and h > 0 is sufficiently small. But by continuity, the last inequality is true for t_{0} = 0. This proves that u is a subsolution of (14.11); the proof that it is a supersolution is similar. The fact that u(0, x) = u_{0}(x) is a direct consequence of the assumption that u_{0} is continuous.

14.3 Approximation theory of viscosity solutions

For simplicity, we consider slightly less general PDEs, namely, those of the form

\frac{∂u}{∂t}(t, x) − F (D^{2}u(t, x), Du(t, x), x) = 0,  (14.13)

where (A, p, x) ↦ F (A, p, x) is admissible, but it is independent of t. This is the case for the functions listed at the beginning of the chapter. For these equations, it is reasonable to expect that the operator S_{t} : u_{0} ↦ u(t, ·) could be approximated by iterations of an operator T_{h}, by which we mean that (T_{h})^{n} → S_{t} in some sense as h → 0, n → ∞. We have seen this in Theorem 2.3, where it was shown that a large class of linear operators converge asymptotically to the heat equation, and we have promised to show that whole classes of nonlinear operators converge asymptotically to other PDEs. We are mainly interested in operators that have been shown to be useful for image analysis and that have been studied in previous chapters. To include all of these operators in the theory, we give three definitions that abstract properties about their asymptotic and local behavior. These are properties one expects scaled operators T_{h} must have if its iterates are to approximate S_{t}.

Definition 14.9. We say that a family of operators T_{h}, 0 < h ≤ 1, is αuniformly consistent with equation (14.13), if for every C^{3} Lipschitz function u the following conditions are true for some α ≥ 1 :

If Du(x) ≠ 0, then

T_{h}u(x) − u(x) = h^{α}F (D^{2}u, Du, x) + o(x, h^{α}),  (14.14)

14.3. APPROXIMATION THEORY OF VISCOSITY SOLUTIONS 223

where o(x, h^{α})/h^{α} → 0 uniformly on any compact set K ⊂ {x | Du(x) ≠ 0}.

There is a continuous function G : (A, p, x) ↦ G(A, p, x) and an ϵ > 0 such that G(0, 0, x) = 0 and

|T_{h}u(x) − u(x)| ≤ h^{α}G(D^{2}u(x), Du(x), x) + C_{K}h^{α+ϵ}  (14.15)

for all x in any compact set K. The constant C_{K} is assumed to depend only on K.

Definition 14.10. We say that a family of operators T_{h}, 0 < h ≤ 1, satisfies a β-uniform local comparison principle if the following is true: Given any L > 0 and any L-Lipschitz functions u and v, there exist κ > 0 and C ≥ 0 such that u(y) ≤ v(y) for y ∈ D(x, r) implies that

T_{h}u(x) ≤ T_{h}v(x) + Ch^{β},  (14.16)

for all h < κ. Furthermore, it is assumed that C and κ depend only on L and r.

Notice that if the condition “u(y) ≤ v(y) for y ∈ D(x, r)” holds uniformly for x in some set S, then the conclusion hold uniformly on S. If T is an inf-sup operator whose structuring elements are uniformly bounded and T_{h} is its scaled version, then (14.16) is satisfied with C = 0.

Definition 14.11. Let T_{h}, 0 < h ≤ 1, be a family of operators that is αuniformly consistent with equation (14.13). The approximate solutions of (14.13) generated by the T_{h} with initial condition u_{0} are the functions (t, x) ↦ u_{h}(t, x) defined for 0 < h ≤ 1 and n ∈ ℕ as follows:

For t = nh^{α}, u_{h}(t, x) = T_{h}^{n}u_{0}(x).

For nh^{α} ≤ t ≤ (n + 1)h^{α},

u_{h}(t, x) = \frac{t − nh^{α}}{h^{α}} u_{h}((n + 1)h^{α}, x) + \frac{(n + 1)h^{α} − t}{h^{α}}u_{h}(nh^{α}, x).

It follows immediately from the definition that |u_{h}(t, x)−u_{h}(t, y)| ≤ L|x−y| whenever u_{0} is L-Lipschitz and T satisfies the hypotheses of Lemma 6.5. Other properties of the approximate solutions will be developed in section 16.3

Here is a lemma will be useful for proving the next proposition. We leave the proof as an exercise. (Note the similarities of the ideas for the proof of this lemma and those used in Exercise 15.1.)

Lemma 14.12. Let v_{k} be a sequence of continuous functions that converges uniformly on a disk \overline{D}(x, r) to a function v. Assume that x is a strict maximum point for v on \overline{D}(x, r), and let x_{k} be any point at which v_{k} assumes its maximum on \overline{D}(x, r). Then x_{k} → x as k → ∞.

The next proposition is known as the Barles–Souganidis principle for the approximation of viscosity solutions.

224 CHAPTER 14. VISCOSITY SOLUTIONS

Proposition 14.13 (Barles–Souganidis convergence theorem). Let T_{h} be a family of operators that is α-uniformly consistent with (14.13), that satisfies a β-uniform comparison principle, and such that the T_{h} commute with the addition of constants. Assume that a sequence of approximate L-Lipschitz solutions u_{h_{k}} converges uniformly on every compact set to a function u as h_{k} → 0. If β > α, then u is a viscosity solution of (14.13).

Proof. To simplify the notation, we will write u_{h} rather than u_{h_{k}}. There should be no ambiguity, but one must keep in mind that we are working with a sequence and not a continuum. Let φ be a C^{∞} test function, and assume that u − φ attains a strict maximum at (t, x). Let D = \overline{D}((t, x), r) be a closed disk such that (u − φ)(s, y) < (u − φ)(t, x) for all (s, y) ∈ D, (s, y) ≠ (t, x). Since u_{h} − φ → u − φ uniformly on D, we know from Lemma 15.12 that there is a sequence (t_{h}, x_{h}) of points where u_{h} − φ takes its maximum value on D and that converges to (t, x). Eventually the points (t_{h}, x_{h}) are in the interior of D, so (u_{h} − φ)(t_{h}, x_{h}) is a local maximum of u_{h} − φ for sufficiently small h (that is, for large enough k).

For each t_{h} there is an n_{h} ∈ ℕ such that n_{h}h^{α} ≤ t_{h} < (n_{h} + 1)h^{α}. Thus, for small enough h, we have

u_{h}((n_{h} − 1)h^{α}, y) − φ((n_{h} − 1)h^{α}, y) ≤ u_{h}(t_{h}, x_{h}) − φ(t_{h}, x_{h}).  (14.17)

The next step is to replace t_{h} with n_{h}h^{α} in the right-hand side of (14.17). Assume first that t_{h} ≠ n_{h}h^{α}. Then by Definition 15.11, the function u_{h} is linear in a neighborhood of t_{h}, and so u_{h} − φ is C^{∞}. Since (u_{h} − φ)(t_{h}, x_{h}) is a local maximum of u_{h} − φ,

\frac{∂u_{h}}{∂t} (t_{h}, x_{h}) − \frac{∂φ}{∂t}(t_{h}, x_{h}) = 0.  (14.18)

Now invoke Lemma 15.5 and assume that φ is of the form φ(s, y) = f(y) + g(s). By expanding g around the point s = t_{h}, we have

g(n_{h}h^{α}) = g(t_{h}) + g′(t_{h})(t_{h} − n_{h}h^{α}) + o_{g}(t_{h} − n_{h}h^{α}),

where |o_{g}(t_{h} − n_{h}h^{α})| ≤ C(t_{h} − n_{h}h^{α})^{2} and C depends only on g and r. This and (14.18) imply that

g(n_{h}h^{α}) − g(t_{h}) = u_{h}(n_{h}h^{α}, x_{h}) − u_{h}(t_{h}, x_{h}) + o_{g}(t_{h} − n_{h}h^{α}).  (14.19)

This relation is also true if t_{h} = n_{h}h^{α}. It follows from inequality (14.17) and (14.19) that

u_{h}((n_{h}−1)h^{α}, y)−φ((n_{h}−1)h^{α}, y) ≤ u_{h}(n_{h}h^{α}, x_{h})−φ(n_{h}h^{α}, x_{h})+o_{g}(t_{h}−n_{h}h^{α}),

for all y such that ((n_{h} − 1)h^{α}, y) ∈ D. Therefore,

u_{h}((n_{h}−1)h^{α}, y) ≤ φ((n_{h}−1)h^{α}, y)+u_{h}(n_{h}h^{α}, x_{h})−φ(n_{h}h^{α}, x_{h})+o_{g}(t_{h}−n_{h}h^{α})

for all sufficiently small h and every y ∈ D(x, r/2). Since u_{h} is Lipschitz and φ is locally Lipschitz, we can apply T_{h} to both sides and of the last inequality and conclude from Definition 15.10 that, for sufficiently small h,

T_{h}u_{h}((n_{h}−1)h^{α}, ·)(x_{h}) ≤ u_{h}(n_{h}h^{α}, x_{h})−φ(n_{h}h^{α}, x_{h})+T_{h}(φ(n_{h}−1)h^{α}, ·)(x_{h})+R(h), 14.3. APPROXIMATION THEORY OF VISCOSITY SOLUTIONS 225

where the error term is now R(h) = o_{g}(t_{h} −n_{h}h^{α})+Ch^{β}. If we write φ in terms of f and g and note that T_{h}(u_{h}(n_{h} − 1)h^{α}, ·)(x) = u_{h}(n_{h}h^{α}, x), we have

g(n_{h}h^{α}) − g((n_{h} − 1)h^{α}) ≤ T_{h}f(x_{h}) − f(x_{h}) + o_{g}(t_{h} − n_{h}h^{α}) + Ch^{β},

where we have again used the commutation of T_{h} with the addition of constants.

We wish to apply the uniform consistency assumption (Definition 15.9), and for this there are two case. First, assume that Df(x) ≠ 0. Then Df(x_{h}) ≠ 0 for sufficiently small h, and we can apply (14.14) to φ = f + g:

T_{h}f(x_{h}) − f(x_{h}) = h^{α}F (D^{2}f(x_{h}), Df(x_{h}), x_{h}) + o(x_{h}, h^{α}).

Thus,

g(n_{h}h^{α})−g((n_{h}−1)h^{α}) ≤ h^{α}F (D^{2}f(x_{h}), Df(x_{h}), x_{h})+o(h^{α})+o_{g}(t_{h}−n_{h}h^{α})+Ch^{β}.

Divide this expression by h^{α} and let h → 0 and n_{h} → ∞ so that (n_{h}h^{α}, x_{h}) → (t, x). Since β > α, |o_{g}(t_{h} − n_{h}h^{α})| ≤ C(t_{h} − n_{h}h^{α})^{2}, and F is continuous, we have

\frac{∂g}{∂t} (t, x) ≤ F (D^{2}f(x), Df(x), x).

Since ∂g/∂t = ∂φ/∂t,

\frac{∂φ}{∂t}(t, x) ≤ F (D^{2}f(x), Df(x), x).

Now assume that Df(x) = 0 and D^{2}f(x) = 0. We know nothing about Df(x_{h}), but in any case we deduce from the uniform consistency assumption (inequality (14.15)), that

|T_{h}f(x_{h}) − f(x_{h})| ≤ h^{α}G(D^{2}f(x_{h}), Df(x_{h}), x_{h})) + o(h^{α}),

where G is continuous and G(0, 0, x) = 0. Again, dividing this by h^{α} and letting h → 0 so that (n_{h}h^{α}, x_{h}) → (t, x) and using the continuity of G shows that

\frac{∂φ}{∂t}(t, x) ≤ 0.

This proves that u is a viscosity subsolution of (14.13). A similar (or dual) proof shows that u is a supersolution.

At this point it should be clear that the last important step in our program is to show that the approximate solutions converge uniformly on compact sets. It should also be clear that Definitions 15.9 and 15.10 were fashioned to abstract from previous results about inf-sup operators, conditions that are sufficient to prove the Barles–Souganidis theorem. We will see in the next chapter that these conditions are also sufficient to prove that the approximate solutions converge. This then will close the gap and show that the iterated operators converge to viscosity solutions of their associated equations.

226 CHAPTER 14. VISCOSITY SOLUTIONS

14.4 A uniqueness result for viscosity solutions

Proving uniqueness is technically quite difficult, and we are going to fudge by quoting a uniqueness result without proof. Our statement has been simplified to cover only those admissible functions F that are associated with image operators. References where one can find this and more general results are given in the next section.

Theorem 14.14 (Uniqueness). Assume that (A, p) ↦ F (A, p) is admissible and that u_{0} and v_{0} are two continuous functions defined on ℝ^{N}. Assume further that these functions are C-periodic with respect to the hypercube C = [−1, 1]^{N}. If u and v are continuous C-periodic viscosity solutions of

\frac{∂φ}{∂t} = F (D^{2}u, Du)  (14.20)

with initial conditions u_{0} and v_{0}, then

sup_{t∈R^{+},x∈ℝ^{N}}(u(t, x) − v(t, x)) ≤ sup_{x∈ℝ^{N}}(u_{0}(x) − v_{0}(x)).  (14.21)

As a consequence, if u_{0} = v_{0}, then u(t, x) = v(t, x) for all x and t.

14.5 Comments and references

The simple definition of viscosity solution given in this chapter was originally proposed by Michael G. Crandall and Pierre-Louis Lions in [?] for first-order PDEs associated with control theory. It was then shown to be applicable to second-order equations, in particular the so-called geometric equations like mean curvature motion. The first complete treatise is User’s guide to viscosity solutions of second order partial differential equations by Crandall, Ishii, and Lions [79]. First-order equations are treated extensively in Barles [35]. An elementary account for first-order equations is give in the textbook by Evans [94]. Crandall’s later presentation of the theory for both first- and second-order equations, published in [?], is a masterpiece of simplicity and brevity. This book contains a rather complete overview of the techniques, results, and applications, although it does not include applications to image analysis.

The approximation theory for viscosity solutions presented here is based on the seminal paper by Barles and Souganidis [38].

Proving uniqueness for viscosity solutions of second-order parabolic or elliptic equations is the technically difficult part of the theory. The key step that leads to the uniqueness results was made by Robert Jensen in his 1988 paper [154]. See also [179] and [280] for resent general uniqueness proofs for the parabolic case. Some alternative (or related) existence and uniqueness theories, namely, the nonlinear semigroup theory and De Giorgi’s theory of barriers, are discussed in [91] and [40].

Chapter 15

Curvature Equations and Iterated Contrast-Invariant Operators

We will prove that the set of approximate solutions {u_{h}} generated by scaled contrast-invariant operators T_{h} contains a sequence {u_{h_{n}}} that converges uniformly on compact sets. This is done under conditions that are general enough to include all of the contrast-invariant filters studied in previous chapters. We use this result and the Barles–Souganidis theorem to show that the limit function is a viscosity solution of the PDE associated with the operators, thus closing the gap in the operator-PDE-solution circle. We apply this result to median filters and to affine-invariant filters generated by sets of 1-localizable structuring elements. We also complete another link in the program by showing that viscosity solutions are contrast invariant: If u is a viscosity solution of a curvature equation from a certain class, then, for any continuous contrast change g, g(u) is also a viscosity solution of the same equation.

15.1 The main curvature equations used for image processing

The curvature equation of most interest to image processing have the general form

\frac{∂u}{∂t} = |Du|β(curv(u))  (15.1)

in two dimensions and the form

\frac{∂u}{∂t} = |Du|β(κ_{1}(u), κ_{2}(u), …, κ_{N−1}(u))  (15.2)

in N dimensions. The real-valued function β is continuous and nondecreasing with respect to each of its variables, and the κ_{i}(u) denote the principal curvatures of the level surface of u. In other words, they are the eigenvalues of the restriction of D^{2}u to the hyperplane orthogonal to Du divided by |Du| (see Definition 4.19).

227 228CHAPTER 15. CURVATURE EQUATIONS AND ITERATED CONTRAST-INVARIANT OPERATORS

Here are some specific examples. In two dimensions, we have shown that the equations

\frac{∂u}{∂t} = |Du|curv(u)  (15.3)

and

\frac{∂u}{∂t} = |Du|(curv(u))^{1/3},  (15.4)

as well as variants like

\frac{∂u}{∂t} = |Du|(curv(u)^{+})^{1/3},  (15.5)

are relevant for image processing. In three dimensions, we will be concerned with

\frac{∂u}{∂t} = |Du|(κ_{1}(u) + κ_{2}(u)).  (15.6)

This is the classical mean curvature motion that is important because it appears as a limit of iterated median filters. Our assumptions also cover variants like

\frac{∂u}{∂t} = |Du| min(κ_{1}(u), κ_{2}(u)).  (15.7)

This filter provides less destructive smoothing of three-dimensional images than the mean curvature motion. Finally, we mention affine-invariant curvature motion, which is a particularly important equation in three dimensions:

\frac{∂u}{∂t} = |Du|((κ_{1}(u)κ_{2}(u))^{+})^{1/4}.  (15.8)

The admissible functions F for most of these equations were listed in section 15.1.

15.2 Contrast invariance and viscosity solutions

We are going to show that the concepts of contrast invariance and viscosity solution are compatible: If u is a viscosity solution of (15.1) or (15.2), then for all continuous nondecreasing functions g, g(u) is also a viscosity solution of the same equation. Of course, we must make some assumptions about β, but by now the appropriate assumptions should be clear: Assume that the PDEs we consider are of the form

∂u/∂t = F (D^{2}u, Du),

where (A, p) ↦ F (A, p) is an admissible function of A and p. We showed in section 15.1 that the equations (15.3)-(15.8) are of this form. Furthermore, all of the functions F fall into the two classes already described: Either F is defined and continuous on 𝔻^{N} = 𝕊^{N} × ℝ^{N}, or F is defined and continuous on 𝔻̃^{N} = 𝕊^{N} × (ℝ^{N} \ {0}), where ℝ^{N} \ {0} is ℝ^{N} with the origin removed. In this latter case, F cannot be extended continuously to 𝔻^{N}, and, in fact, these functions should be thought of as being defined on 𝕊^{N} × {p | p ∈ ℝ^{N}, |p| = 1}. An examination of the functions described in section 15.1 shows that, for this second case, p enters only as an argument of A, so if A = 0, then F (0, p) = 0.

15.2. CONTRAST INVARIANCE AND VISCOSITY SOLUTIONS 229

Lemma 15.1. Assume that (A, p) ↦ F (A, p) is an admissible function of the form F (D^{2}(u), D(u)) = |D(u)|β(κ_{1}(u), κ_{2}(u), …, κ_{N−1}(u)) and that g : ℝ → ℝ is C^{2} with g′(s) > 0 for all s ∈ ℝ.

If Du ≠ 0, then

F (D^{2}(g(u)), D(g(u))) = g′(u)F (D^{2}u, Du)  (15.9)

for any C^{2} function u : ℝ^{N} → ℝ.

If D^{2}u = 0 and Du = 0, then

F (D^{2}(g(u)), D(g(u))) = F (0, 0) = 0.  (15.10)

Proof. If D(u) ≠ 0, then we know from Proposition 4.16 that curv(g(u)) = curv(u) in two dimensions and that κ_{i}(g(u)) = κ_{i}(u) in the N-dimensional case. Thus,

F (D^{2}(g(u)), D(g(u))) =|D(g(u))|β(κ_{1}(g(u)), κ_{2}(g(u)), …, κ_{N−1}(g(u)))
=g′(u)|D(u)|β(κ_{1}(u), κ_{2}(u), …, κ_{N−1}(u))
=g′(u)F (D^{2}(u), D(u)),

as announced.

In general, D(g(u)) = g′(u)D(u) and D^{2}(g(u)) = g′(u)D^{2}(u) + g′′D(u) ⊗ D(u). Thus, if D^{2}(u) = 0 and D(u) = 0, then D^{2}(g(u)) = 0 and D(g(u)) = 0, and

F (D^{2}(g(u)), D(g(u))) = F (0, 0) = 0.

The proof of the next result, the main one of this section, is a slightly more involved because we drop the assumption that g′(s) > 0. This assumption was essential for the lemma, but it disappears in the statement of the proposition.

Proposition 15.2. Assume that u is a viscosity solution of the equation

\frac{∂u}{∂t} = F (D^{2}(u), D(u)),

where F satisfies the conditions of Lemma 16.1. If g : ℝ → ℝ is continuous and nondecreasing, then g(u) is also a viscosity solution of this equation.

Proof. We begin by assuming that g is C^{∞} and that g′(s) > 0, and we write f = g^{−1} for convenience. Let (t, x) be a strict local maximum of g(u) − φ. Without loss of generality, we can assume that g(u(t, x)) − φ(t, x) = 0: Just replace φ with φ − g(u(t, x)). Then (t, x) is also a strict local maximum of u − f(φ). To see this, note that

g(u(s, y)) − φ(s, y) < g(u(t, x)) − φ(t, x) = 0

for (s, y) sufficiently close to (but not equal to) (t, x). Thus,

u(s, y) < f(φ(s, y))

and

u(s, y) − f(φ(s, y)) < 0 = u(t, x) − f(φ(t, x)),

230CHAPTER 15. CURVATURE EQUATIONS AND ITERATED CONTRAST-INVARIANT OPERATORS

again, for (s, y) sufficiently close to (but not equal to) (t, x).

Since f(φ) is C^{∞} and u is a viscosity solution, it follows from the definition of viscosity solution that, for D(f(φ)(t, x) ≠ 0,

\frac{∂(f(φ))}{∂t}(t, x) ≤ F (D^{2}(f(φ))(t, x), D(f(φ))(t, x)).

This implies by Lemma 16.1 that

f′(φ)\frac{∂φ}{∂t}(t, x) ≤ f′(φ)F (D^{2}(φ)(t, x), D(φ)(t, x));

since f′(s) > 0,

\frac{∂φ}{∂t}(t, x) ≤ F (D^{2}(φ)(t, x), D(φ)(t, x)).

If D(f(φ)) = 0 and D^{2}(f(φ)) = 0, then, by Definition 15.2, (∂f(φ)/∂t)(t, x) ≤ 0, and so (∂φ/∂t)(t, x) ≤ 0. This proves that g(u) is a viscosity subsolution when g ∈ C^{∞} and g′(s) > 0.

Now assume that g is simply continuous and nondecreasing. We replace g with g_{ε}, ε > 0, defined by

g_{ε}(s) = ∫_{ℝ} ψ_{ε}(s − t)(g(t) + εt) dt,

where the support of ψ_{ε} is in [−ε, ε], ψ_{ε} is C^{∞}, ψ_{ε}(s) ≥ 0, and ∫_{ℝ} ψ_{ε}(t) dt = 1. It is then a straightforward exercise to show that g_{ε} is C^{∞}, that g′_{ε}(s) ≥ ε, and that g_{ε} → g uniformly on compact subsets of ℝ as ε → 0.

We know from Lemma 15.12 that there is a sequence of points (t_{ε(k)}, x_{ε(k)}), k ∈ ℕ, with the following properties: ε(k) → 0 as k → ∞, (t_{ε(k)}, x_{ε(k)}) → (t, x) as k → ∞, and g_{ε(k)}(u) − φ has a local maximum at (t_{ε(k)}, x_{ε(k)}). (Having fixed this sequence, we will now simplify the notation by writing ε(k) = ε.)

If Dφ(t, x) ≠ 0, then Dφ(t_{ε}, x_{ε}) ≠ 0 for all sufficiently small ε, that is, all sufficiently large k. Since we have shown in the first part of the proof that g_{ε}(u) is a viscosity solution of

\frac{∂u}{∂t} = F (D^{2}(u), D(u)),

it follows from Lemma 15.3 that

\frac{∂φ}{∂t}(t_{ε}, x_{ε}) ≤ F (D^{2}φ(t_{ε}, x_{ε}), Dφ(t_{ε}, x_{ε}))

for all sufficiently small ε. Since both sides of this inequality are continuous, we can pass to the limit as ε → 0 and conclude that

\frac{∂φ}{∂t}(t, x) ≤ F (D^{2}φ(t, x), Dφ(t, x)).

In case D^{2}φ(t, x) = 0 and Dφ(t, x) = 0, we call on Lemma 15.4 and write

\frac{∂φ}{∂t}(t_{ε}, x_{ε}) ≤ G^{+}(D^{2}φ(t_{ε}, x_{ε}), Dφ(t_{ε}, x_{ε})),

15.3. UNIFORM CONTINUITY OF APPROXIMATE SOLUTIONS 231

where G^{+} satisfies the conditions of Definition 15.1. By passing to the limit and using the fact that G^{+}(0, 0) = 0, we see that

\frac{∂φ}{∂t}(t, x) ≤ 0.

This proves that g(u) is a viscosity subsolution of ∂φ/∂t = F (D^{2}u, Du); the dual proof shows that it is a viscosity supersolution.

Exercise 15.1. Note that we wrote Lemma 15.4 assuming that the local maximum was strict, but that we used it in the last part of the proof in a situation where the local maxima were not necessarily strict. Show that this is not a problem.

15.3 Uniform continuity of approximate solutions

This section is a direct continuation of section 15.3. The object is to prove that the approximate solutions of operators that satisfy the conditions of Definitions 15.9 and 15.10 converge uniformly on compact sets. We do this in a sequence of three lemmas.

Lemma 15.3. Assume that the operators T_{h}, 0 < h ≤ 1 are monotone, translation invariant, and commute with the addition of constants. Assume also that they are α- uniformly consistent with an equation of the form

\frac{∂u}{∂t} = F (D^{2}u, Du,)  (15.11)

and that they satisfy a β-uniform local comparison principle with β > α. Let K be an arbitrary compact subset of ℝ^{N}. If u is L-Lipschitz and C^{3}, then there are two constants κ > 0 and C_{K} > 0 such that

|T_{h}^{n}u(x) − u(x)| ≤ C_{K}nh^{α}  (15.12)

for all x ∈ K, all h < κ, and all n ∈ ℕ. Furthermore, κ and C_{K} do not depend on n or x; they depend only on T, u, and K.

Proof. Assume that K is given and define the closed dilation of K by

K_{r} = \overline{D}_{r}(K) = \{y | d(y, K) ≤ r\}

for some r > 0. The value of r is not important, so we take r = 1. By Definition 15.9, there exists a continuous function G, an ϵ > 0, and a constant C(K_{1}), which depends only on K_{1}, and hence only on K, such that

|T_{h}u(x) − u(x)| ≤ h^{α}G(D^{2}u(x), Du(x)) + C(K_{1})h^{α+ϵ}.

Since G is continuous, there is a constant C(G, K_{1}) such that

sup_{x∈K_{1}} G(D^{2}u(x), Du(x)) = C(G, K_{1}).

Thus, for x ∈ K and y ∈ D(x, 1), we have

T_{h}u(y) ≤ u(y) + C(G, K_{1})h^{α} + C(K_{1})h^{α+ϵ}.  (15.13)

232CHAPTER 15. CURVATURE EQUATIONS AND ITERATED CONTRAST-INVARIANT OPERATORS

We wish to apply T_{h} to both sides of this inequality and use the fact that the T_{h} satisfy a β-uniform local comparison principle. We can do this because the functions on either side of the inequality are L-Lipschitz (Lemma 6.5) and because the inequality is true for y ∈ D(x, 1). However, when we apply T_{h} to both sides, we must add an error term and a caveat regarding the values of h. Thus, there is a κ > 0 such that

T_{h}^{2}u(x) ≤ T_{h}u(x) + C(G, K_{1})h^{α} + C(K_{1})h^{α+ϵ} + C(L, r)h^{β}

for all h < κ, and the constant C(L, r) (r = 1) depends only on u and r. This relation is true for all x ∈ K. Now use inequality (15.13) to deduce that

T_{h}^{2}u(x) ≤ u(x) + 2C(G, K_{1})h^{α} + 2C(K_{1})h^{α+ϵ} + C(L, r)h^{β}

for all x ∈ K and all h < κ. This process can be repeated n times to show that

T_{h}^{n}u(x) ≤ u(x) + nC(G, K_{1})h^{α} + nC(K_{1})h^{α+ϵ} + (n − 1)C(K_{1})h^{β}

for all x ∈ K and all h < κ. The important point is that in repeating the process the constants C(G, K_{1}), C(K_{1}), C(K_{1}), and κ do not change. Thus we have

T_{h}^{n}u(x) ≤ u(x) + (C(G, K_{1}) + C(K_{1}) + C(K_{1}))nh^{α},

so if we let C_{K} = C(G, K_{1}) + C(K_{1}) + C(K_{1}) we have half of the inequality (15.12). The other half of (15.12) follows by interchanging T_{h}u and u.

This result provides a bound for the functions T_{h}^{n}u on a given compact set, and this will translate into a bound on the approximate solutions associated with the T_{h}. The next lemma provides a precise statement.

Lemma 15.4. Assume that the operators T_{h} satisfy the conditions given in Lemma 16.3 and let u_{h} be the approximate solutions associated with the T_{h}. Let K be a compact subset of ℝ^{N} and let u_{0} be L-Lipschitz and C^{3} on ℝ^{N}. Then there are constants κ > 0 and C_{K} > 0 such that for all s, t ∈ ℝ^{+}, x, y ∈ K, and h < κ,

|u_{h}(s, x) − u_{h}(t, y)| ≤ L|x − y| + \begin{cases}2C_{K}h^{α}  if |s − t| ≤ h^{α}; \\ 2C_{K}|s − t| if |s − t| ≥ h^{α}. \end{cases}  (15.14)

Proof. Approximate solutions are defined in Definition 15.11 by u_{h}(t, x) = T_{h}^{n}u_{0}(x) if t = nh^{α} and by linear interpolation for t ∈ (nh^{α}, (n + 1)h^{α}). Thus,

u_{h}(t, x) = \frac{t − nh^{α}}{h} u_{h}((n + 1)h^{α}, x) + \frac{(n + 1)h^{α} − t}{h}u_{h}(nh^{α}, x), (15.15)

where nh^{α} ≤ t ≤ (n + 1)h^{α}.

Lemma 6.5 states that T_{h}^{n} maps Lipschitz functions into Lipschitz functions without increasing the Lipschitz constant, so we have immediately that

|u_{h}(nh^{α}, x) − u_{h}(nh^{α}, y)| ≤ L|x − y|

for all x, y ∈ ℝ^{N}. The representation (15.15) shows that this is true for all values of t:

|u_{h}(t, x) − u_{h}(t, y)| ≤ L|x − y|.  (15.16)

15.3. UNIFORM CONTINUITY OF APPROXIMATE SOLUTIONS 233

Now take m ≤ n and note that T_{h}^{n−m}u_{h}(mh^{α}, ·)(x) = u_{h}(nh^{α}, x). Then by Lemma 16.3 we have

\begin{aligned}|u_{h}(mh^{α}, x) − u_{h}(nh^{α}, x)| & = |T_{h}^{n−m}u_{h}(mh^{α}, ·)(x) − u_{h}(mh^{α}, x)| \\ & ≤ C_{K}|m − n|h^{α}, \end{aligned} (15.17)

where this holds for all h < κ and all x ∈ K. The next step is to replace mh^{α} and nh^{α} with s and t where mh^{α} ≤ s < (m + 1)h^{α} and nh^{α} ≤ t < (n + 1)h^{α}. For this, we use (15.15) and write

h^{α}[u_{h}(s, x) − u_{h}(t, x)] =(s − mh^{α})[u_{h}((m + 1)h^{α}, x) − u_{h}((n + 1)h^{α}), x)]
+ ((m + 1)h^{α} − s)[u_{h}(mh^{α}, x) − u_{h}(nh^{α}, x)]

+ ((s − t) − (m − n)h^{α})[u_{h}((n + 1)h^{α}, x) − u_{h}(nh^{α}, x)].

This representation and (15.17) lead directly to the estimate

|u_{h}(s, x) − u_{h}(t, x)| ≤ C_{K}|m − n|h^{α} + |\frac{s − t}{h^{α}} − (m − n)|C_{K}h^{α}.

Since mh^{α} ≤ s < (m + 1)h^{α} and nh^{α} ≤ t < n + 1)h^{α}, we have

|\frac{s − t}{h^{α}} − (m − n)| ≤ 1,

and thus

|u_{h}(s, x) − u_{h}(t, x)| ≤ C_{K}(|m − n| + 1)h^{α}.

By analyzing the cases |s − t| ≤ h^{α} and |s − t| ≥ h^{α}, one arrives at the following dichotomy:

|u_{h}(s, x) − u_{h}(t, x)| ≤ \begin{cases}2C_{K}h^{α}  if |s − t| ≤ h^{α}, \\ 2C_{K}|s − t| if |s − t| ≥ h^{α}, \end{cases}  (15.18)

for x ∈ K and h < κ. Inequalities (15.16) and (15.18) imply (15.14).

Inequality (15.14) allows us to prove the following lemma, which is an application of the Ascoli–Arzela Theorem. This is the result that was announced in the introduction to the section.

Lemma 15.5. Let {h_{n} | n ∈ ℕ} be any decreasing sequence such that h_{n} → 0 as n → ∞. Then there is a subsequence of {u_{h_{n}}} that converges uniformly to a continuous function u on every compact subset ℝ^{+} × ℝ^{N}.

Proof. Consider the compact sets K_{m} = [0, m] × \overline{D}(0, m), m ∈ ℕ. We will prove the result for K = K_{1} and leave the rest of the proof as an exercise.

To apply the Ascoli–Arzela Theorem, we must show that the functions {u_{h_{n}}} are equicontinuous on K. This means that, given ϵ > 0, we must exhibit a δ = δ(ϵ) such that |x − y| + |s − t| ≤ δ implies that |u_{h_{n}}(s, x) − u_{h_{n}}(t, y)| ≤ ϵ uniformly for all n ∈ ℕ and (s, x), (t, y) ∈ K. Since equicontinuity in the spatial variable is obvious, we will concentrate on the time variable. To simplify the notation and thus to exhibit the ideas better, we set up the following situation. Suppose we have continuous functions v_{n} (v_{n} = u_{h_{n}}) such that, for h < κ,

|v_{n}(s) − v_{n}(t)| ≤ \begin{cases}2C_{K}h_{n}^{α} if |s − t| ≤ h_{n}^{α}; \\ 2C_{K}|s − t| if |s − t| ≥ h_{n}^{α}. \end{cases}

234CHAPTER 15. CURVATURE EQUATIONS AND ITERATED CONTRAST-INVARIANT OPERATORS

Given ϵ > 0, choose n′ = n′(ϵ) such that 2C_{K}h_{n′}^{α} ≤ ϵ and h_{n′} < κ. Now choose δ′ = δ′(ϵ) such that |s − t| ≤ δ′ implies that |v_{n}(s) − v_{n}(t)| ≤ ϵ for n ≥ n′. (There are only a finite number of these n, so this is possible.) Finally, let δ = min{h_{n′}^{α}, δ′}. If |s − t| ≤ δ and n ≤ n′, then, by the definition of δ, |v_{n}(s) − v_{n}(t)| ≤ ϵ. On the other hand, if |s − t| ≤ δ and n > n′, then there are two cases: If |s − t| ≤ h_{n}^{α}, then

|v_{n}(s) − v_{n}(t)| ≤ 2C_{K}h_{n}^{α} ≤ 2C_{K}h_{n′}^{α} ≤ ϵ;

if |s − t| ≥ h_{n}, then

|v_{n}(s) − v_{n}(t)| ≤ 2C_{K}|s − t| ≤ 2C_{K}δ ≤ 2C_{K}h_{n′}^{α} ≤ ϵ.

This proves the result, once we restore the space variable and the original notation.

Exercise 15.2. We have just shown that there is a sequence {u_{h_{n}}} that converges uniformly (hence to a continuous function) on the compact set K_{1} = [0, 1] × \overline{D}(0, 1). The same argument shows that there is a subsequence of {u_{h_{n}}} that converges uniformly on K_{2} = [0, 2] × \overline{D}(0, 2), and so on for all K_{n}. Use a diagonal process to show that there is a subsequence of {u_{h_{n}}} that converges uniformly on any compact subset of ℝ^{+} × ℝ^{N}.

15.4 Convergence of iterated median filters to mean curvature motion

The study of median filters has been a major theme of the book. They were introduced in Chapter 9, first as set operators Med_{k} and then as function operators Med_{k}. These filters were defined in terms of a nonnegative measurable weight function k : ℝ^{N} → ℝ that is normalized by

∫_{ℝ^{N}} k(y) dy = 1.

It was shown in Proposition 9.4 that Med_{k} can be represented as

Med_{k}u(x) = sup_{|B|_{k}=1/2} inf_{y∈x+B} u(y),

where |B|_{k} = ∫_{B} k(y) dy. One of the main results, Proposition 9.7, was that, if k is not separable and if u is continuous, then Med_{k}u is continuous and

Med_{k}u(x) = inf_{|B|_{k}≥1/2} sup_{y∈x+B} u(y).

Theorem 10.4 provided an infinitesimal interpretation of Med_{k} in the case k is the uniform measure on the unit disk in ℝ^{2}. It was shown there how the scaled operator (Med_{k})_{h} was related to curvature motion. This result was extended to ℝ^{N} in Theorem 11.3; there we assumed that k was radial, was in the Schwartz class, and had connected compact support; the last assumption ensured that k was not separable. Along the way, we showed that (Med_{k})_{h} = Med_{k_{h}}, where k_{h}(x) = (1/h^{N})k(x/h).

15.5. CONVERGENCE OF ITERATED AFFINE-INVARIANT OPERATORS235

All of the pieces are now in place to prove that the approximate solutions of iterated median filters converge to viscosity solutions of mean curvature equations. Before we do this, it is important to be quite precise about the assumptions we have made to fashion the various components:

• k is radial, it has compact, connected support, and it belongs to the Schwartz class (Theorem 11.3), or k is the uniform distribution on the unit disk in ℝ^{2} (Theorem 10.4).

• Theorem 10.4 was proved under the assumption that u is C^{2}, and Theorem 11.3 was proved assuming that u is C^{3}.

Theorem 15.6 (Convergence of iterated median filters). Assume that the weight function k satisfies the conditions listed above. Let u_{0} be a Lipschitz function that is also C^{3}. Let {u_{h}}, h ≤ 1, be the set of approximate solutions associated with the operators T_{h}^{n}, where T = Med_{k}. Then {u_{h}} has a limit point u in the sense of convergence on compact sets and u is a viscosity solution of the PDE

\frac{∂u}{∂t} = \frac{1}{6}|Du|curv(u)

in the case k is the uniform measure on the unit disk in ℝ^{2}, and u a viscosity solution of

\frac{∂u}{∂t} = \frac{1}{2}c_{k}|Du|curv(u)

in the N-dimensional case. In this case, curv(u) is the mean curvature of u and c_{k} is the constant given in Lemma 11.2.

Proof. The first thing to note is that the operators we are considering are α-uniformly consistent with their associated equations, where α = 2. This is the content of Theorem 10.4 in case k is the uniform measure on the unit disk in ℝ^{2}, and it is the content of Theorem 11.3 in case k is radial, has compact, connected support, and belongs to the Schwartz class. These operators have bounded structuring elements so they clearly satisfy a β-uniform local comparison principle with β = +∞.

Since median filters and u_{0} satisfy all of the other hypotheses of the lemmas of section 16.3, Lemma 16.5 ensures that the set of approximate solutions {u_{h}}, for any one of these filters, has a limit point u in the following sense: There is a sequence h_{n} such that h_{n} → 0 as n → ∞ and such that u_{h_{n}} → u uniformly on compact subsets of ℝ^{+} × ℝ^{N}. The hypotheses being satisfied, the Barles– Souganidis theorem (Proposition 15.13) states that u is a viscosity solution of the PDE associated with the operators T_{h}.

The action of iterated median filters and the action of the corresponding PDE are illustrated and compared in Figures 16.1 and 16.2.

15.5 Convergence of iterated affine-invariant operators to affine-invariant curvature motion

The plan is to prove a theorem for iterated affine-invariant operators like the one we have just demonstrated for median filters. We will do this in ℝ^{2}.

236CHAPTER 15. CURVATURE EQUATIONS AND ITERATED CONTRAST-INVARIANT OPERATORS

Figure 15.1: Scale-space based on iterations of the median filter. From left to right and top to bottom: original shape, size of the disk used for the median filter, and the results of applying the iterated median filter for an increasing number of iterations.

Theorem 15.7. Assume that the set of structuring elements B is affine invariant and 1-localizable and that c_{B}^{+} > 0. Let

IS_{h}u(x) = inf_{B∈hB} sup_{y∈x+B} u(y) and SI_{h}u(x) = sup_{B∈hB} inf_{y∈x+B} u(y),

and let T_{h} denote one to the operators IS_{h}, SI_{h}, or SI_{h}IS_{h}. Let u_{h} be the approximate solutions generated by T_{h} with an initial function u_{0} that is C^{3} and Lipschitz on ℝ^{2}. Then the set {u_{h}} has a unique limit point u in the sense of uniform convergence on compact sets, and u is a viscosity solution of

\frac{∂u}{∂t} = c_{B}|Du|g(curv(u)),

where

g(r) = \begin{cases}(r^{+}/2)^{1/3} if T_{h} = IS_{h}, \\ (r^{−}/2)^{1/3} if T_{h} = SI_{h}, \\ (r/2)^{1/3} if T_{h} = SI_{h}IS_{h}, \end{cases}  (15.19)

and c_{B} = c_{B}^{+}.

15.5. CONVERGENCE OF ITERATED AFFINE-INVARIANT OPERATORS237

Figure 15.2: Comparing an iterated median filter with a curvature motion. Numerically, the iterated median filter and the curvature motion are very close, when the curvatures of the level lines are not too large. Indeed, the iterated median filter converges towards the viscosity solution of the curvature motion. Left: simple shape smoothed by a finite difference scheme of the curvature motion; middle: smoothing with a median filter at the same scale; right: difference between left and middle images. The difference is no greater than the width of one pixel.

Proof. We saw in Lemma 14.4 that 1-localizability and affine invariance imply that c_{B}^{−} = 0. On the other hand, we have shown that c_{B}^{+} > 0 for only two cases (again Lemma14.4). Thus, we need the assumption that c_{B}^{+} > 0 to avoid the possibility of a vacuous result.

From Lemma 14.3 and Theorem 14.4 we know that the operators IS_{h} and SI_{h} are α-uniformly consistent with the first two equations of (15.19). Lemmas 14.5 and 14.6 state that SI_{h}IS_{h} is α-uniformly consistent with the third equation in (15.19). For each case, α = 4/3.

Lemma 13.3 states that IS_{h} and SI_{h} satisfy a β-uniform local comparison principle with β = 2. Although it is not specifically stated there, Lemma 13.4(iv) implies immediately that SI_{h}IS_{h} also satisfies β-uniform local comparison principle in case u is Lipschitz on ℝ^{2}, which is part of the assumptions for Theorem 16.7.

Since the hypotheses of Lemmas 16.3–16.5 are satisfied by all three operators, we know from Lemma 16.5 that the approximate solutions have a limit point 238CHAPTER 15. CURVATURE EQUATIONS AND ITERATED CONTRAST-INVARIANT OPERATORS

u. That is, for each case: IS_{h}, SI_{h}, and SI_{h}IS_{h}, there is a sequence u_{h_{n}} of approximate solutions that converge uniformly on compact subsets of ℝ^{+} × ℝ^{2} to a continuous function u. Finally, Proposition 15.13, the Barles–Souganidis theorem, states that u is a viscosity solution of the appropriate PDE in (15.19).

Figures 16.3 and 16.4 illustrate numerical results showing that affine-invariant filters really are affine invariant. A finite difference scheme is used to compute the action of the PDE in Figure 16.3. Figure 16.4 illustrates the same invariance using an iterated. inf-sup operator.

Figure 15.3: Affine invariance of (AMSS). We check the affine invariance of the affine and morphological scale space (AMSS). A simple shape (top-left) is smoothed using a finite differences discretization of (AMSS) followed by thresholding (bottom-left). We apply an affine transform, with determinant equal to 1, on the same shape (top-right), then the same smoothing process (middle-right), and finally the inverse of the affine transform (down-right). The final results of both processes are experimentally equal.

To the extent that we know viscosity solutions are unique (see Theorem 15.14), we have also shown that the set of approximate solutions {u_{h}}, 0 < h ≤ 1, has only one limit point (or accumulation point) in the topology of uniform convergence on compact sets.

This completes Part IV, and with it a large part of the overall program. But before moving to new topics, we add a few comments about scaling. We chose to 15.5. CONVERGENCE OF ITERATED AFFINE-INVARIANT OPERATORS239

Figure 15.4: Checking the invariance of an affine-invariant inf-sup operator. The images are the final outcomes of the same comparison process shown in Figure 15.3, with T replaced with an affine-invariant inf-sup operator. The structuring set B is an approximately affine-invariant set of 49 ellipses, all with same area. The inf-sup computation is costly and proves to be less affine invariant than the one obtained by a finite difference scheme. This is due to grid effects.

define uniformly consistent and approximate solutions in terms of a parameter α. This has the advantage that it agrees directly with the asymptotic results in Theorems 11.3 and 14.4. For example, Theorem 14.4 states that

T_{h}u(x) − u(x) = h^{4/3}c_{B}|Du(x)|(\frac{1}{2}curv(u)(x)^{+})^{1/3} + o(x, h^{4/3}),

so the T_{h} are α-uniformly consistent with the PDE, where α = 4/3. Another approach is to define a new set of operators S_{h} by S_{h} = T_{h^{3/4}}. Then the S_{h} are 1-uniformly consistent with the PDE. The approximate solutions for S_{h} are defined with α = 1, they converge, and the limit point is a viscosity solution of the PDE. This approach has the advantage that the notation is simpler. In either case, α = 1 or α > 1, the definition of the approximate solutions must be properly scaled.

240CHAPTER 15. CURVATURE EQUATIONS AND ITERATED CONTRAST-INVARIANT OPERATORS

15.6 Comments and references

The existence and uniqueness theory for the viscosity solutions of mean curvature motion and the relations of these solutions with other kinds of solutions (classical, variational) was developed independently by Evans and Spruck [95, 96, 97, 98] and by Chen, Gigo, and Goto [69, 70]. We do not follow their existence proofs, but rather the elegant numerical approximation schemes invented by Merriman, Bence, and Osher [195] and the subsequent convergence proof to the viscosity solution by Barles and Georgelin [37]. Other proofs of the iterated Gaussian median filter toward the mean curvature equation are given in [93] using semigroups and by Ishii [136]. Finally, we note the importance of iterated median filters for denoising applications [26] and [150].

Part V

An Axiomatic Theory of Scale Spaces

241

Chapter 16

Scale Spaces and Partial Differential Equations

This chapter and the next one are devoted to an axiomatic development of image smoothing. Our approach is base on the notion of a scale space:

Definition 16.1. A scale space is a family of image (function) operators {T_{t}}, t ∈ ℝ^{+}, defined on a domain of functions u : ℝ^{N} → ℝ.

Although this concept is completely abstract, it is clearly based on work presented in previous chapters. One can think of an operator T_{t} belonging to a scale space as the asymptotic limit of iterated filters, and in fact, the main purpose for developing this abstract theory is to classify and model the possible asymptotic behaviors of iterated filters. The program proceeds as follows: We first introduce several properties that smoothing operators are reasonably expected to have. These properties will be recognized as abstractions of results about iterated filters that we have already encountered; in particular, the function F that has played such an important role in relating iterated operators to differential equations appears in Definition 17.5. The next step is to define the now-familiar invariants for scale spaces, one at a time, and then deduce properties that F must have based on the assumed invariants of {T_{t}}. This will lead, for example, to a complete characterization of the function F for a linear scale space as the Laplacian.

The chapter contains seven figures that illustrate some of the concepts. Figures 16.4, 16.5, 16.6, and 16.7 are placed at the end of the chapter. Figure 16.4 illustrates numerically that linear smoothing is not contrast invariant. On the other hand, Figure 16.5 shows experimentally that the scale space AMSS is contrast invariant. The significance of contrast invariance for smoothing Tjunctions is illustrated in Figure 16.6. Figure 16.7 illustrates one of the most important twentieth century discoveries about human vision and compares it with computer vision.

243 244CHAPTER 16. SCALE SPACES AND PARTIAL DIFFERENTIAL EQUATIONS

16.1 Basic assumptions about scale spaces

The first order of business is to say something about the domain and range of the operators T_{t} belonging to a scale space {T_{t}}. Our approach requires that these operators are defined on all quadratic function from ℝ^{N} to ℝ. With this in mind, we are going to assume that the T_{t} are defined on all continuous functions that grow no faster than some polynomial. Formally, u is in the domain of T_{t} if there is a constant C and a positive integer n such that

|u(x)| ≤ C(1 + |x|^{n}).  (16.1)

We also assume that the T_{t}u satisfy this kind of condition.

In our context, the operators T_{t} are smoothing operators and the functions u are images. Thus, given an image u_{0}, T_{t}u_{0} = u(t, ·) is the image u_{0} smoothed at scale t. It is natural to abstract the idea that an image smoothed at scale t can be obtained from the image smoothed at scale s, s < t, without having to “go back” to the original image u_{0}. This concept is illustrated in Figure 16.1 and formulated in the next definition.

Definition 16.2. A scale space {T_{t}} is said to be pyramidal if there is another family of operators {T_{t+h,t}}, h ≥ 0, called transition operators, such that

T_{t+h} = T_{t+h,t}T_{t} and T_{0} = I,

where I denotes the identity operator.

We will sometimes denote the transition operators by {T_{s,t}}, 0 ≤ t ≤ s. Then T_{s} = T_{s,t}T_{t}, h = s − t, and T_{t,t} = I. Most, but not all, results are about pyramidal scale spaces. An important exception is Lemma 17.20, which is a key result in our program. We assume that the transition operators have the same domain and range as the T_{t}. This means that they are defined on functions that satisfy (16.1) and they return functions that satisfy (16.1).

A strong version of “pyramidal” is the semigroup property. Recall that we have already encountered this idea in Chapter 8 in connection with a dilation or an erosion generated by a convex set.

Definition 16.3. A scale space {T_{t}} is said to be recursive if T_{0} = I and

T_{s}T_{t} = T_{s+t} for all s, t ∈ ℝ.

Note that if {T_{t}} is recursive, then T_{t} can be obtained by iterating T_{t/n} n times. Another intuitive concept is that of “locality.” The thought that the action of a smoothing operator on a function u at x would be sensitive to what the function did far from x just does not make sense. This means that we want the action of the transition operators to depend essentially on the values of u(y) for y near x. Furthermore, we have had ample opportunity in earlier chapters to see the technical importance of locality. The related property of being monotonic is also intuitively and technically important. We combine these notions is the next definition.

16.1. BASIC ASSUMPTIONS ABOUT SCALE SPACES 245

Definition 16.4. A scale space {T_{t}} satisfies a local comparison principle if the following implications are true: For all u and v in the domain of definition, u(y) < v(y) for y in some neighborhood of x, x ≠ y, implies that

T_{t+h,t}u(x) ≤ T_{t+h,t}v(x) + o(h) for all sufficiently small h.

If u(y) ≤ v(y) for all y ∈ ℝ^{N}, then

T_{t+h,t}u(x) ≤ T_{t+h,t}v(x) for all x ∈ ℝ^{N} and all h > 0.

Our goal is to establish a classification of scale spaces. To do this, we need an assumption stating that a smooth image evolves smoothly with the scale space. From what we have seen in previous chapters, it should not be surprising that it is sufficient to assume this kind of property for quadratic functions. (From the semigroup point of view, the next assumption implies the existence of an infinitesimal generator for the semigroup T_{t}.)

Figure 16.1: The visual pyramid of scale space. Perception is thought of as a flow of images passing through transition operators T_{t,s}. These operators receive an image previously analyzed at scale s and deliver an image analyzed at a larger scale t. The scale t = 0 corresponds to the original percept. In this simple model, the perception process is irreversible: There is no feedback from coarse scales to fine scales.

Definition 16.5. Assume that u is a quadratic function defined for y ∈ ℝ^{N} and represent u as

u(y) = \frac{1}{2}⟨A(y − x), y − x⟩ + ⟨p, y − x⟩ + c,

246CHAPTER 16. SCALE SPACES AND PARTIAL DIFFERENTIAL EQUATIONS

where A = D^{2}u is an N × N matrix, p = Du is a vector in ℝ^{N}, and c = u(x) is a constant. A scale space {T_{t}} is said to be regular if there exists a function

F : (A, p, x, c, t) ↦ F (A, p, x, c, t)

that is continuous with respect to A and such that

\frac{T_{t+h,t}u(x) − u(x)}{h} → F (A, p, x, c, t) as h → 0.  (16.2)

It is useful to write (16.2) as T_{t+h,t}u(x) − u(x) = hF (A, p, x, c, t) + o(h), since replacing t with t − h shows that

T_{t,t−h}u(x) − u(x) = hF (A, p, x, c, t − h) + o(h).  (16.3)

Then, if F is continuous in t,

\frac{T_{t,t−h}u(x) − u(x)}{h} → F (A, p, x, c, t) as h → 0.  (16.4)

We encountered the notion of causality in section 2.6. The idea was quite simple, if not precise: As scale increases, no new features should be introduced by the smoothing operators. The image at scale t′ > t should be simpler than the image at scale t. Since we will constantly be considering scale spaces that are pyramidal and regular, and that satisfy the local comparison principal, it will be convenient to give these scale spaces a name.

Definition 16.6. A scale space {T_{t}} is said to be causal if it is pyramidal and regular, and if it satisfies the local comparison principle.

An immediate consequence of this definition is that the function F is nonincreasing in its first variable. This is the first of several properties of F that will follow from assumptions about the scale space {T_{t}}.

Lemma 16.7. If the scale space {T_{t}} is causal, then the function F is nondecreasing with respect to its first argument, that is, if A ≤ B, where A and B are symmetric matrices, then

F (A, p, c, x, t) ≤ F (B, p, c, x, t).  (16.5)

Proof. Let A and B be any N × N symmetric matrices with A ≤ B, and let p be any N-dimensional vector. Define Q_{A} and Q_{B} by

Q_{A}(y) = c + ⟨p, y − x⟩ + \frac{1}{2}⟨A(y − x), y − x⟩;
Q_{B}(y) = c + ⟨p, y − x⟩ + \frac{1}{2}⟨B(y − x), y − x⟩.

Then for fixed x and all y, Q_{A}(y) ≤ Q_{B}(y). Use the second part of the local comparison principal (monotonicity) to conclude that T_{t+h,t}Q_{A}(x) ≤ T_{t+h,t}Q_{B}(x) for all x ∈ ℝ^{N}. Noting that Q_{A}(x) = Q_{B}(x) = c and using the regularity of {T_{t}}_{t∈ℝ^{+}}, we see that

lim_{h→0} \frac{T_{t+h,t}Q_{A}(x) − Q_{A}(x)}{h} ≤ lim_{h→0} \frac{T_{t+h,t}Q_{B}(x) − Q_{B}(x)}{h},

which is the inequality F (A, p, c, x, t) ≤ F (B, p, c, x, t).

16.2. CAUSAL SCALE SPACES ARE GOVERNED BY PDES 247

We will see in the next section that the causality assumption implies that the scale space is governed by a PDE.

Exercise 16.1. Consider the extreme killer T_{t} defined in section 6.4, where t denotes the area threshold. Show that the family {T_{t}} is pyramidal and satisfies the global comparison principle, but that it does not satisfy the local comparison principle. Show that the family is, however, regular at t = 0 and, more precisely, that

F (A, p, 0) = 0 if p ≠ 0.

16.2 Causal scale spaces are governed by PDEs

The next result, Theorem 17.8, should be no surprise. It just says that for causal scale spaces, the regularity condition, which is defined in terms of quadratic forms, transfers directly to functions u that are C^{2}. This is a fundamental, although easily established, step in our program. Once we have established Theorem 17.8, we are ready to introduce invariants: Postulate that the scale space has certain invariance properties and conclude that F must have certain properties. This will tell us that causal scale spaces with certain invariances will be governed by a general class of PDEs.

Theorem 16.8. Assume that the scale space {T_{t}} is causal and that u is C^{2} at x. Then there exists a function F such that for all x ∈ ℝ^{N},

\frac{T_{t+h,t}u(x) − u(x)}{h} → F (D^{2}u(x), Du(x), u(x), x, t) as h → 0. (16.6)

Proof. The regularity assumption means that there is a function F, which is continuous in its first argument, such that

\frac{T_{t+h,t}v(x) − v(x)}{h} → F (A, p, x, c, t) as h → 0

for any quadratic function

v(y) = \frac{1}{2}⟨A(y − x), y − x⟩ + ⟨p, y − x⟩ + c.

Since we have assumed that u is C^{2} at x, we can expand u near x as

u(y) = u(x) + ⟨Du(x), y − x⟩ + \frac{1}{2}⟨D^{2}u(x)(y − x), y − x⟩ + o(|x − y|^{2}).

For ε > 0, define the quadratic functions Q^{+} and Q^{−} by

Q^{+}(y) = u(x) + ⟨Du(x), y − x⟩ + \frac{1}{2}⟨D^{2}u(x)(y − x), y − x⟩ + \frac{1}{2}ε⟨y − x), y − x⟩;
Q^{−}(y) = u(x) + ⟨Du(x), y − x⟩ + \frac{1}{2}⟨D^{2}u(x)(y − x), y − x⟩ − \frac{1}{2}ε⟨y − x), y − x⟩.

For sufficiently small |y − x|, x ≠ y,

Q^{−}(y) < u(y) < Q^{+}(y).

248CHAPTER 16. SCALE SPACES AND PARTIAL DIFFERENTIAL EQUATIONS

Use the facts that the scale space {T_{t}} is pyramidal (so the transition operators exist) and that it satisfies the local comparison principle to deduce that

T_{t+h,t}Q^{−}(x) ≤ T_{t+h,t}u(x) ≤ T_{t+h,t}Q^{+}(x)

for sufficiently small h. Since Q^{−}(x) = u(x) = Q^{+}(x), we have

T_{t+h,t}Q^{−}(x) − Q^{−}(x) ≤ T_{t+h,t}u(x) − u(x) ≤ T_{t+h,t}Q^{+}(x) − Q^{+}(x).

Now divide by h and let it tend to zero. Since {T_{t}} is regular we have the following limits:

lim_{h→0} \frac{T_{t+h,t}Q^{−}(x) − Q^{−}(x)}{h} ≤ lim inf_{h→0} \frac{T_{t+h,t}u(x) − u(x)}{h} ≤ lim sup_{h→0} \frac{T_{t+h,t}u(x) − u(x)}{h}
≤ lim_{h→0} \frac{T_{t+h,t}Q^{+}(x) − Q^{+}(x)}{h}.

Thus,

F (D^{2}u(x) − εI, Du(x), u(x), x, t) ≤ lim inf_{h→0} \frac{T_{t+h,t}u(x) − u(x)}{h}
≤ lim sup_{h→0} \frac{T_{t+h,t}u(x) − u(x)}{h}
≤F (D^{2}u(x) + εI, Du(x), u(x), x, t).

Part of the regularity assumption is that F is continuous in its first argument, so letting ε tend to zero shows that

lim_{h→0} \frac{T_{t+h,t}u(x) − u(x)}{h} = F (D^{2}u(x), Du(x), u(x), x, t).

This is about all we can conclude concerning the function F; to deduce more about F, we must assume more about the scale space. Most, but not all, of these assumptions will be that the scale space is invariant under some group of operations. Some of these invariants, like affine invariance, are rather special. The first one we consider is, however, an invariance we naturally expect all smoothing operators to have: Smoothing should not alter constants and smoothing should commute with the addition of constants.

Definition 16.9. A pyramidal scale space {T_{t}} is said to be invariant under grey level translations (or commutes with the addition of constants) if

T_{t+h,t}[0](x) = 0 and T_{t+h,t}(u + C)(x) = T_{t+h,t}u(x) + C  (16.7)

for all u, all constants C, and all x ∈ ℝ^{N}.

If T_{t+h,t} is a linear filter defined by T_{t+h,t}u = φ ∗ u, then this axiom is equivalent to the condition ∫ φ(x) dx = 1.

Proposition 16.10. Let {T_{t}} be a causal scale space that is invariant under grey level translations. Then its associated function F : (A, p, c, x, t) ↦ F (A, p, c, x, t) does not depend on c. Furthermore, F (0, 0, c, x, t) = 0.

16.3. SCALE SPACES YIELD VISCOSITY SOLUTIONS 249

Proof. Consider the quadratic function u(y) = (1/2)⟨A(y −x), y −x⟩+⟨p, y − x⟩ + c, and let C be an arbitrary real number. By the regularity assumption

\frac{T_{t+h,t}(u + C)(x) − (u + C)(x)}{h} → F (A, p, c + C, x, t) as h → 0.

Using the grey level translation invariance and regularity again,

\frac{T_{t+h,t}(u + C)(x) − (u + C)(x)}{h} = \frac{T_{t+h,t}u(x) + C − u(x) − C}{h} → F (A, p, c, x, t)

as h → 0. These last two limits imply that F (A, p, c + C, x, t) = F (A, p, c, x, t), so F does not depend on c. If A = 0 and p = 0, then T_{t+h,t}(u)(x) − (u)(x) = c − c = 0 and F (0, 0, c, x, t) = 0.

From now on, we assume that the scale space is causal and invariant under grey level translations. We will thus suppress c and write F (A, p, c, x, t) = F (A, p, x, t).

16.3 Scale spaces yield viscosity solutions

We are going to prove a result that connects a causal scale space {T_{t}} with a viscosity solution of the PDE associated with {T_{t}}. In fact, we will prove that T_{t}u_{0} is a viscosity solution of the equation

\frac{∂u}{∂t}(t, x) = F (D^{2}u(x), Du(x), x, t),  (16.8)

where F is the function associated with {T_{t}} by regularity (Definition 17.5).

Theorem 16.11. Assume that the scale space {T_{t}} is causal and commutes with grey level translations; assume also that F is continuous in t. Then the function u defined by u(t, x) = T_{t}u_{0}(x) is a viscosity solution of (16.8).

Proof. We will show that u is a viscosity subsolution; the proof that it is also a viscosity supersolution is a similar argument with the inequalities going in the opposite direction.

Assume that φ is C^{∞} and that (t, x) ∈ (0, +∞) × ℝ^{N} is a point at which the function u − φ has a strict local maximum. The point (t, x) is fixed, so we will denote the variable by (s, y). We need to show that

\frac{∂φ}{∂t}(t, x) − F (D^{2}φ(t, x), Dφ(t, x), x, t) ≤ 0.  (16.9)

(Note that we would usually consider two cases: (1) Dφ(t, x) ≠ 0; (2) Dφ(t, x) = 0 and D^{2}φ(t, x) = 0. Since F (0, 0, x, t) = 0 by Proposition 18.10, it is sufficient to prove (16.9).)

On the basis of Lemma 15.5, we assume that φ is of the form φ(s, y) = f(y) + g(s). Since the operators commute with the addition of constants, we may also assume that u(t, x) = φ(t, x) = f(x) + g(t). Of course, both f and g are C^{∞}. Thus, for (s, y) in some neighborhood of (t, x), we have u(s, y) ≤ φ(s, y) = f(y) + g(s). In particular, we have

u(t − h, y) ≤ f(y) + g(t − h)

250CHAPTER 16. SCALE SPACES AND PARTIAL DIFFERENTIAL EQUATIONS

for all sufficiently small h > 0. Since the operators T_{t,t−h} satisfy the local comparison principle and commute with the addition of constants, we have

T_{t,t−h}u(t − h, ·)(x) ≤ T_{t,t−h}f(x) + g(t − h).

By definition of the transition operators, T_{t,t−h}u(t − h, ·)(x) = u(t, x) = f(x) + g(t). Thus, we see that

g(t) − g(t − h) ≤ T_{t,t−h}f(x) − f(x),

which by (16.3) we can write as

g(t) − g(t − h) ≤ hF (D^{2}f(x), Df(x), x, t − h) + o(h).

Divide by h, use the fact the F is continuous in t, let h tend to zero, and conclude that

g′(t) ≤ F (D^{2}f(x), Df(x), x, t).

Since ∂φ/∂t = g′, D^{2}φ = D^{2}f, and Dφ = Df, this proves the result.

16.4 Scale space invariants and implications for F

We have already seen the implication for F of assuming that a scale space {T_{t}} is invariant under the addition of constants (Proposition 17.10). This section is devoted to continuing this program: Assume an invariant for {T_{t}} and deduce its implication for F. There are two steps: defining the invariant and deducing its implications.

16.4.1 Translation, rotation, and reflection

These invariants concern the underlying space ℝ^{N}, and they are easily defined. Translation invariance for operators was defined in Definition 6.4 using the translation operator τ_{z}: τ_{z}u(x) = u(x − z) for all x, z ∈ ℝ^{N}.

Definition 16.12. A pyramidal scale space {T_{t}} is said to be translation invariant if

T_{t+h,t}τ_{z} = τ_{z}T_{t+h,t} for all z ∈ ℝ^{N}, t ≥ 0, and h ≥ 0.  (16.10)

Proposition 16.13. Assume that {T_{t}} is a causal, translation invariant scale space. Then its associated function F does not depend on x.

Proof. Consider the two quadratic functions u and τ_{x}u defined by

u(y) = \frac{1}{2}⟨Ay, y⟩ + ⟨p, y⟩ + c and τ_{x}u(y) = \frac{1}{2}⟨Ay − x, y − x⟩ + ⟨p, y − x⟩ + c

By the regularity assumption,

T_{t+h,t}u(0) − u(0) = hF (A, p, 0, c, t) + o(h);  (16.11)
T_{t+h,t}τ_{x}u(x) − τ_{x}u(x) = hF (A, p, x, c, t) + o(h).  (16.12)

16.4. SCALE SPACE INVARIANTS AND IMPLICATIONS FOR F 251

By translation invariance,

T_{t+h,t}τ_{x}u(x) − τ_{x}u(x) = τ_{x}T_{t+h,t}u(x) − τ_{x}u(x) = T_{t+h,t}u(0) − u(0).

Thus we see from (16.11) and (16.12) that hF (A, p, 0, c, t) = hF (A, p, x, c, t) + o(h). Divide both sides by h and let h → 0 to see that

F (A, p, x, c, t) = F (A, p, 0, c, t).

This takes care of translations; rotations and reflections are combined in the group of linear isometries. If P is a linear isometry of ℝ^{N}, then the function P u is defined by P u(x) = u(P x).

Definition 16.14. A pyramidal scale space {T_{t}} is said to be Euclidean invariant (or isotropic) if

P T_{t+h,t} = T_{t+h,t}P  (16.13)

for all linear isometries P of ℝ^{N}, all t ≥ 0, and all h ≥ 0.

We denote the group of linear isometries of ℝ^{N} by O_{N} = O(ℝ^{N}). Any transform P ∈ O_{N} can be represented uniquely by an N × N orthogonal matrix P, assuming an orthonormal basis. We do not make a distinction between the operator P and the matrix P. If P ∈ O_{N}, then recall that its transpose P′ ∈ O_{N}, and P P′ = I. Recall also the given a symmetric matrix A there is always a P ∈ O_{N} such that P AP′ is diagonal. Euclidean invariance is illustrated in Figure 16.2.

Lemma 16.15. If a translation invariant causal scale space {T_{t}} is isotropic, then for every R ∈ O_{N},

F (RAR′, Rp, t) = F (A, p, t),  (16.14)

where F is the function associated with {T_{t}} by regularity.

Proof. Consider a quadratic function u(y) = (1/2)⟨Ay, y⟩+⟨p, y⟩ and let F be the function associated with {T_{t}}. We know that the value of F is determined by the action of T_{t+h,t} on u at t, that is,

T_{t+h,t}u(x) − u(x) = hF (A, p, t) + o(h).

In particular,

lim_{h→0} \frac{T_{t+h,t}u(0) − u(0)}{h} = F (A, p, t).  (16.15)

Let R be any element of O_{N}. Then

u(Ry) = \frac{1}{2}⟨ARy, Ry⟩ + ⟨p, Ry⟩ = \frac{1}{2}⟨R′ARy, y⟩ + ⟨R′p, y⟩.

Thus we know immediately that

T_{t+h,t}(u ∘ R)(x) − u(Rx) = hF (R′AR, R′p, t) + o(h),

252CHAPTER 16. SCALE SPACES AND PARTIAL DIFFERENTIAL EQUATIONS

Figure 16.2: An isotropic filter and a nonisotropic filter. The left frame contains simple shapes that can be deduced from each other by rotations. The center image is the closing of the left image by a horizontal rectangle of size 6 × 2 pixels. This nonisotropic filter produces different results, depending on the shapes orientations. The right image is the closing of the left image by a circle of radius 4 pixels, which has the same area, up to the pixel precision, as the rectangle used in the center image. This filter is isotropic; thus, as one can see, the resulting shapes can be deduced from each other by rotations.

where u ∘ R denotes the function defined by u ∘ R(y) = u(Ry). The assumption that {T_{t}} is isotropic means that T_{t+h,t}(u ∘ R)(x) = T_{t+h,t}u(Rx), so

T_{t+h,t}(u ∘ R)(x) − u(Rx) = T_{t+h,t}u(Rx) − u(Rx).

From this we conclude that

lim_{h→0} \frac{T_{t+h,t}(u ∘ R)(x) − u(x)}{h} = lim_{h→0} \frac{T_{t+h,t}u(Rx) − u(Rx)}{h} = F (R′AR, R′p, t).

By letting x = 0 in these limits, we conclude from (16.15) that

F (A, p, t) = F (R′AR, R′p, t).

Replacing R with R′ completes the proof.

We can summarize this section by saying that if a causal scale space {T_{t}} is invariant under Euclidean motions (including reflections), then the associated 16.4. SCALE SPACE INVARIANTS AND IMPLICATIONS FOR F 253

function F is also invariant under these motions, in the sense that F does not depend on x (translations) and F (A, p, t) = F (RAR′, Rp, t) for all orthogonal matrices R (rotations and reflections).

16.4.2 Contrast invariance

Definition 16.16. A pyramidal scale space {T_{t}} is said to be contrast invariant if

g ∘ T_{t+h,t} = T_{t+h,t} ∘ g

for any nondecreasing continuous function g : ℝ → ℝ.

An immediate consequence of this definition is that a contrast invariant scale space commutes with the addition of constants, that is, it satisfies Definition 17.9. To see this, just take g(s) = 0 and g(s) = s+C. Thus, in the next lemma, the function F does not depend on c.

Lemma 16.17. If a translation invariant causal scale space {T_{t}} is contrast invariant, then its associated function F satisfies the following condition:

F (μA + λp ⊗ p, μp, t) = μF (A, p, t),  (16.16)

where A is any symmetric N × N matrix, p is any N-dimensional vector, λ is any real number, and μ is any real number greater than or equal to zero.

Proof. The assumption is that g ∘ T_{t+h,t} = T_{t+h,t} ∘ g for any nondecreasing continuous function g, but to prove the lemma, we need only to assume that this is true for g that are C^{2}. The arbitrary parameters λ and μ will be associated with g as follows: g′(0) = μ and g′′(0) = λ. The proof depends on the fact that we can choose a C^{2} contrast change g with arbitrary values for g′(0) ≥ 0 and g′′(0) ∈ ℝ. Although we have used the notation before, recall that p ⊗ p denotes the N × N matrix whose entries are p_{i}p_{j}, i, j ∈ {1, 2, …, N}. Here are two more preliminaries: Given a C^{2} function u, we have these two applications of the chain rule:

D(g(u)) = g′(u)Du, and D^{2}(g(u)) = g′(u)D^{2}u + g′′(u)Du ⊗ Du. (16.17)

We can now begin. Choose any quadratic function of the form

u(y) = (1/2)⟨Ay, y⟩ + ⟨p, y⟩.

(We have taken x = 0 without loss of generality based on the assumption that {T_{t}} is translation invariant.) We know from the assumptions and equations (16.17), plus the relations u(0) = 0, Du(0) = p, and D^{2}u(0) = A, that

T_{t+h,t}g(u)(0) − g(0) = hF (g′(0)A + g′′(0)p ⊗ p, g′(0)p, t) + o(h).

Since {T_{t}} is contrast invariant, T_{t+h,t}g(u)(0) = g(T_{t+h,t}u(0)), so

g(T_{t+h,t}u(0)) − g(0) = hF (g′(0)A + g′′(0)p ⊗ p, g′(0)p, t) + o(h).

From regularity, we have

T_{t+h,t}u(0) − u(0) = hF (A, p, t) + o(h),

254CHAPTER 16. SCALE SPACES AND PARTIAL DIFFERENTIAL EQUATIONS

so we can write g(T_{t+h,t}u(0)) = g(hF (A, p, t) + o(h)). Thus for small enough h,

g(T_{t+h,t}u(0)) = g(0) + g′(0)(hF (A, p, t) + o(h)) + o(h)

and

g′(0)hF (A, p, t) + g′(0)o(h)) + o(h) = hF (g′(0)A + g′′(0)p ⊗ p, g′(0)p, t) + o(h).

Dividing this by h and letting h → 0 shows that

g′(0)F (A, p, t) = F (g′(0)A + g′′(0)p ⊗ p, g′(0)p, t),

or

μF (A, p, t) = F (μA + λp ⊗ p, μp, t),

which is what we wished to prove.

16.4.3 Scale and affine invariance

The invariants we have considered can be classified according to the nature of the transformations under which the scale space is invariant. In the case of contrast invariance, the transformation u ↦ g(u) involves the range of the function. The other invariants have been defined by mappings of the domain of u. In fact, we have considered all of the mappings of ℝ^{N} into ℝ^{N} that preserve distance: translations, rotations, and reflections. In this section, we study invariants defined by mappings of ℝ^{N} that do not necessarily preserve distance. Specifically, we consider all linear mappings A : ℝ^{N} → ℝ^{N}, where the matrix A has nonzero determinant. The simplest of these mappings is given by x ↦ λx, λ > 0. This generates a transformation of the real-valued functions u defined on ℝ^{N}, which we denote by H_{λ}. Thus, H_{λ}u(x) = u(λx).

The main purpose of this section is to established a normalized link between scale (t) and space (x). This is the content of Lemma 17.20. We consider families of function operators {T_{t}}, t ≥ 0; they are scale spaces, but they are not necessarily pyramidal. At the abstract level they are not necessarily smoothing operators, but it will be handy to think of them as smoothing operators. We always assume that the mapping t : [0, ∞) ↦ T_{t} is one-to-one. (We do not want the scale space to be cyclic.) In fact, we need no other assumptions about the family {T_{t}}, except the scale-invariance assumptions given in Definitions 17.18. Thus, the results will be true for any scale-invariant family of operators {T_{t}}, where t ∈ ℝ^{+}. The consequences of the normalization between scale and space are therefore wide-ranging. The general principle is that “the relation between scale and space can be normalized,” and more precisely, normalized so t′ = λt, where t is the scale parameter before “zooming” by the factor λ, and t′ is the smoothing scale after “zooming.”

Definition 16.18. A family of operators {T_{t}} is said to be scale invariant if there exists a rescaling function t′ : (t, λ) ↦ t′(t, λ), defined for all λ > 0 and t ≥ 0, such that

H_{λ}T_{t′} = T_{t}H_{λ}.  (16.18)

It {T_{t}} is pyramidal, then

H_{λ}T_{t′,s′} = T_{t,s}H_{λ},  (16.19)

16.4. SCALE SPACE INVARIANTS AND IMPLICATIONS FOR F 255

where t′ = t′(t, λ) and s′ = t′(s, λ). In addition, the function t′ is assumed to be differentiable with respect to t and λ, and the function ϕ defined by ϕ(t) = (∂t′/∂λ)(t, 1) is assumed to be continuous and positive for t > 0.

This definition implies, in particular, that t′ is continuous in t and λ. Condition (16.19), in fact, implies condition (16.18), and is thus a slightly stronger assumption. It has the advantage of making our classification of scale-invariant scale spaces easier. We will see, however, that condition (16.18) is sufficient to prove that the function t′ : (t, λ) ↦ t′(t, λ) can be normalized. (This can be seen by checking the proof of Lemma 17.20, which uses only condition (16.18) and its extension condition (16.20).) We cannot fix a priori the form of t′, since it can vary in specific examples of scale spaces. We can, however, normalize it, which means that we can always rescale a scale-invariant scale space so t′ = λt.

Scale invariance means that the result of applying a scale-invariant scale space {T_{t}} is independent of the size of the analyzed features. This is very important for analyzing natural images, since the same object can be captured at very different distances and at therefore at vary different scales (see Figure 16.3). It is thus essential for the stability of shape analysis that the result of an analysis of an object not yield different “shapes” at different distances. The sequence of shapes obtained by scale space must be independent of the (a priori unknown) size of the object. Of course, we cannot impose the condition t′ = t, since the scale of smoothing and the scale of the image are covariant, as can be appreciated by considering linear filtering and the heat equation.

The assumption that (∂t′/∂λ)(t, 1) > 0 can be interpreted by considering the relation H_{λ}T_{t′} = T_{t}H_{λ} when the scale λ increases before the analysis by T_{t}, that is, when the size of the image is reduced before analysis. Then the corresponding scale before reduction is increase. Informally, we can say that the scale of analysis increases with the size of the picture. This is a natural assumption that is satisfied by all the classical models. The continuity and differentiability assumptions about t′ are also satisfied by the classical models and seems natural. It is easy to determine the function t′ for several classical scale spaces. For the morphological operators, dilation and erosion, it is clear that t′(t, λ) = λt, and for the heat equation and mean curvature motion we have t′(t, λ) = λ^{2}t.

The next definition (axiom) introduces an invariant that implies both isotropic and scale invariance; it also means that the scale space is invariant under any orthographic projection of a planar shape. Combining these invariants leads to an arbitrary nonsingular transformation A of the plane. We write Au(x) = u(Ax).

Definition 16.19. A family of operators {T_{t}} is said to be affine invariant if it is scale invariant and if the following conditions hold: The associated function t′ can be extended to a function t′ : (t, A) ↦ t′(t, A), where t ≥ 0 and A is any linear mapping A : ℝ^{N} → ℝ^{N} with det(A) ≠ 0, such that t′(t, λ) = t′(t, λI) and such that

AT′_{t} = T_{t}A.  (16.20)

If {T_{t}} is pyramidal, the transition operators T_{t,s} satisfy the commutation relation

AT_{t′,s′} = T_{t,s}A  (16.21)

for all 0 ≤ s ≤ t, where t′ = t′(t, A) and s′ = t′(s, A).

256CHAPTER 16. SCALE SPACES AND PARTIAL DIFFERENTIAL EQUATIONS

Figure 16.3: A multiscale world. This series of images is an experiment to show the relative perception of objects seen at different distances. Each photograph, after the first one, was taken by stepping forward to produce a snapshot from a distance closer than the one before. The rectangle in each image outlines the part of the object that appears in the next image. Clearly, as one gets closer to the subject, the visual aspect changes and new structures appear. Thus, computing primitives in an image is always a scale-dependent task, and it depends on the distance to objects. When we look at an object from a certain distance, we do not perceive the very fine structure: For instance, leaves cannot be seen in the two first photographs because we are too far from the trees. Nor do we see them in the last two, since we are now too close. Multiscale smoothing of a digital image tries to emulate this physical and perceptual phenomena by defining a smoothing at different scales that eliminates the finer structures at a scale t, but minimally modifies the image at scales above t.

This property means that the result of applying the scale space {T_{t}} to a image is independent of the distance and orientation in space of the analyzed planar image. Indeed, any affine map u ↦ Au can be thought of as the anamorphosis of a planar image u as it is presented to the eye at any distance sufficiently large with respect to its size and with an arbitrary orientation in space. (Recall that an affine transformation is a projective transformation with the “focal point” at infinity.)

The fact that the function t′ can be different for each scale space may seem mysterious. We will “fix” this in the next lemma by showing that we can, 16.4. SCALE SPACE INVARIANTS AND IMPLICATIONS FOR F 257

without loss of generality, assume that scale-invariant scale spaces have the same scale-space function, namely t′ = λt. This is so because we can by Lemma 17.20 renormalize the scale space. On the other hand, it may be more convenient to use a different normalization, say, t′ = λ^{2}t in the case of a linear scale space. We will see, in the process of identifying all scale spaces having certain invariants, that the next lemma is an essential tool. This is because the scale spaces fall into equivalence classes, where all members of a given class differ only by a rescaling. Thus, we use the phrase “up to a rescaling.”

Lemma 16.20 (Scale normalization). Assume that the mapping t ↦ T_{t}, t ∈ [0, ∞), is one-to-one and that T_{0} = I.

(i) If the family of operators {T_{t}} is scale invariant, then there exists an increasing differentiable function σ : [0, ∞) → [0, ∞) such that t′(t, λ) = σ^{−1}(σ(t)λ). If the operators S_{t} are defined by S_{t} = T_{σ^{−1}(t)}, then

t′(t, λ) = tλ  (16.22)

for the rescaled analysis {S_{t}}.

(ii) If the family {T_{t}} is affine invariant, then the function t′; (t, B) ↦ t′(t, B) depends only on t and |detB|, in particular, t′(t, B) = t′(t, |detB|^{1/N}), and t′ is increasing with respect to t. In addition, there exists an increasing differentiable function σ from [0, ∞) to [0, ∞) such that t′(t, B) = σ^{−1}(σ(t)|detB|^{1/N}). If the operators S_{t} are defined by S_{t} = T_{σ^{−1}(t)}, then

t′(t, B) = t|detB|^{1/N}  (16.23)

for the rescaled analysis {S_{t}}.

Proof. We will prove (ii) and then show how this proof can be reduced to a proof of (i). The first step is to show that

t′(t, AB) = t′(t′(t, A), B)  (16.24)

for any linear transforms A and B with nonzero determinants. To see this, write

ABT_{t′(t,AB)} = T_{t}AB = AT_{t′(t,A)}B = ABT_{t′(t′(t,A),B)}.

Since the determinant of AB does not vanish, we have T_{t′(t,AB)} = T_{t′(t′(t,A),B)}, and since t ↦ T_{t} is one-to-one, we have (16.24).

The second step is to show that the function t′ is increasing with respect to t. We begin by proving that t ↦ t′(t, A) is one-to-one for any A with detA ≠ 0. If this were not the case, then there would be some A, detA ≠ 0, and some s and t, s ≠ t, such that t′(s, A) = t′(t, A). This implies that

T_{s}A = AT_{t′(s,A)} = AT_{t′(t,A)} = T_{t}A.

Since detA ≠ 0, this means that T_{s} = T_{t}, and since t ↦ T_{t} is one-to-one, we have s = t. The conclusion is that t′ is one-to-one. By hypothesis, T_{0} = I, and since AT_{t′(0,A)} = T_{0}A = A, we see that t′(0, A) = 0 for all A. By definition, t′ is continuous and nonnegative; we have just shown that it is also one-to-one and the t′(0, A) = 0. Said another way, t′ ↦ t′(t, A) is a homeomorphism of 258CHAPTER 16. SCALE SPACES AND PARTIAL DIFFERENTIAL EQUATIONS

[0, ∞) onto [0, ∞) for every A with nonzero determinant. It is then an easy exercise to show that t′ is increasing in t: If this were not the case, there would be 0 ≤ s < t such that t′(s, A) ≥ t′(s, A). Then there is an r, 0 ≤ r < s such that t′(r, A) = t′(t, A), which contradicts the fact that t′ is one-to-one.

Let R be an orthogonal matrix. The third step is to show that

t′(t, R) = t.  (16.25)

Define t_{1} = t′(t, R) and t_{n+1} = t′(t_{n}, R). From (16.24) we see that t_{n} = t′(t, R^{n}). There are three cases to consider: (1) t_{1} = t; (2) t_{1} < t; (3) t_{1} > t. If case (1) holds, then we have (16.25) and are done. In case (2), the fact that t′ is strictly increasing in t implies that the sequence t_{n} is strictly decreasing. Similarly, in case (3), the sequence t_{n} is strictly increasing. If R^{m} = I for some m ≠ 0, then the sequence t_{n} is periodic and t_{m+1} = t_{1}. In case (2), this implies that t_{m+1} < t_{1} and t_{m+1} = t_{1}, which is impossible. Similarly, case (3) is untenable. Consequently, if R^{m} = I for some m ≠ 0, then cases (2) and (3) must be rejected and case (1) holds, which means that (16.25) is true. In particular, t′(t, I) = t. To continue, we assume that R^{n} ≠ I for n > 0. Since the set of orthogonal matrices is compact, there is a subsequence n_{k} and an orthogonal matrix P such that R^{n_{k}} → P as k → ∞. Let m_{k} = n_{k+1} − n_{k}. Then R^{m_{k}} → I as k → ∞. Since t′ is continuous, lim_{k→∞} t′(t, R^{m_{k}}) → t′(t, I) = t. In both cases (2) and (3), this leads to the contradiction that t < t: In case (2), we have t = lim_{k→∞} t′(t, R^{m_{k}}) < t, and in case (3) we have t = lim_{k→∞} t′(t, R^{m_{k}}) > t. Once again, cases (2) and (3) are untenable, and case (1) must be true.

The next part of the proof is pure matrix theory. Let B be any N × N nonsingular matrix (linear transform of ℝ^{N}). The B can be written as B = R_{1}DR_{2}, where R_{1} and R_{2} are orthogonal and D is diagonal. Furthermore, d_{ii} = λ_{i} > 0 and the λ_{i} are, up to a sign, the eigenvalues of B. As a consequence, using (16.24) and (16.25), we see that

t′(t, B) = t′(t, D).

The matrix D can be represented as D = A(λ_{1})R_{2}A(λ_{2})R_{2}^{−1} ⋯ R_{N}A(λ_{N})R_{N}^{−1}, where the mapping A(λ_{i}) is defined by (x_{1}, x_{2}, …, x_{N}) ↦ (λ_{i}x_{1}, x_{2}, …, x_{N}) and R_{j} is the orthogonal mapping that interchanges x_{1} and x_{j}. Repeated use of (16.24) and (16.25) and the fact that A(λ_{1})A(λ_{2}) ⋯ A(λ_{N}) = A(λ_{1}λ_{2} ⋯ λ_{N}) shows that

t′(t, D) = t′(t, A(λ_{1}λ_{2} ⋯ λ_{N})).

Now write (λ_{1}λ_{2} ⋯ λ_{N})^{1/N} = λ and consider the matrix λI. As we have done above, we can write λI = A(λ)R_{2}A(λ)R_{2}^{−1} ⋯ R_{N}A(λ)R_{N}^{−1}. Then using (16.24) and (16.25) again, we see that

t′(t, λI) = t′(t, A(λ^{N})) = t′(t, A(λ_{1}λ_{2} ⋯ λ_{N})),

and we conclude that

t′(t, B) = t′(t, D) = t′(t, A(λ_{1}λ_{2} ⋯ λ_{N})) = t′(t, λI).

By definition, t′(t, λI) = t′(t, λ), where we have used the same notation for the function t′ : (t, λ) ↦ t′(t, λ) and its extension t′ : (t, λI) ↦ t′(t, λI). The conclusion is that

t′(t, B) = t′(t, |detB|^{1/N})  (16.26)

16.4. SCALE SPACE INVARIANTS AND IMPLICATIONS FOR F 259

for all transforms B that have nonzero determinants.

We turn now to the rescaling problem. We wish to find an increasing differentiable function σ that satisfies the equation t′(t, λ) = σ^{−1}(σ(t)λ), or equivalently, σ(t′(t, λ)) = σ(t)λ. Differentiating the last equation with respect to λ and then setting λ = 1, shows that

ϕσ′ = σ,  (16.27)

so it is reasonable to define σ by

σ(t) = exp ( ∫_{1}^{t} \frac{ds}{ϕ(s)}).

Since by assumption ϕ is continuous and ϕ(s) > 0 for s > 0, σ is clearly increasing and differentiable. It remains to show that t′(t, λ) = σ^{−1}(σ(t)λ).

From equations (16.24) and (16.25), we know that t′(t, μν) = t′(t′(t, μ), ν) and t′(t, 1) = t for all positive μ and ν. Differentiating both sides of the first equation with respect to μ and then setting μ = 1 and ν = λ shows that

λ\frac{∂t′}{∂λ} (t, λ) = \frac{∂t′}{∂t} (t, λ)\frac{∂t′}{∂λ} (t, 1).  (16.28)

By Definition 17.18, the function ϕ given by ϕ(t) = (∂t′/∂λ)(t, 1) is continuous and positive for t > 0. We have shown that t′ is strictly increasing, thus the right-hand side of (16.28) is nonnegative. This implies that

\frac{∂t′}{∂λ} (t, λ) ≥ 0.  (16.29)

If t′(t, λ) = σ^{−1}(σ(t)λ) is going to be true, then it is also true if we replace λ with λ/σ(t), which is the equation t′(t, λ/σ(t)) = σ^{−1}(λ). This prompts us to examine the function g defined by g(t, λ) = t′(t, λ/σ(t)). When we differentiate g with respect to t, we will see that this derivative is zero:

\frac{∂g}{∂t} (t, λ) = \frac{∂t′}{∂t} (t, λ/σ(t)) − λ\frac{σ′(t)}{σ^{2}(t)} \frac{∂t′}{∂λ} (t, λ/σ(t))

= \frac{∂t′}{∂t} (t, λ/σ(t)) − \frac{σ′(t)}{σ(t)} \frac{∂t′}{∂t} (t, λ/σ(t))ϕ(t) (using (16.28))

= 0 (using σ′ϕ = σ).

Thus, g does not depend on t, and we know from (16.29) that g in nondecreasing. Since g is also differentiable, we conclude that g(t, λ) = β(λ), where β is differentiable and nondecreasing. By replacing λ with λσ(t), we have

t′(t, λ) = β(λσ(t)).  (16.30)

By differentiating both sides of this equation with respect to λ and then letting λ = 1, we see that ϕ(t) = σ(t)β′(σ(t)) = ϕ(t)σ′(t)β′(σ(t)). Dividing both sides by ϕ(t) shows that

\frac{∂β(σ(t))}{∂t}(t) = 1.

Integrating this relation from zero to t yields the equation β(σ(t)) = t+β(σ(0)). Since t′(0, λ) = 0, β(σ(0)) = 0 by (16.30), and we conclude that β = σ^{−1}.

260CHAPTER 16. SCALE SPACES AND PARTIAL DIFFERENTIAL EQUATIONS

To complete the proof of (ii), we must show that the operators S_{t} by S_{t} = T_{σ^{−1}(t)} are affine invariant with t′(t, λ) = λt. Thus let B be any nonsingular linear mapping and let λ = |detB|^{1/N}. Then

S_{t}B = T_{σ^{−1}(t)}B = BT_{t′(σ^{−1}(t),λ)} = BT_{σ^{−1}(λσ(σ^{−1}(t)))} = BT_{σ^{−1}(λt)} = BS_{λt}.

The proof of (i) is just the “image” of the proof of (ii) under the obvious mappings B ↦ |detB|^{1/N} and λI ↦ H_{λ}, which entail t′(t, B) ↦ t′(t, |detB|^{1/N}), and so on.

Lemma 16.21. If a translation invariant causal scale space {T_{t}} is affine invariant, then, after the appropriate renormalization, its associated function satisfies the following condition:

F (BAB′, Bp, t) = |detB|^{1/N}(A, p, |detB|^{1/N}t)  (16.31)

for any nonsingular linear map B. If a translation invariant causal scale space {T_{t}} is scale invariant, then, after the appropriate renormalization, its associated function satisfies the following condition:

F (μ^{2}A, μp, t) = μF (A, p, μt)  (16.32)

for any μ > 0.

Proof. Recall that we have made the blanket assumption that causal spaces are invariant under the addition of constants. Thus, we assume that F does not depend on c or x. Assume that B is a linear map and that λ = |detB|^{1/N}. We also assume that the scale space {T_{t}} is normalized so that T_{t+h,t}B = BT_{λ(t+h),λt}. Let u be the quadratic function defined by u(y) = (1/2)⟨Ay, y⟩ + ⟨p, y⟩. Then

T_{t+h,t}[u(By)](0) = T_{λ(t+h),λt}u(B0) = T_{λ(t+h),λt}u(0).

Since u(B′y) = (1/2)⟨BAB′y, y⟩ + ⟨Bp, y⟩ and by the regularity of {T_{t}},

T_{t+h,t}[u(B′y)](0) = hF (BAB′, Bp, t) + o(h).

Also by regularity

T_{λ(t+h),λt}u(0) = λhF (A, p, λt) + o(λh).

Thus we have hF (BAB′, Bp, t) + o(h) = λhF (A, p, λt) + o(λh). Dividing by h and letting h → 0 proves the first part of the lemma. To prove the second part, just replace B with μI in the proof of the first part.

16.5 Axiomatic approach to linear scale space

We are going to use previous results from this chapter, in particularly Theorem 17.8, to characterize the heat equation ∂u/∂t = Δu as the unique scale space that is both linear and isotropic. A consequence for image processing is that linear smoothing and contrast-invariant smoothing are incompatible. (Recall that we showed in section 2.3 that the heat equation was not contrast invariant.

16.5. AXIOMATIC APPROACH TO LINEAR SCALE SPACE 261

This is illustrated numerically in Figure 16.4.) At some level, this explains the coexistence of at least two different schools of image processing: contrastinvariant mathematical morphology on the one hand, and classical linear scale space on the other, which is essentially convolution with the Gaussian (Theorem 2.3).

For our axiomatic approach, we require that the scale spaces are defined on quadratic functions. Since we are working with the heat equation, this means that we need to extend the work done in Chapter 1. We showed there that u(t, x) = G_{t} ∗ u_{0}(x), where G is the Gaussian, was a classical solution of the heat equation when u_{0} was in L_{C}^{1}, which means that u_{0} was C-periodic and integrable on C = [−1, 1]^{N} (Theorem 1.10). We wish to extend this result to include quadratic functions, and, more generally, functions that grow no faster at infinity than a polynomial. This is the content of the next exercise.

Exercise 16.2. Let u_{0} be a measurable function defined on ℝ^{N} having the property that there exist a constant C > 0 and an integer k such that

|u_{0}(x)| ≤ C(1 + |x|^{k})

for all x ∈ ℝ^{N}. Show that the function u defined by u(t, x) = G_{t} ∗ u_{0}(x) is well defined and C^{∞} on (0, ∞) × ℝ^{N} and that it is a classical solution of the heat equation. Hints: Everything follows from the fact that the Gaussian and all of its derivatives decay exponentially at infinity. Various propositions leading to Theorem 1.10 are quickly extended to the case at hand. Finally, note that there is no question here of uniqueness; we only need the fact that u = G ∗ u_{0} satisfies the heat equation.

Theorem 16.22. Let {T_{t}} be a translation-invariant causal scale space. Assume that {T_{t}} is defined on a family of functions F that has the following property P: For each x ∈ ℝ^{N}, each p ∈ ℝ^{N}, and each N × N symmetric matrix A, there exists a function u ∈ F such that Du(x) = p and D^{2}u(x) = A. If the operators T_{t,s} are linear and isotropic (Euclidean invariant), then F (D^{2}u, Du, t) = c(t)Δu, where c(t) ≥ 0. If, in addition, F is assumed to be continuous in t, then up to a rescaling t′ = h(t), the function u(t, x) = T_{t}u_{0}(x) is a viscosity solution of the heat equation

\frac{∂u}{∂t}(t, x) = Δu(t, x), (t, x) ∈ [0, ∞) × ℝ^{N}.  (16.33)

Proof. Since the scale space is translation invariant, F does not depend on x (Proposition 17.13), and since the scale space commutes with the addition of constants, F does not depend on c (Proposition 17.10). Thus, F (A, p, x, c, t) = F (A, p, t). We know from Theorem 17.8 that T_{t+h,t}u(x)−u(x) = hF (D^{2}u(x), Du(x), t)+ o(h) for any u ∈ C^{2}(ℝ^{N}). Since T_{t+h,t} is linear, we have T_{t+h,t}(ru + sv) = rT_{t+h,t}u + sT_{t+h,t}v for any u, v ∈ C^{2}(ℝ^{N}) and r, s ∈ ℝ. This and Theorem 17.8 imply that

F (D^{2}(ru + sv), D(ru + sv), t) = rF (D^{2}u, Du, t) + sF (D^{2}v, Dv, t),

which means that F is linear in the argument u. (In what follows, we keep t fixed, and for convenience we write F (A, p) rather than F (A, p, t).) Since F satisfies property P, we can choose any values, including zero, for D^{2}u, D^{2}v, Du, and Dv. Thus we have

F (rA + sA′, rp + sp′) = rF (A, p) + sF (A′, p′)

262CHAPTER 16. SCALE SPACES AND PARTIAL DIFFERENTIAL EQUATIONS

Figure 16.4: The heat equation is not contrast invariant. This experiment shows that the linear scale space is not contrast invariant. First row: original image. Second row: Two different contrast changes have been applied to this image. Third row: A convolution by a Gaussian is applied to both images of the second row. Fourth row: The inverse contrast change is applied to the images of the third row. If the linear scale space were contrast invariant, these images should be equal. This is not the case, since the difference (displayed in the fifth row) is not null.

where A and A′ are arbitrary symmetric matrices and p and p′ are arbitrary vectors. From this, we see that F (A, p) = F (A, 0) + F (0, p). Now define F_{1} and F_{2} by F_{1}(p) = F (0, p) and F_{2}(A) = F (A, 0); F_{1} and F_{2} are clearly linear. Using the assumption that the operators are isotropic T_{t,s}, we see from Lemma 17.15 that

F_{1}(p) + F_{2}(A) = F_{1}(Rp) + F_{2}(RAR′),

where R is any linear isometry of ℝ^{N}. Taking A = 0, this implies that F_{1}(Rp) = F_{1}(p) for any linear isometry R. Since F_{1} is linear, this implies that F_{1} is a constant. Since by Proposition 17.10 F (0, 0, t) = 0, we conclude that F_{1}(p) = 0. This proves that F (A) = F (RAR′), where A is an arbitrary N × N symmetric matrix and R is an arbitrary linear isometry.

Given any symmetric matrix A, there is a linear isometry R such that RAR′ is diagonal whenever the coordinate system is orthogonal. Furthermore, any two diagonalizations differ only in the arrangement of the diagonal entries, which are the N eigenvalues of A, and any arrangement of these entries can be achieved by 16.6. COMMENTS AND REFERENCES 263

some linear isometry. This means that the value of F (A) depends only on some symmetric function f of the eigenvalues λ_{1}, λ_{2}, … λ_{N} of A. That means that F (A) = F (f(λ_{1}, λ_{2}, … λ_{N})), where f is a symmetric function of its arguments. Since F is also linear, we have

F (f(rλ_{1}, rλ_{2}, … rλ_{N})) = rF (f(λ_{1}, λ_{2}, … λ_{N})).

Since the only linear symmetric function of N variables is, up to a multiplicative constant, the linear function, we see that

F (A) = c trace(A)

for some constant c. Since F is nondecreasing in A (Lemma 17.7), c is nonnegative. We conclude that F (D^{2}u, Du) = cΔu. Remember that this argument has been made with a fixed t that was not written. Thus, our real conclusion is that F (D^{2}u, Du, t) = c(t)Δu, where c is a nonnegative function of t.

If we assume that F is continuous in t, then t ↦ c(t) is continuous. Then by Theorem 17.11, u(t, x) = T_{t}u_{0}(x) is a viscosity solution of

\frac{∂u}{∂t}(t, x) = c(t)Δu(t, x).

Finally, if we rescale using the function t ↦ t′ defined by ∂t′∂t(t) = c(t), we have the heat equation ∂u/∂t′ = Δu.

If u happens to be in C^{2}(ℝ^{+} × ℝ^{N}), then we know from Proposition 15.7 that u is a classical solution of the heat equation.

Exercise 16.3. Define the linear scale space {T_{t}} by T_{t}u_{0} = G_{t} ∗ u_{0}, where G is the Gaussian. Prove that {T_{t}} is scale invariant (Definition 17.18); in particular, show that for λ > 0, H_{λ}T_{t′} = T_{t}H_{λ} with t′ = tλ^{2}. Prove that the scale space is also isotropic in the sense that T_{t}R = RT_{t} for any linear isometry R of ℝ^{N}. Hint: Show that G_{t}(Rx) = G_{t}(x).

16.6 Comments and references

The presentation in this chapter and the next one follows essentially the work of L. Alvarez, F. Guichard, P.-L. Lions, and J.-M. Morel in [10], [8], and [9]. Their stated objective was to “... describe all multiscale causal, local, stable and shape preserving smoothing operators. This classification contains the classical ‘morphological’ operators, and some new ones.” This axiomatic approach is presented in several survey papers, with increasingly simple sets of axioms: Lions [177]; Alvarez and Morel [14]; Guichard, Lopez, and Morel [126]; and Guichard’s doctoral thesis [125], which was an early version of this book.

Linear scale space. Scale space theory was founded (in the linear framework) by Witkin [274], Marr [190], and Koenderink. An earlier development of linear scale space has been traced to Japan [268, 269]. Many works by Florack, ter Haar Romeny, Koenderink, and Viergever focus on the computation of partial derivatives of any order of an image and their use in image analysis [107, 108, 109, 111]. The concept of causality, used by all of these authors is crucial; it has been reinterpreted in this chapter as the combination of two requirements: a pyramidal structure and a comparison principle. De Giorgi founded 264CHAPTER 16. SCALE SPACES AND PARTIAL DIFFERENTIAL EQUATIONS

his mathematical theory of barriers for geometric motions on similar principles [120]. There are many axiomatic characterizations of linear scale space in terms of causality, invariants, and conservation properties. We mention particularly the early work by Babaud, Witkin, Baudin, and Duda [30] and Hummel [144]. A slight relaxation of the initial axioms led Pauwels and others to discover other possible linear scale spaces, which, however, are less local [218]. There have also been several attempts to define nonlinear scale spaces, which are understood as nonlinear invariant families of smoothing operators. In mathematical morphology, we mention work by Chen [67], Toet [258, 259], and Jackway [152]; Jackway emphasized the scale space properties of multiscale erosions and dilations. After the publication of [10] by Alvarez, Guichard, Lions, and Morel, several different axiomatic approaches have been proposed for nonlinear scale spaces. Weickert insists on grey level conservation, which excludes all of the mathematical morphology operators, and proposes a line of conservative parabolic nonlinear PDEs [265, 266]. The axiomatic presentation of Olver, Sapiro, and Tannenbaum [210] deduces the various scale spaces as invariant heat flows. See also [207]; the book

[255] contains miscellaneous contributions to geometric diffusion.

Extensions. Caselles, Coll, and Morel have questioned the very soundness of applying any of the proposed scale spaces to natural images [58]. They argue following the Kanisza psychophysical theory that occlusions generate T-junctions in images and that these T-junctions should be detected before any smoothing is applied. In [59], the same authors propose the set of level lines of the image, the so-called topographical map, as an alternative multiscale structure for describing images. Geraets and others propose a generalization of scale space to discrete point sets [117].

Contrast invariance. The Wertheimer principle, which states that human visual perception is independent of changes in illumination, was enunciated in 1923 [272]. Contrast invariance appears in mathematical morphology in the work of Serra [244]. Koenderink and van Doorn emphasized this requirement and introduced photometric invariants [166]. Florack and others studied contrast-invariant differential operators in [110]. Romeny and others construct third-order contrast-invariant operators to detect T-junctions [257]. See also [224]. The significance of contrast invariance for smoothing T-junctions is illustrated in Figure 16.6.

Rotation and scale invariance. One of the first discussions of rotationinvariant image operators was given by Beaudet in [39]. See also Lenz [174] for work on rotation-invariant operators. Scale-invariant shape representation is discussed by Baldwin, Geiger, and Hummel in [31]. Alvarez, Gousseau, and Morel use numerical experiments on natural images to confirm their scale invariance [6].

Affine invariance. Affine invariants are viewed as approximate projective invariants by Chang in [72]. The importance of affine invariance for threedimensional object recognition is discussed in [31] and [174]. Work by Forsyth, Munday, and Zisserman has been fundamental and has launched wide-ranging discussion of this theme [112, 201, 202]. Further contributions to the use and 16.6. COMMENTS AND REFERENCES 265 computation of affine and projective differential invariants in image processing can be found in [42], [233], [261], and [270].

266CHAPTER 16. SCALE SPACES AND PARTIAL DIFFERENTIAL EQUATIONS

Figure 16.5: Contrast invariance of the affine morphological scale space (AMSS). First row: original image. Second row: two contrast changes applied to the original. Third row : AMSS applied to both images of the second row, by a finite difference scheme. Fourth row: inverse contrast change applied to the filtered images. A visual check shows that they are almost identical. Bottom image: numerical check by taking the difference of the images in the fourth row. Compare this with the same experiment performed with the linear scale space, Figure 16.4.

16.6. COMMENTS AND REFERENCES 267

Figure 16.6: Same geometric figures, different evolutions under smoothing. 1st row: The four figures have T-junctions that differ in the way grey levels are distributed among the three regions. In the first two figures, the grey levels are monotone in, say, the clockwise direction. This means that they differ by a monotone contrast change. The same is true for the second two figures. However, the first and third figures differ by a nonmonotone contrast change. 2nd row: result of a smoothing by the AMSS model. We see that two different evolutions are possible: If the regions of the image keep the same order of grey levels, then the geometric evolution is identical. If, instead, a nonmonotone contrast change has been applied, the evolutions are geometrically different. 3rd row: result of a smoothing by the linear scale space. All four T-junctions give different evolutions. The evolution depends on the gray-level values of the three level sets, rather than depending only on their order. 4rd row: quantization of the 3rd row to display the shapes of some level lines.

268CHAPTER 16. SCALE SPACES AND PARTIAL DIFFERENTIAL EQUATIONS

Figure 16.7: Hyperdiscrimination of textures by nonlinear scale space. According to the Julesz theory of textons, human perception can discriminate different textures if their average behavior in terms of “texton” density is different. As shown in its mathematical formalization, proposed by C. Lopez, some of the texton densities can be interpreted as densities of the positive and negative parts of the image curvature at different scales. In this remarkable experiment, C. Lopez proved that one of the simplest contrast-invariant scale spaces beats by far the human discrimination performance. From left to right and top to bottom: 1-an original texture pair that is preattentively undiscriminable. The central square of the image consists of rotated “10’s” and the rest of the image of rotated “S’s.” These patterns are different, but have the same number of bars, angles, and so forth. 2-curvature motion applied to the original up to some scale 3-negative part of the curvature at the same scale 4-positive part of the curvature at the same scale 5-multichannel segmentation of the multi-image made of the curvatures 6-negative part of the curvature at scale 0. As seen in 2, 3, 4 and 5, this nonlinear scale space easily discriminates between the two textures.

Chapter 17

The Contrast-Invariant and Affine-Invariant Scale Spaces

This chapter is a direct continuation of Chapter 17. We are going to characterize, up to a multiplicative constant, all of the contrast-invariant scale spaces as curvature evolution equations. In the interest of simplicity and clarity, we will first prove the result in two dimensions. The computations are more intuitive in this case, and they are easily displayed in detail. The results are generalized to any dimension in section 18.2.

17.1 The two-dimensional case

We will show that if a scale space {T_{t}} is causal, isometric, and contrast invariant, then the associated PDE is of the form

\frac{∂u}{∂t} = |Du|G(curv(u), t).  (17.1)

This does not tell us much about G, so the question is, What additional assumptions must be made to have a more specific characterization of G? One answer is this: If we assume that {T_{t}} is affine invariant and that T_{t}(−u) = −T_{t}u (which we call reverse contrast invariant), then there is only one equation that satisfies all of these condition, namely, the so-called affine morphological scale space (AMSS),

\frac{∂u}{∂t} = |Du|(curv(u))^{1/3}.

We are led by Theorem 17.8 to study scale spaces defined by PDEs of the form

\frac{∂u}{∂t} = F (D^{2}u, Du, u, x, t), u(0) = u_{0},

where u_{0} is the original image, u(t, ·) is the image smoothed at scale t, and F is the function associated with {T_{t}}. In the two-dimensional case, A is a 2 × 2 symmetric matrix, p is a two-dimensional vector, c is a constant, x is a point in

269 270CHAPTER 17. THE CONTRAST-INVARIANT AND AFFINE-INVARIANT SCALE SPACES

the plane, and t ∈ ℝ^{+} is the scale. We will be using the following results from Chapter 17:

• If {T_{t}} is translation invariant, then F does not depend on x (Proposition 17.13).

• It {T_{t}} commutes with the addition of constants, then F does not depend on c (Proposition 17.10).

• If {T_{t}} is isotropic, then F (RAR′, Rp, t) = F (A, p, t) for every R ∈ O_{2} (Lemma 17.15).

• It {T_{t}} is contrast invariant, then F (μA + λ(p ⊗ p), μp, t) = μF (A, p, t), for any real numbers λ and μ, μ > 0, any 2 × 2 symmetric matrix A, and any two-dimensional vector p (Lemma 17.17). Recall that the tensor product p⊗p is just the symmetric matrix {p_{i}p_{j}}, i, j ∈ {1, 2}:

p ⊗ p = [\begin{matrix}p_{1}^{2} & p_{1}p_{2} \\ p_{1}p_{2} & p_{2}^{2} \end{matrix}] .

Relations F (RAR′, Rp, t) = F (A, p, t) and F (μA+λ(p⊗p), μp, t) = μF (A, p, t) will be used to show that F depends on two real functions ã_{12} and ã_{22} of A and p. These functions are defined by considering the rotation represented by the matrix

R_{p} = \frac{1}{|p|} [\begin{matrix}p_{1} & p_{2} \\ −p_{2} & p_{1} \end{matrix}] .

R_{p} has been chosen to map the unit vector p/|p| onto the unit vector e_{1} = (1, 0):

R_{p}p = |p|e_{1}.  (17.2)

The functions ã_{ij}, i, j ∈ {1, 2}, are defined by

\frac{1}{|p|}R_{p}AR′_{p} = [\begin{matrix}ã_{11} & ã_{12} \\ ã_{12} & ã_{22} \end{matrix}] .  (17.3)

A straightforward computation shows that

ã_{12} = \frac{1}{|p|^{3}} ((p_{1}^{2} − p_{2}^{2})a_{12} + p_{1}p_{2}(a_{22} − a_{11})) = \frac{A(p, p^{⊥})}{|p|^{3}},  (17.4)
ã_{22} = \frac{1}{|p|^{3}} (a_{11}p_{2}^{2} − 2a_{12}p_{1}p_{2} + a_{22}p_{1}^{2}) = \frac{A(p^{⊥}, p^{⊥})}{|p|^{3}}.  (17.5)

We should keep in mind that A represents D^{2}u and p represents Du so these results and the calculations that follow, while purely algebraic, have interpretations in differential geometry. In particular,

ã_{22}(D^{2}u, Du) = div(\frac{Du}{|Du|}) = curv(u),
ã_{12}(D^{2}u, Du) = div(\frac{Du^{⊥}}{|Du|} ) = anticurv(u).

17.1. THE TWO-DIMENSIONAL CASE 271

Exercise 17.1. This exercise is to give a geometric interpretation of anticurv(u). Show that anticurv(u)(x) is the curvature of the gradient line of u through the point x. The gradient lines are the curves that are tangent to the gradient of u at every point. They form a system of curves that are orthogonal to the level lines of u.

Lemma 17.1. If F satisfies the relations

F (RAR′, Rp, t) = F (A, p, t),  (17.6)
F (μA + λ(p ⊗ p), μp, t) = μF (A, p, t),  (17.7)

then there is a function G of three real variables such that, for p ≠ 0,

F (A, p, t) = |p|G(ã_{12}, ã_{22}, t).  (17.8)

Proof. We first use (17.7) with μ = 1/|p|. Thus,

F (A, p, t) = |p|F (\frac{A}{|p|} + λ(p ⊗ p), \frac{p}{|p|}, t),

where λ is any real number. Next, we apply (17.6) with R = R_{p}:

F (A, p, t) = |p|F (R_{p}(\frac{A}{|p|} + λ(p ⊗ p))R′_{p}, R_{p} \frac{p}{|p|}, t)

= |p|F (R_{p}(\frac{A}{|p|})R′_{p} + λR_{p}(p ⊗ p)R′_{p}, e_{1}, t).

It is easily checked that

R_{p}(p ⊗ p)R′_{p} = [\begin{matrix}|p|^{2} & 0 \\ 0 & 0 \end{matrix}] .

Thus, by (17.3),

F (A, p, t) = |p|F ( [\begin{matrix}ã_{11} + λ|p|^{2} & ã_{12} \\ ã_{12} & ã_{22} \end{matrix}] , e_{1}, t).

Since λ is arbitrary and since p ≠ 0, F depends only on ã_{12} and ã_{22}. To finish, we can define G by

G(ã_{12}, ã_{22}, t) = F ( [\begin{matrix}0 & ã_{12} \\ ã_{12} & ã_{22} \end{matrix}] , e_{1}, t).

Lemma 17.2. The function G depends only on ã_{22} and t.

Proof. We will use the fact established in Lemma 17.7 that F is nondecreasing with respect to its first argument and the assumption in Definition 17.5 that F is continuous in this argument. We will also use the result of Lemma 18.1.

Consider two symmetric matrices A and B such that A ≥ B. In analogy with (17.3), we write

\frac{1}{|p|}R_{P} BR′_{p} = [\begin{matrix}b̃_{11} & b̃_{12} \\ b̃_{12} & b̃_{22} \end{matrix}] .  (17.9)

272CHAPTER 17. THE CONTRAST-INVARIANT AND AFFINE-INVARIANT SCALE SPACES

Then

\frac{1}{|p|}R_{p}(A − B)R′_{p} = [\begin{matrix}ã_{11} − b̃_{11} & ã_{12} − b̃_{12} \\ ã_{12} − b̃_{12} & ã_{22} − b̃_{22} \end{matrix}] .  (17.10)

By assumption, the symmetric matrix A − B is such that A − B ≥ 0. This property is invariant under rotation, in particular, under the mapping A − B ↦ R_{p}(A − B)R′_{p}. Hence, A − B ≥ 0 if and only if R_{p}(A − B)R′_{p} ≥ 0. This means that A ≥ B if and only if

(ã_{11} − b̃_{11})x^{2} + 2(ã_{12} − b̃_{12})xy + (ã_{22} − b̃_{22})y^{2} ≥ 0  (17.11)

for all real numbers x and y, and this is true if and only if

(ã_{11} − b̃_{11})(ã_{22} − b̃_{22}) ≥ (ã_{12} − b̃_{12})^{2}.  (17.12)

Fix A and choose an arbitrary b̃_{12}, b̃_{12} ≠ ã_{12}. Choose any real number b̃_{22} such that ã_{22} −b̃_{22} = ε > 0. Now select b̃_{11} = b̃_{11}(ε) so that (17.12) is satisfied. Then we have

F ( [\begin{matrix}ã_{11} & ã_{12} \\ ã_{12} & ã_{22} \end{matrix}] , e_{1}, t) ≥ F ( [\begin{matrix}b̃_{11}(ε) & b̃_{12} \\ b̃_{12} & ã_{22} − ε \end{matrix}] , e_{1}, t).

The value on the right-hand side of the inequality is independent of b̃_{11}(ε) (Lemma 18.1). Now let ε tend to zero. By the continuity of F in its first argument, we conclude that

F ( [\begin{matrix}ã_{11} & ã_{12} \\ ã_{12} & ã_{22} \end{matrix}] , e_{1}, t) ≥ F ( [\begin{matrix}b̃_{11}(0) & b̃_{12} \\ b̃_{12} & ã_{22} \end{matrix}] , e_{1}, t),

where b̃_{11}(ε) → −∞ as ε → 0. This shows that

G(ã_{12}, ã_{22}, t) ≥ G(b̃_{12}, ã_{22}, t).

A similar argument shows that

G(b̃_{12}, ã_{22}, t) ≥ G(ã_{12}, ã_{22}, t).

We conclude that G(ã_{12}, ã_{22}, t) does not depend on ã_{12}, and hence that G is a function of only ã_{22} and t.

We summarize these last results in the following theorem.

Theorem 17.3. If the two-dimensional scale space {T_{t}} is causal, isometric, and contrast invariant, then its associated PDE has the form

\frac{∂u}{∂t} = |Du|G(curv(u), t),  (17.13)

where G is continuous and nondecreasing in its first variable.

We are now going to introduce scale and affine invariance. We are still working in two dimensions.

17.1. THE TWO-DIMENSIONAL CASE 273

Theorem 17.4. Assume that the scale space {T_{t}} is causal, isometric, and contrast invariant. In addition, assume that it is scale invariant and that it is normalized according to Lemma 17.20. Then its associated PDE has the form

\frac{∂u}{∂t} = |Du|β(tcurv(u)),  (17.14)

where β is continuous and nondecreasing.

If the scale space {T_{t}} is affine invariant and normalized according to Lemma 17.20, then the associated PDE has the form

\frac{∂u}{∂t} = |Du|β(tcurv(u)),  (17.15)

where β(s) = Cs^{1/3} if s > 0 and β(s) = −D|s|^{1/3} if s < 0, for two nonnegative constants C and D. Conversely, this equation defines an affine-invariant scale space.

Proof. By Lemma 17.21, if a causal scale space is affine invariant, then, after appropriate renormalization (Lemma 17.20), its associated function satisfies

F (BAB′, Bp, t) = |detB|^{1/2}F (A, p, |detB|^{1/2}t)  (17.16)

for any linear map B. If we let B = cI, c > 0, then F (c^{2}A, cp, t) = cF (A, p, ct). Since F (A, p, t) = |p|G(ã_{22}(A, p), t), this implies that

c|p|G(ã_{22}(c^{2}A, cp), t) = c|p|G(ã_{22}(A, p), ct),

and since

ã_{22}(c^{2}A, cp) = \frac{c^{2}A(cp^{⊥}, cp^{⊥})}{|cp|^{3}} = cã_{22}(A, p),

we see that

G(ã_{22}(A, p), ct) = G(cã_{22}(A, p), t).

Since this equation is true for all A, p ≠ 0, c > 0, and t > 0, G(cs, t) = G(s, ct) for any s and any positive c and t. This implies that

G(s, t) = G(st, 1) = β(st),

where β is continuous and nondecreasing. This proves the first part of the theorem.

We now assume that the scale space is affine invariant. To identify the power 1/3, we need to exploit the affine invariance only in the particular case of “stretching and shrinking” along the x and y axes, that is, by using the transformation represented by

B(λ) = [\begin{matrix}λ & 0 \\ 0 & λ^{−1} \end{matrix}] .

The reason is that any linear transformation of the plane is a product of isometries, magnifications, and “stretching and shrinking,” and we have already used isometric invariance and scale invariance. First note that

BAB′ = [\begin{matrix}λ^{2}a_{11} & a_{12} \\ a_{12} & λ^{−2}a_{22} \end{matrix}] and Bp = (λp_{1}, p_{2}/λ).

274CHAPTER 17. THE CONTRAST-INVARIANT AND AFFINE-INVARIANT SCALE SPACES

Then we see from (17.5) that

ã_{22}(BAB′, Bp) = \frac{a_{11}p_{2}^{2} − 2a_{12}p_{1}p_{2} + a_{22}p_{1}^{2}}{(λ^{2}p_{1}^{2} + λ^{−2}p_{2}^{2})^{3/2}} .  (17.17)

We know that F (A, p, t) = |p|β(tã_{22}(A, p, t)). This and the affine-invariance relation (17.16) show that

|Bp|β(tã_{22}(BAB′, Bp)) = |p|β(tã_{22}(A, p)).  (17.18)

If we let p_{1} = 1, p_{2} = 0, and a_{22} = 1, then from (17.17) and (17.18) it follows that

|λ|β(s/λ^{3}) = β(s).

The first thing to notice is that β(0) = 0, which is consistent with (and a consequence of) the fact that F (0, 0, t) = 0. On the other hand, nothing we have assumed precludes β(s) = 0 for all s ≥ 0. However, in case β(a) > 0 for some a > 0, we can select λ > 0 so λ^{3} = s/a, and we have

β(s) = a^{−1/3}β(a)s^{1/3},

in which case C = a^{−1/3}β(a) = β(1). A similar argument shows that either β(s) = 0 for all s ≤ 0, or

β(s) = |b|^{−1/3}β(b)|s|^{1/3},

where D = −|b|^{−1/3}β(b) = −β(−1). The result is thus true in all cases.

In general, β(1) ≠ −β(−1), that is, C ≠ D. For example, if D = 0 and C > 0, then we have a pure affine erosion (shapes shrink); if C = 0, then we have a pure affine dilation (shapes expand). However, if the scale space is also reverse contrast invariant, which means that T_{t+h,t}∘g = g∘T_{t+h,t} for continuous nonincreasing g, then D = C. To see this, recall the proof of Lemma 17.17. If the scale space {T_{t}} is reverse contrast invariant, then equation (17.16) is true for negative (as well as positive) μ, and we have F (−A, −p, t) = −F (A, p, t). This then implies that β(−1) = −β(1).

17.2 Contrast-invariant scale space equations in N dimensions

We are going to extend the results of Theorem 18.4 to N dimensions, so we make the same assumptions about the scale space {T_{t}}: It is causal, it commutes with the addition of constants, and it is translation invariant. Our immediate aim is to deduce the general form of F in N dimensions from the assumption that {T_{t}} is contrast invariant. If we also assume that F is isometric, then we will show that, for p ≠ 0,

F (A, p, t) = G(λ_{1}, λ_{1}, …, λ_{N−1}, |p|, t),

where G is a real-valued function defined on ℝ^{N−1} × (0, +∞) × [0, +∞). G is symmetric in the N − 1 variables λ_{1}, λ_{2}, …, λ_{N−1}, is positively homogeneous with respect to λ_{1}, λ_{2}, …, λ_{N−1}, |p|, and is nondecreasing with respect to each 17.2. CONTRAST-INVARIANT SCALE SPACE EQUATIONS IN N DIMENSIONS275

λ_{i}, 1 ≤ i ≤ N − 1 for all t ∈ [0, +∞). (In the interest of notation, we will suppress t in the following discussion.)

We know from Lemma 17.17 that the function F associated with a contrastinvariant scale space {T_{t}} satisfies the relation

F (μA + λ(p ⊗ p), μp) = μF (A, p)  (17.19)

for all λ ∈ ℝ, all μ ≥ 0, any N × N symmetric matrix A ∈ S^{N}, and any vector p ∈ ℝ^{N}. By taking λ = 0, this shows that F is positively homogeneous in (A, p):

F (μA, μp) = μF (A, p).  (17.20)

In particular, F (0, 0) = 0, which we also know from Proposition 17.10.

If we take μ = 1, then (17.19) becomes F (A + λ(p ⊗ p), p) = F (A, p). If N = 1, this means that F depends only on p ∈ ℝ, and we conclude that

F (A, p) = \begin{cases}F (1)p if p ≥ 0, \\ −F (−1)p if p ≤ 0. \end{cases}  (17.21)

More interesting situation occur if N ≥ 2, as we have already seen in case N = 2. From now on we assume that N ≥ 2.

We need to introduce some notation. For p ∈ ℝ^{N}, p ≠ 0, consider the linear operator defined by the matrix Q_{p} = I_{N} −(1/|p|^{2})(p⊗p). It is easy to verify that Q_{p} is the projection of ℝ^{N} onto the hyperplane p^{⊥} (also denoted by (ℝp)^{⊥}). Indeed, If q = λp, then Q_{p}q = 0; if p · q = 0, then Q_{p}q = q. Let A be an N × N symmetric matrix and consider the matrix Q_{p}AQ_{p}. Since Q_{p} is symmetric, it is clear that Q_{p}AQ_{p} is also symmetric. It is also clear that q ∈ (ℝp)^{⊥} implies that Q_{p}AQ_{p}q ∈ (ℝp)^{⊥} and that q ∈ ℝp implies that Q_{p}AQ_{p}q = 0. Since Q_{p}AQ_{p} is symmetric, it has N real eigenvalues, one of which we have just seen to be zero. Let κ_{1}, κ_{2} …, κ_{N−1} denote the N − 1 other eigenvalues. These are the eigenvalues of Q_{p}AQ_{p} restricted to (ℝp)^{⊥}. If A = D^{2}u and p = Du, and if we define λ_{i} = κ_{i}/|p|, 1 ≤ i ≤ N − 1, then the λ_{i} are that principle curvatures of the level hypersurface of u (Definition 4.19). If N = 2, then by Definition 4.14, λ_{1} = (1/|p|)trace(Q_{p}AQ_{p}) = curv(u).

Theorem 17.5 (Giga, Goto []). Let {T_{t}} be a contrast-invariant scale space and assume that N ≥ 2. Then the associated function F satisfies the following relation: For all A ∈ S^{N} and all p ∈ ℝ^{N}, p ≠ 0,

F (A, p, t) = F (Q_{p}AQ_{p}, p, t).  (17.22)

Proof. We begin by fixing p ∈ ℝ^{N}, p ≠ 0 and selecting an orthogonal coordinate system such that p = |p|(0, …, 0, 1). Then p ⊗ p = |p|^{2}(δ_{Ni}δ_{Nj}), 1 ≤ i, j ≤ N. If we write B = A + λ(p ⊗ p), then b_{ij} = a_{ij} + λ|p|^{2}(δ_{Ni}δ_{Nj}). This means that b_{ij} = a_{ij} except for i = N and j = N, in which case b_{NN} = a_{NN} +λ|p|^{2}. Since F (A+λ(p⊗p), p) = F (A, p), this means that F (A, p) does not depend on a_{NN}. We use this fact, combined with the assumption that F is nondecreasing in its first variable, to complete the proof.

Note that, with the coordinate system we have chosen, Q_{p}AQ_{p} is just the matrix A with the last column and last row replaced with zeros: If C = Q_{p}AQ_{p}, then c_{ij} = a_{ij} for 1 ≤ i, j ≤ N − 1 and c_{ij} = 0 if i = N or if j = N.

276CHAPTER 17. THE CONTRAST-INVARIANT AND AFFINE-INVARIANT SCALE SPACES

Now let M = a_{N,1}^{2} + ⋯ + a_{N,N−1}^{2} and consider I_{ε} = εI + (M/ε − ε)(δ_{Ni}δ_{Nj}), 1 ≤ i, j ≤ N. Thus, I_{ε} is an N × N diagonal matrix D, where d_{ii} = ε for 1 ≤ i ≤ N − 1 and d_{NN} = M/ε. One can easily verify that Q_{p}AQ_{p} ≤ A + I_{ε} and that A ≤ Q_{p}AQ_{p} + I_{ε} for small ε > 0. Then we have

F (A, p) ≤ F (Q_{p}AQ_{p} + I_{ε}, p) ≤ F (A + 2I_{ε}, p).

As we let ε → 0, the entries in the matrix A + 2I_{ε} tend to a_{ij} except the entry a_{NN} + 2M/ε, which tends to +∞. But F is independent of the value of NN-entry, and F is continuous in its first variable. Thus,

F (A + 2I_{ε}, p) → F (A, p) and F (Q_{p}AQ_{p} + I_{ε}, p) → F (Q_{p}AQ_{p}, p).

Corollary 17.6. Let {T_{t}} be a contrast-invariant scale space and assume that N ≥ 2. If {T_{t}} is also isometric, then

F (A, p, t) = |p|G(λ_{1}, …, λ_{N−1}, t)  (17.23)

for all A ∈ S^{N}, p ∈ ℝ, p ≠ 0, where G is a continuous function on ℝ^{N−1} that is symmetric in the N −1 variables λ_{1}, …, λ_{N−1} and is nondecreasing with respect to each λ_{i}, 1 ≤ i ≤ N − 1.

Proof. By Lemma 17.15, the function F associated with the scale space satisfies

F (RAR′, Rp) = F (A, p)  (17.24)

for all A ∈ S^{N}, p ∈ ℝ^{N}, p ≠ 0, and R ∈ O_{N}. (Recall that O_{N} denotes the group of linear isometries of ℝ^{N} (section 17.4.1).)

Fix p ≠ 0 and let R be any element of the subgroup O_{N}^{p} of O_{N} that leaves p fixed, that is, Rp = p. Since Q_{p}AQ_{p} ∈ S^{N}, we know from (17.24) that

F (RQ_{p}AQ_{p}R′, Rp) = F (RQ_{p}AQ_{p}R′, p) = F (Q_{p}AQ_{p}, p).

By Theorem 18.5, F (Q_{p}AQ_{p}, p) = F (A, p). These two relations tell us that

F (RQ_{p}AQ_{p}R′, p) = F (A, p)

for all R ∈ O_{N}^{p}. This means that the value of F (A, p) depends only on p and the eigenvalues of Q_{p}AQ_{p}. Indeed, using the same coordinate system used in the proof of Theorem 18.5 based on p = |p|(0, …, 1), there is always an R ∈ O_{N}^{p} so that RQ_{p}AQ_{p}R′ is diagonal with entries κ_{1}, κ_{2}, …, κ_{N−1}, 0, where the κ_{i} are the eigenvalues of Q_{p}AQ_{p} restricted to (ℝp)^{⊥}. Furthermore, we can choose R so that the κ_{i} appear in any order. Thus, there is a function G_{1} such that

F (A, p) = G_{1}(κ_{1}, κ_{2}, …, κ_{N−1}, p);

G_{1} is continuous and symmetric in the κ_{i}, and G_{1} is nondecreasing in each κ_{i}. (These last statements follow from the fact that F is continuous and nondecreasing in its first variable.)

Now take R ∈ O_{N} and let q = Rp (p is still fixed). Then |p| = |q|. Furthermore, given any q ∈ ℝ^{N} such that |p| = |q|, there is an R ∈ O_{N} such that Rp = q. It is easy to verify that R(p⊗p) = q ⊗p, and since (q ⊗p)′ = p⊗q, that 17.3. AFFINE-INVARIANT SCALE SPACES FOR N ≥ 2 277

(p ⊗ p)R′ = p ⊗ q. Thus, R(p ⊗ p)R′ = q ⊗ q, which implies that RQ_{p} = Q_{q}R. Consequently,

Q_{q}RAR′Q_{q} = RQ_{p}AQ_{p}R′,

and this means that Q_{q}RAR′Q_{q} and Q_{p}AQ_{p} have the same eigenvalues. Using (17.24) again, we see that F (RQ_{p}AQ_{p}R′, Rp) = F (Q_{p}AQ_{p}, p), and since Rp = q,

F (RQ_{p}AQ_{p}R′, q) = F (Q_{p}AQ_{p}, p).

Since Q_{q}RAR′Q_{q} and Q_{p}AQ_{p} have the same eigenvalues, this implies that

F (A, p) = G_{1}(κ_{1}, κ_{2}, …, κ_{N−1}, p) = G_{1}(κ_{1}, κ_{2}, …, κ_{N−1}, q)

whenever |p| = |q|. This means that F depends only on the modulus of p, and therefor we can write F (A, p) = G_{1}(κ_{1}, κ_{2}, …, κ_{N−1}, |p|). Since F is homogeneous,

G_{1}(μκ_{1}, μκ_{2}, …, μκ_{N−1}, μ|p|) = μG_{1}(κ_{1}, κ_{2}, …, κ_{N−1}, |p|)

for μ ≥ 0. If we take μ = |p|^{−1}, then

F (A, p) = |p|G_{1}(κ_{1}/|p|, κ_{2}/|p|, …, κ_{N−1}/|p|, 1).

Defining G by G(λ_{1}, λ_{2}, …, λ_{N−1}) = G_{1}(κ_{1}/|p|, κ_{2}/|p|, …, κ_{N−1}/|p|, 1), where the λ_{i} = κ_{i}/|p|, completes the proof.

17.3 Affine-invariant scale spaces for N ≥ 2

There is a function H_{N} in the following theorem that is defined on the set of N integers {−N + 1 + 2k | 0 ≤ k ≤ N − 1}. We will see in the proof of the theorem that H_{N} is nondecreasing and that it vanishes except at the points −(N − 1) and N − 1. It will also be shown that H_{N}(N − 1) ≥ 0. There is not enough information to determine the value of H_{N}(N − 1); however, to avoid the trivial case F ≡ 0, we assume that H_{N}(N − 1) > 0.

Theorem 17.7. Assume that the scale space {T_{t}} is contrast invariant and affine invariant. Assume also that T_{t}(−u) = −T_{t}(u) and that {T_{t}} has been normalized in accordance with Lemma 17.20. Then the PDE associated with {T_{t}} is

\frac{∂u}{∂t} = |Du|t^{\frac{N−1}{N+1}} ∏_{i=1}^{N−1} |λ_{i}|^{\frac{1}{N+1}} H_{N}( ∑_{i=1}^{N−1} sgn(λ_{i})),  (17.25)

where the λ_{i} are the principal curvatures of the level hypersurface of u, sgn(λ_{i}) denotes the sign of λ_{i}, and H_{N} is such that H_{N}(N − 1) > 0, H_{N}(N − 1) = −H_{N}(−(N − 1)), and H(n) = 0 for all −(N − 1) < n < N − 1. In other words, H_{N} is equal to zero if all the λ_{i} do not have the same sign.

Proof. We begin with the result of Corollary 18.6: Since {T_{t}} is contrast invariant and isometric, its associated function F is of the form

F (A, p, t) = |p|G(λ_{1}, λ_{2}, …, λ_{N−1}, t),

278CHAPTER 17. THE CONTRAST-INVARIANT AND AFFINE-INVARIANT SCALE SPACES

and G is symmetric with respect to the λ_{i}. If p ≠ 0, then λ_{i} = κ_{i}/|p|, where the κ_{i}, 1 ≤ i ≤ N − 1 are the eigenvalues of A restricted to the hyperplane p^{⊥} orthogonal to p. To simplify the proof, we prefer to use the more general form of F that appeared in the proof of Corollary 18.6, namely,

F (A, p, t) = G_{1}(κ_{1}, κ_{2}, …, κ_{N−1}, |p|, t).  (17.26)

Since the restriction of A to the hyperplane p^{⊥} is represented by a symmetric matrix, we can choose orthonormal vectors e_{1}, …, e_{N−1} such that κ_{i} = A(e_{i}, e_{i}), 1 ≤ i ≤ N − 1. Each vector e_{i} is orthogonal to p, so we obtain an orthonormal basis for ℝ^{N} by including the vector e_{N} = p/|p|. We now define some special linear affine transformations of ℝ^{N}: Let B_{i}, for 1 ≤ i ≤ N − 1, be the linear transform defined by

B_{i}(e_{1}, …, e_{i}, …, e_{N}) = (e_{1}, …, βe_{i}, …, β^{−1}e_{N}),

where β ∈ ℝ, β ≠ 0. Clearly, |B_{i}| = 1. We are now going to apply the result of Lemma 17.21 with B = B_{1}. This says that F (A, p, t) = F (B_{1}AB′_{1}, B_{1}p, t), and in view of the representation (17.26), this means that

F (A, p, t) = G_{1}(β^{2}κ_{1}, κ_{2}, …, κ_{N−1}, β^{−1}|p|, t).

Assume for the moment that κ_{1} ≠ 0 and take β = |κ_{1}|^{−1/2}. Then

F (A, p, t) = G_{1}(sgn(κ_{1}), κ_{2}, …, κ_{N−1}, |κ_{1}|^{1/2}|p|, t).

Repeat this argument with B = B_{i} for i = 2 to i = N − 1. The result is that

F (A, p, t) = G_{1}(sgn(κ_{1}), …, sgn(κ_{N−1}), |p| ∏_{i=1}^{N−1} |κ_{i}|^{1/2}, t),  (17.27)

assuming that κ_{i} ≠ 0, 1 ≤ i ≤ N − 1. We return again to Lemma 17.21, which is based on the results of Lemma 17.20, and note that

F (μ^{2}A, μp, t) = μF (A, p, μt)

for μ ≥ 0 implies that

G_{1}(sgn(κ_{1}), …, sgn(κ_{N−1}),μ|p|μ^{N−1} ∏_{i=1}^{N−1} |κ_{i}|^{1/2}, t)

= μG_{1}(sgn(κ_{1}), …, sgn(κ_{N−1}), |p| ∏_{i=1}^{N−1} |κ_{i}|^{1/2}, μt).

We deduce that

μ^{−1}G_{1}(sgn(κ_{1}), …, sgn(κ_{N−1}),μ^{N}|p| ∏_{i=1}^{N−1} |κ_{i}|^{1/2}, μ^{−1}t)

= G_{1}(sgn(κ_{1}), …, sgn(κ_{N−1}), |p| ∏_{i=1}^{N−1} |κ_{i}|^{1/2}, t).

17.3. AFFINE-INVARIANT SCALE SPACES FOR N ≥ 2 279

By taking μ = t, we see that

F (A, p, t) = t^{−1}G_{1}(sgn(κ_{1}), …, sgn(κ_{N−1}), t^{N}|p| ∏_{i=1}^{N−1} |κ_{i}|^{1/2}, 1),

and we write

F (A, p, t) = t^{−1}G_{2}(t^{N}|p| ∏_{i=1}^{N−1} |κ_{i}|^{1/2}, sgn(κ_{1}), …, sgn(κ_{N−1})).

At this point we use the fact that {T_{t}} is contrast invariant, and so we have

F (α^{−1}A, α^{−1}p, t) = α^{−1}F (A, p, t)

for all α > 0. Thus,

tF (A, p, t) = αG_{2}(α^{−1}t^{N}|p| ∏_{i=1}^{N−1} |α^{−1}κ_{i}|^{1/2}, sgn(κ_{1}), …, sgn(κ_{N−1})).

By taking α = (t^{N}|p| ∏_{i=1}^{N−1} |κ_{i}|^{1/2})^{2/(N−1)}, the function F is reduced to the following form:

F (A, p, t) = t^{\frac{N−1}{N+1}} |p|^{\frac{2}{N+1}} ∏_{i=1}^{N−1} |κ_{i}|^{\frac{1}{N+1}} G_{2}(1, sgn(κ_{1}), …, sgn(κ_{N−1})

= t^{\frac{N−1}{N+1}} |p|^{\frac{2}{N+1}} ∏_{i=1}^{N−1} |κ_{i}|^{\frac{1}{N+1}} H_{1}(sgn(κ_{1}), …, sgn(κ_{N−1}).

Since λ_{i} = κ_{i}/|p|, we finally have

F (A, p, t) = |p|t^{\frac{N−1}{N+1}} ∏_{i=1}^{N−1} |λ_{i}|^{\frac{1}{N+1}} H_{1}(sgn(λ_{1}), …, sgn(λ_{N−1}).  (17.28)

The derivation of this representation of F was based on the assumption that none of the eigenvalues κ_{i} were zero. If there were κ_{i} = 0, we could have perturbed A by replacing these eigenvalues with ε > 0, done the derivation with positive eigenvalues, and then let ε tend to zero in (17.28).

We must now deal with the function H_{1}, or more precisely, the functions H_{1}, for there is a different function for each value of N ≥ 2. First note that H_{1} must be symmetric in its N − 1 variables: F is invariant under any rotation that leaves p fixed, and we can always find an element of O_{N}^{p} that produces any given permutation of the κ_{i}, and thus of the λ_{i}. If some λ_{i} happens to be zero, we can set H_{1}(sgn(λ_{1}), …, sgn(λ_{N−1})) = 0. Another way to do this is to define sgn(0) = 0 and say that H_{1}(sgn(λ_{1}), …, sgn(λ_{N−1})) = 0 if any of its N − 1 variables is zero. Thus the only interesting situation is in case all of the λ_{i} are nonzero. Then by the invariance of H_{1} under elements of O_{N}^{p}, it is clear that H_{1} depends only on the number variables equal to one and the number of variables equal to minus one. In other words, H_{1} is function of ∑_{i=1}^{N−1} sgn(λ_{i}):

H_{1}(sgn(λ_{1}), …, sgn(λ_{N−1})) = H_{N}( ∑_{i=1}^{N−1} sgn(λ_{i})),

280CHAPTER 17. THE CONTRAST-INVARIANT AND AFFINE-INVARIANT SCALE SPACES

where we have written H_{N} to stress the fact that the functions depend on the dimension N. H_{N} is defined on the N integers {−N + 1 + 2k | 0 ≤ k ≤ N − 1}, but we no information about its range. We do, however, have one last invariant to call on, and that is the assumption that T_{i}(u) = −T_{i}(−u). This translates into the relation F (−A, −p, t) = −F (A, p, t), which in turn implies that H_{N}(n) = −H_{N}(−n). Finally, we know that H_{N} must be nondecreasing based on the fact that F is nondecreasing in its first variable, A. In summary,

F (A, p, t) = |p|t^{\frac{N−1}{N+1}} ∏_{i=1}^{N−1} |λ_{i}|^{\frac{1}{N+1}} H_{N}( ∑_{i=1}^{N−1} sgn(λ_{i})),  (17.29)

where H_{N} is a nondecreasing function defined on the set {−N + 1 + 2k | 0 ≤ k ≤ N −1} and H_{N}(n) = −H_{N}(−n). We are now going to consider three cases.

The case N= 2.

Here we have N − 1 = 1, and (17.29) reads

F (A, p, t) = t^{1/3}|p||λ_{1}|^{1/3} H_{2}(sgn(λ_{1})).

The value of H_{2}(1) is not determined, although it must be positive to avoid the trivial case F ≡ 0 and to ensure that H_{2}(−1) = −H_{2}(1) and that H_{2} is nondecreasing. Thus, up to a positive (multiplicative) constant F (A, p, t) = t^{1/3}|p|sgn(λ_{1})|λ_{1}|^{1/3}, which we have been writing as F (A, p, t) = t^{1/3}|p|(λ_{1})^{1/3}, with the convention that r^{1/3} = (r/|r|)|r|^{1/3}. This can also be written as

F (A, p, t) = t^{1/3}|p|(\frac{1}{|p|}A(\frac{p^{⊥}}{|p|} , \frac{p^{⊥}}{|p|} ))^{1/3}.

If A = D^{2}u and p = Du, then the associated scale space equation if

\frac{∂u}{∂t} = t^{1/3}|Du|(curv(u))^{1/3}.

The case N= 3.

The PDE we obtain in this case is

\frac{∂u}{∂t} = t^{1/2}|Du||λ_{1}λ_{2}|^{1/4}H_{3}(sgn(λ_{1}) + sgn(λ_{2})),

where λ_{1} and λ_{2} are the principal curvatures of the level surface of u. Their product is the Gaussian curvature of the level surface of u, and to highlight this we write the equation as

\frac{∂u}{∂t} = t^{1/2}|Du||G(u)|^{1/4}H_{3}(sgn(λ_{1}) + sgn(λ_{2})).  (17.30)

Since sgn(λ_{1}) + sgn(λ_{2}) takes only the values −2, 0, 2, and since H_{3}(−2) = −H_{3}(−2), we are concerned with only two parameters: H_{3}(2) = b and H_{3}(0) = a. We know that H_{3}(−2) ≤ H_{3}(0) ≤ H_{3}(2), or −b ≤ a ≤ b. Hence, b ≥ 0 and |a| ≤ b. We are now going to show that a = 0 by using the fact that F is increasing with respect to its first variable. We do this by choosing special 17.4. COMMENTS AND REFERENCES 281

values for the pair (λ_{1},λ_{2}). For example, take (−1, α) and (α, α), α > 0. The value of F for the first pair is less than or equal to the value of F for the second:

αH_{3}(0) ≤ α^{2}H_{3}(2).

Letting α tend to zero shows that H_{3}(0) ≤ 0. Making a similar argument with the pairs (−α, α) and (α, α), shows that H_{3}(0) ≥ 0. Thus H_{3}(0) = 0, which means that the left-hand side of (17.30) is zero if the two principal curvatures have opposite signs. Consequently, up to a positive multiplicative constant, equation (17.30) is

\frac{∂u}{∂t} = sgn(λ_{1})t^{1/2}|Du||G(u)^{+}|^{1/4},  (17.31)

where x^{+} stands for sup{0, x}. This equation describes the unique multiscale analysis in three dimensions that is both affine invariant and contrast invariant and satisfies the condition T_{t}(u) = −T_{t}(−u).

The case N> 3.

The only remaining task is to prove that H_{N} has the properties stated in the theorem. This is done by using arguments similar to those use for the threedimensional case: By taking particular values for the λ_{i} and by using the fact that F is nondecreasing, one shows that H_{N}(N − 3) = 0. Then since H_{N} is nondecreasing and since H_{N}(n) = −H_{N}(−n), it follows that H_{N}(n) = 0 except for n = N − 1 and n = −(N − 1). The details are left as an exercise.

Exercise 17.2. Fill in the details for the last part of the proof.

17.4 Comments and references

Axiomatics. In this chapter, we have followed an axiomatic presentation of scale spaces developed in [10], which is a simplified version of the original given in [125]. Other axioms for affine scale space have been proposed by Olver, Sapiro, and Tannenbaum [211, 212].

Curvature motion. The first complete mathematical study on the motion of a surface by its mean curvature is the book by Brakke [45]. The main question in this field is, How regular is a surface that has been smoothed by mean curvature? Huisken proved that a convex surface smoothed by mean curvature was transformed into a sphere before vanishing to a point [141]. This result generalizes to higher dimensions a result by Gage about curve evolution [115]. The question of whether or not a surface smoothed by curvature motion ended in a sphere was first introduced by Firey for Gaussian curvature motion [104]. Osher and Sethian developed the first numerical codes for mean curvature motion, where topological changes of the surface could be dealt with efficiently [214]. Yuille observed that the Koenderink–van Doorn dynamic shape algorithm could create singularities is a dumb-bell shaped surface [277]: The “handle” part ultimately evolved into a thin filament that broke, creating singularities. (See the figures in Chapter 2, particularly Figure 2.9.) This behavior contradicts causality, one of the main axioms of scales space: Smoothing a shape should not create new features. Koenderink comments on this creation of singularities in [165]. The 282CHAPTER 17. THE CONTRAST-INVARIANT AND AFFINE-INVARIANT SCALE SPACES

corresponding mathematical study of this phenomenon is due to Grayson [124]. Bounds on the gradient for the mean curvature equation are given by Barles in [36]. A general survey of singularity formation by mean curvature motion is given by Angenent in [23]. Altschuler, Angenent, and Giga prove the smoothness of the evolution of rotationally symmetric hypersurfaces and estimate the number of singular points [4]. More about regularity and singularities related to mean curvature flow can be found in [92, 25, 142, 149]. Ishii and Souganidis developed a theory of viscosity solutions for general curvature equations, including any power function of the curvature or the Gaussian curvature [151]. Particular mention must be made about the work by Caselles and Sbert on the properties of scale spaces in three dimensions [65]. They prove that the dumb-bell is not “pinched off” by the affine scale space, but they exhibit examples of other surfaces where singularities may appear. Chow proved that the affine scale space deforms strictly convex surfaces into spheres [73, 74]. This result is equivalent to the result by Huisken for mean curvature motion mentioned above.

Extensions of affine scale space in two dimensions. Several authors have attempted to extend the affine scale space in two dimensions to a projectiveinvariant scale space. The results of this chapter have clearly shown that a multiscale analysis that is both local and causal cannot be projective invariant. An affine-invariant scale space must have the form given in equation (17.25), which is completely determined up to a multiplicative constant. Thus, by requiring affine invariance, we have exhausted all degrees of freedom in the choice of the PDE. The way around this is to relax one or more of the other requirements, but not one of the invariants in the projective group. Faugeras and Keriven [101, 102, 103, 100] and Bruckstein and Shaked [49] give up the maximum principle. They then derive higher order PDEs that can hardly be considered smoothing operators. Establishing existence proofs and numerical simulations of this projective curve scale space are open problems. (See also Olver, Sapiro, and Tannenbaum [209].) Dibos does not give up locality or causality, and she is able to simulate her scale space numerically. This scale space no longer depends on a single scale parameter, but rather on two parameters. Geraets and others propose affine-invariant scale spaces for discrete sets with applications to object recognition [117, 118]. One of the first attempts to use the AMSS model for affine-invariant shape recognition was given by Cohignac, Lopez and Morel [75]. A more complete and sophisticated attempt, which performs image comparison by applying the affine scale space to all level lines of each image, is found in [178]. Alvarez and Morales used the affine scale space for corner and T-junction detection in digital images [13].

Appendix A

Jordan Curves

Γ will always denote a Jordan curve that is at least C^{2} and whose length is L. Γ_{i} denotes the open, simply connected set bounded by Γ, and Γ_{e} denotes the unbounded open set bounded by Γ. We will represent Γ with a specific arc-length parameterization: Let C_{L} be the circle at the origin with radius r = L/2π. Let the function c : C_{L} → Γ denote the arc-length parameterization with the property that the point εc′(s)^{⊥} ∈ Γ_{i} for all sufficiently small ε > 0. It follows from Proposition 4.4 and the Jordan curve theorem that we can choose this parameterization.

It is convenient to view C_{L} as a compact Abelian group. One can think of C_{L} as the interval [0, L], where 0 and L are identified by ordinary addition modulo L. This allows us to write s ± t for s, t ∈ C_{L} and have it make sense. This also means that a given point c(s) ∈ Γ is represented by a unique point s ∈ C_{L}.

The signed distance function d : ℝ^{2} → ℝ is defined as follows: If y ∈ Γ_{i}, then by definition d(y) = d(y, Γ) = inf_{s∈ C_{L}} |y − c(s)|. If y ∈ ℝ^{2} \ Γ_{i}, then d(y) = −d(y, Γ). The function d is well defined, and, given y ∈ ℝ^{2}, there is always at least one s ∈ C_{L} such that d(y) = |y − c(s)|. Since the function t ↦ |y−c(t)| has a minimum at this s, (∂|y−c(t)|^{2}/∂t)(s) = 0. This implies that (y − c(s)) · c′(s) = 0, which means that the vector y − c(s) is orthogonal to the unit tangent vector c′(s). Thus, there is a λ ∈ ℝ such that y = c(s) + λc′(s)^{⊥}. Furthermore, the definition of d and the fact that c′(s)^{⊥} “points inward” imply that λ = d(y). One of the main results in this section is that the function y ↦ d(y) assumes the smoothness of Γ, less one derivative, if y is near Γ.

We have assumed that Γ is C^{2}, so the scalar function κ : C_{L} → ℝ defined by κ(s) = c′′(s) · c′(s)^{⊥} is continuous and bounded. Define M = sup_{s∈C_{L}} κ(s) and define δ = 1/M. This notation is used throughout the section.

Exercise A.1. Show that ∫_{0}^{L} κ(s) ds = 2π and, hence, that M > 0.

Consider function z : C_{L} × ℝ → ℝ^{2} defined by z(s, λ) = c(s) + λc′(s)^{⊥}. The Jacobian of this mapping is computed as follows:

\frac{∂z}{∂s} = c′(s) + λc′′(s)^{⊥},
\frac{∂z}{∂λ} = c′(s)^{⊥}.

283 284 APPENDIX A. JORDAN CURVES

These two vectors are orthogonal, so the Jacobian is zero if and only c′(s) + λc′′(s)^{⊥} = 0. Since c′(s) + λc′′(s)^{⊥} is colinear with c′(s), taking the inner product with the unit tangent c′(s) shows that the Jacobian is zero if and only if 1 + λc′′(s)^{⊥} · c′(s) = 0. Note that

1 + λc′′(s)^{⊥} · c′(s) = 1 − λc′′(s) · c′(s)^{⊥} = 1 − λκ(s).

The next result states that z is locally one-to-one if |λ| < δ.

Lemma A.1. Assume that Γ is C^{n} and that|λ| < δ. Then, given (s, λ) ∈ C_{L} × ℝ, there is a neighborhood U of (s, λ) and a neighborhood V of z(s, λ) such that z : U → V is one-to-one and onto. Furthermore, z^{−1} : V → U is C^{n−1}.

Proof. The Jacobian of the mapping z : C_{L} ×ℝ → ℝ^{2} does not vanish at (s, λ), so the inverse mapping theorem applies. The lemma is just a restatement of the inverse mapping theorem for the case at hand.

The next result is similar to Lemma A.1, but it is more geometric and more specific about the size of the neighborhoods.

Lemma A.2. Assume that Γ is C^{2}. Choose any α such that 0 < α < δ. Then there exists a positive number β = β(α) with the following property: If |s−t| < β, s ≠ t, and if |λ| ≤ α and |μ| ≤ α, then

c(s) + λc′(s)^{⊥} ≠ c(t) + μc′(t)^{⊥}.

Proof. Let c(s_{0}) be any (fixed) point on Γ. For convenience of notation, and without loss of generality, we shift the parameterization and assume that s_{0} = 0. Establish a local coordinate systems as follows: c(0) = (0, 0), c′(0) = (1, 0) is the unit vector in the x direction and c′(0)^{⊥} is the unit vector in the y direction. All of the following statements assume that the arc-length parameter s is small and positive. Dual arguments for small negative values give the same results.

Let φ(s) denote the angle between the unit vector c′(s) and the x-axis, that is, the angle between c′(s) and c′(0). Then φ′(s) = κ(s), and

φ(s) = ∫_{0}^{s} κ(t) dt,

which gives the inequality |φ(s)| ≤ sM.

Consider the two straight lines c(0) + μc′(0)^{⊥}, μ ∈ ℝ, (which is the y-axis in our coordinate system), and c(s) + λc′(s)^{⊥}, λ ∈ ℝ. If φ(s) ≠ 0, then these lines intersect in a point (0, Y). We are going to estimate the value of λ = λ(s) for which

c(0) + Y (s)c′(0) = c(s) + λ(s)c′(s)^{⊥}.

Write x(s) = c(s) · c′(0) and y(s) = c(s) · c′(0)^{⊥}. Then

λ(s) = \frac{x(s)}{sin φ(s)}.

285

For small s, φ(s) is small, and | sin φ(s)| ≤ |φ(s)|. We have already mehtioned that φ(s) ≤ sM. Thus, we see that

|λ(s)| = \frac{x(s)}{| sin φ(s)|} ≥ \frac{x(s)}{|φ(s)|} ≥ \frac{x(s)}{sM}.  (1.1)

The next step is to get bounds for x(s)/s. For this, note that x′(s) = c′(s) · c′(0) = cos φ(s), so we have

x(s) = ∫_{0}^{s} cos φ(t) dt.

We get the inequality

∫_{0}^{s} (1 − \frac{φ^{2}(t)}{2}) dt ≤ x(s) ≤ s

by using the fact that 1 − t^{2}/2 ≤ cos t ≤ 1 for small t. Since we are assuming here that s > 0, this shows that

1 − \frac{1}{2s} ∫_{0}^{s} φ^{2}(t) dt ≤ \frac{x(s)}{s} ≤ 1.

Now use the bound |φ(s)| ≤ sM to deduce that

\frac{1}{2s} ∫_{0}^{s} φ^{2}(t) dt ≤ \frac{1}{6}s^{2}M^{2},

and hence that

1 − \frac{1}{6}s^{2}M^{2} ≤ \frac{x(s)}{s} ≤ 1.

For 0 < s < \sqrt{6}δ, the term on the left in positive, and we can combine the last inequality with (1.1) to see that

|λ| ≥ δ − \frac{1}{6}s^{2}M.

It is important to note that this estimate does not depend on properties of the points c(0) and c(s); it depends only on the arc-length distance between these points, that is, on s and on M. Of course, s must be small enough so the trigonometric estimates hold, and this depends on φ. But these estimates are all based on the bound |φ(s)| ≤ sM.

This is what we have shown: Given two points c(s) and c(t), s ≠ t, if |s − t| < \sqrt{6}δ and if the lines c(s) + λc′(s)^{⊥} and c(t) + μc′(t)^{⊥} intersect at a point x then

|x − c(s)| = |λ| > δ − |s − t|^{2}M/6,
|x − c(t)| = |μ| > δ − |s − t|^{2}M/6.

This proves the result: Given 0 < α < δ, let β = \sqrt{6}(min\{δ, \sqrt{(δ − α)δ}\}). Then |s−t| < β, |λ| < α, and |μ| < α, imply that c(s)+λc′(s)^{⊥} ≠ c(t)+μc′(t)^{⊥}.

286 APPENDIX A. JORDAN CURVES

These last two results are strictly local result; in fact, it is possible that the point z(s, λ) has two or more representations of the form z(t, μ) = c(t)+μc′(t)^{⊥}. To eliminate this ambiguity, we analyze the ambiguous sets. Define Δ_{i} to be the set of those points in Γ_{i} with the property that there are (at least) two points s, t ∈ C_{L}, s ≠ t such that d(y) = |y − c(s)| and d(y) = |y − c(t)|. Similarly, let Δ_{e} be the set of those points in Γ_{e} such that there are (at least) two points s, t ∈ C_{L}, s ≠ t such that d(y) = −|y − c(s)| and d(y) = −|y − c(t)|. If Γ is a circle, then Δ_{i} consists of the center point and Δ_{e} is the empty set. Our first result is that Δ_{i} is never empty.

Lemma A.3. If Γ is C^{2}, then Δ_{i} ≠ ∅.

Proof. Let y_{m} ∈ Γ_{i} be a point where d assumes its maximum value. We will show that y_{m} ∈ Δ_{i}. Suppose that this is not the case. Let s_{m} ∈ C_{L} be the unique point such that d(y_{m}) = |y_{m} − c(s_{m})|. Then y_{m} = c(s_{m}) + d(y_{m})c′(s_{m})^{⊥}, and the closed disk D(y_{m}, d(y_{m}) meets Γ in only one point. This implies that, for a sufficiently small ε > 0, the disk y = c(s_{m}) + (d(y_{m}) + ε)c′(s_{m})^{⊥} does not intersect Γ, and hence that d(y) > d(y_{m}) + ε, contradicting the fact that d(y_{m}) was a maximum. The conclusion is that y_{m} ∈ Δ_{i}.

Lemma A.4. Γ ∩ (\overline{Δ}_{i} ∪ \overline{Δ}_{e}) = ∅.

Proof. The proof is indirect, so assume that there is a point s′ ∈ C_{L} such that c(s′) ∈ \overline{Δ}_{i} ∩ Γ. Then there is a sequence of points y_{n} ∈ Δ_{i} that y_{n} → c(s′) as n → ∞. Since y_{n} ∈ Δ_{i}, there are at least two different points s_{n}, t_{n} ∈ C_{L} such that d(y_{n}) = |y_{n} − c(s_{n})| and d(y_{n}) = |y_{n} − c(t_{n})|. Furthermore, the points y_{n} have the two representations

y_{n} = z(s_{n}, d(y_{n})) = c(s_{n}) + d(y_{n})c′(s_{n})^{⊥},
y_{n} = z(t_{n}, d(y_{n})) = c(t_{n}) + d(y_{n})c′(t_{n})^{⊥}.

The sequences {s_{n}} and {t_{n}} each have at least one accumulation point. If they have two different accumulation points s and t, then it follows from the representations above that c(s) = c(t), which contradicts that fact that the mapping c : C_{L} → Γ is one-to-one. The alternative is that the sequences {s_{n}} and {t_{n}} have only one accumulation point s. Then s_{n} → s and t_{n} → s as n → ∞. This implies that s = s′: Since d(y_{n}) → 0, and we are left with y = c(s) = c(s′), and since c is one-to-one, s = s′. Eventually, d(y_{n}) < δ and we have two representations of y_{n} with s_{n} and t_{n} arbitrarily close to s′. This contradicts Lemma A.1. The conclusion is that \overline{Δ}_{i} ∩ Γ = ∅. The same proof works for Δ_{e}, if Δ_{e} ≠ ∅.

Now define the open set Ω as Ω = ℝ^{2} \(\overline{Δ}_{i} ∪\overline{Δ}_{e}). This is a global uniqueness set, in the sense that if y ∈ Ω, then there is a unique point s ∈ C_{L} such that d(y, Γ) = |y − c(s)|. By Lemma A.4, Ω is a neighborhood of Γ, and since Δ_{i} is not empty, the constant γ = d(Γ, \overline{Δ}_{i} ∪ \overline{Δ}_{e}) is finite and positive.

Exercise A.2. Sketch an example illustrating that the condition |λ| < δ is not sufficient to ensure that z(s, λ) ∈ Ω.

Lemma A.5. Assume that Γ is C^{n}, n ≥ 2, and that |λ| < min{γ, δ} is fixed. Then s ↦ z(s, λ) defines a C^{n−1} Jordan curve Γ_{λ}, and Γ_{λ} = {y | d(y) = λ}.

287

Proof. For a fixed λ, s ↦ z(s, λ) is a C^{n−1} mapping of C_{L} onto a plane curve Γ_{λ}. Since |λ| < γ, z(s, λ) ∈ Ω, and it has the unique representation z(s, λ) = c(s) + λc′(s)^{⊥}, where d(z(s, λ)) = λ. This means that the mapping s ↦ z(s, λ) is one-to-one and that Γ_{λ} = {y | d(y) = λ}. It remains to show that the curve Γ_{λ} has an arc-length parameterization that is C^{n−1}. This follows from the assumption that |λ| < δ:

z′(s, λ) = c′(s) + λc′(s)^{⊥} = (1 − λκ(s))c′(s),

where κ(s) = c′′(s)·c′(s)^{⊥}. Since 1−λκ(s) ≠ 0 for s ∈ C_{L}, Γ_{λ} has an arc-length parameterization that is C^{n−1} (see Exercise 4.3).

Theorem A.6. Assume that Γ is C^{n}, n ≥ 2. Then the signed distance function d is C^{n−1} on the open set {y | |d(y)| < min{γ, δ}}.

Proof. Assume that |d(y)| < min{γ, δ}. This implies there is a unique s ∈ C_{L} such that d(y) = ±|y − c(s)| and y has the unique representation

z(s, d(y)) = c(s) + d(y)c′(s)^{⊥}.

This is a global statement base on the fact that |d(y)| < γ. Lemma A.1 states that z^{−1} is locally C^{n−1}. Since z^{−1}(y) = (s, d(y)), we conclude that d is C^{n−1}.

There are two more results that are used when applying the shape local inclusion principle. The first one involves “trapping” a C^{2} Jordan arc locally between the boundaries of disks, and the second one shows the existence of the disks B(x, r) that appear in the definition of local inclusion. It was easy to establish (??), including the definition of the disk B, because we knew everything about the shape of the ellipse E(1). If, instead of having E(1), we have a set X ∈ D whose boundary is C^{2} in some neighborhood of x ∈ ∂X, then we can still “trap” the boundary of X between the boundaries of two disks.

Lemma A.7. Suppose that a Jordan arc Γ is C^{2} in a neighborhood of a point c(s). Then for every sufficiently small ε > 0, there is a neighborhood N = N(ε) of s such that the following conditions hold:

(i) c′′(s) ≠ 0. For s ∈ N, the arc c(s) lies between the boundaries of the two disks D_{1} = D_{1}(c_{1}, r_{1}) and D_{2} = D_{2}(c_{2}, r_{2}), where

c_{1} = c(s) + (κ + ε)^{−1} \frac{c′′(s)}{κ}, r_{1} = (κ + ε)^{−1},
c_{2} = c(s) + (κ − 2ε)^{−1} \frac{c′′(s)}{κ}, r_{2} = (κ − 2ε)^{−1},

where κ = |c′′(s)|.

(ii) c′′(s) = 0. For s ∈ N, the arc c(s) lies between the boundaries of the two disks D_{1} = D_{1}(c_{1}, r_{1}) and D_{2} = D_{2}(c_{2}, r_{2}), where

c_{1} = c(s) + ε^{−1}c′(s)^{⊥}, r_{1} = ε^{−1},
c_{2} = c(s) − ε^{−1}c′(s)^{⊥}, r_{2} = ε^{−1}.

288 APPENDIX A. JORDAN CURVES

Proof. The proof is completely straightforward and is based on the continuity of the curvature. We may assume that the boundary of X at x is represented by the graph of a C^{2} function f, where f(0) = 0 and f′(0) = 0. The curvature of ∂X at x is f′′(0). There are two cases: f′′(0) ≠ 0 and f′′(0) = 0. We begin by assuming that f′′(0) > 0. Choose ε such that 0 < 2ε < f′′(0). By the continuity of f′′, there is a neighborhood N = N(ε) of x = 0 such that

f′′(0) − ε ≤ f′′(x) ≤ f′′(0) + ε

for all x ∈ N. This implies by integration that

(f′′(0) − ε)x ≤ f′(x) ≤ (f′′(0) + ε)x

and that

\frac{x^{2}}{2} (f′′(0) − ε) ≤ f(x) ≤ \frac{x^{2}}{2} (f′′(0) + ε)

for all x ∈ N. Thus, the graph of f is trapped between the graphs of two parabolas as long as x ∈ N. The next step is to trap the graph of f between two circles.

The osculating circle of the graph of the parabola y = (x^{2}/2)(f′′(0) + ε) has radius r = 1/(f′′(0) + ε) and center at (0, r). Furthermore, the parabola lies below the circle for all x in the interval −r < x < r. Stated analytically, this is the inequality

\frac{x^{2}}{2r} ≤ r − \sqrt{r^{2} − x^{2}},

which holds for all |x| < r.

We now want a circle that lies below the parabola y = (x^{2}/2)(f′′(0)−ε). For this, we take the circle whose radius is R = 1/(f′′(0)−2ε), which is slightly larger than the radius of the osculating circle of the graph of y = (x^{2}/2)(f′′(0) − ε). A straightforward computation shows that there is a constant C > 0 such that

R − \sqrt{R^{2} − x^{2}} ≤ \frac{x^{2}}{2} (f′′(0) − ε)

for all x such that x^{2} ≤ Cε. Thus, for all sufficiently small |x|, the graph of f is trapped between two circles:

R − \sqrt{R^{2} − x^{2}} ≤ f(x) ≤ r − \sqrt{r^{2} − x^{2}},

where R = 1/(f′′(0) − 2ε) and r = 1/(f′′(0) + ε).

This covers the case where f′′(0) > 0. The case where f′′(0) < 0 is similar, the function f being trapped between circles with centers on the vegetive y-axis:

−r + \sqrt{r^{2} − x^{2}} ≤ f(x) ≤ −R + \sqrt{R^{2} − x^{2}}.

If f′′(0) = 0, the situation is slightly different. By the continuity of f′′, given ε > 0, there is a neighborhood N such that |f′′(x)| ≤ ε for all x ∈ N. Thus,

−xε ≤ f′(x) ≤ xε

and

−\frac{x^{2}}{2} ε ≤ f(x) ≤ \frac{x^{2}}{2} ε

289

for all x ∈ N. The graph of f is then trapped between the two osculating circles of the two parabolas: −r + \sqrt{r^{2} − x^{2}} ≤ f(x) ≤ r − \sqrt{r^{2} − x^{2}}, where r = 1/ε. Again, this holds for all x ∈ N. The statement of the results in the lemma is just a restatement of the results for f in terms of the vector representation of ∂X.

Lemma A.7 tells us exactly how to trap a C^{2} arc between two disks. The next, and last result of the section, tells us how to select the disks B needed to apply the local inclusion principle.

Lemma A.8. Suppose that a Jordan curve Γ is C^{2} in a neighborhood of a point c(s). Then there is a closed disk D(c(s), r) centered at c(s) with radius r that has the following property: For every closed disk D′(c(s), r′), 0 < r′ ≤ r, Γ ∩ D′ is a Jordan arc that intersects the boundary of D′ in exactly two points, and D′ \ Γ consists of two disjoint sets D′_{i} ⊂ Γ_{i} and D′_{e} ⊂ Γ_{e}.

Proof. As in Lemma A.7, we represent Γ near c(s) as the graph of a function f, where f(0) = 0 and f′(0) = 0. We wish to formalize the idea that, for all sufficiently small r, the graph of a function f intersects the circles C(0, r) in only two points, at one point for x < 0 and one for x > 0. Assume that x > 0. Then what we mean analytically is that the tangent vector (1, f′(x) has a positive component in the direction of the radial vector (x, f(x)). Thus we need to show that (x, f(x)) · (1, f′(x) = x + f(x)f′(x) > 0 for all sufficiently small x > 0. A direct consequence of the computation in the proof of Lemma A.7 is that |f(x)f′(x)| = O(x^{3}) in some neighborhood of x = 0. Thus, for all sufficiently small x > 0, x+f(x)f′(x) > 0, and the piece of Γ represented locally by the graph of f intersects the circles C(0, r) once in the right-half plane for all sufficiently small r. A similar analysis shows that the piece of Γ represented locally by the graph of f intersects the circles C(0, r) once in the left-half plane for all sufficiently small r.

Let Γ_{r} denote the graph of f that lies in the disk D(0, r). We must show that

D(0, r) ∩ Γ = D(0, r) ∩ Γ_{r}

for all sufficiently small r. If this were not the case, there would be a sequence of points c(t_{n}) such that c(t_{n}) → c(s). It should be clear from what we have just done that the points t_{n} ∈ C_{L} are bounded away from s ∈ C_{L}. Then if t is any accumulation point of {t_{n}}, which must exist, t ≠ s, but c(t) = c(s). This contradicts the fact that c is one-to-one.

We have shown that there is an r > such that for all disks D′ = D′(0, r′), 0 < r′ ≤ r, D′ \ Γ_{r} consists of two arc-wise connected sets that do not intersect Γ but whose boundaries intersect Γ. By the Jordan curve theorem, one of the sets must lie in Γ_{i} and the other must be in Γ_{e}.

290 APPENDIX A. JORDAN CURVES

Bibliography

[1] E. H. Adelson and J. R. Bergen, Spatiotemporal energy models for the perception of motion, J. Opt. Soc. Amer. A, 2 (1985), pp. 284–299.

[2] G. J. Agin, Representation and Description of Curved Objects, PhD thesis, Stanford U., October 1972.

[3] F. Almgren, J. E. Taylor, and L. Wang, Curvature-driven flows: A variational approach, SIAM J. Control Optimization, 31 (1993), pp. 387– 438.

[4] S. Altschuler, S. B. Angenent, and Y. Giga, Mean curvature flow through singularities for surfaces of rotation, J. Geom. Anal., 5 (1995), pp. 293–358.

[5] L. Alvarez and J. Esclarìn, Image quantization using reactiondiffusion equations, SIAM J. Appl. Math., 57 (1997), pp. 153–175.

[6] L. Alvarez, Y. Gousseau, and J.-M. Morel, Scales in natural images and a consequence on their bounded variation norm, in Scale-Space Theories in Computer Vision, M. Nielsen, P. Johansen, O. F. Olson, and J. Weickert, eds., Lecture Notes in Computer Science 1682, Springer, 1999, pp. 247–258.

[7] L. Alvarez, Y. Gousseau, and J.-M. Morel, The size of objects in natural and artificial images, in Advances in Imaging and Electron Physics, vol. 111, Academic Press, 1999, pp. 167–242.

[8] L. Alvarez, F. Guichard, P.-L. Lions, and J.-M. Morel, Axiomatisation et nouveaux opérateurs de la morphologie mathématique., C. R. Acad. Sci. Paris Sér. I, 315 (1992), pp. 265–268.

[9] L. Alvarez, F. Guichard, P.-L. Lions, and J.-M. Morel, Axiomes et équations fondamentales du traitement d’images. (Analyse multiéchelle et E.D.P.), C. R. Acad. Sci. Paris Sér. I, 315 (1992), pp. 135–138.

[10] L. Alvarez, F. Guichard, P.-L. Lions, and J.-M. Morel, Axioms and fundamental equations of image processing, Arch. Rational Mech. Anal., 16 (1993), pp. 200–257.

[11] L. Alvarez, P.-L. Lions, and J.-M. Morel, Image selective smoothing and edge detection by nonlinear diffusion (ii), SIAM J. Numer. Anal., 29 (1992), pp. 845–866.

291 292 BIBLIOGRAPHY

[12] L. Alvarez and L. Mazorra, Signal and image restoration using shock filters and anisotropic diffusion, SIAM J. Numer. Anal., 31 (1994), pp. 590–605.

[13] L. Alvarez and F. Morales, Affine morphological multiscale analysis of corners and multiple junctions, Int. J. Comput. Vision, 25 (1997), pp. 95–107.

[14] L. Alvarez and J.-M. Morel, Formalization and computational aspects of image analysis, Acta Numerica, (1999), pp. 1–59.

[15] L. Ambrosio, V. Caselles, S. Masnou, and J.-M. Morel, The connected components of caccioppoli sets and applications to image processing, J. Eur. Math. Soc., 3 (2001), pp. 213–266.

[16] L. Ambrosio, V. Caselles, S. Masnou, and J.-M. Morel, Connected components of sets of finite perimeter and applications to image processing, J. Eur. Math. Soc., 3 (2001), pp. 39–92.

[17] L. Ambrosio and H. M. Soner, Level set approach to mean curvature flow in arbitrary codimension, J. Differential Geom., 43 (1996), pp. 693– 737.

[18] P. Anandan, A computational framework and an algorithm for the measurement of visual motion, Int. J. Comput. Vision, 2 (1989), pp. 283–310.

[19] F. Andreu, C. Ballester, V. Caselles, and J. M. Mazon, Minimizing total variation flow, C. R. Acad. Sci. Paris Sér. I, 331 (2000), pp. 867–872.

[20] F. Andreu, C. Ballester, V. Caselles, and J. M. Mazon, Minimizing total variation flow, Differential and Integral Equations, 4 (2001), pp. 321–360.

[21] S. B. Angenent, Parabolic equations for curves on surfaces. I: Curves with p-integrable curvature, Ann. of Math. (2), 132 (1990), pp. 451–483.

[22] S. B. Angenent, Parabolic equations for curves on surfaces. II: Intersections, blow-up and generalized solutions, Ann. of Math. (2), 133 (1991), pp. 171–215.

[23] S. B. Angenent, Some recent results on mean curvature flow, in Recent advances in partial differential equations, M. A. H. et al., ed., 1994.

[24] S. B. Angenent, G. Sapiro, and A. Tannenbaum, On the affine heat flow for nonconvex curves, J. Amer. Math. Soc., 11 (1998), pp. 601–634.

[25] S. B. Angenent and J. J. L. Velazquez, Degenerate neckpinches in mean curvature flow, J. Reine Angew. Math., 482 (1997), pp. 15–66.

[26] G. R. Arce and N. Gallagher, Stochastic analysis for the recursive median filter process, IEEE Trans. Information Theory, 34 (1988), pp. 669– 679.

BIBLIOGRAPHY 293

[27] G. R. Arce and M. P. McLoughlin, Theoretical analysis of the max/median filter, 35 (1987), pp. 60–69.

[28] H. Asada and M. Brady, The curvature primal sketch, IEEE Trans. Patt. Anal. Mach. Intell., 8 (1986), pp. 2–14.

[29] J. Astola, P. Heinonen, and Y. Neuvo, On root structures of median and median-type filters, IEEE Trans. Acoust. Speech Signal Process., 35 (1987), pp. 1199–1201.

[30] J. Babaud, A. P. Witkin, M. Baudin, and R. O. Duda, Uniqueness of the Gaussian kernel for scale-space filtering, IEEE Trans. Patt. Anal. Mach. Intell., 8 (1986).

[31] B. Baldwin, D. Geiger, and R. A. Hummel, Resolution-appropriate shape representation, in International Conference on Computer Vision, 1998, pp. 460–465.

[32] C. Ballester, V. Caselles, and P. Monasse, The tree of shapes of an image, tech. rep., CMLA, Ecole Normale Supérieure-Cachan, France, 2001. Available at http://www.cmla.ens-

cachan.fr/Cmla/index.html.

[33] C. Ballester, E. Cubero-Castan, M. González, and J.-M. Morel, Fast contrast invariant image intersection. Preprint CERE-MADE, 1998.

[34] J. A. Bangham, P. Ling, and R. Young, Multiscale recursive medians, scale-space, and transforms with applications to image processing, IEEE Trans. Image Process., 5 (1996).

[35] G. Barles.

[36] G. Barles, Interior gradient bounds for the mean curvature equation by viscosity solutions methods, Differential and Integral Equations, 4 (1991).

[37] G. Barles and C. Georgelin, A simple proof of convergence for an approximation scheme for computing motions by mean curvature, SIAM J. Numer. Anal., 32 (1995), pp. 484–500.

[38] G. Barles and P. M. Souganidis, Convergence of approximation schemes for fully nonlinear second order equations, Asymptotic Analysis, 4 (1991), pp. 271–283.

[39] P. R. Beaudet, Rotationally invariant image operators, in Proc. International Conference on Pattern Recognition, 1978, pp. 579–583.

[40] G. Bellettini and M. Novaga, Comparison results between minimal barriers and viscosity solutions for geometric evolutions, Ann. Scuola Norm. Sup. Pisa Cl. Sci. IV, 26 (1998), pp. 97–131.

[41] M. J. Black, G. Sapiro, D. H. Marimont, and D. Heeger, Robust anisotropic diffusion, IEEE Trans. Image Process., 7 (1998), pp. 421–432.

294 BIBLIOGRAPHY

[42] J. Blom, B. M. te Haar Romeny, and J. J. Koenderink, Affine invariant corner detection, tech. rep., 3D Computer Vison Research Group, Utrecht University, NL, 1992.

[43] J. Blom, B. T. te Haar Romeny, A. Bel, and J. J. Koenderink, Spatial derivatives and the propagation of noise in Gaussian scale-space, Tech. Rep. 3DCV 91-03, U. of Utrecht, Dept. of Medical and Physiological Physics, 1991.

[44] S. K. Bose, K. K. Biswas, and S. K. Gupta, Model-based object recognition: The role of affine invariants, AIEng, 10 (1996), pp. 227–234.

[45] K. A. Brakke, The motion of a surface by its mean curvature, Mathematical Notes, 20, Princeton University Press, Princeton, NJ, 1978.

[46] L. Breiman, Probability, Addison-Wesley, 1968.

[47] H. Brezis, Analyse Fonctionnelle, Théorie et Applications, Masson, Paris, 1983.

[48] D. R. K. Brownrigg, The weighted median filter, Communications of the ACM, 27 (1984), pp. 807–818.

[49] A. M. Bruckstein and D. Shaked, On projective invariant smoothing and evolutions of planar curves and polygons, Journal of Mathematical Imaging and Vision, 7 (1997), pp. 225–240.

[50] P. J. Burt and E. H. Adelson, The laplacian pyramid as a compact image code, IEEE Trans. Comm., 31 (1983), pp. 532–540.

[51] A. R. Butz, A class of rank order smoothers, IEEE Trans. Acoust. Speech Signal Process., 34 (1986), pp. 157–165.

[52] J. F. Canny, A computational approach to edge detection, IEEE Trans. Patt. Anal. Mach. Intell., 8 (1986), pp. 679–698.

[53] F. Cao, Absolutely minimizing lipschitz extension with discontinuous boundary data, C. R. Acad. Sci. Paris Sér. I, 327 (1998).

[54] F. Cao, Partial differential equation and mathemetical morphology, J. Math. Pures Appl., 77 (1998), pp. 909–941.

[55] R. A. Carmona and S. F. Zhong, Adaptive smoothing respecting feature directions, IEEE Trans. Image Process., 7 (1998), pp. 353–358.

[56] V. Caselles, F. Catté, T. Coll, and F. Dibos, A geometric model for active contours, Numerische Mathematik, 66 (1993), pp. 1–31.

[57] V. Caselles, B. Coll, and J.-M. Morel, Topographic maps and local contrast changes in natural images, International Journal of Computer Vision, volume=.

[58] V. Caselles, B. Coll, and J.-M. Morel, Partial differential equations and image smoothing. Semin. Equ. Deriv. Partielles, Ec. Polytech., Cent. Math., Palaiseau Semin. 1995-1996, Exp. No.21, 1996.

BIBLIOGRAPHY 295

[59] V. Caselles, B. Coll, and J.-M. Morel, Scale space versus topographic map for natural images, in Scale Space, 1997.

[60] V. Caselles, B. Coll, and J.-M. Morel, Topographic maps and local contrast changes in natural images, Int. J. Comput. Vision, 33 (1999), pp. 5–27.

[61] V. Caselles, R. Kimmel, and G. Sapiro, Geodesic active contours, Int. J. Comput. Vision, 1 (1997), pp. 61–79.

[62] V. Caselles, J. L. Lisani, J.-M. Morel, and G. Sapiro, Shape preserving local histogram modification, 1999.

[63] V. Caselles, J.-M. Morel, and C. Sbert, An axiomatic approach to image interpolation, IEEE Trans. Image Process., 7 (1998), pp. 376–386.

[64] V. Caselles, G. Sapiro, and D. H. Chung, Vector median filters, vector morphology, and coupled PDE’s: Theoretical connections, J. Math. Imaging Vision, 12 (2000), pp. 109–120.

[65] V. Caselles and C. Sbert, What is the best causal scale space for 3D images?, SIAM, 56 (1996).

[66] F. Catté, F. Dibos, and G. Koepfler, A morphological scheme for mean curvature motion and applications to anisotropic diffusion and motion of level sets, SIAM J. Numer. Anal., volume=.

[67] M. H. Chen and P. F. Yan, A multiscaling approach based on morphological filtering, IEEE Trans. Patt. Anal. Mach. Intell., 11 (1989), pp. 6794–700.

[68] R. C. Chen and P. T. Yu, On the optimal design of rational rank selection filters for image restoration, in Proceedings International Conference on Image Processing, 1996, p. 16A9.

[69] Y. G. Chen, Y. Giga, and S. Goto, Uniqueness and existence of viscosity solutions of generalized mean curvature flow equations, Proc. Japan Acad. Ser. A Math. Sci., 65 (1989), pp. 207–210.

[70] Y. G. Chen, Y. Giga, and S. Goto, Uniqueness and existence of viscosity solutions of generalized mean curvature flow equations, J. Differential Geom., 33 (1991), pp. 749–786.

[71] F. Cheng and A. N. Venetsanopoulos, An adaptive morphological filter for image processing, IEEE Trans. Image Process., 1 (1992), pp. 533– 539.

[72] Y. Cheng, Analysis of affine invariants as approximate perspective invariants, Computer Vision and Image Understanding, 63 (1996), pp. 197– 207.

[73] B. Chow, Deforming convex hypersurfaces by the nth root of the gaussian curvature, J. Differential Geom.

296 BIBLIOGRAPHY

[74] B. Chow, Deforming convex hypersurfaces by the square root of the scalar curvature, Invent. Math., 87 (1987), pp. 63–82.

[75] T. Cohignac, C. Lopez, and J.-M. Morel, Integral and local affine invariant parameter and application to shape recognition, in Proceedings International Conference on Pattern Recognition, year=.

[76] R. Courant and F. John, Introduction to Calculus and Analysis, Springer, 2000. Reprint of the 1989 edition.

[77] E. Coyle, N. Gallagher, S. Bass, J. Fitch, and R. Harber, Digital filtering by threshold decomposition, 1987. U.S. Patent 4,868,773.

[78] E. J. Coyle, Rank order operators and the mean absolute error criterion, IEEE Trans. Acoust. Speech Signal Process., 36 (1988), pp. 63–76.

[79] M. G. Crandall, H. Ishii, and P.-L. Lions, User’s guide to viscosity solutions of second order partial differential equations, Bull. Amer. Math. Soc. (N.S.), 27 (1992), pp. 1–67.

[80] J. Damon, Generic structure of two dimensional images under Gaussian blurring, SIAM J. Appl. Math., 59 (1998), pp. 97–138.

[81] P. E. Danielsson, Getting the median faster, Computer Graphics Image Processing, 17 (1981), pp. 71–78.

[82] J. G. Daugman, Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by 2D visual cortical filters, Journal of the Optical Society of America, 2 (1985), pp. 1160–1169.

[83] E. R. Davies, On the noise suppression and image enhancement characteristics of the median, truncated median and mode filters, Pattern Recognition Letters, 7 (1988), pp. 87–97.

[84] R. Deriche, Using canny’s criteria to derive a recursively implemented optimal edge detector, Int. J. Comput. Vision, 1 (1987), pp. 167–187.

[85] R. Deriche and G. Giraudon, A computational approach for corner and vertex detection, International Journal of Computer Vision, 10 (1993), pp. 101–124.

[86] F. Dibos, Affine length and affine dimension of a 1-set of the plane, Proceedings of the Royal Society of Edinburgh, 126 (1996), pp. 985–993.

[87] J. Dieudonné, Foundations of Modern Analysis, Academic Press, 1960.

[88] G. Dudek, Shape Representation from Curvature, PhD thesis, U. Toronto, 1990.

[89] G. Dudek and J. K. Tsotsos, Robustly recognizing curves using curvature-tuned smoothing, in IEEE Conference on Computer Vision and Pattern Recognition, 1991, pp. 35–41.

BIBLIOGRAPHY 297

[90] C. L. Epstein and M. Gage, The curve shortening flow, in Wave motion: theory, modelling, and computation, Springer, NY, 1987, pp. 15–59. Proc. Conf. Hon. 60th Birthday P. D. Lax (Berkeley, CA, 1986) Math. Sci. Res. Inst. Publ. 7.

[91] L. C. Evans, Nonlinear semigroup theory and viscosity solutions of hamilton-jacobi pde, in Nonlinear Semigroups, Partial Differential Equations and Attractors year=.

[92] L. C. Evans, Regularity for fully nonlinear elliptic equations and motion by mean curvature, in Viscosity Solutions and Applications, I. C. D. et al., ed., Lect. Notes Math. 1660, springer, 1987, pp. 97–133. Lectures given at the 2nd session of the Centro Internazionale Matematico Estivo, Montecatini Terme, Italy, June 12-20, 1995.

[93] L. C. Evans, Convergence of an algorithm for mean curvature motion, Indiana Univ. Math. J., 42 (1993), pp. 553–557.

[94] L. C. Evans, Partial Differential Equations, Graduate Studies in Mathematics, 19, American Mathematical Society, Providence, RI, 1998.

[95] L. C. Evans and J. Spruck, Motion of level sets by mean curvature. I., J. Differential Geom., 33 (1991), pp. 635–681.

[96] L. C. Evans and J. Spruck, Motion of level sets by mean curvature. II., Trans. Am. Math. Soc., 330 (1992), pp. 321–332.

[97] L. C. Evans and J. Spruck, Motion of level sets by mean curvature. III., J. Geom. Anal., 2 (1992), pp. 121–150.

[98] L. C. Evans and J. Spruck, Motion of level sets by mean curvature. IV., J. Geom. Anal., 5 (1995), pp. 79–116.

[99] F. Falzon and G. Giraudon, Singularity analysis and derivative scalespace, in IEEE Computer Vision and Pattern Recognition, 1994, pp. 245– 250.

[100] O. Faugeras, Cartan’s moving frame method and its application to the geometry and evolution of curves in the euclidean, affine and projective planes, in Applications of Invariance in Computer Vision, J. L. Mundy, A. Zisserman, and D. Forsyth, eds., Lecture Notes in Computer Science 825, Springer-Verlag, 1994, pp. 11–46.

[101] O. Faugeras and R. Keriven, Affine curvature from affine scale-space, in International Conferenc on Computer Vision, 1995. Submitted.

[102] O. Faugeras and R. Keriven, Scale-spaces and affine curvature, in Proceedings Europe-China Workshop on Geometrical modelling and Invariants for Computer Vision, vol. 1, 1995, pp. 17–24.

[103] O. Faugeras and R. Keriven, Some recent results on the projective evolution of 2-D curves, in International Conference in Image Processing, vol. 3, 1995, pp. 13–16.

[104] W. J. Firey, Shapes of worn stones, Mathematika, 21 (1974), pp. 1–11.

298 BIBLIOGRAPHY

[105] J. P. Fitch, E. J. Coyle, and N. C. Gallagher, Jr., Median filtering by threshold decomposition, IEEE Trans. Acoust. Speech Signal Process., 32 (1984), pp. 1183–1188.

[106] L. M. J. Florack, Image Structure, vol. 10 of Computational Imaging and Vision, Kluwer Academic Publishers, Dordrecht, NL, 1997.

[107] L. M. J. Florack, B. M. ter Haar Romeny, J. J. Koenderink, and M. A. Viergever, Families of tuned scale-space kernels, in Proceedings of the European Conference on Computer Vision, 1992, pp. 19–23.

[108] L. M. J. Florack, B. M. ter Haar Romeny, J. J. Koenderink, and M. A. Viergever, Scale and the differential structure of images, Image and Vision Computing, 10 (1992), pp. 376–388.

[109] L. M. J. Florack, B. M. ter Haar Romeny, J. J. Koenderink, and M. A. Viergever, The multiscale local jet, in Proceedings of the VIP, Utrecht, The Netherlands, 1993, pp. 21–24.

[110] L. M. J. Florack, B. M. ter Haar Romeny, J. J. Koenderink, and M. A. Viergever, General intensity transformations and differential invariants, Journal of Mathematical Imaging and Vision, 4 (1994), pp. 171–187.

[111] L. M. J. Florack, B. M. ter Haar Romeny, J. J. Koenderink, and M. A. Viergever, Linear scale-space, J. Math. Imaging Vision, 4 (1994), pp. 325–351.

[112] D. A. Forsyth, J. L. Mundy, and A. Zisserman, Transformational invariance: A primer, Image and Vision Computing, 10 (1992), pp. 39–45.

[113] D. Gabor, Information theory in electron microscopy, Journal of Laboratory Investigation, 14 (1965), pp. 801–807. The Journal of Laboratory Investigation was published by the International Academy of Pathology.

[114] M. Gage, An isometric inequality with applications to curve shortening, Duke Math. J., 50 (1983), pp. 1225–1229.

[115] M. Gage, Curve shortening makes convex curves circular, Invent. Math., 76 (1984), pp. 357–364.

[116] M. Gage and R. S. Hamilton, The heat equation shrinking convex plane curves, J. Differential Geom., 23 (1986), pp. 69–96.

[117] R. Geraets, A. H. r, B. M. ter Haar Romeny, and M. A. Viergever, Affine scale-space for discrete pointsets, in Proc. Soc. for Neural Networks, Nijmegen, NL, 1995.

[118] R. Geraets, A. H. Salden, B. M. ter Haar Romeny, and M. A. Viergever, Object recognition by affine evolution of measured interest points, in Proc. Computing Science in the Netherlands, Utrecht, the Netherlands, 1995, SION, pp. 86–97.

BIBLIOGRAPHY 299

[119] P. J. Giblin and G. Sapiro, Affine-invariant symmetry sets, in Foundations of Computational Mathematics Conference, Instituto Nacional de Matemtica Pura e Aplicada, 1997.

[120] E. D. Giorgi, New conjectures on flow by mean curvature, in Nonlinear variational problems and partial differential equations. Proceedings of the 3rd conference held in the autumn of 1990 at Isola d’Elba, Italy. publisher=.

[121] E. D. Giorgi and L. Ambrosio, Un nuovo tipo di funzionale del calcolo delle variazioni, Atti. Accad. Naz. Lincei, s.8 (1988), pp. 199–210.

[122] M. J. E. Golay, Smoothing data by least squares procedures and by filtering, IEEE Trans. Computer, 21 (1972), pp. 299–301.

[123] M. A. Grayson, The heat equation shrinks embedded plane curves to round points, J. Differential Geom., 26 (1987), pp. 285–314.

[124] M. A. Grayson, A short note on the evolution of a surface by its mean curvature, Duke Math. J., 58 (1989), pp. 555–558.

[125] F. Guichard, Axiomatisation des analyses multi-échelles d’images et de films - Axiomatization of images and movies scale-space., PhD thesis, Univ. Paris IX-Dauphine, 1994.

[126] F. Guichard, C. Lopez, and J.-M. Morel, Une axiomatization de la vision préattentive, Images des Mathématiques, (1995), pp. 48–57.

[127] F. Guichard and J.-M. Morel, Partial Differential Equations and Image Iterative Filtering, 1995. Washington D.C.

[128] F. Guichard and J.-M. Morel, Partial differential equations and image iterative filtering, in The State of the Art in Numerical Analysis, I. S. D. et al., ed., Inst. Math. Appl. Conf. Ser., New Ser. 63, Institute of Mathematics and its Applications (IMA), Oxford Univ. Press, 1997, pp. 525–562.

[129] R. M. Haralick, Digital step edges from zero-crossing of second directional derivatives, IEEE Trans. Patt. Anal. Mach. Intell., 6 (1984), pp. 58– 68.

[130] R. M. Haralick, S. R. Sternberg, and X. Zhuang, Image analysis using mathematical morphology, IEEE Trans. Patt. Anal. Mach. Intell., 9 (1987), pp. 532–550.

[131] R. C. Hardie and K. E. Barner, Rank conditioned rank selection filters for signal restoration, IEEE Trans. Image Process., 3 (1994), pp. 192–206.

[132] H. J. A. M. Heijmans, Self-dual morphological operators and filters, J. Math. Imaging Vision, 6 (1996), pp. 15–36.

[133] H. J. A. M. Heijmans and C. Ronse, The algebraic basis of mathematical morphology: I. dilations and erosions, Computer Vision, Graphics, and Image Processing, 50 (1990), pp. 245–295.

300 BIBLIOGRAPHY

[134] C. W. Helstrom, Image restoration by the method of least squares, J. Opt. Soc. Amer., 57 (1967), pp. 297–303.

[135] J. M. Hereford and W. T. Rhodes, Nonlinear optical image filtering by time-sequential threshold decomposition, Opt. Eng., 27 (1988), pp. 274– 279.

[136] H.Ishii, A generalization of the bence, merriman and osher algorithm for motion by mean curvature, in Proceedings of the international conference on curvature flows and related topics held in Levico, Italy, A. D. et al., ed., vol. 5 of GAKUTO Int. Ser. Math. Sci. Appl., Tokyo, 1994, Gakkotosho.

[137] B. K. P. Horn and E. J. Weldon, Jr., Filtering closed curves, IEEE Trans. Patt. Anal. Mach. Intell., 8 (1986), pp. 665–668.

[138] C. T. Huang and O. R. Mitchell, A Euclidean distance transform using grayscale morphology decomposition, IEEE Trans. Patt. Anal. Mach. Intell., 16 (1994), pp. 443–448.

[139] T. S. Huang, G. J. Yang, and G. Y. Tang, A fast two-dimensional median filtering algorithm, IEEE Trans. Acoust. Speech Signal Process., 27 (1979), pp. 13–18.

[140] D. H. Hubel and T. N. Wiesel, Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex, J. Physiol., 160 (1962), pp. 106–154.

[141] G. Huisken, Flow by mean curvature of convex surfaces into spheres, J. Differential Geom., 20 (1984), pp. 237–266.

[142] G. Huisken, Singularities of the mean curvature flow, in Tsing Hua lectures on geometry and analysis, S.-T. Yau, ed., Cambridge, MA, 1997, International Press, pp. 125–130. Lecture notes taken by Shun–Cheng Chang, Taiwan, 1990-91.

[143] R. A. Hummel, Representations based on zero-crossing in scale-space, in IEEE Computer Vision and Pattern Recognition, 1986, pp. 204–209.

[144] R. A. Hummel, The scale-space formulation of pyramid data structures, in PCV88, 1988, pp. 107–123.

[145] R. A. Hummel, B. Kimia, and S. W. Zucker, Gaussian blur and the heat equation: Forward and inverse solutions, in IEEE Computer Vision and Pattern Recognition, 1983, pp. 416–421.

[146] R. A. Hummel, B. Kimia, and S. W. Zucker, Deblurring gaussian blur, Computer Vision, Graphics, and Image Processing, 38 (1987), pp. 66–80.

[147] R. A. Hummel and R. Moniot, Reconstructions from zero-crossings in scale-space, IEEE Trans. Acoust. Speech Signal Process., 37 (1989), pp. 2111–2130.

[148] R. A. Hummel and H. J. Wolfson, Affine invariant matching, in DARPA88, 1988, pp. 351–364.

BIBLIOGRAPHY 301

[149] T. Ilmanen, P. Sternberg, and W. P. Ziemer, Equilibrium solutions to generalized motion by mean curvature, J. Geom. Anal., 8 (1998), pp. 845–858.

[150] H. H. S. Ip, D. J. Potter, and D. S. Lebedev, Impulse noise cleaning by iterative threshold median filtering, Pattern Recognition Letters, 2 (1983), pp. 89–93.

[151] H. Ishii and P. Souganidis, Generalized motion of noncompact hypersurfaces with velocity having arbitrary growth on the curvature tensor, Tohoku Math. J. II, 47 (1995), pp. 227–250.

[152] P. T. Jackway and M. Deriche, Scale-space properties of the multiscale morphological dilation erosion, IEEE Trans. Patt. Anal. Mach. Intell., 18 (1996), pp. 38–51.

[153] S. Jaffard, Y. Meyer, and R. D. Ryan, Wavelets: Tools for Science and Technology, Society for Industrial and Applied Mathematics, Philadelphia, 2001.

[154] R. Jensen, The maximum principle for viscosity solutions of fully nonlinear second order partial differential equations, Arch. Rational Mech. Anal., 101 (1988).

[155] B. Julesz, A method of coding TV signals based on edge detection, Bell System Tech. J. 38 (1959), pp. 1001–1020.

[156] M. Kass, A. Witkin, and D. Terzopoulos, Snakes: Active contour models, Int. J. Comput. Vision, 1 (1998), pp. 321–331.

[157] P. Kempenaers, L. VanGool, and A. Oosterlinck, Shape recognition under affine distortions, in VF91, 1991, pp. 323–332.

[158] S. Kichenassamy, The Perona–Malik paradox, SIAM J. Appl. Math., 57 (1997), pp. 1328–1342.

[159] V. Kim and L. Yaroslavskii, Rank algorithms for picture processing, Computer Vision, Graphics, and Image Processing, 35 (1986), pp. 234– 258.

[160] B. Kimia, A. Tannenbaum, and S. W. Zucker, On the evolution of curves via a function of curvature. I. The classical case, J. Math. Anal. Appl., 163 (1992), pp. 438–458.

[161] B. Kimia, A. R. Tannenbaum, and S. W. Zucker, Shapes, shocks, and deformations I: The components of 2-dimensional shape and the reactiondiffusion space, Int. J. Comput. Vision, 15 (1995), pp. 189–224.

[162] R. Kimmel and A. M. Bruckstein, Shape offsets via level sets., Computer Aided Design, 5 (1993), pp. 154–162.

[163] R. Kimmel, D. Shaked, N. Kiryati, and A. M. Bruckstein, Skeletonization via distance maps and level sets, Computer Vision and Image Understanding, 3 (1995), pp. 382–391.

302 BIBLIOGRAPHY

[164] J. J. Koenderink, The structure of images, Biological Cybernetics, 50 (1984), pp. 363–370.

[165] J. J. Koenderink, A hitherto unnoticed singularity of scale-space, IEEE Trans. Patt. Anal. Mach. Intell., 11 (1989), pp. 1222–1224.

[166] J. J. Koenderink and A. J. van Doorn, Photometric invariants related to solid shape, Optica Acta, 27 (1980), pp. 981–996.

[167] J. J. Koenderink and A. J. van Doorn, Dynamic shape, Biological Cybernetics, 53 (1986), pp. 383–396.

[168] L. S. G. Kovasznay and H. M. Joseph, Image processing, Proc. IRE, 43 (1955), p. 560.

[169] H. P. Kramer and J. B. Bruckner, Iterations of a non-linear transformation for enhancement of digital images, Pattern Recognition, 7 (1975).

[170] S. G. Krantz and H. R. Parks, The Geometry of Domains in Space, Birkäuser, 1999.

[171] A. S. Kronrod, On functions of two variables, Uspehi Matem. Nauk (N. S.), 5 (1950), pp. 24–134.

[172] P. D. Lax, Numerical solution of partial differential equations, Amer. Math. Monthly, 72 (1965), pp. 74–85.

[173] Y. H. Lee and S. A. Kassam, Generalized median filtering and related nonlinear filtering techniques, IEEE Trans. Acoust. Speech Signal Process., 33 (1985), pp. 672–683.

[174] R. Lenz, Rotation-invariant operators, in Proc. International Conference on Pattern Recognition, 1986, pp. 1130–1132.

[175] T. Lindeberg, Scale-Space Theory in Computer Vision, vol. 256 of Kluwer International Series in Engineering and Computer Science, Kluwer Academic Publishers, Dordrecht, NL, 1993.

[176] M. Lindenbaum, M. Fischer, and A. M. Bruckstein, On gabor’s contribution to image enhancement, Pattern Recognition, 27 (1994), pp. 1–8.

[177] P.-L. Lions, Axiomatic derivation of image processing models, Math. Models Methods Appl. Sci., 4 (1994).

[178] J.-L. Lisani, L. Moisan, P. Monasse, and J.-M. Morel, Affine invariant mathematical morphology applied to a generic shape recognition algorithm, in Proceedings of the International Symposium of Mathematical Morphology, Palo Alto, CA, 2000. Available at

http://pascal.monasse.free.fr/index.html.

[179] C. Liu, Z. Huan, and M. Tong, Uniqueness of the viscosity solutions for the initial value problem of one type of second order partial differential equation, J. Beijing Inst. Tech., Engl. Ed., 7 (1998), pp. 6–11.

BIBLIOGRAPHY 303

[180] A. Mackworth and F. Mokhtarian, Scale based description and recognition of planar curves and two-dimensional shapes, IEEE Trans. Patt. Anal. Mach. Intell., 8 (1986), pp. 34–43.

[181] A. K. Mackworth and F. Mokhtarian.

[182] J. Maeda, A characteristic property of space curves of constant first affine curvature, Tohoku Math. J., 48 (1941), pp. 148–151.

[183] R. Malladi, J. A. Sethian, and B. C. Vemuri, Shape modeling with front propagation: A level set approach, IEEE Trans. Patt. Anal. Mach. Intell., 17 (1995), pp. 158–175.

[184] P. Maragos, A representation theory for morphological image and signal processing, IEEE Trans. Patt. Anal. Mach. Intell., 11 (1989), pp. 586–599.

[185] P. Maragos, Differential morphology and image-processing, IEEE Trans. Image Process., 5 (1996), pp. 922–937.

[186] P. Maragos, Partial differential equations in image analysis: Continuous modeling, discrete processing, in Eusipco, 1998.

[187] P. Maragos and R. W. Schafer, Morphological filters. Part I: Their set-theoretic analysis and relations to linear shift-invariant filters, IEEE Trans. Acoust. Speech Signal Process., 35 (1987), pp. 1153–1169.

[188] P. Maragos and R. W. Schafer, Morphological filters. Part II: Their relations to median, order-statistic, and stack filters, IEEE Trans. Acoust. Speech Signal Process., 35 (1987), pp. 1170–1184.

[189] P. Maragos and R. D. Ziff, Threshold superposition in morphological image analysis systems, IEEE Trans. Patt. Anal. Mach. Intell., 12 (1990), pp. 498–504.

[190] D. Marr, Vision, W. H. Freeman and Company, New York, NY, 1982.

[191] D. Marr and E. Hildreth, Theory of edge detection, Proc. Roy. Soc. London Ser. B, 207 (1980), pp. 187–217.

[192] Mascarenhas, Diffusion generated motion by mean curvature, tech. rep., CAM Report 92–33, UCLA, 1992.

[193] S. Masnou, Image restoration involving connectedness, in Proceedings of the 6^{th} International Workshop on Digital Image Processing and Computer Graphics, 1998.

[194] G. Matheron, Random Sets and Integral Geometry, John Wiley & Sons, New York, NY, 1975.

[195] B. Merriman, J. Bence, and S. Osher, Diffusion generated motion by mean curvature, in Computational Crystal Growers Workshop. Proceedings of the Geometry Center Workshop held in Minneapolis, Minnesota, February 22–28, 1992., J. E. Taylor, ed., Selected Lectures in Mathematics, American Mathematical Society, 1992, pp. 73–83.

304 BIBLIOGRAPHY

[196] B. Merriman, J. Bence, and S. Osher, Motion of multiple junctions: a level set approach, Journal of Computational Physics, 112 (1994), pp. 334–363.

[197] L. Moisan, Affine plane curve evolution: A fully consistent scheme, IEEE Trans. Image Process., 7 (1998), pp. 411–420.

[198] P. Monasse, Représentation morphologique d’images numériques et application au recalage, PhD thesis, Ecole Normale Supérieure-Cachan, France, 2000. French and English versions available at

http://pascal.monasse.free.fr.

[199] P. Monasse and F. Guichard, Scale-space from a level lines tree, Journal of Visual Communication and Image Representation, 11 (2000),

pp. 224–236. Available at http://pascal.monasse.free.fr.

[200] J. L. Mundy, Object recognition: The search for representation, in ORCV95, 1995, pp. 19–50.

[201] J. L. Mundy and A. Zisserman, eds., Geometric Invariance in Computer Vision, The MIT Press, Cambridge, MA, 1992.

[202] J. L. Mundy, A. Zisserman, and D. Forsyth, eds., Applications of Invariance in Computer Vision, Lecture Notes in Computer Science 825, Springer-Verlag, 1994.

[203] M. Nagao and T. Matsuyama, Edge preserving smoothing, Computer Graphics Image Processing, 9 (1979), pp. 394–407.

[204] Y. Nakagawa and A. Rosenfeld, A note on the use of local min and max operations in digital image processing, IEEE Trans. Systems, Man and Cybernetics, 8 (1978), pp. 632–635.

[205] P. M. Narendra, Noise reduction by median filtering, in IEEE Conf. Pattern Recogn. Image Processing, 1978.

[206] M. H. A. Newman, Elements of the Topology of Plane Sets of Points, Cambridge U. Press, 1951.

[207] W. J. Niessen, B. M. ter Haar Romeny, L. M. J. Florack, and M. A. Viergever, A general framework for geometry-driven evolution equations, International Journal of Computer Vision, 21 (1997), pp. 187– 205.

[208] E. Ochoa, J. P. Allebach, and D. W. Sweeney, Optical median filtering by threshold decomposition, Appl. Opt., 26 (1987), pp. 252–260.

[209] P. J. Olver, G. Sapiro, and A. Tannenbaum, Classification and uniqueness of invariant geometric flows, C. R. Acad. Sci. Paris Sér. I, 319 (1994), pp. 339–344.

[210] P. J. Olver, G. Sapiro, and A. Tannenbaum, Differential invariant signatures and flows in computer vision: A symmetry group approach, in Geometry Driven Diffusion in Computer Vision, Kluwer, 1994.

BIBLIOGRAPHY 305

[211] P. J. Olver, G. Sapiro, and A. Tannenbaum, Affine invariant gradient flows, in 12th international conference on analysis and optimization of Systems:Images, Wavelets and PDE’s, Springer-Verlag, 1996.

[212] P. J. Olver, G. Sapiro, and A. Tannenbaum, Invariant geometric evolutions of surfaces and volumetric smoothing, SIAM J. Appl. Math., 57 (1997), pp. 176–194.

[213] S. Osher and L. I. Rudin, Feature-oriented image enhancement using shock filters, SIAM J. Numer. Anal., 27 (1990), pp. 919–940.

[214] S. Osher and J. Sethian, Fronts propagating with curvature-dependent speed: Algorithms based on the Hamilton–Jacobi formulation, J. Comput. Phys., 79 (1988), pp. 12–49.

[215] H. C. Park and R. T. Chin, Decomposition of arbitrarily-shaped morphological structuring elements, IEEE Trans. Patt. Anal. Mach. Intell., 17 (1995), pp. 2–15.

[216] D. Pasquignon, Computation of skeleton by PDE, in International Conference in Image Processing, IEEE, ed., 1995.

[217] D. Pasquignon, Approximation of viscosity solution by morphological filters, in ESAIM COCV, vol. 4, 1999.

[218] E. J. Pauwels, L. J. VanGool, P. Fiddelaers, and T. Moons, An extended class of scale-invariant and recursive scale space filters, IEEE Trans. Patt. Anal. Mach. Intell., 17 (1995), pp. 691–701.

[219] S. C. Pei, C. L. Lai, and F. Y. Shih, An efficient class of alternating sequential filters in morphology, Graphical Models and Image Processing, 59 (1997), pp. 109–116.

[220] S. S. Perlman, S. Eisenhandler, P. W. Lyons, and M. J. Shumila, Adaptive median filtering for impulse noise elimination in real-time TV signals, IEEE Trans. Communications, 35 (1987), pp. 646–652.

[221] P. Perona and J. Malik, Scale space and edge detection using anisotropic diffusion, in Proc. Workshop on Computer Vision, Representation and Control, IEEE Computer Society, IEEE Press, Piscataway, NJ, 1987, pp. 16–22.

[222] P. Perona and J. Malik, Scale space and edge detection using anisotropic diffusion, IEEE Trans. Patt. Anal. Mach. Intell., 12 (1990), pp. 629–639.

[223] F. Pollick and G. Sapiro, Constant affine velocity predicts the 1/3 power law of planar motion perception and generation, Vision Research, 37 (1997), pp. 347–353.

[224] M. Rahmati and L. G. Hassebrook, Intensity-invariant and distortion-invariant pattern-recognition with complex linear morphology, Pattern Recognition, 27 (1994), pp. 549–568.

306 BIBLIOGRAPHY

[225] L. Rawtani, Image smoothing by median filtering – an efficient algorithm, J.M.A.C.T., 28 (1995), pp. 93–101.

[226] C. S. Regazzoni and A. Teschioni, A new approach to vector median filtering based on space-filling curves, IEEE Trans. Image Process., 6 (1997), pp. 1025–1037.

[227] C. H. Richardson and R. W. Schafer, A lower bound for structuring element decompositions, IEEE Trans. Patt. Anal. Mach. Intell., 13 (1991), pp. 365–369.

[228] F. Riesz and B. Sz.-Nagy, Functional Analysis, Frederick Ungar Publishing Co., New York, 1955.

[229] C. Ronse and H. J. A. M. Heijmans, The algebraic basis of mathematical morphology: II. openings and closings, Computer Vision, Graphics, and Image Processing, 54 (1991), pp. 74–97.

[230] L. I. Rudin, Images, Numerical Analysis of Singularities and Shock Filters, PhD thesis, California Institute of Technology, Pasadena, CA, 1987.

[231] L. I. Rudin, S. Osher, and E. Fatemi, Nonlinear total variation based noise removal algorithms, Physica D: Nonlinear Phenomena, 60 (1992), pp. 259–268.

[232] W. Rudin, Real and Complex Analysis. Third edition, McGraw-Hill Book Co., New York, 1987.

[233] A. H. Salden, B. M. ter Haar Romeny, and M. Viergever, Affine and projective differential geometric invariants of space curves, in Geometric Methods in Computer Vision II, B. Vemuri, ed., 1993, pp. 60–74.

[234] P. Salembier, Structuring element adaptation for morphological filters, Journal of Visual Communication and Image Representation, 3 (1992), pp. 115–136.

[235] P. Salembier and J. Serra, Flat zones filtering, connected operators and filters by reconstruction, IEEE Trans. Image Process., 8 (1995).

[236] G. Sapiro, A. Cohen, and A. M. Bruckstein, A subdivision scheme for continuous scale B-splines and affine invariant progressive smoothing, J. Math. Imaging Vision, 7 (1997), pp. 23–40.

[237] G. Sapiro, R. Kimmel, D. Shaked, B. Kimia, and A. M. Bruckstein, Implementing continuous-scale morphology via curve evolution, Pattern Recognition, 9 (1993), pp. 1363–1372.

[238] G. Sapiro and A. Tannenbaum, Affine invariant scale space, International Journal of Computer Vision, 11 (1993), pp. 25–44.

[239] G. Sapiro and A. Tannenbaum, On invariant curve evolution and image analysis, Indiana Univ. Math. J., 42 (1993), pp. 985–1009.

[240] G. Sapiro and A. Tannenbaum, On affine plane curve evolution, J. Funct. Anal., 119 (1994), pp. 79–120.

BIBLIOGRAPHY 307

[241] D. Schonfeld, Optimal structuring elements for the morphological pattern restoration of binary images, IEEE Trans. Patt. Anal. Mach. Intell., 16 (1994), pp. 589–601.

[242] J. Serra, Image Analysis and Mathematical Morphology, Academic Press, San Diego, CA, 1982.

[243] J. Serra, Introduction to mathematical morphology, Computer Vision, Graphics, and Image Processing, 35 (1986), pp. 283–305.

[244] J. Serra, Image Analysis and Mathematical Morphology. Volume2: Theoretical Advances, Academic Press, 1988.

[245] J. Sethian, An analysis of flow propagation, PhD thesis, UCLA, 1982.

[246] J. A. Sethian, Curvature and the evolution of fronts, Comm. Math. Phys., 101 (1985).

[247] F. Y. Shih and O. R. Mitchell, A mathematical morphology approach to Euclidean distance transformation, IEEE Trans. Image Process., 1 (1992), pp. 197–204.

[248] N. Sochen, R. Kimmel, and R. Malladi, A general framework for low-level vision, IEEE Trans. Image Process., 7 (1998), pp. 310–318.

[249] P. Soille, E. J. Breen, and R. Jones, Recursive implementation of erosions and dilations along discrete lines at arbitrary angles, IEEE Trans. Patt. Anal. Mach. Intell., 18 (1996), pp. 562–567.

[250] D. Struik, Differential Geometry, Addison-Wesley, 1950.

[251] T. Sun and Y. Neuvo, Detail-preserving median based filters in imageprocessing, Pattern Recognition Letters, 15.

[252] I. Svalbe and R. Jones, The design of morphological filters using multiple structuring elements, part I: Openings and closings, Pattern Recognition Letters, 13 (1992), pp. 123–129.

[253] H. D. Tagare and R. J. P. deFigueiredo, Order filters, Proceedings of IEEE, 73 (1985), pp. 163–165.

[254] M. E. Taylor, Partial Differential Equations: Basic Theory, Springer, New York, 1996.

[255] B. M. ter Haar Romeny, ed., vol. 1 of Computational Imaging and Vision, Kluwer Academic Publishers, 1994.

[256] B. M. ter Haar Romeny, Front-End Vision and Multiscale Image Analysis: Introduction to Scale-Space Theory, Kluwer Academic Publishers, Dordrecht, NL, In preparation.

[257] B. M. ter Haar Romeny, L. M. J. Florack, J. J. Koenderink, and M. A. Viergever, Invariant third order properties of isophotes: Tjunction detection, in Proc. 7th Scand. Conf. on Image Analysis, Aalborg, DK, 1991, pp. 346–353.

308 BIBLIOGRAPHY

[258] A. Toet, A morphological pyramidal image decomposition, Pattern Recognition Letters, 9 (1989), pp. 255–261.

[259] A. Toet, A hierarchical morphological image decomposition, Pattern Recognition Letters, 11 (1990), pp. 267–274.

[260] R. van den Boomgaard and A. Smeulders, The morphological structure of images: The differential equations of morphological scale-space, IEEE Trans. Patt. Anal. Mach. Intell., 16 (1994), pp. 1101–1113.

[261] L. J. VanGool, T. Moons, E. Pauwels, and A. Oosterlinck, Semidifferential invariants, in Geometric Invariance in Computer Vision, J. L. Mundy and A. Zisserman, eds., The MIT Press, 1992, pp. 157–192.

[262] B. Vasselle, G. Giraudon, and M. Berthod, Following corners on curves and surfaces in the scale space, in Proc. Third European Conference on Computer Vision, 1994, pp. 109–114.

[263] L. Vincent, Grayscale area openings and closings, their efficient implementation and applications, in Proceedings of the 1^{st} Workshop on Mathematical Morphology and its Applications to Signal Processing, J. Serra and P. Salembier, eds., Barcelona, Spain, 1993, pp. 22–27.

[264] J. von Neumann and O. Morgenstern, Theory of Games and Economic Behavior, Princeton U. Press, Princeton, NJ, 1947.

[265] J. Weickert, Theoretical foundations of anisotropic diffusion in image processing, Computing Suppl., 11 (1996), pp. 221–236.

[266] J. Weickert, Anisotropic Diffusion in Image Processing, ECMI Series, Teubner-Verlag, Stuttgart, 1998.

[267] J. Weickert and B. Benhamouda, A semidiscrete nonlinear scalespace theory and its relation to the Perona–Malik paradox, in Advances in Computer Vision, F. Solina, W. G. Kropatsch, R. Klette, and R. Bajcsy, eds., Springer, 1997, pp. 1–10.

[268] J. Weickert, S. Ishikawa, and A. Imiya, On the history of Gaussian scale-space axiomatics, in Gaussian Scale-Space Theory, Kluwer, 1997, pp. 45–59.

[269] J. Weickert, S. Ishikawa, and A. Imiya, Linear scale-space has first been proposed in japan, J. Math. Imaging Vision, 10 (1999), pp. 237–252.

[270] I. Weiss, Local projective and affine invariants, AMAI, 13 (1995), pp. 203–225.

[271] P. D. Wendt, E. J. Coyle, and N. C. Gallagher, Jr., Stack filters, IEEE Trans. Acoust. Speech Signal Process., 34 (1986), pp. 898–911.

[272] M. Wertheimer, Untersuchungen zur Lehre der Gestalt, II, Psychologische Forschung, 4 (1923), pp. 301–350.

[273] J. Wilburn, Developments in generalized ranked-order filters, Journal of the Optical Society of America, 15 (1998), pp. 1084–1099.

BIBLIOGRAPHY 309

[274] A. P. Witkin, Scale-space filtering, in Proceedings of the International Joint Conference on Artificial Intelligence, Karlsruhe, Germany, 1983, Morgan Kaufmann Publishers, San Francisco, pp. 1019–1021.

[275] L. Wu and Z. Xie, Scaling theorems for zero-crossings, IEEE Trans. Patt. Anal. Mach. Intell., 12 (1990), pp. 46–54.

[276] O. Yli-Harja, J. Astola, and Y. Neuvo, Analysis of the properties of median and weighted median filters using threshold logic and stack filter representation, IEEE Trans. Signal Process., 39 (1991), pp. 395–410.

[277] A. L. Yuille, The creation of structure in dynamic shape, in Proceedings Second International Conference on Computer Vision, 1988, pp. 685–689.

[278] A. L. Yuille and T. A. Poggio, Fingerprint theorems for zerocrossings, J. Opt. Soc. Amer., 2 (1985), pp. 683–692.

[279] A. L. Yuille and T. A. Poggio, Scaling theorems for zero-crossings, IEEE Trans. Patt. Anal. Mach. Intell., 8 (1986), pp. 15–25.

[280] R. Zhu, Existence uniqueness and regularity for viscosity solutions of fully nonlinear parabolic equations under natural structure conditions. (Chinese. English summary), J. Beijing Norm. Univ., Nat. Sci., 30 (1994), pp. 290–300.

[281] X. Zhuang, Decomposition of morphological structuring elements, J. Math. Imaging Vision, 4 (1994), pp. 5–18.

[282] X. Zhuang and R. M. Haralick, Morphological structural element decomposition, Computer Vision, Graphics, and Image Processing, 35 (1986), pp. 370–382.