Motivations ○

Mathematics of Rainfall Images ○○○○

(GSM) Estimation and Fusion ○○○○

Sparse Downscaling and Retrieval ○○○○○○

Adaptive Fusion and Sparse Estimation of Multi-sensor Precipitation

Ardeshir Mohammad Ebtehaj 1,2 1

Efi Foufoula-Georgiou 1

Department of Civil Engineering 2 School of Mathematics

Summary ○

Motivations ○

Mathematics of Rainfall Images ○○○○

(GSM) Estimation and Fusion ○○○○

Sparse Downscaling and Retrieval ○○○○○○

Summary ○

Outline

❶ Motivations and Goals  Multi-sensor Multi-scale estimation, i.e., Downscaling, Fusion and Retrieval of Precipitation

❷ Mathematical Structure of Precipitation Images  Statistics of Precipitation in Transform Domain, Sparsity in the Wavelet Domain

❸ Non-linear Estimation and Multi-sensor Precipitation Fusion

(Bayesian Approach)

 Gaussian Scale Mixture, Localized Estimation and Fusion in the Wavelet Domain ❹ Downscaling and Multi-sensor Retrieval via Sparse Representation  Sparse Inverse Estimator, Downscaling (SPaD) and Retrieval

Motivations •

Mathematics of Rainfall Images ○○○○

(GSM) Estimation and Fusion ○○○○

Sparse Downscaling and Retrieval ○○○○○○

Summary ○

Motivations and Goals

o State-of-the-art mathematical development for downscaling, multi-sensor Fusion and retrieval?

Motivations •

Mathematics of Rainfall Images •○○○

(GSM) Estimation and Fusion ○○○○

Sparse Downscaling and Retrieval ○○○○○○

Summary ○

Rainfall in Transform Domains, Why?

o Transform Domain Representation 𝐱=

𝑛

𝐜𝑖 𝜙𝑖

𝑖=1

e.g.

𝜙𝑖 =

1 𝑁

𝑒 𝑖𝑘𝑛/𝑁

Compact

𝐱 = 𝜙1 |𝜙2 | … |𝜙𝑛 𝐜 = 𝚽𝐜

o Why Transform Domain?

General: 

Efficiency (e.g. Diagonalizing convolution in the Fourier)



Lower dimension due to sparsity of transform coefficients (wavelet)



Decorrelation Capacity (wavelet and Fourier)

Particular:  Multiscale representation  Tractable a priori information (Sparsity!)

Motivations ○

Mathematics of Rainfall Images ••○○

(GSM) Estimation and Fusion ○○○○

Sparse Downscaling and Retrieval ○○○○○○

Summary ○

Data HSTN MELB

14 12 10 8 6 4 2 0

Dataset MELB_19980217_131700

a)

HSTN_19981113_000200

b)

200 independent reflectivity images over two GV sites of the TRMM , Houston, TX. (HSTN) and Melbourne, FL. (MELB)

HSTN_19981113_000200

c)

MELB_20040926_045000

Data structure

d)

• NEXRAD Level III base Reflectivity • 2A25 Orbital near-surface Reflectivity • A Wide range of Strom Types.

Motivations ○

Mathematics of Rainfall Images •••○

(GSM) Estimation and Fusion ○○○○

Sparse Downscaling and Retrieval ○○○○○○

Summary ○

Sparsity in the Wavelet Domain 4

5

a) a)

x 10 x 10 5

4

45 dBZ 45 dBZ

b) b)

4

(V)

(V)

(D)

c)

4

(D)

Dim: Dim: 2[450,600]; Range: [0,45][0,45] 2[450,600]; Range:

3

2

Frequency

(H)

Frequency

(H)

c)

1

0 -30

3

Top

40

40

35

35

30

30

25

25

20

20

15

15

10

10

5

5

0

0

2

20% 1

0 -20 -30

-10 -20

-100

wH

010

wH

20 10

30 20

30

Dim: Dim: [450,600]; Range: [0,45][0,45] [450,600]; Range:

Sparsity of precipitation images in the wavelet domain (a) Sparsity in the wavelet domain (b) Symmetric Distribution with thicker tail than the Gaussian case

(c) Reconstruction by only top 20% of the coefficients (99.8% of the energy is preserved)

Motivations ○

Mathematics of Rainfall Images ••••

(GSM) Estimation and Fusion ○○○○

Sparse Downscaling and Retrieval ○○○○○○

Summary ○

Decorrelation in the Wavelet Domain a)

Horizontal

b)

Vertical

c)

Diagonal

d)

e)

3

5

5

5

10

10

10

2

15

15

15

1.5

20

20

20

25

25

25

2.5

1 0.5 size: [544, 624]

size: [544,624]

5

10

15

20

25

5

10

15

20

25

Local covariance characterization Local covariance of the rainfall wavelet coefficients for a neighborhood of size 5x5 (diagonally dominant)!

Why wavelet for precipitation images? o A well behaved Symmetric Probability Model o Decorrelation capacity ( Karhunen Loéve-like expansion)

o Covariance matrices can be computed locally for neighborhood of coefficients o Only few coefficients explain the rainfall energy o Exploiting of linear estimation theory in an adaptive way !

5

10

15

20

25

Motivations •

Mathematics of Rainfall Images ••••

(GSM) Estimation and Fusion •○○○

Sparse Downscaling and Retrieval ○○○○○○

Summary ○

Gaussian Scale Mixture (GSM) a Probability Model

Generalized Gaussian Distribution GGD spans a symmetric probability continuum ranging from a Dirac delta (𝜶 → 𝟎) to the uniform density (𝜶 → ∞) .

𝑥 𝑓𝑥 𝑥, 𝜃 ∝ 𝐞𝐱𝐩 − 𝑠

𝛼

Problem! A Gaussian Scale Mixture probability model

0 GSM Simulation Fitted GG density

Gaussian Density

A large family of elliptically symmetric density functions with thicker tail than the Gaussian

-10

𝒅 =𝑑 𝑧𝒖 𝚺𝒅 = 𝔼 𝑧 𝚺 𝒖

log(p)

case (e.g., Laplace, Double exponential, etc.)

-5

𝚺𝒅|𝒛 = 𝑧𝚺𝒖

Log-normal Multiplier -15 -15

-10

-5

0 m=1

e.g., Andrews and Mallows (1974), Wainwright et al., (2001)

dH

5

10

15

Motivations ○

Mathematics of Rainfall Images ○○○○

(GSM) Estimation and Fusion ••○○

Sparse Downscaling and Retrieval ○○○○○○

Summary ○

GSM Optimal Estimation (theory)

min 𝐱 − 𝐲 𝐲

2 2

Spatial Domain x

𝐞 = 𝕍𝒂𝒓[𝐱|𝐲]

𝐱=

𝚺𝐱𝐲 𝐱, 𝐲 𝔼[𝐱𝐲] 𝐲= 𝐲 = 𝐲 𝐲, 𝐲 𝔼[𝐲 2 ] 𝚺𝐲𝟐

𝐲 = 𝐻𝐱 + 𝑣 𝐱 −𝒎𝐱 = 𝚺𝐱 𝐻T 𝐻𝚺𝐱 𝐻𝑇 + 𝚺𝑣 𝐻=𝐼 𝒎𝐱 = 𝒎𝐲 = 0

Wavelet Domain Wavelet coefficients of a neighborhood

𝐲=𝐝+𝑣

( 𝐲 −𝒎𝐲 )

𝐱 = 𝚺 𝐱 𝚺 𝐱 + 𝚺𝑣

−𝟏

𝐲

GSM Wiener

𝐝 = 𝑧𝚺𝐮 𝑧𝚺𝐮 + 𝚺𝑣

𝐝= 𝑧𝐮

−1

−𝟏 𝐲

Maximum A Posteriori (MAP) estimation of z 𝑧𝑀𝐴𝑃 = 𝐚𝐫𝐠 𝐦𝐚𝐱 log 𝒑 𝒚 𝑧 + log 𝒑(𝑧) 𝑧

log 𝑧𝑀𝐴𝑃 + 3𝜎𝑧2 2 1 + 𝑧𝑀𝐴𝑃 𝜎𝑧2 2

𝑁

𝑛=1

2 𝑧𝑀𝐴𝑃 − 𝜆−1 𝑛 (𝑣𝑛 − 1) =0 2 𝑧𝑀𝐴𝑃 + 𝜆−1 𝑛

where: 𝑆𝑆 𝑇 = Σ𝑣 Λ, Q = 𝐞𝐢𝐠 𝑆 −1 Σ𝒖 𝑆 −𝑇 𝑣𝑛 ∈ V = 𝑄 𝑇 𝑆 −1 𝒚 Portilla et al., 2003, Ebtehaj A.M., and E. Foufoula-Georgiou 2011b

Motivations ○

Mathematics of Rainfall Images ○○○○

(GSM) Estimation and Fusion •••○

Sparse Downscaling and Retrieval ○○○○○○

GSM Optimal Estimation (1D-Results)

ID implementation of the GSM Wiener (a) A heavy tail noisy signal (similar to rainfall images in marginal statistics) (b) Standard Gaussian Filtering results (c) GSM Wiener filter

Summary ○

Motivations ○

Mathematics of Rainfall Images ○○○○

(GSM) Estimation and Fusion ••••

Sparse Downscaling and Retrieval ○○○○○○

Summary ○

GSM Framework for Fusion of Precipitation Images

Filtering and Fusion in the wavelet domain

Inverse WT

Lateral Projection Filtered low-pass

Filtered high-pass

TRMM-NEX Error Covaraince 𝚺𝐯𝟑

Filtered high-pass SRE Product [1 1] km

a)

Fused Products Size:[224, 784]; Range:[0, 49] dBZ GSM-Fusion [1 1] km

c)

TRMM Error Covariance 𝚺𝐯𝟏

NEXRAD Error Covariance 𝚺𝐯𝟐

Size:[224, 784]; Range:[0, 50] dBZ 50

Vertical Filtering rain-gauge error

Reflectivity [dBZ]

c)

Reduced Uncertainty!

GSM -Fusion SRE

40

NEXRAD TRM M

30 20 10 0

0

100

200 300 400 E. Foufoula-Georgiou 500 600 7002011b800 Ebtehaj A.M., and

Motivations •

Mathematics of Rainfall Images ••••

(GSM) Estimation and Fusion ••••

Sparse Downscaling and Retrieval •○○○○○

Summary ○

Sparse Approximation and Compressive Sensing

Problem

1

𝐲 = A𝐱 + 𝐞

0.5

where,

𝐲 ∈ ℝ𝑛 (n=120 !) 𝐱 ∈ ℝ𝑚 (m=512) 𝐀 ∈ ℝ𝑛×𝑚 𝐀 : Gaussian Orthogonal Ensemble

x

0 -0.5

-1 0

50

100

150

200

250

300

350

400

450

500

node

L2 Recovery

x

2

0.5

𝐦𝐢𝐧 𝐱 0

-0.5 0

50

100

150

200

250

300

350

400

450

𝟐

s. t.

2<

𝐞

2

?

500

node

A𝐱 − 𝐲

𝐦𝐢𝐧 𝐱

𝟎

s. t.

A𝐱 − 𝐲

2<

𝐞

2

2<

𝐞

2

1

L1 Recovery

x

1

0.5

𝐦𝐢𝐧 𝐱

0

𝟏

s. t.

A𝐱 − 𝐲

-0.5

-1 0

50

100

150

200

250

node

300

350

400

450

500

Candes and Tao (2006), Donoho (2006), Elad 2010

Motivations •

Mathematics of Rainfall Images ••••

(GSM) Estimation and Fusion ••••

Sparse Downscaling and Retrieval ••○○○○

Summary ○

Sparse Precipitation Downscaling (SPaD)

Ideas! (a) Optimal recovery of the high-resolution rainfall image from a low-resolution counterpart by Exploiting Sparsity ?

(b) Learning to downscale from a series of coincidental low and high-resolution rainfall images ? (c) Linear approximation of sufficiently small scale local neighborhoods of rainfall fluctuations? 𝐯𝑙 𝑛× 𝑛

=

𝐜1 𝐯𝑙1

nx1

+

𝐜2 𝐯𝑙2

nx1

+

𝐜3 𝐯𝑙3 nx1

nx1

Dim: [29 38]

𝐯ℎ 𝑠𝑛 × 𝑠𝑛

= snx1

Dim: [450 600]

𝐜1 𝐯ℎ1

+ snx1

𝐜2 𝐯ℎ2

+ snx1

Scaling Ratio (s)

𝐜3 𝐯ℎ3 snx1

Motivations •

Mathematics of Rainfall Images ••••

(GSM) Estimation and Fusion ••••

Sparse Downscaling and Retrieval •••○○○

Summary ○

SPaD Implementation 45

a) 40

Analysis

Low Resolution Dictionary

35 30 25 20

Wavelet

15 10 5

Dim: [29,38]; Range: [0,40]

𝐲

𝚿=

0

𝑛×𝑀 Dim: [29 38]

𝐜 = argmin 𝝀 𝐜 c

1 𝟏+ 2

𝐲 − 𝚿𝐜

𝟐 𝟐

High Resolution Dictionary

𝐱 𝒉 = 𝚽𝐜

b)

𝚽=

𝐱 𝐱 = 𝐱𝒍 + 𝐱𝒉 Synthesis Dim: [464,608]; Range: [0,45]

Dim: [450 600]

𝑠𝑛 × 𝑀

Motivations •

Mathematics of Rainfall Images ••••

(GSM) Estimation and Fusion ••••

Sparse Downscaling and Retrieval ••••○○

Summary ○

SPaD Results

Smoothed and Downsampled by factor of 16x16 45

s=16

a) 40 35 30 25 20 15 10 5

Dim: [29,38]; Range: [0,40]

0

Observation (1x1 km)

SPaD Results (1x1 km)

dBZ 45

b)

c)

40 35

Results

30 25

Structural similarity is almost perfect !

20

MSE is less than 1% of image energy

15 10 5

Dim: [464,608]; Range: [0,45]

Dim: [464,608]; Range: [0,45]

0

Motivations ○

Mathematics of Rainfall Images ○○○○

(GSM) Estimation and Fusion ○○○○

Sparse Downscaling and Retrieval •••••○

SPaD Results

4X4 km

8X8 km

dBZ 45

a)

40

16X16 km c)

b)

35 30 25 20 15 10 5

Dim: [113,150]; Range: [0,44]

0

Dim: [57,75]; Range: [0,41]

Dim: [29,38]; Range: [0,40]

1X1 km

1X1 km

1X1 km e)

d)

Dim:[452,600]; Range:[0,48]

f)

Dim:[456,600]; Range:[0,47]

Dim: [464,608]; Range: [0,45]

NBR (%)

MSE (%)

MAE(%)

PSNR

<0.5%

50%

25%

2-3 dB

Summary ○

Motivations •

Mathematics of Rainfall Images ••••

(GSM) Estimation and Fusion •••••

Sparse Downscaling and Retrieval ••••••

Summary ○

Future Research and Perspectives

A New Multi-sensor Precipitation Retrieval (Combined PR and Radiometer)

𝚿

𝚿: Dictionary of TMI (GMI) Brightness Temperature 𝚽: Dictionary of PR (DPR) reflectivity

𝚽

Advantages: • Taking into account the spatial coherence • Retrieval via sparse promoting convex Optimization • Separation of Low and High-pass components Sparse Data Assimilation

min 𝐱 − 𝐲 𝐲

2 2

Orthogonal Projection (e.g., Kalman Filter) • Gaussianity (linear subspace) • Over-determined systems • Error as a priori ! x

min 𝝀 𝐓𝐯 𝐱

1

+ 1/2 𝐱 − 𝐲

2 2

Projection Over any Convex Set • Non-Gaussianity • Under-determined systems • Error not be know necessarily ! x

Motivations •

Mathematics of Rainfall Images ••••

(GSM) Estimation and Fusion ••••

Sparse Downscaling and Retrieval ••••••

 Why Transform domain?

 Gaussian Scale Mixtures (GSM) for Multi-scale Multi-sensor Fusion !

 Sparsity in the wavelet domain and SPaD !

 Opportunities for new retrieval and data assimilation schemes !

Summary •

Motivations ○

Mathematics of Rainfall Images ○○○○

(GSM) Estimation and Fusion ○○○○

Sparse Downscaling and Retrieval ○○○○○○

Summary ○

Acknowledgments •

The research has been mainly supported by: NASA-GPM awards NNX07AD33G and a UMN Interdisciplinary Doctoral fellowship (MATH-CIVIL).

•

Thanks also go to Arthur Hou and Sara Zhang for their interests and supports.

Publications 1- Ebtehaj A.M., and E. Foufoula-Georgiou (2010), Orographic signature on multiscale statistics of extreme rainfall: A storm-scale study, JGR-A

2- Ebtehaj A.M., and E.Foufoula-Georgiou (2011), Statistics of precipitation reflectivity images and cascade of Gaussian‐scale mixtures in the wavelet domain: A formalism for reproducing extremes and coherent multiscale structures , JGR-A 3- Ebtehaj A. M., and E. Foufoula-Georgiou (2011), Adaptive fusion of multisensor precipitation using Gaussian-scale mixtures in the wavelet domain, JGR-A 4- Ebtehaj A.M., E.foufoula-Georgiou, G.Lerman (2011), Sparse Precipitation Downscaling, JGR-A, Under Review

Please visit our website: www.ce.umn.edu/~foufoula and http://umn.edu/home/ebteh001 for released MATLAB codes.

Thank you! Standards are always out of date. That is what makes them standards A. Bennett