EM X IE Introduction Overview Applications
Specification Estimate Maximize
Gaussian E-Step M-Step
Expectation-Maximization Algorithm in Image Segmentation
Results Reference
Shuisheng Xie
March 15, 2007
EM
Overview
X IE Introduction Overview Applications
Specification Estimate Maximize
Gaussian E-Step M-Step
Results Reference
Finding maximum likelihood estimates of parameters in probabilistic models, where the model depends on unobserved latent variables. Two major steps: E step: Computes an expectation of the likelihood by including the latent variables as if they were observed. M step: Computes the maximum likelihood estimates of the parameters by maximizing the expected likelihood found on the E step.
EM
Applications of E-M Algorithm
X IE Introduction Overview Applications
Specification Estimate Maximize
Gaussian E-Step M-Step
Results Reference
Machine Learning Computer Vision and Image Processing Psychometrics Portfolio
EM
Specification of the EM Procedure
X IE Introduction Overview Applications
Definition of the data 1
Y:Incomplete data consisting of values of observable variable
2
X:Missing data
3
(X,Y): Complete data
Specification Estimate Maximize
Gaussian E-Step M-Step
Results Reference
Definite Integral 1
Estimate the unobservable data
2
Maximize expected log-likelihood for the complete dataset
EM X IE Introduction Overview Applications
Specification Estimate Maximize
Gaussian E-Step M-Step
Results Reference
Estimate the unobservable data p(y|x, θ)p(x|θ) p(y, x|θ) =R p(y|θ) p(y|x, θ)p(x|θ)dx For the conditional distribution of the missing data given the observed: the observation likelihood given the unobservable data p(y|x, θ); the probability of the unobservable data p(x|θ).
p(x|y, θ) =
EM X IE
Maximize expected log-likelihood for the complete dataset
Introduction Overview Applications
Specification Estimate Maximize
Gaussian E-Step M-Step
Results Reference
Iteratively improve an initial estimate θ0 and construct new estimates θ1 , . . . , θn , . . . . i h θn+1 = arg max Ex log p (y, x | θ) y θ
Log-likelihood is often used instead of true likelihood. θn+1 : The value that maximizes (M) the conditional expectation (E) of the complete data log-likelihood given the observed variables under the previous parameter value.
EM
Mixture Gaussian
X IE Introduction Overview Applications
Specification Estimate Maximize
Gaussian E-Step M-Step
Results Reference
The samples y1 , . . . , ym , are drawn from the gaussians x1 , . . . , xn P(y|xi , θ) = N (µi , σ i) = 1 −1/2 T −1 −l/2 (2π) |σi | exp − (y − µi ) σi (y − µi ) 2 The model you are trying to estimate: θ = {µ1 , . . . , µn , σ1 , . . . , σn , P(x1 ), . . . , P(xn )}
EM
E-Step
X IE Introduction Overview Applications
Specification Estimate Maximize
Gaussian E-Step M-Step
Results Reference
Estimation for unobserved event (which Gaussian is used),conditioned on the observation, using the values from the last maximization step: P(xi |yj , θt ) p(xi , yj |θt ) = p(yj |θt ) p(yj |xi , θt )P(xi |θt ) = Pn k=1 p(yj |xk , θt )P(xk |θt )
EM
M-Step
X IE Introduction Overview Applications
Specification Estimate Maximize
Q(θ) = Ex ln
m Y
p (yj , x|θ) yj
j=1
Gaussian E-Step M-Step
Results
m X = Ex ln p (yj , x|θ) yj j=1
Reference
=
=
m X
i h Ex ln p (yj , x|θ) yj
j=1 m X n X j=1 i=1
P (xi |yj , θt ) ln p (xi , yj |θ)
EM X IE
µi , σi , P(xi |θ) Pm
Introduction Overview Applications
1
j=1 µi = Pm
j=1 P(xi |yj , θt )
Pm
Specification Estimate Maximize
2
σi =
3
P(xi |θ)
Gaussian E-Step M-Step
Results Reference
P(xi |yj , θt )yj
j=1 P(xi |yj , θt )(yj − µi )(yj − Pm j=1 P(xi |yj , θt ) Pm j=1 P(xi |yj , θt ) = Pn Pm k=1 j=1 P(xk |yj , θt )
µi )T
EM X IE Introduction Overview Applications
Specification Estimate Maximize
Gaussian E-Step M-Step
Results Reference
Experimental Results
EM X IE Introduction Overview Applications
Specification Estimate Maximize
Gaussian E-Step M-Step
Results Reference
Experimental Results
EM X IE Introduction Overview Applications
Specification Estimate Maximize
Gaussian E-Step M-Step
Results Reference
Experimental Results
EM X IE Introduction Overview Applications
Specification Estimate Maximize
Gaussian E-Step M-Step
Results Reference
Experimental Results
EM X IE Introduction Overview Applications
Specification Estimate Maximize
Gaussian E-Step M-Step
Results Reference
Experimental Results
EM X IE Introduction Overview Applications
Specification Estimate Maximize
Gaussian E-Step M-Step
Results Reference
Experimental Results
EM
Reference
X IE Introduction Overview Applications
Specification Estimate Maximize
Gaussian E-Step M-Step
Results Reference
http://en.wikipedia.org/wiki/Expectationmaximization algorithm Xenophon Papademetris, Pavel Shkarin, Lawrence H. Staib, Kevin L. Behar: Regional Whole Body Fat Quantification in Mice. IPMI 2005: 369-380
EM X IE Introduction Overview Applications
Specification Estimate Maximize
Gaussian E-Step M-Step
Results Reference
