Page 233 - Special Topic Session (STS) - Volume 1

STS426 Tanuka C.
(d) Continue iterating (each pair of E & M steps is considered one
iteration) between E-Step and M-Step until convergence is
attained after which an observation can be assigned to the group
for which the corresponding posterior probability is the highest.
Under assumption, the MLE (Maximum-Likelihood Estimate)
method is generally used to check for the convergence of the above
EM. This involves computation of the log-likelihood after each
iteration and stopping the process when there appears to be no
significant change from one iteration to the next. The log-likelihood
is defined as follows:

log (Θ) = ∑ log (∑ ( | , ∑ )) (10)

=1 =1

where, (.) is the Multivariate Gaussian Density.
The results of the above discussed EM are highly dependent on
the initial values. Model-based Hierarchical Clustering can often be a
good source of initial values for datasets that are moderate in size (?;
?; ?).
The EM solution driven by MLE can fail to converge; instead it can
diverge to a point of infinite likelihood. For many mixture models, the
likelihood is unbounded and there are paths in parameter space
along which the likelihood tends to infinity (?). For examples of such
an instance, we refer to the paper by ?. To avoid this instance the
failure of convergence can be tackled by replacing the ML-Estimate
by the maximum a posteriori (MAP) estimate from a Bayesian
analysis. This can be achieved by assuming a prior distribution on the
parameters that eliminates failure due to singularity; while having
little effect on the stable results obtainable without any prior
assumption. Under such setup, the Bayesian predictive density for the
data is assumed to be of the form:

ℒ(| , , ∑ ) ( , , ∑ |) (11)

where ℒ is the mixture likelihood and is given by:

ℒ(| , , ∑ ) = ∏ ∑ ( | , ∑ )

= =
exp {−1/2( − ) ∑ ( − )} (12)
−1

≡ ∏ ∑ det (2∑ ) −1/2
= =
222 | I S I W S C 2 0 1 9