Page 159 - Contributed Paper Session (CPS) - Volume 2

CPS1488 Willem van den B. et al.

Algorithm 1 EP-IS.

Input: Data y, forward map H, and data partition{ }
=1
1. Initialize µk = 01×d and Λk = 0d×d for = 1, . . . , .

2. Iterate the EP updates in (6) derived from the linearization for =
1, . . . , repeatedly for a fixed number of times or until convergence.

3. Iterate the EP updates in (7) derived from importance sampling with
regularization of choice on ΣIS for = 1, . . . , repeatedly for a fixed
number of times or until convergence.

Output: The approximate posterior ()

() = (Λ , Λ )
−1
−1
\
\ \
\
2
2
Computation of the mean and variance of () {− || − ()|| /2 }
\

in (4) is often intractable unless is linear. Consider therefore the first-order
Taylor series expansion of around the current approximation µ of the
posterior mean,
() = () + ( − ) (5)

where is the nk×d Jacobian matrix of . Under this approximation,

2
() {−|| − ̂ ()|| } is Gaussian such that the EP updates of
\

matching the moments in (4) follow as
1

= { − () + } and Λ = ∑ −1 −Λ (6)
\

2
Iterating these updates yields a linearization based posterior approximation
(). Aside from the data partition, this scheme is similar to the Gauss-Newton
algorithm obtained in Steinberg and Bonilla (2014) for a Laplace
approximation to ( | ).
To obtain an approximation that is more accurate than the limitations of
the linearization in (5), we consider importance sampling with the exact for
finding the mean and covariance of the right-hand side in (4). As proposal
distribution, we choose the right-hand side of (4) with the approximation (5),
1 2
that is () {− 2 2 || − ̂ ()|| } up to a proportionality constant. This

\
is a Gaussian and thus easy to sample from. Moreover, it is expected to be
1
reasonably close to the right-hand side of (4), () {− 2 2 || −
\
()|| }, up to a proportionality constant.
2

148 | I S I W S C 2 0 1 9