Page 155 - Contributed Paper Session (CPS) - Volume 4

CPS2164 Jonathan Hosking et al.

Analytical likelihood derivatives for state space
forecasting models
Jonathan Hosking, Ramesh Natarajan
Amazon.com, New York, U.S.A.

Abstract
State space models are a flexible and widely used family of statistical models
for time series analysis and forecasting. Fitting of the models to historical
data is greatly facilitated by the availability of analytical derivatives of the
log-likelihood function. We have obtained a new expression for these
derivatives in terms of quantities routinely computed in Kalman filtering and
smoothing. This result makes it straightforward to construct an optimization
method based on gradient descent using analytical log-likelihood derivatives.
We present the derivation and give some examples of the gain in speed of
parameter estimation when analytical derivatives are used.

Keywords

Time series; maximum likelihood; computation

1. State space models
A state space model for time series data treats the observed data vector yt
as a noisy observation of an unobserved state vector αt that evolves according
to a Markov process. We consider the linear gaussian state space model, which
we write using the notation of Durbin and Koopman (2012, eq. (4.12)):

Vectors yt, αt, and ηt have respective dimensions p, m, and r, with r≤m. The
observation noise εt, state disturbance ηt, and initial state α1 have normal
distributions:

εt ∼N(0,Ht), ηt ∼N(0,Qt), α1 ∼N(a1,P1). (3)

The state space model provides a flexible framework for specification of
forecasting models. Special cases include ARIMA models (Hamilton, 1994),
exponential smoothing (Hyndman et al., 2002), and models involving trend,
seasonality, regression, and noise components, variously known as dynamic
linear models (West and Harrison, 1997; Petris et al., 2009) structural models
(Harvey, 1989). Dynamic linear models, because they decompose a time series
into interpretable components, make particularly effective and understandable

144 | I S I W S C 2 0 1 9