Univariate Time Series: Extended

Before proceeding it’s important to be familiar with OLS regression, in case you aren’t check this:

https://www.sfu.ca/~dsignori/buec333/lecture%208.pdf

Wold Representation Theorem

Any zero-mean covariance stationary time series ${X_t}$ can be decomposed as $X_t = V_t + S_t$, where

• $V_t$ is a linearly deterministic process
• $S_t = \Sigma_{i=0}^{\infty}\Psi_i\eta_{t-i}$ is an infinite moving average process of error terms, where

  • $\Psi_0 = 1$, $\Sigma_{i=0}^{\infty}\Psi_i^2 < \infty$

  • $\{\eta_t\}$ is linearly unpredictable white noise and $\{\eta_t\}$ is uncorrelated with $\{V_t\}$ (In case you forgot correlation check).

    So $E(\eta_t) = 0, \quad E(\eta_t^2) = \sigma^2, \quad E(\eta_t\eta_s) = 0, \quad \forall t,s\ne t \\ E(\eta_tV_s) = 0 \quad\forall t,s$

Using Lag Operator $L()$

The lag operator L() shifts a time series back by one time increment. Recursively, $L^n(X_t) = L(L^{n-1}(X_t)) = X_{t-n}$

Expressing Wold Representation using lag operator, $X_t = \psi(L)\eta_t + V_t$

Some Definitions

• Impulse Response Function: The Impulse Response function of the covariance stationary process $\{X_t\}$ is $IR(j) = \frac{\delta X_t}{\delta \eta_{t-j}} = \psi_j$. The long-run cumulative response of $\{X_t\}$ is $\Sigma_{i=0}^{\infty}IR(j) = \Sigma_{i=0}^{\infty}\Psi_i = \Psi(L)$ with L=1.
• Invertible time series: When $\Psi^{-1}(L)$ exists, the time series $\{X_t\}$ is Invertible and has an auto-regressive representation $X_t = (\Sigma_{i=0}^{\infty}\Psi_i^*X_{t-i}) + \eta_t$

ARMA$(p,q)$ Models

The time series ${X_t}$ follows the ARMA(p, q) Model with auto-regressive order p and moving-average order q if

$$ X_t = \mu + \phi_1(X_{t-1} - \mu) + \phi_2(X_{t-2} - \mu) + \dots \phi_p(X_{t-p} - \mu) + \eta_t + \theta_1\eta_{t-1} + \theta_2\eta_{t-2} + \dots \theta_q\eta_{t-q} $$

where $\{\eta_t\}$ is $WN(0, \sigma^2)$ with

$E(\eta_t) = 0, \forall t \quad; \quad E(\eta_t^2) = \sigma^2 < \infty, \forall t, \text{ and } E(\eta_t\eta_s) = 0, \forall t \ne s$

Yule-Walker Equations & Toeplitz Matrix

For an AR(p) model:

$$ (X_t - \mu) = \phi_1(X_{t-1} - \mu) + \phi_2(X_{t-2} - \mu) + \dots \phi_p(X_{t-p} - \mu) +\eta_t $$

we can write the Yule-Walker Equations $(j=0, 1, \dots)$

$$ E[(X_t - \mu)(X_{t-j} - \mu)] = \phi_1E[(X_{t-1} - \mu)(X_{t-j} - \mu)] + \phi_2E[(X_{t-2} - \mu)(X_{t-j} - \mu)] + \dots \phi_pE[(X_{t-p} - \mu)(X_{t-j} - \mu)] +E[\eta_t(X_{t-j} - \mu)] \\ \gamma(j) = \phi_1\gamma(j-1) + \phi_2\gamma(j-2) + \dots \phi_p\gamma(j-p) + \delta_{0,j}\sigma^2 $$