1 AR(1) model

An autoregressive model of order 1, AR(1), is defined as: \[ y_t = \phi_0 +\phi_1 \ y_{t-1} +\epsilon_t \] with the following assumptions \(\forall t\):

• \(\mathbb{E}[\epsilon_t]= 0\), \(\mathbb{V}[\epsilon_t]= \sigma_{\epsilon}^2\) and the errors \(\epsilon_{t_1}\) and \(\epsilon_{t_2}\) are independent \(\forall t_1\neq t_2\).
• \(\mathbb{E}[y_t]= \mu_y\) (stationary in mean). It can be shown \(\mu_t=\frac{\phi_0}{1-\phi_1}\)
• \(\mathbb{V}[y_t]= \sigma_y^2\) (stationary in variance). It can be shown \(\sigma_y^2=\frac{\sigma_{\epsilon}^2}{1-\phi_1^2}\) imposing the constraint \(-1<\phi_1<1\).

AR(1) can be rewritten: \[ y_t-\mu_y= \phi_1 \ (y_{t-1}-\mu_y) +\epsilon_t \] and it can be shown that the covariance between \(y_t\) and \(y_{t-k}\) is: \[ \mathbb{C}\text{ov}[y_t,y_{t-k}]=(\phi_1)^k \ \sigma_y^2 \] and consequently the correlation between \(y_t\) and \(y_{t-k}\) is: \[ \mathbb{C}\text{or}[y_t,y_{t-k}]=(\phi_1)^k \]

require(fma)
tsdisplay(arima.sim(n=10000,list(ar = c(0.8897)), sd = sqrt(0.1796)))

require(fma)
tsdisplay(arima.sim(n=10000,list(ar = c(-0.8897)), sd = sqrt(0.1796)))

2 AR(2) model

AR(2) can be written: \[ y_t-\mu_y= \phi_1 \ (y_{t-1}-\mu_y) +\phi_2 \ (y_{t-2}-\mu_y) +\epsilon_t \] where \(y_t\) is stationary in mean and variance.

Below is a plot of a simulated time series that follows an AR(2) model with \(\phi_1=0.8897\) and \(\phi_2=-0.5\). Note how the PACF(1) and PACF(2) are not zeroes while all others coefficients PACF(k) are for \(k>2\). The ACF displays a damped sine wave and the time plot shows a contant mean and variance over time (as expected from AR(2)).

require(fma)
tsdisplay(arima.sim(n=10000,list(ar = c(0.8897,-0.5)), sd = sqrt(0.1796)))

Exercise: What contraint is there on the parameters \(\phi_1\) and \(\phi_2\) in an AR(2) ?

3 Using differencing before AR(1) does not create AR(2)

Defining \(x_t=y_t-y_{t-1}\) as the difference of a time series \(y_t\) and Assuming an AR(1) explains \(x_t\): \[ x_t-\mu_x = \phi_1 (x_{t-1}-\mu_x) + \epsilon_t \] For \(y_t\), this means: \[\begin{equation} y_t-y_{t-1}-\mu_x = \phi_1 (y_{t-1}-y_{t-2}-\mu_x) + \epsilon_t \end{equation}\]

Exercise. could \(y_t\) be stationary in mean ?

If \(y_t\) is stationary in mean then \(\mathbb{E}[y_t]=\mu_y\ \forall t\) and \[ \begin{array}{l} \left(\mathbb{E}[y_t-y_{t-1}-\mu_x] =\mathbb{E}[ \phi_1 (y_{t-1}-y_{t-2}-\mu_x) + \epsilon_t]\right)\\ \equiv \left( \mu_y -\mu_y-\mu_x = \phi_1 \ (\mu_y-\mu_y-\mu_x) +0 \right)\\ \equiv \left( -\mu_x =- \phi_1 \ \mu_x \right) \end{array} \] which is only possible if \(\mu_x=0\).

So if \(\mu_x \neq 0\) then \(y_t\) is not stationary in mean (and cannot be explained by an AR(2)).

Exercise. Assuming \(\mu_x=0\), what does it mean if \(y_t\) stationary in variance ?

We assume \(\mu_x=0\), \(\mathbb{E}[y_t]=\mu_y \forall t\), and \(\mathbb{V}[y_t]=\sigma_y^2 \forall t\)

\(x_t\) follows an AR(1) that is assumed stationary in variance then \[ \mathbb{E}[x_t^2]=\sigma_x^2 \quad \forall t \ (\text{since} \ \mu_x=0) \] hence equivalently \[ \mathbb{E}[(y_t-y_{t-1})^2]=\mathbb{E}[(y_t-\mu_y-y_{t-1}+\mu_y)^2]=\sigma_x^2 \quad \forall t \] leading to \[ 2\sigma_y^2 - 2 \gamma_1=\sigma_x^2 \quad (*) \] using notation \(\gamma_1=\mathbb{C}\text{ov}[y_t,y_{t-1}]\ \forall t\).

Using the equation \[ y_t-\mu_y = (1+\phi_1) \ (y_{t-1}-\mu_y) - \phi_1 \ (y_{t-2}-\mu_y) + \epsilon_t \] The covariance \(\gamma_1\) can also be computed leading to \[ \gamma_1=(1+\phi_1) \sigma_y^2- \phi_1 \gamma_1 \] since \(\epsilon_t\) is not correlated with \((y_{t-1}-\mu_y)\). \[ \gamma_1=\sigma_y^2 \quad (**) \] Using \((*)\) and \((**)\) we get \(\sigma_x^2=0\).

Conclusion: \(y_t\) is not an AR(2) in general because it is not stationary in mean and variance at the exception of the case when \(x_t\) is having contant value 0 (!).

4 Using arima.sim in R

Below a time series \(y_t\) is created such that the time series \(x_t=y_t-y_{t-1}\) follows AR(1). Note how the ACF coefficient remains high indicating that there is a trend in the time series.

require(fma)
tsdisplay(arima.sim(list(order = c(1,1,0), ar = 0.7), n = 10000))