1. From the linear trend to autoregressive AR(p) models

Level 9: Time Series - ARIMA and GARCH

In the previous level you learned how to check whether a series is stationary (plots, rolling statistics, the ADF test) and how to read the ACF and the PACF. We take all of that as given: here we start to model and forecast.

The most naive model is the linear model, also called the trend model. It describes a series that can be drawn with a straight line: the value at time t is an initial level plus a slope multiplied by time, plus noise. With no mathematical formulas, in plain Python:

import numpy as np

# x[t] = a0 + a1 * t + w[t]
# a0 -> initial level, a1 -> constant increment per time step, w[t] -> white noise

np.random.seed(7)
w = np.random.randn(250)
x = np.empty_like(w)

a0 = 100.0   # the stock starts at 100 dollars
a1 = 0.5     # and rises 50 cents at every time step

for t in range(len(w)):
    x[t] = a0 + a1 * t + w[t]

It is the statistical equivalent of a company whose sales grow by exactly the same amount every quarter. If we difference the series once, the trend disappears:

dx = np.diff(x)   # removes the deterministic trend

Inspecting the residuals of this model produces the professor's important finding: the distribution is approximately normal, but the residuals are correlated. The PACF shows a significant value at lag 1. Before using the model to forecast, that autocorrelation would have to be absorbed. And the meaning of a significant PACF at lag 1 is concrete: an autoregressive model may be appropriate.

That is the conceptual leap of this level. In finance, a deterministic trend model is almost always a bad idea: it assumes that the future price is determined by the calendar, not by what has happened. Extrapolating a line fitted to historical prices produces forecasts that grow without limit and that incorporate no new information. What does make sense is to let the series' own past explain its future.

That is an autoregressive model of order p, or AR(p). It is a linear regression in which the explanatory variables are the lagged values of the series itself:

# AR(p), written as code rather than as a formula:
# x[t] = alpha_1*x[t-1] + alpha_2*x[t-2] + ... + alpha_p*x[t-p] + w[t]
#
# p         -> number of lagged terms entering the model
# alpha_i   -> coefficient on lag i
# w[t]      -> white noise (zero mean, constant variance)

The AR(p) is a direct extension of the random walk. In fact, an AR(1) with alpha_1 equal to 1 is exactly a random walk, and therefore not stationary. This is one of the model's most important properties: the AR(p) is not always stationary; its stationarity depends on the parameter values, not on the form of the model. For an AR(1) the condition is that the absolute value of alpha_1 be smaller than 1. If alpha_1 is 0.98, the series is technically stationary but reverts to the mean so slowly that in a short sample it will look like a random walk.

In practice, this means that fitting an AR to prices almost never works: you will get an alpha close to 1 that adds nothing. It is fitted to log returns, which are already practically stationary.