4. Moving average (MA) models: modeling the error, not the level

Level 9: Time Series - ARIMA and GARCH

The difference between what was expected and what was forecast is called the residual error. And here comes the elegant idea of this section: the residuals of a forecast form a time series of their own that may have temporal structure. If that structure exists, we can model it, forecast the future error and use that forecast to correct the original prediction.

The model that does this is called the moving average model, or MA. Watch out for the name: it shares its name with moving average smoothing, but it is something completely different. Smoothing averages past values of the series; the MA model takes a weighted linear sum of past shocks (noise terms).

# MA(q) written as code:
# x[t] = w[t] + beta_1*w[t-1] + beta_2*w[t-2] + ... + beta_q*w[t-q]
#
# w[t]    -> white noise, zero mean and constant variance
# beta_j  -> weight of the shock that occurred j periods ago

The motivation is explicit: the MA model explains the shocks of the error process directly, by fitting a model to the error terms themselves. In an AR(p) those shocks are only observed indirectly, through the past values of the series.

A key identification property: by definition, the autocorrelation of an MA(q) is zero for every lag greater than q. That is why a sharp cut-off in the ACF (and not in the PACF) points to an MA.

In finance, an MA(1) with a negative beta is the typical signature of the *bid-ask bounce*: the price bounces between bid and ask, generating a one-step reversion that has nothing to do with economic information.

Autoregression of the residual error. We can model the residual series with an autoregressive model. We start from the persistence model (forecasting that tomorrow's return will be today's) and fit an AR to its errors:

# lagged dataset: column t predicts column t+1
values = pd.DataFrame(Re.values)
Re3 = pd.concat([values.shift(1), values], axis=1)
Re3.columns = ['t', 't+1']
Re3 = Re3[1:]

X = Re3.values
train_size = int(len(X) * 0.66)
train, test = X[1:train_size], X[train_size:]
train_X, train_y = train[:, 0], train[:, 1]
test_X, test_y = test[:, 0], test[:, 1]

# persistence model on the training set
train_pred = [x for x in train_X]
train_resid = [train_y[i] - train_pred[i] for i in range(len(train_pred))]

# we model the residuals with an AR
model_fit = AutoReg(train_resid, lags=1).fit()
window = len(model_fit.model._lags)
coef = model_fit.params
print('Lag=%d, Coef=%s' % (window, coef))

The autoregressive model needs the residual errors of the previous steps, so they have to be kept on hand. We walk through the test set step by step: we forecast, we compute the actual error, we forecast the error of the next step and we update the history:

history = train_resid[len(train_resid) - window:]
history = [history[i] for i in range(len(history))]
predictions, expected_error = [], []

for t in range(len(test_y)):
    yhat = test_X[t]                     # persistence
    error = test_y[t] - yhat             # the error actually observed
    expected_error.append(error)

    length = len(history)
    lag = [history[i] for i in range(length - window, length)]
    pred_error = coef[0]
    for d in range(window):
        pred_error += coef[d + 1] * lag[window - d - 1]
    predictions.append(pred_error)
    history.append(error)

This is a walk-forward, or rolling, forecast applied to the error. The result is two series: the expected residual error and the predicted residual error. Plotting them together gives you an immediate sense of whether the model has any skill at forecasting the error.

And the practical application: the corrected forecast is simply the original forecast plus the predicted error. If the error model gets it right, the correction reduces the overall RMSE.