3. Standard errors and confidence intervals

Level 7: Rigorous Statistical Diagnostics

The formula for the variance of the OLS estimator has an obvious problem: it contains the true error variance, which is unknown. The solution is to estimate it from the residuals, which we do observe. The sample residual variance is computed as the sum of squared residuals divided by the number of observations minus the number of estimated parameters:

# In words: estimated sigma squared = RSS / (N - k)
#   RSS = sum of squared residuals
#   N   = number of observations
#   k   = number of estimated parameters (including the constant)
rss = (model.resid ** 2).sum()
n, k = model.nobs, len(model.params)
sigma2_hat = rss / (n - k)

We divide by N minus k, not by N, because estimating k parameters has used up k degrees of freedom: the residuals are not free, they are tied down by the k restrictions that OLS imposes. This number is only an estimate, but it is the centerpiece of everything that follows: without it there are no test statistics and no intervals.

Since the estimator is a random variable, we can measure its uncertainty with its standard deviation. The square roots of the diagonal elements of the estimated variance-covariance matrix of the estimator are the standard errors of the coefficients. The smaller the better: they describe the precision of the estimator. Read correctly, they are a measure of sampling variability: if we collected several different samples, the OLS estimates would differ in each one, and the standard error quantifies that dispersion.

model.bse
# The b in bse refers to the coefficient vector and se to standard errors.

Three things shrink a coefficient's standard error: more observations, more variation in the regressor and a smaller residual variance. And one thing inflates it silently: collinearity between regressors. When two factors move almost identically, the regression cannot split the credit between them and the standard errors blow up, even though the overall fit may look excellent. It is a classic failure when building multifactor models with redundant factors.

With the standard error in hand we reach interval estimation: a procedure for building ranges of values, called confidence intervals, in which the unknown parameter is likely to lie. Building these intervals leans heavily on the UR.4 normality assumption, because it requires knowing the exact distribution of the estimator.

model.conf_int()             # 95% intervals by default
model.conf_int(alpha=0.01)   # 99%: wider

The correct reading of a 95% interval is not that there is a 95% probability that the parameter lies inside it: the parameter is fixed, what is random is the interval. The correct reading is that the procedure, repeated over many samples, produces intervals that contain the true parameter 95% of the time.

In practice, an interval is far more informative than a bare significant or not significant. If you estimate a fund's beta against the market and obtain an interval from 0.87 to 1.34, you have learned two things at once: that beta is clearly different from zero, but also that your data are perfectly compatible with a fund that tracks the market (beta equal to 1) and with a considerably more aggressive one. That nuance disappears if all you look at is whether the table shows three asterisks.