1. Linear regression in Python: three libraries, three philosophies

Level 6: Regression and Pricing Models

Linear regression is probably the most used tool in all of quantitative trading. Before getting into the theory of pricing models, it is worth knowing which tools estimate a regression in Python, because they are not all good for the same thing.

These are the three options you will meet in 95% of financial code:

• statsmodels: the statsmodels.api.OLS() class (or the formula interface statsmodels.formula.api.ols()). It is the econometricians' library.
• scikit-learn: the sklearn.linear_model.LinearRegression() class. It is the machine learning crowd's library.
• NumPy: the numpy.polyfit() function with degree 1. It is the minimalist option.

And there are more (TensorFlow with the Keras API, PyTorch...), but the choice depends on the specific problem, on how familiar you are with the library and on personal preference.

The practical difference is enormous and worth understanding from the start. statsmodels is built for inference: it wants you to answer questions such as whether a share's beta is statistically different from zero, what its confidence interval is, or whether a fund's alpha is real or noise. That is why its .summary() method hands you coefficients, standard errors, t statistics, p-values and R squared. scikit-learn is built for prediction: it gives you .fit() and .predict(), a .coef_ with the slopes, and that is that. There are no p-values because in the machine learning paradigm what matters is out-of-sample error, not the significance of each parameter. NumPy is built for none of that: it fits a line and hands you two numbers.

import numpy as np
import statsmodels.api as sm
from sklearn.linear_model import LinearRegression

# x: market returns, y: share returns
# 1. NumPy: minimalist. Returns the coefficients from highest to lowest degree.
slope, intercept = np.polyfit(x, y, 1)

# 2. scikit-learn: prediction oriented.
modelo = LinearRegression().fit(x.reshape(-1, 1), y)
print(modelo.coef_, modelo.intercept_)

# 3. statsmodels: inference oriented. Careful: the constant must be added by hand.
X = sm.add_constant(x)
resultado = sm.OLS(y, X).fit()
print(resultado.summary())

There is one detail that causes a great many silent errors: statsmodels does not add the constant term for you. If you call sm.OLS(y, x) without sm.add_constant(x), you are forcing the line through the origin. In a pricing model that means imposing that alpha is exactly zero, which is precisely the hypothesis you wanted to test. scikit-learn does add it by default (fit_intercept=True), and np.polyfit with degree 1 also estimates the intercept.

In this level we will mostly use statsmodels, because pricing models (CAPM, Fama-French) are not used to predict: they are used to decompose a return into explainable parts and find out whether anything is left unexplained.