6. statsmodels in practice: from prices to factor regression

Level 6: Regression and Pricing Models

Let us build the full pipeline: download prices, clean them, turn them into returns, bring in the Fama-French factors and estimate CAPM, FF3 and FF5 in one go.

statsmodels offers a good range of cross-sectional models and some time series ones too. Basic linear regression is provided by OLS. Estimating a model requires first specifying the model and then calling a method to estimate the parameters (called fit in most models). That separation between specifying and estimating is the library's signature.

We start with the data. The list of S&P 500 constituents is taken from Wikipedia, their adjusted prices are downloaded and SPY is added as a proxy for the market:

data = yf.download(tickers, start=start, end=end)['Adj Close']

cleaned_data = data.copy()
# We drop companies (columns) with no data at all across the whole range
cleaned_data.dropna(axis='columns', how='all', inplace=True)
# We drop days (rows) with no data for any company
cleaned_data.dropna(axis='index', how='all', inplace=True)
# And finally the columns with at least one NaN
cleaned_data.dropna(axis='columns', how='any', inplace=True)

# We turn prices into logarithmic returns
return_values = np.log(cleaned_data).diff().dropna()

An honest warning about that last dropna: by removing every column with at least one missing value you are left only with the companies that existed and traded throughout the whole period. That is survivorship bias straight out of the textbook. For a teaching exercise it will do, but if you go on from there to build a strategy, your results will be inflated because you have deleted precisely the ones that went bust.

Now the function that does the work, as the lecturer sets it out:

def assetPriceReg(df_stk):
    import pandas_datareader.data as web

    # Factor data from Kenneth French's library
    df_factors = web.DataReader('F-F_Research_Data_5_Factors_2x3_daily', 'famafrench')[0]
    df_factors.rename(columns={'Mkt-RF': 'MKT'}, inplace=True)
    for col in ['MKT', 'SMB', 'HML', 'RMW', 'CMA', 'RF']:
        df_factors[col] = df_factors[col] / 100

    # We join the share returns and the factors by date
    df_stock_factor = pd.merge(df_stk, df_factors,
                               left_index=True, right_index=True)
    # Excess return over the risk-free asset
    df_stock_factor['XsRet'] = df_stock_factor['Returns'] - df_stock_factor['RF']

    CAPM = sm.ols(formula='XsRet ~ MKT', data=df_stock_factor).fit(
        cov_type='HAC', cov_kwds={'maxlags': 1})
    FF3 = sm.ols(formula='XsRet ~ MKT + SMB + HML', data=df_stock_factor).fit(
        cov_type='HAC', cov_kwds={'maxlags': 1})
    FF5 = sm.ols(formula='XsRet ~ MKT + SMB + HML + RMW + CMA',
                 data=df_stock_factor).fit(cov_type='HAC', cov_kwds={'maxlags': 1})
    return CAPM, FF3, FF5

Three critical details. First, the division by 100: French's library publishes the factors in percentage terms (0.53 means 0.53%), whereas the logarithmic returns from yfinance come as plain decimals. Without that conversion, beta comes out scaled by 100 and the whole model stops making sense. Second, the pd.merge on the index performs an inner join: only the dates present in both sets survive, which is what you want. Third, the dependent variable is never the return, it is the excess return over RF: if you regress the raw return, the intercept is no longer alpha.

For the specific share the logarithmic return is prepared and everything is launched:

df_stk = yf.download('NVDA', start=start, end=end)
df_stk['Returns'] = np.log(df_stk[['Adj Close']]).diff().dropna()
df_stk['Returns'].describe()

df_regOutput = assetPriceReg(df_stk)