1. Preparing time series data: from price to return

Level 8: Time Series - Stationarity

In this level we get into the methodology of univariate time series forecasting, leaning on the main tools of the Python ecosystem. Pandas offers very solid capabilities for handling temporal data, and Statsmodels provides what we need to dissect the data, estimate statistical models and run hypothesis tests.

The quantitative researcher sets out to discern trends, seasonal fluctuations and dependencies in historical financial data by applying statistical time series techniques. The ultimate aim is to generate actionable trading signals. Improving the profitability of our algorithms demands rigorous statistical methods that identify consistent behavior in the asset which can be exploited as an opportunity. That means examining in depth how the properties of the price evolve over time.

To begin, we download the S&P 500 (^GSPC) from Yahoo Finance. The goal of the whole level will be to understand and, later, to forecast this index.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import datetime
import yfinance as yf
import warnings
import statsmodels.api as sm
import statsmodels.tsa.api as smt
import scipy.stats as scs
from statsmodels.tsa.stattools import coint, adfuller
warnings.filterwarnings('ignore')

plt.rc('figure', figsize=(18, 3))
pd.set_option('display.float_format', lambda x: '%.2f' % x)

We load almost seven years of daily history:

start = datetime.datetime(2016, 1, 1)
end = datetime.datetime(2022, 12, 30)
df = yf.download('^GSPC', start=start, end=end, auto_adjust=True)['Close']

df.head()
df.dropna(axis='index', how='all', inplace=True)
df.index[:5]   # the index already comes as datetime, which is what we want

That datetime index is not a cosmetic detail: it is what turns an ordinary DataFrame into a time series. Pandas represents a time series as a Series object, a one-dimensional array with a time label per row.

The second step, and conceptually the most important, is to move from price to log return:

Re = np.log(df).diff().dropna()   # equivalent to np.log(df / df.shift(1))
Re.head()

The difference of the logarithm is the first difference of the series on a log scale. We work with returns rather than price levels for three reasons: returns are approximately stationary while prices are not, they are additive over time (the sum of several days of log returns is the log return of the whole period), and they are comparable across assets with wildly different price scales. The .dropna() is mandatory because the first difference always leaves a null value in the first row: there is no previous day to compare it with.

With the series built, pandas lets you query it using the date index directly, with no need for numeric positions:

ts = pd.Series(df, index=df.index)

df[:'2017'].tail(10)        # everything up to the end of 2017
df['2020-06':'2020-07']     # two specific months
df.size                     # number of observations
df.describe()               # descriptive statistics

Be careful with describe() on prices: it will hand you a perfectly computed and completely useless mean and standard deviation. If the series has a trend, that mean does not describe any real state of the market, but a blend of different states at different moments. It is the first clue as to why stationarity, which we will come to later, is going to be the central concept of the level.