3. Visualization: distributions and scatter plots

Level 8: Time Series - Stationarity

Time series lend themselves naturally to visualization. Line plots of the observations over time are the most common, but there is a whole repertoire of additional plots you can use to get to know your problem better. The more you learn about your data, the more likely you are to develop a better forecasting model.

df.plot()   # S&P 500 levels
Re.plot()   # log returns

The contrast between the two plots already tells the central story of the level: the price series rises and falls without ever returning to a reference level, while the return series oscillates around zero with concentrated bursts of turbulence (March 2020 leaps off the page).

Histograms and density plots.

df.hist()
Re.hist()
df.plot(kind='kde')

This means displaying the values without their temporal ordering. Some linear time series forecasting methods assume a well-behaved distribution of the observations (a Gaussian bell curve, a normal). That can be checked explicitly with hypothesis tests, but plots offer a very useful first check of the distribution, both on the raw observations and after any transformation has been applied.

Kernel density estimation (KDE) is mainly there to provide a smoothed version of the histogram, showing the overall distribution of the data. On a time series it helps you understand what values the series takes, whether they are normally distributed, whether there is skewness, and so on. In the case of S&P 500 returns you will see a density that is more peaked in the center and has fatter tails than a normal: excess kurtosis, the usual signature of financial data.

Lag scatter plots. Time series modeling assumes that a relationship exists between an observation and the previous observations. Previous observations are called lags: the one at the previous step is lag 1, the one two steps back is lag 2, and so on. One very useful plot for exploring the relationship between each observation and one of its lags is the scatter plot. Pandas has a built-in function for exactly that, the lag plot: it puts the observation at time t on the x axis and the lag 1 observation (t-1) on the y axis.

• If the points cluster along a diagonal running from bottom left to top right, it suggests a positive correlation.
• If they cluster along a diagonal running from top left to bottom right, it suggests a negative correlation.
• Either relationship is good news, because both can be modeled.

The more points hug the diagonal, the stronger the relationship; the more scatter around the line, the weaker. A blob in the center or a cloud spread across the whole plot suggests a weak or non-existent relationship.

lag_plot(ts)   # on prices: an almost perfect diagonal
lag_plot(Re)   # on returns: a shapeless blob

And we can generate a grid with several lags at once:

import matplotlib.gridspec as gridspec

values = pd.DataFrame(df.values)
lags = 12
columns = [values]
for i in range(1, lags + 1):
    columns.append(values.shift(i))

dff = pd.concat(columns, axis=1)
dff.columns = ['t'] + ['t-' + str(i) for i in range(1, lags + 1)]

rows = (lags + 3) // 4
gs = gridspec.GridSpec(rows, 4)
fig = plt.figure(figsize=(15, 12))
for i in range(1, lags + 1):
    ax = fig.add_subplot(gs[(i - 1) // 4, (i - 1) % 4])
    ax.set_title('t vs t-' + str(i))
    ax.scatter(x=dff['t'].values, y=dff['t-' + str(i)].values)
plt.tight_layout()
plt.show()