7. Case studies 2 and 3: SVM for regimes and a KNN strategy

Level 11: Neural Networks for Trading

Case 2: support vector machines to predict regime and trend. The plan is this: first, create an unsupervised algorithm that predicts market regimes; second, plot them to visualize them; third, train a support vector classifier using the regime as one of the features; fourth, use that classifier to predict the day's trend at the market open; and fifth, visualize the strategy's performance on the test data.

import numpy as np
import pandas as pd
import yfinance as yf
import seaborn as sns
import matplotlib.pyplot as plt
import talib as ta
from sklearn import mixture as mix
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC

df = yf.download('SPY', start='2000-01-01', end='2012-01-01')
df = df.dropna()[['Open', 'High', 'Low', 'Close']]

n = 10           # 10-day window
t = 0.8          # 80% training, 20% test
split = int(t * len(df))

We shift High, Low and Close by one day so as to access past data only. We then build technical indicators: RSI, simple moving average, ADX, correlation, parabolic SAR and the previous day's return computed open to open.

df['high'] = df['High'].shift(1)
df['low'] = df['Low'].shift(1)
df['close'] = df['Close'].shift(1)
df['RSI'] = ta.RSI(np.array(df['close']), timeperiod=n)
df['SMA'] = df['close'].rolling(window=n).mean()
df['Corr'] = df['SMA'].rolling(window=n).corr(df['close'])
df['SAR'] = ta.SAR(np.array(df['high']), np.array(df['low']), 0.2, 0.2)
df['ADX'] = ta.ADX(np.array(df['high']), np.array(df['low']),
                   np.array(df['close']), timeperiod=n)
df.loc[df.Corr > 1, 'Corr'] = 1
df.loc[df.Corr < -1, 'Corr'] = -1
df['Return'] = np.log(df['Open'] / df['Open'].shift(1))
df = df.dropna()

Now the unsupervised algorithm. A Gaussian mixture splits the market into 4 regimes according to a criterion of its own choosing: we give it no labeled set whatsoever. Note that it is fitted on the training data only and predicts on the later stretch.

ss = StandardScaler()
unsup = mix.GaussianMixture(n_components=4, covariance_type='spherical',
                            n_init=100, random_state=42)
df = df.drop(['High', 'Low', 'Close'], axis=1)
unsup.fit(np.reshape(ss.fit_transform(df[:split]), (-1, df.shape[1])))
regime = unsup.predict(np.reshape(ss.transform(df[split:]), (-1, df.shape[1])))

Regimes = (pd.DataFrame(regime, columns=['Regime'], index=df[split:].index)
           .join(df[split:], how='inner')
           .assign(market_cu_return=df[split:].Return.cumsum())
           .reset_index(drop=False).rename(columns={'index': 'Date'}))

for i in [0, 1, 2, 3]:
    print('Regime %i mean: ' % i, unsup.means_[i][0])
    print('Regime %i covariance: ' % i, unsup.covariances_[i])

Without even looking at the factors used for the classification, some things can be concluded from the chart: there is a low-volatility or sideways zone, a high-volatility or panic zone, and a breakout zone. The data are read as follows: regime 0 has a low mean and high covariance; regime 1, a high mean and high covariance; regime 2, a high mean and low covariance; regime 3, a low mean and low covariance.

We scale the regimes DataFrame excluding the Date and Regime columns, without losing any feature, and we create the signal column that the algorithm will learn to predict.

ss1 = StandardScaler()
columns = Regimes.columns.drop(['Regime', 'Date'])
Regimes['Signal'] = 0
Regimes.loc[Regimes['Return'] > 0, 'Signal'] = 1
Regimes.loc[Regimes['Return'] < 0, 'Signal'] = -1
Regimes['return'] = Regimes['Return'].shift(1)
Regimes = Regimes.dropna()

cls = SVC(C=1.0, kernel='rbf', gamma='auto', degree=3, tol=0.001,
          decision_function_shape='ovr', shrinking=True)

split2 = int(0.8 * len(Regimes))

# the scaler is fitted ONLY on the training slice and then applied to everything:
# fitting it on the whole set would leak the future mean and standard deviation into the test
ss1.fit(Regimes[columns].iloc[:split2])
Regimes[columns] = ss1.transform(Regimes[columns])

X = Regimes.drop(['Signal', 'Return', 'market_cu_return', 'Date'], axis=1)
y = Regimes['Signal']
cls.fit(X[:split2], y[:split2])

p_data = len(X) - split2
df['Pred_Signal'] = 0
df.iloc[-p_data:, df.columns.get_loc('Pred_Signal')] = cls.predict(X[split2:])
df['str_ret'] = df['Pred_Signal'] * df['Return'].shift(-1)

The 'rbf' kernel is what makes the SVM non-linear: it implicitly projects the data into a very high-dimensional space where a linear boundary separates classes that were not separable in the original space. C controls the trade-off between a wide margin and classification errors.

Case 3: a trading strategy with KNN. The nearest neighbor is the simplest possible model and serves as the benchmark against which to measure everything above.

from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score

df = yf.download('SPY', start='2012-01-01', end='2021-01-01').dropna()
df = df[['Open', 'High', 'Low', 'Close']]

df['Open-Close'] = df.Open - df.Close
df['High-Low'] = df.High - df.Low
df = df.dropna()
X = df[['Open-Close', 'High-Low']]
Y = np.where(df['Close'].shift(-1) > df['Close'], 1, -1)

split = int(0.7 * len(df))
X_train, X_test = X[:split], X[split:]
Y_train, Y_test = Y[:split], Y[split:]

knn = KNeighborsClassifier(n_neighbors=15)
knn.fit(X_train, Y_train)
print('Train accuracy: %.2f' % accuracy_score(Y_train, knn.predict(X_train)))
print('Test accuracy:  %.2f' % accuracy_score(Y_test, knn.predict(X_test)))

df['Predicted_Signal'] = knn.predict(X)
df['SPY_returns'] = np.log(df['Close'] / df['Close'].shift(1))
df['Strategy_returns'] = df['SPY_returns'] * df['Predicted_Signal'].shift(1)
Cumulative_SPY = df[split:]['SPY_returns'].cumsum() * 100
Cumulative_Strategy = df[split:]['Strategy_returns'].cumsum() * 100

KNN learns no parameters: it stores the data and classifies each new day by a vote of its 15 closest neighbors. A small k memorizes the noise; a large one smooths until the signal is gone. And pay attention to the shift(1) on Predicted_Signal: yesterday's signal is applied to today's return, which is the only thing you can actually execute.