4. Supervised learning: learning with a teacher
Level 10: Machine Learning Applied to Trading
Machine learning problems fall into a handful of types depending on what information we have available. The first and most common is supervised learning.
Algorithms of this type work on the teacher model. That means we have a dataset with both input and output values, that is, labeled data. We define a method for obtaining the known outputs from the inputs we have. As we improve that method using all the input-output pairs (this is what we call training), we become able to predict output values for new input data. That is supervised learning. We learn the model using the training data and test it on test data it has not seen. In other words, we work with both input and output data.
A typical example of supervised learning is predicting house prices. Another is saying which digit, which animal or which object appears in an image.
Within supervised learning there are two big families, and the difference lies in the nature of y:
- Regression: y is continuous. The canonical example is linear regression. In trading: predicting tomorrow's return, next week's realized volatility, or the expected spread.
- Classification: y is binary or multiclass. The canonical example is logistic regression. In trading: predicting whether tomorrow's close will be above today's, or classifying the market regime as bullish, sideways or panic.
In finance there is one detail that never shows up in generic tutorials and that ruins more backtests than any other mistake: the split between training and test cannot be random. If you shuffle the rows, the model trains on future days and is evaluated on past days, learning relationships that were not available at the moment of trading. The split has to respect the time ordering.
import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
# df has columns Open, High, Low, Close for an asset (for instance, SPY)
df = df.dropna().copy()
# Input variables (X): information available AT TODAY'S CLOSE
df["Open-Close"] = df["Open"] - df["Close"]
df["High-Low"] = df["High"] - df["Low"]
df["Ret_1d"] = np.log(df["Close"] / df["Close"].shift(1))
# Output variable (y): binary label, 1 if tomorrow closes higher
df["y"] = np.where(df["Close"].shift(-1) > df["Close"], 1, 0)
df = df.dropna()
X = df[["Open-Close", "High-Low", "Ret_1d"]]
y = df["y"]
# TIME-BASED split, never random: 70% past for training, 30% future for test
split = int(0.7 * len(df))
X_train, X_test = X[:split], X[split:]
y_train, y_test = y[:split], y[split:]
model = LogisticRegression(max_iter=1000).fit(X_train, y_train)
print("Train accuracy:", accuracy_score(y_train, model.predict(X_train)))
print("Test accuracy :", accuracy_score(y_test, model.predict(X_test)))If train accuracy is much higher than test accuracy, you have overfitting. And if test accuracy hovers around 50% on a balanced binary task, your model knows nothing: it is tossing an expensive coin.