2. Which metrics to measure in a trading system

Level 10: Knowing what is going on

Measuring is not storing numbers just in case. A metric only earns its place if there is a decision you would take differently depending on its value. With that filter, a trading system's list comes out short and clear, and it splits into two families that should never be mixed on the same dashboard.

Two families answering different questions

The system metrics answer "is the machinery working?". There are five, and with those five you cover nearly everything:

Data delay: seconds between the data's timestamp and the moment your process has it in memory.
Duration of each task: how long the download, the signal computation, the order sending take.
Rejected orders: how many and for what reason, grouped by cause.
Errors per hour: caught exceptions, by module.
Gap between the signal price and the execution price: the slippage actually paid.

The strategy metrics answer "how much am I making or losing?": position by symbol, gross and net exposure, the day's result and drawdown from peak equity.

Keep the two families on separate dashboards, and not for aesthetics: they are read at different moments. The system ones, every morning before trading; the strategy ones, at the close of the day. Mixing them produces a dashboard of forty numbers in which the eye cannot find the one that matters, which is the debug-level logging problem transferred to charts.

The system ones warn before the strategy loses money

This is the module's central idea. Strategy metrics are lagging: by the time the drawdown from peak worsens, the money has already gone. System metrics are leading: they describe the conditions under which each decision is being taken, and they degrade before the effect reaches the result.

A typical example: the data delay goes from three seconds to ninety over the course of a morning. The strategy is still in the green, there are no rejections and nobody has noticed anything. But the signals are being computed on stale candles, and that misalignment between what the model believes it sees and what the market is doing will eventually show up in the result days later. If you only look at the result, you find out when there is nothing left to do.

import time
from collections import defaultdict

class Metricas:
    def __init__(self):
        self.contadores = defaultdict(float)
        self.muestras = defaultdict(list)

    def _clave(self, nombre, etiquetas):
        return (nombre, tuple(sorted(etiquetas.items())))

    def contar(self, nombre, valor=1.0, **etiquetas):
        self.contadores[self._clave(nombre, etiquetas)] += valor

    def observar(self, nombre, valor, **etiquetas):
        self.muestras[self._clave(nombre, etiquetas)].append(valor)

    def percentil(self, nombre, p, **etiquetas):
        datos = sorted(self.muestras[self._clave(nombre, etiquetas)])
        if not datos:
            return None
        return datos[min(len(datos) - 1, int(p / 100.0 * len(datos)))]

M = Metricas()

class Cronometro:
    def __init__(self, tarea):
        self.tarea = tarea

    def __enter__(self):
        self.t0 = time.perf_counter()
        return self

    def __exit__(self, tipo, valor, traza):
        M.observar("tarea_segundos", time.perf_counter() - self.t0, tarea=self.tarea)
        if tipo is not None:
            M.contar("errores", tarea=self.tarea)
        return False

Always measure percentiles, not means. The mean duration of a task is flattened by the good days; what breaks a system is the tail, the 95th percentile that one Tuesday eats the entire execution window.

Real slippage against expected slippage is the thermometer

Of all the metrics, the one that detects a silent degradation earliest is the comparison between the slippage you pay and the one your simulation assumed. Your backtest used a number, typically a few basis points. That number is a hypothesis, and in production you have the chance to test it every day.

def registrar_deslizamiento(orden, ejecucion):
    signo = 1.0 if orden["lado"] == "compra" else -1.0
    base = orden["precio_senal"]
    real_pb = signo * (ejecucion["precio"] - base) / base * 10000.0
    esperado_pb = orden["deslizamiento_esperado_pb"]
    M.observar("deslizamiento_pb", real_pb, simbolo=orden["simbolo"])
    M.observar("exceso_pb", real_pb - esperado_pb, simbolo=orden["simbolo"])
    registrar(logging.INFO, "deslizamiento", simbolo=orden["simbolo"],
              precio_senal=base, precio_ejecucion=ejecucion["precio"],
              real_pb=round(real_pb, 2), esperado_pb=esperado_pb)

If the average excess moves away from zero in a sustained way, something has shifted: the name's liquidity, the time you send at, the relative size of your order or the execution route. None of those things shows up legibly in the day's result; in this metric they show up immediately.

The strategy ones say how much it hurts, and must be read together

Gross exposure sums the absolute value of every position; net sums them with sign. Reading only one leads to false conclusions: a portfolio with gross of 180,000 and net of 4,000 on equity of 100,000 is not "almost in cash", it is heavily leveraged and almost market-neutral, with a risk that comes not from direction but from its pairs ceasing to behave as you expected.

def instantanea_estrategia(posiciones, precios, efectivo, maximo_previo):
    bruta = sum(abs(c) * precios[s] for s, c in posiciones.items())
    neta = sum(c * precios[s] for s, c in posiciones.items())
    patrimonio = efectivo + neta
    maximo = max(maximo_previo, patrimonio)
    caida = 0.0 if maximo <= 0 else (patrimonio / maximo) - 1.0
    M.observar("exposicion_bruta", bruta)
    M.observar("exposicion_neta", neta)
    M.observar("caida_desde_maximo", caida)
    return patrimonio, maximo, caida

Worked scenario: three points that were not in the model

A scenario invented for teaching purposes. A daily strategy runs for a month three percentage points below simulation. The signals match the backtest's one for one and there was not a single rejection. The excess slippage metric tells the story: an average of 17 basis points against the 4 assumed, concentrated in the two least liquid names in the basket and in the orders sent in the first minute of the session. The strategy was not broken; the execution was. Without that metric, the team would have reoptimized the model's parameters and made both things worse: it would still be paying the same execution cost and now on top of a signal fitted to one particular month. The fix was to split the orders in the two illiquid names and to delay the send by fifteen minutes, until the book settles. This is education, not financial advice: the number that matters is not 17 or 4, but the habit of comparing what you pay with what you assumed.