1. Testing a trading system is harder than it looks

Level 12: Testing, deploying and taking the step

You arrive here with a system that already exists: a deterministic structure with no hidden effects, idempotent state governed by a state machine, a scheduler that respects the calendar, logging, metrics, alerts and reconciliation, and kill switches for the known failure modes. None of that gets explained again. The question that opens this last level is a different and uncomfortable one: how do you prove all of that works before the market proves it to you with money on the table.

Why this is not tested the way a website is tested

A decent test needs three things: a reproducible input, a unit isolated from the world and a cheap failure. A trading system breaks all three. The market does not deliver the same data twice, so if your test downloads prices it is no longer a test, it is an experiment: when it goes red you will not know whether you have broken the code or whether the market has behaved differently. The piece you want to test talks to a broker that is not yours, one you cannot restart or ask to reject an order at 10:31 to see what your code does. And failure does not cost a support ticket, it costs a position.

From that come this module's four decisions, and none of them is optional: isolate the pure logic, replace the broker with a double you control, provoke the rare cases deliberately and compare each version with the previous one on the same data.

Pure logic is tested with frozen data

The part that decides does not read from disk, does not call an API and does not look at the clock. It takes data and state, returns proposed orders, and with the same input it always returns the same output. If your decision function checks the time internally, you do not have a test: you have a result with an expiry date. The data is written by hand into the repository, with numbers chosen to make the case obvious.

import pandas as pd
from estrategia import decidir

CONFIG = {"ventana": 3, "unidades": 10}

def indice(n):
    return pd.date_range("2024-03-01", periods=n, freq="B", tz="UTC")

def test_el_cruce_dispara_una_sola_compra():
    velas = pd.DataFrame({"cierre": [100.0, 101.0, 99.0, 103.0, 104.0]}, index=indice(5))
    ordenes = decidir(velas, posicion=0, config=CONFIG)
    assert [(o.lado, o.cantidad) for o in ordenes] == [("compra", 10)]

def test_sin_senal_no_manda_nada():
    velas = pd.DataFrame({"cierre": [100.0] * 5}, index=indice(5))
    assert decidir(velas, posicion=0, config=CONFIG) == []

Look at the second test. Almost everybody checks that the signal fires and forgets to check that it does not fire when it should not. In production the frequent case is the second one, and a spurious order costs exactly as much as a good one.

A fake broker that misbehaves on purpose

You do not test against the real broker: it is slow, it has rate limits and its rejections cannot be ordered to measure. You test against a double that implements the same interface and to which you dictate the script. Its value is not in returning "filled", but in returning what the real one returns on a bad Tuesday.

class BrokerFalso:
    def __init__(self, guion):
        self.guion = list(guion)
        self.enviadas = []

    def enviar(self, orden):
        self.enviadas.append(orden)
        if not self.guion:
            raise AssertionError("el sistema mando mas ordenes de las previstas")
        return self.guion.pop(0)

def test_la_ejecucion_parcial_no_infla_la_posicion():
    broker = BrokerFalso([
        {"estado": "parcial", "pedido": 10, "ejecutado": 4, "id": "A1"},
        {"estado": "rechazada", "motivo": "fuera de horario", "id": "A2"},
    ])
    motor = Motor(broker, posicion=0)
    motor.ciclo(senal="compra 10")
    assert motor.posicion == 4
    motor.ciclo(senal="compra 6")
    assert motor.posicion == 4
    assert len(broker.enviadas) == 2

That double is also the place where you check Level 8's idempotency: call the same cycle twice with the same intent identifier and verify that the broker received one order, not two.

The rare cases are the normal case

• Partial fill. You ask for ten, you get four. Your position is no longer what your code believes.
• Rejected order. Funds, price out of band, market closed, rate limit. A "no" is an answer too and has to be handled.
• Gap in the data. A candle is missing, or two arrive with the same timestamp. Your rolling window is computed over fewer observations than you think and nobody warns you.
• Reply that takes too long. The worst: you do not know whether the order arrived. The mandatory case is timeout followed by querying the real status, never automatic resending.
• Repeated confirmation. The broker announces the same fill twice. If you add them up, your accounting doubles and Level 10's reconciliation spends the day shouting.

Comparing the new version with the previous one

The last family of tests does not check that the result is correct, but that it has not changed without you asking. You store the current version's output over a fixed stretch of data and every subsequent change is measured against it.

import json, pathlib

REFERENCIA = pathlib.Path("pruebas/referencia_2019_2021.json")

def test_las_decisiones_no_cambian_sin_avisar():
    salida = [o.como_dict() for o in reproducir("datos/tramo_2019_2021.parquet")]
    esperado = json.loads(REFERENCIA.read_text(encoding="utf-8"))
    assert salida == esperado, "cambian las decisiones: revisa antes de tocar la referencia"

When that test goes red there are two legitimate outcomes: either you have broken something, or you have changed the strategy on purpose and you update the reference in the same commit that explains it. Updating it because it was annoying is not one of them.

Worked scenario

This is not a historical case, it is an example put together with concrete figures. A system slices large orders into lots. A refactoring moves the lot-size rounding from one module to another. All 140 unit tests pass. The comparison test fails on 3 of 512 decisions in the reference stretch: a size of 97 units now gets sliced into 50 and 47 instead of 49 and 48. It looks cosmetic. It is not: the same rounding applied to closing the position left an orphan unit, and that unit showed up every few weeks as a discrepancy in the daily reconciliation, with its corresponding alert at seven in the morning. 0.6% of decisions different and not one unit test able to see it.