4. Wrapping up: the checklist, and where to go next

Level 6: From data to decision

This is where the first half of the course closes, the data half, before we go into the six production levels. The right place to gather those six levels into a tool you can use on Monday, and to say clearly what you are still missing and where to get it.

The first six levels, one sentence each

• Level 1, where the data comes from: what each type of vendor gives you, what you are paying for when you pay, and why an identifier that gets recycled destroys an entire history.
• Level 2, the biases that break a backtest: survivorship, look-ahead and choice of period, the three mistakes that produce spectacular results without anything going wrong.
• Level 3, adjustments and corporate actions: why the raw price lies after a split or a dividend, and what a merger does to the continuity of a series.
• Level 4, cleaning and validation: how you detect the impossible, the improbable and the missing, and why deleting is not always the answer.
• Level 5, storage and updating: columnar formats, partitions and an incremental update that neither duplicates nor loses anything.
• And this level: alignment with no future leaking in, alternative data with your feet on the ground, and traceability from end to end.

The checklist

Apply it to any dataset before you believe a backtest run on it, whether it is yours or somebody is showing it to you. The first six points are blocking: if any of them fails, the result is not up for discussion, it is discarded and redone. From the seventh onwards they do not invalidate anything on their own, but every one that fails is a question you will not be able to answer the day the numbers change.

• 1. Does the universe include the companies that went bust, merged or dropped out of the index at each date, or only the ones still there today?
• 2. Is every fundamental joined on its publication date and not on the end of the period it describes?
• 3. Is every macro series stored by version, with the figure that was published back then and not the one revised today?
• 4. Is there at least one day between the signal date and the execution date, and does the return you collect correspond to the stretch after it?
• 5. Does no transformation look at the complete series, no global normalization, no interpolation, no backward fill?
• 6. Are prices adjusted for splits and dividends with factors from the right date?
• 7. Does the original raw file still exist, untouched, with its download date and its cryptographic hash?
• 8. Is the dataset rebuilt by running a whole script from the raw data, with no manual steps?
• 9. Are all the transformations applied written down, in order?
• 10. Are the decisions about doubtful data written down, with the criterion and the alternative that was rejected?
• 11. Is coverage checked period by period, that is, how many symbols there are each year and who disappears?
• 12. Are duplicate keys, future dates, impossible values and extreme jumps all checked?
• 13. Are the identifiers stable over time and do they survive a name or ticker change?
• 14. How many genuinely independent observations are there behind the result, rather than how many rows?
• 15. How many variants were tried before settling on this one, and is that number written down?

The same list, in code

The mechanical points get automated and put in front of the backtest, not behind it.

def puerta(df, universo_pit, notas_decisiones, manifiesto):
    """Runs BEFORE the backtest. If it returns False, nothing runs."""
    controles = {
        "universo_punto_en_el_tiempo": universo_pit is not None,
        "fundamentales_por_publicacion": "fecha_publicacion" in df.columns,
        "senal_desfasada": df["posicion"].equals(df.groupby("simbolo")["senal"].shift(1)),
        "sin_fechas_futuras": df["fecha"].max() <= pd.Timestamp.today(),
        "sin_claves_duplicadas": not df.duplicated(["simbolo", "fecha"]).any(),
        "bruto_con_huella": all("sha256" in f for f in manifiesto["fuentes"]),
        "decisiones_escritas": len(notas_decisiones) > 0,
    }
    for nombre, ok in controles.items():
        print(("OK  " if ok else "FALLA ") + nombre)
    return all(controles.values())

Points 14 and 15 cannot be automated and they are the ones that bring down the most results. They are written by hand in the same file as the rest, before looking at the curve.

registro = {
    "observaciones_independientes": "20 trimestres, no 6000 filas",
    "variantes_probadas": 37,
    "regimen_cubierto": "2016-2025, tipos cero y su normalizacion posterior",
}

What comes next

With this you have a table that can be trusted. What you do not have yet is a system: something that runs on its own every morning, that knows where it left off yesterday, that does not duplicate an order if it restarts halfway through the job, and that raises the alarm when something goes wrong. That is what the next six levels, 7 to 12, are about, and they are the other half of this course.

And getting the data right does not tell you whether your result means anything either. That lives in other places.

• The three no-code courses: Market theory, to understand what a price is and who is on the other side; Macro, to know what environment any idea is exposed to; and Costs, which explains why the gap between the backtest price and the one you actually get eats more strategies than any failed signal ever does.
• The three intermediate ones: Math and statistics, where questions 14 and 15 on the list turn into significance tests, a penalty for multiple testing and out-of-sample validation; Derivatives, to understand options and futures; and Risk and portfolio management, which answers how much to put into each position and how several ideas get combined.
• The other two advanced tracks, because this is one of the three: Technical and Python, which starts from basic programming and goes as far as connecting to a real broker, managing orders and running live positions, and for which these six data levels are the foundation, because everything else rests on the table being honest; and Expectations and edge, which does not carry a single line of code and is advanced by sheer demand, because it is about what is already priced in and where there is room to disagree with it.

A reminder at the halfway point

All this material is education, not financial advice. Nothing you have read is a recommendation to buy or sell any asset, nor is it tailored to your situation, your objectives or your risk tolerance. A flawless dataset removes an entire family of errors; it does not make an idea good, it does not validate a strategy and it does not predict anything. Investing carries the risk of loss, including the total loss of your capital. If you are going to take decisions with real money, do it with your own analysis and, if you need it, with a professional authorized to give it.