1. Where data comes from, and what free really costs

Level 1: Where they come from and what they carry inside

There is a question almost nobody asks before writing the first line of a strategy: who touched this number before it reached my hard drive. A price does not appear by spontaneous generation inside a CSV. It has passed through a chain of hands, and every hand charges something, filters something or delays something. Understanding that chain is what separates a backtest that means something from a backtest that only means you know how to use pandas.

The four links in the chain

• The exchange. NYSE, Nasdaq, Cboe, BME, Eurex. This is where the event happens: two orders cross and a price is born. The exchange is the original owner of that data and licenses it out. For the big exchange operators, selling information is a major line of business, of the same order of magnitude as trading revenue, and that explains a good deal of what follows.
• The consolidator. In the United States there is a regulated infrastructure that collects trades and best quotes from every trading venue and publishes a single tape. In Europe, consolidation has historically been more fragmented, and for years there was no equivalent consolidated tape.
• The vendor. Buys or licenses the data, normalizes it, attaches identifiers, cleans it, wraps it in an API and sells it to you. This is where everything from the expensive terminals to the monthly subscription APIs lives.
• You. The last link: the one who pays, or the one who accepts the terms of whoever is not charging.

Every hop adds latency, cost and one more opportunity to introduce an error. When two sources disagree, it is almost always because they have traveled different routes down this chain. It is also worth knowing which link you have plugged into: a vendor that consolidates on its own account from several direct feeds will not give you the same thing as one that resells the official consolidated tape, even if both sell you a column with the same name.

The license constrains you more than the price does

A beginner's mistake is to compare vendors by euros per month. What actually decides whether you can use a source is its terms:

• Redistribution. Almost no cheap license lets you show the data to third parties. If your project is a public dashboard or a newsletter with prices in it, you are redistributing, even if you charge nothing.
• Commercial versus personal use. Many free APIs are explicitly for personal, non-commercial use. Trading your own money usually fits; managing somebody else's almost never does.
• Requests per minute and per day. The rate limit defines your entire architecture: on five requests per minute you are not going to refresh 500 index constituents every minute, full stop.
• Retention and derived works. Some licenses forbid you from storing the history or publishing figures derived from it.
• Non-display use. If an algorithm consumes the data without a human ever looking at it, many exchanges apply specific fees, and they tend to be higher than the display fee.

What you pay when you do not pay

Free never means free: it means the payment is not in money. The bill arrives in five currencies:

• Delay. Fifteen minutes is the standard for delayed data. For a daily strategy it may not matter; for anything intraday it is lethal.
• Incomplete coverage. Small caps are missing, non-US markets are missing and, above all, companies that stopped trading are missing.
• No revision history. You get today's snapshot, not the one that was visible back then.
• No proper timestamp. Many free sources do not tell you when the data was published, only which period it refers to.
• No guarantee of continuity. Nobody owes you anything. The service can change format or disappear on a Tuesday.

The first thing any serious script should do is measure the real delay of its source instead of trusting the brochure:

import pandas as pd
import requests

# Placeholder: the key never goes hard-coded in the source or pushed to a repository.
CLAVE = "tu_clave_aqui"

def ultima_marca(simbolo):
    """Timestamp of the most recent candle the source admits to having."""
    r = requests.get(
        "https://api.proveedor-ejemplo.com/v1/intradia",
        params={"symbol": simbolo, "interval": "1min", "apikey": CLAVE},
        timeout=10,
    )
    r.raise_for_status()
    velas = pd.DataFrame(r.json()["values"])
    velas["datetime"] = pd.to_datetime(velas["datetime"], utc=True)
    return velas["datetime"].max()

ultima = ultima_marca("AAPL")
ahora = pd.Timestamp.utcnow()
print("Ultimo dato:", ultima)
print("Retraso aparente en minutos:", (ahora - ultima).total_seconds() / 60)

And the second is to audit the historical series before building anything on top of it. A gap of three consecutive sessions in the middle of a year is not a public holiday: it is an outage that nobody ever backfilled.

precios = pd.read_csv("aapl_gratis.csv", parse_dates=["date"], index_col="date").sort_index()

esperados = pd.bdate_range(precios.index.min(), precios.index.max())
faltan = esperados.difference(precios.index)

print("Sesiones esperadas:", len(esperados))
print("Sesiones recibidas:", len(precios))
print("Ausentes (incluye festivos):", len(faltan))

# Blocks of consecutive missing days: the long ones smell like an outage.
huecos = pd.Series(faltan)
bloques = (huecos.diff() != pd.Timedelta(days=1)).cumsum()
resumen = huecos.groupby(bloques).agg(["min", "max", "count"])
print(resumen[resumen["count"] >= 3])

A real case: the Tuesdays that broke thousands of scripts

In May 2017, Yahoo Finance withdrew without prior warning the historical CSV download endpoint that had been the community's de facto standard for a decade. Any project that depended on the Yahoo function in pandas-datareader stopped working overnight; the reaction was a swarm of unofficial reverse-engineered libraries, one of which ended up being yfinance.

Less than a year later, in April 2018, Quandl announced that its WIKI dataset of US daily equity prices would no longer be actively maintained. It was not deleted: it stayed there, frozen. That is the worse of the two failures, because it is silent: a script pointing at WIKI kept downloading, kept returning a DataFrame and kept producing a backtest that looked perfectly normal, except that it ran on data that no longer moved forward or got corrected.

Neither source breached anything: there was no contract to breach. That is exactly the currency in which free data is paid for. All this material is education, not financial advice.