1. Survivorship bias: the graveyard you never see

Level 2: The biases that break a backtest

You download today's list of S&P 500 constituents, pull twenty years of prices for those five hundred tickers and build the backtest. It is the first line of code in almost every amateur backtest in existence, and with it you have already lost. Not because of a programming fault: the script runs, the table has the right shape and the equity curve goes up. You have lost because the sample you are measuring on was selected using information that nobody had in 2006.

The sample reaches you already filtered

For a company to appear in an index's constituent list today, it has to have met a very demanding condition: to still exist, to still be listed and to still be large and liquid enough to deserve a place. Applying that list backwards means your 2006 universe consists exclusively of companies that we know, from the future, were going to make it to today alive.

The ones that did not make it are absent, and they vanished from your sample by very different routes:

• Bankruptcy or liquidation. The company stops trading and the shareholder is left with very little or with nothing at all.
• Takeover or merger. Another company buys it and the ticker is extinguished, sometimes at a premium and sometimes at half price.
• Removal from the index. The committee takes it out for size, for liquidity or for failing requirements, almost always after a collapse.
• Change of market. It moves to the over-the-counter market or switches venue, and your vendor simply stops following it.

A large index turns over on the order of twenty constituents a year; over two decades that means several hundred additions and deletions, and not one of those that left by the back door appears in your download.

The filter never points both ways

The temptation is to think that deletions are randomly distributed and the effect washes out. They are not randomly distributed: the main route out of an index is having done badly. Systematically removing the ones that did badly leaves a sample whose average return is inflated by construction, and the inflation is not a simple shift, because it also erases the left tail of the distribution: the ninety percent drawdowns disappear, and with them the real volatility, drawdown and skew.

There is a nuance worth keeping in mind, because it is the most reasonable argument against all this: some of the deletions are acquisitions at a premium, and those do push the other way. That is true, and it is not enough. In aggregate the net effect is clearly positive, and there is one detail that makes it worse: the bias interacts with your strategy's criterion. If you buy whatever is cheapest on book value, or whatever has been most beaten up over the past year, you are asking for precisely the candidates to disappear, and the current universe has already taken them off your plate.

Building the universe with a date

The solution is not statistical, it is a data problem: you need a historical table of additions and deletions, and you need to recompute the universe at every decision date.

import pandas as pd

# One row per membership: ticker, fecha_alta, fecha_baja (NaT if still in)
historico = pd.read_csv(
    "sp500_composicion.csv",
    parse_dates=["fecha_alta", "fecha_baja"],
)

def universo_en(fecha):
    """Tickers that were in the index THAT day, not today's ones."""
    dentro = (historico["fecha_alta"] <= fecha) & (
        historico["fecha_baja"].isna() | (historico["fecha_baja"] > fecha)
    )
    return set(historico.loc[dentro, "ticker"])

fechas = pd.date_range("2006-01-31", "2025-12-31", freq="ME")
universos = {f: universo_en(f) for f in fechas}

primero, ultimo = universos[fechas[0]], universos[fechas[-1]]
print(len(primero), len(ultimo), len(primero & ultimo))
# the overlap between the two extremes is far smaller than it looks

Note the NaT condition. Companies still in the index have an empty fecha_baja, and any comparison against NaT returns False, so without that isna() you would be throwing every one of them out. It is the most common mistake when writing this function and it raises no exception: it just hands you a universe that keeps getting smaller.

Delistings have returns too

Rebuilding the universe is not enough if your prices stop on the day the company is delisted. If the code does pct_change() and then dropna(), the position evaporates at its last known price and the final collapse is never accounted for. You need the delisting return.

retornos = precios.pct_change()

for _, fila in bajas.iterrows():
    tk = fila["ticker"]
    dia = retornos.index[retornos.index <= fila["fecha_baja"]][-1]
    if fila["motivo"] == "exclusion_o_quiebra":
        # liquidation or move to the gray market: very negative exit return
        retornos.loc[dia, tk] = fila["retorno_baja"]
    elif fila["motivo"] == "opa":
        # you collect the offer price, not zero
        retornos.loc[dia, tk] = fila["precio_oferta"] / fila["ultimo_precio"] - 1

Shumway's classic work on delisting bias in US price databases put the average return on performance-related delistings at around minus thirty percent on the traditional exchanges, and at worse figures still on the electronic market. Ignoring that number amounts to assuming that companies going bankrupt get sold at market price the day before.

A real case: how much fake return is being measured

The ground on which this bias has been measured best is not equities but funds, because the commercial databases used to delete closed funds. The literature of the nineties, with Malkiel and with Elton, Gruber and Blake among the reference names, put the excess return introduced by the disappearance of dead funds at an order of magnitude running from under a point to around a point and a half a year. In hedge fund databases the estimates are considerably larger, on the order of several points a year, because there you also have managers who only report when it suits them.

Carry that over to an equity backtest with the current universe applied backwards, crossed on top of that with a criterion that hunts for companies in difficulty, and the order of magnitude stops being decorative: it is perfectly capable of turning a mediocre strategy into one that appears to beat the market comfortably. The operating rule is simple and admits no shortcuts: if your vendor does not give you historical composition and data on delisted companies, you do not have the data to run this backtest.