4. Type I and type II errors: false positives and false negatives when evaluating a strategy

Level 6: Hypothesis testing and its traps in backtesting

There are two ways of getting it wrong when judging a strategy, and only one of them shows up on your bank statement.

That asymmetry is what decides, in practice, where the bar should be set. But to see it you first have to separate the two kinds of failure quite carefully.

The false positive and what it costs

Every hypothesis test can fail in two distinct ways, and in the context of evaluating a strategy it is worth being absolutely clear about both, because they have very different consequences for your money.

A type I error is a false positive: you reject H0, concluding that there is a real edge, when in fact H0 was true and there was no edge at all.

In trading, this translates into allocating real capital to a strategy that is fundamentally noise. Sooner or later that non-existent "edge" evaporates in live trading, and if you were also using leverage, the disappointment can be severe.

The false negative and what it costs

A type II error is a false negative: you fail to reject H0, concluding that there is not enough evidence, when in fact H1 was true and a genuine edge did exist.

Here the cost is of a different nature. You discard a good idea because your test was not sensitive enough to detect it, with too little data or too much background noise.

It is an opportunity cost. Less dramatic than losing money already deployed, but just as real.

What the threshold controls exactly

The significance level you choose — the threshold we conventionally called 0.05 in module 2 — directly controls the rate of type I errors.

Saying "I use a threshold of 0.05" is equivalent to saying this: if I applied this same rule to a great many strategies that truly have no edge, roughly 5% of them would still clear my filter, by pure chance.

That figure, 5% of false positives built into the method itself, is the seed of the multiple comparisons problem you will tackle in the next module.

There is no threshold that wins on both sides

Between the two kinds of error there is a structural tension, and certainly not a free lunch:

• If you tighten the threshold, demanding a p-value below 0.01 instead of 0.05, you protect yourself better from false positives. But you mechanically increase the probability of also rejecting strategies with a real but modest or noisy edge, that is, more type II errors.
• If you loosen the threshold the opposite happens: fewer missed opportunities, but more noise slipping through as if it were signal.

There is no threshold that minimizes both errors at once. Choosing where to place it is a decision, not an automatic calculation.

Power: what it takes to detect something faint

The power of a test is a concept worth handling even if only qualitatively. It is the probability that the test correctly detects a real edge, given that the edge genuinely exists.

Power moves in two fairly intuitive directions:

• It increases the more independent data you have.
• It decreases when the real edge is small, or when returns are very noisy.

This explains a frustrating and very real phenomenon. Test a subtle idea, with a genuinely small edge, over only two years of data on a single illiquid stock, and it will probably fail the test.

But note the reason for the failure. It does not fail because the idea is bad, but because the test, as framed, lacks the power needed to detect something so faint with so little information.

Discarding the idea at that point would quite likely be a type II error dressed up as rigor.

The two errors do not weigh the same

In live trading practice, the two errors do not weigh the same, and that is the asymmetry announced at the start.

A type I error is usually more costly because it involves capital already deployed and real losses, sometimes amplified by leverage. A type II error is a missed opportunity, but not a hole in the account.

This asymmetry is exactly why many quantitative teams prefer to be conservative, using stricter significance thresholds, in capital allocation decisions. They gladly accept more missed opportunities in exchange for fewer disastrous "discoveries".

The four cells and where to spread the risk

To fix the four possibilities in mind it helps to think of them as a two-by-two table, with one axis for reality and one for the verdict:

• On the reality axis: "there is no real edge" versus "there is a real edge".
• On the axis of what the test says: "the test says there is no edge" versus "the test says there is an edge".

Crossing both axes gives you four cells: correctly discarding noise, a type I error or false positive, a type II error or false negative, and correctly detecting a genuine edge.

Any decision about where to place the significance threshold is, at bottom, a decision about how to distribute risk across those four cells.

One concrete application makes it clear. A quant desk may demand a stricter threshold, say 0.01, for any strategy that will trade with leverage above 2x, while keeping the standard 0.05 threshold for small, unlevered allocations.

The logic is not arbitrary: the more expensive a type I error is in a given context, the more sense it makes to pay the price of more type II errors to avoid it.