5. Outliers: noise to ignore, or the real signal

Level 3: Descriptive and inferential statistics applied to markets

Take the three worst market days of the last decade out of a backtest and the equity curve turns beautiful.

It also turns into a strategy nobody could ever have executed, because nobody knows in advance which days it is best not to be invested.

Deciding what to do with an extreme data point is, in financial data analysis, one of the most expensive decisions there is. And no formula settles it.

What an outlier formally is

An outlier, or atypical value, is an observation that departs markedly from the rest of the data in the series.

The most widely used formal definition relies on quartiles. If you sort all the data and divide them into four equal parts:

• The first quartile (Q1) is the value below which 25% of the observations fall.
• The third quartile (Q3) is the value below which 75% fall.
• The difference between the two, Q3 minus Q1, is called the interquartile range, and it measures the dispersion of the central half of the data, ignoring the extremes.

A common convention flags as an outlier any observation falling below Q1 minus 1.5 times the interquartile range, or above Q3 plus 1.5 times that same range.

It is a mechanical rule, useful for spotting outlier candidates systematically. But here is the important point of this module: detecting an outlier is not the same thing as deciding what to do with it, and there no formula replaces judgment.

Spurious ones: those to be corrected

There are two categories of outlier in financial data, and confusing them is one of the most expensive errors a quantitative analyst can make.

The first is the spurious outlier: an extreme value that reflects nothing real about the market, but rather a failure in the capture or processing of the data. Typical examples:

• A share split not correctly adjusted in the price history. It produces a 50% fall from one day to the next that never really happened in terms of shareholder value.
• A badly recorded dividend.
• A decimal-point error at the data provider.
• A wrong price captured during a technical halt in trading.

These values must be corrected or removed. They carry no information about the behavior of the asset: they only contaminate any statistic computed afterwards.

Genuine ones: those to be looked at

The second category is the genuine outlier: an extreme value that did happen and does reflect the real behavior of the market, usually associated with an identifiable event.

A 20% fall in a single day driven by widespread panic. A sudden rise after a surprising set of corporate results. The collapse of October 1987, or that of March 2020.

These values are not noise to be cleaned up. They are often the most valuable information in the whole series, and for a concrete reason: extreme events are what determine the real risk of a strategy, the size of the largest possible losses and the validity of the assumptions made by risk management tools.

Removing these outliers from a backtest so that the equity curve looks cleaner is not data cleaning. It is manipulating the evidence to conceal precisely the kind of event an investor needs to understand before risking capital.

Let us go back to the opening example. A strategy that looks extraordinary in a backtest from which the three worst market days of the last decade have been withdrawn is not the same strategy that is really being evaluated. It is a fictional version that could never have been executed in reality.

How to tell them apart

The criterion for distinguishing between the two categories is not statistical, it is domain knowledge: you have to investigate where the data point came from.

Three questions to ask:

• Is there a news item, a macroeconomic event, a corporate announcement or a known crisis that coincides with that date? If so, it is probably a genuine outlier.
• Does the move coincide exactly with the date of a split or of a dividend payment not adjusted in the database? If so, it is probably a processing error.
• Do other independent data providers confirm the same extreme move on the same date? If it only shows up in one source, suspect a technical error at that particular source before a real market event.

A third, intermediate category

There is a third category worth mentioning: genuine outliers that are potentially irrelevant to the question being asked.

For example, when analyzing the behavior of an individual share, an extreme move caused by news specific to that company —a lawsuit, an accounting scandal— is real. But it may say nothing useful about how that share will behave in the future under normal market circumstances, if the event that caused it was truly idiosyncratic and non-recurring.

Here the decision to include it or to treat it separately depends on the aim of the analysis:

• If the aim is to estimate the realistic tail risk of holding that share in a portfolio, it must be included. That kind of idiosyncratic event is also part of the real risk of investing in individual shares.
• If the aim is to isolate the effect of a specific systematic factor —the effect of momentum in general, say, rather than of that particular company—, it may make sense to treat it separately and document it explicitly. Never to hide it without more.

The general rule

The rule that sums up this module is easy to state and hard to follow.

Faced with any outlier, the first question is never how do I remove it so that my statistics look better. It is what caused it, and what that tells me about the system I am trying to understand.

Ignoring a genuine outlier out of statistical discomfort is, frequently, the most direct way of building a strategy that looks robust on paper and breaks in the first real crisis it goes through.