3. Model risk: correlations estimated from noise

Risk 6: The limits of the tools

So far we have been saying that measures describe the future badly. This module asks something more uncomfortable: what if the problem is not that the data is insufficient, but that the very procedure you use to build the portfolio amplifies the errors instead of correcting them?

Model risk: when the tool is the source of the risk

Model risk is the name for the possibility that losses come from having used a model that was wrong, badly calibrated or applied outside its own terrain. It is not a market risk the model failed to see: it is a risk the model created, because the decisions you take depend on its outputs.

It has one particularly treacherous property: it does not show up in the portfolio's statistics. A position badly sized because of a badly estimated correlation looks exactly like a well-sized one, right up to the day the real correlation makes itself known.

A correlation matrix is, above all, noise

Here the figures speak for themselves. With 50 assets you have to estimate 50 variances and 1,225 distinct correlations, because the number of pairs grows roughly with the square of the number of assets. If you do it with a year of daily data, you have 250 observations per asset to pin down 1,275 parameters. The ratio does not work, and it does not work by a wide margin.

The upshot is not that the matrix is approximate: it is that most of its fine structure is sampling noise. If you repeat the exercise with the previous year, many correlations change substantially and quite a few change sign. Neither matrix is the true one: both are photographs of a different noise.

On top of this come two problems you already know from level 5. First, that correlation is not a constant of the asset but a property of the period. Second, that it tends to rise precisely in stress episodes, so that the estimate computed from mostly calm data describes the state of the world in which you need it least.

The optimizer as an error maximizer

Richard Michaud put it more than thirty years ago in an idea that sums up the whole problem: a mean-variance optimizer is, in practice, an estimation-error maximizer. The mechanism is transparent. The procedure looks for the assets with the best expected return, the lowest volatility and the lowest correlation with the rest. If an asset combines those three virtues because its estimate is biased upwards by chance, the optimizer does not detect it: it rewards it, and in proportion to how extreme its error is.

That is why portfolios that are optimal on paper come out so odd: concentrated in two or three positions, with enormous weights, sometimes with negative signs, and radically different if you shift the estimation window by a few months. That instability is not an implementation defect: it is the fingerprint of a procedure that has taken noise for signal.

Marcos López de Prado adds a technical layer: when the covariance matrix contains directions the data barely determines, inverting it, which is what the classic optimizer does under the bonnet, amplifies the errors out of all proportion. Hence his proposals group assets by similarity and allocate in blocks.

Estimating expected returns is far harder than estimating risks

There is an asymmetry here that settles where the effort is worth spending.

• Volatility can be measured better by observing more often. Going from monthly data to daily multiplies the observations over the same period by twenty and reduces the estimation error appreciably.
• The average return does not work that way. Its precision depends on the total time elapsed, not on how many times you look inside it. Sampling every minute of a single year does nothing at all to improve the estimate of the annual mean.

The consequence is harsh: to tell an asset returning 8% a year from one returning 6% with reasonable confidence, both with volatility of 20%, you need decades of data. And the optimizer's most uncertain inputs are precisely those means, which are also the ones its result is most sensitive to. Classic work estimates that an error in expected return costs around ten times as much as an equivalent error in variance.

Why the simple and robust usually wins

Out of all of the above comes a recommendation that sounds like surrender and is not: when the inputs are noisy, a method that ignores part of the information usually beats one that uses all of it. Splitting equally, capping maximum weights, grouping into blocks, not allowing negative positions. All of that destroys theoretical optimality and buys stability, and stability is what translates into out-of-sample results.

The rule worth internalizing is this: prefer the method whose result does not change much if you shift the data window. If re-estimating with six more months makes the proposed portfolio unrecognizable, what you have is not a model, it is a well-presented noise amplifier.

A real case: two hundred and fifty years of data

In 2009 a systematic comparison was published between fourteen portfolio construction models and the simplest alternative imaginable, splitting equally. Across seven different data sets and out of sample, none of the fourteen consistently beat the simple split on return relative to risk.

The most eloquent figure is not that one, but the one that explains why. The authors calculated how much history would be needed for the optimized method genuinely to beat the naive split, and came out with something of the order of 3,000 months for a portfolio of 25 assets, some 250 years, a figure that also grows with the number of assets. The theoretical advantage of the optimum exists, but it sits at a distance in data that no investor is ever going to travel. Meanwhile, whoever splits equally gives up an optimality that was never going to materialize and spares themselves the estimation error entirely.