4. What mathematical optimization actually means

Level 7: Linear algebra and portfolio optimization

Putting limits on an optimizer never improves the theoretical result, and yet almost everybody puts them there.

To understand that apparent contradiction you first have to unpack what optimizing means. The word "optimize" is used very loosely, but in mathematics it has a precise meaning.

What optimizing means, exactly

To optimize is to look for the maximum or minimum value of something within a set of permitted possibilities.

That definition hides two pieces worth naming separately from the start:

• What is maximized or minimized is the objective function.
• The set of permitted possibilities is delimited by the constraints.

In the Markowitz problem, the objective function can be "minimize portfolio variance" or "maximize expected return", depending on which version of the problem you pose.

Constraints, from the basic to the realistic

The typical constraints are that the weights add up to 1 and, if short positions are not allowed, that none is negative.

Additional and considerably more realistic constraints are sometimes added to those basic ones:

• That no asset weighs more than 5%.
• That a sector does not exceed 20% of the total.
• That the portfolio contains only assets with sufficient liquidity.

Why pen and paper stop working

With two or three assets, finding the optimum is literally a geometry exercise. It can be solved with pen and paper or a calculator, and the efficient frontier from the previous module can be traced by hand by combining the variance formulas for two or three assets.

The problem is that this approach does not scale, so look at what is on the table with 500 assets, the full S&P 500 universe:

• 500 weights to decide simultaneously.
• Subject to adding up to 1 and to all the additional constraints you may wish to impose.
• And a 500 by 500 covariance matrix, with the 124,750 cross terms from the previous module, entering the variance calculation for every possible combination.

It is not that the problem is conceptually different from the 2-asset one: it is exactly the same question.

What changes is the scale, because the space of possible weight combinations is so enormous that no person could explore it by hand in a whole lifetime. This is where the computer comes in, not as a luxury, but as a structural requirement of the problem.

How the algorithms walk down the mountain

The algorithms that solve this are known generically as numerical optimization methods. They neither try combinations at random nor enumerate them all, which would be infeasible even for a computer, since the weights are continuous numbers, not a finite set of options.

Instead, they start from an initial combination of weights and adjust it systematically, step by step, in the direction that improves the objective function, until further adjustment no longer improves the result appreciably.

It is similar, allowing for the differences, to walking down a mountain in thick fog. You cannot see the whole valley, but at each point you can feel which way the ground slopes and take a step in that direction. Repeating the process many times, you end up at the bottom without ever having seen the whole landscape at once.

Local minimum and global minimum

An important distinction appears here, between two kinds of valley floor:

• A local minimum is a point that is better than all its immediate neighbors, but not necessarily the best in the whole terrain.
• A global minimum is the best possible point in the entire search space.

In many optimization problems, the algorithm can get trapped in a local minimum without realizing that a deeper valley exists elsewhere on the map.

The property that saves Markowitz

The good news is that the classical Markowitz problem, minimizing variance subject to weights adding up to 1 and no short positions, has a favorable mathematical property called convexity.

In a convex problem, the terrain has a single bowl shape, with no secondary valleys. So any local minimum found is automatically the global minimum.

That property is precisely what makes the Markowitz problem reliably solvable, even with hundreds of assets.

The price of adding constraints

However, that guarantee is easily lost as soon as more realistic constraints or objective functions are added.

Some variants preserve that property and others destroy it completely:

• Requiring that no asset weighs more than 5%, or allowing short positions within certain limits, keeps the problem convex in most common cases.
• Forcing the number of assets in the portfolio to be at most 20, choosing which of the 500, turns the problem into something computationally far harder, where there is no longer any guarantee of finding the global optimum in a reasonable time.

And there is one important side effect of adding constraints. The new, constrained efficient frontier always lies below or at best equal to the theoretical unconstrained frontier.

Put another way: restricting the search space can never improve the optimal result, only worsen it or leave it unchanged.

So why does everyone restrict, if the theoretical optimum can only get worse? Because what you gain from constraints is realism and control over specific risks, not more theoretical return.

One example of that control makes it clear: preventing the optimizer from concentrating 80% of the capital in a single stock because of an estimation artefact.