4. The random walk: tomorrow does not follow today
Level 5: Time series: why the past does not repeat itself the same way
A share has just risen five days in a row. What is it more likely to do on the sixth?
Almost everyone has an answer. Some say it is due a correction, because it has already risen enough. Others say the trend will continue, because it has been doing so for days.
Under the model you will see in this module, both answers are equally unfounded.
What the model says
The random walk model formalizes an idea that was already introduced intuitively in the market theory course when discussing market efficiency.
The idea is that the price of an asset evolves as the previous period's price plus an unpredictable increment, without that increment depending systematically on what happened in the past.
In simple notation, with no heavy formulae: today's price = yesterday's price + a random change whose expected value is zero (or a small constant drift) and which is not correlated with the changes of previous days.
The direct consequence is that the best point forecast for tomorrow's price, given everything that is known today, is simply today's price.
- It is not today's price plus an extrapolation of the recent trend.
- It is not today's price corrected for a supposed imminent reversion.
The reason is that the part of the change that is not the constant drift is, by construction, noise with no predictable structure.
The connection with autocorrelation
This connects directly with the previous module. The random walk is, in essence, the assertion that the autocorrelation of returns at any lag is statistically indistinguishable from zero.
Suppose you compute the autocorrelation function of the daily returns of a highly liquid share and find that no lag departs significantly from zero, within the reference band of two standard errors seen in the previous module.
That evidence is compatible with the random walk hypothesis, though it is not definitive proof of it, and by extension it is compatible with a weak form of market efficiency: past prices, on their own, contain no exploitable information about future prices.
What the random walk does not assert
It is worth being careful about what the random walk asserts and what it does not assert, because this is where the most common errors of interpretation occur:
- It does not assert that the price will not move. In fact it moves constantly. It asserts that the direction of each individual movement is not predictable from the price history.
- Nor does it assert that the market is completely unpredictable in every sense. As you saw in the previous module, the size of the movements, that is, volatility, may exhibit a predictable temporal structure. It happens even when the direction does not, and it is the volatility clustering of module 6.
A market can be, simultaneously, almost a random walk as regards direction and quite predictable as regards the expected size of its swings.
These two assertions do not contradict each other, they describe two different aspects of the price's behavior.
The two fallacies of runs
A classic error that the random walk helps to diagnose is the gambler's fallacy applied to markets: the belief that after a run of rises, a fall becomes more likely because the asset is somehow exhausted or due for a correction.
Its opposite is the extrapolation fallacy: the belief that after several consecutive rises the trend is going to continue simply because it has already been doing so for several days.
Go back to the scene at the start. Under a pure random walk with zero autocorrelation, a run of five consecutive rises provides no statistical information about what will happen on the sixth day. The probability of rising or falling remains, approximately, the same as on any other day, neither higher out of inertia nor lower out of exhaustion.
The condition almost nobody states
It is important to note the condition attached to that conclusion: this holds under a pure random walk.
Suppose that in a particular asset an autocorrelation genuinely different from zero has been measured, for example +0.2, well above the noise band for the available sample size.
Then that specific asset does not behave as a pure random walk, and a recent run would in fact carry a small amount of real predictive information, however modest.
The key is always the same: measure the specific autocorrelation of the asset in question before assuming either of the two behaviors.
The drift: minuscule by the day, decisive by the year
The random walk usually also includes a drift. It is a small but systematically non-zero expected increment in each period, representing, for example, the risk premium the market demands on average for holding an asset.
That drift is minuscule day by day. If the expected annual return of a diversified portfolio is around 8%, the implied daily drift is of an order of magnitude close to 0.03%, well below the typical daily noise of 1% or more.
But that minuscule drift accumulates. Over 252 days, a small and constant daily drift ends up explaining most of the price's long-term trend, whereas the unpredictable component dominates the short-term variation.
This explains an apparent paradox in the price's behavior. At a daily level the market can look like almost pure noise with no discernible direction, and yet at the level of years it exhibits a clear and consistent upward trend.
Both observations are compatible with the same random walk with drift model. The long-term trend does not come from each individual day being predictable, but from the accumulation of a small and constant bias over a very large number of days.
What it implies for a trading rule
One final implication remains, which connects with the analysis of strategies seen in the market theory course.
Suppose the autocorrelation of an asset's returns is effectively indistinguishable from noise. Then any trading rule based exclusively on past price patterns, of the kind buy after two consecutive rises, or sell after touching a certain level, should not, on average and after allowing for transaction costs, generate a systematic advantage over simply holding the position.
And now the part that is too often read wrongly. This does not mean that no strategy can ever work. It means two things:
- That its justification cannot rest solely on past price patterns without solid statistical evidence of genuine autocorrelation.
- That any positive result found by testing many rules on historical data must be submitted to the overfitting scrutiny you already saw in the backtesting block of the previous course.