2. The refutation criterion: what would make you change your mind

Level 6: From reading to decision

A written thesis has a manufacturing defect: it was written by the same person who later judges it. And that person has money on the table. The previous module gave you an explicit bet; this one gives you the only thing that stops that bet turning, without your noticing, into a permanent justification for doing nothing.

A thesis rewrites itself

The mechanism is silent. You buy because the margin is going to rise from 18% to 24%. Two quarters pass, the margin does not move, but the company announces a new product and you are pleased. Two more quarters and the margin is still flat, though now the market is punishing the whole sector and you decide the problem is sector-wide and temporary. The following year the price has fallen and you explain to anyone who asks that the company has a promising product and that the sector is being unfairly penalized.

You have not lied to anybody. The margin thesis died months ago and nobody signed the death certificate. Because you never sold, you never had to admit you had changed your mind: the opinion kept mutating to prop up the position you already held. It is the most expensive bias there is, because it does not feel like a bias, it feels like keeping yourself informed.

Annie Duke puts it from the other side: almost everybody is clear about when to get in and nobody has written down when to get out, so quitting gets decided late and in the heat of the moment, once the pain has already done the job that belonged to a criterion.

Anatomy of a refutation criterion

It is written before you buy, on the same sheet of paper as the thesis, and it has four pieces:

• An observable quantity, which has to measure the driver from line 3. Not the price, not sentiment, not "how things are going": the specific figure your bet depends on.
• A threshold. A figure. Above or below which the reading stops holding up.
• A deadline. A date. Without a date, any thesis stays alive indefinitely because it always just needs more time.
• An agreed consequence. What you do if it triggers: close, halve, stop adding. Written down, not implied.

And one quality test that throws out half the criteria people write: what probability do you give it of triggering? If, looking at it honestly, you would answer "almost none", you have not written a refutation criterion, you have written an alibi. A criterion that only fires if the company goes bust will never make you change your mind, because by then your opinion no longer matters.

A falling price refutes nothing, but it is not noise either

Two claims are worth separating. The first: a fall in the price is not evidence about your thesis, unless your thesis was about the price. Your bet was about a margin, and the margin has not said a word.

The second, which many people skip: a fall is evidence about something else, namely that more people disagree with you, and with more conviction, than you supposed. It does not prove you are wrong, but it is a reasonable ground for looking at the driver again and asking yourself what they are seeing.

"It has fallen, so it is cheaper" is the most expensive way of refuting nothing

That sentence deserves a section of its own, because it looks like discipline and is the opposite. Its problem is one of logical structure: it turns any evidence against you into a buy signal. If it goes up, you were right; if it goes down, it is an opportunity. A thesis in which no possible outcome counts as a refutation is not a thesis.

And there is an arithmetic problem on top of the logical one. The price does not fall in a vacuum: it usually falls because the driver your thesis depended on has disappointed. If the driver deteriorates, value falls too, and it may have fallen further than the price: the gap narrows exactly when you believe it is widening. The only way to know is to redo the reverse reading at the new price. You may well find that it now only demands a margin of 19%, and that your 24% needs a defense it did not need before.

The finishing touch is that averaging down increases the position at precisely the moment your confidence ought to be falling, which is the opposite of what you will see in the next module.

Updating is not surrendering

Tetlock observed that the most accurate forecasters update often and in small steps, while everybody else swings between two symmetrical errors: never moving at all, anchored to the first belief, or swerving with every headline. The refutation criterion is the tool that separates the two, because it defines in advance how much evidence deserves a large move and leaves everything else as minor adjustments to confidence.

And it cuts both ways. Just as it can be refuted, it can be confirmed: if the figure arrives where you said and on the date you said, you have gained information about your own ability to read that driver, and that is worth having for the next twenty decisions.

Worked example (continuing from the previous module)

Thesis: the high-margin segment goes from 30% to 45% of sales in four quarters, and the consolidated margin passes 20%. Criterion written before going in: if by the fourth quarter the segment is not above 36% of sales, or the consolidated margin is not above 19%, the position is closed.

The third quarter arrives. The segment stands at 31.5%, the consolidated margin at 18.1%, and the price has fallen 22%. The tempting reading is that the market is overreacting and that the gap has never been this wide. The honest reading is arithmetic: in nine months the driver has advanced a point and a half of the fifteen it needed, and three remain before the 36% threshold. The criterion has not triggered yet, but the trajectory is already informative, and what follows from it is redoing the reverse reading at the new price and lowering your confidence, not doubling the position.

What is not allowed is the one thing almost everybody does: moving the threshold from 36% to 33% now that the figure is in front of you. That is not updating, it is rewriting the thesis so that it cannot lose.