1. Why a notebook that works is not a system
Level 7: From the notebook to the process
You have a notebook that works. It downloads prices, computes the signal, sizes the position and, in the last cell, sends the order to the broker. You have run it twenty times and all twenty it did the right thing. The natural conclusion is that you already have a system and all that is left is to leave it running. It is exactly the other way round: what you have is an experiment that has gone well twenty times while somebody was watching it.
The real state is not in the code, it is in the interpreter
A notebook has two different things: the text of the cells, which is what you see and what you share, and the interpreter's memory, which is what actually rules. That memory holds variables created by cells you have already deleted, a series you trimmed by hand an hour ago and a constant you changed just to try something and never put back.
# Cell 3, run at 09:12
capital = 100000
# Cell 3, edited at 09:41 for a quick test and never re-run
capital = 1000
# Cell 9, run at 09:55: it uses what is in memory, not what is written
tamano = calcular_tamano(capital, precio, volatilidad)Nobody opening that file can know which capital the size was computed with. The code says one thousand; the order went out with one hundred thousand. There is no visible error anywhere: there are two truths at once, and the one that reached the market is not the one that is written down.
The execution order is never recorded
In a notebook you run the cells in whatever order suits you: you go down, you go back up, you repeat the middle one, you skip the download because it takes ages. That order is essential information and it is not saved anywhere. The file only keeps an execution number next to each cell, and that number resets the moment you restart the kernel.
The consequence is hard to accept: a notebook does not describe a computation, it describes a session. Two people with the same file can get two different results without either of them having done anything wrong. And you yourself, next week, are one of those two people.
It depends on somebody being there
A notebook takes for granted a human operator who watches. Without realizing it, that operator does five jobs a system has to do on its own:
- Decides when it starts. Opens the laptop and launches it. If that day they are on a train, it does not get launched.
- Interprets the errors. Sees a traceback in red, understands that the data provider has returned a gap and decides whether to carry on or stop.
- Retries. Re-runs the cell that failed, by eye, with no written criterion.
- Checks the result. Looks at the number, finds it reasonable and moves on.
- Remembers. Knows whether it ran yesterday, even though that knowledge only exists inside their head.
Take the operator away and none of that is left. There is no record of what ran yesterday, no way of knowing whether it ran twice, no recovery after a failure and nobody to notice that today it has not run at all. A notebook's silence and its success look far too much alike.
The expensive failure: the same order twice
This is the concrete risk, and it is worth looking at head on:
# Last cell of the notebook
respuesta = broker.enviar_orden(simbolo='SPY', lado='compra', cantidad=120)
print(respuesta)The request hangs for forty seconds. You interrupt the kernel, you see no confirmation, you run the cell again. As far as the broker is concerned, two legitimate and indistinguishable requests have arrived, and now you hold two hundred and forty shares where you wanted one hundred and twenty. The calculation was right; the signal was a single one. What was missing was a mark saying this particular intent has already been attempted, and a notebook has nowhere to put it, because it has no memory between sessions. How that mark is built is the whole of Level 8; here it is enough to understand why the notebook cannot have it.
What a notebook is genuinely good for
None of this means throwing it away. The notebook is an excellent laboratory: looking at data, trying an idea out in ten minutes, drawing a curve. What it is not is a factory. The practical rule is simple: the notebook may read and compute anything it likes, but it must not write anywhere that anybody else depends on, and least of all send orders. The moment a cell produces an effect on the outside world, that cell has stopped belonging to the laboratory.
The healthy way to live with it is the opposite of the usual one: instead of turning the notebook into the program, the notebook imports the program. The functions live in ordinary files, versioned and tested, and the notebook does nothing but call them and plot the result. That way what you explore and what runs tomorrow on the server are literally the same code.
Real case: a risk model that lived in spreadsheets
In 2012 JPMorgan lost more than six billion dollars in the episode known as the London Whale. The bank's own internal report, published in January 2013, describes how the new risk model for that book was operated by hand: data was copied and pasted from one workbook to another, and one of the formulas divided by the sum of two figures instead of by their average, which halved the volatility being computed.
The spreadsheet did not cause the loss on its own, and it would be false to tell it that way: there were enormous positions and highly questionable decisions behind it. But it explains something important, namely why for months nobody saw the real size of what was there. A manual artefact, with no version control, no tests and operated by a person copying and pasting, can hold up a wrong figure for a very long time without the error leaving a single trace. A notebook with a cell that sends orders is exactly the same kind of artefact.