Most rewrites ship late, and some never ship

Every team with an old system has had the conversation. The code is tangled, nobody understands parts of it, and starting again looks so much cleaner. Sometimes it is the right call. This lesson is about why it so often isn’t, and about the one property that separates the safe ways to change a system from the dangerous ones.

What the famous warning says

In April 2000 Joel Spolsky called Netscape’s decision to rewrite its browser “the single worst strategic mistake that any software company can make”. His argument was about knowledge: “When you throw away code and start from scratch, you are throwing away all that knowledge.” Years of bug fixes for odd cases live in the old code, often without a comment to say why.

Two corrections to the way the story is usually told. Spolsky’s essay doesn’t say the rewrite alone killed Netscape. And the long gap between releases wasn’t all rewrite: the decision to replace the old layout code came in October 1998, well after Netscape 4. The rewritten engine, Gecko, later became the engine of Firefox. The lesson is not “never rewrite”; it’s that a rewrite has to relearn everything the old code knew, while the old system keeps changing underneath it.

TSB: the headline and the documents

In April 2018 the UK bank TSB moved its customers to a new platform in a single weekend, and the result was months of disruption. The popular version is “the data migration failed” or “big bang was the mistake”. The official documents say otherwise:

• The Financial Conduct Authority’s Final Notice says the data migration itself was successful. What failed was the new platform, which wasn’t ready to support TSB’s full customer base.
• The independent review by Slaughter and May said a largely single-event migration would not necessarily have been the wrong approach, had the right mitigants been in place.
• Rolling back was essentially impossible once customers were on the new platform.
• Testing of the two data centres working together had been cut back, in a decision taken outside the forum that should have approved it. The FCA says that testing would likely have found the configuration problem that took online banking down.

Read that way, TSB is a story about a one-way step and a check removed in front of it.

Changes are the usual trigger

Google’s SRE book puts it bluntly: “SRE has found that roughly 70% of outages are due to changes in a live system.” The book doesn’t say what that figure rests on. Google’s later workbook does publish data: across thousands of its own postmortems from 2010 to 2017, binary pushes triggered 37% of outages and configuration pushes 31%. That is one company’s data, and it counts triggers, not root causes, so quote both as Google’s findings, not industry statistics. But the direction is clear: the risk is in changing things, and a rewrite is the biggest change there is.

Google’s book on software engineering adds the cost side: “Evolving a system in place is usually cheaper than replacing it with a new one, when turndown costs are included.”

Try it

Sort four real kinds of decision into the ones you could walk back and the ones you couldn’t. That distinction is what every pattern in the rest of this trail is built on.