To simulate a business

It was about 2014 when I first got excited about a start-up building something called foundationdb, a sort of magic-break-all-the-rules-of-the-time database. I remember seeing them demo it working on a mini rack of raspberrypi-like machines, pulling out cables and killing pi's to show how the database kept working and keeping its guarantees whatever they threw at it.

I worked on a big distributed system at the time, and we were burning thousands of real monies spinning up clusters of machines, poking them, watching them fail, then digging around in a mountain of logs and grafana metrics trying to rebuild a model in our heads of what happened.

How were the foundationdb team making it look so easy?

It wasn't the raspberry pi's. As legend has it, when the team set out to build foundationdb they didn't start by building a database at all, they started building a simulator for the world the database would run in. Almost like a game world for databases, only one where its always storming and the ground gives way under foot.

That game world is the deterministic simulator, it enabled the team to run whole clusters of machines in one process. They could make networks fail, disks fail or corrupt, slow CPU processes on some machines, all in memory and run them as part of a test suite.

This approach in hindsight seems so simple, you go from tests being expensive, slow, impossible to replicate, infrastructure spaghetti nightmares, to fast, expansive in scope and repeatable.

It's still a great idea in 2026, just take a look at tigerbeatledb.

So simulation testing makes building complex systems simpler, allowing us to cheaply and quickly find out how to break things, and then repeat it in a test case.

Why then I have I never encountered this approach in the day-to-day building of business systems?

Do those not need to be correct? Are they not distributed systems with complex unexpected failure scenarios and enormous state spaces?

OK, so if a database needs network, time, disks (etc etc) what do we need to simulate in order to run a business in its own adversarial game world? What is the outside world for a business, the IO or Effects that it wants to cause or that impact it? And could we compress down the logic of a business such that we could run it in a single process?

For an online shop that world might be things like customers, payments, stock, fulfilment, time and emails etc. Our simulator could explore what happens when a customer places an order, then deletes their payment details, only for fillment to fail and retry placing the order 3 days later.

Languages with first class algebraic effects force us to answer part of that explicitly within the type system as we go. They also conveniently invert the normal flow of control of code, allowing us to drive execution of the system directly with a simulator. That could mean stepping through time or some kind of queue of business events like CustomerComplaintRecieved.

As the world tends to ever more code with less and less people able to understand it, perhaps a harness for the world makes more sense than ever.