Deterministic Simulation Testing in Celld
The developers of celld, a runtime for Cloudflare Workers, explain their use of deterministic simulation testing to improve distributed system reliability. By controlling event ordering and timing, the team can reproduce and fix complex bugs that are otherwise difficult to isolate.
Why it matters
Deterministic simulation testing is a critical methodology for ensuring the stability of distributed cloud infrastructure, reducing downtime and development cycles.
We are building celld , a runtime for Cloudflare Workers and Durable Objects applications on your own machines. It can run with an S3-compatible object store as its only external service dependency.
celld is a distributed system. Making distributed systems reliable is hard, partly because we can unintentionally rely on assumptions that don’t hold in practice, even when we know the pitfalls. Peter Deutsch’s The Eight Fallacies of Distributed Computing lists eight such assumptions, including “The network is reliable” and “Latency is zero.”
A bug can depend on a particular sequence of delayed messages, failed writes, and node restarts. Those events can happen in a different order on the next test run, making the failure difficult to reproduce. We need to be able to repeat the failing run so we can investigate the cause and check whether a proposed fix really resolves the problem.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in