System Design: gossip protocols
This course explains gossip protocols: how a large group of machines agrees on facts about itself when no machine is in charge. It starts from an application that coordinates by polling a shared database and a health check, and shows why that design fits its size. The lessons cover how gossip spreads in logarithmic rounds, failure detection with SWIM and indirect probes, which data shapes merge safely, systems such as Consul, Cassandra and Redis Cluster, and cheaper options to try before adopting gossip.
What you'll learn
- ✓Polling intervals as a coordination budget: queries per second times participants
- ✓The gossip loop and why it converges in about log N rounds
- ✓Failure detection with SWIM, indirect probes and refutation by version
- ✓Merge rules that must be commutative, associative and idempotent, as in CRDTs
- ✓How Consul, Serf, Cassandra and Redis Cluster use gossip, and why Kubernetes uses etcd
- ✓Cheaper alternatives to try before adopting a gossip protocol
Coordination without a center
A gossip protocol is how a large group of machines agrees on facts about itself when no machine is in charge. The way to understand it is to look first at what this application does instead.
Read the full lesson free →Curriculum
7 lessonsCoordination without a center
How this application coordinates by polling one shared authority, and why gossip starts where no reachable authority exists.
How gossip spreads
The four-step gossip loop, why random peers give fault tolerance, and why convergence takes about log N rounds.
Failure detection
Why a node cannot tell a dead peer from a slow network, and how SWIM uses indirect probes.
Convergence and its limits
Which data shapes converge under gossip, the merge rules behind CRDTs, and four limits on convergence.
Where you meet this in practice
How Consul, Cassandra and Redis Cluster use gossip, why Kubernetes uses etcd, and how to diagnose failures.
When not to reach for it
The single precondition for gossip, four cheaper alternatives to try first, and the cost of adopting it early.
Putting it together
A recap of the course's main ideas and a challenge to write a method that counts gossip rounds.