Heartbeat

A small periodic message sent from one node to another (or to a coordinator) signalling that the sender is alive and healthy. Absence of heartbeats for some threshold triggers failover, removal from a load-balancer pool, or alerts.

Distributed systems need to know when a peer has failed. TCP connections can stay 'open' long after the other side has crashed, so without explicit heartbeats, you'd discover failures only when the next request times out, potentially minutes later.

Each node sends a heartbeat every T seconds (typically 1-5s). The receiver tracks the last-seen timestamp. If no heartbeat arrives within K × T (typically K=3), the receiver declares the sender dead. Action: leader election kicks off, the dead node is removed from rotation, alerts fire.

• Fast failure detection (seconds, not minutes)
• Simple to implement and reason about
• Used by basically every cluster manager (Kubernetes, Consul, Zookeeper)

• False positives: network blips cause healthy nodes to be marked dead
• Heartbeat traffic adds up in large clusters (gossip helps but adds complexity)
• Tuning the threshold is a real tradeoff: too short = flapping, too long = slow detection

• WhatsApp →

  Implicit in the WebSocket layer, where ping/pong frames detect dropped connections

• Distributed Cache →

  ZooKeeper monitors cache node liveness via session heartbeats

• Configure thresholds based on real network behavior. 1-second heartbeats with 3-strike-you're-out works for healthy LANs; cellular networks need much more slack.
• Heartbeat-based failover can be wrong: the 'dead' node might just be partitioned. Pair with quorum to avoid split-brain.
• Gossip protocols (Cassandra) scale heartbeats to thousands of nodes, which direct N×N heartbeats can't.