Cluster engineering notes
Written by the engineers doing the work. No sponsored posts, no thought-leadership, no article that could have been generated without ever touching a cluster.
5 published · RSS
-
Backups you have never restored are not backups
The backup job going green every night is the most reassuring and least informative signal in infrastructure. Here is a method for turning it into a number you can quote, and the failure modes that hide behind a successful job.
-
Kubernetes DRA: what actually changes for GPU clusters
Dynamic Resource Allocation replaces the device plugin model with something closer to how GPUs really work — but the operational questions it answers are narrower than the marketing suggests. Here is what changes, and what does not.
-
PostgreSQL HA: Patroni, repmgr and pgpool, compared honestly
These three tools are frequently compared as if they solve the same problem. They do not. One manages cluster state, one manages replication, and one is a router that people mistake for a failover mechanism.
-
Ceph CRUSH map mistakes we keep finding
Durability in Ceph comes from the CRUSH map, not from the replication number. A three-replica pool whose copies land on three OSDs in the same host is a one-copy pool with extra ceremony. These are the configurations we find most often.
-
Why your HA cluster has never actually failed over
Fencing is not a feature of high availability — it is the precondition for it. A cluster that cannot isolate a failed peer will, on a split, run both sides and write to the same data. Here is how to find out whether yours can.
More pieces are in progress. The resources page has the checklist and the runbooks in the meantime.
Have a topic you want covered?
If you are dealing with something specific and you would read an honest write-up of it, tell us. That is how most of the published list got there.