A single MySQL primary became a synchronous three-node Galera cluster with a tested cutover
Galera cluster (MariaDB / Percona XtraDB)
Where they started
One primary and one asynchronous replica. A primary failure meant a manual promotion and a period of acknowledged data loss the business had never formally accepted.
The problem, precisely
Synchronous multi-master changes write behaviour: every transaction now waits for the slowest node, and heavy writes can stall the cluster.
What we did
Chose Galera for synchronous certification, then spent the first half of the project on the workload itself — hot spots, missing indexes and an oversized batch job that would have made certification the bottleneck.
Delivered
- Three-node Galera cluster with garbd on a separate host for split-brain protection
- Write-path tuning: index work and batch reshaping so certification does not stall
- Streaming replication flow control tuned and checked under peak load
- Cutover rehearsed twice on a staging copy, then executed with application owners watching
- Node recovery procedure: how to rejoin a node that has been down for a day
Technologies used
Measured result
Every number below was measured and reported to the customer. Where we do not have a measured figure, the entry is a fact rather than a percentage — we would rather be precise than impressive.
- 3 nodes synchronous, single-node failure tolerated
- 0 acknowledged data loss in the new design
- 2.4s measured cutover, executed in the window
- 2 cutover rehearsals before touching production
Synchronous replication moves the bottleneck to your worst write pattern. Fix the workload first, or you will blame the cluster for your own query plan.