tidaldb/tidal-server
jordan 5b3cfe59d9 fix(cluster): never boot-pull against the topology leader post-election
This is the defect that kept tidaldb-0 looping, and the per-key instrument named it
exactly. Live group 1 held:

  keys: [[0, 13540659], [1, 13540652], [2, 13540661]]

Current leader tidaldb-2 is key 2 and the group was fully converged there at
13540661. Key 1 is a STALE position left from when tidaldb-1 led the group.

`node.rs`'s follower boot self-heal pulled `shard_of_region(leader)` where `leader`
is the BOOT TOPOLOGY leader — dead config after any election, as the topology
comment itself says. For group 1 that is tidaldb-1, i.e. key 1, so the pull went
out at 13540652 + 1 = 13540653, which tidaldb-1's WAL had compacted below (earliest
13540657). Permanent `snapshot-required` → marker latch → `reseed_self_restart` →
repeat. The old comment claimed "term fencing + later election traffic rescue it";
they do not, because the refusal re-latches faster than the rescue converges.

The boot pull is now confined to the genuine topology era (durable term 0), where
the topology leader IS authoritative. Post-election, convergence is driven by the
heartbeat path (which carries the CURRENT leader's frontier and works on an idle
cluster since m12p5) and by the receiver's gap detection on real ship traffic —
both keyed to the leader actually shipping, never a historical one.

Gates: mp_follower_reseeds_via_snapshot_after_compaction and
mp_multi_group_node_converges_after_reseeding_several_groups both pass;
mp_quarantined_node_reseeds_without_wipe still passes, which is the term-0 path
this change deliberately leaves intact.
2026-08-21 11:35:31 -06:00
..
benches feat(m11): cluster security (m11p7) + perf instrumentation floor 2026-06-13 01:25:35 -06:00
config feat: kubernetes deployment, OpenAPI spec, guides, and docker consolidation 2026-06-09 17:06:34 -06:00
src fix(cluster): never boot-pull against the topology leader post-election 2026-08-21 11:35:31 -06:00
tests fix(cluster): readiness must prove convergence, not merely lack a marker 2026-08-21 10:41:38 -06:00
BUILD.bazel feat: Bazel build, crate docs/ai-lookup, docker images, and engine hardening 2026-06-07 18:29:38 -06:00
Cargo.toml feat(m11): continuous correctness (m11p9) — fault classes, invariant checkers, soak gates, nightly pipeline 2026-06-13 15:23:59 -06:00