tidaldb/tidal-server/src
jordan 5b3cfe59d9 fix(cluster): never boot-pull against the topology leader post-election
This is the defect that kept tidaldb-0 looping, and the per-key instrument named it
exactly. Live group 1 held:

  keys: [[0, 13540659], [1, 13540652], [2, 13540661]]

Current leader tidaldb-2 is key 2 and the group was fully converged there at
13540661. Key 1 is a STALE position left from when tidaldb-1 led the group.

`node.rs`'s follower boot self-heal pulled `shard_of_region(leader)` where `leader`
is the BOOT TOPOLOGY leader — dead config after any election, as the topology
comment itself says. For group 1 that is tidaldb-1, i.e. key 1, so the pull went
out at 13540652 + 1 = 13540653, which tidaldb-1's WAL had compacted below (earliest
13540657). Permanent `snapshot-required` → marker latch → `reseed_self_restart` →
repeat. The old comment claimed "term fencing + later election traffic rescue it";
they do not, because the refusal re-latches faster than the rescue converges.

The boot pull is now confined to the genuine topology era (durable term 0), where
the topology leader IS authoritative. Post-election, convergence is driven by the
heartbeat path (which carries the CURRENT leader's frontier and works on an idle
cluster since m12p5) and by the receiver's gap detection on real ship traffic —
both keyed to the leader actually shipping, never a historical one.

Gates: mp_follower_reseeds_via_snapshot_after_compaction and
mp_multi_group_node_converges_after_reseeding_several_groups both pass;
mp_quarantined_node_reseeds_without_wipe still passes, which is the term-0 path
this change deliberately leaves intact.
2026-08-21 11:35:31 -06:00
..
cluster fix(cluster): never boot-pull against the topology leader post-election 2026-08-21 11:35:31 -06:00
cluster_config.rs feat(m9/m10/p1): community policy engine, signal revocation, agent capability boundaries, feedback loop, metrics instrumentation 2026-03-16 05:59:42 -06:00
config.rs feat(m12): multi-vector user preference modeling + ANN candidate-gen 2026-06-23 09:52:36 -06:00
dto.rs feat(m12p4): sharded ingestion — scatter-gather pool + cross-shard unified reads (L4) 2026-06-14 15:17:35 -06:00
error.rs docs: withdraw the pre-release "not ready for production" disclaimer 2026-07-30 19:03:34 -06:00
health.rs fix: M0-M10 third-pass remediation — durability, replication, and CLI hardening 2026-06-08 10:28:34 -06:00
lib.rs feat(m11): membership, snapshot install, and reseed (m11p5) 2026-06-12 19:55:54 -06:00
main.rs docs: withdraw the pre-release "not ready for production" disclaimer 2026-07-30 19:03:34 -06:00
offload.rs fix(m12-rc13): read-SLA collapse + WAL_RETENTION_SEGMENTS 16 + tidalctl S3 DR 2026-06-17 15:47:37 -06:00
openapi.rs docs: withdraw the pre-release "not ready for production" disclaimer 2026-07-30 19:03:34 -06:00
router.rs fix(cluster): reconcile could not run at production scale 2026-08-18 10:07:19 -06:00
scatter_gather.rs fix(cluster): size the read fan-out budget for the transport it crosses 2026-08-17 20:28:21 -06:00
self_exit.rs feat(m11): membership, snapshot install, and reseed (m11p5) 2026-06-12 19:55:54 -06:00
state.rs feat(m12): vector retrieval G1/G2 — recall harness, ANN in RETRIEVE, index tuning 2026-06-14 11:07:09 -06:00