tidaldb/tidal-server
jordan 12c7edc374 fix(cluster): the frontier pair has one writer, and 0 is not a commit index
`TidalDBClusterQuorumLag` sat CRITICAL all session against the live three-voter
cluster while every region reported lag 0 and every per-peer ship queue was
empty. Two independent defects fed it:

- `observe_ship` bumped `relay_last_seq` on every batch ship while
  `relay_durable_seq` only moved when a signal write completed. The two are
  documented as a subtractable pair, so a shipping-but-not-committing node
  reported the whole relay log (13.3M events) as quorum lag. The ship path now
  feeds only its own per-peer queue-depth gauge; the pair has one writer.
- `set_frontier_gauges` published `CommitIndex::committed()` verbatim, but that
  returns 0 as a SENTINEL for "no quorum information in this term yet". It now
  publishes both halves or neither, and every satisfied `await_quorum` -- not
  just signal writes -- refreshes them, so item and embedding workloads keep
  the pair live.

Regression test asserts a busy ship loop leaves both halves at 0 (lag 0, not
13.3M) and that the single writer still moves them together.

Also excludes the `tmp` emptyDir from velero fs-backup: three 0-byte
PodVolumeBackups a night whose only other outcome is failing the whole fleet
backup when a scratch file vanishes mid-snapshot.
2026-08-18 09:50:19 -06:00
..
benches feat(m11): cluster security (m11p7) + perf instrumentation floor 2026-06-13 01:25:35 -06:00
config feat: kubernetes deployment, OpenAPI spec, guides, and docker consolidation 2026-06-09 17:06:34 -06:00
src fix(cluster): the frontier pair has one writer, and 0 is not a commit index 2026-08-18 09:50:19 -06:00
tests fleet remediation: make the workspace gate runnable, then fix what it caught 2026-08-16 12:38:14 -06:00
BUILD.bazel feat: Bazel build, crate docs/ai-lookup, docker images, and engine hardening 2026-06-07 18:29:38 -06:00
Cargo.toml feat(m11): continuous correctness (m11p9) — fault classes, invariant checkers, soak gates, nightly pipeline 2026-06-13 15:23:59 -06:00