67a175e19a
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
The status aggregator probed every peer's /cluster/status/local with NO credential. That route is token-gated, so on any cluster with TIDAL_API_KEY set each peer answered 401, and every peer row collapsed to the honest-unknown placeholder: reachable false, partitioned true, applied_events null, lag_events null, version "". Only the OWN region survived, because it is served in-process with no HTTP hop. The result: the one surface an operator reads to clear the N/N+1 version skew before a rolling upgrade - and the deploy runbook's own step 4 - reported the whole cluster partitioned while it was perfectly healthy, with every peer's version blank so the skew check was blind. Observed on the GKE cluster: all three pods answered /health 200 under leader tidaldb-1, replication applied, and a curl between the exact same pod FQDNs returned 200, while /cluster/status insisted both siblings were unreachable. security::bearer_from_env documents this precise trap - a node that "dials an authenticated peer with NO credential" - and count_alive_other_voters already attaches the bearer. This forwards the CALLER's Authorization header instead of reaching for creds.bearer(), matching the relayed-operator-hop convention that /cluster/promote already uses, so a weakly-authenticated caller cannot borrow the node's own credential to read peers it could not read directly. Why it escaped: every other multi-process test runs with no TIDAL_API_KEY, where a credential-less probe succeeds - including cluster_multiproc's all-reachable assertion. The new test carries the key. Verified differential: it fails on the reverted code with exactly the observed shape (us-east reachable, both peers null/false/empty) and passes with the fix. cluster_multiproc still 5/5. |
||
|---|---|---|
| .. | ||
| support | ||
| cluster_chaos.rs | ||
| cluster_cross_shard_reads.rs | ||
| cluster_e2e.rs | ||
| cluster_election.rs | ||
| cluster_faults.rs | ||
| cluster_graph_persistence.rs | ||
| cluster_grpc.rs | ||
| cluster_lifecycle.rs | ||
| cluster_membership.rs | ||
| cluster_multiproc.rs | ||
| cluster_poison_embedding.rs | ||
| cluster_quorum.rs | ||
| cluster_region.rs | ||
| cluster_reseed.rs | ||
| cluster_routes.rs | ||
| cluster_runbook.rs | ||
| cluster_security.rs | ||
| cluster_sharding.rs | ||
| exact_rank.rs | ||
| middleware.rs | ||
| reseed_install.rs | ||
| standalone_offload.rs | ||
| standalone.rs | ||
| vector_search.rs | ||