tidaldb/tidal/src
jordan 6ad8c51cfa
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
fix(cluster): stop discarding signal context on every clustered write
`POST /signals` on a cluster node applied (signal, entity, weight) and
dropped `user_id`/`creator_id` while still answering 204. Everything
user-scoped was silently lost: hard negatives, seen tracking, (user,
creator) interaction weight, preference vector, per-user trending index,
cohort attribution, community forwarding. A clustered deployment looked
healthy, accepted every behavioural signal, and learned nothing — with no
wire evidence of the loss.

This was known and worked around rather than fixed: `dto.rs` documented the
fields as "(standalone path only)", and thepeach's staging tofu pinned its
discover corpus to a STANDALONE instance citing this exact defect
(infra/tofu/envs/staging/svc_api.tf). thepeach's client does populate both
fields (crates/tidaldb-client/src/lib.rs), so the loss was live, not
theoretical.

Cause: `stage_signal_local` called `signal_staged`, which takes no context.
`signal_with_context` (standalone) takes both and runs the side effects
after the base write.

Fix, engine side. Extract the two halves of `signal_with_context` that were
not reusable: `validate_context_entity` (the u32 item-slot guard, which must
reject BEFORE anything is written — a truncated id in a durable Tag::HardNeg
row is a permanent cross-item collision that survives restart) and
`apply_signal_context` (every side effect). `StagedSignal` now optionally
carries the context and applies it in `wait()` AFTER durability, so the
ordering matches the synchronous path and a crash cannot leave a side effect
whose base signal was never logged. `signal_with_context_staged` is the new
entry point; both staged constructors share one admission sequence.

Not routed through the synchronous `signal_with_context` on purpose: that
would serialise every context-carrying write on its own fsync, and since
essentially every real signal carries a user, it would have cost the whole
m11p1 group-commit win on the hot path.

Fix, server side. `stage_signal_local` takes the two ids and ALWAYS uses the
context-carrying stage — it degrades to the plain staged write when both are
None, so there is no second path to keep in step. Both production handlers
(`write_signal`, `sharded_write_signal` in cluster/node.rs, served by
`build_region_router`, which is what `run_seed_join_cluster` and
`run_region_cluster` boot — the deployed RF3 topology) now pass them.

The experimental single-process router (cluster/routes.rs, `SimulatedCluster`
+ scatter_gather) genuinely cannot honour context: its relay applies
(signal, entity, weight). It now REFUSES such a request with 400 naming the
supported route, rather than accepting and discarding. A 204 over a dropped
user_id is the failure mode that caused this.

Verified, not assumed: reverted the fix to context-always-None and the new
differential tests fail exactly as the bug did — observed (false, false,
0.0) against the synchronous path's (false, true, 1.0), i.e. no seen bit and
no interaction weight. Tests are differential (staged vs synchronous end
state, with positive controls so two empty states cannot agree) precisely
because that is the check the original bug got past. Existing suites: 2144
engine + 197 server, zero failures.
2026-09-15 17:48:28 -06:00
..
cohort Merge remote-tracking branch 'origin/main' (m11/m12 cluster) into m9/m10 2026-08-03 02:16:04 -06:00
db fix(cluster): stop discarding signal context on every clustered write 2026-09-15 17:48:28 -06:00
entities vector search: normalize the query, instrument the blob path, expose per-group vector counts 2026-08-30 13:57:36 -06:00
experiment Merge remote-tracking branch 'origin/main' (m11/m12 cluster) into m9/m10 2026-08-03 02:16:04 -06:00
governance fix: M0-M10 code-review pass2 remediation — all 91 findings 2026-06-09 12:21:00 -06:00
load fix: M0-M10 third-pass remediation — durability, replication, and CLI hardening 2026-06-08 10:28:34 -06:00
query feat: add exact qualified-hot ranking API 2026-09-09 09:34:26 -06:00
ranking feat: add exact qualified-hot ranking API 2026-09-09 09:34:26 -06:00
replication harden: validate embeddings before the WAL, fix the reseed-latch leak, run every test suite 2026-08-31 00:46:00 -06:00
schema Merge remote-tracking branch 'origin/main' (m11/m12 cluster) into m9/m10 2026-08-03 02:16:04 -06:00
session Merge remote-tracking branch 'origin/main' (m11/m12 cluster) into m9/m10 2026-08-03 02:16:04 -06:00
signals Merge remote-tracking branch 'origin/main' (m11/m12 cluster) into m9/m10 2026-08-03 02:16:04 -06:00
storage ranking: fix two BLOCKERs in the age-aware sorts, and stop trusting created_at units 2026-08-31 21:31:31 -06:00
testing feat(m11): observability+ops (m11p8) + perf-sweep wave 2 T2 2026-06-13 09:17:49 -06:00
text fix: M0-M10 code-review pass2 remediation — all 91 findings 2026-06-09 12:21:00 -06:00
wal vector search: normalize the query, instrument the blob path, expose per-group vector counts 2026-08-30 13:57:36 -06:00
fault.rs feat(m11): continuous correctness (m11p9) — fault classes, invariant checkers, soak gates, nightly pipeline 2026-06-13 15:23:59 -06:00
lib.rs feat(observability): HTTP metrics, structured logs, dashboard, live tidalctl 2026-08-23 10:31:57 -06:00