tidaldb/tidal
jordan 6ad8c51cfa
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
fix(cluster): stop discarding signal context on every clustered write
`POST /signals` on a cluster node applied (signal, entity, weight) and
dropped `user_id`/`creator_id` while still answering 204. Everything
user-scoped was silently lost: hard negatives, seen tracking, (user,
creator) interaction weight, preference vector, per-user trending index,
cohort attribution, community forwarding. A clustered deployment looked
healthy, accepted every behavioural signal, and learned nothing — with no
wire evidence of the loss.

This was known and worked around rather than fixed: `dto.rs` documented the
fields as "(standalone path only)", and thepeach's staging tofu pinned its
discover corpus to a STANDALONE instance citing this exact defect
(infra/tofu/envs/staging/svc_api.tf). thepeach's client does populate both
fields (crates/tidaldb-client/src/lib.rs), so the loss was live, not
theoretical.

Cause: `stage_signal_local` called `signal_staged`, which takes no context.
`signal_with_context` (standalone) takes both and runs the side effects
after the base write.

Fix, engine side. Extract the two halves of `signal_with_context` that were
not reusable: `validate_context_entity` (the u32 item-slot guard, which must
reject BEFORE anything is written — a truncated id in a durable Tag::HardNeg
row is a permanent cross-item collision that survives restart) and
`apply_signal_context` (every side effect). `StagedSignal` now optionally
carries the context and applies it in `wait()` AFTER durability, so the
ordering matches the synchronous path and a crash cannot leave a side effect
whose base signal was never logged. `signal_with_context_staged` is the new
entry point; both staged constructors share one admission sequence.

Not routed through the synchronous `signal_with_context` on purpose: that
would serialise every context-carrying write on its own fsync, and since
essentially every real signal carries a user, it would have cost the whole
m11p1 group-commit win on the hot path.

Fix, server side. `stage_signal_local` takes the two ids and ALWAYS uses the
context-carrying stage — it degrades to the plain staged write when both are
None, so there is no second path to keep in step. Both production handlers
(`write_signal`, `sharded_write_signal` in cluster/node.rs, served by
`build_region_router`, which is what `run_seed_join_cluster` and
`run_region_cluster` boot — the deployed RF3 topology) now pass them.

The experimental single-process router (cluster/routes.rs, `SimulatedCluster`
+ scatter_gather) genuinely cannot honour context: its relay applies
(signal, entity, weight). It now REFUSES such a request with 400 naming the
supported route, rather than accepting and discarding. A 204 over a dropped
user_id is the failure mode that caused this.

Verified, not assumed: reverted the fix to context-always-None and the new
differential tests fail exactly as the bug did — observed (false, false,
0.0) against the synchronous path's (false, true, 1.0), i.e. no seen bit and
no interaction weight. Tests are differential (staged vs synchronous end
state, with positive controls so two empty states cannot agree) precisely
because that is the check the original bug got past. Existing suites: 2144
engine + 197 server, zero failures.
2026-09-15 17:48:28 -06:00
..
benches Merge remote-tracking branch 'origin/main' (m11/m12 cluster) into m9/m10 2026-08-03 02:16:04 -06:00
examples feat: add exact qualified-hot ranking API 2026-09-09 09:34:26 -06:00
proptest-regressions feat(m8p10): multi-process cluster mode — scatter-gather, reconcile relay, chaos/UAT suites 2026-06-10 14:07:33 -06:00
src fix(cluster): stop discarding signal context on every clustered write 2026-09-15 17:48:28 -06:00
tests fix(cluster): stop discarding signal context on every clustered write 2026-09-15 17:48:28 -06:00
BUILD.bazel feat: Bazel build, crate docs/ai-lookup, docker images, and engine hardening 2026-06-07 18:29:38 -06:00
build.rs feat: M0p1 runtime skeleton, M0p2 tooling & diagnostics, m1p4 signal ledger 2026-02-20 20:32:00 -07:00
Cargo.lock feat: M0p1 runtime skeleton, M0p2 tooling & diagnostics, m1p4 signal ledger 2026-02-20 20:32:00 -07:00
Cargo.toml Merge remote-tracking branch 'origin/main' (m11/m12 cluster) into m9/m10 2026-08-03 02:16:04 -06:00
CLAUDE.md feat: kubernetes deployment, OpenAPI spec, guides, and docker consolidation 2026-06-09 17:06:34 -06:00