Reconciles two independently-developed lines from base 006d3d0:
ours — M9/M10 community layers, retroactive purge + re-materialization,
signal revocation, agent capability boundaries, P1 feedback loop,
reason labels, instrumented metrics
theirs — M11/M12 cluster mode (tidal-net gRPC transport, tidal-server
cluster/scatter-gather, tidal-stress), multi-vector preference,
ANN candidate-gen, warm-tier day buckets, keyed signal snapshots
Notable semantic resolutions:
* storage::keys::Tag — both sides allocated 0x0E..0x11 for different
records. Kept theirs' 0x0E..0x1A (shipped on-disk format) and renumbered
ours to 0x1B..0x1E (CommunityMembership/Revocation/PurgeManifest/
CommunityLeave); Tag::ALL grown to 30 so the contiguity drift guard holds.
* ranking executor — took theirs' rewrite (SignalReadPlan pre-pass, keyed
SignalKey snapshots, Result-returning reads, finalize()) and re-applied
ours' M10 read suppression at the chokepoints it introduced:
single_signal_score, score_hot/trending/controversial,
CreatorEngagementRate, and the Stage-4 boost loop.
* signals::warm — theirs' day-bucket/read-time-rotation rewrite, with ours'
subtract_bucket and Clone extended to the new day tier; ours' test split
kept (warm/tests.rs, warm/proptests.rs) carrying theirs' updated bodies.
* db::signals — kept ours' contribution-logging try_cohort_attribution in
signal_dispatch.rs and theirs' event-time try_update_preference_vector;
dropped the superseded duplicates.
* db::mod / from_parts — theirs' constructors, with ours' purge/
re-materialization/revocation/community/skip-counter fields and restart
rebuilds; from_parts kept in its own file per the 600-line guideline.
* schema::validation::builders — ours' module split with theirs' expanded
tests; policy validation runs both sides' checks (read-signal lists +
profile overrides, then the zero-duration limit guard).
* feedback Unhide no longer writes a -1.0 "hide" signal: theirs' engine
rejects negative weights (spec §8). Reverses index state only, matching
every other undo action.
* SessionState::new is now the single construction path (gains
overrides_rejected/default_profile); AuditEntry gains kind on the
deserialize path, inferred from the accepted flag as before.
* Removed tidal/src/replication/tcp_transport.rs and its test: never
declared in replication/mod.rs on either branch, so it had never
compiled and nothing referenced it. Superseded by tidal-net's
GrpcTransport.
Verified: cargo clippy -p tidaldb (lib) clean; --all-targets compiles for
tidaldb/tidal-net/tidal-server/tidal-stress; 2094/2094 lib tests and the
integration suite pass except m8p3_reconcile_production's two CRDT-count
assertions, which fail identically on MERGE_HEAD (pre-existing).
tidalctl cannot build locally: its aws-sdk deps need rustc 1.91.1, local
toolchain is 1.91.0.
Root cause: after a checkpoint-based snapshot install the engine WAL is empty,
so wal_term_mark() reports tail_term=0. decide_join compares (tail_term, frontier)
lexicographically — tail_term FIRST — so 0 < leader_term classifies the reseeded
shard ReseedRequired on EVERY boot regardless of the correctly-seeded frontier,
re-latching the marker and self-restarting forever. Observed live on tidaldb-2:
30 CrashLoopBackOff restarts, leader tidaldb-1 term 5, baseline=536647, the
frontier seeded correctly (from_seqno=536647) yet the loop persists because the
(tail_term, frontier) compare never reaches the frontier.
Fix 1 (already in tree): seed the post-open frontier from sentinel.snapshot_seq,
not last_wal_seq() (which a checkpoint restore leaves at 0).
Fix 2 (loop-breaker): durably synthesize the artifact's kind-3 TERM_MARKER WAL
record in the post-open reseed seed, at the artifact's captured term + the
reseed-leader region (threaded through an extended 18-byte install sentinel,
back-compat with 10/8-byte). Makes wal_term_mark() truthful on this boot AND
every reboot (blob records are NOT checkpoint-filtered on recovery), so
decide_join returns Clean. Truthful, not a bypass: the artifact IS the leader's
authoritative state at (term, seq); a genuinely-divergent node (no install
sentinel) still surfaces tail_term > term -> Quarantine. Crash-idempotent via a
monotonic-by-term guard.
Fix 3: node-level reseed-restart coordinator — the single process-wide exit fires
once, only after every hosted shard requests a restart or a bounded grace
elapses, so one shard's self-restart never aborts a co-hosted sibling's
in-flight install (S>1). No-op on the S=1 production topology.
Fix 4: is_ready() returns 503 while any reseed marker (SnapshotRequired or
Quarantine) is latched, closing the plain-restart serve-while-behind gap;
readiness is bounded staleness, not "ready the instant the process is up".
Tests: decide_join loop/fix/bounded-reseed unit; install-sentinel 18-byte
round-trip + back-compat; engine durability (term marker survives a checkpoint
advanced past it + crash-reopen); reseed-restart gate (5 cases); reseed_install
carries the term. Verified: cluster_reseed 4/4 (zero-loss rolling restart x2,
quarantine reseed, failover oracle), reseed_install 3/3, m12_reseed_term_marker
4/4, cluster_membership mp_idle/mp_dns/mp_remove x2, tidal-server lib 154/154.
mp_scale_3_5_3 and mp_seed_join_snapshot_catchup OOM on this host (22GB colima
VM); their /health/startup failure is process-down, not the is_ready path Fix 4
touches.
Boot now LOADS the per-slot HNSW graph instead of rebuilding it. Clean
shutdown writes {data_dir}/vector/<kind>__<slot>.usearch; the next open loads
it when it matches the durable corpus (seconds), falling back to a full rebuild
only when the graph is missing/stale/corrupt. Eliminates the multi-minute boot
rebuild (~50-70 min at 1M/1536-D) that let the WAL compact past a restarting
node and triggered the reseed cascade.
Graceful SIGTERM now actually runs the close: bounded_drain caps the post-signal
HTTP drain (TIDAL_SHUTDOWN_DRAIN_MS, default 15s) then runs the deterministic
close regardless — sibling keep-alive connections no longer block the drain past
the k8s 60s grace into a SIGKILL (which cannot run Drop). ClusterNode and
ShardReplica::shutdown are now &self (db handle is an ArcSwapOption) so the close
fires even when a stuck connection task holds an Arc.
Fix USearch insert to be a true upsert (remove+add): it was unconditional add,
which a multi:false index rejects on a reseeding follower's post-snapshot WAL
replay -> applied_events stalls -> catch-up deadlock -> unrecoverable cluster.
Also: circuit-breaker peer last-contact tracking; real k3s 1536-dim deploy +
recall findings (recall@10 0.9869, read p99 8.71ms @ 200rps @ 100k) in
docs/profiling/m12-cluster-deploy-findings.md; new tidal-stress k8s jobs and
m12p6 graph-persistence + SIGTERM tier-3 regression tests.
Replace ScoredCandidate.signal_snapshot Vec<(String,f64)> with
SmallVec<[(SignalKey,f64); 4]> where SignalKey is Static(&'static str)
| Owned(Arc<str>):
- Compile-time-constant labels (sort bases, relevance, co_engagement,
preference_affinity) -> Static: zero allocation, pointer-copy clone.
- Dynamic {signal}_boost/_penalty/_decay labels built once per query in a
RuleLabels hoist (was format! per-candidate-per-rule), shared by Arc.
Cohort rescore hoisted the same way.
- scored accumulator pre-sized to candidates.len().
- Owned Strings rebuilt only at the two response-assembly sites (<= limit).
Byte-identical output: full 1896-test lib suite green. Measured win
(cargo bench --bench ranking, committed-base vs working-tree):
score_200_hot 23.89->21.04us (-11.9%), score_200_trending
27.66->25.85us (-6.5%), score_200_full_pipeline 27.88->26.97us (-3.3%).
smallvec promoted from the lock to a direct dep (no new dependency
surface). perf-sweep doc updated; T2 (per-term DashMap collapse) next.
Resolves every finding in docs/reviews/M0-M10-code-review-2026-06-08-pass2.md
across the engine, network, server, and CLI crates: session restore,
replication/CRDT, WAL format and recovery, storage indexes, query/ranking
executors, cohort/community governance, and scatter-gather routing.
Adds regression tests:
- review_pass2_creator_search_filter
- review_pass2_d_replication
- review_pass2_query_for_session
- review_pass2_storage_indexes_bitmap_cache
- review_pass2_zone_a_sessions
Verified: cargo clippy -D warnings and full test suite green across all crates.
Resolves the 142 findings from tidal/docs/reviews/CODE_REVIEW_m0-m10.md across
the engine, server, net, and CLI surfaces:
- WAL/session-journal durability, checkpoint format, and crash-recovery hardening
- Replication shipper/receiver, tenant isolation, and migration paths
- Cluster scatter-gather, router, standalone server + health/offload endpoints
- tidalctl refactored into command modules with JSON output and WAL-state tooling
- Cohort, governance, signal-ledger, and vector-registry correctness fixes
- Expanded UAT/integration/durability test coverage across all milestones
- m0p3: CONTRIBUTING.md with run-samples checklist, all 4 examples
(quickstart, cli_embedding, axum_embedding, actix_embedding), doc-test
coverage for every public API surface
- m1p5: TidalDb public API — write_item, signal, read_decay_score,
read_windowed_count, read_velocity; StorageBox enum routing memory vs
fjall; WalSender/WalHandleWriter bridge; WAL replay on open
- Periodic checkpoint: 30s background thread for persistent+schema mode;
FjallBackend::Clone (O(1), fjall::Keyspace is ref-counted); graceful
shutdown via Arc<AtomicBool> + join before final checkpoint
- ROADMAP.md: M0 and M1 fully marked COMPLETE (341 tests passing)
- Milestone 2 planning scaffolding added under docs/planning/milestone-2/
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
