Commit Graph

5 Commits

Author SHA1 Message Date
jordan
cdbe9cb453 Merge remote-tracking branch 'origin/main' (m11/m12 cluster) into m9/m10
Reconciles two independently-developed lines from base 006d3d0:
  ours   — M9/M10 community layers, retroactive purge + re-materialization,
           signal revocation, agent capability boundaries, P1 feedback loop,
           reason labels, instrumented metrics
  theirs — M11/M12 cluster mode (tidal-net gRPC transport, tidal-server
           cluster/scatter-gather, tidal-stress), multi-vector preference,
           ANN candidate-gen, warm-tier day buckets, keyed signal snapshots

Notable semantic resolutions:

* storage::keys::Tag — both sides allocated 0x0E..0x11 for different
  records. Kept theirs' 0x0E..0x1A (shipped on-disk format) and renumbered
  ours to 0x1B..0x1E (CommunityMembership/Revocation/PurgeManifest/
  CommunityLeave); Tag::ALL grown to 30 so the contiguity drift guard holds.

* ranking executor — took theirs' rewrite (SignalReadPlan pre-pass, keyed
  SignalKey snapshots, Result-returning reads, finalize()) and re-applied
  ours' M10 read suppression at the chokepoints it introduced:
  single_signal_score, score_hot/trending/controversial,
  CreatorEngagementRate, and the Stage-4 boost loop.

* signals::warm — theirs' day-bucket/read-time-rotation rewrite, with ours'
  subtract_bucket and Clone extended to the new day tier; ours' test split
  kept (warm/tests.rs, warm/proptests.rs) carrying theirs' updated bodies.

* db::signals — kept ours' contribution-logging try_cohort_attribution in
  signal_dispatch.rs and theirs' event-time try_update_preference_vector;
  dropped the superseded duplicates.

* db::mod / from_parts — theirs' constructors, with ours' purge/
  re-materialization/revocation/community/skip-counter fields and restart
  rebuilds; from_parts kept in its own file per the 600-line guideline.

* schema::validation::builders — ours' module split with theirs' expanded
  tests; policy validation runs both sides' checks (read-signal lists +
  profile overrides, then the zero-duration limit guard).

* feedback Unhide no longer writes a -1.0 "hide" signal: theirs' engine
  rejects negative weights (spec §8). Reverses index state only, matching
  every other undo action.

* SessionState::new is now the single construction path (gains
  overrides_rejected/default_profile); AuditEntry gains kind on the
  deserialize path, inferred from the accepted flag as before.

* Removed tidal/src/replication/tcp_transport.rs and its test: never
  declared in replication/mod.rs on either branch, so it had never
  compiled and nothing referenced it. Superseded by tidal-net's
  GrpcTransport.

Verified: cargo clippy -p tidaldb (lib) clean; --all-targets compiles for
tidaldb/tidal-net/tidal-server/tidal-stress; 2094/2094 lib tests and the
integration suite pass except m8p3_reconcile_production's two CRDT-count
assertions, which fail identically on MERGE_HEAD (pre-existing).
tidalctl cannot build locally: its aws-sdk deps need rustc 1.91.1, local
toolchain is 1.91.0.
2026-08-03 02:16:04 -06:00
jx12n
b55ad70141 fix: M0-M10 third-pass remediation — durability, replication, and CLI hardening
Resolves the 142 findings from tidal/docs/reviews/CODE_REVIEW_m0-m10.md across
the engine, server, net, and CLI surfaces:

- WAL/session-journal durability, checkpoint format, and crash-recovery hardening
- Replication shipper/receiver, tenant isolation, and migration paths
- Cluster scatter-gather, router, standalone server + health/offload endpoints
- tidalctl refactored into command modules with JSON output and WAL-state tooling
- Cohort, governance, signal-ledger, and vector-registry correctness fixes
- Expanded UAT/integration/durability test coverage across all milestones
2026-06-08 10:28:34 -06:00
jx12n
3bcfb3c576 feat: Bazel build, crate docs/ai-lookup, docker images, and engine hardening
- Add BUILD.bazel across tidal, tidal-net, tidal-server, tidalctl for bzlmod build
- Add tidal/ crate docs (README, CHANGELOG, CONTRIBUTING, AGENTS, CLAUDE, API, ARCHITECTURE) and ai-lookup reference
- Add docker standalone/cluster/deploy images, compose, and prometheus config
- Harden WAL (batch format, writer, dedup, diagnostics), text syncer/collectors, and vector registry
- Expand tidalctl CLI and tests; restructure WAL/visibility integration test suites
- Refine tidal-net transport/client/server and tidal-server cluster/scatter-gather
2026-06-07 18:29:38 -06:00
jordan
fd95dfc2be fix: restore session metadata and annotations after crash recovery
deserialize_start_record now returns the full (session_id, user_id,
started_at_ns, metadata) tuple — the metadata bytes were already written
by serialize_start_record but silently discarded on read.

restore_session_wal_events looks up the persisted start record in storage
for each open session and uses the deserialized metadata instead of
HashMap::new(), so fields like {"tool":"planner"} survive a crash.

The signal replay loop no longer discards _annotation — annotations are
now pushed into state.annotations during WAL replay, restoring preference
hints like "more jazz today" so FOR SESSION ranking works post-restart.

Two new integration tests in session_durability.rs verify both fixes
against a real persistent store with simulated crash (drop without
close_session). session/serde.rs split into serde/mod.rs + serde/start_record.rs
to satisfy the 600-line limit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-22 00:12:58 -07:00
jordan
192c473f55 feat: complete Milestone 5 — full-text search, RRF fusion, and creator search
- M5p1: BM25 text indexing via Tantivy with background syncer (0.26ms @ 10K docs)
- M5p2: RRF fusion layer combining BM25 + ANN scores (46µs @ 1K candidates)
- M5p3: unified Search query API (8-stage pipeline, BM25 + vector + ranking)
- M5p4: creator text + vector indexing and creator search executor (< 20ms @ 200 creators)
- Refactor db/mod.rs into focused sub-modules (creators, items, sessions, signals, etc.)
- Decompose monolithic files into directory modules (query/executor, ranking/diversity, etc.)
- Split brute.rs → brute/mod.rs + brute/tests.rs; extract search executor helpers
- Add benches: fusion, search, session, text_index
- Add M5 UAT test suites (m5_uat, m5_search, m5p4_creator_search, text_index)
- Update blog posts, roadmap, content strategy, and M5 planning docs
- Add tmp/ and .claude/worktrees/ to .gitignore

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-21 23:53:16 -07:00