Commit Graph

4 Commits

Author SHA1 Message Date
jordan.washburn
fe711870be feat: M8 phases 7-10 — gRPC transport, cluster server, scatter-gather, multi-node UAT
Delivers the distributed fabric's network layer and HTTP cluster surface:

**m8p7: tidal-net crate (gRPC transport)**
- GrpcTransport implementing Transport trait via tonic 0.12
- Per-peer circuit breaker (Closed/Open/HalfOpen), mutual TLS via rustls
- Boxed error types (clippy-clean), graceful mutex recovery, debug_assert
  against calling block_on from tokio context
- Proto: WalShipping service (ShipSegment, StreamSegments stub, Heartbeat)
- 19 tests: contract, mTLS, reconnection, multi-node UAT, benchmarks

**m8p8: cluster subcommand + HTTP routes**
- ClusterState wrapping SimulatedCluster with region name mapping
- Routes: /health, /cluster/status, /cluster/promote, /partition, /heal
- Data routes: /items, /embeddings, /signals, /feed, /search (region-aware)
- Ranking profiles wired through ClusterConfig to all cluster nodes
- Topology YAML config, docker/cluster/Dockerfile (ENTRYPOINT+CMD, non-root)

**m8p9: scatter-gather query routing**
- Entity-sharded writes via Knuth multiplicative hash
- Scatter-gather RETRIEVE and SEARCH with deadline propagation (50ms-5ms)
- Partial failure: degraded=true with unavailable_shards metadata
- 6 tests: distribution, determinism, multi-shard retrieve, degraded
  partial results, deadline propagation, scatter-gather search

**m8p10: gRPC transport integration tests**
- 8 tests over real gRPC: replication convergence, idempotent replay,
  mixed signals, 3-node fan-out, partition/heal, degraded follower, perf
- Documented as tier-2 (in-process+gRPC); tier-3 multi-process pending
2026-04-11 13:51:08 -06:00
jordan
eca7765e8d fix: heal_region re-delivers missed WAL batches so partitioned followers converge immediately after heal
- Extract redeliver_missed(tx, db, log) helper into cluster_transport.rs
- heal_region now removes partition then immediately ships any missed
  batch-log entries to the healed follower's channel
- await_convergence refactored to call the same helper (no logic change)
- tidal-server: reload_text_index before search in cluster mode
- tidal-server: write_signal returns Result instead of panicking on unknown signal
- tidal-server: leader shows lag_events=0 (writes directly, no receiver thread)
- tidal-server: fix cluster mode error propagation (ServerError::from)
- docs/runbooks/cluster.md: add full cluster operations runbook
- docker/: add Dockerfile for containerised cluster deployment
- README.md: add tidal-server HTTP API getting-started section
- Split oversized source files per CODING_GUIDELINES §9

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-25 11:57:01 -07:00
jordan
51b4d1bbd6 fix: repair tidal-server compilation and verify standalone HTTP server
Fix 9 compilation errors across tidal-server and testing/cluster.rs so
that `cargo run -p tidal-server -- standalone` works end-to-end.

Bugs fixed:
- cluster.rs: wrong return types `RetrieveResult`→`Results` and
  `SearchResult`→`SearchResults` on retrieve/search helpers
- state.rs: `RegionId` imported from private path; now uses
  `tidaldb::replication::RegionId`
- state.rs: missing `Ok()` wrapper on `ServerState::cluster()` return
- state.rs: cluster match arms returned `TidalError` where `ServerError`
  required; added `.map_err(ServerError::from)` on write_item,
  write_embedding, retrieve, search
- error.rs: `Result<T>` alias lacked default E param; callers in router
  used two-arg form `Result<T, AppError>` — changed to
  `Result<T, E = ServerError>`
- router.rs: `with_state()` called before cluster routes were added,
  making `app` `Router<()>`; restructured to call `with_state` once at end
- router.rs: `TidalErrorWrapper(TidalError)` used to map `QueryError`;
  fixed with `|e| TidalErrorWrapper(e.into())`
- router.rs: `Search::limit()` takes `u32` but code cast to `usize`
- router.rs: `bm25_score`/`semantic_score` are `f32` in SearchResultItem
  but `f64` in response struct; added `.map(f64::from)` conversion

Also split cluster.rs into cluster.rs + cluster_transport.rs to stay
under the 600-line limit required by CODING_GUIDELINES §9.

Verified all README curl examples work:
  POST /items, POST /embeddings, POST /signals, GET /feed, GET /search,
  GET /health all return correct HTTP status codes and JSON responses.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-25 01:45:09 -07:00
jordan
98bdc18a49 feat: add iknowyou app + complete M8 replication extensions + Aeries agents/skills
- applications/iknowyou: new Next.js chat application with persona-aware conversations,
  briefing API, cohort logic, vLLM streaming, and sidebar navigation
- tidal M8: add replication control plane (control.rs), tenant migration state machine
  (migration.rs), tenant/upgrade coordinators, cluster/fault test harnesses
- tidal M8 tests: expand m8p2/m8p3/m8p4 test suites; add m8p5_multitenancy and m8_uat
- tidal db: split replication_ops out of db/mod.rs (was 647 lines, now 574)
- .claude: add kai-park, kaya-osei, mira-vasquez agents; add aeries-design-architect,
  aeries-fullstack-engineer, aeries-product-visionary skills
- docs: update ROADMAP.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 21:09:11 -07:00