jordan c97aaa8e5b fleet remediation: make the workspace gate runnable, then fix what it caught

`cargo test --workspace` could not run at all: dependency resolution failed with
"aws-types@1.3.16 requires rustc 1.91.1" on the 1.91.0 default toolchain, so the
gate the project documents was dead. Making it run exposed a compile break and
two wrong tests that had been invisible for months. Now green end to end:
143 suites, 3155 tests, exit 0.

Toolchain
- rust-toolchain.toml pins the DEV toolchain to 1.91.1. The published MSRV stays
  `rust-version = "1.91"` (the engine builds on 1.91.0); only tidalctl's AWS SDK
  chain needs the patch release, and it now declares that itself.

Consumer crates migrated to the current engine API (clean cutover)
- iknowyou-engine: `AgentPolicy` gained five m10 read/profile-override fields;
  the literal now spreads `..AgentPolicy::default()` as the engine's own doc
  example does, so future fields do not break it again.
- forage-engine: `RetrieveResult` gained p1 `reasons`. The app builds its own
  candidate pool, so it now tags what it knows: PreferenceMatch for the
  preference-vector blend, SemanticMatch (with the seed item) for
  similar-to-saved, ExplorationBudget for pinned discoveries.
- forage-engine: `url_to_item_id` folded into the u32 item universe. The engine
  narrows item IDs to a u32 slot in durable per-user state and rejects anything
  above u32::MAX rather than alias two items forever, so every add_item with a
  64-bit FNV hash failed. 9 of 28 smoke tests were failing on this alone.
- forage-engine: bridge items read the top-2 preference CLUSTERS via
  `query_vectors`, not the single centroid from `preference_vectors().get()`.
  Since m12 that accessor returns only the strongest cluster, so a tech+jazz user
  whose interests split into two clusters looked single-interest and never
  bridged. Falls back to top-2 dimensions when a user has one cluster.

Reconcile tests corrected to the shipped contract
- tidal/tests/m8p3_reconcile_production.rs asserted `3 + 5 == 8` for a windowed
  count after heal. `take_crdt_snapshot` deliberately keys signal contributions
  to ONE canonical contributor (ShardId::SINGLE) because signals are relayed from
  a single writer, so per-node attribution double-counted every replicated event
  on every reconcile. Merge is therefore LWW on (last_update_ns, score) plus
  PN-counter per-node max: nodes converge on the more complete accumulator. The
  old expectation was asserting the bug that fix removed.
- Rewrote to assert convergence, count survival (not 0), and no inflation, and
  added `repeated_reconcile_of_converged_nodes_does_not_creep` - the regression
  guard for the creep itself, which nothing covered.

Pre-commit hook unified
- hooks/pre-commit dropped `-D warnings`: each crate's `[lints]` table is the
  source of truth (`clippy::all`/`unwrap_used` deny, `pedantic` warn), and the
  flag promoted ~58 deliberate pedantic warnings in integration tests to errors,
  making every Rust commit impossible.
- It now lints all five tidal crates instead of path-matching `tidal/`, which
  silently skipped tidal-server, tidal-net, tidal-stress, tidalctl and
  applications/ - the rot above lived in exactly those crates. Ported the
  CODING_GUIDELINES file-length, println, and unsafe-SAFETY checks from the
  divergent untracked copy that this replaces.
- CONTRIBUTING.md now documents the real commands and the toolchain/MSRV split.

Fleet recovery and soak
- scripts/restore-fleet.sh: the fail-closed selective restore, promoted out of an
  ignored tmp/ directory into the repository. Preflights retained storage,
  digest-pinned images, parked state, and aggregate plus per-PV-node scheduler
  headroom before the first scale; writes a durable transcript under
  tmp/restore-logs/ with structured start/error/rollback/complete events.
- k8s manifests park the standalone store, the RF3 cluster, and the soak monitor
  at zero replicas with restore-fleet.sh as the only supported scale-up path.
- soak-eval/soak-watch and the nightly CronJob fail closed on stale or missing
  restart evidence instead of silently skipping the restart-aware half of the gate.
- docs/ops/capacity-planning.md corrects the RAM envelope to the real hot-tier
  formula and separates analytic totals from the measured process envelope.

2026-08-16 12:38:14 -06:00

3.3 KiB

Raw Blame History

Contributing to tidalDB

Quick Start

# Clone the repo
git clone https://github.com/orchard9/tidaldb && cd tidaldb

# Confirm the engine compiles and all tests pass
cargo test --manifest-path tidal/Cargo.toml

# Confirm doc tests and examples compile and run
cargo test --doc --manifest-path tidal/Cargo.toml
cargo test --examples --manifest-path tidal/Cargo.toml

Toolchain

rust-toolchain.toml pins the development toolchain to 1.91.1; rustup honors it automatically. That pin is not the library's MSRV — tidaldb and its siblings publish rust-version = "1.91" and build on 1.91.0. The extra patch release is required by tidalctl's AWS SDK chain (aws-types declares 1.91.1), and without it every workspace-wide command fails during dependency resolution instead of compiling, which silently disables the gates below.

Run Samples Checklist

Before opening a PR that touches public API or examples, verify all samples still work:

# Doc tests (default features)
cargo test --doc --manifest-path tidal/Cargo.toml

# Doc tests with optional features
cargo test --doc --manifest-path tidal/Cargo.toml --features test-utils,metrics

# All four examples compile and run
cargo run --example quickstart       --manifest-path tidal/Cargo.toml
cargo run --example cli_embedding    --manifest-path tidal/Cargo.toml
cargo run --example axum_embedding   --manifest-path tidal/Cargo.toml  # Ctrl+C to stop
cargo run --example actix_embedding  --manifest-path tidal/Cargo.toml  # Ctrl+C to stop

Expected output for quickstart:

build:   dev
uptime:  0.000s
health:  ok
tidalDB opened, verified, and closed. M0 complete.

Full Quality Gate

The pre-commit hook (hooks/pre-commit, activated by scripts/install-hooks.sh) enforces these on staged Rust files:

cargo fmt
cargo clippy -p tidaldb -p tidal-net -p tidal-server -p tidal-stress -p tidalctl --all-targets
cargo test -p tidaldb --lib

Clippy runs without -D warnings on purpose. Each crate's [lints] table is the single source of truth: clippy::all and unwrap_used are deny (so they fail the build), while pedantic and nursery are warn (advisory). Passing -D warnings on the command line overrides that and turns ~58 deliberate pedantic warnings in the integration tests into hard errors.

Run the complete gate manually, across the whole workspace:

cargo fmt --check
cargo clippy --workspace --all-targets
cargo test --workspace
cargo bench --manifest-path tidal/Cargo.toml --no-run   # ensure benches compile

Project Layout

This is a Cargo workspace — the tidaldb engine crate is at tidal/, with tidal-net/, tidal-server/, and tidalctl/ as siblings and example consumers under applications/. See CLAUDE.md § Repository Structure for the full, canonical layout (including the tidal/src/ module map and the docs/ doc homes).

Coding Standards

See CODING_GUIDELINES.md for the full engineering standards.

Key rules:

Result<T, TidalError> everywhere — no panics on recoverable failures
#![forbid(unsafe_code)] — relaxed only at explicit FFI boundaries with // SAFETY: comment
Property tests for invariants, criterion benchmarks for performance claims
Deny-level clippy (clippy::all, unwrap_used) must pass; pedantic/nursery are advisory

3.3 KiB Raw Blame History