tidaldb/tidal-stress/k8s/soak-results-pvc.yaml
jordan c97aaa8e5b fleet remediation: make the workspace gate runnable, then fix what it caught
`cargo test --workspace` could not run at all: dependency resolution failed with
"aws-types@1.3.16 requires rustc 1.91.1" on the 1.91.0 default toolchain, so the
gate the project documents was dead. Making it run exposed a compile break and
two wrong tests that had been invisible for months. Now green end to end:
143 suites, 3155 tests, exit 0.

Toolchain
- rust-toolchain.toml pins the DEV toolchain to 1.91.1. The published MSRV stays
  `rust-version = "1.91"` (the engine builds on 1.91.0); only tidalctl's AWS SDK
  chain needs the patch release, and it now declares that itself.

Consumer crates migrated to the current engine API (clean cutover)
- iknowyou-engine: `AgentPolicy` gained five m10 read/profile-override fields;
  the literal now spreads `..AgentPolicy::default()` as the engine's own doc
  example does, so future fields do not break it again.
- forage-engine: `RetrieveResult` gained p1 `reasons`. The app builds its own
  candidate pool, so it now tags what it knows: PreferenceMatch for the
  preference-vector blend, SemanticMatch (with the seed item) for
  similar-to-saved, ExplorationBudget for pinned discoveries.
- forage-engine: `url_to_item_id` folded into the u32 item universe. The engine
  narrows item IDs to a u32 slot in durable per-user state and rejects anything
  above u32::MAX rather than alias two items forever, so every add_item with a
  64-bit FNV hash failed. 9 of 28 smoke tests were failing on this alone.
- forage-engine: bridge items read the top-2 preference CLUSTERS via
  `query_vectors`, not the single centroid from `preference_vectors().get()`.
  Since m12 that accessor returns only the strongest cluster, so a tech+jazz user
  whose interests split into two clusters looked single-interest and never
  bridged. Falls back to top-2 dimensions when a user has one cluster.

Reconcile tests corrected to the shipped contract
- tidal/tests/m8p3_reconcile_production.rs asserted `3 + 5 == 8` for a windowed
  count after heal. `take_crdt_snapshot` deliberately keys signal contributions
  to ONE canonical contributor (ShardId::SINGLE) because signals are relayed from
  a single writer, so per-node attribution double-counted every replicated event
  on every reconcile. Merge is therefore LWW on (last_update_ns, score) plus
  PN-counter per-node max: nodes converge on the more complete accumulator. The
  old expectation was asserting the bug that fix removed.
- Rewrote to assert convergence, count survival (not 0), and no inflation, and
  added `repeated_reconcile_of_converged_nodes_does_not_creep` - the regression
  guard for the creep itself, which nothing covered.

Pre-commit hook unified
- hooks/pre-commit dropped `-D warnings`: each crate's `[lints]` table is the
  source of truth (`clippy::all`/`unwrap_used` deny, `pedantic` warn), and the
  flag promoted ~58 deliberate pedantic warnings in integration tests to errors,
  making every Rust commit impossible.
- It now lints all five tidal crates instead of path-matching `tidal/`, which
  silently skipped tidal-server, tidal-net, tidal-stress, tidalctl and
  applications/ - the rot above lived in exactly those crates. Ported the
  CODING_GUIDELINES file-length, println, and unsafe-SAFETY checks from the
  divergent untracked copy that this replaces.
- CONTRIBUTING.md now documents the real commands and the toolchain/MSRV split.

Fleet recovery and soak
- scripts/restore-fleet.sh: the fail-closed selective restore, promoted out of an
  ignored tmp/ directory into the repository. Preflights retained storage,
  digest-pinned images, parked state, and aggregate plus per-PV-node scheduler
  headroom before the first scale; writes a durable transcript under
  tmp/restore-logs/ with structured start/error/rollback/complete events.
- k8s manifests park the standalone store, the RF3 cluster, and the soak monitor
  at zero replicas with restore-fleet.sh as the only supported scale-up path.
- soak-eval/soak-watch and the nightly CronJob fail closed on stale or missing
  restart evidence instead of silently skipping the restart-aware half of the gate.
- docs/ops/capacity-planning.md corrects the RAM envelope to the real hot-tier
  formula and separates analytic totals from the measured process envelope.
2026-08-16 12:38:14 -06:00

35 lines
1.5 KiB
YAML
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Durable result sink for the 30-night soak (GA bar: nightly suite green 30
# consecutive days — docs/planning/milestone-11/phase-9.md "Exit gate").
#
# longhorn-rwx: RWX so the nightly soak CronJob pods (writers) and the always-on
# soak-monitor (reader + evaluator) can mount it concurrently, and Retain reclaim
# policy so the 30 nights of JSON summaries survive a PVC delete / job churn /
# operator session ending. Each night writes /results/soak-YYYY-MM-DD.json +
# appends one line to /results/ledger.tsv; the monitor appends restarts.tsv every
# 5 min and the evaluator rewrites streak.tsv.
#
# SIZING (30-night budget): 30 × soak-*.json (per-stage latency histograms, up to
# ~30 MiB each on a full run) ≈ 1 GiB, + restarts.tsv (288 samples/day × 3 pods ×
# 30 nights ≈ 26k rows ≈ a few MiB) + ledger/streak (KiB). 5Gi gives ~2× headroom.
# The evaluator container PRUNES soak-*.json older than 31 days each loop so the
# Retain volume stays bounded across successive windows (it never auto-shrinks).
#
# Apply: kubectl apply -f tidal-stress/k8s/soak-results-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: tidal-soak-results
namespace: tidaldb-cluster
labels:
app.kubernetes.io/name: tidal-soak
app.kubernetes.io/part-of: tidaldb
backup.orchard9.ai/class: protected
backup.orchard9.ai/method: velero-kopia
spec:
accessModes:
- ReadWriteMany
storageClassName: longhorn-rwx
resources:
requests:
storage: 5Gi