tidaldb

jordan/tidaldb

Fork 0

Commit Graph

Author	SHA1	Message	Date
jordan	c97aaa8e5b	fleet remediation: make the workspace gate runnable, then fix what it caught `cargo test --workspace` could not run at all: dependency resolution failed with "aws-types@1.3.16 requires rustc 1.91.1" on the 1.91.0 default toolchain, so the gate the project documents was dead. Making it run exposed a compile break and two wrong tests that had been invisible for months. Now green end to end: 143 suites, 3155 tests, exit 0. Toolchain - rust-toolchain.toml pins the DEV toolchain to 1.91.1. The published MSRV stays `rust-version = "1.91"` (the engine builds on 1.91.0); only tidalctl's AWS SDK chain needs the patch release, and it now declares that itself. Consumer crates migrated to the current engine API (clean cutover) - iknowyou-engine: `AgentPolicy` gained five m10 read/profile-override fields; the literal now spreads `..AgentPolicy::default()` as the engine's own doc example does, so future fields do not break it again. - forage-engine: `RetrieveResult` gained p1 `reasons`. The app builds its own candidate pool, so it now tags what it knows: PreferenceMatch for the preference-vector blend, SemanticMatch (with the seed item) for similar-to-saved, ExplorationBudget for pinned discoveries. - forage-engine: `url_to_item_id` folded into the u32 item universe. The engine narrows item IDs to a u32 slot in durable per-user state and rejects anything above u32::MAX rather than alias two items forever, so every add_item with a 64-bit FNV hash failed. 9 of 28 smoke tests were failing on this alone. - forage-engine: bridge items read the top-2 preference CLUSTERS via `query_vectors`, not the single centroid from `preference_vectors().get()`. Since m12 that accessor returns only the strongest cluster, so a tech+jazz user whose interests split into two clusters looked single-interest and never bridged. Falls back to top-2 dimensions when a user has one cluster. Reconcile tests corrected to the shipped contract - tidal/tests/m8p3_reconcile_production.rs asserted `3 + 5 == 8` for a windowed count after heal. `take_crdt_snapshot` deliberately keys signal contributions to ONE canonical contributor (ShardId::SINGLE) because signals are relayed from a single writer, so per-node attribution double-counted every replicated event on every reconcile. Merge is therefore LWW on (last_update_ns, score) plus PN-counter per-node max: nodes converge on the more complete accumulator. The old expectation was asserting the bug that fix removed. - Rewrote to assert convergence, count survival (not 0), and no inflation, and added `repeated_reconcile_of_converged_nodes_does_not_creep` - the regression guard for the creep itself, which nothing covered. Pre-commit hook unified - hooks/pre-commit dropped `-D warnings`: each crate's `[lints]` table is the source of truth (`clippy::all`/`unwrap_used` deny, `pedantic` warn), and the flag promoted ~58 deliberate pedantic warnings in integration tests to errors, making every Rust commit impossible. - It now lints all five tidal crates instead of path-matching `tidal/`, which silently skipped tidal-server, tidal-net, tidal-stress, tidalctl and applications/ - the rot above lived in exactly those crates. Ported the CODING_GUIDELINES file-length, println, and unsafe-SAFETY checks from the divergent untracked copy that this replaces. - CONTRIBUTING.md now documents the real commands and the toolchain/MSRV split. Fleet recovery and soak - scripts/restore-fleet.sh: the fail-closed selective restore, promoted out of an ignored tmp/ directory into the repository. Preflights retained storage, digest-pinned images, parked state, and aggregate plus per-PV-node scheduler headroom before the first scale; writes a durable transcript under tmp/restore-logs/ with structured start/error/rollback/complete events. - k8s manifests park the standalone store, the RF3 cluster, and the soak monitor at zero replicas with restore-fleet.sh as the only supported scale-up path. - soak-eval/soak-watch and the nightly CronJob fail closed on stale or missing restart evidence instead of silently skipping the restart-aware half of the gate. - docs/ops/capacity-planning.md corrects the RAM envelope to the real hot-tier formula and separates analytic totals from the measured process envelope.	2026-08-16 12:38:14 -06:00
jx12n	c22a3b65a6	docs: withdraw the pre-release "not ready for production" disclaimer M0-M12 are shipped and the HA cluster runs in production on k3s, so the pre-release disclaimer no longer describes the project. Removes it from the canonical doc set and corrects the readiness text that had gone stale. - README.md: replace the "Pre-release / not yet recommended for production" banner with a production-ready statement; drop "(experimental)" from the cluster status bullet; state the post-1.0 versioning posture (additive in minor releases, breaking changes get a documented migration path). - CLAUDE.md / QUICKSTART.md / docs/guides/server-deployment.md / docs/runbooks/cluster.md: same withdrawal; reframe the cluster opt-in as a guard against standing up a multi-node fabric by accident rather than a readiness warning. - CHANGELOG.md: record the stability posture under [Unreleased], superseding the historical 0.1.0 "no stability guarantees" note (left intact as history). - k8s/statefulset.yaml: the "NOT production HA, tracked as m8p10" comment was stale (m8p10 shipped); point at k8s/cluster/ for the HA deployment instead. Also corrects text that was factually wrong since m11p3/m11p4: the multi-process cluster gate, its CLI help, and the served OpenAPI description all still claimed quorum-ack writes and automatic failure detection did not exist. They do. Historical records (docs/reviews/, docs/profiling/, past CHANGELOG entries, the kubernetes.md rc7 fix note) are left unchanged. Verified against a running binary, not just the build: the opt-in gate's refusal message, the startup WARN, /health 200, and the served /openapi.json description all carry the new text. cargo fmt clean; clippy -D warnings clean on tidaldb and the tidal-server lib; 1943 engine + 155 server lib tests pass; scripts/check-docs.sh OK. Claude-Session: https://claude.ai/code/session_01QdqSDw1tUhK1JT9Pb1vryP	2026-07-30 19:03:34 -06:00
jx12n	1092d34c39	feat: kubernetes deployment, OpenAPI spec, guides, and docker consolidation - Add k8s/ manifests (StatefulSet, kustomize, PDB, ServiceMonitor) + docs/runbooks/kubernetes.md - Add tidal-server/src/openapi.rs (utoipa OpenAPI spec) and wire into router - Add docs/guides/ (build-a-feed-app, embeddings, server-deployment) + foryou_feed example - Consolidate tidal/docker/ into root docker/ (single canonical home) - Update API.md, QUICKSTART.md, README.md, CLAUDE.md, check-docs.sh accordingly	2026-06-09 17:06:34 -06:00

Author

SHA1

Message

Date

jordan

c97aaa8e5b

fleet remediation: make the workspace gate runnable, then fix what it caught

`cargo test --workspace` could not run at all: dependency resolution failed with
"aws-types@1.3.16 requires rustc 1.91.1" on the 1.91.0 default toolchain, so the
gate the project documents was dead. Making it run exposed a compile break and
two wrong tests that had been invisible for months. Now green end to end:
143 suites, 3155 tests, exit 0.

Toolchain
- rust-toolchain.toml pins the DEV toolchain to 1.91.1. The published MSRV stays
  `rust-version = "1.91"` (the engine builds on 1.91.0); only tidalctl's AWS SDK
  chain needs the patch release, and it now declares that itself.

Consumer crates migrated to the current engine API (clean cutover)
- iknowyou-engine: `AgentPolicy` gained five m10 read/profile-override fields;
  the literal now spreads `..AgentPolicy::default()` as the engine's own doc
  example does, so future fields do not break it again.
- forage-engine: `RetrieveResult` gained p1 `reasons`. The app builds its own
  candidate pool, so it now tags what it knows: PreferenceMatch for the
  preference-vector blend, SemanticMatch (with the seed item) for
  similar-to-saved, ExplorationBudget for pinned discoveries.
- forage-engine: `url_to_item_id` folded into the u32 item universe. The engine
  narrows item IDs to a u32 slot in durable per-user state and rejects anything
  above u32::MAX rather than alias two items forever, so every add_item with a
  64-bit FNV hash failed. 9 of 28 smoke tests were failing on this alone.
- forage-engine: bridge items read the top-2 preference CLUSTERS via
  `query_vectors`, not the single centroid from `preference_vectors().get()`.
  Since m12 that accessor returns only the strongest cluster, so a tech+jazz user
  whose interests split into two clusters looked single-interest and never
  bridged. Falls back to top-2 dimensions when a user has one cluster.

Reconcile tests corrected to the shipped contract
- tidal/tests/m8p3_reconcile_production.rs asserted `3 + 5 == 8` for a windowed
  count after heal. `take_crdt_snapshot` deliberately keys signal contributions
  to ONE canonical contributor (ShardId::SINGLE) because signals are relayed from
  a single writer, so per-node attribution double-counted every replicated event
  on every reconcile. Merge is therefore LWW on (last_update_ns, score) plus
  PN-counter per-node max: nodes converge on the more complete accumulator. The
  old expectation was asserting the bug that fix removed.
- Rewrote to assert convergence, count survival (not 0), and no inflation, and
  added `repeated_reconcile_of_converged_nodes_does_not_creep` - the regression
  guard for the creep itself, which nothing covered.

Pre-commit hook unified
- hooks/pre-commit dropped `-D warnings`: each crate's `[lints]` table is the
  source of truth (`clippy::all`/`unwrap_used` deny, `pedantic` warn), and the
  flag promoted ~58 deliberate pedantic warnings in integration tests to errors,
  making every Rust commit impossible.
- It now lints all five tidal crates instead of path-matching `tidal/`, which
  silently skipped tidal-server, tidal-net, tidal-stress, tidalctl and
  applications/ - the rot above lived in exactly those crates. Ported the
  CODING_GUIDELINES file-length, println, and unsafe-SAFETY checks from the
  divergent untracked copy that this replaces.
- CONTRIBUTING.md now documents the real commands and the toolchain/MSRV split.

Fleet recovery and soak
- scripts/restore-fleet.sh: the fail-closed selective restore, promoted out of an
  ignored tmp/ directory into the repository. Preflights retained storage,
  digest-pinned images, parked state, and aggregate plus per-PV-node scheduler
  headroom before the first scale; writes a durable transcript under
  tmp/restore-logs/ with structured start/error/rollback/complete events.
- k8s manifests park the standalone store, the RF3 cluster, and the soak monitor
  at zero replicas with restore-fleet.sh as the only supported scale-up path.
- soak-eval/soak-watch and the nightly CronJob fail closed on stale or missing
  restart evidence instead of silently skipping the restart-aware half of the gate.
- docs/ops/capacity-planning.md corrects the RAM envelope to the real hot-tier
  formula and separates analytic totals from the measured process envelope.

2026-08-16 12:38:14 -06:00

jx12n

c22a3b65a6

docs: withdraw the pre-release "not ready for production" disclaimer

M0-M12 are shipped and the HA cluster runs in production on k3s, so the
pre-release disclaimer no longer describes the project. Removes it from the
canonical doc set and corrects the readiness text that had gone stale.

- README.md: replace the "Pre-release / not yet recommended for production"
  banner with a production-ready statement; drop "(experimental)" from the
  cluster status bullet; state the post-1.0 versioning posture (additive in
  minor releases, breaking changes get a documented migration path).
- CLAUDE.md / QUICKSTART.md / docs/guides/server-deployment.md /
  docs/runbooks/cluster.md: same withdrawal; reframe the cluster opt-in as a
  guard against standing up a multi-node fabric by accident rather than a
  readiness warning.
- CHANGELOG.md: record the stability posture under [Unreleased], superseding
  the historical 0.1.0 "no stability guarantees" note (left intact as history).
- k8s/statefulset.yaml: the "NOT production HA, tracked as m8p10" comment was
  stale (m8p10 shipped); point at k8s/cluster/ for the HA deployment instead.

Also corrects text that was factually wrong since m11p3/m11p4: the
multi-process cluster gate, its CLI help, and the served OpenAPI description
all still claimed quorum-ack writes and automatic failure detection did not
exist. They do.

Historical records (docs/reviews/, docs/profiling/, past CHANGELOG entries,
the kubernetes.md rc7 fix note) are left unchanged.

Verified against a running binary, not just the build: the opt-in gate's
refusal message, the startup WARN, /health 200, and the served
/openapi.json description all carry the new text. cargo fmt clean; clippy
-D warnings clean on tidaldb and the tidal-server lib; 1943 engine + 155
server lib tests pass; scripts/check-docs.sh OK.

Claude-Session: https://claude.ai/code/session_01QdqSDw1tUhK1JT9Pb1vryP

2026-07-30 19:03:34 -06:00

jx12n

1092d34c39

feat: kubernetes deployment, OpenAPI spec, guides, and docker consolidation

- Add k8s/ manifests (StatefulSet, kustomize, PDB, ServiceMonitor) + docs/runbooks/kubernetes.md
- Add tidal-server/src/openapi.rs (utoipa OpenAPI spec) and wire into router
- Add docs/guides/ (build-a-feed-app, embeddings, server-deployment) + foryou_feed example
- Consolidate tidal/docker/ into root docker/ (single canonical home)
- Update API.md, QUICKSTART.md, README.md, CLAUDE.md, check-docs.sh accordingly

2026-06-09 17:06:34 -06:00

3 Commits