9523f6da43
The existing 32 checks prove the deployment answers -- TLS, auth, quorum commit,
convergence, isolation, dashboards, backups. Not one wrote a signal and observed
an order change, so VISION.md:17 "Ranking is not a feature. It is a primitive."
was unverified. This adds a 60-item content-feed app and five assertions that
verify the product's semantics, on a hermetic standalone node.
Added
- tests/e2e/app/: fixture contract (60 items, 4 categories, each owning one
unoccupied 100-id embedding cluster), a deep-module harness owning the whole
lifecycle behind startApp(), the product page, and an app:dev entry point.
- tidal-stress/src/bin/feed-fixture.rs: seeds the catalog and emits brute-force
ground truth, reusing recall::embedding_for rather than adding a third copy of
the corpus generator (tidal/src/db/items.rs already holds a second).
- GroundTruth::from_ids: the oracle now serves sparse id sets. build() delegates,
so there is no transient copy even at 1M, and top_k indexes positionally.
- 10-ranking-semantics.spec.ts (5 hermetic checks) and
11-ranking-integrity.spec.ts (2 cluster tripwires).
- playwright.semantics.config.ts + CAP-016 demo beat (walkthrough 82s -> 90s).
Measured, not merely green
- like: index 59 -> 0, like_boost 2.0, with no sleep between write and read.
- decay: implied half-lives 7.0007 d and 14.0014 d against a schema declaring
7 d and 14 d, recovered from a 4-second window via H = t*ln2 / -ln(v2/v1) and
compared against the schema the node actually loaded, not a hardcoded copy.
- ANN: top-10 identical to brute-force cosine on all four probes; self-distance
0.0148-0.0197 against a 0.05 tolerance.
- rank: dense 1..60 on standalone vs [1,1,1,2,2,3,4,3,4,5,6,5] on the cluster.
Three product findings, pinned and routed to @tidal-engineer
- BUG-018 (High) skip is durably accepted and query-time inert. Penalty is fully
implemented (ranking/profile.rs:227 -> executor/signal_values.rs:183, labelled
{signal}_penalty at executor/mod.rs:65) but skeleton() sets penalties: vec![]
(ranking/builtins.rs:62) and none of the 27 built-ins overrides it. So
VISION.md:187 "negative signals are equal citizens" holds for no shipped
profile. Same anti-pattern as the reseed defects and scatter_merge: a guard
present on one path, absent on its sibling.
- BUG-019 (Medium) three built-ins read signals this schema does not declare --
trending/share_velocity, hidden_gems/completion, controversial/dislike -- so
those terms are permanently 0 and trending ranks on view_velocity alone.
- BUG-020 (Low) for_you declares Scan{sort_field:"created_at"} but ignores a
created_at metadata value; an order matching neither id-asc nor
created_at-desc came back strictly id-ascending.
Two assertions therefore report a gap rather than a success, written as tripwires
whose failure message says what to do when the gap closes. The rank defect is
localised, not fixed: scatter_merge (cluster/node.rs:7542) returns a merged slice
without re-stamping rank while scores stay correctly ordered, so the fault is the
missing stamp and not the merge's sort.
Notes
- Hermetic by construction: its own config, because FullConfig.projects is not
filtered by --project and globalSetup publishes credentials into the main
process that forked workers inherit -- so a setup project cannot replace it,
and weakening globalSetup would destroy the fail-loud behaviour that is its
purpose. Verified with KUBECONFIG=/nonexistent and all E2E_* unset.
- Never touches the deployed corpus: skip is permanent: true, so seeding it into
production would be irreversible.
- The page contains no sort, no hostname and no credential; the harness proxy
injects auth server-side so no bearer reaches a browser or a capture.
- Schema comes from k8s/cluster/schema-configmap.yaml, asserted at 1536 dims;
tidal-server/config/default-schema.yaml declares 128 and would 422 every write.
Verification: 5 semantics + 34 regression + 10 demo captures green; tsc clean;
tidal-stress clippy clean under clippy::all=deny with unwrap_used=deny; 2101
tidaldb lib tests; preflight 10/10 perfect; render 90.05s/2700 frames with zero
empty boundary frames; zero orphan processes or temp dirs after teardown.
174 lines
7.7 KiB
JSON
174 lines
7.7 KiB
JSON
{
|
|
"schemaVersion": 1,
|
|
"buildRevision": "d21a202",
|
|
"verifiedImage": "registry.threesix.ai/tidal/server:m12-admin-gate-20260823@sha256:6e220060a342658b734d258245b20f6233d96e26415b3a44956b1c3bceebe48c",
|
|
"capturedEnvironment": "orchard9-k3sf / namespace tidaldb-cluster (live)",
|
|
"viewport": {
|
|
"width": 1600,
|
|
"height": 900
|
|
},
|
|
"captures": [
|
|
{
|
|
"id": "CAP-002-convergence",
|
|
"capabilityId": "CAP-002",
|
|
"testId": "workflows/deploy-verification.demo.spec.ts :: demo capture \u2014 deploy verification :: CAP-002 every node is converged",
|
|
"file": "captures/CAP-002-convergence.png",
|
|
"expected": "Three nodes, zero lag on every shard group, no reseed pending",
|
|
"businessPurpose": "Quorum with one-node fault tolerance actually exists, rather than being assumed from pod readiness",
|
|
"personas": [
|
|
"cluster operator"
|
|
],
|
|
"width": 1600,
|
|
"height": 900,
|
|
"contentHash": "sha256:69da2d8601585268be468d049afc4747aa2e6ae4c7c1f46b927e0de7ab5ca1b9",
|
|
"audienceVerdict": "perfect",
|
|
"auditStatus": "pass"
|
|
},
|
|
{
|
|
"id": "CAP-006-quorum-write",
|
|
"capabilityId": "CAP-006",
|
|
"testId": "workflows/deploy-verification.demo.spec.ts :: demo capture \u2014 deploy verification :: CAP-005 CAP-006 the boundary refuses, then a quorum write commits",
|
|
"file": "captures/CAP-006-quorum-write.png",
|
|
"expected": "401, 401, 200, then 201 for a quorum-acked write",
|
|
"businessPurpose": "The single strongest available proof: the full stack works and the data plane is closed to strangers",
|
|
"personas": [
|
|
"cluster operator"
|
|
],
|
|
"width": 1600,
|
|
"height": 900,
|
|
"contentHash": "sha256:f1fe76f19643382349006988d5bbddb68a501c1bf3037720c07e98a5a27ddfea",
|
|
"audienceVerdict": "perfect",
|
|
"auditStatus": "pass"
|
|
},
|
|
{
|
|
"id": "CAP-016-feed-reorder",
|
|
"capabilityId": "CAP-016",
|
|
"testId": "workflows/feed-app.demo.spec.ts :: feed-app product surface :: CAP-016 a signal write reorders the feed immediately",
|
|
"file": "captures/CAP-016-feed-reorder.png",
|
|
"expected": "The same query, before and after one like: the liked item moves from last to first, with its like_boost visible",
|
|
"businessPurpose": "A signal write changes the order with no ETL in between \u2014 the product thesis, not the deployment",
|
|
"personas": [
|
|
"cluster operator",
|
|
"application developer"
|
|
],
|
|
"width": 1600,
|
|
"height": 900,
|
|
"contentHash": "sha256:ede7ca3fab758d46cabdf1185f6645bc3055c7e8bbb25e380f8cff509d2136b4",
|
|
"audienceVerdict": "perfect",
|
|
"auditStatus": "pass"
|
|
},
|
|
{
|
|
"id": "CAP-008-network-isolation",
|
|
"capabilityId": "CAP-008",
|
|
"testId": "workflows/deploy-verification.demo.spec.ts :: demo capture \u2014 deploy verification :: CAP-008 the metrics port is closed to foreign pods but open to the scraper",
|
|
"file": "captures/CAP-008-network-isolation.png",
|
|
"expected": "Connection refused from a foreign namespace; hundreds of series to the scraper",
|
|
"businessPurpose": "Least-privilege network access without blinding the monitoring stack",
|
|
"personas": [
|
|
"cluster operator"
|
|
],
|
|
"width": 1600,
|
|
"height": 900,
|
|
"contentHash": "sha256:247fce3324c519319176a20095a1c7a7b2b5eb5ad6745ff9046e8b6886271c6a",
|
|
"audienceVerdict": "perfect",
|
|
"auditStatus": "pass"
|
|
},
|
|
{
|
|
"id": "CAP-010-dashboard",
|
|
"capabilityId": "CAP-010",
|
|
"testId": "workflows/deploy-verification.demo.spec.ts :: demo capture \u2014 deploy verification :: CAP-010 the operator dashboard renders live data",
|
|
"file": "captures/CAP-010-dashboard.png",
|
|
"expected": "Cluster health OK, reseed none, corpus size, and per-node latency charts \u2014 legible at delivery resolution",
|
|
"businessPurpose": "The first surface an operator opens during an incident actually shows the cluster",
|
|
"personas": [
|
|
"cluster operator"
|
|
],
|
|
"width": 1600,
|
|
"height": 502,
|
|
"contentHash": "sha256:296570addb43426d4fbb1f8bb69b8a1fac9c691d9cdccf2e927a5db81e4407fe",
|
|
"audienceVerdict": "perfect",
|
|
"auditStatus": "pass"
|
|
},
|
|
{
|
|
"id": "CAP-014-authority",
|
|
"capabilityId": "CAP-014",
|
|
"testId": "workflows/deploy-verification.demo.spec.ts :: demo capture \u2014 deploy verification :: CAP-014 operator authority is separate from data access",
|
|
"file": "captures/CAP-014-authority.png",
|
|
"expected": "403 for the data credential, not-403 for the admin credential",
|
|
"businessPurpose": "Blast radius of a leaked application key is bounded to data, not cluster topology",
|
|
"personas": [
|
|
"cluster operator"
|
|
],
|
|
"width": 1600,
|
|
"height": 900,
|
|
"contentHash": "sha256:9897670f83c5c408b57a9c88c0097d3fcc43f983ad463be7bfbcabc49e3522b1",
|
|
"audienceVerdict": "perfect",
|
|
"auditStatus": "pass"
|
|
},
|
|
{
|
|
"id": "CAP-014-drift",
|
|
"capabilityId": "CAP-014",
|
|
"testId": "workflows/deploy-verification.demo.spec.ts :: demo capture \u2014 deploy verification :: CAP-014 CAP-015 the harness corrected its own runbook",
|
|
"file": "captures/CAP-014-drift.png",
|
|
"expected": "The superseded claim beside the live probe that contradicts it",
|
|
"businessPurpose": "Verification that audits its own documentation instead of drifting away from it",
|
|
"personas": [
|
|
"cluster operator"
|
|
],
|
|
"width": 1600,
|
|
"height": 900,
|
|
"contentHash": "sha256:7901e70f8df834a91f1399d1ad8fcf4398546bef14fa46ad23ec605e9d12a344",
|
|
"audienceVerdict": "perfect",
|
|
"auditStatus": "pass"
|
|
},
|
|
{
|
|
"id": "CAP-013-backup",
|
|
"capabilityId": "CAP-013",
|
|
"testId": "workflows/deploy-verification.demo.spec.ts :: demo capture \u2014 deploy verification :: CAP-013 the fleet backup captured every volume",
|
|
"file": "captures/CAP-013-backup.png",
|
|
"expected": "Completed, all items, every PodVolumeBackup Completed",
|
|
"businessPurpose": "The cluster can actually be restored, and the alert is trustworthy",
|
|
"personas": [
|
|
"cluster operator"
|
|
],
|
|
"width": 1600,
|
|
"height": 900,
|
|
"contentHash": "sha256:e570917de51ac352a18949117dbf6c9e990fc75127b1bc6c71d0473e2f21371f",
|
|
"audienceVerdict": "perfect",
|
|
"auditStatus": "pass"
|
|
},
|
|
{
|
|
"id": "CAP-015-inert",
|
|
"capabilityId": "CAP-015",
|
|
"testId": "workflows/deploy-verification.demo.spec.ts :: demo capture \u2014 deploy verification :: CAP-015 what is NOT verified is stated",
|
|
"file": "captures/CAP-015-inert.png",
|
|
"expected": "Zero tidaldb_http_* families while baseline tidaldb_* families are present",
|
|
"businessPurpose": "A verification that hides its gaps cannot be trusted about the parts it claims",
|
|
"personas": [
|
|
"cluster operator"
|
|
],
|
|
"width": 1600,
|
|
"height": 900,
|
|
"contentHash": "sha256:671ce44f8e3a629f88cab8f807c9389e068660f51a00e68232c9f92c87b55a76",
|
|
"audienceVerdict": "perfect",
|
|
"auditStatus": "pass"
|
|
},
|
|
{
|
|
"id": "CAP-012-tidalctl",
|
|
"capabilityId": "CAP-012",
|
|
"testId": "workflows/deploy-verification.demo.spec.ts :: demo capture \u2014 deploy verification :: CAP-012 tidalctl gives an operator a live view and an exit code",
|
|
"file": "captures/CAP-012-tidalctl.png",
|
|
"expected": "Leader, region table with NO REPORT markers, shard table, exit 2",
|
|
"businessPurpose": "An operator can interrogate the cluster without hand-rolling curl, and is told what the tool cannot see",
|
|
"personas": [
|
|
"cluster operator"
|
|
],
|
|
"width": 1600,
|
|
"height": 900,
|
|
"contentHash": "sha256:e13e0bfecc766d02f839334487cf0364472c43bf0f1fbe1a11159e6312a58aa6",
|
|
"audienceVerdict": "perfect",
|
|
"auditStatus": "pass"
|
|
}
|
|
]
|
|
}
|