The existing 32 checks prove the deployment answers -- TLS, auth, quorum commit,
convergence, isolation, dashboards, backups. Not one wrote a signal and observed
an order change, so VISION.md:17 "Ranking is not a feature. It is a primitive."
was unverified. This adds a 60-item content-feed app and five assertions that
verify the product's semantics, on a hermetic standalone node.
Added
- tests/e2e/app/: fixture contract (60 items, 4 categories, each owning one
unoccupied 100-id embedding cluster), a deep-module harness owning the whole
lifecycle behind startApp(), the product page, and an app:dev entry point.
- tidal-stress/src/bin/feed-fixture.rs: seeds the catalog and emits brute-force
ground truth, reusing recall::embedding_for rather than adding a third copy of
the corpus generator (tidal/src/db/items.rs already holds a second).
- GroundTruth::from_ids: the oracle now serves sparse id sets. build() delegates,
so there is no transient copy even at 1M, and top_k indexes positionally.
- 10-ranking-semantics.spec.ts (5 hermetic checks) and
11-ranking-integrity.spec.ts (2 cluster tripwires).
- playwright.semantics.config.ts + CAP-016 demo beat (walkthrough 82s -> 90s).
Measured, not merely green
- like: index 59 -> 0, like_boost 2.0, with no sleep between write and read.
- decay: implied half-lives 7.0007 d and 14.0014 d against a schema declaring
7 d and 14 d, recovered from a 4-second window via H = t*ln2 / -ln(v2/v1) and
compared against the schema the node actually loaded, not a hardcoded copy.
- ANN: top-10 identical to brute-force cosine on all four probes; self-distance
0.0148-0.0197 against a 0.05 tolerance.
- rank: dense 1..60 on standalone vs [1,1,1,2,2,3,4,3,4,5,6,5] on the cluster.
Three product findings, pinned and routed to @tidal-engineer
- BUG-018 (High) skip is durably accepted and query-time inert. Penalty is fully
implemented (ranking/profile.rs:227 -> executor/signal_values.rs:183, labelled
{signal}_penalty at executor/mod.rs:65) but skeleton() sets penalties: vec![]
(ranking/builtins.rs:62) and none of the 27 built-ins overrides it. So
VISION.md:187 "negative signals are equal citizens" holds for no shipped
profile. Same anti-pattern as the reseed defects and scatter_merge: a guard
present on one path, absent on its sibling.
- BUG-019 (Medium) three built-ins read signals this schema does not declare --
trending/share_velocity, hidden_gems/completion, controversial/dislike -- so
those terms are permanently 0 and trending ranks on view_velocity alone.
- BUG-020 (Low) for_you declares Scan{sort_field:"created_at"} but ignores a
created_at metadata value; an order matching neither id-asc nor
created_at-desc came back strictly id-ascending.
Two assertions therefore report a gap rather than a success, written as tripwires
whose failure message says what to do when the gap closes. The rank defect is
localised, not fixed: scatter_merge (cluster/node.rs:7542) returns a merged slice
without re-stamping rank while scores stay correctly ordered, so the fault is the
missing stamp and not the merge's sort.
Notes
- Hermetic by construction: its own config, because FullConfig.projects is not
filtered by --project and globalSetup publishes credentials into the main
process that forked workers inherit -- so a setup project cannot replace it,
and weakening globalSetup would destroy the fail-loud behaviour that is its
purpose. Verified with KUBECONFIG=/nonexistent and all E2E_* unset.
- Never touches the deployed corpus: skip is permanent: true, so seeding it into
production would be irreversible.
- The page contains no sort, no hostname and no credential; the harness proxy
injects auth server-side so no bearer reaches a browser or a capture.
- Schema comes from k8s/cluster/schema-configmap.yaml, asserted at 1536 dims;
tidal-server/config/default-schema.yaml declares 128 and would 422 every write.
Verification: 5 semantics + 34 regression + 10 demo captures green; tsc clean;
tidal-stress clippy clean under clippy::all=deny with unwrap_used=deny; 2101
tidaldb lib tests; preflight 10/10 perfect; render 90.05s/2700 frames with zero
empty boundary frames; zero orphan processes or temp dirs after teardown.
187 lines
6.7 KiB
TypeScript
187 lines
6.7 KiB
TypeScript
/**
|
|
* The content-feed app's fixture contract — the single home for the catalog and
|
|
* for the shape of the ground truth the Rust fixture binary emits.
|
|
*
|
|
* Everything else imports this: the harness (to seed and to serve
|
|
* `/catalog.json`), the page (via that endpoint), and the specs (to pick probe
|
|
* items and to read the oracle). Without one contract, five consumers each
|
|
* decide independently what an item is and which ids exist.
|
|
*
|
|
* Data and types only. No I/O, no HTTP, and — deliberately — no ranking
|
|
* arithmetic: `CODING_GUIDELINES.md:88` puts scoring in a named ranking profile
|
|
* inside the database, and this app exists to demonstrate exactly that.
|
|
*/
|
|
|
|
/** The embedding width the schema declares (`k8s/cluster/schema-configmap.yaml:51`). */
|
|
export const EMBEDDING_DIM = 1536;
|
|
|
|
/**
|
|
* Items per category. 15 keeps a whole category inside one 100-id embedding
|
|
* cluster; crossing 100 would spill into the next cluster and silently destroy
|
|
* the ground-truth separation the oracle depends on.
|
|
*/
|
|
export const ITEMS_PER_CATEGORY = 15;
|
|
|
|
/**
|
|
* Each category owns ONE embedding cluster.
|
|
*
|
|
* `embedding_for` assigns cluster = id / 100 (`tidal-stress/src/recall.rs:104`),
|
|
* so a category's ids must share a 100-id band. These bands were probed against
|
|
* the live corpus and are unoccupied: the nearest existing vector sits at
|
|
* distance ~2.04 while intra-cluster neighbours sit at ~0.435. That 4.7x
|
|
* separation is what makes a brute-force top-k over just these 60 items the
|
|
* *global* top-k — no 6 GB oracle required.
|
|
*
|
|
* The `999_000_0xx` band is deliberately avoided: earlier verification probes
|
|
* already wrote items there, so its cluster is polluted.
|
|
*/
|
|
export const CATEGORIES = [
|
|
{ name: 'Field recordings', idBase: 900_000_000 },
|
|
{ name: 'Analog synthesis', idBase: 900_000_100 },
|
|
{ name: 'Choral & sacred', idBase: 900_000_200 },
|
|
{ name: 'Free jazz', idBase: 900_000_300 },
|
|
] as const;
|
|
|
|
export type CatalogItem = {
|
|
entityId: number;
|
|
title: string;
|
|
category: string;
|
|
};
|
|
|
|
/**
|
|
* Titles per category, in id order. These are the only prose a viewer reads on
|
|
* screen, so they are real and human rather than `post-1` — a bare id would make
|
|
* the recording read as a developer artifact rather than a product.
|
|
*/
|
|
const TITLES: Record<string, readonly string[]> = {
|
|
'Field recordings': [
|
|
'Harbour Ice at Thaw',
|
|
'Nightjars, Dungeness Shingle',
|
|
'Tram Depot, 04:40',
|
|
'Rain on a Zinc Roof',
|
|
'Cicadas Before the Storm',
|
|
'Understory, Monsoon Week',
|
|
'Fog Signal, Outer Channel',
|
|
'Beehive Interior, Midsummer',
|
|
'Grain Elevator, Idling',
|
|
'Salt Flats, Wind Only',
|
|
'Cathedral Steps at Dusk',
|
|
'Snowmelt Under Rock',
|
|
'Ferry Wake, North Passage',
|
|
'Sparrows in a Bus Shelter',
|
|
'Powerlines, Dry Heat',
|
|
],
|
|
'Analog synthesis': [
|
|
'Ladder Filter Sketch No. 4',
|
|
'Two Oscillators, Slight Detune',
|
|
'Ring Modulator Study',
|
|
'Tape Delay, Self-Oscillating',
|
|
'Sample and Hold Lullaby',
|
|
'Patchbay at Low Voltage',
|
|
'Sawtooth Descending',
|
|
'Envelope Follower Duet',
|
|
'Noise Source, Filtered Slowly',
|
|
'Sequencer Drift',
|
|
'Bucket Brigade Chorus',
|
|
'Sync Lead, Held Open',
|
|
'Resonance at the Edge',
|
|
'Pulse Width Breathing',
|
|
'Cold Start, Warm Bias',
|
|
],
|
|
'Choral & sacred': [
|
|
'Vespers for a Small Room',
|
|
'Antiphon in Two Voices',
|
|
'Kyrie, Winter Setting',
|
|
'Plainchant, Reconstructed',
|
|
'Nunc Dimittis at Compline',
|
|
'Motet for Eight Parts',
|
|
'Requiem Fragment, Anonymous',
|
|
'Magnificat in the Old Style',
|
|
'Litany with Drone',
|
|
'Alleluia, Second Mode',
|
|
'Lament for Holy Saturday',
|
|
'Te Deum, Village Choir',
|
|
'Canticle of the Three',
|
|
'Hymn at the Lighting of Lamps',
|
|
'Psalm 130, Unaccompanied',
|
|
],
|
|
'Free jazz': [
|
|
'Ashfall Quartet, Take 3',
|
|
'Blindfold Duet',
|
|
'Circular Breathing Suite',
|
|
'Downtown Loft, Second Set',
|
|
'Extended Technique No. 9',
|
|
'Fractured Standard',
|
|
'Glass Reeds',
|
|
'Horns Against a Wall',
|
|
'Inside the Piano',
|
|
'Junk Percussion Trio',
|
|
'Kinetic Sculpture Session',
|
|
'Long Tones, No Meter',
|
|
'Multiphonic Conversation',
|
|
'Nine Bells and a Bass',
|
|
'Overblown Ballad',
|
|
],
|
|
};
|
|
|
|
/**
|
|
* The 60-item catalog: 4 categories x 15 items, ids inside each category's band.
|
|
*
|
|
* Insertion order is category-grouped, and the feed's initial order is
|
|
* id-ascending, so the first page shows one category. That is not a layout bug:
|
|
* a freshly seeded corpus has no signals, every item ties on score, and
|
|
* `for_you`'s tie-break is the entity id. Interleaving was tried and reverted —
|
|
* the `for_you` candidate scan declares `sort_field: "created_at"`
|
|
* (`tidal/src/ranking/builtins.rs:49`) but ignores a `created_at` metadata value
|
|
* entirely (measured: an order matching neither id-ascending nor
|
|
* created_at-descending came back strictly id-ascending), so insertion order and
|
|
* timestamps have no observable effect. The page stops looking uniform the
|
|
* instant a signal is written, which is the point.
|
|
*/
|
|
export const CATALOG: readonly CatalogItem[] = CATEGORIES.flatMap(({ name, idBase }) =>
|
|
(TITLES[name] ?? []).map((title, offset) => ({
|
|
entityId: idBase + offset,
|
|
title,
|
|
category: name,
|
|
})),
|
|
);
|
|
|
|
/** Id -> item, for the id→title join the page performs (`/feed` returns no metadata). */
|
|
export const CATALOG_BY_ID: Record<string, CatalogItem> = Object.fromEntries(
|
|
CATALOG.map((item) => [String(item.entityId), item]),
|
|
);
|
|
|
|
/**
|
|
* Items the spec probes for the ANN assertion — one per category, so every
|
|
* cluster is covered rather than just the first.
|
|
*/
|
|
export const PROBE_IDS = [900_000_007, 900_000_107, 900_000_207, 900_000_307] as const;
|
|
|
|
/**
|
|
* What `tidal-stress`'s `feed-fixture` binary writes. A generated artifact, never
|
|
* committed: a checked-in oracle drifts from its generator the first time either
|
|
* one changes, and the drift is silent.
|
|
*/
|
|
export type FixtureGroundTruth = {
|
|
dim: number;
|
|
/** entityId -> its true cosine-ranked neighbour ids, nearest first. */
|
|
neighboursByItem: Record<string, number[]>;
|
|
/**
|
|
* entityId -> its raw 1536-float vector, for the handful of items the spec
|
|
* probes. The ANN assertion has to POST a query vector, and the vectors
|
|
* otherwise exist only inside the Rust binary. Emitting a few probes rather
|
|
* than all 60 keeps this artifact ~100 KB instead of ~1 MB.
|
|
*/
|
|
probeVectors: Record<string, number[]>;
|
|
/**
|
|
* How far an item's own vector may land from itself and still count as "zero".
|
|
* Non-zero purely because the vector round-trips through f32 JSON and the
|
|
* engine re-normalises on write.
|
|
*/
|
|
selfDistanceTolerance: number;
|
|
};
|
|
|
|
/** The three signal names the schema declares. Nothing else exists. */
|
|
export const SIGNALS = ['view', 'like', 'skip'] as const;
|
|
export type SignalName = (typeof SIGNALS)[number];
|