9523f6da43
The existing 32 checks prove the deployment answers -- TLS, auth, quorum commit,
convergence, isolation, dashboards, backups. Not one wrote a signal and observed
an order change, so VISION.md:17 "Ranking is not a feature. It is a primitive."
was unverified. This adds a 60-item content-feed app and five assertions that
verify the product's semantics, on a hermetic standalone node.
Added
- tests/e2e/app/: fixture contract (60 items, 4 categories, each owning one
unoccupied 100-id embedding cluster), a deep-module harness owning the whole
lifecycle behind startApp(), the product page, and an app:dev entry point.
- tidal-stress/src/bin/feed-fixture.rs: seeds the catalog and emits brute-force
ground truth, reusing recall::embedding_for rather than adding a third copy of
the corpus generator (tidal/src/db/items.rs already holds a second).
- GroundTruth::from_ids: the oracle now serves sparse id sets. build() delegates,
so there is no transient copy even at 1M, and top_k indexes positionally.
- 10-ranking-semantics.spec.ts (5 hermetic checks) and
11-ranking-integrity.spec.ts (2 cluster tripwires).
- playwright.semantics.config.ts + CAP-016 demo beat (walkthrough 82s -> 90s).
Measured, not merely green
- like: index 59 -> 0, like_boost 2.0, with no sleep between write and read.
- decay: implied half-lives 7.0007 d and 14.0014 d against a schema declaring
7 d and 14 d, recovered from a 4-second window via H = t*ln2 / -ln(v2/v1) and
compared against the schema the node actually loaded, not a hardcoded copy.
- ANN: top-10 identical to brute-force cosine on all four probes; self-distance
0.0148-0.0197 against a 0.05 tolerance.
- rank: dense 1..60 on standalone vs [1,1,1,2,2,3,4,3,4,5,6,5] on the cluster.
Three product findings, pinned and routed to @tidal-engineer
- BUG-018 (High) skip is durably accepted and query-time inert. Penalty is fully
implemented (ranking/profile.rs:227 -> executor/signal_values.rs:183, labelled
{signal}_penalty at executor/mod.rs:65) but skeleton() sets penalties: vec![]
(ranking/builtins.rs:62) and none of the 27 built-ins overrides it. So
VISION.md:187 "negative signals are equal citizens" holds for no shipped
profile. Same anti-pattern as the reseed defects and scatter_merge: a guard
present on one path, absent on its sibling.
- BUG-019 (Medium) three built-ins read signals this schema does not declare --
trending/share_velocity, hidden_gems/completion, controversial/dislike -- so
those terms are permanently 0 and trending ranks on view_velocity alone.
- BUG-020 (Low) for_you declares Scan{sort_field:"created_at"} but ignores a
created_at metadata value; an order matching neither id-asc nor
created_at-desc came back strictly id-ascending.
Two assertions therefore report a gap rather than a success, written as tripwires
whose failure message says what to do when the gap closes. The rank defect is
localised, not fixed: scatter_merge (cluster/node.rs:7542) returns a merged slice
without re-stamping rank while scores stay correctly ordered, so the fault is the
missing stamp and not the merge's sort.
Notes
- Hermetic by construction: its own config, because FullConfig.projects is not
filtered by --project and globalSetup publishes credentials into the main
process that forked workers inherit -- so a setup project cannot replace it,
and weakening globalSetup would destroy the fail-loud behaviour that is its
purpose. Verified with KUBECONFIG=/nonexistent and all E2E_* unset.
- Never touches the deployed corpus: skip is permanent: true, so seeding it into
production would be irreversible.
- The page contains no sort, no hostname and no credential; the harness proxy
injects auth server-side so no bearer reaches a browser or a capture.
- Schema comes from k8s/cluster/schema-configmap.yaml, asserted at 1536 dims;
tidal-server/config/default-schema.yaml declares 128 and would 422 every write.
Verification: 5 semantics + 34 regression + 10 demo captures green; tsc clean;
tidal-stress clippy clean under clippy::all=deny with unwrap_used=deny; 2101
tidaldb lib tests; preflight 10/10 perfect; render 90.05s/2700 frames with zero
empty boundary frames; zero orphan processes or temp dirs after teardown.
165 lines
7.1 KiB
TypeScript
165 lines
7.1 KiB
TypeScript
/**
|
|
* Demo capture — the product working, not just the deployment answering.
|
|
*
|
|
* The walkthrough's other nine captures prove the cluster is real: it converges,
|
|
* it refuses bad credentials, it commits a quorum write, it can be observed and
|
|
* restored. None of them shows what tidalDB is FOR. This one does, and the claim
|
|
* is deliberately narrow: a signal write changes the order of the next query,
|
|
* with nothing in between.
|
|
*
|
|
* It is one image holding both states. Two separate stills of a list would force
|
|
* a viewer to diff two frames from memory; stacked before/after makes the
|
|
* movement self-evident. The lists are composed at their native resolution rather
|
|
* than scaled to fill the frame — a downscaled list is the BUG-010 mistake.
|
|
*
|
|
* This runs against a LOCAL standalone node with fixture data, and the frame says
|
|
* so. `demo/audience-brief.md` forbids implying a dataset that is not what it
|
|
* appears to be, and `skip` being `permanent: true` is exactly why the fixture
|
|
* never touches production.
|
|
*/
|
|
|
|
import { expect, test } from '@playwright/test';
|
|
import {
|
|
CAPTURE_HEIGHT,
|
|
CAPTURE_WIDTH,
|
|
recordScreenshot,
|
|
writeManifestFragment,
|
|
type CaptureRecord,
|
|
} from '../support/proof-panel.ts';
|
|
import { startApp } from '../../app/harness.ts';
|
|
|
|
const records: CaptureRecord[] = [];
|
|
|
|
/**
|
|
* Rows shown in each half.
|
|
*
|
|
* Five, not six: two six-row lists plus their labels came to ~922px inside a
|
|
* 900px frame, which pushed the environment footer off the bottom and clipped the
|
|
* last row. The footer is what stops a fixture screenshot reading as production,
|
|
* so it is not optional — and the fix is fewer rows, never a downscaled list
|
|
* (that was the BUG-010 mistake).
|
|
*/
|
|
const ROWS = 5;
|
|
|
|
/** Viewport that makes the page's centred column fill the frame exactly. */
|
|
const APP_VIEWPORT = { width: 1240, height: 900 };
|
|
|
|
type Row = { position: string; title: string };
|
|
|
|
function composition(before: string, after: string, movedTitle: string, rows: Row[]): string {
|
|
const label = (text: string) => `<div class="label">${text}</div>`;
|
|
return `<!doctype html>
|
|
<html><head><meta charset="utf-8"><style>
|
|
:root { --bg:#0d1117; --line:#222c3a; --text:#d6deeb; --dim:#7c8ba1; --good:#56d364; }
|
|
* { box-sizing: border-box; }
|
|
html, body { margin:0; width:${CAPTURE_WIDTH}px; height:${CAPTURE_HEIGHT}px; }
|
|
body {
|
|
background: var(--bg); color: var(--text); padding: 30px 40px;
|
|
display: flex; flex-direction: column; align-items: center; gap: 13px;
|
|
font: 15px/1.5 ui-sans-serif, -apple-system, "Segoe UI", system-ui, sans-serif;
|
|
}
|
|
.label { align-self: flex-start; margin-left: 184px; color: var(--dim); font-size: 15px; }
|
|
.label b { color: var(--text); font-weight: 650; }
|
|
.label .up { color: var(--good); font-weight: 700; }
|
|
img { display: block; border-radius: 8px; }
|
|
footer {
|
|
margin-top: auto; color: var(--dim); font-size: 13.5px; align-self: flex-start;
|
|
margin-left: 184px;
|
|
}
|
|
</style></head>
|
|
<body>
|
|
${label(`<b>before</b> — ${rows.length} of 60 items, no signals written yet. Everything ties, so the order is the tie-break.`)}
|
|
<img src="${before}" width="1152">
|
|
${label(`<b>after</b> — one <b>like</b> on “${movedTitle}”, then the same query again. <span class="up">It is now first.</span>`)}
|
|
<img src="${after}" width="1152">
|
|
<footer>local standalone node · 60-item fixture catalog · profile <code>for_you</code> · no ETL, no reindex, no second system</footer>
|
|
</body></html>`;
|
|
}
|
|
|
|
test.describe('feed-app product surface', () => {
|
|
test('CAP-016 a signal write reorders the feed immediately', async ({ page }, testInfo) => {
|
|
const app = await startApp();
|
|
try {
|
|
await page.setViewportSize(APP_VIEWPORT);
|
|
await page.goto(`${app.url}/?limit=${ROWS}`, { waitUntil: 'networkidle' });
|
|
await page.waitForSelector('#feed li');
|
|
await page.evaluate(() => document.fonts.ready);
|
|
|
|
const list = page.locator('#feed');
|
|
const readRows = (): Promise<Row[]> =>
|
|
page.$$eval('#feed li', (nodes) =>
|
|
nodes.map((li) => ({
|
|
position: li.querySelector('.pos')!.textContent!.trim().replace(/\s+/g, ' '),
|
|
title: li.querySelector('.title')!.textContent!.trim().replace(/\s+/g, ' '),
|
|
})),
|
|
);
|
|
|
|
const before = await readRows();
|
|
expect(before.length, `the page must render ${ROWS} rows to compose from`).toBe(ROWS);
|
|
const beforeShot = await list.screenshot();
|
|
|
|
// The last visible row, so the movement spans the whole frame.
|
|
const targetTitle = before[before.length - 1]!.title;
|
|
await page.locator('#feed li').last().locator('button[data-signal="like"]').click();
|
|
|
|
// Wait for the re-read to land by watching the thing under test change,
|
|
// rather than sleeping a guessed interval. `startsWith`, not equality: once
|
|
// a row moves, its title element also carries the delta badge ("\u25b25").
|
|
await page.waitForFunction(
|
|
(title) =>
|
|
document.querySelector('#feed li .title')?.textContent?.trim().startsWith(title) === true,
|
|
targetTitle,
|
|
{ timeout: 15_000 },
|
|
);
|
|
|
|
const after = await readRows();
|
|
const afterIndex = after.findIndex((row) => row.title.startsWith(targetTitle));
|
|
|
|
// Assert BEFORE photographing. A capture is a photograph of an
|
|
// already-proven state; if the order did not change there is nothing
|
|
// honest to show.
|
|
expect(
|
|
afterIndex,
|
|
`the liked item must have moved to the top; "${targetTitle}" is at index ${afterIndex}`,
|
|
).toBe(0);
|
|
|
|
// Move the pointer off the list and drop focus before photographing.
|
|
// The cursor physically stays where it clicked, so after the re-render a
|
|
// DIFFERENT row's button sits under it and picks up `:hover` — a highlighted
|
|
// control on an unrelated row, which reads as meaningful when it is not.
|
|
await page.mouse.move(0, 0);
|
|
await page.evaluate(() => (document.activeElement as HTMLElement | null)?.blur());
|
|
const afterShot = await list.screenshot();
|
|
const dataUri = (buffer: Buffer): string =>
|
|
`data:image/png;base64,${buffer.toString('base64')}`;
|
|
|
|
await page.setViewportSize({ width: CAPTURE_WIDTH, height: CAPTURE_HEIGHT });
|
|
await page.setContent(
|
|
composition(dataUri(beforeShot), dataUri(afterShot), targetTitle, before),
|
|
{ waitUntil: 'load' },
|
|
);
|
|
await page.evaluate(() => document.fonts.ready);
|
|
|
|
records.push(
|
|
await recordScreenshot(await page.screenshot(), testInfo, {
|
|
captureId: 'CAP-016-feed-reorder',
|
|
capabilityId: 'CAP-016',
|
|
expected:
|
|
'The same query, before and after one like: the liked item moves from last to first, with its like_boost visible',
|
|
businessPurpose:
|
|
'A signal write changes the order with no ETL in between — the product thesis, not the deployment',
|
|
personas: ['cluster operator', 'application developer'],
|
|
width: CAPTURE_WIDTH,
|
|
height: CAPTURE_HEIGHT,
|
|
}),
|
|
);
|
|
} finally {
|
|
await app.close();
|
|
}
|
|
});
|
|
|
|
test.afterAll(async () => {
|
|
await writeManifestFragment('feed-app', records);
|
|
});
|
|
});
|