tidaldb/docs/guides/build-a-feed-app.md
jx12n 1092d34c39 feat: kubernetes deployment, OpenAPI spec, guides, and docker consolidation
- Add k8s/ manifests (StatefulSet, kustomize, PDB, ServiceMonitor) + docs/runbooks/kubernetes.md
- Add tidal-server/src/openapi.rs (utoipa OpenAPI spec) and wire into router
- Add docs/guides/ (build-a-feed-app, embeddings, server-deployment) + foryou_feed example
- Consolidate tidal/docker/ into root docker/ (single canonical home)
- Update API.md, QUICKSTART.md, README.md, CLAUDE.md, check-docs.sh accordingly
2026-06-09 17:06:34 -06:00

621 lines
28 KiB
Markdown

# Build a Short-Video "For You" Feed on tidalDB
This is an end-to-end tutorial for building a TikTok/Reels-style short-video feed —
a swipeable home feed ("For You"), a Following feed, and a Discover/Trending tab —
on top of tidalDB. You will model the signals a short-video app actually emits,
ingest videos, wire the swipe feedback loop so a user's taste updates *immediately*,
and serve all three feed surfaces. You will do it twice: once embedded in a Rust
process, and once over HTTP with `curl`.
There is a verified, runnable version of the embedded path in this repo:
```bash
cargo run -p tidaldb --example foryou_feed
```
That example ([`tidal/examples/foryou_feed.rs`](../../tidal/examples/foryou_feed.rs))
is the source of truth for every Rust call shown below — this guide reproduces and
explains its key calls. Skim it after reading section 6.
**New to tidalDB?** Read [QUICKSTART.md](../../QUICKSTART.md) first for the 5-minute
schema → ingest → rank loop. This guide assumes you have run the quickstart and
understand schemas, signals, and the `RETRIEVE` query. The product framing for this
surface is [USE_CASES.md → UC-01 Personalized Feed (For You)](../../USE_CASES.md#uc-01--personalized-feed--for-you).
---
## 1. What tidalDB owns vs. what you build
tidalDB answers exactly one question: **given a user and a context, what content
should they see, in what order?** It owns *retrieval and ranking* — and the
feedback loop that learns taste from engagement. It owns nothing else.
Everything that produces, stores, or serves the actual video bytes is yours.
| Concern | Who owns it |
|---|---|
| Video upload, blob storage, object store | **You** |
| CDN, transcoding, HLS/DASH packaging, thumbnails | **You** |
| Embedding generation (run the video through a multimodal model) | **You** — see [docs/guides/embeddings.md](./embeddings.md) |
| Authentication, identity, sessions | **You** |
| Moderation, trust & safety, takedowns | **You** |
| Payments, creator payouts | **You** |
| Player UI, swipe gestures, autoplay | **You** |
| **Candidate retrieval (ANN + filters)** | **tidalDB** |
| **Signal ledgers: decay, velocity, windowed counts** | **tidalDB** |
| **Per-user preference (taste) vectors** | **tidalDB** |
| **Ranking profiles (`for_you`, `following`, `trending`, …)** | **tidalDB** |
| **Diversity enforcement + exploration** | **tidalDB** |
| **The swipe feedback loop** (engagement → updated taste, no Kafka lag) | **tidalDB** |
### Architecture sketch
```
┌──────────────┐ upload ┌─────────────────────────────────────┐
│ Creator app │────────────▶│ YOUR services │
└──────────────┘ │ • blob store + CDN + transcode │
│ • multimodal model → embedding │
│ • auth / identity / moderation │
└───────────────┬─────────────────────┘
write_item_with_metadata(id, meta)
write_item_embedding(id, vector) ← you bring the vector
┌──────────────┐ ┌───────────────────────────────┐
│ Viewer app │ GET feed │ tidalDB │
│ (player UI) │◀──── retrieve ────│ HNSW + signal ledgers + │
│ │ │ preference vectors + │
│ swipe ─────┼──── signals ─────▶│ ranking profiles + diversity │
│ (complete, │ signal_with_ │ │
│ like,skip) │ context(…) │ one process, one query API │
└──────────────┘ └───────────────────────────────┘
```
The viewer app does two things against tidalDB: it **reads** a ranked feed, and as
the user swipes it **writes** signals back. The write updates that user's taste in
the same process that serves the next read — there is no Kafka topic, no feature
store, and no batch job in between (section 4).
> tidalDB does **not** generate embeddings. Your model turns the video
> (frames + audio + caption) into a vector; tidalDB L2-normalizes it, inserts it
> into the HNSW index, and ranks over it. See [docs/guides/embeddings.md](./embeddings.md).
---
## 2. Model the signals for short video
A signal is a typed, timestamped event stream with native decay, velocity, and
windowed aggregation. You declare each signal once in the schema; the engine then
maintains its running decay score, per-window counts, and (optionally) velocity for
free. For short video, six signals capture the surface:
| Signal | Decay half-life | `positive_engagement` | Why |
|---|---|---|---|
| `view` | 7 days | no | The raw impression. Low-intent — a glance, not a preference. Drives trending velocity, not taste. |
| `completion` | 1 year (barely decays) | **yes** | Finishing a short video is the strongest, most durable taste signal there is. Weight = fraction watched ∈ [0,1]. |
| `like` | 30 days | **yes** | An explicit, durable positive. |
| `share` | 3 days (bursty) | no | Tracks `velocity` — shares spike, then fade; great for trending, not a private taste signal. |
| `skip` | 1 day | **no** | The swipe-away negative. Must **never** pull taste toward the skipped video, and marks it as a hard negative for that user. |
| `replay` | 14 days | **yes** | Re-watching is a meaningful, deliberate positive. |
### The `positive_engagement(true)` rule — read this twice
This is the single most important correctness rule for a personalized feed:
> **Preference-vector personalization fires only for signals declared
> `.positive_engagement(true)`.** When you record an engagement through
> `signal_with_context`, tidalDB folds the *item's content embedding* into the
> *acting user's* preference (taste) vector — but only if that signal type is a
> declared positive-engagement signal.
Why each classification matters:
- **`completion`, `like`, `replay` are `positive_engagement(true)`** → engaging
with a video nudges the user's taste vector toward that video's content. This is
the learning step. Get it right and the feed converges on what the user loves.
- **`view` is NOT positive-engagement** → a view is just "the video was shown."
If views folded into taste, a user's vector would be dragged toward everything
the feed happened to surface — a feedback bubble that learns nothing.
- **`skip` is NOT positive-engagement** → this is the load-bearing negative. A
swipe-away must never pull taste *toward* the rejected content. Routed through
`signal_with_context`, `skip` instead marks the video as a **hard negative** and
**seen** for that user, so the feed stops surfacing it.
If you accidentally mark `skip` or `view` as positive-engagement, the feed slowly
poisons every user's taste vector. Declare them honestly.
> **Subtle but important — the rule is all-or-nothing per database.** If *any*
> signal in your schema declares `positive_engagement(true)`, then **only** the
> signals you explicitly flagged fold the preference vector; nothing else does.
> If *no* signal declares it, tidalDB falls back to a built-in name allowlist
> (`like`, `share`, `completion`, `search_click`). The embedded schema below
> declares the flags explicitly, so it is fully in control. This matters for the
> HTTP path — see the caveat in section 9.
### Schema (embedded Rust)
This is lifted directly from
[`tidal/examples/foryou_feed.rs`](../../tidal/examples/foryou_feed.rs):
```rust
use std::time::Duration;
use tidaldb::schema::{DecaySpec, EntityKind, SchemaBuilder, Window};
let mut schema = SchemaBuilder::new();
// view: 7-day half-life, velocity + windows. Low-intent — NOT positive engagement.
let _ = schema
.signal("view", EntityKind::Item,
DecaySpec::Exponential { half_life: Duration::from_secs(7 * 24 * 3600) })
.windows(&[Window::OneHour, Window::TwentyFourHours, Window::AllTime])
.velocity(true)
.add();
// completion: ~1-year half-life. Weight = fraction watched. POSITIVE engagement.
let _ = schema
.signal("completion", EntityKind::Item,
DecaySpec::Exponential { half_life: Duration::from_secs(365 * 24 * 3600) })
.windows(&[Window::AllTime])
.positive_engagement(true)
.add();
// like: 30-day half-life. POSITIVE engagement.
let _ = schema
.signal("like", EntityKind::Item,
DecaySpec::Exponential { half_life: Duration::from_secs(30 * 24 * 3600) })
.windows(&[Window::AllTime])
.positive_engagement(true)
.add();
// share: 3-day half-life, bursty → velocity. NOT positive engagement.
let _ = schema
.signal("share", EntityKind::Item,
DecaySpec::Exponential { half_life: Duration::from_secs(3 * 24 * 3600) })
.windows(&[Window::TwentyFourHours, Window::AllTime])
.velocity(true)
.add();
// skip: fast 1-day decay — the swipe-away negative. NOT positive engagement.
let _ = schema
.signal("skip", EntityKind::Item,
DecaySpec::Exponential { half_life: Duration::from_secs(24 * 3600) })
.windows(&[Window::OneHour, Window::TwentyFourHours])
.add();
// replay: 14-day half-life. POSITIVE engagement.
let _ = schema
.signal("replay", EntityKind::Item,
DecaySpec::Exponential { half_life: Duration::from_secs(14 * 24 * 3600) })
.windows(&[Window::AllTime])
.positive_engagement(true)
.add();
// Content embedding slot for items: 128 dimensions (your model's output size).
let _ = schema.embedding_slot("content", EntityKind::Item, 128);
let schema = schema.build()?;
```
`DecaySpec` has three variants: `Exponential { half_life }`, `Linear { lifetime }`,
and `Permanent`. Available windows: `OneHour`, `TwentyFourHours`, `SevenDays`,
`ThirtyDays`, `AllTime`. Only declare the windows a profile actually reads —
each one is real bookkeeping.
---
## 3. Ingest videos (metadata + embeddings)
After your services have uploaded, transcoded, and embedded a video, write two
records into tidalDB: the metadata and the embedding vector.
```rust
use std::collections::HashMap;
use tidaldb::{TidalDb, schema::{EntityId, Timestamp}};
let db = TidalDb::builder().ephemeral().with_schema(schema).open()?;
// For durability: .with_data_dir("/var/lib/tidaldb") instead of .ephemeral()
let now = Timestamp::now();
let mut meta = HashMap::new();
meta.insert("title".to_string(), "Lo-fi piano loop".to_string());
meta.insert("category".to_string(), "music".to_string());
meta.insert("format".to_string(), "short".to_string());
meta.insert("creator_id".to_string(), "42".to_string());
meta.insert("duration".to_string(), "27".to_string()); // seconds
meta.insert("created_at".to_string(), now.as_nanos().to_string());
db.write_item_with_metadata(EntityId::new(1001), &meta)?;
// You bring the vector. This MUST be your model's output for THIS video.
let embedding: Vec<f32> = your_model.embed(&video); // length == slot dimensions (128)
db.write_item_embedding(EntityId::new(1001), &embedding)?;
```
Indexed metadata keys the engine understands: `title`, `category`, `format`,
`creator_id`, `tags`, `duration`, `created_at`. `creator_id` is what the
`max_per_creator` diversity cap (section 5) keys on, so always set it.
**Embedding rules (strict — read [docs/guides/embeddings.md](./embeddings.md) for
real vectors):**
- `write_item_embedding` L2-normalizes the vector and inserts it into the HNSW index.
- Dimensions are strict: **min 2, max 4096**, and the length **must equal the slot's
declared dimensions** (128 here) or the insert fails.
- **Zero-norm vectors are rejected.**
- **Multi-slot caveat:** `RETRIEVE`/`SEARCH` route through the **first declared slot
only**. A multi-modal app (separate text and image vectors) must fuse them offline
into one vector per slot, or model the modalities as separate entity kinds.
The example seeds 128-D *random* vectors purely so it runs standalone. A real app
must use genuine embeddings — random vectors give you random nearest-neighbors.
---
## 4. The swipe feedback loop (no Kafka, no feature-store lag)
This is what makes tidalDB a *feed* engine and not just a vector store. As the user
swipes, you record each engagement through `signal_with_context`, passing the user
id and the video's creator id. tidalDB writes the signal, updates windowed counts
and velocity, marks seen/hard-negative state, and — for positive-engagement signals —
folds the video's embedding into *that user's* preference vector. The next
`retrieve` call in the same process sees the updated taste. No queue, no batch job,
no feature store to keep in sync.
```rust
let user_id: u64 = 1001;
let creator_id: u64 = 42;
let video = EntityId::new(2001);
let now = Timestamp::now();
// User watched the whole thing → completion weight is the fraction watched (1.0).
// completion is positive-engagement → folds the video's embedding into the user's taste.
db.signal_with_context("completion", video, 1.0, now, Some(user_id), Some(creator_id))?;
// User tapped like → another positive fold, plus strengthens the (user, creator) edge.
db.signal_with_context("like", video, 1.0, now, Some(user_id), Some(creator_id))?;
// Different video — user swiped away after a glance. Partial view, then skip.
let rejected = EntityId::new(2002);
db.signal_with_context("view", rejected, 0.1, now, Some(user_id), Some(creator_id))?;
// skip is NOT positive-engagement: it does NOT pull taste toward `rejected`.
// It marks `rejected` as a hard negative + seen for this user, so the feed drops it.
db.signal_with_context("skip", rejected, 1.0, now, Some(user_id), Some(creator_id))?;
```
Contrast the two write APIs:
- `db.signal(type, id, weight, ts)`**global** signal, no user context. Use it for
aggregate counters that feed `trending` (e.g. a server-side view counter). It does
**not** touch any preference vector or seen-state.
- `db.signal_with_context(type, id, weight, ts, Some(user_id), Some(creator_id))`
**personalized**. Updates the user's preference vector (positive-engagement only),
seen-state, and hard negatives (skip/hide/block). This is the swipe-loop API.
A `completion` weight is the **fraction watched** in `[0.0, 1.0]` — 0.3 for a
three-second glance at a ten-second clip, 1.0 for a full watch (or a watch past the
end, if you clamp). The decay score blends these naturally: many partial completions
sum to less than a few full ones.
You can verify the loop landed by reading the live decay score:
```rust
let score = db.read_decay_score(EntityId::new(2001), "completion", 0)?; // window index 0
```
---
## 5. Serve the three feed surfaces
tidalDB ships **25 built-in ranking profiles**, registered automatically on
`open()`. A short-video app needs three of them. None require any extra schema — you
just name the profile in the query.
### For You — `profile("for_you")`
The personalized home feed. It blends interaction-weighted decay scores
(`view·1 + like·2 + share-velocity·1.5`) with the user's preference vector, sorts
with a Hot decay (`gravity = 1.5`), then enforces diversity and exploration. The
built-in defaults are:
- `max_per_creator = 2` — no single creator can dominate the feed.
- `format_mix_max_fraction = 0.4` — no one format swamps the mix.
- `exploration = 0.1`**10% of slots are exploration**: unseen/random items
injected to break the filter bubble and let *new creators surface*. Without this,
a feed converges to a narrow loop and new content never gets a chance.
Seen items and hard negatives (the videos this user already watched or skipped) are
filtered out automatically — a feed must surface fresh content.
```rust
use tidaldb::query::retrieve::Retrieve;
let for_you = Retrieve::builder()
.for_user(user_id) // personalization + seen/hard-negative filtering
.profile("for_you")
.limit(10)
.build()?;
let results = db.retrieve(&for_you)?;
for item in &results.items {
println!("#{} video={} score={:.4}", item.rank, item.entity_id.as_u64(), item.score);
}
```
Want to override the defaults for one query? Use `.diversity(...)`:
```rust
use tidaldb::ranking::diversity::DiversityConstraints;
let q = Retrieve::builder()
.for_user(user_id)
.profile("for_you")
.diversity(DiversityConstraints {
max_per_creator: Some(1), // stricter: one video per creator
format_mix_max_fraction: Some(0.5),
..DiversityConstraints::new()
})
.limit(20)
.build()?;
```
### Following — `profile("following")`
Content from creators the user follows, sourced from the relationship graph and
ranked by recency with a view-velocity boost (`max_per_creator = 3`). This requires
that you have recorded follow relationships (a follows edge) for the user — see
[USE_CASES.md](../../USE_CASES.md) for the relationship model.
```rust
let following = Retrieve::builder()
.for_user(user_id)
.profile("following")
.limit(20)
.build()?;
let results = db.retrieve(&following)?;
```
### Discover / Trending — `profile("trending")`
The global, *non-personalized* tab — what a brand-new visitor with no taste history
sees. It ranks every item by velocity (`view-velocity + 2·share-velocity` over the
24h window), capped at `max_per_creator = 1` for maximum variety. Note: velocity
needs signals arriving over real elapsed time to populate hour-level buckets, so a
freshly-seeded demo will show sparse trending scores — that is expected.
```rust
let trending = Retrieve::builder() // no .for_user — trending is global
.profile("trending")
.limit(20)
.build()?;
let results = db.retrieve(&trending)?;
```
> The payoff to look for in [`foryou_feed.rs`](../../tidal/examples/foryou_feed.rs):
> after a user completes + likes music videos and skips everything else, the top of
> their `for_you` feed is the *fresh* music videos — surfaced because the swipe
> session built a strong taste signal, **not** because of the (random) embeddings.
> The ordering is driven by signals, which is why it is stable across runs.
---
## 6. The whole thing, embedded (Rust)
The complete, verified program is at
[`tidal/examples/foryou_feed.rs`](../../tidal/examples/foryou_feed.rs). Run it:
```bash
cargo run -p tidaldb --example foryou_feed
```
It walks the full arc end to end:
1. Builds the six-signal short-video schema from section 2.
2. Opens an ephemeral DB and ingests ~24 videos across 5 creators.
3. Simulates one user's swipe session — completions + likes + a share + a replay on
music videos, skips on the rest — all via `signal_with_context`.
4. Retrieves `for_you` (diversity-capped, 10% exploration) and prints the ranked feed.
5. Retrieves global `trending` for contrast.
`Arc<TidalDb>` is `Send + Sync`, so the same handle backs every request thread in a
real server: clone the `Arc` into your axum/actix handlers, write signals on the
swipe endpoint, and read feeds on the feed endpoint. See the embedding-integration
examples (`axum_embedding`, `actix_embedding`, `cli_embedding`) in `tidal/examples/`
for the web-handler wiring.
A `Results` value contains `items: Vec<FeedItem>`; each `FeedItem` exposes `.rank`,
`.entity_id.as_u64()`, `.score`, and `.signals` (per-item signal snapshots for
explainability), plus `results.total_candidates` for the pre-diversity pool size.
For text + vector hybrid search (a search box rather than a feed), use the parallel
`Search::builder().query("...").vector(vec).for_user(uid).limit(n).build()?` then
`db.search(&q)?` — covered in [QUICKSTART.md](../../QUICKSTART.md) and
[API.md](../../API.md).
---
## 7. The whole thing, over HTTP
The same database behind a JSON HTTP API is `tidal-server`. Start a standalone
node with your schema:
```bash
cargo run -p tidal-server -- standalone \
--listen 0.0.0.0:9400 \
--schema ./short-video-schema.yaml \
--data-dir /var/lib/tidaldb \
--metrics 127.0.0.1:9091
```
Set `TIDAL_API_KEY` to require Bearer auth on the data routes. **If `TIDAL_API_KEY`
is unset, the server runs fully unauthenticated and logs a WARN** — never do that
outside a private dev box.
```bash
export TIDAL_API_KEY="$(openssl rand -hex 32)"
```
### Schema YAML
The HTTP server loads its schema from YAML. The short-video schema looks like:
```yaml
signals:
- name: view
entity: item
decay: { exponential: { half_life_seconds: 604800 } } # 7 days
windows: [one_hour, twenty_four_hours, all_time]
velocity: true
- name: completion
entity: item
decay: { exponential: { half_life_seconds: 31536000 } } # 1 year
windows: [all_time]
positive_engagement: true
- name: like
entity: item
decay: { exponential: { half_life_seconds: 2592000 } } # 30 days
windows: [all_time]
positive_engagement: true
- name: share
entity: item
decay: { exponential: { half_life_seconds: 259200 } } # 3 days
windows: [twenty_four_hours, all_time]
velocity: true
- name: skip
entity: item
decay: { exponential: { half_life_seconds: 86400 } } # 1 day
windows: [one_hour, twenty_four_hours]
- name: replay
entity: item
decay: { exponential: { half_life_seconds: 1209600 } } # 14 days
windows: [all_time]
positive_engagement: true
text_fields:
- { name: title, kind: text }
- { name: category, kind: keyword }
embedding_slots:
- { name: content, entity: item, dimensions: 128 }
```
> **`positive_engagement` works in YAML too.** Declaring it per signal (as
> above) gives the HTTP server the same explicit control as the embedded
> `SchemaBuilder` — `completion`, `like`, and `replay` fold into the user's
> preference vector; `view` and `skip` do not. If you omit the field on *every*
> signal, tidalDB falls back to a built-in name allowlist (`like`, `share`,
> `completion`, `search_click`); declaring it explicitly (recommended) removes
> any ambiguity about which signals shape taste.
The 25 built-in profiles (`for_you`, `following`, `trending`, …) are available on
the HTTP server with no `profiles:` block — they are registered automatically. Add a
`profiles:` block only to override a built-in or define your own.
### Routes
All routes are JSON. Data routes require `Authorization: Bearer $TIDAL_API_KEY` when
the key is set; `GET /health*` and `GET /openapi.json` are always unauthenticated.
```bash
# Ingest a video's metadata → 201 Created
curl -X POST http://localhost:9400/items \
-H "Authorization: Bearer $TIDAL_API_KEY" \
-H 'Content-Type: application/json' \
-d '{ "entity_id": 1001,
"metadata": { "title": "Lo-fi piano loop", "category": "music",
"format": "short", "creator_id": "42", "duration": "27" } }'
# Ingest its embedding (length MUST equal slot dimensions) → 204 No Content
curl -X POST http://localhost:9400/embeddings \
-H "Authorization: Bearer $TIDAL_API_KEY" \
-H 'Content-Type: application/json' \
-d '{ "entity_id": 1001, "values": [0.013, -0.041, /* … 128 floats … */ 0.092] }'
# The swipe loop: record engagement with user + creator context → 204 No Content
curl -X POST http://localhost:9400/signals \
-H "Authorization: Bearer $TIDAL_API_KEY" \
-H 'Content-Type: application/json' \
-d '{ "entity_id": 1001, "signal": "completion", "weight": 1.0,
"user_id": 1001, "creator_id": 42 }'
curl -X POST http://localhost:9400/signals \
-H "Authorization: Bearer $TIDAL_API_KEY" \
-H 'Content-Type: application/json' \
-d '{ "entity_id": 1001, "signal": "like", "weight": 1.0,
"user_id": 1001, "creator_id": 42 }'
# The swipe-away negative on a different video
curl -X POST http://localhost:9400/signals \
-H "Authorization: Bearer $TIDAL_API_KEY" \
-H 'Content-Type: application/json' \
-d '{ "entity_id": 1002, "signal": "skip", "weight": 1.0,
"user_id": 1001, "creator_id": 7 }'
# Serve the For You feed
curl -H "Authorization: Bearer $TIDAL_API_KEY" \
"http://localhost:9400/feed?user_id=1001&profile=for_you&limit=20"
# Following feed and the Discover/Trending tab
curl -H "Authorization: Bearer $TIDAL_API_KEY" \
"http://localhost:9400/feed?user_id=1001&profile=following&limit=20"
curl -H "Authorization: Bearer $TIDAL_API_KEY" \
"http://localhost:9400/feed?profile=trending&limit=20" # no user_id — global
```
When `?user_id=` and `?profile=` are omitted, `/feed` defaults to `profile=for_you`.
Passing `user_id` and `creator_id` on `/signals` is what activates the personalized
loop — omit them and the signal is recorded globally (the `signal` vs.
`signal_with_context` distinction from section 4).
Middleware on the data routes: 30s timeout (`408`), 100 max in-flight (`429`), 2 MB
body limit (`413`), and an `x-request-id` echoed on every response. The canonical,
always-current HTTP reference is the served spec at `GET /openapi.json` (unauthenticated).
---
## 8. Going to production — checklist
- [ ] **Durable storage.** Open with `.with_data_dir(...)` (embedded) or
`--data-dir` (server), not ephemeral. The WAL persists signals across restarts.
- [ ] **Real embeddings.** Replace the random vectors with your multimodal model's
output. Same model for ingest and query. See
[docs/guides/embeddings.md](./embeddings.md).
- [ ] **Auth on.** Set `TIDAL_API_KEY`. An unset key means an unauthenticated server
(it logs a WARN). Terminate TLS at your ingress.
- [ ] **Lock down metrics.** The `--metrics` Prometheus endpoint is **unauthenticated**
— bind it to loopback or a cluster-internal address only, never the public
interface.
- [ ] **Graceful shutdown.** On `SIGTERM` the server flips readiness to `503`,
drains in-flight requests, then checkpoints and fsyncs the WAL. Wire your
orchestrator's `preStop`/termination grace period to allow the drain.
- [ ] **Health probes.** Use `GET /health` for readiness (`200` ok / `503` draining),
`GET /health/startup` and `GET /health/live` for startup/liveness.
- [ ] **Observability.** Scrape the metrics endpoint; import the dashboard and alerts
from [docs/ops/](../ops/) (`grafana-dashboard.json`, `prometheus-alerts.yaml`,
`monitoring.md`). Plan headroom with [docs/ops/capacity-planning.md](../ops/capacity-planning.md).
- [ ] **Backup & recovery.** Rehearse restore with [docs/ops/recovery.md](../ops/recovery.md).
- [ ] **Deployment.** Follow [docs/guides/server-deployment.md](./server-deployment.md)
for packaging and rollout, and the Kubernetes runbook in
[docs/runbooks/](../runbooks/) for cluster orchestration. The container images
live at `docker/` (`standalone` / `cluster` / `deploy`) — build from the repo
root so `COPY . .` sees the whole workspace.
> **On cluster mode — be honest with yourself.** tidalDB's cluster mode is
> **experimental** and currently runs **all regions in a single process** over
> loopback gRPC. It is *not* production multi-process HA (true multi-process HA is
> tracked as m8p10). It also replicates **global signals only** — `user_id` /
> `creator_id` contexts are rejected in cluster mode, so the personalized swipe loop
> (section 4) is a **single-node** feature today. Ship the For You feed on a
> vertically-scaled standalone node; reach for the experimental cluster only for
> read-scale experiments, per the [cluster runbook](../runbooks/cluster.md).
---
## See also
- [QUICKSTART.md](../../QUICKSTART.md) — the 5-minute ingest → rank loop.
- [USE_CASES.md → UC-01](../../USE_CASES.md#uc-01--personalized-feed--for-you) — product framing for the For You surface.
- [docs/guides/embeddings.md](./embeddings.md) — generating and writing real vectors.
- [docs/guides/server-deployment.md](./server-deployment.md) — packaging and rollout.
- [API.md](../../API.md) — full embedded + HTTP API reference.
- [`tidal/examples/foryou_feed.rs`](../../tidal/examples/foryou_feed.rs) — the verified, runnable version of this guide.