tidaldb/SEQUENCE.md

# Sequence Diagrams

User-perspective flows for every major surface. Each diagram shows what the application sends, what tidalDB does internally, and what signals flow back to close the loop.

---

## Table of Contents

- [UC-01 · For You Feed](#uc-01--for-you-feed)
- [UC-02 · Search with Personalized Ranking](#uc-02--search-with-personalized-ranking)
- [UC-03 · Trending / Popular](#uc-03--trending--popular)
- [UC-04 · Following Feed](#uc-04--following-feed)
- [UC-05 · Related Content / Up Next](#uc-05--related-content--up-next)
- [UC-06 · Browse / Category Discovery](#uc-06--browse--category-discovery)
- [UC-07 · Notification Prioritization](#uc-07--notification-prioritization)
- [UC-15 · Cohort-Scoped Trending](#uc-15--cohort-scoped-trending)
- [Core · Engagement Feedback Loop](#core--engagement-feedback-loop)
- [Write · Content Ingest](#write--content-ingest)

---

## UC-01 · For You Feed

User opens the app. tidalDB retrieves candidates, scores them against the user's preference profile, enforces diversity, and returns a ready-to-render batch. Pagination continues with previously-returned IDs excluded.

```mermaid
sequenceDiagram
    actor User
    participant App
    participant TidalDB

    User->>App: Opens app / pulls to refresh

    App->>TidalDB: RETRIEVE items\nFOR USER @u123\nCONTEXT feed\nUSING PROFILE for_you\nFILTER unseen, unblocked\nDIVERSITY max_per_creator:2\nLIMIT 50

    Note over TidalDB: 1. Load user preference vector\n2. ANN retrieval over item embeddings\n3. Apply seen / blocked filters\n4. Score via for_you profile:\n   preference_match\n   × engagement_velocity(24h)\n   × recency_decay\n   × social_proof\n5. Enforce diversity constraints\n6. Return ranked batch

    TidalDB-->>App: [{item_id, score, signals_snapshot} × 50]
    App-->>User: Feed renders

    User->>App: Scrolls to bottom

    App->>TidalDB: RETRIEVE items\nFOR USER @u123\nCONTEXT feed\nUSING PROFILE for_you\nFILTER unseen, unblocked\nEXCLUDE [previously_returned_ids]\nLIMIT 50

    TidalDB-->>App: Next batch of 50
    App-->>User: Feed extends
```

**Signals powering this query:** user preference vector (built from history), item view velocity (24h window), item completion rate, user→creator interaction weight, social graph engagement, item age with decay curve, skip and hide signals.

---

## UC-02 · Search with Personalized Ranking

Text relevance is the floor — an irrelevant result never appears just because the user likes the creator. Personalization reorders within the relevant candidate set. A beginner and an expert searching the same query get different orderings.

```mermaid
sequenceDiagram
    actor User
    participant App
    participant Embed as Embedding Service
    participant TidalDB

    User->>App: Types "rust tutorial beginner"

    App->>Embed: embed("rust tutorial beginner")
    Embed-->>App: query_vector[1536]

    App->>TidalDB: SEARCH items\nQUERY "rust tutorial beginner"\nVECTOR query_vector\nFOR USER @u123\nUSING PROFILE search\nDIVERSITY max_per_creator:2\nLIMIT 20

    Note over TidalDB: 1. BM25 text match → relevance scores\n2. ANN over item embeddings → semantic scores\n3. Merge: text(0.6) + semantic(0.4)\n4. Personalization layer:\n   user topic engagement history\n   item quality signals (completion, like ratio)\n   recency curve (slow decay for tutorials)\n5. Diversity pass\n6. Return ranked results

    TidalDB-->>App: [{item_id, relevance_score, rank} × 20]
    App-->>User: Search results render

    User->>App: Clicks result #3

    App->>TidalDB: SIGNAL search_click\nitem: @item_id\nuser: @u123\nquery_context: "rust tutorial beginner"\nrank_at_click: 3

    Note over TidalDB: Updates item relevance signal\nfor this query category.\nBoosts user→topic weight.

    TidalDB-->>App: ok

    User->>App: Watches 80% of video

    App->>TidalDB: SIGNAL completion\nitem: @item_id\nuser: @u123\nratio: 0.80

    Note over TidalDB: Strong positive on item quality.\nUpdates user preference vector\ntoward this topic and format.

    TidalDB-->>App: ok
```

---

## UC-03 · Trending / Popular

Velocity, not volume. A video posted 4 hours ago with a high share rate outranks a video with 10× the total views but slow recent growth. The same profile applies to global, category-scoped, social-graph-scoped, and cohort-scoped trending — only the candidate set and signal scope change.

```mermaid
sequenceDiagram
    actor User
    participant App
    participant TidalDB

    User->>App: Taps "Trending" tab

    App->>TidalDB: RETRIEVE items\nUSING PROFILE trending\nWINDOW 24h\nDIVERSITY max_per_creator:1\nLIMIT 25

    Note over TidalDB: Profile: trending\nPrimary:   share_velocity(6h)\nSecondary: view_velocity(6h)\nBoost:     new_user_reach (virality)\nGate:      engagement_ratio > 0.03\nNo personalization.\nNo total-count signals.

    TidalDB-->>App: [{item_id, trending_score, velocity_snapshot} × 25]
    App-->>User: Trending page renders

    User->>App: Taps "Jazz" category filter

    App->>TidalDB: RETRIEVE items\nUSING PROFILE trending\nFILTER category: jazz\nWINDOW 24h\nDIVERSITY max_per_creator:1\nLIMIT 25

    Note over TidalDB: Same profile. Same velocity signals.\nHard filter on category metadata.\nScoped candidate set.

    TidalDB-->>App: Trending in Jazz
    App-->>User: "Trending in Jazz" renders

    User->>App: Switches to "Among People I Follow"

    App->>TidalDB: RETRIEVE items\nUSING PROFILE trending\nFILTER social_graph: @u123 depth:2\nWINDOW 24h\nLIMIT 25

    Note over TidalDB: Same profile.\nCandidates constrained to items\nengaged by users @u123 follows.

    TidalDB-->>App: Trending among follows
    App-->>User: Social trending renders

    User->>App: Switches to "Trending in My Demo"

    App->>TidalDB: RETRIEVE items\nUSING PROFILE trending\nCOHORT locale:en-US, age:18-24\nWINDOW 24h\nLIMIT 25

    Note over TidalDB: Same profile.\nSignal aggregation scoped to\nusers matching cohort predicate.

    TidalDB-->>App: Cohort-scoped trending
    App-->>User: Demographic trending renders
```

**One profile, four scopes.** Global, category, social, and cohort trending are the same ranking profile applied to different signal scopes. No code changes — just query parameters.

---

## UC-04 · Following Feed

The surface where users expect control. Recency-dominant, minimal algorithmic intervention. A creator's worst-performing video still appears here — because the user chose to follow them.

```mermaid
sequenceDiagram
    actor User
    participant App
    participant TidalDB

    User->>App: Taps "Following" tab

    App->>TidalDB: RETRIEVE items\nFOR USER @u123\nFILTER relationship: follows\nUSING PROFILE following\nFILTER unseen\nLIMIT 50

    Note over TidalDB: Profile: following\nPrimary sort:  created_at DESC\nTiebreaker:    completion_rate\nHard filter:   creator in user's follows\nHard filter:   unseen by this user\nNo exploration budget.\nNo aggressive personalization.

    TidalDB-->>App: Chronological feed from follows
    App-->>User: Subscription feed renders

    User->>App: Scrolls — catching up from 3 days ago

    App->>TidalDB: RETRIEVE items\nFOR USER @u123\nFILTER relationship: follows\nFILTER created_at < @cursor_timestamp\nUSING PROFILE following\nLIMIT 50

    TidalDB-->>App: Older batch (cursor pagination)
    App-->>User: Older content loads

    User->>App: Follows a new creator

    App->>TidalDB: WRITE relationship\ntype: follows\nfrom: @u123\nto: @creator_id\ntimestamp: now()

    Note over TidalDB: Adds edge to relationship graph.\nUpdates user preference vector\nto include creator's topic signals.\nNew creator appears in next query.

    TidalDB-->>App: ok
```

---

## UC-05 · Related Content / Up Next

Anchored to the item just consumed. Blends semantic similarity with collaborative filtering and user taste. The autoplay accept/skip signal strengthens or decays the item-to-item pairing for future recommendations.

```mermaid
sequenceDiagram
    actor User
    participant App
    participant TidalDB

    User->>App: Finishes watching @item_abc (Rust tutorial, beginner)

    App->>TidalDB: SIGNAL completion\nitem: @item_abc\nuser: @u123\nratio: 0.94

    Note over TidalDB: Appends to @item_abc signal ledger.\nUpdates user→item_abc relationship.\nUpdates user preference vector\ntoward rust/programming/tutorials.

    TidalDB-->>App: ok

    App->>TidalDB: RETRIEVE items\nSIMILAR TO @item_abc\nFOR USER @u123\nUSING PROFILE related\nFILTER unseen\nEXCLUDE creator: @item_abc.creator_id (top 3 only)\nLIMIT 10

    Note over TidalDB: Profile: related\n1. ANN: items near @item_abc embedding\n2. Collaborative: items co-engaged with @item_abc\n3. Personalize: re-rank by user preference match\n4. Quality gate: completion_rate > 0.4\n5. Diversity: avoid same creator in top 3\n6. Return

    TidalDB-->>App: [{item_id, similarity_score, collab_score} × 10]
    App-->>User: Up Next queue renders

    User->>App: Autoplay begins on result #1

    App->>TidalDB: SIGNAL autoplay_accept\nsource_item: @item_abc\ntarget_item: @item_xyz\nuser: @u123

    Note over TidalDB: Strong positive on item_abc → item_xyz pairing.\nStrengthens collaborative edge.

    TidalDB-->>App: ok

    User->>App: Skips after 8 seconds

    App->>TidalDB: SIGNAL skip\nitem: @item_xyz\nuser: @u123\ndwell_ms: 8200\ncontext: autoplay_from @item_abc

    Note over TidalDB: Negative on this pairing.\nDecays item_abc → item_xyz\ncollaborative edge slightly.

    TidalDB-->>App: ok
```

---

## UC-06 · Browse / Category Discovery

Quality-dominant ranking within a filtered candidate set. Mix of established content and breakout newcomers. Sort mode switches (Top, New, All Time) are different profiles on the same candidate set — no application logic needed.

```mermaid
sequenceDiagram
    actor User
    participant App
    participant TidalDB

    User->>App: Taps "Jazz" category

    App->>TidalDB: RETRIEVE items\nFILTER category: jazz\nUSING PROFILE browse\nDIVERSITY max_per_creator:2\nLIMIT 20

    Note over TidalDB: Profile: browse\nPrimary: quality_score\n  completion_rate(0.5)\n  + like_ratio(0.3)\n  + reach(0.2)\nRecency boost: 30d half-life\nBreakout bonus: age < 14d\n  AND velocity_percentile > 0.9

    TidalDB-->>App: [{item_id, quality_score} × 20]
    App-->>User: Jazz browse page renders

    User->>App: Switches sort to "New"

    App->>TidalDB: RETRIEVE items\nFILTER category: jazz\nUSING PROFILE new\nLIMIT 20

    Note over TidalDB: Profile: new\nSort: created_at DESC\nNo quality gate.

    TidalDB-->>App: Latest jazz content
    App-->>User: New jazz renders

    User->>App: Switches sort to "Top All Time"

    App->>TidalDB: RETRIEVE items\nFILTER category: jazz\nUSING PROFILE top_all_time\nDIVERSITY max_per_creator:3\nLIMIT 20

    Note over TidalDB: Profile: top_all_time\nSort: total_completion_weighted_views DESC\nNo recency boost. No decay.\nPure historical quality.

    TidalDB-->>App: All-time top jazz
    App-->>User: Top jazz renders
```

---

## UC-07 · Notification Prioritization

Of all events since the user was last active, tidalDB decides which are worth a push and in what order they appear in the notification center. Open and dismiss signals feed back to adjust future notification priority per creator.

```mermaid
sequenceDiagram
    participant Job as Background Job
    participant TidalDB
    participant Push as Push Service
    actor User

    Job->>TidalDB: RETRIEVE notifications\nFOR USER @u123\nSINCE @last_seen_timestamp\nUSING PROFILE notification\nLIMIT push:3, inbox:20

    Note over TidalDB: Profile: notification\nCandidates: events from followed creators\n  + social graph activity\nScore by:\n  relationship_strength(user, creator)\n  × item engagement_velocity at event time\n  × user notification_open_rate\nHard filter: event_age > 48h → suppress\nHard filter: max 1 per creator per day

    TidalDB-->>Job: push_candidates[3], inbox_candidates[20]

    Job->>Push: Send push notifications (top 3)
    Push->>User: Push notification arrives

    User->>Push: Taps notification
    Push->>Job: opened — item @item_id
    Job->>TidalDB: SIGNAL notification_open\nuser: @u123\ncreator: @creator_id\nitem: @item_id

    Note over TidalDB: Strong positive on user→creator\nrelationship for notification context.\nIncreases future notification priority\nfor this creator.

    TidalDB-->>Job: ok

    User->>Push: Swipes to dismiss
    Push->>Job: dismissed
    Job->>TidalDB: SIGNAL notification_dismiss\nuser: @u123\ncreator: @creator_id

    Note over TidalDB: Mild negative on relationship\nfor notification context.\nReduces push frequency\nfrom this creator slightly.

    TidalDB-->>Job: ok
```

---

## UC-15 · Cohort-Scoped Trending

User explores what's trending within their audience segment. tidalDB scopes signal aggregation to users matching the cohort predicate, ranks by velocity within that cohort, and supports search composition on top.

```mermaid
sequenceDiagram
    actor User
    participant App
    participant TidalDB

    User->>App: Opens "Trending For You"

    Note over App: App resolves user's primary cohort<br/>from their attributes:<br/>locale:en-US, age:18-24, interest:music

    App->>TidalDB: RETRIEVE items<br/>USING PROFILE trending<br/>COHORT locale:en-US, age:18-24, interest:music<br/>WINDOW 24h<br/>DIVERSITY max_per_creator:1<br/>LIMIT 25

    Note over TidalDB: 1. Resolve cohort: users matching predicate<br/>2. Load cohort-scoped signal aggregates<br/>   (view_velocity, share_velocity scoped<br/>    to signals from cohort members)<br/>3. Rank by cohort-scoped velocity<br/>4. Gate: engagement_ratio > 0.03<br/>5. Enforce diversity<br/>6. Return ranked batch

    TidalDB-->>App: [{item_id, cohort_trending_score, velocity_snapshot} × 25]
    App-->>User: "Trending in your world" renders

    User->>App: Searches "jazz piano" within trending

    App->>TidalDB: SEARCH items<br/>QUERY "jazz piano"<br/>WITHIN TRENDING<br/>COHORT locale:en-US, age:18-24, interest:music<br/>WINDOW 24h<br/>LIMIT 20

    Note over TidalDB: 1. Cohort-scoped trending candidates<br/>2. BM25 text match within candidates<br/>3. Merge: trending_score × text_relevance<br/>4. Return intersection

    TidalDB-->>App: [{item_id, composite_score} × 20]
    App-->>User: Search-within-trending results render

    User->>App: Switches to "Trending Globally"

    App->>TidalDB: RETRIEVE items<br/>USING PROFILE trending<br/>WINDOW 24h<br/>DIVERSITY max_per_creator:1<br/>LIMIT 25

    Note over TidalDB: Same profile, no cohort scope.<br/>Global signal aggregates used.<br/>Different results than cohort view.

    TidalDB-->>App: Global trending results
    App-->>User: "Trending Globally" renders
```

**Three layers, one engine.** Global trending, cohort trending, and search-within-cohort-trending are the same ranking profile applied to different signal scopes. The profile doesn't change — the signal aggregation scope does.

---

## Core · Engagement Feedback Loop

Every interaction closes the loop in a single write transaction. There is no ETL, no Kafka consumer, no feature store sync. The next ranking query — even 100ms later — sees the updated state.

```mermaid
sequenceDiagram
    actor User
    participant App
    participant TidalDB

    User->>App: Likes a video

    App->>TidalDB: SIGNAL like\nitem: @item_id\nuser: @u123\ntimestamp: now()

    Note over TidalDB: Atomic write transaction:\n1. Append event to item signal ledger\n2. Update windowed aggregates:\n   like_count_1h, _24h, _7d\n3. Recompute like_velocity\n4. Update user→item relationship weight\n5. Increment user→creator interaction_weight\n6. Update user preference vector\n   toward item's topic embedding\n7. Attribute signal to user's cohorts\n   (increment cohort-scoped counters)\n8. Commit

    TidalDB-->>App: ok

    Note over App: Next RETRIEVE for this user\nreflects updated signals immediately.

    User->>App: Hides a video ("Not interested")

    App->>TidalDB: SIGNAL hide\nitem: @item_id\nuser: @u123

    Note over TidalDB: 1. Set permanent hard-negative flag\n   on user→item relationship\n2. Decay user→creator interaction_weight\n3. Update user preference vector\n   AWAY from item's topic embedding\n4. Item excluded from all future\n   queries for this user

    TidalDB-->>App: ok

    User->>App: Blocks a creator

    App->>TidalDB: SIGNAL block\nuser: @u123\ntarget_creator: @creator_id

    Note over TidalDB: 1. Set permanent hard block\n   on user→creator relationship\n2. All items by @creator_id excluded\n   from every query for this user\n3. Existing relationship edges zeroed

    TidalDB-->>App: ok
```

---

## Write · Content Ingest

How a new item enters the system and becomes immediately retrievable and rankable. Cold start is handled by the database via an exploration budget — the application does not manage this.

```mermaid
sequenceDiagram
    actor Creator
    participant App
    participant Embed as Embedding Service
    participant TidalDB

    Creator->>App: Uploads video\n(title, description, tags, category, transcript)

    App->>Embed: embed(title + description + transcript)
    Embed-->>App: content_vector[1536]

    App->>TidalDB: WRITE item\nid: @item_id\ncreator: @creator_id\nmetadata: {title, description, tags, category, duration}\nembedding: content_vector\ncreated_at: now()

    Note over TidalDB: 1. Store metadata in entity store\n2. Index text fields into inverted index\n3. Insert vector into ANN index (HNSW)\n4. Initialize signal ledger (all zeros)\n5. Apply new-item exploration budget:\n   appears in a small % of for_you feeds\n   before signals accumulate\n6. Link to creator entity\n7. Commit — item is immediately queryable

    TidalDB-->>App: ok, @item_id indexed

    App-->>Creator: "Your content is live"

    Note over TidalDB: Item is now retrievable in:\n• Search (text + semantic)\n• Following feeds of creator's followers\n• Browse / category pages\n• Trending (once signals accumulate)\n• For You (exploration budget active)
```

**Cold start is a first-class problem.** New content has no engagement signals. tidalDB handles this natively — new items receive a configurable exploration window proportional to the creator's relationship strength with their existing followers. The application does not manage this.

---

## Signal Reference

| Signal | Type | Decay | Primary Use |
|---|---|---|---|
| `view` | count | slow (7d half-life) | baseline engagement |
| `completion` | ratio 0–1 | very slow | quality signal |
| `like` | count | slow | positive sentiment |
| `share` | count | medium | virality |
| `comment` | count | medium | community |
| `skip` | count | fast (1d half-life) | negative quality |
| `hide` | bool | permanent | hard negative |
| `block` | bool | permanent | hard filter |
| `follow` | bool | permanent | relationship |
| `interaction_weight` | float | slow | relationship strength |
| `dwell_time` | duration | medium | true engagement |
| `autoplay_accept` | bool | medium | recommendation quality |
| `notification_open` | bool | slow | creator notification priority |
| `notification_dismiss` | bool | medium | reduce push frequency |
| `search_click` | count + rank | medium | query relevance signal |