129 lines
7.5 KiB
Markdown
129 lines
7.5 KiB
Markdown
# Spec: Briefing UX & Reason Labels
|
|
|
|
## Problem
|
|
|
|
When tidalDB returns ranked results via `retrieve()` or `search()`, the application receives entity IDs and scores but no structured explanation of **why** each item was surfaced. The `signal_snapshot` field on `ScoredCandidate` is always empty (`vec![]`), and no "reason label" taxonomy exists. Without this, the consuming application cannot:
|
|
|
|
1. Show users *why* an item appeared in their feed ("From a creator you follow", "Trending in Jazz", "Because you liked similar content")
|
|
2. Build trust through transparency -- users who understand why they see something engage more and churn less
|
|
3. Debug ranking behavior -- operators cannot tell which signal dominated a result's position
|
|
4. Close the feedback loop with reason-aware negative signals ("Not interested *because* of this reason")
|
|
|
|
The p1 milestone ("Concierge Alpha") requires a daily ranked brief where each card carries a human-readable reason label. This feature delivers the engine-side infrastructure for that.
|
|
|
|
## Solution
|
|
|
|
Add a **reason label** system to tidalDB's query response types. Each ranked result carries one or more `ReasonLabel` values that describe the primary factors that caused the item to appear and rank where it did. Reasons are derived deterministically from the scoring pipeline -- not guessed after the fact.
|
|
|
|
### Reason Taxonomy
|
|
|
|
A closed enum of reason codes, each with a structured payload:
|
|
|
|
| Reason Code | Description | Example Label |
|
|
|---|---|---|
|
|
| `FollowedCreator` | Item is from a creator the user follows | "From Jazz Academy" |
|
|
| `TrendingGlobal` | Item is in the global trending set | "Trending now" |
|
|
| `TrendingInCategory` | Item is trending within a filtered category | "Trending in Jazz" |
|
|
| `TrendingInCohort` | Item is trending within the user's cohort | "Popular with listeners like you" |
|
|
| `PreferenceMatch` | Item's embedding is close to user's preference vector | "Matches your interests" |
|
|
| `SocialProof` | People the user follows engaged with this item | "Liked by people you follow" |
|
|
| `HighQuality` | Item has high completion rate and engagement ratio | "Highly rated" |
|
|
| `HiddenGem` | Item scored via the hidden_gems profile | "Hidden gem" |
|
|
| `NewFromFollowed` | Recent item from a followed creator | "New from Jazz Academy" |
|
|
| `Rising` | Item is overperforming its baseline | "Rising" |
|
|
| `Controversial` | Item scored via the controversial profile | "Generating discussion" |
|
|
| `ExplorationBudget` | Item was injected via the exploration mechanism | "Something new to try" |
|
|
| `TopInWindow` | Item ranked highly in a windowed top sort | "Top this week" |
|
|
| `SemanticMatch` | Item matched via ANN/embedding similarity | "Similar to content you enjoy" |
|
|
| `TextRelevance` | Item matched via BM25 text search | "Matches your search" |
|
|
| `CoEngagement` | Item was co-engaged with the seed item | "Viewers also watched" |
|
|
| `SessionContext` | Item surfaced due to active session/agent signals | "Based on this conversation" |
|
|
| `CohortPopular` | Item is popular within the user's cohort | "Popular in your area" |
|
|
| `SavedSearchMatch` | Item matched a user's saved search | "Matches your saved search" |
|
|
|
|
### Source Exposure Rules
|
|
|
|
Not all internal scoring details should be exposed to end users. The system distinguishes between:
|
|
|
|
- **User-facing reasons**: Reasons suitable for display in the UI (e.g., "From Jazz Academy", "Trending now"). These use natural language templates.
|
|
- **Operator-facing reasons**: Full signal decomposition including weights, decay values, and pipeline stage. Always available in `QueryStats` or debug mode.
|
|
|
|
The `ReasonLabel` struct carries both:
|
|
```rust
|
|
pub struct ReasonLabel {
|
|
pub code: ReasonCode,
|
|
pub context: HashMap<String, String>,
|
|
pub weight: f64,
|
|
}
|
|
```
|
|
|
|
### Population Rules
|
|
|
|
Reason labels are populated during the scoring pipeline, not post-hoc:
|
|
|
|
1. **Stage 1 (Candidate Generation)**: Tag candidates with their source strategy (`FollowedCreator`, `TrendingGlobal`, `ExplorationBudget`, `CoEngagement`).
|
|
2. **Stage 3 (Signal Scoring)**: Tag with dominant signal contributors (`PreferenceMatch`, `SocialProof`, `HighQuality`, `Rising`, `Controversial`, `HiddenGem`, `TopInWindow`).
|
|
3. **Stage 3b (Search Scoring)**: Tag with retrieval method (`TextRelevance`, `SemanticMatch`).
|
|
4. **Stage 4 (Diversity)**: No new labels, but diversity-displaced items retain their original reasons.
|
|
5. **Assembly**: Select the top 1-3 reasons by weight for each result. Attach to `RetrieveResult` and `SearchResultItem`.
|
|
|
|
### Dominance Threshold
|
|
|
|
A reason is included only if its contribution exceeds a minimum threshold (default: 10% of the item's total score contribution). This prevents noisy labels like "0.2% from social proof" from appearing.
|
|
|
|
### API Surface Changes
|
|
|
|
```rust
|
|
pub struct RetrieveResult {
|
|
pub entity_id: EntityId,
|
|
pub score: f64,
|
|
pub rank: usize,
|
|
pub signals: Vec<Signal>,
|
|
pub reasons: Vec<ReasonLabel>, // NEW
|
|
}
|
|
|
|
pub struct SearchResultItem {
|
|
pub entity_id: EntityId,
|
|
pub score: f64,
|
|
pub rank: usize,
|
|
pub bm25_score: Option<f32>,
|
|
pub semantic_score: Option<f32>,
|
|
pub signals: Vec<Signal>,
|
|
pub metadata: Option<HashMap<String, String>>,
|
|
pub reasons: Vec<ReasonLabel>, // NEW
|
|
}
|
|
```
|
|
|
|
The `reasons` field is always populated (never `None`). When no reason can be determined (e.g., anonymous query with no profile context), the vec is empty.
|
|
|
|
## Non-Goals
|
|
|
|
- **Natural language template rendering**: The engine provides structured reason codes and context. The application layer renders them into localized strings. tidalDB does not own i18n.
|
|
- **Reason-based re-ranking**: Reasons are observational, not prescriptive. They do not change the ranking order.
|
|
- **User-configurable reason visibility**: All reasons are returned; the application decides what to show. tidalDB does not filter reasons based on user preferences.
|
|
- **A/B testing of reason labels**: The engine always populates reasons. Whether to show them is an application decision.
|
|
- **Retroactive reason computation**: Reasons are computed live during query execution. There is no stored history of "why was item X shown to user Y on date Z."
|
|
|
|
## Success Criteria
|
|
|
|
1. Every `RetrieveResult` and `SearchResultItem` carries a `reasons: Vec<ReasonLabel>` field.
|
|
2. For a `for_you` query with `for_user` set, at least 80% of results have at least one non-empty reason label.
|
|
3. For a `trending` query, 100% of results carry the `TrendingGlobal` or `TrendingInCategory` reason.
|
|
4. For a `following` query, 100% of results carry the `FollowedCreator` reason.
|
|
5. For a `search` query, 100% of results carry either `TextRelevance` or `SemanticMatch` (or both).
|
|
6. Reason population adds less than 5% overhead to the query pipeline (measured via the existing benchmark suite).
|
|
7. The `signal_snapshot` field on `ScoredCandidate` is populated with actual signal values during scoring (fixing the current empty-vec behavior).
|
|
8. `ReasonCode` is a closed, exhaustive enum -- no string-based reason codes.
|
|
9. Integration tests verify reason labels for each profile type (for_you, trending, following, search, hidden_gems, rising, controversial, related).
|
|
|
|
## Dependencies
|
|
|
|
- Existing `ScoredCandidate.signal_snapshot` infrastructure (currently unused but structurally present)
|
|
- Profile registry and `RankingProfile` definitions
|
|
- `PreferenceVectors` for preference-match detection
|
|
- `CreatorItemsBitmap` and `UserStateIndex` for followed-creator detection
|
|
- `CohortSignalLedger` for cohort-trending detection
|
|
|
|
## Open Questions
|
|
|
|
None -- the taxonomy and architecture are well-constrained by the existing scoring pipeline. |