# Design: Quality & Diversity Baseline

## Overview

This is a backend-only change. No UI, no API surface change, no schema migration. The deliverable is a new built-in ranking profile (`brief`) and integration tests proving quality and diversity invariants.

## Architecture

All changes are confined to three areas:

```
tidal/src/ranking/builtins.rs        ← new profile definition + registration
tidal/src/query/executor/mod.rs      ← format enrichment fix + metadata loading trigger
tidal/tests/p1_quality_diversity.rs  ← integration tests
```

No new modules, no new types, no new traits. The existing `Gate`, `DiversitySpec`, `Boost`, and `Sort` types are sufficient.

## Profile Definition

```
ranking/builtins.rs
└── fn brief() -> RankingProfile
    ├── sort: Hot { gravity: 1.5 }
    ├── gates:
    │   ├── Gate { signal: "view", agg: Value, window: AllTime, min_threshold: 3.0 }
    │   └── Gate { signal: "completion", agg: Value, window: AllTime, min_threshold: 1.0 }
    ├── boosts:
    │   ├── Boost { signal: "view", agg: DecayScore, window: AllTime, weight: 1.0 }
    │   ├── Boost { signal: "like", agg: DecayScore, window: AllTime, weight: 2.0 }
    │   └── Boost { signal: "completion", agg: DecayScore, window: AllTime, weight: 1.5 }
    ├── diversity:
    │   ├── max_per_creator: 2
    │   └── format_mix_max_fraction: 0.6
    └── exploration: 0.0  (disabled — bypasses quality gates)
```

### Gate Behavior

Gates are evaluated in `passes_gates()` (in `ranking/executor/helpers.rs`) which runs **before** scoring. This means:

1. `view / Value / AllTime >= 3.0` — reads the AllTime windowed count for "view". Items with fewer than 3 total views are excluded before any scoring happens.
2. `completion / Value / AllTime >= 1.0` — reads the AllTime windowed count for "completion". Items with zero completions are excluded.

Both gates use `DegradationLevel::Full` (never coarsened), ensuring gate evaluation precision is never sacrificed under load.

**Schema dependency:** Gates reference signals `view` and `completion`. If either signal is not in the schema, `read_agg` returns 0.0, which fails the gate threshold, meaning all items are excluded. This is the correct behavior — a brief without view/completion signals has no quality signal to rank on and should surface nothing rather than surface garbage.

### Exploration Disabled

Exploration is set to 0.0 because `inject_exploration()` runs **after** gate filtering and injects random candidates from the unscored universe. These candidates bypass quality gates, defeating the curated-brief guarantee. This was discovered during integration testing.

### Diversity Behavior

Diversity is enforced by the existing `DiversitySelector` in Stage 5 of the scoring pipeline:

1. **max_per_creator: 2** — Greedy selection skips items from any creator that already has 2 items in the result set. Multi-stage relaxation ensures the result count invariant (INV-RANK-5) holds.
2. **format_mix_max_fraction: 0.6** — No single format can exceed 60% of results. For a 10-item brief, max 6 items of one format.

These are profile-level defaults. Callers can override via `RetrieveBuilder::diversity()` for query-level control.

### Pipeline Fixes

Two fixes in the scoring pipeline were required:

1. **Format enrichment** (`query/executor/mod.rs`): The metadata enrichment loop only populated `ScoredCandidate.creator_id`. Extended to also populate `ScoredCandidate.format` from item metadata's "format" field.

2. **Metadata loading trigger** (`query/executor/mod.rs`): `needs_metadata_for_creator_grouping` only checked `max_per_creator`. Extended to also check `format_mix_max_fraction`, ensuring metadata is loaded when format diversity is configured.

### Scoring Formula

The sort mode is `Hot { gravity: 1.5 }`, which uses the Reddit/HN age-decay formula. This balances recency with engagement — appropriate for a daily brief cadence where yesterday's high-engagement content should still appear but not dominate.

Three boosts add quality signals:
- `view * 1.0` — baseline popularity
- `like * 2.0` — quality signal (2x weight reflects explicit positive engagement)
- `completion * 1.5` — deeper engagement signal

## Registration

In `register_builtins()`:

```rust
registry.register(brief())?;  // added after date_saved()
```

Total built-in profiles: 27 (includes profiles from parallel features).

## Integration Test Design

`tidal/tests/p1_quality_diversity.rs`:

### Test 1: `quality_gate_excludes_low_view_items`
- Create schema with `view`, `completion`, `like` signals.
- Write 10 items: 5 with >= 3 views + >= 1 completion, 5 with 1-2 views.
- Retrieve using `brief` profile.
- Assert: only the 5 high-quality items appear.

### Test 2: `quality_gate_excludes_zero_completion_items`
- Write 10 items: all have >= 3 views, but 5 have 0 completions.
- Retrieve using `brief` profile.
- Assert: only the 5 items with completions appear.

### Test 3: `creator_diversity_enforced`
- Write 30 items from 5 creators (10 from creator 1, 5 each from creators 2-5).
- All pass quality gates.
- Retrieve with limit 10.
- Assert: no single creator has all 10 items; at least 2 creators represented.

### Test 4: `format_diversity_enforced`
- Write 12 items: 6 video, 6 podcast, all different creators, all pass quality gates.
- Retrieve with limit 10.
- Assert: no format exceeds 60%.

### Test 5: `brief_profile_registered`
- Open DB, retrieve with `brief` profile on empty DB.
- Assert: no error (profile exists), empty results.

### Test 6: `combined_quality_and_diversity`
- 13 items: 10 high-quality from 5 creators, 3 low-quality.
- Retrieve brief with limit 10.
- Assert all constraints: quality gates exclude low-quality, creator diversity <= 2 per creator, format diversity <= 60%.

## Data Flow

```
RETRIEVE items USING PROFILE brief LIMIT 10
  │
  ├─ Stage 1: Candidate Generation (Scan)
  │   └─ All items from universe
  │
  ├─ Stage 2: Filter Evaluation
  │   └─ User-level filters (unseen, unblocked, etc.)
  │
  ├─ Stage 3: Signal Scoring
  │   ├─ Gate: view/Value/AllTime >= 3 → exclude if below
  │   ├─ Gate: completion/Value/AllTime >= 1 → exclude if below
  │   ├─ Base: Hot(gravity=1.5) scoring
  │   ├─ Boost: view * 1.0 + like * 2.0 + completion * 1.5
  │   └─ Normalize [0, 1]
  │
  ├─ Stage 5: Diversity
  │   ├─ max_per_creator: 2
  │   └─ format_mix: 0.6
  │
  └─ Stage 6: Result Assembly
      └─ Top 10, cursor, stats
```

## Risks & Mitigations

| Risk | Mitigation |
|------|-----------|
| Schema missing `view` or `completion` signal | Gates return 0.0 for unknown signals, excluding all items. This is correct — no quality signal = no brief. Integration tests verify this is the expected behavior. |
| Small catalog where all items are from same creator | DiversitySelector's multi-stage relaxation fills to target count even when constraints are unsatisfiable. The result set reports `constraints_satisfied: false` but still returns results. |
| Gate thresholds too aggressive for small alpha catalog | Thresholds are deliberately low (3 views, 1 completion). Can be tuned by registering a custom profile with adjusted gates. |
| Exploration re-introduces gate-failing candidates | Exploration disabled (0.0) for this profile. Discovery surfaces use separate profiles with exploration enabled. |