tidaldb/.sdlc/features/pg1-baseline-comparison/design.md

Baseline Comparison Study — Design

Architecture

Pure backend/library feature. No UI. Three components added:

1. experiment module (tidal/src/experiment/) — types, assignment function, report builder
2. db/experiment.rs — TidalDb impl block wiring experiment methods to existing infrastructure
3. chronological builtin profile — registered in ranking/builtins.rs

Module Layout

tidal/src/
├── experiment/
│   ├── mod.rs          # ExperimentConfig, ExperimentGroup, ExperimentReport,
│   │                   # GroupMetrics, MetricLift, assign_group()
│   └── report.rs       # ReportBuilder: scans UserSignalIndex per group
├── db/
│   └── experiment.rs   # TidalDb::experiment_group/profile/report
└── ranking/
    └── builtins.rs     # + chronological() profile

Assignment

FNV-1a hash over experiment_id.as_bytes() concatenated with user_id.to_le_bytes(). Bucket = hash % 10,000. Treatment if bucket < threshold. Pure function, no I/O, deterministic across platforms.

Report

ReportBuilder takes &Arc<SignalLedger> and &UserSignalIndex. Partitions users via assign_group, then for each group iterates users calling user_signal_count() (new method on UserSignalIndex that sums AllTime counts across all entities for a user+signal_type pair) and user_activity_split() for return rate.

Key Design Decisions

1. No new storage — all metrics derived from existing UserSignalIndex DashMap scans
2. New UserSignalIndex methods — user_signal_count(user_id, type_id) and user_activity_split(user_id, boundary_ns) added to support per-user aggregation across all entities
3. chronological profile — uses existing Sort::New, no new sort variant needed
4. Thread safety — all types are Send + Sync, assign_group is pure, report takes &self

Error Handling

• InvalidInput: treatment_fraction out of range, empty signal lists
• Internal: no schema/ledger available
• No new error variants needed