# Baseline Comparison Study — QA Plan

## Scope
Verify the A/B experiment framework: chronological profile, deterministic assignment, metric aggregation, lift calculation, and integration test coverage.

## Test Matrix

### 1. Chronological Profile
| # | Scenario | Method | Expected |
|---|----------|--------|----------|
| 1.1 | Reverse created_at ordering | Integration test | Items returned newest-first |
| 1.2 | Deterministic across users | Integration test | Same query produces identical order |
| 1.3 | Profile registered as builtin | Unit test | `chronological` in ProfileRegistry |

### 2. Experiment Assignment
| # | Scenario | Method | Expected |
|---|----------|--------|----------|
| 2.1 | Deterministic assignment | Integration test | Same user_id + config always yields same group |
| 2.2 | Balanced 50/50 split | Integration test | 10k users within 4750–5250 treatment |
| 2.3 | Profile name matches group | Integration test | Treatment -> treatment_profile, Control -> control_profile |
| 2.4 | FNV-1a hash quality | Unit test | No systematic bias across user_id ranges |

### 3. Report Aggregation
| # | Scenario | Method | Expected |
|---|----------|--------|----------|
| 3.1 | Synthetic signal report | Integration test | Treatment CTR > Control CTR with known signal pattern |
| 3.2 | Group sizes sum to total | Integration test | treatment_users + control_users == user_ids.len() |
| 3.3 | Reproducible report | Integration test | Two calls with same inputs yield identical output |
| 3.4 | Zero-signal edge case | Unit test | GroupMetrics with 0 views yields 0.0 rates (no NaN/panic) |
| 3.5 | CTR lift positive | Integration test | lift.ctr_lift > 0.0 when treatment has more clicks |

### 4. UserSignalIndex Extensions
| # | Scenario | Method | Expected |
|---|----------|--------|----------|
| 4.1 | user_signal_count aggregation | Unit/Integration | Counts across all entities for a user+signal_type |
| 4.2 | user_activity_split detection | Unit/Integration | Returns (has_any, has_recent) correctly |

## Pass Criteria
- All 7 integration tests in `pg1_baseline.rs` pass
- All 16 unit tests in `experiment/mod.rs` pass
- `cargo check --manifest-path tidal/Cargo.toml` clean (no warnings from experiment code)
- No panics on edge cases (zero views, empty user list)