# Baseline Comparison Study — QA Results

## Test Execution Summary

| Suite | Tests | Passed | Failed | Duration |
|-------|-------|--------|--------|----------|
| Unit tests (`experiment::tests`) | 17 | 17 | 0 | 0.00s |
| Builtin tests (`chronological`) | 2 | 2 | 0 | 0.00s |
| Integration tests (`pg1_baseline`) | 7 | 7 | 0 | 0.04s |
| **Total** | **26** | **26** | **0** | **< 1s** |

## Compilation

- `cargo check --manifest-path tidal/Cargo.toml --lib` — CLEAN (no errors, no warnings from experiment code)
- `cargo test --manifest-path tidal/Cargo.toml --lib -- experiment` — 17 passed
- `cargo test --manifest-path tidal/Cargo.toml --test pg1_baseline` — 7 passed

## QA Plan Coverage

### 1. Chronological Profile
| # | Scenario | Result | Notes |
|---|----------|--------|-------|
| 1.1 | Reverse created_at ordering | PASS | `chronological_profile_returns_reverse_created_at` — newest item (id=10) first, oldest (id=1) last |
| 1.2 | Deterministic across users | PASS | `chronological_produces_same_order_for_different_users` — same query, identical order |
| 1.3 | Profile registered as builtin | PASS | `all_twentyseven_builtins_registered` — 27 builtins including chronological |

### 2. Experiment Assignment
| # | Scenario | Result | Notes |
|---|----------|--------|-------|
| 2.1 | Deterministic assignment | PASS | `experiment_group_assignment_is_deterministic` — 100 iterations same result |
| 2.2 | Balanced 50/50 split | PASS | `experiment_group_balanced_split` — 10k users within 4750-5250 |
| 2.3 | Profile name matches group | PASS | `experiment_profile_returns_correct_name` — Treatment=for_you, Control=chronological |
| 2.4 | FNV-1a hash quality | PASS | `treatment_fraction_0_5_produces_balanced_split` + `treatment_fraction_0_1_produces_10_percent_split` |

### 3. Report Aggregation
| # | Scenario | Result | Notes |
|---|----------|--------|-------|
| 3.1 | Synthetic signal report | PASS | `experiment_report_with_synthetic_signals` — treatment CTR > control CTR |
| 3.2 | Group sizes sum to total | PASS | Verified: treatment_users + control_users == 100 |
| 3.3 | Reproducible report | PASS | `experiment_report_is_reproducible` — two calls identical |
| 3.4 | Zero-signal edge case | PASS | `GroupMetrics::zero()` returns 0.0 rates; `relative_lift(0.0, 0.0)` returns 0.0 |
| 3.5 | CTR lift positive | PASS | `report.lift.ctr_lift > 0.0` verified in integration test |

### 4. UserSignalIndex Extensions
| # | Scenario | Result | Notes |
|---|----------|--------|-------|
| 4.1 | user_signal_count aggregation | PASS | Used in report builder; verified via synthetic signal patterns |
| 4.2 | user_activity_split detection | PASS | Used in return rate calculation; verified implicitly |

## Issues Found During QA

1. **`signal_with_context` signature change:** The method now takes 6 arguments (added `creator_id: Option<u64>` from uncommitted M10 work). Fixed integration test to pass `None` for the 6th parameter. This is a pre-existing API change, not a defect in the experiment code.

## Verdict

**PASSED.** All 26 tests pass. All QA plan scenarios verified. One minor test fix applied for API compatibility with uncommitted M10 changes. No defects found in the experiment framework itself.