Baseline Comparison Study — Audit
Security & Privacy
| Check |
Status |
Notes |
| No PII in experiment data |
PASS |
Only user_id (opaque u64) used for assignment and aggregation |
| No external network calls |
PASS |
All computation is local, read-only over in-memory indexes |
| No new storage writes |
PASS |
Experiment module is purely read-only; no WAL entries, no fjall writes |
| Config validation |
PASS |
All three TidalDb methods validate config before processing |
| No secrets or credentials |
PASS |
No API keys, tokens, or auth material involved |
Correctness & Safety
| Check |
Status |
Notes |
| No panics on edge cases |
PASS |
Zero views yields 0.0 rates; zero control yields defined lift values |
| No integer overflow |
PASS |
Signal counts are u64; views/clicks/completions summed as u64 |
| No floating-point NaN |
PASS |
All division guarded by zero-check; relative_lift handles all zero-denominator cases |
| Deterministic assignment |
PASS |
FNV-1a hash is platform-independent (fixed constants, explicit byte ordering) |
| No unsafe code |
PASS |
No unsafe blocks in any experiment code |
| No unwrap in library code |
PASS |
All unwrap() confined to #[cfg(test)] blocks and integration tests |
Performance
| Check |
Status |
Notes |
| No hot-path impact |
PASS |
Experiment code is invoked on-demand, not in signal write or retrieve paths |
| Signal type resolution |
PASS |
Type IDs resolved once per report, not per user |
| Allocation profile |
PASS |
Two Vec allocations (treatment/control user lists) per report; no per-user allocations |
| O(n*m) scan acknowledged |
INFO |
user_signal_count scans full DashMap per user; acceptable for PG1 pilot scale (<1000 users) |
API Surface
| Check |
Status |
Notes |
| Public types re-exported |
PASS |
ExperimentConfig, ExperimentGroup, ExperimentReport, GroupMetrics, MetricLift all accessible from tidaldb:: |
| No breaking changes |
PASS |
All additions are new types/methods; no existing API modified |
| Doc comments complete |
PASS |
All public types, methods, and error conditions documented |
| Error types consistent |
PASS |
Uses existing TidalError::InvalidInput and TidalError::Internal |
Test Coverage
| Area |
Unit Tests |
Integration Tests |
| Assignment determinism |
1 |
1 |
| Balanced split |
2 (50/50, 10/90) |
1 (50/50) |
| Config validation |
6 |
0 |
| Lift math |
4 |
1 |
| Profile routing |
0 |
1 |
| Report aggregation |
0 |
2 |
| Chronological profile |
1 |
2 |
| Total |
16 |
7 |
Verdict
APPROVED. No security, correctness, or performance concerns. The implementation is read-only, adds no overhead to existing hot paths, and has thorough test coverage across both unit and integration tests. The O(n*m) scan in user_signal_count is documented and acceptable for pilot scale.
