6f26d03c77
Adds the M9 purge re-materialization feature: a WAL-replay background
engine that rebuilds community cohort aggregates for a (user, community)
pair after retroactive signal purge, restoring ranking correctness without
modifying the immutable WAL.
Key additions:
- cohort::rematerialization module: PurgeJobQueue, RematerializationEngine,
WAL replay, atomic CohortSignalLedger swap, BLAKE3 audit log, metrics counters
- TidalDb::{submit_purge_job, purge_job_status, rematerialization_metrics}
public API (db/rematerialization.rs)
- Engine auto-starts in persistent mode; clean shutdown before WAL teardown
- 6 integration tests in tests/m9_purge_remat.rs covering ephemeral and
persistent modes, job lifecycle, and multi-job independence
- Split oversized files to comply with 600-line limit: db/mod.rs →
db/from_parts.rs, entities/revocation.rs → revocation/{mod,tests}.rs,
schema/validation/builders.rs → builders/{mod,tests}.rs,
signals/warm.rs → warm/{mod,tests,proptests}.rs
- Fix pre-existing bootstrap errors: export AuditKind from session module,
add overrides_rejected to SessionSnapshot deserialization
63 lines
2.6 KiB
Markdown
63 lines
2.6 KiB
Markdown
# QA Plan: Re-materialization after Purge
|
|
|
|
## Unit Tests (lib)
|
|
|
|
### job.rs
|
|
- `submit_and_claim`: submit job → claim → status Running
|
|
- `fifo_order`: three jobs claimed in submission order
|
|
- `status_transitions`: Running → Succeeded
|
|
- `permanently_failed_after_max_attempts`: 5 failures → PermanentlyFailed
|
|
- `requeue_interrupted`: Running → Pending on requeue_interrupted
|
|
- `list_jobs_preserves_order`: order matches submission
|
|
- `serde_round_trip`: JSON roundtrip of PurgeJob
|
|
|
|
### metrics.rs (covered by integration)
|
|
|
|
### replay.rs
|
|
- `empty_contributions_returns_zero_excluded`: no filter → no exclusions
|
|
- `exclusion_set_excludes_matching_events`: exact 4-tuple match
|
|
- `events_not_in_set_are_replayed`: non-matching events pass through
|
|
|
|
### swap.rs
|
|
- `swap_replaces_entry_values`: scratch count wins over live count
|
|
- `absent_from_scratch_removes_from_live`: zero-event entries removed
|
|
- `unrelated_community_untouched`: other cohorts not affected
|
|
|
|
### audit.rs
|
|
- `write_and_read_single_entry`: roundtrip success entry
|
|
- `read_audit_log_multiple_entries`: multiple entries preserved in order
|
|
- `checksum_mismatch_detected`: corrupt entry detected
|
|
- `write_failure_entry`: failure entry fields correct
|
|
- `verification_checksum_is_deterministic`: same ledger → same hash
|
|
|
|
### mod.rs (engine)
|
|
- `engine_starts_and_stops`: worker starts and stops cleanly
|
|
- `job_with_empty_wal_succeeds`: empty WAL → Succeeded
|
|
- `shutdown_within_one_second`: shutdown latency < 1s
|
|
|
|
### purge.rs (fixed pre-existing)
|
|
- `purge_returns_manifest_with_drained_entries`
|
|
- `second_purge_produces_empty_entries`
|
|
- `purge_retracts_from_ledger` (fixed ancient timestamp bug)
|
|
- `manifest_json_roundtrip`
|
|
- `manifest_from_malformed_json_returns_none`
|
|
|
|
## Integration Tests (m9_purge_remat.rs)
|
|
|
|
1. `submit_purge_job_returns_job_id` — API surface works, JobId is non-zero
|
|
2. `purge_job_status_pending_after_submit` — Pending immediately after submit (ephemeral, no engine)
|
|
3. `unknown_job_id_returns_none` — unknown job → None
|
|
4. `rematerialization_metrics_snapshot_is_valid` — fresh db has zero counters
|
|
5. `job_succeeds_in_persistent_mode` — engine starts, processes job, metrics updated (< 10s)
|
|
6. `multiple_jobs_processed_independently` — two jobs for different users both succeed (< 15s)
|
|
|
|
## Regression Tests (m9_retroactive_purge.rs)
|
|
Existing 8 tests cover `request_community_purge` correctness; all must still pass.
|
|
|
|
## Acceptance Criteria
|
|
- [ ] All 1299 lib tests pass
|
|
- [ ] All 6 m9_purge_remat integration tests pass
|
|
- [ ] All 8 m9_retroactive_purge integration tests pass
|
|
- [ ] `cargo clippy -- -D warnings` clean
|
|
- [ ] `cargo fmt --check` clean
|