jordan/tidaldb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
da736b8eb2
tidaldb/.sdlc/features/m9-purge-rematerialization/qa-plan.md
jordan 6f26d03c77 feat(m9): implement purge re-materialization engine 
Adds the M9 purge re-materialization feature: a WAL-replay background
engine that rebuilds community cohort aggregates for a (user, community)
pair after retroactive signal purge, restoring ranking correctness without
modifying the immutable WAL.

Key additions:
- cohort::rematerialization module: PurgeJobQueue, RematerializationEngine,
  WAL replay, atomic CohortSignalLedger swap, BLAKE3 audit log, metrics counters
- TidalDb::{submit_purge_job, purge_job_status, rematerialization_metrics}
  public API (db/rematerialization.rs)
- Engine auto-starts in persistent mode; clean shutdown before WAL teardown
- 6 integration tests in tests/m9_purge_remat.rs covering ephemeral and
  persistent modes, job lifecycle, and multi-job independence
- Split oversized files to comply with 600-line limit: db/mod.rs →
  db/from_parts.rs, entities/revocation.rs → revocation/{mod,tests}.rs,
  schema/validation/builders.rs → builders/{mod,tests}.rs,
  signals/warm.rs → warm/{mod,tests,proptests}.rs
- Fix pre-existing bootstrap errors: export AuditKind from session module,
  add overrides_rejected to SessionSnapshot deserialization
2026-03-03 19:18:16 -07:00

2.6 KiB
Raw Blame History

QA Plan: Re-materialization after Purge

Unit Tests (lib)

job.rs

  • submit_and_claim: submit job → claim → status Running
  • fifo_order: three jobs claimed in submission order
  • status_transitions: Running → Succeeded
  • permanently_failed_after_max_attempts: 5 failures → PermanentlyFailed
  • requeue_interrupted: Running → Pending on requeue_interrupted
  • list_jobs_preserves_order: order matches submission
  • serde_round_trip: JSON roundtrip of PurgeJob

metrics.rs (covered by integration)

replay.rs

  • empty_contributions_returns_zero_excluded: no filter → no exclusions
  • exclusion_set_excludes_matching_events: exact 4-tuple match
  • events_not_in_set_are_replayed: non-matching events pass through

swap.rs

  • swap_replaces_entry_values: scratch count wins over live count
  • absent_from_scratch_removes_from_live: zero-event entries removed
  • unrelated_community_untouched: other cohorts not affected

audit.rs

  • write_and_read_single_entry: roundtrip success entry
  • read_audit_log_multiple_entries: multiple entries preserved in order
  • checksum_mismatch_detected: corrupt entry detected
  • write_failure_entry: failure entry fields correct
  • verification_checksum_is_deterministic: same ledger → same hash

mod.rs (engine)

  • engine_starts_and_stops: worker starts and stops cleanly
  • job_with_empty_wal_succeeds: empty WAL → Succeeded
  • shutdown_within_one_second: shutdown latency < 1s

purge.rs (fixed pre-existing)

  • purge_returns_manifest_with_drained_entries
  • second_purge_produces_empty_entries
  • purge_retracts_from_ledger (fixed ancient timestamp bug)
  • manifest_json_roundtrip
  • manifest_from_malformed_json_returns_none

Integration Tests (m9_purge_remat.rs)

  1. submit_purge_job_returns_job_id — API surface works, JobId is non-zero
  2. purge_job_status_pending_after_submit — Pending immediately after submit (ephemeral, no engine)
  3. unknown_job_id_returns_none — unknown job → None
  4. rematerialization_metrics_snapshot_is_valid — fresh db has zero counters
  5. job_succeeds_in_persistent_mode — engine starts, processes job, metrics updated (< 10s)
  6. multiple_jobs_processed_independently — two jobs for different users both succeed (< 15s)

Regression Tests (m9_retroactive_purge.rs)

Existing 8 tests cover request_community_purge correctness; all must still pass.

Acceptance Criteria

  • All 1299 lib tests pass
  • All 6 m9_purge_remat integration tests pass
  • All 8 m9_retroactive_purge integration tests pass
  • cargo clippy -- -D warnings clean
  • cargo fmt --check clean