6f26d03c77
Adds the M9 purge re-materialization feature: a WAL-replay background
engine that rebuilds community cohort aggregates for a (user, community)
pair after retroactive signal purge, restoring ranking correctness without
modifying the immutable WAL.
Key additions:
- cohort::rematerialization module: PurgeJobQueue, RematerializationEngine,
WAL replay, atomic CohortSignalLedger swap, BLAKE3 audit log, metrics counters
- TidalDb::{submit_purge_job, purge_job_status, rematerialization_metrics}
public API (db/rematerialization.rs)
- Engine auto-starts in persistent mode; clean shutdown before WAL teardown
- 6 integration tests in tests/m9_purge_remat.rs covering ephemeral and
persistent modes, job lifecycle, and multi-job independence
- Split oversized files to comply with 600-line limit: db/mod.rs →
db/from_parts.rs, entities/revocation.rs → revocation/{mod,tests}.rs,
schema/validation/builders.rs → builders/{mod,tests}.rs,
signals/warm.rs → warm/{mod,tests,proptests}.rs
- Fix pre-existing bootstrap errors: export AuditKind from session module,
add overrides_rejected to SessionSnapshot deserialization
70 lines
4.8 KiB
Markdown
70 lines
4.8 KiB
Markdown
# Review: Re-materialization after Purge
|
|
|
|
## Summary
|
|
|
|
Implementation is complete. All 9 tasks are delivered. 1299 lib tests pass, 6 new integration tests pass, 8 existing retroactive purge tests pass. Clippy clean, fmt clean.
|
|
|
|
## Files Changed
|
|
|
|
### New Files
|
|
- `tidal/src/cohort/rematerialization/mod.rs` — engine orchestrator, `RematerializationConfig`, `RematerializationHandle`, `start()`, `worker_loop()`
|
|
- `tidal/src/cohort/rematerialization/job.rs` — `PurgeJob`, `PurgeJobQueue`, `PurgeJobStatus`, `JobId`
|
|
- `tidal/src/cohort/rematerialization/replay.rs` — `filtered_replay()`, `ExclusionKey`, `ReplayResult`, `ReplayStats`
|
|
- `tidal/src/cohort/rematerialization/swap.rs` — `atomic_swap()`
|
|
- `tidal/src/cohort/rematerialization/audit.rs` — `AuditLogWriter`, `PurgeAuditEntry`, BLAKE3-framed binary format
|
|
- `tidal/src/cohort/rematerialization/metrics.rs` — `RematerializationMetrics`, `RematerializationMetricsSnapshot`
|
|
- `tidal/src/db/rematerialization.rs` — public API: `submit_purge_job`, `purge_job_status`, `rematerialization_metrics`
|
|
- `tidal/tests/m9_purge_remat.rs` — 6 integration tests
|
|
|
|
### Modified Files
|
|
- `tidal/src/cohort/ledger.rs` — added `remove_entry()`, `drain_community_into()`
|
|
- `tidal/src/cohort/mod.rs` — added `pub mod contribution`, `pub mod purge`, `pub mod rematerialization`
|
|
- `tidal/src/db/mod.rs` — added 3 fields, `mod rematerialization`, engine start in `from_parts`, `mod rematerialization`
|
|
- `tidal/src/db/lifecycle.rs` — engine shutdown before WAL in `shutdown_inner()`
|
|
- `tidal/src/cohort/purge.rs` — fixed `purge_retracts_from_ledger` test (ancient timestamp)
|
|
- `tidal/src/entities/revocation.rs` — replaced `uuid::Uuid` with `[u8; 16]` (not in Cargo.toml)
|
|
- `tidal/src/query/retrieve/types/tests.rs` — added `community: None` to 3 direct `Retrieve` struct initializers
|
|
- `tidal/src/ranking/executor/mod.rs` — fixed `if let Some` on `Result` (bootstrapped code bug)
|
|
- `tidal/src/entities/community.rs` — fixed temporary-dropped-while-borrowed (DashMap entry)
|
|
|
|
## Design Review
|
|
|
|
### Correctness
|
|
- **ExclusionKey 4-tuple** (`entity_id`, `signal_type_id`, `weight_bits`, `timestamp_nanos`) precisely identifies WAL events to skip. Exact match avoids false exclusions.
|
|
- **Atomic swap** uses `DashMap::remove + insert` to transfer `EntitySignalEntry` ownership without cloning (`HotSignalState` contains `AtomicU64`, not `Clone`).
|
|
- **Score floor**: `HotSignalState::retract()` uses saturating subtraction, so scores never go negative.
|
|
- **Idempotency**: Second `submit_purge_job` + completed job correctly processes an empty contribution log, producing a no-op swap. Score unchanged.
|
|
- **Crash safety**: `requeue_interrupted()` at startup re-queues any `Running` jobs found in the queue.
|
|
|
|
### Concurrency
|
|
- Background worker is a single thread — no per-job parallelism needed. Job queue mutex is only held briefly (claim + update).
|
|
- `DashMap` shard locking ensures ranking queries on disjoint entries proceed during the atomic swap.
|
|
- Worker polls every 500ms, responsive to shutdown within one sleep interval.
|
|
|
|
### Observability
|
|
- `RematerializationMetrics` AtomicU64 counters: `jobs_pending`, `jobs_succeeded`, `jobs_failed`, `last_job_duration_ms`, `last_job_entries_updated`.
|
|
- Audit log: BLAKE3-framed binary entries with verification checksum over live ledger state.
|
|
- Tracing spans: `info!` on success, `warn!` on failure, `error!` on audit log open failure.
|
|
|
|
### Error Handling
|
|
- WAL read errors propagate as `TidalError::Internal` and mark job `Failed` (retried up to 5 times).
|
|
- Storage write failures in `request_community_purge` return `TidalError::Internal`; in-memory retraction already applied (eventual consistency).
|
|
- Audit log write failure is non-fatal (warn log only) to prevent blocking job completion.
|
|
- Mutex poison produces `TidalError::Internal` in public API methods.
|
|
|
|
## Issues Found and Fixed
|
|
|
|
1. **purge_retracts_from_ledger test**: Pre-existing bootstrapped test used `ts = 1_000_000_000u64` (nanoseconds = ~1970 epoch). Signal fully decays over 55 years. Fixed to `Timestamp::now()`.
|
|
|
|
2. **Retrieve struct direct initializers**: Bootstrapped `CommunityContext` field added to `Retrieve` struct but 3 test sites constructed the struct directly without `community: None`. Fixed.
|
|
|
|
3. **ranking/executor/mod.rs**: Bootstrapped code used `if let Some(type_id) = self.ledger.resolve_signal_type(...)` but `resolve_signal_type` returns `Result`, not `Option`. Fixed to `if let Ok`.
|
|
|
|
4. **entities/revocation.rs**: Bootstrapped code imported `uuid::Uuid` which is not in `Cargo.toml`. Replaced with `[u8; 16]` backed by `Timestamp::now().as_nanos()` + `AtomicU64` counter.
|
|
|
|
5. **entities/community.rs**: DashMap entry temporary dropped while borrowed. Fixed with explicit `let mut entry` binding (clippy `significant_drop_tightening`).
|
|
|
|
## Verdict
|
|
|
|
APPROVED. Implementation is correct, complete, and consistent with the spec and design. All tests pass.
|