61 lines
3.9 KiB
Markdown
61 lines
3.9 KiB
Markdown
# Spec: Retroactive Signal Purge
|
|
|
|
## Summary
|
|
|
|
When a user explicitly requests a purge of their contributed signals from community aggregates (cohort ledgers), tidalDB must atomically mark those contributions as purged and schedule deterministic re-materialization. This feature covers the purge request API, the contribution-tracking ledger entries needed to make the purge deterministic, and the coordination handoff to the re-materialization engine (m9-purge-rematerialization).
|
|
|
|
## Problem
|
|
|
|
The `CohortSignalLedger` accumulates per-cohort signal state from all users who contributed signals while members of a cohort. When a user exercises their right to remove their data from community aggregates — either via a Leave + Purge flow or a standalone explicit purge — there is currently no mechanism to:
|
|
|
|
1. Record which (cohort, entity, signal_type, timestamp, weight) tuples the user contributed.
|
|
2. Remove those contributions from the live in-memory aggregate state.
|
|
3. Trigger the background re-materialization pipeline to recompute aggregate state from the remaining WAL history.
|
|
|
|
Without contribution attribution, removing a user's data from community signals requires replaying the entire community WAL minus the user's events — which is the job of the re-materialization engine (m9-purge-rematerialization). This feature provides the purge request surface, the contribution-attribution record needed to make the purge bounded and auditable, and the handoff to re-materialization.
|
|
|
|
## Functional Requirements
|
|
|
|
### FR-1: Contribution Logging
|
|
|
|
Every call to `try_cohort_attribution` that successfully records a signal into a cohort ledger entry MUST also append a `ContributionRecord { user_id, cohort: String, entity_id, signal_type_id, weight: f32, timestamp_ns: u64 }` to the per-db `CohortContributionLog`.
|
|
|
|
The log is bounded: it retains contributions for at most the `max_contribution_log_entries` configured limit (default: 5 million entries). When the cap is exceeded, oldest entries are evicted (FIFO ring-buffer semantics). An evicted entry is unretractable; purge proceeds with what is in the log and defers full correctness to re-materialization.
|
|
|
|
### FR-2: Ledger Retraction
|
|
|
|
`CohortSignalLedger` gains a `retract(cohort, entity_id, type_id, weight, timestamp_ns)` method that subtracts the contribution from the live aggregate via CAS loops on `HotSignalState` and `BucketedCounter`. Scores floor at 0.0 and never go negative.
|
|
|
|
### FR-3: Purge Manifest
|
|
|
|
After applying in-memory retractions, `PurgeCoordinator` serializes all retracted entries into a `PurgeManifest` and writes it to durable storage at key `[user_id: 8B BE][0x00][Tag::PurgeManifest=0x10][purge_id: 16B BE]` in the items keyspace.
|
|
|
|
### FR-4: Public API
|
|
|
|
```rust
|
|
pub fn request_community_purge(
|
|
&self,
|
|
user_id: u64,
|
|
cohort_name: &str,
|
|
) -> crate::Result<(PurgeId, PurgeManifest)>;
|
|
|
|
pub fn list_purge_manifests(&self, user_id: u64) -> crate::Result<Vec<PurgeManifest>>;
|
|
```
|
|
|
|
### FR-5: Idempotency
|
|
|
|
If `request_community_purge` is called twice for the same (user_id, cohort_name), the second call returns a new `PurgeId` and an empty-entries manifest — no additional retractions are applied (the contribution log entries are drained after first retraction).
|
|
|
|
### FR-6: Thread Safety
|
|
|
|
`CohortContributionLog` uses `Mutex<VecDeque<ContributionRecord>>` — concurrent attribution writes and infrequent purge drains are serialized without contention on the happy path.
|
|
|
|
## Acceptance Criteria
|
|
|
|
1. After `request_community_purge(user, "cohort")`, `CohortSignalLedger::read_decay_score` reflects the subtracted contributions.
|
|
2. A `PurgeManifest` is durably written to storage (in persistent mode) and listable via `list_purge_manifests`.
|
|
3. Calling `request_community_purge` twice for the same user+cohort causes no double-retraction.
|
|
4. Scores never go negative after purge.
|
|
5. Purging user A does not affect user B's contributions.
|
|
6. Purging a user who is not a member of a cohort returns an empty manifest without error.
|