Spec: Retroactive Signal Purge

Summary

When a user explicitly requests a purge of their contributed signals from community aggregates (cohort ledgers), tidalDB must atomically mark those contributions as purged and schedule deterministic re-materialization. This feature covers the purge request API, the contribution-tracking ledger entries needed to make the purge deterministic, and the coordination handoff to the re-materialization engine (m9-purge-rematerialization).

Problem

The CohortSignalLedger accumulates per-cohort signal state from all users who contributed signals while members of a cohort. When a user exercises their right to remove their data from community aggregates — either via a Leave + Purge flow or a standalone explicit purge — there is currently no mechanism to:

Record which (cohort, entity, signal_type, timestamp, weight) tuples the user contributed.
Remove those contributions from the live in-memory aggregate state.
Trigger the background re-materialization pipeline to recompute aggregate state from the remaining WAL history.

Without contribution attribution, removing a user's data from community signals requires replaying the entire community WAL minus the user's events — which is the job of the re-materialization engine (m9-purge-rematerialization). This feature provides the purge request surface, the contribution-attribution record needed to make the purge bounded and auditable, and the handoff to re-materialization.

Functional Requirements

FR-1: Contribution Logging

Every call to try_cohort_attribution that successfully records a signal into a cohort ledger entry MUST also append a ContributionRecord { user_id, cohort: String, entity_id, signal_type_id, weight: f32, timestamp_ns: u64 } to the per-db CohortContributionLog.

The log is bounded: it retains contributions for at most the max_contribution_log_entries configured limit (default: 5 million entries). When the cap is exceeded, oldest entries are evicted (FIFO ring-buffer semantics). An evicted entry is unretractable; purge proceeds with what is in the log and defers full correctness to re-materialization.

FR-2: Ledger Retraction

CohortSignalLedger gains a retract(cohort, entity_id, type_id, weight, timestamp_ns) method that subtracts the contribution from the live aggregate via CAS loops on HotSignalState and BucketedCounter. Scores floor at 0.0 and never go negative.

FR-3: Purge Manifest

After applying in-memory retractions, PurgeCoordinator serializes all retracted entries into a PurgeManifest and writes it to durable storage at key [user_id: 8B BE][0x00][Tag::PurgeManifest=0x10][purge_id: 16B BE] in the items keyspace.

FR-4: Public API

pub fn request_community_purge(
    &self,
    user_id: u64,
    cohort_name: &str,
) -> crate::Result<(PurgeId, PurgeManifest)>;

pub fn list_purge_manifests(&self, user_id: u64) -> crate::Result<Vec<PurgeManifest>>;

FR-5: Idempotency

If request_community_purge is called twice for the same (user_id, cohort_name), the second call returns a new PurgeId and an empty-entries manifest — no additional retractions are applied (the contribution log entries are drained after first retraction).

FR-6: Thread Safety

CohortContributionLog uses Mutex<VecDeque<ContributionRecord>> — concurrent attribution writes and infrequent purge drains are serialized without contention on the happy path.

Acceptance Criteria

After request_community_purge(user, "cohort"), CohortSignalLedger::read_decay_score reflects the subtracted contributions.
A PurgeManifest is durably written to storage (in persistent mode) and listable via list_purge_manifests.
Calling request_community_purge twice for the same user+cohort causes no double-retraction.
Scores never go negative after purge.
Purging user A does not affect user B's contributions.
Purging a user who is not a member of a cohort returns an empty manifest without error.

3.9 KiB Raw Blame History