# m9p2: Membership Lifecycle and Stop-Forward ## Delivers The join / leave / rejoin lifecycle for community personalization overlays, built on the scope and provenance primitives delivered in m9p1. After this phase, a user can join a community with a `SharePolicy`, leave it with a `stop_forward` guarantee that blocks new community contributions in under a second, and rejoin into a fresh membership *epoch* so that replay never crosses epoch boundaries ambiguously. The `stop_forward` gate is an `O(1)` atomic read on the synchronous signal write path -- it never depends on the 60s session sweeper. Membership is durable (its own `Tag::Membership` keyspace), survives restart, and the active epoch is surfaced in query result metadata for debugging and explainability. This is the lifecycle phase: m9p1 made scope-aware events shippable; m9p2 decides *whose* community-scoped events are eligible to ship right now and under which epoch they are stamped. No retroactive purge yet (that is m9p3) -- this phase only governs forward contributions. Deliverables: - `MembershipState` enum (`Joined`, `LeavingStopForward`, `Left`, `Rejoined`) and `Membership` struct with an atomic `stop_forward_at_ns` gate - `MembershipRegistry`: in-memory `DashMap<(UserId, CommunityId), Membership>` source of truth, backed by a durable `Tag::Membership` checkpoint - Public lifecycle API on `TidalDb` in a new `db/communities.rs`: `join_community`, `leave_community`, `rejoin_community`, `membership_epoch` - A synchronous stop-forward gate in `signal_with_context` that drops community-scoped contributions for a member in `LeavingStopForward`/`Left` state without touching the local profile - `membership_epoch` per community advanced on every rejoin, stamped into the m9p1 `SignalProvenance.membership_epoch` field and the WAL event - `membership_epoch` surfaced in `Results` via `policy_metadata`, plus the active epoch added to `QueryStats` - All existing M0-M8 tests pass unchanged; single-node default has no memberships and a zero-cost gate ## Dependencies - **Requires:** m9p1 complete -- `SignalScope`, `CommunityId`, `SignalProvenance`, `SharePolicy`/`ShareMode`, the WAL V3 `EventRecord` carrying `scope` + `membership_epoch`, and the `governance` module root wired into `lib.rs`. Also requires M8 (`ReplicationState`, shard identity) and M4 (`AgentId`, session policy) which m9p1 already extends. - **Files modified:** - `tidal/src/db/signals.rs` -- stop-forward gate + epoch stamping inside `signal_with_context` community dispatch (`signals.rs:219`); reject path for community contributions when membership is in a stop-forward state - `tidal/src/db/mod.rs` -- add `membership_registry` field to `TidalDb`, declare `mod communities;`, restore the registry in `open()` - `tidal/src/replication/state.rs` -- add per-community `membership_epoch` tracking alongside the existing per-shard high-water-mark - `tidal/src/query/retrieve/types.rs` -- add `policy_metadata: PolicyMetadata` to `Results` (`types.rs:434`) carrying the active `membership_epoch` - `tidal/src/query/stats.rs` -- add `membership_epoch: Option` to `QueryStats` - `tidal/src/governance/mod.rs` -- re-export membership types - `tidal/src/storage/keys.rs` -- add `Tag::Membership = 0x0E` - **Files created:** - `tidal/src/governance/membership.rs` -- `MembershipState`, `Membership`, `MembershipRegistry`, `LeaveMode`, `UserId` newtype - `tidal/src/db/communities.rs` -- `join_community`, `leave_community`, `rejoin_community`, `membership_epoch` on `TidalDb` ## Research References - `/tmp/m9m10_brief.md` -- section 1.3 (Membership Epoch primitive), section 2 (M9 Phase 2 acceptance->work table), section 4 invariants 4 (<1s gates are synchronous atomic reads, never the 60s sweeper) and 5 (local-profile-intact) - `docs/research/tidaldb_wal.md` -- WAL event envelope, epoch stamping on the event payload - `thoughts.md` -- Part II.1 (WAL convergence / replay determinism), Part V.12 (subject-prefix key encoding for the membership keyspace) - `docs/planning/milestone-8/phase-1/` -- ReplicationState extension pattern, identity-newtype conventions ## Acceptance Criteria (Phase Level) - [ ] `MembershipState` enum has exactly `Joined`, `LeavingStopForward`, `Left`, `Rejoined`; derives `Copy + Clone + Debug + PartialEq + Eq + Serialize + Deserialize`; round-trips through serde; has a `u8` wire discriminant - [ ] `Membership` holds `user_id`, `community_id`, `state`, `epoch: u32`, `share_policy_version: u16`, `joined_at_ns: u64`, and an atomic `stop_forward_at_ns: AtomicU64` (0 = not stop-forwarded) - [ ] `TidalDb::join_community(user, community, SharePolicy)` creates a `Joined` membership at `epoch = 0` (or `Rejoined` at the next epoch if a prior `Left` membership exists), persists it to `Tag::Membership`, and is idempotent if already joined at the current epoch - [ ] `TidalDb::leave_community(user, community, LeaveMode::StopForward)` transitions to `LeavingStopForward`, sets `stop_forward_at_ns` to the call wall-clock in a single atomic store, then settles to `Left`; the membership row persists - [ ] After `leave_community(StopForward)`, a community-scoped `signal_with_context` for that `(user, community)` records **zero** new community contributions; the gate is verified to be an `O(1)` atomic read (no sweeper, no scan) -- measured p99 latency for the gated path < 1s, in practice sub-microsecond - [ ] The same post-leave signal still records the user's **local** profile update unchanged (local-profile-intact: `SignalScope::Local` side effects fire regardless of membership state) - [ ] `TidalDb::rejoin_community(...)` increments the per-community epoch by exactly 1, sets state `Rejoined`, clears `stop_forward_at_ns` to 0, and the new epoch is stamped into `SignalProvenance.membership_epoch` and the WAL event for subsequent community contributions - [ ] A pre-rejoin (old-epoch) WAL event replayed after a leave->rejoin does **not** revive community contributions under the new epoch: `apply_wal_event` / CRDT merge segregates events by `membership_epoch` - [ ] `TidalDb::membership_epoch(user, community) -> Option` returns the active epoch; `Results.policy_metadata.membership_epoch` and `QueryStats.membership_epoch` expose it on community-scoped queries - [ ] Membership state survives close/reopen: registry rebuilt from `Tag::Membership` checkpoint on `open()`; epochs and stop-forward state preserved - [ ] All existing M0-M8 tests pass unchanged; a DB with no memberships pays a single `DashMap::is_empty()` check on the community dispatch path - [ ] `cargo fmt` clean, `cargo clippy -p tidaldb -D warnings` clean, all `cargo test -p tidaldb --lib` pass ## Task Execution Order ``` Task 01: Membership Types ───────────┬──> Task 03: Lifecycle API (db/communities.rs) (membership.rs, UserId, LeaveMode) │ │ │ v Task 02: Membership Registry ────────┘ Task 04: Stop-Forward Gate (DashMap + Tag::Membership persist) (signal_with_context) │ v Task 05: Epoch Stamping + Cross-Epoch Replay (provenance.membership_epoch, apply_wal_event) │ v Task 06: Epoch in Query Results (policy_metadata, QueryStats, ReplicationState) ``` Task 01 and Task 02 are parallelizable (types vs. persistence wiring). Task 03 depends on both. Task 04 depends on 03 (needs the registry lookup in the write path). Task 05 depends on 04 (epoch must be readable to stamp). Task 06 depends on 05 (epoch must be authoritative before it is surfaced). ## Module Location | File | Status | Contains | |------|--------|----------| | `tidal/src/governance/membership.rs` | NEW | `MembershipState`, `Membership`, `MembershipRegistry`, `LeaveMode`, `UserId` | | `tidal/src/db/communities.rs` | NEW | `join_community`, `leave_community`, `rejoin_community`, `membership_epoch` on `TidalDb` | | `tidal/src/governance/mod.rs` | MODIFIED | Re-export membership types (`pub mod membership;`) | | `tidal/src/db/mod.rs` | MODIFIED | `membership_registry` field; `mod communities;`; restore in `open()` | | `tidal/src/db/signals.rs` | MODIFIED | Stop-forward gate + epoch stamping in `signal_with_context` community dispatch | | `tidal/src/replication/state.rs` | MODIFIED | Per-community `membership_epoch` tracking | | `tidal/src/query/retrieve/types.rs` | MODIFIED | `PolicyMetadata` + `Results.policy_metadata` | | `tidal/src/query/stats.rs` | MODIFIED | `QueryStats.membership_epoch` | | `tidal/src/storage/keys.rs` | MODIFIED | `Tag::Membership = 0x0E` | ## Technical Design ### Membership state and the atomic stop-forward gate `Membership` is the per-`(user, community)` lifecycle row. The single load-bearing field is `stop_forward_at_ns`: an `AtomicU64` that the write path reads with `Ordering::Acquire`. Zero means "contributions flow"; non-zero means "stop-forwarded at this wall-clock ns". Because the gate is one atomic load, the <1s p99 requirement is met by construction -- there is no sweeper, no scan, no lock on the read side. ```rust // tidal/src/governance/membership.rs use std::sync::atomic::{AtomicU64, Ordering}; use serde::{Deserialize, Serialize}; use crate::governance::scope::CommunityId; // from m9p1 /// Identifies a user entity participating in a community overlay. /// /// A thin newtype over the entity id so membership keys never collide with /// raw `EntityId` usage elsewhere. `UserId(0)` is reserved (no user). #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize, Deserialize)] pub struct UserId(pub u64); /// The lifecycle state of a `(user, community)` membership. /// /// Transitions: `Joined -> LeavingStopForward -> Left -> Rejoined -> ...`. /// Each `Rejoined` opens a new epoch; replay never crosses epochs. #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] #[repr(u8)] pub enum MembershipState { /// Active member; community-scoped contributions flow. Joined = 0, /// Leave initiated; `stop_forward_at_ns` is set and new community /// contributions are gated off, but no purge has occurred. LeavingStopForward = 1, /// Settled "left" state; contributions stay gated until a rejoin. Left = 2, /// Re-entered the community under a new epoch. Rejoined = 3, } /// How a leave is performed. /// /// `StopForward` is the m9p2 guarantee: block new contributions immediately, /// leave existing aggregates untouched. `Purge` (a superset, m9p3) additionally /// removes prior contributions and is declared here so the API surface is /// stable, but is rejected with `TidalError::Unsupported` until m9p3 lands. #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] pub enum LeaveMode { /// Block new contributions; keep prior aggregates (m9p2). StopForward, /// Stop-forward **and** retroactively purge prior contributions (m9p3). Purge, } /// A durable per-`(user, community)` membership row. /// /// The source of truth lives in `MembershipRegistry`; this struct is what /// gets checkpointed under `Tag::Membership`. `stop_forward_at_ns` is the /// only mutable hot-path field and is an atomic so the signal write path can /// gate without taking a lock. #[derive(Debug, Serialize, Deserialize)] pub struct Membership { /// The participating user. pub user_id: UserId, /// The community overlay being joined. pub community_id: CommunityId, /// Current lifecycle state. pub state: MembershipState, /// Monotonic epoch; increments by 1 on each rejoin. First join = 0. pub epoch: u32, /// The `SharePolicy` version in force for this membership (m9p1). pub share_policy_version: u16, /// Wall-clock ns at first join (never changes across epochs). pub joined_at_ns: u64, /// Wall-clock ns at which contributions were stop-forwarded. /// `0` means "not stop-forwarded -- contributions flow". Read on the /// synchronous signal write path with `Ordering::Acquire`. #[serde(with = "atomic_u64_serde")] pub stop_forward_at_ns: AtomicU64, } impl Membership { /// Whether community-scoped contributions are currently gated off. /// /// `O(1)` atomic load -- this is the <1s stop-forward gate. Never routes /// through the sweeper. #[must_use] pub fn is_stop_forwarded(&self) -> bool { self.stop_forward_at_ns.load(Ordering::Acquire) != 0 } /// Engage the stop-forward gate at `now_ns`. Idempotent: the first /// non-zero store wins; later calls do not move the timestamp backward. pub fn engage_stop_forward(&self, now_ns: u64) { let _ = self.stop_forward_at_ns.compare_exchange( 0, now_ns, Ordering::AcqRel, Ordering::Acquire, ); } /// Clear the gate (used on rejoin so the new epoch flows). pub fn clear_stop_forward(&self) { self.stop_forward_at_ns.store(0, Ordering::Release); } } ``` ### Membership registry: in-memory source of truth + durable checkpoint The registry mirrors the `CohortRegistry`/`ReplicationState` conventions: a `DashMap` for lock-free reads on the write path, a JSON checkpoint under `Tag::Membership` at a sentinel entity id for durability, and an `is_empty()` fast path so a DB with no communities pays nothing. ```rust // tidal/src/governance/membership.rs (continued) use dashmap::DashMap; /// In-memory registry of all `(user, community)` memberships. /// /// Source of truth for the lifecycle gate. Reads on the signal write path go /// through `DashMap::get` (sharded, lock-free for distinct keys). Persisted to /// `Tag::Membership` on checkpoint and rebuilt on `open()`. #[derive(Debug, Default)] pub struct MembershipRegistry { rows: DashMap<(UserId, CommunityId), Membership>, } impl MembershipRegistry { #[must_use] pub fn new() -> Self { Self { rows: DashMap::new() } } /// Fast path used on the signal write path before any community work. #[must_use] pub fn is_empty(&self) -> bool { self.rows.is_empty() } /// The active epoch for `(user, community)`, if a membership exists. #[must_use] pub fn epoch(&self, user: UserId, community: CommunityId) -> Option { self.rows.get(&(user, community)).map(|m| m.epoch) } /// Whether a community-scoped contribution for `(user, community)` is /// currently gated off (stop-forwarded, left, or no membership). /// /// Returns `true` (gated) when no membership exists -- a non-member cannot /// contribute to a community. `O(1)`. #[must_use] pub fn is_contribution_gated(&self, user: UserId, community: CommunityId) -> bool { match self.rows.get(&(user, community)) { Some(m) => m.is_stop_forwarded() || matches!(m.state, MembershipState::Left | MembershipState::LeavingStopForward), None => true, } } /// Serialize all rows for checkpoint persistence (`Tag::Membership`). #[must_use] pub fn to_checkpoint_bytes(&self) -> Vec { /* serde_json over a Vec */ } /// Rebuild from checkpoint bytes; malformed bytes yield an empty registry. #[must_use] pub fn from_checkpoint_bytes(bytes: &[u8]) -> Self { /* ... */ } } ``` ### Lifecycle API on `TidalDb` The lifecycle methods live in the new `db/communities.rs`, mirroring how `db/cohorts.rs` and `db/sessions.rs` extend `impl TidalDb`. Each method mutates the registry, persists the row under `Tag::Membership`, and advances the per-community epoch counter in `ReplicationState`. ```rust // tidal/src/db/communities.rs use crate::{ governance::membership::{LeaveMode, Membership, MembershipState, UserId}, governance::scope::CommunityId, schema::{TidalError, Timestamp}, }; impl super::TidalDb { /// Join a community overlay under an explicit `SharePolicy`. /// /// Idempotent if already `Joined`/`Rejoined` at the current epoch. If a /// prior membership is `Left`, this is equivalent to `rejoin_community` /// (next epoch). The new row is persisted to `Tag::Membership`. /// /// # Errors /// - `TidalError::Internal` if no storage backend is wired. /// - `TidalError::Storage` on persistence failure. pub fn join_community( &self, user: UserId, community: CommunityId, policy: crate::governance::share_policy::SharePolicy, // from m9p1 ) -> crate::Result /* returns active epoch */ { /* ... */ } /// Leave a community. `LeaveMode::StopForward` engages the atomic gate /// (<1s, synchronous) and settles the row to `Left`. `LeaveMode::Purge` /// returns `TidalError::Unsupported` until m9p3. /// /// # Errors /// - `TidalError::NotFound` if no membership exists. /// - `TidalError::Unsupported` for `LeaveMode::Purge` (m9p3). pub fn leave_community( &self, user: UserId, community: CommunityId, mode: LeaveMode, ) -> crate::Result<()> { /* ... */ } /// Rejoin a community after leaving: opens a new epoch (`epoch + 1`), /// clears the stop-forward gate, and sets state `Rejoined`. /// /// # Errors /// - `TidalError::NotFound` if no prior membership exists. pub fn rejoin_community( &self, user: UserId, community: CommunityId, policy: crate::governance::share_policy::SharePolicy, ) -> crate::Result /* returns the new epoch */ { /* ... */ } /// The active membership epoch for `(user, community)`, or `None` if the /// user is not a member. Surfaced into query result metadata. #[must_use] pub fn membership_epoch(&self, user: UserId, community: CommunityId) -> Option { self.membership_registry.epoch(user, community) } } ``` ### Stop-forward gate on the synchronous write path The gate slots into `signal_with_context` (`db/signals.rs:219`) inside the community-scoped dispatch branch added in m9p1. It runs *before* the contribution is attributed to the community, and it leaves the local-profile side effects (steps 1-7 in the existing dispatch) entirely alone -- only the community attribution is gated. This is the local-profile-intact invariant in code. ```rust // tidal/src/db/signals.rs — inside the community-scope branch of signal_with_context // `scope` and `community` come from the m9p1 SignalScope dispatch. if let crate::governance::scope::SignalScope::Community(community) = scope { let uid = crate::governance::membership::UserId(user_id); // <1s STOP-FORWARD GATE: single atomic Acquire load. Never the sweeper. if self.membership_registry.is_contribution_gated(uid, community) { // Local profile already updated above; community contribution dropped. return Ok(()); } // Stamp the active epoch into provenance + the WAL community event. let epoch = self .membership_registry .epoch(uid, community) .unwrap_or(0); let provenance = crate::governance::provenance::SignalProvenance { membership_epoch: epoch, scope, ..provenance // writer_agent_id, origin_shard, share_policy_version, hlc from m9p1 }; self.attribute_community_contribution(community, entity_id, weight, timestamp, provenance); } ``` ### Per-community epoch tracking in `ReplicationState` `ReplicationState` already tracks per-shard high-water-marks with atomics; the membership epoch is added as a parallel monotonic-per-community counter so the epoch authority is the same place CRDT/replay machinery already consults. ```rust // tidal/src/replication/state.rs — additive use crate::governance::scope::CommunityId; impl ReplicationState { /// The current membership epoch for a community (0 if never joined). /// Monotonic; advanced only by `advance_membership_epoch`. #[must_use] pub fn membership_epoch(&self, community: CommunityId) -> u32 { /* atomic load */ } /// Advance a community's membership epoch on rejoin. CAS loop enforces /// monotonicity exactly like `advance` does for seqnos. pub fn advance_membership_epoch(&self, community: CommunityId, epoch: u32) { /* ... */ } } ``` ### Surfacing the active epoch in query results `Results` gains a `policy_metadata` field (a small struct so m9p3/m10 can extend it with `purge_watermark` / `policy_version` without further widening `Results`). `QueryStats` carries the epoch for the operational-visibility path. ```rust // tidal/src/query/retrieve/types.rs — additive /// Governance metadata attached to a query response. /// /// Carries the active membership epoch for community-scoped queries. /// Extended by m9p3 (`purge_watermark`) and m10 (`policy_version`). #[derive(Debug, Clone, Default)] pub struct PolicyMetadata { /// Active membership epoch when the query targets a community overlay. /// `None` for purely local queries. pub membership_epoch: Option, } // added to `pub struct Results`: // /// Governance metadata (membership epoch, later purge/policy versions). // pub policy_metadata: PolicyMetadata, ``` ```rust // tidal/src/query/stats.rs — additive field on QueryStats // /// Active membership epoch for community-scoped queries (None otherwise). // pub membership_epoch: Option, ``` ### New storage tag ```rust // tidal/src/storage/keys.rs — next free discriminant after SchemaFingerprint (0x0D) // /// Community membership lifecycle rows (M9p2). // Membership = 0x0E, // plus the matching `0x0E => Some(Self::Membership)` arm in `Tag::from_byte`. ``` ## Notes ### The stop-forward gate is an atomic read, never the sweeper Invariant 4 from the brief: both the <1s revocation gate (m10p2) and this <1s stop-forward gate are `O(1)` atomic loads on the synchronous write path. The 60s `db/sweeper.rs` is explicitly *not* in this path. Routing stop-forward through the sweeper would blow the p99 SLA by ~60x and is a hard "no". The gate returns early from `signal_with_context` after the local-profile update, before community attribution -- so the cost when a member is active is a single `DashMap::get` + atomic load, and the cost when there are no communities at all is one `is_empty()` check. ### Local profile is sacred (local-profile-intact) Invariant 5: a stop-forwarded or `Left` membership gates **only** the community contribution. Every `SignalScope::Local` side effect -- seen tracking, hard negatives, preference-vector blend, interaction weights -- still fires. A routing bug that lets the gate suppress local side effects is silent data loss for the user's own profile. Tests must assert the local feed/score is unchanged across join/leave/rejoin. ### Rejoin = new epoch; replay never crosses epochs ambiguously Invariant 3 from the brief: `epoch += 1` on every rejoin, stamped into both `SignalProvenance.membership_epoch` and the WAL community event. On replay, `apply_wal_event` and CRDT merge must segregate by epoch so a late, pre-rejoin (old-epoch) event cannot revive contributions under the new epoch. The epoch is a `u32` carried in the V3 `EventRecord` from m9p1 -- this phase only populates and reads it; it does not change the wire format. ### `LeaveMode::Purge` is declared but deferred `leave_community(.., LeaveMode::Purge)` is part of the stable API surface but returns `TidalError::Unsupported` until m9p3 wires the rematerialization engine. Declaring it now avoids a breaking signature change in m9p3. ### Backward compatibility No WAL or checkpoint format change in this phase -- m9p1 already bumped `FORMAT_VERSION` to 3 and added `membership_epoch` to the event. The only new durable artifact is the `Tag::Membership` keyspace, which is additive: a pre-m9p2 database simply has zero membership rows and the gate is a no-op. A DB opened by m9p2 code that never joins a community is byte-identical in behavior to M8. ### Idempotency `join_community` is idempotent at the current epoch; `engage_stop_forward` uses a `compare_exchange(0, now)` so concurrent leaves settle on the first timestamp. Membership mutations are persisted under a deterministic key so checkpoint replay is order-independent for distinct `(user, community)` pairs. ## Done When A user can `join_community` with a `SharePolicy`, contribute community-scoped signals that blend into the community feed, then `leave_community(StopForward)` and observe that (a) **zero** new community contributions are recorded while their **local** profile keeps updating, (b) the gate is a single atomic read with sub-microsecond p99 (never the sweeper), and (c) after `rejoin_community` the active epoch has incremented by one, old-epoch replayed events do not revive contributions, and the active epoch is visible in `Results.policy_metadata` and `QueryStats`. Membership survives close/reopen, and all existing M0-M8 tests pass unchanged.