tidaldb/tidal/docs/planning/milestone-10/phase-2/OVERVIEW.md

m10p2: Agent Capability and Scope Controls

Delivers

Per-agent capability tokens that gate signal reads and writes by SignalScope (Local / Community / Session / Agent) with a hard TTL, plus synchronous, atomic revocation that takes effect in under one second. After this phase, an agent can only write community-scoped signals if it holds a live, unexpired, unrevoked CapabilityToken granting write on that scope; every denial emits a typed PolicyViolation and an audit-log entry; and granted/revoked capabilities survive restart because they are persisted to durable storage and replayed at startup. This is the enforcement layer that makes the M10 UAT's "A_experimental is denied community write" and "U revokes A_trusted community scope immediately" steps real.

Deliverables:

• CapabilityToken { token_id, agent_id, scopes: Vec<ScopePermission>, created_at_ns, expires_at_ns, revoked_at_ns: AtomicU64 } and ScopePermission { scope, read, write } in new tidal/src/governance/capability.rs
• CapabilityRegistry: in-memory DashMap<String, Arc<CapabilityToken>> (keyed by token_id) with O(1) live-status lookup, owned by TidalDb
• Public API on TidalDb: grant_capability(...) -> Result<CapabilityToken> and revoke_capability(token_id) -> Result<()>
• SessionState.capability_token: Option<Arc<CapabilityToken>> binding a session to its agent's token
• PolicyEvaluator gains a capability-check phase; new PolicyViolationKind::{InsufficientCapability, CapabilityRevoked, CapabilityExpired}
• Enforcement wired into session_signal (db/sessions.rs) and signal_for_tenant (db/replication_ops.rs); both deny out-of-scope writes and audit them
• Atomic revoked_at_ns gate read at the top of the write path -> revocation is O(1) and synchronous; NEVER routed through the 60s sweeper
• Durable persistence: grants written to Tag::Capability, revocations to Tag::CapabilityRevocation; replayed on startup via db/session_restore.rs
• tidalctl agents and tidalctl revocations offline read-only subcommands

Dependencies

• Requires: M9 complete (SignalScope, CommunityId, SignalProvenance in tidal/src/governance/; WAL V3 event envelope; membership + share-policy machinery). M10p1 complete (GovernancePolicy, GovernanceRegistry, governance policies loaded post-with_schema in db/open.rs). M8 session layer (SessionState, PolicyEvaluator, AuditLog, AgentPolicy, SessionWalEvent).
• Files modified:
  • tidal/src/governance/mod.rs -- re-export CapabilityToken, ScopePermission, CapabilityRegistry
  • tidal/src/schema/validation/policies.rs -- AgentPolicy gains required_scopes: Vec<ScopePermission> and required_token: bool (capability binding)
  • tidal/src/session/policy.rs -- PolicyEvaluator capability phase; new PolicyViolationKind variants
  • tidal/src/session/state.rs -- SessionState.capability_token field
  • tidal/src/session/audit.rs -- audit entries already capture accepted/reason; denial reason strings extended (no shape change)
  • tidal/src/db/sessions.rs -- session_signal revocation gate + capability eval before policy eval
  • tidal/src/db/replication_ops.rs -- signal_for_tenant capability enforcement for tenant/community writes
  • tidal/src/db/mod.rs -- capability_registry field on TidalDb
  • tidal/src/db/session_restore.rs -- replay Capability / CapabilityRevocation WAL events at startup
  • tidal/src/storage/keys.rs -- Tag::Capability = 0x0E, Tag::CapabilityRevocation = 0x0F
  • tidal/src/schema/error.rs -- TidalError capability variants; SchemaError capability-binding validation
  • tidalctl/src/main.rs -- agents and revocations subcommands
• Files created:
  • tidal/src/governance/capability.rs -- CapabilityToken, ScopePermission, CapabilityRegistry
  • tidal/src/db/capabilities.rs -- grant_capability / revoke_capability API + persistence
  • tidal/tests/m10p2_capabilities.rs -- integration tests (grant/deny/revoke/restart)

Research References

• docs/research/tidaldb_wal.md -- WAL record framing, BLAKE3 per-record checksum (capability/revocation events reuse the session-journal envelope)
• thoughts.md -- Part V.12 (subject-prefix key encoding; capability keys keyed under the agent's reserved entity range)
• /tmp/m9m10_brief.md -- section 1.5 (canonical CapabilityToken shape), section 2 M10p2 table, section 4 invariants (<1s synchronous revocation, local-profile-intact), section 5 test strategy
• docs/planning/milestone-8/phase-1/ -- doc-format precedent (this OVERVIEW mirrors it)

Acceptance Criteria (Phase Level)

• CapabilityToken carries agent_id: AgentId, scopes: Vec<ScopePermission>, created_at_ns: u64, expires_at_ns: u64, and revoked_at_ns: AtomicU64 (0 = not revoked); ScopePermission { scope: SignalScope, read: bool, write: bool }
• CapabilityToken::is_live(now_ns) returns false when now_ns >= expires_at_ns (TTL elapsed) OR revoked_at_ns != 0 && now_ns >= revoked_at_ns; true otherwise — verified by unit test across all four corners
• CapabilityToken::permits(scope, want_read, want_write) returns true only when a matching ScopePermission grants every requested mode; absent scope = deny
• grant_capability(agent_id, scopes, ttl) returns a CapabilityToken, inserts it into CapabilityRegistry, and persists a Tag::Capability record before returning
• revoke_capability(token_id) performs an atomic revoked_at_ns.store(now_ns) on the live token (O(1)), persists a Tag::CapabilityRevocation record, and is idempotent (second revoke is a no-op)
• A session bound to a token with no community-write ScopePermission is rejected on a community-scoped session_signal with TidalError::PolicyViolation whose kind is PolicyViolationKind::InsufficientCapability (matches UAT step 2)
• Every capability denial appends an AuditEntry { accepted: false, reason: Some(..) } to the session AuditLog
• After revoke_capability, the next community-scoped write by that agent is rejected with PolicyViolationKind::CapabilityRevoked; measured wall-clock from revoke-return to first-denial is < 1s p99 (the gate is an O(1) atomic read, so effectively immediate) and does NOT depend on the 60s sweeper
• Local-scope (SignalScope::Local) writes are NEVER blocked by capability checks — an agent with no token still writes its local profile (local-profile-intact guarantee)
• Grants and revocations survive close/reopen: a token granted then revoked before shutdown is replayed at startup such that a post-restart community write by that agent is still denied with CapabilityRevoked
• tidalctl agents --path <dir> prints JSON listing each agent's tokens (token_id, scopes, expires_at_ns, live/expired/revoked status); tidalctl revocations --path <dir> prints the revocation history. Both are offline read-only scans (no DB open)
• WAL/checkpoint backward-compat: a data dir written by M10p1 (no capability records) opens cleanly; absence of capability records means "no capabilities granted", not an error
• cargo fmt clean, cargo clippy -p tidaldb -D warnings clean, all unit + m10p2_capabilities integration tests pass

Task Execution Order

Task 01: CapabilityToken + ScopePermission ──┐
         (governance/capability.rs)          │
                                             ├──> Task 03: PolicyEvaluator capability phase
Task 02: CapabilityRegistry + Tags ──────────┤        (session/policy.rs, state.rs, error.rs)
         (registry, keys.rs, mod.rs)         │                     │
                                             │                     v
                                             └──> Task 04: grant/revoke API + persistence
                                                         (db/capabilities.rs, db/mod.rs)
                                                                   │
                                                                   v
                                              Task 05: Write-path enforcement + audit
                                                       (db/sessions.rs, db/replication_ops.rs)
                                                                   │
                                                                   v
                                              Task 06: Startup replay + restart durability
                                                       (db/session_restore.rs)
                                                                   │
                                                                   v
                                              Task 07: tidalctl agents/revocations + m10p2 UAT
                                                       (tidalctl/src/main.rs, tests/)

Tasks 01 and 02 are fully parallelizable (pure types vs. registry + tag bytes). Task 03 depends on 01 (needs permits/is_live). Task 04 depends on 02 + 03. Task 05 depends on 04 (needs grant/revoke + registry). Task 06 depends on 04/05 (replays the same records the API writes). Task 07 depends on all (CLI scans the persisted records; UAT exercises the full grant->deny->revoke->restart path).

Module Location

File	Status	Contains
tidal/src/governance/capability.rs	NEW	CapabilityToken, ScopePermission, CapabilityRegistry, is_live/permits
tidal/src/db/capabilities.rs	NEW	TidalDb::grant_capability, revoke_capability, persistence helpers
tidal/tests/m10p2_capabilities.rs	NEW	Grant/deny/revoke/restart integration tests
tidal/src/governance/mod.rs	MODIFIED	Re-export capability types
tidal/src/storage/keys.rs	MODIFIED	Tag::Capability = 0x0E, Tag::CapabilityRevocation = 0x0F + from_byte
tidal/src/schema/validation/policies.rs	MODIFIED	AgentPolicy.required_scopes, required_token
tidal/src/session/policy.rs	MODIFIED	Capability phase; new PolicyViolationKind variants
tidal/src/session/state.rs	MODIFIED	SessionState.capability_token field
tidal/src/db/sessions.rs	MODIFIED	session_signal revocation gate + capability eval
tidal/src/db/replication_ops.rs	MODIFIED	signal_for_tenant capability enforcement
tidal/src/db/mod.rs	MODIFIED	capability_registry: Arc<CapabilityRegistry> field
tidal/src/db/session_restore.rs	MODIFIED	Replay capability/revocation WAL events
tidal/src/schema/error.rs	MODIFIED	TidalError capability variants; SchemaError capability validation
tidalctl/src/main.rs	MODIFIED	agents, revocations subcommands

Technical Design

CapabilityToken and ScopePermission

// tidal/src/governance/capability.rs

use std::sync::atomic::{AtomicU64, Ordering};

use crate::governance::scope::SignalScope;
use crate::session::types::AgentId;

/// One scope's read/write grant inside a capability token.
///
/// Absence of a `ScopePermission` for a scope is an implicit deny: a token
/// only permits what it explicitly lists.
#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
pub struct ScopePermission {
    /// The signal scope this permission applies to.
    pub scope: SignalScope,
    /// Whether the agent may read signals at this scope.
    pub read: bool,
    /// Whether the agent may write signals at this scope.
    pub write: bool,
}

impl ScopePermission {
    /// A read+write grant for a single scope.
    #[must_use]
    pub const fn read_write(scope: SignalScope) -> Self {
        Self { scope, read: true, write: true }
    }
}

/// A capability granted to an agent: which scopes it may read/write, and for
/// how long. Revocation is an atomic `revoked_at_ns` store on the live token,
/// checked synchronously on the write path (NOT via the 60s sweeper).
///
/// Cloning is intentionally not derived: the live token is shared as
/// `Arc<CapabilityToken>` so that a revoke on the registry copy is visible to
/// every session holding a clone of the `Arc`.
#[derive(Debug, serde::Serialize, serde::Deserialize)]
pub struct CapabilityToken {
    /// Stable identifier (UUID-like string) used as the persistence + registry key.
    pub token_id: String,
    /// The agent this token authorizes.
    pub agent_id: AgentId,
    /// Per-scope read/write grants. Empty = a token that permits nothing.
    pub scopes: Vec<ScopePermission>,
    /// Nanoseconds since Unix epoch when the token was granted.
    pub created_at_ns: u64,
    /// Nanoseconds since Unix epoch when the token expires (TTL boundary).
    pub expires_at_ns: u64,
    /// `0` = not revoked. Otherwise the revocation instant in ns since epoch.
    /// Atomic so revocation is a lock-free O(1) store visible to all readers.
    #[serde(with = "atomic_u64_serde")]
    pub revoked_at_ns: AtomicU64,
}

impl CapabilityToken {
    /// Whether this token is currently usable at `now_ns`.
    ///
    /// `false` once the TTL has elapsed (`now_ns >= expires_at_ns`) or the token
    /// has been revoked at-or-before `now_ns`. This is the single source of
    /// truth for liveness — both the TTL and the revocation gate funnel through
    /// it, so there is no second code path to keep in sync.
    #[must_use]
    pub fn is_live(&self, now_ns: u64) -> bool {
        if now_ns >= self.expires_at_ns {
            return false;
        }
        let revoked = self.revoked_at_ns.load(Ordering::Acquire);
        revoked == 0 || now_ns < revoked
    }

    /// Whether this token grants the requested access at `scope`.
    ///
    /// Liveness is NOT checked here; callers gate on `is_live` first so the
    /// denial reason (`Expired`/`Revoked` vs `Insufficient`) is distinguishable.
    #[must_use]
    pub fn permits(&self, scope: SignalScope, want_read: bool, want_write: bool) -> bool {
        self.scopes.iter().any(|p| {
            p.scope == scope && (!want_read || p.read) && (!want_write || p.write)
        })
    }

    /// Atomically mark the token revoked at `now_ns`. Idempotent: an already-
    /// revoked token keeps its earlier revocation instant.
    pub fn revoke(&self, now_ns: u64) {
        // CAS-from-zero so the first revoke wins and later ones are no-ops.
        let _ = self.revoked_at_ns.compare_exchange(
            0, now_ns, Ordering::AcqRel, Ordering::Acquire,
        );
    }
}

CapabilityRegistry

// tidal/src/governance/capability.rs (continued)

use std::sync::Arc;
use dashmap::DashMap;

/// In-memory index of granted capability tokens, owned by `TidalDb`.
///
/// Keyed by `token_id`; a secondary index maps `agent_id -> Vec<token_id>` so
/// the write path can resolve "does this agent hold a live token granting
/// (scope, write)?" in O(tokens-per-agent), which is tiny in practice.
#[derive(Debug, Default)]
pub struct CapabilityRegistry {
    by_id: DashMap<String, Arc<CapabilityToken>>,
    by_agent: DashMap<String, Vec<String>>, // agent_id.as_str() -> token_ids
}

impl CapabilityRegistry {
    #[must_use]
    pub fn new() -> Self {
        Self::default()
    }

    /// Insert a freshly granted token (also used by startup replay).
    pub fn insert(&self, token: Arc<CapabilityToken>) {
        self.by_agent
            .entry(token.agent_id.as_str().to_owned())
            .or_default()
            .push(token.token_id.clone());
        self.by_id.insert(token.token_id.clone(), token);
    }

    /// Fetch a token by id for revocation / inspection.
    #[must_use]
    pub fn get(&self, token_id: &str) -> Option<Arc<CapabilityToken>> {
        self.by_id.get(token_id).map(|e| Arc::clone(e.value()))
    }

    /// True if `agent` holds at least one live token permitting (scope, modes).
    #[must_use]
    pub fn agent_permits(
        &self,
        agent: &AgentId,
        scope: SignalScope,
        want_read: bool,
        want_write: bool,
        now_ns: u64,
    ) -> bool {
        let Some(ids) = self.by_agent.get(agent.as_str()) else {
            return false;
        };
        ids.iter().filter_map(|id| self.by_id.get(id)).any(|tok| {
            tok.is_live(now_ns) && tok.permits(scope, want_read, want_write)
        })
    }
}

PolicyViolationKind and PolicyEvaluator capability phase

// tidal/src/session/policy.rs  (extends the existing enum)

#[derive(Debug, Clone, PartialEq, Eq)]
pub enum PolicyViolationKind {
    Expired,
    CountCap,
    Denied,
    NotAllowed,
    /// No live token grants the (scope, mode) this write requires.
    InsufficientCapability,
    /// The agent's capability token was revoked at or before this write.
    CapabilityRevoked,
    /// The agent's capability token TTL has elapsed.
    CapabilityExpired,
}

// tidal/src/session/policy.rs  (new method on PolicyEvaluator)

impl<'a> PolicyEvaluator<'a> {
    /// Capability phase: run AFTER the existing allow/deny/count/duration checks
    /// and ONLY for non-local scopes. `Local` scope is exempt — the local
    /// profile is never gated by capabilities.
    ///
    /// `token` is the session's bound token (if any). `now_ns` is the write
    /// timestamp. Returns the same `PolicyViolation` shape as `check`.
    ///
    /// # Errors
    ///
    /// `InsufficientCapability` when no token / no matching scope; `CapabilityExpired`
    /// when the token's TTL elapsed; `CapabilityRevoked` when it was revoked.
    pub fn check_capability(
        &self,
        signal_type: &str,
        scope: SignalScope,
        token: Option<&CapabilityToken>,
        now_ns: u64,
    ) -> Result<(), PolicyViolation> {
        if scope == SignalScope::Local {
            return Ok(()); // local-profile-intact: never gated
        }
        let make = |kind: PolicyViolationKind, reason: String| PolicyViolation {
            kind,
            signal_type: signal_type.to_owned(),
            policy_name: self.policy_name.to_owned(),
            reason,
        };
        let Some(tok) = token else {
            return Err(make(
                PolicyViolationKind::InsufficientCapability,
                format!("no capability token bound for scope {scope:?}"),
            ));
        };
        if !tok.is_live(now_ns) {
            let revoked = tok.revoked_at_ns.load(std::sync::atomic::Ordering::Acquire);
            let kind = if revoked != 0 && now_ns >= revoked {
                PolicyViolationKind::CapabilityRevoked
            } else {
                PolicyViolationKind::CapabilityExpired
            };
            return Err(make(kind, format!("token '{}' not live", tok.token_id)));
        }
        if !tok.permits(scope, false, true) {
            return Err(make(
                PolicyViolationKind::InsufficientCapability,
                format!("token '{}' lacks write at scope {scope:?}", tok.token_id),
            ));
        }
        Ok(())
    }
}

grant / revoke API and TidalError variants

// tidal/src/db/capabilities.rs

impl TidalDb {
    /// Grant `agent` the listed scope permissions for `ttl`. Persists the grant
    /// to `Tag::Capability` before returning so it survives restart.
    ///
    /// # Errors
    ///
    /// - `TidalError::ReadOnly` on a follower node.
    /// - `TidalError::Storage` if the durable write fails.
    pub fn grant_capability(
        &self,
        agent_id: AgentId,
        scopes: Vec<ScopePermission>,
        ttl: std::time::Duration,
    ) -> crate::Result<Arc<CapabilityToken>> { /* ... */ }

    /// Revoke a token by id. Atomic O(1) `revoked_at_ns` store on the live token
    /// + a durable `Tag::CapabilityRevocation` record. Idempotent.
    ///
    /// # Errors
    ///
    /// - `TidalError::ReadOnly` on a follower node.
    /// - `TidalError::CapabilityNotFound` if no token has `token_id`.
    pub fn revoke_capability(&self, token_id: &str) -> crate::Result<()> { /* ... */ }
}

// tidal/src/schema/error.rs  (new TidalError variants)

/// A signal write was rejected because the agent lacks a live capability
/// granting the requested scope. Carries the typed denial reason for dispatch.
#[error("capability denied: agent '{agent_id}' for scope '{scope}': {reason}")]
CapabilityDenied {
    agent_id: String,
    scope: String,
    reason: String,
},
/// `revoke_capability` was called for an unknown token id.
#[error("capability not found: '{0}'")]
CapabilityNotFound(String),

AgentPolicy capability binding

// tidal/src/schema/validation/policies.rs  (AgentPolicy gains two fields)

pub struct AgentPolicy {
    pub allowed_signals: Vec<String>,
    pub denied_signals: Vec<String>,
    pub max_session_duration: Duration,
    pub max_signals_per_session: u32,
    /// Scopes (beyond `Local`) this policy requires a live capability token to
    /// write. Empty = community/session writes are unrestricted by capability.
    pub required_scopes: Vec<ScopePermission>,
    /// If `true`, a session under this policy MUST be bound to a live token to
    /// write any non-local scope. Defaults to `false` for backward compat.
    pub required_token: bool,
}

Tag additions and persisted record shape

// tidal/src/storage/keys.rs  (extend Tag)

pub enum Tag {
    // ... existing 0x01..=0x0D ...
    /// Capability grant records (M10p2).
    Capability = 0x0E,
    /// Capability revocation records (M10p2).
    CapabilityRevocation = 0x0F,
}
// from_byte gains: 0x0E => Some(Self::Capability), 0x0F => Some(Self::CapabilityRevocation)

// tidal/src/wal/format/session.rs  (extend SessionWalEvent for durable replay)

pub enum SessionWalEvent {
    Start { /* ... */ },
    Signal { /* ... */ },
    Close { session_id: u64 },
    /// A capability token was granted (M10p2). Serialized token bytes mirror the
    /// `Tag::Capability` storage record so replay reconstructs the registry.
    CapabilityGrant {
        token_id: String,
        agent_id: String,
        scopes: Vec<(u8, bool, bool)>, // (scope discriminant, read, write)
        created_at_ns: u64,
        expires_at_ns: u64,
    },
    /// A capability token was revoked (M10p2).
    CapabilityRevoke { token_id: String, revoked_at_ns: u64 },
}

Notes

Revocation is synchronous and atomic — never the sweeper

The <1s p99 revocation gate is an AtomicU64::load of revoked_at_ns on the session signal write path (session_signal, before policy eval) and on signal_for_tenant. It is read directly off the Arc<CapabilityToken> the session holds, which is the same Arc the registry mutated in revoke, so the store is visible on the next write with no propagation step. The 60s session TTL sweeper (db/sweeper.rs) is explicitly NOT involved — routing revocation through it would blow the SLA by 60x. This mirrors the M9 stop-forward gate (stop_forward_at_ns) exactly.

Local-profile-intact guarantee

SignalScope::Local writes bypass the capability phase entirely (check_capability returns Ok(()) for Local). An agent with no token, an expired token, or a revoked token can still write its local profile. The UAT's "U queries local-only ... views" step depends on this: revoking A_trusted's community scope must not touch U's local feed. A routing bug that gates local writes is silent data loss — the integration test asserts the local feed is byte-identical across grant/deny/revoke.

Capability eval ordering and audit

The capability phase runs AFTER the existing duration/count/deny/allow checks so that a session that is expired-by-duration still reports Expired, not CapabilityExpired. Every capability denial appends an AuditEntry { accepted: false, reason: Some(..) } to the session AuditLog exactly like the existing policy denials — no new audit structure, just new reason strings keyed by the typed PolicyViolationKind. signals_rejected is incremented on the same path.

Backward compatibility is non-negotiable

Tag::Capability / Tag::CapabilityRevocation are new tag bytes; older data dirs simply have none, which decodes as "no capabilities granted". The SessionWalEvent enum gains two variants encoded behind a new record-type byte in the session journal; decode_session_events already tolerates unknown trailing bytes per-record (length-prefixed + BLAKE3-checksummed), and v1/v2 records continue to decode. AgentPolicy's two new fields default to empty/false, so existing schemas build unchanged.

Token id and entity-range keying

token_id is a process-unique string (UUID-style). Capability storage records are keyed under a reserved entity-id range derived from a stable hash of agent_id so a prefix scan over Tag::Capability enumerates an agent's tokens without a DB open — which is what tidalctl agents relies on.

Done When

A developer can grant_capability(agent, [ScopePermission::read_write(Community(c))], ttl), bind it to a session, and have community-scoped writes succeed; an agent without that grant is rejected with PolicyViolationKind::InsufficientCapability and an audit entry; revoke_capability(token_id) causes the very next community write by that agent to fail with CapabilityRevoked in under one second (synchronous atomic gate, not the sweeper); local writes are never affected; the grant and revocation survive close/reopen via durable replay; and tidalctl agents / tidalctl revocations show the capability and revocation history from an offline scan. All existing M0–M10p1 tests pass unchanged.