tidaldb/docs/planning/milestone-8/phase-4/task-01-session-seqno.md
jordan f4cfd6c81f feat: complete M8 replication primitives + forage enhancements + docs
Milestone 8 (phases 1-4):
- Shard-aware WAL segment naming, BatchHeader v2, ShardRouter
- Transport trait, InProcessTransport, WalShipper, FollowerDb
- HLC, PNCounter, LWWRegister, CrdtSignalState, ReconciliationEngine
- Session replication bridge with SeqNo/HWM, idempotency store

Forage application:
- Multi-source discovery engine with MAB exploration
- Embedding-based label system, server handlers, UI refresh

Other:
- QUICKSTART.md, README.md, milestone-8 planning docs
- Hard negative union semantics, RLHF export enhancements
- Recovery benchmark and visibility test expansions
- Split 8 oversized source files per CODING_GUIDELINES §9

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 13:17:19 -07:00

140 lines
4.6 KiB
Markdown

# Task 01: SessionSeqNo + WAL Format Extension
## Delivers
`SessionSeqNo(u64)` type added to `tidal/src/wal/format/session.rs` and `tidal/src/session/state.rs`. Every session write operation carries a monotonically incrementing sequence number. The receiver's high-water-mark (HWM) rejects writes with `seqno <= hwm` as idempotent no-ops.
## Complexity: S
## Dependencies
- Phase 8.2 (WAL shipping, SegmentReceiver)
## Technical Design
```rust
// tidal/src/wal/format/session.rs
/// Monotonic sequence number for session writes.
///
/// Incremented once per session write operation (preference signal,
/// annotation, search query, interaction). Used by the receiver to
/// enforce idempotent replay and exactly-once semantics.
#[derive(
Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash,
serde::Serialize, serde::Deserialize,
)]
pub struct SessionSeqNo(pub u64);
impl SessionSeqNo {
pub const ZERO: Self = Self(0);
pub fn next(self) -> Self {
Self(self.0 + 1)
}
}
impl std::fmt::Display for SessionSeqNo {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "ssn:{}", self.0)
}
}
/// Extended session WAL event -- backward-compatible with existing format.
///
/// The `session_seqno` and `idempotency_key` fields are appended to the
/// existing `SessionWalEvent` bytes. Old readers that don't understand
/// the extension fields still decode the core event; they will silently
/// ignore the extra bytes (length-prefixed framing ensures this).
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct SessionWalEvent {
// --- existing fields (unchanged) ---
pub session_id: SessionId,
pub kind: SessionEventKind,
pub timestamp_ns: u64,
pub payload: SessionEventPayload,
// --- new fields (m8p4 extension) ---
/// Monotonically increasing sequence number for this session's writes.
/// Starts at 1 for the first write in a session.
#[serde(default)]
pub session_seqno: Option<SessionSeqNo>,
/// BLAKE3-derived idempotency key for exactly-once delivery.
/// `None` for events written before m8p4.
#[serde(default)]
pub idempotency_key: Option<u128>,
}
```
```rust
// tidal/src/session/state.rs (additions only)
/// Per-session monotonic write counter.
///
/// Tracks the highest seqno applied locally. Writes with seqno <= hwm
/// are silently dropped (idempotent replay is safe; the state is already
/// reflected in local storage).
#[derive(Debug, Default)]
pub struct SessionSeqNoTracker {
/// Map from SessionId to highest applied SessionSeqNo.
hwm: DashMap<SessionId, SessionSeqNo>,
}
impl SessionSeqNoTracker {
pub fn new() -> Self {
Self { hwm: DashMap::new() }
}
/// Returns `true` if this write should be applied (seqno > hwm).
/// Returns `false` if the write is a duplicate and should be skipped.
/// Updates the HWM on accept.
pub fn should_apply(&self, session_id: SessionId, seqno: SessionSeqNo) -> bool {
let mut entry = self.hwm.entry(session_id).or_insert(SessionSeqNo::ZERO);
if seqno > *entry {
*entry = seqno;
true
} else {
false
}
}
/// Current HWM for a session (returns ZERO if unknown).
pub fn hwm(&self, session_id: SessionId) -> SessionSeqNo {
self.hwm.get(&session_id)
.map(|v| *v)
.unwrap_or(SessionSeqNo::ZERO)
}
/// Initialize or reset HWM for a session (used on follower startup).
pub fn set_hwm(&self, session_id: SessionId, seqno: SessionSeqNo) {
self.hwm.insert(session_id, seqno);
}
}
```
### Sequence Number Assignment
```rust
// In session write path (tidal/src/session/mod.rs)
impl SessionManager {
fn next_seqno(&self, session_id: SessionId) -> SessionSeqNo {
// Fetch-and-increment per session.
let mut counter = self.seqno_counters
.entry(session_id)
.or_insert(SessionSeqNo::ZERO);
*counter = counter.next();
*counter
}
}
```
## Acceptance Criteria
- [ ] `SessionSeqNo` is `Copy + Clone + Ord + Hash + Serialize + Deserialize`
- [ ] `SessionSeqNoTracker::should_apply(id, seqno)` returns `true` for the first call with a given seqno, `false` on duplicate, and `true` again for a higher seqno
- [ ] HWM persists in memory; on follower node restart, WAL replay re-establishes HWM by scanning all `SessionWalEvent` entries in order
- [ ] `SessionWalEvent` with `session_seqno: None` (pre-m8p4 events) is decoded without error; `should_apply` returns `true` for all legacy events
- [ ] `cargo clippy -D warnings` and `cargo fmt` pass