f4cfd6c81f
Milestone 8 (phases 1-4): - Shard-aware WAL segment naming, BatchHeader v2, ShardRouter - Transport trait, InProcessTransport, WalShipper, FollowerDb - HLC, PNCounter, LWWRegister, CrdtSignalState, ReconciliationEngine - Session replication bridge with SeqNo/HWM, idempotency store Forage application: - Multi-source discovery engine with MAB exploration - Embedding-based label system, server handlers, UI refresh Other: - QUICKSTART.md, README.md, milestone-8 planning docs - Hard negative union semantics, RLHF export enhancements - Recovery benchmark and visibility test expansions - Split 8 oversized source files per CODING_GUIDELINES §9 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
87 lines
5.4 KiB
Markdown
87 lines
5.4 KiB
Markdown
# m8p4: Session Continuity and Agent Memory Across Regions
|
|
|
|
## Delivers
|
|
|
|
Session writes carry monotonic sequence numbers and idempotency keys, enabling
|
|
agents to roam between regions without losing session state or violating memory
|
|
guarantees. Hard negatives are monotonic: once hidden, an item never appears
|
|
to the user even while replicas are converging. Cross-region session visibility
|
|
is achieved within the replication lag window (< 2 seconds).
|
|
|
|
Deliverables:
|
|
- `SessionSeqNo(u64)`: monotonic sequence number per session write, included in WAL event
|
|
- `IdempotencyKey(u128)`: BLAKE3-derived key per session operation for exactly-once semantics
|
|
- `SessionReplicationBridge`: replicates session journal entries via the `Transport` trait alongside WAL segments
|
|
- Cross-region agent memory: a session started in us-east is readable in eu-west after replication lag
|
|
- Hard-negative monotonicity: during convergence, the union of all hard negatives is applied (never the intersection)
|
|
|
|
## Dependencies
|
|
|
|
- **Requires:** Phase 8.2 (WAL shipping, SegmentReceiver), Phase 8.3 (LWWRegister for hard negatives, HLC)
|
|
- **Files modified:**
|
|
- `tidal/src/wal/format/session.rs` -- add `session_seqno` and `idempotency_key` fields to `SessionWalEvent`
|
|
- `tidal/src/session/state.rs` -- track per-session high-water-mark seqno
|
|
- `tidal/src/entities/hard_neg.rs` -- union-based merge during convergence (never remove a hard negative during replication)
|
|
- `tidal/src/wal/session_journal.rs` -- include session events in replication payload
|
|
- **Files created:**
|
|
- `tidal/src/replication/session_bridge.rs` -- `SessionReplicationBridge`
|
|
- `tidal/src/replication/idempotency.rs` -- `IdempotencyKey`, `IdempotencyStore` (bounded LRU)
|
|
|
|
## Research References
|
|
|
|
- `VISION.md` -- Sessions / Agent Context section: "Sessions can be forked, merged, and policy-limited so an agent only sees what it is allowed to remember"
|
|
|
|
## Acceptance Criteria (Phase Level)
|
|
|
|
- [ ] `SessionSeqNo` is a monotonically increasing u64 per session; writes with seqno <= high-water-mark on the receiver are idempotent no-ops
|
|
- [ ] `IdempotencyKey` is derived from `BLAKE3(session_id || seqno || operation_bytes)`; stored in a bounded LRU of 100K entries per node
|
|
- [ ] Duplicate session writes (same idempotency key) across regions produce exactly one state change
|
|
- [ ] Session started in region A is visible (session metadata + preference hints + annotations) in region B within 2 seconds (in-process transport)
|
|
- [ ] Hard negatives are replicated with union semantics: if shard A has `hide(user, item)` and shard B does not, after replication both shards have the hide; during convergence the stricter (hide) always wins
|
|
- [ ] Agent roaming test: create session in us-east, write 5 preference signals; switch to eu-west follower; read session signals within 2 seconds; all 5 signals visible
|
|
- [ ] No phantom un-hides: once a hard negative is applied, it is never removed by replication (only by explicit user action with a higher HLC timestamp)
|
|
|
|
## Task Execution Order
|
|
|
|
```
|
|
Task 01: SessionSeqNo + WAL Format ──────┐
|
|
├──> Task 03: SessionReplicationBridge
|
|
Task 02: IdempotencyKey + Store ──────────┘ │
|
|
v
|
|
Task 04: HardNeg Monotonicity
|
|
│
|
|
v
|
|
Task 05: Cross-Region Session Tests
|
|
```
|
|
|
|
Tasks 01 and 02 are parallelizable. Task 03 depends on both. Task 04 depends on 03. Task 05 depends on all.
|
|
|
|
## Module Location
|
|
|
|
| File | Status | Contains |
|
|
|------|--------|----------|
|
|
| `tidal/src/replication/session_bridge.rs` | NEW | `SessionReplicationBridge` |
|
|
| `tidal/src/replication/idempotency.rs` | NEW | `IdempotencyKey`, `IdempotencyStore` |
|
|
| `tidal/src/wal/format/session.rs` | MODIFIED | `session_seqno`, `idempotency_key` fields |
|
|
| `tidal/src/session/state.rs` | MODIFIED | Per-session high-water-mark seqno |
|
|
| `tidal/src/entities/hard_neg.rs` | MODIFIED | Union-based merge, LWW with HLC |
|
|
| `tidal/src/wal/session_journal.rs` | MODIFIED | Session events in replication payload |
|
|
|
|
## Notes
|
|
|
|
### Union, not LWW, for hard negatives during convergence
|
|
|
|
The safety property is: a hidden item must never appear to the user, even while replicas are still converging. This means during the convergence window, we take the union of all hard negatives from all shards. Only after full convergence can LWW semantics resolve explicit unhide operations.
|
|
|
|
### Bounded idempotency store
|
|
|
|
The LRU holds 100K entries (~1.6 MB). This means idempotency is guaranteed for the last 100K operations per session. Older operations that are replayed are handled by the seqno high-water-mark check (which is unbounded and monotonic).
|
|
|
|
### Session replication piggybacks on WAL shipping
|
|
|
|
Session journal entries are bundled into a separate channel on the same `Transport`, not mixed into the signal WAL segments. This keeps the signal WAL path fast and the session path independently tunable.
|
|
|
|
## Done When
|
|
|
|
An agent creates a session in one region, writes preference signals and hard negatives, then the session is readable from a follower in another region within 2 seconds. Duplicate operations across regions produce no double-counting. Items hidden in one region are never visible in another region during convergence.
|