tidaldb/docs/planning/milestone-4/phase-4.md

m4p4 — Session-Aware Ranking and M4 UAT (✅ COMPLETE 2026-02-21)

Phase spec and acceptance criteria: ROADMAP · Milestone 4 · Phase 4. Milestone index: README.md. Backfilled record.

What shipped

1. FOR SESSION on both query surfaces. RetrieveBuilder::for_session(session_id) and the SEARCH equivalent (tidal/src/query/search/executor.rs:265 — with_session(context, snapshot)). The db layer loads the session snapshot and derives a SessionContext (tidal/src/db/query_ops.rs:172,440), so an archived session works exactly like a live one — frozen values instead of decayed-to-now values.

2. SessionContext as the ranking input (tidal/src/session/snapshot.rs:54). Built by SessionContext::from_snapshot: annotation text is split on whitespace, lowercased, and de-duplicated through a HashSet into keywords; reward_velocity is the current (or frozen) score of the reward signal; session metadata rides along.

3. The boost formula (ProfileExecutor::session_boost, tidal/src/ranking/executor/mod.rs:605-627):

   hint_score = matched_keywords / total_keywords     // [0,1]
   vel_norm   = reward_velocity / (reward_velocity+1) // Michaelis-Menten saturation
   boost      = hint_score * 0.3 + vel_norm * 0.2
   

   A keyword matches if any of the candidate's metadata values contains it, case-insensitively. The boost is additive and applied after base scoring, before min-max normalization, so it layers onto personalization instead of replacing it. Keywords are lowercased once per query (lowered_session_keywords), not once per candidate.

4. Session state travels back with the results. Results.session_snapshot: Option<SessionSnapshot> is populated whenever for_session is present, so an agent gets ranked items and its own session state in one round trip.

5. A ranking reason code. ReasonCode::SessionContext (tidal/src/ranking/reason.rs:71,232) makes a session-influenced result explainable rather than mysterious.

6. The M4 UAT (tidal/tests/m4_uat.rs) — 12 #[test] functions, one per scenario step, covering lifecycle, signals, policy accept/reject, annotations, snapshot, archive, FOR SESSION ranking, isolation, and AgentId validation.

Evidence

Criterion	Proof
FOR SESSION query returns results and attaches the snapshot	m4_uat.rs::step8_for_session_ranking_boost
Keyword hints move ranking	session_durability.rs::hint_keywords_boost_matching_items; unit test ranking/executor/mod.rs::session_boost_keyword_match_is_case_insensitive
Empty keyword set is a no-op (no divide-by-zero, no phantom boost)	unit test ranking/executor/mod.rs::session_boost_empty_keywords_is_noop
Sessions do not leak across each other in ranking inputs	m4_uat.rs::step9_session_isolation
Archived session usable as query context	m4_uat.rs::step7_closed_session_snapshot
RETRIEVE and SEARCH behave identically on a swept session	review_pass2_query_for_session.rs::retrieve_and_search_degrade_identically_on_missing_session

Divergence from the plan

• Missing session degrades, it does not error — deliberately reversed. The ROADMAP criterion reads: "When for_session references a non-existent session, LumenError::Query("session not found") returned." That behaviour shipped and was then removed on purpose by the M0–M10 review pass 2 (9728194): FOR SESSION <swept-id> is a well-formed query, and CODING_GUIDELINES.md §6 ("graceful degradation, never failure") says it must execute without the boost. Before the fix RETRIEVE returned Err(SessionNotFound) while the structurally identical SEARCH degraded — a surface-specific outage the moment the session sweeper ran. Both surfaces now degrade. The ROADMAP criterion is stale; the current behaviour is correct.
• The < 5 ms session-context overhead figure is unevidenced. tidal/benches/session.rs has exactly the right benchmark (retrieve_1k_items/{without_session,with_session}), but no run of it is recorded in docs/. Measurable on demand (cargo bench -p tidaldb --bench session); not currently measured.

Known dead field — recorded, not silently tolerated

SessionContext.signaled_entities is populated by SessionContext::from_snapshot and documented "for entity-level boost", and tidal/src/session/audit.rs says the set "feeds the FOR SESSION entity-level boost". No such boost exists. session_boost reads only keywords and reward_velocity; a repo-wide search finds signaled_entities reached from a SessionContext only in a test fixture that sets it to HashSet::new(). The field on SessionSnapshot is genuinely used — but by cross-session preference aggregation (tidal/src/db/sessions.rs:547, m6p4), never by ranking.

Consequences worth stating plainly:

• The set is computed and cloned on every FOR SESSION query for no effect.
• m4_uat.rs::step8_for_session_ranking_boost is weaker than its comments suggest. With no annotation set, hint_score is 0 and every candidate receives the same uniform vel_norm * 0.2, which cannot reorder anything, so its assertion (rank_with <= rank_without) holds trivially. The keyword half of the boost is genuinely proven — by session_durability.rs::hint_keywords_boost_matching_items and the case-insensitivity unit test — so the mechanism works; only the entity-identity claim in step 8's comments is unbacked.

Not fixed here: this record is a planning-history backfill and does not touch Rust source. Either the entity boost should be implemented (a session that rewarded entity 5 arguably should rank entity 5 up) or the field and the two doc comments should go. Both are real changes needing a real owner.