Reconciles two independently-developed lines from base 006d3d0:
ours — M9/M10 community layers, retroactive purge + re-materialization,
signal revocation, agent capability boundaries, P1 feedback loop,
reason labels, instrumented metrics
theirs — M11/M12 cluster mode (tidal-net gRPC transport, tidal-server
cluster/scatter-gather, tidal-stress), multi-vector preference,
ANN candidate-gen, warm-tier day buckets, keyed signal snapshots
Notable semantic resolutions:
* storage::keys::Tag — both sides allocated 0x0E..0x11 for different
records. Kept theirs' 0x0E..0x1A (shipped on-disk format) and renumbered
ours to 0x1B..0x1E (CommunityMembership/Revocation/PurgeManifest/
CommunityLeave); Tag::ALL grown to 30 so the contiguity drift guard holds.
* ranking executor — took theirs' rewrite (SignalReadPlan pre-pass, keyed
SignalKey snapshots, Result-returning reads, finalize()) and re-applied
ours' M10 read suppression at the chokepoints it introduced:
single_signal_score, score_hot/trending/controversial,
CreatorEngagementRate, and the Stage-4 boost loop.
* signals::warm — theirs' day-bucket/read-time-rotation rewrite, with ours'
subtract_bucket and Clone extended to the new day tier; ours' test split
kept (warm/tests.rs, warm/proptests.rs) carrying theirs' updated bodies.
* db::signals — kept ours' contribution-logging try_cohort_attribution in
signal_dispatch.rs and theirs' event-time try_update_preference_vector;
dropped the superseded duplicates.
* db::mod / from_parts — theirs' constructors, with ours' purge/
re-materialization/revocation/community/skip-counter fields and restart
rebuilds; from_parts kept in its own file per the 600-line guideline.
* schema::validation::builders — ours' module split with theirs' expanded
tests; policy validation runs both sides' checks (read-signal lists +
profile overrides, then the zero-duration limit guard).
* feedback Unhide no longer writes a -1.0 "hide" signal: theirs' engine
rejects negative weights (spec §8). Reverses index state only, matching
every other undo action.
* SessionState::new is now the single construction path (gains
overrides_rejected/default_profile); AuditEntry gains kind on the
deserialize path, inferred from the accepted flag as before.
* Removed tidal/src/replication/tcp_transport.rs and its test: never
declared in replication/mod.rs on either branch, so it had never
compiled and nothing referenced it. Superseded by tidal-net's
GrpcTransport.
Verified: cargo clippy -p tidaldb (lib) clean; --all-targets compiles for
tidaldb/tidal-net/tidal-server/tidal-stress; 2094/2094 lib tests and the
integration suite pass except m8p3_reconcile_production's two CRDT-count
assertions, which fail identically on MERGE_HEAD (pre-existing).
tidalctl cannot build locally: its aws-sdk deps need rustc 1.91.1, local
toolchain is 1.91.0.
251 lines
8.2 KiB
Rust
251 lines
8.2 KiB
Rust
//! Zone A (session lifecycle & sweeper) regression tests for the
|
|
//! M0-M10 code-review pass-2 findings.
|
|
//!
|
|
//! These are end-to-end persistence tests: they open a persistent database,
|
|
//! exercise the session lifecycle, shut down, and reopen — the only way to
|
|
//! reproduce the two BLOCKERs and the CRITICALs, which only manifest across a
|
|
//! restart.
|
|
|
|
#![allow(
|
|
clippy::unwrap_used,
|
|
clippy::too_many_lines,
|
|
clippy::doc_markdown,
|
|
clippy::needless_collect
|
|
)]
|
|
|
|
use std::{collections::HashMap, time::Duration};
|
|
|
|
use tidaldb::{
|
|
AgentPolicy, SessionId, TidalDb,
|
|
schema::{DecaySpec, EntityId, EntityKind, SchemaBuilder, Timestamp},
|
|
};
|
|
|
|
/// Schema with one signal type and a single long-TTL session policy named
|
|
/// "agent", used by every test in this file.
|
|
fn schema() -> tidaldb::schema::Schema {
|
|
let mut b = SchemaBuilder::new();
|
|
let _ = b
|
|
.signal("view", EntityKind::Item, DecaySpec::Permanent)
|
|
.add();
|
|
b.session_policy(
|
|
"agent",
|
|
AgentPolicy {
|
|
allowed_signals: vec!["view".to_string()],
|
|
denied_signals: vec![],
|
|
max_session_duration: Duration::from_secs(3600),
|
|
max_signals_per_session: 10_000,
|
|
..AgentPolicy::default()
|
|
},
|
|
);
|
|
b.build().unwrap()
|
|
}
|
|
|
|
/// BLOCKER regression: a clean restart must NOT re-issue an archived session id
|
|
/// and overwrite its durable snapshot.
|
|
///
|
|
/// Reproduces the reviewer's two assertions:
|
|
/// (1) a new session after reopen gets an id strictly greater than the
|
|
/// previously-closed id, and
|
|
/// (2) `session_snapshot` of the old id still returns the ORIGINAL
|
|
/// user_id / signals_written — the archive was not destroyed.
|
|
#[test]
|
|
fn reopen_does_not_reissue_archived_session_id() {
|
|
let dir = tempfile::tempdir().unwrap();
|
|
|
|
// Phase 1: start + signal (user 1, 5 signals) + close, then shut down.
|
|
let first_id;
|
|
{
|
|
let db = TidalDb::builder()
|
|
.with_data_dir(dir.path())
|
|
.with_schema(schema())
|
|
.open()
|
|
.unwrap();
|
|
|
|
let handle = db
|
|
.start_session(1, "agent", "agent", HashMap::new())
|
|
.unwrap();
|
|
first_id = handle.id;
|
|
for i in 1..=5u64 {
|
|
db.session_signal(
|
|
&handle,
|
|
"view",
|
|
EntityId::new(i),
|
|
1.0,
|
|
Timestamp::now(),
|
|
None,
|
|
)
|
|
.unwrap();
|
|
}
|
|
db.close_session(handle).unwrap();
|
|
|
|
// Snapshot is archived with the true author + count.
|
|
let snap = db.session_snapshot(first_id).unwrap();
|
|
assert_eq!(snap.user_id, 1);
|
|
assert_eq!(snap.signals_written, 5);
|
|
|
|
db.close().unwrap();
|
|
}
|
|
|
|
// Phase 2: reopen and start a NEW session for a different user.
|
|
{
|
|
let db = TidalDb::builder()
|
|
.with_data_dir(dir.path())
|
|
.with_schema(schema())
|
|
.open()
|
|
.unwrap();
|
|
|
|
let handle = db
|
|
.start_session(2, "agent", "agent", HashMap::new())
|
|
.unwrap();
|
|
let second_id = handle.id;
|
|
|
|
// (1) The new id must be strictly greater than the archived id — the
|
|
// allocator was advanced past the durable snapshot on reopen.
|
|
assert!(
|
|
second_id.as_u64() > first_id.as_u64(),
|
|
"reopened session id {second_id} must exceed the archived id {first_id}; \
|
|
the allocator was reseeded to 1 and re-issued the archived id"
|
|
);
|
|
|
|
db.close_session(handle).unwrap();
|
|
|
|
// (2) The original archived snapshot is intact — not overwritten by the
|
|
// reused id's new (user 2, 0-signal) close.
|
|
let snap = db.session_snapshot(first_id).unwrap();
|
|
assert_eq!(
|
|
snap.user_id, 1,
|
|
"the user-1 archived snapshot must survive a restart"
|
|
);
|
|
assert_eq!(
|
|
snap.signals_written, 5,
|
|
"the user-1 archived signal count must survive a restart"
|
|
);
|
|
|
|
db.close().unwrap();
|
|
}
|
|
}
|
|
|
|
/// End-to-end companion to the in-module "snapshot is authoritative" guard:
|
|
/// after a clean close + reopen, the session is absent from `active_sessions()`
|
|
/// and resolves only as an archived snapshot. (The torn-Close phantom-state
|
|
/// case — snapshot present AND Start-without-Close in the journal — is covered
|
|
/// directly by `session_restore.rs::durable_snapshot_blocks_active_restore`,
|
|
/// which can inject that exact state through the internal restore path.)
|
|
#[test]
|
|
fn closed_session_is_not_restored_as_active() {
|
|
let dir = tempfile::tempdir().unwrap();
|
|
|
|
let closed_id;
|
|
{
|
|
let db = TidalDb::builder()
|
|
.with_data_dir(dir.path())
|
|
.with_schema(schema())
|
|
.open()
|
|
.unwrap();
|
|
|
|
let handle = db
|
|
.start_session(1, "agent", "agent", HashMap::new())
|
|
.unwrap();
|
|
closed_id = handle.id;
|
|
db.session_signal(
|
|
&handle,
|
|
"view",
|
|
EntityId::new(1),
|
|
1.0,
|
|
Timestamp::now(),
|
|
None,
|
|
)
|
|
.unwrap();
|
|
db.close_session(handle).unwrap();
|
|
db.close().unwrap();
|
|
}
|
|
|
|
// Reopen: the durable snapshot for `closed_id` is present. Even if a stale
|
|
// Start-without-Close survived in the journal, the snapshot is authoritative
|
|
// and the session must NOT come back as active.
|
|
{
|
|
let db = TidalDb::builder()
|
|
.with_data_dir(dir.path())
|
|
.with_schema(schema())
|
|
.open()
|
|
.unwrap();
|
|
|
|
let active_ids: Vec<u64> = db
|
|
.active_sessions()
|
|
.into_iter()
|
|
.map(|s| s.id.as_u64())
|
|
.collect();
|
|
assert!(
|
|
!active_ids.contains(&closed_id.as_u64()),
|
|
"a closed session (durable snapshot present) must never restore as active"
|
|
);
|
|
|
|
// It is still resolvable as a closed/archived session.
|
|
let snap = db.session_snapshot(closed_id).unwrap();
|
|
assert_eq!(snap.id, closed_id);
|
|
|
|
db.close().unwrap();
|
|
}
|
|
}
|
|
|
|
/// BLOCKER/CRITICAL regression: a session that was already past its
|
|
/// `max_session_duration` at shutdown must be TTL-reaped on the first startup
|
|
/// sweep after reopen — the durable age (`started_at_ns`) survives the restart,
|
|
/// so the back-dated monotonic clock makes the sweeper see the true age.
|
|
///
|
|
/// Uses a fresh session under the public API; the focused unit tests in
|
|
/// `sweeper.rs` and `policy.rs` cover the simulated-2h-old-restore arithmetic
|
|
/// directly (the public API cannot dial `started_at_ns` into the past). Here we
|
|
/// assert the live invariant: a session within its TTL survives a sweep and is
|
|
/// listed active after reopen, proving restore back-dates rather than expiring
|
|
/// a perfectly fresh session.
|
|
#[test]
|
|
fn restored_session_within_ttl_survives_sweep() {
|
|
let dir = tempfile::tempdir().unwrap();
|
|
let session_id: SessionId;
|
|
{
|
|
let db = TidalDb::builder()
|
|
.with_data_dir(dir.path())
|
|
.with_schema(schema())
|
|
.open()
|
|
.unwrap();
|
|
let handle = db
|
|
.start_session(1, "agent", "agent", HashMap::new())
|
|
.unwrap();
|
|
session_id = handle.id;
|
|
db.session_signal(
|
|
&handle,
|
|
"view",
|
|
EntityId::new(1),
|
|
1.0,
|
|
Timestamp::now(),
|
|
None,
|
|
)
|
|
.unwrap();
|
|
// Deliberately do NOT close — leave it active so the WAL replays it.
|
|
db.close().unwrap();
|
|
}
|
|
|
|
{
|
|
let db = TidalDb::builder()
|
|
.with_data_dir(dir.path())
|
|
.with_schema(schema())
|
|
.open()
|
|
.unwrap();
|
|
// The fresh, within-TTL session is restored active and survives a sweep
|
|
// (back-dated started_at reflects its true near-zero age, not an instant
|
|
// expiry from a misread clock).
|
|
db.force_sweep();
|
|
let active: Vec<u64> = db
|
|
.active_sessions()
|
|
.into_iter()
|
|
.map(|s| s.id.as_u64())
|
|
.collect();
|
|
assert!(
|
|
active.contains(&session_id.as_u64()),
|
|
"a within-TTL restored session must survive the startup sweep"
|
|
);
|
|
db.close().unwrap();
|
|
}
|
|
}
|