Reconciles two independently-developed lines from base 006d3d0:
ours — M9/M10 community layers, retroactive purge + re-materialization,
signal revocation, agent capability boundaries, P1 feedback loop,
reason labels, instrumented metrics
theirs — M11/M12 cluster mode (tidal-net gRPC transport, tidal-server
cluster/scatter-gather, tidal-stress), multi-vector preference,
ANN candidate-gen, warm-tier day buckets, keyed signal snapshots
Notable semantic resolutions:
* storage::keys::Tag — both sides allocated 0x0E..0x11 for different
records. Kept theirs' 0x0E..0x1A (shipped on-disk format) and renumbered
ours to 0x1B..0x1E (CommunityMembership/Revocation/PurgeManifest/
CommunityLeave); Tag::ALL grown to 30 so the contiguity drift guard holds.
* ranking executor — took theirs' rewrite (SignalReadPlan pre-pass, keyed
SignalKey snapshots, Result-returning reads, finalize()) and re-applied
ours' M10 read suppression at the chokepoints it introduced:
single_signal_score, score_hot/trending/controversial,
CreatorEngagementRate, and the Stage-4 boost loop.
* signals::warm — theirs' day-bucket/read-time-rotation rewrite, with ours'
subtract_bucket and Clone extended to the new day tier; ours' test split
kept (warm/tests.rs, warm/proptests.rs) carrying theirs' updated bodies.
* db::signals — kept ours' contribution-logging try_cohort_attribution in
signal_dispatch.rs and theirs' event-time try_update_preference_vector;
dropped the superseded duplicates.
* db::mod / from_parts — theirs' constructors, with ours' purge/
re-materialization/revocation/community/skip-counter fields and restart
rebuilds; from_parts kept in its own file per the 600-line guideline.
* schema::validation::builders — ours' module split with theirs' expanded
tests; policy validation runs both sides' checks (read-signal lists +
profile overrides, then the zero-duration limit guard).
* feedback Unhide no longer writes a -1.0 "hide" signal: theirs' engine
rejects negative weights (spec §8). Reverses index state only, matching
every other undo action.
* SessionState::new is now the single construction path (gains
overrides_rejected/default_profile); AuditEntry gains kind on the
deserialize path, inferred from the accepted flag as before.
* Removed tidal/src/replication/tcp_transport.rs and its test: never
declared in replication/mod.rs on either branch, so it had never
compiled and nothing referenced it. Superseded by tidal-net's
GrpcTransport.
Verified: cargo clippy -p tidaldb (lib) clean; --all-targets compiles for
tidaldb/tidal-net/tidal-server/tidal-stress; 2094/2094 lib tests and the
integration suite pass except m8p3_reconcile_production's two CRDT-count
assertions, which fail identically on MERGE_HEAD (pre-existing).
tidalctl cannot build locally: its aws-sdk deps need rustc 1.91.1, local
toolchain is 1.91.0.
318 lines
11 KiB
Rust
318 lines
11 KiB
Rust
//! M9 Community Profile Sync Integration Tests.
|
|
//!
|
|
//! Tests the opt-in community personalization layer:
|
|
//! - join_community_layer / is_community_member / get_community_memberships
|
|
//! - Signal forwarding to community aggregates via signal_with_context
|
|
//! - Membership persistence across restart
|
|
//! - Validation of community name constraints
|
|
|
|
#![allow(clippy::unwrap_used)]
|
|
|
|
use std::time::Duration;
|
|
|
|
use tidaldb::TidalDb;
|
|
use tidaldb::schema::{DecaySpec, EntityId, EntityKind, SchemaBuilder, Timestamp, Window};
|
|
|
|
// ── Schema ──────────────────────────────────────────────────────────────────
|
|
|
|
fn community_schema() -> tidaldb::schema::Schema {
|
|
let mut builder = SchemaBuilder::new();
|
|
let _ = builder
|
|
.signal(
|
|
"view",
|
|
EntityKind::Item,
|
|
DecaySpec::Exponential {
|
|
half_life: Duration::from_secs(7 * 24 * 3600),
|
|
},
|
|
)
|
|
.windows(&[Window::OneHour, Window::TwentyFourHours, Window::AllTime])
|
|
.velocity(true)
|
|
.add();
|
|
let _ = builder
|
|
.signal(
|
|
"like",
|
|
EntityKind::Item,
|
|
DecaySpec::Exponential {
|
|
half_life: Duration::from_secs(14 * 24 * 3600),
|
|
},
|
|
)
|
|
.windows(&[Window::AllTime])
|
|
.velocity(false)
|
|
.add();
|
|
builder.build().expect("community schema must be valid")
|
|
}
|
|
|
|
fn open_ephemeral() -> TidalDb {
|
|
TidalDb::builder()
|
|
.ephemeral()
|
|
.with_schema(community_schema())
|
|
.open()
|
|
.expect("ephemeral open")
|
|
}
|
|
|
|
// ── Test 1: join_and_query_membership ────────────────────────────────────────
|
|
|
|
#[test]
|
|
fn join_and_query_membership() {
|
|
let db = open_ephemeral();
|
|
|
|
assert!(!db.is_community_member(1, "jazz").unwrap());
|
|
assert_eq!(
|
|
db.get_community_memberships(1).unwrap(),
|
|
Vec::<String>::new()
|
|
);
|
|
|
|
db.join_community_layer(1, "jazz").unwrap();
|
|
|
|
assert!(db.is_community_member(1, "jazz").unwrap());
|
|
assert!(!db.is_community_member(1, "blues").unwrap()); // different community
|
|
assert!(!db.is_community_member(2, "jazz").unwrap()); // different user
|
|
|
|
let memberships = db.get_community_memberships(1).unwrap();
|
|
assert_eq!(memberships, vec!["jazz"]);
|
|
}
|
|
|
|
// ── Test 2: signal_routes_to_community_aggregate ─────────────────────────────
|
|
|
|
#[test]
|
|
fn signal_routes_to_community_aggregate() {
|
|
let db = open_ephemeral();
|
|
let user_id = 10u64;
|
|
let item_id = EntityId::new(100);
|
|
|
|
db.join_community_layer(user_id, "jazz").unwrap();
|
|
|
|
let ts = Timestamp::now();
|
|
db.signal_with_context("view", item_id, 1.0, ts, Some(user_id), None)
|
|
.unwrap();
|
|
|
|
// Global ledger should have the signal.
|
|
let global_score = db.read_decay_score(item_id, "view", 0).unwrap();
|
|
assert!(global_score.is_some(), "global ledger must have the signal");
|
|
assert!(global_score.unwrap() > 0.0);
|
|
|
|
// Community aggregate should also have the signal.
|
|
let community_count = db
|
|
.cohort_ledger()
|
|
.read_windowed_count("community::jazz", item_id, "view", Window::AllTime)
|
|
.unwrap();
|
|
assert_eq!(
|
|
community_count, 1,
|
|
"community aggregate must reflect the forwarded signal"
|
|
);
|
|
}
|
|
|
|
// ── Test 3: nonmember_signal_not_forwarded ───────────────────────────────────
|
|
|
|
#[test]
|
|
fn nonmember_signal_not_forwarded() {
|
|
let db = open_ephemeral();
|
|
let nonmember_id = 20u64;
|
|
let item_id = EntityId::new(200);
|
|
|
|
// User 20 never joins any community.
|
|
let ts = Timestamp::now();
|
|
db.signal_with_context("view", item_id, 1.0, ts, Some(nonmember_id), None)
|
|
.unwrap();
|
|
|
|
// Global ledger should have the signal.
|
|
let global_score = db.read_decay_score(item_id, "view", 0).unwrap();
|
|
assert!(global_score.is_some());
|
|
|
|
// Community aggregate must NOT have any signal (user was not a member).
|
|
let community_count = db
|
|
.cohort_ledger()
|
|
.read_windowed_count("community::jazz", item_id, "view", Window::AllTime)
|
|
.unwrap();
|
|
assert_eq!(
|
|
community_count, 0,
|
|
"non-member signal must not appear in community aggregate"
|
|
);
|
|
}
|
|
|
|
// ── Test 4: multiple_communities_independent ─────────────────────────────────
|
|
|
|
#[test]
|
|
fn multiple_communities_independent() {
|
|
let db = open_ephemeral();
|
|
let user_id = 30u64;
|
|
let item_id = EntityId::new(300);
|
|
|
|
db.join_community_layer(user_id, "jazz").unwrap();
|
|
db.join_community_layer(user_id, "blues").unwrap();
|
|
|
|
let memberships = db.get_community_memberships(user_id).unwrap();
|
|
assert_eq!(memberships, vec!["blues", "jazz"]); // sorted
|
|
|
|
let ts = Timestamp::now();
|
|
db.signal_with_context("view", item_id, 1.0, ts, Some(user_id), None)
|
|
.unwrap();
|
|
|
|
// Both community aggregates should have the signal.
|
|
let jazz_count = db
|
|
.cohort_ledger()
|
|
.read_windowed_count("community::jazz", item_id, "view", Window::AllTime)
|
|
.unwrap();
|
|
let blues_count = db
|
|
.cohort_ledger()
|
|
.read_windowed_count("community::blues", item_id, "view", Window::AllTime)
|
|
.unwrap();
|
|
|
|
assert_eq!(jazz_count, 1, "jazz aggregate must have the signal");
|
|
assert_eq!(blues_count, 1, "blues aggregate must have the signal");
|
|
}
|
|
|
|
// ── Test 5: membership_persists_across_restart ───────────────────────────────
|
|
|
|
#[test]
|
|
fn membership_persists_across_restart() {
|
|
let dir = tempfile::tempdir().unwrap();
|
|
let schema = community_schema();
|
|
|
|
// First open: join community.
|
|
{
|
|
let db = TidalDb::builder()
|
|
.with_data_dir(dir.path())
|
|
.with_schema(schema.clone())
|
|
.open()
|
|
.unwrap();
|
|
|
|
db.join_community_layer(1, "jazz").unwrap();
|
|
db.join_community_layer(1, "blues").unwrap();
|
|
assert!(db.is_community_member(1, "jazz").unwrap());
|
|
|
|
db.close().unwrap();
|
|
}
|
|
|
|
// Second open: memberships should be loaded from storage.
|
|
{
|
|
let db = TidalDb::builder()
|
|
.with_data_dir(dir.path())
|
|
.with_schema(schema)
|
|
.open()
|
|
.unwrap();
|
|
|
|
let memberships = db.get_community_memberships(1).unwrap();
|
|
assert_eq!(
|
|
memberships,
|
|
vec!["blues", "jazz"],
|
|
"memberships must survive restart"
|
|
);
|
|
assert!(db.is_community_member(1, "jazz").unwrap());
|
|
assert!(db.is_community_member(1, "blues").unwrap());
|
|
assert!(!db.is_community_member(1, "rock").unwrap());
|
|
|
|
db.close().unwrap();
|
|
}
|
|
}
|
|
|
|
// ── Test 6: join_idempotent ───────────────────────────────────────────────────
|
|
|
|
#[test]
|
|
fn join_idempotent() {
|
|
let db = open_ephemeral();
|
|
|
|
db.join_community_layer(1, "jazz").unwrap();
|
|
db.join_community_layer(1, "jazz").unwrap(); // second call — idempotent
|
|
|
|
let memberships = db.get_community_memberships(1).unwrap();
|
|
assert_eq!(memberships.len(), 1, "idempotent join must not duplicate");
|
|
assert_eq!(memberships[0], "jazz");
|
|
}
|
|
|
|
// ── Test 7: invalid_community_name_empty ─────────────────────────────────────
|
|
|
|
#[test]
|
|
fn invalid_community_name_empty() {
|
|
let db = open_ephemeral();
|
|
let result = db.join_community_layer(1, "");
|
|
assert!(result.is_err(), "empty community name must return an error");
|
|
let err = result.unwrap_err();
|
|
assert!(
|
|
matches!(err, tidaldb::schema::TidalError::InvalidInput { .. }),
|
|
"expected InvalidInput, got {err:?}"
|
|
);
|
|
}
|
|
|
|
// ── Test 8: invalid_community_name_too_long ──────────────────────────────────
|
|
|
|
#[test]
|
|
fn invalid_community_name_too_long() {
|
|
let db = open_ephemeral();
|
|
let long_name = "a".repeat(65); // 65 bytes, exceeds 64-byte limit
|
|
let result = db.join_community_layer(1, &long_name);
|
|
assert!(
|
|
result.is_err(),
|
|
"65-byte community name must return an error"
|
|
);
|
|
let err = result.unwrap_err();
|
|
assert!(
|
|
matches!(err, tidaldb::schema::TidalError::InvalidInput { .. }),
|
|
"expected InvalidInput, got {err:?}"
|
|
);
|
|
}
|
|
|
|
// ── Test 9: forwarding_does_not_block_primary_write ──────────────────────────
|
|
|
|
#[test]
|
|
fn forwarding_does_not_block_primary_write() {
|
|
// This test verifies that the primary signal write path is unaffected by
|
|
// community state. The community_membership index is always populated, but
|
|
// even if forwarding encountered any issue, the primary write must succeed.
|
|
let db = open_ephemeral();
|
|
let user_id = 99u64;
|
|
let item_id = EntityId::new(999);
|
|
|
|
db.join_community_layer(user_id, "jazz").unwrap();
|
|
|
|
let ts = Timestamp::now();
|
|
let result = db.signal_with_context("view", item_id, 1.0, ts, Some(user_id), None);
|
|
assert!(
|
|
result.is_ok(),
|
|
"signal_with_context must succeed for community member"
|
|
);
|
|
|
|
// Primary ledger is updated.
|
|
let score = db.read_decay_score(item_id, "view", 0).unwrap();
|
|
assert!(score.is_some(), "primary ledger must have the signal");
|
|
assert!(score.unwrap() > 0.0);
|
|
}
|
|
|
|
// ── Test 10: community_aggregate_windowed_count ───────────────────────────────
|
|
|
|
#[test]
|
|
fn community_aggregate_windowed_count() {
|
|
let db = open_ephemeral();
|
|
let user_id = 50u64;
|
|
let item_id = EntityId::new(500);
|
|
|
|
db.join_community_layer(user_id, "tech").unwrap();
|
|
|
|
let ts = Timestamp::now();
|
|
// Write 3 signals.
|
|
for _ in 0..3 {
|
|
db.signal_with_context("view", item_id, 1.0, ts, Some(user_id), None)
|
|
.unwrap();
|
|
}
|
|
|
|
let count = db
|
|
.cohort_ledger()
|
|
.read_windowed_count("community::tech", item_id, "view", Window::AllTime)
|
|
.unwrap();
|
|
assert_eq!(
|
|
count, 3,
|
|
"community aggregate must count all 3 forwarded signals"
|
|
);
|
|
|
|
// A second item from the same user also gets forwarded.
|
|
let item2_id = EntityId::new(501);
|
|
db.signal_with_context("like", item2_id, 1.0, ts, Some(user_id), None)
|
|
.unwrap();
|
|
|
|
let like_count = db
|
|
.cohort_ledger()
|
|
.read_windowed_count("community::tech", item2_id, "like", Window::AllTime)
|
|
.unwrap();
|
|
assert_eq!(like_count, 1, "like signal must be forwarded to community");
|
|
}
|