tidaldb/docs/planning/milestone-3/phase-2/task-02-interaction-weight-ledger.md

Task 02: Interaction Weight Ledger

Context

Milestone: 3 -- Personalized Ranking Phase: m3p2 -- Feedback Loop Depends On: Task 01 (entity lookup patterns, preference module structure), m3p1 Task 02 (relationship graph: RelationshipType::InteractionWeight edge), m1p4 (decay infrastructure: DecayModel::Exponential, lambda computation) Blocks: Task 04 (Atomic Signal Dispatch updates interaction weights), m3p3 (Personalized Profiles use interaction weights for scoring) Complexity: M

Objective

Deliver the user-to-creator interaction weight system: a decayed weight representing how strongly a user prefers a particular creator, updated atomically on every engagement signal. The interaction weight uses the same DecayModel::Exponential infrastructure from m1p4, not a separate decay system.

When a user engages with an item (view, like, share, completion), the interaction weight for the user-to-creator pair increases. When a user skips an item, the weight decreases. The weight decays over time using the same exponential formula as signal scores: W(t) = W(t_prev) * exp(-lambda * dt) + delta_weight.

The interaction weight is stored as a RelationshipType::InteractionWeight edge in the users keyspace (established in m3p1 Task 02). This task adds the decay-aware update logic on top of the raw relationship edge storage.

In m3p3, the for_you profile uses interaction weights as a scoring boost: items from creators with higher interaction weights get a score multiplier. This makes the feed naturally favor creators the user has historically engaged with.

Requirements

• InteractionWeightLedger struct: manages in-memory user->creator weights with decay
• ledger.update_weight(user_id, creator_id, delta, timestamp) applies decayed weight update
• ledger.read_weight(user_id, creator_id, now) returns current decayed weight
• ledger.read_top_creators(user_id, n, now) returns top-n creators by decayed weight
• Decay formula: W(t) = W(prev) * exp(-lambda * (t - t_prev)) + delta
• Lambda from configurable half-life (default: 14 days, matching the "like" signal)
• Initial weight for first interaction: delta is added to 0.0
• Delta values: +1.0 for like, +0.5 for view, +2.0 for share, +1.5 for completion, -0.5 for skip
• In-memory cache: DashMap<(u64, u64), InteractionWeightEntry> keyed by (user_id, creator_id)
• Persistence via RelationshipType::InteractionWeight edges in users keyspace
• Checkpoint to storage on TidalDb::close() and periodic checkpoint
• Restore from storage on startup

Technical Design

Module Structure

tidal/src/
  entities/
    interaction.rs -- InteractionWeightLedger, InteractionWeightEntry, config

Core Types

// === entities/interaction.rs ===

use dashmap::DashMap;
use crate::schema::{EntityId, Timestamp};

/// Configuration for interaction weight decay.
#[derive(Debug, Clone)]
pub struct InteractionWeightConfig {
    /// Half-life for interaction weight decay.
    /// Default: 14 days (same as "like" signal).
    pub half_life_secs: f64,
    /// Pre-computed lambda = ln(2) / half_life_secs.
    pub lambda: f64,
    /// Delta weights per signal type.
    pub deltas: InteractionDeltas,
}

impl InteractionWeightConfig {
    pub fn new(half_life_secs: f64) -> Self {
        let lambda = std::f64::consts::LN_2 / half_life_secs;
        Self {
            half_life_secs,
            lambda,
            deltas: InteractionDeltas::default(),
        }
    }
}

impl Default for InteractionWeightConfig {
    fn default() -> Self {
        Self::new(14.0 * 24.0 * 3600.0) // 14 days
    }
}

/// Delta weight values for each signal type.
#[derive(Debug, Clone)]
pub struct InteractionDeltas {
    pub view: f64,
    pub like: f64,
    pub share: f64,
    pub completion: f64,
    pub skip: f64,
}

impl Default for InteractionDeltas {
    fn default() -> Self {
        Self {
            view: 0.5,
            like: 1.0,
            share: 2.0,
            completion: 1.5,
            skip: -0.5,
        }
    }
}

/// In-memory entry for a single user->creator interaction weight.
///
/// Mirrors the `HotSignalState` pattern from m1p4: stores the running
/// decayed score and the timestamp of the last update.
#[derive(Debug, Clone)]
pub struct InteractionWeightEntry {
    /// Running decayed weight. Always >= 0.0 (clamped).
    pub weight: f64,
    /// Timestamp of the last update (nanoseconds).
    pub last_update_ns: u64,
}

impl InteractionWeightEntry {
    /// Compute the current decayed weight at time `now_ns`.
    pub fn current_weight(&self, now_ns: u64, lambda: f64) -> f64 {
        if now_ns <= self.last_update_ns {
            return self.weight;
        }
        let dt_secs = (now_ns - self.last_update_ns) as f64 / 1_000_000_000.0;
        (self.weight * (-lambda * dt_secs).exp()).max(0.0)
    }

    /// Apply a delta to the weight with lazy decay.
    ///
    /// Decays the existing weight to `now_ns`, then adds delta.
    /// Clamps result to >= 0.0.
    pub fn update(&mut self, delta: f64, now_ns: u64, lambda: f64) {
        let decayed = self.current_weight(now_ns, lambda);
        self.weight = (decayed + delta).max(0.0);
        self.last_update_ns = now_ns;
    }
}

/// Manages in-memory user-to-creator interaction weights with decay.
///
/// The ledger is the in-memory hot tier. Weights are persisted as
/// `RelationshipType::InteractionWeight` edges in the users keyspace
/// and restored on startup.
pub struct InteractionWeightLedger {
    /// Per-(user_id, creator_id) weight entries.
    entries: DashMap<(u64, u64), InteractionWeightEntry>,
    /// Decay configuration.
    config: InteractionWeightConfig,
}

impl InteractionWeightLedger {
    pub fn new(config: InteractionWeightConfig) -> Self {
        Self {
            entries: DashMap::new(),
            config,
        }
    }

    /// Update the interaction weight for a user->creator pair.
    ///
    /// Applies lazy decay to the existing weight, then adds the delta.
    /// Creates a new entry if this is the first interaction.
    pub fn update_weight(
        &self,
        user_id: EntityId,
        creator_id: EntityId,
        delta: f64,
        timestamp: Timestamp,
    ) {
        let key = (user_id.as_u64(), creator_id.as_u64());
        let now_ns = timestamp.as_nanos();

        self.entries
            .entry(key)
            .and_modify(|entry| {
                entry.update(delta, now_ns, self.config.lambda);
            })
            .or_insert_with(|| InteractionWeightEntry {
                weight: delta.max(0.0),
                last_update_ns: now_ns,
            });
    }

    /// Read the current decayed weight for a user->creator pair.
    ///
    /// Returns 0.0 if no interaction history exists.
    pub fn read_weight(
        &self,
        user_id: EntityId,
        creator_id: EntityId,
        now: Timestamp,
    ) -> f64 {
        let key = (user_id.as_u64(), creator_id.as_u64());
        self.entries
            .get(&key)
            .map_or(0.0, |entry| {
                entry.current_weight(now.as_nanos(), self.config.lambda)
            })
    }

    /// Read the top-N creators by decayed interaction weight for a user.
    ///
    /// Returns `(creator_id, weight)` pairs sorted by descending weight.
    pub fn read_top_creators(
        &self,
        user_id: EntityId,
        n: usize,
        now: Timestamp,
    ) -> Vec<(EntityId, f64)> {
        let now_ns = now.as_nanos();
        let prefix = user_id.as_u64();

        let mut weights: Vec<(EntityId, f64)> = self.entries
            .iter()
            .filter(|entry| entry.key().0 == prefix)
            .map(|entry| {
                let creator_id = EntityId::new(entry.key().1);
                let weight = entry.current_weight(now_ns, self.config.lambda);
                (creator_id, weight)
            })
            .filter(|(_, w)| *w > f64::EPSILON)
            .collect();

        weights.sort_by(|a, b| b.1.partial_cmp(&a.1).unwrap_or(std::cmp::Ordering::Equal));
        weights.truncate(n);
        weights
    }

    /// Resolve the delta weight for a signal type name.
    pub fn delta_for_signal(&self, signal_type: &str) -> f64 {
        match signal_type {
            "view" => self.config.deltas.view,
            "like" => self.config.deltas.like,
            "share" => self.config.deltas.share,
            "completion" => self.config.deltas.completion,
            "skip" => self.config.deltas.skip,
            _ => 0.0, // Unknown signal types do not affect interaction weight.
        }
    }

    // ── Persistence ───────────────────────────────────────────────────

    /// Checkpoint all interaction weights to storage as relationship edges.
    pub fn checkpoint(
        &self,
        db_writer: &dyn Fn(EntityId, EntityId, f64, u64) -> crate::Result<()>,
    ) -> crate::Result<()> {
        for entry in self.entries.iter() {
            let (user_id, creator_id) = *entry.key();
            let e = entry.value();
            db_writer(
                EntityId::new(user_id),
                EntityId::new(creator_id),
                e.weight,
                e.last_update_ns,
            )?;
        }
        Ok(())
    }

    /// Restore interaction weights from stored relationship edges.
    pub fn restore_entry(
        &self,
        user_id: EntityId,
        creator_id: EntityId,
        weight: f64,
        timestamp_ns: u64,
    ) {
        let key = (user_id.as_u64(), creator_id.as_u64());
        self.entries.insert(key, InteractionWeightEntry {
            weight,
            last_update_ns: timestamp_ns,
        });
    }
}

Test Strategy

Unit Tests

#[test]
fn new_interaction_starts_at_delta() {
    let ledger = InteractionWeightLedger::new(InteractionWeightConfig::default());
    let user = EntityId::new(1);
    let creator = EntityId::new(10);
    let ts = Timestamp::from_nanos(1_000_000_000_000_000_000);

    ledger.update_weight(user, creator, 1.0, ts);
    let weight = ledger.read_weight(user, creator, ts);
    assert!((weight - 1.0).abs() < f64::EPSILON);
}

#[test]
fn weight_decays_over_time() {
    let config = InteractionWeightConfig::new(7.0 * 24.0 * 3600.0); // 7 day half-life
    let ledger = InteractionWeightLedger::new(config.clone());
    let user = EntityId::new(1);
    let creator = EntityId::new(10);
    let t0 = Timestamp::from_nanos(1_000_000_000_000_000_000);

    ledger.update_weight(user, creator, 10.0, t0);

    // After one half-life (7 days), weight should be ~5.0.
    let seven_days_ns = 7 * 24 * 3600 * 1_000_000_000u64;
    let t1 = Timestamp::from_nanos(t0.as_nanos() + seven_days_ns);
    let weight = ledger.read_weight(user, creator, t1);
    assert!((weight - 5.0).abs() < 0.1, "weight after 1 half-life: {}", weight);
}

#[test]
fn weight_accumulates_with_decay() {
    let config = InteractionWeightConfig::default();
    let ledger = InteractionWeightLedger::new(config.clone());
    let user = EntityId::new(1);
    let creator = EntityId::new(10);
    let t0 = Timestamp::from_nanos(1_000_000_000_000_000_000);

    ledger.update_weight(user, creator, 1.0, t0);

    // Second update 1 second later.
    let t1 = Timestamp::from_nanos(t0.as_nanos() + 1_000_000_000);
    ledger.update_weight(user, creator, 1.0, t1);

    let weight = ledger.read_weight(user, creator, t1);
    // Should be slightly less than 2.0 (first event decayed slightly).
    assert!(weight > 1.9, "accumulated weight: {}", weight);
    assert!(weight < 2.0, "accumulated weight: {}", weight);
}

#[test]
fn negative_delta_clamps_to_zero() {
    let ledger = InteractionWeightLedger::new(InteractionWeightConfig::default());
    let user = EntityId::new(1);
    let creator = EntityId::new(10);
    let ts = Timestamp::from_nanos(1_000_000_000_000_000_000);

    ledger.update_weight(user, creator, 0.5, ts);
    ledger.update_weight(user, creator, -10.0, ts); // Large negative
    let weight = ledger.read_weight(user, creator, ts);
    assert!((weight - 0.0).abs() < f64::EPSILON, "clamped to zero: {}", weight);
}

#[test]
fn read_nonexistent_returns_zero() {
    let ledger = InteractionWeightLedger::new(InteractionWeightConfig::default());
    let weight = ledger.read_weight(EntityId::new(1), EntityId::new(99), Timestamp::now());
    assert!((weight - 0.0).abs() < f64::EPSILON);
}

#[test]
fn top_creators_returns_sorted() {
    let ledger = InteractionWeightLedger::new(InteractionWeightConfig::default());
    let user = EntityId::new(1);
    let ts = Timestamp::from_nanos(1_000_000_000_000_000_000);

    ledger.update_weight(user, EntityId::new(10), 5.0, ts);
    ledger.update_weight(user, EntityId::new(20), 10.0, ts);
    ledger.update_weight(user, EntityId::new(30), 1.0, ts);

    let top = ledger.read_top_creators(user, 2, ts);
    assert_eq!(top.len(), 2);
    assert_eq!(top[0].0, EntityId::new(20)); // highest weight
    assert_eq!(top[1].0, EntityId::new(10)); // second highest
}

#[test]
fn top_creators_excludes_zero_weight() {
    let ledger = InteractionWeightLedger::new(InteractionWeightConfig::default());
    let user = EntityId::new(1);
    let ts = Timestamp::from_nanos(1_000_000_000_000_000_000);

    ledger.update_weight(user, EntityId::new(10), 0.0, ts);
    ledger.update_weight(user, EntityId::new(20), 1.0, ts);

    let top = ledger.read_top_creators(user, 10, ts);
    assert_eq!(top.len(), 1);
    assert_eq!(top[0].0, EntityId::new(20));
}

#[test]
fn delta_for_signal_resolves_correctly() {
    let ledger = InteractionWeightLedger::new(InteractionWeightConfig::default());
    assert!((ledger.delta_for_signal("view") - 0.5).abs() < f64::EPSILON);
    assert!((ledger.delta_for_signal("like") - 1.0).abs() < f64::EPSILON);
    assert!((ledger.delta_for_signal("share") - 2.0).abs() < f64::EPSILON);
    assert!((ledger.delta_for_signal("completion") - 1.5).abs() < f64::EPSILON);
    assert!((ledger.delta_for_signal("skip") - (-0.5)).abs() < f64::EPSILON);
    assert!((ledger.delta_for_signal("unknown") - 0.0).abs() < f64::EPSILON);
}

#[test]
fn restore_entry_populates_ledger() {
    let ledger = InteractionWeightLedger::new(InteractionWeightConfig::default());
    ledger.restore_entry(EntityId::new(1), EntityId::new(10), 5.0, 1_000_000_000_000_000_000);
    let ts = Timestamp::from_nanos(1_000_000_000_000_000_000);
    let weight = ledger.read_weight(EntityId::new(1), EntityId::new(10), ts);
    assert!((weight - 5.0).abs() < f64::EPSILON);
}

#[test]
fn different_users_independent() {
    let ledger = InteractionWeightLedger::new(InteractionWeightConfig::default());
    let ts = Timestamp::from_nanos(1_000_000_000_000_000_000);
    ledger.update_weight(EntityId::new(1), EntityId::new(10), 5.0, ts);
    ledger.update_weight(EntityId::new(2), EntityId::new(10), 3.0, ts);

    let w1 = ledger.read_weight(EntityId::new(1), EntityId::new(10), ts);
    let w2 = ledger.read_weight(EntityId::new(2), EntityId::new(10), ts);
    assert!((w1 - 5.0).abs() < f64::EPSILON);
    assert!((w2 - 3.0).abs() < f64::EPSILON);
}

Property Tests

use proptest::prelude::*;

proptest! {
    #[test]
    fn weight_never_negative(
        deltas in proptest::collection::vec(-5.0f64..5.0, 1..20),
        half_life_secs in 3600.0f64..30.0 * 24.0 * 3600.0,
    ) {
        let config = InteractionWeightConfig::new(half_life_secs);
        let ledger = InteractionWeightLedger::new(config);
        let user = EntityId::new(1);
        let creator = EntityId::new(10);

        let mut ts_ns = 1_000_000_000_000_000_000u64;
        for delta in deltas {
            let ts = Timestamp::from_nanos(ts_ns);
            ledger.update_weight(user, creator, delta, ts);
            ts_ns += 1_000_000_000; // 1 second between updates
        }

        let final_ts = Timestamp::from_nanos(ts_ns);
        let weight = ledger.read_weight(user, creator, final_ts);
        prop_assert!(weight >= 0.0, "weight should never be negative: {}", weight);
    }

    #[test]
    fn decay_formula_matches_analytical(
        initial_weight in 0.1f64..100.0,
        dt_secs in 1.0f64..30.0 * 24.0 * 3600.0,
        half_life_secs in 3600.0f64..30.0 * 24.0 * 3600.0,
    ) {
        let lambda = std::f64::consts::LN_2 / half_life_secs;
        let entry = InteractionWeightEntry {
            weight: initial_weight,
            last_update_ns: 0,
        };
        let now_ns = (dt_secs * 1_000_000_000.0) as u64;

        let actual = entry.current_weight(now_ns, lambda);
        let expected = initial_weight * (-lambda * dt_secs).exp();

        prop_assert!((actual - expected).abs() < 1e-6,
            "decay mismatch: actual={}, expected={}", actual, expected);
    }
}

Acceptance Criteria

• InteractionWeightEntry::update applies lazy decay + delta, clamps to >= 0.0
• InteractionWeightEntry::current_weight returns correctly decayed value
• Decay formula matches W(prev) * exp(-lambda * dt) + delta analytically (property tested)
• InteractionWeightLedger::update_weight creates new entries and updates existing ones
• InteractionWeightLedger::read_weight returns decayed weight, 0.0 for nonexistent
• InteractionWeightLedger::read_top_creators returns sorted results, excludes zero-weight
• delta_for_signal maps signal type names to correct delta values
• Weight is always >= 0.0 (property tested across arbitrary delta sequences)
• Different users have independent weight state
• restore_entry correctly populates the in-memory ledger
• checkpoint writes all entries via the provided writer closure
• Lambda computed from half_life as ln(2) / half_life_secs
• cargo clippy -- -D warnings passes
• All tests pass

Research References

• docs/research/tidaldb_signal_ledger.md -- Running decay formula, O(1) lazy decay
• thoughts.md -- Part V.16 (interaction weight as decay-based signal)
• VISION.md -- Relationship weights are first-class

Implementation Notes

• The InteractionWeightLedger uses the same O(1) lazy decay pattern as HotSignalState from m1p4. The weight is stored as a running value with a timestamp. On read, decay is computed lazily. On update, the existing weight is first decayed to the current time, then the delta is added.
• The DashMap<(u64, u64), InteractionWeightEntry> key uses raw u64 values (not EntityId) for efficiency. The user_id and creator_id are extracted from EntityId::as_u64().
• read_top_creators iterates all entries with a matching user_id prefix. For M3 at up to 500 users and ~200 creators, this linear scan is fast enough. For M6+ with larger user bases, a per-user sorted index should be considered.
• The checkpoint method takes a writer closure rather than a direct storage reference. This allows the TidalDb to provide a closure that writes RelationshipType::InteractionWeight edges through its existing API. The closure signature is Fn(user_id, creator_id, weight, timestamp_ns) -> Result<()>.
• Delta values are configurable via InteractionDeltas. The defaults are tuned to make likes worth more than views, shares worth more than likes, and skips a mild negative. These values can be adjusted without code changes by setting them in InteractionWeightConfig.
• Do NOT implement the persistence/checkpoint wiring in this task. The checkpoint and restore_entry methods are public but the actual wiring into TidalDb::close() and TidalDb::open() is done in Task 04 (Atomic Signal Dispatch) where all state management is coordinated.