---
title: "The feedback loop that closes in one write"
date: "2026-02-21"
author: "Jordan Washburn"
description: "When a user likes an item, tidalDB atomically updates the item's signal ledger, the user-to-creator interaction weight, and the user's preference vector. One db.signal_with_context() call. No Kafka consumer to lag. No feature store to sync. The next ranking query reflects the change."
tags: ["signals", "personalization", "architecture", "feedback-loop"]
---

Here is the feedback loop in the 6-system stack:

1. User likes a video.
2. The application publishes a `like` event to Kafka.
3. A Kafka consumer reads the event (median lag: 200ms, p99: 4 seconds).
4. The consumer increments a Redis counter for the item's like count.
5. A separate consumer updates the feature store with the user's latest engagement vector.
6. A third consumer updates the user-to-creator interaction weight in a graph database.
7. A cron job recomputes the user's preference embedding from the feature store every 15 minutes.
8. The next ranking query reads stale data from at least one of those systems.

Between step 1 and the moment all downstream systems converge, there is a window where the user's feed does not reflect what they just did. They liked a jazz piano video. The next feed refresh shows them the same content it would have shown before the like. The preference vector has not updated. The interaction weight has not changed. The like count is still the old value in Elasticsearch because the cron job that syncs Redis to Elasticsearch runs every 5 minutes.

This window is where trust erodes. The user did something. The system did not respond. The feedback loop is open.

## What closes the loop

In tidalDB, a single function call does everything:

```rust
db.signal_with_context(
    "like",           // signal type
    item_id,          // the item being liked
    1.0,              // weight
    Timestamp::now(), // timestamp
    Some(user_id),    // user context
    Some(creator_id), // creator of the item
)?;
```

That call triggers up to four operations atomically, within the same process, within the same memory space:

1. **Item signal ledger** -- the item's like decay score updates via the O(1) forward-decay formula. Windowed counters increment. Velocity recomputes.
2. **Seen tracking** -- the item is marked as seen in the user's `RoaringBitmap`. Future `FOR USER` queries exclude it automatically.
3. **Interaction weight** -- the user-to-creator interaction weight increments by the caller's `weight` argument. The weight uses the same lazy exponential decay as signal scores. Items from this creator will rank higher in the user's next `for_you` query.
4. **Preference vector** -- for positive engagement signals only (like, share, completion): if the item has a stored embedding, the user's preference vector shifts toward it via exponential moving average. View signals skip this step -- they are low-intent and do not shift taste.

For a `like`, all four updates complete before the function returns. For a `view`, the first three fire but the preference vector is unchanged. Either way, the next ranking query -- even one issued 100 milliseconds later -- sees the updated state.

## The four targets

### 1. Item signal ledger

This is the update path from the [signal engine](/blog/running-decay-scores-are-o1). The running decay score updates in O(1):

```
S(t) = S(t_prev) * exp(-lambda * dt) + weight
```

One `exp()` call, one multiply-add. The windowed counter (bucketed circular buffer) increments the current minute bucket. The all-time counter increments. No raw event scanning. No batch aggregation. The score is current because the score is computed, not cached.

### 2. Seen tracking

The user state index maintains a `RoaringBitmap` per user of all item IDs they have interacted with:

```rust
// From the signal dispatch path
self.user_state.mark_seen(user_id, item_u32);
```

Bitmap insertion is O(1). The `FOR USER` clause in a `RETRIEVE` query intersects this bitmap with the candidate set during the filter stage. Seen items are removed before scoring. The user never sees the same item twice unless they explicitly request it.

### 3. Interaction weight

The `InteractionLedger` tracks per-(user, creator) interaction strength using the same lazy decay formula as signal scores:

```rust
pub fn record(&self, user_id: u64, creator_id: u64, weight: f64, timestamp_ns: u64) {
    let user_map = self.inner.entry(user_id).or_default();
    let mut entry = user_map.entry(creator_id).or_insert(InteractionEntry {
        score: 0.0,
        last_update_ns: timestamp_ns,
    });

    let dt_secs = if timestamp_ns > entry.last_update_ns {
        (timestamp_ns - entry.last_update_ns) as f64 / 1_000_000_000.0
    } else {
        0.0
    };

    // Lazy decay + accumulate.
    entry.score = entry.score.mul_add((-self.lambda * dt_secs).exp(), weight);
    entry.last_update_ns = timestamp_ns;
}
```

The structure is a nested `DashMap`: outer key is `user_id`, inner key is `creator_id`. This makes `top_creators(user_id)` a scan over that user's creators only -- O(M) where M is the number of distinct creators the user has interacted with, not O(N*M) over all pairs.

The `weight` argument is passed directly by the caller -- `signal_with_context` does not look up a weight table automatically. The `InteractionWeights` struct exists as a recommended-defaults helper for applications that want conventional scaling:

| Signal | Recommended weight |
|--------|-------------------|
| `view` | 1.0 |
| `completion` | 2.0 |
| `like` | 3.0 |
| `save` | 4.0 |
| `share` | 5.0 |

These are conventions, not automatic mappings. The application chooses what weight to pass. A user who shares 3 videos from a creator (at weight 5.0 each) builds a stronger interaction weight than a user who views 10 (at weight 1.0 each). The decay half-life is 7 days by default -- recent interactions matter more than old ones, but a creator the user engaged with heavily a month ago still carries residual weight.

At query time, the `for_you` profile reads the interaction weight and applies it as a scoring boost. Items from high-weight creators rank higher. The boost is proportional to the decayed interaction score, not a binary threshold. The ranking is a gradient, not a gate.

### 4. Preference vector

The preference vector is the user's taste, represented as a 128-dimensional embedding that evolves with every positive engagement:

```rust
// From tidal/src/entities/preference.rs
pub fn update(&self, user_id: u64, interaction_embedding: &[f32]) -> bool {
    let lr = self.learning_rate;
    match self.inner.entry(user_id) {
        Entry::Occupied(mut occ) => {
            let pref = occ.get_mut();
            for (p, &i) in pref.iter_mut().zip(interaction_embedding.iter()) {
                *p = (1.0 - lr).mul_add(*p, lr * i);
            }
            l2_normalize(pref);
        }
        Entry::Vacant(vac) => {
            // Cold start: first interaction becomes the initial preference.
            let mut v = interaction_embedding.to_vec();
            l2_normalize(&mut v);
            vac.insert(v);
        }
    }
    // ...
}
```

The update is an exponential moving average: the new preference is `(1 - lr) * current + lr * item_embedding`, then L2-normalized back to unit length. The default learning rate is 0.1 -- each interaction contributes 10% to the updated preference.

Cold-start users have no preference vector. The first positive interaction initializes it from the item's embedding. From that point forward, every like, share, and completion shifts the vector toward the engaged content. A user who likes three jazz piano videos and two cooking tutorials will have a preference vector that sits in the embedding space between jazz piano and cooking, weighted toward whatever they interacted with most recently.

The normalization invariant is critical. The preference vector is always unit length, which means cosine similarity via dot product works without rescaling. When the personalized profile uses this vector as an ANN query against the item embedding index, the distance computation is a single SIMD dot product per candidate. No normalization at query time. No score calibration. The geometry is correct by construction.

## The dispatch

The signal dispatch is a branch on signal type. Positive engagement signals (like, share, completion) trigger all four updates including the preference vector. View signals trigger the first three (signal ledger, seen tracking, interaction weight) but not the preference vector -- views are low-intent and do not shift the user's taste representation. Hard negatives (skip, hide, dislike, block) trigger exclusion. The branching happens in `signal_with_context`:

```rust
// Record the base signal (item ledger, WAL, windowed counters).
self.signal(signal_type, entity_id, weight, timestamp)?;

if let Some(user_id) = for_user {
    // 1. Hard negatives: skip/hide/dislike/block -> exclusion bitmap.
    if HardNegIndex::is_hard_neg_signal(signal_type) {
        self.hard_negatives.add(user_id, item_u32);
    }

    // 2. Seen tracking.
    self.user_state.mark_seen(user_id, item_u32);

    // 3. Interaction weight.
    if let Some(cid) = creator_id {
        self.interaction_ledger
            .record(user_id, cid, weight, timestamp.as_nanos());
    }

    // 4. Preference vector (positive signals only).
    if is_positive_engagement_signal(signal_type) {
        self.try_update_preference_vector(user_id, entity_id);
    }
}
```

The base signal write goes through the same WAL-first path as every other signal: hash for deduplication, append to WAL, update in-memory decay score, update windowed counter. The user-context side effects are additional in-memory operations on top of that. No second WAL write. No additional disk I/O. The overhead is the cost of a `DashMap` insertion (seen tracking), a `DashMap` update with one `exp()` call (interaction weight), and a 128-element vector blend with L2 normalization (preference vector).

When no user context is provided -- `for_user: None` -- the dispatch skips all user-level updates. The signal is item-level only. This preserves backward compatibility with the signal write API from earlier milestones. The application chooses when to provide user context. The database handles the rest.

## The immediate visibility test

The acceptance test writes signals with user context and immediately queries:

```rust
// User views items from creator 100.
for i in 1..=3u64 {
    db.signal_with_context(
        "view", EntityId::new(i), 3.0, ts,
        Some(user_id), Some(100),
    )?;
}

// User blocks creator 300 — all items from this creator are excluded from future queries.
db.write_relationship(
    EntityId::new(user_id),
    RelationshipType::Blocks,
    EntityId::new(300),
    1.0, ts,
)?;

// User skips item 5 — added to the hard-negative bitmap.
db.signal_with_context(
    "skip", EntityId::new(5), 1.0, ts,
    Some(user_id), None,
)?;

// Query immediately — no consumer lag, no cache to invalidate.
let query = RetrieveBuilder::new(EntityKind::Item, ProfileRef::new("new"))
    .for_user(user_id)
    .limit(20)
    .build()?;
let results = db.retrieve(&query)?;

// Results contain only unseen, unblocked, non-negative items.
// Items from creator 300 are absent. Item 5 is absent. Viewed items 1–3 are absent.
```

There is no delay between the signal writes and the query. No background consumer to wait for. No cache to invalidate. No eventual consistency window. The signal updates in-memory state. The query reads in-memory state. They share a process and a memory space. The loop closes in the time it takes to execute the function call.

The interaction ledger test verifies the decay formula in isolation:

```rust
// Record interaction with weight 10.0.
il.record(1, 100, 10.0, base_ns);
let score_now = il.score(1, 100, base_ns);
// score_now ≈ 10.0 — immediate, no decay elapsed.

// After one half-life (7 days): score halves.
let one_week_later = base_ns + 7 * 24 * 3600 * 1_000_000_000;
let score_later = il.score(1, 100, one_week_later);
// score_later ≈ 5.0 — same O(1) lazy decay as the signal ledger.
```

Same O(1) lazy decay as the signal ledger. Same formula. Same correctness guarantees. Different data.

## What this eliminates

Here is the dependency graph for a feedback loop in the 6-system stack:

```
User likes a video
  -> Application publishes to Kafka
       -> Consumer 1: increment Redis like counter
       -> Consumer 2: update feature store (user engagement features)
       -> Consumer 3: update graph DB (user-creator edge weight)
  -> Cron job (every 15 min): recompute user preference embedding
  -> Cron job (every 5 min): sync Redis counters to Elasticsearch
  -> Next ranking query:
       reads preference vector (up to 15 min stale)
       reads like count (up to 5 min stale)
       reads interaction weight (up to 4 sec consumer lag)
```

Three Kafka consumers. Two cron jobs. Three consistency models. A preference vector that can be 15 minutes out of date. A like count that can be 5 minutes out of date. An interaction weight that depends on consumer group lag. And every seam between these systems is a place where a bug can hide, where a consumer can fall behind, where a cron job can fail silently, where data can diverge.

Here is the same feedback loop in tidalDB:

```
User likes a video
  -> db.signal_with_context("like", item_id, 1.0, ts, Some(user), Some(creator))
       -> item like decay score updated (O(1))
       -> item windowed counter incremented
       -> user seen bitmap updated
       -> user->creator interaction weight updated (O(1) lazy decay)
       -> user preference vector shifted toward item embedding (EMA)
  -> Next ranking query:
       reads all updated state from the same memory space
```

One function call. One process. One consistency model. The data is never stale because the write path and the read path share the same data structures. There is no consumer to lag. There is no cron job to wait for. There is no cache to invalidate. The loop closes in the time it takes to execute an `exp()` call and a vector blend.

## What is honest about this

The preference vector and interaction weight updates are in-memory. They survive within the process lifetime. If the process crashes, the base signal survives via WAL replay, but the user-level side effects are reconstructed from durable storage on restart -- the interaction ledger rebuilds from stored relationship edges, the hard negative index rebuilds from relationship records. The preference vector, being updated on every positive engagement, converges back to its pre-crash state as signals replay.

This is a deliberate choice. The base signal is the source of truth. The WAL is the durability boundary. Everything else is derived state that can be reconstructed. The same architectural principle that makes the signal ledger correct after a crash makes the entire feedback loop correct after a crash -- at the cost of a restart that replays events.

The seen set for regular views is intentionally ephemeral. Users should see content again after a restart. Only explicit hides and blocks are durably written via relationship edges that survive crashes. This is not a limitation. It is a decision about what "seen" means.

---

The 6-system stack's feedback loop is open because it is distributed. Events traverse network boundaries, consumer groups, batch jobs, and cache layers before they converge into a consistent view. Every boundary is a place where latency accumulates and correctness degrades.

tidalDB's feedback loop is closed because it is embedded. The signal write and the ranking query share a process, a memory space, and a set of data structures. When the user acts, the system responds. Not eventually. Now.

---

*The signal dispatch is at [tidal/src/db/mod.rs](https://github.com/orchard9/tidalDB/blob/main/tidal/src/db/mod.rs). The interaction ledger is at [tidal/src/entities/interaction.rs](https://github.com/orchard9/tidalDB/blob/main/tidal/src/entities/interaction.rs). The preference vectors are at [tidal/src/entities/preference.rs](https://github.com/orchard9/tidalDB/blob/main/tidal/src/entities/preference.rs). The feedback loop tests are at [tidal/tests/](https://github.com/orchard9/tidalDB/tree/main/tidal/tests). Follow the build on [GitHub](https://github.com/orchard9/tidalDB).*