tidaldb/tidal/docs/planning/milestone-8/phase-3/task-04-crdt-signal-state.md
jx12n 3bcfb3c576 feat: Bazel build, crate docs/ai-lookup, docker images, and engine hardening
- Add BUILD.bazel across tidal, tidal-net, tidal-server, tidalctl for bzlmod build
- Add tidal/ crate docs (README, CHANGELOG, CONTRIBUTING, AGENTS, CLAUDE, API, ARCHITECTURE) and ai-lookup reference
- Add docker standalone/cluster/deploy images, compose, and prometheus config
- Harden WAL (batch format, writer, dedup, diagnostics), text syncer/collectors, and vector registry
- Expand tidalctl CLI and tests; restructure WAL/visibility integration test suites
- Refine tidal-net transport/client/server and tidal-server cluster/scatter-gather
2026-06-07 18:29:38 -06:00

112 lines
4.4 KiB
Markdown

# Task 04: CrdtSignalState
## Delivers
`CrdtSignalState` wrapping `HotSignalState` and `BucketedCounter` with per-node CRDT semantics. Per-node decay accumulators that sum on merge. Per-node bucket arrays that max on merge. Merge produces correct decay scores regardless of order.
## Complexity: L
## Dependencies
- Task 02 (PNCounter)
- Phase 8.1 (ShardId as node identifier)
## Technical Design
The key insight: exponential decay scores are sums of weighted exponentials.
`S_total(t) = sum_i(w_i * exp(-lambda * (t - t_i)))`. Each node maintains its
own running partial sum. On merge, partial sums add (each covers disjoint events
since each node processes distinct WAL segments). This is mathematically exact.
```rust
// tidal/src/replication/crdt/signal_state.rs
/// CRDT-aware signal state for a single entity+signal_type pair.
///
/// Extends the existing HotSignalState and BucketedCounter with per-node
/// accounting that enables correct merge after partitioned writes.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct CrdtSignalState {
/// Per-node running decay score.
///
/// Each node contributes its own partial decay sum.
/// Global score = sum of all node contributions at query time.
node_decay_scores: HashMap<ShardId, f64>,
/// Timestamp of last event per node (for decay math on merge).
node_last_update_ns: HashMap<ShardId, u64>,
/// Per-node windowed counters.
///
/// Each node tracks its own bucket increments.
/// On merge, per-node buckets are merged by taking per-node max
/// (idempotent since same-node events are identical across replicas).
node_buckets: HashMap<ShardId, PNCounter>,
/// Lambda (decay rate) -- identical across all nodes for this signal.
lambda: f64,
}
impl CrdtSignalState {
pub fn new(lambda: f64) -> Self {
Self {
node_decay_scores: HashMap::new(),
node_last_update_ns: HashMap::new(),
node_buckets: HashMap::new(),
lambda,
}
}
/// Record a new signal event from `node`.
pub fn on_signal(&mut self, node: ShardId, weight: f64, now_ns: u64) {
let entry = self.node_decay_scores.entry(node).or_default();
let last = self.node_last_update_ns.entry(node).or_insert(now_ns);
// Decay existing score, then add new event weight.
let dt = (now_ns.saturating_sub(*last)) as f64 / 1e9;
*entry = *entry * (-self.lambda * dt).exp() + weight;
*last = now_ns;
}
/// Global decay score: sum of all per-node contributions at `now_ns`.
pub fn decay_score(&self, now_ns: u64) -> f64 {
self.node_decay_scores.iter()
.zip(self.node_last_update_ns.values())
.map(|((_, &score), &last)| {
let dt = (now_ns.saturating_sub(last)) as f64 / 1e9;
score * (-self.lambda * dt).exp()
})
.sum()
}
/// Merge another CrdtSignalState into this one.
///
/// Per-node scores are summed (each node contributes distinct events).
/// Per-node buckets are merged via PNCounter merge (per-node max).
pub fn merge(&mut self, other: &CrdtSignalState) {
for (&node, &other_score) in &other.node_decay_scores {
*self.node_decay_scores.entry(node).or_default() += other_score;
}
for (&node, &other_ts) in &other.node_last_update_ns {
let entry = self.node_last_update_ns.entry(node).or_default();
*entry = (*entry).max(other_ts);
}
for (node, other_bucket) in &other.node_buckets {
self.node_buckets
.entry(*node)
.or_default()
.merge(other_bucket);
}
}
}
```
## Acceptance Criteria
- [ ] `CrdtSignalState::decay_score(now_ns)` returns sum of all per-node contributions decayed to `now_ns`
- [ ] Two nodes process 500 events each (non-overlapping); after merge, `decay_score` == sum of both individual scores (property test: 1000 random event sequences)
- [ ] `merge` is commutative and associative (property tests)
- [ ] `merge` does not double-count: same-node events produce the same score regardless of how many times the node's state is merged (idempotent per node)
- [ ] `BucketedCounter` equivalent: per-node bucket increments merged by PNCounter; total windowed count = sum of distinct events across all nodes; no double-counting
- [ ] `cargo clippy -D warnings` and `cargo fmt` pass