# Milestone 1, Phase 4: Signal Ledger -- Decay Scores and Windowed Aggregation

## Phase Deliverable

The in-memory per-entity signal state: running exponential decay scores with O(1) update and O(1) read, bucketed windowed counters for 1h/24h/7d aggregate queries, raw velocity computation, and checkpoint/restore for crash recovery. This is the core temporal engine that makes signals a database primitive instead of application math.

## Acceptance Criteria

- [ ] `HotSignalState` is `#[repr(C, align(64))]` -- one L1 cache line per signal type per entity
- [ ] Running decay formula `S(t) = S(t_prev) * exp(-lambda * dt) + weight` is mathematically exact, verified against analytical brute-force computation to 6 decimal places across 10,000 random event sequences (property test P2)
- [ ] Out-of-order events handled correctly: when `t_event < last_update`, weight is pre-decayed: `score += weight * exp(-lambda * (last_update - t_event))` -- no timestamp regression
- [ ] Decay scores monotonically decrease without new events (property test P1)
- [ ] Decay scores are always non-negative (invariant INV-SIG-3)
- [ ] Windowed counts use `BucketedCounter` with per-minute buckets (60) and per-hour buckets (168), supporting 1h/24h/7d windows via bucket summation
- [ ] Velocity = `windowed_count / window_duration_seconds` -- raw velocity for all configured windows
- [ ] `SignalLedger` coordinates hot and warm tiers with `DashMap<(EntityId, SignalTypeId), _>` for concurrent access
- [ ] State checkpointed to `StorageEngine` via `Tag::Sig`; restore from checkpoint reconstructs exact state
- [ ] Property tests P1-P4 pass: monotonic decrease, analytical match, windowed count correctness, out-of-order commutativity

## Dependencies

- **Requires:** m1p1 (types: `EntityId`, `Timestamp`, `DecayModel`, `Window`, `WindowSet`, `SignalTypeDef`), m1p2 (WAL: `WalEvent` type for replay interface -- m1p4 defines the `WalWriter` trait but does NOT implement WAL; the trait is a dependency boundary), m1p3 (storage: `StorageEngine` trait, `Tag::Sig`, key encoding for checkpoint persistence)
- **Blocks:** m1p5 (Entity CRUD and Signal Write API)

## Research References

- [docs/research/tidaldb_signal_ledger.md](../../../research/tidaldb_signal_ledger.md) -- three-tier architecture, running-score formula proof, BucketedCounter design, EntityState struct (~128 bytes), performance estimates (~36ns write, ~15ns read), Scotty stream-slicing approach
- [thoughts.md](../../../../thoughts.md) -- Part V.5 (quarantine-first signal ingestion), Part V.6 (group commit), Part V.14 (cache-line alignment for hot-path structs)

## Spec References

- [docs/specs/03-signal-system.md](../../../specs/03-signal-system.md) -- HotSignalState layout (Section 3), decay computation (Section 4), velocity computation (Section 5), windowed aggregation (Section 6), write path (Section 8), invariants INV-SIG-1 through INV-SIG-5, INV-CON-1 through INV-CON-3, property tests P1-P4, performance targets (Section 12)
- [docs/specs/00-architecture-overview.md](../../../specs/00-architecture-overview.md) -- Materializer trait (`on_event`, `checkpoint`, `restore`), signal write walkthrough (Section 5), code module map showing `signal/hot.rs`, `signal/warm.rs`

## Task Index

| # | Task | Delivers | Depends On | Complexity |
|---|------|----------|------------|------------|
| 01 | Hot-Tier Signal State | `HotSignalState`, atomic decay score CAS, out-of-order handling, lazy read-time decay | None | L |
| 02 | Warm-Tier Bucketed Counters | `BucketedCounter`, per-minute/per-hour buckets, windowed count queries, all-time counter | None | M |
| 03 | Signal Ledger and Velocity | `SignalLedger` coordinating hot+warm, DashMap concurrent access, velocity computation, `WalWriter` trait boundary | Task 01, Task 02 | L |
| 04 | Checkpoint and Restore | Serialization of hot+warm state to `StorageEngine`, restore from checkpoint, integration with key encoding | Task 03 | M |

## Task Dependency DAG

```
Task 01: Hot-Tier Signal State      Task 02: Warm-Tier Bucketed Counters
    |                                   |
    +-----------------------------------+
                    |
                    v
    Task 03: Signal Ledger and Velocity
                    |
                    v
    Task 04: Checkpoint and Restore
```

Tasks 01 and 02 are fully parallelizable -- they share no types or state. Task 03 composes them. Task 04 adds persistence.

## File Layout

```
tidal/src/
  signals/
    mod.rs            -- pub use re-exports, SignalTypeId newtype
    hot.rs            -- Task 01: HotSignalState, on_signal, current_score
    warm.rs           -- Task 02: BucketedCounter, windowed_count, all_time_count
    ledger.rs         -- Task 03: SignalLedger, WalWriter trait, velocity
    checkpoint.rs     -- Task 04: checkpoint, restore, serialization
  lib.rs              -- (unchanged, already declares pub mod signals)
```

## Open Questions

1. **`unsafe_code` and `#[repr(C, align(64))]`** -- The crate uses `#![forbid(unsafe_code)]`. `#[repr(C, align(64))]` itself does not require `unsafe` -- it is a layout attribute on a safe struct. Atomic operations (`AtomicU64`) are safe Rust. No `unsafe` is needed for m1p4. Confirmed: the spec's `HotSignalState` uses `AtomicU64` for f64 bit patterns via `f64::from_bits`/`f64::to_bits`, which are safe functions.

2. **`DashMap` dependency** -- `dashmap` crate needs to be added to `Cargo.toml`. It is a well-maintained, production-quality concurrent hash map with sharded locks. Alternatives (`crossbeam::SkipList`, manual sharded `RwLock<HashMap>`) are less ergonomic. The crossbeam dependency already exists. Decision: use `dashmap`.

3. **WAL trait boundary** -- m1p4 defines a `WalWriter` trait with a single method (`append`) that m1p2 will implement. For m1p4 testing, a no-op `WalWriter` is used. This allows m1p4 to be built and tested independently of m1p2, while establishing the correct dependency boundary. The `SignalLedger` takes a `Box<dyn WalWriter>` at construction.

4. **`SignalTypeId` representation** -- The spec uses `u16` for `signal_type_id`. Since the maximum is 64 signal types per entity kind, `u16` is generous but matches the spec. Introduce a `SignalTypeId(u16)` newtype in `signals/mod.rs`, assigned by the schema at registration time.

5. **Three decay scores vs one** -- The spec allocates space for 3 decay rates per signal type (for signals participating in multiple ranking profiles with different half-lives). For M1, only the primary decay rate (index 0) is used. The other two slots are zeroed. This matches the spec layout without requiring multi-profile support.