3.1 KiB
Social Scale Performance Analysis
CoEngagementIndex Eviction
Correctness Tests
All tests in tidal/tests/m7p3_social_scale.rs pass:
| Test | Invariant Verified |
|---|---|
eviction_correctness_at_2x_capacity |
edge_count <= capacity after every insert at 2× load |
high_weight_edge_survival_under_eviction |
High-weight edges (score=100) survive low-weight flood |
co_engagement_memory_bounded_at_2x_insertions |
10 users × 40 items stays within capacity=200 |
no_self_loops_after_eviction |
No (a, a) edges created under eviction pressure |
top_candidates_consistent_after_eviction |
top_candidates() works correctly after eviction |
Eviction Benchmarks
| Capacity | Mean Eviction Latency | Notes |
|---|---|---|
| 10K edges | ~2ms | O(N log N) sort over 10K entries |
| 50K edges | ~12ms | O(N log N) sort over 50K entries |
| 100K edges | ~26ms | O(N log N) sort over 100K entries |
Benchmark: cargo bench --manifest-path tidal/Cargo.toml --bench social -- co_engagement_eviction
Note: Eviction is amortised. With USER_RECENT_CAPACITY=50, each record_positive adds at
most 50 new edges. At default capacity=50K, eviction fires at most once every ~50K/50=1000
positive engagements — approximately once per active user session.
Social Graph Filter at 1M Items
Setup
- 100 followed creators
- 500 followers per creator
- 10K items per creator = 1M items total
Benchmark Results
| Depth | Mean Latency | p99 Latency | Notes |
|---|---|---|---|
| Depth-1 (followed creator items) | ~3ms | ~8ms | Direct bitmap union over 100 creators |
| Depth-2 (co-followers' seen items) | ~25ms | ~45ms | Fan-out: 100 creators × 500 followers |
Target: p99 < 50ms — ✅ achieved
Run: cargo bench --manifest-path tidal/Cargo.toml --bench social -- social_graph_1m
Fan-out Cap
The social graph filter uses DEPTH2_FAN_OUT_CAP to bound the number of co-followers
processed. Without this cap, depth-2 at 500 followers × 100 creators = 50K co-follower
lookups. With the cap, this is bounded to prevent p99 blowup.
Cross-Session Preference Merge
Algorithm
PreferenceVectors::update_with_custom_rate(user_id, &interaction, lr):
pref = (1 - lr) * pref + lr * interaction
pref = L2_normalize(pref)
For 128D: 128 multiply-adds + L2 normalize = ~256 FP ops + sqrt.
Benchmark Results
| Operation | Mean Latency | Target |
|---|---|---|
| Single 128D EMA (100K users) | ~3µs | < 1ms ✅ |
| 10-session batch (100K users) | ~28µs | < 10ms ✅ |
Both targets easily met. The bottleneck is DashMap lookup (~2µs overhead), not the EMA computation itself (~1µs for 128D).
Run: cargo bench --manifest-path tidal/Cargo.toml --bench social -- preference_merge
Adaptive Learning Rate
The update() method (used in production) uses an adaptive learning rate that decays as:
alpha = base_alpha / (1 + ln(update_count + 1))
This means users with many updates converge slower (preferences are more stable).
First-session users get alpha = 0.1; users with 100+ sessions get alpha ≈ 0.02.