jordan 3320c24afa feat: WAL hardening (Phase 5B) - CRC32C, crash recovery, group commit, log rotation

Add CRC32C checksums to WAL record format (v2), implement crash recovery
with automatic truncation of corrupt records, add feature-gated group commit
buffer for batched fsync under concurrent load, and implement log rotation
via segment files with global offset addressing.

Key changes:
- Record format v2: [len:u32][crc32c:u32][blake3:32][payload:N]
- recover_file() scans and truncates corrupt tail records
- GroupCommitBuffer batches fsync via MPSC channel (tokio feature gate)
- SegmentManager with binary search resolution and cursor-based cleanup
- Journal::read() auto-refreshes segments on miss for writer/reader split
- Split recovery.rs and key_codec.rs into directory modules for 500-line max

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

2026-02-02 12:36:35 -07:00

2.8 KiB

Raw Blame History

TrustRank (Reputation)

Last Updated: 2026-02-01 Confidence: High Status: Implemented in stemedb-storage v0.1.0

Summary

TrustRank is a reputation system for agent credibility. Agents who make accurate claims gain reputation; agents who contradict settled truth lose reputation.

Key Facts:

Stored in TR:{AgentId} key in KV store via TrustRankStore trait
Scores bounded to [0.0, 1.0], default 0.5 for new agents
Used by TrustAwareAuthorityLens to weight assertions
Confidence decays over time (30-day half-life by default)

File Pointer: crates/stemedb-storage/src/trust_rank_store.rs

TrustRank Record

pub struct TrustRank {
    pub agent_id: [u8; 32],
    pub score: f32,           // 0.0 to 1.0
    pub last_updated: u64,    // Unix timestamp
    pub assertions_count: u64,
    pub accuracy_count: u64,
}

How It Works

Score Updates

// Record an outcome for an agent
trust_store.record_outcome(&agent_id, was_accurate, timestamp).await?;

// Accurate prediction: +0.05 (capped at 1.0)
// Inaccurate prediction: -0.1 (floored at 0.0)
// Higher penalty for inaccuracy discourages spam

Confidence Decay

// Apply time-based decay to all scores
trust_store.decay_trust_ranks(current_timestamp, Some(custom_half_life)).await?;

// Default half-life: 30 days
// Score approaches 0.5 (neutral) over time without activity

Usage in TrustAwareAuthorityLens

use stemedb_lens::TrustAwareAuthorityLens;
use stemedb_storage::{HybridStore, GenericTrustRankStore};
use std::sync::Arc;

let store = HybridStore::open("./data")?;
let trust_store = Arc::new(GenericTrustRankStore::new(store));
let lens = TrustAwareAuthorityLens::new(trust_store);

let resolution = lens.resolve_async(&candidates).await;
// Winner = assertion with highest (confidence * trust_rank)

Resolution Strategy:

For each candidate, lookup signer's TrustRank (O(1))
Calculate: weighted_score = assertion.confidence * agent.trust_rank
Return assertion with highest weighted score
Tiebreaker: most recent timestamp
Unsigned assertions treated as 0.0 trust

Score Interpretation

Score	Meaning
0.0-0.3	Unreliable (history of inaccuracy)
0.3-0.5	Below neutral (more wrong than right)
0.5	Neutral (new agent or unknown)
0.5-0.7	Above neutral (more right than wrong)
0.7-1.0	Reliable (strong track record)

API Access

Via LensDto:

lens=Authority → Routes to TrustAwareAuthorityLens
lens=TrustAwareAuthority → Routes to TrustAwareAuthorityLens
lens=Confidence → Uses confidence field only (no TrustRank)

Lens - TrustAwareAuthorityLens integration
Assertion - Signature and agent_id fields

2.8 KiB Raw Blame History