jordan 3cfaa1e1d3 feat: Complete Phase 1 (The Spine) - storage foundation

Phase 1 delivers the complete durability and storage layer:

- WAL with crash recovery: Append-only journal with BLAKE3 checksums,
  fsync guarantees, and proper seek-to-EOF on reopen
- Storage engine: sled-backed KVStore with scan_prefix for range queries
- Content-addressed storage: H:{hash}, V:{hash}, E:{hash} key patterns
- Ingestor: Background worker tailing WAL, writing to KV with 8-byte
  aligned record headers for rkyv zero-copy deserialization
- Comprehensive tests: 31 tests covering crash recovery, round-trips,
  and multi-cycle durability

New crates: stemedb-wal, stemedb-storage, stemedb-ingest

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

2026-01-31 14:15:34 -07:00

2.4 KiB

Raw Blame History

Tracing Coverage Remediation

AUDIT (2026-01-31)

Pattern: Missing structured tracing in critical paths Found: 6 files with tracing dependency but insufficient instrumentation

Current State (Before)

File	Has Tracing Dep	Has Spans	Critical Ops Instrumented
`stemedb-wal/journal.rs`	✓ (via crate)	✗	✗ (append, recover)
`stemedb-storage/sled_backend.rs`	✓	✗	✗ (get, put, delete)
`stemedb-ingest/ingestor.rs`	✓ (via crate)	✗	✗ (start, process)
`stemedb-ingest/worker.rs`	✓	✓	Partial (has info!/debug!)
`stemedb-wal/durability.rs`	✓	Minimal	Only error!() on sync fail
`stemedb-sim/main.rs`	✓	✓	info!() calls present

FIX (2026-01-31)

crates/stemedb-wal/src/journal.rs
- Added #[instrument] to: open, append, read, recover, open_current_file
- Added debug!/info!/warn! for key events (file creation, recovery, record appends)
crates/stemedb-storage/src/sled_backend.rs
- Added #[instrument] to: open, get, put, delete, flush
- Fields include: key_len, value_len, found status
crates/stemedb-ingest/src/ingestor.rs
- Added #[instrument] to: start, process_pending
- Added lifecycle logging (info! on start, debug! on completion)
crates/stemedb-wal/src/durability.rs
- Added #[instrument] to: lock_exclusive, force_sync
- Added debug! confirmations for lock acquisition and sync completion

VERIFY (2026-01-31)

$ grep -rn "#\[instrument" crates/ --include="*.rs" | wc -l
15

All critical paths now have spans. Tests pass.

ENFORCE (2026-01-31)

Added to CLAUDE.md Critical Rules:

- **Instrument Critical Paths:** Use `#[instrument]` on public methods in WAL, storage, and ingestion code. Include meaningful fields (key_len, payload_len, offset).

DOCUMENT (2026-01-31)

Updated .claude/skills/stemedb-core/SKILL.md:

Added "Tracing Pattern" section with code example
Added "Add public methods without #[instrument]" to Do Not list
Added "Instrument public methods" to Do list

Summary

Before: 0 #[instrument] spans in critical paths After: 14 #[instrument] spans covering WAL, storage, and ingestion

Enforcement: CLAUDE.md rule prevents regression Documentation: Skill updated with pattern and anti-pattern

2.4 KiB Raw Blame History