stemedb/crates/stemedb-ingest/src/worker
jordan 42d4e09508 feat: Index persistence (Phase 5C) - vector hot/cold, visual checkpoint
Phase 5C (Index Persistence) implementation:
- PersistentVectorIndex with hot/cold architecture
  - Hot: in-memory HNSW for recent vectors
  - Cold: memory-mapped HNSW loaded from disk
  - Background builder for WAL replay and atomic swap
  - BLAKE3 integrity verification
- PersistentVisualIndex with checkpoint persistence
  - BkTreeSnapshot with rkyv serialization
  - CRC32C corruption detection
  - Atomic write pattern (temp → fsync → rename)
- Key codec additions for vector index metadata
- Split large files into modules (<500 lines each)
  - battery_pre_sentinel.rs → battery/ directory
  - visual_index.rs → visual_index/ directory
  - persistent.rs → persistent/ directory
- Refactored ingest worker tests for clarity
- Updated roadmap to mark Phase 5 complete

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 15:43:18 -07:00
..
tests feat: Index persistence (Phase 5C) - vector hot/cold, visual checkpoint 2026-02-02 15:43:18 -07:00
mod.rs feat: WAL hardening (Phase 5B) - CRC32C, crash recovery, group commit, log rotation 2026-02-02 12:36:35 -07:00
processing.rs feat: WAL hardening (Phase 5B) - CRC32C, crash recovery, group commit, log rotation 2026-02-02 12:36:35 -07:00
record_types.rs refactor: Split all files to enforce 500-line max 2026-02-02 01:13:45 -07:00
run.rs refactor: Split all files to enforce 500-line max 2026-02-02 01:13:45 -07:00
storage.rs feat: WAL hardening (Phase 5B) - CRC32C, crash recovery, group commit, log rotation 2026-02-02 12:36:35 -07:00