jordan/stemedb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
81 Commits 1 Branch 0 Tags 18 MiB
4360a17dd3
Commit Graph

3 Commits

Author SHA1 Message Date
jordan
02ecac9a07 fix: merge upstream 10 commits, fix DashMap deadlock, deterministic sim ingestion 
Merged 10 upstream commits (MemTable, read-your-writes tests, feed endpoint,
security hardening, signed assertions, source registry, dashboard enhancements)
and fixed all test failures across the full workspace (2656/2656 passing).

Key fixes:
- fix(cluster): DashMap deadlock in swim.rs suspect_node/fail_node/alive_node
  - DashMap::get_mut RefMut + iter() on same map = non-reentrant write lock deadlock
  - Fix: extract clone in scoped block to drop RefMut before calling update_node_gauges()
  - 6 previously-hanging SWIM tests now pass in <2s
- fix(sim): replace background-task+polling ingestion with synchronous process_pending()
  - smoke_high_volume_simulation was CPU-starved under 2656 parallel tests
  - Removed ingestor.start() + wait_until_ingested() pattern throughout sim
  - All arena functions now call ingestor.process_pending() directly (deterministic)
- fix(test): v2 signature helper used wrong hash (rkyv vs canonical compute_content_hash_v2)
- fix(test): quota test signed "test" but v1 requires "subject:predicate" format
- fix(test): http_validation now accepts 400 for valid-format-but-invalid-crypto hex
- fix(test): scale_adaptive micro tier assertions updated (auto_promote upstream change)
- config: add nextest.toml with slow-timeout for background-task-tests group

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-02-20 20:27:32 -07:00
jordan
137a588ed0 feat: Concept hierarchy (Phase 5D) - ConceptPath, source schemes, AliasStore 
Implements hierarchical subject identifiers with scheme-based source tier inference:

- ConceptPath type with parse/wire_format, leaf/parent, prefix matching
- SourceScheme registry mapping schemes to default SourceClass tiers:
  - rfc://, fda://, ietf:// → Regulatory (Tier 0)
  - peer://, pubmed:// → PeerReviewed (Tier 1)
  - code://, wiki:// → Expert (Tier 3)
  - blog://, anon:// → Anecdotal (Tier 5)
- AliasStore for cross-scheme entity resolution (bidirectional indexing)
- API endpoints for concept operations
- Battery tests 8, 9 & 10 for concepts, aliases, and advanced signatures
- Go SDK updates for concept types and signing

Completes Phase 5, advancing to Phase 6 (Distributed Writes).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-02 17:44:54 -07:00
jordan
42d4e09508 feat: Index persistence (Phase 5C) - vector hot/cold, visual checkpoint 
Phase 5C (Index Persistence) implementation:
- PersistentVectorIndex with hot/cold architecture
  - Hot: in-memory HNSW for recent vectors
  - Cold: memory-mapped HNSW loaded from disk
  - Background builder for WAL replay and atomic swap
  - BLAKE3 integrity verification
- PersistentVisualIndex with checkpoint persistence
  - BkTreeSnapshot with rkyv serialization
  - CRC32C corruption detection
  - Atomic write pattern (temp → fsync → rename)
- Key codec additions for vector index metadata
- Split large files into modules (<500 lines each)
  - battery_pre_sentinel.rs → battery/ directory
  - visual_index.rs → visual_index/ directory
  - persistent.rs → persistent/ directory
- Refactored ingest worker tests for clarity
- Updated roadmap to mark Phase 5 complete

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-02 15:43:18 -07:00