tidaldb/tidal/src/storage
jordan e5fd19eb73 fix(vector): load a persisted slot graph by its own backend, not an assumed one
Found by running the standalone server end to end: after a clean shutdown that
logged "persisted HNSW graphs to disk (next boot loads instead of rebuilding)",
the next boot logged

  WARN persisted HNSW graph failed to load; falling back to rebuild
       slot="content_vector" error=USearch load failed: Failed to read vectors

and rebuilt the index. Every boot. Correct results, no data loss, a WARN nobody
reads - and the optimization was dead.

Cause: `checkpoint_graphs` saves whichever index the slot holds, and
`build_slot_index` holds a BRUTE-FORCE index below the dimension-aware
crossover. Both write to the same `<kind>__<slot>.usearch` path, so a small slot
persisted a `BFVI` file that `load_persisted_slot` then handed to the USearch
reader. Observed on a 6-item, 128-D store; at production scale the slot is
USearch-backed, which is why the cluster never surfaced it - but every dev,
staging, and small-tenant instance pays a full index rebuild on every start, and
that rebuild is what the 20-minute startup probe budget exists for.

Fix: sniff the file's magic (brute-force `MAGIC` is now `pub(crate)` so the
reader and writer cannot drift) and dispatch to the matching loader. A
brute-force graph is still rejected when `expected_count` has grown past the
crossover, so the rebuild upgrades the backend to HNSW rather than pinning a
linear scan forever. Log lines now say "vector graph" and name the backend
instead of claiming HNSW for both.

Tests: `brute_force_slot_graph_round_trips_through_registry_persistence` covers
the case that was broken (the existing round-trip test forces USearch, which is
why it passed throughout), and
`grown_corpus_rejects_a_brute_force_graph_so_the_rebuild_upgrades_it` pins the
upgrade path. Verified live: the same data directory that produced the WARN now
logs `loaded persisted vector graph from disk (skipped full rebuild)
backend="brute-force" count=6`, with feed, text search, and vector search all
returning the pre-restart results.
2026-08-17 17:47:55 -06:00
..
indexes fix: M0-M10 code-review pass2 remediation — all 91 findings 2026-06-09 12:21:00 -06:00
vector fix(vector): load a persisted slot graph by its own backend, not an assumed one 2026-08-17 17:47:55 -06:00
batch.rs feat: implement Milestone 1 phases 1-3 — schema, WAL, and storage layer 2026-02-20 16:43:24 -07:00
engine.rs fix: M0-M10 code-review pass2 remediation — all 91 findings 2026-06-09 12:21:00 -06:00
error.rs fix: M0-M10 code-review pass2 remediation — all 91 findings 2026-06-09 12:21:00 -06:00
fjall.rs fix: M0-M10 code-review pass2 remediation — all 91 findings 2026-06-09 12:21:00 -06:00
iterator.rs feat: implement Milestone 1 phases 1-3 — schema, WAL, and storage layer 2026-02-20 16:43:24 -07:00
keys.rs Merge remote-tracking branch 'origin/main' (m11/m12 cluster) into m9/m10 2026-08-03 02:16:04 -06:00
memory.rs fix: M0-M10 code-review pass2 remediation — all 91 findings 2026-06-09 12:21:00 -06:00
mod.rs feat: complete Milestones 2–4 — RETRIEVE query, vector index, ranking profiles, diversity, entity system, sessions 2026-02-21 16:24:48 -07:00