tidaldb

jordan/tidaldb

Fork 0

Commit Graph

Author	SHA1	Message	Date
jx12n	a0399550d6	feat(m12p6): persist HNSW graph + bounded SIGTERM drain — boot loads, no rebuild Boot now LOADS the per-slot HNSW graph instead of rebuilding it. Clean shutdown writes {data_dir}/vector/<kind>__<slot>.usearch; the next open loads it when it matches the durable corpus (seconds), falling back to a full rebuild only when the graph is missing/stale/corrupt. Eliminates the multi-minute boot rebuild (~50-70 min at 1M/1536-D) that let the WAL compact past a restarting node and triggered the reseed cascade. Graceful SIGTERM now actually runs the close: bounded_drain caps the post-signal HTTP drain (TIDAL_SHUTDOWN_DRAIN_MS, default 15s) then runs the deterministic close regardless — sibling keep-alive connections no longer block the drain past the k8s 60s grace into a SIGKILL (which cannot run Drop). ClusterNode and ShardReplica::shutdown are now &self (db handle is an ArcSwapOption) so the close fires even when a stuck connection task holds an Arc. Fix USearch insert to be a true upsert (remove+add): it was unconditional add, which a multi:false index rejects on a reseeding follower's post-snapshot WAL replay -> applied_events stalls -> catch-up deadlock -> unrecoverable cluster. Also: circuit-breaker peer last-contact tracking; real k3s 1536-dim deploy + recall findings (recall@10 0.9869, read p99 8.71ms @ 200rps @ 100k) in docs/profiling/m12-cluster-deploy-findings.md; new tidal-stress k8s jobs and m12p6 graph-persistence + SIGTERM tier-3 regression tests.	2026-06-15 13:09:20 -06:00
jx12n	bb21e69ae6	feat(m12): vector retrieval G1/G2 — recall harness, ANN in RETRIEVE, index tuning m12p1 (measurement truth): TidalDb::vector_search_items pure k-NN probe + POST /vector_search (standalone + region node, merge-by-distance) + tidal-stress --verify-recall (deterministic id-keyed corpus, in-RAM brute-force cosine oracle, open-loop ramp → recall@k + true p99 + read-knee + JSON/gate exit). Repaired fabricated p99 columns (mean-as-p99) in social-scale.md / scale.rs. Verified real: recall@10=0.9997 at 20k/1536-D vs brute-force. m12p2 (G1 unblock): ANN candidate-gen wired into RETRIEVE — for_you=preference vector, related=seed embedding (similar_to), graceful scan-fallback. Cached per-signal-type top-K (signals/ledger/hot_top_k.rs, decay-order-invariant) so trending serves O(K). related over HTTP (FeedQuery.similar_to). Harness gains --feed-profile / --seed-preferences. Verified: trending retrieve p99 3.5-7.7ms. m12p3 (G2): per-query ef_search now honored (RwLock epoch-guard with_expansion, shared guard for same-ef concurrency) + dimension-aware brute→HNSW crossover usearch_min_vectors(dim) + memory_usage() + examples/ann_grid_search.rs. Measured 1536-D/100k clustered: default M=16/ef_c=400/F16/ef_s=200 clears G1+G2 (recall 0.997, p99 1.4ms); F16 -0.25% vs F32; Int8 rejected (-28%). Recall corpus is now clustered (Gaussian mixture) in grid + harness.	2026-06-14 11:07:09 -06:00

Author

SHA1

Message

Date

jx12n

a0399550d6

feat(m12p6): persist HNSW graph + bounded SIGTERM drain — boot loads, no rebuild

Boot now LOADS the per-slot HNSW graph instead of rebuilding it. Clean
shutdown writes {data_dir}/vector/<kind>__<slot>.usearch; the next open loads
it when it matches the durable corpus (seconds), falling back to a full rebuild
only when the graph is missing/stale/corrupt. Eliminates the multi-minute boot
rebuild (~50-70 min at 1M/1536-D) that let the WAL compact past a restarting
node and triggered the reseed cascade.

Graceful SIGTERM now actually runs the close: bounded_drain caps the post-signal
HTTP drain (TIDAL_SHUTDOWN_DRAIN_MS, default 15s) then runs the deterministic
close regardless — sibling keep-alive connections no longer block the drain past
the k8s 60s grace into a SIGKILL (which cannot run Drop). ClusterNode and
ShardReplica::shutdown are now &self (db handle is an ArcSwapOption) so the close
fires even when a stuck connection task holds an Arc.

Fix USearch insert to be a true upsert (remove+add): it was unconditional add,
which a multi:false index rejects on a reseeding follower's post-snapshot WAL
replay -> applied_events stalls -> catch-up deadlock -> unrecoverable cluster.

Also: circuit-breaker peer last-contact tracking; real k3s 1536-dim deploy +
recall findings (recall@10 0.9869, read p99 8.71ms @ 200rps @ 100k) in
docs/profiling/m12-cluster-deploy-findings.md; new tidal-stress k8s jobs and
m12p6 graph-persistence + SIGTERM tier-3 regression tests.

2026-06-15 13:09:20 -06:00

jx12n

bb21e69ae6

feat(m12): vector retrieval G1/G2 — recall harness, ANN in RETRIEVE, index tuning

m12p1 (measurement truth): TidalDb::vector_search_items pure k-NN probe +
POST /vector_search (standalone + region node, merge-by-distance) +
tidal-stress --verify-recall (deterministic id-keyed corpus, in-RAM brute-force
cosine oracle, open-loop ramp → recall@k + true p99 + read-knee + JSON/gate exit).
Repaired fabricated p99 columns (mean-as-p99) in social-scale.md / scale.rs.
Verified real: recall@10=0.9997 at 20k/1536-D vs brute-force.

m12p2 (G1 unblock): ANN candidate-gen wired into RETRIEVE — for_you=preference
vector, related=seed embedding (similar_to), graceful scan-fallback. Cached
per-signal-type top-K (signals/ledger/hot_top_k.rs, decay-order-invariant) so
trending serves O(K). related over HTTP (FeedQuery.similar_to). Harness gains
--feed-profile / --seed-preferences. Verified: trending retrieve p99 3.5-7.7ms.

m12p3 (G2): per-query ef_search now honored (RwLock epoch-guard with_expansion,
shared guard for same-ef concurrency) + dimension-aware brute→HNSW crossover
usearch_min_vectors(dim) + memory_usage() + examples/ann_grid_search.rs.
Measured 1536-D/100k clustered: default M=16/ef_c=400/F16/ef_s=200 clears
G1+G2 (recall 0.997, p99 1.4ms); F16 -0.25% vs F32; Int8 rejected (-28%).
Recall corpus is now clustered (Gaussian mixture) in grid + harness.

2026-06-14 11:07:09 -06:00

2 Commits