tidaldb/tidal/src
jordan 7c1c80dd90 fix(vector): grow the HNSW graph on insert; it could never accept a new vector once full
Measured on the live RF3 cluster 2026-08-30 while verifying it end to end. Every
NEW embedding returned HTTP 500:

  POST /sharded/embeddings -> 500
  {"error":"... [op=write_item_embedding] backend error:
    USearch insert failed: Reserve capacity ahead of insertions!"}

while POST /sharded/items returned 201, POST /sharded/signals 204, text search
and feed 200, and re-embedding an ALREADY-INDEXED entity returned 204. So the
store had silently become read/update-only for vectors: a consumer could write
items and signals all day and only its embeddings would fail, with nothing
alerting on it.

Cause: USearch's add() cannot grow the graph. build_slot_index() reserves the
rebuild's expected count once and its comment claimed 'the write path grows it
further as needed' - the write path never reserved anything. insert() went
straight to add() for a new key, so the moment size() reached the reservation
every new key failed permanently. The upsert path kept working because remove()
tombstones and size() excludes tombstones, so the re-add lands in the slot just
freed - which is exactly why this looked healthy from the outside.

insert() now reserves before adding a new key at capacity, growing by
max(size/8, 1024) so reserve's reallocation is amortised rather than per-insert.

The regression test drives off the REPORTED capacity, not the requested one:
USearch rounds a reservation up (reserve(4) reported 64), so a hardcoded insert
count sits inside the reservation and exercises nothing. Verified against the
pre-fix source it fails with the production error at insert 64 of capacity 64,
and it asserts the grown graph still answers searches and still upserts.

storage::vector 101 passed, db::items 7 passed.
2026-08-30 11:33:03 -06:00
..
cohort Merge remote-tracking branch 'origin/main' (m11/m12 cluster) into m9/m10 2026-08-03 02:16:04 -06:00
db feat(observability): HTTP metrics, structured logs, dashboard, live tidalctl 2026-08-23 10:31:57 -06:00
entities Merge remote-tracking branch 'origin/main' (m11/m12 cluster) into m9/m10 2026-08-03 02:16:04 -06:00
experiment Merge remote-tracking branch 'origin/main' (m11/m12 cluster) into m9/m10 2026-08-03 02:16:04 -06:00
governance fix: M0-M10 code-review pass2 remediation — all 91 findings 2026-06-09 12:21:00 -06:00
load fix: M0-M10 third-pass remediation — durability, replication, and CLI hardening 2026-06-08 10:28:34 -06:00
query Merge remote-tracking branch 'origin/main' (m11/m12 cluster) into m9/m10 2026-08-03 02:16:04 -06:00
ranking Merge remote-tracking branch 'origin/main' (m11/m12 cluster) into m9/m10 2026-08-03 02:16:04 -06:00
replication feat(cluster): expose every tracked replication stream key's position 2026-08-21 11:01:46 -06:00
schema Merge remote-tracking branch 'origin/main' (m11/m12 cluster) into m9/m10 2026-08-03 02:16:04 -06:00
session Merge remote-tracking branch 'origin/main' (m11/m12 cluster) into m9/m10 2026-08-03 02:16:04 -06:00
signals Merge remote-tracking branch 'origin/main' (m11/m12 cluster) into m9/m10 2026-08-03 02:16:04 -06:00
storage fix(vector): grow the HNSW graph on insert; it could never accept a new vector once full 2026-08-30 11:33:03 -06:00
testing feat(m11): observability+ops (m11p8) + perf-sweep wave 2 T2 2026-06-13 09:17:49 -06:00
text fix: M0-M10 code-review pass2 remediation — all 91 findings 2026-06-09 12:21:00 -06:00
wal fix(m12-rc13): read-SLA collapse + WAL_RETENTION_SEGMENTS 16 + tidalctl S3 DR 2026-06-17 15:47:37 -06:00
fault.rs feat(m11): continuous correctness (m11p9) — fault classes, invariant checkers, soak gates, nightly pipeline 2026-06-13 15:23:59 -06:00
lib.rs feat(observability): HTTP metrics, structured logs, dashboard, live tidalctl 2026-08-23 10:31:57 -06:00