jx12n
bb21e69ae6
feat(m12): vector retrieval G1/G2 — recall harness, ANN in RETRIEVE, index tuning
...
m12p1 (measurement truth): TidalDb::vector_search_items pure k-NN probe +
POST /vector_search (standalone + region node, merge-by-distance) +
tidal-stress --verify-recall (deterministic id-keyed corpus, in-RAM brute-force
cosine oracle, open-loop ramp → recall@k + true p99 + read-knee + JSON/gate exit).
Repaired fabricated p99 columns (mean-as-p99) in social-scale.md / scale.rs.
Verified real: recall@10=0.9997 at 20k/1536-D vs brute-force.
m12p2 (G1 unblock): ANN candidate-gen wired into RETRIEVE — for_you=preference
vector, related=seed embedding (similar_to), graceful scan-fallback. Cached
per-signal-type top-K (signals/ledger/hot_top_k.rs, decay-order-invariant) so
trending serves O(K). related over HTTP (FeedQuery.similar_to). Harness gains
--feed-profile / --seed-preferences. Verified: trending retrieve p99 3.5-7.7ms.
m12p3 (G2): per-query ef_search now honored (RwLock epoch-guard with_expansion,
shared guard for same-ef concurrency) + dimension-aware brute→HNSW crossover
usearch_min_vectors(dim) + memory_usage() + examples/ann_grid_search.rs.
Measured 1536-D/100k clustered: default M=16/ef_c=400/F16/ef_s=200 clears
G1+G2 (recall 0.997, p99 1.4ms); F16 -0.25% vs F32; Int8 rejected (-28%).
Recall corpus is now clustered (Gaussian mixture) in grid + harness.
2026-06-14 11:07:09 -06:00
jx12n
ad4134e280
chore: doc consolidation, seven-dimension review fixes, and commit hooks
...
- Eliminate the tidal/ self-contained doc mirror; docs now have two canonical
homes (root *.md and docs/), with planning/specs/research/reviews moved up
- Remove stale .agents/skills and .ai mirrors; canonicalize skills under .claude/
- Add pre-commit hook + scripts/check-docs.sh doc-guard + scripts/install-hooks.sh
- Implement M0-M10 seven-dimension review findings across engine, net, server,
and tidalctl (durability, replication, query, WAL, storage, CLI hardening)
2026-06-08 22:46:28 -06:00
jordan
413b712c0a
chore: initialize tidalDB repository with schema foundation and standards
...
- Schema phase 1 (tasks 01-02): EntityId, EntityKind, Timestamp, Score, SignalTypeDef, DecayModel, Window, WindowSet — all with property tests and benchmarks scaffolding
- Stub modules for storage, signals, query, ranking
- Full documentation suite: VISION, USE_CASES, SEQUENCE, API, CODING_GUIDELINES, ai-lookup, research docs, specs, roadmap, planning docs
- Marketing site (Next.js) with blog infrastructure
- .claude/ agents and skills for the tidalDB development workflow
- Foundation standards enforced: thiserror + tracing declared as dependencies, clippy::unwrap_used = deny added to lint config
- .gitignore hardened: .next/, node_modules/, .env, secrets, logs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-20 12:52:20 -07:00
