History

jx12n aa94fd9b1f feat(m12p5): idle-readiness convergence via heartbeat live frontier + wildcard cert SAN Leader heartbeat now carries its live flushed WAL frontier (leader_last_seq, proto field 14) so a snapshot-installed joiner converges its sticky readiness latch from the heartbeat — which flows even on a fully idle cluster — instead of only from observed ship traffic or an external status poll. Fixes the idle-readiness stall (WORKLOG 2026-06-13: an 11.5h /health 503 hang where a caught-up joiner never joined the Service VIP). - proto: HeartbeatRequest.leader_last_seq (field 14); 0 = pre-m12p5 leader → fall back to the status-poll readiness path - ElectionHooks::on_heartbeat threads leader_last_seq through net + driver - ShardReplica::note_leader_frontier_for_readiness folds the frontier into the lag gauge (monotonic per shard) and drives the readiness latch using a REAL leader frontier (never the uninitialized-0 gauge, which would false-converge a still-behind joiner); a joiner that WINS leadership converges trivially - tier-3 regression: mp_idle_cluster_snapshot_joiner_flips_ready_without_traffic — snapshot joiner flips /health ready on an idle cluster with zero writes and no status poll, then proves content parity (honest convergence) - certs: wildcard pod SAN (*.tidaldb-peers...) in k8s/cluster/certs.yaml and scripts/gen-cluster-certs.sh so StatefulSet scale-up/down with --seed needs no cert re-issue (T4 scale-to-5 broke mTLS on tidaldb-3/4); explicit per-pod names kept as belt-and-suspenders - docs/profiling/m12p5-idle-readiness-elasticity.md: root-cause + fix writeup		2026-06-14 16:21:00 -06:00
..
guides	feat(m11): cluster security (m11p7) + perf instrumentation floor	2026-06-13 01:25:35 -06:00
ops	bench(1536): production-shape capacity — read path is cheap, quorum write is the ceiling	2026-06-13 21:40:58 -06:00
planning	feat(m11): sharding × replication + rebalancing (m11p6 L3-L5)	2026-06-13 18:23:43 -06:00
profiling	feat(m12p5): idle-readiness convergence via heartbeat live frontier + wildcard cert SAN	2026-06-14 16:21:00 -06:00
research	chore: doc consolidation, seven-dimension review fixes, and commit hooks	2026-06-08 22:46:28 -06:00
reviews	feat(m11): observability+ops (m11p8) + perf-sweep wave 2 T2	2026-06-13 09:17:49 -06:00
runbooks	feat(m11): sharding × replication + rebalancing (m11p6 L3-L5)	2026-06-13 18:23:43 -06:00
specs	feat(m12): vector retrieval G1/G2 — recall harness, ANN in RETRIEVE, index tuning	2026-06-14 11:07:09 -06:00
content-strategy.md	feat: complete Milestone 5 — full-text search, RRF fusion, and creator search	2026-02-21 23:53:16 -07:00
personal-briefing-beachhead.md	feat: M0p1 runtime skeleton, M0p2 tooling & diagnostics, m1p4 signal ledger	2026-02-20 20:32:00 -07:00
README.md	feat(m11): membership, snapshot install, and reseed (m11p5)	2026-06-12 19:55:54 -06:00
roadmap-to-cluster.md	feat(m11): sharding × replication + rebalancing (m11p6 L3-L5)	2026-06-13 18:23:43 -06:00

README.md

tidalDB Engineering Docs

The engineering documentation home. Top-level product docs (VISION, USE_CASES, SEQUENCE, ARCHITECTURE, API, QUICKSTART, CODING_GUIDELINES, thoughts) live at the repository root; everything below is the deeper engineering record.

This and the repo root are the two canonical doc homes. There is intentionally no per-crate doc mirror (no tidal/docs/). Edit the canonical file, never a copy.

Component specs — `specs/`

The authoritative component specifications (status: Implemented, M0–M8).

#	Spec	#	Spec
00	Architecture overview	08	Query engine
01	Storage engine	09	Ranking & scoring
02	Entity model	10	Feedback loop
03	Signal system	11	Schema
04	Relationships	12	Cold start
05	Cohorts	13	Concurrency
06	Text retrieval	14	Scale architecture
07	Vector retrieval

Planning — `planning/`

ROADMAP.md — milestones M0–M11, phase status, known gaps
roadmap-to-cluster.md — adopted M11 plan: gap analysis + phase specs taking the multi-process cluster from experimental to enterprise-grade (m11p1–p5 ✅; p6–p9 planned), grounded in the 2026-06-10 live stress-test baselines
PRODUCT_ROADMAP.md · architecture-review.md · roadmap-cohort-analysis.md · site-cohort-analysis.md
Per-milestone phase/task archive: planning/milestone-0,1,2,3,5,7,8,9,10,11,p/

Code reviews — `reviews/`

M0–M10 code review — 2026-06-07 — seven-dimension re-review, 88 verified findings
M0–M10 code review — 2026-06-08 — seven-dimension review, 142 findings (latest pass)
M0–M10 seven-dimension review — additional pass (2 BLOCKERs: signal-checkpoint trim, 30-day window)

Guides — `guides/`

Task-oriented, build-an-app docs (complements the root QUICKSTART.md and API.md):

Build a feed app — end-to-end TikTok/Reels-style "For You" feed, embedded and over HTTP
Embedding integration — wiring a real embedding model (OpenAI / Cohere / local) into the write + query paths
Server deployment — running the tidal-server HTTP service: config, auth, the served OpenAPI spec, Docker
Ranking-profile reference: ai-lookup/services/ranking-profiles.md — all 25 built-in profiles