History

jx12n 25296bcc5b docs: refresh ops runbooks to the live rc7 / full-placement reality The runbooks had drifted to the retired m8/m11p5 design while all m12 production reality (topology, perf, fixes, DR) sat only in a profiling doc no operator opens. This promotes that reality into the runbooks and fixes the contradictions. Contradictions fixed: - runbooks/cluster.md: the "NEITHER IS QUORUM-ACKED HA YET" status banner was FALSE (quorum-ack + automatic election have been live since m11p3/p4). Rewritten to state the deployed reality (single-StatefulSet full-placement RF3, rc7). - README.md: the cluster section called the HA cluster a "built-in simulated cluster / multi-region fabric" demo and showed promote-by-region as failover. Rewritten — real quorum HA, automatic failover, /cluster/promote is a maintenance verb. Kept the honest caveats (experimental gate, global-signals-only). Reality promoted into the runbooks: - Live topology (single STS, ns tidaldb-cluster, 3 voters, full-placement RF3, gRPC 9601/9602/9603, HTTPS+mTLS :9500), the five shipped fixes, and the real build+digest-pin procedure (cross-compile -> trixie -> amd64 PLATFORM manifest, not the index/attestation digest) in cluster.md + kubernetes.md. - Stale constants: soak ramp 3900 -> 200 rps; cluster grace 60 -> 600s; the pre-m12 4.5k/s signal-write perf table annotated + the 1536-D read reality added. - ops/capacity-planning.md: new "Ref-A 3-node fleet — measured capacity" section (read p99/ceiling, ~250 rps write knee, 1M needs >16GiB nodes, pod resources). - ops/recovery.md: new cluster-recovery routing section + scoped the quiesce-and- copy note to standalone (the cluster uses tidalctl + the DR runbook). New docs: - runbooks/disaster-recovery.md: the proven S3/R2 backup -> restore -> byte-verify -> query-proof procedure, full-cluster rebuild, PITR posture (previously undocumented despite being proven against real S3). - runbooks/on-call.md: incident response — symptom -> golden signal -> runbook, severity, escalation, and the open alert-wiring step. - runbooks/README.md: the runbook index + current production facts. Open follow-up (infra, not docs): ops/prometheus-alerts.yaml is accurate but design-reference; promoting it to a live PrometheusRule is the one unwired step.		2026-06-19 19:53:29 -06:00
..
guides	feat(m11): cluster security (m11p7) + perf instrumentation floor	2026-06-13 01:25:35 -06:00
ops	docs: refresh ops runbooks to the live rc7 / full-placement reality	2026-06-19 19:53:29 -06:00
planning	fix(net): classify ship deadline as timeout, not partition (write-burst false-partition)	2026-06-19 16:25:50 -06:00
profiling	fix(m12p6): 6-bug k3s 3-shard cluster repair (rc8+rc9)	2026-06-16 22:34:21 -06:00
research	chore: doc consolidation, seven-dimension review fixes, and commit hooks	2026-06-08 22:46:28 -06:00
reviews	feat(m11): observability+ops (m11p8) + perf-sweep wave 2 T2	2026-06-13 09:17:49 -06:00
runbooks	docs: refresh ops runbooks to the live rc7 / full-placement reality	2026-06-19 19:53:29 -06:00
specs	feat(m12): vector retrieval G1/G2 — recall harness, ANN in RETRIEVE, index tuning	2026-06-14 11:07:09 -06:00
content-strategy.md	feat: complete Milestone 5 — full-text search, RRF fusion, and creator search	2026-02-21 23:53:16 -07:00
personal-briefing-beachhead.md	feat: M0p1 runtime skeleton, M0p2 tooling & diagnostics, m1p4 signal ledger	2026-02-20 20:32:00 -07:00
README.md	feat(m11): membership, snapshot install, and reseed (m11p5)	2026-06-12 19:55:54 -06:00
roadmap-to-cluster.md	feat(m11): sharding × replication + rebalancing (m11p6 L3-L5)	2026-06-13 18:23:43 -06:00

README.md

tidalDB Engineering Docs

The engineering documentation home. Top-level product docs (VISION, USE_CASES, SEQUENCE, ARCHITECTURE, API, QUICKSTART, CODING_GUIDELINES, thoughts) live at the repository root; everything below is the deeper engineering record.

This and the repo root are the two canonical doc homes. There is intentionally no per-crate doc mirror (no tidal/docs/). Edit the canonical file, never a copy.

Component specs — `specs/`

The authoritative component specifications (status: Implemented, M0–M8).

#	Spec	#	Spec
00	Architecture overview	08	Query engine
01	Storage engine	09	Ranking & scoring
02	Entity model	10	Feedback loop
03	Signal system	11	Schema
04	Relationships	12	Cold start
05	Cohorts	13	Concurrency
06	Text retrieval	14	Scale architecture
07	Vector retrieval

Planning — `planning/`

ROADMAP.md — milestones M0–M11, phase status, known gaps
roadmap-to-cluster.md — adopted M11 plan: gap analysis + phase specs taking the multi-process cluster from experimental to enterprise-grade (m11p1–p5 ✅; p6–p9 planned), grounded in the 2026-06-10 live stress-test baselines
PRODUCT_ROADMAP.md · architecture-review.md · roadmap-cohort-analysis.md · site-cohort-analysis.md
Per-milestone phase/task archive: planning/milestone-0,1,2,3,5,7,8,9,10,11,p/

Code reviews — `reviews/`

M0–M10 code review — 2026-06-07 — seven-dimension re-review, 88 verified findings
M0–M10 code review — 2026-06-08 — seven-dimension review, 142 findings (latest pass)
M0–M10 seven-dimension review — additional pass (2 BLOCKERs: signal-checkpoint trim, 30-day window)

Guides — `guides/`

Task-oriented, build-an-app docs (complements the root QUICKSTART.md and API.md):

Build a feed app — end-to-end TikTok/Reels-style "For You" feed, embedded and over HTTP
Embedding integration — wiring a real embedding model (OpenAI / Cohere / local) into the write + query paths
Server deployment — running the tidal-server HTTP service: config, auth, the served OpenAPI spec, Docker
Ranking-profile reference: ai-lookup/services/ranking-profiles.md — all 25 built-in profiles