jordan/tidaldb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5ed2edb211
tidaldb/docs
History
jx12n 5ed2edb211 feat(m11): quorum-acked writes — ack=leader|quorum, commit index, durable frontier reports (m11p3) 
ack=quorum gates replicated writes on a majority of the replica set durably
holding them: followers push their durably-applied frontier (ReportApplied,
once per apply round, decoupled from ship acks), the leader folds frontier
reports + ship-ack hints + heal resumes into a leadership-scoped CommitIndex
(k-th-largest durable mark), and handlers await it through an async
watch-channel bridge (zero parked threads per waiter). Honest timeouts:
retryable 503 naming the laggards; x-tidal-seq on every cluster write.
Follower blob applies are batched under group-commit fsyncs (22x seeding).
Exit gate: 167/167 leader-SIGKILL kill points, zero acked-write loss.

Seven-dimension review pass (all confirmed findings fixed):
- WAL blob drain now ABORTS on the first write failure instead of reusing
  the failed seqno mid-drain (a torn record buried mid-segment would
  truncate every later acked record on replay)
- apply_replicated_blobs waits every staged append even after a mid-batch
  failure, parses metadata once, and moves records into Arcs shared with
  the WAL writer (no deep clone per record on the follower apply path)
- CommitIndex: zero-peer fast path now respects demotion (active checked
  under lock before the single-replica return), k-th-largest uses
  select_nth over a reused scratch buffer
- await_quorum: re-reads the index once after the deadline fires (no false
  503 for a write that committed in the race window), warns when the
  commit-watch bridge dies outside shutdown, zero-peer path checks active
- notify_applied report failures: WARN on the first failure of a streak,
  INFO on recovery (a silently stalling frontier reads as unexplained
  quorum 503s); receiver skips re-notifying unadvanced frontiers
- x-tidal-deduplicated: 1 marks dedup-suppressed signal writes (relayed
  through forwards) so durability cursors can tell dedup from no-seqno
- docs: 167/167 kill-point record corrected in CHANGELOG; rolling-upgrade
  order (leader first — a pre-m11p3 leader silently downgrades quorum
  requests to leader-ack) in CHANGELOG + runbook §8; monitoring note for
  report-loss diagnosis on the quorum-timeout alert

Verified: workspace clippy -D warnings (incl. cluster-e2e targets), full
tidaldb/tidal-net/tidal-server/tidalctl suites green, tier-3 multi-process
quorum suite green (8/8 kill points, zero acked loss, partition gate/recover).
2026-06-11 13:28:08 -06:00
..
guides feat(m8p10): multi-process cluster mode — scatter-gather, reconcile relay, chaos/UAT suites 2026-06-10 14:07:33 -06:00
ops feat(m11): quorum-acked writes — ack=leader|quorum, commit index, durable frontier reports (m11p3) 2026-06-11 13:28:08 -06:00
planning feat(m11): quorum-acked writes — ack=leader|quorum, commit index, durable frontier reports (m11p3) 2026-06-11 13:28:08 -06:00
profiling feat: complete M6-M7 + Enterprise Readiness milestones; split oversized source files per CODING_GUIDELINES §9 2026-02-23 22:41:16 -07:00
research chore: doc consolidation, seven-dimension review fixes, and commit hooks 2026-06-08 22:46:28 -06:00
reviews fix: M0-M10 code-review pass2 remediation — all 91 findings 2026-06-09 12:21:00 -06:00
runbooks feat(m11): quorum-acked writes — ack=leader|quorum, commit index, durable frontier reports (m11p3) 2026-06-11 13:28:08 -06:00
specs chore: doc consolidation, seven-dimension review fixes, and commit hooks 2026-06-08 22:46:28 -06:00
content-strategy.md feat: complete Milestone 5 — full-text search, RRF fusion, and creator search 2026-02-21 23:53:16 -07:00
personal-briefing-beachhead.md feat: M0p1 runtime skeleton, M0p2 tooling & diagnostics, m1p4 signal ledger 2026-02-20 20:32:00 -07:00
README.md docs: roadmap to an enterprise-grade cluster (M11 candidate) 2026-06-10 22:20:15 -06:00
roadmap-to-cluster.md feat(m11): quorum-acked writes — ack=leader|quorum, commit index, durable frontier reports (m11p3) 2026-06-11 13:28:08 -06:00

README.md

tidalDB Engineering Docs

The engineering documentation home. Top-level product docs (VISION, USE_CASES, SEQUENCE, ARCHITECTURE, API, QUICKSTART, CODING_GUIDELINES, thoughts) live at the repository root; everything below is the deeper engineering record.

This and the repo root are the two canonical doc homes. There is intentionally no per-crate doc mirror (no tidal/docs/). Edit the canonical file, never a copy.

Component specs — specs/

The authoritative component specifications (status: Implemented, M0–M8).

# Spec # Spec
00 Architecture overview 08 Query engine
01 Storage engine 09 Ranking & scoring
02 Entity model 10 Feedback loop
03 Signal system 11 Schema
04 Relationships 12 Cold start
05 Cohorts 13 Concurrency
06 Text retrieval 14 Scale architecture
07 Vector retrieval

Planning — planning/

Code reviews — reviews/

Guides — guides/

Task-oriented, build-an-app docs (complements the root QUICKSTART.md and API.md):

Operations — ops/ and runbooks/

Research — research/

ANN (1, 2) · Tantivy (1, 2) · Signal ledger (1, 2) · WAL · Type system · Tooling & diagnostics · Enterprise-readiness risks

Profiling — profiling/

Hotspot analysis · Scale baselines · Signal memory · Signal rollup eval · Social scale · Tantivy merge tuning · USearch tuning

Strategy

Content strategy · Personal-briefing beachhead