jordan/tidaldb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
d0a52e4530
tidaldb/docs
History
jx12n d0a52e4530 feat(m11): catch-up timer retry + TSEG segment version header (m11p4) 
WAL segment format: 8-byte TSEG header (magic + version byte + 3 reserved)
prepended to every new segment. Legacy headerless segments (m0-m11p3) read
as implicit v0 — no migration. Unknown magic/version surfaces as
WalError::SegmentFormatUnknown at open time; foreign files are never
repaired or truncated (fixes the silent data-loss path from the p3 rollout
incident where torn-tail repair zeroed a follower's unreadable segments).

Catch-up transport: FAILED_PRECONDITION ("snapshot required") and stream
errors that skip the shard now arm a timer retry (re-arm-on-skip is the
load-bearing liveness fix — without it a skipped pull never re-fires and
the follower stays permanently behind). Single retry pending per shard;
CatchupRunner owns the Arc'd state shared between the retry tasks and the
transport. Test: tidal-net/tests/catchup_retry.rs covers the retry path.

Stress: k8s stress-job-t2a/t2b yaml + ops/stress-test-p3-t2 runbook.
2026-06-11 17:05:20 -06:00
..
guides feat(m8p10): multi-process cluster mode — scatter-gather, reconcile relay, chaos/UAT suites 2026-06-10 14:07:33 -06:00
ops feat(m11): catch-up timer retry + TSEG segment version header (m11p4) 2026-06-11 17:05:20 -06:00
planning feat(m11): quorum-acked writes — ack=leader|quorum, commit index, durable frontier reports (m11p3) 2026-06-11 13:28:08 -06:00
profiling feat: complete M6-M7 + Enterprise Readiness milestones; split oversized source files per CODING_GUIDELINES §9 2026-02-23 22:41:16 -07:00
research chore: doc consolidation, seven-dimension review fixes, and commit hooks 2026-06-08 22:46:28 -06:00
reviews fix: M0-M10 code-review pass2 remediation — all 91 findings 2026-06-09 12:21:00 -06:00
runbooks feat(m11): catch-up timer retry + TSEG segment version header (m11p4) 2026-06-11 17:05:20 -06:00
specs feat(m11): catch-up timer retry + TSEG segment version header (m11p4) 2026-06-11 17:05:20 -06:00
content-strategy.md feat: complete Milestone 5 — full-text search, RRF fusion, and creator search 2026-02-21 23:53:16 -07:00
personal-briefing-beachhead.md feat: M0p1 runtime skeleton, M0p2 tooling & diagnostics, m1p4 signal ledger 2026-02-20 20:32:00 -07:00
README.md docs: roadmap to an enterprise-grade cluster (M11 candidate) 2026-06-10 22:20:15 -06:00
roadmap-to-cluster.md feat(m11): quorum-acked writes — ack=leader|quorum, commit index, durable frontier reports (m11p3) 2026-06-11 13:28:08 -06:00

README.md

tidalDB Engineering Docs

The engineering documentation home. Top-level product docs (VISION, USE_CASES, SEQUENCE, ARCHITECTURE, API, QUICKSTART, CODING_GUIDELINES, thoughts) live at the repository root; everything below is the deeper engineering record.

This and the repo root are the two canonical doc homes. There is intentionally no per-crate doc mirror (no tidal/docs/). Edit the canonical file, never a copy.

Component specs — specs/

The authoritative component specifications (status: Implemented, M0–M8).

# Spec # Spec
00 Architecture overview 08 Query engine
01 Storage engine 09 Ranking & scoring
02 Entity model 10 Feedback loop
03 Signal system 11 Schema
04 Relationships 12 Cold start
05 Cohorts 13 Concurrency
06 Text retrieval 14 Scale architecture
07 Vector retrieval

Planning — planning/

Code reviews — reviews/

Guides — guides/

Task-oriented, build-an-app docs (complements the root QUICKSTART.md and API.md):

Operations — ops/ and runbooks/

Research — research/

ANN (1, 2) · Tantivy (1, 2) · Signal ledger (1, 2) · WAL · Type system · Tooling & diagnostics · Enterprise-readiness risks

Profiling — profiling/

Hotspot analysis · Scale baselines · Signal memory · Signal rollup eval · Social scale · Tantivy merge tuning · USearch tuning

Strategy

Content strategy · Personal-briefing beachhead