7e8b7b013a
Gap analysis + 9-phase plan (m11p1-p9) from the experimental m8p10 cluster to an enterprise-grade one, grounded in the 2026-06-10 live stress-test baselines: replicated /signals ~90/s vs sharded 3,669/s vs embedded ~82ns, plus four live-observed incidents (restart leadership amnesia, side-channel broadcast bugs, breaker-eaten heal, the 178ms ship) promoted to requirements. Phases: perf floor, one replicated log, quorum acks (G4), election+fencing (G5), membership/discovery, sharding x RF, security, observability/ops, chaos CI — each with a testable Ref-A exit gate; release waves v0.9 credible / v1.0 scalable / v1.1 enterprise. Indexed in docs/README.md; pointer from ROADMAP's Post-M8 Follow-ups.
63 lines
4.5 KiB
Markdown
63 lines
4.5 KiB
Markdown
# tidalDB Engineering Docs
|
||
|
||
The engineering documentation home. Top-level product docs (VISION, USE_CASES, SEQUENCE,
|
||
ARCHITECTURE, API, QUICKSTART, CODING_GUIDELINES, thoughts) live at the
|
||
[repository root](../CLAUDE.md); everything below is the deeper engineering record.
|
||
|
||
> This and the repo root are the **two canonical doc homes**. There is intentionally no
|
||
> per-crate doc mirror (no `tidal/docs/`). Edit the canonical file, never a copy.
|
||
|
||
## Component specs — `specs/`
|
||
|
||
The authoritative component specifications (status: Implemented, M0–M8).
|
||
|
||
| # | Spec | # | Spec |
|
||
|---|------|---|------|
|
||
| 00 | [Architecture overview](specs/00-architecture-overview.md) | 08 | [Query engine](specs/08-query-engine.md) |
|
||
| 01 | [Storage engine](specs/01-storage-engine.md) | 09 | [Ranking & scoring](specs/09-ranking-scoring.md) |
|
||
| 02 | [Entity model](specs/02-entity-model.md) | 10 | [Feedback loop](specs/10-feedback-loop.md) |
|
||
| 03 | [Signal system](specs/03-signal-system.md) | 11 | [Schema](specs/11-schema.md) |
|
||
| 04 | [Relationships](specs/04-relationships.md) | 12 | [Cold start](specs/12-cold-start.md) |
|
||
| 05 | [Cohorts](specs/05-cohorts.md) | 13 | [Concurrency](specs/13-concurrency.md) |
|
||
| 06 | [Text retrieval](specs/06-text-retrieval.md) | 14 | [Scale architecture](specs/14-scale-architecture.md) |
|
||
| 07 | [Vector retrieval](specs/07-vector-retrieval.md) | | |
|
||
|
||
## Planning — `planning/`
|
||
|
||
- [**ROADMAP.md**](planning/ROADMAP.md) — milestones M0–M10, phase status, known gaps
|
||
- [**roadmap-to-cluster.md**](roadmap-to-cluster.md) — gap analysis + phase plan (M11 candidate) from the experimental multi-process cluster to an enterprise-grade one, grounded in the 2026-06-10 live stress-test baselines
|
||
- [PRODUCT_ROADMAP.md](planning/PRODUCT_ROADMAP.md) · [architecture-review.md](planning/architecture-review.md) · [roadmap-cohort-analysis.md](planning/roadmap-cohort-analysis.md) · [site-cohort-analysis.md](planning/site-cohort-analysis.md)
|
||
- Per-milestone phase/task archive: `planning/milestone-0,1,2,3,5,7,8,9,10,p/`
|
||
|
||
## Code reviews — `reviews/`
|
||
|
||
- [M0–M10 code review — 2026-06-07](reviews/M0-M10-code-review-2026-06-07.md) — seven-dimension re-review, 88 verified findings
|
||
- [M0–M10 code review — 2026-06-08](reviews/M0-M10-code-review-2026-06-08.md) — seven-dimension review, 142 findings (latest pass)
|
||
- [M0–M10 seven-dimension review](reviews/M0-M10-seven-dimension-review.md) — additional pass (2 BLOCKERs: signal-checkpoint trim, 30-day window)
|
||
|
||
## Guides — `guides/`
|
||
|
||
Task-oriented, build-an-app docs (complements the root [QUICKSTART.md](../QUICKSTART.md) and [API.md](../API.md)):
|
||
|
||
- [**Build a feed app**](guides/build-a-feed-app.md) — end-to-end TikTok/Reels-style "For You" feed, embedded and over HTTP
|
||
- [**Embedding integration**](guides/embeddings.md) — wiring a real embedding model (OpenAI / Cohere / local) into the write + query paths
|
||
- [**Server deployment**](guides/server-deployment.md) — running the `tidal-server` HTTP service: config, auth, the served OpenAPI spec, Docker
|
||
- Ranking-profile reference: [ai-lookup/services/ranking-profiles.md](../ai-lookup/services/ranking-profiles.md) — all 25 built-in profiles
|
||
|
||
## Operations — `ops/` and `runbooks/`
|
||
|
||
- [Monitoring](ops/monitoring.md) · [Prometheus alerts](ops/prometheus-alerts.yaml) · [Grafana dashboard](ops/grafana-dashboard.json) · [Capacity planning](ops/capacity-planning.md) · [Recovery](ops/recovery.md)
|
||
- Runbooks: [Kubernetes](runbooks/kubernetes.md) · [Cluster (experimental)](runbooks/cluster.md)
|
||
|
||
## Research — `research/`
|
||
|
||
ANN ([1](research/ann_for_tidaldb.md), [2](research/ann_for_tidaldb_gemini.md)) · Tantivy ([1](research/tantivy.md), [2](research/tantivy_gemini.md)) · Signal ledger ([1](research/tidaldb_signal_ledger.md), [2](research/tidaldb_signal_ledger_gemini.md)) · [WAL](research/tidaldb_wal.md) · [Type system](research/phase1_1_type_system.md) · [Tooling & diagnostics](research/tidaldb_tooling_and_diagnostics.md) · [Enterprise-readiness risks](research/enterprise_readiness_risks.md)
|
||
|
||
## Profiling — `profiling/`
|
||
|
||
[Hotspot analysis](profiling/hotspot-analysis.md) · [Scale baselines](profiling/scale-baselines.md) · [Signal memory](profiling/signal-memory-analysis.md) · [Signal rollup eval](profiling/signal-rollup-eval.md) · [Social scale](profiling/social-scale.md) · [Tantivy merge tuning](profiling/tantivy-merge-tuning.md) · [USearch tuning](profiling/usearch-tuning.md)
|
||
|
||
## Strategy
|
||
|
||
[Content strategy](content-strategy.md) · [Personal-briefing beachhead](personal-briefing-beachhead.md)
|