docs: withdraw the pre-release "not ready for production" disclaimer
M0-M12 are shipped and the HA cluster runs in production on k3s, so the pre-release disclaimer no longer describes the project. Removes it from the canonical doc set and corrects the readiness text that had gone stale. - README.md: replace the "Pre-release / not yet recommended for production" banner with a production-ready statement; drop "(experimental)" from the cluster status bullet; state the post-1.0 versioning posture (additive in minor releases, breaking changes get a documented migration path). - CLAUDE.md / QUICKSTART.md / docs/guides/server-deployment.md / docs/runbooks/cluster.md: same withdrawal; reframe the cluster opt-in as a guard against standing up a multi-node fabric by accident rather than a readiness warning. - CHANGELOG.md: record the stability posture under [Unreleased], superseding the historical 0.1.0 "no stability guarantees" note (left intact as history). - k8s/statefulset.yaml: the "NOT production HA, tracked as m8p10" comment was stale (m8p10 shipped); point at k8s/cluster/ for the HA deployment instead. Also corrects text that was factually wrong since m11p3/m11p4: the multi-process cluster gate, its CLI help, and the served OpenAPI description all still claimed quorum-ack writes and automatic failure detection did not exist. They do. Historical records (docs/reviews/, docs/profiling/, past CHANGELOG entries, the kubernetes.md rc7 fix note) are left unchanged. Verified against a running binary, not just the build: the opt-in gate's refusal message, the startup WARN, /health 200, and the served /openapi.json description all carry the new text. cargo fmt clean; clippy -D warnings clean on tidaldb and the tidal-server lib; 1943 engine + 155 server lib tests pass; scripts/check-docs.sh OK. Claude-Session: https://claude.ai/code/session_01QdqSDw1tUhK1JT9Pb1vryP
This commit is contained in:
jx12n
parent
4051077cff
commit
c22a3b65a6
10
CHANGELOG.md
10
CHANGELOG.md
@ -4,6 +4,16 @@ All notable changes to tidalDB will be documented in this file.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Stability
|
||||
|
||||
tidalDB is **production-ready**. M0–M12 are shipped and the pre-release
|
||||
"not yet recommended for production" disclaimer has been withdrawn from the
|
||||
documentation set. The API surface and on-disk data format are stable for
|
||||
shipped features: additive changes ship in minor releases, and any breaking
|
||||
change to a public API or a persisted format ships with a documented migration
|
||||
path in this file. This supersedes the `0.1.0` "no stability guarantees" note
|
||||
below.
|
||||
|
||||
### Added
|
||||
|
||||
**Multi-vector user preference modeling + ANN candidate-gen (M12) — a warm user is many interests, not one averaged vector: per-user preference clusters drive a top-M ANN fan-out in `for_you`**
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
|
||||
A single-node-first, embeddable Rust database for the **personalized content ranking problem**. Replaces the 6-system stack (Elasticsearch + Redis + Kafka + feature store + vector DB + ranking service) with a single process, single query interface, and single operational model.
|
||||
|
||||
**Status:** Implemented — M0–M12 shipped (embeddable engine + multi-region cluster mode). M11 (Enterprise-Grade Cluster) closed all nine phases 2026-06-13; M12 (Vector Retrieval at production shape) shipped the recall/ANN/index-tuning G1/G2 work, sharded ingestion, cluster elasticity (idle-readiness, TLS scale-up), and multi-vector preference modeling. This repository is a standalone Cargo workspace: the engine is the `tidaldb` crate at `tidal/`, with `tidal-net/`, `tidal-server/`, and `tidalctl/` as workspace siblings and example consumers under `applications/`. Pre-1.0 — APIs are stable for shipped features, but breaking changes are possible before 1.0. See [CHANGELOG.md](CHANGELOG.md) for milestone history and [docs/planning/ROADMAP.md](docs/planning/ROADMAP.md) for status and known gaps.
|
||||
**Status:** Implemented — M0–M12 shipped (embeddable engine + multi-region cluster mode). M11 (Enterprise-Grade Cluster) closed all nine phases 2026-06-13; M12 (Vector Retrieval at production shape) shipped the recall/ANN/index-tuning G1/G2 work, sharded ingestion, cluster elasticity (idle-readiness, TLS scale-up), and multi-vector preference modeling. This repository is a standalone Cargo workspace: the engine is the `tidaldb` crate at `tidal/`, with `tidal-net/`, `tidal-server/`, and `tidalctl/` as workspace siblings and example consumers under `applications/`. Production-ready — APIs are stable for shipped features; breaking changes ship with a documented migration path. See [CHANGELOG.md](CHANGELOG.md) for milestone history and [docs/planning/ROADMAP.md](docs/planning/ROADMAP.md) for status and known gaps.
|
||||
|
||||
## Find Your Guide
|
||||
|
||||
|
||||
@ -307,4 +307,4 @@ This flushes the WAL, checkpoints signal state, and persists indexes. In persist
|
||||
| Axum embedding example | `tidal/examples/axum_embedding.rs` |
|
||||
| 14 content discovery surfaces | [USE_CASES.md](USE_CASES.md) |
|
||||
| Architecture and design decisions | [ARCHITECTURE.md](ARCHITECTURE.md) |
|
||||
| Cluster mode (experimental) | [docs/runbooks/cluster.md](docs/runbooks/cluster.md) |
|
||||
| Cluster mode (high availability) | [docs/runbooks/cluster.md](docs/runbooks/cluster.md) |
|
||||
|
||||
21
README.md
21
README.md
@ -2,7 +2,9 @@
|
||||
|
||||
**An embeddable Rust database for the personalized content ranking problem.**
|
||||
|
||||
> Pre-release. API is stabilizing. Not yet recommended for production.
|
||||
> Production-ready. M0–M12 shipped: crash-safe storage, ranked retrieval, hybrid
|
||||
> search, ANN vector retrieval, and a quorum-acked HA cluster running in
|
||||
> production on k3s. The API surface is stable for shipped features.
|
||||
|
||||
---
|
||||
|
||||
@ -220,8 +222,13 @@ leader election + failover, elastic seed-join membership, inter-node mTLS,
|
||||
and per-node Prometheus metrics — deployed in production on k3s as one
|
||||
`StatefulSet` (3 pods = 3 regions = 3 voters, full-placement RF3 so every
|
||||
pod hosts all shard groups, HTTPS + mTLS on `:9500`). It exposes `/signals`,
|
||||
`/feed`, `/search` plus cluster-management routes. It stays behind the
|
||||
`--experimental-cluster` gate.
|
||||
`/feed`, `/search` plus cluster-management routes.
|
||||
|
||||
Because a standalone node is the right answer for most deployments, cluster
|
||||
mode requires an explicit opt-in flag (`--experimental-cluster`, or
|
||||
`TIDAL_ALLOW_EXPERIMENTAL_CLUSTER=1`) so nobody starts a multi-node fabric by
|
||||
accident. Reach for it deliberately when you need multi-node availability or
|
||||
read-scale.
|
||||
|
||||
```bash
|
||||
cargo run -p tidal-server -- \
|
||||
@ -311,6 +318,10 @@ Milestones completed:
|
||||
- Session and agent context (short-lived signals, preference decay)
|
||||
- Crash recovery, graceful degradation, rate limiting, diagnostics
|
||||
- Scale: tested to 1M items; scale benchmarks passing
|
||||
- High-availability cluster (experimental): quorum-acked writes, automatic election + failover, elastic seed-join membership, inter-node mTLS, per-node Prometheus — running in production on k3s
|
||||
- High-availability cluster: quorum-acked writes, automatic election + failover, elastic seed-join membership, inter-node mTLS, per-node Prometheus — running in production on k3s
|
||||
|
||||
The API surface is stable for the implemented features. Breaking changes are possible before 1.0.
|
||||
tidalDB is production-ready. The API surface is stable for the implemented
|
||||
features, and every shipped guarantee is covered by the chaos and soak suites in
|
||||
[docs/planning/ROADMAP.md](docs/planning/ROADMAP.md). Semantic versioning applies
|
||||
from here: additive changes ship in minor releases, and any breaking change gets
|
||||
a documented migration path in [CHANGELOG.md](CHANGELOG.md).
|
||||
|
||||
@ -16,7 +16,7 @@ Operational reference for running `tidal-server` — the HTTP wrapper around the
|
||||
| Crash recovery, WAL replay, backup/restore | [docs/ops/recovery.md](../ops/recovery.md) |
|
||||
| Sizing memory/disk/CPU, scrape budgets | [docs/ops/capacity-planning.md](../ops/capacity-planning.md) |
|
||||
| Kubernetes deployment (probes, volumes) | [docs/runbooks/kubernetes.md](../runbooks/kubernetes.md) |
|
||||
| Cluster operation (experimental) | [docs/runbooks/cluster.md](../runbooks/cluster.md) |
|
||||
| Cluster operation (high availability) | [docs/runbooks/cluster.md](../runbooks/cluster.md) |
|
||||
|
||||
---
|
||||
|
||||
@ -36,11 +36,11 @@ tidal-server standalone \
|
||||
--metrics 127.0.0.1:9091
|
||||
```
|
||||
|
||||
### Cluster — EXPERIMENTAL, not production HA
|
||||
### Cluster — production HA, opt-in
|
||||
|
||||
Cluster mode runs multiple "regions" behind one HTTP surface and replicates between them over the **real `tidal-net` gRPC transport**. It has two shapes: a **multi-process** mode (`--region`, one process per region, real gRPC peering between siblings, real process/host isolation — shipped in m8p10) and a single-process dev/demo default (all regions in one process over loopback). Since M11 it is genuinely HA: **quorum-acked writes** (`ack=quorum`, m11p3), **automatic election/failover** (m11p4), and **elastic membership** — DNS peers, seed join, online add/remove/replace with snapshot+stream catch-up (m11p5) — all exist. It remains **`--experimental-cluster`-gated**, so the recommended default is still a **single standalone node** (simpler, scales vertically); reach for cluster mode deliberately when you need multi-node availability or read-scale. See the [cluster runbook](../runbooks/cluster.md) and [`k8s/cluster/`](../../k8s/cluster/) for the reference deployment.
|
||||
Cluster mode runs multiple "regions" behind one HTTP surface and replicates between them over the **real `tidal-net` gRPC transport**. It has two shapes: a **multi-process** mode (`--region`, one process per region, real gRPC peering between siblings, real process/host isolation — the production shape) and a single-process dev/demo default (all regions in one process over loopback). Since M11 it is genuinely HA: **quorum-acked writes** (`ack=quorum`, m11p3), **automatic election/failover** (m11p4), and **elastic membership** — DNS peers, seed join, online add/remove/replace with snapshot+stream catch-up (m11p5). M12 added sharded ingestion and TLS scale-up. The reference deployment runs in production on k3s. The recommended default is still a **single standalone node** (simpler, scales vertically); reach for cluster mode deliberately when you need multi-node availability or read-scale. See the [cluster runbook](../runbooks/cluster.md) and [`k8s/cluster/`](../../k8s/cluster/) for the reference deployment.
|
||||
|
||||
The mode refuses to start unless you opt in explicitly (`--experimental-cluster` or `TIDAL_ALLOW_EXPERIMENTAL_CLUSTER=1`) and emits a loud `WARN` describing exactly what it is and is not. Cluster mode currently replicates **global signals only** — `user_id`/`creator_id` signal context is dropped on the cluster path so followers stay consistent with the leader's WAL.
|
||||
Because standalone is the right answer for most deployments, cluster mode refuses to start unless you opt in explicitly (`--experimental-cluster` or `TIDAL_ALLOW_EXPERIMENTAL_CLUSTER=1`) — a guard against starting a multi-node fabric by accident — and emits a startup `WARN` describing exactly which shape it is running. Cluster mode replicates **global signals only** — `user_id`/`creator_id` signal context is dropped on the cluster path so followers stay consistent with the leader's WAL.
|
||||
|
||||
For everything cluster-specific (topology file, leader promotion, partition/heal, scatter-gather routes), see the **[cluster runbook](../runbooks/cluster.md)**. The rest of this guide focuses on standalone.
|
||||
|
||||
|
||||
@ -20,10 +20,12 @@ write-durability contract.
|
||||
>
|
||||
> **Honest caveats that still hold:** cluster mode replicates **global** retrieval
|
||||
> signals only (no per-user personalization on the `/signals` route — see the
|
||||
> personalization note in [§3](#3-topology-yaml)); and both launch modes still
|
||||
> **refuse to start** unless you explicitly opt in with `--experimental-cluster` or
|
||||
> `TIDAL_ALLOW_EXPERIMENTAL_CLUSTER=1` (the k8s manifests set the env var). The two
|
||||
> launch shapes below remain:
|
||||
> personalization note in [§3](#3-topology-yaml)); and because a standalone node is
|
||||
> the right answer for most deployments, both launch modes **refuse to start**
|
||||
> unless you explicitly opt in with `--experimental-cluster` or
|
||||
> `TIDAL_ALLOW_EXPERIMENTAL_CLUSTER=1` (the k8s manifests set the env var) — a guard
|
||||
> against standing up a multi-node fabric by accident, not a readiness warning. The
|
||||
> two launch shapes below remain:
|
||||
>
|
||||
> **1. Multi-process (`--region`) — real process isolation (the production shape).** Each
|
||||
> `tidal-server cluster --region <name>` process owns **exactly one region**: one
|
||||
|
||||
@ -4,9 +4,10 @@
|
||||
# embeddable — the server wraps ONE engine instance whose state (WAL +
|
||||
# checkpoints + indexes) lives on a durable data dir. It scales VERTICALLY
|
||||
# (bigger node), not by adding replicas. `replicas: 1` is intentional and load-
|
||||
# bearing: there is no shared-storage multi-writer mode. The experimental
|
||||
# multi-region `cluster` subcommand runs every region in ONE process (no host
|
||||
# isolation — NOT production HA, tracked as m8p10), so it does not change this.
|
||||
# bearing: there is no shared-storage multi-writer mode. If you need multi-node
|
||||
# HA, that is a DIFFERENT deployment: the multi-process `cluster` subcommand
|
||||
# (one process per region, quorum-acked writes, automatic failover) ships as its
|
||||
# own StatefulSet under k8s/cluster/ — it does not change this manifest.
|
||||
# See docs/runbooks/kubernetes.md and docs/runbooks/cluster.md.
|
||||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
|
||||
@ -27,7 +27,7 @@
|
||||
//!
|
||||
//! * [`topology`] — `TopologySpec` / `RegionSpec` / `load_topology` / validation
|
||||
//! * [`transport`] — single-process self-loop gRPC transport wiring
|
||||
//! * [`state`] — [`ClusterState`] (single-process) + the experimental gate
|
||||
//! * [`state`] — [`ClusterState`] (single-process) + the cluster opt-in gate
|
||||
//! * [`routes`] — the single-process router + handlers
|
||||
//! * [`node`] — [`ClusterNode`] (the process) hosting [`ShardReplica`] groups
|
||||
//! (multi-process) + its gateway router/handlers
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
//! `ClusterState`: the single-process multi-region fabric behind the HTTP
|
||||
//! surface, plus the experimental-mode opt-in gate shared by both cluster modes.
|
||||
//! surface, plus the explicit opt-in gate shared by both cluster modes.
|
||||
|
||||
use std::{
|
||||
collections::HashMap,
|
||||
@ -26,14 +26,16 @@ use crate::{
|
||||
/// used by the distributed fabric. All cluster operations go through this
|
||||
/// layer.
|
||||
///
|
||||
/// # Experimental
|
||||
/// # Single-process shape
|
||||
///
|
||||
/// Replication between regions runs over the **real `tidal-net` gRPC
|
||||
/// transport** (see the module-level docs), but every region still lives inside
|
||||
/// this single process: there is no process or host isolation, so a crash takes
|
||||
/// the whole "cluster" down. It provides faithful multi-region replication
|
||||
/// semantics, not production high-availability. Cluster mode is therefore gated
|
||||
/// behind an explicit operator opt-in — see [`ensure_experimental_enabled`].
|
||||
/// semantics for dev and demo, not high-availability — production HA is the
|
||||
/// multi-process `--region` shape. Both shapes are gated behind an explicit
|
||||
/// operator opt-in so a multi-node fabric is never started by accident — see
|
||||
/// [`ensure_experimental_enabled`].
|
||||
pub struct ClusterState {
|
||||
/// `Some` for the lifetime of the server; consumed by [`shutdown`] /
|
||||
/// [`Drop`] so every node's `TidalDb` is dropped (checkpoint + WAL fsync +
|
||||
@ -58,27 +60,28 @@ pub struct ClusterState {
|
||||
write_pool: ClusterWritePool,
|
||||
}
|
||||
|
||||
/// Environment variable that opts in to the experimental cluster mode.
|
||||
/// Environment variable that opts in to cluster mode.
|
||||
pub const EXPERIMENTAL_CLUSTER_ENV: &str = "TIDAL_ALLOW_EXPERIMENTAL_CLUSTER";
|
||||
|
||||
/// Which cluster mode the experimental gate is being checked for, so the loud
|
||||
/// Which cluster mode the opt-in gate is being checked for, so the startup
|
||||
/// WARN can describe exactly what each mode does and does NOT provide.
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub enum ClusterMode {
|
||||
/// Every region runs inside ONE process (m8p8). No host/process isolation.
|
||||
/// Every region runs inside ONE process (m8p8). No host/process isolation —
|
||||
/// the dev/demo shape.
|
||||
SingleProcess,
|
||||
/// Exactly ONE region runs in this process and peers with sibling processes
|
||||
/// over real gRPC (m8p10). Real process isolation, but still no quorum-ack
|
||||
/// writes and no automatic failure detection.
|
||||
/// over real gRPC (m8p10). Real process isolation, quorum-acked writes
|
||||
/// (m11p3), and automatic election/failover (m11p4) — the production shape.
|
||||
MultiProcess,
|
||||
}
|
||||
|
||||
/// Gate cluster mode behind an explicit operator opt-in.
|
||||
///
|
||||
/// Both cluster modes are experimental and must not be started by accident:
|
||||
/// single-process has no host/process isolation, and multi-process — while it
|
||||
/// gives real process isolation — still lacks quorum-ack writes and automatic
|
||||
/// failure detection. Starting is permitted only when either:
|
||||
/// A standalone node is the right answer for most deployments, so neither
|
||||
/// cluster shape may be started by accident. This is a guard against standing up
|
||||
/// a multi-node fabric unintentionally, not a readiness warning: the
|
||||
/// multi-process shape is production HA. Starting is permitted only when either:
|
||||
///
|
||||
/// * the `--experimental-cluster` CLI flag is passed (`flag_set == true`), or
|
||||
/// * the [`EXPERIMENTAL_CLUSTER_ENV`] env var is set to a truthy value
|
||||
@ -99,12 +102,11 @@ pub fn ensure_experimental_enabled(flag_set: bool, mode: ClusterMode) -> Result<
|
||||
let reason = match mode {
|
||||
ClusterMode::SingleProcess => {
|
||||
"cluster mode replicates over real gRPC but runs all regions in ONE process \
|
||||
(no host/process isolation) — it is not production HA."
|
||||
(no host/process isolation) — it is the dev/demo shape, not production HA."
|
||||
}
|
||||
ClusterMode::MultiProcess => {
|
||||
"multi-process cluster mode (--region) gives real process isolation but does \
|
||||
NOT yet provide quorum-ack writes or automatic failure detection — it is not \
|
||||
production HA."
|
||||
"multi-process cluster mode (--region) stands up a real multi-node fabric; \
|
||||
a standalone node is the right default for most deployments."
|
||||
}
|
||||
};
|
||||
return Err(ServerError::ExperimentalDisabled(format!(
|
||||
@ -115,17 +117,17 @@ pub fn ensure_experimental_enabled(flag_set: bool, mode: ClusterMode) -> Result<
|
||||
|
||||
match mode {
|
||||
ClusterMode::SingleProcess => tracing::warn!(
|
||||
"EXPERIMENTAL cluster mode enabled. Regions replicate over the real tidal-net gRPC \
|
||||
transport (loopback), but all regions run in a SINGLE process — there is no host or \
|
||||
process isolation, so this provides NO production high-availability. True \
|
||||
multi-process deployment is the --region mode. Do NOT use this for production traffic."
|
||||
"Single-process cluster mode enabled (dev/demo shape). Regions replicate over the \
|
||||
real tidal-net gRPC transport (loopback), but all regions run in a SINGLE process — \
|
||||
there is no host or process isolation, so this provides NO high-availability. Use \
|
||||
the --region multi-process mode for production traffic."
|
||||
),
|
||||
ClusterMode::MultiProcess => tracing::warn!(
|
||||
"EXPERIMENTAL multi-process cluster mode enabled (--region). This process owns ONE \
|
||||
region and peers with siblings over real gRPC, so process isolation IS real — but \
|
||||
quorum-ack writes and automatic failure detection are NOT yet provided (a 204 means \
|
||||
leader durability only, and failover is operator-driven). Do NOT use this for \
|
||||
production traffic."
|
||||
"Multi-process cluster mode enabled (--region) — the production HA shape. This \
|
||||
process owns ONE region and peers with siblings over real gRPC: process isolation \
|
||||
is real, writes are quorum-acked when the topology sets replication.ack=quorum, and \
|
||||
leader election/failover is automatic. Cluster mode replicates GLOBAL signals only \
|
||||
(user_id/creator_id context is dropped on the cluster path)."
|
||||
),
|
||||
}
|
||||
Ok(())
|
||||
|
||||
@ -30,7 +30,7 @@ pub enum ServerError {
|
||||
Network(#[from] std::io::Error),
|
||||
#[error("bad request: {0}")]
|
||||
BadRequest(String),
|
||||
#[error("cluster mode is experimental and disabled: {0}")]
|
||||
#[error("cluster mode requires an explicit opt-in: {0}")]
|
||||
ExperimentalDisabled(String),
|
||||
#[error("cluster error: {0}")]
|
||||
Cluster(String),
|
||||
|
||||
@ -41,14 +41,18 @@ struct ClusterArgs {
|
||||
/// `TIDAL_CONFIG` env var.
|
||||
#[arg(long, env = "TIDAL_CONFIG")]
|
||||
config_dir: Option<PathBuf>,
|
||||
/// Opt in to the EXPERIMENTAL cluster mode (single- or multi-process).
|
||||
/// Opt in to cluster mode (single- or multi-process).
|
||||
///
|
||||
/// A standalone node is the right default for most deployments, so cluster
|
||||
/// mode refuses to start without this flag (or the
|
||||
/// `TIDAL_ALLOW_EXPERIMENTAL_CLUSTER=1` env var) — a guard against standing
|
||||
/// up a multi-node fabric by accident.
|
||||
///
|
||||
/// Single-process cluster mode replicates between regions over the real
|
||||
/// tidal-net gRPC transport (loopback), but every region runs inside THIS
|
||||
/// one process — no host/process isolation. Multi-process mode (`--region`)
|
||||
/// gives real process isolation but still lacks quorum-ack writes and
|
||||
/// automatic failure detection. Either way it refuses to start without this
|
||||
/// flag (or the `TIDAL_ALLOW_EXPERIMENTAL_CLUSTER=1` env var).
|
||||
/// one process — no host/process isolation; it is the dev/demo shape.
|
||||
/// Multi-process mode (`--region`) is the production HA shape: real process
|
||||
/// isolation, quorum-acked writes, and automatic election/failover.
|
||||
#[arg(long)]
|
||||
experimental_cluster: bool,
|
||||
/// Run ONLY this region in this process (multi-process cluster mode).
|
||||
@ -233,9 +237,9 @@ async fn run_cluster(args: ClusterArgs) -> Result<()> {
|
||||
async fn run_single_process_cluster(args: ClusterArgs) -> Result<()> {
|
||||
// Honest gate: single-process cluster mode replicates between regions over
|
||||
// the real tidal-net gRPC transport, but every region runs inside this one
|
||||
// process (no host/process isolation), so it is not production HA. Refuse to
|
||||
// start (and emit a loud WARN when permitted) so no operator mistakes it for
|
||||
// production HA.
|
||||
// process (no host/process isolation), so it is the dev/demo shape, not HA.
|
||||
// Refuse to start (and emit a startup WARN when permitted) so no operator
|
||||
// mistakes it for the multi-process production shape.
|
||||
ensure_experimental_enabled(args.experimental_cluster, ClusterMode::SingleProcess)?;
|
||||
|
||||
let schema_path = resolve_config_path(
|
||||
@ -283,8 +287,9 @@ async fn run_single_process_cluster(args: ClusterArgs) -> Result<()> {
|
||||
}
|
||||
|
||||
async fn run_region_cluster(args: ClusterArgs, region: String) -> Result<()> {
|
||||
// Multi-process mode is experimental too: real process isolation, but no
|
||||
// quorum-ack writes or automatic failure detection yet. The gate's WARN text
|
||||
// Multi-process mode is the production HA shape (real process isolation,
|
||||
// quorum-acked writes, automatic election/failover) but still opt-in, so a
|
||||
// multi-node fabric is never stood up by accident. The gate's WARN text
|
||||
// differentiates the two modes honestly.
|
||||
ensure_experimental_enabled(args.experimental_cluster, ClusterMode::MultiProcess)?;
|
||||
|
||||
@ -579,7 +584,7 @@ impl ServeState for ClusterState {
|
||||
// `SimulatedCluster` (out of this crate's edit scope), whose nodes can
|
||||
// only be closed by dropping the `Arc<SimulatedCluster>` — which needs
|
||||
// `&mut self` here. This branch is not reached in practice (this path is
|
||||
// experimental and not the deployed cluster; in steady state
|
||||
// the dev/demo shape, not the deployed cluster; in steady state
|
||||
// `try_unwrap` succeeds and `shutdown_owned` runs). Flip readiness and
|
||||
// rely on the last-`Arc` `Drop` (which runs the same `&mut self` close).
|
||||
Self::set_shutting_down(self);
|
||||
|
||||
@ -129,8 +129,10 @@ pub struct StandaloneApiDoc;
|
||||
version = env!("CARGO_PKG_VERSION"),
|
||||
description = "Cluster-mode tidalDB HTTP surface: the standalone data + health \
|
||||
routes plus multi-region cluster management and sharded \
|
||||
scatter-gather routes. EXPERIMENTAL — all regions run in one \
|
||||
process (no host/process isolation).",
|
||||
scatter-gather routes. This is the single-process dev/demo \
|
||||
shape — all regions run in one process (no host/process \
|
||||
isolation). For production HA run one process per region \
|
||||
with --region.",
|
||||
),
|
||||
paths(
|
||||
crate::cluster::routes::cluster_status,
|
||||
@ -193,8 +195,8 @@ pub struct ClusterApiDoc;
|
||||
peers with siblings over real gRPC, presenting the cluster as one \
|
||||
coherent HTTP surface (leader forwarding, broadcast, status \
|
||||
aggregation, cross-process reconcile, and sharded scatter-gather). \
|
||||
EXPERIMENTAL — real process isolation, but no quorum-ack writes or \
|
||||
automatic failure detection yet.",
|
||||
This is the production HA shape: real process isolation, \
|
||||
quorum-acked writes, and automatic election/failover.",
|
||||
),
|
||||
paths(
|
||||
crate::cluster::node::status_local,
|
||||
|
||||
Loading…
Reference in New Issue
Block a user