44b768b8c6
End the "replicated XOR sharded" split: S shard groups, each a
replication group at RF with its own elected leader, leaders balanced
across nodes; any gateway hash-routes.
- One unified write surface: /items,/embeddings,/signals hash-route to
the owning shard group's leader (ShardRouter FNV-1a) AND replicate at
RF. x-tidal-ack/x-tidal-seq, quorum await, NotLeader/QuorumTimeout are
per-group; NotLeader names the group.
- Rebalance verbs (L3): POST /cluster/shards/{id}/transfer (fenced
leadership move) + /cluster/shards/{id}/replicas (add/remove replica).
A ?shard= selector threads through every per-shard admin verb and is
propagated on intra-group forwards (ShardReplica::admin_path). S=1 is
byte-for-byte (no selector, no shard in NotLeader body).
- Tier-3 exit gate (cluster_sharding.rs): 3 nodes × 3 shards × RF=3 over
real OS processes — SIGKILL a node under ack=quorum load → only its
shard-leaderships re-elect, reads never stop, zero acked loss across
random kill points; plus a rebalance-verb test. Harness:
MultiProcCluster::start_sharded.
- tidal-stress drives the single path (WritePath::Leader|Sharded gone),
spreading writes round-robin across gateways or pinning --leader-url.
- Throughput: local 3×3 sustains 3,000 quorum signal-writes/s @ 0% err,
~30% CPU, lag ~0 (generator-bound). ≥5,000/s + ≥2.5× scaling is Ref-A.
Known follow-up (tracked): per-group-aware node readiness and cross-node
read fan-out under PARTIAL placement.
353 lines
14 KiB
Rust
353 lines
14 KiB
Rust
//! Machine-readable `OpenAPI` 3.1 specification for the tidalDB HTTP API.
|
|
//!
|
|
//! Two [`OpenApi`](utoipa::OpenApi) documents are derived from the
|
|
//! `#[utoipa::path(...)]` attributes on the handlers and the
|
|
//! `#[derive(ToSchema)]` on the DTOs:
|
|
//!
|
|
//! * [`StandaloneApiDoc`] — the data + health surface served by
|
|
//! [`crate::router::build_router`].
|
|
//! * [`ClusterApiDoc`] — the same surface PLUS the cluster-management and
|
|
//! sharded (scatter-gather) routes served by
|
|
//! [`crate::cluster::build_cluster_router`].
|
|
//!
|
|
//! Both are served UNAUTHENTICATED at `GET /openapi.json` (sibling to the
|
|
//! `/health/*` probes): the document describes the API contract only — it
|
|
//! carries no entity data, signals, or secrets — so gating it behind the same
|
|
//! bearer token a client needs the spec to learn how to send would be
|
|
//! self-defeating. The `bearerAuth` security scheme is declared via the
|
|
//! [`SecurityAddon`] modifier so generated clients know the data routes require
|
|
//! a token; the spec endpoint itself is exempt.
|
|
|
|
use axum::Json;
|
|
use utoipa::{
|
|
Modify, OpenApi,
|
|
openapi::security::{Http, HttpAuthScheme, SecurityScheme},
|
|
};
|
|
|
|
/// `GET /openapi.json` handler for the **standalone** surface.
|
|
///
|
|
/// Mounted unauthenticated alongside the `/health/*` probes by
|
|
/// [`crate::router::build_router`]. Returns the derived `OpenAPI` 3.1 document
|
|
/// as JSON (`utoipa::openapi::OpenApi` is `Serialize`).
|
|
pub(crate) async fn serve_standalone() -> Json<utoipa::openapi::OpenApi> {
|
|
Json(StandaloneApiDoc::openapi())
|
|
}
|
|
|
|
/// `GET /openapi.json` handler for the **cluster** surface — the superset
|
|
/// document that also describes the `/cluster/*` and `/sharded/*` routes.
|
|
/// Mounted by [`crate::cluster::build_cluster_router`].
|
|
pub(crate) async fn serve_cluster() -> Json<utoipa::openapi::OpenApi> {
|
|
Json(ClusterApiDoc::openapi())
|
|
}
|
|
|
|
/// `GET /openapi.json` handler for the **multi-process region node** surface
|
|
/// (`--region`). Distinct from [`serve_cluster`]: a region node owns one region
|
|
/// but forwards/broadcasts/aggregates across the cluster, and its `/items` /
|
|
/// `/signals` paths carry region-local routing text — so it is a separate
|
|
/// document, not the single-process superset. Mounted by
|
|
/// [`crate::cluster::build_region_router`].
|
|
pub(crate) async fn serve_region() -> Json<utoipa::openapi::OpenApi> {
|
|
Json(RegionApiDoc::openapi())
|
|
}
|
|
|
|
/// Injects the `bearerAuth` HTTP security scheme into the components.
|
|
///
|
|
/// The per-handler `security(("bearerAuth" = []))` attributes reference this
|
|
/// scheme by name; without registering it here the generated document would
|
|
/// name an undefined scheme. Health and `/openapi.json` carry no `security`
|
|
/// requirement, so they remain open.
|
|
struct SecurityAddon;
|
|
|
|
impl Modify for SecurityAddon {
|
|
fn modify(&self, openapi: &mut utoipa::openapi::OpenApi) {
|
|
// `components` is always `Some` once any schema is registered, but guard
|
|
// rather than unwrap so an empty-component build degrades gracefully
|
|
// (the crate forbids `unwrap`/`expect` in non-test code).
|
|
let components = openapi.components.get_or_insert_with(Default::default);
|
|
components.add_security_scheme(
|
|
"bearerAuth",
|
|
SecurityScheme::Http(Http::new(HttpAuthScheme::Bearer)),
|
|
);
|
|
}
|
|
}
|
|
|
|
/// `OpenAPI` document for the standalone HTTP surface.
|
|
#[derive(OpenApi)]
|
|
#[openapi(
|
|
info(
|
|
title = "tidalDB HTTP API",
|
|
version = env!("CARGO_PKG_VERSION"),
|
|
description = "Embeddable, single-node-first database for the personalized \
|
|
content ranking problem. Write items, embeddings, and signals; \
|
|
retrieve ranked feeds and run hybrid search. Data routes require \
|
|
a Bearer token when TIDAL_API_KEY is configured; health probes \
|
|
and this spec do not.",
|
|
),
|
|
paths(
|
|
crate::router::health,
|
|
crate::router::create_item,
|
|
crate::router::write_embedding,
|
|
crate::router::write_signal,
|
|
crate::router::feed,
|
|
crate::router::search,
|
|
),
|
|
components(schemas(
|
|
crate::dto::ItemRequest,
|
|
crate::dto::EmbeddingRequest,
|
|
crate::dto::SignalRequest,
|
|
crate::dto::FeedResponse,
|
|
crate::dto::FeedItem,
|
|
crate::dto::SignalValue,
|
|
crate::dto::SearchResponse,
|
|
crate::dto::SearchItem,
|
|
)),
|
|
modifiers(&SecurityAddon),
|
|
tags(
|
|
(name = "health", description = "Liveness / readiness probes (unauthenticated)."),
|
|
(name = "data", description = "Item, embedding, signal writes and ranked reads."),
|
|
),
|
|
)]
|
|
pub struct StandaloneApiDoc;
|
|
|
|
/// `OpenAPI` document for the cluster HTTP surface.
|
|
///
|
|
/// Superset of [`StandaloneApiDoc`]: the same data + health routes plus the
|
|
/// cluster-management (`/cluster/*`) and sharded scatter-gather
|
|
/// (`/sharded/*`) routes. The cluster handlers register the SAME `/items`,
|
|
/// `/embeddings`, `/signals`, `/feed`, `/search` paths as the standalone ones;
|
|
/// because this is a distinct document the shared paths are described once here
|
|
/// from the cluster handlers' attributes (region-aware response text), so there
|
|
/// is no path-key collision.
|
|
#[derive(OpenApi)]
|
|
#[openapi(
|
|
info(
|
|
title = "tidalDB HTTP API",
|
|
version = env!("CARGO_PKG_VERSION"),
|
|
description = "Cluster-mode tidalDB HTTP surface: the standalone data + health \
|
|
routes plus multi-region cluster management and sharded \
|
|
scatter-gather routes. EXPERIMENTAL — all regions run in one \
|
|
process (no host/process isolation).",
|
|
),
|
|
paths(
|
|
crate::cluster::routes::cluster_status,
|
|
crate::cluster::routes::cluster_promote,
|
|
crate::cluster::routes::cluster_partition,
|
|
crate::cluster::routes::cluster_heal,
|
|
crate::cluster::routes::create_item,
|
|
crate::cluster::routes::write_embedding,
|
|
crate::cluster::routes::write_signal,
|
|
crate::cluster::routes::feed,
|
|
crate::cluster::routes::search,
|
|
crate::cluster::routes::sharded_create_item,
|
|
crate::cluster::routes::sharded_write_embedding,
|
|
crate::cluster::routes::sharded_write_signal,
|
|
crate::cluster::routes::sharded_feed,
|
|
crate::cluster::routes::sharded_search,
|
|
),
|
|
components(schemas(
|
|
crate::dto::ItemRequest,
|
|
crate::dto::EmbeddingRequest,
|
|
crate::dto::SignalRequest,
|
|
crate::dto::FeedResponse,
|
|
crate::dto::FeedItem,
|
|
crate::dto::SignalValue,
|
|
crate::dto::SearchResponse,
|
|
crate::dto::SearchItem,
|
|
crate::cluster::routes::ClusterStatusResponse,
|
|
crate::cluster::routes::RegionStatus,
|
|
crate::cluster::routes::RegionRequest,
|
|
crate::cluster::routes::ScatterGatherInfo,
|
|
crate::cluster::routes::ShardedFeedResponse,
|
|
crate::cluster::routes::ShardedSearchResponse,
|
|
)),
|
|
modifiers(&SecurityAddon),
|
|
tags(
|
|
(name = "health", description = "Liveness / readiness probes (unauthenticated)."),
|
|
(name = "data", description = "Item, embedding, signal writes and ranked reads."),
|
|
(name = "cluster", description = "Multi-region cluster management."),
|
|
(name = "sharded", description = "Scatter-gather routes fanning across all shards."),
|
|
),
|
|
)]
|
|
pub struct ClusterApiDoc;
|
|
|
|
/// `OpenAPI` document for the multi-process region node HTTP surface (`--region`).
|
|
///
|
|
/// A region node owns ONE region but presents the cluster as a single coherent
|
|
/// surface (m8p10 task 03): it serves the data routes (`/items`, `/embeddings`,
|
|
/// `/signals`, `/hardnegs`, `/feed`, `/search`) with leader-forwarding +
|
|
/// broadcast, aggregated `/cluster/status` plus the local-management routes
|
|
/// (`/cluster/promote`, `/cluster/partition`, `/cluster/heal`,
|
|
/// `/cluster/reconcile`, `/cluster/reconcile/snapshot`), and the cross-process
|
|
/// `/sharded/*` scatter-gather routes. A SEPARATE document from
|
|
/// [`ClusterApiDoc`] because the shared paths carry region-local routing text.
|
|
#[derive(OpenApi)]
|
|
#[openapi(
|
|
info(
|
|
title = "tidalDB HTTP API",
|
|
version = env!("CARGO_PKG_VERSION"),
|
|
description = "Multi-process cluster region node: this process owns ONE region and \
|
|
peers with siblings over real gRPC, presenting the cluster as one \
|
|
coherent HTTP surface (leader forwarding, broadcast, status \
|
|
aggregation, cross-process reconcile, and sharded scatter-gather). \
|
|
EXPERIMENTAL — real process isolation, but no quorum-ack writes or \
|
|
automatic failure detection yet.",
|
|
),
|
|
paths(
|
|
crate::cluster::node::status_local,
|
|
crate::cluster::node::cluster_status,
|
|
crate::cluster::node::cluster_promote,
|
|
crate::cluster::node::cluster_partition,
|
|
crate::cluster::node::cluster_heal,
|
|
crate::cluster::node::cluster_reseed,
|
|
crate::cluster::node::shard_replicas,
|
|
crate::cluster::node::shard_transfer,
|
|
crate::cluster::node::cluster_reconcile,
|
|
crate::cluster::node::cluster_reconcile_snapshot,
|
|
crate::cluster::node::create_item,
|
|
crate::cluster::node::write_embedding,
|
|
crate::cluster::node::write_signal,
|
|
crate::cluster::node::write_hardneg,
|
|
crate::cluster::node::feed,
|
|
crate::cluster::node::search,
|
|
crate::cluster::node::sharded_create_item,
|
|
crate::cluster::node::sharded_write_embedding,
|
|
crate::cluster::node::sharded_write_signal,
|
|
crate::cluster::node::sharded_feed,
|
|
crate::cluster::node::sharded_search,
|
|
),
|
|
components(schemas(
|
|
crate::dto::ItemRequest,
|
|
crate::dto::EmbeddingRequest,
|
|
crate::dto::SignalRequest,
|
|
crate::dto::FeedResponse,
|
|
crate::dto::FeedItem,
|
|
crate::dto::SignalValue,
|
|
crate::dto::SearchResponse,
|
|
crate::dto::SearchItem,
|
|
crate::cluster::node::LocalStatusResponse,
|
|
crate::cluster::node::AggregatedStatusResponse,
|
|
crate::cluster::node::AggregatedRegionStatus,
|
|
crate::cluster::node::RegionRequest,
|
|
crate::cluster::node::ShardReplicaChange,
|
|
crate::cluster::node::HardNegRequest,
|
|
crate::cluster::node::ReconcileResponse,
|
|
crate::cluster::node::ReconcileSnapshotResponse,
|
|
crate::cluster::routes::ScatterGatherInfo,
|
|
crate::cluster::routes::ShardedFeedResponse,
|
|
crate::cluster::routes::ShardedSearchResponse,
|
|
)),
|
|
modifiers(&SecurityAddon),
|
|
tags(
|
|
(name = "health", description = "Liveness / readiness probes (unauthenticated)."),
|
|
(name = "data", description = "Item, embedding, signal, hard-negative writes and ranked reads."),
|
|
(name = "cluster", description = "Cluster management, status aggregation, and reconcile."),
|
|
(name = "sharded", description = "Cross-process scatter-gather routes fanning across all regions."),
|
|
),
|
|
)]
|
|
pub struct RegionApiDoc;
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
/// The standalone document must enumerate the data + health paths and carry
|
|
/// the bearerAuth scheme so generated clients know how to authenticate.
|
|
#[test]
|
|
fn standalone_doc_lists_core_paths() {
|
|
let doc = StandaloneApiDoc::openapi();
|
|
let paths = &doc.paths.paths;
|
|
for p in [
|
|
"/health",
|
|
"/items",
|
|
"/embeddings",
|
|
"/signals",
|
|
"/feed",
|
|
"/search",
|
|
] {
|
|
assert!(paths.contains_key(p), "standalone doc missing path {p}");
|
|
}
|
|
// The cluster-only routes must NOT leak into the standalone document.
|
|
assert!(
|
|
!paths.contains_key("/cluster/status"),
|
|
"standalone doc must not advertise cluster routes"
|
|
);
|
|
let schemes = &doc
|
|
.components
|
|
.as_ref()
|
|
.expect("components present after schema registration")
|
|
.security_schemes;
|
|
assert!(
|
|
schemes.contains_key("bearerAuth"),
|
|
"SecurityAddon must register the bearerAuth scheme"
|
|
);
|
|
}
|
|
|
|
/// The cluster document is a superset: data + health + cluster + sharded.
|
|
#[test]
|
|
fn cluster_doc_lists_cluster_and_sharded_paths() {
|
|
let doc = ClusterApiDoc::openapi();
|
|
let paths = &doc.paths.paths;
|
|
for p in [
|
|
"/feed",
|
|
"/cluster/status",
|
|
"/cluster/promote",
|
|
"/cluster/partition",
|
|
"/cluster/heal",
|
|
"/sharded/items",
|
|
"/sharded/feed",
|
|
"/sharded/search",
|
|
] {
|
|
assert!(paths.contains_key(p), "cluster doc missing path {p}");
|
|
}
|
|
}
|
|
|
|
/// The multi-process region document enumerates the region node's full
|
|
/// cross-process surface (task 03): data routes with forwarding, aggregated
|
|
/// `/cluster/status`, the reconcile pair, and the `/sharded/*` scatter-gather
|
|
/// routes — and carries the bearerAuth scheme.
|
|
#[test]
|
|
fn region_doc_lists_region_paths() {
|
|
let doc = RegionApiDoc::openapi();
|
|
let paths = &doc.paths.paths;
|
|
for p in [
|
|
"/items",
|
|
"/embeddings",
|
|
"/signals",
|
|
"/hardnegs",
|
|
"/feed",
|
|
"/search",
|
|
"/cluster/status/local",
|
|
"/cluster/status",
|
|
"/cluster/promote",
|
|
"/cluster/partition",
|
|
"/cluster/heal",
|
|
"/cluster/reconcile",
|
|
"/cluster/reconcile/snapshot",
|
|
"/sharded/items",
|
|
"/sharded/embeddings",
|
|
"/sharded/signals",
|
|
"/sharded/feed",
|
|
"/sharded/search",
|
|
] {
|
|
assert!(paths.contains_key(p), "region doc missing path {p}");
|
|
}
|
|
let schemes = &doc
|
|
.components
|
|
.as_ref()
|
|
.expect("components present after schema registration")
|
|
.security_schemes;
|
|
assert!(
|
|
schemes.contains_key("bearerAuth"),
|
|
"region doc must register the bearerAuth scheme"
|
|
);
|
|
}
|
|
|
|
/// The served version string is the crate version, exposed for the route's
|
|
/// contract test and the verification report.
|
|
#[test]
|
|
fn doc_version_is_crate_version() {
|
|
let doc = StandaloneApiDoc::openapi();
|
|
assert_eq!(doc.info.version, env!("CARGO_PKG_VERSION"));
|
|
}
|
|
}
|