44b768b8c6
End the "replicated XOR sharded" split: S shard groups, each a
replication group at RF with its own elected leader, leaders balanced
across nodes; any gateway hash-routes.
- One unified write surface: /items,/embeddings,/signals hash-route to
the owning shard group's leader (ShardRouter FNV-1a) AND replicate at
RF. x-tidal-ack/x-tidal-seq, quorum await, NotLeader/QuorumTimeout are
per-group; NotLeader names the group.
- Rebalance verbs (L3): POST /cluster/shards/{id}/transfer (fenced
leadership move) + /cluster/shards/{id}/replicas (add/remove replica).
A ?shard= selector threads through every per-shard admin verb and is
propagated on intra-group forwards (ShardReplica::admin_path). S=1 is
byte-for-byte (no selector, no shard in NotLeader body).
- Tier-3 exit gate (cluster_sharding.rs): 3 nodes × 3 shards × RF=3 over
real OS processes — SIGKILL a node under ack=quorum load → only its
shard-leaderships re-elect, reads never stop, zero acked loss across
random kill points; plus a rebalance-verb test. Harness:
MultiProcCluster::start_sharded.
- tidal-stress drives the single path (WritePath::Leader|Sharded gone),
spreading writes round-robin across gateways or pinning --leader-url.
- Throughput: local 3×3 sustains 3,000 quorum signal-writes/s @ 0% err,
~30% CPU, lag ~0 (generator-bound). ≥5,000/s + ≥2.5× scaling is Ref-A.
Known follow-up (tracked): per-group-aware node readiness and cross-node
read fan-out under PARTIAL placement.
82 lines
3.2 KiB
Rust
82 lines
3.2 KiB
Rust
//! Hot-path micro-benchmarks for the load generator's OWN per-request cost.
|
|
//!
|
|
//! These run with NO server and NO network, so they isolate exactly the work
|
|
//! the generator does per dispatched request — request construction
|
|
//! ([`Workload::next`]) and latency aggregation ([`LatencyHistogram::record`]).
|
|
//! That is the only honest way to answer "is this allocation material?": the
|
|
//! number here is the generator's CPU floor, the thing that must stay far below
|
|
//! the network round-trip it measures, and the before/after the perf sweep is
|
|
//! graded against.
|
|
|
|
use std::hint::black_box;
|
|
use std::time::Duration;
|
|
|
|
use criterion::{Criterion, criterion_group, criterion_main};
|
|
|
|
use tidal_stress::metrics::LatencyHistogram;
|
|
use tidal_stress::workload::{Workload, parse_mix};
|
|
|
|
fn make_workload(mix: &str) -> Workload {
|
|
Workload::new(
|
|
// Three read gateways (round-robin), writes pinned to one gateway =>
|
|
// write_bases len == 1, the headline `--leader-url` path.
|
|
vec![
|
|
"http://10.0.0.1:9500".into(),
|
|
"http://10.0.0.2:9500".into(),
|
|
"http://10.0.0.3:9500".into(),
|
|
],
|
|
vec!["http://10.0.0.1:9500".into()],
|
|
parse_mix(mix).expect("mix preset parses"),
|
|
10_000, // corpus
|
|
50_000, // users
|
|
1.3, // hot_skew
|
|
24, // feed_limit
|
|
128, // embedding_dim
|
|
)
|
|
}
|
|
|
|
/// `Workload::next` — the per-dispatched-request construction cost (URL String +
|
|
/// body). `peach` is the headline signal-dominated mix; `writes` forces the body
|
|
/// path on every call; `reads` isolates the URL-only feed/search path.
|
|
fn bench_next(c: &mut Criterion) {
|
|
let wl_peach = make_workload("peach");
|
|
let wl_writes = make_workload("writes");
|
|
let wl_reads = make_workload("reads");
|
|
let mut rng = rand::rng();
|
|
|
|
let mut g = c.benchmark_group("workload_next");
|
|
g.bench_function("peach", |b| b.iter(|| black_box(wl_peach.next(&mut rng))));
|
|
g.bench_function("writes", |b| b.iter(|| black_box(wl_writes.next(&mut rng))));
|
|
g.bench_function("reads", |b| b.iter(|| black_box(wl_reads.next(&mut rng))));
|
|
g.finish();
|
|
}
|
|
|
|
/// `LatencyHistogram::record` — the per-OK-request collector cost (single-writer,
|
|
/// allocation-free), plus the per-stage `percentile` report cost.
|
|
fn bench_histogram(c: &mut Criterion) {
|
|
c.bench_function("histogram_record", |b| {
|
|
let mut h = LatencyHistogram::default();
|
|
// Cheap LCG so successive records land in different buckets rather than
|
|
// hammering one — closer to a real latency distribution.
|
|
let mut state = 0x2545_F491_4F6C_DD1Du64;
|
|
b.iter(|| {
|
|
state = state
|
|
.wrapping_mul(6_364_136_223_846_793_005)
|
|
.wrapping_add(1_442_695_040_888_963_407);
|
|
let ns = (state >> 33) % 200_000_000 + 1; // ~1ns..200ms
|
|
h.record(black_box(Duration::from_nanos(ns)));
|
|
});
|
|
});
|
|
|
|
let mut h = LatencyHistogram::default();
|
|
for i in 1..=100_000u64 {
|
|
h.record(Duration::from_nanos(i * 1_500));
|
|
}
|
|
c.bench_function("histogram_p99", |b| {
|
|
b.iter(|| black_box(h.percentile(black_box(0.99))));
|
|
});
|
|
}
|
|
|
|
criterion_group!(benches, bench_next, bench_histogram);
|
|
criterion_main!(benches);
|