# Benchmark — 1536-dim production shape, peach mix (2026-06-14)

Live 3-node m11p6 cluster (`m11-44b768b`), mTLS, single replication group,
ack=quorum, local-path PVCs, 2 vCPU / 2Gi per pod. Schema `content_vector`
**1536-dim** (text-embedding-3-small — thepeach production width). Mix `peach`:
feed-profile reads + search + signal writes in production ratio (writes dominate;
the feed read hits `/feed?profile=…` — the `db.retrieve(profile)` path thepeach
E2/R8 will call). 20k-item corpus, 8-stage ramp, 120s/stage.

## Per-stage (total rps / write p99 / feed-read p99 / error)

| Stage | Target | Achieved | write p99 | feed p99 | search p99 | error | verdict |
|------:|-------:|---------:|----------:|---------:|-----------:|------:|---------|
| 1 | 50   | 50    | 43ms | 9ms  | 45ms* | 0.00% | clean |
| 2 | 150  | 150   | 40ms | 6ms  | 5ms   | 0.00% | clean |
| 3 | 400  | 400   | 40ms | 8ms  | 6ms   | 0.00% | clean |
| 4 | 800  | 800   | 43ms | 9ms  | 7ms   | 0.00% | clean, comfortable |
| **5** | **1500** | **1498** | **57ms** | **11ms** | **9ms** | **0.14%** | **highest within SLO** |
| 6 | 3000 | 2976  | 4.56s | 170ms | 173ms | 6.67% | **knee — SLO breach** |
| 7 | 5000 | 3700  | 4.61s | 151ms | 153ms | 8.78% | saturated |
| 8 | 8000 | 3718  | 7.32s | 156ms | 165ms | 10.72% | saturated |

\* stage-1 search p99 is a cold-start single-sample artifact (97 requests).

## Findings

1. **1536-dim costs ~nothing on throughput vs 128-dim.** Knee is stage 6
   (~2,976 rps) — identical to the 128-dim run (2,981). The write bottleneck is
   quorum-commit + the 2-worker leader pool, **not** vector size. Larger
   embeddings did not move the throughput ceiling.

2. **The vector read path is cheap and is NOT the bottleneck.** Feed-profile
   retrieve stays p99 3–11ms through stage 5 and only ~170ms even past the knee,
   while writes blow up to 4.5s. The thing we were worried about — vector search
   at production width — is a non-issue for latency. Search p99 ≤ 9ms through 1500 rps.

3. **Quorum writes are the sole ceiling**, and it's CPU-bound on the leader's
   2-core / ~2-worker pool. Zero 503s at every stage (quorum never timed out);
   the knee is 429/in-flight-cap backpressure, not server quorum failure.
   Post-ramp lag=0, no pod restarts.

4. **Memory is the dim-sensitive resource.** 20k×1536 items → 567–751 MiB/pod
   (vs 229–408 MiB at 128-dim, ~12× per-vector). Modest at 20k corpus; the
   binding constraint at real corpus scale. Capacity-plan RAM = corpus × 1536 ×
   4B × index-overhead, not throughput.

## Recommended operating target

The knee is ~3,000 rps. For a **high-quality sustained target with real margin**
(absorbs spikes, survives a single-node failover that transiently ~halves write
capacity, keeps write p99 within the 50ms SLA):

**Target: ≤ 1,000 signal-ingest rps sustained (~1,200 rps full peach mix).**

- ~40% of the knee → 2.5× headroom; survives one node loss without breaching.
- Write p99 ~45ms (within the 50ms in-process SLA), feed p99 < 10ms, error ~0%.
- ≈ 23k DAU at a realistic 5× evening peak, or ≈ 117k DAU against average load.
- 1,500 rps is the *highest within SLO* but write p99 (57ms) and p999 (≈150ms)
  are at the edge — operate below it, not at it.

**Scale levers when traffic grows past this** (both available, both unproven —
gated behind T5): (a) more CPU per leader (2→8 workers, ~linear on the write
pool); (b) m11p6 sharding — hash-route writes across S groups for ~S× the knee.