# Stress test regression — p4 (m11p4) (2026-06-11)

Live 3-StatefulSet k3s cluster, m11p4 image (`sha256:86dd59c4...`). Fresh
deployment over existing PVCs (no reset needed — m11p4 WAL reader accepts m11p3
segments as implicit v0). All 3 pods Running, lag=0 at test start.

---

## Catch-up self-healing validation

Validates the m11p4 fix: failed/missed catch-up pulls now retry on a 30s timer
rather than waiting for the next push event.

**Procedure:**
1. Confirmed baseline lag=0 on all 3 nodes.
2. Killed tidaldb-eu-west-0; wrote 500 events to leader (ack=leader) while pod
   was down. Pod restarted via StatefulSet (fast restart).
3. Stopped all writes with eu-west at lag=2,035,769.
4. Polled `/cluster/status` every 10s — zero writes issued.

**Result:**

| t+ | eu-west lag |
|---|---|
| 0s (write-stop) | 2,035,769 |
| 10s | **0** |

eu-west caught up to lag=0 within 10s of pod start with no new writes to the
cluster. Prior to m11p4, this scenario left the follower stuck indefinitely
(the 2026-06-11 p3 rollout incident: both followers stuck at lag=136507 until
the cluster received new writes).

**Verdict: timer-triggered catch-up CONFIRMED.**

---

## Throughput regression (p3 gate re-check)

`tidal-stress` open-loop, `--ack quorum`, `--mix writes`, `peach-100k` ramp,
60s/stage, 20k corpus, 100k users, `--stop-on-knee`.

| Stage | Target rps | Signal ok/s | Error rate | p99 write | Max lag | Pass |
|---|---|---|---|---|---|---|
| 1 | 50 | 50 | 0.00% | 35 ms | 0 | ✓ |
| 2 | 150 | 150 | 0.00% | 35 ms | 0 | ✓ |
| 3 | 400 | 400 | 0.00% | 37 ms | 0 | ✓ |
| 4 | 800 | 800 | 0.00% | 43 ms | 8 | ✓ |
| 5 | 1,500 | 1,499 | 0.04% | 41 ms | 0 | ✓ |
| **6** | **3,000** | **2,975** | **0.82%** | **49 ms** | **27** | **✓ highest** |
| 7 ⚠ | 5,000 | 4,411 | 11.76% | 164 ms | 24 | ✗ knee |

No 503 quorum timeouts at any stage.

### Regression verdict

| Metric | T2-A (m11p3) | Regression (m11p4) | Delta | Result |
|---|---|---|---|---|
| Quorum signal-writes/s | 2,980/s | **2,975/s** | −0.2% | **✓ no regression** |
| Write p99 at stage 6 | 49 ms | **49 ms** | 0 | **✓ no regression** |
| Error rate at stage 6 | 0.67% | **0.82%** | +0.15% | **✓ within gate (<1%)** |
| Capacity knee | stage 7 (5k) | **stage 7 (5k)** | none | **✓ no regression** |
| Lag at stage 6 | 23 events | **27 events** | +4 | **✓ within gate (≤2s)** |

**p3 gate (≥1,000/s quorum, p99 ≤50ms, error <1%) holds through m11p4.**