Commit Graph

14 Commits

Author SHA1 Message Date
jx12n
bd2b270dee build(standalone): committed multi-arch pullable standalone dev image path
Consumers (e.g. thepeach) need a PULLABLE tidaldb image — not every dev has the
source for a compose `build:` context. Publishes registry.threesix.ai/tidal/standalone
as a manifest list: linux/amd64 (Linux/CI) + linux/arm64 (Apple-silicon Macs),
defaulting to STANDALONE mode (:9400, --data-dir /data) so it drops into a consumer's
docker-compose like postgres.

Hybrid build because `rustc` SIGSEGVs under QEMU (an in-container amd64 cross-build on
an arm64 host fails):
- amd64: docker/standalone/amd64.Dockerfile packages the HOST cross-compiled x86_64
  binary (the proven build-release path; trixie-slim for glibc 2.41/libmvec) — apt+COPY only.
- arm64: native in-container build from docker/standalone/Dockerfile.
- scripts/build-standalone-image.sh stitches both into one manifest list (:m12 + :latest).

Verified: pulled :m12 (arm64), booted standalone, GET /health -> {ok:true,mode:standalone}.
2026-06-19 21:34:06 -06:00
jx12n
580142df49 feat(m12): election-divergence-fix + soak-eval streak + release tooling
Durable `leader_acked` frontier in `ShardReplica` tracks the highest seqno
acked under `ack=leader` (journal-only, un-replicated); `decide_join` now
quarantines on THIS node's own frontier rather than comparing stream numbers
across stream boundaries — eliminates false-quarantine churn on rolling
restarts. `SHUTDOWN_HANDOFF_WAIT` (3s) drains the leader's tail to quorum
before step-down so the next leader inherits a clean prefix. New
`load_leader_acked`/`persist_leader_acked` helpers; `cluster_reseed.rs` gains
the divergence-fix regression suite; `replication_ops.rs` threads the signal.

Soak-eval: `tidal_stress::soak_eval` + `soak-eval` binary implement the
30-night streak (ledger.tsv × restarts.tsv → streak.tsv); monitor and nightly
CronJob k8s YAMLs updated; phase-9 doc clarifies the dual-stream streak
definition (ledger PASS AND zero pod restarts in window). `run-reliability.sh`
gates the election-divergence suite before any k8s push.

Release tooling: `docker/release/` multi-stage Dockerfile + DR image;
`scripts/build-release.sh` single repeatable cross-compile+buildx path.
2026-06-18 13:08:53 -06:00
jx12n
f640764d89 feat(tidal-stress): open-loop capacity load generator (thepeach feed workload)
New workspace crate: an open-loop, coordinated-omission-corrected HTTP load
generator + capacity ramp for the standalone and multi-process cluster surfaces,
modeling a thepeach feed session (feed reads + view/like/skip signals + search,
signal-dominated per their user-graph spec). Throttleable target rate, ramp
presets (smoke/quick/peach-100k/max) or rps:secs specs, peach/reads/writes/custom
mixes, leader vs sharded write paths, per-op p50/p90/p99/p999/max latency, a
backpressure-aware status breakdown (429/408/503/4xx/5xx/transport), and a verdict
translated to supported DAU. Runs in-cluster as a k8s Job (tidal-stress/k8s/).

Open-loop scheduler (scheduler.rs) fires at a fixed arrival rate and measures
latency from each request's intended send time, so a server stall inflates the
percentiles a closed-loop test hides; it shed-and-counts rather than blocking when
the in-flight cap is reached. Pure-Rust (tokio + reqwest/rustls), no engine deps.

Findings on the live 3-region k3s cluster (docs/ops/stress-test-thepeach.md):
reads scale to thousands/s at <15ms p99; the replicated /signals path saturates at
~90 signals/s (single-leader funnel + 2-worker write pool + synchronous gRPC ship);
the sharded path sustains 3,669 signals/s at 0 errors and ~27% cluster CPU (≈ the
100k-DAU peak, knee not reached). Overload degrades gracefully (429; 0 pod
restarts). thepeach's planned in-process embedding sidesteps all of it (write ≈82ns).
2026-06-10 21:54:21 -06:00
jx12n
1092d34c39 feat: kubernetes deployment, OpenAPI spec, guides, and docker consolidation
- Add k8s/ manifests (StatefulSet, kustomize, PDB, ServiceMonitor) + docs/runbooks/kubernetes.md
- Add tidal-server/src/openapi.rs (utoipa OpenAPI spec) and wire into router
- Add docs/guides/ (build-a-feed-app, embeddings, server-deployment) + foryou_feed example
- Consolidate tidal/docker/ into root docker/ (single canonical home)
- Update API.md, QUICKSTART.md, README.md, CLAUDE.md, check-docs.sh accordingly
2026-06-09 17:06:34 -06:00
jx12n
69ae6e64a1 fix: M0-M10 remediation — deferred post-filters, WAL hardening, docker build fixes
- Extract shared deferred post-filter (InCollection/MinSignal/MaxSignal/
  NearLocation/SocialGraph) into query/executor/post_filter.rs so RETRIEVE and
  SEARCH apply identical predicates; SEARCH previously ignored four variants
- Harden WAL writer/compaction/dedup paths
- Brute-force vector store fixes + tests
- Governance community ledger updates
- Pin docker builders to bookworm (glibc/libmvec match) and add
  protobuf-compiler for tidal-net's tonic-build
- Add M0-M10 code-review document
2026-06-07 20:47:17 -06:00
jordan.washburn
fe711870be feat: M8 phases 7-10 — gRPC transport, cluster server, scatter-gather, multi-node UAT
Delivers the distributed fabric's network layer and HTTP cluster surface:

**m8p7: tidal-net crate (gRPC transport)**
- GrpcTransport implementing Transport trait via tonic 0.12
- Per-peer circuit breaker (Closed/Open/HalfOpen), mutual TLS via rustls
- Boxed error types (clippy-clean), graceful mutex recovery, debug_assert
  against calling block_on from tokio context
- Proto: WalShipping service (ShipSegment, StreamSegments stub, Heartbeat)
- 19 tests: contract, mTLS, reconnection, multi-node UAT, benchmarks

**m8p8: cluster subcommand + HTTP routes**
- ClusterState wrapping SimulatedCluster with region name mapping
- Routes: /health, /cluster/status, /cluster/promote, /partition, /heal
- Data routes: /items, /embeddings, /signals, /feed, /search (region-aware)
- Ranking profiles wired through ClusterConfig to all cluster nodes
- Topology YAML config, docker/cluster/Dockerfile (ENTRYPOINT+CMD, non-root)

**m8p9: scatter-gather query routing**
- Entity-sharded writes via Knuth multiplicative hash
- Scatter-gather RETRIEVE and SEARCH with deadline propagation (50ms-5ms)
- Partial failure: degraded=true with unavailable_shards metadata
- 6 tests: distribution, determinism, multi-shard retrieve, degraded
  partial results, deadline propagation, scatter-gather search

**m8p10: gRPC transport integration tests**
- 8 tests over real gRPC: replication convergence, idempotent replay,
  mixed signals, 3-node fan-out, partition/heal, degraded follower, perf
- Documented as tier-2 (in-process+gRPC); tier-3 multi-process pending
2026-04-11 13:51:08 -06:00
Alan Kahn
889c746bc1 fix: add g++ to deploy Dockerfile for cc-rs compilation
rust:1.91-slim doesn't include a C++ compiler, which cc-rs needs.
The standalone/cluster Dockerfiles use the full rust:1.91 image so
they weren't affected.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 14:53:48 -05:00
Alan Kahn
dd6c709cbe feat: support PORT env var for listen address
Clap now reads PORT from the environment, accepting either a bare port
number (e.g. 8080 -> 0.0.0.0:8080) or a full host:port. CLI --listen
flag still takes precedence. Deploy Dockerfile defaults PORT=9500 and
removes the hardcoded --listen argument.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 12:25:58 -05:00
Alan Kahn
c12cde46e8 chore: rename schema mount from eros-schema.yaml to schema.yaml
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 10:06:28 -05:00
Alan Kahn
82925cee7c feat: add deploy Dockerfile, restore standalone to generic
docker/standalone/Dockerfile stays generic (port 9400, baked-in config,
auth healthcheck) for anyone running tidalDB independently.

docker/deploy/Dockerfile is purpose-built for k8s: port 9500, schema
mounted at runtime, persistent data dir, fixed UID for PVC ownership.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 10:03:01 -05:00
Alan Kahn
3101789d32 feat: k8s health lifecycle + standalone Dockerfile for internal deployment
Add /health/startup and /health/live probes, flip readiness to 503 on
SIGTERM so k8s stops routing before drain. Update standalone Dockerfile
for internal deployment: port 9500, schema mounted at runtime (not baked
in), persistent data dir, non-root user with fixed UID 10001.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 09:57:19 -05:00
jordan
a0a33f4d9a feat: harden tidal-server for production (Weeks 1–3)
Week 1 — deployment prerequisites:
- Add TIDAL_API_KEY Bearer auth middleware (constant-time comparison)
- Handle SIGTERM alongside ctrl-c for graceful shutdown
- Remove test-utils feature from production tidal-server binary
- Fix standalone Dockerfile; add cluster Dockerfile and docker-compose
- Extract MultiRegionState into state.rs with per-region TidalDb map

Week 2 — operational middleware and observability:
- Add body limit (2MB), request timeout (30s), concurrency limit (100)
- Add SetRequestIdLayer + PropagateRequestIdLayer (x-request-id header)
- Add TraceLayer with structured spans including request ID
- Activate Prometheus /metrics endpoint via --metrics flag
- Add monitoring.md, recovery.md, prometheus-alerts.yaml, grafana-dashboard.json

Week 3 — query latency histograms and middleware integration tests:
- Add QUERY_LATENCY_BOUNDS (100µs–10s) histogram to tidal library
- Instrument retrieve() and search() with tidaldb_retrieve/search_latency_us
- Fix: search() latency now recorded on error paths (was skipped via ?)
- Lib+bin split in tidal-server enabling integration tests
- Add 8 middleware integration tests (auth, body limit, request ID)
- Add 2 Prometheus alert rules and 2 Grafana latency panels

Post-review fixes:
- Fix SIGTERM handler compilation on non-Unix targets (#[cfg(unix)] guard)
- Exempt /health from TimeoutLayer + ConcurrencyLimitLayer (prevents false liveness failures under load)
- Case-insensitive Bearer scheme matching per RFC 7235 §2.1
2026-02-27 20:32:39 -07:00
Alan Kahn
928bcf98d6 fix: bump Rust version to 1.91 in cluster Dockerfile
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 15:00:48 -05:00
jordan
eca7765e8d fix: heal_region re-delivers missed WAL batches so partitioned followers converge immediately after heal
- Extract redeliver_missed(tx, db, log) helper into cluster_transport.rs
- heal_region now removes partition then immediately ships any missed
  batch-log entries to the healed follower's channel
- await_convergence refactored to call the same helper (no logic change)
- tidal-server: reload_text_index before search in cluster mode
- tidal-server: write_signal returns Result instead of panicking on unknown signal
- tidal-server: leader shows lag_events=0 (writes directly, no receiver thread)
- tidal-server: fix cluster mode error propagation (ServerError::from)
- docs/runbooks/cluster.md: add full cluster operations runbook
- docker/: add Dockerfile for containerised cluster deployment
- README.md: add tidal-server HTTP API getting-started section
- Split oversized source files per CODING_GUIDELINES §9

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-25 11:57:01 -07:00