jx12n
|
3bcfb3c576
|
feat: Bazel build, crate docs/ai-lookup, docker images, and engine hardening
- Add BUILD.bazel across tidal, tidal-net, tidal-server, tidalctl for bzlmod build
- Add tidal/ crate docs (README, CHANGELOG, CONTRIBUTING, AGENTS, CLAUDE, API, ARCHITECTURE) and ai-lookup reference
- Add docker standalone/cluster/deploy images, compose, and prometheus config
- Harden WAL (batch format, writer, dedup, diagnostics), text syncer/collectors, and vector registry
- Expand tidalctl CLI and tests; restructure WAL/visibility integration test suites
- Refine tidal-net transport/client/server and tidal-server cluster/scatter-gather
|
2026-06-07 18:29:38 -06:00 |
|
jordan.washburn
|
fe711870be
|
feat: M8 phases 7-10 — gRPC transport, cluster server, scatter-gather, multi-node UAT
Delivers the distributed fabric's network layer and HTTP cluster surface:
**m8p7: tidal-net crate (gRPC transport)**
- GrpcTransport implementing Transport trait via tonic 0.12
- Per-peer circuit breaker (Closed/Open/HalfOpen), mutual TLS via rustls
- Boxed error types (clippy-clean), graceful mutex recovery, debug_assert
against calling block_on from tokio context
- Proto: WalShipping service (ShipSegment, StreamSegments stub, Heartbeat)
- 19 tests: contract, mTLS, reconnection, multi-node UAT, benchmarks
**m8p8: cluster subcommand + HTTP routes**
- ClusterState wrapping SimulatedCluster with region name mapping
- Routes: /health, /cluster/status, /cluster/promote, /partition, /heal
- Data routes: /items, /embeddings, /signals, /feed, /search (region-aware)
- Ranking profiles wired through ClusterConfig to all cluster nodes
- Topology YAML config, docker/cluster/Dockerfile (ENTRYPOINT+CMD, non-root)
**m8p9: scatter-gather query routing**
- Entity-sharded writes via Knuth multiplicative hash
- Scatter-gather RETRIEVE and SEARCH with deadline propagation (50ms-5ms)
- Partial failure: degraded=true with unavailable_shards metadata
- 6 tests: distribution, determinism, multi-shard retrieve, degraded
partial results, deadline propagation, scatter-gather search
**m8p10: gRPC transport integration tests**
- 8 tests over real gRPC: replication convergence, idempotent replay,
mixed signals, 3-node fan-out, partition/heal, degraded follower, perf
- Documented as tier-2 (in-process+gRPC); tier-3 multi-process pending
|
2026-04-11 13:51:08 -06:00 |
|
jordan
|
a0a33f4d9a
|
feat: harden tidal-server for production (Weeks 1–3)
Week 1 — deployment prerequisites:
- Add TIDAL_API_KEY Bearer auth middleware (constant-time comparison)
- Handle SIGTERM alongside ctrl-c for graceful shutdown
- Remove test-utils feature from production tidal-server binary
- Fix standalone Dockerfile; add cluster Dockerfile and docker-compose
- Extract MultiRegionState into state.rs with per-region TidalDb map
Week 2 — operational middleware and observability:
- Add body limit (2MB), request timeout (30s), concurrency limit (100)
- Add SetRequestIdLayer + PropagateRequestIdLayer (x-request-id header)
- Add TraceLayer with structured spans including request ID
- Activate Prometheus /metrics endpoint via --metrics flag
- Add monitoring.md, recovery.md, prometheus-alerts.yaml, grafana-dashboard.json
Week 3 — query latency histograms and middleware integration tests:
- Add QUERY_LATENCY_BOUNDS (100µs–10s) histogram to tidal library
- Instrument retrieve() and search() with tidaldb_retrieve/search_latency_us
- Fix: search() latency now recorded on error paths (was skipped via ?)
- Lib+bin split in tidal-server enabling integration tests
- Add 8 middleware integration tests (auth, body limit, request ID)
- Add 2 Prometheus alert rules and 2 Grafana latency panels
Post-review fixes:
- Fix SIGTERM handler compilation on non-Unix targets (#[cfg(unix)] guard)
- Exempt /health from TimeoutLayer + ConcurrencyLimitLayer (prevents false liveness failures under load)
- Case-insensitive Bearer scheme matching per RFC 7235 §2.1
|
2026-02-27 20:32:39 -07:00 |
|