tidaldb

jordan/tidaldb

Fork 0

Commit Graph

Author	SHA1	Message	Date
jx12n	3bcfb3c576	feat: Bazel build, crate docs/ai-lookup, docker images, and engine hardening - Add BUILD.bazel across tidal, tidal-net, tidal-server, tidalctl for bzlmod build - Add tidal/ crate docs (README, CHANGELOG, CONTRIBUTING, AGENTS, CLAUDE, API, ARCHITECTURE) and ai-lookup reference - Add docker standalone/cluster/deploy images, compose, and prometheus config - Harden WAL (batch format, writer, dedup, diagnostics), text syncer/collectors, and vector registry - Expand tidalctl CLI and tests; restructure WAL/visibility integration test suites - Refine tidal-net transport/client/server and tidal-server cluster/scatter-gather	2026-06-07 18:29:38 -06:00
jordan.washburn	b16025b8b2	feat: pluggable cluster transport + multi-process E2E test harness Three changes closing M8 gaps identified during verification: 1. ROADMAP.md: Mark m8p8 and m8p10 as PARTIAL (not COMPLETE). Added Known Gaps table (G1: in-process transport, G2: tier-3 tests, G3: hash inconsistency). 2. SimulatedCluster transport now pluggable via ClusterConfig.transports. Default (None) uses new ChannelTransport (crossbeam, same behavior). When Some, accepts external transports (e.g., GrpcTransport from tidal-net). Updated redeliver_missed to use &dyn Transport. Zero regressions: all 1209 lib + 8 m8_uat tests pass unchanged. 3. Multi-process E2E test harness (tidal-server/tests/cluster_e2e.rs). ClusterHarness spawns real tidal-server cluster OS processes, allocates dynamic ports, generates topology YAML, polls health, and cleans up via SIGTERM. Two tests: smoke (write + converge + verify follower reads) and promote (leader change + continued writes). Feature-gated behind cluster-e2e.	2026-04-11 19:57:25 -06:00
jordan.washburn	fe711870be	feat: M8 phases 7-10 — gRPC transport, cluster server, scatter-gather, multi-node UAT Delivers the distributed fabric's network layer and HTTP cluster surface: m8p7: tidal-net crate (gRPC transport) - GrpcTransport implementing Transport trait via tonic 0.12 - Per-peer circuit breaker (Closed/Open/HalfOpen), mutual TLS via rustls - Boxed error types (clippy-clean), graceful mutex recovery, debug_assert against calling block_on from tokio context - Proto: WalShipping service (ShipSegment, StreamSegments stub, Heartbeat) - 19 tests: contract, mTLS, reconnection, multi-node UAT, benchmarks m8p8: cluster subcommand + HTTP routes - ClusterState wrapping SimulatedCluster with region name mapping - Routes: /health, /cluster/status, /cluster/promote, /partition, /heal - Data routes: /items, /embeddings, /signals, /feed, /search (region-aware) - Ranking profiles wired through ClusterConfig to all cluster nodes - Topology YAML config, docker/cluster/Dockerfile (ENTRYPOINT+CMD, non-root) m8p9: scatter-gather query routing - Entity-sharded writes via Knuth multiplicative hash - Scatter-gather RETRIEVE and SEARCH with deadline propagation (50ms-5ms) - Partial failure: degraded=true with unavailable_shards metadata - 6 tests: distribution, determinism, multi-shard retrieve, degraded partial results, deadline propagation, scatter-gather search m8p10: gRPC transport integration tests - 8 tests over real gRPC: replication convergence, idempotent replay, mixed signals, 3-node fan-out, partition/heal, degraded follower, perf - Documented as tier-2 (in-process+gRPC); tier-3 multi-process pending	2026-04-11 13:51:08 -06:00

Author

SHA1

Message

Date

jx12n

3bcfb3c576

feat: Bazel build, crate docs/ai-lookup, docker images, and engine hardening

- Add BUILD.bazel across tidal, tidal-net, tidal-server, tidalctl for bzlmod build
- Add tidal/ crate docs (README, CHANGELOG, CONTRIBUTING, AGENTS, CLAUDE, API, ARCHITECTURE) and ai-lookup reference
- Add docker standalone/cluster/deploy images, compose, and prometheus config
- Harden WAL (batch format, writer, dedup, diagnostics), text syncer/collectors, and vector registry
- Expand tidalctl CLI and tests; restructure WAL/visibility integration test suites
- Refine tidal-net transport/client/server and tidal-server cluster/scatter-gather

2026-06-07 18:29:38 -06:00

jordan.washburn

b16025b8b2

feat: pluggable cluster transport + multi-process E2E test harness

Three changes closing M8 gaps identified during verification:

1. ROADMAP.md: Mark m8p8 and m8p10 as PARTIAL (not COMPLETE).
   Added Known Gaps table (G1: in-process transport, G2: tier-3
   tests, G3: hash inconsistency).

2. SimulatedCluster transport now pluggable via ClusterConfig.transports.
   Default (None) uses new ChannelTransport (crossbeam, same behavior).
   When Some, accepts external transports (e.g., GrpcTransport from
   tidal-net). Updated redeliver_missed to use &dyn Transport.
   Zero regressions: all 1209 lib + 8 m8_uat tests pass unchanged.

3. Multi-process E2E test harness (tidal-server/tests/cluster_e2e.rs).
   ClusterHarness spawns real tidal-server cluster OS processes,
   allocates dynamic ports, generates topology YAML, polls health,
   and cleans up via SIGTERM. Two tests: smoke (write + converge +
   verify follower reads) and promote (leader change + continued writes).
   Feature-gated behind cluster-e2e.

2026-04-11 19:57:25 -06:00

jordan.washburn

fe711870be

feat: M8 phases 7-10 — gRPC transport, cluster server, scatter-gather, multi-node UAT

Delivers the distributed fabric's network layer and HTTP cluster surface:

**m8p7: tidal-net crate (gRPC transport)**
- GrpcTransport implementing Transport trait via tonic 0.12
- Per-peer circuit breaker (Closed/Open/HalfOpen), mutual TLS via rustls
- Boxed error types (clippy-clean), graceful mutex recovery, debug_assert
  against calling block_on from tokio context
- Proto: WalShipping service (ShipSegment, StreamSegments stub, Heartbeat)
- 19 tests: contract, mTLS, reconnection, multi-node UAT, benchmarks

**m8p8: cluster subcommand + HTTP routes**
- ClusterState wrapping SimulatedCluster with region name mapping
- Routes: /health, /cluster/status, /cluster/promote, /partition, /heal
- Data routes: /items, /embeddings, /signals, /feed, /search (region-aware)
- Ranking profiles wired through ClusterConfig to all cluster nodes
- Topology YAML config, docker/cluster/Dockerfile (ENTRYPOINT+CMD, non-root)

**m8p9: scatter-gather query routing**
- Entity-sharded writes via Knuth multiplicative hash
- Scatter-gather RETRIEVE and SEARCH with deadline propagation (50ms-5ms)
- Partial failure: degraded=true with unavailable_shards metadata
- 6 tests: distribution, determinism, multi-shard retrieve, degraded
  partial results, deadline propagation, scatter-gather search

**m8p10: gRPC transport integration tests**
- 8 tests over real gRPC: replication convergence, idempotent replay,
  mixed signals, 3-node fan-out, partition/heal, degraded follower, perf
- Documented as tier-2 (in-process+gRPC); tier-3 multi-process pending

2026-04-11 13:51:08 -06:00

3 Commits