jx12n 3bcfb3c576 feat: Bazel build, crate docs/ai-lookup, docker images, and engine hardening

- Add BUILD.bazel across tidal, tidal-net, tidal-server, tidalctl for bzlmod build
- Add tidal/ crate docs (README, CHANGELOG, CONTRIBUTING, AGENTS, CLAUDE, API, ARCHITECTURE) and ai-lookup reference
- Add docker standalone/cluster/deploy images, compose, and prometheus config
- Harden WAL (batch format, writer, dedup, diagnostics), text syncer/collectors, and vector registry
- Expand tidalctl CLI and tests; restructure WAL/visibility integration test suites
- Refine tidal-net transport/client/server and tidal-server cluster/scatter-gather

2026-06-07 18:29:38 -06:00

3.2 KiB

Raw Blame History

Task 07: Replication Integration Tests

Delivers

Integration tests in tidal/tests/m8p2_replication.rs covering the full replication stack: leader->follower segment delivery, decay score equivalence to 6 decimal places, follower read-only enforcement, lag convergence, and segment corruption rejection.

Complexity: M

Dependencies

Tasks 01-06 complete

Technical Design

// tidal/tests/m8p2_replication.rs

use tidaldb::{TidalDb, TidalDbBuilder, NodeRole, ShardId, RegionId, NodeConfig};
use tidaldb::replication::{InProcessTransportFactory, ReplicationLagGauge};

fn leader_config(data_dir: &Path) -> Config {
    Config {
        cluster: NodeConfig {
            role: NodeRole::Leader,
            shard_id: ShardId(0),
            ..Default::default()
        },
        ..Config::with_data_dir(data_dir)
    }
}

fn follower_config(data_dir: &Path) -> Config {
    Config {
        cluster: NodeConfig {
            role: NodeRole::Follower,
            shard_id: ShardId(0),
            ..Default::default()
        },
        ..Config::with_data_dir(data_dir)
    }
}

#[tokio::test]
async fn replication_decay_scores_match() {
    // Leader writes 1,000 signals.
    // Follower replays all segments.
    // Verify: read_decay_score on follower matches leader to 6 decimal places.
}

#[tokio::test]
async fn follower_rejects_writes() {
    // Open follower. Attempt signal() write.
    // Verify: returns TidalError::ReadOnly.
}

#[tokio::test]
async fn follower_serves_retrieve_queries() {
    // Leader writes items + signals.
    // Follower applies.
    // Follower.retrieve() returns ranked results.
}

#[tokio::test]
async fn replication_lag_converges_to_zero() {
    // Leader writes 500 segments.
    // Wait for follower to apply all.
    // Assert: lag_seqno(follower) == 0 within 5 seconds.
}

#[tokio::test]
async fn corrupted_segment_is_rejected() {
    // Manually corrupt BLAKE3 checksum in segment bytes.
    // Send to follower via transport.
    // Verify: segment is not applied (decay scores unchanged).
}

#[tokio::test]
async fn leader_restart_follower_continues() {
    // Leader writes 100 signals.
    // Leader shuts down.
    // Follower serves read queries from replayed state.
    // Leader restarts; ships remaining segments.
    // Follower catches up.
}

#[tokio::test]
async fn idempotent_segment_replay() {
    // Ship same segment twice to follower.
    // Verify: signal counts NOT doubled (seqno idempotency).
}

Acceptance Criteria

All 7 integration tests pass under cargo test --test m8p2_replication
Test replication_decay_scores_match: leader 1K signals -> follower matches to 6 decimal places
Test follower_rejects_writes: TidalError::ReadOnly on all write methods
Test follower_serves_retrieve_queries: follower returns correct ranked results
Test replication_lag_converges_to_zero: lag = 0 within 5 seconds of leader quiesce
Test corrupted_segment_is_rejected: corrupt checksums rejected, no state change
Test leader_restart_follower_continues: follower serves reads after leader crash
Test idempotent_segment_replay: no double-counting on duplicate segments
cargo clippy -D warnings and cargo fmt pass

3.2 KiB Raw Blame History