3bcfb3c576
- Add BUILD.bazel across tidal, tidal-net, tidal-server, tidalctl for bzlmod build - Add tidal/ crate docs (README, CHANGELOG, CONTRIBUTING, AGENTS, CLAUDE, API, ARCHITECTURE) and ai-lookup reference - Add docker standalone/cluster/deploy images, compose, and prometheus config - Harden WAL (batch format, writer, dedup, diagnostics), text syncer/collectors, and vector registry - Expand tidalctl CLI and tests; restructure WAL/visibility integration test suites - Refine tidal-net transport/client/server and tidal-server cluster/scatter-gather
104 lines
3.2 KiB
Markdown
104 lines
3.2 KiB
Markdown
# Task 07: Replication Integration Tests
|
|
|
|
## Delivers
|
|
|
|
Integration tests in `tidal/tests/m8p2_replication.rs` covering the full replication stack: leader->follower segment delivery, decay score equivalence to 6 decimal places, follower read-only enforcement, lag convergence, and segment corruption rejection.
|
|
|
|
## Complexity: M
|
|
|
|
## Dependencies
|
|
|
|
- Tasks 01-06 complete
|
|
|
|
## Technical Design
|
|
|
|
```rust
|
|
// tidal/tests/m8p2_replication.rs
|
|
|
|
use tidaldb::{TidalDb, TidalDbBuilder, NodeRole, ShardId, RegionId, NodeConfig};
|
|
use tidaldb::replication::{InProcessTransportFactory, ReplicationLagGauge};
|
|
|
|
fn leader_config(data_dir: &Path) -> Config {
|
|
Config {
|
|
cluster: NodeConfig {
|
|
role: NodeRole::Leader,
|
|
shard_id: ShardId(0),
|
|
..Default::default()
|
|
},
|
|
..Config::with_data_dir(data_dir)
|
|
}
|
|
}
|
|
|
|
fn follower_config(data_dir: &Path) -> Config {
|
|
Config {
|
|
cluster: NodeConfig {
|
|
role: NodeRole::Follower,
|
|
shard_id: ShardId(0),
|
|
..Default::default()
|
|
},
|
|
..Config::with_data_dir(data_dir)
|
|
}
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn replication_decay_scores_match() {
|
|
// Leader writes 1,000 signals.
|
|
// Follower replays all segments.
|
|
// Verify: read_decay_score on follower matches leader to 6 decimal places.
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn follower_rejects_writes() {
|
|
// Open follower. Attempt signal() write.
|
|
// Verify: returns TidalError::ReadOnly.
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn follower_serves_retrieve_queries() {
|
|
// Leader writes items + signals.
|
|
// Follower applies.
|
|
// Follower.retrieve() returns ranked results.
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn replication_lag_converges_to_zero() {
|
|
// Leader writes 500 segments.
|
|
// Wait for follower to apply all.
|
|
// Assert: lag_seqno(follower) == 0 within 5 seconds.
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn corrupted_segment_is_rejected() {
|
|
// Manually corrupt BLAKE3 checksum in segment bytes.
|
|
// Send to follower via transport.
|
|
// Verify: segment is not applied (decay scores unchanged).
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn leader_restart_follower_continues() {
|
|
// Leader writes 100 signals.
|
|
// Leader shuts down.
|
|
// Follower serves read queries from replayed state.
|
|
// Leader restarts; ships remaining segments.
|
|
// Follower catches up.
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn idempotent_segment_replay() {
|
|
// Ship same segment twice to follower.
|
|
// Verify: signal counts NOT doubled (seqno idempotency).
|
|
}
|
|
```
|
|
|
|
## Acceptance Criteria
|
|
|
|
- [ ] All 7 integration tests pass under `cargo test --test m8p2_replication`
|
|
- [ ] Test `replication_decay_scores_match`: leader 1K signals -> follower matches to 6 decimal places
|
|
- [ ] Test `follower_rejects_writes`: `TidalError::ReadOnly` on all write methods
|
|
- [ ] Test `follower_serves_retrieve_queries`: follower returns correct ranked results
|
|
- [ ] Test `replication_lag_converges_to_zero`: lag = 0 within 5 seconds of leader quiesce
|
|
- [ ] Test `corrupted_segment_is_rejected`: corrupt checksums rejected, no state change
|
|
- [ ] Test `leader_restart_follower_continues`: follower serves reads after leader crash
|
|
- [ ] Test `idempotent_segment_replay`: no double-counting on duplicate segments
|
|
- [ ] `cargo clippy -D warnings` and `cargo fmt` pass
|