tidaldb/docs/planning/milestone-7/phase-5/task-04-regression-gate.md
2026-02-23 22:41:16 -07:00

5.1 KiB

Task 04: Regression Gate

Delivers

Verification that all prior milestone UAT suites (m2_uat through m6_uat) continue to pass after M7 changes. This is a validation step, not new Rust code. The task ensures M7's production hardening additions (new error variants, new fields on Results/SearchResults, new TidalDb builder methods, session sweeper background thread) have not introduced regressions in existing behavior.

Complexity: S

Dependencies

  • Tasks 01, 02, 03 complete (all m7_uat tests written and passing)
  • m7p1-m7p4 complete (all M7 features implemented)

Technical Design

Step 1: Run all prior UAT suites

cargo test --manifest-path tidal/Cargo.toml --test m2_uat    2>&1
cargo test --manifest-path tidal/Cargo.toml --test m3_uat    2>&1
cargo test --manifest-path tidal/Cargo.toml --test m4_uat    2>&1
cargo test --manifest-path tidal/Cargo.toml --test m5_uat    2>&1
cargo test --manifest-path tidal/Cargo.toml --test m6_uat    2>&1

All five commands must exit with status 0 and report 0 failures.

Step 2: Run the new M7 UAT suite

cargo test --manifest-path tidal/Cargo.toml --test m7_uat    2>&1

Must exit with status 0 and report 0 failures.

Step 3: Run all integration tests together

cargo test --manifest-path tidal/Cargo.toml --tests          2>&1

Must exit with status 0. This catches any cross-test interference (e.g., a background sweeper thread from one test leaking into another, port conflicts, temp directory collisions).

Step 4: Run the full lib test suite

cargo test --manifest-path tidal/Cargo.toml --lib             2>&1

Must exit with status 0.

Step 5: Clippy clean

cargo clippy --manifest-path tidal/Cargo.toml -- -D warnings  2>&1

Must exit with status 0.

Known Regression Vectors

M7 introduces several API changes that could break existing tests:

Change Potential breakage Mitigation
Results.stats: QueryStats new field Existing tests constructing Results manually QueryStats should impl Default; existing tests unaffected if they access results.items only
SearchResults.stats: QueryStats new field Same as above Same mitigation
SessionSummary.auto_closed: bool new field Tests checking SessionSummary exhaustively Field defaults to false for manually-closed sessions
TidalError::RateLimited new variant Tests matching on TidalError exhaustively No existing tests should do exhaustive match; they use is_err()
TidalError::Backpressure new variant Same as above Same mitigation
Session sweeper background thread Tests that open ephemeral DBs without calling close() Sweeper should be no-op on ephemeral DBs with no sessions; Drop impl on TidalDb should cancel the sweeper
DegradationLevel in response Tests asserting exact Results structure Field should default to DegradationLevel::Full (no degradation)

Debugging Regression Failures

If a prior UAT test fails after M7:

  1. Identify the test: which specific #[test] function failed?
  2. Check the error: compile error (API change) or runtime assertion failure?
  3. If compile error: the M7 API change broke backwards compatibility. Fix the M7 implementation to provide backward-compatible defaults (e.g., #[derive(Default)] on new types, default field values).
  4. If runtime assertion: run the failing test in isolation. If it passes alone but fails with --tests, it is a cross-test interference issue (likely the sweeper thread or a shared global). Fix the isolation.
  5. Never weaken the existing assertion. If an M6 UAT test asserts something that was true before M7 and is no longer true, M7 broke a contract. Fix M7, not the test.

Acceptance Criteria

  • cargo test --manifest-path tidal/Cargo.toml --test m2_uat passes (0 failures)
  • cargo test --manifest-path tidal/Cargo.toml --test m3_uat passes (0 failures)
  • cargo test --manifest-path tidal/Cargo.toml --test m4_uat passes (0 failures)
  • cargo test --manifest-path tidal/Cargo.toml --test m5_uat passes (0 failures)
  • cargo test --manifest-path tidal/Cargo.toml --test m6_uat passes (0 failures)
  • cargo test --manifest-path tidal/Cargo.toml --test m7_uat passes (0 failures)
  • cargo test --manifest-path tidal/Cargo.toml --tests passes (0 failures, no cross-test interference)
  • cargo test --manifest-path tidal/Cargo.toml --lib passes (0 failures)
  • cargo clippy --manifest-path tidal/Cargo.toml -- -D warnings passes
  • No existing test was modified to make it pass (if a test needed modification, the M7 implementation is fixed instead)

Test Strategy

This task produces no new Rust code. It is a verification gate that runs existing test suites and confirms they remain green. The value is in the systematic check and the documented regression vectors, which guide the implementer to preemptively fix backward-compatibility issues during m7p1-m7p4 rather than discovering them at UAT time.

If any prior UAT suite fails, the fix belongs in the M7 implementation (m7p1-m7p4), not in this task. This task only passes when all suites are green without modification.