102 lines
5.1 KiB
Markdown
102 lines
5.1 KiB
Markdown
# Task 04: Regression Gate
|
|
|
|
## Delivers
|
|
|
|
Verification that all prior milestone UAT suites (m2_uat through m6_uat) continue to pass after M7 changes. This is a validation step, not new Rust code. The task ensures M7's production hardening additions (new error variants, new fields on Results/SearchResults, new TidalDb builder methods, session sweeper background thread) have not introduced regressions in existing behavior.
|
|
|
|
## Complexity: S
|
|
|
|
## Dependencies
|
|
|
|
- Tasks 01, 02, 03 complete (all m7_uat tests written and passing)
|
|
- m7p1-m7p4 complete (all M7 features implemented)
|
|
|
|
## Technical Design
|
|
|
|
### Step 1: Run all prior UAT suites
|
|
|
|
```bash
|
|
cargo test --manifest-path tidal/Cargo.toml --test m2_uat 2>&1
|
|
cargo test --manifest-path tidal/Cargo.toml --test m3_uat 2>&1
|
|
cargo test --manifest-path tidal/Cargo.toml --test m4_uat 2>&1
|
|
cargo test --manifest-path tidal/Cargo.toml --test m5_uat 2>&1
|
|
cargo test --manifest-path tidal/Cargo.toml --test m6_uat 2>&1
|
|
```
|
|
|
|
All five commands must exit with status 0 and report 0 failures.
|
|
|
|
### Step 2: Run the new M7 UAT suite
|
|
|
|
```bash
|
|
cargo test --manifest-path tidal/Cargo.toml --test m7_uat 2>&1
|
|
```
|
|
|
|
Must exit with status 0 and report 0 failures.
|
|
|
|
### Step 3: Run all integration tests together
|
|
|
|
```bash
|
|
cargo test --manifest-path tidal/Cargo.toml --tests 2>&1
|
|
```
|
|
|
|
Must exit with status 0. This catches any cross-test interference (e.g., a background sweeper thread from one test leaking into another, port conflicts, temp directory collisions).
|
|
|
|
### Step 4: Run the full lib test suite
|
|
|
|
```bash
|
|
cargo test --manifest-path tidal/Cargo.toml --lib 2>&1
|
|
```
|
|
|
|
Must exit with status 0.
|
|
|
|
### Step 5: Clippy clean
|
|
|
|
```bash
|
|
cargo clippy --manifest-path tidal/Cargo.toml -- -D warnings 2>&1
|
|
```
|
|
|
|
Must exit with status 0.
|
|
|
|
### Known Regression Vectors
|
|
|
|
M7 introduces several API changes that could break existing tests:
|
|
|
|
| Change | Potential breakage | Mitigation |
|
|
|--------|--------------------|------------|
|
|
| `Results.stats: QueryStats` new field | Existing tests constructing `Results` manually | `QueryStats` should impl `Default`; existing tests unaffected if they access `results.items` only |
|
|
| `SearchResults.stats: QueryStats` new field | Same as above | Same mitigation |
|
|
| `SessionSummary.auto_closed: bool` new field | Tests checking `SessionSummary` exhaustively | Field defaults to `false` for manually-closed sessions |
|
|
| `TidalError::RateLimited` new variant | Tests matching on `TidalError` exhaustively | No existing tests should do exhaustive match; they use `is_err()` |
|
|
| `TidalError::Backpressure` new variant | Same as above | Same mitigation |
|
|
| Session sweeper background thread | Tests that open ephemeral DBs without calling `close()` | Sweeper should be no-op on ephemeral DBs with no sessions; `Drop` impl on `TidalDb` should cancel the sweeper |
|
|
| `DegradationLevel` in response | Tests asserting exact `Results` structure | Field should default to `DegradationLevel::Full` (no degradation) |
|
|
|
|
### Debugging Regression Failures
|
|
|
|
If a prior UAT test fails after M7:
|
|
|
|
1. **Identify the test:** which specific `#[test]` function failed?
|
|
2. **Check the error:** compile error (API change) or runtime assertion failure?
|
|
3. **If compile error:** the M7 API change broke backwards compatibility. Fix the M7 implementation to provide backward-compatible defaults (e.g., `#[derive(Default)]` on new types, default field values).
|
|
4. **If runtime assertion:** run the failing test in isolation. If it passes alone but fails with `--tests`, it is a cross-test interference issue (likely the sweeper thread or a shared global). Fix the isolation.
|
|
5. **Never weaken the existing assertion.** If an M6 UAT test asserts something that was true before M7 and is no longer true, M7 broke a contract. Fix M7, not the test.
|
|
|
|
## Acceptance Criteria
|
|
|
|
- [ ] `cargo test --manifest-path tidal/Cargo.toml --test m2_uat` passes (0 failures)
|
|
- [ ] `cargo test --manifest-path tidal/Cargo.toml --test m3_uat` passes (0 failures)
|
|
- [ ] `cargo test --manifest-path tidal/Cargo.toml --test m4_uat` passes (0 failures)
|
|
- [ ] `cargo test --manifest-path tidal/Cargo.toml --test m5_uat` passes (0 failures)
|
|
- [ ] `cargo test --manifest-path tidal/Cargo.toml --test m6_uat` passes (0 failures)
|
|
- [ ] `cargo test --manifest-path tidal/Cargo.toml --test m7_uat` passes (0 failures)
|
|
- [ ] `cargo test --manifest-path tidal/Cargo.toml --tests` passes (0 failures, no cross-test interference)
|
|
- [ ] `cargo test --manifest-path tidal/Cargo.toml --lib` passes (0 failures)
|
|
- [ ] `cargo clippy --manifest-path tidal/Cargo.toml -- -D warnings` passes
|
|
- [ ] No existing test was modified to make it pass (if a test needed modification, the M7 implementation is fixed instead)
|
|
|
|
## Test Strategy
|
|
|
|
This task produces no new Rust code. It is a verification gate that runs existing test suites and confirms they remain green. The value is in the systematic check and the documented regression vectors, which guide the implementer to preemptively fix backward-compatibility issues during m7p1-m7p4 rather than discovering them at UAT time.
|
|
|
|
If any prior UAT suite fails, the fix belongs in the M7 implementation (m7p1-m7p4), not in this task. This task only passes when all suites are green without modification.
|