125 lines
7.1 KiB
Markdown
125 lines
7.1 KiB
Markdown
# QA Plan: Agent Capability Boundaries
|
|
|
|
## Scope
|
|
|
|
This QA plan covers the policy enforcement layer added by `m10-agent-capability-boundaries`. All testing is automated via the Rust test suite. No manual or UI testing is required — this feature has no frontend surface.
|
|
|
|
## Test Levels
|
|
|
|
### 1. Unit Tests (`cargo test --lib`)
|
|
|
|
All unit tests live in `#[cfg(test)]` blocks within the modules they test.
|
|
|
|
#### `session/policy.rs` — `PolicyEvaluator` read checks
|
|
|
|
| Test | Description | Expected |
|
|
|---|---|---|
|
|
| `check_read_empty_lists_allows_any` | Empty `allowed_read_signals` and `denied_read_signals` | `Ok(())` for any signal |
|
|
| `check_read_allow_list_permits_listed_signal` | `allowed_read_signals: ["view"]`, check `"view"` | `Ok(())` |
|
|
| `check_read_allow_list_rejects_unlisted_signal` | `allowed_read_signals: ["view"]`, check `"like"` | `Err(ReadNotAllowed)` |
|
|
| `check_read_deny_list_rejects_denied_signal` | `denied_read_signals: ["hide"]`, check `"hide"` | `Err(ReadDenied)` |
|
|
| `check_read_deny_list_takes_priority_over_allow` | `allowed: ["hide"]`, `denied: ["hide"]`, check `"hide"` | `Err(ReadDenied)` |
|
|
| `check_attribute_read_empty_lists_allows_any` | All attribute lists empty | `Ok(())` for any key |
|
|
| `check_attribute_read_allow_list_permits_key` | `allowed_user_attributes: ["locale"]`, key `"locale"` | `Ok(())` |
|
|
| `check_attribute_read_allow_list_rejects_unlisted_key` | `allowed_user_attributes: ["locale"]`, key `"age_range"` | `Err(AttributeReadNotAllowed)` |
|
|
| `check_attribute_read_deny_list_rejects_key` | `denied_user_attributes: ["email"]`, key `"email"` | `Err(AttributeReadDenied)` |
|
|
| `check_profile_override_empty_list_rejects_any` | `allowed_profile_overrides: []`, profile `"search"` | `Err(ProfileOverrideNotAllowed)` |
|
|
| `check_profile_override_listed_profile_allowed` | `allowed_profile_overrides: ["search"]`, profile `"search"` | `Ok(())` |
|
|
| `check_profile_override_unlisted_profile_rejected` | `allowed_profile_overrides: ["search"]`, profile `"for_you"` | `Err(ProfileOverrideNotAllowed)` |
|
|
|
|
#### `schema/validation/builders.rs` — Schema build validation
|
|
|
|
| Test | Description | Expected |
|
|
|---|---|---|
|
|
| `schema_policy_valid_read_signals_builds` | Valid signal names in read lists | `Ok(schema)` |
|
|
| `schema_policy_unknown_signal_in_allowed_read_fails` | `allowed_read_signals: ["nonexistent"]` | `Err(SchemaError)` |
|
|
| `schema_policy_unknown_signal_in_denied_read_fails` | `denied_read_signals: ["nonexistent"]` | `Err(SchemaError)` |
|
|
| `schema_policy_conflict_allow_deny_read_fails` | Same signal in both lists | `Err(SchemaError)` |
|
|
| `schema_policy_unknown_profile_override_fails` | `allowed_profile_overrides: ["made_up_profile"]` | `Err(SchemaError)` |
|
|
| `schema_policy_star_sentinel_expands_to_all_profiles` | `allowed_profile_overrides: ["*"]` | Schema builds; list contains all known profiles |
|
|
| `schema_policy_empty_new_fields_is_valid` | All new fields empty | `Ok(schema)` (no regression) |
|
|
|
|
#### `session/state.rs` + `session/snapshot.rs`
|
|
|
|
| Test | Description | Expected |
|
|
|---|---|---|
|
|
| `overrides_rejected_starts_at_zero` | Freshly created `SessionState` | `overrides_rejected.load() == 0` |
|
|
| `snapshot_includes_overrides_rejected` | Snapshot from state with `overrides_rejected = 3` | `snapshot.overrides_rejected == 3` |
|
|
| `frozen_snapshot_includes_overrides_rejected` | Frozen snapshot from same state | `frozen_snapshot.overrides_rejected == 3` |
|
|
|
|
### 2. Integration Tests (`tidal/tests/m10_agent_capability.rs`)
|
|
|
|
These tests exercise the full `TidalDb` API stack. All run with an ephemeral in-memory database.
|
|
|
|
| Test | Description | Expected |
|
|
|---|---|---|
|
|
| `read_allowed_signal_succeeds` | Session with `allowed_read_signals: ["view"]`; call `read_decay_score_for_session` for `view` | `Ok(Some(_))` |
|
|
| `read_disallowed_signal_fails_not_allowed` | Same session; call for `like` (not in allow list) | `Err` with `ReadNotAllowed` kind |
|
|
| `read_denied_signal_fails_denied` | Session with `denied_read_signals: ["hide"]`; call for `hide` | `Err` with `ReadDenied` kind |
|
|
| `read_without_session_unrestricted` | `read_decay_score` (no session) for `hide` | `Ok(_)` |
|
|
| `attribute_read_allowed_succeeds` | Session with `allowed_user_attributes: ["locale"]`; read `locale` | `Ok(Some(_))` |
|
|
| `attribute_read_disallowed_fails` | Same session; read `age_range` | `Err` with `AttributeReadNotAllowed` |
|
|
| `profile_override_allowed_proceeds` | Session with `allowed_profile_overrides: ["search"]`; retrieve with `profile: "search"` | `Ok(results)` |
|
|
| `profile_override_disallowed_fails` | Same session; retrieve with `profile: "for_you"` | `Err` with `ProfileOverrideNotAllowed` |
|
|
| `overrides_rejected_incremented` | After one disallowed profile override | `session_snapshot(sid).overrides_rejected == 1` |
|
|
| `audit_log_records_read_denial` | After a `ReadDenied` violation | `session_audit(sid)` contains entry with appropriate kind |
|
|
| `signals_rejected_incremented_on_read_denial` | After a read denial | `session_snapshot(sid).signals_rejected >= 1` |
|
|
| `empty_policy_no_regression` | Policy with all new fields empty; reads of any signal | `Ok(_)` (same as before) |
|
|
| `schema_build_fails_unknown_read_signal` | Policy with `allowed_read_signals: ["ghost"]` | `SchemaError` at build time |
|
|
| `schema_build_fails_conflict` | Same signal in both read allow and deny lists | `SchemaError` at build time |
|
|
| `schema_sentinel_star_allows_all_profiles` | `allowed_profile_overrides: ["*"]`; retrieve with `profile: "for_you"` | `Ok(results)` |
|
|
|
|
### 3. Regression: Existing Test Suite
|
|
|
|
Run the full library test suite to verify no regressions:
|
|
|
|
```bash
|
|
cargo test --manifest-path tidal/Cargo.toml --lib
|
|
```
|
|
|
|
All tests passing before this feature must continue to pass. The new fields on `AgentPolicy` default to empty and must not change behavior for any existing code path.
|
|
|
|
Run existing integration test suites:
|
|
|
|
```bash
|
|
cargo test --manifest-path tidal/Cargo.toml --test m7_uat
|
|
cargo test --manifest-path tidal/Cargo.toml --test m5_uat
|
|
cargo test --manifest-path tidal/Cargo.toml --test m6_uat
|
|
```
|
|
|
|
These must all remain green.
|
|
|
|
## Test Commands
|
|
|
|
```bash
|
|
# Full library unit tests (must all pass, no regressions)
|
|
cargo test --manifest-path tidal/Cargo.toml --lib
|
|
|
|
# New integration tests
|
|
cargo test --manifest-path tidal/Cargo.toml --test m10_agent_capability
|
|
|
|
# Lint and format
|
|
cargo fmt --manifest-path tidal/Cargo.toml --check
|
|
cargo clippy --manifest-path tidal/Cargo.toml -D warnings
|
|
|
|
# Prior UAT suites (regression check)
|
|
cargo test --manifest-path tidal/Cargo.toml --test m7_uat
|
|
cargo test --manifest-path tidal/Cargo.toml --test m5_uat
|
|
cargo test --manifest-path tidal/Cargo.toml --test m6_uat
|
|
```
|
|
|
|
## Pass Criteria
|
|
|
|
- All unit tests in `session/policy.rs`, `schema/validation/builders.rs`, and `session/snapshot.rs` pass.
|
|
- All 15 integration tests in `m10_agent_capability.rs` pass.
|
|
- Full library test suite passes with no regressions.
|
|
- `cargo clippy -D warnings` is clean.
|
|
- `cargo fmt --check` is clean.
|
|
- Prior M5, M6, M7 UAT suites are all green.
|
|
|
|
## Exclusions
|
|
|
|
- No manual browser or UI testing required (no UI surface).
|
|
- No load/benchmark testing required for this feature (enforcement cost is sub-100ns per check; no new hot paths).
|
|
- No WAL or storage format testing required (no on-disk changes).
|