tidaldb/.sdlc/features/m10-community-policy-engine/qa-plan.md

5.3 KiB
Raw Blame History

QA Plan: Community Policy Engine

Scope

Verify correctness and robustness of the community policy engine across schema validation, signal write enforcement, ranking read suppression, and error handling. Confirm zero regression in existing test suites.


Test Suites

1. Unit tests — schema validation (schema/validation/builders.rs, schema/validation/community_policy.rs)

# Test Expected
U1 Build schema with valid member and moderator community policies build() succeeds; schema.community_policy("member") returns Some
U2 Build schema with duplicate community policy name Err(SchemaError::DuplicateCommunityPolicyName)
U3 Build schema with signal in policy that does not exist in schema Err(SchemaError::CommunityPolicySignalNotInSchema)
U4 Build schema with write allow/deny conflict Err(SchemaError::CommunityPolicySignalConflict)
U5 Build schema with read allow/deny conflict Err(SchemaError::CommunityPolicySignalConflict)
U6 Build schema with invalid community policy name (uppercase, spaces) Err(SchemaError::InvalidCommunityPolicyName)
U7 Schema::community_policy_count() returns correct count after build Count matches number of registered policies
U8 CommunityPolicy with empty allow/deny lists (admin-like) validates cleanly build() succeeds

2. Unit tests — write evaluator (db/community.rs)

# Test Expected
U9 check_write with signal in allowed_write_signals, no deny list Ok(())
U10 check_write with signal in denied_write_signals Err(PolicyViolation { kind: CommunityWriteDenied, .. })
U11 check_write with non-empty allow list and signal not in it Err(PolicyViolation { kind: CommunityWriteNotAllowed, .. })
U12 check_write with empty allow list and empty deny list Ok(()) (all signals allowed)
U13 check_write with signal in deny list AND allow list — deny takes precedence Err(PolicyViolation { kind: CommunityWriteDenied, .. })

3. Unit tests — read suppression (ranking/executor/)

# Test Expected
U14 build_suppressed_set with signal in denied_read_signals Set contains that SignalTypeId
U15 build_suppressed_set with empty allowed_read_signals and denied_read_signals Empty set
U16 build_suppressed_set with non-empty allowed_read_signals and signal not in it Set contains that SignalTypeId
U17 Scoring loop skips contribution for signal in suppressed set Score lower than unsuppressed equivalent
U18 Scoring loop includes all contributions when suppressed set is empty Score identical to no-community-context case

4. Integration tests — tidal/tests/m10_community_policy.rs

# Test Expected
I1 Write allowed signal with member role Signal recorded; read_decay_score returns positive value
I2 Write denied signal with member role Err(TidalError::PolicyViolation) with CommunityWriteDenied kind
I3 Write signal not in allow list with member role Err(TidalError::PolicyViolation) with CommunityWriteNotAllowed kind
I4 Write any signal with admin role (empty allow/deny) Signal recorded; no error
I5 Retrieve with community context suppressing signal: candidate with suppressed signal scores lower than candidate without Score ordering: unsuppressed > suppressed
I6 Retrieve without community context: all signal contributions included Score identical to pre-feature baseline
I7 Schema registration: duplicate policy name Err(SchemaError::DuplicateCommunityPolicyName) at build()
I8 Schema registration: unknown signal in policy Err(SchemaError::CommunityPolicySignalNotInSchema) at build()
I9 Schema registration: allow/deny conflict Err(SchemaError::CommunityPolicySignalConflict) at build()
I10 signal_with_community_policy with unknown role name Err(TidalError::NotFound)

5. Regression tests

Run existing integration suites without modification. All must pass:

Suite Command
m5_uat cargo test --test m5_uat
m6_uat cargo test --test m6_uat
m7_uat cargo test --test m7_uat
m8_uat cargo test --test m8_uat
m2_uat cargo test --test m2_uat
signal_api cargo test --test signal_api
vector_usearch cargo test --test vector_usearch

Performance Check

After implementation, verify the scoring hot path has not regressed:

  • Existing retrieve benchmarks (if any) must not show > 5% regression.
  • For a 200-candidate retrieve query without community context, confirm the fast-path branch (empty suppressed set, no contains calls) is taken.

Error Message Quality

For each TidalError::PolicyViolation returned:

  • The reason string must name the signal type and the role.
  • Example: "signal 'pin' denied by community role 'member'".

For each SchemaError returned:

  • Must include enough context to identify the offending policy and signal by name.

Pass Criteria

  • All U1U18 unit tests pass.
  • All I1I10 integration tests pass.
  • All regression suites pass without modification.
  • cargo clippy --manifest-path tidal/Cargo.toml -- -D warnings produces zero warnings.
  • cargo fmt --manifest-path tidal/Cargo.toml --check produces no diffs.