- Add PolicySourceStore for tracking where policies come from - Implement claim extraction skill and API endpoints - Add community UI text selection extractor component - Create Go SDK aphoria client for policy operations - Document patent specifications and legal disclosures - Add guides: golden path loop, policy audit trails, pre-flight checks - Expand Unreal Engine config extractor with source tracking - Add UAT reports for policy source tracking validation - Refactor tests.rs into modular test files Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
156 lines
4.4 KiB
Markdown
156 lines
4.4 KiB
Markdown
# UAT Plan: Policy Source Tracking in Persistent Mode
|
|
|
|
**Goal:** Verify that persistent mode now tracks and displays policy source attribution (pack name, version, issuer) in conflict reports — matching ephemeral mode behavior.
|
|
|
|
**Hypothesis:** Teams adopting Aphoria's persistent mode need full provenance in conflict reports. Without knowing *which* Trust Pack flagged a violation, developers can't escalate to the right team or understand why a policy exists.
|
|
|
|
## 1. Test Environment
|
|
|
|
**Aphoria Version:** 0.1.0 + PackSourceStore
|
|
**Configuration:**
|
|
```toml
|
|
# Project A: Policy Publisher
|
|
[scan]
|
|
mode = "persistent"
|
|
|
|
# Project B: Policy Consumer
|
|
[scan]
|
|
mode = "persistent"
|
|
policies = ["../project_a/security-standard.pack"]
|
|
```
|
|
|
|
## 2. Success Criteria
|
|
|
|
| Test Case | Expected Result | Status |
|
|
|-----------|-----------------|--------|
|
|
| Import stores pack source | `PackSourceStore` contains entry for each assertion subject | |
|
|
| Conflict shows policy_source | `ConflictingSource.policy_source` is `Some(...)` not `None` | |
|
|
| Pack name correct | `policy_source.pack_name` matches exported pack name | |
|
|
| Pack version correct | `policy_source.pack_version` matches exported version | |
|
|
| Issuer hex correct | `policy_source.issuer_hex` is 8 chars (4 bytes of public key) | |
|
|
| Multiple packs isolated | Assertions from different packs show their respective sources | |
|
|
| Persistence survives restart | Reopen `LocalEpisteme`, pack sources still queryable | |
|
|
|
|
## 3. Execution Plan
|
|
|
|
### Step 1: Create Policy Publisher (Project A)
|
|
|
|
```bash
|
|
mkdir -p /tmp/uat-policy-source/project_a
|
|
cd /tmp/uat-policy-source/project_a
|
|
|
|
# Initialize Aphoria
|
|
mkdir -p .aphoria
|
|
cat > aphoria.toml << 'EOF'
|
|
[episteme]
|
|
data_dir = ".aphoria/db"
|
|
EOF
|
|
|
|
# Create a TLS policy assertion
|
|
aphoria bless "code://rust/acme/tls/cert_verification" enabled true \
|
|
--reason "All Acme services MUST verify TLS certificates"
|
|
|
|
# Export as Trust Pack
|
|
aphoria export --name "Acme Security Standard" --output security-standard.pack
|
|
```
|
|
|
|
**Checkpoint:** Verify `security-standard.pack` exists and is valid:
|
|
```bash
|
|
ls -la security-standard.pack
|
|
# Should be non-empty binary file
|
|
```
|
|
|
|
### Step 2: Create Policy Consumer (Project B)
|
|
|
|
```bash
|
|
mkdir -p /tmp/uat-policy-source/project_b/src
|
|
cd /tmp/uat-policy-source/project_b
|
|
|
|
# Create Cargo.toml
|
|
cat > Cargo.toml << 'EOF'
|
|
[package]
|
|
name = "project_b"
|
|
version = "0.1.0"
|
|
EOF
|
|
|
|
# Create violating code
|
|
cat > src/client.rs << 'EOF'
|
|
fn create_client() -> Client {
|
|
reqwest::Client::builder()
|
|
.danger_accept_invalid_certs(true) // Violates policy!
|
|
.build()
|
|
.unwrap()
|
|
}
|
|
EOF
|
|
|
|
# Configure Aphoria with imported policy
|
|
cat > aphoria.toml << 'EOF'
|
|
[episteme]
|
|
data_dir = ".aphoria/db"
|
|
|
|
[policies]
|
|
security = "../project_a/security-standard.pack"
|
|
EOF
|
|
```
|
|
|
|
### Step 3: Import Policy and Scan
|
|
|
|
```bash
|
|
cd /tmp/uat-policy-source/project_b
|
|
|
|
# Import the policy (stores pack sources)
|
|
aphoria import ../project_a/security-standard.pack
|
|
|
|
# Run persistent mode scan
|
|
aphoria scan . --mode persistent --format json > scan_result.json
|
|
```
|
|
|
|
### Step 4: Verify Policy Source Attribution
|
|
|
|
```bash
|
|
# Check JSON output for policy_source field
|
|
cat scan_result.json | jq '.findings[].conflicting_sources[].policy_source'
|
|
```
|
|
|
|
**Expected Output:**
|
|
```json
|
|
{
|
|
"pack_name": "Acme Security Standard",
|
|
"pack_version": "0.1.0",
|
|
"issuer_hex": "a1b2c3d4"
|
|
}
|
|
```
|
|
|
|
### Step 5: Verify Persistence
|
|
|
|
```bash
|
|
# Restart and scan again (should still have policy source)
|
|
aphoria scan . --mode persistent --format table
|
|
```
|
|
|
|
**Expected:** Table output shows "Source: Acme Security Standard (a1b2c3d4)" in conflict details.
|
|
|
|
## 4. Edge Cases
|
|
|
|
| Scenario | Expected Behavior |
|
|
|----------|-------------------|
|
|
| Hardcoded corpus assertion (no pack) | `policy_source: null` |
|
|
| Same subject from multiple packs | Last imported pack wins |
|
|
| Empty pack (no assertions) | No pack sources stored |
|
|
| Corrupted pack file | Import fails with signature error |
|
|
|
|
## 5. Artifacts
|
|
|
|
Upon completion, create:
|
|
- `2026-02-04-uat-policy-source-results.md` with pass/fail for each criterion
|
|
- Screenshot of table output showing policy source attribution
|
|
|
|
## 6. Risk Assessment
|
|
|
|
- **False Negative:** If pack source lookup fails silently, conflicts show `policy_source: None` and we lose provenance. Mitigated by `warn!` logging on lookup errors.
|
|
- **Performance:** JSON serialization for every subject adds latency. Acceptable for policy import (one-time cost), not in hot path.
|
|
|
|
---
|
|
|
|
**Next Step:** Execute Steps 1-5 and record results.
|