stemedb/applications/aphoria/uat/2026-02-04-uat-policy-source-tracking.md
jordan 1cc453c97b feat: Aphoria policy source tracking + claim extraction pipeline
- Add PolicySourceStore for tracking where policies come from
- Implement claim extraction skill and API endpoints
- Add community UI text selection extractor component
- Create Go SDK aphoria client for policy operations
- Document patent specifications and legal disclosures
- Add guides: golden path loop, policy audit trails, pre-flight checks
- Expand Unreal Engine config extractor with source tracking
- Add UAT reports for policy source tracking validation
- Refactor tests.rs into modular test files

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 02:35:02 -07:00

156 lines
4.4 KiB
Markdown

# UAT Plan: Policy Source Tracking in Persistent Mode
**Goal:** Verify that persistent mode now tracks and displays policy source attribution (pack name, version, issuer) in conflict reports — matching ephemeral mode behavior.
**Hypothesis:** Teams adopting Aphoria's persistent mode need full provenance in conflict reports. Without knowing *which* Trust Pack flagged a violation, developers can't escalate to the right team or understand why a policy exists.
## 1. Test Environment
**Aphoria Version:** 0.1.0 + PackSourceStore
**Configuration:**
```toml
# Project A: Policy Publisher
[scan]
mode = "persistent"
# Project B: Policy Consumer
[scan]
mode = "persistent"
policies = ["../project_a/security-standard.pack"]
```
## 2. Success Criteria
| Test Case | Expected Result | Status |
|-----------|-----------------|--------|
| Import stores pack source | `PackSourceStore` contains entry for each assertion subject | |
| Conflict shows policy_source | `ConflictingSource.policy_source` is `Some(...)` not `None` | |
| Pack name correct | `policy_source.pack_name` matches exported pack name | |
| Pack version correct | `policy_source.pack_version` matches exported version | |
| Issuer hex correct | `policy_source.issuer_hex` is 8 chars (4 bytes of public key) | |
| Multiple packs isolated | Assertions from different packs show their respective sources | |
| Persistence survives restart | Reopen `LocalEpisteme`, pack sources still queryable | |
## 3. Execution Plan
### Step 1: Create Policy Publisher (Project A)
```bash
mkdir -p /tmp/uat-policy-source/project_a
cd /tmp/uat-policy-source/project_a
# Initialize Aphoria
mkdir -p .aphoria
cat > aphoria.toml << 'EOF'
[episteme]
data_dir = ".aphoria/db"
EOF
# Create a TLS policy assertion
aphoria bless "code://rust/acme/tls/cert_verification" enabled true \
--reason "All Acme services MUST verify TLS certificates"
# Export as Trust Pack
aphoria export --name "Acme Security Standard" --output security-standard.pack
```
**Checkpoint:** Verify `security-standard.pack` exists and is valid:
```bash
ls -la security-standard.pack
# Should be non-empty binary file
```
### Step 2: Create Policy Consumer (Project B)
```bash
mkdir -p /tmp/uat-policy-source/project_b/src
cd /tmp/uat-policy-source/project_b
# Create Cargo.toml
cat > Cargo.toml << 'EOF'
[package]
name = "project_b"
version = "0.1.0"
EOF
# Create violating code
cat > src/client.rs << 'EOF'
fn create_client() -> Client {
reqwest::Client::builder()
.danger_accept_invalid_certs(true) // Violates policy!
.build()
.unwrap()
}
EOF
# Configure Aphoria with imported policy
cat > aphoria.toml << 'EOF'
[episteme]
data_dir = ".aphoria/db"
[policies]
security = "../project_a/security-standard.pack"
EOF
```
### Step 3: Import Policy and Scan
```bash
cd /tmp/uat-policy-source/project_b
# Import the policy (stores pack sources)
aphoria import ../project_a/security-standard.pack
# Run persistent mode scan
aphoria scan . --mode persistent --format json > scan_result.json
```
### Step 4: Verify Policy Source Attribution
```bash
# Check JSON output for policy_source field
cat scan_result.json | jq '.findings[].conflicting_sources[].policy_source'
```
**Expected Output:**
```json
{
"pack_name": "Acme Security Standard",
"pack_version": "0.1.0",
"issuer_hex": "a1b2c3d4"
}
```
### Step 5: Verify Persistence
```bash
# Restart and scan again (should still have policy source)
aphoria scan . --mode persistent --format table
```
**Expected:** Table output shows "Source: Acme Security Standard (a1b2c3d4)" in conflict details.
## 4. Edge Cases
| Scenario | Expected Behavior |
|----------|-------------------|
| Hardcoded corpus assertion (no pack) | `policy_source: null` |
| Same subject from multiple packs | Last imported pack wins |
| Empty pack (no assertions) | No pack sources stored |
| Corrupted pack file | Import fails with signature error |
## 5. Artifacts
Upon completion, create:
- `2026-02-04-uat-policy-source-results.md` with pass/fail for each criterion
- Screenshot of table output showing policy source attribution
## 6. Risk Assessment
- **False Negative:** If pack source lookup fails silently, conflicts show `policy_source: None` and we lose provenance. Mitigated by `warn!` logging on lookup errors.
- **Performance:** JSON serialization for every subject adds latency. Acceptable for policy import (one-time cost), not in hot path.
---
**Next Step:** Execute Steps 1-5 and record results.