stemedb/applications/aphoria/uat/2026-02-04-uat-plan-unreal.md
jordan 1cc453c97b feat: Aphoria policy source tracking + claim extraction pipeline
- Add PolicySourceStore for tracking where policies come from
- Implement claim extraction skill and API endpoints
- Add community UI text selection extractor component
- Create Go SDK aphoria client for policy operations
- Document patent specifications and legal disclosures
- Add guides: golden path loop, policy audit trails, pre-flight checks
- Expand Unreal Engine config extractor with source tracking
- Add UAT reports for policy source tracking validation
- Refactor tests.rs into modular test files

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 02:35:02 -07:00

3.1 KiB

UAT Plan: Unreal Engine Audit (Masq Project)

Goal: Prove Aphoria's value for game development by detecting specific performance, security, and architectural issues in a real-world Unreal Engine project (MasqMain).

Hypothesis: Game developers struggle with invisible "drift" in large C++/Blueprint codebases — hardcoded paths, synchronous loading hitches, and insecure config defaults. Aphoria can surface these instantly using the same knowledge-graph approach that worked for VulnBank.

1. Test Environment

Target Codebase: /opt/MasqMain/UE (Masquerade Unreal Client) Aphoria Version: 0.1.0 + Unreal Extractors Configuration:

# aphoria.toml
[scan]
include_tests = false
max_file_size = 1048576 # 1MB

[extractors]
enabled = ["unreal_cpp", "unreal_config", "unreal_performance", "hardcoded_secrets"]

2. Success Criteria

We will consider this UAT a success if Aphoria detects at least 5 distinct issues with 100% precision (no false positives).

Category Finding Expected Verdict Why it matters
Performance LoadSynchronous() in C++ files FLAG Causes frame hitches during gameplay.

What We DON'T Flag (By Design)

Pattern Reason NOT Flagged
Hardcoded /Game/... paths in INI Standard Unreal practice - asset registry handles resolution
Empty ApiKey= placeholder Empty is safe - only non-empty credentials are a problem
UFUNCTION(Exec) Not present in Masq codebase
MaxClientRate settings Not configured in Masq INI files

3. Execution Plan

Step 1: Baseline Scan

Run Aphoria against the project root to establish the current state of "epistemic drift."

cd /opt/MasqMain/UE
aphoria scan . --format table

Step 2: Verification of Findings

For each finding, verify:

  1. Context: Is it actually code we own? (Ignore Engine/ if scanning externally, but we are inside project).
  2. Authority: Does the citation (vendor://unreal/...) make sense?
  3. Accuracy: Is LoadSynchronous actually on the game thread? (Yes, in Initialize()).

Step 3: Fix Workflow (Simulated)

Demonstrate how a developer would resolve one issue using the ack workflow vs. a code fix.

  • Scenario A (Fix): Change LoadSynchronous() to StreamableManager.RequestAsyncLoad().
  • Scenario B (Ack): Acknowledge UFUNCTION(Exec) on a debug cheat function that is stripped in shipping.

4. Expected Output Artifact

A report titled 2026-02-04-masq-unreal-audit.md in applications/aphoria/uat/ containing:

  • Summary of findings.
  • "Show stopper" issues found (e.g., the Sync Load in Subsystem).
  • Comparison of how long this would take a human reviewer vs. Aphoria (0.5s).

5. Risk Assessment

  • False Positives: TEXT("/Game/...") might be valid in ConstructorHelpers (only runs at startup). We need to distinguish runtime usage from CDO initialization.
  • Engine Code: If we scan Plugins/ that are third-party, we might find issues we can't fix. We should focus on Source/Masq/.

Next Step: Execute the scan?