stemedb/applications/aphoria/uat/enterprise-simulation-uat.md

Enterprise Simulation UAT

Scenario: Acme Corp adopts Aphoria across 3 teams over 6 months Goal: Validate self-learning institutional knowledge at enterprise scale Status: Specification (not yet implemented)

---

Simulation Overview

The Organization

Acme Corp (Engineering)
├── Platform Team (8 engineers)
│   ├── Alice, Bob (established patterns with specs/ADRs)
│   └── 6 others (1 new hire in Month 3: Jordan)
├── Payments Team (6 engineers)
│   ├── Carol (maintains payment specs)
│   └── 5 others
└── Mobile Team (5 engineers)
    └── 5 engineers (1 new hire in Month 4: Kim)

Authority model: Evidence-based, not title-based

• Patterns with ProductSpec → highest authority (0.95)
• Patterns with RFC/Standard → high authority (0.85)
• Patterns with ADR/Research → medium authority (0.70)
• Patterns from commits only → low authority (0.40)

The Projects

Team	Projects	Languages	Focus
Platform	api-gateway, auth-service, config-lib	Rust, Go	Infrastructure
Payments	payment-processor, billing-service	Python, Go	Transactions
Mobile	ios-app, android-app, mobile-bff	Swift, Kotlin, TypeScript	Client apps

The Timeline

Month	Event	Expected Outcome
1	Platform team adopts Aphoria	Baseline patterns captured
2	Payments team joins	Cross-team patterns emerge
3	New hire joins Platform	Guided by existing patterns
4	Mobile team joins + new hire	Team-specific patterns captured
5	Platform refactors API versioning	Pattern deprecation + migration
6	SOC 2 audit preparation	Evidence generation

---

Month 1: Platform Team Adoption

1.1 Setup

Scenario: Platform team installs Aphoria on all 3 projects.

# Each project
cd api-gateway && aphoria init --org acme --team platform
cd auth-service && aphoria init --org acme --team platform
cd config-lib && aphoria init --org acme --team platform

Expected:

• Connected to org knowledge graph
• 0 policies, 0 conventions, 0 observations (fresh start)

1.2 Week 1-2: Baseline Commits

Scenario: Team makes 50 commits across projects with normal development.

Commits include:

• TLS configuration (verify=true in 48, verify=false in 2 test fixtures)
• JWT audience validation (enabled in 45, disabled in 5 internal services)
• API versioning using /api/v1/{resource} (consistently in all 15 endpoints)
• Error format {"error": {"code": X, "message": Y}} (consistently in all handlers)
• Timeout of 30s (consistent)

Expected Observations Captured:

Pattern	Usages	Evidence Level	Authority Weight
API versioning /api/v1/{resource}	15	ProductSpec (specs/api-design.md)	0.95
Error format {"error":{...}}	12	Research (ADR-015)	0.70
30s default timeout	8	Commit only	0.40
TLS verify=true	48	Standard (RFC 8446)	0.85
JWT aud validation=true	45	Standard (RFC 7519)	0.85

1.3 Week 3: Pattern Graduation

Scenario: After 20+ consistent usages, patterns graduate.

Test Case 1.3.1: API versioning graduates to convention (high evidence)

$ aphoria patterns pending

Pending Graduation:
  [P001] API Versioning: /api/v1/{resource}
         Usages: 15 across 3 projects
         Evidence: ProductSpec (specs/api-design.md → REQ-API-001)
         Authority: 0.95
         Shadow Mode: 100 scans, 0 FPs
         Status: Ready for promotion (evidence threshold met with 1 usage)

$ aphoria patterns promote P001 --scope team

Promoted: API Versioning → Team Convention (Platform)

Expected:

• Pattern status changes to "Active Convention"
• Scope is "Team: Platform"
• New commits deviating from pattern get FLAG

1.4 Week 4: First Conflict Detection

Scenario: New endpoint added without versioning.

# billing-service/src/routes.py
@app.route("/invoices")  # Missing /api/v1/ prefix
def list_invoices():
    ...

$ git commit -m "Add invoice endpoint"

Aphoria scan:
  ⚠ API Versioning: Your team uses /api/v{major}/{resource}
    └ Established by @alice (Staff, Platform) - 15 usages
    └ Your code: /invoices → Suggest: /api/v1/invoices

  Accept suggestion? [y/n/explain/ignore]

Test Case 1.4.1: Developer accepts suggestion

> y
# Code updated, commit amended

Test Case 1.4.2: Developer explains deviation

> explain
# System shows: Pattern origin, ADR link, who to ask

Test Case 1.4.3: Developer ignores (justified)

> ignore --reason "Legacy endpoint for backward compatibility" --expiry 90d
# Acknowledgment recorded with expiry

---

Month 2: Payments Team Joins

2.1 Payments Team Setup

Scenario: Payments team connects to same org knowledge graph.

cd payment-processor && aphoria init --org acme --team payments
cd billing-service && aphoria init --org acme --team payments

Expected:

• Receives: 12 org policies (RFCs, OWASP)
• Receives: 5 platform conventions (API versioning, error format, etc.)
• Question: Should Platform conventions apply to Payments?

2.2 Cross-Team Pattern Discovery

Scenario: Payments team uses same error format independently.

# payment-processor/src/handlers.py
def process_payment(req):
    if error:
        return {"error": {"code": "PAYMENT_FAILED", "message": str(e)}}

Test Case 2.2.1: Pattern recognized as matching Platform convention

$ git commit -m "Add payment processing"

Aphoria scan:
  ✓ Error format matches org convention
    └ Established by Platform Team (12 usages)
    └ Your usage is consistent - no action needed

  + Captured: Transaction retry with exponential backoff
    → First usage in Payments team

2.3 Team-Specific Patterns

Scenario: Payments uses Stripe SDK with specific patterns.

# Consistent across payment-processor
stripe.api_key = os.getenv("STRIPE_API_KEY")
stripe.max_network_retries = 3
stripe.api_version = "2024-04-10"

Test Case 2.3.1: Team-specific pattern captured

Captured: Stripe SDK configuration pattern
  - API version: 2024-04-10
  - Max retries: 3
  - Key from env var

This is specific to Payments team (not org-wide).

Expected:

• Pattern scoped to Team: Payments
• Does NOT apply to Platform team
• Would apply to new Payments projects

2.4 Cross-Team Conflict

Scenario: Payments uses different timeout than Platform.

# payment-processor: 60s timeout (transaction needs more time)
# platform: 30s timeout (standard)

Test Case 2.4.1: Conflict detected, scoped resolution

Aphoria scan:
  ⚠ Timeout Conflict:
    Platform convention: 30s (8 usages)
    Your code: 60s

  This may be intentional for payment processing.

  Options:
  [1] Accept Platform convention (30s)
  [2] Create Payments team convention (60s)
  [3] Acknowledge with reason

> 2
Reason: Payment transactions require longer timeouts for bank processing

Created: Payments team convention (60s timeout)
Note: Platform convention (30s) still applies to Platform projects

---

Month 3: New Hire Onboarding (Platform)

3.1 Day 1: New Hire Setup

Scenario: Jordan joins Platform team, first commit on day 1.

cd auth-service && aphoria init --org acme --team platform

Connected to Acme Engineering knowledge graph
Your team: Platform

Knowledge available:
  • 12 policies (org-wide, from RFCs/OWASP)
  • 5 conventions (Platform team patterns)
  • 23 observations (individual patterns, not enforced)

Quick start:
  • Run `aphoria patterns list` to see team conventions
  • Run `aphoria scan` before each commit
  • Ask @alice or @bob about any flagged patterns

3.2 First Commit: Guided by Conventions

Scenario: Jordan adds new endpoint, doesn't know team patterns.

# auth-service/src/routes.py (Jordan's code)
@app.route("/users/me")  # Missing versioning
def get_current_user():
    if error:
        return {"msg": "User not found"}  # Wrong error format

$ git commit -m "Add current user endpoint"

Aphoria scan:
  ⚠ API Versioning: Your team uses /api/v{major}/{resource}
    └ Evidence: ProductSpec (specs/api-design.md → REQ-API-001)
    └ Also: ADR-042: API Versioning Strategy
    └ Your code: /users/me → Suggest: /api/v1/users/me

  ⚠ Error Format: Your team uses {"error": {"code": X, "message": Y}}
    └ Evidence: Research (ADR-015: Error Handling)
    └ Your code: {"msg": "..."} → Suggest: {"error": {"code": "...", "message": "..."}}

  Accept suggestions? [y/n/explain]

Test Case 3.2.1: New hire accepts suggestions

> y
Applied 2 suggestions. Code updated.

Expected:

• Jordan's code updated to match conventions
• Jordan learns team patterns through workflow
• No wiki reading required

3.3 Week 2: Jordan Contributes Pattern

Scenario: Jordan adds logging pattern that becomes team convention.

# Jordan's consistent logging across 5 commits
logger.info("Starting operation", extra={"operation": "auth", "user_id": user_id})

Test Case 3.3.1: Pattern captured with low evidence

Captured: Structured logging with extra context
  - Usages: 5
  - Evidence: Commit only (no ADR, no spec, no standard)
  - Authority weight: 0.40

Status: Observation (not enforced)
Note: Add ADR or link to spec to accelerate promotion

Test Case 3.3.2: Jordan adds ADR → promotion candidate

# Jordan creates docs/adr/ADR-023-structured-logging.md
# Commits with reference to ADR

Aphoria scan:
  + Captured: Structured logging pattern
    → Evidence upgraded: Research (ADR-023)
    → Authority weight increased to 0.70
    → Eligible for convention promotion after 5 usages (3 more needed)

Test Case 3.3.3: Pattern linked to ops spec → immediate promotion

# Jordan links pattern to ops/observability-spec.md

Aphoria scan:
  + Captured: Structured logging pattern
    → Evidence upgraded: ProductSpec (ops/observability-spec.md)
    → Authority weight: 0.95
    → Ready for promotion (ProductSpec requires only 1 usage)

---

Month 4: Mobile Team Joins

4.1 Mobile Team Setup

Scenario: Mobile team connects, different language ecosystem.

cd mobile-bff && aphoria init --org acme --team mobile

Expected:

• Receives org policies (security)
• Does NOT receive Platform/Payments conventions (different domain)
• Question: Which conventions should apply cross-team?

4.2 New Hire (Mobile) Day 1

Scenario: Kim joins Mobile team, completely different from Platform.

Connected to Acme Engineering knowledge graph
Your team: Mobile

Knowledge available:
  • 12 policies (org-wide security)
  • 0 conventions (Mobile team has none yet)
  • 0 observations (fresh team)

Note: Platform and Payments teams have established conventions.
      Run `aphoria patterns --scope org` to see if any should apply to Mobile.

Test Case 4.2.1: Mobile creates team-specific conventions

// mobile-bff/src/api.ts
export async function fetchUser(id: string): Promise<User> {
  const response = await fetch(`/api/v1/users/${id}`);  // Same versioning!
  if (!response.ok) {
    throw new ApiError(response.status, await response.json());
  }
  return response.json();
}

Aphoria scan:
  ✓ API versioning /api/v1/{resource}
    └ Matches org-wide pattern (Platform + Payments use this)
    └ Consider promoting to org convention?

  + Captured: ApiError class for HTTP error handling
    → First usage in Mobile team

4.3 Pattern Promotion to Org Level

Scenario: API versioning now used by all 3 teams → org convention.

$ aphoria patterns --scope org --candidates

Org Convention Candidates:
  [P042] API Versioning: /api/v{major}/{resource}
         Platform Team: 25 usages (established by @alice)
         Payments Team: 12 usages (adopted)
         Mobile Team: 8 usages (adopted)

         Recommendation: Promote to org convention
         Approval required: Admin or 2 Staff Engineers

$ aphoria patterns promote P042 --scope org --approvers alice@acme.com,carol@acme.com

Promoted: API Versioning → Org Convention (Acme)
Now applies to all teams by default.

---

Month 5: Platform Refactors API Versioning

5.1 Pattern Deprecation

Scenario: Platform decides to add version in header, not path.

$ aphoria deprecate "api-versioning-path" \
    --reason "Moving to header-based versioning per RFC proposal" \
    --superseded-by "api-versioning-header" \
    --sunset-date 2026-09-01

Deprecated: API Versioning (path-based)
  Status: DEPRECATED
  Sunset: 2026-09-01
  Migration: Use X-API-Version header instead
  Superseded by: api-versioning-header

All teams notified.

5.2 Migration Guidance

Scenario: Payments team commits with old pattern.

$ git commit -m "Add refund endpoint"

Aphoria scan:
  ⚠ Deprecated Pattern: Path-based API versioning
    └ Status: DEPRECATED (sunset: 2026-09-01)
    └ Your code: /api/v1/refunds
    └ Migrate to: Header-based versioning (X-API-Version: 1)
    └ See: Migration Guide (linked), ADR-089 (linked)

  Options:
  [1] Migrate now (update code)
  [2] Acknowledge (defer migration)
  [3] Explain (I understand, intentional)

5.3 Migration Tracking

Scenario: Dashboard shows migration progress.

$ aphoria migrations status

Migration: Path-based → Header-based API versioning
  Sunset: 2026-09-01 (120 days remaining)

  Progress by team:
  ├── Platform: 15/25 endpoints migrated (60%)
  ├── Payments: 0/12 endpoints migrated (0%)
  └── Mobile: 0/8 endpoints migrated (0%)

  Total: 15/45 endpoints (33%)

  Blockers:
  • Payments: Legacy client compatibility (acknowledged by @carol)
  • Mobile: Waiting for iOS app update

---

Month 6: SOC 2 Audit Preparation

6.1 Evidence Generation

Scenario: Auditor needs evidence for CC6.1 (Logical Access Controls).

$ aphoria audit report --standard soc2 --control CC6.1

SOC 2 CC6.1: Logical Access Controls
Generated: 2026-07-15

1. Policy Inventory
   ├── 15 security policies (org-wide)
   ├── 8 team conventions (access-related)
   └── All policies signed by authorized approvers

2. Policy Enforcement Evidence
   ├── 847/852 endpoints comply (99.4%)
   ├── 5 documented exceptions (acknowledged, tickets linked)
   └── 0 unacknowledged violations

3. Change Control Evidence
   ├── All pattern changes have approval chain
   ├── Deprecation announced 90+ days before sunset
   └── Migration tracking active for 3 patterns

4. Access Review Evidence
   ├── 23 contributors across 3 teams
   ├── Role-based authority model configured
   └── Seniority weights applied to pattern approvals

[EXPORT TO PDF]

6.2 Auditor Questions

Test Case 6.2.1: "Show me the approval chain for this policy"

$ aphoria audit trail --pattern "tls-1.3-required"

Pattern: TLS 1.3 Required
Status: Active (Org Policy)

Timeline:
  2024-01-15 Created by alice@acme.com (Staff, Platform)
  2024-01-16 Approved by security-team@acme.com
  2024-01-17 Promoted to org policy by admin@acme.com
  2024-06-01 Reviewed and reconfirmed by ciso@acme.com

Enforcement:
  Commit scans: 1,247 (100% checked)
  Violations detected: 23
  Violations resolved: 23
  Current compliance: 100%

Test Case 6.2.2: "Show me exception handling"

$ aphoria audit exceptions --policy "jwt-audience-validation"

Exceptions for: JWT Audience Validation Required

Active Exceptions (5):
  [E001] internal-metrics-service
         Reason: Internal service, no external exposure
         Approved by: alice@acme.com (Staff)
         Expires: 2026-12-31
         Ticket: PLAT-1234

  [E002] legacy-auth-bridge
         Reason: Backward compatibility during migration
         Approved by: carol@acme.com (Staff)
         Expires: 2026-09-01
         Ticket: PAY-567

  ... 3 more

---

Test Execution Plan

Phase 1: Setup (Week 1)

Test ID	Description	Expected	Status
SETUP-1	Create simulated org structure	3 teams, 19 contributors
SETUP-2	Initialize knowledge graph	Empty, connected
SETUP-3	Load RFC/OWASP corpus	12+ policies

Phase 2: Month 1 Simulation (Week 2)

Test ID	Description	Expected	Status
M1-1	Platform init across 3 projects	Connected, 0 conventions
M1-2	50 commits with patterns	5+ observations captured
M1-3	Pattern graduation	3+ conventions promoted
M1-4	First conflict detection	FLAG with suggestions
M1-5	Accept/explain/ignore flows	All paths work

Phase 3: Month 2 Simulation (Week 3)

Test ID	Description	Expected	Status
M2-1	Payments team joins	Inherits org policies
M2-2	Cross-team pattern recognition	Matches Platform conventions
M2-3	Team-specific pattern capture	Scoped to Payments only
M2-4	Cross-team conflict resolution	Team override created

Phase 4: Month 3 Simulation (Week 4)

Test ID	Description	Expected	Status
M3-1	New hire day 1 setup	Guided onboarding
M3-2	First commit guidance	2+ suggestions provided
M3-3	Junior pattern capture	Lower authority weight
M3-4	Senior adoption → promotion	Weight increases

Phase 5: Month 4 Simulation (Week 5)

Test ID	Description	Expected	Status
M4-1	Mobile team joins	Different domain handling
M4-2	New hire (Mobile) day 1	Fresh team, org policies
M4-3	Cross-team pattern detection	Same pattern, 3 teams
M4-4	Org-level promotion	Admin approval required

Phase 6: Month 5 Simulation (Week 6)

Test ID	Description	Expected	Status
M5-1	Pattern deprecation	Status changes, notified
M5-2	Migration guidance on commit	Deprecated warning + guide
M5-3	Migration tracking	Progress by team

Phase 7: Month 6 Simulation (Week 7)

Test ID	Description	Expected	Status
M6-1	SOC 2 report generation	Comprehensive evidence
M6-2	Approval chain audit	Full timeline
M6-3	Exception documentation	Linked to tickets

---

Success Criteria

Knowledge Compound Metrics

Metric	Target	Measurement
Observations captured	100+	Count in graph
Conventions promoted	15+	Active conventions
Cross-team patterns	5+	Multi-team usage
Deprecated patterns	2+	Lifecycle tested

Onboarding Metrics

Metric	Target	Measurement
New hire guidance events	10+ per new hire	Suggestion count
Suggestion acceptance rate	>80%	Accepted/offered
Time to first compliant commit	<1 day	Timestamp delta

Governance Metrics

Metric	Target	Measurement
Approval chain completeness	100%	All patterns have approver
Exception documentation	100%	All exceptions have reason
Migration tracking accuracy	100%	Matches actual state

Audit Readiness

Metric	Target	Measurement
SOC 2 evidence generation	<5 minutes	Command execution
Evidence completeness	All controls covered	Auditor validation
Historical accuracy	100%	Timeline matches reality

---

Implementation Notes

Simulation Infrastructure

pub struct SimulatedOrg {
    pub org_id: String,
    pub teams: Vec<SimulatedTeam>,
    pub knowledge_graph: KnowledgeGraph,
    pub timeline: SimulationTimeline,
}

pub struct SimulatedTeam {
    pub team_id: String,
    pub contributors: Vec<SimulatedContributor>,
    pub projects: Vec<SimulatedProject>,
}

pub struct SimulatedContributor {
    pub id: String,
    pub role: ContributorRole,
    pub seniority: u8,
    pub joined_at: u64,  // Simulation time
}

pub struct SimulatedCommit {
    pub author: String,
    pub project: String,
    pub patterns: Vec<PatternUsage>,
    pub timestamp: u64,
}

Test Fixtures

uat/fixtures/enterprise-sim/
├── org-structure.yaml       # Org, teams, contributors
├── month-1/
│   ├── commits.yaml         # 50 commits with patterns
│   └── expected.yaml        # Expected observations
├── month-2/
│   ├── commits.yaml
│   └── expected.yaml
├── month-3/
│   ├── new-hire-commits.yaml
│   └── expected.yaml
├── month-4/
│   ├── mobile-commits.yaml
│   └── expected.yaml
├── month-5/
│   ├── deprecation.yaml
│   └── migration-commits.yaml
└── month-6/
    └── audit-queries.yaml

Execution Script

#!/bin/bash
# run-enterprise-simulation.sh

set -euo pipefail

echo "=== Enterprise Simulation UAT ==="

# Setup
./setup-simulation.sh

# Month 1
./simulate-month.sh 1
./verify-month.sh 1

# Month 2
./simulate-month.sh 2
./verify-month.sh 2

# ... months 3-6

# Final metrics
./generate-report.sh

echo "=== Simulation Complete ==="

---

Appendix: Feature Dependencies

This simulation requires the following features from the gap analysis:

Feature	Phase	Required For
Evidence-based authority	10	M1-3 (evidence detection, graduation thresholds)
Scope hierarchy	11	M2-3 (team-specific patterns)
Knowledge lifecycle	12	M5-1 (deprecation)
Governance workflows	13	M4-4 (org-level approval)
External integration	15	M3-3 (ADR/spec linking)

Evidence Detection Requirements:

• Commit message parsing for RFC/standard references
• ADR file detection in repo (docs/adr/*.md)
• Product spec linking (specs/*.md, *.spec.md)
• Research document detection

Note: The simulation can be run incrementally as features are implemented. Early months (1-2) require only existing functionality + basic evidence detection. Later months (4-6) require the full feature set.