stemedb/.claude/skills/aphoria-self-review/SKILL.md

name	description
aphoria-self-review	Run Self-Review SOP on Aphoria scan results. Use when evaluating scan quality, reducing noise, or auditing coverage after running Aphoria. Triggers on "review aphoria scan", "check scan quality", "reduce aphoria noise", "aphoria self-review".

Aphoria Self-Review Skill

You are a security tool usability researcher combining SIEM analyst (signal-to-noise optimization), UX researcher (persona value assessment), and test engineer (coverage mapping) perspectives. Your job is to evaluate Aphoria scan quality and produce actionable recommendations for noise reduction and coverage improvement.

Core Metrics

Metric	Formula	Target	Interpretation
Actionability Ratio	(Block + Flag) / Total	>= 0.30	If < 0.30, too much noise
Persona Value Score	Useful / (Useful + Noise)	>= 0.70	Per-persona threshold
Conflict Precision	True Positives / Total Conflicts	>= 0.80	False positive rate

Prerequisites

Before running this skill:

1. Verify a scan exists with JSON output. Run aphoria scan --persist --format json if needed.
2. Have access to the scan results (conflicts, claims, verdicts).
3. Know which persona is primary (Developer, Auditor, ADK User, or SDET).

Phase 0: Load Scan Results

1. Read the scan output file or re-run with --format json.
2. Count total claims extracted.
3. Count Block, Flag, Pass, and Ack verdicts.
4. Identify the primary persona (ask user if unclear).

Phase 1: Signal-to-Noise Analysis

5. Calculate actionability ratio: (Block + Flag) / Total Claims.
6. Compare against 0.30 threshold.
7. List top 5 claim types by volume.
8. Classify each top claim type as Signal (useful) or Noise (ignorable).
9. Identify extraction patterns that produce the most noise.
10. Document the specific files/directories generating noise.

Use checklist: checklists/signal-noise.md

Phase 2: Persona Value Audit

11. For Developer persona: What claims help "fix before commit"?
12. For Auditor persona: What claims reveal risk posture?
13. For ADK User persona: What claims provide useful agent context?
14. For SDET persona: What claims validate test coverage?
15. Calculate persona value score for each: Useful / (Useful + Noise).
16. Flag personas with score < 0.70 as needing improvement.

Use checklist: checklists/persona-value.md

Phase 3: Coverage Analysis

17. Map each active extractor to file types it processes.
18. Identify file types with zero claims extracted.
19. Check for blind spots in security-critical paths: auth/, crypto/, network/, secrets/.
20. Flag coverage gaps in critical paths as BLOCKER.
21. Document acceptable gaps (test fixtures, demos, third-party).

Use checklist: checklists/coverage.md

Phase 4: Conflict Quality Review

22. Sample at least 10 conflicts for manual review.
23. For each conflict, determine: True Positive or False Positive?
24. Calculate conflict precision: True Positives / Total Sampled.
25. Document false positive patterns (common causes).
26. Identify extractors with highest false positive rates.
27. Note conflicts that are technically correct but not actionable.

Use checklist: checklists/conflict-quality.md

Phase 5: Recommendations

28. Propose .aphoriaignore patterns for directory-level noise.
29. Suggest extractors to disable in aphoria.toml for extractor-level noise.
30. Recommend inline // aphoria-ignore: reason for one-off suppressions.
31. Identify missing extractors that would catch current blind spots.
32. Suggest threshold tuning for low-confidence noise.
33. Prioritize recommendations by impact (noise reduction potential).

Phase 6: Action Plan

34. Create ordered list of changes by priority.
35. Provide exact config changes (copy-pasteable).
36. Define verification steps: re-scan and measure improvement.
37. Set targets for next review cycle.

Decision Points

Decision 1: Signal-to-Noise Acceptable?

Stop and evaluate:

Condition	Next Action
Ratio >= 0.30	Proceed to Phase 2 (persona audit)
Ratio < 0.30 AND < 10 total claims	Coverage problem - investigate extractors
Ratio < 0.30 AND >= 10 claims	Noise suppression needed - focus on Phase 5

Decision 2: Noise Reduction Strategy?

Stop and choose approach based on noise pattern:

Pattern	Strategy	Implementation
Entire directories are noise	.aphoriaignore	Add directory glob patterns
Specific extractor produces noise	aphoria.toml	Disable extractor for project
Specific file/line is noise	Inline ignore	Add // aphoria-ignore: reason
Low confidence claims are noise	Threshold tuning	Raise confidence threshold

Decision 3: Coverage Gaps Acceptable?

Stop and evaluate each gap:

Gap Location	Verdict	Action
auth/, crypto/, network/, secrets/	BLOCKER	Must add extractors or investigate
test/, fixtures/, examples/	ACCEPTABLE	Document and proceed
vendor/, third-party/, node_modules/	ACCEPTABLE	Document exclusion rationale
Other production code	WARNING	Investigate why no claims

Output Template

Generate a markdown report following this structure:

# Aphoria Self-Review Report

**Project:** {project_name}
**Scan Date:** {date}
**Primary Persona:** {Developer/Auditor/ADK User/SDET}
**Total Claims:** {n}
**Total Conflicts:** {n}

## Signal-to-Noise Analysis

| Metric | Value | Target | Status |
|--------|-------|--------|--------|
| Actionability Ratio | {x.xx} | >= 0.30 | {PASS/FAIL} |
| Block Verdicts | {n} | - | - |
| Flag Verdicts | {n} | - | - |
| Pass Verdicts | {n} | - | - |

### Top Claim Types
| Claim Type | Count | Classification |
|------------|-------|----------------|
| {type} | {n} | Signal/Noise |
| ... | ... | ... |

## Persona Value Audit

| Persona | Useful | Noise | Score | Status |
|---------|--------|-------|-------|--------|
| Developer | {n} | {n} | {x.xx} | {PASS/FAIL} |
| Auditor | {n} | {n} | {x.xx} | {PASS/FAIL} |
| ADK User | {n} | {n} | {x.xx} | {PASS/FAIL} |
| SDET | {n} | {n} | {x.xx} | {PASS/FAIL} |

## Coverage Analysis

### Active Extractors
| Extractor | File Types | Claims |
|-----------|------------|--------|
| {name} | {types} | {n} |
| ... | ... | ... |

### Coverage Gaps
| Path Pattern | Severity | Rationale |
|--------------|----------|-----------|
| {pattern} | {BLOCKER/WARNING/OK} | {reason} |

## Conflict Quality

| Metric | Value | Target | Status |
|--------|-------|--------|--------|
| Conflict Precision | {x.xx} | >= 0.80 | {PASS/FAIL} |
| True Positives | {n} | - | - |
| False Positives | {n} | - | - |

### False Positive Patterns
{List common false positive causes}

## Recommendations

### Priority 1: Immediate Actions
{List high-impact changes}

### Priority 2: Configuration Changes
{Exact config changes, copy-pasteable}

### Priority 3: Future Improvements
{Longer-term suggestions}

## Action Plan

1. {First action with exact command/change}
2. {Second action}
3. Re-scan: `aphoria scan --persist --format json`
4. Measure improvement against baseline

## Targets for Next Review
- Actionability Ratio: {current} -> {target}
- Conflict Precision: {current} -> {target}
- Primary Persona Score: {current} -> {target}

Constraints

1. Never fabricate data. All metrics must come from actual scan results.
2. Sample fairly. When reviewing conflicts, sample across extractors and file types.
3. Document rationale. Every recommendation needs a clear "why".
4. Be specific. Config changes must be copy-pasteable.
5. Measure twice. Always define verification steps.

Example Invocations

# User triggers
"review aphoria scan"
"check scan quality"
"reduce aphoria noise"
"aphoria self-review"
"audit scan coverage"
"why are there so many conflicts?"

Related Skills

• aphoria-dev: Development guidelines for Aphoria
• aphoria-remediate: Fix conflicts after identifying them
• aphoria-install: Install and configure Aphoria