stemedb/.claude/skills/aphoria-self-review/checklists/conflict-quality.md

Conflict Quality Checklist

Use this checklist during Phase 4 of the Self-Review SOP.

Sampling Strategy

• Total conflicts in scan: ___
• Sample size (minimum 10): ___
• Sampling approach: Random / Stratified by extractor / Stratified by file

Conflict Review Template

For each sampled conflict:

Conflict #1

• File:
• Line:
• Claim:
• Conflicting Authority:
• Conflict Score:
• Verdict: Block / Flag / Pass

Assessment:

• True Positive: Real conflict that matters
• False Positive: Not actually a conflict
• True but Not Actionable: Correct but user can't/won't fix

Rationale:

---

Conflict #2

• File:
• Line:
• Claim:
• Conflicting Authority:
• Conflict Score:
• Verdict: Block / Flag / Pass

Assessment:

• True Positive
• False Positive
• True but Not Actionable

Rationale:

---

(Repeat for 10+ conflicts)

Precision Calculation

Category	Count
True Positives
False Positives
True but Not Actionable
Total Sampled

Conflict Precision: True Positives / Total = ___ / ___ = ___

Status: PASS (>= 0.80) / FAIL (< 0.80)

False Positive Pattern Analysis

Group false positives by cause:

Pattern	Count	Examples	Root Cause
Wrong concept path matching
Stale authority data
Context not considered
Overly broad regex
Test/fixture misidentified
Other: ___

Extractor Quality

Extractor	Conflicts Sampled	True Positive Rate	Issues

High-Value True Positives

List conflicts that provided genuine value:

1. File: ___ — Why: ___
2. File: ___ — Why: ___
3. File: ___ — Why: ___

Problematic False Positives

List conflicts that wasted time or caused confusion:

1. File: ___ — Problem: ___
2. File: ___ — Problem: ___
3. File: ___ — Problem: ___

Authority Quality Issues

Authority Source	Issue	Impact

Recommendations from Conflict Review

Based on the conflict quality analysis:

1. Suppress these patterns: ___
2. Fix these extractors: ___
3. Update these authorities: ___
4. Tune these thresholds: ___

Outcome

• Sample size >= 10
• All conflicts assessed
• Precision calculated
• False positive patterns documented
• Problematic extractors identified
• Recommendations generated