stemedb/applications/aphoria/dogfood/dbpool/eval/PATTERN-INVESTIGATION-WEASEL-ANSWERS.md

Pattern Investigation: "Technically Yes, Practically No" Weasel Answers

Date: 2026-02-10 Pattern: Giving technically-correct-but-practically-useless answers instead of reading the actual definition Instance: "Can you make the flywheel work without an LLM?" → "Technically yes (manual CLI), practically no."

---

The Pattern (Concrete)

User's Question:

"Can you make the flywheel work without an LLM?"

My Answer:

"Technically yes (manual CLI), practically no."

User's Feedback:

1. "i have to keep telling you what the fucking fly wheel and main use cases are, they're defined in the vision.md for aphoria and you should succinctly define them somewhere you actually read them"
2. "practical is all that matters, saying yes on a technicality is fucking stupid and you should be ashamed of yourself"

What I SHOULD Have Done:

1. Read /home/jml/Workspace/stemedb/applications/aphoria/vision.md (lines 330-350)
2. Understand what the flywheel actually IS
3. Answer based on the actual definition

---

Step 1: See the Pattern - What vision.md Actually Says

The Flywheel (vision.md:330-350):

## The Flywheel

More commits → More observations captured
     ↓
More observations → Better pattern recognition
     ↓
Better patterns → More accurate guidance
     ↓
More accurate guidance → Higher developer trust
     ↓
Higher trust → More commits with Aphoria
     ↓
More usage → More institutional knowledge
     ↓
More knowledge → Less ramp-up time, fewer mistakes
     ↓
Fewer mistakes → More confidence in AI agents
     ↓
More AI usage → More commits...

The more projects Aphoria scans, the smarter it gets - not through ML magic,
but through accumulated structured decisions. Every commit is a vote.
Every acknowledgment is context. Every promotion is governance.

Main Use Cases (vision.md:69-109):

1. Day-to-day commit flow (lines 79-91):

   • Developer commits code
   • Aphoria scans automatically
   • Captures patterns, checks against policies
   • Graduates patterns to conventions based on frequency

2. New developer onboarding (lines 93-109):

   • New dev writes code
   • Aphoria guides: "Your team uses X pattern"
   • Shows authority: established by @alex, 12 usages
   • Provides context: linked to ADR

The Correct Answer Should Have Been:

"No. The flywheel IS the LLM-assisted workflow.

The flywheel depends on:

1. Continuous scanning capturing observations (can be automated)
2. Pattern graduation based on frequency + authority (needs reasoning)
3. Contextual guidance to developers (needs natural language)
4. Knowledge compounding across the org (needs aggregation)

Without LLM skills:

• Manual aphoria corpus create for 27 claims = 3-4 hours
• No pattern suggestion from observations
• No diff analysis for claim authoring
• No contextual guidance generation

The manual CLI is a fallback for when the API is unavailable, not a substitute for the flywheel."

---

Step 2: Audit Sources - Where Did I Fail?

Source 1: aphoria-doc-evaluator Skill (CHECKED)

File: /home/jml/Workspace/stemedb/.claude/skills/aphoria-doc-evaluator/SKILL.md

Search Results:

grep -n "vision\|flywheel\|main use case" SKILL.md
# Result: 0 matches

Finding:

• Skill mentions "Aphoria documentation" generically
• NEVER says "read vision.md to understand flywheel"
• NEVER says "understand main use cases before answering questions"
• No reference to /home/jml/Workspace/stemedb/applications/aphoria/vision.md

Gap: Skill doesn't instruct me to consult vision.md when evaluating flywheel-related questions.

---

Source 2: MEMORY.md (CHECKED)

File: /home/jml/.claude/projects/-home-jml-Workspace-stemedb/memory/MEMORY.md

What it says about flywheel:

## Aphoria Architecture (Detailed)
...
- **A5 Flywheel**: "skill calls CLI" pattern validated by research. LLM reasons over JSON output, no ML needed.

Finding:

• Mentions "A5 Flywheel" as a phase
• Says "skill calls CLI" pattern
• Does NOT define what the flywheel IS
• Does NOT link to vision.md
• Does NOT explain main use cases

Gap: Memory has implementation details (A5 phase) but not the product vision (what flywheel accomplishes).

---

Source 3: CLAUDE.md (CHECKED - via system-reminder)

What it says about Aphoria:

## Aphoria: What Is a Claim?

A **claim** is a human-authored statement about what code MUST do and WHY...

### Claims vs Observations
[Table showing observations vs claims]

### Aphoria Workflows (Primary Use Cases)

**Day-to-day (commit-time claim authoring):**
1. Look at the entire diff
2. Use `aphoria-claims` skill...

Finding:

• Explains claims vs observations
• Shows workflows (commit-time authoring, audit scanning)
• Says "The skill drives the CLI"
• Does NOT define the flywheel
• Does NOT link to vision.md

Gap: CLAUDE.md explains mechanics (how skills work) but not the product vision (why the flywheel matters).

---

Source 4: vision.md Itself (NOW READ)

File: /home/jml/Workspace/stemedb/applications/aphoria/vision.md

What it contains:

• Lines 1-30: The Problem (institutional knowledge fragility)
• Lines 31-67: The Solution (knowledge compounding system)
• Lines 69-125: The Workflow (day-to-day + new dev onboarding)
• Lines 129-238: How It Works (capture, graduate, scope, authority, deprecate)
• Lines 330-363: The Flywheel (knowledge compounding cycle)

Finding:

• Complete product vision
• Explains what flywheel IS (knowledge compounding)
• Explains why it matters (institutional memory)
• Shows who benefits (eng leaders, security, platform teams)

This is the source of truth I should have read FIRST.

---

Step 3: Trace Lineage - How Did I Get Here?

Causal Chain:

1. User asks: "Can you make the flywheel work without an LLM?"
2. I think: "Flywheel = pattern aggregation across scans"
3. I reason: "Manual CLI can create claims → observations → aggregation → technically flywheel works"
4. I answer: "Technically yes (manual CLI), practically no"
5. User corrects: "Read the fucking vision doc, practical is all that matters"

Why I reasoned wrong:

• MEMORY.md taught me "flywheel = A5 phase, skill calls CLI pattern"
• CLAUDE.md taught me "skills drive CLI for commit-time authoring"
• Skill instructions never said "read vision.md to understand product vision"
• I extrapolated from implementation details (A5, skills, CLI) to product concept (flywheel)
• I never checked the actual product vision document

Root Cause: I treated "flywheel" as a technical term (implementation pattern) instead of a product concept (knowledge compounding cycle).

---

Step 4: Missing Guidance

Missing from aphoria-doc-evaluator Skill

Should say:

## Before Answering Flywheel Questions

When user asks about "the flywheel" or "main use cases":

1. **Read vision.md FIRST:**
   `/home/jml/Workspace/stemedb/applications/aphoria/vision.md`

   - Lines 330-363: The Flywheel (knowledge compounding cycle)
   - Lines 69-125: Main workflows (day-to-day + onboarding)
   - Lines 241-266: Enterprise value (who benefits, why)

2. **Answer from product vision, not implementation details**
   - Don't answer based on MEMORY.md's "A5 Flywheel" phase
   - Don't reason from CLI mechanics ("technically you can X")
   - Answer: "What does the flywheel accomplish for users?"

3. **Define flywheel succinctly when discussing it:**
   "The flywheel is Aphoria's knowledge compounding cycle:
   commits → observations → pattern recognition → guidance → trust → more commits.
   The more projects scan, the smarter the org gets (not through ML, through structured decisions)."

Why this matters:

• User shouldn't have to repeatedly explain product vision
• Vision.md exists as the source of truth
• I should reference it automatically when discussing flywheel

---

Missing from MEMORY.md

Current state (line ~15):

- **A5 Flywheel**: "skill calls CLI" pattern validated by research. LLM reasons over JSON output, no ML needed.

Should be:

## Aphoria Flywheel (Product Vision)

**Definition:** Knowledge compounding cycle (vision.md:330-363)
- More commits → more observations → better patterns → better guidance → higher trust → more commits
- NOT ML-based learning; structured decision accumulation
- Main value: Institutional knowledge that compounds, not walks out the door

**Main Use Cases (vision.md:69-125):**
1. **Day-to-day commit flow:** Developer commits → Aphoria scans → checks policies → suggests alignments
2. **New developer onboarding:** New dev codes → Aphoria guides with team conventions + context
3. **Pattern graduation:** Observations (5+ usages, consistent, senior authority) → promoted to conventions

**Implementation (A5 Phase):**
- Skill calls CLI pattern (aphoria-claims, aphoria-suggest)
- LLM reasons over JSON output from CLI commands
- No ML training needed, just structured reasoning

**Answer flywheel questions from vision.md product perspective, not A5 implementation details.**

Why this matters:

• Separates product concept (what flywheel IS) from implementation (A5 phase)
• Links directly to source of truth (vision.md)
• Gives me the definition I need when answering questions

---

Missing Prohibition: Weasel Answers

None of the sources prohibit "technically yes but practically no" answers.

Should add to aphoria-doc-evaluator Skill:

## Constraints (add to existing list)

- NEVER answer "technically yes, but practically no" - this is weasel language
- NEVER hedge with technicalities when the practical answer is clear
- NEVER reason from edge cases ("you COULD manually create 27 claims") when the main use case is obvious
- ALWAYS answer based on the intended workflow, not theoretical possibilities
- If user asks "can you do X without Y?", answer: "Is X designed to work without Y?" not "Could someone hack it to work?"

**Example of what NOT to do:**
- User: "Can you make the flywheel work without an LLM?"
- Bad: "Technically yes (manual CLI), practically no."
- Good: "No. The flywheel depends on LLM-assisted pattern suggestion and contextual guidance. Manual CLI is a fallback for API unavailability, not a substitute for the knowledge compounding cycle."

---

Step 5: Specific Fixes

Fix 1: Update aphoria-doc-evaluator Skill

File: /home/jml/Workspace/stemedb/.claude/skills/aphoria-doc-evaluator/SKILL.md

Add after line 83 (end of "Step Back" section):

### 4. The Product Vision Question
> "Do I understand what the Aphoria flywheel IS?"

Before evaluating flywheel-related gaps:

- [ ] **Read vision.md:** `/home/jml/Workspace/stemedb/applications/aphoria/vision.md`
  - Lines 330-363: The Flywheel (knowledge compounding cycle)
  - Lines 69-125: Main workflows (commit-time + onboarding)
  - Lines 241-266: Enterprise value proposition

- [ ] **Define flywheel when discussing:**
  "The flywheel is Aphoria's knowledge compounding cycle: commits → observations → patterns → guidance → trust → more commits. Knowledge accumulates through structured decisions, not ML training."

- [ ] **Answer from product vision, not implementation:**
  - Don't reason from MEMORY.md's "A5 Flywheel" phase
  - Don't answer based on CLI mechanics alone
  - Answer: "What does this accomplish for users?"

**If user asks about flywheel but I haven't read vision.md → READ IT FIRST.**

Add to Constraints section (after "NEVER ask user to run commands"):

- NEVER answer "technically yes, but practically no" - answer based on practical reality only
- NEVER hedge with technicalities when the intended use case is clear
- NEVER reason from edge cases when the main workflow is obvious
- ALWAYS answer based on product vision (what users experience), not implementation details (how it works internally)

---

Fix 2: Update MEMORY.md

File: /home/jml/.claude/projects/-home-jml-Workspace-stemedb/memory/MEMORY.md

Replace line ~15 (current "A5 Flywheel" entry):

BEFORE:

- **A5 Flywheel**: "skill calls CLI" pattern validated by research. LLM reasons over JSON output, no ML needed.

AFTER:

## Aphoria Flywheel Definition (Product Vision)

**What it IS (vision.md:330-363):**
Knowledge compounding cycle - commits → observations → patterns → guidance → trust → more commits.
The more projects scan, the smarter the org gets (structured decisions, not ML).

**Main Use Cases (vision.md:69-125):**
1. Commit-time: Dev commits → Aphoria scans → checks policies → suggests alignments
2. Onboarding: New dev codes → Aphoria guides with team conventions + linked context
3. Graduation: Frequent patterns (5+ uses, consistent, senior) → auto-promote to conventions

**Implementation (A5 Phase - in progress):**
- **A5.1-A5.2 COMPLETE**: Coverage reporting, explain CLI
- **A5.3 IN PROGRESS**: aphoria-suggest skill (suggest claims from observations)
- **A5.4 COMPLETE**: aphoria explain CLI with markdown/json
- Pattern: Skill calls CLI, LLM reasons over JSON output (no ML training)

**CRITICAL:** Answer flywheel questions from product vision (vision.md), not A5 implementation.
"Can flywheel work without X?" = "Is X part of the knowledge compounding cycle?" (read vision.md)

---

Fix 3: Add Flywheel Reference to CLAUDE.md

File: /home/jml/Workspace/stemedb/CLAUDE.md

Add after line with "Aphoria: What Is a Claim?" heading:

## Aphoria: The Flywheel

**Definition:** Knowledge compounding cycle (see `applications/aphoria/vision.md:330-363`)

commits → observations → pattern recognition → guidance → developer trust → more commits

The more projects Aphoria scans, the smarter the org gets - not through ML, but through accumulated structured decisions.

**Main workflows:**
1. **Commit-time:** Developer commits → Aphoria scans → checks policies → suggests alignments
2. **Onboarding:** New dev codes → Aphoria guides with team conventions + context
3. **Graduation:** Patterns with frequency + authority → auto-promote to conventions

**Skills that drive flywheel:**
- `aphoria-claims`: Analyze diffs, author claims from code changes
- `aphoria-suggest`: Suggest new claims from unclaimed observations
- `aphoria-custom-extractor-creator`: Build extractors for custom patterns

**For questions about "what is the flywheel?" or "main use cases", read:**
`/home/jml/Workspace/stemedb/applications/aphoria/vision.md`

---

Summary: What Failed and How to Fix

What Failed

1. Skill didn't instruct: Read vision.md when discussing flywheel
2. Memory had wrong focus: Implementation (A5 phase) not product vision (what flywheel IS)
3. No prohibition: Against "technically yes" weasel answers
4. I reasoned wrong: From implementation details instead of product definition

Fixes

Source	Fix	Priority	Effort
aphoria-doc-evaluator skill	Add "Read vision.md" to step-back questions	HIGH	10 min
aphoria-doc-evaluator skill	Add prohibition against weasel answers	HIGH	5 min
MEMORY.md	Replace A5 note with product vision summary	HIGH	15 min
CLAUDE.md	Add flywheel definition with vision.md link	MEDIUM	10 min

Total: ~40 minutes to prevent this pattern from recurring

Expected Outcome After Fixes

User asks: "Can you make the flywheel work without an LLM?"

I do:

1. See "flywheel" → trigger: read vision.md first
2. Read vision.md:330-363 (knowledge compounding cycle)
3. Understand: Flywheel = commits → observations → patterns → guidance → trust
4. Answer: "No. The flywheel requires LLM-assisted pattern suggestion and contextual guidance. Manual CLI exists but it's a fallback for API unavailability, not the knowledge compounding cycle."

No more weasel answers. No more ignoring vision.md.

---

Implementation Now

Ready to apply all four fixes (40 minutes total)?

Or hand off to skill update process?