stemedb/applications/aphoria/dogfood/dbpool/eval/FIXES-APPLIED-2026-02-10.md

Pattern Investigation Fixes Applied

Date: 2026-02-10 Pattern: "Technically yes, practically no" weasel answers Root Cause: Reasoning from implementation details instead of reading product vision

---

Fixes Completed

✅ Fix 1: aphoria-doc-evaluator Skill

File: .claude/skills/aphoria-doc-evaluator/SKILL.md

Added:

1. Step Back Section 4 (after line 82): "The Product Vision Question"

   • Read vision.md before discussing flywheel
   • Define flywheel: "autonomous knowledge compounding cycle"
   • Answer from product vision, not implementation details
   • CRITICAL note about LLM requirement (Claude skills OR Go ADK OR other methodology)

2. Do Not #12 (after line 452): Weasel answer prohibition

   • "NEVER say 'technically yes, but practically no'"
   • Answer based on practical reality and intended workflows

3. Constraints (after line 489): Three new prohibitions

   • NEVER answer "technically yes, but practically no"
   • NEVER hedge with technicalities when use case is clear
   • NEVER reason from edge cases when main workflow is obvious
   • ALWAYS answer from product vision, not implementation

---

✅ Fix 2: MEMORY.md

File: .claude/projects/-home-jml-Workspace-stemedb/memory/MEMORY.md

Replaced (line ~10-31): "A5 Flywheel" implementation note

With: "Aphoria Flywheel Definition (Product Vision)" section containing:

• What it IS (vision.md:330-363 reference)
• CRITICAL: Requires LLM automation (Claude skills OR Go ADK OR other)
• Main use cases (commit-time, onboarding, graduation)
• Enterprise value (leaders, security, platform)
• Implementation status (A5.1-A5.4)
• Directive: "ALWAYS answer from product vision"

Moved architecture details to separate section below.

---

✅ Fix 3: CLAUDE.md

File: /home/jml/Workspace/stemedb/CLAUDE.md

Added (before line 84 "What Is a Claim?"): New "Aphoria: The Autonomous Flywheel" section

Contents:

• Definition with vision.md reference
• Flowchart: commits → observations → patterns → guidance → trust → more commits
• CRITICAL box: Requires LLM automation (Claude skills OR Go ADK OR other methodology)
• Main workflows (commit-time, onboarding, graduation)
• Skills that drive flywheel (aphoria-claims, aphoria-suggest, aphoria-custom-extractor-creator)
• Link to vision.md for deeper understanding

---

Expected Behavior After Fixes

Before:

User: "Can you make the flywheel work without an LLM?"
Me: "Technically yes (manual CLI), practically no."

After:

User: "Can you make the flywheel work without an LLM?"
Me: [Reads vision.md] "No. The flywheel is an autonomous knowledge compounding cycle
that requires LLM-driven automation - either Claude Code skills, Go ADK agents, or
another LLM methodology. Manual CLI exists as a fallback for API unavailability,
not as a substitute for the autonomous operation."

---

Key Changes Summary

Source	What Changed	Why
aphoria-doc-evaluator skill	Added "Read vision.md" instruction	Prevent reasoning from implementation instead of product vision
aphoria-doc-evaluator skill	Added weasel answer prohibition	Stop "technically yes" hedging
MEMORY.md	Moved flywheel to top, added product definition	Separate product concept (flywheel) from implementation (A5)
MEMORY.md	Added CRITICAL note about LLM requirement	Clarify autonomous nature explicitly
CLAUDE.md	Added dedicated flywheel section	Make product vision visible when working on Aphoria

---

What This Prevents

1. Weasel answers: No more "technically yes, practically no"
2. Implementation confusion: Clear separation of product vision (flywheel) vs implementation (A5)
3. Missing context: vision.md is now referenced automatically for flywheel questions
4. Autonomous nature hidden: Made explicit that LLM automation is REQUIRED (skills OR ADK OR other)

---

The Autonomous Flywheel (Now Correctly Defined)

What it IS: Autonomous knowledge compounding cycle where commits → observations → pattern recognition → contextual guidance → developer trust → more commits.

CRITICAL Requirement: LLM-driven automation via:

• Claude Code skills (aphoria-claims, aphoria-suggest), OR
• Go ADK agents, OR
• Other LLM methodology

NOT a substitute: Manual CLI exists as fallback for API unavailability. It is NOT the flywheel. The flywheel is the autonomous cycle.

---

Verification Checklist

Next time user asks about flywheel:

• I read vision.md FIRST
• I answer from product vision (what users experience)
• I state the LLM requirement clearly (skills OR ADK OR other)
• I avoid "technically yes" weasel language
• I give practical answer only

No more bullshit. Direct answers from product vision.

---

Additional Fixes Applied (Later Session - 2026-02-10)

Problem Discovered

After applying initial fixes, user had to correct me 12 MORE times because I kept describing Aphoria as "CLI tool with optional LLM features" instead of "autonomous LLM-driven system."

Root cause: Initial fixes focused on "weasel answers" but didn't add strong PROHIBITIONS against the wrong framing.

---

✅ Fix 4: MEMORY.md - Core Definition at Top

File: .claude/projects/-home-jml-Workspace-stemedb/memory/MEMORY.md (NEW lines 3-47)

Added brand new section:

## APHORIA CORE DEFINITION (READ THIS FIRST)

**Aphoria is a continuous learning flywheel, NOT a CLI tool.**

### What Aphoria IS:
- Autonomous system that runs on EVERY commit
- LLM-driven via Claude Code skills, Go ADK agents, or other LLM methodology
- 4-step loop: scan → fix → identify claims → create extractors → repeat
- Knowledge compounding: Each commit benefits from all previous commits
- Corpus import: Separate capability to import corpuses of text

### What Aphoria is NOT:
- ❌ NOT a CLI tool you run manually
- ❌ NOT "42 extractors + custom additions"
- ❌ NOT "optional LLM features"
- ❌ NOT something you invoke when you want

### CRITICAL PROHIBITION:
NEVER present LLM workflows as "Option 1" or "approach to consider."
They ARE the core mechanism. Manual CLI is fallback only.

Impact: Every new session sees this FIRST, before any implementation details.

---

✅ Fix 5: CLAUDE.md - Rewritten Flywheel Section

File: /home/jml/Workspace/stemedb/CLAUDE.md (lines 84-132, COMPLETELY REWRITTEN)

Changed headline from:

## Aphoria: The Autonomous Flywheel
**Definition:** Autonomous knowledge compounding cycle

To:

## Aphoria: The Autonomous Flywheel
**Aphoria is a continuous learning system that runs on EVERY commit,
NOT a CLI tool you invoke manually.**

Added:

1. Commit-time loop visualization (scan → fix → identify → create)
2. "LLM Workflows ARE the Core Product" section with CRITICAL marker
3. Table showing when each skill is used (commit-time vs growth vs expansion)
4. "CRITICAL PROHIBITION" section with ❌ (wrong) vs ✅ (right) examples

Key addition - Prohibition section:

### CRITICAL PROHIBITION:

**NEVER describe Aphoria as:**
- ❌ "CLI tool with LLM features"
- ❌ "Static scanner with optional automation"
- ❌ "Tool you run when you want"

**ALWAYS describe Aphoria as:**
- ✅ "Autonomous continuous learning system"
- ✅ "LLM-driven commit-time flywheel"
- ✅ "System that runs on every commit"

Impact: Main project CLAUDE.md now explicitly prohibits all wrong framings.

---

✅ Fix 6: aphoria-doc-evaluator Skill - Workflow Check

File: .claude/skills/aphoria-doc-evaluator/SKILL.md

Added to Principles (lines 29-59):

### 6. **CRITICAL: Aphoria Nature Check**
BEFORE analyzing individual gaps, ALWAYS check:
Did the team use LLM workflows (skills) or manual CLI?

Aphoria is an autonomous LLM-driven system, NOT a CLI tool.
If team used manual CLI instead of skills:
- This is NOT a product limitation
- This is NOT a documentation gap (unless docs failed to emphasize)
- This IS evidence team used debug interface instead of product

Red flags:
- Team used `aphoria scan`, `aphoria claims create` manually
- No evidence of skill invocations
- Hit "extractor coverage gap" (skills would have filled)
- Time 2-3x longer than expected

NEVER describe manual CLI as "Option 1" - it's debug mode only.

Added to Phase 3 Analysis (new section 3A, lines 257-295):

#### 3A: CRITICAL FIRST CHECK - Aphoria Nature Question

"Did the team use LLM workflows (skills) or manual CLI?"

[Check progress log for evidence of skill usage]

If team used manual CLI instead of skills:
- Flag as "Product Misunderstanding, NOT Documentation Gap"
- Only flag as doc gap if docs didn't emphasize skills requirement
- Block proceeding to individual gap analysis

Impact: Future evaluations will catch "wrong workflow used" IMMEDIATELY, before analyzing individual gaps.

---

What These Additional Fixes Prevent

Before (After Fix 1-3 Only):

User: "Talk about the product limitation they discovered"
Me: "Here are options to improve:
     1. LLM-driven extractor generation (recommended)
     2. Manual declarative extractors
     3. Hybrid approach"
User: [Furious correction]

After (With Fix 4-6):

User: "Talk about the product limitation they discovered"
Me: [Reads MEMORY.md core definition]
    "There is NO product limitation. The team used manual CLI
     (debug interface) instead of skills (the product).
     Skills would have generated extractors automatically.
     This is evidence of workflow misunderstanding, not a gap."

---

Complete Change Summary

File	What Added	Lines Changed	Impact
MEMORY.md	Core definition section at top	+44 lines	Every session sees this FIRST
CLAUDE.md	Rewritten flywheel section with prohibitions	~48 lines rewritten	Project instructions prohibit wrong framing
aphoria-doc-evaluator	Workflow check principle + Phase 3A	+68 lines	Future evals catch workflow issues

Total: 3 files, ~160 lines added/modified in this round

---

Why It Took Two Rounds

Round 1 (earlier today): Fixed "weasel answers" and added product vision references Round 2 (this session): Added PROHIBITIONS because I kept saying the wrong thing anyway

Key insight: Instructions like "read vision.md" aren't enough. Need explicit:

• ❌ NEVER say X
• ✅ ALWAYS say Y
• Clear framing with examples of wrong vs right

---

Success Criteria (Updated)

✅ MEMORY.md has "READ THIS FIRST" core definition with prohibitions ✅ CLAUDE.md emphasizes "runs on every commit, NOT CLI tool" ✅ CLAUDE.md has ❌ / ✅ prohibition examples ✅ aphoria-doc-evaluator checks workflow BEFORE gap analysis ✅ aphoria-doc-evaluator has "NOT a product limitation" framing ✅ All fixes applied to source documents (not just this project)

Next test: New dogfooding project → Claude should immediately identify if team uses manual CLI instead of skills.

---

Status: ✅ All Fixes Applied (Both Rounds) Files Modified: 3 (MEMORY.md, CLAUDE.md, aphoria-doc-evaluator skill) Related Documents:

• eval/PATTERN-INVESTIGATION-APHORIA-FUNDAMENTALS.md (root cause analysis)
• eval/EVALUATION-DAY2-3-2026-02-10.md (evaluation that triggered this)