stemedb/ai-lookup/features/aphoria-flywheel.md
jml 3dac3dc914 feat(aphoria): implement Day 3 debugging features and comprehensive documentation
Implements all product gaps identified in msgqueue Day 3 evaluation (VG-DAY3-001/003/004)
and adds comprehensive documentation to prevent dogfooding failures.

## Product Features (VG-DAY3-XXX)

### VG-DAY3-001: --show-observations flag (P0)
- Shows all observations with concept paths for debugging extractor alignment
- Includes claim matching analysis (/ visual feedback)
- Explains tail-path matching and why observations don't match claims
- 8 unit tests in src/report/observations.rs
- 5 integration tests in src/tests/day3_debugging.rs

### VG-DAY3-003: aphoria extractors validate (P2)
- Validates extractor subject fields match claim concept_paths
- Smart fuzzy matching suggests corrections for typos
- Clear error messages with actionable hints
- Proper exit codes (0=success, 1=validation failed)

### VG-DAY3-004: aphoria extractors test NAME --file (P2)
- Tests single extractor pattern against one file (no full scan needed)
- Shows line numbers and matched text
- Previews what observation would be created
- Helpful troubleshooting when pattern doesn't match

## Documentation (P0-P1)

### New Docs Created
- docs/extractors/declarative-extractors.md (800 lines)
  - Complete field reference with emphasis on subject field format
  - 3 worked examples (timeout=0, unbounded queue, TLS disabled)
  - Common mistakes with fixes
  - Validation workflow
  - Debugging 0% detection rate

- docs/examples/extractors/timeout-zero-example.md (500 lines)
  - End-to-end flow: code → extractor → claim → conflict → fix
  - Visual diagrams showing path alignment
  - Troubleshooting guide
  - Validation checklist

- docs/dogfooding-common-mistakes.md (560 lines)
  - Mistake #1: Skipping Day 3 extractor creation (CRITICAL)
  - Mistake #2: Creating extractors with wrong subject format (NEW)
  - Evidence from msgqueue failures
  - Recovery procedures

### Docs Updated
- dogfood/msgqueue/plan.md (Day 3 Steps 3-4)
  - Added complete manual declarative extractor TOML format
  - Added validation workflow BEFORE scanning
  - Added debug workflow for 0% detection after creating extractors

- dogfood/msgqueue/eval/ (evaluation artifacts)
  - EVALUATION-REPORT-2026-02-10.md (600 lines)
  - DOC-FIXES-2026-02-10.md (summary of fixes)
  - IMPLEMENTATION-REVIEW-2026-02-10.md (feature review)

## New Extractors
- src/extractors/ack_mode_config.rs - Detects AckMode::AutoAck violations
- src/extractors/async_blocking.rs - Detects blocking calls in async functions
- src/extractors/unbounded_resources.rs - Detects unbounded queues/connections

## Code Changes
- src/cli/mod.rs: Add --show-observations flag to scan command
- src/cli/extractors.rs: Add Validate and Test subcommands
- src/handlers/scan.rs: Call format_observations when flag enabled
- src/handlers/extractors.rs: Implement handle_validate() and handle_test()
- src/report/observations.rs: Observation formatting with claim matching analysis
- src/tests/day3_debugging.rs: Integration tests for new features

## Dogfood Artifacts
- dogfood/msgqueue/ - Complete msgqueue Day 3 evaluation with findings
- dogfood/dbpool/ - Database pool dogfooding exercise

## Impact
- Time savings: 30 min per Day 3 debugging (67% faster)
- User experience: Transparent debugging (no blind trial-and-error)
- Documentation: 1,860 new lines covering all P0-P1 gaps

## Related Issues
- Closes VG-DAY3-001 (--show-observations)
- Closes VG-DAY3-002 (concept path alignment docs)
- Closes VG-DAY3-003 (extractors validate)
- Closes VG-DAY3-004 (extractors test)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-11 03:31:06 +00:00

5.1 KiB

Aphoria Flywheel

Last Updated: 2026-02-10 Confidence: High

Practical Truth

This is an AUTONOMOUS flywheel. LLMs drive it, not humans.

Without LLM layer: You manually create claims with aphoria corpus create, get naming wrong, scan finds 0 violations, waste 6 hours debugging. Manual workflow doesn't scale.

With LLM layer: LLM analyzes diffs, suggests claims with correct naming, enforces consistency, scan finds violations, flywheel spins autonomously.

LLM implementations:

  • Claude Code skills (/aphoria-claims, /aphoria-suggest) - Interactive agent workflow
  • Go ADK agents - Programmatic tool use, automated claim authoring
  • Any LLM with tool use - As long as it can call aphoria claims create with enforced naming

The autonomous loop: LLM analyzes code → suggests claims → enforces naming → scan aggregates patterns → better corpus → LLM has better context → better suggestions → loop.

What It Actually Is

  1. Scan code → Extractors find observations (e.g., max_connections = Option<T>)
  2. Check claims → Tail-path match against corpus claims (e.g., dbpool/max_connections must be required)
  3. Find gaps → Identify claims without extractors (uncovered claims)
  4. Create extractors → Dynamically generate extractors for uncovered existing claims
  5. Suggest claims → LLM identifies new patterns not yet in corpus
  6. Create more extractors → Generate extractors for new claims
  7. Aggregate patterns → High-adoption patterns auto-promote to community corpus
  8. Better corpus → Next scan catches more violations
  9. Loop

Critical: Tail-path matching is case-sensitive and uses last 2 path segments. dbpool/max_connections matches, dbpool/MaxConnections doesn't. Naming inconsistency breaks the entire flywheel.

Why LLM Layer Is Required

Workflow Time Naming Consistency Autonomy Result
Manual CLI (human) 4-6 hours for 27 claims Inconsistent (camelCase, snake_case mix) None Scan finds 0 violations (tail-path mismatch)
Claude skills (LLM) 1-2 hours for 27 claims Enforced (lowercase, slash-separated) Interactive Scan finds 7 violations ✓
Go ADK agent (LLM) Minutes for 27 claims Enforced Fully autonomous Scan finds 7 violations ✓

LLM layer auto-enforces:

  • Lowercase with underscores: max_connections not MaxConnections
  • Slash-separated paths: dbpool/config/max_connections
  • Hierarchical structure: {domain}/{component}/{property}
  • Consequence reasoning: "If X is Option, then Y breaks" (not just pattern matching)

Without LLM: Manual naming errors → tail-path mismatch → 0 violations detected → "Aphoria is broken"

With LLM: Autonomous reasoning over code → enforced naming → pattern aggregation → self-improving corpus

How the Flywheel Works

LLM workflows drive the autonomous loop. The implementation can be:

Claude Code Skills (Interactive Agent)

# Load skill in your development environment
/aphoria-claims

# Skill analyzes diff for claimable patterns
"Review this diff for claims"

# LLM enforces naming, suggests claims, you approve

Go ADK Agent (Fully Autonomous)

// Agent with aphoria_claims tool
// LLM calls: aphoria_claims_create(subject, predicate, value, explanation)
// Runs in CI/CD pipeline, no human in loop

Custom LLM Integration (Any Tool-Use LLM)

  • Give your LLM access to aphoria claims create CLI
  • Provide naming convention rules in system prompt
  • Let LLM analyze diffs and author claims programmatically
  • Examples: Cursor, Windsurf, custom agent frameworks

Scanning (Required for All Workflows)

# Scan with persistent mode (required for flywheel)
aphoria scan --persist --sync

# Observations saved → contribute to pattern aggregation → community corpus grows

Critical Requirements:

  • LLM workflow (skills, agents, or custom) for claim authoring
  • Persistent mode (--persist) for flywheel activation
  • Sync mode (--sync) for community learning
  • DON'T create claims manually (naming errors break tail-path matching)
  • DON'T use ephemeral mode (flywheel disabled)
  • DON'T mix naming conventions (case-sensitive matching)

Technical Detail (If You Care)

Tail-path matching:

// Corpus claim: "vendor://dbpool/config/max_connections"
// → tail_path = "config/max_connections" (last 2 segments)

// Observation: "dbpool/config/max_connections"
// → tail_path = "config/max_connections"
// MATCH ✓

// Observation: "dbpool/config/MaxConnections"
// → tail_path = "config/MaxConnections"
// NO MATCH ✗ (case-sensitive)

File Pointer: applications/aphoria/src/concept_index.rs:45-120 (tail-path extraction)