stemedb/ai-lookup/features/aphoria-flywheel.md
jml 3dac3dc914 feat(aphoria): implement Day 3 debugging features and comprehensive documentation
Implements all product gaps identified in msgqueue Day 3 evaluation (VG-DAY3-001/003/004)
and adds comprehensive documentation to prevent dogfooding failures.

## Product Features (VG-DAY3-XXX)

### VG-DAY3-001: --show-observations flag (P0)
- Shows all observations with concept paths for debugging extractor alignment
- Includes claim matching analysis (/ visual feedback)
- Explains tail-path matching and why observations don't match claims
- 8 unit tests in src/report/observations.rs
- 5 integration tests in src/tests/day3_debugging.rs

### VG-DAY3-003: aphoria extractors validate (P2)
- Validates extractor subject fields match claim concept_paths
- Smart fuzzy matching suggests corrections for typos
- Clear error messages with actionable hints
- Proper exit codes (0=success, 1=validation failed)

### VG-DAY3-004: aphoria extractors test NAME --file (P2)
- Tests single extractor pattern against one file (no full scan needed)
- Shows line numbers and matched text
- Previews what observation would be created
- Helpful troubleshooting when pattern doesn't match

## Documentation (P0-P1)

### New Docs Created
- docs/extractors/declarative-extractors.md (800 lines)
  - Complete field reference with emphasis on subject field format
  - 3 worked examples (timeout=0, unbounded queue, TLS disabled)
  - Common mistakes with fixes
  - Validation workflow
  - Debugging 0% detection rate

- docs/examples/extractors/timeout-zero-example.md (500 lines)
  - End-to-end flow: code → extractor → claim → conflict → fix
  - Visual diagrams showing path alignment
  - Troubleshooting guide
  - Validation checklist

- docs/dogfooding-common-mistakes.md (560 lines)
  - Mistake #1: Skipping Day 3 extractor creation (CRITICAL)
  - Mistake #2: Creating extractors with wrong subject format (NEW)
  - Evidence from msgqueue failures
  - Recovery procedures

### Docs Updated
- dogfood/msgqueue/plan.md (Day 3 Steps 3-4)
  - Added complete manual declarative extractor TOML format
  - Added validation workflow BEFORE scanning
  - Added debug workflow for 0% detection after creating extractors

- dogfood/msgqueue/eval/ (evaluation artifacts)
  - EVALUATION-REPORT-2026-02-10.md (600 lines)
  - DOC-FIXES-2026-02-10.md (summary of fixes)
  - IMPLEMENTATION-REVIEW-2026-02-10.md (feature review)

## New Extractors
- src/extractors/ack_mode_config.rs - Detects AckMode::AutoAck violations
- src/extractors/async_blocking.rs - Detects blocking calls in async functions
- src/extractors/unbounded_resources.rs - Detects unbounded queues/connections

## Code Changes
- src/cli/mod.rs: Add --show-observations flag to scan command
- src/cli/extractors.rs: Add Validate and Test subcommands
- src/handlers/scan.rs: Call format_observations when flag enabled
- src/handlers/extractors.rs: Implement handle_validate() and handle_test()
- src/report/observations.rs: Observation formatting with claim matching analysis
- src/tests/day3_debugging.rs: Integration tests for new features

## Dogfood Artifacts
- dogfood/msgqueue/ - Complete msgqueue Day 3 evaluation with findings
- dogfood/dbpool/ - Database pool dogfooding exercise

## Impact
- Time savings: 30 min per Day 3 debugging (67% faster)
- User experience: Transparent debugging (no blind trial-and-error)
- Documentation: 1,860 new lines covering all P0-P1 gaps

## Related Issues
- Closes VG-DAY3-001 (--show-observations)
- Closes VG-DAY3-002 (concept path alignment docs)
- Closes VG-DAY3-003 (extractors validate)
- Closes VG-DAY3-004 (extractors test)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-11 03:31:06 +00:00

120 lines
5.1 KiB
Markdown

# Aphoria Flywheel
**Last Updated:** 2026-02-10
**Confidence:** High
## Practical Truth
**This is an AUTONOMOUS flywheel.** LLMs drive it, not humans.
**Without LLM layer:** You manually create claims with `aphoria corpus create`, get naming wrong, scan finds 0 violations, waste 6 hours debugging. Manual workflow doesn't scale.
**With LLM layer:** LLM analyzes diffs, suggests claims with correct naming, enforces consistency, scan finds violations, flywheel spins autonomously.
**LLM implementations:**
- **Claude Code skills** (`/aphoria-claims`, `/aphoria-suggest`) - Interactive agent workflow
- **Go ADK agents** - Programmatic tool use, automated claim authoring
- **Any LLM with tool use** - As long as it can call `aphoria claims create` with enforced naming
**The autonomous loop:** LLM analyzes code → suggests claims → enforces naming → scan aggregates patterns → better corpus → LLM has better context → better suggestions → loop.
## What It Actually Is
1. **Scan code** → Extractors find observations (e.g., `max_connections = Option<T>`)
2. **Check claims** → Tail-path match against corpus claims (e.g., `dbpool/max_connections must be required`)
3. **Find gaps** → Identify claims without extractors (uncovered claims)
4. **Create extractors** → Dynamically generate extractors for uncovered existing claims
5. **Suggest claims** → LLM identifies new patterns not yet in corpus
6. **Create more extractors** → Generate extractors for new claims
7. **Aggregate patterns** → High-adoption patterns auto-promote to community corpus
8. **Better corpus** → Next scan catches more violations
9. **Loop**
**Critical:** Tail-path matching is case-sensitive and uses last 2 path segments. `dbpool/max_connections` matches, `dbpool/MaxConnections` doesn't. Naming inconsistency breaks the entire flywheel.
## Why LLM Layer Is Required
| Workflow | Time | Naming Consistency | Autonomy | Result |
|----------|------|-------------------|----------|--------|
| Manual CLI (human) | 4-6 hours for 27 claims | Inconsistent (camelCase, snake_case mix) | None | Scan finds 0 violations (tail-path mismatch) |
| Claude skills (LLM) | 1-2 hours for 27 claims | Enforced (lowercase, slash-separated) | Interactive | Scan finds 7 violations ✓ |
| Go ADK agent (LLM) | Minutes for 27 claims | Enforced | Fully autonomous | Scan finds 7 violations ✓ |
**LLM layer auto-enforces:**
- Lowercase with underscores: `max_connections` not `MaxConnections`
- Slash-separated paths: `dbpool/config/max_connections`
- Hierarchical structure: `{domain}/{component}/{property}`
- Consequence reasoning: "If X is Option<T>, then Y breaks" (not just pattern matching)
**Without LLM:** Manual naming errors → tail-path mismatch → 0 violations detected → "Aphoria is broken"
**With LLM:** Autonomous reasoning over code → enforced naming → pattern aggregation → self-improving corpus
## How the Flywheel Works
**LLM workflows drive the autonomous loop.** The implementation can be:
### Claude Code Skills (Interactive Agent)
```bash
# Load skill in your development environment
/aphoria-claims
# Skill analyzes diff for claimable patterns
"Review this diff for claims"
# LLM enforces naming, suggests claims, you approve
```
### Go ADK Agent (Fully Autonomous)
```go
// Agent with aphoria_claims tool
// LLM calls: aphoria_claims_create(subject, predicate, value, explanation)
// Runs in CI/CD pipeline, no human in loop
```
### Custom LLM Integration (Any Tool-Use LLM)
- Give your LLM access to `aphoria claims create` CLI
- Provide naming convention rules in system prompt
- Let LLM analyze diffs and author claims programmatically
- **Examples:** Cursor, Windsurf, custom agent frameworks
### Scanning (Required for All Workflows)
```bash
# Scan with persistent mode (required for flywheel)
aphoria scan --persist --sync
# Observations saved → contribute to pattern aggregation → community corpus grows
```
**Critical Requirements:**
- ✅ LLM workflow (skills, agents, or custom) for claim authoring
- ✅ Persistent mode (`--persist`) for flywheel activation
- ✅ Sync mode (`--sync`) for community learning
-**DON'T** create claims manually (naming errors break tail-path matching)
-**DON'T** use ephemeral mode (flywheel disabled)
-**DON'T** mix naming conventions (case-sensitive matching)
## Technical Detail (If You Care)
**Tail-path matching:**
```rust
// Corpus claim: "vendor://dbpool/config/max_connections"
// → tail_path = "config/max_connections" (last 2 segments)
// Observation: "dbpool/config/max_connections"
// → tail_path = "config/max_connections"
// MATCH ✓
// Observation: "dbpool/config/MaxConnections"
// → tail_path = "config/MaxConnections"
// NO MATCH ✗ (case-sensitive)
```
**File Pointer:** `applications/aphoria/src/concept_index.rs:45-120` (tail-path extraction)
## Related
- [Aphoria Claims Workflow](../../CLAUDE.md#aphoria-workflows-primary-use-cases) - Day-to-day usage
- [Claims vs Observations](../../CLAUDE.md#claims-vs-observations) - What's the difference
- [Naming Conventions](../../applications/aphoria/dogfood/dbpool/CHECKLIST.md) - Strict rules (coming)