stemedb/applications/aphoria/dogfood/SYSTEMATIC-FIXES-2026-02-10.md
jml 3dac3dc914 feat(aphoria): implement Day 3 debugging features and comprehensive documentation
Implements all product gaps identified in msgqueue Day 3 evaluation (VG-DAY3-001/003/004)
and adds comprehensive documentation to prevent dogfooding failures.

## Product Features (VG-DAY3-XXX)

### VG-DAY3-001: --show-observations flag (P0)
- Shows all observations with concept paths for debugging extractor alignment
- Includes claim matching analysis (/ visual feedback)
- Explains tail-path matching and why observations don't match claims
- 8 unit tests in src/report/observations.rs
- 5 integration tests in src/tests/day3_debugging.rs

### VG-DAY3-003: aphoria extractors validate (P2)
- Validates extractor subject fields match claim concept_paths
- Smart fuzzy matching suggests corrections for typos
- Clear error messages with actionable hints
- Proper exit codes (0=success, 1=validation failed)

### VG-DAY3-004: aphoria extractors test NAME --file (P2)
- Tests single extractor pattern against one file (no full scan needed)
- Shows line numbers and matched text
- Previews what observation would be created
- Helpful troubleshooting when pattern doesn't match

## Documentation (P0-P1)

### New Docs Created
- docs/extractors/declarative-extractors.md (800 lines)
  - Complete field reference with emphasis on subject field format
  - 3 worked examples (timeout=0, unbounded queue, TLS disabled)
  - Common mistakes with fixes
  - Validation workflow
  - Debugging 0% detection rate

- docs/examples/extractors/timeout-zero-example.md (500 lines)
  - End-to-end flow: code → extractor → claim → conflict → fix
  - Visual diagrams showing path alignment
  - Troubleshooting guide
  - Validation checklist

- docs/dogfooding-common-mistakes.md (560 lines)
  - Mistake #1: Skipping Day 3 extractor creation (CRITICAL)
  - Mistake #2: Creating extractors with wrong subject format (NEW)
  - Evidence from msgqueue failures
  - Recovery procedures

### Docs Updated
- dogfood/msgqueue/plan.md (Day 3 Steps 3-4)
  - Added complete manual declarative extractor TOML format
  - Added validation workflow BEFORE scanning
  - Added debug workflow for 0% detection after creating extractors

- dogfood/msgqueue/eval/ (evaluation artifacts)
  - EVALUATION-REPORT-2026-02-10.md (600 lines)
  - DOC-FIXES-2026-02-10.md (summary of fixes)
  - IMPLEMENTATION-REVIEW-2026-02-10.md (feature review)

## New Extractors
- src/extractors/ack_mode_config.rs - Detects AckMode::AutoAck violations
- src/extractors/async_blocking.rs - Detects blocking calls in async functions
- src/extractors/unbounded_resources.rs - Detects unbounded queues/connections

## Code Changes
- src/cli/mod.rs: Add --show-observations flag to scan command
- src/cli/extractors.rs: Add Validate and Test subcommands
- src/handlers/scan.rs: Call format_observations when flag enabled
- src/handlers/extractors.rs: Implement handle_validate() and handle_test()
- src/report/observations.rs: Observation formatting with claim matching analysis
- src/tests/day3_debugging.rs: Integration tests for new features

## Dogfood Artifacts
- dogfood/msgqueue/ - Complete msgqueue Day 3 evaluation with findings
- dogfood/dbpool/ - Database pool dogfooding exercise

## Impact
- Time savings: 30 min per Day 3 debugging (67% faster)
- User experience: Transparent debugging (no blind trial-and-error)
- Documentation: 1,860 new lines covering all P0-P1 gaps

## Related Issues
- Closes VG-DAY3-001 (--show-observations)
- Closes VG-DAY3-002 (concept path alignment docs)
- Closes VG-DAY3-003 (extractors validate)
- Closes VG-DAY3-004 (extractors test)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-11 03:31:06 +00:00

10 KiB

Systematic Fixes: Invalid Comparison Modes Across Dogfood Exercises

Date: 2026-02-10 Issue: Invalid ComparisonMode values (greater_than, less_than, in_range, max_value, min_value) used throughout dogfood exercises Root Cause: Template creation assumed numeric comparison operators exist in Aphoria Valid Modes: equals, not_equals, present, absent, contains, not_contains


Summary of Audit

msgqueue - FIXED

  • claims-template.toml - Fixed with valid comparison modes
  • FIXES-APPLIED.md - Documented the fix
  • .aphoria/claims.toml - Contains valid claims

dbpool - BROKEN (Extensive)

  • .aphoria/claims.toml - HAS ACTIVE INVALID CLAIMS
  • plan.md - Shows invalid predicates in tables
  • CHECKLIST.md - Templates teach invalid modes
  • CLAUDE.md - Documentation references invalid modes
  • claim-extraction-example.md - Examples use invalid modes
  • .aphoria/config.toml - Comments reference invalid modes
  • src/config.rs - Code comments reference invalid modes
  • docs/multi-project-setup.md - Guide shows invalid modes
  • Historical/archived docs - ⚠️ Contain invalid modes (leave as-is, historical record)

httpclient - UNKNOWN

  • No invalid modes found in active files
  • User reported immediate failure, but no template files exist

~/.claude/skills/ - CLEAN

  • No invalid comparison modes found

applications/aphoria/docs/ - CLEAN

  • No invalid comparison modes found

Critical Files to Fix

1. dbpool/.aphoria/claims.toml (ACTIVE CLAIMS - HIGHEST PRIORITY)

Location: /home/jml/Workspace/stemedb/applications/aphoria/dogfood/dbpool/.aphoria/claims.toml

Problem: Line 256 has predicate = "min_value" which is INVALID.

Impact: CRITICAL - This is the active claims file. Aphoria will fail to parse it.

Fix Required:

# WRONG:
[[claim]]
id = "dbpool-006"
concept_path = "dbpool/min_connections"
predicate = "min_value"
value = 2
comparison = "equals"

# CORRECT:
[[claim]]
id = "dbpool-006"
concept_path = "dbpool/min_connections"
predicate = "minimum"
value = 2
comparison = "equals"

Pattern: For numeric constraints, encode the constraint in the predicate name:

  • predicate = "min_value" predicate = "minimum"
  • predicate = "max_value" predicate = "maximum"
  • Or use predicate = "bounded" with value = true and document range in invariant

2. dbpool/plan.md (DOCUMENTATION - HIGH PRIORITY)

Location: /home/jml/Workspace/stemedb/applications/aphoria/dogfood/dbpool/plan.md

Problems:

  • Lines 111-113, 118: Tables show max_value, min_value as predicates
  • Lines 189, 197: Examples reference these invalid modes

Fix Required: Update all tables to show valid patterns:

# WRONG:
| `dbpool/min_connections` | `min_value` | `2` | HikariCP | 2 |

# CORRECT:
| `dbpool/min_connections` | `minimum` | `2` | HikariCP | 2 |
# OR
| `dbpool/min_connections/minimum` | `value` | `2` | HikariCP | 2 |
# OR (preferred - matches msgqueue pattern)
| `dbpool/min_connections` | `bounded` | `true` | HikariCP | 2 |
# (with invariant: "min_connections MUST be >= 2")

3. dbpool/CHECKLIST.md (TEMPLATES - HIGH PRIORITY)

Location: /home/jml/Workspace/stemedb/applications/aphoria/dogfood/dbpool/CHECKLIST.md

Problems:

  • Line 409: Template command shows max_value|min_value as options
  • Lines 468, 498-499, 504, 506, 630, 636: Examples use invalid modes

Fix Required: Update template to show only valid predicates:

# WRONG:
aphoria corpus create \
  --predicate "{required|recommended|max_value|min_value}" \

# CORRECT:
aphoria corpus create \
  --predicate "{required|recommended|bounded|version}" \

4. dbpool/CLAUDE.md (GUIDANCE - HIGH PRIORITY)

Location: /home/jml/Workspace/stemedb/applications/aphoria/dogfood/dbpool/CLAUDE.md

Problems:

  • Line 71: Template shows invalid predicates
  • Lines 178-179: Table references invalid modes

Fix Required: Same as CHECKLIST.md - update templates to valid modes.

5. dbpool/docs/claim-extraction-example.md (EXAMPLES - HIGH PRIORITY)

Location: /home/jml/Workspace/stemedb/applications/aphoria/dogfood/dbpool/docs/claim-extraction-example.md

Problems:

  • Lines 201, 222: Examples create claims with max_value, min_value

Fix Required: Rewrite examples to use valid comparison modes with proper encoding:

# WRONG:
aphoria corpus create \
  --subject "dbpool/connection_timeout" \
  --predicate "max_value" \
  --value "30"

# CORRECT (Option 1 - encode in predicate):
aphoria corpus create \
  --subject "dbpool/connection_timeout" \
  --predicate "maximum" \
  --value 30 \
  --comparison "equals"

# CORRECT (Option 2 - encode in concept_path):
aphoria corpus create \
  --subject "dbpool/connection_timeout/maximum" \
  --predicate "value" \
  --value 30 \
  --comparison "equals"

# CORRECT (Option 3 - msgqueue pattern):
aphoria corpus create \
  --subject "dbpool/connection_timeout" \
  --predicate "excessive" \
  --value 30 \
  --comparison "not_equals" \
  --invariant "connection_timeout MUST NOT exceed 30s (HikariCP)"

6. dbpool/.aphoria/config.toml (COMMENTS - MEDIUM PRIORITY)

Location: /home/jml/Workspace/stemedb/applications/aphoria/dogfood/dbpool/.aphoria/config.toml

Problems:

  • Lines 107, 123, 133: Comments reference max_value, min_value

Fix Required: Update comments to reflect actual valid predicates.

7. dbpool/src/config.rs (CODE COMMENTS - LOW PRIORITY)

Location: /home/jml/Workspace/stemedb/applications/aphoria/dogfood/dbpool/src/config.rs

Problems:

  • Lines 45, 57: Code comments reference invalid modes

Fix Required: Update comments to match actual claim structure after fixes.

8. dbpool/docs/multi-project-setup.md (GUIDE - MEDIUM PRIORITY)

Location: /home/jml/Workspace/stemedb/applications/aphoria/dogfood/dbpool/docs/multi-project-setup.md

Problems:

  • Lines 53, 119: Examples show max_value predicate

Fix Required: Update examples to valid predicates.


Files That DON'T Need Fixes

Historical/Archived Documents (Keep as-is)

  • dbpool/eval-archive-2026-02-09/ - Historical evaluation, keep for record
  • dbpool/DAY2-COMPLETE.md - Historical completion report
  • dbpool/verify-results-v1.json - Historical scan results
  • msgqueue/FIXES-APPLIED.md - Already documents the issue

Rationale: These are historical artifacts showing what happened on specific dates. Changing them would falsify the historical record.


Fix Strategy

Phase 1: Fix Active Claims (CRITICAL - Do First)

  1. msgqueue/.aphoria/claims.toml - Already fixed
  2. dbpool/.aphoria/claims.toml - Fix immediately

Phase 2: Fix Documentation (HIGH - Prevent Recurrence)

  1. dbpool/plan.md - Update tables
  2. dbpool/CHECKLIST.md - Update templates
  3. dbpool/CLAUDE.md - Update guidance
  4. dbpool/docs/claim-extraction-example.md - Update examples

Phase 3: Fix Auxiliary Files (MEDIUM - Consistency)

  1. dbpool/.aphoria/config.toml - Update comments
  2. dbpool/docs/multi-project-setup.md - Update guide

Phase 4: Fix Code Comments (LOW - Polish)

  1. dbpool/src/config.rs - Update comments

Validation After Fixes

Test 1: Validate Claims Files

# msgqueue
cd applications/aphoria/dogfood/msgqueue
aphoria claims import claims-template.toml --validate-only
# Expected: ✓ Validation passed

# dbpool
cd applications/aphoria/dogfood/dbpool
aphoria claims list --format json | jq '.claims[] | .predicate' | sort -u
# Expected: Only valid predicates (required, recommended, bounded, etc.)

Test 2: Search for Remaining Invalid Modes

cd applications/aphoria/dogfood
grep -r "greater_than\|less_than\|in_range\|max_value\|min_value" . \
  --exclude-dir=eval-archive-2026-02-09 \
  --exclude="*FIXES-APPLIED.md" \
  --exclude="*SYSTEMATIC-FIXES*.md" \
  --exclude="DAY2-COMPLETE.md" \
  --exclude="verify-results-v1.json"

# Expected: Only matches in historical/archived files

Test 3: Verify No Broken Templates

# Check that all templates use only valid comparison modes
grep -r "comparison.*=" applications/aphoria/dogfood/ \
  --include="*.toml" \
  --include="*.md" \
  | grep -v "equals\|not_equals\|present\|absent\|contains\|not_contains" \
  | grep -v "eval-archive"

# Expected: No matches (or only in historical files)

Prevention: Add to Skill Templates

Update ~/.claude/skills/aphoria-dogfood/SKILL.md to include warning:

## CRITICAL: Valid ComparisonMode Values

Aphoria's ComparisonMode enum ONLY supports:
-`equals`
-`not_equals`
-`present`
-`absent`
-`contains`
-`not_contains`

❌ INVALID (do not use):
-`greater_than`
-`less_than`
-`in_range`
-`max_value`
-`min_value`

**For numeric constraints:**
Encode the constraint in the predicate or concept_path:

Example:
```toml
# WRONG:
predicate = "max_value"
value = 30
comparison = "greater_than"  # INVALID

# CORRECT (Option 1):
predicate = "maximum"
value = 30
comparison = "equals"

# CORRECT (Option 2):
concept_path = "dbpool/connection_timeout/maximum"
predicate = "value"
value = 30
comparison = "equals"

# CORRECT (Option 3 - msgqueue pattern):
predicate = "excessive"
value = 30
comparison = "not_equals"

---

## Summary

### What Was Wrong
- **msgqueue:** Template used invalid comparison modes (fixed)
- **dbpool:** Extensive use of invalid predicates throughout documentation and active claims file
- **Root cause:** Template creation assumed numeric operators existed

### What's Fixed
- ✅ msgqueue claims template
- ✅ msgqueue documentation

### What's NOT Fixed
- ❌ dbpool active claims file (.aphoria/claims.toml)
- ❌ dbpool documentation (plan, checklist, CLAUDE.md)
- ❌ dbpool examples and guides

### Next Steps
1. Fix dbpool/.aphoria/claims.toml immediately (CRITICAL)
2. Fix all dbpool documentation systematically
3. Add validation to aphoria-dogfood skill to prevent recurrence
4. Consider adding a `--validate-claims` command to Aphoria CLI

### User Question Answered
> "did you fix things systemtically? our ~/.claude/**/*aphoria skills and our docs?"

**Answer:** NO
- ✅ ~/.claude/skills/ are clean (no invalid modes found)
- ✅ applications/aphoria/docs/ are clean
- ❌ dbpool dogfood is BROKEN extensively
- ✅ msgqueue dogfood is FIXED

**Action required:** Systematic fix of dbpool dogfood exercise following this report.