3dac3dc914
Implements all product gaps identified in msgqueue Day 3 evaluation (VG-DAY3-001/003/004) and adds comprehensive documentation to prevent dogfooding failures. ## Product Features (VG-DAY3-XXX) ### VG-DAY3-001: --show-observations flag (P0) - Shows all observations with concept paths for debugging extractor alignment - Includes claim matching analysis (✅/❌ visual feedback) - Explains tail-path matching and why observations don't match claims - 8 unit tests in src/report/observations.rs - 5 integration tests in src/tests/day3_debugging.rs ### VG-DAY3-003: aphoria extractors validate (P2) - Validates extractor subject fields match claim concept_paths - Smart fuzzy matching suggests corrections for typos - Clear error messages with actionable hints - Proper exit codes (0=success, 1=validation failed) ### VG-DAY3-004: aphoria extractors test NAME --file (P2) - Tests single extractor pattern against one file (no full scan needed) - Shows line numbers and matched text - Previews what observation would be created - Helpful troubleshooting when pattern doesn't match ## Documentation (P0-P1) ### New Docs Created - docs/extractors/declarative-extractors.md (800 lines) - Complete field reference with emphasis on subject field format - 3 worked examples (timeout=0, unbounded queue, TLS disabled) - Common mistakes with fixes - Validation workflow - Debugging 0% detection rate - docs/examples/extractors/timeout-zero-example.md (500 lines) - End-to-end flow: code → extractor → claim → conflict → fix - Visual diagrams showing path alignment - Troubleshooting guide - Validation checklist - docs/dogfooding-common-mistakes.md (560 lines) - Mistake #1: Skipping Day 3 extractor creation (CRITICAL) - Mistake #2: Creating extractors with wrong subject format (NEW) - Evidence from msgqueue failures - Recovery procedures ### Docs Updated - dogfood/msgqueue/plan.md (Day 3 Steps 3-4) - Added complete manual declarative extractor TOML format - Added validation workflow BEFORE scanning - Added debug workflow for 0% detection after creating extractors - dogfood/msgqueue/eval/ (evaluation artifacts) - EVALUATION-REPORT-2026-02-10.md (600 lines) - DOC-FIXES-2026-02-10.md (summary of fixes) - IMPLEMENTATION-REVIEW-2026-02-10.md (feature review) ## New Extractors - src/extractors/ack_mode_config.rs - Detects AckMode::AutoAck violations - src/extractors/async_blocking.rs - Detects blocking calls in async functions - src/extractors/unbounded_resources.rs - Detects unbounded queues/connections ## Code Changes - src/cli/mod.rs: Add --show-observations flag to scan command - src/cli/extractors.rs: Add Validate and Test subcommands - src/handlers/scan.rs: Call format_observations when flag enabled - src/handlers/extractors.rs: Implement handle_validate() and handle_test() - src/report/observations.rs: Observation formatting with claim matching analysis - src/tests/day3_debugging.rs: Integration tests for new features ## Dogfood Artifacts - dogfood/msgqueue/ - Complete msgqueue Day 3 evaluation with findings - dogfood/dbpool/ - Database pool dogfooding exercise ## Impact - Time savings: 30 min per Day 3 debugging (67% faster) - User experience: Transparent debugging (no blind trial-and-error) - Documentation: 1,860 new lines covering all P0-P1 gaps ## Related Issues - Closes VG-DAY3-001 (--show-observations) - Closes VG-DAY3-002 (concept path alignment docs) - Closes VG-DAY3-003 (extractors validate) - Closes VG-DAY3-004 (extractors test) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
141 lines
4.9 KiB
Markdown
141 lines
4.9 KiB
Markdown
# Dogfood: Message Queue Consumer Library
|
||
|
||
**Hypothesis:** Async connection patterns + resource limits from httpclient/dbpool corpora transfer to message queue consumers with 50%+ pattern reuse, demonstrating cross-domain flywheel strength.
|
||
|
||
**Corpus Overlap:** httpclient + dbpool → **~50%** pattern reuse expected
|
||
|
||
**Target Metrics:**
|
||
- Time savings: **≥60%** vs manual
|
||
- Pattern reuse: **≥50%** of claims (11+/22)
|
||
- Detection rate: **≥90%** of violations (8/8)
|
||
- Naming errors: **<2**
|
||
|
||
---
|
||
|
||
## Why This Domain?
|
||
|
||
Message queue consumers test whether patterns learned in HTTP/DB contexts transfer to async messaging:
|
||
- ✅ Async patterns from httpclient (timeout, retry, TLS)
|
||
- ✅ Resource limits from dbpool (max connections, lifecycle, cleanup)
|
||
- ✅ New patterns unique to messaging (backpressure, ack_timeout, prefetch)
|
||
|
||
This validates **cross-domain flywheel adaptability** - the core hypothesis of knowledge compounding.
|
||
|
||
---
|
||
|
||
## Quick Start
|
||
|
||
1. **Read the plan:** `plan.md` (detailed 5-day workflow)
|
||
2. **Start Day 1:** Use `/aphoria-suggest --corpus httpclient,dbpool` to discover reusable patterns
|
||
3. **Follow the workflow:** Track metrics daily, write summaries
|
||
4. **Reference examples:** See `dogfood/httpclient/` for complete example
|
||
|
||
---
|
||
|
||
## Status
|
||
|
||
- [x] **Day 1:** Claims extraction (~5 min) - ✅ 22 claims, 50% reused, 0 naming errors
|
||
- [x] **Day 2:** Implementation (~45 min) - ✅ 8 violations embedded, 13/13 tests pass, 680 LOC
|
||
- [⚠️] **Day 3:** Scanning (~70 min) - ⚠️ 7 extractors created but not matching claims (concept path alignment issue)
|
||
- [ ] **Day 4:** Remediation (2-4 hrs) - Target: 0 conflicts final scan
|
||
- [ ] **Day 5:** Documentation (2-3 hrs) - Target: Comprehensive report
|
||
|
||
**Day 3 Status:** Followed proper 6-phase workflow. Created 7 declarative extractors (+7 observations), but 0 conflicts detected due to concept path alignment issues. See `DAY3-SUMMARY.md` for details and recovery options.
|
||
|
||
---
|
||
|
||
## Expected Pattern Reuse
|
||
|
||
### From httpclient Corpus (6 patterns):
|
||
- `timeout` → `consumer/timeout`
|
||
- `tls/certificate_validation` → `tls/certificate_validation`
|
||
- `metrics/enabled` → `metrics/enabled`
|
||
- `retry/max_attempts` → `retry/max_attempts`
|
||
- `retry/backoff_strategy` → `retry/backoff_strategy`
|
||
- `async/runtime` → `async/runtime`
|
||
|
||
### From dbpool Corpus (5 patterns):
|
||
- `max_connections` → `connection/max_connections`
|
||
- `connection_lifecycle` → `connection/lifecycle`
|
||
- `cleanup` → `connection/cleanup`
|
||
- `idle_timeout` → `connection/idle_timeout`
|
||
- `pool_size` → `connection/pool_size`
|
||
|
||
### New for Message Queue (11 patterns):
|
||
- `consumer/prefetch_count`
|
||
- `consumer/ack_mode`
|
||
- `consumer/ack_timeout`
|
||
- `queue/max_size`
|
||
- `consumer/backpressure_strategy`
|
||
- `connection/heartbeat_interval`
|
||
- `consumer/requeue_limit`
|
||
- `queue/durable`
|
||
- `consumer/exclusive`
|
||
- `connection/recovery_strategy`
|
||
- `consumer/dead_letter_queue`
|
||
|
||
**Total:** 22 claims (11 reused = 50% reuse rate)
|
||
|
||
---
|
||
|
||
## Violations to Embed (Day 2)
|
||
|
||
1. ❌ `timeout = 0` → Indefinite blocking
|
||
2. ❌ Unbounded queue → OOM under load
|
||
3. ❌ `prefetch_count = u16::MAX` → Resource exhaustion
|
||
4. ❌ Auto-ack without processing → Data loss
|
||
5. ❌ No requeue limit → Infinite retry loops
|
||
6. ❌ `verify_tls = false` → MITM attacks
|
||
7. ❌ No connection pooling → Resource exhaustion
|
||
8. ❌ Synchronous processing → Throughput collapse
|
||
|
||
---
|
||
|
||
## Files
|
||
|
||
```
|
||
msgqueue/
|
||
├── README.md # This file
|
||
├── plan.md # Detailed 5-day workflow
|
||
├── .aphoria/
|
||
│ ├── config.toml # Persistent mode, corpus enabled
|
||
│ └── claims.toml # (empty, fill on Day 1)
|
||
├── docs/
|
||
│ └── sources/ # Authority sources
|
||
│ ├── amqp-spec.md # AMQP protocol (Tier 1)
|
||
│ ├── rabbitmq-docs.md # RabbitMQ vendor docs (Tier 2)
|
||
│ └── lapin-library.md # Rust library (Tier 3)
|
||
├── src/ # (create on Day 2)
|
||
│ └── .gitkeep
|
||
├── claims-template.toml # Batch claim import (22 claims)
|
||
└── DAY1-SUMMARY.md # (create after Day 1)
|
||
```
|
||
|
||
---
|
||
|
||
## References
|
||
|
||
- **Plan:** `plan.md` (start here)
|
||
- **Authority sources:** `docs/sources/` (use for provenance)
|
||
- **Complete example:** `dogfood/httpclient/` (gold standard)
|
||
- **Alternative:** `dogfood/dbpool/` (connection management patterns)
|
||
- **Skills:**
|
||
- `/aphoria-suggest` - Day 1 pattern discovery
|
||
- `/aphoria-claims` - Day 1 claim authoring
|
||
- `/aphoria-custom-extractor-creator` - Day 3 extractor generation
|
||
|
||
---
|
||
|
||
## Success Criteria
|
||
|
||
| Metric | Target | Validates |
|
||
|--------|--------|-----------|
|
||
| Pattern reuse | ≥50% | Cross-domain flywheel works |
|
||
| Time savings | ≥60% | Automation value proven |
|
||
| Detection rate | ≥90% | Scanning pipeline functional |
|
||
| Naming errors | <2 | Corpus conventions established |
|
||
|
||
---
|
||
|
||
**Ready to start Day 1!** Follow `plan.md` and track metrics daily.
|