stemedb/applications/aphoria/dogfood/httpclient/plan.md

# Dogfood Project 2: HTTP Client Library (`httpclient`)

**Status:** 🚀 READY TO START (Project 1 complete)
**Start Date:** TBD
**Target Completion:** TBD + 4 days (faster than Project 1)
**Owner:** Aphoria Development Team

---

## Executive Summary

Build a production-ready HTTP client library with **intentional violations** of HTTP best practices and security standards, then use Aphoria **skills** to detect violations through pattern reuse from Project 1 (dbpool). This demonstrates Aphoria's autonomous flywheel: **knowledge compounds across projects**.

**Key Metrics:**
- **Claims to Extract:** ~22 total (8-10 reused from dbpool, 12-14 new HTTP-specific)
- **Time for Day 1:** <2 hours (vs Project 1's 4 hours) - **50% faster via skills**
- **Intentional Violations:** 7-8
- **Expected Detection Rate:** 100% (with custom extractors via skills)
- **Final State:** 0 conflicts, production-ready

**Demonstration Value:**
- ✅ **Flywheel proof:** Reuses connection/timeout/TLS patterns from dbpool
- ✅ **Skills-driven:** `/aphoria-suggest` discovers patterns, `/aphoria-claims` enforces naming
- ✅ **Time savings:** 50-60% reduction through pattern reuse
- ✅ **Consistency:** 0 naming errors (skills enforce automatically)

---

## Product Overview

### What We're Building

**httpclient:** A safe, opinionated HTTP client library for Rust with connection pooling, timeout management, and TLS enforcement.

**Why This Product:**
1. **Pattern Reuse:** Shares connection management patterns with dbpool (demonstrates flywheel)
2. **High Stakes:** HTTP client misconfigurations cause timeout cascades, redirect loops, security vulnerabilities
3. **Clear Authority:** HTTP RFCs, Mozilla docs, Requests library provide canonical best practices
4. **Common Mistakes:** Developers frequently misconfigure timeouts, redirect limits, TLS verification
5. **Measurable ROI:** "Aphoria prevented timeout cascade in production" + "50% faster via pattern reuse"

### Scope

**Initial Implementation (v0.1.0):**
- HTTP client with connection pooling (reuses dbpool connection patterns)
- Timeout management (connection, request, total) - **ALIGNED with dbpool patterns**
- Redirect handling with configurable limits
- TLS configuration and certificate validation - **ALIGNED with dbpool TLS patterns**
- Retry logic with exponential backoff
- Request/response metrics

**Lines of Code:** ~700 (intentionally small for clarity)

**Dependencies:**
- `reqwest` for HTTP client (or build on `hyper` directly)
- `tokio` for async runtime
- `rustls` for TLS
- `serde` for configuration

---

## Authority Sources

### Primary Sources

1. **HTTP/1.1 RFCs (RFC 7230-7235)**
   - **URL:** https://tools.ietf.org/html/rfc7230
   - **Authority Tier:** Tier 0 (Standards)
   - **Expected Claims:** 5-7
   - **Key Topics:**
     - Redirect limits (RFC 7231: 10 max recommended)
     - Timeout behavior
     - Connection management
     - Header handling

2. **Mozilla HTTP Documentation**
   - **URL:** https://developer.mozilla.org/en-US/docs/Web/HTTP
   - **Authority Tier:** Tier 2 (Vendor/Industry Standard)
   - **Expected Claims:** 6-8
   - **Key Topics:**
     - Request timeouts
     - TLS/SSL best practices
     - Redirect policies
     - Connection pooling

3. **Requests Library Best Practices (Python)**
   - **URL:** https://requests.readthedocs.io/
   - **Authority Tier:** Tier 2 (Vendor - widely adopted HTTP library)
   - **Expected Claims:** 5-7
   - **Key Topics:**
     - Timeout configuration (connect vs read)
     - Session pooling
     - TLS verification
     - Retry strategies

### Secondary Sources (Reused from dbpool)

4. **OWASP A07:2021 - Identification and Authentication Failures**
   - **Claims Reused:** TLS enforcement, certificate validation, credential handling
   - **Expected Reuse:** 3-4 claims

5. **dbpool Connection Patterns**
   - **Claims Reused:** connection_timeout, max_connections patterns (adapted to HTTP context)
   - **Expected Reuse:** 4-5 claims

---

## Intentional Violations (7-8 Total)

### Safety Violations (4)

1. **Unbounded Redirect Limit**
   - **Violation:** `max_redirects: Option<usize>` = None (unbounded)
   - **Authority:** RFC 7231 Section 6.4 (10 redirects max recommended)
   - **Consequence:** Infinite redirect loop exhausts client resources
   - **Claim:** `httpclient/redirect_limit` predicate:`max_value` value:10
   - **Reuse:** Similar to dbpool's `max_connections` limit pattern

2. **Excessive Request Timeout**
   - **Violation:** `request_timeout: Duration` = 120s (too long)
   - **Authority:** Mozilla HTTP docs (30s recommended for requests)
   - **Consequence:** Slow external services block thread pool
   - **Claim:** `httpclient/request_timeout` predicate:`max_value` value:30
   - **Reuse:** ✅ **ALIGNED with dbpool/connection_timeout pattern**

3. **Excessive Connection Timeout**
   - **Violation:** `connect_timeout: Duration` = 60s
   - **Authority:** Requests library (10s recommended for connections)
   - **Consequence:** Unresponsive endpoints block connection establishment
   - **Claim:** `httpclient/connect_timeout` predicate:`max_value` value:10
   - **Reuse:** ✅ **ALIGNED with dbpool/connection_timeout pattern**

4. **Missing Idle Connection Timeout**
   - **Violation:** `idle_timeout: Option<Duration>` = None
   - **Authority:** HTTP keep-alive best practices
   - **Consequence:** Stale connections accumulate, wastes resources
   - **Claim:** `httpclient/idle_timeout` predicate:`required` value:true
   - **Reuse:** ✅ **ALIGNED with dbpool/max_lifetime pattern**

### Security Violations (3)

5. **TLS Certificate Verification Disabled**
   - **Violation:** `verify_tls: bool` = false
   - **Authority:** OWASP A07:2021, Mozilla TLS docs
   - **Consequence:** Man-in-the-middle attacks, credential exposure
   - **Claim:** `httpclient/tls/certificate_validation` predicate:`required` value:true
   - **Reuse:** ✅ **DIRECTLY reused from dbpool TLS pattern**

6. **Minimum TLS Version Too Low**
   - **Violation:** `min_tls_version: TlsVersion` = TLS 1.0
   - **Authority:** OWASP, Mozilla Security Guidelines (TLS 1.2 minimum)
   - **Consequence:** Vulnerable to protocol downgrade attacks
   - **Claim:** `httpclient/tls/min_version` predicate:`min_value` value:"1.2"
   - **Reuse:** ✅ **ALIGNED with dbpool TLS patterns**

7. **No Retry Limit**
   - **Violation:** `max_retries: Option<u32>` = None (unbounded)
   - **Authority:** Requests library (3 retries recommended)
   - **Consequence:** Retry storms amplify cascading failures
   - **Claim:** `httpclient/retry/max_attempts` predicate:`max_value` value:3
   - **Reuse:** Similar to dbpool's bounded resource pattern

### Optional Warning (Documentation)

8. **Missing Metrics Exposure**
   - **Violation:** No `metrics` field in config
   - **Authority:** Observability best practices
   - **Consequence:** Cannot monitor client health in production
   - **Claim:** `httpclient/metrics/enabled` predicate:`recommended` value:true
   - **Reuse:** ✅ **ALIGNED with dbpool/metrics pattern**

---

## Pattern Reuse from Project 1 (dbpool)

### Direct Reuse (5-6 claims)

| dbpool Claim | httpclient Claim | Adaptation |
|--------------|------------------|------------|
| `dbpool/connection_timeout` max_value:30 | `httpclient/request_timeout` max_value:30 | Same timeout, different context |
| `dbpool/tls/enabled` required | `httpclient/tls/certificate_validation` required | Same security requirement |
| `dbpool/tls/min_version` min_value:"1.2" | `httpclient/tls/min_version` min_value:"1.2" | Identical TLS policy |
| `dbpool/max_connections` required | `httpclient/max_redirects` max_value:10 | Bounded resource pattern |
| `dbpool/max_lifetime` required | `httpclient/idle_timeout` required | Connection lifecycle management |
| `dbpool/metrics/enabled` recommended | `httpclient/metrics/enabled` recommended | Observability pattern |

### Semantic Alignment (Naming Consistency)

**Pattern discovered by `/aphoria-suggest`:**
- dbpool uses `connection_timeout` not `timeout`
- dbpool uses `max_connections` not `connection_limit`
- dbpool uses `tls/` prefix for all TLS settings

**httpclient will align:**
- Use `connect_timeout` and `request_timeout` (not `timeout`)
- Use `max_redirects` (not `redirect_limit`)
- Use `tls/` prefix for certificate_validation, min_version

**Result:** Cross-project naming consistency enforced by skills

---

## 5-Day Plan

### Day 1: Extract Claims with Skills (1-2 hours, vs 4 hours for Project 1)

**PRIMARY WORKFLOW: Skills-Driven**

**Step 1: Pattern Discovery (15 min)**
```
/aphoria-suggest

"I'm building an HTTP client library. What patterns from dbpool should I reuse?
Focus on connection management, timeouts, and TLS."
```

**Expected skill output:**
- 5-6 reusable claims from dbpool
- Naming patterns to align with
- Cross-project consistency recommendations

**Step 2: Fetch HTTP Authority Sources (30 min)**
- Download RFC 7230-7235 sections
- Save Mozilla HTTP docs
- Save Requests library best practices
- **Save to:** `docs/sources/`

**Step 3: Extract Claims with Skills (30-45 min)**
```
/aphoria-claims

"Read docs/sources/http-rfcs.md and extract claims for HTTP client.

ALIGN NAMING with dbpool patterns:
- Use 'connect_timeout' and 'request_timeout' (match dbpool pattern)
- Use 'max_redirects' (match dbpool's max_connections pattern)
- Use 'tls/' prefix for all TLS settings (match dbpool)

Project prefix: httpclient/"
```

**Expected outcome:**
- 22 claims created (8-10 reused, 12-14 new)
- Perfect naming alignment with dbpool
- Completed in 1-2 hours (50% faster than Project 1)

---

### Day 2: Implementation (4-5 hours)

**Files to Create:**
```
src/
├── lib.rs           # Library root
├── config.rs        # ClientConfig (5 violations)
├── client.rs        # HttpClient (2-3 violations)
├── connection.rs    # Connection pool wrapper
├── retry.rs         # Retry logic
└── error.rs         # Error types

tests/
└── basic.rs         # Integration tests (23+ tests)

Cargo.toml           # Package manifest
```

**Implementation with Violations:**
- `config.rs`: Embed 5 violations (unbounded redirects, excessive timeouts, TLS disabled, etc.)
- `client.rs`: Embed 2-3 violations (no retry limit, missing metrics)
- **Document each violation inline** with `// VIOLATION:` comment
- All tests passing except violations are intentional

---

### Day 3: Scan with Skills (2-3 hours)

**Step 1: Initial Scan**
```bash
aphoria scan --persist --format json > scan-results-v1.json
```

**Expected (with built-in extractors only):** 2-3/7 violations detected (TLS, plaintext patterns)

**Step 2: Generate Custom Extractors (if needed)**
```
/aphoria-custom-extractor-creator

"Generate extractors for these HTTP client violations:
- redirect_limit exceeds 10
- request_timeout exceeds 30s
- connect_timeout exceeds 10s
- idle_timeout missing
- tls/certificate_validation disabled
- tls/min_version below 1.2
- max_retries unbounded"
```

**Expected:** Skill generates declarative extractors, 7/7 violations detected

---

### Day 4: Remediation (4-5 hours)

**Fix violations one at a time:**
1. Set `max_redirects: 10`
2. Set `request_timeout: 30s`
3. Set `connect_timeout: 10s`
4. Set `idle_timeout: Some(60s)`
5. Enable TLS verification
6. Set TLS minimum version to 1.2
7. Set `max_retries: 3`

**After each fix:**
- Re-scan with incremented version (scan-v2.json, scan-v3.json, ...)
- Verify violation count decreased
- Git commit with context

**Final scan:** 0 conflicts

---

### Day 5: Documentation (3-4 hours)

**Deliverables:**
1. **SUCCESS-STORY.md** - Flywheel demonstration with metrics
2. **DEMO-SCRIPT.md** - How to present to stakeholders
3. **Flywheel metrics:**
   - Time: 1.5 hours vs 4 hours (62.5% reduction)
   - Pattern reuse: 9/22 claims from dbpool (41%)
   - Naming consistency: 0 errors (skills enforced)

---

## Success Criteria

### Minimum Success

- ✅ Day 1 completed in <2 hours (vs 4 hours for Project 1)
- ✅ 8+ claims reused from Project 1
- ✅ 0 naming errors (skills enforce consistency)
- ✅ 7/7 violations detected (with skills-generated extractors)

### Full Success (Demonstrates Flywheel)

- ✅ All of above, plus:
- ✅ Skills generated all custom extractors needed
- ✅ Documentation shows measurable flywheel value:
  - Time savings: 60%+ reduction
  - Pattern reuse: 40%+ claims
  - Consistency: 100% aligned naming
- ✅ Can demo: "Project 2 proved Aphoria compounds knowledge across projects"

---

## Differences from Project 1 (dbpool)

| Aspect | Project 1 (dbpool) | Project 2 (httpclient) |
|--------|-------------------|----------------------|
| **Day 1 Workflow** | Manual CLI (4 hours) | Skills-driven (1-2 hours) |
| **Claim Creation** | Start from scratch (27 new) | Pattern discovery (8-10 reused, 12-14 new) |
| **Naming** | Manual (2-3 errors) | Skills enforce (0 errors) |
| **Extractor Creation** | Manual TOML or skip | Skills generate automatically |
| **Purpose** | Establish baseline | Demonstrate flywheel |
| **Evidence** | Violations detected | Time saved + patterns reused + consistency |

---

## Files to Create

**Required:**
- `plan.md` (this file) - ✅ COMPLETE
- `CHECKLIST.md` - Day-by-day execution (adapt from dbpool)
- `README.md` - Project overview
- `.aphoria/config.toml` - Persistent mode config

**Documentation:**
- `docs/sources/http-rfcs.md` - RFC 7230-7235 excerpts
- `docs/sources/mozilla-http.md` - Mozilla HTTP best practices
- `docs/sources/requests-docs.md` - Requests library patterns

**Implementation (Day 2):**
- `src/*.rs` - HTTP client library with violations
- `tests/basic.rs` - Integration tests
- `Cargo.toml` - Dependencies

**Evidence (Day 5):**
- `SUCCESS-STORY.md` - Flywheel demonstration
- `DEMO-SCRIPT.md` - Presentation guide

---

**Status:** Plan complete, ready for CHECKLIST.md and README.md
**Next:** Create execution checklist with skills-first workflow