stemedb/applications/aphoria/roadmap-archive.md

Aphoria Roadmap Archive

Completed phases moved from roadmap.md. Full implementation details preserved in git history.

---

Phase 0: StemeDB Foundation ✅

ConceptPath type, hierarchical index, alias store, source class inference, concept API endpoints. All shipped as Phase 5D of the main StemeDB roadmap.

Spec: docs/specs/concept-hierarchy.md

---

Phase 2: CLI Core ✅

End-to-end CLI pipeline with 10 extractors and bootstrapped corpus of 11 hardcoded assertions.

Task	Status
2.1 Project Walker	✅ walker/mod.rs, walker/path_mapper.rs, walker/language.rs
2.2 Extractors (10)	✅ tls_verify, jwt_config, hardcoded_secrets, timeout_config, dep_versions, cors_config, rate_limit, weak_crypto, command_injection, sql_injection
2.3 Ingestion Bridge	✅ bridge.rs — BLAKE3 hashing, Ed25519 signing, claim→assertion conversion
2.4 Conflict Query	✅ episteme.rs — LocalEpisteme with check_conflicts()
2.5 Report Output	✅ report/ — table (comfy-table), JSON, SARIF 2.1.0, markdown
2.6 Acknowledge Command	✅ lib.rs acknowledge()
Baseline & Diff	✅ lib.rs set_baseline(), show_diff()
Status Command	✅ lib.rs show_status()

Phase 2 Code Quality Fixes ✅

• DES/RC4 concept path misclassification: Split into check_hash_pattern() and check_encryption_pattern()
• SHA1 edge case: Documented as intentionally broad
• JS exec() regex: Tightened to require child_process. prefix

---

Phase 2A: Concept Matching ✅

• 2A.1 Leaf-Based Matching: ConceptIndex with tail-path matching (last 2 segments + predicate)
• 2A.2 Alias Resolution: Wired AliasStore into QueryEngine.execute() with resolve_aliases: bool
• 2A.3 Auto-Alias Creation: Auto-creates aliases when code and authority share leaf names

---

Phase 1: Authoritative Corpus Expansion ✅

Expanded from 11 hardcoded assertions to pluggable corpus system.

• 1.1 CorpusBuilder Trait ✅ — name, scheme, default_tier, build, requires_network
• 1.2 RFC Ingester ✅ — HTTP fetching, RFC 2119 keyword parsing, 8 RFC-specific parsers
• 1.3 OWASP Ingester ✅ — GitHub raw content, 9 cheat sheet parsers
• 1.4 Vendor Docs ✅ — PostgreSQL, Redis, reqwest, hyper, Go net/http, tokio-postgres, SQLx
• 1.5 Hardcoded Refactor ✅ — Original 11 assertions as HardcodedCorpusBuilder
• 1.6 CLI Integration ✅ — aphoria corpus build/list, --only, --offline, --clear-cache
• 1.7 Error Handling ✅ — Per-source graceful degradation

Files: corpus/mod.rs, corpus/hardcoded.rs, corpus/rfc.rs, corpus/owasp.rs, corpus/vendor.rs

---

Phase 3: Skill Integration ✅

• 3.1 Claude Code Skill ✅ — /aphoria scan, scan --fix, ack, status, diff, init, baseline
• 3.2 Agent Pre-Flight Hook ✅ — --exit-code (2=BLOCK, 1=FLAG, 0=clean), --strict
• 3.3 Alias Suggestion ✅ — Auto-alias creation from Phase 2A.3

---

Phase 4: Full-Cycle Pre-Commit (Scan + Sync) ✅

Bidirectional knowledge sync: extract → check → classify → update → gate.

• 4A Observational Claims ✅ — --sync records novel claims as Tier 4 observations
• 4B Self-Conflict Detection ✅ — Drift detection with Verdict::Drift
• 4C Diff-Only Scanning ✅ — --staged for fast pre-commit hooks
• 4D Enhanced Ack ✅ — --reason, aphoria update for policy changes
• 4E Hosted Mode ✅ — Team aggregation via central StemeDB server, HostedClient

---

Phase 4.5: Ephemeral Scan Mode ✅

40x faster scans by skipping Episteme storage. Default mode ~0.25s, persistent ~1-2s.

• ScanMode enum (Ephemeral default, Persistent opt-in with --persist)
• EphemeralDetector — in-memory corpus + ConceptIndex
• check_conflicts_pure() extracted as standalone function

---

Phase 5: Research Agent Loop ✅

• 5.1 Gap Detection ✅ — detect_gaps() compares claims against ConceptIndex
• 5.2 Gap Storage ✅ — JSON-backed persistent storage with eligibility tracking
• 5.3 Quality Validation ✅ — Source attribution, normative language, vague content detection
• 5.4 Research Execution ✅ — HTTP fetching, normative extraction, confidence scoring
• 5.5 CLI Integration ✅ — aphoria research run/status/gaps
• 5.6 Community Corpus ✅ — Opt-in anonymous pattern sharing with privacy-preserving anonymization
• 5.7 Security Extractors ✅ — weak_crypto, command_injection, sql_injection

---

Phase 6: Federated Policy & Trust Packs ✅

• 6.1 Trust Pack Format ✅ — rkyv serialization, Ed25519 signing
• 6.2 Policy Management ✅ — Local and remote loading with caching
• 6.3 Core Integration ✅ — EphemeralDetector + LocalEpisteme policy ingestion
• 6.4 CLI Commands ✅ — aphoria policy export, auto-loading

---

Phase 6.5: Trust Pack Extensions ✅

• 6.5.1 Predicate Aliases ✅ — enabled ↔ required ↔ mandatory ↔ enforced
• 6.5.2 Pack Signing Key Rotation ✅ — aphoria policy resign, signature chain audit trail

---

Phase 7: Declarative Extractors ✅

TOML-defined custom extractors without Rust code.

• 7.1 Core Types ✅ — DeclarativeExtractorDef, DeclarativeExtractor
• 7.2 Configuration ✅ — [[extractors.declarative]] in aphoria.toml
• 7.3 Validation ✅ — ReDoS protection, confidence validation
• 7.4 Registry Integration ✅ — Enable/disable, Trust Pack integration
• 7.5 Error Handling ✅
• 7.6 Tests ✅ — 22 unit + 7 integration tests

---

Phase 7.5: LLM-in-the-Loop Extraction ✅

Gemini-powered semantic extraction for high-value files.

• 7.5.1 LLM Extractor ✅ — GeminiClient, structured JSON output
• 7.5.2 Selective Triggering ✅ — is_high_value_file(), token budget
• 7.5.3 Cost Controls ✅ — BLAKE3 caching, budget enforcement
• 7.5.4 Configuration ✅ — [llm] section in aphoria.toml

---

Phase 7.6: Pattern Learning Store ✅

Remember patterns LLM finds for promotion to declarative extractors.

• 7.6.1 Schema ✅ — LearnedPattern, ClaimTemplate, ValueType
• 7.6.2 PatternStore ✅ — JSON-backed, RwLock thread safety, Levenshtein dedup
• 7.6.3 Normalization ✅ — Version/boolean/number/string placeholder replacement
• 7.6.4 Configuration ✅ — [learning] section
• 7.6.5 Scan Integration ✅ — Project hash, record/update patterns

---

Phase 7.7: Pattern → Extractor Promotion ✅

Learned patterns become declarative extractors via LLM regex generation.

• 7.7.1 Pipeline ✅ — PromotionPipeline, RegexGenerator, ExtractorValidator, YamlWriter
• 7.7.2 Regex Generation ✅ — Multi-example prompt, ReDoS safety
• 7.7.3 Validation ✅ — Positive tests, timing validation
• 7.7.4 Human Review ✅ — aphoria extractors review/stats/candidates/promote
• 7.7.5 Extractor Output ✅ — YAML files in .aphoria/extractors/learned/

---

Phase 7.8: LLM Prompt Evaluation ✅

Golden fixtures with precision/recall metrics and regression detection.

• 7.8.1 Fixture Format ✅ — TOML-based with must_contain/must_not_contain
• 7.8.2 Claim Matching ✅ — Tail-path matching, type coercion
• 7.8.3 Metrics ✅ — Precision/Recall/F1, per-category breakdown
• 7.8.4 Harness ✅ — Live/Cached/Mock modes, regression detection
• 7.8.5 Reports ✅ — Table, JSON, Markdown
• 7.8.6 CLI ✅ — aphoria eval run/baseline/update-baseline/list-fixtures/validate-fixtures
• 7.8.7 Seed Fixtures ✅ — 10 fixtures across tls, jwt, secrets, auth, negative, edge

---

Phase 8: Enterprise Extractor Improvements ✅

42 extractors total. Enterprise-grade detection for production codebases.

• 8.1 High-Entropy Secrets ✅ — Shannon entropy, known prefixes (AWS/Stripe/GitHub/GitLab/Slack)
• 8.2 Framework Extractors (10) ✅ — Spring, Django, Express, Rails, ASP.NET, Laravel, FastAPI, Next.js, Flask, NestJS
• 8.3 Config Deep Parsing ✅ — YAML/JSON/TOML tree walking, 11 security rules
• 8.4 Semantic TLS Version ✅ — Cross-language const detection, Terraform, Kubernetes
• 8.5 ORM SQL Injection ✅ — Django/SQLAlchemy/GORM/ActiveRecord/Prisma/Sequelize
• 8.6 Path Traversal ✅
• 8.7 Unvalidated Redirects ✅
• 8.8 Weak Password ✅
• 8.9 Security Headers ✅
• 8.10 Insecure Deserialization ✅
• 8.11 SSRF ✅

---

Phase 9: Autonomous Extractor Generation ✅

Fully self-improving extraction system.

• 9.1 Autonomous Promotion ✅ — >0.95 confidence, >10 projects, full audit trail
• 9.2 Shadow Mode Testing ✅ — Isolated metrics, graduation gate, FP tracking
• 9.3 Auto-Rollback ✅ — FP rate >15% triggers automatic rollback
• 9.4 Cross-Project Learning ✅ — Privacy-preserving pattern sync, community extractors
• 9.5 Extractor Versioning ✅ — Changelogs, rollback, A/B comparison

---

Phase 10.1: Acknowledgment Expiry ✅

Time-limited exceptions with --expires flag.

• --expires 90d or --expires 2026-12-31 (ISO 8601)
• Expired acks resurface as BLOCK
• Preserved for audit trail per patent claim 25
• All report formatters show expiry info

Files: src/expiry.rs, cli.rs, report/*.rs

---

Phase 11: Evidence-Based Authority ✅

Evidence levels (ProductSpec > Standard > Research > Commit-only) with evidence-aware graduation.

• 11.1 Types ✅ — EvidenceLevel, PatternEvidence with ADR/spec/RFC references
• 11.2 Detection ✅ — Commit message parsing, ADR/spec file detection
• 11.3 Graduation ✅ — Thresholds vary by evidence (ProductSpec: 1 usage, Commit-only: 10)
• 11.4 Display ✅ — Evidence chain in output, --evidence filter

Files: src/evidence/mod.rs, evidence/types.rs, evidence/detection.rs

---

Phase 12: Knowledge Scope Hierarchy ✅

Organization → Team → Project scope levels with inheritance.

• 12.1 Scope Types ✅ — ScopeLevel enum, ScopeConfig
• 12.2 Inheritance ✅ — Security: no opt-out, Conventions: override with justification
• 12.3 Override Workflow ✅ — Justification + evidence required
• 12.4 Cross-Scope Queries ✅ — --scope org/team/project, --exclude-inherited

Files: src/scope/mod.rs, scope/config.rs, scope/resolver.rs, scope/override_record.rs, scope/store.rs

---

Phase 13: Knowledge Lifecycle Management ✅

Active → Deprecated → Superseded → Archived lifecycle for patterns.

• 13.1 Status Types ✅ — KnowledgeStatus enum with history tracking
• 13.2 Deprecation ✅ — aphoria deprecate with --reason, --superseded-by, --sunset-date
• 13.3 Migration Guidance ✅ — Warnings in scan output, links to replacements
• 13.4 Migration Dashboard ✅ — aphoria migrations status, progress tracking, export

Files: src/lifecycle/mod.rs, lifecycle/store.rs, lifecycle/migration.rs

---

Phase 16: Ignore & Exclusion System ✅

Clean scans by excluding test fixtures and intentional patterns.

• 16.1 Glob Patterns ✅ — globset with **, *, ? support
• 16.2 .aphoriaignore ✅ — Gitignore-style patterns, merged with aphoria.toml
• 16.3 Inline Comments ✅ — // aphoria:ignore, ignore-next-line, ignore-block
• 16.4 Ack Export/Import ✅ — .aphoria/acks.toml, version-controllable

---

Phase 17: Pattern Enrichment & Best Practices Infrastructure ✅

Backend infrastructure for enriched corpus patterns and team guideline ingestion.

Note: Backend only — UI integration not implemented. Patterns have metadata but dashboard doesn't display it yet.

17.1 Enriched Pattern Metadata ✅

What: Transform bare patterns like "md5: true" into actionable insights "MD5 is deprecated (NIST 2010)".

Task	Status
Add enrichment fields to PatternAggregate (category, verdict, explanation, authority_source)	✅
Add pattern_metadata() method to Extractor trait	✅
Create PatternEnricher service with exact/wildcard matching + noise detection	✅ corpus/enricher.rs
Implement pattern_metadata() for 10 security extractors	✅ See below

Enriched Extractors:

• WeakCryptoExtractor — MD5, SHA1, DES, RC4 deprecated
• TlsVersionExtractor — TLS 1.0/1.1 deprecated, 1.2/1.3 recommended
• TlsVerifyExtractor — cert_verification: false insecure
• JwtConfigExtractor — algorithm: none forbidden
• CorsConfigExtractor — allow_origin: * insecure
• HardcodedSecretsExtractor — API keys/passwords critical
• SqlInjectionExtractor — string interpolation vulnerable
• CommandInjectionExtractor — shell exec vulnerable
• PathTraversalExtractor — user-controlled paths vulnerable
• InsecureDeserializationExtractor — pickle/yaml.load unsafe

17.2 TeamPolicy Authority Tier ✅

What: New tier 2.5 between Observational and Expert for team-level architectural guidelines.

Task	Status
Add TeamPolicy variant to SourceClass enum	✅ stemedb-core/src/types/source.rs
Add tier_fractional() for 2.5 representation	✅
Update authority_weight() (0.6) and default_decay_days() (180)	✅
Add "team_policy" parsing to parse_authority_tier()	✅ aphoria/src/types/authored_claim.rs
Update all DTO conversions (API, ontology)	✅

17.3 Best Practices Import CLI ✅

What: Batch import claims from TOML files (e.g., hexagonal architecture guidelines).

Task	Status
Add Import subcommand to ClaimsCommands	✅ cli/claims.rs
Implement handle_claims_import() with merge strategies	✅ handlers/claims.rs
Support --authority-tier override	✅
Support --source-guide for tracking	✅
Support --dry-run for preview	✅
Merge strategies: skip_existing, overwrite, fail_on_duplicate	✅

Usage:

aphoria claims import docs/guidelines.toml \
  --authority-tier team_policy \
  --source-guide "hexagonal-arch" \
  --dry-run

17.4 Guideline Tracking System ✅

What: Track which guidelines have been imported for compliance filtering and change detection.

Task	Status
Create GuidelineMetadata struct	✅ types/ingested_guides.rs
Create IngestedGuidesFile with TOML persistence	✅
Track: id, name, source_path, document_hash, claim_ids	✅
Integrate with import command	✅
Store in .aphoria/ingested_guides.toml	✅

17.5 Updated Comparison Modes ✅

Task	Status
Add Contains comparison mode	✅ types/authored_claim.rs
Add NotContains comparison mode	✅
Update API DTOs	✅ stemedb-api/src/dto/aphoria/types.rs

What's NOT Implemented

❌ Dashboard UI Integration — Enrichment metadata exists in backend but no UI to display it ❌ Category Badges — No visual badges for security/architecture/performance ❌ Verdict Badges — No visual indicators for deprecated/recommended ❌ Filtering UI — No dropdown to filter patterns by category ❌ "Hide Noise" Toggle — Noise detection works but no UI control ❌ --check-policy Flag — Backend ready but scan filtering not implemented

Files Modified

Core:

• crates/stemedb-core/src/types/source.rs — TeamPolicy tier
• crates/stemedb-storage/src/pattern_aggregate_store/mod.rs — Enrichment fields

Aphoria:

• applications/aphoria/src/extractors/traits.rs — pattern_metadata() method
• applications/aphoria/src/corpus/enricher.rs — NEW Pattern enrichment service
• applications/aphoria/src/types/authored_claim.rs — Contains/NotContains modes
• applications/aphoria/src/types/ingested_guides.rs — NEW Guideline tracking
• applications/aphoria/src/cli/claims.rs — Import subcommand
• applications/aphoria/src/handlers/claims.rs — Import handler
• 10 extractor files with pattern_metadata() implementations

API:

• crates/stemedb-api/src/dto/enums.rs — TeamPolicy DTO
• crates/stemedb-api/src/dto/aphoria/types.rs — Contains/NotContains DTOs
• crates/stemedb-api/src/handlers/aphoria/claims.rs — Comparison mode conversion
• crates/stemedb-api/src/handlers/layered.rs — SourceClass conversion

Ontology:

• crates/stemedb-ontology/src/dto/enums.rs — TeamPolicy DTO
• crates/stemedb-ontology/src/dto/conversions.rs — Conversion functions

---

The Self-Learning Vision (Complete)

Phase 7: Declarative Extractors                          ✅
    ↓
Phase 7.5: LLM-in-the-Loop (Gemini semantic extraction) ✅
    ↓
Phase 7.6: Pattern Learning (remember what LLM finds)    ✅
    ↓
Phase 7.7: Pattern Promotion (patterns → extractors)     ✅
    ↓
Phase 7.8: LLM Prompt Evaluation (measure & improve)     ✅
    ↓
Phase 8: Enterprise Extractors (42 total)                ✅
    ↓
Phase 9: Autonomous Generation (fully self-improving)     ✅

Milestone Summary (Completed)

Phase	Deliverable	Status
0	ConceptPath in StemeDB	✅
2	Aphoria CLI (scan, report, ack)	✅
2A	Concept matching (leaf, alias, auto-alias)	✅
1	Authoritative corpus expansion	✅
3	Claude Code skill + hooks	✅
4	Full-cycle pre-commit (sync, drift, staged, hosted)	✅
4.5	Ephemeral scan mode (40x faster)	✅
5	Research agent loop + community corpus	✅
6	Federated Policy & Trust Packs	✅
6.5	Trust Pack Extensions	✅
7	Declarative Extractors	✅
7.5	LLM-in-the-Loop Extraction	✅
7.6	Pattern Learning Store	✅
7.7	Pattern → Extractor Promotion	✅
7.8	LLM Prompt Evaluation	✅
8	Enterprise Extractors (42 total)	✅
9	Autonomous Extractor Generation	✅
10.1	Acknowledgment Expiry	✅
11	Evidence-Based Authority	✅
12	Knowledge Scope Hierarchy	✅
13	Knowledge Lifecycle Management	✅
16	Ignore & Exclusion System	✅
17	Pattern Enrichment & Best Practices Infrastructure	✅ (backend only)