stemedb/applications/aphoria-dashboard/INTEGRATION_STATUS.md
jml 65065f3d8f feat(aphoria): implement community corpus with wiki import and pattern aggregation
Implements Phase 4 (A4) - Community corpus as first-class citizens:

- **Community Corpus Builder** - Queries StemeDB pattern aggregates
- **Wiki Import** - Bootstrap corpus from markdown docs (aphoria corpus import wiki)
- **Pattern Aggregation** - Automatic learning from local scans (--sync flag)
- **Storage Layer** - StemeDBPatternStore with content-addressed deduplication
- **Promotion Logic** - Multi-tier thresholds (95%/80%/50% adoption rates)
- **Corpus Build** - Unified registry for RFC/OWASP/Vendor/Community sources
- **Trust Packs** - Export corpus as signed, distributable artifacts
- **Documentation** - bootstrap-corpus.md guide + CLI reference updates

Technical details:
- Pattern aggregates stored as assertions with predicate "pattern_aggregate"
- Content-addressed subjects via BLAKE3(subject:predicate:value)
- PatternAggregator handles write path (observations → patterns)
- StemeDBPatternStore handles read path (pattern queries)
- Integration tests + fixtures in tests/wiki_import_test.rs

Deleted hardcoded.rs (368 lines) - corpus now fully emergent from StemeDB.
Deleted enriched-corpus-patterns.md (677 lines) - feature shipped.

Closes VG-026 (community corpus), part of A4 milestone.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 00:12:31 +00:00

8.8 KiB

Aphoria Dashboard Integration Status

Date: 2026-02-08 Phase CC Complete: All community corpus features integrated


What's Integrated

1. API Endpoint

File: crates/stemedb-api/src/handlers/aphoria/report.rs:232

pub async fn get_patterns(
    State(state): State<AppState>,
    Query(params): Query<GetPatternsRequest>,
) -> Result<Json<GetPatternsResponse>> {
    // ✅ Uses GenericPatternAggregateStore (new infrastructure)
    let pattern_store = GenericPatternAggregateStore::new(state.store.clone());

    // ✅ Queries pattern aggregates from StemeDB
    let aggregates = if let Some(prefix) = &params.subject_prefix {
        pattern_store.get_patterns_for_subject_prefix(...)
    } else {
        pattern_store.get_popular_patterns(...)
    };

    // ✅ Uses PatternEnricher for query-time enrichment
    let enricher = PatternEnricher::from_registry(&registry);
}

Endpoint: GET /v1/aphoria/patterns

Features:

  • Queries pattern aggregates from StemeDB (not flat files)
  • Filters by subject_prefix (e.g., "tls/cert")
  • Filters by min_projects threshold
  • Query-time enrichment (category, verdict, explanation)
  • Authority source matching

2. Dashboard UI

File: applications/aphoria-dashboard/src/app/corpus/page.tsx

export default function CorpusPage() {
  return (
    <>
      <Header title="Community Corpus" />
      <CorpusPanel />
    </>
  );
}

Components:

  • CorpusPanel - Main corpus view with filters
  • CorpusFilters - Subject prefix, min projects, category, hide noise
  • CorpusList - Pattern cards with enrichment data
  • CorpusRow - Individual pattern display
  • Empty states, loading skeletons, error handling

URL: http://aphoria.local/corpus or http://localhost:3000/corpus


3. API Client

File: applications/aphoria-dashboard/src/lib/api/client.ts:191

async getPatterns(params: {
  subjectPrefix?: string;
  minProjects?: number;
  limit?: number;
} = {}): Promise<GetPatternsResponse> {
  const searchParams = new URLSearchParams();
  if (params.subjectPrefix) searchParams.set("subject_prefix", params.subjectPrefix);
  if (params.minProjects !== undefined) searchParams.set("min_projects", String(params.minProjects));
  if (params.limit !== undefined) searchParams.set("limit", String(params.limit));
  return this.fetch<GetPatternsResponse>(`/v1/aphoria/patterns?${searchParams}`);
}

Integration:

  • Calls /v1/aphoria/patterns endpoint
  • Supports filtering by subject prefix, min projects, limit
  • Returns typed GetPatternsResponse with pattern DTOs
  • Error handling (404 → empty patterns, other → error state)

4. Pattern Enrichment

The dashboard shows enriched pattern data:

  • Category badge - Security, Performance, Configuration, etc.
  • Verdict badge - Best Practice, Noise, Neutral
  • Explanation - Why this pattern matters
  • Authority source - RFC, OWASP references
  • Project count - How many projects use this pattern
  • Observation count - Total observations across projects

All enrichment happens via the PatternEnricher at query time or write time.


🔄 Data Flow (End-to-End)

1. User scans project
   └─> aphoria scan --persist --sync

2. Observations recorded
   └─> Stored as Tier 4 assertions in StemeDB

3. Pattern aggregation (CC.6)
   └─> Observations grouped by (subject, predicate, value)
   └─> Pattern aggregates created/updated in StemeDB
   └─> Stored as assertions with predicate "pattern_aggregate"

4. Community corpus (CC.7)
   └─> CommunityCorpusBuilder queries pattern aggregates
   └─> Promotion thresholds evaluated
   └─> High-confidence patterns promoted to corpus

5. Dashboard queries patterns
   └─> GET /v1/aphoria/patterns
   └─> GenericPatternAggregateStore.get_popular_patterns()
   └─> Returns pattern aggregates from StemeDB

6. UI displays enriched patterns
   └─> CorpusPanel renders patterns with enrichment
   └─> Shows category, verdict, explanation, project count

Integration Verification

Test 1: API Endpoint Works

# Start API server
cargo run --bin stemedb-api

# Query patterns (should return JSON)
curl http://localhost:18180/v1/aphoria/patterns | jq

Expected:

{
  "patterns": [
    {
      "subject": "code://*/tls/cert_verification",
      "predicate": "enabled",
      "value_display": "true",
      "project_count": 5,
      "observation_count": 12,
      "category": "security",
      "verdict": "best_practice",
      "explanation": "TLS certificate verification prevents MITM attacks"
    }
  ],
  "total_matching": 1
}

Test 2: Dashboard Displays Patterns

# Start dashboard
cd applications/aphoria-dashboard
npm run dev

# Open in browser
open http://localhost:3000/corpus

Expected:

  • Corpus page loads
  • Patterns displayed if available
  • Filters work (subject prefix, min projects)
  • Empty state shown if no patterns

Test 3: Pattern Aggregation Works

# Scan a project to create observations
aphoria scan --persist --sync /path/to/project

# Verify pattern aggregates created
RUST_LOG=aphoria=debug aphoria scan --persist . 2>&1 | grep "pattern_aggregate\|patterns created\|patterns updated"

Expected:

✅ Pattern aggregation: 5 patterns created/updated
✅ Patterns stored with predicate "pattern_aggregate"

⚠️ Known Gaps (If Any)

Gap 1: No Real-Time Updates

Status: Not a blocker

The dashboard doesn't auto-refresh when new patterns are added. User must manually refresh the page.

Solution (optional): Add polling or WebSocket support for live updates.

Gap 2: Hosted Mode vs Community Corpus

Status: Documented in CORPUS_STATUS.md

Two separate features that were initially conflated:

  • Hosted Mode: /v1/aphoria/observations - team server
  • Community Corpus: /v1/aphoria/community/observations - public patterns

Current State: API serves patterns from StemeDB pattern aggregates created during local scans. Hosted mode integration is separate.


Summary: Integration Complete

Component Status Notes
Pattern Aggregation Complete CC.6 - observations → aggregates
Community Corpus Builder Complete CC.7 - async, enabled by default
Pattern Store API Complete GenericPatternAggregateStore
API Endpoint Complete GET /v1/aphoria/patterns
Dashboard UI Complete /corpus page with filters
API Client Complete getPatterns() method
Pattern Enrichment Complete Query-time via PatternEnricher
Empty States Complete Handles zero patterns gracefully
Error Handling Complete 404 → empty, other → error

🎯 What's Working Right Now

  1. Local Scans → Pattern aggregates stored in StemeDB
  2. API Endpoint → Queries pattern aggregates from StemeDB
  3. Dashboard → Displays patterns with enrichment
  4. Filters → Subject prefix, min projects, category
  5. Empty State → Shown when no patterns exist

🚀 Next Steps (Optional Enhancements)

Enhancement 1: Pattern Promotion UI

Add UI for reviewing and promoting patterns that meet thresholds:

  • Show promotion candidates (50+ projects, not yet promoted)
  • Manual approve/reject buttons
  • Maps to Phase 14 (Governance Workflows)

Enhancement 2: Pattern Timeline

Show how patterns evolve over time:

  • Graph of project_count over time
  • When pattern first appeared
  • Adoption trend (growing/declining)

Enhancement 3: Pattern Details Page

Click pattern → see:

  • All projects using this pattern
  • Code examples
  • Related patterns
  • Authority sources (RFC sections, OWASP references)

📋 Files Changed for Integration

File Purpose Status
crates/stemedb-api/src/handlers/aphoria/report.rs API endpoint for patterns
applications/aphoria-dashboard/src/app/corpus/page.tsx Corpus page route
applications/aphoria-dashboard/src/components/corpus/corpus-panel.tsx Main corpus component
applications/aphoria-dashboard/src/lib/api/client.ts API client method
applications/aphoria-dashboard/src/lib/api/types.ts TypeScript types

Conclusion

The Aphoria Dashboard is fully integrated with the new community corpus infrastructure (Phase CC).

All core features work:

  • Pattern aggregation (CC.6)
  • Community corpus builder (CC.7)
  • API endpoint serving patterns
  • Dashboard UI displaying patterns
  • Filters and enrichment working

No integration gaps detected. The dashboard is ready to display community patterns as soon as they're created through scans.