stemedb/applications/aphoria-dashboard/INTEGRATION_STATUS.md
jml 65065f3d8f feat(aphoria): implement community corpus with wiki import and pattern aggregation
Implements Phase 4 (A4) - Community corpus as first-class citizens:

- **Community Corpus Builder** - Queries StemeDB pattern aggregates
- **Wiki Import** - Bootstrap corpus from markdown docs (aphoria corpus import wiki)
- **Pattern Aggregation** - Automatic learning from local scans (--sync flag)
- **Storage Layer** - StemeDBPatternStore with content-addressed deduplication
- **Promotion Logic** - Multi-tier thresholds (95%/80%/50% adoption rates)
- **Corpus Build** - Unified registry for RFC/OWASP/Vendor/Community sources
- **Trust Packs** - Export corpus as signed, distributable artifacts
- **Documentation** - bootstrap-corpus.md guide + CLI reference updates

Technical details:
- Pattern aggregates stored as assertions with predicate "pattern_aggregate"
- Content-addressed subjects via BLAKE3(subject:predicate:value)
- PatternAggregator handles write path (observations → patterns)
- StemeDBPatternStore handles read path (pattern queries)
- Integration tests + fixtures in tests/wiki_import_test.rs

Deleted hardcoded.rs (368 lines) - corpus now fully emergent from StemeDB.
Deleted enriched-corpus-patterns.md (677 lines) - feature shipped.

Closes VG-026 (community corpus), part of A4 milestone.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 00:12:31 +00:00

294 lines
8.8 KiB
Markdown

# Aphoria Dashboard Integration Status
> **Date**: 2026-02-08
> **Phase CC Complete**: All community corpus features integrated ✅
---
## ✅ What's Integrated
### 1. **API Endpoint** ✅
**File**: `crates/stemedb-api/src/handlers/aphoria/report.rs:232`
```rust
pub async fn get_patterns(
State(state): State<AppState>,
Query(params): Query<GetPatternsRequest>,
) -> Result<Json<GetPatternsResponse>> {
// ✅ Uses GenericPatternAggregateStore (new infrastructure)
let pattern_store = GenericPatternAggregateStore::new(state.store.clone());
// ✅ Queries pattern aggregates from StemeDB
let aggregates = if let Some(prefix) = &params.subject_prefix {
pattern_store.get_patterns_for_subject_prefix(...)
} else {
pattern_store.get_popular_patterns(...)
};
// ✅ Uses PatternEnricher for query-time enrichment
let enricher = PatternEnricher::from_registry(&registry);
}
```
**Endpoint**: `GET /v1/aphoria/patterns`
**Features**:
- ✅ Queries pattern aggregates from StemeDB (not flat files)
- ✅ Filters by `subject_prefix` (e.g., "tls/cert")
- ✅ Filters by `min_projects` threshold
- ✅ Query-time enrichment (category, verdict, explanation)
- ✅ Authority source matching
---
### 2. **Dashboard UI** ✅
**File**: `applications/aphoria-dashboard/src/app/corpus/page.tsx`
```tsx
export default function CorpusPage() {
return (
<>
<Header title="Community Corpus" />
<CorpusPanel />
</>
);
}
```
**Components**:
-`CorpusPanel` - Main corpus view with filters
-`CorpusFilters` - Subject prefix, min projects, category, hide noise
-`CorpusList` - Pattern cards with enrichment data
-`CorpusRow` - Individual pattern display
- ✅ Empty states, loading skeletons, error handling
**URL**: `http://aphoria.local/corpus` or `http://localhost:3000/corpus`
---
### 3. **API Client** ✅
**File**: `applications/aphoria-dashboard/src/lib/api/client.ts:191`
```typescript
async getPatterns(params: {
subjectPrefix?: string;
minProjects?: number;
limit?: number;
} = {}): Promise<GetPatternsResponse> {
const searchParams = new URLSearchParams();
if (params.subjectPrefix) searchParams.set("subject_prefix", params.subjectPrefix);
if (params.minProjects !== undefined) searchParams.set("min_projects", String(params.minProjects));
if (params.limit !== undefined) searchParams.set("limit", String(params.limit));
return this.fetch<GetPatternsResponse>(`/v1/aphoria/patterns?${searchParams}`);
}
```
**Integration**:
- ✅ Calls `/v1/aphoria/patterns` endpoint
- ✅ Supports filtering by subject prefix, min projects, limit
- ✅ Returns typed `GetPatternsResponse` with pattern DTOs
- ✅ Error handling (404 → empty patterns, other → error state)
---
### 4. **Pattern Enrichment** ✅
The dashboard shows enriched pattern data:
-**Category badge** - Security, Performance, Configuration, etc.
-**Verdict badge** - Best Practice, Noise, Neutral
-**Explanation** - Why this pattern matters
-**Authority source** - RFC, OWASP references
-**Project count** - How many projects use this pattern
-**Observation count** - Total observations across projects
All enrichment happens via the `PatternEnricher` at query time or write time.
---
## 🔄 Data Flow (End-to-End)
```
1. User scans project
└─> aphoria scan --persist --sync
2. Observations recorded
└─> Stored as Tier 4 assertions in StemeDB
3. Pattern aggregation (CC.6)
└─> Observations grouped by (subject, predicate, value)
└─> Pattern aggregates created/updated in StemeDB
└─> Stored as assertions with predicate "pattern_aggregate"
4. Community corpus (CC.7)
└─> CommunityCorpusBuilder queries pattern aggregates
└─> Promotion thresholds evaluated
└─> High-confidence patterns promoted to corpus
5. Dashboard queries patterns
└─> GET /v1/aphoria/patterns
└─> GenericPatternAggregateStore.get_popular_patterns()
└─> Returns pattern aggregates from StemeDB
6. UI displays enriched patterns
└─> CorpusPanel renders patterns with enrichment
└─> Shows category, verdict, explanation, project count
```
---
## ✅ Integration Verification
### Test 1: API Endpoint Works
```bash
# Start API server
cargo run --bin stemedb-api
# Query patterns (should return JSON)
curl http://localhost:18180/v1/aphoria/patterns | jq
```
**Expected**:
```json
{
"patterns": [
{
"subject": "code://*/tls/cert_verification",
"predicate": "enabled",
"value_display": "true",
"project_count": 5,
"observation_count": 12,
"category": "security",
"verdict": "best_practice",
"explanation": "TLS certificate verification prevents MITM attacks"
}
],
"total_matching": 1
}
```
### Test 2: Dashboard Displays Patterns
```bash
# Start dashboard
cd applications/aphoria-dashboard
npm run dev
# Open in browser
open http://localhost:3000/corpus
```
**Expected**:
- Corpus page loads
- Patterns displayed if available
- Filters work (subject prefix, min projects)
- Empty state shown if no patterns
### Test 3: Pattern Aggregation Works
```bash
# Scan a project to create observations
aphoria scan --persist --sync /path/to/project
# Verify pattern aggregates created
RUST_LOG=aphoria=debug aphoria scan --persist . 2>&1 | grep "pattern_aggregate\|patterns created\|patterns updated"
```
**Expected**:
```
✅ Pattern aggregation: 5 patterns created/updated
✅ Patterns stored with predicate "pattern_aggregate"
```
---
## ⚠️ Known Gaps (If Any)
### Gap 1: No Real-Time Updates
**Status**: Not a blocker
The dashboard doesn't auto-refresh when new patterns are added. User must manually refresh the page.
**Solution** (optional): Add polling or WebSocket support for live updates.
### Gap 2: Hosted Mode vs Community Corpus
**Status**: Documented in CORPUS_STATUS.md
Two separate features that were initially conflated:
- **Hosted Mode**: `/v1/aphoria/observations` - team server
- **Community Corpus**: `/v1/aphoria/community/observations` - public patterns
**Current State**: API serves patterns from StemeDB pattern aggregates created during local scans. Hosted mode integration is separate.
---
## ✅ Summary: Integration Complete
| Component | Status | Notes |
|-----------|--------|-------|
| **Pattern Aggregation** | ✅ Complete | CC.6 - observations → aggregates |
| **Community Corpus Builder** | ✅ Complete | CC.7 - async, enabled by default |
| **Pattern Store API** | ✅ Complete | GenericPatternAggregateStore |
| **API Endpoint** | ✅ Complete | GET /v1/aphoria/patterns |
| **Dashboard UI** | ✅ Complete | /corpus page with filters |
| **API Client** | ✅ Complete | getPatterns() method |
| **Pattern Enrichment** | ✅ Complete | Query-time via PatternEnricher |
| **Empty States** | ✅ Complete | Handles zero patterns gracefully |
| **Error Handling** | ✅ Complete | 404 → empty, other → error |
---
## 🎯 What's Working Right Now
1. **Local Scans** → Pattern aggregates stored in StemeDB
2. **API Endpoint** → Queries pattern aggregates from StemeDB
3. **Dashboard** → Displays patterns with enrichment
4. **Filters** → Subject prefix, min projects, category
5. **Empty State** → Shown when no patterns exist
---
## 🚀 Next Steps (Optional Enhancements)
### Enhancement 1: Pattern Promotion UI
Add UI for reviewing and promoting patterns that meet thresholds:
- Show promotion candidates (50+ projects, not yet promoted)
- Manual approve/reject buttons
- Maps to Phase 14 (Governance Workflows)
### Enhancement 2: Pattern Timeline
Show how patterns evolve over time:
- Graph of project_count over time
- When pattern first appeared
- Adoption trend (growing/declining)
### Enhancement 3: Pattern Details Page
Click pattern → see:
- All projects using this pattern
- Code examples
- Related patterns
- Authority sources (RFC sections, OWASP references)
---
## 📋 Files Changed for Integration
| File | Purpose | Status |
|------|---------|--------|
| `crates/stemedb-api/src/handlers/aphoria/report.rs` | API endpoint for patterns | ✅ |
| `applications/aphoria-dashboard/src/app/corpus/page.tsx` | Corpus page route | ✅ |
| `applications/aphoria-dashboard/src/components/corpus/corpus-panel.tsx` | Main corpus component | ✅ |
| `applications/aphoria-dashboard/src/lib/api/client.ts` | API client method | ✅ |
| `applications/aphoria-dashboard/src/lib/api/types.ts` | TypeScript types | ✅ |
---
## ✅ Conclusion
**The Aphoria Dashboard is fully integrated with the new community corpus infrastructure (Phase CC).**
All core features work:
- Pattern aggregation (CC.6) ✅
- Community corpus builder (CC.7) ✅
- API endpoint serving patterns ✅
- Dashboard UI displaying patterns ✅
- Filters and enrichment working ✅
**No integration gaps detected.** The dashboard is ready to display community patterns as soon as they're created through scans.