Implements Phase 4 (A4) - Community corpus as first-class citizens: - **Community Corpus Builder** - Queries StemeDB pattern aggregates - **Wiki Import** - Bootstrap corpus from markdown docs (aphoria corpus import wiki) - **Pattern Aggregation** - Automatic learning from local scans (--sync flag) - **Storage Layer** - StemeDBPatternStore with content-addressed deduplication - **Promotion Logic** - Multi-tier thresholds (95%/80%/50% adoption rates) - **Corpus Build** - Unified registry for RFC/OWASP/Vendor/Community sources - **Trust Packs** - Export corpus as signed, distributable artifacts - **Documentation** - bootstrap-corpus.md guide + CLI reference updates Technical details: - Pattern aggregates stored as assertions with predicate "pattern_aggregate" - Content-addressed subjects via BLAKE3(subject:predicate:value) - PatternAggregator handles write path (observations → patterns) - StemeDBPatternStore handles read path (pattern queries) - Integration tests + fixtures in tests/wiki_import_test.rs Deleted hardcoded.rs (368 lines) - corpus now fully emergent from StemeDB. Deleted enriched-corpus-patterns.md (677 lines) - feature shipped. Closes VG-026 (community corpus), part of A4 milestone. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
294 lines
8.8 KiB
Markdown
294 lines
8.8 KiB
Markdown
# Aphoria Dashboard Integration Status
|
|
> **Date**: 2026-02-08
|
|
> **Phase CC Complete**: All community corpus features integrated ✅
|
|
|
|
---
|
|
|
|
## ✅ What's Integrated
|
|
|
|
### 1. **API Endpoint** ✅
|
|
**File**: `crates/stemedb-api/src/handlers/aphoria/report.rs:232`
|
|
|
|
```rust
|
|
pub async fn get_patterns(
|
|
State(state): State<AppState>,
|
|
Query(params): Query<GetPatternsRequest>,
|
|
) -> Result<Json<GetPatternsResponse>> {
|
|
// ✅ Uses GenericPatternAggregateStore (new infrastructure)
|
|
let pattern_store = GenericPatternAggregateStore::new(state.store.clone());
|
|
|
|
// ✅ Queries pattern aggregates from StemeDB
|
|
let aggregates = if let Some(prefix) = ¶ms.subject_prefix {
|
|
pattern_store.get_patterns_for_subject_prefix(...)
|
|
} else {
|
|
pattern_store.get_popular_patterns(...)
|
|
};
|
|
|
|
// ✅ Uses PatternEnricher for query-time enrichment
|
|
let enricher = PatternEnricher::from_registry(®istry);
|
|
}
|
|
```
|
|
|
|
**Endpoint**: `GET /v1/aphoria/patterns`
|
|
|
|
**Features**:
|
|
- ✅ Queries pattern aggregates from StemeDB (not flat files)
|
|
- ✅ Filters by `subject_prefix` (e.g., "tls/cert")
|
|
- ✅ Filters by `min_projects` threshold
|
|
- ✅ Query-time enrichment (category, verdict, explanation)
|
|
- ✅ Authority source matching
|
|
|
|
---
|
|
|
|
### 2. **Dashboard UI** ✅
|
|
**File**: `applications/aphoria-dashboard/src/app/corpus/page.tsx`
|
|
|
|
```tsx
|
|
export default function CorpusPage() {
|
|
return (
|
|
<>
|
|
<Header title="Community Corpus" />
|
|
<CorpusPanel />
|
|
</>
|
|
);
|
|
}
|
|
```
|
|
|
|
**Components**:
|
|
- ✅ `CorpusPanel` - Main corpus view with filters
|
|
- ✅ `CorpusFilters` - Subject prefix, min projects, category, hide noise
|
|
- ✅ `CorpusList` - Pattern cards with enrichment data
|
|
- ✅ `CorpusRow` - Individual pattern display
|
|
- ✅ Empty states, loading skeletons, error handling
|
|
|
|
**URL**: `http://aphoria.local/corpus` or `http://localhost:3000/corpus`
|
|
|
|
---
|
|
|
|
### 3. **API Client** ✅
|
|
**File**: `applications/aphoria-dashboard/src/lib/api/client.ts:191`
|
|
|
|
```typescript
|
|
async getPatterns(params: {
|
|
subjectPrefix?: string;
|
|
minProjects?: number;
|
|
limit?: number;
|
|
} = {}): Promise<GetPatternsResponse> {
|
|
const searchParams = new URLSearchParams();
|
|
if (params.subjectPrefix) searchParams.set("subject_prefix", params.subjectPrefix);
|
|
if (params.minProjects !== undefined) searchParams.set("min_projects", String(params.minProjects));
|
|
if (params.limit !== undefined) searchParams.set("limit", String(params.limit));
|
|
return this.fetch<GetPatternsResponse>(`/v1/aphoria/patterns?${searchParams}`);
|
|
}
|
|
```
|
|
|
|
**Integration**:
|
|
- ✅ Calls `/v1/aphoria/patterns` endpoint
|
|
- ✅ Supports filtering by subject prefix, min projects, limit
|
|
- ✅ Returns typed `GetPatternsResponse` with pattern DTOs
|
|
- ✅ Error handling (404 → empty patterns, other → error state)
|
|
|
|
---
|
|
|
|
### 4. **Pattern Enrichment** ✅
|
|
|
|
The dashboard shows enriched pattern data:
|
|
- ✅ **Category badge** - Security, Performance, Configuration, etc.
|
|
- ✅ **Verdict badge** - Best Practice, Noise, Neutral
|
|
- ✅ **Explanation** - Why this pattern matters
|
|
- ✅ **Authority source** - RFC, OWASP references
|
|
- ✅ **Project count** - How many projects use this pattern
|
|
- ✅ **Observation count** - Total observations across projects
|
|
|
|
All enrichment happens via the `PatternEnricher` at query time or write time.
|
|
|
|
---
|
|
|
|
## 🔄 Data Flow (End-to-End)
|
|
|
|
```
|
|
1. User scans project
|
|
└─> aphoria scan --persist --sync
|
|
|
|
2. Observations recorded
|
|
└─> Stored as Tier 4 assertions in StemeDB
|
|
|
|
3. Pattern aggregation (CC.6)
|
|
└─> Observations grouped by (subject, predicate, value)
|
|
└─> Pattern aggregates created/updated in StemeDB
|
|
└─> Stored as assertions with predicate "pattern_aggregate"
|
|
|
|
4. Community corpus (CC.7)
|
|
└─> CommunityCorpusBuilder queries pattern aggregates
|
|
└─> Promotion thresholds evaluated
|
|
└─> High-confidence patterns promoted to corpus
|
|
|
|
5. Dashboard queries patterns
|
|
└─> GET /v1/aphoria/patterns
|
|
└─> GenericPatternAggregateStore.get_popular_patterns()
|
|
└─> Returns pattern aggregates from StemeDB
|
|
|
|
6. UI displays enriched patterns
|
|
└─> CorpusPanel renders patterns with enrichment
|
|
└─> Shows category, verdict, explanation, project count
|
|
```
|
|
|
|
---
|
|
|
|
## ✅ Integration Verification
|
|
|
|
### Test 1: API Endpoint Works
|
|
```bash
|
|
# Start API server
|
|
cargo run --bin stemedb-api
|
|
|
|
# Query patterns (should return JSON)
|
|
curl http://localhost:18180/v1/aphoria/patterns | jq
|
|
```
|
|
|
|
**Expected**:
|
|
```json
|
|
{
|
|
"patterns": [
|
|
{
|
|
"subject": "code://*/tls/cert_verification",
|
|
"predicate": "enabled",
|
|
"value_display": "true",
|
|
"project_count": 5,
|
|
"observation_count": 12,
|
|
"category": "security",
|
|
"verdict": "best_practice",
|
|
"explanation": "TLS certificate verification prevents MITM attacks"
|
|
}
|
|
],
|
|
"total_matching": 1
|
|
}
|
|
```
|
|
|
|
### Test 2: Dashboard Displays Patterns
|
|
```bash
|
|
# Start dashboard
|
|
cd applications/aphoria-dashboard
|
|
npm run dev
|
|
|
|
# Open in browser
|
|
open http://localhost:3000/corpus
|
|
```
|
|
|
|
**Expected**:
|
|
- Corpus page loads
|
|
- Patterns displayed if available
|
|
- Filters work (subject prefix, min projects)
|
|
- Empty state shown if no patterns
|
|
|
|
### Test 3: Pattern Aggregation Works
|
|
```bash
|
|
# Scan a project to create observations
|
|
aphoria scan --persist --sync /path/to/project
|
|
|
|
# Verify pattern aggregates created
|
|
RUST_LOG=aphoria=debug aphoria scan --persist . 2>&1 | grep "pattern_aggregate\|patterns created\|patterns updated"
|
|
```
|
|
|
|
**Expected**:
|
|
```
|
|
✅ Pattern aggregation: 5 patterns created/updated
|
|
✅ Patterns stored with predicate "pattern_aggregate"
|
|
```
|
|
|
|
---
|
|
|
|
## ⚠️ Known Gaps (If Any)
|
|
|
|
### Gap 1: No Real-Time Updates
|
|
**Status**: Not a blocker
|
|
|
|
The dashboard doesn't auto-refresh when new patterns are added. User must manually refresh the page.
|
|
|
|
**Solution** (optional): Add polling or WebSocket support for live updates.
|
|
|
|
### Gap 2: Hosted Mode vs Community Corpus
|
|
**Status**: Documented in CORPUS_STATUS.md
|
|
|
|
Two separate features that were initially conflated:
|
|
- **Hosted Mode**: `/v1/aphoria/observations` - team server
|
|
- **Community Corpus**: `/v1/aphoria/community/observations` - public patterns
|
|
|
|
**Current State**: API serves patterns from StemeDB pattern aggregates created during local scans. Hosted mode integration is separate.
|
|
|
|
---
|
|
|
|
## ✅ Summary: Integration Complete
|
|
|
|
| Component | Status | Notes |
|
|
|-----------|--------|-------|
|
|
| **Pattern Aggregation** | ✅ Complete | CC.6 - observations → aggregates |
|
|
| **Community Corpus Builder** | ✅ Complete | CC.7 - async, enabled by default |
|
|
| **Pattern Store API** | ✅ Complete | GenericPatternAggregateStore |
|
|
| **API Endpoint** | ✅ Complete | GET /v1/aphoria/patterns |
|
|
| **Dashboard UI** | ✅ Complete | /corpus page with filters |
|
|
| **API Client** | ✅ Complete | getPatterns() method |
|
|
| **Pattern Enrichment** | ✅ Complete | Query-time via PatternEnricher |
|
|
| **Empty States** | ✅ Complete | Handles zero patterns gracefully |
|
|
| **Error Handling** | ✅ Complete | 404 → empty, other → error |
|
|
|
|
---
|
|
|
|
## 🎯 What's Working Right Now
|
|
|
|
1. **Local Scans** → Pattern aggregates stored in StemeDB
|
|
2. **API Endpoint** → Queries pattern aggregates from StemeDB
|
|
3. **Dashboard** → Displays patterns with enrichment
|
|
4. **Filters** → Subject prefix, min projects, category
|
|
5. **Empty State** → Shown when no patterns exist
|
|
|
|
---
|
|
|
|
## 🚀 Next Steps (Optional Enhancements)
|
|
|
|
### Enhancement 1: Pattern Promotion UI
|
|
Add UI for reviewing and promoting patterns that meet thresholds:
|
|
- Show promotion candidates (50+ projects, not yet promoted)
|
|
- Manual approve/reject buttons
|
|
- Maps to Phase 14 (Governance Workflows)
|
|
|
|
### Enhancement 2: Pattern Timeline
|
|
Show how patterns evolve over time:
|
|
- Graph of project_count over time
|
|
- When pattern first appeared
|
|
- Adoption trend (growing/declining)
|
|
|
|
### Enhancement 3: Pattern Details Page
|
|
Click pattern → see:
|
|
- All projects using this pattern
|
|
- Code examples
|
|
- Related patterns
|
|
- Authority sources (RFC sections, OWASP references)
|
|
|
|
---
|
|
|
|
## 📋 Files Changed for Integration
|
|
|
|
| File | Purpose | Status |
|
|
|------|---------|--------|
|
|
| `crates/stemedb-api/src/handlers/aphoria/report.rs` | API endpoint for patterns | ✅ |
|
|
| `applications/aphoria-dashboard/src/app/corpus/page.tsx` | Corpus page route | ✅ |
|
|
| `applications/aphoria-dashboard/src/components/corpus/corpus-panel.tsx` | Main corpus component | ✅ |
|
|
| `applications/aphoria-dashboard/src/lib/api/client.ts` | API client method | ✅ |
|
|
| `applications/aphoria-dashboard/src/lib/api/types.ts` | TypeScript types | ✅ |
|
|
|
|
---
|
|
|
|
## ✅ Conclusion
|
|
|
|
**The Aphoria Dashboard is fully integrated with the new community corpus infrastructure (Phase CC).**
|
|
|
|
All core features work:
|
|
- Pattern aggregation (CC.6) ✅
|
|
- Community corpus builder (CC.7) ✅
|
|
- API endpoint serving patterns ✅
|
|
- Dashboard UI displaying patterns ✅
|
|
- Filters and enrichment working ✅
|
|
|
|
**No integration gaps detected.** The dashboard is ready to display community patterns as soon as they're created through scans.
|