jordan/stemedb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
71 Commits 1 Branch 0 Tags 18 MiB
f1244a84d7
Commit Graph

8 Commits

Author SHA1 Message Date
jordan
ad07a75d0a feat: add source content to source registry, signed assertions, feed endpoint, dashboard enhancements 
- Add `content: Option<String>` to SourceRecord with rkyv schema evolution
  (LegacySourceRecord compat deserializer for backward compatibility)
- Add MAX_SOURCE_CONTENT_LEN (1MB) limit with API validation
- Strip content from list responses, include in single-source GET
- Update Go SDK RegisterSourceRequest with Content field
- FCM pipeline extracts PDF text via pdftotext and passes to registration
- Dashboard impact panel fetches and displays source content with expand/collapse
- Add feed endpoint, dashboard feed panel, and signed assertion support
- Update data-structures.md, API docs, and storage docs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-19 21:54:27 -07:00
jordan
28fc3b5391 feat(aphoria): add C language support and streamline documentation 
Add Language::C variant with file detection (.c, Makefile, CMakeLists.txt)
and integration across prompts, regex_gen, and path_mapper. Simplify
README and guides to be more concise and scannable.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-12 03:02:33 -07:00
jml
65065f3d8f feat(aphoria): implement community corpus with wiki import and pattern aggregation 
Implements Phase 4 (A4) - Community corpus as first-class citizens:

- **Community Corpus Builder** - Queries StemeDB pattern aggregates
- **Wiki Import** - Bootstrap corpus from markdown docs (aphoria corpus import wiki)
- **Pattern Aggregation** - Automatic learning from local scans (--sync flag)
- **Storage Layer** - StemeDBPatternStore with content-addressed deduplication
- **Promotion Logic** - Multi-tier thresholds (95%/80%/50% adoption rates)
- **Corpus Build** - Unified registry for RFC/OWASP/Vendor/Community sources
- **Trust Packs** - Export corpus as signed, distributable artifacts
- **Documentation** - bootstrap-corpus.md guide + CLI reference updates

Technical details:
- Pattern aggregates stored as assertions with predicate "pattern_aggregate"
- Content-addressed subjects via BLAKE3(subject:predicate:value)
- PatternAggregator handles write path (observations → patterns)
- StemeDBPatternStore handles read path (pattern queries)
- Integration tests + fixtures in tests/wiki_import_test.rs

Deleted hardcoded.rs (368 lines) - corpus now fully emergent from StemeDB.
Deleted enriched-corpus-patterns.md (677 lines) - feature shipped.

Closes VG-026 (community corpus), part of A4 milestone.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-02-09 00:12:31 +00:00
jml
6430ff0fd6 fix(aphoria): move claims.toml to project root and fix verify integration 
## Root Cause
Claims file was in applications/aphoria/.aphoria/ but all commands looked
for .aphoria/claims.toml relative to project root. Additionally, .aphoria/
was fully gitignored, preventing version control of claims.

## Changes

### Path Fixes
- Move claims.toml from applications/aphoria/.aphoria/ to .aphoria/ at project root
- Update .gitignore: .aphoria/ → .aphoria/* with !.aphoria/claims.toml exception
- Now claims can be version controlled while keys remain secret

### Verify Integration (Scanner)
- scanner.rs: Load claims from ClaimsFile and call verify_claims()
- ScanResult: Add verify field with VerifyReport
- Report formatters: Add claim verification sections showing PASS/CONFLICT/MISSING

### Clippy Fix
- report/json.rs: Replace filter().map().expect() with filter_map()

## Verification
- aphoria scan . → Shows claim verification with verdicts
- aphoria verify run → Per-claim verification results
- aphoria verify map → Extractor coverage mapping (7/10 claims = 70%)
- aphoria claims list → Reads from project root
- aphoria claims create → Writes to project root
- All tests pass (1120+ aphoria tests)
- clippy --workspace passes

## Impact
Both primary use cases now work:
1. Day-to-day (commit-time): Skills can read/create claims via CLI
2. Audit (scan-time): Scanner verifies code against authored claims

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-02-08 11:09:57 +00:00
jml
3b5f88b4f0 feat(aphoria): implement claims architecture (A1-A5) with verify engine, corpus, coverage, and explain 
Complete Aphoria claims system overhaul:
- A1: Rename ExtractedClaim to Observation (extractors produce observations, not claims)
- A2: Add AuthoredClaim with full provenance, invariants, and authority tiers
- A3: Verify engine comparing observations against authored claims, CLI + formatters
- A4: Corpus as first-class assertions with predicate indexing, authority lens, trust packs
- A5: Coverage analysis, explain/docs generation, self-audit extractor, claim suggester skill

Also includes: 42 extractors updated for Observation type, verifiable_predicates trait,
conflict detection with comparison modes, claims TOML persistence, Grafana dashboard,
backup/restore scripts, and comprehensive test coverage.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-08 09:11:47 +00:00
jordan
157dbbb9eb feat: Complete Aphoria Phase 8-9 + UAT suite (90/90 tests passing) 
## Phase 8: Enterprise Extractor Improvements ✅
- 14 security extractors (TLS, JWT, SQL injection, XSS, etc.)
- 10 framework-specific extractors (Spring, Django, Rails, etc.)
- Config file security detection (YAML, TOML)

## Phase 9: Autonomous Extractor Generation ✅
- Shadow mode executor with TP/FP tracking
- Graduation pipeline with confidence thresholds
- Auto-rollback on regression detection
- Cross-project pattern syncing

## UAT Suite Complete (14 scripts, 90 tests)
- test-core-detection.sh (6 tests)
- test-declarative-extractors.sh (5 tests)
- test-domain-frameworks.sh (5 tests)
- test-domain-unreal.sh (3 tests)
- test-llm-extraction.sh (6 tests)
- test-eval-harness.sh (5 tests)
- test-cross-language.sh (3 tests)
- test-precommit-performance.sh (4 tests)
- test-output-formats.sh (8 tests)
- test-drift-detection.sh (6 tests)
- test-exit-codes.sh (12 tests)
+ 3 more scripts

## Other Changes
- Updated roadmap to mark Phase 8-9 complete
- Added .gitignore entries for build artifacts
- Updated pre-commit: 800 line limit, exclude tests/data/cmd

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-06 22:50:55 -07:00
jordan
bbe6aedc40 feat: Aphoria security extractors + LLM evaluation architecture + ontology docs 
New security extractors:
- insecure_deserialization, orm_injection, path_traversal, security_headers
- ssrf, unvalidated_redirects, weak_password, xxe
- Enhanced tls_version extractor with comprehensive cipher/protocol checks

Architecture docs:
- Scout-judge extraction pattern for LLM-based code analysis
- LLM prompt evaluation framework
- LLM eval implementation guide

Core improvements:
- stemedb-ontology README and client enhancements
- WAL journal/segment instrumentation
- Signing and ingestion refinements
- Consumer health demo script

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-05 15:22:55 -07:00
jordan
41c676a78e feat: Aphoria enterprise features + ontology SDK + file length compliance 
Enterprise Features:
- Hosted mode with remote sync for team pattern aggregation
- Community sharing with privacy-preserving anonymization
- LLM-based semantic claim extraction with Gemini integration
- Pattern learning with promotion to declarative extractors
- High-entropy secrets extractor with configurable thresholds
- Auth bypass and insecure cookies extractors

Module Refactoring:
- Split oversized files to comply with 500-line limit
- Config split: types/core.rs, types/extractors.rs, types/hosted.rs, etc.
- Handlers split: scan.rs, policy.rs, report.rs modules
- Extractors split: declarative/, high_entropy_secrets/, insecure_cookies/
- Learning split: store modules with metrics and persistence

SDK & Ontology:
- stemedb-ontology SDK with fluent builders and StemeDB client
- Pharma domain extractors for FDA Orange Book data
- Consumer health UAT test infrastructure

Code Quality:
- Fixed clippy warnings (needless_borrows_for_generic_args)
- Added KVStore trait imports where needed
- Fixed utoipa path re-exports for OpenAPI docs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-05 12:55:29 -07:00