jordan/stemedb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
46 Commits 1 Branch 0 Tags 18 MiB
65065f3d8f
Commit Graph

11 Commits

Author SHA1 Message Date
jml
65065f3d8f feat(aphoria): implement community corpus with wiki import and pattern aggregation 
Implements Phase 4 (A4) - Community corpus as first-class citizens:

- **Community Corpus Builder** - Queries StemeDB pattern aggregates
- **Wiki Import** - Bootstrap corpus from markdown docs (aphoria corpus import wiki)
- **Pattern Aggregation** - Automatic learning from local scans (--sync flag)
- **Storage Layer** - StemeDBPatternStore with content-addressed deduplication
- **Promotion Logic** - Multi-tier thresholds (95%/80%/50% adoption rates)
- **Corpus Build** - Unified registry for RFC/OWASP/Vendor/Community sources
- **Trust Packs** - Export corpus as signed, distributable artifacts
- **Documentation** - bootstrap-corpus.md guide + CLI reference updates

Technical details:
- Pattern aggregates stored as assertions with predicate "pattern_aggregate"
- Content-addressed subjects via BLAKE3(subject:predicate:value)
- PatternAggregator handles write path (observations → patterns)
- StemeDBPatternStore handles read path (pattern queries)
- Integration tests + fixtures in tests/wiki_import_test.rs

Deleted hardcoded.rs (368 lines) - corpus now fully emergent from StemeDB.
Deleted enriched-corpus-patterns.md (677 lines) - feature shipped.

Closes VG-026 (community corpus), part of A4 milestone.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-02-09 00:12:31 +00:00
jml
3b5f88b4f0 feat(aphoria): implement claims architecture (A1-A5) with verify engine, corpus, coverage, and explain 
Complete Aphoria claims system overhaul:
- A1: Rename ExtractedClaim to Observation (extractors produce observations, not claims)
- A2: Add AuthoredClaim with full provenance, invariants, and authority tiers
- A3: Verify engine comparing observations against authored claims, CLI + formatters
- A4: Corpus as first-class assertions with predicate indexing, authority lens, trust packs
- A5: Coverage analysis, explain/docs generation, self-audit extractor, claim suggester skill

Also includes: 42 extractors updated for Observation type, verifiable_predicates trait,
conflict detection with comparison modes, claims TOML persistence, Grafana dashboard,
backup/restore scripts, and comprehensive test coverage.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-08 09:11:47 +00:00
jordan
c65066fd1c feat(aphoria): implement ignore & exclusion system (Phase 16) 
Reduces scan noise by 96% through proper exclusion of test fixtures,
demo apps, and intentional vulnerabilities.

Phase 16.1 - Glob Pattern Matching:
- Replace starts_with() with globset for ** and * patterns
- Backwards compatible with legacy prefix patterns
- Add walker/mod.rs tests for glob exclusions

Phase 16.2 - .aphoriaignore File:
- Create walker/ignore_file.rs for gitignore-style parsing
- Merge with aphoria.toml excludes
- Support # comments and whitespace trimming

Phase 16.3 - Inline Ignore Comments:
- Create extractors/ignore_comments.rs parser
- Support // aphoria:ignore, // aphoria:ignore-next-line
- Support // aphoria:ignore-block / // aphoria:end-ignore
- Multiple comment styles: //, #, /*, --, <!--
- Integrate with ExtractorRegistry.extract_all()

Phase 16.4 - Ack Export/Import:
- Create ack_file.rs for TOML serialization
- Add 'aphoria ack add' subcommand
- Add 'aphoria ack export' to .aphoria/acks.toml
- Add 'aphoria ack import' from .aphoria/acks.toml
- Preserve expiry and reason fields

Also configures stemedb with:
- aphoria.toml with glob excludes for vulnbank, extractors, fixtures
- .aphoriaignore for dashboard, community, latent, SDK examples

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-07 17:28:50 -07:00
jordan
c849627620 feat: add Aphoria dashboard scans and corpus UI 
- Add scans panel with finding details, verdict badges, and filters
- Add corpus panel for managing knowledge sources
- Add scan cache for API state management
- Update sidebar navigation with new routes
- Extend API types for scans and corpus endpoints
- Add .aphoria/ to gitignore (contains project keys)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-07 15:56:49 -07:00
jordan
8af9b48ac7 feat: Complete Aphoria Phase 14 - Governance Workflows 
Implement structured approval workflows for pattern promotion with full
audit trails for SOC 2 compliance.

Core Components:
- governance/types.rs: ApprovalRequest, ApprovalStatus, ApprovalDecision
- governance/workflow.rs: ApprovalWorkflow, ApprovalStage with escalation
- governance/store.rs: JSONL persistence for requests and decisions
- governance/state_machine.rs: Approval state transitions with auto-advance
- governance/audit.rs: AuditTrail with JSON/CSV/Markdown export

CLI Commands:
- aphoria governance pending/approve/reject/escalate/status/create
- aphoria audit trail/export/summary

Integration:
- Pipeline gate blocks promotion until governance approval
- Auto-creates approval requests when governance enabled
- Evidence-based auto-approval for high-confidence patterns

Also includes:
- Phase 11-13: Evidence, Lifecycle, Scope modules
- 62+ governance-specific tests (946 total passing)
- Clippy clean with -D warnings
- Refactored cli.rs into submodules (governance, lifecycle, scope, etc.)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-07 05:16:26 -07:00
jordan
157dbbb9eb feat: Complete Aphoria Phase 8-9 + UAT suite (90/90 tests passing) 
## Phase 8: Enterprise Extractor Improvements ✅
- 14 security extractors (TLS, JWT, SQL injection, XSS, etc.)
- 10 framework-specific extractors (Spring, Django, Rails, etc.)
- Config file security detection (YAML, TOML)

## Phase 9: Autonomous Extractor Generation ✅
- Shadow mode executor with TP/FP tracking
- Graduation pipeline with confidence thresholds
- Auto-rollback on regression detection
- Cross-project pattern syncing

## UAT Suite Complete (14 scripts, 90 tests)
- test-core-detection.sh (6 tests)
- test-declarative-extractors.sh (5 tests)
- test-domain-frameworks.sh (5 tests)
- test-domain-unreal.sh (3 tests)
- test-llm-extraction.sh (6 tests)
- test-eval-harness.sh (5 tests)
- test-cross-language.sh (3 tests)
- test-precommit-performance.sh (4 tests)
- test-output-formats.sh (8 tests)
- test-drift-detection.sh (6 tests)
- test-exit-codes.sh (12 tests)
+ 3 more scripts

## Other Changes
- Updated roadmap to mark Phase 8-9 complete
- Added .gitignore entries for build artifacts
- Updated pre-commit: 800 line limit, exclude tests/data/cmd

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-06 22:50:55 -07:00
jordan
41c676a78e feat: Aphoria enterprise features + ontology SDK + file length compliance 
Enterprise Features:
- Hosted mode with remote sync for team pattern aggregation
- Community sharing with privacy-preserving anonymization
- LLM-based semantic claim extraction with Gemini integration
- Pattern learning with promotion to declarative extractors
- High-entropy secrets extractor with configurable thresholds
- Auth bypass and insecure cookies extractors

Module Refactoring:
- Split oversized files to comply with 500-line limit
- Config split: types/core.rs, types/extractors.rs, types/hosted.rs, etc.
- Handlers split: scan.rs, policy.rs, report.rs modules
- Extractors split: declarative/, high_entropy_secrets/, insecure_cookies/
- Learning split: store modules with metrics and persistence

SDK & Ontology:
- stemedb-ontology SDK with fluent builders and StemeDB client
- Pharma domain extractors for FDA Orange Book data
- Consumer health UAT test infrastructure

Code Quality:
- Fixed clippy warnings (needless_borrows_for_generic_args)
- Added KVStore trait imports where needed
- Fixed utoipa path re-exports for OpenAPI docs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-05 12:55:29 -07:00
jordan
1cc453c97b feat: Aphoria policy source tracking + claim extraction pipeline 
- Add PolicySourceStore for tracking where policies come from
- Implement claim extraction skill and API endpoints
- Add community UI text selection extractor component
- Create Go SDK aphoria client for policy operations
- Document patent specifications and legal disclosures
- Add guides: golden path loop, policy audit trails, pre-flight checks
- Expand Unreal Engine config extractor with source tracking
- Add UAT reports for policy source tracking validation
- Refactor tests.rs into modular test files

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-04 02:35:02 -07:00
jordan
b3e8a9a058 feat: Multi-application expansion with chaos testing and community UI 
Major additions:
- Community Next.js app (port 18187) for browsing claims with API docs
- stemedb-chaos crate: Fault injection, chaos testing, CRDT properties
- Latent ingestion system: Reddit/FDA ingesters with ADK-Go agents
- Disputed claims handling: Manual review workflows and validation
- Aphoria security scanner: New extractors (SQL injection, command
  injection, weak crypto, TLS version), policy-based ignores, UAT reports
- Docker infrastructure: Dockerfile, docker-compose.yml for full stack
- VulnBank demo: Intentionally vulnerable multi-language test corpus

SDK & API enhancements:
- Source registry handlers for tracking data provenance
- Metrics endpoint
- Skeptic filtering improvements

Code quality:
- Split 14 large files (>500 lines) into focused modules
- All files now under 500-line limit per project guidelines

Documentation:
- Chaos testing guide, circuit breakers, observability docs
- Phase 7 UAT documentation updates
- Martin Kleppmann technical writer agent

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-04 01:24:14 -07:00
jordan
d3a88585fe feat: Phase 6 UAT - Admission control, HLC recency, cluster coordination 
This commit includes comprehensive work on Phase 6 features:

## Admission Control (Phase 6 admission middleware)
- AdmissionStore implementation backed by TrustRankStore
- PoW verification with tier-based difficulty computation
- Trust tier progression (Newcomer → Established → Trusted → Authority)
- API integration with admission status endpoints

## HLC Recency Lens (Phase 6C)
- HlcRecencyLens for distributed system ordering
- Hybrid logical clock integration with causality preservation

## Cluster Coordination (Phase 6C)
- Multi-node cluster tests (availability, partition tolerance)
- CRDT convergence tests for anti-entropy sync
- Gateway handler improvements

## Aphoria Code Linter (Phase 2A)
- RFC/OWASP corpus builders with network fetching and caching
- Concept hierarchy with auto-alias creation on conflict detection
- Multiple security extractors (TLS, JWT, CORS, secrets, rate limiting)

## Code Organization
- Split large files into modules to comply with 500-line limit
- Improved test organization with separate test modules
- Fixed rkyv serialization for EigenTrustState (AgentScore struct)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-03 00:43:37 -07:00
jordan
2b0923f20e feat: Distributed replication foundation (Phase 6A) - HLC, Merkle trees, CRDT stores, sync protocol 
- Add Hybrid Logical Clock (HLC) for causality tracking across nodes
- Implement Merkle tree for efficient diff/sync with BLAKE3 hashing
- Add CRDT-aware stores for assertions and votes with vector clocks
- Create stemedb-sync crate with anti-entropy and gossip protocols
- Add stemedb-rpc crate with gRPC sync service (proto definitions)
- Implement SupersessionChain for tracking assertion lifecycles
- Add Aphoria application for code analysis/reporting
- Add battery11 replication test scaffolding
- Fix .gitignore to exclude nested target directories

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-02 19:31:54 -07:00