157dbbb9eb
## Phase 8: Enterprise Extractor Improvements ✅ - 14 security extractors (TLS, JWT, SQL injection, XSS, etc.) - 10 framework-specific extractors (Spring, Django, Rails, etc.) - Config file security detection (YAML, TOML) ## Phase 9: Autonomous Extractor Generation ✅ - Shadow mode executor with TP/FP tracking - Graduation pipeline with confidence thresholds - Auto-rollback on regression detection - Cross-project pattern syncing ## UAT Suite Complete (14 scripts, 90 tests) - test-core-detection.sh (6 tests) - test-declarative-extractors.sh (5 tests) - test-domain-frameworks.sh (5 tests) - test-domain-unreal.sh (3 tests) - test-llm-extraction.sh (6 tests) - test-eval-harness.sh (5 tests) - test-cross-language.sh (3 tests) - test-precommit-performance.sh (4 tests) - test-output-formats.sh (8 tests) - test-drift-detection.sh (6 tests) - test-exit-codes.sh (12 tests) + 3 more scripts ## Other Changes - Updated roadmap to mark Phase 8-9 complete - Added .gitignore entries for build artifacts - Updated pre-commit: 800 line limit, exclude tests/data/cmd Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
139 lines
4.9 KiB
Markdown
139 lines
4.9 KiB
Markdown
# Production Readiness Verification
|
|
|
|
Systematic verification checklist for deploying StemeDB in production environments.
|
|
|
|
## Quick Reference
|
|
|
|
| Category | Status | Last Verified |
|
|
|----------|--------|---------------|
|
|
| Crash Recovery | ✅ Pass | 2026-02-05 |
|
|
| Signature Verification | ✅ Pass | 2026-02-05 |
|
|
| End-to-End Pipeline | ✅ Pass | 2026-02-05 |
|
|
| Load Testing | ✅ Tooling ready | Run `./scripts/run-load-test.sh` |
|
|
| API Security | ❌ Not done | - |
|
|
| Backup/Restore | ❌ Not done | - |
|
|
| Observability | ⚠️ Partial | - |
|
|
|
|
## Verification Areas
|
|
|
|
### 1. Data Integrity & Durability
|
|
|
|
| Check | Command | Expected |
|
|
|-------|---------|----------|
|
|
| WAL crash recovery | `cargo test -p stemedb-ingest test_crash_recovery` | All pass |
|
|
| No duplicate assertions | `cargo test -p stemedb-ingest test_p0_crash_recovery` | All pass |
|
|
| Cursor checkpoint | `cargo test -p stemedb-ingest test_cursor` | All pass |
|
|
|
|
### 2. Signature Verification
|
|
|
|
| Check | Command | Expected |
|
|
|-------|---------|----------|
|
|
| v1 signatures (legacy) | `cargo test -p stemedb-ingest test_ingest_assertion` | Pass |
|
|
| v2 signatures (enterprise) | Pharma-ingest with real keys | All assertions accepted |
|
|
| Invalid signature rejection | `cargo test -p stemedb-ingest test_rejects_invalid` | Pass |
|
|
| Unsigned assertion rejection | `cargo test -p stemedb-ingest test_rejects_unsigned` | Pass |
|
|
|
|
### 3. End-to-End Pipeline
|
|
|
|
| Check | Command | Expected |
|
|
|-------|---------|----------|
|
|
| API server starts | `cargo run --bin stemedb-api` | Binds to :18180 |
|
|
| Assertion ingestion | `POST /v1/assert` | Returns hash |
|
|
| Query returns data | `GET /v1/query?subject=X` | Returns assertions |
|
|
| Skeptic conflict analysis | `GET /v1/skeptic?subject=X&predicate=Y` | Returns conflict_score |
|
|
| Health check | `GET /v1/health` | assertions_count > 0 |
|
|
|
|
### 4. Load Testing
|
|
|
|
**Tool:** `cmd/load-test` (Go-based with native Ed25519 signing)
|
|
|
|
| Scenario | Command | Target | Status |
|
|
|----------|---------|--------|--------|
|
|
| Baseline latency | `--scenario baseline` | 10K assertions, p99 < 200ms | ✅ Ready |
|
|
| Sustained writes | `--scenario sustained` | 1K/sec for 1 hour, p99 < 200ms | ✅ Ready |
|
|
| Concurrent readers | `--scenario concurrent` | 100 readers, <2x degradation | ✅ Ready |
|
|
|
|
**Quick Start:**
|
|
```bash
|
|
# Run all scenarios (5 min sustained by default)
|
|
./scripts/run-load-test.sh
|
|
|
|
# Run full 1-hour sustained test
|
|
LOAD_TEST_DURATION=1h ./scripts/run-load-test.sh
|
|
|
|
# Run specific scenario
|
|
./scripts/run-load-test.sh --scenario baseline
|
|
go run ./cmd/load-test --api-url http://localhost:18180 --scenario sustained --duration 10m
|
|
```
|
|
|
|
**Prerequisites:**
|
|
- Set `STEMEDB_METER_ENABLED=false` for accurate sustained test results
|
|
- Ensure ~10-20GB disk space for 1-hour tests (~3.6M assertions)
|
|
- Results saved to `uat/production-readiness/results/`
|
|
|
|
### 5. API Security (TODO)
|
|
|
|
| Check | Implementation | Status |
|
|
|-------|----------------|--------|
|
|
| Authentication | JWT or API keys | Not implemented |
|
|
| Rate limiting | Per-client limits | Not implemented |
|
|
| Input validation | Oversized payloads rejected | Partial |
|
|
| TLS in transit | HTTPS termination | External (nginx/LB) |
|
|
|
|
### 6. Backup & Restore (TODO)
|
|
|
|
| Check | Procedure | Status |
|
|
|-------|-----------|--------|
|
|
| Point-in-time recovery | WAL replay | Not documented |
|
|
| WAL archival | S3/GCS upload | Not implemented |
|
|
| Restore test | Full recovery | Not tested |
|
|
|
|
### 7. Observability (Partial)
|
|
|
|
| Check | Implementation | Status |
|
|
|-------|----------------|--------|
|
|
| Structured logs | `tracing` crate | ✅ Implemented |
|
|
| Metrics endpoint | `/metrics` Prometheus | Not implemented |
|
|
| Distributed tracing | OpenTelemetry | Not implemented |
|
|
| Alerting | WAL lag, errors | Not implemented |
|
|
|
|
## Running Full Verification
|
|
|
|
```bash
|
|
# 1. Run all unit tests
|
|
cargo test --workspace --lib
|
|
|
|
# 2. Start fresh API server
|
|
rm -rf /tmp/stemedb-prod-test && mkdir -p /tmp/stemedb-prod-test
|
|
STEMEDB_DATA_DIR=/tmp/stemedb-prod-test cargo run --bin stemedb-api &
|
|
sleep 5
|
|
|
|
# 3. Run pharma-ingest (tests v2 signatures)
|
|
cargo run -p stemedb-ontology --bin pharma-ingest -- --with-conflicts
|
|
|
|
# 4. Verify endpoints
|
|
curl http://localhost:18180/v1/health
|
|
curl "http://localhost:18180/v1/query?subject=Semaglutide"
|
|
curl "http://localhost:18180/v1/skeptic?subject=Semaglutide&predicate=nausea_rate"
|
|
|
|
# 5. Kill and restart (crash recovery test)
|
|
pkill -9 -f stemedb-api
|
|
STEMEDB_DATA_DIR=/tmp/stemedb-prod-test cargo run --bin stemedb-api &
|
|
sleep 3
|
|
curl http://localhost:18180/v1/health # Should show same assertion count
|
|
```
|
|
|
|
## Results Archive
|
|
|
|
Date-stamped verification results:
|
|
|
|
| Date | Report | Summary |
|
|
|------|--------|---------|
|
|
| 2026-02-05 | [wal-sync-fix.md](./results/2026-02-05-wal-sync-fix.md) | WAL segment cache fix, all tests pass |
|
|
|
|
## Related
|
|
|
|
- [UAT Report Template](../how-to.md)
|
|
- [Consumer Health UAT](../consumer-health/README.md)
|
|
- [Production Readiness Facts](../../ai-lookup/features/production-readiness.md)
|