157dbbb9eb
## Phase 8: Enterprise Extractor Improvements ✅ - 14 security extractors (TLS, JWT, SQL injection, XSS, etc.) - 10 framework-specific extractors (Spring, Django, Rails, etc.) - Config file security detection (YAML, TOML) ## Phase 9: Autonomous Extractor Generation ✅ - Shadow mode executor with TP/FP tracking - Graduation pipeline with confidence thresholds - Auto-rollback on regression detection - Cross-project pattern syncing ## UAT Suite Complete (14 scripts, 90 tests) - test-core-detection.sh (6 tests) - test-declarative-extractors.sh (5 tests) - test-domain-frameworks.sh (5 tests) - test-domain-unreal.sh (3 tests) - test-llm-extraction.sh (6 tests) - test-eval-harness.sh (5 tests) - test-cross-language.sh (3 tests) - test-precommit-performance.sh (4 tests) - test-output-formats.sh (8 tests) - test-drift-detection.sh (6 tests) - test-exit-codes.sh (12 tests) + 3 more scripts ## Other Changes - Updated roadmap to mark Phase 8-9 complete - Added .gitignore entries for build artifacts - Updated pre-commit: 800 line limit, exclude tests/data/cmd Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
116 lines
3.7 KiB
Markdown
116 lines
3.7 KiB
Markdown
# UAT Report: WAL Segment Synchronization Fix
|
|
|
|
**Date:** 2026-02-05
|
|
**Phase/Feature:** WAL/IngestWorker synchronization
|
|
**Tester:** Claude Code
|
|
**Status:** PASS
|
|
|
|
## Summary
|
|
|
|
Fixed WAL segment size cache bug causing IngestWorker to fail reading newly-written records. All 180+ core tests now pass, and end-to-end pipeline verified working.
|
|
|
|
## Scope
|
|
|
|
What was tested:
|
|
- WAL append and read operations
|
|
- IngestWorker record processing
|
|
- v1 and v2 signature verification
|
|
- End-to-end ingestion pipeline (pharma-ingest)
|
|
- Skeptic lens conflict analysis
|
|
|
|
What was NOT tested:
|
|
- Load testing (sustained throughput)
|
|
- Crash recovery under load
|
|
- Distributed cluster scenarios
|
|
|
|
## Environment
|
|
|
|
- Rust version: stable (1.x)
|
|
- OS: macOS Darwin 23.6.0
|
|
- Commit: Working tree (uncommitted fix)
|
|
|
|
## Test Results
|
|
|
|
### WAL Tests (stemedb-wal)
|
|
|
|
| Test | Expected | Actual | Status |
|
|
|------|----------|--------|--------|
|
|
| test_read_across_segments | Records readable after append | Records readable | PASS |
|
|
| All 40 WAL tests | Pass | 40/40 pass | PASS |
|
|
|
|
### Ingest Tests (stemedb-ingest)
|
|
|
|
| Test | Expected | Actual | Status |
|
|
|------|----------|--------|--------|
|
|
| test_ingest_assertion | Process assertion from WAL | Processed successfully | PASS |
|
|
| test_ingest_vote | Process vote from WAL | Processed successfully | PASS |
|
|
| test_crash_recovery_* | Survive crash, no data loss | All recovery tests pass | PASS |
|
|
| test_rejects_invalid_signature | Reject tampered assertion | Rejected correctly | PASS |
|
|
| All 41 ingest tests | Pass | 41/41 pass | PASS |
|
|
|
|
### End-to-End Pipeline
|
|
|
|
| Test | Expected | Actual | Status |
|
|
|------|----------|--------|--------|
|
|
| API server startup | Binds to :18180 | Started successfully | PASS |
|
|
| pharma-ingest | 11 assertions, 0 errors | 11/11 ingested | PASS |
|
|
| /v1/health | assertions_count > 0 | assertions_count: 38 | PASS |
|
|
| /v1/query?subject=Semaglutide | Returns assertions | 28 assertions | PASS |
|
|
| /v1/skeptic | Returns conflict analysis | conflict_score: 0.995 | PASS |
|
|
|
|
## Issues Found
|
|
|
|
### WAL Segment Size Cache Stale
|
|
|
|
**Severity:** Critical
|
|
**Status:** Fixed
|
|
|
|
**Description:** `Journal::append()` updated `current_offset` but not the `SegmentManager`'s cached segment `size`. When `Journal::read()` tried to read, it used the stale cached size (8 bytes = header only) and rejected reads as "beyond segment size".
|
|
|
|
**Root Cause:** `SegmentManager` only updated segment sizes in `open()` and `refresh()`, not after writes.
|
|
|
|
**Fix Applied:**
|
|
1. Added `update_current_segment_size()` method to `SegmentManager`
|
|
2. Called it from `Journal::append()` after each write
|
|
|
|
## Fixes Applied
|
|
|
|
- `crates/stemedb-wal/src/segment.rs`: Added `update_current_segment_size()` method
|
|
- `crates/stemedb-wal/src/journal.rs`: Call `update_current_segment_size()` after `guard.write()`
|
|
|
|
## Artifacts
|
|
|
|
Test output:
|
|
```
|
|
stemedb-core: 99/99 pass
|
|
stemedb-ingest: 41/41 pass
|
|
stemedb-wal: 40/40 pass
|
|
Full workspace: 532/534 pass (2 pre-existing failures in aphoria TLS tests)
|
|
```
|
|
|
|
Skeptic query result (Semaglutide nausea_rate):
|
|
```json
|
|
{
|
|
"status": "Contested",
|
|
"conflict_score": 0.99540675,
|
|
"claims": [
|
|
{"value": {"type": "Number", "value": 0.25}, "weight_share": 0.54},
|
|
{"value": {"type": "Number", "value": 0.38}, "weight_share": 0.46}
|
|
]
|
|
}
|
|
```
|
|
|
|
## Recommendations
|
|
|
|
1. **Add load testing** - Verify sustained write throughput
|
|
2. **Fix aphoria TLS tests** - 2 pre-existing failures in `extractors::tls_version::tests`
|
|
3. **Document backup/restore** - WAL archival and recovery procedures
|
|
4. **Add API authentication** - Currently no auth on endpoints
|
|
|
|
## Sign-Off
|
|
|
|
- [x] All critical tests pass
|
|
- [x] No blocking issues remain
|
|
- [ ] Documentation updated (this report)
|
|
- [ ] Ready for release (pending load testing)
|