Major additions: - Staged scanning modes (working tree, staged, committed) with git integration - Drift detection for baseline vs current state comparisons - Hosted API handlers for policy CRUD operations via StemeDB API - stemedb-ontology crate with domain definitions and medical extractors - Consumer health vertical UAT scenarios (GLP-1, gastroparesis, etc.) - Aphoria development skill documentation Code organization: - Split large files into focused modules to stay under 500-line limit - Extracted config tests, episteme helpers/drift/aliases, API helpers Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
158 lines
4.3 KiB
Markdown
158 lines
4.3 KiB
Markdown
# UAT: FDA Label Paradigm Shift (Epoch Supersession)
|
|
|
|
**Date:** YYYY-MM-DD
|
|
**Feature:** Epoch-Based Invalidation
|
|
**Status:** [ ] PASS / [ ] FAIL / [ ] BLOCKED
|
|
|
|
## Scenario
|
|
|
|
FDA updates the drug label for Semaglutide with significant new safety information. This creates a new "epoch" that supersedes hundreds of pre-existing assertions based on outdated label information.
|
|
|
|
Key validation: The epoch switch must invalidate O(1), not iterate O(N) through all assertions.
|
|
|
|
## Acceptance Criteria
|
|
|
|
| Criterion | Expected | Met? |
|
|
|-----------|----------|------|
|
|
| Old epoch assertions | Exist in storage | [ ] |
|
|
| New epoch created | Hash returned | [ ] |
|
|
| EpochAware query | Returns only new epoch | [ ] |
|
|
| Invalidation performance | O(1) not O(N) | [ ] |
|
|
| Old data still queryable | With explicit epoch filter | [ ] |
|
|
|
|
## Test Matrix
|
|
|
|
| Step | Action | Expected | Actual | Status |
|
|
|------|--------|----------|--------|--------|
|
|
| 1 | Create 100 old-epoch assertions | 100 hashes returned | | [ ] |
|
|
| 2 | Create new epoch | Epoch hash returned | | [ ] |
|
|
| 3 | Create new-epoch assertion | Hash returned | | [ ] |
|
|
| 4 | Query with EpochAware lens | Only new assertion | | [ ] |
|
|
| 5 | Query old epoch explicitly | Old assertions returned | | [ ] |
|
|
| 6 | Measure invalidation time | < 10ms | | [ ] |
|
|
|
|
## Setup Commands
|
|
|
|
```bash
|
|
# Start StemeDB
|
|
cargo run --bin stemedb-api &
|
|
sleep 2
|
|
```
|
|
|
|
## Test Commands
|
|
|
|
### Step 1: Ingest Old-Epoch Assertions (Pre-FDA Update)
|
|
|
|
```bash
|
|
# Create 100 assertions in the "pre-2024-label" epoch
|
|
for i in $(seq 1 100); do
|
|
curl -s -X POST http://localhost:18180/v1/assertions \
|
|
-H "Content-Type: application/json" \
|
|
-d "{
|
|
\"subject\": \"Semaglutide\",
|
|
\"predicate\": \"has_boxed_warning\",
|
|
\"object\": {\"Boolean\": false},
|
|
\"confidence\": 0.9,
|
|
\"source_class\": \"Clinical\",
|
|
\"epoch\": \"0000000000000000000000000000000000000000000000000000000000000001\"
|
|
}" > /dev/null
|
|
done
|
|
echo "Created 100 old-epoch assertions"
|
|
```
|
|
|
|
**Expected:** 100 assertions created
|
|
**Actual:**
|
|
**Status:** [ ]
|
|
|
|
### Step 2: Create New Epoch (FDA Label Update)
|
|
|
|
```bash
|
|
curl -X POST http://localhost:18180/v1/epochs \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"name": "FDA Label Update 2024",
|
|
"description": "New boxed warning added for thyroid C-cell tumors",
|
|
"supersedes": "0000000000000000000000000000000000000000000000000000000000000001",
|
|
"effective_date": "2024-06-15T00:00:00Z"
|
|
}'
|
|
```
|
|
|
|
**Expected:** `{"epoch_id": "..."}`
|
|
**Actual:**
|
|
**Status:** [ ]
|
|
|
|
### Step 3: Create New-Epoch Assertion
|
|
|
|
```bash
|
|
curl -X POST http://localhost:18180/v1/assertions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"subject": "Semaglutide",
|
|
"predicate": "has_boxed_warning",
|
|
"object": {"Boolean": true},
|
|
"confidence": 1.0,
|
|
"source_class": "Regulatory",
|
|
"epoch": "<NEW_EPOCH_ID_FROM_STEP_2>"
|
|
}'
|
|
```
|
|
|
|
**Expected:** Hash returned
|
|
**Actual:**
|
|
**Status:** [ ]
|
|
|
|
### Step 4: Query with EpochAware Lens
|
|
|
|
```bash
|
|
curl "http://localhost:18180/v1/query?subject=Semaglutide&predicate=has_boxed_warning&lens=epoch-aware"
|
|
```
|
|
|
|
**Expected:** Only the new-epoch assertion (has_boxed_warning = true)
|
|
**Actual:**
|
|
**Status:** [ ]
|
|
|
|
### Step 5: Query Old Epoch Explicitly
|
|
|
|
```bash
|
|
curl "http://localhost:18180/v1/query?subject=Semaglutide&predicate=has_boxed_warning&epoch=0000000000000000000000000000000000000000000000000000000000000001"
|
|
```
|
|
|
|
**Expected:** Returns old assertions (has_boxed_warning = false)
|
|
**Actual:**
|
|
**Status:** [ ]
|
|
|
|
### Step 6: Measure Invalidation Performance
|
|
|
|
The epoch creation (Step 2) should complete in O(1) time, regardless of how many assertions exist in the old epoch.
|
|
|
|
```bash
|
|
# Time the epoch creation
|
|
time curl -X POST http://localhost:18180/v1/epochs \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"name": "Performance Test Epoch",
|
|
"supersedes": "0000000000000000000000000000000000000000000000000000000000000001"
|
|
}'
|
|
```
|
|
|
|
**Expected:** < 10ms (should not scan 100 assertions)
|
|
**Actual:**
|
|
**Status:** [ ]
|
|
|
|
## Sign-Off Checklist
|
|
|
|
- [ ] Old epoch assertions preserved in storage
|
|
- [ ] New epoch created successfully
|
|
- [ ] EpochAware lens filters to current epoch
|
|
- [ ] Time-travel query to old epoch works
|
|
- [ ] Invalidation is O(1) not O(N)
|
|
|
|
## Notes
|
|
|
|
*Verify that epoch supersession happens at query time (via lens), not at write time (no mass updates).*
|
|
|
|
---
|
|
|
|
**Tester:**
|
|
**Date:**
|
|
**Result:**
|