Enterprise Features: - Hosted mode with remote sync for team pattern aggregation - Community sharing with privacy-preserving anonymization - LLM-based semantic claim extraction with Gemini integration - Pattern learning with promotion to declarative extractors - High-entropy secrets extractor with configurable thresholds - Auth bypass and insecure cookies extractors Module Refactoring: - Split oversized files to comply with 500-line limit - Config split: types/core.rs, types/extractors.rs, types/hosted.rs, etc. - Handlers split: scan.rs, policy.rs, report.rs modules - Extractors split: declarative/, high_entropy_secrets/, insecure_cookies/ - Learning split: store modules with metrics and persistence SDK & Ontology: - stemedb-ontology SDK with fluent builders and StemeDB client - Pharma domain extractors for FDA Orange Book data - Consumer health UAT test infrastructure Code Quality: - Fixed clippy warnings (needless_borrows_for_generic_args) - Added KVStore trait imports where needed - Fixed utoipa path re-exports for OpenAPI docs Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
223 lines
5.3 KiB
Markdown
223 lines
5.3 KiB
Markdown
# UAT Quick Start Guide
|
|
|
|
Get Week 4 UAT scenarios running in 5 minutes.
|
|
|
|
## Prerequisites
|
|
|
|
```bash
|
|
# 1. Ensure you're in the stemedb repo
|
|
cd /Users/jordanwashburn/Workspace/orchard9/stemedb
|
|
|
|
# 2. Build everything (one-time setup)
|
|
cargo build --workspace
|
|
```
|
|
|
|
## Step-by-Step Execution
|
|
|
|
### 1. Start the API Server (Terminal 1)
|
|
|
|
```bash
|
|
cargo run -p stemedb-api
|
|
```
|
|
|
|
Wait for output like:
|
|
```
|
|
INFO stemedb_api] Server running on http://0.0.0.0:18180
|
|
```
|
|
|
|
### 2. Validate API Readiness (Terminal 2)
|
|
|
|
```bash
|
|
cd uat/consumer-health
|
|
./validate_api_readiness.sh http://localhost:18180
|
|
```
|
|
|
|
Expected output:
|
|
```
|
|
✅ API is READY for UAT execution
|
|
```
|
|
|
|
### 3. Run UAT Tests
|
|
|
|
**Option A: All scenarios at once**
|
|
```bash
|
|
STEMEDB_API_URL=http://localhost:18180 \
|
|
cargo test --test consumer_health_uat run_all_uat_scenarios -- --ignored --nocapture
|
|
```
|
|
|
|
**Option B: Individual scenarios**
|
|
```bash
|
|
# GLP-1 Muscle Loss Contradiction
|
|
STEMEDB_API_URL=http://localhost:18180 \
|
|
cargo test --test consumer_health_uat uat_glp1_muscle_loss_contradiction -- --ignored --nocapture
|
|
|
|
# Gastroparesis Multi-Source
|
|
STEMEDB_API_URL=http://localhost:18180 \
|
|
cargo test --test consumer_health_uat uat_gastroparesis_multi_source -- --ignored --nocapture
|
|
|
|
# Layered Consensus
|
|
STEMEDB_API_URL=http://localhost:18180 \
|
|
cargo test --test consumer_health_uat uat_layered_consensus -- --ignored --nocapture
|
|
```
|
|
|
|
## What You Should See
|
|
|
|
### Successful Test Output
|
|
|
|
```
|
|
=== UAT: GLP-1 Muscle Loss Contradiction ===
|
|
Step 1: Ingest Study A (muscle loss observed)
|
|
✓ Study A hash: abcd1234...
|
|
Step 2: Ingest Study B (muscle mass preserved)
|
|
✓ Study B hash: ef567890...
|
|
Step 3: Query Skeptic Lens
|
|
Status: Contested
|
|
Conflict Score: 0.88
|
|
Claims: 2
|
|
Candidates: 2
|
|
✓ PASS: GLP-1 Muscle Loss Contradiction
|
|
```
|
|
|
|
### Failed Test (Common Issues)
|
|
|
|
**Issue 1: Connection refused**
|
|
```
|
|
API error 000: Connection refused
|
|
```
|
|
**Fix:** Make sure API server is running in Terminal 1.
|
|
|
|
**Issue 2: 404 Not Found**
|
|
```
|
|
API error 404: No assertions found
|
|
```
|
|
**Fix:** This is normal if database is empty. Tests create data first.
|
|
|
|
**Issue 3: 400 Bad Request - Invalid signature**
|
|
```
|
|
API error 400: At least one signature is required
|
|
```
|
|
**Fix:** Signature validation is enforced. See "Known Issues" below.
|
|
|
|
## Known Issues
|
|
|
|
### Signature Validation
|
|
|
|
If tests fail with signature errors:
|
|
|
|
1. **Check if API has test mode** (recommended)
|
|
2. **OR generate real signatures** (complex, not recommended for UAT)
|
|
|
|
### Database State
|
|
|
|
Tests write to a persistent database. To start fresh:
|
|
|
|
```bash
|
|
# Stop API server (Ctrl+C in Terminal 1)
|
|
# Remove database
|
|
rm -rf /tmp/stemedb-data # or wherever your DB is
|
|
# Restart API server
|
|
cargo run -p stemedb-api
|
|
```
|
|
|
|
## Next Steps After Successful Run
|
|
|
|
1. **Capture Results**
|
|
- Take screenshots of test output
|
|
- Copy actual values into UAT markdown files
|
|
- Update test matrices
|
|
|
|
2. **Sign Off**
|
|
- Mark scenarios as PASS in markdown files
|
|
- Update `crates/stemedb-ontology/ROADMAP.md`
|
|
- Commit results
|
|
|
|
3. **Report Issues**
|
|
- Create GitHub issues for any failing tests
|
|
- Document unexpected behavior
|
|
- Propose fixes
|
|
|
|
## Advanced Usage
|
|
|
|
### Run with different API URL
|
|
|
|
```bash
|
|
STEMEDB_API_URL=http://staging.example.com:18180 \
|
|
cargo test --test consumer_health_uat -- --ignored --nocapture
|
|
```
|
|
|
|
### Run specific test multiple times
|
|
|
|
```bash
|
|
for i in {1..5}; do
|
|
echo "=== Run $i ==="
|
|
STEMEDB_API_URL=http://localhost:18180 \
|
|
cargo test --test consumer_health_uat uat_glp1_muscle_loss_contradiction -- --ignored --nocapture
|
|
done
|
|
```
|
|
|
|
### Save test output to file
|
|
|
|
```bash
|
|
STEMEDB_API_URL=http://localhost:18180 \
|
|
cargo test --test consumer_health_uat run_all_uat_scenarios -- --ignored --nocapture \
|
|
2>&1 | tee uat_results_$(date +%Y%m%d_%H%M%S).log
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### "cargo: command not found"
|
|
Install Rust: https://rustup.rs/
|
|
|
|
### "package stemedb-api not found"
|
|
```bash
|
|
cargo build --workspace
|
|
```
|
|
|
|
### "Permission denied" for validate script
|
|
```bash
|
|
chmod +x uat/consumer-health/validate_api_readiness.sh
|
|
```
|
|
|
|
### Tests are slow / timing out
|
|
Increase sleep durations in test code:
|
|
```rust
|
|
std::thread::sleep(std::time::Duration::from_secs(5)); // was 2
|
|
```
|
|
|
|
## Documentation Links
|
|
|
|
- [Full Execution Plan](./WEEK4_EXECUTION_PLAN.md) - Detailed instructions
|
|
- [Delivery Summary](./WEEK4_DELIVERY_SUMMARY.md) - What was built
|
|
- [UAT Scenarios](./README.md) - All 12 scenarios
|
|
- [API Documentation](http://localhost:18180/swagger-ui/) - While server is running
|
|
|
|
## Getting Help
|
|
|
|
**Check logs:**
|
|
- API server logs in Terminal 1
|
|
- Test output in Terminal 2
|
|
|
|
**Common commands:**
|
|
```bash
|
|
# Check if API is running
|
|
curl http://localhost:18180/v1/health
|
|
|
|
# View OpenAPI spec
|
|
curl http://localhost:18180/api-docs/openapi.json | jq .
|
|
|
|
# Check database location
|
|
# (Look for STEMEDB_DATA_DIR in API server logs)
|
|
```
|
|
|
|
---
|
|
|
|
**Quick Reference Card**
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ Terminal 1: cargo run -p stemedb-api │
|
|
│ Terminal 2: ./validate_api_readiness.sh http://localhost... │
|
|
│ Terminal 2: STEMEDB_API_URL=... cargo test --test ... │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
```
|