b3e8a9a058
Major additions: - Community Next.js app (port 18187) for browsing claims with API docs - stemedb-chaos crate: Fault injection, chaos testing, CRDT properties - Latent ingestion system: Reddit/FDA ingesters with ADK-Go agents - Disputed claims handling: Manual review workflows and validation - Aphoria security scanner: New extractors (SQL injection, command injection, weak crypto, TLS version), policy-based ignores, UAT reports - Docker infrastructure: Dockerfile, docker-compose.yml for full stack - VulnBank demo: Intentionally vulnerable multi-language test corpus SDK & API enhancements: - Source registry handlers for tracking data provenance - Metrics endpoint - Skeptic filtering improvements Code quality: - Split 14 large files (>500 lines) into focused modules - All files now under 500-line limit per project guidelines Documentation: - Chaos testing guide, circuit breakers, observability docs - Phase 7 UAT documentation updates - Martin Kleppmann technical writer agent Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
130 lines
3.9 KiB
Markdown
130 lines
3.9 KiB
Markdown
# Latent: System Sequence Diagrams
|
|
|
|
These diagrams illustrate the data flow, conflict detection, and user interaction within the Latent platform.
|
|
|
|
## 1. Ingestion & Conflict Detection Loop (The Backend)
|
|
|
|
How the system continuously ingests diverse data sources and computes divergence.
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant S_FDA as Source: FDA/Regulatory (Tier 0)
|
|
participant S_Social as Source: Reddit/Twitter (Tier 5)
|
|
participant Ingest as Ingestion Service
|
|
participant StemeDB as StemeDB (Graph)
|
|
participant Conflict as Conflict Engine
|
|
participant Alert as Alert Service
|
|
|
|
Note over S_FDA, S_Social: Continuous Data Streams
|
|
|
|
par Ingest Regulatory
|
|
S_FDA->>Ingest: New Label Update (PDF/JSON)
|
|
Ingest->>Ingest: Extract Assertions (NLP)
|
|
Ingest->>StemeDB: Write Assertion (Tier 0)
|
|
and Ingest Social
|
|
S_Social->>Ingest: Raw Posts/Tweets
|
|
Ingest->>Ingest: Extract Claims & Cluster
|
|
Ingest->>StemeDB: Write Assertion (Tier 5)
|
|
end
|
|
|
|
loop Every 10 Minutes
|
|
Conflict->>StemeDB: Query(Subject="Semaglutide", Lens="Skeptic")
|
|
StemeDB-->>Conflict: Return { Official: "Transient", Latent: "Paralysis" }
|
|
|
|
Conflict->>Conflict: Calculate Divergence Score (0.0 - 1.0)
|
|
|
|
alt Score > Threshold (0.6)
|
|
Conflict->>Alert: Trigger "Divergence Alert"
|
|
Alert->>StemeDB: Log Alert Metadata
|
|
end
|
|
end
|
|
```
|
|
|
|
## 2. Analyst Investigation (The Frontend)
|
|
|
|
How a user interacts with the system to investigate a signal.
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
actor Analyst
|
|
participant UI as Latent Dashboard
|
|
participant API as Latent API
|
|
participant Lens as Episteme Lens Engine
|
|
participant DB as StemeDB Storage
|
|
|
|
Analyst->>UI: Select "Semaglutide"
|
|
UI->>API: GET /molecule/semaglutide/conflict
|
|
|
|
API->>Lens: Apply Lens: "Layered Consensus"
|
|
|
|
par Tier 0 Query
|
|
Lens->>DB: Fetch Assertions (SourceClass=0)
|
|
DB-->>Lens: [FDA Label, EMA Filing]
|
|
and Tier 5 Query
|
|
Lens->>DB: Fetch Assertions (SourceClass=5)
|
|
DB-->>Lens: [Reddit Cluster #8492]
|
|
end
|
|
|
|
Lens->>Lens: Compute Diff & Confidence
|
|
Lens-->>API: Result { ConflictScore: 0.88, Drivers: [...] }
|
|
API-->>UI: Render Conflict Heatmap
|
|
|
|
Analyst->>UI: Click "Timeline View"
|
|
UI->>API: GET /molecule/semaglutide/timeline?step=1mo
|
|
|
|
API->>DB: Time-Travel Query (AsOf: 2023-Q1, 2023-Q2...)
|
|
DB-->>API: Historical Snapshots
|
|
API-->>UI: Render "Lag Chart" (Social vs. Regulatory)
|
|
```
|
|
|
|
## 3. The "Alpha" Signal (Hedge Fund Use Case)
|
|
|
|
How an automated trading algorithm might use Latent.
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Algo as Trading Algo
|
|
participant API as Latent API
|
|
participant Broker as Brokerage
|
|
|
|
loop Daily Pre-Market
|
|
Algo->>API: GET /signals/top-divergence?sector=biotech
|
|
API-->>Algo: List [ { Ticker: "NVO", Divergence: 0.88, Signal: "Safety" } ]
|
|
|
|
Algo->>Algo: Check Portfolio Exposure
|
|
|
|
alt Divergence > 0.8 AND Sentiment == Negative
|
|
Algo->>Broker: Sell / Short NVO
|
|
Algo->>API: Log Action (for correlation)
|
|
end
|
|
end
|
|
```
|
|
|
|
## 4. Source Decay & Invalidation
|
|
|
|
How the system handles old data fading away vs. regulatory permanence.
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Clock as Scheduler
|
|
participant Decay as Decay Service
|
|
participant StemeDB as StemeDB
|
|
|
|
Clock->>Decay: Run Daily Decay
|
|
|
|
Decay->>StemeDB: Scan Active Assertions
|
|
|
|
loop For Each Assertion
|
|
alt Source Class == 0 (Regulatory)
|
|
Decay->>Decay: Do Nothing (Permanent)
|
|
else Source Class == 5 (Social)
|
|
Decay->>Decay: Calculate Age
|
|
alt Age > 30 Days
|
|
Decay->>StemeDB: Update Confidence (Reduce by 50%)
|
|
else Age > 90 Days
|
|
Decay->>StemeDB: Mark "Archived" (Remove from Hot Path)
|
|
end
|
|
end
|
|
end
|
|
```
|