jordan b3e8a9a058 feat: Multi-application expansion with chaos testing and community UI

Major additions:
- Community Next.js app (port 18187) for browsing claims with API docs
- stemedb-chaos crate: Fault injection, chaos testing, CRDT properties
- Latent ingestion system: Reddit/FDA ingesters with ADK-Go agents
- Disputed claims handling: Manual review workflows and validation
- Aphoria security scanner: New extractors (SQL injection, command
  injection, weak crypto, TLS version), policy-based ignores, UAT reports
- Docker infrastructure: Dockerfile, docker-compose.yml for full stack
- VulnBank demo: Intentionally vulnerable multi-language test corpus

SDK & API enhancements:
- Source registry handlers for tracking data provenance
- Metrics endpoint
- Skeptic filtering improvements

Code quality:
- Split 14 large files (>500 lines) into focused modules
- All files now under 500-line limit per project guidelines

Documentation:
- Chaos testing guide, circuit breakers, observability docs
- Phase 7 UAT documentation updates
- Martin Kleppmann technical writer agent

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

2026-02-04 01:24:14 -07:00

3.9 KiB

Raw Blame History

Latent: System Sequence Diagrams

These diagrams illustrate the data flow, conflict detection, and user interaction within the Latent platform.

1. Ingestion & Conflict Detection Loop (The Backend)

How the system continuously ingests diverse data sources and computes divergence.

sequenceDiagram
    participant S_FDA as Source: FDA/Regulatory (Tier 0)
    participant S_Social as Source: Reddit/Twitter (Tier 5)
    participant Ingest as Ingestion Service
    participant StemeDB as StemeDB (Graph)
    participant Conflict as Conflict Engine
    participant Alert as Alert Service

    Note over S_FDA, S_Social: Continuous Data Streams

    par Ingest Regulatory
        S_FDA->>Ingest: New Label Update (PDF/JSON)
        Ingest->>Ingest: Extract Assertions (NLP)
        Ingest->>StemeDB: Write Assertion (Tier 0)
    and Ingest Social
        S_Social->>Ingest: Raw Posts/Tweets
        Ingest->>Ingest: Extract Claims & Cluster
        Ingest->>StemeDB: Write Assertion (Tier 5)
    end

    loop Every 10 Minutes
        Conflict->>StemeDB: Query(Subject="Semaglutide", Lens="Skeptic")
        StemeDB-->>Conflict: Return { Official: "Transient", Latent: "Paralysis" }
        
        Conflict->>Conflict: Calculate Divergence Score (0.0 - 1.0)
        
        alt Score > Threshold (0.6)
            Conflict->>Alert: Trigger "Divergence Alert"
            Alert->>StemeDB: Log Alert Metadata
        end
    end

2. Analyst Investigation (The Frontend)

How a user interacts with the system to investigate a signal.

sequenceDiagram
    actor Analyst
    participant UI as Latent Dashboard
    participant API as Latent API
    participant Lens as Episteme Lens Engine
    participant DB as StemeDB Storage

    Analyst->>UI: Select "Semaglutide"
    UI->>API: GET /molecule/semaglutide/conflict
    
    API->>Lens: Apply Lens: "Layered Consensus"
    
    par Tier 0 Query
        Lens->>DB: Fetch Assertions (SourceClass=0)
        DB-->>Lens: [FDA Label, EMA Filing]
    and Tier 5 Query
        Lens->>DB: Fetch Assertions (SourceClass=5)
        DB-->>Lens: [Reddit Cluster #8492]
    end
    
    Lens->>Lens: Compute Diff & Confidence
    Lens-->>API: Result { ConflictScore: 0.88, Drivers: [...] }
    API-->>UI: Render Conflict Heatmap
    
    Analyst->>UI: Click "Timeline View"
    UI->>API: GET /molecule/semaglutide/timeline?step=1mo
    
    API->>DB: Time-Travel Query (AsOf: 2023-Q1, 2023-Q2...)
    DB-->>API: Historical Snapshots
    API-->>UI: Render "Lag Chart" (Social vs. Regulatory)

3. The "Alpha" Signal (Hedge Fund Use Case)

How an automated trading algorithm might use Latent.

sequenceDiagram
    participant Algo as Trading Algo
    participant API as Latent API
    participant Broker as Brokerage

    loop Daily Pre-Market
        Algo->>API: GET /signals/top-divergence?sector=biotech
        API-->>Algo: List [ { Ticker: "NVO", Divergence: 0.88, Signal: "Safety" } ]
        
        Algo->>Algo: Check Portfolio Exposure
        
        alt Divergence > 0.8 AND Sentiment == Negative
            Algo->>Broker: Sell / Short NVO
            Algo->>API: Log Action (for correlation)
        end
    end

4. Source Decay & Invalidation

How the system handles old data fading away vs. regulatory permanence.

sequenceDiagram
    participant Clock as Scheduler
    participant Decay as Decay Service
    participant StemeDB as StemeDB

    Clock->>Decay: Run Daily Decay
    
    Decay->>StemeDB: Scan Active Assertions
    
    loop For Each Assertion
        alt Source Class == 0 (Regulatory)
            Decay->>Decay: Do Nothing (Permanent)
        else Source Class == 5 (Social)
            Decay->>Decay: Calculate Age
            alt Age > 30 Days
                Decay->>StemeDB: Update Confidence (Reduce by 50%)
            else Age > 90 Days
                Decay->>StemeDB: Mark "Archived" (Remove from Hot Path)
            end
        end
    end

3.9 KiB Raw Blame History