stemedb/usage.md
jordan 3cfaa1e1d3 feat: Complete Phase 1 (The Spine) - storage foundation
Phase 1 delivers the complete durability and storage layer:

- WAL with crash recovery: Append-only journal with BLAKE3 checksums,
  fsync guarantees, and proper seek-to-EOF on reopen
- Storage engine: sled-backed KVStore with scan_prefix for range queries
- Content-addressed storage: H:{hash}, V:{hash}, E:{hash} key patterns
- Ingestor: Background worker tailing WAL, writing to KV with 8-byte
  aligned record headers for rkyv zero-copy deserialization
- Comprehensive tests: 31 tests covering crash recovery, round-trips,
  and multi-cycle durability

New crates: stemedb-wal, stemedb-storage, stemedb-ingest

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 14:15:34 -07:00

5.1 KiB

StemeDB Usage Guide (Go)

Philosophy: State as a Side Effect. Truth as a Consensus. Package: github.com/orchard9/stemedb-go

1. The "Invisible" Integration (Job-Aware)

StemeDB is the "Flight Recorder" for your Agents. The Context carries the Job ID, binding every assertion to a persistent execution trace.

Setup with Job Binding

package main

import (
    "context"
    "github.com/orchard9/stemedb-go/steme"
)

func main() {
    // 1. Bind to a Persistent Job (The "Interaction Object")
    // This allows MCP/SSE clients to track progress in real-time.
    ctx := steme.BindJob(context.Background(), "job-uuid-123")
    
    // 2. Initialize Paradigm Context
    ctx = steme.NewContext(ctx, steme.Config{
        Project: "data-migration-v1",
        AgentID: "worker-01",
        // 3. Set Default Lens (Shared Reality)
        // Agents default to Consensus to ensure they "hallucinate together"
        DefaultLens: steme.LensConsensus,
    })
    
    // Updates status to "INDEXING" automatically via MCP
    steme.UpdateStatus(ctx, steme.StatusIndexing)
    
    // ... run app ...
}

2. Durable Execution (The Auto-Resume Pattern)

Stop writing checkpointing logic. Let the DB handle state continuity.

type FileProcessor struct {
    steme.Task // Embed Steme tracking
}

func (p *FileProcessor) Process(ctx context.Context, files []string) {
    // PRE-FLIGHT CHECK: Lens::Constraints
    // Before doing ANY work, check for constraints on this domain.
    // This prevents the "Context Drift" problem where agents forget rules.
    constraints := steme.Query(ctx, steme.Query{
        Subject: "FileProcessing",
        Lens:    "Constraints", // Returns 'must_use', 'forbidden', etc.
    })
    
    if err := p.validateAgainst(constraints); err != nil {
        panic(err) // Fail fast if constraints violated
    }
    
    for _, file := range files {
        // 1. Check Reality: Is this done in the current Epoch?
        if steme.IsDone(ctx, file) {
            continue // Auto-skip (Durable Execution)
        }

        // 2. Visual Anchoring (Handling "Entropy of the Wild Web")
        // Anchors the assertion to a Perceptual Hash (pHash) of the source.
        snapshot := captureScreenshot(file)
        
        // 3. Assert with Proof
        steme.Assert(ctx, file, "status", "PROCESSED",
            steme.WithVisualProof(snapshot), // Stores pHash
            steme.WithConfidence(0.95),
        )
    }
}

3. Deep Research Primitives (Consensus & Decay)

Agents must collaborate to build truth, not just overwrite each other.

The "Co-Signing" Pattern (Gap A)

If Agent B agrees with Agent A, it shouldn't write a duplicate assertion. It should Sign the existing one to boost its TrustRank.

func ValidateFact(ctx context.Context, factID string) {
    // Agent B verifies the fact
    if verify(factID) {
        // Boosts the consensus score without adding a new node.
        // Cryptographically signs the existing hash.
        steme.Sign(ctx, factID, steme.Confidence(0.9)) 
    } else {
        // Create a Counter-Assertion (Conflict)
        steme.Assert(ctx, factID, "status", "DISPUTED", 
            steme.WithReason("Verification Failed"),
        )
    }
}

The "Reinforce" Pattern (Gap C)

Truth decays over time (t_1/2). Agents must actively maintain knowledge.

func MaintenanceLoop(ctx context.Context) {
    // 1. Find decaying facts using the 'Decay' Lens
    facts := steme.Query(ctx, steme.Query{
        Lens: "Decay", // Returns low-confidence / old facts
    })

    for _, fact := range facts {
        // 2. Re-verify
        if stillTrue(fact) {
            // Resets the decay timer (Back-Propagation)
            steme.Reinforce(ctx, fact.Hash)
        }
    }
}

4. Lenses: Defined vs. Custom

Agents use Defined Lenses for 99% of operations to ensure a shared reality.

Defined Lenses (The Standard Library)

Available via steme.Lens* constants.

  • Consensus: The default. "What does the group believe?"
  • Authority: "What do the experts believe?" (TrustRank weighted).
  • Constraints: "What are the rules?" (Pre-flight checks).
  • Decay: "What is fading?" (Maintenance targets).

Custom Lenses (The Exception)

Used only for specific simulations or "What If" scenarios.

// Advanced: Define a transient lens for a simulation
lens := steme.DefineLens(ctx, steme.LensDefinition{
    Name: "Hypothetical-High-Trust",
    Logic: `return candidates.filter(c => c.agent_id == "Agent-X")`, // WASM logic
})

// Query using the custom lens
result := steme.Query(ctx, steme.Query{
    Subject: "Tesla",
    Lens:    lens.ID,
})

5. The "Paradigm Shift" (Bulk Invalidation)

Handling massive changes (e.g., deprecating an API version) is a single API call.

func PivotToV2(ctx context.Context) {
    // Supersede the entire "V1" Epoch
    err := steme.SupersedeEpoch(ctx, steme.SupersedeReq{
        OldEpoch: "api-v1-semantics",
        NewEpoch: "api-v2-semantics",
        Type:     steme.SupersessionInvalidate, // "V1 was wrong"
        Reason:   "Security Vulnerability",
    })
}