Phase 1 delivers the complete durability and storage layer:
- WAL with crash recovery: Append-only journal with BLAKE3 checksums,
fsync guarantees, and proper seek-to-EOF on reopen
- Storage engine: sled-backed KVStore with scan_prefix for range queries
- Content-addressed storage: H:{hash}, V:{hash}, E:{hash} key patterns
- Ingestor: Background worker tailing WAL, writing to KV with 8-byte
aligned record headers for rkyv zero-copy deserialization
- Comprehensive tests: 31 tests covering crash recovery, round-trips,
and multi-cycle durability
New crates: stemedb-wal, stemedb-storage, stemedb-ingest
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
5.1 KiB
StemeDB Usage Guide (Go)
Philosophy: State as a Side Effect. Truth as a Consensus. Package:
github.com/orchard9/stemedb-go
1. The "Invisible" Integration (Job-Aware)
StemeDB is the "Flight Recorder" for your Agents. The Context carries the Job ID, binding every assertion to a persistent execution trace.
Setup with Job Binding
package main
import (
"context"
"github.com/orchard9/stemedb-go/steme"
)
func main() {
// 1. Bind to a Persistent Job (The "Interaction Object")
// This allows MCP/SSE clients to track progress in real-time.
ctx := steme.BindJob(context.Background(), "job-uuid-123")
// 2. Initialize Paradigm Context
ctx = steme.NewContext(ctx, steme.Config{
Project: "data-migration-v1",
AgentID: "worker-01",
// 3. Set Default Lens (Shared Reality)
// Agents default to Consensus to ensure they "hallucinate together"
DefaultLens: steme.LensConsensus,
})
// Updates status to "INDEXING" automatically via MCP
steme.UpdateStatus(ctx, steme.StatusIndexing)
// ... run app ...
}
2. Durable Execution (The Auto-Resume Pattern)
Stop writing checkpointing logic. Let the DB handle state continuity.
type FileProcessor struct {
steme.Task // Embed Steme tracking
}
func (p *FileProcessor) Process(ctx context.Context, files []string) {
// PRE-FLIGHT CHECK: Lens::Constraints
// Before doing ANY work, check for constraints on this domain.
// This prevents the "Context Drift" problem where agents forget rules.
constraints := steme.Query(ctx, steme.Query{
Subject: "FileProcessing",
Lens: "Constraints", // Returns 'must_use', 'forbidden', etc.
})
if err := p.validateAgainst(constraints); err != nil {
panic(err) // Fail fast if constraints violated
}
for _, file := range files {
// 1. Check Reality: Is this done in the current Epoch?
if steme.IsDone(ctx, file) {
continue // Auto-skip (Durable Execution)
}
// 2. Visual Anchoring (Handling "Entropy of the Wild Web")
// Anchors the assertion to a Perceptual Hash (pHash) of the source.
snapshot := captureScreenshot(file)
// 3. Assert with Proof
steme.Assert(ctx, file, "status", "PROCESSED",
steme.WithVisualProof(snapshot), // Stores pHash
steme.WithConfidence(0.95),
)
}
}
3. Deep Research Primitives (Consensus & Decay)
Agents must collaborate to build truth, not just overwrite each other.
The "Co-Signing" Pattern (Gap A)
If Agent B agrees with Agent A, it shouldn't write a duplicate assertion. It should Sign the existing one to boost its TrustRank.
func ValidateFact(ctx context.Context, factID string) {
// Agent B verifies the fact
if verify(factID) {
// Boosts the consensus score without adding a new node.
// Cryptographically signs the existing hash.
steme.Sign(ctx, factID, steme.Confidence(0.9))
} else {
// Create a Counter-Assertion (Conflict)
steme.Assert(ctx, factID, "status", "DISPUTED",
steme.WithReason("Verification Failed"),
)
}
}
The "Reinforce" Pattern (Gap C)
Truth decays over time (t_1/2). Agents must actively maintain knowledge.
func MaintenanceLoop(ctx context.Context) {
// 1. Find decaying facts using the 'Decay' Lens
facts := steme.Query(ctx, steme.Query{
Lens: "Decay", // Returns low-confidence / old facts
})
for _, fact := range facts {
// 2. Re-verify
if stillTrue(fact) {
// Resets the decay timer (Back-Propagation)
steme.Reinforce(ctx, fact.Hash)
}
}
}
4. Lenses: Defined vs. Custom
Agents use Defined Lenses for 99% of operations to ensure a shared reality.
Defined Lenses (The Standard Library)
Available via steme.Lens* constants.
- Consensus: The default. "What does the group believe?"
- Authority: "What do the experts believe?" (TrustRank weighted).
- Constraints: "What are the rules?" (Pre-flight checks).
- Decay: "What is fading?" (Maintenance targets).
Custom Lenses (The Exception)
Used only for specific simulations or "What If" scenarios.
// Advanced: Define a transient lens for a simulation
lens := steme.DefineLens(ctx, steme.LensDefinition{
Name: "Hypothetical-High-Trust",
Logic: `return candidates.filter(c => c.agent_id == "Agent-X")`, // WASM logic
})
// Query using the custom lens
result := steme.Query(ctx, steme.Query{
Subject: "Tesla",
Lens: lens.ID,
})
5. The "Paradigm Shift" (Bulk Invalidation)
Handling massive changes (e.g., deprecating an API version) is a single API call.
func PivotToV2(ctx context.Context) {
// Supersede the entire "V1" Epoch
err := steme.SupersedeEpoch(ctx, steme.SupersedeReq{
OldEpoch: "api-v1-semantics",
NewEpoch: "api-v2-semantics",
Type: steme.SupersessionInvalidate, // "V1 was wrong"
Reason: "Security Vulnerability",
})
}