Phase 1 delivers the complete durability and storage layer:
- WAL with crash recovery: Append-only journal with BLAKE3 checksums,
fsync guarantees, and proper seek-to-EOF on reopen
- Storage engine: sled-backed KVStore with scan_prefix for range queries
- Content-addressed storage: H:{hash}, V:{hash}, E:{hash} key patterns
- Ingestor: Background worker tailing WAL, writing to KV with 8-byte
aligned record headers for rkyv zero-copy deserialization
- Comprehensive tests: 31 tests covering crash recovery, round-trips,
and multi-cycle durability
New crates: stemedb-wal, stemedb-storage, stemedb-ingest
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
175 lines
5.1 KiB
Markdown
175 lines
5.1 KiB
Markdown
# StemeDB Usage Guide (Go)
|
|
|
|
> **Philosophy:** State as a Side Effect. Truth as a Consensus.
|
|
> **Package:** `github.com/orchard9/stemedb-go`
|
|
|
|
## 1. The "Invisible" Integration (Job-Aware)
|
|
|
|
StemeDB is the "Flight Recorder" for your Agents. The Context carries the **Job ID**, binding every assertion to a persistent execution trace.
|
|
|
|
### Setup with Job Binding
|
|
|
|
```go
|
|
package main
|
|
|
|
import (
|
|
"context"
|
|
"github.com/orchard9/stemedb-go/steme"
|
|
)
|
|
|
|
func main() {
|
|
// 1. Bind to a Persistent Job (The "Interaction Object")
|
|
// This allows MCP/SSE clients to track progress in real-time.
|
|
ctx := steme.BindJob(context.Background(), "job-uuid-123")
|
|
|
|
// 2. Initialize Paradigm Context
|
|
ctx = steme.NewContext(ctx, steme.Config{
|
|
Project: "data-migration-v1",
|
|
AgentID: "worker-01",
|
|
// 3. Set Default Lens (Shared Reality)
|
|
// Agents default to Consensus to ensure they "hallucinate together"
|
|
DefaultLens: steme.LensConsensus,
|
|
})
|
|
|
|
// Updates status to "INDEXING" automatically via MCP
|
|
steme.UpdateStatus(ctx, steme.StatusIndexing)
|
|
|
|
// ... run app ...
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 2. Durable Execution (The Auto-Resume Pattern)
|
|
|
|
Stop writing checkpointing logic. Let the DB handle state continuity.
|
|
|
|
```go
|
|
type FileProcessor struct {
|
|
steme.Task // Embed Steme tracking
|
|
}
|
|
|
|
func (p *FileProcessor) Process(ctx context.Context, files []string) {
|
|
// PRE-FLIGHT CHECK: Lens::Constraints
|
|
// Before doing ANY work, check for constraints on this domain.
|
|
// This prevents the "Context Drift" problem where agents forget rules.
|
|
constraints := steme.Query(ctx, steme.Query{
|
|
Subject: "FileProcessing",
|
|
Lens: "Constraints", // Returns 'must_use', 'forbidden', etc.
|
|
})
|
|
|
|
if err := p.validateAgainst(constraints); err != nil {
|
|
panic(err) // Fail fast if constraints violated
|
|
}
|
|
|
|
for _, file := range files {
|
|
// 1. Check Reality: Is this done in the current Epoch?
|
|
if steme.IsDone(ctx, file) {
|
|
continue // Auto-skip (Durable Execution)
|
|
}
|
|
|
|
// 2. Visual Anchoring (Handling "Entropy of the Wild Web")
|
|
// Anchors the assertion to a Perceptual Hash (pHash) of the source.
|
|
snapshot := captureScreenshot(file)
|
|
|
|
// 3. Assert with Proof
|
|
steme.Assert(ctx, file, "status", "PROCESSED",
|
|
steme.WithVisualProof(snapshot), // Stores pHash
|
|
steme.WithConfidence(0.95),
|
|
)
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 3. Deep Research Primitives (Consensus & Decay)
|
|
|
|
Agents must collaborate to build truth, not just overwrite each other.
|
|
|
|
### The "Co-Signing" Pattern (Gap A)
|
|
If Agent B agrees with Agent A, it shouldn't write a duplicate assertion. It should **Sign** the existing one to boost its TrustRank.
|
|
|
|
```go
|
|
func ValidateFact(ctx context.Context, factID string) {
|
|
// Agent B verifies the fact
|
|
if verify(factID) {
|
|
// Boosts the consensus score without adding a new node.
|
|
// Cryptographically signs the existing hash.
|
|
steme.Sign(ctx, factID, steme.Confidence(0.9))
|
|
} else {
|
|
// Create a Counter-Assertion (Conflict)
|
|
steme.Assert(ctx, factID, "status", "DISPUTED",
|
|
steme.WithReason("Verification Failed"),
|
|
)
|
|
}
|
|
}
|
|
```
|
|
|
|
### The "Reinforce" Pattern (Gap C)
|
|
Truth decays over time (`t_1/2`). Agents must actively maintain knowledge.
|
|
|
|
```go
|
|
func MaintenanceLoop(ctx context.Context) {
|
|
// 1. Find decaying facts using the 'Decay' Lens
|
|
facts := steme.Query(ctx, steme.Query{
|
|
Lens: "Decay", // Returns low-confidence / old facts
|
|
})
|
|
|
|
for _, fact := range facts {
|
|
// 2. Re-verify
|
|
if stillTrue(fact) {
|
|
// Resets the decay timer (Back-Propagation)
|
|
steme.Reinforce(ctx, fact.Hash)
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 4. Lenses: Defined vs. Custom
|
|
|
|
Agents use **Defined Lenses** for 99% of operations to ensure a shared reality.
|
|
|
|
### Defined Lenses (The Standard Library)
|
|
Available via `steme.Lens*` constants.
|
|
* **Consensus**: The default. "What does the group believe?"
|
|
* **Authority**: "What do the experts believe?" (TrustRank weighted).
|
|
* **Constraints**: "What are the rules?" (Pre-flight checks).
|
|
* **Decay**: "What is fading?" (Maintenance targets).
|
|
|
|
### Custom Lenses (The Exception)
|
|
Used only for specific simulations or "What If" scenarios.
|
|
|
|
```go
|
|
// Advanced: Define a transient lens for a simulation
|
|
lens := steme.DefineLens(ctx, steme.LensDefinition{
|
|
Name: "Hypothetical-High-Trust",
|
|
Logic: `return candidates.filter(c => c.agent_id == "Agent-X")`, // WASM logic
|
|
})
|
|
|
|
// Query using the custom lens
|
|
result := steme.Query(ctx, steme.Query{
|
|
Subject: "Tesla",
|
|
Lens: lens.ID,
|
|
})
|
|
```
|
|
|
|
---
|
|
|
|
## 5. The "Paradigm Shift" (Bulk Invalidation)
|
|
|
|
Handling massive changes (e.g., deprecating an API version) is a single API call.
|
|
|
|
```go
|
|
func PivotToV2(ctx context.Context) {
|
|
// Supersede the entire "V1" Epoch
|
|
err := steme.SupersedeEpoch(ctx, steme.SupersedeReq{
|
|
OldEpoch: "api-v1-semantics",
|
|
NewEpoch: "api-v2-semantics",
|
|
Type: steme.SupersessionInvalidate, // "V1 was wrong"
|
|
Reason: "Security Vulnerability",
|
|
})
|
|
}
|
|
``` |