- Add `content: Option<String>` to SourceRecord with rkyv schema evolution (LegacySourceRecord compat deserializer for backward compatibility) - Add MAX_SOURCE_CONTENT_LEN (1MB) limit with API validation - Strip content from list responses, include in single-source GET - Update Go SDK RegisterSourceRequest with Content field - FCM pipeline extracts PDF text via pdftotext and passes to registration - Dashboard impact panel fetches and displays source content with expand/collapse - Add feed endpoint, dashboard feed panel, and signed assertion support - Update data-structures.md, API docs, and storage docs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
330 lines
9.8 KiB
Markdown
330 lines
9.8 KiB
Markdown
# SDK - Go Client Libraries
|
|
|
|
**Last Updated:** 2026-02-19
|
|
**Confidence:** High
|
|
|
|
## Summary
|
|
|
|
Official Go SDK for Episteme with two layers: a low-level HTTP client (`steme`) and an ADK-Go integration layer (`adk`) for building AI agents. Both libraries handle Ed25519 signing, content-addressing, and fluent assertion building. The ADK layer exposes Episteme as Google ADK-Go tools with pre-flight constraint checking and audit callbacks.
|
|
|
|
**Key Facts:**
|
|
- Languages: Go (initial), Python/TypeScript planned
|
|
- Location: `sdk/go/steme/` (core client), `sdk/go/adk/` (ADK integration)
|
|
- Signing: Ed25519 with BLAKE3 content-addressing
|
|
- Transport: HTTP/JSON (matches `stemedb-api`)
|
|
- ADK Tools: Query, Assert, ConstraintCheck, Trace, Supersede
|
|
- Test Coverage: 26 tests passing (12 core + 14 ADK)
|
|
|
|
**File Pointers:**
|
|
- `sdk/go/steme/client.go` - HTTP client with auto-signing
|
|
- `sdk/go/steme/assertion.go` - Fluent builder for assertions
|
|
- `sdk/go/steme/signer.go` - Ed25519 signature generation
|
|
- `sdk/go/adk/tools.go` - ADK tool definitions
|
|
- `sdk/go/adk/callbacks.go` - Pre-flight checks and audit hooks
|
|
- `sdk/go/adk/config.go` - Agent configurations (Lead, Research, Implementation, etc.)
|
|
|
|
## Core SDK (`steme`)
|
|
|
|
### Installation
|
|
|
|
```go
|
|
import "github.com/orchard9/stemedb/sdk/go/steme"
|
|
```
|
|
|
|
### Client Creation
|
|
|
|
```go
|
|
// Create with Ed25519 keypair
|
|
privateKey := ed25519.NewKeyFromSeed(seed)
|
|
client := steme.NewClient("https://episteme.example.com", privateKey)
|
|
|
|
// Or use environment-based config
|
|
client := steme.NewClientFromEnv() // STEME_URL, STEME_PRIVATE_KEY
|
|
```
|
|
|
|
### Fluent Assertion Builder
|
|
|
|
```go
|
|
assertion, err := steme.NewAssertion().
|
|
Subject("auth/jwt").
|
|
Predicate("signing_algorithm").
|
|
StringObject("RS256").
|
|
Confidence(0.95).
|
|
SourceHash(evidenceHash).
|
|
Lifecycle("approved").
|
|
Build()
|
|
|
|
// Client auto-signs on submit
|
|
hash, err := client.Assert(ctx, assertion)
|
|
```
|
|
|
|
### Query API
|
|
|
|
```go
|
|
result, err := client.Query(ctx, steme.QueryRequest{
|
|
Subject: "auth/jwt",
|
|
Predicate: "signing_algorithm",
|
|
Lens: "authority",
|
|
Lifecycle: "approved",
|
|
MinConfidence: 0.8,
|
|
AsOf: time.Now().Add(-24 * time.Hour), // Time-travel
|
|
})
|
|
|
|
fmt.Printf("Value: %v (confidence: %.2f)\n", result.Value, result.Confidence)
|
|
```
|
|
|
|
### Voting
|
|
|
|
```go
|
|
err := client.Vote(ctx, steme.VoteRequest{
|
|
AssertionHash: targetHash,
|
|
Upvote: true,
|
|
AgentID: myAgentID,
|
|
VoteHash: voteContentHash,
|
|
})
|
|
```
|
|
|
|
## ADK Integration (`adk`)
|
|
|
|
### Installation
|
|
|
|
```go
|
|
import (
|
|
"github.com/orchard9/stemedb/sdk/go/adk"
|
|
"google.golang.org/adk/llmagent"
|
|
)
|
|
```
|
|
|
|
### Agent Configuration
|
|
|
|
Pre-configured agent personas with appropriate tool access and callbacks:
|
|
|
|
```go
|
|
// Implementation Agent - Uses approved patterns only
|
|
config := adk.ImplementationAgentConfig(stemeClient, model)
|
|
agent, err := llmagent.New(config)
|
|
|
|
// Research Agent - Stores conflicting information without flattening
|
|
config := adk.ResearchAgentConfig(stemeClient, model)
|
|
agent, err := llmagent.New(config)
|
|
|
|
// Lead Agent - Fast queries with confidence thresholds
|
|
config := adk.LeadAgentConfig(stemeClient, model)
|
|
agent, err := llmagent.New(config)
|
|
|
|
// Supervisor Agent - Time-travel and corrections
|
|
config := adk.SupervisorAgentConfig(stemeClient, model)
|
|
agent, err := llmagent.New(config)
|
|
|
|
// On-Call Agent - Traces and debugging
|
|
config := adk.OnCallAgentConfig(stemeClient, model)
|
|
agent, err := llmagent.New(config)
|
|
```
|
|
|
|
### Available Tools
|
|
|
|
All tools are ADK-compatible with structured input/output and OpenAPI schema generation:
|
|
|
|
**Query Tool**
|
|
```go
|
|
type QueryInput struct {
|
|
Subject string `json:"subject"`
|
|
Predicate string `json:"predicate"`
|
|
Lens string `json:"lens,omitempty"`
|
|
Lifecycle string `json:"lifecycle,omitempty"`
|
|
MinConfidence float32 `json:"min_confidence,omitempty"`
|
|
AsOf string `json:"as_of,omitempty"`
|
|
}
|
|
```
|
|
|
|
**Assert Tool**
|
|
```go
|
|
type AssertInput struct {
|
|
Subject string `json:"subject"`
|
|
Predicate string `json:"predicate"`
|
|
Object interface{} `json:"object"`
|
|
SourceHash string `json:"source_hash"`
|
|
Confidence float32 `json:"confidence"`
|
|
Lifecycle string `json:"lifecycle,omitempty"`
|
|
ParentHash string `json:"parent_hash,omitempty"`
|
|
}
|
|
```
|
|
|
|
**ConstraintCheck Tool**
|
|
```go
|
|
type ConstraintCheckInput struct {
|
|
Context string `json:"context"` // e.g., "python_http", "auth_jwt"
|
|
}
|
|
```
|
|
|
|
**Trace Tool** (SRE debugging)
|
|
```go
|
|
type TraceInput struct {
|
|
AgentID string `json:"agent_id"`
|
|
From string `json:"from"` // ISO8601 or relative like "-6h"
|
|
Subject string `json:"subject,omitempty"` // Filter pattern
|
|
}
|
|
```
|
|
|
|
**Supersede Tool** (corrections)
|
|
```go
|
|
type SupersedeInput struct {
|
|
Hash string `json:"hash"` // Assertion to correct
|
|
Reason string `json:"reason"` // Why it's being invalidated
|
|
Type string `json:"type"` // "Invalidate" or "Update"
|
|
}
|
|
```
|
|
|
|
### Callbacks
|
|
|
|
Three callback hooks for safety and observability:
|
|
|
|
**ConstraintEnforcement** (BeforeToolCallback)
|
|
```go
|
|
// Blocks tool calls that violate organizational constraints
|
|
// Example: Prevents using SHA1 if auth/jwt has "forbidden: sha1"
|
|
BeforeToolCallback: adk.ConstraintEnforcementCallback(client)
|
|
```
|
|
|
|
**AuditLogging** (AfterModelCallback)
|
|
```go
|
|
// Logs agent decisions for debugging and replay
|
|
AfterModelCallback: adk.AuditLoggingCallback(client)
|
|
```
|
|
|
|
**ConfidenceEscalation** (AfterToolCallback)
|
|
```go
|
|
// Escalates to human when confidence drops below threshold
|
|
// Triggers Gardener on tool failures for TrustRank back-propagation
|
|
AfterToolCallback: adk.ConfidenceEscalationCallback(client, 0.7)
|
|
```
|
|
|
|
### Custom Agent Example
|
|
|
|
```go
|
|
import (
|
|
"github.com/orchard9/stemedb/sdk/go/adk"
|
|
"google.golang.org/adk/llmagent"
|
|
"google.golang.org/adk/tool"
|
|
)
|
|
|
|
func main() {
|
|
stemeClient := steme.NewClientFromEnv()
|
|
model := llm.NewClient()
|
|
|
|
// Build ADK tools
|
|
queryTool, _ := adk.NewQueryTool(stemeClient)
|
|
assertTool, _ := adk.NewAssertTool(stemeClient)
|
|
constraintTool, _ := adk.NewConstraintCheckTool(stemeClient)
|
|
|
|
// Create agent with callbacks
|
|
agent, err := llmagent.New(llmagent.Config{
|
|
Name: "custom_agent",
|
|
Model: model,
|
|
Tools: []tool.Tool{queryTool, assertTool, constraintTool},
|
|
|
|
// Pre-flight constraint checking
|
|
BeforeToolCallback: adk.ConstraintEnforcementCallback(stemeClient),
|
|
|
|
// Audit all decisions
|
|
AfterModelCallback: adk.AuditLoggingCallback(stemeClient),
|
|
|
|
// Learn from failures
|
|
AfterToolCallback: adk.ConfidenceEscalationCallback(stemeClient, 0.8),
|
|
})
|
|
|
|
// Run agent
|
|
ctx := context.Background()
|
|
response, err := agent.Generate(ctx, llmagent.Input{
|
|
Prompt: "Check if we have an approved JWT signing algorithm",
|
|
})
|
|
}
|
|
```
|
|
|
|
## E2E Test Coverage
|
|
|
|
The SDK implementation is validated by comprehensive integration tests:
|
|
|
|
**File:** `crates/stemedb-api/tests/e2e_flow_test.rs`
|
|
|
|
**Tests:**
|
|
1. `test_basic_write_materialize_query` - Write → MV → Read flow
|
|
2. `test_write_multiple_query_recency` - Recency lens with multiple assertions
|
|
3. `test_vote_and_query_consensus` - Voting + Consensus lens
|
|
4. `test_event_driven_materialization` - MV updates on write events
|
|
|
|
**Coverage:**
|
|
- Real Ed25519 signature generation and verification
|
|
- Event-driven materialization (WAL → MV update)
|
|
- All lens types (Recency, Consensus, Authority)
|
|
- Time-travel queries
|
|
- Vote aggregation with TrustRank
|
|
|
|
## Type Hierarchy
|
|
|
|
```
|
|
Go SDK (serde JSON) Rust API (serde JSON) Rust Core (rkyv)
|
|
| | |
|
|
Assertion struct dto::AssertionRequest types::Assertion
|
|
| | |
|
|
+-- Subject: string +-- subject: String +-- subject: EntityId
|
|
+-- Predicate: string +-- predicate: String +-- predicate: RelationId
|
|
+-- Object: interface{} +-- object: Value +-- object: ObjectValue
|
|
+-- Confidence: float32 +-- confidence: f32 +-- confidence: f32
|
|
+-- SourceHash: string +-- source_hash: Hash +-- source_hash: Hash
|
|
+-- Lifecycle: string +-- lifecycle: String +-- lifecycle: LifecycleStage
|
|
```
|
|
|
|
## Design Principles
|
|
|
|
**Content-Addressed Identity**
|
|
- Assertion hash = BLAKE3(canonical JSON serialization)
|
|
- Same content = same hash across all SDKs
|
|
- Idempotent writes (re-asserting produces same hash)
|
|
|
|
**Multi-Signature Support**
|
|
- Client signs entire assertion with Ed25519
|
|
- API verifies signature before writing to WAL
|
|
- Multiple agents can co-sign the same assertion (multi-sig)
|
|
|
|
**Fluent Builder Pattern**
|
|
- Type-safe construction with compile-time validation
|
|
- Prevents partial/invalid assertions
|
|
- Generates content hash automatically on Build()
|
|
|
|
**Zero Dependencies on Rust**
|
|
- Pure Go implementation with standard library + ADK
|
|
- JSON over HTTP (no gRPC/protobuf yet)
|
|
- Works with any Episteme deployment
|
|
|
|
## Error Handling
|
|
|
|
```go
|
|
// SDK errors implement error interface
|
|
_, err := client.Assert(ctx, assertion)
|
|
if err != nil {
|
|
var apiErr *steme.APIError
|
|
if errors.As(err, &apiErr) {
|
|
switch apiErr.Code {
|
|
case steme.ErrInvalidSignature:
|
|
// Re-sign with correct key
|
|
case steme.ErrDuplicateHash:
|
|
// Already exists (idempotent - not an error)
|
|
case steme.ErrConstraintViolation:
|
|
// Check constraints with ConstraintCheck tool
|
|
default:
|
|
// Log and retry with backoff
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
## Related Topics
|
|
|
|
- [API Surface](./api.md) - HTTP endpoints and OpenAPI schema
|
|
- [Assertion](./assertion.md) - Core data structure
|
|
- [Lens](./lens.md) - Read-time resolution strategies
|
|
- [ADK Integration Pattern](../patterns/adk-integration.md) - Tool and callback details
|
|
- [Query Audit](../features/query-audit.md) - Trace agent decisions
|
|
- [TrustRank](../features/trust-rank.md) - Agent reputation and Gardener
|