stemedb/ai-lookup/services/sdk.md
jordan ad07a75d0a feat: add source content to source registry, signed assertions, feed endpoint, dashboard enhancements
- Add `content: Option<String>` to SourceRecord with rkyv schema evolution
  (LegacySourceRecord compat deserializer for backward compatibility)
- Add MAX_SOURCE_CONTENT_LEN (1MB) limit with API validation
- Strip content from list responses, include in single-source GET
- Update Go SDK RegisterSourceRequest with Content field
- FCM pipeline extracts PDF text via pdftotext and passes to registration
- Dashboard impact panel fetches and displays source content with expand/collapse
- Add feed endpoint, dashboard feed panel, and signed assertion support
- Update data-structures.md, API docs, and storage docs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 21:54:27 -07:00

330 lines
9.8 KiB
Markdown

# SDK - Go Client Libraries
**Last Updated:** 2026-02-19
**Confidence:** High
## Summary
Official Go SDK for Episteme with two layers: a low-level HTTP client (`steme`) and an ADK-Go integration layer (`adk`) for building AI agents. Both libraries handle Ed25519 signing, content-addressing, and fluent assertion building. The ADK layer exposes Episteme as Google ADK-Go tools with pre-flight constraint checking and audit callbacks.
**Key Facts:**
- Languages: Go (initial), Python/TypeScript planned
- Location: `sdk/go/steme/` (core client), `sdk/go/adk/` (ADK integration)
- Signing: Ed25519 with BLAKE3 content-addressing
- Transport: HTTP/JSON (matches `stemedb-api`)
- ADK Tools: Query, Assert, ConstraintCheck, Trace, Supersede
- Test Coverage: 26 tests passing (12 core + 14 ADK)
**File Pointers:**
- `sdk/go/steme/client.go` - HTTP client with auto-signing
- `sdk/go/steme/assertion.go` - Fluent builder for assertions
- `sdk/go/steme/signer.go` - Ed25519 signature generation
- `sdk/go/adk/tools.go` - ADK tool definitions
- `sdk/go/adk/callbacks.go` - Pre-flight checks and audit hooks
- `sdk/go/adk/config.go` - Agent configurations (Lead, Research, Implementation, etc.)
## Core SDK (`steme`)
### Installation
```go
import "github.com/orchard9/stemedb/sdk/go/steme"
```
### Client Creation
```go
// Create with Ed25519 keypair
privateKey := ed25519.NewKeyFromSeed(seed)
client := steme.NewClient("https://episteme.example.com", privateKey)
// Or use environment-based config
client := steme.NewClientFromEnv() // STEME_URL, STEME_PRIVATE_KEY
```
### Fluent Assertion Builder
```go
assertion, err := steme.NewAssertion().
Subject("auth/jwt").
Predicate("signing_algorithm").
StringObject("RS256").
Confidence(0.95).
SourceHash(evidenceHash).
Lifecycle("approved").
Build()
// Client auto-signs on submit
hash, err := client.Assert(ctx, assertion)
```
### Query API
```go
result, err := client.Query(ctx, steme.QueryRequest{
Subject: "auth/jwt",
Predicate: "signing_algorithm",
Lens: "authority",
Lifecycle: "approved",
MinConfidence: 0.8,
AsOf: time.Now().Add(-24 * time.Hour), // Time-travel
})
fmt.Printf("Value: %v (confidence: %.2f)\n", result.Value, result.Confidence)
```
### Voting
```go
err := client.Vote(ctx, steme.VoteRequest{
AssertionHash: targetHash,
Upvote: true,
AgentID: myAgentID,
VoteHash: voteContentHash,
})
```
## ADK Integration (`adk`)
### Installation
```go
import (
"github.com/orchard9/stemedb/sdk/go/adk"
"google.golang.org/adk/llmagent"
)
```
### Agent Configuration
Pre-configured agent personas with appropriate tool access and callbacks:
```go
// Implementation Agent - Uses approved patterns only
config := adk.ImplementationAgentConfig(stemeClient, model)
agent, err := llmagent.New(config)
// Research Agent - Stores conflicting information without flattening
config := adk.ResearchAgentConfig(stemeClient, model)
agent, err := llmagent.New(config)
// Lead Agent - Fast queries with confidence thresholds
config := adk.LeadAgentConfig(stemeClient, model)
agent, err := llmagent.New(config)
// Supervisor Agent - Time-travel and corrections
config := adk.SupervisorAgentConfig(stemeClient, model)
agent, err := llmagent.New(config)
// On-Call Agent - Traces and debugging
config := adk.OnCallAgentConfig(stemeClient, model)
agent, err := llmagent.New(config)
```
### Available Tools
All tools are ADK-compatible with structured input/output and OpenAPI schema generation:
**Query Tool**
```go
type QueryInput struct {
Subject string `json:"subject"`
Predicate string `json:"predicate"`
Lens string `json:"lens,omitempty"`
Lifecycle string `json:"lifecycle,omitempty"`
MinConfidence float32 `json:"min_confidence,omitempty"`
AsOf string `json:"as_of,omitempty"`
}
```
**Assert Tool**
```go
type AssertInput struct {
Subject string `json:"subject"`
Predicate string `json:"predicate"`
Object interface{} `json:"object"`
SourceHash string `json:"source_hash"`
Confidence float32 `json:"confidence"`
Lifecycle string `json:"lifecycle,omitempty"`
ParentHash string `json:"parent_hash,omitempty"`
}
```
**ConstraintCheck Tool**
```go
type ConstraintCheckInput struct {
Context string `json:"context"` // e.g., "python_http", "auth_jwt"
}
```
**Trace Tool** (SRE debugging)
```go
type TraceInput struct {
AgentID string `json:"agent_id"`
From string `json:"from"` // ISO8601 or relative like "-6h"
Subject string `json:"subject,omitempty"` // Filter pattern
}
```
**Supersede Tool** (corrections)
```go
type SupersedeInput struct {
Hash string `json:"hash"` // Assertion to correct
Reason string `json:"reason"` // Why it's being invalidated
Type string `json:"type"` // "Invalidate" or "Update"
}
```
### Callbacks
Three callback hooks for safety and observability:
**ConstraintEnforcement** (BeforeToolCallback)
```go
// Blocks tool calls that violate organizational constraints
// Example: Prevents using SHA1 if auth/jwt has "forbidden: sha1"
BeforeToolCallback: adk.ConstraintEnforcementCallback(client)
```
**AuditLogging** (AfterModelCallback)
```go
// Logs agent decisions for debugging and replay
AfterModelCallback: adk.AuditLoggingCallback(client)
```
**ConfidenceEscalation** (AfterToolCallback)
```go
// Escalates to human when confidence drops below threshold
// Triggers Gardener on tool failures for TrustRank back-propagation
AfterToolCallback: adk.ConfidenceEscalationCallback(client, 0.7)
```
### Custom Agent Example
```go
import (
"github.com/orchard9/stemedb/sdk/go/adk"
"google.golang.org/adk/llmagent"
"google.golang.org/adk/tool"
)
func main() {
stemeClient := steme.NewClientFromEnv()
model := llm.NewClient()
// Build ADK tools
queryTool, _ := adk.NewQueryTool(stemeClient)
assertTool, _ := adk.NewAssertTool(stemeClient)
constraintTool, _ := adk.NewConstraintCheckTool(stemeClient)
// Create agent with callbacks
agent, err := llmagent.New(llmagent.Config{
Name: "custom_agent",
Model: model,
Tools: []tool.Tool{queryTool, assertTool, constraintTool},
// Pre-flight constraint checking
BeforeToolCallback: adk.ConstraintEnforcementCallback(stemeClient),
// Audit all decisions
AfterModelCallback: adk.AuditLoggingCallback(stemeClient),
// Learn from failures
AfterToolCallback: adk.ConfidenceEscalationCallback(stemeClient, 0.8),
})
// Run agent
ctx := context.Background()
response, err := agent.Generate(ctx, llmagent.Input{
Prompt: "Check if we have an approved JWT signing algorithm",
})
}
```
## E2E Test Coverage
The SDK implementation is validated by comprehensive integration tests:
**File:** `crates/stemedb-api/tests/e2e_flow_test.rs`
**Tests:**
1. `test_basic_write_materialize_query` - Write → MV → Read flow
2. `test_write_multiple_query_recency` - Recency lens with multiple assertions
3. `test_vote_and_query_consensus` - Voting + Consensus lens
4. `test_event_driven_materialization` - MV updates on write events
**Coverage:**
- Real Ed25519 signature generation and verification
- Event-driven materialization (WAL → MV update)
- All lens types (Recency, Consensus, Authority)
- Time-travel queries
- Vote aggregation with TrustRank
## Type Hierarchy
```
Go SDK (serde JSON) Rust API (serde JSON) Rust Core (rkyv)
| | |
Assertion struct dto::AssertionRequest types::Assertion
| | |
+-- Subject: string +-- subject: String +-- subject: EntityId
+-- Predicate: string +-- predicate: String +-- predicate: RelationId
+-- Object: interface{} +-- object: Value +-- object: ObjectValue
+-- Confidence: float32 +-- confidence: f32 +-- confidence: f32
+-- SourceHash: string +-- source_hash: Hash +-- source_hash: Hash
+-- Lifecycle: string +-- lifecycle: String +-- lifecycle: LifecycleStage
```
## Design Principles
**Content-Addressed Identity**
- Assertion hash = BLAKE3(canonical JSON serialization)
- Same content = same hash across all SDKs
- Idempotent writes (re-asserting produces same hash)
**Multi-Signature Support**
- Client signs entire assertion with Ed25519
- API verifies signature before writing to WAL
- Multiple agents can co-sign the same assertion (multi-sig)
**Fluent Builder Pattern**
- Type-safe construction with compile-time validation
- Prevents partial/invalid assertions
- Generates content hash automatically on Build()
**Zero Dependencies on Rust**
- Pure Go implementation with standard library + ADK
- JSON over HTTP (no gRPC/protobuf yet)
- Works with any Episteme deployment
## Error Handling
```go
// SDK errors implement error interface
_, err := client.Assert(ctx, assertion)
if err != nil {
var apiErr *steme.APIError
if errors.As(err, &apiErr) {
switch apiErr.Code {
case steme.ErrInvalidSignature:
// Re-sign with correct key
case steme.ErrDuplicateHash:
// Already exists (idempotent - not an error)
case steme.ErrConstraintViolation:
// Check constraints with ConstraintCheck tool
default:
// Log and retry with backoff
}
}
}
```
## Related Topics
- [API Surface](./api.md) - HTTP endpoints and OpenAPI schema
- [Assertion](./assertion.md) - Core data structure
- [Lens](./lens.md) - Read-time resolution strategies
- [ADK Integration Pattern](../patterns/adk-integration.md) - Tool and callback details
- [Query Audit](../features/query-audit.md) - Trace agent decisions
- [TrustRank](../features/trust-rank.md) - Agent reputation and Gardener