1ce4004807
This commit adds the read path (Cortex) to complement the write path (Spine): ## Crates - stemedb-api: HTTP API with axum + utoipa OpenAPI - /v1/assert, /v1/query, /v1/epoch, /v1/skeptic, /v1/trace, /v1/audit - Metered endpoints with quota enforcement - Ed25519 signature verification - stemedb-lens: Truth resolution lenses - RecencyLens, ConsensusLens, ConfidenceLens - VoteAwareConsensusLens (Ballot Box pattern) - TrustAwareAuthorityLens (The Hive pattern) - SkepticLens (conflict analysis) - EpochAwareLens (paradigm-safe queries) - stemedb-query: Query engine with materialized views ## Storage Extensions - VoteStore: Vote aggregation with cached counts - TrustRankStore: Agent reputation with decay - AuditStore: Query audit trail - IndexStore: SP/P/S index structures - SupersessionStore: Epoch supersession chains ## SDKs - sdk/go/steme: Go HTTP client with Ed25519 signing - sdk/go/adk: ADK-Go tools for AI agents ## Documentation - Updated CLAUDE.md, architecture.md, roadmap.md - New ai-lookup entries for all services - Use case docs for consumer health intelligence - Arena roadmap for simulation advancement Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
511 lines
16 KiB
Markdown
511 lines
16 KiB
Markdown
# ADK-Go + Episteme Integration Guide
|
|
|
|
> **Purpose:** Technical reference for integrating AI agents with Episteme using Google's Agent Development Kit for Go (ADK-Go).
|
|
> **For use case context:** See [use-cases/agile-agent-team.md](../../../use-cases/agile-agent-team.md)
|
|
|
|
## Overview
|
|
|
|
This guide shows how AI agents integrate with Episteme for:
|
|
- **Query** - Retrieve knowledge with lens-based resolution
|
|
- **Assert** - Store knowledge with lifecycle and confidence
|
|
- **Constraint Check** - Pre-flight validation before actions
|
|
- **Trace** - Audit trail for incident investigation
|
|
- **Supersede** - Correct errors with cascade tracking
|
|
|
|
Each agent type has specific patterns for tool usage and callback integration.
|
|
|
|
---
|
|
|
|
## Tool Definitions
|
|
|
|
Every Episteme operation is exposed as an ADK-Go tool:
|
|
|
|
```go
|
|
package episteme
|
|
|
|
import (
|
|
"google.golang.org/adk/tool"
|
|
"google.golang.org/adk/tool/functiontool"
|
|
)
|
|
|
|
// === QUERY TOOL ===
|
|
// Used by: All agents
|
|
|
|
type QueryInput struct {
|
|
Subject string `json:"subject" jsonschema:"Entity to query (e.g., auth/jwt)"`
|
|
Predicate string `json:"predicate" jsonschema:"Relation to query (e.g., signing_algorithm)"`
|
|
Lens string `json:"lens,omitempty" jsonschema:"Resolution: consensus, authority, recency, constraints"`
|
|
Lifecycle string `json:"lifecycle,omitempty" jsonschema:"Filter: proposed, approved, deprecated"`
|
|
MinConfidence float32 `json:"min_confidence,omitempty" jsonschema:"Minimum confidence threshold (0.0-1.0)"`
|
|
AsOf string `json:"as_of,omitempty" jsonschema:"Time-travel: ISO8601 timestamp"`
|
|
}
|
|
|
|
type QueryOutput struct {
|
|
Value interface{} `json:"value"`
|
|
Confidence float32 `json:"confidence"`
|
|
Lifecycle string `json:"lifecycle"`
|
|
Sources []Source `json:"sources"`
|
|
QueryID string `json:"query_id"` // CRITICAL: for audit trail
|
|
}
|
|
|
|
type Source struct {
|
|
Hash string `json:"hash"`
|
|
SourceHash string `json:"source_hash"`
|
|
Weight float32 `json:"weight"`
|
|
}
|
|
|
|
func queryEpisteme(ctx tool.Context, input QueryInput) QueryOutput {
|
|
// Call Episteme API
|
|
result, err := epistemeClient.Query(ctx, input)
|
|
if err != nil {
|
|
return QueryOutput{Error: err.Error()}
|
|
}
|
|
return result
|
|
}
|
|
|
|
// === ASSERT TOOL ===
|
|
// Used by: Research Agent, Human Supervisor
|
|
|
|
type AssertInput struct {
|
|
Subject string `json:"subject" jsonschema:"Entity being described"`
|
|
Predicate string `json:"predicate" jsonschema:"Relation being asserted"`
|
|
Object interface{} `json:"object" jsonschema:"Value being claimed"`
|
|
SourceHash string `json:"source_hash" jsonschema:"BLAKE3 hash of evidence"`
|
|
Confidence float32 `json:"confidence" jsonschema:"Certainty level (0.0-1.0)"`
|
|
Lifecycle string `json:"lifecycle,omitempty" jsonschema:"proposed, under_review, approved"`
|
|
ParentHash string `json:"parent_hash,omitempty" jsonschema:"Hash of assertion being updated"`
|
|
Meta *AssertMeta `json:"meta,omitempty" jsonschema:"Negative constraints and metadata"`
|
|
}
|
|
|
|
type AssertMeta struct {
|
|
ForbiddenAlternative string `json:"forbidden_alternative,omitempty"`
|
|
Reason string `json:"reason,omitempty"`
|
|
}
|
|
|
|
type AssertOutput struct {
|
|
Hash string `json:"hash"`
|
|
Success bool `json:"success"`
|
|
Error string `json:"error,omitempty"`
|
|
}
|
|
|
|
// === CONSTRAINT CHECK TOOL ===
|
|
// Used by: Implementation Agent (pre-flight)
|
|
|
|
type ConstraintCheckInput struct {
|
|
Context string `json:"context" jsonschema:"Domain context (e.g., python_http, auth_jwt)"`
|
|
}
|
|
|
|
type ConstraintCheckOutput struct {
|
|
Constraints []Constraint `json:"constraints"`
|
|
}
|
|
|
|
type Constraint struct {
|
|
Subject string `json:"subject"`
|
|
MustUse string `json:"must_use,omitempty"`
|
|
Forbidden string `json:"forbidden,omitempty"`
|
|
Reason string `json:"reason"`
|
|
}
|
|
|
|
// === TRACE TOOL ===
|
|
// Used by: On-Call SRE, Human Supervisor
|
|
|
|
type TraceInput struct {
|
|
AgentID string `json:"agent_id" jsonschema:"Agent to trace"`
|
|
From string `json:"from" jsonschema:"Start time (ISO8601 or relative like -6h)"`
|
|
To string `json:"to,omitempty" jsonschema:"End time (default: now)"`
|
|
Subject string `json:"subject,omitempty" jsonschema:"Filter by subject pattern (e.g., auth/*)"`
|
|
}
|
|
|
|
type TraceOutput struct {
|
|
Queries []QueryTrace `json:"queries"`
|
|
}
|
|
|
|
type QueryTrace struct {
|
|
QueryID string `json:"query_id"`
|
|
Timestamp string `json:"timestamp"`
|
|
Subject string `json:"subject"`
|
|
Predicate string `json:"predicate"`
|
|
Lens string `json:"lens"`
|
|
Result string `json:"result"`
|
|
Confidence float32 `json:"confidence"`
|
|
Contributing []string `json:"contributing_assertions"`
|
|
}
|
|
|
|
// === SUPERSEDE TOOL ===
|
|
// Used by: Human Supervisor
|
|
|
|
type SupersedeInput struct {
|
|
Hash string `json:"hash" jsonschema:"Assertion hash to supersede"`
|
|
Reason string `json:"reason" jsonschema:"Why this is being superseded"`
|
|
Type string `json:"type" jsonschema:"Invalidate, Temporal, RequiresReview, Additive"`
|
|
}
|
|
|
|
type SupersedeOutput struct {
|
|
NewHash string `json:"new_hash"`
|
|
AffectedAssertions []string `json:"affected_assertions"`
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Callback Integration
|
|
|
|
Callbacks are critical for enforcing constraints and maintaining audit trails:
|
|
|
|
```go
|
|
import (
|
|
"google.golang.org/adk/agent"
|
|
"google.golang.org/adk/agent/llmagent"
|
|
"google.golang.org/adk/model"
|
|
"google.golang.org/adk/tool"
|
|
)
|
|
|
|
// Implementation Agent with BeforeToolCallback for constraint checking
|
|
implementationAgent, err := llmagent.New(llmagent.Config{
|
|
Name: "implementation_agent",
|
|
Model: model,
|
|
Description: "Writes code against current approved patterns",
|
|
Instruction: "You write code. Always use approved patterns only.",
|
|
Tools: []tool.Tool{queryTool, constraintTool},
|
|
|
|
// CRITICAL: Check constraints BEFORE any tool that generates code
|
|
BeforeToolCallback: func(ctx agent.CallbackContext, call *tool.Call) (*tool.Call, error) {
|
|
// If agent is about to write code, check constraints first
|
|
if needsConstraintCheck(call) {
|
|
context := extractDomainContext(call) // e.g., "python_http"
|
|
constraints, err := checkConstraints(ctx, context)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("constraint check failed: %w", err)
|
|
}
|
|
|
|
for _, c := range constraints {
|
|
if violatesConstraint(call, c) {
|
|
return nil, fmt.Errorf("blocked: %s is forbidden - %s",
|
|
c.Forbidden, c.Reason)
|
|
}
|
|
}
|
|
}
|
|
return call, nil
|
|
},
|
|
|
|
// Log decisions for audit trail
|
|
AfterModelCallback: func(ctx agent.CallbackContext, resp *model.LLMResponse) (*model.LLMResponse, error) {
|
|
decision := extractDecision(resp)
|
|
logToEpisteme(ctx, AuditEntry{
|
|
AgentID: ctx.Agent().Name(),
|
|
Timestamp: time.Now(),
|
|
Decision: decision,
|
|
SessionID: ctx.Session().ID(),
|
|
})
|
|
return resp, nil
|
|
},
|
|
})
|
|
|
|
// Lead Orchestrator with confidence threshold escalation
|
|
leadOrchestrator, err := llmagent.New(llmagent.Config{
|
|
Name: "lead_orchestrator",
|
|
Model: model,
|
|
Description: "Coordinates agent team, routes work based on knowledge",
|
|
Instruction: `You coordinate the agent team. Query Episteme for current state.
|
|
If confidence < 0.8, escalate to human supervisor.`,
|
|
Tools: []tool.Tool{queryTool, delegateTool},
|
|
OutputKey: "orchestrator_decision", // Pass to downstream agents
|
|
|
|
// Check confidence scores and escalate if too low
|
|
AfterToolCallback: func(ctx agent.CallbackContext, call *tool.Call, result *tool.Result) (*tool.Result, error) {
|
|
if call.Name == "episteme_query" {
|
|
var queryResult QueryOutput
|
|
json.Unmarshal(result.Output, &queryResult)
|
|
|
|
if queryResult.Confidence < 0.8 {
|
|
// Mark for human review
|
|
ctx.Session().State().Set("needs_human_review", true)
|
|
ctx.Session().State().Set("low_confidence_query", queryResult.QueryID)
|
|
}
|
|
}
|
|
return result, nil
|
|
},
|
|
})
|
|
```
|
|
|
|
---
|
|
|
|
## Agent-Specific Patterns
|
|
|
|
### Lead Orchestrator
|
|
|
|
Fast queries with confidence thresholds for routing decisions:
|
|
|
|
```go
|
|
// Query current auth config with confidence threshold
|
|
result := queryEpisteme(ctx, QueryInput{
|
|
Subject: "auth/jwt",
|
|
Predicate: "signing_algorithm",
|
|
Lens: "authority",
|
|
MinConfidence: 0.8,
|
|
})
|
|
|
|
if result.Confidence < 0.8 {
|
|
// Escalate to human - can't route confidently
|
|
ctx.Session().State().Set("escalation_reason",
|
|
fmt.Sprintf("Confidence %.2f below threshold for %s/%s",
|
|
result.Confidence, result.Subject, result.Predicate))
|
|
return escalateToHuman(ctx)
|
|
}
|
|
|
|
// Route to implementation agent with high confidence
|
|
ctx.Session().State().Set("auth_config", result.Value)
|
|
return delegateTo(ctx, "implementation_agent")
|
|
```
|
|
|
|
### Implementation Agent
|
|
|
|
Queries approved patterns only, with pre-flight constraint checks:
|
|
|
|
```go
|
|
// CRITICAL: Filter to approved lifecycle ONLY
|
|
result := queryEpisteme(ctx, QueryInput{
|
|
Subject: "auth/jwt",
|
|
Predicate: "signing_algorithm",
|
|
Lens: "authority",
|
|
Lifecycle: "approved", // Never use proposed patterns!
|
|
})
|
|
|
|
if result.Lifecycle != "approved" {
|
|
return fmt.Errorf("no approved pattern found for %s/%s",
|
|
input.Subject, input.Predicate)
|
|
}
|
|
|
|
// Pre-flight: check for forbidden alternatives
|
|
constraints := checkConstraints(ctx, "auth_jwt")
|
|
for _, c := range constraints.Constraints {
|
|
// Agent now knows: "use X, don't use Y, because Z"
|
|
// This enables contrastive learning
|
|
}
|
|
```
|
|
|
|
### Research Agent
|
|
|
|
Stores conflicting information with uncertainty:
|
|
|
|
```go
|
|
// Store first source
|
|
assertKnowledge(ctx, AssertInput{
|
|
Subject: "jwt_rotation",
|
|
Predicate: "best_practice",
|
|
Object: "rotate_daily",
|
|
SourceHash: hashURL("https://security.io/jwt-best-practices"),
|
|
Confidence: 0.7, // Express uncertainty
|
|
Lifecycle: "proposed", // Not approved yet
|
|
})
|
|
|
|
// Store contradicting source - don't flatten!
|
|
assertKnowledge(ctx, AssertInput{
|
|
Subject: "jwt_rotation",
|
|
Predicate: "best_practice",
|
|
Object: "rotate_hourly",
|
|
SourceHash: hashURL("https://owasp.org/jwt-rotation"),
|
|
Confidence: 0.8,
|
|
Lifecycle: "proposed",
|
|
})
|
|
|
|
// Let Lead Orchestrator resolve via Lens::Consensus or Lens::Authority
|
|
```
|
|
|
|
### Human Supervisor
|
|
|
|
Time-travel queries and corrections with impact analysis:
|
|
|
|
```go
|
|
// What was believed during the incident?
|
|
result := queryEpisteme(ctx, QueryInput{
|
|
Subject: "auth/jwt",
|
|
Predicate: "signing_algorithm",
|
|
AsOf: "2024-01-15T21:00:00Z", // Time of incident
|
|
})
|
|
|
|
// Trace agent queries during the incident window
|
|
traces := traceQueries(ctx, TraceInput{
|
|
AgentID: "deployment-agent",
|
|
From: "2024-01-15T20:00:00Z",
|
|
To: "2024-01-15T22:00:00Z",
|
|
Subject: "auth/*",
|
|
})
|
|
|
|
// Found the bug: agent queried without lifecycle filter
|
|
// Correct the record
|
|
impact := supersede(ctx, SupersedeInput{
|
|
Hash: badAssertionHash,
|
|
Reason: "Proposal treated as approved - agent didn't filter by lifecycle",
|
|
Type: "Invalidate",
|
|
})
|
|
|
|
// impact.AffectedAssertions shows downstream effects
|
|
fmt.Printf("Corrected. %d downstream assertions affected.\n",
|
|
len(impact.AffectedAssertions))
|
|
```
|
|
|
|
### On-Call SRE
|
|
|
|
Sub-second trace commands for incident investigation:
|
|
|
|
```go
|
|
// It's 3am. Auth is broken. What happened?
|
|
|
|
// Step 1: What did deployment agent query? (<500ms required)
|
|
traces := traceQueries(ctx, TraceInput{
|
|
AgentID: "deployment-agent",
|
|
From: "-6h",
|
|
Subject: "auth/*",
|
|
})
|
|
|
|
for _, t := range traces.Queries {
|
|
fmt.Printf("[%s] %s/%s via %s -> %s (conf: %.2f)\n",
|
|
t.Timestamp, t.Subject, t.Predicate,
|
|
t.Lens, t.Result, t.Confidence)
|
|
|
|
if len(t.Contributing) > 0 {
|
|
fmt.Printf(" Contributing: %v\n", t.Contributing)
|
|
}
|
|
}
|
|
|
|
// Step 2: What changed in the last 24 hours?
|
|
diff := queryDiff(ctx, DiffInput{
|
|
Subject: "auth/jwt",
|
|
From: "-24h",
|
|
})
|
|
|
|
// Step 3: Mark bad assertion (found via trace)
|
|
supersede(ctx, SupersedeInput{
|
|
Hash: badAssertionHash,
|
|
Reason: "RFC proposal incorrectly treated as approved config",
|
|
Type: "Invalidate",
|
|
})
|
|
|
|
// Step 4: Verify fix
|
|
result := queryEpisteme(ctx, QueryInput{
|
|
Subject: "auth/jwt",
|
|
Predicate: "signing_algorithm",
|
|
Lifecycle: "approved",
|
|
})
|
|
fmt.Printf("Current approved value: %s\n", result.Value)
|
|
```
|
|
|
|
---
|
|
|
|
## Multi-Agent Pipeline Example
|
|
|
|
Complete example showing agents coordinating through Episteme:
|
|
|
|
```go
|
|
package main
|
|
|
|
import (
|
|
"context"
|
|
"google.golang.org/adk/agent"
|
|
"google.golang.org/adk/agent/llmagent"
|
|
"google.golang.org/adk/agent/sequentialagent"
|
|
"google.golang.org/adk/model/gemini"
|
|
)
|
|
|
|
func main() {
|
|
ctx := context.Background()
|
|
model, _ := gemini.NewModel(ctx, "gemini-3-flash-preview", nil)
|
|
|
|
// Research Agent: Discovers and stores knowledge
|
|
researchAgent, _ := llmagent.New(llmagent.Config{
|
|
Name: "research_agent",
|
|
Model: model,
|
|
Description: "Researches and stores knowledge with source attribution",
|
|
Instruction: `Research the topic. Store findings with confidence scores.
|
|
Mark conflicting sources - don't resolve them.`,
|
|
Tools: []tool.Tool{assertTool, searchTool},
|
|
OutputKey: "research_findings",
|
|
})
|
|
|
|
// Lead Orchestrator: Queries and routes
|
|
leadAgent, _ := llmagent.New(llmagent.Config{
|
|
Name: "lead_orchestrator",
|
|
Model: model,
|
|
Description: "Coordinates team based on current knowledge state",
|
|
Instruction: `Query Episteme for current approved patterns.
|
|
Research findings available in {research_findings}.
|
|
If confidence >= 0.8, route to implementation.
|
|
If confidence < 0.8, flag for human review.`,
|
|
Tools: []tool.Tool{queryTool},
|
|
OutputKey: "routing_decision",
|
|
|
|
AfterToolCallback: confidenceEscalationCallback,
|
|
})
|
|
|
|
// Implementation Agent: Writes code
|
|
implAgent, _ := llmagent.New(llmagent.Config{
|
|
Name: "implementation_agent",
|
|
Model: model,
|
|
Description: "Writes code against approved patterns only",
|
|
Instruction: `Write code for the task.
|
|
Routing decision in {routing_decision}.
|
|
ONLY use approved patterns. Check constraints first.`,
|
|
Tools: []tool.Tool{queryTool, constraintTool, writeCodeTool},
|
|
OutputKey: "implementation",
|
|
|
|
BeforeToolCallback: constraintEnforcementCallback,
|
|
})
|
|
|
|
// Sequential pipeline
|
|
pipeline, _ := sequentialagent.New(sequentialagent.Config{
|
|
AgentConfig: agent.Config{
|
|
Name: "AgileDevPipeline",
|
|
Description: "Research -> Route -> Implement with Episteme coordination",
|
|
SubAgents: []agent.Agent{researchAgent, leadAgent, implAgent},
|
|
},
|
|
})
|
|
|
|
// Run the pipeline
|
|
runner := agent.NewRunner(pipeline, sessionService)
|
|
for event, err := range runner.Run(ctx, userID, sessionID, userMessage, nil) {
|
|
if err != nil {
|
|
log.Printf("Pipeline error: %v", err)
|
|
}
|
|
handleEvent(event)
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Session State vs. Episteme
|
|
|
|
| Use Case | Mechanism | Why |
|
|
|----------|-----------|-----|
|
|
| Agent-to-agent handoff in pipeline | Session State + `OutputKey` | Ephemeral, same conversation |
|
|
| Permanent organizational knowledge | Episteme `Assert` | Persists across sessions/agents |
|
|
| "What was decided in this conversation" | Session State | Temporary, conversation-scoped |
|
|
| "What should all agents know forever" | Episteme `Assert` + `lifecycle: approved` | Permanent, query-audited |
|
|
| Temporary working data | `temp:` prefixed state keys | Auto-cleared after session |
|
|
| Agent decision audit trail | Episteme `QueryAudit` | Immutable, time-travel enabled |
|
|
|
|
---
|
|
|
|
## CLI Commands for Debugging
|
|
|
|
```bash
|
|
# Trace what an agent queried
|
|
episteme trace --agent deployment-agent --time "6 hours ago" --subject "auth/*"
|
|
|
|
# Show assertion provenance
|
|
episteme show --hash $ASSERTION_HASH --provenance
|
|
|
|
# Supersede bad assertion
|
|
episteme supersede --hash $BAD_HASH --reason "Proposal treated as approved" \
|
|
--type "Invalidate"
|
|
|
|
# See cascade impact
|
|
episteme cascade --root $BAD_HASH
|
|
|
|
# Verify correction
|
|
episteme query --subject auth/jwt --predicate signing_algorithm --lifecycle approved
|
|
```
|
|
|
|
**SRE's 3am workflow:** Query -> Trace -> Identify -> Correct -> Verify. Under 10 minutes.
|