From 8ce28fc40c9b75a8b8953aea2a69fbfab5f70502 Mon Sep 17 00:00:00 2001 From: jordan Date: Sat, 24 Jan 2026 20:17:46 -0700 Subject: [PATCH] docs: Add comprehensive development plan (v0.1-v0.6) PLAN.md covers: - v0.1: Base case (complete) - v0.2: Real workspaces with init container clone - v0.3: Git integration with deploy keys - v0.4: Go REST API for controlling claudebox pods - v0.5: SSE streaming for real-time output - v0.6: Production hardening (auth, rate limits, audit) Architecture: External clients (Discord, Slack, CLI) connect to rdev-api which kubectl exec's into claudebox pods. Co-Authored-By: Claude Opus 4.5 --- PLAN.md | 404 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 404 insertions(+) create mode 100644 PLAN.md diff --git a/PLAN.md b/PLAN.md new file mode 100644 index 0000000..60dbefb --- /dev/null +++ b/PLAN.md @@ -0,0 +1,404 @@ +# rdev Development Plan + +Remote Developer - Claude Code on k3s with API control. + +## Vision + +Run Claude Code in isolated Kubernetes pods, controlled via a REST API with SSE streaming. External clients (Discord bots, Slack bots, CLI tools) connect to the API - keeping the core infrastructure separate from integration concerns. + +## Architecture + +``` +External Clients rdev (k3s namespace) +┌─────────────────┐ ┌─────────────────────────────────────┐ +│ Discord Bot │──┐ │ │ +│ (orchard9/ │ │ │ ┌─────────────────────────────┐ │ +│ rdev-discord) │ │ │ │ rdev-api (Go) │ │ +└─────────────────┘ │ │ │ │ │ + │ HTTP/SSE │ │ POST /projects/:id/claude │ │ +┌─────────────────┐ │ ─────────────▶ │ │ POST /projects/:id/shell │ │ +│ Slack Bot │──┤ │ │ POST /projects/:id/git │ │ +│ (future) │ │ │ │ GET /projects/:id/events │ │ +└─────────────────┘ │ │ │ GET /projects │ │ + │ │ │ GET /health │ │ +┌─────────────────┐ │ │ └──────────────┬──────────────┘ │ +│ CLI Tool │──┘ │ │ │ +│ (future) │ │ │ kubectl exec │ +└─────────────────┘ │ ▼ │ + │ ┌──────────────────────────────┐ │ + │ │ claudebox pods │ │ + │ │ │ │ + │ │ ┌────────────────────────┐ │ │ + │ │ │ claudebox-pantheon │ │ │ + │ │ │ /workspace: pantheon │ │ │ + │ │ │ Claude Code CLI │ │ │ + │ │ └────────────────────────┘ │ │ + │ │ │ │ + │ │ ┌────────────────────────┐ │ │ + │ │ │ claudebox-aeries │ │ │ + │ │ │ /workspace: aeries │ │ │ + │ │ │ Claude Code CLI │ │ │ + │ │ └────────────────────────┘ │ │ + │ └──────────────────────────────┘ │ + └─────────────────────────────────────┘ +``` + +## Versions + +### v0.1 - Base Case ✅ +**Status**: Complete (2026-01-24) + +Single claudebox pod running Claude Code CLI. + +**Delivered**: +- claudebox Docker image (`ghcr.io/orchard9/rdev-claudebox:v0.1.0`) +- Kubernetes manifests (StatefulSet, PVCs, Service) +- Manual interaction via `kubectl exec` +- Claude authentication persists in PVC + +**Verify**: +```bash +export KUBECONFIG=~/.kube/orchard9-k3sf.yaml +kubectl exec -it -n rdev claudebox-0 -- claude "say hello" +``` + +--- + +### v0.2 - Real Workspaces +**Status**: Planned + +Mount actual project repos (pantheon, aeries) into dedicated claudebox pods. + +**Deliverables**: +- [ ] `claudebox-pantheon` StatefulSet with pantheon repo +- [ ] `claudebox-aeries` StatefulSet with aeries repo +- [ ] Init container that clones repo on first start +- [ ] Git SSH deploy keys as Kubernetes secrets +- [ ] Project-specific CLAUDE.md mounted via ConfigMap + +**Architecture**: +```yaml +# Each project gets its own StatefulSet +claudebox-pantheon: + initContainer: + - clone github.com/orchard9/pantheon if /workspace empty + - else: git fetch origin + volumes: + - workspace-pantheon (PVC, 20Gi) + - claude-config-pantheon (PVC, 1Gi) + - ssh-keys (Secret) + - project-config (ConfigMap with CLAUDE.md) +``` + +**Init Container Logic**: +```bash +if [ ! -d /workspace/.git ]; then + git clone git@github.com:orchard9/pantheon.git /workspace +else + cd /workspace && git fetch origin +fi +``` + +**Deploy Keys**: +- Generate per-repo deploy key: `ssh-keygen -t ed25519 -f pantheon-deploy-key` +- Add public key to GitHub repo Settings → Deploy Keys (read/write) +- Store private key in K8s secret + +**Verify**: +```bash +kubectl exec -n rdev claudebox-pantheon-0 -- ls /workspace +# Should show pantheon repo files + +kubectl exec -n rdev claudebox-pantheon-0 -- git -C /workspace status +# Should show git status +``` + +--- + +### v0.3 - Git Integration +**Status**: Planned + +Pods can commit and push changes back to GitHub. + +**Deliverables**: +- [ ] SSH keys mounted at `/root/.ssh/` +- [ ] Git config (user.name, user.email) set in container +- [ ] known_hosts includes github.com +- [ ] Test push from inside pod + +**Git Config** (via ConfigMap or Dockerfile): +```bash +git config --global user.name "rdev-bot" +git config --global user.email "rdev@orchard9.ai" +``` + +**SSH Setup**: +```yaml +volumes: + - name: ssh-keys + secret: + secretName: github-deploy-keys + defaultMode: 0600 + items: + - key: pantheon-deploy-key + path: id_ed25519 + - key: known_hosts + path: known_hosts +``` + +**Verify**: +```bash +kubectl exec -n rdev claudebox-pantheon-0 -- bash -c " + cd /workspace && + echo 'test' >> test.txt && + git add test.txt && + git commit -m 'test commit from rdev' && + git push origin HEAD +" +``` + +--- + +### v0.4 - API Server +**Status**: Planned + +Go API server for controlling claudebox pods. + +**Deliverables**: +- [ ] `rdev-api` Go service +- [ ] REST endpoints for claude, shell, git commands +- [ ] SSE endpoint for streaming output +- [ ] Kubernetes RBAC for pod exec +- [ ] Project registry (which pods exist) + +**API Endpoints**: + +| Method | Path | Description | +|--------|------|-------------| +| GET | `/health` | Health check | +| GET | `/projects` | List available projects | +| GET | `/projects/:id` | Get project details | +| POST | `/projects/:id/claude` | Run claude command | +| POST | `/projects/:id/shell` | Run shell command | +| POST | `/projects/:id/git` | Run git command | +| GET | `/projects/:id/events` | SSE stream for output | + +**Request/Response**: + +```bash +# Start a claude command +POST /projects/pantheon/claude +Content-Type: application/json + +{ + "prompt": "fix the bug in auth handler", + "stream_id": "abc123" # For SSE correlation +} + +# Response +{ + "id": "cmd-xyz", + "status": "running", + "stream_url": "/projects/pantheon/events?stream_id=abc123" +} +``` + +```bash +# Stream output via SSE +GET /projects/pantheon/events?stream_id=abc123 + +event: output +data: {"line": "Reading auth handler...", "timestamp": "..."} + +event: output +data: {"line": "Found issue on line 142", "timestamp": "..."} + +event: complete +data: {"exit_code": 0, "duration_ms": 4532} +``` + +**Architecture**: +``` +cmd/rdev-api/ +├── main.go +internal/ +├── api/ +│ ├── handlers.go # HTTP handlers +│ ├── sse.go # SSE streaming +│ └── middleware.go # Auth, logging +├── executor/ +│ ├── kubectl.go # kubectl exec wrapper +│ └── stream.go # Output streaming +├── projects/ +│ └── registry.go # Project configuration +└── config/ + └── config.go +``` + +**RBAC**: +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: rdev-api + namespace: rdev +rules: + - apiGroups: [""] + resources: ["pods"] + verbs: ["get", "list"] + - apiGroups: [""] + resources: ["pods/exec"] + verbs: ["create"] +``` + +**Verify**: +```bash +curl http://rdev-api.rdev.svc/health +# {"status": "ok"} + +curl -X POST http://rdev-api.rdev.svc/projects/pantheon/claude \ + -H "Content-Type: application/json" \ + -d '{"prompt": "what files are in this project?"}' +``` + +--- + +### v0.5 - Streaming & Events +**Status**: Planned + +Real-time output streaming via SSE. + +**Deliverables**: +- [ ] SSE endpoint with proper event formatting +- [ ] Output buffering and chunking +- [ ] Connection management (heartbeats, reconnection) +- [ ] Event types: output, error, complete, heartbeat + +**SSE Event Types**: + +``` +event: heartbeat +data: {"timestamp": "2026-01-24T20:00:00Z"} + +event: output +data: {"line": "Analyzing code...", "stream": "stdout"} + +event: error +data: {"line": "Permission denied", "stream": "stderr"} + +event: complete +data: {"exit_code": 0, "duration_ms": 1234} +``` + +**Client Example**: +```javascript +const events = new EventSource('/projects/pantheon/events?stream_id=abc123'); + +events.addEventListener('output', (e) => { + const data = JSON.parse(e.data); + console.log(data.line); +}); + +events.addEventListener('complete', (e) => { + const data = JSON.parse(e.data); + console.log(`Done in ${data.duration_ms}ms`); + events.close(); +}); +``` + +--- + +### v0.6 - Production Ready +**Status**: Planned + +Hardening for production use. + +**Deliverables**: +- [ ] Authentication (API keys or OAuth) +- [ ] Rate limiting +- [ ] Project-level permissions +- [ ] Audit logging +- [ ] Health monitoring and alerts +- [ ] Auto-commit checkpoints +- [ ] Metrics (Prometheus) + +**Auth Options**: +1. **API Keys**: Simple, stored in K8s secret, passed via header +2. **OAuth/OIDC**: Integrate with existing auth (Discord OAuth?) +3. **Service Accounts**: For bot-to-API communication + +**Audit Log**: +```json +{ + "timestamp": "2026-01-24T20:00:00Z", + "project": "pantheon", + "action": "claude", + "prompt": "fix the auth bug", + "user": "discord:123456", + "duration_ms": 4532, + "exit_code": 0 +} +``` + +--- + +## File Structure + +``` +rdev/ +├── CLAUDE.md # Project instructions +├── PLAN.md # This file +├── README.md # Quick start +├── Dockerfile # claudebox image +├── cmd/ +│ └── rdev-api/ +│ └── main.go # API server entry (v0.4+) +├── internal/ # Go packages (v0.4+) +│ ├── api/ +│ ├── executor/ +│ └── projects/ +├── deployments/ +│ └── k8s/ +│ └── base/ +│ ├── kustomization.yaml +│ ├── namespace.yaml +│ ├── claudebox.yaml # Generic claudebox (v0.1) +│ ├── claudebox-pantheon.yaml # Project-specific (v0.2+) +│ ├── claudebox-aeries.yaml # Project-specific (v0.2+) +│ ├── rdev-api.yaml # API server (v0.4+) +│ ├── pvc.yaml +│ ├── secrets.yaml # Deploy keys (v0.3+) +│ └── rbac.yaml # API permissions (v0.4+) +├── scripts/ +│ ├── build-push.sh +│ ├── deploy.sh +│ └── verify.sh +├── docs/ +│ └── reference.md # Original reference + k3s notes +└── history/ + ├── v0.1.0.md # Release notes + ├── v0.2.0.md + └── ... +``` + +--- + +## Timeline + +| Version | Scope | Dependencies | +|---------|-------|--------------| +| v0.1 ✅ | Base case | None | +| v0.2 | Real workspaces | Deploy keys created | +| v0.3 | Git integration | v0.2 | +| v0.4 | API server | v0.3 | +| v0.5 | SSE streaming | v0.4 | +| v0.6 | Production ready | v0.5 | + +--- + +## Open Questions + +1. **Project config**: Where do we define which projects exist? ConfigMap? Code? +2. **Ingress**: Expose API externally or keep internal? If external, auth is critical. +3. **Multi-tenancy**: One rdev instance per org, or shared with project isolation? +4. **Claude context**: Should API support conversation IDs for context continuity?