release: v0.10.52 - feat: SDLC worker routing for skeleton projects with auto-init
This commit is contained in:
jordan
parent
46c8bfeec2
commit
4766a54314
491
.claude/guides/services/composable-monorepo.md
Normal file
491
.claude/guides/services/composable-monorepo.md
Normal file
@ -0,0 +1,491 @@
|
||||
# Composable Monorepo Templates
|
||||
|
||||
**When to use:** Implementing the monorepo skeleton, component templates, or the component addition API.
|
||||
|
||||
## Overview
|
||||
|
||||
Composable Monorepo Templates evolve rdev from single-template projects to full monorepo scaffolding:
|
||||
|
||||
1. **Skeleton**: Every `POST /projects` creates the monorepo base (shared `pkg/`, scripts, CI)
|
||||
2. **Component Templates**: `POST /projects/{id}/components` adds services/workers/apps/cli
|
||||
3. **Deployment**: Target whole monorepo or individual components
|
||||
|
||||
**Terminology:** "skeleton" = generated project code; "platform" = rdev itself. See [ai-lookup/terminology.md](../../../ai-lookup/terminology.md).
|
||||
|
||||
**Plan Document:** `tmp/template-monorepo-plan.md`
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1: Skeleton Template
|
||||
|
||||
Create `templates/skeleton/` with base monorepo structure.
|
||||
|
||||
**Files to create:**
|
||||
```
|
||||
internal/adapter/templates/templates/skeleton/
|
||||
├── CLAUDE.md.tmpl
|
||||
├── README.md.tmpl
|
||||
├── Procfile.tmpl
|
||||
├── docker-compose.yml.tmpl
|
||||
├── go.work.tmpl
|
||||
├── .golangci.yml
|
||||
├── .gitignore
|
||||
├── .woodpecker.yml.tmpl # ⚠️ Template-provided CI (never AI-generated)
|
||||
├── .claude/
|
||||
│ ├── settings.local.json
|
||||
│ ├── guides/
|
||||
│ └── skills/
|
||||
├── scripts/
|
||||
│ ├── install.sh
|
||||
│ ├── quality.sh
|
||||
│ ├── dev.sh
|
||||
│ └── discover.sh
|
||||
└── pkg/
|
||||
├── go.mod.tmpl
|
||||
├── app/
|
||||
├── middleware/
|
||||
├── httpcontext/
|
||||
├── httpresponse/
|
||||
├── httpvalidation/
|
||||
├── logging/
|
||||
├── config/
|
||||
└── httpclient/
|
||||
```
|
||||
|
||||
**Critical: CI Must Be Templated**
|
||||
|
||||
AI-generated `.woodpecker.yml` produces invalid YAML (broken anchor syntax). All CI comes from templates:
|
||||
- Skeleton provides base `.woodpecker.yml.tmpl`
|
||||
- Components provide `.woodpecker.step.yml.tmpl`
|
||||
- AddComponent service inserts component steps into main pipeline
|
||||
|
||||
**Template Variables:**
|
||||
```go
|
||||
{{.ProjectName}} // "acme"
|
||||
{{.GoModule}} // "github.com/orchard9/acme"
|
||||
{{.Description}} // "Project description"
|
||||
```
|
||||
|
||||
**Provider Changes:**
|
||||
```go
|
||||
// internal/port/template_provider.go
|
||||
type TemplateProvider interface {
|
||||
// Existing
|
||||
GetTemplate(name string) (*Template, error)
|
||||
ListTemplates() []Template
|
||||
|
||||
// New
|
||||
GetSkeleton() (*Template, error)
|
||||
GetComponentTemplate(componentType, templateName string) (*Template, error)
|
||||
ListComponentTemplates(componentType string) []Template
|
||||
}
|
||||
```
|
||||
|
||||
### Phase 1.5: Shared Packages (pkg/)
|
||||
|
||||
Extract and combine packages from Aeries + Colix:
|
||||
|
||||
| Package | Source | Key Files |
|
||||
|---------|--------|-----------|
|
||||
| `app/` | Aeries `pkg/chassis/` | `app.go` |
|
||||
| `middleware/` | Colix `pkg/middleware/` | `cors.go`, `recovery.go`, `request_id.go`, `logger.go` |
|
||||
| `httpcontext/` | Colix `pkg/httpcontext/` | `keys.go` |
|
||||
| `httpresponse/` | Both | `response.go`, `envelope.go` |
|
||||
| `httpvalidation/` | Colix `pkg/httpvalidation/` | `validator.go`, `errors.go` |
|
||||
| `logging/` | Both | `logger.go` |
|
||||
| `config/` | Aeries `pkg/config/` | `config.go` |
|
||||
| `httpclient/` | Both | `client.go` |
|
||||
|
||||
### Phase 2: Component Templates
|
||||
|
||||
Migrate existing templates to component format:
|
||||
|
||||
```
|
||||
internal/adapter/templates/templates/components/
|
||||
├── service/ # Renamed from go-api
|
||||
│ ├── cmd/server/main.go.tmpl
|
||||
│ ├── internal/
|
||||
│ ├── Makefile.tmpl
|
||||
│ ├── Dockerfile.tmpl
|
||||
│ ├── component.yaml.tmpl
|
||||
│ └── .woodpecker.step.yml.tmpl # CI step for this component type
|
||||
├── worker/ # NEW
|
||||
├── app-astro/ # Renamed from astro-landing
|
||||
├── app-react/ # NEW
|
||||
└── cli/ # NEW
|
||||
```
|
||||
|
||||
Each component includes `.woodpecker.step.yml.tmpl` that defines its Kaniko build step.
|
||||
When components are added, the service renders this template and inserts it into the main pipeline.
|
||||
|
||||
**Component Variables:**
|
||||
```go
|
||||
{{.ProjectName}} // "acme"
|
||||
{{.ComponentName}} // "auth-api"
|
||||
{{.ComponentNameCamel}} // "AuthApi"
|
||||
{{.GoModule}} // "github.com/orchard9/acme"
|
||||
{{.Port}} // 8080
|
||||
```
|
||||
|
||||
### Phase 3: Add Component API
|
||||
|
||||
Create endpoint for adding components to existing projects.
|
||||
|
||||
**Handler:**
|
||||
```go
|
||||
// internal/handlers/components.go
|
||||
func (h *Handler) AddComponent(w http.ResponseWriter, r *http.Request) {
|
||||
projectID := chi.URLParam(r, "projectID")
|
||||
|
||||
var req AddComponentRequest
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||
api.Error(w, http.StatusBadRequest, "invalid request")
|
||||
return
|
||||
}
|
||||
|
||||
component, err := h.svc.AddComponent(r.Context(), projectID, req.Type, req.Name, req.Template)
|
||||
if err != nil {
|
||||
api.Error(w, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
api.OK(w, component)
|
||||
}
|
||||
```
|
||||
|
||||
**Request/Response:**
|
||||
```go
|
||||
type AddComponentRequest struct {
|
||||
Type string `json:"type"` // service, worker, app, cli
|
||||
Name string `json:"name"` // auth-api, dashboard, etc.
|
||||
Template string `json:"template"` // optional: specific variant
|
||||
}
|
||||
```
|
||||
|
||||
**Service Logic:**
|
||||
```go
|
||||
func (s *Service) AddComponent(ctx context.Context, projectID, compType, name, template string) (*domain.Component, error) {
|
||||
// 1. Render component template
|
||||
// 2. Commit files to project repo
|
||||
// 3. Update Procfile with new entry
|
||||
// 4. Update go.work if Go component
|
||||
// 5. Update CLAUDE.md routing
|
||||
// 6. Update .woodpecker.yml with component's build step
|
||||
// 7. Return component metadata
|
||||
}
|
||||
```
|
||||
|
||||
### Phase 4: Component-Aware Deployment
|
||||
|
||||
Extend deploy to support component targets:
|
||||
|
||||
```go
|
||||
type DeployRequest struct {
|
||||
Component string `json:"component,omitempty"` // e.g., "services/auth-api"
|
||||
}
|
||||
|
||||
// POST /projects/{id}/deploy → full monorepo
|
||||
// POST /projects/{id}/deploy/services/auth-api → single component
|
||||
```
|
||||
|
||||
### Phase 5: Discovery Scripts
|
||||
|
||||
Scripts that walk the monorepo and operate on all components:
|
||||
|
||||
```bash
|
||||
# scripts/discover.sh
|
||||
#!/bin/bash
|
||||
for type in services workers apps cli; do
|
||||
for dir in "$type"/*/; do
|
||||
[ -d "$dir" ] && echo "$type/$(basename $dir)"
|
||||
done
|
||||
done
|
||||
|
||||
# scripts/install.sh
|
||||
#!/bin/bash
|
||||
for service in services/*/; do
|
||||
[ -f "$service/go.mod" ] && (cd "$service" && go mod download)
|
||||
done
|
||||
for app in apps/*/; do
|
||||
[ -f "$app/package.json" ] && (cd "$app" && npm install)
|
||||
done
|
||||
|
||||
# scripts/quality.sh
|
||||
#!/bin/bash
|
||||
golangci-lint run ./services/... ./workers/... ./cli/...
|
||||
for app in apps/*/; do
|
||||
[ -f "$app/package.json" ] && (cd "$app" && npm run lint)
|
||||
done
|
||||
|
||||
# scripts/dev.sh
|
||||
#!/bin/bash
|
||||
docker-compose up -d
|
||||
overmind start
|
||||
```
|
||||
|
||||
### Phase 6: Quality Hooks
|
||||
|
||||
Pre-commit hooks for monorepo quality:
|
||||
|
||||
```bash
|
||||
# .githooks/pre-commit
|
||||
#!/bin/bash
|
||||
# File length check (500 lines)
|
||||
for file in $(git diff --cached --name-only --diff-filter=ACM | grep -E '\.(go|ts|tsx)$'); do
|
||||
lines=$(wc -l < "$file")
|
||||
if [ "$lines" -gt 500 ]; then
|
||||
echo "Error: $file exceeds 500 lines ($lines)"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
# Go formatting
|
||||
gofmt -l -w $(git diff --cached --name-only --diff-filter=ACM | grep '\.go$')
|
||||
|
||||
# Linting
|
||||
golangci-lint run ./services/... ./workers/... ./cli/...
|
||||
```
|
||||
|
||||
## Domain Model
|
||||
|
||||
```go
|
||||
// internal/domain/component.go
|
||||
type ComponentType string
|
||||
|
||||
const (
|
||||
// Code components (scaffold template files)
|
||||
ComponentTypeService ComponentType = "service"
|
||||
ComponentTypeWorker ComponentType = "worker"
|
||||
ComponentTypeAppAstro ComponentType = "app-astro"
|
||||
ComponentTypeAppReact ComponentType = "app-react"
|
||||
ComponentTypeCLI ComponentType = "cli"
|
||||
|
||||
// Infrastructure components (provision resources)
|
||||
ComponentTypePostgres ComponentType = "postgres"
|
||||
ComponentTypeRedis ComponentType = "redis"
|
||||
)
|
||||
|
||||
type Component struct {
|
||||
Type ComponentType
|
||||
Name string
|
||||
Template string
|
||||
Path string // "services/auth-api" or "infra/postgres"
|
||||
Port int // from component.yaml or auto-assigned
|
||||
Dependencies []string // postgres, redis, etc.
|
||||
BuildOrder int
|
||||
}
|
||||
```
|
||||
|
||||
## Infrastructure Components
|
||||
|
||||
Infrastructure components (`postgres`, `redis`) don't scaffold files — they provision actual resources:
|
||||
|
||||
### Adding Database (CockroachDB)
|
||||
|
||||
```bash
|
||||
POST /projects/acme/components
|
||||
{
|
||||
"type": "postgres",
|
||||
"name": "main-db"
|
||||
}
|
||||
```
|
||||
|
||||
This:
|
||||
1. Creates a CockroachDB database: `project_acme`
|
||||
2. Creates a database user with full permissions
|
||||
3. Stores `DATABASE_URL` and `DATABASE_URL_STAGING` in credential store
|
||||
|
||||
### Adding Cache (Redis)
|
||||
|
||||
```bash
|
||||
POST /projects/acme/components
|
||||
{
|
||||
"type": "redis",
|
||||
"name": "job-queue"
|
||||
}
|
||||
```
|
||||
|
||||
This:
|
||||
1. Creates a Redis ACL user: `proj-acme`
|
||||
2. Scopes access to keys matching `project:acme:*`
|
||||
3. Stores `REDIS_URL`, `REDIS_URL_STAGING`, and `REDIS_PREFIX` in credential store
|
||||
|
||||
## Credential Injection
|
||||
|
||||
**Critical:** When code components are deployed, credentials are automatically injected.
|
||||
|
||||
The `createInitialComponentDeployment()` function:
|
||||
1. Calls `fetchProjectCredentials(projectID)` to retrieve stored credentials
|
||||
2. Populates `DeploySpec.Secrets` with `DATABASE_URL`, `REDIS_URL`, etc.
|
||||
3. Deployer creates K8s Secret and mounts it as environment variables
|
||||
|
||||
**Available credentials (if provisioned):**
|
||||
- `DATABASE_URL` - CockroachDB connection string
|
||||
- `DATABASE_URL_STAGING` - Staging database (same as prod currently)
|
||||
- `REDIS_URL` - Redis connection string with auth
|
||||
- `REDIS_URL_STAGING` - Staging Redis (same as prod currently)
|
||||
- `REDIS_PREFIX` - Key prefix for isolation (e.g., `project:acme:`)
|
||||
|
||||
**Service Discovery:** Sibling services are also injected as env vars:
|
||||
- `AUTH_SVC_URL=http://acme-auth-svc:8001`
|
||||
- `CHAT_SVC_URL=http://acme-chat-svc:8002`
|
||||
|
||||
**File Pointers:**
|
||||
- Credential injection: `internal/service/component_deploy.go:35-48, 160-212`
|
||||
- Infrastructure provisioning: `internal/service/component_infra.go`
|
||||
- Deployer secret creation: `internal/adapter/deployer/resources.go:81-135`
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
**Important Route Note:** Project CRUD uses `/project` (singular), but component/build/pipeline operations use `/projects/{id}/...` (plural).
|
||||
|
||||
### Create Project (Skeleton)
|
||||
|
||||
```bash
|
||||
POST /project
|
||||
{
|
||||
"name": "acme",
|
||||
"description": "Acme Corp platform"
|
||||
}
|
||||
|
||||
# Response: Creates monorepo skeleton in repo
|
||||
```
|
||||
|
||||
### Add Component
|
||||
|
||||
```bash
|
||||
POST /projects/acme/components
|
||||
{
|
||||
"type": "service",
|
||||
"name": "auth-api",
|
||||
"template": "service" # optional
|
||||
}
|
||||
|
||||
# Response:
|
||||
{
|
||||
"type": "service",
|
||||
"name": "auth-api",
|
||||
"path": "services/auth-api",
|
||||
"port": 8001
|
||||
}
|
||||
```
|
||||
|
||||
### Deploy
|
||||
|
||||
```bash
|
||||
# Component deploy (single component at a time)
|
||||
POST /projects/acme/deploy
|
||||
{
|
||||
"component": "services/auth-api"
|
||||
}
|
||||
|
||||
# Note: Full monorepo deploy is handled by CI pipeline, not a single API call
|
||||
```
|
||||
|
||||
### List Components
|
||||
|
||||
```bash
|
||||
GET /projects/acme/components
|
||||
|
||||
# Response:
|
||||
{
|
||||
"components": [
|
||||
{"type": "service", "name": "auth-api", "path": "services/auth-api", "port": 8001},
|
||||
{"type": "app-react", "name": "dashboard", "path": "apps/dashboard", "port": 3001}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Delete Project
|
||||
|
||||
```bash
|
||||
DELETE /project/acme
|
||||
```
|
||||
|
||||
### List Component Templates
|
||||
|
||||
```bash
|
||||
GET /templates/components
|
||||
|
||||
# Response shows available component types: service, worker, app-astro, app-react, cli
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
```go
|
||||
func TestAddComponent(t *testing.T) {
|
||||
// Setup project first
|
||||
project := createTestProject(t, "test-monorepo")
|
||||
|
||||
// Add service component
|
||||
req := AddComponentRequest{
|
||||
Type: "service",
|
||||
Name: "auth-api",
|
||||
Template: "go-api",
|
||||
}
|
||||
|
||||
component, err := svc.AddComponent(ctx, project.ID, req.Type, req.Name, req.Template)
|
||||
require.NoError(t, err)
|
||||
|
||||
assert.Equal(t, "service", string(component.Type))
|
||||
assert.Equal(t, "auth-api", component.Name)
|
||||
assert.Equal(t, "services/auth-api", component.Path)
|
||||
|
||||
// Verify files created in repo
|
||||
files, err := gitea.ListFiles(project.Owner, project.Name, "services/auth-api")
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, files, "cmd/server/main.go")
|
||||
assert.Contains(t, files, "Makefile")
|
||||
}
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Component endpoint returns 404
|
||||
|
||||
The component service is only initialized when `GITEA_URL`, `GITEA_TOKEN`, and template provider are all configured. Check startup logs for "component service initialized":
|
||||
|
||||
```bash
|
||||
kubectl logs -n rdev deployment/rdev-api | grep "component service"
|
||||
```
|
||||
|
||||
If missing, verify the rdev-api deployment has the required env vars.
|
||||
|
||||
### Component addition fails with "directory exists"
|
||||
|
||||
The component path already exists. Either delete it or choose a different name.
|
||||
|
||||
### Component addition fails with "connection refused"
|
||||
|
||||
**Hairpin NAT issue.** rdev-api is trying to reach Gitea via external URL which doesn't work from within the cluster. Use internal service hostnames:
|
||||
|
||||
```yaml
|
||||
# In deployments/k8s/base/rdev-api.yaml
|
||||
- name: GITEA_URL
|
||||
value: "http://gitea.threesix.svc.cluster.local"
|
||||
```
|
||||
|
||||
See [ops/networking.md](../ops/networking.md) for details.
|
||||
|
||||
### CI pipeline linter errors
|
||||
|
||||
Woodpecker may show linter warnings about oneOf/anyOf validation. These are usually benign. Check for actual schema violations like invalid `when.event` values.
|
||||
|
||||
**Critical:** The `.woodpecker.yml` marker must be exactly `# COMPONENT_STEPS_BELOW` on its own line. If the marker has trailing text (like `- Do not remove`), it will be appended to the last line of inserted component steps, corrupting the YAML.
|
||||
|
||||
**File Pointer:** `internal/adapter/templates/templates/skeleton/.woodpecker.yml.tmpl:11`
|
||||
|
||||
### Procfile not updating
|
||||
|
||||
Check that the template includes the Procfile.tmpl and the service has write access to the repo.
|
||||
|
||||
### go.work not syncing
|
||||
|
||||
Run `./scripts/discover.sh` to verify component detection, then manually run `go work sync`.
|
||||
|
||||
## Related
|
||||
|
||||
- [Project Templates](./templates.md) - Current single-template system
|
||||
- [Build Orchestration](./build-orchestration.md) - Component builds
|
||||
- [ai-lookup: Composable Monorepo](../../ai-lookup/features/composable-monorepo.md) - Quick facts
|
||||
321
.claude/guides/services/cookbook-trees.md
Normal file
321
.claude/guides/services/cookbook-trees.md
Normal file
@ -0,0 +1,321 @@
|
||||
# Cookbook Tree System
|
||||
|
||||
Checkpoint-based cookbook execution with YAML tree definitions. Enables resumable, debuggable E2E test workflows.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
```bash
|
||||
# Run a tree (creates checkpoint on each step)
|
||||
./cookbooks/scripts/tree-runner.sh run landing-page --project-name my-test
|
||||
|
||||
# Resume from last checkpoint after failure
|
||||
./cookbooks/scripts/tree-runner.sh resume landing-page
|
||||
|
||||
# Run only a specific step (debugging)
|
||||
./cookbooks/scripts/tree-runner.sh only landing-page wait-pipeline
|
||||
|
||||
# Check status of a tree run
|
||||
./cookbooks/scripts/tree-runner.sh status landing-page
|
||||
|
||||
# Teardown resources (runs tree's teardown section)
|
||||
./cookbooks/scripts/tree-runner.sh teardown landing-page
|
||||
|
||||
# List all available trees
|
||||
./cookbooks/scripts/tree-runner.sh list
|
||||
|
||||
# Clean checkpoint (discard state)
|
||||
./cookbooks/scripts/tree-runner.sh clean landing-page
|
||||
```
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `yq` - YAML parser (`brew install yq`)
|
||||
- `jq` - JSON parser (already required by common.sh)
|
||||
|
||||
## Tree YAML Format
|
||||
|
||||
Tree definitions live in `cookbooks/trees/` and define workflow steps as a DAG.
|
||||
|
||||
```yaml
|
||||
name: landing-page
|
||||
description: Deploy a landing page
|
||||
version: 1
|
||||
|
||||
# Variables (can be overridden via --var-name)
|
||||
vars:
|
||||
project_name: "" # Required, no default
|
||||
template: "app-astro" # Optional, has default
|
||||
|
||||
steps:
|
||||
create-project:
|
||||
description: Create the project skeleton
|
||||
action: api
|
||||
method: POST
|
||||
endpoint: /project
|
||||
body:
|
||||
name: "{{ .vars.project_name }}"
|
||||
description: "Landing page E2E test"
|
||||
outputs:
|
||||
- project_id: .data.name
|
||||
- domain: .data.domain
|
||||
|
||||
add-component:
|
||||
description: Add landing page component
|
||||
depends_on: [create-project]
|
||||
action: api
|
||||
method: POST
|
||||
endpoint: "/projects/{{ .outputs.create-project.project_id }}/components"
|
||||
body:
|
||||
type: "{{ .vars.template }}"
|
||||
name: landing
|
||||
template: "{{ .vars.template }}"
|
||||
|
||||
wait-pipeline:
|
||||
description: Wait for CI pipeline to complete
|
||||
depends_on: [add-component]
|
||||
action: wait_pipeline
|
||||
project_id: "{{ .outputs.create-project.project_id }}"
|
||||
on_error: continue # Don't fail the whole tree
|
||||
|
||||
verify-site:
|
||||
description: Verify site is accessible
|
||||
depends_on: [wait-pipeline]
|
||||
action: wait_site
|
||||
domain: "{{ .outputs.create-project.domain }}"
|
||||
project_id: "{{ .outputs.create-project.project_id }}"
|
||||
|
||||
# Teardown runs in reverse order on failure or explicit teardown
|
||||
teardown:
|
||||
- description: Delete project
|
||||
action: api
|
||||
method: DELETE
|
||||
endpoint: "/project/{{ .outputs.create-project.project_id }}"
|
||||
```
|
||||
|
||||
### Step Properties
|
||||
|
||||
| Property | Required | Description |
|
||||
|----------|----------|-------------|
|
||||
| `description` | No | Human-readable description |
|
||||
| `action` | Yes | Action type: `api`, `wait_pipeline`, `wait_site`, `diagnose`, `shell` |
|
||||
| `depends_on` | No | Array of step names that must complete first |
|
||||
| `on_error` | No | `fail` (default) or `continue` |
|
||||
| `outputs` | No | Extract values from response (jq paths) |
|
||||
|
||||
### Action Types
|
||||
|
||||
#### api
|
||||
Make an authenticated API call.
|
||||
|
||||
```yaml
|
||||
action: api
|
||||
method: POST # GET, POST, DELETE, PUT, PATCH
|
||||
endpoint: /projects/{{ .project_id }}/components
|
||||
body: # Optional, for POST/PUT/PATCH
|
||||
type: service
|
||||
name: api
|
||||
```
|
||||
|
||||
#### wait_pipeline
|
||||
Wait for a CI pipeline to complete.
|
||||
|
||||
```yaml
|
||||
action: wait_pipeline
|
||||
project_id: "{{ .outputs.create-project.project_id }}"
|
||||
max_attempts: 60 # Optional, default 60
|
||||
poll_interval: 5 # Optional, default 5 seconds
|
||||
```
|
||||
|
||||
#### wait_site
|
||||
Wait for a site to be accessible.
|
||||
|
||||
```yaml
|
||||
action: wait_site
|
||||
domain: "{{ .outputs.create-project.domain }}"
|
||||
project_id: "{{ .outputs.create-project.project_id }}" # For diagnostics
|
||||
max_attempts: 30
|
||||
poll_interval: 5
|
||||
```
|
||||
|
||||
#### diagnose
|
||||
Run diagnostic checks.
|
||||
|
||||
```yaml
|
||||
action: diagnose
|
||||
type: pipeline # or 'site'
|
||||
project_id: "{{ .outputs.create-project.project_id }}"
|
||||
domain: "{{ .outputs.create-project.domain }}" # For site diagnostics
|
||||
```
|
||||
|
||||
#### shell
|
||||
Run a shell command.
|
||||
|
||||
```yaml
|
||||
action: shell
|
||||
command: "curl -s https://{{ .outputs.create-project.domain }}/api/health | jq ."
|
||||
outputs:
|
||||
- health_status: .status
|
||||
```
|
||||
|
||||
### Template Variables
|
||||
|
||||
Variables are expanded using Go template syntax (`{{ .path }}`):
|
||||
|
||||
- `.vars.<name>` - Variables from CLI flags or tree defaults
|
||||
- `.outputs.<step>.<key>` - Outputs captured from previous steps
|
||||
|
||||
## Checkpoint Format
|
||||
|
||||
Checkpoints are stored in `cookbooks/.checkpoints/` (gitignored) as JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"tree": "landing-page",
|
||||
"run_id": "landing-page-1706889600",
|
||||
"status": "partial",
|
||||
"vars": {
|
||||
"project_name": "test-landing"
|
||||
},
|
||||
"steps": {
|
||||
"create-project": {
|
||||
"status": "completed",
|
||||
"started_at": "2025-02-01T10:00:00Z",
|
||||
"completed_at": "2025-02-01T10:00:05Z",
|
||||
"output": {
|
||||
"project_id": "test-landing",
|
||||
"domain": "test-landing.threesix.ai"
|
||||
}
|
||||
},
|
||||
"wait-pipeline": {
|
||||
"status": "failed",
|
||||
"started_at": "2025-02-01T10:00:05Z",
|
||||
"completed_at": "2025-02-01T10:05:00Z",
|
||||
"error": "Pipeline #3 failed with status: failure"
|
||||
}
|
||||
},
|
||||
"last_completed_step": "create-project"
|
||||
}
|
||||
```
|
||||
|
||||
### Checkpoint Status Values
|
||||
|
||||
- `pending` - Tree started but no steps completed
|
||||
- `partial` - Some steps completed, some pending/failed
|
||||
- `completed` - All steps completed successfully
|
||||
- `failed` - A step failed with `on_error: fail`
|
||||
|
||||
## Creating a New Tree
|
||||
|
||||
1. Create `cookbooks/trees/<name>.yaml`
|
||||
2. Define steps with dependencies
|
||||
3. Add teardown section
|
||||
4. Test with `tree-runner.sh run <name> --project-name test-$(date +%s)`
|
||||
|
||||
### Best Practices
|
||||
|
||||
- **Always include teardown** - Clean up resources even if the tree fails
|
||||
- **Use descriptive step names** - They appear in status output
|
||||
- **Set on_error: continue for non-critical steps** - Pipeline failures shouldn't block site verification
|
||||
- **Capture outputs** - Pass data between steps via outputs, not hardcoded values
|
||||
- **Use vars for inputs** - Makes trees reusable with different parameters
|
||||
|
||||
## Migrating from Script to Tree
|
||||
|
||||
Compare script steps to tree steps:
|
||||
|
||||
| Script Pattern | Tree Equivalent |
|
||||
|----------------|-----------------|
|
||||
| `api_call POST /project "$json"` | `action: api`, `method: POST` |
|
||||
| `wait_for_pipeline "$project"` | `action: wait_pipeline` |
|
||||
| `wait_for_site "$domain" 30 5 "$project"` | `action: wait_site` |
|
||||
| `diagnose_pipeline_failure "$project"` | `action: diagnose`, `type: pipeline` |
|
||||
| `curl ... \| jq ...` | `action: shell`, `command: "..."` |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Tree not found
|
||||
```
|
||||
Error: Tree 'foo' not found
|
||||
Available trees: landing-page, composable-app, sdlc-flow
|
||||
```
|
||||
Check that `cookbooks/trees/foo.yaml` exists.
|
||||
|
||||
### yq not found
|
||||
```
|
||||
Error: yq is required but not installed
|
||||
```
|
||||
Install with `brew install yq`.
|
||||
|
||||
### Resume finds no checkpoint
|
||||
```
|
||||
No checkpoint found for tree 'landing-page'
|
||||
```
|
||||
Run `tree-runner.sh run landing-page ...` first.
|
||||
|
||||
### Step failed but outputs missing
|
||||
```
|
||||
Error: Output 'project_id' not found in step 'create-project'
|
||||
```
|
||||
The step may have failed silently. Check the checkpoint file:
|
||||
```bash
|
||||
cat cookbooks/.checkpoints/landing-page.json | jq '.steps["create-project"]'
|
||||
```
|
||||
|
||||
## Available Trees
|
||||
|
||||
### Basic Trees
|
||||
|
||||
| Tree | Description |
|
||||
|------|-------------|
|
||||
| `landing-page` | Single-page landing site with astro |
|
||||
| `composable-app` | Multi-component monorepo with service + app |
|
||||
| `sdlc-flow` | Feature lifecycle with SDLC orchestration |
|
||||
|
||||
### Slackpath Trees (Reference Architectures)
|
||||
|
||||
Progressive complexity paths for building Slack-like platforms:
|
||||
|
||||
| Tree | Description | Infrastructure |
|
||||
|------|-------------|----------------|
|
||||
| `slackpath-1-authenticated-service` | Identity layer: User auth, JWT, protected routes | CockroachDB |
|
||||
| `slackpath-2-async-worker-pipeline` | Background jobs: Producer/consumer with Redis | Redis |
|
||||
| `slackpath-3-realtime-chat` | WebSockets: Pub/sub broadcasting | Redis |
|
||||
| `slackpath-4-microservice-constellation` | Service mesh: Auth + Chat + Worker coordination | CockroachDB + Redis |
|
||||
|
||||
**Running a slackpath:**
|
||||
```bash
|
||||
./cookbooks/scripts/tree-runner.sh run slackpath-1-authenticated-service \
|
||||
--project-name auth-test-$(date +%s)
|
||||
```
|
||||
|
||||
These trees demonstrate:
|
||||
- Infrastructure provisioning (`type: postgres`, `type: redis`)
|
||||
- Automatic credential injection (`DATABASE_URL`, `REDIS_URL`)
|
||||
- SDLC-driven implementation via `/implement-feature` prompts
|
||||
- End-to-end verification scripts
|
||||
|
||||
## Files
|
||||
|
||||
```
|
||||
cookbooks/
|
||||
├── .checkpoints/ # Checkpoint storage (gitignored)
|
||||
│ └── landing-page.json
|
||||
├── scripts/
|
||||
│ ├── lib/
|
||||
│ │ ├── checkpoint.sh # Checkpoint I/O
|
||||
│ │ └── tree-parser.sh # YAML parsing
|
||||
│ └── tree-runner.sh # Main executable
|
||||
└── trees/
|
||||
├── landing-page.yaml
|
||||
├── composable-app.yaml
|
||||
├── sdlc-flow.yaml
|
||||
├── slackpath-1-authenticated-service.yaml
|
||||
├── slackpath-2-async-worker-pipeline.yaml
|
||||
├── slackpath-3-realtime-chat.yaml
|
||||
└── slackpath-4-microservice-constellation.yaml
|
||||
```
|
||||
|
||||
## Related
|
||||
|
||||
- [E2E Testing Strategy](./e2e-testing-strategy.md) — When to run trees, philosophy, history tracking
|
||||
- [Composable Monorepo Templates](./composable-monorepo.md) — Template structure tested by trees
|
||||
@ -35,6 +35,7 @@ When discussing code: "add to **platform**" = edit rdev; "add to **skeleton**" =
|
||||
| **Composable monorepo templates** | [services/composable-monorepo.md](.claude/guides/services/composable-monorepo.md) |
|
||||
| **E2E testing strategy** | [services/e2e-testing-strategy.md](.claude/guides/services/e2e-testing-strategy.md) |
|
||||
| **Cookbook tree system (commands)** | [services/cookbook-trees.md](.claude/guides/services/cookbook-trees.md) |
|
||||
| **Slackpath reference architectures** | [services/cookbook-trees.md](.claude/guides/services/cookbook-trees.md#slackpath-trees-reference-architectures) |
|
||||
| **Write E2E cookbook scripts** | [cookbook-scripts/SKILL.md](.claude/skills/cookbook-scripts/SKILL.md) |
|
||||
| **Build orchestration** | [services/build-orchestration.md](.claude/guides/services/build-orchestration.md) |
|
||||
| **Build event streaming** | [services/build-streaming.md](.claude/guides/services/build-streaming.md) |
|
||||
@ -101,6 +102,9 @@ go test ./...
|
||||
kubectl apply -f deployments/k8s/base/rdev-api.yaml
|
||||
kubectl rollout restart -n rdev deployment/rdev-api
|
||||
|
||||
# Deploy claudebox worker (when Dockerfile changes)
|
||||
./scripts/build-push.sh v0.4.0 claudebox && kubectl apply -f deployments/k8s/base/claudebox.yaml && kubectl rollout restart -n rdev statefulset/claudebox
|
||||
|
||||
# Verify pods
|
||||
kubectl get pods -n rdev
|
||||
|
||||
|
||||
@ -11,10 +11,12 @@ Composable Monorepo Templates evolve rdev's project scaffolding from single temp
|
||||
|
||||
**Key Facts:**
|
||||
- `POST /projects` creates monorepo skeleton (not single template)
|
||||
- `POST /projects/{id}/components` adds services/workers/apps/cli
|
||||
- `POST /projects/{id}/components` adds services/workers/apps/cli (code) or postgres/redis (infrastructure)
|
||||
- **Infrastructure provisioning:** `type: postgres` creates CockroachDB database, `type: redis` creates Redis cache
|
||||
- **Automatic credential injection:** Components deployed after infrastructure get `DATABASE_URL`, `REDIS_URL` as env vars
|
||||
- Convention-based discovery: `services/*/`, `workers/*/`, `apps/*/`, `cli/*/`
|
||||
- Optional `component.yaml` per component for ports, dependencies, build order
|
||||
- Shared `pkg/` from Aeries chassis + Colix patterns (8 packages)
|
||||
- Shared `pkg/` from Aeries chassis + Colix patterns (8+ packages including queue, auth, database, realtime)
|
||||
- Deployment supports whole-monorepo or individual-component targets
|
||||
- **CI is template-provided** - skeleton has `.woodpecker.yml.tmpl`, components have `.woodpecker.step.yml.tmpl`
|
||||
|
||||
@ -95,10 +97,25 @@ Combines best patterns from Aeries (chassis) and Colix (modular):
|
||||
|
||||
| Type | Directory | Template | Identifier |
|
||||
|------|-----------|----------|------------|
|
||||
| Service | `services/` | go-api | `Makefile` or `go.mod` |
|
||||
| Service | `services/` | service | `Makefile` or `go.mod` |
|
||||
| Worker | `workers/` | worker | `Makefile` or `go.mod` |
|
||||
| App | `apps/` | app-astro, app-react | `package.json` |
|
||||
| App | `apps/` | app-astro, app-react, app-nextjs | `package.json` |
|
||||
| CLI | `cli/` | cli | `Makefile` or `go.mod` |
|
||||
| Postgres | `infra/postgres` | (provisioned) | CockroachDB database |
|
||||
| Redis | `infra/redis` | (provisioned) | Redis cache with ACL |
|
||||
|
||||
## Infrastructure Provisioning
|
||||
|
||||
When you add `type: postgres` or `type: redis`:
|
||||
|
||||
1. **Database (postgres):** Creates CockroachDB database + user, stores `DATABASE_URL` in credential store
|
||||
2. **Cache (redis):** Creates Redis ACL user with scoped prefix, stores `REDIS_URL` and `REDIS_PREFIX`
|
||||
|
||||
**Credential Injection:** When code components (service, worker, app) are deployed, the system automatically fetches stored credentials and injects them as K8s secrets → env vars.
|
||||
|
||||
**File Pointers:**
|
||||
- Provisioning: `internal/service/component_infra.go`
|
||||
- Credential injection: `internal/service/component_deploy.go:fetchProjectCredentials()`
|
||||
|
||||
## Template Migration
|
||||
|
||||
|
||||
@ -24,7 +24,7 @@ Quick reference for rdev concepts and facts.
|
||||
| SSE Streaming | [features/sse-streaming.md](./features/sse-streaming.md) | High | 2025-01 | Real-time output streaming |
|
||||
| Infrastructure Management | [features/infrastructure.md](./features/infrastructure.md) | High | 2025-01 | Gitea, Cloudflare, deployment |
|
||||
| Build Orchestration | [features/build-orchestration.md](./features/build-orchestration.md) | High | 2026-01 | Bot-driven build specs with audit trail |
|
||||
| Composable Monorepo | [features/composable-monorepo.md](./features/composable-monorepo.md) | High | 2026-01 | Monorepo skeleton + component templates |
|
||||
| Composable Monorepo | [features/composable-monorepo.md](./features/composable-monorepo.md) | High | 2026-02 | Monorepo skeleton + component templates + infra provisioning |
|
||||
| **SDLC** |
|
||||
| SDLC Orchestration | [services/sdlc.md](./services/sdlc.md) | High | 2026-02 | Feature lifecycle, classifier engine, rdev API integration |
|
||||
|
||||
|
||||
11
changelog/v0.10.52.md
Normal file
11
changelog/v0.10.52.md
Normal file
@ -0,0 +1,11 @@
|
||||
# v0.10.52
|
||||
|
||||
**Released:** 2026-02-05
|
||||
|
||||
## Changes
|
||||
|
||||
feat: SDLC worker routing for skeleton projects with auto-init
|
||||
|
||||
---
|
||||
|
||||
**Image:** `ghcr.io/orchard9/rdev-api:v0.10.52`
|
||||
@ -24,7 +24,7 @@ spec:
|
||||
serviceAccountName: rdev-api
|
||||
containers:
|
||||
- name: rdev-api
|
||||
image: ghcr.io/orchard9/rdev-api:v0.10.51
|
||||
image: ghcr.io/orchard9/rdev-api:v0.10.52
|
||||
imagePullPolicy: Always
|
||||
|
||||
ports:
|
||||
|
||||
Loading…
Reference in New Issue
Block a user