jordan bc47e426b0 feat: Add CI pipeline proxy, DNS alias management, and worker executor system

- Add ListPipelines/GetPipeline to CIProvider port with Woodpecker adapter
- Add DNS alias endpoints: GET/POST/DELETE /projects/{id}/domains
- Implement worker executor daemon, build executor, and git operations
- Add build service, worker service, and build audit tracking
- Add worker registry with PostgreSQL adapter and migration
- Add multi-provider code agent interface (Claude Code + OpenCode)
- Add create-and-build combo endpoint
- Update landing-page cookbook to reflect all gaps closed
- Fix tech debt: unified validation, auth scopes, error wrapping, slog patterns

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

2026-01-27 21:05:28 -07:00

12 KiB

Raw Blame History

Landing Page Cookbook

Deploy a static landing page through the threesix.ai infrastructure with agent-driven development.

Overview

This cookbook creates and deploys a simple landing page using the full threesix.ai autonomous infrastructure:

POST /project → Gitea repo + DNS + Woodpecker CI + template seed → Claude agent → git push → CI build → K8s deployment

Target: landing.threesix.ai (with future DNS aliases for www/root) Stack: Astro (static site generator) via astro-landing template Status: Coming Soon page

What's Automated Today

POST /project orchestrates the full infrastructure setup in a single call:

Step	Status	How
Gitea repo creation	Automated	`port.GitRepository` adapter
DNS A record	Automated	`port.DNSProvider` (Cloudflare) adapter
Woodpecker CI activation	Automated	`port.CIProvider` adapter, called during project creation
Template seeding	Automated	`port.TemplateProvider` with `astro-landing` template
K8s deployment	Automated	`port.Deployer` adapter (triggered by CI webhook)

Full Pipeline Status

All infrastructure gaps have been closed. The full pipeline from project creation through code generation, CI monitoring, and multi-domain DNS is operational:

Capability	Endpoint	Status
Project creation	`POST /project`	Operational
Code generation (worker)	`POST /projects/{id}/builds`	Operational
Create + build combo	`POST /project/create-and-build`	Operational
CI pipeline monitoring	`GET /projects/{id}/pipelines`	Operational
DNS alias management	`POST /projects/{id}/domains`	Operational

Prerequisites

Credentials Required

Secret	Location	Purpose
RDEV_ADMIN_KEY	`rdev-credentials` secret	rdev-api authentication
GITEA_TOKEN	`rdev-credentials` secret	Gitea API access
WOODPECKER_API_TOKEN	`rdev-credentials` secret	Woodpecker repo activation
CLOUDFLARE_API_TOKEN	`rdev-credentials` secret	DNS management

Infrastructure Required

rdev-api running with infrastructure handlers
Gitea at https://git.threesix.ai
Woodpecker CI at https://ci.threesix.ai
Zot registry at zot.threesix.svc.cluster.local:5000
projects namespace in K8s with RBAC
Wildcard TLS cert for *.threesix.ai

Architecture

┌──────────────────────────────────────────────────────────────────────┐
│                       Landing Page Flow                              │
│                                                                      │
│  1. Create Project (single API call)                                │
│     POST /project {"name": "landing", "template": "astro-landing"} │
│          │                                                           │
│          ├──▶ Creates Gitea repo: threesix/landing                  │
│          ├──▶ Creates DNS: landing.threesix.ai → cluster IP         │
│          ├──▶ Activates Woodpecker CI (auto)                        │
│          └──▶ Seeds repo with astro-landing template                │
│                                                                      │
│  2. Generate Code (3 options)                                       │
│     Via worker pool, claudebox, or local Claude Code                │
│          │                                                           │
│          ├──▶ Customizes Astro landing page                         │
│          ├──▶ Commits and pushes to Gitea                           │
│          └──▶ Worker executor polls queue, dispatches to agent      │
│                                                                      │
│  3. CI/CD Pipeline (automatic on push)                              │
│     Woodpecker triggered by git push                                │
│          │                                                           │
│          ├──▶ npm install + npm build                               │
│          ├──▶ Docker build (nginx)                                  │
│          ├──▶ Push to Zot registry                                  │
│          └──▶ kubectl set image (deploy)                            │
│                                                                      │
│  4. Live at https://landing.threesix.ai                             │
│                                                                      │
└──────────────────────────────────────────────────────────────────────┘

Step-by-Step Implementation

Step 1: Create Project via rdev-api

This single call creates the Gitea repo, DNS record, activates Woodpecker CI, and seeds the repo with the astro-landing template.

curl -X POST https://rdev.masq-ops.orchard9.ai/project \
  -H "Authorization: Bearer $RDEV_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "landing",
    "description": "threesix.ai landing page",
    "template": "astro-landing"
  }'

Response:

{
  "project_id": "landing",
  "name": "landing",
  "description": "threesix.ai landing page",
  "git": {
    "owner": "threesix",
    "name": "landing",
    "clone_ssh": "git@git.threesix.ai:threesix/landing.git",
    "clone_http": "https://git.threesix.ai/threesix/landing.git",
    "html_url": "https://git.threesix.ai/threesix/landing"
  },
  "domain": "landing.threesix.ai",
  "url": "https://landing.threesix.ai",
  "next_steps": []
}

If any infrastructure step fails, next_steps will contain manual instructions for that step. The remaining steps still execute.

Step 2: Generate Code

The astro-landing template seeds the repo with a working Astro project, Dockerfile, nginx.conf, and .woodpecker.yml. You can customize it via Claude.

Option A: Local Claude Code (recommended for now)

git clone https://git.threesix.ai/threesix/landing.git
cd landing
# Use Claude Code to customize the landing page
# Then commit and push

Option B: Via existing claudebox

curl -X POST "https://rdev.masq-ops.orchard9.ai/projects/pantheon/claude" \
  -H "Authorization: Bearer $RDEV_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "Clone https://git.threesix.ai/threesix/landing.git to /tmp/landing, then customize the Astro landing page with: Coming Soon message, threesix.ai branding (dark theme, gradient background), responsive layout. Commit and push when done."
  }'

Option C: Via work queue (build endpoint)

# Enqueue a build task for a worker to execute
curl -X POST "https://rdev.masq-ops.orchard9.ai/projects/landing/builds" \
  -H "Authorization: Bearer $RDEV_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "Customize the landing page with Coming Soon message and threesix.ai branding",
    "auto_commit": true,
    "auto_push": true
  }'

Option D: Create + build in one call

curl -X POST "https://rdev.masq-ops.orchard9.ai/project/create-and-build" \
  -H "Authorization: Bearer $RDEV_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "landing",
    "description": "threesix.ai landing page",
    "template": "astro-landing",
    "build": {
      "prompt": "Customize with Coming Soon message and threesix.ai branding",
      "auto_commit": true,
      "auto_push": true
    }
  }'

Step 3: Monitor Build

The git push triggers Woodpecker CI automatically.

Via rdev-api (recommended):

# List recent pipelines
curl -s "https://rdev.masq-ops.orchard9.ai/projects/landing/pipelines" \
  -H "Authorization: Bearer $RDEV_KEY" | jq '.data.pipelines'

# Get specific pipeline
curl -s "https://rdev.masq-ops.orchard9.ai/projects/landing/pipelines/1" \
  -H "Authorization: Bearer $RDEV_KEY" | jq '.data'

Via Woodpecker UI:

https://ci.threesix.ai/threesix/landing

Step 4: Verify Deployment

# Check project status via rdev-api
curl -s "https://rdev.masq-ops.orchard9.ai/project/landing" \
  -H "Authorization: Bearer $RDEV_KEY" | jq '.data.deployment'

# Check deployment via K8s
export KUBECONFIG=~/.kube/orchard9-k3sf.yaml
kubectl get deploy -n projects landing

# Check the site
curl -I https://landing.threesix.ai

Step 5: Configure DNS Aliases (Optional)

Point www.threesix.ai and threesix.ai to the landing page via the rdev-api domain alias endpoints.

# Add www.threesix.ai as a CNAME alias
curl -X POST "https://rdev.masq-ops.orchard9.ai/projects/landing/domains" \
  -H "Authorization: Bearer $RDEV_KEY" \
  -H "Content-Type: application/json" \
  -d '{"domain": "www.threesix.ai", "type": "CNAME"}'

# Add root A record
curl -X POST "https://rdev.masq-ops.orchard9.ai/projects/landing/domains" \
  -H "Authorization: Bearer $RDEV_KEY" \
  -H "Content-Type: application/json" \
  -d '{"domain": "threesix.ai"}'

# List all domains for the project
curl -s "https://rdev.masq-ops.orchard9.ai/projects/landing/domains" \
  -H "Authorization: Bearer $RDEV_KEY" | jq '.data.domains'

# Remove an alias
curl -X DELETE "https://rdev.masq-ops.orchard9.ai/projects/landing/domains/www.threesix.ai" \
  -H "Authorization: Bearer $RDEV_KEY"

Template: astro-landing

The astro-landing template (deployments/k8s/base/templates/astro-landing/) seeds the repo with:

File	Purpose
`.woodpecker.yml`	CI pipeline: npm build, Docker build, push, deploy
`.claude/CLAUDE.md`	Project instructions for Claude Code
`Dockerfile`	Multi-stage build (Node 20 build, nginx serve)
`nginx.conf`	Production config with gzip, caching, SPA fallback
`package.json`	Astro 4.0+ with Tailwind CSS
`astro.config.mjs`	Astro configuration
`tailwind.config.mjs`	Tailwind configuration
`src/pages/index.astro`	Landing page (dark theme with gradient)
`src/layouts/Layout.astro`	Base HTML layout
`README.md`	Development and deployment docs

Variables substituted during seeding: {{PROJECT_NAME}}, {{DOMAIN}}, {{GIT_URL}}

Implementation Status

All components for the full landing page pipeline are implemented:

Component	Location	Status
Work queue (enqueue/dequeue)	`internal/adapter/postgres/work_queue.go`	Implemented
Worker registry	`internal/adapter/postgres/worker_registry.go`	Implemented
Build audit tracking	`internal/adapter/postgres/build_audit.go`	Implemented
Build service	`internal/service/build_service.go`	Implemented
Worker service	`internal/service/worker_service.go`	Implemented
Work handlers (REST)	`internal/handlers/work.go`	Implemented
Code agent interface	`internal/port/code_agent.go`	Implemented
Worker executor daemon	`internal/worker/work_executor.go`	Implemented
BuildSpec-to-agent bridge	`internal/worker/build_executor.go`	Implemented
Git credential resolution	`internal/service/credential_service.go`	Implemented
DNS alias endpoints	`internal/handlers/infrastructure_domains.go`	Implemented
CI pipeline proxy	`internal/handlers/infrastructure_pipelines.go`	Implemented
Create-and-build endpoint	`internal/handlers/create_and_build.go`	Implemented

Verification

After deployment, verify:

# Check DNS
dig landing.threesix.ai

# Check site
curl -I https://landing.threesix.ai

# Check deployment status
curl https://rdev.masq-ops.orchard9.ai/project/landing \
  -H "Authorization: Bearer $RDEV_KEY"

Rollback

To remove the landing page:

# Delete via rdev-api (removes DNS, K8s deployment; Gitea repo preserved for safety)
curl -X DELETE https://rdev.masq-ops.orchard9.ai/project/landing \
  -H "Authorization: Bearer $RDEV_KEY"

Build Orchestration - Build system documentation
Worker Pool - Worker pool management
Work Queue - Work queue guide
Templates - Project template guide

12 KiB Raw Blame History