rdev/docs/plans/THREESIX_INFRASTRUCTURE.md

# threesix.ai Infrastructure Implementation Plan

> Self-hosted git, CI/CD, and deployment infrastructure for agent-driven development.

## Overview

Replace GitHub dependency with self-hosted infrastructure on k3s:
- **Gitea** - Git server (full-featured, web UI, native Woodpecker integration)
- **Zot** - Container registry (OCI-native)
- **Woodpecker** - CI/CD pipelines
- **rdev-api** - Orchestration layer with DNS management

## Architecture

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                              threesix.ai                                     │
│                                                                              │
│  git.threesix.ai ──────▶ gitea (web UI + SSH :22)                           │
│  registry.threesix.ai ─▶ zot (internal only, HTTPS for UI)                  │
│  ci.threesix.ai ───────▶ woodpecker (web UI)                                │
│  *.threesix.ai ────────▶ project deployments                                │
│                                                                              │
└─────────────────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────────────────┐
│                           k3s cluster                                        │
│                                                                              │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐                   │
│  │    gitea     │───▶│  woodpecker  │───▶│     zot      │                   │
│  │  (git repos) │    │  (CI/CD)     │    │  (registry)  │                   │
│  └──────────────┘    └──────────────┘    └──────────────┘                   │
│         │                   │                   │                            │
│         │                   ▼                   │                            │
│         │            ┌──────────────┐           │                            │
│         └───────────▶│   rdev-api   │◀──────────┘                           │
│                      │              │                                        │
│                      │ - Create repos                                        │
│                      │ - Deploy apps                                         │
│                      │ - Manage DNS                                          │
│                      └──────────────┘                                        │
│                             │                                                │
│                             ▼                                                │
│                      ┌──────────────┐                                        │
│                      │  Cloudflare  │                                        │
│                      │  DNS API     │                                        │
│                      └──────────────┘                                        │
│                                                                              │
└─────────────────────────────────────────────────────────────────────────────┘
```

## Configuration

### Credentials (from .secrets)

| Key | Value | Purpose |
|-----|-------|---------|
| CLOUDFLARE_API_TOKEN | `nGoDhG6Za...` | DNS management |
| CLOUDFLARE_ZONE_ID | `e0bc8d51...` | threesix.ai zone |
| GITEA_ADMIN_PASSWORD | (generate) | Gitea admin login |
| GITEA_SECRET_KEY | (generate: `openssl rand -hex 32`) | Gitea internal security |
| GITEA_API_TOKEN | (create in Gitea UI) | rdev-api access to Gitea |
| GITEA_OAUTH_CLIENT_ID | (create in Gitea UI) | Woodpecker OAuth |
| GITEA_OAUTH_CLIENT_SECRET | (create in Gitea UI) | Woodpecker OAuth |
| WOODPECKER_AGENT_SECRET | (generate: `openssl rand -hex 32`) | Agent-server auth |

### Network

| Resource | Value |
|----------|-------|
| External IP | 208.122.204.172 |
| Let's Encrypt Email | jordan@threesix.ai |
| Domain | threesix.ai |

### Admin Access

Admin SSH key (add in Gitea UI: Settings → SSH/GPG Keys):
```
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDZwQF0Ro0E0foFo0oro/NrfUb5abEec/A0OP2qO8dVn jordanwashburn@jordanmacstudio.lan
```

---

## Phase 1: Foundation (K8s Infrastructure)

### 1.1 Create Namespace and Secrets

```yaml
# deployments/k8s/base/threesix/namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: threesix
---
# Cloudflare API secret for cert-manager and rdev-api
apiVersion: v1
kind: Secret
metadata:
  name: cloudflare-api
  namespace: threesix
type: Opaque
stringData:
  api-token: "${CLOUDFLARE_API_TOKEN}"
  zone-id: "${CLOUDFLARE_ZONE_ID}"
```

### 1.2 Configure cert-manager for Wildcard Certs

```yaml
# deployments/k8s/base/threesix/cluster-issuer.yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-threesix
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: jordan@threesix.ai
    privateKeySecretRef:
      name: letsencrypt-threesix-account
    solvers:
    - dns01:
        cloudflare:
          apiTokenSecretRef:
            name: cloudflare-api
            key: api-token
      selector:
        dnsZones:
        - "threesix.ai"
---
# Wildcard certificate
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: threesix-wildcard
  namespace: threesix
spec:
  secretName: threesix-wildcard-tls
  issuerRef:
    name: letsencrypt-threesix
    kind: ClusterIssuer
  dnsNames:
  - "threesix.ai"
  - "*.threesix.ai"
```

### 1.3 Deploy Gitea

```yaml
# deployments/k8s/base/threesix/gitea.yaml
apiVersion: v1
kind: Secret
metadata:
  name: gitea-admin
  namespace: threesix
type: Opaque
stringData:
  username: jordan
  password: "${GITEA_ADMIN_PASSWORD}"
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: gitea-config
  namespace: threesix
data:
  app.ini: |
    APP_NAME = threesix
    RUN_MODE = prod

    [server]
    DOMAIN = git.threesix.ai
    SSH_DOMAIN = git.threesix.ai
    ROOT_URL = https://git.threesix.ai/
    HTTP_PORT = 3000
    SSH_PORT = 22
    SSH_LISTEN_PORT = 22
    LFS_START_SERVER = true

    [database]
    DB_TYPE = sqlite3
    PATH = /data/gitea/gitea.db

    [repository]
    ROOT = /data/git/repositories
    DEFAULT_BRANCH = main

    [security]
    INSTALL_LOCK = true
    SECRET_KEY = ${GITEA_SECRET_KEY}

    [service]
    DISABLE_REGISTRATION = true
    REQUIRE_SIGNIN_VIEW = false

    [oauth2_client]
    ENABLE_AUTO_REGISTRATION = false

    [webhook]
    ALLOWED_HOST_LIST = woodpecker-server.threesix.svc.cluster.local    
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: gitea
  namespace: threesix
spec:
  serviceName: gitea
  replicas: 1
  selector:
    matchLabels:
      app: gitea
  template:
    metadata:
      labels:
        app: gitea
    spec:
      initContainers:
      - name: init-config
        image: gitea/gitea:latest
        command: ['sh', '-c', 'cp /etc/gitea/app.ini /data/gitea/conf/app.ini']
        volumeMounts:
        - name: data
          mountPath: /data
        - name: config
          mountPath: /etc/gitea
      containers:
      - name: gitea
        image: gitea/gitea:latest
        ports:
        - containerPort: 3000
          name: http
        - containerPort: 22
          name: ssh
        env:
        - name: GITEA_ADMIN_USERNAME
          valueFrom:
            secretKeyRef:
              name: gitea-admin
              key: username
        - name: GITEA_ADMIN_PASSWORD
          valueFrom:
            secretKeyRef:
              name: gitea-admin
              key: password
        volumeMounts:
        - name: data
          mountPath: /data
        resources:
          requests:
            memory: "256Mi"
            cpu: "100m"
          limits:
            memory: "1Gi"
            cpu: "1000m"
      volumes:
      - name: config
        configMap:
          name: gitea-config
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: ["ReadWriteOnce"]
      storageClassName: longhorn
      resources:
        requests:
          storage: 20Gi
---
apiVersion: v1
kind: Service
metadata:
  name: gitea
  namespace: threesix
spec:
  selector:
    app: gitea
  ports:
  - name: http
    port: 3000
    targetPort: 3000
  - name: ssh
    port: 22
    targetPort: 22
---
# External SSH access via LoadBalancer
apiVersion: v1
kind: Service
metadata:
  name: gitea-ssh
  namespace: threesix
spec:
  type: LoadBalancer
  selector:
    app: gitea
  ports:
  - name: ssh
    port: 22
    targetPort: 22
---
# HTTP access via Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gitea
  namespace: threesix
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-threesix
spec:
  ingressClassName: traefik
  tls:
  - hosts:
    - git.threesix.ai
    secretName: git-threesix-tls
  rules:
  - host: git.threesix.ai
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: gitea
            port:
              number: 3000
```

### 1.4 Deploy Zot Registry

```yaml
# deployments/k8s/base/threesix/zot.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: zot-config
  namespace: threesix
data:
  config.json: |
    {
      "distSpecVersion": "1.1.0",
      "storage": {
        "rootDirectory": "/var/lib/zot",
        "gc": true,
        "gcDelay": "1h"
      },
      "http": {
        "address": "0.0.0.0",
        "port": "5000"
      },
      "log": {
        "level": "info"
      },
      "extensions": {
        "search": {
          "enable": true
        },
        "ui": {
          "enable": true
        }
      }
    }    
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: zot
  namespace: threesix
spec:
  serviceName: zot
  replicas: 1
  selector:
    matchLabels:
      app: zot
  template:
    metadata:
      labels:
        app: zot
    spec:
      containers:
      - name: zot
        image: ghcr.io/project-zot/zot-linux-amd64:latest
        ports:
        - containerPort: 5000
        volumeMounts:
        - name: data
          mountPath: /var/lib/zot
        - name: config
          mountPath: /etc/zot/config.json
          subPath: config.json
        resources:
          requests:
            memory: "128Mi"
            cpu: "100m"
          limits:
            memory: "512Mi"
            cpu: "1000m"
      volumes:
      - name: config
        configMap:
          name: zot-config
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: ["ReadWriteOnce"]
      storageClassName: longhorn
      resources:
        requests:
          storage: 50Gi
---
apiVersion: v1
kind: Service
metadata:
  name: zot
  namespace: threesix
spec:
  selector:
    app: zot
  ports:
  - port: 5000
    targetPort: 5000
---
# Internal DNS name for cluster access
# Pods can pull from: zot.threesix.svc.cluster.local:5000/image:tag
---
# Optional: External UI access
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: zot
  namespace: threesix
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-threesix
spec:
  ingressClassName: traefik
  tls:
  - hosts:
    - registry.threesix.ai
    secretName: registry-threesix-tls
  rules:
  - host: registry.threesix.ai
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: zot
            port:
              number: 5000
```

### 1.5 Initial DNS Records

Create via Cloudflare API or dashboard:

| Type | Name | Value | Proxy |
|------|------|-------|-------|
| A | git | 208.122.204.172 | No (SSH needs direct) |
| A | registry | 208.122.204.172 | No |
| A | ci | 208.122.204.172 | Yes (optional) |
| A | * | 208.122.204.172 | Yes (optional) |

---

## Phase 2: CI/CD (Woodpecker)

### 2.1 Create Gitea OAuth Application

Before deploying Woodpecker, create an OAuth application in Gitea:

1. Login to https://git.threesix.ai as admin
2. Go to Site Administration → Applications → Create OAuth2 Application
3. Application Name: `Woodpecker CI`
4. Redirect URI: `https://ci.threesix.ai/authorize`
5. Save the Client ID and Client Secret

### 2.2 Deploy Woodpecker Server

```yaml
# deployments/k8s/base/threesix/woodpecker-server.yaml
apiVersion: v1
kind: Secret
metadata:
  name: woodpecker-secrets
  namespace: threesix
type: Opaque
stringData:
  # Generate with: openssl rand -hex 32
  WOODPECKER_AGENT_SECRET: "${WOODPECKER_AGENT_SECRET}"
  # From Gitea OAuth application
  WOODPECKER_GITEA_CLIENT: "${GITEA_OAUTH_CLIENT_ID}"
  WOODPECKER_GITEA_SECRET: "${GITEA_OAUTH_CLIENT_SECRET}"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: woodpecker-server
  namespace: threesix
spec:
  replicas: 1
  selector:
    matchLabels:
      app: woodpecker-server
  template:
    metadata:
      labels:
        app: woodpecker-server
    spec:
      containers:
      - name: woodpecker
        image: woodpeckerci/woodpecker-server:latest
        ports:
        - containerPort: 8000
        env:
        - name: WOODPECKER_HOST
          value: "https://ci.threesix.ai"
        - name: WOODPECKER_OPEN
          value: "true"
        - name: WOODPECKER_ADMIN
          value: "jordan"
        # Gitea forge integration
        - name: WOODPECKER_GITEA
          value: "true"
        - name: WOODPECKER_GITEA_URL
          value: "https://git.threesix.ai"
        envFrom:
        - secretRef:
            name: woodpecker-secrets
        volumeMounts:
        - name: data
          mountPath: /var/lib/woodpecker
      volumes:
      - name: data
        persistentVolumeClaim:
          claimName: woodpecker-data
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: woodpecker-data
  namespace: threesix
spec:
  accessModes: ["ReadWriteOnce"]
  storageClassName: longhorn
  resources:
    requests:
      storage: 5Gi
---
apiVersion: v1
kind: Service
metadata:
  name: woodpecker-server
  namespace: threesix
spec:
  selector:
    app: woodpecker-server
  ports:
  - name: http
    port: 8000
    targetPort: 8000
  - name: grpc
    port: 9000
    targetPort: 9000
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: woodpecker
  namespace: threesix
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-threesix
spec:
  ingressClassName: traefik
  tls:
  - hosts:
    - ci.threesix.ai
    secretName: ci-threesix-tls
  rules:
  - host: ci.threesix.ai
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: woodpecker-server
            port:
              number: 8000
```

### 2.3 Deploy Woodpecker Agent (with Kaniko)

```yaml
# deployments/k8s/base/threesix/woodpecker-agent.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: woodpecker-agent
  namespace: threesix
spec:
  replicas: 2
  selector:
    matchLabels:
      app: woodpecker-agent
  template:
    metadata:
      labels:
        app: woodpecker-agent
    spec:
      containers:
      - name: agent
        image: woodpeckerci/woodpecker-agent:latest
        env:
        - name: WOODPECKER_SERVER
          value: "woodpecker-server.threesix.svc:9000"
        - name: WOODPECKER_BACKEND
          value: "kubernetes"
        - name: WOODPECKER_BACKEND_K8S_NAMESPACE
          value: "threesix"
        - name: WOODPECKER_BACKEND_K8S_STORAGE_CLASS
          value: "longhorn"
        - name: WOODPECKER_BACKEND_K8S_VOLUME_SIZE
          value: "10Gi"
        envFrom:
        - secretRef:
            name: woodpecker-secrets
      serviceAccountName: woodpecker-agent
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: woodpecker-agent
  namespace: threesix
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: woodpecker-agent
  namespace: threesix
rules:
- apiGroups: [""]
  resources: ["pods", "pods/log", "secrets", "configmaps", "persistentvolumeclaims"]
  verbs: ["*"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: woodpecker-agent
  namespace: threesix
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: woodpecker-agent
subjects:
- kind: ServiceAccount
  name: woodpecker-agent
  namespace: threesix
```

---

## Phase 3: rdev-api Extensions

### 3.1 New Port Interfaces

```go
// internal/port/git.go
package port

import "context"

// GitRepository manages git repositories via Gitea API.
type GitRepository interface {
    // CreateRepo creates a new git repository.
    CreateRepo(ctx context.Context, name, description string, private bool) (*Repo, error)

    // DeleteRepo deletes a repository.
    DeleteRepo(ctx context.Context, owner, name string) error

    // ListRepos returns all repositories for an owner.
    ListRepos(ctx context.Context, owner string) ([]*Repo, error)

    // GetRepo returns a single repository.
    GetRepo(ctx context.Context, owner, name string) (*Repo, error)

    // AddCollaborator adds a user as collaborator to a repo.
    AddCollaborator(ctx context.Context, owner, repo, username string, permission string) error

    // AddDeployKey adds a deploy key to a repo for read-only or read-write access.
    AddDeployKey(ctx context.Context, owner, repo, title, publicKey string, readOnly bool) error

    // CreateWebhook creates a webhook to trigger on push events.
    CreateWebhook(ctx context.Context, owner, repo, url, secret string, events []string) error
}

// Repo represents a git repository.
type Repo struct {
    ID          int64
    Owner       string
    Name        string
    FullName    string  // owner/name
    Description string
    Private     bool
    CloneSSH    string  // git@git.threesix.ai:owner/name.git
    CloneHTTP   string  // https://git.threesix.ai/owner/name.git
    HTMLURL     string  // https://git.threesix.ai/owner/name
    CreatedAt   time.Time
    UpdatedAt   time.Time
}
```

```go
// internal/port/dns.go
package port

import "context"

// DNSProvider manages DNS records.
type DNSProvider interface {
    // CreateRecord creates a DNS record.
    CreateRecord(ctx context.Context, record DNSRecord) error

    // DeleteRecord removes a DNS record.
    DeleteRecord(ctx context.Context, recordType, name string) error

    // ListRecords returns all records for the zone.
    ListRecords(ctx context.Context) ([]*DNSRecord, error)
}

// DNSRecord represents a DNS record.
type DNSRecord struct {
    Type    string // A, CNAME, TXT
    Name    string // subdomain or @ for root
    Content string // IP or target
    TTL     int    // seconds, 1 = auto
    Proxied bool   // Cloudflare proxy
}
```

```go
// internal/port/deployer.go
package port

import "context"

// Deployer manages application deployments.
type Deployer interface {
    // Deploy creates or updates a deployment.
    Deploy(ctx context.Context, spec DeploySpec) error

    // Undeploy removes a deployment.
    Undeploy(ctx context.Context, projectName string) error

    // GetStatus returns deployment status.
    GetStatus(ctx context.Context, projectName string) (*DeployStatus, error)
}

// DeploySpec defines a deployment.
type DeploySpec struct {
    ProjectName string
    Image       string
    Domain      string   // e.g., "myapp.threesix.ai"
    Port        int      // container port
    Replicas    int
    EnvVars     map[string]string
    Secrets     map[string]string
}

// DeployStatus represents current deployment state.
type DeployStatus struct {
    ProjectName    string
    Image          string
    Replicas       int
    ReadyReplicas  int
    URL            string
    Status         string // "running", "pending", "failed"
}
```

### 3.2 New Adapters

```
internal/adapter/
├── gitea/           # Gitea REST API client
│   └── client.go
├── cloudflare/      # Cloudflare DNS API client
│   └── client.go
├── deployer/        # K8s deployment manager
│   └── deployer.go
└── registry/        # Zot registry client (optional)
    └── client.go
```

#### Gitea Client Example

```go
// internal/adapter/gitea/client.go
package gitea

import (
    "code.gitea.io/sdk/gitea"
    "github.com/orchard9/rdev/internal/port"
)

type Client struct {
    client *gitea.Client
    owner  string // default owner/org for repos
}

func NewClient(url, token, defaultOwner string) (*Client, error) {
    client, err := gitea.NewClient(url, gitea.SetToken(token))
    if err != nil {
        return nil, err
    }
    return &Client{client: client, owner: defaultOwner}, nil
}

func (c *Client) CreateRepo(ctx context.Context, name, description string, private bool) (*port.Repo, error) {
    repo, _, err := c.client.CreateOrgRepo(c.owner, gitea.CreateRepoOption{
        Name:        name,
        Description: description,
        Private:     private,
        AutoInit:    true,
    })
    if err != nil {
        return nil, err
    }
    return &port.Repo{
        ID:          repo.ID,
        Owner:       repo.Owner.UserName,
        Name:        repo.Name,
        FullName:    repo.FullName,
        Description: repo.Description,
        Private:     repo.Private,
        CloneSSH:    repo.SSHURL,
        CloneHTTP:   repo.CloneURL,
        HTMLURL:     repo.HTMLURL,
        CreatedAt:   repo.Created,
        UpdatedAt:   repo.Updated,
    }, nil
}
```

### 3.3 New Handlers

```go
// internal/handlers/projects_git.go

// POST /projects/{id}/repo - Create git repo for project
// DELETE /projects/{id}/repo - Delete git repo
// GET /projects/{id}/repo - Get repo info

// POST /projects/{id}/deploy - Deploy project
// DELETE /projects/{id}/deploy - Undeploy project
// GET /projects/{id}/deploy/status - Get deployment status

// POST /projects/{id}/domain - Set custom domain
// DELETE /projects/{id}/domain - Remove custom domain
```

### 3.4 New API Endpoints

| Method | Path | Description |
|--------|------|-------------|
| POST | `/projects/{id}/repo` | Create git repo |
| DELETE | `/projects/{id}/repo` | Delete git repo |
| GET | `/projects/{id}/repo` | Get repo info (clone URLs) |
| POST | `/projects/{id}/deploy` | Deploy from image |
| DELETE | `/projects/{id}/deploy` | Remove deployment |
| GET | `/projects/{id}/deploy/status` | Deployment status |
| POST | `/projects/{id}/domain` | Add custom domain |
| DELETE | `/projects/{id}/domain` | Remove custom domain |

---

## Phase 4: Database Schema

### 4.1 Migration: Add Git and Deployment Fields

```sql
-- migrations/010_project_infrastructure.up.sql

-- Add infrastructure fields to projects
ALTER TABLE projects ADD COLUMN IF NOT EXISTS
    git_repo_name VARCHAR(255);

ALTER TABLE projects ADD COLUMN IF NOT EXISTS
    git_clone_ssh VARCHAR(512);

ALTER TABLE projects ADD COLUMN IF NOT EXISTS
    git_clone_http VARCHAR(512);

ALTER TABLE projects ADD COLUMN IF NOT EXISTS
    domain VARCHAR(255);

ALTER TABLE projects ADD COLUMN IF NOT EXISTS
    custom_domain VARCHAR(255);

ALTER TABLE projects ADD COLUMN IF NOT EXISTS
    deployment_image VARCHAR(512);

ALTER TABLE projects ADD COLUMN IF NOT EXISTS
    deployment_status VARCHAR(50) DEFAULT 'none';

ALTER TABLE projects ADD COLUMN IF NOT EXISTS
    deployment_replicas INTEGER DEFAULT 1;

-- Index for domain lookups
CREATE INDEX IF NOT EXISTS idx_projects_domain ON projects(domain);
CREATE INDEX IF NOT EXISTS idx_projects_custom_domain ON projects(custom_domain);
```

---

## Phase 5: Pantheon Integration

### 5.1 New Commands for Agents

```
/project create <name>
  → Creates project in DB
  → Creates git repo in Gitea (threesix/<name>)
  → Activates repo in Woodpecker CI
  → Creates DNS record (<name>.threesix.ai)
  → Returns clone URLs:
      SSH:   git@git.threesix.ai:threesix/<name>.git
      HTTPS: https://git.threesix.ai/threesix/<name>.git

/project deploy <name>
  → Triggers build from latest commit
  → Deploys to k8s
  → Returns live URL

/project status <name>
  → Shows git repo, deployment status, URLs

/project domain <name> <custom-domain>
  → Adds custom domain to project
  → Instructions for DNS pointing
```

### 5.2 Webhook Flow

```
Agent pushes code to Gitea
        │
        ▼
Gitea receives push, fires webhook to Woodpecker
        │
        ▼
Woodpecker reads .woodpecker.yml from repo
        │
        ▼
Kaniko builds image, pushes to zot
        │
        ▼
Woodpecker calls rdev-api: POST /projects/{id}/deploy
        │
        ▼
rdev-api creates/updates K8s Deployment + Ingress
        │
        ▼
Project live at https://{name}.threesix.ai
```

**Note:** When you activate a repo in Woodpecker's UI, it automatically creates the webhook in Gitea via OAuth. No manual webhook configuration needed.

---

## Implementation Checklist

### Phase 1: Foundation ✅ COMPLETED (2026-01-26)
- [x] Create `threesix` namespace
- [x] Create Cloudflare API secret
- [x] Configure Issuer for DNS-01 challenge (namespace-scoped, not ClusterIssuer)
- [x] Request wildcard certificate (*.threesix.ai)
- [x] Deploy Gitea StatefulSet (rootless image, PostgreSQL backend, writable config)
- [x] Configure Gitea LoadBalancer for SSH (208.122.204.172:22)
- [x] Deploy Zot registry (10Gi storage)
- [x] Create DNS records: git.threesix.ai, registry.threesix.ai → 208.122.204.172
- [x] Test: `https://git.threesix.ai` shows Gitea UI ✅
- [x] Test: `https://registry.threesix.ai/v2/_catalog` returns `{"repositories":[]}` ✅
- [x] Complete Gitea installation wizard ✅

**Implementation Notes:**
- Used namespace-scoped `Issuer` instead of `ClusterIssuer` (cert-manager couldn't access secrets across namespaces)
- Gitea uses PostgreSQL (`postgres.databases.svc.cluster.local`) instead of SQLite
- Gitea credentials: `gitea` user, password in `/tmp/gitea-db-password.txt`
- Rootless Gitea image requires `securityContext.fsGroup: 1000` and writable `/etc/gitea` via volume subPath

### Phase 2: CI/CD ✅ COMPLETED (2026-01-26)
- [x] Create Gitea OAuth application for Woodpecker
- [x] Generate Woodpecker agent secret
- [x] Create DNS record: ci.threesix.ai → 208.122.204.172
- [x] Deploy Woodpecker server with Gitea forge
- [x] Deploy Woodpecker agents (2 replicas, K8s backend)
- [x] TLS certificate issued for ci.threesix.ai
- [ ] Test: Login to Woodpecker via Gitea OAuth
- [ ] Test: Activate repo in Woodpecker (auto-creates webhook)
- [ ] Test: Push triggers build
- [ ] Test: Kaniko builds and pushes to Zot

**Secrets saved:**
- Agent secret: `/tmp/woodpecker-agent-secret.txt`
- Gitea OAuth: Client ID `7548afec-43e0-486a-b6eb-e2a7d5c88d41`

### Phase 3: rdev-api ✅ COMPLETED (2026-01-26)
- [x] Add GitRepository port interface (`internal/port/git_repository.go`)
- [x] Add DNSProvider port interface (`internal/port/dns_provider.go`)
- [x] Add Deployer port interface (`internal/port/deployer.go`)
- [x] Implement Gitea adapter (`internal/adapter/gitea/client.go`) using `code.gitea.io/sdk/gitea`
- [x] Implement Cloudflare adapter (`internal/adapter/cloudflare/client.go`)
- [x] Implement K8s deployer adapter (`internal/adapter/deployer/deployer.go`)
- [x] Add database migration (`internal/db/migrations/008_project_infrastructure.sql`)
- [x] Add infrastructure handler (`internal/handlers/infrastructure.go`)
- [x] Wire up in main.go with environment variables
- [ ] Test: API can create repos
- [ ] Test: API can manage DNS
- [ ] Test: API can deploy apps

**New Environment Variables:**
```
GITEA_URL=https://git.threesix.ai
GITEA_TOKEN=<from Gitea UI: Settings → Applications → Access Tokens>
GITEA_DEFAULT_ORG=threesix
CLOUDFLARE_API_TOKEN=<existing>
CLOUDFLARE_ZONE_ID=<existing>
DEFAULT_DOMAIN=threesix.ai
DEPLOY_NAMESPACE=projects
DEPLOY_TLS_ISSUER=letsencrypt-threesix
CLUSTER_IP=208.122.204.172
```

**New API Endpoints:**
- `POST /projects/{id}/repo` - Create git repo
- `GET /projects/{id}/repo` - Get repo info
- `DELETE /projects/{id}/repo` - Delete repo
- `POST /projects/{id}/deploy` - Deploy from image
- `GET /projects/{id}/deploy/status` - Get deployment status
- `DELETE /projects/{id}/deploy` - Undeploy
- `POST /projects/{id}/deploy/restart` - Restart deployment
- `POST /projects/{id}/deploy/scale` - Scale replicas
- `GET /projects/{id}/deploy/logs` - Get logs
- `POST /projects/{id}/domain` - Add custom domain
- `DELETE /projects/{id}/domain` - Remove domain

### Phase 4: Integration ✅ COMPLETED (2026-01-26)
- [x] Wire up webhook: build → deploy (`internal/handlers/woodpecker_webhook.go`)
- [x] Add project commands to Pantheon (`.claude/commands/project-*.md`)
- [x] Create project infrastructure service (`internal/service/project_infra.go`)
- [x] Create project management handler (`internal/handlers/project_management.go`)
- [x] Update ExternalSecret with infrastructure credentials
- [x] Create .woodpecker.yml pipeline template
- [ ] Test: Login to Woodpecker via Gitea OAuth
- [ ] Test: end-to-end "create project" → "push code" → "live site"

**New Files Created:**
- `internal/handlers/woodpecker_webhook.go` - Handles Woodpecker CI webhooks for auto-deployment
- `internal/service/project_infra.go` - Orchestrates full project lifecycle (DB → Gitea → DNS → K8s)
- `internal/handlers/project_management.go` - HTTP handlers for project CRUD operations
- `.claude/commands/project-create.md` - /project create command
- `.claude/commands/project-status.md` - /project status command
- `.claude/commands/project-deploy.md` - /project deploy command
- `.claude/commands/project-list.md` - /project list command

**New API Endpoints:**
- `POST /project` - Create new project with git repo and DNS
- `GET /project` - List all projects with status
- `GET /project/{name}` - Get single project status
- `DELETE /project/{name}` - Delete project and all resources
- `POST /webhook/woodpecker` - Woodpecker CI webhook (auto-deploy on build success)

**Environment Variables Added:**
```
REGISTRY_URL=zot.threesix.svc.cluster.local:5000
WOODPECKER_WEBHOOK_SECRET=<from GCP Secret Manager>
```

### Phase 5: Polish
- [ ] Custom domain support
- [ ] Build notifications to Pantheon
- [ ] Deployment logs streaming
- [ ] Resource limits per project
- [ ] Usage metrics

---

## Resource Estimates

| Component | CPU Request | Memory Request | Storage |
|-----------|-------------|----------------|---------|
| Gitea | 100m | 256Mi | 20Gi |
| Zot | 100m | 128Mi | 50Gi |
| Woodpecker Server | 100m | 128Mi | 5Gi |
| Woodpecker Agent (x2) | 200m each | 256Mi each | - |
| **Total** | ~700m | ~1Gi | 75Gi |

---

## Security Considerations

1. **Gitea admin** - Registration disabled, only admin user can create accounts
2. **Gitea API token** - Create a dedicated token for rdev-api with repo scope
3. **Registry access** - Internal only, no auth needed (ClusterIP)
4. **Woodpecker** - OAuth via Gitea, inherits Gitea permissions
5. **Cloudflare token** - Scoped to DNS edit only
6. **Deploy permissions** - rdev-api ServiceAccount limited to `threesix` and `projects` namespaces

---

## Next Steps

1. Review this plan
2. I deploy Phase 1 infrastructure
3. Test git and registry
4. Deploy Phase 2 CI/CD
5. Implement Phase 3 rdev-api changes
6. Integration testing
7. Pantheon integration