rdev/docs/operations/deployment.md
jordan a9ad3d8304
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
chore: accumulated platform hardening and CI fixes
CI / Woodpecker:
- Add explicit depends_on to all .woodpecker.yml steps (rdev + templates)
- Fix skip_tls_verify -> skip-tls-verify (correct Kaniko flag name)
- Add replicasets get/list to deployer RBAC for rollout status
- Skeleton template: add failure:ignore on docs steps, Traefik TLS
  annotations on ingress, depends_on on verify step

Component templates:
- Fix container name in deploy steps (PROJECT_NAME-COMPONENT_NAME)
- Replace kubectl scale with kubectl patch for replicas
- Add post-deploy image verification and rollout status checks
- Applied consistently across all 5 component templates

Adapters:
- gitea: Add HTTP client timeout (30s), context cancellation checks,
  handle 404 on GetRepo/DeleteRepo
- zot: Add retry with exponential backoff (doWithRetry), limit response
  body reads to 10MB
- cockroach: Use net.JoinHostPort for IPv6-safe DSN construction
- woodpecker: Fix error wrapping (%v -> %w)
- redis: Fix error wrapping (%v -> %w)
- deployer: Add context cancellation checks

Services:
- apikey_service: Fix error wrapping (%v -> %w)
- component_deploy: Fix error wrapping (%v -> %w)
- project_infra: Fix error wrapping (%v -> %w)
- webhook/dispatcher: Fix error wrapping (%v -> %w)

Other:
- CLAUDE.md: Add guide links for Gitea, Go 1.25, Woodpecker v3,
  Traefik v3, Zot registry
- circuitbreaker: Add test for error wrapping
- docs: Update deployment, troubleshooting, and runbook docs
- health: Fix error wrapping (%v -> %w)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 23:16:56 -07:00

7.5 KiB

Deployment Guide

This guide covers deploying rdev API to the k3s cluster.

Prerequisites

# REQUIRED: Set kubeconfig before any kubectl command
export KUBECONFIG=~/.kube/orchard9-k3sf.yaml
  • k3s cluster (orchard9-k3sf)
  • kubectl configured with correct kubeconfig
  • PostgreSQL database
  • Container registry access (ghcr.io/orchard9)

Quick Deploy

# Release + deploy (recommended)
./scripts/release.sh v0.10.1 "Description of changes" --deploy

# Or manual deploy
kubectl apply -f deployments/k8s/base/rdev-api.yaml
kubectl rollout restart -n rdev deployment/rdev-api

# Verify deployment
kubectl -n rdev get pods
kubectl -n rdev get svc

Configuration

Environment Variables

Variable Description Required Default
PORT HTTP server port No 8080
POSTGRES_HOST Database host Yes -
POSTGRES_PORT Database port No 5432
POSTGRES_USER Database user Yes -
POSTGRES_PASSWORD Database password Yes -
POSTGRES_DB Database name No rdev
RDEV_NAMESPACE K8s namespace for pods No default
RATE_LIMIT_RPS Requests per second No 10
CONCURRENT_COMMANDS Max concurrent commands No 5

Secrets

Create a secret for database credentials:

kubectl -n rdev create secret generic rdev-api-secrets \
  --from-literal=postgres-password=your-password

Or use the manifest:

apiVersion: v1
kind: Secret
metadata:
  name: rdev-api-secrets
  namespace: rdev
type: Opaque
stringData:
  postgres-password: your-secure-password

ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  name: rdev-api-config
  namespace: rdev
data:
  POSTGRES_HOST: "postgres.databases.svc"
  POSTGRES_DB: "rdev"
  RDEV_NAMESPACE: "rdev"
  RATE_LIMIT_RPS: "10"
  CONCURRENT_COMMANDS: "5"

Kubernetes Manifests

Namespace

apiVersion: v1
kind: Namespace
metadata:
  name: rdev
  labels:
    app.kubernetes.io/name: rdev

Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: rdev-api
  namespace: rdev
spec:
  replicas: 2
  selector:
    matchLabels:
      app: rdev-api
  template:
    metadata:
      labels:
        app: rdev-api
    spec:
      serviceAccountName: rdev-api
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
      containers:
        - name: rdev-api
          image: your-registry/rdev-api:latest
          ports:
            - containerPort: 8080
          env:
            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: rdev-api-secrets
                  key: postgres-password
          envFrom:
            - configMapRef:
                name: rdev-api-config
          securityContext:
            readOnlyRootFilesystem: true
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - ALL
          resources:
            requests:
              memory: "128Mi"
              cpu: "100m"
            limits:
              memory: "512Mi"
              cpu: "500m"
          livenessProbe:
            httpGet:
              path: /health
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /ready
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 5

Service

apiVersion: v1
kind: Service
metadata:
  name: rdev-api
  namespace: rdev
spec:
  selector:
    app: rdev-api
  ports:
    - port: 80
      targetPort: 8080

Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: rdev-api
  namespace: rdev
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: websecure
    traefik.ingress.kubernetes.io/router.tls: "true"
spec:
  ingressClassName: traefik
  rules:
    - host: rdev.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: rdev-api
                port:
                  number: 80
  tls:
    - hosts:
        - rdev.example.com
      secretName: rdev-tls

RBAC

apiVersion: v1
kind: ServiceAccount
metadata:
  name: rdev-api
  namespace: rdev
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: rdev-api-role
  namespace: rdev
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: ["create"]
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: rdev-api-binding
  namespace: rdev
subjects:
  - kind: ServiceAccount
    name: rdev-api
roleRef:
  kind: Role
  name: rdev-api-role
  apiGroup: rbac.authorization.k8s.io

Pod Disruption Budget

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: rdev-api-pdb
  namespace: rdev
spec:
  minAvailable: 1
  selector:
    matchLabels:
      app: rdev-api

Network Policy

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: rdev-api-policy
  namespace: rdev
spec:
  podSelector:
    matchLabels:
      app: rdev-api
  policyTypes:
    - Ingress
    - Egress
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: ingress-nginx
  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: databases
      ports:
        - protocol: TCP
          port: 5432

Database Setup

Create Database

CREATE DATABASE rdev;
CREATE USER rdev_user WITH PASSWORD 'secure-password';
GRANT ALL PRIVILEGES ON DATABASE rdev TO rdev_user;

Migrations

Migrations run automatically on startup. To run manually:

# Connect to pod
kubectl -n rdev exec -it deployment/rdev-api -- sh

# Check migration status
psql $DATABASE_URL -c "SELECT * FROM schema_migrations;"

Scaling

Horizontal Pod Autoscaler

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: rdev-api-hpa
  namespace: rdev
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: rdev-api
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

Upgrading

Rolling Update

# Update image
kubectl -n rdev set image deployment/rdev-api \
  rdev-api=your-registry/rdev-api:new-version

# Watch rollout
kubectl -n rdev rollout status deployment/rdev-api

Rollback

# Rollback to previous version
kubectl -n rdev rollout undo deployment/rdev-api

# Rollback to specific revision
kubectl -n rdev rollout undo deployment/rdev-api --to-revision=2

Health Checks

Liveness

curl http://rdev-api/health

Returns 200 OK if the service is running.

Readiness

curl http://rdev-api/ready

Returns 200 OK if database and K8s are connected.

Troubleshooting

Pod Not Starting

# Check pod events
kubectl -n rdev describe pod -l app=rdev-api

# Check logs
kubectl -n rdev logs -l app=rdev-api

Database Connection Failed

  1. Check secret is mounted correctly
  2. Verify database host is reachable
  3. Check network policy allows egress

K8s API Errors

  1. Verify ServiceAccount has correct RBAC
  2. Check namespace configuration
  3. Verify API server connectivity