rdev/docs/operations/deployment.md
jordan 72d16929ca feat: Implement hexagonal architecture with services, webhooks, queue, and telemetry
Major refactoring to hexagonal (ports & adapters) architecture:

- Add service layer (apikey_service, project_service) for business logic
- Add webhook system with dispatcher and delivery tracking
- Add command queue with priority-based processing
- Add rate limiting with sliding window algorithm
- Add audit logging for command execution
- Add OpenTelemetry integration (traces, metrics, spans)
- Add circuit breaker for fault tolerance
- Add cached repository wrapper for performance
- Add comprehensive validation package
- Add Kubernetes client integration for pod management
- Add database migrations (allowed_ips, audit_log, rate_limiting, queue, webhooks)
- Add network policy and PodDisruptionBudget for k8s
- Remove legacy executor and projects/registry packages
- Untrack secrets.yaml (now managed via envault)
- Add coverage.out to .gitignore
- Add e2e test infrastructure with docker-compose
- Add comprehensive documentation (API, architecture, operations, plans)
- Add golangci-lint config and pre-commit hook

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 19:57:46 -07:00

7.2 KiB

Deployment Guide

This guide covers deploying rdev API to a Kubernetes cluster.

Prerequisites

  • Kubernetes cluster (1.24+)
  • kubectl configured
  • PostgreSQL database
  • Container registry access

Quick Deploy

# Apply all manifests
kubectl apply -k deployments/k8s/base/

# Verify deployment
kubectl -n rdev get pods
kubectl -n rdev get svc

Configuration

Environment Variables

Variable Description Required Default
PORT HTTP server port No 8080
POSTGRES_HOST Database host Yes -
POSTGRES_PORT Database port No 5432
POSTGRES_USER Database user Yes -
POSTGRES_PASSWORD Database password Yes -
POSTGRES_DB Database name No rdev
RDEV_NAMESPACE K8s namespace for pods No default
RATE_LIMIT_RPS Requests per second No 10
CONCURRENT_COMMANDS Max concurrent commands No 5

Secrets

Create a secret for database credentials:

kubectl -n rdev create secret generic rdev-api-secrets \
  --from-literal=postgres-password=your-password

Or use the manifest:

apiVersion: v1
kind: Secret
metadata:
  name: rdev-api-secrets
  namespace: rdev
type: Opaque
stringData:
  postgres-password: your-secure-password

ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  name: rdev-api-config
  namespace: rdev
data:
  POSTGRES_HOST: "postgres.databases.svc"
  POSTGRES_DB: "rdev"
  RDEV_NAMESPACE: "rdev"
  RATE_LIMIT_RPS: "10"
  CONCURRENT_COMMANDS: "5"

Kubernetes Manifests

Namespace

apiVersion: v1
kind: Namespace
metadata:
  name: rdev
  labels:
    app.kubernetes.io/name: rdev

Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: rdev-api
  namespace: rdev
spec:
  replicas: 2
  selector:
    matchLabels:
      app: rdev-api
  template:
    metadata:
      labels:
        app: rdev-api
    spec:
      serviceAccountName: rdev-api
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
      containers:
        - name: rdev-api
          image: your-registry/rdev-api:latest
          ports:
            - containerPort: 8080
          env:
            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: rdev-api-secrets
                  key: postgres-password
          envFrom:
            - configMapRef:
                name: rdev-api-config
          securityContext:
            readOnlyRootFilesystem: true
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - ALL
          resources:
            requests:
              memory: "128Mi"
              cpu: "100m"
            limits:
              memory: "512Mi"
              cpu: "500m"
          livenessProbe:
            httpGet:
              path: /health
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /ready
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 5

Service

apiVersion: v1
kind: Service
metadata:
  name: rdev-api
  namespace: rdev
spec:
  selector:
    app: rdev-api
  ports:
    - port: 80
      targetPort: 8080

Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: rdev-api
  namespace: rdev
  annotations:
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
spec:
  ingressClassName: nginx
  rules:
    - host: rdev.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: rdev-api
                port:
                  number: 80
  tls:
    - hosts:
        - rdev.example.com
      secretName: rdev-tls

RBAC

apiVersion: v1
kind: ServiceAccount
metadata:
  name: rdev-api
  namespace: rdev
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: rdev-api-role
  namespace: rdev
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: ["create"]
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: rdev-api-binding
  namespace: rdev
subjects:
  - kind: ServiceAccount
    name: rdev-api
roleRef:
  kind: Role
  name: rdev-api-role
  apiGroup: rbac.authorization.k8s.io

Pod Disruption Budget

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: rdev-api-pdb
  namespace: rdev
spec:
  minAvailable: 1
  selector:
    matchLabels:
      app: rdev-api

Network Policy

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: rdev-api-policy
  namespace: rdev
spec:
  podSelector:
    matchLabels:
      app: rdev-api
  policyTypes:
    - Ingress
    - Egress
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: ingress-nginx
  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: databases
      ports:
        - protocol: TCP
          port: 5432

Database Setup

Create Database

CREATE DATABASE rdev;
CREATE USER rdev_user WITH PASSWORD 'secure-password';
GRANT ALL PRIVILEGES ON DATABASE rdev TO rdev_user;

Migrations

Migrations run automatically on startup. To run manually:

# Connect to pod
kubectl -n rdev exec -it deployment/rdev-api -- sh

# Check migration status
psql $DATABASE_URL -c "SELECT * FROM schema_migrations;"

Scaling

Horizontal Pod Autoscaler

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: rdev-api-hpa
  namespace: rdev
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: rdev-api
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

Upgrading

Rolling Update

# Update image
kubectl -n rdev set image deployment/rdev-api \
  rdev-api=your-registry/rdev-api:new-version

# Watch rollout
kubectl -n rdev rollout status deployment/rdev-api

Rollback

# Rollback to previous version
kubectl -n rdev rollout undo deployment/rdev-api

# Rollback to specific revision
kubectl -n rdev rollout undo deployment/rdev-api --to-revision=2

Health Checks

Liveness

curl http://rdev-api/health

Returns 200 OK if the service is running.

Readiness

curl http://rdev-api/ready

Returns 200 OK if database and K8s are connected.

Troubleshooting

Pod Not Starting

# Check pod events
kubectl -n rdev describe pod -l app=rdev-api

# Check logs
kubectl -n rdev logs -l app=rdev-api

Database Connection Failed

  1. Check secret is mounted correctly
  2. Verify database host is reachable
  3. Check network policy allows egress

K8s API Errors

  1. Verify ServiceAccount has correct RBAC
  2. Check namespace configuration
  3. Verify API server connectivity