rdev/docs/operations/deployment.md
jordan 72d16929ca feat: Implement hexagonal architecture with services, webhooks, queue, and telemetry
Major refactoring to hexagonal (ports & adapters) architecture:

- Add service layer (apikey_service, project_service) for business logic
- Add webhook system with dispatcher and delivery tracking
- Add command queue with priority-based processing
- Add rate limiting with sliding window algorithm
- Add audit logging for command execution
- Add OpenTelemetry integration (traces, metrics, spans)
- Add circuit breaker for fault tolerance
- Add cached repository wrapper for performance
- Add comprehensive validation package
- Add Kubernetes client integration for pod management
- Add database migrations (allowed_ips, audit_log, rate_limiting, queue, webhooks)
- Add network policy and PodDisruptionBudget for k8s
- Remove legacy executor and projects/registry packages
- Untrack secrets.yaml (now managed via envault)
- Add coverage.out to .gitignore
- Add e2e test infrastructure with docker-compose
- Add comprehensive documentation (API, architecture, operations, plans)
- Add golangci-lint config and pre-commit hook

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 19:57:46 -07:00

395 lines
7.2 KiB
Markdown

# Deployment Guide
This guide covers deploying rdev API to a Kubernetes cluster.
## Prerequisites
- Kubernetes cluster (1.24+)
- kubectl configured
- PostgreSQL database
- Container registry access
## Quick Deploy
```bash
# Apply all manifests
kubectl apply -k deployments/k8s/base/
# Verify deployment
kubectl -n rdev get pods
kubectl -n rdev get svc
```
## Configuration
### Environment Variables
| Variable | Description | Required | Default |
|----------|-------------|----------|---------|
| `PORT` | HTTP server port | No | 8080 |
| `POSTGRES_HOST` | Database host | Yes | - |
| `POSTGRES_PORT` | Database port | No | 5432 |
| `POSTGRES_USER` | Database user | Yes | - |
| `POSTGRES_PASSWORD` | Database password | Yes | - |
| `POSTGRES_DB` | Database name | No | rdev |
| `RDEV_NAMESPACE` | K8s namespace for pods | No | default |
| `RATE_LIMIT_RPS` | Requests per second | No | 10 |
| `CONCURRENT_COMMANDS` | Max concurrent commands | No | 5 |
### Secrets
Create a secret for database credentials:
```bash
kubectl -n rdev create secret generic rdev-api-secrets \
--from-literal=postgres-password=your-password
```
Or use the manifest:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: rdev-api-secrets
namespace: rdev
type: Opaque
stringData:
postgres-password: your-secure-password
```
### ConfigMap
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: rdev-api-config
namespace: rdev
data:
POSTGRES_HOST: "postgres.databases.svc"
POSTGRES_DB: "rdev"
RDEV_NAMESPACE: "rdev"
RATE_LIMIT_RPS: "10"
CONCURRENT_COMMANDS: "5"
```
## Kubernetes Manifests
### Namespace
```yaml
apiVersion: v1
kind: Namespace
metadata:
name: rdev
labels:
app.kubernetes.io/name: rdev
```
### Deployment
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: rdev-api
namespace: rdev
spec:
replicas: 2
selector:
matchLabels:
app: rdev-api
template:
metadata:
labels:
app: rdev-api
spec:
serviceAccountName: rdev-api
securityContext:
runAsNonRoot: true
runAsUser: 1000
containers:
- name: rdev-api
image: your-registry/rdev-api:latest
ports:
- containerPort: 8080
env:
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: rdev-api-secrets
key: postgres-password
envFrom:
- configMapRef:
name: rdev-api-config
securityContext:
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
resources:
requests:
memory: "128Mi"
cpu: "100m"
limits:
memory: "512Mi"
cpu: "500m"
livenessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 5
periodSeconds: 10
readinessProbe:
httpGet:
path: /ready
port: 8080
initialDelaySeconds: 5
periodSeconds: 5
```
### Service
```yaml
apiVersion: v1
kind: Service
metadata:
name: rdev-api
namespace: rdev
spec:
selector:
app: rdev-api
ports:
- port: 80
targetPort: 8080
```
### Ingress
```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: rdev-api
namespace: rdev
annotations:
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
spec:
ingressClassName: nginx
rules:
- host: rdev.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: rdev-api
port:
number: 80
tls:
- hosts:
- rdev.example.com
secretName: rdev-tls
```
### RBAC
```yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: rdev-api
namespace: rdev
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: rdev-api-role
namespace: rdev
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create"]
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rdev-api-binding
namespace: rdev
subjects:
- kind: ServiceAccount
name: rdev-api
roleRef:
kind: Role
name: rdev-api-role
apiGroup: rbac.authorization.k8s.io
```
### Pod Disruption Budget
```yaml
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: rdev-api-pdb
namespace: rdev
spec:
minAvailable: 1
selector:
matchLabels:
app: rdev-api
```
### Network Policy
```yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: rdev-api-policy
namespace: rdev
spec:
podSelector:
matchLabels:
app: rdev-api
policyTypes:
- Ingress
- Egress
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: ingress-nginx
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: databases
ports:
- protocol: TCP
port: 5432
```
## Database Setup
### Create Database
```sql
CREATE DATABASE rdev;
CREATE USER rdev_user WITH PASSWORD 'secure-password';
GRANT ALL PRIVILEGES ON DATABASE rdev TO rdev_user;
```
### Migrations
Migrations run automatically on startup. To run manually:
```bash
# Connect to pod
kubectl -n rdev exec -it deployment/rdev-api -- sh
# Check migration status
psql $DATABASE_URL -c "SELECT * FROM schema_migrations;"
```
## Scaling
### Horizontal Pod Autoscaler
```yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: rdev-api-hpa
namespace: rdev
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: rdev-api
minReplicas: 2
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
```
## Upgrading
### Rolling Update
```bash
# Update image
kubectl -n rdev set image deployment/rdev-api \
rdev-api=your-registry/rdev-api:new-version
# Watch rollout
kubectl -n rdev rollout status deployment/rdev-api
```
### Rollback
```bash
# Rollback to previous version
kubectl -n rdev rollout undo deployment/rdev-api
# Rollback to specific revision
kubectl -n rdev rollout undo deployment/rdev-api --to-revision=2
```
## Health Checks
### Liveness
```bash
curl http://rdev-api/health
```
Returns `200 OK` if the service is running.
### Readiness
```bash
curl http://rdev-api/ready
```
Returns `200 OK` if database and K8s are connected.
## Troubleshooting
### Pod Not Starting
```bash
# Check pod events
kubectl -n rdev describe pod -l app=rdev-api
# Check logs
kubectl -n rdev logs -l app=rdev-api
```
### Database Connection Failed
1. Check secret is mounted correctly
2. Verify database host is reachable
3. Check network policy allows egress
### K8s API Errors
1. Verify ServiceAccount has correct RBAC
2. Check namespace configuration
3. Verify API server connectivity