a9ad3d8304
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
CI / Woodpecker: - Add explicit depends_on to all .woodpecker.yml steps (rdev + templates) - Fix skip_tls_verify -> skip-tls-verify (correct Kaniko flag name) - Add replicasets get/list to deployer RBAC for rollout status - Skeleton template: add failure:ignore on docs steps, Traefik TLS annotations on ingress, depends_on on verify step Component templates: - Fix container name in deploy steps (PROJECT_NAME-COMPONENT_NAME) - Replace kubectl scale with kubectl patch for replicas - Add post-deploy image verification and rollout status checks - Applied consistently across all 5 component templates Adapters: - gitea: Add HTTP client timeout (30s), context cancellation checks, handle 404 on GetRepo/DeleteRepo - zot: Add retry with exponential backoff (doWithRetry), limit response body reads to 10MB - cockroach: Use net.JoinHostPort for IPv6-safe DSN construction - woodpecker: Fix error wrapping (%v -> %w) - redis: Fix error wrapping (%v -> %w) - deployer: Add context cancellation checks Services: - apikey_service: Fix error wrapping (%v -> %w) - component_deploy: Fix error wrapping (%v -> %w) - project_infra: Fix error wrapping (%v -> %w) - webhook/dispatcher: Fix error wrapping (%v -> %w) Other: - CLAUDE.md: Add guide links for Gitea, Go 1.25, Woodpecker v3, Traefik v3, Zot registry - circuitbreaker: Add test for error wrapping - docs: Update deployment, troubleshooting, and runbook docs - health: Fix error wrapping (%v -> %w) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
404 lines
7.5 KiB
Markdown
404 lines
7.5 KiB
Markdown
# Deployment Guide
|
|
|
|
This guide covers deploying rdev API to the k3s cluster.
|
|
|
|
## Prerequisites
|
|
|
|
```bash
|
|
# REQUIRED: Set kubeconfig before any kubectl command
|
|
export KUBECONFIG=~/.kube/orchard9-k3sf.yaml
|
|
```
|
|
|
|
- k3s cluster (orchard9-k3sf)
|
|
- kubectl configured with correct kubeconfig
|
|
- PostgreSQL database
|
|
- Container registry access (ghcr.io/orchard9)
|
|
|
|
## Quick Deploy
|
|
|
|
```bash
|
|
# Release + deploy (recommended)
|
|
./scripts/release.sh v0.10.1 "Description of changes" --deploy
|
|
|
|
# Or manual deploy
|
|
kubectl apply -f deployments/k8s/base/rdev-api.yaml
|
|
kubectl rollout restart -n rdev deployment/rdev-api
|
|
|
|
# Verify deployment
|
|
kubectl -n rdev get pods
|
|
kubectl -n rdev get svc
|
|
```
|
|
|
|
## Configuration
|
|
|
|
### Environment Variables
|
|
|
|
| Variable | Description | Required | Default |
|
|
|----------|-------------|----------|---------|
|
|
| `PORT` | HTTP server port | No | 8080 |
|
|
| `POSTGRES_HOST` | Database host | Yes | - |
|
|
| `POSTGRES_PORT` | Database port | No | 5432 |
|
|
| `POSTGRES_USER` | Database user | Yes | - |
|
|
| `POSTGRES_PASSWORD` | Database password | Yes | - |
|
|
| `POSTGRES_DB` | Database name | No | rdev |
|
|
| `RDEV_NAMESPACE` | K8s namespace for pods | No | default |
|
|
| `RATE_LIMIT_RPS` | Requests per second | No | 10 |
|
|
| `CONCURRENT_COMMANDS` | Max concurrent commands | No | 5 |
|
|
|
|
### Secrets
|
|
|
|
Create a secret for database credentials:
|
|
|
|
```bash
|
|
kubectl -n rdev create secret generic rdev-api-secrets \
|
|
--from-literal=postgres-password=your-password
|
|
```
|
|
|
|
Or use the manifest:
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: Secret
|
|
metadata:
|
|
name: rdev-api-secrets
|
|
namespace: rdev
|
|
type: Opaque
|
|
stringData:
|
|
postgres-password: your-secure-password
|
|
```
|
|
|
|
### ConfigMap
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: ConfigMap
|
|
metadata:
|
|
name: rdev-api-config
|
|
namespace: rdev
|
|
data:
|
|
POSTGRES_HOST: "postgres.databases.svc"
|
|
POSTGRES_DB: "rdev"
|
|
RDEV_NAMESPACE: "rdev"
|
|
RATE_LIMIT_RPS: "10"
|
|
CONCURRENT_COMMANDS: "5"
|
|
```
|
|
|
|
## Kubernetes Manifests
|
|
|
|
### Namespace
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: Namespace
|
|
metadata:
|
|
name: rdev
|
|
labels:
|
|
app.kubernetes.io/name: rdev
|
|
```
|
|
|
|
### Deployment
|
|
|
|
```yaml
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
name: rdev-api
|
|
namespace: rdev
|
|
spec:
|
|
replicas: 2
|
|
selector:
|
|
matchLabels:
|
|
app: rdev-api
|
|
template:
|
|
metadata:
|
|
labels:
|
|
app: rdev-api
|
|
spec:
|
|
serviceAccountName: rdev-api
|
|
securityContext:
|
|
runAsNonRoot: true
|
|
runAsUser: 1000
|
|
containers:
|
|
- name: rdev-api
|
|
image: your-registry/rdev-api:latest
|
|
ports:
|
|
- containerPort: 8080
|
|
env:
|
|
- name: POSTGRES_PASSWORD
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: rdev-api-secrets
|
|
key: postgres-password
|
|
envFrom:
|
|
- configMapRef:
|
|
name: rdev-api-config
|
|
securityContext:
|
|
readOnlyRootFilesystem: true
|
|
allowPrivilegeEscalation: false
|
|
capabilities:
|
|
drop:
|
|
- ALL
|
|
resources:
|
|
requests:
|
|
memory: "128Mi"
|
|
cpu: "100m"
|
|
limits:
|
|
memory: "512Mi"
|
|
cpu: "500m"
|
|
livenessProbe:
|
|
httpGet:
|
|
path: /health
|
|
port: 8080
|
|
initialDelaySeconds: 5
|
|
periodSeconds: 10
|
|
readinessProbe:
|
|
httpGet:
|
|
path: /ready
|
|
port: 8080
|
|
initialDelaySeconds: 5
|
|
periodSeconds: 5
|
|
```
|
|
|
|
### Service
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: rdev-api
|
|
namespace: rdev
|
|
spec:
|
|
selector:
|
|
app: rdev-api
|
|
ports:
|
|
- port: 80
|
|
targetPort: 8080
|
|
```
|
|
|
|
### Ingress
|
|
|
|
```yaml
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: Ingress
|
|
metadata:
|
|
name: rdev-api
|
|
namespace: rdev
|
|
annotations:
|
|
traefik.ingress.kubernetes.io/router.entrypoints: websecure
|
|
traefik.ingress.kubernetes.io/router.tls: "true"
|
|
spec:
|
|
ingressClassName: traefik
|
|
rules:
|
|
- host: rdev.example.com
|
|
http:
|
|
paths:
|
|
- path: /
|
|
pathType: Prefix
|
|
backend:
|
|
service:
|
|
name: rdev-api
|
|
port:
|
|
number: 80
|
|
tls:
|
|
- hosts:
|
|
- rdev.example.com
|
|
secretName: rdev-tls
|
|
```
|
|
|
|
### RBAC
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: ServiceAccount
|
|
metadata:
|
|
name: rdev-api
|
|
namespace: rdev
|
|
---
|
|
apiVersion: rbac.authorization.k8s.io/v1
|
|
kind: Role
|
|
metadata:
|
|
name: rdev-api-role
|
|
namespace: rdev
|
|
rules:
|
|
- apiGroups: [""]
|
|
resources: ["pods"]
|
|
verbs: ["get", "list", "watch"]
|
|
- apiGroups: [""]
|
|
resources: ["pods/exec"]
|
|
verbs: ["create"]
|
|
- apiGroups: [""]
|
|
resources: ["configmaps"]
|
|
verbs: ["get", "list", "watch"]
|
|
---
|
|
apiVersion: rbac.authorization.k8s.io/v1
|
|
kind: RoleBinding
|
|
metadata:
|
|
name: rdev-api-binding
|
|
namespace: rdev
|
|
subjects:
|
|
- kind: ServiceAccount
|
|
name: rdev-api
|
|
roleRef:
|
|
kind: Role
|
|
name: rdev-api-role
|
|
apiGroup: rbac.authorization.k8s.io
|
|
```
|
|
|
|
### Pod Disruption Budget
|
|
|
|
```yaml
|
|
apiVersion: policy/v1
|
|
kind: PodDisruptionBudget
|
|
metadata:
|
|
name: rdev-api-pdb
|
|
namespace: rdev
|
|
spec:
|
|
minAvailable: 1
|
|
selector:
|
|
matchLabels:
|
|
app: rdev-api
|
|
```
|
|
|
|
### Network Policy
|
|
|
|
```yaml
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: rdev-api-policy
|
|
namespace: rdev
|
|
spec:
|
|
podSelector:
|
|
matchLabels:
|
|
app: rdev-api
|
|
policyTypes:
|
|
- Ingress
|
|
- Egress
|
|
ingress:
|
|
- from:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: ingress-nginx
|
|
egress:
|
|
- to:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: databases
|
|
ports:
|
|
- protocol: TCP
|
|
port: 5432
|
|
```
|
|
|
|
## Database Setup
|
|
|
|
### Create Database
|
|
|
|
```sql
|
|
CREATE DATABASE rdev;
|
|
CREATE USER rdev_user WITH PASSWORD 'secure-password';
|
|
GRANT ALL PRIVILEGES ON DATABASE rdev TO rdev_user;
|
|
```
|
|
|
|
### Migrations
|
|
|
|
Migrations run automatically on startup. To run manually:
|
|
|
|
```bash
|
|
# Connect to pod
|
|
kubectl -n rdev exec -it deployment/rdev-api -- sh
|
|
|
|
# Check migration status
|
|
psql $DATABASE_URL -c "SELECT * FROM schema_migrations;"
|
|
```
|
|
|
|
## Scaling
|
|
|
|
### Horizontal Pod Autoscaler
|
|
|
|
```yaml
|
|
apiVersion: autoscaling/v2
|
|
kind: HorizontalPodAutoscaler
|
|
metadata:
|
|
name: rdev-api-hpa
|
|
namespace: rdev
|
|
spec:
|
|
scaleTargetRef:
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
name: rdev-api
|
|
minReplicas: 2
|
|
maxReplicas: 10
|
|
metrics:
|
|
- type: Resource
|
|
resource:
|
|
name: cpu
|
|
target:
|
|
type: Utilization
|
|
averageUtilization: 70
|
|
```
|
|
|
|
## Upgrading
|
|
|
|
### Rolling Update
|
|
|
|
```bash
|
|
# Update image
|
|
kubectl -n rdev set image deployment/rdev-api \
|
|
rdev-api=your-registry/rdev-api:new-version
|
|
|
|
# Watch rollout
|
|
kubectl -n rdev rollout status deployment/rdev-api
|
|
```
|
|
|
|
### Rollback
|
|
|
|
```bash
|
|
# Rollback to previous version
|
|
kubectl -n rdev rollout undo deployment/rdev-api
|
|
|
|
# Rollback to specific revision
|
|
kubectl -n rdev rollout undo deployment/rdev-api --to-revision=2
|
|
```
|
|
|
|
## Health Checks
|
|
|
|
### Liveness
|
|
|
|
```bash
|
|
curl http://rdev-api/health
|
|
```
|
|
|
|
Returns `200 OK` if the service is running.
|
|
|
|
### Readiness
|
|
|
|
```bash
|
|
curl http://rdev-api/ready
|
|
```
|
|
|
|
Returns `200 OK` if database and K8s are connected.
|
|
|
|
## Troubleshooting
|
|
|
|
### Pod Not Starting
|
|
|
|
```bash
|
|
# Check pod events
|
|
kubectl -n rdev describe pod -l app=rdev-api
|
|
|
|
# Check logs
|
|
kubectl -n rdev logs -l app=rdev-api
|
|
```
|
|
|
|
### Database Connection Failed
|
|
|
|
1. Check secret is mounted correctly
|
|
2. Verify database host is reachable
|
|
3. Check network policy allows egress
|
|
|
|
### K8s API Errors
|
|
|
|
1. Verify ServiceAccount has correct RBAC
|
|
2. Check namespace configuration
|
|
3. Verify API server connectivity
|