087b83154a
tidaldb.threesix.ai now serves the cluster's data surface over a Let's Encrypt cert, verified end-to-end from the internet: 401 without a bearer, 401 with a wrong one, 200 with the real key, and a quorum-acked write returning 201 on all three node IPs. Three things this had to get right, each of which failed first: * The backend is HTTPS, not HTTP. Pods serve :9500 over TLS with the internal cluster CA whenever grpc_tls is configured, so a plaintext backend dial answers 500. Added a ServersTransport that VERIFIES that hop - every pod mounts the same tidaldb-cluster-tls leaf and its SANs include the client-Service DNS name, so serverName pinning validates it without insecureSkipVerify. * `service.*` annotations are read from the Service, not the Ingress. Putting serversscheme/serverstransport on the Ingress is silently ignored and presents exactly as a broken backend. * http01 cannot be used behind any gateway gate that rejects unknown callers, because it rejects the ACME challenge too. Uses the Cloudflare dns01 solver. Deliberately unpublished: /cluster/* (every mutating admin verb shares the SAME single bearer as the data routes, so a client key could remove members or transfer shards), /cluster/status (unauthenticated - leaks leader, membership, seqnos), /openapi.json (unauthenticated, enumerates the admin routes), and /metrics (only on the headless peers Service, unreachable here). Documents two controls that are NOT available and why: an IP allowlist cannot work while the shared Traefik Service runs externalTrafficPolicy=Cluster (svclb SNATs the client address), and Traefik basicAuth cannot stack in front of the bearer because both occupy the Authorization header.
40 lines
1.4 KiB
YAML
40 lines
1.4 KiB
YAML
# tidalDB CLUSTER deployment (m11p5 §4). Apply with:
|
|
# kubectl apply -k k8s/cluster/
|
|
#
|
|
# MUTUALLY EXCLUSIVE with the standalone set (k8s/, namespace `tidaldb`). This
|
|
# set lives in namespace `tidaldb-cluster`. Deploy ONE or the OTHER per
|
|
# namespace — never both (they share the StatefulSet name `tidaldb` and the
|
|
# standalone set's replicas:1 is load-bearing).
|
|
#
|
|
# Create the credentials secret FIRST (deliberately excluded so no key is
|
|
# committed — see secret.example.yaml). m11p7 shape carries BOTH the bearer and
|
|
# the cluster key:
|
|
# kubectl -n tidaldb-cluster create secret generic tidaldb-credentials \
|
|
# --from-literal=TIDAL_API_KEY="$(openssl rand -hex 32)" \
|
|
# --from-literal=TIDAL_CLUSTER_KEY="$(openssl rand -hex 32)"
|
|
#
|
|
# certs.yaml (m11p7 inter-node TLS) requires cert-manager. If you do NOT run
|
|
# cert-manager, comment certs.yaml out and provision the `tidaldb-cluster-tls`
|
|
# Secret with scripts/gen-cluster-certs.sh instead.
|
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
kind: Kustomization
|
|
|
|
namespace: tidaldb-cluster
|
|
|
|
resources:
|
|
- namespace.yaml
|
|
- schema-configmap.yaml
|
|
- topology-configmap.yaml
|
|
- certs.yaml
|
|
- service-peers.yaml
|
|
- service-client.yaml
|
|
- statefulset.yaml
|
|
- poddisruptionbudget.yaml
|
|
# Public exposure (Traefik + Let's Encrypt). Remove for internal-only.
|
|
- ingress.yaml
|
|
|
|
labels:
|
|
- pairs:
|
|
app.kubernetes.io/part-of: tidaldb
|
|
includeSelectors: false
|