31ee612f27
Scale write throughput across data-shard groups while keeping a single unified read surface: - scatter_gather.rs: pooled fan-out across shard groups (replaces per-request client construction); cross-shard query results merged on one node - cluster/node.rs: cross-shard read routing — a read on any node gathers from every shard group's leader and unions results - cluster/forward.rs: fix h2 204 forward-relay bug (relay_forwarded skips body for 1xx/204/304 — synthesized JSON body on a 204 triggered HTTP/2 RST_STREAM on the real mTLS plane) - dto.rs: cross-shard query/result DTOs - k8s/cluster/: enable 3-group `shards:` topology (statefulset, service-peers, topology-configmap) - k8s/cluster-local-kind/: local-kind overlay to run the T5 gate without Ref-A - tidal-stress/k8s/stress-job-t5.yaml: 2-generator sharded throughput job - tests: cluster_cross_shard_reads.rs + multiproc support; ran real on kind - docs/profiling/m12p4-t5-sharded-throughput.md: T5 throughput findings
44 lines
1.7 KiB
YAML
44 lines
1.7 KiB
YAML
# Local-kind overlay for the cluster set — used to run the m12p4 T5 sharded
|
|
# throughput gate on a local `kind` cluster (no private registry, no Ref-A).
|
|
#
|
|
# It reuses the canonical k8s/cluster/ base verbatim (including the 3-group
|
|
# `shards:` topology and cert-manager certs) and only swaps the registry-pinned
|
|
# server image for a locally-built tag that `kind load docker-image` puts on the
|
|
# node. cert-manager must be present in the kind cluster (the shared `canopy`
|
|
# cluster already has it); the credentials Secret is still created out-of-band.
|
|
#
|
|
# Run:
|
|
# docker build -f docker/deploy/Dockerfile -t tidaldb-server:m12p4-local .
|
|
# kind load docker-image tidaldb-server:m12p4-local --name canopy
|
|
# kubectl create namespace tidaldb-cluster
|
|
# kubectl -n tidaldb-cluster create secret generic tidaldb-credentials \
|
|
# --from-literal=TIDAL_API_KEY="$(openssl rand -hex 32)" \
|
|
# --from-literal=TIDAL_CLUSTER_KEY="$(openssl rand -hex 32)"
|
|
# kubectl apply -k k8s/cluster-local-kind/
|
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
kind: Kustomization
|
|
|
|
namespace: tidaldb-cluster
|
|
|
|
resources:
|
|
- ../cluster
|
|
|
|
# Swap the registry-pinned (digest) server image for the locally-loaded tag.
|
|
# Kustomize matches on the name part before the `@sha256:` digest.
|
|
images:
|
|
- name: registry.threesix.ai/tidal/server
|
|
newName: tidaldb-server
|
|
newTag: m12p4-local
|
|
|
|
# kind ships a default StorageClass named `standard` (rancher.io/local-path),
|
|
# not `local-path` as the canonical StatefulSet names. Repoint the PVC template
|
|
# at the kind default so the claims bind instead of hanging Pending.
|
|
patches:
|
|
- target:
|
|
kind: StatefulSet
|
|
name: tidaldb
|
|
patch: |-
|
|
- op: replace
|
|
path: /spec/volumeClaimTemplates/0/spec/storageClassName
|
|
value: standard
|