From 37cafb2b75cdf717ec36d24a4108f2e4e70e8915 Mon Sep 17 00:00:00 2001 From: jx12n Date: Sat, 5 Sep 2026 14:30:54 -0600 Subject: [PATCH] release: stop building under the server's egress policy MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The build Job's pods carried `app: hush`, which is the podSelector on hush's NetworkPolicy — a default-deny policy that permits egress to DNS and Redis and nothing else. That is a true statement about the server and a false one about a build, which needs Gitea, the registry, Docker Hub and gcr.io. kube-router REJECTS rather than drops, so the symptom was `connection refused` from whichever host the build reached for next: four pushes to registry.threesix.ai, then a blob fetch from production.cloudfront.docker.com. Intermittently, because policy sync leaves windows where the rules are briefly absent — which is why an earlier build pushed successfully and read as normal. Measured 2026-09-05 from pods in this namespace: `app=hush` reached Docker Hub 10 of 12 times, `app=hush-build` 12 of 12, and in-cluster destinations 16 of 16 under both. The Job is now `app: hush-build`, so hush's policy no longer selects it, and the Service selector no longer matches it either. --- scripts/release.sh | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/scripts/release.sh b/scripts/release.sh index b9106a7..ecba064 100755 --- a/scripts/release.sh +++ b/scripts/release.sh @@ -72,18 +72,26 @@ echo " build resolves git+registry to traefik at $TRAEFIK_IP" # re-created; replacing it is the idempotent thing to do. kubectl -n "$NS" delete job "$JOB" --ignore-not-found >/dev/null +# NOTE the label: `app: hush-build`, NOT `app: hush`. hush's NetworkPolicy +# selects `app: hush` and permits egress to DNS and Redis only — a true +# statement about the SERVER, and a build pod that inherits it cannot reach +# Docker Hub, gcr.io, Gitea or the registry. kube-router REJECTS, so that +# arrives as `connection refused` from whichever host the build happened to +# need next, intermittently, because policy sync leaves windows where the rules +# are briefly absent. Measured 2026-09-05: five consecutive kaniko builds +# failed on three different external hosts under `app: hush`. kubectl -n "$NS" apply -f - >/dev/null <