From 62c95f8c2f41d7fc88a4cac9bfc372e8d38def6d Mon Sep 17 00:00:00 2001 From: jx12n Date: Thu, 3 Sep 2026 00:18:53 -0600 Subject: [PATCH] docs, ops scripts, and the MCP install MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Written after the service was live, so every command and every number here was run against the real deployment rather than assumed: * DEPLOY.md records the things a rebuild needs and git does not hold — the Redis ACL user (and why +getdel is the one to notice), the GCP Secret Manager entry, the DNS record, and the three coordinated edits vmalert needs because it has no ConfigMap auto-discovery. * It also records two blockers rather than hiding them: Woodpecker is NOT activated (the token in rdev-credentials returns 401), so pushes do not deploy yet and the Kaniko Job is the interim path; and the host is hush.threesix.ai rather than hush.orchard9.ai because orchard9.ai is on GoDaddy and no GoDaddy credential exists anywhere I can reach. * OPERATIONS.md is one section per alert, plus the failure modes that are not alerts — chiefly that "gone" cannot distinguish already-revealed from expired from LRU-evicted, on purpose, so the operator's default reading of an unexpected "gone" is that the secret is compromised and should be rotated. * scripts/logs.sh and alerts-check.sh verify rather than assert: alerts-check asks vmalert what it actually loaded AND checks each rule's series exists, because a rule reading a metric nothing exports can never fire and looks exactly like a healthy service. * scripts/smoke.sh is a real client — it generates a key, encrypts, posts only ciphertext, reveals, decrypts, then asserts the second reveal is 410, that three GETs did not consume the secret, that missing and malformed ids are indistinguishable, and that a plaintext field is refused. install-mcp.sh proves the MCP handshake before writing any config, backs up mcp.json, and rewrites only hush's entry — a config pointing at a broken server surfaces as an opaque host-side connect failure, which is worth one extra check to avoid. --- docs/DEPLOY.md | 197 ++++++++++++++++++++++++++++++++++++++++ docs/MCP.md | 111 ++++++++++++++++++++++ docs/OPERATIONS.md | 178 ++++++++++++++++++++++++++++++++++++ scripts/alerts-check.sh | 50 ++++++++++ scripts/format-logs.py | 40 ++++++++ scripts/install-mcp.sh | 79 ++++++++++++++++ scripts/logs.sh | 34 +++++++ 7 files changed, 689 insertions(+) create mode 100644 docs/DEPLOY.md create mode 100644 docs/MCP.md create mode 100644 docs/OPERATIONS.md create mode 100755 scripts/alerts-check.sh create mode 100755 scripts/format-logs.py create mode 100755 scripts/install-mcp.sh create mode 100755 scripts/logs.sh diff --git a/docs/DEPLOY.md b/docs/DEPLOY.md new file mode 100644 index 0000000..4579c5f --- /dev/null +++ b/docs/DEPLOY.md @@ -0,0 +1,197 @@ +# Deploying hush + +Live at , one replica in the `projects` namespace on +the orchard9 k3s cluster. + +``` +git push origin main → Gitea webhook → Woodpecker → Kaniko (amd64, in-cluster) + → Zot registry → kubectl set image → projects/hush +``` + +`origin` **must** be Gitea (`git.threesix.ai`); that remote carries the webhook. +The GitHub mirror is a backup and pushing there deploys nothing. + +## Never build the image locally + +Two reasons, both of which cost real time to discover: + +1. A laptop build on Apple Silicon produces **arm64**; the cluster runs amd64. +2. `registry.threesix.ai` accepts **only OCI image manifests** — not Docker + schema2, and not an OCI *index*. `docker push` and `crane push` both fail + `MANIFEST_INVALID`, and buildx wraps even a single-platform build in an index. + +Kaniko sidesteps both. If you must build outside the pipeline, use a Job — see +"Bootstrap" below, which is exactly what the first deploy did. + +## Dependencies are vendored, deliberately + +`github.com/orchard9/go-chassis` is a **private** module. Neither the Woodpecker +test container nor the Kaniko build holds a git credential, so a build that +resolved dependencies from the network would fail: + +``` +$ curl https://proxy.golang.org/github.com/orchard9/go-chassis/@v/list +404 ... could not read Username for 'https://github.com' +``` + +So `vendor/` is committed and both CI and the Dockerfile run `-mod=vendor` with +`GOPROXY=off`. `GOPROXY=off` is the important half: it turns "silently fetched +from somewhere" into a hard failure. `make vendor` is the only way versions +move, and `make verify` proves the tree still builds with no network. + +## One-time setup, already done + +Recorded because it is what a rebuild would need, and none of it is in git. + +### 1. Redis ACL user + +hush connects as its own ACL user, scoped to `~hush:*`, on **db 5** (0 and 3 and +4 are taken by pantheon/rdev, reel, and jit): + +```bash +redis-cli ACL SETUSER hush on '>PASSWORD' '~hush:*' resetchannels \ + -@all +ping +set +getdel +incr +pexpire +pttl +select +redis-cli ACL SAVE # persists to /data/users.acl +``` + +**`+getdel` is the one to notice.** No other service's ACL user has it, because +no other service needs an atomic read-and-destroy. Omit it and creates keep +working while every reveal fails `NOPERM` — a service that accepts secrets and +cannot deliver them. `+incr +pexpire +pttl` are the rate limiter. + +### 2. The credential + +A JSON object in GCP Secret Manager, pulled into the namespace by ESO: + +```bash +gcloud secrets create k3sf-hush-credentials --project orchard9 \ + --replication-policy=automatic --data-file=- <<< \ + '{"REDIS_URL":"redis://hush:PASSWORD@redis.databases.svc.cluster.local:6379/5"}' +``` + +`ExternalSecret/hush-credentials` (in `deployments/k8s/hush.yaml`) syncs it to a +Secret of the same name. Confirm with: + +```bash +kubectl -n projects get externalsecret hush-credentials \ + -o jsonpath='{.status.conditions[0].reason}' # want SecretSynced +``` + +### 3. DNS + +`hush.threesix.ai` → `208.122.204.172`, A record, **DNS-only** (not proxied), +TTL 120 — matching every other `*.threesix.ai` service. There is no wildcard on +the zone, so each host needs its own record. cert-manager then issues TLS over +HTTP-01 with no DNS credential needed. + +> `hush.orchard9.ai` was the original intent and is **not** what shipped. +> `orchard9.ai` is on GoDaddy and no GoDaddy credential exists in rdev, the +> cluster, or `~/.squiddy-dns`. Moving the host there needs that credential; +> everything else is a one-line Ingress change plus a new record. + +### 4. Alert rules + +vmalert has **no ConfigMap auto-discovery**. Three coordinated edits in +`orchard9-k3sf`, and missing any one leaves the rules silently absent: + +1. `observability/hush-alerting-rules.yaml` — the ConfigMap +2. `observability/kustomization.yaml` — list it under `resources:` +3. `observability/victoria-metrics.yaml` — vmalert needs a matching + `-rule=/etc/rules-hush/*.yaml`, `volumeMount` and `volume` + +`make alerts-check` asks vmalert what it actually loaded, and separately checks +that every series the rules reference exists — a rule reading a metric nothing +exports can never fire and looks exactly like a healthy service. + +## Bootstrap (what the first deploy did) + +The pipeline's deploy step runs `kubectl set image`, so a Deployment must exist +first. But the committed image tag cannot be `:latest`: the cluster's +`stable-controller-images.orchard9.ai` admission policy refuses +`latest|main|master|dev|edge|canary|nightly|snapshot`, because a floating tag +cannot pin a rollback. + +So the manifest carries `:bootstrap`, which is policy-legal and does not exist. +Apply it, then build once by hand: + +```bash +make deploy-manifests # pod sits in ImagePullBackOff — expected + +SHA=$(git rev-parse --short=8 HEAD) +kubectl -n projects create job hush-build-$SHA --dry-run=client -o yaml ... # see below +kubectl -n projects set image deployment/hush hushd=registry.threesix.ai/hush/api:$SHA +``` + +The build Job, which is what Woodpecker's Kaniko step does by hand: + +```yaml +apiVersion: batch/v1 +kind: Job +metadata: { name: hush-build, namespace: projects } +spec: + backoffLimit: 1 + ttlSecondsAfterFinished: 3600 + template: + spec: + restartPolicy: Never + containers: + - name: kaniko + image: gcr.io/kaniko-project/executor:v1.23.2 + args: + - --context=git://git.threesix.ai/jordan/hush.git#refs/heads/main + - --dockerfile=Dockerfile + - --destination=registry.threesix.ai/hush/api:SHA + - --skip-tls-verify + - --skip-tls-verify-pull + - --single-snapshot + resources: + requests: { cpu: 500m, memory: 1Gi } + limits: { cpu: "2", memory: 3Gi } +``` + +The git context needs no credential because the Gitea repo is public. + +## Woodpecker is NOT yet activated + +The repo exists on Gitea and `.woodpecker.yml` is committed, but activation +failed: the `WOODPECKER_API_TOKEN` in `rdev/rdev-credentials` returns +`401 User not authorized`. + +Until a valid token replaces it, **pushes do not deploy** — use the Kaniko Job +above and `kubectl set image`. To finish it: + +```bash +WP=$(curl -s -H "X-API-Key: $RDEV_API_KEY" "$RDEV_API_URL/credentials/WOODPECKER_API_TOKEN" | jq -r '.data.value') +curl -X POST "https://ci.threesix.ai/api/repos?forge_remote_id=jordan/hush" -H "Authorization: Bearer $WP" +``` + +A fresh token comes from Woodpecker → User Settings → Token, and belongs back in +rdev rather than anywhere else. + +## Rollback + +Every build is SHA-tagged, so rollback is naming the previous one: + +```bash +kubectl -n projects rollout undo deployment/hush +# or explicitly +kubectl -n projects set image deployment/hush hushd=registry.threesix.ai/hush/api: +``` + +Nothing to migrate and no schema: Redis holds only TTL'd ciphertext, and a +rollback cannot invalidate an outstanding link because the id and the wire +format are stable. + +## Verifying a deploy + +```bash +make deploy-status # rollout, pods, ingress, certificate +BASE=https://hush.threesix.ai make smoke # real crypto, create → reveal → gone +make logs # the lifecycle in VictoriaLogs +make alerts-check # rules loaded, series present +``` + +`make smoke` is the one that matters. It encrypts with a real AES-256-GCM key, +posts only ciphertext, reveals once, decrypts, and then asserts the second +reveal is `410`, that three `GET`s did not consume the secret, that missing and +malformed ids are indistinguishable, and that a plaintext field is refused. diff --git a/docs/MCP.md b/docs/MCP.md new file mode 100644 index 0000000..f5f2968 --- /dev/null +++ b/docs/MCP.md @@ -0,0 +1,111 @@ +# The MCP server + +`cmd/hush-mcp` gives an MCP host (omp, Claude Code, any client) two tools: + +| Tool | Does | +|---|---| +| `hush_create` | Encrypts a secret locally, stores the ciphertext, returns a one-time link | +| `hush_reveal` | Fetches and decrypts a link, **destroying it** | + +## Install + +```bash +make mcp +``` + +That builds the binary to `~/.local/bin/hush-mcp`, proves the MCP handshake +works before wiring anything, and adds a `hush` entry to +`~/.omp/agent/mcp.json` — backing the file up first and leaving every other +server alone. Restart omp to pick it up. + +```json +{ + "mcpServers": { + "hush": { + "type": "stdio", + "command": "/Users/you/.local/bin/hush-mcp", + "env": { "HUSH_BASE_URL": "https://hush.threesix.ai" }, + "timeout": 20000 + } + } +} +``` + +The same file shape works for Claude Code (`~/.claude.json`), Cursor and VS +Code — the stdio transport is the portable part. + +## Why it runs locally instead of being an endpoint on hushd + +hushd could serve `/mcp` and encrypt on the server. It deliberately does not. + +If the server did the encrypting, the server would see every plaintext created +through MCP. hush's guarantee — *we cannot read your secrets* — would then hold +for browser users and quietly not hold for agent users, and no one could tell +which they had by looking at a link. Two guarantees behind one URL is worse +than one honest guarantee. + +So `hush-mcp` is a peer of the browser, not of the server: it mints the AES-256 +key, encrypts, posts only ciphertext, and assembles the `#fragment` link +itself. hushd sees exactly what it sees from a browser and no more. + +The cost is that this is a local binary to install rather than a URL to +configure. That is the right trade for a service whose entire value is where +the key sits. + +## Wire compatibility + +Three implementations produce and consume one format — the browser +(`internal/web/templates/base.html`), this server, and `scripts/smoke.sh`: + +``` +AES-256-GCM, 96-bit nonce PREPENDED to the ciphertext, +both ciphertext and key base64url-encoded WITHOUT padding +``` + +A link minted by any of the three opens in the other two. +`TestWireFormatMatchesAnIndependentImplementation` pins that by round-tripping +Go↔Python in both directions, so a change to one implementation's encoding +fails the build rather than producing links that only work in the client that +made them. + +## Configuration + +| Variable | Default | Purpose | +|---|---|---| +| `HUSH_BASE_URL` | `https://hush.threesix.ai` | Which deployment `hush_create` posts to | +| `HUSH_CREATE_TOKEN` | unset | Only needed if that deployment has `HUSH_REQUIRE_AUTH=true` | + +`hush_reveal` ignores `HUSH_BASE_URL` and reveals against **the link's own +origin**. A link from another hush deployment must not be posted to this one, +where its id would be meaningless — and silently revealing against the wrong +host would report `gone` for a secret that was never touched. + +## Behaviour worth knowing before you call it + +- **`hush_reveal` is destructive and irreversible.** After it returns, the link + is dead and the intended recipient cannot open it. The tool description says + so, because a model that calls it to "check" a link has burned it. +- **`hush_create` returns the link once.** It cannot be recovered: the key was + never sent to the server, so nothing can rebuild it. +- **A link with no `#fragment` is reported as an error without touching the + secret.** Chat and email clients truncate fragments, and this is the commonest + real failure. The secret is intact and the fix is to ask for the full link — + which the error says, rather than reporting a generic failure. +- **Errors come back as tool errors** (`isError: true`), not protocol errors, so + the model reads the message and can act on it instead of seeing an opaque + transport failure. + +## Protocol notes + +Implemented directly against the JSON-RPC 2.0 stdio transport rather than via an +SDK: the surface needed is `initialize`, `notifications/initialized`, +`tools/list`, `tools/call` and `ping`, which is less code than an SDK +dependency's API churn would cost. + +Two rules the implementation is careful about, both of which produce +hard-to-diagnose host-side failures when broken: + +- **stdout carries protocol frames only.** Every diagnostic goes to stderr. A + stray `Println` corrupts the stream and the host reports an opaque parse error. +- **A notification (no `id`) is never answered.** Replying to one desyncs the + host, which then attributes the unsolicited response to the next request. diff --git a/docs/OPERATIONS.md b/docs/OPERATIONS.md new file mode 100644 index 0000000..51a45c7 --- /dev/null +++ b/docs/OPERATIONS.md @@ -0,0 +1,178 @@ +# Operating hush + +hush is a stateless Go process in front of TTL'd Redis keys. There is no +schema, no queue, no background worker and no durable state of its own, so +almost every incident is one of: Redis is unreachable, the pod is not being +scraped, or someone is abusing anonymous create. + +## Reading the logs + +```bash +make logs # hush, last hour +./scripts/logs.sh 'service:hush level:error' +./scripts/logs.sh 'service:hush category:secret' # the create/reveal/gone lifecycle +./scripts/logs.sh 'service:hush sid:fb26b024452a' # one secret, end to end +``` + +Vector collects pod stdout cluster-wide with no annotation, so hush's JSON lands +in VictoriaLogs automatically. Indexed stream fields are `service`, `level`, +`host`, `unit` — everything else (`request_id`, `sid`, `category`, `error_type`) +is exact-match queryable and deliberately not indexed. + +`level` is lowercase in the corpus. `{level="ERROR"}` matches nothing. + +### `sid`, and why no id is ever logged + +The secret id is the capability: anyone holding it can reveal the secret. It is +never logged. The correlation handle is `sid = sha256(id)[:12]`, which follows +one secret across `secret.created` → `secret.revealed` → `secret.gone` and is +useless for opening it. + +Verified rather than asserted: creating a secret and searching the whole corpus +for its raw id returns zero hits, while its `sid` returns the lifecycle. If you +ever see a 43-character base64url string in a hush log line, that is a **P1 +capability leak** — the id type is built so it cannot happen (see +`internal/secret/id.go`) and a regression means someone added a `Value()` call +at a log site. + +## Alerts + +Routing (`alertmanager.yaml`): `critical` and `high` reach Discord **and** open a +Pantheon incident; `warning` is Discord only. + +### HushRedisUnreachable — critical + +`hush_store_up == 0` for 2m. hush can neither store nor deliver a secret; +readiness fails and the pod has left the Service. + +Nothing is lost — Redis owns the secrets and hush fails closed — but the URL is +down. In order: + +```bash +kubectl -n databases get pod -l app=redis # is Redis up? +kubectl -n projects logs -l app=hush --tail=50 # what does hushd say? +kubectl -n databases exec redis-0 -- redis-cli --no-auth-warning -a "$PW" ACL LIST | grep hush +``` + +That last check matters more than it looks: the Redis pod's init container +reconciles the `default` ACL user on every start. If a change ever dropped the +`hush` user, or dropped `+getdel` from it, the symptom is identical to an +outage — and a missing `+getdel` specifically breaks *only* reveal while create +keeps succeeding, so the service looks half-healthy. + +### HushMetricsAbsent — high + +`absent(hush_store_up)` for 10m. Every other rule reads a hush metric, so +absence silently disables the whole group. + +Most likely cause is not a dead pod but a **dropped scrape target**. vmagent +gates on `prometheus.io/scrape=true` AND a `prometheus.io/port` that *equals* a +declared `containerPort` — via `keepequal`, which drops a mismatch **silently**: +no error, no `up=0`, the target simply never appears. + +```bash +kubectl -n projects get pod -l app=hush -o jsonpath='{.items[0].metadata.annotations}' +kubectl -n projects get pod -l app=hush -o jsonpath='{.items[0].spec.containers[0].ports}' +# the annotation value and the containerPort number must be identical strings +``` + +Then confirm the NetworkPolicy still admits `observability` on 18500; without +it vmagent discovers the target and every scrape is connection-refused. + +### Hush5xxRateHigh — warning + +>5% 5xx over 15m on non-trivial traffic. A 5xx means a secret was +accepted-but-not-stored, or a reveal failed **without** destroying the secret. +Neither loses data, but a caller who saw a 500 on create does not know whether +their link exists. Check `error_type` in the logs. + +### HushRateLimitSustained — warning + +Steady create refusals for 30m. hush is anonymous-create by design, so this is +how bulk automation shows up; a single user retrying cannot sustain it, because +the limit is per IP. + +If it is abuse rather than a busy NAT: + +```bash +kubectl -n projects set env deployment/hush HUSH_REQUIRE_AUTH=true HUSH_CREATE_TOKEN= +``` + +Reveal stays anonymous either way — the recipient is external and holds no +credential. That asymmetry is the design, not an oversight. + +### HushCreateRejectionsHigh — warning + +More than half of creates failing validation. Break down by reason: + +```bash +./scripts/logs.sh 'service:hush level:warn' +# or in Grafana: hush_secrets_rejected_total by (reason) +``` + +`ciphertext_invalid` in bulk means a client is posting something that is not +base64url — either a broken page deploy or someone treating hush as a plaintext +API. `ciphertext_too_large` means someone is trying to use it as a file host. + +## Failure modes that are not alerts + +### A user says "the link says gone" and swears they never opened it + +Three possible causes and hush deliberately cannot tell them apart, because +distinguishing them would leak whether a given link was real: + +1. Someone else opened it — **treat the secret as compromised and rotate it.** +2. It expired. +3. Redis evicted it early (below). + +Assume (1) unless the TTL clearly elapsed. That is the conservative reading and +it is cheap: rotating a credential costs less than a leaked one. + +### Redis evicted a secret before its TTL + +The shared Redis runs `maxmemory-policy allkeys-lru` at 256 MiB, so under memory +pressure it can drop a hush key **before** its TTL fires. + +This is an **availability** risk and never a confidentiality one: eviction only +deletes. A secret can become unavailable early; it can never outlive its TTL and +can never be read twice. For a secret courier that is the correct direction to +fail, which is why `410 gone` does not distinguish it — the user-visible +contract is identical. + +If it starts happening, the fix is upstream (Redis memory, or `volatile-lru`, +which is a cluster-wide change affecting every tenant) rather than anything in +hush. + +### The pod restarts + +Nothing is lost. All state is in Redis. In-flight requests get the two-phase +drain: readiness flips to 503, the load balancer stops sending traffic, then the +process shuts down. `terminationGracePeriodSeconds: 45` exceeds the chassis's +5s drain + 25s shutdown, so the kubelet does not SIGKILL mid-drain. + +### Someone reports a link that "lost its #" + +Chat and email clients truncate URL fragments. The secret is **intact and +unopened** — the fragment never reaches the server, so nothing was consumed. The +reveal page and the MCP tool both say this explicitly rather than reporting a +generic failure. Ask the sender to re-send the whole link. + +## Routine checks + +```bash +make deploy-status # rollout, pods, ingress, certificate +BASE=https://hush.threesix.ai make smoke # end-to-end with real crypto +make alerts-check # rules loaded AND their series exist +``` + +`make smoke` creates and burns a real secret against production. It is safe to +run any time; it touches nothing but its own secret. + +## What has no runbook because it cannot happen + +- **Reading a stored secret as an operator.** There is no key. `kubectl exec` into + Redis and you get ciphertext. +- **Restoring a revealed secret.** `GETDEL` is atomic and there is no backup of + a value that existed for one read. +- **Listing outstanding secrets.** The store contract has no `List` and no + `Exists`. `--scan` in Redis yields opaque keys and opaque values. diff --git a/scripts/alerts-check.sh b/scripts/alerts-check.sh new file mode 100755 index 0000000..d6afd88 --- /dev/null +++ b/scripts/alerts-check.sh @@ -0,0 +1,50 @@ +#!/usr/bin/env bash +# Confirm vmalert has LOADED hush's rules and can evaluate them. +# +# Reading the ConfigMap is not verification: vmalert has no auto-discovery, so a +# rules file can be applied and committed and still be absent because +# victoria-metrics.yaml lacks the matching -rule=, volumeMount and volume. This +# asks vmalert what it actually has. +# +# It also checks each rule's series EXISTS. A rule whose expression references a +# metric nothing exports is a rule that can never fire, which reads identically +# to a healthy service. +set -euo pipefail + +export KUBECONFIG="${KUBECONFIG:-$HOME/.kube/orchard9-k3sf.yaml}" + +kubectl -n observability port-forward svc/vmalert 8880:8880 >/dev/null 2>&1 & +PFA=$! +kubectl -n observability port-forward svc/vmsingle 8428:8428 >/dev/null 2>&1 & +PFS=$! +trap 'kill $PFA $PFS 2>/dev/null || true' EXIT +sleep 3 + +echo "=== rules vmalert has loaded for hush ===" +curl -sS http://localhost:8880/api/v1/rules \ +| python3 -c ' +import json, sys +d = json.load(sys.stdin) +groups = [g for g in d["data"]["groups"] if g["name"] == "hush"] +if not groups: + print(" NO hush group loaded — check the -rule=, volumeMount and volume in victoria-metrics.yaml") + sys.exit(1) +for g in groups: + print(" group {} (file {})".format(g["name"], g.get("file", "?"))) + for r in g["rules"]: + print(" {:32} state={:8} health={} for={}".format( + r.get("name", "?"), r.get("state", "?"), r.get("health", "?"), r.get("duration", "?"))) + if r.get("lastError"): + print(" lastError:", r["lastError"]) +' + +echo +echo "=== do the series each rule reads actually exist? ===" +for metric in hush_store_up hush_secrets_created_total hush_secrets_revealed_total \ + hush_secrets_rejected_total hush_rate_limited_total http_requests_total; do + n=$(curl -sS -G http://localhost:8428/prometheus/api/v1/query \ + --data-urlencode "query=count(${metric}{service=\"hush\"}) or count(${metric})" \ + | python3 -c 'import json,sys; r=json.load(sys.stdin)["data"]["result"]; print(int(float(r[0]["value"][1])) if r else 0)') + if [ "$n" -gt 0 ]; then printf ' ok %-32s %s series\n' "$metric" "$n" + else printf ' MISSING %-29s 0 series — a rule on this can never fire\n' "$metric"; fi +done diff --git a/scripts/format-logs.py b/scripts/format-logs.py new file mode 100755 index 0000000..6a7b236 --- /dev/null +++ b/scripts/format-logs.py @@ -0,0 +1,40 @@ +#!/usr/bin/env python3 +"""Render VictoriaLogs' JSON-lines output as one readable line per entry. + +Kept as a file rather than inlined in logs.sh: quoting a python f-string inside +a shell heredoc inside a pipeline is how you get a SyntaxError that only shows +up against the live cluster. +""" +import json +import sys + +# Fields Vector or the wire format always sets. They are shown in fixed columns +# or are pod plumbing, so they are not repeated in the trailing key=value list. +FIXED = { + "_time", "_stream", "_stream_id", "_msg", "msg", + "level", "service", "env", "host", "unit", "k8s_pod", "k8s_container", +} + +rows = [] +for line in sys.stdin: + line = line.strip() + if not line: + continue + try: + rows.append(json.loads(line)) + except ValueError: + print("unparseable line:", line[:200], file=sys.stderr) + +if not rows: + print("no lines matched — has hush served a request in this window?") + sys.exit(0) + +# Oldest first, so reading top-to-bottom follows the sequence of events. +for r in sorted(rows, key=lambda x: x.get("_time", "")): + ts = r.get("_time", "")[:23] + level = r.get("level", "") + msg = r.get("_msg") or r.get("msg", "") + extra = " ".join(f"{k}={v}" for k, v in sorted(r.items()) if k not in FIXED) + print("{:24} {:8} {:34} {}".format(ts, level, msg, extra)) + +print("\n{} lines".format(len(rows)), file=sys.stderr) diff --git a/scripts/install-mcp.sh b/scripts/install-mcp.sh new file mode 100755 index 0000000..a5026bc --- /dev/null +++ b/scripts/install-mcp.sh @@ -0,0 +1,79 @@ +#!/usr/bin/env bash +# Build hush-mcp and register it with omp. +# +# The server runs LOCALLY and does the AES-256-GCM itself. That is the point: if +# hushd served an MCP endpoint and encrypted server-side, the server could read +# every secret created through MCP, and hush's guarantee would hold for browser +# users while quietly not holding for agent users. This binary is a peer of the +# browser, not of the server. +# +# Idempotent: re-running rebuilds the binary and rewrites only hush's entry in +# ~/.omp/agent/mcp.json, leaving every other server alone. +set -euo pipefail + +BIN_DIR="${BIN_DIR:-$HOME/.local/bin}" +MCP_JSON="${MCP_JSON:-$HOME/.omp/agent/mcp.json}" +BASE_URL="${HUSH_BASE_URL:-https://hush.threesix.ai}" +ROOT="$(cd "$(dirname "$0")/.." && pwd)" + +echo "building hush-mcp" +mkdir -p "$BIN_DIR" +(cd "$ROOT" && go build -trimpath -ldflags="-s -w" -o "$BIN_DIR/hush-mcp" ./cmd/hush-mcp) +echo " installed $BIN_DIR/hush-mcp" + +# Prove the binary speaks MCP before wiring it in. A config pointing at a broken +# server surfaces as an opaque host-side connect failure, so the handshake is +# checked here where the error is legible. +echo "verifying the MCP handshake" +HANDSHAKE=$(printf '%s\n' \ + '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{}}}' \ + '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' \ + | HUSH_BASE_URL="$BASE_URL" "$BIN_DIR/hush-mcp" 2>/dev/null) +echo "$HANDSHAKE" | python3 -c ' +import json, sys +tools = None +for line in sys.stdin: + m = json.loads(line) + if m.get("id") == 1: + print(" protocol", m["result"]["protocolVersion"], "server", m["result"]["serverInfo"]["name"]) + if m.get("id") == 2: + tools = [t["name"] for t in m["result"]["tools"]] +if not tools: + sys.exit(" the server did not answer tools/list") +print(" tools:", ", ".join(tools)) +' + +echo "registering with omp at $MCP_JSON" +mkdir -p "$(dirname "$MCP_JSON")" +[ -f "$MCP_JSON" ] || printf '{"mcpServers":{}}\n' > "$MCP_JSON" +# Back up before touching a config that may hold other servers. +cp "$MCP_JSON" "$MCP_JSON.bak.$(date +%Y%m%d%H%M%S)" + +BIN="$BIN_DIR/hush-mcp" BASE="$BASE_URL" TARGET="$MCP_JSON" python3 - <<'PY' +import json, os + +target = os.environ["TARGET"] +with open(target) as f: + cfg = json.load(f) + +cfg.setdefault("$schema", + "https://raw.githubusercontent.com/can1357/oh-my-pi/main/packages/coding-agent/src/config/mcp-schema.json") +servers = cfg.setdefault("mcpServers", {}) + +# stdio, not http: the encryption has to happen on this machine. +servers["hush"] = { + "type": "stdio", + "command": os.environ["BIN"], + "env": {"HUSH_BASE_URL": os.environ["BASE"]}, + "timeout": 20000, +} + +with open(target, "w") as f: + json.dump(cfg, f, indent=2) + f.write("\n") + +print(" servers now configured:", ", ".join(sorted(servers))) +PY + +echo +echo "done. Restart omp to pick up the new server, then: hush_create / hush_reveal" diff --git a/scripts/logs.sh b/scripts/logs.sh new file mode 100755 index 0000000..fd8bf7e --- /dev/null +++ b/scripts/logs.sh @@ -0,0 +1,34 @@ +#!/usr/bin/env bash +# hush's structured logs out of VictoriaLogs. +# +# VictoriaLogs has no read-path ingress (the only ingress, telemetry.threesix.ai, +# fronts vmauth-WRITE and is bearer-gated), and its NetworkPolicy admits only +# vector, vmauth-write, vmagent and grafana. So a laptop reads it through a +# port-forward, which goes node→apiserver→pod and bypasses the pod-to-pod policy. +# +# Usage: +# ./scripts/logs.sh # hush, last hour +# ./scripts/logs.sh 'service:hush level:error' # any LogsQL +# LIMIT=200 ./scripts/logs.sh 'service:hush category:secret' +set -euo pipefail + +export KUBECONFIG="${KUBECONFIG:-$HOME/.kube/orchard9-k3sf.yaml}" +PORT="${PORT:-9428}" +QUERY="${1:-service:hush _time:1h}" +LIMIT="${LIMIT:-50}" + +kubectl -n observability port-forward svc/victoria-logs "$PORT:9428" >/dev/null 2>&1 & +PF=$! +trap 'kill $PF 2>/dev/null || true' EXIT +for _ in $(seq 1 40); do + curl -sf -o /dev/null -G "http://localhost:$PORT/select/logsql/query" \ + --data-urlencode 'query=*' --data-urlencode 'limit=1' && break + sleep 0.25 +done + +curl -sS -G "http://localhost:$PORT/select/logsql/query" \ + --data-urlencode "query=$QUERY" --data-urlencode "limit=$LIMIT" \ + > /tmp/hush-logs.$$ + +python3 "$(dirname "$0")/format-logs.py" < /tmp/hush-logs.$$ +rm -f /tmp/hush-logs.$$