62c95f8c2f
Written after the service was live, so every command and every number here was
run against the real deployment rather than assumed:
* DEPLOY.md records the things a rebuild needs and git does not hold — the
Redis ACL user (and why +getdel is the one to notice), the GCP Secret
Manager entry, the DNS record, and the three coordinated edits vmalert
needs because it has no ConfigMap auto-discovery.
* It also records two blockers rather than hiding them: Woodpecker is NOT
activated (the token in rdev-credentials returns 401), so pushes do not
deploy yet and the Kaniko Job is the interim path; and the host is
hush.threesix.ai rather than hush.orchard9.ai because orchard9.ai is on
GoDaddy and no GoDaddy credential exists anywhere I can reach.
* OPERATIONS.md is one section per alert, plus the failure modes that are not
alerts — chiefly that "gone" cannot distinguish already-revealed from
expired from LRU-evicted, on purpose, so the operator's default reading of
an unexpected "gone" is that the secret is compromised and should be
rotated.
* scripts/logs.sh and alerts-check.sh verify rather than assert:
alerts-check asks vmalert what it actually loaded AND checks each rule's
series exists, because a rule reading a metric nothing exports can never
fire and looks exactly like a healthy service.
* scripts/smoke.sh is a real client — it generates a key, encrypts, posts only
ciphertext, reveals, decrypts, then asserts the second reveal is 410, that
three GETs did not consume the secret, that missing and malformed ids are
indistinguishable, and that a plaintext field is refused.
install-mcp.sh proves the MCP handshake before writing any config, backs up
mcp.json, and rewrites only hush's entry — a config pointing at a broken server
surfaces as an opaque host-side connect failure, which is worth one extra check
to avoid.
41 lines
1.4 KiB
Python
Executable File
41 lines
1.4 KiB
Python
Executable File
#!/usr/bin/env python3
|
|
"""Render VictoriaLogs' JSON-lines output as one readable line per entry.
|
|
|
|
Kept as a file rather than inlined in logs.sh: quoting a python f-string inside
|
|
a shell heredoc inside a pipeline is how you get a SyntaxError that only shows
|
|
up against the live cluster.
|
|
"""
|
|
import json
|
|
import sys
|
|
|
|
# Fields Vector or the wire format always sets. They are shown in fixed columns
|
|
# or are pod plumbing, so they are not repeated in the trailing key=value list.
|
|
FIXED = {
|
|
"_time", "_stream", "_stream_id", "_msg", "msg",
|
|
"level", "service", "env", "host", "unit", "k8s_pod", "k8s_container",
|
|
}
|
|
|
|
rows = []
|
|
for line in sys.stdin:
|
|
line = line.strip()
|
|
if not line:
|
|
continue
|
|
try:
|
|
rows.append(json.loads(line))
|
|
except ValueError:
|
|
print("unparseable line:", line[:200], file=sys.stderr)
|
|
|
|
if not rows:
|
|
print("no lines matched — has hush served a request in this window?")
|
|
sys.exit(0)
|
|
|
|
# Oldest first, so reading top-to-bottom follows the sequence of events.
|
|
for r in sorted(rows, key=lambda x: x.get("_time", "")):
|
|
ts = r.get("_time", "")[:23]
|
|
level = r.get("level", "")
|
|
msg = r.get("_msg") or r.get("msg", "")
|
|
extra = " ".join(f"{k}={v}" for k, v in sorted(r.items()) if k not in FIXED)
|
|
print("{:24} {:8} {:34} {}".format(ts, level, msg, extra))
|
|
|
|
print("\n{} lines".format(len(rows)), file=sys.stderr)
|