hush: one-time secret links the server cannot read
Paste a secret, get a link, send it. The first person to open it and press
Reveal sees the secret; the link dies at that moment. The recipient needs a
browser and nothing else — no account, no client, no installed tooling.
The server cannot read what it stores. AES-256-GCM happens in the browser and
the key lives in the URL fragment, which browsers never transmit, so hushd
holds ciphertext and no key material. That is a property of where the key sits
rather than a promise about our conduct, which is why there is deliberately no
endpoint accepting a plaintext secret and no server-side-encryption fallback:
two guarantees behind one URL would be worse than one honest guarantee.
Three decisions carry the design:
* GET /s/{id} touches NO storage, not even to check existence. Slack, Teams,
WhatsApp, iMessage and Outlook Safe Links all fetch a URL before a human
sees it, so destroying on GET would destroy most secrets in transit and the
recipient's "already used" would be indistinguishable from interception.
Only POST /reveal consumes. Bot user-agent detection is an arms race;
removing the side effect from GET is not. Pinned by
TestGettingTheRevealPageNeverConsumesTheSecret.
* Destruction is one Redis GETDEL, which is atomic. GET-then-DEL has a window
where two simultaneous readers both win, and for a one-time secret that
window is the product. The store contract demands atomicity and the same
concurrency test runs against both implementations.
* Missing, already-revealed, expired and evicted are ONE indistinguishable
410. Separating them would confirm to a prober that a given link was real.
The secret id IS the capability, so secret.ID is a struct whose every
accidental path — %v, %s, String(), slog, json.Marshal — emits a redacted
handle or refuses, and the raw value needs an explicit Value(). The first
version tried to prevent leaks by implementing no String() at all; its own test
caught that Go's fmt prints unexported fields anyway, so forbidding the method
had removed the control rather than the leak.
Operationally: structured JSON on stdout in the fleet's wire format, which
Vector already collects with no annotation; six hush_* metrics on the chassis
registry with no id, IP or path in any label; five alert rules wired into
vmalert. The public Ingress enumerates /, /s/ and /api/ so /metrics, /healthz
and /readyz share the port but are unreachable from the internet — no
basic-auth middleware to maintain and get wrong.
Dependencies are vendored because go-chassis is private: the Woodpecker test
step and the in-cluster Kaniko build both run -mod=vendor with GOPROXY=off and
hold no git credential.
cmd/hush-mcp is a stdio MCP server doing the same client-side crypto locally,
so using hush from an agent preserves the same guarantee as using it from a
browser.
This commit is contained in:
commit
4d9a26498e
2
.gitignore
vendored
Normal file
2
.gitignore
vendored
Normal file
@ -0,0 +1,2 @@
|
||||
.build/
|
||||
*.tmp
|
||||
62
.woodpecker.yml
Normal file
62
.woodpecker.yml
Normal file
@ -0,0 +1,62 @@
|
||||
# hush CI/CD. Origin must be Gitea (git.threesix.ai) — that remote carries the
|
||||
# webhook. Pushing to the GitHub mirror does not deploy anything.
|
||||
#
|
||||
# push to main → test → Kaniko amd64 build → Zot → kubectl set image on `projects`
|
||||
|
||||
clone:
|
||||
git:
|
||||
image: woodpeckerci/plugin-git
|
||||
settings:
|
||||
depth: 1
|
||||
|
||||
steps:
|
||||
test:
|
||||
# 1.26, not the 1.25 in the template: go-chassis declares `go 1.26.0` and an
|
||||
# older toolchain refuses to build it.
|
||||
image: golang:1.26-alpine
|
||||
environment:
|
||||
# Hermetic, and deliberately so. go-chassis is a PRIVATE module and this
|
||||
# container has no git credential, so a build that reached for the network
|
||||
# would fail here — the vendored tree is what makes CI possible at all.
|
||||
# GOPROXY=off turns "silently fetched from a proxy" into a hard error.
|
||||
GOFLAGS: -mod=vendor
|
||||
GOPROXY: "off"
|
||||
commands:
|
||||
- go vet ./...
|
||||
- go test ./...
|
||||
# gofmt as a gate, not a suggestion. -l lists offenders; the test fails if
|
||||
# the list is non-empty. vendor/ is excluded because it is not ours.
|
||||
- test -z "$(gofmt -l ./cmd ./internal)" || { gofmt -l ./cmd ./internal; echo "gofmt"; exit 1; }
|
||||
when:
|
||||
branch: main
|
||||
event: push
|
||||
|
||||
build:
|
||||
image: woodpeckerci/plugin-kaniko
|
||||
settings:
|
||||
registry: registry.threesix.ai
|
||||
repo: hush/api
|
||||
tags:
|
||||
- latest
|
||||
# SHA-tagged as well as latest: `latest` cannot pin a rollback.
|
||||
- ${CI_COMMIT_SHA:0:8}
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
cache: true
|
||||
skip_tls_verify: true # internal Zot registry, self-signed cert
|
||||
when:
|
||||
branch: main
|
||||
event: push
|
||||
|
||||
deploy:
|
||||
image: bitnami/kubectl:latest
|
||||
commands:
|
||||
- kubectl set image deployment/hush hushd=registry.threesix.ai/hush/api:${CI_COMMIT_SHA:0:8} -n projects
|
||||
- kubectl rollout status deployment/hush -n projects --timeout=120s
|
||||
# Prove the rolled pod actually serves, not merely that it became Ready.
|
||||
# A pod can pass readiness and still be the wrong image if `set image`
|
||||
# silently matched nothing.
|
||||
- kubectl -n projects get deployment hush -o jsonpath='{.spec.template.spec.containers[0].image}{"\n"}'
|
||||
when:
|
||||
branch: main
|
||||
event: push
|
||||
44
Dockerfile
Normal file
44
Dockerfile
Normal file
@ -0,0 +1,44 @@
|
||||
# hushd. Built amd64 IN-CLUSTER by Kaniko — never `docker build` locally, which
|
||||
# on an Apple Silicon laptop produces an arm64 image the cluster cannot run.
|
||||
#
|
||||
# The build is HERMETIC: `-mod=vendor` with `GOPROXY=off` and `GOFLAGS=-mod=vendor`
|
||||
# means no module download and no network. That is not an optimisation — it is
|
||||
# required, because github.com/orchard9/go-chassis is a private module and the
|
||||
# build container holds no git credential. `make vendor` is what moves versions.
|
||||
FROM golang:1.26-alpine AS build
|
||||
WORKDIR /src
|
||||
|
||||
# Vendored, so this is the whole dependency graph: no go.sum verification step,
|
||||
# no proxy, no cache warm-up.
|
||||
COPY go.mod go.sum ./
|
||||
COPY vendor/ ./vendor/
|
||||
COPY cmd/ ./cmd/
|
||||
COPY internal/ ./internal/
|
||||
|
||||
# Fail loudly if anything reaches for the network, rather than silently falling
|
||||
# back to a proxy that will not be there in CI.
|
||||
ENV GOFLAGS=-mod=vendor GOPROXY=off CGO_ENABLED=0
|
||||
|
||||
# Tests run in the CI step, not here: a Kaniko layer that runs tests caches
|
||||
# their result and stops re-running them. Keep the image build to building.
|
||||
RUN go build -trimpath -ldflags="-s -w" -o /out/hushd ./cmd/hushd
|
||||
|
||||
# distroless static + nonroot: no shell, no package manager, no libc surface.
|
||||
# hushd needs only TCP and the CA bundle distroless already carries.
|
||||
FROM gcr.io/distroless/static-debian12:nonroot AS runtime
|
||||
WORKDIR /
|
||||
|
||||
COPY --from=build /out/hushd /hushd
|
||||
|
||||
# Numeric, not "nonroot": with a non-numeric USER the kubelet cannot verify
|
||||
# runAsNonRoot and refuses to start the pod.
|
||||
USER 65532:65532
|
||||
|
||||
# HTML templates are embedded in the binary (internal/web, go:embed), so there
|
||||
# is no asset directory to mount, drift, or go missing at runtime.
|
||||
EXPOSE 18500
|
||||
|
||||
# Exec form so SIGTERM reaches the process directly. The chassis's two-phase
|
||||
# drain (readiness 503, wait, then shutdown) depends on receiving it, and the
|
||||
# Deployment's terminationGracePeriodSeconds is sized to outlast that drain.
|
||||
ENTRYPOINT ["/hushd"]
|
||||
91
Makefile
Normal file
91
Makefile
Normal file
@ -0,0 +1,91 @@
|
||||
.DEFAULT_GOAL := help
|
||||
.PHONY: help fmt vet test test-redis build run dev dev-stop smoke vendor verify ci \
|
||||
mcp mcp-install deploy-manifests deploy-status logs alerts-check
|
||||
|
||||
# Local development Redis. A real server, not a mock: the one-time guarantee
|
||||
# rests on GETDEL being atomic, and a fake cannot prove that.
|
||||
DEV_REDIS_PORT ?= 6389
|
||||
DEV_REDIS_URL ?= redis://127.0.0.1:$(DEV_REDIS_PORT)/5
|
||||
HUSH_PORT ?= 18500
|
||||
BASE ?= http://127.0.0.1:$(HUSH_PORT)
|
||||
|
||||
# The public deployment. `hushd` runs one replica in the `projects` namespace.
|
||||
KUBECONFIG_FILE ?= $(HOME)/.kube/orchard9-k3sf.yaml
|
||||
NS ?= projects
|
||||
HOST ?= hush.threesix.ai
|
||||
|
||||
help: ## Show this help
|
||||
@grep -hE '^[a-zA-Z_-]+:.*?## ' $(MAKEFILE_LIST) \
|
||||
| awk 'BEGIN{FS=":.*?## "}{printf " \033[36m%-18s\033[0m %s\n", $$1, $$2}'
|
||||
|
||||
fmt: ## Format everything outside vendor/
|
||||
@gofmt -w ./cmd ./internal
|
||||
|
||||
vet: ## go vet
|
||||
@go vet ./...
|
||||
|
||||
test: ## Unit tests. No Redis, no network, no container.
|
||||
@go test ./...
|
||||
|
||||
test-redis: ## Run the store contract suite against a REAL Redis as well
|
||||
@$(MAKE) --no-print-directory dev-redis
|
||||
@HUSH_TEST_REDIS_URL=$(DEV_REDIS_URL) go test ./internal/store/... -count=1
|
||||
|
||||
build: ## Build both binaries into .build/
|
||||
@mkdir -p .build
|
||||
@go build -o .build/hushd ./cmd/hushd
|
||||
@go build -o .build/hush-mcp ./cmd/hush-mcp
|
||||
|
||||
# `vendor` is not a convenience: github.com/orchard9/go-chassis is a PRIVATE
|
||||
# module, so neither the Woodpecker test step nor the in-cluster Kaniko build
|
||||
# can fetch it. Vendoring makes both hermetic — they build with -mod=vendor and
|
||||
# no network and no credential. This target is the only way dep versions move.
|
||||
vendor: ## Refresh vendor/ after a dependency change
|
||||
@GOPRIVATE=github.com/orchard9 go mod tidy
|
||||
@GOPRIVATE=github.com/orchard9 go mod vendor
|
||||
@go build -mod=vendor ./... && echo "vendor/ builds hermetically"
|
||||
|
||||
verify: ## Prove the vendored tree builds with no network, exactly as CI does
|
||||
@GOFLAGS=-mod=vendor GOPROXY=off go build ./... \
|
||||
&& GOFLAGS=-mod=vendor GOPROXY=off go test ./... >/dev/null \
|
||||
&& echo "hermetic build OK (GOPROXY=off, -mod=vendor)"
|
||||
|
||||
ci: fmt vet test verify ## Everything the pipeline runs
|
||||
|
||||
dev-redis:
|
||||
@redis-cli -p $(DEV_REDIS_PORT) ping >/dev/null 2>&1 || { \
|
||||
echo "starting redis on :$(DEV_REDIS_PORT)"; \
|
||||
redis-server --port $(DEV_REDIS_PORT) --daemonize yes --save '' --appendonly no; \
|
||||
for i in $$(seq 1 30); do redis-cli -p $(DEV_REDIS_PORT) ping >/dev/null 2>&1 && break; sleep 0.2; done; }
|
||||
@redis-cli -p $(DEV_REDIS_PORT) ping | sed 's/^/redis: /'
|
||||
|
||||
dev: dev-redis build ## Run hushd locally against a local Redis
|
||||
@echo "hushd on $(BASE) — open it in a browser"
|
||||
@APP_ENV=dev REDIS_URL=$(DEV_REDIS_URL) HUSH_PORT=$(HUSH_PORT) .build/hushd
|
||||
|
||||
dev-stop: ## Stop the local Redis
|
||||
@redis-cli -p $(DEV_REDIS_PORT) shutdown nosave 2>/dev/null || true
|
||||
@echo "stopped"
|
||||
|
||||
smoke: ## Full create -> reveal -> gone against a locally running hushd
|
||||
@BASE=$(BASE) ./scripts/smoke.sh
|
||||
|
||||
mcp: ## Build and install the MCP server, then register it with omp
|
||||
@$(MAKE) --no-print-directory mcp-install
|
||||
|
||||
mcp-install:
|
||||
@./scripts/install-mcp.sh
|
||||
|
||||
deploy-manifests: ## Apply the k8s manifests (do this BEFORE the first push)
|
||||
@KUBECONFIG=$(KUBECONFIG_FILE) kubectl apply -f deployments/k8s/hush.yaml
|
||||
|
||||
deploy-status: ## Rollout, pods, ingress and certificate
|
||||
@KUBECONFIG=$(KUBECONFIG_FILE) kubectl -n $(NS) rollout status deployment/hush --timeout=90s
|
||||
@KUBECONFIG=$(KUBECONFIG_FILE) kubectl -n $(NS) get pod,svc,ingress -l app=hush
|
||||
@KUBECONFIG=$(KUBECONFIG_FILE) kubectl -n $(NS) get certificate hush-tls 2>/dev/null || true
|
||||
|
||||
logs: ## Tail hush's structured logs out of VictoriaLogs
|
||||
@./scripts/logs.sh
|
||||
|
||||
alerts-check: ## Confirm vmalert has loaded hush's rules
|
||||
@./scripts/alerts-check.sh
|
||||
118
README.md
Normal file
118
README.md
Normal file
@ -0,0 +1,118 @@
|
||||
# hush
|
||||
|
||||
Send someone a secret over a link that works once.
|
||||
|
||||
**Production:** <https://hush.threesix.ai>
|
||||
|
||||
Paste a secret, get a link, send the link. The first person to open it and press
|
||||
**Reveal** sees the secret; the link is dead from that moment. Nobody needs an
|
||||
account, a client, or anything installed — a browser is the whole requirement.
|
||||
|
||||
The server cannot read what you sent. Encryption happens in your browser and the
|
||||
key lives in the URL *fragment* (`…/s/ID#KEY`), which browsers never transmit.
|
||||
hush stores ciphertext it has no way to open. That is not a promise about our
|
||||
operational discipline; it is a property of where the key sits.
|
||||
|
||||
## What one-time actually buys you
|
||||
|
||||
Worth being precise, because "one-time link" is often oversold:
|
||||
|
||||
- **Bounded exposure.** The secret is fetchable once, for at most its TTL, then it
|
||||
is gone. A credential sitting in a Slack thread is fetchable forever by anyone
|
||||
who later gains access to that thread.
|
||||
- **Tamper evidence.** If your recipient says "already used", someone else opened
|
||||
it. You have learned something a plain paste never tells you.
|
||||
- **Nothing at rest to steal.** A dump of hush's Redis yields ciphertext and no keys.
|
||||
|
||||
And what it does not buy you:
|
||||
|
||||
- **It does not protect the link.** Whatever channel carries the link could be read
|
||||
by whoever can read that channel. One-time-ness limits the damage and makes it
|
||||
detectable; it does not make the channel private.
|
||||
- **It does not authenticate the reader.** Anyone holding the link can open it.
|
||||
The link *is* the capability. Treat it like the secret it carries.
|
||||
|
||||
If a secret must reach one specific verified human and nobody else, this is the
|
||||
wrong tool — use a channel with identity.
|
||||
|
||||
## Usage
|
||||
|
||||
### In a browser
|
||||
|
||||
1. Open <https://hush.threesix.ai>.
|
||||
2. Paste the secret, pick a lifetime, press **Create link**.
|
||||
3. Copy the link and send it however you like.
|
||||
4. The recipient opens it, presses **Reveal**, and reads it once.
|
||||
|
||||
### Why there is a button
|
||||
|
||||
Slack, Teams, WhatsApp, iMessage and Outlook Safe Links all fetch a URL to build
|
||||
a preview *before* any human sees it. A service that destroys on `GET` therefore
|
||||
destroys most secrets in transit, and the recipient's "already used" is
|
||||
indistinguishable from a real interception.
|
||||
|
||||
So in hush, `GET /s/{id}` is a static page that touches no storage at all. Only
|
||||
`POST /s/{id}/reveal` reads and destroys. Link previewers are harmless by
|
||||
construction, not by user-agent guessing.
|
||||
|
||||
### API
|
||||
|
||||
The API takes **ciphertext**. There is no endpoint that accepts a plaintext
|
||||
secret, because such an endpoint would make the server able to read secrets and
|
||||
the claim at the top of this file would become a matter of trust rather than
|
||||
arithmetic.
|
||||
|
||||
```
|
||||
POST /api/secrets
|
||||
{ "ciphertext": "<base64url AES-256-GCM, nonce prepended>", "ttl_seconds": 86400 }
|
||||
→ 201 { "id": "…", "expires_at": "2026-09-04T…Z", "ttl_seconds": 86400 }
|
||||
|
||||
POST /api/secrets/{id}/reveal
|
||||
→ 200 { "ciphertext": "…" } first caller only, secret destroyed
|
||||
→ 410 { "error": { "code": "gone" } } every other case
|
||||
```
|
||||
|
||||
`410 gone` is returned identically whether the id never existed, was already
|
||||
revealed, or expired. Distinguishing those would confirm to an attacker that a
|
||||
particular link once existed.
|
||||
|
||||
`GET /` serves the create page, `GET /s/{id}` the reveal page. `/healthz`,
|
||||
`/readyz` and `/metrics` are served on the same port but are **not routed by the
|
||||
public ingress** — they are reachable in-cluster only.
|
||||
|
||||
### From an agent, over MCP
|
||||
|
||||
`cmd/hush-mcp` is a stdio MCP server exposing two tools, `hush_create` and
|
||||
`hush_reveal`. It runs **locally** and does the encryption on your machine, so
|
||||
using hush from an agent preserves the same zero-knowledge property as using it
|
||||
from a browser. See [docs/MCP.md](docs/MCP.md).
|
||||
|
||||
## Limits
|
||||
|
||||
| Thing | Value | Why |
|
||||
|---|---|---|
|
||||
| Ciphertext | ≤ 64 KiB | It is a courier for credentials, not a file host |
|
||||
| TTL | 5m … 7d, default 24h | Long enough to be useful, short enough to bound exposure |
|
||||
| Rate limit | 30 creates / 10 min / IP | Anonymous create is otherwise a free blob host |
|
||||
| Reveals per secret | exactly 1 | The product |
|
||||
|
||||
## Operating it
|
||||
|
||||
- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — how it works and why each choice
|
||||
- [docs/DEPLOY.md](docs/DEPLOY.md) — pipeline, DNS, credentials, first deploy
|
||||
- [docs/OPERATIONS.md](docs/OPERATIONS.md) — alert runbook, log queries, failure modes
|
||||
- [docs/MCP.md](docs/MCP.md) — the MCP server and how to install it
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
make help # every target
|
||||
make test # unit tests, no external dependencies
|
||||
make dev # a local Redis in Docker + hushd on :18500
|
||||
make smoke # full create → reveal → gone against the local instance
|
||||
make vendor # refresh vendor/ after a dependency change
|
||||
```
|
||||
|
||||
`go-chassis` is a private module, so dependencies are **vendored** and both CI
|
||||
and the container build run with `-mod=vendor` and no network. `make vendor` is
|
||||
the only way dependency versions change.
|
||||
141
cmd/hush-mcp/crypto_test.go
Normal file
141
cmd/hush-mcp/crypto_test.go
Normal file
@ -0,0 +1,141 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/base64"
|
||||
"os/exec"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestSealOpenRoundTrip(t *testing.T) {
|
||||
for _, plain := range []string{
|
||||
"hunter2",
|
||||
"",
|
||||
strings.Repeat("x", 40000),
|
||||
"unicode: ✓ 漢字 🔐",
|
||||
"multi\nline\nwith\ttabs",
|
||||
} {
|
||||
ct, key, err := seal(plain)
|
||||
if err != nil {
|
||||
t.Fatalf("seal(%d bytes): %v", len(plain), err)
|
||||
}
|
||||
got, err := open(ct, key)
|
||||
if err != nil {
|
||||
t.Fatalf("open: %v", err)
|
||||
}
|
||||
if got != plain {
|
||||
t.Fatalf("round trip changed the plaintext (%d bytes)", len(plain))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestSealProducesTheBrowsersWireFormat(t *testing.T) {
|
||||
ct, key, err := seal("x")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// base64url, unpadded, matching base64.RawURLEncoding on the server and
|
||||
// the b64u helper in templates/base.html. Three implementations, one
|
||||
// spelling — a mismatch here means a link minted by one client cannot be
|
||||
// opened by another.
|
||||
for name, v := range map[string]string{"ciphertext": ct, "key": key} {
|
||||
if strings.ContainsAny(v, "+/=") {
|
||||
t.Fatalf("%s %q uses the standard base64 alphabet or padding; it must be base64url unpadded", name, v)
|
||||
}
|
||||
if _, err := base64.RawURLEncoding.DecodeString(v); err != nil {
|
||||
t.Fatalf("%s does not decode as base64url: %v", name, err)
|
||||
}
|
||||
}
|
||||
// 256-bit key.
|
||||
raw, _ := base64.RawURLEncoding.DecodeString(key)
|
||||
if len(raw) != 32 {
|
||||
t.Fatalf("key is %d bytes, want 32 (AES-256)", len(raw))
|
||||
}
|
||||
// 96-bit nonce prepended, then at least the GCM tag.
|
||||
blob, _ := base64.RawURLEncoding.DecodeString(ct)
|
||||
if len(blob) < 12+16 {
|
||||
t.Fatalf("ciphertext is %d bytes, too short for a 12-byte nonce plus a 16-byte tag", len(blob))
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenRefusesAWrongKeyAndTamperedCiphertext(t *testing.T) {
|
||||
ct, _, err := seal("secret")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_, otherKey, _ := seal("unrelated")
|
||||
|
||||
if _, err := open(ct, otherKey); err == nil {
|
||||
t.Fatal("open() accepted a key that does not belong to this ciphertext")
|
||||
}
|
||||
|
||||
// GCM is authenticated: a flipped byte must fail, not decrypt to garbage.
|
||||
blob, _ := base64.RawURLEncoding.DecodeString(ct)
|
||||
blob[len(blob)-1] ^= 0xff
|
||||
_, key, _ := seal("x")
|
||||
if _, err := open(base64.RawURLEncoding.EncodeToString(blob), key); err == nil {
|
||||
t.Fatal("open() accepted tampered ciphertext")
|
||||
}
|
||||
|
||||
if _, err := open("!!!not base64!!!", key); err == nil {
|
||||
t.Fatal("open() accepted a non-base64url ciphertext")
|
||||
}
|
||||
if _, err := open(ct, "!!!"); err == nil {
|
||||
t.Fatal("open() accepted a non-base64url key")
|
||||
}
|
||||
}
|
||||
|
||||
// Cross-implementation check: a blob sealed by this Go client must decrypt with
|
||||
// Python's AES-GCM, and vice versa. This is what proves the MCP client, the
|
||||
// browser and the smoke script really share one format rather than three
|
||||
// self-consistent ones.
|
||||
func TestWireFormatMatchesAnIndependentImplementation(t *testing.T) {
|
||||
if _, err := exec.LookPath("python3"); err != nil {
|
||||
t.Skip("python3 unavailable")
|
||||
}
|
||||
const plain = "cross-implementation-secret"
|
||||
|
||||
ct, key, err := seal(plain)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// Go -> Python
|
||||
out, err := exec.Command("python3", "-c", `
|
||||
import base64, sys
|
||||
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
|
||||
u = lambda s: base64.urlsafe_b64decode(s + "=" * (-len(s) % 4))
|
||||
blob, key = u(sys.argv[1]), u(sys.argv[2])
|
||||
sys.stdout.write(AESGCM(key).decrypt(blob[:12], blob[12:], None).decode())
|
||||
`, ct, key).Output()
|
||||
if err != nil {
|
||||
t.Skipf("python cryptography unavailable: %v", err)
|
||||
}
|
||||
if string(out) != plain {
|
||||
t.Fatalf("python decrypted our ciphertext to %q, want %q", out, plain)
|
||||
}
|
||||
|
||||
// Python -> Go
|
||||
pyOut, err := exec.Command("python3", "-c", `
|
||||
import base64, os
|
||||
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
|
||||
key = AESGCM.generate_key(bit_length=256); nonce = os.urandom(12)
|
||||
blob = nonce + AESGCM(key).encrypt(nonce, b"from-python", None)
|
||||
b = lambda x: base64.urlsafe_b64encode(x).decode().rstrip("=")
|
||||
print(b(blob), b(key))
|
||||
`).Output()
|
||||
if err != nil {
|
||||
t.Fatalf("python encrypt failed: %v", err)
|
||||
}
|
||||
parts := strings.Fields(string(pyOut))
|
||||
if len(parts) != 2 {
|
||||
t.Fatalf("unexpected python output %q", pyOut)
|
||||
}
|
||||
got, err := open(parts[0], parts[1])
|
||||
if err != nil {
|
||||
t.Fatalf("could not open python's ciphertext: %v", err)
|
||||
}
|
||||
if got != "from-python" {
|
||||
t.Fatalf("opened python's ciphertext to %q", got)
|
||||
}
|
||||
}
|
||||
304
cmd/hush-mcp/main.go
Normal file
304
cmd/hush-mcp/main.go
Normal file
@ -0,0 +1,304 @@
|
||||
// Command hush-mcp exposes hush to an MCP host (omp, Claude Code, any client)
|
||||
// as two tools: hush_create and hush_reveal.
|
||||
//
|
||||
// It runs LOCALLY, on the operator's machine, and does the AES-256-GCM itself.
|
||||
// That is the whole reason it is a stdio binary rather than an endpoint served
|
||||
// by hushd: if the server did the encrypting, the server could read every
|
||||
// secret created through MCP, and hush's central claim would hold for browser
|
||||
// users and quietly not hold for agent users. Two guarantees behind one URL is
|
||||
// worse than one honest guarantee.
|
||||
//
|
||||
// So this binary is a peer of the browser, not a peer of the server: it mints
|
||||
// the key, encrypts, posts ciphertext, and assembles the `#fragment` link.
|
||||
// hushd sees exactly what it sees from a browser.
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"crypto/aes"
|
||||
"crypto/cipher"
|
||||
"crypto/rand"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
const version = "0.1.0"
|
||||
|
||||
// stderr is indirected so tests can capture diagnostics. stdout is reserved
|
||||
// for protocol frames — a stray write there corrupts the stream.
|
||||
var stderr io.Writer = os.Stderr
|
||||
|
||||
// b64 matches the server and the browser: base64url, unpadded. One spelling of
|
||||
// the wire format across all three implementations.
|
||||
var b64 = base64.RawURLEncoding
|
||||
|
||||
func main() {
|
||||
base := strings.TrimSuffix(os.Getenv("HUSH_BASE_URL"), "/")
|
||||
if base == "" {
|
||||
base = "https://hush.threesix.ai"
|
||||
}
|
||||
c := &client{
|
||||
base: base,
|
||||
http: &http.Client{Timeout: 15 * time.Second},
|
||||
// A create token is only needed if the deployment has closed anonymous
|
||||
// create (HUSH_REQUIRE_AUTH). Empty is the normal case.
|
||||
token: os.Getenv("HUSH_CREATE_TOKEN"),
|
||||
}
|
||||
|
||||
srv := NewServer("hush", version, os.Stdout, tools(c))
|
||||
if err := srv.Serve(os.Stdin); err != nil {
|
||||
fmt.Fprintf(stderr, "hush-mcp: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func tools(c *client) []Tool {
|
||||
return []Tool{
|
||||
{
|
||||
Name: "hush_create",
|
||||
Title: "Create a one-time secret link",
|
||||
Description: "Encrypt a secret locally and store the ciphertext on hush, returning a " +
|
||||
"link that works exactly once. The encryption key is generated on this machine and " +
|
||||
"travels only in the link's #fragment, so the hush server never receives it and " +
|
||||
"cannot read the secret. Use this to hand a credential to someone instead of " +
|
||||
"pasting it into chat or email. The returned link is shown once and cannot be " +
|
||||
"recovered — pass it on immediately.",
|
||||
Schema: map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"secret": map[string]any{
|
||||
"type": "string",
|
||||
"description": "The plaintext to send. Never leaves this machine unencrypted.",
|
||||
},
|
||||
"ttl_seconds": map[string]any{
|
||||
"type": "integer",
|
||||
"description": "Lifetime in seconds, 300 to 604800. Defaults to 86400 (24h).",
|
||||
"minimum": 300,
|
||||
"maximum": 604800,
|
||||
},
|
||||
},
|
||||
"required": []string{"secret"},
|
||||
"additionalProperties": false,
|
||||
},
|
||||
Handler: c.create,
|
||||
},
|
||||
{
|
||||
Name: "hush_reveal",
|
||||
Title: "Open a one-time secret link",
|
||||
Description: "Fetch and decrypt a hush link, DESTROYING it in the process. This is " +
|
||||
"irreversible: after this call the link is dead and nobody else can open it, " +
|
||||
"including the intended recipient. Only call this on a link meant for you.",
|
||||
Schema: map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"link": map[string]any{
|
||||
"type": "string",
|
||||
"description": "The full hush link including the #fragment key.",
|
||||
},
|
||||
},
|
||||
"required": []string{"link"},
|
||||
"additionalProperties": false,
|
||||
},
|
||||
Handler: c.reveal,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
type client struct {
|
||||
base string
|
||||
http *http.Client
|
||||
token string
|
||||
}
|
||||
|
||||
func (c *client) create(raw json.RawMessage) (string, error) {
|
||||
var args struct {
|
||||
Secret string `json:"secret"`
|
||||
TTLSeconds int64 `json:"ttl_seconds"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &args); err != nil {
|
||||
return "", fmt.Errorf("bad arguments: %w", err)
|
||||
}
|
||||
if args.Secret == "" {
|
||||
return "", errors.New("secret is empty — nothing to send")
|
||||
}
|
||||
|
||||
ciphertext, key, err := seal(args.Secret)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("encrypt locally: %w", err)
|
||||
}
|
||||
|
||||
body, _ := json.Marshal(map[string]any{"ciphertext": ciphertext, "ttl_seconds": args.TTLSeconds})
|
||||
req, err := http.NewRequest(http.MethodPost, c.base+"/api/secrets", bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
if c.token != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+c.token)
|
||||
}
|
||||
|
||||
res, err := c.http.Do(req)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("reach hush at %s: %w", c.base, err)
|
||||
}
|
||||
defer res.Body.Close()
|
||||
payload, _ := io.ReadAll(io.LimitReader(res.Body, 1<<20))
|
||||
|
||||
if res.StatusCode != http.StatusCreated {
|
||||
return "", fmt.Errorf("hush refused the secret (%s): %s", res.Status, apiMessage(payload))
|
||||
}
|
||||
var out struct {
|
||||
ID string `json:"id"`
|
||||
ExpiresAt string `json:"expires_at"`
|
||||
TTLSeconds int64 `json:"ttl_seconds"`
|
||||
}
|
||||
if err := json.Unmarshal(payload, &out); err != nil || out.ID == "" {
|
||||
return "", fmt.Errorf("unexpected response from hush: %s", string(payload))
|
||||
}
|
||||
|
||||
// The key is appended HERE, on this machine. It was never in the request.
|
||||
link := c.base + "/s/" + out.ID + "#" + key
|
||||
return fmt.Sprintf(
|
||||
"%s\n\nOne-time link — works exactly once, expires %s.\n"+
|
||||
"The key is in the #fragment, so hush cannot read the secret.\n"+
|
||||
"This link cannot be shown again: send it now.",
|
||||
link, out.ExpiresAt), nil
|
||||
}
|
||||
|
||||
func (c *client) reveal(raw json.RawMessage) (string, error) {
|
||||
var args struct {
|
||||
Link string `json:"link"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &args); err != nil {
|
||||
return "", fmt.Errorf("bad arguments: %w", err)
|
||||
}
|
||||
|
||||
u, err := url.Parse(strings.TrimSpace(args.Link))
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("not a URL: %w", err)
|
||||
}
|
||||
if u.Fragment == "" {
|
||||
// The commonest real failure: a chat client or mail rewriter dropped
|
||||
// the fragment. Say so precisely, because the secret is still intact
|
||||
// and the fix is to ask the sender for the whole link.
|
||||
return "", errors.New("this link has no #fragment, so it carries no key. " +
|
||||
"Chat and email clients sometimes truncate it — ask the sender for the full link. " +
|
||||
"The secret has NOT been opened.")
|
||||
}
|
||||
id := strings.TrimPrefix(u.Path, "/s/")
|
||||
if id == "" || strings.Contains(id, "/") {
|
||||
return "", fmt.Errorf("cannot find a secret id in the path %q", u.Path)
|
||||
}
|
||||
|
||||
// Reveal against the link's OWN origin, not the configured base: a link
|
||||
// from a different hush deployment must not be posted to this one, where
|
||||
// the id would be meaningless.
|
||||
origin := u.Scheme + "://" + u.Host
|
||||
res, err := c.http.Post(origin+"/api/secrets/"+url.PathEscape(id)+"/reveal", "application/json", nil)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("reach hush at %s: %w", origin, err)
|
||||
}
|
||||
defer res.Body.Close()
|
||||
payload, _ := io.ReadAll(io.LimitReader(res.Body, 1<<20))
|
||||
|
||||
if res.StatusCode == http.StatusGone {
|
||||
return "", errors.New("gone: this link was already opened, expired, or never existed. " +
|
||||
"If you did not open it yourself, assume someone else did and ask the sender to rotate the secret.")
|
||||
}
|
||||
if res.StatusCode != http.StatusOK {
|
||||
return "", fmt.Errorf("hush returned %s: %s", res.Status, apiMessage(payload))
|
||||
}
|
||||
|
||||
var out struct {
|
||||
Ciphertext string `json:"ciphertext"`
|
||||
}
|
||||
if err := json.Unmarshal(payload, &out); err != nil {
|
||||
return "", fmt.Errorf("unexpected response from hush: %s", string(payload))
|
||||
}
|
||||
plain, err := open(out.Ciphertext, u.Fragment)
|
||||
if err != nil {
|
||||
// The ciphertext is already destroyed at this point, so there is
|
||||
// nothing to retry. Say that plainly.
|
||||
return "", fmt.Errorf("the key in this link does not open this secret, and the ciphertext "+
|
||||
"has now been destroyed (%w). The link was probably altered in transit; ask for a new one", err)
|
||||
}
|
||||
return plain, nil
|
||||
}
|
||||
|
||||
// seal encrypts with AES-256-GCM and returns (ciphertext, key), both base64url.
|
||||
// The nonce is prepended to the ciphertext so the stored blob is self-contained
|
||||
// — byte-for-byte the format templates/base.html produces.
|
||||
func seal(plaintext string) (string, string, error) {
|
||||
key := make([]byte, 32)
|
||||
if _, err := rand.Read(key); err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
block, err := aes.NewCipher(key)
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
gcm, err := cipher.NewGCM(block)
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
nonce := make([]byte, gcm.NonceSize())
|
||||
if _, err := rand.Read(nonce); err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
blob := gcm.Seal(nonce, nonce, []byte(plaintext), nil)
|
||||
return b64.EncodeToString(blob), b64.EncodeToString(key), nil
|
||||
}
|
||||
|
||||
func open(ciphertext, keyStr string) (string, error) {
|
||||
blob, err := b64.DecodeString(ciphertext)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("ciphertext is not base64url: %w", err)
|
||||
}
|
||||
key, err := b64.DecodeString(keyStr)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("key is not base64url: %w", err)
|
||||
}
|
||||
block, err := aes.NewCipher(key)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
gcm, err := cipher.NewGCM(block)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if len(blob) < gcm.NonceSize() {
|
||||
return "", errors.New("ciphertext is too short to contain a nonce")
|
||||
}
|
||||
plain, err := gcm.Open(nil, blob[:gcm.NonceSize()], blob[gcm.NonceSize():], nil)
|
||||
if err != nil {
|
||||
return "", errors.New("authentication failed")
|
||||
}
|
||||
return string(plain), nil
|
||||
}
|
||||
|
||||
// apiMessage pulls the human message out of hush's error envelope, falling back
|
||||
// to the raw body so a proxy's HTML error page is still readable.
|
||||
func apiMessage(payload []byte) string {
|
||||
var e struct {
|
||||
Error struct {
|
||||
Code string `json:"code"`
|
||||
Message string `json:"message"`
|
||||
} `json:"error"`
|
||||
}
|
||||
if err := json.Unmarshal(payload, &e); err == nil && e.Error.Message != "" {
|
||||
return e.Error.Code + ": " + e.Error.Message
|
||||
}
|
||||
s := strings.TrimSpace(string(payload))
|
||||
if len(s) > 300 {
|
||||
s = s[:300] + "…"
|
||||
}
|
||||
return s
|
||||
}
|
||||
195
cmd/hush-mcp/mcp.go
Normal file
195
cmd/hush-mcp/mcp.go
Normal file
@ -0,0 +1,195 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// A minimal, dependency-free MCP server over stdio.
|
||||
//
|
||||
// MCP on stdio is newline-delimited JSON-RPC 2.0. The surface this server needs
|
||||
// is four methods — initialize, notifications/initialized, tools/list,
|
||||
// tools/call — so it is implemented directly rather than pulling in an SDK
|
||||
// whose API churn would be a bigger maintenance surface than the protocol.
|
||||
//
|
||||
// The one rule that matters for a stdio server: stdout carries protocol frames
|
||||
// ONLY. Anything diagnostic goes to stderr, because a stray Println on stdout
|
||||
// corrupts the stream and the host reports an opaque parse failure.
|
||||
|
||||
// protocolVersion is the MCP revision this server implements. The host sends
|
||||
// its own in initialize; the spec has the server answer with the version it
|
||||
// will actually speak rather than echoing the client's.
|
||||
const protocolVersion = "2025-06-18"
|
||||
|
||||
type request struct {
|
||||
JSONRPC string `json:"jsonrpc"`
|
||||
ID json.RawMessage `json:"id,omitempty"`
|
||||
Method string `json:"method"`
|
||||
Params json.RawMessage `json:"params,omitempty"`
|
||||
}
|
||||
|
||||
type response struct {
|
||||
JSONRPC string `json:"jsonrpc"`
|
||||
ID json.RawMessage `json:"id,omitempty"`
|
||||
Result any `json:"result,omitempty"`
|
||||
Error *rpcError `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
type rpcError struct {
|
||||
Code int `json:"code"`
|
||||
Message string `json:"message"`
|
||||
}
|
||||
|
||||
// JSON-RPC 2.0 reserved codes. -32602 is the one that matters here: a bad tool
|
||||
// argument is an invalid-params error, not a transport failure.
|
||||
const (
|
||||
codeInvalidParams = -32602
|
||||
codeMethodMissing = -32601
|
||||
codeInternal = -32603
|
||||
)
|
||||
|
||||
// Tool is one callable tool. Schema is the raw JSON Schema advertised to the
|
||||
// host, which is what makes the arguments self-documenting in the client.
|
||||
type Tool struct {
|
||||
Name string
|
||||
Title string
|
||||
Description string
|
||||
Schema map[string]any
|
||||
// Handler returns the text to show the caller. An error is reported as a
|
||||
// TOOL error (isError on the result) rather than a protocol error, so the
|
||||
// model sees the message and can act on it instead of the call appearing
|
||||
// to have failed at the transport level.
|
||||
Handler func(args json.RawMessage) (string, error)
|
||||
}
|
||||
|
||||
// Server dispatches MCP over a reader/writer pair.
|
||||
type Server struct {
|
||||
name string
|
||||
version string
|
||||
tools []Tool
|
||||
|
||||
mu sync.Mutex // serialises writes: one frame per line, never interleaved
|
||||
out *json.Encoder
|
||||
w io.Writer
|
||||
}
|
||||
|
||||
func NewServer(name, version string, out io.Writer, tools []Tool) *Server {
|
||||
return &Server{name: name, version: version, tools: tools, out: json.NewEncoder(out), w: out}
|
||||
}
|
||||
|
||||
// Serve reads frames until stdin closes, which is how a stdio host signals
|
||||
// shutdown.
|
||||
func (s *Server) Serve(in io.Reader) error {
|
||||
sc := bufio.NewScanner(in)
|
||||
// A tool result can carry a secret link, which is small, but the buffer is
|
||||
// raised so a large argument cannot truncate a frame into a parse error.
|
||||
sc.Buffer(make([]byte, 0, 64*1024), 4*1024*1024)
|
||||
for sc.Scan() {
|
||||
line := sc.Bytes()
|
||||
if len(line) == 0 {
|
||||
continue
|
||||
}
|
||||
var req request
|
||||
if err := json.Unmarshal(line, &req); err != nil {
|
||||
// Malformed frame with no id: nothing to reply to. Report on stderr
|
||||
// and keep the stream alive.
|
||||
fmt.Fprintf(stderr, "hush-mcp: unparseable frame: %v\n", err)
|
||||
continue
|
||||
}
|
||||
s.dispatch(req)
|
||||
}
|
||||
return sc.Err()
|
||||
}
|
||||
|
||||
func (s *Server) dispatch(req request) {
|
||||
// A notification has no id and MUST NOT be answered. Replying to one is the
|
||||
// classic stdio bug: the host sees an unsolicited response and desyncs.
|
||||
isNotification := len(req.ID) == 0
|
||||
|
||||
switch req.Method {
|
||||
case "initialize":
|
||||
s.reply(req.ID, map[string]any{
|
||||
"protocolVersion": protocolVersion,
|
||||
"capabilities": map[string]any{"tools": map[string]any{}},
|
||||
"serverInfo": map[string]any{"name": s.name, "version": s.version},
|
||||
})
|
||||
case "notifications/initialized":
|
||||
// Handshake complete. Nothing to send.
|
||||
case "ping":
|
||||
s.reply(req.ID, map[string]any{})
|
||||
case "tools/list":
|
||||
list := make([]map[string]any, 0, len(s.tools))
|
||||
for _, t := range s.tools {
|
||||
list = append(list, map[string]any{
|
||||
"name": t.Name,
|
||||
"title": t.Title,
|
||||
"description": t.Description,
|
||||
"inputSchema": t.Schema,
|
||||
})
|
||||
}
|
||||
s.reply(req.ID, map[string]any{"tools": list})
|
||||
case "tools/call":
|
||||
s.call(req)
|
||||
default:
|
||||
if !isNotification {
|
||||
s.fail(req.ID, codeMethodMissing, "unsupported method: "+req.Method)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Server) call(req request) {
|
||||
var p struct {
|
||||
Name string `json:"name"`
|
||||
Arguments json.RawMessage `json:"arguments"`
|
||||
}
|
||||
if err := json.Unmarshal(req.Params, &p); err != nil {
|
||||
s.fail(req.ID, codeInvalidParams, "malformed tools/call params")
|
||||
return
|
||||
}
|
||||
for _, t := range s.tools {
|
||||
if t.Name != p.Name {
|
||||
continue
|
||||
}
|
||||
text, err := t.Handler(p.Arguments)
|
||||
if err != nil {
|
||||
// isError:true keeps this a TOOL failure the model can read and
|
||||
// react to, rather than a protocol error that looks like the server
|
||||
// broke.
|
||||
s.reply(req.ID, map[string]any{
|
||||
"content": []map[string]any{{"type": "text", "text": err.Error()}},
|
||||
"isError": true,
|
||||
})
|
||||
return
|
||||
}
|
||||
s.reply(req.ID, map[string]any{
|
||||
"content": []map[string]any{{"type": "text", "text": text}},
|
||||
})
|
||||
return
|
||||
}
|
||||
s.fail(req.ID, codeInvalidParams, "unknown tool: "+p.Name)
|
||||
}
|
||||
|
||||
func (s *Server) reply(id json.RawMessage, result any) {
|
||||
if len(id) == 0 {
|
||||
return
|
||||
}
|
||||
s.write(response{JSONRPC: "2.0", ID: id, Result: result})
|
||||
}
|
||||
|
||||
func (s *Server) fail(id json.RawMessage, code int, msg string) {
|
||||
if len(id) == 0 {
|
||||
return
|
||||
}
|
||||
s.write(response{JSONRPC: "2.0", ID: id, Error: &rpcError{Code: code, Message: msg}})
|
||||
}
|
||||
|
||||
func (s *Server) write(r response) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if err := s.out.Encode(r); err != nil {
|
||||
fmt.Fprintf(stderr, "hush-mcp: write failed: %v\n", err)
|
||||
}
|
||||
}
|
||||
65
cmd/hushd/config.go
Normal file
65
cmd/hushd/config.go
Normal file
@ -0,0 +1,65 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"github.com/orchard9/go-chassis/config"
|
||||
)
|
||||
|
||||
// Config is every knob hushd has. Loaded once at boot; a missing or malformed
|
||||
// required value stops the process rather than being defaulted, so a
|
||||
// misconfigured pod crashloops visibly instead of serving something subtly
|
||||
// wrong.
|
||||
type Config struct {
|
||||
Env string
|
||||
Port int
|
||||
|
||||
// RedisURL is REQUIRED. There is an in-memory store in the codebase for
|
||||
// tests, and requiring this is what makes it impossible to select by
|
||||
// accident in production: a pod with no REDIS_URL does not boot, rather
|
||||
// than booting with a store that loses every secret on restart.
|
||||
RedisURL string
|
||||
|
||||
// RateLimit bounds anonymous creates per client IP. Reveals are not
|
||||
// limited by this: a recipient gets exactly one successful reveal by
|
||||
// construction, so there is nothing to throttle, and throttling would let
|
||||
// one noisy NAT block a colleague's delivery.
|
||||
RateLimitCreates int
|
||||
RateLimitWindow time.Duration
|
||||
|
||||
// TrustedProxyHops is how many Traefik hops sit in front of hushd, used to
|
||||
// pick the real client IP out of X-Forwarded-For. Wrong-high lets a caller
|
||||
// spoof their IP and evade the rate limit; wrong-low rate-limits the
|
||||
// ingress itself and throttles everyone together.
|
||||
TrustedProxyHops int
|
||||
|
||||
// AllowOrigins is the CORS allowlist. Empty is correct for the deployed
|
||||
// service: both pages are same-origin, so no cross-origin caller is
|
||||
// legitimate.
|
||||
AllowOrigins []string
|
||||
|
||||
// RequireAuthToCreate closes anonymous create if the service is abused.
|
||||
// Reveal stays anonymous regardless — the recipient is external and holds
|
||||
// no credential.
|
||||
RequireAuthToCreate bool
|
||||
CreateToken string
|
||||
}
|
||||
|
||||
func loadConfig() (Config, error) {
|
||||
l := config.New()
|
||||
cfg := Config{
|
||||
Env: l.OneOf("APP_ENV", "dev", "dev", "staging", "prod"),
|
||||
Port: l.Port("HUSH_PORT", 18500),
|
||||
RedisURL: l.Required("REDIS_URL"),
|
||||
RateLimitCreates: l.Int("HUSH_RATE_LIMIT_CREATES", 30),
|
||||
RateLimitWindow: l.Duration("HUSH_RATE_LIMIT_WINDOW", 10*time.Minute),
|
||||
TrustedProxyHops: l.Int("HUSH_TRUSTED_PROXY_HOPS", 1),
|
||||
AllowOrigins: l.Strings("HUSH_ALLOW_ORIGINS", nil),
|
||||
RequireAuthToCreate: l.Bool("HUSH_REQUIRE_AUTH", false),
|
||||
CreateToken: l.String("HUSH_CREATE_TOKEN", ""),
|
||||
}
|
||||
if err := l.Err(); err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
return cfg, nil
|
||||
}
|
||||
183
cmd/hushd/handlers.go
Normal file
183
cmd/hushd/handlers.go
Normal file
@ -0,0 +1,183 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/orchard9/go-chassis/chassis"
|
||||
|
||||
"github.com/orchard9/hush/internal/secret"
|
||||
"github.com/orchard9/hush/internal/store"
|
||||
"github.com/orchard9/hush/internal/web"
|
||||
)
|
||||
|
||||
// Server holds the handler dependencies.
|
||||
type Server struct {
|
||||
cfg Config
|
||||
store store.Store
|
||||
pages *web.Pages
|
||||
metrics *Metrics
|
||||
}
|
||||
|
||||
// pageData is the same for every render: the limits, so the browser enforces
|
||||
// what the server enforces. It carries nothing request-specific, which is why
|
||||
// both pages are safely static.
|
||||
func (s *Server) pageData() web.Data {
|
||||
return web.Data{
|
||||
MaxCiphertextBytes: secret.MaxCiphertextBytes,
|
||||
DefaultTTLSeconds: int(secret.DefaultTTL.Seconds()),
|
||||
MinTTLSeconds: int(secret.MinTTL.Seconds()),
|
||||
MaxTTLSeconds: int(secret.MaxTTL.Seconds()),
|
||||
}
|
||||
}
|
||||
|
||||
// handleCreatePage serves GET /.
|
||||
func (s *Server) handleCreatePage(c *chassis.Context) error {
|
||||
return s.pages.Create(c.Writer(), s.pageData())
|
||||
}
|
||||
|
||||
// handleRevealPage serves GET /s/{id}.
|
||||
//
|
||||
// It touches NO storage — not even to check whether the id exists. That is the
|
||||
// design's load-bearing property: Slack, Teams, WhatsApp, iMessage and Outlook
|
||||
// Safe Links all fetch URLs before a human sees them, so any read here would
|
||||
// destroy most secrets in transit. It also means this response cannot leak
|
||||
// whether an id exists.
|
||||
//
|
||||
// The id is not even parsed: a malformed one gets the same page, and the POST
|
||||
// is what answers. Validating here would make this endpoint an id oracle.
|
||||
func (s *Server) handleRevealPage(c *chassis.Context) error {
|
||||
return s.pages.Reveal(c.Writer(), s.pageData())
|
||||
}
|
||||
|
||||
type createRequest struct {
|
||||
Ciphertext string `json:"ciphertext"`
|
||||
TTLSeconds int64 `json:"ttl_seconds"`
|
||||
}
|
||||
|
||||
type createResponse struct {
|
||||
ID string `json:"id"`
|
||||
ExpiresAt string `json:"expires_at"`
|
||||
TTLSeconds int64 `json:"ttl_seconds"`
|
||||
}
|
||||
|
||||
// handleCreate serves POST /api/secrets.
|
||||
//
|
||||
// It accepts ciphertext only. There is deliberately no endpoint taking a
|
||||
// plaintext secret: one would make the server able to read secrets, and then
|
||||
// nobody could tell from a link which guarantee they had.
|
||||
func (s *Server) handleCreate(c *chassis.Context) error {
|
||||
var req createRequest
|
||||
if err := c.Bind(&req); err != nil {
|
||||
s.metrics.Rejected.WithLabelValues(reasonMalformed).Inc()
|
||||
return err
|
||||
}
|
||||
|
||||
if err := secret.ValidateCiphertext(req.Ciphertext); err != nil {
|
||||
reason, code := classifyCiphertextError(err)
|
||||
s.metrics.Rejected.WithLabelValues(reason).Inc()
|
||||
// err carries sizes, never content: ValidateCiphertext never puts the
|
||||
// ciphertext in its message.
|
||||
return &chassis.Error{Status: http.StatusUnprocessableEntity, Code: code, Msg: err.Error()}
|
||||
}
|
||||
|
||||
ttl, err := secret.ResolveTTL(time.Duration(req.TTLSeconds) * time.Second)
|
||||
if err != nil {
|
||||
s.metrics.Rejected.WithLabelValues(reasonTTL).Inc()
|
||||
return &chassis.Error{Status: http.StatusUnprocessableEntity, Code: reasonTTL, Msg: err.Error()}
|
||||
}
|
||||
|
||||
id, err := secret.NewID()
|
||||
if err != nil {
|
||||
// No entropy means no safe id. Refuse rather than mint a guessable one.
|
||||
return chassis.Internal(err)
|
||||
}
|
||||
|
||||
if err := s.store.Put(c.Context(), id, req.Ciphertext, ttl); err != nil {
|
||||
if errors.Is(err, store.ErrIDCollision) {
|
||||
// Impossible at 256 bits, so it means an id-generation bug. Surfaced
|
||||
// rather than retried, because a retry loop would hide it.
|
||||
c.Log().Error("secret.id_collision", "category", "secret",
|
||||
"error_type", "id_collision", "sid", id.LogHandle())
|
||||
return chassis.Internal(err)
|
||||
}
|
||||
return chassis.Internal(err)
|
||||
}
|
||||
|
||||
s.metrics.Created.Inc()
|
||||
s.metrics.Bytes.Observe(float64(len(req.Ciphertext)))
|
||||
// sid, not id: the id is the capability and never reaches a log.
|
||||
c.Log().Info("secret.created", "category", "secret",
|
||||
"sid", id.LogHandle(), "ttl_seconds", int64(ttl.Seconds()),
|
||||
"ciphertext_bytes", len(req.Ciphertext))
|
||||
|
||||
return c.Created(createResponse{
|
||||
ID: id.Value(),
|
||||
ExpiresAt: time.Now().UTC().Add(ttl).Format(time.RFC3339),
|
||||
TTLSeconds: int64(ttl.Seconds()),
|
||||
})
|
||||
}
|
||||
|
||||
type revealResponse struct {
|
||||
Ciphertext string `json:"ciphertext"`
|
||||
}
|
||||
|
||||
// gone is the single response for every unavailable secret: never existed,
|
||||
// already revealed, expired, or evicted early by Redis.
|
||||
//
|
||||
// One response for four causes is deliberate. A caller able to tell "already
|
||||
// revealed" from "never existed" learns that a particular link was real, which
|
||||
// is information about someone else's secret.
|
||||
func gone() *chassis.Error {
|
||||
return &chassis.Error{
|
||||
Status: http.StatusGone,
|
||||
Code: "gone",
|
||||
Msg: "this secret is not available: it was already revealed, expired, or never existed",
|
||||
}
|
||||
}
|
||||
|
||||
// handleReveal serves POST /api/secrets/{id}/reveal — the only destructive
|
||||
// route, and the reason GET is inert.
|
||||
func (s *Server) handleReveal(c *chassis.Context) error {
|
||||
id, err := secret.ParseID(c.PathValue("id"))
|
||||
if err != nil {
|
||||
// A malformed id gets the SAME 410 as a missing one. A 400 here would
|
||||
// separate "not an id we could have minted" from "an id that is gone",
|
||||
// which is a free oracle for anyone probing the id space.
|
||||
s.metrics.Revealed.WithLabelValues("gone").Inc()
|
||||
return gone()
|
||||
}
|
||||
|
||||
ciphertext, err := s.store.Take(c.Context(), id)
|
||||
switch {
|
||||
case errors.Is(err, store.ErrGone):
|
||||
s.metrics.Revealed.WithLabelValues("gone").Inc()
|
||||
c.Log().Info("secret.gone", "category", "secret", "sid", id.LogHandle())
|
||||
return gone()
|
||||
case err != nil:
|
||||
// A store error is NOT reported as gone. The secret may still exist,
|
||||
// and telling the recipient it is gone would send them to rotate a
|
||||
// credential that was never delivered.
|
||||
return chassis.Internal(err)
|
||||
}
|
||||
|
||||
s.metrics.Revealed.WithLabelValues("ok").Inc()
|
||||
c.Log().Info("secret.revealed", "category", "secret", "sid", id.LogHandle())
|
||||
return c.OK(revealResponse{Ciphertext: ciphertext})
|
||||
}
|
||||
|
||||
// classifyCiphertextError maps a validation failure to its metric reason and
|
||||
// API error code, which are the same vocabulary on purpose.
|
||||
func classifyCiphertextError(err error) (reason, code string) {
|
||||
switch {
|
||||
case errors.Is(err, secret.ErrCiphertextTooLarge):
|
||||
return reasonTooLarge, reasonTooLarge
|
||||
case errors.Is(err, secret.ErrCiphertextInvalid):
|
||||
return reasonInvalid, reasonInvalid
|
||||
case errors.Is(err, secret.ErrCiphertextEmpty):
|
||||
return reasonEmpty, reasonEmpty
|
||||
default:
|
||||
return reasonMalformed, reasonMalformed
|
||||
}
|
||||
}
|
||||
294
cmd/hushd/handlers_test.go
Normal file
294
cmd/hushd/handlers_test.go
Normal file
@ -0,0 +1,294 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/orchard9/go-chassis/chassis"
|
||||
|
||||
"github.com/orchard9/hush/internal/secret"
|
||||
"github.com/orchard9/hush/internal/store"
|
||||
"github.com/orchard9/hush/internal/web"
|
||||
)
|
||||
|
||||
// testApp builds the real route table over an in-memory store, so these tests
|
||||
// exercise the actual middleware chain, binder and error envelopes rather than
|
||||
// calling handlers directly.
|
||||
func testApp(t *testing.T) (http.Handler, *store.Memory) {
|
||||
t.Helper()
|
||||
pages, err := web.New()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
mem := store.NewMemory()
|
||||
srv := &Server{
|
||||
cfg: Config{Env: "dev"},
|
||||
store: mem,
|
||||
pages: pages,
|
||||
metrics: newMetrics(),
|
||||
}
|
||||
srv.metrics.Prime()
|
||||
|
||||
log := slog.New(slog.NewJSONHandler(io.Discard, nil))
|
||||
app := chassis.New(chassis.Config{Service: "hush", Env: "dev", MaxBodyBytes: 128 * 1024}, log)
|
||||
app.Get("/", srv.handleCreatePage)
|
||||
app.Get("/s/{id}", srv.handleRevealPage)
|
||||
app.Route("/api", func(r *chassis.Router) {
|
||||
r.Post("/secrets", srv.handleCreate)
|
||||
r.Post("/secrets/{id}/reveal", srv.handleReveal)
|
||||
})
|
||||
return app.Handler(), mem
|
||||
}
|
||||
|
||||
func ciphertext(s string) string {
|
||||
return base64.RawURLEncoding.EncodeToString([]byte(s))
|
||||
}
|
||||
|
||||
func do(t *testing.T, h http.Handler, method, path, body string) (int, map[string]any, string) {
|
||||
t.Helper()
|
||||
var r *http.Request
|
||||
if body == "" {
|
||||
r = httptest.NewRequest(method, path, nil)
|
||||
} else {
|
||||
r = httptest.NewRequest(method, path, strings.NewReader(body))
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, r)
|
||||
raw := w.Body.String()
|
||||
var parsed map[string]any
|
||||
_ = json.Unmarshal([]byte(raw), &parsed)
|
||||
return w.Code, parsed, raw
|
||||
}
|
||||
|
||||
func create(t *testing.T, h http.Handler, ct string) string {
|
||||
t.Helper()
|
||||
code, body, raw := do(t, h, http.MethodPost, "/api/secrets",
|
||||
`{"ciphertext":"`+ct+`","ttl_seconds":3600}`)
|
||||
if code != http.StatusCreated {
|
||||
t.Fatalf("create = %d, want 201: %s", code, raw)
|
||||
}
|
||||
id, ok := body["id"].(string)
|
||||
if !ok || id == "" {
|
||||
t.Fatalf("create response carried no id: %s", raw)
|
||||
}
|
||||
return id
|
||||
}
|
||||
|
||||
func TestCreateThenRevealThenGone(t *testing.T) {
|
||||
h, _ := testApp(t)
|
||||
ct := ciphertext("the-secret-bytes")
|
||||
id := create(t, h, ct)
|
||||
|
||||
code, body, raw := do(t, h, http.MethodPost, "/api/secrets/"+id+"/reveal", "")
|
||||
if code != http.StatusOK {
|
||||
t.Fatalf("first reveal = %d, want 200: %s", code, raw)
|
||||
}
|
||||
if body["ciphertext"] != ct {
|
||||
t.Fatalf("reveal returned %v, want the stored ciphertext", body["ciphertext"])
|
||||
}
|
||||
|
||||
code, _, raw = do(t, h, http.MethodPost, "/api/secrets/"+id+"/reveal", "")
|
||||
if code != http.StatusGone {
|
||||
t.Fatalf("second reveal = %d, want 410: %s", code, raw)
|
||||
}
|
||||
}
|
||||
|
||||
// THE regression test for this whole design. Slack, Teams, WhatsApp, iMessage
|
||||
// and Outlook Safe Links fetch a URL to build a preview before any human sees
|
||||
// it. If GET consumed the secret, most secrets would be destroyed in transit
|
||||
// and the recipient's "already used" would be indistinguishable from a real
|
||||
// interception.
|
||||
//
|
||||
// So: fetching the reveal page any number of times must leave the secret intact.
|
||||
func TestGettingTheRevealPageNeverConsumesTheSecret(t *testing.T) {
|
||||
h, _ := testApp(t)
|
||||
ct := ciphertext("survives-the-previewers")
|
||||
id := create(t, h, ct)
|
||||
|
||||
for i := range 5 {
|
||||
code, _, _ := do(t, h, http.MethodGet, "/s/"+id, "")
|
||||
if code != http.StatusOK {
|
||||
t.Fatalf("GET /s/{id} attempt %d = %d, want 200", i, code)
|
||||
}
|
||||
}
|
||||
|
||||
code, body, raw := do(t, h, http.MethodPost, "/api/secrets/"+id+"/reveal", "")
|
||||
if code != http.StatusOK {
|
||||
t.Fatalf("reveal after 5 page loads = %d, want 200 — a GET consumed the secret: %s", code, raw)
|
||||
}
|
||||
if body["ciphertext"] != ct {
|
||||
t.Fatal("the ciphertext changed across page loads")
|
||||
}
|
||||
}
|
||||
|
||||
// The reveal page must be identical for every id, including ids that were never
|
||||
// minted. If it 404'd on an unknown id it would become an oracle for whether a
|
||||
// link was ever real.
|
||||
func TestTheRevealPageDoesNotDiscloseWhetherASecretExists(t *testing.T) {
|
||||
h, _ := testApp(t)
|
||||
id := create(t, h, ciphertext("real"))
|
||||
|
||||
_, _, real := do(t, h, http.MethodGet, "/s/"+id, "")
|
||||
_, _, fake := do(t, h, http.MethodGet, "/s/"+strings.Repeat("A", 43), "")
|
||||
_, _, junk := do(t, h, http.MethodGet, "/s/not-an-id", "")
|
||||
|
||||
if real != fake || real != junk {
|
||||
t.Fatal("the reveal page differs between a real id, a well-formed unknown id, and junk — " +
|
||||
"it must not disclose existence")
|
||||
}
|
||||
}
|
||||
|
||||
// A malformed id must produce the SAME 410 as a missing one. A 400 here would
|
||||
// separate "not an id this service could mint" from "an id that is gone", which
|
||||
// hands a probe a free classifier.
|
||||
func TestMalformedAndMissingIDsAreIndistinguishable(t *testing.T) {
|
||||
h, _ := testApp(t)
|
||||
|
||||
unknown, err := secret.NewID()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
var bodies []string
|
||||
for _, id := range []string{
|
||||
unknown.Value(), // well-formed, never stored
|
||||
strings.Repeat("A", 43), // well-formed, never minted
|
||||
"short", // wrong length
|
||||
"!!!", // not base64url
|
||||
} {
|
||||
code, body, raw := do(t, h, http.MethodPost, "/api/secrets/"+id+"/reveal", "")
|
||||
if code != http.StatusGone {
|
||||
t.Fatalf("reveal(%q) = %d, want 410: %s", id, code, raw)
|
||||
}
|
||||
errObj, _ := body["error"].(map[string]any)
|
||||
if errObj["code"] != "gone" {
|
||||
t.Fatalf("reveal(%q) code = %v, want \"gone\"", id, errObj["code"])
|
||||
}
|
||||
bodies = append(bodies, errObj["message"].(string))
|
||||
}
|
||||
for i := range bodies {
|
||||
if bodies[i] != bodies[0] {
|
||||
t.Fatalf("the gone message differs between causes (%q vs %q) — they must be identical",
|
||||
bodies[0], bodies[i])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestCreateRejectsWhatItCannotStore(t *testing.T) {
|
||||
h, _ := testApp(t)
|
||||
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
body string
|
||||
code string
|
||||
}{
|
||||
{"empty ciphertext", `{"ciphertext":"","ttl_seconds":3600}`, reasonEmpty},
|
||||
{"not base64url", `{"ciphertext":"!!!!","ttl_seconds":3600}`, reasonInvalid},
|
||||
{"oversized", `{"ciphertext":"` + strings.Repeat("A", secret.MaxCiphertextBytes+1) + `","ttl_seconds":3600}`, reasonTooLarge},
|
||||
{"ttl too short", `{"ciphertext":"` + ciphertext("x") + `","ttl_seconds":60}`, reasonTTL},
|
||||
{"ttl too long", `{"ciphertext":"` + ciphertext("x") + `","ttl_seconds":2592000}`, reasonTTL},
|
||||
{"negative ttl", `{"ciphertext":"` + ciphertext("x") + `","ttl_seconds":-1}`, reasonTTL},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
code, body, raw := do(t, h, http.MethodPost, "/api/secrets", tc.body)
|
||||
if code != http.StatusUnprocessableEntity {
|
||||
t.Fatalf("create = %d, want 422: %s", code, raw)
|
||||
}
|
||||
errObj, _ := body["error"].(map[string]any)
|
||||
if errObj["code"] != tc.code {
|
||||
t.Fatalf("error code = %v, want %q", errObj["code"], tc.code)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestOmittedTTLGetsTheDefault(t *testing.T) {
|
||||
h, _ := testApp(t)
|
||||
code, body, raw := do(t, h, http.MethodPost, "/api/secrets",
|
||||
`{"ciphertext":"`+ciphertext("x")+`"}`)
|
||||
if code != http.StatusCreated {
|
||||
t.Fatalf("create without ttl_seconds = %d, want 201: %s", code, raw)
|
||||
}
|
||||
if got := body["ttl_seconds"].(float64); int(got) != int(secret.DefaultTTL.Seconds()) {
|
||||
t.Fatalf("ttl_seconds = %v, want the default %v", got, secret.DefaultTTL.Seconds())
|
||||
}
|
||||
}
|
||||
|
||||
// There must be no way to hand hush a plaintext secret. If a field like
|
||||
// "secret" or "plaintext" were ever accepted, the server would become able to
|
||||
// read secrets and the guarantee in the README would quietly become a promise
|
||||
// about our conduct instead of a property of the design.
|
||||
func TestThereIsNoPlaintextIntakeField(t *testing.T) {
|
||||
h, _ := testApp(t)
|
||||
for _, body := range []string{
|
||||
`{"secret":"hunter2","ttl_seconds":3600}`,
|
||||
`{"plaintext":"hunter2","ttl_seconds":3600}`,
|
||||
`{"ciphertext":"` + ciphertext("x") + `","plaintext":"hunter2"}`,
|
||||
} {
|
||||
code, _, raw := do(t, h, http.MethodPost, "/api/secrets", body)
|
||||
if code == http.StatusCreated {
|
||||
t.Fatalf("create accepted a plaintext field: %s -> %s", body, raw)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The client-side crypto IS the product. If either page stops shipping it, the
|
||||
// service silently becomes a plaintext store or a broken reader, and every
|
||||
// other test here would still pass.
|
||||
func TestPagesShipTheClientSideCrypto(t *testing.T) {
|
||||
h, _ := testApp(t)
|
||||
|
||||
_, _, create := do(t, h, http.MethodGet, "/", "")
|
||||
for _, want := range []string{
|
||||
"AES-GCM", // the cipher
|
||||
"crypto.subtle", // in the browser, not on the server
|
||||
"generateKey", // the key is minted client-side
|
||||
`"#" + sealed.key`, // and leaves only in the fragment
|
||||
"/api/secrets", // ciphertext is what gets posted
|
||||
} {
|
||||
if !strings.Contains(create, want) {
|
||||
t.Fatalf("the create page no longer contains %q — check it still encrypts client-side", want)
|
||||
}
|
||||
}
|
||||
// The create page must never post a plaintext field.
|
||||
for _, forbidden := range []string{`"plaintext"`, `"secret":`} {
|
||||
if strings.Contains(create, forbidden) {
|
||||
t.Fatalf("the create page mentions %s, which suggests it sends plaintext", forbidden)
|
||||
}
|
||||
}
|
||||
|
||||
_, _, reveal := do(t, h, http.MethodGet, "/s/"+strings.Repeat("A", 43), "")
|
||||
for _, want := range []string{
|
||||
"location.hash", // the key comes from the fragment
|
||||
"crypto.subtle", // decryption is client-side
|
||||
"/reveal", // and only POST consumes
|
||||
"history.replaceState", // the key is dropped from the address bar after
|
||||
} {
|
||||
if !strings.Contains(reveal, want) {
|
||||
t.Fatalf("the reveal page no longer contains %q", want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestPagesAreNotCacheable(t *testing.T) {
|
||||
h, _ := testApp(t)
|
||||
for _, path := range []string{"/", "/s/" + strings.Repeat("A", 43)} {
|
||||
r := httptest.NewRequest(http.MethodGet, path, nil)
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, r)
|
||||
// A cached reveal page in a shared proxy is a copy of a one-time URL.
|
||||
if cc := w.Header().Get("Cache-Control"); !strings.Contains(cc, "no-store") {
|
||||
t.Fatalf("%s Cache-Control = %q, want no-store", path, cc)
|
||||
}
|
||||
if ref := w.Header().Get("Referrer-Policy"); ref != "no-referrer" {
|
||||
t.Fatalf("%s Referrer-Policy = %q, want no-referrer — a click could otherwise leak the URL", path, ref)
|
||||
}
|
||||
}
|
||||
}
|
||||
166
cmd/hushd/main.go
Normal file
166
cmd/hushd/main.go
Normal file
@ -0,0 +1,166 @@
|
||||
// Command hushd serves hush: one-time secret links.
|
||||
//
|
||||
// It stores ciphertext it cannot read. The encryption key is generated in the
|
||||
// browser and travels only in the URL fragment, which browsers never transmit.
|
||||
// See docs/ARCHITECTURE.md.
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/orchard9/go-chassis/chassis"
|
||||
"github.com/orchard9/go-chassis/logging"
|
||||
|
||||
"github.com/orchard9/hush/internal/secret"
|
||||
"github.com/orchard9/hush/internal/store"
|
||||
"github.com/orchard9/hush/internal/web"
|
||||
)
|
||||
|
||||
// service is the log corpus's `service` value and the metrics prefix. It is a
|
||||
// member of a closed enum shared with every other orchard9 emitter — the
|
||||
// cluster's Vector sink indexes `service` as a stream field, so a new value is
|
||||
// a coordinated decision, not a free string.
|
||||
const service = "hush"
|
||||
|
||||
func main() {
|
||||
if err := run(); err != nil {
|
||||
// Boot failures go to stderr as well as the log: if logging itself is
|
||||
// what failed, the process must still say why before exiting.
|
||||
fmt.Fprintf(os.Stderr, "hushd: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func run() error {
|
||||
cfg, cfgErr := loadConfig()
|
||||
|
||||
// The logger is built before the config error is returned, so a
|
||||
// misconfiguration is REPORTED through the same corpus as everything else
|
||||
// rather than dying silently. cfg.Env is the zero value on failure, which
|
||||
// logging.Env normalises.
|
||||
log := logging.New(logging.Config{Service: service, Env: logging.Env(cfg.Env)})
|
||||
logging.SetFallback(log)
|
||||
|
||||
if cfgErr != nil {
|
||||
// critical + category boot is the fail-closed contract: this is a
|
||||
// refusal to serve, which is the only thing critical is for.
|
||||
logging.Critical(context.Background(), log, "boot.config_invalid",
|
||||
"category", "boot", "error_type", "config_invalid", "error_msg", cfgErr.Error())
|
||||
return cfgErr
|
||||
}
|
||||
|
||||
pages, err := web.New()
|
||||
if err != nil {
|
||||
logging.Critical(context.Background(), log, "boot.templates_invalid",
|
||||
"category", "boot", "error_type", "template_invalid", "error_msg", err.Error())
|
||||
return err
|
||||
}
|
||||
|
||||
rdb, err := store.NewRedis(cfg.RedisURL)
|
||||
if err != nil {
|
||||
// The URL is malformed — a config defect, not an outage. Refuse to boot
|
||||
// rather than serve a store that can never work.
|
||||
logging.Critical(context.Background(), log, "boot.store_invalid",
|
||||
"category", "boot", "error_type", "store_invalid", "error_msg", err.Error())
|
||||
return err
|
||||
}
|
||||
defer func() { _ = rdb.Close() }()
|
||||
|
||||
metrics := newMetrics()
|
||||
metrics.Prime()
|
||||
|
||||
srv := &Server{cfg: cfg, store: rdb, pages: pages, metrics: metrics}
|
||||
|
||||
app := chassis.New(chassis.Config{
|
||||
Service: service,
|
||||
Env: cfg.Env,
|
||||
Addr: fmt.Sprintf(":%d", cfg.Port),
|
||||
// 128 KiB caps the body at the edge so an oversized upload is refused
|
||||
// before a handler allocates it. The 64 KiB ciphertext cap is enforced
|
||||
// separately in the handler; this is the outer bound including JSON
|
||||
// framing.
|
||||
MaxBodyBytes: 128 * 1024,
|
||||
AllowOrigins: cfg.AllowOrigins,
|
||||
Collectors: metrics.Collectors(),
|
||||
}, log)
|
||||
|
||||
// Readiness doubles as the store_up gauge, so the alert on Redis
|
||||
// reachability reads the same probe Kubernetes uses to route traffic —
|
||||
// rather than a second, separately-drifting health notion.
|
||||
app.Health("redis", func(ctx context.Context) error {
|
||||
err := rdb.Ping(ctx)
|
||||
if err != nil {
|
||||
metrics.StoreUp.Set(0)
|
||||
return err
|
||||
}
|
||||
metrics.StoreUp.Set(1)
|
||||
return nil
|
||||
})
|
||||
|
||||
// Pages: no storage access, no rate limit. A link previewer hitting either
|
||||
// of these must be free and harmless.
|
||||
app.Get("/", srv.handleCreatePage)
|
||||
app.Get("/s/{id}", srv.handleRevealPage)
|
||||
|
||||
limiter := &redisLimiter{store: rdb, cfg: cfg, metrics: metrics}
|
||||
app.Route("/api", func(r *chassis.Router) {
|
||||
// Rate limit applies to create only. A reveal succeeds at most once per
|
||||
// secret by construction, so there is nothing to throttle, and
|
||||
// throttling would let one busy NAT block a colleague's delivery.
|
||||
createMW := []chassis.Middleware{chassis.RateLimit(limiter, chassis.RateKey(cfg.TrustedProxyHops))}
|
||||
if cfg.RequireAuthToCreate {
|
||||
// The escape hatch for abuse. Reveal stays anonymous either way:
|
||||
// the recipient is external and holds no credential.
|
||||
createMW = append(createMW, chassis.RequireAuth(chassis.NewStaticToken(cfg.CreateToken)))
|
||||
}
|
||||
r.Post("/secrets", srv.handleCreate, createMW...)
|
||||
r.Post("/secrets/{id}/reveal", srv.handleReveal)
|
||||
})
|
||||
|
||||
log.Info("boot.ready", "category", "boot",
|
||||
"port", cfg.Port,
|
||||
"rate_limit_creates", cfg.RateLimitCreates,
|
||||
"rate_limit_window_seconds", int64(cfg.RateLimitWindow.Seconds()),
|
||||
"max_ciphertext_bytes", secret.MaxCiphertextBytes,
|
||||
"default_ttl_seconds", int64(secret.DefaultTTL.Seconds()),
|
||||
"require_auth_to_create", cfg.RequireAuthToCreate)
|
||||
|
||||
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
|
||||
defer stop()
|
||||
if err := app.Run(ctx); err != nil && !errors.Is(err, http.ErrServerClosed) {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// redisLimiter adapts the Redis fixed-window counter to chassis.RateLimiter and
|
||||
// counts refusals, which is the abuse signal the alert rules watch.
|
||||
type redisLimiter struct {
|
||||
store *store.Redis
|
||||
cfg Config
|
||||
metrics *Metrics
|
||||
}
|
||||
|
||||
func (l *redisLimiter) Allow(ctx context.Context, key string) (bool, time.Duration, error) {
|
||||
ok, retry, err := l.store.AllowN(ctx, key, l.cfg.RateLimitCreates, l.cfg.RateLimitWindow)
|
||||
if err != nil {
|
||||
// Redis is unreachable. FAIL OPEN on the limiter specifically: the
|
||||
// alternative is that a Redis blip turns the rate limiter into a total
|
||||
// outage of a service whose whole job is delivering credentials during
|
||||
// incidents. The store call immediately after will fail anyway if Redis
|
||||
// is really down, so this cannot silently accept a secret it cannot
|
||||
// store — it just refuses at the right layer, with the right error.
|
||||
return true, 0, nil
|
||||
}
|
||||
if !ok {
|
||||
l.metrics.Limited.Inc()
|
||||
}
|
||||
return ok, retry, nil
|
||||
}
|
||||
84
cmd/hushd/metrics.go
Normal file
84
cmd/hushd/metrics.go
Normal file
@ -0,0 +1,84 @@
|
||||
package main
|
||||
|
||||
import "github.com/prometheus/client_golang/prometheus"
|
||||
|
||||
// Metrics are hush's own series, registered on the chassis's private registry
|
||||
// via chassis.Config.Collectors so /metrics is one scrape.
|
||||
//
|
||||
// Cardinality rule applied throughout: NO label ever carries a secret id, a
|
||||
// client IP, or a URL path. A metric label lands in the time series index and
|
||||
// stays there, so a per-secret label would be both a capability leak and an
|
||||
// unbounded index. Every label below is a closed enum.
|
||||
type Metrics struct {
|
||||
Created prometheus.Counter
|
||||
Revealed *prometheus.CounterVec // result: ok | gone
|
||||
Rejected *prometheus.CounterVec // reason: closed enum, see rejection reasons
|
||||
Bytes prometheus.Histogram
|
||||
StoreUp prometheus.Gauge
|
||||
Limited prometheus.Counter
|
||||
}
|
||||
|
||||
// Rejection reasons. A closed set, mirrored by the API error codes, so
|
||||
// `hush_secrets_rejected_total{reason="ciphertext_too_large"}` and the 422 a
|
||||
// caller saw are the same vocabulary.
|
||||
const (
|
||||
reasonTooLarge = "ciphertext_too_large"
|
||||
reasonInvalid = "ciphertext_invalid"
|
||||
reasonEmpty = "ciphertext_empty"
|
||||
reasonTTL = "ttl_out_of_range"
|
||||
reasonMalformed = "malformed_request"
|
||||
)
|
||||
|
||||
func newMetrics() *Metrics {
|
||||
return &Metrics{
|
||||
Created: prometheus.NewCounter(prometheus.CounterOpts{
|
||||
Name: "hush_secrets_created_total",
|
||||
Help: "Secrets accepted and stored.",
|
||||
}),
|
||||
Revealed: prometheus.NewCounterVec(prometheus.CounterOpts{
|
||||
Name: "hush_secrets_revealed_total",
|
||||
Help: "Reveal attempts by outcome. `gone` covers already-revealed, expired, never-existed and evicted, which the service does not distinguish.",
|
||||
}, []string{"result"}),
|
||||
Rejected: prometheus.NewCounterVec(prometheus.CounterOpts{
|
||||
Name: "hush_secrets_rejected_total",
|
||||
Help: "Create attempts refused by validation, by reason.",
|
||||
}, []string{"reason"}),
|
||||
Bytes: prometheus.NewHistogram(prometheus.HistogramOpts{
|
||||
Name: "hush_secret_ciphertext_bytes",
|
||||
Help: "Size of stored ciphertext.",
|
||||
// Hand-picked rather than DefBuckets (which tops out at 10): this
|
||||
// measures bytes up to a 64 KiB cap, so the buckets straddle
|
||||
// credential-sized (hundreds of bytes) through the limit.
|
||||
Buckets: []float64{256, 1024, 4096, 16384, 65536},
|
||||
}),
|
||||
StoreUp: prometheus.NewGauge(prometheus.GaugeOpts{
|
||||
Name: "hush_store_up",
|
||||
Help: "1 when the last readiness probe reached Redis, 0 otherwise.",
|
||||
}),
|
||||
Limited: prometheus.NewCounter(prometheus.CounterOpts{
|
||||
Name: "hush_rate_limited_total",
|
||||
Help: "Requests refused by the create rate limit. A sustained rise is the abuse signal.",
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
// Collectors is what gets handed to chassis.Config.Collectors. The chassis owns
|
||||
// a private registry and exposes no accessor, so this slice is the only way
|
||||
// hush's series reach /metrics.
|
||||
func (m *Metrics) Collectors() []prometheus.Collector {
|
||||
return []prometheus.Collector{m.Created, m.Revealed, m.Rejected, m.Bytes, m.StoreUp, m.Limited}
|
||||
}
|
||||
|
||||
// Prime initialises the label combinations that alerting queries reference.
|
||||
//
|
||||
// Without this, `rate(hush_secrets_revealed_total{result="gone"}[15m])` returns
|
||||
// no data until the first `gone` ever happens, and an alert written against a
|
||||
// missing series is an alert that cannot fire. Priming makes the series exist
|
||||
// at zero from boot.
|
||||
func (m *Metrics) Prime() {
|
||||
m.Revealed.WithLabelValues("ok")
|
||||
m.Revealed.WithLabelValues("gone")
|
||||
for _, r := range []string{reasonTooLarge, reasonInvalid, reasonEmpty, reasonTTL, reasonMalformed} {
|
||||
m.Rejected.WithLabelValues(r)
|
||||
}
|
||||
}
|
||||
249
deployments/k8s/hush.yaml
Normal file
249
deployments/k8s/hush.yaml
Normal file
@ -0,0 +1,249 @@
|
||||
# hush on the orchard9 k3s cluster. Apply this BEFORE the first push, because
|
||||
# the pipeline's deploy step runs `kubectl set image` and needs a Deployment to
|
||||
# set it on.
|
||||
#
|
||||
# KUBECONFIG=~/.kube/orchard9-k3sf.yaml kubectl apply -f deployments/k8s/hush.yaml
|
||||
#
|
||||
# Everything hush needs is here: the credential, the workload, the Service, the
|
||||
# network boundary and the public route.
|
||||
---
|
||||
# The Redis credential. hush connects as its OWN Redis ACL user, scoped to
|
||||
# `~hush:*` with a minimal command set (+ping +set +getdel +incr +pexpire
|
||||
# +select), so a bug in hush cannot read or write another tenant's keys and a
|
||||
# compromise of hush cannot enumerate the keyspace.
|
||||
#
|
||||
# `+getdel` is the one that needs saying out loud: no other service's ACL user
|
||||
# has it, because no other service needs an atomic read-and-destroy. Omitting it
|
||||
# makes every reveal fail NOPERM while creates keep working — a service that
|
||||
# accepts secrets and cannot deliver them.
|
||||
apiVersion: external-secrets.io/v1
|
||||
kind: ExternalSecret
|
||||
metadata:
|
||||
name: hush-credentials
|
||||
namespace: projects
|
||||
spec:
|
||||
refreshInterval: 1h
|
||||
secretStoreRef:
|
||||
name: gcp-secret-manager
|
||||
kind: ClusterSecretStore
|
||||
target:
|
||||
name: hush-credentials
|
||||
creationPolicy: Owner
|
||||
dataFrom:
|
||||
- extract:
|
||||
key: k3sf-hush-credentials
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: hush
|
||||
namespace: projects
|
||||
labels:
|
||||
app: hush
|
||||
spec:
|
||||
# One replica is sufficient and not a bottleneck: every request is a single
|
||||
# Redis round trip and all state lives in Redis, so this scales horizontally
|
||||
# whenever it needs to. The rate limiter is already Redis-backed for exactly
|
||||
# that reason.
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: hush
|
||||
strategy:
|
||||
rollingUpdate:
|
||||
maxUnavailable: 0
|
||||
maxSurge: 1
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: hush
|
||||
annotations:
|
||||
# Metrics scraping. All three are required, and the port must EQUAL a
|
||||
# declared containerPort as a string — vmagent's relabel config uses
|
||||
# `keepequal`, so a mismatch silently drops the target with no error
|
||||
# and no up=0 to alert on.
|
||||
prometheus.io/scrape: "true"
|
||||
prometheus.io/port: "18500"
|
||||
prometheus.io/path: /metrics
|
||||
spec:
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 65532
|
||||
runAsGroup: 65532
|
||||
fsGroup: 65532
|
||||
seccompProfile:
|
||||
type: RuntimeDefault
|
||||
# The chassis drains in two phases — flip readiness to 503, wait 5s for
|
||||
# the load balancer to notice, then shut down with a 25s deadline. That is
|
||||
# 30s, so the grace period must exceed it or the kubelet SIGKILLs mid-drain
|
||||
# and in-flight reveals are lost.
|
||||
terminationGracePeriodSeconds: 45
|
||||
containers:
|
||||
- name: hushd
|
||||
# `bootstrap` is a placeholder, replaced by CI's `kubectl set image`
|
||||
# with the commit-SHA tag on the first push. It is deliberately NOT
|
||||
# `:latest`: the cluster's stable-controller-images admission policy
|
||||
# refuses latest/main/master/dev/edge/canary/nightly/snapshot, because
|
||||
# a floating tag cannot pin a rollback. Until the first pipeline run
|
||||
# this image does not exist and the pod sits in ImagePullBackOff,
|
||||
# which is the expected bootstrap state.
|
||||
image: registry.threesix.ai/hush/api:bootstrap
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 18500
|
||||
env:
|
||||
- name: APP_ENV
|
||||
value: prod
|
||||
- name: HUSH_PORT
|
||||
value: "18500"
|
||||
- name: REDIS_URL
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hush-credentials
|
||||
key: REDIS_URL
|
||||
# One Traefik hop sits in front, so the client IP is the last entry
|
||||
# of X-Forwarded-For. Too high lets a caller spoof their IP past the
|
||||
# rate limit; too low rate-limits the ingress itself and throttles
|
||||
# every user together.
|
||||
- name: HUSH_TRUSTED_PROXY_HOPS
|
||||
value: "1"
|
||||
- name: HUSH_RATE_LIMIT_CREATES
|
||||
value: "30"
|
||||
- name: HUSH_RATE_LIMIT_WINDOW
|
||||
value: 10m
|
||||
securityContext:
|
||||
allowPrivilegeEscalation: false
|
||||
readOnlyRootFilesystem: true
|
||||
capabilities:
|
||||
drop: ["ALL"]
|
||||
# Liveness stays 200 during the drain by design, so this restarts the
|
||||
# pod only when the process is genuinely wedged — never merely because
|
||||
# it is shutting down or because Redis is down.
|
||||
livenessProbe:
|
||||
httpGet: { path: /healthz, port: http }
|
||||
initialDelaySeconds: 3
|
||||
periodSeconds: 10
|
||||
timeoutSeconds: 2
|
||||
failureThreshold: 3
|
||||
# Readiness pings Redis. A Redis outage takes hush out of the Service
|
||||
# rather than leaving it to serve 500s from a pod the load balancer
|
||||
# still trusts.
|
||||
readinessProbe:
|
||||
httpGet: { path: /readyz, port: http }
|
||||
initialDelaySeconds: 2
|
||||
periodSeconds: 5
|
||||
timeoutSeconds: 3
|
||||
failureThreshold: 2
|
||||
resources:
|
||||
requests: { memory: 32Mi, cpu: 20m }
|
||||
# 128Mi is generous for a service whose largest allocation is a
|
||||
# 64 KiB ciphertext, and leaves headroom for Go's GC pacing.
|
||||
limits: { memory: 128Mi, cpu: 500m }
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: hush
|
||||
namespace: projects
|
||||
labels:
|
||||
app: hush
|
||||
spec:
|
||||
selector:
|
||||
app: hush
|
||||
ports:
|
||||
- name: http
|
||||
port: 80
|
||||
targetPort: http
|
||||
---
|
||||
# Default-deny both directions, then open exactly what hush needs. Written as
|
||||
# one policy because the ingress and egress rules are a single statement about
|
||||
# this pod: Traefik in, DNS and Redis out, nothing else.
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: hush
|
||||
namespace: projects
|
||||
spec:
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app: hush
|
||||
policyTypes: [Ingress, Egress]
|
||||
ingress:
|
||||
# Public traffic, via Traefik only.
|
||||
- from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: kube-system
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: traefik
|
||||
ports:
|
||||
- { protocol: TCP, port: 18500 }
|
||||
# vmagent scrapes /metrics on the pod IP directly. Without this rule the
|
||||
# target is discovered and every scrape is connection-refused, which shows
|
||||
# up as up=0 and fires ScrapeTargetDown rather than as a policy error.
|
||||
- from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: observability
|
||||
ports:
|
||||
- { protocol: TCP, port: 18500 }
|
||||
egress:
|
||||
# DNS. Redis is a headless Service, so its name resolves to a pod IP that
|
||||
# changes when Redis restarts — this must be re-resolvable, not cached.
|
||||
- to:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: kube-system
|
||||
ports:
|
||||
- { protocol: UDP, port: 53 }
|
||||
- { protocol: TCP, port: 53 }
|
||||
# Redis, and nothing else. hush makes no other outbound connection: it does
|
||||
# not fetch, does not call a provider, and does not ship its own logs
|
||||
# (Vector reads stdout off the node filesystem).
|
||||
- to:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: databases
|
||||
ports:
|
||||
- { protocol: TCP, port: 6379 }
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: hush
|
||||
namespace: projects
|
||||
annotations:
|
||||
cert-manager.io/cluster-issuer: letsencrypt-prod
|
||||
spec:
|
||||
tls:
|
||||
- hosts: [hush.threesix.ai]
|
||||
secretName: hush-tls
|
||||
rules:
|
||||
- host: hush.threesix.ai
|
||||
http:
|
||||
paths:
|
||||
# The paths are enumerated deliberately, and `/` is Exact rather than
|
||||
# Prefix. A Prefix `/` would route EVERYTHING, publishing /metrics,
|
||||
# /healthz and /readyz to the internet. /metrics leaks how many
|
||||
# secrets are created and when; the others are just noise. Enumerating
|
||||
# instead means Traefik 404s them at the edge and there is no
|
||||
# basic-auth middleware to maintain and get wrong.
|
||||
- path: /
|
||||
pathType: Exact
|
||||
backend:
|
||||
service:
|
||||
name: hush
|
||||
port: { name: http }
|
||||
- path: /s/
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: hush
|
||||
port: { name: http }
|
||||
- path: /api/
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: hush
|
||||
port: { name: http }
|
||||
180
docs/ARCHITECTURE.md
Normal file
180
docs/ARCHITECTURE.md
Normal file
@ -0,0 +1,180 @@
|
||||
# Architecture
|
||||
|
||||
One Go binary, one Redis key per secret, no database. The interesting parts are
|
||||
all about *where the key sits* and *what destroys the ciphertext*.
|
||||
|
||||
## Components
|
||||
|
||||
```
|
||||
browser ──► Traefik ──► hushd (projects ns, 1 replica) ──► Redis (databases ns, db 5)
|
||||
│
|
||||
├─ stdout JSON ──► Vector (DaemonSet) ──► VictoriaLogs
|
||||
└─ /metrics ─────► vmagent ──► vmsingle ──► vmalert ──► Alertmanager
|
||||
```
|
||||
|
||||
`hushd` holds no durable state. Redis holds every secret and nothing else.
|
||||
|
||||
## The zero-knowledge split
|
||||
|
||||
```
|
||||
create: plaintext ──[AES-256-GCM in the browser]──► ciphertext ──► POST /api/secrets
|
||||
key ──────────────────────────────────────► URL fragment, never sent
|
||||
|
||||
reveal: POST /api/secrets/{id}/reveal ──► ciphertext ──[decrypt in browser]──► plaintext
|
||||
key read from location.hash
|
||||
```
|
||||
|
||||
The fragment is the whole trick. Per RFC 3986 §3.5 the fragment is a client-side
|
||||
construct: browsers do not put it in the request line, so it never reaches
|
||||
Traefik, hushd, Redis, an access log, or a proxy. hushd receives a 256-bit AES-GCM
|
||||
ciphertext with a prepended 96-bit nonce and has no key material at any point.
|
||||
|
||||
Consequences worth stating plainly:
|
||||
|
||||
- A Redis dump is worthless. A hushd core dump is worthless. Our own operators
|
||||
cannot read a secret, and neither can anyone who compromises the service.
|
||||
- **A URL in someone's browser history contains the key.** The fragment is not
|
||||
transmitted, but it *is* stored locally. This is the residual exposure and it
|
||||
is why TTLs are short.
|
||||
- There is deliberately no server-side-encryption fallback mode. A second mode
|
||||
where the server sees plaintext would mean nobody could tell, from a link,
|
||||
which guarantee they had.
|
||||
|
||||
## Why GET never touches storage
|
||||
|
||||
`GET /s/{id}` renders a static page and makes zero calls to Redis. It does not
|
||||
even check whether the id exists.
|
||||
|
||||
That is not laziness — it is the only way to be correct in the presence of link
|
||||
previewers. Slack, Teams, WhatsApp, iMessage and Outlook Safe Links fetch URLs
|
||||
before a human sees them. Any design that destroys on `GET` destroys most secrets
|
||||
in transit. Bot user-agent detection is a losing arms race; removing the
|
||||
side effect from `GET` is not.
|
||||
|
||||
A secondary benefit: because `GET` does not look the id up, the reveal page cannot
|
||||
leak whether an id exists. Existence is only ever answered by a `POST`, and that
|
||||
answer is identical for missing, revealed and expired.
|
||||
|
||||
## Storage and destruction
|
||||
|
||||
One key per secret:
|
||||
|
||||
```
|
||||
key hush:s:<id> id = 256 bits from crypto/rand, base64url (43 chars)
|
||||
value <ciphertext> opaque bytes, ≤ 64 KiB
|
||||
write SET key val EX <ttl> NX
|
||||
read GETDEL key
|
||||
```
|
||||
|
||||
`GETDEL` (Redis 6.2+; the cluster runs 7.4.8) is atomic, which is the reason it
|
||||
is used instead of `GET` followed by `DEL`. Two people opening the same link
|
||||
simultaneously cannot both receive the plaintext — exactly one `GETDEL` returns
|
||||
the value and the other returns nil. A `GET`+`DEL` pair has a window between the
|
||||
two commands where both callers succeed, and for a one-time secret that window is
|
||||
the entire product.
|
||||
|
||||
`NX` on write means an id collision never overwrites an existing secret. At 256
|
||||
bits of entropy a collision will not happen; the flag costs nothing and turns a
|
||||
theoretical silent overwrite into a visible error.
|
||||
|
||||
TTL is Redis-native, so expiry needs no sweeper, no cron and nothing to wedge.
|
||||
|
||||
### Eviction is an availability risk, not a confidentiality one
|
||||
|
||||
The shared Redis runs `maxmemory-policy allkeys-lru` with `maxmemory 256MiB`.
|
||||
Under memory pressure Redis may evict a hush key **before** its TTL fires. That
|
||||
means a secret can become unavailable early.
|
||||
|
||||
It cannot become *more* available: eviction only ever deletes. So the failure mode
|
||||
is "your recipient has to ask you again", never "the secret outlived its TTL" and
|
||||
never "someone read it twice". For a secret courier that is the correct direction
|
||||
to fail, and it is why `410 gone` deliberately does not distinguish causes — the
|
||||
user-visible contract is identical either way.
|
||||
|
||||
Operationally this is watched via `HushRedisUnreachable` and the Redis memory
|
||||
alerts, not by trying to tell eviction and reveal apart. See
|
||||
[OPERATIONS.md](OPERATIONS.md).
|
||||
|
||||
## Identifiers and what gets logged
|
||||
|
||||
The id **is** the capability. Anyone holding it can reveal the secret, so it is
|
||||
treated like a bearer token:
|
||||
|
||||
- Never logged. Not at debug, not in an error, not in a panic.
|
||||
- The log correlation handle is `sid = sha256(id)[:12]` — enough to follow one
|
||||
secret's create → reveal → gone across a corpus, useless for revealing it.
|
||||
- Never in a metric label (that would put it in the time series index forever).
|
||||
|
||||
`internal/secret.ID.LogHandle()` is the only way to get a loggable form, and the
|
||||
`ID` type does not implement `String()` or `MarshalText()`, so it cannot be
|
||||
accidentally interpolated into a log line or JSON body. That is enforced by
|
||||
`internal/secret/id_test.go`.
|
||||
|
||||
The chassis logger additionally redacts any field *named* `secret`, `token`,
|
||||
`password`, `api_key`, `authorization` and friends. Field names here avoid those
|
||||
words entirely (`ciphertext`, `sid`, `ttl_seconds`) so nothing depends on that
|
||||
backstop.
|
||||
|
||||
## Request path
|
||||
|
||||
```
|
||||
GET / create page (static HTML+JS, no storage access)
|
||||
GET /s/{id} reveal page (static HTML+JS, no storage access)
|
||||
POST /api/secrets store ciphertext rate limited
|
||||
POST /api/secrets/{id}/reveal GETDEL, destroy, return once rate limited
|
||||
GET /healthz liveness — 200 while draining
|
||||
GET /readyz readiness — Redis PING, 503 while draining
|
||||
GET /metrics Prometheus
|
||||
```
|
||||
|
||||
Built on `github.com/orchard9/go-chassis`, which supplies routing, request ids,
|
||||
the panic recovery envelope, RED metrics, secure headers, the two-phase drain,
|
||||
and `/healthz`, `/readyz`, `/metrics`. hush contributes handlers, a store, a
|
||||
rate limiter and templates — not a framework.
|
||||
|
||||
The public Ingress routes `/` (exact), `/s/` and `/api/` only. `/metrics`,
|
||||
`/healthz` and `/readyz` share the port but are unreachable from the internet;
|
||||
vmagent scrapes the pod IP directly. This is why there is no metrics basic-auth
|
||||
middleware to maintain.
|
||||
|
||||
## Abuse posture
|
||||
|
||||
Create is anonymous by design, which makes the service a free blob host and a
|
||||
phishing kit borrowing a `threesix.ai` name. Mitigations, all cheap:
|
||||
|
||||
| Control | Value |
|
||||
|---|---|
|
||||
| Ciphertext cap | 64 KiB, enforced before Redis |
|
||||
| Request body cap | 128 KiB, enforced by the chassis at the edge |
|
||||
| TTL clamp | 5m … 7d, out-of-range is a 422, not a silent clamp |
|
||||
| Rate limit | 30 creates / 10 min / IP, Redis fixed-window |
|
||||
| Id entropy | 256 bits — enumeration is not a threat model |
|
||||
| No listing route | there is no way to ask "what secrets exist" |
|
||||
| Identical `gone` | missing, revealed and expired are one response |
|
||||
|
||||
If it is ever abused, `HUSH_REQUIRE_AUTH=true` puts create behind the chassis
|
||||
authenticator while leaving reveal anonymous — the asymmetry the design assumes.
|
||||
Reveal must stay anonymous: the recipient is external and has no credential.
|
||||
|
||||
## Failure modes
|
||||
|
||||
| Failure | Behaviour |
|
||||
|---|---|
|
||||
| Redis down | `/readyz` 503, pod leaves the Service, creates and reveals 503. No secret is lost that was already written. |
|
||||
| Redis evicts a key early | That link returns `410 gone`. Sender must re-send. |
|
||||
| hushd restarts | Nothing lost; all state is in Redis. |
|
||||
| Two simultaneous reveals | Exactly one wins, atomically. |
|
||||
| Body over 128 KiB | 413 at the edge, never reaches a handler. |
|
||||
| Ciphertext over 64 KiB | 422 `ciphertext_too_large`. |
|
||||
| Malformed base64 | 422 `ciphertext_invalid`. hushd validates the encoding but cannot validate the plaintext. |
|
||||
| Clock skew | TTL is Redis-relative, so skew between hushd and the browser cannot extend a secret's life. |
|
||||
|
||||
## What is deliberately absent
|
||||
|
||||
Accounts. Passphrases on top of the link. File uploads. Multi-read links. An
|
||||
audit UI. Email delivery. Each is a real request and each doubles the surface.
|
||||
|
||||
The one with a genuine argument is **notify-on-read**: it confirms delivery and,
|
||||
if it fires before the recipient says they opened it, that is a compromise
|
||||
signal. It needs an email path, `notify` already exists to provide one, and it is
|
||||
the first thing to add if hush proves useful.
|
||||
21
go.mod
Normal file
21
go.mod
Normal file
@ -0,0 +1,21 @@
|
||||
module github.com/orchard9/hush
|
||||
|
||||
go 1.26.0
|
||||
|
||||
require (
|
||||
github.com/orchard9/go-chassis v0.1.0
|
||||
github.com/prometheus/client_golang v1.24.0
|
||||
github.com/redis/go-redis/v9 v9.22.0
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/beorn7/perks v1.0.1 // indirect
|
||||
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
||||
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
|
||||
github.com/prometheus/client_model v0.6.2 // indirect
|
||||
github.com/prometheus/common v0.70.0 // indirect
|
||||
github.com/prometheus/procfs v0.21.1 // indirect
|
||||
go.uber.org/atomic v1.11.0 // indirect
|
||||
golang.org/x/sys v0.47.0 // indirect
|
||||
google.golang.org/protobuf v1.36.11 // indirect
|
||||
)
|
||||
50
go.sum
Normal file
50
go.sum
Normal file
@ -0,0 +1,50 @@
|
||||
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
|
||||
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
|
||||
github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs=
|
||||
github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c=
|
||||
github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA=
|
||||
github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0=
|
||||
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
|
||||
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
||||
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
|
||||
github.com/klauspost/compress v1.19.0 h1:sXLILfc9jV2QYWkzFOPWStmcUVH2RHEB1JCdY2oVvCQ=
|
||||
github.com/klauspost/compress v1.19.0/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
|
||||
github.com/klauspost/cpuid/v2 v2.2.10 h1:tBs3QSyvjDyFTq3uoc/9xFpCuOsJQFNPiAhYdw2skhE=
|
||||
github.com/klauspost/cpuid/v2 v2.2.10/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
|
||||
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
|
||||
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
|
||||
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
|
||||
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
|
||||
github.com/orchard9/go-chassis v0.1.0 h1:epi8NWfYIqhS0GkHMRLVS0v9//brI4Ithd7RyaoYxA4=
|
||||
github.com/orchard9/go-chassis v0.1.0/go.mod h1:lhBicqc77tUEogh1jScEWEzxPqnsw2fpbzid/1bKVcQ=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/prometheus/client_golang v1.24.0 h1:5XStIklKuAtJSNpdD3s8XJj/Yv78IQmE1kbNk87JrAI=
|
||||
github.com/prometheus/client_golang v1.24.0/go.mod h1:QcsNdotprC2nS4BTM2ucbcqxd2CeXTEa9jW7zHO9iDE=
|
||||
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
|
||||
github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=
|
||||
github.com/prometheus/common v0.70.0 h1:bcpru3tWPVnxGnETLgOV5jbp/JRXgYEyv65CuBLAMMI=
|
||||
github.com/prometheus/common v0.70.0/go.mod h1:S/SFasQmgGiYH6C81LKCtYa8QACgthGg5zxL2udV7SY=
|
||||
github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
|
||||
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
|
||||
github.com/redis/go-redis/v9 v9.22.0 h1:laDvpYXTJtZLloinw1fA5Kqd6HAEH2XKxOkG/PDq2F0=
|
||||
github.com/redis/go-redis/v9 v9.22.0/go.mod h1:y2g0Wj8rQvuK0ELM+oxSudcLtC09JScs98I/X9gRWY4=
|
||||
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
|
||||
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
|
||||
github.com/zeebo/xxh3 v1.1.0 h1:s7DLGDK45Dyfg7++yxI0khrfwq9661w9EN78eP/UZVs=
|
||||
github.com/zeebo/xxh3 v1.1.0/go.mod h1:IisAie1LELR4xhVinxWS5+zf1lA4p0MW4T+w+W07F5s=
|
||||
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
|
||||
go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
|
||||
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
|
||||
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
|
||||
go.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ=
|
||||
go.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ=
|
||||
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
|
||||
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
|
||||
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
|
||||
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
131
internal/secret/id.go
Normal file
131
internal/secret/id.go
Normal file
@ -0,0 +1,131 @@
|
||||
// Package secret holds hush's domain types and policy: identifiers, size and
|
||||
// lifetime limits. It imports nothing outside the standard library, so the
|
||||
// rules live in one place and are testable without Redis or HTTP.
|
||||
package secret
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"log/slog"
|
||||
)
|
||||
|
||||
// IDBytes is the entropy behind a secret id. 256 bits makes enumeration a
|
||||
// non-threat: an attacker guessing ids is not a scenario this service defends
|
||||
// against with rate limits, it is a scenario arithmetic forecloses.
|
||||
const IDBytes = 32
|
||||
|
||||
// idEncoding is base64url without padding, so an id is URL-safe, 43 characters,
|
||||
// and needs no escaping in a path or a fragment.
|
||||
var idEncoding = base64.RawURLEncoding
|
||||
|
||||
// ErrMalformedID is returned for a value that cannot be an id this service
|
||||
// minted. Callers turn it into the same 410 as a missing secret — telling a
|
||||
// caller that their id was well-formed but absent confirms the id space.
|
||||
var ErrMalformedID = errors.New("malformed secret id")
|
||||
|
||||
// ID is a secret's identifier, and it IS the capability: anyone holding it can
|
||||
// reveal the secret exactly once. It is therefore treated like a bearer token.
|
||||
//
|
||||
// Every accidental path prints a REDACTED form. An earlier version tried to
|
||||
// prevent leaks by implementing no String() at all, which was wrong: Go's fmt
|
||||
// prints unexported struct fields anyway, so `fmt.Sprintf("%v", id)` emitted
|
||||
// the live id. Forbidding the method did not remove the leak, it only removed
|
||||
// the chance to control it.
|
||||
//
|
||||
// So instead the safe form is the DEFAULT and the raw value needs an explicit
|
||||
// call:
|
||||
//
|
||||
// String() -> "ID(a1b2c3d4e5f6)" fmt %v, %s, string concatenation
|
||||
// LogValue() -> "a1b2c3d4e5f6" every slog call site
|
||||
// MarshalJSON -> refuses a response struct cannot leak one silently
|
||||
// Value() -> the raw id the two places it must escape
|
||||
//
|
||||
// Enforced by id_test.go, which fails if any of those starts emitting the raw
|
||||
// value.
|
||||
type ID struct {
|
||||
raw string
|
||||
}
|
||||
|
||||
// NewID mints a fresh identifier from crypto/rand. It returns an error rather
|
||||
// than panicking: a service that cannot get entropy must refuse to mint a
|
||||
// secret, not mint a guessable one.
|
||||
func NewID() (ID, error) {
|
||||
b := make([]byte, IDBytes)
|
||||
if _, err := rand.Read(b); err != nil {
|
||||
return ID{}, err
|
||||
}
|
||||
return ID{raw: idEncoding.EncodeToString(b)}, nil
|
||||
}
|
||||
|
||||
// ParseID validates an id from a URL path. It checks the encoding and the
|
||||
// decoded LENGTH, so a short-but-valid base64 string cannot become an id and
|
||||
// widen the space a scanner has to cover.
|
||||
func ParseID(s string) (ID, error) {
|
||||
if len(s) != idEncoding.EncodedLen(IDBytes) {
|
||||
return ID{}, ErrMalformedID
|
||||
}
|
||||
b, err := idEncoding.DecodeString(s)
|
||||
if err != nil || len(b) != IDBytes {
|
||||
return ID{}, ErrMalformedID
|
||||
}
|
||||
return ID{raw: s}, nil
|
||||
}
|
||||
|
||||
// Value returns the raw id. Every call site is a place where the capability
|
||||
// escapes, so there are deliberately few: the storage key and the created
|
||||
// response body.
|
||||
func (id ID) Value() string { return id.raw }
|
||||
|
||||
// IsZero reports whether this is the zero ID, which no minting path produces.
|
||||
func (id ID) IsZero() bool { return id.raw == "" }
|
||||
|
||||
// LogHandle is the only form of an id that may be logged or reported: the first
|
||||
// 12 hex characters of its SHA-256. It is stable, so one secret's create,
|
||||
// reveal and gone lines correlate across the log corpus, and it is one-way, so
|
||||
// a log reader cannot reveal the secret it refers to.
|
||||
//
|
||||
// 48 bits of a hash is not a secret-strength value and is not treated as one —
|
||||
// it is a correlation handle. The preimage is 256 bits of entropy, so recovering
|
||||
// an id from a handle is not feasible even though the handle is short.
|
||||
func (id ID) LogHandle() string {
|
||||
if id.raw == "" {
|
||||
return ""
|
||||
}
|
||||
sum := sha256.Sum256([]byte(id.raw))
|
||||
return hex.EncodeToString(sum[:])[:12]
|
||||
}
|
||||
|
||||
// String is the redacted form, so `%v`, `%s` and string concatenation are all
|
||||
// safe by default. Reaching the raw id requires Value().
|
||||
func (id ID) String() string {
|
||||
if id.raw == "" {
|
||||
return "ID(zero)"
|
||||
}
|
||||
return "ID(" + id.LogHandle() + ")"
|
||||
}
|
||||
|
||||
// LogValue makes every slog call site safe without the caller thinking about
|
||||
// it: `log.Info("secret.created", "id", id)` emits the handle, not the
|
||||
// capability. This is why the handlers can pass ids around without a review
|
||||
// checklist for each log line.
|
||||
func (id ID) LogValue() slog.Value { return slog.StringValue(id.LogHandle()) }
|
||||
|
||||
// MarshalJSON REFUSES rather than emitting either form.
|
||||
//
|
||||
// Emitting the raw id would leak a capability into any response struct that
|
||||
// happened to embed an ID. Emitting the redacted form would be worse: it would
|
||||
// produce a response that looks like it carries an id and does not, and the bug
|
||||
// would surface as a broken link rather than a failed request. The two places
|
||||
// an id legitimately reaches a client both call Value() explicitly.
|
||||
func (id ID) MarshalJSON() ([]byte, error) {
|
||||
return nil, errors.New("secret.ID must not be serialised: call Value() at the one site that needs it")
|
||||
}
|
||||
|
||||
// StorageKey is the Redis key holding this secret's ciphertext. The `hush:`
|
||||
// prefix is what the Redis ACL user is scoped to (`~hush:*`), so a bug that
|
||||
// built a key outside this prefix would be refused by the server rather than
|
||||
// touching another tenant's keyspace.
|
||||
func (id ID) StorageKey() string { return "hush:s:" + id.raw }
|
||||
190
internal/secret/id_test.go
Normal file
190
internal/secret/id_test.go
Normal file
@ -0,0 +1,190 @@
|
||||
package secret
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The id IS the capability, so every path that could print one is checked here.
|
||||
//
|
||||
// The first version of this test forbade String() outright, on the theory that
|
||||
// a type with no Stringer cannot be interpolated. That was wrong and this test
|
||||
// caught it: Go's fmt prints unexported struct fields regardless, so `%v` was
|
||||
// emitting live ids. The design now makes the REDACTED form the default on
|
||||
// every accidental path and requires Value() for the raw one.
|
||||
//
|
||||
// Read this test first if you are wondering why ID has a String() that throws
|
||||
// away information and a MarshalJSON that refuses.
|
||||
func TestNoAccidentalPathEmitsTheRawID(t *testing.T) {
|
||||
id, err := NewID()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
raw := id.Value()
|
||||
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
got string
|
||||
}{
|
||||
{"fmt %v", fmt.Sprintf("%v", id)},
|
||||
{"fmt %s", fmt.Sprintf("%s", id)},
|
||||
{"fmt %+v", fmt.Sprintf("%+v", id)},
|
||||
{"fmt %#v of a wrapper struct", fmt.Sprintf("%v", struct{ ID ID }{id})},
|
||||
{"String()", id.String()},
|
||||
{"LogValue()", id.LogValue().String()},
|
||||
{"inside a slice", fmt.Sprintf("%v", []ID{id})},
|
||||
{"inside a map", fmt.Sprintf("%v", map[string]ID{"k": id})},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
if strings.Contains(tc.got, raw) {
|
||||
t.Fatalf("%s emitted the live id (%q) — this value would appear in a log line "+
|
||||
"and anyone reading it could reveal the secret", tc.name, tc.got)
|
||||
}
|
||||
if !strings.Contains(tc.got, id.LogHandle()) {
|
||||
t.Fatalf("%s = %q, which carries neither the id nor its handle; a log line "+
|
||||
"with no handle cannot be correlated", tc.name, tc.got)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// A real slog handler, because that is where ids actually pass through.
|
||||
func TestSlogEmitsTheHandleNotTheID(t *testing.T) {
|
||||
id, err := NewID()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var buf bytes.Buffer
|
||||
log := slog.New(slog.NewJSONHandler(&buf, nil))
|
||||
|
||||
// Every shape a call site might use, including the careless one.
|
||||
log.Info("secret.created", "id", id)
|
||||
log.Info("secret.created", "sid", id.LogHandle())
|
||||
log.With("id", id).Info("secret.revealed")
|
||||
log.Info("secret.gone", "ids", []ID{id})
|
||||
|
||||
out := buf.String()
|
||||
if strings.Contains(out, id.Value()) {
|
||||
t.Fatalf("slog output contains the live id:\n%s", out)
|
||||
}
|
||||
if !strings.Contains(out, id.LogHandle()) {
|
||||
t.Fatalf("slog output contains no correlation handle:\n%s", out)
|
||||
}
|
||||
}
|
||||
|
||||
// Serialising an ID must FAIL rather than quietly emit either form. The raw
|
||||
// value would leak a capability out of any response struct embedding an ID;
|
||||
// the redacted value would produce a response that looks like it carries an id
|
||||
// and does not, turning a server bug into a broken link for the recipient.
|
||||
func TestMarshallingAnIDIsAnError(t *testing.T) {
|
||||
id, err := NewID()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if b, err := json.Marshal(id); err == nil {
|
||||
t.Fatalf("json.Marshal(ID) succeeded with %s; it must refuse", b)
|
||||
}
|
||||
// And from inside a struct, which is how it would actually happen.
|
||||
if b, err := json.Marshal(struct {
|
||||
ID ID `json:"id"`
|
||||
}{id}); err == nil {
|
||||
t.Fatalf("marshalling a struct containing an ID succeeded with %s; it must refuse", b)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewIDIsUnguessablyWideAndUnique(t *testing.T) {
|
||||
seen := make(map[string]bool, 1000)
|
||||
for range 1000 {
|
||||
id, err := NewID()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(id.Value()) != 43 {
|
||||
t.Fatalf("id %q is %d chars, want 43 (256 bits base64url unpadded)", id.Value(), len(id.Value()))
|
||||
}
|
||||
if seen[id.Value()] {
|
||||
t.Fatal("NewID repeated a value in 1000 draws — entropy is broken")
|
||||
}
|
||||
seen[id.Value()] = true
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseIDRoundTripsAndRefusesEverythingElse(t *testing.T) {
|
||||
id, err := NewID()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
back, err := ParseID(id.Value())
|
||||
if err != nil {
|
||||
t.Fatalf("ParseID rejected a freshly minted id: %v", err)
|
||||
}
|
||||
if back.Value() != id.Value() {
|
||||
t.Fatalf("round trip changed the id: %q -> %q", id.Value(), back.Value())
|
||||
}
|
||||
|
||||
for _, bad := range []struct{ name, in string }{
|
||||
{"empty", ""},
|
||||
{"too short", "abc"},
|
||||
{"one char short", id.Value()[:42]},
|
||||
{"one char long", id.Value() + "a"},
|
||||
{"not base64url", strings.Repeat("!", 43)},
|
||||
// A shorter secret encoded in a 43-char field would widen the space a
|
||||
// scanner must cover, so the DECODED length is checked too.
|
||||
{"padded base64", strings.Repeat("A", 40) + "==="},
|
||||
{"path traversal", "../../etc/passwd"},
|
||||
{"redis glob", strings.Repeat("*", 43)},
|
||||
} {
|
||||
t.Run(bad.name, func(t *testing.T) {
|
||||
if _, err := ParseID(bad.in); err == nil {
|
||||
t.Fatalf("ParseID(%q) accepted a value NewID cannot produce", bad.in)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestLogHandleIsStableShortAndNotTheID(t *testing.T) {
|
||||
id, err := NewID()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
h := id.LogHandle()
|
||||
if len(h) != 12 {
|
||||
t.Fatalf("LogHandle() = %q, want 12 hex chars", h)
|
||||
}
|
||||
if id.LogHandle() != h {
|
||||
t.Fatal("LogHandle is not stable across calls, so log lines for one secret will not correlate")
|
||||
}
|
||||
if strings.Contains(h, id.Value()) || strings.Contains(id.Value(), h) {
|
||||
t.Fatalf("LogHandle %q overlaps the raw id %q — it must be a hash, not a prefix", h, id.Value())
|
||||
}
|
||||
|
||||
// Different ids must not collide into one handle, or correlation is wrong.
|
||||
other, err := NewID()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if other.LogHandle() == h {
|
||||
t.Fatal("two ids produced the same LogHandle")
|
||||
}
|
||||
|
||||
if (ID{}).LogHandle() != "" {
|
||||
t.Fatal("the zero ID must produce an empty handle, not a hash of the empty string")
|
||||
}
|
||||
}
|
||||
|
||||
func TestStorageKeyStaysInsideTheACLPrefix(t *testing.T) {
|
||||
id, err := NewID()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// The Redis ACL user is scoped to ~hush:*. A key built outside that prefix
|
||||
// is refused by the server, so this test is what keeps a rename from
|
||||
// turning every write into a NOPERM at runtime.
|
||||
if !strings.HasPrefix(id.StorageKey(), "hush:") {
|
||||
t.Fatalf("StorageKey() = %q, which is outside the ~hush:* ACL scope", id.StorageKey())
|
||||
}
|
||||
}
|
||||
80
internal/secret/policy.go
Normal file
80
internal/secret/policy.go
Normal file
@ -0,0 +1,80 @@
|
||||
package secret
|
||||
|
||||
import (
|
||||
"encoding/base64"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Size and lifetime policy. Every bound here is a product decision with a
|
||||
// reason, not a tuning knob:
|
||||
//
|
||||
// - MaxCiphertextBytes keeps hush a courier for credentials rather than a file
|
||||
// host. 64 KiB of AES-GCM holds roughly 48 KiB of plaintext, which is a very
|
||||
// large credential and a very small file.
|
||||
// - MinTTL exists because a link that expires before the recipient reads their
|
||||
// messages is a support ticket, not a security win.
|
||||
// - MaxTTL bounds exposure. The residual risk in this design is the key sitting
|
||||
// in a browser history entry, and a week is as long as that is defensible.
|
||||
const (
|
||||
MaxCiphertextBytes = 64 * 1024
|
||||
MinTTL = 5 * time.Minute
|
||||
MaxTTL = 7 * 24 * time.Hour
|
||||
DefaultTTL = 24 * time.Hour
|
||||
)
|
||||
|
||||
// Validation failures. Each maps to one API error code, so a caller can branch
|
||||
// on the cause without parsing prose.
|
||||
var (
|
||||
ErrCiphertextEmpty = errors.New("ciphertext is empty")
|
||||
ErrCiphertextTooLarge = errors.New("ciphertext exceeds the size limit")
|
||||
ErrCiphertextInvalid = errors.New("ciphertext is not valid base64url")
|
||||
ErrTTLOutOfRange = errors.New("ttl is outside the permitted range")
|
||||
)
|
||||
|
||||
// ciphertextEncoding matches what the browser produces: base64url, unpadded.
|
||||
var ciphertextEncoding = base64.RawURLEncoding
|
||||
|
||||
// ValidateCiphertext checks what the server is ABLE to check. hush cannot
|
||||
// verify that the bytes decrypt, because it has no key — by design. So it
|
||||
// verifies the two things it can: that the encoding is what this service's
|
||||
// clients produce, and that the size is inside the cap.
|
||||
//
|
||||
// The encoding check is not cosmetic. Without it, hush becomes a store for
|
||||
// arbitrary bytes addressable by URL, which is a different and much less
|
||||
// defensible service than the one described in the README.
|
||||
func ValidateCiphertext(ciphertext string) error {
|
||||
switch {
|
||||
case ciphertext == "":
|
||||
return ErrCiphertextEmpty
|
||||
case len(ciphertext) > MaxCiphertextBytes:
|
||||
// Measured on the encoded form, which is what is stored and what
|
||||
// bounds memory. Checked BEFORE decoding so an oversized body is
|
||||
// rejected without allocating its decoded copy.
|
||||
return fmt.Errorf("%w: %d > %d bytes", ErrCiphertextTooLarge, len(ciphertext), MaxCiphertextBytes)
|
||||
}
|
||||
if _, err := ciphertextEncoding.DecodeString(ciphertext); err != nil {
|
||||
return ErrCiphertextInvalid
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ResolveTTL turns a caller's requested lifetime into the one that will be
|
||||
// used. Zero means "unspecified" and gets the default.
|
||||
//
|
||||
// An out-of-range value is an ERROR, never a silent clamp. A caller who asked
|
||||
// for 30 days and got 7 without being told would believe their link outlives
|
||||
// its actual expiry, and would find out when the recipient could not open it.
|
||||
func ResolveTTL(requested time.Duration) (time.Duration, error) {
|
||||
if requested == 0 {
|
||||
return DefaultTTL, nil
|
||||
}
|
||||
if requested < MinTTL || requested > MaxTTL {
|
||||
return 0, fmt.Errorf("%w: %s not in [%s, %s]", ErrTTLOutOfRange, requested, MinTTL, MaxTTL)
|
||||
}
|
||||
// Truncate to whole seconds: the wire format is seconds and Redis EX takes
|
||||
// seconds, so keeping sub-second precision would make the expires_at we
|
||||
// report disagree with the expiry Redis enforces.
|
||||
return requested.Truncate(time.Second), nil
|
||||
}
|
||||
91
internal/secret/policy_test.go
Normal file
91
internal/secret/policy_test.go
Normal file
@ -0,0 +1,91 @@
|
||||
package secret
|
||||
|
||||
import (
|
||||
"encoding/base64"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestValidateCiphertext(t *testing.T) {
|
||||
valid := base64.RawURLEncoding.EncodeToString([]byte("nonce+ciphertext bytes"))
|
||||
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
in string
|
||||
want error
|
||||
}{
|
||||
{"a real base64url blob", valid, nil},
|
||||
{"empty", "", ErrCiphertextEmpty},
|
||||
{"not base64url", "!!!not base64!!!", ErrCiphertextInvalid},
|
||||
// Standard base64 uses + and /, which are not URL-safe. Accepting them
|
||||
// would mean two spellings of one ciphertext and a client that works in
|
||||
// one browser and not another.
|
||||
{"standard base64 alphabet", "YWJj+/8=", ErrCiphertextInvalid},
|
||||
{"padded", "YWJjZA==", ErrCiphertextInvalid},
|
||||
{"at the cap", strings.Repeat("A", MaxCiphertextBytes), nil},
|
||||
{"one byte over the cap", strings.Repeat("A", MaxCiphertextBytes+1), ErrCiphertextTooLarge},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
err := ValidateCiphertext(tc.in)
|
||||
if !errors.Is(err, tc.want) {
|
||||
t.Fatalf("ValidateCiphertext() = %v, want %v", err, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The size check must happen BEFORE the decode, or a 64 KiB+ body costs a
|
||||
// decoded copy before being refused — which is the cheap half of a memory DoS.
|
||||
func TestOversizedCiphertextIsRefusedWithoutDecoding(t *testing.T) {
|
||||
// Deliberately not valid base64. If the implementation decoded first, this
|
||||
// would come back as ErrCiphertextInvalid instead of ErrCiphertextTooLarge.
|
||||
huge := strings.Repeat("!", MaxCiphertextBytes+1)
|
||||
if err := ValidateCiphertext(huge); !errors.Is(err, ErrCiphertextTooLarge) {
|
||||
t.Fatalf("ValidateCiphertext(oversized invalid) = %v, want ErrCiphertextTooLarge — "+
|
||||
"the size gate must precede the decode", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveTTL(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
in time.Duration
|
||||
want time.Duration
|
||||
err error
|
||||
}{
|
||||
{"unspecified takes the default", 0, DefaultTTL, nil},
|
||||
{"at the minimum", MinTTL, MinTTL, nil},
|
||||
{"at the maximum", MaxTTL, MaxTTL, nil},
|
||||
{"a normal day", 24 * time.Hour, 24 * time.Hour, nil},
|
||||
{"below the minimum", MinTTL - time.Second, 0, ErrTTLOutOfRange},
|
||||
{"above the maximum", MaxTTL + time.Second, 0, ErrTTLOutOfRange},
|
||||
{"negative", -time.Hour, 0, ErrTTLOutOfRange},
|
||||
{"absurd", 3650 * 24 * time.Hour, 0, ErrTTLOutOfRange},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
got, err := ResolveTTL(tc.in)
|
||||
if !errors.Is(err, tc.err) {
|
||||
t.Fatalf("ResolveTTL(%s) error = %v, want %v", tc.in, err, tc.err)
|
||||
}
|
||||
if err == nil && got != tc.want {
|
||||
t.Fatalf("ResolveTTL(%s) = %s, want %s", tc.in, got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// An out-of-range TTL must be an ERROR and never a silent clamp. A caller who
|
||||
// asked for 30 days and was quietly given 7 would believe their link outlives
|
||||
// its real expiry, and would discover otherwise when the recipient could not
|
||||
// open it.
|
||||
func TestOutOfRangeTTLIsRefusedNotClamped(t *testing.T) {
|
||||
got, err := ResolveTTL(30 * 24 * time.Hour)
|
||||
if err == nil {
|
||||
t.Fatalf("ResolveTTL(30d) silently returned %s instead of refusing", got)
|
||||
}
|
||||
if got != 0 {
|
||||
t.Fatalf("ResolveTTL returned %s alongside an error; callers must not see a usable value", got)
|
||||
}
|
||||
}
|
||||
80
internal/store/memory.go
Normal file
80
internal/store/memory.go
Normal file
@ -0,0 +1,80 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/orchard9/hush/internal/secret"
|
||||
)
|
||||
|
||||
// Memory is an in-process Store for tests and `make dev` without Redis.
|
||||
//
|
||||
// It is NOT a deployment option, and the deployment path cannot select it: the
|
||||
// store is chosen by whether REDIS_URL is set, and REDIS_URL is Required in
|
||||
// config, so a misconfigured pod fails to boot rather than silently serving
|
||||
// secrets from a store that dies with the process. This type exists so the
|
||||
// handler tests do not need a container.
|
||||
type Memory struct {
|
||||
mu sync.Mutex
|
||||
items map[string]memItem
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
type memItem struct {
|
||||
ciphertext string
|
||||
expiresAt time.Time
|
||||
}
|
||||
|
||||
// NewMemory returns an empty store using the real clock.
|
||||
func NewMemory() *Memory {
|
||||
return &Memory{items: map[string]memItem{}, now: time.Now}
|
||||
}
|
||||
|
||||
// NewMemoryAt returns a store driven by a caller-supplied clock, so expiry is
|
||||
// testable without sleeping.
|
||||
func NewMemoryAt(now func() time.Time) *Memory {
|
||||
return &Memory{items: map[string]memItem{}, now: now}
|
||||
}
|
||||
|
||||
func (m *Memory) Put(_ context.Context, id secret.ID, ciphertext string, ttl time.Duration) error {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
k := id.StorageKey()
|
||||
// Mirror Redis SET NX semantics, including that an EXPIRED key is treated
|
||||
// as absent and may be overwritten. A memory store that rejected a write
|
||||
// against a stale entry would pass tests Redis fails.
|
||||
if it, ok := m.items[k]; ok && m.now().Before(it.expiresAt) {
|
||||
return ErrIDCollision
|
||||
}
|
||||
m.items[k] = memItem{ciphertext: ciphertext, expiresAt: m.now().Add(ttl)}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Take mirrors GETDEL: the read and the delete happen under one lock, so the
|
||||
// atomicity the one-time guarantee depends on holds here too and the
|
||||
// concurrency test is meaningful against both implementations.
|
||||
func (m *Memory) Take(_ context.Context, id secret.ID) (string, error) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
k := id.StorageKey()
|
||||
it, ok := m.items[k]
|
||||
if !ok {
|
||||
return "", ErrGone
|
||||
}
|
||||
delete(m.items, k)
|
||||
if !m.now().Before(it.expiresAt) {
|
||||
return "", ErrGone
|
||||
}
|
||||
return it.ciphertext, nil
|
||||
}
|
||||
|
||||
func (m *Memory) Ping(context.Context) error { return nil }
|
||||
func (m *Memory) Close() error { return nil }
|
||||
|
||||
// AllowN is an always-allow limiter: rate limiting is an abuse control on the
|
||||
// public deployment, and silently enforcing one in tests would make handler
|
||||
// tests order-dependent and flaky.
|
||||
func (m *Memory) AllowN(context.Context, string, int, time.Duration) (bool, time.Duration, error) {
|
||||
return true, 0, nil
|
||||
}
|
||||
117
internal/store/redis.go
Normal file
117
internal/store/redis.go
Normal file
@ -0,0 +1,117 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/redis/go-redis/v9"
|
||||
|
||||
"github.com/orchard9/hush/internal/secret"
|
||||
)
|
||||
|
||||
// Redis is the production store. It keeps one key per secret and lets Redis own
|
||||
// expiry, so nothing in hush sweeps, scans, or holds a timer.
|
||||
type Redis struct {
|
||||
client *redis.Client
|
||||
}
|
||||
|
||||
// NewRedis dials Redis from a URL of the form
|
||||
// redis://user:password@host:6379/5 — the ACL user, the password and the db
|
||||
// index all ride in the URL, matching every other orchard9 service.
|
||||
func NewRedis(url string) (*Redis, error) {
|
||||
opt, err := redis.ParseURL(url)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("parse redis url: %w", err)
|
||||
}
|
||||
// A secret write must not hang a request behind a slow dependency: the
|
||||
// chassis request timeout would fire and the caller would see a 500 with no
|
||||
// idea whether the secret was stored. Short, explicit timeouts make the
|
||||
// failure fast and unambiguous.
|
||||
opt.DialTimeout = 3 * time.Second
|
||||
opt.ReadTimeout = 2 * time.Second
|
||||
opt.WriteTimeout = 2 * time.Second
|
||||
opt.MaxRetries = 2
|
||||
return &Redis{client: redis.NewClient(opt)}, nil
|
||||
}
|
||||
|
||||
// Put stores ciphertext with SET ... EX ttl NX.
|
||||
//
|
||||
// NX is what makes an id collision an error instead of a silent overwrite of a
|
||||
// live secret. It cannot fire by chance at 256 bits, which is exactly why a
|
||||
// false return is worth surfacing: it means something is wrong with id
|
||||
// generation, and the alternative is destroying a secret somebody is waiting on.
|
||||
func (r *Redis) Put(ctx context.Context, id secret.ID, ciphertext string, ttl time.Duration) error {
|
||||
ok, err := r.client.SetNX(ctx, id.StorageKey(), ciphertext, ttl).Result()
|
||||
if err != nil {
|
||||
return fmt.Errorf("redis set: %w", err)
|
||||
}
|
||||
if !ok {
|
||||
return ErrIDCollision
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Take reads and destroys in ONE command.
|
||||
//
|
||||
// GETDEL (Redis 6.2+) is atomic, which is the entire one-time guarantee. A
|
||||
// GET followed by a DEL has a window between the two round trips where two
|
||||
// simultaneous readers both get the plaintext, and for this service that window
|
||||
// is the product. Redis 7.4.8 runs in the cluster; the ACL user must carry
|
||||
// `+getdel` or every reveal returns NOPERM.
|
||||
//
|
||||
// redis.Nil covers all four "not available" causes and collapses to ErrGone —
|
||||
// see the doc comment on ErrGone for why they are not separated.
|
||||
func (r *Redis) Take(ctx context.Context, id secret.ID) (string, error) {
|
||||
v, err := r.client.GetDel(ctx, id.StorageKey()).Result()
|
||||
switch {
|
||||
case errors.Is(err, redis.Nil):
|
||||
return "", ErrGone
|
||||
case err != nil:
|
||||
return "", fmt.Errorf("redis getdel: %w", err)
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
|
||||
// Ping backs /readyz. It is the reason a Redis outage takes the pod out of the
|
||||
// Service rather than serving 500s from a pod the load balancer still trusts.
|
||||
func (r *Redis) Ping(ctx context.Context) error {
|
||||
return r.client.Ping(ctx).Err()
|
||||
}
|
||||
|
||||
// Close shuts the pool down.
|
||||
func (r *Redis) Close() error { return r.client.Close() }
|
||||
|
||||
// AllowN implements a fixed-window rate limit in Redis: INCR the window key,
|
||||
// set its expiry on first use, and refuse once the count passes the limit.
|
||||
//
|
||||
// Fixed window rather than a sliding one because it costs two commands, needs
|
||||
// no Lua, and the failure it permits — up to 2x the limit across a window
|
||||
// boundary — is irrelevant for an abuse control whose job is to stop bulk
|
||||
// automation, not to meter precisely.
|
||||
//
|
||||
// It lives in Redis rather than in process memory so the limit still holds if
|
||||
// hush is ever scaled past one replica, and so a restart cannot be used to
|
||||
// reset it.
|
||||
func (r *Redis) AllowN(ctx context.Context, key string, limit int, window time.Duration) (bool, time.Duration, error) {
|
||||
full := "hush:rl:" + key
|
||||
pipe := r.client.Pipeline()
|
||||
incr := pipe.Incr(ctx, full)
|
||||
// NX so a long-running window is not extended by later requests inside it;
|
||||
// without it a steady stream of calls would push the expiry forward forever
|
||||
// and the window would never reset.
|
||||
pipe.ExpireNX(ctx, full, window)
|
||||
if _, err := pipe.Exec(ctx); err != nil {
|
||||
return false, 0, fmt.Errorf("redis ratelimit: %w", err)
|
||||
}
|
||||
count := incr.Val()
|
||||
if count > int64(limit) {
|
||||
retry, err := r.client.PTTL(ctx, full).Result()
|
||||
if err != nil || retry < 0 {
|
||||
retry = window
|
||||
}
|
||||
return false, retry, nil
|
||||
}
|
||||
return true, 0, nil
|
||||
}
|
||||
50
internal/store/store.go
Normal file
50
internal/store/store.go
Normal file
@ -0,0 +1,50 @@
|
||||
// Package store persists ciphertext for at most a TTL and destroys it on the
|
||||
// first successful read.
|
||||
package store
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"time"
|
||||
|
||||
"github.com/orchard9/hush/internal/secret"
|
||||
)
|
||||
|
||||
// ErrGone is returned by Take for every reason a secret is not available: it
|
||||
// never existed, it was already revealed, it expired, or Redis evicted it early
|
||||
// under memory pressure.
|
||||
//
|
||||
// The causes are deliberately NOT distinguished. Not because they are hard to
|
||||
// tell apart, but because telling them apart is the leak: a caller who can
|
||||
// separate "never existed" from "already revealed" can confirm that a given
|
||||
// link was real, which is information about someone else's secret. The API maps
|
||||
// this one error to one 410 for all four cases.
|
||||
var ErrGone = errors.New("secret is gone")
|
||||
|
||||
// ErrIDCollision means Put found the key already occupied. At 256 bits of id
|
||||
// entropy this cannot happen by chance, so it means a bug or a repeated id, and
|
||||
// it is surfaced rather than silently overwriting a live secret.
|
||||
var ErrIDCollision = errors.New("secret id already exists")
|
||||
|
||||
// Store is the whole persistence contract. Two methods, both destructive-safe:
|
||||
// there is no Get, no List, and no Exists — a store that could answer "does
|
||||
// this id exist" without consuming it would let the reveal page leak existence,
|
||||
// and a store that could list would make a compromise catastrophic instead of
|
||||
// merely bad.
|
||||
type Store interface {
|
||||
// Put writes ciphertext under id, expiring after ttl. It must fail rather
|
||||
// than overwrite an existing key.
|
||||
Put(ctx context.Context, id secret.ID, ciphertext string, ttl time.Duration) error
|
||||
|
||||
// Take returns the ciphertext and destroys it ATOMICALLY. Two concurrent
|
||||
// callers must not both receive a value; exactly one gets it and the other
|
||||
// gets ErrGone. This atomicity is the one-time property — an implementation
|
||||
// that reads then deletes has a window in which both callers succeed.
|
||||
Take(ctx context.Context, id secret.ID) (string, error)
|
||||
|
||||
// Ping reports whether the backing store is usable, for readiness.
|
||||
Ping(ctx context.Context) error
|
||||
|
||||
// Close releases resources.
|
||||
Close() error
|
||||
}
|
||||
246
internal/store/store_test.go
Normal file
246
internal/store/store_test.go
Normal file
@ -0,0 +1,246 @@
|
||||
package store_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"os"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/orchard9/hush/internal/secret"
|
||||
"github.com/orchard9/hush/internal/store"
|
||||
)
|
||||
|
||||
// One contract, two implementations. The suite runs against Memory always and
|
||||
// against Redis whenever HUSH_TEST_REDIS_URL is set (`make test-redis`, and CI
|
||||
// where a Redis service is available).
|
||||
//
|
||||
// Running the SAME assertions against both is the point: Memory exists so
|
||||
// handler tests need no container, and it is only trustworthy if it is held to
|
||||
// the behaviour Redis actually has — including that an expired key may be
|
||||
// overwritten and that Take is atomic.
|
||||
func eachStore(t *testing.T, fn func(t *testing.T, s store.Store)) {
|
||||
t.Helper()
|
||||
|
||||
t.Run("memory", func(t *testing.T) {
|
||||
fn(t, store.NewMemory())
|
||||
})
|
||||
|
||||
url := os.Getenv("HUSH_TEST_REDIS_URL")
|
||||
if url == "" {
|
||||
t.Log("HUSH_TEST_REDIS_URL unset: skipping the Redis half of the contract suite")
|
||||
return
|
||||
}
|
||||
t.Run("redis", func(t *testing.T) {
|
||||
r, err := store.NewRedis(url)
|
||||
if err != nil {
|
||||
t.Fatalf("dial redis: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { _ = r.Close() })
|
||||
if err := r.Ping(context.Background()); err != nil {
|
||||
t.Fatalf("ping redis at %s: %v", url, err)
|
||||
}
|
||||
fn(t, r)
|
||||
})
|
||||
}
|
||||
|
||||
func newID(t *testing.T) secret.ID {
|
||||
t.Helper()
|
||||
id, err := secret.NewID()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return id
|
||||
}
|
||||
|
||||
func TestPutThenTakeReturnsTheCiphertextExactlyOnce(t *testing.T) {
|
||||
eachStore(t, func(t *testing.T, s store.Store) {
|
||||
ctx := context.Background()
|
||||
id := newID(t)
|
||||
const ct = "bm9uY2UtYW5kLWNpcGhlcnRleHQ"
|
||||
|
||||
if err := s.Put(ctx, id, ct, time.Minute); err != nil {
|
||||
t.Fatalf("Put: %v", err)
|
||||
}
|
||||
|
||||
got, err := s.Take(ctx, id)
|
||||
if err != nil {
|
||||
t.Fatalf("first Take: %v", err)
|
||||
}
|
||||
if got != ct {
|
||||
t.Fatalf("first Take = %q, want %q", got, ct)
|
||||
}
|
||||
|
||||
// The whole product: the second read must find nothing.
|
||||
if _, err := s.Take(ctx, id); !errors.Is(err, store.ErrGone) {
|
||||
t.Fatalf("second Take = %v, want ErrGone — the secret was not destroyed", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestTakeOfAnUnknownIDIsGone(t *testing.T) {
|
||||
eachStore(t, func(t *testing.T, s store.Store) {
|
||||
if _, err := s.Take(context.Background(), newID(t)); !errors.Is(err, store.ErrGone) {
|
||||
t.Fatalf("Take(unknown) = %v, want ErrGone", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestPutRefusesToOverwriteALiveSecret(t *testing.T) {
|
||||
eachStore(t, func(t *testing.T, s store.Store) {
|
||||
ctx := context.Background()
|
||||
id := newID(t)
|
||||
if err := s.Put(ctx, id, "first", time.Minute); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// At 256 bits this cannot happen by chance, so if it ever does it is an
|
||||
// id-generation bug. Destroying the live secret instead of reporting it
|
||||
// would lose a secret someone is waiting on.
|
||||
if err := s.Put(ctx, id, "second", time.Minute); !errors.Is(err, store.ErrIDCollision) {
|
||||
t.Fatalf("second Put = %v, want ErrIDCollision", err)
|
||||
}
|
||||
got, err := s.Take(ctx, id)
|
||||
if err != nil || got != "first" {
|
||||
t.Fatalf("Take after refused overwrite = (%q, %v), want (\"first\", nil)", got, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// Exactly one winner under concurrency. This is why the store contract demands
|
||||
// an atomic read-and-destroy: a GET followed by a DEL has a window in which two
|
||||
// readers both receive the plaintext, and for a one-time secret that window is
|
||||
// the entire guarantee.
|
||||
func TestConcurrentTakesProduceExactlyOneWinner(t *testing.T) {
|
||||
eachStore(t, func(t *testing.T, s store.Store) {
|
||||
ctx := context.Background()
|
||||
const racers = 32
|
||||
|
||||
for round := range 20 {
|
||||
id := newID(t)
|
||||
if err := s.Put(ctx, id, "only-once", time.Minute); err != nil {
|
||||
t.Fatalf("round %d Put: %v", round, err)
|
||||
}
|
||||
|
||||
var (
|
||||
wg sync.WaitGroup
|
||||
mu sync.Mutex
|
||||
wins int
|
||||
gone int
|
||||
othererr error
|
||||
start = make(chan struct{})
|
||||
)
|
||||
for range racers {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
<-start // release them together to maximise the overlap
|
||||
v, err := s.Take(ctx, id)
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
switch {
|
||||
case err == nil && v == "only-once":
|
||||
wins++
|
||||
case errors.Is(err, store.ErrGone):
|
||||
gone++
|
||||
default:
|
||||
othererr = err
|
||||
}
|
||||
}()
|
||||
}
|
||||
close(start)
|
||||
wg.Wait()
|
||||
|
||||
if othererr != nil {
|
||||
t.Fatalf("round %d: unexpected error from Take: %v", round, othererr)
|
||||
}
|
||||
if wins != 1 {
|
||||
t.Fatalf("round %d: %d goroutines received the secret, want exactly 1 (%d saw gone)", round, wins, gone)
|
||||
}
|
||||
if gone != racers-1 {
|
||||
t.Fatalf("round %d: %d saw gone, want %d", round, gone, racers-1)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestAnExpiredSecretIsGoneAndItsIDIsReusable(t *testing.T) {
|
||||
// Driven by a fake clock so this does not sleep. The Redis half of the
|
||||
// contract is covered by TestRedisHonoursTTL below, which uses a short real
|
||||
// TTL because Redis owns that clock.
|
||||
now := time.Now()
|
||||
s := store.NewMemoryAt(func() time.Time { return now })
|
||||
ctx := context.Background()
|
||||
id := newID(t)
|
||||
|
||||
if err := s.Put(ctx, id, "vanishing", time.Minute); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
now = now.Add(time.Minute + time.Second)
|
||||
|
||||
if _, err := s.Take(ctx, id); !errors.Is(err, store.ErrGone) {
|
||||
t.Fatalf("Take after expiry = %v, want ErrGone", err)
|
||||
}
|
||||
// Redis treats an expired key as absent, so Put must succeed here. A
|
||||
// memory store that refused would pass tests Redis fails.
|
||||
if err := s.Put(ctx, id, "reused", time.Minute); err != nil {
|
||||
t.Fatalf("Put over an expired key = %v, want nil (Redis SET NX succeeds on an expired key)", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRedisHonoursTTL(t *testing.T) {
|
||||
url := os.Getenv("HUSH_TEST_REDIS_URL")
|
||||
if url == "" {
|
||||
t.Skip("HUSH_TEST_REDIS_URL unset")
|
||||
}
|
||||
r, err := store.NewRedis(url)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { _ = r.Close() })
|
||||
|
||||
ctx := context.Background()
|
||||
id := newID(t)
|
||||
// Redis EX takes whole seconds, so 1s is the shortest observable TTL.
|
||||
if err := r.Put(ctx, id, "brief", time.Second); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
time.Sleep(1500 * time.Millisecond)
|
||||
if _, err := r.Take(ctx, id); !errors.Is(err, store.ErrGone) {
|
||||
t.Fatalf("Take after the TTL elapsed = %v, want ErrGone", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRedisRateLimitCountsWithinAWindowAndThenRefuses(t *testing.T) {
|
||||
url := os.Getenv("HUSH_TEST_REDIS_URL")
|
||||
if url == "" {
|
||||
t.Skip("HUSH_TEST_REDIS_URL unset")
|
||||
}
|
||||
r, err := store.NewRedis(url)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { _ = r.Close() })
|
||||
|
||||
ctx := context.Background()
|
||||
// A unique key per run so a re-run is not throttled by the previous one.
|
||||
id := newID(t)
|
||||
key := "test-" + id.LogHandle()
|
||||
|
||||
for i := 1; i <= 3; i++ {
|
||||
ok, _, err := r.AllowN(ctx, key, 3, 5*time.Second)
|
||||
if err != nil || !ok {
|
||||
t.Fatalf("call %d: AllowN = (%v, %v), want allowed", i, ok, err)
|
||||
}
|
||||
}
|
||||
ok, retry, err := r.AllowN(ctx, key, 3, 5*time.Second)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if ok {
|
||||
t.Fatal("the 4th call in a limit-3 window was allowed")
|
||||
}
|
||||
if retry <= 0 || retry > 5*time.Second {
|
||||
t.Fatalf("retryAfter = %s, want a positive value inside the window", retry)
|
||||
}
|
||||
}
|
||||
94
internal/web/templates/base.html
Normal file
94
internal/web/templates/base.html
Normal file
@ -0,0 +1,94 @@
|
||||
{{define "base.html"}}<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>hush</title>
|
||||
<!-- No external origins: no CDN, no font host, no analytics. A third-party
|
||||
script on this page could read the key out of the fragment, so the CSP
|
||||
forbids one existing. 'unsafe-inline' covers the inline script and style
|
||||
below, which are same-document and reviewed in this repo. -->
|
||||
<meta http-equiv="Content-Security-Policy"
|
||||
content="default-src 'none'; script-src 'unsafe-inline'; style-src 'unsafe-inline'; connect-src 'self'; form-action 'none'; base-uri 'none'; frame-ancestors 'none'">
|
||||
<meta name="referrer" content="no-referrer">
|
||||
<style>
|
||||
:root{color-scheme:dark;--bg:#0b0d10;--fg:#e7ebf0;--dim:#8b95a3;--line:#1e242c;--accent:#7fd1ae;--warn:#f0b866;--bad:#e8737d}
|
||||
*{box-sizing:border-box}
|
||||
body{margin:0;background:var(--bg);color:var(--fg);font:15px/1.55 ui-monospace,SFMono-Regular,Menlo,monospace;display:flex;min-height:100vh;align-items:center;justify-content:center;padding:24px}
|
||||
main{width:100%;max-width:620px}
|
||||
h1{font-size:20px;margin:0 0 4px;letter-spacing:.02em}
|
||||
h1 span{color:var(--accent)}
|
||||
p.lede{color:var(--dim);margin:0 0 24px;font-size:13px}
|
||||
label{display:block;font-size:12px;color:var(--dim);margin:16px 0 6px;text-transform:uppercase;letter-spacing:.08em}
|
||||
textarea,input,select,button{font:inherit;color:var(--fg);background:#12161b;border:1px solid var(--line);border-radius:6px;padding:10px 12px;width:100%}
|
||||
textarea{min-height:150px;resize:vertical}
|
||||
button{background:var(--accent);color:#06231a;border:0;font-weight:600;cursor:pointer;margin-top:20px;padding:12px}
|
||||
button:hover{filter:brightness(1.08)}
|
||||
button:disabled{opacity:.5;cursor:not-allowed;filter:none}
|
||||
button.secondary{background:#12161b;color:var(--fg);border:1px solid var(--line);font-weight:400}
|
||||
.row{display:flex;gap:12px}
|
||||
.row>*{flex:1}
|
||||
.out{margin-top:24px;padding:14px;border:1px solid var(--line);border-radius:6px;background:#0f1418;word-break:break-all;font-size:13px}
|
||||
.note{color:var(--dim);font-size:12px;margin-top:10px}
|
||||
.err{color:var(--bad)}
|
||||
.warn{color:var(--warn)}
|
||||
.ok{color:var(--accent)}
|
||||
.hide{display:none}
|
||||
footer{margin-top:28px;color:var(--dim);font-size:11px;border-top:1px solid var(--line);padding-top:12px}
|
||||
a{color:var(--dim)}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<main>
|
||||
{{template "content" .}}
|
||||
<footer>
|
||||
Encrypted in your browser. The key travels in the link's <code>#fragment</code>,
|
||||
which browsers never send to a server — hush stores ciphertext it cannot open.
|
||||
The link works once.
|
||||
</footer>
|
||||
</main>
|
||||
<script>
|
||||
// Shared crypto. AES-256-GCM via WebCrypto; the key is generated here, never
|
||||
// transmitted, and carried only in the URL fragment.
|
||||
//
|
||||
// b64u: base64url without padding, matching Go's base64.RawURLEncoding on the
|
||||
// server so the wire format has exactly one spelling.
|
||||
const b64u = {
|
||||
enc(bytes) {
|
||||
let s = ""; for (const b of bytes) s += String.fromCharCode(b);
|
||||
return btoa(s).replace(/\+/g, "-").replace(/\//g, "_").replace(/=+$/, "");
|
||||
},
|
||||
dec(str) {
|
||||
const p = str.replace(/-/g, "+").replace(/_/g, "/");
|
||||
const raw = atob(p + "=".repeat((4 - (p.length % 4)) % 4));
|
||||
const out = new Uint8Array(raw.length);
|
||||
for (let i = 0; i < raw.length; i++) out[i] = raw.charCodeAt(i);
|
||||
return out;
|
||||
},
|
||||
};
|
||||
|
||||
async function seal(plaintext) {
|
||||
const key = await crypto.subtle.generateKey({ name: "AES-GCM", length: 256 }, true, ["encrypt", "decrypt"]);
|
||||
// 96-bit nonce is the AES-GCM standard size and is generated per secret. It
|
||||
// is prepended to the ciphertext rather than sent separately so a stored blob
|
||||
// is self-contained.
|
||||
const nonce = crypto.getRandomValues(new Uint8Array(12));
|
||||
const body = new Uint8Array(await crypto.subtle.encrypt(
|
||||
{ name: "AES-GCM", iv: nonce }, key, new TextEncoder().encode(plaintext)));
|
||||
const blob = new Uint8Array(nonce.length + body.length);
|
||||
blob.set(nonce, 0); blob.set(body, nonce.length);
|
||||
const rawKey = new Uint8Array(await crypto.subtle.exportKey("raw", key));
|
||||
return { ciphertext: b64u.enc(blob), key: b64u.enc(rawKey) };
|
||||
}
|
||||
|
||||
async function open(ciphertext, keyStr) {
|
||||
const blob = b64u.dec(ciphertext);
|
||||
const key = await crypto.subtle.importKey("raw", b64u.dec(keyStr), "AES-GCM", false, ["decrypt"]);
|
||||
const plain = await crypto.subtle.decrypt(
|
||||
{ name: "AES-GCM", iv: blob.slice(0, 12) }, key, blob.slice(12));
|
||||
return new TextDecoder().decode(plain);
|
||||
}
|
||||
</script>
|
||||
{{template "script" .}}
|
||||
</body>
|
||||
</html>{{end}}
|
||||
118
internal/web/templates/create.html
Normal file
118
internal/web/templates/create.html
Normal file
@ -0,0 +1,118 @@
|
||||
{{define "content"}}
|
||||
<h1>hush<span>.</span></h1>
|
||||
<p class="lede">Paste a secret. Get a link that works once.</p>
|
||||
|
||||
<div id="form">
|
||||
<label for="secret">Secret</label>
|
||||
<textarea id="secret" autofocus autocomplete="off" spellcheck="false"
|
||||
placeholder="API key, password, connection string…"></textarea>
|
||||
|
||||
<div class="row">
|
||||
<div>
|
||||
<label for="ttl">Expires after</label>
|
||||
<select id="ttl">
|
||||
<option value="900">15 minutes</option>
|
||||
<option value="3600">1 hour</option>
|
||||
<option value="86400" selected>24 hours</option>
|
||||
<option value="259200">3 days</option>
|
||||
<option value="604800">7 days</option>
|
||||
</select>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<button id="go">Create link</button>
|
||||
<p class="note" id="msg"></p>
|
||||
</div>
|
||||
|
||||
<div id="result" class="hide">
|
||||
<label>Your one-time link</label>
|
||||
<div class="out" id="link"></div>
|
||||
<div class="row">
|
||||
<button id="copy">Copy link</button>
|
||||
<button id="again" class="secondary">Create another</button>
|
||||
</div>
|
||||
<p class="note warn">
|
||||
This is shown once and is not recoverable — hush cannot rebuild it, because
|
||||
the key it contains was never sent to the server. Copy it now.
|
||||
</p>
|
||||
</div>
|
||||
{{end}}
|
||||
|
||||
{{define "script"}}
|
||||
<script>
|
||||
const $ = (id) => document.getElementById(id);
|
||||
const MAX = {{.MaxCiphertextBytes}};
|
||||
|
||||
async function create() {
|
||||
const text = $("secret").value;
|
||||
if (!text) { return fail("Nothing to send."); }
|
||||
|
||||
$("go").disabled = true;
|
||||
$("msg").className = "note";
|
||||
$("msg").textContent = "Encrypting…";
|
||||
|
||||
let sealed;
|
||||
try {
|
||||
sealed = await seal(text);
|
||||
} catch (e) {
|
||||
return fail("Encryption failed in this browser: " + e.message);
|
||||
}
|
||||
|
||||
// Check the size of the CIPHERTEXT, which is what the server caps, so the
|
||||
// message names the same number the server would reject on.
|
||||
if (sealed.ciphertext.length > MAX) {
|
||||
return fail("Too large: " + sealed.ciphertext.length + " > " + MAX + " bytes encrypted.");
|
||||
}
|
||||
|
||||
let res, body;
|
||||
try {
|
||||
res = await fetch("/api/secrets", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ ciphertext: sealed.ciphertext, ttl_seconds: Number($("ttl").value) }),
|
||||
});
|
||||
body = await res.json();
|
||||
} catch (e) {
|
||||
return fail("Could not reach hush: " + e.message);
|
||||
}
|
||||
if (!res.ok) {
|
||||
return fail(body && body.error ? body.error.message : "Server refused the secret (" + res.status + ").");
|
||||
}
|
||||
|
||||
// The key goes in the fragment and ONLY in the fragment. Assembling the URL
|
||||
// here — not on the server — is what keeps the key client-side.
|
||||
const url = location.origin + "/s/" + body.id + "#" + sealed.key;
|
||||
$("link").textContent = url;
|
||||
$("form").classList.add("hide");
|
||||
$("result").classList.remove("hide");
|
||||
$("copy").focus();
|
||||
}
|
||||
|
||||
function fail(m) {
|
||||
$("go").disabled = false;
|
||||
$("msg").className = "note err";
|
||||
$("msg").textContent = m;
|
||||
}
|
||||
|
||||
$("go").addEventListener("click", create);
|
||||
$("secret").addEventListener("keydown", (e) => {
|
||||
if ((e.metaKey || e.ctrlKey) && e.key === "Enter") create();
|
||||
});
|
||||
$("copy").addEventListener("click", async () => {
|
||||
try {
|
||||
await navigator.clipboard.writeText($("link").textContent);
|
||||
$("copy").textContent = "Copied";
|
||||
setTimeout(() => ($("copy").textContent = "Copy link"), 1500);
|
||||
} catch {
|
||||
// Clipboard needs a permission this browser withheld; selecting the text
|
||||
// is a working fallback rather than a dead button.
|
||||
const r = document.createRange();
|
||||
r.selectNodeContents($("link"));
|
||||
getSelection().removeAllRanges();
|
||||
getSelection().addRange(r);
|
||||
$("copy").textContent = "Selected — press copy";
|
||||
}
|
||||
});
|
||||
$("again").addEventListener("click", () => location.assign("/"));
|
||||
</script>
|
||||
{{end}}
|
||||
118
internal/web/templates/reveal.html
Normal file
118
internal/web/templates/reveal.html
Normal file
@ -0,0 +1,118 @@
|
||||
{{define "content"}}
|
||||
<h1>hush<span>.</span></h1>
|
||||
<p class="lede">Someone sent you a secret. It can be opened once.</p>
|
||||
|
||||
<div id="gate">
|
||||
<p class="note">
|
||||
Nothing has been read yet. This page has not touched the secret — opening it
|
||||
is the button below, so a link preview in a chat app cannot consume it.
|
||||
</p>
|
||||
<button id="go">Reveal the secret</button>
|
||||
<p class="note" id="msg"></p>
|
||||
</div>
|
||||
|
||||
<div id="result" class="hide">
|
||||
<label>The secret</label>
|
||||
<div class="out" id="plain"></div>
|
||||
<div class="row">
|
||||
<button id="copy">Copy</button>
|
||||
</div>
|
||||
<p class="note warn">
|
||||
Destroyed. Reloading this page will not show it again — copy it now.
|
||||
</p>
|
||||
</div>
|
||||
{{end}}
|
||||
|
||||
{{define "script"}}
|
||||
<script>
|
||||
const $ = (id) => document.getElementById(id);
|
||||
|
||||
// The id comes from the path and the key from the fragment. Neither is
|
||||
// interpolated by the server, so this page renders identically for every secret
|
||||
// and reflects nothing.
|
||||
const id = location.pathname.replace(/^\/s\//, "");
|
||||
const key = location.hash.slice(1);
|
||||
|
||||
if (!key) {
|
||||
$("go").disabled = true;
|
||||
$("msg").className = "note err";
|
||||
$("msg").textContent =
|
||||
"This link is missing its key — the part after '#'. Chat apps and email " +
|
||||
"clients sometimes truncate it. Ask the sender for the full link; the " +
|
||||
"secret is intact and has not been opened.";
|
||||
}
|
||||
|
||||
async function reveal() {
|
||||
$("go").disabled = true;
|
||||
$("msg").className = "note";
|
||||
$("msg").textContent = "Opening…";
|
||||
|
||||
let res, body;
|
||||
try {
|
||||
res = await fetch("/api/secrets/" + encodeURIComponent(id) + "/reveal", { method: "POST" });
|
||||
body = await res.json();
|
||||
} catch (e) {
|
||||
// The secret is very likely consumed at this point, so do not offer a retry
|
||||
// that would report "gone" and read as a lie about what happened.
|
||||
$("msg").className = "note err";
|
||||
$("msg").textContent = "Could not reach hush: " + e.message +
|
||||
". If the request left your browser, the secret is already destroyed.";
|
||||
return;
|
||||
}
|
||||
|
||||
if (res.status === 410) {
|
||||
$("msg").className = "note err";
|
||||
$("msg").textContent =
|
||||
"Gone. This link was already opened, expired, or never existed. " +
|
||||
"If you have not opened it yourself, assume someone else did and ask the " +
|
||||
"sender to rotate the secret.";
|
||||
return;
|
||||
}
|
||||
if (!res.ok) {
|
||||
$("go").disabled = false;
|
||||
$("msg").className = "note err";
|
||||
$("msg").textContent = body && body.error ? body.error.message : "Server error (" + res.status + ").";
|
||||
return;
|
||||
}
|
||||
|
||||
let text;
|
||||
try {
|
||||
text = await open(body.ciphertext, key);
|
||||
} catch {
|
||||
// Decryption failed AFTER the server destroyed the ciphertext, so there is
|
||||
// nothing to retry. Say so, because the alternative is a user reloading
|
||||
// forever against a secret that no longer exists.
|
||||
$("msg").className = "note err";
|
||||
$("msg").textContent =
|
||||
"The key in this link does not open this secret, and the ciphertext has " +
|
||||
"now been destroyed. The link was probably altered in transit. Ask the " +
|
||||
"sender to create a new one.";
|
||||
return;
|
||||
}
|
||||
|
||||
$("plain").textContent = text;
|
||||
$("gate").classList.add("hide");
|
||||
$("result").classList.remove("hide");
|
||||
// Drop the key from the address bar so a screenshot, a shoulder-surfer or a
|
||||
// later copy of the URL does not carry it. The secret is already destroyed
|
||||
// server-side, so this only reduces incidental exposure.
|
||||
history.replaceState(null, "", location.pathname);
|
||||
$("copy").focus();
|
||||
}
|
||||
|
||||
$("go").addEventListener("click", reveal);
|
||||
$("copy").addEventListener("click", async () => {
|
||||
try {
|
||||
await navigator.clipboard.writeText($("plain").textContent);
|
||||
$("copy").textContent = "Copied";
|
||||
setTimeout(() => ($("copy").textContent = "Copy"), 1500);
|
||||
} catch {
|
||||
const r = document.createRange();
|
||||
r.selectNodeContents($("plain"));
|
||||
getSelection().removeAllRanges();
|
||||
getSelection().addRange(r);
|
||||
$("copy").textContent = "Selected — press copy";
|
||||
}
|
||||
});
|
||||
</script>
|
||||
{{end}}
|
||||
73
internal/web/web.go
Normal file
73
internal/web/web.go
Normal file
@ -0,0 +1,73 @@
|
||||
// Package web serves hush's two pages. Both are static: they read no storage,
|
||||
// so a link previewer fetching either one cannot destroy a secret.
|
||||
package web
|
||||
|
||||
import (
|
||||
"embed"
|
||||
"fmt"
|
||||
"html/template"
|
||||
"net/http"
|
||||
)
|
||||
|
||||
//go:embed templates/*.html
|
||||
var files embed.FS
|
||||
|
||||
// Pages renders the create and reveal pages. Templates are embedded, so the
|
||||
// container carries no template directory to go missing at runtime.
|
||||
type Pages struct {
|
||||
create *template.Template
|
||||
reveal *template.Template
|
||||
}
|
||||
|
||||
// Data is everything a page needs. The limits are passed through so the browser
|
||||
// enforces the same caps the server does and a user learns their secret is too
|
||||
// large before uploading it, not after.
|
||||
type Data struct {
|
||||
MaxCiphertextBytes int
|
||||
DefaultTTLSeconds int
|
||||
MinTTLSeconds int
|
||||
MaxTTLSeconds int
|
||||
}
|
||||
|
||||
// New parses the embedded templates. It fails at boot rather than on first
|
||||
// request: a template error is a build defect and should not wait for traffic
|
||||
// to surface.
|
||||
func New() (*Pages, error) {
|
||||
create, err := template.ParseFS(files, "templates/base.html", "templates/create.html")
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("parse create template: %w", err)
|
||||
}
|
||||
reveal, err := template.ParseFS(files, "templates/base.html", "templates/reveal.html")
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("parse reveal template: %w", err)
|
||||
}
|
||||
return &Pages{create: create, reveal: reveal}, nil
|
||||
}
|
||||
|
||||
// Create writes the create page.
|
||||
func (p *Pages) Create(w http.ResponseWriter, d Data) error {
|
||||
return render(w, p.create, d)
|
||||
}
|
||||
|
||||
// Reveal writes the reveal page.
|
||||
//
|
||||
// The secret id is NOT passed in and is NOT interpolated into the HTML. The
|
||||
// page reads it from location.pathname in the browser, alongside the key it
|
||||
// reads from location.hash. That keeps the template free of any value that
|
||||
// could be reflected, and means this handler needs no escaping decisions about
|
||||
// a capability.
|
||||
func (p *Pages) Reveal(w http.ResponseWriter, d Data) error {
|
||||
return render(w, p.reveal, d)
|
||||
}
|
||||
|
||||
func render(w http.ResponseWriter, t *template.Template, d Data) error {
|
||||
// no-store on both pages: a cached create page is harmless, but a cached
|
||||
// reveal page in a shared proxy would be a copy of a one-time URL.
|
||||
w.Header().Set("Cache-Control", "no-store, max-age=0")
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
// Referrer-Policy is load-bearing here, not boilerplate: without it a click
|
||||
// on any link from the reveal page could send the full URL — including the
|
||||
// fragment-adjacent path — to a third party.
|
||||
w.Header().Set("Referrer-Policy", "no-referrer")
|
||||
return t.ExecuteTemplate(w, "base.html", d)
|
||||
}
|
||||
94
scripts/smoke.sh
Executable file
94
scripts/smoke.sh
Executable file
@ -0,0 +1,94 @@
|
||||
#!/usr/bin/env bash
|
||||
# End-to-end proof against a RUNNING hushd, doing the real client-side crypto.
|
||||
#
|
||||
# This is not a mock: it generates an AES-256-GCM key, encrypts a plaintext,
|
||||
# posts only the ciphertext, reveals it once, decrypts it, and compares. Then it
|
||||
# checks the three properties the design rests on:
|
||||
#
|
||||
# 1. a second reveal is 410 gone
|
||||
# 2. GET on the reveal page does NOT consume the secret (link previewers)
|
||||
# 3. a malformed id and a missing id are indistinguishable
|
||||
#
|
||||
# Usage: BASE=http://127.0.0.1:18500 ./scripts/smoke.sh
|
||||
set -euo pipefail
|
||||
|
||||
BASE="${BASE:-http://127.0.0.1:18500}"
|
||||
PLAINTEXT="${PLAINTEXT:-hunter2-$(date +%s)-$RANDOM}"
|
||||
|
||||
pass() { printf ' \033[32mok\033[0m %s\n' "$1"; }
|
||||
fail() { printf ' \033[31mFAIL\033[0m %s\n' "$1"; exit 1; }
|
||||
|
||||
command -v python3 >/dev/null || { echo "python3 is required for the crypto half"; exit 1; }
|
||||
|
||||
echo "hush smoke against $BASE"
|
||||
|
||||
# --- encrypt exactly as the browser does -----------------------------------
|
||||
# AES-256-GCM, 96-bit nonce prepended, base64url unpadded. Same wire format as
|
||||
# templates/base.html, which is what makes this a real client.
|
||||
read -r CIPHERTEXT KEY <<<"$(python3 - "$PLAINTEXT" <<'PY'
|
||||
import base64, os, sys
|
||||
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
|
||||
key = AESGCM.generate_key(bit_length=256)
|
||||
nonce = os.urandom(12)
|
||||
blob = nonce + AESGCM(key).encrypt(nonce, sys.argv[1].encode(), None)
|
||||
b = lambda x: base64.urlsafe_b64encode(x).decode().rstrip("=")
|
||||
print(b(blob), b(key))
|
||||
PY
|
||||
)"
|
||||
[ -n "$CIPHERTEXT" ] || fail "could not encrypt (is 'cryptography' installed? pip install cryptography)"
|
||||
pass "encrypted client-side (${#CIPHERTEXT} bytes of ciphertext)"
|
||||
|
||||
# --- create ----------------------------------------------------------------
|
||||
CREATE=$(curl -sS -X POST "$BASE/api/secrets" -H 'Content-Type: application/json' \
|
||||
-d "{\"ciphertext\":\"$CIPHERTEXT\",\"ttl_seconds\":900}")
|
||||
ID=$(printf '%s' "$CREATE" | python3 -c 'import json,sys; print(json.load(sys.stdin).get("id",""))')
|
||||
[ -n "$ID" ] || fail "create returned no id: $CREATE"
|
||||
pass "created — id is ${#ID} chars"
|
||||
|
||||
# --- the previewer property, checked BEFORE revealing ----------------------
|
||||
for _ in 1 2 3; do
|
||||
code=$(curl -sS -o /dev/null -w '%{http_code}' "$BASE/s/$ID")
|
||||
[ "$code" = "200" ] || fail "GET /s/{id} returned $code"
|
||||
done
|
||||
pass "GET on the reveal page x3 — Slack/Outlook previews are harmless"
|
||||
|
||||
# --- reveal once and decrypt ----------------------------------------------
|
||||
REVEAL=$(curl -sS -X POST "$BASE/api/secrets/$ID/reveal")
|
||||
GOT_CT=$(printf '%s' "$REVEAL" | python3 -c 'import json,sys; print(json.load(sys.stdin).get("ciphertext",""))')
|
||||
[ -n "$GOT_CT" ] || fail "reveal returned no ciphertext: $REVEAL"
|
||||
|
||||
DECRYPTED=$(python3 - "$GOT_CT" "$KEY" <<'PY'
|
||||
import base64, sys
|
||||
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
|
||||
u = lambda s: base64.urlsafe_b64decode(s + "=" * (-len(s) % 4))
|
||||
blob, key = u(sys.argv[1]), u(sys.argv[2])
|
||||
sys.stdout.write(AESGCM(key).decrypt(blob[:12], blob[12:], None).decode())
|
||||
PY
|
||||
)
|
||||
[ "$DECRYPTED" = "$PLAINTEXT" ] || fail "decrypted to '$DECRYPTED', expected '$PLAINTEXT'"
|
||||
pass "revealed and decrypted — round trip matches"
|
||||
|
||||
# --- and it is gone --------------------------------------------------------
|
||||
code=$(curl -sS -o /tmp/hush-gone.$$ -w '%{http_code}' -X POST "$BASE/api/secrets/$ID/reveal")
|
||||
[ "$code" = "410" ] || fail "second reveal returned $code, want 410"
|
||||
GONE_BODY=$(cat /tmp/hush-gone.$$); rm -f /tmp/hush-gone.$$
|
||||
pass "second reveal is 410 gone"
|
||||
|
||||
# --- missing and malformed are one response --------------------------------
|
||||
MISSING=$(curl -sS -X POST "$BASE/api/secrets/$(python3 -c 'print("A"*43)')/reveal")
|
||||
MALFORMED=$(curl -sS -X POST "$BASE/api/secrets/not-an-id/reveal")
|
||||
for body in "$MISSING" "$MALFORMED"; do
|
||||
norm=$(printf '%s' "$body" | python3 -c 'import json,sys; e=json.load(sys.stdin)["error"]; print(e["code"], e["message"])')
|
||||
gone=$(printf '%s' "$GONE_BODY" | python3 -c 'import json,sys; e=json.load(sys.stdin)["error"]; print(e["code"], e["message"])')
|
||||
[ "$norm" = "$gone" ] || fail "responses differ between causes: '$norm' vs '$gone'"
|
||||
done
|
||||
pass "revealed, expired, missing and malformed are one indistinguishable response"
|
||||
|
||||
# --- the server must refuse plaintext -------------------------------------
|
||||
code=$(curl -sS -o /dev/null -w '%{http_code}' -X POST "$BASE/api/secrets" \
|
||||
-H 'Content-Type: application/json' -d '{"secret":"hunter2"}')
|
||||
[ "$code" != "201" ] || fail "the server ACCEPTED a plaintext field — the zero-knowledge claim is broken"
|
||||
pass "a plaintext field is refused ($code)"
|
||||
|
||||
echo
|
||||
printf '\033[32mall smoke checks passed\033[0m\n'
|
||||
20
vendor/github.com/beorn7/perks/LICENSE
generated
vendored
Normal file
20
vendor/github.com/beorn7/perks/LICENSE
generated
vendored
Normal file
@ -0,0 +1,20 @@
|
||||
Copyright (C) 2013 Blake Mizerany
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining
|
||||
a copy of this software and associated documentation files (the
|
||||
"Software"), to deal in the Software without restriction, including
|
||||
without limitation the rights to use, copy, modify, merge, publish,
|
||||
distribute, sublicense, and/or sell copies of the Software, and to
|
||||
permit persons to whom the Software is furnished to do so, subject to
|
||||
the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be
|
||||
included in all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
||||
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
||||
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
||||
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
2388
vendor/github.com/beorn7/perks/quantile/exampledata.txt
generated
vendored
Normal file
2388
vendor/github.com/beorn7/perks/quantile/exampledata.txt
generated
vendored
Normal file
File diff suppressed because it is too large
Load Diff
316
vendor/github.com/beorn7/perks/quantile/stream.go
generated
vendored
Normal file
316
vendor/github.com/beorn7/perks/quantile/stream.go
generated
vendored
Normal file
@ -0,0 +1,316 @@
|
||||
// Package quantile computes approximate quantiles over an unbounded data
|
||||
// stream within low memory and CPU bounds.
|
||||
//
|
||||
// A small amount of accuracy is traded to achieve the above properties.
|
||||
//
|
||||
// Multiple streams can be merged before calling Query to generate a single set
|
||||
// of results. This is meaningful when the streams represent the same type of
|
||||
// data. See Merge and Samples.
|
||||
//
|
||||
// For more detailed information about the algorithm used, see:
|
||||
//
|
||||
// Effective Computation of Biased Quantiles over Data Streams
|
||||
//
|
||||
// http://www.cs.rutgers.edu/~muthu/bquant.pdf
|
||||
package quantile
|
||||
|
||||
import (
|
||||
"math"
|
||||
"sort"
|
||||
)
|
||||
|
||||
// Sample holds an observed value and meta information for compression. JSON
|
||||
// tags have been added for convenience.
|
||||
type Sample struct {
|
||||
Value float64 `json:",string"`
|
||||
Width float64 `json:",string"`
|
||||
Delta float64 `json:",string"`
|
||||
}
|
||||
|
||||
// Samples represents a slice of samples. It implements sort.Interface.
|
||||
type Samples []Sample
|
||||
|
||||
func (a Samples) Len() int { return len(a) }
|
||||
func (a Samples) Less(i, j int) bool { return a[i].Value < a[j].Value }
|
||||
func (a Samples) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
|
||||
|
||||
type invariant func(s *stream, r float64) float64
|
||||
|
||||
// NewLowBiased returns an initialized Stream for low-biased quantiles
|
||||
// (e.g. 0.01, 0.1, 0.5) where the needed quantiles are not known a priori, but
|
||||
// error guarantees can still be given even for the lower ranks of the data
|
||||
// distribution.
|
||||
//
|
||||
// The provided epsilon is a relative error, i.e. the true quantile of a value
|
||||
// returned by a query is guaranteed to be within (1±Epsilon)*Quantile.
|
||||
//
|
||||
// See http://www.cs.rutgers.edu/~muthu/bquant.pdf for time, space, and error
|
||||
// properties.
|
||||
func NewLowBiased(epsilon float64) *Stream {
|
||||
ƒ := func(s *stream, r float64) float64 {
|
||||
return 2 * epsilon * r
|
||||
}
|
||||
return newStream(ƒ)
|
||||
}
|
||||
|
||||
// NewHighBiased returns an initialized Stream for high-biased quantiles
|
||||
// (e.g. 0.01, 0.1, 0.5) where the needed quantiles are not known a priori, but
|
||||
// error guarantees can still be given even for the higher ranks of the data
|
||||
// distribution.
|
||||
//
|
||||
// The provided epsilon is a relative error, i.e. the true quantile of a value
|
||||
// returned by a query is guaranteed to be within 1-(1±Epsilon)*(1-Quantile).
|
||||
//
|
||||
// See http://www.cs.rutgers.edu/~muthu/bquant.pdf for time, space, and error
|
||||
// properties.
|
||||
func NewHighBiased(epsilon float64) *Stream {
|
||||
ƒ := func(s *stream, r float64) float64 {
|
||||
return 2 * epsilon * (s.n - r)
|
||||
}
|
||||
return newStream(ƒ)
|
||||
}
|
||||
|
||||
// NewTargeted returns an initialized Stream concerned with a particular set of
|
||||
// quantile values that are supplied a priori. Knowing these a priori reduces
|
||||
// space and computation time. The targets map maps the desired quantiles to
|
||||
// their absolute errors, i.e. the true quantile of a value returned by a query
|
||||
// is guaranteed to be within (Quantile±Epsilon).
|
||||
//
|
||||
// See http://www.cs.rutgers.edu/~muthu/bquant.pdf for time, space, and error properties.
|
||||
func NewTargeted(targetMap map[float64]float64) *Stream {
|
||||
// Convert map to slice to avoid slow iterations on a map.
|
||||
// ƒ is called on the hot path, so converting the map to a slice
|
||||
// beforehand results in significant CPU savings.
|
||||
targets := targetMapToSlice(targetMap)
|
||||
|
||||
ƒ := func(s *stream, r float64) float64 {
|
||||
var m = math.MaxFloat64
|
||||
var f float64
|
||||
for _, t := range targets {
|
||||
if t.quantile*s.n <= r {
|
||||
f = (2 * t.epsilon * r) / t.quantile
|
||||
} else {
|
||||
f = (2 * t.epsilon * (s.n - r)) / (1 - t.quantile)
|
||||
}
|
||||
if f < m {
|
||||
m = f
|
||||
}
|
||||
}
|
||||
return m
|
||||
}
|
||||
return newStream(ƒ)
|
||||
}
|
||||
|
||||
type target struct {
|
||||
quantile float64
|
||||
epsilon float64
|
||||
}
|
||||
|
||||
func targetMapToSlice(targetMap map[float64]float64) []target {
|
||||
targets := make([]target, 0, len(targetMap))
|
||||
|
||||
for quantile, epsilon := range targetMap {
|
||||
t := target{
|
||||
quantile: quantile,
|
||||
epsilon: epsilon,
|
||||
}
|
||||
targets = append(targets, t)
|
||||
}
|
||||
|
||||
return targets
|
||||
}
|
||||
|
||||
// Stream computes quantiles for a stream of float64s. It is not thread-safe by
|
||||
// design. Take care when using across multiple goroutines.
|
||||
type Stream struct {
|
||||
*stream
|
||||
b Samples
|
||||
sorted bool
|
||||
}
|
||||
|
||||
func newStream(ƒ invariant) *Stream {
|
||||
x := &stream{ƒ: ƒ}
|
||||
return &Stream{x, make(Samples, 0, 500), true}
|
||||
}
|
||||
|
||||
// Insert inserts v into the stream.
|
||||
func (s *Stream) Insert(v float64) {
|
||||
s.insert(Sample{Value: v, Width: 1})
|
||||
}
|
||||
|
||||
func (s *Stream) insert(sample Sample) {
|
||||
s.b = append(s.b, sample)
|
||||
s.sorted = false
|
||||
if len(s.b) == cap(s.b) {
|
||||
s.flush()
|
||||
}
|
||||
}
|
||||
|
||||
// Query returns the computed qth percentiles value. If s was created with
|
||||
// NewTargeted, and q is not in the set of quantiles provided a priori, Query
|
||||
// will return an unspecified result.
|
||||
func (s *Stream) Query(q float64) float64 {
|
||||
if !s.flushed() {
|
||||
// Fast path when there hasn't been enough data for a flush;
|
||||
// this also yields better accuracy for small sets of data.
|
||||
l := len(s.b)
|
||||
if l == 0 {
|
||||
return 0
|
||||
}
|
||||
i := int(math.Ceil(float64(l) * q))
|
||||
if i > 0 {
|
||||
i -= 1
|
||||
}
|
||||
s.maybeSort()
|
||||
return s.b[i].Value
|
||||
}
|
||||
s.flush()
|
||||
return s.stream.query(q)
|
||||
}
|
||||
|
||||
// Merge merges samples into the underlying streams samples. This is handy when
|
||||
// merging multiple streams from separate threads, database shards, etc.
|
||||
//
|
||||
// ATTENTION: This method is broken and does not yield correct results. The
|
||||
// underlying algorithm is not capable of merging streams correctly.
|
||||
func (s *Stream) Merge(samples Samples) {
|
||||
sort.Sort(samples)
|
||||
s.stream.merge(samples)
|
||||
}
|
||||
|
||||
// Reset reinitializes and clears the list reusing the samples buffer memory.
|
||||
func (s *Stream) Reset() {
|
||||
s.stream.reset()
|
||||
s.b = s.b[:0]
|
||||
}
|
||||
|
||||
// Samples returns stream samples held by s.
|
||||
func (s *Stream) Samples() Samples {
|
||||
if !s.flushed() {
|
||||
return s.b
|
||||
}
|
||||
s.flush()
|
||||
return s.stream.samples()
|
||||
}
|
||||
|
||||
// Count returns the total number of samples observed in the stream
|
||||
// since initialization.
|
||||
func (s *Stream) Count() int {
|
||||
return len(s.b) + s.stream.count()
|
||||
}
|
||||
|
||||
func (s *Stream) flush() {
|
||||
s.maybeSort()
|
||||
s.stream.merge(s.b)
|
||||
s.b = s.b[:0]
|
||||
}
|
||||
|
||||
func (s *Stream) maybeSort() {
|
||||
if !s.sorted {
|
||||
s.sorted = true
|
||||
sort.Sort(s.b)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Stream) flushed() bool {
|
||||
return len(s.stream.l) > 0
|
||||
}
|
||||
|
||||
type stream struct {
|
||||
n float64
|
||||
l []Sample
|
||||
ƒ invariant
|
||||
}
|
||||
|
||||
func (s *stream) reset() {
|
||||
s.l = s.l[:0]
|
||||
s.n = 0
|
||||
}
|
||||
|
||||
func (s *stream) insert(v float64) {
|
||||
s.merge(Samples{{v, 1, 0}})
|
||||
}
|
||||
|
||||
func (s *stream) merge(samples Samples) {
|
||||
// TODO(beorn7): This tries to merge not only individual samples, but
|
||||
// whole summaries. The paper doesn't mention merging summaries at
|
||||
// all. Unittests show that the merging is inaccurate. Find out how to
|
||||
// do merges properly.
|
||||
var r float64
|
||||
i := 0
|
||||
for _, sample := range samples {
|
||||
for ; i < len(s.l); i++ {
|
||||
c := s.l[i]
|
||||
if c.Value > sample.Value {
|
||||
// Insert at position i.
|
||||
s.l = append(s.l, Sample{})
|
||||
copy(s.l[i+1:], s.l[i:])
|
||||
s.l[i] = Sample{
|
||||
sample.Value,
|
||||
sample.Width,
|
||||
math.Max(sample.Delta, math.Floor(s.ƒ(s, r))-1),
|
||||
// TODO(beorn7): How to calculate delta correctly?
|
||||
}
|
||||
i++
|
||||
goto inserted
|
||||
}
|
||||
r += c.Width
|
||||
}
|
||||
s.l = append(s.l, Sample{sample.Value, sample.Width, 0})
|
||||
i++
|
||||
inserted:
|
||||
s.n += sample.Width
|
||||
r += sample.Width
|
||||
}
|
||||
s.compress()
|
||||
}
|
||||
|
||||
func (s *stream) count() int {
|
||||
return int(s.n)
|
||||
}
|
||||
|
||||
func (s *stream) query(q float64) float64 {
|
||||
t := math.Ceil(q * s.n)
|
||||
t += math.Ceil(s.ƒ(s, t) / 2)
|
||||
p := s.l[0]
|
||||
var r float64
|
||||
for _, c := range s.l[1:] {
|
||||
r += p.Width
|
||||
if r+c.Width+c.Delta > t {
|
||||
return p.Value
|
||||
}
|
||||
p = c
|
||||
}
|
||||
return p.Value
|
||||
}
|
||||
|
||||
func (s *stream) compress() {
|
||||
if len(s.l) < 2 {
|
||||
return
|
||||
}
|
||||
x := s.l[len(s.l)-1]
|
||||
xi := len(s.l) - 1
|
||||
r := s.n - 1 - x.Width
|
||||
|
||||
for i := len(s.l) - 2; i >= 0; i-- {
|
||||
c := s.l[i]
|
||||
if c.Width+x.Width+x.Delta <= s.ƒ(s, r) {
|
||||
x.Width += c.Width
|
||||
s.l[xi] = x
|
||||
// Remove element at i.
|
||||
copy(s.l[i:], s.l[i+1:])
|
||||
s.l = s.l[:len(s.l)-1]
|
||||
xi -= 1
|
||||
} else {
|
||||
x = c
|
||||
xi = i
|
||||
}
|
||||
r -= c.Width
|
||||
}
|
||||
}
|
||||
|
||||
func (s *stream) samples() Samples {
|
||||
samples := make(Samples, len(s.l))
|
||||
copy(samples, s.l)
|
||||
return samples
|
||||
}
|
||||
22
vendor/github.com/cespare/xxhash/v2/LICENSE.txt
generated
vendored
Normal file
22
vendor/github.com/cespare/xxhash/v2/LICENSE.txt
generated
vendored
Normal file
@ -0,0 +1,22 @@
|
||||
Copyright (c) 2016 Caleb Spare
|
||||
|
||||
MIT License
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining
|
||||
a copy of this software and associated documentation files (the
|
||||
"Software"), to deal in the Software without restriction, including
|
||||
without limitation the rights to use, copy, modify, merge, publish,
|
||||
distribute, sublicense, and/or sell copies of the Software, and to
|
||||
permit persons to whom the Software is furnished to do so, subject to
|
||||
the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be
|
||||
included in all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
||||
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
||||
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
||||
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
74
vendor/github.com/cespare/xxhash/v2/README.md
generated
vendored
Normal file
74
vendor/github.com/cespare/xxhash/v2/README.md
generated
vendored
Normal file
@ -0,0 +1,74 @@
|
||||
# xxhash
|
||||
|
||||
[](https://pkg.go.dev/github.com/cespare/xxhash/v2)
|
||||
[](https://github.com/cespare/xxhash/actions/workflows/test.yml)
|
||||
|
||||
xxhash is a Go implementation of the 64-bit [xxHash] algorithm, XXH64. This is a
|
||||
high-quality hashing algorithm that is much faster than anything in the Go
|
||||
standard library.
|
||||
|
||||
This package provides a straightforward API:
|
||||
|
||||
```
|
||||
func Sum64(b []byte) uint64
|
||||
func Sum64String(s string) uint64
|
||||
type Digest struct{ ... }
|
||||
func New() *Digest
|
||||
```
|
||||
|
||||
The `Digest` type implements hash.Hash64. Its key methods are:
|
||||
|
||||
```
|
||||
func (*Digest) Write([]byte) (int, error)
|
||||
func (*Digest) WriteString(string) (int, error)
|
||||
func (*Digest) Sum64() uint64
|
||||
```
|
||||
|
||||
The package is written with optimized pure Go and also contains even faster
|
||||
assembly implementations for amd64 and arm64. If desired, the `purego` build tag
|
||||
opts into using the Go code even on those architectures.
|
||||
|
||||
[xxHash]: http://cyan4973.github.io/xxHash/
|
||||
|
||||
## Compatibility
|
||||
|
||||
This package is in a module and the latest code is in version 2 of the module.
|
||||
You need a version of Go with at least "minimal module compatibility" to use
|
||||
github.com/cespare/xxhash/v2:
|
||||
|
||||
* 1.9.7+ for Go 1.9
|
||||
* 1.10.3+ for Go 1.10
|
||||
* Go 1.11 or later
|
||||
|
||||
I recommend using the latest release of Go.
|
||||
|
||||
## Benchmarks
|
||||
|
||||
Here are some quick benchmarks comparing the pure-Go and assembly
|
||||
implementations of Sum64.
|
||||
|
||||
| input size | purego | asm |
|
||||
| ---------- | --------- | --------- |
|
||||
| 4 B | 1.3 GB/s | 1.2 GB/s |
|
||||
| 16 B | 2.9 GB/s | 3.5 GB/s |
|
||||
| 100 B | 6.9 GB/s | 8.1 GB/s |
|
||||
| 4 KB | 11.7 GB/s | 16.7 GB/s |
|
||||
| 10 MB | 12.0 GB/s | 17.3 GB/s |
|
||||
|
||||
These numbers were generated on Ubuntu 20.04 with an Intel Xeon Platinum 8252C
|
||||
CPU using the following commands under Go 1.19.2:
|
||||
|
||||
```
|
||||
benchstat <(go test -tags purego -benchtime 500ms -count 15 -bench 'Sum64$')
|
||||
benchstat <(go test -benchtime 500ms -count 15 -bench 'Sum64$')
|
||||
```
|
||||
|
||||
## Projects using this package
|
||||
|
||||
- [InfluxDB](https://github.com/influxdata/influxdb)
|
||||
- [Prometheus](https://github.com/prometheus/prometheus)
|
||||
- [VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics)
|
||||
- [FreeCache](https://github.com/coocood/freecache)
|
||||
- [FastCache](https://github.com/VictoriaMetrics/fastcache)
|
||||
- [Ristretto](https://github.com/dgraph-io/ristretto)
|
||||
- [Badger](https://github.com/dgraph-io/badger)
|
||||
10
vendor/github.com/cespare/xxhash/v2/testall.sh
generated
vendored
Normal file
10
vendor/github.com/cespare/xxhash/v2/testall.sh
generated
vendored
Normal file
@ -0,0 +1,10 @@
|
||||
#!/bin/bash
|
||||
set -eu -o pipefail
|
||||
|
||||
# Small convenience script for running the tests with various combinations of
|
||||
# arch/tags. This assumes we're running on amd64 and have qemu available.
|
||||
|
||||
go test ./...
|
||||
go test -tags purego ./...
|
||||
GOARCH=arm64 go test
|
||||
GOARCH=arm64 go test -tags purego
|
||||
243
vendor/github.com/cespare/xxhash/v2/xxhash.go
generated
vendored
Normal file
243
vendor/github.com/cespare/xxhash/v2/xxhash.go
generated
vendored
Normal file
@ -0,0 +1,243 @@
|
||||
// Package xxhash implements the 64-bit variant of xxHash (XXH64) as described
|
||||
// at http://cyan4973.github.io/xxHash/.
|
||||
package xxhash
|
||||
|
||||
import (
|
||||
"encoding/binary"
|
||||
"errors"
|
||||
"math/bits"
|
||||
)
|
||||
|
||||
const (
|
||||
prime1 uint64 = 11400714785074694791
|
||||
prime2 uint64 = 14029467366897019727
|
||||
prime3 uint64 = 1609587929392839161
|
||||
prime4 uint64 = 9650029242287828579
|
||||
prime5 uint64 = 2870177450012600261
|
||||
)
|
||||
|
||||
// Store the primes in an array as well.
|
||||
//
|
||||
// The consts are used when possible in Go code to avoid MOVs but we need a
|
||||
// contiguous array for the assembly code.
|
||||
var primes = [...]uint64{prime1, prime2, prime3, prime4, prime5}
|
||||
|
||||
// Digest implements hash.Hash64.
|
||||
//
|
||||
// Note that a zero-valued Digest is not ready to receive writes.
|
||||
// Call Reset or create a Digest using New before calling other methods.
|
||||
type Digest struct {
|
||||
v1 uint64
|
||||
v2 uint64
|
||||
v3 uint64
|
||||
v4 uint64
|
||||
total uint64
|
||||
mem [32]byte
|
||||
n int // how much of mem is used
|
||||
}
|
||||
|
||||
// New creates a new Digest with a zero seed.
|
||||
func New() *Digest {
|
||||
return NewWithSeed(0)
|
||||
}
|
||||
|
||||
// NewWithSeed creates a new Digest with the given seed.
|
||||
func NewWithSeed(seed uint64) *Digest {
|
||||
var d Digest
|
||||
d.ResetWithSeed(seed)
|
||||
return &d
|
||||
}
|
||||
|
||||
// Reset clears the Digest's state so that it can be reused.
|
||||
// It uses a seed value of zero.
|
||||
func (d *Digest) Reset() {
|
||||
d.ResetWithSeed(0)
|
||||
}
|
||||
|
||||
// ResetWithSeed clears the Digest's state so that it can be reused.
|
||||
// It uses the given seed to initialize the state.
|
||||
func (d *Digest) ResetWithSeed(seed uint64) {
|
||||
d.v1 = seed + prime1 + prime2
|
||||
d.v2 = seed + prime2
|
||||
d.v3 = seed
|
||||
d.v4 = seed - prime1
|
||||
d.total = 0
|
||||
d.n = 0
|
||||
}
|
||||
|
||||
// Size always returns 8 bytes.
|
||||
func (d *Digest) Size() int { return 8 }
|
||||
|
||||
// BlockSize always returns 32 bytes.
|
||||
func (d *Digest) BlockSize() int { return 32 }
|
||||
|
||||
// Write adds more data to d. It always returns len(b), nil.
|
||||
func (d *Digest) Write(b []byte) (n int, err error) {
|
||||
n = len(b)
|
||||
d.total += uint64(n)
|
||||
|
||||
memleft := d.mem[d.n&(len(d.mem)-1):]
|
||||
|
||||
if d.n+n < 32 {
|
||||
// This new data doesn't even fill the current block.
|
||||
copy(memleft, b)
|
||||
d.n += n
|
||||
return
|
||||
}
|
||||
|
||||
if d.n > 0 {
|
||||
// Finish off the partial block.
|
||||
c := copy(memleft, b)
|
||||
d.v1 = round(d.v1, u64(d.mem[0:8]))
|
||||
d.v2 = round(d.v2, u64(d.mem[8:16]))
|
||||
d.v3 = round(d.v3, u64(d.mem[16:24]))
|
||||
d.v4 = round(d.v4, u64(d.mem[24:32]))
|
||||
b = b[c:]
|
||||
d.n = 0
|
||||
}
|
||||
|
||||
if len(b) >= 32 {
|
||||
// One or more full blocks left.
|
||||
nw := writeBlocks(d, b)
|
||||
b = b[nw:]
|
||||
}
|
||||
|
||||
// Store any remaining partial block.
|
||||
copy(d.mem[:], b)
|
||||
d.n = len(b)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
// Sum appends the current hash to b and returns the resulting slice.
|
||||
func (d *Digest) Sum(b []byte) []byte {
|
||||
s := d.Sum64()
|
||||
return append(
|
||||
b,
|
||||
byte(s>>56),
|
||||
byte(s>>48),
|
||||
byte(s>>40),
|
||||
byte(s>>32),
|
||||
byte(s>>24),
|
||||
byte(s>>16),
|
||||
byte(s>>8),
|
||||
byte(s),
|
||||
)
|
||||
}
|
||||
|
||||
// Sum64 returns the current hash.
|
||||
func (d *Digest) Sum64() uint64 {
|
||||
var h uint64
|
||||
|
||||
if d.total >= 32 {
|
||||
v1, v2, v3, v4 := d.v1, d.v2, d.v3, d.v4
|
||||
h = rol1(v1) + rol7(v2) + rol12(v3) + rol18(v4)
|
||||
h = mergeRound(h, v1)
|
||||
h = mergeRound(h, v2)
|
||||
h = mergeRound(h, v3)
|
||||
h = mergeRound(h, v4)
|
||||
} else {
|
||||
h = d.v3 + prime5
|
||||
}
|
||||
|
||||
h += d.total
|
||||
|
||||
b := d.mem[:d.n&(len(d.mem)-1)]
|
||||
for ; len(b) >= 8; b = b[8:] {
|
||||
k1 := round(0, u64(b[:8]))
|
||||
h ^= k1
|
||||
h = rol27(h)*prime1 + prime4
|
||||
}
|
||||
if len(b) >= 4 {
|
||||
h ^= uint64(u32(b[:4])) * prime1
|
||||
h = rol23(h)*prime2 + prime3
|
||||
b = b[4:]
|
||||
}
|
||||
for ; len(b) > 0; b = b[1:] {
|
||||
h ^= uint64(b[0]) * prime5
|
||||
h = rol11(h) * prime1
|
||||
}
|
||||
|
||||
h ^= h >> 33
|
||||
h *= prime2
|
||||
h ^= h >> 29
|
||||
h *= prime3
|
||||
h ^= h >> 32
|
||||
|
||||
return h
|
||||
}
|
||||
|
||||
const (
|
||||
magic = "xxh\x06"
|
||||
marshaledSize = len(magic) + 8*5 + 32
|
||||
)
|
||||
|
||||
// MarshalBinary implements the encoding.BinaryMarshaler interface.
|
||||
func (d *Digest) MarshalBinary() ([]byte, error) {
|
||||
b := make([]byte, 0, marshaledSize)
|
||||
b = append(b, magic...)
|
||||
b = appendUint64(b, d.v1)
|
||||
b = appendUint64(b, d.v2)
|
||||
b = appendUint64(b, d.v3)
|
||||
b = appendUint64(b, d.v4)
|
||||
b = appendUint64(b, d.total)
|
||||
b = append(b, d.mem[:d.n]...)
|
||||
b = b[:len(b)+len(d.mem)-d.n]
|
||||
return b, nil
|
||||
}
|
||||
|
||||
// UnmarshalBinary implements the encoding.BinaryUnmarshaler interface.
|
||||
func (d *Digest) UnmarshalBinary(b []byte) error {
|
||||
if len(b) < len(magic) || string(b[:len(magic)]) != magic {
|
||||
return errors.New("xxhash: invalid hash state identifier")
|
||||
}
|
||||
if len(b) != marshaledSize {
|
||||
return errors.New("xxhash: invalid hash state size")
|
||||
}
|
||||
b = b[len(magic):]
|
||||
b, d.v1 = consumeUint64(b)
|
||||
b, d.v2 = consumeUint64(b)
|
||||
b, d.v3 = consumeUint64(b)
|
||||
b, d.v4 = consumeUint64(b)
|
||||
b, d.total = consumeUint64(b)
|
||||
copy(d.mem[:], b)
|
||||
d.n = int(d.total % uint64(len(d.mem)))
|
||||
return nil
|
||||
}
|
||||
|
||||
func appendUint64(b []byte, x uint64) []byte {
|
||||
var a [8]byte
|
||||
binary.LittleEndian.PutUint64(a[:], x)
|
||||
return append(b, a[:]...)
|
||||
}
|
||||
|
||||
func consumeUint64(b []byte) ([]byte, uint64) {
|
||||
x := u64(b)
|
||||
return b[8:], x
|
||||
}
|
||||
|
||||
func u64(b []byte) uint64 { return binary.LittleEndian.Uint64(b) }
|
||||
func u32(b []byte) uint32 { return binary.LittleEndian.Uint32(b) }
|
||||
|
||||
func round(acc, input uint64) uint64 {
|
||||
acc += input * prime2
|
||||
acc = rol31(acc)
|
||||
acc *= prime1
|
||||
return acc
|
||||
}
|
||||
|
||||
func mergeRound(acc, val uint64) uint64 {
|
||||
val = round(0, val)
|
||||
acc ^= val
|
||||
acc = acc*prime1 + prime4
|
||||
return acc
|
||||
}
|
||||
|
||||
func rol1(x uint64) uint64 { return bits.RotateLeft64(x, 1) }
|
||||
func rol7(x uint64) uint64 { return bits.RotateLeft64(x, 7) }
|
||||
func rol11(x uint64) uint64 { return bits.RotateLeft64(x, 11) }
|
||||
func rol12(x uint64) uint64 { return bits.RotateLeft64(x, 12) }
|
||||
func rol18(x uint64) uint64 { return bits.RotateLeft64(x, 18) }
|
||||
func rol23(x uint64) uint64 { return bits.RotateLeft64(x, 23) }
|
||||
func rol27(x uint64) uint64 { return bits.RotateLeft64(x, 27) }
|
||||
func rol31(x uint64) uint64 { return bits.RotateLeft64(x, 31) }
|
||||
209
vendor/github.com/cespare/xxhash/v2/xxhash_amd64.s
generated
vendored
Normal file
209
vendor/github.com/cespare/xxhash/v2/xxhash_amd64.s
generated
vendored
Normal file
@ -0,0 +1,209 @@
|
||||
//go:build !appengine && gc && !purego
|
||||
// +build !appengine
|
||||
// +build gc
|
||||
// +build !purego
|
||||
|
||||
#include "textflag.h"
|
||||
|
||||
// Registers:
|
||||
#define h AX
|
||||
#define d AX
|
||||
#define p SI // pointer to advance through b
|
||||
#define n DX
|
||||
#define end BX // loop end
|
||||
#define v1 R8
|
||||
#define v2 R9
|
||||
#define v3 R10
|
||||
#define v4 R11
|
||||
#define x R12
|
||||
#define prime1 R13
|
||||
#define prime2 R14
|
||||
#define prime4 DI
|
||||
|
||||
#define round(acc, x) \
|
||||
IMULQ prime2, x \
|
||||
ADDQ x, acc \
|
||||
ROLQ $31, acc \
|
||||
IMULQ prime1, acc
|
||||
|
||||
// round0 performs the operation x = round(0, x).
|
||||
#define round0(x) \
|
||||
IMULQ prime2, x \
|
||||
ROLQ $31, x \
|
||||
IMULQ prime1, x
|
||||
|
||||
// mergeRound applies a merge round on the two registers acc and x.
|
||||
// It assumes that prime1, prime2, and prime4 have been loaded.
|
||||
#define mergeRound(acc, x) \
|
||||
round0(x) \
|
||||
XORQ x, acc \
|
||||
IMULQ prime1, acc \
|
||||
ADDQ prime4, acc
|
||||
|
||||
// blockLoop processes as many 32-byte blocks as possible,
|
||||
// updating v1, v2, v3, and v4. It assumes that there is at least one block
|
||||
// to process.
|
||||
#define blockLoop() \
|
||||
loop: \
|
||||
MOVQ +0(p), x \
|
||||
round(v1, x) \
|
||||
MOVQ +8(p), x \
|
||||
round(v2, x) \
|
||||
MOVQ +16(p), x \
|
||||
round(v3, x) \
|
||||
MOVQ +24(p), x \
|
||||
round(v4, x) \
|
||||
ADDQ $32, p \
|
||||
CMPQ p, end \
|
||||
JLE loop
|
||||
|
||||
// func Sum64(b []byte) uint64
|
||||
TEXT ·Sum64(SB), NOSPLIT|NOFRAME, $0-32
|
||||
// Load fixed primes.
|
||||
MOVQ ·primes+0(SB), prime1
|
||||
MOVQ ·primes+8(SB), prime2
|
||||
MOVQ ·primes+24(SB), prime4
|
||||
|
||||
// Load slice.
|
||||
MOVQ b_base+0(FP), p
|
||||
MOVQ b_len+8(FP), n
|
||||
LEAQ (p)(n*1), end
|
||||
|
||||
// The first loop limit will be len(b)-32.
|
||||
SUBQ $32, end
|
||||
|
||||
// Check whether we have at least one block.
|
||||
CMPQ n, $32
|
||||
JLT noBlocks
|
||||
|
||||
// Set up initial state (v1, v2, v3, v4).
|
||||
MOVQ prime1, v1
|
||||
ADDQ prime2, v1
|
||||
MOVQ prime2, v2
|
||||
XORQ v3, v3
|
||||
XORQ v4, v4
|
||||
SUBQ prime1, v4
|
||||
|
||||
blockLoop()
|
||||
|
||||
MOVQ v1, h
|
||||
ROLQ $1, h
|
||||
MOVQ v2, x
|
||||
ROLQ $7, x
|
||||
ADDQ x, h
|
||||
MOVQ v3, x
|
||||
ROLQ $12, x
|
||||
ADDQ x, h
|
||||
MOVQ v4, x
|
||||
ROLQ $18, x
|
||||
ADDQ x, h
|
||||
|
||||
mergeRound(h, v1)
|
||||
mergeRound(h, v2)
|
||||
mergeRound(h, v3)
|
||||
mergeRound(h, v4)
|
||||
|
||||
JMP afterBlocks
|
||||
|
||||
noBlocks:
|
||||
MOVQ ·primes+32(SB), h
|
||||
|
||||
afterBlocks:
|
||||
ADDQ n, h
|
||||
|
||||
ADDQ $24, end
|
||||
CMPQ p, end
|
||||
JG try4
|
||||
|
||||
loop8:
|
||||
MOVQ (p), x
|
||||
ADDQ $8, p
|
||||
round0(x)
|
||||
XORQ x, h
|
||||
ROLQ $27, h
|
||||
IMULQ prime1, h
|
||||
ADDQ prime4, h
|
||||
|
||||
CMPQ p, end
|
||||
JLE loop8
|
||||
|
||||
try4:
|
||||
ADDQ $4, end
|
||||
CMPQ p, end
|
||||
JG try1
|
||||
|
||||
MOVL (p), x
|
||||
ADDQ $4, p
|
||||
IMULQ prime1, x
|
||||
XORQ x, h
|
||||
|
||||
ROLQ $23, h
|
||||
IMULQ prime2, h
|
||||
ADDQ ·primes+16(SB), h
|
||||
|
||||
try1:
|
||||
ADDQ $4, end
|
||||
CMPQ p, end
|
||||
JGE finalize
|
||||
|
||||
loop1:
|
||||
MOVBQZX (p), x
|
||||
ADDQ $1, p
|
||||
IMULQ ·primes+32(SB), x
|
||||
XORQ x, h
|
||||
ROLQ $11, h
|
||||
IMULQ prime1, h
|
||||
|
||||
CMPQ p, end
|
||||
JL loop1
|
||||
|
||||
finalize:
|
||||
MOVQ h, x
|
||||
SHRQ $33, x
|
||||
XORQ x, h
|
||||
IMULQ prime2, h
|
||||
MOVQ h, x
|
||||
SHRQ $29, x
|
||||
XORQ x, h
|
||||
IMULQ ·primes+16(SB), h
|
||||
MOVQ h, x
|
||||
SHRQ $32, x
|
||||
XORQ x, h
|
||||
|
||||
MOVQ h, ret+24(FP)
|
||||
RET
|
||||
|
||||
// func writeBlocks(d *Digest, b []byte) int
|
||||
TEXT ·writeBlocks(SB), NOSPLIT|NOFRAME, $0-40
|
||||
// Load fixed primes needed for round.
|
||||
MOVQ ·primes+0(SB), prime1
|
||||
MOVQ ·primes+8(SB), prime2
|
||||
|
||||
// Load slice.
|
||||
MOVQ b_base+8(FP), p
|
||||
MOVQ b_len+16(FP), n
|
||||
LEAQ (p)(n*1), end
|
||||
SUBQ $32, end
|
||||
|
||||
// Load vN from d.
|
||||
MOVQ s+0(FP), d
|
||||
MOVQ 0(d), v1
|
||||
MOVQ 8(d), v2
|
||||
MOVQ 16(d), v3
|
||||
MOVQ 24(d), v4
|
||||
|
||||
// We don't need to check the loop condition here; this function is
|
||||
// always called with at least one block of data to process.
|
||||
blockLoop()
|
||||
|
||||
// Copy vN back to d.
|
||||
MOVQ v1, 0(d)
|
||||
MOVQ v2, 8(d)
|
||||
MOVQ v3, 16(d)
|
||||
MOVQ v4, 24(d)
|
||||
|
||||
// The number of bytes written is p minus the old base pointer.
|
||||
SUBQ b_base+8(FP), p
|
||||
MOVQ p, ret+32(FP)
|
||||
|
||||
RET
|
||||
183
vendor/github.com/cespare/xxhash/v2/xxhash_arm64.s
generated
vendored
Normal file
183
vendor/github.com/cespare/xxhash/v2/xxhash_arm64.s
generated
vendored
Normal file
@ -0,0 +1,183 @@
|
||||
//go:build !appengine && gc && !purego
|
||||
// +build !appengine
|
||||
// +build gc
|
||||
// +build !purego
|
||||
|
||||
#include "textflag.h"
|
||||
|
||||
// Registers:
|
||||
#define digest R1
|
||||
#define h R2 // return value
|
||||
#define p R3 // input pointer
|
||||
#define n R4 // input length
|
||||
#define nblocks R5 // n / 32
|
||||
#define prime1 R7
|
||||
#define prime2 R8
|
||||
#define prime3 R9
|
||||
#define prime4 R10
|
||||
#define prime5 R11
|
||||
#define v1 R12
|
||||
#define v2 R13
|
||||
#define v3 R14
|
||||
#define v4 R15
|
||||
#define x1 R20
|
||||
#define x2 R21
|
||||
#define x3 R22
|
||||
#define x4 R23
|
||||
|
||||
#define round(acc, x) \
|
||||
MADD prime2, acc, x, acc \
|
||||
ROR $64-31, acc \
|
||||
MUL prime1, acc
|
||||
|
||||
// round0 performs the operation x = round(0, x).
|
||||
#define round0(x) \
|
||||
MUL prime2, x \
|
||||
ROR $64-31, x \
|
||||
MUL prime1, x
|
||||
|
||||
#define mergeRound(acc, x) \
|
||||
round0(x) \
|
||||
EOR x, acc \
|
||||
MADD acc, prime4, prime1, acc
|
||||
|
||||
// blockLoop processes as many 32-byte blocks as possible,
|
||||
// updating v1, v2, v3, and v4. It assumes that n >= 32.
|
||||
#define blockLoop() \
|
||||
LSR $5, n, nblocks \
|
||||
PCALIGN $16 \
|
||||
loop: \
|
||||
LDP.P 16(p), (x1, x2) \
|
||||
LDP.P 16(p), (x3, x4) \
|
||||
round(v1, x1) \
|
||||
round(v2, x2) \
|
||||
round(v3, x3) \
|
||||
round(v4, x4) \
|
||||
SUB $1, nblocks \
|
||||
CBNZ nblocks, loop
|
||||
|
||||
// func Sum64(b []byte) uint64
|
||||
TEXT ·Sum64(SB), NOSPLIT|NOFRAME, $0-32
|
||||
LDP b_base+0(FP), (p, n)
|
||||
|
||||
LDP ·primes+0(SB), (prime1, prime2)
|
||||
LDP ·primes+16(SB), (prime3, prime4)
|
||||
MOVD ·primes+32(SB), prime5
|
||||
|
||||
CMP $32, n
|
||||
CSEL LT, prime5, ZR, h // if n < 32 { h = prime5 } else { h = 0 }
|
||||
BLT afterLoop
|
||||
|
||||
ADD prime1, prime2, v1
|
||||
MOVD prime2, v2
|
||||
MOVD $0, v3
|
||||
NEG prime1, v4
|
||||
|
||||
blockLoop()
|
||||
|
||||
ROR $64-1, v1, x1
|
||||
ROR $64-7, v2, x2
|
||||
ADD x1, x2
|
||||
ROR $64-12, v3, x3
|
||||
ROR $64-18, v4, x4
|
||||
ADD x3, x4
|
||||
ADD x2, x4, h
|
||||
|
||||
mergeRound(h, v1)
|
||||
mergeRound(h, v2)
|
||||
mergeRound(h, v3)
|
||||
mergeRound(h, v4)
|
||||
|
||||
afterLoop:
|
||||
ADD n, h
|
||||
|
||||
TBZ $4, n, try8
|
||||
LDP.P 16(p), (x1, x2)
|
||||
|
||||
round0(x1)
|
||||
|
||||
// NOTE: here and below, sequencing the EOR after the ROR (using a
|
||||
// rotated register) is worth a small but measurable speedup for small
|
||||
// inputs.
|
||||
ROR $64-27, h
|
||||
EOR x1 @> 64-27, h, h
|
||||
MADD h, prime4, prime1, h
|
||||
|
||||
round0(x2)
|
||||
ROR $64-27, h
|
||||
EOR x2 @> 64-27, h, h
|
||||
MADD h, prime4, prime1, h
|
||||
|
||||
try8:
|
||||
TBZ $3, n, try4
|
||||
MOVD.P 8(p), x1
|
||||
|
||||
round0(x1)
|
||||
ROR $64-27, h
|
||||
EOR x1 @> 64-27, h, h
|
||||
MADD h, prime4, prime1, h
|
||||
|
||||
try4:
|
||||
TBZ $2, n, try2
|
||||
MOVWU.P 4(p), x2
|
||||
|
||||
MUL prime1, x2
|
||||
ROR $64-23, h
|
||||
EOR x2 @> 64-23, h, h
|
||||
MADD h, prime3, prime2, h
|
||||
|
||||
try2:
|
||||
TBZ $1, n, try1
|
||||
MOVHU.P 2(p), x3
|
||||
AND $255, x3, x1
|
||||
LSR $8, x3, x2
|
||||
|
||||
MUL prime5, x1
|
||||
ROR $64-11, h
|
||||
EOR x1 @> 64-11, h, h
|
||||
MUL prime1, h
|
||||
|
||||
MUL prime5, x2
|
||||
ROR $64-11, h
|
||||
EOR x2 @> 64-11, h, h
|
||||
MUL prime1, h
|
||||
|
||||
try1:
|
||||
TBZ $0, n, finalize
|
||||
MOVBU (p), x4
|
||||
|
||||
MUL prime5, x4
|
||||
ROR $64-11, h
|
||||
EOR x4 @> 64-11, h, h
|
||||
MUL prime1, h
|
||||
|
||||
finalize:
|
||||
EOR h >> 33, h
|
||||
MUL prime2, h
|
||||
EOR h >> 29, h
|
||||
MUL prime3, h
|
||||
EOR h >> 32, h
|
||||
|
||||
MOVD h, ret+24(FP)
|
||||
RET
|
||||
|
||||
// func writeBlocks(d *Digest, b []byte) int
|
||||
TEXT ·writeBlocks(SB), NOSPLIT|NOFRAME, $0-40
|
||||
LDP ·primes+0(SB), (prime1, prime2)
|
||||
|
||||
// Load state. Assume v[1-4] are stored contiguously.
|
||||
MOVD d+0(FP), digest
|
||||
LDP 0(digest), (v1, v2)
|
||||
LDP 16(digest), (v3, v4)
|
||||
|
||||
LDP b_base+8(FP), (p, n)
|
||||
|
||||
blockLoop()
|
||||
|
||||
// Store updated state.
|
||||
STP (v1, v2), 0(digest)
|
||||
STP (v3, v4), 16(digest)
|
||||
|
||||
BIC $31, n
|
||||
MOVD n, ret+32(FP)
|
||||
RET
|
||||
15
vendor/github.com/cespare/xxhash/v2/xxhash_asm.go
generated
vendored
Normal file
15
vendor/github.com/cespare/xxhash/v2/xxhash_asm.go
generated
vendored
Normal file
@ -0,0 +1,15 @@
|
||||
//go:build (amd64 || arm64) && !appengine && gc && !purego
|
||||
// +build amd64 arm64
|
||||
// +build !appengine
|
||||
// +build gc
|
||||
// +build !purego
|
||||
|
||||
package xxhash
|
||||
|
||||
// Sum64 computes the 64-bit xxHash digest of b with a zero seed.
|
||||
//
|
||||
//go:noescape
|
||||
func Sum64(b []byte) uint64
|
||||
|
||||
//go:noescape
|
||||
func writeBlocks(d *Digest, b []byte) int
|
||||
76
vendor/github.com/cespare/xxhash/v2/xxhash_other.go
generated
vendored
Normal file
76
vendor/github.com/cespare/xxhash/v2/xxhash_other.go
generated
vendored
Normal file
@ -0,0 +1,76 @@
|
||||
//go:build (!amd64 && !arm64) || appengine || !gc || purego
|
||||
// +build !amd64,!arm64 appengine !gc purego
|
||||
|
||||
package xxhash
|
||||
|
||||
// Sum64 computes the 64-bit xxHash digest of b with a zero seed.
|
||||
func Sum64(b []byte) uint64 {
|
||||
// A simpler version would be
|
||||
// d := New()
|
||||
// d.Write(b)
|
||||
// return d.Sum64()
|
||||
// but this is faster, particularly for small inputs.
|
||||
|
||||
n := len(b)
|
||||
var h uint64
|
||||
|
||||
if n >= 32 {
|
||||
v1 := primes[0] + prime2
|
||||
v2 := prime2
|
||||
v3 := uint64(0)
|
||||
v4 := -primes[0]
|
||||
for len(b) >= 32 {
|
||||
v1 = round(v1, u64(b[0:8:len(b)]))
|
||||
v2 = round(v2, u64(b[8:16:len(b)]))
|
||||
v3 = round(v3, u64(b[16:24:len(b)]))
|
||||
v4 = round(v4, u64(b[24:32:len(b)]))
|
||||
b = b[32:len(b):len(b)]
|
||||
}
|
||||
h = rol1(v1) + rol7(v2) + rol12(v3) + rol18(v4)
|
||||
h = mergeRound(h, v1)
|
||||
h = mergeRound(h, v2)
|
||||
h = mergeRound(h, v3)
|
||||
h = mergeRound(h, v4)
|
||||
} else {
|
||||
h = prime5
|
||||
}
|
||||
|
||||
h += uint64(n)
|
||||
|
||||
for ; len(b) >= 8; b = b[8:] {
|
||||
k1 := round(0, u64(b[:8]))
|
||||
h ^= k1
|
||||
h = rol27(h)*prime1 + prime4
|
||||
}
|
||||
if len(b) >= 4 {
|
||||
h ^= uint64(u32(b[:4])) * prime1
|
||||
h = rol23(h)*prime2 + prime3
|
||||
b = b[4:]
|
||||
}
|
||||
for ; len(b) > 0; b = b[1:] {
|
||||
h ^= uint64(b[0]) * prime5
|
||||
h = rol11(h) * prime1
|
||||
}
|
||||
|
||||
h ^= h >> 33
|
||||
h *= prime2
|
||||
h ^= h >> 29
|
||||
h *= prime3
|
||||
h ^= h >> 32
|
||||
|
||||
return h
|
||||
}
|
||||
|
||||
func writeBlocks(d *Digest, b []byte) int {
|
||||
v1, v2, v3, v4 := d.v1, d.v2, d.v3, d.v4
|
||||
n := len(b)
|
||||
for len(b) >= 32 {
|
||||
v1 = round(v1, u64(b[0:8:len(b)]))
|
||||
v2 = round(v2, u64(b[8:16:len(b)]))
|
||||
v3 = round(v3, u64(b[16:24:len(b)]))
|
||||
v4 = round(v4, u64(b[24:32:len(b)]))
|
||||
b = b[32:len(b):len(b)]
|
||||
}
|
||||
d.v1, d.v2, d.v3, d.v4 = v1, v2, v3, v4
|
||||
return n - len(b)
|
||||
}
|
||||
16
vendor/github.com/cespare/xxhash/v2/xxhash_safe.go
generated
vendored
Normal file
16
vendor/github.com/cespare/xxhash/v2/xxhash_safe.go
generated
vendored
Normal file
@ -0,0 +1,16 @@
|
||||
//go:build appengine
|
||||
// +build appengine
|
||||
|
||||
// This file contains the safe implementations of otherwise unsafe-using code.
|
||||
|
||||
package xxhash
|
||||
|
||||
// Sum64String computes the 64-bit xxHash digest of s with a zero seed.
|
||||
func Sum64String(s string) uint64 {
|
||||
return Sum64([]byte(s))
|
||||
}
|
||||
|
||||
// WriteString adds more data to d. It always returns len(s), nil.
|
||||
func (d *Digest) WriteString(s string) (n int, err error) {
|
||||
return d.Write([]byte(s))
|
||||
}
|
||||
58
vendor/github.com/cespare/xxhash/v2/xxhash_unsafe.go
generated
vendored
Normal file
58
vendor/github.com/cespare/xxhash/v2/xxhash_unsafe.go
generated
vendored
Normal file
@ -0,0 +1,58 @@
|
||||
//go:build !appengine
|
||||
// +build !appengine
|
||||
|
||||
// This file encapsulates usage of unsafe.
|
||||
// xxhash_safe.go contains the safe implementations.
|
||||
|
||||
package xxhash
|
||||
|
||||
import (
|
||||
"unsafe"
|
||||
)
|
||||
|
||||
// In the future it's possible that compiler optimizations will make these
|
||||
// XxxString functions unnecessary by realizing that calls such as
|
||||
// Sum64([]byte(s)) don't need to copy s. See https://go.dev/issue/2205.
|
||||
// If that happens, even if we keep these functions they can be replaced with
|
||||
// the trivial safe code.
|
||||
|
||||
// NOTE: The usual way of doing an unsafe string-to-[]byte conversion is:
|
||||
//
|
||||
// var b []byte
|
||||
// bh := (*reflect.SliceHeader)(unsafe.Pointer(&b))
|
||||
// bh.Data = (*reflect.StringHeader)(unsafe.Pointer(&s)).Data
|
||||
// bh.Len = len(s)
|
||||
// bh.Cap = len(s)
|
||||
//
|
||||
// Unfortunately, as of Go 1.15.3 the inliner's cost model assigns a high enough
|
||||
// weight to this sequence of expressions that any function that uses it will
|
||||
// not be inlined. Instead, the functions below use a different unsafe
|
||||
// conversion designed to minimize the inliner weight and allow both to be
|
||||
// inlined. There is also a test (TestInlining) which verifies that these are
|
||||
// inlined.
|
||||
//
|
||||
// See https://github.com/golang/go/issues/42739 for discussion.
|
||||
|
||||
// Sum64String computes the 64-bit xxHash digest of s with a zero seed.
|
||||
// It may be faster than Sum64([]byte(s)) by avoiding a copy.
|
||||
func Sum64String(s string) uint64 {
|
||||
b := *(*[]byte)(unsafe.Pointer(&sliceHeader{s, len(s)}))
|
||||
return Sum64(b)
|
||||
}
|
||||
|
||||
// WriteString adds more data to d. It always returns len(s), nil.
|
||||
// It may be faster than Write([]byte(s)) by avoiding a copy.
|
||||
func (d *Digest) WriteString(s string) (n int, err error) {
|
||||
d.Write(*(*[]byte)(unsafe.Pointer(&sliceHeader{s, len(s)})))
|
||||
// d.Write always returns len(s), nil.
|
||||
// Ignoring the return output and returning these fixed values buys a
|
||||
// savings of 6 in the inliner's cost model.
|
||||
return len(s), nil
|
||||
}
|
||||
|
||||
// sliceHeader is similar to reflect.SliceHeader, but it assumes that the layout
|
||||
// of the first two words is the same as the layout of a string.
|
||||
type sliceHeader struct {
|
||||
s string
|
||||
cap int
|
||||
}
|
||||
31
vendor/github.com/munnerz/goautoneg/LICENSE
generated
vendored
Normal file
31
vendor/github.com/munnerz/goautoneg/LICENSE
generated
vendored
Normal file
@ -0,0 +1,31 @@
|
||||
Copyright (c) 2011, Open Knowledge Foundation Ltd.
|
||||
All rights reserved.
|
||||
|
||||
Redistribution and use in source and binary forms, with or without
|
||||
modification, are permitted provided that the following conditions are
|
||||
met:
|
||||
|
||||
Redistributions of source code must retain the above copyright
|
||||
notice, this list of conditions and the following disclaimer.
|
||||
|
||||
Redistributions in binary form must reproduce the above copyright
|
||||
notice, this list of conditions and the following disclaimer in
|
||||
the documentation and/or other materials provided with the
|
||||
distribution.
|
||||
|
||||
Neither the name of the Open Knowledge Foundation Ltd. nor the
|
||||
names of its contributors may be used to endorse or promote
|
||||
products derived from this software without specific prior written
|
||||
permission.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
13
vendor/github.com/munnerz/goautoneg/Makefile
generated
vendored
Normal file
13
vendor/github.com/munnerz/goautoneg/Makefile
generated
vendored
Normal file
@ -0,0 +1,13 @@
|
||||
include $(GOROOT)/src/Make.inc
|
||||
|
||||
TARG=bitbucket.org/ww/goautoneg
|
||||
GOFILES=autoneg.go
|
||||
|
||||
include $(GOROOT)/src/Make.pkg
|
||||
|
||||
format:
|
||||
gofmt -w *.go
|
||||
|
||||
docs:
|
||||
gomake clean
|
||||
godoc ${TARG} > README.txt
|
||||
67
vendor/github.com/munnerz/goautoneg/README.txt
generated
vendored
Normal file
67
vendor/github.com/munnerz/goautoneg/README.txt
generated
vendored
Normal file
@ -0,0 +1,67 @@
|
||||
PACKAGE
|
||||
|
||||
package goautoneg
|
||||
import "bitbucket.org/ww/goautoneg"
|
||||
|
||||
HTTP Content-Type Autonegotiation.
|
||||
|
||||
The functions in this package implement the behaviour specified in
|
||||
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
|
||||
|
||||
Copyright (c) 2011, Open Knowledge Foundation Ltd.
|
||||
All rights reserved.
|
||||
|
||||
Redistribution and use in source and binary forms, with or without
|
||||
modification, are permitted provided that the following conditions are
|
||||
met:
|
||||
|
||||
Redistributions of source code must retain the above copyright
|
||||
notice, this list of conditions and the following disclaimer.
|
||||
|
||||
Redistributions in binary form must reproduce the above copyright
|
||||
notice, this list of conditions and the following disclaimer in
|
||||
the documentation and/or other materials provided with the
|
||||
distribution.
|
||||
|
||||
Neither the name of the Open Knowledge Foundation Ltd. nor the
|
||||
names of its contributors may be used to endorse or promote
|
||||
products derived from this software without specific prior written
|
||||
permission.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
|
||||
FUNCTIONS
|
||||
|
||||
func Negotiate(header string, alternatives []string) (content_type string)
|
||||
Negotiate the most appropriate content_type given the accept header
|
||||
and a list of alternatives.
|
||||
|
||||
func ParseAccept(header string) (accept []Accept)
|
||||
Parse an Accept Header string returning a sorted list
|
||||
of clauses
|
||||
|
||||
|
||||
TYPES
|
||||
|
||||
type Accept struct {
|
||||
Type, SubType string
|
||||
Q float32
|
||||
Params map[string]string
|
||||
}
|
||||
Structure to represent a clause in an HTTP Accept Header
|
||||
|
||||
|
||||
SUBDIRECTORIES
|
||||
|
||||
.hg
|
||||
189
vendor/github.com/munnerz/goautoneg/autoneg.go
generated
vendored
Normal file
189
vendor/github.com/munnerz/goautoneg/autoneg.go
generated
vendored
Normal file
@ -0,0 +1,189 @@
|
||||
/*
|
||||
HTTP Content-Type Autonegotiation.
|
||||
|
||||
The functions in this package implement the behaviour specified in
|
||||
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
|
||||
|
||||
Copyright (c) 2011, Open Knowledge Foundation Ltd.
|
||||
All rights reserved.
|
||||
|
||||
Redistribution and use in source and binary forms, with or without
|
||||
modification, are permitted provided that the following conditions are
|
||||
met:
|
||||
|
||||
Redistributions of source code must retain the above copyright
|
||||
notice, this list of conditions and the following disclaimer.
|
||||
|
||||
Redistributions in binary form must reproduce the above copyright
|
||||
notice, this list of conditions and the following disclaimer in
|
||||
the documentation and/or other materials provided with the
|
||||
distribution.
|
||||
|
||||
Neither the name of the Open Knowledge Foundation Ltd. nor the
|
||||
names of its contributors may be used to endorse or promote
|
||||
products derived from this software without specific prior written
|
||||
permission.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
*/
|
||||
|
||||
package goautoneg
|
||||
|
||||
import (
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Structure to represent a clause in an HTTP Accept Header
|
||||
type Accept struct {
|
||||
Type, SubType string
|
||||
Q float64
|
||||
Params map[string]string
|
||||
}
|
||||
|
||||
// acceptSlice is defined to implement sort interface.
|
||||
type acceptSlice []Accept
|
||||
|
||||
func (slice acceptSlice) Len() int {
|
||||
return len(slice)
|
||||
}
|
||||
|
||||
func (slice acceptSlice) Less(i, j int) bool {
|
||||
ai, aj := slice[i], slice[j]
|
||||
if ai.Q > aj.Q {
|
||||
return true
|
||||
}
|
||||
if ai.Type != "*" && aj.Type == "*" {
|
||||
return true
|
||||
}
|
||||
if ai.SubType != "*" && aj.SubType == "*" {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (slice acceptSlice) Swap(i, j int) {
|
||||
slice[i], slice[j] = slice[j], slice[i]
|
||||
}
|
||||
|
||||
func stringTrimSpaceCutset(r rune) bool {
|
||||
return r == ' '
|
||||
}
|
||||
|
||||
func nextSplitElement(s, sep string) (item string, remaining string) {
|
||||
if index := strings.Index(s, sep); index != -1 {
|
||||
return s[:index], s[index+1:]
|
||||
}
|
||||
return s, ""
|
||||
}
|
||||
|
||||
// Parse an Accept Header string returning a sorted list
|
||||
// of clauses
|
||||
func ParseAccept(header string) acceptSlice {
|
||||
partsCount := 0
|
||||
remaining := header
|
||||
for len(remaining) > 0 {
|
||||
partsCount++
|
||||
_, remaining = nextSplitElement(remaining, ",")
|
||||
}
|
||||
accept := make(acceptSlice, 0, partsCount)
|
||||
|
||||
remaining = header
|
||||
var part string
|
||||
for len(remaining) > 0 {
|
||||
part, remaining = nextSplitElement(remaining, ",")
|
||||
part = strings.TrimFunc(part, stringTrimSpaceCutset)
|
||||
|
||||
a := Accept{
|
||||
Q: 1.0,
|
||||
}
|
||||
|
||||
sp, remainingPart := nextSplitElement(part, ";")
|
||||
|
||||
sp0, spRemaining := nextSplitElement(sp, "/")
|
||||
a.Type = strings.TrimFunc(sp0, stringTrimSpaceCutset)
|
||||
|
||||
switch {
|
||||
case len(spRemaining) == 0:
|
||||
if a.Type == "*" {
|
||||
a.SubType = "*"
|
||||
} else {
|
||||
continue
|
||||
}
|
||||
default:
|
||||
var sp1 string
|
||||
sp1, spRemaining = nextSplitElement(spRemaining, "/")
|
||||
if len(spRemaining) > 0 {
|
||||
continue
|
||||
}
|
||||
a.SubType = strings.TrimFunc(sp1, stringTrimSpaceCutset)
|
||||
}
|
||||
|
||||
if len(remainingPart) == 0 {
|
||||
accept = append(accept, a)
|
||||
continue
|
||||
}
|
||||
|
||||
a.Params = make(map[string]string)
|
||||
for len(remainingPart) > 0 {
|
||||
sp, remainingPart = nextSplitElement(remainingPart, ";")
|
||||
sp0, spRemaining = nextSplitElement(sp, "=")
|
||||
if len(spRemaining) == 0 {
|
||||
continue
|
||||
}
|
||||
var sp1 string
|
||||
sp1, spRemaining = nextSplitElement(spRemaining, "=")
|
||||
if len(spRemaining) != 0 {
|
||||
continue
|
||||
}
|
||||
token := strings.TrimFunc(sp0, stringTrimSpaceCutset)
|
||||
if token == "q" {
|
||||
a.Q, _ = strconv.ParseFloat(sp1, 32)
|
||||
} else {
|
||||
a.Params[token] = strings.TrimFunc(sp1, stringTrimSpaceCutset)
|
||||
}
|
||||
}
|
||||
|
||||
accept = append(accept, a)
|
||||
}
|
||||
|
||||
sort.Sort(accept)
|
||||
return accept
|
||||
}
|
||||
|
||||
// Negotiate the most appropriate content_type given the accept header
|
||||
// and a list of alternatives.
|
||||
func Negotiate(header string, alternatives []string) (content_type string) {
|
||||
asp := make([][]string, 0, len(alternatives))
|
||||
for _, ctype := range alternatives {
|
||||
asp = append(asp, strings.SplitN(ctype, "/", 2))
|
||||
}
|
||||
for _, clause := range ParseAccept(header) {
|
||||
for i, ctsp := range asp {
|
||||
if clause.Type == ctsp[0] && clause.SubType == ctsp[1] {
|
||||
content_type = alternatives[i]
|
||||
return
|
||||
}
|
||||
if clause.Type == ctsp[0] && clause.SubType == "*" {
|
||||
content_type = alternatives[i]
|
||||
return
|
||||
}
|
||||
if clause.Type == "*" && clause.SubType == "*" {
|
||||
content_type = alternatives[i]
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
393
vendor/github.com/orchard9/go-chassis/chassis/app.go
generated
vendored
Normal file
393
vendor/github.com/orchard9/go-chassis/chassis/app.go
generated
vendored
Normal file
@ -0,0 +1,393 @@
|
||||
// Package chassis is the shared HTTP framework every service surface is built
|
||||
// on (the API service, any second service, the worker's health endpoint). It
|
||||
// owns the edge concerns so handlers stay thin: routing (stdlib ServeMux), the
|
||||
// request Context + JSON envelope, the error model, the edge middleware chain
|
||||
// (recover, request-id, structured logging, RED metrics, security headers,
|
||||
// CORS), pluggable health probes, /metrics, an auth seam, and two-phase
|
||||
// graceful shutdown. See patterns/go-chassis.md.
|
||||
package chassis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"runtime/debug"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/prometheus/client_golang/prometheus"
|
||||
)
|
||||
|
||||
// Config is the chassis runtime configuration. Zero values get safe defaults
|
||||
// (withDefaults); Service + Addr are the only fields a caller must set.
|
||||
type Config struct {
|
||||
Service string // closed-enum service name (metrics/logs)
|
||||
Env string // dev | staging | prod
|
||||
Addr string // listen address, e.g. ":16150"
|
||||
AllowOrigins []string // exact-match CORS allowlist for browser UIs (empty = none)
|
||||
MaxBodyBytes int64 // request body cap (Bind) — default 1 MiB
|
||||
RequestTimeout time.Duration // per-request context deadline — default 15s
|
||||
ReadyTimeout time.Duration // readiness check budget — default 2s
|
||||
DrainDelay time.Duration // phase-1 drain wait before Shutdown — default 5s
|
||||
ShutdownTimeout time.Duration // phase-2 in-flight drain — default 25s
|
||||
|
||||
// Collectors are service-owned Prometheus collectors registered on the
|
||||
// app's private registry alongside the RED/Go/process ones, so /metrics is
|
||||
// one scrape and there is no package-level default registry to collide in.
|
||||
// Composition roots pass domain metrics here.
|
||||
Collectors []prometheus.Collector
|
||||
|
||||
// Validator, if set, runs on every Context.Bind after JSON decode; a non-nil
|
||||
// result becomes 422 Unprocessable. Wire shared/validate at the composition root.
|
||||
Validator func(any) error
|
||||
// EdgeMiddleware are extra http.Handler wrappers applied OUTERMOST (before
|
||||
// secureHeaders) — e.g. a tracing span middleware from shared/tracing.
|
||||
// Composition roots inject these; the chassis stays dependency-light.
|
||||
EdgeMiddleware []func(http.Handler) http.Handler
|
||||
}
|
||||
|
||||
// socketHeadroom is how far the socket deadlines outlive the per-request
|
||||
// context deadline.
|
||||
//
|
||||
// ReadTimeout and WriteTimeout used to be hardcoded at 15s while
|
||||
// RequestTimeout was configurable, so a surface that raised the request budget
|
||||
// (reeld runs 120s so a worker can stream a rendered mp4 on the completion
|
||||
// call) still had its socket cut at 15s. That reads as a proxy 502 rather than
|
||||
// the board's own 413/504, which is a completely different bug to chase.
|
||||
//
|
||||
// The socket MUST outlive the context, never the reverse: when the context
|
||||
// expires the handler returns and the error envelope is written on a socket
|
||||
// that is still open. Cutting the socket first truncates the response mid-body.
|
||||
const socketHeadroom = 30 * time.Second
|
||||
|
||||
func (c *Config) withDefaults() {
|
||||
if c.MaxBodyBytes == 0 {
|
||||
c.MaxBodyBytes = 1 << 20
|
||||
}
|
||||
if c.RequestTimeout == 0 {
|
||||
c.RequestTimeout = 15 * time.Second
|
||||
}
|
||||
if c.ReadyTimeout == 0 {
|
||||
c.ReadyTimeout = 2 * time.Second
|
||||
}
|
||||
if c.DrainDelay == 0 {
|
||||
c.DrainDelay = 5 * time.Second
|
||||
}
|
||||
if c.ShutdownTimeout == 0 {
|
||||
c.ShutdownTimeout = 25 * time.Second
|
||||
}
|
||||
}
|
||||
|
||||
// App is one HTTP surface: routes + health checks + background workers wired at
|
||||
// the composition root, served by Run.
|
||||
type App struct {
|
||||
cfg Config
|
||||
log *slog.Logger
|
||||
mux *http.ServeMux
|
||||
metrics *metrics
|
||||
checks []namedCheck
|
||||
bg []func(context.Context) error
|
||||
routes []Route
|
||||
draining atomic.Bool
|
||||
// closing is closed when phase-1 drain starts. Request handlers that hold
|
||||
// a connection open indefinitely (SSE) select on it so they end at the
|
||||
// drain delay instead of stalling Shutdown for the full ShutdownTimeout.
|
||||
closing chan struct{}
|
||||
}
|
||||
|
||||
// Route is one registered endpoint. Exposed so a spec-drift test can reconcile
|
||||
// the OpenAPI document against what the router actually serves — a hand-kept
|
||||
// list of paths diverges from the code silently, which is the whole failure
|
||||
// mode API docs have.
|
||||
type Route struct {
|
||||
Method string
|
||||
Pattern string
|
||||
}
|
||||
|
||||
// New builds an App and registers the always-on public routes: /metrics,
|
||||
// /healthz (liveness), /readyz (readiness).
|
||||
func New(cfg Config, log *slog.Logger) *App {
|
||||
cfg.withDefaults()
|
||||
a := &App{
|
||||
cfg: cfg, log: log,
|
||||
mux: http.NewServeMux(),
|
||||
metrics: newMetrics(cfg.Service, cfg.Collectors...),
|
||||
closing: make(chan struct{}),
|
||||
}
|
||||
a.mux.Handle("GET /metrics", a.metrics.handler())
|
||||
a.mux.Handle("GET /healthz", a.toHTTP(a.handleLive))
|
||||
a.mux.Handle("GET /readyz", a.toHTTP(a.handleReady))
|
||||
return a
|
||||
}
|
||||
|
||||
// toHTTP adapts a HandlerFunc to net/http, mapping a returned error to the
|
||||
// JSON error envelope.
|
||||
func (a *App) toHTTP(h HandlerFunc) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
c := &Context{
|
||||
w: w, r: r,
|
||||
maxBytes: a.cfg.MaxBodyBytes,
|
||||
validate: a.cfg.Validator,
|
||||
closing: a.closing,
|
||||
}
|
||||
if err := h(c); err != nil {
|
||||
a.writeError(c, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// writeError logs the failure (5xx at error, 4xx at debug — both with the
|
||||
// internal cause) and writes the client envelope, which never carries the cause
|
||||
// or any secret.
|
||||
func (a *App) writeError(c *Context, err error) {
|
||||
e := asError(err)
|
||||
log := c.Log()
|
||||
// The cause rides on BOTH branches. Error.cause is documented as
|
||||
// "logged server-side", and dropping it on 4xx made that false for every
|
||||
// cause the framework attaches — Bind attaches one on 400, 413 and 422 and
|
||||
// nothing else ever sees it. The visible symptom: a rejected body logged as
|
||||
// bare `bad_request` with no field name, so the same unexplained-rejection
|
||||
// hunt WorkerEligible's reason string exists to prevent on the lease path.
|
||||
cause := e.Msg
|
||||
if e.cause != nil {
|
||||
cause = e.cause.Error()
|
||||
}
|
||||
if e.Status >= http.StatusInternalServerError {
|
||||
log.Error("request.error", "category", "request",
|
||||
"error_type", e.Code, "http_status", e.Status, "error_msg", cause)
|
||||
} else {
|
||||
log.Debug("request.rejected", "category", "request",
|
||||
"error_type", e.Code, "http_status", e.Status, "error_msg", cause)
|
||||
}
|
||||
// request_id rides in the body, not just the X-Request-Id header. A browser
|
||||
// on a cross-origin deployment cannot read a response header unless it is
|
||||
// explicitly exposed, so a body without it leaves the user with an error and
|
||||
// nothing to quote to support. The Rust track's error.rs declares this exact
|
||||
// triple as the contract for every surface; omitting it here made that claim
|
||||
// false for half the projects this skill generates.
|
||||
rid := c.w.Header().Get("X-Request-Id")
|
||||
if rid == "" {
|
||||
// Sentinel, matching the Rust track. A literal "-" is itself the signal
|
||||
// that the request-id middleware is not wired, which an absent key is not.
|
||||
rid = "-"
|
||||
}
|
||||
_ = c.JSON(e.Status, map[string]any{
|
||||
"error": map[string]any{"code": e.Code, "message": e.Msg, "request_id": rid},
|
||||
})
|
||||
}
|
||||
|
||||
func (a *App) register(method, pattern string, h HandlerFunc, mw []Middleware) {
|
||||
for i := len(mw) - 1; i >= 0; i-- { // mw[0] outermost
|
||||
h = mw[i](h)
|
||||
}
|
||||
a.routes = append(a.routes, Route{Method: method, Pattern: pattern})
|
||||
a.mux.Handle(method+" "+pattern, a.toHTTP(h))
|
||||
}
|
||||
|
||||
// Routes returns every route registered through Get/Post/Handle/Route, in
|
||||
// registration order. The always-on probes (/healthz, /readyz, /metrics) are
|
||||
// registered directly on the mux and deliberately excluded — they are chassis
|
||||
// infrastructure, not part of a service's documented API surface.
|
||||
func (a *App) Routes() []Route {
|
||||
out := make([]Route, len(a.routes))
|
||||
copy(out, a.routes)
|
||||
return out
|
||||
}
|
||||
|
||||
// Get/Post/Handle register a top-level route with optional route middleware.
|
||||
func (a *App) Get(pattern string, h HandlerFunc, mw ...Middleware) { a.register("GET", pattern, h, mw) }
|
||||
func (a *App) Post(pattern string, h HandlerFunc, mw ...Middleware) {
|
||||
a.register("POST", pattern, h, mw)
|
||||
}
|
||||
func (a *App) Handle(method, pattern string, h HandlerFunc, mw ...Middleware) {
|
||||
a.register(method, pattern, h, mw)
|
||||
}
|
||||
|
||||
// Route groups routes under a path prefix with shared middleware (e.g. auth).
|
||||
func (a *App) Route(prefix string, fn func(r *Router)) { fn(&Router{app: a, prefix: prefix}) }
|
||||
|
||||
// Router registers routes under a prefix, applying group middleware to each.
|
||||
type Router struct {
|
||||
app *App
|
||||
prefix string
|
||||
mw []Middleware
|
||||
}
|
||||
|
||||
// Use adds middleware applied to every route registered on this Router.
|
||||
func (r *Router) Use(mw ...Middleware) { r.mw = append(r.mw, mw...) }
|
||||
|
||||
func (r *Router) Get(pattern string, h HandlerFunc, mw ...Middleware) {
|
||||
r.handle("GET", pattern, h, mw)
|
||||
}
|
||||
func (r *Router) Post(pattern string, h HandlerFunc, mw ...Middleware) {
|
||||
r.handle("POST", pattern, h, mw)
|
||||
}
|
||||
func (r *Router) Delete(pattern string, h HandlerFunc, mw ...Middleware) {
|
||||
r.handle("DELETE", pattern, h, mw)
|
||||
}
|
||||
|
||||
// Handle registers any method under the group prefix.
|
||||
func (r *Router) Handle(method, pattern string, h HandlerFunc, mw ...Middleware) {
|
||||
r.handle(method, pattern, h, mw)
|
||||
}
|
||||
|
||||
func (r *Router) handle(method, pattern string, h HandlerFunc, mw []Middleware) {
|
||||
all := make([]Middleware, 0, len(r.mw)+len(mw))
|
||||
all = append(all, r.mw...)
|
||||
all = append(all, mw...)
|
||||
r.app.register(method, r.prefix+pattern, h, all)
|
||||
}
|
||||
|
||||
// Health registers a named readiness dependency check.
|
||||
func (a *App) Health(name string, fn CheckFunc) {
|
||||
a.checks = append(a.checks, namedCheck{name: name, fn: fn})
|
||||
}
|
||||
|
||||
// Background registers a worker run with the server lifecycle; it receives a
|
||||
// context cancelled on shutdown and Run waits for it to return.
|
||||
func (a *App) Background(fn func(context.Context) error) { a.bg = append(a.bg, fn) }
|
||||
|
||||
// Handler returns the fully composed edge chain over the route mux — used by Run
|
||||
// and available for in-process tests. Order (outermost first): any injected
|
||||
// EdgeMiddleware (e.g. tracing), then secureHeaders, instrument, CORS.
|
||||
func (a *App) Handler() http.Handler {
|
||||
mw := make([]func(http.Handler) http.Handler, 0, len(a.cfg.EdgeMiddleware)+3)
|
||||
mw = append(mw, a.cfg.EdgeMiddleware...)
|
||||
mw = append(mw, a.secureHeaders, a.instrument, a.cors)
|
||||
return chain(a.mux, mw...)
|
||||
}
|
||||
|
||||
// Run starts the server and blocks until ctx is cancelled (SIGINT/SIGTERM) or
|
||||
// the listener fails. Shutdown is two-phase: flip readiness to 503 so the load
|
||||
// balancer drains the pod, wait DrainDelay, then Shutdown in-flight requests
|
||||
// under ShutdownTimeout (DrainDelay + ShutdownTimeout MUST be < the k8s grace
|
||||
// period). Liveness stays 200 throughout so the pod is drained, never killed.
|
||||
func (a *App) Run(ctx context.Context) error {
|
||||
// The socket deadlines track RequestTimeout, they are not independent
|
||||
// knobs: see socketHeadroom. ReadTimeout has to clear a full body upload
|
||||
// (reeld takes a rendered mp4 on the completion call) and WriteTimeout a
|
||||
// full response, both of which are bounded by the request budget.
|
||||
srv := &http.Server{
|
||||
Addr: a.cfg.Addr,
|
||||
Handler: a.Handler(),
|
||||
ReadHeaderTimeout: 5 * time.Second, // Slowloris guard
|
||||
ReadTimeout: a.cfg.RequestTimeout + socketHeadroom,
|
||||
WriteTimeout: a.cfg.RequestTimeout + socketHeadroom,
|
||||
IdleTimeout: 60 * time.Second,
|
||||
MaxHeaderBytes: 1 << 20,
|
||||
}
|
||||
|
||||
runCtx, cancel := context.WithCancel(ctx)
|
||||
defer cancel()
|
||||
|
||||
var wg sync.WaitGroup
|
||||
for _, fn := range a.bg {
|
||||
wg.Add(1)
|
||||
go func(fn func(context.Context) error) {
|
||||
defer wg.Done()
|
||||
a.superviseBackground(runCtx, fn)
|
||||
}(fn)
|
||||
}
|
||||
|
||||
errc := make(chan error, 1)
|
||||
go func() {
|
||||
a.log.Info("server.listening", "http_addr", a.cfg.Addr)
|
||||
if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
|
||||
errc <- err
|
||||
}
|
||||
}()
|
||||
|
||||
select {
|
||||
case err := <-errc:
|
||||
cancel()
|
||||
wg.Wait()
|
||||
return err
|
||||
case <-runCtx.Done():
|
||||
a.draining.Store(true) // phase 1: readiness -> 503, LB drains the pod
|
||||
// Long-lived streams end here rather than at phase 2: an SSE client
|
||||
// that reconnects during the drain delay lands on a pod that is still
|
||||
// serving, instead of one that is 25s from closing under it.
|
||||
close(a.closing)
|
||||
a.log.Info("server.draining", "category", "shutdown", "drain_delay_ms", a.cfg.DrainDelay.Milliseconds())
|
||||
time.Sleep(a.cfg.DrainDelay)
|
||||
|
||||
sctx, scancel := context.WithTimeout(context.Background(), a.cfg.ShutdownTimeout) // phase 2: drain in-flight
|
||||
defer scancel()
|
||||
err := srv.Shutdown(sctx)
|
||||
cancel()
|
||||
wg.Wait()
|
||||
a.log.Info("server.stopped", "category", "shutdown")
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
// superviseBackground runs one background worker, restarting it with backoff if
|
||||
// it panics or returns an error.
|
||||
//
|
||||
// Without this a panic in an auxiliary worker takes the whole process down,
|
||||
// including HTTP serving — so a bug in, say, artifact retention would stop the
|
||||
// board handing out leases. That asymmetry is surprising, because request
|
||||
// handlers already get panic recovery from the instrument middleware; this
|
||||
// gives background work the same protection.
|
||||
//
|
||||
// Restarting rather than merely recovering matters just as much: a worker that
|
||||
// dies quietly leaves its job undone forever with a green readiness probe, and
|
||||
// nobody discovers retention stopped until a disk fills.
|
||||
func (a *App) superviseBackground(ctx context.Context, fn func(context.Context) error) {
|
||||
const (
|
||||
minBackoff = time.Second
|
||||
maxBackoff = time.Minute
|
||||
)
|
||||
backoff := minBackoff
|
||||
|
||||
for {
|
||||
err, panicked := runBackgroundOnce(ctx, fn)
|
||||
switch {
|
||||
case ctx.Err() != nil:
|
||||
// Shutdown. A worker returning on a cancelled context is the
|
||||
// normal path, not a failure.
|
||||
return
|
||||
case err == nil && !panicked:
|
||||
// A clean return before shutdown means the worker considers its
|
||||
// job finished. Respect that rather than spinning it forever.
|
||||
return
|
||||
case panicked:
|
||||
a.log.Error("background.panicked", "category", "worker",
|
||||
"error_type", "worker_panic", "error_msg", err.Error(),
|
||||
"restart_in_ms", backoff.Milliseconds())
|
||||
default:
|
||||
a.log.Error("background.failed", "category", "worker",
|
||||
"error_type", "worker_error", "error_msg", err.Error(),
|
||||
"restart_in_ms", backoff.Milliseconds())
|
||||
}
|
||||
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-time.After(backoff):
|
||||
}
|
||||
if backoff *= 2; backoff > maxBackoff {
|
||||
backoff = maxBackoff
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// runBackgroundOnce invokes fn, converting a panic into an error so the
|
||||
// supervisor can treat both failure modes alike. The stack is attached because
|
||||
// a recovered panic with no stack is nearly unactionable.
|
||||
func runBackgroundOnce(ctx context.Context, fn func(context.Context) error) (err error, panicked bool) {
|
||||
defer func() {
|
||||
if r := recover(); r != nil {
|
||||
panicked = true
|
||||
err = fmt.Errorf("panic: %v\n%s", r, debug.Stack())
|
||||
}
|
||||
}()
|
||||
err = fn(ctx)
|
||||
if errors.Is(err, context.Canceled) {
|
||||
err = nil
|
||||
}
|
||||
return err, false
|
||||
}
|
||||
104
vendor/github.com/orchard9/go-chassis/chassis/auth.go
generated
vendored
Normal file
104
vendor/github.com/orchard9/go-chassis/chassis/auth.go
generated
vendored
Normal file
@ -0,0 +1,104 @@
|
||||
package chassis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/subtle"
|
||||
"net/http"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Identity is the authenticated principal attached to the request context by an
|
||||
// auth Middleware. Subject is the stable principal id; OrgID is the tenant the
|
||||
// principal is acting in (the multi-tenant scoping key — every tenant row
|
||||
// filters by it); Scopes are the granted permissions; Claims carries any extra
|
||||
// verified attributes without coupling chassis to a domain type.
|
||||
type Identity struct {
|
||||
Subject string
|
||||
OrgID string
|
||||
Scopes []string
|
||||
Claims map[string]string
|
||||
}
|
||||
|
||||
// HasScope reports whether the identity holds the required scope. A held scope
|
||||
// ending in ":*" dominates any required scope sharing its prefix, so "admin:*"
|
||||
// satisfies "admin:read". An exact match always passes.
|
||||
func (id *Identity) HasScope(required string) bool {
|
||||
if id == nil {
|
||||
return false
|
||||
}
|
||||
for _, s := range id.Scopes {
|
||||
if s == required {
|
||||
return true
|
||||
}
|
||||
if prefix, ok := strings.CutSuffix(s, ":*"); ok {
|
||||
if reqPrefix, _, found := strings.Cut(required, ":"); found && reqPrefix == prefix {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Authenticator verifies a request and returns its Identity, or an *Error
|
||||
// (Unauthorized/Forbidden) when verification fails. Implementations:
|
||||
// - NoAuth — local default, everyone is anonymous (documented seam).
|
||||
// - StaticToken — shared bearer token, constant-time compared (local/CI).
|
||||
// - (later) OIDC — coreos/go-oidc verifier, JWKS cache, refresh-on-kid-miss.
|
||||
// - (later) APIKey — hashed key behind a Secrets port, constant-time compare.
|
||||
type Authenticator interface {
|
||||
Authenticate(r *http.Request) (*Identity, error)
|
||||
}
|
||||
|
||||
type identityKey struct{}
|
||||
|
||||
// IdentityFrom returns the authenticated identity, or false when the route was
|
||||
// not behind RequireAuth.
|
||||
func IdentityFrom(ctx context.Context) (*Identity, bool) {
|
||||
id, ok := ctx.Value(identityKey{}).(*Identity)
|
||||
return id, ok
|
||||
}
|
||||
|
||||
// RequireAuth is route/group middleware that runs the Authenticator, rejecting
|
||||
// the request (deny-by-default) when it fails and otherwise stashing the
|
||||
// Identity in the context for handlers.
|
||||
func RequireAuth(a Authenticator) Middleware {
|
||||
return func(next HandlerFunc) HandlerFunc {
|
||||
return func(c *Context) error {
|
||||
id, err := a.Authenticate(c.r)
|
||||
if err != nil {
|
||||
return err // already an *Error (Unauthorized/Forbidden)
|
||||
}
|
||||
c.r = c.r.WithContext(context.WithValue(c.r.Context(), identityKey{}, id))
|
||||
return next(c)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// NoAuth treats every caller as anonymous. The explicit local default — never
|
||||
// select it for a protected environment.
|
||||
type NoAuth struct{}
|
||||
|
||||
func (NoAuth) Authenticate(*http.Request) (*Identity, error) {
|
||||
return &Identity{Subject: "anonymous"}, nil
|
||||
}
|
||||
|
||||
// StaticToken authenticates a single shared bearer token in constant time
|
||||
// (crypto/subtle) to avoid leaking the token via response timing. For local/CI
|
||||
// and internal service-to-service; real principals use OIDC/API-key later.
|
||||
type StaticToken struct{ token string }
|
||||
|
||||
// NewStaticToken builds a StaticToken; an empty token rejects every request.
|
||||
func NewStaticToken(token string) StaticToken { return StaticToken{token: token} }
|
||||
|
||||
func (s StaticToken) Authenticate(r *http.Request) (*Identity, error) {
|
||||
const prefix = "Bearer "
|
||||
h := r.Header.Get("Authorization")
|
||||
if s.token == "" || !strings.HasPrefix(h, prefix) {
|
||||
return nil, Unauthorized("missing or malformed bearer token")
|
||||
}
|
||||
got := strings.TrimPrefix(h, prefix)
|
||||
if subtle.ConstantTimeCompare([]byte(got), []byte(s.token)) != 1 {
|
||||
return nil, Unauthorized("invalid token")
|
||||
}
|
||||
return &Identity{Subject: "static-token"}, nil
|
||||
}
|
||||
37
vendor/github.com/orchard9/go-chassis/chassis/authz.go
generated
vendored
Normal file
37
vendor/github.com/orchard9/go-chassis/chassis/authz.go
generated
vendored
Normal file
@ -0,0 +1,37 @@
|
||||
package chassis
|
||||
|
||||
// Authorization middleware — the layer above authentication. RequireAuth proves
|
||||
// WHO; these prove WHAT they may do and WHICH tenant they act in. Apply after
|
||||
// RequireAuth on a route group.
|
||||
|
||||
// RequireScope rejects callers lacking the given scope (scope dominance applies:
|
||||
// a held "x:*" satisfies "x:read"). 401 if unauthenticated, 403 if under-scoped.
|
||||
func RequireScope(scope string) Middleware {
|
||||
return func(next HandlerFunc) HandlerFunc {
|
||||
return func(c *Context) error {
|
||||
id, ok := c.Identity()
|
||||
if !ok {
|
||||
return Unauthorized("authentication required")
|
||||
}
|
||||
if !id.HasScope(scope) {
|
||||
return Forbidden("missing required scope: " + scope)
|
||||
}
|
||||
return next(c)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// RequireOrg enforces the multi-tenant invariant: the caller must have a resolved
|
||||
// active organization. Handlers behind it can rely on a non-empty Context.OrgID
|
||||
// and MUST filter every tenant query by it.
|
||||
func RequireOrg() Middleware {
|
||||
return func(next HandlerFunc) HandlerFunc {
|
||||
return func(c *Context) error {
|
||||
id, ok := c.Identity()
|
||||
if !ok || id.OrgID == "" {
|
||||
return Forbidden("no active organization")
|
||||
}
|
||||
return next(c)
|
||||
}
|
||||
}
|
||||
}
|
||||
139
vendor/github.com/orchard9/go-chassis/chassis/context.go
generated
vendored
Normal file
139
vendor/github.com/orchard9/go-chassis/chassis/context.go
generated
vendored
Normal file
@ -0,0 +1,139 @@
|
||||
package chassis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/orchard9/go-chassis/logging"
|
||||
)
|
||||
|
||||
// HandlerFunc is the chassis handler signature: return an error and the
|
||||
// framework maps it to the JSON error envelope. See patterns/go-chassis.md.
|
||||
type HandlerFunc func(*Context) error
|
||||
|
||||
// Middleware decorates a HandlerFunc (route/group scope: auth, rate-limit).
|
||||
// Cross-cutting infra (recover, request-id, logging, metrics) is applied once
|
||||
// at the server edge as http.Handler middleware, not here.
|
||||
type Middleware func(HandlerFunc) HandlerFunc
|
||||
|
||||
// Context carries the request/response for one call plus typed helpers.
|
||||
type Context struct {
|
||||
w http.ResponseWriter
|
||||
r *http.Request
|
||||
maxBytes int64
|
||||
validate func(any) error // optional; set from Config.Validator
|
||||
// closing fires when the app begins draining, so a Stream ends with the
|
||||
// pod instead of holding Shutdown open.
|
||||
closing <-chan struct{}
|
||||
}
|
||||
|
||||
// Context returns the request context (deadline + request-scoped logger).
|
||||
func (c *Context) Context() context.Context { return c.r.Context() }
|
||||
|
||||
// Request exposes the raw request for the rare case a helper does not cover.
|
||||
func (c *Context) Request() *http.Request { return c.r }
|
||||
|
||||
// Writer exposes the raw ResponseWriter for the three responses the JSON
|
||||
// envelope cannot carry: the Scalar docs page (HTML), the OpenAPI document
|
||||
// (pre-rendered bytes), and a streamed artifact (video/mp4).
|
||||
func (c *Context) Writer() http.ResponseWriter { return c.w }
|
||||
|
||||
// Log returns the request-scoped logger (carries request_id).
|
||||
func (c *Context) Log() *slog.Logger { return logging.From(c.r.Context()) }
|
||||
|
||||
// PathValue returns a ServeMux wildcard value, e.g. {id} from "/v1/x/{id}".
|
||||
func (c *Context) PathValue(key string) string { return c.r.PathValue(key) }
|
||||
|
||||
// Identity returns the authenticated principal, or false when the route was not
|
||||
// behind RequireAuth.
|
||||
func (c *Context) Identity() (*Identity, bool) { return IdentityFrom(c.r.Context()) }
|
||||
|
||||
// OrgID returns the active tenant for the caller ("" when unauthenticated or the
|
||||
// identity carries no org). Repos MUST filter every tenant query by it.
|
||||
func (c *Context) OrgID() string {
|
||||
if id, ok := c.Identity(); ok {
|
||||
return id.OrgID
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// TraceID returns the request's trace id (W3C traceparent or generated).
|
||||
func (c *Context) TraceID() string {
|
||||
id, _ := TraceIDFrom(c.r.Context())
|
||||
return id
|
||||
}
|
||||
|
||||
// Bind enforces the body-size limit, then JSON-decodes into v rejecting unknown
|
||||
// fields. An oversized body becomes 413; a malformed body becomes 400 — neither
|
||||
// leaks internals to the client.
|
||||
func (c *Context) Bind(v any) error {
|
||||
c.r.Body = http.MaxBytesReader(c.w, c.r.Body, c.maxBytes)
|
||||
dec := json.NewDecoder(c.r.Body)
|
||||
dec.DisallowUnknownFields()
|
||||
if err := dec.Decode(v); err != nil {
|
||||
var maxErr *http.MaxBytesError
|
||||
if errors.As(err, &maxErr) {
|
||||
return PayloadTooLarge("request body too large").WithCause(err)
|
||||
}
|
||||
if f, ok := unknownField(err); ok {
|
||||
return BadRequest(fmt.Sprintf("unknown field %q", f)).WithCause(err)
|
||||
}
|
||||
return BadRequest("invalid JSON body").WithCause(err)
|
||||
}
|
||||
if c.validate != nil {
|
||||
if err := c.validate(v); err != nil {
|
||||
return Unprocessable("request failed validation").WithCause(err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// unknownFieldPrefix is what encoding/json returns under
|
||||
// DisallowUnknownFields. The stdlib gives no typed error for it, so matching the
|
||||
// text is the only way to tell "you sent a field we do not accept" from "your
|
||||
// JSON is broken" — and they are different bugs on the caller's side.
|
||||
const unknownFieldPrefix = `json: unknown field `
|
||||
|
||||
// unknownField extracts the rejected field name, if that is why decoding failed.
|
||||
//
|
||||
// The name is echoed to the client because the alternative is what this cost us:
|
||||
// a worker POSTing a well-formed body with one extra key is told "invalid JSON
|
||||
// body", which is false, and has nothing to act on. Echoing a key the caller
|
||||
// just sent leaks nothing. The rest of the decoder's errors stay generic — an
|
||||
// UnmarshalTypeError names Go struct fields and types, which is internal detail.
|
||||
func unknownField(err error) (string, bool) {
|
||||
msg := err.Error()
|
||||
if !strings.HasPrefix(msg, unknownFieldPrefix) {
|
||||
return "", false
|
||||
}
|
||||
name, uerr := strconv.Unquote(strings.TrimPrefix(msg, unknownFieldPrefix))
|
||||
if uerr != nil {
|
||||
return "", false
|
||||
}
|
||||
return name, true
|
||||
}
|
||||
|
||||
// OK writes 200 + JSON. Created writes 201. NoContent writes 204.
|
||||
func (c *Context) OK(v any) error { return c.JSON(http.StatusOK, v) }
|
||||
func (c *Context) Created(v any) error { return c.JSON(http.StatusCreated, v) }
|
||||
|
||||
func (c *Context) NoContent() error {
|
||||
c.w.WriteHeader(http.StatusNoContent)
|
||||
return nil
|
||||
}
|
||||
|
||||
// JSON writes the status code and JSON-encodes v.
|
||||
func (c *Context) JSON(code int, v any) error {
|
||||
c.w.Header().Set("Content-Type", "application/json; charset=utf-8")
|
||||
c.w.WriteHeader(code)
|
||||
if v == nil {
|
||||
return nil
|
||||
}
|
||||
return json.NewEncoder(c.w).Encode(v)
|
||||
}
|
||||
74
vendor/github.com/orchard9/go-chassis/chassis/errors.go
generated
vendored
Normal file
74
vendor/github.com/orchard9/go-chassis/chassis/errors.go
generated
vendored
Normal file
@ -0,0 +1,74 @@
|
||||
package chassis
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
)
|
||||
|
||||
// Error is the HTTP error model: a stable machine Code + a SAFE public Msg
|
||||
// (no secrets, no internal detail) shown to the client, plus an internal cause
|
||||
// that is logged server-side and NEVER serialized to the response. Handlers
|
||||
// return these; the framework maps them to the JSON error envelope.
|
||||
//
|
||||
// This is distinct from any domain result/status enum a handler returns as a
|
||||
// SUCCESS-response field — those are 200 bodies, not errors.
|
||||
type Error struct {
|
||||
Status int // HTTP status code
|
||||
Code string // stable machine-readable code (snake_case)
|
||||
Msg string // safe public message
|
||||
cause error // internal cause: logged, never sent to the client
|
||||
}
|
||||
|
||||
func (e *Error) Error() string {
|
||||
if e.cause != nil {
|
||||
return fmt.Sprintf("%s: %s: %v", e.Code, e.Msg, e.cause)
|
||||
}
|
||||
return fmt.Sprintf("%s: %s", e.Code, e.Msg)
|
||||
}
|
||||
|
||||
func (e *Error) Unwrap() error { return e.cause }
|
||||
|
||||
// WithCause attaches the internal cause (logged, never sent). Returns a copy so
|
||||
// the package-level constructors stay immutable.
|
||||
func (e *Error) WithCause(err error) *Error {
|
||||
c := *e
|
||||
c.cause = err
|
||||
return &c
|
||||
}
|
||||
|
||||
func newErr(status int, code, msg string) *Error {
|
||||
return &Error{Status: status, Code: code, Msg: msg}
|
||||
}
|
||||
|
||||
// Constructors — public Msg defaults are safe and generic; override per call.
|
||||
func BadRequest(msg string) *Error { return newErr(http.StatusBadRequest, "bad_request", msg) }
|
||||
func Unauthorized(msg string) *Error { return newErr(http.StatusUnauthorized, "unauthorized", msg) }
|
||||
func Forbidden(msg string) *Error { return newErr(http.StatusForbidden, "forbidden", msg) }
|
||||
func NotFound(msg string) *Error { return newErr(http.StatusNotFound, "not_found", msg) }
|
||||
func Conflict(msg string) *Error { return newErr(http.StatusConflict, "conflict", msg) }
|
||||
func Unprocessable(msg string) *Error {
|
||||
return newErr(http.StatusUnprocessableEntity, "unprocessable", msg)
|
||||
}
|
||||
func TooManyRequests(msg string) *Error {
|
||||
return newErr(http.StatusTooManyRequests, "rate_limited", msg)
|
||||
}
|
||||
func PayloadTooLarge(msg string) *Error {
|
||||
return newErr(http.StatusRequestEntityTooLarge, "payload_too_large", msg)
|
||||
}
|
||||
|
||||
// Internal wraps any non-Error as a 500 with a generic public message; the real
|
||||
// cause is preserved for logging but hidden from the client.
|
||||
func Internal(cause error) *Error {
|
||||
return (&Error{Status: http.StatusInternalServerError, Code: "internal", Msg: "internal error"}).WithCause(cause)
|
||||
}
|
||||
|
||||
// asError maps any error to an *Error, defaulting unknown errors to a 500 whose
|
||||
// detail is hidden from the client.
|
||||
func asError(err error) *Error {
|
||||
var e *Error
|
||||
if errors.As(err, &e) {
|
||||
return e
|
||||
}
|
||||
return Internal(err)
|
||||
}
|
||||
50
vendor/github.com/orchard9/go-chassis/chassis/health.go
generated
vendored
Normal file
50
vendor/github.com/orchard9/go-chassis/chassis/health.go
generated
vendored
Normal file
@ -0,0 +1,50 @@
|
||||
package chassis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
)
|
||||
|
||||
// CheckFunc reports whether a dependency is usable right now. It MUST honor the
|
||||
// passed ctx deadline and return a non-nil error (with no secrets) when unhealthy.
|
||||
type CheckFunc func(ctx context.Context) error
|
||||
|
||||
type namedCheck struct {
|
||||
name string
|
||||
fn CheckFunc
|
||||
}
|
||||
|
||||
// handleLive is the k8s liveness probe: 200 while the process runs. It does NOT
|
||||
// check dependencies and stays 200 during drain — a flapping dep or an
|
||||
// in-progress shutdown must never trigger a restart, only LB removal.
|
||||
func (a *App) handleLive(c *Context) error {
|
||||
return c.OK(map[string]any{"status": "ok"})
|
||||
}
|
||||
|
||||
// handleReady is the k8s readiness probe. It returns 503 the moment shutdown
|
||||
// begins (two-phase drain: the LB pulls the pod before connections close), and
|
||||
// otherwise 200 only when every registered dependency check passes.
|
||||
func (a *App) handleReady(c *Context) error {
|
||||
if a.draining.Load() {
|
||||
return c.JSON(http.StatusServiceUnavailable, map[string]any{"status": "draining"})
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(c.Context(), a.cfg.ReadyTimeout)
|
||||
defer cancel()
|
||||
|
||||
checks := make(map[string]string, len(a.checks))
|
||||
ok := true
|
||||
for _, ch := range a.checks {
|
||||
if err := ch.fn(ctx); err != nil {
|
||||
checks[ch.name] = "error: " + err.Error()
|
||||
ok = false
|
||||
continue
|
||||
}
|
||||
checks[ch.name] = "ok"
|
||||
}
|
||||
|
||||
status, code := "ok", http.StatusOK
|
||||
if !ok {
|
||||
status, code = "unready", http.StatusServiceUnavailable
|
||||
}
|
||||
return c.JSON(code, map[string]any{"status": status, "checks": checks})
|
||||
}
|
||||
119
vendor/github.com/orchard9/go-chassis/chassis/idempotency.go
generated
vendored
Normal file
119
vendor/github.com/orchard9/go-chassis/chassis/idempotency.go
generated
vendored
Normal file
@ -0,0 +1,119 @@
|
||||
package chassis
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"net/http"
|
||||
"time"
|
||||
)
|
||||
|
||||
// StoredResponse is a cached successful response, replayed verbatim (status +
|
||||
// headers + body) on a duplicate Idempotency-Key.
|
||||
type StoredResponse struct {
|
||||
Status int
|
||||
Header http.Header
|
||||
Body []byte
|
||||
}
|
||||
|
||||
// IdempotencyStore persists and replays responses keyed by Idempotency-Key.
|
||||
// Back it with Redis (TTL'd) in shared/adapters/redis. Lookup returns found=false
|
||||
// when the key is new; Save records a successful response.
|
||||
type IdempotencyStore interface {
|
||||
Lookup(ctx context.Context, key string) (StoredResponse, bool, error)
|
||||
Save(ctx context.Context, key string, resp StoredResponse, ttl time.Duration) error
|
||||
}
|
||||
|
||||
// Idempotent dedups mutating requests by the Idempotency-Key header: a duplicate
|
||||
// returns the original response without re-running the handler. Apply it to
|
||||
// POST/PUT route groups. Requests without the header pass straight through. The
|
||||
// key is scoped by the caller's org so keys can't collide across tenants. The
|
||||
// store is best-effort: a backend error fails open (the request proceeds) rather
|
||||
// than blocking writes.
|
||||
func Idempotent(store IdempotencyStore, ttl time.Duration) Middleware {
|
||||
return func(next HandlerFunc) HandlerFunc {
|
||||
return func(c *Context) error {
|
||||
key := c.r.Header.Get("Idempotency-Key")
|
||||
if key == "" {
|
||||
return next(c)
|
||||
}
|
||||
// Scope by tenant. With no resolved org we refuse to cache rather than
|
||||
// share a global (cross-tenant) key — pair this with RequireOrg.
|
||||
org := c.OrgID()
|
||||
if org == "" {
|
||||
c.Log().Warn("idempotency.skipped_no_org", "category", "idempotency")
|
||||
return next(c)
|
||||
}
|
||||
scoped := org + ":" + key
|
||||
ctx := c.r.Context()
|
||||
|
||||
if sr, found, err := store.Lookup(ctx, scoped); err != nil {
|
||||
c.Log().Warn("idempotency.lookup_failed", "category", "idempotency", "error_msg", err.Error())
|
||||
} else if found {
|
||||
h := c.w.Header()
|
||||
for k, vs := range sr.Header { // replay the original headers verbatim
|
||||
for _, v := range vs {
|
||||
h.Add(k, v)
|
||||
}
|
||||
}
|
||||
h.Set("Idempotency-Replayed", "true")
|
||||
c.w.WriteHeader(sr.Status)
|
||||
_, werr := c.w.Write(sr.Body)
|
||||
return werr
|
||||
}
|
||||
|
||||
// Capture the handler's response so we can persist + flush it. Restore
|
||||
// the writer via defer so a panic in next() can't leave c.w dangling.
|
||||
rec := &captureWriter{header: http.Header{}, status: http.StatusOK}
|
||||
orig := c.w
|
||||
c.w = rec
|
||||
defer func() { c.w = orig }()
|
||||
if err := next(c); err != nil {
|
||||
return err // errors aren't cached; the framework writes the envelope to orig
|
||||
}
|
||||
|
||||
for k, vs := range rec.header {
|
||||
for _, v := range vs {
|
||||
orig.Header().Add(k, v)
|
||||
}
|
||||
}
|
||||
orig.WriteHeader(rec.status)
|
||||
if _, werr := orig.Write(rec.buf.Bytes()); werr != nil {
|
||||
return werr
|
||||
}
|
||||
if rec.status >= 200 && rec.status < 300 {
|
||||
body := append([]byte(nil), rec.buf.Bytes()...)
|
||||
stored := StoredResponse{Status: rec.status, Header: rec.header.Clone(), Body: body}
|
||||
if serr := store.Save(ctx, scoped, stored, ttl); serr != nil {
|
||||
c.Log().Warn("idempotency.save_failed", "category", "idempotency", "error_msg", serr.Error())
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// captureWriter buffers a handler's response so the idempotency middleware can
|
||||
// persist and replay it. It implements http.ResponseWriter.
|
||||
type captureWriter struct {
|
||||
header http.Header
|
||||
buf bytes.Buffer
|
||||
status int
|
||||
wrote bool
|
||||
}
|
||||
|
||||
func (c *captureWriter) Header() http.Header { return c.header }
|
||||
|
||||
func (c *captureWriter) WriteHeader(code int) {
|
||||
if c.wrote {
|
||||
return
|
||||
}
|
||||
c.status = code
|
||||
c.wrote = true
|
||||
}
|
||||
|
||||
func (c *captureWriter) Write(b []byte) (int, error) {
|
||||
if !c.wrote {
|
||||
c.WriteHeader(http.StatusOK)
|
||||
}
|
||||
return c.buf.Write(b)
|
||||
}
|
||||
67
vendor/github.com/orchard9/go-chassis/chassis/metrics.go
generated
vendored
Normal file
67
vendor/github.com/orchard9/go-chassis/chassis/metrics.go
generated
vendored
Normal file
@ -0,0 +1,67 @@
|
||||
package chassis
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/prometheus/client_golang/prometheus"
|
||||
"github.com/prometheus/client_golang/prometheus/collectors"
|
||||
"github.com/prometheus/client_golang/prometheus/promhttp"
|
||||
)
|
||||
|
||||
// metrics holds the RED HTTP collectors on a private registry (no global state,
|
||||
// so tests and multiple apps never collide). The route label is the matched
|
||||
// ServeMux pattern — bounded cardinality, never the raw path.
|
||||
type metrics struct {
|
||||
reg *prometheus.Registry
|
||||
reqs *prometheus.CounterVec
|
||||
dur *prometheus.HistogramVec
|
||||
inflight prometheus.Gauge
|
||||
}
|
||||
|
||||
// newMetrics builds the RED collectors and registers them alongside any
|
||||
// service-owned collectors (Config.Collectors) on the same private registry —
|
||||
// so /metrics is one scrape and a domain gauge cannot be lost to a second
|
||||
// endpoint nobody remembers to scrape.
|
||||
func newMetrics(service string, extra ...prometheus.Collector) *metrics {
|
||||
reg := prometheus.NewRegistry()
|
||||
labels := prometheus.Labels{"service": service}
|
||||
m := &metrics{
|
||||
reg: reg,
|
||||
reqs: prometheus.NewCounterVec(prometheus.CounterOpts{
|
||||
Name: "http_requests_total", Help: "Total HTTP requests.", ConstLabels: labels,
|
||||
}, []string{"method", "route", "status"}),
|
||||
dur: prometheus.NewHistogramVec(prometheus.HistogramOpts{
|
||||
Name: "http_request_duration_seconds", Help: "HTTP request latency.",
|
||||
Buckets: prometheus.DefBuckets, ConstLabels: labels,
|
||||
}, []string{"method", "route"}),
|
||||
inflight: prometheus.NewGauge(prometheus.GaugeOpts{
|
||||
Name: "http_requests_in_flight", Help: "In-flight HTTP requests.", ConstLabels: labels,
|
||||
}),
|
||||
}
|
||||
reg.MustRegister(m.reqs, m.dur, m.inflight,
|
||||
collectors.NewGoCollector(),
|
||||
collectors.NewProcessCollector(collectors.ProcessCollectorOpts{}))
|
||||
for _, c := range extra {
|
||||
reg.MustRegister(c)
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
func (m *metrics) observe(method, route string, status int, d time.Duration) {
|
||||
m.countOnly(method, route, status)
|
||||
m.dur.WithLabelValues(method, route).Observe(d.Seconds())
|
||||
}
|
||||
|
||||
// countOnly records a request without timing it. For responses whose elapsed
|
||||
// time is not a latency — an event stream ends when the operator closes the
|
||||
// tab, and putting that in the histogram makes every latency alert lie.
|
||||
func (m *metrics) countOnly(method, route string, status int) {
|
||||
m.reqs.WithLabelValues(method, route, strconv.Itoa(status)).Inc()
|
||||
}
|
||||
|
||||
// handler serves the Prometheus exposition for the metrics agent to scrape.
|
||||
func (m *metrics) handler() http.Handler {
|
||||
return promhttp.HandlerFor(m.reg, promhttp.HandlerOpts{Registry: m.reg})
|
||||
}
|
||||
244
vendor/github.com/orchard9/go-chassis/chassis/middleware.go
generated
vendored
Normal file
244
vendor/github.com/orchard9/go-chassis/chassis/middleware.go
generated
vendored
Normal file
@ -0,0 +1,244 @@
|
||||
package chassis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"runtime/debug"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/orchard9/go-chassis/logging"
|
||||
)
|
||||
|
||||
// chain composes edge middleware so mw[0] is outermost (runs first).
|
||||
func chain(h http.Handler, mw ...func(http.Handler) http.Handler) http.Handler {
|
||||
for i := len(mw) - 1; i >= 0; i-- {
|
||||
h = mw[i](h)
|
||||
}
|
||||
return h
|
||||
}
|
||||
|
||||
// statusRecorder captures the response status for logging/metrics and survives
|
||||
// double WriteHeader; it forwards Flush so SSE/streaming handlers still work.
|
||||
type statusRecorder struct {
|
||||
http.ResponseWriter
|
||||
status int
|
||||
wrote bool
|
||||
// streamed is set by Context.Stream. It keeps a connection an operator
|
||||
// held open for twenty minutes out of the request-latency histogram.
|
||||
streamed bool
|
||||
}
|
||||
|
||||
func (r *statusRecorder) WriteHeader(code int) {
|
||||
if r.wrote {
|
||||
return
|
||||
}
|
||||
r.status = code
|
||||
r.wrote = true
|
||||
r.ResponseWriter.WriteHeader(code)
|
||||
}
|
||||
|
||||
func (r *statusRecorder) Write(b []byte) (int, error) {
|
||||
if !r.wrote {
|
||||
r.WriteHeader(http.StatusOK)
|
||||
}
|
||||
return r.ResponseWriter.Write(b)
|
||||
}
|
||||
|
||||
func (r *statusRecorder) Flush() {
|
||||
if f, ok := r.ResponseWriter.(http.Flusher); ok {
|
||||
f.Flush()
|
||||
}
|
||||
}
|
||||
|
||||
// Unwrap exposes the wrapped writer to http.ResponseController, which is how a
|
||||
// streaming handler clears the server's WriteTimeout for its own connection.
|
||||
// Without it ResponseController stops at this wrapper and every deadline call
|
||||
// returns ErrNotSupported — the response is then cut mid-stream at
|
||||
// RequestTimeout+socketHeadroom with no error the handler can report.
|
||||
func (r *statusRecorder) Unwrap() http.ResponseWriter { return r.ResponseWriter }
|
||||
|
||||
func newRequestID() string {
|
||||
var b [16]byte
|
||||
if _, err := rand.Read(b[:]); err != nil {
|
||||
return strconv.FormatInt(time.Now().UnixNano(), 16)
|
||||
}
|
||||
return hex.EncodeToString(b[:])
|
||||
}
|
||||
|
||||
type traceIDKey struct{}
|
||||
|
||||
// TraceIDFrom returns the request's trace id, or false outside a chassis-handled
|
||||
// request. shared/httpclient uses it to propagate traceparent downstream.
|
||||
func TraceIDFrom(ctx context.Context) (string, bool) {
|
||||
id, ok := ctx.Value(traceIDKey{}).(string)
|
||||
return id, ok
|
||||
}
|
||||
|
||||
// traceIDFromHeader extracts the 32-hex trace-id from a W3C traceparent header
|
||||
// ("00-<32hex>-<16hex>-<flags>"); on absence/malformation it generates one.
|
||||
func traceIDFromHeader(traceparent string) string {
|
||||
parts := strings.Split(traceparent, "-")
|
||||
if len(parts) == 4 && len(parts[1]) == 32 && parts[1] != strings.Repeat("0", 32) {
|
||||
if _, err := hex.DecodeString(parts[1]); err == nil {
|
||||
return parts[1]
|
||||
}
|
||||
}
|
||||
return newRequestID()
|
||||
}
|
||||
|
||||
// instrument is the core edge middleware: it assigns/propagates a request_id,
|
||||
// builds the request-scoped logger, applies the per-request timeout, counts
|
||||
// in-flight requests, RECOVERS panics (so the access log + metrics observe the
|
||||
// real 500), and emits one access line + RED metrics. The route label is the
|
||||
// matched ServeMux pattern (bounded cardinality), never the raw path.
|
||||
func (a *App) instrument(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.Header.Get("X-Request-Id")
|
||||
if id == "" {
|
||||
id = newRequestID()
|
||||
}
|
||||
w.Header().Set("X-Request-Id", id)
|
||||
|
||||
// Derive the trace id from W3C traceparent (or generate one) so logs +
|
||||
// downstream calls correlate. trace_id is a regular log field, NEVER a
|
||||
// stream field (high cardinality).
|
||||
trace := traceIDFromHeader(r.Header.Get("traceparent"))
|
||||
w.Header().Set("X-Trace-Id", trace)
|
||||
|
||||
log := a.log.With("request_id", id, "trace_id", trace)
|
||||
ctx := context.WithValue(r.Context(), traceIDKey{}, trace)
|
||||
ctx = logging.Into(ctx, log)
|
||||
if a.cfg.RequestTimeout > 0 {
|
||||
// A deadline cannot be removed from a derived context, so the
|
||||
// pre-timeout one is carried alongside for Context.Stream: a
|
||||
// server-sent-events response is open for minutes by design and
|
||||
// must stay bound to client disconnect and drain, not to the
|
||||
// per-request budget every other route wants.
|
||||
ctx = context.WithValue(ctx, untimedKey{}, ctx)
|
||||
var cancel context.CancelFunc
|
||||
ctx, cancel = context.WithTimeout(ctx, a.cfg.RequestTimeout)
|
||||
defer cancel()
|
||||
}
|
||||
r2 := r.WithContext(ctx)
|
||||
|
||||
rec := &statusRecorder{ResponseWriter: w, status: http.StatusOK}
|
||||
a.metrics.inflight.Inc()
|
||||
defer a.metrics.inflight.Dec()
|
||||
start := time.Now()
|
||||
|
||||
func() {
|
||||
defer func() {
|
||||
if rv := recover(); rv != nil {
|
||||
log.Error("panic.recovered", "category", "panic",
|
||||
"error_type", "panic", "error_msg", fmt.Sprint(rv),
|
||||
"stack", string(debug.Stack()))
|
||||
if !rec.wrote {
|
||||
rec.Header().Set("Content-Type", "application/json; charset=utf-8")
|
||||
rec.WriteHeader(http.StatusInternalServerError)
|
||||
// Same envelope as writeError, request_id included. A panic
|
||||
// is precisely when a user needs an id to quote, so this is
|
||||
// the worst response to leave it out of.
|
||||
_, _ = fmt.Fprintf(rec,
|
||||
`{"error":{"code":"internal","message":"internal error","request_id":%q}}`, id)
|
||||
}
|
||||
}
|
||||
}()
|
||||
next.ServeHTTP(rec, r2)
|
||||
}()
|
||||
|
||||
route := routeLabel(r2.Pattern)
|
||||
dur := time.Since(start)
|
||||
log.Debug("http.request",
|
||||
"http_method", r.Method, "http_route", route, "http_path", r.URL.Path,
|
||||
"http_status", rec.status, "http_duration_ms", dur.Milliseconds(),
|
||||
"http_streamed", rec.streamed)
|
||||
// A stream's elapsed time is how long an operator left a tab open, not
|
||||
// how long the service took to answer. Feeding it to the latency
|
||||
// histogram fired HighRequestLatency the first time somebody opened the
|
||||
// console and would poison every latency panel for the whole service,
|
||||
// so a streamed response is counted but not timed.
|
||||
if rec.streamed {
|
||||
a.metrics.countOnly(r.Method, route, rec.status)
|
||||
return
|
||||
}
|
||||
a.metrics.observe(r.Method, route, rec.status, dur)
|
||||
})
|
||||
}
|
||||
|
||||
// routeLabel reduces a matched ServeMux pattern ("GET /v1/x/{id}") to its path
|
||||
// shape ("/v1/x/{id}") for a bounded metrics/log label — never the raw path.
|
||||
func routeLabel(pattern string) string {
|
||||
if pattern == "" {
|
||||
return "other"
|
||||
}
|
||||
if i := strings.IndexByte(pattern, ' '); i >= 0 { // drop the "METHOD " prefix
|
||||
pattern = pattern[i+1:]
|
||||
}
|
||||
return pattern
|
||||
}
|
||||
|
||||
// secureHeaders sets conservative response headers for a JSON API (no inline
|
||||
// content, no framing). HSTS is emitted only in prod, where TLS terminates.
|
||||
func (a *App) secureHeaders(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
h := w.Header()
|
||||
h.Set("X-Content-Type-Options", "nosniff")
|
||||
h.Set("X-Frame-Options", "DENY")
|
||||
h.Set("Referrer-Policy", "no-referrer")
|
||||
h.Set("Content-Security-Policy", "default-src 'none'; frame-ancestors 'none'")
|
||||
if a.cfg.Env == "prod" {
|
||||
h.Set("Strict-Transport-Security", "max-age=31536000; includeSubDomains")
|
||||
}
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
|
||||
// cors echoes the request Origin when it is on the allowlist and answers
|
||||
// preflight. A platform serves more than one browser surface (admin, tenant,
|
||||
// operator), so the allowlist is a set and the response echoes the matched
|
||||
// origin rather than a single fixed value — echoing a fixed origin breaks every
|
||||
// UI but one, and echoing the request unchecked is an open CORS policy.
|
||||
func (a *App) cors(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
h := w.Header()
|
||||
// The response body depends on Origin whenever an allowlist is
|
||||
// configured, so Vary is set even on a non-match — otherwise a shared
|
||||
// cache can serve an allowed origin's response to a denied one.
|
||||
if len(a.cfg.AllowOrigins) > 0 {
|
||||
h.Add("Vary", "Origin")
|
||||
}
|
||||
if origin := r.Header.Get("Origin"); origin != "" && a.originAllowed(origin) {
|
||||
h.Set("Access-Control-Allow-Origin", origin)
|
||||
h.Set("Access-Control-Allow-Methods", "GET, POST, PUT, PATCH, DELETE, OPTIONS")
|
||||
h.Set("Access-Control-Allow-Headers", "Content-Type, Authorization, X-Request-Id, Idempotency-Key")
|
||||
// Without this the browser can send X-Request-Id but cannot READ the
|
||||
// one we echo back, so a cross-origin SPA has no id to show the user
|
||||
// next to an error toast. Allow-Headers governs the request; only
|
||||
// Expose-Headers governs what JS may read off the response.
|
||||
h.Set("Access-Control-Expose-Headers", "X-Request-Id, X-Trace-Id")
|
||||
h.Set("Access-Control-Max-Age", "600")
|
||||
}
|
||||
if r.Method == http.MethodOptions {
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
return
|
||||
}
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
|
||||
// originAllowed reports whether origin is on the configured allowlist. Matching
|
||||
// is exact: no suffix or wildcard matching, because "endswith example.com"
|
||||
// also matches "evil-example.com".
|
||||
func (a *App) originAllowed(origin string) bool {
|
||||
for _, allowed := range a.cfg.AllowOrigins {
|
||||
if allowed == origin {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
44
vendor/github.com/orchard9/go-chassis/chassis/pagination.go
generated
vendored
Normal file
44
vendor/github.com/orchard9/go-chassis/chassis/pagination.go
generated
vendored
Normal file
@ -0,0 +1,44 @@
|
||||
package chassis
|
||||
|
||||
import "strconv"
|
||||
|
||||
// Page is a parsed cursor-pagination request (?limit=&cursor=). Cursor is opaque
|
||||
// to the chassis; handlers encode/decode it (e.g. an id or keyset token).
|
||||
type Page struct {
|
||||
Limit int
|
||||
Cursor string
|
||||
}
|
||||
|
||||
// Page parses pagination params from the query string, clamping limit to
|
||||
// [1, maxLimit] and defaulting to defLimit.
|
||||
func (c *Context) Page(defLimit, maxLimit int) Page {
|
||||
q := c.r.URL.Query()
|
||||
p := Page{Limit: defLimit, Cursor: q.Get("cursor")}
|
||||
if v := q.Get("limit"); v != "" {
|
||||
if n, err := strconv.Atoi(v); err == nil && n > 0 {
|
||||
p.Limit = n
|
||||
}
|
||||
}
|
||||
if p.Limit > maxLimit {
|
||||
p.Limit = maxLimit
|
||||
}
|
||||
if p.Limit < 1 {
|
||||
p.Limit = 1
|
||||
}
|
||||
return p
|
||||
}
|
||||
|
||||
// ListResponse is the standard list envelope. NextCursor is "" on the last page.
|
||||
type ListResponse[T any] struct {
|
||||
Items []T `json:"items"`
|
||||
NextCursor string `json:"next_cursor,omitempty"`
|
||||
}
|
||||
|
||||
// List builds a ListResponse, normalizing a nil slice to []. nextCursor is the
|
||||
// token a client passes as ?cursor= to fetch the next page ("" = no more).
|
||||
func List[T any](items []T, nextCursor string) ListResponse[T] {
|
||||
if items == nil {
|
||||
items = []T{}
|
||||
}
|
||||
return ListResponse[T]{Items: items, NextCursor: nextCursor}
|
||||
}
|
||||
89
vendor/github.com/orchard9/go-chassis/chassis/ratelimit.go
generated
vendored
Normal file
89
vendor/github.com/orchard9/go-chassis/chassis/ratelimit.go
generated
vendored
Normal file
@ -0,0 +1,89 @@
|
||||
package chassis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"math"
|
||||
"net"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// RateLimiter decides whether a key may proceed now. Back it with a Redis token
|
||||
// bucket in shared/adapters/redis (tiered by actor). retryAfter hints when to
|
||||
// retry; it is surfaced as the Retry-After header on a 429.
|
||||
type RateLimiter interface {
|
||||
Allow(ctx context.Context, key string) (allowed bool, retryAfter time.Duration, err error)
|
||||
}
|
||||
|
||||
// RateLimit rejects requests over the limit with 429 + Retry-After. keyFn maps a
|
||||
// request to a bucket key (default: authenticated subject, else client IP). The
|
||||
// limiter is best-effort: a backend error fails open (request proceeds) so a
|
||||
// degraded Redis can't take down the API.
|
||||
func RateLimit(limiter RateLimiter, keyFn func(*Context) string) Middleware {
|
||||
if keyFn == nil {
|
||||
keyFn = DefaultRateKey
|
||||
}
|
||||
return func(next HandlerFunc) HandlerFunc {
|
||||
return func(c *Context) error {
|
||||
allowed, retryAfter, err := limiter.Allow(c.r.Context(), keyFn(c))
|
||||
if err != nil {
|
||||
c.Log().Warn("ratelimit.failed", "category", "ratelimit", "error_msg", err.Error())
|
||||
return next(c)
|
||||
}
|
||||
if !allowed {
|
||||
if retryAfter > 0 {
|
||||
c.w.Header().Set("Retry-After", strconv.Itoa(int(math.Ceil(retryAfter.Seconds()))))
|
||||
}
|
||||
return TooManyRequests("rate limit exceeded")
|
||||
}
|
||||
return next(c)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// DefaultRateKey buckets by authenticated subject when present, else by client
|
||||
// IP. It does NOT trust X-Forwarded-For (trustedHops=0), so a spoofed XFF can't
|
||||
// mint a fresh bucket per request. Behind N trusted proxies/LBs, use RateKey(N).
|
||||
func DefaultRateKey(c *Context) string { return rateKey(c, 0) }
|
||||
|
||||
// RateKey returns a key function for a deployment behind trustedHops proxies/LBs
|
||||
// (the IP is taken trustedHops entries from the right of X-Forwarded-For).
|
||||
func RateKey(trustedHops int) func(*Context) string {
|
||||
return func(c *Context) string { return rateKey(c, trustedHops) }
|
||||
}
|
||||
|
||||
func rateKey(c *Context, trustedHops int) string {
|
||||
if id, ok := c.Identity(); ok && id.Subject != "" && id.Subject != "anonymous" {
|
||||
return "sub:" + id.Subject
|
||||
}
|
||||
return "ip:" + clientIP(c.r, trustedHops)
|
||||
}
|
||||
|
||||
// clientIP derives the client address. With trustedHops<=0 it uses the connection
|
||||
// RemoteAddr (XFF is attacker-controlled and ignored). Behind trustedHops trusted
|
||||
// proxies it takes the entry that many hops from the right of X-Forwarded-For —
|
||||
// the first address the trust boundary did not append.
|
||||
func clientIP(r *http.Request, trustedHops int) string {
|
||||
host, _, err := net.SplitHostPort(r.RemoteAddr)
|
||||
if err != nil {
|
||||
host = r.RemoteAddr
|
||||
}
|
||||
if trustedHops <= 0 {
|
||||
return host
|
||||
}
|
||||
xff := r.Header.Get("X-Forwarded-For")
|
||||
if xff == "" {
|
||||
return host
|
||||
}
|
||||
parts := strings.Split(xff, ",")
|
||||
idx := len(parts) - trustedHops
|
||||
if idx < 0 || idx >= len(parts) {
|
||||
return host
|
||||
}
|
||||
if v := strings.TrimSpace(parts[idx]); v != "" {
|
||||
return v
|
||||
}
|
||||
return host
|
||||
}
|
||||
154
vendor/github.com/orchard9/go-chassis/chassis/stream.go
generated
vendored
Normal file
154
vendor/github.com/orchard9/go-chassis/chassis/stream.go
generated
vendored
Normal file
@ -0,0 +1,154 @@
|
||||
package chassis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"time"
|
||||
)
|
||||
|
||||
// untimedKey carries the pre-deadline request context so Stream can shed the
|
||||
// per-request budget. See instrument.
|
||||
type untimedKey struct{}
|
||||
|
||||
// heartbeatEvery bounds how long a stream stays silent. Any proxy between the
|
||||
// board and a browser will drop an idle connection eventually — Traefik's
|
||||
// default is 0 (never) but Cloudflare's is 100s and a corporate egress proxy's
|
||||
// is anyone's guess — so a comment frame goes out on every tick even when the
|
||||
// board state has not moved. It is two bytes and it is the difference between
|
||||
// a live dashboard and one that silently stopped updating an hour ago.
|
||||
const heartbeatEvery = 20 * time.Second
|
||||
|
||||
// Stream is an open server-sent-events response.
|
||||
//
|
||||
// SSE rather than a websocket because the traffic is strictly one-way (the
|
||||
// board pushes state, the browser never talks back), it survives every proxy
|
||||
// that speaks HTTP/1.1, and EventSource reconnects on its own — so a board
|
||||
// restart costs the dashboard a few seconds rather than a page reload.
|
||||
type Stream struct {
|
||||
w http.ResponseWriter
|
||||
rc *http.ResponseController
|
||||
ctx context.Context
|
||||
closing <-chan struct{}
|
||||
}
|
||||
|
||||
// Stream converts the response into an event stream and returns a handle.
|
||||
//
|
||||
// It clears this connection's write deadline (the server sets one from
|
||||
// RequestTimeout for every other route) and detaches from the per-request
|
||||
// context deadline, leaving the stream bound to exactly two things: the client
|
||||
// hanging up, and the server beginning to drain.
|
||||
//
|
||||
// The handler MUST NOT write to the Context afterwards — the response is
|
||||
// committed the moment this returns.
|
||||
func (c *Context) Stream() (*Stream, error) {
|
||||
w := c.w
|
||||
rc := http.NewResponseController(w)
|
||||
// A stream lives past any per-request deadline by definition. This is the
|
||||
// call that needs statusRecorder.Unwrap; without it the connection is cut
|
||||
// mid-stream at RequestTimeout+socketHeadroom.
|
||||
if err := rc.SetWriteDeadline(time.Time{}); err != nil {
|
||||
return nil, fmt.Errorf("chassis: stream needs a deadline-capable writer: %w", err)
|
||||
}
|
||||
// Tell the edge not to time this response. How long a stream stays open is
|
||||
// how long an operator left a tab open; in the latency histogram it fires
|
||||
// HighRequestLatency and drags every percentile for the whole service.
|
||||
if rec, ok := w.(*statusRecorder); ok {
|
||||
rec.streamed = true
|
||||
}
|
||||
|
||||
ctx := c.r.Context()
|
||||
if untimed, ok := ctx.Value(untimedKey{}).(context.Context); ok {
|
||||
ctx = untimed
|
||||
}
|
||||
|
||||
h := w.Header()
|
||||
h.Set("Content-Type", "text/event-stream")
|
||||
h.Set("Cache-Control", "no-cache, no-transform")
|
||||
h.Set("Connection", "keep-alive")
|
||||
// Traefik does not buffer, but this response passes through whatever the
|
||||
// operator puts in front of it and an accumulating proxy turns a live
|
||||
// stream into a batch delivered at close.
|
||||
h.Set("X-Accel-Buffering", "no")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
// Past this point the response is committed, so a flush failure must not
|
||||
// become a returned error — the chassis would write a JSON envelope on top
|
||||
// of a 200 event stream. The first Send surfaces a dead connection.
|
||||
_ = rc.Flush()
|
||||
|
||||
return &Stream{w: w, rc: rc, ctx: ctx, closing: c.closing}, nil
|
||||
}
|
||||
|
||||
// Context is the stream's lifetime: cancelled when the client disconnects.
|
||||
func (s *Stream) Context() context.Context { return s.ctx }
|
||||
|
||||
// Send JSON-encodes v as one named event and flushes it. A write error means
|
||||
// the client is gone; the caller returns and the handler ends.
|
||||
func (s *Stream) Send(event string, v any) error {
|
||||
body, err := json.Marshal(v)
|
||||
if err != nil {
|
||||
return fmt.Errorf("chassis: stream encode %s: %w", event, err)
|
||||
}
|
||||
// The payload is compact JSON from encoding/json, so it contains no raw
|
||||
// newline and needs no multi-line data: continuation.
|
||||
if _, err := fmt.Fprintf(s.w, "event: %s\ndata: %s\n\n", event, body); err != nil {
|
||||
return err
|
||||
}
|
||||
return s.rc.Flush()
|
||||
}
|
||||
|
||||
// Run pushes an initial frame, then one per tick, until the client disconnects
|
||||
// or the server drains. It returns nil on every ordinary end — a browser
|
||||
// closing a tab is not a server error and must not be logged as one.
|
||||
func (s *Stream) Run(every time.Duration, frame func(context.Context) (any, error)) error {
|
||||
send := func() error {
|
||||
v, err := frame(s.ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return s.Send("state", v)
|
||||
}
|
||||
|
||||
if err := send(); err != nil {
|
||||
return s.classify(err)
|
||||
}
|
||||
|
||||
tick := time.NewTicker(every)
|
||||
defer tick.Stop()
|
||||
beat := time.NewTicker(heartbeatEvery)
|
||||
defer beat.Stop()
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-s.ctx.Done():
|
||||
return nil
|
||||
case <-s.closing:
|
||||
// Tell the browser to come back rather than letting it infer a
|
||||
// dead board from a closed socket.
|
||||
_ = s.Send("bye", map[string]string{"reason": "draining"})
|
||||
return nil
|
||||
case <-beat.C:
|
||||
if _, err := fmt.Fprint(s.w, ": ping\n\n"); err != nil {
|
||||
return nil
|
||||
}
|
||||
if err := s.rc.Flush(); err != nil {
|
||||
return nil
|
||||
}
|
||||
case <-tick.C:
|
||||
if err := send(); err != nil {
|
||||
return s.classify(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// classify swallows the errors that mean "the client left". A disconnect races
|
||||
// every write, so treating it as a failure would fill the log with 500s every
|
||||
// time somebody closes a dashboard tab.
|
||||
func (s *Stream) classify(err error) error {
|
||||
if s.ctx.Err() != nil {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
147
vendor/github.com/orchard9/go-chassis/config/config.go
generated
vendored
Normal file
147
vendor/github.com/orchard9/go-chassis/config/config.go
generated
vendored
Normal file
@ -0,0 +1,147 @@
|
||||
// Package config is the typed, validated environment loader used at the
|
||||
// composition root (services/*/main.go, workers/*/main.go) — the ONLY place env
|
||||
// is read. A Loader accumulates parse/validation errors so boot fails closed
|
||||
// with one actionable message instead of silently running on bad config.
|
||||
//
|
||||
// Secrets (DB passwords, API keys) do NOT belong here — they come from a
|
||||
// secret source (manager / file). This loads non-secret wiring only.
|
||||
package config
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Loader reads env vars with defaults and validation, collecting every error.
|
||||
type Loader struct {
|
||||
errs []error
|
||||
}
|
||||
|
||||
// New returns an empty Loader.
|
||||
func New() *Loader { return &Loader{} }
|
||||
|
||||
// String returns the env value or def when unset/empty.
|
||||
func (l *Loader) String(key, def string) string {
|
||||
if v := strings.TrimSpace(os.Getenv(key)); v != "" {
|
||||
return v
|
||||
}
|
||||
return def
|
||||
}
|
||||
|
||||
// Strings splits a comma-separated env value, trimming each entry and dropping
|
||||
// empties, or returns def when unset. Used for allowlists (CORS origins,
|
||||
// trusted proxies) where a platform has more than one legitimate value and a
|
||||
// single-string field would silently serve only the first.
|
||||
func (l *Loader) Strings(key string, def []string) []string {
|
||||
raw := strings.TrimSpace(os.Getenv(key))
|
||||
if raw == "" {
|
||||
return def
|
||||
}
|
||||
parts := strings.Split(raw, ",")
|
||||
out := make([]string, 0, len(parts))
|
||||
for _, p := range parts {
|
||||
if v := strings.TrimSpace(p); v != "" {
|
||||
out = append(out, v)
|
||||
}
|
||||
}
|
||||
if len(out) == 0 {
|
||||
return def
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Required returns the env value or records an error when unset/empty.
|
||||
func (l *Loader) Required(key string) string {
|
||||
v := strings.TrimSpace(os.Getenv(key))
|
||||
if v == "" {
|
||||
l.errs = append(l.errs, fmt.Errorf("%s is required", key))
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
// Int parses an integer env value, recording an error on a malformed value.
|
||||
func (l *Loader) Int(key string, def int) int {
|
||||
raw := strings.TrimSpace(os.Getenv(key))
|
||||
if raw == "" {
|
||||
return def
|
||||
}
|
||||
n, err := strconv.Atoi(raw)
|
||||
if err != nil {
|
||||
l.errs = append(l.errs, fmt.Errorf("%s: invalid int %q", key, raw))
|
||||
return def
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// Int64 parses a 64-bit integer env value, recording an error on a malformed
|
||||
// value. Separate from Int because byte-size limits (body caps, artifact
|
||||
// ceilings) legitimately exceed a 32-bit int on some platforms.
|
||||
func (l *Loader) Int64(key string, def int64) int64 {
|
||||
raw := strings.TrimSpace(os.Getenv(key))
|
||||
if raw == "" {
|
||||
return def
|
||||
}
|
||||
n, err := strconv.ParseInt(raw, 10, 64)
|
||||
if err != nil {
|
||||
l.errs = append(l.errs, fmt.Errorf("%s: invalid int64 %q", key, raw))
|
||||
return def
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// Port parses and bounds-checks a TCP port (1..65535).
|
||||
func (l *Loader) Port(key string, def int) int {
|
||||
n := l.Int(key, def)
|
||||
if n < 1 || n > 65535 {
|
||||
l.errs = append(l.errs, fmt.Errorf("%s: port %d out of range 1..65535", key, n))
|
||||
return def
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// Duration parses a Go duration (e.g. 5s, 1h), recording an error if malformed.
|
||||
func (l *Loader) Duration(key string, def time.Duration) time.Duration {
|
||||
raw := strings.TrimSpace(os.Getenv(key))
|
||||
if raw == "" {
|
||||
return def
|
||||
}
|
||||
d, err := time.ParseDuration(raw)
|
||||
if err != nil {
|
||||
l.errs = append(l.errs, fmt.Errorf("%s: invalid duration %q", key, raw))
|
||||
return def
|
||||
}
|
||||
return d
|
||||
}
|
||||
|
||||
// Bool parses a boolean (1/t/true/0/f/false), recording an error if malformed.
|
||||
func (l *Loader) Bool(key string, def bool) bool {
|
||||
raw := strings.TrimSpace(os.Getenv(key))
|
||||
if raw == "" {
|
||||
return def
|
||||
}
|
||||
b, err := strconv.ParseBool(raw)
|
||||
if err != nil {
|
||||
l.errs = append(l.errs, fmt.Errorf("%s: invalid bool %q", key, raw))
|
||||
return def
|
||||
}
|
||||
return b
|
||||
}
|
||||
|
||||
// OneOf returns the env value when it is in allowed, else records an error.
|
||||
func (l *Loader) OneOf(key, def string, allowed ...string) string {
|
||||
v := l.String(key, def)
|
||||
for _, a := range allowed {
|
||||
if v == a {
|
||||
return v
|
||||
}
|
||||
}
|
||||
l.errs = append(l.errs, fmt.Errorf("%s: %q not one of %s", key, v, strings.Join(allowed, "|")))
|
||||
return def
|
||||
}
|
||||
|
||||
// Err returns the joined validation errors, or nil when the config is clean.
|
||||
func (l *Loader) Err() error { return errors.Join(l.errs...) }
|
||||
25
vendor/github.com/orchard9/go-chassis/logging/context.go
generated
vendored
Normal file
25
vendor/github.com/orchard9/go-chassis/logging/context.go
generated
vendored
Normal file
@ -0,0 +1,25 @@
|
||||
package logging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log/slog"
|
||||
)
|
||||
|
||||
type ctxKey int
|
||||
|
||||
const loggerKey ctxKey = iota
|
||||
|
||||
// Into returns a child context carrying log, so request-scoped fields
|
||||
// (request_id, route, ...) added at the edge flow to every downstream call.
|
||||
func Into(ctx context.Context, log *slog.Logger) context.Context {
|
||||
return context.WithValue(ctx, loggerKey, log)
|
||||
}
|
||||
|
||||
// From returns the request-scoped logger stored in ctx, or the package fallback
|
||||
// (set by SetFallback at boot) so a missing logger never panics or drops logs.
|
||||
func From(ctx context.Context) *slog.Logger {
|
||||
if log, ok := ctx.Value(loggerKey).(*slog.Logger); ok && log != nil {
|
||||
return log
|
||||
}
|
||||
return fallback()
|
||||
}
|
||||
139
vendor/github.com/orchard9/go-chassis/logging/logging.go
generated
vendored
Normal file
139
vendor/github.com/orchard9/go-chassis/logging/logging.go
generated
vendored
Normal file
@ -0,0 +1,139 @@
|
||||
// Package logging is the core structured logger: one JSON object per line on
|
||||
// stdout, ready for any log agent (Fluent Bit, vector, the cloud's native
|
||||
// collector) to ship. Fields: ts (ISO-8601 UTC, ms, Z), level, service + env
|
||||
// (the only indexed stream fields — keep them closed enums), msg, plus caller
|
||||
// attrs. Secrets/PII MUST NOT be logged; forbiddenKeys redacts common offenders
|
||||
// as defense-in-depth behind code review + lint. See patterns/go-chassis.md.
|
||||
package logging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
"log/slog"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// LevelCritical sits above slog.LevelError; it maps to the "critical" enum and
|
||||
// is reserved for fail-closed boot/serve refusal (wire it to a page).
|
||||
const LevelCritical = slog.Level(12)
|
||||
|
||||
// Config selects the closed-enum stream fields the log store indexes.
|
||||
type Config struct {
|
||||
Service string // closed enum, keep small (e.g. api, job-worker)
|
||||
Env string // dev | staging | prod
|
||||
}
|
||||
|
||||
// New returns the core logger writing the JSON wire format to stdout.
|
||||
func New(cfg Config) *slog.Logger { return newTo(os.Stdout, cfg) }
|
||||
|
||||
// NewTo returns the core logger writing the JSON wire format to w. Server mode
|
||||
// uses New (stdout); a bounded maintenance command uses this to send its
|
||||
// operational diagnostics to stderr, keeping stdout a clean machine-readable
|
||||
// result stream a caller can parse without stripping log lines out of it.
|
||||
func NewTo(w io.Writer, cfg Config) *slog.Logger { return newTo(w, cfg) }
|
||||
|
||||
func newTo(w io.Writer, cfg Config) *slog.Logger {
|
||||
h := slog.NewJSONHandler(w, &slog.HandlerOptions{
|
||||
Level: slog.LevelDebug,
|
||||
ReplaceAttr: replace,
|
||||
})
|
||||
return slog.New(h).With("service", cfg.Service, "env", cfg.Env)
|
||||
}
|
||||
|
||||
func replace(groups []string, a slog.Attr) slog.Attr {
|
||||
if forbiddenKeys[a.Key] { // defense-in-depth secret/PII redaction
|
||||
return slog.String(a.Key, "[REDACTED]")
|
||||
}
|
||||
// The built-in rewrites below apply only to the record's own time/level
|
||||
// attrs, which slog always passes at the top level. A caller attr that
|
||||
// happens to be named "time" or "level" — log.Info("m", "level", "high") —
|
||||
// arrives here too, so both the group depth and the value kind are checked
|
||||
// before converting. An unchecked assertion here panics the process inside
|
||||
// the logger every binary depends on.
|
||||
if len(groups) > 0 {
|
||||
return a
|
||||
}
|
||||
switch a.Key {
|
||||
case slog.TimeKey:
|
||||
if a.Value.Kind() != slog.KindTime {
|
||||
return a
|
||||
}
|
||||
// ts: ISO-8601 UTC, ms precision, Z suffix.
|
||||
a.Key = "ts"
|
||||
a.Value = slog.StringValue(a.Value.Time().UTC().Format("2006-01-02T15:04:05.000Z07:00"))
|
||||
case slog.LevelKey:
|
||||
lvl, ok := a.Value.Any().(slog.Level)
|
||||
if !ok {
|
||||
return a
|
||||
}
|
||||
a.Key = "level"
|
||||
a.Value = slog.StringValue(levelString(lvl))
|
||||
}
|
||||
return a
|
||||
}
|
||||
|
||||
func levelString(l slog.Level) string {
|
||||
switch {
|
||||
case l >= LevelCritical:
|
||||
return "critical"
|
||||
case l >= slog.LevelError:
|
||||
return "error"
|
||||
case l >= slog.LevelWarn:
|
||||
return "warn"
|
||||
case l >= slog.LevelInfo:
|
||||
return "info"
|
||||
default:
|
||||
return "debug"
|
||||
}
|
||||
}
|
||||
|
||||
// forbiddenKeys are field names that MUST NOT reach the log corpus. Extend it
|
||||
// with YOUR product's sensitive fields (PII, PII, financial, tokens).
|
||||
var forbiddenKeys = map[string]bool{
|
||||
"password": true, "passwd": true, "secret": true, "token": true,
|
||||
"api_key": true, "apikey": true, "authorization": true, "cookie": true,
|
||||
"ssn": true, "email": true, "phone": true,
|
||||
}
|
||||
|
||||
var (
|
||||
fallbackMu sync.RWMutex
|
||||
fallbackLog *slog.Logger
|
||||
)
|
||||
|
||||
// SetFallback installs the logger From returns when no request-scoped logger is
|
||||
// in context. Set once at boot from the composition root.
|
||||
func SetFallback(l *slog.Logger) {
|
||||
fallbackMu.Lock()
|
||||
defer fallbackMu.Unlock()
|
||||
fallbackLog = l
|
||||
}
|
||||
|
||||
func fallback() *slog.Logger {
|
||||
fallbackMu.RLock()
|
||||
l := fallbackLog
|
||||
fallbackMu.RUnlock()
|
||||
if l != nil {
|
||||
return l
|
||||
}
|
||||
return New(Config{Service: "unknown", Env: "dev"})
|
||||
}
|
||||
|
||||
// Critical logs at the fail-closed level (boot/serve refusal). Bind to a page.
|
||||
func Critical(ctx context.Context, l *slog.Logger, msg string, args ...any) {
|
||||
l.Log(ctx, LevelCritical, msg, args...)
|
||||
}
|
||||
|
||||
// Env normalizes an APP_ENV value (local|dev|staging|prod, or a gcp-* / aws-*
|
||||
// prefix) to the log env enum.
|
||||
func Env(appEnv string) string {
|
||||
switch {
|
||||
case strings.HasSuffix(appEnv, "prod"):
|
||||
return "prod"
|
||||
case strings.HasSuffix(appEnv, "staging"):
|
||||
return "staging"
|
||||
default:
|
||||
return "dev"
|
||||
}
|
||||
}
|
||||
53
vendor/github.com/orchard9/go-chassis/logging/redact.go
generated
vendored
Normal file
53
vendor/github.com/orchard9/go-chassis/logging/redact.go
generated
vendored
Normal file
@ -0,0 +1,53 @@
|
||||
package logging
|
||||
|
||||
import "encoding/json"
|
||||
|
||||
// Forbidden reports whether key is a secrets/PII field name that must never be
|
||||
// persisted in cleartext (logs, audit snapshots, alert payloads).
|
||||
func Forbidden(key string) bool { return forbiddenKeys[key] }
|
||||
|
||||
// Redact returns "[REDACTED]" when key is forbidden, otherwise val. Use it when
|
||||
// projecting attributes into a non-log sink (e.g. alert events).
|
||||
func Redact(key, val string) string {
|
||||
if forbiddenKeys[key] {
|
||||
return "[REDACTED]"
|
||||
}
|
||||
return val
|
||||
}
|
||||
|
||||
// RedactJSON returns a copy of a JSON document with every forbidden key's value
|
||||
// (at any depth) replaced by "[REDACTED]". This is the defense-in-depth scrubber
|
||||
// the audit sink applies to before/after snapshots so PII cannot land in the
|
||||
// immutable audit log. Non-JSON or unparseable input is returned unchanged.
|
||||
func RedactJSON(b []byte) []byte {
|
||||
if len(b) == 0 {
|
||||
return b
|
||||
}
|
||||
var v any
|
||||
if err := json.Unmarshal(b, &v); err != nil {
|
||||
return b
|
||||
}
|
||||
redactValue(v)
|
||||
out, err := json.Marshal(v)
|
||||
if err != nil {
|
||||
return b
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func redactValue(v any) {
|
||||
switch t := v.(type) {
|
||||
case map[string]any:
|
||||
for k, val := range t {
|
||||
if forbiddenKeys[k] {
|
||||
t[k] = "[REDACTED]"
|
||||
continue
|
||||
}
|
||||
redactValue(val)
|
||||
}
|
||||
case []any:
|
||||
for _, e := range t {
|
||||
redactValue(e)
|
||||
}
|
||||
}
|
||||
}
|
||||
201
vendor/github.com/prometheus/client_golang/LICENSE
generated
vendored
Normal file
201
vendor/github.com/prometheus/client_golang/LICENSE
generated
vendored
Normal file
@ -0,0 +1,201 @@
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright [yyyy] [name of copyright owner]
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
18
vendor/github.com/prometheus/client_golang/NOTICE
generated
vendored
Normal file
18
vendor/github.com/prometheus/client_golang/NOTICE
generated
vendored
Normal file
@ -0,0 +1,18 @@
|
||||
Prometheus instrumentation library for Go applications
|
||||
Copyright 2012-2015 The Prometheus Authors
|
||||
|
||||
This product includes software developed at
|
||||
SoundCloud Ltd. (http://soundcloud.com/).
|
||||
|
||||
|
||||
The following components are included in this product:
|
||||
|
||||
perks - a fork of https://github.com/bmizerany/perks
|
||||
https://github.com/beorn7/perks
|
||||
Copyright 2013-2015 Blake Mizerany, Björn Rabenstein
|
||||
See https://github.com/beorn7/perks/blob/master/README.md for license details.
|
||||
|
||||
Go support for Protocol Buffers - Google's data interchange format
|
||||
http://github.com/golang/protobuf/
|
||||
Copyright 2010 The Go Authors
|
||||
See source code for license details.
|
||||
27
vendor/github.com/prometheus/client_golang/internal/github.com/golang/gddo/LICENSE
generated
vendored
Normal file
27
vendor/github.com/prometheus/client_golang/internal/github.com/golang/gddo/LICENSE
generated
vendored
Normal file
@ -0,0 +1,27 @@
|
||||
Copyright (c) 2013 The Go Authors. All rights reserved.
|
||||
|
||||
Redistribution and use in source and binary forms, with or without
|
||||
modification, are permitted provided that the following conditions are
|
||||
met:
|
||||
|
||||
* Redistributions of source code must retain the above copyright
|
||||
notice, this list of conditions and the following disclaimer.
|
||||
* Redistributions in binary form must reproduce the above
|
||||
copyright notice, this list of conditions and the following disclaimer
|
||||
in the documentation and/or other materials provided with the
|
||||
distribution.
|
||||
* Neither the name of Google Inc. nor the names of its
|
||||
contributors may be used to endorse or promote products derived from
|
||||
this software without specific prior written permission.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
145
vendor/github.com/prometheus/client_golang/internal/github.com/golang/gddo/httputil/header/header.go
generated
vendored
Normal file
145
vendor/github.com/prometheus/client_golang/internal/github.com/golang/gddo/httputil/header/header.go
generated
vendored
Normal file
@ -0,0 +1,145 @@
|
||||
// Copyright 2013 The Go Authors. All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file or at
|
||||
// https://developers.google.com/open-source/licenses/bsd.
|
||||
|
||||
// Package header provides functions for parsing HTTP headers.
|
||||
package header
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Octet types from RFC 2616.
|
||||
var octetTypes [256]octetType
|
||||
|
||||
type octetType byte
|
||||
|
||||
const (
|
||||
isToken octetType = 1 << iota
|
||||
isSpace
|
||||
)
|
||||
|
||||
func init() {
|
||||
// OCTET = <any 8-bit sequence of data>
|
||||
// CHAR = <any US-ASCII character (octets 0 - 127)>
|
||||
// CTL = <any US-ASCII control character (octets 0 - 31) and DEL (127)>
|
||||
// CR = <US-ASCII CR, carriage return (13)>
|
||||
// LF = <US-ASCII LF, linefeed (10)>
|
||||
// SP = <US-ASCII SP, space (32)>
|
||||
// HT = <US-ASCII HT, horizontal-tab (9)>
|
||||
// <"> = <US-ASCII double-quote mark (34)>
|
||||
// CRLF = CR LF
|
||||
// LWS = [CRLF] 1*( SP | HT )
|
||||
// TEXT = <any OCTET except CTLs, but including LWS>
|
||||
// separators = "(" | ")" | "<" | ">" | "@" | "," | ";" | ":" | "\" | <">
|
||||
// | "/" | "[" | "]" | "?" | "=" | "{" | "}" | SP | HT
|
||||
// token = 1*<any CHAR except CTLs or separators>
|
||||
// qdtext = <any TEXT except <">>
|
||||
|
||||
for c := 0; c < 256; c++ {
|
||||
var t octetType
|
||||
isCtl := c <= 31 || c == 127
|
||||
isChar := 0 <= c && c <= 127
|
||||
isSeparator := strings.ContainsRune(" \t\"(),/:;<=>?@[]\\{}", rune(c))
|
||||
if strings.ContainsRune(" \t\r\n", rune(c)) {
|
||||
t |= isSpace
|
||||
}
|
||||
if isChar && !isCtl && !isSeparator {
|
||||
t |= isToken
|
||||
}
|
||||
octetTypes[c] = t
|
||||
}
|
||||
}
|
||||
|
||||
// AcceptSpec describes an Accept* header.
|
||||
type AcceptSpec struct {
|
||||
Value string
|
||||
Q float64
|
||||
}
|
||||
|
||||
// ParseAccept parses Accept* headers.
|
||||
func ParseAccept(header http.Header, key string) (specs []AcceptSpec) {
|
||||
loop:
|
||||
for _, s := range header[key] {
|
||||
for {
|
||||
var spec AcceptSpec
|
||||
spec.Value, s = expectTokenSlash(s)
|
||||
if spec.Value == "" {
|
||||
continue loop
|
||||
}
|
||||
spec.Q = 1.0
|
||||
s = skipSpace(s)
|
||||
if strings.HasPrefix(s, ";") {
|
||||
s = skipSpace(s[1:])
|
||||
if !strings.HasPrefix(s, "q=") {
|
||||
continue loop
|
||||
}
|
||||
spec.Q, s = expectQuality(s[2:])
|
||||
if spec.Q < 0.0 {
|
||||
continue loop
|
||||
}
|
||||
}
|
||||
specs = append(specs, spec)
|
||||
s = skipSpace(s)
|
||||
if !strings.HasPrefix(s, ",") {
|
||||
continue loop
|
||||
}
|
||||
s = skipSpace(s[1:])
|
||||
}
|
||||
}
|
||||
return specs
|
||||
}
|
||||
|
||||
func skipSpace(s string) (rest string) {
|
||||
i := 0
|
||||
for ; i < len(s); i++ {
|
||||
if octetTypes[s[i]]&isSpace == 0 {
|
||||
break
|
||||
}
|
||||
}
|
||||
return s[i:]
|
||||
}
|
||||
|
||||
func expectTokenSlash(s string) (token, rest string) {
|
||||
i := 0
|
||||
for ; i < len(s); i++ {
|
||||
b := s[i]
|
||||
if (octetTypes[b]&isToken == 0) && b != '/' {
|
||||
break
|
||||
}
|
||||
}
|
||||
return s[:i], s[i:]
|
||||
}
|
||||
|
||||
func expectQuality(s string) (q float64, rest string) {
|
||||
switch {
|
||||
case len(s) == 0:
|
||||
return -1, ""
|
||||
case s[0] == '0':
|
||||
q = 0
|
||||
case s[0] == '1':
|
||||
q = 1
|
||||
default:
|
||||
return -1, ""
|
||||
}
|
||||
s = s[1:]
|
||||
if !strings.HasPrefix(s, ".") {
|
||||
return q, s
|
||||
}
|
||||
s = s[1:]
|
||||
i := 0
|
||||
n := 0
|
||||
d := 1
|
||||
for ; i < len(s); i++ {
|
||||
b := s[i]
|
||||
if b < '0' || b > '9' {
|
||||
break
|
||||
}
|
||||
n = n*10 + int(b) - '0'
|
||||
d *= 10
|
||||
}
|
||||
return q + float64(n)/float64(d), s[i:]
|
||||
}
|
||||
36
vendor/github.com/prometheus/client_golang/internal/github.com/golang/gddo/httputil/negotiate.go
generated
vendored
Normal file
36
vendor/github.com/prometheus/client_golang/internal/github.com/golang/gddo/httputil/negotiate.go
generated
vendored
Normal file
@ -0,0 +1,36 @@
|
||||
// Copyright 2013 The Go Authors. All rights reserved.
|
||||
//
|
||||
// Use of this source code is governed by a BSD-style
|
||||
// license that can be found in the LICENSE file or at
|
||||
// https://developers.google.com/open-source/licenses/bsd.
|
||||
|
||||
package httputil
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
|
||||
"github.com/prometheus/client_golang/internal/github.com/golang/gddo/httputil/header"
|
||||
)
|
||||
|
||||
// NegotiateContentEncoding returns the best offered content encoding for the
|
||||
// request's Accept-Encoding header. If two offers match with equal weight and
|
||||
// then the offer earlier in the list is preferred. If no offers are
|
||||
// acceptable, then "" is returned.
|
||||
func NegotiateContentEncoding(r *http.Request, offers []string) string {
|
||||
bestOffer := "identity"
|
||||
bestQ := -1.0
|
||||
specs := header.ParseAccept(r.Header, "Accept-Encoding")
|
||||
for _, offer := range offers {
|
||||
for _, spec := range specs {
|
||||
if spec.Q > bestQ &&
|
||||
(spec.Value == "*" || spec.Value == offer) {
|
||||
bestQ = spec.Q
|
||||
bestOffer = offer
|
||||
}
|
||||
}
|
||||
}
|
||||
if bestQ == 0 {
|
||||
bestOffer = ""
|
||||
}
|
||||
return bestOffer
|
||||
}
|
||||
1
vendor/github.com/prometheus/client_golang/prometheus/.gitignore
generated
vendored
Normal file
1
vendor/github.com/prometheus/client_golang/prometheus/.gitignore
generated
vendored
Normal file
@ -0,0 +1 @@
|
||||
command-line-arguments.test
|
||||
1
vendor/github.com/prometheus/client_golang/prometheus/README.md
generated
vendored
Normal file
1
vendor/github.com/prometheus/client_golang/prometheus/README.md
generated
vendored
Normal file
@ -0,0 +1 @@
|
||||
See [](https://pkg.go.dev/github.com/prometheus/client_golang/prometheus).
|
||||
38
vendor/github.com/prometheus/client_golang/prometheus/build_info_collector.go
generated
vendored
Normal file
38
vendor/github.com/prometheus/client_golang/prometheus/build_info_collector.go
generated
vendored
Normal file
@ -0,0 +1,38 @@
|
||||
// Copyright 2021 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package prometheus
|
||||
|
||||
import "runtime/debug"
|
||||
|
||||
// NewBuildInfoCollector is the obsolete version of collectors.NewBuildInfoCollector.
|
||||
// See there for documentation.
|
||||
//
|
||||
// Deprecated: Use collectors.NewBuildInfoCollector instead.
|
||||
func NewBuildInfoCollector() Collector {
|
||||
path, version, sum := "unknown", "unknown", "unknown"
|
||||
if bi, ok := debug.ReadBuildInfo(); ok {
|
||||
path = bi.Main.Path
|
||||
version = bi.Main.Version
|
||||
sum = bi.Main.Sum
|
||||
}
|
||||
c := &selfCollector{MustNewConstMetric(
|
||||
NewDesc(
|
||||
"go_build_info",
|
||||
"Build information about the main Go module.",
|
||||
nil, Labels{"path": path, "version": version, "checksum": sum},
|
||||
),
|
||||
GaugeValue, 1)}
|
||||
c.init(c.self)
|
||||
return c
|
||||
}
|
||||
128
vendor/github.com/prometheus/client_golang/prometheus/collector.go
generated
vendored
Normal file
128
vendor/github.com/prometheus/client_golang/prometheus/collector.go
generated
vendored
Normal file
@ -0,0 +1,128 @@
|
||||
// Copyright 2014 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package prometheus
|
||||
|
||||
// Collector is the interface implemented by anything that can be used by
|
||||
// Prometheus to collect metrics. A Collector has to be registered for
|
||||
// collection. See Registerer.Register.
|
||||
//
|
||||
// The stock metrics provided by this package (Gauge, Counter, Summary,
|
||||
// Histogram, Untyped) are also Collectors (which only ever collect one metric,
|
||||
// namely itself). An implementer of Collector may, however, collect multiple
|
||||
// metrics in a coordinated fashion and/or create metrics on the fly. Examples
|
||||
// for collectors already implemented in this library are the metric vectors
|
||||
// (i.e. collection of multiple instances of the same Metric but with different
|
||||
// label values) like GaugeVec or SummaryVec, and the ExpvarCollector.
|
||||
type Collector interface {
|
||||
// Describe sends the super-set of all possible descriptors of metrics
|
||||
// collected by this Collector to the provided channel and returns once
|
||||
// the last descriptor has been sent. The sent descriptors fulfill the
|
||||
// consistency and uniqueness requirements described in the Desc
|
||||
// documentation.
|
||||
//
|
||||
// It is valid if one and the same Collector sends duplicate
|
||||
// descriptors. Those duplicates are simply ignored. However, two
|
||||
// different Collectors must not send duplicate descriptors.
|
||||
//
|
||||
// Sending no descriptor at all marks the Collector as “unchecked”,
|
||||
// i.e. no checks will be performed at registration time, and the
|
||||
// Collector may yield any Metric it sees fit in its Collect method.
|
||||
//
|
||||
// This method idempotently sends the same descriptors throughout the
|
||||
// lifetime of the Collector. It may be called concurrently and
|
||||
// therefore must be implemented in a concurrency safe way.
|
||||
//
|
||||
// If a Collector encounters an error while executing this method, it
|
||||
// must send an invalid descriptor (created with NewInvalidDesc) to
|
||||
// signal the error to the registry.
|
||||
Describe(chan<- *Desc)
|
||||
// Collect is called by the Prometheus registry when collecting
|
||||
// metrics. The implementation sends each collected metric via the
|
||||
// provided channel and returns once the last metric has been sent. The
|
||||
// descriptor of each sent metric is one of those returned by Describe
|
||||
// (unless the Collector is unchecked, see above). Returned metrics that
|
||||
// share the same descriptor must differ in their variable label
|
||||
// values.
|
||||
//
|
||||
// This method may be called concurrently and must therefore be
|
||||
// implemented in a concurrency safe way. Blocking occurs at the expense
|
||||
// of total performance of rendering all registered metrics. Ideally,
|
||||
// Collector implementations support concurrent readers.
|
||||
Collect(chan<- Metric)
|
||||
}
|
||||
|
||||
// DescribeByCollect is a helper to implement the Describe method of a custom
|
||||
// Collector. It collects the metrics from the provided Collector and sends
|
||||
// their descriptors to the provided channel.
|
||||
//
|
||||
// If a Collector collects the same metrics throughout its lifetime, its
|
||||
// Describe method can simply be implemented as:
|
||||
//
|
||||
// func (c customCollector) Describe(ch chan<- *Desc) {
|
||||
// DescribeByCollect(c, ch)
|
||||
// }
|
||||
//
|
||||
// However, this will not work if the metrics collected change dynamically over
|
||||
// the lifetime of the Collector in a way that their combined set of descriptors
|
||||
// changes as well. The shortcut implementation will then violate the contract
|
||||
// of the Describe method. If a Collector sometimes collects no metrics at all
|
||||
// (for example vectors like CounterVec, GaugeVec, etc., which only collect
|
||||
// metrics after a metric with a fully specified label set has been accessed),
|
||||
// it might even get registered as an unchecked Collector (cf. the Register
|
||||
// method of the Registerer interface). Hence, only use this shortcut
|
||||
// implementation of Describe if you are certain to fulfill the contract.
|
||||
//
|
||||
// The Collector example demonstrates a use of DescribeByCollect.
|
||||
func DescribeByCollect(c Collector, descs chan<- *Desc) {
|
||||
metrics := make(chan Metric)
|
||||
go func() {
|
||||
c.Collect(metrics)
|
||||
close(metrics)
|
||||
}()
|
||||
for m := range metrics {
|
||||
descs <- m.Desc()
|
||||
}
|
||||
}
|
||||
|
||||
// selfCollector implements Collector for a single Metric so that the Metric
|
||||
// collects itself. Add it as an anonymous field to a struct that implements
|
||||
// Metric, and call init with the Metric itself as an argument.
|
||||
type selfCollector struct {
|
||||
self Metric
|
||||
}
|
||||
|
||||
// init provides the selfCollector with a reference to the metric it is supposed
|
||||
// to collect. It is usually called within the factory function to create a
|
||||
// metric. See example.
|
||||
func (c *selfCollector) init(self Metric) {
|
||||
c.self = self
|
||||
}
|
||||
|
||||
// Describe implements Collector.
|
||||
func (c *selfCollector) Describe(ch chan<- *Desc) {
|
||||
ch <- c.self.Desc()
|
||||
}
|
||||
|
||||
// Collect implements Collector.
|
||||
func (c *selfCollector) Collect(ch chan<- Metric) {
|
||||
ch <- c.self
|
||||
}
|
||||
|
||||
// collectorMetric is a metric that is also a collector.
|
||||
// Because of selfCollector, most (if not all) Metrics in
|
||||
// this package are also collectors.
|
||||
type collectorMetric interface {
|
||||
Metric
|
||||
Collector
|
||||
}
|
||||
30
vendor/github.com/prometheus/client_golang/prometheus/collectorfunc.go
generated
vendored
Normal file
30
vendor/github.com/prometheus/client_golang/prometheus/collectorfunc.go
generated
vendored
Normal file
@ -0,0 +1,30 @@
|
||||
// Copyright 2025 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package prometheus
|
||||
|
||||
// CollectorFunc is a convenient way to implement a Prometheus Collector
|
||||
// without interface boilerplate.
|
||||
// This implementation is based on DescribeByCollect method.
|
||||
// familiarize yourself to it before using.
|
||||
type CollectorFunc func(chan<- Metric)
|
||||
|
||||
// Collect calls the defined CollectorFunc function with the provided Metrics channel
|
||||
func (f CollectorFunc) Collect(ch chan<- Metric) {
|
||||
f(ch)
|
||||
}
|
||||
|
||||
// Describe sends the descriptor information using DescribeByCollect
|
||||
func (f CollectorFunc) Describe(ch chan<- *Desc) {
|
||||
DescribeByCollect(f, ch)
|
||||
}
|
||||
40
vendor/github.com/prometheus/client_golang/prometheus/collectors/collectors.go
generated
vendored
Normal file
40
vendor/github.com/prometheus/client_golang/prometheus/collectors/collectors.go
generated
vendored
Normal file
@ -0,0 +1,40 @@
|
||||
// Copyright 2021 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
// Package collectors provides implementations of prometheus.Collector to
|
||||
// conveniently collect process and Go-related metrics.
|
||||
package collectors
|
||||
|
||||
import "github.com/prometheus/client_golang/prometheus"
|
||||
|
||||
// NewBuildInfoCollector returns a collector collecting a single metric
|
||||
// "go_build_info" with the constant value 1 and three labels "path", "version",
|
||||
// and "checksum". Their label values contain the main module path, version, and
|
||||
// checksum, respectively. The labels will only have meaningful values if the
|
||||
// binary is built with Go module support and from source code retrieved from
|
||||
// the source repository (rather than the local file system). This is usually
|
||||
// accomplished by building from outside of GOPATH, specifying the full address
|
||||
// of the main package, e.g. "GO111MODULE=on go run
|
||||
// github.com/prometheus/client_golang/examples/random". If built without Go
|
||||
// module support, all label values will be "unknown". If built with Go module
|
||||
// support but using the source code from the local file system, the "path" will
|
||||
// be set appropriately, but "checksum" will be empty and "version" will be
|
||||
// "(devel)".
|
||||
//
|
||||
// This collector uses only the build information for the main module. See
|
||||
// https://github.com/povilasv/prommod for an example of a collector for the
|
||||
// module dependencies.
|
||||
func NewBuildInfoCollector() prometheus.Collector {
|
||||
//nolint:staticcheck // Ignore SA1019 until v2.
|
||||
return prometheus.NewBuildInfoCollector()
|
||||
}
|
||||
119
vendor/github.com/prometheus/client_golang/prometheus/collectors/dbstats_collector.go
generated
vendored
Normal file
119
vendor/github.com/prometheus/client_golang/prometheus/collectors/dbstats_collector.go
generated
vendored
Normal file
@ -0,0 +1,119 @@
|
||||
// Copyright 2021 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package collectors
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
|
||||
"github.com/prometheus/client_golang/prometheus"
|
||||
)
|
||||
|
||||
type dbStatsCollector struct {
|
||||
db *sql.DB
|
||||
|
||||
maxOpenConnections *prometheus.Desc
|
||||
|
||||
openConnections *prometheus.Desc
|
||||
inUseConnections *prometheus.Desc
|
||||
idleConnections *prometheus.Desc
|
||||
|
||||
waitCount *prometheus.Desc
|
||||
waitDuration *prometheus.Desc
|
||||
maxIdleClosed *prometheus.Desc
|
||||
maxIdleTimeClosed *prometheus.Desc
|
||||
maxLifetimeClosed *prometheus.Desc
|
||||
}
|
||||
|
||||
// NewDBStatsCollector returns a collector that exports metrics about the given *sql.DB.
|
||||
// See https://golang.org/pkg/database/sql/#DBStats for more information on stats.
|
||||
func NewDBStatsCollector(db *sql.DB, dbName string) prometheus.Collector {
|
||||
fqName := func(name string) string {
|
||||
return "go_sql_" + name
|
||||
}
|
||||
return &dbStatsCollector{
|
||||
db: db,
|
||||
maxOpenConnections: prometheus.NewDesc(
|
||||
fqName("max_open_connections"),
|
||||
"Maximum number of open connections to the database.",
|
||||
nil, prometheus.Labels{"db_name": dbName},
|
||||
),
|
||||
openConnections: prometheus.NewDesc(
|
||||
fqName("open_connections"),
|
||||
"The number of established connections both in use and idle.",
|
||||
nil, prometheus.Labels{"db_name": dbName},
|
||||
),
|
||||
inUseConnections: prometheus.NewDesc(
|
||||
fqName("in_use_connections"),
|
||||
"The number of connections currently in use.",
|
||||
nil, prometheus.Labels{"db_name": dbName},
|
||||
),
|
||||
idleConnections: prometheus.NewDesc(
|
||||
fqName("idle_connections"),
|
||||
"The number of idle connections.",
|
||||
nil, prometheus.Labels{"db_name": dbName},
|
||||
),
|
||||
waitCount: prometheus.NewDesc(
|
||||
fqName("wait_count_total"),
|
||||
"The total number of connections waited for.",
|
||||
nil, prometheus.Labels{"db_name": dbName},
|
||||
),
|
||||
waitDuration: prometheus.NewDesc(
|
||||
fqName("wait_duration_seconds_total"),
|
||||
"The total time blocked waiting for a new connection.",
|
||||
nil, prometheus.Labels{"db_name": dbName},
|
||||
),
|
||||
maxIdleClosed: prometheus.NewDesc(
|
||||
fqName("max_idle_closed_total"),
|
||||
"The total number of connections closed due to SetMaxIdleConns.",
|
||||
nil, prometheus.Labels{"db_name": dbName},
|
||||
),
|
||||
maxIdleTimeClosed: prometheus.NewDesc(
|
||||
fqName("max_idle_time_closed_total"),
|
||||
"The total number of connections closed due to SetConnMaxIdleTime.",
|
||||
nil, prometheus.Labels{"db_name": dbName},
|
||||
),
|
||||
maxLifetimeClosed: prometheus.NewDesc(
|
||||
fqName("max_lifetime_closed_total"),
|
||||
"The total number of connections closed due to SetConnMaxLifetime.",
|
||||
nil, prometheus.Labels{"db_name": dbName},
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
// Describe implements Collector.
|
||||
func (c *dbStatsCollector) Describe(ch chan<- *prometheus.Desc) {
|
||||
ch <- c.maxOpenConnections
|
||||
ch <- c.openConnections
|
||||
ch <- c.inUseConnections
|
||||
ch <- c.idleConnections
|
||||
ch <- c.waitCount
|
||||
ch <- c.waitDuration
|
||||
ch <- c.maxIdleClosed
|
||||
ch <- c.maxLifetimeClosed
|
||||
ch <- c.maxIdleTimeClosed
|
||||
}
|
||||
|
||||
// Collect implements Collector.
|
||||
func (c *dbStatsCollector) Collect(ch chan<- prometheus.Metric) {
|
||||
stats := c.db.Stats()
|
||||
ch <- prometheus.MustNewConstMetric(c.maxOpenConnections, prometheus.GaugeValue, float64(stats.MaxOpenConnections))
|
||||
ch <- prometheus.MustNewConstMetric(c.openConnections, prometheus.GaugeValue, float64(stats.OpenConnections))
|
||||
ch <- prometheus.MustNewConstMetric(c.inUseConnections, prometheus.GaugeValue, float64(stats.InUse))
|
||||
ch <- prometheus.MustNewConstMetric(c.idleConnections, prometheus.GaugeValue, float64(stats.Idle))
|
||||
ch <- prometheus.MustNewConstMetric(c.waitCount, prometheus.CounterValue, float64(stats.WaitCount))
|
||||
ch <- prometheus.MustNewConstMetric(c.waitDuration, prometheus.CounterValue, stats.WaitDuration.Seconds())
|
||||
ch <- prometheus.MustNewConstMetric(c.maxIdleClosed, prometheus.CounterValue, float64(stats.MaxIdleClosed))
|
||||
ch <- prometheus.MustNewConstMetric(c.maxLifetimeClosed, prometheus.CounterValue, float64(stats.MaxLifetimeClosed))
|
||||
ch <- prometheus.MustNewConstMetric(c.maxIdleTimeClosed, prometheus.CounterValue, float64(stats.MaxIdleTimeClosed))
|
||||
}
|
||||
57
vendor/github.com/prometheus/client_golang/prometheus/collectors/expvar_collector.go
generated
vendored
Normal file
57
vendor/github.com/prometheus/client_golang/prometheus/collectors/expvar_collector.go
generated
vendored
Normal file
@ -0,0 +1,57 @@
|
||||
// Copyright 2021 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package collectors
|
||||
|
||||
import "github.com/prometheus/client_golang/prometheus"
|
||||
|
||||
// NewExpvarCollector returns a newly allocated expvar Collector.
|
||||
//
|
||||
// An expvar Collector collects metrics from the expvar interface. It provides a
|
||||
// quick way to expose numeric values that are already exported via expvar as
|
||||
// Prometheus metrics. Note that the data models of expvar and Prometheus are
|
||||
// fundamentally different, and that the expvar Collector is inherently slower
|
||||
// than native Prometheus metrics. Thus, the expvar Collector is probably great
|
||||
// for experiments and prototyping, but you should seriously consider a more
|
||||
// direct implementation of Prometheus metrics for monitoring production
|
||||
// systems.
|
||||
//
|
||||
// The exports map has the following meaning:
|
||||
//
|
||||
// The keys in the map correspond to expvar keys, i.e. for every expvar key you
|
||||
// want to export as Prometheus metric, you need an entry in the exports
|
||||
// map. The descriptor mapped to each key describes how to export the expvar
|
||||
// value. It defines the name and the help string of the Prometheus metric
|
||||
// proxying the expvar value. The type will always be Untyped.
|
||||
//
|
||||
// For descriptors without variable labels, the expvar value must be a number or
|
||||
// a bool. The number is then directly exported as the Prometheus sample
|
||||
// value. (For a bool, 'false' translates to 0 and 'true' to 1). Expvar values
|
||||
// that are not numbers or bools are silently ignored.
|
||||
//
|
||||
// If the descriptor has one variable label, the expvar value must be an expvar
|
||||
// map. The keys in the expvar map become the various values of the one
|
||||
// Prometheus label. The values in the expvar map must be numbers or bools again
|
||||
// as above.
|
||||
//
|
||||
// For descriptors with more than one variable label, the expvar must be a
|
||||
// nested expvar map, i.e. where the values of the topmost map are maps again
|
||||
// etc. until a depth is reached that corresponds to the number of labels. The
|
||||
// leaves of that structure must be numbers or bools as above to serve as the
|
||||
// sample values.
|
||||
//
|
||||
// Anything that does not fit into the scheme above is silently ignored.
|
||||
func NewExpvarCollector(exports map[string]*prometheus.Desc) prometheus.Collector {
|
||||
//nolint:staticcheck // Ignore SA1019 until v2.
|
||||
return prometheus.NewExpvarCollector(exports)
|
||||
}
|
||||
167
vendor/github.com/prometheus/client_golang/prometheus/collectors/go_collector_latest.go
generated
vendored
Normal file
167
vendor/github.com/prometheus/client_golang/prometheus/collectors/go_collector_latest.go
generated
vendored
Normal file
@ -0,0 +1,167 @@
|
||||
// Copyright 2021 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
//go:build go1.17
|
||||
// +build go1.17
|
||||
|
||||
package collectors
|
||||
|
||||
import (
|
||||
"regexp"
|
||||
|
||||
"github.com/prometheus/client_golang/prometheus"
|
||||
"github.com/prometheus/client_golang/prometheus/internal"
|
||||
)
|
||||
|
||||
var (
|
||||
// MetricsAll allows all the metrics to be collected from Go runtime.
|
||||
MetricsAll = GoRuntimeMetricsRule{regexp.MustCompile("/.*")}
|
||||
// MetricsGC allows only GC metrics to be collected from Go runtime.
|
||||
// e.g. go_gc_cycles_automatic_gc_cycles_total
|
||||
// NOTE: This does not include new class of "/cpu/classes/gc/..." metrics.
|
||||
// Use custom metric rule to access those.
|
||||
MetricsGC = GoRuntimeMetricsRule{regexp.MustCompile(`^/gc/.*`)}
|
||||
// MetricsMemory allows only memory metrics to be collected from Go runtime.
|
||||
// e.g. go_memory_classes_heap_free_bytes
|
||||
MetricsMemory = GoRuntimeMetricsRule{regexp.MustCompile(`^/memory/.*`)}
|
||||
// MetricsScheduler allows only scheduler metrics to be collected from Go runtime.
|
||||
// e.g. go_sched_goroutines_goroutines
|
||||
MetricsScheduler = GoRuntimeMetricsRule{regexp.MustCompile(`^/sched/.*`)}
|
||||
// MetricsDebug allows only debug metrics to be collected from Go runtime.
|
||||
// e.g. go_godebug_non_default_behavior_gocachetest_events_total
|
||||
MetricsDebug = GoRuntimeMetricsRule{regexp.MustCompile(`^/godebug/.*`)}
|
||||
)
|
||||
|
||||
// WithGoCollectorMemStatsMetricsDisabled disables metrics that is gathered in runtime.MemStats structure such as:
|
||||
//
|
||||
// go_memstats_alloc_bytes
|
||||
// go_memstats_alloc_bytes_total
|
||||
// go_memstats_sys_bytes
|
||||
// go_memstats_mallocs_total
|
||||
// go_memstats_frees_total
|
||||
// go_memstats_heap_alloc_bytes
|
||||
// go_memstats_heap_sys_bytes
|
||||
// go_memstats_heap_idle_bytes
|
||||
// go_memstats_heap_inuse_bytes
|
||||
// go_memstats_heap_released_bytes
|
||||
// go_memstats_heap_objects
|
||||
// go_memstats_stack_inuse_bytes
|
||||
// go_memstats_stack_sys_bytes
|
||||
// go_memstats_mspan_inuse_bytes
|
||||
// go_memstats_mspan_sys_bytes
|
||||
// go_memstats_mcache_inuse_bytes
|
||||
// go_memstats_mcache_sys_bytes
|
||||
// go_memstats_buck_hash_sys_bytes
|
||||
// go_memstats_gc_sys_bytes
|
||||
// go_memstats_other_sys_bytes
|
||||
// go_memstats_next_gc_bytes
|
||||
//
|
||||
// so the metrics known from pre client_golang v1.12.0,
|
||||
//
|
||||
// NOTE(bwplotka): The above represents runtime.MemStats statistics, but they are
|
||||
// actually implemented using new runtime/metrics package. (except skipped go_memstats_gc_cpu_fraction
|
||||
// -- see https://github.com/prometheus/client_golang/issues/842#issuecomment-861812034 for explanation).
|
||||
//
|
||||
// Some users might want to disable this on collector level (although you can use scrape relabelling on Prometheus),
|
||||
// because similar metrics can be now obtained using WithGoCollectorRuntimeMetrics. Note that the semantics of new
|
||||
// metrics might be different, plus the names can be change over time with different Go version.
|
||||
//
|
||||
// NOTE(bwplotka): Changing metric names can be tedious at times as the alerts, recording rules and dashboards have to be adjusted.
|
||||
// The old metrics are also very useful, with many guides and books written about how to interpret them.
|
||||
//
|
||||
// As a result our recommendation would be to stick with MemStats like metrics and enable other runtime/metrics if you are interested
|
||||
// in advanced insights Go provides. See ExampleGoCollector_WithAdvancedGoMetrics.
|
||||
func WithGoCollectorMemStatsMetricsDisabled() func(options *internal.GoCollectorOptions) {
|
||||
return func(o *internal.GoCollectorOptions) {
|
||||
o.DisableMemStatsLikeMetrics = true
|
||||
}
|
||||
}
|
||||
|
||||
// GoRuntimeMetricsRule allow enabling and configuring particular group of runtime/metrics.
|
||||
// TODO(bwplotka): Consider adding ability to adjust buckets.
|
||||
type GoRuntimeMetricsRule struct {
|
||||
// Matcher represents RE2 expression will match the runtime/metrics from https://pkg.go.dev/runtime/metrics
|
||||
// Use `regexp.MustCompile` or `regexp.Compile` to create this field.
|
||||
Matcher *regexp.Regexp
|
||||
}
|
||||
|
||||
// WithGoCollectorRuntimeMetrics allows enabling and configuring particular group of runtime/metrics.
|
||||
// See the list of metrics https://pkg.go.dev/runtime/metrics (pick the Go version you use there!).
|
||||
// You can use this option in repeated manner, which will add new rules. The order of rules is important, the last rule
|
||||
// that matches particular metrics is applied.
|
||||
func WithGoCollectorRuntimeMetrics(rules ...GoRuntimeMetricsRule) func(options *internal.GoCollectorOptions) {
|
||||
rs := make([]internal.GoCollectorRule, len(rules))
|
||||
for i, r := range rules {
|
||||
rs[i] = internal.GoCollectorRule{
|
||||
Matcher: r.Matcher,
|
||||
}
|
||||
}
|
||||
|
||||
return func(o *internal.GoCollectorOptions) {
|
||||
o.RuntimeMetricRules = append(o.RuntimeMetricRules, rs...)
|
||||
}
|
||||
}
|
||||
|
||||
// WithoutGoCollectorRuntimeMetrics allows disabling group of runtime/metrics that you might have added in WithGoCollectorRuntimeMetrics.
|
||||
// It behaves similarly to WithGoCollectorRuntimeMetrics just with deny-list semantics.
|
||||
func WithoutGoCollectorRuntimeMetrics(matchers ...*regexp.Regexp) func(options *internal.GoCollectorOptions) {
|
||||
rs := make([]internal.GoCollectorRule, len(matchers))
|
||||
for i, m := range matchers {
|
||||
rs[i] = internal.GoCollectorRule{
|
||||
Matcher: m,
|
||||
Deny: true,
|
||||
}
|
||||
}
|
||||
|
||||
return func(o *internal.GoCollectorOptions) {
|
||||
o.RuntimeMetricRules = append(o.RuntimeMetricRules, rs...)
|
||||
}
|
||||
}
|
||||
|
||||
// GoCollectionOption represents Go collection option flag.
|
||||
// Deprecated.
|
||||
type GoCollectionOption uint32
|
||||
|
||||
const (
|
||||
// GoRuntimeMemStatsCollection represents the metrics represented by runtime.MemStats structure.
|
||||
//
|
||||
// Deprecated: Use WithGoCollectorMemStatsMetricsDisabled() function to disable those metrics in the collector.
|
||||
GoRuntimeMemStatsCollection GoCollectionOption = 1 << iota
|
||||
// GoRuntimeMetricsCollection is the new set of metrics represented by runtime/metrics package.
|
||||
//
|
||||
// Deprecated: Use WithGoCollectorRuntimeMetrics(GoRuntimeMetricsRule{Matcher: regexp.MustCompile("/.*")})
|
||||
// function to enable those metrics in the collector.
|
||||
GoRuntimeMetricsCollection
|
||||
)
|
||||
|
||||
// WithGoCollections allows enabling different collections for Go collector on top of base metrics.
|
||||
//
|
||||
// Deprecated: Use WithGoCollectorRuntimeMetrics() and WithGoCollectorMemStatsMetricsDisabled() instead to control metrics.
|
||||
func WithGoCollections(flags GoCollectionOption) func(options *internal.GoCollectorOptions) {
|
||||
return func(options *internal.GoCollectorOptions) {
|
||||
if flags&GoRuntimeMemStatsCollection == 0 {
|
||||
WithGoCollectorMemStatsMetricsDisabled()(options)
|
||||
}
|
||||
|
||||
if flags&GoRuntimeMetricsCollection != 0 {
|
||||
WithGoCollectorRuntimeMetrics(GoRuntimeMetricsRule{Matcher: regexp.MustCompile("/.*")})(options)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// NewGoCollector returns a collector that exports metrics about the current Go
|
||||
// process using debug.GCStats (base metrics) and runtime/metrics (both in MemStats style and new ones).
|
||||
func NewGoCollector(opts ...func(o *internal.GoCollectorOptions)) prometheus.Collector {
|
||||
//nolint:staticcheck // Ignore SA1019 until v2.
|
||||
return prometheus.NewGoCollector(opts...)
|
||||
}
|
||||
56
vendor/github.com/prometheus/client_golang/prometheus/collectors/process_collector.go
generated
vendored
Normal file
56
vendor/github.com/prometheus/client_golang/prometheus/collectors/process_collector.go
generated
vendored
Normal file
@ -0,0 +1,56 @@
|
||||
// Copyright 2021 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package collectors
|
||||
|
||||
import "github.com/prometheus/client_golang/prometheus"
|
||||
|
||||
// ProcessCollectorOpts defines the behavior of a process metrics collector
|
||||
// created with NewProcessCollector.
|
||||
type ProcessCollectorOpts struct {
|
||||
// PidFn returns the PID of the process the collector collects metrics
|
||||
// for. It is called upon each collection. By default, the PID of the
|
||||
// current process is used, as determined on construction time by
|
||||
// calling os.Getpid().
|
||||
PidFn func() (int, error)
|
||||
// If non-empty, each of the collected metrics is prefixed by the
|
||||
// provided string and an underscore ("_").
|
||||
Namespace string
|
||||
// If true, any error encountered during collection is reported as an
|
||||
// invalid metric (see NewInvalidMetric). Otherwise, errors are ignored
|
||||
// and the collected metrics will be incomplete. (Possibly, no metrics
|
||||
// will be collected at all.) While that's usually not desired, it is
|
||||
// appropriate for the common "mix-in" of process metrics, where process
|
||||
// metrics are nice to have, but failing to collect them should not
|
||||
// disrupt the collection of the remaining metrics.
|
||||
ReportErrors bool
|
||||
}
|
||||
|
||||
// NewProcessCollector returns a collector which exports the current state of
|
||||
// process metrics including CPU, memory and file descriptor usage as well as
|
||||
// the process start time. The detailed behavior is defined by the provided
|
||||
// ProcessCollectorOpts. The zero value of ProcessCollectorOpts creates a
|
||||
// collector for the current process with an empty namespace string and no error
|
||||
// reporting.
|
||||
//
|
||||
// The collector only works on operating systems with a Linux-style proc
|
||||
// filesystem and on Microsoft Windows. On other operating systems, it will not
|
||||
// collect any metrics.
|
||||
func NewProcessCollector(opts ProcessCollectorOpts) prometheus.Collector {
|
||||
//nolint:staticcheck // Ignore SA1019 until v2.
|
||||
return prometheus.NewProcessCollector(prometheus.ProcessCollectorOpts{
|
||||
PidFn: opts.PidFn,
|
||||
Namespace: opts.Namespace,
|
||||
ReportErrors: opts.ReportErrors,
|
||||
})
|
||||
}
|
||||
361
vendor/github.com/prometheus/client_golang/prometheus/counter.go
generated
vendored
Normal file
361
vendor/github.com/prometheus/client_golang/prometheus/counter.go
generated
vendored
Normal file
@ -0,0 +1,361 @@
|
||||
// Copyright 2014 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package prometheus
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"math"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
dto "github.com/prometheus/client_model/go"
|
||||
"google.golang.org/protobuf/types/known/timestamppb"
|
||||
)
|
||||
|
||||
// Counter is a Metric that represents a single numerical value that only ever
|
||||
// goes up. That implies that it cannot be used to count items whose number can
|
||||
// also go down, e.g. the number of currently running goroutines. Those
|
||||
// "counters" are represented by Gauges.
|
||||
//
|
||||
// A Counter is typically used to count requests served, tasks completed, errors
|
||||
// occurred, etc.
|
||||
//
|
||||
// To create Counter instances, use NewCounter.
|
||||
type Counter interface {
|
||||
Metric
|
||||
Collector
|
||||
|
||||
// Inc increments the counter by 1. Use Add to increment it by arbitrary
|
||||
// non-negative values.
|
||||
Inc()
|
||||
// Add adds the given value to the counter. It panics if the value is <
|
||||
// 0.
|
||||
Add(float64)
|
||||
}
|
||||
|
||||
// ExemplarAdder is implemented by Counters that offer the option of adding a
|
||||
// value to the Counter together with an exemplar. Its AddWithExemplar method
|
||||
// works like the Add method of the Counter interface but also replaces the
|
||||
// currently saved exemplar (if any) with a new one, created from the provided
|
||||
// value, the current time as timestamp, and the provided labels. Empty Labels
|
||||
// will lead to a valid (label-less) exemplar. But if Labels is nil, the current
|
||||
// exemplar is left in place. AddWithExemplar panics if the value is < 0, if any
|
||||
// of the provided labels are invalid, or if the provided labels contain more
|
||||
// than 128 runes in total.
|
||||
type ExemplarAdder interface {
|
||||
AddWithExemplar(value float64, exemplar Labels)
|
||||
}
|
||||
|
||||
// CounterOpts is an alias for Opts. See there for doc comments.
|
||||
type CounterOpts Opts
|
||||
|
||||
// CounterVecOpts bundles the options to create a CounterVec metric.
|
||||
// It is mandatory to set CounterOpts, see there for mandatory fields. VariableLabels
|
||||
// is optional and can safely be left to its default value.
|
||||
type CounterVecOpts struct {
|
||||
CounterOpts
|
||||
|
||||
// VariableLabels are used to partition the metric vector by the given set
|
||||
// of labels. Each label value will be constrained with the optional Constraint
|
||||
// function, if provided.
|
||||
VariableLabels ConstrainableLabels
|
||||
}
|
||||
|
||||
// NewCounter creates a new Counter based on the provided CounterOpts.
|
||||
//
|
||||
// The returned implementation also implements ExemplarAdder. It is safe to
|
||||
// perform the corresponding type assertion.
|
||||
//
|
||||
// The returned implementation tracks the counter value in two separate
|
||||
// variables, a float64 and a uint64. The latter is used to track calls of the
|
||||
// Inc method and calls of the Add method with a value that can be represented
|
||||
// as a uint64. This allows atomic increments of the counter with optimal
|
||||
// performance. (It is common to have an Inc call in very hot execution paths.)
|
||||
// Both internal tracking values are added up in the Write method. This has to
|
||||
// be taken into account when it comes to precision and overflow behavior.
|
||||
func NewCounter(opts CounterOpts) Counter {
|
||||
desc := V2.NewDesc(
|
||||
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name),
|
||||
opts.Help,
|
||||
UnconstrainedLabels(nil),
|
||||
opts.ConstLabels,
|
||||
WithUnit(opts.Unit),
|
||||
)
|
||||
if opts.now == nil {
|
||||
opts.now = time.Now
|
||||
}
|
||||
result := &counter{desc: desc, labelPairs: desc.constLabelPairs, now: opts.now}
|
||||
result.init(result) // Init self-collection.
|
||||
result.createdTs = timestamppb.New(opts.now())
|
||||
return result
|
||||
}
|
||||
|
||||
type counter struct {
|
||||
// valBits contains the bits of the represented float64 value, while
|
||||
// valInt stores values that are exact integers. Both have to go first
|
||||
// in the struct to guarantee alignment for atomic operations.
|
||||
// http://golang.org/pkg/sync/atomic/#pkg-note-BUG
|
||||
valBits uint64
|
||||
valInt uint64
|
||||
|
||||
selfCollector
|
||||
desc *Desc
|
||||
|
||||
createdTs *timestamppb.Timestamp
|
||||
labelPairs []*dto.LabelPair
|
||||
exemplar atomic.Value // Containing nil or a *dto.Exemplar.
|
||||
|
||||
// now is for testing purposes, by default it's time.Now.
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
func (c *counter) Desc() *Desc {
|
||||
return c.desc
|
||||
}
|
||||
|
||||
func (c *counter) Add(v float64) {
|
||||
if v < 0 {
|
||||
panic(errors.New("counter cannot decrease in value"))
|
||||
}
|
||||
|
||||
ival := uint64(v)
|
||||
if float64(ival) == v {
|
||||
atomic.AddUint64(&c.valInt, ival)
|
||||
return
|
||||
}
|
||||
|
||||
for {
|
||||
oldBits := atomic.LoadUint64(&c.valBits)
|
||||
newBits := math.Float64bits(math.Float64frombits(oldBits) + v)
|
||||
if atomic.CompareAndSwapUint64(&c.valBits, oldBits, newBits) {
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (c *counter) AddWithExemplar(v float64, e Labels) {
|
||||
c.Add(v)
|
||||
c.updateExemplar(v, e)
|
||||
}
|
||||
|
||||
func (c *counter) Inc() {
|
||||
atomic.AddUint64(&c.valInt, 1)
|
||||
}
|
||||
|
||||
func (c *counter) get() float64 {
|
||||
fval := math.Float64frombits(atomic.LoadUint64(&c.valBits))
|
||||
ival := atomic.LoadUint64(&c.valInt)
|
||||
return fval + float64(ival)
|
||||
}
|
||||
|
||||
func (c *counter) Write(out *dto.Metric) error {
|
||||
// Read the Exemplar first and the value second. This is to avoid a race condition
|
||||
// where users see an exemplar for a not-yet-existing observation.
|
||||
var exemplar *dto.Exemplar
|
||||
if e := c.exemplar.Load(); e != nil {
|
||||
exemplar = e.(*dto.Exemplar)
|
||||
}
|
||||
val := c.get()
|
||||
return populateMetric(CounterValue, val, c.labelPairs, exemplar, out, c.createdTs)
|
||||
}
|
||||
|
||||
func (c *counter) updateExemplar(v float64, l Labels) {
|
||||
if l == nil {
|
||||
return
|
||||
}
|
||||
e, err := newExemplar(v, c.now(), l)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
c.exemplar.Store(e)
|
||||
}
|
||||
|
||||
// CounterVec is a Collector that bundles a set of Counters that all share the
|
||||
// same Desc, but have different values for their variable labels. This is used
|
||||
// if you want to count the same thing partitioned by various dimensions
|
||||
// (e.g. number of HTTP requests, partitioned by response code and
|
||||
// method). Create instances with NewCounterVec.
|
||||
type CounterVec struct {
|
||||
*MetricVec
|
||||
}
|
||||
|
||||
// NewCounterVec creates a new CounterVec based on the provided CounterOpts and
|
||||
// partitioned by the given label names.
|
||||
func NewCounterVec(opts CounterOpts, labelNames []string) *CounterVec {
|
||||
return V2.NewCounterVec(CounterVecOpts{
|
||||
CounterOpts: opts,
|
||||
VariableLabels: UnconstrainedLabels(labelNames),
|
||||
})
|
||||
}
|
||||
|
||||
// NewCounterVec creates a new CounterVec based on the provided CounterVecOpts.
|
||||
func (v2) NewCounterVec(opts CounterVecOpts) *CounterVec {
|
||||
desc := V2.NewDesc(
|
||||
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name),
|
||||
opts.Help,
|
||||
opts.VariableLabels,
|
||||
opts.ConstLabels,
|
||||
WithUnit(opts.Unit),
|
||||
)
|
||||
if opts.now == nil {
|
||||
opts.now = time.Now
|
||||
}
|
||||
return &CounterVec{
|
||||
MetricVec: NewMetricVec(desc, func(lvs ...string) Metric {
|
||||
if len(lvs) != len(desc.variableLabels.names) {
|
||||
panic(makeInconsistentCardinalityError(desc.fqName, desc.variableLabels.names, lvs))
|
||||
}
|
||||
result := &counter{desc: desc, labelPairs: MakeLabelPairs(desc, lvs), now: opts.now}
|
||||
result.init(result) // Init self-collection.
|
||||
result.createdTs = timestamppb.New(opts.now())
|
||||
return result
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
// GetMetricWithLabelValues returns the Counter for the given slice of label
|
||||
// values (same order as the variable labels in Desc). If that combination of
|
||||
// label values is accessed for the first time, a new Counter is created.
|
||||
//
|
||||
// It is possible to call this method without using the returned Counter to only
|
||||
// create the new Counter but leave it at its starting value 0. See also the
|
||||
// SummaryVec example.
|
||||
//
|
||||
// Keeping the Counter for later use is possible (and should be considered if
|
||||
// performance is critical), but keep in mind that Reset, DeleteLabelValues and
|
||||
// Delete can be used to delete the Counter from the CounterVec. In that case,
|
||||
// the Counter will still exist, but it will not be exported anymore, even if a
|
||||
// Counter with the same label values is created later.
|
||||
//
|
||||
// An error is returned if the number of label values is not the same as the
|
||||
// number of variable labels in Desc (minus any curried labels).
|
||||
//
|
||||
// Note that for more than one label value, this method is prone to mistakes
|
||||
// caused by an incorrect order of arguments. Consider GetMetricWith(Labels) as
|
||||
// an alternative to avoid that type of mistake. For higher label numbers, the
|
||||
// latter has a much more readable (albeit more verbose) syntax, but it comes
|
||||
// with a performance overhead (for creating and processing the Labels map).
|
||||
// See also the GaugeVec example.
|
||||
func (v *CounterVec) GetMetricWithLabelValues(lvs ...string) (Counter, error) {
|
||||
metric, err := v.MetricVec.GetMetricWithLabelValues(lvs...)
|
||||
if metric != nil {
|
||||
return metric.(Counter), err
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// GetMetricWith returns the Counter for the given Labels map (the label names
|
||||
// must match those of the variable labels in Desc). If that label map is
|
||||
// accessed for the first time, a new Counter is created. Implications of
|
||||
// creating a Counter without using it and keeping the Counter for later use are
|
||||
// the same as for GetMetricWithLabelValues.
|
||||
//
|
||||
// An error is returned if the number and names of the Labels are inconsistent
|
||||
// with those of the variable labels in Desc (minus any curried labels).
|
||||
//
|
||||
// This method is used for the same purpose as
|
||||
// GetMetricWithLabelValues(...string). See there for pros and cons of the two
|
||||
// methods.
|
||||
func (v *CounterVec) GetMetricWith(labels Labels) (Counter, error) {
|
||||
metric, err := v.MetricVec.GetMetricWith(labels)
|
||||
if metric != nil {
|
||||
return metric.(Counter), err
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// WithLabelValues works as GetMetricWithLabelValues, but panics where
|
||||
// GetMetricWithLabelValues would have returned an error. Not returning an
|
||||
// error allows shortcuts like
|
||||
//
|
||||
// myVec.WithLabelValues("404", "GET").Add(42)
|
||||
func (v *CounterVec) WithLabelValues(lvs ...string) Counter {
|
||||
c, err := v.GetMetricWithLabelValues(lvs...)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
// With works as GetMetricWith, but panics where GetMetricWithLabels would have
|
||||
// returned an error. Not returning an error allows shortcuts like
|
||||
//
|
||||
// myVec.With(prometheus.Labels{"code": "404", "method": "GET"}).Add(42)
|
||||
func (v *CounterVec) With(labels Labels) Counter {
|
||||
c, err := v.GetMetricWith(labels)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
// CurryWith returns a vector curried with the provided labels, i.e. the
|
||||
// returned vector has those labels pre-set for all labeled operations performed
|
||||
// on it. The cardinality of the curried vector is reduced accordingly. The
|
||||
// order of the remaining labels stays the same (just with the curried labels
|
||||
// taken out of the sequence – which is relevant for the
|
||||
// (GetMetric)WithLabelValues methods). It is possible to curry a curried
|
||||
// vector, but only with labels not yet used for currying before.
|
||||
//
|
||||
// The metrics contained in the CounterVec are shared between the curried and
|
||||
// uncurried vectors. They are just accessed differently. Curried and uncurried
|
||||
// vectors behave identically in terms of collection. Only one must be
|
||||
// registered with a given registry (usually the uncurried version). The Reset
|
||||
// method deletes all metrics, even if called on a curried vector.
|
||||
func (v *CounterVec) CurryWith(labels Labels) (*CounterVec, error) {
|
||||
vec, err := v.MetricVec.CurryWith(labels)
|
||||
if vec != nil {
|
||||
return &CounterVec{vec}, err
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// MustCurryWith works as CurryWith but panics where CurryWith would have
|
||||
// returned an error.
|
||||
func (v *CounterVec) MustCurryWith(labels Labels) *CounterVec {
|
||||
vec, err := v.CurryWith(labels)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return vec
|
||||
}
|
||||
|
||||
// CounterFunc is a Counter whose value is determined at collect time by calling a
|
||||
// provided function.
|
||||
//
|
||||
// To create CounterFunc instances, use NewCounterFunc.
|
||||
type CounterFunc interface {
|
||||
Metric
|
||||
Collector
|
||||
}
|
||||
|
||||
// NewCounterFunc creates a new CounterFunc based on the provided
|
||||
// CounterOpts. The value reported is determined by calling the given function
|
||||
// from within the Write method. Take into account that metric collection may
|
||||
// happen concurrently. If that results in concurrent calls to Write, like in
|
||||
// the case where a CounterFunc is directly registered with Prometheus, the
|
||||
// provided function must be concurrency-safe. The function should also honor
|
||||
// the contract for a Counter (values only go up, not down), but compliance will
|
||||
// not be checked.
|
||||
//
|
||||
// Check out the ExampleGaugeFunc examples for the similar GaugeFunc.
|
||||
func NewCounterFunc(opts CounterOpts, function func() float64) CounterFunc {
|
||||
return newValueFunc(V2.NewDesc(
|
||||
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name),
|
||||
opts.Help,
|
||||
UnconstrainedLabels(nil),
|
||||
opts.ConstLabels,
|
||||
WithUnit(opts.Unit),
|
||||
), CounterValue, function)
|
||||
}
|
||||
238
vendor/github.com/prometheus/client_golang/prometheus/desc.go
generated
vendored
Normal file
238
vendor/github.com/prometheus/client_golang/prometheus/desc.go
generated
vendored
Normal file
@ -0,0 +1,238 @@
|
||||
// Copyright 2016 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package prometheus
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"github.com/cespare/xxhash/v2"
|
||||
dto "github.com/prometheus/client_model/go"
|
||||
"github.com/prometheus/common/model"
|
||||
"google.golang.org/protobuf/proto"
|
||||
|
||||
"github.com/prometheus/client_golang/prometheus/internal"
|
||||
)
|
||||
|
||||
// Desc is the descriptor used by every Prometheus Metric. It is essentially
|
||||
// the immutable meta-data of a Metric. The normal Metric implementations
|
||||
// included in this package manage their Desc under the hood. Users only have to
|
||||
// deal with Desc if they use advanced features like the ExpvarCollector or
|
||||
// custom Collectors and Metrics.
|
||||
//
|
||||
// Descriptors registered with the same registry have to fulfill certain
|
||||
// consistency and uniqueness criteria if they share the same fully-qualified
|
||||
// name: They must have the same help string and the same label names (aka label
|
||||
// dimensions) in each, constLabels and variableLabels, but they must differ in
|
||||
// the values of the constLabels.
|
||||
//
|
||||
// Descriptors that share the same fully-qualified names and the same label
|
||||
// values of their constLabels are considered equal.
|
||||
//
|
||||
// Use NewDesc to create new Desc instances.
|
||||
type Desc struct {
|
||||
// fqName has been built from Namespace, Subsystem, and Name.
|
||||
fqName string
|
||||
// help provides some helpful information about this metric.
|
||||
help string
|
||||
// unit provides the unit of this metric.
|
||||
unit string
|
||||
// constLabelPairs contains precalculated DTO label pairs based on
|
||||
// the constant labels.
|
||||
constLabelPairs []*dto.LabelPair
|
||||
// variableLabels contains names of labels and normalization function for
|
||||
// which the metric maintains variable values.
|
||||
variableLabels *compiledLabels
|
||||
// id is a hash of the values of the ConstLabels and fqName. This
|
||||
// must be unique among all registered descriptors and can therefore be
|
||||
// used as an identifier of the descriptor.
|
||||
id uint64
|
||||
// dimHash is a hash of the label names (preset and variable) and the
|
||||
// Help string. Each Desc with the same fqName must have the same
|
||||
// dimHash.
|
||||
dimHash uint64
|
||||
// err is an error that occurred during construction. It is reported on
|
||||
// registration time.
|
||||
err error
|
||||
}
|
||||
|
||||
// DescOpt allows setting optional fields for NewDesc.
|
||||
type DescOpt func(*Desc)
|
||||
|
||||
// WithUnit sets the unit for a Desc.
|
||||
func WithUnit(unit string) DescOpt {
|
||||
return func(d *Desc) {
|
||||
d.unit = unit
|
||||
}
|
||||
}
|
||||
|
||||
// NewDesc allocates and initializes a new Desc. Errors are recorded in the Desc
|
||||
// and will be reported on registration time. variableLabels and constLabels can
|
||||
// be nil if no such labels should be set. fqName must not be empty.
|
||||
//
|
||||
// variableLabels only contain the label names. Their label values are variable
|
||||
// and therefore not part of the Desc. (They are managed within the Metric.)
|
||||
//
|
||||
// For constLabels, the label values are constant. Therefore, they are fully
|
||||
// specified in the Desc. See the Collector example for a usage pattern.
|
||||
func NewDesc(fqName, help string, variableLabels []string, constLabels Labels) *Desc {
|
||||
return V2.NewDesc(fqName, help, UnconstrainedLabels(variableLabels), constLabels)
|
||||
}
|
||||
|
||||
// NewDesc allocates and initializes a new Desc. Errors are recorded in the Desc
|
||||
// and will be reported on registration time. variableLabels and constLabels can
|
||||
// be nil if no such labels should be set. fqName must not be empty.
|
||||
//
|
||||
// variableLabels only contain the label names and normalization functions. Their
|
||||
// label values are variable and therefore not part of the Desc. (They are managed
|
||||
// within the Metric.)
|
||||
//
|
||||
// For constLabels, the label values are constant. Therefore, they are fully
|
||||
// specified in the Desc. See the Collector example for a usage pattern.
|
||||
func (v2) NewDesc(fqName, help string, variableLabels ConstrainableLabels, constLabels Labels, opts ...DescOpt) *Desc {
|
||||
d := &Desc{
|
||||
fqName: fqName,
|
||||
help: help,
|
||||
variableLabels: variableLabels.compile(),
|
||||
}
|
||||
|
||||
for _, opt := range opts {
|
||||
opt(d)
|
||||
}
|
||||
if !model.UTF8Validation.IsValidMetricName(fqName) {
|
||||
d.err = fmt.Errorf("%q is not a valid metric name", fqName)
|
||||
return d
|
||||
}
|
||||
// labelValues contains the label values of const labels (in order of
|
||||
// their sorted label names) plus the fqName (at position 0).
|
||||
labelValues := make([]string, 1, len(constLabels)+1)
|
||||
labelValues[0] = fqName
|
||||
labelNames := make([]string, 0, len(constLabels)+len(d.variableLabels.names))
|
||||
labelNameSet := map[string]struct{}{}
|
||||
// First add only the const label names and sort them...
|
||||
for labelName := range constLabels {
|
||||
if !checkLabelName(labelName) {
|
||||
d.err = fmt.Errorf("%q is not a valid label name for metric %q", labelName, fqName)
|
||||
return d
|
||||
}
|
||||
labelNames = append(labelNames, labelName)
|
||||
labelNameSet[labelName] = struct{}{}
|
||||
}
|
||||
sort.Strings(labelNames)
|
||||
// ... so that we can now add const label values in the order of their names.
|
||||
for _, labelName := range labelNames {
|
||||
labelValues = append(labelValues, constLabels[labelName])
|
||||
}
|
||||
// Validate the const label values. They can't have a wrong cardinality, so
|
||||
// use in len(labelValues) as expectedNumberOfValues.
|
||||
if err := validateLabelValues(labelValues, len(labelValues)); err != nil {
|
||||
d.err = err
|
||||
return d
|
||||
}
|
||||
// Now add the variable label names, but prefix them with something that
|
||||
// cannot be in a regular label name. That prevents matching the label
|
||||
// dimension with a different mix between preset and variable labels.
|
||||
for _, label := range d.variableLabels.names {
|
||||
if !checkLabelName(label) {
|
||||
d.err = fmt.Errorf("%q is not a valid label name for metric %q", label, fqName)
|
||||
return d
|
||||
}
|
||||
labelNames = append(labelNames, "$"+label)
|
||||
labelNameSet[label] = struct{}{}
|
||||
}
|
||||
if len(labelNames) != len(labelNameSet) {
|
||||
d.err = fmt.Errorf("duplicate label names in constant and variable labels for metric %q", fqName)
|
||||
return d
|
||||
}
|
||||
|
||||
xxh := xxhash.New()
|
||||
for _, val := range labelValues {
|
||||
xxh.WriteString(val)
|
||||
xxh.Write(separatorByteSlice)
|
||||
}
|
||||
d.id = xxh.Sum64()
|
||||
// Sort labelNames so that order doesn't matter for the hash.
|
||||
sort.Strings(labelNames)
|
||||
// Now hash together (in this order) the help string, the unit string and the sorted
|
||||
// label names.
|
||||
xxh.Reset()
|
||||
xxh.WriteString(help)
|
||||
xxh.Write(separatorByteSlice)
|
||||
xxh.WriteString(d.unit)
|
||||
xxh.Write(separatorByteSlice)
|
||||
for _, labelName := range labelNames {
|
||||
xxh.WriteString(labelName)
|
||||
xxh.Write(separatorByteSlice)
|
||||
}
|
||||
d.dimHash = xxh.Sum64()
|
||||
|
||||
d.constLabelPairs = make([]*dto.LabelPair, 0, len(constLabels))
|
||||
for n, v := range constLabels {
|
||||
d.constLabelPairs = append(d.constLabelPairs, &dto.LabelPair{
|
||||
Name: proto.String(n),
|
||||
Value: proto.String(v),
|
||||
})
|
||||
}
|
||||
sort.Sort(internal.LabelPairSorter(d.constLabelPairs))
|
||||
return d
|
||||
}
|
||||
|
||||
// NewInvalidDesc returns an invalid descriptor, i.e. a descriptor with the
|
||||
// provided error set. If a collector returning such a descriptor is registered,
|
||||
// registration will fail with the provided error. NewInvalidDesc can be used by
|
||||
// a Collector to signal inability to describe itself.
|
||||
func NewInvalidDesc(err error) *Desc {
|
||||
return &Desc{
|
||||
err: err,
|
||||
}
|
||||
}
|
||||
|
||||
// Err returns an error that occurred during construction, if any.
|
||||
//
|
||||
// Calling this method is optional. It can be used to detect construction
|
||||
// errors early, before invoking other methods on the Desc. If an error is
|
||||
// present, later operations may not behave as expected.
|
||||
func (d *Desc) Err() error {
|
||||
return d.err
|
||||
}
|
||||
|
||||
func (d *Desc) String() string {
|
||||
lpStrings := make([]string, 0, len(d.constLabelPairs))
|
||||
for _, lp := range d.constLabelPairs {
|
||||
lpStrings = append(
|
||||
lpStrings,
|
||||
fmt.Sprintf("%s=%q", lp.GetName(), lp.GetValue()),
|
||||
)
|
||||
}
|
||||
vlStrings := []string{}
|
||||
if d.variableLabels != nil {
|
||||
vlStrings = make([]string, 0, len(d.variableLabels.names))
|
||||
for _, vl := range d.variableLabels.names {
|
||||
if fn, ok := d.variableLabels.labelConstraints[vl]; ok && fn != nil {
|
||||
vlStrings = append(vlStrings, fmt.Sprintf("c(%s)", vl))
|
||||
} else {
|
||||
vlStrings = append(vlStrings, vl)
|
||||
}
|
||||
}
|
||||
}
|
||||
return fmt.Sprintf(
|
||||
"Desc{fqName: %q, help: %q, unit: %q, constLabels: {%s}, variableLabels: {%s}}",
|
||||
d.fqName,
|
||||
d.help,
|
||||
d.unit,
|
||||
strings.Join(lpStrings, ","),
|
||||
strings.Join(vlStrings, ","),
|
||||
)
|
||||
}
|
||||
210
vendor/github.com/prometheus/client_golang/prometheus/doc.go
generated
vendored
Normal file
210
vendor/github.com/prometheus/client_golang/prometheus/doc.go
generated
vendored
Normal file
@ -0,0 +1,210 @@
|
||||
// Copyright 2014 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
// Package prometheus is the core instrumentation package. It provides metrics
|
||||
// primitives to instrument code for monitoring. It also offers a registry for
|
||||
// metrics. Sub-packages allow to expose the registered metrics via HTTP
|
||||
// (package promhttp) or push them to a Pushgateway (package push). There is
|
||||
// also a sub-package promauto, which provides metrics constructors with
|
||||
// automatic registration.
|
||||
//
|
||||
// All exported functions and methods are safe to be used concurrently unless
|
||||
// specified otherwise.
|
||||
//
|
||||
// # A Basic Example
|
||||
//
|
||||
// As a starting point, a very basic usage example:
|
||||
//
|
||||
// package main
|
||||
//
|
||||
// import (
|
||||
// "log"
|
||||
// "net/http"
|
||||
//
|
||||
// "github.com/prometheus/client_golang/prometheus"
|
||||
// "github.com/prometheus/client_golang/prometheus/promhttp"
|
||||
// )
|
||||
//
|
||||
// type metrics struct {
|
||||
// cpuTemp prometheus.Gauge
|
||||
// hdFailures *prometheus.CounterVec
|
||||
// }
|
||||
//
|
||||
// func NewMetrics(reg prometheus.Registerer) *metrics {
|
||||
// m := &metrics{
|
||||
// cpuTemp: prometheus.NewGauge(prometheus.GaugeOpts{
|
||||
// Name: "cpu_temperature_celsius",
|
||||
// Help: "Current temperature of the CPU.",
|
||||
// }),
|
||||
// hdFailures: prometheus.NewCounterVec(
|
||||
// prometheus.CounterOpts{
|
||||
// Name: "hd_errors_total",
|
||||
// Help: "Number of hard-disk errors.",
|
||||
// },
|
||||
// []string{"device"},
|
||||
// ),
|
||||
// }
|
||||
// reg.MustRegister(m.cpuTemp)
|
||||
// reg.MustRegister(m.hdFailures)
|
||||
// return m
|
||||
// }
|
||||
//
|
||||
// func main() {
|
||||
// // Create a non-global registry.
|
||||
// reg := prometheus.NewRegistry()
|
||||
//
|
||||
// // Create new metrics and register them using the custom registry.
|
||||
// m := NewMetrics(reg)
|
||||
// // Set values for the new created metrics.
|
||||
// m.cpuTemp.Set(65.3)
|
||||
// m.hdFailures.With(prometheus.Labels{"device":"/dev/sda"}).Inc()
|
||||
//
|
||||
// // Expose metrics and custom registry via an HTTP server
|
||||
// // using the HandleFor function. "/metrics" is the usual endpoint for that.
|
||||
// http.Handle("/metrics", promhttp.HandlerFor(reg, promhttp.HandlerOpts{Registry: reg}))
|
||||
// log.Fatal(http.ListenAndServe(":8080", nil))
|
||||
// }
|
||||
//
|
||||
// This is a complete program that exports two metrics, a Gauge and a Counter,
|
||||
// the latter with a label attached to turn it into a (one-dimensional) vector.
|
||||
// It register the metrics using a custom registry and exposes them via an HTTP server
|
||||
// on the /metrics endpoint.
|
||||
//
|
||||
// # Metrics
|
||||
//
|
||||
// The number of exported identifiers in this package might appear a bit
|
||||
// overwhelming. However, in addition to the basic plumbing shown in the example
|
||||
// above, you only need to understand the different metric types and their
|
||||
// vector versions for basic usage. Furthermore, if you are not concerned with
|
||||
// fine-grained control of when and how to register metrics with the registry,
|
||||
// have a look at the promauto package, which will effectively allow you to
|
||||
// ignore registration altogether in simple cases.
|
||||
//
|
||||
// Above, you have already touched the Counter and the Gauge. There are two more
|
||||
// advanced metric types: the Summary and Histogram. A more thorough description
|
||||
// of those four metric types can be found in the Prometheus docs:
|
||||
// https://prometheus.io/docs/concepts/metric_types/
|
||||
//
|
||||
// In addition to the fundamental metric types Gauge, Counter, Summary, and
|
||||
// Histogram, a very important part of the Prometheus data model is the
|
||||
// partitioning of samples along dimensions called labels, which results in
|
||||
// metric vectors. The fundamental types are GaugeVec, CounterVec, SummaryVec,
|
||||
// and HistogramVec.
|
||||
//
|
||||
// While only the fundamental metric types implement the Metric interface, both
|
||||
// the metrics and their vector versions implement the Collector interface. A
|
||||
// Collector manages the collection of a number of Metrics, but for convenience,
|
||||
// a Metric can also “collect itself”. Note that Gauge, Counter, Summary, and
|
||||
// Histogram are interfaces themselves while GaugeVec, CounterVec, SummaryVec,
|
||||
// and HistogramVec are not.
|
||||
//
|
||||
// To create instances of Metrics and their vector versions, you need a suitable
|
||||
// …Opts struct, i.e. GaugeOpts, CounterOpts, SummaryOpts, or HistogramOpts.
|
||||
//
|
||||
// # Custom Collectors and constant Metrics
|
||||
//
|
||||
// While you could create your own implementations of Metric, most likely you
|
||||
// will only ever implement the Collector interface on your own. At a first
|
||||
// glance, a custom Collector seems handy to bundle Metrics for common
|
||||
// registration (with the prime example of the different metric vectors above,
|
||||
// which bundle all the metrics of the same name but with different labels).
|
||||
//
|
||||
// There is a more involved use case, too: If you already have metrics
|
||||
// available, created outside of the Prometheus context, you don't need the
|
||||
// interface of the various Metric types. You essentially want to mirror the
|
||||
// existing numbers into Prometheus Metrics during collection. An own
|
||||
// implementation of the Collector interface is perfect for that. You can create
|
||||
// Metric instances “on the fly” using NewConstMetric, NewConstHistogram, and
|
||||
// NewConstSummary (and their respective Must… versions). NewConstMetric is used
|
||||
// for all metric types with just a float64 as their value: Counter, Gauge, and
|
||||
// a special “type” called Untyped. Use the latter if you are not sure if the
|
||||
// mirrored metric is a Counter or a Gauge. Creation of the Metric instance
|
||||
// happens in the Collect method. The Describe method has to return separate
|
||||
// Desc instances, representative of the “throw-away” metrics to be created
|
||||
// later. NewDesc comes in handy to create those Desc instances. Alternatively,
|
||||
// you could return no Desc at all, which will mark the Collector “unchecked”.
|
||||
// No checks are performed at registration time, but metric consistency will
|
||||
// still be ensured at scrape time, i.e. any inconsistencies will lead to scrape
|
||||
// errors. Thus, with unchecked Collectors, the responsibility to not collect
|
||||
// metrics that lead to inconsistencies in the total scrape result lies with the
|
||||
// implementer of the Collector. While this is not a desirable state, it is
|
||||
// sometimes necessary. The typical use case is a situation where the exact
|
||||
// metrics to be returned by a Collector cannot be predicted at registration
|
||||
// time, but the implementer has sufficient knowledge of the whole system to
|
||||
// guarantee metric consistency.
|
||||
//
|
||||
// The Collector example illustrates the use case. You can also look at the
|
||||
// source code of the processCollector (mirroring process metrics), the
|
||||
// goCollector (mirroring Go metrics), or the expvarCollector (mirroring expvar
|
||||
// metrics) as examples that are used in this package itself.
|
||||
//
|
||||
// If you just need to call a function to get a single float value to collect as
|
||||
// a metric, GaugeFunc, CounterFunc, or UntypedFunc might be interesting
|
||||
// shortcuts.
|
||||
//
|
||||
// # Advanced Uses of the Registry
|
||||
//
|
||||
// While MustRegister is the by far most common way of registering a Collector,
|
||||
// sometimes you might want to handle the errors the registration might cause.
|
||||
// As suggested by the name, MustRegister panics if an error occurs. With the
|
||||
// Register function, the error is returned and can be handled.
|
||||
//
|
||||
// An error is returned if the registered Collector is incompatible or
|
||||
// inconsistent with already registered metrics. The registry aims for
|
||||
// consistency of the collected metrics according to the Prometheus data model.
|
||||
// Inconsistencies are ideally detected at registration time, not at collect
|
||||
// time. The former will usually be detected at start-up time of a program,
|
||||
// while the latter will only happen at scrape time, possibly not even on the
|
||||
// first scrape if the inconsistency only becomes relevant later. That is the
|
||||
// main reason why a Collector and a Metric have to describe themselves to the
|
||||
// registry.
|
||||
//
|
||||
// So far, everything we did operated on the so-called default registry, as it
|
||||
// can be found in the global DefaultRegisterer variable. With NewRegistry, you
|
||||
// can create a custom registry, or you can even implement the Registerer or
|
||||
// Gatherer interfaces yourself. The methods Register and Unregister work in the
|
||||
// same way on a custom registry as the global functions Register and Unregister
|
||||
// on the default registry.
|
||||
//
|
||||
// There are a number of uses for custom registries: You can use registries with
|
||||
// special properties, see NewPedanticRegistry. You can avoid global state, as
|
||||
// it is imposed by the DefaultRegisterer. You can use multiple registries at
|
||||
// the same time to expose different metrics in different ways. You can use
|
||||
// separate registries for testing purposes.
|
||||
//
|
||||
// Also note that the DefaultRegisterer comes registered with a Collector for Go
|
||||
// runtime metrics (via NewGoCollector) and a Collector for process metrics (via
|
||||
// NewProcessCollector). With a custom registry, you are in control and decide
|
||||
// yourself about the Collectors to register.
|
||||
//
|
||||
// # HTTP Exposition
|
||||
//
|
||||
// The Registry implements the Gatherer interface. The caller of the Gather
|
||||
// method can then expose the gathered metrics in some way. Usually, the metrics
|
||||
// are served via HTTP on the /metrics endpoint. That's happening in the example
|
||||
// above. The tools to expose metrics via HTTP are in the promhttp sub-package.
|
||||
//
|
||||
// # Pushing to the Pushgateway
|
||||
//
|
||||
// Function for pushing to the Pushgateway can be found in the push sub-package.
|
||||
//
|
||||
// # Graphite Bridge
|
||||
//
|
||||
// Functions and examples to push metrics from a Gatherer to Graphite can be
|
||||
// found in the graphite sub-package.
|
||||
//
|
||||
// # Other Means of Exposition
|
||||
//
|
||||
// More ways of exposing metrics can easily be added by following the approaches
|
||||
// of the existing implementations.
|
||||
package prometheus
|
||||
86
vendor/github.com/prometheus/client_golang/prometheus/expvar_collector.go
generated
vendored
Normal file
86
vendor/github.com/prometheus/client_golang/prometheus/expvar_collector.go
generated
vendored
Normal file
@ -0,0 +1,86 @@
|
||||
// Copyright 2014 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package prometheus
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"expvar"
|
||||
)
|
||||
|
||||
type expvarCollector struct {
|
||||
exports map[string]*Desc
|
||||
}
|
||||
|
||||
// NewExpvarCollector is the obsolete version of collectors.NewExpvarCollector.
|
||||
// See there for documentation.
|
||||
//
|
||||
// Deprecated: Use collectors.NewExpvarCollector instead.
|
||||
func NewExpvarCollector(exports map[string]*Desc) Collector {
|
||||
return &expvarCollector{
|
||||
exports: exports,
|
||||
}
|
||||
}
|
||||
|
||||
// Describe implements Collector.
|
||||
func (e *expvarCollector) Describe(ch chan<- *Desc) {
|
||||
for _, desc := range e.exports {
|
||||
ch <- desc
|
||||
}
|
||||
}
|
||||
|
||||
// Collect implements Collector.
|
||||
func (e *expvarCollector) Collect(ch chan<- Metric) {
|
||||
for name, desc := range e.exports {
|
||||
var m Metric
|
||||
expVar := expvar.Get(name)
|
||||
if expVar == nil {
|
||||
continue
|
||||
}
|
||||
var v any
|
||||
labels := make([]string, len(desc.variableLabels.names))
|
||||
if err := json.Unmarshal([]byte(expVar.String()), &v); err != nil {
|
||||
ch <- NewInvalidMetric(desc, err)
|
||||
continue
|
||||
}
|
||||
var processValue func(v any, i int)
|
||||
processValue = func(v any, i int) {
|
||||
if i >= len(labels) {
|
||||
copiedLabels := append(make([]string, 0, len(labels)), labels...)
|
||||
switch v := v.(type) {
|
||||
case float64:
|
||||
m = MustNewConstMetric(desc, UntypedValue, v, copiedLabels...)
|
||||
case bool:
|
||||
if v {
|
||||
m = MustNewConstMetric(desc, UntypedValue, 1, copiedLabels...)
|
||||
} else {
|
||||
m = MustNewConstMetric(desc, UntypedValue, 0, copiedLabels...)
|
||||
}
|
||||
default:
|
||||
return
|
||||
}
|
||||
ch <- m
|
||||
return
|
||||
}
|
||||
vm, ok := v.(map[string]any)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
for lv, val := range vm {
|
||||
labels[i] = lv
|
||||
processValue(val, i+1)
|
||||
}
|
||||
}
|
||||
processValue(v, 0)
|
||||
}
|
||||
}
|
||||
42
vendor/github.com/prometheus/client_golang/prometheus/fnv.go
generated
vendored
Normal file
42
vendor/github.com/prometheus/client_golang/prometheus/fnv.go
generated
vendored
Normal file
@ -0,0 +1,42 @@
|
||||
// Copyright 2018 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package prometheus
|
||||
|
||||
// Inline and byte-free variant of hash/fnv's fnv64a.
|
||||
|
||||
const (
|
||||
offset64 = 14695981039346656037
|
||||
prime64 = 1099511628211
|
||||
)
|
||||
|
||||
// hashNew initializies a new fnv64a hash value.
|
||||
func hashNew() uint64 {
|
||||
return offset64
|
||||
}
|
||||
|
||||
// hashAdd adds a string to a fnv64a hash value, returning the updated hash.
|
||||
func hashAdd(h uint64, s string) uint64 {
|
||||
for i := 0; i < len(s); i++ {
|
||||
h ^= uint64(s[i])
|
||||
h *= prime64
|
||||
}
|
||||
return h
|
||||
}
|
||||
|
||||
// hashAddByte adds a byte to a fnv64a hash value, returning the updated hash.
|
||||
func hashAddByte(h uint64, b byte) uint64 {
|
||||
h ^= uint64(b)
|
||||
h *= prime64
|
||||
return h
|
||||
}
|
||||
314
vendor/github.com/prometheus/client_golang/prometheus/gauge.go
generated
vendored
Normal file
314
vendor/github.com/prometheus/client_golang/prometheus/gauge.go
generated
vendored
Normal file
@ -0,0 +1,314 @@
|
||||
// Copyright 2014 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package prometheus
|
||||
|
||||
import (
|
||||
"math"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
dto "github.com/prometheus/client_model/go"
|
||||
)
|
||||
|
||||
// Gauge is a Metric that represents a single numerical value that can
|
||||
// arbitrarily go up and down.
|
||||
//
|
||||
// A Gauge is typically used for measured values like temperatures or current
|
||||
// memory usage, but also "counts" that can go up and down, like the number of
|
||||
// running goroutines.
|
||||
//
|
||||
// To create Gauge instances, use NewGauge.
|
||||
type Gauge interface {
|
||||
Metric
|
||||
Collector
|
||||
|
||||
// Set sets the Gauge to an arbitrary value.
|
||||
Set(float64)
|
||||
// Inc increments the Gauge by 1. Use Add to increment it by arbitrary
|
||||
// values.
|
||||
Inc()
|
||||
// Dec decrements the Gauge by 1. Use Sub to decrement it by arbitrary
|
||||
// values.
|
||||
Dec()
|
||||
// Add adds the given value to the Gauge. (The value can be negative,
|
||||
// resulting in a decrease of the Gauge.)
|
||||
Add(float64)
|
||||
// Sub subtracts the given value from the Gauge. (The value can be
|
||||
// negative, resulting in an increase of the Gauge.)
|
||||
Sub(float64)
|
||||
|
||||
// SetToCurrentTime sets the Gauge to the current Unix time in seconds.
|
||||
SetToCurrentTime()
|
||||
}
|
||||
|
||||
// GaugeOpts is an alias for Opts. See there for doc comments.
|
||||
type GaugeOpts Opts
|
||||
|
||||
// GaugeVecOpts bundles the options to create a GaugeVec metric.
|
||||
// It is mandatory to set GaugeOpts, see there for mandatory fields. VariableLabels
|
||||
// is optional and can safely be left to its default value.
|
||||
type GaugeVecOpts struct {
|
||||
GaugeOpts
|
||||
|
||||
// VariableLabels are used to partition the metric vector by the given set
|
||||
// of labels. Each label value will be constrained with the optional Constraint
|
||||
// function, if provided.
|
||||
VariableLabels ConstrainableLabels
|
||||
}
|
||||
|
||||
// NewGauge creates a new Gauge based on the provided GaugeOpts.
|
||||
//
|
||||
// The returned implementation is optimized for a fast Set method. If you have a
|
||||
// choice for managing the value of a Gauge via Set vs. Inc/Dec/Add/Sub, pick
|
||||
// the former. For example, the Inc method of the returned Gauge is slower than
|
||||
// the Inc method of a Counter returned by NewCounter. This matches the typical
|
||||
// scenarios for Gauges and Counters, where the former tends to be Set-heavy and
|
||||
// the latter Inc-heavy.
|
||||
func NewGauge(opts GaugeOpts) Gauge {
|
||||
desc := V2.NewDesc(
|
||||
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name),
|
||||
opts.Help,
|
||||
UnconstrainedLabels(nil),
|
||||
opts.ConstLabels,
|
||||
WithUnit(opts.Unit),
|
||||
)
|
||||
result := &gauge{desc: desc, labelPairs: desc.constLabelPairs}
|
||||
result.init(result) // Init self-collection.
|
||||
return result
|
||||
}
|
||||
|
||||
type gauge struct {
|
||||
// valBits contains the bits of the represented float64 value. It has
|
||||
// to go first in the struct to guarantee alignment for atomic
|
||||
// operations. http://golang.org/pkg/sync/atomic/#pkg-note-BUG
|
||||
valBits uint64
|
||||
|
||||
selfCollector
|
||||
|
||||
desc *Desc
|
||||
labelPairs []*dto.LabelPair
|
||||
}
|
||||
|
||||
func (g *gauge) Desc() *Desc {
|
||||
return g.desc
|
||||
}
|
||||
|
||||
func (g *gauge) Set(val float64) {
|
||||
atomic.StoreUint64(&g.valBits, math.Float64bits(val))
|
||||
}
|
||||
|
||||
func (g *gauge) SetToCurrentTime() {
|
||||
g.Set(float64(time.Now().UnixNano()) / 1e9)
|
||||
}
|
||||
|
||||
func (g *gauge) Inc() {
|
||||
g.Add(1)
|
||||
}
|
||||
|
||||
func (g *gauge) Dec() {
|
||||
g.Add(-1)
|
||||
}
|
||||
|
||||
func (g *gauge) Add(val float64) {
|
||||
for {
|
||||
oldBits := atomic.LoadUint64(&g.valBits)
|
||||
newBits := math.Float64bits(math.Float64frombits(oldBits) + val)
|
||||
if atomic.CompareAndSwapUint64(&g.valBits, oldBits, newBits) {
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (g *gauge) Sub(val float64) {
|
||||
g.Add(val * -1)
|
||||
}
|
||||
|
||||
func (g *gauge) Write(out *dto.Metric) error {
|
||||
val := math.Float64frombits(atomic.LoadUint64(&g.valBits))
|
||||
return populateMetric(GaugeValue, val, g.labelPairs, nil, out, nil)
|
||||
}
|
||||
|
||||
// GaugeVec is a Collector that bundles a set of Gauges that all share the same
|
||||
// Desc, but have different values for their variable labels. This is used if
|
||||
// you want to count the same thing partitioned by various dimensions
|
||||
// (e.g. number of operations queued, partitioned by user and operation
|
||||
// type). Create instances with NewGaugeVec.
|
||||
type GaugeVec struct {
|
||||
*MetricVec
|
||||
}
|
||||
|
||||
// NewGaugeVec creates a new GaugeVec based on the provided GaugeOpts and
|
||||
// partitioned by the given label names.
|
||||
func NewGaugeVec(opts GaugeOpts, labelNames []string) *GaugeVec {
|
||||
return V2.NewGaugeVec(GaugeVecOpts{
|
||||
GaugeOpts: opts,
|
||||
VariableLabels: UnconstrainedLabels(labelNames),
|
||||
})
|
||||
}
|
||||
|
||||
// NewGaugeVec creates a new GaugeVec based on the provided GaugeVecOpts.
|
||||
func (v2) NewGaugeVec(opts GaugeVecOpts) *GaugeVec {
|
||||
desc := V2.NewDesc(
|
||||
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name),
|
||||
opts.Help,
|
||||
opts.VariableLabels,
|
||||
opts.ConstLabels,
|
||||
WithUnit(opts.Unit),
|
||||
)
|
||||
return &GaugeVec{
|
||||
MetricVec: NewMetricVec(desc, func(lvs ...string) Metric {
|
||||
if len(lvs) != len(desc.variableLabels.names) {
|
||||
panic(makeInconsistentCardinalityError(desc.fqName, desc.variableLabels.names, lvs))
|
||||
}
|
||||
result := &gauge{desc: desc, labelPairs: MakeLabelPairs(desc, lvs)}
|
||||
result.init(result) // Init self-collection.
|
||||
return result
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
// GetMetricWithLabelValues returns the Gauge for the given slice of label
|
||||
// values (same order as the variable labels in Desc). If that combination of
|
||||
// label values is accessed for the first time, a new Gauge is created.
|
||||
//
|
||||
// It is possible to call this method without using the returned Gauge to only
|
||||
// create the new Gauge but leave it at its starting value 0. See also the
|
||||
// SummaryVec example.
|
||||
//
|
||||
// Keeping the Gauge for later use is possible (and should be considered if
|
||||
// performance is critical), but keep in mind that Reset, DeleteLabelValues and
|
||||
// Delete can be used to delete the Gauge from the GaugeVec. In that case, the
|
||||
// Gauge will still exist, but it will not be exported anymore, even if a
|
||||
// Gauge with the same label values is created later. See also the CounterVec
|
||||
// example.
|
||||
//
|
||||
// An error is returned if the number of label values is not the same as the
|
||||
// number of variable labels in Desc (minus any curried labels).
|
||||
//
|
||||
// Note that for more than one label value, this method is prone to mistakes
|
||||
// caused by an incorrect order of arguments. Consider GetMetricWith(Labels) as
|
||||
// an alternative to avoid that type of mistake. For higher label numbers, the
|
||||
// latter has a much more readable (albeit more verbose) syntax, but it comes
|
||||
// with a performance overhead (for creating and processing the Labels map).
|
||||
func (v *GaugeVec) GetMetricWithLabelValues(lvs ...string) (Gauge, error) {
|
||||
metric, err := v.MetricVec.GetMetricWithLabelValues(lvs...)
|
||||
if metric != nil {
|
||||
return metric.(Gauge), err
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// GetMetricWith returns the Gauge for the given Labels map (the label names
|
||||
// must match those of the variable labels in Desc). If that label map is
|
||||
// accessed for the first time, a new Gauge is created. Implications of
|
||||
// creating a Gauge without using it and keeping the Gauge for later use are
|
||||
// the same as for GetMetricWithLabelValues.
|
||||
//
|
||||
// An error is returned if the number and names of the Labels are inconsistent
|
||||
// with those of the variable labels in Desc (minus any curried labels).
|
||||
//
|
||||
// This method is used for the same purpose as
|
||||
// GetMetricWithLabelValues(...string). See there for pros and cons of the two
|
||||
// methods.
|
||||
func (v *GaugeVec) GetMetricWith(labels Labels) (Gauge, error) {
|
||||
metric, err := v.MetricVec.GetMetricWith(labels)
|
||||
if metric != nil {
|
||||
return metric.(Gauge), err
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// WithLabelValues works as GetMetricWithLabelValues, but panics where
|
||||
// GetMetricWithLabelValues would have returned an error. Not returning an
|
||||
// error allows shortcuts like
|
||||
//
|
||||
// myVec.WithLabelValues("404", "GET").Add(42)
|
||||
func (v *GaugeVec) WithLabelValues(lvs ...string) Gauge {
|
||||
g, err := v.GetMetricWithLabelValues(lvs...)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return g
|
||||
}
|
||||
|
||||
// With works as GetMetricWith, but panics where GetMetricWithLabels would have
|
||||
// returned an error. Not returning an error allows shortcuts like
|
||||
//
|
||||
// myVec.With(prometheus.Labels{"code": "404", "method": "GET"}).Add(42)
|
||||
func (v *GaugeVec) With(labels Labels) Gauge {
|
||||
g, err := v.GetMetricWith(labels)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return g
|
||||
}
|
||||
|
||||
// CurryWith returns a vector curried with the provided labels, i.e. the
|
||||
// returned vector has those labels pre-set for all labeled operations performed
|
||||
// on it. The cardinality of the curried vector is reduced accordingly. The
|
||||
// order of the remaining labels stays the same (just with the curried labels
|
||||
// taken out of the sequence – which is relevant for the
|
||||
// (GetMetric)WithLabelValues methods). It is possible to curry a curried
|
||||
// vector, but only with labels not yet used for currying before.
|
||||
//
|
||||
// The metrics contained in the GaugeVec are shared between the curried and
|
||||
// uncurried vectors. They are just accessed differently. Curried and uncurried
|
||||
// vectors behave identically in terms of collection. Only one must be
|
||||
// registered with a given registry (usually the uncurried version). The Reset
|
||||
// method deletes all metrics, even if called on a curried vector.
|
||||
func (v *GaugeVec) CurryWith(labels Labels) (*GaugeVec, error) {
|
||||
vec, err := v.MetricVec.CurryWith(labels)
|
||||
if vec != nil {
|
||||
return &GaugeVec{vec}, err
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// MustCurryWith works as CurryWith but panics where CurryWith would have
|
||||
// returned an error.
|
||||
func (v *GaugeVec) MustCurryWith(labels Labels) *GaugeVec {
|
||||
vec, err := v.CurryWith(labels)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return vec
|
||||
}
|
||||
|
||||
// GaugeFunc is a Gauge whose value is determined at collect time by calling a
|
||||
// provided function.
|
||||
//
|
||||
// To create GaugeFunc instances, use NewGaugeFunc.
|
||||
type GaugeFunc interface {
|
||||
Metric
|
||||
Collector
|
||||
}
|
||||
|
||||
// NewGaugeFunc creates a new GaugeFunc based on the provided GaugeOpts. The
|
||||
// value reported is determined by calling the given function from within the
|
||||
// Write method. Take into account that metric collection may happen
|
||||
// concurrently. Therefore, it must be safe to call the provided function
|
||||
// concurrently.
|
||||
//
|
||||
// NewGaugeFunc is a good way to create an “info” style metric with a constant
|
||||
// value of 1. Example:
|
||||
// https://github.com/prometheus/common/blob/8558a5b7db3c84fa38b4766966059a7bd5bfa2ee/version/info.go#L36-L56
|
||||
func NewGaugeFunc(opts GaugeOpts, function func() float64) GaugeFunc {
|
||||
return newValueFunc(V2.NewDesc(
|
||||
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name),
|
||||
opts.Help,
|
||||
UnconstrainedLabels(nil),
|
||||
opts.ConstLabels,
|
||||
WithUnit(opts.Unit),
|
||||
), GaugeValue, function)
|
||||
}
|
||||
26
vendor/github.com/prometheus/client_golang/prometheus/get_pid.go
generated
vendored
Normal file
26
vendor/github.com/prometheus/client_golang/prometheus/get_pid.go
generated
vendored
Normal file
@ -0,0 +1,26 @@
|
||||
// Copyright 2015 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
//go:build !js || wasm
|
||||
// +build !js wasm
|
||||
|
||||
package prometheus
|
||||
|
||||
import "os"
|
||||
|
||||
func getPIDFn() func() (int, error) {
|
||||
pid := os.Getpid()
|
||||
return func() (int, error) {
|
||||
return pid, nil
|
||||
}
|
||||
}
|
||||
23
vendor/github.com/prometheus/client_golang/prometheus/get_pid_gopherjs.go
generated
vendored
Normal file
23
vendor/github.com/prometheus/client_golang/prometheus/get_pid_gopherjs.go
generated
vendored
Normal file
@ -0,0 +1,23 @@
|
||||
// Copyright 2015 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
//go:build js && !wasm
|
||||
// +build js,!wasm
|
||||
|
||||
package prometheus
|
||||
|
||||
func getPIDFn() func() (int, error) {
|
||||
return func() (int, error) {
|
||||
return 1, nil
|
||||
}
|
||||
}
|
||||
274
vendor/github.com/prometheus/client_golang/prometheus/go_collector.go
generated
vendored
Normal file
274
vendor/github.com/prometheus/client_golang/prometheus/go_collector.go
generated
vendored
Normal file
@ -0,0 +1,274 @@
|
||||
// Copyright 2018 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package prometheus
|
||||
|
||||
import (
|
||||
"runtime"
|
||||
"runtime/debug"
|
||||
"time"
|
||||
)
|
||||
|
||||
// goRuntimeMemStats provides the metrics initially provided by runtime.ReadMemStats.
|
||||
// From Go 1.17 those similar (and better) statistics are provided by runtime/metrics, so
|
||||
// while eval closure works on runtime.MemStats, the struct from Go 1.17+ is
|
||||
// populated using runtime/metrics. Those are the defaults we can't alter.
|
||||
func goRuntimeMemStats() memStatsMetrics {
|
||||
return memStatsMetrics{
|
||||
{
|
||||
desc: NewDesc(
|
||||
memstatNamespace("alloc_bytes"),
|
||||
"Number of bytes allocated in heap and currently in use. Equals to /memory/classes/heap/objects:bytes.",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.Alloc) },
|
||||
valType: GaugeValue,
|
||||
}, {
|
||||
desc: NewDesc(
|
||||
memstatNamespace("alloc_bytes_total"),
|
||||
"Total number of bytes allocated in heap until now, even if released already. Equals to /gc/heap/allocs:bytes.",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.TotalAlloc) },
|
||||
valType: CounterValue,
|
||||
}, {
|
||||
desc: NewDesc(
|
||||
memstatNamespace("sys_bytes"),
|
||||
"Number of bytes obtained from system. Equals to /memory/classes/total:byte.",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.Sys) },
|
||||
valType: GaugeValue,
|
||||
}, {
|
||||
desc: NewDesc(
|
||||
memstatNamespace("mallocs_total"),
|
||||
// TODO(bwplotka): We could add go_memstats_heap_objects, probably useful for discovery. Let's gather more feedback, kind of a waste of bytes for everybody for compatibility reasons to keep both, and we can't really rename/remove useful metric.
|
||||
"Total number of heap objects allocated, both live and gc-ed. Semantically a counter version for go_memstats_heap_objects gauge. Equals to /gc/heap/allocs:objects + /gc/heap/tiny/allocs:objects.",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.Mallocs) },
|
||||
valType: CounterValue,
|
||||
}, {
|
||||
desc: NewDesc(
|
||||
memstatNamespace("frees_total"),
|
||||
"Total number of heap objects frees. Equals to /gc/heap/frees:objects + /gc/heap/tiny/allocs:objects.",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.Frees) },
|
||||
valType: CounterValue,
|
||||
}, {
|
||||
desc: NewDesc(
|
||||
memstatNamespace("heap_alloc_bytes"),
|
||||
"Number of heap bytes allocated and currently in use, same as go_memstats_alloc_bytes. Equals to /memory/classes/heap/objects:bytes.",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.HeapAlloc) },
|
||||
valType: GaugeValue,
|
||||
}, {
|
||||
desc: NewDesc(
|
||||
memstatNamespace("heap_sys_bytes"),
|
||||
"Number of heap bytes obtained from system. Equals to /memory/classes/heap/objects:bytes + /memory/classes/heap/unused:bytes + /memory/classes/heap/released:bytes + /memory/classes/heap/free:bytes.",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.HeapSys) },
|
||||
valType: GaugeValue,
|
||||
}, {
|
||||
desc: NewDesc(
|
||||
memstatNamespace("heap_idle_bytes"),
|
||||
"Number of heap bytes waiting to be used. Equals to /memory/classes/heap/released:bytes + /memory/classes/heap/free:bytes.",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.HeapIdle) },
|
||||
valType: GaugeValue,
|
||||
}, {
|
||||
desc: NewDesc(
|
||||
memstatNamespace("heap_inuse_bytes"),
|
||||
"Number of heap bytes that are in use. Equals to /memory/classes/heap/objects:bytes + /memory/classes/heap/unused:bytes",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.HeapInuse) },
|
||||
valType: GaugeValue,
|
||||
}, {
|
||||
desc: NewDesc(
|
||||
memstatNamespace("heap_released_bytes"),
|
||||
"Number of heap bytes released to OS. Equals to /memory/classes/heap/released:bytes.",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.HeapReleased) },
|
||||
valType: GaugeValue,
|
||||
}, {
|
||||
desc: NewDesc(
|
||||
memstatNamespace("heap_objects"),
|
||||
"Number of currently allocated objects. Equals to /gc/heap/objects:objects.",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.HeapObjects) },
|
||||
valType: GaugeValue,
|
||||
}, {
|
||||
desc: NewDesc(
|
||||
memstatNamespace("stack_inuse_bytes"),
|
||||
"Number of bytes obtained from system for stack allocator in non-CGO environments. Equals to /memory/classes/heap/stacks:bytes.",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.StackInuse) },
|
||||
valType: GaugeValue,
|
||||
}, {
|
||||
desc: NewDesc(
|
||||
memstatNamespace("stack_sys_bytes"),
|
||||
"Number of bytes obtained from system for stack allocator. Equals to /memory/classes/heap/stacks:bytes + /memory/classes/os-stacks:bytes.",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.StackSys) },
|
||||
valType: GaugeValue,
|
||||
}, {
|
||||
desc: NewDesc(
|
||||
memstatNamespace("mspan_inuse_bytes"),
|
||||
"Number of bytes in use by mspan structures. Equals to /memory/classes/metadata/mspan/inuse:bytes.",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.MSpanInuse) },
|
||||
valType: GaugeValue,
|
||||
}, {
|
||||
desc: NewDesc(
|
||||
memstatNamespace("mspan_sys_bytes"),
|
||||
"Number of bytes used for mspan structures obtained from system. Equals to /memory/classes/metadata/mspan/inuse:bytes + /memory/classes/metadata/mspan/free:bytes.",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.MSpanSys) },
|
||||
valType: GaugeValue,
|
||||
}, {
|
||||
desc: NewDesc(
|
||||
memstatNamespace("mcache_inuse_bytes"),
|
||||
"Number of bytes in use by mcache structures. Equals to /memory/classes/metadata/mcache/inuse:bytes.",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.MCacheInuse) },
|
||||
valType: GaugeValue,
|
||||
}, {
|
||||
desc: NewDesc(
|
||||
memstatNamespace("mcache_sys_bytes"),
|
||||
"Number of bytes used for mcache structures obtained from system. Equals to /memory/classes/metadata/mcache/inuse:bytes + /memory/classes/metadata/mcache/free:bytes.",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.MCacheSys) },
|
||||
valType: GaugeValue,
|
||||
}, {
|
||||
desc: NewDesc(
|
||||
memstatNamespace("buck_hash_sys_bytes"),
|
||||
"Number of bytes used by the profiling bucket hash table. Equals to /memory/classes/profiling/buckets:bytes.",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.BuckHashSys) },
|
||||
valType: GaugeValue,
|
||||
}, {
|
||||
desc: NewDesc(
|
||||
memstatNamespace("gc_sys_bytes"),
|
||||
"Number of bytes used for garbage collection system metadata. Equals to /memory/classes/metadata/other:bytes.",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.GCSys) },
|
||||
valType: GaugeValue,
|
||||
}, {
|
||||
desc: NewDesc(
|
||||
memstatNamespace("other_sys_bytes"),
|
||||
"Number of bytes used for other system allocations. Equals to /memory/classes/other:bytes.",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.OtherSys) },
|
||||
valType: GaugeValue,
|
||||
}, {
|
||||
desc: NewDesc(
|
||||
memstatNamespace("next_gc_bytes"),
|
||||
"Number of heap bytes when next garbage collection will take place. Equals to /gc/heap/goal:bytes.",
|
||||
nil, nil,
|
||||
),
|
||||
eval: func(ms *runtime.MemStats) float64 { return float64(ms.NextGC) },
|
||||
valType: GaugeValue,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
type baseGoCollector struct {
|
||||
goroutinesDesc *Desc
|
||||
threadsDesc *Desc
|
||||
gcDesc *Desc
|
||||
gcLastTimeDesc *Desc
|
||||
goInfoDesc *Desc
|
||||
}
|
||||
|
||||
func newBaseGoCollector() baseGoCollector {
|
||||
return baseGoCollector{
|
||||
goroutinesDesc: NewDesc(
|
||||
"go_goroutines",
|
||||
"Number of goroutines that currently exist.",
|
||||
nil, nil),
|
||||
threadsDesc: NewDesc(
|
||||
"go_threads",
|
||||
"Number of OS threads created.",
|
||||
nil, nil),
|
||||
gcDesc: NewDesc(
|
||||
"go_gc_duration_seconds",
|
||||
"A summary of the wall-time pause (stop-the-world) duration in garbage collection cycles.",
|
||||
nil, nil),
|
||||
gcLastTimeDesc: NewDesc(
|
||||
"go_memstats_last_gc_time_seconds",
|
||||
"Number of seconds since 1970 of last garbage collection.",
|
||||
nil, nil),
|
||||
goInfoDesc: NewDesc(
|
||||
"go_info",
|
||||
"Information about the Go environment.",
|
||||
nil, Labels{"version": runtime.Version()}),
|
||||
}
|
||||
}
|
||||
|
||||
// Describe returns all descriptions of the collector.
|
||||
func (c *baseGoCollector) Describe(ch chan<- *Desc) {
|
||||
ch <- c.goroutinesDesc
|
||||
ch <- c.threadsDesc
|
||||
ch <- c.gcDesc
|
||||
ch <- c.gcLastTimeDesc
|
||||
ch <- c.goInfoDesc
|
||||
}
|
||||
|
||||
// Collect returns the current state of all metrics of the collector.
|
||||
func (c *baseGoCollector) Collect(ch chan<- Metric) {
|
||||
ch <- MustNewConstMetric(c.goroutinesDesc, GaugeValue, float64(runtime.NumGoroutine()))
|
||||
|
||||
n := getRuntimeNumThreads()
|
||||
ch <- MustNewConstMetric(c.threadsDesc, GaugeValue, n)
|
||||
|
||||
var stats debug.GCStats
|
||||
stats.PauseQuantiles = make([]time.Duration, 5)
|
||||
debug.ReadGCStats(&stats)
|
||||
|
||||
quantiles := make(map[float64]float64)
|
||||
for idx, pq := range stats.PauseQuantiles[1:] {
|
||||
quantiles[float64(idx+1)/float64(len(stats.PauseQuantiles)-1)] = pq.Seconds()
|
||||
}
|
||||
quantiles[0.0] = stats.PauseQuantiles[0].Seconds()
|
||||
ch <- MustNewConstSummary(c.gcDesc, uint64(stats.NumGC), stats.PauseTotal.Seconds(), quantiles)
|
||||
ch <- MustNewConstMetric(c.gcLastTimeDesc, GaugeValue, float64(stats.LastGC.UnixNano())/1e9)
|
||||
ch <- MustNewConstMetric(c.goInfoDesc, GaugeValue, 1)
|
||||
}
|
||||
|
||||
func memstatNamespace(s string) string {
|
||||
return "go_memstats_" + s
|
||||
}
|
||||
|
||||
// memStatsMetrics provide description, evaluator, runtime/metrics name, and
|
||||
// value type for memstat metrics.
|
||||
type memStatsMetrics []struct {
|
||||
desc *Desc
|
||||
eval func(*runtime.MemStats) float64
|
||||
valType ValueType
|
||||
}
|
||||
586
vendor/github.com/prometheus/client_golang/prometheus/go_collector_latest.go
generated
vendored
Normal file
586
vendor/github.com/prometheus/client_golang/prometheus/go_collector_latest.go
generated
vendored
Normal file
@ -0,0 +1,586 @@
|
||||
// Copyright 2021 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
//go:build go1.17
|
||||
// +build go1.17
|
||||
|
||||
package prometheus
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"math"
|
||||
"runtime"
|
||||
"runtime/metrics"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"github.com/prometheus/client_golang/prometheus/internal"
|
||||
|
||||
dto "github.com/prometheus/client_model/go"
|
||||
"google.golang.org/protobuf/proto"
|
||||
)
|
||||
|
||||
const (
|
||||
// constants for strings referenced more than once.
|
||||
goGCHeapTinyAllocsObjects = "/gc/heap/tiny/allocs:objects"
|
||||
goGCHeapAllocsObjects = "/gc/heap/allocs:objects"
|
||||
goGCHeapFreesObjects = "/gc/heap/frees:objects"
|
||||
goGCHeapFreesBytes = "/gc/heap/frees:bytes"
|
||||
goGCHeapAllocsBytes = "/gc/heap/allocs:bytes"
|
||||
goGCHeapObjects = "/gc/heap/objects:objects"
|
||||
goGCHeapGoalBytes = "/gc/heap/goal:bytes"
|
||||
goMemoryClassesTotalBytes = "/memory/classes/total:bytes"
|
||||
goMemoryClassesHeapObjectsBytes = "/memory/classes/heap/objects:bytes"
|
||||
goMemoryClassesHeapUnusedBytes = "/memory/classes/heap/unused:bytes"
|
||||
goMemoryClassesHeapReleasedBytes = "/memory/classes/heap/released:bytes"
|
||||
goMemoryClassesHeapFreeBytes = "/memory/classes/heap/free:bytes"
|
||||
goMemoryClassesHeapStacksBytes = "/memory/classes/heap/stacks:bytes"
|
||||
goMemoryClassesOSStacksBytes = "/memory/classes/os-stacks:bytes"
|
||||
goMemoryClassesMetadataMSpanInuseBytes = "/memory/classes/metadata/mspan/inuse:bytes"
|
||||
goMemoryClassesMetadataMSPanFreeBytes = "/memory/classes/metadata/mspan/free:bytes"
|
||||
goMemoryClassesMetadataMCacheInuseBytes = "/memory/classes/metadata/mcache/inuse:bytes"
|
||||
goMemoryClassesMetadataMCacheFreeBytes = "/memory/classes/metadata/mcache/free:bytes"
|
||||
goMemoryClassesProfilingBucketsBytes = "/memory/classes/profiling/buckets:bytes"
|
||||
goMemoryClassesMetadataOtherBytes = "/memory/classes/metadata/other:bytes"
|
||||
goMemoryClassesOtherBytes = "/memory/classes/other:bytes"
|
||||
)
|
||||
|
||||
// rmNamesForMemStatsMetrics represents runtime/metrics names required to populate goRuntimeMemStats from like logic.
|
||||
var rmNamesForMemStatsMetrics = []string{
|
||||
goGCHeapTinyAllocsObjects,
|
||||
goGCHeapAllocsObjects,
|
||||
goGCHeapFreesObjects,
|
||||
goGCHeapAllocsBytes,
|
||||
goGCHeapObjects,
|
||||
goGCHeapGoalBytes,
|
||||
goMemoryClassesTotalBytes,
|
||||
goMemoryClassesHeapObjectsBytes,
|
||||
goMemoryClassesHeapUnusedBytes,
|
||||
goMemoryClassesHeapReleasedBytes,
|
||||
goMemoryClassesHeapFreeBytes,
|
||||
goMemoryClassesHeapStacksBytes,
|
||||
goMemoryClassesOSStacksBytes,
|
||||
goMemoryClassesMetadataMSpanInuseBytes,
|
||||
goMemoryClassesMetadataMSPanFreeBytes,
|
||||
goMemoryClassesMetadataMCacheInuseBytes,
|
||||
goMemoryClassesMetadataMCacheFreeBytes,
|
||||
goMemoryClassesProfilingBucketsBytes,
|
||||
goMemoryClassesMetadataOtherBytes,
|
||||
goMemoryClassesOtherBytes,
|
||||
}
|
||||
|
||||
func bestEffortLookupRM(lookup []string) []metrics.Description {
|
||||
ret := make([]metrics.Description, 0, len(lookup))
|
||||
for _, rm := range metrics.All() {
|
||||
for _, m := range lookup {
|
||||
if m == rm.Name {
|
||||
ret = append(ret, rm)
|
||||
}
|
||||
}
|
||||
}
|
||||
return ret
|
||||
}
|
||||
|
||||
type goCollector struct {
|
||||
base baseGoCollector
|
||||
|
||||
// mu protects updates to all fields ensuring a consistent
|
||||
// snapshot is always produced by Collect.
|
||||
mu sync.Mutex
|
||||
|
||||
// Contains all samples that have to be retrieved from runtime/metrics (not all of them will be exposed).
|
||||
sampleBuf []metrics.Sample
|
||||
// sampleMap allows lookup for MemStats metrics and runtime/metrics histograms for exact sums.
|
||||
sampleMap map[string]*metrics.Sample
|
||||
|
||||
// rmExposedMetrics represents all runtime/metrics package metrics
|
||||
// that were configured to be exposed.
|
||||
rmExposedMetrics []collectorMetric
|
||||
rmExactSumMapForHist map[string]string
|
||||
|
||||
// With Go 1.17, the runtime/metrics package was introduced.
|
||||
// From that point on, metric names produced by the runtime/metrics
|
||||
// package could be generated from runtime/metrics names. However,
|
||||
// these differ from the old names for the same values.
|
||||
//
|
||||
// This field exists to export the same values under the old names
|
||||
// as well.
|
||||
msMetrics memStatsMetrics
|
||||
msMetricsEnabled bool
|
||||
}
|
||||
|
||||
type rmMetricDesc struct {
|
||||
metrics.Description
|
||||
}
|
||||
|
||||
func matchRuntimeMetricsRules(rules []internal.GoCollectorRule) []rmMetricDesc {
|
||||
var descs []rmMetricDesc
|
||||
for _, d := range metrics.All() {
|
||||
var (
|
||||
deny = true
|
||||
desc rmMetricDesc
|
||||
)
|
||||
|
||||
for _, r := range rules {
|
||||
if !r.Matcher.MatchString(d.Name) {
|
||||
continue
|
||||
}
|
||||
deny = r.Deny
|
||||
}
|
||||
if deny {
|
||||
continue
|
||||
}
|
||||
|
||||
desc.Description = d
|
||||
descs = append(descs, desc)
|
||||
}
|
||||
return descs
|
||||
}
|
||||
|
||||
func defaultGoCollectorOptions() internal.GoCollectorOptions {
|
||||
return internal.GoCollectorOptions{
|
||||
RuntimeMetricSumForHist: map[string]string{
|
||||
"/gc/heap/allocs-by-size:bytes": goGCHeapAllocsBytes,
|
||||
"/gc/heap/frees-by-size:bytes": goGCHeapFreesBytes,
|
||||
},
|
||||
RuntimeMetricRules: []internal.GoCollectorRule{
|
||||
// Recommended metrics we want by default from runtime/metrics.
|
||||
{Matcher: internal.GoCollectorDefaultRuntimeMetrics},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// NewGoCollector is the obsolete version of collectors.NewGoCollector.
|
||||
// See there for documentation.
|
||||
//
|
||||
// Deprecated: Use collectors.NewGoCollector instead.
|
||||
func NewGoCollector(opts ...func(o *internal.GoCollectorOptions)) Collector {
|
||||
opt := defaultGoCollectorOptions()
|
||||
for _, o := range opts {
|
||||
o(&opt)
|
||||
}
|
||||
|
||||
exposedDescriptions := matchRuntimeMetricsRules(opt.RuntimeMetricRules)
|
||||
|
||||
// Collect all histogram samples so that we can get their buckets.
|
||||
// The API guarantees that the buckets are always fixed for the lifetime
|
||||
// of the process.
|
||||
var histograms []metrics.Sample
|
||||
for _, d := range exposedDescriptions {
|
||||
if d.Kind == metrics.KindFloat64Histogram {
|
||||
histograms = append(histograms, metrics.Sample{Name: d.Name})
|
||||
}
|
||||
}
|
||||
|
||||
if len(histograms) > 0 {
|
||||
metrics.Read(histograms)
|
||||
}
|
||||
|
||||
bucketsMap := make(map[string][]float64)
|
||||
for i := range histograms {
|
||||
bucketsMap[histograms[i].Name] = histograms[i].Value.Float64Histogram().Buckets
|
||||
}
|
||||
|
||||
// Generate a collector for each exposed runtime/metrics metric.
|
||||
metricSet := make([]collectorMetric, 0, len(exposedDescriptions))
|
||||
// SampleBuf is used for reading from runtime/metrics.
|
||||
// We are assuming the largest case to have stable pointers for sampleMap purposes.
|
||||
sampleBuf := make([]metrics.Sample, 0, len(exposedDescriptions)+len(opt.RuntimeMetricSumForHist)+len(rmNamesForMemStatsMetrics))
|
||||
sampleMap := make(map[string]*metrics.Sample, len(exposedDescriptions))
|
||||
for _, d := range exposedDescriptions {
|
||||
namespace, subsystem, name, ok := internal.RuntimeMetricsToProm(&d.Description)
|
||||
if !ok {
|
||||
// Just ignore this metric; we can't do anything with it here.
|
||||
// If a user decides to use the latest version of Go, we don't want
|
||||
// to fail here. This condition is tested in TestExpectedRuntimeMetrics.
|
||||
continue
|
||||
}
|
||||
help := attachOriginalName(d.Description.Description, d.Name)
|
||||
|
||||
sampleBuf = append(sampleBuf, metrics.Sample{Name: d.Name})
|
||||
sampleMap[d.Name] = &sampleBuf[len(sampleBuf)-1]
|
||||
|
||||
// Extract unit from the runtime/metrics name (e.g., "/gc/heap/allocs:bytes" -> "bytes")
|
||||
// and sanitize to match Prometheus naming conventions (e.g., "cpu-seconds" -> "cpu_seconds")
|
||||
var unit string
|
||||
if idx := strings.IndexRune(d.Name, ':'); idx >= 0 {
|
||||
unit = d.Name[idx+1:]
|
||||
unit = strings.ReplaceAll(unit, "-", "_")
|
||||
unit = strings.ReplaceAll(unit, "*", "_")
|
||||
unit = strings.ReplaceAll(unit, "/", "_per_")
|
||||
}
|
||||
|
||||
var m collectorMetric
|
||||
if d.Kind == metrics.KindFloat64Histogram {
|
||||
_, hasSum := opt.RuntimeMetricSumForHist[d.Name]
|
||||
m = newBatchHistogram(
|
||||
V2.NewDesc(
|
||||
BuildFQName(namespace, subsystem, name),
|
||||
help,
|
||||
UnconstrainedLabels(nil),
|
||||
nil,
|
||||
WithUnit(unit),
|
||||
),
|
||||
internal.RuntimeMetricsBucketsForUnit(bucketsMap[d.Name], unit),
|
||||
hasSum,
|
||||
)
|
||||
} else if d.Cumulative {
|
||||
m = NewCounter(CounterOpts{
|
||||
Namespace: namespace,
|
||||
Subsystem: subsystem,
|
||||
Name: name,
|
||||
Help: help,
|
||||
Unit: unit,
|
||||
},
|
||||
)
|
||||
} else {
|
||||
m = NewGauge(GaugeOpts{
|
||||
Namespace: namespace,
|
||||
Subsystem: subsystem,
|
||||
Name: name,
|
||||
Help: help,
|
||||
Unit: unit,
|
||||
})
|
||||
}
|
||||
metricSet = append(metricSet, m)
|
||||
}
|
||||
|
||||
// Add exact sum metrics to sampleBuf if not added before.
|
||||
for _, h := range histograms {
|
||||
sumMetric, ok := opt.RuntimeMetricSumForHist[h.Name]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
|
||||
if _, ok := sampleMap[sumMetric]; ok {
|
||||
continue
|
||||
}
|
||||
sampleBuf = append(sampleBuf, metrics.Sample{Name: sumMetric})
|
||||
sampleMap[sumMetric] = &sampleBuf[len(sampleBuf)-1]
|
||||
}
|
||||
|
||||
var (
|
||||
msMetrics memStatsMetrics
|
||||
msDescriptions []metrics.Description
|
||||
)
|
||||
|
||||
if !opt.DisableMemStatsLikeMetrics {
|
||||
msMetrics = goRuntimeMemStats()
|
||||
msDescriptions = bestEffortLookupRM(rmNamesForMemStatsMetrics)
|
||||
|
||||
// Check if metric was not exposed before and if not, add to sampleBuf.
|
||||
for _, mdDesc := range msDescriptions {
|
||||
if _, ok := sampleMap[mdDesc.Name]; ok {
|
||||
continue
|
||||
}
|
||||
sampleBuf = append(sampleBuf, metrics.Sample{Name: mdDesc.Name})
|
||||
sampleMap[mdDesc.Name] = &sampleBuf[len(sampleBuf)-1]
|
||||
}
|
||||
}
|
||||
|
||||
return &goCollector{
|
||||
base: newBaseGoCollector(),
|
||||
sampleBuf: sampleBuf,
|
||||
sampleMap: sampleMap,
|
||||
rmExposedMetrics: metricSet,
|
||||
rmExactSumMapForHist: opt.RuntimeMetricSumForHist,
|
||||
msMetrics: msMetrics,
|
||||
msMetricsEnabled: !opt.DisableMemStatsLikeMetrics,
|
||||
}
|
||||
}
|
||||
|
||||
func attachOriginalName(desc, origName string) string {
|
||||
return fmt.Sprintf("%s Sourced from %s.", desc, origName)
|
||||
}
|
||||
|
||||
// Describe returns all descriptions of the collector.
|
||||
func (c *goCollector) Describe(ch chan<- *Desc) {
|
||||
c.base.Describe(ch)
|
||||
for _, i := range c.msMetrics {
|
||||
ch <- i.desc
|
||||
}
|
||||
for _, m := range c.rmExposedMetrics {
|
||||
ch <- m.Desc()
|
||||
}
|
||||
}
|
||||
|
||||
// Collect returns the current state of all metrics of the collector.
|
||||
func (c *goCollector) Collect(ch chan<- Metric) {
|
||||
// Collect base non-memory metrics.
|
||||
c.base.Collect(ch)
|
||||
|
||||
if len(c.sampleBuf) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
// Collect must be thread-safe, so prevent concurrent use of
|
||||
// sampleBuf elements. Just read into sampleBuf but write all the data
|
||||
// we get into our Metrics or MemStats.
|
||||
//
|
||||
// This lock also ensures that the Metrics we send out are all from
|
||||
// the same updates, ensuring their mutual consistency insofar as
|
||||
// is guaranteed by the runtime/metrics package.
|
||||
//
|
||||
// N.B. This locking is heavy-handed, but Collect is expected to be called
|
||||
// relatively infrequently. Also the core operation here, metrics.Read,
|
||||
// is fast (O(tens of microseconds)) so contention should certainly be
|
||||
// low, though channel operations and any allocations may add to that.
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
|
||||
// Populate runtime/metrics sample buffer.
|
||||
metrics.Read(c.sampleBuf)
|
||||
|
||||
// Collect all our runtime/metrics user chose to expose from sampleBuf (if any).
|
||||
for i, metric := range c.rmExposedMetrics {
|
||||
// We created samples for exposed metrics first in order, so indexes match.
|
||||
sample := c.sampleBuf[i]
|
||||
|
||||
// N.B. switch on concrete type because it's significantly more efficient
|
||||
// than checking for the Counter and Gauge interface implementations. In
|
||||
// this case, we control all the types here.
|
||||
switch m := metric.(type) {
|
||||
case *counter:
|
||||
// Guard against decreases. This should never happen, but a failure
|
||||
// to do so will result in a panic, which is a harsh consequence for
|
||||
// a metrics collection bug.
|
||||
v0, v1 := m.get(), unwrapScalarRMValue(sample.Value)
|
||||
if v1 > v0 {
|
||||
m.Add(unwrapScalarRMValue(sample.Value) - m.get())
|
||||
}
|
||||
m.Collect(ch)
|
||||
case *gauge:
|
||||
m.Set(unwrapScalarRMValue(sample.Value))
|
||||
m.Collect(ch)
|
||||
case *batchHistogram:
|
||||
m.update(sample.Value.Float64Histogram(), c.exactSumFor(sample.Name))
|
||||
m.Collect(ch)
|
||||
default:
|
||||
panic("unexpected metric type")
|
||||
}
|
||||
}
|
||||
|
||||
if c.msMetricsEnabled {
|
||||
// ms is a dummy MemStats that we populate ourselves so that we can
|
||||
// populate the old metrics from it if goMemStatsCollection is enabled.
|
||||
var ms runtime.MemStats
|
||||
memStatsFromRM(&ms, c.sampleMap)
|
||||
for _, i := range c.msMetrics {
|
||||
ch <- MustNewConstMetric(i.desc, i.valType, i.eval(&ms))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// unwrapScalarRMValue unwraps a runtime/metrics value that is assumed
|
||||
// to be scalar and returns the equivalent float64 value. Panics if the
|
||||
// value is not scalar.
|
||||
func unwrapScalarRMValue(v metrics.Value) float64 {
|
||||
switch v.Kind() {
|
||||
case metrics.KindUint64:
|
||||
return float64(v.Uint64())
|
||||
case metrics.KindFloat64:
|
||||
return v.Float64()
|
||||
case metrics.KindBad:
|
||||
// Unsupported metric.
|
||||
//
|
||||
// This should never happen because we always populate our metric
|
||||
// set from the runtime/metrics package.
|
||||
panic("unexpected bad kind metric")
|
||||
default:
|
||||
// Unsupported metric kind.
|
||||
//
|
||||
// This should never happen because we check for this during initialization
|
||||
// and flag and filter metrics whose kinds we don't understand.
|
||||
panic(fmt.Sprintf("unexpected unsupported metric: %v", v.Kind()))
|
||||
}
|
||||
}
|
||||
|
||||
// exactSumFor takes a runtime/metrics metric name (that is assumed to
|
||||
// be of kind KindFloat64Histogram) and returns its exact sum and whether
|
||||
// its exact sum exists.
|
||||
//
|
||||
// The runtime/metrics API for histograms doesn't currently expose exact
|
||||
// sums, but some of the other metrics are in fact exact sums of histograms.
|
||||
func (c *goCollector) exactSumFor(rmName string) float64 {
|
||||
sumName, ok := c.rmExactSumMapForHist[rmName]
|
||||
if !ok {
|
||||
return 0
|
||||
}
|
||||
s, ok := c.sampleMap[sumName]
|
||||
if !ok {
|
||||
return 0
|
||||
}
|
||||
return unwrapScalarRMValue(s.Value)
|
||||
}
|
||||
|
||||
func memStatsFromRM(ms *runtime.MemStats, rm map[string]*metrics.Sample) {
|
||||
lookupOrZero := func(name string) uint64 {
|
||||
if s, ok := rm[name]; ok {
|
||||
return s.Value.Uint64()
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// Currently, MemStats adds tiny alloc count to both Mallocs AND Frees.
|
||||
// The reason for this is because MemStats couldn't be extended at the time
|
||||
// but there was a desire to have Mallocs at least be a little more representative,
|
||||
// while having Mallocs - Frees still represent a live object count.
|
||||
// Unfortunately, MemStats doesn't actually export a large allocation count,
|
||||
// so it's impossible to pull this number out directly.
|
||||
tinyAllocs := lookupOrZero(goGCHeapTinyAllocsObjects)
|
||||
ms.Mallocs = lookupOrZero(goGCHeapAllocsObjects) + tinyAllocs
|
||||
ms.Frees = lookupOrZero(goGCHeapFreesObjects) + tinyAllocs
|
||||
|
||||
ms.TotalAlloc = lookupOrZero(goGCHeapAllocsBytes)
|
||||
ms.Sys = lookupOrZero(goMemoryClassesTotalBytes)
|
||||
ms.Lookups = 0 // Already always zero.
|
||||
ms.HeapAlloc = lookupOrZero(goMemoryClassesHeapObjectsBytes)
|
||||
ms.Alloc = ms.HeapAlloc
|
||||
ms.HeapInuse = ms.HeapAlloc + lookupOrZero(goMemoryClassesHeapUnusedBytes)
|
||||
ms.HeapReleased = lookupOrZero(goMemoryClassesHeapReleasedBytes)
|
||||
ms.HeapIdle = ms.HeapReleased + lookupOrZero(goMemoryClassesHeapFreeBytes)
|
||||
ms.HeapSys = ms.HeapInuse + ms.HeapIdle
|
||||
ms.HeapObjects = lookupOrZero(goGCHeapObjects)
|
||||
ms.StackInuse = lookupOrZero(goMemoryClassesHeapStacksBytes)
|
||||
ms.StackSys = ms.StackInuse + lookupOrZero(goMemoryClassesOSStacksBytes)
|
||||
ms.MSpanInuse = lookupOrZero(goMemoryClassesMetadataMSpanInuseBytes)
|
||||
ms.MSpanSys = ms.MSpanInuse + lookupOrZero(goMemoryClassesMetadataMSPanFreeBytes)
|
||||
ms.MCacheInuse = lookupOrZero(goMemoryClassesMetadataMCacheInuseBytes)
|
||||
ms.MCacheSys = ms.MCacheInuse + lookupOrZero(goMemoryClassesMetadataMCacheFreeBytes)
|
||||
ms.BuckHashSys = lookupOrZero(goMemoryClassesProfilingBucketsBytes)
|
||||
ms.GCSys = lookupOrZero(goMemoryClassesMetadataOtherBytes)
|
||||
ms.OtherSys = lookupOrZero(goMemoryClassesOtherBytes)
|
||||
ms.NextGC = lookupOrZero(goGCHeapGoalBytes)
|
||||
|
||||
// N.B. GCCPUFraction is intentionally omitted. This metric is not useful,
|
||||
// and often misleading due to the fact that it's an average over the lifetime
|
||||
// of the process.
|
||||
// See https://github.com/prometheus/client_golang/issues/842#issuecomment-861812034
|
||||
// for more details.
|
||||
ms.GCCPUFraction = 0
|
||||
}
|
||||
|
||||
// batchHistogram is a mutable histogram that is updated
|
||||
// in batches.
|
||||
type batchHistogram struct {
|
||||
selfCollector
|
||||
|
||||
// Static fields updated only once.
|
||||
desc *Desc
|
||||
hasSum bool
|
||||
|
||||
// Because this histogram operates in batches, it just uses a
|
||||
// single mutex for everything. updates are always serialized
|
||||
// but Write calls may operate concurrently with updates.
|
||||
// Contention between these two sources should be rare.
|
||||
mu sync.Mutex
|
||||
buckets []float64 // Inclusive lower bounds, like runtime/metrics.
|
||||
counts []uint64
|
||||
sum float64 // Used if hasSum is true.
|
||||
}
|
||||
|
||||
// newBatchHistogram creates a new batch histogram value with the given
|
||||
// Desc, buckets, and whether or not it has an exact sum available.
|
||||
//
|
||||
// buckets must always be from the runtime/metrics package, following
|
||||
// the same conventions.
|
||||
func newBatchHistogram(desc *Desc, buckets []float64, hasSum bool) *batchHistogram {
|
||||
// We need to remove -Inf values. runtime/metrics keeps them around.
|
||||
// But -Inf bucket should not be allowed for prometheus histograms.
|
||||
if buckets[0] == math.Inf(-1) {
|
||||
buckets = buckets[1:]
|
||||
}
|
||||
h := &batchHistogram{
|
||||
desc: desc,
|
||||
buckets: buckets,
|
||||
// Because buckets follows runtime/metrics conventions, there's
|
||||
// 1 more value in the buckets list than there are buckets represented,
|
||||
// because in runtime/metrics, the bucket values represent *boundaries*,
|
||||
// and non-Inf boundaries are inclusive lower bounds for that bucket.
|
||||
counts: make([]uint64, len(buckets)-1),
|
||||
hasSum: hasSum,
|
||||
}
|
||||
h.init(h)
|
||||
return h
|
||||
}
|
||||
|
||||
// update updates the batchHistogram from a runtime/metrics histogram.
|
||||
//
|
||||
// sum must be provided if the batchHistogram was created to have an exact sum.
|
||||
// h.buckets must be a strict subset of his.Buckets.
|
||||
func (h *batchHistogram) update(his *metrics.Float64Histogram, sum float64) {
|
||||
counts, buckets := his.Counts, his.Buckets
|
||||
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
|
||||
// Clear buckets.
|
||||
for i := range h.counts {
|
||||
h.counts[i] = 0
|
||||
}
|
||||
// Copy and reduce buckets.
|
||||
var j int
|
||||
for i, count := range counts {
|
||||
h.counts[j] += count
|
||||
if buckets[i+1] == h.buckets[j+1] {
|
||||
j++
|
||||
}
|
||||
}
|
||||
if h.hasSum {
|
||||
h.sum = sum
|
||||
}
|
||||
}
|
||||
|
||||
func (h *batchHistogram) Desc() *Desc {
|
||||
return h.desc
|
||||
}
|
||||
|
||||
func (h *batchHistogram) Write(out *dto.Metric) error {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
|
||||
sum := float64(0)
|
||||
if h.hasSum {
|
||||
sum = h.sum
|
||||
}
|
||||
dtoBuckets := make([]*dto.Bucket, 0, len(h.counts))
|
||||
totalCount := uint64(0)
|
||||
for i, count := range h.counts {
|
||||
totalCount += count
|
||||
if !h.hasSum {
|
||||
if count != 0 {
|
||||
// N.B. This computed sum is an underestimate.
|
||||
sum += h.buckets[i] * float64(count)
|
||||
}
|
||||
}
|
||||
|
||||
// Skip the +Inf bucket, but only for the bucket list.
|
||||
// It must still count for sum and totalCount.
|
||||
if math.IsInf(h.buckets[i+1], 1) {
|
||||
break
|
||||
}
|
||||
// Float64Histogram's upper bound is exclusive, so make it inclusive
|
||||
// by obtaining the next float64 value down, in order.
|
||||
upperBound := math.Nextafter(h.buckets[i+1], h.buckets[i])
|
||||
dtoBuckets = append(dtoBuckets, &dto.Bucket{
|
||||
CumulativeCount: proto.Uint64(totalCount),
|
||||
UpperBound: proto.Float64(upperBound),
|
||||
})
|
||||
}
|
||||
out.Histogram = &dto.Histogram{
|
||||
Bucket: dtoBuckets,
|
||||
SampleCount: proto.Uint64(totalCount),
|
||||
SampleSum: proto.Float64(sum),
|
||||
}
|
||||
return nil
|
||||
}
|
||||
2061
vendor/github.com/prometheus/client_golang/prometheus/histogram.go
generated
vendored
Normal file
2061
vendor/github.com/prometheus/client_golang/prometheus/histogram.go
generated
vendored
Normal file
File diff suppressed because it is too large
Load Diff
60
vendor/github.com/prometheus/client_golang/prometheus/internal/almost_equal.go
generated
vendored
Normal file
60
vendor/github.com/prometheus/client_golang/prometheus/internal/almost_equal.go
generated
vendored
Normal file
@ -0,0 +1,60 @@
|
||||
// Copyright (c) 2015 Björn Rabenstein
|
||||
//
|
||||
// Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
// of this software and associated documentation files (the "Software"), to deal
|
||||
// in the Software without restriction, including without limitation the rights
|
||||
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
// copies of the Software, and to permit persons to whom the Software is
|
||||
// furnished to do so, subject to the following conditions:
|
||||
//
|
||||
// The above copyright notice and this permission notice shall be included in all
|
||||
// copies or substantial portions of the Software.
|
||||
//
|
||||
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
// SOFTWARE.
|
||||
//
|
||||
// The code in this package is copy/paste to avoid a dependency. Hence this file
|
||||
// carries the copyright of the original repo.
|
||||
// https://github.com/beorn7/floats
|
||||
package internal
|
||||
|
||||
import (
|
||||
"math"
|
||||
)
|
||||
|
||||
// minNormalFloat64 is the smallest positive normal value of type float64.
|
||||
var minNormalFloat64 = math.Float64frombits(0x0010000000000000)
|
||||
|
||||
// AlmostEqualFloat64 returns true if a and b are equal within a relative error
|
||||
// of epsilon. See http://floating-point-gui.de/errors/comparison/ for the
|
||||
// details of the applied method.
|
||||
func AlmostEqualFloat64(a, b, epsilon float64) bool {
|
||||
if a == b {
|
||||
return true
|
||||
}
|
||||
absA := math.Abs(a)
|
||||
absB := math.Abs(b)
|
||||
diff := math.Abs(a - b)
|
||||
if a == 0 || b == 0 || absA+absB < minNormalFloat64 {
|
||||
return diff < epsilon*minNormalFloat64
|
||||
}
|
||||
return diff/math.Min(absA+absB, math.MaxFloat64) < epsilon
|
||||
}
|
||||
|
||||
// AlmostEqualFloat64s is the slice form of AlmostEqualFloat64.
|
||||
func AlmostEqualFloat64s(a, b []float64, epsilon float64) bool {
|
||||
if len(a) != len(b) {
|
||||
return false
|
||||
}
|
||||
for i := range a {
|
||||
if !AlmostEqualFloat64(a[i], b[i], epsilon) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
655
vendor/github.com/prometheus/client_golang/prometheus/internal/difflib.go
generated
vendored
Normal file
655
vendor/github.com/prometheus/client_golang/prometheus/internal/difflib.go
generated
vendored
Normal file
@ -0,0 +1,655 @@
|
||||
// Copyright 2022 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
// It provides tools to compare sequences of strings and generate textual diffs.
|
||||
//
|
||||
// Maintaining `GetUnifiedDiffString` here because original repository
|
||||
// (https://github.com/pmezard/go-difflib) is no longer maintained.
|
||||
package internal
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"fmt"
|
||||
"io"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
func minInt(a, b int) int {
|
||||
if a < b {
|
||||
return a
|
||||
}
|
||||
return b
|
||||
}
|
||||
|
||||
func maxInt(a, b int) int {
|
||||
if a > b {
|
||||
return a
|
||||
}
|
||||
return b
|
||||
}
|
||||
|
||||
func calculateRatio(matches, length int) float64 {
|
||||
if length > 0 {
|
||||
return 2.0 * float64(matches) / float64(length)
|
||||
}
|
||||
return 1.0
|
||||
}
|
||||
|
||||
type Match struct {
|
||||
A int
|
||||
B int
|
||||
Size int
|
||||
}
|
||||
|
||||
type OpCode struct {
|
||||
Tag byte
|
||||
I1 int
|
||||
I2 int
|
||||
J1 int
|
||||
J2 int
|
||||
}
|
||||
|
||||
// SequenceMatcher compares sequence of strings. The basic
|
||||
// algorithm predates, and is a little fancier than, an algorithm
|
||||
// published in the late 1980's by Ratcliff and Obershelp under the
|
||||
// hyperbolic name "gestalt pattern matching". The basic idea is to find
|
||||
// the longest contiguous matching subsequence that contains no "junk"
|
||||
// elements (R-O doesn't address junk). The same idea is then applied
|
||||
// recursively to the pieces of the sequences to the left and to the right
|
||||
// of the matching subsequence. This does not yield minimal edit
|
||||
// sequences, but does tend to yield matches that "look right" to people.
|
||||
//
|
||||
// SequenceMatcher tries to compute a "human-friendly diff" between two
|
||||
// sequences. Unlike e.g. UNIX(tm) diff, the fundamental notion is the
|
||||
// longest *contiguous* & junk-free matching subsequence. That's what
|
||||
// catches peoples' eyes. The Windows(tm) windiff has another interesting
|
||||
// notion, pairing up elements that appear uniquely in each sequence.
|
||||
// That, and the method here, appear to yield more intuitive difference
|
||||
// reports than does diff. This method appears to be the least vulnerable
|
||||
// to syncing up on blocks of "junk lines", though (like blank lines in
|
||||
// ordinary text files, or maybe "<P>" lines in HTML files). That may be
|
||||
// because this is the only method of the 3 that has a *concept* of
|
||||
// "junk" <wink>.
|
||||
//
|
||||
// Timing: Basic R-O is cubic time worst case and quadratic time expected
|
||||
// case. SequenceMatcher is quadratic time for the worst case and has
|
||||
// expected-case behavior dependent in a complicated way on how many
|
||||
// elements the sequences have in common; best case time is linear.
|
||||
type SequenceMatcher struct {
|
||||
a []string
|
||||
b []string
|
||||
b2j map[string][]int
|
||||
IsJunk func(string) bool
|
||||
autoJunk bool
|
||||
bJunk map[string]struct{}
|
||||
matchingBlocks []Match
|
||||
fullBCount map[string]int
|
||||
bPopular map[string]struct{}
|
||||
opCodes []OpCode
|
||||
}
|
||||
|
||||
func NewMatcher(a, b []string) *SequenceMatcher {
|
||||
m := SequenceMatcher{autoJunk: true}
|
||||
m.SetSeqs(a, b)
|
||||
return &m
|
||||
}
|
||||
|
||||
func NewMatcherWithJunk(a, b []string, autoJunk bool,
|
||||
isJunk func(string) bool,
|
||||
) *SequenceMatcher {
|
||||
m := SequenceMatcher{IsJunk: isJunk, autoJunk: autoJunk}
|
||||
m.SetSeqs(a, b)
|
||||
return &m
|
||||
}
|
||||
|
||||
// Set two sequences to be compared.
|
||||
func (m *SequenceMatcher) SetSeqs(a, b []string) {
|
||||
m.SetSeq1(a)
|
||||
m.SetSeq2(b)
|
||||
}
|
||||
|
||||
// Set the first sequence to be compared. The second sequence to be compared is
|
||||
// not changed.
|
||||
//
|
||||
// SequenceMatcher computes and caches detailed information about the second
|
||||
// sequence, so if you want to compare one sequence S against many sequences,
|
||||
// use .SetSeq2(s) once and call .SetSeq1(x) repeatedly for each of the other
|
||||
// sequences.
|
||||
//
|
||||
// See also SetSeqs() and SetSeq2().
|
||||
func (m *SequenceMatcher) SetSeq1(a []string) {
|
||||
if &a == &m.a {
|
||||
return
|
||||
}
|
||||
m.a = a
|
||||
m.matchingBlocks = nil
|
||||
m.opCodes = nil
|
||||
}
|
||||
|
||||
// Set the second sequence to be compared. The first sequence to be compared is
|
||||
// not changed.
|
||||
func (m *SequenceMatcher) SetSeq2(b []string) {
|
||||
if &b == &m.b {
|
||||
return
|
||||
}
|
||||
m.b = b
|
||||
m.matchingBlocks = nil
|
||||
m.opCodes = nil
|
||||
m.fullBCount = nil
|
||||
m.chainB()
|
||||
}
|
||||
|
||||
func (m *SequenceMatcher) chainB() {
|
||||
// Populate line -> index mapping
|
||||
b2j := map[string][]int{}
|
||||
for i, s := range m.b {
|
||||
indices := b2j[s]
|
||||
indices = append(indices, i)
|
||||
b2j[s] = indices
|
||||
}
|
||||
|
||||
// Purge junk elements
|
||||
m.bJunk = map[string]struct{}{}
|
||||
if m.IsJunk != nil {
|
||||
junk := m.bJunk
|
||||
for s := range b2j {
|
||||
if m.IsJunk(s) {
|
||||
junk[s] = struct{}{}
|
||||
}
|
||||
}
|
||||
for s := range junk {
|
||||
delete(b2j, s)
|
||||
}
|
||||
}
|
||||
|
||||
// Purge remaining popular elements
|
||||
popular := map[string]struct{}{}
|
||||
n := len(m.b)
|
||||
if m.autoJunk && n >= 200 {
|
||||
ntest := n/100 + 1
|
||||
for s, indices := range b2j {
|
||||
if len(indices) > ntest {
|
||||
popular[s] = struct{}{}
|
||||
}
|
||||
}
|
||||
for s := range popular {
|
||||
delete(b2j, s)
|
||||
}
|
||||
}
|
||||
m.bPopular = popular
|
||||
m.b2j = b2j
|
||||
}
|
||||
|
||||
func (m *SequenceMatcher) isBJunk(s string) bool {
|
||||
_, ok := m.bJunk[s]
|
||||
return ok
|
||||
}
|
||||
|
||||
// Find longest matching block in a[alo:ahi] and b[blo:bhi].
|
||||
//
|
||||
// If IsJunk is not defined:
|
||||
//
|
||||
// Return (i,j,k) such that a[i:i+k] is equal to b[j:j+k], where
|
||||
//
|
||||
// alo <= i <= i+k <= ahi
|
||||
// blo <= j <= j+k <= bhi
|
||||
//
|
||||
// and for all (i',j',k') meeting those conditions,
|
||||
//
|
||||
// k >= k'
|
||||
// i <= i'
|
||||
// and if i == i', j <= j'
|
||||
//
|
||||
// In other words, of all maximal matching blocks, return one that
|
||||
// starts earliest in a, and of all those maximal matching blocks that
|
||||
// start earliest in a, return the one that starts earliest in b.
|
||||
//
|
||||
// If IsJunk is defined, first the longest matching block is
|
||||
// determined as above, but with the additional restriction that no
|
||||
// junk element appears in the block. Then that block is extended as
|
||||
// far as possible by matching (only) junk elements on both sides. So
|
||||
// the resulting block never matches on junk except as identical junk
|
||||
// happens to be adjacent to an "interesting" match.
|
||||
//
|
||||
// If no blocks match, return (alo, blo, 0).
|
||||
func (m *SequenceMatcher) findLongestMatch(alo, ahi, blo, bhi int) Match {
|
||||
// CAUTION: stripping common prefix or suffix would be incorrect.
|
||||
// E.g.,
|
||||
// ab
|
||||
// acab
|
||||
// Longest matching block is "ab", but if common prefix is
|
||||
// stripped, it's "a" (tied with "b"). UNIX(tm) diff does so
|
||||
// strip, so ends up claiming that ab is changed to acab by
|
||||
// inserting "ca" in the middle. That's minimal but unintuitive:
|
||||
// "it's obvious" that someone inserted "ac" at the front.
|
||||
// Windiff ends up at the same place as diff, but by pairing up
|
||||
// the unique 'b's and then matching the first two 'a's.
|
||||
besti, bestj, bestsize := alo, blo, 0
|
||||
|
||||
// find longest junk-free match
|
||||
// during an iteration of the loop, j2len[j] = length of longest
|
||||
// junk-free match ending with a[i-1] and b[j]
|
||||
j2len := map[int]int{}
|
||||
for i := alo; i != ahi; i++ {
|
||||
// look at all instances of a[i] in b; note that because
|
||||
// b2j has no junk keys, the loop is skipped if a[i] is junk
|
||||
newj2len := map[int]int{}
|
||||
for _, j := range m.b2j[m.a[i]] {
|
||||
// a[i] matches b[j]
|
||||
if j < blo {
|
||||
continue
|
||||
}
|
||||
if j >= bhi {
|
||||
break
|
||||
}
|
||||
k := j2len[j-1] + 1
|
||||
newj2len[j] = k
|
||||
if k > bestsize {
|
||||
besti, bestj, bestsize = i-k+1, j-k+1, k
|
||||
}
|
||||
}
|
||||
j2len = newj2len
|
||||
}
|
||||
|
||||
// Extend the best by non-junk elements on each end. In particular,
|
||||
// "popular" non-junk elements aren't in b2j, which greatly speeds
|
||||
// the inner loop above, but also means "the best" match so far
|
||||
// doesn't contain any junk *or* popular non-junk elements.
|
||||
for besti > alo && bestj > blo && !m.isBJunk(m.b[bestj-1]) &&
|
||||
m.a[besti-1] == m.b[bestj-1] {
|
||||
besti, bestj, bestsize = besti-1, bestj-1, bestsize+1
|
||||
}
|
||||
for besti+bestsize < ahi && bestj+bestsize < bhi &&
|
||||
!m.isBJunk(m.b[bestj+bestsize]) &&
|
||||
m.a[besti+bestsize] == m.b[bestj+bestsize] {
|
||||
bestsize++
|
||||
}
|
||||
|
||||
// Now that we have a wholly interesting match (albeit possibly
|
||||
// empty!), we may as well suck up the matching junk on each
|
||||
// side of it too. Can't think of a good reason not to, and it
|
||||
// saves post-processing the (possibly considerable) expense of
|
||||
// figuring out what to do with it. In the case of an empty
|
||||
// interesting match, this is clearly the right thing to do,
|
||||
// because no other kind of match is possible in the regions.
|
||||
for besti > alo && bestj > blo && m.isBJunk(m.b[bestj-1]) &&
|
||||
m.a[besti-1] == m.b[bestj-1] {
|
||||
besti, bestj, bestsize = besti-1, bestj-1, bestsize+1
|
||||
}
|
||||
for besti+bestsize < ahi && bestj+bestsize < bhi &&
|
||||
m.isBJunk(m.b[bestj+bestsize]) &&
|
||||
m.a[besti+bestsize] == m.b[bestj+bestsize] {
|
||||
bestsize++
|
||||
}
|
||||
|
||||
return Match{A: besti, B: bestj, Size: bestsize}
|
||||
}
|
||||
|
||||
// Return list of triples describing matching subsequences.
|
||||
//
|
||||
// Each triple is of the form (i, j, n), and means that
|
||||
// a[i:i+n] == b[j:j+n]. The triples are monotonically increasing in
|
||||
// i and in j. It's also guaranteed that if (i, j, n) and (i', j', n') are
|
||||
// adjacent triples in the list, and the second is not the last triple in the
|
||||
// list, then i+n != i' or j+n != j'. IOW, adjacent triples never describe
|
||||
// adjacent equal blocks.
|
||||
//
|
||||
// The last triple is a dummy, (len(a), len(b), 0), and is the only
|
||||
// triple with n==0.
|
||||
func (m *SequenceMatcher) GetMatchingBlocks() []Match {
|
||||
if m.matchingBlocks != nil {
|
||||
return m.matchingBlocks
|
||||
}
|
||||
|
||||
var matchBlocks func(alo, ahi, blo, bhi int, matched []Match) []Match
|
||||
matchBlocks = func(alo, ahi, blo, bhi int, matched []Match) []Match {
|
||||
match := m.findLongestMatch(alo, ahi, blo, bhi)
|
||||
i, j, k := match.A, match.B, match.Size
|
||||
if match.Size > 0 {
|
||||
if alo < i && blo < j {
|
||||
matched = matchBlocks(alo, i, blo, j, matched)
|
||||
}
|
||||
matched = append(matched, match)
|
||||
if i+k < ahi && j+k < bhi {
|
||||
matched = matchBlocks(i+k, ahi, j+k, bhi, matched)
|
||||
}
|
||||
}
|
||||
return matched
|
||||
}
|
||||
matched := matchBlocks(0, len(m.a), 0, len(m.b), nil)
|
||||
|
||||
// It's possible that we have adjacent equal blocks in the
|
||||
// matching_blocks list now.
|
||||
nonAdjacent := []Match{}
|
||||
i1, j1, k1 := 0, 0, 0
|
||||
for _, b := range matched {
|
||||
// Is this block adjacent to i1, j1, k1?
|
||||
i2, j2, k2 := b.A, b.B, b.Size
|
||||
if i1+k1 == i2 && j1+k1 == j2 {
|
||||
// Yes, so collapse them -- this just increases the length of
|
||||
// the first block by the length of the second, and the first
|
||||
// block so lengthened remains the block to compare against.
|
||||
k1 += k2
|
||||
} else {
|
||||
// Not adjacent. Remember the first block (k1==0 means it's
|
||||
// the dummy we started with), and make the second block the
|
||||
// new block to compare against.
|
||||
if k1 > 0 {
|
||||
nonAdjacent = append(nonAdjacent, Match{i1, j1, k1})
|
||||
}
|
||||
i1, j1, k1 = i2, j2, k2
|
||||
}
|
||||
}
|
||||
if k1 > 0 {
|
||||
nonAdjacent = append(nonAdjacent, Match{i1, j1, k1})
|
||||
}
|
||||
|
||||
nonAdjacent = append(nonAdjacent, Match{len(m.a), len(m.b), 0})
|
||||
m.matchingBlocks = nonAdjacent
|
||||
return m.matchingBlocks
|
||||
}
|
||||
|
||||
// Return list of 5-tuples describing how to turn a into b.
|
||||
//
|
||||
// Each tuple is of the form (tag, i1, i2, j1, j2). The first tuple
|
||||
// has i1 == j1 == 0, and remaining tuples have i1 == the i2 from the
|
||||
// tuple preceding it, and likewise for j1 == the previous j2.
|
||||
//
|
||||
// The tags are characters, with these meanings:
|
||||
//
|
||||
// 'r' (replace): a[i1:i2] should be replaced by b[j1:j2]
|
||||
//
|
||||
// 'd' (delete): a[i1:i2] should be deleted, j1==j2 in this case.
|
||||
//
|
||||
// 'i' (insert): b[j1:j2] should be inserted at a[i1:i1], i1==i2 in this case.
|
||||
//
|
||||
// 'e' (equal): a[i1:i2] == b[j1:j2]
|
||||
func (m *SequenceMatcher) GetOpCodes() []OpCode {
|
||||
if m.opCodes != nil {
|
||||
return m.opCodes
|
||||
}
|
||||
i, j := 0, 0
|
||||
matching := m.GetMatchingBlocks()
|
||||
opCodes := make([]OpCode, 0, len(matching))
|
||||
for _, m := range matching {
|
||||
// invariant: we've pumped out correct diffs to change
|
||||
// a[:i] into b[:j], and the next matching block is
|
||||
// a[ai:ai+size] == b[bj:bj+size]. So we need to pump
|
||||
// out a diff to change a[i:ai] into b[j:bj], pump out
|
||||
// the matching block, and move (i,j) beyond the match
|
||||
ai, bj, size := m.A, m.B, m.Size
|
||||
tag := byte(0)
|
||||
if i < ai && j < bj {
|
||||
tag = 'r'
|
||||
} else if i < ai {
|
||||
tag = 'd'
|
||||
} else if j < bj {
|
||||
tag = 'i'
|
||||
}
|
||||
if tag > 0 {
|
||||
opCodes = append(opCodes, OpCode{tag, i, ai, j, bj})
|
||||
}
|
||||
i, j = ai+size, bj+size
|
||||
// the list of matching blocks is terminated by a
|
||||
// sentinel with size 0
|
||||
if size > 0 {
|
||||
opCodes = append(opCodes, OpCode{'e', ai, i, bj, j})
|
||||
}
|
||||
}
|
||||
m.opCodes = opCodes
|
||||
return m.opCodes
|
||||
}
|
||||
|
||||
// Isolate change clusters by eliminating ranges with no changes.
|
||||
//
|
||||
// Return a generator of groups with up to n lines of context.
|
||||
// Each group is in the same format as returned by GetOpCodes().
|
||||
func (m *SequenceMatcher) GetGroupedOpCodes(n int) [][]OpCode {
|
||||
if n < 0 {
|
||||
n = 3
|
||||
}
|
||||
codes := m.GetOpCodes()
|
||||
if len(codes) == 0 {
|
||||
codes = []OpCode{{'e', 0, 1, 0, 1}}
|
||||
}
|
||||
// Fixup leading and trailing groups if they show no changes.
|
||||
if codes[0].Tag == 'e' {
|
||||
c := codes[0]
|
||||
i1, i2, j1, j2 := c.I1, c.I2, c.J1, c.J2
|
||||
codes[0] = OpCode{c.Tag, maxInt(i1, i2-n), i2, maxInt(j1, j2-n), j2}
|
||||
}
|
||||
if codes[len(codes)-1].Tag == 'e' {
|
||||
c := codes[len(codes)-1]
|
||||
i1, i2, j1, j2 := c.I1, c.I2, c.J1, c.J2
|
||||
codes[len(codes)-1] = OpCode{c.Tag, i1, minInt(i2, i1+n), j1, minInt(j2, j1+n)}
|
||||
}
|
||||
nn := n + n
|
||||
groups := [][]OpCode{}
|
||||
group := []OpCode{}
|
||||
for _, c := range codes {
|
||||
i1, i2, j1, j2 := c.I1, c.I2, c.J1, c.J2
|
||||
// End the current group and start a new one whenever
|
||||
// there is a large range with no changes.
|
||||
if c.Tag == 'e' && i2-i1 > nn {
|
||||
group = append(group, OpCode{
|
||||
c.Tag, i1, minInt(i2, i1+n),
|
||||
j1, minInt(j2, j1+n),
|
||||
})
|
||||
groups = append(groups, group)
|
||||
group = []OpCode{}
|
||||
i1, j1 = maxInt(i1, i2-n), maxInt(j1, j2-n)
|
||||
}
|
||||
group = append(group, OpCode{c.Tag, i1, i2, j1, j2})
|
||||
}
|
||||
if len(group) > 0 && (len(group) != 1 || group[0].Tag != 'e') {
|
||||
groups = append(groups, group)
|
||||
}
|
||||
return groups
|
||||
}
|
||||
|
||||
// Return a measure of the sequences' similarity (float in [0,1]).
|
||||
//
|
||||
// Where T is the total number of elements in both sequences, and
|
||||
// M is the number of matches, this is 2.0*M / T.
|
||||
// Note that this is 1 if the sequences are identical, and 0 if
|
||||
// they have nothing in common.
|
||||
//
|
||||
// .Ratio() is expensive to compute if you haven't already computed
|
||||
// .GetMatchingBlocks() or .GetOpCodes(), in which case you may
|
||||
// want to try .QuickRatio() or .RealQuickRation() first to get an
|
||||
// upper bound.
|
||||
func (m *SequenceMatcher) Ratio() float64 {
|
||||
matches := 0
|
||||
for _, m := range m.GetMatchingBlocks() {
|
||||
matches += m.Size
|
||||
}
|
||||
return calculateRatio(matches, len(m.a)+len(m.b))
|
||||
}
|
||||
|
||||
// Return an upper bound on ratio() relatively quickly.
|
||||
//
|
||||
// This isn't defined beyond that it is an upper bound on .Ratio(), and
|
||||
// is faster to compute.
|
||||
func (m *SequenceMatcher) QuickRatio() float64 {
|
||||
// viewing a and b as multisets, set matches to the cardinality
|
||||
// of their intersection; this counts the number of matches
|
||||
// without regard to order, so is clearly an upper bound
|
||||
if m.fullBCount == nil {
|
||||
m.fullBCount = map[string]int{}
|
||||
for _, s := range m.b {
|
||||
m.fullBCount[s]++
|
||||
}
|
||||
}
|
||||
|
||||
// avail[x] is the number of times x appears in 'b' less the
|
||||
// number of times we've seen it in 'a' so far ... kinda
|
||||
avail := map[string]int{}
|
||||
matches := 0
|
||||
for _, s := range m.a {
|
||||
n, ok := avail[s]
|
||||
if !ok {
|
||||
n = m.fullBCount[s]
|
||||
}
|
||||
avail[s] = n - 1
|
||||
if n > 0 {
|
||||
matches++
|
||||
}
|
||||
}
|
||||
return calculateRatio(matches, len(m.a)+len(m.b))
|
||||
}
|
||||
|
||||
// Return an upper bound on ratio() very quickly.
|
||||
//
|
||||
// This isn't defined beyond that it is an upper bound on .Ratio(), and
|
||||
// is faster to compute than either .Ratio() or .QuickRatio().
|
||||
func (m *SequenceMatcher) RealQuickRatio() float64 {
|
||||
la, lb := len(m.a), len(m.b)
|
||||
return calculateRatio(minInt(la, lb), la+lb)
|
||||
}
|
||||
|
||||
// Convert range to the "ed" format
|
||||
func formatRangeUnified(start, stop int) string {
|
||||
// Per the diff spec at http://www.unix.org/single_unix_specification/
|
||||
beginning := start + 1 // lines start numbering with one
|
||||
length := stop - start
|
||||
if length == 1 {
|
||||
return strconv.Itoa(beginning)
|
||||
}
|
||||
if length == 0 {
|
||||
beginning-- // empty ranges begin at line just before the range
|
||||
}
|
||||
return fmt.Sprintf("%d,%d", beginning, length)
|
||||
}
|
||||
|
||||
// Unified diff parameters
|
||||
type UnifiedDiff struct {
|
||||
A []string // First sequence lines
|
||||
FromFile string // First file name
|
||||
FromDate string // First file time
|
||||
B []string // Second sequence lines
|
||||
ToFile string // Second file name
|
||||
ToDate string // Second file time
|
||||
Eol string // Headers end of line, defaults to LF
|
||||
Context int // Number of context lines
|
||||
}
|
||||
|
||||
// Compare two sequences of lines; generate the delta as a unified diff.
|
||||
//
|
||||
// Unified diffs are a compact way of showing line changes and a few
|
||||
// lines of context. The number of context lines is set by 'n' which
|
||||
// defaults to three.
|
||||
//
|
||||
// By default, the diff control lines (those with ---, +++, or @@) are
|
||||
// created with a trailing newline. This is helpful so that inputs
|
||||
// created from file.readlines() result in diffs that are suitable for
|
||||
// file.writelines() since both the inputs and outputs have trailing
|
||||
// newlines.
|
||||
//
|
||||
// For inputs that do not have trailing newlines, set the lineterm
|
||||
// argument to "" so that the output will be uniformly newline free.
|
||||
//
|
||||
// The unidiff format normally has a header for filenames and modification
|
||||
// times. Any or all of these may be specified using strings for
|
||||
// 'fromfile', 'tofile', 'fromfiledate', and 'tofiledate'.
|
||||
// The modification times are normally expressed in the ISO 8601 format.
|
||||
func WriteUnifiedDiff(writer io.Writer, diff UnifiedDiff) error {
|
||||
buf := bufio.NewWriter(writer)
|
||||
defer buf.Flush()
|
||||
wf := func(format string, args ...any) error {
|
||||
_, err := fmt.Fprintf(buf, format, args...)
|
||||
return err
|
||||
}
|
||||
ws := func(s string) error {
|
||||
_, err := buf.WriteString(s)
|
||||
return err
|
||||
}
|
||||
|
||||
if len(diff.Eol) == 0 {
|
||||
diff.Eol = "\n"
|
||||
}
|
||||
|
||||
started := false
|
||||
m := NewMatcher(diff.A, diff.B)
|
||||
for _, g := range m.GetGroupedOpCodes(diff.Context) {
|
||||
if !started {
|
||||
started = true
|
||||
fromDate := ""
|
||||
if len(diff.FromDate) > 0 {
|
||||
fromDate = "\t" + diff.FromDate
|
||||
}
|
||||
toDate := ""
|
||||
if len(diff.ToDate) > 0 {
|
||||
toDate = "\t" + diff.ToDate
|
||||
}
|
||||
if diff.FromFile != "" || diff.ToFile != "" {
|
||||
err := wf("--- %s%s%s", diff.FromFile, fromDate, diff.Eol)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
err = wf("+++ %s%s%s", diff.ToFile, toDate, diff.Eol)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
first, last := g[0], g[len(g)-1]
|
||||
range1 := formatRangeUnified(first.I1, last.I2)
|
||||
range2 := formatRangeUnified(first.J1, last.J2)
|
||||
if err := wf("@@ -%s +%s @@%s", range1, range2, diff.Eol); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, c := range g {
|
||||
i1, i2, j1, j2 := c.I1, c.I2, c.J1, c.J2
|
||||
if c.Tag == 'e' {
|
||||
for _, line := range diff.A[i1:i2] {
|
||||
if err := ws(" " + line); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
if c.Tag == 'r' || c.Tag == 'd' {
|
||||
for _, line := range diff.A[i1:i2] {
|
||||
if err := ws("-" + line); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
if c.Tag == 'r' || c.Tag == 'i' {
|
||||
for _, line := range diff.B[j1:j2] {
|
||||
if err := ws("+" + line); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Like WriteUnifiedDiff but returns the diff a string.
|
||||
func GetUnifiedDiffString(diff UnifiedDiff) (string, error) {
|
||||
w := &bytes.Buffer{}
|
||||
err := WriteUnifiedDiff(w, diff)
|
||||
return w.String(), err
|
||||
}
|
||||
|
||||
// Split a string on "\n" while preserving them. The output can be used
|
||||
// as input for UnifiedDiff and ContextDiff structures.
|
||||
func SplitLines(s string) []string {
|
||||
lines := strings.SplitAfter(s, "\n")
|
||||
lines[len(lines)-1] += "\n"
|
||||
return lines
|
||||
}
|
||||
34
vendor/github.com/prometheus/client_golang/prometheus/internal/go_collector_options.go
generated
vendored
Normal file
34
vendor/github.com/prometheus/client_golang/prometheus/internal/go_collector_options.go
generated
vendored
Normal file
@ -0,0 +1,34 @@
|
||||
// Copyright 2021 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package internal
|
||||
|
||||
import "regexp"
|
||||
|
||||
type GoCollectorRule struct {
|
||||
Matcher *regexp.Regexp
|
||||
Deny bool
|
||||
}
|
||||
|
||||
// GoCollectorOptions should not be used be directly by anything, except `collectors` package.
|
||||
// Use it via collectors package instead. See issue
|
||||
// https://github.com/prometheus/client_golang/issues/1030.
|
||||
//
|
||||
// This is internal, so external users only can use it via `collector.WithGoCollector*` methods
|
||||
type GoCollectorOptions struct {
|
||||
DisableMemStatsLikeMetrics bool
|
||||
RuntimeMetricSumForHist map[string]string
|
||||
RuntimeMetricRules []GoCollectorRule
|
||||
}
|
||||
|
||||
var GoCollectorDefaultRuntimeMetrics = regexp.MustCompile(`/gc/gogc:percent|/gc/gomemlimit:bytes|/sched/gomaxprocs:threads`)
|
||||
143
vendor/github.com/prometheus/client_golang/prometheus/internal/go_runtime_metrics.go
generated
vendored
Normal file
143
vendor/github.com/prometheus/client_golang/prometheus/internal/go_runtime_metrics.go
generated
vendored
Normal file
@ -0,0 +1,143 @@
|
||||
// Copyright 2021 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
//go:build go1.17
|
||||
// +build go1.17
|
||||
|
||||
package internal
|
||||
|
||||
import (
|
||||
"math"
|
||||
"path"
|
||||
"runtime/metrics"
|
||||
"strings"
|
||||
|
||||
"github.com/prometheus/common/model"
|
||||
)
|
||||
|
||||
// RuntimeMetricsToProm produces a Prometheus metric name from a runtime/metrics
|
||||
// metric description and validates whether the metric is suitable for integration
|
||||
// with Prometheus.
|
||||
//
|
||||
// Returns false if a name could not be produced, or if Prometheus does not understand
|
||||
// the runtime/metrics Kind.
|
||||
//
|
||||
// Note that the main reason a name couldn't be produced is if the runtime/metrics
|
||||
// package exports a name with characters outside the valid Prometheus metric name
|
||||
// character set. This is theoretically possible, but should never happen in practice.
|
||||
// Still, don't rely on it.
|
||||
func RuntimeMetricsToProm(d *metrics.Description) (string, string, string, bool) {
|
||||
namespace := "go"
|
||||
|
||||
comp := strings.SplitN(d.Name, ":", 2)
|
||||
key := comp[0]
|
||||
unit := comp[1]
|
||||
|
||||
// The last path element in the key is the name,
|
||||
// the rest is the subsystem.
|
||||
subsystem := path.Dir(key[1:] /* remove leading / */)
|
||||
name := path.Base(key)
|
||||
|
||||
// subsystem is translated by replacing all / and - with _.
|
||||
subsystem = strings.ReplaceAll(subsystem, "/", "_")
|
||||
subsystem = strings.ReplaceAll(subsystem, "-", "_")
|
||||
|
||||
// unit is translated assuming that the unit contains no
|
||||
// non-ASCII characters.
|
||||
unit = strings.ReplaceAll(unit, "-", "_")
|
||||
unit = strings.ReplaceAll(unit, "*", "_")
|
||||
unit = strings.ReplaceAll(unit, "/", "_per_")
|
||||
|
||||
// name has - replaced with _ and is concatenated with the unit and
|
||||
// other data.
|
||||
name = strings.ReplaceAll(name, "-", "_")
|
||||
name += "_" + unit
|
||||
if d.Cumulative && d.Kind != metrics.KindFloat64Histogram {
|
||||
name += "_total"
|
||||
}
|
||||
|
||||
// Our current conversion moves to legacy naming, so use legacy validation.
|
||||
valid := model.LegacyValidation.IsValidMetricName(namespace + "_" + subsystem + "_" + name)
|
||||
switch d.Kind {
|
||||
case metrics.KindUint64:
|
||||
case metrics.KindFloat64:
|
||||
case metrics.KindFloat64Histogram:
|
||||
default:
|
||||
valid = false
|
||||
}
|
||||
return namespace, subsystem, name, valid
|
||||
}
|
||||
|
||||
// RuntimeMetricsBucketsForUnit takes a set of buckets obtained for a runtime/metrics histogram
|
||||
// type (so, lower-bound inclusive) and a unit from a runtime/metrics name, and produces
|
||||
// a reduced set of buckets. This function always removes any -Inf bucket as it's represented
|
||||
// as the bottom-most upper-bound inclusive bucket in Prometheus.
|
||||
func RuntimeMetricsBucketsForUnit(buckets []float64, unit string) []float64 {
|
||||
switch unit {
|
||||
case "bytes":
|
||||
// Re-bucket as powers of 2.
|
||||
return reBucketExp(buckets, 2)
|
||||
case "seconds":
|
||||
// Re-bucket as powers of 10 and then merge all buckets greater
|
||||
// than 1 second into the +Inf bucket.
|
||||
b := reBucketExp(buckets, 10)
|
||||
for i := range b {
|
||||
if b[i] <= 1 {
|
||||
continue
|
||||
}
|
||||
b[i] = math.Inf(1)
|
||||
b = b[:i+1]
|
||||
break
|
||||
}
|
||||
return b
|
||||
}
|
||||
return buckets
|
||||
}
|
||||
|
||||
// reBucketExp takes a list of bucket boundaries (lower bound inclusive) and
|
||||
// downsamples the buckets to those a multiple of base apart. The end result
|
||||
// is a roughly exponential (in many cases, perfectly exponential) bucketing
|
||||
// scheme.
|
||||
func reBucketExp(buckets []float64, base float64) []float64 {
|
||||
bucket := buckets[0]
|
||||
var newBuckets []float64
|
||||
// We may see a -Inf here, in which case, add it and skip it
|
||||
// since we risk producing NaNs otherwise.
|
||||
//
|
||||
// We need to preserve -Inf values to maintain runtime/metrics
|
||||
// conventions. We'll strip it out later.
|
||||
if bucket == math.Inf(-1) {
|
||||
newBuckets = append(newBuckets, bucket)
|
||||
buckets = buckets[1:]
|
||||
bucket = buckets[0]
|
||||
}
|
||||
// From now on, bucket should always have a non-Inf value because
|
||||
// Infs are only ever at the ends of the bucket lists, so
|
||||
// arithmetic operations on it are non-NaN.
|
||||
for i := 1; i < len(buckets); i++ {
|
||||
if bucket >= 0 && buckets[i] < bucket*base {
|
||||
// The next bucket we want to include is at least bucket*base.
|
||||
continue
|
||||
} else if bucket < 0 && buckets[i] < bucket/base {
|
||||
// In this case the bucket we're targeting is negative, and since
|
||||
// we're ascending through buckets here, we need to divide to get
|
||||
// closer to zero exponentially.
|
||||
continue
|
||||
}
|
||||
// The +Inf bucket will always be the last one, and we'll always
|
||||
// end up including it here because bucket
|
||||
newBuckets = append(newBuckets, bucket)
|
||||
bucket = buckets[i]
|
||||
}
|
||||
return append(newBuckets, bucket)
|
||||
}
|
||||
101
vendor/github.com/prometheus/client_golang/prometheus/internal/metric.go
generated
vendored
Normal file
101
vendor/github.com/prometheus/client_golang/prometheus/internal/metric.go
generated
vendored
Normal file
@ -0,0 +1,101 @@
|
||||
// Copyright 2018 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package internal
|
||||
|
||||
import (
|
||||
"sort"
|
||||
|
||||
dto "github.com/prometheus/client_model/go"
|
||||
)
|
||||
|
||||
// LabelPairSorter implements sort.Interface. It is used to sort a slice of
|
||||
// dto.LabelPair pointers.
|
||||
type LabelPairSorter []*dto.LabelPair
|
||||
|
||||
func (s LabelPairSorter) Len() int {
|
||||
return len(s)
|
||||
}
|
||||
|
||||
func (s LabelPairSorter) Swap(i, j int) {
|
||||
s[i], s[j] = s[j], s[i]
|
||||
}
|
||||
|
||||
func (s LabelPairSorter) Less(i, j int) bool {
|
||||
return s[i].GetName() < s[j].GetName()
|
||||
}
|
||||
|
||||
// MetricSorter is a sortable slice of *dto.Metric.
|
||||
type MetricSorter []*dto.Metric
|
||||
|
||||
func (s MetricSorter) Len() int {
|
||||
return len(s)
|
||||
}
|
||||
|
||||
func (s MetricSorter) Swap(i, j int) {
|
||||
s[i], s[j] = s[j], s[i]
|
||||
}
|
||||
|
||||
func (s MetricSorter) Less(i, j int) bool {
|
||||
if len(s[i].Label) != len(s[j].Label) {
|
||||
// This should not happen. The metrics are
|
||||
// inconsistent. However, we have to deal with the fact, as
|
||||
// people might use custom collectors or metric family injection
|
||||
// to create inconsistent metrics. So let's simply compare the
|
||||
// number of labels in this case. That will still yield
|
||||
// reproducible sorting.
|
||||
return len(s[i].Label) < len(s[j].Label)
|
||||
}
|
||||
for n, lp := range s[i].Label {
|
||||
vi := lp.GetValue()
|
||||
vj := s[j].Label[n].GetValue()
|
||||
if vi != vj {
|
||||
return vi < vj
|
||||
}
|
||||
}
|
||||
|
||||
// We should never arrive here. Multiple metrics with the same
|
||||
// label set in the same scrape will lead to undefined ingestion
|
||||
// behavior. However, as above, we have to provide stable sorting
|
||||
// here, even for inconsistent metrics. So sort equal metrics
|
||||
// by their timestamp, with missing timestamps (implying "now")
|
||||
// coming last.
|
||||
if s[i].TimestampMs == nil {
|
||||
return false
|
||||
}
|
||||
if s[j].TimestampMs == nil {
|
||||
return true
|
||||
}
|
||||
return s[i].GetTimestampMs() < s[j].GetTimestampMs()
|
||||
}
|
||||
|
||||
// NormalizeMetricFamilies returns a MetricFamily slice with empty
|
||||
// MetricFamilies pruned and the remaining MetricFamilies sorted by name within
|
||||
// the slice, with the contained Metrics sorted within each MetricFamily.
|
||||
func NormalizeMetricFamilies(metricFamiliesByName map[string]*dto.MetricFamily) []*dto.MetricFamily {
|
||||
for _, mf := range metricFamiliesByName {
|
||||
sort.Sort(MetricSorter(mf.Metric))
|
||||
}
|
||||
names := make([]string, 0, len(metricFamiliesByName))
|
||||
for name, mf := range metricFamiliesByName {
|
||||
if len(mf.Metric) > 0 {
|
||||
names = append(names, name)
|
||||
}
|
||||
}
|
||||
sort.Strings(names)
|
||||
result := make([]*dto.MetricFamily, 0, len(names))
|
||||
for _, name := range names {
|
||||
result = append(result, metricFamiliesByName[name])
|
||||
}
|
||||
return result
|
||||
}
|
||||
188
vendor/github.com/prometheus/client_golang/prometheus/labels.go
generated
vendored
Normal file
188
vendor/github.com/prometheus/client_golang/prometheus/labels.go
generated
vendored
Normal file
@ -0,0 +1,188 @@
|
||||
// Copyright 2018 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package prometheus
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/prometheus/common/model"
|
||||
)
|
||||
|
||||
// Labels represents a collection of label name -> value mappings. This type is
|
||||
// commonly used with the With(Labels) and GetMetricWith(Labels) methods of
|
||||
// metric vector Collectors, e.g.:
|
||||
//
|
||||
// myVec.With(Labels{"code": "404", "method": "GET"}).Add(42)
|
||||
//
|
||||
// The other use-case is the specification of constant label pairs in Opts or to
|
||||
// create a Desc.
|
||||
type Labels map[string]string
|
||||
|
||||
// LabelConstraint normalizes label values.
|
||||
type LabelConstraint func(string) string
|
||||
|
||||
// ConstrainedLabels represents a label name and its constrain function
|
||||
// to normalize label values. This type is commonly used when constructing
|
||||
// metric vector Collectors.
|
||||
type ConstrainedLabel struct {
|
||||
Name string
|
||||
Constraint LabelConstraint
|
||||
}
|
||||
|
||||
// ConstrainableLabels is an interface that allows creating of labels that can
|
||||
// be optionally constrained.
|
||||
//
|
||||
// prometheus.V2().NewCounterVec(CounterVecOpts{
|
||||
// CounterOpts: {...}, // Usual CounterOpts fields
|
||||
// VariableLabels: []ConstrainedLabels{
|
||||
// {Name: "A"},
|
||||
// {Name: "B", Constraint: func(v string) string { ... }},
|
||||
// },
|
||||
// })
|
||||
type ConstrainableLabels interface {
|
||||
compile() *compiledLabels
|
||||
labelNames() []string
|
||||
}
|
||||
|
||||
// ConstrainedLabels represents a collection of label name -> constrain function
|
||||
// to normalize label values. This type is commonly used when constructing
|
||||
// metric vector Collectors.
|
||||
type ConstrainedLabels []ConstrainedLabel
|
||||
|
||||
func (cls ConstrainedLabels) compile() *compiledLabels {
|
||||
compiled := &compiledLabels{
|
||||
names: make([]string, len(cls)),
|
||||
labelConstraints: map[string]LabelConstraint{},
|
||||
}
|
||||
|
||||
for i, label := range cls {
|
||||
compiled.names[i] = label.Name
|
||||
if label.Constraint != nil {
|
||||
compiled.labelConstraints[label.Name] = label.Constraint
|
||||
}
|
||||
}
|
||||
|
||||
return compiled
|
||||
}
|
||||
|
||||
func (cls ConstrainedLabels) labelNames() []string {
|
||||
names := make([]string, len(cls))
|
||||
for i, label := range cls {
|
||||
names[i] = label.Name
|
||||
}
|
||||
return names
|
||||
}
|
||||
|
||||
// UnconstrainedLabels represents collection of label without any constraint on
|
||||
// their value. Thus, it is simply a collection of label names.
|
||||
//
|
||||
// UnconstrainedLabels([]string{ "A", "B" })
|
||||
//
|
||||
// is equivalent to
|
||||
//
|
||||
// ConstrainedLabels {
|
||||
// { Name: "A" },
|
||||
// { Name: "B" },
|
||||
// }
|
||||
type UnconstrainedLabels []string
|
||||
|
||||
func (uls UnconstrainedLabels) compile() *compiledLabels {
|
||||
return &compiledLabels{
|
||||
names: uls,
|
||||
}
|
||||
}
|
||||
|
||||
func (uls UnconstrainedLabels) labelNames() []string {
|
||||
return uls
|
||||
}
|
||||
|
||||
type compiledLabels struct {
|
||||
names []string
|
||||
labelConstraints map[string]LabelConstraint
|
||||
}
|
||||
|
||||
func (cls *compiledLabels) compile() *compiledLabels {
|
||||
return cls
|
||||
}
|
||||
|
||||
func (cls *compiledLabels) labelNames() []string {
|
||||
return cls.names
|
||||
}
|
||||
|
||||
func (cls *compiledLabels) constrain(labelName, value string) string {
|
||||
if fn, ok := cls.labelConstraints[labelName]; ok && fn != nil {
|
||||
return fn(value)
|
||||
}
|
||||
return value
|
||||
}
|
||||
|
||||
// reservedLabelPrefix is a prefix which is not legal in user-supplied
|
||||
// label names.
|
||||
const reservedLabelPrefix = "__"
|
||||
|
||||
var errInconsistentCardinality = errors.New("inconsistent label cardinality")
|
||||
|
||||
func makeInconsistentCardinalityError(fqName string, labels, labelValues []string) error {
|
||||
return fmt.Errorf(
|
||||
"%w: %q has %d variable labels named %q but %d values %q were provided",
|
||||
errInconsistentCardinality, fqName,
|
||||
len(labels), labels,
|
||||
len(labelValues), labelValues,
|
||||
)
|
||||
}
|
||||
|
||||
func validateValuesInLabels(labels Labels, expectedNumberOfValues int) error {
|
||||
if len(labels) != expectedNumberOfValues {
|
||||
return fmt.Errorf(
|
||||
"%w: expected %d label values but got %d in %#v",
|
||||
errInconsistentCardinality, expectedNumberOfValues,
|
||||
len(labels), labels,
|
||||
)
|
||||
}
|
||||
|
||||
for name, val := range labels {
|
||||
if !utf8.ValidString(val) {
|
||||
return fmt.Errorf("label %s: value %q is not valid UTF-8", name, val)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func validateLabelValues(vals []string, expectedNumberOfValues int) error {
|
||||
if len(vals) != expectedNumberOfValues {
|
||||
// The call below makes vals escape, copy them to avoid that.
|
||||
vals := append([]string(nil), vals...)
|
||||
return fmt.Errorf(
|
||||
"%w: expected %d label values but got %d in %#v",
|
||||
errInconsistentCardinality, expectedNumberOfValues,
|
||||
len(vals), vals,
|
||||
)
|
||||
}
|
||||
|
||||
for _, val := range vals {
|
||||
if !utf8.ValidString(val) {
|
||||
return fmt.Errorf("label value %q is not valid UTF-8", val)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func checkLabelName(l string) bool {
|
||||
return model.UTF8Validation.IsValidLabelName(l) && !strings.HasPrefix(l, reservedLabelPrefix)
|
||||
}
|
||||
279
vendor/github.com/prometheus/client_golang/prometheus/metric.go
generated
vendored
Normal file
279
vendor/github.com/prometheus/client_golang/prometheus/metric.go
generated
vendored
Normal file
@ -0,0 +1,279 @@
|
||||
// Copyright 2014 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package prometheus
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"math"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
dto "github.com/prometheus/client_model/go"
|
||||
"github.com/prometheus/common/model"
|
||||
"google.golang.org/protobuf/proto"
|
||||
)
|
||||
|
||||
var separatorByteSlice = []byte{model.SeparatorByte} // For convenient use with xxhash.
|
||||
|
||||
// A Metric models a single sample value with its meta data being exported to
|
||||
// Prometheus. Implementations of Metric in this package are Gauge, Counter,
|
||||
// Histogram, Summary, and Untyped.
|
||||
type Metric interface {
|
||||
// Desc returns the descriptor for the Metric. This method idempotently
|
||||
// returns the same descriptor throughout the lifetime of the
|
||||
// Metric. The returned descriptor is immutable by contract. A Metric
|
||||
// unable to describe itself must return an invalid descriptor (created
|
||||
// with NewInvalidDesc).
|
||||
Desc() *Desc
|
||||
// Write encodes the Metric into a "Metric" Protocol Buffer data
|
||||
// transmission object.
|
||||
//
|
||||
// Metric implementations must observe concurrency safety as reads of
|
||||
// this metric may occur at any time, and any blocking occurs at the
|
||||
// expense of total performance of rendering all registered
|
||||
// metrics. Ideally, Metric implementations should support concurrent
|
||||
// readers.
|
||||
//
|
||||
// While populating dto.Metric, it is the responsibility of the
|
||||
// implementation to ensure validity of the Metric protobuf (like valid
|
||||
// UTF-8 strings or syntactically valid metric and label names). It is
|
||||
// recommended to sort labels lexicographically. Callers of Write should
|
||||
// still make sure of sorting if they depend on it.
|
||||
Write(*dto.Metric) error
|
||||
// TODO(beorn7): The original rationale of passing in a pre-allocated
|
||||
// dto.Metric protobuf to save allocations has disappeared. The
|
||||
// signature of this method should be changed to "Write() (*dto.Metric,
|
||||
// error)".
|
||||
}
|
||||
|
||||
// Opts bundles the options for creating most Metric types. Each metric
|
||||
// implementation XXX has its own XXXOpts type, but in most cases, it is just
|
||||
// an alias of this type (which might change when the requirement arises.)
|
||||
//
|
||||
// It is mandatory to set Name to a non-empty string. All other fields are
|
||||
// optional and can safely be left at their zero value, although it is strongly
|
||||
// encouraged to set a Help string.
|
||||
type Opts struct {
|
||||
// Namespace, Subsystem, and Name are components of the fully-qualified
|
||||
// name of the Metric (created by joining these components with
|
||||
// "_"). Only Name is mandatory, the others merely help structuring the
|
||||
// name. Note that the fully-qualified name of the metric must be a
|
||||
// valid Prometheus metric name.
|
||||
Namespace string
|
||||
Subsystem string
|
||||
Name string
|
||||
|
||||
// Help provides information about this metric.
|
||||
//
|
||||
// Metrics with the same fully-qualified name must have the same Help
|
||||
// string.
|
||||
Help string
|
||||
|
||||
// Unit provides the unit of this metric as per https://prometheus.io/docs/specs/om
|
||||
Unit string
|
||||
|
||||
// ConstLabels are used to attach fixed labels to this metric. Metrics
|
||||
// with the same fully-qualified name must have the same label names in
|
||||
// their ConstLabels.
|
||||
//
|
||||
// ConstLabels are only used rarely. In particular, do not use them to
|
||||
// attach the same labels to all your metrics. Those use cases are
|
||||
// better covered by target labels set by the scraping Prometheus
|
||||
// server, or by one specific metric (e.g. a build_info or a
|
||||
// machine_role metric). See also
|
||||
// https://prometheus.io/docs/instrumenting/writing_exporters/#target-labels-not-static-scraped-labels
|
||||
ConstLabels Labels
|
||||
|
||||
// now is for testing purposes, by default it's time.Now.
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
// BuildFQName joins the given three name components by "_". Empty name
|
||||
// components are ignored. If the name parameter itself is empty, an empty
|
||||
// string is returned, no matter what. Metric implementations included in this
|
||||
// library use this function internally to generate the fully-qualified metric
|
||||
// name from the name component in their Opts. Users of the library will only
|
||||
// need this function if they implement their own Metric or instantiate a Desc
|
||||
// (with NewDesc) directly.
|
||||
func BuildFQName(namespace, subsystem, name string) string {
|
||||
if name == "" {
|
||||
return ""
|
||||
}
|
||||
|
||||
sb := strings.Builder{}
|
||||
sb.Grow(len(namespace) + len(subsystem) + len(name) + 2)
|
||||
|
||||
if namespace != "" {
|
||||
sb.WriteString(namespace)
|
||||
sb.WriteString("_")
|
||||
}
|
||||
|
||||
if subsystem != "" {
|
||||
sb.WriteString(subsystem)
|
||||
sb.WriteString("_")
|
||||
}
|
||||
|
||||
sb.WriteString(name)
|
||||
|
||||
return sb.String()
|
||||
}
|
||||
|
||||
type invalidMetric struct {
|
||||
desc *Desc
|
||||
err error
|
||||
}
|
||||
|
||||
// NewInvalidMetric returns a metric whose Write method always returns the
|
||||
// provided error. It is useful if a Collector finds itself unable to collect
|
||||
// a metric and wishes to report an error to the registry.
|
||||
func NewInvalidMetric(desc *Desc, err error) Metric {
|
||||
return &invalidMetric{desc, err}
|
||||
}
|
||||
|
||||
func (m *invalidMetric) Desc() *Desc { return m.desc }
|
||||
|
||||
func (m *invalidMetric) Write(*dto.Metric) error { return m.err }
|
||||
|
||||
type timestampedMetric struct {
|
||||
Metric
|
||||
t time.Time
|
||||
}
|
||||
|
||||
func (m timestampedMetric) Write(pb *dto.Metric) error {
|
||||
e := m.Metric.Write(pb)
|
||||
pb.TimestampMs = proto.Int64(m.t.Unix()*1000 + int64(m.t.Nanosecond()/1000000))
|
||||
return e
|
||||
}
|
||||
|
||||
// NewMetricWithTimestamp returns a new Metric wrapping the provided Metric in a
|
||||
// way that it has an explicit timestamp set to the provided Time. This is only
|
||||
// useful in rare cases as the timestamp of a Prometheus metric should usually
|
||||
// be set by the Prometheus server during scraping. Exceptions include mirroring
|
||||
// metrics with given timestamps from other metric
|
||||
// sources.
|
||||
//
|
||||
// NewMetricWithTimestamp works best with MustNewConstMetric,
|
||||
// MustNewConstHistogram, and MustNewConstSummary, see example.
|
||||
//
|
||||
// Currently, the exposition formats used by Prometheus are limited to
|
||||
// millisecond resolution. Thus, the provided time will be rounded down to the
|
||||
// next full millisecond value.
|
||||
func NewMetricWithTimestamp(t time.Time, m Metric) Metric {
|
||||
return timestampedMetric{Metric: m, t: t}
|
||||
}
|
||||
|
||||
type withExemplarsMetric struct {
|
||||
Metric
|
||||
|
||||
exemplars []*dto.Exemplar
|
||||
}
|
||||
|
||||
func (m *withExemplarsMetric) Write(pb *dto.Metric) error {
|
||||
if err := m.Metric.Write(pb); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
switch {
|
||||
case pb.Counter != nil:
|
||||
pb.Counter.Exemplar = m.exemplars[len(m.exemplars)-1]
|
||||
case pb.Histogram != nil:
|
||||
h := pb.Histogram
|
||||
for _, e := range m.exemplars {
|
||||
if (h.GetZeroThreshold() != 0 || h.GetZeroCount() != 0 ||
|
||||
len(h.PositiveSpan) != 0 || len(h.NegativeSpan) != 0) &&
|
||||
e.GetTimestamp() != nil {
|
||||
h.Exemplars = append(h.Exemplars, e)
|
||||
if len(h.Bucket) == 0 {
|
||||
// Don't proceed to classic buckets if there are none.
|
||||
continue
|
||||
}
|
||||
}
|
||||
// h.Bucket are sorted by UpperBound.
|
||||
i := sort.Search(len(h.Bucket), func(i int) bool {
|
||||
return h.Bucket[i].GetUpperBound() >= e.GetValue()
|
||||
})
|
||||
if i < len(h.Bucket) {
|
||||
h.Bucket[i].Exemplar = e
|
||||
} else {
|
||||
// The +Inf bucket should be explicitly added if there is an exemplar for it, similar to non-const histogram logic in https://github.com/prometheus/client_golang/blob/main/prometheus/histogram.go#L357-L365.
|
||||
b := &dto.Bucket{
|
||||
CumulativeCount: proto.Uint64(h.GetSampleCount()),
|
||||
UpperBound: proto.Float64(math.Inf(1)),
|
||||
Exemplar: e,
|
||||
}
|
||||
h.Bucket = append(h.Bucket, b)
|
||||
}
|
||||
}
|
||||
default:
|
||||
// TODO(bwplotka): Implement Gauge?
|
||||
return errors.New("cannot inject exemplar into Gauge, Summary or Untyped")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// Exemplar is easier to use, user-facing representation of *dto.Exemplar.
|
||||
type Exemplar struct {
|
||||
Value float64
|
||||
Labels Labels
|
||||
// Optional.
|
||||
// Default value (time.Time{}) indicates its empty, which should be
|
||||
// understood as time.Now() time at the moment of creation of metric.
|
||||
Timestamp time.Time
|
||||
}
|
||||
|
||||
// NewMetricWithExemplars returns a new Metric wrapping the provided Metric with given
|
||||
// exemplars. Exemplars are validated.
|
||||
//
|
||||
// Only last applicable exemplar is injected from the list.
|
||||
// For example for Counter it means last exemplar is injected.
|
||||
// For Histogram, it means last applicable exemplar for each bucket is injected.
|
||||
// For a Native Histogram, all valid exemplars are injected.
|
||||
//
|
||||
// NewMetricWithExemplars works best with MustNewConstMetric and
|
||||
// MustNewConstHistogram, see example.
|
||||
func NewMetricWithExemplars(m Metric, exemplars ...Exemplar) (Metric, error) {
|
||||
if len(exemplars) == 0 {
|
||||
return nil, errors.New("no exemplar was passed for NewMetricWithExemplars")
|
||||
}
|
||||
|
||||
var (
|
||||
now = time.Now()
|
||||
exs = make([]*dto.Exemplar, len(exemplars))
|
||||
err error
|
||||
)
|
||||
for i, e := range exemplars {
|
||||
ts := e.Timestamp
|
||||
if ts.IsZero() {
|
||||
ts = now
|
||||
}
|
||||
exs[i], err = newExemplar(e.Value, ts, e.Labels)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
return &withExemplarsMetric{Metric: m, exemplars: exs}, nil
|
||||
}
|
||||
|
||||
// MustNewMetricWithExemplars is a version of NewMetricWithExemplars that panics where
|
||||
// NewMetricWithExemplars would have returned an error.
|
||||
func MustNewMetricWithExemplars(m Metric, exemplars ...Exemplar) Metric {
|
||||
ret, err := NewMetricWithExemplars(m, exemplars...)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return ret
|
||||
}
|
||||
25
vendor/github.com/prometheus/client_golang/prometheus/num_threads.go
generated
vendored
Normal file
25
vendor/github.com/prometheus/client_golang/prometheus/num_threads.go
generated
vendored
Normal file
@ -0,0 +1,25 @@
|
||||
// Copyright 2018 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
//go:build !js || wasm
|
||||
// +build !js wasm
|
||||
|
||||
package prometheus
|
||||
|
||||
import "runtime"
|
||||
|
||||
// getRuntimeNumThreads returns the number of open OS threads.
|
||||
func getRuntimeNumThreads() float64 {
|
||||
n, _ := runtime.ThreadCreateProfile(nil)
|
||||
return float64(n)
|
||||
}
|
||||
22
vendor/github.com/prometheus/client_golang/prometheus/num_threads_gopherjs.go
generated
vendored
Normal file
22
vendor/github.com/prometheus/client_golang/prometheus/num_threads_gopherjs.go
generated
vendored
Normal file
@ -0,0 +1,22 @@
|
||||
// Copyright 2018 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
//go:build js && !wasm
|
||||
// +build js,!wasm
|
||||
|
||||
package prometheus
|
||||
|
||||
// getRuntimeNumThreads returns the number of open OS threads.
|
||||
func getRuntimeNumThreads() float64 {
|
||||
return 1
|
||||
}
|
||||
64
vendor/github.com/prometheus/client_golang/prometheus/observer.go
generated
vendored
Normal file
64
vendor/github.com/prometheus/client_golang/prometheus/observer.go
generated
vendored
Normal file
@ -0,0 +1,64 @@
|
||||
// Copyright 2017 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package prometheus
|
||||
|
||||
// Observer is the interface that wraps the Observe method, which is used by
|
||||
// Histogram and Summary to add observations.
|
||||
type Observer interface {
|
||||
Observe(float64)
|
||||
}
|
||||
|
||||
// The ObserverFunc type is an adapter to allow the use of ordinary
|
||||
// functions as Observers. If f is a function with the appropriate
|
||||
// signature, ObserverFunc(f) is an Observer that calls f.
|
||||
//
|
||||
// This adapter is usually used in connection with the Timer type, and there are
|
||||
// two general use cases:
|
||||
//
|
||||
// The most common one is to use a Gauge as the Observer for a Timer.
|
||||
// See the "Gauge" Timer example.
|
||||
//
|
||||
// The more advanced use case is to create a function that dynamically decides
|
||||
// which Observer to use for observing the duration. See the "Complex" Timer
|
||||
// example.
|
||||
type ObserverFunc func(float64)
|
||||
|
||||
// Observe calls f(value). It implements Observer.
|
||||
func (f ObserverFunc) Observe(value float64) {
|
||||
f(value)
|
||||
}
|
||||
|
||||
// ObserverVec is an interface implemented by `HistogramVec` and `SummaryVec`.
|
||||
type ObserverVec interface {
|
||||
GetMetricWith(Labels) (Observer, error)
|
||||
GetMetricWithLabelValues(lvs ...string) (Observer, error)
|
||||
With(Labels) Observer
|
||||
WithLabelValues(...string) Observer
|
||||
CurryWith(Labels) (ObserverVec, error)
|
||||
MustCurryWith(Labels) ObserverVec
|
||||
|
||||
Collector
|
||||
}
|
||||
|
||||
// ExemplarObserver is implemented by Observers that offer the option of
|
||||
// observing a value together with an exemplar. Its ObserveWithExemplar method
|
||||
// works like the Observe method of an Observer but also replaces the currently
|
||||
// saved exemplar (if any) with a new one, created from the provided value, the
|
||||
// current time as timestamp, and the provided Labels. Empty Labels will lead to
|
||||
// a valid (label-less) exemplar. But if Labels is nil, the current exemplar is
|
||||
// left in place. ObserveWithExemplar panics if any of the provided labels are
|
||||
// invalid or if the provided labels contain more than 128 runes in total.
|
||||
type ExemplarObserver interface {
|
||||
ObserveWithExemplar(value float64, exemplar Labels)
|
||||
}
|
||||
180
vendor/github.com/prometheus/client_golang/prometheus/process_collector.go
generated
vendored
Normal file
180
vendor/github.com/prometheus/client_golang/prometheus/process_collector.go
generated
vendored
Normal file
@ -0,0 +1,180 @@
|
||||
// Copyright 2015 The Prometheus Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package prometheus
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type processCollector struct {
|
||||
collectFn func(chan<- Metric)
|
||||
describeFn func(chan<- *Desc)
|
||||
pidFn func() (int, error)
|
||||
reportErrors bool
|
||||
cpuTotal *Desc
|
||||
openFDs, maxFDs *Desc
|
||||
vsize, maxVsize *Desc
|
||||
rss *Desc
|
||||
startTime *Desc
|
||||
inBytes, outBytes *Desc
|
||||
}
|
||||
|
||||
// ProcessCollectorOpts defines the behavior of a process metrics collector
|
||||
// created with NewProcessCollector.
|
||||
type ProcessCollectorOpts struct {
|
||||
// PidFn returns the PID of the process the collector collects metrics
|
||||
// for. It is called upon each collection. By default, the PID of the
|
||||
// current process is used, as determined on construction time by
|
||||
// calling os.Getpid().
|
||||
PidFn func() (int, error)
|
||||
// If non-empty, each of the collected metrics is prefixed by the
|
||||
// provided string and an underscore ("_").
|
||||
Namespace string
|
||||
// If true, any error encountered during collection is reported as an
|
||||
// invalid metric (see NewInvalidMetric). Otherwise, errors are ignored
|
||||
// and the collected metrics will be incomplete. (Possibly, no metrics
|
||||
// will be collected at all.) While that's usually not desired, it is
|
||||
// appropriate for the common "mix-in" of process metrics, where process
|
||||
// metrics are nice to have, but failing to collect them should not
|
||||
// disrupt the collection of the remaining metrics.
|
||||
ReportErrors bool
|
||||
}
|
||||
|
||||
// NewProcessCollector is the obsolete version of collectors.NewProcessCollector.
|
||||
// See there for documentation.
|
||||
//
|
||||
// Deprecated: Use collectors.NewProcessCollector instead.
|
||||
func NewProcessCollector(opts ProcessCollectorOpts) Collector {
|
||||
ns := ""
|
||||
if len(opts.Namespace) > 0 {
|
||||
ns = opts.Namespace + "_"
|
||||
}
|
||||
|
||||
c := &processCollector{
|
||||
reportErrors: opts.ReportErrors,
|
||||
cpuTotal: NewDesc(
|
||||
ns+"process_cpu_seconds_total",
|
||||
"Total user and system CPU time spent in seconds.",
|
||||
nil, nil,
|
||||
),
|
||||
openFDs: NewDesc(
|
||||
ns+"process_open_fds",
|
||||
"Number of open file descriptors.",
|
||||
nil, nil,
|
||||
),
|
||||
maxFDs: NewDesc(
|
||||
ns+"process_max_fds",
|
||||
"Maximum number of open file descriptors.",
|
||||
nil, nil,
|
||||
),
|
||||
vsize: NewDesc(
|
||||
ns+"process_virtual_memory_bytes",
|
||||
"Virtual memory size in bytes.",
|
||||
nil, nil,
|
||||
),
|
||||
maxVsize: NewDesc(
|
||||
ns+"process_virtual_memory_max_bytes",
|
||||
"Maximum amount of virtual memory available in bytes.",
|
||||
nil, nil,
|
||||
),
|
||||
rss: NewDesc(
|
||||
ns+"process_resident_memory_bytes",
|
||||
"Resident memory size in bytes.",
|
||||
nil, nil,
|
||||
),
|
||||
startTime: NewDesc(
|
||||
ns+"process_start_time_seconds",
|
||||
"Start time of the process since unix epoch in seconds.",
|
||||
nil, nil,
|
||||
),
|
||||
inBytes: NewDesc(
|
||||
ns+"process_network_receive_bytes_total",
|
||||
"Number of bytes received by the process over the network.",
|
||||
nil, nil,
|
||||
),
|
||||
outBytes: NewDesc(
|
||||
ns+"process_network_transmit_bytes_total",
|
||||
"Number of bytes sent by the process over the network.",
|
||||
nil, nil,
|
||||
),
|
||||
}
|
||||
|
||||
if opts.PidFn == nil {
|
||||
c.pidFn = getPIDFn()
|
||||
} else {
|
||||
c.pidFn = opts.PidFn
|
||||
}
|
||||
|
||||
// Set up process metric collection if supported by the runtime.
|
||||
if canCollectProcess() {
|
||||
c.collectFn = c.processCollect
|
||||
c.describeFn = c.describe
|
||||
} else {
|
||||
c.collectFn = c.errorCollectFn
|
||||
c.describeFn = c.errorDescribeFn
|
||||
}
|
||||
|
||||
return c
|
||||
}
|
||||
|
||||
func (c *processCollector) errorCollectFn(ch chan<- Metric) {
|
||||
c.reportError(ch, nil, errors.New("process metrics not supported on this platform"))
|
||||
}
|
||||
|
||||
func (c *processCollector) errorDescribeFn(ch chan<- *Desc) {
|
||||
if c.reportErrors {
|
||||
ch <- NewInvalidDesc(errors.New("process metrics not supported on this platform"))
|
||||
}
|
||||
}
|
||||
|
||||
// Collect returns the current state of all metrics of the collector.
|
||||
func (c *processCollector) Collect(ch chan<- Metric) {
|
||||
c.collectFn(ch)
|
||||
}
|
||||
|
||||
// Describe returns all descriptions of the collector.
|
||||
func (c *processCollector) Describe(ch chan<- *Desc) {
|
||||
c.describeFn(ch)
|
||||
}
|
||||
|
||||
func (c *processCollector) reportError(ch chan<- Metric, desc *Desc, err error) {
|
||||
if !c.reportErrors {
|
||||
return
|
||||
}
|
||||
if desc == nil {
|
||||
desc = NewInvalidDesc(err)
|
||||
}
|
||||
ch <- NewInvalidMetric(desc, err)
|
||||
}
|
||||
|
||||
// NewPidFileFn returns a function that retrieves a pid from the specified file.
|
||||
// It is meant to be used for the PidFn field in ProcessCollectorOpts.
|
||||
func NewPidFileFn(pidFilePath string) func() (int, error) {
|
||||
return func() (int, error) {
|
||||
content, err := os.ReadFile(pidFilePath)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("can't read pid file %q: %w", pidFilePath, err)
|
||||
}
|
||||
pid, err := strconv.Atoi(strings.TrimSpace(string(content)))
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("can't parse pid file %q: %w", pidFilePath, err)
|
||||
}
|
||||
|
||||
return pid, nil
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user