hush: one-time secret links the server cannot read
Paste a secret, get a link, send it. The first person to open it and press
Reveal sees the secret; the link dies at that moment. The recipient needs a
browser and nothing else — no account, no client, no installed tooling.
The server cannot read what it stores. AES-256-GCM happens in the browser and
the key lives in the URL fragment, which browsers never transmit, so hushd
holds ciphertext and no key material. That is a property of where the key sits
rather than a promise about our conduct, which is why there is deliberately no
endpoint accepting a plaintext secret and no server-side-encryption fallback:
two guarantees behind one URL would be worse than one honest guarantee.
Three decisions carry the design:
* GET /s/{id} touches NO storage, not even to check existence. Slack, Teams,
WhatsApp, iMessage and Outlook Safe Links all fetch a URL before a human
sees it, so destroying on GET would destroy most secrets in transit and the
recipient's "already used" would be indistinguishable from interception.
Only POST /reveal consumes. Bot user-agent detection is an arms race;
removing the side effect from GET is not. Pinned by
TestGettingTheRevealPageNeverConsumesTheSecret.
* Destruction is one Redis GETDEL, which is atomic. GET-then-DEL has a window
where two simultaneous readers both win, and for a one-time secret that
window is the product. The store contract demands atomicity and the same
concurrency test runs against both implementations.
* Missing, already-revealed, expired and evicted are ONE indistinguishable
410. Separating them would confirm to a prober that a given link was real.
The secret id IS the capability, so secret.ID is a struct whose every
accidental path — %v, %s, String(), slog, json.Marshal — emits a redacted
handle or refuses, and the raw value needs an explicit Value(). The first
version tried to prevent leaks by implementing no String() at all; its own test
caught that Go's fmt prints unexported fields anyway, so forbidding the method
had removed the control rather than the leak.
Operationally: structured JSON on stdout in the fleet's wire format, which
Vector already collects with no annotation; six hush_* metrics on the chassis
registry with no id, IP or path in any label; five alert rules wired into
vmalert. The public Ingress enumerates /, /s/ and /api/ so /metrics, /healthz
and /readyz share the port but are unreachable from the internet — no
basic-auth middleware to maintain and get wrong.
Dependencies are vendored because go-chassis is private: the Woodpecker test
step and the in-cluster Kaniko build both run -mod=vendor with GOPROXY=off and
hold no git credential.
cmd/hush-mcp is a stdio MCP server doing the same client-side crypto locally,
so using hush from an agent preserves the same guarantee as using it from a
browser.
This commit is contained in:
commit
4d9a26498e
2
.gitignore
vendored
Normal file
2
.gitignore
vendored
Normal file
@ -0,0 +1,2 @@
|
|||||||
|
.build/
|
||||||
|
*.tmp
|
||||||
62
.woodpecker.yml
Normal file
62
.woodpecker.yml
Normal file
@ -0,0 +1,62 @@
|
|||||||
|
# hush CI/CD. Origin must be Gitea (git.threesix.ai) — that remote carries the
|
||||||
|
# webhook. Pushing to the GitHub mirror does not deploy anything.
|
||||||
|
#
|
||||||
|
# push to main → test → Kaniko amd64 build → Zot → kubectl set image on `projects`
|
||||||
|
|
||||||
|
clone:
|
||||||
|
git:
|
||||||
|
image: woodpeckerci/plugin-git
|
||||||
|
settings:
|
||||||
|
depth: 1
|
||||||
|
|
||||||
|
steps:
|
||||||
|
test:
|
||||||
|
# 1.26, not the 1.25 in the template: go-chassis declares `go 1.26.0` and an
|
||||||
|
# older toolchain refuses to build it.
|
||||||
|
image: golang:1.26-alpine
|
||||||
|
environment:
|
||||||
|
# Hermetic, and deliberately so. go-chassis is a PRIVATE module and this
|
||||||
|
# container has no git credential, so a build that reached for the network
|
||||||
|
# would fail here — the vendored tree is what makes CI possible at all.
|
||||||
|
# GOPROXY=off turns "silently fetched from a proxy" into a hard error.
|
||||||
|
GOFLAGS: -mod=vendor
|
||||||
|
GOPROXY: "off"
|
||||||
|
commands:
|
||||||
|
- go vet ./...
|
||||||
|
- go test ./...
|
||||||
|
# gofmt as a gate, not a suggestion. -l lists offenders; the test fails if
|
||||||
|
# the list is non-empty. vendor/ is excluded because it is not ours.
|
||||||
|
- test -z "$(gofmt -l ./cmd ./internal)" || { gofmt -l ./cmd ./internal; echo "gofmt"; exit 1; }
|
||||||
|
when:
|
||||||
|
branch: main
|
||||||
|
event: push
|
||||||
|
|
||||||
|
build:
|
||||||
|
image: woodpeckerci/plugin-kaniko
|
||||||
|
settings:
|
||||||
|
registry: registry.threesix.ai
|
||||||
|
repo: hush/api
|
||||||
|
tags:
|
||||||
|
- latest
|
||||||
|
# SHA-tagged as well as latest: `latest` cannot pin a rollback.
|
||||||
|
- ${CI_COMMIT_SHA:0:8}
|
||||||
|
context: .
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
cache: true
|
||||||
|
skip_tls_verify: true # internal Zot registry, self-signed cert
|
||||||
|
when:
|
||||||
|
branch: main
|
||||||
|
event: push
|
||||||
|
|
||||||
|
deploy:
|
||||||
|
image: bitnami/kubectl:latest
|
||||||
|
commands:
|
||||||
|
- kubectl set image deployment/hush hushd=registry.threesix.ai/hush/api:${CI_COMMIT_SHA:0:8} -n projects
|
||||||
|
- kubectl rollout status deployment/hush -n projects --timeout=120s
|
||||||
|
# Prove the rolled pod actually serves, not merely that it became Ready.
|
||||||
|
# A pod can pass readiness and still be the wrong image if `set image`
|
||||||
|
# silently matched nothing.
|
||||||
|
- kubectl -n projects get deployment hush -o jsonpath='{.spec.template.spec.containers[0].image}{"\n"}'
|
||||||
|
when:
|
||||||
|
branch: main
|
||||||
|
event: push
|
||||||
44
Dockerfile
Normal file
44
Dockerfile
Normal file
@ -0,0 +1,44 @@
|
|||||||
|
# hushd. Built amd64 IN-CLUSTER by Kaniko — never `docker build` locally, which
|
||||||
|
# on an Apple Silicon laptop produces an arm64 image the cluster cannot run.
|
||||||
|
#
|
||||||
|
# The build is HERMETIC: `-mod=vendor` with `GOPROXY=off` and `GOFLAGS=-mod=vendor`
|
||||||
|
# means no module download and no network. That is not an optimisation — it is
|
||||||
|
# required, because github.com/orchard9/go-chassis is a private module and the
|
||||||
|
# build container holds no git credential. `make vendor` is what moves versions.
|
||||||
|
FROM golang:1.26-alpine AS build
|
||||||
|
WORKDIR /src
|
||||||
|
|
||||||
|
# Vendored, so this is the whole dependency graph: no go.sum verification step,
|
||||||
|
# no proxy, no cache warm-up.
|
||||||
|
COPY go.mod go.sum ./
|
||||||
|
COPY vendor/ ./vendor/
|
||||||
|
COPY cmd/ ./cmd/
|
||||||
|
COPY internal/ ./internal/
|
||||||
|
|
||||||
|
# Fail loudly if anything reaches for the network, rather than silently falling
|
||||||
|
# back to a proxy that will not be there in CI.
|
||||||
|
ENV GOFLAGS=-mod=vendor GOPROXY=off CGO_ENABLED=0
|
||||||
|
|
||||||
|
# Tests run in the CI step, not here: a Kaniko layer that runs tests caches
|
||||||
|
# their result and stops re-running them. Keep the image build to building.
|
||||||
|
RUN go build -trimpath -ldflags="-s -w" -o /out/hushd ./cmd/hushd
|
||||||
|
|
||||||
|
# distroless static + nonroot: no shell, no package manager, no libc surface.
|
||||||
|
# hushd needs only TCP and the CA bundle distroless already carries.
|
||||||
|
FROM gcr.io/distroless/static-debian12:nonroot AS runtime
|
||||||
|
WORKDIR /
|
||||||
|
|
||||||
|
COPY --from=build /out/hushd /hushd
|
||||||
|
|
||||||
|
# Numeric, not "nonroot": with a non-numeric USER the kubelet cannot verify
|
||||||
|
# runAsNonRoot and refuses to start the pod.
|
||||||
|
USER 65532:65532
|
||||||
|
|
||||||
|
# HTML templates are embedded in the binary (internal/web, go:embed), so there
|
||||||
|
# is no asset directory to mount, drift, or go missing at runtime.
|
||||||
|
EXPOSE 18500
|
||||||
|
|
||||||
|
# Exec form so SIGTERM reaches the process directly. The chassis's two-phase
|
||||||
|
# drain (readiness 503, wait, then shutdown) depends on receiving it, and the
|
||||||
|
# Deployment's terminationGracePeriodSeconds is sized to outlast that drain.
|
||||||
|
ENTRYPOINT ["/hushd"]
|
||||||
91
Makefile
Normal file
91
Makefile
Normal file
@ -0,0 +1,91 @@
|
|||||||
|
.DEFAULT_GOAL := help
|
||||||
|
.PHONY: help fmt vet test test-redis build run dev dev-stop smoke vendor verify ci \
|
||||||
|
mcp mcp-install deploy-manifests deploy-status logs alerts-check
|
||||||
|
|
||||||
|
# Local development Redis. A real server, not a mock: the one-time guarantee
|
||||||
|
# rests on GETDEL being atomic, and a fake cannot prove that.
|
||||||
|
DEV_REDIS_PORT ?= 6389
|
||||||
|
DEV_REDIS_URL ?= redis://127.0.0.1:$(DEV_REDIS_PORT)/5
|
||||||
|
HUSH_PORT ?= 18500
|
||||||
|
BASE ?= http://127.0.0.1:$(HUSH_PORT)
|
||||||
|
|
||||||
|
# The public deployment. `hushd` runs one replica in the `projects` namespace.
|
||||||
|
KUBECONFIG_FILE ?= $(HOME)/.kube/orchard9-k3sf.yaml
|
||||||
|
NS ?= projects
|
||||||
|
HOST ?= hush.threesix.ai
|
||||||
|
|
||||||
|
help: ## Show this help
|
||||||
|
@grep -hE '^[a-zA-Z_-]+:.*?## ' $(MAKEFILE_LIST) \
|
||||||
|
| awk 'BEGIN{FS=":.*?## "}{printf " \033[36m%-18s\033[0m %s\n", $$1, $$2}'
|
||||||
|
|
||||||
|
fmt: ## Format everything outside vendor/
|
||||||
|
@gofmt -w ./cmd ./internal
|
||||||
|
|
||||||
|
vet: ## go vet
|
||||||
|
@go vet ./...
|
||||||
|
|
||||||
|
test: ## Unit tests. No Redis, no network, no container.
|
||||||
|
@go test ./...
|
||||||
|
|
||||||
|
test-redis: ## Run the store contract suite against a REAL Redis as well
|
||||||
|
@$(MAKE) --no-print-directory dev-redis
|
||||||
|
@HUSH_TEST_REDIS_URL=$(DEV_REDIS_URL) go test ./internal/store/... -count=1
|
||||||
|
|
||||||
|
build: ## Build both binaries into .build/
|
||||||
|
@mkdir -p .build
|
||||||
|
@go build -o .build/hushd ./cmd/hushd
|
||||||
|
@go build -o .build/hush-mcp ./cmd/hush-mcp
|
||||||
|
|
||||||
|
# `vendor` is not a convenience: github.com/orchard9/go-chassis is a PRIVATE
|
||||||
|
# module, so neither the Woodpecker test step nor the in-cluster Kaniko build
|
||||||
|
# can fetch it. Vendoring makes both hermetic — they build with -mod=vendor and
|
||||||
|
# no network and no credential. This target is the only way dep versions move.
|
||||||
|
vendor: ## Refresh vendor/ after a dependency change
|
||||||
|
@GOPRIVATE=github.com/orchard9 go mod tidy
|
||||||
|
@GOPRIVATE=github.com/orchard9 go mod vendor
|
||||||
|
@go build -mod=vendor ./... && echo "vendor/ builds hermetically"
|
||||||
|
|
||||||
|
verify: ## Prove the vendored tree builds with no network, exactly as CI does
|
||||||
|
@GOFLAGS=-mod=vendor GOPROXY=off go build ./... \
|
||||||
|
&& GOFLAGS=-mod=vendor GOPROXY=off go test ./... >/dev/null \
|
||||||
|
&& echo "hermetic build OK (GOPROXY=off, -mod=vendor)"
|
||||||
|
|
||||||
|
ci: fmt vet test verify ## Everything the pipeline runs
|
||||||
|
|
||||||
|
dev-redis:
|
||||||
|
@redis-cli -p $(DEV_REDIS_PORT) ping >/dev/null 2>&1 || { \
|
||||||
|
echo "starting redis on :$(DEV_REDIS_PORT)"; \
|
||||||
|
redis-server --port $(DEV_REDIS_PORT) --daemonize yes --save '' --appendonly no; \
|
||||||
|
for i in $$(seq 1 30); do redis-cli -p $(DEV_REDIS_PORT) ping >/dev/null 2>&1 && break; sleep 0.2; done; }
|
||||||
|
@redis-cli -p $(DEV_REDIS_PORT) ping | sed 's/^/redis: /'
|
||||||
|
|
||||||
|
dev: dev-redis build ## Run hushd locally against a local Redis
|
||||||
|
@echo "hushd on $(BASE) — open it in a browser"
|
||||||
|
@APP_ENV=dev REDIS_URL=$(DEV_REDIS_URL) HUSH_PORT=$(HUSH_PORT) .build/hushd
|
||||||
|
|
||||||
|
dev-stop: ## Stop the local Redis
|
||||||
|
@redis-cli -p $(DEV_REDIS_PORT) shutdown nosave 2>/dev/null || true
|
||||||
|
@echo "stopped"
|
||||||
|
|
||||||
|
smoke: ## Full create -> reveal -> gone against a locally running hushd
|
||||||
|
@BASE=$(BASE) ./scripts/smoke.sh
|
||||||
|
|
||||||
|
mcp: ## Build and install the MCP server, then register it with omp
|
||||||
|
@$(MAKE) --no-print-directory mcp-install
|
||||||
|
|
||||||
|
mcp-install:
|
||||||
|
@./scripts/install-mcp.sh
|
||||||
|
|
||||||
|
deploy-manifests: ## Apply the k8s manifests (do this BEFORE the first push)
|
||||||
|
@KUBECONFIG=$(KUBECONFIG_FILE) kubectl apply -f deployments/k8s/hush.yaml
|
||||||
|
|
||||||
|
deploy-status: ## Rollout, pods, ingress and certificate
|
||||||
|
@KUBECONFIG=$(KUBECONFIG_FILE) kubectl -n $(NS) rollout status deployment/hush --timeout=90s
|
||||||
|
@KUBECONFIG=$(KUBECONFIG_FILE) kubectl -n $(NS) get pod,svc,ingress -l app=hush
|
||||||
|
@KUBECONFIG=$(KUBECONFIG_FILE) kubectl -n $(NS) get certificate hush-tls 2>/dev/null || true
|
||||||
|
|
||||||
|
logs: ## Tail hush's structured logs out of VictoriaLogs
|
||||||
|
@./scripts/logs.sh
|
||||||
|
|
||||||
|
alerts-check: ## Confirm vmalert has loaded hush's rules
|
||||||
|
@./scripts/alerts-check.sh
|
||||||
118
README.md
Normal file
118
README.md
Normal file
@ -0,0 +1,118 @@
|
|||||||
|
# hush
|
||||||
|
|
||||||
|
Send someone a secret over a link that works once.
|
||||||
|
|
||||||
|
**Production:** <https://hush.threesix.ai>
|
||||||
|
|
||||||
|
Paste a secret, get a link, send the link. The first person to open it and press
|
||||||
|
**Reveal** sees the secret; the link is dead from that moment. Nobody needs an
|
||||||
|
account, a client, or anything installed — a browser is the whole requirement.
|
||||||
|
|
||||||
|
The server cannot read what you sent. Encryption happens in your browser and the
|
||||||
|
key lives in the URL *fragment* (`…/s/ID#KEY`), which browsers never transmit.
|
||||||
|
hush stores ciphertext it has no way to open. That is not a promise about our
|
||||||
|
operational discipline; it is a property of where the key sits.
|
||||||
|
|
||||||
|
## What one-time actually buys you
|
||||||
|
|
||||||
|
Worth being precise, because "one-time link" is often oversold:
|
||||||
|
|
||||||
|
- **Bounded exposure.** The secret is fetchable once, for at most its TTL, then it
|
||||||
|
is gone. A credential sitting in a Slack thread is fetchable forever by anyone
|
||||||
|
who later gains access to that thread.
|
||||||
|
- **Tamper evidence.** If your recipient says "already used", someone else opened
|
||||||
|
it. You have learned something a plain paste never tells you.
|
||||||
|
- **Nothing at rest to steal.** A dump of hush's Redis yields ciphertext and no keys.
|
||||||
|
|
||||||
|
And what it does not buy you:
|
||||||
|
|
||||||
|
- **It does not protect the link.** Whatever channel carries the link could be read
|
||||||
|
by whoever can read that channel. One-time-ness limits the damage and makes it
|
||||||
|
detectable; it does not make the channel private.
|
||||||
|
- **It does not authenticate the reader.** Anyone holding the link can open it.
|
||||||
|
The link *is* the capability. Treat it like the secret it carries.
|
||||||
|
|
||||||
|
If a secret must reach one specific verified human and nobody else, this is the
|
||||||
|
wrong tool — use a channel with identity.
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
### In a browser
|
||||||
|
|
||||||
|
1. Open <https://hush.threesix.ai>.
|
||||||
|
2. Paste the secret, pick a lifetime, press **Create link**.
|
||||||
|
3. Copy the link and send it however you like.
|
||||||
|
4. The recipient opens it, presses **Reveal**, and reads it once.
|
||||||
|
|
||||||
|
### Why there is a button
|
||||||
|
|
||||||
|
Slack, Teams, WhatsApp, iMessage and Outlook Safe Links all fetch a URL to build
|
||||||
|
a preview *before* any human sees it. A service that destroys on `GET` therefore
|
||||||
|
destroys most secrets in transit, and the recipient's "already used" is
|
||||||
|
indistinguishable from a real interception.
|
||||||
|
|
||||||
|
So in hush, `GET /s/{id}` is a static page that touches no storage at all. Only
|
||||||
|
`POST /s/{id}/reveal` reads and destroys. Link previewers are harmless by
|
||||||
|
construction, not by user-agent guessing.
|
||||||
|
|
||||||
|
### API
|
||||||
|
|
||||||
|
The API takes **ciphertext**. There is no endpoint that accepts a plaintext
|
||||||
|
secret, because such an endpoint would make the server able to read secrets and
|
||||||
|
the claim at the top of this file would become a matter of trust rather than
|
||||||
|
arithmetic.
|
||||||
|
|
||||||
|
```
|
||||||
|
POST /api/secrets
|
||||||
|
{ "ciphertext": "<base64url AES-256-GCM, nonce prepended>", "ttl_seconds": 86400 }
|
||||||
|
→ 201 { "id": "…", "expires_at": "2026-09-04T…Z", "ttl_seconds": 86400 }
|
||||||
|
|
||||||
|
POST /api/secrets/{id}/reveal
|
||||||
|
→ 200 { "ciphertext": "…" } first caller only, secret destroyed
|
||||||
|
→ 410 { "error": { "code": "gone" } } every other case
|
||||||
|
```
|
||||||
|
|
||||||
|
`410 gone` is returned identically whether the id never existed, was already
|
||||||
|
revealed, or expired. Distinguishing those would confirm to an attacker that a
|
||||||
|
particular link once existed.
|
||||||
|
|
||||||
|
`GET /` serves the create page, `GET /s/{id}` the reveal page. `/healthz`,
|
||||||
|
`/readyz` and `/metrics` are served on the same port but are **not routed by the
|
||||||
|
public ingress** — they are reachable in-cluster only.
|
||||||
|
|
||||||
|
### From an agent, over MCP
|
||||||
|
|
||||||
|
`cmd/hush-mcp` is a stdio MCP server exposing two tools, `hush_create` and
|
||||||
|
`hush_reveal`. It runs **locally** and does the encryption on your machine, so
|
||||||
|
using hush from an agent preserves the same zero-knowledge property as using it
|
||||||
|
from a browser. See [docs/MCP.md](docs/MCP.md).
|
||||||
|
|
||||||
|
## Limits
|
||||||
|
|
||||||
|
| Thing | Value | Why |
|
||||||
|
|---|---|---|
|
||||||
|
| Ciphertext | ≤ 64 KiB | It is a courier for credentials, not a file host |
|
||||||
|
| TTL | 5m … 7d, default 24h | Long enough to be useful, short enough to bound exposure |
|
||||||
|
| Rate limit | 30 creates / 10 min / IP | Anonymous create is otherwise a free blob host |
|
||||||
|
| Reveals per secret | exactly 1 | The product |
|
||||||
|
|
||||||
|
## Operating it
|
||||||
|
|
||||||
|
- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — how it works and why each choice
|
||||||
|
- [docs/DEPLOY.md](docs/DEPLOY.md) — pipeline, DNS, credentials, first deploy
|
||||||
|
- [docs/OPERATIONS.md](docs/OPERATIONS.md) — alert runbook, log queries, failure modes
|
||||||
|
- [docs/MCP.md](docs/MCP.md) — the MCP server and how to install it
|
||||||
|
|
||||||
|
## Development
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make help # every target
|
||||||
|
make test # unit tests, no external dependencies
|
||||||
|
make dev # a local Redis in Docker + hushd on :18500
|
||||||
|
make smoke # full create → reveal → gone against the local instance
|
||||||
|
make vendor # refresh vendor/ after a dependency change
|
||||||
|
```
|
||||||
|
|
||||||
|
`go-chassis` is a private module, so dependencies are **vendored** and both CI
|
||||||
|
and the container build run with `-mod=vendor` and no network. `make vendor` is
|
||||||
|
the only way dependency versions change.
|
||||||
141
cmd/hush-mcp/crypto_test.go
Normal file
141
cmd/hush-mcp/crypto_test.go
Normal file
@ -0,0 +1,141 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/base64"
|
||||||
|
"os/exec"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestSealOpenRoundTrip(t *testing.T) {
|
||||||
|
for _, plain := range []string{
|
||||||
|
"hunter2",
|
||||||
|
"",
|
||||||
|
strings.Repeat("x", 40000),
|
||||||
|
"unicode: ✓ 漢字 🔐",
|
||||||
|
"multi\nline\nwith\ttabs",
|
||||||
|
} {
|
||||||
|
ct, key, err := seal(plain)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("seal(%d bytes): %v", len(plain), err)
|
||||||
|
}
|
||||||
|
got, err := open(ct, key)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("open: %v", err)
|
||||||
|
}
|
||||||
|
if got != plain {
|
||||||
|
t.Fatalf("round trip changed the plaintext (%d bytes)", len(plain))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSealProducesTheBrowsersWireFormat(t *testing.T) {
|
||||||
|
ct, key, err := seal("x")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// base64url, unpadded, matching base64.RawURLEncoding on the server and
|
||||||
|
// the b64u helper in templates/base.html. Three implementations, one
|
||||||
|
// spelling — a mismatch here means a link minted by one client cannot be
|
||||||
|
// opened by another.
|
||||||
|
for name, v := range map[string]string{"ciphertext": ct, "key": key} {
|
||||||
|
if strings.ContainsAny(v, "+/=") {
|
||||||
|
t.Fatalf("%s %q uses the standard base64 alphabet or padding; it must be base64url unpadded", name, v)
|
||||||
|
}
|
||||||
|
if _, err := base64.RawURLEncoding.DecodeString(v); err != nil {
|
||||||
|
t.Fatalf("%s does not decode as base64url: %v", name, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// 256-bit key.
|
||||||
|
raw, _ := base64.RawURLEncoding.DecodeString(key)
|
||||||
|
if len(raw) != 32 {
|
||||||
|
t.Fatalf("key is %d bytes, want 32 (AES-256)", len(raw))
|
||||||
|
}
|
||||||
|
// 96-bit nonce prepended, then at least the GCM tag.
|
||||||
|
blob, _ := base64.RawURLEncoding.DecodeString(ct)
|
||||||
|
if len(blob) < 12+16 {
|
||||||
|
t.Fatalf("ciphertext is %d bytes, too short for a 12-byte nonce plus a 16-byte tag", len(blob))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestOpenRefusesAWrongKeyAndTamperedCiphertext(t *testing.T) {
|
||||||
|
ct, _, err := seal("secret")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
_, otherKey, _ := seal("unrelated")
|
||||||
|
|
||||||
|
if _, err := open(ct, otherKey); err == nil {
|
||||||
|
t.Fatal("open() accepted a key that does not belong to this ciphertext")
|
||||||
|
}
|
||||||
|
|
||||||
|
// GCM is authenticated: a flipped byte must fail, not decrypt to garbage.
|
||||||
|
blob, _ := base64.RawURLEncoding.DecodeString(ct)
|
||||||
|
blob[len(blob)-1] ^= 0xff
|
||||||
|
_, key, _ := seal("x")
|
||||||
|
if _, err := open(base64.RawURLEncoding.EncodeToString(blob), key); err == nil {
|
||||||
|
t.Fatal("open() accepted tampered ciphertext")
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := open("!!!not base64!!!", key); err == nil {
|
||||||
|
t.Fatal("open() accepted a non-base64url ciphertext")
|
||||||
|
}
|
||||||
|
if _, err := open(ct, "!!!"); err == nil {
|
||||||
|
t.Fatal("open() accepted a non-base64url key")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Cross-implementation check: a blob sealed by this Go client must decrypt with
|
||||||
|
// Python's AES-GCM, and vice versa. This is what proves the MCP client, the
|
||||||
|
// browser and the smoke script really share one format rather than three
|
||||||
|
// self-consistent ones.
|
||||||
|
func TestWireFormatMatchesAnIndependentImplementation(t *testing.T) {
|
||||||
|
if _, err := exec.LookPath("python3"); err != nil {
|
||||||
|
t.Skip("python3 unavailable")
|
||||||
|
}
|
||||||
|
const plain = "cross-implementation-secret"
|
||||||
|
|
||||||
|
ct, key, err := seal(plain)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Go -> Python
|
||||||
|
out, err := exec.Command("python3", "-c", `
|
||||||
|
import base64, sys
|
||||||
|
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
|
||||||
|
u = lambda s: base64.urlsafe_b64decode(s + "=" * (-len(s) % 4))
|
||||||
|
blob, key = u(sys.argv[1]), u(sys.argv[2])
|
||||||
|
sys.stdout.write(AESGCM(key).decrypt(blob[:12], blob[12:], None).decode())
|
||||||
|
`, ct, key).Output()
|
||||||
|
if err != nil {
|
||||||
|
t.Skipf("python cryptography unavailable: %v", err)
|
||||||
|
}
|
||||||
|
if string(out) != plain {
|
||||||
|
t.Fatalf("python decrypted our ciphertext to %q, want %q", out, plain)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Python -> Go
|
||||||
|
pyOut, err := exec.Command("python3", "-c", `
|
||||||
|
import base64, os
|
||||||
|
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
|
||||||
|
key = AESGCM.generate_key(bit_length=256); nonce = os.urandom(12)
|
||||||
|
blob = nonce + AESGCM(key).encrypt(nonce, b"from-python", None)
|
||||||
|
b = lambda x: base64.urlsafe_b64encode(x).decode().rstrip("=")
|
||||||
|
print(b(blob), b(key))
|
||||||
|
`).Output()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("python encrypt failed: %v", err)
|
||||||
|
}
|
||||||
|
parts := strings.Fields(string(pyOut))
|
||||||
|
if len(parts) != 2 {
|
||||||
|
t.Fatalf("unexpected python output %q", pyOut)
|
||||||
|
}
|
||||||
|
got, err := open(parts[0], parts[1])
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("could not open python's ciphertext: %v", err)
|
||||||
|
}
|
||||||
|
if got != "from-python" {
|
||||||
|
t.Fatalf("opened python's ciphertext to %q", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
304
cmd/hush-mcp/main.go
Normal file
304
cmd/hush-mcp/main.go
Normal file
@ -0,0 +1,304 @@
|
|||||||
|
// Command hush-mcp exposes hush to an MCP host (omp, Claude Code, any client)
|
||||||
|
// as two tools: hush_create and hush_reveal.
|
||||||
|
//
|
||||||
|
// It runs LOCALLY, on the operator's machine, and does the AES-256-GCM itself.
|
||||||
|
// That is the whole reason it is a stdio binary rather than an endpoint served
|
||||||
|
// by hushd: if the server did the encrypting, the server could read every
|
||||||
|
// secret created through MCP, and hush's central claim would hold for browser
|
||||||
|
// users and quietly not hold for agent users. Two guarantees behind one URL is
|
||||||
|
// worse than one honest guarantee.
|
||||||
|
//
|
||||||
|
// So this binary is a peer of the browser, not a peer of the server: it mints
|
||||||
|
// the key, encrypts, posts ciphertext, and assembles the `#fragment` link.
|
||||||
|
// hushd sees exactly what it sees from a browser.
|
||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"crypto/aes"
|
||||||
|
"crypto/cipher"
|
||||||
|
"crypto/rand"
|
||||||
|
"encoding/base64"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"net/url"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
const version = "0.1.0"
|
||||||
|
|
||||||
|
// stderr is indirected so tests can capture diagnostics. stdout is reserved
|
||||||
|
// for protocol frames — a stray write there corrupts the stream.
|
||||||
|
var stderr io.Writer = os.Stderr
|
||||||
|
|
||||||
|
// b64 matches the server and the browser: base64url, unpadded. One spelling of
|
||||||
|
// the wire format across all three implementations.
|
||||||
|
var b64 = base64.RawURLEncoding
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
base := strings.TrimSuffix(os.Getenv("HUSH_BASE_URL"), "/")
|
||||||
|
if base == "" {
|
||||||
|
base = "https://hush.threesix.ai"
|
||||||
|
}
|
||||||
|
c := &client{
|
||||||
|
base: base,
|
||||||
|
http: &http.Client{Timeout: 15 * time.Second},
|
||||||
|
// A create token is only needed if the deployment has closed anonymous
|
||||||
|
// create (HUSH_REQUIRE_AUTH). Empty is the normal case.
|
||||||
|
token: os.Getenv("HUSH_CREATE_TOKEN"),
|
||||||
|
}
|
||||||
|
|
||||||
|
srv := NewServer("hush", version, os.Stdout, tools(c))
|
||||||
|
if err := srv.Serve(os.Stdin); err != nil {
|
||||||
|
fmt.Fprintf(stderr, "hush-mcp: %v\n", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func tools(c *client) []Tool {
|
||||||
|
return []Tool{
|
||||||
|
{
|
||||||
|
Name: "hush_create",
|
||||||
|
Title: "Create a one-time secret link",
|
||||||
|
Description: "Encrypt a secret locally and store the ciphertext on hush, returning a " +
|
||||||
|
"link that works exactly once. The encryption key is generated on this machine and " +
|
||||||
|
"travels only in the link's #fragment, so the hush server never receives it and " +
|
||||||
|
"cannot read the secret. Use this to hand a credential to someone instead of " +
|
||||||
|
"pasting it into chat or email. The returned link is shown once and cannot be " +
|
||||||
|
"recovered — pass it on immediately.",
|
||||||
|
Schema: map[string]any{
|
||||||
|
"type": "object",
|
||||||
|
"properties": map[string]any{
|
||||||
|
"secret": map[string]any{
|
||||||
|
"type": "string",
|
||||||
|
"description": "The plaintext to send. Never leaves this machine unencrypted.",
|
||||||
|
},
|
||||||
|
"ttl_seconds": map[string]any{
|
||||||
|
"type": "integer",
|
||||||
|
"description": "Lifetime in seconds, 300 to 604800. Defaults to 86400 (24h).",
|
||||||
|
"minimum": 300,
|
||||||
|
"maximum": 604800,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"required": []string{"secret"},
|
||||||
|
"additionalProperties": false,
|
||||||
|
},
|
||||||
|
Handler: c.create,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Name: "hush_reveal",
|
||||||
|
Title: "Open a one-time secret link",
|
||||||
|
Description: "Fetch and decrypt a hush link, DESTROYING it in the process. This is " +
|
||||||
|
"irreversible: after this call the link is dead and nobody else can open it, " +
|
||||||
|
"including the intended recipient. Only call this on a link meant for you.",
|
||||||
|
Schema: map[string]any{
|
||||||
|
"type": "object",
|
||||||
|
"properties": map[string]any{
|
||||||
|
"link": map[string]any{
|
||||||
|
"type": "string",
|
||||||
|
"description": "The full hush link including the #fragment key.",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"required": []string{"link"},
|
||||||
|
"additionalProperties": false,
|
||||||
|
},
|
||||||
|
Handler: c.reveal,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type client struct {
|
||||||
|
base string
|
||||||
|
http *http.Client
|
||||||
|
token string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *client) create(raw json.RawMessage) (string, error) {
|
||||||
|
var args struct {
|
||||||
|
Secret string `json:"secret"`
|
||||||
|
TTLSeconds int64 `json:"ttl_seconds"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(raw, &args); err != nil {
|
||||||
|
return "", fmt.Errorf("bad arguments: %w", err)
|
||||||
|
}
|
||||||
|
if args.Secret == "" {
|
||||||
|
return "", errors.New("secret is empty — nothing to send")
|
||||||
|
}
|
||||||
|
|
||||||
|
ciphertext, key, err := seal(args.Secret)
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("encrypt locally: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
body, _ := json.Marshal(map[string]any{"ciphertext": ciphertext, "ttl_seconds": args.TTLSeconds})
|
||||||
|
req, err := http.NewRequest(http.MethodPost, c.base+"/api/secrets", bytes.NewReader(body))
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
req.Header.Set("Content-Type", "application/json")
|
||||||
|
if c.token != "" {
|
||||||
|
req.Header.Set("Authorization", "Bearer "+c.token)
|
||||||
|
}
|
||||||
|
|
||||||
|
res, err := c.http.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("reach hush at %s: %w", c.base, err)
|
||||||
|
}
|
||||||
|
defer res.Body.Close()
|
||||||
|
payload, _ := io.ReadAll(io.LimitReader(res.Body, 1<<20))
|
||||||
|
|
||||||
|
if res.StatusCode != http.StatusCreated {
|
||||||
|
return "", fmt.Errorf("hush refused the secret (%s): %s", res.Status, apiMessage(payload))
|
||||||
|
}
|
||||||
|
var out struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
ExpiresAt string `json:"expires_at"`
|
||||||
|
TTLSeconds int64 `json:"ttl_seconds"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(payload, &out); err != nil || out.ID == "" {
|
||||||
|
return "", fmt.Errorf("unexpected response from hush: %s", string(payload))
|
||||||
|
}
|
||||||
|
|
||||||
|
// The key is appended HERE, on this machine. It was never in the request.
|
||||||
|
link := c.base + "/s/" + out.ID + "#" + key
|
||||||
|
return fmt.Sprintf(
|
||||||
|
"%s\n\nOne-time link — works exactly once, expires %s.\n"+
|
||||||
|
"The key is in the #fragment, so hush cannot read the secret.\n"+
|
||||||
|
"This link cannot be shown again: send it now.",
|
||||||
|
link, out.ExpiresAt), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *client) reveal(raw json.RawMessage) (string, error) {
|
||||||
|
var args struct {
|
||||||
|
Link string `json:"link"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(raw, &args); err != nil {
|
||||||
|
return "", fmt.Errorf("bad arguments: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
u, err := url.Parse(strings.TrimSpace(args.Link))
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("not a URL: %w", err)
|
||||||
|
}
|
||||||
|
if u.Fragment == "" {
|
||||||
|
// The commonest real failure: a chat client or mail rewriter dropped
|
||||||
|
// the fragment. Say so precisely, because the secret is still intact
|
||||||
|
// and the fix is to ask the sender for the whole link.
|
||||||
|
return "", errors.New("this link has no #fragment, so it carries no key. " +
|
||||||
|
"Chat and email clients sometimes truncate it — ask the sender for the full link. " +
|
||||||
|
"The secret has NOT been opened.")
|
||||||
|
}
|
||||||
|
id := strings.TrimPrefix(u.Path, "/s/")
|
||||||
|
if id == "" || strings.Contains(id, "/") {
|
||||||
|
return "", fmt.Errorf("cannot find a secret id in the path %q", u.Path)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Reveal against the link's OWN origin, not the configured base: a link
|
||||||
|
// from a different hush deployment must not be posted to this one, where
|
||||||
|
// the id would be meaningless.
|
||||||
|
origin := u.Scheme + "://" + u.Host
|
||||||
|
res, err := c.http.Post(origin+"/api/secrets/"+url.PathEscape(id)+"/reveal", "application/json", nil)
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("reach hush at %s: %w", origin, err)
|
||||||
|
}
|
||||||
|
defer res.Body.Close()
|
||||||
|
payload, _ := io.ReadAll(io.LimitReader(res.Body, 1<<20))
|
||||||
|
|
||||||
|
if res.StatusCode == http.StatusGone {
|
||||||
|
return "", errors.New("gone: this link was already opened, expired, or never existed. " +
|
||||||
|
"If you did not open it yourself, assume someone else did and ask the sender to rotate the secret.")
|
||||||
|
}
|
||||||
|
if res.StatusCode != http.StatusOK {
|
||||||
|
return "", fmt.Errorf("hush returned %s: %s", res.Status, apiMessage(payload))
|
||||||
|
}
|
||||||
|
|
||||||
|
var out struct {
|
||||||
|
Ciphertext string `json:"ciphertext"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(payload, &out); err != nil {
|
||||||
|
return "", fmt.Errorf("unexpected response from hush: %s", string(payload))
|
||||||
|
}
|
||||||
|
plain, err := open(out.Ciphertext, u.Fragment)
|
||||||
|
if err != nil {
|
||||||
|
// The ciphertext is already destroyed at this point, so there is
|
||||||
|
// nothing to retry. Say that plainly.
|
||||||
|
return "", fmt.Errorf("the key in this link does not open this secret, and the ciphertext "+
|
||||||
|
"has now been destroyed (%w). The link was probably altered in transit; ask for a new one", err)
|
||||||
|
}
|
||||||
|
return plain, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// seal encrypts with AES-256-GCM and returns (ciphertext, key), both base64url.
|
||||||
|
// The nonce is prepended to the ciphertext so the stored blob is self-contained
|
||||||
|
// — byte-for-byte the format templates/base.html produces.
|
||||||
|
func seal(plaintext string) (string, string, error) {
|
||||||
|
key := make([]byte, 32)
|
||||||
|
if _, err := rand.Read(key); err != nil {
|
||||||
|
return "", "", err
|
||||||
|
}
|
||||||
|
block, err := aes.NewCipher(key)
|
||||||
|
if err != nil {
|
||||||
|
return "", "", err
|
||||||
|
}
|
||||||
|
gcm, err := cipher.NewGCM(block)
|
||||||
|
if err != nil {
|
||||||
|
return "", "", err
|
||||||
|
}
|
||||||
|
nonce := make([]byte, gcm.NonceSize())
|
||||||
|
if _, err := rand.Read(nonce); err != nil {
|
||||||
|
return "", "", err
|
||||||
|
}
|
||||||
|
blob := gcm.Seal(nonce, nonce, []byte(plaintext), nil)
|
||||||
|
return b64.EncodeToString(blob), b64.EncodeToString(key), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func open(ciphertext, keyStr string) (string, error) {
|
||||||
|
blob, err := b64.DecodeString(ciphertext)
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("ciphertext is not base64url: %w", err)
|
||||||
|
}
|
||||||
|
key, err := b64.DecodeString(keyStr)
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("key is not base64url: %w", err)
|
||||||
|
}
|
||||||
|
block, err := aes.NewCipher(key)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
gcm, err := cipher.NewGCM(block)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
if len(blob) < gcm.NonceSize() {
|
||||||
|
return "", errors.New("ciphertext is too short to contain a nonce")
|
||||||
|
}
|
||||||
|
plain, err := gcm.Open(nil, blob[:gcm.NonceSize()], blob[gcm.NonceSize():], nil)
|
||||||
|
if err != nil {
|
||||||
|
return "", errors.New("authentication failed")
|
||||||
|
}
|
||||||
|
return string(plain), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// apiMessage pulls the human message out of hush's error envelope, falling back
|
||||||
|
// to the raw body so a proxy's HTML error page is still readable.
|
||||||
|
func apiMessage(payload []byte) string {
|
||||||
|
var e struct {
|
||||||
|
Error struct {
|
||||||
|
Code string `json:"code"`
|
||||||
|
Message string `json:"message"`
|
||||||
|
} `json:"error"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(payload, &e); err == nil && e.Error.Message != "" {
|
||||||
|
return e.Error.Code + ": " + e.Error.Message
|
||||||
|
}
|
||||||
|
s := strings.TrimSpace(string(payload))
|
||||||
|
if len(s) > 300 {
|
||||||
|
s = s[:300] + "…"
|
||||||
|
}
|
||||||
|
return s
|
||||||
|
}
|
||||||
195
cmd/hush-mcp/mcp.go
Normal file
195
cmd/hush-mcp/mcp.go
Normal file
@ -0,0 +1,195 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bufio"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"sync"
|
||||||
|
)
|
||||||
|
|
||||||
|
// A minimal, dependency-free MCP server over stdio.
|
||||||
|
//
|
||||||
|
// MCP on stdio is newline-delimited JSON-RPC 2.0. The surface this server needs
|
||||||
|
// is four methods — initialize, notifications/initialized, tools/list,
|
||||||
|
// tools/call — so it is implemented directly rather than pulling in an SDK
|
||||||
|
// whose API churn would be a bigger maintenance surface than the protocol.
|
||||||
|
//
|
||||||
|
// The one rule that matters for a stdio server: stdout carries protocol frames
|
||||||
|
// ONLY. Anything diagnostic goes to stderr, because a stray Println on stdout
|
||||||
|
// corrupts the stream and the host reports an opaque parse failure.
|
||||||
|
|
||||||
|
// protocolVersion is the MCP revision this server implements. The host sends
|
||||||
|
// its own in initialize; the spec has the server answer with the version it
|
||||||
|
// will actually speak rather than echoing the client's.
|
||||||
|
const protocolVersion = "2025-06-18"
|
||||||
|
|
||||||
|
type request struct {
|
||||||
|
JSONRPC string `json:"jsonrpc"`
|
||||||
|
ID json.RawMessage `json:"id,omitempty"`
|
||||||
|
Method string `json:"method"`
|
||||||
|
Params json.RawMessage `json:"params,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type response struct {
|
||||||
|
JSONRPC string `json:"jsonrpc"`
|
||||||
|
ID json.RawMessage `json:"id,omitempty"`
|
||||||
|
Result any `json:"result,omitempty"`
|
||||||
|
Error *rpcError `json:"error,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type rpcError struct {
|
||||||
|
Code int `json:"code"`
|
||||||
|
Message string `json:"message"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// JSON-RPC 2.0 reserved codes. -32602 is the one that matters here: a bad tool
|
||||||
|
// argument is an invalid-params error, not a transport failure.
|
||||||
|
const (
|
||||||
|
codeInvalidParams = -32602
|
||||||
|
codeMethodMissing = -32601
|
||||||
|
codeInternal = -32603
|
||||||
|
)
|
||||||
|
|
||||||
|
// Tool is one callable tool. Schema is the raw JSON Schema advertised to the
|
||||||
|
// host, which is what makes the arguments self-documenting in the client.
|
||||||
|
type Tool struct {
|
||||||
|
Name string
|
||||||
|
Title string
|
||||||
|
Description string
|
||||||
|
Schema map[string]any
|
||||||
|
// Handler returns the text to show the caller. An error is reported as a
|
||||||
|
// TOOL error (isError on the result) rather than a protocol error, so the
|
||||||
|
// model sees the message and can act on it instead of the call appearing
|
||||||
|
// to have failed at the transport level.
|
||||||
|
Handler func(args json.RawMessage) (string, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Server dispatches MCP over a reader/writer pair.
|
||||||
|
type Server struct {
|
||||||
|
name string
|
||||||
|
version string
|
||||||
|
tools []Tool
|
||||||
|
|
||||||
|
mu sync.Mutex // serialises writes: one frame per line, never interleaved
|
||||||
|
out *json.Encoder
|
||||||
|
w io.Writer
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewServer(name, version string, out io.Writer, tools []Tool) *Server {
|
||||||
|
return &Server{name: name, version: version, tools: tools, out: json.NewEncoder(out), w: out}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Serve reads frames until stdin closes, which is how a stdio host signals
|
||||||
|
// shutdown.
|
||||||
|
func (s *Server) Serve(in io.Reader) error {
|
||||||
|
sc := bufio.NewScanner(in)
|
||||||
|
// A tool result can carry a secret link, which is small, but the buffer is
|
||||||
|
// raised so a large argument cannot truncate a frame into a parse error.
|
||||||
|
sc.Buffer(make([]byte, 0, 64*1024), 4*1024*1024)
|
||||||
|
for sc.Scan() {
|
||||||
|
line := sc.Bytes()
|
||||||
|
if len(line) == 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
var req request
|
||||||
|
if err := json.Unmarshal(line, &req); err != nil {
|
||||||
|
// Malformed frame with no id: nothing to reply to. Report on stderr
|
||||||
|
// and keep the stream alive.
|
||||||
|
fmt.Fprintf(stderr, "hush-mcp: unparseable frame: %v\n", err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
s.dispatch(req)
|
||||||
|
}
|
||||||
|
return sc.Err()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) dispatch(req request) {
|
||||||
|
// A notification has no id and MUST NOT be answered. Replying to one is the
|
||||||
|
// classic stdio bug: the host sees an unsolicited response and desyncs.
|
||||||
|
isNotification := len(req.ID) == 0
|
||||||
|
|
||||||
|
switch req.Method {
|
||||||
|
case "initialize":
|
||||||
|
s.reply(req.ID, map[string]any{
|
||||||
|
"protocolVersion": protocolVersion,
|
||||||
|
"capabilities": map[string]any{"tools": map[string]any{}},
|
||||||
|
"serverInfo": map[string]any{"name": s.name, "version": s.version},
|
||||||
|
})
|
||||||
|
case "notifications/initialized":
|
||||||
|
// Handshake complete. Nothing to send.
|
||||||
|
case "ping":
|
||||||
|
s.reply(req.ID, map[string]any{})
|
||||||
|
case "tools/list":
|
||||||
|
list := make([]map[string]any, 0, len(s.tools))
|
||||||
|
for _, t := range s.tools {
|
||||||
|
list = append(list, map[string]any{
|
||||||
|
"name": t.Name,
|
||||||
|
"title": t.Title,
|
||||||
|
"description": t.Description,
|
||||||
|
"inputSchema": t.Schema,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
s.reply(req.ID, map[string]any{"tools": list})
|
||||||
|
case "tools/call":
|
||||||
|
s.call(req)
|
||||||
|
default:
|
||||||
|
if !isNotification {
|
||||||
|
s.fail(req.ID, codeMethodMissing, "unsupported method: "+req.Method)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) call(req request) {
|
||||||
|
var p struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Arguments json.RawMessage `json:"arguments"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(req.Params, &p); err != nil {
|
||||||
|
s.fail(req.ID, codeInvalidParams, "malformed tools/call params")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
for _, t := range s.tools {
|
||||||
|
if t.Name != p.Name {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
text, err := t.Handler(p.Arguments)
|
||||||
|
if err != nil {
|
||||||
|
// isError:true keeps this a TOOL failure the model can read and
|
||||||
|
// react to, rather than a protocol error that looks like the server
|
||||||
|
// broke.
|
||||||
|
s.reply(req.ID, map[string]any{
|
||||||
|
"content": []map[string]any{{"type": "text", "text": err.Error()}},
|
||||||
|
"isError": true,
|
||||||
|
})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.reply(req.ID, map[string]any{
|
||||||
|
"content": []map[string]any{{"type": "text", "text": text}},
|
||||||
|
})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.fail(req.ID, codeInvalidParams, "unknown tool: "+p.Name)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) reply(id json.RawMessage, result any) {
|
||||||
|
if len(id) == 0 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.write(response{JSONRPC: "2.0", ID: id, Result: result})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) fail(id json.RawMessage, code int, msg string) {
|
||||||
|
if len(id) == 0 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.write(response{JSONRPC: "2.0", ID: id, Error: &rpcError{Code: code, Message: msg}})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) write(r response) {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
if err := s.out.Encode(r); err != nil {
|
||||||
|
fmt.Fprintf(stderr, "hush-mcp: write failed: %v\n", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
65
cmd/hushd/config.go
Normal file
65
cmd/hushd/config.go
Normal file
@ -0,0 +1,65 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/orchard9/go-chassis/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Config is every knob hushd has. Loaded once at boot; a missing or malformed
|
||||||
|
// required value stops the process rather than being defaulted, so a
|
||||||
|
// misconfigured pod crashloops visibly instead of serving something subtly
|
||||||
|
// wrong.
|
||||||
|
type Config struct {
|
||||||
|
Env string
|
||||||
|
Port int
|
||||||
|
|
||||||
|
// RedisURL is REQUIRED. There is an in-memory store in the codebase for
|
||||||
|
// tests, and requiring this is what makes it impossible to select by
|
||||||
|
// accident in production: a pod with no REDIS_URL does not boot, rather
|
||||||
|
// than booting with a store that loses every secret on restart.
|
||||||
|
RedisURL string
|
||||||
|
|
||||||
|
// RateLimit bounds anonymous creates per client IP. Reveals are not
|
||||||
|
// limited by this: a recipient gets exactly one successful reveal by
|
||||||
|
// construction, so there is nothing to throttle, and throttling would let
|
||||||
|
// one noisy NAT block a colleague's delivery.
|
||||||
|
RateLimitCreates int
|
||||||
|
RateLimitWindow time.Duration
|
||||||
|
|
||||||
|
// TrustedProxyHops is how many Traefik hops sit in front of hushd, used to
|
||||||
|
// pick the real client IP out of X-Forwarded-For. Wrong-high lets a caller
|
||||||
|
// spoof their IP and evade the rate limit; wrong-low rate-limits the
|
||||||
|
// ingress itself and throttles everyone together.
|
||||||
|
TrustedProxyHops int
|
||||||
|
|
||||||
|
// AllowOrigins is the CORS allowlist. Empty is correct for the deployed
|
||||||
|
// service: both pages are same-origin, so no cross-origin caller is
|
||||||
|
// legitimate.
|
||||||
|
AllowOrigins []string
|
||||||
|
|
||||||
|
// RequireAuthToCreate closes anonymous create if the service is abused.
|
||||||
|
// Reveal stays anonymous regardless — the recipient is external and holds
|
||||||
|
// no credential.
|
||||||
|
RequireAuthToCreate bool
|
||||||
|
CreateToken string
|
||||||
|
}
|
||||||
|
|
||||||
|
func loadConfig() (Config, error) {
|
||||||
|
l := config.New()
|
||||||
|
cfg := Config{
|
||||||
|
Env: l.OneOf("APP_ENV", "dev", "dev", "staging", "prod"),
|
||||||
|
Port: l.Port("HUSH_PORT", 18500),
|
||||||
|
RedisURL: l.Required("REDIS_URL"),
|
||||||
|
RateLimitCreates: l.Int("HUSH_RATE_LIMIT_CREATES", 30),
|
||||||
|
RateLimitWindow: l.Duration("HUSH_RATE_LIMIT_WINDOW", 10*time.Minute),
|
||||||
|
TrustedProxyHops: l.Int("HUSH_TRUSTED_PROXY_HOPS", 1),
|
||||||
|
AllowOrigins: l.Strings("HUSH_ALLOW_ORIGINS", nil),
|
||||||
|
RequireAuthToCreate: l.Bool("HUSH_REQUIRE_AUTH", false),
|
||||||
|
CreateToken: l.String("HUSH_CREATE_TOKEN", ""),
|
||||||
|
}
|
||||||
|
if err := l.Err(); err != nil {
|
||||||
|
return Config{}, err
|
||||||
|
}
|
||||||
|
return cfg, nil
|
||||||
|
}
|
||||||
183
cmd/hushd/handlers.go
Normal file
183
cmd/hushd/handlers.go
Normal file
@ -0,0 +1,183 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"net/http"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/orchard9/go-chassis/chassis"
|
||||||
|
|
||||||
|
"github.com/orchard9/hush/internal/secret"
|
||||||
|
"github.com/orchard9/hush/internal/store"
|
||||||
|
"github.com/orchard9/hush/internal/web"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Server holds the handler dependencies.
|
||||||
|
type Server struct {
|
||||||
|
cfg Config
|
||||||
|
store store.Store
|
||||||
|
pages *web.Pages
|
||||||
|
metrics *Metrics
|
||||||
|
}
|
||||||
|
|
||||||
|
// pageData is the same for every render: the limits, so the browser enforces
|
||||||
|
// what the server enforces. It carries nothing request-specific, which is why
|
||||||
|
// both pages are safely static.
|
||||||
|
func (s *Server) pageData() web.Data {
|
||||||
|
return web.Data{
|
||||||
|
MaxCiphertextBytes: secret.MaxCiphertextBytes,
|
||||||
|
DefaultTTLSeconds: int(secret.DefaultTTL.Seconds()),
|
||||||
|
MinTTLSeconds: int(secret.MinTTL.Seconds()),
|
||||||
|
MaxTTLSeconds: int(secret.MaxTTL.Seconds()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// handleCreatePage serves GET /.
|
||||||
|
func (s *Server) handleCreatePage(c *chassis.Context) error {
|
||||||
|
return s.pages.Create(c.Writer(), s.pageData())
|
||||||
|
}
|
||||||
|
|
||||||
|
// handleRevealPage serves GET /s/{id}.
|
||||||
|
//
|
||||||
|
// It touches NO storage — not even to check whether the id exists. That is the
|
||||||
|
// design's load-bearing property: Slack, Teams, WhatsApp, iMessage and Outlook
|
||||||
|
// Safe Links all fetch URLs before a human sees them, so any read here would
|
||||||
|
// destroy most secrets in transit. It also means this response cannot leak
|
||||||
|
// whether an id exists.
|
||||||
|
//
|
||||||
|
// The id is not even parsed: a malformed one gets the same page, and the POST
|
||||||
|
// is what answers. Validating here would make this endpoint an id oracle.
|
||||||
|
func (s *Server) handleRevealPage(c *chassis.Context) error {
|
||||||
|
return s.pages.Reveal(c.Writer(), s.pageData())
|
||||||
|
}
|
||||||
|
|
||||||
|
type createRequest struct {
|
||||||
|
Ciphertext string `json:"ciphertext"`
|
||||||
|
TTLSeconds int64 `json:"ttl_seconds"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type createResponse struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
ExpiresAt string `json:"expires_at"`
|
||||||
|
TTLSeconds int64 `json:"ttl_seconds"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// handleCreate serves POST /api/secrets.
|
||||||
|
//
|
||||||
|
// It accepts ciphertext only. There is deliberately no endpoint taking a
|
||||||
|
// plaintext secret: one would make the server able to read secrets, and then
|
||||||
|
// nobody could tell from a link which guarantee they had.
|
||||||
|
func (s *Server) handleCreate(c *chassis.Context) error {
|
||||||
|
var req createRequest
|
||||||
|
if err := c.Bind(&req); err != nil {
|
||||||
|
s.metrics.Rejected.WithLabelValues(reasonMalformed).Inc()
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := secret.ValidateCiphertext(req.Ciphertext); err != nil {
|
||||||
|
reason, code := classifyCiphertextError(err)
|
||||||
|
s.metrics.Rejected.WithLabelValues(reason).Inc()
|
||||||
|
// err carries sizes, never content: ValidateCiphertext never puts the
|
||||||
|
// ciphertext in its message.
|
||||||
|
return &chassis.Error{Status: http.StatusUnprocessableEntity, Code: code, Msg: err.Error()}
|
||||||
|
}
|
||||||
|
|
||||||
|
ttl, err := secret.ResolveTTL(time.Duration(req.TTLSeconds) * time.Second)
|
||||||
|
if err != nil {
|
||||||
|
s.metrics.Rejected.WithLabelValues(reasonTTL).Inc()
|
||||||
|
return &chassis.Error{Status: http.StatusUnprocessableEntity, Code: reasonTTL, Msg: err.Error()}
|
||||||
|
}
|
||||||
|
|
||||||
|
id, err := secret.NewID()
|
||||||
|
if err != nil {
|
||||||
|
// No entropy means no safe id. Refuse rather than mint a guessable one.
|
||||||
|
return chassis.Internal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := s.store.Put(c.Context(), id, req.Ciphertext, ttl); err != nil {
|
||||||
|
if errors.Is(err, store.ErrIDCollision) {
|
||||||
|
// Impossible at 256 bits, so it means an id-generation bug. Surfaced
|
||||||
|
// rather than retried, because a retry loop would hide it.
|
||||||
|
c.Log().Error("secret.id_collision", "category", "secret",
|
||||||
|
"error_type", "id_collision", "sid", id.LogHandle())
|
||||||
|
return chassis.Internal(err)
|
||||||
|
}
|
||||||
|
return chassis.Internal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.metrics.Created.Inc()
|
||||||
|
s.metrics.Bytes.Observe(float64(len(req.Ciphertext)))
|
||||||
|
// sid, not id: the id is the capability and never reaches a log.
|
||||||
|
c.Log().Info("secret.created", "category", "secret",
|
||||||
|
"sid", id.LogHandle(), "ttl_seconds", int64(ttl.Seconds()),
|
||||||
|
"ciphertext_bytes", len(req.Ciphertext))
|
||||||
|
|
||||||
|
return c.Created(createResponse{
|
||||||
|
ID: id.Value(),
|
||||||
|
ExpiresAt: time.Now().UTC().Add(ttl).Format(time.RFC3339),
|
||||||
|
TTLSeconds: int64(ttl.Seconds()),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
type revealResponse struct {
|
||||||
|
Ciphertext string `json:"ciphertext"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// gone is the single response for every unavailable secret: never existed,
|
||||||
|
// already revealed, expired, or evicted early by Redis.
|
||||||
|
//
|
||||||
|
// One response for four causes is deliberate. A caller able to tell "already
|
||||||
|
// revealed" from "never existed" learns that a particular link was real, which
|
||||||
|
// is information about someone else's secret.
|
||||||
|
func gone() *chassis.Error {
|
||||||
|
return &chassis.Error{
|
||||||
|
Status: http.StatusGone,
|
||||||
|
Code: "gone",
|
||||||
|
Msg: "this secret is not available: it was already revealed, expired, or never existed",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// handleReveal serves POST /api/secrets/{id}/reveal — the only destructive
|
||||||
|
// route, and the reason GET is inert.
|
||||||
|
func (s *Server) handleReveal(c *chassis.Context) error {
|
||||||
|
id, err := secret.ParseID(c.PathValue("id"))
|
||||||
|
if err != nil {
|
||||||
|
// A malformed id gets the SAME 410 as a missing one. A 400 here would
|
||||||
|
// separate "not an id we could have minted" from "an id that is gone",
|
||||||
|
// which is a free oracle for anyone probing the id space.
|
||||||
|
s.metrics.Revealed.WithLabelValues("gone").Inc()
|
||||||
|
return gone()
|
||||||
|
}
|
||||||
|
|
||||||
|
ciphertext, err := s.store.Take(c.Context(), id)
|
||||||
|
switch {
|
||||||
|
case errors.Is(err, store.ErrGone):
|
||||||
|
s.metrics.Revealed.WithLabelValues("gone").Inc()
|
||||||
|
c.Log().Info("secret.gone", "category", "secret", "sid", id.LogHandle())
|
||||||
|
return gone()
|
||||||
|
case err != nil:
|
||||||
|
// A store error is NOT reported as gone. The secret may still exist,
|
||||||
|
// and telling the recipient it is gone would send them to rotate a
|
||||||
|
// credential that was never delivered.
|
||||||
|
return chassis.Internal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.metrics.Revealed.WithLabelValues("ok").Inc()
|
||||||
|
c.Log().Info("secret.revealed", "category", "secret", "sid", id.LogHandle())
|
||||||
|
return c.OK(revealResponse{Ciphertext: ciphertext})
|
||||||
|
}
|
||||||
|
|
||||||
|
// classifyCiphertextError maps a validation failure to its metric reason and
|
||||||
|
// API error code, which are the same vocabulary on purpose.
|
||||||
|
func classifyCiphertextError(err error) (reason, code string) {
|
||||||
|
switch {
|
||||||
|
case errors.Is(err, secret.ErrCiphertextTooLarge):
|
||||||
|
return reasonTooLarge, reasonTooLarge
|
||||||
|
case errors.Is(err, secret.ErrCiphertextInvalid):
|
||||||
|
return reasonInvalid, reasonInvalid
|
||||||
|
case errors.Is(err, secret.ErrCiphertextEmpty):
|
||||||
|
return reasonEmpty, reasonEmpty
|
||||||
|
default:
|
||||||
|
return reasonMalformed, reasonMalformed
|
||||||
|
}
|
||||||
|
}
|
||||||
294
cmd/hushd/handlers_test.go
Normal file
294
cmd/hushd/handlers_test.go
Normal file
@ -0,0 +1,294 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/base64"
|
||||||
|
"encoding/json"
|
||||||
|
"io"
|
||||||
|
"log/slog"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/orchard9/go-chassis/chassis"
|
||||||
|
|
||||||
|
"github.com/orchard9/hush/internal/secret"
|
||||||
|
"github.com/orchard9/hush/internal/store"
|
||||||
|
"github.com/orchard9/hush/internal/web"
|
||||||
|
)
|
||||||
|
|
||||||
|
// testApp builds the real route table over an in-memory store, so these tests
|
||||||
|
// exercise the actual middleware chain, binder and error envelopes rather than
|
||||||
|
// calling handlers directly.
|
||||||
|
func testApp(t *testing.T) (http.Handler, *store.Memory) {
|
||||||
|
t.Helper()
|
||||||
|
pages, err := web.New()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
mem := store.NewMemory()
|
||||||
|
srv := &Server{
|
||||||
|
cfg: Config{Env: "dev"},
|
||||||
|
store: mem,
|
||||||
|
pages: pages,
|
||||||
|
metrics: newMetrics(),
|
||||||
|
}
|
||||||
|
srv.metrics.Prime()
|
||||||
|
|
||||||
|
log := slog.New(slog.NewJSONHandler(io.Discard, nil))
|
||||||
|
app := chassis.New(chassis.Config{Service: "hush", Env: "dev", MaxBodyBytes: 128 * 1024}, log)
|
||||||
|
app.Get("/", srv.handleCreatePage)
|
||||||
|
app.Get("/s/{id}", srv.handleRevealPage)
|
||||||
|
app.Route("/api", func(r *chassis.Router) {
|
||||||
|
r.Post("/secrets", srv.handleCreate)
|
||||||
|
r.Post("/secrets/{id}/reveal", srv.handleReveal)
|
||||||
|
})
|
||||||
|
return app.Handler(), mem
|
||||||
|
}
|
||||||
|
|
||||||
|
func ciphertext(s string) string {
|
||||||
|
return base64.RawURLEncoding.EncodeToString([]byte(s))
|
||||||
|
}
|
||||||
|
|
||||||
|
func do(t *testing.T, h http.Handler, method, path, body string) (int, map[string]any, string) {
|
||||||
|
t.Helper()
|
||||||
|
var r *http.Request
|
||||||
|
if body == "" {
|
||||||
|
r = httptest.NewRequest(method, path, nil)
|
||||||
|
} else {
|
||||||
|
r = httptest.NewRequest(method, path, strings.NewReader(body))
|
||||||
|
r.Header.Set("Content-Type", "application/json")
|
||||||
|
}
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
h.ServeHTTP(w, r)
|
||||||
|
raw := w.Body.String()
|
||||||
|
var parsed map[string]any
|
||||||
|
_ = json.Unmarshal([]byte(raw), &parsed)
|
||||||
|
return w.Code, parsed, raw
|
||||||
|
}
|
||||||
|
|
||||||
|
func create(t *testing.T, h http.Handler, ct string) string {
|
||||||
|
t.Helper()
|
||||||
|
code, body, raw := do(t, h, http.MethodPost, "/api/secrets",
|
||||||
|
`{"ciphertext":"`+ct+`","ttl_seconds":3600}`)
|
||||||
|
if code != http.StatusCreated {
|
||||||
|
t.Fatalf("create = %d, want 201: %s", code, raw)
|
||||||
|
}
|
||||||
|
id, ok := body["id"].(string)
|
||||||
|
if !ok || id == "" {
|
||||||
|
t.Fatalf("create response carried no id: %s", raw)
|
||||||
|
}
|
||||||
|
return id
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCreateThenRevealThenGone(t *testing.T) {
|
||||||
|
h, _ := testApp(t)
|
||||||
|
ct := ciphertext("the-secret-bytes")
|
||||||
|
id := create(t, h, ct)
|
||||||
|
|
||||||
|
code, body, raw := do(t, h, http.MethodPost, "/api/secrets/"+id+"/reveal", "")
|
||||||
|
if code != http.StatusOK {
|
||||||
|
t.Fatalf("first reveal = %d, want 200: %s", code, raw)
|
||||||
|
}
|
||||||
|
if body["ciphertext"] != ct {
|
||||||
|
t.Fatalf("reveal returned %v, want the stored ciphertext", body["ciphertext"])
|
||||||
|
}
|
||||||
|
|
||||||
|
code, _, raw = do(t, h, http.MethodPost, "/api/secrets/"+id+"/reveal", "")
|
||||||
|
if code != http.StatusGone {
|
||||||
|
t.Fatalf("second reveal = %d, want 410: %s", code, raw)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// THE regression test for this whole design. Slack, Teams, WhatsApp, iMessage
|
||||||
|
// and Outlook Safe Links fetch a URL to build a preview before any human sees
|
||||||
|
// it. If GET consumed the secret, most secrets would be destroyed in transit
|
||||||
|
// and the recipient's "already used" would be indistinguishable from a real
|
||||||
|
// interception.
|
||||||
|
//
|
||||||
|
// So: fetching the reveal page any number of times must leave the secret intact.
|
||||||
|
func TestGettingTheRevealPageNeverConsumesTheSecret(t *testing.T) {
|
||||||
|
h, _ := testApp(t)
|
||||||
|
ct := ciphertext("survives-the-previewers")
|
||||||
|
id := create(t, h, ct)
|
||||||
|
|
||||||
|
for i := range 5 {
|
||||||
|
code, _, _ := do(t, h, http.MethodGet, "/s/"+id, "")
|
||||||
|
if code != http.StatusOK {
|
||||||
|
t.Fatalf("GET /s/{id} attempt %d = %d, want 200", i, code)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
code, body, raw := do(t, h, http.MethodPost, "/api/secrets/"+id+"/reveal", "")
|
||||||
|
if code != http.StatusOK {
|
||||||
|
t.Fatalf("reveal after 5 page loads = %d, want 200 — a GET consumed the secret: %s", code, raw)
|
||||||
|
}
|
||||||
|
if body["ciphertext"] != ct {
|
||||||
|
t.Fatal("the ciphertext changed across page loads")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The reveal page must be identical for every id, including ids that were never
|
||||||
|
// minted. If it 404'd on an unknown id it would become an oracle for whether a
|
||||||
|
// link was ever real.
|
||||||
|
func TestTheRevealPageDoesNotDiscloseWhetherASecretExists(t *testing.T) {
|
||||||
|
h, _ := testApp(t)
|
||||||
|
id := create(t, h, ciphertext("real"))
|
||||||
|
|
||||||
|
_, _, real := do(t, h, http.MethodGet, "/s/"+id, "")
|
||||||
|
_, _, fake := do(t, h, http.MethodGet, "/s/"+strings.Repeat("A", 43), "")
|
||||||
|
_, _, junk := do(t, h, http.MethodGet, "/s/not-an-id", "")
|
||||||
|
|
||||||
|
if real != fake || real != junk {
|
||||||
|
t.Fatal("the reveal page differs between a real id, a well-formed unknown id, and junk — " +
|
||||||
|
"it must not disclose existence")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A malformed id must produce the SAME 410 as a missing one. A 400 here would
|
||||||
|
// separate "not an id this service could mint" from "an id that is gone", which
|
||||||
|
// hands a probe a free classifier.
|
||||||
|
func TestMalformedAndMissingIDsAreIndistinguishable(t *testing.T) {
|
||||||
|
h, _ := testApp(t)
|
||||||
|
|
||||||
|
unknown, err := secret.NewID()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var bodies []string
|
||||||
|
for _, id := range []string{
|
||||||
|
unknown.Value(), // well-formed, never stored
|
||||||
|
strings.Repeat("A", 43), // well-formed, never minted
|
||||||
|
"short", // wrong length
|
||||||
|
"!!!", // not base64url
|
||||||
|
} {
|
||||||
|
code, body, raw := do(t, h, http.MethodPost, "/api/secrets/"+id+"/reveal", "")
|
||||||
|
if code != http.StatusGone {
|
||||||
|
t.Fatalf("reveal(%q) = %d, want 410: %s", id, code, raw)
|
||||||
|
}
|
||||||
|
errObj, _ := body["error"].(map[string]any)
|
||||||
|
if errObj["code"] != "gone" {
|
||||||
|
t.Fatalf("reveal(%q) code = %v, want \"gone\"", id, errObj["code"])
|
||||||
|
}
|
||||||
|
bodies = append(bodies, errObj["message"].(string))
|
||||||
|
}
|
||||||
|
for i := range bodies {
|
||||||
|
if bodies[i] != bodies[0] {
|
||||||
|
t.Fatalf("the gone message differs between causes (%q vs %q) — they must be identical",
|
||||||
|
bodies[0], bodies[i])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCreateRejectsWhatItCannotStore(t *testing.T) {
|
||||||
|
h, _ := testApp(t)
|
||||||
|
|
||||||
|
for _, tc := range []struct {
|
||||||
|
name string
|
||||||
|
body string
|
||||||
|
code string
|
||||||
|
}{
|
||||||
|
{"empty ciphertext", `{"ciphertext":"","ttl_seconds":3600}`, reasonEmpty},
|
||||||
|
{"not base64url", `{"ciphertext":"!!!!","ttl_seconds":3600}`, reasonInvalid},
|
||||||
|
{"oversized", `{"ciphertext":"` + strings.Repeat("A", secret.MaxCiphertextBytes+1) + `","ttl_seconds":3600}`, reasonTooLarge},
|
||||||
|
{"ttl too short", `{"ciphertext":"` + ciphertext("x") + `","ttl_seconds":60}`, reasonTTL},
|
||||||
|
{"ttl too long", `{"ciphertext":"` + ciphertext("x") + `","ttl_seconds":2592000}`, reasonTTL},
|
||||||
|
{"negative ttl", `{"ciphertext":"` + ciphertext("x") + `","ttl_seconds":-1}`, reasonTTL},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
code, body, raw := do(t, h, http.MethodPost, "/api/secrets", tc.body)
|
||||||
|
if code != http.StatusUnprocessableEntity {
|
||||||
|
t.Fatalf("create = %d, want 422: %s", code, raw)
|
||||||
|
}
|
||||||
|
errObj, _ := body["error"].(map[string]any)
|
||||||
|
if errObj["code"] != tc.code {
|
||||||
|
t.Fatalf("error code = %v, want %q", errObj["code"], tc.code)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestOmittedTTLGetsTheDefault(t *testing.T) {
|
||||||
|
h, _ := testApp(t)
|
||||||
|
code, body, raw := do(t, h, http.MethodPost, "/api/secrets",
|
||||||
|
`{"ciphertext":"`+ciphertext("x")+`"}`)
|
||||||
|
if code != http.StatusCreated {
|
||||||
|
t.Fatalf("create without ttl_seconds = %d, want 201: %s", code, raw)
|
||||||
|
}
|
||||||
|
if got := body["ttl_seconds"].(float64); int(got) != int(secret.DefaultTTL.Seconds()) {
|
||||||
|
t.Fatalf("ttl_seconds = %v, want the default %v", got, secret.DefaultTTL.Seconds())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// There must be no way to hand hush a plaintext secret. If a field like
|
||||||
|
// "secret" or "plaintext" were ever accepted, the server would become able to
|
||||||
|
// read secrets and the guarantee in the README would quietly become a promise
|
||||||
|
// about our conduct instead of a property of the design.
|
||||||
|
func TestThereIsNoPlaintextIntakeField(t *testing.T) {
|
||||||
|
h, _ := testApp(t)
|
||||||
|
for _, body := range []string{
|
||||||
|
`{"secret":"hunter2","ttl_seconds":3600}`,
|
||||||
|
`{"plaintext":"hunter2","ttl_seconds":3600}`,
|
||||||
|
`{"ciphertext":"` + ciphertext("x") + `","plaintext":"hunter2"}`,
|
||||||
|
} {
|
||||||
|
code, _, raw := do(t, h, http.MethodPost, "/api/secrets", body)
|
||||||
|
if code == http.StatusCreated {
|
||||||
|
t.Fatalf("create accepted a plaintext field: %s -> %s", body, raw)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The client-side crypto IS the product. If either page stops shipping it, the
|
||||||
|
// service silently becomes a plaintext store or a broken reader, and every
|
||||||
|
// other test here would still pass.
|
||||||
|
func TestPagesShipTheClientSideCrypto(t *testing.T) {
|
||||||
|
h, _ := testApp(t)
|
||||||
|
|
||||||
|
_, _, create := do(t, h, http.MethodGet, "/", "")
|
||||||
|
for _, want := range []string{
|
||||||
|
"AES-GCM", // the cipher
|
||||||
|
"crypto.subtle", // in the browser, not on the server
|
||||||
|
"generateKey", // the key is minted client-side
|
||||||
|
`"#" + sealed.key`, // and leaves only in the fragment
|
||||||
|
"/api/secrets", // ciphertext is what gets posted
|
||||||
|
} {
|
||||||
|
if !strings.Contains(create, want) {
|
||||||
|
t.Fatalf("the create page no longer contains %q — check it still encrypts client-side", want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// The create page must never post a plaintext field.
|
||||||
|
for _, forbidden := range []string{`"plaintext"`, `"secret":`} {
|
||||||
|
if strings.Contains(create, forbidden) {
|
||||||
|
t.Fatalf("the create page mentions %s, which suggests it sends plaintext", forbidden)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
_, _, reveal := do(t, h, http.MethodGet, "/s/"+strings.Repeat("A", 43), "")
|
||||||
|
for _, want := range []string{
|
||||||
|
"location.hash", // the key comes from the fragment
|
||||||
|
"crypto.subtle", // decryption is client-side
|
||||||
|
"/reveal", // and only POST consumes
|
||||||
|
"history.replaceState", // the key is dropped from the address bar after
|
||||||
|
} {
|
||||||
|
if !strings.Contains(reveal, want) {
|
||||||
|
t.Fatalf("the reveal page no longer contains %q", want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPagesAreNotCacheable(t *testing.T) {
|
||||||
|
h, _ := testApp(t)
|
||||||
|
for _, path := range []string{"/", "/s/" + strings.Repeat("A", 43)} {
|
||||||
|
r := httptest.NewRequest(http.MethodGet, path, nil)
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
h.ServeHTTP(w, r)
|
||||||
|
// A cached reveal page in a shared proxy is a copy of a one-time URL.
|
||||||
|
if cc := w.Header().Get("Cache-Control"); !strings.Contains(cc, "no-store") {
|
||||||
|
t.Fatalf("%s Cache-Control = %q, want no-store", path, cc)
|
||||||
|
}
|
||||||
|
if ref := w.Header().Get("Referrer-Policy"); ref != "no-referrer" {
|
||||||
|
t.Fatalf("%s Referrer-Policy = %q, want no-referrer — a click could otherwise leak the URL", path, ref)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
166
cmd/hushd/main.go
Normal file
166
cmd/hushd/main.go
Normal file
@ -0,0 +1,166 @@
|
|||||||
|
// Command hushd serves hush: one-time secret links.
|
||||||
|
//
|
||||||
|
// It stores ciphertext it cannot read. The encryption key is generated in the
|
||||||
|
// browser and travels only in the URL fragment, which browsers never transmit.
|
||||||
|
// See docs/ARCHITECTURE.md.
|
||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"os"
|
||||||
|
"os/signal"
|
||||||
|
"syscall"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/orchard9/go-chassis/chassis"
|
||||||
|
"github.com/orchard9/go-chassis/logging"
|
||||||
|
|
||||||
|
"github.com/orchard9/hush/internal/secret"
|
||||||
|
"github.com/orchard9/hush/internal/store"
|
||||||
|
"github.com/orchard9/hush/internal/web"
|
||||||
|
)
|
||||||
|
|
||||||
|
// service is the log corpus's `service` value and the metrics prefix. It is a
|
||||||
|
// member of a closed enum shared with every other orchard9 emitter — the
|
||||||
|
// cluster's Vector sink indexes `service` as a stream field, so a new value is
|
||||||
|
// a coordinated decision, not a free string.
|
||||||
|
const service = "hush"
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
if err := run(); err != nil {
|
||||||
|
// Boot failures go to stderr as well as the log: if logging itself is
|
||||||
|
// what failed, the process must still say why before exiting.
|
||||||
|
fmt.Fprintf(os.Stderr, "hushd: %v\n", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func run() error {
|
||||||
|
cfg, cfgErr := loadConfig()
|
||||||
|
|
||||||
|
// The logger is built before the config error is returned, so a
|
||||||
|
// misconfiguration is REPORTED through the same corpus as everything else
|
||||||
|
// rather than dying silently. cfg.Env is the zero value on failure, which
|
||||||
|
// logging.Env normalises.
|
||||||
|
log := logging.New(logging.Config{Service: service, Env: logging.Env(cfg.Env)})
|
||||||
|
logging.SetFallback(log)
|
||||||
|
|
||||||
|
if cfgErr != nil {
|
||||||
|
// critical + category boot is the fail-closed contract: this is a
|
||||||
|
// refusal to serve, which is the only thing critical is for.
|
||||||
|
logging.Critical(context.Background(), log, "boot.config_invalid",
|
||||||
|
"category", "boot", "error_type", "config_invalid", "error_msg", cfgErr.Error())
|
||||||
|
return cfgErr
|
||||||
|
}
|
||||||
|
|
||||||
|
pages, err := web.New()
|
||||||
|
if err != nil {
|
||||||
|
logging.Critical(context.Background(), log, "boot.templates_invalid",
|
||||||
|
"category", "boot", "error_type", "template_invalid", "error_msg", err.Error())
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
rdb, err := store.NewRedis(cfg.RedisURL)
|
||||||
|
if err != nil {
|
||||||
|
// The URL is malformed — a config defect, not an outage. Refuse to boot
|
||||||
|
// rather than serve a store that can never work.
|
||||||
|
logging.Critical(context.Background(), log, "boot.store_invalid",
|
||||||
|
"category", "boot", "error_type", "store_invalid", "error_msg", err.Error())
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer func() { _ = rdb.Close() }()
|
||||||
|
|
||||||
|
metrics := newMetrics()
|
||||||
|
metrics.Prime()
|
||||||
|
|
||||||
|
srv := &Server{cfg: cfg, store: rdb, pages: pages, metrics: metrics}
|
||||||
|
|
||||||
|
app := chassis.New(chassis.Config{
|
||||||
|
Service: service,
|
||||||
|
Env: cfg.Env,
|
||||||
|
Addr: fmt.Sprintf(":%d", cfg.Port),
|
||||||
|
// 128 KiB caps the body at the edge so an oversized upload is refused
|
||||||
|
// before a handler allocates it. The 64 KiB ciphertext cap is enforced
|
||||||
|
// separately in the handler; this is the outer bound including JSON
|
||||||
|
// framing.
|
||||||
|
MaxBodyBytes: 128 * 1024,
|
||||||
|
AllowOrigins: cfg.AllowOrigins,
|
||||||
|
Collectors: metrics.Collectors(),
|
||||||
|
}, log)
|
||||||
|
|
||||||
|
// Readiness doubles as the store_up gauge, so the alert on Redis
|
||||||
|
// reachability reads the same probe Kubernetes uses to route traffic —
|
||||||
|
// rather than a second, separately-drifting health notion.
|
||||||
|
app.Health("redis", func(ctx context.Context) error {
|
||||||
|
err := rdb.Ping(ctx)
|
||||||
|
if err != nil {
|
||||||
|
metrics.StoreUp.Set(0)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
metrics.StoreUp.Set(1)
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
|
||||||
|
// Pages: no storage access, no rate limit. A link previewer hitting either
|
||||||
|
// of these must be free and harmless.
|
||||||
|
app.Get("/", srv.handleCreatePage)
|
||||||
|
app.Get("/s/{id}", srv.handleRevealPage)
|
||||||
|
|
||||||
|
limiter := &redisLimiter{store: rdb, cfg: cfg, metrics: metrics}
|
||||||
|
app.Route("/api", func(r *chassis.Router) {
|
||||||
|
// Rate limit applies to create only. A reveal succeeds at most once per
|
||||||
|
// secret by construction, so there is nothing to throttle, and
|
||||||
|
// throttling would let one busy NAT block a colleague's delivery.
|
||||||
|
createMW := []chassis.Middleware{chassis.RateLimit(limiter, chassis.RateKey(cfg.TrustedProxyHops))}
|
||||||
|
if cfg.RequireAuthToCreate {
|
||||||
|
// The escape hatch for abuse. Reveal stays anonymous either way:
|
||||||
|
// the recipient is external and holds no credential.
|
||||||
|
createMW = append(createMW, chassis.RequireAuth(chassis.NewStaticToken(cfg.CreateToken)))
|
||||||
|
}
|
||||||
|
r.Post("/secrets", srv.handleCreate, createMW...)
|
||||||
|
r.Post("/secrets/{id}/reveal", srv.handleReveal)
|
||||||
|
})
|
||||||
|
|
||||||
|
log.Info("boot.ready", "category", "boot",
|
||||||
|
"port", cfg.Port,
|
||||||
|
"rate_limit_creates", cfg.RateLimitCreates,
|
||||||
|
"rate_limit_window_seconds", int64(cfg.RateLimitWindow.Seconds()),
|
||||||
|
"max_ciphertext_bytes", secret.MaxCiphertextBytes,
|
||||||
|
"default_ttl_seconds", int64(secret.DefaultTTL.Seconds()),
|
||||||
|
"require_auth_to_create", cfg.RequireAuthToCreate)
|
||||||
|
|
||||||
|
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
|
||||||
|
defer stop()
|
||||||
|
if err := app.Run(ctx); err != nil && !errors.Is(err, http.ErrServerClosed) {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// redisLimiter adapts the Redis fixed-window counter to chassis.RateLimiter and
|
||||||
|
// counts refusals, which is the abuse signal the alert rules watch.
|
||||||
|
type redisLimiter struct {
|
||||||
|
store *store.Redis
|
||||||
|
cfg Config
|
||||||
|
metrics *Metrics
|
||||||
|
}
|
||||||
|
|
||||||
|
func (l *redisLimiter) Allow(ctx context.Context, key string) (bool, time.Duration, error) {
|
||||||
|
ok, retry, err := l.store.AllowN(ctx, key, l.cfg.RateLimitCreates, l.cfg.RateLimitWindow)
|
||||||
|
if err != nil {
|
||||||
|
// Redis is unreachable. FAIL OPEN on the limiter specifically: the
|
||||||
|
// alternative is that a Redis blip turns the rate limiter into a total
|
||||||
|
// outage of a service whose whole job is delivering credentials during
|
||||||
|
// incidents. The store call immediately after will fail anyway if Redis
|
||||||
|
// is really down, so this cannot silently accept a secret it cannot
|
||||||
|
// store — it just refuses at the right layer, with the right error.
|
||||||
|
return true, 0, nil
|
||||||
|
}
|
||||||
|
if !ok {
|
||||||
|
l.metrics.Limited.Inc()
|
||||||
|
}
|
||||||
|
return ok, retry, nil
|
||||||
|
}
|
||||||
84
cmd/hushd/metrics.go
Normal file
84
cmd/hushd/metrics.go
Normal file
@ -0,0 +1,84 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import "github.com/prometheus/client_golang/prometheus"
|
||||||
|
|
||||||
|
// Metrics are hush's own series, registered on the chassis's private registry
|
||||||
|
// via chassis.Config.Collectors so /metrics is one scrape.
|
||||||
|
//
|
||||||
|
// Cardinality rule applied throughout: NO label ever carries a secret id, a
|
||||||
|
// client IP, or a URL path. A metric label lands in the time series index and
|
||||||
|
// stays there, so a per-secret label would be both a capability leak and an
|
||||||
|
// unbounded index. Every label below is a closed enum.
|
||||||
|
type Metrics struct {
|
||||||
|
Created prometheus.Counter
|
||||||
|
Revealed *prometheus.CounterVec // result: ok | gone
|
||||||
|
Rejected *prometheus.CounterVec // reason: closed enum, see rejection reasons
|
||||||
|
Bytes prometheus.Histogram
|
||||||
|
StoreUp prometheus.Gauge
|
||||||
|
Limited prometheus.Counter
|
||||||
|
}
|
||||||
|
|
||||||
|
// Rejection reasons. A closed set, mirrored by the API error codes, so
|
||||||
|
// `hush_secrets_rejected_total{reason="ciphertext_too_large"}` and the 422 a
|
||||||
|
// caller saw are the same vocabulary.
|
||||||
|
const (
|
||||||
|
reasonTooLarge = "ciphertext_too_large"
|
||||||
|
reasonInvalid = "ciphertext_invalid"
|
||||||
|
reasonEmpty = "ciphertext_empty"
|
||||||
|
reasonTTL = "ttl_out_of_range"
|
||||||
|
reasonMalformed = "malformed_request"
|
||||||
|
)
|
||||||
|
|
||||||
|
func newMetrics() *Metrics {
|
||||||
|
return &Metrics{
|
||||||
|
Created: prometheus.NewCounter(prometheus.CounterOpts{
|
||||||
|
Name: "hush_secrets_created_total",
|
||||||
|
Help: "Secrets accepted and stored.",
|
||||||
|
}),
|
||||||
|
Revealed: prometheus.NewCounterVec(prometheus.CounterOpts{
|
||||||
|
Name: "hush_secrets_revealed_total",
|
||||||
|
Help: "Reveal attempts by outcome. `gone` covers already-revealed, expired, never-existed and evicted, which the service does not distinguish.",
|
||||||
|
}, []string{"result"}),
|
||||||
|
Rejected: prometheus.NewCounterVec(prometheus.CounterOpts{
|
||||||
|
Name: "hush_secrets_rejected_total",
|
||||||
|
Help: "Create attempts refused by validation, by reason.",
|
||||||
|
}, []string{"reason"}),
|
||||||
|
Bytes: prometheus.NewHistogram(prometheus.HistogramOpts{
|
||||||
|
Name: "hush_secret_ciphertext_bytes",
|
||||||
|
Help: "Size of stored ciphertext.",
|
||||||
|
// Hand-picked rather than DefBuckets (which tops out at 10): this
|
||||||
|
// measures bytes up to a 64 KiB cap, so the buckets straddle
|
||||||
|
// credential-sized (hundreds of bytes) through the limit.
|
||||||
|
Buckets: []float64{256, 1024, 4096, 16384, 65536},
|
||||||
|
}),
|
||||||
|
StoreUp: prometheus.NewGauge(prometheus.GaugeOpts{
|
||||||
|
Name: "hush_store_up",
|
||||||
|
Help: "1 when the last readiness probe reached Redis, 0 otherwise.",
|
||||||
|
}),
|
||||||
|
Limited: prometheus.NewCounter(prometheus.CounterOpts{
|
||||||
|
Name: "hush_rate_limited_total",
|
||||||
|
Help: "Requests refused by the create rate limit. A sustained rise is the abuse signal.",
|
||||||
|
}),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Collectors is what gets handed to chassis.Config.Collectors. The chassis owns
|
||||||
|
// a private registry and exposes no accessor, so this slice is the only way
|
||||||
|
// hush's series reach /metrics.
|
||||||
|
func (m *Metrics) Collectors() []prometheus.Collector {
|
||||||
|
return []prometheus.Collector{m.Created, m.Revealed, m.Rejected, m.Bytes, m.StoreUp, m.Limited}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Prime initialises the label combinations that alerting queries reference.
|
||||||
|
//
|
||||||
|
// Without this, `rate(hush_secrets_revealed_total{result="gone"}[15m])` returns
|
||||||
|
// no data until the first `gone` ever happens, and an alert written against a
|
||||||
|
// missing series is an alert that cannot fire. Priming makes the series exist
|
||||||
|
// at zero from boot.
|
||||||
|
func (m *Metrics) Prime() {
|
||||||
|
m.Revealed.WithLabelValues("ok")
|
||||||
|
m.Revealed.WithLabelValues("gone")
|
||||||
|
for _, r := range []string{reasonTooLarge, reasonInvalid, reasonEmpty, reasonTTL, reasonMalformed} {
|
||||||
|
m.Rejected.WithLabelValues(r)
|
||||||
|
}
|
||||||
|
}
|
||||||
249
deployments/k8s/hush.yaml
Normal file
249
deployments/k8s/hush.yaml
Normal file
@ -0,0 +1,249 @@
|
|||||||
|
# hush on the orchard9 k3s cluster. Apply this BEFORE the first push, because
|
||||||
|
# the pipeline's deploy step runs `kubectl set image` and needs a Deployment to
|
||||||
|
# set it on.
|
||||||
|
#
|
||||||
|
# KUBECONFIG=~/.kube/orchard9-k3sf.yaml kubectl apply -f deployments/k8s/hush.yaml
|
||||||
|
#
|
||||||
|
# Everything hush needs is here: the credential, the workload, the Service, the
|
||||||
|
# network boundary and the public route.
|
||||||
|
---
|
||||||
|
# The Redis credential. hush connects as its OWN Redis ACL user, scoped to
|
||||||
|
# `~hush:*` with a minimal command set (+ping +set +getdel +incr +pexpire
|
||||||
|
# +select), so a bug in hush cannot read or write another tenant's keys and a
|
||||||
|
# compromise of hush cannot enumerate the keyspace.
|
||||||
|
#
|
||||||
|
# `+getdel` is the one that needs saying out loud: no other service's ACL user
|
||||||
|
# has it, because no other service needs an atomic read-and-destroy. Omitting it
|
||||||
|
# makes every reveal fail NOPERM while creates keep working — a service that
|
||||||
|
# accepts secrets and cannot deliver them.
|
||||||
|
apiVersion: external-secrets.io/v1
|
||||||
|
kind: ExternalSecret
|
||||||
|
metadata:
|
||||||
|
name: hush-credentials
|
||||||
|
namespace: projects
|
||||||
|
spec:
|
||||||
|
refreshInterval: 1h
|
||||||
|
secretStoreRef:
|
||||||
|
name: gcp-secret-manager
|
||||||
|
kind: ClusterSecretStore
|
||||||
|
target:
|
||||||
|
name: hush-credentials
|
||||||
|
creationPolicy: Owner
|
||||||
|
dataFrom:
|
||||||
|
- extract:
|
||||||
|
key: k3sf-hush-credentials
|
||||||
|
---
|
||||||
|
apiVersion: apps/v1
|
||||||
|
kind: Deployment
|
||||||
|
metadata:
|
||||||
|
name: hush
|
||||||
|
namespace: projects
|
||||||
|
labels:
|
||||||
|
app: hush
|
||||||
|
spec:
|
||||||
|
# One replica is sufficient and not a bottleneck: every request is a single
|
||||||
|
# Redis round trip and all state lives in Redis, so this scales horizontally
|
||||||
|
# whenever it needs to. The rate limiter is already Redis-backed for exactly
|
||||||
|
# that reason.
|
||||||
|
replicas: 1
|
||||||
|
selector:
|
||||||
|
matchLabels:
|
||||||
|
app: hush
|
||||||
|
strategy:
|
||||||
|
rollingUpdate:
|
||||||
|
maxUnavailable: 0
|
||||||
|
maxSurge: 1
|
||||||
|
template:
|
||||||
|
metadata:
|
||||||
|
labels:
|
||||||
|
app: hush
|
||||||
|
annotations:
|
||||||
|
# Metrics scraping. All three are required, and the port must EQUAL a
|
||||||
|
# declared containerPort as a string — vmagent's relabel config uses
|
||||||
|
# `keepequal`, so a mismatch silently drops the target with no error
|
||||||
|
# and no up=0 to alert on.
|
||||||
|
prometheus.io/scrape: "true"
|
||||||
|
prometheus.io/port: "18500"
|
||||||
|
prometheus.io/path: /metrics
|
||||||
|
spec:
|
||||||
|
securityContext:
|
||||||
|
runAsNonRoot: true
|
||||||
|
runAsUser: 65532
|
||||||
|
runAsGroup: 65532
|
||||||
|
fsGroup: 65532
|
||||||
|
seccompProfile:
|
||||||
|
type: RuntimeDefault
|
||||||
|
# The chassis drains in two phases — flip readiness to 503, wait 5s for
|
||||||
|
# the load balancer to notice, then shut down with a 25s deadline. That is
|
||||||
|
# 30s, so the grace period must exceed it or the kubelet SIGKILLs mid-drain
|
||||||
|
# and in-flight reveals are lost.
|
||||||
|
terminationGracePeriodSeconds: 45
|
||||||
|
containers:
|
||||||
|
- name: hushd
|
||||||
|
# `bootstrap` is a placeholder, replaced by CI's `kubectl set image`
|
||||||
|
# with the commit-SHA tag on the first push. It is deliberately NOT
|
||||||
|
# `:latest`: the cluster's stable-controller-images admission policy
|
||||||
|
# refuses latest/main/master/dev/edge/canary/nightly/snapshot, because
|
||||||
|
# a floating tag cannot pin a rollback. Until the first pipeline run
|
||||||
|
# this image does not exist and the pod sits in ImagePullBackOff,
|
||||||
|
# which is the expected bootstrap state.
|
||||||
|
image: registry.threesix.ai/hush/api:bootstrap
|
||||||
|
ports:
|
||||||
|
- name: http
|
||||||
|
containerPort: 18500
|
||||||
|
env:
|
||||||
|
- name: APP_ENV
|
||||||
|
value: prod
|
||||||
|
- name: HUSH_PORT
|
||||||
|
value: "18500"
|
||||||
|
- name: REDIS_URL
|
||||||
|
valueFrom:
|
||||||
|
secretKeyRef:
|
||||||
|
name: hush-credentials
|
||||||
|
key: REDIS_URL
|
||||||
|
# One Traefik hop sits in front, so the client IP is the last entry
|
||||||
|
# of X-Forwarded-For. Too high lets a caller spoof their IP past the
|
||||||
|
# rate limit; too low rate-limits the ingress itself and throttles
|
||||||
|
# every user together.
|
||||||
|
- name: HUSH_TRUSTED_PROXY_HOPS
|
||||||
|
value: "1"
|
||||||
|
- name: HUSH_RATE_LIMIT_CREATES
|
||||||
|
value: "30"
|
||||||
|
- name: HUSH_RATE_LIMIT_WINDOW
|
||||||
|
value: 10m
|
||||||
|
securityContext:
|
||||||
|
allowPrivilegeEscalation: false
|
||||||
|
readOnlyRootFilesystem: true
|
||||||
|
capabilities:
|
||||||
|
drop: ["ALL"]
|
||||||
|
# Liveness stays 200 during the drain by design, so this restarts the
|
||||||
|
# pod only when the process is genuinely wedged — never merely because
|
||||||
|
# it is shutting down or because Redis is down.
|
||||||
|
livenessProbe:
|
||||||
|
httpGet: { path: /healthz, port: http }
|
||||||
|
initialDelaySeconds: 3
|
||||||
|
periodSeconds: 10
|
||||||
|
timeoutSeconds: 2
|
||||||
|
failureThreshold: 3
|
||||||
|
# Readiness pings Redis. A Redis outage takes hush out of the Service
|
||||||
|
# rather than leaving it to serve 500s from a pod the load balancer
|
||||||
|
# still trusts.
|
||||||
|
readinessProbe:
|
||||||
|
httpGet: { path: /readyz, port: http }
|
||||||
|
initialDelaySeconds: 2
|
||||||
|
periodSeconds: 5
|
||||||
|
timeoutSeconds: 3
|
||||||
|
failureThreshold: 2
|
||||||
|
resources:
|
||||||
|
requests: { memory: 32Mi, cpu: 20m }
|
||||||
|
# 128Mi is generous for a service whose largest allocation is a
|
||||||
|
# 64 KiB ciphertext, and leaves headroom for Go's GC pacing.
|
||||||
|
limits: { memory: 128Mi, cpu: 500m }
|
||||||
|
---
|
||||||
|
apiVersion: v1
|
||||||
|
kind: Service
|
||||||
|
metadata:
|
||||||
|
name: hush
|
||||||
|
namespace: projects
|
||||||
|
labels:
|
||||||
|
app: hush
|
||||||
|
spec:
|
||||||
|
selector:
|
||||||
|
app: hush
|
||||||
|
ports:
|
||||||
|
- name: http
|
||||||
|
port: 80
|
||||||
|
targetPort: http
|
||||||
|
---
|
||||||
|
# Default-deny both directions, then open exactly what hush needs. Written as
|
||||||
|
# one policy because the ingress and egress rules are a single statement about
|
||||||
|
# this pod: Traefik in, DNS and Redis out, nothing else.
|
||||||
|
apiVersion: networking.k8s.io/v1
|
||||||
|
kind: NetworkPolicy
|
||||||
|
metadata:
|
||||||
|
name: hush
|
||||||
|
namespace: projects
|
||||||
|
spec:
|
||||||
|
podSelector:
|
||||||
|
matchLabels:
|
||||||
|
app: hush
|
||||||
|
policyTypes: [Ingress, Egress]
|
||||||
|
ingress:
|
||||||
|
# Public traffic, via Traefik only.
|
||||||
|
- from:
|
||||||
|
- namespaceSelector:
|
||||||
|
matchLabels:
|
||||||
|
kubernetes.io/metadata.name: kube-system
|
||||||
|
podSelector:
|
||||||
|
matchLabels:
|
||||||
|
app.kubernetes.io/name: traefik
|
||||||
|
ports:
|
||||||
|
- { protocol: TCP, port: 18500 }
|
||||||
|
# vmagent scrapes /metrics on the pod IP directly. Without this rule the
|
||||||
|
# target is discovered and every scrape is connection-refused, which shows
|
||||||
|
# up as up=0 and fires ScrapeTargetDown rather than as a policy error.
|
||||||
|
- from:
|
||||||
|
- namespaceSelector:
|
||||||
|
matchLabels:
|
||||||
|
kubernetes.io/metadata.name: observability
|
||||||
|
ports:
|
||||||
|
- { protocol: TCP, port: 18500 }
|
||||||
|
egress:
|
||||||
|
# DNS. Redis is a headless Service, so its name resolves to a pod IP that
|
||||||
|
# changes when Redis restarts — this must be re-resolvable, not cached.
|
||||||
|
- to:
|
||||||
|
- namespaceSelector:
|
||||||
|
matchLabels:
|
||||||
|
kubernetes.io/metadata.name: kube-system
|
||||||
|
ports:
|
||||||
|
- { protocol: UDP, port: 53 }
|
||||||
|
- { protocol: TCP, port: 53 }
|
||||||
|
# Redis, and nothing else. hush makes no other outbound connection: it does
|
||||||
|
# not fetch, does not call a provider, and does not ship its own logs
|
||||||
|
# (Vector reads stdout off the node filesystem).
|
||||||
|
- to:
|
||||||
|
- namespaceSelector:
|
||||||
|
matchLabels:
|
||||||
|
kubernetes.io/metadata.name: databases
|
||||||
|
ports:
|
||||||
|
- { protocol: TCP, port: 6379 }
|
||||||
|
---
|
||||||
|
apiVersion: networking.k8s.io/v1
|
||||||
|
kind: Ingress
|
||||||
|
metadata:
|
||||||
|
name: hush
|
||||||
|
namespace: projects
|
||||||
|
annotations:
|
||||||
|
cert-manager.io/cluster-issuer: letsencrypt-prod
|
||||||
|
spec:
|
||||||
|
tls:
|
||||||
|
- hosts: [hush.threesix.ai]
|
||||||
|
secretName: hush-tls
|
||||||
|
rules:
|
||||||
|
- host: hush.threesix.ai
|
||||||
|
http:
|
||||||
|
paths:
|
||||||
|
# The paths are enumerated deliberately, and `/` is Exact rather than
|
||||||
|
# Prefix. A Prefix `/` would route EVERYTHING, publishing /metrics,
|
||||||
|
# /healthz and /readyz to the internet. /metrics leaks how many
|
||||||
|
# secrets are created and when; the others are just noise. Enumerating
|
||||||
|
# instead means Traefik 404s them at the edge and there is no
|
||||||
|
# basic-auth middleware to maintain and get wrong.
|
||||||
|
- path: /
|
||||||
|
pathType: Exact
|
||||||
|
backend:
|
||||||
|
service:
|
||||||
|
name: hush
|
||||||
|
port: { name: http }
|
||||||
|
- path: /s/
|
||||||
|
pathType: Prefix
|
||||||
|
backend:
|
||||||
|
service:
|
||||||
|
name: hush
|
||||||
|
port: { name: http }
|
||||||
|
- path: /api/
|
||||||
|
pathType: Prefix
|
||||||
|
backend:
|
||||||
|
service:
|
||||||
|
name: hush
|
||||||
|
port: { name: http }
|
||||||
180
docs/ARCHITECTURE.md
Normal file
180
docs/ARCHITECTURE.md
Normal file
@ -0,0 +1,180 @@
|
|||||||
|
# Architecture
|
||||||
|
|
||||||
|
One Go binary, one Redis key per secret, no database. The interesting parts are
|
||||||
|
all about *where the key sits* and *what destroys the ciphertext*.
|
||||||
|
|
||||||
|
## Components
|
||||||
|
|
||||||
|
```
|
||||||
|
browser ──► Traefik ──► hushd (projects ns, 1 replica) ──► Redis (databases ns, db 5)
|
||||||
|
│
|
||||||
|
├─ stdout JSON ──► Vector (DaemonSet) ──► VictoriaLogs
|
||||||
|
└─ /metrics ─────► vmagent ──► vmsingle ──► vmalert ──► Alertmanager
|
||||||
|
```
|
||||||
|
|
||||||
|
`hushd` holds no durable state. Redis holds every secret and nothing else.
|
||||||
|
|
||||||
|
## The zero-knowledge split
|
||||||
|
|
||||||
|
```
|
||||||
|
create: plaintext ──[AES-256-GCM in the browser]──► ciphertext ──► POST /api/secrets
|
||||||
|
key ──────────────────────────────────────► URL fragment, never sent
|
||||||
|
|
||||||
|
reveal: POST /api/secrets/{id}/reveal ──► ciphertext ──[decrypt in browser]──► plaintext
|
||||||
|
key read from location.hash
|
||||||
|
```
|
||||||
|
|
||||||
|
The fragment is the whole trick. Per RFC 3986 §3.5 the fragment is a client-side
|
||||||
|
construct: browsers do not put it in the request line, so it never reaches
|
||||||
|
Traefik, hushd, Redis, an access log, or a proxy. hushd receives a 256-bit AES-GCM
|
||||||
|
ciphertext with a prepended 96-bit nonce and has no key material at any point.
|
||||||
|
|
||||||
|
Consequences worth stating plainly:
|
||||||
|
|
||||||
|
- A Redis dump is worthless. A hushd core dump is worthless. Our own operators
|
||||||
|
cannot read a secret, and neither can anyone who compromises the service.
|
||||||
|
- **A URL in someone's browser history contains the key.** The fragment is not
|
||||||
|
transmitted, but it *is* stored locally. This is the residual exposure and it
|
||||||
|
is why TTLs are short.
|
||||||
|
- There is deliberately no server-side-encryption fallback mode. A second mode
|
||||||
|
where the server sees plaintext would mean nobody could tell, from a link,
|
||||||
|
which guarantee they had.
|
||||||
|
|
||||||
|
## Why GET never touches storage
|
||||||
|
|
||||||
|
`GET /s/{id}` renders a static page and makes zero calls to Redis. It does not
|
||||||
|
even check whether the id exists.
|
||||||
|
|
||||||
|
That is not laziness — it is the only way to be correct in the presence of link
|
||||||
|
previewers. Slack, Teams, WhatsApp, iMessage and Outlook Safe Links fetch URLs
|
||||||
|
before a human sees them. Any design that destroys on `GET` destroys most secrets
|
||||||
|
in transit. Bot user-agent detection is a losing arms race; removing the
|
||||||
|
side effect from `GET` is not.
|
||||||
|
|
||||||
|
A secondary benefit: because `GET` does not look the id up, the reveal page cannot
|
||||||
|
leak whether an id exists. Existence is only ever answered by a `POST`, and that
|
||||||
|
answer is identical for missing, revealed and expired.
|
||||||
|
|
||||||
|
## Storage and destruction
|
||||||
|
|
||||||
|
One key per secret:
|
||||||
|
|
||||||
|
```
|
||||||
|
key hush:s:<id> id = 256 bits from crypto/rand, base64url (43 chars)
|
||||||
|
value <ciphertext> opaque bytes, ≤ 64 KiB
|
||||||
|
write SET key val EX <ttl> NX
|
||||||
|
read GETDEL key
|
||||||
|
```
|
||||||
|
|
||||||
|
`GETDEL` (Redis 6.2+; the cluster runs 7.4.8) is atomic, which is the reason it
|
||||||
|
is used instead of `GET` followed by `DEL`. Two people opening the same link
|
||||||
|
simultaneously cannot both receive the plaintext — exactly one `GETDEL` returns
|
||||||
|
the value and the other returns nil. A `GET`+`DEL` pair has a window between the
|
||||||
|
two commands where both callers succeed, and for a one-time secret that window is
|
||||||
|
the entire product.
|
||||||
|
|
||||||
|
`NX` on write means an id collision never overwrites an existing secret. At 256
|
||||||
|
bits of entropy a collision will not happen; the flag costs nothing and turns a
|
||||||
|
theoretical silent overwrite into a visible error.
|
||||||
|
|
||||||
|
TTL is Redis-native, so expiry needs no sweeper, no cron and nothing to wedge.
|
||||||
|
|
||||||
|
### Eviction is an availability risk, not a confidentiality one
|
||||||
|
|
||||||
|
The shared Redis runs `maxmemory-policy allkeys-lru` with `maxmemory 256MiB`.
|
||||||
|
Under memory pressure Redis may evict a hush key **before** its TTL fires. That
|
||||||
|
means a secret can become unavailable early.
|
||||||
|
|
||||||
|
It cannot become *more* available: eviction only ever deletes. So the failure mode
|
||||||
|
is "your recipient has to ask you again", never "the secret outlived its TTL" and
|
||||||
|
never "someone read it twice". For a secret courier that is the correct direction
|
||||||
|
to fail, and it is why `410 gone` deliberately does not distinguish causes — the
|
||||||
|
user-visible contract is identical either way.
|
||||||
|
|
||||||
|
Operationally this is watched via `HushRedisUnreachable` and the Redis memory
|
||||||
|
alerts, not by trying to tell eviction and reveal apart. See
|
||||||
|
[OPERATIONS.md](OPERATIONS.md).
|
||||||
|
|
||||||
|
## Identifiers and what gets logged
|
||||||
|
|
||||||
|
The id **is** the capability. Anyone holding it can reveal the secret, so it is
|
||||||
|
treated like a bearer token:
|
||||||
|
|
||||||
|
- Never logged. Not at debug, not in an error, not in a panic.
|
||||||
|
- The log correlation handle is `sid = sha256(id)[:12]` — enough to follow one
|
||||||
|
secret's create → reveal → gone across a corpus, useless for revealing it.
|
||||||
|
- Never in a metric label (that would put it in the time series index forever).
|
||||||
|
|
||||||
|
`internal/secret.ID.LogHandle()` is the only way to get a loggable form, and the
|
||||||
|
`ID` type does not implement `String()` or `MarshalText()`, so it cannot be
|
||||||
|
accidentally interpolated into a log line or JSON body. That is enforced by
|
||||||
|
`internal/secret/id_test.go`.
|
||||||
|
|
||||||
|
The chassis logger additionally redacts any field *named* `secret`, `token`,
|
||||||
|
`password`, `api_key`, `authorization` and friends. Field names here avoid those
|
||||||
|
words entirely (`ciphertext`, `sid`, `ttl_seconds`) so nothing depends on that
|
||||||
|
backstop.
|
||||||
|
|
||||||
|
## Request path
|
||||||
|
|
||||||
|
```
|
||||||
|
GET / create page (static HTML+JS, no storage access)
|
||||||
|
GET /s/{id} reveal page (static HTML+JS, no storage access)
|
||||||
|
POST /api/secrets store ciphertext rate limited
|
||||||
|
POST /api/secrets/{id}/reveal GETDEL, destroy, return once rate limited
|
||||||
|
GET /healthz liveness — 200 while draining
|
||||||
|
GET /readyz readiness — Redis PING, 503 while draining
|
||||||
|
GET /metrics Prometheus
|
||||||
|
```
|
||||||
|
|
||||||
|
Built on `github.com/orchard9/go-chassis`, which supplies routing, request ids,
|
||||||
|
the panic recovery envelope, RED metrics, secure headers, the two-phase drain,
|
||||||
|
and `/healthz`, `/readyz`, `/metrics`. hush contributes handlers, a store, a
|
||||||
|
rate limiter and templates — not a framework.
|
||||||
|
|
||||||
|
The public Ingress routes `/` (exact), `/s/` and `/api/` only. `/metrics`,
|
||||||
|
`/healthz` and `/readyz` share the port but are unreachable from the internet;
|
||||||
|
vmagent scrapes the pod IP directly. This is why there is no metrics basic-auth
|
||||||
|
middleware to maintain.
|
||||||
|
|
||||||
|
## Abuse posture
|
||||||
|
|
||||||
|
Create is anonymous by design, which makes the service a free blob host and a
|
||||||
|
phishing kit borrowing a `threesix.ai` name. Mitigations, all cheap:
|
||||||
|
|
||||||
|
| Control | Value |
|
||||||
|
|---|---|
|
||||||
|
| Ciphertext cap | 64 KiB, enforced before Redis |
|
||||||
|
| Request body cap | 128 KiB, enforced by the chassis at the edge |
|
||||||
|
| TTL clamp | 5m … 7d, out-of-range is a 422, not a silent clamp |
|
||||||
|
| Rate limit | 30 creates / 10 min / IP, Redis fixed-window |
|
||||||
|
| Id entropy | 256 bits — enumeration is not a threat model |
|
||||||
|
| No listing route | there is no way to ask "what secrets exist" |
|
||||||
|
| Identical `gone` | missing, revealed and expired are one response |
|
||||||
|
|
||||||
|
If it is ever abused, `HUSH_REQUIRE_AUTH=true` puts create behind the chassis
|
||||||
|
authenticator while leaving reveal anonymous — the asymmetry the design assumes.
|
||||||
|
Reveal must stay anonymous: the recipient is external and has no credential.
|
||||||
|
|
||||||
|
## Failure modes
|
||||||
|
|
||||||
|
| Failure | Behaviour |
|
||||||
|
|---|---|
|
||||||
|
| Redis down | `/readyz` 503, pod leaves the Service, creates and reveals 503. No secret is lost that was already written. |
|
||||||
|
| Redis evicts a key early | That link returns `410 gone`. Sender must re-send. |
|
||||||
|
| hushd restarts | Nothing lost; all state is in Redis. |
|
||||||
|
| Two simultaneous reveals | Exactly one wins, atomically. |
|
||||||
|
| Body over 128 KiB | 413 at the edge, never reaches a handler. |
|
||||||
|
| Ciphertext over 64 KiB | 422 `ciphertext_too_large`. |
|
||||||
|
| Malformed base64 | 422 `ciphertext_invalid`. hushd validates the encoding but cannot validate the plaintext. |
|
||||||
|
| Clock skew | TTL is Redis-relative, so skew between hushd and the browser cannot extend a secret's life. |
|
||||||
|
|
||||||
|
## What is deliberately absent
|
||||||
|
|
||||||
|
Accounts. Passphrases on top of the link. File uploads. Multi-read links. An
|
||||||
|
audit UI. Email delivery. Each is a real request and each doubles the surface.
|
||||||
|
|
||||||
|
The one with a genuine argument is **notify-on-read**: it confirms delivery and,
|
||||||
|
if it fires before the recipient says they opened it, that is a compromise
|
||||||
|
signal. It needs an email path, `notify` already exists to provide one, and it is
|
||||||
|
the first thing to add if hush proves useful.
|
||||||
21
go.mod
Normal file
21
go.mod
Normal file
@ -0,0 +1,21 @@
|
|||||||
|
module github.com/orchard9/hush
|
||||||
|
|
||||||
|
go 1.26.0
|
||||||
|
|
||||||
|
require (
|
||||||
|
github.com/orchard9/go-chassis v0.1.0
|
||||||
|
github.com/prometheus/client_golang v1.24.0
|
||||||
|
github.com/redis/go-redis/v9 v9.22.0
|
||||||
|
)
|
||||||
|
|
||||||
|
require (
|
||||||
|
github.com/beorn7/perks v1.0.1 // indirect
|
||||||
|
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
||||||
|
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
|
||||||
|
github.com/prometheus/client_model v0.6.2 // indirect
|
||||||
|
github.com/prometheus/common v0.70.0 // indirect
|
||||||
|
github.com/prometheus/procfs v0.21.1 // indirect
|
||||||
|
go.uber.org/atomic v1.11.0 // indirect
|
||||||
|
golang.org/x/sys v0.47.0 // indirect
|
||||||
|
google.golang.org/protobuf v1.36.11 // indirect
|
||||||
|
)
|
||||||
50
go.sum
Normal file
50
go.sum
Normal file
@ -0,0 +1,50 @@
|
|||||||
|
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
|
||||||
|
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
|
||||||
|
github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs=
|
||||||
|
github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c=
|
||||||
|
github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA=
|
||||||
|
github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0=
|
||||||
|
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
|
||||||
|
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
|
||||||
|
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||||
|
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||||
|
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
||||||
|
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
|
||||||
|
github.com/klauspost/compress v1.19.0 h1:sXLILfc9jV2QYWkzFOPWStmcUVH2RHEB1JCdY2oVvCQ=
|
||||||
|
github.com/klauspost/compress v1.19.0/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
|
||||||
|
github.com/klauspost/cpuid/v2 v2.2.10 h1:tBs3QSyvjDyFTq3uoc/9xFpCuOsJQFNPiAhYdw2skhE=
|
||||||
|
github.com/klauspost/cpuid/v2 v2.2.10/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
|
||||||
|
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
|
||||||
|
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
|
||||||
|
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
|
||||||
|
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
|
||||||
|
github.com/orchard9/go-chassis v0.1.0 h1:epi8NWfYIqhS0GkHMRLVS0v9//brI4Ithd7RyaoYxA4=
|
||||||
|
github.com/orchard9/go-chassis v0.1.0/go.mod h1:lhBicqc77tUEogh1jScEWEzxPqnsw2fpbzid/1bKVcQ=
|
||||||
|
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||||
|
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||||
|
github.com/prometheus/client_golang v1.24.0 h1:5XStIklKuAtJSNpdD3s8XJj/Yv78IQmE1kbNk87JrAI=
|
||||||
|
github.com/prometheus/client_golang v1.24.0/go.mod h1:QcsNdotprC2nS4BTM2ucbcqxd2CeXTEa9jW7zHO9iDE=
|
||||||
|
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
|
||||||
|
github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=
|
||||||
|
github.com/prometheus/common v0.70.0 h1:bcpru3tWPVnxGnETLgOV5jbp/JRXgYEyv65CuBLAMMI=
|
||||||
|
github.com/prometheus/common v0.70.0/go.mod h1:S/SFasQmgGiYH6C81LKCtYa8QACgthGg5zxL2udV7SY=
|
||||||
|
github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
|
||||||
|
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
|
||||||
|
github.com/redis/go-redis/v9 v9.22.0 h1:laDvpYXTJtZLloinw1fA5Kqd6HAEH2XKxOkG/PDq2F0=
|
||||||
|
github.com/redis/go-redis/v9 v9.22.0/go.mod h1:y2g0Wj8rQvuK0ELM+oxSudcLtC09JScs98I/X9gRWY4=
|
||||||
|
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
|
||||||
|
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
|
||||||
|
github.com/zeebo/xxh3 v1.1.0 h1:s7DLGDK45Dyfg7++yxI0khrfwq9661w9EN78eP/UZVs=
|
||||||
|
github.com/zeebo/xxh3 v1.1.0/go.mod h1:IisAie1LELR4xhVinxWS5+zf1lA4p0MW4T+w+W07F5s=
|
||||||
|
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
|
||||||
|
go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
|
||||||
|
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
|
||||||
|
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
|
||||||
|
go.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ=
|
||||||
|
go.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ=
|
||||||
|
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
|
||||||
|
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
|
||||||
|
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
|
||||||
|
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
|
||||||
|
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||||
|
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||||
131
internal/secret/id.go
Normal file
131
internal/secret/id.go
Normal file
@ -0,0 +1,131 @@
|
|||||||
|
// Package secret holds hush's domain types and policy: identifiers, size and
|
||||||
|
// lifetime limits. It imports nothing outside the standard library, so the
|
||||||
|
// rules live in one place and are testable without Redis or HTTP.
|
||||||
|
package secret
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/rand"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/base64"
|
||||||
|
"encoding/hex"
|
||||||
|
"errors"
|
||||||
|
"log/slog"
|
||||||
|
)
|
||||||
|
|
||||||
|
// IDBytes is the entropy behind a secret id. 256 bits makes enumeration a
|
||||||
|
// non-threat: an attacker guessing ids is not a scenario this service defends
|
||||||
|
// against with rate limits, it is a scenario arithmetic forecloses.
|
||||||
|
const IDBytes = 32
|
||||||
|
|
||||||
|
// idEncoding is base64url without padding, so an id is URL-safe, 43 characters,
|
||||||
|
// and needs no escaping in a path or a fragment.
|
||||||
|
var idEncoding = base64.RawURLEncoding
|
||||||
|
|
||||||
|
// ErrMalformedID is returned for a value that cannot be an id this service
|
||||||
|
// minted. Callers turn it into the same 410 as a missing secret — telling a
|
||||||
|
// caller that their id was well-formed but absent confirms the id space.
|
||||||
|
var ErrMalformedID = errors.New("malformed secret id")
|
||||||
|
|
||||||
|
// ID is a secret's identifier, and it IS the capability: anyone holding it can
|
||||||
|
// reveal the secret exactly once. It is therefore treated like a bearer token.
|
||||||
|
//
|
||||||
|
// Every accidental path prints a REDACTED form. An earlier version tried to
|
||||||
|
// prevent leaks by implementing no String() at all, which was wrong: Go's fmt
|
||||||
|
// prints unexported struct fields anyway, so `fmt.Sprintf("%v", id)` emitted
|
||||||
|
// the live id. Forbidding the method did not remove the leak, it only removed
|
||||||
|
// the chance to control it.
|
||||||
|
//
|
||||||
|
// So instead the safe form is the DEFAULT and the raw value needs an explicit
|
||||||
|
// call:
|
||||||
|
//
|
||||||
|
// String() -> "ID(a1b2c3d4e5f6)" fmt %v, %s, string concatenation
|
||||||
|
// LogValue() -> "a1b2c3d4e5f6" every slog call site
|
||||||
|
// MarshalJSON -> refuses a response struct cannot leak one silently
|
||||||
|
// Value() -> the raw id the two places it must escape
|
||||||
|
//
|
||||||
|
// Enforced by id_test.go, which fails if any of those starts emitting the raw
|
||||||
|
// value.
|
||||||
|
type ID struct {
|
||||||
|
raw string
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewID mints a fresh identifier from crypto/rand. It returns an error rather
|
||||||
|
// than panicking: a service that cannot get entropy must refuse to mint a
|
||||||
|
// secret, not mint a guessable one.
|
||||||
|
func NewID() (ID, error) {
|
||||||
|
b := make([]byte, IDBytes)
|
||||||
|
if _, err := rand.Read(b); err != nil {
|
||||||
|
return ID{}, err
|
||||||
|
}
|
||||||
|
return ID{raw: idEncoding.EncodeToString(b)}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ParseID validates an id from a URL path. It checks the encoding and the
|
||||||
|
// decoded LENGTH, so a short-but-valid base64 string cannot become an id and
|
||||||
|
// widen the space a scanner has to cover.
|
||||||
|
func ParseID(s string) (ID, error) {
|
||||||
|
if len(s) != idEncoding.EncodedLen(IDBytes) {
|
||||||
|
return ID{}, ErrMalformedID
|
||||||
|
}
|
||||||
|
b, err := idEncoding.DecodeString(s)
|
||||||
|
if err != nil || len(b) != IDBytes {
|
||||||
|
return ID{}, ErrMalformedID
|
||||||
|
}
|
||||||
|
return ID{raw: s}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Value returns the raw id. Every call site is a place where the capability
|
||||||
|
// escapes, so there are deliberately few: the storage key and the created
|
||||||
|
// response body.
|
||||||
|
func (id ID) Value() string { return id.raw }
|
||||||
|
|
||||||
|
// IsZero reports whether this is the zero ID, which no minting path produces.
|
||||||
|
func (id ID) IsZero() bool { return id.raw == "" }
|
||||||
|
|
||||||
|
// LogHandle is the only form of an id that may be logged or reported: the first
|
||||||
|
// 12 hex characters of its SHA-256. It is stable, so one secret's create,
|
||||||
|
// reveal and gone lines correlate across the log corpus, and it is one-way, so
|
||||||
|
// a log reader cannot reveal the secret it refers to.
|
||||||
|
//
|
||||||
|
// 48 bits of a hash is not a secret-strength value and is not treated as one —
|
||||||
|
// it is a correlation handle. The preimage is 256 bits of entropy, so recovering
|
||||||
|
// an id from a handle is not feasible even though the handle is short.
|
||||||
|
func (id ID) LogHandle() string {
|
||||||
|
if id.raw == "" {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
sum := sha256.Sum256([]byte(id.raw))
|
||||||
|
return hex.EncodeToString(sum[:])[:12]
|
||||||
|
}
|
||||||
|
|
||||||
|
// String is the redacted form, so `%v`, `%s` and string concatenation are all
|
||||||
|
// safe by default. Reaching the raw id requires Value().
|
||||||
|
func (id ID) String() string {
|
||||||
|
if id.raw == "" {
|
||||||
|
return "ID(zero)"
|
||||||
|
}
|
||||||
|
return "ID(" + id.LogHandle() + ")"
|
||||||
|
}
|
||||||
|
|
||||||
|
// LogValue makes every slog call site safe without the caller thinking about
|
||||||
|
// it: `log.Info("secret.created", "id", id)` emits the handle, not the
|
||||||
|
// capability. This is why the handlers can pass ids around without a review
|
||||||
|
// checklist for each log line.
|
||||||
|
func (id ID) LogValue() slog.Value { return slog.StringValue(id.LogHandle()) }
|
||||||
|
|
||||||
|
// MarshalJSON REFUSES rather than emitting either form.
|
||||||
|
//
|
||||||
|
// Emitting the raw id would leak a capability into any response struct that
|
||||||
|
// happened to embed an ID. Emitting the redacted form would be worse: it would
|
||||||
|
// produce a response that looks like it carries an id and does not, and the bug
|
||||||
|
// would surface as a broken link rather than a failed request. The two places
|
||||||
|
// an id legitimately reaches a client both call Value() explicitly.
|
||||||
|
func (id ID) MarshalJSON() ([]byte, error) {
|
||||||
|
return nil, errors.New("secret.ID must not be serialised: call Value() at the one site that needs it")
|
||||||
|
}
|
||||||
|
|
||||||
|
// StorageKey is the Redis key holding this secret's ciphertext. The `hush:`
|
||||||
|
// prefix is what the Redis ACL user is scoped to (`~hush:*`), so a bug that
|
||||||
|
// built a key outside this prefix would be refused by the server rather than
|
||||||
|
// touching another tenant's keyspace.
|
||||||
|
func (id ID) StorageKey() string { return "hush:s:" + id.raw }
|
||||||
190
internal/secret/id_test.go
Normal file
190
internal/secret/id_test.go
Normal file
@ -0,0 +1,190 @@
|
|||||||
|
package secret
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"log/slog"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The id IS the capability, so every path that could print one is checked here.
|
||||||
|
//
|
||||||
|
// The first version of this test forbade String() outright, on the theory that
|
||||||
|
// a type with no Stringer cannot be interpolated. That was wrong and this test
|
||||||
|
// caught it: Go's fmt prints unexported struct fields regardless, so `%v` was
|
||||||
|
// emitting live ids. The design now makes the REDACTED form the default on
|
||||||
|
// every accidental path and requires Value() for the raw one.
|
||||||
|
//
|
||||||
|
// Read this test first if you are wondering why ID has a String() that throws
|
||||||
|
// away information and a MarshalJSON that refuses.
|
||||||
|
func TestNoAccidentalPathEmitsTheRawID(t *testing.T) {
|
||||||
|
id, err := NewID()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
raw := id.Value()
|
||||||
|
|
||||||
|
for _, tc := range []struct {
|
||||||
|
name string
|
||||||
|
got string
|
||||||
|
}{
|
||||||
|
{"fmt %v", fmt.Sprintf("%v", id)},
|
||||||
|
{"fmt %s", fmt.Sprintf("%s", id)},
|
||||||
|
{"fmt %+v", fmt.Sprintf("%+v", id)},
|
||||||
|
{"fmt %#v of a wrapper struct", fmt.Sprintf("%v", struct{ ID ID }{id})},
|
||||||
|
{"String()", id.String()},
|
||||||
|
{"LogValue()", id.LogValue().String()},
|
||||||
|
{"inside a slice", fmt.Sprintf("%v", []ID{id})},
|
||||||
|
{"inside a map", fmt.Sprintf("%v", map[string]ID{"k": id})},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
if strings.Contains(tc.got, raw) {
|
||||||
|
t.Fatalf("%s emitted the live id (%q) — this value would appear in a log line "+
|
||||||
|
"and anyone reading it could reveal the secret", tc.name, tc.got)
|
||||||
|
}
|
||||||
|
if !strings.Contains(tc.got, id.LogHandle()) {
|
||||||
|
t.Fatalf("%s = %q, which carries neither the id nor its handle; a log line "+
|
||||||
|
"with no handle cannot be correlated", tc.name, tc.got)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A real slog handler, because that is where ids actually pass through.
|
||||||
|
func TestSlogEmitsTheHandleNotTheID(t *testing.T) {
|
||||||
|
id, err := NewID()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var buf bytes.Buffer
|
||||||
|
log := slog.New(slog.NewJSONHandler(&buf, nil))
|
||||||
|
|
||||||
|
// Every shape a call site might use, including the careless one.
|
||||||
|
log.Info("secret.created", "id", id)
|
||||||
|
log.Info("secret.created", "sid", id.LogHandle())
|
||||||
|
log.With("id", id).Info("secret.revealed")
|
||||||
|
log.Info("secret.gone", "ids", []ID{id})
|
||||||
|
|
||||||
|
out := buf.String()
|
||||||
|
if strings.Contains(out, id.Value()) {
|
||||||
|
t.Fatalf("slog output contains the live id:\n%s", out)
|
||||||
|
}
|
||||||
|
if !strings.Contains(out, id.LogHandle()) {
|
||||||
|
t.Fatalf("slog output contains no correlation handle:\n%s", out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Serialising an ID must FAIL rather than quietly emit either form. The raw
|
||||||
|
// value would leak a capability out of any response struct embedding an ID;
|
||||||
|
// the redacted value would produce a response that looks like it carries an id
|
||||||
|
// and does not, turning a server bug into a broken link for the recipient.
|
||||||
|
func TestMarshallingAnIDIsAnError(t *testing.T) {
|
||||||
|
id, err := NewID()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if b, err := json.Marshal(id); err == nil {
|
||||||
|
t.Fatalf("json.Marshal(ID) succeeded with %s; it must refuse", b)
|
||||||
|
}
|
||||||
|
// And from inside a struct, which is how it would actually happen.
|
||||||
|
if b, err := json.Marshal(struct {
|
||||||
|
ID ID `json:"id"`
|
||||||
|
}{id}); err == nil {
|
||||||
|
t.Fatalf("marshalling a struct containing an ID succeeded with %s; it must refuse", b)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNewIDIsUnguessablyWideAndUnique(t *testing.T) {
|
||||||
|
seen := make(map[string]bool, 1000)
|
||||||
|
for range 1000 {
|
||||||
|
id, err := NewID()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(id.Value()) != 43 {
|
||||||
|
t.Fatalf("id %q is %d chars, want 43 (256 bits base64url unpadded)", id.Value(), len(id.Value()))
|
||||||
|
}
|
||||||
|
if seen[id.Value()] {
|
||||||
|
t.Fatal("NewID repeated a value in 1000 draws — entropy is broken")
|
||||||
|
}
|
||||||
|
seen[id.Value()] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseIDRoundTripsAndRefusesEverythingElse(t *testing.T) {
|
||||||
|
id, err := NewID()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
back, err := ParseID(id.Value())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ParseID rejected a freshly minted id: %v", err)
|
||||||
|
}
|
||||||
|
if back.Value() != id.Value() {
|
||||||
|
t.Fatalf("round trip changed the id: %q -> %q", id.Value(), back.Value())
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, bad := range []struct{ name, in string }{
|
||||||
|
{"empty", ""},
|
||||||
|
{"too short", "abc"},
|
||||||
|
{"one char short", id.Value()[:42]},
|
||||||
|
{"one char long", id.Value() + "a"},
|
||||||
|
{"not base64url", strings.Repeat("!", 43)},
|
||||||
|
// A shorter secret encoded in a 43-char field would widen the space a
|
||||||
|
// scanner must cover, so the DECODED length is checked too.
|
||||||
|
{"padded base64", strings.Repeat("A", 40) + "==="},
|
||||||
|
{"path traversal", "../../etc/passwd"},
|
||||||
|
{"redis glob", strings.Repeat("*", 43)},
|
||||||
|
} {
|
||||||
|
t.Run(bad.name, func(t *testing.T) {
|
||||||
|
if _, err := ParseID(bad.in); err == nil {
|
||||||
|
t.Fatalf("ParseID(%q) accepted a value NewID cannot produce", bad.in)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLogHandleIsStableShortAndNotTheID(t *testing.T) {
|
||||||
|
id, err := NewID()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
h := id.LogHandle()
|
||||||
|
if len(h) != 12 {
|
||||||
|
t.Fatalf("LogHandle() = %q, want 12 hex chars", h)
|
||||||
|
}
|
||||||
|
if id.LogHandle() != h {
|
||||||
|
t.Fatal("LogHandle is not stable across calls, so log lines for one secret will not correlate")
|
||||||
|
}
|
||||||
|
if strings.Contains(h, id.Value()) || strings.Contains(id.Value(), h) {
|
||||||
|
t.Fatalf("LogHandle %q overlaps the raw id %q — it must be a hash, not a prefix", h, id.Value())
|
||||||
|
}
|
||||||
|
|
||||||
|
// Different ids must not collide into one handle, or correlation is wrong.
|
||||||
|
other, err := NewID()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if other.LogHandle() == h {
|
||||||
|
t.Fatal("two ids produced the same LogHandle")
|
||||||
|
}
|
||||||
|
|
||||||
|
if (ID{}).LogHandle() != "" {
|
||||||
|
t.Fatal("the zero ID must produce an empty handle, not a hash of the empty string")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestStorageKeyStaysInsideTheACLPrefix(t *testing.T) {
|
||||||
|
id, err := NewID()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// The Redis ACL user is scoped to ~hush:*. A key built outside that prefix
|
||||||
|
// is refused by the server, so this test is what keeps a rename from
|
||||||
|
// turning every write into a NOPERM at runtime.
|
||||||
|
if !strings.HasPrefix(id.StorageKey(), "hush:") {
|
||||||
|
t.Fatalf("StorageKey() = %q, which is outside the ~hush:* ACL scope", id.StorageKey())
|
||||||
|
}
|
||||||
|
}
|
||||||
80
internal/secret/policy.go
Normal file
80
internal/secret/policy.go
Normal file
@ -0,0 +1,80 @@
|
|||||||
|
package secret
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/base64"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Size and lifetime policy. Every bound here is a product decision with a
|
||||||
|
// reason, not a tuning knob:
|
||||||
|
//
|
||||||
|
// - MaxCiphertextBytes keeps hush a courier for credentials rather than a file
|
||||||
|
// host. 64 KiB of AES-GCM holds roughly 48 KiB of plaintext, which is a very
|
||||||
|
// large credential and a very small file.
|
||||||
|
// - MinTTL exists because a link that expires before the recipient reads their
|
||||||
|
// messages is a support ticket, not a security win.
|
||||||
|
// - MaxTTL bounds exposure. The residual risk in this design is the key sitting
|
||||||
|
// in a browser history entry, and a week is as long as that is defensible.
|
||||||
|
const (
|
||||||
|
MaxCiphertextBytes = 64 * 1024
|
||||||
|
MinTTL = 5 * time.Minute
|
||||||
|
MaxTTL = 7 * 24 * time.Hour
|
||||||
|
DefaultTTL = 24 * time.Hour
|
||||||
|
)
|
||||||
|
|
||||||
|
// Validation failures. Each maps to one API error code, so a caller can branch
|
||||||
|
// on the cause without parsing prose.
|
||||||
|
var (
|
||||||
|
ErrCiphertextEmpty = errors.New("ciphertext is empty")
|
||||||
|
ErrCiphertextTooLarge = errors.New("ciphertext exceeds the size limit")
|
||||||
|
ErrCiphertextInvalid = errors.New("ciphertext is not valid base64url")
|
||||||
|
ErrTTLOutOfRange = errors.New("ttl is outside the permitted range")
|
||||||
|
)
|
||||||
|
|
||||||
|
// ciphertextEncoding matches what the browser produces: base64url, unpadded.
|
||||||
|
var ciphertextEncoding = base64.RawURLEncoding
|
||||||
|
|
||||||
|
// ValidateCiphertext checks what the server is ABLE to check. hush cannot
|
||||||
|
// verify that the bytes decrypt, because it has no key — by design. So it
|
||||||
|
// verifies the two things it can: that the encoding is what this service's
|
||||||
|
// clients produce, and that the size is inside the cap.
|
||||||
|
//
|
||||||
|
// The encoding check is not cosmetic. Without it, hush becomes a store for
|
||||||
|
// arbitrary bytes addressable by URL, which is a different and much less
|
||||||
|
// defensible service than the one described in the README.
|
||||||
|
func ValidateCiphertext(ciphertext string) error {
|
||||||
|
switch {
|
||||||
|
case ciphertext == "":
|
||||||
|
return ErrCiphertextEmpty
|
||||||
|
case len(ciphertext) > MaxCiphertextBytes:
|
||||||
|
// Measured on the encoded form, which is what is stored and what
|
||||||
|
// bounds memory. Checked BEFORE decoding so an oversized body is
|
||||||
|
// rejected without allocating its decoded copy.
|
||||||
|
return fmt.Errorf("%w: %d > %d bytes", ErrCiphertextTooLarge, len(ciphertext), MaxCiphertextBytes)
|
||||||
|
}
|
||||||
|
if _, err := ciphertextEncoding.DecodeString(ciphertext); err != nil {
|
||||||
|
return ErrCiphertextInvalid
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ResolveTTL turns a caller's requested lifetime into the one that will be
|
||||||
|
// used. Zero means "unspecified" and gets the default.
|
||||||
|
//
|
||||||
|
// An out-of-range value is an ERROR, never a silent clamp. A caller who asked
|
||||||
|
// for 30 days and got 7 without being told would believe their link outlives
|
||||||
|
// its actual expiry, and would find out when the recipient could not open it.
|
||||||
|
func ResolveTTL(requested time.Duration) (time.Duration, error) {
|
||||||
|
if requested == 0 {
|
||||||
|
return DefaultTTL, nil
|
||||||
|
}
|
||||||
|
if requested < MinTTL || requested > MaxTTL {
|
||||||
|
return 0, fmt.Errorf("%w: %s not in [%s, %s]", ErrTTLOutOfRange, requested, MinTTL, MaxTTL)
|
||||||
|
}
|
||||||
|
// Truncate to whole seconds: the wire format is seconds and Redis EX takes
|
||||||
|
// seconds, so keeping sub-second precision would make the expires_at we
|
||||||
|
// report disagree with the expiry Redis enforces.
|
||||||
|
return requested.Truncate(time.Second), nil
|
||||||
|
}
|
||||||
91
internal/secret/policy_test.go
Normal file
91
internal/secret/policy_test.go
Normal file
@ -0,0 +1,91 @@
|
|||||||
|
package secret
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/base64"
|
||||||
|
"errors"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestValidateCiphertext(t *testing.T) {
|
||||||
|
valid := base64.RawURLEncoding.EncodeToString([]byte("nonce+ciphertext bytes"))
|
||||||
|
|
||||||
|
for _, tc := range []struct {
|
||||||
|
name string
|
||||||
|
in string
|
||||||
|
want error
|
||||||
|
}{
|
||||||
|
{"a real base64url blob", valid, nil},
|
||||||
|
{"empty", "", ErrCiphertextEmpty},
|
||||||
|
{"not base64url", "!!!not base64!!!", ErrCiphertextInvalid},
|
||||||
|
// Standard base64 uses + and /, which are not URL-safe. Accepting them
|
||||||
|
// would mean two spellings of one ciphertext and a client that works in
|
||||||
|
// one browser and not another.
|
||||||
|
{"standard base64 alphabet", "YWJj+/8=", ErrCiphertextInvalid},
|
||||||
|
{"padded", "YWJjZA==", ErrCiphertextInvalid},
|
||||||
|
{"at the cap", strings.Repeat("A", MaxCiphertextBytes), nil},
|
||||||
|
{"one byte over the cap", strings.Repeat("A", MaxCiphertextBytes+1), ErrCiphertextTooLarge},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
err := ValidateCiphertext(tc.in)
|
||||||
|
if !errors.Is(err, tc.want) {
|
||||||
|
t.Fatalf("ValidateCiphertext() = %v, want %v", err, tc.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The size check must happen BEFORE the decode, or a 64 KiB+ body costs a
|
||||||
|
// decoded copy before being refused — which is the cheap half of a memory DoS.
|
||||||
|
func TestOversizedCiphertextIsRefusedWithoutDecoding(t *testing.T) {
|
||||||
|
// Deliberately not valid base64. If the implementation decoded first, this
|
||||||
|
// would come back as ErrCiphertextInvalid instead of ErrCiphertextTooLarge.
|
||||||
|
huge := strings.Repeat("!", MaxCiphertextBytes+1)
|
||||||
|
if err := ValidateCiphertext(huge); !errors.Is(err, ErrCiphertextTooLarge) {
|
||||||
|
t.Fatalf("ValidateCiphertext(oversized invalid) = %v, want ErrCiphertextTooLarge — "+
|
||||||
|
"the size gate must precede the decode", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveTTL(t *testing.T) {
|
||||||
|
for _, tc := range []struct {
|
||||||
|
name string
|
||||||
|
in time.Duration
|
||||||
|
want time.Duration
|
||||||
|
err error
|
||||||
|
}{
|
||||||
|
{"unspecified takes the default", 0, DefaultTTL, nil},
|
||||||
|
{"at the minimum", MinTTL, MinTTL, nil},
|
||||||
|
{"at the maximum", MaxTTL, MaxTTL, nil},
|
||||||
|
{"a normal day", 24 * time.Hour, 24 * time.Hour, nil},
|
||||||
|
{"below the minimum", MinTTL - time.Second, 0, ErrTTLOutOfRange},
|
||||||
|
{"above the maximum", MaxTTL + time.Second, 0, ErrTTLOutOfRange},
|
||||||
|
{"negative", -time.Hour, 0, ErrTTLOutOfRange},
|
||||||
|
{"absurd", 3650 * 24 * time.Hour, 0, ErrTTLOutOfRange},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
got, err := ResolveTTL(tc.in)
|
||||||
|
if !errors.Is(err, tc.err) {
|
||||||
|
t.Fatalf("ResolveTTL(%s) error = %v, want %v", tc.in, err, tc.err)
|
||||||
|
}
|
||||||
|
if err == nil && got != tc.want {
|
||||||
|
t.Fatalf("ResolveTTL(%s) = %s, want %s", tc.in, got, tc.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// An out-of-range TTL must be an ERROR and never a silent clamp. A caller who
|
||||||
|
// asked for 30 days and was quietly given 7 would believe their link outlives
|
||||||
|
// its real expiry, and would discover otherwise when the recipient could not
|
||||||
|
// open it.
|
||||||
|
func TestOutOfRangeTTLIsRefusedNotClamped(t *testing.T) {
|
||||||
|
got, err := ResolveTTL(30 * 24 * time.Hour)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatalf("ResolveTTL(30d) silently returned %s instead of refusing", got)
|
||||||
|
}
|
||||||
|
if got != 0 {
|
||||||
|
t.Fatalf("ResolveTTL returned %s alongside an error; callers must not see a usable value", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
80
internal/store/memory.go
Normal file
80
internal/store/memory.go
Normal file
@ -0,0 +1,80 @@
|
|||||||
|
package store
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/orchard9/hush/internal/secret"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Memory is an in-process Store for tests and `make dev` without Redis.
|
||||||
|
//
|
||||||
|
// It is NOT a deployment option, and the deployment path cannot select it: the
|
||||||
|
// store is chosen by whether REDIS_URL is set, and REDIS_URL is Required in
|
||||||
|
// config, so a misconfigured pod fails to boot rather than silently serving
|
||||||
|
// secrets from a store that dies with the process. This type exists so the
|
||||||
|
// handler tests do not need a container.
|
||||||
|
type Memory struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
items map[string]memItem
|
||||||
|
now func() time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
type memItem struct {
|
||||||
|
ciphertext string
|
||||||
|
expiresAt time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewMemory returns an empty store using the real clock.
|
||||||
|
func NewMemory() *Memory {
|
||||||
|
return &Memory{items: map[string]memItem{}, now: time.Now}
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewMemoryAt returns a store driven by a caller-supplied clock, so expiry is
|
||||||
|
// testable without sleeping.
|
||||||
|
func NewMemoryAt(now func() time.Time) *Memory {
|
||||||
|
return &Memory{items: map[string]memItem{}, now: now}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *Memory) Put(_ context.Context, id secret.ID, ciphertext string, ttl time.Duration) error {
|
||||||
|
m.mu.Lock()
|
||||||
|
defer m.mu.Unlock()
|
||||||
|
k := id.StorageKey()
|
||||||
|
// Mirror Redis SET NX semantics, including that an EXPIRED key is treated
|
||||||
|
// as absent and may be overwritten. A memory store that rejected a write
|
||||||
|
// against a stale entry would pass tests Redis fails.
|
||||||
|
if it, ok := m.items[k]; ok && m.now().Before(it.expiresAt) {
|
||||||
|
return ErrIDCollision
|
||||||
|
}
|
||||||
|
m.items[k] = memItem{ciphertext: ciphertext, expiresAt: m.now().Add(ttl)}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Take mirrors GETDEL: the read and the delete happen under one lock, so the
|
||||||
|
// atomicity the one-time guarantee depends on holds here too and the
|
||||||
|
// concurrency test is meaningful against both implementations.
|
||||||
|
func (m *Memory) Take(_ context.Context, id secret.ID) (string, error) {
|
||||||
|
m.mu.Lock()
|
||||||
|
defer m.mu.Unlock()
|
||||||
|
k := id.StorageKey()
|
||||||
|
it, ok := m.items[k]
|
||||||
|
if !ok {
|
||||||
|
return "", ErrGone
|
||||||
|
}
|
||||||
|
delete(m.items, k)
|
||||||
|
if !m.now().Before(it.expiresAt) {
|
||||||
|
return "", ErrGone
|
||||||
|
}
|
||||||
|
return it.ciphertext, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *Memory) Ping(context.Context) error { return nil }
|
||||||
|
func (m *Memory) Close() error { return nil }
|
||||||
|
|
||||||
|
// AllowN is an always-allow limiter: rate limiting is an abuse control on the
|
||||||
|
// public deployment, and silently enforcing one in tests would make handler
|
||||||
|
// tests order-dependent and flaky.
|
||||||
|
func (m *Memory) AllowN(context.Context, string, int, time.Duration) (bool, time.Duration, error) {
|
||||||
|
return true, 0, nil
|
||||||
|
}
|
||||||
117
internal/store/redis.go
Normal file
117
internal/store/redis.go
Normal file
@ -0,0 +1,117 @@
|
|||||||
|
package store
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/redis/go-redis/v9"
|
||||||
|
|
||||||
|
"github.com/orchard9/hush/internal/secret"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Redis is the production store. It keeps one key per secret and lets Redis own
|
||||||
|
// expiry, so nothing in hush sweeps, scans, or holds a timer.
|
||||||
|
type Redis struct {
|
||||||
|
client *redis.Client
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewRedis dials Redis from a URL of the form
|
||||||
|
// redis://user:password@host:6379/5 — the ACL user, the password and the db
|
||||||
|
// index all ride in the URL, matching every other orchard9 service.
|
||||||
|
func NewRedis(url string) (*Redis, error) {
|
||||||
|
opt, err := redis.ParseURL(url)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("parse redis url: %w", err)
|
||||||
|
}
|
||||||
|
// A secret write must not hang a request behind a slow dependency: the
|
||||||
|
// chassis request timeout would fire and the caller would see a 500 with no
|
||||||
|
// idea whether the secret was stored. Short, explicit timeouts make the
|
||||||
|
// failure fast and unambiguous.
|
||||||
|
opt.DialTimeout = 3 * time.Second
|
||||||
|
opt.ReadTimeout = 2 * time.Second
|
||||||
|
opt.WriteTimeout = 2 * time.Second
|
||||||
|
opt.MaxRetries = 2
|
||||||
|
return &Redis{client: redis.NewClient(opt)}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Put stores ciphertext with SET ... EX ttl NX.
|
||||||
|
//
|
||||||
|
// NX is what makes an id collision an error instead of a silent overwrite of a
|
||||||
|
// live secret. It cannot fire by chance at 256 bits, which is exactly why a
|
||||||
|
// false return is worth surfacing: it means something is wrong with id
|
||||||
|
// generation, and the alternative is destroying a secret somebody is waiting on.
|
||||||
|
func (r *Redis) Put(ctx context.Context, id secret.ID, ciphertext string, ttl time.Duration) error {
|
||||||
|
ok, err := r.client.SetNX(ctx, id.StorageKey(), ciphertext, ttl).Result()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("redis set: %w", err)
|
||||||
|
}
|
||||||
|
if !ok {
|
||||||
|
return ErrIDCollision
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Take reads and destroys in ONE command.
|
||||||
|
//
|
||||||
|
// GETDEL (Redis 6.2+) is atomic, which is the entire one-time guarantee. A
|
||||||
|
// GET followed by a DEL has a window between the two round trips where two
|
||||||
|
// simultaneous readers both get the plaintext, and for this service that window
|
||||||
|
// is the product. Redis 7.4.8 runs in the cluster; the ACL user must carry
|
||||||
|
// `+getdel` or every reveal returns NOPERM.
|
||||||
|
//
|
||||||
|
// redis.Nil covers all four "not available" causes and collapses to ErrGone —
|
||||||
|
// see the doc comment on ErrGone for why they are not separated.
|
||||||
|
func (r *Redis) Take(ctx context.Context, id secret.ID) (string, error) {
|
||||||
|
v, err := r.client.GetDel(ctx, id.StorageKey()).Result()
|
||||||
|
switch {
|
||||||
|
case errors.Is(err, redis.Nil):
|
||||||
|
return "", ErrGone
|
||||||
|
case err != nil:
|
||||||
|
return "", fmt.Errorf("redis getdel: %w", err)
|
||||||
|
}
|
||||||
|
return v, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ping backs /readyz. It is the reason a Redis outage takes the pod out of the
|
||||||
|
// Service rather than serving 500s from a pod the load balancer still trusts.
|
||||||
|
func (r *Redis) Ping(ctx context.Context) error {
|
||||||
|
return r.client.Ping(ctx).Err()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Close shuts the pool down.
|
||||||
|
func (r *Redis) Close() error { return r.client.Close() }
|
||||||
|
|
||||||
|
// AllowN implements a fixed-window rate limit in Redis: INCR the window key,
|
||||||
|
// set its expiry on first use, and refuse once the count passes the limit.
|
||||||
|
//
|
||||||
|
// Fixed window rather than a sliding one because it costs two commands, needs
|
||||||
|
// no Lua, and the failure it permits — up to 2x the limit across a window
|
||||||
|
// boundary — is irrelevant for an abuse control whose job is to stop bulk
|
||||||
|
// automation, not to meter precisely.
|
||||||
|
//
|
||||||
|
// It lives in Redis rather than in process memory so the limit still holds if
|
||||||
|
// hush is ever scaled past one replica, and so a restart cannot be used to
|
||||||
|
// reset it.
|
||||||
|
func (r *Redis) AllowN(ctx context.Context, key string, limit int, window time.Duration) (bool, time.Duration, error) {
|
||||||
|
full := "hush:rl:" + key
|
||||||
|
pipe := r.client.Pipeline()
|
||||||
|
incr := pipe.Incr(ctx, full)
|
||||||
|
// NX so a long-running window is not extended by later requests inside it;
|
||||||
|
// without it a steady stream of calls would push the expiry forward forever
|
||||||
|
// and the window would never reset.
|
||||||
|
pipe.ExpireNX(ctx, full, window)
|
||||||
|
if _, err := pipe.Exec(ctx); err != nil {
|
||||||
|
return false, 0, fmt.Errorf("redis ratelimit: %w", err)
|
||||||
|
}
|
||||||
|
count := incr.Val()
|
||||||
|
if count > int64(limit) {
|
||||||
|
retry, err := r.client.PTTL(ctx, full).Result()
|
||||||
|
if err != nil || retry < 0 {
|
||||||
|
retry = window
|
||||||
|
}
|
||||||
|
return false, retry, nil
|
||||||
|
}
|
||||||
|
return true, 0, nil
|
||||||
|
}
|
||||||
50
internal/store/store.go
Normal file
50
internal/store/store.go
Normal file
@ -0,0 +1,50 @@
|
|||||||
|
// Package store persists ciphertext for at most a TTL and destroys it on the
|
||||||
|
// first successful read.
|
||||||
|
package store
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/orchard9/hush/internal/secret"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ErrGone is returned by Take for every reason a secret is not available: it
|
||||||
|
// never existed, it was already revealed, it expired, or Redis evicted it early
|
||||||
|
// under memory pressure.
|
||||||
|
//
|
||||||
|
// The causes are deliberately NOT distinguished. Not because they are hard to
|
||||||
|
// tell apart, but because telling them apart is the leak: a caller who can
|
||||||
|
// separate "never existed" from "already revealed" can confirm that a given
|
||||||
|
// link was real, which is information about someone else's secret. The API maps
|
||||||
|
// this one error to one 410 for all four cases.
|
||||||
|
var ErrGone = errors.New("secret is gone")
|
||||||
|
|
||||||
|
// ErrIDCollision means Put found the key already occupied. At 256 bits of id
|
||||||
|
// entropy this cannot happen by chance, so it means a bug or a repeated id, and
|
||||||
|
// it is surfaced rather than silently overwriting a live secret.
|
||||||
|
var ErrIDCollision = errors.New("secret id already exists")
|
||||||
|
|
||||||
|
// Store is the whole persistence contract. Two methods, both destructive-safe:
|
||||||
|
// there is no Get, no List, and no Exists — a store that could answer "does
|
||||||
|
// this id exist" without consuming it would let the reveal page leak existence,
|
||||||
|
// and a store that could list would make a compromise catastrophic instead of
|
||||||
|
// merely bad.
|
||||||
|
type Store interface {
|
||||||
|
// Put writes ciphertext under id, expiring after ttl. It must fail rather
|
||||||
|
// than overwrite an existing key.
|
||||||
|
Put(ctx context.Context, id secret.ID, ciphertext string, ttl time.Duration) error
|
||||||
|
|
||||||
|
// Take returns the ciphertext and destroys it ATOMICALLY. Two concurrent
|
||||||
|
// callers must not both receive a value; exactly one gets it and the other
|
||||||
|
// gets ErrGone. This atomicity is the one-time property — an implementation
|
||||||
|
// that reads then deletes has a window in which both callers succeed.
|
||||||
|
Take(ctx context.Context, id secret.ID) (string, error)
|
||||||
|
|
||||||
|
// Ping reports whether the backing store is usable, for readiness.
|
||||||
|
Ping(ctx context.Context) error
|
||||||
|
|
||||||
|
// Close releases resources.
|
||||||
|
Close() error
|
||||||
|
}
|
||||||
246
internal/store/store_test.go
Normal file
246
internal/store/store_test.go
Normal file
@ -0,0 +1,246 @@
|
|||||||
|
package store_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/orchard9/hush/internal/secret"
|
||||||
|
"github.com/orchard9/hush/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
// One contract, two implementations. The suite runs against Memory always and
|
||||||
|
// against Redis whenever HUSH_TEST_REDIS_URL is set (`make test-redis`, and CI
|
||||||
|
// where a Redis service is available).
|
||||||
|
//
|
||||||
|
// Running the SAME assertions against both is the point: Memory exists so
|
||||||
|
// handler tests need no container, and it is only trustworthy if it is held to
|
||||||
|
// the behaviour Redis actually has — including that an expired key may be
|
||||||
|
// overwritten and that Take is atomic.
|
||||||
|
func eachStore(t *testing.T, fn func(t *testing.T, s store.Store)) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
t.Run("memory", func(t *testing.T) {
|
||||||
|
fn(t, store.NewMemory())
|
||||||
|
})
|
||||||
|
|
||||||
|
url := os.Getenv("HUSH_TEST_REDIS_URL")
|
||||||
|
if url == "" {
|
||||||
|
t.Log("HUSH_TEST_REDIS_URL unset: skipping the Redis half of the contract suite")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
t.Run("redis", func(t *testing.T) {
|
||||||
|
r, err := store.NewRedis(url)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("dial redis: %v", err)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() { _ = r.Close() })
|
||||||
|
if err := r.Ping(context.Background()); err != nil {
|
||||||
|
t.Fatalf("ping redis at %s: %v", url, err)
|
||||||
|
}
|
||||||
|
fn(t, r)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func newID(t *testing.T) secret.ID {
|
||||||
|
t.Helper()
|
||||||
|
id, err := secret.NewID()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return id
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPutThenTakeReturnsTheCiphertextExactlyOnce(t *testing.T) {
|
||||||
|
eachStore(t, func(t *testing.T, s store.Store) {
|
||||||
|
ctx := context.Background()
|
||||||
|
id := newID(t)
|
||||||
|
const ct = "bm9uY2UtYW5kLWNpcGhlcnRleHQ"
|
||||||
|
|
||||||
|
if err := s.Put(ctx, id, ct, time.Minute); err != nil {
|
||||||
|
t.Fatalf("Put: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
got, err := s.Take(ctx, id)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("first Take: %v", err)
|
||||||
|
}
|
||||||
|
if got != ct {
|
||||||
|
t.Fatalf("first Take = %q, want %q", got, ct)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The whole product: the second read must find nothing.
|
||||||
|
if _, err := s.Take(ctx, id); !errors.Is(err, store.ErrGone) {
|
||||||
|
t.Fatalf("second Take = %v, want ErrGone — the secret was not destroyed", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTakeOfAnUnknownIDIsGone(t *testing.T) {
|
||||||
|
eachStore(t, func(t *testing.T, s store.Store) {
|
||||||
|
if _, err := s.Take(context.Background(), newID(t)); !errors.Is(err, store.ErrGone) {
|
||||||
|
t.Fatalf("Take(unknown) = %v, want ErrGone", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPutRefusesToOverwriteALiveSecret(t *testing.T) {
|
||||||
|
eachStore(t, func(t *testing.T, s store.Store) {
|
||||||
|
ctx := context.Background()
|
||||||
|
id := newID(t)
|
||||||
|
if err := s.Put(ctx, id, "first", time.Minute); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// At 256 bits this cannot happen by chance, so if it ever does it is an
|
||||||
|
// id-generation bug. Destroying the live secret instead of reporting it
|
||||||
|
// would lose a secret someone is waiting on.
|
||||||
|
if err := s.Put(ctx, id, "second", time.Minute); !errors.Is(err, store.ErrIDCollision) {
|
||||||
|
t.Fatalf("second Put = %v, want ErrIDCollision", err)
|
||||||
|
}
|
||||||
|
got, err := s.Take(ctx, id)
|
||||||
|
if err != nil || got != "first" {
|
||||||
|
t.Fatalf("Take after refused overwrite = (%q, %v), want (\"first\", nil)", got, err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// Exactly one winner under concurrency. This is why the store contract demands
|
||||||
|
// an atomic read-and-destroy: a GET followed by a DEL has a window in which two
|
||||||
|
// readers both receive the plaintext, and for a one-time secret that window is
|
||||||
|
// the entire guarantee.
|
||||||
|
func TestConcurrentTakesProduceExactlyOneWinner(t *testing.T) {
|
||||||
|
eachStore(t, func(t *testing.T, s store.Store) {
|
||||||
|
ctx := context.Background()
|
||||||
|
const racers = 32
|
||||||
|
|
||||||
|
for round := range 20 {
|
||||||
|
id := newID(t)
|
||||||
|
if err := s.Put(ctx, id, "only-once", time.Minute); err != nil {
|
||||||
|
t.Fatalf("round %d Put: %v", round, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
wg sync.WaitGroup
|
||||||
|
mu sync.Mutex
|
||||||
|
wins int
|
||||||
|
gone int
|
||||||
|
othererr error
|
||||||
|
start = make(chan struct{})
|
||||||
|
)
|
||||||
|
for range racers {
|
||||||
|
wg.Add(1)
|
||||||
|
go func() {
|
||||||
|
defer wg.Done()
|
||||||
|
<-start // release them together to maximise the overlap
|
||||||
|
v, err := s.Take(ctx, id)
|
||||||
|
mu.Lock()
|
||||||
|
defer mu.Unlock()
|
||||||
|
switch {
|
||||||
|
case err == nil && v == "only-once":
|
||||||
|
wins++
|
||||||
|
case errors.Is(err, store.ErrGone):
|
||||||
|
gone++
|
||||||
|
default:
|
||||||
|
othererr = err
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
close(start)
|
||||||
|
wg.Wait()
|
||||||
|
|
||||||
|
if othererr != nil {
|
||||||
|
t.Fatalf("round %d: unexpected error from Take: %v", round, othererr)
|
||||||
|
}
|
||||||
|
if wins != 1 {
|
||||||
|
t.Fatalf("round %d: %d goroutines received the secret, want exactly 1 (%d saw gone)", round, wins, gone)
|
||||||
|
}
|
||||||
|
if gone != racers-1 {
|
||||||
|
t.Fatalf("round %d: %d saw gone, want %d", round, gone, racers-1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAnExpiredSecretIsGoneAndItsIDIsReusable(t *testing.T) {
|
||||||
|
// Driven by a fake clock so this does not sleep. The Redis half of the
|
||||||
|
// contract is covered by TestRedisHonoursTTL below, which uses a short real
|
||||||
|
// TTL because Redis owns that clock.
|
||||||
|
now := time.Now()
|
||||||
|
s := store.NewMemoryAt(func() time.Time { return now })
|
||||||
|
ctx := context.Background()
|
||||||
|
id := newID(t)
|
||||||
|
|
||||||
|
if err := s.Put(ctx, id, "vanishing", time.Minute); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
now = now.Add(time.Minute + time.Second)
|
||||||
|
|
||||||
|
if _, err := s.Take(ctx, id); !errors.Is(err, store.ErrGone) {
|
||||||
|
t.Fatalf("Take after expiry = %v, want ErrGone", err)
|
||||||
|
}
|
||||||
|
// Redis treats an expired key as absent, so Put must succeed here. A
|
||||||
|
// memory store that refused would pass tests Redis fails.
|
||||||
|
if err := s.Put(ctx, id, "reused", time.Minute); err != nil {
|
||||||
|
t.Fatalf("Put over an expired key = %v, want nil (Redis SET NX succeeds on an expired key)", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRedisHonoursTTL(t *testing.T) {
|
||||||
|
url := os.Getenv("HUSH_TEST_REDIS_URL")
|
||||||
|
if url == "" {
|
||||||
|
t.Skip("HUSH_TEST_REDIS_URL unset")
|
||||||
|
}
|
||||||
|
r, err := store.NewRedis(url)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() { _ = r.Close() })
|
||||||
|
|
||||||
|
ctx := context.Background()
|
||||||
|
id := newID(t)
|
||||||
|
// Redis EX takes whole seconds, so 1s is the shortest observable TTL.
|
||||||
|
if err := r.Put(ctx, id, "brief", time.Second); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
time.Sleep(1500 * time.Millisecond)
|
||||||
|
if _, err := r.Take(ctx, id); !errors.Is(err, store.ErrGone) {
|
||||||
|
t.Fatalf("Take after the TTL elapsed = %v, want ErrGone", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRedisRateLimitCountsWithinAWindowAndThenRefuses(t *testing.T) {
|
||||||
|
url := os.Getenv("HUSH_TEST_REDIS_URL")
|
||||||
|
if url == "" {
|
||||||
|
t.Skip("HUSH_TEST_REDIS_URL unset")
|
||||||
|
}
|
||||||
|
r, err := store.NewRedis(url)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() { _ = r.Close() })
|
||||||
|
|
||||||
|
ctx := context.Background()
|
||||||
|
// A unique key per run so a re-run is not throttled by the previous one.
|
||||||
|
id := newID(t)
|
||||||
|
key := "test-" + id.LogHandle()
|
||||||
|
|
||||||
|
for i := 1; i <= 3; i++ {
|
||||||
|
ok, _, err := r.AllowN(ctx, key, 3, 5*time.Second)
|
||||||
|
if err != nil || !ok {
|
||||||
|
t.Fatalf("call %d: AllowN = (%v, %v), want allowed", i, ok, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ok, retry, err := r.AllowN(ctx, key, 3, 5*time.Second)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if ok {
|
||||||
|
t.Fatal("the 4th call in a limit-3 window was allowed")
|
||||||
|
}
|
||||||
|
if retry <= 0 || retry > 5*time.Second {
|
||||||
|
t.Fatalf("retryAfter = %s, want a positive value inside the window", retry)
|
||||||
|
}
|
||||||
|
}
|
||||||
94
internal/web/templates/base.html
Normal file
94
internal/web/templates/base.html
Normal file
@ -0,0 +1,94 @@
|
|||||||
|
{{define "base.html"}}<!doctype html>
|
||||||
|
<html lang="en">
|
||||||
|
<head>
|
||||||
|
<meta charset="utf-8">
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||||
|
<title>hush</title>
|
||||||
|
<!-- No external origins: no CDN, no font host, no analytics. A third-party
|
||||||
|
script on this page could read the key out of the fragment, so the CSP
|
||||||
|
forbids one existing. 'unsafe-inline' covers the inline script and style
|
||||||
|
below, which are same-document and reviewed in this repo. -->
|
||||||
|
<meta http-equiv="Content-Security-Policy"
|
||||||
|
content="default-src 'none'; script-src 'unsafe-inline'; style-src 'unsafe-inline'; connect-src 'self'; form-action 'none'; base-uri 'none'; frame-ancestors 'none'">
|
||||||
|
<meta name="referrer" content="no-referrer">
|
||||||
|
<style>
|
||||||
|
:root{color-scheme:dark;--bg:#0b0d10;--fg:#e7ebf0;--dim:#8b95a3;--line:#1e242c;--accent:#7fd1ae;--warn:#f0b866;--bad:#e8737d}
|
||||||
|
*{box-sizing:border-box}
|
||||||
|
body{margin:0;background:var(--bg);color:var(--fg);font:15px/1.55 ui-monospace,SFMono-Regular,Menlo,monospace;display:flex;min-height:100vh;align-items:center;justify-content:center;padding:24px}
|
||||||
|
main{width:100%;max-width:620px}
|
||||||
|
h1{font-size:20px;margin:0 0 4px;letter-spacing:.02em}
|
||||||
|
h1 span{color:var(--accent)}
|
||||||
|
p.lede{color:var(--dim);margin:0 0 24px;font-size:13px}
|
||||||
|
label{display:block;font-size:12px;color:var(--dim);margin:16px 0 6px;text-transform:uppercase;letter-spacing:.08em}
|
||||||
|
textarea,input,select,button{font:inherit;color:var(--fg);background:#12161b;border:1px solid var(--line);border-radius:6px;padding:10px 12px;width:100%}
|
||||||
|
textarea{min-height:150px;resize:vertical}
|
||||||
|
button{background:var(--accent);color:#06231a;border:0;font-weight:600;cursor:pointer;margin-top:20px;padding:12px}
|
||||||
|
button:hover{filter:brightness(1.08)}
|
||||||
|
button:disabled{opacity:.5;cursor:not-allowed;filter:none}
|
||||||
|
button.secondary{background:#12161b;color:var(--fg);border:1px solid var(--line);font-weight:400}
|
||||||
|
.row{display:flex;gap:12px}
|
||||||
|
.row>*{flex:1}
|
||||||
|
.out{margin-top:24px;padding:14px;border:1px solid var(--line);border-radius:6px;background:#0f1418;word-break:break-all;font-size:13px}
|
||||||
|
.note{color:var(--dim);font-size:12px;margin-top:10px}
|
||||||
|
.err{color:var(--bad)}
|
||||||
|
.warn{color:var(--warn)}
|
||||||
|
.ok{color:var(--accent)}
|
||||||
|
.hide{display:none}
|
||||||
|
footer{margin-top:28px;color:var(--dim);font-size:11px;border-top:1px solid var(--line);padding-top:12px}
|
||||||
|
a{color:var(--dim)}
|
||||||
|
</style>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<main>
|
||||||
|
{{template "content" .}}
|
||||||
|
<footer>
|
||||||
|
Encrypted in your browser. The key travels in the link's <code>#fragment</code>,
|
||||||
|
which browsers never send to a server — hush stores ciphertext it cannot open.
|
||||||
|
The link works once.
|
||||||
|
</footer>
|
||||||
|
</main>
|
||||||
|
<script>
|
||||||
|
// Shared crypto. AES-256-GCM via WebCrypto; the key is generated here, never
|
||||||
|
// transmitted, and carried only in the URL fragment.
|
||||||
|
//
|
||||||
|
// b64u: base64url without padding, matching Go's base64.RawURLEncoding on the
|
||||||
|
// server so the wire format has exactly one spelling.
|
||||||
|
const b64u = {
|
||||||
|
enc(bytes) {
|
||||||
|
let s = ""; for (const b of bytes) s += String.fromCharCode(b);
|
||||||
|
return btoa(s).replace(/\+/g, "-").replace(/\//g, "_").replace(/=+$/, "");
|
||||||
|
},
|
||||||
|
dec(str) {
|
||||||
|
const p = str.replace(/-/g, "+").replace(/_/g, "/");
|
||||||
|
const raw = atob(p + "=".repeat((4 - (p.length % 4)) % 4));
|
||||||
|
const out = new Uint8Array(raw.length);
|
||||||
|
for (let i = 0; i < raw.length; i++) out[i] = raw.charCodeAt(i);
|
||||||
|
return out;
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
async function seal(plaintext) {
|
||||||
|
const key = await crypto.subtle.generateKey({ name: "AES-GCM", length: 256 }, true, ["encrypt", "decrypt"]);
|
||||||
|
// 96-bit nonce is the AES-GCM standard size and is generated per secret. It
|
||||||
|
// is prepended to the ciphertext rather than sent separately so a stored blob
|
||||||
|
// is self-contained.
|
||||||
|
const nonce = crypto.getRandomValues(new Uint8Array(12));
|
||||||
|
const body = new Uint8Array(await crypto.subtle.encrypt(
|
||||||
|
{ name: "AES-GCM", iv: nonce }, key, new TextEncoder().encode(plaintext)));
|
||||||
|
const blob = new Uint8Array(nonce.length + body.length);
|
||||||
|
blob.set(nonce, 0); blob.set(body, nonce.length);
|
||||||
|
const rawKey = new Uint8Array(await crypto.subtle.exportKey("raw", key));
|
||||||
|
return { ciphertext: b64u.enc(blob), key: b64u.enc(rawKey) };
|
||||||
|
}
|
||||||
|
|
||||||
|
async function open(ciphertext, keyStr) {
|
||||||
|
const blob = b64u.dec(ciphertext);
|
||||||
|
const key = await crypto.subtle.importKey("raw", b64u.dec(keyStr), "AES-GCM", false, ["decrypt"]);
|
||||||
|
const plain = await crypto.subtle.decrypt(
|
||||||
|
{ name: "AES-GCM", iv: blob.slice(0, 12) }, key, blob.slice(12));
|
||||||
|
return new TextDecoder().decode(plain);
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
{{template "script" .}}
|
||||||
|
</body>
|
||||||
|
</html>{{end}}
|
||||||
118
internal/web/templates/create.html
Normal file
118
internal/web/templates/create.html
Normal file
@ -0,0 +1,118 @@
|
|||||||
|
{{define "content"}}
|
||||||
|
<h1>hush<span>.</span></h1>
|
||||||
|
<p class="lede">Paste a secret. Get a link that works once.</p>
|
||||||
|
|
||||||
|
<div id="form">
|
||||||
|
<label for="secret">Secret</label>
|
||||||
|
<textarea id="secret" autofocus autocomplete="off" spellcheck="false"
|
||||||
|
placeholder="API key, password, connection string…"></textarea>
|
||||||
|
|
||||||
|
<div class="row">
|
||||||
|
<div>
|
||||||
|
<label for="ttl">Expires after</label>
|
||||||
|
<select id="ttl">
|
||||||
|
<option value="900">15 minutes</option>
|
||||||
|
<option value="3600">1 hour</option>
|
||||||
|
<option value="86400" selected>24 hours</option>
|
||||||
|
<option value="259200">3 days</option>
|
||||||
|
<option value="604800">7 days</option>
|
||||||
|
</select>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<button id="go">Create link</button>
|
||||||
|
<p class="note" id="msg"></p>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div id="result" class="hide">
|
||||||
|
<label>Your one-time link</label>
|
||||||
|
<div class="out" id="link"></div>
|
||||||
|
<div class="row">
|
||||||
|
<button id="copy">Copy link</button>
|
||||||
|
<button id="again" class="secondary">Create another</button>
|
||||||
|
</div>
|
||||||
|
<p class="note warn">
|
||||||
|
This is shown once and is not recoverable — hush cannot rebuild it, because
|
||||||
|
the key it contains was never sent to the server. Copy it now.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
{{end}}
|
||||||
|
|
||||||
|
{{define "script"}}
|
||||||
|
<script>
|
||||||
|
const $ = (id) => document.getElementById(id);
|
||||||
|
const MAX = {{.MaxCiphertextBytes}};
|
||||||
|
|
||||||
|
async function create() {
|
||||||
|
const text = $("secret").value;
|
||||||
|
if (!text) { return fail("Nothing to send."); }
|
||||||
|
|
||||||
|
$("go").disabled = true;
|
||||||
|
$("msg").className = "note";
|
||||||
|
$("msg").textContent = "Encrypting…";
|
||||||
|
|
||||||
|
let sealed;
|
||||||
|
try {
|
||||||
|
sealed = await seal(text);
|
||||||
|
} catch (e) {
|
||||||
|
return fail("Encryption failed in this browser: " + e.message);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check the size of the CIPHERTEXT, which is what the server caps, so the
|
||||||
|
// message names the same number the server would reject on.
|
||||||
|
if (sealed.ciphertext.length > MAX) {
|
||||||
|
return fail("Too large: " + sealed.ciphertext.length + " > " + MAX + " bytes encrypted.");
|
||||||
|
}
|
||||||
|
|
||||||
|
let res, body;
|
||||||
|
try {
|
||||||
|
res = await fetch("/api/secrets", {
|
||||||
|
method: "POST",
|
||||||
|
headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify({ ciphertext: sealed.ciphertext, ttl_seconds: Number($("ttl").value) }),
|
||||||
|
});
|
||||||
|
body = await res.json();
|
||||||
|
} catch (e) {
|
||||||
|
return fail("Could not reach hush: " + e.message);
|
||||||
|
}
|
||||||
|
if (!res.ok) {
|
||||||
|
return fail(body && body.error ? body.error.message : "Server refused the secret (" + res.status + ").");
|
||||||
|
}
|
||||||
|
|
||||||
|
// The key goes in the fragment and ONLY in the fragment. Assembling the URL
|
||||||
|
// here — not on the server — is what keeps the key client-side.
|
||||||
|
const url = location.origin + "/s/" + body.id + "#" + sealed.key;
|
||||||
|
$("link").textContent = url;
|
||||||
|
$("form").classList.add("hide");
|
||||||
|
$("result").classList.remove("hide");
|
||||||
|
$("copy").focus();
|
||||||
|
}
|
||||||
|
|
||||||
|
function fail(m) {
|
||||||
|
$("go").disabled = false;
|
||||||
|
$("msg").className = "note err";
|
||||||
|
$("msg").textContent = m;
|
||||||
|
}
|
||||||
|
|
||||||
|
$("go").addEventListener("click", create);
|
||||||
|
$("secret").addEventListener("keydown", (e) => {
|
||||||
|
if ((e.metaKey || e.ctrlKey) && e.key === "Enter") create();
|
||||||
|
});
|
||||||
|
$("copy").addEventListener("click", async () => {
|
||||||
|
try {
|
||||||
|
await navigator.clipboard.writeText($("link").textContent);
|
||||||
|
$("copy").textContent = "Copied";
|
||||||
|
setTimeout(() => ($("copy").textContent = "Copy link"), 1500);
|
||||||
|
} catch {
|
||||||
|
// Clipboard needs a permission this browser withheld; selecting the text
|
||||||
|
// is a working fallback rather than a dead button.
|
||||||
|
const r = document.createRange();
|
||||||
|
r.selectNodeContents($("link"));
|
||||||
|
getSelection().removeAllRanges();
|
||||||
|
getSelection().addRange(r);
|
||||||
|
$("copy").textContent = "Selected — press copy";
|
||||||
|
}
|
||||||
|
});
|
||||||
|
$("again").addEventListener("click", () => location.assign("/"));
|
||||||
|
</script>
|
||||||
|
{{end}}
|
||||||
118
internal/web/templates/reveal.html
Normal file
118
internal/web/templates/reveal.html
Normal file
@ -0,0 +1,118 @@
|
|||||||
|
{{define "content"}}
|
||||||
|
<h1>hush<span>.</span></h1>
|
||||||
|
<p class="lede">Someone sent you a secret. It can be opened once.</p>
|
||||||
|
|
||||||
|
<div id="gate">
|
||||||
|
<p class="note">
|
||||||
|
Nothing has been read yet. This page has not touched the secret — opening it
|
||||||
|
is the button below, so a link preview in a chat app cannot consume it.
|
||||||
|
</p>
|
||||||
|
<button id="go">Reveal the secret</button>
|
||||||
|
<p class="note" id="msg"></p>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div id="result" class="hide">
|
||||||
|
<label>The secret</label>
|
||||||
|
<div class="out" id="plain"></div>
|
||||||
|
<div class="row">
|
||||||
|
<button id="copy">Copy</button>
|
||||||
|
</div>
|
||||||
|
<p class="note warn">
|
||||||
|
Destroyed. Reloading this page will not show it again — copy it now.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
{{end}}
|
||||||
|
|
||||||
|
{{define "script"}}
|
||||||
|
<script>
|
||||||
|
const $ = (id) => document.getElementById(id);
|
||||||
|
|
||||||
|
// The id comes from the path and the key from the fragment. Neither is
|
||||||
|
// interpolated by the server, so this page renders identically for every secret
|
||||||
|
// and reflects nothing.
|
||||||
|
const id = location.pathname.replace(/^\/s\//, "");
|
||||||
|
const key = location.hash.slice(1);
|
||||||
|
|
||||||
|
if (!key) {
|
||||||
|
$("go").disabled = true;
|
||||||
|
$("msg").className = "note err";
|
||||||
|
$("msg").textContent =
|
||||||
|
"This link is missing its key — the part after '#'. Chat apps and email " +
|
||||||
|
"clients sometimes truncate it. Ask the sender for the full link; the " +
|
||||||
|
"secret is intact and has not been opened.";
|
||||||
|
}
|
||||||
|
|
||||||
|
async function reveal() {
|
||||||
|
$("go").disabled = true;
|
||||||
|
$("msg").className = "note";
|
||||||
|
$("msg").textContent = "Opening…";
|
||||||
|
|
||||||
|
let res, body;
|
||||||
|
try {
|
||||||
|
res = await fetch("/api/secrets/" + encodeURIComponent(id) + "/reveal", { method: "POST" });
|
||||||
|
body = await res.json();
|
||||||
|
} catch (e) {
|
||||||
|
// The secret is very likely consumed at this point, so do not offer a retry
|
||||||
|
// that would report "gone" and read as a lie about what happened.
|
||||||
|
$("msg").className = "note err";
|
||||||
|
$("msg").textContent = "Could not reach hush: " + e.message +
|
||||||
|
". If the request left your browser, the secret is already destroyed.";
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (res.status === 410) {
|
||||||
|
$("msg").className = "note err";
|
||||||
|
$("msg").textContent =
|
||||||
|
"Gone. This link was already opened, expired, or never existed. " +
|
||||||
|
"If you have not opened it yourself, assume someone else did and ask the " +
|
||||||
|
"sender to rotate the secret.";
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (!res.ok) {
|
||||||
|
$("go").disabled = false;
|
||||||
|
$("msg").className = "note err";
|
||||||
|
$("msg").textContent = body && body.error ? body.error.message : "Server error (" + res.status + ").";
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
let text;
|
||||||
|
try {
|
||||||
|
text = await open(body.ciphertext, key);
|
||||||
|
} catch {
|
||||||
|
// Decryption failed AFTER the server destroyed the ciphertext, so there is
|
||||||
|
// nothing to retry. Say so, because the alternative is a user reloading
|
||||||
|
// forever against a secret that no longer exists.
|
||||||
|
$("msg").className = "note err";
|
||||||
|
$("msg").textContent =
|
||||||
|
"The key in this link does not open this secret, and the ciphertext has " +
|
||||||
|
"now been destroyed. The link was probably altered in transit. Ask the " +
|
||||||
|
"sender to create a new one.";
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
$("plain").textContent = text;
|
||||||
|
$("gate").classList.add("hide");
|
||||||
|
$("result").classList.remove("hide");
|
||||||
|
// Drop the key from the address bar so a screenshot, a shoulder-surfer or a
|
||||||
|
// later copy of the URL does not carry it. The secret is already destroyed
|
||||||
|
// server-side, so this only reduces incidental exposure.
|
||||||
|
history.replaceState(null, "", location.pathname);
|
||||||
|
$("copy").focus();
|
||||||
|
}
|
||||||
|
|
||||||
|
$("go").addEventListener("click", reveal);
|
||||||
|
$("copy").addEventListener("click", async () => {
|
||||||
|
try {
|
||||||
|
await navigator.clipboard.writeText($("plain").textContent);
|
||||||
|
$("copy").textContent = "Copied";
|
||||||
|
setTimeout(() => ($("copy").textContent = "Copy"), 1500);
|
||||||
|
} catch {
|
||||||
|
const r = document.createRange();
|
||||||
|
r.selectNodeContents($("plain"));
|
||||||
|
getSelection().removeAllRanges();
|
||||||
|
getSelection().addRange(r);
|
||||||
|
$("copy").textContent = "Selected — press copy";
|
||||||
|
}
|
||||||
|
});
|
||||||
|
</script>
|
||||||
|
{{end}}
|
||||||
73
internal/web/web.go
Normal file
73
internal/web/web.go
Normal file
@ -0,0 +1,73 @@
|
|||||||
|
// Package web serves hush's two pages. Both are static: they read no storage,
|
||||||
|
// so a link previewer fetching either one cannot destroy a secret.
|
||||||
|
package web
|
||||||
|
|
||||||
|
import (
|
||||||
|
"embed"
|
||||||
|
"fmt"
|
||||||
|
"html/template"
|
||||||
|
"net/http"
|
||||||
|
)
|
||||||
|
|
||||||
|
//go:embed templates/*.html
|
||||||
|
var files embed.FS
|
||||||
|
|
||||||
|
// Pages renders the create and reveal pages. Templates are embedded, so the
|
||||||
|
// container carries no template directory to go missing at runtime.
|
||||||
|
type Pages struct {
|
||||||
|
create *template.Template
|
||||||
|
reveal *template.Template
|
||||||
|
}
|
||||||
|
|
||||||
|
// Data is everything a page needs. The limits are passed through so the browser
|
||||||
|
// enforces the same caps the server does and a user learns their secret is too
|
||||||
|
// large before uploading it, not after.
|
||||||
|
type Data struct {
|
||||||
|
MaxCiphertextBytes int
|
||||||
|
DefaultTTLSeconds int
|
||||||
|
MinTTLSeconds int
|
||||||
|
MaxTTLSeconds int
|
||||||
|
}
|
||||||
|
|
||||||
|
// New parses the embedded templates. It fails at boot rather than on first
|
||||||
|
// request: a template error is a build defect and should not wait for traffic
|
||||||
|
// to surface.
|
||||||
|
func New() (*Pages, error) {
|
||||||
|
create, err := template.ParseFS(files, "templates/base.html", "templates/create.html")
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("parse create template: %w", err)
|
||||||
|
}
|
||||||
|
reveal, err := template.ParseFS(files, "templates/base.html", "templates/reveal.html")
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("parse reveal template: %w", err)
|
||||||
|
}
|
||||||
|
return &Pages{create: create, reveal: reveal}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Create writes the create page.
|
||||||
|
func (p *Pages) Create(w http.ResponseWriter, d Data) error {
|
||||||
|
return render(w, p.create, d)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Reveal writes the reveal page.
|
||||||
|
//
|
||||||
|
// The secret id is NOT passed in and is NOT interpolated into the HTML. The
|
||||||
|
// page reads it from location.pathname in the browser, alongside the key it
|
||||||
|
// reads from location.hash. That keeps the template free of any value that
|
||||||
|
// could be reflected, and means this handler needs no escaping decisions about
|
||||||
|
// a capability.
|
||||||
|
func (p *Pages) Reveal(w http.ResponseWriter, d Data) error {
|
||||||
|
return render(w, p.reveal, d)
|
||||||
|
}
|
||||||
|
|
||||||
|
func render(w http.ResponseWriter, t *template.Template, d Data) error {
|
||||||
|
// no-store on both pages: a cached create page is harmless, but a cached
|
||||||
|
// reveal page in a shared proxy would be a copy of a one-time URL.
|
||||||
|
w.Header().Set("Cache-Control", "no-store, max-age=0")
|
||||||
|
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||||
|
// Referrer-Policy is load-bearing here, not boilerplate: without it a click
|
||||||
|
// on any link from the reveal page could send the full URL — including the
|
||||||
|
// fragment-adjacent path — to a third party.
|
||||||
|
w.Header().Set("Referrer-Policy", "no-referrer")
|
||||||
|
return t.ExecuteTemplate(w, "base.html", d)
|
||||||
|
}
|
||||||
94
scripts/smoke.sh
Executable file
94
scripts/smoke.sh
Executable file
@ -0,0 +1,94 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# End-to-end proof against a RUNNING hushd, doing the real client-side crypto.
|
||||||
|
#
|
||||||
|
# This is not a mock: it generates an AES-256-GCM key, encrypts a plaintext,
|
||||||
|
# posts only the ciphertext, reveals it once, decrypts it, and compares. Then it
|
||||||
|
# checks the three properties the design rests on:
|
||||||
|
#
|
||||||
|
# 1. a second reveal is 410 gone
|
||||||
|
# 2. GET on the reveal page does NOT consume the secret (link previewers)
|
||||||
|
# 3. a malformed id and a missing id are indistinguishable
|
||||||
|
#
|
||||||
|
# Usage: BASE=http://127.0.0.1:18500 ./scripts/smoke.sh
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
BASE="${BASE:-http://127.0.0.1:18500}"
|
||||||
|
PLAINTEXT="${PLAINTEXT:-hunter2-$(date +%s)-$RANDOM}"
|
||||||
|
|
||||||
|
pass() { printf ' \033[32mok\033[0m %s\n' "$1"; }
|
||||||
|
fail() { printf ' \033[31mFAIL\033[0m %s\n' "$1"; exit 1; }
|
||||||
|
|
||||||
|
command -v python3 >/dev/null || { echo "python3 is required for the crypto half"; exit 1; }
|
||||||
|
|
||||||
|
echo "hush smoke against $BASE"
|
||||||
|
|
||||||
|
# --- encrypt exactly as the browser does -----------------------------------
|
||||||
|
# AES-256-GCM, 96-bit nonce prepended, base64url unpadded. Same wire format as
|
||||||
|
# templates/base.html, which is what makes this a real client.
|
||||||
|
read -r CIPHERTEXT KEY <<<"$(python3 - "$PLAINTEXT" <<'PY'
|
||||||
|
import base64, os, sys
|
||||||
|
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
|
||||||
|
key = AESGCM.generate_key(bit_length=256)
|
||||||
|
nonce = os.urandom(12)
|
||||||
|
blob = nonce + AESGCM(key).encrypt(nonce, sys.argv[1].encode(), None)
|
||||||
|
b = lambda x: base64.urlsafe_b64encode(x).decode().rstrip("=")
|
||||||
|
print(b(blob), b(key))
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
[ -n "$CIPHERTEXT" ] || fail "could not encrypt (is 'cryptography' installed? pip install cryptography)"
|
||||||
|
pass "encrypted client-side (${#CIPHERTEXT} bytes of ciphertext)"
|
||||||
|
|
||||||
|
# --- create ----------------------------------------------------------------
|
||||||
|
CREATE=$(curl -sS -X POST "$BASE/api/secrets" -H 'Content-Type: application/json' \
|
||||||
|
-d "{\"ciphertext\":\"$CIPHERTEXT\",\"ttl_seconds\":900}")
|
||||||
|
ID=$(printf '%s' "$CREATE" | python3 -c 'import json,sys; print(json.load(sys.stdin).get("id",""))')
|
||||||
|
[ -n "$ID" ] || fail "create returned no id: $CREATE"
|
||||||
|
pass "created — id is ${#ID} chars"
|
||||||
|
|
||||||
|
# --- the previewer property, checked BEFORE revealing ----------------------
|
||||||
|
for _ in 1 2 3; do
|
||||||
|
code=$(curl -sS -o /dev/null -w '%{http_code}' "$BASE/s/$ID")
|
||||||
|
[ "$code" = "200" ] || fail "GET /s/{id} returned $code"
|
||||||
|
done
|
||||||
|
pass "GET on the reveal page x3 — Slack/Outlook previews are harmless"
|
||||||
|
|
||||||
|
# --- reveal once and decrypt ----------------------------------------------
|
||||||
|
REVEAL=$(curl -sS -X POST "$BASE/api/secrets/$ID/reveal")
|
||||||
|
GOT_CT=$(printf '%s' "$REVEAL" | python3 -c 'import json,sys; print(json.load(sys.stdin).get("ciphertext",""))')
|
||||||
|
[ -n "$GOT_CT" ] || fail "reveal returned no ciphertext: $REVEAL"
|
||||||
|
|
||||||
|
DECRYPTED=$(python3 - "$GOT_CT" "$KEY" <<'PY'
|
||||||
|
import base64, sys
|
||||||
|
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
|
||||||
|
u = lambda s: base64.urlsafe_b64decode(s + "=" * (-len(s) % 4))
|
||||||
|
blob, key = u(sys.argv[1]), u(sys.argv[2])
|
||||||
|
sys.stdout.write(AESGCM(key).decrypt(blob[:12], blob[12:], None).decode())
|
||||||
|
PY
|
||||||
|
)
|
||||||
|
[ "$DECRYPTED" = "$PLAINTEXT" ] || fail "decrypted to '$DECRYPTED', expected '$PLAINTEXT'"
|
||||||
|
pass "revealed and decrypted — round trip matches"
|
||||||
|
|
||||||
|
# --- and it is gone --------------------------------------------------------
|
||||||
|
code=$(curl -sS -o /tmp/hush-gone.$$ -w '%{http_code}' -X POST "$BASE/api/secrets/$ID/reveal")
|
||||||
|
[ "$code" = "410" ] || fail "second reveal returned $code, want 410"
|
||||||
|
GONE_BODY=$(cat /tmp/hush-gone.$$); rm -f /tmp/hush-gone.$$
|
||||||
|
pass "second reveal is 410 gone"
|
||||||
|
|
||||||
|
# --- missing and malformed are one response --------------------------------
|
||||||
|
MISSING=$(curl -sS -X POST "$BASE/api/secrets/$(python3 -c 'print("A"*43)')/reveal")
|
||||||
|
MALFORMED=$(curl -sS -X POST "$BASE/api/secrets/not-an-id/reveal")
|
||||||
|
for body in "$MISSING" "$MALFORMED"; do
|
||||||
|
norm=$(printf '%s' "$body" | python3 -c 'import json,sys; e=json.load(sys.stdin)["error"]; print(e["code"], e["message"])')
|
||||||
|
gone=$(printf '%s' "$GONE_BODY" | python3 -c 'import json,sys; e=json.load(sys.stdin)["error"]; print(e["code"], e["message"])')
|
||||||
|
[ "$norm" = "$gone" ] || fail "responses differ between causes: '$norm' vs '$gone'"
|
||||||
|
done
|
||||||
|
pass "revealed, expired, missing and malformed are one indistinguishable response"
|
||||||
|
|
||||||
|
# --- the server must refuse plaintext -------------------------------------
|
||||||
|
code=$(curl -sS -o /dev/null -w '%{http_code}' -X POST "$BASE/api/secrets" \
|
||||||
|
-H 'Content-Type: application/json' -d '{"secret":"hunter2"}')
|
||||||
|
[ "$code" != "201" ] || fail "the server ACCEPTED a plaintext field — the zero-knowledge claim is broken"
|
||||||
|
pass "a plaintext field is refused ($code)"
|
||||||
|
|
||||||
|
echo
|
||||||
|
printf '\033[32mall smoke checks passed\033[0m\n'
|
||||||
20
vendor/github.com/beorn7/perks/LICENSE
generated
vendored
Normal file
20
vendor/github.com/beorn7/perks/LICENSE
generated
vendored
Normal file
@ -0,0 +1,20 @@
|
|||||||
|
Copyright (C) 2013 Blake Mizerany
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining
|
||||||
|
a copy of this software and associated documentation files (the
|
||||||
|
"Software"), to deal in the Software without restriction, including
|
||||||
|
without limitation the rights to use, copy, modify, merge, publish,
|
||||||
|
distribute, sublicense, and/or sell copies of the Software, and to
|
||||||
|
permit persons to whom the Software is furnished to do so, subject to
|
||||||
|
the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be
|
||||||
|
included in all copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||||
|
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
||||||
|
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||||
|
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
||||||
|
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
||||||
|
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||||
|
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||||
2388
vendor/github.com/beorn7/perks/quantile/exampledata.txt
generated
vendored
Normal file
2388
vendor/github.com/beorn7/perks/quantile/exampledata.txt
generated
vendored
Normal file
File diff suppressed because it is too large
Load Diff
316
vendor/github.com/beorn7/perks/quantile/stream.go
generated
vendored
Normal file
316
vendor/github.com/beorn7/perks/quantile/stream.go
generated
vendored
Normal file
@ -0,0 +1,316 @@
|
|||||||
|
// Package quantile computes approximate quantiles over an unbounded data
|
||||||
|
// stream within low memory and CPU bounds.
|
||||||
|
//
|
||||||
|
// A small amount of accuracy is traded to achieve the above properties.
|
||||||
|
//
|
||||||
|
// Multiple streams can be merged before calling Query to generate a single set
|
||||||
|
// of results. This is meaningful when the streams represent the same type of
|
||||||
|
// data. See Merge and Samples.
|
||||||
|
//
|
||||||
|
// For more detailed information about the algorithm used, see:
|
||||||
|
//
|
||||||
|
// Effective Computation of Biased Quantiles over Data Streams
|
||||||
|
//
|
||||||
|
// http://www.cs.rutgers.edu/~muthu/bquant.pdf
|
||||||
|
package quantile
|
||||||
|
|
||||||
|
import (
|
||||||
|
"math"
|
||||||
|
"sort"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Sample holds an observed value and meta information for compression. JSON
|
||||||
|
// tags have been added for convenience.
|
||||||
|
type Sample struct {
|
||||||
|
Value float64 `json:",string"`
|
||||||
|
Width float64 `json:",string"`
|
||||||
|
Delta float64 `json:",string"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Samples represents a slice of samples. It implements sort.Interface.
|
||||||
|
type Samples []Sample
|
||||||
|
|
||||||
|
func (a Samples) Len() int { return len(a) }
|
||||||
|
func (a Samples) Less(i, j int) bool { return a[i].Value < a[j].Value }
|
||||||
|
func (a Samples) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
|
||||||
|
|
||||||
|
type invariant func(s *stream, r float64) float64
|
||||||
|
|
||||||
|
// NewLowBiased returns an initialized Stream for low-biased quantiles
|
||||||
|
// (e.g. 0.01, 0.1, 0.5) where the needed quantiles are not known a priori, but
|
||||||
|
// error guarantees can still be given even for the lower ranks of the data
|
||||||
|
// distribution.
|
||||||
|
//
|
||||||
|
// The provided epsilon is a relative error, i.e. the true quantile of a value
|
||||||
|
// returned by a query is guaranteed to be within (1±Epsilon)*Quantile.
|
||||||
|
//
|
||||||
|
// See http://www.cs.rutgers.edu/~muthu/bquant.pdf for time, space, and error
|
||||||
|
// properties.
|
||||||
|
func NewLowBiased(epsilon float64) *Stream {
|
||||||
|
ƒ := func(s *stream, r float64) float64 {
|
||||||
|
return 2 * epsilon * r
|
||||||
|
}
|
||||||
|
return newStream(ƒ)
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewHighBiased returns an initialized Stream for high-biased quantiles
|
||||||
|
// (e.g. 0.01, 0.1, 0.5) where the needed quantiles are not known a priori, but
|
||||||
|
// error guarantees can still be given even for the higher ranks of the data
|
||||||
|
// distribution.
|
||||||
|
//
|
||||||
|
// The provided epsilon is a relative error, i.e. the true quantile of a value
|
||||||
|
// returned by a query is guaranteed to be within 1-(1±Epsilon)*(1-Quantile).
|
||||||
|
//
|
||||||
|
// See http://www.cs.rutgers.edu/~muthu/bquant.pdf for time, space, and error
|
||||||
|
// properties.
|
||||||
|
func NewHighBiased(epsilon float64) *Stream {
|
||||||
|
ƒ := func(s *stream, r float64) float64 {
|
||||||
|
return 2 * epsilon * (s.n - r)
|
||||||
|
}
|
||||||
|
return newStream(ƒ)
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewTargeted returns an initialized Stream concerned with a particular set of
|
||||||
|
// quantile values that are supplied a priori. Knowing these a priori reduces
|
||||||
|
// space and computation time. The targets map maps the desired quantiles to
|
||||||
|
// their absolute errors, i.e. the true quantile of a value returned by a query
|
||||||
|
// is guaranteed to be within (Quantile±Epsilon).
|
||||||
|
//
|
||||||
|
// See http://www.cs.rutgers.edu/~muthu/bquant.pdf for time, space, and error properties.
|
||||||
|
func NewTargeted(targetMap map[float64]float64) *Stream {
|
||||||
|
// Convert map to slice to avoid slow iterations on a map.
|
||||||
|
// ƒ is called on the hot path, so converting the map to a slice
|
||||||
|
// beforehand results in significant CPU savings.
|
||||||
|
targets := targetMapToSlice(targetMap)
|
||||||
|
|
||||||
|
ƒ := func(s *stream, r float64) float64 {
|
||||||
|
var m = math.MaxFloat64
|
||||||
|
var f float64
|
||||||
|
for _, t := range targets {
|
||||||
|
if t.quantile*s.n <= r {
|
||||||
|
f = (2 * t.epsilon * r) / t.quantile
|
||||||
|
} else {
|
||||||
|
f = (2 * t.epsilon * (s.n - r)) / (1 - t.quantile)
|
||||||
|
}
|
||||||
|
if f < m {
|
||||||
|
m = f
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
return newStream(ƒ)
|
||||||
|
}
|
||||||
|
|
||||||
|
type target struct {
|
||||||
|
quantile float64
|
||||||
|
epsilon float64
|
||||||
|
}
|
||||||
|
|
||||||
|
func targetMapToSlice(targetMap map[float64]float64) []target {
|
||||||
|
targets := make([]target, 0, len(targetMap))
|
||||||
|
|
||||||
|
for quantile, epsilon := range targetMap {
|
||||||
|
t := target{
|
||||||
|
quantile: quantile,
|
||||||
|
epsilon: epsilon,
|
||||||
|
}
|
||||||
|
targets = append(targets, t)
|
||||||
|
}
|
||||||
|
|
||||||
|
return targets
|
||||||
|
}
|
||||||
|
|
||||||
|
// Stream computes quantiles for a stream of float64s. It is not thread-safe by
|
||||||
|
// design. Take care when using across multiple goroutines.
|
||||||
|
type Stream struct {
|
||||||
|
*stream
|
||||||
|
b Samples
|
||||||
|
sorted bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func newStream(ƒ invariant) *Stream {
|
||||||
|
x := &stream{ƒ: ƒ}
|
||||||
|
return &Stream{x, make(Samples, 0, 500), true}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Insert inserts v into the stream.
|
||||||
|
func (s *Stream) Insert(v float64) {
|
||||||
|
s.insert(Sample{Value: v, Width: 1})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Stream) insert(sample Sample) {
|
||||||
|
s.b = append(s.b, sample)
|
||||||
|
s.sorted = false
|
||||||
|
if len(s.b) == cap(s.b) {
|
||||||
|
s.flush()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Query returns the computed qth percentiles value. If s was created with
|
||||||
|
// NewTargeted, and q is not in the set of quantiles provided a priori, Query
|
||||||
|
// will return an unspecified result.
|
||||||
|
func (s *Stream) Query(q float64) float64 {
|
||||||
|
if !s.flushed() {
|
||||||
|
// Fast path when there hasn't been enough data for a flush;
|
||||||
|
// this also yields better accuracy for small sets of data.
|
||||||
|
l := len(s.b)
|
||||||
|
if l == 0 {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
i := int(math.Ceil(float64(l) * q))
|
||||||
|
if i > 0 {
|
||||||
|
i -= 1
|
||||||
|
}
|
||||||
|
s.maybeSort()
|
||||||
|
return s.b[i].Value
|
||||||
|
}
|
||||||
|
s.flush()
|
||||||
|
return s.stream.query(q)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Merge merges samples into the underlying streams samples. This is handy when
|
||||||
|
// merging multiple streams from separate threads, database shards, etc.
|
||||||
|
//
|
||||||
|
// ATTENTION: This method is broken and does not yield correct results. The
|
||||||
|
// underlying algorithm is not capable of merging streams correctly.
|
||||||
|
func (s *Stream) Merge(samples Samples) {
|
||||||
|
sort.Sort(samples)
|
||||||
|
s.stream.merge(samples)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Reset reinitializes and clears the list reusing the samples buffer memory.
|
||||||
|
func (s *Stream) Reset() {
|
||||||
|
s.stream.reset()
|
||||||
|
s.b = s.b[:0]
|
||||||
|
}
|
||||||
|
|
||||||
|
// Samples returns stream samples held by s.
|
||||||
|
func (s *Stream) Samples() Samples {
|
||||||
|
if !s.flushed() {
|
||||||
|
return s.b
|
||||||
|
}
|
||||||
|
s.flush()
|
||||||
|
return s.stream.samples()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Count returns the total number of samples observed in the stream
|
||||||
|
// since initialization.
|
||||||
|
func (s *Stream) Count() int {
|
||||||
|
return len(s.b) + s.stream.count()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Stream) flush() {
|
||||||
|
s.maybeSort()
|
||||||
|
s.stream.merge(s.b)
|
||||||
|
s.b = s.b[:0]
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Stream) maybeSort() {
|
||||||
|
if !s.sorted {
|
||||||
|
s.sorted = true
|
||||||
|
sort.Sort(s.b)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Stream) flushed() bool {
|
||||||
|
return len(s.stream.l) > 0
|
||||||
|
}
|
||||||
|
|
||||||
|
type stream struct {
|
||||||
|
n float64
|
||||||
|
l []Sample
|
||||||
|
ƒ invariant
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *stream) reset() {
|
||||||
|
s.l = s.l[:0]
|
||||||
|
s.n = 0
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *stream) insert(v float64) {
|
||||||
|
s.merge(Samples{{v, 1, 0}})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *stream) merge(samples Samples) {
|
||||||
|
// TODO(beorn7): This tries to merge not only individual samples, but
|
||||||
|
// whole summaries. The paper doesn't mention merging summaries at
|
||||||
|
// all. Unittests show that the merging is inaccurate. Find out how to
|
||||||
|
// do merges properly.
|
||||||
|
var r float64
|
||||||
|
i := 0
|
||||||
|
for _, sample := range samples {
|
||||||
|
for ; i < len(s.l); i++ {
|
||||||
|
c := s.l[i]
|
||||||
|
if c.Value > sample.Value {
|
||||||
|
// Insert at position i.
|
||||||
|
s.l = append(s.l, Sample{})
|
||||||
|
copy(s.l[i+1:], s.l[i:])
|
||||||
|
s.l[i] = Sample{
|
||||||
|
sample.Value,
|
||||||
|
sample.Width,
|
||||||
|
math.Max(sample.Delta, math.Floor(s.ƒ(s, r))-1),
|
||||||
|
// TODO(beorn7): How to calculate delta correctly?
|
||||||
|
}
|
||||||
|
i++
|
||||||
|
goto inserted
|
||||||
|
}
|
||||||
|
r += c.Width
|
||||||
|
}
|
||||||
|
s.l = append(s.l, Sample{sample.Value, sample.Width, 0})
|
||||||
|
i++
|
||||||
|
inserted:
|
||||||
|
s.n += sample.Width
|
||||||
|
r += sample.Width
|
||||||
|
}
|
||||||
|
s.compress()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *stream) count() int {
|
||||||
|
return int(s.n)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *stream) query(q float64) float64 {
|
||||||
|
t := math.Ceil(q * s.n)
|
||||||
|
t += math.Ceil(s.ƒ(s, t) / 2)
|
||||||
|
p := s.l[0]
|
||||||
|
var r float64
|
||||||
|
for _, c := range s.l[1:] {
|
||||||
|
r += p.Width
|
||||||
|
if r+c.Width+c.Delta > t {
|
||||||
|
return p.Value
|
||||||
|
}
|
||||||
|
p = c
|
||||||
|
}
|
||||||
|
return p.Value
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *stream) compress() {
|
||||||
|
if len(s.l) < 2 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
x := s.l[len(s.l)-1]
|
||||||
|
xi := len(s.l) - 1
|
||||||
|
r := s.n - 1 - x.Width
|
||||||
|
|
||||||
|
for i := len(s.l) - 2; i >= 0; i-- {
|
||||||
|
c := s.l[i]
|
||||||
|
if c.Width+x.Width+x.Delta <= s.ƒ(s, r) {
|
||||||
|
x.Width += c.Width
|
||||||
|
s.l[xi] = x
|
||||||
|
// Remove element at i.
|
||||||
|
copy(s.l[i:], s.l[i+1:])
|
||||||
|
s.l = s.l[:len(s.l)-1]
|
||||||
|
xi -= 1
|
||||||
|
} else {
|
||||||
|
x = c
|
||||||
|
xi = i
|
||||||
|
}
|
||||||
|
r -= c.Width
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *stream) samples() Samples {
|
||||||
|
samples := make(Samples, len(s.l))
|
||||||
|
copy(samples, s.l)
|
||||||
|
return samples
|
||||||
|
}
|
||||||
22
vendor/github.com/cespare/xxhash/v2/LICENSE.txt
generated
vendored
Normal file
22
vendor/github.com/cespare/xxhash/v2/LICENSE.txt
generated
vendored
Normal file
@ -0,0 +1,22 @@
|
|||||||
|
Copyright (c) 2016 Caleb Spare
|
||||||
|
|
||||||
|
MIT License
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining
|
||||||
|
a copy of this software and associated documentation files (the
|
||||||
|
"Software"), to deal in the Software without restriction, including
|
||||||
|
without limitation the rights to use, copy, modify, merge, publish,
|
||||||
|
distribute, sublicense, and/or sell copies of the Software, and to
|
||||||
|
permit persons to whom the Software is furnished to do so, subject to
|
||||||
|
the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be
|
||||||
|
included in all copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||||
|
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
||||||
|
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||||
|
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
||||||
|
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
||||||
|
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||||
|
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||||
74
vendor/github.com/cespare/xxhash/v2/README.md
generated
vendored
Normal file
74
vendor/github.com/cespare/xxhash/v2/README.md
generated
vendored
Normal file
@ -0,0 +1,74 @@
|
|||||||
|
# xxhash
|
||||||
|
|
||||||
|
[](https://pkg.go.dev/github.com/cespare/xxhash/v2)
|
||||||
|
[](https://github.com/cespare/xxhash/actions/workflows/test.yml)
|
||||||
|
|
||||||
|
xxhash is a Go implementation of the 64-bit [xxHash] algorithm, XXH64. This is a
|
||||||
|
high-quality hashing algorithm that is much faster than anything in the Go
|
||||||
|
standard library.
|
||||||
|
|
||||||
|
This package provides a straightforward API:
|
||||||
|
|
||||||
|
```
|
||||||
|
func Sum64(b []byte) uint64
|
||||||
|
func Sum64String(s string) uint64
|
||||||
|
type Digest struct{ ... }
|
||||||
|
func New() *Digest
|
||||||
|
```
|
||||||
|
|
||||||
|
The `Digest` type implements hash.Hash64. Its key methods are:
|
||||||
|
|
||||||
|
```
|
||||||
|
func (*Digest) Write([]byte) (int, error)
|
||||||
|
func (*Digest) WriteString(string) (int, error)
|
||||||
|
func (*Digest) Sum64() uint64
|
||||||
|
```
|
||||||
|
|
||||||
|
The package is written with optimized pure Go and also contains even faster
|
||||||
|
assembly implementations for amd64 and arm64. If desired, the `purego` build tag
|
||||||
|
opts into using the Go code even on those architectures.
|
||||||
|
|
||||||
|
[xxHash]: http://cyan4973.github.io/xxHash/
|
||||||
|
|
||||||
|
## Compatibility
|
||||||
|
|
||||||
|
This package is in a module and the latest code is in version 2 of the module.
|
||||||
|
You need a version of Go with at least "minimal module compatibility" to use
|
||||||
|
github.com/cespare/xxhash/v2:
|
||||||
|
|
||||||
|
* 1.9.7+ for Go 1.9
|
||||||
|
* 1.10.3+ for Go 1.10
|
||||||
|
* Go 1.11 or later
|
||||||
|
|
||||||
|
I recommend using the latest release of Go.
|
||||||
|
|
||||||
|
## Benchmarks
|
||||||
|
|
||||||
|
Here are some quick benchmarks comparing the pure-Go and assembly
|
||||||
|
implementations of Sum64.
|
||||||
|
|
||||||
|
| input size | purego | asm |
|
||||||
|
| ---------- | --------- | --------- |
|
||||||
|
| 4 B | 1.3 GB/s | 1.2 GB/s |
|
||||||
|
| 16 B | 2.9 GB/s | 3.5 GB/s |
|
||||||
|
| 100 B | 6.9 GB/s | 8.1 GB/s |
|
||||||
|
| 4 KB | 11.7 GB/s | 16.7 GB/s |
|
||||||
|
| 10 MB | 12.0 GB/s | 17.3 GB/s |
|
||||||
|
|
||||||
|
These numbers were generated on Ubuntu 20.04 with an Intel Xeon Platinum 8252C
|
||||||
|
CPU using the following commands under Go 1.19.2:
|
||||||
|
|
||||||
|
```
|
||||||
|
benchstat <(go test -tags purego -benchtime 500ms -count 15 -bench 'Sum64$')
|
||||||
|
benchstat <(go test -benchtime 500ms -count 15 -bench 'Sum64$')
|
||||||
|
```
|
||||||
|
|
||||||
|
## Projects using this package
|
||||||
|
|
||||||
|
- [InfluxDB](https://github.com/influxdata/influxdb)
|
||||||
|
- [Prometheus](https://github.com/prometheus/prometheus)
|
||||||
|
- [VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics)
|
||||||
|
- [FreeCache](https://github.com/coocood/freecache)
|
||||||
|
- [FastCache](https://github.com/VictoriaMetrics/fastcache)
|
||||||
|
- [Ristretto](https://github.com/dgraph-io/ristretto)
|
||||||
|
- [Badger](https://github.com/dgraph-io/badger)
|
||||||
10
vendor/github.com/cespare/xxhash/v2/testall.sh
generated
vendored
Normal file
10
vendor/github.com/cespare/xxhash/v2/testall.sh
generated
vendored
Normal file
@ -0,0 +1,10 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
set -eu -o pipefail
|
||||||
|
|
||||||
|
# Small convenience script for running the tests with various combinations of
|
||||||
|
# arch/tags. This assumes we're running on amd64 and have qemu available.
|
||||||
|
|
||||||
|
go test ./...
|
||||||
|
go test -tags purego ./...
|
||||||
|
GOARCH=arm64 go test
|
||||||
|
GOARCH=arm64 go test -tags purego
|
||||||
243
vendor/github.com/cespare/xxhash/v2/xxhash.go
generated
vendored
Normal file
243
vendor/github.com/cespare/xxhash/v2/xxhash.go
generated
vendored
Normal file
@ -0,0 +1,243 @@
|
|||||||
|
// Package xxhash implements the 64-bit variant of xxHash (XXH64) as described
|
||||||
|
// at http://cyan4973.github.io/xxHash/.
|
||||||
|
package xxhash
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/binary"
|
||||||
|
"errors"
|
||||||
|
"math/bits"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
prime1 uint64 = 11400714785074694791
|
||||||
|
prime2 uint64 = 14029467366897019727
|
||||||
|
prime3 uint64 = 1609587929392839161
|
||||||
|
prime4 uint64 = 9650029242287828579
|
||||||
|
prime5 uint64 = 2870177450012600261
|
||||||
|
)
|
||||||
|
|
||||||
|
// Store the primes in an array as well.
|
||||||
|
//
|
||||||
|
// The consts are used when possible in Go code to avoid MOVs but we need a
|
||||||
|
// contiguous array for the assembly code.
|
||||||
|
var primes = [...]uint64{prime1, prime2, prime3, prime4, prime5}
|
||||||
|
|
||||||
|
// Digest implements hash.Hash64.
|
||||||
|
//
|
||||||
|
// Note that a zero-valued Digest is not ready to receive writes.
|
||||||
|
// Call Reset or create a Digest using New before calling other methods.
|
||||||
|
type Digest struct {
|
||||||
|
v1 uint64
|
||||||
|
v2 uint64
|
||||||
|
v3 uint64
|
||||||
|
v4 uint64
|
||||||
|
total uint64
|
||||||
|
mem [32]byte
|
||||||
|
n int // how much of mem is used
|
||||||
|
}
|
||||||
|
|
||||||
|
// New creates a new Digest with a zero seed.
|
||||||
|
func New() *Digest {
|
||||||
|
return NewWithSeed(0)
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewWithSeed creates a new Digest with the given seed.
|
||||||
|
func NewWithSeed(seed uint64) *Digest {
|
||||||
|
var d Digest
|
||||||
|
d.ResetWithSeed(seed)
|
||||||
|
return &d
|
||||||
|
}
|
||||||
|
|
||||||
|
// Reset clears the Digest's state so that it can be reused.
|
||||||
|
// It uses a seed value of zero.
|
||||||
|
func (d *Digest) Reset() {
|
||||||
|
d.ResetWithSeed(0)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ResetWithSeed clears the Digest's state so that it can be reused.
|
||||||
|
// It uses the given seed to initialize the state.
|
||||||
|
func (d *Digest) ResetWithSeed(seed uint64) {
|
||||||
|
d.v1 = seed + prime1 + prime2
|
||||||
|
d.v2 = seed + prime2
|
||||||
|
d.v3 = seed
|
||||||
|
d.v4 = seed - prime1
|
||||||
|
d.total = 0
|
||||||
|
d.n = 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// Size always returns 8 bytes.
|
||||||
|
func (d *Digest) Size() int { return 8 }
|
||||||
|
|
||||||
|
// BlockSize always returns 32 bytes.
|
||||||
|
func (d *Digest) BlockSize() int { return 32 }
|
||||||
|
|
||||||
|
// Write adds more data to d. It always returns len(b), nil.
|
||||||
|
func (d *Digest) Write(b []byte) (n int, err error) {
|
||||||
|
n = len(b)
|
||||||
|
d.total += uint64(n)
|
||||||
|
|
||||||
|
memleft := d.mem[d.n&(len(d.mem)-1):]
|
||||||
|
|
||||||
|
if d.n+n < 32 {
|
||||||
|
// This new data doesn't even fill the current block.
|
||||||
|
copy(memleft, b)
|
||||||
|
d.n += n
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if d.n > 0 {
|
||||||
|
// Finish off the partial block.
|
||||||
|
c := copy(memleft, b)
|
||||||
|
d.v1 = round(d.v1, u64(d.mem[0:8]))
|
||||||
|
d.v2 = round(d.v2, u64(d.mem[8:16]))
|
||||||
|
d.v3 = round(d.v3, u64(d.mem[16:24]))
|
||||||
|
d.v4 = round(d.v4, u64(d.mem[24:32]))
|
||||||
|
b = b[c:]
|
||||||
|
d.n = 0
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(b) >= 32 {
|
||||||
|
// One or more full blocks left.
|
||||||
|
nw := writeBlocks(d, b)
|
||||||
|
b = b[nw:]
|
||||||
|
}
|
||||||
|
|
||||||
|
// Store any remaining partial block.
|
||||||
|
copy(d.mem[:], b)
|
||||||
|
d.n = len(b)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sum appends the current hash to b and returns the resulting slice.
|
||||||
|
func (d *Digest) Sum(b []byte) []byte {
|
||||||
|
s := d.Sum64()
|
||||||
|
return append(
|
||||||
|
b,
|
||||||
|
byte(s>>56),
|
||||||
|
byte(s>>48),
|
||||||
|
byte(s>>40),
|
||||||
|
byte(s>>32),
|
||||||
|
byte(s>>24),
|
||||||
|
byte(s>>16),
|
||||||
|
byte(s>>8),
|
||||||
|
byte(s),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sum64 returns the current hash.
|
||||||
|
func (d *Digest) Sum64() uint64 {
|
||||||
|
var h uint64
|
||||||
|
|
||||||
|
if d.total >= 32 {
|
||||||
|
v1, v2, v3, v4 := d.v1, d.v2, d.v3, d.v4
|
||||||
|
h = rol1(v1) + rol7(v2) + rol12(v3) + rol18(v4)
|
||||||
|
h = mergeRound(h, v1)
|
||||||
|
h = mergeRound(h, v2)
|
||||||
|
h = mergeRound(h, v3)
|
||||||
|
h = mergeRound(h, v4)
|
||||||
|
} else {
|
||||||
|
h = d.v3 + prime5
|
||||||
|
}
|
||||||
|
|
||||||
|
h += d.total
|
||||||
|
|
||||||
|
b := d.mem[:d.n&(len(d.mem)-1)]
|
||||||
|
for ; len(b) >= 8; b = b[8:] {
|
||||||
|
k1 := round(0, u64(b[:8]))
|
||||||
|
h ^= k1
|
||||||
|
h = rol27(h)*prime1 + prime4
|
||||||
|
}
|
||||||
|
if len(b) >= 4 {
|
||||||
|
h ^= uint64(u32(b[:4])) * prime1
|
||||||
|
h = rol23(h)*prime2 + prime3
|
||||||
|
b = b[4:]
|
||||||
|
}
|
||||||
|
for ; len(b) > 0; b = b[1:] {
|
||||||
|
h ^= uint64(b[0]) * prime5
|
||||||
|
h = rol11(h) * prime1
|
||||||
|
}
|
||||||
|
|
||||||
|
h ^= h >> 33
|
||||||
|
h *= prime2
|
||||||
|
h ^= h >> 29
|
||||||
|
h *= prime3
|
||||||
|
h ^= h >> 32
|
||||||
|
|
||||||
|
return h
|
||||||
|
}
|
||||||
|
|
||||||
|
const (
|
||||||
|
magic = "xxh\x06"
|
||||||
|
marshaledSize = len(magic) + 8*5 + 32
|
||||||
|
)
|
||||||
|
|
||||||
|
// MarshalBinary implements the encoding.BinaryMarshaler interface.
|
||||||
|
func (d *Digest) MarshalBinary() ([]byte, error) {
|
||||||
|
b := make([]byte, 0, marshaledSize)
|
||||||
|
b = append(b, magic...)
|
||||||
|
b = appendUint64(b, d.v1)
|
||||||
|
b = appendUint64(b, d.v2)
|
||||||
|
b = appendUint64(b, d.v3)
|
||||||
|
b = appendUint64(b, d.v4)
|
||||||
|
b = appendUint64(b, d.total)
|
||||||
|
b = append(b, d.mem[:d.n]...)
|
||||||
|
b = b[:len(b)+len(d.mem)-d.n]
|
||||||
|
return b, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// UnmarshalBinary implements the encoding.BinaryUnmarshaler interface.
|
||||||
|
func (d *Digest) UnmarshalBinary(b []byte) error {
|
||||||
|
if len(b) < len(magic) || string(b[:len(magic)]) != magic {
|
||||||
|
return errors.New("xxhash: invalid hash state identifier")
|
||||||
|
}
|
||||||
|
if len(b) != marshaledSize {
|
||||||
|
return errors.New("xxhash: invalid hash state size")
|
||||||
|
}
|
||||||
|
b = b[len(magic):]
|
||||||
|
b, d.v1 = consumeUint64(b)
|
||||||
|
b, d.v2 = consumeUint64(b)
|
||||||
|
b, d.v3 = consumeUint64(b)
|
||||||
|
b, d.v4 = consumeUint64(b)
|
||||||
|
b, d.total = consumeUint64(b)
|
||||||
|
copy(d.mem[:], b)
|
||||||
|
d.n = int(d.total % uint64(len(d.mem)))
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func appendUint64(b []byte, x uint64) []byte {
|
||||||
|
var a [8]byte
|
||||||
|
binary.LittleEndian.PutUint64(a[:], x)
|
||||||
|
return append(b, a[:]...)
|
||||||
|
}
|
||||||
|
|
||||||
|
func consumeUint64(b []byte) ([]byte, uint64) {
|
||||||
|
x := u64(b)
|
||||||
|
return b[8:], x
|
||||||
|
}
|
||||||
|
|
||||||
|
func u64(b []byte) uint64 { return binary.LittleEndian.Uint64(b) }
|
||||||
|
func u32(b []byte) uint32 { return binary.LittleEndian.Uint32(b) }
|
||||||
|
|
||||||
|
func round(acc, input uint64) uint64 {
|
||||||
|
acc += input * prime2
|
||||||
|
acc = rol31(acc)
|
||||||
|
acc *= prime1
|
||||||
|
return acc
|
||||||
|
}
|
||||||
|
|
||||||
|
func mergeRound(acc, val uint64) uint64 {
|
||||||
|
val = round(0, val)
|
||||||
|
acc ^= val
|
||||||
|
acc = acc*prime1 + prime4
|
||||||
|
return acc
|
||||||
|
}
|
||||||
|
|
||||||
|
func rol1(x uint64) uint64 { return bits.RotateLeft64(x, 1) }
|
||||||
|
func rol7(x uint64) uint64 { return bits.RotateLeft64(x, 7) }
|
||||||
|
func rol11(x uint64) uint64 { return bits.RotateLeft64(x, 11) }
|
||||||
|
func rol12(x uint64) uint64 { return bits.RotateLeft64(x, 12) }
|
||||||
|
func rol18(x uint64) uint64 { return bits.RotateLeft64(x, 18) }
|
||||||
|
func rol23(x uint64) uint64 { return bits.RotateLeft64(x, 23) }
|
||||||
|
func rol27(x uint64) uint64 { return bits.RotateLeft64(x, 27) }
|
||||||
|
func rol31(x uint64) uint64 { return bits.RotateLeft64(x, 31) }
|
||||||
209
vendor/github.com/cespare/xxhash/v2/xxhash_amd64.s
generated
vendored
Normal file
209
vendor/github.com/cespare/xxhash/v2/xxhash_amd64.s
generated
vendored
Normal file
@ -0,0 +1,209 @@
|
|||||||
|
//go:build !appengine && gc && !purego
|
||||||
|
// +build !appengine
|
||||||
|
// +build gc
|
||||||
|
// +build !purego
|
||||||
|
|
||||||
|
#include "textflag.h"
|
||||||
|
|
||||||
|
// Registers:
|
||||||
|
#define h AX
|
||||||
|
#define d AX
|
||||||
|
#define p SI // pointer to advance through b
|
||||||
|
#define n DX
|
||||||
|
#define end BX // loop end
|
||||||
|
#define v1 R8
|
||||||
|
#define v2 R9
|
||||||
|
#define v3 R10
|
||||||
|
#define v4 R11
|
||||||
|
#define x R12
|
||||||
|
#define prime1 R13
|
||||||
|
#define prime2 R14
|
||||||
|
#define prime4 DI
|
||||||
|
|
||||||
|
#define round(acc, x) \
|
||||||
|
IMULQ prime2, x \
|
||||||
|
ADDQ x, acc \
|
||||||
|
ROLQ $31, acc \
|
||||||
|
IMULQ prime1, acc
|
||||||
|
|
||||||
|
// round0 performs the operation x = round(0, x).
|
||||||
|
#define round0(x) \
|
||||||
|
IMULQ prime2, x \
|
||||||
|
ROLQ $31, x \
|
||||||
|
IMULQ prime1, x
|
||||||
|
|
||||||
|
// mergeRound applies a merge round on the two registers acc and x.
|
||||||
|
// It assumes that prime1, prime2, and prime4 have been loaded.
|
||||||
|
#define mergeRound(acc, x) \
|
||||||
|
round0(x) \
|
||||||
|
XORQ x, acc \
|
||||||
|
IMULQ prime1, acc \
|
||||||
|
ADDQ prime4, acc
|
||||||
|
|
||||||
|
// blockLoop processes as many 32-byte blocks as possible,
|
||||||
|
// updating v1, v2, v3, and v4. It assumes that there is at least one block
|
||||||
|
// to process.
|
||||||
|
#define blockLoop() \
|
||||||
|
loop: \
|
||||||
|
MOVQ +0(p), x \
|
||||||
|
round(v1, x) \
|
||||||
|
MOVQ +8(p), x \
|
||||||
|
round(v2, x) \
|
||||||
|
MOVQ +16(p), x \
|
||||||
|
round(v3, x) \
|
||||||
|
MOVQ +24(p), x \
|
||||||
|
round(v4, x) \
|
||||||
|
ADDQ $32, p \
|
||||||
|
CMPQ p, end \
|
||||||
|
JLE loop
|
||||||
|
|
||||||
|
// func Sum64(b []byte) uint64
|
||||||
|
TEXT ·Sum64(SB), NOSPLIT|NOFRAME, $0-32
|
||||||
|
// Load fixed primes.
|
||||||
|
MOVQ ·primes+0(SB), prime1
|
||||||
|
MOVQ ·primes+8(SB), prime2
|
||||||
|
MOVQ ·primes+24(SB), prime4
|
||||||
|
|
||||||
|
// Load slice.
|
||||||
|
MOVQ b_base+0(FP), p
|
||||||
|
MOVQ b_len+8(FP), n
|
||||||
|
LEAQ (p)(n*1), end
|
||||||
|
|
||||||
|
// The first loop limit will be len(b)-32.
|
||||||
|
SUBQ $32, end
|
||||||
|
|
||||||
|
// Check whether we have at least one block.
|
||||||
|
CMPQ n, $32
|
||||||
|
JLT noBlocks
|
||||||
|
|
||||||
|
// Set up initial state (v1, v2, v3, v4).
|
||||||
|
MOVQ prime1, v1
|
||||||
|
ADDQ prime2, v1
|
||||||
|
MOVQ prime2, v2
|
||||||
|
XORQ v3, v3
|
||||||
|
XORQ v4, v4
|
||||||
|
SUBQ prime1, v4
|
||||||
|
|
||||||
|
blockLoop()
|
||||||
|
|
||||||
|
MOVQ v1, h
|
||||||
|
ROLQ $1, h
|
||||||
|
MOVQ v2, x
|
||||||
|
ROLQ $7, x
|
||||||
|
ADDQ x, h
|
||||||
|
MOVQ v3, x
|
||||||
|
ROLQ $12, x
|
||||||
|
ADDQ x, h
|
||||||
|
MOVQ v4, x
|
||||||
|
ROLQ $18, x
|
||||||
|
ADDQ x, h
|
||||||
|
|
||||||
|
mergeRound(h, v1)
|
||||||
|
mergeRound(h, v2)
|
||||||
|
mergeRound(h, v3)
|
||||||
|
mergeRound(h, v4)
|
||||||
|
|
||||||
|
JMP afterBlocks
|
||||||
|
|
||||||
|
noBlocks:
|
||||||
|
MOVQ ·primes+32(SB), h
|
||||||
|
|
||||||
|
afterBlocks:
|
||||||
|
ADDQ n, h
|
||||||
|
|
||||||
|
ADDQ $24, end
|
||||||
|
CMPQ p, end
|
||||||
|
JG try4
|
||||||
|
|
||||||
|
loop8:
|
||||||
|
MOVQ (p), x
|
||||||
|
ADDQ $8, p
|
||||||
|
round0(x)
|
||||||
|
XORQ x, h
|
||||||
|
ROLQ $27, h
|
||||||
|
IMULQ prime1, h
|
||||||
|
ADDQ prime4, h
|
||||||
|
|
||||||
|
CMPQ p, end
|
||||||
|
JLE loop8
|
||||||
|
|
||||||
|
try4:
|
||||||
|
ADDQ $4, end
|
||||||
|
CMPQ p, end
|
||||||
|
JG try1
|
||||||
|
|
||||||
|
MOVL (p), x
|
||||||
|
ADDQ $4, p
|
||||||
|
IMULQ prime1, x
|
||||||
|
XORQ x, h
|
||||||
|
|
||||||
|
ROLQ $23, h
|
||||||
|
IMULQ prime2, h
|
||||||
|
ADDQ ·primes+16(SB), h
|
||||||
|
|
||||||
|
try1:
|
||||||
|
ADDQ $4, end
|
||||||
|
CMPQ p, end
|
||||||
|
JGE finalize
|
||||||
|
|
||||||
|
loop1:
|
||||||
|
MOVBQZX (p), x
|
||||||
|
ADDQ $1, p
|
||||||
|
IMULQ ·primes+32(SB), x
|
||||||
|
XORQ x, h
|
||||||
|
ROLQ $11, h
|
||||||
|
IMULQ prime1, h
|
||||||
|
|
||||||
|
CMPQ p, end
|
||||||
|
JL loop1
|
||||||
|
|
||||||
|
finalize:
|
||||||
|
MOVQ h, x
|
||||||
|
SHRQ $33, x
|
||||||
|
XORQ x, h
|
||||||
|
IMULQ prime2, h
|
||||||
|
MOVQ h, x
|
||||||
|
SHRQ $29, x
|
||||||
|
XORQ x, h
|
||||||
|
IMULQ ·primes+16(SB), h
|
||||||
|
MOVQ h, x
|
||||||
|
SHRQ $32, x
|
||||||
|
XORQ x, h
|
||||||
|
|
||||||
|
MOVQ h, ret+24(FP)
|
||||||
|
RET
|
||||||
|
|
||||||
|
// func writeBlocks(d *Digest, b []byte) int
|
||||||
|
TEXT ·writeBlocks(SB), NOSPLIT|NOFRAME, $0-40
|
||||||
|
// Load fixed primes needed for round.
|
||||||
|
MOVQ ·primes+0(SB), prime1
|
||||||
|
MOVQ ·primes+8(SB), prime2
|
||||||
|
|
||||||
|
// Load slice.
|
||||||
|
MOVQ b_base+8(FP), p
|
||||||
|
MOVQ b_len+16(FP), n
|
||||||
|
LEAQ (p)(n*1), end
|
||||||
|
SUBQ $32, end
|
||||||
|
|
||||||
|
// Load vN from d.
|
||||||
|
MOVQ s+0(FP), d
|
||||||
|
MOVQ 0(d), v1
|
||||||
|
MOVQ 8(d), v2
|
||||||
|
MOVQ 16(d), v3
|
||||||
|
MOVQ 24(d), v4
|
||||||
|
|
||||||
|
// We don't need to check the loop condition here; this function is
|
||||||
|
// always called with at least one block of data to process.
|
||||||
|
blockLoop()
|
||||||
|
|
||||||
|
// Copy vN back to d.
|
||||||
|
MOVQ v1, 0(d)
|
||||||
|
MOVQ v2, 8(d)
|
||||||
|
MOVQ v3, 16(d)
|
||||||
|
MOVQ v4, 24(d)
|
||||||
|
|
||||||
|
// The number of bytes written is p minus the old base pointer.
|
||||||
|
SUBQ b_base+8(FP), p
|
||||||
|
MOVQ p, ret+32(FP)
|
||||||
|
|
||||||
|
RET
|
||||||
183
vendor/github.com/cespare/xxhash/v2/xxhash_arm64.s
generated
vendored
Normal file
183
vendor/github.com/cespare/xxhash/v2/xxhash_arm64.s
generated
vendored
Normal file
@ -0,0 +1,183 @@
|
|||||||
|
//go:build !appengine && gc && !purego
|
||||||
|
// +build !appengine
|
||||||
|
// +build gc
|
||||||
|
// +build !purego
|
||||||
|
|
||||||
|
#include "textflag.h"
|
||||||
|
|
||||||
|
// Registers:
|
||||||
|
#define digest R1
|
||||||
|
#define h R2 // return value
|
||||||
|
#define p R3 // input pointer
|
||||||
|
#define n R4 // input length
|
||||||
|
#define nblocks R5 // n / 32
|
||||||
|
#define prime1 R7
|
||||||
|
#define prime2 R8
|
||||||
|
#define prime3 R9
|
||||||
|
#define prime4 R10
|
||||||
|
#define prime5 R11
|
||||||
|
#define v1 R12
|
||||||
|
#define v2 R13
|
||||||
|
#define v3 R14
|
||||||
|
#define v4 R15
|
||||||
|
#define x1 R20
|
||||||
|
#define x2 R21
|
||||||
|
#define x3 R22
|
||||||
|
#define x4 R23
|
||||||
|
|
||||||
|
#define round(acc, x) \
|
||||||
|
MADD prime2, acc, x, acc \
|
||||||
|
ROR $64-31, acc \
|
||||||
|
MUL prime1, acc
|
||||||
|
|
||||||
|
// round0 performs the operation x = round(0, x).
|
||||||
|
#define round0(x) \
|
||||||
|
MUL prime2, x \
|
||||||
|
ROR $64-31, x \
|
||||||
|
MUL prime1, x
|
||||||
|
|
||||||
|
#define mergeRound(acc, x) \
|
||||||
|
round0(x) \
|
||||||
|
EOR x, acc \
|
||||||
|
MADD acc, prime4, prime1, acc
|
||||||
|
|
||||||
|
// blockLoop processes as many 32-byte blocks as possible,
|
||||||
|
// updating v1, v2, v3, and v4. It assumes that n >= 32.
|
||||||
|
#define blockLoop() \
|
||||||
|
LSR $5, n, nblocks \
|
||||||
|
PCALIGN $16 \
|
||||||
|
loop: \
|
||||||
|
LDP.P 16(p), (x1, x2) \
|
||||||
|
LDP.P 16(p), (x3, x4) \
|
||||||
|
round(v1, x1) \
|
||||||
|
round(v2, x2) \
|
||||||
|
round(v3, x3) \
|
||||||
|
round(v4, x4) \
|
||||||
|
SUB $1, nblocks \
|
||||||
|
CBNZ nblocks, loop
|
||||||
|
|
||||||
|
// func Sum64(b []byte) uint64
|
||||||
|
TEXT ·Sum64(SB), NOSPLIT|NOFRAME, $0-32
|
||||||
|
LDP b_base+0(FP), (p, n)
|
||||||
|
|
||||||
|
LDP ·primes+0(SB), (prime1, prime2)
|
||||||
|
LDP ·primes+16(SB), (prime3, prime4)
|
||||||
|
MOVD ·primes+32(SB), prime5
|
||||||
|
|
||||||
|
CMP $32, n
|
||||||
|
CSEL LT, prime5, ZR, h // if n < 32 { h = prime5 } else { h = 0 }
|
||||||
|
BLT afterLoop
|
||||||
|
|
||||||
|
ADD prime1, prime2, v1
|
||||||
|
MOVD prime2, v2
|
||||||
|
MOVD $0, v3
|
||||||
|
NEG prime1, v4
|
||||||
|
|
||||||
|
blockLoop()
|
||||||
|
|
||||||
|
ROR $64-1, v1, x1
|
||||||
|
ROR $64-7, v2, x2
|
||||||
|
ADD x1, x2
|
||||||
|
ROR $64-12, v3, x3
|
||||||
|
ROR $64-18, v4, x4
|
||||||
|
ADD x3, x4
|
||||||
|
ADD x2, x4, h
|
||||||
|
|
||||||
|
mergeRound(h, v1)
|
||||||
|
mergeRound(h, v2)
|
||||||
|
mergeRound(h, v3)
|
||||||
|
mergeRound(h, v4)
|
||||||
|
|
||||||
|
afterLoop:
|
||||||
|
ADD n, h
|
||||||
|
|
||||||
|
TBZ $4, n, try8
|
||||||
|
LDP.P 16(p), (x1, x2)
|
||||||
|
|
||||||
|
round0(x1)
|
||||||
|
|
||||||
|
// NOTE: here and below, sequencing the EOR after the ROR (using a
|
||||||
|
// rotated register) is worth a small but measurable speedup for small
|
||||||
|
// inputs.
|
||||||
|
ROR $64-27, h
|
||||||
|
EOR x1 @> 64-27, h, h
|
||||||
|
MADD h, prime4, prime1, h
|
||||||
|
|
||||||
|
round0(x2)
|
||||||
|
ROR $64-27, h
|
||||||
|
EOR x2 @> 64-27, h, h
|
||||||
|
MADD h, prime4, prime1, h
|
||||||
|
|
||||||
|
try8:
|
||||||
|
TBZ $3, n, try4
|
||||||
|
MOVD.P 8(p), x1
|
||||||
|
|
||||||
|
round0(x1)
|
||||||
|
ROR $64-27, h
|
||||||
|
EOR x1 @> 64-27, h, h
|
||||||
|
MADD h, prime4, prime1, h
|
||||||
|
|
||||||
|
try4:
|
||||||
|
TBZ $2, n, try2
|
||||||
|
MOVWU.P 4(p), x2
|
||||||
|
|
||||||
|
MUL prime1, x2
|
||||||
|
ROR $64-23, h
|
||||||
|
EOR x2 @> 64-23, h, h
|
||||||
|
MADD h, prime3, prime2, h
|
||||||
|
|
||||||
|
try2:
|
||||||
|
TBZ $1, n, try1
|
||||||
|
MOVHU.P 2(p), x3
|
||||||
|
AND $255, x3, x1
|
||||||
|
LSR $8, x3, x2
|
||||||
|
|
||||||
|
MUL prime5, x1
|
||||||
|
ROR $64-11, h
|
||||||
|
EOR x1 @> 64-11, h, h
|
||||||
|
MUL prime1, h
|
||||||
|
|
||||||
|
MUL prime5, x2
|
||||||
|
ROR $64-11, h
|
||||||
|
EOR x2 @> 64-11, h, h
|
||||||
|
MUL prime1, h
|
||||||
|
|
||||||
|
try1:
|
||||||
|
TBZ $0, n, finalize
|
||||||
|
MOVBU (p), x4
|
||||||
|
|
||||||
|
MUL prime5, x4
|
||||||
|
ROR $64-11, h
|
||||||
|
EOR x4 @> 64-11, h, h
|
||||||
|
MUL prime1, h
|
||||||
|
|
||||||
|
finalize:
|
||||||
|
EOR h >> 33, h
|
||||||
|
MUL prime2, h
|
||||||
|
EOR h >> 29, h
|
||||||
|
MUL prime3, h
|
||||||
|
EOR h >> 32, h
|
||||||
|
|
||||||
|
MOVD h, ret+24(FP)
|
||||||
|
RET
|
||||||
|
|
||||||
|
// func writeBlocks(d *Digest, b []byte) int
|
||||||
|
TEXT ·writeBlocks(SB), NOSPLIT|NOFRAME, $0-40
|
||||||
|
LDP ·primes+0(SB), (prime1, prime2)
|
||||||
|
|
||||||
|
// Load state. Assume v[1-4] are stored contiguously.
|
||||||
|
MOVD d+0(FP), digest
|
||||||
|
LDP 0(digest), (v1, v2)
|
||||||
|
LDP 16(digest), (v3, v4)
|
||||||
|
|
||||||
|
LDP b_base+8(FP), (p, n)
|
||||||
|
|
||||||
|
blockLoop()
|
||||||
|
|
||||||
|
// Store updated state.
|
||||||
|
STP (v1, v2), 0(digest)
|
||||||
|
STP (v3, v4), 16(digest)
|
||||||
|
|
||||||
|
BIC $31, n
|
||||||
|
MOVD n, ret+32(FP)
|
||||||
|
RET
|
||||||
15
vendor/github.com/cespare/xxhash/v2/xxhash_asm.go
generated
vendored
Normal file
15
vendor/github.com/cespare/xxhash/v2/xxhash_asm.go
generated
vendored
Normal file
@ -0,0 +1,15 @@
|
|||||||
|
//go:build (amd64 || arm64) && !appengine && gc && !purego
|
||||||
|
// +build amd64 arm64
|
||||||
|
// +build !appengine
|
||||||
|
// +build gc
|
||||||
|
// +build !purego
|
||||||
|
|
||||||
|
package xxhash
|
||||||
|
|
||||||
|
// Sum64 computes the 64-bit xxHash digest of b with a zero seed.
|
||||||
|
//
|
||||||
|
//go:noescape
|
||||||
|
func Sum64(b []byte) uint64
|
||||||
|
|
||||||
|
//go:noescape
|
||||||
|
func writeBlocks(d *Digest, b []byte) int
|
||||||
76
vendor/github.com/cespare/xxhash/v2/xxhash_other.go
generated
vendored
Normal file
76
vendor/github.com/cespare/xxhash/v2/xxhash_other.go
generated
vendored
Normal file
@ -0,0 +1,76 @@
|
|||||||
|
//go:build (!amd64 && !arm64) || appengine || !gc || purego
|
||||||
|
// +build !amd64,!arm64 appengine !gc purego
|
||||||
|
|
||||||
|
package xxhash
|
||||||
|
|
||||||
|
// Sum64 computes the 64-bit xxHash digest of b with a zero seed.
|
||||||
|
func Sum64(b []byte) uint64 {
|
||||||
|
// A simpler version would be
|
||||||
|
// d := New()
|
||||||
|
// d.Write(b)
|
||||||
|
// return d.Sum64()
|
||||||
|
// but this is faster, particularly for small inputs.
|
||||||
|
|
||||||
|
n := len(b)
|
||||||
|
var h uint64
|
||||||
|
|
||||||
|
if n >= 32 {
|
||||||
|
v1 := primes[0] + prime2
|
||||||
|
v2 := prime2
|
||||||
|
v3 := uint64(0)
|
||||||
|
v4 := -primes[0]
|
||||||
|
for len(b) >= 32 {
|
||||||
|
v1 = round(v1, u64(b[0:8:len(b)]))
|
||||||
|
v2 = round(v2, u64(b[8:16:len(b)]))
|
||||||
|
v3 = round(v3, u64(b[16:24:len(b)]))
|
||||||
|
v4 = round(v4, u64(b[24:32:len(b)]))
|
||||||
|
b = b[32:len(b):len(b)]
|
||||||
|
}
|
||||||
|
h = rol1(v1) + rol7(v2) + rol12(v3) + rol18(v4)
|
||||||
|
h = mergeRound(h, v1)
|
||||||
|
h = mergeRound(h, v2)
|
||||||
|
h = mergeRound(h, v3)
|
||||||
|
h = mergeRound(h, v4)
|
||||||
|
} else {
|
||||||
|
h = prime5
|
||||||
|
}
|
||||||
|
|
||||||
|
h += uint64(n)
|
||||||
|
|
||||||
|
for ; len(b) >= 8; b = b[8:] {
|
||||||
|
k1 := round(0, u64(b[:8]))
|
||||||
|
h ^= k1
|
||||||
|
h = rol27(h)*prime1 + prime4
|
||||||
|
}
|
||||||
|
if len(b) >= 4 {
|
||||||
|
h ^= uint64(u32(b[:4])) * prime1
|
||||||
|
h = rol23(h)*prime2 + prime3
|
||||||
|
b = b[4:]
|
||||||
|
}
|
||||||
|
for ; len(b) > 0; b = b[1:] {
|
||||||
|
h ^= uint64(b[0]) * prime5
|
||||||
|
h = rol11(h) * prime1
|
||||||
|
}
|
||||||
|
|
||||||
|
h ^= h >> 33
|
||||||
|
h *= prime2
|
||||||
|
h ^= h >> 29
|
||||||
|
h *= prime3
|
||||||
|
h ^= h >> 32
|
||||||
|
|
||||||
|
return h
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeBlocks(d *Digest, b []byte) int {
|
||||||
|
v1, v2, v3, v4 := d.v1, d.v2, d.v3, d.v4
|
||||||
|
n := len(b)
|
||||||
|
for len(b) >= 32 {
|
||||||
|
v1 = round(v1, u64(b[0:8:len(b)]))
|
||||||
|
v2 = round(v2, u64(b[8:16:len(b)]))
|
||||||
|
v3 = round(v3, u64(b[16:24:len(b)]))
|
||||||
|
v4 = round(v4, u64(b[24:32:len(b)]))
|
||||||
|
b = b[32:len(b):len(b)]
|
||||||
|
}
|
||||||
|
d.v1, d.v2, d.v3, d.v4 = v1, v2, v3, v4
|
||||||
|
return n - len(b)
|
||||||
|
}
|
||||||
16
vendor/github.com/cespare/xxhash/v2/xxhash_safe.go
generated
vendored
Normal file
16
vendor/github.com/cespare/xxhash/v2/xxhash_safe.go
generated
vendored
Normal file
@ -0,0 +1,16 @@
|
|||||||
|
//go:build appengine
|
||||||
|
// +build appengine
|
||||||
|
|
||||||
|
// This file contains the safe implementations of otherwise unsafe-using code.
|
||||||
|
|
||||||
|
package xxhash
|
||||||
|
|
||||||
|
// Sum64String computes the 64-bit xxHash digest of s with a zero seed.
|
||||||
|
func Sum64String(s string) uint64 {
|
||||||
|
return Sum64([]byte(s))
|
||||||
|
}
|
||||||
|
|
||||||
|
// WriteString adds more data to d. It always returns len(s), nil.
|
||||||
|
func (d *Digest) WriteString(s string) (n int, err error) {
|
||||||
|
return d.Write([]byte(s))
|
||||||
|
}
|
||||||
58
vendor/github.com/cespare/xxhash/v2/xxhash_unsafe.go
generated
vendored
Normal file
58
vendor/github.com/cespare/xxhash/v2/xxhash_unsafe.go
generated
vendored
Normal file
@ -0,0 +1,58 @@
|
|||||||
|
//go:build !appengine
|
||||||
|
// +build !appengine
|
||||||
|
|
||||||
|
// This file encapsulates usage of unsafe.
|
||||||
|
// xxhash_safe.go contains the safe implementations.
|
||||||
|
|
||||||
|
package xxhash
|
||||||
|
|
||||||
|
import (
|
||||||
|
"unsafe"
|
||||||
|
)
|
||||||
|
|
||||||
|
// In the future it's possible that compiler optimizations will make these
|
||||||
|
// XxxString functions unnecessary by realizing that calls such as
|
||||||
|
// Sum64([]byte(s)) don't need to copy s. See https://go.dev/issue/2205.
|
||||||
|
// If that happens, even if we keep these functions they can be replaced with
|
||||||
|
// the trivial safe code.
|
||||||
|
|
||||||
|
// NOTE: The usual way of doing an unsafe string-to-[]byte conversion is:
|
||||||
|
//
|
||||||
|
// var b []byte
|
||||||
|
// bh := (*reflect.SliceHeader)(unsafe.Pointer(&b))
|
||||||
|
// bh.Data = (*reflect.StringHeader)(unsafe.Pointer(&s)).Data
|
||||||
|
// bh.Len = len(s)
|
||||||
|
// bh.Cap = len(s)
|
||||||
|
//
|
||||||
|
// Unfortunately, as of Go 1.15.3 the inliner's cost model assigns a high enough
|
||||||
|
// weight to this sequence of expressions that any function that uses it will
|
||||||
|
// not be inlined. Instead, the functions below use a different unsafe
|
||||||
|
// conversion designed to minimize the inliner weight and allow both to be
|
||||||
|
// inlined. There is also a test (TestInlining) which verifies that these are
|
||||||
|
// inlined.
|
||||||
|
//
|
||||||
|
// See https://github.com/golang/go/issues/42739 for discussion.
|
||||||
|
|
||||||
|
// Sum64String computes the 64-bit xxHash digest of s with a zero seed.
|
||||||
|
// It may be faster than Sum64([]byte(s)) by avoiding a copy.
|
||||||
|
func Sum64String(s string) uint64 {
|
||||||
|
b := *(*[]byte)(unsafe.Pointer(&sliceHeader{s, len(s)}))
|
||||||
|
return Sum64(b)
|
||||||
|
}
|
||||||
|
|
||||||
|
// WriteString adds more data to d. It always returns len(s), nil.
|
||||||
|
// It may be faster than Write([]byte(s)) by avoiding a copy.
|
||||||
|
func (d *Digest) WriteString(s string) (n int, err error) {
|
||||||
|
d.Write(*(*[]byte)(unsafe.Pointer(&sliceHeader{s, len(s)})))
|
||||||
|
// d.Write always returns len(s), nil.
|
||||||
|
// Ignoring the return output and returning these fixed values buys a
|
||||||
|
// savings of 6 in the inliner's cost model.
|
||||||
|
return len(s), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// sliceHeader is similar to reflect.SliceHeader, but it assumes that the layout
|
||||||
|
// of the first two words is the same as the layout of a string.
|
||||||
|
type sliceHeader struct {
|
||||||
|
s string
|
||||||
|
cap int
|
||||||
|
}
|
||||||
31
vendor/github.com/munnerz/goautoneg/LICENSE
generated
vendored
Normal file
31
vendor/github.com/munnerz/goautoneg/LICENSE
generated
vendored
Normal file
@ -0,0 +1,31 @@
|
|||||||
|
Copyright (c) 2011, Open Knowledge Foundation Ltd.
|
||||||
|
All rights reserved.
|
||||||
|
|
||||||
|
Redistribution and use in source and binary forms, with or without
|
||||||
|
modification, are permitted provided that the following conditions are
|
||||||
|
met:
|
||||||
|
|
||||||
|
Redistributions of source code must retain the above copyright
|
||||||
|
notice, this list of conditions and the following disclaimer.
|
||||||
|
|
||||||
|
Redistributions in binary form must reproduce the above copyright
|
||||||
|
notice, this list of conditions and the following disclaimer in
|
||||||
|
the documentation and/or other materials provided with the
|
||||||
|
distribution.
|
||||||
|
|
||||||
|
Neither the name of the Open Knowledge Foundation Ltd. nor the
|
||||||
|
names of its contributors may be used to endorse or promote
|
||||||
|
products derived from this software without specific prior written
|
||||||
|
permission.
|
||||||
|
|
||||||
|
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
13
vendor/github.com/munnerz/goautoneg/Makefile
generated
vendored
Normal file
13
vendor/github.com/munnerz/goautoneg/Makefile
generated
vendored
Normal file
@ -0,0 +1,13 @@
|
|||||||
|
include $(GOROOT)/src/Make.inc
|
||||||
|
|
||||||
|
TARG=bitbucket.org/ww/goautoneg
|
||||||
|
GOFILES=autoneg.go
|
||||||
|
|
||||||
|
include $(GOROOT)/src/Make.pkg
|
||||||
|
|
||||||
|
format:
|
||||||
|
gofmt -w *.go
|
||||||
|
|
||||||
|
docs:
|
||||||
|
gomake clean
|
||||||
|
godoc ${TARG} > README.txt
|
||||||
67
vendor/github.com/munnerz/goautoneg/README.txt
generated
vendored
Normal file
67
vendor/github.com/munnerz/goautoneg/README.txt
generated
vendored
Normal file
@ -0,0 +1,67 @@
|
|||||||
|
PACKAGE
|
||||||
|
|
||||||
|
package goautoneg
|
||||||
|
import "bitbucket.org/ww/goautoneg"
|
||||||
|
|
||||||
|
HTTP Content-Type Autonegotiation.
|
||||||
|
|
||||||
|
The functions in this package implement the behaviour specified in
|
||||||
|
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
|
||||||
|
|
||||||
|
Copyright (c) 2011, Open Knowledge Foundation Ltd.
|
||||||
|
All rights reserved.
|
||||||
|
|
||||||
|
Redistribution and use in source and binary forms, with or without
|
||||||
|
modification, are permitted provided that the following conditions are
|
||||||
|
met:
|
||||||
|
|
||||||
|
Redistributions of source code must retain the above copyright
|
||||||
|
notice, this list of conditions and the following disclaimer.
|
||||||
|
|
||||||
|
Redistributions in binary form must reproduce the above copyright
|
||||||
|
notice, this list of conditions and the following disclaimer in
|
||||||
|
the documentation and/or other materials provided with the
|
||||||
|
distribution.
|
||||||
|
|
||||||
|
Neither the name of the Open Knowledge Foundation Ltd. nor the
|
||||||
|
names of its contributors may be used to endorse or promote
|
||||||
|
products derived from this software without specific prior written
|
||||||
|
permission.
|
||||||
|
|
||||||
|
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
|
||||||
|
|
||||||
|
FUNCTIONS
|
||||||
|
|
||||||
|
func Negotiate(header string, alternatives []string) (content_type string)
|
||||||
|
Negotiate the most appropriate content_type given the accept header
|
||||||
|
and a list of alternatives.
|
||||||
|
|
||||||
|
func ParseAccept(header string) (accept []Accept)
|
||||||
|
Parse an Accept Header string returning a sorted list
|
||||||
|
of clauses
|
||||||
|
|
||||||
|
|
||||||
|
TYPES
|
||||||
|
|
||||||
|
type Accept struct {
|
||||||
|
Type, SubType string
|
||||||
|
Q float32
|
||||||
|
Params map[string]string
|
||||||
|
}
|
||||||
|
Structure to represent a clause in an HTTP Accept Header
|
||||||
|
|
||||||
|
|
||||||
|
SUBDIRECTORIES
|
||||||
|
|
||||||
|
.hg
|
||||||
189
vendor/github.com/munnerz/goautoneg/autoneg.go
generated
vendored
Normal file
189
vendor/github.com/munnerz/goautoneg/autoneg.go
generated
vendored
Normal file
@ -0,0 +1,189 @@
|
|||||||
|
/*
|
||||||
|
HTTP Content-Type Autonegotiation.
|
||||||
|
|
||||||
|
The functions in this package implement the behaviour specified in
|
||||||
|
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
|
||||||
|
|
||||||
|
Copyright (c) 2011, Open Knowledge Foundation Ltd.
|
||||||
|
All rights reserved.
|
||||||
|
|
||||||
|
Redistribution and use in source and binary forms, with or without
|
||||||
|
modification, are permitted provided that the following conditions are
|
||||||
|
met:
|
||||||
|
|
||||||
|
Redistributions of source code must retain the above copyright
|
||||||
|
notice, this list of conditions and the following disclaimer.
|
||||||
|
|
||||||
|
Redistributions in binary form must reproduce the above copyright
|
||||||
|
notice, this list of conditions and the following disclaimer in
|
||||||
|
the documentation and/or other materials provided with the
|
||||||
|
distribution.
|
||||||
|
|
||||||
|
Neither the name of the Open Knowledge Foundation Ltd. nor the
|
||||||
|
names of its contributors may be used to endorse or promote
|
||||||
|
products derived from this software without specific prior written
|
||||||
|
permission.
|
||||||
|
|
||||||
|
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
*/
|
||||||
|
|
||||||
|
package goautoneg
|
||||||
|
|
||||||
|
import (
|
||||||
|
"sort"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Structure to represent a clause in an HTTP Accept Header
|
||||||
|
type Accept struct {
|
||||||
|
Type, SubType string
|
||||||
|
Q float64
|
||||||
|
Params map[string]string
|
||||||
|
}
|
||||||
|
|
||||||
|
// acceptSlice is defined to implement sort interface.
|
||||||
|
type acceptSlice []Accept
|
||||||
|
|
||||||
|
func (slice acceptSlice) Len() int {
|
||||||
|
return len(slice)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (slice acceptSlice) Less(i, j int) bool {
|
||||||
|
ai, aj := slice[i], slice[j]
|
||||||
|
if ai.Q > aj.Q {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if ai.Type != "*" && aj.Type == "*" {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if ai.SubType != "*" && aj.SubType == "*" {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
func (slice acceptSlice) Swap(i, j int) {
|
||||||
|
slice[i], slice[j] = slice[j], slice[i]
|
||||||
|
}
|
||||||
|
|
||||||
|
func stringTrimSpaceCutset(r rune) bool {
|
||||||
|
return r == ' '
|
||||||
|
}
|
||||||
|
|
||||||
|
func nextSplitElement(s, sep string) (item string, remaining string) {
|
||||||
|
if index := strings.Index(s, sep); index != -1 {
|
||||||
|
return s[:index], s[index+1:]
|
||||||
|
}
|
||||||
|
return s, ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse an Accept Header string returning a sorted list
|
||||||
|
// of clauses
|
||||||
|
func ParseAccept(header string) acceptSlice {
|
||||||
|
partsCount := 0
|
||||||
|
remaining := header
|
||||||
|
for len(remaining) > 0 {
|
||||||
|
partsCount++
|
||||||
|
_, remaining = nextSplitElement(remaining, ",")
|
||||||
|
}
|
||||||
|
accept := make(acceptSlice, 0, partsCount)
|
||||||
|
|
||||||
|
remaining = header
|
||||||
|
var part string
|
||||||
|
for len(remaining) > 0 {
|
||||||
|
part, remaining = nextSplitElement(remaining, ",")
|
||||||
|
part = strings.TrimFunc(part, stringTrimSpaceCutset)
|
||||||
|
|
||||||
|
a := Accept{
|
||||||
|
Q: 1.0,
|
||||||
|
}
|
||||||
|
|
||||||
|
sp, remainingPart := nextSplitElement(part, ";")
|
||||||
|
|
||||||
|
sp0, spRemaining := nextSplitElement(sp, "/")
|
||||||
|
a.Type = strings.TrimFunc(sp0, stringTrimSpaceCutset)
|
||||||
|
|
||||||
|
switch {
|
||||||
|
case len(spRemaining) == 0:
|
||||||
|
if a.Type == "*" {
|
||||||
|
a.SubType = "*"
|
||||||
|
} else {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
var sp1 string
|
||||||
|
sp1, spRemaining = nextSplitElement(spRemaining, "/")
|
||||||
|
if len(spRemaining) > 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
a.SubType = strings.TrimFunc(sp1, stringTrimSpaceCutset)
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(remainingPart) == 0 {
|
||||||
|
accept = append(accept, a)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
a.Params = make(map[string]string)
|
||||||
|
for len(remainingPart) > 0 {
|
||||||
|
sp, remainingPart = nextSplitElement(remainingPart, ";")
|
||||||
|
sp0, spRemaining = nextSplitElement(sp, "=")
|
||||||
|
if len(spRemaining) == 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
var sp1 string
|
||||||
|
sp1, spRemaining = nextSplitElement(spRemaining, "=")
|
||||||
|
if len(spRemaining) != 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
token := strings.TrimFunc(sp0, stringTrimSpaceCutset)
|
||||||
|
if token == "q" {
|
||||||
|
a.Q, _ = strconv.ParseFloat(sp1, 32)
|
||||||
|
} else {
|
||||||
|
a.Params[token] = strings.TrimFunc(sp1, stringTrimSpaceCutset)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
accept = append(accept, a)
|
||||||
|
}
|
||||||
|
|
||||||
|
sort.Sort(accept)
|
||||||
|
return accept
|
||||||
|
}
|
||||||
|
|
||||||
|
// Negotiate the most appropriate content_type given the accept header
|
||||||
|
// and a list of alternatives.
|
||||||
|
func Negotiate(header string, alternatives []string) (content_type string) {
|
||||||
|
asp := make([][]string, 0, len(alternatives))
|
||||||
|
for _, ctype := range alternatives {
|
||||||
|
asp = append(asp, strings.SplitN(ctype, "/", 2))
|
||||||
|
}
|
||||||
|
for _, clause := range ParseAccept(header) {
|
||||||
|
for i, ctsp := range asp {
|
||||||
|
if clause.Type == ctsp[0] && clause.SubType == ctsp[1] {
|
||||||
|
content_type = alternatives[i]
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if clause.Type == ctsp[0] && clause.SubType == "*" {
|
||||||
|
content_type = alternatives[i]
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if clause.Type == "*" && clause.SubType == "*" {
|
||||||
|
content_type = alternatives[i]
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
393
vendor/github.com/orchard9/go-chassis/chassis/app.go
generated
vendored
Normal file
393
vendor/github.com/orchard9/go-chassis/chassis/app.go
generated
vendored
Normal file
@ -0,0 +1,393 @@
|
|||||||
|
// Package chassis is the shared HTTP framework every service surface is built
|
||||||
|
// on (the API service, any second service, the worker's health endpoint). It
|
||||||
|
// owns the edge concerns so handlers stay thin: routing (stdlib ServeMux), the
|
||||||
|
// request Context + JSON envelope, the error model, the edge middleware chain
|
||||||
|
// (recover, request-id, structured logging, RED metrics, security headers,
|
||||||
|
// CORS), pluggable health probes, /metrics, an auth seam, and two-phase
|
||||||
|
// graceful shutdown. See patterns/go-chassis.md.
|
||||||
|
package chassis
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"log/slog"
|
||||||
|
"net/http"
|
||||||
|
"runtime/debug"
|
||||||
|
"sync"
|
||||||
|
"sync/atomic"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/prometheus/client_golang/prometheus"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Config is the chassis runtime configuration. Zero values get safe defaults
|
||||||
|
// (withDefaults); Service + Addr are the only fields a caller must set.
|
||||||
|
type Config struct {
|
||||||
|
Service string // closed-enum service name (metrics/logs)
|
||||||
|
Env string // dev | staging | prod
|
||||||
|
Addr string // listen address, e.g. ":16150"
|
||||||
|
AllowOrigins []string // exact-match CORS allowlist for browser UIs (empty = none)
|
||||||
|
MaxBodyBytes int64 // request body cap (Bind) — default 1 MiB
|
||||||
|
RequestTimeout time.Duration // per-request context deadline — default 15s
|
||||||
|
ReadyTimeout time.Duration // readiness check budget — default 2s
|
||||||
|
DrainDelay time.Duration // phase-1 drain wait before Shutdown — default 5s
|
||||||
|
ShutdownTimeout time.Duration // phase-2 in-flight drain — default 25s
|
||||||
|
|
||||||
|
// Collectors are service-owned Prometheus collectors registered on the
|
||||||
|
// app's private registry alongside the RED/Go/process ones, so /metrics is
|
||||||
|
// one scrape and there is no package-level default registry to collide in.
|
||||||
|
// Composition roots pass domain metrics here.
|
||||||
|
Collectors []prometheus.Collector
|
||||||
|
|
||||||
|
// Validator, if set, runs on every Context.Bind after JSON decode; a non-nil
|
||||||
|
// result becomes 422 Unprocessable. Wire shared/validate at the composition root.
|
||||||
|
Validator func(any) error
|
||||||
|
// EdgeMiddleware are extra http.Handler wrappers applied OUTERMOST (before
|
||||||
|
// secureHeaders) — e.g. a tracing span middleware from shared/tracing.
|
||||||
|
// Composition roots inject these; the chassis stays dependency-light.
|
||||||
|
EdgeMiddleware []func(http.Handler) http.Handler
|
||||||
|
}
|
||||||
|
|
||||||
|
// socketHeadroom is how far the socket deadlines outlive the per-request
|
||||||
|
// context deadline.
|
||||||
|
//
|
||||||
|
// ReadTimeout and WriteTimeout used to be hardcoded at 15s while
|
||||||
|
// RequestTimeout was configurable, so a surface that raised the request budget
|
||||||
|
// (reeld runs 120s so a worker can stream a rendered mp4 on the completion
|
||||||
|
// call) still had its socket cut at 15s. That reads as a proxy 502 rather than
|
||||||
|
// the board's own 413/504, which is a completely different bug to chase.
|
||||||
|
//
|
||||||
|
// The socket MUST outlive the context, never the reverse: when the context
|
||||||
|
// expires the handler returns and the error envelope is written on a socket
|
||||||
|
// that is still open. Cutting the socket first truncates the response mid-body.
|
||||||
|
const socketHeadroom = 30 * time.Second
|
||||||
|
|
||||||
|
func (c *Config) withDefaults() {
|
||||||
|
if c.MaxBodyBytes == 0 {
|
||||||
|
c.MaxBodyBytes = 1 << 20
|
||||||
|
}
|
||||||
|
if c.RequestTimeout == 0 {
|
||||||
|
c.RequestTimeout = 15 * time.Second
|
||||||
|
}
|
||||||
|
if c.ReadyTimeout == 0 {
|
||||||
|
c.ReadyTimeout = 2 * time.Second
|
||||||
|
}
|
||||||
|
if c.DrainDelay == 0 {
|
||||||
|
c.DrainDelay = 5 * time.Second
|
||||||
|
}
|
||||||
|
if c.ShutdownTimeout == 0 {
|
||||||
|
c.ShutdownTimeout = 25 * time.Second
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// App is one HTTP surface: routes + health checks + background workers wired at
|
||||||
|
// the composition root, served by Run.
|
||||||
|
type App struct {
|
||||||
|
cfg Config
|
||||||
|
log *slog.Logger
|
||||||
|
mux *http.ServeMux
|
||||||
|
metrics *metrics
|
||||||
|
checks []namedCheck
|
||||||
|
bg []func(context.Context) error
|
||||||
|
routes []Route
|
||||||
|
draining atomic.Bool
|
||||||
|
// closing is closed when phase-1 drain starts. Request handlers that hold
|
||||||
|
// a connection open indefinitely (SSE) select on it so they end at the
|
||||||
|
// drain delay instead of stalling Shutdown for the full ShutdownTimeout.
|
||||||
|
closing chan struct{}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Route is one registered endpoint. Exposed so a spec-drift test can reconcile
|
||||||
|
// the OpenAPI document against what the router actually serves — a hand-kept
|
||||||
|
// list of paths diverges from the code silently, which is the whole failure
|
||||||
|
// mode API docs have.
|
||||||
|
type Route struct {
|
||||||
|
Method string
|
||||||
|
Pattern string
|
||||||
|
}
|
||||||
|
|
||||||
|
// New builds an App and registers the always-on public routes: /metrics,
|
||||||
|
// /healthz (liveness), /readyz (readiness).
|
||||||
|
func New(cfg Config, log *slog.Logger) *App {
|
||||||
|
cfg.withDefaults()
|
||||||
|
a := &App{
|
||||||
|
cfg: cfg, log: log,
|
||||||
|
mux: http.NewServeMux(),
|
||||||
|
metrics: newMetrics(cfg.Service, cfg.Collectors...),
|
||||||
|
closing: make(chan struct{}),
|
||||||
|
}
|
||||||
|
a.mux.Handle("GET /metrics", a.metrics.handler())
|
||||||
|
a.mux.Handle("GET /healthz", a.toHTTP(a.handleLive))
|
||||||
|
a.mux.Handle("GET /readyz", a.toHTTP(a.handleReady))
|
||||||
|
return a
|
||||||
|
}
|
||||||
|
|
||||||
|
// toHTTP adapts a HandlerFunc to net/http, mapping a returned error to the
|
||||||
|
// JSON error envelope.
|
||||||
|
func (a *App) toHTTP(h HandlerFunc) http.Handler {
|
||||||
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
c := &Context{
|
||||||
|
w: w, r: r,
|
||||||
|
maxBytes: a.cfg.MaxBodyBytes,
|
||||||
|
validate: a.cfg.Validator,
|
||||||
|
closing: a.closing,
|
||||||
|
}
|
||||||
|
if err := h(c); err != nil {
|
||||||
|
a.writeError(c, err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// writeError logs the failure (5xx at error, 4xx at debug — both with the
|
||||||
|
// internal cause) and writes the client envelope, which never carries the cause
|
||||||
|
// or any secret.
|
||||||
|
func (a *App) writeError(c *Context, err error) {
|
||||||
|
e := asError(err)
|
||||||
|
log := c.Log()
|
||||||
|
// The cause rides on BOTH branches. Error.cause is documented as
|
||||||
|
// "logged server-side", and dropping it on 4xx made that false for every
|
||||||
|
// cause the framework attaches — Bind attaches one on 400, 413 and 422 and
|
||||||
|
// nothing else ever sees it. The visible symptom: a rejected body logged as
|
||||||
|
// bare `bad_request` with no field name, so the same unexplained-rejection
|
||||||
|
// hunt WorkerEligible's reason string exists to prevent on the lease path.
|
||||||
|
cause := e.Msg
|
||||||
|
if e.cause != nil {
|
||||||
|
cause = e.cause.Error()
|
||||||
|
}
|
||||||
|
if e.Status >= http.StatusInternalServerError {
|
||||||
|
log.Error("request.error", "category", "request",
|
||||||
|
"error_type", e.Code, "http_status", e.Status, "error_msg", cause)
|
||||||
|
} else {
|
||||||
|
log.Debug("request.rejected", "category", "request",
|
||||||
|
"error_type", e.Code, "http_status", e.Status, "error_msg", cause)
|
||||||
|
}
|
||||||
|
// request_id rides in the body, not just the X-Request-Id header. A browser
|
||||||
|
// on a cross-origin deployment cannot read a response header unless it is
|
||||||
|
// explicitly exposed, so a body without it leaves the user with an error and
|
||||||
|
// nothing to quote to support. The Rust track's error.rs declares this exact
|
||||||
|
// triple as the contract for every surface; omitting it here made that claim
|
||||||
|
// false for half the projects this skill generates.
|
||||||
|
rid := c.w.Header().Get("X-Request-Id")
|
||||||
|
if rid == "" {
|
||||||
|
// Sentinel, matching the Rust track. A literal "-" is itself the signal
|
||||||
|
// that the request-id middleware is not wired, which an absent key is not.
|
||||||
|
rid = "-"
|
||||||
|
}
|
||||||
|
_ = c.JSON(e.Status, map[string]any{
|
||||||
|
"error": map[string]any{"code": e.Code, "message": e.Msg, "request_id": rid},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (a *App) register(method, pattern string, h HandlerFunc, mw []Middleware) {
|
||||||
|
for i := len(mw) - 1; i >= 0; i-- { // mw[0] outermost
|
||||||
|
h = mw[i](h)
|
||||||
|
}
|
||||||
|
a.routes = append(a.routes, Route{Method: method, Pattern: pattern})
|
||||||
|
a.mux.Handle(method+" "+pattern, a.toHTTP(h))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Routes returns every route registered through Get/Post/Handle/Route, in
|
||||||
|
// registration order. The always-on probes (/healthz, /readyz, /metrics) are
|
||||||
|
// registered directly on the mux and deliberately excluded — they are chassis
|
||||||
|
// infrastructure, not part of a service's documented API surface.
|
||||||
|
func (a *App) Routes() []Route {
|
||||||
|
out := make([]Route, len(a.routes))
|
||||||
|
copy(out, a.routes)
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get/Post/Handle register a top-level route with optional route middleware.
|
||||||
|
func (a *App) Get(pattern string, h HandlerFunc, mw ...Middleware) { a.register("GET", pattern, h, mw) }
|
||||||
|
func (a *App) Post(pattern string, h HandlerFunc, mw ...Middleware) {
|
||||||
|
a.register("POST", pattern, h, mw)
|
||||||
|
}
|
||||||
|
func (a *App) Handle(method, pattern string, h HandlerFunc, mw ...Middleware) {
|
||||||
|
a.register(method, pattern, h, mw)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Route groups routes under a path prefix with shared middleware (e.g. auth).
|
||||||
|
func (a *App) Route(prefix string, fn func(r *Router)) { fn(&Router{app: a, prefix: prefix}) }
|
||||||
|
|
||||||
|
// Router registers routes under a prefix, applying group middleware to each.
|
||||||
|
type Router struct {
|
||||||
|
app *App
|
||||||
|
prefix string
|
||||||
|
mw []Middleware
|
||||||
|
}
|
||||||
|
|
||||||
|
// Use adds middleware applied to every route registered on this Router.
|
||||||
|
func (r *Router) Use(mw ...Middleware) { r.mw = append(r.mw, mw...) }
|
||||||
|
|
||||||
|
func (r *Router) Get(pattern string, h HandlerFunc, mw ...Middleware) {
|
||||||
|
r.handle("GET", pattern, h, mw)
|
||||||
|
}
|
||||||
|
func (r *Router) Post(pattern string, h HandlerFunc, mw ...Middleware) {
|
||||||
|
r.handle("POST", pattern, h, mw)
|
||||||
|
}
|
||||||
|
func (r *Router) Delete(pattern string, h HandlerFunc, mw ...Middleware) {
|
||||||
|
r.handle("DELETE", pattern, h, mw)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Handle registers any method under the group prefix.
|
||||||
|
func (r *Router) Handle(method, pattern string, h HandlerFunc, mw ...Middleware) {
|
||||||
|
r.handle(method, pattern, h, mw)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Router) handle(method, pattern string, h HandlerFunc, mw []Middleware) {
|
||||||
|
all := make([]Middleware, 0, len(r.mw)+len(mw))
|
||||||
|
all = append(all, r.mw...)
|
||||||
|
all = append(all, mw...)
|
||||||
|
r.app.register(method, r.prefix+pattern, h, all)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Health registers a named readiness dependency check.
|
||||||
|
func (a *App) Health(name string, fn CheckFunc) {
|
||||||
|
a.checks = append(a.checks, namedCheck{name: name, fn: fn})
|
||||||
|
}
|
||||||
|
|
||||||
|
// Background registers a worker run with the server lifecycle; it receives a
|
||||||
|
// context cancelled on shutdown and Run waits for it to return.
|
||||||
|
func (a *App) Background(fn func(context.Context) error) { a.bg = append(a.bg, fn) }
|
||||||
|
|
||||||
|
// Handler returns the fully composed edge chain over the route mux — used by Run
|
||||||
|
// and available for in-process tests. Order (outermost first): any injected
|
||||||
|
// EdgeMiddleware (e.g. tracing), then secureHeaders, instrument, CORS.
|
||||||
|
func (a *App) Handler() http.Handler {
|
||||||
|
mw := make([]func(http.Handler) http.Handler, 0, len(a.cfg.EdgeMiddleware)+3)
|
||||||
|
mw = append(mw, a.cfg.EdgeMiddleware...)
|
||||||
|
mw = append(mw, a.secureHeaders, a.instrument, a.cors)
|
||||||
|
return chain(a.mux, mw...)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Run starts the server and blocks until ctx is cancelled (SIGINT/SIGTERM) or
|
||||||
|
// the listener fails. Shutdown is two-phase: flip readiness to 503 so the load
|
||||||
|
// balancer drains the pod, wait DrainDelay, then Shutdown in-flight requests
|
||||||
|
// under ShutdownTimeout (DrainDelay + ShutdownTimeout MUST be < the k8s grace
|
||||||
|
// period). Liveness stays 200 throughout so the pod is drained, never killed.
|
||||||
|
func (a *App) Run(ctx context.Context) error {
|
||||||
|
// The socket deadlines track RequestTimeout, they are not independent
|
||||||
|
// knobs: see socketHeadroom. ReadTimeout has to clear a full body upload
|
||||||
|
// (reeld takes a rendered mp4 on the completion call) and WriteTimeout a
|
||||||
|
// full response, both of which are bounded by the request budget.
|
||||||
|
srv := &http.Server{
|
||||||
|
Addr: a.cfg.Addr,
|
||||||
|
Handler: a.Handler(),
|
||||||
|
ReadHeaderTimeout: 5 * time.Second, // Slowloris guard
|
||||||
|
ReadTimeout: a.cfg.RequestTimeout + socketHeadroom,
|
||||||
|
WriteTimeout: a.cfg.RequestTimeout + socketHeadroom,
|
||||||
|
IdleTimeout: 60 * time.Second,
|
||||||
|
MaxHeaderBytes: 1 << 20,
|
||||||
|
}
|
||||||
|
|
||||||
|
runCtx, cancel := context.WithCancel(ctx)
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
for _, fn := range a.bg {
|
||||||
|
wg.Add(1)
|
||||||
|
go func(fn func(context.Context) error) {
|
||||||
|
defer wg.Done()
|
||||||
|
a.superviseBackground(runCtx, fn)
|
||||||
|
}(fn)
|
||||||
|
}
|
||||||
|
|
||||||
|
errc := make(chan error, 1)
|
||||||
|
go func() {
|
||||||
|
a.log.Info("server.listening", "http_addr", a.cfg.Addr)
|
||||||
|
if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
|
||||||
|
errc <- err
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
select {
|
||||||
|
case err := <-errc:
|
||||||
|
cancel()
|
||||||
|
wg.Wait()
|
||||||
|
return err
|
||||||
|
case <-runCtx.Done():
|
||||||
|
a.draining.Store(true) // phase 1: readiness -> 503, LB drains the pod
|
||||||
|
// Long-lived streams end here rather than at phase 2: an SSE client
|
||||||
|
// that reconnects during the drain delay lands on a pod that is still
|
||||||
|
// serving, instead of one that is 25s from closing under it.
|
||||||
|
close(a.closing)
|
||||||
|
a.log.Info("server.draining", "category", "shutdown", "drain_delay_ms", a.cfg.DrainDelay.Milliseconds())
|
||||||
|
time.Sleep(a.cfg.DrainDelay)
|
||||||
|
|
||||||
|
sctx, scancel := context.WithTimeout(context.Background(), a.cfg.ShutdownTimeout) // phase 2: drain in-flight
|
||||||
|
defer scancel()
|
||||||
|
err := srv.Shutdown(sctx)
|
||||||
|
cancel()
|
||||||
|
wg.Wait()
|
||||||
|
a.log.Info("server.stopped", "category", "shutdown")
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// superviseBackground runs one background worker, restarting it with backoff if
|
||||||
|
// it panics or returns an error.
|
||||||
|
//
|
||||||
|
// Without this a panic in an auxiliary worker takes the whole process down,
|
||||||
|
// including HTTP serving — so a bug in, say, artifact retention would stop the
|
||||||
|
// board handing out leases. That asymmetry is surprising, because request
|
||||||
|
// handlers already get panic recovery from the instrument middleware; this
|
||||||
|
// gives background work the same protection.
|
||||||
|
//
|
||||||
|
// Restarting rather than merely recovering matters just as much: a worker that
|
||||||
|
// dies quietly leaves its job undone forever with a green readiness probe, and
|
||||||
|
// nobody discovers retention stopped until a disk fills.
|
||||||
|
func (a *App) superviseBackground(ctx context.Context, fn func(context.Context) error) {
|
||||||
|
const (
|
||||||
|
minBackoff = time.Second
|
||||||
|
maxBackoff = time.Minute
|
||||||
|
)
|
||||||
|
backoff := minBackoff
|
||||||
|
|
||||||
|
for {
|
||||||
|
err, panicked := runBackgroundOnce(ctx, fn)
|
||||||
|
switch {
|
||||||
|
case ctx.Err() != nil:
|
||||||
|
// Shutdown. A worker returning on a cancelled context is the
|
||||||
|
// normal path, not a failure.
|
||||||
|
return
|
||||||
|
case err == nil && !panicked:
|
||||||
|
// A clean return before shutdown means the worker considers its
|
||||||
|
// job finished. Respect that rather than spinning it forever.
|
||||||
|
return
|
||||||
|
case panicked:
|
||||||
|
a.log.Error("background.panicked", "category", "worker",
|
||||||
|
"error_type", "worker_panic", "error_msg", err.Error(),
|
||||||
|
"restart_in_ms", backoff.Milliseconds())
|
||||||
|
default:
|
||||||
|
a.log.Error("background.failed", "category", "worker",
|
||||||
|
"error_type", "worker_error", "error_msg", err.Error(),
|
||||||
|
"restart_in_ms", backoff.Milliseconds())
|
||||||
|
}
|
||||||
|
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
case <-time.After(backoff):
|
||||||
|
}
|
||||||
|
if backoff *= 2; backoff > maxBackoff {
|
||||||
|
backoff = maxBackoff
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// runBackgroundOnce invokes fn, converting a panic into an error so the
|
||||||
|
// supervisor can treat both failure modes alike. The stack is attached because
|
||||||
|
// a recovered panic with no stack is nearly unactionable.
|
||||||
|
func runBackgroundOnce(ctx context.Context, fn func(context.Context) error) (err error, panicked bool) {
|
||||||
|
defer func() {
|
||||||
|
if r := recover(); r != nil {
|
||||||
|
panicked = true
|
||||||
|
err = fmt.Errorf("panic: %v\n%s", r, debug.Stack())
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
err = fn(ctx)
|
||||||
|
if errors.Is(err, context.Canceled) {
|
||||||
|
err = nil
|
||||||
|
}
|
||||||
|
return err, false
|
||||||
|
}
|
||||||
104
vendor/github.com/orchard9/go-chassis/chassis/auth.go
generated
vendored
Normal file
104
vendor/github.com/orchard9/go-chassis/chassis/auth.go
generated
vendored
Normal file
@ -0,0 +1,104 @@
|
|||||||
|
package chassis
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/subtle"
|
||||||
|
"net/http"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Identity is the authenticated principal attached to the request context by an
|
||||||
|
// auth Middleware. Subject is the stable principal id; OrgID is the tenant the
|
||||||
|
// principal is acting in (the multi-tenant scoping key — every tenant row
|
||||||
|
// filters by it); Scopes are the granted permissions; Claims carries any extra
|
||||||
|
// verified attributes without coupling chassis to a domain type.
|
||||||
|
type Identity struct {
|
||||||
|
Subject string
|
||||||
|
OrgID string
|
||||||
|
Scopes []string
|
||||||
|
Claims map[string]string
|
||||||
|
}
|
||||||
|
|
||||||
|
// HasScope reports whether the identity holds the required scope. A held scope
|
||||||
|
// ending in ":*" dominates any required scope sharing its prefix, so "admin:*"
|
||||||
|
// satisfies "admin:read". An exact match always passes.
|
||||||
|
func (id *Identity) HasScope(required string) bool {
|
||||||
|
if id == nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, s := range id.Scopes {
|
||||||
|
if s == required {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if prefix, ok := strings.CutSuffix(s, ":*"); ok {
|
||||||
|
if reqPrefix, _, found := strings.Cut(required, ":"); found && reqPrefix == prefix {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// Authenticator verifies a request and returns its Identity, or an *Error
|
||||||
|
// (Unauthorized/Forbidden) when verification fails. Implementations:
|
||||||
|
// - NoAuth — local default, everyone is anonymous (documented seam).
|
||||||
|
// - StaticToken — shared bearer token, constant-time compared (local/CI).
|
||||||
|
// - (later) OIDC — coreos/go-oidc verifier, JWKS cache, refresh-on-kid-miss.
|
||||||
|
// - (later) APIKey — hashed key behind a Secrets port, constant-time compare.
|
||||||
|
type Authenticator interface {
|
||||||
|
Authenticate(r *http.Request) (*Identity, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
type identityKey struct{}
|
||||||
|
|
||||||
|
// IdentityFrom returns the authenticated identity, or false when the route was
|
||||||
|
// not behind RequireAuth.
|
||||||
|
func IdentityFrom(ctx context.Context) (*Identity, bool) {
|
||||||
|
id, ok := ctx.Value(identityKey{}).(*Identity)
|
||||||
|
return id, ok
|
||||||
|
}
|
||||||
|
|
||||||
|
// RequireAuth is route/group middleware that runs the Authenticator, rejecting
|
||||||
|
// the request (deny-by-default) when it fails and otherwise stashing the
|
||||||
|
// Identity in the context for handlers.
|
||||||
|
func RequireAuth(a Authenticator) Middleware {
|
||||||
|
return func(next HandlerFunc) HandlerFunc {
|
||||||
|
return func(c *Context) error {
|
||||||
|
id, err := a.Authenticate(c.r)
|
||||||
|
if err != nil {
|
||||||
|
return err // already an *Error (Unauthorized/Forbidden)
|
||||||
|
}
|
||||||
|
c.r = c.r.WithContext(context.WithValue(c.r.Context(), identityKey{}, id))
|
||||||
|
return next(c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// NoAuth treats every caller as anonymous. The explicit local default — never
|
||||||
|
// select it for a protected environment.
|
||||||
|
type NoAuth struct{}
|
||||||
|
|
||||||
|
func (NoAuth) Authenticate(*http.Request) (*Identity, error) {
|
||||||
|
return &Identity{Subject: "anonymous"}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// StaticToken authenticates a single shared bearer token in constant time
|
||||||
|
// (crypto/subtle) to avoid leaking the token via response timing. For local/CI
|
||||||
|
// and internal service-to-service; real principals use OIDC/API-key later.
|
||||||
|
type StaticToken struct{ token string }
|
||||||
|
|
||||||
|
// NewStaticToken builds a StaticToken; an empty token rejects every request.
|
||||||
|
func NewStaticToken(token string) StaticToken { return StaticToken{token: token} }
|
||||||
|
|
||||||
|
func (s StaticToken) Authenticate(r *http.Request) (*Identity, error) {
|
||||||
|
const prefix = "Bearer "
|
||||||
|
h := r.Header.Get("Authorization")
|
||||||
|
if s.token == "" || !strings.HasPrefix(h, prefix) {
|
||||||
|
return nil, Unauthorized("missing or malformed bearer token")
|
||||||
|
}
|
||||||
|
got := strings.TrimPrefix(h, prefix)
|
||||||
|
if subtle.ConstantTimeCompare([]byte(got), []byte(s.token)) != 1 {
|
||||||
|
return nil, Unauthorized("invalid token")
|
||||||
|
}
|
||||||
|
return &Identity{Subject: "static-token"}, nil
|
||||||
|
}
|
||||||
37
vendor/github.com/orchard9/go-chassis/chassis/authz.go
generated
vendored
Normal file
37
vendor/github.com/orchard9/go-chassis/chassis/authz.go
generated
vendored
Normal file
@ -0,0 +1,37 @@
|
|||||||
|
package chassis
|
||||||
|
|
||||||
|
// Authorization middleware — the layer above authentication. RequireAuth proves
|
||||||
|
// WHO; these prove WHAT they may do and WHICH tenant they act in. Apply after
|
||||||
|
// RequireAuth on a route group.
|
||||||
|
|
||||||
|
// RequireScope rejects callers lacking the given scope (scope dominance applies:
|
||||||
|
// a held "x:*" satisfies "x:read"). 401 if unauthenticated, 403 if under-scoped.
|
||||||
|
func RequireScope(scope string) Middleware {
|
||||||
|
return func(next HandlerFunc) HandlerFunc {
|
||||||
|
return func(c *Context) error {
|
||||||
|
id, ok := c.Identity()
|
||||||
|
if !ok {
|
||||||
|
return Unauthorized("authentication required")
|
||||||
|
}
|
||||||
|
if !id.HasScope(scope) {
|
||||||
|
return Forbidden("missing required scope: " + scope)
|
||||||
|
}
|
||||||
|
return next(c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// RequireOrg enforces the multi-tenant invariant: the caller must have a resolved
|
||||||
|
// active organization. Handlers behind it can rely on a non-empty Context.OrgID
|
||||||
|
// and MUST filter every tenant query by it.
|
||||||
|
func RequireOrg() Middleware {
|
||||||
|
return func(next HandlerFunc) HandlerFunc {
|
||||||
|
return func(c *Context) error {
|
||||||
|
id, ok := c.Identity()
|
||||||
|
if !ok || id.OrgID == "" {
|
||||||
|
return Forbidden("no active organization")
|
||||||
|
}
|
||||||
|
return next(c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
139
vendor/github.com/orchard9/go-chassis/chassis/context.go
generated
vendored
Normal file
139
vendor/github.com/orchard9/go-chassis/chassis/context.go
generated
vendored
Normal file
@ -0,0 +1,139 @@
|
|||||||
|
package chassis
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"log/slog"
|
||||||
|
"net/http"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/orchard9/go-chassis/logging"
|
||||||
|
)
|
||||||
|
|
||||||
|
// HandlerFunc is the chassis handler signature: return an error and the
|
||||||
|
// framework maps it to the JSON error envelope. See patterns/go-chassis.md.
|
||||||
|
type HandlerFunc func(*Context) error
|
||||||
|
|
||||||
|
// Middleware decorates a HandlerFunc (route/group scope: auth, rate-limit).
|
||||||
|
// Cross-cutting infra (recover, request-id, logging, metrics) is applied once
|
||||||
|
// at the server edge as http.Handler middleware, not here.
|
||||||
|
type Middleware func(HandlerFunc) HandlerFunc
|
||||||
|
|
||||||
|
// Context carries the request/response for one call plus typed helpers.
|
||||||
|
type Context struct {
|
||||||
|
w http.ResponseWriter
|
||||||
|
r *http.Request
|
||||||
|
maxBytes int64
|
||||||
|
validate func(any) error // optional; set from Config.Validator
|
||||||
|
// closing fires when the app begins draining, so a Stream ends with the
|
||||||
|
// pod instead of holding Shutdown open.
|
||||||
|
closing <-chan struct{}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Context returns the request context (deadline + request-scoped logger).
|
||||||
|
func (c *Context) Context() context.Context { return c.r.Context() }
|
||||||
|
|
||||||
|
// Request exposes the raw request for the rare case a helper does not cover.
|
||||||
|
func (c *Context) Request() *http.Request { return c.r }
|
||||||
|
|
||||||
|
// Writer exposes the raw ResponseWriter for the three responses the JSON
|
||||||
|
// envelope cannot carry: the Scalar docs page (HTML), the OpenAPI document
|
||||||
|
// (pre-rendered bytes), and a streamed artifact (video/mp4).
|
||||||
|
func (c *Context) Writer() http.ResponseWriter { return c.w }
|
||||||
|
|
||||||
|
// Log returns the request-scoped logger (carries request_id).
|
||||||
|
func (c *Context) Log() *slog.Logger { return logging.From(c.r.Context()) }
|
||||||
|
|
||||||
|
// PathValue returns a ServeMux wildcard value, e.g. {id} from "/v1/x/{id}".
|
||||||
|
func (c *Context) PathValue(key string) string { return c.r.PathValue(key) }
|
||||||
|
|
||||||
|
// Identity returns the authenticated principal, or false when the route was not
|
||||||
|
// behind RequireAuth.
|
||||||
|
func (c *Context) Identity() (*Identity, bool) { return IdentityFrom(c.r.Context()) }
|
||||||
|
|
||||||
|
// OrgID returns the active tenant for the caller ("" when unauthenticated or the
|
||||||
|
// identity carries no org). Repos MUST filter every tenant query by it.
|
||||||
|
func (c *Context) OrgID() string {
|
||||||
|
if id, ok := c.Identity(); ok {
|
||||||
|
return id.OrgID
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// TraceID returns the request's trace id (W3C traceparent or generated).
|
||||||
|
func (c *Context) TraceID() string {
|
||||||
|
id, _ := TraceIDFrom(c.r.Context())
|
||||||
|
return id
|
||||||
|
}
|
||||||
|
|
||||||
|
// Bind enforces the body-size limit, then JSON-decodes into v rejecting unknown
|
||||||
|
// fields. An oversized body becomes 413; a malformed body becomes 400 — neither
|
||||||
|
// leaks internals to the client.
|
||||||
|
func (c *Context) Bind(v any) error {
|
||||||
|
c.r.Body = http.MaxBytesReader(c.w, c.r.Body, c.maxBytes)
|
||||||
|
dec := json.NewDecoder(c.r.Body)
|
||||||
|
dec.DisallowUnknownFields()
|
||||||
|
if err := dec.Decode(v); err != nil {
|
||||||
|
var maxErr *http.MaxBytesError
|
||||||
|
if errors.As(err, &maxErr) {
|
||||||
|
return PayloadTooLarge("request body too large").WithCause(err)
|
||||||
|
}
|
||||||
|
if f, ok := unknownField(err); ok {
|
||||||
|
return BadRequest(fmt.Sprintf("unknown field %q", f)).WithCause(err)
|
||||||
|
}
|
||||||
|
return BadRequest("invalid JSON body").WithCause(err)
|
||||||
|
}
|
||||||
|
if c.validate != nil {
|
||||||
|
if err := c.validate(v); err != nil {
|
||||||
|
return Unprocessable("request failed validation").WithCause(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// unknownFieldPrefix is what encoding/json returns under
|
||||||
|
// DisallowUnknownFields. The stdlib gives no typed error for it, so matching the
|
||||||
|
// text is the only way to tell "you sent a field we do not accept" from "your
|
||||||
|
// JSON is broken" — and they are different bugs on the caller's side.
|
||||||
|
const unknownFieldPrefix = `json: unknown field `
|
||||||
|
|
||||||
|
// unknownField extracts the rejected field name, if that is why decoding failed.
|
||||||
|
//
|
||||||
|
// The name is echoed to the client because the alternative is what this cost us:
|
||||||
|
// a worker POSTing a well-formed body with one extra key is told "invalid JSON
|
||||||
|
// body", which is false, and has nothing to act on. Echoing a key the caller
|
||||||
|
// just sent leaks nothing. The rest of the decoder's errors stay generic — an
|
||||||
|
// UnmarshalTypeError names Go struct fields and types, which is internal detail.
|
||||||
|
func unknownField(err error) (string, bool) {
|
||||||
|
msg := err.Error()
|
||||||
|
if !strings.HasPrefix(msg, unknownFieldPrefix) {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
name, uerr := strconv.Unquote(strings.TrimPrefix(msg, unknownFieldPrefix))
|
||||||
|
if uerr != nil {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
return name, true
|
||||||
|
}
|
||||||
|
|
||||||
|
// OK writes 200 + JSON. Created writes 201. NoContent writes 204.
|
||||||
|
func (c *Context) OK(v any) error { return c.JSON(http.StatusOK, v) }
|
||||||
|
func (c *Context) Created(v any) error { return c.JSON(http.StatusCreated, v) }
|
||||||
|
|
||||||
|
func (c *Context) NoContent() error {
|
||||||
|
c.w.WriteHeader(http.StatusNoContent)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// JSON writes the status code and JSON-encodes v.
|
||||||
|
func (c *Context) JSON(code int, v any) error {
|
||||||
|
c.w.Header().Set("Content-Type", "application/json; charset=utf-8")
|
||||||
|
c.w.WriteHeader(code)
|
||||||
|
if v == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return json.NewEncoder(c.w).Encode(v)
|
||||||
|
}
|
||||||
74
vendor/github.com/orchard9/go-chassis/chassis/errors.go
generated
vendored
Normal file
74
vendor/github.com/orchard9/go-chassis/chassis/errors.go
generated
vendored
Normal file
@ -0,0 +1,74 @@
|
|||||||
|
package chassis
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Error is the HTTP error model: a stable machine Code + a SAFE public Msg
|
||||||
|
// (no secrets, no internal detail) shown to the client, plus an internal cause
|
||||||
|
// that is logged server-side and NEVER serialized to the response. Handlers
|
||||||
|
// return these; the framework maps them to the JSON error envelope.
|
||||||
|
//
|
||||||
|
// This is distinct from any domain result/status enum a handler returns as a
|
||||||
|
// SUCCESS-response field — those are 200 bodies, not errors.
|
||||||
|
type Error struct {
|
||||||
|
Status int // HTTP status code
|
||||||
|
Code string // stable machine-readable code (snake_case)
|
||||||
|
Msg string // safe public message
|
||||||
|
cause error // internal cause: logged, never sent to the client
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *Error) Error() string {
|
||||||
|
if e.cause != nil {
|
||||||
|
return fmt.Sprintf("%s: %s: %v", e.Code, e.Msg, e.cause)
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("%s: %s", e.Code, e.Msg)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *Error) Unwrap() error { return e.cause }
|
||||||
|
|
||||||
|
// WithCause attaches the internal cause (logged, never sent). Returns a copy so
|
||||||
|
// the package-level constructors stay immutable.
|
||||||
|
func (e *Error) WithCause(err error) *Error {
|
||||||
|
c := *e
|
||||||
|
c.cause = err
|
||||||
|
return &c
|
||||||
|
}
|
||||||
|
|
||||||
|
func newErr(status int, code, msg string) *Error {
|
||||||
|
return &Error{Status: status, Code: code, Msg: msg}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Constructors — public Msg defaults are safe and generic; override per call.
|
||||||
|
func BadRequest(msg string) *Error { return newErr(http.StatusBadRequest, "bad_request", msg) }
|
||||||
|
func Unauthorized(msg string) *Error { return newErr(http.StatusUnauthorized, "unauthorized", msg) }
|
||||||
|
func Forbidden(msg string) *Error { return newErr(http.StatusForbidden, "forbidden", msg) }
|
||||||
|
func NotFound(msg string) *Error { return newErr(http.StatusNotFound, "not_found", msg) }
|
||||||
|
func Conflict(msg string) *Error { return newErr(http.StatusConflict, "conflict", msg) }
|
||||||
|
func Unprocessable(msg string) *Error {
|
||||||
|
return newErr(http.StatusUnprocessableEntity, "unprocessable", msg)
|
||||||
|
}
|
||||||
|
func TooManyRequests(msg string) *Error {
|
||||||
|
return newErr(http.StatusTooManyRequests, "rate_limited", msg)
|
||||||
|
}
|
||||||
|
func PayloadTooLarge(msg string) *Error {
|
||||||
|
return newErr(http.StatusRequestEntityTooLarge, "payload_too_large", msg)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Internal wraps any non-Error as a 500 with a generic public message; the real
|
||||||
|
// cause is preserved for logging but hidden from the client.
|
||||||
|
func Internal(cause error) *Error {
|
||||||
|
return (&Error{Status: http.StatusInternalServerError, Code: "internal", Msg: "internal error"}).WithCause(cause)
|
||||||
|
}
|
||||||
|
|
||||||
|
// asError maps any error to an *Error, defaulting unknown errors to a 500 whose
|
||||||
|
// detail is hidden from the client.
|
||||||
|
func asError(err error) *Error {
|
||||||
|
var e *Error
|
||||||
|
if errors.As(err, &e) {
|
||||||
|
return e
|
||||||
|
}
|
||||||
|
return Internal(err)
|
||||||
|
}
|
||||||
50
vendor/github.com/orchard9/go-chassis/chassis/health.go
generated
vendored
Normal file
50
vendor/github.com/orchard9/go-chassis/chassis/health.go
generated
vendored
Normal file
@ -0,0 +1,50 @@
|
|||||||
|
package chassis
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"net/http"
|
||||||
|
)
|
||||||
|
|
||||||
|
// CheckFunc reports whether a dependency is usable right now. It MUST honor the
|
||||||
|
// passed ctx deadline and return a non-nil error (with no secrets) when unhealthy.
|
||||||
|
type CheckFunc func(ctx context.Context) error
|
||||||
|
|
||||||
|
type namedCheck struct {
|
||||||
|
name string
|
||||||
|
fn CheckFunc
|
||||||
|
}
|
||||||
|
|
||||||
|
// handleLive is the k8s liveness probe: 200 while the process runs. It does NOT
|
||||||
|
// check dependencies and stays 200 during drain — a flapping dep or an
|
||||||
|
// in-progress shutdown must never trigger a restart, only LB removal.
|
||||||
|
func (a *App) handleLive(c *Context) error {
|
||||||
|
return c.OK(map[string]any{"status": "ok"})
|
||||||
|
}
|
||||||
|
|
||||||
|
// handleReady is the k8s readiness probe. It returns 503 the moment shutdown
|
||||||
|
// begins (two-phase drain: the LB pulls the pod before connections close), and
|
||||||
|
// otherwise 200 only when every registered dependency check passes.
|
||||||
|
func (a *App) handleReady(c *Context) error {
|
||||||
|
if a.draining.Load() {
|
||||||
|
return c.JSON(http.StatusServiceUnavailable, map[string]any{"status": "draining"})
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(c.Context(), a.cfg.ReadyTimeout)
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
checks := make(map[string]string, len(a.checks))
|
||||||
|
ok := true
|
||||||
|
for _, ch := range a.checks {
|
||||||
|
if err := ch.fn(ctx); err != nil {
|
||||||
|
checks[ch.name] = "error: " + err.Error()
|
||||||
|
ok = false
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
checks[ch.name] = "ok"
|
||||||
|
}
|
||||||
|
|
||||||
|
status, code := "ok", http.StatusOK
|
||||||
|
if !ok {
|
||||||
|
status, code = "unready", http.StatusServiceUnavailable
|
||||||
|
}
|
||||||
|
return c.JSON(code, map[string]any{"status": status, "checks": checks})
|
||||||
|
}
|
||||||
119
vendor/github.com/orchard9/go-chassis/chassis/idempotency.go
generated
vendored
Normal file
119
vendor/github.com/orchard9/go-chassis/chassis/idempotency.go
generated
vendored
Normal file
@ -0,0 +1,119 @@
|
|||||||
|
package chassis
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"net/http"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// StoredResponse is a cached successful response, replayed verbatim (status +
|
||||||
|
// headers + body) on a duplicate Idempotency-Key.
|
||||||
|
type StoredResponse struct {
|
||||||
|
Status int
|
||||||
|
Header http.Header
|
||||||
|
Body []byte
|
||||||
|
}
|
||||||
|
|
||||||
|
// IdempotencyStore persists and replays responses keyed by Idempotency-Key.
|
||||||
|
// Back it with Redis (TTL'd) in shared/adapters/redis. Lookup returns found=false
|
||||||
|
// when the key is new; Save records a successful response.
|
||||||
|
type IdempotencyStore interface {
|
||||||
|
Lookup(ctx context.Context, key string) (StoredResponse, bool, error)
|
||||||
|
Save(ctx context.Context, key string, resp StoredResponse, ttl time.Duration) error
|
||||||
|
}
|
||||||
|
|
||||||
|
// Idempotent dedups mutating requests by the Idempotency-Key header: a duplicate
|
||||||
|
// returns the original response without re-running the handler. Apply it to
|
||||||
|
// POST/PUT route groups. Requests without the header pass straight through. The
|
||||||
|
// key is scoped by the caller's org so keys can't collide across tenants. The
|
||||||
|
// store is best-effort: a backend error fails open (the request proceeds) rather
|
||||||
|
// than blocking writes.
|
||||||
|
func Idempotent(store IdempotencyStore, ttl time.Duration) Middleware {
|
||||||
|
return func(next HandlerFunc) HandlerFunc {
|
||||||
|
return func(c *Context) error {
|
||||||
|
key := c.r.Header.Get("Idempotency-Key")
|
||||||
|
if key == "" {
|
||||||
|
return next(c)
|
||||||
|
}
|
||||||
|
// Scope by tenant. With no resolved org we refuse to cache rather than
|
||||||
|
// share a global (cross-tenant) key — pair this with RequireOrg.
|
||||||
|
org := c.OrgID()
|
||||||
|
if org == "" {
|
||||||
|
c.Log().Warn("idempotency.skipped_no_org", "category", "idempotency")
|
||||||
|
return next(c)
|
||||||
|
}
|
||||||
|
scoped := org + ":" + key
|
||||||
|
ctx := c.r.Context()
|
||||||
|
|
||||||
|
if sr, found, err := store.Lookup(ctx, scoped); err != nil {
|
||||||
|
c.Log().Warn("idempotency.lookup_failed", "category", "idempotency", "error_msg", err.Error())
|
||||||
|
} else if found {
|
||||||
|
h := c.w.Header()
|
||||||
|
for k, vs := range sr.Header { // replay the original headers verbatim
|
||||||
|
for _, v := range vs {
|
||||||
|
h.Add(k, v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
h.Set("Idempotency-Replayed", "true")
|
||||||
|
c.w.WriteHeader(sr.Status)
|
||||||
|
_, werr := c.w.Write(sr.Body)
|
||||||
|
return werr
|
||||||
|
}
|
||||||
|
|
||||||
|
// Capture the handler's response so we can persist + flush it. Restore
|
||||||
|
// the writer via defer so a panic in next() can't leave c.w dangling.
|
||||||
|
rec := &captureWriter{header: http.Header{}, status: http.StatusOK}
|
||||||
|
orig := c.w
|
||||||
|
c.w = rec
|
||||||
|
defer func() { c.w = orig }()
|
||||||
|
if err := next(c); err != nil {
|
||||||
|
return err // errors aren't cached; the framework writes the envelope to orig
|
||||||
|
}
|
||||||
|
|
||||||
|
for k, vs := range rec.header {
|
||||||
|
for _, v := range vs {
|
||||||
|
orig.Header().Add(k, v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
orig.WriteHeader(rec.status)
|
||||||
|
if _, werr := orig.Write(rec.buf.Bytes()); werr != nil {
|
||||||
|
return werr
|
||||||
|
}
|
||||||
|
if rec.status >= 200 && rec.status < 300 {
|
||||||
|
body := append([]byte(nil), rec.buf.Bytes()...)
|
||||||
|
stored := StoredResponse{Status: rec.status, Header: rec.header.Clone(), Body: body}
|
||||||
|
if serr := store.Save(ctx, scoped, stored, ttl); serr != nil {
|
||||||
|
c.Log().Warn("idempotency.save_failed", "category", "idempotency", "error_msg", serr.Error())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// captureWriter buffers a handler's response so the idempotency middleware can
|
||||||
|
// persist and replay it. It implements http.ResponseWriter.
|
||||||
|
type captureWriter struct {
|
||||||
|
header http.Header
|
||||||
|
buf bytes.Buffer
|
||||||
|
status int
|
||||||
|
wrote bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *captureWriter) Header() http.Header { return c.header }
|
||||||
|
|
||||||
|
func (c *captureWriter) WriteHeader(code int) {
|
||||||
|
if c.wrote {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
c.status = code
|
||||||
|
c.wrote = true
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *captureWriter) Write(b []byte) (int, error) {
|
||||||
|
if !c.wrote {
|
||||||
|
c.WriteHeader(http.StatusOK)
|
||||||
|
}
|
||||||
|
return c.buf.Write(b)
|
||||||
|
}
|
||||||
67
vendor/github.com/orchard9/go-chassis/chassis/metrics.go
generated
vendored
Normal file
67
vendor/github.com/orchard9/go-chassis/chassis/metrics.go
generated
vendored
Normal file
@ -0,0 +1,67 @@
|
|||||||
|
package chassis
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
"strconv"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/prometheus/client_golang/prometheus"
|
||||||
|
"github.com/prometheus/client_golang/prometheus/collectors"
|
||||||
|
"github.com/prometheus/client_golang/prometheus/promhttp"
|
||||||
|
)
|
||||||
|
|
||||||
|
// metrics holds the RED HTTP collectors on a private registry (no global state,
|
||||||
|
// so tests and multiple apps never collide). The route label is the matched
|
||||||
|
// ServeMux pattern — bounded cardinality, never the raw path.
|
||||||
|
type metrics struct {
|
||||||
|
reg *prometheus.Registry
|
||||||
|
reqs *prometheus.CounterVec
|
||||||
|
dur *prometheus.HistogramVec
|
||||||
|
inflight prometheus.Gauge
|
||||||
|
}
|
||||||
|
|
||||||
|
// newMetrics builds the RED collectors and registers them alongside any
|
||||||
|
// service-owned collectors (Config.Collectors) on the same private registry —
|
||||||
|
// so /metrics is one scrape and a domain gauge cannot be lost to a second
|
||||||
|
// endpoint nobody remembers to scrape.
|
||||||
|
func newMetrics(service string, extra ...prometheus.Collector) *metrics {
|
||||||
|
reg := prometheus.NewRegistry()
|
||||||
|
labels := prometheus.Labels{"service": service}
|
||||||
|
m := &metrics{
|
||||||
|
reg: reg,
|
||||||
|
reqs: prometheus.NewCounterVec(prometheus.CounterOpts{
|
||||||
|
Name: "http_requests_total", Help: "Total HTTP requests.", ConstLabels: labels,
|
||||||
|
}, []string{"method", "route", "status"}),
|
||||||
|
dur: prometheus.NewHistogramVec(prometheus.HistogramOpts{
|
||||||
|
Name: "http_request_duration_seconds", Help: "HTTP request latency.",
|
||||||
|
Buckets: prometheus.DefBuckets, ConstLabels: labels,
|
||||||
|
}, []string{"method", "route"}),
|
||||||
|
inflight: prometheus.NewGauge(prometheus.GaugeOpts{
|
||||||
|
Name: "http_requests_in_flight", Help: "In-flight HTTP requests.", ConstLabels: labels,
|
||||||
|
}),
|
||||||
|
}
|
||||||
|
reg.MustRegister(m.reqs, m.dur, m.inflight,
|
||||||
|
collectors.NewGoCollector(),
|
||||||
|
collectors.NewProcessCollector(collectors.ProcessCollectorOpts{}))
|
||||||
|
for _, c := range extra {
|
||||||
|
reg.MustRegister(c)
|
||||||
|
}
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *metrics) observe(method, route string, status int, d time.Duration) {
|
||||||
|
m.countOnly(method, route, status)
|
||||||
|
m.dur.WithLabelValues(method, route).Observe(d.Seconds())
|
||||||
|
}
|
||||||
|
|
||||||
|
// countOnly records a request without timing it. For responses whose elapsed
|
||||||
|
// time is not a latency — an event stream ends when the operator closes the
|
||||||
|
// tab, and putting that in the histogram makes every latency alert lie.
|
||||||
|
func (m *metrics) countOnly(method, route string, status int) {
|
||||||
|
m.reqs.WithLabelValues(method, route, strconv.Itoa(status)).Inc()
|
||||||
|
}
|
||||||
|
|
||||||
|
// handler serves the Prometheus exposition for the metrics agent to scrape.
|
||||||
|
func (m *metrics) handler() http.Handler {
|
||||||
|
return promhttp.HandlerFor(m.reg, promhttp.HandlerOpts{Registry: m.reg})
|
||||||
|
}
|
||||||
244
vendor/github.com/orchard9/go-chassis/chassis/middleware.go
generated
vendored
Normal file
244
vendor/github.com/orchard9/go-chassis/chassis/middleware.go
generated
vendored
Normal file
@ -0,0 +1,244 @@
|
|||||||
|
package chassis
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/rand"
|
||||||
|
"encoding/hex"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"runtime/debug"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/orchard9/go-chassis/logging"
|
||||||
|
)
|
||||||
|
|
||||||
|
// chain composes edge middleware so mw[0] is outermost (runs first).
|
||||||
|
func chain(h http.Handler, mw ...func(http.Handler) http.Handler) http.Handler {
|
||||||
|
for i := len(mw) - 1; i >= 0; i-- {
|
||||||
|
h = mw[i](h)
|
||||||
|
}
|
||||||
|
return h
|
||||||
|
}
|
||||||
|
|
||||||
|
// statusRecorder captures the response status for logging/metrics and survives
|
||||||
|
// double WriteHeader; it forwards Flush so SSE/streaming handlers still work.
|
||||||
|
type statusRecorder struct {
|
||||||
|
http.ResponseWriter
|
||||||
|
status int
|
||||||
|
wrote bool
|
||||||
|
// streamed is set by Context.Stream. It keeps a connection an operator
|
||||||
|
// held open for twenty minutes out of the request-latency histogram.
|
||||||
|
streamed bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *statusRecorder) WriteHeader(code int) {
|
||||||
|
if r.wrote {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
r.status = code
|
||||||
|
r.wrote = true
|
||||||
|
r.ResponseWriter.WriteHeader(code)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *statusRecorder) Write(b []byte) (int, error) {
|
||||||
|
if !r.wrote {
|
||||||
|
r.WriteHeader(http.StatusOK)
|
||||||
|
}
|
||||||
|
return r.ResponseWriter.Write(b)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *statusRecorder) Flush() {
|
||||||
|
if f, ok := r.ResponseWriter.(http.Flusher); ok {
|
||||||
|
f.Flush()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Unwrap exposes the wrapped writer to http.ResponseController, which is how a
|
||||||
|
// streaming handler clears the server's WriteTimeout for its own connection.
|
||||||
|
// Without it ResponseController stops at this wrapper and every deadline call
|
||||||
|
// returns ErrNotSupported — the response is then cut mid-stream at
|
||||||
|
// RequestTimeout+socketHeadroom with no error the handler can report.
|
||||||
|
func (r *statusRecorder) Unwrap() http.ResponseWriter { return r.ResponseWriter }
|
||||||
|
|
||||||
|
func newRequestID() string {
|
||||||
|
var b [16]byte
|
||||||
|
if _, err := rand.Read(b[:]); err != nil {
|
||||||
|
return strconv.FormatInt(time.Now().UnixNano(), 16)
|
||||||
|
}
|
||||||
|
return hex.EncodeToString(b[:])
|
||||||
|
}
|
||||||
|
|
||||||
|
type traceIDKey struct{}
|
||||||
|
|
||||||
|
// TraceIDFrom returns the request's trace id, or false outside a chassis-handled
|
||||||
|
// request. shared/httpclient uses it to propagate traceparent downstream.
|
||||||
|
func TraceIDFrom(ctx context.Context) (string, bool) {
|
||||||
|
id, ok := ctx.Value(traceIDKey{}).(string)
|
||||||
|
return id, ok
|
||||||
|
}
|
||||||
|
|
||||||
|
// traceIDFromHeader extracts the 32-hex trace-id from a W3C traceparent header
|
||||||
|
// ("00-<32hex>-<16hex>-<flags>"); on absence/malformation it generates one.
|
||||||
|
func traceIDFromHeader(traceparent string) string {
|
||||||
|
parts := strings.Split(traceparent, "-")
|
||||||
|
if len(parts) == 4 && len(parts[1]) == 32 && parts[1] != strings.Repeat("0", 32) {
|
||||||
|
if _, err := hex.DecodeString(parts[1]); err == nil {
|
||||||
|
return parts[1]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return newRequestID()
|
||||||
|
}
|
||||||
|
|
||||||
|
// instrument is the core edge middleware: it assigns/propagates a request_id,
|
||||||
|
// builds the request-scoped logger, applies the per-request timeout, counts
|
||||||
|
// in-flight requests, RECOVERS panics (so the access log + metrics observe the
|
||||||
|
// real 500), and emits one access line + RED metrics. The route label is the
|
||||||
|
// matched ServeMux pattern (bounded cardinality), never the raw path.
|
||||||
|
func (a *App) instrument(next http.Handler) http.Handler {
|
||||||
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
id := r.Header.Get("X-Request-Id")
|
||||||
|
if id == "" {
|
||||||
|
id = newRequestID()
|
||||||
|
}
|
||||||
|
w.Header().Set("X-Request-Id", id)
|
||||||
|
|
||||||
|
// Derive the trace id from W3C traceparent (or generate one) so logs +
|
||||||
|
// downstream calls correlate. trace_id is a regular log field, NEVER a
|
||||||
|
// stream field (high cardinality).
|
||||||
|
trace := traceIDFromHeader(r.Header.Get("traceparent"))
|
||||||
|
w.Header().Set("X-Trace-Id", trace)
|
||||||
|
|
||||||
|
log := a.log.With("request_id", id, "trace_id", trace)
|
||||||
|
ctx := context.WithValue(r.Context(), traceIDKey{}, trace)
|
||||||
|
ctx = logging.Into(ctx, log)
|
||||||
|
if a.cfg.RequestTimeout > 0 {
|
||||||
|
// A deadline cannot be removed from a derived context, so the
|
||||||
|
// pre-timeout one is carried alongside for Context.Stream: a
|
||||||
|
// server-sent-events response is open for minutes by design and
|
||||||
|
// must stay bound to client disconnect and drain, not to the
|
||||||
|
// per-request budget every other route wants.
|
||||||
|
ctx = context.WithValue(ctx, untimedKey{}, ctx)
|
||||||
|
var cancel context.CancelFunc
|
||||||
|
ctx, cancel = context.WithTimeout(ctx, a.cfg.RequestTimeout)
|
||||||
|
defer cancel()
|
||||||
|
}
|
||||||
|
r2 := r.WithContext(ctx)
|
||||||
|
|
||||||
|
rec := &statusRecorder{ResponseWriter: w, status: http.StatusOK}
|
||||||
|
a.metrics.inflight.Inc()
|
||||||
|
defer a.metrics.inflight.Dec()
|
||||||
|
start := time.Now()
|
||||||
|
|
||||||
|
func() {
|
||||||
|
defer func() {
|
||||||
|
if rv := recover(); rv != nil {
|
||||||
|
log.Error("panic.recovered", "category", "panic",
|
||||||
|
"error_type", "panic", "error_msg", fmt.Sprint(rv),
|
||||||
|
"stack", string(debug.Stack()))
|
||||||
|
if !rec.wrote {
|
||||||
|
rec.Header().Set("Content-Type", "application/json; charset=utf-8")
|
||||||
|
rec.WriteHeader(http.StatusInternalServerError)
|
||||||
|
// Same envelope as writeError, request_id included. A panic
|
||||||
|
// is precisely when a user needs an id to quote, so this is
|
||||||
|
// the worst response to leave it out of.
|
||||||
|
_, _ = fmt.Fprintf(rec,
|
||||||
|
`{"error":{"code":"internal","message":"internal error","request_id":%q}}`, id)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
next.ServeHTTP(rec, r2)
|
||||||
|
}()
|
||||||
|
|
||||||
|
route := routeLabel(r2.Pattern)
|
||||||
|
dur := time.Since(start)
|
||||||
|
log.Debug("http.request",
|
||||||
|
"http_method", r.Method, "http_route", route, "http_path", r.URL.Path,
|
||||||
|
"http_status", rec.status, "http_duration_ms", dur.Milliseconds(),
|
||||||
|
"http_streamed", rec.streamed)
|
||||||
|
// A stream's elapsed time is how long an operator left a tab open, not
|
||||||
|
// how long the service took to answer. Feeding it to the latency
|
||||||
|
// histogram fired HighRequestLatency the first time somebody opened the
|
||||||
|
// console and would poison every latency panel for the whole service,
|
||||||
|
// so a streamed response is counted but not timed.
|
||||||
|
if rec.streamed {
|
||||||
|
a.metrics.countOnly(r.Method, route, rec.status)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
a.metrics.observe(r.Method, route, rec.status, dur)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// routeLabel reduces a matched ServeMux pattern ("GET /v1/x/{id}") to its path
|
||||||
|
// shape ("/v1/x/{id}") for a bounded metrics/log label — never the raw path.
|
||||||
|
func routeLabel(pattern string) string {
|
||||||
|
if pattern == "" {
|
||||||
|
return "other"
|
||||||
|
}
|
||||||
|
if i := strings.IndexByte(pattern, ' '); i >= 0 { // drop the "METHOD " prefix
|
||||||
|
pattern = pattern[i+1:]
|
||||||
|
}
|
||||||
|
return pattern
|
||||||
|
}
|
||||||
|
|
||||||
|
// secureHeaders sets conservative response headers for a JSON API (no inline
|
||||||
|
// content, no framing). HSTS is emitted only in prod, where TLS terminates.
|
||||||
|
func (a *App) secureHeaders(next http.Handler) http.Handler {
|
||||||
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
h := w.Header()
|
||||||
|
h.Set("X-Content-Type-Options", "nosniff")
|
||||||
|
h.Set("X-Frame-Options", "DENY")
|
||||||
|
h.Set("Referrer-Policy", "no-referrer")
|
||||||
|
h.Set("Content-Security-Policy", "default-src 'none'; frame-ancestors 'none'")
|
||||||
|
if a.cfg.Env == "prod" {
|
||||||
|
h.Set("Strict-Transport-Security", "max-age=31536000; includeSubDomains")
|
||||||
|
}
|
||||||
|
next.ServeHTTP(w, r)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// cors echoes the request Origin when it is on the allowlist and answers
|
||||||
|
// preflight. A platform serves more than one browser surface (admin, tenant,
|
||||||
|
// operator), so the allowlist is a set and the response echoes the matched
|
||||||
|
// origin rather than a single fixed value — echoing a fixed origin breaks every
|
||||||
|
// UI but one, and echoing the request unchecked is an open CORS policy.
|
||||||
|
func (a *App) cors(next http.Handler) http.Handler {
|
||||||
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
h := w.Header()
|
||||||
|
// The response body depends on Origin whenever an allowlist is
|
||||||
|
// configured, so Vary is set even on a non-match — otherwise a shared
|
||||||
|
// cache can serve an allowed origin's response to a denied one.
|
||||||
|
if len(a.cfg.AllowOrigins) > 0 {
|
||||||
|
h.Add("Vary", "Origin")
|
||||||
|
}
|
||||||
|
if origin := r.Header.Get("Origin"); origin != "" && a.originAllowed(origin) {
|
||||||
|
h.Set("Access-Control-Allow-Origin", origin)
|
||||||
|
h.Set("Access-Control-Allow-Methods", "GET, POST, PUT, PATCH, DELETE, OPTIONS")
|
||||||
|
h.Set("Access-Control-Allow-Headers", "Content-Type, Authorization, X-Request-Id, Idempotency-Key")
|
||||||
|
// Without this the browser can send X-Request-Id but cannot READ the
|
||||||
|
// one we echo back, so a cross-origin SPA has no id to show the user
|
||||||
|
// next to an error toast. Allow-Headers governs the request; only
|
||||||
|
// Expose-Headers governs what JS may read off the response.
|
||||||
|
h.Set("Access-Control-Expose-Headers", "X-Request-Id, X-Trace-Id")
|
||||||
|
h.Set("Access-Control-Max-Age", "600")
|
||||||
|
}
|
||||||
|
if r.Method == http.MethodOptions {
|
||||||
|
w.WriteHeader(http.StatusNoContent)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
next.ServeHTTP(w, r)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// originAllowed reports whether origin is on the configured allowlist. Matching
|
||||||
|
// is exact: no suffix or wildcard matching, because "endswith example.com"
|
||||||
|
// also matches "evil-example.com".
|
||||||
|
func (a *App) originAllowed(origin string) bool {
|
||||||
|
for _, allowed := range a.cfg.AllowOrigins {
|
||||||
|
if allowed == origin {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
44
vendor/github.com/orchard9/go-chassis/chassis/pagination.go
generated
vendored
Normal file
44
vendor/github.com/orchard9/go-chassis/chassis/pagination.go
generated
vendored
Normal file
@ -0,0 +1,44 @@
|
|||||||
|
package chassis
|
||||||
|
|
||||||
|
import "strconv"
|
||||||
|
|
||||||
|
// Page is a parsed cursor-pagination request (?limit=&cursor=). Cursor is opaque
|
||||||
|
// to the chassis; handlers encode/decode it (e.g. an id or keyset token).
|
||||||
|
type Page struct {
|
||||||
|
Limit int
|
||||||
|
Cursor string
|
||||||
|
}
|
||||||
|
|
||||||
|
// Page parses pagination params from the query string, clamping limit to
|
||||||
|
// [1, maxLimit] and defaulting to defLimit.
|
||||||
|
func (c *Context) Page(defLimit, maxLimit int) Page {
|
||||||
|
q := c.r.URL.Query()
|
||||||
|
p := Page{Limit: defLimit, Cursor: q.Get("cursor")}
|
||||||
|
if v := q.Get("limit"); v != "" {
|
||||||
|
if n, err := strconv.Atoi(v); err == nil && n > 0 {
|
||||||
|
p.Limit = n
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if p.Limit > maxLimit {
|
||||||
|
p.Limit = maxLimit
|
||||||
|
}
|
||||||
|
if p.Limit < 1 {
|
||||||
|
p.Limit = 1
|
||||||
|
}
|
||||||
|
return p
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListResponse is the standard list envelope. NextCursor is "" on the last page.
|
||||||
|
type ListResponse[T any] struct {
|
||||||
|
Items []T `json:"items"`
|
||||||
|
NextCursor string `json:"next_cursor,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// List builds a ListResponse, normalizing a nil slice to []. nextCursor is the
|
||||||
|
// token a client passes as ?cursor= to fetch the next page ("" = no more).
|
||||||
|
func List[T any](items []T, nextCursor string) ListResponse[T] {
|
||||||
|
if items == nil {
|
||||||
|
items = []T{}
|
||||||
|
}
|
||||||
|
return ListResponse[T]{Items: items, NextCursor: nextCursor}
|
||||||
|
}
|
||||||
89
vendor/github.com/orchard9/go-chassis/chassis/ratelimit.go
generated
vendored
Normal file
89
vendor/github.com/orchard9/go-chassis/chassis/ratelimit.go
generated
vendored
Normal file
@ -0,0 +1,89 @@
|
|||||||
|
package chassis
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"math"
|
||||||
|
"net"
|
||||||
|
"net/http"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// RateLimiter decides whether a key may proceed now. Back it with a Redis token
|
||||||
|
// bucket in shared/adapters/redis (tiered by actor). retryAfter hints when to
|
||||||
|
// retry; it is surfaced as the Retry-After header on a 429.
|
||||||
|
type RateLimiter interface {
|
||||||
|
Allow(ctx context.Context, key string) (allowed bool, retryAfter time.Duration, err error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// RateLimit rejects requests over the limit with 429 + Retry-After. keyFn maps a
|
||||||
|
// request to a bucket key (default: authenticated subject, else client IP). The
|
||||||
|
// limiter is best-effort: a backend error fails open (request proceeds) so a
|
||||||
|
// degraded Redis can't take down the API.
|
||||||
|
func RateLimit(limiter RateLimiter, keyFn func(*Context) string) Middleware {
|
||||||
|
if keyFn == nil {
|
||||||
|
keyFn = DefaultRateKey
|
||||||
|
}
|
||||||
|
return func(next HandlerFunc) HandlerFunc {
|
||||||
|
return func(c *Context) error {
|
||||||
|
allowed, retryAfter, err := limiter.Allow(c.r.Context(), keyFn(c))
|
||||||
|
if err != nil {
|
||||||
|
c.Log().Warn("ratelimit.failed", "category", "ratelimit", "error_msg", err.Error())
|
||||||
|
return next(c)
|
||||||
|
}
|
||||||
|
if !allowed {
|
||||||
|
if retryAfter > 0 {
|
||||||
|
c.w.Header().Set("Retry-After", strconv.Itoa(int(math.Ceil(retryAfter.Seconds()))))
|
||||||
|
}
|
||||||
|
return TooManyRequests("rate limit exceeded")
|
||||||
|
}
|
||||||
|
return next(c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// DefaultRateKey buckets by authenticated subject when present, else by client
|
||||||
|
// IP. It does NOT trust X-Forwarded-For (trustedHops=0), so a spoofed XFF can't
|
||||||
|
// mint a fresh bucket per request. Behind N trusted proxies/LBs, use RateKey(N).
|
||||||
|
func DefaultRateKey(c *Context) string { return rateKey(c, 0) }
|
||||||
|
|
||||||
|
// RateKey returns a key function for a deployment behind trustedHops proxies/LBs
|
||||||
|
// (the IP is taken trustedHops entries from the right of X-Forwarded-For).
|
||||||
|
func RateKey(trustedHops int) func(*Context) string {
|
||||||
|
return func(c *Context) string { return rateKey(c, trustedHops) }
|
||||||
|
}
|
||||||
|
|
||||||
|
func rateKey(c *Context, trustedHops int) string {
|
||||||
|
if id, ok := c.Identity(); ok && id.Subject != "" && id.Subject != "anonymous" {
|
||||||
|
return "sub:" + id.Subject
|
||||||
|
}
|
||||||
|
return "ip:" + clientIP(c.r, trustedHops)
|
||||||
|
}
|
||||||
|
|
||||||
|
// clientIP derives the client address. With trustedHops<=0 it uses the connection
|
||||||
|
// RemoteAddr (XFF is attacker-controlled and ignored). Behind trustedHops trusted
|
||||||
|
// proxies it takes the entry that many hops from the right of X-Forwarded-For —
|
||||||
|
// the first address the trust boundary did not append.
|
||||||
|
func clientIP(r *http.Request, trustedHops int) string {
|
||||||
|
host, _, err := net.SplitHostPort(r.RemoteAddr)
|
||||||
|
if err != nil {
|
||||||
|
host = r.RemoteAddr
|
||||||
|
}
|
||||||
|
if trustedHops <= 0 {
|
||||||
|
return host
|
||||||
|
}
|
||||||
|
xff := r.Header.Get("X-Forwarded-For")
|
||||||
|
if xff == "" {
|
||||||
|
return host
|
||||||
|
}
|
||||||
|
parts := strings.Split(xff, ",")
|
||||||
|
idx := len(parts) - trustedHops
|
||||||
|
if idx < 0 || idx >= len(parts) {
|
||||||
|
return host
|
||||||
|
}
|
||||||
|
if v := strings.TrimSpace(parts[idx]); v != "" {
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
return host
|
||||||
|
}
|
||||||
154
vendor/github.com/orchard9/go-chassis/chassis/stream.go
generated
vendored
Normal file
154
vendor/github.com/orchard9/go-chassis/chassis/stream.go
generated
vendored
Normal file
@ -0,0 +1,154 @@
|
|||||||
|
package chassis
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// untimedKey carries the pre-deadline request context so Stream can shed the
|
||||||
|
// per-request budget. See instrument.
|
||||||
|
type untimedKey struct{}
|
||||||
|
|
||||||
|
// heartbeatEvery bounds how long a stream stays silent. Any proxy between the
|
||||||
|
// board and a browser will drop an idle connection eventually — Traefik's
|
||||||
|
// default is 0 (never) but Cloudflare's is 100s and a corporate egress proxy's
|
||||||
|
// is anyone's guess — so a comment frame goes out on every tick even when the
|
||||||
|
// board state has not moved. It is two bytes and it is the difference between
|
||||||
|
// a live dashboard and one that silently stopped updating an hour ago.
|
||||||
|
const heartbeatEvery = 20 * time.Second
|
||||||
|
|
||||||
|
// Stream is an open server-sent-events response.
|
||||||
|
//
|
||||||
|
// SSE rather than a websocket because the traffic is strictly one-way (the
|
||||||
|
// board pushes state, the browser never talks back), it survives every proxy
|
||||||
|
// that speaks HTTP/1.1, and EventSource reconnects on its own — so a board
|
||||||
|
// restart costs the dashboard a few seconds rather than a page reload.
|
||||||
|
type Stream struct {
|
||||||
|
w http.ResponseWriter
|
||||||
|
rc *http.ResponseController
|
||||||
|
ctx context.Context
|
||||||
|
closing <-chan struct{}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Stream converts the response into an event stream and returns a handle.
|
||||||
|
//
|
||||||
|
// It clears this connection's write deadline (the server sets one from
|
||||||
|
// RequestTimeout for every other route) and detaches from the per-request
|
||||||
|
// context deadline, leaving the stream bound to exactly two things: the client
|
||||||
|
// hanging up, and the server beginning to drain.
|
||||||
|
//
|
||||||
|
// The handler MUST NOT write to the Context afterwards — the response is
|
||||||
|
// committed the moment this returns.
|
||||||
|
func (c *Context) Stream() (*Stream, error) {
|
||||||
|
w := c.w
|
||||||
|
rc := http.NewResponseController(w)
|
||||||
|
// A stream lives past any per-request deadline by definition. This is the
|
||||||
|
// call that needs statusRecorder.Unwrap; without it the connection is cut
|
||||||
|
// mid-stream at RequestTimeout+socketHeadroom.
|
||||||
|
if err := rc.SetWriteDeadline(time.Time{}); err != nil {
|
||||||
|
return nil, fmt.Errorf("chassis: stream needs a deadline-capable writer: %w", err)
|
||||||
|
}
|
||||||
|
// Tell the edge not to time this response. How long a stream stays open is
|
||||||
|
// how long an operator left a tab open; in the latency histogram it fires
|
||||||
|
// HighRequestLatency and drags every percentile for the whole service.
|
||||||
|
if rec, ok := w.(*statusRecorder); ok {
|
||||||
|
rec.streamed = true
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx := c.r.Context()
|
||||||
|
if untimed, ok := ctx.Value(untimedKey{}).(context.Context); ok {
|
||||||
|
ctx = untimed
|
||||||
|
}
|
||||||
|
|
||||||
|
h := w.Header()
|
||||||
|
h.Set("Content-Type", "text/event-stream")
|
||||||
|
h.Set("Cache-Control", "no-cache, no-transform")
|
||||||
|
h.Set("Connection", "keep-alive")
|
||||||
|
// Traefik does not buffer, but this response passes through whatever the
|
||||||
|
// operator puts in front of it and an accumulating proxy turns a live
|
||||||
|
// stream into a batch delivered at close.
|
||||||
|
h.Set("X-Accel-Buffering", "no")
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
// Past this point the response is committed, so a flush failure must not
|
||||||
|
// become a returned error — the chassis would write a JSON envelope on top
|
||||||
|
// of a 200 event stream. The first Send surfaces a dead connection.
|
||||||
|
_ = rc.Flush()
|
||||||
|
|
||||||
|
return &Stream{w: w, rc: rc, ctx: ctx, closing: c.closing}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Context is the stream's lifetime: cancelled when the client disconnects.
|
||||||
|
func (s *Stream) Context() context.Context { return s.ctx }
|
||||||
|
|
||||||
|
// Send JSON-encodes v as one named event and flushes it. A write error means
|
||||||
|
// the client is gone; the caller returns and the handler ends.
|
||||||
|
func (s *Stream) Send(event string, v any) error {
|
||||||
|
body, err := json.Marshal(v)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("chassis: stream encode %s: %w", event, err)
|
||||||
|
}
|
||||||
|
// The payload is compact JSON from encoding/json, so it contains no raw
|
||||||
|
// newline and needs no multi-line data: continuation.
|
||||||
|
if _, err := fmt.Fprintf(s.w, "event: %s\ndata: %s\n\n", event, body); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return s.rc.Flush()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Run pushes an initial frame, then one per tick, until the client disconnects
|
||||||
|
// or the server drains. It returns nil on every ordinary end — a browser
|
||||||
|
// closing a tab is not a server error and must not be logged as one.
|
||||||
|
func (s *Stream) Run(every time.Duration, frame func(context.Context) (any, error)) error {
|
||||||
|
send := func() error {
|
||||||
|
v, err := frame(s.ctx)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return s.Send("state", v)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := send(); err != nil {
|
||||||
|
return s.classify(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
tick := time.NewTicker(every)
|
||||||
|
defer tick.Stop()
|
||||||
|
beat := time.NewTicker(heartbeatEvery)
|
||||||
|
defer beat.Stop()
|
||||||
|
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-s.ctx.Done():
|
||||||
|
return nil
|
||||||
|
case <-s.closing:
|
||||||
|
// Tell the browser to come back rather than letting it infer a
|
||||||
|
// dead board from a closed socket.
|
||||||
|
_ = s.Send("bye", map[string]string{"reason": "draining"})
|
||||||
|
return nil
|
||||||
|
case <-beat.C:
|
||||||
|
if _, err := fmt.Fprint(s.w, ": ping\n\n"); err != nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err := s.rc.Flush(); err != nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
case <-tick.C:
|
||||||
|
if err := send(); err != nil {
|
||||||
|
return s.classify(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// classify swallows the errors that mean "the client left". A disconnect races
|
||||||
|
// every write, so treating it as a failure would fill the log with 500s every
|
||||||
|
// time somebody closes a dashboard tab.
|
||||||
|
func (s *Stream) classify(err error) error {
|
||||||
|
if s.ctx.Err() != nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
147
vendor/github.com/orchard9/go-chassis/config/config.go
generated
vendored
Normal file
147
vendor/github.com/orchard9/go-chassis/config/config.go
generated
vendored
Normal file
@ -0,0 +1,147 @@
|
|||||||
|
// Package config is the typed, validated environment loader used at the
|
||||||
|
// composition root (services/*/main.go, workers/*/main.go) — the ONLY place env
|
||||||
|
// is read. A Loader accumulates parse/validation errors so boot fails closed
|
||||||
|
// with one actionable message instead of silently running on bad config.
|
||||||
|
//
|
||||||
|
// Secrets (DB passwords, API keys) do NOT belong here — they come from a
|
||||||
|
// secret source (manager / file). This loads non-secret wiring only.
|
||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Loader reads env vars with defaults and validation, collecting every error.
|
||||||
|
type Loader struct {
|
||||||
|
errs []error
|
||||||
|
}
|
||||||
|
|
||||||
|
// New returns an empty Loader.
|
||||||
|
func New() *Loader { return &Loader{} }
|
||||||
|
|
||||||
|
// String returns the env value or def when unset/empty.
|
||||||
|
func (l *Loader) String(key, def string) string {
|
||||||
|
if v := strings.TrimSpace(os.Getenv(key)); v != "" {
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
return def
|
||||||
|
}
|
||||||
|
|
||||||
|
// Strings splits a comma-separated env value, trimming each entry and dropping
|
||||||
|
// empties, or returns def when unset. Used for allowlists (CORS origins,
|
||||||
|
// trusted proxies) where a platform has more than one legitimate value and a
|
||||||
|
// single-string field would silently serve only the first.
|
||||||
|
func (l *Loader) Strings(key string, def []string) []string {
|
||||||
|
raw := strings.TrimSpace(os.Getenv(key))
|
||||||
|
if raw == "" {
|
||||||
|
return def
|
||||||
|
}
|
||||||
|
parts := strings.Split(raw, ",")
|
||||||
|
out := make([]string, 0, len(parts))
|
||||||
|
for _, p := range parts {
|
||||||
|
if v := strings.TrimSpace(p); v != "" {
|
||||||
|
out = append(out, v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(out) == 0 {
|
||||||
|
return def
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// Required returns the env value or records an error when unset/empty.
|
||||||
|
func (l *Loader) Required(key string) string {
|
||||||
|
v := strings.TrimSpace(os.Getenv(key))
|
||||||
|
if v == "" {
|
||||||
|
l.errs = append(l.errs, fmt.Errorf("%s is required", key))
|
||||||
|
}
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
|
||||||
|
// Int parses an integer env value, recording an error on a malformed value.
|
||||||
|
func (l *Loader) Int(key string, def int) int {
|
||||||
|
raw := strings.TrimSpace(os.Getenv(key))
|
||||||
|
if raw == "" {
|
||||||
|
return def
|
||||||
|
}
|
||||||
|
n, err := strconv.Atoi(raw)
|
||||||
|
if err != nil {
|
||||||
|
l.errs = append(l.errs, fmt.Errorf("%s: invalid int %q", key, raw))
|
||||||
|
return def
|
||||||
|
}
|
||||||
|
return n
|
||||||
|
}
|
||||||
|
|
||||||
|
// Int64 parses a 64-bit integer env value, recording an error on a malformed
|
||||||
|
// value. Separate from Int because byte-size limits (body caps, artifact
|
||||||
|
// ceilings) legitimately exceed a 32-bit int on some platforms.
|
||||||
|
func (l *Loader) Int64(key string, def int64) int64 {
|
||||||
|
raw := strings.TrimSpace(os.Getenv(key))
|
||||||
|
if raw == "" {
|
||||||
|
return def
|
||||||
|
}
|
||||||
|
n, err := strconv.ParseInt(raw, 10, 64)
|
||||||
|
if err != nil {
|
||||||
|
l.errs = append(l.errs, fmt.Errorf("%s: invalid int64 %q", key, raw))
|
||||||
|
return def
|
||||||
|
}
|
||||||
|
return n
|
||||||
|
}
|
||||||
|
|
||||||
|
// Port parses and bounds-checks a TCP port (1..65535).
|
||||||
|
func (l *Loader) Port(key string, def int) int {
|
||||||
|
n := l.Int(key, def)
|
||||||
|
if n < 1 || n > 65535 {
|
||||||
|
l.errs = append(l.errs, fmt.Errorf("%s: port %d out of range 1..65535", key, n))
|
||||||
|
return def
|
||||||
|
}
|
||||||
|
return n
|
||||||
|
}
|
||||||
|
|
||||||
|
// Duration parses a Go duration (e.g. 5s, 1h), recording an error if malformed.
|
||||||
|
func (l *Loader) Duration(key string, def time.Duration) time.Duration {
|
||||||
|
raw := strings.TrimSpace(os.Getenv(key))
|
||||||
|
if raw == "" {
|
||||||
|
return def
|
||||||
|
}
|
||||||
|
d, err := time.ParseDuration(raw)
|
||||||
|
if err != nil {
|
||||||
|
l.errs = append(l.errs, fmt.Errorf("%s: invalid duration %q", key, raw))
|
||||||
|
return def
|
||||||
|
}
|
||||||
|
return d
|
||||||
|
}
|
||||||
|
|
||||||
|
// Bool parses a boolean (1/t/true/0/f/false), recording an error if malformed.
|
||||||
|
func (l *Loader) Bool(key string, def bool) bool {
|
||||||
|
raw := strings.TrimSpace(os.Getenv(key))
|
||||||
|
if raw == "" {
|
||||||
|
return def
|
||||||
|
}
|
||||||
|
b, err := strconv.ParseBool(raw)
|
||||||
|
if err != nil {
|
||||||
|
l.errs = append(l.errs, fmt.Errorf("%s: invalid bool %q", key, raw))
|
||||||
|
return def
|
||||||
|
}
|
||||||
|
return b
|
||||||
|
}
|
||||||
|
|
||||||
|
// OneOf returns the env value when it is in allowed, else records an error.
|
||||||
|
func (l *Loader) OneOf(key, def string, allowed ...string) string {
|
||||||
|
v := l.String(key, def)
|
||||||
|
for _, a := range allowed {
|
||||||
|
if v == a {
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
}
|
||||||
|
l.errs = append(l.errs, fmt.Errorf("%s: %q not one of %s", key, v, strings.Join(allowed, "|")))
|
||||||
|
return def
|
||||||
|
}
|
||||||
|
|
||||||
|
// Err returns the joined validation errors, or nil when the config is clean.
|
||||||
|
func (l *Loader) Err() error { return errors.Join(l.errs...) }
|
||||||
25
vendor/github.com/orchard9/go-chassis/logging/context.go
generated
vendored
Normal file
25
vendor/github.com/orchard9/go-chassis/logging/context.go
generated
vendored
Normal file
@ -0,0 +1,25 @@
|
|||||||
|
package logging
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"log/slog"
|
||||||
|
)
|
||||||
|
|
||||||
|
type ctxKey int
|
||||||
|
|
||||||
|
const loggerKey ctxKey = iota
|
||||||
|
|
||||||
|
// Into returns a child context carrying log, so request-scoped fields
|
||||||
|
// (request_id, route, ...) added at the edge flow to every downstream call.
|
||||||
|
func Into(ctx context.Context, log *slog.Logger) context.Context {
|
||||||
|
return context.WithValue(ctx, loggerKey, log)
|
||||||
|
}
|
||||||
|
|
||||||
|
// From returns the request-scoped logger stored in ctx, or the package fallback
|
||||||
|
// (set by SetFallback at boot) so a missing logger never panics or drops logs.
|
||||||
|
func From(ctx context.Context) *slog.Logger {
|
||||||
|
if log, ok := ctx.Value(loggerKey).(*slog.Logger); ok && log != nil {
|
||||||
|
return log
|
||||||
|
}
|
||||||
|
return fallback()
|
||||||
|
}
|
||||||
139
vendor/github.com/orchard9/go-chassis/logging/logging.go
generated
vendored
Normal file
139
vendor/github.com/orchard9/go-chassis/logging/logging.go
generated
vendored
Normal file
@ -0,0 +1,139 @@
|
|||||||
|
// Package logging is the core structured logger: one JSON object per line on
|
||||||
|
// stdout, ready for any log agent (Fluent Bit, vector, the cloud's native
|
||||||
|
// collector) to ship. Fields: ts (ISO-8601 UTC, ms, Z), level, service + env
|
||||||
|
// (the only indexed stream fields — keep them closed enums), msg, plus caller
|
||||||
|
// attrs. Secrets/PII MUST NOT be logged; forbiddenKeys redacts common offenders
|
||||||
|
// as defense-in-depth behind code review + lint. See patterns/go-chassis.md.
|
||||||
|
package logging
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"io"
|
||||||
|
"log/slog"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
)
|
||||||
|
|
||||||
|
// LevelCritical sits above slog.LevelError; it maps to the "critical" enum and
|
||||||
|
// is reserved for fail-closed boot/serve refusal (wire it to a page).
|
||||||
|
const LevelCritical = slog.Level(12)
|
||||||
|
|
||||||
|
// Config selects the closed-enum stream fields the log store indexes.
|
||||||
|
type Config struct {
|
||||||
|
Service string // closed enum, keep small (e.g. api, job-worker)
|
||||||
|
Env string // dev | staging | prod
|
||||||
|
}
|
||||||
|
|
||||||
|
// New returns the core logger writing the JSON wire format to stdout.
|
||||||
|
func New(cfg Config) *slog.Logger { return newTo(os.Stdout, cfg) }
|
||||||
|
|
||||||
|
// NewTo returns the core logger writing the JSON wire format to w. Server mode
|
||||||
|
// uses New (stdout); a bounded maintenance command uses this to send its
|
||||||
|
// operational diagnostics to stderr, keeping stdout a clean machine-readable
|
||||||
|
// result stream a caller can parse without stripping log lines out of it.
|
||||||
|
func NewTo(w io.Writer, cfg Config) *slog.Logger { return newTo(w, cfg) }
|
||||||
|
|
||||||
|
func newTo(w io.Writer, cfg Config) *slog.Logger {
|
||||||
|
h := slog.NewJSONHandler(w, &slog.HandlerOptions{
|
||||||
|
Level: slog.LevelDebug,
|
||||||
|
ReplaceAttr: replace,
|
||||||
|
})
|
||||||
|
return slog.New(h).With("service", cfg.Service, "env", cfg.Env)
|
||||||
|
}
|
||||||
|
|
||||||
|
func replace(groups []string, a slog.Attr) slog.Attr {
|
||||||
|
if forbiddenKeys[a.Key] { // defense-in-depth secret/PII redaction
|
||||||
|
return slog.String(a.Key, "[REDACTED]")
|
||||||
|
}
|
||||||
|
// The built-in rewrites below apply only to the record's own time/level
|
||||||
|
// attrs, which slog always passes at the top level. A caller attr that
|
||||||
|
// happens to be named "time" or "level" — log.Info("m", "level", "high") —
|
||||||
|
// arrives here too, so both the group depth and the value kind are checked
|
||||||
|
// before converting. An unchecked assertion here panics the process inside
|
||||||
|
// the logger every binary depends on.
|
||||||
|
if len(groups) > 0 {
|
||||||
|
return a
|
||||||
|
}
|
||||||
|
switch a.Key {
|
||||||
|
case slog.TimeKey:
|
||||||
|
if a.Value.Kind() != slog.KindTime {
|
||||||
|
return a
|
||||||
|
}
|
||||||
|
// ts: ISO-8601 UTC, ms precision, Z suffix.
|
||||||
|
a.Key = "ts"
|
||||||
|
a.Value = slog.StringValue(a.Value.Time().UTC().Format("2006-01-02T15:04:05.000Z07:00"))
|
||||||
|
case slog.LevelKey:
|
||||||
|
lvl, ok := a.Value.Any().(slog.Level)
|
||||||
|
if !ok {
|
||||||
|
return a
|
||||||
|
}
|
||||||
|
a.Key = "level"
|
||||||
|
a.Value = slog.StringValue(levelString(lvl))
|
||||||
|
}
|
||||||
|
return a
|
||||||
|
}
|
||||||
|
|
||||||
|
func levelString(l slog.Level) string {
|
||||||
|
switch {
|
||||||
|
case l >= LevelCritical:
|
||||||
|
return "critical"
|
||||||
|
case l >= slog.LevelError:
|
||||||
|
return "error"
|
||||||
|
case l >= slog.LevelWarn:
|
||||||
|
return "warn"
|
||||||
|
case l >= slog.LevelInfo:
|
||||||
|
return "info"
|
||||||
|
default:
|
||||||
|
return "debug"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// forbiddenKeys are field names that MUST NOT reach the log corpus. Extend it
|
||||||
|
// with YOUR product's sensitive fields (PII, PII, financial, tokens).
|
||||||
|
var forbiddenKeys = map[string]bool{
|
||||||
|
"password": true, "passwd": true, "secret": true, "token": true,
|
||||||
|
"api_key": true, "apikey": true, "authorization": true, "cookie": true,
|
||||||
|
"ssn": true, "email": true, "phone": true,
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
fallbackMu sync.RWMutex
|
||||||
|
fallbackLog *slog.Logger
|
||||||
|
)
|
||||||
|
|
||||||
|
// SetFallback installs the logger From returns when no request-scoped logger is
|
||||||
|
// in context. Set once at boot from the composition root.
|
||||||
|
func SetFallback(l *slog.Logger) {
|
||||||
|
fallbackMu.Lock()
|
||||||
|
defer fallbackMu.Unlock()
|
||||||
|
fallbackLog = l
|
||||||
|
}
|
||||||
|
|
||||||
|
func fallback() *slog.Logger {
|
||||||
|
fallbackMu.RLock()
|
||||||
|
l := fallbackLog
|
||||||
|
fallbackMu.RUnlock()
|
||||||
|
if l != nil {
|
||||||
|
return l
|
||||||
|
}
|
||||||
|
return New(Config{Service: "unknown", Env: "dev"})
|
||||||
|
}
|
||||||
|
|
||||||
|
// Critical logs at the fail-closed level (boot/serve refusal). Bind to a page.
|
||||||
|
func Critical(ctx context.Context, l *slog.Logger, msg string, args ...any) {
|
||||||
|
l.Log(ctx, LevelCritical, msg, args...)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Env normalizes an APP_ENV value (local|dev|staging|prod, or a gcp-* / aws-*
|
||||||
|
// prefix) to the log env enum.
|
||||||
|
func Env(appEnv string) string {
|
||||||
|
switch {
|
||||||
|
case strings.HasSuffix(appEnv, "prod"):
|
||||||
|
return "prod"
|
||||||
|
case strings.HasSuffix(appEnv, "staging"):
|
||||||
|
return "staging"
|
||||||
|
default:
|
||||||
|
return "dev"
|
||||||
|
}
|
||||||
|
}
|
||||||
53
vendor/github.com/orchard9/go-chassis/logging/redact.go
generated
vendored
Normal file
53
vendor/github.com/orchard9/go-chassis/logging/redact.go
generated
vendored
Normal file
@ -0,0 +1,53 @@
|
|||||||
|
package logging
|
||||||
|
|
||||||
|
import "encoding/json"
|
||||||
|
|
||||||
|
// Forbidden reports whether key is a secrets/PII field name that must never be
|
||||||
|
// persisted in cleartext (logs, audit snapshots, alert payloads).
|
||||||
|
func Forbidden(key string) bool { return forbiddenKeys[key] }
|
||||||
|
|
||||||
|
// Redact returns "[REDACTED]" when key is forbidden, otherwise val. Use it when
|
||||||
|
// projecting attributes into a non-log sink (e.g. alert events).
|
||||||
|
func Redact(key, val string) string {
|
||||||
|
if forbiddenKeys[key] {
|
||||||
|
return "[REDACTED]"
|
||||||
|
}
|
||||||
|
return val
|
||||||
|
}
|
||||||
|
|
||||||
|
// RedactJSON returns a copy of a JSON document with every forbidden key's value
|
||||||
|
// (at any depth) replaced by "[REDACTED]". This is the defense-in-depth scrubber
|
||||||
|
// the audit sink applies to before/after snapshots so PII cannot land in the
|
||||||
|
// immutable audit log. Non-JSON or unparseable input is returned unchanged.
|
||||||
|
func RedactJSON(b []byte) []byte {
|
||||||
|
if len(b) == 0 {
|
||||||
|
return b
|
||||||
|
}
|
||||||
|
var v any
|
||||||
|
if err := json.Unmarshal(b, &v); err != nil {
|
||||||
|
return b
|
||||||
|
}
|
||||||
|
redactValue(v)
|
||||||
|
out, err := json.Marshal(v)
|
||||||
|
if err != nil {
|
||||||
|
return b
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func redactValue(v any) {
|
||||||
|
switch t := v.(type) {
|
||||||
|
case map[string]any:
|
||||||
|
for k, val := range t {
|
||||||
|
if forbiddenKeys[k] {
|
||||||
|
t[k] = "[REDACTED]"
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
redactValue(val)
|
||||||
|
}
|
||||||
|
case []any:
|
||||||
|
for _, e := range t {
|
||||||
|
redactValue(e)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
201
vendor/github.com/prometheus/client_golang/LICENSE
generated
vendored
Normal file
201
vendor/github.com/prometheus/client_golang/LICENSE
generated
vendored
Normal file
@ -0,0 +1,201 @@
|
|||||||
|
Apache License
|
||||||
|
Version 2.0, January 2004
|
||||||
|
http://www.apache.org/licenses/
|
||||||
|
|
||||||
|
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||||
|
|
||||||
|
1. Definitions.
|
||||||
|
|
||||||
|
"License" shall mean the terms and conditions for use, reproduction,
|
||||||
|
and distribution as defined by Sections 1 through 9 of this document.
|
||||||
|
|
||||||
|
"Licensor" shall mean the copyright owner or entity authorized by
|
||||||
|
the copyright owner that is granting the License.
|
||||||
|
|
||||||
|
"Legal Entity" shall mean the union of the acting entity and all
|
||||||
|
other entities that control, are controlled by, or are under common
|
||||||
|
control with that entity. For the purposes of this definition,
|
||||||
|
"control" means (i) the power, direct or indirect, to cause the
|
||||||
|
direction or management of such entity, whether by contract or
|
||||||
|
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||||
|
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||||
|
|
||||||
|
"You" (or "Your") shall mean an individual or Legal Entity
|
||||||
|
exercising permissions granted by this License.
|
||||||
|
|
||||||
|
"Source" form shall mean the preferred form for making modifications,
|
||||||
|
including but not limited to software source code, documentation
|
||||||
|
source, and configuration files.
|
||||||
|
|
||||||
|
"Object" form shall mean any form resulting from mechanical
|
||||||
|
transformation or translation of a Source form, including but
|
||||||
|
not limited to compiled object code, generated documentation,
|
||||||
|
and conversions to other media types.
|
||||||
|
|
||||||
|
"Work" shall mean the work of authorship, whether in Source or
|
||||||
|
Object form, made available under the License, as indicated by a
|
||||||
|
copyright notice that is included in or attached to the work
|
||||||
|
(an example is provided in the Appendix below).
|
||||||
|
|
||||||
|
"Derivative Works" shall mean any work, whether in Source or Object
|
||||||
|
form, that is based on (or derived from) the Work and for which the
|
||||||
|
editorial revisions, annotations, elaborations, or other modifications
|
||||||
|
represent, as a whole, an original work of authorship. For the purposes
|
||||||
|
of this License, Derivative Works shall not include works that remain
|
||||||
|
separable from, or merely link (or bind by name) to the interfaces of,
|
||||||
|
the Work and Derivative Works thereof.
|
||||||
|
|
||||||
|
"Contribution" shall mean any work of authorship, including
|
||||||
|
the original version of the Work and any modifications or additions
|
||||||
|
to that Work or Derivative Works thereof, that is intentionally
|
||||||
|
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||||
|
or by an individual or Legal Entity authorized to submit on behalf of
|
||||||
|
the copyright owner. For the purposes of this definition, "submitted"
|
||||||
|
means any form of electronic, verbal, or written communication sent
|
||||||
|
to the Licensor or its representatives, including but not limited to
|
||||||
|
communication on electronic mailing lists, source code control systems,
|
||||||
|
and issue tracking systems that are managed by, or on behalf of, the
|
||||||
|
Licensor for the purpose of discussing and improving the Work, but
|
||||||
|
excluding communication that is conspicuously marked or otherwise
|
||||||
|
designated in writing by the copyright owner as "Not a Contribution."
|
||||||
|
|
||||||
|
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||||
|
on behalf of whom a Contribution has been received by Licensor and
|
||||||
|
subsequently incorporated within the Work.
|
||||||
|
|
||||||
|
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||||
|
this License, each Contributor hereby grants to You a perpetual,
|
||||||
|
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||||
|
copyright license to reproduce, prepare Derivative Works of,
|
||||||
|
publicly display, publicly perform, sublicense, and distribute the
|
||||||
|
Work and such Derivative Works in Source or Object form.
|
||||||
|
|
||||||
|
3. Grant of Patent License. Subject to the terms and conditions of
|
||||||
|
this License, each Contributor hereby grants to You a perpetual,
|
||||||
|
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||||
|
(except as stated in this section) patent license to make, have made,
|
||||||
|
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||||
|
where such license applies only to those patent claims licensable
|
||||||
|
by such Contributor that are necessarily infringed by their
|
||||||
|
Contribution(s) alone or by combination of their Contribution(s)
|
||||||
|
with the Work to which such Contribution(s) was submitted. If You
|
||||||
|
institute patent litigation against any entity (including a
|
||||||
|
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||||
|
or a Contribution incorporated within the Work constitutes direct
|
||||||
|
or contributory patent infringement, then any patent licenses
|
||||||
|
granted to You under this License for that Work shall terminate
|
||||||
|
as of the date such litigation is filed.
|
||||||
|
|
||||||
|
4. Redistribution. You may reproduce and distribute copies of the
|
||||||
|
Work or Derivative Works thereof in any medium, with or without
|
||||||
|
modifications, and in Source or Object form, provided that You
|
||||||
|
meet the following conditions:
|
||||||
|
|
||||||
|
(a) You must give any other recipients of the Work or
|
||||||
|
Derivative Works a copy of this License; and
|
||||||
|
|
||||||
|
(b) You must cause any modified files to carry prominent notices
|
||||||
|
stating that You changed the files; and
|
||||||
|
|
||||||
|
(c) You must retain, in the Source form of any Derivative Works
|
||||||
|
that You distribute, all copyright, patent, trademark, and
|
||||||
|
attribution notices from the Source form of the Work,
|
||||||
|
excluding those notices that do not pertain to any part of
|
||||||
|
the Derivative Works; and
|
||||||
|
|
||||||
|
(d) If the Work includes a "NOTICE" text file as part of its
|
||||||
|
distribution, then any Derivative Works that You distribute must
|
||||||
|
include a readable copy of the attribution notices contained
|
||||||
|
within such NOTICE file, excluding those notices that do not
|
||||||
|
pertain to any part of the Derivative Works, in at least one
|
||||||
|
of the following places: within a NOTICE text file distributed
|
||||||
|
as part of the Derivative Works; within the Source form or
|
||||||
|
documentation, if provided along with the Derivative Works; or,
|
||||||
|
within a display generated by the Derivative Works, if and
|
||||||
|
wherever such third-party notices normally appear. The contents
|
||||||
|
of the NOTICE file are for informational purposes only and
|
||||||
|
do not modify the License. You may add Your own attribution
|
||||||
|
notices within Derivative Works that You distribute, alongside
|
||||||
|
or as an addendum to the NOTICE text from the Work, provided
|
||||||
|
that such additional attribution notices cannot be construed
|
||||||
|
as modifying the License.
|
||||||
|
|
||||||
|
You may add Your own copyright statement to Your modifications and
|
||||||
|
may provide additional or different license terms and conditions
|
||||||
|
for use, reproduction, or distribution of Your modifications, or
|
||||||
|
for any such Derivative Works as a whole, provided Your use,
|
||||||
|
reproduction, and distribution of the Work otherwise complies with
|
||||||
|
the conditions stated in this License.
|
||||||
|
|
||||||
|
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||||
|
any Contribution intentionally submitted for inclusion in the Work
|
||||||
|
by You to the Licensor shall be under the terms and conditions of
|
||||||
|
this License, without any additional terms or conditions.
|
||||||
|
Notwithstanding the above, nothing herein shall supersede or modify
|
||||||
|
the terms of any separate license agreement you may have executed
|
||||||
|
with Licensor regarding such Contributions.
|
||||||
|
|
||||||
|
6. Trademarks. This License does not grant permission to use the trade
|
||||||
|
names, trademarks, service marks, or product names of the Licensor,
|
||||||
|
except as required for reasonable and customary use in describing the
|
||||||
|
origin of the Work and reproducing the content of the NOTICE file.
|
||||||
|
|
||||||
|
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||||
|
agreed to in writing, Licensor provides the Work (and each
|
||||||
|
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||||
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||||
|
implied, including, without limitation, any warranties or conditions
|
||||||
|
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||||
|
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||||
|
appropriateness of using or redistributing the Work and assume any
|
||||||
|
risks associated with Your exercise of permissions under this License.
|
||||||
|
|
||||||
|
8. Limitation of Liability. In no event and under no legal theory,
|
||||||
|
whether in tort (including negligence), contract, or otherwise,
|
||||||
|
unless required by applicable law (such as deliberate and grossly
|
||||||
|
negligent acts) or agreed to in writing, shall any Contributor be
|
||||||
|
liable to You for damages, including any direct, indirect, special,
|
||||||
|
incidental, or consequential damages of any character arising as a
|
||||||
|
result of this License or out of the use or inability to use the
|
||||||
|
Work (including but not limited to damages for loss of goodwill,
|
||||||
|
work stoppage, computer failure or malfunction, or any and all
|
||||||
|
other commercial damages or losses), even if such Contributor
|
||||||
|
has been advised of the possibility of such damages.
|
||||||
|
|
||||||
|
9. Accepting Warranty or Additional Liability. While redistributing
|
||||||
|
the Work or Derivative Works thereof, You may choose to offer,
|
||||||
|
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||||
|
or other liability obligations and/or rights consistent with this
|
||||||
|
License. However, in accepting such obligations, You may act only
|
||||||
|
on Your own behalf and on Your sole responsibility, not on behalf
|
||||||
|
of any other Contributor, and only if You agree to indemnify,
|
||||||
|
defend, and hold each Contributor harmless for any liability
|
||||||
|
incurred by, or claims asserted against, such Contributor by reason
|
||||||
|
of your accepting any such warranty or additional liability.
|
||||||
|
|
||||||
|
END OF TERMS AND CONDITIONS
|
||||||
|
|
||||||
|
APPENDIX: How to apply the Apache License to your work.
|
||||||
|
|
||||||
|
To apply the Apache License to your work, attach the following
|
||||||
|
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||||
|
replaced with your own identifying information. (Don't include
|
||||||
|
the brackets!) The text should be enclosed in the appropriate
|
||||||
|
comment syntax for the file format. We also recommend that a
|
||||||
|
file or class name and description of purpose be included on the
|
||||||
|
same "printed page" as the copyright notice for easier
|
||||||
|
identification within third-party archives.
|
||||||
|
|
||||||
|
Copyright [yyyy] [name of copyright owner]
|
||||||
|
|
||||||
|
Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
you may not use this file except in compliance with the License.
|
||||||
|
You may obtain a copy of the License at
|
||||||
|
|
||||||
|
http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
|
||||||
|
Unless required by applicable law or agreed to in writing, software
|
||||||
|
distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
See the License for the specific language governing permissions and
|
||||||
|
limitations under the License.
|
||||||
18
vendor/github.com/prometheus/client_golang/NOTICE
generated
vendored
Normal file
18
vendor/github.com/prometheus/client_golang/NOTICE
generated
vendored
Normal file
@ -0,0 +1,18 @@
|
|||||||
|
Prometheus instrumentation library for Go applications
|
||||||
|
Copyright 2012-2015 The Prometheus Authors
|
||||||
|
|
||||||
|
This product includes software developed at
|
||||||
|
SoundCloud Ltd. (http://soundcloud.com/).
|
||||||
|
|
||||||
|
|
||||||
|
The following components are included in this product:
|
||||||
|
|
||||||
|
perks - a fork of https://github.com/bmizerany/perks
|
||||||
|
https://github.com/beorn7/perks
|
||||||
|
Copyright 2013-2015 Blake Mizerany, Björn Rabenstein
|
||||||
|
See https://github.com/beorn7/perks/blob/master/README.md for license details.
|
||||||
|
|
||||||
|
Go support for Protocol Buffers - Google's data interchange format
|
||||||
|
http://github.com/golang/protobuf/
|
||||||
|
Copyright 2010 The Go Authors
|
||||||
|
See source code for license details.
|
||||||
27
vendor/github.com/prometheus/client_golang/internal/github.com/golang/gddo/LICENSE
generated
vendored
Normal file
27
vendor/github.com/prometheus/client_golang/internal/github.com/golang/gddo/LICENSE
generated
vendored
Normal file
@ -0,0 +1,27 @@
|
|||||||
|
Copyright (c) 2013 The Go Authors. All rights reserved.
|
||||||
|
|
||||||
|
Redistribution and use in source and binary forms, with or without
|
||||||
|
modification, are permitted provided that the following conditions are
|
||||||
|
met:
|
||||||
|
|
||||||
|
* Redistributions of source code must retain the above copyright
|
||||||
|
notice, this list of conditions and the following disclaimer.
|
||||||
|
* Redistributions in binary form must reproduce the above
|
||||||
|
copyright notice, this list of conditions and the following disclaimer
|
||||||
|
in the documentation and/or other materials provided with the
|
||||||
|
distribution.
|
||||||
|
* Neither the name of Google Inc. nor the names of its
|
||||||
|
contributors may be used to endorse or promote products derived from
|
||||||
|
this software without specific prior written permission.
|
||||||
|
|
||||||
|
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
145
vendor/github.com/prometheus/client_golang/internal/github.com/golang/gddo/httputil/header/header.go
generated
vendored
Normal file
145
vendor/github.com/prometheus/client_golang/internal/github.com/golang/gddo/httputil/header/header.go
generated
vendored
Normal file
@ -0,0 +1,145 @@
|
|||||||
|
// Copyright 2013 The Go Authors. All rights reserved.
|
||||||
|
//
|
||||||
|
// Use of this source code is governed by a BSD-style
|
||||||
|
// license that can be found in the LICENSE file or at
|
||||||
|
// https://developers.google.com/open-source/licenses/bsd.
|
||||||
|
|
||||||
|
// Package header provides functions for parsing HTTP headers.
|
||||||
|
package header
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Octet types from RFC 2616.
|
||||||
|
var octetTypes [256]octetType
|
||||||
|
|
||||||
|
type octetType byte
|
||||||
|
|
||||||
|
const (
|
||||||
|
isToken octetType = 1 << iota
|
||||||
|
isSpace
|
||||||
|
)
|
||||||
|
|
||||||
|
func init() {
|
||||||
|
// OCTET = <any 8-bit sequence of data>
|
||||||
|
// CHAR = <any US-ASCII character (octets 0 - 127)>
|
||||||
|
// CTL = <any US-ASCII control character (octets 0 - 31) and DEL (127)>
|
||||||
|
// CR = <US-ASCII CR, carriage return (13)>
|
||||||
|
// LF = <US-ASCII LF, linefeed (10)>
|
||||||
|
// SP = <US-ASCII SP, space (32)>
|
||||||
|
// HT = <US-ASCII HT, horizontal-tab (9)>
|
||||||
|
// <"> = <US-ASCII double-quote mark (34)>
|
||||||
|
// CRLF = CR LF
|
||||||
|
// LWS = [CRLF] 1*( SP | HT )
|
||||||
|
// TEXT = <any OCTET except CTLs, but including LWS>
|
||||||
|
// separators = "(" | ")" | "<" | ">" | "@" | "," | ";" | ":" | "\" | <">
|
||||||
|
// | "/" | "[" | "]" | "?" | "=" | "{" | "}" | SP | HT
|
||||||
|
// token = 1*<any CHAR except CTLs or separators>
|
||||||
|
// qdtext = <any TEXT except <">>
|
||||||
|
|
||||||
|
for c := 0; c < 256; c++ {
|
||||||
|
var t octetType
|
||||||
|
isCtl := c <= 31 || c == 127
|
||||||
|
isChar := 0 <= c && c <= 127
|
||||||
|
isSeparator := strings.ContainsRune(" \t\"(),/:;<=>?@[]\\{}", rune(c))
|
||||||
|
if strings.ContainsRune(" \t\r\n", rune(c)) {
|
||||||
|
t |= isSpace
|
||||||
|
}
|
||||||
|
if isChar && !isCtl && !isSeparator {
|
||||||
|
t |= isToken
|
||||||
|
}
|
||||||
|
octetTypes[c] = t
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// AcceptSpec describes an Accept* header.
|
||||||
|
type AcceptSpec struct {
|
||||||
|
Value string
|
||||||
|
Q float64
|
||||||
|
}
|
||||||
|
|
||||||
|
// ParseAccept parses Accept* headers.
|
||||||
|
func ParseAccept(header http.Header, key string) (specs []AcceptSpec) {
|
||||||
|
loop:
|
||||||
|
for _, s := range header[key] {
|
||||||
|
for {
|
||||||
|
var spec AcceptSpec
|
||||||
|
spec.Value, s = expectTokenSlash(s)
|
||||||
|
if spec.Value == "" {
|
||||||
|
continue loop
|
||||||
|
}
|
||||||
|
spec.Q = 1.0
|
||||||
|
s = skipSpace(s)
|
||||||
|
if strings.HasPrefix(s, ";") {
|
||||||
|
s = skipSpace(s[1:])
|
||||||
|
if !strings.HasPrefix(s, "q=") {
|
||||||
|
continue loop
|
||||||
|
}
|
||||||
|
spec.Q, s = expectQuality(s[2:])
|
||||||
|
if spec.Q < 0.0 {
|
||||||
|
continue loop
|
||||||
|
}
|
||||||
|
}
|
||||||
|
specs = append(specs, spec)
|
||||||
|
s = skipSpace(s)
|
||||||
|
if !strings.HasPrefix(s, ",") {
|
||||||
|
continue loop
|
||||||
|
}
|
||||||
|
s = skipSpace(s[1:])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return specs
|
||||||
|
}
|
||||||
|
|
||||||
|
func skipSpace(s string) (rest string) {
|
||||||
|
i := 0
|
||||||
|
for ; i < len(s); i++ {
|
||||||
|
if octetTypes[s[i]]&isSpace == 0 {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return s[i:]
|
||||||
|
}
|
||||||
|
|
||||||
|
func expectTokenSlash(s string) (token, rest string) {
|
||||||
|
i := 0
|
||||||
|
for ; i < len(s); i++ {
|
||||||
|
b := s[i]
|
||||||
|
if (octetTypes[b]&isToken == 0) && b != '/' {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return s[:i], s[i:]
|
||||||
|
}
|
||||||
|
|
||||||
|
func expectQuality(s string) (q float64, rest string) {
|
||||||
|
switch {
|
||||||
|
case len(s) == 0:
|
||||||
|
return -1, ""
|
||||||
|
case s[0] == '0':
|
||||||
|
q = 0
|
||||||
|
case s[0] == '1':
|
||||||
|
q = 1
|
||||||
|
default:
|
||||||
|
return -1, ""
|
||||||
|
}
|
||||||
|
s = s[1:]
|
||||||
|
if !strings.HasPrefix(s, ".") {
|
||||||
|
return q, s
|
||||||
|
}
|
||||||
|
s = s[1:]
|
||||||
|
i := 0
|
||||||
|
n := 0
|
||||||
|
d := 1
|
||||||
|
for ; i < len(s); i++ {
|
||||||
|
b := s[i]
|
||||||
|
if b < '0' || b > '9' {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
n = n*10 + int(b) - '0'
|
||||||
|
d *= 10
|
||||||
|
}
|
||||||
|
return q + float64(n)/float64(d), s[i:]
|
||||||
|
}
|
||||||
36
vendor/github.com/prometheus/client_golang/internal/github.com/golang/gddo/httputil/negotiate.go
generated
vendored
Normal file
36
vendor/github.com/prometheus/client_golang/internal/github.com/golang/gddo/httputil/negotiate.go
generated
vendored
Normal file
@ -0,0 +1,36 @@
|
|||||||
|
// Copyright 2013 The Go Authors. All rights reserved.
|
||||||
|
//
|
||||||
|
// Use of this source code is governed by a BSD-style
|
||||||
|
// license that can be found in the LICENSE file or at
|
||||||
|
// https://developers.google.com/open-source/licenses/bsd.
|
||||||
|
|
||||||
|
package httputil
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
|
||||||
|
"github.com/prometheus/client_golang/internal/github.com/golang/gddo/httputil/header"
|
||||||
|
)
|
||||||
|
|
||||||
|
// NegotiateContentEncoding returns the best offered content encoding for the
|
||||||
|
// request's Accept-Encoding header. If two offers match with equal weight and
|
||||||
|
// then the offer earlier in the list is preferred. If no offers are
|
||||||
|
// acceptable, then "" is returned.
|
||||||
|
func NegotiateContentEncoding(r *http.Request, offers []string) string {
|
||||||
|
bestOffer := "identity"
|
||||||
|
bestQ := -1.0
|
||||||
|
specs := header.ParseAccept(r.Header, "Accept-Encoding")
|
||||||
|
for _, offer := range offers {
|
||||||
|
for _, spec := range specs {
|
||||||
|
if spec.Q > bestQ &&
|
||||||
|
(spec.Value == "*" || spec.Value == offer) {
|
||||||
|
bestQ = spec.Q
|
||||||
|
bestOffer = offer
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if bestQ == 0 {
|
||||||
|
bestOffer = ""
|
||||||
|
}
|
||||||
|
return bestOffer
|
||||||
|
}
|
||||||
1
vendor/github.com/prometheus/client_golang/prometheus/.gitignore
generated
vendored
Normal file
1
vendor/github.com/prometheus/client_golang/prometheus/.gitignore
generated
vendored
Normal file
@ -0,0 +1 @@
|
|||||||
|
command-line-arguments.test
|
||||||
1
vendor/github.com/prometheus/client_golang/prometheus/README.md
generated
vendored
Normal file
1
vendor/github.com/prometheus/client_golang/prometheus/README.md
generated
vendored
Normal file
@ -0,0 +1 @@
|
|||||||
|
See [](https://pkg.go.dev/github.com/prometheus/client_golang/prometheus).
|
||||||
38
vendor/github.com/prometheus/client_golang/prometheus/build_info_collector.go
generated
vendored
Normal file
38
vendor/github.com/prometheus/client_golang/prometheus/build_info_collector.go
generated
vendored
Normal file
@ -0,0 +1,38 @@
|
|||||||
|
// Copyright 2021 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus
|
||||||
|
|
||||||
|
import "runtime/debug"
|
||||||
|
|
||||||
|
// NewBuildInfoCollector is the obsolete version of collectors.NewBuildInfoCollector.
|
||||||
|
// See there for documentation.
|
||||||
|
//
|
||||||
|
// Deprecated: Use collectors.NewBuildInfoCollector instead.
|
||||||
|
func NewBuildInfoCollector() Collector {
|
||||||
|
path, version, sum := "unknown", "unknown", "unknown"
|
||||||
|
if bi, ok := debug.ReadBuildInfo(); ok {
|
||||||
|
path = bi.Main.Path
|
||||||
|
version = bi.Main.Version
|
||||||
|
sum = bi.Main.Sum
|
||||||
|
}
|
||||||
|
c := &selfCollector{MustNewConstMetric(
|
||||||
|
NewDesc(
|
||||||
|
"go_build_info",
|
||||||
|
"Build information about the main Go module.",
|
||||||
|
nil, Labels{"path": path, "version": version, "checksum": sum},
|
||||||
|
),
|
||||||
|
GaugeValue, 1)}
|
||||||
|
c.init(c.self)
|
||||||
|
return c
|
||||||
|
}
|
||||||
128
vendor/github.com/prometheus/client_golang/prometheus/collector.go
generated
vendored
Normal file
128
vendor/github.com/prometheus/client_golang/prometheus/collector.go
generated
vendored
Normal file
@ -0,0 +1,128 @@
|
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus
|
||||||
|
|
||||||
|
// Collector is the interface implemented by anything that can be used by
|
||||||
|
// Prometheus to collect metrics. A Collector has to be registered for
|
||||||
|
// collection. See Registerer.Register.
|
||||||
|
//
|
||||||
|
// The stock metrics provided by this package (Gauge, Counter, Summary,
|
||||||
|
// Histogram, Untyped) are also Collectors (which only ever collect one metric,
|
||||||
|
// namely itself). An implementer of Collector may, however, collect multiple
|
||||||
|
// metrics in a coordinated fashion and/or create metrics on the fly. Examples
|
||||||
|
// for collectors already implemented in this library are the metric vectors
|
||||||
|
// (i.e. collection of multiple instances of the same Metric but with different
|
||||||
|
// label values) like GaugeVec or SummaryVec, and the ExpvarCollector.
|
||||||
|
type Collector interface {
|
||||||
|
// Describe sends the super-set of all possible descriptors of metrics
|
||||||
|
// collected by this Collector to the provided channel and returns once
|
||||||
|
// the last descriptor has been sent. The sent descriptors fulfill the
|
||||||
|
// consistency and uniqueness requirements described in the Desc
|
||||||
|
// documentation.
|
||||||
|
//
|
||||||
|
// It is valid if one and the same Collector sends duplicate
|
||||||
|
// descriptors. Those duplicates are simply ignored. However, two
|
||||||
|
// different Collectors must not send duplicate descriptors.
|
||||||
|
//
|
||||||
|
// Sending no descriptor at all marks the Collector as “unchecked”,
|
||||||
|
// i.e. no checks will be performed at registration time, and the
|
||||||
|
// Collector may yield any Metric it sees fit in its Collect method.
|
||||||
|
//
|
||||||
|
// This method idempotently sends the same descriptors throughout the
|
||||||
|
// lifetime of the Collector. It may be called concurrently and
|
||||||
|
// therefore must be implemented in a concurrency safe way.
|
||||||
|
//
|
||||||
|
// If a Collector encounters an error while executing this method, it
|
||||||
|
// must send an invalid descriptor (created with NewInvalidDesc) to
|
||||||
|
// signal the error to the registry.
|
||||||
|
Describe(chan<- *Desc)
|
||||||
|
// Collect is called by the Prometheus registry when collecting
|
||||||
|
// metrics. The implementation sends each collected metric via the
|
||||||
|
// provided channel and returns once the last metric has been sent. The
|
||||||
|
// descriptor of each sent metric is one of those returned by Describe
|
||||||
|
// (unless the Collector is unchecked, see above). Returned metrics that
|
||||||
|
// share the same descriptor must differ in their variable label
|
||||||
|
// values.
|
||||||
|
//
|
||||||
|
// This method may be called concurrently and must therefore be
|
||||||
|
// implemented in a concurrency safe way. Blocking occurs at the expense
|
||||||
|
// of total performance of rendering all registered metrics. Ideally,
|
||||||
|
// Collector implementations support concurrent readers.
|
||||||
|
Collect(chan<- Metric)
|
||||||
|
}
|
||||||
|
|
||||||
|
// DescribeByCollect is a helper to implement the Describe method of a custom
|
||||||
|
// Collector. It collects the metrics from the provided Collector and sends
|
||||||
|
// their descriptors to the provided channel.
|
||||||
|
//
|
||||||
|
// If a Collector collects the same metrics throughout its lifetime, its
|
||||||
|
// Describe method can simply be implemented as:
|
||||||
|
//
|
||||||
|
// func (c customCollector) Describe(ch chan<- *Desc) {
|
||||||
|
// DescribeByCollect(c, ch)
|
||||||
|
// }
|
||||||
|
//
|
||||||
|
// However, this will not work if the metrics collected change dynamically over
|
||||||
|
// the lifetime of the Collector in a way that their combined set of descriptors
|
||||||
|
// changes as well. The shortcut implementation will then violate the contract
|
||||||
|
// of the Describe method. If a Collector sometimes collects no metrics at all
|
||||||
|
// (for example vectors like CounterVec, GaugeVec, etc., which only collect
|
||||||
|
// metrics after a metric with a fully specified label set has been accessed),
|
||||||
|
// it might even get registered as an unchecked Collector (cf. the Register
|
||||||
|
// method of the Registerer interface). Hence, only use this shortcut
|
||||||
|
// implementation of Describe if you are certain to fulfill the contract.
|
||||||
|
//
|
||||||
|
// The Collector example demonstrates a use of DescribeByCollect.
|
||||||
|
func DescribeByCollect(c Collector, descs chan<- *Desc) {
|
||||||
|
metrics := make(chan Metric)
|
||||||
|
go func() {
|
||||||
|
c.Collect(metrics)
|
||||||
|
close(metrics)
|
||||||
|
}()
|
||||||
|
for m := range metrics {
|
||||||
|
descs <- m.Desc()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// selfCollector implements Collector for a single Metric so that the Metric
|
||||||
|
// collects itself. Add it as an anonymous field to a struct that implements
|
||||||
|
// Metric, and call init with the Metric itself as an argument.
|
||||||
|
type selfCollector struct {
|
||||||
|
self Metric
|
||||||
|
}
|
||||||
|
|
||||||
|
// init provides the selfCollector with a reference to the metric it is supposed
|
||||||
|
// to collect. It is usually called within the factory function to create a
|
||||||
|
// metric. See example.
|
||||||
|
func (c *selfCollector) init(self Metric) {
|
||||||
|
c.self = self
|
||||||
|
}
|
||||||
|
|
||||||
|
// Describe implements Collector.
|
||||||
|
func (c *selfCollector) Describe(ch chan<- *Desc) {
|
||||||
|
ch <- c.self.Desc()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Collect implements Collector.
|
||||||
|
func (c *selfCollector) Collect(ch chan<- Metric) {
|
||||||
|
ch <- c.self
|
||||||
|
}
|
||||||
|
|
||||||
|
// collectorMetric is a metric that is also a collector.
|
||||||
|
// Because of selfCollector, most (if not all) Metrics in
|
||||||
|
// this package are also collectors.
|
||||||
|
type collectorMetric interface {
|
||||||
|
Metric
|
||||||
|
Collector
|
||||||
|
}
|
||||||
30
vendor/github.com/prometheus/client_golang/prometheus/collectorfunc.go
generated
vendored
Normal file
30
vendor/github.com/prometheus/client_golang/prometheus/collectorfunc.go
generated
vendored
Normal file
@ -0,0 +1,30 @@
|
|||||||
|
// Copyright 2025 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus
|
||||||
|
|
||||||
|
// CollectorFunc is a convenient way to implement a Prometheus Collector
|
||||||
|
// without interface boilerplate.
|
||||||
|
// This implementation is based on DescribeByCollect method.
|
||||||
|
// familiarize yourself to it before using.
|
||||||
|
type CollectorFunc func(chan<- Metric)
|
||||||
|
|
||||||
|
// Collect calls the defined CollectorFunc function with the provided Metrics channel
|
||||||
|
func (f CollectorFunc) Collect(ch chan<- Metric) {
|
||||||
|
f(ch)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Describe sends the descriptor information using DescribeByCollect
|
||||||
|
func (f CollectorFunc) Describe(ch chan<- *Desc) {
|
||||||
|
DescribeByCollect(f, ch)
|
||||||
|
}
|
||||||
40
vendor/github.com/prometheus/client_golang/prometheus/collectors/collectors.go
generated
vendored
Normal file
40
vendor/github.com/prometheus/client_golang/prometheus/collectors/collectors.go
generated
vendored
Normal file
@ -0,0 +1,40 @@
|
|||||||
|
// Copyright 2021 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
// Package collectors provides implementations of prometheus.Collector to
|
||||||
|
// conveniently collect process and Go-related metrics.
|
||||||
|
package collectors
|
||||||
|
|
||||||
|
import "github.com/prometheus/client_golang/prometheus"
|
||||||
|
|
||||||
|
// NewBuildInfoCollector returns a collector collecting a single metric
|
||||||
|
// "go_build_info" with the constant value 1 and three labels "path", "version",
|
||||||
|
// and "checksum". Their label values contain the main module path, version, and
|
||||||
|
// checksum, respectively. The labels will only have meaningful values if the
|
||||||
|
// binary is built with Go module support and from source code retrieved from
|
||||||
|
// the source repository (rather than the local file system). This is usually
|
||||||
|
// accomplished by building from outside of GOPATH, specifying the full address
|
||||||
|
// of the main package, e.g. "GO111MODULE=on go run
|
||||||
|
// github.com/prometheus/client_golang/examples/random". If built without Go
|
||||||
|
// module support, all label values will be "unknown". If built with Go module
|
||||||
|
// support but using the source code from the local file system, the "path" will
|
||||||
|
// be set appropriately, but "checksum" will be empty and "version" will be
|
||||||
|
// "(devel)".
|
||||||
|
//
|
||||||
|
// This collector uses only the build information for the main module. See
|
||||||
|
// https://github.com/povilasv/prommod for an example of a collector for the
|
||||||
|
// module dependencies.
|
||||||
|
func NewBuildInfoCollector() prometheus.Collector {
|
||||||
|
//nolint:staticcheck // Ignore SA1019 until v2.
|
||||||
|
return prometheus.NewBuildInfoCollector()
|
||||||
|
}
|
||||||
119
vendor/github.com/prometheus/client_golang/prometheus/collectors/dbstats_collector.go
generated
vendored
Normal file
119
vendor/github.com/prometheus/client_golang/prometheus/collectors/dbstats_collector.go
generated
vendored
Normal file
@ -0,0 +1,119 @@
|
|||||||
|
// Copyright 2021 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package collectors
|
||||||
|
|
||||||
|
import (
|
||||||
|
"database/sql"
|
||||||
|
|
||||||
|
"github.com/prometheus/client_golang/prometheus"
|
||||||
|
)
|
||||||
|
|
||||||
|
type dbStatsCollector struct {
|
||||||
|
db *sql.DB
|
||||||
|
|
||||||
|
maxOpenConnections *prometheus.Desc
|
||||||
|
|
||||||
|
openConnections *prometheus.Desc
|
||||||
|
inUseConnections *prometheus.Desc
|
||||||
|
idleConnections *prometheus.Desc
|
||||||
|
|
||||||
|
waitCount *prometheus.Desc
|
||||||
|
waitDuration *prometheus.Desc
|
||||||
|
maxIdleClosed *prometheus.Desc
|
||||||
|
maxIdleTimeClosed *prometheus.Desc
|
||||||
|
maxLifetimeClosed *prometheus.Desc
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewDBStatsCollector returns a collector that exports metrics about the given *sql.DB.
|
||||||
|
// See https://golang.org/pkg/database/sql/#DBStats for more information on stats.
|
||||||
|
func NewDBStatsCollector(db *sql.DB, dbName string) prometheus.Collector {
|
||||||
|
fqName := func(name string) string {
|
||||||
|
return "go_sql_" + name
|
||||||
|
}
|
||||||
|
return &dbStatsCollector{
|
||||||
|
db: db,
|
||||||
|
maxOpenConnections: prometheus.NewDesc(
|
||||||
|
fqName("max_open_connections"),
|
||||||
|
"Maximum number of open connections to the database.",
|
||||||
|
nil, prometheus.Labels{"db_name": dbName},
|
||||||
|
),
|
||||||
|
openConnections: prometheus.NewDesc(
|
||||||
|
fqName("open_connections"),
|
||||||
|
"The number of established connections both in use and idle.",
|
||||||
|
nil, prometheus.Labels{"db_name": dbName},
|
||||||
|
),
|
||||||
|
inUseConnections: prometheus.NewDesc(
|
||||||
|
fqName("in_use_connections"),
|
||||||
|
"The number of connections currently in use.",
|
||||||
|
nil, prometheus.Labels{"db_name": dbName},
|
||||||
|
),
|
||||||
|
idleConnections: prometheus.NewDesc(
|
||||||
|
fqName("idle_connections"),
|
||||||
|
"The number of idle connections.",
|
||||||
|
nil, prometheus.Labels{"db_name": dbName},
|
||||||
|
),
|
||||||
|
waitCount: prometheus.NewDesc(
|
||||||
|
fqName("wait_count_total"),
|
||||||
|
"The total number of connections waited for.",
|
||||||
|
nil, prometheus.Labels{"db_name": dbName},
|
||||||
|
),
|
||||||
|
waitDuration: prometheus.NewDesc(
|
||||||
|
fqName("wait_duration_seconds_total"),
|
||||||
|
"The total time blocked waiting for a new connection.",
|
||||||
|
nil, prometheus.Labels{"db_name": dbName},
|
||||||
|
),
|
||||||
|
maxIdleClosed: prometheus.NewDesc(
|
||||||
|
fqName("max_idle_closed_total"),
|
||||||
|
"The total number of connections closed due to SetMaxIdleConns.",
|
||||||
|
nil, prometheus.Labels{"db_name": dbName},
|
||||||
|
),
|
||||||
|
maxIdleTimeClosed: prometheus.NewDesc(
|
||||||
|
fqName("max_idle_time_closed_total"),
|
||||||
|
"The total number of connections closed due to SetConnMaxIdleTime.",
|
||||||
|
nil, prometheus.Labels{"db_name": dbName},
|
||||||
|
),
|
||||||
|
maxLifetimeClosed: prometheus.NewDesc(
|
||||||
|
fqName("max_lifetime_closed_total"),
|
||||||
|
"The total number of connections closed due to SetConnMaxLifetime.",
|
||||||
|
nil, prometheus.Labels{"db_name": dbName},
|
||||||
|
),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Describe implements Collector.
|
||||||
|
func (c *dbStatsCollector) Describe(ch chan<- *prometheus.Desc) {
|
||||||
|
ch <- c.maxOpenConnections
|
||||||
|
ch <- c.openConnections
|
||||||
|
ch <- c.inUseConnections
|
||||||
|
ch <- c.idleConnections
|
||||||
|
ch <- c.waitCount
|
||||||
|
ch <- c.waitDuration
|
||||||
|
ch <- c.maxIdleClosed
|
||||||
|
ch <- c.maxLifetimeClosed
|
||||||
|
ch <- c.maxIdleTimeClosed
|
||||||
|
}
|
||||||
|
|
||||||
|
// Collect implements Collector.
|
||||||
|
func (c *dbStatsCollector) Collect(ch chan<- prometheus.Metric) {
|
||||||
|
stats := c.db.Stats()
|
||||||
|
ch <- prometheus.MustNewConstMetric(c.maxOpenConnections, prometheus.GaugeValue, float64(stats.MaxOpenConnections))
|
||||||
|
ch <- prometheus.MustNewConstMetric(c.openConnections, prometheus.GaugeValue, float64(stats.OpenConnections))
|
||||||
|
ch <- prometheus.MustNewConstMetric(c.inUseConnections, prometheus.GaugeValue, float64(stats.InUse))
|
||||||
|
ch <- prometheus.MustNewConstMetric(c.idleConnections, prometheus.GaugeValue, float64(stats.Idle))
|
||||||
|
ch <- prometheus.MustNewConstMetric(c.waitCount, prometheus.CounterValue, float64(stats.WaitCount))
|
||||||
|
ch <- prometheus.MustNewConstMetric(c.waitDuration, prometheus.CounterValue, stats.WaitDuration.Seconds())
|
||||||
|
ch <- prometheus.MustNewConstMetric(c.maxIdleClosed, prometheus.CounterValue, float64(stats.MaxIdleClosed))
|
||||||
|
ch <- prometheus.MustNewConstMetric(c.maxLifetimeClosed, prometheus.CounterValue, float64(stats.MaxLifetimeClosed))
|
||||||
|
ch <- prometheus.MustNewConstMetric(c.maxIdleTimeClosed, prometheus.CounterValue, float64(stats.MaxIdleTimeClosed))
|
||||||
|
}
|
||||||
57
vendor/github.com/prometheus/client_golang/prometheus/collectors/expvar_collector.go
generated
vendored
Normal file
57
vendor/github.com/prometheus/client_golang/prometheus/collectors/expvar_collector.go
generated
vendored
Normal file
@ -0,0 +1,57 @@
|
|||||||
|
// Copyright 2021 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package collectors
|
||||||
|
|
||||||
|
import "github.com/prometheus/client_golang/prometheus"
|
||||||
|
|
||||||
|
// NewExpvarCollector returns a newly allocated expvar Collector.
|
||||||
|
//
|
||||||
|
// An expvar Collector collects metrics from the expvar interface. It provides a
|
||||||
|
// quick way to expose numeric values that are already exported via expvar as
|
||||||
|
// Prometheus metrics. Note that the data models of expvar and Prometheus are
|
||||||
|
// fundamentally different, and that the expvar Collector is inherently slower
|
||||||
|
// than native Prometheus metrics. Thus, the expvar Collector is probably great
|
||||||
|
// for experiments and prototyping, but you should seriously consider a more
|
||||||
|
// direct implementation of Prometheus metrics for monitoring production
|
||||||
|
// systems.
|
||||||
|
//
|
||||||
|
// The exports map has the following meaning:
|
||||||
|
//
|
||||||
|
// The keys in the map correspond to expvar keys, i.e. for every expvar key you
|
||||||
|
// want to export as Prometheus metric, you need an entry in the exports
|
||||||
|
// map. The descriptor mapped to each key describes how to export the expvar
|
||||||
|
// value. It defines the name and the help string of the Prometheus metric
|
||||||
|
// proxying the expvar value. The type will always be Untyped.
|
||||||
|
//
|
||||||
|
// For descriptors without variable labels, the expvar value must be a number or
|
||||||
|
// a bool. The number is then directly exported as the Prometheus sample
|
||||||
|
// value. (For a bool, 'false' translates to 0 and 'true' to 1). Expvar values
|
||||||
|
// that are not numbers or bools are silently ignored.
|
||||||
|
//
|
||||||
|
// If the descriptor has one variable label, the expvar value must be an expvar
|
||||||
|
// map. The keys in the expvar map become the various values of the one
|
||||||
|
// Prometheus label. The values in the expvar map must be numbers or bools again
|
||||||
|
// as above.
|
||||||
|
//
|
||||||
|
// For descriptors with more than one variable label, the expvar must be a
|
||||||
|
// nested expvar map, i.e. where the values of the topmost map are maps again
|
||||||
|
// etc. until a depth is reached that corresponds to the number of labels. The
|
||||||
|
// leaves of that structure must be numbers or bools as above to serve as the
|
||||||
|
// sample values.
|
||||||
|
//
|
||||||
|
// Anything that does not fit into the scheme above is silently ignored.
|
||||||
|
func NewExpvarCollector(exports map[string]*prometheus.Desc) prometheus.Collector {
|
||||||
|
//nolint:staticcheck // Ignore SA1019 until v2.
|
||||||
|
return prometheus.NewExpvarCollector(exports)
|
||||||
|
}
|
||||||
167
vendor/github.com/prometheus/client_golang/prometheus/collectors/go_collector_latest.go
generated
vendored
Normal file
167
vendor/github.com/prometheus/client_golang/prometheus/collectors/go_collector_latest.go
generated
vendored
Normal file
@ -0,0 +1,167 @@
|
|||||||
|
// Copyright 2021 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
//go:build go1.17
|
||||||
|
// +build go1.17
|
||||||
|
|
||||||
|
package collectors
|
||||||
|
|
||||||
|
import (
|
||||||
|
"regexp"
|
||||||
|
|
||||||
|
"github.com/prometheus/client_golang/prometheus"
|
||||||
|
"github.com/prometheus/client_golang/prometheus/internal"
|
||||||
|
)
|
||||||
|
|
||||||
|
var (
|
||||||
|
// MetricsAll allows all the metrics to be collected from Go runtime.
|
||||||
|
MetricsAll = GoRuntimeMetricsRule{regexp.MustCompile("/.*")}
|
||||||
|
// MetricsGC allows only GC metrics to be collected from Go runtime.
|
||||||
|
// e.g. go_gc_cycles_automatic_gc_cycles_total
|
||||||
|
// NOTE: This does not include new class of "/cpu/classes/gc/..." metrics.
|
||||||
|
// Use custom metric rule to access those.
|
||||||
|
MetricsGC = GoRuntimeMetricsRule{regexp.MustCompile(`^/gc/.*`)}
|
||||||
|
// MetricsMemory allows only memory metrics to be collected from Go runtime.
|
||||||
|
// e.g. go_memory_classes_heap_free_bytes
|
||||||
|
MetricsMemory = GoRuntimeMetricsRule{regexp.MustCompile(`^/memory/.*`)}
|
||||||
|
// MetricsScheduler allows only scheduler metrics to be collected from Go runtime.
|
||||||
|
// e.g. go_sched_goroutines_goroutines
|
||||||
|
MetricsScheduler = GoRuntimeMetricsRule{regexp.MustCompile(`^/sched/.*`)}
|
||||||
|
// MetricsDebug allows only debug metrics to be collected from Go runtime.
|
||||||
|
// e.g. go_godebug_non_default_behavior_gocachetest_events_total
|
||||||
|
MetricsDebug = GoRuntimeMetricsRule{regexp.MustCompile(`^/godebug/.*`)}
|
||||||
|
)
|
||||||
|
|
||||||
|
// WithGoCollectorMemStatsMetricsDisabled disables metrics that is gathered in runtime.MemStats structure such as:
|
||||||
|
//
|
||||||
|
// go_memstats_alloc_bytes
|
||||||
|
// go_memstats_alloc_bytes_total
|
||||||
|
// go_memstats_sys_bytes
|
||||||
|
// go_memstats_mallocs_total
|
||||||
|
// go_memstats_frees_total
|
||||||
|
// go_memstats_heap_alloc_bytes
|
||||||
|
// go_memstats_heap_sys_bytes
|
||||||
|
// go_memstats_heap_idle_bytes
|
||||||
|
// go_memstats_heap_inuse_bytes
|
||||||
|
// go_memstats_heap_released_bytes
|
||||||
|
// go_memstats_heap_objects
|
||||||
|
// go_memstats_stack_inuse_bytes
|
||||||
|
// go_memstats_stack_sys_bytes
|
||||||
|
// go_memstats_mspan_inuse_bytes
|
||||||
|
// go_memstats_mspan_sys_bytes
|
||||||
|
// go_memstats_mcache_inuse_bytes
|
||||||
|
// go_memstats_mcache_sys_bytes
|
||||||
|
// go_memstats_buck_hash_sys_bytes
|
||||||
|
// go_memstats_gc_sys_bytes
|
||||||
|
// go_memstats_other_sys_bytes
|
||||||
|
// go_memstats_next_gc_bytes
|
||||||
|
//
|
||||||
|
// so the metrics known from pre client_golang v1.12.0,
|
||||||
|
//
|
||||||
|
// NOTE(bwplotka): The above represents runtime.MemStats statistics, but they are
|
||||||
|
// actually implemented using new runtime/metrics package. (except skipped go_memstats_gc_cpu_fraction
|
||||||
|
// -- see https://github.com/prometheus/client_golang/issues/842#issuecomment-861812034 for explanation).
|
||||||
|
//
|
||||||
|
// Some users might want to disable this on collector level (although you can use scrape relabelling on Prometheus),
|
||||||
|
// because similar metrics can be now obtained using WithGoCollectorRuntimeMetrics. Note that the semantics of new
|
||||||
|
// metrics might be different, plus the names can be change over time with different Go version.
|
||||||
|
//
|
||||||
|
// NOTE(bwplotka): Changing metric names can be tedious at times as the alerts, recording rules and dashboards have to be adjusted.
|
||||||
|
// The old metrics are also very useful, with many guides and books written about how to interpret them.
|
||||||
|
//
|
||||||
|
// As a result our recommendation would be to stick with MemStats like metrics and enable other runtime/metrics if you are interested
|
||||||
|
// in advanced insights Go provides. See ExampleGoCollector_WithAdvancedGoMetrics.
|
||||||
|
func WithGoCollectorMemStatsMetricsDisabled() func(options *internal.GoCollectorOptions) {
|
||||||
|
return func(o *internal.GoCollectorOptions) {
|
||||||
|
o.DisableMemStatsLikeMetrics = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// GoRuntimeMetricsRule allow enabling and configuring particular group of runtime/metrics.
|
||||||
|
// TODO(bwplotka): Consider adding ability to adjust buckets.
|
||||||
|
type GoRuntimeMetricsRule struct {
|
||||||
|
// Matcher represents RE2 expression will match the runtime/metrics from https://pkg.go.dev/runtime/metrics
|
||||||
|
// Use `regexp.MustCompile` or `regexp.Compile` to create this field.
|
||||||
|
Matcher *regexp.Regexp
|
||||||
|
}
|
||||||
|
|
||||||
|
// WithGoCollectorRuntimeMetrics allows enabling and configuring particular group of runtime/metrics.
|
||||||
|
// See the list of metrics https://pkg.go.dev/runtime/metrics (pick the Go version you use there!).
|
||||||
|
// You can use this option in repeated manner, which will add new rules. The order of rules is important, the last rule
|
||||||
|
// that matches particular metrics is applied.
|
||||||
|
func WithGoCollectorRuntimeMetrics(rules ...GoRuntimeMetricsRule) func(options *internal.GoCollectorOptions) {
|
||||||
|
rs := make([]internal.GoCollectorRule, len(rules))
|
||||||
|
for i, r := range rules {
|
||||||
|
rs[i] = internal.GoCollectorRule{
|
||||||
|
Matcher: r.Matcher,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return func(o *internal.GoCollectorOptions) {
|
||||||
|
o.RuntimeMetricRules = append(o.RuntimeMetricRules, rs...)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// WithoutGoCollectorRuntimeMetrics allows disabling group of runtime/metrics that you might have added in WithGoCollectorRuntimeMetrics.
|
||||||
|
// It behaves similarly to WithGoCollectorRuntimeMetrics just with deny-list semantics.
|
||||||
|
func WithoutGoCollectorRuntimeMetrics(matchers ...*regexp.Regexp) func(options *internal.GoCollectorOptions) {
|
||||||
|
rs := make([]internal.GoCollectorRule, len(matchers))
|
||||||
|
for i, m := range matchers {
|
||||||
|
rs[i] = internal.GoCollectorRule{
|
||||||
|
Matcher: m,
|
||||||
|
Deny: true,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return func(o *internal.GoCollectorOptions) {
|
||||||
|
o.RuntimeMetricRules = append(o.RuntimeMetricRules, rs...)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// GoCollectionOption represents Go collection option flag.
|
||||||
|
// Deprecated.
|
||||||
|
type GoCollectionOption uint32
|
||||||
|
|
||||||
|
const (
|
||||||
|
// GoRuntimeMemStatsCollection represents the metrics represented by runtime.MemStats structure.
|
||||||
|
//
|
||||||
|
// Deprecated: Use WithGoCollectorMemStatsMetricsDisabled() function to disable those metrics in the collector.
|
||||||
|
GoRuntimeMemStatsCollection GoCollectionOption = 1 << iota
|
||||||
|
// GoRuntimeMetricsCollection is the new set of metrics represented by runtime/metrics package.
|
||||||
|
//
|
||||||
|
// Deprecated: Use WithGoCollectorRuntimeMetrics(GoRuntimeMetricsRule{Matcher: regexp.MustCompile("/.*")})
|
||||||
|
// function to enable those metrics in the collector.
|
||||||
|
GoRuntimeMetricsCollection
|
||||||
|
)
|
||||||
|
|
||||||
|
// WithGoCollections allows enabling different collections for Go collector on top of base metrics.
|
||||||
|
//
|
||||||
|
// Deprecated: Use WithGoCollectorRuntimeMetrics() and WithGoCollectorMemStatsMetricsDisabled() instead to control metrics.
|
||||||
|
func WithGoCollections(flags GoCollectionOption) func(options *internal.GoCollectorOptions) {
|
||||||
|
return func(options *internal.GoCollectorOptions) {
|
||||||
|
if flags&GoRuntimeMemStatsCollection == 0 {
|
||||||
|
WithGoCollectorMemStatsMetricsDisabled()(options)
|
||||||
|
}
|
||||||
|
|
||||||
|
if flags&GoRuntimeMetricsCollection != 0 {
|
||||||
|
WithGoCollectorRuntimeMetrics(GoRuntimeMetricsRule{Matcher: regexp.MustCompile("/.*")})(options)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewGoCollector returns a collector that exports metrics about the current Go
|
||||||
|
// process using debug.GCStats (base metrics) and runtime/metrics (both in MemStats style and new ones).
|
||||||
|
func NewGoCollector(opts ...func(o *internal.GoCollectorOptions)) prometheus.Collector {
|
||||||
|
//nolint:staticcheck // Ignore SA1019 until v2.
|
||||||
|
return prometheus.NewGoCollector(opts...)
|
||||||
|
}
|
||||||
56
vendor/github.com/prometheus/client_golang/prometheus/collectors/process_collector.go
generated
vendored
Normal file
56
vendor/github.com/prometheus/client_golang/prometheus/collectors/process_collector.go
generated
vendored
Normal file
@ -0,0 +1,56 @@
|
|||||||
|
// Copyright 2021 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package collectors
|
||||||
|
|
||||||
|
import "github.com/prometheus/client_golang/prometheus"
|
||||||
|
|
||||||
|
// ProcessCollectorOpts defines the behavior of a process metrics collector
|
||||||
|
// created with NewProcessCollector.
|
||||||
|
type ProcessCollectorOpts struct {
|
||||||
|
// PidFn returns the PID of the process the collector collects metrics
|
||||||
|
// for. It is called upon each collection. By default, the PID of the
|
||||||
|
// current process is used, as determined on construction time by
|
||||||
|
// calling os.Getpid().
|
||||||
|
PidFn func() (int, error)
|
||||||
|
// If non-empty, each of the collected metrics is prefixed by the
|
||||||
|
// provided string and an underscore ("_").
|
||||||
|
Namespace string
|
||||||
|
// If true, any error encountered during collection is reported as an
|
||||||
|
// invalid metric (see NewInvalidMetric). Otherwise, errors are ignored
|
||||||
|
// and the collected metrics will be incomplete. (Possibly, no metrics
|
||||||
|
// will be collected at all.) While that's usually not desired, it is
|
||||||
|
// appropriate for the common "mix-in" of process metrics, where process
|
||||||
|
// metrics are nice to have, but failing to collect them should not
|
||||||
|
// disrupt the collection of the remaining metrics.
|
||||||
|
ReportErrors bool
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewProcessCollector returns a collector which exports the current state of
|
||||||
|
// process metrics including CPU, memory and file descriptor usage as well as
|
||||||
|
// the process start time. The detailed behavior is defined by the provided
|
||||||
|
// ProcessCollectorOpts. The zero value of ProcessCollectorOpts creates a
|
||||||
|
// collector for the current process with an empty namespace string and no error
|
||||||
|
// reporting.
|
||||||
|
//
|
||||||
|
// The collector only works on operating systems with a Linux-style proc
|
||||||
|
// filesystem and on Microsoft Windows. On other operating systems, it will not
|
||||||
|
// collect any metrics.
|
||||||
|
func NewProcessCollector(opts ProcessCollectorOpts) prometheus.Collector {
|
||||||
|
//nolint:staticcheck // Ignore SA1019 until v2.
|
||||||
|
return prometheus.NewProcessCollector(prometheus.ProcessCollectorOpts{
|
||||||
|
PidFn: opts.PidFn,
|
||||||
|
Namespace: opts.Namespace,
|
||||||
|
ReportErrors: opts.ReportErrors,
|
||||||
|
})
|
||||||
|
}
|
||||||
361
vendor/github.com/prometheus/client_golang/prometheus/counter.go
generated
vendored
Normal file
361
vendor/github.com/prometheus/client_golang/prometheus/counter.go
generated
vendored
Normal file
@ -0,0 +1,361 @@
|
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"math"
|
||||||
|
"sync/atomic"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
dto "github.com/prometheus/client_model/go"
|
||||||
|
"google.golang.org/protobuf/types/known/timestamppb"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Counter is a Metric that represents a single numerical value that only ever
|
||||||
|
// goes up. That implies that it cannot be used to count items whose number can
|
||||||
|
// also go down, e.g. the number of currently running goroutines. Those
|
||||||
|
// "counters" are represented by Gauges.
|
||||||
|
//
|
||||||
|
// A Counter is typically used to count requests served, tasks completed, errors
|
||||||
|
// occurred, etc.
|
||||||
|
//
|
||||||
|
// To create Counter instances, use NewCounter.
|
||||||
|
type Counter interface {
|
||||||
|
Metric
|
||||||
|
Collector
|
||||||
|
|
||||||
|
// Inc increments the counter by 1. Use Add to increment it by arbitrary
|
||||||
|
// non-negative values.
|
||||||
|
Inc()
|
||||||
|
// Add adds the given value to the counter. It panics if the value is <
|
||||||
|
// 0.
|
||||||
|
Add(float64)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ExemplarAdder is implemented by Counters that offer the option of adding a
|
||||||
|
// value to the Counter together with an exemplar. Its AddWithExemplar method
|
||||||
|
// works like the Add method of the Counter interface but also replaces the
|
||||||
|
// currently saved exemplar (if any) with a new one, created from the provided
|
||||||
|
// value, the current time as timestamp, and the provided labels. Empty Labels
|
||||||
|
// will lead to a valid (label-less) exemplar. But if Labels is nil, the current
|
||||||
|
// exemplar is left in place. AddWithExemplar panics if the value is < 0, if any
|
||||||
|
// of the provided labels are invalid, or if the provided labels contain more
|
||||||
|
// than 128 runes in total.
|
||||||
|
type ExemplarAdder interface {
|
||||||
|
AddWithExemplar(value float64, exemplar Labels)
|
||||||
|
}
|
||||||
|
|
||||||
|
// CounterOpts is an alias for Opts. See there for doc comments.
|
||||||
|
type CounterOpts Opts
|
||||||
|
|
||||||
|
// CounterVecOpts bundles the options to create a CounterVec metric.
|
||||||
|
// It is mandatory to set CounterOpts, see there for mandatory fields. VariableLabels
|
||||||
|
// is optional and can safely be left to its default value.
|
||||||
|
type CounterVecOpts struct {
|
||||||
|
CounterOpts
|
||||||
|
|
||||||
|
// VariableLabels are used to partition the metric vector by the given set
|
||||||
|
// of labels. Each label value will be constrained with the optional Constraint
|
||||||
|
// function, if provided.
|
||||||
|
VariableLabels ConstrainableLabels
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewCounter creates a new Counter based on the provided CounterOpts.
|
||||||
|
//
|
||||||
|
// The returned implementation also implements ExemplarAdder. It is safe to
|
||||||
|
// perform the corresponding type assertion.
|
||||||
|
//
|
||||||
|
// The returned implementation tracks the counter value in two separate
|
||||||
|
// variables, a float64 and a uint64. The latter is used to track calls of the
|
||||||
|
// Inc method and calls of the Add method with a value that can be represented
|
||||||
|
// as a uint64. This allows atomic increments of the counter with optimal
|
||||||
|
// performance. (It is common to have an Inc call in very hot execution paths.)
|
||||||
|
// Both internal tracking values are added up in the Write method. This has to
|
||||||
|
// be taken into account when it comes to precision and overflow behavior.
|
||||||
|
func NewCounter(opts CounterOpts) Counter {
|
||||||
|
desc := V2.NewDesc(
|
||||||
|
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name),
|
||||||
|
opts.Help,
|
||||||
|
UnconstrainedLabels(nil),
|
||||||
|
opts.ConstLabels,
|
||||||
|
WithUnit(opts.Unit),
|
||||||
|
)
|
||||||
|
if opts.now == nil {
|
||||||
|
opts.now = time.Now
|
||||||
|
}
|
||||||
|
result := &counter{desc: desc, labelPairs: desc.constLabelPairs, now: opts.now}
|
||||||
|
result.init(result) // Init self-collection.
|
||||||
|
result.createdTs = timestamppb.New(opts.now())
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|
||||||
|
type counter struct {
|
||||||
|
// valBits contains the bits of the represented float64 value, while
|
||||||
|
// valInt stores values that are exact integers. Both have to go first
|
||||||
|
// in the struct to guarantee alignment for atomic operations.
|
||||||
|
// http://golang.org/pkg/sync/atomic/#pkg-note-BUG
|
||||||
|
valBits uint64
|
||||||
|
valInt uint64
|
||||||
|
|
||||||
|
selfCollector
|
||||||
|
desc *Desc
|
||||||
|
|
||||||
|
createdTs *timestamppb.Timestamp
|
||||||
|
labelPairs []*dto.LabelPair
|
||||||
|
exemplar atomic.Value // Containing nil or a *dto.Exemplar.
|
||||||
|
|
||||||
|
// now is for testing purposes, by default it's time.Now.
|
||||||
|
now func() time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *counter) Desc() *Desc {
|
||||||
|
return c.desc
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *counter) Add(v float64) {
|
||||||
|
if v < 0 {
|
||||||
|
panic(errors.New("counter cannot decrease in value"))
|
||||||
|
}
|
||||||
|
|
||||||
|
ival := uint64(v)
|
||||||
|
if float64(ival) == v {
|
||||||
|
atomic.AddUint64(&c.valInt, ival)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
for {
|
||||||
|
oldBits := atomic.LoadUint64(&c.valBits)
|
||||||
|
newBits := math.Float64bits(math.Float64frombits(oldBits) + v)
|
||||||
|
if atomic.CompareAndSwapUint64(&c.valBits, oldBits, newBits) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *counter) AddWithExemplar(v float64, e Labels) {
|
||||||
|
c.Add(v)
|
||||||
|
c.updateExemplar(v, e)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *counter) Inc() {
|
||||||
|
atomic.AddUint64(&c.valInt, 1)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *counter) get() float64 {
|
||||||
|
fval := math.Float64frombits(atomic.LoadUint64(&c.valBits))
|
||||||
|
ival := atomic.LoadUint64(&c.valInt)
|
||||||
|
return fval + float64(ival)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *counter) Write(out *dto.Metric) error {
|
||||||
|
// Read the Exemplar first and the value second. This is to avoid a race condition
|
||||||
|
// where users see an exemplar for a not-yet-existing observation.
|
||||||
|
var exemplar *dto.Exemplar
|
||||||
|
if e := c.exemplar.Load(); e != nil {
|
||||||
|
exemplar = e.(*dto.Exemplar)
|
||||||
|
}
|
||||||
|
val := c.get()
|
||||||
|
return populateMetric(CounterValue, val, c.labelPairs, exemplar, out, c.createdTs)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *counter) updateExemplar(v float64, l Labels) {
|
||||||
|
if l == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
e, err := newExemplar(v, c.now(), l)
|
||||||
|
if err != nil {
|
||||||
|
panic(err)
|
||||||
|
}
|
||||||
|
c.exemplar.Store(e)
|
||||||
|
}
|
||||||
|
|
||||||
|
// CounterVec is a Collector that bundles a set of Counters that all share the
|
||||||
|
// same Desc, but have different values for their variable labels. This is used
|
||||||
|
// if you want to count the same thing partitioned by various dimensions
|
||||||
|
// (e.g. number of HTTP requests, partitioned by response code and
|
||||||
|
// method). Create instances with NewCounterVec.
|
||||||
|
type CounterVec struct {
|
||||||
|
*MetricVec
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewCounterVec creates a new CounterVec based on the provided CounterOpts and
|
||||||
|
// partitioned by the given label names.
|
||||||
|
func NewCounterVec(opts CounterOpts, labelNames []string) *CounterVec {
|
||||||
|
return V2.NewCounterVec(CounterVecOpts{
|
||||||
|
CounterOpts: opts,
|
||||||
|
VariableLabels: UnconstrainedLabels(labelNames),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewCounterVec creates a new CounterVec based on the provided CounterVecOpts.
|
||||||
|
func (v2) NewCounterVec(opts CounterVecOpts) *CounterVec {
|
||||||
|
desc := V2.NewDesc(
|
||||||
|
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name),
|
||||||
|
opts.Help,
|
||||||
|
opts.VariableLabels,
|
||||||
|
opts.ConstLabels,
|
||||||
|
WithUnit(opts.Unit),
|
||||||
|
)
|
||||||
|
if opts.now == nil {
|
||||||
|
opts.now = time.Now
|
||||||
|
}
|
||||||
|
return &CounterVec{
|
||||||
|
MetricVec: NewMetricVec(desc, func(lvs ...string) Metric {
|
||||||
|
if len(lvs) != len(desc.variableLabels.names) {
|
||||||
|
panic(makeInconsistentCardinalityError(desc.fqName, desc.variableLabels.names, lvs))
|
||||||
|
}
|
||||||
|
result := &counter{desc: desc, labelPairs: MakeLabelPairs(desc, lvs), now: opts.now}
|
||||||
|
result.init(result) // Init self-collection.
|
||||||
|
result.createdTs = timestamppb.New(opts.now())
|
||||||
|
return result
|
||||||
|
}),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetMetricWithLabelValues returns the Counter for the given slice of label
|
||||||
|
// values (same order as the variable labels in Desc). If that combination of
|
||||||
|
// label values is accessed for the first time, a new Counter is created.
|
||||||
|
//
|
||||||
|
// It is possible to call this method without using the returned Counter to only
|
||||||
|
// create the new Counter but leave it at its starting value 0. See also the
|
||||||
|
// SummaryVec example.
|
||||||
|
//
|
||||||
|
// Keeping the Counter for later use is possible (and should be considered if
|
||||||
|
// performance is critical), but keep in mind that Reset, DeleteLabelValues and
|
||||||
|
// Delete can be used to delete the Counter from the CounterVec. In that case,
|
||||||
|
// the Counter will still exist, but it will not be exported anymore, even if a
|
||||||
|
// Counter with the same label values is created later.
|
||||||
|
//
|
||||||
|
// An error is returned if the number of label values is not the same as the
|
||||||
|
// number of variable labels in Desc (minus any curried labels).
|
||||||
|
//
|
||||||
|
// Note that for more than one label value, this method is prone to mistakes
|
||||||
|
// caused by an incorrect order of arguments. Consider GetMetricWith(Labels) as
|
||||||
|
// an alternative to avoid that type of mistake. For higher label numbers, the
|
||||||
|
// latter has a much more readable (albeit more verbose) syntax, but it comes
|
||||||
|
// with a performance overhead (for creating and processing the Labels map).
|
||||||
|
// See also the GaugeVec example.
|
||||||
|
func (v *CounterVec) GetMetricWithLabelValues(lvs ...string) (Counter, error) {
|
||||||
|
metric, err := v.MetricVec.GetMetricWithLabelValues(lvs...)
|
||||||
|
if metric != nil {
|
||||||
|
return metric.(Counter), err
|
||||||
|
}
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetMetricWith returns the Counter for the given Labels map (the label names
|
||||||
|
// must match those of the variable labels in Desc). If that label map is
|
||||||
|
// accessed for the first time, a new Counter is created. Implications of
|
||||||
|
// creating a Counter without using it and keeping the Counter for later use are
|
||||||
|
// the same as for GetMetricWithLabelValues.
|
||||||
|
//
|
||||||
|
// An error is returned if the number and names of the Labels are inconsistent
|
||||||
|
// with those of the variable labels in Desc (minus any curried labels).
|
||||||
|
//
|
||||||
|
// This method is used for the same purpose as
|
||||||
|
// GetMetricWithLabelValues(...string). See there for pros and cons of the two
|
||||||
|
// methods.
|
||||||
|
func (v *CounterVec) GetMetricWith(labels Labels) (Counter, error) {
|
||||||
|
metric, err := v.MetricVec.GetMetricWith(labels)
|
||||||
|
if metric != nil {
|
||||||
|
return metric.(Counter), err
|
||||||
|
}
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// WithLabelValues works as GetMetricWithLabelValues, but panics where
|
||||||
|
// GetMetricWithLabelValues would have returned an error. Not returning an
|
||||||
|
// error allows shortcuts like
|
||||||
|
//
|
||||||
|
// myVec.WithLabelValues("404", "GET").Add(42)
|
||||||
|
func (v *CounterVec) WithLabelValues(lvs ...string) Counter {
|
||||||
|
c, err := v.GetMetricWithLabelValues(lvs...)
|
||||||
|
if err != nil {
|
||||||
|
panic(err)
|
||||||
|
}
|
||||||
|
return c
|
||||||
|
}
|
||||||
|
|
||||||
|
// With works as GetMetricWith, but panics where GetMetricWithLabels would have
|
||||||
|
// returned an error. Not returning an error allows shortcuts like
|
||||||
|
//
|
||||||
|
// myVec.With(prometheus.Labels{"code": "404", "method": "GET"}).Add(42)
|
||||||
|
func (v *CounterVec) With(labels Labels) Counter {
|
||||||
|
c, err := v.GetMetricWith(labels)
|
||||||
|
if err != nil {
|
||||||
|
panic(err)
|
||||||
|
}
|
||||||
|
return c
|
||||||
|
}
|
||||||
|
|
||||||
|
// CurryWith returns a vector curried with the provided labels, i.e. the
|
||||||
|
// returned vector has those labels pre-set for all labeled operations performed
|
||||||
|
// on it. The cardinality of the curried vector is reduced accordingly. The
|
||||||
|
// order of the remaining labels stays the same (just with the curried labels
|
||||||
|
// taken out of the sequence – which is relevant for the
|
||||||
|
// (GetMetric)WithLabelValues methods). It is possible to curry a curried
|
||||||
|
// vector, but only with labels not yet used for currying before.
|
||||||
|
//
|
||||||
|
// The metrics contained in the CounterVec are shared between the curried and
|
||||||
|
// uncurried vectors. They are just accessed differently. Curried and uncurried
|
||||||
|
// vectors behave identically in terms of collection. Only one must be
|
||||||
|
// registered with a given registry (usually the uncurried version). The Reset
|
||||||
|
// method deletes all metrics, even if called on a curried vector.
|
||||||
|
func (v *CounterVec) CurryWith(labels Labels) (*CounterVec, error) {
|
||||||
|
vec, err := v.MetricVec.CurryWith(labels)
|
||||||
|
if vec != nil {
|
||||||
|
return &CounterVec{vec}, err
|
||||||
|
}
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// MustCurryWith works as CurryWith but panics where CurryWith would have
|
||||||
|
// returned an error.
|
||||||
|
func (v *CounterVec) MustCurryWith(labels Labels) *CounterVec {
|
||||||
|
vec, err := v.CurryWith(labels)
|
||||||
|
if err != nil {
|
||||||
|
panic(err)
|
||||||
|
}
|
||||||
|
return vec
|
||||||
|
}
|
||||||
|
|
||||||
|
// CounterFunc is a Counter whose value is determined at collect time by calling a
|
||||||
|
// provided function.
|
||||||
|
//
|
||||||
|
// To create CounterFunc instances, use NewCounterFunc.
|
||||||
|
type CounterFunc interface {
|
||||||
|
Metric
|
||||||
|
Collector
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewCounterFunc creates a new CounterFunc based on the provided
|
||||||
|
// CounterOpts. The value reported is determined by calling the given function
|
||||||
|
// from within the Write method. Take into account that metric collection may
|
||||||
|
// happen concurrently. If that results in concurrent calls to Write, like in
|
||||||
|
// the case where a CounterFunc is directly registered with Prometheus, the
|
||||||
|
// provided function must be concurrency-safe. The function should also honor
|
||||||
|
// the contract for a Counter (values only go up, not down), but compliance will
|
||||||
|
// not be checked.
|
||||||
|
//
|
||||||
|
// Check out the ExampleGaugeFunc examples for the similar GaugeFunc.
|
||||||
|
func NewCounterFunc(opts CounterOpts, function func() float64) CounterFunc {
|
||||||
|
return newValueFunc(V2.NewDesc(
|
||||||
|
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name),
|
||||||
|
opts.Help,
|
||||||
|
UnconstrainedLabels(nil),
|
||||||
|
opts.ConstLabels,
|
||||||
|
WithUnit(opts.Unit),
|
||||||
|
), CounterValue, function)
|
||||||
|
}
|
||||||
238
vendor/github.com/prometheus/client_golang/prometheus/desc.go
generated
vendored
Normal file
238
vendor/github.com/prometheus/client_golang/prometheus/desc.go
generated
vendored
Normal file
@ -0,0 +1,238 @@
|
|||||||
|
// Copyright 2016 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/cespare/xxhash/v2"
|
||||||
|
dto "github.com/prometheus/client_model/go"
|
||||||
|
"github.com/prometheus/common/model"
|
||||||
|
"google.golang.org/protobuf/proto"
|
||||||
|
|
||||||
|
"github.com/prometheus/client_golang/prometheus/internal"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Desc is the descriptor used by every Prometheus Metric. It is essentially
|
||||||
|
// the immutable meta-data of a Metric. The normal Metric implementations
|
||||||
|
// included in this package manage their Desc under the hood. Users only have to
|
||||||
|
// deal with Desc if they use advanced features like the ExpvarCollector or
|
||||||
|
// custom Collectors and Metrics.
|
||||||
|
//
|
||||||
|
// Descriptors registered with the same registry have to fulfill certain
|
||||||
|
// consistency and uniqueness criteria if they share the same fully-qualified
|
||||||
|
// name: They must have the same help string and the same label names (aka label
|
||||||
|
// dimensions) in each, constLabels and variableLabels, but they must differ in
|
||||||
|
// the values of the constLabels.
|
||||||
|
//
|
||||||
|
// Descriptors that share the same fully-qualified names and the same label
|
||||||
|
// values of their constLabels are considered equal.
|
||||||
|
//
|
||||||
|
// Use NewDesc to create new Desc instances.
|
||||||
|
type Desc struct {
|
||||||
|
// fqName has been built from Namespace, Subsystem, and Name.
|
||||||
|
fqName string
|
||||||
|
// help provides some helpful information about this metric.
|
||||||
|
help string
|
||||||
|
// unit provides the unit of this metric.
|
||||||
|
unit string
|
||||||
|
// constLabelPairs contains precalculated DTO label pairs based on
|
||||||
|
// the constant labels.
|
||||||
|
constLabelPairs []*dto.LabelPair
|
||||||
|
// variableLabels contains names of labels and normalization function for
|
||||||
|
// which the metric maintains variable values.
|
||||||
|
variableLabels *compiledLabels
|
||||||
|
// id is a hash of the values of the ConstLabels and fqName. This
|
||||||
|
// must be unique among all registered descriptors and can therefore be
|
||||||
|
// used as an identifier of the descriptor.
|
||||||
|
id uint64
|
||||||
|
// dimHash is a hash of the label names (preset and variable) and the
|
||||||
|
// Help string. Each Desc with the same fqName must have the same
|
||||||
|
// dimHash.
|
||||||
|
dimHash uint64
|
||||||
|
// err is an error that occurred during construction. It is reported on
|
||||||
|
// registration time.
|
||||||
|
err error
|
||||||
|
}
|
||||||
|
|
||||||
|
// DescOpt allows setting optional fields for NewDesc.
|
||||||
|
type DescOpt func(*Desc)
|
||||||
|
|
||||||
|
// WithUnit sets the unit for a Desc.
|
||||||
|
func WithUnit(unit string) DescOpt {
|
||||||
|
return func(d *Desc) {
|
||||||
|
d.unit = unit
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewDesc allocates and initializes a new Desc. Errors are recorded in the Desc
|
||||||
|
// and will be reported on registration time. variableLabels and constLabels can
|
||||||
|
// be nil if no such labels should be set. fqName must not be empty.
|
||||||
|
//
|
||||||
|
// variableLabels only contain the label names. Their label values are variable
|
||||||
|
// and therefore not part of the Desc. (They are managed within the Metric.)
|
||||||
|
//
|
||||||
|
// For constLabels, the label values are constant. Therefore, they are fully
|
||||||
|
// specified in the Desc. See the Collector example for a usage pattern.
|
||||||
|
func NewDesc(fqName, help string, variableLabels []string, constLabels Labels) *Desc {
|
||||||
|
return V2.NewDesc(fqName, help, UnconstrainedLabels(variableLabels), constLabels)
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewDesc allocates and initializes a new Desc. Errors are recorded in the Desc
|
||||||
|
// and will be reported on registration time. variableLabels and constLabels can
|
||||||
|
// be nil if no such labels should be set. fqName must not be empty.
|
||||||
|
//
|
||||||
|
// variableLabels only contain the label names and normalization functions. Their
|
||||||
|
// label values are variable and therefore not part of the Desc. (They are managed
|
||||||
|
// within the Metric.)
|
||||||
|
//
|
||||||
|
// For constLabels, the label values are constant. Therefore, they are fully
|
||||||
|
// specified in the Desc. See the Collector example for a usage pattern.
|
||||||
|
func (v2) NewDesc(fqName, help string, variableLabels ConstrainableLabels, constLabels Labels, opts ...DescOpt) *Desc {
|
||||||
|
d := &Desc{
|
||||||
|
fqName: fqName,
|
||||||
|
help: help,
|
||||||
|
variableLabels: variableLabels.compile(),
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, opt := range opts {
|
||||||
|
opt(d)
|
||||||
|
}
|
||||||
|
if !model.UTF8Validation.IsValidMetricName(fqName) {
|
||||||
|
d.err = fmt.Errorf("%q is not a valid metric name", fqName)
|
||||||
|
return d
|
||||||
|
}
|
||||||
|
// labelValues contains the label values of const labels (in order of
|
||||||
|
// their sorted label names) plus the fqName (at position 0).
|
||||||
|
labelValues := make([]string, 1, len(constLabels)+1)
|
||||||
|
labelValues[0] = fqName
|
||||||
|
labelNames := make([]string, 0, len(constLabels)+len(d.variableLabels.names))
|
||||||
|
labelNameSet := map[string]struct{}{}
|
||||||
|
// First add only the const label names and sort them...
|
||||||
|
for labelName := range constLabels {
|
||||||
|
if !checkLabelName(labelName) {
|
||||||
|
d.err = fmt.Errorf("%q is not a valid label name for metric %q", labelName, fqName)
|
||||||
|
return d
|
||||||
|
}
|
||||||
|
labelNames = append(labelNames, labelName)
|
||||||
|
labelNameSet[labelName] = struct{}{}
|
||||||
|
}
|
||||||
|
sort.Strings(labelNames)
|
||||||
|
// ... so that we can now add const label values in the order of their names.
|
||||||
|
for _, labelName := range labelNames {
|
||||||
|
labelValues = append(labelValues, constLabels[labelName])
|
||||||
|
}
|
||||||
|
// Validate the const label values. They can't have a wrong cardinality, so
|
||||||
|
// use in len(labelValues) as expectedNumberOfValues.
|
||||||
|
if err := validateLabelValues(labelValues, len(labelValues)); err != nil {
|
||||||
|
d.err = err
|
||||||
|
return d
|
||||||
|
}
|
||||||
|
// Now add the variable label names, but prefix them with something that
|
||||||
|
// cannot be in a regular label name. That prevents matching the label
|
||||||
|
// dimension with a different mix between preset and variable labels.
|
||||||
|
for _, label := range d.variableLabels.names {
|
||||||
|
if !checkLabelName(label) {
|
||||||
|
d.err = fmt.Errorf("%q is not a valid label name for metric %q", label, fqName)
|
||||||
|
return d
|
||||||
|
}
|
||||||
|
labelNames = append(labelNames, "$"+label)
|
||||||
|
labelNameSet[label] = struct{}{}
|
||||||
|
}
|
||||||
|
if len(labelNames) != len(labelNameSet) {
|
||||||
|
d.err = fmt.Errorf("duplicate label names in constant and variable labels for metric %q", fqName)
|
||||||
|
return d
|
||||||
|
}
|
||||||
|
|
||||||
|
xxh := xxhash.New()
|
||||||
|
for _, val := range labelValues {
|
||||||
|
xxh.WriteString(val)
|
||||||
|
xxh.Write(separatorByteSlice)
|
||||||
|
}
|
||||||
|
d.id = xxh.Sum64()
|
||||||
|
// Sort labelNames so that order doesn't matter for the hash.
|
||||||
|
sort.Strings(labelNames)
|
||||||
|
// Now hash together (in this order) the help string, the unit string and the sorted
|
||||||
|
// label names.
|
||||||
|
xxh.Reset()
|
||||||
|
xxh.WriteString(help)
|
||||||
|
xxh.Write(separatorByteSlice)
|
||||||
|
xxh.WriteString(d.unit)
|
||||||
|
xxh.Write(separatorByteSlice)
|
||||||
|
for _, labelName := range labelNames {
|
||||||
|
xxh.WriteString(labelName)
|
||||||
|
xxh.Write(separatorByteSlice)
|
||||||
|
}
|
||||||
|
d.dimHash = xxh.Sum64()
|
||||||
|
|
||||||
|
d.constLabelPairs = make([]*dto.LabelPair, 0, len(constLabels))
|
||||||
|
for n, v := range constLabels {
|
||||||
|
d.constLabelPairs = append(d.constLabelPairs, &dto.LabelPair{
|
||||||
|
Name: proto.String(n),
|
||||||
|
Value: proto.String(v),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
sort.Sort(internal.LabelPairSorter(d.constLabelPairs))
|
||||||
|
return d
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewInvalidDesc returns an invalid descriptor, i.e. a descriptor with the
|
||||||
|
// provided error set. If a collector returning such a descriptor is registered,
|
||||||
|
// registration will fail with the provided error. NewInvalidDesc can be used by
|
||||||
|
// a Collector to signal inability to describe itself.
|
||||||
|
func NewInvalidDesc(err error) *Desc {
|
||||||
|
return &Desc{
|
||||||
|
err: err,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Err returns an error that occurred during construction, if any.
|
||||||
|
//
|
||||||
|
// Calling this method is optional. It can be used to detect construction
|
||||||
|
// errors early, before invoking other methods on the Desc. If an error is
|
||||||
|
// present, later operations may not behave as expected.
|
||||||
|
func (d *Desc) Err() error {
|
||||||
|
return d.err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (d *Desc) String() string {
|
||||||
|
lpStrings := make([]string, 0, len(d.constLabelPairs))
|
||||||
|
for _, lp := range d.constLabelPairs {
|
||||||
|
lpStrings = append(
|
||||||
|
lpStrings,
|
||||||
|
fmt.Sprintf("%s=%q", lp.GetName(), lp.GetValue()),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
vlStrings := []string{}
|
||||||
|
if d.variableLabels != nil {
|
||||||
|
vlStrings = make([]string, 0, len(d.variableLabels.names))
|
||||||
|
for _, vl := range d.variableLabels.names {
|
||||||
|
if fn, ok := d.variableLabels.labelConstraints[vl]; ok && fn != nil {
|
||||||
|
vlStrings = append(vlStrings, fmt.Sprintf("c(%s)", vl))
|
||||||
|
} else {
|
||||||
|
vlStrings = append(vlStrings, vl)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return fmt.Sprintf(
|
||||||
|
"Desc{fqName: %q, help: %q, unit: %q, constLabels: {%s}, variableLabels: {%s}}",
|
||||||
|
d.fqName,
|
||||||
|
d.help,
|
||||||
|
d.unit,
|
||||||
|
strings.Join(lpStrings, ","),
|
||||||
|
strings.Join(vlStrings, ","),
|
||||||
|
)
|
||||||
|
}
|
||||||
210
vendor/github.com/prometheus/client_golang/prometheus/doc.go
generated
vendored
Normal file
210
vendor/github.com/prometheus/client_golang/prometheus/doc.go
generated
vendored
Normal file
@ -0,0 +1,210 @@
|
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
// Package prometheus is the core instrumentation package. It provides metrics
|
||||||
|
// primitives to instrument code for monitoring. It also offers a registry for
|
||||||
|
// metrics. Sub-packages allow to expose the registered metrics via HTTP
|
||||||
|
// (package promhttp) or push them to a Pushgateway (package push). There is
|
||||||
|
// also a sub-package promauto, which provides metrics constructors with
|
||||||
|
// automatic registration.
|
||||||
|
//
|
||||||
|
// All exported functions and methods are safe to be used concurrently unless
|
||||||
|
// specified otherwise.
|
||||||
|
//
|
||||||
|
// # A Basic Example
|
||||||
|
//
|
||||||
|
// As a starting point, a very basic usage example:
|
||||||
|
//
|
||||||
|
// package main
|
||||||
|
//
|
||||||
|
// import (
|
||||||
|
// "log"
|
||||||
|
// "net/http"
|
||||||
|
//
|
||||||
|
// "github.com/prometheus/client_golang/prometheus"
|
||||||
|
// "github.com/prometheus/client_golang/prometheus/promhttp"
|
||||||
|
// )
|
||||||
|
//
|
||||||
|
// type metrics struct {
|
||||||
|
// cpuTemp prometheus.Gauge
|
||||||
|
// hdFailures *prometheus.CounterVec
|
||||||
|
// }
|
||||||
|
//
|
||||||
|
// func NewMetrics(reg prometheus.Registerer) *metrics {
|
||||||
|
// m := &metrics{
|
||||||
|
// cpuTemp: prometheus.NewGauge(prometheus.GaugeOpts{
|
||||||
|
// Name: "cpu_temperature_celsius",
|
||||||
|
// Help: "Current temperature of the CPU.",
|
||||||
|
// }),
|
||||||
|
// hdFailures: prometheus.NewCounterVec(
|
||||||
|
// prometheus.CounterOpts{
|
||||||
|
// Name: "hd_errors_total",
|
||||||
|
// Help: "Number of hard-disk errors.",
|
||||||
|
// },
|
||||||
|
// []string{"device"},
|
||||||
|
// ),
|
||||||
|
// }
|
||||||
|
// reg.MustRegister(m.cpuTemp)
|
||||||
|
// reg.MustRegister(m.hdFailures)
|
||||||
|
// return m
|
||||||
|
// }
|
||||||
|
//
|
||||||
|
// func main() {
|
||||||
|
// // Create a non-global registry.
|
||||||
|
// reg := prometheus.NewRegistry()
|
||||||
|
//
|
||||||
|
// // Create new metrics and register them using the custom registry.
|
||||||
|
// m := NewMetrics(reg)
|
||||||
|
// // Set values for the new created metrics.
|
||||||
|
// m.cpuTemp.Set(65.3)
|
||||||
|
// m.hdFailures.With(prometheus.Labels{"device":"/dev/sda"}).Inc()
|
||||||
|
//
|
||||||
|
// // Expose metrics and custom registry via an HTTP server
|
||||||
|
// // using the HandleFor function. "/metrics" is the usual endpoint for that.
|
||||||
|
// http.Handle("/metrics", promhttp.HandlerFor(reg, promhttp.HandlerOpts{Registry: reg}))
|
||||||
|
// log.Fatal(http.ListenAndServe(":8080", nil))
|
||||||
|
// }
|
||||||
|
//
|
||||||
|
// This is a complete program that exports two metrics, a Gauge and a Counter,
|
||||||
|
// the latter with a label attached to turn it into a (one-dimensional) vector.
|
||||||
|
// It register the metrics using a custom registry and exposes them via an HTTP server
|
||||||
|
// on the /metrics endpoint.
|
||||||
|
//
|
||||||
|
// # Metrics
|
||||||
|
//
|
||||||
|
// The number of exported identifiers in this package might appear a bit
|
||||||
|
// overwhelming. However, in addition to the basic plumbing shown in the example
|
||||||
|
// above, you only need to understand the different metric types and their
|
||||||
|
// vector versions for basic usage. Furthermore, if you are not concerned with
|
||||||
|
// fine-grained control of when and how to register metrics with the registry,
|
||||||
|
// have a look at the promauto package, which will effectively allow you to
|
||||||
|
// ignore registration altogether in simple cases.
|
||||||
|
//
|
||||||
|
// Above, you have already touched the Counter and the Gauge. There are two more
|
||||||
|
// advanced metric types: the Summary and Histogram. A more thorough description
|
||||||
|
// of those four metric types can be found in the Prometheus docs:
|
||||||
|
// https://prometheus.io/docs/concepts/metric_types/
|
||||||
|
//
|
||||||
|
// In addition to the fundamental metric types Gauge, Counter, Summary, and
|
||||||
|
// Histogram, a very important part of the Prometheus data model is the
|
||||||
|
// partitioning of samples along dimensions called labels, which results in
|
||||||
|
// metric vectors. The fundamental types are GaugeVec, CounterVec, SummaryVec,
|
||||||
|
// and HistogramVec.
|
||||||
|
//
|
||||||
|
// While only the fundamental metric types implement the Metric interface, both
|
||||||
|
// the metrics and their vector versions implement the Collector interface. A
|
||||||
|
// Collector manages the collection of a number of Metrics, but for convenience,
|
||||||
|
// a Metric can also “collect itself”. Note that Gauge, Counter, Summary, and
|
||||||
|
// Histogram are interfaces themselves while GaugeVec, CounterVec, SummaryVec,
|
||||||
|
// and HistogramVec are not.
|
||||||
|
//
|
||||||
|
// To create instances of Metrics and their vector versions, you need a suitable
|
||||||
|
// …Opts struct, i.e. GaugeOpts, CounterOpts, SummaryOpts, or HistogramOpts.
|
||||||
|
//
|
||||||
|
// # Custom Collectors and constant Metrics
|
||||||
|
//
|
||||||
|
// While you could create your own implementations of Metric, most likely you
|
||||||
|
// will only ever implement the Collector interface on your own. At a first
|
||||||
|
// glance, a custom Collector seems handy to bundle Metrics for common
|
||||||
|
// registration (with the prime example of the different metric vectors above,
|
||||||
|
// which bundle all the metrics of the same name but with different labels).
|
||||||
|
//
|
||||||
|
// There is a more involved use case, too: If you already have metrics
|
||||||
|
// available, created outside of the Prometheus context, you don't need the
|
||||||
|
// interface of the various Metric types. You essentially want to mirror the
|
||||||
|
// existing numbers into Prometheus Metrics during collection. An own
|
||||||
|
// implementation of the Collector interface is perfect for that. You can create
|
||||||
|
// Metric instances “on the fly” using NewConstMetric, NewConstHistogram, and
|
||||||
|
// NewConstSummary (and their respective Must… versions). NewConstMetric is used
|
||||||
|
// for all metric types with just a float64 as their value: Counter, Gauge, and
|
||||||
|
// a special “type” called Untyped. Use the latter if you are not sure if the
|
||||||
|
// mirrored metric is a Counter or a Gauge. Creation of the Metric instance
|
||||||
|
// happens in the Collect method. The Describe method has to return separate
|
||||||
|
// Desc instances, representative of the “throw-away” metrics to be created
|
||||||
|
// later. NewDesc comes in handy to create those Desc instances. Alternatively,
|
||||||
|
// you could return no Desc at all, which will mark the Collector “unchecked”.
|
||||||
|
// No checks are performed at registration time, but metric consistency will
|
||||||
|
// still be ensured at scrape time, i.e. any inconsistencies will lead to scrape
|
||||||
|
// errors. Thus, with unchecked Collectors, the responsibility to not collect
|
||||||
|
// metrics that lead to inconsistencies in the total scrape result lies with the
|
||||||
|
// implementer of the Collector. While this is not a desirable state, it is
|
||||||
|
// sometimes necessary. The typical use case is a situation where the exact
|
||||||
|
// metrics to be returned by a Collector cannot be predicted at registration
|
||||||
|
// time, but the implementer has sufficient knowledge of the whole system to
|
||||||
|
// guarantee metric consistency.
|
||||||
|
//
|
||||||
|
// The Collector example illustrates the use case. You can also look at the
|
||||||
|
// source code of the processCollector (mirroring process metrics), the
|
||||||
|
// goCollector (mirroring Go metrics), or the expvarCollector (mirroring expvar
|
||||||
|
// metrics) as examples that are used in this package itself.
|
||||||
|
//
|
||||||
|
// If you just need to call a function to get a single float value to collect as
|
||||||
|
// a metric, GaugeFunc, CounterFunc, or UntypedFunc might be interesting
|
||||||
|
// shortcuts.
|
||||||
|
//
|
||||||
|
// # Advanced Uses of the Registry
|
||||||
|
//
|
||||||
|
// While MustRegister is the by far most common way of registering a Collector,
|
||||||
|
// sometimes you might want to handle the errors the registration might cause.
|
||||||
|
// As suggested by the name, MustRegister panics if an error occurs. With the
|
||||||
|
// Register function, the error is returned and can be handled.
|
||||||
|
//
|
||||||
|
// An error is returned if the registered Collector is incompatible or
|
||||||
|
// inconsistent with already registered metrics. The registry aims for
|
||||||
|
// consistency of the collected metrics according to the Prometheus data model.
|
||||||
|
// Inconsistencies are ideally detected at registration time, not at collect
|
||||||
|
// time. The former will usually be detected at start-up time of a program,
|
||||||
|
// while the latter will only happen at scrape time, possibly not even on the
|
||||||
|
// first scrape if the inconsistency only becomes relevant later. That is the
|
||||||
|
// main reason why a Collector and a Metric have to describe themselves to the
|
||||||
|
// registry.
|
||||||
|
//
|
||||||
|
// So far, everything we did operated on the so-called default registry, as it
|
||||||
|
// can be found in the global DefaultRegisterer variable. With NewRegistry, you
|
||||||
|
// can create a custom registry, or you can even implement the Registerer or
|
||||||
|
// Gatherer interfaces yourself. The methods Register and Unregister work in the
|
||||||
|
// same way on a custom registry as the global functions Register and Unregister
|
||||||
|
// on the default registry.
|
||||||
|
//
|
||||||
|
// There are a number of uses for custom registries: You can use registries with
|
||||||
|
// special properties, see NewPedanticRegistry. You can avoid global state, as
|
||||||
|
// it is imposed by the DefaultRegisterer. You can use multiple registries at
|
||||||
|
// the same time to expose different metrics in different ways. You can use
|
||||||
|
// separate registries for testing purposes.
|
||||||
|
//
|
||||||
|
// Also note that the DefaultRegisterer comes registered with a Collector for Go
|
||||||
|
// runtime metrics (via NewGoCollector) and a Collector for process metrics (via
|
||||||
|
// NewProcessCollector). With a custom registry, you are in control and decide
|
||||||
|
// yourself about the Collectors to register.
|
||||||
|
//
|
||||||
|
// # HTTP Exposition
|
||||||
|
//
|
||||||
|
// The Registry implements the Gatherer interface. The caller of the Gather
|
||||||
|
// method can then expose the gathered metrics in some way. Usually, the metrics
|
||||||
|
// are served via HTTP on the /metrics endpoint. That's happening in the example
|
||||||
|
// above. The tools to expose metrics via HTTP are in the promhttp sub-package.
|
||||||
|
//
|
||||||
|
// # Pushing to the Pushgateway
|
||||||
|
//
|
||||||
|
// Function for pushing to the Pushgateway can be found in the push sub-package.
|
||||||
|
//
|
||||||
|
// # Graphite Bridge
|
||||||
|
//
|
||||||
|
// Functions and examples to push metrics from a Gatherer to Graphite can be
|
||||||
|
// found in the graphite sub-package.
|
||||||
|
//
|
||||||
|
// # Other Means of Exposition
|
||||||
|
//
|
||||||
|
// More ways of exposing metrics can easily be added by following the approaches
|
||||||
|
// of the existing implementations.
|
||||||
|
package prometheus
|
||||||
86
vendor/github.com/prometheus/client_golang/prometheus/expvar_collector.go
generated
vendored
Normal file
86
vendor/github.com/prometheus/client_golang/prometheus/expvar_collector.go
generated
vendored
Normal file
@ -0,0 +1,86 @@
|
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"expvar"
|
||||||
|
)
|
||||||
|
|
||||||
|
type expvarCollector struct {
|
||||||
|
exports map[string]*Desc
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewExpvarCollector is the obsolete version of collectors.NewExpvarCollector.
|
||||||
|
// See there for documentation.
|
||||||
|
//
|
||||||
|
// Deprecated: Use collectors.NewExpvarCollector instead.
|
||||||
|
func NewExpvarCollector(exports map[string]*Desc) Collector {
|
||||||
|
return &expvarCollector{
|
||||||
|
exports: exports,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Describe implements Collector.
|
||||||
|
func (e *expvarCollector) Describe(ch chan<- *Desc) {
|
||||||
|
for _, desc := range e.exports {
|
||||||
|
ch <- desc
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Collect implements Collector.
|
||||||
|
func (e *expvarCollector) Collect(ch chan<- Metric) {
|
||||||
|
for name, desc := range e.exports {
|
||||||
|
var m Metric
|
||||||
|
expVar := expvar.Get(name)
|
||||||
|
if expVar == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
var v any
|
||||||
|
labels := make([]string, len(desc.variableLabels.names))
|
||||||
|
if err := json.Unmarshal([]byte(expVar.String()), &v); err != nil {
|
||||||
|
ch <- NewInvalidMetric(desc, err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
var processValue func(v any, i int)
|
||||||
|
processValue = func(v any, i int) {
|
||||||
|
if i >= len(labels) {
|
||||||
|
copiedLabels := append(make([]string, 0, len(labels)), labels...)
|
||||||
|
switch v := v.(type) {
|
||||||
|
case float64:
|
||||||
|
m = MustNewConstMetric(desc, UntypedValue, v, copiedLabels...)
|
||||||
|
case bool:
|
||||||
|
if v {
|
||||||
|
m = MustNewConstMetric(desc, UntypedValue, 1, copiedLabels...)
|
||||||
|
} else {
|
||||||
|
m = MustNewConstMetric(desc, UntypedValue, 0, copiedLabels...)
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ch <- m
|
||||||
|
return
|
||||||
|
}
|
||||||
|
vm, ok := v.(map[string]any)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
for lv, val := range vm {
|
||||||
|
labels[i] = lv
|
||||||
|
processValue(val, i+1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
processValue(v, 0)
|
||||||
|
}
|
||||||
|
}
|
||||||
42
vendor/github.com/prometheus/client_golang/prometheus/fnv.go
generated
vendored
Normal file
42
vendor/github.com/prometheus/client_golang/prometheus/fnv.go
generated
vendored
Normal file
@ -0,0 +1,42 @@
|
|||||||
|
// Copyright 2018 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus
|
||||||
|
|
||||||
|
// Inline and byte-free variant of hash/fnv's fnv64a.
|
||||||
|
|
||||||
|
const (
|
||||||
|
offset64 = 14695981039346656037
|
||||||
|
prime64 = 1099511628211
|
||||||
|
)
|
||||||
|
|
||||||
|
// hashNew initializies a new fnv64a hash value.
|
||||||
|
func hashNew() uint64 {
|
||||||
|
return offset64
|
||||||
|
}
|
||||||
|
|
||||||
|
// hashAdd adds a string to a fnv64a hash value, returning the updated hash.
|
||||||
|
func hashAdd(h uint64, s string) uint64 {
|
||||||
|
for i := 0; i < len(s); i++ {
|
||||||
|
h ^= uint64(s[i])
|
||||||
|
h *= prime64
|
||||||
|
}
|
||||||
|
return h
|
||||||
|
}
|
||||||
|
|
||||||
|
// hashAddByte adds a byte to a fnv64a hash value, returning the updated hash.
|
||||||
|
func hashAddByte(h uint64, b byte) uint64 {
|
||||||
|
h ^= uint64(b)
|
||||||
|
h *= prime64
|
||||||
|
return h
|
||||||
|
}
|
||||||
314
vendor/github.com/prometheus/client_golang/prometheus/gauge.go
generated
vendored
Normal file
314
vendor/github.com/prometheus/client_golang/prometheus/gauge.go
generated
vendored
Normal file
@ -0,0 +1,314 @@
|
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus
|
||||||
|
|
||||||
|
import (
|
||||||
|
"math"
|
||||||
|
"sync/atomic"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
dto "github.com/prometheus/client_model/go"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Gauge is a Metric that represents a single numerical value that can
|
||||||
|
// arbitrarily go up and down.
|
||||||
|
//
|
||||||
|
// A Gauge is typically used for measured values like temperatures or current
|
||||||
|
// memory usage, but also "counts" that can go up and down, like the number of
|
||||||
|
// running goroutines.
|
||||||
|
//
|
||||||
|
// To create Gauge instances, use NewGauge.
|
||||||
|
type Gauge interface {
|
||||||
|
Metric
|
||||||
|
Collector
|
||||||
|
|
||||||
|
// Set sets the Gauge to an arbitrary value.
|
||||||
|
Set(float64)
|
||||||
|
// Inc increments the Gauge by 1. Use Add to increment it by arbitrary
|
||||||
|
// values.
|
||||||
|
Inc()
|
||||||
|
// Dec decrements the Gauge by 1. Use Sub to decrement it by arbitrary
|
||||||
|
// values.
|
||||||
|
Dec()
|
||||||
|
// Add adds the given value to the Gauge. (The value can be negative,
|
||||||
|
// resulting in a decrease of the Gauge.)
|
||||||
|
Add(float64)
|
||||||
|
// Sub subtracts the given value from the Gauge. (The value can be
|
||||||
|
// negative, resulting in an increase of the Gauge.)
|
||||||
|
Sub(float64)
|
||||||
|
|
||||||
|
// SetToCurrentTime sets the Gauge to the current Unix time in seconds.
|
||||||
|
SetToCurrentTime()
|
||||||
|
}
|
||||||
|
|
||||||
|
// GaugeOpts is an alias for Opts. See there for doc comments.
|
||||||
|
type GaugeOpts Opts
|
||||||
|
|
||||||
|
// GaugeVecOpts bundles the options to create a GaugeVec metric.
|
||||||
|
// It is mandatory to set GaugeOpts, see there for mandatory fields. VariableLabels
|
||||||
|
// is optional and can safely be left to its default value.
|
||||||
|
type GaugeVecOpts struct {
|
||||||
|
GaugeOpts
|
||||||
|
|
||||||
|
// VariableLabels are used to partition the metric vector by the given set
|
||||||
|
// of labels. Each label value will be constrained with the optional Constraint
|
||||||
|
// function, if provided.
|
||||||
|
VariableLabels ConstrainableLabels
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewGauge creates a new Gauge based on the provided GaugeOpts.
|
||||||
|
//
|
||||||
|
// The returned implementation is optimized for a fast Set method. If you have a
|
||||||
|
// choice for managing the value of a Gauge via Set vs. Inc/Dec/Add/Sub, pick
|
||||||
|
// the former. For example, the Inc method of the returned Gauge is slower than
|
||||||
|
// the Inc method of a Counter returned by NewCounter. This matches the typical
|
||||||
|
// scenarios for Gauges and Counters, where the former tends to be Set-heavy and
|
||||||
|
// the latter Inc-heavy.
|
||||||
|
func NewGauge(opts GaugeOpts) Gauge {
|
||||||
|
desc := V2.NewDesc(
|
||||||
|
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name),
|
||||||
|
opts.Help,
|
||||||
|
UnconstrainedLabels(nil),
|
||||||
|
opts.ConstLabels,
|
||||||
|
WithUnit(opts.Unit),
|
||||||
|
)
|
||||||
|
result := &gauge{desc: desc, labelPairs: desc.constLabelPairs}
|
||||||
|
result.init(result) // Init self-collection.
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|
||||||
|
type gauge struct {
|
||||||
|
// valBits contains the bits of the represented float64 value. It has
|
||||||
|
// to go first in the struct to guarantee alignment for atomic
|
||||||
|
// operations. http://golang.org/pkg/sync/atomic/#pkg-note-BUG
|
||||||
|
valBits uint64
|
||||||
|
|
||||||
|
selfCollector
|
||||||
|
|
||||||
|
desc *Desc
|
||||||
|
labelPairs []*dto.LabelPair
|
||||||
|
}
|
||||||
|
|
||||||
|
func (g *gauge) Desc() *Desc {
|
||||||
|
return g.desc
|
||||||
|
}
|
||||||
|
|
||||||
|
func (g *gauge) Set(val float64) {
|
||||||
|
atomic.StoreUint64(&g.valBits, math.Float64bits(val))
|
||||||
|
}
|
||||||
|
|
||||||
|
func (g *gauge) SetToCurrentTime() {
|
||||||
|
g.Set(float64(time.Now().UnixNano()) / 1e9)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (g *gauge) Inc() {
|
||||||
|
g.Add(1)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (g *gauge) Dec() {
|
||||||
|
g.Add(-1)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (g *gauge) Add(val float64) {
|
||||||
|
for {
|
||||||
|
oldBits := atomic.LoadUint64(&g.valBits)
|
||||||
|
newBits := math.Float64bits(math.Float64frombits(oldBits) + val)
|
||||||
|
if atomic.CompareAndSwapUint64(&g.valBits, oldBits, newBits) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (g *gauge) Sub(val float64) {
|
||||||
|
g.Add(val * -1)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (g *gauge) Write(out *dto.Metric) error {
|
||||||
|
val := math.Float64frombits(atomic.LoadUint64(&g.valBits))
|
||||||
|
return populateMetric(GaugeValue, val, g.labelPairs, nil, out, nil)
|
||||||
|
}
|
||||||
|
|
||||||
|
// GaugeVec is a Collector that bundles a set of Gauges that all share the same
|
||||||
|
// Desc, but have different values for their variable labels. This is used if
|
||||||
|
// you want to count the same thing partitioned by various dimensions
|
||||||
|
// (e.g. number of operations queued, partitioned by user and operation
|
||||||
|
// type). Create instances with NewGaugeVec.
|
||||||
|
type GaugeVec struct {
|
||||||
|
*MetricVec
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewGaugeVec creates a new GaugeVec based on the provided GaugeOpts and
|
||||||
|
// partitioned by the given label names.
|
||||||
|
func NewGaugeVec(opts GaugeOpts, labelNames []string) *GaugeVec {
|
||||||
|
return V2.NewGaugeVec(GaugeVecOpts{
|
||||||
|
GaugeOpts: opts,
|
||||||
|
VariableLabels: UnconstrainedLabels(labelNames),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewGaugeVec creates a new GaugeVec based on the provided GaugeVecOpts.
|
||||||
|
func (v2) NewGaugeVec(opts GaugeVecOpts) *GaugeVec {
|
||||||
|
desc := V2.NewDesc(
|
||||||
|
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name),
|
||||||
|
opts.Help,
|
||||||
|
opts.VariableLabels,
|
||||||
|
opts.ConstLabels,
|
||||||
|
WithUnit(opts.Unit),
|
||||||
|
)
|
||||||
|
return &GaugeVec{
|
||||||
|
MetricVec: NewMetricVec(desc, func(lvs ...string) Metric {
|
||||||
|
if len(lvs) != len(desc.variableLabels.names) {
|
||||||
|
panic(makeInconsistentCardinalityError(desc.fqName, desc.variableLabels.names, lvs))
|
||||||
|
}
|
||||||
|
result := &gauge{desc: desc, labelPairs: MakeLabelPairs(desc, lvs)}
|
||||||
|
result.init(result) // Init self-collection.
|
||||||
|
return result
|
||||||
|
}),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetMetricWithLabelValues returns the Gauge for the given slice of label
|
||||||
|
// values (same order as the variable labels in Desc). If that combination of
|
||||||
|
// label values is accessed for the first time, a new Gauge is created.
|
||||||
|
//
|
||||||
|
// It is possible to call this method without using the returned Gauge to only
|
||||||
|
// create the new Gauge but leave it at its starting value 0. See also the
|
||||||
|
// SummaryVec example.
|
||||||
|
//
|
||||||
|
// Keeping the Gauge for later use is possible (and should be considered if
|
||||||
|
// performance is critical), but keep in mind that Reset, DeleteLabelValues and
|
||||||
|
// Delete can be used to delete the Gauge from the GaugeVec. In that case, the
|
||||||
|
// Gauge will still exist, but it will not be exported anymore, even if a
|
||||||
|
// Gauge with the same label values is created later. See also the CounterVec
|
||||||
|
// example.
|
||||||
|
//
|
||||||
|
// An error is returned if the number of label values is not the same as the
|
||||||
|
// number of variable labels in Desc (minus any curried labels).
|
||||||
|
//
|
||||||
|
// Note that for more than one label value, this method is prone to mistakes
|
||||||
|
// caused by an incorrect order of arguments. Consider GetMetricWith(Labels) as
|
||||||
|
// an alternative to avoid that type of mistake. For higher label numbers, the
|
||||||
|
// latter has a much more readable (albeit more verbose) syntax, but it comes
|
||||||
|
// with a performance overhead (for creating and processing the Labels map).
|
||||||
|
func (v *GaugeVec) GetMetricWithLabelValues(lvs ...string) (Gauge, error) {
|
||||||
|
metric, err := v.MetricVec.GetMetricWithLabelValues(lvs...)
|
||||||
|
if metric != nil {
|
||||||
|
return metric.(Gauge), err
|
||||||
|
}
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetMetricWith returns the Gauge for the given Labels map (the label names
|
||||||
|
// must match those of the variable labels in Desc). If that label map is
|
||||||
|
// accessed for the first time, a new Gauge is created. Implications of
|
||||||
|
// creating a Gauge without using it and keeping the Gauge for later use are
|
||||||
|
// the same as for GetMetricWithLabelValues.
|
||||||
|
//
|
||||||
|
// An error is returned if the number and names of the Labels are inconsistent
|
||||||
|
// with those of the variable labels in Desc (minus any curried labels).
|
||||||
|
//
|
||||||
|
// This method is used for the same purpose as
|
||||||
|
// GetMetricWithLabelValues(...string). See there for pros and cons of the two
|
||||||
|
// methods.
|
||||||
|
func (v *GaugeVec) GetMetricWith(labels Labels) (Gauge, error) {
|
||||||
|
metric, err := v.MetricVec.GetMetricWith(labels)
|
||||||
|
if metric != nil {
|
||||||
|
return metric.(Gauge), err
|
||||||
|
}
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// WithLabelValues works as GetMetricWithLabelValues, but panics where
|
||||||
|
// GetMetricWithLabelValues would have returned an error. Not returning an
|
||||||
|
// error allows shortcuts like
|
||||||
|
//
|
||||||
|
// myVec.WithLabelValues("404", "GET").Add(42)
|
||||||
|
func (v *GaugeVec) WithLabelValues(lvs ...string) Gauge {
|
||||||
|
g, err := v.GetMetricWithLabelValues(lvs...)
|
||||||
|
if err != nil {
|
||||||
|
panic(err)
|
||||||
|
}
|
||||||
|
return g
|
||||||
|
}
|
||||||
|
|
||||||
|
// With works as GetMetricWith, but panics where GetMetricWithLabels would have
|
||||||
|
// returned an error. Not returning an error allows shortcuts like
|
||||||
|
//
|
||||||
|
// myVec.With(prometheus.Labels{"code": "404", "method": "GET"}).Add(42)
|
||||||
|
func (v *GaugeVec) With(labels Labels) Gauge {
|
||||||
|
g, err := v.GetMetricWith(labels)
|
||||||
|
if err != nil {
|
||||||
|
panic(err)
|
||||||
|
}
|
||||||
|
return g
|
||||||
|
}
|
||||||
|
|
||||||
|
// CurryWith returns a vector curried with the provided labels, i.e. the
|
||||||
|
// returned vector has those labels pre-set for all labeled operations performed
|
||||||
|
// on it. The cardinality of the curried vector is reduced accordingly. The
|
||||||
|
// order of the remaining labels stays the same (just with the curried labels
|
||||||
|
// taken out of the sequence – which is relevant for the
|
||||||
|
// (GetMetric)WithLabelValues methods). It is possible to curry a curried
|
||||||
|
// vector, but only with labels not yet used for currying before.
|
||||||
|
//
|
||||||
|
// The metrics contained in the GaugeVec are shared between the curried and
|
||||||
|
// uncurried vectors. They are just accessed differently. Curried and uncurried
|
||||||
|
// vectors behave identically in terms of collection. Only one must be
|
||||||
|
// registered with a given registry (usually the uncurried version). The Reset
|
||||||
|
// method deletes all metrics, even if called on a curried vector.
|
||||||
|
func (v *GaugeVec) CurryWith(labels Labels) (*GaugeVec, error) {
|
||||||
|
vec, err := v.MetricVec.CurryWith(labels)
|
||||||
|
if vec != nil {
|
||||||
|
return &GaugeVec{vec}, err
|
||||||
|
}
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// MustCurryWith works as CurryWith but panics where CurryWith would have
|
||||||
|
// returned an error.
|
||||||
|
func (v *GaugeVec) MustCurryWith(labels Labels) *GaugeVec {
|
||||||
|
vec, err := v.CurryWith(labels)
|
||||||
|
if err != nil {
|
||||||
|
panic(err)
|
||||||
|
}
|
||||||
|
return vec
|
||||||
|
}
|
||||||
|
|
||||||
|
// GaugeFunc is a Gauge whose value is determined at collect time by calling a
|
||||||
|
// provided function.
|
||||||
|
//
|
||||||
|
// To create GaugeFunc instances, use NewGaugeFunc.
|
||||||
|
type GaugeFunc interface {
|
||||||
|
Metric
|
||||||
|
Collector
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewGaugeFunc creates a new GaugeFunc based on the provided GaugeOpts. The
|
||||||
|
// value reported is determined by calling the given function from within the
|
||||||
|
// Write method. Take into account that metric collection may happen
|
||||||
|
// concurrently. Therefore, it must be safe to call the provided function
|
||||||
|
// concurrently.
|
||||||
|
//
|
||||||
|
// NewGaugeFunc is a good way to create an “info” style metric with a constant
|
||||||
|
// value of 1. Example:
|
||||||
|
// https://github.com/prometheus/common/blob/8558a5b7db3c84fa38b4766966059a7bd5bfa2ee/version/info.go#L36-L56
|
||||||
|
func NewGaugeFunc(opts GaugeOpts, function func() float64) GaugeFunc {
|
||||||
|
return newValueFunc(V2.NewDesc(
|
||||||
|
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name),
|
||||||
|
opts.Help,
|
||||||
|
UnconstrainedLabels(nil),
|
||||||
|
opts.ConstLabels,
|
||||||
|
WithUnit(opts.Unit),
|
||||||
|
), GaugeValue, function)
|
||||||
|
}
|
||||||
26
vendor/github.com/prometheus/client_golang/prometheus/get_pid.go
generated
vendored
Normal file
26
vendor/github.com/prometheus/client_golang/prometheus/get_pid.go
generated
vendored
Normal file
@ -0,0 +1,26 @@
|
|||||||
|
// Copyright 2015 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
//go:build !js || wasm
|
||||||
|
// +build !js wasm
|
||||||
|
|
||||||
|
package prometheus
|
||||||
|
|
||||||
|
import "os"
|
||||||
|
|
||||||
|
func getPIDFn() func() (int, error) {
|
||||||
|
pid := os.Getpid()
|
||||||
|
return func() (int, error) {
|
||||||
|
return pid, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
23
vendor/github.com/prometheus/client_golang/prometheus/get_pid_gopherjs.go
generated
vendored
Normal file
23
vendor/github.com/prometheus/client_golang/prometheus/get_pid_gopherjs.go
generated
vendored
Normal file
@ -0,0 +1,23 @@
|
|||||||
|
// Copyright 2015 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
//go:build js && !wasm
|
||||||
|
// +build js,!wasm
|
||||||
|
|
||||||
|
package prometheus
|
||||||
|
|
||||||
|
func getPIDFn() func() (int, error) {
|
||||||
|
return func() (int, error) {
|
||||||
|
return 1, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
274
vendor/github.com/prometheus/client_golang/prometheus/go_collector.go
generated
vendored
Normal file
274
vendor/github.com/prometheus/client_golang/prometheus/go_collector.go
generated
vendored
Normal file
@ -0,0 +1,274 @@
|
|||||||
|
// Copyright 2018 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus
|
||||||
|
|
||||||
|
import (
|
||||||
|
"runtime"
|
||||||
|
"runtime/debug"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// goRuntimeMemStats provides the metrics initially provided by runtime.ReadMemStats.
|
||||||
|
// From Go 1.17 those similar (and better) statistics are provided by runtime/metrics, so
|
||||||
|
// while eval closure works on runtime.MemStats, the struct from Go 1.17+ is
|
||||||
|
// populated using runtime/metrics. Those are the defaults we can't alter.
|
||||||
|
func goRuntimeMemStats() memStatsMetrics {
|
||||||
|
return memStatsMetrics{
|
||||||
|
{
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("alloc_bytes"),
|
||||||
|
"Number of bytes allocated in heap and currently in use. Equals to /memory/classes/heap/objects:bytes.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.Alloc) },
|
||||||
|
valType: GaugeValue,
|
||||||
|
}, {
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("alloc_bytes_total"),
|
||||||
|
"Total number of bytes allocated in heap until now, even if released already. Equals to /gc/heap/allocs:bytes.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.TotalAlloc) },
|
||||||
|
valType: CounterValue,
|
||||||
|
}, {
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("sys_bytes"),
|
||||||
|
"Number of bytes obtained from system. Equals to /memory/classes/total:byte.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.Sys) },
|
||||||
|
valType: GaugeValue,
|
||||||
|
}, {
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("mallocs_total"),
|
||||||
|
// TODO(bwplotka): We could add go_memstats_heap_objects, probably useful for discovery. Let's gather more feedback, kind of a waste of bytes for everybody for compatibility reasons to keep both, and we can't really rename/remove useful metric.
|
||||||
|
"Total number of heap objects allocated, both live and gc-ed. Semantically a counter version for go_memstats_heap_objects gauge. Equals to /gc/heap/allocs:objects + /gc/heap/tiny/allocs:objects.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.Mallocs) },
|
||||||
|
valType: CounterValue,
|
||||||
|
}, {
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("frees_total"),
|
||||||
|
"Total number of heap objects frees. Equals to /gc/heap/frees:objects + /gc/heap/tiny/allocs:objects.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.Frees) },
|
||||||
|
valType: CounterValue,
|
||||||
|
}, {
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("heap_alloc_bytes"),
|
||||||
|
"Number of heap bytes allocated and currently in use, same as go_memstats_alloc_bytes. Equals to /memory/classes/heap/objects:bytes.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.HeapAlloc) },
|
||||||
|
valType: GaugeValue,
|
||||||
|
}, {
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("heap_sys_bytes"),
|
||||||
|
"Number of heap bytes obtained from system. Equals to /memory/classes/heap/objects:bytes + /memory/classes/heap/unused:bytes + /memory/classes/heap/released:bytes + /memory/classes/heap/free:bytes.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.HeapSys) },
|
||||||
|
valType: GaugeValue,
|
||||||
|
}, {
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("heap_idle_bytes"),
|
||||||
|
"Number of heap bytes waiting to be used. Equals to /memory/classes/heap/released:bytes + /memory/classes/heap/free:bytes.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.HeapIdle) },
|
||||||
|
valType: GaugeValue,
|
||||||
|
}, {
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("heap_inuse_bytes"),
|
||||||
|
"Number of heap bytes that are in use. Equals to /memory/classes/heap/objects:bytes + /memory/classes/heap/unused:bytes",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.HeapInuse) },
|
||||||
|
valType: GaugeValue,
|
||||||
|
}, {
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("heap_released_bytes"),
|
||||||
|
"Number of heap bytes released to OS. Equals to /memory/classes/heap/released:bytes.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.HeapReleased) },
|
||||||
|
valType: GaugeValue,
|
||||||
|
}, {
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("heap_objects"),
|
||||||
|
"Number of currently allocated objects. Equals to /gc/heap/objects:objects.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.HeapObjects) },
|
||||||
|
valType: GaugeValue,
|
||||||
|
}, {
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("stack_inuse_bytes"),
|
||||||
|
"Number of bytes obtained from system for stack allocator in non-CGO environments. Equals to /memory/classes/heap/stacks:bytes.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.StackInuse) },
|
||||||
|
valType: GaugeValue,
|
||||||
|
}, {
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("stack_sys_bytes"),
|
||||||
|
"Number of bytes obtained from system for stack allocator. Equals to /memory/classes/heap/stacks:bytes + /memory/classes/os-stacks:bytes.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.StackSys) },
|
||||||
|
valType: GaugeValue,
|
||||||
|
}, {
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("mspan_inuse_bytes"),
|
||||||
|
"Number of bytes in use by mspan structures. Equals to /memory/classes/metadata/mspan/inuse:bytes.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.MSpanInuse) },
|
||||||
|
valType: GaugeValue,
|
||||||
|
}, {
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("mspan_sys_bytes"),
|
||||||
|
"Number of bytes used for mspan structures obtained from system. Equals to /memory/classes/metadata/mspan/inuse:bytes + /memory/classes/metadata/mspan/free:bytes.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.MSpanSys) },
|
||||||
|
valType: GaugeValue,
|
||||||
|
}, {
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("mcache_inuse_bytes"),
|
||||||
|
"Number of bytes in use by mcache structures. Equals to /memory/classes/metadata/mcache/inuse:bytes.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.MCacheInuse) },
|
||||||
|
valType: GaugeValue,
|
||||||
|
}, {
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("mcache_sys_bytes"),
|
||||||
|
"Number of bytes used for mcache structures obtained from system. Equals to /memory/classes/metadata/mcache/inuse:bytes + /memory/classes/metadata/mcache/free:bytes.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.MCacheSys) },
|
||||||
|
valType: GaugeValue,
|
||||||
|
}, {
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("buck_hash_sys_bytes"),
|
||||||
|
"Number of bytes used by the profiling bucket hash table. Equals to /memory/classes/profiling/buckets:bytes.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.BuckHashSys) },
|
||||||
|
valType: GaugeValue,
|
||||||
|
}, {
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("gc_sys_bytes"),
|
||||||
|
"Number of bytes used for garbage collection system metadata. Equals to /memory/classes/metadata/other:bytes.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.GCSys) },
|
||||||
|
valType: GaugeValue,
|
||||||
|
}, {
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("other_sys_bytes"),
|
||||||
|
"Number of bytes used for other system allocations. Equals to /memory/classes/other:bytes.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.OtherSys) },
|
||||||
|
valType: GaugeValue,
|
||||||
|
}, {
|
||||||
|
desc: NewDesc(
|
||||||
|
memstatNamespace("next_gc_bytes"),
|
||||||
|
"Number of heap bytes when next garbage collection will take place. Equals to /gc/heap/goal:bytes.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.NextGC) },
|
||||||
|
valType: GaugeValue,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type baseGoCollector struct {
|
||||||
|
goroutinesDesc *Desc
|
||||||
|
threadsDesc *Desc
|
||||||
|
gcDesc *Desc
|
||||||
|
gcLastTimeDesc *Desc
|
||||||
|
goInfoDesc *Desc
|
||||||
|
}
|
||||||
|
|
||||||
|
func newBaseGoCollector() baseGoCollector {
|
||||||
|
return baseGoCollector{
|
||||||
|
goroutinesDesc: NewDesc(
|
||||||
|
"go_goroutines",
|
||||||
|
"Number of goroutines that currently exist.",
|
||||||
|
nil, nil),
|
||||||
|
threadsDesc: NewDesc(
|
||||||
|
"go_threads",
|
||||||
|
"Number of OS threads created.",
|
||||||
|
nil, nil),
|
||||||
|
gcDesc: NewDesc(
|
||||||
|
"go_gc_duration_seconds",
|
||||||
|
"A summary of the wall-time pause (stop-the-world) duration in garbage collection cycles.",
|
||||||
|
nil, nil),
|
||||||
|
gcLastTimeDesc: NewDesc(
|
||||||
|
"go_memstats_last_gc_time_seconds",
|
||||||
|
"Number of seconds since 1970 of last garbage collection.",
|
||||||
|
nil, nil),
|
||||||
|
goInfoDesc: NewDesc(
|
||||||
|
"go_info",
|
||||||
|
"Information about the Go environment.",
|
||||||
|
nil, Labels{"version": runtime.Version()}),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Describe returns all descriptions of the collector.
|
||||||
|
func (c *baseGoCollector) Describe(ch chan<- *Desc) {
|
||||||
|
ch <- c.goroutinesDesc
|
||||||
|
ch <- c.threadsDesc
|
||||||
|
ch <- c.gcDesc
|
||||||
|
ch <- c.gcLastTimeDesc
|
||||||
|
ch <- c.goInfoDesc
|
||||||
|
}
|
||||||
|
|
||||||
|
// Collect returns the current state of all metrics of the collector.
|
||||||
|
func (c *baseGoCollector) Collect(ch chan<- Metric) {
|
||||||
|
ch <- MustNewConstMetric(c.goroutinesDesc, GaugeValue, float64(runtime.NumGoroutine()))
|
||||||
|
|
||||||
|
n := getRuntimeNumThreads()
|
||||||
|
ch <- MustNewConstMetric(c.threadsDesc, GaugeValue, n)
|
||||||
|
|
||||||
|
var stats debug.GCStats
|
||||||
|
stats.PauseQuantiles = make([]time.Duration, 5)
|
||||||
|
debug.ReadGCStats(&stats)
|
||||||
|
|
||||||
|
quantiles := make(map[float64]float64)
|
||||||
|
for idx, pq := range stats.PauseQuantiles[1:] {
|
||||||
|
quantiles[float64(idx+1)/float64(len(stats.PauseQuantiles)-1)] = pq.Seconds()
|
||||||
|
}
|
||||||
|
quantiles[0.0] = stats.PauseQuantiles[0].Seconds()
|
||||||
|
ch <- MustNewConstSummary(c.gcDesc, uint64(stats.NumGC), stats.PauseTotal.Seconds(), quantiles)
|
||||||
|
ch <- MustNewConstMetric(c.gcLastTimeDesc, GaugeValue, float64(stats.LastGC.UnixNano())/1e9)
|
||||||
|
ch <- MustNewConstMetric(c.goInfoDesc, GaugeValue, 1)
|
||||||
|
}
|
||||||
|
|
||||||
|
func memstatNamespace(s string) string {
|
||||||
|
return "go_memstats_" + s
|
||||||
|
}
|
||||||
|
|
||||||
|
// memStatsMetrics provide description, evaluator, runtime/metrics name, and
|
||||||
|
// value type for memstat metrics.
|
||||||
|
type memStatsMetrics []struct {
|
||||||
|
desc *Desc
|
||||||
|
eval func(*runtime.MemStats) float64
|
||||||
|
valType ValueType
|
||||||
|
}
|
||||||
586
vendor/github.com/prometheus/client_golang/prometheus/go_collector_latest.go
generated
vendored
Normal file
586
vendor/github.com/prometheus/client_golang/prometheus/go_collector_latest.go
generated
vendored
Normal file
@ -0,0 +1,586 @@
|
|||||||
|
// Copyright 2021 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
//go:build go1.17
|
||||||
|
// +build go1.17
|
||||||
|
|
||||||
|
package prometheus
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"math"
|
||||||
|
"runtime"
|
||||||
|
"runtime/metrics"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
|
||||||
|
"github.com/prometheus/client_golang/prometheus/internal"
|
||||||
|
|
||||||
|
dto "github.com/prometheus/client_model/go"
|
||||||
|
"google.golang.org/protobuf/proto"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
// constants for strings referenced more than once.
|
||||||
|
goGCHeapTinyAllocsObjects = "/gc/heap/tiny/allocs:objects"
|
||||||
|
goGCHeapAllocsObjects = "/gc/heap/allocs:objects"
|
||||||
|
goGCHeapFreesObjects = "/gc/heap/frees:objects"
|
||||||
|
goGCHeapFreesBytes = "/gc/heap/frees:bytes"
|
||||||
|
goGCHeapAllocsBytes = "/gc/heap/allocs:bytes"
|
||||||
|
goGCHeapObjects = "/gc/heap/objects:objects"
|
||||||
|
goGCHeapGoalBytes = "/gc/heap/goal:bytes"
|
||||||
|
goMemoryClassesTotalBytes = "/memory/classes/total:bytes"
|
||||||
|
goMemoryClassesHeapObjectsBytes = "/memory/classes/heap/objects:bytes"
|
||||||
|
goMemoryClassesHeapUnusedBytes = "/memory/classes/heap/unused:bytes"
|
||||||
|
goMemoryClassesHeapReleasedBytes = "/memory/classes/heap/released:bytes"
|
||||||
|
goMemoryClassesHeapFreeBytes = "/memory/classes/heap/free:bytes"
|
||||||
|
goMemoryClassesHeapStacksBytes = "/memory/classes/heap/stacks:bytes"
|
||||||
|
goMemoryClassesOSStacksBytes = "/memory/classes/os-stacks:bytes"
|
||||||
|
goMemoryClassesMetadataMSpanInuseBytes = "/memory/classes/metadata/mspan/inuse:bytes"
|
||||||
|
goMemoryClassesMetadataMSPanFreeBytes = "/memory/classes/metadata/mspan/free:bytes"
|
||||||
|
goMemoryClassesMetadataMCacheInuseBytes = "/memory/classes/metadata/mcache/inuse:bytes"
|
||||||
|
goMemoryClassesMetadataMCacheFreeBytes = "/memory/classes/metadata/mcache/free:bytes"
|
||||||
|
goMemoryClassesProfilingBucketsBytes = "/memory/classes/profiling/buckets:bytes"
|
||||||
|
goMemoryClassesMetadataOtherBytes = "/memory/classes/metadata/other:bytes"
|
||||||
|
goMemoryClassesOtherBytes = "/memory/classes/other:bytes"
|
||||||
|
)
|
||||||
|
|
||||||
|
// rmNamesForMemStatsMetrics represents runtime/metrics names required to populate goRuntimeMemStats from like logic.
|
||||||
|
var rmNamesForMemStatsMetrics = []string{
|
||||||
|
goGCHeapTinyAllocsObjects,
|
||||||
|
goGCHeapAllocsObjects,
|
||||||
|
goGCHeapFreesObjects,
|
||||||
|
goGCHeapAllocsBytes,
|
||||||
|
goGCHeapObjects,
|
||||||
|
goGCHeapGoalBytes,
|
||||||
|
goMemoryClassesTotalBytes,
|
||||||
|
goMemoryClassesHeapObjectsBytes,
|
||||||
|
goMemoryClassesHeapUnusedBytes,
|
||||||
|
goMemoryClassesHeapReleasedBytes,
|
||||||
|
goMemoryClassesHeapFreeBytes,
|
||||||
|
goMemoryClassesHeapStacksBytes,
|
||||||
|
goMemoryClassesOSStacksBytes,
|
||||||
|
goMemoryClassesMetadataMSpanInuseBytes,
|
||||||
|
goMemoryClassesMetadataMSPanFreeBytes,
|
||||||
|
goMemoryClassesMetadataMCacheInuseBytes,
|
||||||
|
goMemoryClassesMetadataMCacheFreeBytes,
|
||||||
|
goMemoryClassesProfilingBucketsBytes,
|
||||||
|
goMemoryClassesMetadataOtherBytes,
|
||||||
|
goMemoryClassesOtherBytes,
|
||||||
|
}
|
||||||
|
|
||||||
|
func bestEffortLookupRM(lookup []string) []metrics.Description {
|
||||||
|
ret := make([]metrics.Description, 0, len(lookup))
|
||||||
|
for _, rm := range metrics.All() {
|
||||||
|
for _, m := range lookup {
|
||||||
|
if m == rm.Name {
|
||||||
|
ret = append(ret, rm)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ret
|
||||||
|
}
|
||||||
|
|
||||||
|
type goCollector struct {
|
||||||
|
base baseGoCollector
|
||||||
|
|
||||||
|
// mu protects updates to all fields ensuring a consistent
|
||||||
|
// snapshot is always produced by Collect.
|
||||||
|
mu sync.Mutex
|
||||||
|
|
||||||
|
// Contains all samples that have to be retrieved from runtime/metrics (not all of them will be exposed).
|
||||||
|
sampleBuf []metrics.Sample
|
||||||
|
// sampleMap allows lookup for MemStats metrics and runtime/metrics histograms for exact sums.
|
||||||
|
sampleMap map[string]*metrics.Sample
|
||||||
|
|
||||||
|
// rmExposedMetrics represents all runtime/metrics package metrics
|
||||||
|
// that were configured to be exposed.
|
||||||
|
rmExposedMetrics []collectorMetric
|
||||||
|
rmExactSumMapForHist map[string]string
|
||||||
|
|
||||||
|
// With Go 1.17, the runtime/metrics package was introduced.
|
||||||
|
// From that point on, metric names produced by the runtime/metrics
|
||||||
|
// package could be generated from runtime/metrics names. However,
|
||||||
|
// these differ from the old names for the same values.
|
||||||
|
//
|
||||||
|
// This field exists to export the same values under the old names
|
||||||
|
// as well.
|
||||||
|
msMetrics memStatsMetrics
|
||||||
|
msMetricsEnabled bool
|
||||||
|
}
|
||||||
|
|
||||||
|
type rmMetricDesc struct {
|
||||||
|
metrics.Description
|
||||||
|
}
|
||||||
|
|
||||||
|
func matchRuntimeMetricsRules(rules []internal.GoCollectorRule) []rmMetricDesc {
|
||||||
|
var descs []rmMetricDesc
|
||||||
|
for _, d := range metrics.All() {
|
||||||
|
var (
|
||||||
|
deny = true
|
||||||
|
desc rmMetricDesc
|
||||||
|
)
|
||||||
|
|
||||||
|
for _, r := range rules {
|
||||||
|
if !r.Matcher.MatchString(d.Name) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
deny = r.Deny
|
||||||
|
}
|
||||||
|
if deny {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
desc.Description = d
|
||||||
|
descs = append(descs, desc)
|
||||||
|
}
|
||||||
|
return descs
|
||||||
|
}
|
||||||
|
|
||||||
|
func defaultGoCollectorOptions() internal.GoCollectorOptions {
|
||||||
|
return internal.GoCollectorOptions{
|
||||||
|
RuntimeMetricSumForHist: map[string]string{
|
||||||
|
"/gc/heap/allocs-by-size:bytes": goGCHeapAllocsBytes,
|
||||||
|
"/gc/heap/frees-by-size:bytes": goGCHeapFreesBytes,
|
||||||
|
},
|
||||||
|
RuntimeMetricRules: []internal.GoCollectorRule{
|
||||||
|
// Recommended metrics we want by default from runtime/metrics.
|
||||||
|
{Matcher: internal.GoCollectorDefaultRuntimeMetrics},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewGoCollector is the obsolete version of collectors.NewGoCollector.
|
||||||
|
// See there for documentation.
|
||||||
|
//
|
||||||
|
// Deprecated: Use collectors.NewGoCollector instead.
|
||||||
|
func NewGoCollector(opts ...func(o *internal.GoCollectorOptions)) Collector {
|
||||||
|
opt := defaultGoCollectorOptions()
|
||||||
|
for _, o := range opts {
|
||||||
|
o(&opt)
|
||||||
|
}
|
||||||
|
|
||||||
|
exposedDescriptions := matchRuntimeMetricsRules(opt.RuntimeMetricRules)
|
||||||
|
|
||||||
|
// Collect all histogram samples so that we can get their buckets.
|
||||||
|
// The API guarantees that the buckets are always fixed for the lifetime
|
||||||
|
// of the process.
|
||||||
|
var histograms []metrics.Sample
|
||||||
|
for _, d := range exposedDescriptions {
|
||||||
|
if d.Kind == metrics.KindFloat64Histogram {
|
||||||
|
histograms = append(histograms, metrics.Sample{Name: d.Name})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(histograms) > 0 {
|
||||||
|
metrics.Read(histograms)
|
||||||
|
}
|
||||||
|
|
||||||
|
bucketsMap := make(map[string][]float64)
|
||||||
|
for i := range histograms {
|
||||||
|
bucketsMap[histograms[i].Name] = histograms[i].Value.Float64Histogram().Buckets
|
||||||
|
}
|
||||||
|
|
||||||
|
// Generate a collector for each exposed runtime/metrics metric.
|
||||||
|
metricSet := make([]collectorMetric, 0, len(exposedDescriptions))
|
||||||
|
// SampleBuf is used for reading from runtime/metrics.
|
||||||
|
// We are assuming the largest case to have stable pointers for sampleMap purposes.
|
||||||
|
sampleBuf := make([]metrics.Sample, 0, len(exposedDescriptions)+len(opt.RuntimeMetricSumForHist)+len(rmNamesForMemStatsMetrics))
|
||||||
|
sampleMap := make(map[string]*metrics.Sample, len(exposedDescriptions))
|
||||||
|
for _, d := range exposedDescriptions {
|
||||||
|
namespace, subsystem, name, ok := internal.RuntimeMetricsToProm(&d.Description)
|
||||||
|
if !ok {
|
||||||
|
// Just ignore this metric; we can't do anything with it here.
|
||||||
|
// If a user decides to use the latest version of Go, we don't want
|
||||||
|
// to fail here. This condition is tested in TestExpectedRuntimeMetrics.
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
help := attachOriginalName(d.Description.Description, d.Name)
|
||||||
|
|
||||||
|
sampleBuf = append(sampleBuf, metrics.Sample{Name: d.Name})
|
||||||
|
sampleMap[d.Name] = &sampleBuf[len(sampleBuf)-1]
|
||||||
|
|
||||||
|
// Extract unit from the runtime/metrics name (e.g., "/gc/heap/allocs:bytes" -> "bytes")
|
||||||
|
// and sanitize to match Prometheus naming conventions (e.g., "cpu-seconds" -> "cpu_seconds")
|
||||||
|
var unit string
|
||||||
|
if idx := strings.IndexRune(d.Name, ':'); idx >= 0 {
|
||||||
|
unit = d.Name[idx+1:]
|
||||||
|
unit = strings.ReplaceAll(unit, "-", "_")
|
||||||
|
unit = strings.ReplaceAll(unit, "*", "_")
|
||||||
|
unit = strings.ReplaceAll(unit, "/", "_per_")
|
||||||
|
}
|
||||||
|
|
||||||
|
var m collectorMetric
|
||||||
|
if d.Kind == metrics.KindFloat64Histogram {
|
||||||
|
_, hasSum := opt.RuntimeMetricSumForHist[d.Name]
|
||||||
|
m = newBatchHistogram(
|
||||||
|
V2.NewDesc(
|
||||||
|
BuildFQName(namespace, subsystem, name),
|
||||||
|
help,
|
||||||
|
UnconstrainedLabels(nil),
|
||||||
|
nil,
|
||||||
|
WithUnit(unit),
|
||||||
|
),
|
||||||
|
internal.RuntimeMetricsBucketsForUnit(bucketsMap[d.Name], unit),
|
||||||
|
hasSum,
|
||||||
|
)
|
||||||
|
} else if d.Cumulative {
|
||||||
|
m = NewCounter(CounterOpts{
|
||||||
|
Namespace: namespace,
|
||||||
|
Subsystem: subsystem,
|
||||||
|
Name: name,
|
||||||
|
Help: help,
|
||||||
|
Unit: unit,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
} else {
|
||||||
|
m = NewGauge(GaugeOpts{
|
||||||
|
Namespace: namespace,
|
||||||
|
Subsystem: subsystem,
|
||||||
|
Name: name,
|
||||||
|
Help: help,
|
||||||
|
Unit: unit,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
metricSet = append(metricSet, m)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Add exact sum metrics to sampleBuf if not added before.
|
||||||
|
for _, h := range histograms {
|
||||||
|
sumMetric, ok := opt.RuntimeMetricSumForHist[h.Name]
|
||||||
|
if !ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, ok := sampleMap[sumMetric]; ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
sampleBuf = append(sampleBuf, metrics.Sample{Name: sumMetric})
|
||||||
|
sampleMap[sumMetric] = &sampleBuf[len(sampleBuf)-1]
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
msMetrics memStatsMetrics
|
||||||
|
msDescriptions []metrics.Description
|
||||||
|
)
|
||||||
|
|
||||||
|
if !opt.DisableMemStatsLikeMetrics {
|
||||||
|
msMetrics = goRuntimeMemStats()
|
||||||
|
msDescriptions = bestEffortLookupRM(rmNamesForMemStatsMetrics)
|
||||||
|
|
||||||
|
// Check if metric was not exposed before and if not, add to sampleBuf.
|
||||||
|
for _, mdDesc := range msDescriptions {
|
||||||
|
if _, ok := sampleMap[mdDesc.Name]; ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
sampleBuf = append(sampleBuf, metrics.Sample{Name: mdDesc.Name})
|
||||||
|
sampleMap[mdDesc.Name] = &sampleBuf[len(sampleBuf)-1]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return &goCollector{
|
||||||
|
base: newBaseGoCollector(),
|
||||||
|
sampleBuf: sampleBuf,
|
||||||
|
sampleMap: sampleMap,
|
||||||
|
rmExposedMetrics: metricSet,
|
||||||
|
rmExactSumMapForHist: opt.RuntimeMetricSumForHist,
|
||||||
|
msMetrics: msMetrics,
|
||||||
|
msMetricsEnabled: !opt.DisableMemStatsLikeMetrics,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func attachOriginalName(desc, origName string) string {
|
||||||
|
return fmt.Sprintf("%s Sourced from %s.", desc, origName)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Describe returns all descriptions of the collector.
|
||||||
|
func (c *goCollector) Describe(ch chan<- *Desc) {
|
||||||
|
c.base.Describe(ch)
|
||||||
|
for _, i := range c.msMetrics {
|
||||||
|
ch <- i.desc
|
||||||
|
}
|
||||||
|
for _, m := range c.rmExposedMetrics {
|
||||||
|
ch <- m.Desc()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Collect returns the current state of all metrics of the collector.
|
||||||
|
func (c *goCollector) Collect(ch chan<- Metric) {
|
||||||
|
// Collect base non-memory metrics.
|
||||||
|
c.base.Collect(ch)
|
||||||
|
|
||||||
|
if len(c.sampleBuf) == 0 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Collect must be thread-safe, so prevent concurrent use of
|
||||||
|
// sampleBuf elements. Just read into sampleBuf but write all the data
|
||||||
|
// we get into our Metrics or MemStats.
|
||||||
|
//
|
||||||
|
// This lock also ensures that the Metrics we send out are all from
|
||||||
|
// the same updates, ensuring their mutual consistency insofar as
|
||||||
|
// is guaranteed by the runtime/metrics package.
|
||||||
|
//
|
||||||
|
// N.B. This locking is heavy-handed, but Collect is expected to be called
|
||||||
|
// relatively infrequently. Also the core operation here, metrics.Read,
|
||||||
|
// is fast (O(tens of microseconds)) so contention should certainly be
|
||||||
|
// low, though channel operations and any allocations may add to that.
|
||||||
|
c.mu.Lock()
|
||||||
|
defer c.mu.Unlock()
|
||||||
|
|
||||||
|
// Populate runtime/metrics sample buffer.
|
||||||
|
metrics.Read(c.sampleBuf)
|
||||||
|
|
||||||
|
// Collect all our runtime/metrics user chose to expose from sampleBuf (if any).
|
||||||
|
for i, metric := range c.rmExposedMetrics {
|
||||||
|
// We created samples for exposed metrics first in order, so indexes match.
|
||||||
|
sample := c.sampleBuf[i]
|
||||||
|
|
||||||
|
// N.B. switch on concrete type because it's significantly more efficient
|
||||||
|
// than checking for the Counter and Gauge interface implementations. In
|
||||||
|
// this case, we control all the types here.
|
||||||
|
switch m := metric.(type) {
|
||||||
|
case *counter:
|
||||||
|
// Guard against decreases. This should never happen, but a failure
|
||||||
|
// to do so will result in a panic, which is a harsh consequence for
|
||||||
|
// a metrics collection bug.
|
||||||
|
v0, v1 := m.get(), unwrapScalarRMValue(sample.Value)
|
||||||
|
if v1 > v0 {
|
||||||
|
m.Add(unwrapScalarRMValue(sample.Value) - m.get())
|
||||||
|
}
|
||||||
|
m.Collect(ch)
|
||||||
|
case *gauge:
|
||||||
|
m.Set(unwrapScalarRMValue(sample.Value))
|
||||||
|
m.Collect(ch)
|
||||||
|
case *batchHistogram:
|
||||||
|
m.update(sample.Value.Float64Histogram(), c.exactSumFor(sample.Name))
|
||||||
|
m.Collect(ch)
|
||||||
|
default:
|
||||||
|
panic("unexpected metric type")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if c.msMetricsEnabled {
|
||||||
|
// ms is a dummy MemStats that we populate ourselves so that we can
|
||||||
|
// populate the old metrics from it if goMemStatsCollection is enabled.
|
||||||
|
var ms runtime.MemStats
|
||||||
|
memStatsFromRM(&ms, c.sampleMap)
|
||||||
|
for _, i := range c.msMetrics {
|
||||||
|
ch <- MustNewConstMetric(i.desc, i.valType, i.eval(&ms))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// unwrapScalarRMValue unwraps a runtime/metrics value that is assumed
|
||||||
|
// to be scalar and returns the equivalent float64 value. Panics if the
|
||||||
|
// value is not scalar.
|
||||||
|
func unwrapScalarRMValue(v metrics.Value) float64 {
|
||||||
|
switch v.Kind() {
|
||||||
|
case metrics.KindUint64:
|
||||||
|
return float64(v.Uint64())
|
||||||
|
case metrics.KindFloat64:
|
||||||
|
return v.Float64()
|
||||||
|
case metrics.KindBad:
|
||||||
|
// Unsupported metric.
|
||||||
|
//
|
||||||
|
// This should never happen because we always populate our metric
|
||||||
|
// set from the runtime/metrics package.
|
||||||
|
panic("unexpected bad kind metric")
|
||||||
|
default:
|
||||||
|
// Unsupported metric kind.
|
||||||
|
//
|
||||||
|
// This should never happen because we check for this during initialization
|
||||||
|
// and flag and filter metrics whose kinds we don't understand.
|
||||||
|
panic(fmt.Sprintf("unexpected unsupported metric: %v", v.Kind()))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// exactSumFor takes a runtime/metrics metric name (that is assumed to
|
||||||
|
// be of kind KindFloat64Histogram) and returns its exact sum and whether
|
||||||
|
// its exact sum exists.
|
||||||
|
//
|
||||||
|
// The runtime/metrics API for histograms doesn't currently expose exact
|
||||||
|
// sums, but some of the other metrics are in fact exact sums of histograms.
|
||||||
|
func (c *goCollector) exactSumFor(rmName string) float64 {
|
||||||
|
sumName, ok := c.rmExactSumMapForHist[rmName]
|
||||||
|
if !ok {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
s, ok := c.sampleMap[sumName]
|
||||||
|
if !ok {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
return unwrapScalarRMValue(s.Value)
|
||||||
|
}
|
||||||
|
|
||||||
|
func memStatsFromRM(ms *runtime.MemStats, rm map[string]*metrics.Sample) {
|
||||||
|
lookupOrZero := func(name string) uint64 {
|
||||||
|
if s, ok := rm[name]; ok {
|
||||||
|
return s.Value.Uint64()
|
||||||
|
}
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// Currently, MemStats adds tiny alloc count to both Mallocs AND Frees.
|
||||||
|
// The reason for this is because MemStats couldn't be extended at the time
|
||||||
|
// but there was a desire to have Mallocs at least be a little more representative,
|
||||||
|
// while having Mallocs - Frees still represent a live object count.
|
||||||
|
// Unfortunately, MemStats doesn't actually export a large allocation count,
|
||||||
|
// so it's impossible to pull this number out directly.
|
||||||
|
tinyAllocs := lookupOrZero(goGCHeapTinyAllocsObjects)
|
||||||
|
ms.Mallocs = lookupOrZero(goGCHeapAllocsObjects) + tinyAllocs
|
||||||
|
ms.Frees = lookupOrZero(goGCHeapFreesObjects) + tinyAllocs
|
||||||
|
|
||||||
|
ms.TotalAlloc = lookupOrZero(goGCHeapAllocsBytes)
|
||||||
|
ms.Sys = lookupOrZero(goMemoryClassesTotalBytes)
|
||||||
|
ms.Lookups = 0 // Already always zero.
|
||||||
|
ms.HeapAlloc = lookupOrZero(goMemoryClassesHeapObjectsBytes)
|
||||||
|
ms.Alloc = ms.HeapAlloc
|
||||||
|
ms.HeapInuse = ms.HeapAlloc + lookupOrZero(goMemoryClassesHeapUnusedBytes)
|
||||||
|
ms.HeapReleased = lookupOrZero(goMemoryClassesHeapReleasedBytes)
|
||||||
|
ms.HeapIdle = ms.HeapReleased + lookupOrZero(goMemoryClassesHeapFreeBytes)
|
||||||
|
ms.HeapSys = ms.HeapInuse + ms.HeapIdle
|
||||||
|
ms.HeapObjects = lookupOrZero(goGCHeapObjects)
|
||||||
|
ms.StackInuse = lookupOrZero(goMemoryClassesHeapStacksBytes)
|
||||||
|
ms.StackSys = ms.StackInuse + lookupOrZero(goMemoryClassesOSStacksBytes)
|
||||||
|
ms.MSpanInuse = lookupOrZero(goMemoryClassesMetadataMSpanInuseBytes)
|
||||||
|
ms.MSpanSys = ms.MSpanInuse + lookupOrZero(goMemoryClassesMetadataMSPanFreeBytes)
|
||||||
|
ms.MCacheInuse = lookupOrZero(goMemoryClassesMetadataMCacheInuseBytes)
|
||||||
|
ms.MCacheSys = ms.MCacheInuse + lookupOrZero(goMemoryClassesMetadataMCacheFreeBytes)
|
||||||
|
ms.BuckHashSys = lookupOrZero(goMemoryClassesProfilingBucketsBytes)
|
||||||
|
ms.GCSys = lookupOrZero(goMemoryClassesMetadataOtherBytes)
|
||||||
|
ms.OtherSys = lookupOrZero(goMemoryClassesOtherBytes)
|
||||||
|
ms.NextGC = lookupOrZero(goGCHeapGoalBytes)
|
||||||
|
|
||||||
|
// N.B. GCCPUFraction is intentionally omitted. This metric is not useful,
|
||||||
|
// and often misleading due to the fact that it's an average over the lifetime
|
||||||
|
// of the process.
|
||||||
|
// See https://github.com/prometheus/client_golang/issues/842#issuecomment-861812034
|
||||||
|
// for more details.
|
||||||
|
ms.GCCPUFraction = 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// batchHistogram is a mutable histogram that is updated
|
||||||
|
// in batches.
|
||||||
|
type batchHistogram struct {
|
||||||
|
selfCollector
|
||||||
|
|
||||||
|
// Static fields updated only once.
|
||||||
|
desc *Desc
|
||||||
|
hasSum bool
|
||||||
|
|
||||||
|
// Because this histogram operates in batches, it just uses a
|
||||||
|
// single mutex for everything. updates are always serialized
|
||||||
|
// but Write calls may operate concurrently with updates.
|
||||||
|
// Contention between these two sources should be rare.
|
||||||
|
mu sync.Mutex
|
||||||
|
buckets []float64 // Inclusive lower bounds, like runtime/metrics.
|
||||||
|
counts []uint64
|
||||||
|
sum float64 // Used if hasSum is true.
|
||||||
|
}
|
||||||
|
|
||||||
|
// newBatchHistogram creates a new batch histogram value with the given
|
||||||
|
// Desc, buckets, and whether or not it has an exact sum available.
|
||||||
|
//
|
||||||
|
// buckets must always be from the runtime/metrics package, following
|
||||||
|
// the same conventions.
|
||||||
|
func newBatchHistogram(desc *Desc, buckets []float64, hasSum bool) *batchHistogram {
|
||||||
|
// We need to remove -Inf values. runtime/metrics keeps them around.
|
||||||
|
// But -Inf bucket should not be allowed for prometheus histograms.
|
||||||
|
if buckets[0] == math.Inf(-1) {
|
||||||
|
buckets = buckets[1:]
|
||||||
|
}
|
||||||
|
h := &batchHistogram{
|
||||||
|
desc: desc,
|
||||||
|
buckets: buckets,
|
||||||
|
// Because buckets follows runtime/metrics conventions, there's
|
||||||
|
// 1 more value in the buckets list than there are buckets represented,
|
||||||
|
// because in runtime/metrics, the bucket values represent *boundaries*,
|
||||||
|
// and non-Inf boundaries are inclusive lower bounds for that bucket.
|
||||||
|
counts: make([]uint64, len(buckets)-1),
|
||||||
|
hasSum: hasSum,
|
||||||
|
}
|
||||||
|
h.init(h)
|
||||||
|
return h
|
||||||
|
}
|
||||||
|
|
||||||
|
// update updates the batchHistogram from a runtime/metrics histogram.
|
||||||
|
//
|
||||||
|
// sum must be provided if the batchHistogram was created to have an exact sum.
|
||||||
|
// h.buckets must be a strict subset of his.Buckets.
|
||||||
|
func (h *batchHistogram) update(his *metrics.Float64Histogram, sum float64) {
|
||||||
|
counts, buckets := his.Counts, his.Buckets
|
||||||
|
|
||||||
|
h.mu.Lock()
|
||||||
|
defer h.mu.Unlock()
|
||||||
|
|
||||||
|
// Clear buckets.
|
||||||
|
for i := range h.counts {
|
||||||
|
h.counts[i] = 0
|
||||||
|
}
|
||||||
|
// Copy and reduce buckets.
|
||||||
|
var j int
|
||||||
|
for i, count := range counts {
|
||||||
|
h.counts[j] += count
|
||||||
|
if buckets[i+1] == h.buckets[j+1] {
|
||||||
|
j++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if h.hasSum {
|
||||||
|
h.sum = sum
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (h *batchHistogram) Desc() *Desc {
|
||||||
|
return h.desc
|
||||||
|
}
|
||||||
|
|
||||||
|
func (h *batchHistogram) Write(out *dto.Metric) error {
|
||||||
|
h.mu.Lock()
|
||||||
|
defer h.mu.Unlock()
|
||||||
|
|
||||||
|
sum := float64(0)
|
||||||
|
if h.hasSum {
|
||||||
|
sum = h.sum
|
||||||
|
}
|
||||||
|
dtoBuckets := make([]*dto.Bucket, 0, len(h.counts))
|
||||||
|
totalCount := uint64(0)
|
||||||
|
for i, count := range h.counts {
|
||||||
|
totalCount += count
|
||||||
|
if !h.hasSum {
|
||||||
|
if count != 0 {
|
||||||
|
// N.B. This computed sum is an underestimate.
|
||||||
|
sum += h.buckets[i] * float64(count)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Skip the +Inf bucket, but only for the bucket list.
|
||||||
|
// It must still count for sum and totalCount.
|
||||||
|
if math.IsInf(h.buckets[i+1], 1) {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
// Float64Histogram's upper bound is exclusive, so make it inclusive
|
||||||
|
// by obtaining the next float64 value down, in order.
|
||||||
|
upperBound := math.Nextafter(h.buckets[i+1], h.buckets[i])
|
||||||
|
dtoBuckets = append(dtoBuckets, &dto.Bucket{
|
||||||
|
CumulativeCount: proto.Uint64(totalCount),
|
||||||
|
UpperBound: proto.Float64(upperBound),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
out.Histogram = &dto.Histogram{
|
||||||
|
Bucket: dtoBuckets,
|
||||||
|
SampleCount: proto.Uint64(totalCount),
|
||||||
|
SampleSum: proto.Float64(sum),
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
2061
vendor/github.com/prometheus/client_golang/prometheus/histogram.go
generated
vendored
Normal file
2061
vendor/github.com/prometheus/client_golang/prometheus/histogram.go
generated
vendored
Normal file
File diff suppressed because it is too large
Load Diff
60
vendor/github.com/prometheus/client_golang/prometheus/internal/almost_equal.go
generated
vendored
Normal file
60
vendor/github.com/prometheus/client_golang/prometheus/internal/almost_equal.go
generated
vendored
Normal file
@ -0,0 +1,60 @@
|
|||||||
|
// Copyright (c) 2015 Björn Rabenstein
|
||||||
|
//
|
||||||
|
// Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
// of this software and associated documentation files (the "Software"), to deal
|
||||||
|
// in the Software without restriction, including without limitation the rights
|
||||||
|
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
// copies of the Software, and to permit persons to whom the Software is
|
||||||
|
// furnished to do so, subject to the following conditions:
|
||||||
|
//
|
||||||
|
// The above copyright notice and this permission notice shall be included in all
|
||||||
|
// copies or substantial portions of the Software.
|
||||||
|
//
|
||||||
|
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
// SOFTWARE.
|
||||||
|
//
|
||||||
|
// The code in this package is copy/paste to avoid a dependency. Hence this file
|
||||||
|
// carries the copyright of the original repo.
|
||||||
|
// https://github.com/beorn7/floats
|
||||||
|
package internal
|
||||||
|
|
||||||
|
import (
|
||||||
|
"math"
|
||||||
|
)
|
||||||
|
|
||||||
|
// minNormalFloat64 is the smallest positive normal value of type float64.
|
||||||
|
var minNormalFloat64 = math.Float64frombits(0x0010000000000000)
|
||||||
|
|
||||||
|
// AlmostEqualFloat64 returns true if a and b are equal within a relative error
|
||||||
|
// of epsilon. See http://floating-point-gui.de/errors/comparison/ for the
|
||||||
|
// details of the applied method.
|
||||||
|
func AlmostEqualFloat64(a, b, epsilon float64) bool {
|
||||||
|
if a == b {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
absA := math.Abs(a)
|
||||||
|
absB := math.Abs(b)
|
||||||
|
diff := math.Abs(a - b)
|
||||||
|
if a == 0 || b == 0 || absA+absB < minNormalFloat64 {
|
||||||
|
return diff < epsilon*minNormalFloat64
|
||||||
|
}
|
||||||
|
return diff/math.Min(absA+absB, math.MaxFloat64) < epsilon
|
||||||
|
}
|
||||||
|
|
||||||
|
// AlmostEqualFloat64s is the slice form of AlmostEqualFloat64.
|
||||||
|
func AlmostEqualFloat64s(a, b []float64, epsilon float64) bool {
|
||||||
|
if len(a) != len(b) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for i := range a {
|
||||||
|
if !AlmostEqualFloat64(a[i], b[i], epsilon) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
655
vendor/github.com/prometheus/client_golang/prometheus/internal/difflib.go
generated
vendored
Normal file
655
vendor/github.com/prometheus/client_golang/prometheus/internal/difflib.go
generated
vendored
Normal file
@ -0,0 +1,655 @@
|
|||||||
|
// Copyright 2022 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
//
|
||||||
|
// It provides tools to compare sequences of strings and generate textual diffs.
|
||||||
|
//
|
||||||
|
// Maintaining `GetUnifiedDiffString` here because original repository
|
||||||
|
// (https://github.com/pmezard/go-difflib) is no longer maintained.
|
||||||
|
package internal
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bufio"
|
||||||
|
"bytes"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
func minInt(a, b int) int {
|
||||||
|
if a < b {
|
||||||
|
return a
|
||||||
|
}
|
||||||
|
return b
|
||||||
|
}
|
||||||
|
|
||||||
|
func maxInt(a, b int) int {
|
||||||
|
if a > b {
|
||||||
|
return a
|
||||||
|
}
|
||||||
|
return b
|
||||||
|
}
|
||||||
|
|
||||||
|
func calculateRatio(matches, length int) float64 {
|
||||||
|
if length > 0 {
|
||||||
|
return 2.0 * float64(matches) / float64(length)
|
||||||
|
}
|
||||||
|
return 1.0
|
||||||
|
}
|
||||||
|
|
||||||
|
type Match struct {
|
||||||
|
A int
|
||||||
|
B int
|
||||||
|
Size int
|
||||||
|
}
|
||||||
|
|
||||||
|
type OpCode struct {
|
||||||
|
Tag byte
|
||||||
|
I1 int
|
||||||
|
I2 int
|
||||||
|
J1 int
|
||||||
|
J2 int
|
||||||
|
}
|
||||||
|
|
||||||
|
// SequenceMatcher compares sequence of strings. The basic
|
||||||
|
// algorithm predates, and is a little fancier than, an algorithm
|
||||||
|
// published in the late 1980's by Ratcliff and Obershelp under the
|
||||||
|
// hyperbolic name "gestalt pattern matching". The basic idea is to find
|
||||||
|
// the longest contiguous matching subsequence that contains no "junk"
|
||||||
|
// elements (R-O doesn't address junk). The same idea is then applied
|
||||||
|
// recursively to the pieces of the sequences to the left and to the right
|
||||||
|
// of the matching subsequence. This does not yield minimal edit
|
||||||
|
// sequences, but does tend to yield matches that "look right" to people.
|
||||||
|
//
|
||||||
|
// SequenceMatcher tries to compute a "human-friendly diff" between two
|
||||||
|
// sequences. Unlike e.g. UNIX(tm) diff, the fundamental notion is the
|
||||||
|
// longest *contiguous* & junk-free matching subsequence. That's what
|
||||||
|
// catches peoples' eyes. The Windows(tm) windiff has another interesting
|
||||||
|
// notion, pairing up elements that appear uniquely in each sequence.
|
||||||
|
// That, and the method here, appear to yield more intuitive difference
|
||||||
|
// reports than does diff. This method appears to be the least vulnerable
|
||||||
|
// to syncing up on blocks of "junk lines", though (like blank lines in
|
||||||
|
// ordinary text files, or maybe "<P>" lines in HTML files). That may be
|
||||||
|
// because this is the only method of the 3 that has a *concept* of
|
||||||
|
// "junk" <wink>.
|
||||||
|
//
|
||||||
|
// Timing: Basic R-O is cubic time worst case and quadratic time expected
|
||||||
|
// case. SequenceMatcher is quadratic time for the worst case and has
|
||||||
|
// expected-case behavior dependent in a complicated way on how many
|
||||||
|
// elements the sequences have in common; best case time is linear.
|
||||||
|
type SequenceMatcher struct {
|
||||||
|
a []string
|
||||||
|
b []string
|
||||||
|
b2j map[string][]int
|
||||||
|
IsJunk func(string) bool
|
||||||
|
autoJunk bool
|
||||||
|
bJunk map[string]struct{}
|
||||||
|
matchingBlocks []Match
|
||||||
|
fullBCount map[string]int
|
||||||
|
bPopular map[string]struct{}
|
||||||
|
opCodes []OpCode
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewMatcher(a, b []string) *SequenceMatcher {
|
||||||
|
m := SequenceMatcher{autoJunk: true}
|
||||||
|
m.SetSeqs(a, b)
|
||||||
|
return &m
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewMatcherWithJunk(a, b []string, autoJunk bool,
|
||||||
|
isJunk func(string) bool,
|
||||||
|
) *SequenceMatcher {
|
||||||
|
m := SequenceMatcher{IsJunk: isJunk, autoJunk: autoJunk}
|
||||||
|
m.SetSeqs(a, b)
|
||||||
|
return &m
|
||||||
|
}
|
||||||
|
|
||||||
|
// Set two sequences to be compared.
|
||||||
|
func (m *SequenceMatcher) SetSeqs(a, b []string) {
|
||||||
|
m.SetSeq1(a)
|
||||||
|
m.SetSeq2(b)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Set the first sequence to be compared. The second sequence to be compared is
|
||||||
|
// not changed.
|
||||||
|
//
|
||||||
|
// SequenceMatcher computes and caches detailed information about the second
|
||||||
|
// sequence, so if you want to compare one sequence S against many sequences,
|
||||||
|
// use .SetSeq2(s) once and call .SetSeq1(x) repeatedly for each of the other
|
||||||
|
// sequences.
|
||||||
|
//
|
||||||
|
// See also SetSeqs() and SetSeq2().
|
||||||
|
func (m *SequenceMatcher) SetSeq1(a []string) {
|
||||||
|
if &a == &m.a {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
m.a = a
|
||||||
|
m.matchingBlocks = nil
|
||||||
|
m.opCodes = nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Set the second sequence to be compared. The first sequence to be compared is
|
||||||
|
// not changed.
|
||||||
|
func (m *SequenceMatcher) SetSeq2(b []string) {
|
||||||
|
if &b == &m.b {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
m.b = b
|
||||||
|
m.matchingBlocks = nil
|
||||||
|
m.opCodes = nil
|
||||||
|
m.fullBCount = nil
|
||||||
|
m.chainB()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *SequenceMatcher) chainB() {
|
||||||
|
// Populate line -> index mapping
|
||||||
|
b2j := map[string][]int{}
|
||||||
|
for i, s := range m.b {
|
||||||
|
indices := b2j[s]
|
||||||
|
indices = append(indices, i)
|
||||||
|
b2j[s] = indices
|
||||||
|
}
|
||||||
|
|
||||||
|
// Purge junk elements
|
||||||
|
m.bJunk = map[string]struct{}{}
|
||||||
|
if m.IsJunk != nil {
|
||||||
|
junk := m.bJunk
|
||||||
|
for s := range b2j {
|
||||||
|
if m.IsJunk(s) {
|
||||||
|
junk[s] = struct{}{}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for s := range junk {
|
||||||
|
delete(b2j, s)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Purge remaining popular elements
|
||||||
|
popular := map[string]struct{}{}
|
||||||
|
n := len(m.b)
|
||||||
|
if m.autoJunk && n >= 200 {
|
||||||
|
ntest := n/100 + 1
|
||||||
|
for s, indices := range b2j {
|
||||||
|
if len(indices) > ntest {
|
||||||
|
popular[s] = struct{}{}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for s := range popular {
|
||||||
|
delete(b2j, s)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
m.bPopular = popular
|
||||||
|
m.b2j = b2j
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *SequenceMatcher) isBJunk(s string) bool {
|
||||||
|
_, ok := m.bJunk[s]
|
||||||
|
return ok
|
||||||
|
}
|
||||||
|
|
||||||
|
// Find longest matching block in a[alo:ahi] and b[blo:bhi].
|
||||||
|
//
|
||||||
|
// If IsJunk is not defined:
|
||||||
|
//
|
||||||
|
// Return (i,j,k) such that a[i:i+k] is equal to b[j:j+k], where
|
||||||
|
//
|
||||||
|
// alo <= i <= i+k <= ahi
|
||||||
|
// blo <= j <= j+k <= bhi
|
||||||
|
//
|
||||||
|
// and for all (i',j',k') meeting those conditions,
|
||||||
|
//
|
||||||
|
// k >= k'
|
||||||
|
// i <= i'
|
||||||
|
// and if i == i', j <= j'
|
||||||
|
//
|
||||||
|
// In other words, of all maximal matching blocks, return one that
|
||||||
|
// starts earliest in a, and of all those maximal matching blocks that
|
||||||
|
// start earliest in a, return the one that starts earliest in b.
|
||||||
|
//
|
||||||
|
// If IsJunk is defined, first the longest matching block is
|
||||||
|
// determined as above, but with the additional restriction that no
|
||||||
|
// junk element appears in the block. Then that block is extended as
|
||||||
|
// far as possible by matching (only) junk elements on both sides. So
|
||||||
|
// the resulting block never matches on junk except as identical junk
|
||||||
|
// happens to be adjacent to an "interesting" match.
|
||||||
|
//
|
||||||
|
// If no blocks match, return (alo, blo, 0).
|
||||||
|
func (m *SequenceMatcher) findLongestMatch(alo, ahi, blo, bhi int) Match {
|
||||||
|
// CAUTION: stripping common prefix or suffix would be incorrect.
|
||||||
|
// E.g.,
|
||||||
|
// ab
|
||||||
|
// acab
|
||||||
|
// Longest matching block is "ab", but if common prefix is
|
||||||
|
// stripped, it's "a" (tied with "b"). UNIX(tm) diff does so
|
||||||
|
// strip, so ends up claiming that ab is changed to acab by
|
||||||
|
// inserting "ca" in the middle. That's minimal but unintuitive:
|
||||||
|
// "it's obvious" that someone inserted "ac" at the front.
|
||||||
|
// Windiff ends up at the same place as diff, but by pairing up
|
||||||
|
// the unique 'b's and then matching the first two 'a's.
|
||||||
|
besti, bestj, bestsize := alo, blo, 0
|
||||||
|
|
||||||
|
// find longest junk-free match
|
||||||
|
// during an iteration of the loop, j2len[j] = length of longest
|
||||||
|
// junk-free match ending with a[i-1] and b[j]
|
||||||
|
j2len := map[int]int{}
|
||||||
|
for i := alo; i != ahi; i++ {
|
||||||
|
// look at all instances of a[i] in b; note that because
|
||||||
|
// b2j has no junk keys, the loop is skipped if a[i] is junk
|
||||||
|
newj2len := map[int]int{}
|
||||||
|
for _, j := range m.b2j[m.a[i]] {
|
||||||
|
// a[i] matches b[j]
|
||||||
|
if j < blo {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if j >= bhi {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
k := j2len[j-1] + 1
|
||||||
|
newj2len[j] = k
|
||||||
|
if k > bestsize {
|
||||||
|
besti, bestj, bestsize = i-k+1, j-k+1, k
|
||||||
|
}
|
||||||
|
}
|
||||||
|
j2len = newj2len
|
||||||
|
}
|
||||||
|
|
||||||
|
// Extend the best by non-junk elements on each end. In particular,
|
||||||
|
// "popular" non-junk elements aren't in b2j, which greatly speeds
|
||||||
|
// the inner loop above, but also means "the best" match so far
|
||||||
|
// doesn't contain any junk *or* popular non-junk elements.
|
||||||
|
for besti > alo && bestj > blo && !m.isBJunk(m.b[bestj-1]) &&
|
||||||
|
m.a[besti-1] == m.b[bestj-1] {
|
||||||
|
besti, bestj, bestsize = besti-1, bestj-1, bestsize+1
|
||||||
|
}
|
||||||
|
for besti+bestsize < ahi && bestj+bestsize < bhi &&
|
||||||
|
!m.isBJunk(m.b[bestj+bestsize]) &&
|
||||||
|
m.a[besti+bestsize] == m.b[bestj+bestsize] {
|
||||||
|
bestsize++
|
||||||
|
}
|
||||||
|
|
||||||
|
// Now that we have a wholly interesting match (albeit possibly
|
||||||
|
// empty!), we may as well suck up the matching junk on each
|
||||||
|
// side of it too. Can't think of a good reason not to, and it
|
||||||
|
// saves post-processing the (possibly considerable) expense of
|
||||||
|
// figuring out what to do with it. In the case of an empty
|
||||||
|
// interesting match, this is clearly the right thing to do,
|
||||||
|
// because no other kind of match is possible in the regions.
|
||||||
|
for besti > alo && bestj > blo && m.isBJunk(m.b[bestj-1]) &&
|
||||||
|
m.a[besti-1] == m.b[bestj-1] {
|
||||||
|
besti, bestj, bestsize = besti-1, bestj-1, bestsize+1
|
||||||
|
}
|
||||||
|
for besti+bestsize < ahi && bestj+bestsize < bhi &&
|
||||||
|
m.isBJunk(m.b[bestj+bestsize]) &&
|
||||||
|
m.a[besti+bestsize] == m.b[bestj+bestsize] {
|
||||||
|
bestsize++
|
||||||
|
}
|
||||||
|
|
||||||
|
return Match{A: besti, B: bestj, Size: bestsize}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Return list of triples describing matching subsequences.
|
||||||
|
//
|
||||||
|
// Each triple is of the form (i, j, n), and means that
|
||||||
|
// a[i:i+n] == b[j:j+n]. The triples are monotonically increasing in
|
||||||
|
// i and in j. It's also guaranteed that if (i, j, n) and (i', j', n') are
|
||||||
|
// adjacent triples in the list, and the second is not the last triple in the
|
||||||
|
// list, then i+n != i' or j+n != j'. IOW, adjacent triples never describe
|
||||||
|
// adjacent equal blocks.
|
||||||
|
//
|
||||||
|
// The last triple is a dummy, (len(a), len(b), 0), and is the only
|
||||||
|
// triple with n==0.
|
||||||
|
func (m *SequenceMatcher) GetMatchingBlocks() []Match {
|
||||||
|
if m.matchingBlocks != nil {
|
||||||
|
return m.matchingBlocks
|
||||||
|
}
|
||||||
|
|
||||||
|
var matchBlocks func(alo, ahi, blo, bhi int, matched []Match) []Match
|
||||||
|
matchBlocks = func(alo, ahi, blo, bhi int, matched []Match) []Match {
|
||||||
|
match := m.findLongestMatch(alo, ahi, blo, bhi)
|
||||||
|
i, j, k := match.A, match.B, match.Size
|
||||||
|
if match.Size > 0 {
|
||||||
|
if alo < i && blo < j {
|
||||||
|
matched = matchBlocks(alo, i, blo, j, matched)
|
||||||
|
}
|
||||||
|
matched = append(matched, match)
|
||||||
|
if i+k < ahi && j+k < bhi {
|
||||||
|
matched = matchBlocks(i+k, ahi, j+k, bhi, matched)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return matched
|
||||||
|
}
|
||||||
|
matched := matchBlocks(0, len(m.a), 0, len(m.b), nil)
|
||||||
|
|
||||||
|
// It's possible that we have adjacent equal blocks in the
|
||||||
|
// matching_blocks list now.
|
||||||
|
nonAdjacent := []Match{}
|
||||||
|
i1, j1, k1 := 0, 0, 0
|
||||||
|
for _, b := range matched {
|
||||||
|
// Is this block adjacent to i1, j1, k1?
|
||||||
|
i2, j2, k2 := b.A, b.B, b.Size
|
||||||
|
if i1+k1 == i2 && j1+k1 == j2 {
|
||||||
|
// Yes, so collapse them -- this just increases the length of
|
||||||
|
// the first block by the length of the second, and the first
|
||||||
|
// block so lengthened remains the block to compare against.
|
||||||
|
k1 += k2
|
||||||
|
} else {
|
||||||
|
// Not adjacent. Remember the first block (k1==0 means it's
|
||||||
|
// the dummy we started with), and make the second block the
|
||||||
|
// new block to compare against.
|
||||||
|
if k1 > 0 {
|
||||||
|
nonAdjacent = append(nonAdjacent, Match{i1, j1, k1})
|
||||||
|
}
|
||||||
|
i1, j1, k1 = i2, j2, k2
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if k1 > 0 {
|
||||||
|
nonAdjacent = append(nonAdjacent, Match{i1, j1, k1})
|
||||||
|
}
|
||||||
|
|
||||||
|
nonAdjacent = append(nonAdjacent, Match{len(m.a), len(m.b), 0})
|
||||||
|
m.matchingBlocks = nonAdjacent
|
||||||
|
return m.matchingBlocks
|
||||||
|
}
|
||||||
|
|
||||||
|
// Return list of 5-tuples describing how to turn a into b.
|
||||||
|
//
|
||||||
|
// Each tuple is of the form (tag, i1, i2, j1, j2). The first tuple
|
||||||
|
// has i1 == j1 == 0, and remaining tuples have i1 == the i2 from the
|
||||||
|
// tuple preceding it, and likewise for j1 == the previous j2.
|
||||||
|
//
|
||||||
|
// The tags are characters, with these meanings:
|
||||||
|
//
|
||||||
|
// 'r' (replace): a[i1:i2] should be replaced by b[j1:j2]
|
||||||
|
//
|
||||||
|
// 'd' (delete): a[i1:i2] should be deleted, j1==j2 in this case.
|
||||||
|
//
|
||||||
|
// 'i' (insert): b[j1:j2] should be inserted at a[i1:i1], i1==i2 in this case.
|
||||||
|
//
|
||||||
|
// 'e' (equal): a[i1:i2] == b[j1:j2]
|
||||||
|
func (m *SequenceMatcher) GetOpCodes() []OpCode {
|
||||||
|
if m.opCodes != nil {
|
||||||
|
return m.opCodes
|
||||||
|
}
|
||||||
|
i, j := 0, 0
|
||||||
|
matching := m.GetMatchingBlocks()
|
||||||
|
opCodes := make([]OpCode, 0, len(matching))
|
||||||
|
for _, m := range matching {
|
||||||
|
// invariant: we've pumped out correct diffs to change
|
||||||
|
// a[:i] into b[:j], and the next matching block is
|
||||||
|
// a[ai:ai+size] == b[bj:bj+size]. So we need to pump
|
||||||
|
// out a diff to change a[i:ai] into b[j:bj], pump out
|
||||||
|
// the matching block, and move (i,j) beyond the match
|
||||||
|
ai, bj, size := m.A, m.B, m.Size
|
||||||
|
tag := byte(0)
|
||||||
|
if i < ai && j < bj {
|
||||||
|
tag = 'r'
|
||||||
|
} else if i < ai {
|
||||||
|
tag = 'd'
|
||||||
|
} else if j < bj {
|
||||||
|
tag = 'i'
|
||||||
|
}
|
||||||
|
if tag > 0 {
|
||||||
|
opCodes = append(opCodes, OpCode{tag, i, ai, j, bj})
|
||||||
|
}
|
||||||
|
i, j = ai+size, bj+size
|
||||||
|
// the list of matching blocks is terminated by a
|
||||||
|
// sentinel with size 0
|
||||||
|
if size > 0 {
|
||||||
|
opCodes = append(opCodes, OpCode{'e', ai, i, bj, j})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
m.opCodes = opCodes
|
||||||
|
return m.opCodes
|
||||||
|
}
|
||||||
|
|
||||||
|
// Isolate change clusters by eliminating ranges with no changes.
|
||||||
|
//
|
||||||
|
// Return a generator of groups with up to n lines of context.
|
||||||
|
// Each group is in the same format as returned by GetOpCodes().
|
||||||
|
func (m *SequenceMatcher) GetGroupedOpCodes(n int) [][]OpCode {
|
||||||
|
if n < 0 {
|
||||||
|
n = 3
|
||||||
|
}
|
||||||
|
codes := m.GetOpCodes()
|
||||||
|
if len(codes) == 0 {
|
||||||
|
codes = []OpCode{{'e', 0, 1, 0, 1}}
|
||||||
|
}
|
||||||
|
// Fixup leading and trailing groups if they show no changes.
|
||||||
|
if codes[0].Tag == 'e' {
|
||||||
|
c := codes[0]
|
||||||
|
i1, i2, j1, j2 := c.I1, c.I2, c.J1, c.J2
|
||||||
|
codes[0] = OpCode{c.Tag, maxInt(i1, i2-n), i2, maxInt(j1, j2-n), j2}
|
||||||
|
}
|
||||||
|
if codes[len(codes)-1].Tag == 'e' {
|
||||||
|
c := codes[len(codes)-1]
|
||||||
|
i1, i2, j1, j2 := c.I1, c.I2, c.J1, c.J2
|
||||||
|
codes[len(codes)-1] = OpCode{c.Tag, i1, minInt(i2, i1+n), j1, minInt(j2, j1+n)}
|
||||||
|
}
|
||||||
|
nn := n + n
|
||||||
|
groups := [][]OpCode{}
|
||||||
|
group := []OpCode{}
|
||||||
|
for _, c := range codes {
|
||||||
|
i1, i2, j1, j2 := c.I1, c.I2, c.J1, c.J2
|
||||||
|
// End the current group and start a new one whenever
|
||||||
|
// there is a large range with no changes.
|
||||||
|
if c.Tag == 'e' && i2-i1 > nn {
|
||||||
|
group = append(group, OpCode{
|
||||||
|
c.Tag, i1, minInt(i2, i1+n),
|
||||||
|
j1, minInt(j2, j1+n),
|
||||||
|
})
|
||||||
|
groups = append(groups, group)
|
||||||
|
group = []OpCode{}
|
||||||
|
i1, j1 = maxInt(i1, i2-n), maxInt(j1, j2-n)
|
||||||
|
}
|
||||||
|
group = append(group, OpCode{c.Tag, i1, i2, j1, j2})
|
||||||
|
}
|
||||||
|
if len(group) > 0 && (len(group) != 1 || group[0].Tag != 'e') {
|
||||||
|
groups = append(groups, group)
|
||||||
|
}
|
||||||
|
return groups
|
||||||
|
}
|
||||||
|
|
||||||
|
// Return a measure of the sequences' similarity (float in [0,1]).
|
||||||
|
//
|
||||||
|
// Where T is the total number of elements in both sequences, and
|
||||||
|
// M is the number of matches, this is 2.0*M / T.
|
||||||
|
// Note that this is 1 if the sequences are identical, and 0 if
|
||||||
|
// they have nothing in common.
|
||||||
|
//
|
||||||
|
// .Ratio() is expensive to compute if you haven't already computed
|
||||||
|
// .GetMatchingBlocks() or .GetOpCodes(), in which case you may
|
||||||
|
// want to try .QuickRatio() or .RealQuickRation() first to get an
|
||||||
|
// upper bound.
|
||||||
|
func (m *SequenceMatcher) Ratio() float64 {
|
||||||
|
matches := 0
|
||||||
|
for _, m := range m.GetMatchingBlocks() {
|
||||||
|
matches += m.Size
|
||||||
|
}
|
||||||
|
return calculateRatio(matches, len(m.a)+len(m.b))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Return an upper bound on ratio() relatively quickly.
|
||||||
|
//
|
||||||
|
// This isn't defined beyond that it is an upper bound on .Ratio(), and
|
||||||
|
// is faster to compute.
|
||||||
|
func (m *SequenceMatcher) QuickRatio() float64 {
|
||||||
|
// viewing a and b as multisets, set matches to the cardinality
|
||||||
|
// of their intersection; this counts the number of matches
|
||||||
|
// without regard to order, so is clearly an upper bound
|
||||||
|
if m.fullBCount == nil {
|
||||||
|
m.fullBCount = map[string]int{}
|
||||||
|
for _, s := range m.b {
|
||||||
|
m.fullBCount[s]++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// avail[x] is the number of times x appears in 'b' less the
|
||||||
|
// number of times we've seen it in 'a' so far ... kinda
|
||||||
|
avail := map[string]int{}
|
||||||
|
matches := 0
|
||||||
|
for _, s := range m.a {
|
||||||
|
n, ok := avail[s]
|
||||||
|
if !ok {
|
||||||
|
n = m.fullBCount[s]
|
||||||
|
}
|
||||||
|
avail[s] = n - 1
|
||||||
|
if n > 0 {
|
||||||
|
matches++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return calculateRatio(matches, len(m.a)+len(m.b))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Return an upper bound on ratio() very quickly.
|
||||||
|
//
|
||||||
|
// This isn't defined beyond that it is an upper bound on .Ratio(), and
|
||||||
|
// is faster to compute than either .Ratio() or .QuickRatio().
|
||||||
|
func (m *SequenceMatcher) RealQuickRatio() float64 {
|
||||||
|
la, lb := len(m.a), len(m.b)
|
||||||
|
return calculateRatio(minInt(la, lb), la+lb)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Convert range to the "ed" format
|
||||||
|
func formatRangeUnified(start, stop int) string {
|
||||||
|
// Per the diff spec at http://www.unix.org/single_unix_specification/
|
||||||
|
beginning := start + 1 // lines start numbering with one
|
||||||
|
length := stop - start
|
||||||
|
if length == 1 {
|
||||||
|
return strconv.Itoa(beginning)
|
||||||
|
}
|
||||||
|
if length == 0 {
|
||||||
|
beginning-- // empty ranges begin at line just before the range
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("%d,%d", beginning, length)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Unified diff parameters
|
||||||
|
type UnifiedDiff struct {
|
||||||
|
A []string // First sequence lines
|
||||||
|
FromFile string // First file name
|
||||||
|
FromDate string // First file time
|
||||||
|
B []string // Second sequence lines
|
||||||
|
ToFile string // Second file name
|
||||||
|
ToDate string // Second file time
|
||||||
|
Eol string // Headers end of line, defaults to LF
|
||||||
|
Context int // Number of context lines
|
||||||
|
}
|
||||||
|
|
||||||
|
// Compare two sequences of lines; generate the delta as a unified diff.
|
||||||
|
//
|
||||||
|
// Unified diffs are a compact way of showing line changes and a few
|
||||||
|
// lines of context. The number of context lines is set by 'n' which
|
||||||
|
// defaults to three.
|
||||||
|
//
|
||||||
|
// By default, the diff control lines (those with ---, +++, or @@) are
|
||||||
|
// created with a trailing newline. This is helpful so that inputs
|
||||||
|
// created from file.readlines() result in diffs that are suitable for
|
||||||
|
// file.writelines() since both the inputs and outputs have trailing
|
||||||
|
// newlines.
|
||||||
|
//
|
||||||
|
// For inputs that do not have trailing newlines, set the lineterm
|
||||||
|
// argument to "" so that the output will be uniformly newline free.
|
||||||
|
//
|
||||||
|
// The unidiff format normally has a header for filenames and modification
|
||||||
|
// times. Any or all of these may be specified using strings for
|
||||||
|
// 'fromfile', 'tofile', 'fromfiledate', and 'tofiledate'.
|
||||||
|
// The modification times are normally expressed in the ISO 8601 format.
|
||||||
|
func WriteUnifiedDiff(writer io.Writer, diff UnifiedDiff) error {
|
||||||
|
buf := bufio.NewWriter(writer)
|
||||||
|
defer buf.Flush()
|
||||||
|
wf := func(format string, args ...any) error {
|
||||||
|
_, err := fmt.Fprintf(buf, format, args...)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
ws := func(s string) error {
|
||||||
|
_, err := buf.WriteString(s)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(diff.Eol) == 0 {
|
||||||
|
diff.Eol = "\n"
|
||||||
|
}
|
||||||
|
|
||||||
|
started := false
|
||||||
|
m := NewMatcher(diff.A, diff.B)
|
||||||
|
for _, g := range m.GetGroupedOpCodes(diff.Context) {
|
||||||
|
if !started {
|
||||||
|
started = true
|
||||||
|
fromDate := ""
|
||||||
|
if len(diff.FromDate) > 0 {
|
||||||
|
fromDate = "\t" + diff.FromDate
|
||||||
|
}
|
||||||
|
toDate := ""
|
||||||
|
if len(diff.ToDate) > 0 {
|
||||||
|
toDate = "\t" + diff.ToDate
|
||||||
|
}
|
||||||
|
if diff.FromFile != "" || diff.ToFile != "" {
|
||||||
|
err := wf("--- %s%s%s", diff.FromFile, fromDate, diff.Eol)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
err = wf("+++ %s%s%s", diff.ToFile, toDate, diff.Eol)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
first, last := g[0], g[len(g)-1]
|
||||||
|
range1 := formatRangeUnified(first.I1, last.I2)
|
||||||
|
range2 := formatRangeUnified(first.J1, last.J2)
|
||||||
|
if err := wf("@@ -%s +%s @@%s", range1, range2, diff.Eol); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for _, c := range g {
|
||||||
|
i1, i2, j1, j2 := c.I1, c.I2, c.J1, c.J2
|
||||||
|
if c.Tag == 'e' {
|
||||||
|
for _, line := range diff.A[i1:i2] {
|
||||||
|
if err := ws(" " + line); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if c.Tag == 'r' || c.Tag == 'd' {
|
||||||
|
for _, line := range diff.A[i1:i2] {
|
||||||
|
if err := ws("-" + line); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if c.Tag == 'r' || c.Tag == 'i' {
|
||||||
|
for _, line := range diff.B[j1:j2] {
|
||||||
|
if err := ws("+" + line); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Like WriteUnifiedDiff but returns the diff a string.
|
||||||
|
func GetUnifiedDiffString(diff UnifiedDiff) (string, error) {
|
||||||
|
w := &bytes.Buffer{}
|
||||||
|
err := WriteUnifiedDiff(w, diff)
|
||||||
|
return w.String(), err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Split a string on "\n" while preserving them. The output can be used
|
||||||
|
// as input for UnifiedDiff and ContextDiff structures.
|
||||||
|
func SplitLines(s string) []string {
|
||||||
|
lines := strings.SplitAfter(s, "\n")
|
||||||
|
lines[len(lines)-1] += "\n"
|
||||||
|
return lines
|
||||||
|
}
|
||||||
34
vendor/github.com/prometheus/client_golang/prometheus/internal/go_collector_options.go
generated
vendored
Normal file
34
vendor/github.com/prometheus/client_golang/prometheus/internal/go_collector_options.go
generated
vendored
Normal file
@ -0,0 +1,34 @@
|
|||||||
|
// Copyright 2021 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package internal
|
||||||
|
|
||||||
|
import "regexp"
|
||||||
|
|
||||||
|
type GoCollectorRule struct {
|
||||||
|
Matcher *regexp.Regexp
|
||||||
|
Deny bool
|
||||||
|
}
|
||||||
|
|
||||||
|
// GoCollectorOptions should not be used be directly by anything, except `collectors` package.
|
||||||
|
// Use it via collectors package instead. See issue
|
||||||
|
// https://github.com/prometheus/client_golang/issues/1030.
|
||||||
|
//
|
||||||
|
// This is internal, so external users only can use it via `collector.WithGoCollector*` methods
|
||||||
|
type GoCollectorOptions struct {
|
||||||
|
DisableMemStatsLikeMetrics bool
|
||||||
|
RuntimeMetricSumForHist map[string]string
|
||||||
|
RuntimeMetricRules []GoCollectorRule
|
||||||
|
}
|
||||||
|
|
||||||
|
var GoCollectorDefaultRuntimeMetrics = regexp.MustCompile(`/gc/gogc:percent|/gc/gomemlimit:bytes|/sched/gomaxprocs:threads`)
|
||||||
143
vendor/github.com/prometheus/client_golang/prometheus/internal/go_runtime_metrics.go
generated
vendored
Normal file
143
vendor/github.com/prometheus/client_golang/prometheus/internal/go_runtime_metrics.go
generated
vendored
Normal file
@ -0,0 +1,143 @@
|
|||||||
|
// Copyright 2021 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
//go:build go1.17
|
||||||
|
// +build go1.17
|
||||||
|
|
||||||
|
package internal
|
||||||
|
|
||||||
|
import (
|
||||||
|
"math"
|
||||||
|
"path"
|
||||||
|
"runtime/metrics"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/prometheus/common/model"
|
||||||
|
)
|
||||||
|
|
||||||
|
// RuntimeMetricsToProm produces a Prometheus metric name from a runtime/metrics
|
||||||
|
// metric description and validates whether the metric is suitable for integration
|
||||||
|
// with Prometheus.
|
||||||
|
//
|
||||||
|
// Returns false if a name could not be produced, or if Prometheus does not understand
|
||||||
|
// the runtime/metrics Kind.
|
||||||
|
//
|
||||||
|
// Note that the main reason a name couldn't be produced is if the runtime/metrics
|
||||||
|
// package exports a name with characters outside the valid Prometheus metric name
|
||||||
|
// character set. This is theoretically possible, but should never happen in practice.
|
||||||
|
// Still, don't rely on it.
|
||||||
|
func RuntimeMetricsToProm(d *metrics.Description) (string, string, string, bool) {
|
||||||
|
namespace := "go"
|
||||||
|
|
||||||
|
comp := strings.SplitN(d.Name, ":", 2)
|
||||||
|
key := comp[0]
|
||||||
|
unit := comp[1]
|
||||||
|
|
||||||
|
// The last path element in the key is the name,
|
||||||
|
// the rest is the subsystem.
|
||||||
|
subsystem := path.Dir(key[1:] /* remove leading / */)
|
||||||
|
name := path.Base(key)
|
||||||
|
|
||||||
|
// subsystem is translated by replacing all / and - with _.
|
||||||
|
subsystem = strings.ReplaceAll(subsystem, "/", "_")
|
||||||
|
subsystem = strings.ReplaceAll(subsystem, "-", "_")
|
||||||
|
|
||||||
|
// unit is translated assuming that the unit contains no
|
||||||
|
// non-ASCII characters.
|
||||||
|
unit = strings.ReplaceAll(unit, "-", "_")
|
||||||
|
unit = strings.ReplaceAll(unit, "*", "_")
|
||||||
|
unit = strings.ReplaceAll(unit, "/", "_per_")
|
||||||
|
|
||||||
|
// name has - replaced with _ and is concatenated with the unit and
|
||||||
|
// other data.
|
||||||
|
name = strings.ReplaceAll(name, "-", "_")
|
||||||
|
name += "_" + unit
|
||||||
|
if d.Cumulative && d.Kind != metrics.KindFloat64Histogram {
|
||||||
|
name += "_total"
|
||||||
|
}
|
||||||
|
|
||||||
|
// Our current conversion moves to legacy naming, so use legacy validation.
|
||||||
|
valid := model.LegacyValidation.IsValidMetricName(namespace + "_" + subsystem + "_" + name)
|
||||||
|
switch d.Kind {
|
||||||
|
case metrics.KindUint64:
|
||||||
|
case metrics.KindFloat64:
|
||||||
|
case metrics.KindFloat64Histogram:
|
||||||
|
default:
|
||||||
|
valid = false
|
||||||
|
}
|
||||||
|
return namespace, subsystem, name, valid
|
||||||
|
}
|
||||||
|
|
||||||
|
// RuntimeMetricsBucketsForUnit takes a set of buckets obtained for a runtime/metrics histogram
|
||||||
|
// type (so, lower-bound inclusive) and a unit from a runtime/metrics name, and produces
|
||||||
|
// a reduced set of buckets. This function always removes any -Inf bucket as it's represented
|
||||||
|
// as the bottom-most upper-bound inclusive bucket in Prometheus.
|
||||||
|
func RuntimeMetricsBucketsForUnit(buckets []float64, unit string) []float64 {
|
||||||
|
switch unit {
|
||||||
|
case "bytes":
|
||||||
|
// Re-bucket as powers of 2.
|
||||||
|
return reBucketExp(buckets, 2)
|
||||||
|
case "seconds":
|
||||||
|
// Re-bucket as powers of 10 and then merge all buckets greater
|
||||||
|
// than 1 second into the +Inf bucket.
|
||||||
|
b := reBucketExp(buckets, 10)
|
||||||
|
for i := range b {
|
||||||
|
if b[i] <= 1 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
b[i] = math.Inf(1)
|
||||||
|
b = b[:i+1]
|
||||||
|
break
|
||||||
|
}
|
||||||
|
return b
|
||||||
|
}
|
||||||
|
return buckets
|
||||||
|
}
|
||||||
|
|
||||||
|
// reBucketExp takes a list of bucket boundaries (lower bound inclusive) and
|
||||||
|
// downsamples the buckets to those a multiple of base apart. The end result
|
||||||
|
// is a roughly exponential (in many cases, perfectly exponential) bucketing
|
||||||
|
// scheme.
|
||||||
|
func reBucketExp(buckets []float64, base float64) []float64 {
|
||||||
|
bucket := buckets[0]
|
||||||
|
var newBuckets []float64
|
||||||
|
// We may see a -Inf here, in which case, add it and skip it
|
||||||
|
// since we risk producing NaNs otherwise.
|
||||||
|
//
|
||||||
|
// We need to preserve -Inf values to maintain runtime/metrics
|
||||||
|
// conventions. We'll strip it out later.
|
||||||
|
if bucket == math.Inf(-1) {
|
||||||
|
newBuckets = append(newBuckets, bucket)
|
||||||
|
buckets = buckets[1:]
|
||||||
|
bucket = buckets[0]
|
||||||
|
}
|
||||||
|
// From now on, bucket should always have a non-Inf value because
|
||||||
|
// Infs are only ever at the ends of the bucket lists, so
|
||||||
|
// arithmetic operations on it are non-NaN.
|
||||||
|
for i := 1; i < len(buckets); i++ {
|
||||||
|
if bucket >= 0 && buckets[i] < bucket*base {
|
||||||
|
// The next bucket we want to include is at least bucket*base.
|
||||||
|
continue
|
||||||
|
} else if bucket < 0 && buckets[i] < bucket/base {
|
||||||
|
// In this case the bucket we're targeting is negative, and since
|
||||||
|
// we're ascending through buckets here, we need to divide to get
|
||||||
|
// closer to zero exponentially.
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
// The +Inf bucket will always be the last one, and we'll always
|
||||||
|
// end up including it here because bucket
|
||||||
|
newBuckets = append(newBuckets, bucket)
|
||||||
|
bucket = buckets[i]
|
||||||
|
}
|
||||||
|
return append(newBuckets, bucket)
|
||||||
|
}
|
||||||
101
vendor/github.com/prometheus/client_golang/prometheus/internal/metric.go
generated
vendored
Normal file
101
vendor/github.com/prometheus/client_golang/prometheus/internal/metric.go
generated
vendored
Normal file
@ -0,0 +1,101 @@
|
|||||||
|
// Copyright 2018 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package internal
|
||||||
|
|
||||||
|
import (
|
||||||
|
"sort"
|
||||||
|
|
||||||
|
dto "github.com/prometheus/client_model/go"
|
||||||
|
)
|
||||||
|
|
||||||
|
// LabelPairSorter implements sort.Interface. It is used to sort a slice of
|
||||||
|
// dto.LabelPair pointers.
|
||||||
|
type LabelPairSorter []*dto.LabelPair
|
||||||
|
|
||||||
|
func (s LabelPairSorter) Len() int {
|
||||||
|
return len(s)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s LabelPairSorter) Swap(i, j int) {
|
||||||
|
s[i], s[j] = s[j], s[i]
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s LabelPairSorter) Less(i, j int) bool {
|
||||||
|
return s[i].GetName() < s[j].GetName()
|
||||||
|
}
|
||||||
|
|
||||||
|
// MetricSorter is a sortable slice of *dto.Metric.
|
||||||
|
type MetricSorter []*dto.Metric
|
||||||
|
|
||||||
|
func (s MetricSorter) Len() int {
|
||||||
|
return len(s)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s MetricSorter) Swap(i, j int) {
|
||||||
|
s[i], s[j] = s[j], s[i]
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s MetricSorter) Less(i, j int) bool {
|
||||||
|
if len(s[i].Label) != len(s[j].Label) {
|
||||||
|
// This should not happen. The metrics are
|
||||||
|
// inconsistent. However, we have to deal with the fact, as
|
||||||
|
// people might use custom collectors or metric family injection
|
||||||
|
// to create inconsistent metrics. So let's simply compare the
|
||||||
|
// number of labels in this case. That will still yield
|
||||||
|
// reproducible sorting.
|
||||||
|
return len(s[i].Label) < len(s[j].Label)
|
||||||
|
}
|
||||||
|
for n, lp := range s[i].Label {
|
||||||
|
vi := lp.GetValue()
|
||||||
|
vj := s[j].Label[n].GetValue()
|
||||||
|
if vi != vj {
|
||||||
|
return vi < vj
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// We should never arrive here. Multiple metrics with the same
|
||||||
|
// label set in the same scrape will lead to undefined ingestion
|
||||||
|
// behavior. However, as above, we have to provide stable sorting
|
||||||
|
// here, even for inconsistent metrics. So sort equal metrics
|
||||||
|
// by their timestamp, with missing timestamps (implying "now")
|
||||||
|
// coming last.
|
||||||
|
if s[i].TimestampMs == nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if s[j].TimestampMs == nil {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return s[i].GetTimestampMs() < s[j].GetTimestampMs()
|
||||||
|
}
|
||||||
|
|
||||||
|
// NormalizeMetricFamilies returns a MetricFamily slice with empty
|
||||||
|
// MetricFamilies pruned and the remaining MetricFamilies sorted by name within
|
||||||
|
// the slice, with the contained Metrics sorted within each MetricFamily.
|
||||||
|
func NormalizeMetricFamilies(metricFamiliesByName map[string]*dto.MetricFamily) []*dto.MetricFamily {
|
||||||
|
for _, mf := range metricFamiliesByName {
|
||||||
|
sort.Sort(MetricSorter(mf.Metric))
|
||||||
|
}
|
||||||
|
names := make([]string, 0, len(metricFamiliesByName))
|
||||||
|
for name, mf := range metricFamiliesByName {
|
||||||
|
if len(mf.Metric) > 0 {
|
||||||
|
names = append(names, name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.Strings(names)
|
||||||
|
result := make([]*dto.MetricFamily, 0, len(names))
|
||||||
|
for _, name := range names {
|
||||||
|
result = append(result, metricFamiliesByName[name])
|
||||||
|
}
|
||||||
|
return result
|
||||||
|
}
|
||||||
188
vendor/github.com/prometheus/client_golang/prometheus/labels.go
generated
vendored
Normal file
188
vendor/github.com/prometheus/client_golang/prometheus/labels.go
generated
vendored
Normal file
@ -0,0 +1,188 @@
|
|||||||
|
// Copyright 2018 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
"unicode/utf8"
|
||||||
|
|
||||||
|
"github.com/prometheus/common/model"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Labels represents a collection of label name -> value mappings. This type is
|
||||||
|
// commonly used with the With(Labels) and GetMetricWith(Labels) methods of
|
||||||
|
// metric vector Collectors, e.g.:
|
||||||
|
//
|
||||||
|
// myVec.With(Labels{"code": "404", "method": "GET"}).Add(42)
|
||||||
|
//
|
||||||
|
// The other use-case is the specification of constant label pairs in Opts or to
|
||||||
|
// create a Desc.
|
||||||
|
type Labels map[string]string
|
||||||
|
|
||||||
|
// LabelConstraint normalizes label values.
|
||||||
|
type LabelConstraint func(string) string
|
||||||
|
|
||||||
|
// ConstrainedLabels represents a label name and its constrain function
|
||||||
|
// to normalize label values. This type is commonly used when constructing
|
||||||
|
// metric vector Collectors.
|
||||||
|
type ConstrainedLabel struct {
|
||||||
|
Name string
|
||||||
|
Constraint LabelConstraint
|
||||||
|
}
|
||||||
|
|
||||||
|
// ConstrainableLabels is an interface that allows creating of labels that can
|
||||||
|
// be optionally constrained.
|
||||||
|
//
|
||||||
|
// prometheus.V2().NewCounterVec(CounterVecOpts{
|
||||||
|
// CounterOpts: {...}, // Usual CounterOpts fields
|
||||||
|
// VariableLabels: []ConstrainedLabels{
|
||||||
|
// {Name: "A"},
|
||||||
|
// {Name: "B", Constraint: func(v string) string { ... }},
|
||||||
|
// },
|
||||||
|
// })
|
||||||
|
type ConstrainableLabels interface {
|
||||||
|
compile() *compiledLabels
|
||||||
|
labelNames() []string
|
||||||
|
}
|
||||||
|
|
||||||
|
// ConstrainedLabels represents a collection of label name -> constrain function
|
||||||
|
// to normalize label values. This type is commonly used when constructing
|
||||||
|
// metric vector Collectors.
|
||||||
|
type ConstrainedLabels []ConstrainedLabel
|
||||||
|
|
||||||
|
func (cls ConstrainedLabels) compile() *compiledLabels {
|
||||||
|
compiled := &compiledLabels{
|
||||||
|
names: make([]string, len(cls)),
|
||||||
|
labelConstraints: map[string]LabelConstraint{},
|
||||||
|
}
|
||||||
|
|
||||||
|
for i, label := range cls {
|
||||||
|
compiled.names[i] = label.Name
|
||||||
|
if label.Constraint != nil {
|
||||||
|
compiled.labelConstraints[label.Name] = label.Constraint
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return compiled
|
||||||
|
}
|
||||||
|
|
||||||
|
func (cls ConstrainedLabels) labelNames() []string {
|
||||||
|
names := make([]string, len(cls))
|
||||||
|
for i, label := range cls {
|
||||||
|
names[i] = label.Name
|
||||||
|
}
|
||||||
|
return names
|
||||||
|
}
|
||||||
|
|
||||||
|
// UnconstrainedLabels represents collection of label without any constraint on
|
||||||
|
// their value. Thus, it is simply a collection of label names.
|
||||||
|
//
|
||||||
|
// UnconstrainedLabels([]string{ "A", "B" })
|
||||||
|
//
|
||||||
|
// is equivalent to
|
||||||
|
//
|
||||||
|
// ConstrainedLabels {
|
||||||
|
// { Name: "A" },
|
||||||
|
// { Name: "B" },
|
||||||
|
// }
|
||||||
|
type UnconstrainedLabels []string
|
||||||
|
|
||||||
|
func (uls UnconstrainedLabels) compile() *compiledLabels {
|
||||||
|
return &compiledLabels{
|
||||||
|
names: uls,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (uls UnconstrainedLabels) labelNames() []string {
|
||||||
|
return uls
|
||||||
|
}
|
||||||
|
|
||||||
|
type compiledLabels struct {
|
||||||
|
names []string
|
||||||
|
labelConstraints map[string]LabelConstraint
|
||||||
|
}
|
||||||
|
|
||||||
|
func (cls *compiledLabels) compile() *compiledLabels {
|
||||||
|
return cls
|
||||||
|
}
|
||||||
|
|
||||||
|
func (cls *compiledLabels) labelNames() []string {
|
||||||
|
return cls.names
|
||||||
|
}
|
||||||
|
|
||||||
|
func (cls *compiledLabels) constrain(labelName, value string) string {
|
||||||
|
if fn, ok := cls.labelConstraints[labelName]; ok && fn != nil {
|
||||||
|
return fn(value)
|
||||||
|
}
|
||||||
|
return value
|
||||||
|
}
|
||||||
|
|
||||||
|
// reservedLabelPrefix is a prefix which is not legal in user-supplied
|
||||||
|
// label names.
|
||||||
|
const reservedLabelPrefix = "__"
|
||||||
|
|
||||||
|
var errInconsistentCardinality = errors.New("inconsistent label cardinality")
|
||||||
|
|
||||||
|
func makeInconsistentCardinalityError(fqName string, labels, labelValues []string) error {
|
||||||
|
return fmt.Errorf(
|
||||||
|
"%w: %q has %d variable labels named %q but %d values %q were provided",
|
||||||
|
errInconsistentCardinality, fqName,
|
||||||
|
len(labels), labels,
|
||||||
|
len(labelValues), labelValues,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateValuesInLabels(labels Labels, expectedNumberOfValues int) error {
|
||||||
|
if len(labels) != expectedNumberOfValues {
|
||||||
|
return fmt.Errorf(
|
||||||
|
"%w: expected %d label values but got %d in %#v",
|
||||||
|
errInconsistentCardinality, expectedNumberOfValues,
|
||||||
|
len(labels), labels,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
for name, val := range labels {
|
||||||
|
if !utf8.ValidString(val) {
|
||||||
|
return fmt.Errorf("label %s: value %q is not valid UTF-8", name, val)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateLabelValues(vals []string, expectedNumberOfValues int) error {
|
||||||
|
if len(vals) != expectedNumberOfValues {
|
||||||
|
// The call below makes vals escape, copy them to avoid that.
|
||||||
|
vals := append([]string(nil), vals...)
|
||||||
|
return fmt.Errorf(
|
||||||
|
"%w: expected %d label values but got %d in %#v",
|
||||||
|
errInconsistentCardinality, expectedNumberOfValues,
|
||||||
|
len(vals), vals,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, val := range vals {
|
||||||
|
if !utf8.ValidString(val) {
|
||||||
|
return fmt.Errorf("label value %q is not valid UTF-8", val)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func checkLabelName(l string) bool {
|
||||||
|
return model.UTF8Validation.IsValidLabelName(l) && !strings.HasPrefix(l, reservedLabelPrefix)
|
||||||
|
}
|
||||||
279
vendor/github.com/prometheus/client_golang/prometheus/metric.go
generated
vendored
Normal file
279
vendor/github.com/prometheus/client_golang/prometheus/metric.go
generated
vendored
Normal file
@ -0,0 +1,279 @@
|
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"math"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
dto "github.com/prometheus/client_model/go"
|
||||||
|
"github.com/prometheus/common/model"
|
||||||
|
"google.golang.org/protobuf/proto"
|
||||||
|
)
|
||||||
|
|
||||||
|
var separatorByteSlice = []byte{model.SeparatorByte} // For convenient use with xxhash.
|
||||||
|
|
||||||
|
// A Metric models a single sample value with its meta data being exported to
|
||||||
|
// Prometheus. Implementations of Metric in this package are Gauge, Counter,
|
||||||
|
// Histogram, Summary, and Untyped.
|
||||||
|
type Metric interface {
|
||||||
|
// Desc returns the descriptor for the Metric. This method idempotently
|
||||||
|
// returns the same descriptor throughout the lifetime of the
|
||||||
|
// Metric. The returned descriptor is immutable by contract. A Metric
|
||||||
|
// unable to describe itself must return an invalid descriptor (created
|
||||||
|
// with NewInvalidDesc).
|
||||||
|
Desc() *Desc
|
||||||
|
// Write encodes the Metric into a "Metric" Protocol Buffer data
|
||||||
|
// transmission object.
|
||||||
|
//
|
||||||
|
// Metric implementations must observe concurrency safety as reads of
|
||||||
|
// this metric may occur at any time, and any blocking occurs at the
|
||||||
|
// expense of total performance of rendering all registered
|
||||||
|
// metrics. Ideally, Metric implementations should support concurrent
|
||||||
|
// readers.
|
||||||
|
//
|
||||||
|
// While populating dto.Metric, it is the responsibility of the
|
||||||
|
// implementation to ensure validity of the Metric protobuf (like valid
|
||||||
|
// UTF-8 strings or syntactically valid metric and label names). It is
|
||||||
|
// recommended to sort labels lexicographically. Callers of Write should
|
||||||
|
// still make sure of sorting if they depend on it.
|
||||||
|
Write(*dto.Metric) error
|
||||||
|
// TODO(beorn7): The original rationale of passing in a pre-allocated
|
||||||
|
// dto.Metric protobuf to save allocations has disappeared. The
|
||||||
|
// signature of this method should be changed to "Write() (*dto.Metric,
|
||||||
|
// error)".
|
||||||
|
}
|
||||||
|
|
||||||
|
// Opts bundles the options for creating most Metric types. Each metric
|
||||||
|
// implementation XXX has its own XXXOpts type, but in most cases, it is just
|
||||||
|
// an alias of this type (which might change when the requirement arises.)
|
||||||
|
//
|
||||||
|
// It is mandatory to set Name to a non-empty string. All other fields are
|
||||||
|
// optional and can safely be left at their zero value, although it is strongly
|
||||||
|
// encouraged to set a Help string.
|
||||||
|
type Opts struct {
|
||||||
|
// Namespace, Subsystem, and Name are components of the fully-qualified
|
||||||
|
// name of the Metric (created by joining these components with
|
||||||
|
// "_"). Only Name is mandatory, the others merely help structuring the
|
||||||
|
// name. Note that the fully-qualified name of the metric must be a
|
||||||
|
// valid Prometheus metric name.
|
||||||
|
Namespace string
|
||||||
|
Subsystem string
|
||||||
|
Name string
|
||||||
|
|
||||||
|
// Help provides information about this metric.
|
||||||
|
//
|
||||||
|
// Metrics with the same fully-qualified name must have the same Help
|
||||||
|
// string.
|
||||||
|
Help string
|
||||||
|
|
||||||
|
// Unit provides the unit of this metric as per https://prometheus.io/docs/specs/om
|
||||||
|
Unit string
|
||||||
|
|
||||||
|
// ConstLabels are used to attach fixed labels to this metric. Metrics
|
||||||
|
// with the same fully-qualified name must have the same label names in
|
||||||
|
// their ConstLabels.
|
||||||
|
//
|
||||||
|
// ConstLabels are only used rarely. In particular, do not use them to
|
||||||
|
// attach the same labels to all your metrics. Those use cases are
|
||||||
|
// better covered by target labels set by the scraping Prometheus
|
||||||
|
// server, or by one specific metric (e.g. a build_info or a
|
||||||
|
// machine_role metric). See also
|
||||||
|
// https://prometheus.io/docs/instrumenting/writing_exporters/#target-labels-not-static-scraped-labels
|
||||||
|
ConstLabels Labels
|
||||||
|
|
||||||
|
// now is for testing purposes, by default it's time.Now.
|
||||||
|
now func() time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// BuildFQName joins the given three name components by "_". Empty name
|
||||||
|
// components are ignored. If the name parameter itself is empty, an empty
|
||||||
|
// string is returned, no matter what. Metric implementations included in this
|
||||||
|
// library use this function internally to generate the fully-qualified metric
|
||||||
|
// name from the name component in their Opts. Users of the library will only
|
||||||
|
// need this function if they implement their own Metric or instantiate a Desc
|
||||||
|
// (with NewDesc) directly.
|
||||||
|
func BuildFQName(namespace, subsystem, name string) string {
|
||||||
|
if name == "" {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
sb := strings.Builder{}
|
||||||
|
sb.Grow(len(namespace) + len(subsystem) + len(name) + 2)
|
||||||
|
|
||||||
|
if namespace != "" {
|
||||||
|
sb.WriteString(namespace)
|
||||||
|
sb.WriteString("_")
|
||||||
|
}
|
||||||
|
|
||||||
|
if subsystem != "" {
|
||||||
|
sb.WriteString(subsystem)
|
||||||
|
sb.WriteString("_")
|
||||||
|
}
|
||||||
|
|
||||||
|
sb.WriteString(name)
|
||||||
|
|
||||||
|
return sb.String()
|
||||||
|
}
|
||||||
|
|
||||||
|
type invalidMetric struct {
|
||||||
|
desc *Desc
|
||||||
|
err error
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewInvalidMetric returns a metric whose Write method always returns the
|
||||||
|
// provided error. It is useful if a Collector finds itself unable to collect
|
||||||
|
// a metric and wishes to report an error to the registry.
|
||||||
|
func NewInvalidMetric(desc *Desc, err error) Metric {
|
||||||
|
return &invalidMetric{desc, err}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *invalidMetric) Desc() *Desc { return m.desc }
|
||||||
|
|
||||||
|
func (m *invalidMetric) Write(*dto.Metric) error { return m.err }
|
||||||
|
|
||||||
|
type timestampedMetric struct {
|
||||||
|
Metric
|
||||||
|
t time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m timestampedMetric) Write(pb *dto.Metric) error {
|
||||||
|
e := m.Metric.Write(pb)
|
||||||
|
pb.TimestampMs = proto.Int64(m.t.Unix()*1000 + int64(m.t.Nanosecond()/1000000))
|
||||||
|
return e
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewMetricWithTimestamp returns a new Metric wrapping the provided Metric in a
|
||||||
|
// way that it has an explicit timestamp set to the provided Time. This is only
|
||||||
|
// useful in rare cases as the timestamp of a Prometheus metric should usually
|
||||||
|
// be set by the Prometheus server during scraping. Exceptions include mirroring
|
||||||
|
// metrics with given timestamps from other metric
|
||||||
|
// sources.
|
||||||
|
//
|
||||||
|
// NewMetricWithTimestamp works best with MustNewConstMetric,
|
||||||
|
// MustNewConstHistogram, and MustNewConstSummary, see example.
|
||||||
|
//
|
||||||
|
// Currently, the exposition formats used by Prometheus are limited to
|
||||||
|
// millisecond resolution. Thus, the provided time will be rounded down to the
|
||||||
|
// next full millisecond value.
|
||||||
|
func NewMetricWithTimestamp(t time.Time, m Metric) Metric {
|
||||||
|
return timestampedMetric{Metric: m, t: t}
|
||||||
|
}
|
||||||
|
|
||||||
|
type withExemplarsMetric struct {
|
||||||
|
Metric
|
||||||
|
|
||||||
|
exemplars []*dto.Exemplar
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *withExemplarsMetric) Write(pb *dto.Metric) error {
|
||||||
|
if err := m.Metric.Write(pb); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
switch {
|
||||||
|
case pb.Counter != nil:
|
||||||
|
pb.Counter.Exemplar = m.exemplars[len(m.exemplars)-1]
|
||||||
|
case pb.Histogram != nil:
|
||||||
|
h := pb.Histogram
|
||||||
|
for _, e := range m.exemplars {
|
||||||
|
if (h.GetZeroThreshold() != 0 || h.GetZeroCount() != 0 ||
|
||||||
|
len(h.PositiveSpan) != 0 || len(h.NegativeSpan) != 0) &&
|
||||||
|
e.GetTimestamp() != nil {
|
||||||
|
h.Exemplars = append(h.Exemplars, e)
|
||||||
|
if len(h.Bucket) == 0 {
|
||||||
|
// Don't proceed to classic buckets if there are none.
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// h.Bucket are sorted by UpperBound.
|
||||||
|
i := sort.Search(len(h.Bucket), func(i int) bool {
|
||||||
|
return h.Bucket[i].GetUpperBound() >= e.GetValue()
|
||||||
|
})
|
||||||
|
if i < len(h.Bucket) {
|
||||||
|
h.Bucket[i].Exemplar = e
|
||||||
|
} else {
|
||||||
|
// The +Inf bucket should be explicitly added if there is an exemplar for it, similar to non-const histogram logic in https://github.com/prometheus/client_golang/blob/main/prometheus/histogram.go#L357-L365.
|
||||||
|
b := &dto.Bucket{
|
||||||
|
CumulativeCount: proto.Uint64(h.GetSampleCount()),
|
||||||
|
UpperBound: proto.Float64(math.Inf(1)),
|
||||||
|
Exemplar: e,
|
||||||
|
}
|
||||||
|
h.Bucket = append(h.Bucket, b)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
// TODO(bwplotka): Implement Gauge?
|
||||||
|
return errors.New("cannot inject exemplar into Gauge, Summary or Untyped")
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Exemplar is easier to use, user-facing representation of *dto.Exemplar.
|
||||||
|
type Exemplar struct {
|
||||||
|
Value float64
|
||||||
|
Labels Labels
|
||||||
|
// Optional.
|
||||||
|
// Default value (time.Time{}) indicates its empty, which should be
|
||||||
|
// understood as time.Now() time at the moment of creation of metric.
|
||||||
|
Timestamp time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewMetricWithExemplars returns a new Metric wrapping the provided Metric with given
|
||||||
|
// exemplars. Exemplars are validated.
|
||||||
|
//
|
||||||
|
// Only last applicable exemplar is injected from the list.
|
||||||
|
// For example for Counter it means last exemplar is injected.
|
||||||
|
// For Histogram, it means last applicable exemplar for each bucket is injected.
|
||||||
|
// For a Native Histogram, all valid exemplars are injected.
|
||||||
|
//
|
||||||
|
// NewMetricWithExemplars works best with MustNewConstMetric and
|
||||||
|
// MustNewConstHistogram, see example.
|
||||||
|
func NewMetricWithExemplars(m Metric, exemplars ...Exemplar) (Metric, error) {
|
||||||
|
if len(exemplars) == 0 {
|
||||||
|
return nil, errors.New("no exemplar was passed for NewMetricWithExemplars")
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
now = time.Now()
|
||||||
|
exs = make([]*dto.Exemplar, len(exemplars))
|
||||||
|
err error
|
||||||
|
)
|
||||||
|
for i, e := range exemplars {
|
||||||
|
ts := e.Timestamp
|
||||||
|
if ts.IsZero() {
|
||||||
|
ts = now
|
||||||
|
}
|
||||||
|
exs[i], err = newExemplar(e.Value, ts, e.Labels)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return &withExemplarsMetric{Metric: m, exemplars: exs}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// MustNewMetricWithExemplars is a version of NewMetricWithExemplars that panics where
|
||||||
|
// NewMetricWithExemplars would have returned an error.
|
||||||
|
func MustNewMetricWithExemplars(m Metric, exemplars ...Exemplar) Metric {
|
||||||
|
ret, err := NewMetricWithExemplars(m, exemplars...)
|
||||||
|
if err != nil {
|
||||||
|
panic(err)
|
||||||
|
}
|
||||||
|
return ret
|
||||||
|
}
|
||||||
25
vendor/github.com/prometheus/client_golang/prometheus/num_threads.go
generated
vendored
Normal file
25
vendor/github.com/prometheus/client_golang/prometheus/num_threads.go
generated
vendored
Normal file
@ -0,0 +1,25 @@
|
|||||||
|
// Copyright 2018 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
//go:build !js || wasm
|
||||||
|
// +build !js wasm
|
||||||
|
|
||||||
|
package prometheus
|
||||||
|
|
||||||
|
import "runtime"
|
||||||
|
|
||||||
|
// getRuntimeNumThreads returns the number of open OS threads.
|
||||||
|
func getRuntimeNumThreads() float64 {
|
||||||
|
n, _ := runtime.ThreadCreateProfile(nil)
|
||||||
|
return float64(n)
|
||||||
|
}
|
||||||
22
vendor/github.com/prometheus/client_golang/prometheus/num_threads_gopherjs.go
generated
vendored
Normal file
22
vendor/github.com/prometheus/client_golang/prometheus/num_threads_gopherjs.go
generated
vendored
Normal file
@ -0,0 +1,22 @@
|
|||||||
|
// Copyright 2018 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
//go:build js && !wasm
|
||||||
|
// +build js,!wasm
|
||||||
|
|
||||||
|
package prometheus
|
||||||
|
|
||||||
|
// getRuntimeNumThreads returns the number of open OS threads.
|
||||||
|
func getRuntimeNumThreads() float64 {
|
||||||
|
return 1
|
||||||
|
}
|
||||||
64
vendor/github.com/prometheus/client_golang/prometheus/observer.go
generated
vendored
Normal file
64
vendor/github.com/prometheus/client_golang/prometheus/observer.go
generated
vendored
Normal file
@ -0,0 +1,64 @@
|
|||||||
|
// Copyright 2017 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus
|
||||||
|
|
||||||
|
// Observer is the interface that wraps the Observe method, which is used by
|
||||||
|
// Histogram and Summary to add observations.
|
||||||
|
type Observer interface {
|
||||||
|
Observe(float64)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The ObserverFunc type is an adapter to allow the use of ordinary
|
||||||
|
// functions as Observers. If f is a function with the appropriate
|
||||||
|
// signature, ObserverFunc(f) is an Observer that calls f.
|
||||||
|
//
|
||||||
|
// This adapter is usually used in connection with the Timer type, and there are
|
||||||
|
// two general use cases:
|
||||||
|
//
|
||||||
|
// The most common one is to use a Gauge as the Observer for a Timer.
|
||||||
|
// See the "Gauge" Timer example.
|
||||||
|
//
|
||||||
|
// The more advanced use case is to create a function that dynamically decides
|
||||||
|
// which Observer to use for observing the duration. See the "Complex" Timer
|
||||||
|
// example.
|
||||||
|
type ObserverFunc func(float64)
|
||||||
|
|
||||||
|
// Observe calls f(value). It implements Observer.
|
||||||
|
func (f ObserverFunc) Observe(value float64) {
|
||||||
|
f(value)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ObserverVec is an interface implemented by `HistogramVec` and `SummaryVec`.
|
||||||
|
type ObserverVec interface {
|
||||||
|
GetMetricWith(Labels) (Observer, error)
|
||||||
|
GetMetricWithLabelValues(lvs ...string) (Observer, error)
|
||||||
|
With(Labels) Observer
|
||||||
|
WithLabelValues(...string) Observer
|
||||||
|
CurryWith(Labels) (ObserverVec, error)
|
||||||
|
MustCurryWith(Labels) ObserverVec
|
||||||
|
|
||||||
|
Collector
|
||||||
|
}
|
||||||
|
|
||||||
|
// ExemplarObserver is implemented by Observers that offer the option of
|
||||||
|
// observing a value together with an exemplar. Its ObserveWithExemplar method
|
||||||
|
// works like the Observe method of an Observer but also replaces the currently
|
||||||
|
// saved exemplar (if any) with a new one, created from the provided value, the
|
||||||
|
// current time as timestamp, and the provided Labels. Empty Labels will lead to
|
||||||
|
// a valid (label-less) exemplar. But if Labels is nil, the current exemplar is
|
||||||
|
// left in place. ObserveWithExemplar panics if any of the provided labels are
|
||||||
|
// invalid or if the provided labels contain more than 128 runes in total.
|
||||||
|
type ExemplarObserver interface {
|
||||||
|
ObserveWithExemplar(value float64, exemplar Labels)
|
||||||
|
}
|
||||||
180
vendor/github.com/prometheus/client_golang/prometheus/process_collector.go
generated
vendored
Normal file
180
vendor/github.com/prometheus/client_golang/prometheus/process_collector.go
generated
vendored
Normal file
@ -0,0 +1,180 @@
|
|||||||
|
// Copyright 2015 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
type processCollector struct {
|
||||||
|
collectFn func(chan<- Metric)
|
||||||
|
describeFn func(chan<- *Desc)
|
||||||
|
pidFn func() (int, error)
|
||||||
|
reportErrors bool
|
||||||
|
cpuTotal *Desc
|
||||||
|
openFDs, maxFDs *Desc
|
||||||
|
vsize, maxVsize *Desc
|
||||||
|
rss *Desc
|
||||||
|
startTime *Desc
|
||||||
|
inBytes, outBytes *Desc
|
||||||
|
}
|
||||||
|
|
||||||
|
// ProcessCollectorOpts defines the behavior of a process metrics collector
|
||||||
|
// created with NewProcessCollector.
|
||||||
|
type ProcessCollectorOpts struct {
|
||||||
|
// PidFn returns the PID of the process the collector collects metrics
|
||||||
|
// for. It is called upon each collection. By default, the PID of the
|
||||||
|
// current process is used, as determined on construction time by
|
||||||
|
// calling os.Getpid().
|
||||||
|
PidFn func() (int, error)
|
||||||
|
// If non-empty, each of the collected metrics is prefixed by the
|
||||||
|
// provided string and an underscore ("_").
|
||||||
|
Namespace string
|
||||||
|
// If true, any error encountered during collection is reported as an
|
||||||
|
// invalid metric (see NewInvalidMetric). Otherwise, errors are ignored
|
||||||
|
// and the collected metrics will be incomplete. (Possibly, no metrics
|
||||||
|
// will be collected at all.) While that's usually not desired, it is
|
||||||
|
// appropriate for the common "mix-in" of process metrics, where process
|
||||||
|
// metrics are nice to have, but failing to collect them should not
|
||||||
|
// disrupt the collection of the remaining metrics.
|
||||||
|
ReportErrors bool
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewProcessCollector is the obsolete version of collectors.NewProcessCollector.
|
||||||
|
// See there for documentation.
|
||||||
|
//
|
||||||
|
// Deprecated: Use collectors.NewProcessCollector instead.
|
||||||
|
func NewProcessCollector(opts ProcessCollectorOpts) Collector {
|
||||||
|
ns := ""
|
||||||
|
if len(opts.Namespace) > 0 {
|
||||||
|
ns = opts.Namespace + "_"
|
||||||
|
}
|
||||||
|
|
||||||
|
c := &processCollector{
|
||||||
|
reportErrors: opts.ReportErrors,
|
||||||
|
cpuTotal: NewDesc(
|
||||||
|
ns+"process_cpu_seconds_total",
|
||||||
|
"Total user and system CPU time spent in seconds.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
openFDs: NewDesc(
|
||||||
|
ns+"process_open_fds",
|
||||||
|
"Number of open file descriptors.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
maxFDs: NewDesc(
|
||||||
|
ns+"process_max_fds",
|
||||||
|
"Maximum number of open file descriptors.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
vsize: NewDesc(
|
||||||
|
ns+"process_virtual_memory_bytes",
|
||||||
|
"Virtual memory size in bytes.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
maxVsize: NewDesc(
|
||||||
|
ns+"process_virtual_memory_max_bytes",
|
||||||
|
"Maximum amount of virtual memory available in bytes.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
rss: NewDesc(
|
||||||
|
ns+"process_resident_memory_bytes",
|
||||||
|
"Resident memory size in bytes.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
startTime: NewDesc(
|
||||||
|
ns+"process_start_time_seconds",
|
||||||
|
"Start time of the process since unix epoch in seconds.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
inBytes: NewDesc(
|
||||||
|
ns+"process_network_receive_bytes_total",
|
||||||
|
"Number of bytes received by the process over the network.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
outBytes: NewDesc(
|
||||||
|
ns+"process_network_transmit_bytes_total",
|
||||||
|
"Number of bytes sent by the process over the network.",
|
||||||
|
nil, nil,
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
if opts.PidFn == nil {
|
||||||
|
c.pidFn = getPIDFn()
|
||||||
|
} else {
|
||||||
|
c.pidFn = opts.PidFn
|
||||||
|
}
|
||||||
|
|
||||||
|
// Set up process metric collection if supported by the runtime.
|
||||||
|
if canCollectProcess() {
|
||||||
|
c.collectFn = c.processCollect
|
||||||
|
c.describeFn = c.describe
|
||||||
|
} else {
|
||||||
|
c.collectFn = c.errorCollectFn
|
||||||
|
c.describeFn = c.errorDescribeFn
|
||||||
|
}
|
||||||
|
|
||||||
|
return c
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *processCollector) errorCollectFn(ch chan<- Metric) {
|
||||||
|
c.reportError(ch, nil, errors.New("process metrics not supported on this platform"))
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *processCollector) errorDescribeFn(ch chan<- *Desc) {
|
||||||
|
if c.reportErrors {
|
||||||
|
ch <- NewInvalidDesc(errors.New("process metrics not supported on this platform"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Collect returns the current state of all metrics of the collector.
|
||||||
|
func (c *processCollector) Collect(ch chan<- Metric) {
|
||||||
|
c.collectFn(ch)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Describe returns all descriptions of the collector.
|
||||||
|
func (c *processCollector) Describe(ch chan<- *Desc) {
|
||||||
|
c.describeFn(ch)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *processCollector) reportError(ch chan<- Metric, desc *Desc, err error) {
|
||||||
|
if !c.reportErrors {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if desc == nil {
|
||||||
|
desc = NewInvalidDesc(err)
|
||||||
|
}
|
||||||
|
ch <- NewInvalidMetric(desc, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewPidFileFn returns a function that retrieves a pid from the specified file.
|
||||||
|
// It is meant to be used for the PidFn field in ProcessCollectorOpts.
|
||||||
|
func NewPidFileFn(pidFilePath string) func() (int, error) {
|
||||||
|
return func() (int, error) {
|
||||||
|
content, err := os.ReadFile(pidFilePath)
|
||||||
|
if err != nil {
|
||||||
|
return 0, fmt.Errorf("can't read pid file %q: %w", pidFilePath, err)
|
||||||
|
}
|
||||||
|
pid, err := strconv.Atoi(strings.TrimSpace(string(content)))
|
||||||
|
if err != nil {
|
||||||
|
return 0, fmt.Errorf("can't parse pid file %q: %w", pidFilePath, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return pid, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user