unkin-agent c20c6fa836
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
docs: reword reports/events union without old/new framing
2026-09-05 11:56:50 +10:00

pdbmux — merging PuppetDB proxy

pdbmux is a small HTTP daemon that fronts several PuppetDB backends and serves a single, merged PuppetDB v4 query surface on one address. Point Puppetboard, or any other PuppetDB API client, at pdbmux instead of a raw PuppetDB and it sees one consistent view spanning all of them.

Why

Running more than one PuppetDB — during a migration between two of them, or across regions — means a given node's current data lives in exactly one at any moment, and consumers have to know which, or query each in turn. pdbmux merges them all so consumers don't have to know (or query twice) which PuppetDB a node currently lives in.

All backends are equal — pdbmux is never told which one to favour. Backend names are arbitrary labels and there is no fixed number of them. The configured order is used only as a tie-break, so output is reproducible.

Endpoints

pdbmux proxies GET requests only. The query param (PuppetDB AST JSON, not PQL) is forwarded verbatim.

Path Behaviour
GET /pdb/query/v4/nodes Fan out to all backends, dedupe by certname, keep the record with the newer report_timestamp.
GET /pdb/query/v4/facts Fan out to all, and per certname keep all facts from the backend that owns that node (see merge semantics).
GET /pdb/query/v4/reports Fan out to all and serve the union, deduped by report hash, re-ordered and re-paged across backends.
GET /pdb/query/v4/events Fan out to all and serve the union, deduped by record identity, re-ordered and re-paged.
GET /pdb/query/v4/reports/<hash>/{events,logs,metrics} Ask every backend; serve the answer from whichever backend actually holds that report. 404 when none does.
GET /pdb/query/v4/* (any other) No merge rule, so backends are tried in configured order and the first success is streamed back verbatim; if all reject it, the first upstream error response is replayed.
GET /healthz Per-backend reachability. 200 {"status":"ok"} if all reachable, 200 degraded if some fail, 503 down if all fail.

Fan-out is concurrent. If one backend errors or times out, pdbmux serves the surviving backends' results and logs a warning; a merged endpoint only returns 502 when every backend fails. Response records are passed through as raw JSON so unknown fields survive untouched.

Merge semantics

  • /nodes — dedupe by certname; the record with the strictly-newer report_timestamp wins. On a tie, the backend listed first in backends supplies the record — a tie-break only, so the merged output is deterministic.
  • /facts — node-level granularity. For a certname present in more than one backend, pdbmux keeps all of that node's facts from one backend and drops the others', chosen by the merge strategy:
    • freshness (default) — attribute each certname to whichever backend holds its newer report_timestamp. pdbmux derives this from a per-certname freshness map built by querying /nodes from every backend, cached for freshness_ttl (default 30s).
    • static — skip the extra /nodes query and take each shared node's facts from the first backend in configured order that holds it.
    • A node present in only one backend always appears (falls back to whichever backend actually returned facts for it).
  • /reports, /eventsunion, not a per-node winner. Reports are immutable history, so a node's reports can legitimately exist in more than one backend and all of them belong in the merged view. Reports dedupe on hash; events, which carry no id of their own, dedupe on the verbatim record (a node reporting to more than one backend stores identical records in each). Records the merge cannot key — extract/group_by aggregate rows — are never deduped, so every backend's rows pass through even when byte-identical; summing those aggregates across backends is not implemented yet.

Paging and ordering on the merged endpoints

Each backend applies order_by/limit/offset to its own slice only, so pdbmux re-does all three over the union:

  • order_by is parsed and the merged set re-sorted by those fields (ties keep the merged set's existing order). A record missing an ordered field sorts first.
  • Backends are asked for the first offset + limit records — never an offset — and the requested window is then cut from the merged, re-sorted set.
  • include_total=true makes pdbmux sum each backend's X-Records header into one merged header. Deduped records are counted once per backend, so the total is an upper bound.
  • A malformed limit, offset or order_by gets a 400 rather than being forwarded.

Config

Precedence (lowest → highest): defaults < config file < env vars (PDBMUX_*) < flags.

Config file: $XDG_CONFIG_HOME/pdbmux/config.yaml. In a container, configuration is supplied entirely via PDBMUX_* env vars (no config file) — see Deployment.

backends has no default. pdbmux refuses to start until at least one backend is configured, via the config file or PDBMUX_BACKENDS.

# ~/.config/pdbmux/config.yaml  (local dev; in a container use PDBMUX_* env instead)
listen: ":8080"
backends:                          # order is a tie-break only, not a ranking
  - name: pdb-a
    url: http://puppetdb1.example.com:8080
  - name: pdb-b
    url: https://puppetdb2.example.com
merge: freshness      # freshness | static
timeout: 10s          # per-upstream request timeout
freshness_ttl: 30s    # freshness-map cache TTL (freshness merge only)

backends[*].url is a base URL (scheme://host[:port]), without the /pdb/query/v4/... path — pdbmux appends the path per request.

Env var Overrides
PDBMUX_LISTEN listen
PDBMUX_MERGE merge
PDBMUX_TIMEOUT timeout (Go duration, e.g. 10s)
PDBMUX_FRESHNESS_TTL freshness_ttl
PDBMUX_BACKENDS whole backend list, as name=url,name=url

Flags: --listen, --merge.

Running

pdbmux                 # start the proxy (serve is the default action)
pdbmux serve           # explicit
pdbmux config init     # write an example config file to edit
pdbmux config show     # print active config after all overrides
pdbmux version

Point a consumer at it — any PuppetDB v4 client works, it just needs the pdbmux base URL in place of a PuppetDB one. For Puppetboard, that is its PuppetDB host/port setting; for a raw query:

curl -s --get http://localhost:8080/pdb/query/v4/nodes \
  --data-urlencode 'query=["=","certname","host1.example.com"]'

Build

make build             # -> dist/pdbmux (CGO disabled, static)
make test              # go test -race ./...
make lint              # golangci-lint

Requires Go 1.25+. Dependencies: github.com/spf13/cobra (CLI), gopkg.in/yaml.v3 (config file).

Deployment

pdbmux is distributed as a container image only — there is no OS package. The image is built and pushed on every v* tag (.woodpecker/docker.yaml); the registry and repository are pipeline settings, so point them at your own.

It is a minimal static (CGO_ENABLED=0) binary on a distroless base (Dockerfile), configured entirely via PDBMUX_* env vars, with a single HTTP listener and /healthz for liveness/readiness probes. It is stateless, so run as many replicas as you like behind an ordinary Service/Ingress.

A container needs at minimum PDBMUX_BACKENDS; everything else has a default:

env:
  - name: PDBMUX_BACKENDS
    value: "pdb-a=http://puppetdb1.example.com:8080,pdb-b=http://puppetdb2.example.com:8080"

Locally you can run the binary directly for development:

PDBMUX_BACKENDS='pdb-a=http://puppetdb1.example.com:8080,pdb-b=http://puppetdb2.example.com:8080' \
  pdbmux serve
curl -s localhost:8080/healthz

Version bumps

make patch             # tag vX.Y.(Z+1) and push (triggers the docker release)
make minor             # tag vX.(Y+1).0
make major             # tag v(X+1).0.0
S
Description
Merging HTTP proxy over two PuppetDB backends, presenting a single merged PuppetDB v4 query surface during the VM to k8s Puppet migration. Deployed in-cluster via argocd-apps.
Readme 2.2 MiB
Languages
Go 99.7%
Makefile 0.2%