Add pdbmux: a merging PuppetDB proxy for the VM->k8s migration
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful

During the VM->k8s Puppet migration there are two PuppetDBs - the legacy
Consul-registered one (http://puppetdbapi.service.consul:8080) and the new
k8s one (https://puppetdb.k8s.syd1.au.unkin.net) - and nodes move between
them as they migrate. node-lookup and pblastreport need a single, consistent
merged view without knowing which PuppetDB a node currently lives in.

This adds pdbmux, a small HTTP daemon that fronts both backends:

- Adds cmd/pdbmux/ (config.go, merge.go, server.go, main.go): a cobra tool
  whose default action (also `serve`) starts the proxy, plus config init/show
  and version subcommands, following the repo's config precedence pattern
  (defaults < config file < env PDBMUX_* < flags).
- Merges GET /pdb/query/v4/nodes: dedupes by certname, keeping the record with
  the newer report_timestamp.
- Merges GET /pdb/query/v4/facts at node granularity: keeps all facts from the
  backend owning each certname, chosen by the freshness strategy (per-certname
  report_timestamp map from /nodes, cached for freshness_ttl) or a static
  prefer-backend fallback.
- Fans out to both backends concurrently, serves the survivor if one fails, and
  returns 502 only when both fail; passes records through as raw JSON so unknown
  fields survive.
- Transparently proxies any other /pdb/query/v4/* path to the configurable
  primary, and exposes /healthz with per-backend reachability (200 ok /
  200 degraded / 503 down).
- Adds table-driven tests (go test -race, no network) covering merge logic,
  handler behaviour with httptest backends, query passthrough, one/both backend
  down, and config precedence/validation.
- Wires pdbmux into the build/release: Makefile BINARIES, scripts/build-rpm.sh,
  nfpm packaging (binary + completions + a systemd unit), and the release
  pipeline's cross-platform build + Gitea asset list.
- Documents pdbmux (what/why/endpoints/merge-semantics/config/deployment) in a
  new README.md and updates AGENTS.md.
This commit is contained in:
Ben Vincent
2026-07-24 22:50:18 +10:00
parent 2aaffd7e31
commit 2aa94f0de7
15 changed files with 1743 additions and 9 deletions
+2 -2
View File
@@ -12,7 +12,7 @@ cd "${ROOT_DIR}"
VERSION="${1:-${CI_COMMIT_TAG:-0.0.0-dev}}"
VERSION="${VERSION#v}" # strip a leading v
BINARY="node-lookup"
BINARIES=(node-lookup pburl pblastreport)
BINARIES=(node-lookup pburl pblastreport pdbmux)
DIST="dist"
for b in "${BINARIES[@]}"; do
@@ -37,7 +37,7 @@ export PACKAGE_VERSION="${VERSION}"
export PACKAGE_RELEASE="1"
export PACKAGE_ARCH="amd64"
export PACKAGE_PLATFORM="linux"
export PACKAGE_DESCRIPTION="CLI tools for PuppetDB: node-lookup (fact lookup/filtering) plus pburl and pblastreport (Puppetboard URLs and last-report times)"
export PACKAGE_DESCRIPTION="CLI tools for PuppetDB: node-lookup (fact lookup/filtering), pburl and pblastreport (Puppetboard URLs and last-report times), plus pdbmux (merging proxy over the old + new PuppetDB during migration)"
export PACKAGE_MAINTAINER="Ben Vincent <ben@unkin.net>"
export PACKAGE_HOMEPAGE="https://git.unkin.net/unkin/node-lookup"
export PACKAGE_LICENSE="MIT"