Add pdbmux: a merging PuppetDB proxy for the VM->k8s migration
During the VM->k8s Puppet migration there are two PuppetDBs - the legacy Consul-registered one (http://puppetdbapi.service.consul:8080) and the new k8s one (https://puppetdb.k8s.syd1.au.unkin.net) - and nodes move between them as they migrate. node-lookup and pblastreport need a single, consistent merged view without knowing which PuppetDB a node currently lives in. This adds pdbmux, a small HTTP daemon that fronts both backends: - Adds cmd/pdbmux/ (config.go, merge.go, server.go, main.go): a cobra tool whose default action (also `serve`) starts the proxy, plus config init/show and version subcommands, following the repo's config precedence pattern (defaults < config file < env PDBMUX_* < flags). - Merges GET /pdb/query/v4/nodes: dedupes by certname, keeping the record with the newer report_timestamp. - Merges GET /pdb/query/v4/facts at node granularity: keeps all facts from the backend owning each certname, chosen by the freshness strategy (per-certname report_timestamp map from /nodes, cached for freshness_ttl) or a static prefer-backend fallback. - Fans out to both backends concurrently, serves the survivor if one fails, and returns 502 only when both fail; passes records through as raw JSON so unknown fields survive. - Transparently proxies any other /pdb/query/v4/* path to the configurable primary, and exposes /healthz with per-backend reachability (200 ok / 200 degraded / 503 down). - Adds table-driven tests (go test -race, no network) covering merge logic, handler behaviour with httptest backends, query passthrough, one/both backend down, and config precedence/validation. - Wires pdbmux into the build/release: Makefile BINARIES, scripts/build-rpm.sh, nfpm packaging (binary + completions + a systemd unit), and the release pipeline's cross-platform build + Gitea asset list. - Documents pdbmux (what/why/endpoints/merge-semantics/config/deployment) in a new README.md and updates AGENTS.md.
This commit is contained in:
@@ -2,7 +2,7 @@
|
||||
|
||||
## Project Overview
|
||||
|
||||
This repo ships three related Puppet CLIs in one RPM:
|
||||
This repo ships four related Puppet tools in one RPM:
|
||||
|
||||
- **`node-lookup`** — queries the PuppetDB API to retrieve and filter node facts.
|
||||
- **`pburl`** — prints the Puppetboard node-page URL for each host (reads hosts
|
||||
@@ -10,10 +10,16 @@ This repo ships three related Puppet CLIs in one RPM:
|
||||
- **`pblastreport`** — prints each host's last Puppet report time and its
|
||||
Puppetboard URL. Output: `<host>\t<time>\t<url>`. Supports `--relative`/`-r`
|
||||
(relative age) and `--timezone`/`-z <IANA>` (default: local timezone).
|
||||
- **`pdbmux`** — a long-running HTTP daemon that presents a single merged
|
||||
PuppetDB v4 query surface over the old (Consul) and new (k8s) PuppetDBs during
|
||||
the VM→k8s migration. Merges `/pdb/query/v4/{nodes,facts}`, transparently
|
||||
proxies other v4 paths to the primary, and exposes `/healthz`. See README.md
|
||||
for full config/merge semantics.
|
||||
|
||||
`node-lookup` is the module root; `pburl` and `pblastreport` live under `cmd/`
|
||||
and share the `internal/puppet` package (config, PuppetDB `nodes` queries,
|
||||
Puppetboard URL construction, stdin host reading).
|
||||
`node-lookup` is the module root; `pburl`, `pblastreport` and `pdbmux` live
|
||||
under `cmd/`. The three CLI tools share the `internal/puppet` package (config,
|
||||
PuppetDB `nodes` queries, Puppetboard URL construction, stdin host reading);
|
||||
`pdbmux` is self-contained (its own config + HTTP server).
|
||||
|
||||
## Structure
|
||||
|
||||
@@ -22,11 +28,13 @@ main.go # node-lookup CLI source (module root, package mai
|
||||
main_test.go # node-lookup unit tests (mock PuppetDB via httptest)
|
||||
cmd/pburl/main.go # pburl CLI
|
||||
cmd/pblastreport/main.go # pblastreport CLI (report.go: report-time formatting)
|
||||
cmd/pdbmux/ # pdbmux daemon: main.go, config.go, merge.go, server.go
|
||||
internal/puppet/ # shared: config, puppetdb nodes query, board URLs, stdin
|
||||
go.mod # Go module (module name: node-lookup)
|
||||
go.sum # dependency checksums
|
||||
Makefile # build / test / lint / completions / rpm / version-bump targets
|
||||
packaging/nfpm.yaml # nfpm spec (envsubst-templated) for the RPM (all 3 binaries)
|
||||
packaging/nfpm.yaml # nfpm spec (envsubst-templated) for the RPM (all 4 binaries)
|
||||
packaging/pdbmux.service # systemd unit for the pdbmux daemon
|
||||
scripts/build-rpm.sh # generates completions + packages the RPM with nfpm
|
||||
.woodpecker/ # CI: build, test, pre-commit (PR) + release (tag)
|
||||
dist/ # build output: binaries, completions, RPM (not committed)
|
||||
|
||||
Reference in New Issue
Block a user