Ben Vincent
2aa94f0de7
During the VM->k8s Puppet migration there are two PuppetDBs - the legacy Consul-registered one (http://puppetdbapi.service.consul:8080) and the new k8s one (https://puppetdb.k8s.syd1.au.unkin.net) - and nodes move between them as they migrate. node-lookup and pblastreport need a single, consistent merged view without knowing which PuppetDB a node currently lives in. This adds pdbmux, a small HTTP daemon that fronts both backends: - Adds cmd/pdbmux/ (config.go, merge.go, server.go, main.go): a cobra tool whose default action (also `serve`) starts the proxy, plus config init/show and version subcommands, following the repo's config precedence pattern (defaults < config file < env PDBMUX_* < flags). - Merges GET /pdb/query/v4/nodes: dedupes by certname, keeping the record with the newer report_timestamp. - Merges GET /pdb/query/v4/facts at node granularity: keeps all facts from the backend owning each certname, chosen by the freshness strategy (per-certname report_timestamp map from /nodes, cached for freshness_ttl) or a static prefer-backend fallback. - Fans out to both backends concurrently, serves the survivor if one fails, and returns 502 only when both fail; passes records through as raw JSON so unknown fields survive. - Transparently proxies any other /pdb/query/v4/* path to the configurable primary, and exposes /healthz with per-backend reachability (200 ok / 200 degraded / 503 down). - Adds table-driven tests (go test -race, no network) covering merge logic, handler behaviour with httptest backends, query passthrough, one/both backend down, and config precedence/validation. - Wires pdbmux into the build/release: Makefile BINARIES, scripts/build-rpm.sh, nfpm packaging (binary + completions + a systemd unit), and the release pipeline's cross-platform build + Gitea asset list. - Documents pdbmux (what/why/endpoints/merge-semantics/config/deployment) in a new README.md and updates AGENTS.md.
138 lines
5.3 KiB
Markdown
138 lines
5.3 KiB
Markdown
# node-lookup tools
|
|
|
|
PuppetDB CLIs and one proxy daemon, shipped together in a single RPM:
|
|
|
|
- **`node-lookup`** — query and filter PuppetDB node facts.
|
|
- **`pburl`** — print each host's Puppetboard node-page URL.
|
|
- **`pblastreport`** — print each host's last Puppet report time + Puppetboard URL.
|
|
- **`pdbmux`** — merging HTTP proxy over two PuppetDBs (see below).
|
|
|
|
See [AGENTS.md](AGENTS.md) for the CLI tools' flags, config, and internals. This
|
|
README covers **pdbmux**.
|
|
|
|
---
|
|
|
|
## pdbmux — merging PuppetDB proxy
|
|
|
|
### What
|
|
|
|
`pdbmux` is a small HTTP daemon that fronts **two** PuppetDB backends and serves
|
|
a single, merged PuppetDB v4 query surface on one address. Point `node-lookup`,
|
|
`pblastreport`, or anything else at `pdbmux` instead of a raw PuppetDB and it
|
|
sees one consistent view spanning both.
|
|
|
|
### Why
|
|
|
|
During the VM→k8s Puppet migration there are two PuppetDBs:
|
|
|
|
- **old** — the legacy Consul-registered `http://puppetdbapi.service.consul:8080`
|
|
- **new** — the k8s `https://puppetdb.k8s.syd1.au.unkin.net` (TLS terminated at
|
|
the gateway; backends are plain PuppetDB on 8080)
|
|
|
|
Nodes move from old to new as they migrate, so at any moment a given node's
|
|
current data lives in exactly one of them. `pdbmux` merges both so consumers
|
|
don't have to know (or query twice) which PuppetDB a node currently lives in.
|
|
|
|
### Endpoints
|
|
|
|
`pdbmux` proxies **GET** requests only. The `query` param (PuppetDB AST JSON,
|
|
not PQL) is forwarded verbatim.
|
|
|
|
| Path | Behaviour |
|
|
|---|---|
|
|
| `GET /pdb/query/v4/nodes` | Fan out to both backends, dedupe by `certname`, keep the record with the newer `report_timestamp`. |
|
|
| `GET /pdb/query/v4/facts` | Fan out to both, and per `certname` keep **all** facts from the backend that owns that node (see merge semantics). |
|
|
| `GET /pdb/query/v4/*` (any other) | Transparently proxied to the **primary** backend, unmerged, streamed verbatim. |
|
|
| `GET /healthz` | Per-backend reachability. `200 {"status":"ok"}` if all reachable, `200 degraded` if some fail, `503 down` if all fail. |
|
|
|
|
Fan-out is concurrent. If one backend errors or times out, `pdbmux` serves the
|
|
survivor's results and logs a warning; a merged endpoint only returns `502` when
|
|
**every** backend fails. Response records are passed through as raw JSON so
|
|
unknown fields survive untouched.
|
|
|
|
### Merge semantics
|
|
|
|
- **`/nodes`** — dedupe by `certname`; the record with the strictly-newer
|
|
`report_timestamp` wins. On a tie (or when a node exists in only one backend),
|
|
the **preferred** backend's record is kept.
|
|
- **`/facts`** — node-level granularity. For a `certname` present in both
|
|
backends, `pdbmux` keeps **all** of that node's facts from **one** backend and
|
|
drops the other's, chosen by the merge strategy:
|
|
- **`freshness`** (default) — attribute each `certname` to whichever backend
|
|
holds its newer `report_timestamp`. `pdbmux` derives this from a per-certname
|
|
freshness map built by querying `/nodes` from both backends, cached for
|
|
`freshness_ttl` (default 30s). Ties/fallbacks use `prefer`.
|
|
- **`static`** — always keep the `prefer` backend's facts for shared nodes.
|
|
No extra `/nodes` query.
|
|
- A node present in only one backend always appears (falls back to whichever
|
|
backend actually returned facts for it).
|
|
|
|
### Config
|
|
|
|
Precedence (lowest → highest): **defaults < config file < env vars (`PDBMUX_*`) < flags**.
|
|
|
|
Config file: `$XDG_CONFIG_HOME/pdbmux/config.yaml` (as an RPM/systemd service:
|
|
`/etc/pdbmux/config.yaml`).
|
|
|
|
```yaml
|
|
# ~/.config/pdbmux/config.yaml (or /etc/pdbmux/config.yaml under systemd)
|
|
listen: ":8080"
|
|
backends:
|
|
- name: old
|
|
url: http://puppetdbapi.service.consul:8080
|
|
- name: new
|
|
url: https://puppetdb.k8s.syd1.au.unkin.net
|
|
primary: new # backend used for non-merged /pdb/query/v4/* pass-through
|
|
merge: freshness # freshness | static
|
|
prefer: new # winner on ties / static merge / fallback
|
|
timeout: 10s # per-upstream request timeout
|
|
freshness_ttl: 30s # freshness-map cache TTL (freshness merge only)
|
|
```
|
|
|
|
`backends[*].url` is a **base** URL (`scheme://host[:port]`), without the
|
|
`/pdb/query/v4/...` path — `pdbmux` appends the path per request.
|
|
|
|
| Env var | Overrides |
|
|
|---|---|
|
|
| `PDBMUX_LISTEN` | `listen` |
|
|
| `PDBMUX_PRIMARY` | `primary` |
|
|
| `PDBMUX_MERGE` | `merge` |
|
|
| `PDBMUX_PREFER` | `prefer` |
|
|
| `PDBMUX_TIMEOUT` | `timeout` (Go duration, e.g. `10s`) |
|
|
| `PDBMUX_FRESHNESS_TTL` | `freshness_ttl` |
|
|
| `PDBMUX_BACKENDS` | whole backend list, as `name=url,name=url` |
|
|
|
|
Flags: `--listen`, `--primary`, `--merge`.
|
|
|
|
### Running
|
|
|
|
```bash
|
|
pdbmux # start the proxy (serve is the default action)
|
|
pdbmux serve # explicit
|
|
pdbmux config init # write a default config file
|
|
pdbmux config show # print active config after all overrides
|
|
pdbmux version
|
|
```
|
|
|
|
Point a consumer at it:
|
|
|
|
```bash
|
|
node-lookup --url http://localhost:8080/pdb/query/v4/facts -R
|
|
NODE_LOOKUP_URL=http://localhost:8080/pdb/query/v4/facts pblastreport somehost
|
|
```
|
|
|
|
### Deployment
|
|
|
|
The RPM installs `/usr/bin/pdbmux` plus a systemd unit at
|
|
`/usr/lib/systemd/system/pdbmux.service` (reads `/etc/pdbmux/config.yaml`):
|
|
|
|
```bash
|
|
systemctl enable --now pdbmux
|
|
curl -s localhost:8080/healthz
|
|
```
|
|
|
|
Suited to run as an RPM-managed daemon on a VM alongside the existing Puppet
|
|
infra during migration; the same static binary is container-ready for a future
|
|
k8s deployment (single listener, `/healthz` liveness/readiness probe, config via
|
|
`PDBMUX_*` env).
|