unkin/node-lookup
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 19 Wiki Activity

Add pdbmux: a merging PuppetDB proxy for the VM->k8s migration #17

Closed
unkinben wants to merge 2 commits from benvin/pdbmux into main
pull from: benvin/pdbmux
merge into: unkin:main
Conversation 2 Commits 2 Files Changed 16
+1783 -9
unkinben commented 2026-07-24 22:50:54 +10:00
Owner
Copy Link
Copy Source

Why

During the VM→k8s Puppet migration there are two PuppetDBs — the legacy Consul-registered http://puppetdbapi.service.consul:8080 and the new k8s https://puppetdb.k8s.syd1.au.unkin.net — and nodes move from one to the other as they migrate. node-lookup and pblastreport need a single, consistent merged view without having to know (or query twice) which PuppetDB a given node currently lives in.

This adds pdbmux, a small HTTP daemon that fronts both backends and serves one merged PuppetDB v4 query surface.

Changes

  • Adds cmd/pdbmux/ (config.go, merge.go, server.go, main.go): a cobra tool whose default action (also serve) starts the proxy, plus config init/config show and version subcommands, following the repo's config precedence pattern (defaults < config file < env PDBMUX_* < flags).
  • Merges GET /pdb/query/v4/nodes: dedupes by certname, keeping the record with the newer report_timestamp.
  • Merges GET /pdb/query/v4/facts at node granularity: keeps all facts from the backend owning each certname, chosen by the freshness strategy (per-certname report_timestamp map from /nodes, cached for freshness_ttl, default 30s) or a static prefer-backend fallback.
  • Fans out to both backends concurrently, serves the survivor if one fails, and returns 502 only when both fail; passes records through as raw JSON so unknown fields survive.
  • Transparently proxies any other /pdb/query/v4/* path to the configurable primary, and exposes /healthz with per-backend reachability (200 ok / 200 degraded / 503 down).
  • Adds table-driven tests (go test -race, no network) covering merge logic (both-have-node newer-wins, one-backend-only, one-backend-down, both-down 502), handler behaviour with httptest backends, query-param passthrough, and config precedence/validation.
  • Wires pdbmux into build/release: Makefile BINARIES, scripts/build-rpm.sh, nfpm packaging (binary + completions + a systemd unit), and the release pipeline's cross-platform build + Gitea asset list.
  • Documents pdbmux in a new README.md and updates AGENTS.md.

Config example

# /etc/pdbmux/config.yaml (systemd) or ~/.config/pdbmux/config.yaml
listen: ":8080"
backends:
  - name: old
    url: http://puppetdbapi.service.consul:8080
  - name: new
    url: https://puppetdb.k8s.syd1.au.unkin.net
primary: new          # non-merged /pdb/query/v4/* pass-through target
merge: freshness      # freshness | static
prefer: new           # tie / static / fallback winner
timeout: 10s
freshness_ttl: 30s

Env overrides: PDBMUX_LISTEN, PDBMUX_PRIMARY, PDBMUX_MERGE, PDBMUX_PREFER, PDBMUX_TIMEOUT, PDBMUX_FRESHNESS_TTL, PDBMUX_BACKENDS (name=url,name=url).

Merge semantics

  • /nodes — dedupe by certname; strictly-newer report_timestamp wins; ties/single-backend nodes fall back to prefer.
  • /facts — per certname, keep all facts from one backend: the one holding that node's newer report (freshness, from a short-TTL /nodes map) or the prefer backend (static). A node present in only one backend always appears.
  • other v4 paths — pass through verbatim to primary; only GET is proxied.
  • One backend down → serve the other + log a warning; both down → 502.

Verification

  • go build ./..., go vet ./..., go test -race ./... all pass; gofmt clean; pre-commit hooks pass.
  • make build + make completions build all four binaries and pdbmux completions; scripts/build-rpm.sh produces an RPM containing /usr/bin/pdbmux, the systemd unit, and completions.
  • Smoke-tested locally against both live PuppetDBs (read-only GETs): /healthz → {"status":"ok"} with both backends ok; /nodes and /facts proxy and merge (HTTP 200).

Deployment

Ships an RPM (/usr/bin/pdbmux + pdbmux.service reading /etc/pdbmux/config.yaml) suited to run as a systemd daemon on a VM during migration. The same static binary is container-ready for a future k8s deployment (single listener, /healthz probe, PDBMUX_* env config).

## Why During the VM→k8s Puppet migration there are **two** PuppetDBs — the legacy Consul-registered `http://puppetdbapi.service.consul:8080` and the new k8s `https://puppetdb.k8s.syd1.au.unkin.net` — and nodes move from one to the other as they migrate. `node-lookup` and `pblastreport` need a single, consistent merged view without having to know (or query twice) which PuppetDB a given node currently lives in. This adds **`pdbmux`**, a small HTTP daemon that fronts both backends and serves one merged PuppetDB v4 query surface. ## Changes - Adds `cmd/pdbmux/` (`config.go`, `merge.go`, `server.go`, `main.go`): a cobra tool whose default action (also `serve`) starts the proxy, plus `config init`/`config show` and `version` subcommands, following the repo's config precedence pattern (defaults < config file < env `PDBMUX_*` < flags). - Merges `GET /pdb/query/v4/nodes`: dedupes by `certname`, keeping the record with the newer `report_timestamp`. - Merges `GET /pdb/query/v4/facts` at node granularity: keeps **all** facts from the backend owning each `certname`, chosen by the **freshness** strategy (per-certname `report_timestamp` map from `/nodes`, cached for `freshness_ttl`, default 30s) or a **static** prefer-backend fallback. - Fans out to both backends concurrently, serves the survivor if one fails, and returns `502` only when both fail; passes records through as raw JSON so unknown fields survive. - Transparently proxies any other `/pdb/query/v4/*` path to the configurable **primary**, and exposes `/healthz` with per-backend reachability (`200 ok` / `200 degraded` / `503 down`). - Adds table-driven tests (`go test -race`, no network) covering merge logic (both-have-node newer-wins, one-backend-only, one-backend-down, both-down 502), handler behaviour with `httptest` backends, query-param passthrough, and config precedence/validation. - Wires `pdbmux` into build/release: `Makefile` `BINARIES`, `scripts/build-rpm.sh`, nfpm packaging (binary + completions + a systemd unit), and the release pipeline's cross-platform build + Gitea asset list. - Documents `pdbmux` in a new `README.md` and updates `AGENTS.md`. ## Config example ```yaml # /etc/pdbmux/config.yaml (systemd) or ~/.config/pdbmux/config.yaml listen: ":8080" backends: - name: old url: http://puppetdbapi.service.consul:8080 - name: new url: https://puppetdb.k8s.syd1.au.unkin.net primary: new # non-merged /pdb/query/v4/* pass-through target merge: freshness # freshness | static prefer: new # tie / static / fallback winner timeout: 10s freshness_ttl: 30s ``` Env overrides: `PDBMUX_LISTEN`, `PDBMUX_PRIMARY`, `PDBMUX_MERGE`, `PDBMUX_PREFER`, `PDBMUX_TIMEOUT`, `PDBMUX_FRESHNESS_TTL`, `PDBMUX_BACKENDS` (`name=url,name=url`). ## Merge semantics - **`/nodes`** — dedupe by `certname`; strictly-newer `report_timestamp` wins; ties/single-backend nodes fall back to `prefer`. - **`/facts`** — per `certname`, keep **all** facts from one backend: the one holding that node's newer report (`freshness`, from a short-TTL `/nodes` map) or the `prefer` backend (`static`). A node present in only one backend always appears. - **other v4 paths** — pass through verbatim to `primary`; **only GET** is proxied. - One backend down → serve the other + log a warning; both down → `502`. ## Verification - `go build ./...`, `go vet ./...`, `go test -race ./...` all pass; `gofmt` clean; pre-commit hooks pass. - `make build` + `make completions` build all four binaries and pdbmux completions; `scripts/build-rpm.sh` produces an RPM containing `/usr/bin/pdbmux`, the systemd unit, and completions. - Smoke-tested locally against both live PuppetDBs (read-only GETs): `/healthz` → `{"status":"ok"}` with both backends `ok`; `/nodes` and `/facts` proxy and merge (HTTP 200). ## Deployment Ships an RPM (`/usr/bin/pdbmux` + `pdbmux.service` reading `/etc/pdbmux/config.yaml`) suited to run as a systemd daemon on a VM during migration. The same static binary is container-ready for a future k8s deployment (single listener, `/healthz` probe, `PDBMUX_*` env config).
unkinben added 1 commit 2026-07-24 22:50:54 +10:00
Add pdbmux: a merging PuppetDB proxy for the VM->k8s migration
ci/woodpecker/pr/build Pipeline was successful
Details
ci/woodpecker/pr/test Pipeline was successful
Details
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
2aa94f0de7 
During the VM->k8s Puppet migration there are two PuppetDBs - the legacy
Consul-registered one (http://puppetdbapi.service.consul:8080) and the new
k8s one (https://puppetdb.k8s.syd1.au.unkin.net) - and nodes move between
them as they migrate. node-lookup and pblastreport need a single, consistent
merged view without knowing which PuppetDB a node currently lives in.

This adds pdbmux, a small HTTP daemon that fronts both backends:

- Adds cmd/pdbmux/ (config.go, merge.go, server.go, main.go): a cobra tool
  whose default action (also `serve`) starts the proxy, plus config init/show
  and version subcommands, following the repo's config precedence pattern
  (defaults < config file < env PDBMUX_* < flags).
- Merges GET /pdb/query/v4/nodes: dedupes by certname, keeping the record with
  the newer report_timestamp.
- Merges GET /pdb/query/v4/facts at node granularity: keeps all facts from the
  backend owning each certname, chosen by the freshness strategy (per-certname
  report_timestamp map from /nodes, cached for freshness_ttl) or a static
  prefer-backend fallback.
- Fans out to both backends concurrently, serves the survivor if one fails, and
  returns 502 only when both fail; passes records through as raw JSON so unknown
  fields survive.
- Transparently proxies any other /pdb/query/v4/* path to the configurable
  primary, and exposes /healthz with per-backend reachability (200 ok /
  200 degraded / 503 down).
- Adds table-driven tests (go test -race, no network) covering merge logic,
  handler behaviour with httptest backends, query passthrough, one/both backend
  down, and config precedence/validation.
- Wires pdbmux into the build/release: Makefile BINARIES, scripts/build-rpm.sh,
  nfpm packaging (binary + completions + a systemd unit), and the release
  pipeline's cross-platform build + Gitea asset list.
- Documents pdbmux (what/why/endpoints/merge-semantics/config/deployment) in a
  new README.md and updates AGENTS.md.
unkinben added 1 commit 2026-07-24 23:09:49 +10:00
pdbmux: ship as k8s container, drop per-VM systemd/RPM delivery
ci/woodpecker/pr/build Pipeline was successful
Details
ci/woodpecker/pr/test Pipeline was successful
Details
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
17ded87439 
The estate direction is all-in-kubernetes, so pdbmux (a long-running daemon)
should run as an in-cluster service rather than an RPM-installed systemd unit
on each VM. The RPM is for workstation/VM CLI tools only; a daemon does not
belong there.

- Remove packaging/pdbmux.service and drop pdbmux (binary, systemd unit,
  completions) from the RPM/nfpm spec and build-rpm.sh.
- Keep pdbmux in the Makefile build and the test suite.
- Add Dockerfile.pdbmux building a static CGO_ENABLED=0 binary on distroless
  (mirrors encapi's image style).
- Add .woodpecker/docker.yaml to build+push git.unkin.net/unkin/pdbmux:<tag>
  on v* tags via the docker-buildx plugin (droneci/DRONECI_PASSWORD creds,
  same as encapi), with k8s resources set.
- Update README/AGENTS.md: deployment is k8s, config via PDBMUX_* env.
unkinben commented 2026-07-24 23:10:20 +10:00
Author
Owner
Copy Link
Copy Source

Rework: pdbmux is now a k8s container, not a per-VM RPM/systemd service

Per the all-in-kubernetes estate direction, pdbmux (a long-running daemon) now
ships as an in-cluster container image instead of an RPM-installed systemd unit
on each VM. The RPM remains the workstation/VM CLI package only.

Removed

  • packaging/pdbmux.service (systemd unit, DynamicUser) deleted.
  • pdbmux dropped from the RPM: nfpm spec no longer installs /usr/bin/pdbmux,
    the systemd unit, or pdbmux completions; build-rpm.sh no longer builds/lists
    it. Verified rpm -qlp shows only node-lookup/pburl/pblastreport (+ their
    completions).

Added

  • Dockerfile.pdbmux — static CGO_ENABLED=0 binary on
    gcr.io/distroless/static-debian12:nonroot, EXPOSE 8080,
    ENTRYPOINT ["pdbmux","serve"] (mirrors encapi's image style). Builds clean.
  • .woodpecker/docker.yaml — on v* tags, woodpeckerci/plugin-docker-buildx
    builds and pushes git.unkin.net/unkin/pdbmux:<tag> + :latest using the
    droneci / DRONECI_PASSWORD credentials (same mechanism/secret as encapi).
    k8s resources + serviceAccountName set.

Kept

  • pdbmux still built by make build and covered by go test -race ./....

Docs

  • README/AGENTS.md updated: deployment is k8s; configuration via PDBMUX_* env
    (no config file in-cluster); image + argocd-apps deployment described.

Verification

  • go build ./... OK, go test -race ./... OK, go vet ./cmd/pdbmux/... OK.
  • make build builds all four binaries; build-rpm.sh produces a clean RPM
    without any pdbmux/systemd files.
  • docker build -f Dockerfile.pdbmux succeeds; entrypoint is pdbmux serve.

The k8s Deployment/Service/Gateway manifests are in a companion argocd-apps PR
(namespace pdbmux, exposed at pdbmux.k8s.syd1.au.unkin.net). That PR is gated
on this one merging and a v* tag being cut so the image exists.

### Rework: pdbmux is now a k8s container, not a per-VM RPM/systemd service Per the all-in-kubernetes estate direction, pdbmux (a long-running daemon) now ships as an in-cluster container image instead of an RPM-installed systemd unit on each VM. The RPM remains the workstation/VM CLI package only. **Removed** - `packaging/pdbmux.service` (systemd unit, DynamicUser) deleted. - pdbmux dropped from the RPM: nfpm spec no longer installs `/usr/bin/pdbmux`, the systemd unit, or pdbmux completions; `build-rpm.sh` no longer builds/lists it. Verified `rpm -qlp` shows only node-lookup/pburl/pblastreport (+ their completions). **Added** - `Dockerfile.pdbmux` — static `CGO_ENABLED=0` binary on `gcr.io/distroless/static-debian12:nonroot`, `EXPOSE 8080`, `ENTRYPOINT ["pdbmux","serve"]` (mirrors encapi's image style). Builds clean. - `.woodpecker/docker.yaml` — on `v*` tags, `woodpeckerci/plugin-docker-buildx` builds and pushes `git.unkin.net/unkin/pdbmux:<tag>` + `:latest` using the `droneci` / `DRONECI_PASSWORD` credentials (same mechanism/secret as encapi). k8s resources + serviceAccountName set. **Kept** - pdbmux still built by `make build` and covered by `go test -race ./...`. **Docs** - README/AGENTS.md updated: deployment is k8s; configuration via `PDBMUX_*` env (no config file in-cluster); image + argocd-apps deployment described. **Verification** - `go build ./...` OK, `go test -race ./...` OK, `go vet ./cmd/pdbmux/...` OK. - `make build` builds all four binaries; `build-rpm.sh` produces a clean RPM without any pdbmux/systemd files. - `docker build -f Dockerfile.pdbmux` succeeds; entrypoint is `pdbmux serve`. The k8s Deployment/Service/Gateway manifests are in a companion `argocd-apps` PR (namespace `pdbmux`, exposed at `pdbmux.k8s.syd1.au.unkin.net`). That PR is gated on this one merging and a `v*` tag being cut so the image exists.
unkinben commented 2026-07-24 23:26:16 +10:00
Author
Owner
Copy Link
Copy Source

pdbmux has moved to its own repository: https://git.unkin.net/unkin/pdbmux (initial content PR: unkin/pdbmux#1). Closing this PR — the code is preserved there. node-lookup stays a CLI/RPM-only module.

pdbmux has moved to its own repository: https://git.unkin.net/unkin/pdbmux (initial content PR: https://git.unkin.net/unkin/pdbmux/pulls/1). Closing this PR — the code is preserved there. node-lookup stays a CLI/RPM-only module.
unkinben closed this pull request 2026-07-24 23:26:16 +10:00
All checks were successful
ci/woodpecker/pr/build Pipeline was successful
Required
Details
ci/woodpecker/pr/test Pipeline was successful
Required
Details
ci/woodpecker/pr/pre-commit Pipeline was successful
Required
Details

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
benvin (BenVincent) unkin-agent (Unkin Agent) unkinben (Ben Vincent)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: unkin/node-lookup#17
Delete Branch "benvin/pdbmux"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?