docs: strip over-commenting from README and source
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful

This commit is contained in:
2026-09-05 11:38:24 +10:00
parent e2e9004784
commit 31ad4ae457
7 changed files with 35 additions and 169 deletions
+1 -4
View File
@@ -1,7 +1,4 @@
# Build and push the pdbmux container image on a v* tag. pdbmux is a k8s-only
# daemon (deployed via argocd-apps), so it ships as an image. Mirrors the estate
# convention: the CA-baked plugin-docker-buildx image pushes to the
# artifactapi local docker registry (unauthenticated in-cluster push).
# plugin-docker-buildx is the CA-baked variant; artifactapi's cert is not in the default trust store.
when:
- event: tag
ref: refs/tags/v*
-3
View File
@@ -1,6 +1,3 @@
# Container image for pdbmux, the merging PuppetDB proxy daemon. pdbmux is a
# k8s-only service (deployed via argocd-apps), so it ships as a distroless
# static image rather than an RPM.
FROM golang:1.25-alpine AS builder
RUN apk add --no-cache git
-3
View File
@@ -9,7 +9,6 @@ ARCH ?= $(shell go env GOARCH)
all: build
# Build the single static binary into dist/.
build:
CGO_ENABLED=0 GOOS=$(OS) GOARCH=$(ARCH) go build $(GOFLAGS) -o $(DIST)/$(BINARY) .
@@ -28,8 +27,6 @@ clean:
install:
go install $(GOFLAGS) .
# Bump helpers — read the latest semver tag and create the next one.
# If no tag exists yet, start from v0.0.0.
_LATEST := $(shell git tag --sort=-v:refname | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$$' | head -1)
_BASE := $(if $(_LATEST),$(_LATEST),v0.0.0)
_MAJ := $(shell echo $(_BASE) | sed 's/^v//' | cut -d. -f1)
+14 -58
View File
@@ -55,12 +55,10 @@ unknown fields survive untouched.
Precedence (lowest → highest): **defaults < config file < env vars (`PDBMUX_*`) < flags**.
Config file: `$XDG_CONFIG_HOME/pdbmux/config.yaml`. In Kubernetes, configuration
is supplied entirely via `PDBMUX_*` env vars (no config file), which is the
supported deployment path — see [Deployment](#deployment).
Config file: `$XDG_CONFIG_HOME/pdbmux/config.yaml`. In Kubernetes there is no
config file — everything comes from `PDBMUX_*` env vars.
```yaml
# ~/.config/pdbmux/config.yaml (local dev; in k8s use PDBMUX_* env instead)
listen: ":8080"
backends:
- name: old
@@ -74,8 +72,8 @@ timeout: 10s # per-upstream request timeout
freshness_ttl: 30s # freshness-map cache TTL (freshness merge only)
```
`backends[*].url` is a **base** URL (`scheme://host[:port]`), without the
`/pdb/query/v4/...` path — `pdbmux` appends the path per request.
`backends[*].url` is a **base** URL (`scheme://host[:port]`); `pdbmux` appends
the `/pdb/query/v4/...` path per request.
| Env var | Overrides |
|---|---|
@@ -91,66 +89,24 @@ Flags: `--listen`, `--primary`, `--merge`.
## Running
```bash
pdbmux # start the proxy (serve is the default action)
pdbmux serve # explicit
pdbmux config init # write a default config file
pdbmux config show # print active config after all overrides
pdbmux version
```
Point a consumer at it:
Subcommands: `serve` (default), `config init`, `config show`, `version`. Run
`pdbmux --help` for details.
```bash
PDBMUX_BACKENDS='old=http://puppetdbapi.service.consul:8080,new=https://puppetdb.k8s.syd1.au.unkin.net' pdbmux
node-lookup --url http://localhost:8080/pdb/query/v4/facts -R
NODE_LOOKUP_URL=http://localhost:8080/pdb/query/v4/facts pblastreport somehost
```
## Build
```bash
make build # -> dist/pdbmux (CGO disabled, static)
make test # go test -race ./...
make lint # golangci-lint
```
Requires Go 1.25+. Dependencies: `github.com/spf13/cobra` (CLI),
`gopkg.in/yaml.v3` (config file).
`make build` (static binary into `dist/`), `make test`, `make lint`. Requires Go 1.25+.
## Deployment
`pdbmux` runs **in Kubernetes** as a container, in line with the all-in-k8s
estate direction — it is not shipped as a per-VM RPM/systemd service. The image
is built and pushed on every `v*` tag (`.woodpecker/docker.yaml`) to:
Kubernetes only — no RPM. Every `v*` tag builds and pushes
`artifactapi.k8s.syd1.au.unkin.net/docker-internal/pdbmux:<tag>`
(`.woodpecker/docker.yaml`); tag with `make patch` / `minor` / `major`.
```
artifactapi.k8s.syd1.au.unkin.net/docker-internal/pdbmux:<tag>
```
It is a minimal static (`CGO_ENABLED=0`) binary on a distroless base
(`Dockerfile`), configured entirely via `PDBMUX_*` env vars, with a single HTTP
listener and `/healthz` for liveness/readiness probes.
The Deployment/Service/Gateway manifests live in the estate's `argocd-apps` repo
under `apps/base/pdbmux/` (namespace `pdbmux`, 2 replicas), and it is exposed to
VM/workstation `node-lookup` consumers over HTTPS at:
```
https://pdbmux.k8s.syd1.au.unkin.net
```
Locally you can still run the binary directly for development:
```bash
PDBMUX_BACKENDS='old=http://puppetdbapi.service.consul:8080,new=http://puppetdb.puppet.svc.cluster.local:8080' \
pdbmux serve
curl -s localhost:8080/healthz
```
## Version bumps
```bash
make patch # tag vX.Y.(Z+1) and push (triggers the docker release)
make minor # tag vX.(Y+1).0
make major # tag v(X+1).0.0
```
Manifests live in `argocd-apps` under `apps/base/pdbmux/` (namespace `pdbmux`,
2 replicas). Use `/healthz` for liveness/readiness probes. Reachable from VMs and
workstations at `https://pdbmux.k8s.syd1.au.unkin.net`.
+12 -49
View File
@@ -16,49 +16,27 @@ const (
configFileName = "config.yaml"
envPrefix = "PDBMUX_"
// defaultListen is the default HTTP listen address.
defaultListen = ":8080"
// defaultOldURL / defaultNewURL are the two PuppetDBs merged during the
// VM -> k8s migration. old = legacy Consul-registered puppetdbapi; new =
// the k8s PuppetDB behind the gateway (TLS terminated there).
defaultOldURL = "http://puppetdbapi.service.consul:8080"
defaultNewURL = "https://puppetdb.k8s.syd1.au.unkin.net"
// defaultPrimary is the backend name used for pass-through (non-merged)
// /pdb/query/v4/* paths and as static precedence for merge fallback.
defaultListen = ":8080"
defaultOldURL = "http://puppetdbapi.service.consul:8080"
defaultNewURL = "https://puppetdb.k8s.syd1.au.unkin.net"
defaultPrimary = "new"
defaultTimeout = 10 * time.Second
defaultFreshnessTTL = 30 * time.Second
)
// Backend is one upstream PuppetDB. URL is the base URL (scheme://host[:port]),
// without the /pdb/query/v4/... path — that is appended per request.
type Backend struct {
Name string `yaml:"name"`
URL string `yaml:"url"`
URL string `yaml:"url"` // base URL only; the query path is appended per request
}
// Config holds every configurable value. Fields map 1:1 to config-file keys and
// env vars (PDBMUX_*). See Load for precedence.
type Config struct {
// Listen is the HTTP listen address (host:port).
Listen string `yaml:"listen"`
// Backends is the ordered list of upstream PuppetDBs to fan out to.
Backends []Backend `yaml:"backends"`
// Primary is the backend Name used for transparent pass-through of
// non-merged /pdb/query/v4/* paths.
Primary string `yaml:"primary"`
// Merge selects how /facts records are attributed to a backend when a
// certname appears in both: "freshness" (query /nodes report_timestamp,
// newer wins) or "static" (always prefer the Prefer backend).
Merge string `yaml:"merge"`
// Prefer names the backend that wins under static merge and as the
// tie-breaker/fallback under freshness merge.
Prefer string `yaml:"prefer"`
// Timeout bounds each upstream request.
Timeout time.Duration `yaml:"timeout"`
// FreshnessTTL is how long a per-certname freshness map (from /nodes) is
// cached under the "freshness" merge strategy.
Listen string `yaml:"listen"`
Backends []Backend `yaml:"backends"`
Primary string `yaml:"primary"`
Merge string `yaml:"merge"`
Prefer string `yaml:"prefer"` // wins under static merge, and breaks ties under freshness merge
Timeout time.Duration `yaml:"timeout"`
FreshnessTTL time.Duration `yaml:"freshness_ttl"`
}
@@ -67,8 +45,6 @@ const (
mergeStatic = "static"
)
// DefaultConfig returns the built-in defaults: both migration PuppetDBs,
// freshness merge, "new" primary/preferred.
func DefaultConfig() Config {
return Config{
Listen: defaultListen,
@@ -84,7 +60,6 @@ func DefaultConfig() Config {
}
}
// ConfigDir returns the XDG_CONFIG_HOME/pdbmux directory.
func ConfigDir() string {
base := os.Getenv("XDG_CONFIG_HOME")
if base == "" {
@@ -94,15 +69,11 @@ func ConfigDir() string {
return filepath.Join(base, appName)
}
// ConfigPath returns the full path to the config file.
func ConfigPath() string {
return filepath.Join(ConfigDir(), configFileName)
}
// Load reads the config file (if present), then applies env var overrides.
// Precedence (lowest -> highest): defaults < config file < env vars < flags
// (flags are applied by the caller). Backends can be overridden wholesale via
// PDBMUX_BACKENDS ("name=url,name=url").
// Precedence: defaults < config file < env vars < flags, and flags are applied by the caller.
func Load() (Config, error) {
cfg := DefaultConfig()
@@ -125,7 +96,6 @@ func Load() (Config, error) {
return cfg, nil
}
// applyEnv overlays PDBMUX_* env vars onto cfg. getenv is injected for testing.
func applyEnv(cfg *Config, getenv func(string) string) {
if v := getenv(envPrefix + "LISTEN"); v != "" {
cfg.Listen = v
@@ -156,8 +126,7 @@ func applyEnv(cfg *Config, getenv func(string) string) {
}
}
// parseBackends parses "name=url,name=url" into Backends. Entries without an
// "=" are skipped. Used for the PDBMUX_BACKENDS env override.
// Parses the PDBMUX_BACKENDS form "name=url,name=url"; entries without an "=" are skipped.
func parseBackends(s string) []Backend {
var out []Backend
for _, part := range strings.Split(s, ",") {
@@ -175,7 +144,6 @@ func parseBackends(s string) []Backend {
return out
}
// Validate checks the config is internally consistent and usable.
func (c Config) Validate() error {
if len(c.Backends) == 0 {
return fmt.Errorf("no backends configured")
@@ -207,8 +175,6 @@ func (c Config) Validate() error {
return nil
}
// PrimaryBackend returns the backend named by Primary (guaranteed present after
// Validate).
func (c Config) PrimaryBackend() Backend {
for _, b := range c.Backends {
if b.Name == c.Primary {
@@ -218,7 +184,6 @@ func (c Config) PrimaryBackend() Backend {
return c.Backends[0]
}
// writeDefaultConfig creates the config dir and writes a default config file.
func writeDefaultConfig() error {
dir := ConfigDir()
if err := os.MkdirAll(dir, 0o755); err != nil {
@@ -240,8 +205,6 @@ func writeDefaultConfig() error {
return nil
}
// durationString renders a duration for `config show` (falls back to a plain
// seconds count for zero to avoid "0s" ambiguity in logs).
func durationString(d time.Duration) string {
if d == 0 {
return "0"
+1 -21
View File
@@ -1,21 +1,4 @@
// Command pdbmux is a small merging HTTP proxy over two PuppetDB backends.
//
// During the VM -> k8s Puppet migration there are two PuppetDBs — the legacy
// Consul-registered one and the new k8s one — and nodes move between them as
// they migrate. pdbmux presents a single merged PuppetDB v4 query surface so
// node-lookup and pblastreport (and anything else) see one consistent view:
//
// - GET /pdb/query/v4/nodes — fan out to both backends, dedupe by certname,
// keep the record with the newer report_timestamp.
// - GET /pdb/query/v4/facts — fan out to both, and for a certname present in
// both keep ALL facts from the backend holding that node's newer report
// (freshness merge) or a static preferred backend (static merge).
// - any other GET /pdb/query/v4/* — transparently proxied to the primary.
// - GET /healthz — per-backend reachability.
//
// The query param is forwarded verbatim (PuppetDB AST JSON). If one backend
// errors/times out, the other's results are served; only if both fail does a
// merged endpoint return 502.
// Command pdbmux serves one merged PuppetDB v4 query surface over two PuppetDBs.
package main
import (
@@ -119,8 +102,6 @@ func main() {
}
}
// runServer starts the HTTP server and blocks until SIGINT/SIGTERM, then
// gracefully shuts down.
func runServer(cfg Config) error {
logger := log.New(os.Stderr, "pdbmux: ", log.LstdFlags)
srv := NewServer(cfg, logger)
@@ -155,7 +136,6 @@ func runServer(cfg Config) error {
}
}
// printConfig renders the active config for `config show`.
func printConfig(cfg Config) {
fmt.Printf("config file : %s\n", ConfigPath())
fmt.Printf("listen : %s\n", cfg.Listen)
+7 -31
View File
@@ -5,24 +5,18 @@ import (
"time"
)
// record is a single PuppetDB result element kept as raw JSON so unknown fields
// survive the merge untouched. certname/report_timestamp are decoded only for
// merge decisions.
// Raw is kept verbatim so unknown PuppetDB fields survive the merge.
type record struct {
Raw json.RawMessage
Certname string
ReportTimestamp string // only populated for /nodes records
}
// recordMeta is the subset we decode from any /nodes or /facts element to drive
// merge decisions.
type recordMeta struct {
Certname string `json:"certname"`
ReportTimestamp string `json:"report_timestamp"`
}
// decodeRecords turns a raw PuppetDB JSON array into records, preserving each
// element verbatim in Raw. A body that is not a JSON array yields (nil, err).
func decodeRecords(body []byte) ([]record, error) {
var raws []json.RawMessage
if err := json.Unmarshal(body, &raws); err != nil {
@@ -41,8 +35,7 @@ func decodeRecords(body []byte) ([]record, error) {
return out, nil
}
// parseTimestamp parses a PuppetDB RFC3339(nano) timestamp. Zero time on
// failure sorts oldest, so a backend with a well-formed newer timestamp wins.
// An unparseable timestamp yields the zero time, which sorts oldest.
func parseTimestamp(s string) time.Time {
if s == "" {
return time.Time{}
@@ -53,10 +46,7 @@ func parseTimestamp(s string) time.Time {
return time.Time{}
}
// mergeNodes dedupes /nodes records by certname, keeping the one with the newer
// report_timestamp. backends is the ordered list of (name, records) results;
// when timestamps tie (or both are zero), the earlier backend in the slice
// wins, so callers should order by precedence.
// results must be ordered by precedence: ties keep the earlier backend's record.
func mergeNodes(results []backendResult) []json.RawMessage {
type pick struct {
raw json.RawMessage
@@ -73,7 +63,6 @@ func mergeNodes(results []backendResult) []json.RawMessage {
order = append(order, rec.Certname)
continue
}
// Strictly-newer wins; ties keep the existing (earlier-backend) pick.
if ts.After(cur.ts) {
best[rec.Certname] = pick{raw: rec.Raw, ts: ts}
}
@@ -86,12 +75,10 @@ func mergeNodes(results []backendResult) []json.RawMessage {
return out
}
// freshness maps certname -> backend name that holds that node's newest report.
// certname -> name of the backend holding that node's newest report.
type freshness map[string]string
// buildFreshness computes, per certname, which backend has the newer
// report_timestamp. results must be ordered by precedence; on a tie the
// earlier backend wins.
// results must be ordered by precedence: ties keep the earlier backend.
func buildFreshness(results []backendResult) freshness {
type pick struct {
backend string
@@ -114,19 +101,9 @@ func buildFreshness(results []backendResult) freshness {
return f
}
// mergeFacts merges /facts records at node granularity: for each certname, all
// facts from the winning backend are kept and the other backend's facts for
// that certname are dropped.
//
// The winner is chosen per certname by `owner(certname)`. Callers supply owner
// from either a freshness map (freshness merge) or a constant preferred backend
// (static merge). When owner returns a backend that has no facts for a certname
// (or a name not in results), records fall back to precedence order so a node
// present in only one backend still appears.
// owner names the winning backend per certname; when it holds no facts for that certname, precedence order wins.
func mergeFacts(results []backendResult, owner func(certname string) string) []json.RawMessage {
// Which backends actually returned facts for each certname, in precedence
// order, so we can fall back if the chosen owner has none.
present := map[string][]string{} // certname -> ordered backend names
present := map[string][]string{} // certname -> backend names, in precedence order
byKey := map[string][]json.RawMessage{}
for _, res := range results {
for _, rec := range res.records {
@@ -154,7 +131,6 @@ func mergeFacts(results []backendResult, owner func(certname string) string) []j
for _, cn := range order {
backends := present[cn]
chosen := owner(cn)
// Fall back to precedence order if the chosen backend has no facts here.
if !contains(backends, chosen) {
chosen = backends[0]
}