Every backend gets the same query, so one they all refuse is the client's
mistake; flattening it into "all backends failed" threw openvoxdb's own
explanation away and logged a typo as an outage.
- Carry status, content type and body on a typed upstreamError
- Replay the status and explanation when every backend refuses alike
- Redact backend addresses from replayed bodies
- Keep a refused query out of the partial counters and the cache
distinct_resources sends /events to openvoxdb's legacy compiler, which has no
function or group_by, so every backend failed and the client saw a generic 502
instead of the reason.
- Refuse an aggregate carrying a truthy distinct_resources with 400, before any fan-out
- Read the param the way openvoxdb does, so any capitalisation of "true" counts
- Leave non-aggregate distinct_resources queries and every other route alone
- Document the refusal
An extract carrying a ["function", ...] column returns counts, not events, so
the union's verbatim-record key folded two backends' identical rows into one
number.
- Route /events through the central aggregate guard with its own fan-out path
- Drop its unsummed opt-out so the route-table property tests cover it
- Document the combined path on /events
A repeated extract function names one response column twice, which openvoxdb
aliases as <name>_2: unknown to the merge spec, it froze at the first backend's
value. A limit pushed upstream truncated each backend's groups before the
cross-backend fold, so a group could be partly counted or missed.
- Refuses any extract projecting one response column twice, naming the clash
- Fetches every group and applies limit/offset after the fold
- Documents the float64 avg divergence from Postgres numeric
sumRows folded every numeric column by addition, which is only correct
for count and sum, so min/max returned a sum, avg an average of
averages, and a to_string extract collapsed into one empty-key row.
- Combine count and sum by adding, min and max by the extreme, on text
columns as well as numeric ones
- Rewrite an avg extract into an upstream sum and count and divide the
totals, answering under the avg key the client asked for
- Refuse an aggregate pdbmux cannot merge with 400 naming the clash
- Treat to_string and jsonb_typeof as row functions that group rather
than fold, and key groups on every non-aggregate projected column
- Give the e2e fixture per-node resource line numbers and titles whose
extremes differ per backend
A `/facts` aggregate row carries no certname, so the per-certname fact
merge collapsed every backend's rows into one bucket and served a single
backend's numbers.
- Route a `/facts` query whose `extract` carries a `function` column to
serveSummed, as /nodes, /resources and /reports already do
- Document which extract functions combine correctly across backends
A down backend costs a full timeout stall on every request, since fan-out
has no way to know before it asks, and the client is never told the answer
came from fewer backends than are configured.
- poll each backend's status endpoint in the background, one goroutine per
backend, with failure/success thresholds so a blip cannot flap it
- skip backends the prober has down, and fall open to querying all of them
when none is left healthy
- treat a not-yet-probed backend as healthy so a restart drops no traffic
- log only up/down transitions
- stamp merged responses with X-Backends: <contributed>/<configured>
- report per-backend probe state and the last round's partiality on /healthz
- add health_probe_enabled, health_probe_path, health_probe_interval,
health_probe_timeout, health_probe_failures and health_probe_successes,
with matching PDBMUX_* env vars and a --health-probe flag
## Why
With several PuppetDBs behind one endpoint, consumers cannot tell which backend a node's data came from.
## How
- Add a synthetic `pdbmux_source` fact per certname on `/facts`, valued with the backend that won the facts merge, and stamp the same key on merged `/nodes` records.
- Emit all four fact keys including `environment`, which clients index directly.
- Skip injection for top-level `extract` queries, so `count()` and other aggregates keep the backends' own numbers, and for `/facts` queries constraining `name`; a `name` filter inside an `in` subquery still injects.
- Replace, never duplicate, an upstream fact of the configured name.
- Configure via `source_fact` / `source_fact_enabled` (`PDBMUX_SOURCE_FACT`, `PDBMUX_SOURCE_FACT_ENABLED`), defaulting to `pdbmux_source` enabled.
## Why
Puppetboard 7.0.1 cannot run against pdbmux: it exits at import when
/pdb/meta/v1/version 404s, and its landing page, metrics and radiator views
404 on the Jolokia surface.
## How
- Serve /pdb/meta/v1/version, reporting the lowest version any backend runs,
and /pdb/meta/v1/server-time from the first reachable backend.
- Merge the Jolokia surface (/metrics/v2/read, /metrics/v2/list,
/metrics/v1/mbeans): objects union, numeric attributes sum by default, and
Min/Max/Uptime/StartTime plus the distribution stats take a bound or a mean.
- Route /nodes extract-count queries to the summing path ahead of the certname
merge, and give /resources aggregates the same path.
- Document the endpoints and merge semantics in the README.
- Cover version disagreement, metric rules, escaped MBean names, count summing
and the non-aggregate /nodes merge with httptest backends.
Reports are immutable history, so a node that migrated has reports in the
old PuppetDB and the new one; serve the union rather than picking a single
owning backend as /facts does.
Re-apply order_by/limit/offset over the merged set and sum X-Records, since
each backend only orders and pages its own slice.
Split out from node-lookup PR #17 into its own repo. pdbmux presents a
single merged PuppetDB v4 query surface over the old (Consul) and new
(k8s) PuppetDBs during the VM to k8s migration, and is deployed in-cluster
via argocd-apps as a container image.
unkin-agent
benvin