Inject a pdbmux_source provenance fact #13

Merged

benvin merged 4 commits from benvin/source-fact into main 2026-09-05 22:49:33 +10:00

Conversation 0 Commits 4 Files Changed 10

+1021 -38

unkin-agent commented 2026-09-05 21:00:25 +10:00

Member

Copy Link

Copy Source

Why

With several PuppetDBs behind one endpoint, consumers cannot tell which backend a node's data came from.

How

• Add a synthetic pdbmux_source fact per certname on /facts, valued with the backend that won the facts merge, and stamp the same key on merged /nodes records.
• Emit all four fact keys including environment, which clients index directly.
• Skip injection for top-level extract queries, so count() and other aggregates keep the backends' own numbers, and for /facts queries constraining name; a name filter inside an in subquery still injects.
• Replace, never duplicate, an upstream fact of the configured name.
• Configure via source_fact / source_fact_enabled (PDBMUX_SOURCE_FACT, PDBMUX_SOURCE_FACT_ENABLED), defaulting to pdbmux_source enabled.

Server-side filtering on the fact is a documented v1 limitation; /factsets and /inventory are untouched because pdbmux does not merge them.

## Why With several PuppetDBs behind one endpoint, consumers cannot tell which backend a node's data came from. ## How - Add a synthetic `pdbmux_source` fact per certname on `/facts`, valued with the backend that won the facts merge, and stamp the same key on merged `/nodes` records. - Emit all four fact keys including `environment`, which clients index directly. - Skip injection for top-level `extract` queries, so `count()` and other aggregates keep the backends' own numbers, and for `/facts` queries constraining `name`; a `name` filter inside an `in` subquery still injects. - Replace, never duplicate, an upstream fact of the configured name. - Configure via `source_fact` / `source_fact_enabled` (`PDBMUX_SOURCE_FACT`, `PDBMUX_SOURCE_FACT_ENABLED`), defaulting to `pdbmux_source` enabled. Server-side filtering on the fact is a documented v1 limitation; `/factsets` and `/inventory` are untouched because pdbmux does not merge them.

unkin-agent added 1 commit 2026-09-05 21:00:25 +10:00

Inject a pdbmux_source provenance fact ... c935b20a54

## Why
With several PuppetDBs behind one endpoint, consumers cannot tell which backend a node's data came from.

## How
- Add a synthetic `pdbmux_source` fact per certname on `/facts`, valued with the backend that won the facts merge, and stamp the same key on merged `/nodes` records.
- Emit all four fact keys including `environment`, which clients index directly.
- Skip injection for top-level `extract` queries, so `count()` and other aggregates keep the backends' own numbers, and for `/facts` queries constraining `name`; a `name` filter inside an `in` subquery still injects.
- Replace, never duplicate, an upstream fact of the configured name.
- Configure via `source_fact` / `source_fact_enabled` (`PDBMUX_SOURCE_FACT`, `PDBMUX_SOURCE_FACT_ENABLED`), defaulting to `pdbmux_source` enabled.

unkin-agent added 1 commit 2026-09-05 21:05:27 +10:00

Merge main into benvin/source-fact ... 6cfded36fe

Route /nodes through serveNodes so aggregate queries still sum, and pass the
per-request source injector into the merged path it keeps.

unkin-agent added 1 commit 2026-09-05 21:31:55 +10:00

Override the source fact on every query shape ... a4a29866e1

- drop upstream facts of the configured name whenever the feature is enabled,
  independent of the per-query injection gate, and log the drop once per request
- walk the whole AST for a nested extract and skip injection when one is found
  outside an in subquery
- skip the environment scan on /facts when nothing is injected
- document the override rule and that PQL-syntax queries never get the fact

unkin-agent added 1 commit 2026-09-05 21:49:19 +10:00

Scope the extract gate to subqueries and correct the suppression doc ... c228597fb9

Suppression matches a record's own `name` field, so a projection that
filters on `name` without returning it carries an upstream value through.
The README claimed the record was always dropped on every query shape.
State the rule the code implements and pin the shape with a test.

`hasExtract` exempted only `in`. openvoxdb's `valid-operator?`
(src/puppetlabs/puppetdb/query_eng/engine.clj:2779-2784) lists `subquery`
separately, and the AST-rewrite stage (:2111-2123) expands
["subquery" entity expr] into ["in" cols ["extract" cols ["select_x" expr]]]
before any plan node is built, so its operand is projected into a subquery
exactly like `in`'s (:2705-2712). Exempt `subquery` and the explicit
`select_<entity>` forms (:1889-1911).

Signed-off-by: unkin-agent <unkin-agent@unkin.net>

benvin merged commit 083fb6ba53 into main 2026-09-05 22:49:33 +10:00

benvin deleted branch benvin/source-fact 2026-09-05 22:49:34 +10:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

bug

Something is not working

duplicate

This issue or pull request already exists

enhancement

New feature

help wanted

Need some help

invalid

Something is wrong

question

More information is needed

wontfix

This won't be fixed

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

benvin (BenVincent) unkin-agent (Unkin Agent) unkinben (Ben Vincent)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unkin/pdbmux#13