puppet-catalog-tools/AGENTS.md

AGENTS.md

Project Overview

Go CLI tooling for the Puppet estate, shipped as one RPM (puppet-catalog-tools). Every tool is a separate main package under cmd/; shared code lives in internal/.

• catalog-diff — compiles a host's catalog twice (base environment and the environment built from the branch under test) over the puppetserver API and diffs the result.
• catalog-impact — maps a control repo change onto the hosts it can affect, by reading hieradata paths as hierarchy positions and tracing manifest classes backwards to role classes.

Structure

cmd/catalog-diff/       # the CLI (flags, exit codes, orchestration)
cmd/catalog-impact/     # the CLI (flags, output modes)
internal/catalog/       # catalog types, normalisation, diff engine, renderers,
                        # branch.go (BranchToEnvironment)
internal/puppetserver/  # mTLS client for POST /puppet/v4/catalog + /status/v1/simple
internal/impact/        # path -> selector rules, repo index, reverse lookup,
                        # estate fact table, renderers
internal/puppetdb/      # unauthenticated fact client for PuppetDB
internal/config/        # XDG config file + env var loading
packaging/nfpm.yaml     # RPM spec (envsubst-templated)
scripts/build-rpm.sh    # completions + nfpm packaging
.woodpecker/            # CI: build, test, pre-commit (PR) + release (tag)

Adding a binary: create cmd/<name>/. The Makefile, scripts/build-rpm.sh, packaging/nfpm.yaml and the release pipeline all take the shipped list from cmd/, so nothing else needs editing.

Transport

• Catalogs: POST /puppet/v4/catalog with persistence off, options.prefer_requested_environment: true, facts and synthesised trusted facts in the body. Auth is client mTLS against the estate Puppet CA; the certname must be allowed in the puppetserver auth.conf.
• Facts: GET /pdb/query/v4/facts with the AST query ["=","certname","<certname>"]. No auth, no client cert. The response is a flat array of {certname, environment, name, value} records; a record for another node is never folded into the result.
• Estate-wide facts: the same endpoint and record shape, with an extract query selecting the wanted fact names (puppetdb.Client.FactsByName). The estate is ~150 nodes, so catalog-impact pulls the whole table once and filters in Go rather than querying per host.
• The two services have different auth models and stay separate clients.

Gotchas

• The compiler wraps its response as {"catalog": {...}}. decodeCatalog unwraps it and still accepts a bare catalog; decoding straight into catalog.Catalog silently yields an empty one.
• Only type, title, tags and exported are present on every resource. parameters is absent on class/stage/anchor resources and file/line on resources with no manifest location, so Resource.Line is a pointer and Resource.Params() maps absent onto empty.
• An ENC-supplied environment outranks the requested one. Every compile checks the returned catalog's environment and hard-fails on a mismatch — never relax that check. A returned environment that is empty is a decode failure, not a mismatch, and is reported as such.
• The compiler returns HTTP 500 for manifest errors too, so retryability is decided from the error text: a failed PuppetDB query (server_urls, /pdb/query/) is transient, a manifest error never is.
• Branch names are converted the way g10k does it (\W -> _, case preserved). The rule lives in one function, catalog.BranchToEnvironment.
• Exit codes are a contract: 0 same, 1 compile failed, 2 differences, 3 transport/auth/mismatch/usage. --testonly never returns 2.
• Numeric flags are validated up front (validate). A negative --retry used to skip the compile loop and leave a nil error inside exitError; that type now prints safely whatever it holds.
• The two compiles share a cancellable context: the first failure abandons its sibling. An error that is only that cancellation is not the cause and is not what gets reported.
• Normalisation carries a pile of false-positive fixes (classes and exported resources skipped, before/require dropped, notify/subscribe sorted, File titles keyed by path, datatype-only changes suppressed). Removing one floods the output with noise.

catalog-impact gotchas

• The reverse lookup must search hieradata as well as manifests. profiles::base declares ($hiera_include - $hiera_exclude).include, and some modules — the jellyfin one, for instance — are applied only by a hiera_include list in a role yaml. A manifest-only search reports them as orphans.
• hieradata/roles/<t1>/<t2>.yaml is a prefix match on the role tiers, not an exact one. site/roles/manifests/.../<t3>.pp is exact.
• The bias is to over-include: an unmapped path widens to the whole estate and says so on stderr. Never turn a widening into a silent empty result.
• The same applies in the parser. A class reference it cannot reduce to literal names becomes an UnresolvedRef against its file, and every class that reference could have named is treated as reaching the classes the file declares. Never drop a reference the parser failed on.
• The repo index comes from the --repo checkout, not from --branch. CI runs on the branch checkout, so the two agree there.

Testing

make test     # go test -v -race ./...

Everything is stubbed with httptest (a TLS server for puppetserver, a plain one for PuppetDB) — no live estate access is required. Renderer output is covered by golden files under internal/catalog/testdata; regenerate with go test ./internal/catalog -update.

internal/puppetserver/testdata/catalog_response.json is a synthetic OpenVoxServer 8.8 response, kept for its shape: the catalog envelope, a string version, a null code_id, and resources that omit parameters, file and line, plus the exported and sensitive_parameters cases. This repo is public — hostnames, class names and paths in fixtures stay fake.