# AGENTS.md ## Project Overview Go CLI tooling for the Puppet estate, shipped as one RPM (`puppet-catalog-tools`). Every tool is a separate `main` package under `cmd/`; shared code lives in `internal/`. - **`catalog-diff`** — compiles a host's catalog twice (base environment and the environment built from the branch under test) over the puppetserver API and diffs the result. - **`catalog-impact`** — maps a control repo change onto the hosts it can affect, by reading hieradata paths as hierarchy positions and tracing manifest classes backwards to role classes. ## Structure ``` cmd/catalog-diff/ # the CLI (flags, exit codes, orchestration) cmd/catalog-impact/ # the CLI (flags, output modes) internal/catalog/ # catalog types, normalisation, diff engine, renderers, # branch.go (BranchToEnvironment) internal/puppetserver/ # mTLS client for POST /puppet/v4/catalog + /status/v1/simple internal/impact/ # path -> selector rules, repo index, reverse lookup, # estate fact table, renderers internal/puppetdb/ # unauthenticated fact client for PuppetDB internal/config/ # XDG config file + env var loading packaging/nfpm.yaml # RPM spec (envsubst-templated) scripts/build-rpm.sh # completions + nfpm packaging .woodpecker/ # CI: build, test, pre-commit (PR) + release (tag) ``` Adding a binary: create `cmd//`. The Makefile, `scripts/build-rpm.sh`, `packaging/nfpm.yaml` and the release pipeline all take the shipped list from `cmd/`, so nothing else needs editing. ## Transport - Catalogs: `POST /puppet/v4/catalog` with `persistence` off, `options.prefer_requested_environment: true`, facts and synthesised trusted facts in the body. Auth is client mTLS against the estate Puppet CA; the certname must be allowed in the puppetserver `auth.conf`. - Facts: `GET /pdb/query/v4/facts` with the AST query `["=","certname",""]`. No auth, no client cert. The response is a flat array of `{certname, environment, name, value}` records; a record for another node is never folded into the result. - Estate-wide facts: the same endpoint and record shape, with an `extract` query selecting the wanted fact names (`puppetdb.Client.FactsByName`). The estate is ~150 nodes, so `catalog-impact` pulls the whole table once and filters in Go rather than querying per host. - The two services have different auth models and stay separate clients. ## Gotchas - The compiler wraps its response as `{"catalog": {...}}`. `decodeCatalog` unwraps it and still accepts a bare catalog; decoding straight into `catalog.Catalog` silently yields an empty one. - Only `type`, `title`, `tags` and `exported` are present on every resource. `parameters` is absent on class/stage/anchor resources and `file`/`line` on resources with no manifest location, so `Resource.Line` is a pointer and `Resource.Params()` maps absent onto empty. - An ENC-supplied environment outranks the requested one. Every compile checks the returned catalog's `environment` and hard-fails on a mismatch — never relax that check. A returned environment that is empty is a decode failure, not a mismatch, and is reported as such. - The compiler returns HTTP 500 for manifest errors too, so retryability is decided from the error text: a failed PuppetDB query (`server_urls`, `/pdb/query/`) is transient, a manifest error never is. - Branch names are converted the way g10k does it (`\W` -> `_`, case preserved). The rule lives in one function, `catalog.BranchToEnvironment`. - Exit codes are a contract: 0 same, 1 compile failed, 2 differences, 3 transport/auth/mismatch/usage. `--testonly` never returns 2. - Numeric flags are validated up front (`validate`). A negative `--retry` used to skip the compile loop and leave a nil error inside `exitError`; that type now prints safely whatever it holds. - The two compiles share a cancellable context: the first failure abandons its sibling. An error that is only that cancellation is not the cause and is not what gets reported. - Normalisation carries a pile of false-positive fixes (classes and exported resources skipped, `before`/`require` dropped, `notify`/`subscribe` sorted, File titles keyed by `path`, datatype-only changes suppressed). Removing one floods the output with noise. ## catalog-impact gotchas - The reverse lookup must search hieradata as well as manifests. `profiles::base` declares `($hiera_include - $hiera_exclude).include`, and some modules — the `jellyfin` one, for instance — are applied *only* by a `hiera_include` list in a role yaml. A manifest-only search reports them as orphans. - `hieradata/roles//.yaml` is a **prefix** match on the role tiers, not an exact one. `site/roles/manifests/.../.pp` is exact. - The bias is to over-include: an unmapped path widens to the whole estate and says so on stderr. Never turn a widening into a silent empty result. - The same applies in the parser. A class reference it cannot reduce to literal names becomes an `UnresolvedRef` against its file, and every class that reference could have named is treated as reaching the classes the file declares. Never drop a reference the parser failed on. - The repo index comes from the `--repo` checkout, not from `--branch`. CI runs on the branch checkout, so the two agree there. ## Testing ```bash make test # go test -v -race ./... ``` Everything is stubbed with `httptest` (a TLS server for puppetserver, a plain one for PuppetDB) — no live estate access is required. Renderer output is covered by golden files under `internal/catalog/testdata`; regenerate with `go test ./internal/catalog -update`. `internal/puppetserver/testdata/catalog_response.json` is a synthetic OpenVoxServer 8.8 response, kept for its shape: the `catalog` envelope, a string `version`, a null `code_id`, and resources that omit `parameters`, `file` and `line`, plus the exported and `sensitive_parameters` cases. This repo is public — hostnames, class names and paths in fixtures stay fake.