puppet-catalog-tools/AGENTS.md

# AGENTS.md

## Project Overview

Go CLI tooling for the Puppet estate, shipped as one RPM
(`puppet-catalog-tools`). Every tool is a separate `main` package under `cmd/`;
shared code lives in `internal/`.

- **`catalog-diff`** — compiles a host's catalog twice (base environment and
  the environment built from the branch under test) over the puppetserver API
  and diffs the result.
- **`catalog-impact`** — maps a control repo change onto the hosts it can
  affect, by reading hieradata paths as hierarchy positions and tracing
  manifest classes backwards to role classes.

## Structure

```
cmd/catalog-diff/       # the CLI (flags, exit codes, orchestration)
cmd/catalog-impact/     # the CLI (flags, output modes)
internal/catalog/       # catalog types, normalisation, diff engine, renderers,
                        # branch.go (BranchToEnvironment)
internal/puppetserver/  # mTLS client for POST /puppet/v4/catalog + /status/v1/simple
internal/impact/        # path -> selector rules, repo index, reverse lookup,
                        # estate fact table, renderers
internal/puppetdb/      # unauthenticated fact client for PuppetDB
internal/config/        # XDG config file + env var loading
packaging/nfpm.yaml     # RPM spec (envsubst-templated)
scripts/build-rpm.sh    # completions + nfpm packaging
.woodpecker/            # CI: build, test, pre-commit (PR) + release (tag)
```

Adding a binary: create `cmd/<name>/`. The Makefile, `scripts/build-rpm.sh`,
`packaging/nfpm.yaml` and the release pipeline all take the shipped list from
`cmd/`, so nothing else needs editing.

## Transport

- Catalogs: `POST /puppet/v4/catalog` with `persistence` off,
  `options.prefer_requested_environment: true`, facts and synthesised trusted
  facts in the body. Auth is client mTLS against the estate Puppet CA; the
  certname must be allowed in the puppetserver `auth.conf`.
- Facts: `GET /pdb/query/v4/facts` with the AST query
  `["=","certname","<certname>"]`. No auth, no client cert. The response is a
  flat array of `{certname, environment, name, value}` records; a record for
  another node is never folded into the result.
- Estate-wide facts: the same endpoint and record shape, with an `extract`
  query selecting the wanted fact names (`puppetdb.Client.FactsByName`). The
  estate is ~150 nodes, so `catalog-impact` pulls the whole table once and
  filters in Go rather than querying per host.
- The two services have different auth models and stay separate clients.

## Gotchas

- The compiler wraps its response as `{"catalog": {...}}`. `decodeCatalog`
  unwraps it and still accepts a bare catalog; decoding straight into
  `catalog.Catalog` silently yields an empty one.
- Only `type`, `title`, `tags` and `exported` are present on every resource.
  `parameters` is absent on class/stage/anchor resources and `file`/`line` on
  resources with no manifest location, so `Resource.Line` is a pointer and
  `Resource.Params()` maps absent onto empty.
- An ENC-supplied environment outranks the requested one. Every compile checks
  the returned catalog's `environment` and hard-fails on a mismatch — never
  relax that check. A returned environment that is empty is a decode failure,
  not a mismatch, and is reported as such.
- The compiler returns HTTP 500 for manifest errors too, so retryability is
  decided from the error text: a failed PuppetDB query (`server_urls`,
  `/pdb/query/`) is transient, a manifest error never is.
- Branch names are converted the way g10k does it (`\W` -> `_`, case
  preserved). The rule lives in one function, `catalog.BranchToEnvironment`.
- Exit codes are a contract: 0 same, 1 compile failed, 2 differences, 3
  transport/auth/mismatch/usage. `--testonly` never returns 2.
- Numeric flags are validated up front (`validate`). A negative `--retry` used
  to skip the compile loop and leave a nil error inside `exitError`; that type
  now prints safely whatever it holds.
- The two compiles share a cancellable context: the first failure abandons its
  sibling. An error that is only that cancellation is not the cause and is not
  what gets reported.
- Normalisation carries a pile of false-positive fixes (classes and exported
  resources skipped, `before`/`require` dropped, `notify`/`subscribe` sorted,
  File titles keyed by `path`, datatype-only changes suppressed). Removing one
  floods the output with noise.

## catalog-impact gotchas

- The reverse lookup must search hieradata as well as manifests. `profiles::base`
  declares `($hiera_include - $hiera_exclude).include`, and some modules — the
  `jellyfin` one, for instance — are applied *only* by a `hiera_include` list in
  a role yaml. A manifest-only search reports them as orphans.
- `hieradata/roles/<t1>/<t2>.yaml` is a **prefix** match on the role tiers, not
  an exact one. `site/roles/manifests/.../<t3>.pp` is exact.
- The bias is to over-include: an unmapped path widens to the whole estate and
  says so on stderr. Never turn a widening into a silent empty result.
- The same applies in the parser. A class reference it cannot reduce to literal
  names becomes an `UnresolvedRef` against its file, and every class that
  reference could have named is treated as reaching the classes the file
  declares. Never drop a reference the parser failed on.
- The repo index comes from the `--repo` checkout, not from `--branch`. CI runs
  on the branch checkout, so the two agree there.

## Testing

```bash
make test     # go test -v -race ./...
```

Everything is stubbed with `httptest` (a TLS server for puppetserver, a plain
one for PuppetDB) — no live estate access is required. Renderer output is
covered by golden files under `internal/catalog/testdata`; regenerate with
`go test ./internal/catalog -update`.

`internal/puppetserver/testdata/catalog_response.json` is a synthetic
OpenVoxServer 8.8 response, kept for its shape: the `catalog` envelope, a
string `version`, a null `code_id`, and resources that omit `parameters`,
`file` and `line`, plus the exported and `sensitive_parameters` cases. This
repo is public — hostnames, class names and paths in fixtures stay fake.