unkin-agent
807d3df71c
Short -n node names (no dot) now silently returned nothing because the PuppetDB certname filter needs a FQDN. Auto-qualify a dotless name to <name>.<domain> (domain defaults to main.unkin.net) before the query. - Add qualifyNode() pure helper: dotless names get .<domain> appended; names already containing a dot (any domain) are left unchanged; a single trailing dot is stripped; empty input is preserved. - Apply it to the -n value and to stdin-sourced node names in run(). - Make the domain configurable via config key domain, NODE_LOOKUP_DOMAIN env var, and --domain flag (default main.unkin.net). - Surface domain in config show / config init and document in AGENTS.md. - Add table-driven qualifyNode tests and a domain env-override test.
183 lines
9.4 KiB
Markdown
183 lines
9.4 KiB
Markdown
# AGENTS.md
|
|
|
|
## Project Overview
|
|
|
|
This repo ships three related Puppet CLIs in one RPM:
|
|
|
|
- **`node-lookup`** — queries the PuppetDB API to retrieve and filter node facts.
|
|
- **`pburl`** — prints the Puppetboard node-page URL for each host (reads hosts
|
|
from args or piped `node-lookup` output). Output: `<host> <url>`.
|
|
- **`pblastreport`** — prints each host's last Puppet report time and its
|
|
Puppetboard URL. Output: `<host>\t<time>\t<url>`. Supports `--relative`/`-r`
|
|
(relative age) and `--timezone`/`-z <IANA>` (default: local timezone).
|
|
|
|
`node-lookup` is the module root; `pburl` and `pblastreport` live under `cmd/`
|
|
and share the `internal/puppet` package (config, PuppetDB `nodes` queries,
|
|
Puppetboard URL construction, stdin host reading).
|
|
|
|
## Structure
|
|
|
|
```
|
|
main.go # node-lookup CLI source (module root, package main)
|
|
main_test.go # node-lookup unit tests (mock PuppetDB via httptest)
|
|
cmd/pburl/main.go # pburl CLI
|
|
cmd/pblastreport/main.go # pblastreport CLI (report.go: report-time formatting)
|
|
internal/puppet/ # shared: config, puppetdb nodes query, board URLs, stdin
|
|
go.mod # Go module (module name: node-lookup)
|
|
go.sum # dependency checksums
|
|
Makefile # build / test / lint / completions / rpm / version-bump targets
|
|
packaging/nfpm.yaml # nfpm spec (envsubst-templated) for the RPM (all 3 binaries)
|
|
scripts/build-rpm.sh # generates completions + packages the RPM with nfpm
|
|
.woodpecker/ # CI: build, test, pre-commit (PR) + release (tag)
|
|
dist/ # build output: binaries, completions, RPM (not committed)
|
|
```
|
|
|
|
Every binary is a separate `main` package, so `make build` builds each with its
|
|
own `-o` (a single `go build ./...` can't emit multiple mains to one file).
|
|
|
|
## Build
|
|
|
|
```bash
|
|
make build # -> dist/node-lookup (CGO disabled, static)
|
|
# or directly:
|
|
go build -o node-lookup ./...
|
|
```
|
|
|
|
Requires Go 1.21+. Dependencies: `github.com/spf13/cobra` (CLI), `gopkg.in/yaml.v3` (Ansible output).
|
|
|
|
## Packaging (RPM)
|
|
|
|
```bash
|
|
make rpm # build the binary + package it into dist/*.rpm via nfpm
|
|
```
|
|
|
|
`scripts/build-rpm.sh` generates bash/zsh/fish completions from the built binary
|
|
and bundles them alongside `/usr/bin/node-lookup`. On a `v*` tag the release
|
|
pipeline builds the RPM and `PUT`s it to the artifactapi `rpm-internal` repo.
|
|
|
|
## Shell completions
|
|
|
|
Cobra provides a `completion` subcommand:
|
|
|
|
```bash
|
|
node-lookup completion bash # or zsh / fish / powershell
|
|
```
|
|
|
|
The RPM installs completions to the standard system paths
|
|
(`/usr/share/bash-completion/completions/`, `/usr/share/zsh/site-functions/`,
|
|
`/usr/share/fish/vendor_completions.d/`), so they work automatically once
|
|
installed. To load ad-hoc in the current shell, e.g. zsh:
|
|
`source <(node-lookup completion zsh)`.
|
|
|
|
## Running the Tool
|
|
|
|
```bash
|
|
./node-lookup --help
|
|
./node-lookup -R # show all nodes with role fact
|
|
./node-lookup -n <hostname> # lookup a specific node
|
|
./node-lookup -F <fact_name> # filter by fact name
|
|
./node-lookup -jF ipaddress,enc_role # several facts at once (comma-separated)
|
|
./node-lookup -R -m <value> # exact value match (-m)
|
|
./node-lookup -R -pm <value> # partial/regex match (-p -m combined)
|
|
./node-lookup -R -im <value> # inverse exact match (-i -m combined)
|
|
./node-lookup -R -ipm <value> # inverse partial match (-i -p -m combined)
|
|
./node-lookup -R -p <value> # value may also be given positionally
|
|
./node-lookup -R -1 # node names only
|
|
./node-lookup -R -2 # values only
|
|
./node-lookup -R -C # count occurrences
|
|
./node-lookup -R -A # output as Ansible YAML inventory (queried facts become host vars)
|
|
./node-lookup -j # output as JSON { host → { fact → value } }
|
|
./node-lookup --url http://host:8080/... # override PuppetDB URL for this invocation
|
|
echo -e "node1\nnode2" | ./node-lookup -R # pipe node names via stdin
|
|
```
|
|
|
|
### Companion tools
|
|
|
|
```bash
|
|
node-lookup -R | pburl # <host> <puppetboard-url> per line
|
|
pburl host1 host2 # hosts as args instead of stdin
|
|
|
|
node-lookup -R | pblastreport # <host> <last-report-time> <url>
|
|
pblastreport -r host1 # relative age (e.g. "3h ago")
|
|
pblastreport -z Asia/Singapore host1 # render the time in a specific IANA tz
|
|
```
|
|
|
|
Both read hostnames from arguments or the first field of each piped line (so
|
|
any `node-lookup` output mode works), de-duplicate, and share `node-lookup`'s
|
|
config file / env vars. `pblastreport` reads `report_timestamp` from the
|
|
PuppetDB v4 `nodes` endpoint (derived from the configured facts URL).
|
|
|
|
## Configuration
|
|
|
|
Precedence (lowest → highest): **defaults < config file < env vars < `--url` flag**
|
|
|
|
### Config file
|
|
|
|
XDG location: `$XDG_CONFIG_HOME/node-lookup/config.yaml` (default: `~/.config/node-lookup/config.yaml`)
|
|
|
|
```yaml
|
|
puppetdb_url: http://puppetdbapi.service.consul:8080/pdb/query/v4/facts
|
|
role_fact: enc_role
|
|
puppetboard_url: https://puppetboard.k8s.syd1.au.unkin.net # used by pburl / pblastreport
|
|
```
|
|
|
|
Generate the default config file:
|
|
```bash
|
|
./node-lookup config init
|
|
```
|
|
|
|
Show the active configuration (after all overrides applied):
|
|
```bash
|
|
./node-lookup config show
|
|
```
|
|
|
|
### Environment variables
|
|
|
|
| Variable | Config key | Description |
|
|
|---|---|---|
|
|
| `NODE_LOOKUP_URL` | `puppetdb_url` | PuppetDB facts endpoint |
|
|
| `NODE_LOOKUP_ROLE_FACT` | `role_fact` | Fact name used by `-R` flag |
|
|
| `NODE_LOOKUP_PUPPETBOARD_URL` | `puppetboard_url` | Puppetboard base URL (pburl / pblastreport) |
|
|
| `NODE_LOOKUP_DOMAIN` | `domain` | Domain appended to short (dotless) `-n` node names (default `main.unkin.net`) |
|
|
|
|
### CLI flags
|
|
|
|
`--url <url>` overrides the PuppetDB URL for a single invocation (highest precedence).
|
|
`--domain <domain>` overrides the auto-qualify domain for a single invocation.
|
|
|
|
## Code Patterns
|
|
|
|
- **`loadConfig()`**: reads config file → applies env vars → returns `config` struct. Called once at startup in `main()`.
|
|
- **`buildQuery()`**: returns a PuppetDB PQL-compatible JSON array string. Uses `roleFact` from config (not hardcoded). Match modifiers: `-p` (partial/regex, uses `~` op), `-i` (inverse, wraps with `not`), composable.
|
|
- **Multiple facts**: `-F` accepts a comma-separated list (`ipaddress,enc_role`). `splitFactNames()`/`nameFilter()` turn several names into an `or` over `["=","name",<n>]` clauses; JSON output keys each value by the fact's real name so all requested facts appear per host.
|
|
- **Match value / `matchValue()`**: the value to match comes from `-m/--match` or, if that is empty, an optional positional argument. The positional fallback exists because pflag does not attach a space-separated value to a string flag grouped with a bool flag, so in `-pm k8s` the `k8s` arrives as a positional. `-m` still wins when both are given.
|
|
- **`queryPuppetDB(url, query)`**: takes the URL as a parameter — never reads globals.
|
|
- **`processResults()`**: iterates facts, returns sorted `"certname value"` strings. JSON string values are unquoted; other JSON types rendered as compact JSON.
|
|
- **Output modes**: JSON (`-j`), count (`-C`), Ansible YAML (`-A`), node-only (`-1`), value-only (`-2`), default (node + value). `-j` and `-A` share `factsByHost()`, so both attach the queried fact(s) per host — as an object under the host (`-j`) or as inventory host vars (`-A`).
|
|
- **Short node names / `qualifyNode()`**: a `-n` value (and stdin-sourced node names) with no dot is auto-qualified to `<name>.<domain>` (domain defaults to `main.unkin.net`, overridable via `--domain`/`NODE_LOOKUP_DOMAIN`), so `-n ausyd1nxvm2120` resolves the same as its FQDN. A name that already contains a dot (any domain) is left unchanged; a single trailing dot is stripped; empty input is preserved.
|
|
- **Stdin support**: `stdinReader()` reads node names from stdin only when it is a real pipe/redirect carrying data (and no `-n` given). Terminals, `/dev/null`, and empty/closed pipes fall through to a normal query — so running without a TTY (e.g. invoked by an agent or CI) behaves like an interactive run instead of consuming empty input.
|
|
- **SIGPIPE handling**: `signal.Ignore(syscall.SIGPIPE)` so pipes to `head` etc. work cleanly.
|
|
|
|
## CLI Framework
|
|
|
|
Uses [Cobra](https://github.com/spf13/cobra). Root command is the query command. `config` is a subcommand with `init` and `show` sub-subcommands.
|
|
|
|
## Testing
|
|
|
|
```bash
|
|
make test # go test -v -race ./...
|
|
```
|
|
|
|
`main_test.go` covers query construction (all `-m`/`-p`/`-i` combinations), value
|
|
rendering, result processing/counting, config precedence (defaults < file < env),
|
|
`writeDefaultConfig`, the `stdinReader` no-TTY behavior, and every `run()` output
|
|
mode (default, `-1`, `-2`, `-C`, `-j`, `-A`, `-a`). PuppetDB is stubbed with
|
|
`httptest` — no live Consul/PuppetDB access is required.
|
|
|
|
## Gotchas
|
|
|
|
- `-1`, `-2`, `-C`, and `-A` all require `-R` or `-F`; the tool exits with an error otherwise.
|
|
- `-C` (count) with stdin reads all lines as pre-fetched `"node value"` output for counting — it does **not** query PuppetDB per line.
|
|
- JSON output (`-j`) builds `{ hostname: { factname: value } }` keyed by each result's actual fact name (so `-F ipaddress,enc_role` yields both per host); it falls back to the `-F` value, the `role_fact` config value (if `-R`), or `"value"` only when a result carries no name.
|
|
- `config init` fails if the config file already exists (will not overwrite).
|