6 Commits

Author SHA1 Message Date
unkin-agent 35edc9c547 node-lookup: auto-qualify short node names to .main.unkin.net (#19)
ci/woodpecker/tag/release Pipeline was successful
Short `-n` node names silently returned nothing: `node-lookup -R -n ausyd1nxvm2120` found nothing while `-n ausyd1nxvm2120.main.unkin.net` worked, because the PuppetDB `certname` filter needs a FQDN. This auto-qualifies a dotless name before the lookup.

## Changes
- Add `qualifyNode()` pure helper: a dotless name gets `.<domain>` appended; a name already containing a dot (any domain, incl. `*.k8s.syd1.au.unkin.net`) is left unchanged; a single trailing dot is stripped first; empty input is preserved (same "no node" behavior as today).
- Apply normalization to the `-n` value and to stdin-sourced node names in `run()`, so both entry points behave consistently.
- Make the domain configurable: config key `domain`, `NODE_LOOKUP_DOMAIN` env var, and `--domain` flag, all defaulting to `main.unkin.net`.
- Surface `domain` in `config show` / `config init` output and document the new env var/flag/behavior in AGENTS.md.
- Add table-driven `qualifyNode` tests (short name appended, FQDN unchanged, multi-label other-domain FQDN unchanged, trailing-dot handling, empty input, custom domain) and a `NODE_LOOKUP_DOMAIN` env-override test.

Companion tools `pburl`/`pblastreport` take already-qualified hostnames (typically piped from `node-lookup`) via `puppet.ReadHosts` and do not share the `-n` code path, so they are intentionally left out to keep this PR atomic.

## Validation
- `gofmt -l .` clean, `go vet ./...` clean
- `go test -race ./...` pass
- `make build` builds all three binaries

Reviewed-on: #19
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-08-15 14:50:35 +10:00
unkinben 182bd326b8 Fix release changelog range and attach RPM + checksums (#18)
ci/woodpecker/tag/release Pipeline was successful
## Why

The `v0.5.4` release step failed with `Error: open node-lookup-linux-amd64: no such file or directory`, so the Gitea releases page carries **none** of the binaries. Root cause: at v0.5.4 the build loop's shell variables (`${name}`, `${pkg}`, `${osarch%/*}`) were unescaped, so Woodpecker substituted them to empty at YAML-parse time and no cross-compiled binaries were produced. The same blanking left the release `--note` empty (`git log "..v0.5.4"`).

The build-loop escaping was already fixed in #16. This PR fixes the two remaining release-step defects and enriches the release assets.

## Changes

- Replace the `git describe --tags --abbrev=0 HEAD^` changelog anchor with a previous-tag scan that skips tags on the current commit and picks the newest semver **ancestor** tag. Several tags point at the same commit (v0.5.3 and v0.5.4 both on `f296056`), so `describe HEAD^` jumps the range back to v0.5.1; the scan correctly selects v0.5.2.
- Attach the packaged RPM (from `dist/`) and a generated `sha256sums.txt` alongside the 12 cross-compiled binaries.
- Build the asset list once and checksum exactly what is uploaded.

Keeps `serviceAccountName: default` and the k8s resource requests/limits on the release step unchanged.

## Validation

- `check-yaml` / `trailing-whitespace` pre-commit hooks pass.
- Local dry-run of the exact release-step shell logic (with `$$`→`$`) from the worktree: `PREV_TAG=v0.5.2`, non-empty notes, and all 13 assets (12 binaries + RPM) plus `sha256sums.txt` resolve on disk.

---------

Co-authored-by: Ben Vincent <neotheo@gmail.com>
Reviewed-on: #18
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-25 14:58:24 +10:00
unkinben 2aaffd7e31 Fix release pipeline: escape shell vars so cross-compiled assets build (#16)
## Why

The **v0.5.3** release pipeline failed at the asset-upload step with `open node-lookup-linux-amd64: no such file or directory`, and release notes came out empty. Woodpecker substitutes `${...}` expressions in `commands` **at parse time**, treating them as pipeline variables. The build loop's shell parameter expansions (`${name}`, `${osarch%/*}`, `${pkg}`) and the release step's `${PREV_TAG}`/`${NOTES}` were blanked before the shell ran, so the per-os/arch binaries were written to garbled names and never matched the upload list.

The RPM itself was unaffected (it's built via `make build`, internal to the Makefile) and `node-lookup-0.5.3-1.x86_64.rpm` published to `rpm-internal` correctly — only the Gitea release binary assets and notes were broken.

## Changes

- Escape all shell variables and command substitutions in the build loop and the release-notes block as `$$`, matching the existing `upload-rpm` step's convention. Genuine Woodpecker vars (`${CI_COMMIT_TAG}`, `${CI_REPO}`) stay single-`$`.

## Follow-up

The v0.5.3 Gitea release assets are being backfilled manually; this fix ensures v0.5.4+ produce them automatically.

Reviewed-on: #16
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-16 23:08:09 +10:00
unkinben f296056360 Add pburl and pblastreport companion tools to the RPM (#15)
ci/woodpecker/tag/release Pipeline failed
## Why

`node-lookup` output is handy for pivoting to Puppetboard, but there was no quick way to turn a list of hosts into Puppetboard node-page URLs, or to see when each host last ran Puppet. These two small tools close that gap and ship in the **same RPM** so they're available wherever `node-lookup` is.

## Changes

- Add **`pburl`**: reads hostnames from args or piped `node-lookup` output (first field of each line, de-duped) and prints `<host> <puppetboard-node-page-url>`.
- Add **`pblastreport`**: prints `<host>\t<last-report-time>\t<url>` using `report_timestamp` from the PuppetDB v4 `nodes` endpoint. Supports `--relative`/`-r` (relative age) and `--timezone`/`-z <IANA>` (default: local timezone).
- Add **`internal/puppet`** package shared by both tools: config load, PuppetDB `nodes` query, Puppetboard URL construction (`<base>/node/<certname>`), and no-TTY-safe stdin host reading.
- Add **`puppetboard_url`** config key (env `NODE_LOOKUP_PUPPETBOARD_URL`, default `https://puppetboard.k8s.syd1.au.unkin.net`) to the shared config so `config init`/`config show` scaffold it for the whole tool family. `node-lookup`'s own query behaviour is unchanged.
- Build all three binaries individually (each is its own `main` package — a single `go build ./...` can't emit multiple mains) and generate per-binary bash/zsh/fish completions in the Makefile, `build-rpm.sh`, and nfpm spec.
- Cross-compile and attach all three tools per os/arch in the release pipeline; extend `.gitignore`; `go mod tidy` promotes cobra/yaml to direct deps.
- Document the tools, config key, and env var in `AGENTS.md`.

## Testing

- `go test -race ./...` passes (new tests cover config precedence, `nodes` endpoint derivation, host-page URLs, `LookupNode`, stdin host parsing, and the report-time formatting incl. timezone/relative/edge cases).
- Built the RPM locally and confirmed it installs all 3 binaries + 9 completion files.
- Smoke-tested both tools end-to-end against a mock PuppetDB (timezone conversion, relative time, and error handling all correct).

No cross-repo changes needed: the release reuses the existing `default` ServiceAccount and the artifactapi `rpm-internal` upload.

Reviewed-on: #15
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-16 22:37:26 +10:00
unkinben 5982d257d5 Fix -pm <value> parsing and add comma-separated multi-fact -F (#14)
Two related query-ergonomics fixes surfaced while using the tool interactively.

## 1. `-pm <value>` failed with `unknown command "k8s"`

pflag does not attach a space-separated value to a string flag (`-m`) grouped with a bool flag (`-p`), so `k8s` was left as a stray positional. Only `-pm=k8s` or the un-grouped `-p -m k8s` worked.

- Allow one positional argument (`cobra.MaximumNArgs(1)`) and fall back to it for the match value when `-m` is empty (`matchValue()`). `-pm/-im/-ipm <value>` and a bare `-p <value>` now all work; `-m` still wins when both are given.

## 2. `-F a,b` (multiple facts) returned `{}`

`-jF ipaddress,enc_role` queried a single fact literally named `ipaddress,enc_role`.

- Split `-F` on commas (`splitFactNames`) and match any of them via an `or` over `["=","name",<n>]` clauses (`nameFilter`); a single name keeps the plain `=` form.
- Key JSON output by each result's real fact name so all requested facts appear under the host.

Both paths have unit tests; verified live: `node-lookup -R -pm externaldns | node-lookup -jF ipaddress,enc_role` returns both facts per host. AGENTS.md updated.

Reviewed-on: #14
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-05 20:05:45 +10:00
unkinben e070357d3f Relocate packaging: RPM, shell completions, no-TTY fix, repaired tests (#13)
ci/woodpecker/tag/release Pipeline was successful
## Why

The unit tests stopped compiling after the `--pm` → `-p`/`-i` match-modifier refactor was left uncommitted, there was no RPM/completions distribution story, and invoking the tool without a TTY against an empty pipe silently returned nothing. This makes the project releasable and safe to run from agents/CI.

## Changes

- Make stdin handling robust: replace the fragile `!isTerminal` check with `stdinReader()`, which only reads node names when stdin is a real pipe/redirect carrying data. Terminals, `/dev/null`, and empty/closed pipes now fall through to a normal query, so running without a TTY behaves like an interactive run.
- Repair and expand `main_test.go` to match the current `buildQuery`/`run` signatures; add coverage for the match modifiers, all output modes, config precedence, and the new `stdinReader` logic. `httptest` stubs PuppetDB (no live deps).
- Add nfpm packaging (`packaging/nfpm.yaml`, `scripts/build-rpm.sh`): installs the binary to `/usr/bin/node-lookup` and bundles generated bash/zsh/fish completions under the standard system paths.
- Rework the Makefile to build into `dist/` and add `completions`/`rpm` targets.
- Split PR CI into `build`, `test`, and `pre-commit` workflows and extend `release` to build the RPM and `PUT` it to the artifactapi `rpm-internal` repo. Every step sets a `serviceAccount` and k8s resources.

The project directory has also been relocated under `prodenv`.

Reviewed-on: #13
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-05 00:02:34 +10:00
24 changed files with 1889 additions and 213 deletions
+8 -1
View File
@@ -1,2 +1,9 @@
node-lookup # built binaries (repo root only — not the cmd/ source dirs)
/node-lookup
/pburl
/pblastreport
# cross-compiled release artifacts (e.g. node-lookup-linux-amd64)
/node-lookup-*
/pburl-*
/pblastreport-*
dist/ dist/
+18
View File
@@ -0,0 +1,18 @@
when:
- event: pull_request
steps:
- name: build
image: golang:1.25
commands:
- make build
backend_options:
kubernetes:
serviceAccountName: default
resources:
requests:
memory: 512Mi
cpu: 1
limits:
memory: 2Gi
cpu: 2
-8
View File
@@ -1,8 +0,0 @@
when:
- event: pull_request
steps:
- name: lint
image: golangci/golangci-lint:latest
commands:
- golangci-lint run ./...
+11 -1
View File
@@ -3,6 +3,16 @@ when:
steps: steps:
- name: pre-commit - name: pre-commit
image: git.unkin.net/unkin/almalinux9-gobuilder:20260325 image: git.unkin.net/unkin/almalinux9-gobuilder:20260606
commands: commands:
- uvx pre-commit run --all-files - uvx pre-commit run --all-files
backend_options:
kubernetes:
serviceAccountName: default
resources:
requests:
memory: 512Mi
cpu: 1
limits:
memory: 2Gi
cpu: 2
+126 -31
View File
@@ -3,36 +3,107 @@ when:
steps: steps:
- name: test - name: test
image: golang:latest image: golang:1.25
commands: commands:
- go test ./... - go test -race ./...
backend_options:
kubernetes:
serviceAccountName: default
resources:
requests:
memory: 512Mi
cpu: 1
limits:
memory: 2Gi
cpu: 2
- name: build-linux-amd64 # Build all binaries into dist/ (consumed by the RPM step) plus the
image: golang:latest # cross-platform binaries attached to the Gitea release. Each tool is a
# separate main package, so they are built individually per os/arch.
- name: build
image: git.unkin.net/unkin/almalinux9-gobuilder:20260606
commands: commands:
- GOOS=linux GOARCH=amd64 go build -ldflags="-s -w -X main.version=${CI_COMMIT_TAG}" -o node-lookup-linux-amd64 ./... - make build VERSION=${CI_COMMIT_TAG}
# Shell variables/expansions are escaped as $$ so Woodpecker leaves them
# for the shell instead of substituting them (as pipeline vars) at parse
# time. ${CI_COMMIT_TAG} is a real Woodpecker var and stays single-$.
- |
for entry in "node-lookup:." "pburl:./cmd/pburl" "pblastreport:./cmd/pblastreport"; do
name="$${entry%%:*}"; pkg="$${entry##*:}"
for osarch in linux/amd64 linux/arm64 darwin/amd64 darwin/arm64; do
os="$${osarch%/*}"; arch="$${osarch#*/}"
GOOS="$$os" GOARCH="$$arch" \
go build -ldflags="-s -w -X main.version=${CI_COMMIT_TAG}" \
-o "$${name}-$${os}-$${arch}" "$$pkg"
done
done
depends_on: [test] depends_on: [test]
backend_options:
kubernetes:
serviceAccountName: default
resources:
requests:
memory: 512Mi
cpu: 1
limits:
memory: 2Gi
cpu: 2
- name: build-linux-arm64 # Package the built binary + generated shell completions into an RPM.
image: golang:latest - name: package
image: git.unkin.net/unkin/almalinux9-rpmbuilder:latest
commands: commands:
- GOOS=linux GOARCH=arm64 go build -ldflags="-s -w -X main.version=${CI_COMMIT_TAG}" -o node-lookup-linux-arm64 ./... - ./scripts/build-rpm.sh ${CI_COMMIT_TAG}
depends_on: [test] depends_on: [build]
backend_options:
kubernetes:
serviceAccountName: default
resources:
requests:
memory: 512Mi
cpu: 1
limits:
memory: 2Gi
cpu: 2
- name: build-darwin-amd64 # Publish the RPM to the artifactapi local rpm repo (a real yum repo;
image: golang:latest # repodata regenerates automatically).
- name: upload-rpm
image: git.unkin.net/unkin/almalinux9-base:20260606
commands: commands:
- GOOS=darwin GOARCH=amd64 go build -ldflags="-s -w -X main.version=${CI_COMMIT_TAG}" -o node-lookup-darwin-amd64 ./... - |
depends_on: [test] HOST="https://artifactapi.k8s.syd1.au.unkin.net"
REPO="rpm-internal"
- name: build-darwin-arm64 for rpm in dist/*.rpm; do
image: golang:latest FILE=$$(basename "$$rpm")
commands: # artifactapi has no HEAD route (returns 405); probe with GET against
- GOOS=darwin GOARCH=arm64 go build -ldflags="-s -w -X main.version=${CI_COMMIT_TAG}" -o node-lookup-darwin-arm64 ./... # the served path (RPMs are stored under Packages/) to avoid re-upload.
depends_on: [test] code=$$(curl -s -o /dev/null -w '%{http_code}' "$$HOST/api/v2/remotes/$$REPO/files/Packages/$$FILE" || true)
if [ "$$code" = "200" ]; then
echo "$$FILE already exists in $$REPO (HTTP $$code); skipping upload"
continue
fi
echo "Uploading $$FILE to $$REPO (existence probe returned $$code)"
curl -f -X PUT \
"$$HOST/api/v2/remotes/$$REPO/files/$$FILE" \
-H "Content-Type: application/x-rpm" \
--data-binary @"$$rpm"
done
depends_on: [package]
backend_options:
kubernetes:
serviceAccountName: default
resources:
requests:
memory: 128Mi
cpu: 100m
limits:
memory: 512Mi
cpu: 500m
# Cut a Gitea release with the cross-platform binaries attached.
- name: release - name: release
image: git.unkin.net/unkin/almalinux9-base:20260325 image: git.unkin.net/unkin/almalinux9-base:20260606
environment: environment:
RELEASER_TOKEN: RELEASER_TOKEN:
from_secret: RELEASER_TOKEN from_secret: RELEASER_TOKEN
@@ -40,20 +111,44 @@ steps:
- | - |
curl --output /usr/local/bin/tea https://artifactapi.k8s.syd1.au.unkin.net/api/v1/remote/gitea-dl/tea/0.12.0/tea-0.12.0-linux-amd64 && chmod +x /usr/local/bin/tea curl --output /usr/local/bin/tea https://artifactapi.k8s.syd1.au.unkin.net/api/v1/remote/gitea-dl/tea/0.12.0/tea-0.12.0-linux-amd64 && chmod +x /usr/local/bin/tea
tea logins add --name gitea --url https://git.unkin.net --token "$${RELEASER_TOKEN}" --no-version-check tea logins add --name gitea --url https://git.unkin.net --token "$${RELEASER_TOKEN}" --no-version-check
PREV_TAG=$(git describe --tags --abbrev=0 HEAD^ 2>/dev/null || echo "") # $$ escapes shell vars/substitutions so Woodpecker doesn't blank them
if [ -n "$PREV_TAG" ]; then # at parse time; ${CI_COMMIT_TAG}/${CI_REPO} are real Woodpecker vars.
NOTES=$(git log "${PREV_TAG}..${CI_COMMIT_TAG}" --pretty=format:"- %s") # Find the previous release tag for the changelog range. Several tags can
# point at the same commit (e.g. v0.5.3 and v0.5.4), so we skip tags on
# the current commit and pick the newest semver tag that is a real
# ancestor of this one -- describe HEAD^ would jump too far back.
CUR_SHA=$$(git rev-list -n1 "${CI_COMMIT_TAG}")
PREV_TAG=""
for t in $$(git tag --sort=-v:refname); do
[ "$$t" = "${CI_COMMIT_TAG}" ] && continue
[ "$$(git rev-list -n1 "$$t")" = "$$CUR_SHA" ] && continue
if git merge-base --is-ancestor "$$t" "${CI_COMMIT_TAG}" 2>/dev/null; then
PREV_TAG="$$t"; break
fi
done
if [ -n "$$PREV_TAG" ]; then
NOTES=$$(git log "$${PREV_TAG}..${CI_COMMIT_TAG}" --pretty=format:"- %s")
else else
NOTES=$(git log --pretty=format:"- %s") NOTES=$$(git log --pretty=format:"- %s")
fi fi
tea releases create --tag "${CI_COMMIT_TAG}" --title "${CI_COMMIT_TAG}" --note "${NOTES}" --login gitea --repo "${CI_REPO}" tea releases create --tag "${CI_COMMIT_TAG}" --title "${CI_COMMIT_TAG}" --note "$${NOTES}" --login gitea --repo "${CI_REPO}"
tea releases assets create "${CI_COMMIT_TAG}" \ # The build step writes the 12 cross-compiled binaries into the workspace
node-lookup-linux-amd64 \ # root; the package step writes the RPM to dist/. Generate a checksums
node-lookup-linux-arm64 \ # manifest over everything we attach so downloads can be verified.
node-lookup-darwin-amd64 \ RPM=$$(ls dist/*.rpm 2>/dev/null | head -1)
node-lookup-darwin-arm64 \ ASSETS="node-lookup-linux-amd64 node-lookup-linux-arm64 node-lookup-darwin-amd64 node-lookup-darwin-arm64 pburl-linux-amd64 pburl-linux-arm64 pburl-darwin-amd64 pburl-darwin-arm64 pblastreport-linux-amd64 pblastreport-linux-arm64 pblastreport-darwin-amd64 pblastreport-darwin-arm64"
[ -n "$$RPM" ] && ASSETS="$$ASSETS $$RPM"
sha256sum $$ASSETS > sha256sums.txt
tea releases assets create "${CI_COMMIT_TAG}" $$ASSETS sha256sums.txt \
--login gitea --repo "${CI_REPO}" --login gitea --repo "${CI_REPO}"
depends_on: [upload-rpm]
backend_options: backend_options:
kubernetes: kubernetes:
serviceAccountName: default serviceAccountName: default
depends_on: [build-linux-amd64, build-linux-arm64, build-darwin-amd64, build-darwin-arm64] resources:
requests:
memory: 128Mi
cpu: 100m
limits:
memory: 512Mi
cpu: 500m
+33
View File
@@ -0,0 +1,33 @@
when:
- event: pull_request
steps:
- name: lint
image: golangci/golangci-lint:latest
commands:
- golangci-lint run ./...
backend_options:
kubernetes:
serviceAccountName: default
resources:
requests:
memory: 512Mi
cpu: 1
limits:
memory: 2Gi
cpu: 2
- name: test
image: golang:1.25
commands:
- go test -v -race ./...
backend_options:
kubernetes:
serviceAccountName: default
resources:
requests:
memory: 512Mi
cpu: 1
limits:
memory: 2Gi
cpu: 2
-8
View File
@@ -1,8 +0,0 @@
when:
- event: pull_request
steps:
- name: unit-tests
image: golang:latest
commands:
- go test -v -race ./...
+97 -14
View File
@@ -2,25 +2,73 @@
## Project Overview ## Project Overview
`node-lookup` is a Go CLI tool that queries a PuppetDB API to retrieve and filter node facts. This repo ships three related Puppet CLIs in one RPM:
- **`node-lookup`** — queries the PuppetDB API to retrieve and filter node facts.
- **`pburl`** — prints the Puppetboard node-page URL for each host (reads hosts
from args or piped `node-lookup` output). Output: `<host> <url>`.
- **`pblastreport`** — prints each host's last Puppet report time and its
Puppetboard URL. Output: `<host>\t<time>\t<url>`. Supports `--relative`/`-r`
(relative age) and `--timezone`/`-z <IANA>` (default: local timezone).
`node-lookup` is the module root; `pburl` and `pblastreport` live under `cmd/`
and share the `internal/puppet` package (config, PuppetDB `nodes` queries,
Puppetboard URL construction, stdin host reading).
## Structure ## Structure
``` ```
main.go # entire application source main.go # node-lookup CLI source (module root, package main)
go.mod # Go module (module name: node-lookup) main_test.go # node-lookup unit tests (mock PuppetDB via httptest)
go.sum # dependency checksums cmd/pburl/main.go # pburl CLI
node-lookup # compiled binary (not committed) cmd/pblastreport/main.go # pblastreport CLI (report.go: report-time formatting)
internal/puppet/ # shared: config, puppetdb nodes query, board URLs, stdin
go.mod # Go module (module name: node-lookup)
go.sum # dependency checksums
Makefile # build / test / lint / completions / rpm / version-bump targets
packaging/nfpm.yaml # nfpm spec (envsubst-templated) for the RPM (all 3 binaries)
scripts/build-rpm.sh # generates completions + packages the RPM with nfpm
.woodpecker/ # CI: build, test, pre-commit (PR) + release (tag)
dist/ # build output: binaries, completions, RPM (not committed)
``` ```
Every binary is a separate `main` package, so `make build` builds each with its
own `-o` (a single `go build ./...` can't emit multiple mains to one file).
## Build ## Build
```bash ```bash
make build # -> dist/node-lookup (CGO disabled, static)
# or directly:
go build -o node-lookup ./... go build -o node-lookup ./...
``` ```
Requires Go 1.21+. Dependencies: `github.com/spf13/cobra` (CLI), `gopkg.in/yaml.v3` (Ansible output). Requires Go 1.21+. Dependencies: `github.com/spf13/cobra` (CLI), `gopkg.in/yaml.v3` (Ansible output).
## Packaging (RPM)
```bash
make rpm # build the binary + package it into dist/*.rpm via nfpm
```
`scripts/build-rpm.sh` generates bash/zsh/fish completions from the built binary
and bundles them alongside `/usr/bin/node-lookup`. On a `v*` tag the release
pipeline builds the RPM and `PUT`s it to the artifactapi `rpm-internal` repo.
## Shell completions
Cobra provides a `completion` subcommand:
```bash
node-lookup completion bash # or zsh / fish / powershell
```
The RPM installs completions to the standard system paths
(`/usr/share/bash-completion/completions/`, `/usr/share/zsh/site-functions/`,
`/usr/share/fish/vendor_completions.d/`), so they work automatically once
installed. To load ad-hoc in the current shell, e.g. zsh:
`source <(node-lookup completion zsh)`.
## Running the Tool ## Running the Tool
```bash ```bash
@@ -28,17 +76,37 @@ Requires Go 1.21+. Dependencies: `github.com/spf13/cobra` (CLI), `gopkg.in/yaml.
./node-lookup -R # show all nodes with role fact ./node-lookup -R # show all nodes with role fact
./node-lookup -n <hostname> # lookup a specific node ./node-lookup -n <hostname> # lookup a specific node
./node-lookup -F <fact_name> # filter by fact name ./node-lookup -F <fact_name> # filter by fact name
./node-lookup -m <value> # exact value match ./node-lookup -jF ipaddress,enc_role # several facts at once (comma-separated)
./node-lookup --pm <pattern> # partial/regex match on value ./node-lookup -R -m <value> # exact value match (-m)
./node-lookup -R -pm <value> # partial/regex match (-p -m combined)
./node-lookup -R -im <value> # inverse exact match (-i -m combined)
./node-lookup -R -ipm <value> # inverse partial match (-i -p -m combined)
./node-lookup -R -p <value> # value may also be given positionally
./node-lookup -R -1 # node names only ./node-lookup -R -1 # node names only
./node-lookup -R -2 # values only ./node-lookup -R -2 # values only
./node-lookup -R -C # count occurrences ./node-lookup -R -C # count occurrences
./node-lookup -R -A # output as Ansible YAML inventory ./node-lookup -R -A # output as Ansible YAML inventory (queried facts become host vars)
./node-lookup -j # output as JSON { host → { fact → value } } ./node-lookup -j # output as JSON { host → { fact → value } }
./node-lookup --url http://host:8080/... # override PuppetDB URL for this invocation ./node-lookup --url http://host:8080/... # override PuppetDB URL for this invocation
echo -e "node1\nnode2" | ./node-lookup -R # pipe node names via stdin echo -e "node1\nnode2" | ./node-lookup -R # pipe node names via stdin
``` ```
### Companion tools
```bash
node-lookup -R | pburl # <host> <puppetboard-url> per line
pburl host1 host2 # hosts as args instead of stdin
node-lookup -R | pblastreport # <host> <last-report-time> <url>
pblastreport -r host1 # relative age (e.g. "3h ago")
pblastreport -z Asia/Singapore host1 # render the time in a specific IANA tz
```
Both read hostnames from arguments or the first field of each piped line (so
any `node-lookup` output mode works), de-duplicate, and share `node-lookup`'s
config file / env vars. `pblastreport` reads `report_timestamp` from the
PuppetDB v4 `nodes` endpoint (derived from the configured facts URL).
## Configuration ## Configuration
Precedence (lowest → highest): **defaults < config file < env vars < `--url` flag** Precedence (lowest → highest): **defaults < config file < env vars < `--url` flag**
@@ -50,6 +118,7 @@ XDG location: `$XDG_CONFIG_HOME/node-lookup/config.yaml` (default: `~/.config/no
```yaml ```yaml
puppetdb_url: http://puppetdbapi.service.consul:8080/pdb/query/v4/facts puppetdb_url: http://puppetdbapi.service.consul:8080/pdb/query/v4/facts
role_fact: enc_role role_fact: enc_role
puppetboard_url: https://puppetboard.k8s.syd1.au.unkin.net # used by pburl / pblastreport
``` ```
Generate the default config file: Generate the default config file:
@@ -68,19 +137,25 @@ Show the active configuration (after all overrides applied):
|---|---|---| |---|---|---|
| `NODE_LOOKUP_URL` | `puppetdb_url` | PuppetDB facts endpoint | | `NODE_LOOKUP_URL` | `puppetdb_url` | PuppetDB facts endpoint |
| `NODE_LOOKUP_ROLE_FACT` | `role_fact` | Fact name used by `-R` flag | | `NODE_LOOKUP_ROLE_FACT` | `role_fact` | Fact name used by `-R` flag |
| `NODE_LOOKUP_PUPPETBOARD_URL` | `puppetboard_url` | Puppetboard base URL (pburl / pblastreport) |
| `NODE_LOOKUP_DOMAIN` | `domain` | Domain appended to short (dotless) `-n` node names (default `main.unkin.net`) |
### CLI flag ### CLI flags
`--url <url>` overrides the PuppetDB URL for a single invocation (highest precedence). `--url <url>` overrides the PuppetDB URL for a single invocation (highest precedence).
`--domain <domain>` overrides the auto-qualify domain for a single invocation.
## Code Patterns ## Code Patterns
- **`loadConfig()`**: reads config file → applies env vars → returns `config` struct. Called once at startup in `main()`. - **`loadConfig()`**: reads config file → applies env vars → returns `config` struct. Called once at startup in `main()`.
- **`buildQuery()`**: returns a PuppetDB PQL-compatible JSON array string. Uses `roleFact` from config (not hardcoded). - **`buildQuery()`**: returns a PuppetDB PQL-compatible JSON array string. Uses `roleFact` from config (not hardcoded). Match modifiers: `-p` (partial/regex, uses `~` op), `-i` (inverse, wraps with `not`), composable.
- **Multiple facts**: `-F` accepts a comma-separated list (`ipaddress,enc_role`). `splitFactNames()`/`nameFilter()` turn several names into an `or` over `["=","name",<n>]` clauses; JSON output keys each value by the fact's real name so all requested facts appear per host.
- **Match value / `matchValue()`**: the value to match comes from `-m/--match` or, if that is empty, an optional positional argument. The positional fallback exists because pflag does not attach a space-separated value to a string flag grouped with a bool flag, so in `-pm k8s` the `k8s` arrives as a positional. `-m` still wins when both are given.
- **`queryPuppetDB(url, query)`**: takes the URL as a parameter — never reads globals. - **`queryPuppetDB(url, query)`**: takes the URL as a parameter — never reads globals.
- **`processResults()`**: iterates facts, returns sorted `"certname value"` strings. JSON string values are unquoted; other JSON types rendered as compact JSON. - **`processResults()`**: iterates facts, returns sorted `"certname value"` strings. JSON string values are unquoted; other JSON types rendered as compact JSON.
- **Output modes**: JSON (`-j`), count (`-C`), Ansible YAML (`-A`), node-only (`-1`), value-only (`-2`), default (node + value). - **Output modes**: JSON (`-j`), count (`-C`), Ansible YAML (`-A`), node-only (`-1`), value-only (`-2`), default (node + value). `-j` and `-A` share `factsByHost()`, so both attach the queried fact(s) per host — as an object under the host (`-j`) or as inventory host vars (`-A`).
- **Stdin support**: when stdin is not a TTY and no `-n` is given, node names are read line-by-line and queried individually (one HTTP request per node). - **Short node names / `qualifyNode()`**: a `-n` value (and stdin-sourced node names) with no dot is auto-qualified to `<name>.<domain>` (domain defaults to `main.unkin.net`, overridable via `--domain`/`NODE_LOOKUP_DOMAIN`), so `-n ausyd1nxvm2120` resolves the same as its FQDN. A name that already contains a dot (any domain) is left unchanged; a single trailing dot is stripped; empty input is preserved.
- **Stdin support**: `stdinReader()` reads node names from stdin only when it is a real pipe/redirect carrying data (and no `-n` given). Terminals, `/dev/null`, and empty/closed pipes fall through to a normal query — so running without a TTY (e.g. invoked by an agent or CI) behaves like an interactive run instead of consuming empty input.
- **SIGPIPE handling**: `signal.Ignore(syscall.SIGPIPE)` so pipes to `head` etc. work cleanly. - **SIGPIPE handling**: `signal.Ignore(syscall.SIGPIPE)` so pipes to `head` etc. work cleanly.
## CLI Framework ## CLI Framework
@@ -89,11 +164,19 @@ Uses [Cobra](https://github.com/spf13/cobra). Root command is the query command.
## Testing ## Testing
No test suite exists. Manual testing requires access to the Consul/PuppetDB environment or a mock HTTP server. ```bash
make test # go test -v -race ./...
```
`main_test.go` covers query construction (all `-m`/`-p`/`-i` combinations), value
rendering, result processing/counting, config precedence (defaults < file < env),
`writeDefaultConfig`, the `stdinReader` no-TTY behavior, and every `run()` output
mode (default, `-1`, `-2`, `-C`, `-j`, `-A`, `-a`). PuppetDB is stubbed with
`httptest` — no live Consul/PuppetDB access is required.
## Gotchas ## Gotchas
- `-1`, `-2`, `-C`, and `-A` all require `-R` or `-F`; the tool exits with an error otherwise. - `-1`, `-2`, `-C`, and `-A` all require `-R` or `-F`; the tool exits with an error otherwise.
- `-C` (count) with stdin reads all lines as pre-fetched `"node value"` output for counting — it does **not** query PuppetDB per line. - `-C` (count) with stdin reads all lines as pre-fetched `"node value"` output for counting — it does **not** query PuppetDB per line.
- JSON output (`-j`) builds `{ hostname: { factname: value } }` where the fact key is the `-F` value, the `role_fact` config value (if `-R`), or `"value"` as fallback. - JSON output (`-j`) builds `{ hostname: { factname: value } }` keyed by each result's actual fact name (so `-F ipaddress,enc_role` yields both per host); it falls back to the `-F` value, the `role_fact` config value (if `-R`), or `"value"` only when a result carries no name.
- `config init` fails if the config file already exists (will not overwrite). - `config init` fails if the config file already exists (will not overwrite).
+41 -6
View File
@@ -1,13 +1,29 @@
BINARY := node-lookup BINARY := node-lookup
VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo dev) # All shipped binaries and the package path each is built from. node-lookup is
GOFLAGS := -ldflags="-s -w -X main.version=$(VERSION)" # the module root; the companion tools live under cmd/.
BINARIES := node-lookup pburl pblastreport
DIST := dist
VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo dev)
GOFLAGS := -ldflags="-s -w -X main.version=$(VERSION)"
OS ?= $(shell go env GOOS)
ARCH ?= $(shell go env GOARCH)
.PHONY: all build test lint clean install patch minor major _tag # The Go package path for a binary: node-lookup is the module root, the
# companion tools live under cmd/. Usable inside a shell for-loop over $(BINARIES).
pkgpath = $$([ "$$b" = "node-lookup" ] && echo . || echo ./cmd/$$b)
.PHONY: all build test lint fmt clean install completions rpm rpm-package patch minor major _tag
all: build all: build
# Build every binary into dist/ so the nfpm packaging step
# (scripts/build-rpm.sh) can find them. Each main package needs its own -o, so
# they are built individually rather than with a single ./... invocation.
build: build:
go build $(GOFLAGS) -o $(BINARY) ./... @for b in $(BINARIES); do \
echo "building $$b"; \
CGO_ENABLED=0 GOOS=$(OS) GOARCH=$(ARCH) go build $(GOFLAGS) -o $(DIST)/$$b $(pkgpath) || exit 1; \
done
test: test:
go test -v -race ./... go test -v -race ./...
@@ -15,12 +31,31 @@ test:
lint: lint:
golangci-lint run ./... golangci-lint run ./...
fmt:
gofmt -w .
clean: clean:
rm -f $(BINARY) rm -rf $(DIST) $(BINARIES)
install: install:
go install $(GOFLAGS) ./... go install $(GOFLAGS) ./...
# Generate bash/zsh/fish completions for every binary into dist/completions.
completions: build
@mkdir -p $(DIST)/completions
@for b in $(BINARIES); do \
$(DIST)/$$b completion bash > $(DIST)/completions/$$b.bash; \
$(DIST)/$$b completion zsh > $(DIST)/completions/_$$b; \
$(DIST)/$$b completion fish > $(DIST)/completions/$$b.fish; \
done
# Build the binary then package it (with completions) into an RPM via nfpm.
rpm: build rpm-package
# Package an already-built binary into an RPM (used by CI after the build step).
rpm-package:
./scripts/build-rpm.sh $(VERSION)
# Bump helpers — reads the latest semver tag and creates the next one. # Bump helpers — reads the latest semver tag and creates the next one.
# If no tag exists yet, starts from v0.0.0. # If no tag exists yet, starts from v0.0.0.
_LATEST := $(shell git tag --sort=-v:refname | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$$' | head -1) _LATEST := $(shell git tag --sort=-v:refname | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$$' | head -1)
+97
View File
@@ -0,0 +1,97 @@
// Command pblastreport shows each host's last Puppet report time alongside its
// Puppetboard node-page URL.
//
// It reads hostnames from its arguments or from piped node-lookup output,
// queries PuppetDB for each node's report_timestamp, and prints a
// tab-separated "<host> <last-report> <puppetboard-url>" line:
//
// node-lookup -R | pblastreport
// pblastreport --relative host1.example.net
// pblastreport --timezone Asia/Singapore host1.example.net
package main
import (
"fmt"
"os"
"time"
"node-lookup/internal/puppet"
"github.com/spf13/cobra"
)
var version = "dev"
func main() {
cfg, err := puppet.Load()
if err != nil {
fmt.Fprintln(os.Stderr, "config error:", err)
os.Exit(1)
}
var (
relative bool
tz string
boardURL string
pdbURL string
)
root := &cobra.Command{
Use: "pblastreport [host...]",
Short: "Show each host's last Puppet report time and Puppetboard URL.",
Long: "Reads hostnames from arguments or piped node-lookup output and prints, per\n" +
"host, the time of its last Puppet report and its Puppetboard node-page URL.\n" +
"Times are shown in the local timezone unless --timezone is given, or as a\n" +
"relative age with --relative. Example: node-lookup -R | pblastreport -r",
Args: cobra.ArbitraryArgs,
SilenceUsage: true,
RunE: func(cmd *cobra.Command, args []string) error {
if cmd.Flags().Changed("puppetboard-url") {
cfg.PuppetboardURL = boardURL
}
if cmd.Flags().Changed("url") {
cfg.PuppetDBURL = pdbURL
}
loc := time.Local
if tz != "" {
l, err := time.LoadLocation(tz)
if err != nil {
return fmt.Errorf("invalid timezone %q: %w", tz, err)
}
loc = l
}
hosts := puppet.ReadHosts(os.Stdin, args)
if len(hosts) == 0 {
return fmt.Errorf("no hosts given (pass as arguments or pipe node-lookup output)")
}
nodesURL := puppet.NodesEndpoint(cfg.PuppetDBURL)
now := time.Now()
for _, h := range hosts {
node, lookupErr := puppet.LookupNode(nodesURL, h)
when := formatWhen(node, lookupErr, relative, loc, now)
fmt.Printf("%s\t%s\t%s\n", h, when, puppet.HostPageURL(cfg.PuppetboardURL, h))
}
return nil
},
}
f := root.Flags()
f.BoolVarP(&relative, "relative", "r", false, "Show the report time as a relative age (e.g. '3h ago')")
f.StringVarP(&tz, "timezone", "z", "", "IANA timezone for the report time (e.g. Asia/Singapore); default local")
f.StringVar(&boardURL, "puppetboard-url", cfg.PuppetboardURL, "Puppetboard base URL (overrides config and NODE_LOOKUP_PUPPETBOARD_URL)")
f.StringVar(&pdbURL, "url", cfg.PuppetDBURL, "PuppetDB facts URL (overrides config and NODE_LOOKUP_URL)")
root.AddCommand(&cobra.Command{
Use: "version",
Short: "Print the version",
Run: func(cmd *cobra.Command, args []string) { fmt.Println(version) },
SilenceUsage: true,
})
if err := root.Execute(); err != nil {
os.Exit(1)
}
}
+53
View File
@@ -0,0 +1,53 @@
package main
import (
"fmt"
"time"
"node-lookup/internal/puppet"
)
// formatWhen renders the "last report" column for a node lookup result. It
// handles the lookup error, the unknown-node, and the never-reported cases so
// the output always has a value in every column.
func formatWhen(node *puppet.Node, lookupErr error, relative bool, loc *time.Location, now time.Time) string {
if lookupErr != nil {
return "error: " + lookupErr.Error()
}
if node == nil {
return "unknown node"
}
if node.ReportTimestamp == "" {
return "no report"
}
ts, err := time.Parse(time.RFC3339Nano, node.ReportTimestamp)
if err != nil {
return node.ReportTimestamp // fall back to the raw value
}
if relative {
return humanizeSince(now, ts)
}
return ts.In(loc).Format("2006-01-02 15:04:05 MST")
}
// humanizeSince renders the gap between now and ts as a coarse relative string
// ("42s ago", "9m ago", "3h ago", "5d ago"). Future timestamps (clock skew)
// render as "in <d>".
func humanizeSince(now, ts time.Time) string {
d := now.Sub(ts)
suffix := "ago"
if d < 0 {
d = -d
suffix = "from now"
}
switch {
case d < time.Minute:
return fmt.Sprintf("%ds %s", int(d.Seconds()), suffix)
case d < time.Hour:
return fmt.Sprintf("%dm %s", int(d.Minutes()), suffix)
case d < 24*time.Hour:
return fmt.Sprintf("%dh %s", int(d.Hours()), suffix)
default:
return fmt.Sprintf("%dd %s", int(d.Hours()/24), suffix)
}
}
+84
View File
@@ -0,0 +1,84 @@
package main
import (
"errors"
"strings"
"testing"
"time"
"node-lookup/internal/puppet"
)
func mustTime(t *testing.T, s string) time.Time {
t.Helper()
ts, err := time.Parse(time.RFC3339Nano, s)
if err != nil {
t.Fatal(err)
}
return ts
}
func TestFormatWhen_Absolute(t *testing.T) {
node := &puppet.Node{ReportTimestamp: "2026-07-15T04:05:06.000Z"}
now := mustTime(t, "2026-07-15T10:00:00Z")
got := formatWhen(node, nil, false, time.UTC, now)
if got != "2026-07-15 04:05:06 UTC" {
t.Fatalf("unexpected absolute time: %q", got)
}
}
func TestFormatWhen_TimezoneApplied(t *testing.T) {
loc, err := time.LoadLocation("Asia/Singapore") // UTC+8, no DST
if err != nil {
t.Skipf("tzdata unavailable: %v", err)
}
node := &puppet.Node{ReportTimestamp: "2026-07-15T04:05:06Z"}
now := mustTime(t, "2026-07-15T10:00:00Z")
got := formatWhen(node, nil, false, loc, now)
if !strings.HasPrefix(got, "2026-07-15 12:05:06") {
t.Fatalf("expected +08 time, got %q", got)
}
}
func TestFormatWhen_Relative(t *testing.T) {
node := &puppet.Node{ReportTimestamp: "2026-07-15T07:00:00Z"}
now := mustTime(t, "2026-07-15T10:00:00Z")
if got := formatWhen(node, nil, true, time.UTC, now); got != "3h ago" {
t.Fatalf("expected '3h ago', got %q", got)
}
}
func TestFormatWhen_EdgeCases(t *testing.T) {
now := mustTime(t, "2026-07-15T10:00:00Z")
if got := formatWhen(nil, errors.New("down"), false, time.UTC, now); !strings.HasPrefix(got, "error:") {
t.Fatalf("expected error passthrough, got %q", got)
}
if got := formatWhen(nil, nil, false, time.UTC, now); got != "unknown node" {
t.Fatalf("expected unknown node, got %q", got)
}
if got := formatWhen(&puppet.Node{ReportTimestamp: ""}, nil, false, time.UTC, now); got != "no report" {
t.Fatalf("expected no report, got %q", got)
}
if got := formatWhen(&puppet.Node{ReportTimestamp: "garbage"}, nil, false, time.UTC, now); got != "garbage" {
t.Fatalf("expected raw fallback, got %q", got)
}
}
func TestHumanizeSince(t *testing.T) {
base := mustTime(t, "2026-07-15T10:00:00Z")
cases := []struct {
ts string
want string
}{
{"2026-07-15T09:59:30Z", "30s ago"},
{"2026-07-15T09:45:00Z", "15m ago"},
{"2026-07-15T05:00:00Z", "5h ago"},
{"2026-07-13T10:00:00Z", "2d ago"},
{"2026-07-15T10:01:00Z", "1m from now"},
}
for _, c := range cases {
if got := humanizeSince(base, mustTime(t, c.ts)); got != c.want {
t.Errorf("humanizeSince(%s) = %q, want %q", c.ts, got, c.want)
}
}
}
+63
View File
@@ -0,0 +1,63 @@
// Command pburl prints the Puppetboard node-page URL for each host it is given.
//
// It reads hostnames from its arguments or from piped node-lookup output and
// emits "<host> <puppetboard-url>" per line:
//
// node-lookup -R | pburl
// pburl host1.example.net host2.example.net
package main
import (
"fmt"
"os"
"node-lookup/internal/puppet"
"github.com/spf13/cobra"
)
var version = "dev"
func main() {
cfg, err := puppet.Load()
if err != nil {
fmt.Fprintln(os.Stderr, "config error:", err)
os.Exit(1)
}
var boardURL string
root := &cobra.Command{
Use: "pburl [host...]",
Short: "Print the Puppetboard node-page URL for each host.",
Long: "Reads hostnames from arguments or piped node-lookup output and prints\n" +
"'<host> <puppetboard-url>' for each. Example: node-lookup -R | pburl",
Args: cobra.ArbitraryArgs,
SilenceUsage: true,
RunE: func(cmd *cobra.Command, args []string) error {
if cmd.Flags().Changed("puppetboard-url") {
cfg.PuppetboardURL = boardURL
}
hosts := puppet.ReadHosts(os.Stdin, args)
if len(hosts) == 0 {
return fmt.Errorf("no hosts given (pass as arguments or pipe node-lookup output)")
}
for _, h := range hosts {
fmt.Printf("%s %s\n", h, puppet.HostPageURL(cfg.PuppetboardURL, h))
}
return nil
},
}
root.Flags().StringVar(&boardURL, "puppetboard-url", cfg.PuppetboardURL, "Puppetboard base URL (overrides config and NODE_LOOKUP_PUPPETBOARD_URL)")
root.AddCommand(&cobra.Command{
Use: "version",
Short: "Print the version",
Run: func(cmd *cobra.Command, args []string) { fmt.Println(version) },
SilenceUsage: true,
})
if err := root.Execute(); err != nil {
os.Exit(1)
}
}
+7 -4
View File
@@ -3,8 +3,11 @@ module node-lookup
go 1.25.7 go 1.25.7
require ( require (
github.com/inconshreveable/mousetrap v1.1.0 // indirect github.com/spf13/cobra v1.10.2
github.com/spf13/cobra v1.10.2 // indirect gopkg.in/yaml.v3 v3.0.1
github.com/spf13/pflag v1.0.9 // indirect )
gopkg.in/yaml.v3 v3.0.1 // indirect
require (
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/spf13/pflag v1.0.9 // indirect
) )
+1
View File
@@ -7,6 +7,7 @@ github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiT
github.com/spf13/pflag v1.0.9 h1:9exaQaMOCwffKiiiYk6/BndUBv+iRViNW+4lEMi0PvY= github.com/spf13/pflag v1.0.9 h1:9exaQaMOCwffKiiiYk6/BndUBv+iRViNW+4lEMi0PvY=
github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+9
View File
@@ -0,0 +1,9 @@
package puppet
import "strings"
// HostPageURL returns the Puppetboard node-detail page URL for a certname,
// e.g. https://puppetboard.example.net/node/host1.example.net.
func HostPageURL(base, certname string) string {
return strings.TrimRight(base, "/") + "/node/" + certname
}
+90
View File
@@ -0,0 +1,90 @@
// Package puppet holds the small pieces of PuppetDB/Puppetboard plumbing shared
// by the node-lookup companion tools (pburl, pblastreport): config loading,
// PuppetDB "nodes" queries, Puppetboard URL construction, and reading hostnames
// from piped node-lookup output.
//
// It intentionally reads the SAME config file, env vars, and defaults as the
// node-lookup CLI so a single `~/.config/node-lookup/config.yaml` configures
// every tool in the family.
package puppet
import (
"fmt"
"os"
"path/filepath"
"gopkg.in/yaml.v3"
)
const (
// DefaultPuppetDBURL is the PuppetDB v4 facts endpoint (shared with node-lookup).
DefaultPuppetDBURL = "http://puppetdbapi.service.consul:8080/pdb/query/v4/facts"
// DefaultRoleFact is the role fact node-lookup queries with -R.
DefaultRoleFact = "enc_role"
// DefaultPuppetboardURL is the base URL of the Puppetboard web UI.
DefaultPuppetboardURL = "https://puppetboard.k8s.syd1.au.unkin.net"
appName = "node-lookup"
configFileName = "config.yaml"
)
// Config mirrors node-lookup's config plus the puppetboard_url key used by the
// companion tools. Fields map 1:1 to config file keys and env vars.
type Config struct {
PuppetDBURL string `yaml:"puppetdb_url"`
RoleFact string `yaml:"role_fact"`
PuppetboardURL string `yaml:"puppetboard_url"`
}
// DefaultConfig returns the built-in defaults.
func DefaultConfig() Config {
return Config{
PuppetDBURL: DefaultPuppetDBURL,
RoleFact: DefaultRoleFact,
PuppetboardURL: DefaultPuppetboardURL,
}
}
// ConfigDir returns the XDG_CONFIG_HOME/node-lookup directory.
func ConfigDir() string {
base := os.Getenv("XDG_CONFIG_HOME")
if base == "" {
home, _ := os.UserHomeDir()
base = filepath.Join(home, ".config")
}
return filepath.Join(base, appName)
}
// ConfigPath returns the full path to the shared config file.
func ConfigPath() string {
return filepath.Join(ConfigDir(), configFileName)
}
// Load reads the config file (if present), then applies env var overrides.
// Precedence (lowest → highest): defaults < config file < env vars.
func Load() (Config, error) {
cfg := DefaultConfig()
path := ConfigPath()
data, err := os.ReadFile(path)
if err != nil && !os.IsNotExist(err) {
return cfg, fmt.Errorf("reading config %s: %w", path, err)
}
if err == nil {
if err := yaml.Unmarshal(data, &cfg); err != nil {
return cfg, fmt.Errorf("parsing config %s: %w", path, err)
}
}
if v := os.Getenv("NODE_LOOKUP_URL"); v != "" {
cfg.PuppetDBURL = v
}
if v := os.Getenv("NODE_LOOKUP_ROLE_FACT"); v != "" {
cfg.RoleFact = v
}
if v := os.Getenv("NODE_LOOKUP_PUPPETBOARD_URL"); v != "" {
cfg.PuppetboardURL = v
}
return cfg, nil
}
+177
View File
@@ -0,0 +1,177 @@
package puppet
import (
"encoding/json"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
)
// ---- config -----------------------------------------------------------------
func TestLoad_Defaults(t *testing.T) {
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
t.Setenv("NODE_LOOKUP_URL", "")
t.Setenv("NODE_LOOKUP_ROLE_FACT", "")
t.Setenv("NODE_LOOKUP_PUPPETBOARD_URL", "")
cfg, err := Load()
if err != nil {
t.Fatal(err)
}
if cfg.PuppetDBURL != DefaultPuppetDBURL {
t.Fatalf("expected default puppetdb url, got %s", cfg.PuppetDBURL)
}
if cfg.PuppetboardURL != DefaultPuppetboardURL {
t.Fatalf("expected default puppetboard url, got %s", cfg.PuppetboardURL)
}
}
func TestLoad_FileAndEnvOverride(t *testing.T) {
dir := t.TempDir()
t.Setenv("XDG_CONFIG_HOME", dir)
t.Setenv("NODE_LOOKUP_URL", "")
t.Setenv("NODE_LOOKUP_ROLE_FACT", "")
t.Setenv("NODE_LOOKUP_PUPPETBOARD_URL", "https://env.example.net")
cfgDir := filepath.Join(dir, appName)
if err := os.MkdirAll(cfgDir, 0o755); err != nil {
t.Fatal(err)
}
body := "puppetdb_url: http://file:8080/pdb/query/v4/facts\npuppetboard_url: https://file.example.net\n"
if err := os.WriteFile(filepath.Join(cfgDir, configFileName), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
cfg, err := Load()
if err != nil {
t.Fatal(err)
}
if cfg.PuppetDBURL != "http://file:8080/pdb/query/v4/facts" {
t.Fatalf("file override failed: %s", cfg.PuppetDBURL)
}
// env beats the file for puppetboard_url
if cfg.PuppetboardURL != "https://env.example.net" {
t.Fatalf("env should beat file: %s", cfg.PuppetboardURL)
}
}
// ---- NodesEndpoint ----------------------------------------------------------
func TestNodesEndpoint(t *testing.T) {
cases := map[string]string{
"http://puppetdbapi.service.consul:8080/pdb/query/v4/facts": "http://puppetdbapi.service.consul:8080/pdb/query/v4/nodes",
"http://h:8080/pdb/query/v4/facts/": "http://h:8080/pdb/query/v4/nodes",
"https://h/facts": "https://h/nodes",
}
for in, want := range cases {
if got := NodesEndpoint(in); got != want {
t.Errorf("NodesEndpoint(%q) = %q, want %q", in, got, want)
}
}
}
// ---- HostPageURL ------------------------------------------------------------
func TestHostPageURL(t *testing.T) {
if got := HostPageURL("https://pb.example.net", "h1.example.net"); got != "https://pb.example.net/node/h1.example.net" {
t.Fatalf("unexpected url: %s", got)
}
// trailing slash on the base is trimmed
if got := HostPageURL("https://pb.example.net/", "h1"); got != "https://pb.example.net/node/h1" {
t.Fatalf("trailing slash not handled: %s", got)
}
}
// ---- LookupNode -------------------------------------------------------------
func TestLookupNode_Found(t *testing.T) {
var gotQuery string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotQuery = r.URL.Query().Get("query")
_ = json.NewEncoder(w).Encode([]Node{{
Certname: "h1",
ReportTimestamp: "2026-07-15T04:05:06.000Z",
LatestReportStatus: "changed",
}})
}))
defer srv.Close()
node, err := LookupNode(srv.URL, "h1")
if err != nil {
t.Fatal(err)
}
if node == nil || node.ReportTimestamp != "2026-07-15T04:05:06.000Z" {
t.Fatalf("unexpected node: %+v", node)
}
if !strings.Contains(gotQuery, "certname") || !strings.Contains(gotQuery, "h1") {
t.Fatalf("query missing certname filter: %s", gotQuery)
}
}
func TestLookupNode_Unknown(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_ = json.NewEncoder(w).Encode([]Node{})
}))
defer srv.Close()
node, err := LookupNode(srv.URL, "nope")
if err != nil {
t.Fatal(err)
}
if node != nil {
t.Fatalf("expected nil node for unknown certname, got %+v", node)
}
}
func TestLookupNode_HTTPError(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Error(w, "boom", http.StatusInternalServerError)
}))
defer srv.Close()
if _, err := LookupNode(srv.URL, "h1"); err == nil {
t.Fatal("expected error for HTTP 500")
}
}
// ---- ReadHosts --------------------------------------------------------------
func TestReadHosts_ArgsWin(t *testing.T) {
// A real pipe with data present, but explicit args should take precedence.
r, w, _ := os.Pipe()
go func() { _, _ = w.WriteString("piped\n"); _ = w.Close() }()
defer func() { _ = r.Close() }()
got := ReadHosts(r, []string{"a", "b", "a"})
if strings.Join(got, ",") != "a,b" {
t.Fatalf("expected deduped args, got %v", got)
}
}
func TestReadHosts_StdinFirstField(t *testing.T) {
r, w, _ := os.Pipe()
go func() {
// node-lookup default output: "host value"; also a bare host and a dup.
_, _ = w.WriteString("host1 roles::web\nhost2 roles::db\nhost1 roles::web\nhost3\n")
_ = w.Close()
}()
defer func() { _ = r.Close() }()
got := ReadHosts(r, nil)
if strings.Join(got, ",") != "host1,host2,host3" {
t.Fatalf("expected first-field hosts deduped, got %v", got)
}
}
func TestReadHosts_NoInput(t *testing.T) {
// /dev/null is a char device: no args, no pipe data -> nil.
f, _ := os.Open(os.DevNull)
defer func() { _ = f.Close() }()
if got := ReadHosts(f, nil); got != nil {
t.Fatalf("expected nil for no input, got %v", got)
}
}
+71
View File
@@ -0,0 +1,71 @@
package puppet
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"strings"
)
// Node is the subset of a PuppetDB v4 "nodes" record the companion tools use.
type Node struct {
Certname string `json:"certname"`
ReportTimestamp string `json:"report_timestamp"`
LatestReportStatus string `json:"latest_report_status"`
}
// NodesEndpoint derives the PuppetDB v4 "nodes" query endpoint from the
// configured facts endpoint by swapping the final path segment
// (…/pdb/query/v4/facts → …/pdb/query/v4/nodes). It leaves scheme/host/query
// untouched, so a custom NODE_LOOKUP_URL still resolves correctly.
func NodesEndpoint(factsURL string) string {
u, err := url.Parse(factsURL)
if err != nil {
return factsURL
}
p := strings.TrimRight(u.Path, "/")
if i := strings.LastIndex(p, "/"); i >= 0 {
p = p[:i] + "/nodes"
} else {
p = "/nodes"
}
u.Path = p
return u.String()
}
// LookupNode fetches the single PuppetDB node record for certname. It returns
// (nil, nil) when PuppetDB knows of no such node.
func LookupNode(nodesURL, certname string) (*Node, error) {
q, _ := json.Marshal([]interface{}{"=", "certname", certname})
nodes, err := queryNodes(nodesURL, string(q))
if err != nil {
return nil, err
}
if len(nodes) == 0 {
return nil, nil
}
return &nodes[0], nil
}
func queryNodes(nodesURL, query string) ([]Node, error) {
params := url.Values{}
params.Set("query", query)
resp, err := http.Get(nodesURL + "?" + params.Encode())
if err != nil {
return nil, fmt.Errorf("request failed: %w", err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
return nil, fmt.Errorf("HTTP %d: %s", resp.StatusCode, string(body))
}
var nodes []Node
if err := json.NewDecoder(resp.Body).Decode(&nodes); err != nil {
return nil, fmt.Errorf("decode error: %w", err)
}
return nodes, nil
}
+66
View File
@@ -0,0 +1,66 @@
package puppet
import (
"bufio"
"os"
"strings"
)
// StdinReader returns a buffered reader over f and true only when f actually
// carries piped/redirected data. Terminals and character devices such as
// /dev/null return false, and an empty pipe or empty file (immediate EOF on
// peek) also returns false. This mirrors node-lookup's no-TTY behaviour: when
// invoked without a real pipe the caller can fall back to arguments instead of
// blocking on or silently consuming empty input.
func StdinReader(f *os.File) (*bufio.Reader, bool) {
fi, err := f.Stat()
if err != nil {
return nil, false
}
if (fi.Mode() & os.ModeCharDevice) != 0 {
return nil, false // terminal or /dev/null
}
r := bufio.NewReader(f)
if _, err := r.Peek(1); err != nil {
return nil, false // empty pipe / empty file (EOF)
}
return r, true
}
// ReadHosts resolves the list of hostnames to act on. Explicit args win; failing
// that it reads the first whitespace-separated field of each non-empty line from
// stdin (so `node-lookup -R | pburl` and `node-lookup -1 | pburl` both work).
// Order is preserved and duplicates are removed. Returns nil when neither args
// nor piped stdin data are present.
func ReadHosts(stdin *os.File, args []string) []string {
if len(args) > 0 {
return dedupe(args)
}
r, ok := StdinReader(stdin)
if !ok {
return nil
}
var hosts []string
sc := bufio.NewScanner(r)
for sc.Scan() {
fields := strings.Fields(sc.Text())
if len(fields) == 0 {
continue
}
hosts = append(hosts, fields[0])
}
return dedupe(hosts)
}
func dedupe(in []string) []string {
seen := make(map[string]struct{}, len(in))
out := make([]string, 0, len(in))
for _, s := range in {
if _, ok := seen[s]; ok {
continue
}
seen[s] = struct{}{}
out = append(out, s)
}
return out
}
+183 -60
View File
@@ -19,10 +19,12 @@ import (
) )
const ( const (
defaultPuppetDBURL = "http://puppetdbapi.service.consul:8080/pdb/query/v4/facts" defaultPuppetDBURL = "http://puppetdbapi.service.consul:8080/pdb/query/v4/facts"
defaultRoleFact = "enc_role" defaultRoleFact = "enc_role"
configFileName = "config.yaml" defaultPuppetboardURL = "https://puppetboard.k8s.syd1.au.unkin.net"
appName = "node-lookup" defaultDomain = "main.unkin.net"
configFileName = "config.yaml"
appName = "node-lookup"
) )
var version = "dev" var version = "dev"
@@ -32,12 +34,20 @@ var version = "dev"
type config struct { type config struct {
PuppetDBURL string `yaml:"puppetdb_url"` PuppetDBURL string `yaml:"puppetdb_url"`
RoleFact string `yaml:"role_fact"` RoleFact string `yaml:"role_fact"`
// PuppetboardURL is not used by node-lookup itself; it is scaffolded here so
// the shared config file also configures the companion tools (pburl,
// pblastreport) that read this same file.
PuppetboardURL string `yaml:"puppetboard_url"`
// Domain is appended to a short (dotless) -n node name to form its FQDN.
Domain string `yaml:"domain"`
} }
func defaultConfig() config { func defaultConfig() config {
return config{ return config{
PuppetDBURL: defaultPuppetDBURL, PuppetDBURL: defaultPuppetDBURL,
RoleFact: defaultRoleFact, RoleFact: defaultRoleFact,
PuppetboardURL: defaultPuppetboardURL,
Domain: defaultDomain,
} }
} }
@@ -78,6 +88,12 @@ func loadConfig() (config, error) {
if v := os.Getenv("NODE_LOOKUP_ROLE_FACT"); v != "" { if v := os.Getenv("NODE_LOOKUP_ROLE_FACT"); v != "" {
cfg.RoleFact = v cfg.RoleFact = v
} }
if v := os.Getenv("NODE_LOOKUP_PUPPETBOARD_URL"); v != "" {
cfg.PuppetboardURL = v
}
if v := os.Getenv("NODE_LOOKUP_DOMAIN"); v != "" {
cfg.Domain = v
}
return cfg, nil return cfg, nil
} }
@@ -96,7 +112,7 @@ func writeDefaultConfig() error {
cfg := defaultConfig() cfg := defaultConfig()
data, _ := yaml.Marshal(cfg) data, _ := yaml.Marshal(cfg)
header := []byte("# node-lookup configuration\n# Fields can be overridden with env vars: NODE_LOOKUP_URL, NODE_LOOKUP_ROLE_FACT\n\n") header := []byte("# node-lookup configuration\n# Fields can be overridden with env vars: NODE_LOOKUP_URL, NODE_LOOKUP_ROLE_FACT, NODE_LOOKUP_PUPPETBOARD_URL, NODE_LOOKUP_DOMAIN\n# puppetboard_url is used by the companion tools (pburl, pblastreport).\n# domain is appended to short (dotless) -n node names to form their FQDN.\n\n")
if err := os.WriteFile(path, append(header, data...), 0o644); err != nil { if err := os.WriteFile(path, append(header, data...), 0o644); err != nil {
return fmt.Errorf("writing config: %w", err) return fmt.Errorf("writing config: %w", err)
} }
@@ -110,23 +126,70 @@ type fact struct {
Value json.RawMessage `json:"value"` Value json.RawMessage `json:"value"`
} }
func buildQuery(node, factName, match, partialMatch, roleFact string, showRole bool) string { // splitFactNames splits a comma-separated -F value into trimmed, non-empty
// names, so `-F ipaddress,enc_role` queries both facts.
func splitFactNames(factName string) []string {
var names []string
for _, n := range strings.Split(factName, ",") {
if n = strings.TrimSpace(n); n != "" {
names = append(names, n)
}
}
return names
}
// nameFilter returns a PQL filter matching any of the given fact names: a plain
// equality for one name, an "or" over per-name equalities for several.
func nameFilter(names []string) []interface{} {
if len(names) == 1 {
return []interface{}{"=", "name", names[0]}
}
or := []interface{}{"or"}
for _, n := range names {
or = append(or, []interface{}{"=", "name", n})
}
return or
}
// qualifyNode auto-qualifies a short (dotless) node name by appending
// ".<domain>", so `-n ausyd1nxvm2120` resolves the same as its FQDN. A name
// that already contains a dot is treated as already-qualified (including names
// in other domains like *.k8s.syd1.au.unkin.net) and returned unchanged. A
// single trailing dot is stripped first, so a dotless name with a trailing dot
// is still qualified. Empty input is returned unchanged to preserve the
// existing "no node given" behavior.
func qualifyNode(name, domain string) string {
name = strings.TrimSuffix(name, ".")
if name == "" || strings.Contains(name, ".") {
return name
}
return name + "." + domain
}
func buildQuery(node, factName, match, roleFact string, showRole, partial, inverse bool) string {
type filter = []interface{} type filter = []interface{}
var filters []filter var filters []filter
if node != "" { if node != "" {
filters = append(filters, filter{"=", "certname", node}) filters = append(filters, filter{"=", "certname", node})
} }
if factName != "" { if names := splitFactNames(factName); len(names) > 0 {
filters = append(filters, filter{"=", "name", factName}) filters = append(filters, nameFilter(names))
} else if showRole { } else if showRole {
filters = append(filters, filter{"=", "name", roleFact}) filters = append(filters, filter{"=", "name", roleFact})
} }
if match != "" { if match != "" {
filters = append(filters, filter{"=", "value", match}) op := "="
} else if partialMatch != "" { if partial {
filters = append(filters, filter{"~", "value", partialMatch}) op = "~"
}
inner := filter{op, "value", match}
if inverse {
filters = append(filters, filter{"not", inner})
} else {
filters = append(filters, inner)
}
} }
if len(filters) == 0 { if len(filters) == 0 {
@@ -207,12 +270,66 @@ func countResults(lines []string) []string {
return out return out
} }
func isTerminal(f *os.File) bool { // stdinReader returns a buffered reader over f and true only when f actually
// carries piped/redirected data to consume as node names. Terminals and
// character devices such as /dev/null return false, and an empty pipe or empty
// file (immediate EOF on peek) also returns false. This means running without a
// TTY — e.g. invoked by an agent or CI where stdin is /dev/null or a closed
// pipe — falls through to a normal query instead of silently consuming empty
// input and printing nothing.
func stdinReader(f *os.File) (*bufio.Reader, bool) {
fi, err := f.Stat() fi, err := f.Stat()
if err != nil { if err != nil {
return false return nil, false
} }
return (fi.Mode() & os.ModeCharDevice) != 0 if (fi.Mode() & os.ModeCharDevice) != 0 {
return nil, false // terminal or /dev/null
}
r := bufio.NewReader(f)
if _, err := r.Peek(1); err != nil {
return nil, false // empty pipe / empty file (EOF)
}
return r, true
}
// matchValue resolves the value to match against. The -m/--match flag wins; if
// it is empty, the (optional) positional argument is used instead. The
// positional fallback exists so combined shorthands like `-pm k8s` work — pflag
// leaves the space-separated `k8s` as a positional rather than attaching it to
// the grouped -m flag.
func matchValue(flagMatch string, args []string) string {
if flagMatch != "" {
return flagMatch
}
if len(args) > 0 {
return args[0]
}
return ""
}
// factsByHost groups collected facts into {certname: {factname: value}}. Each
// value is keyed by the fact's real name, so multiple -F facts each appear
// under the host; it falls back to the -F string, the role fact (with -R), or
// "value" only when a result carries no name. Shared by the -j and -A outputs.
func factsByHost(collected []fact, factName, roleFact string, showRole bool) map[string]map[string]interface{} {
out := map[string]map[string]interface{}{}
for _, f := range collected {
if _, ok := out[f.Certname]; !ok {
out[f.Certname] = map[string]interface{}{}
}
key := f.Name
if key == "" {
if key = factName; key == "" {
if showRole {
key = roleFact
} else {
key = "value"
}
}
}
out[f.Certname][key] = valueAny(f.Value)
}
return out
} }
func allFactsForNode(puppetDBURL, node string) ([]fact, error) { func allFactsForNode(puppetDBURL, node string) ([]fact, error) {
@@ -220,9 +337,11 @@ func allFactsForNode(puppetDBURL, node string) ([]fact, error) {
return queryPuppetDB(puppetDBURL, string(query)) return queryPuppetDB(puppetDBURL, string(query))
} }
func run(cfg config, nodeName, factName, match, partialMatch string, showRole, nodeOnly, valueOnly, count, ansible, jsonMode, allFacts bool) error { func run(cfg config, nodeName, factName, match string, showRole, partial, inverse, nodeOnly, valueOnly, count, ansible, jsonMode, allFacts bool) error {
signal.Ignore(syscall.SIGPIPE) signal.Ignore(syscall.SIGPIPE)
nodeName = qualifyNode(nodeName, cfg.Domain)
if allFacts { if allFacts {
if nodeName == "" { if nodeName == "" {
return fmt.Errorf("-a requires -n") return fmt.Errorf("-a requires -n")
@@ -241,12 +360,15 @@ func run(cfg config, nodeName, factName, match, partialMatch string, showRole, n
if (nodeOnly || valueOnly || count || ansible) && !showRole && factName == "" { if (nodeOnly || valueOnly || count || ansible) && !showRole && factName == "" {
return fmt.Errorf("-R or -F must be used with -1, -2, -C, or -A") return fmt.Errorf("-R or -F must be used with -1, -2, -C, or -A")
} }
if (match != "" || partial || inverse) && !showRole && factName == "" {
return fmt.Errorf("-R or -F must be used with -m, -p, or -i")
}
var collected []fact var collected []fact
var stdinLines []string var stdinLines []string
doQuery := func(node string) error { doQuery := func(node string) error {
query := buildQuery(node, factName, match, partialMatch, cfg.RoleFact, showRole) query := buildQuery(node, factName, match, cfg.RoleFact, showRole, partial, inverse)
facts, err := queryPuppetDB(cfg.PuppetDBURL, query) facts, err := queryPuppetDB(cfg.PuppetDBURL, query)
if err != nil { if err != nil {
return err return err
@@ -255,8 +377,8 @@ func run(cfg config, nodeName, factName, match, partialMatch string, showRole, n
return nil return nil
} }
if nodeName == "" && !isTerminal(os.Stdin) { if reader, ok := stdinReader(os.Stdin); ok && nodeName == "" {
scanner := bufio.NewScanner(os.Stdin) scanner := bufio.NewScanner(reader)
if count { if count {
for scanner.Scan() { for scanner.Scan() {
stdinLines = append(stdinLines, scanner.Text()) stdinLines = append(stdinLines, scanner.Text())
@@ -267,7 +389,7 @@ func run(cfg config, nodeName, factName, match, partialMatch string, showRole, n
if len(fields) == 0 { if len(fields) == 0 {
continue continue
} }
if err := doQuery(fields[0]); err != nil { if err := doQuery(qualifyNode(fields[0], cfg.Domain)); err != nil {
fmt.Fprintln(os.Stderr, "error:", err) fmt.Fprintln(os.Stderr, "error:", err)
} }
} }
@@ -282,25 +404,10 @@ func run(cfg config, nodeName, factName, match, partialMatch string, showRole, n
switch { switch {
case jsonMode: case jsonMode:
hostFactMap := map[string]map[string]interface{}{}
for _, f := range collected {
if _, ok := hostFactMap[f.Certname]; !ok {
hostFactMap[f.Certname] = map[string]interface{}{}
}
key := factName
if key == "" {
if showRole {
key = cfg.RoleFact
} else {
key = "value"
}
}
hostFactMap[f.Certname][key] = valueAny(f.Value)
}
enc := json.NewEncoder(os.Stdout) enc := json.NewEncoder(os.Stdout)
enc.SetIndent("", " ") enc.SetIndent("", " ")
enc.SetEscapeHTML(false) enc.SetEscapeHTML(false)
_ = enc.Encode(hostFactMap) _ = enc.Encode(factsByHost(collected, factName, cfg.RoleFact, showRole))
case count: case count:
values := stdinLines values := stdinLines
@@ -310,10 +417,11 @@ func run(cfg config, nodeName, factName, match, partialMatch string, showRole, n
fmt.Println(strings.Join(countResults(values), "\n")) fmt.Println(strings.Join(countResults(values), "\n"))
case ansible: case ansible:
// Attach each host's queried fact(s) as inventory host vars, e.g.
// `-F ipaddress,enc_role -A` yields hosts with ipaddress + enc_role set.
hosts := map[string]interface{}{} hosts := map[string]interface{}{}
for _, line := range returnData { for host, vars := range factsByHost(collected, factName, cfg.RoleFact, showRole) {
host := strings.Fields(line)[0] hosts[host] = vars
hosts[host] = map[string]interface{}{}
} }
inventory := map[string]interface{}{ inventory := map[string]interface{}{
"all": map[string]interface{}{"hosts": hosts}, "all": map[string]interface{}{"hosts": hosts},
@@ -349,47 +457,60 @@ func main() {
} }
var ( var (
nodeName string nodeName string
factName string factName string
showRole bool showRole bool
match string match string
partialMatch string partial bool
nodeOnly bool inverse bool
valueOnly bool nodeOnly bool
count bool valueOnly bool
ansible bool count bool
jsonMode bool ansible bool
allFacts bool jsonMode bool
puppetDBURL string allFacts bool
puppetDBURL string
domain string
) )
rootCmd := &cobra.Command{ rootCmd := &cobra.Command{
Use: appName, Use: appName + " [value]",
Short: "Query PuppetDB for nodes.", Short: "Query PuppetDB for nodes.",
// Accept an optional positional match value in addition to -m. This makes
// combined shorthands like `-pm k8s` work: pflag does not attach a
// space-separated value to a string flag grouped with a bool flag (only
// `-pm=k8s` or `-p -m k8s` do), so `k8s` arrives here as a positional
// argument instead. Falling back to it keeps the ergonomic form working.
Args: cobra.MaximumNArgs(1),
PersistentPreRunE: func(cmd *cobra.Command, args []string) error { PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
if cmd.Flags().Changed("url") { if cmd.Flags().Changed("url") {
cfg.PuppetDBURL = puppetDBURL cfg.PuppetDBURL = puppetDBURL
} }
if cmd.Flags().Changed("domain") {
cfg.Domain = domain
}
return nil return nil
}, },
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
return run(cfg, nodeName, factName, match, partialMatch, showRole, nodeOnly, valueOnly, count, ansible, jsonMode, allFacts) return run(cfg, nodeName, factName, matchValue(match, args), showRole, partial, inverse, nodeOnly, valueOnly, count, ansible, jsonMode, allFacts)
}, },
SilenceUsage: true, SilenceUsage: true,
} }
f := rootCmd.Flags() f := rootCmd.Flags()
f.StringVarP(&nodeName, "node", "n", "", "Node name") f.StringVarP(&nodeName, "node", "n", "", "Node name")
f.StringVarP(&factName, "fact", "F", "", "Fact name") f.StringVarP(&factName, "fact", "F", "", "Fact name (comma-separated for several, e.g. -F ipaddress,enc_role)")
f.BoolVarP(&showRole, "role", "R", false, "Show role fact ("+defaultRoleFact+" by default)") f.BoolVarP(&showRole, "role", "R", false, "Show role fact ("+defaultRoleFact+" by default)")
f.StringVarP(&match, "match", "m", "", "Exact value match") f.StringVarP(&match, "match", "m", "", "Value to match (use with -p and/or -i)")
f.StringVar(&partialMatch, "pm", "", "Partial match on value") f.BoolVarP(&partial, "partial", "p", false, "Partial/regex match modifier (combine with -m)")
f.BoolVarP(&inverse, "inverse", "i", false, "Inverse match modifier (combine with -m)")
f.BoolVarP(&nodeOnly, "nodeonly", "1", false, "Show only the node name") f.BoolVarP(&nodeOnly, "nodeonly", "1", false, "Show only the node name")
f.BoolVarP(&valueOnly, "valueonly", "2", false, "Show only the value") f.BoolVarP(&valueOnly, "valueonly", "2", false, "Show only the value")
f.BoolVarP(&count, "count", "C", false, "Count fact occurrences") f.BoolVarP(&count, "count", "C", false, "Count fact occurrences")
f.BoolVarP(&ansible, "ansible", "A", false, "Output as Ansible inventory") f.BoolVarP(&ansible, "ansible", "A", false, "Output as Ansible inventory")
f.BoolVarP(&jsonMode, "json", "j", false, "Emit valid JSON for all output") f.BoolVarP(&jsonMode, "json", "j", false, "Emit valid JSON for all output")
f.BoolVarP(&allFacts, "all", "a", false, "Show all facts for a node (requires -n)") f.BoolVarP(&allFacts, "all", "a", false, "Show all facts for a node (requires -n)")
f.StringVar(&domain, "domain", cfg.Domain, "Domain appended to short (dotless) -n node names (overrides config and NODE_LOOKUP_DOMAIN)")
rootCmd.PersistentFlags().StringVar(&puppetDBURL, "url", cfg.PuppetDBURL, "PuppetDB facts URL (overrides config and NODE_LOOKUP_URL)") rootCmd.PersistentFlags().StringVar(&puppetDBURL, "url", cfg.PuppetDBURL, "PuppetDB facts URL (overrides config and NODE_LOOKUP_URL)")
configCmd := &cobra.Command{ configCmd := &cobra.Command{
@@ -410,9 +531,11 @@ func main() {
Use: "show", Use: "show",
Short: "Print the active configuration", Short: "Print the active configuration",
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
fmt.Printf("config file : %s\n", configPath()) fmt.Printf("config file : %s\n", configPath())
fmt.Printf("puppetdb_url: %s\n", cfg.PuppetDBURL) fmt.Printf("puppetdb_url : %s\n", cfg.PuppetDBURL)
fmt.Printf("role_fact : %s\n", cfg.RoleFact) fmt.Printf("role_fact : %s\n", cfg.RoleFact)
fmt.Printf("puppetboard_url: %s\n", cfg.PuppetboardURL)
fmt.Printf("domain : %s\n", cfg.Domain)
return nil return nil
}, },
SilenceUsage: true, SilenceUsage: true,
+523 -80
View File
@@ -9,6 +9,8 @@ import (
"path/filepath" "path/filepath"
"strings" "strings"
"testing" "testing"
"gopkg.in/yaml.v3"
) )
// ---- helpers ---------------------------------------------------------------- // ---- helpers ----------------------------------------------------------------
@@ -26,74 +28,285 @@ func rawJSON(v interface{}) json.RawMessage {
return json.RawMessage(b) return json.RawMessage(b)
} }
// captureStdout redirects os.Stdout for the duration of fn and returns whatever
// was written. run() writes directly to os.Stdout, so the output modes are
// exercised end-to-end this way.
func captureStdout(t *testing.T, fn func()) string {
t.Helper()
old := os.Stdout
r, w, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
os.Stdout = w
defer func() { os.Stdout = old }()
fn()
_ = w.Close()
var buf strings.Builder
_, _ = io.Copy(&buf, r)
return buf.String()
}
// ---- buildQuery ------------------------------------------------------------- // ---- buildQuery -------------------------------------------------------------
func TestBuildQuery_NoFilters(t *testing.T) { func TestBuildQuery_NoFilters(t *testing.T) {
q := buildQuery("", "", "", "", "enc_role", false) // With no node/fact/match, buildQuery falls back to a role-fact query.
if !strings.Contains(q, "enc_role") { q := buildQuery("", "", "", "enc_role", false, false, false)
if !strings.Contains(q, "enc_role") || !strings.Contains(q, "name") {
t.Fatalf("expected default role query, got %s", q) t.Fatalf("expected default role query, got %s", q)
} }
} }
func TestBuildQuery_Node(t *testing.T) { func TestBuildQuery_Node(t *testing.T) {
q := buildQuery("host1", "", "", "", "enc_role", false) q := buildQuery("host1", "", "", "enc_role", false, false, false)
if !strings.Contains(q, "certname") || !strings.Contains(q, "host1") { if !strings.Contains(q, "certname") || !strings.Contains(q, "host1") {
t.Fatalf("unexpected query: %s", q) t.Fatalf("unexpected query: %s", q)
} }
} }
func TestBuildQuery_FactAndMatch(t *testing.T) { func TestBuildQuery_FactAndMatch(t *testing.T) {
q := buildQuery("", "region", "syd1", "", "enc_role", false) q := buildQuery("", "region", "syd1", "enc_role", false, false, false)
if !strings.Contains(q, "region") || !strings.Contains(q, "syd1") { if !strings.Contains(q, "region") || !strings.Contains(q, "syd1") {
t.Fatalf("unexpected query: %s", q) t.Fatalf("unexpected query: %s", q)
} }
// exact match uses the "=" operator, never "~".
if strings.Contains(q, `"~"`) {
t.Fatalf("exact match should not use ~ operator: %s", q)
}
} }
func TestBuildQuery_PartialMatch(t *testing.T) { func TestBuildQuery_PartialMatch(t *testing.T) {
q := buildQuery("", "enc_role", "", "dns", "enc_role", false) // -p turns the value match into a regex ("~") comparison.
if !strings.Contains(q, "~") || !strings.Contains(q, "dns") { q := buildQuery("", "enc_role", "dns", "enc_role", false, true, false)
t.Fatalf("expected partial match query, got %s", q) if !strings.Contains(q, `"~"`) || !strings.Contains(q, "dns") {
t.Fatalf("expected partial (~) match query, got %s", q)
}
}
func TestBuildQuery_InverseMatch(t *testing.T) {
// -i wraps the value comparison in a "not".
q := buildQuery("", "enc_role", "dns", "enc_role", false, false, true)
if !strings.Contains(q, `"not"`) {
t.Fatalf("expected inverse (not) match query, got %s", q)
}
}
func TestBuildQuery_InversePartialMatch(t *testing.T) {
// -i and -p compose: a negated regex match.
q := buildQuery("", "enc_role", "dns", "enc_role", false, true, true)
if !strings.Contains(q, `"not"`) || !strings.Contains(q, `"~"`) {
t.Fatalf("expected inverse partial match query, got %s", q)
}
// The regex operator must sit inside the "not" wrapper, not beside it.
notIdx := strings.Index(q, `"not"`)
tildeIdx := strings.Index(q, `"~"`)
if notIdx < 0 || tildeIdx < 0 || tildeIdx < notIdx {
t.Fatalf("expected ~ nested inside not, got %s", q)
} }
} }
func TestBuildQuery_ShowRole(t *testing.T) { func TestBuildQuery_ShowRole(t *testing.T) {
q := buildQuery("", "", "", "", "my_role_fact", true) q := buildQuery("", "", "", "my_role_fact", true, false, false)
if !strings.Contains(q, "my_role_fact") { if !strings.Contains(q, "my_role_fact") {
t.Fatalf("expected role fact in query, got %s", q) t.Fatalf("expected role fact in query, got %s", q)
} }
} }
func TestBuildQuery_CustomRoleFact(t *testing.T) { func TestBuildQuery_CustomRoleFact(t *testing.T) {
q := buildQuery("", "", "", "", "custom_role", true) q := buildQuery("", "", "", "custom_role", true, false, false)
if !strings.Contains(q, "custom_role") { if !strings.Contains(q, "custom_role") {
t.Fatalf("expected custom role fact, got %s", q) t.Fatalf("expected custom role fact, got %s", q)
} }
} }
// ---- valueString ------------------------------------------------------------ func TestBuildQuery_FactBeatsRole(t *testing.T) {
// An explicit -F fact name takes precedence over the role fact.
q := buildQuery("", "region", "", "enc_role", true, false, false)
if !strings.Contains(q, "region") {
t.Fatalf("expected explicit fact name, got %s", q)
}
if strings.Contains(q, "enc_role") {
t.Fatalf("role fact should not appear when -F is set: %s", q)
}
}
func TestBuildQuery_ValidJSON(t *testing.T) {
// Whatever the flag combination, the query must be a valid JSON array.
q := buildQuery("host1", "region", "syd1", "enc_role", false, true, true)
var v []interface{}
if err := json.Unmarshal([]byte(q), &v); err != nil {
t.Fatalf("query is not valid JSON: %v (%s)", err, q)
}
if v[0] != "and" {
t.Fatalf("expected combined query to start with 'and', got %v", v[0])
}
}
// ---- multi-fact -F (comma-separated) ----------------------------------------
func TestSplitFactNames(t *testing.T) {
cases := map[string][]string{
"ipaddress": {"ipaddress"},
"ipaddress,enc_role": {"ipaddress", "enc_role"},
"ipaddress, enc_role ": {"ipaddress", "enc_role"}, // trims spaces
"a,,b,": {"a", "b"}, // drops empties
"": nil,
}
for in, want := range cases {
got := splitFactNames(in)
if len(got) != len(want) {
t.Fatalf("splitFactNames(%q) = %v, want %v", in, got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("splitFactNames(%q) = %v, want %v", in, got, want)
}
}
}
}
func TestQualifyNode(t *testing.T) {
const domain = "main.unkin.net"
cases := []struct {
name string
in string
want string
}{
{"short name appends domain", "ausyd1nxvm2120", "ausyd1nxvm2120.main.unkin.net"},
{"fqdn in default domain unchanged", "ausyd1nxvm2120.main.unkin.net", "ausyd1nxvm2120.main.unkin.net"},
{"multi-label fqdn other domain unchanged", "foo.k8s.syd1.au.unkin.net", "foo.k8s.syd1.au.unkin.net"},
{"short name with trailing dot qualified", "ausyd1nxvm2120.", "ausyd1nxvm2120.main.unkin.net"},
{"fqdn with trailing dot stripped", "foo.main.unkin.net.", "foo.main.unkin.net"},
{"empty unchanged", "", ""},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if got := qualifyNode(tc.in, domain); got != tc.want {
t.Fatalf("qualifyNode(%q, %q) = %q, want %q", tc.in, domain, got, tc.want)
}
})
}
}
func TestQualifyNode_CustomDomain(t *testing.T) {
if got := qualifyNode("host1", "example.com"); got != "host1.example.com" {
t.Fatalf("qualifyNode with custom domain = %q, want host1.example.com", got)
}
}
func TestBuildQuery_SingleFact_NoOr(t *testing.T) {
q := buildQuery("", "ipaddress", "", "enc_role", false, false, false)
if strings.Contains(q, `"or"`) {
t.Fatalf("single fact should not use 'or': %s", q)
}
if !strings.Contains(q, "ipaddress") {
t.Fatalf("expected fact name in query: %s", q)
}
}
func TestBuildQuery_MultiFact_UsesOr(t *testing.T) {
q := buildQuery("host1", "ipaddress,enc_role", "", "enc_role", false, false, false)
if !strings.Contains(q, `"or"`) {
t.Fatalf("expected 'or' over fact names: %s", q)
}
if !strings.Contains(q, "ipaddress") || !strings.Contains(q, "enc_role") {
t.Fatalf("expected both fact names: %s", q)
}
// Must remain valid PQL JSON, combined under "and" with the certname filter.
var v []interface{}
if err := json.Unmarshal([]byte(q), &v); err != nil {
t.Fatalf("query is not valid JSON: %v (%s)", err, q)
}
if v[0] != "and" {
t.Fatalf("expected top-level 'and', got %v", v[0])
}
}
func TestRun_JSON_MultipleFacts(t *testing.T) {
// Two facts returned for one host must both appear, keyed by their real name.
facts := []fact{
{Certname: "hosta", Name: "ipaddress", Value: rawJSON("198.18.0.1")},
{Certname: "hosta", Name: "enc_role", Value: rawJSON("roles::dns")},
}
out := runToString(t, facts, func(a *runArgs) {
a.showRole = false
a.factName = "ipaddress,enc_role"
a.jsonMode = true
})
var parsed map[string]map[string]interface{}
if err := json.Unmarshal([]byte(out), &parsed); err != nil {
t.Fatalf("output is not valid JSON: %v (%s)", err, out)
}
host := parsed["hosta"]
if host["ipaddress"] != "198.18.0.1" || host["enc_role"] != "roles::dns" {
t.Fatalf("expected both facts under host, got: %v", host)
}
}
// ---- matchValue (positional fallback for `-pm value`) -----------------------
func TestMatchValue_FlagWins(t *testing.T) {
// An explicit -m value takes precedence over any positional arg.
if got := matchValue("flagval", []string{"posval"}); got != "flagval" {
t.Fatalf("expected flag value to win, got %q", got)
}
}
func TestMatchValue_PositionalFallback(t *testing.T) {
// This is the `-pm k8s` case: pflag leaves k8s as a positional because the
// grouped -m flag does not attach the space-separated value.
if got := matchValue("", []string{"k8s"}); got != "k8s" {
t.Fatalf("expected positional fallback, got %q", got)
}
}
func TestMatchValue_NoneGiven(t *testing.T) {
if got := matchValue("", nil); got != "" {
t.Fatalf("expected empty, got %q", got)
}
}
// ---- valueString / valueAny -------------------------------------------------
func TestValueString_String(t *testing.T) { func TestValueString_String(t *testing.T) {
raw := rawJSON("hello") if got := valueString(rawJSON("hello")); got != "hello" {
if got := valueString(raw); got != "hello" {
t.Fatalf("expected hello, got %s", got) t.Fatalf("expected hello, got %s", got)
} }
} }
func TestValueString_Number(t *testing.T) { func TestValueString_Number(t *testing.T) {
raw := rawJSON(42) if got := valueString(rawJSON(42)); got != "42" {
if got := valueString(raw); got != "42" {
t.Fatalf("expected 42, got %s", got) t.Fatalf("expected 42, got %s", got)
} }
} }
func TestValueString_Bool(t *testing.T) {
if got := valueString(rawJSON(true)); got != "true" {
t.Fatalf("expected true, got %s", got)
}
}
func TestValueString_Object(t *testing.T) { func TestValueString_Object(t *testing.T) {
raw := json.RawMessage(`{"a":1}`) raw := json.RawMessage(`{"a":1}`)
got := valueString(raw) if got := valueString(raw); got != `{"a":1}` {
if got != `{"a":1}` {
t.Fatalf("expected compact JSON, got %s", got) t.Fatalf("expected compact JSON, got %s", got)
} }
} }
func TestValueAny_TypesPreserved(t *testing.T) {
if v := valueAny(rawJSON("s")); v != "s" {
t.Fatalf("expected string, got %v", v)
}
if v := valueAny(rawJSON(3)); v != float64(3) {
t.Fatalf("expected float64(3), got %T %v", v, v)
}
if v := valueAny(rawJSON(true)); v != true {
t.Fatalf("expected bool true, got %v", v)
}
}
// ---- processResults --------------------------------------------------------- // ---- processResults ---------------------------------------------------------
func TestProcessResults_Sorted(t *testing.T) { func TestProcessResults_Sorted(t *testing.T) {
@@ -108,8 +321,7 @@ func TestProcessResults_Sorted(t *testing.T) {
} }
func TestProcessResults_Empty(t *testing.T) { func TestProcessResults_Empty(t *testing.T) {
out := processResults(nil) if out := processResults(nil); len(out) != 0 {
if len(out) != 0 {
t.Fatalf("expected empty, got %v", out) t.Fatalf("expected empty, got %v", out)
} }
} }
@@ -117,11 +329,7 @@ func TestProcessResults_Empty(t *testing.T) {
// ---- countResults ----------------------------------------------------------- // ---- countResults -----------------------------------------------------------
func TestCountResults_Basic(t *testing.T) { func TestCountResults_Basic(t *testing.T) {
lines := []string{ lines := []string{"host1 syd1", "host2 syd1", "host3 mel1"}
"host1 syd1",
"host2 syd1",
"host3 mel1",
}
out := countResults(lines) out := countResults(lines)
found := map[string]bool{} found := map[string]bool{}
for _, l := range out { for _, l := range out {
@@ -136,9 +344,8 @@ func TestCountResults_Basic(t *testing.T) {
} }
func TestCountResults_Sorted(t *testing.T) { func TestCountResults_Sorted(t *testing.T) {
lines := []string{"h1 z", "h2 a", "h3 a"} // Lexicographic sort of the "N: value" strings: "1: z" < "2: a".
out := countResults(lines) out := countResults([]string{"h1 z", "h2 a", "h3 a"})
// lexicographic sort: "1: z" < "2: a"
if out[0] != "1: z" || out[1] != "2: a" { if out[0] != "1: z" || out[1] != "2: a" {
t.Fatalf("unexpected order: %v", out) t.Fatalf("unexpected order: %v", out)
} }
@@ -147,13 +354,11 @@ func TestCountResults_Sorted(t *testing.T) {
// ---- queryPuppetDB ---------------------------------------------------------- // ---- queryPuppetDB ----------------------------------------------------------
func TestQueryPuppetDB_Success(t *testing.T) { func TestQueryPuppetDB_Success(t *testing.T) {
want := []fact{ want := []fact{{Certname: "node1", Name: "enc_role", Value: rawJSON("roles::dns")}}
{Certname: "node1", Name: "enc_role", Value: rawJSON("roles::dns")},
}
srv := newTestServer(t, want) srv := newTestServer(t, want)
defer srv.Close() defer srv.Close()
got, err := queryPuppetDB(srv.URL+"/pdb/query/v4/facts", `["=","name","enc_role"]`) got, err := queryPuppetDB(srv.URL, `["=","name","enc_role"]`)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -162,14 +367,27 @@ func TestQueryPuppetDB_Success(t *testing.T) {
} }
} }
func TestQueryPuppetDB_SendsQueryParam(t *testing.T) {
var got string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
got = r.URL.Query().Get("query")
_ = json.NewEncoder(w).Encode([]fact{})
}))
defer srv.Close()
_, _ = queryPuppetDB(srv.URL, `["=","name","enc_role"]`)
if got != `["=","name","enc_role"]` {
t.Fatalf("query param not forwarded correctly, got %q", got)
}
}
func TestQueryPuppetDB_HTTPError(t *testing.T) { func TestQueryPuppetDB_HTTPError(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Error(w, "not found", http.StatusNotFound) http.Error(w, "not found", http.StatusNotFound)
})) }))
defer srv.Close() defer srv.Close()
_, err := queryPuppetDB(srv.URL+"/pdb/query/v4/facts", `["=","name","enc_role"]`) if _, err := queryPuppetDB(srv.URL, `[]`); err == nil {
if err == nil {
t.Fatal("expected error for 404") t.Fatal("expected error for 404")
} }
} }
@@ -180,15 +398,13 @@ func TestQueryPuppetDB_BadJSON(t *testing.T) {
})) }))
defer srv.Close() defer srv.Close()
_, err := queryPuppetDB(srv.URL+"/pdb/query/v4/facts", `["=","name","enc_role"]`) if _, err := queryPuppetDB(srv.URL, `[]`); err == nil {
if err == nil {
t.Fatal("expected decode error") t.Fatal("expected decode error")
} }
} }
func TestQueryPuppetDB_ConnectionRefused(t *testing.T) { func TestQueryPuppetDB_ConnectionRefused(t *testing.T) {
_, err := queryPuppetDB("http://127.0.0.1:1/facts", `["=","name","enc_role"]`) if _, err := queryPuppetDB("http://127.0.0.1:1/facts", `[]`); err == nil {
if err == nil {
t.Fatal("expected connection error") t.Fatal("expected connection error")
} }
} }
@@ -210,6 +426,22 @@ func TestLoadConfig_Defaults(t *testing.T) {
if cfg.RoleFact != defaultRoleFact { if cfg.RoleFact != defaultRoleFact {
t.Fatalf("expected default role fact, got %s", cfg.RoleFact) t.Fatalf("expected default role fact, got %s", cfg.RoleFact)
} }
if cfg.Domain != defaultDomain {
t.Fatalf("expected default domain, got %s", cfg.Domain)
}
}
func TestLoadConfig_DomainEnvOverride(t *testing.T) {
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
t.Setenv("NODE_LOOKUP_DOMAIN", "example.com")
cfg, err := loadConfig()
if err != nil {
t.Fatal(err)
}
if cfg.Domain != "example.com" {
t.Fatalf("domain env override failed: %s", cfg.Domain)
}
} }
func TestLoadConfig_EnvOverride(t *testing.T) { func TestLoadConfig_EnvOverride(t *testing.T) {
@@ -292,8 +524,7 @@ func TestLoadConfig_InvalidYAML(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
_, err := loadConfig() if _, err := loadConfig(); err == nil {
if err == nil {
t.Fatal("expected error for invalid YAML") t.Fatal("expected error for invalid YAML")
} }
} }
@@ -305,9 +536,7 @@ func TestWriteDefaultConfig(t *testing.T) {
if err := writeDefaultConfig(); err != nil { if err := writeDefaultConfig(); err != nil {
t.Fatal(err) t.Fatal(err)
} }
data, err := os.ReadFile(filepath.Join(dir, appName, configFileName))
path := filepath.Join(dir, appName, configFileName)
data, err := os.ReadFile(path)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -323,15 +552,14 @@ func TestWriteDefaultConfig_AlreadyExists(t *testing.T) {
if err := writeDefaultConfig(); err != nil { if err := writeDefaultConfig(); err != nil {
t.Fatal(err) t.Fatal(err)
} }
err := writeDefaultConfig() if err := writeDefaultConfig(); err == nil {
if err == nil {
t.Fatal("expected error when config already exists") t.Fatal("expected error when config already exists")
} }
} }
// ---- allFactsForNode -------------------------------------------------------- // ---- allFactsForNode --------------------------------------------------------
func TestAllFactsForNode_ReturnsSortedFacts(t *testing.T) { func TestAllFactsForNode_ReturnsFacts(t *testing.T) {
facts := []fact{ facts := []fact{
{Certname: "node1", Name: "zebra", Value: rawJSON("z-val")}, {Certname: "node1", Name: "zebra", Value: rawJSON("z-val")},
{Certname: "node1", Name: "alpha", Value: rawJSON("a-val")}, {Certname: "node1", Name: "alpha", Value: rawJSON("a-val")},
@@ -340,7 +568,7 @@ func TestAllFactsForNode_ReturnsSortedFacts(t *testing.T) {
srv := newTestServer(t, facts) srv := newTestServer(t, facts)
defer srv.Close() defer srv.Close()
got, err := allFactsForNode(srv.URL+"/pdb/query/v4/facts", "node1") got, err := allFactsForNode(srv.URL, "node1")
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -353,12 +581,11 @@ func TestAllFactsForNode_QueryContainsCertname(t *testing.T) {
var receivedQuery string var receivedQuery string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
receivedQuery = r.URL.Query().Get("query") receivedQuery = r.URL.Query().Get("query")
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode([]fact{}) _ = json.NewEncoder(w).Encode([]fact{})
})) }))
defer srv.Close() defer srv.Close()
_, _ = allFactsForNode(srv.URL+"/pdb/query/v4/facts", "mynode.example.com") _, _ = allFactsForNode(srv.URL, "mynode.example.com")
if !strings.Contains(receivedQuery, "mynode.example.com") { if !strings.Contains(receivedQuery, "mynode.example.com") {
t.Fatalf("expected certname in query, got: %s", receivedQuery) t.Fatalf("expected certname in query, got: %s", receivedQuery)
} }
@@ -373,59 +600,275 @@ func TestAllFactsForNode_HTTPError(t *testing.T) {
})) }))
defer srv.Close() defer srv.Close()
_, err := allFactsForNode(srv.URL+"/pdb/query/v4/facts", "node1") if _, err := allFactsForNode(srv.URL, "node1"); err == nil {
if err == nil {
t.Fatal("expected error for HTTP 500") t.Fatal("expected error for HTTP 500")
} }
} }
// ---- run with -a flag ------------------------------------------------------- // ---- stdinReader (no-TTY behavior) ------------------------------------------
func TestStdinReader_CharDeviceFallsThrough(t *testing.T) {
// /dev/null is a character device — the same shape stdin has when the tool
// is invoked by an agent/CI without a TTY. It must NOT be treated as input.
f, err := os.Open(os.DevNull)
if err != nil {
t.Fatal(err)
}
defer func() { _ = f.Close() }()
if _, ok := stdinReader(f); ok {
t.Fatal("expected /dev/null to be treated as no stdin data")
}
}
func TestStdinReader_EmptyPipeFallsThrough(t *testing.T) {
// A closed, empty pipe (e.g. `true | node-lookup`) must fall through to a
// normal query rather than consuming empty input and printing nothing.
r, w, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
_ = w.Close() // no data written; reader sees immediate EOF
defer func() { _ = r.Close() }()
if _, ok := stdinReader(r); ok {
t.Fatal("expected empty pipe to be treated as no stdin data")
}
}
func TestStdinReader_PipeWithDataIsRead(t *testing.T) {
r, w, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
go func() {
_, _ = io.WriteString(w, "node1\nnode2\n")
_ = w.Close()
}()
defer func() { _ = r.Close() }()
reader, ok := stdinReader(r)
if !ok {
t.Fatal("expected pipe with data to be treated as stdin input")
}
got, _ := reader.ReadString('\n')
if strings.TrimSpace(got) != "node1" {
t.Fatalf("expected first line 'node1', got %q", got)
}
}
func TestStdinReader_RegularFileWithData(t *testing.T) {
path := filepath.Join(t.TempDir(), "nodes.txt")
if err := os.WriteFile(path, []byte("host1\nhost2\n"), 0o644); err != nil {
t.Fatal(err)
}
f, err := os.Open(path)
if err != nil {
t.Fatal(err)
}
defer func() { _ = f.Close() }()
if _, ok := stdinReader(f); !ok {
t.Fatal("expected redirected file with data to be treated as stdin input")
}
}
func TestStdinReader_EmptyFileFallsThrough(t *testing.T) {
path := filepath.Join(t.TempDir(), "empty.txt")
if err := os.WriteFile(path, nil, 0o644); err != nil {
t.Fatal(err)
}
f, err := os.Open(path)
if err != nil {
t.Fatal(err)
}
defer func() { _ = f.Close() }()
if _, ok := stdinReader(f); ok {
t.Fatal("expected empty file to be treated as no stdin data")
}
}
// ---- run: validation --------------------------------------------------------
func TestRun_AllFacts_RequiresNode(t *testing.T) { func TestRun_AllFacts_RequiresNode(t *testing.T) {
cfg := config{PuppetDBURL: "http://unused", RoleFact: "enc_role"} cfg := config{PuppetDBURL: "http://unused", RoleFact: "enc_role"}
err := run(cfg, "", "", "", "", false, false, false, false, false, false, true) err := run(cfg, "", "", "", false, false, false, false, false, false, false, false, true)
if err == nil || !strings.Contains(err.Error(), "-a requires -n") { if err == nil || !strings.Contains(err.Error(), "-a requires -n") {
t.Fatalf("expected -a requires -n error, got: %v", err) t.Fatalf("expected -a requires -n error, got: %v", err)
} }
} }
func TestRun_OutputFlagsRequireRoleOrFact(t *testing.T) {
cfg := config{PuppetDBURL: "http://unused", RoleFact: "enc_role"}
// -1 (nodeOnly) with neither -R nor -F must error before any HTTP call.
err := run(cfg, "", "", "", false, false, false, true, false, false, false, false, false)
if err == nil || !strings.Contains(err.Error(), "-R or -F") {
t.Fatalf("expected -R/-F requirement error, got: %v", err)
}
}
func TestRun_MatchRequiresRoleOrFact(t *testing.T) {
cfg := config{PuppetDBURL: "http://unused", RoleFact: "enc_role"}
// -m with neither -R nor -F must error before any HTTP call.
err := run(cfg, "", "", "someval", false, false, false, false, false, false, false, false, false)
if err == nil || !strings.Contains(err.Error(), "-R or -F") {
t.Fatalf("expected -R/-F requirement error, got: %v", err)
}
}
// ---- run: output modes ------------------------------------------------------
type runArgs struct {
cfg config
nodeName, factName, match string
showRole, partial, inverse, nodeOnly, valueOnly, count bool
ansible, jsonMode, allFacts bool
}
// runToString invokes run against a mock PuppetDB returning facts, capturing
// stdout. nodeName defaults to "n1" so the stdin path is skipped.
func runToString(t *testing.T, facts []fact, mutate func(*runArgs)) string {
t.Helper()
srv := newTestServer(t, facts)
t.Cleanup(srv.Close)
a := runArgs{
cfg: config{PuppetDBURL: srv.URL, RoleFact: "enc_role"},
nodeName: "n1",
showRole: true,
}
if mutate != nil {
mutate(&a)
}
return captureStdout(t, func() {
if err := run(a.cfg, a.nodeName, a.factName, a.match, a.showRole, a.partial, a.inverse, a.nodeOnly, a.valueOnly, a.count, a.ansible, a.jsonMode, a.allFacts); err != nil {
t.Fatalf("run returned error: %v", err)
}
})
}
func TestRun_DefaultOutput(t *testing.T) {
facts := []fact{
{Certname: "hostb", Name: "enc_role", Value: rawJSON("roles::web")},
{Certname: "hosta", Name: "enc_role", Value: rawJSON("roles::db")},
}
out := strings.TrimSpace(runToString(t, facts, nil))
lines := strings.Split(out, "\n")
if len(lines) != 2 || lines[0] != "hosta roles::db" || lines[1] != "hostb roles::web" {
t.Fatalf("unexpected default output: %q", out)
}
}
func TestRun_NodeOnly(t *testing.T) {
facts := []fact{
{Certname: "hosta", Name: "enc_role", Value: rawJSON("roles::db")},
{Certname: "hostb", Name: "enc_role", Value: rawJSON("roles::web")},
}
out := strings.TrimSpace(runToString(t, facts, func(a *runArgs) { a.nodeOnly = true }))
if out != "hosta\nhostb" {
t.Fatalf("unexpected -1 output: %q", out)
}
}
func TestRun_ValueOnly(t *testing.T) {
facts := []fact{
{Certname: "hosta", Name: "enc_role", Value: rawJSON("roles::db")},
{Certname: "hostb", Name: "enc_role", Value: rawJSON("roles::web")},
}
out := strings.TrimSpace(runToString(t, facts, func(a *runArgs) { a.valueOnly = true }))
if out != "roles::db\nroles::web" {
t.Fatalf("unexpected -2 output: %q", out)
}
}
func TestRun_Count(t *testing.T) {
facts := []fact{
{Certname: "hosta", Name: "enc_role", Value: rawJSON("roles::web")},
{Certname: "hostb", Name: "enc_role", Value: rawJSON("roles::web")},
{Certname: "hostc", Name: "enc_role", Value: rawJSON("roles::db")},
}
out := strings.TrimSpace(runToString(t, facts, func(a *runArgs) { a.count = true }))
if !strings.Contains(out, "2: roles::web") || !strings.Contains(out, "1: roles::db") {
t.Fatalf("unexpected -C output: %q", out)
}
}
func TestRun_JSON(t *testing.T) {
facts := []fact{{Certname: "hosta", Name: "enc_role", Value: rawJSON("roles::db")}}
out := runToString(t, facts, func(a *runArgs) { a.jsonMode = true })
var parsed map[string]map[string]interface{}
if err := json.Unmarshal([]byte(out), &parsed); err != nil {
t.Fatalf("output is not valid JSON: %v (%s)", err, out)
}
if parsed["hosta"]["enc_role"] != "roles::db" {
t.Fatalf("unexpected JSON structure: %v", parsed)
}
}
func TestRun_Ansible(t *testing.T) {
facts := []fact{
{Certname: "hosta", Name: "enc_role", Value: rawJSON("roles::db")},
{Certname: "hostb", Name: "enc_role", Value: rawJSON("roles::web")},
}
out := runToString(t, facts, func(a *runArgs) { a.ansible = true })
if !strings.Contains(out, "all:") || !strings.Contains(out, "hosts:") {
t.Fatalf("expected Ansible inventory structure, got: %q", out)
}
if !strings.Contains(out, "hosta:") || !strings.Contains(out, "hostb:") {
t.Fatalf("expected both hosts in inventory, got: %q", out)
}
// The queried fact is attached as a host var.
if !strings.Contains(out, "enc_role: roles::db") || !strings.Contains(out, "enc_role: roles::web") {
t.Fatalf("expected fact host vars in inventory, got: %q", out)
}
}
func TestRun_Ansible_MultipleFacts(t *testing.T) {
// -F ipaddress,enc_role -A must include both facts as host vars.
facts := []fact{
{Certname: "hosta", Name: "ipaddress", Value: rawJSON("198.18.0.1")},
{Certname: "hosta", Name: "enc_role", Value: rawJSON("roles::dns")},
}
out := runToString(t, facts, func(a *runArgs) {
a.showRole = false
a.factName = "ipaddress,enc_role"
a.ansible = true
})
// Parse it back as YAML and assert the structure precisely.
var inv struct {
All struct {
Hosts map[string]map[string]interface{} `yaml:"hosts"`
} `yaml:"all"`
}
if err := yaml.Unmarshal([]byte(out), &inv); err != nil {
t.Fatalf("inventory is not valid YAML: %v (%s)", err, out)
}
host := inv.All.Hosts["hosta"]
if host["ipaddress"] != "198.18.0.1" || host["enc_role"] != "roles::dns" {
t.Fatalf("expected both facts as host vars, got: %v", host)
}
}
func TestRun_AllFacts_PrintsSortedByName(t *testing.T) { func TestRun_AllFacts_PrintsSortedByName(t *testing.T) {
facts := []fact{ facts := []fact{
{Certname: "node1", Name: "zzz_fact", Value: rawJSON("last")}, {Certname: "node1", Name: "zzz_fact", Value: rawJSON("last")},
{Certname: "node1", Name: "aaa_fact", Value: rawJSON("first")}, {Certname: "node1", Name: "aaa_fact", Value: rawJSON("first")},
{Certname: "node1", Name: "mmm_fact", Value: rawJSON(true)}, {Certname: "node1", Name: "mmm_fact", Value: rawJSON(true)},
} }
srv := newTestServer(t, facts) out := strings.TrimSpace(runToString(t, facts, func(a *runArgs) {
defer srv.Close() a.showRole = false
a.allFacts = true
old := os.Stdout }))
r, w, _ := os.Pipe() lines := strings.Split(out, "\n")
os.Stdout = w
cfg := config{PuppetDBURL: srv.URL + "/pdb/query/v4/facts", RoleFact: "enc_role"}
err := run(cfg, "node1", "", "", "", false, false, false, false, false, false, true)
_ = w.Close()
os.Stdout = old
var buf strings.Builder
_, _ = io.Copy(&buf, r)
out := buf.String()
if err != nil {
t.Fatal(err)
}
lines := strings.Split(strings.TrimSpace(out), "\n")
if len(lines) != 3 { if len(lines) != 3 {
t.Fatalf("expected 3 lines, got %d: %q", len(lines), out) t.Fatalf("expected 3 lines, got %d: %q", len(lines), out)
} }
if !strings.HasPrefix(lines[0], "aaa_fact") { if !strings.HasPrefix(lines[0], "aaa_fact") ||
t.Errorf("first line should be aaa_fact, got: %s", lines[0]) !strings.HasPrefix(lines[1], "mmm_fact") ||
} !strings.HasPrefix(lines[2], "zzz_fact") {
if !strings.HasPrefix(lines[1], "mmm_fact") { t.Fatalf("facts not sorted by name: %v", lines)
t.Errorf("second line should be mmm_fact, got: %s", lines[1])
}
if !strings.HasPrefix(lines[2], "zzz_fact") {
t.Errorf("third line should be zzz_fact, got: %s", lines[2])
} }
} }
+82
View File
@@ -0,0 +1,82 @@
---
# nfpm config for building the node-lookup RPM.
# Rendered through envsubst (see scripts/build-rpm.sh) then fed to `nfpm pkg`.
name: ${PACKAGE_NAME}
version: ${PACKAGE_VERSION}
release: ${PACKAGE_RELEASE}
arch: ${PACKAGE_ARCH}
platform: ${PACKAGE_PLATFORM}
section: default
priority: extra
description: "${PACKAGE_DESCRIPTION}"
maintainer: ${PACKAGE_MAINTAINER}
homepage: ${PACKAGE_HOMEPAGE}
license: ${PACKAGE_LICENSE}
disable_globbing: false
replaces:
- node-lookup
provides:
- node-lookup
contents:
# The CLI binaries: node-lookup and its companion tools.
- src: dist/node-lookup
dst: /usr/bin/node-lookup
file_info:
mode: 0755
owner: root
group: root
- src: dist/pburl
dst: /usr/bin/pburl
file_info:
mode: 0755
owner: root
group: root
- src: dist/pblastreport
dst: /usr/bin/pblastreport
file_info:
mode: 0755
owner: root
group: root
# Shell completions (generated by scripts/build-rpm.sh before packaging).
- src: dist/completions/node-lookup.bash
dst: /usr/share/bash-completion/completions/node-lookup
file_info:
mode: 0644
- src: dist/completions/_node-lookup
dst: /usr/share/zsh/site-functions/_node-lookup
file_info:
mode: 0644
- src: dist/completions/node-lookup.fish
dst: /usr/share/fish/vendor_completions.d/node-lookup.fish
file_info:
mode: 0644
- src: dist/completions/pburl.bash
dst: /usr/share/bash-completion/completions/pburl
file_info:
mode: 0644
- src: dist/completions/_pburl
dst: /usr/share/zsh/site-functions/_pburl
file_info:
mode: 0644
- src: dist/completions/pburl.fish
dst: /usr/share/fish/vendor_completions.d/pburl.fish
file_info:
mode: 0644
- src: dist/completions/pblastreport.bash
dst: /usr/share/bash-completion/completions/pblastreport
file_info:
mode: 0644
- src: dist/completions/_pblastreport
dst: /usr/share/zsh/site-functions/_pblastreport
file_info:
mode: 0644
- src: dist/completions/pblastreport.fish
dst: /usr/share/fish/vendor_completions.d/pblastreport.fish
file_info:
mode: 0644
+49
View File
@@ -0,0 +1,49 @@
#!/usr/bin/env bash
#
# Package the (already built) node-lookup, pburl and pblastreport binaries into
# an RPM with nfpm, bundling generated bash/zsh/fish shell completions.
# Usage: scripts/build-rpm.sh [version] (version defaults to $CI_COMMIT_TAG)
#
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "${ROOT_DIR}"
VERSION="${1:-${CI_COMMIT_TAG:-0.0.0-dev}}"
VERSION="${VERSION#v}" # strip a leading v
BINARY="node-lookup"
BINARIES=(node-lookup pburl pblastreport)
DIST="dist"
for b in "${BINARIES[@]}"; do
if [ ! -f "${DIST}/${b}" ]; then
echo "ERROR: ${DIST}/${b} not found; run 'make build' first" >&2
exit 1
fi
done
# Generate shell completions from the freshly built binaries so they always
# match the shipped flags/subcommands.
COMP_DIR="${DIST}/completions"
mkdir -p "${COMP_DIR}"
for b in "${BINARIES[@]}"; do
"./${DIST}/${b}" completion bash >"${COMP_DIR}/${b}.bash"
"./${DIST}/${b}" completion zsh >"${COMP_DIR}/_${b}"
"./${DIST}/${b}" completion fish >"${COMP_DIR}/${b}.fish"
done
export PACKAGE_NAME="${BINARY}"
export PACKAGE_VERSION="${VERSION}"
export PACKAGE_RELEASE="1"
export PACKAGE_ARCH="amd64"
export PACKAGE_PLATFORM="linux"
export PACKAGE_DESCRIPTION="CLI tools for PuppetDB: node-lookup (fact lookup/filtering) plus pburl and pblastreport (Puppetboard URLs and last-report times)"
export PACKAGE_MAINTAINER="Ben Vincent <ben@unkin.net>"
export PACKAGE_HOMEPAGE="https://git.unkin.net/unkin/node-lookup"
export PACKAGE_LICENSE="MIT"
envsubst <packaging/nfpm.yaml >"${DIST}/nfpm.yaml"
nfpm pkg --config "${DIST}/nfpm.yaml" --target "${DIST}" --packager rpm
echo "Built:"
ls -1 "${DIST}"/*.rpm