6 Commits

Author SHA1 Message Date
unkin-agent 35edc9c547 node-lookup: auto-qualify short node names to .main.unkin.net (#19)
ci/woodpecker/tag/release Pipeline was successful
Short `-n` node names silently returned nothing: `node-lookup -R -n ausyd1nxvm2120` found nothing while `-n ausyd1nxvm2120.main.unkin.net` worked, because the PuppetDB `certname` filter needs a FQDN. This auto-qualifies a dotless name before the lookup.

## Changes
- Add `qualifyNode()` pure helper: a dotless name gets `.<domain>` appended; a name already containing a dot (any domain, incl. `*.k8s.syd1.au.unkin.net`) is left unchanged; a single trailing dot is stripped first; empty input is preserved (same "no node" behavior as today).
- Apply normalization to the `-n` value and to stdin-sourced node names in `run()`, so both entry points behave consistently.
- Make the domain configurable: config key `domain`, `NODE_LOOKUP_DOMAIN` env var, and `--domain` flag, all defaulting to `main.unkin.net`.
- Surface `domain` in `config show` / `config init` output and document the new env var/flag/behavior in AGENTS.md.
- Add table-driven `qualifyNode` tests (short name appended, FQDN unchanged, multi-label other-domain FQDN unchanged, trailing-dot handling, empty input, custom domain) and a `NODE_LOOKUP_DOMAIN` env-override test.

Companion tools `pburl`/`pblastreport` take already-qualified hostnames (typically piped from `node-lookup`) via `puppet.ReadHosts` and do not share the `-n` code path, so they are intentionally left out to keep this PR atomic.

## Validation
- `gofmt -l .` clean, `go vet ./...` clean
- `go test -race ./...` pass
- `make build` builds all three binaries

Reviewed-on: #19
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-08-15 14:50:35 +10:00
unkinben 182bd326b8 Fix release changelog range and attach RPM + checksums (#18)
ci/woodpecker/tag/release Pipeline was successful
## Why

The `v0.5.4` release step failed with `Error: open node-lookup-linux-amd64: no such file or directory`, so the Gitea releases page carries **none** of the binaries. Root cause: at v0.5.4 the build loop's shell variables (`${name}`, `${pkg}`, `${osarch%/*}`) were unescaped, so Woodpecker substituted them to empty at YAML-parse time and no cross-compiled binaries were produced. The same blanking left the release `--note` empty (`git log "..v0.5.4"`).

The build-loop escaping was already fixed in #16. This PR fixes the two remaining release-step defects and enriches the release assets.

## Changes

- Replace the `git describe --tags --abbrev=0 HEAD^` changelog anchor with a previous-tag scan that skips tags on the current commit and picks the newest semver **ancestor** tag. Several tags point at the same commit (v0.5.3 and v0.5.4 both on `f296056`), so `describe HEAD^` jumps the range back to v0.5.1; the scan correctly selects v0.5.2.
- Attach the packaged RPM (from `dist/`) and a generated `sha256sums.txt` alongside the 12 cross-compiled binaries.
- Build the asset list once and checksum exactly what is uploaded.

Keeps `serviceAccountName: default` and the k8s resource requests/limits on the release step unchanged.

## Validation

- `check-yaml` / `trailing-whitespace` pre-commit hooks pass.
- Local dry-run of the exact release-step shell logic (with `$$`→`$`) from the worktree: `PREV_TAG=v0.5.2`, non-empty notes, and all 13 assets (12 binaries + RPM) plus `sha256sums.txt` resolve on disk.

---------

Co-authored-by: Ben Vincent <neotheo@gmail.com>
Reviewed-on: #18
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-25 14:58:24 +10:00
unkinben 2aaffd7e31 Fix release pipeline: escape shell vars so cross-compiled assets build (#16)
## Why

The **v0.5.3** release pipeline failed at the asset-upload step with `open node-lookup-linux-amd64: no such file or directory`, and release notes came out empty. Woodpecker substitutes `${...}` expressions in `commands` **at parse time**, treating them as pipeline variables. The build loop's shell parameter expansions (`${name}`, `${osarch%/*}`, `${pkg}`) and the release step's `${PREV_TAG}`/`${NOTES}` were blanked before the shell ran, so the per-os/arch binaries were written to garbled names and never matched the upload list.

The RPM itself was unaffected (it's built via `make build`, internal to the Makefile) and `node-lookup-0.5.3-1.x86_64.rpm` published to `rpm-internal` correctly — only the Gitea release binary assets and notes were broken.

## Changes

- Escape all shell variables and command substitutions in the build loop and the release-notes block as `$$`, matching the existing `upload-rpm` step's convention. Genuine Woodpecker vars (`${CI_COMMIT_TAG}`, `${CI_REPO}`) stay single-`$`.

## Follow-up

The v0.5.3 Gitea release assets are being backfilled manually; this fix ensures v0.5.4+ produce them automatically.

Reviewed-on: #16
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-16 23:08:09 +10:00
unkinben f296056360 Add pburl and pblastreport companion tools to the RPM (#15)
ci/woodpecker/tag/release Pipeline failed
## Why

`node-lookup` output is handy for pivoting to Puppetboard, but there was no quick way to turn a list of hosts into Puppetboard node-page URLs, or to see when each host last ran Puppet. These two small tools close that gap and ship in the **same RPM** so they're available wherever `node-lookup` is.

## Changes

- Add **`pburl`**: reads hostnames from args or piped `node-lookup` output (first field of each line, de-duped) and prints `<host> <puppetboard-node-page-url>`.
- Add **`pblastreport`**: prints `<host>\t<last-report-time>\t<url>` using `report_timestamp` from the PuppetDB v4 `nodes` endpoint. Supports `--relative`/`-r` (relative age) and `--timezone`/`-z <IANA>` (default: local timezone).
- Add **`internal/puppet`** package shared by both tools: config load, PuppetDB `nodes` query, Puppetboard URL construction (`<base>/node/<certname>`), and no-TTY-safe stdin host reading.
- Add **`puppetboard_url`** config key (env `NODE_LOOKUP_PUPPETBOARD_URL`, default `https://puppetboard.k8s.syd1.au.unkin.net`) to the shared config so `config init`/`config show` scaffold it for the whole tool family. `node-lookup`'s own query behaviour is unchanged.
- Build all three binaries individually (each is its own `main` package — a single `go build ./...` can't emit multiple mains) and generate per-binary bash/zsh/fish completions in the Makefile, `build-rpm.sh`, and nfpm spec.
- Cross-compile and attach all three tools per os/arch in the release pipeline; extend `.gitignore`; `go mod tidy` promotes cobra/yaml to direct deps.
- Document the tools, config key, and env var in `AGENTS.md`.

## Testing

- `go test -race ./...` passes (new tests cover config precedence, `nodes` endpoint derivation, host-page URLs, `LookupNode`, stdin host parsing, and the report-time formatting incl. timezone/relative/edge cases).
- Built the RPM locally and confirmed it installs all 3 binaries + 9 completion files.
- Smoke-tested both tools end-to-end against a mock PuppetDB (timezone conversion, relative time, and error handling all correct).

No cross-repo changes needed: the release reuses the existing `default` ServiceAccount and the artifactapi `rpm-internal` upload.

Reviewed-on: #15
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-16 22:37:26 +10:00
unkinben 5982d257d5 Fix -pm <value> parsing and add comma-separated multi-fact -F (#14)
Two related query-ergonomics fixes surfaced while using the tool interactively.

## 1. `-pm <value>` failed with `unknown command "k8s"`

pflag does not attach a space-separated value to a string flag (`-m`) grouped with a bool flag (`-p`), so `k8s` was left as a stray positional. Only `-pm=k8s` or the un-grouped `-p -m k8s` worked.

- Allow one positional argument (`cobra.MaximumNArgs(1)`) and fall back to it for the match value when `-m` is empty (`matchValue()`). `-pm/-im/-ipm <value>` and a bare `-p <value>` now all work; `-m` still wins when both are given.

## 2. `-F a,b` (multiple facts) returned `{}`

`-jF ipaddress,enc_role` queried a single fact literally named `ipaddress,enc_role`.

- Split `-F` on commas (`splitFactNames`) and match any of them via an `or` over `["=","name",<n>]` clauses (`nameFilter`); a single name keeps the plain `=` form.
- Key JSON output by each result's real fact name so all requested facts appear under the host.

Both paths have unit tests; verified live: `node-lookup -R -pm externaldns | node-lookup -jF ipaddress,enc_role` returns both facts per host. AGENTS.md updated.

Reviewed-on: #14
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-05 20:05:45 +10:00
unkinben e070357d3f Relocate packaging: RPM, shell completions, no-TTY fix, repaired tests (#13)
ci/woodpecker/tag/release Pipeline was successful
## Why

The unit tests stopped compiling after the `--pm` → `-p`/`-i` match-modifier refactor was left uncommitted, there was no RPM/completions distribution story, and invoking the tool without a TTY against an empty pipe silently returned nothing. This makes the project releasable and safe to run from agents/CI.

## Changes

- Make stdin handling robust: replace the fragile `!isTerminal` check with `stdinReader()`, which only reads node names when stdin is a real pipe/redirect carrying data. Terminals, `/dev/null`, and empty/closed pipes now fall through to a normal query, so running without a TTY behaves like an interactive run.
- Repair and expand `main_test.go` to match the current `buildQuery`/`run` signatures; add coverage for the match modifiers, all output modes, config precedence, and the new `stdinReader` logic. `httptest` stubs PuppetDB (no live deps).
- Add nfpm packaging (`packaging/nfpm.yaml`, `scripts/build-rpm.sh`): installs the binary to `/usr/bin/node-lookup` and bundles generated bash/zsh/fish completions under the standard system paths.
- Rework the Makefile to build into `dist/` and add `completions`/`rpm` targets.
- Split PR CI into `build`, `test`, and `pre-commit` workflows and extend `release` to build the RPM and `PUT` it to the artifactapi `rpm-internal` repo. Every step sets a `serviceAccount` and k8s resources.

The project directory has also been relocated under `prodenv`.

Reviewed-on: #13
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-05 00:02:34 +10:00
24 changed files with 1889 additions and 213 deletions
+8 -1
View File
@@ -1,2 +1,9 @@
node-lookup # built binaries (repo root only — not the cmd/ source dirs)
/node-lookup
/pburl
/pblastreport
# cross-compiled release artifacts (e.g. node-lookup-linux-amd64)
/node-lookup-*
/pburl-*
/pblastreport-*
dist/ dist/
+18
View File
@@ -0,0 +1,18 @@
when:
- event: pull_request
steps:
- name: build
image: golang:1.25
commands:
- make build
backend_options:
kubernetes:
serviceAccountName: default
resources:
requests:
memory: 512Mi
cpu: 1
limits:
memory: 2Gi
cpu: 2
-8
View File
@@ -1,8 +0,0 @@
when:
- event: pull_request
steps:
- name: lint
image: golangci/golangci-lint:latest
commands:
- golangci-lint run ./...
+11 -1
View File
@@ -3,6 +3,16 @@ when:
steps: steps:
- name: pre-commit - name: pre-commit
image: git.unkin.net/unkin/almalinux9-gobuilder:20260325 image: git.unkin.net/unkin/almalinux9-gobuilder:20260606
commands: commands:
- uvx pre-commit run --all-files - uvx pre-commit run --all-files
backend_options:
kubernetes:
serviceAccountName: default
resources:
requests:
memory: 512Mi
cpu: 1
limits:
memory: 2Gi
cpu: 2
+127 -32
View File
@@ -3,36 +3,107 @@ when:
steps: steps:
- name: test - name: test
image: golang:latest image: golang:1.25
commands: commands:
- go test ./... - go test -race ./...
backend_options:
kubernetes:
serviceAccountName: default
resources:
requests:
memory: 512Mi
cpu: 1
limits:
memory: 2Gi
cpu: 2
- name: build-linux-amd64 # Build all binaries into dist/ (consumed by the RPM step) plus the
image: golang:latest # cross-platform binaries attached to the Gitea release. Each tool is a
# separate main package, so they are built individually per os/arch.
- name: build
image: git.unkin.net/unkin/almalinux9-gobuilder:20260606
commands: commands:
- GOOS=linux GOARCH=amd64 go build -ldflags="-s -w -X main.version=${CI_COMMIT_TAG}" -o node-lookup-linux-amd64 ./... - make build VERSION=${CI_COMMIT_TAG}
# Shell variables/expansions are escaped as $$ so Woodpecker leaves them
# for the shell instead of substituting them (as pipeline vars) at parse
# time. ${CI_COMMIT_TAG} is a real Woodpecker var and stays single-$.
- |
for entry in "node-lookup:." "pburl:./cmd/pburl" "pblastreport:./cmd/pblastreport"; do
name="$${entry%%:*}"; pkg="$${entry##*:}"
for osarch in linux/amd64 linux/arm64 darwin/amd64 darwin/arm64; do
os="$${osarch%/*}"; arch="$${osarch#*/}"
GOOS="$$os" GOARCH="$$arch" \
go build -ldflags="-s -w -X main.version=${CI_COMMIT_TAG}" \
-o "$${name}-$${os}-$${arch}" "$$pkg"
done
done
depends_on: [test] depends_on: [test]
backend_options:
kubernetes:
serviceAccountName: default
resources:
requests:
memory: 512Mi
cpu: 1
limits:
memory: 2Gi
cpu: 2
- name: build-linux-arm64 # Package the built binary + generated shell completions into an RPM.
image: golang:latest - name: package
image: git.unkin.net/unkin/almalinux9-rpmbuilder:latest
commands: commands:
- GOOS=linux GOARCH=arm64 go build -ldflags="-s -w -X main.version=${CI_COMMIT_TAG}" -o node-lookup-linux-arm64 ./... - ./scripts/build-rpm.sh ${CI_COMMIT_TAG}
depends_on: [test] depends_on: [build]
backend_options:
kubernetes:
serviceAccountName: default
resources:
requests:
memory: 512Mi
cpu: 1
limits:
memory: 2Gi
cpu: 2
- name: build-darwin-amd64 # Publish the RPM to the artifactapi local rpm repo (a real yum repo;
image: golang:latest # repodata regenerates automatically).
- name: upload-rpm
image: git.unkin.net/unkin/almalinux9-base:20260606
commands: commands:
- GOOS=darwin GOARCH=amd64 go build -ldflags="-s -w -X main.version=${CI_COMMIT_TAG}" -o node-lookup-darwin-amd64 ./... - |
depends_on: [test] HOST="https://artifactapi.k8s.syd1.au.unkin.net"
REPO="rpm-internal"
- name: build-darwin-arm64 for rpm in dist/*.rpm; do
image: golang:latest FILE=$$(basename "$$rpm")
commands: # artifactapi has no HEAD route (returns 405); probe with GET against
- GOOS=darwin GOARCH=arm64 go build -ldflags="-s -w -X main.version=${CI_COMMIT_TAG}" -o node-lookup-darwin-arm64 ./... # the served path (RPMs are stored under Packages/) to avoid re-upload.
depends_on: [test] code=$$(curl -s -o /dev/null -w '%{http_code}' "$$HOST/api/v2/remotes/$$REPO/files/Packages/$$FILE" || true)
if [ "$$code" = "200" ]; then
echo "$$FILE already exists in $$REPO (HTTP $$code); skipping upload"
continue
fi
echo "Uploading $$FILE to $$REPO (existence probe returned $$code)"
curl -f -X PUT \
"$$HOST/api/v2/remotes/$$REPO/files/$$FILE" \
-H "Content-Type: application/x-rpm" \
--data-binary @"$$rpm"
done
depends_on: [package]
backend_options:
kubernetes:
serviceAccountName: default
resources:
requests:
memory: 128Mi
cpu: 100m
limits:
memory: 512Mi
cpu: 500m
# Cut a Gitea release with the cross-platform binaries attached.
- name: release - name: release
image: git.unkin.net/unkin/almalinux9-base:20260325 image: git.unkin.net/unkin/almalinux9-base:20260606
environment: environment:
RELEASER_TOKEN: RELEASER_TOKEN:
from_secret: RELEASER_TOKEN from_secret: RELEASER_TOKEN
@@ -40,20 +111,44 @@ steps:
- | - |
curl --output /usr/local/bin/tea https://artifactapi.k8s.syd1.au.unkin.net/api/v1/remote/gitea-dl/tea/0.12.0/tea-0.12.0-linux-amd64 && chmod +x /usr/local/bin/tea curl --output /usr/local/bin/tea https://artifactapi.k8s.syd1.au.unkin.net/api/v1/remote/gitea-dl/tea/0.12.0/tea-0.12.0-linux-amd64 && chmod +x /usr/local/bin/tea
tea logins add --name gitea --url https://git.unkin.net --token "$${RELEASER_TOKEN}" --no-version-check tea logins add --name gitea --url https://git.unkin.net --token "$${RELEASER_TOKEN}" --no-version-check
PREV_TAG=$(git describe --tags --abbrev=0 HEAD^ 2>/dev/null || echo "") # $$ escapes shell vars/substitutions so Woodpecker doesn't blank them
if [ -n "$PREV_TAG" ]; then # at parse time; ${CI_COMMIT_TAG}/${CI_REPO} are real Woodpecker vars.
NOTES=$(git log "${PREV_TAG}..${CI_COMMIT_TAG}" --pretty=format:"- %s") # Find the previous release tag for the changelog range. Several tags can
else # point at the same commit (e.g. v0.5.3 and v0.5.4), so we skip tags on
NOTES=$(git log --pretty=format:"- %s") # the current commit and pick the newest semver tag that is a real
# ancestor of this one -- describe HEAD^ would jump too far back.
CUR_SHA=$$(git rev-list -n1 "${CI_COMMIT_TAG}")
PREV_TAG=""
for t in $$(git tag --sort=-v:refname); do
[ "$$t" = "${CI_COMMIT_TAG}" ] && continue
[ "$$(git rev-list -n1 "$$t")" = "$$CUR_SHA" ] && continue
if git merge-base --is-ancestor "$$t" "${CI_COMMIT_TAG}" 2>/dev/null; then
PREV_TAG="$$t"; break
fi fi
tea releases create --tag "${CI_COMMIT_TAG}" --title "${CI_COMMIT_TAG}" --note "${NOTES}" --login gitea --repo "${CI_REPO}" done
tea releases assets create "${CI_COMMIT_TAG}" \ if [ -n "$$PREV_TAG" ]; then
node-lookup-linux-amd64 \ NOTES=$$(git log "$${PREV_TAG}..${CI_COMMIT_TAG}" --pretty=format:"- %s")
node-lookup-linux-arm64 \ else
node-lookup-darwin-amd64 \ NOTES=$$(git log --pretty=format:"- %s")
node-lookup-darwin-arm64 \ fi
tea releases create --tag "${CI_COMMIT_TAG}" --title "${CI_COMMIT_TAG}" --note "$${NOTES}" --login gitea --repo "${CI_REPO}"
# The build step writes the 12 cross-compiled binaries into the workspace
# root; the package step writes the RPM to dist/. Generate a checksums
# manifest over everything we attach so downloads can be verified.
RPM=$$(ls dist/*.rpm 2>/dev/null | head -1)
ASSETS="node-lookup-linux-amd64 node-lookup-linux-arm64 node-lookup-darwin-amd64 node-lookup-darwin-arm64 pburl-linux-amd64 pburl-linux-arm64 pburl-darwin-amd64 pburl-darwin-arm64 pblastreport-linux-amd64 pblastreport-linux-arm64 pblastreport-darwin-amd64 pblastreport-darwin-arm64"
[ -n "$$RPM" ] && ASSETS="$$ASSETS $$RPM"
sha256sum $$ASSETS > sha256sums.txt
tea releases assets create "${CI_COMMIT_TAG}" $$ASSETS sha256sums.txt \
--login gitea --repo "${CI_REPO}" --login gitea --repo "${CI_REPO}"
depends_on: [upload-rpm]
backend_options: backend_options:
kubernetes: kubernetes:
serviceAccountName: default serviceAccountName: default
depends_on: [build-linux-amd64, build-linux-arm64, build-darwin-amd64, build-darwin-arm64] resources:
requests:
memory: 128Mi
cpu: 100m
limits:
memory: 512Mi
cpu: 500m
+33
View File
@@ -0,0 +1,33 @@
when:
- event: pull_request
steps:
- name: lint
image: golangci/golangci-lint:latest
commands:
- golangci-lint run ./...
backend_options:
kubernetes:
serviceAccountName: default
resources:
requests:
memory: 512Mi
cpu: 1
limits:
memory: 2Gi
cpu: 2
- name: test
image: golang:1.25
commands:
- go test -v -race ./...
backend_options:
kubernetes:
serviceAccountName: default
resources:
requests:
memory: 512Mi
cpu: 1
limits:
memory: 2Gi
cpu: 2
-8
View File
@@ -1,8 +0,0 @@
when:
- event: pull_request
steps:
- name: unit-tests
image: golang:latest
commands:
- go test -v -race ./...
+95 -12
View File
@@ -2,25 +2,73 @@
## Project Overview ## Project Overview
`node-lookup` is a Go CLI tool that queries a PuppetDB API to retrieve and filter node facts. This repo ships three related Puppet CLIs in one RPM:
- **`node-lookup`** — queries the PuppetDB API to retrieve and filter node facts.
- **`pburl`** — prints the Puppetboard node-page URL for each host (reads hosts
from args or piped `node-lookup` output). Output: `<host> <url>`.
- **`pblastreport`** — prints each host's last Puppet report time and its
Puppetboard URL. Output: `<host>\t<time>\t<url>`. Supports `--relative`/`-r`
(relative age) and `--timezone`/`-z <IANA>` (default: local timezone).
`node-lookup` is the module root; `pburl` and `pblastreport` live under `cmd/`
and share the `internal/puppet` package (config, PuppetDB `nodes` queries,
Puppetboard URL construction, stdin host reading).
## Structure ## Structure
``` ```
main.go # entire application source main.go # node-lookup CLI source (module root, package main)
main_test.go # node-lookup unit tests (mock PuppetDB via httptest)
cmd/pburl/main.go # pburl CLI
cmd/pblastreport/main.go # pblastreport CLI (report.go: report-time formatting)
internal/puppet/ # shared: config, puppetdb nodes query, board URLs, stdin
go.mod # Go module (module name: node-lookup) go.mod # Go module (module name: node-lookup)
go.sum # dependency checksums go.sum # dependency checksums
node-lookup # compiled binary (not committed) Makefile # build / test / lint / completions / rpm / version-bump targets
packaging/nfpm.yaml # nfpm spec (envsubst-templated) for the RPM (all 3 binaries)
scripts/build-rpm.sh # generates completions + packages the RPM with nfpm
.woodpecker/ # CI: build, test, pre-commit (PR) + release (tag)
dist/ # build output: binaries, completions, RPM (not committed)
``` ```
Every binary is a separate `main` package, so `make build` builds each with its
own `-o` (a single `go build ./...` can't emit multiple mains to one file).
## Build ## Build
```bash ```bash
make build # -> dist/node-lookup (CGO disabled, static)
# or directly:
go build -o node-lookup ./... go build -o node-lookup ./...
``` ```
Requires Go 1.21+. Dependencies: `github.com/spf13/cobra` (CLI), `gopkg.in/yaml.v3` (Ansible output). Requires Go 1.21+. Dependencies: `github.com/spf13/cobra` (CLI), `gopkg.in/yaml.v3` (Ansible output).
## Packaging (RPM)
```bash
make rpm # build the binary + package it into dist/*.rpm via nfpm
```
`scripts/build-rpm.sh` generates bash/zsh/fish completions from the built binary
and bundles them alongside `/usr/bin/node-lookup`. On a `v*` tag the release
pipeline builds the RPM and `PUT`s it to the artifactapi `rpm-internal` repo.
## Shell completions
Cobra provides a `completion` subcommand:
```bash
node-lookup completion bash # or zsh / fish / powershell
```
The RPM installs completions to the standard system paths
(`/usr/share/bash-completion/completions/`, `/usr/share/zsh/site-functions/`,
`/usr/share/fish/vendor_completions.d/`), so they work automatically once
installed. To load ad-hoc in the current shell, e.g. zsh:
`source <(node-lookup completion zsh)`.
## Running the Tool ## Running the Tool
```bash ```bash
@@ -28,17 +76,37 @@ Requires Go 1.21+. Dependencies: `github.com/spf13/cobra` (CLI), `gopkg.in/yaml.
./node-lookup -R # show all nodes with role fact ./node-lookup -R # show all nodes with role fact
./node-lookup -n <hostname> # lookup a specific node ./node-lookup -n <hostname> # lookup a specific node
./node-lookup -F <fact_name> # filter by fact name ./node-lookup -F <fact_name> # filter by fact name
./node-lookup -m <value> # exact value match ./node-lookup -jF ipaddress,enc_role # several facts at once (comma-separated)
./node-lookup --pm <pattern> # partial/regex match on value ./node-lookup -R -m <value> # exact value match (-m)
./node-lookup -R -pm <value> # partial/regex match (-p -m combined)
./node-lookup -R -im <value> # inverse exact match (-i -m combined)
./node-lookup -R -ipm <value> # inverse partial match (-i -p -m combined)
./node-lookup -R -p <value> # value may also be given positionally
./node-lookup -R -1 # node names only ./node-lookup -R -1 # node names only
./node-lookup -R -2 # values only ./node-lookup -R -2 # values only
./node-lookup -R -C # count occurrences ./node-lookup -R -C # count occurrences
./node-lookup -R -A # output as Ansible YAML inventory ./node-lookup -R -A # output as Ansible YAML inventory (queried facts become host vars)
./node-lookup -j # output as JSON { host → { fact → value } } ./node-lookup -j # output as JSON { host → { fact → value } }
./node-lookup --url http://host:8080/... # override PuppetDB URL for this invocation ./node-lookup --url http://host:8080/... # override PuppetDB URL for this invocation
echo -e "node1\nnode2" | ./node-lookup -R # pipe node names via stdin echo -e "node1\nnode2" | ./node-lookup -R # pipe node names via stdin
``` ```
### Companion tools
```bash
node-lookup -R | pburl # <host> <puppetboard-url> per line
pburl host1 host2 # hosts as args instead of stdin
node-lookup -R | pblastreport # <host> <last-report-time> <url>
pblastreport -r host1 # relative age (e.g. "3h ago")
pblastreport -z Asia/Singapore host1 # render the time in a specific IANA tz
```
Both read hostnames from arguments or the first field of each piped line (so
any `node-lookup` output mode works), de-duplicate, and share `node-lookup`'s
config file / env vars. `pblastreport` reads `report_timestamp` from the
PuppetDB v4 `nodes` endpoint (derived from the configured facts URL).
## Configuration ## Configuration
Precedence (lowest → highest): **defaults < config file < env vars < `--url` flag** Precedence (lowest → highest): **defaults < config file < env vars < `--url` flag**
@@ -50,6 +118,7 @@ XDG location: `$XDG_CONFIG_HOME/node-lookup/config.yaml` (default: `~/.config/no
```yaml ```yaml
puppetdb_url: http://puppetdbapi.service.consul:8080/pdb/query/v4/facts puppetdb_url: http://puppetdbapi.service.consul:8080/pdb/query/v4/facts
role_fact: enc_role role_fact: enc_role
puppetboard_url: https://puppetboard.k8s.syd1.au.unkin.net # used by pburl / pblastreport
``` ```
Generate the default config file: Generate the default config file:
@@ -68,19 +137,25 @@ Show the active configuration (after all overrides applied):
|---|---|---| |---|---|---|
| `NODE_LOOKUP_URL` | `puppetdb_url` | PuppetDB facts endpoint | | `NODE_LOOKUP_URL` | `puppetdb_url` | PuppetDB facts endpoint |
| `NODE_LOOKUP_ROLE_FACT` | `role_fact` | Fact name used by `-R` flag | | `NODE_LOOKUP_ROLE_FACT` | `role_fact` | Fact name used by `-R` flag |
| `NODE_LOOKUP_PUPPETBOARD_URL` | `puppetboard_url` | Puppetboard base URL (pburl / pblastreport) |
| `NODE_LOOKUP_DOMAIN` | `domain` | Domain appended to short (dotless) `-n` node names (default `main.unkin.net`) |
### CLI flag ### CLI flags
`--url <url>` overrides the PuppetDB URL for a single invocation (highest precedence). `--url <url>` overrides the PuppetDB URL for a single invocation (highest precedence).
`--domain <domain>` overrides the auto-qualify domain for a single invocation.
## Code Patterns ## Code Patterns
- **`loadConfig()`**: reads config file → applies env vars → returns `config` struct. Called once at startup in `main()`. - **`loadConfig()`**: reads config file → applies env vars → returns `config` struct. Called once at startup in `main()`.
- **`buildQuery()`**: returns a PuppetDB PQL-compatible JSON array string. Uses `roleFact` from config (not hardcoded). - **`buildQuery()`**: returns a PuppetDB PQL-compatible JSON array string. Uses `roleFact` from config (not hardcoded). Match modifiers: `-p` (partial/regex, uses `~` op), `-i` (inverse, wraps with `not`), composable.
- **Multiple facts**: `-F` accepts a comma-separated list (`ipaddress,enc_role`). `splitFactNames()`/`nameFilter()` turn several names into an `or` over `["=","name",<n>]` clauses; JSON output keys each value by the fact's real name so all requested facts appear per host.
- **Match value / `matchValue()`**: the value to match comes from `-m/--match` or, if that is empty, an optional positional argument. The positional fallback exists because pflag does not attach a space-separated value to a string flag grouped with a bool flag, so in `-pm k8s` the `k8s` arrives as a positional. `-m` still wins when both are given.
- **`queryPuppetDB(url, query)`**: takes the URL as a parameter — never reads globals. - **`queryPuppetDB(url, query)`**: takes the URL as a parameter — never reads globals.
- **`processResults()`**: iterates facts, returns sorted `"certname value"` strings. JSON string values are unquoted; other JSON types rendered as compact JSON. - **`processResults()`**: iterates facts, returns sorted `"certname value"` strings. JSON string values are unquoted; other JSON types rendered as compact JSON.
- **Output modes**: JSON (`-j`), count (`-C`), Ansible YAML (`-A`), node-only (`-1`), value-only (`-2`), default (node + value). - **Output modes**: JSON (`-j`), count (`-C`), Ansible YAML (`-A`), node-only (`-1`), value-only (`-2`), default (node + value). `-j` and `-A` share `factsByHost()`, so both attach the queried fact(s) per host — as an object under the host (`-j`) or as inventory host vars (`-A`).
- **Stdin support**: when stdin is not a TTY and no `-n` is given, node names are read line-by-line and queried individually (one HTTP request per node). - **Short node names / `qualifyNode()`**: a `-n` value (and stdin-sourced node names) with no dot is auto-qualified to `<name>.<domain>` (domain defaults to `main.unkin.net`, overridable via `--domain`/`NODE_LOOKUP_DOMAIN`), so `-n ausyd1nxvm2120` resolves the same as its FQDN. A name that already contains a dot (any domain) is left unchanged; a single trailing dot is stripped; empty input is preserved.
- **Stdin support**: `stdinReader()` reads node names from stdin only when it is a real pipe/redirect carrying data (and no `-n` given). Terminals, `/dev/null`, and empty/closed pipes fall through to a normal query — so running without a TTY (e.g. invoked by an agent or CI) behaves like an interactive run instead of consuming empty input.
- **SIGPIPE handling**: `signal.Ignore(syscall.SIGPIPE)` so pipes to `head` etc. work cleanly. - **SIGPIPE handling**: `signal.Ignore(syscall.SIGPIPE)` so pipes to `head` etc. work cleanly.
## CLI Framework ## CLI Framework
@@ -89,11 +164,19 @@ Uses [Cobra](https://github.com/spf13/cobra). Root command is the query command.
## Testing ## Testing
No test suite exists. Manual testing requires access to the Consul/PuppetDB environment or a mock HTTP server. ```bash
make test # go test -v -race ./...
```
`main_test.go` covers query construction (all `-m`/`-p`/`-i` combinations), value
rendering, result processing/counting, config precedence (defaults < file < env),
`writeDefaultConfig`, the `stdinReader` no-TTY behavior, and every `run()` output
mode (default, `-1`, `-2`, `-C`, `-j`, `-A`, `-a`). PuppetDB is stubbed with
`httptest` — no live Consul/PuppetDB access is required.
## Gotchas ## Gotchas
- `-1`, `-2`, `-C`, and `-A` all require `-R` or `-F`; the tool exits with an error otherwise. - `-1`, `-2`, `-C`, and `-A` all require `-R` or `-F`; the tool exits with an error otherwise.
- `-C` (count) with stdin reads all lines as pre-fetched `"node value"` output for counting — it does **not** query PuppetDB per line. - `-C` (count) with stdin reads all lines as pre-fetched `"node value"` output for counting — it does **not** query PuppetDB per line.
- JSON output (`-j`) builds `{ hostname: { factname: value } }` where the fact key is the `-F` value, the `role_fact` config value (if `-R`), or `"value"` as fallback. - JSON output (`-j`) builds `{ hostname: { factname: value } }` keyed by each result's actual fact name (so `-F ipaddress,enc_role` yields both per host); it falls back to the `-F` value, the `role_fact` config value (if `-R`), or `"value"` only when a result carries no name.
- `config init` fails if the config file already exists (will not overwrite). - `config init` fails if the config file already exists (will not overwrite).
+38 -3
View File
@@ -1,13 +1,29 @@
BINARY := node-lookup BINARY := node-lookup
# All shipped binaries and the package path each is built from. node-lookup is
# the module root; the companion tools live under cmd/.
BINARIES := node-lookup pburl pblastreport
DIST := dist
VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo dev) VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo dev)
GOFLAGS := -ldflags="-s -w -X main.version=$(VERSION)" GOFLAGS := -ldflags="-s -w -X main.version=$(VERSION)"
OS ?= $(shell go env GOOS)
ARCH ?= $(shell go env GOARCH)
.PHONY: all build test lint clean install patch minor major _tag # The Go package path for a binary: node-lookup is the module root, the
# companion tools live under cmd/. Usable inside a shell for-loop over $(BINARIES).
pkgpath = $$([ "$$b" = "node-lookup" ] && echo . || echo ./cmd/$$b)
.PHONY: all build test lint fmt clean install completions rpm rpm-package patch minor major _tag
all: build all: build
# Build every binary into dist/ so the nfpm packaging step
# (scripts/build-rpm.sh) can find them. Each main package needs its own -o, so
# they are built individually rather than with a single ./... invocation.
build: build:
go build $(GOFLAGS) -o $(BINARY) ./... @for b in $(BINARIES); do \
echo "building $$b"; \
CGO_ENABLED=0 GOOS=$(OS) GOARCH=$(ARCH) go build $(GOFLAGS) -o $(DIST)/$$b $(pkgpath) || exit 1; \
done
test: test:
go test -v -race ./... go test -v -race ./...
@@ -15,12 +31,31 @@ test:
lint: lint:
golangci-lint run ./... golangci-lint run ./...
fmt:
gofmt -w .
clean: clean:
rm -f $(BINARY) rm -rf $(DIST) $(BINARIES)
install: install:
go install $(GOFLAGS) ./... go install $(GOFLAGS) ./...
# Generate bash/zsh/fish completions for every binary into dist/completions.
completions: build
@mkdir -p $(DIST)/completions
@for b in $(BINARIES); do \
$(DIST)/$$b completion bash > $(DIST)/completions/$$b.bash; \
$(DIST)/$$b completion zsh > $(DIST)/completions/_$$b; \
$(DIST)/$$b completion fish > $(DIST)/completions/$$b.fish; \
done
# Build the binary then package it (with completions) into an RPM via nfpm.
rpm: build rpm-package
# Package an already-built binary into an RPM (used by CI after the build step).
rpm-package:
./scripts/build-rpm.sh $(VERSION)
# Bump helpers — reads the latest semver tag and creates the next one. # Bump helpers — reads the latest semver tag and creates the next one.
# If no tag exists yet, starts from v0.0.0. # If no tag exists yet, starts from v0.0.0.
_LATEST := $(shell git tag --sort=-v:refname | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$$' | head -1) _LATEST := $(shell git tag --sort=-v:refname | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$$' | head -1)
+97
View File
@@ -0,0 +1,97 @@
// Command pblastreport shows each host's last Puppet report time alongside its
// Puppetboard node-page URL.
//
// It reads hostnames from its arguments or from piped node-lookup output,
// queries PuppetDB for each node's report_timestamp, and prints a
// tab-separated "<host> <last-report> <puppetboard-url>" line:
//
// node-lookup -R | pblastreport
// pblastreport --relative host1.example.net
// pblastreport --timezone Asia/Singapore host1.example.net
package main
import (
"fmt"
"os"
"time"
"node-lookup/internal/puppet"
"github.com/spf13/cobra"
)
var version = "dev"
func main() {
cfg, err := puppet.Load()
if err != nil {
fmt.Fprintln(os.Stderr, "config error:", err)
os.Exit(1)
}
var (
relative bool
tz string
boardURL string
pdbURL string
)
root := &cobra.Command{
Use: "pblastreport [host...]",
Short: "Show each host's last Puppet report time and Puppetboard URL.",
Long: "Reads hostnames from arguments or piped node-lookup output and prints, per\n" +
"host, the time of its last Puppet report and its Puppetboard node-page URL.\n" +
"Times are shown in the local timezone unless --timezone is given, or as a\n" +
"relative age with --relative. Example: node-lookup -R | pblastreport -r",
Args: cobra.ArbitraryArgs,
SilenceUsage: true,
RunE: func(cmd *cobra.Command, args []string) error {
if cmd.Flags().Changed("puppetboard-url") {
cfg.PuppetboardURL = boardURL
}
if cmd.Flags().Changed("url") {
cfg.PuppetDBURL = pdbURL
}
loc := time.Local
if tz != "" {
l, err := time.LoadLocation(tz)
if err != nil {
return fmt.Errorf("invalid timezone %q: %w", tz, err)
}
loc = l
}
hosts := puppet.ReadHosts(os.Stdin, args)
if len(hosts) == 0 {
return fmt.Errorf("no hosts given (pass as arguments or pipe node-lookup output)")
}
nodesURL := puppet.NodesEndpoint(cfg.PuppetDBURL)
now := time.Now()
for _, h := range hosts {
node, lookupErr := puppet.LookupNode(nodesURL, h)
when := formatWhen(node, lookupErr, relative, loc, now)
fmt.Printf("%s\t%s\t%s\n", h, when, puppet.HostPageURL(cfg.PuppetboardURL, h))
}
return nil
},
}
f := root.Flags()
f.BoolVarP(&relative, "relative", "r", false, "Show the report time as a relative age (e.g. '3h ago')")
f.StringVarP(&tz, "timezone", "z", "", "IANA timezone for the report time (e.g. Asia/Singapore); default local")
f.StringVar(&boardURL, "puppetboard-url", cfg.PuppetboardURL, "Puppetboard base URL (overrides config and NODE_LOOKUP_PUPPETBOARD_URL)")
f.StringVar(&pdbURL, "url", cfg.PuppetDBURL, "PuppetDB facts URL (overrides config and NODE_LOOKUP_URL)")
root.AddCommand(&cobra.Command{
Use: "version",
Short: "Print the version",
Run: func(cmd *cobra.Command, args []string) { fmt.Println(version) },
SilenceUsage: true,
})
if err := root.Execute(); err != nil {
os.Exit(1)
}
}
+53
View File
@@ -0,0 +1,53 @@
package main
import (
"fmt"
"time"
"node-lookup/internal/puppet"
)
// formatWhen renders the "last report" column for a node lookup result. It
// handles the lookup error, the unknown-node, and the never-reported cases so
// the output always has a value in every column.
func formatWhen(node *puppet.Node, lookupErr error, relative bool, loc *time.Location, now time.Time) string {
if lookupErr != nil {
return "error: " + lookupErr.Error()
}
if node == nil {
return "unknown node"
}
if node.ReportTimestamp == "" {
return "no report"
}
ts, err := time.Parse(time.RFC3339Nano, node.ReportTimestamp)
if err != nil {
return node.ReportTimestamp // fall back to the raw value
}
if relative {
return humanizeSince(now, ts)
}
return ts.In(loc).Format("2006-01-02 15:04:05 MST")
}
// humanizeSince renders the gap between now and ts as a coarse relative string
// ("42s ago", "9m ago", "3h ago", "5d ago"). Future timestamps (clock skew)
// render as "in <d>".
func humanizeSince(now, ts time.Time) string {
d := now.Sub(ts)
suffix := "ago"
if d < 0 {
d = -d
suffix = "from now"
}
switch {
case d < time.Minute:
return fmt.Sprintf("%ds %s", int(d.Seconds()), suffix)
case d < time.Hour:
return fmt.Sprintf("%dm %s", int(d.Minutes()), suffix)
case d < 24*time.Hour:
return fmt.Sprintf("%dh %s", int(d.Hours()), suffix)
default:
return fmt.Sprintf("%dd %s", int(d.Hours()/24), suffix)
}
}
+84
View File
@@ -0,0 +1,84 @@
package main
import (
"errors"
"strings"
"testing"
"time"
"node-lookup/internal/puppet"
)
func mustTime(t *testing.T, s string) time.Time {
t.Helper()
ts, err := time.Parse(time.RFC3339Nano, s)
if err != nil {
t.Fatal(err)
}
return ts
}
func TestFormatWhen_Absolute(t *testing.T) {
node := &puppet.Node{ReportTimestamp: "2026-07-15T04:05:06.000Z"}
now := mustTime(t, "2026-07-15T10:00:00Z")
got := formatWhen(node, nil, false, time.UTC, now)
if got != "2026-07-15 04:05:06 UTC" {
t.Fatalf("unexpected absolute time: %q", got)
}
}
func TestFormatWhen_TimezoneApplied(t *testing.T) {
loc, err := time.LoadLocation("Asia/Singapore") // UTC+8, no DST
if err != nil {
t.Skipf("tzdata unavailable: %v", err)
}
node := &puppet.Node{ReportTimestamp: "2026-07-15T04:05:06Z"}
now := mustTime(t, "2026-07-15T10:00:00Z")
got := formatWhen(node, nil, false, loc, now)
if !strings.HasPrefix(got, "2026-07-15 12:05:06") {
t.Fatalf("expected +08 time, got %q", got)
}
}
func TestFormatWhen_Relative(t *testing.T) {
node := &puppet.Node{ReportTimestamp: "2026-07-15T07:00:00Z"}
now := mustTime(t, "2026-07-15T10:00:00Z")
if got := formatWhen(node, nil, true, time.UTC, now); got != "3h ago" {
t.Fatalf("expected '3h ago', got %q", got)
}
}
func TestFormatWhen_EdgeCases(t *testing.T) {
now := mustTime(t, "2026-07-15T10:00:00Z")
if got := formatWhen(nil, errors.New("down"), false, time.UTC, now); !strings.HasPrefix(got, "error:") {
t.Fatalf("expected error passthrough, got %q", got)
}
if got := formatWhen(nil, nil, false, time.UTC, now); got != "unknown node" {
t.Fatalf("expected unknown node, got %q", got)
}
if got := formatWhen(&puppet.Node{ReportTimestamp: ""}, nil, false, time.UTC, now); got != "no report" {
t.Fatalf("expected no report, got %q", got)
}
if got := formatWhen(&puppet.Node{ReportTimestamp: "garbage"}, nil, false, time.UTC, now); got != "garbage" {
t.Fatalf("expected raw fallback, got %q", got)
}
}
func TestHumanizeSince(t *testing.T) {
base := mustTime(t, "2026-07-15T10:00:00Z")
cases := []struct {
ts string
want string
}{
{"2026-07-15T09:59:30Z", "30s ago"},
{"2026-07-15T09:45:00Z", "15m ago"},
{"2026-07-15T05:00:00Z", "5h ago"},
{"2026-07-13T10:00:00Z", "2d ago"},
{"2026-07-15T10:01:00Z", "1m from now"},
}
for _, c := range cases {
if got := humanizeSince(base, mustTime(t, c.ts)); got != c.want {
t.Errorf("humanizeSince(%s) = %q, want %q", c.ts, got, c.want)
}
}
}
+63
View File
@@ -0,0 +1,63 @@
// Command pburl prints the Puppetboard node-page URL for each host it is given.
//
// It reads hostnames from its arguments or from piped node-lookup output and
// emits "<host> <puppetboard-url>" per line:
//
// node-lookup -R | pburl
// pburl host1.example.net host2.example.net
package main
import (
"fmt"
"os"
"node-lookup/internal/puppet"
"github.com/spf13/cobra"
)
var version = "dev"
func main() {
cfg, err := puppet.Load()
if err != nil {
fmt.Fprintln(os.Stderr, "config error:", err)
os.Exit(1)
}
var boardURL string
root := &cobra.Command{
Use: "pburl [host...]",
Short: "Print the Puppetboard node-page URL for each host.",
Long: "Reads hostnames from arguments or piped node-lookup output and prints\n" +
"'<host> <puppetboard-url>' for each. Example: node-lookup -R | pburl",
Args: cobra.ArbitraryArgs,
SilenceUsage: true,
RunE: func(cmd *cobra.Command, args []string) error {
if cmd.Flags().Changed("puppetboard-url") {
cfg.PuppetboardURL = boardURL
}
hosts := puppet.ReadHosts(os.Stdin, args)
if len(hosts) == 0 {
return fmt.Errorf("no hosts given (pass as arguments or pipe node-lookup output)")
}
for _, h := range hosts {
fmt.Printf("%s %s\n", h, puppet.HostPageURL(cfg.PuppetboardURL, h))
}
return nil
},
}
root.Flags().StringVar(&boardURL, "puppetboard-url", cfg.PuppetboardURL, "Puppetboard base URL (overrides config and NODE_LOOKUP_PUPPETBOARD_URL)")
root.AddCommand(&cobra.Command{
Use: "version",
Short: "Print the version",
Run: func(cmd *cobra.Command, args []string) { fmt.Println(version) },
SilenceUsage: true,
})
if err := root.Execute(); err != nil {
os.Exit(1)
}
}
+7 -4
View File
@@ -3,8 +3,11 @@ module node-lookup
go 1.25.7 go 1.25.7
require ( require (
github.com/inconshreveable/mousetrap v1.1.0 // indirect github.com/spf13/cobra v1.10.2
github.com/spf13/cobra v1.10.2 // indirect gopkg.in/yaml.v3 v3.0.1
github.com/spf13/pflag v1.0.9 // indirect )
gopkg.in/yaml.v3 v3.0.1 // indirect
require (
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/spf13/pflag v1.0.9 // indirect
) )
+1
View File
@@ -7,6 +7,7 @@ github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiT
github.com/spf13/pflag v1.0.9 h1:9exaQaMOCwffKiiiYk6/BndUBv+iRViNW+4lEMi0PvY= github.com/spf13/pflag v1.0.9 h1:9exaQaMOCwffKiiiYk6/BndUBv+iRViNW+4lEMi0PvY=
github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+9
View File
@@ -0,0 +1,9 @@
package puppet
import "strings"
// HostPageURL returns the Puppetboard node-detail page URL for a certname,
// e.g. https://puppetboard.example.net/node/host1.example.net.
func HostPageURL(base, certname string) string {
return strings.TrimRight(base, "/") + "/node/" + certname
}
+90
View File
@@ -0,0 +1,90 @@
// Package puppet holds the small pieces of PuppetDB/Puppetboard plumbing shared
// by the node-lookup companion tools (pburl, pblastreport): config loading,
// PuppetDB "nodes" queries, Puppetboard URL construction, and reading hostnames
// from piped node-lookup output.
//
// It intentionally reads the SAME config file, env vars, and defaults as the
// node-lookup CLI so a single `~/.config/node-lookup/config.yaml` configures
// every tool in the family.
package puppet
import (
"fmt"
"os"
"path/filepath"
"gopkg.in/yaml.v3"
)
const (
// DefaultPuppetDBURL is the PuppetDB v4 facts endpoint (shared with node-lookup).
DefaultPuppetDBURL = "http://puppetdbapi.service.consul:8080/pdb/query/v4/facts"
// DefaultRoleFact is the role fact node-lookup queries with -R.
DefaultRoleFact = "enc_role"
// DefaultPuppetboardURL is the base URL of the Puppetboard web UI.
DefaultPuppetboardURL = "https://puppetboard.k8s.syd1.au.unkin.net"
appName = "node-lookup"
configFileName = "config.yaml"
)
// Config mirrors node-lookup's config plus the puppetboard_url key used by the
// companion tools. Fields map 1:1 to config file keys and env vars.
type Config struct {
PuppetDBURL string `yaml:"puppetdb_url"`
RoleFact string `yaml:"role_fact"`
PuppetboardURL string `yaml:"puppetboard_url"`
}
// DefaultConfig returns the built-in defaults.
func DefaultConfig() Config {
return Config{
PuppetDBURL: DefaultPuppetDBURL,
RoleFact: DefaultRoleFact,
PuppetboardURL: DefaultPuppetboardURL,
}
}
// ConfigDir returns the XDG_CONFIG_HOME/node-lookup directory.
func ConfigDir() string {
base := os.Getenv("XDG_CONFIG_HOME")
if base == "" {
home, _ := os.UserHomeDir()
base = filepath.Join(home, ".config")
}
return filepath.Join(base, appName)
}
// ConfigPath returns the full path to the shared config file.
func ConfigPath() string {
return filepath.Join(ConfigDir(), configFileName)
}
// Load reads the config file (if present), then applies env var overrides.
// Precedence (lowest → highest): defaults < config file < env vars.
func Load() (Config, error) {
cfg := DefaultConfig()
path := ConfigPath()
data, err := os.ReadFile(path)
if err != nil && !os.IsNotExist(err) {
return cfg, fmt.Errorf("reading config %s: %w", path, err)
}
if err == nil {
if err := yaml.Unmarshal(data, &cfg); err != nil {
return cfg, fmt.Errorf("parsing config %s: %w", path, err)
}
}
if v := os.Getenv("NODE_LOOKUP_URL"); v != "" {
cfg.PuppetDBURL = v
}
if v := os.Getenv("NODE_LOOKUP_ROLE_FACT"); v != "" {
cfg.RoleFact = v
}
if v := os.Getenv("NODE_LOOKUP_PUPPETBOARD_URL"); v != "" {
cfg.PuppetboardURL = v
}
return cfg, nil
}
+177
View File
@@ -0,0 +1,177 @@
package puppet
import (
"encoding/json"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
)
// ---- config -----------------------------------------------------------------
func TestLoad_Defaults(t *testing.T) {
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
t.Setenv("NODE_LOOKUP_URL", "")
t.Setenv("NODE_LOOKUP_ROLE_FACT", "")
t.Setenv("NODE_LOOKUP_PUPPETBOARD_URL", "")
cfg, err := Load()
if err != nil {
t.Fatal(err)
}
if cfg.PuppetDBURL != DefaultPuppetDBURL {
t.Fatalf("expected default puppetdb url, got %s", cfg.PuppetDBURL)
}
if cfg.PuppetboardURL != DefaultPuppetboardURL {
t.Fatalf("expected default puppetboard url, got %s", cfg.PuppetboardURL)
}
}
func TestLoad_FileAndEnvOverride(t *testing.T) {
dir := t.TempDir()
t.Setenv("XDG_CONFIG_HOME", dir)
t.Setenv("NODE_LOOKUP_URL", "")
t.Setenv("NODE_LOOKUP_ROLE_FACT", "")
t.Setenv("NODE_LOOKUP_PUPPETBOARD_URL", "https://env.example.net")
cfgDir := filepath.Join(dir, appName)
if err := os.MkdirAll(cfgDir, 0o755); err != nil {
t.Fatal(err)
}
body := "puppetdb_url: http://file:8080/pdb/query/v4/facts\npuppetboard_url: https://file.example.net\n"
if err := os.WriteFile(filepath.Join(cfgDir, configFileName), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
cfg, err := Load()
if err != nil {
t.Fatal(err)
}
if cfg.PuppetDBURL != "http://file:8080/pdb/query/v4/facts" {
t.Fatalf("file override failed: %s", cfg.PuppetDBURL)
}
// env beats the file for puppetboard_url
if cfg.PuppetboardURL != "https://env.example.net" {
t.Fatalf("env should beat file: %s", cfg.PuppetboardURL)
}
}
// ---- NodesEndpoint ----------------------------------------------------------
func TestNodesEndpoint(t *testing.T) {
cases := map[string]string{
"http://puppetdbapi.service.consul:8080/pdb/query/v4/facts": "http://puppetdbapi.service.consul:8080/pdb/query/v4/nodes",
"http://h:8080/pdb/query/v4/facts/": "http://h:8080/pdb/query/v4/nodes",
"https://h/facts": "https://h/nodes",
}
for in, want := range cases {
if got := NodesEndpoint(in); got != want {
t.Errorf("NodesEndpoint(%q) = %q, want %q", in, got, want)
}
}
}
// ---- HostPageURL ------------------------------------------------------------
func TestHostPageURL(t *testing.T) {
if got := HostPageURL("https://pb.example.net", "h1.example.net"); got != "https://pb.example.net/node/h1.example.net" {
t.Fatalf("unexpected url: %s", got)
}
// trailing slash on the base is trimmed
if got := HostPageURL("https://pb.example.net/", "h1"); got != "https://pb.example.net/node/h1" {
t.Fatalf("trailing slash not handled: %s", got)
}
}
// ---- LookupNode -------------------------------------------------------------
func TestLookupNode_Found(t *testing.T) {
var gotQuery string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotQuery = r.URL.Query().Get("query")
_ = json.NewEncoder(w).Encode([]Node{{
Certname: "h1",
ReportTimestamp: "2026-07-15T04:05:06.000Z",
LatestReportStatus: "changed",
}})
}))
defer srv.Close()
node, err := LookupNode(srv.URL, "h1")
if err != nil {
t.Fatal(err)
}
if node == nil || node.ReportTimestamp != "2026-07-15T04:05:06.000Z" {
t.Fatalf("unexpected node: %+v", node)
}
if !strings.Contains(gotQuery, "certname") || !strings.Contains(gotQuery, "h1") {
t.Fatalf("query missing certname filter: %s", gotQuery)
}
}
func TestLookupNode_Unknown(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_ = json.NewEncoder(w).Encode([]Node{})
}))
defer srv.Close()
node, err := LookupNode(srv.URL, "nope")
if err != nil {
t.Fatal(err)
}
if node != nil {
t.Fatalf("expected nil node for unknown certname, got %+v", node)
}
}
func TestLookupNode_HTTPError(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Error(w, "boom", http.StatusInternalServerError)
}))
defer srv.Close()
if _, err := LookupNode(srv.URL, "h1"); err == nil {
t.Fatal("expected error for HTTP 500")
}
}
// ---- ReadHosts --------------------------------------------------------------
func TestReadHosts_ArgsWin(t *testing.T) {
// A real pipe with data present, but explicit args should take precedence.
r, w, _ := os.Pipe()
go func() { _, _ = w.WriteString("piped\n"); _ = w.Close() }()
defer func() { _ = r.Close() }()
got := ReadHosts(r, []string{"a", "b", "a"})
if strings.Join(got, ",") != "a,b" {
t.Fatalf("expected deduped args, got %v", got)
}
}
func TestReadHosts_StdinFirstField(t *testing.T) {
r, w, _ := os.Pipe()
go func() {
// node-lookup default output: "host value"; also a bare host and a dup.
_, _ = w.WriteString("host1 roles::web\nhost2 roles::db\nhost1 roles::web\nhost3\n")
_ = w.Close()
}()
defer func() { _ = r.Close() }()
got := ReadHosts(r, nil)
if strings.Join(got, ",") != "host1,host2,host3" {
t.Fatalf("expected first-field hosts deduped, got %v", got)
}
}
func TestReadHosts_NoInput(t *testing.T) {
// /dev/null is a char device: no args, no pipe data -> nil.
f, _ := os.Open(os.DevNull)
defer func() { _ = f.Close() }()
if got := ReadHosts(f, nil); got != nil {
t.Fatalf("expected nil for no input, got %v", got)
}
}
+71
View File
@@ -0,0 +1,71 @@
package puppet
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"strings"
)
// Node is the subset of a PuppetDB v4 "nodes" record the companion tools use.
type Node struct {
Certname string `json:"certname"`
ReportTimestamp string `json:"report_timestamp"`
LatestReportStatus string `json:"latest_report_status"`
}
// NodesEndpoint derives the PuppetDB v4 "nodes" query endpoint from the
// configured facts endpoint by swapping the final path segment
// (…/pdb/query/v4/facts → …/pdb/query/v4/nodes). It leaves scheme/host/query
// untouched, so a custom NODE_LOOKUP_URL still resolves correctly.
func NodesEndpoint(factsURL string) string {
u, err := url.Parse(factsURL)
if err != nil {
return factsURL
}
p := strings.TrimRight(u.Path, "/")
if i := strings.LastIndex(p, "/"); i >= 0 {
p = p[:i] + "/nodes"
} else {
p = "/nodes"
}
u.Path = p
return u.String()
}
// LookupNode fetches the single PuppetDB node record for certname. It returns
// (nil, nil) when PuppetDB knows of no such node.
func LookupNode(nodesURL, certname string) (*Node, error) {
q, _ := json.Marshal([]interface{}{"=", "certname", certname})
nodes, err := queryNodes(nodesURL, string(q))
if err != nil {
return nil, err
}
if len(nodes) == 0 {
return nil, nil
}
return &nodes[0], nil
}
func queryNodes(nodesURL, query string) ([]Node, error) {
params := url.Values{}
params.Set("query", query)
resp, err := http.Get(nodesURL + "?" + params.Encode())
if err != nil {
return nil, fmt.Errorf("request failed: %w", err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
return nil, fmt.Errorf("HTTP %d: %s", resp.StatusCode, string(body))
}
var nodes []Node
if err := json.NewDecoder(resp.Body).Decode(&nodes); err != nil {
return nil, fmt.Errorf("decode error: %w", err)
}
return nodes, nil
}
+66
View File
@@ -0,0 +1,66 @@
package puppet
import (
"bufio"
"os"
"strings"
)
// StdinReader returns a buffered reader over f and true only when f actually
// carries piped/redirected data. Terminals and character devices such as
// /dev/null return false, and an empty pipe or empty file (immediate EOF on
// peek) also returns false. This mirrors node-lookup's no-TTY behaviour: when
// invoked without a real pipe the caller can fall back to arguments instead of
// blocking on or silently consuming empty input.
func StdinReader(f *os.File) (*bufio.Reader, bool) {
fi, err := f.Stat()
if err != nil {
return nil, false
}
if (fi.Mode() & os.ModeCharDevice) != 0 {
return nil, false // terminal or /dev/null
}
r := bufio.NewReader(f)
if _, err := r.Peek(1); err != nil {
return nil, false // empty pipe / empty file (EOF)
}
return r, true
}
// ReadHosts resolves the list of hostnames to act on. Explicit args win; failing
// that it reads the first whitespace-separated field of each non-empty line from
// stdin (so `node-lookup -R | pburl` and `node-lookup -1 | pburl` both work).
// Order is preserved and duplicates are removed. Returns nil when neither args
// nor piped stdin data are present.
func ReadHosts(stdin *os.File, args []string) []string {
if len(args) > 0 {
return dedupe(args)
}
r, ok := StdinReader(stdin)
if !ok {
return nil
}
var hosts []string
sc := bufio.NewScanner(r)
for sc.Scan() {
fields := strings.Fields(sc.Text())
if len(fields) == 0 {
continue
}
hosts = append(hosts, fields[0])
}
return dedupe(hosts)
}
func dedupe(in []string) []string {
seen := make(map[string]struct{}, len(in))
out := make([]string, 0, len(in))
for _, s := range in {
if _, ok := seen[s]; ok {
continue
}
seen[s] = struct{}{}
out = append(out, s)
}
return out
}
+163 -40
View File
@@ -21,6 +21,8 @@ import (
const ( const (
defaultPuppetDBURL = "http://puppetdbapi.service.consul:8080/pdb/query/v4/facts" defaultPuppetDBURL = "http://puppetdbapi.service.consul:8080/pdb/query/v4/facts"
defaultRoleFact = "enc_role" defaultRoleFact = "enc_role"
defaultPuppetboardURL = "https://puppetboard.k8s.syd1.au.unkin.net"
defaultDomain = "main.unkin.net"
configFileName = "config.yaml" configFileName = "config.yaml"
appName = "node-lookup" appName = "node-lookup"
) )
@@ -32,12 +34,20 @@ var version = "dev"
type config struct { type config struct {
PuppetDBURL string `yaml:"puppetdb_url"` PuppetDBURL string `yaml:"puppetdb_url"`
RoleFact string `yaml:"role_fact"` RoleFact string `yaml:"role_fact"`
// PuppetboardURL is not used by node-lookup itself; it is scaffolded here so
// the shared config file also configures the companion tools (pburl,
// pblastreport) that read this same file.
PuppetboardURL string `yaml:"puppetboard_url"`
// Domain is appended to a short (dotless) -n node name to form its FQDN.
Domain string `yaml:"domain"`
} }
func defaultConfig() config { func defaultConfig() config {
return config{ return config{
PuppetDBURL: defaultPuppetDBURL, PuppetDBURL: defaultPuppetDBURL,
RoleFact: defaultRoleFact, RoleFact: defaultRoleFact,
PuppetboardURL: defaultPuppetboardURL,
Domain: defaultDomain,
} }
} }
@@ -78,6 +88,12 @@ func loadConfig() (config, error) {
if v := os.Getenv("NODE_LOOKUP_ROLE_FACT"); v != "" { if v := os.Getenv("NODE_LOOKUP_ROLE_FACT"); v != "" {
cfg.RoleFact = v cfg.RoleFact = v
} }
if v := os.Getenv("NODE_LOOKUP_PUPPETBOARD_URL"); v != "" {
cfg.PuppetboardURL = v
}
if v := os.Getenv("NODE_LOOKUP_DOMAIN"); v != "" {
cfg.Domain = v
}
return cfg, nil return cfg, nil
} }
@@ -96,7 +112,7 @@ func writeDefaultConfig() error {
cfg := defaultConfig() cfg := defaultConfig()
data, _ := yaml.Marshal(cfg) data, _ := yaml.Marshal(cfg)
header := []byte("# node-lookup configuration\n# Fields can be overridden with env vars: NODE_LOOKUP_URL, NODE_LOOKUP_ROLE_FACT\n\n") header := []byte("# node-lookup configuration\n# Fields can be overridden with env vars: NODE_LOOKUP_URL, NODE_LOOKUP_ROLE_FACT, NODE_LOOKUP_PUPPETBOARD_URL, NODE_LOOKUP_DOMAIN\n# puppetboard_url is used by the companion tools (pburl, pblastreport).\n# domain is appended to short (dotless) -n node names to form their FQDN.\n\n")
if err := os.WriteFile(path, append(header, data...), 0o644); err != nil { if err := os.WriteFile(path, append(header, data...), 0o644); err != nil {
return fmt.Errorf("writing config: %w", err) return fmt.Errorf("writing config: %w", err)
} }
@@ -110,23 +126,70 @@ type fact struct {
Value json.RawMessage `json:"value"` Value json.RawMessage `json:"value"`
} }
func buildQuery(node, factName, match, partialMatch, roleFact string, showRole bool) string { // splitFactNames splits a comma-separated -F value into trimmed, non-empty
// names, so `-F ipaddress,enc_role` queries both facts.
func splitFactNames(factName string) []string {
var names []string
for _, n := range strings.Split(factName, ",") {
if n = strings.TrimSpace(n); n != "" {
names = append(names, n)
}
}
return names
}
// nameFilter returns a PQL filter matching any of the given fact names: a plain
// equality for one name, an "or" over per-name equalities for several.
func nameFilter(names []string) []interface{} {
if len(names) == 1 {
return []interface{}{"=", "name", names[0]}
}
or := []interface{}{"or"}
for _, n := range names {
or = append(or, []interface{}{"=", "name", n})
}
return or
}
// qualifyNode auto-qualifies a short (dotless) node name by appending
// ".<domain>", so `-n ausyd1nxvm2120` resolves the same as its FQDN. A name
// that already contains a dot is treated as already-qualified (including names
// in other domains like *.k8s.syd1.au.unkin.net) and returned unchanged. A
// single trailing dot is stripped first, so a dotless name with a trailing dot
// is still qualified. Empty input is returned unchanged to preserve the
// existing "no node given" behavior.
func qualifyNode(name, domain string) string {
name = strings.TrimSuffix(name, ".")
if name == "" || strings.Contains(name, ".") {
return name
}
return name + "." + domain
}
func buildQuery(node, factName, match, roleFact string, showRole, partial, inverse bool) string {
type filter = []interface{} type filter = []interface{}
var filters []filter var filters []filter
if node != "" { if node != "" {
filters = append(filters, filter{"=", "certname", node}) filters = append(filters, filter{"=", "certname", node})
} }
if factName != "" { if names := splitFactNames(factName); len(names) > 0 {
filters = append(filters, filter{"=", "name", factName}) filters = append(filters, nameFilter(names))
} else if showRole { } else if showRole {
filters = append(filters, filter{"=", "name", roleFact}) filters = append(filters, filter{"=", "name", roleFact})
} }
if match != "" { if match != "" {
filters = append(filters, filter{"=", "value", match}) op := "="
} else if partialMatch != "" { if partial {
filters = append(filters, filter{"~", "value", partialMatch}) op = "~"
}
inner := filter{op, "value", match}
if inverse {
filters = append(filters, filter{"not", inner})
} else {
filters = append(filters, inner)
}
} }
if len(filters) == 0 { if len(filters) == 0 {
@@ -207,12 +270,66 @@ func countResults(lines []string) []string {
return out return out
} }
func isTerminal(f *os.File) bool { // stdinReader returns a buffered reader over f and true only when f actually
// carries piped/redirected data to consume as node names. Terminals and
// character devices such as /dev/null return false, and an empty pipe or empty
// file (immediate EOF on peek) also returns false. This means running without a
// TTY — e.g. invoked by an agent or CI where stdin is /dev/null or a closed
// pipe — falls through to a normal query instead of silently consuming empty
// input and printing nothing.
func stdinReader(f *os.File) (*bufio.Reader, bool) {
fi, err := f.Stat() fi, err := f.Stat()
if err != nil { if err != nil {
return false return nil, false
} }
return (fi.Mode() & os.ModeCharDevice) != 0 if (fi.Mode() & os.ModeCharDevice) != 0 {
return nil, false // terminal or /dev/null
}
r := bufio.NewReader(f)
if _, err := r.Peek(1); err != nil {
return nil, false // empty pipe / empty file (EOF)
}
return r, true
}
// matchValue resolves the value to match against. The -m/--match flag wins; if
// it is empty, the (optional) positional argument is used instead. The
// positional fallback exists so combined shorthands like `-pm k8s` work — pflag
// leaves the space-separated `k8s` as a positional rather than attaching it to
// the grouped -m flag.
func matchValue(flagMatch string, args []string) string {
if flagMatch != "" {
return flagMatch
}
if len(args) > 0 {
return args[0]
}
return ""
}
// factsByHost groups collected facts into {certname: {factname: value}}. Each
// value is keyed by the fact's real name, so multiple -F facts each appear
// under the host; it falls back to the -F string, the role fact (with -R), or
// "value" only when a result carries no name. Shared by the -j and -A outputs.
func factsByHost(collected []fact, factName, roleFact string, showRole bool) map[string]map[string]interface{} {
out := map[string]map[string]interface{}{}
for _, f := range collected {
if _, ok := out[f.Certname]; !ok {
out[f.Certname] = map[string]interface{}{}
}
key := f.Name
if key == "" {
if key = factName; key == "" {
if showRole {
key = roleFact
} else {
key = "value"
}
}
}
out[f.Certname][key] = valueAny(f.Value)
}
return out
} }
func allFactsForNode(puppetDBURL, node string) ([]fact, error) { func allFactsForNode(puppetDBURL, node string) ([]fact, error) {
@@ -220,9 +337,11 @@ func allFactsForNode(puppetDBURL, node string) ([]fact, error) {
return queryPuppetDB(puppetDBURL, string(query)) return queryPuppetDB(puppetDBURL, string(query))
} }
func run(cfg config, nodeName, factName, match, partialMatch string, showRole, nodeOnly, valueOnly, count, ansible, jsonMode, allFacts bool) error { func run(cfg config, nodeName, factName, match string, showRole, partial, inverse, nodeOnly, valueOnly, count, ansible, jsonMode, allFacts bool) error {
signal.Ignore(syscall.SIGPIPE) signal.Ignore(syscall.SIGPIPE)
nodeName = qualifyNode(nodeName, cfg.Domain)
if allFacts { if allFacts {
if nodeName == "" { if nodeName == "" {
return fmt.Errorf("-a requires -n") return fmt.Errorf("-a requires -n")
@@ -241,12 +360,15 @@ func run(cfg config, nodeName, factName, match, partialMatch string, showRole, n
if (nodeOnly || valueOnly || count || ansible) && !showRole && factName == "" { if (nodeOnly || valueOnly || count || ansible) && !showRole && factName == "" {
return fmt.Errorf("-R or -F must be used with -1, -2, -C, or -A") return fmt.Errorf("-R or -F must be used with -1, -2, -C, or -A")
} }
if (match != "" || partial || inverse) && !showRole && factName == "" {
return fmt.Errorf("-R or -F must be used with -m, -p, or -i")
}
var collected []fact var collected []fact
var stdinLines []string var stdinLines []string
doQuery := func(node string) error { doQuery := func(node string) error {
query := buildQuery(node, factName, match, partialMatch, cfg.RoleFact, showRole) query := buildQuery(node, factName, match, cfg.RoleFact, showRole, partial, inverse)
facts, err := queryPuppetDB(cfg.PuppetDBURL, query) facts, err := queryPuppetDB(cfg.PuppetDBURL, query)
if err != nil { if err != nil {
return err return err
@@ -255,8 +377,8 @@ func run(cfg config, nodeName, factName, match, partialMatch string, showRole, n
return nil return nil
} }
if nodeName == "" && !isTerminal(os.Stdin) { if reader, ok := stdinReader(os.Stdin); ok && nodeName == "" {
scanner := bufio.NewScanner(os.Stdin) scanner := bufio.NewScanner(reader)
if count { if count {
for scanner.Scan() { for scanner.Scan() {
stdinLines = append(stdinLines, scanner.Text()) stdinLines = append(stdinLines, scanner.Text())
@@ -267,7 +389,7 @@ func run(cfg config, nodeName, factName, match, partialMatch string, showRole, n
if len(fields) == 0 { if len(fields) == 0 {
continue continue
} }
if err := doQuery(fields[0]); err != nil { if err := doQuery(qualifyNode(fields[0], cfg.Domain)); err != nil {
fmt.Fprintln(os.Stderr, "error:", err) fmt.Fprintln(os.Stderr, "error:", err)
} }
} }
@@ -282,25 +404,10 @@ func run(cfg config, nodeName, factName, match, partialMatch string, showRole, n
switch { switch {
case jsonMode: case jsonMode:
hostFactMap := map[string]map[string]interface{}{}
for _, f := range collected {
if _, ok := hostFactMap[f.Certname]; !ok {
hostFactMap[f.Certname] = map[string]interface{}{}
}
key := factName
if key == "" {
if showRole {
key = cfg.RoleFact
} else {
key = "value"
}
}
hostFactMap[f.Certname][key] = valueAny(f.Value)
}
enc := json.NewEncoder(os.Stdout) enc := json.NewEncoder(os.Stdout)
enc.SetIndent("", " ") enc.SetIndent("", " ")
enc.SetEscapeHTML(false) enc.SetEscapeHTML(false)
_ = enc.Encode(hostFactMap) _ = enc.Encode(factsByHost(collected, factName, cfg.RoleFact, showRole))
case count: case count:
values := stdinLines values := stdinLines
@@ -310,10 +417,11 @@ func run(cfg config, nodeName, factName, match, partialMatch string, showRole, n
fmt.Println(strings.Join(countResults(values), "\n")) fmt.Println(strings.Join(countResults(values), "\n"))
case ansible: case ansible:
// Attach each host's queried fact(s) as inventory host vars, e.g.
// `-F ipaddress,enc_role -A` yields hosts with ipaddress + enc_role set.
hosts := map[string]interface{}{} hosts := map[string]interface{}{}
for _, line := range returnData { for host, vars := range factsByHost(collected, factName, cfg.RoleFact, showRole) {
host := strings.Fields(line)[0] hosts[host] = vars
hosts[host] = map[string]interface{}{}
} }
inventory := map[string]interface{}{ inventory := map[string]interface{}{
"all": map[string]interface{}{"hosts": hosts}, "all": map[string]interface{}{"hosts": hosts},
@@ -353,7 +461,8 @@ func main() {
factName string factName string
showRole bool showRole bool
match string match string
partialMatch string partial bool
inverse bool
nodeOnly bool nodeOnly bool
valueOnly bool valueOnly bool
count bool count bool
@@ -361,35 +470,47 @@ func main() {
jsonMode bool jsonMode bool
allFacts bool allFacts bool
puppetDBURL string puppetDBURL string
domain string
) )
rootCmd := &cobra.Command{ rootCmd := &cobra.Command{
Use: appName, Use: appName + " [value]",
Short: "Query PuppetDB for nodes.", Short: "Query PuppetDB for nodes.",
// Accept an optional positional match value in addition to -m. This makes
// combined shorthands like `-pm k8s` work: pflag does not attach a
// space-separated value to a string flag grouped with a bool flag (only
// `-pm=k8s` or `-p -m k8s` do), so `k8s` arrives here as a positional
// argument instead. Falling back to it keeps the ergonomic form working.
Args: cobra.MaximumNArgs(1),
PersistentPreRunE: func(cmd *cobra.Command, args []string) error { PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
if cmd.Flags().Changed("url") { if cmd.Flags().Changed("url") {
cfg.PuppetDBURL = puppetDBURL cfg.PuppetDBURL = puppetDBURL
} }
if cmd.Flags().Changed("domain") {
cfg.Domain = domain
}
return nil return nil
}, },
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
return run(cfg, nodeName, factName, match, partialMatch, showRole, nodeOnly, valueOnly, count, ansible, jsonMode, allFacts) return run(cfg, nodeName, factName, matchValue(match, args), showRole, partial, inverse, nodeOnly, valueOnly, count, ansible, jsonMode, allFacts)
}, },
SilenceUsage: true, SilenceUsage: true,
} }
f := rootCmd.Flags() f := rootCmd.Flags()
f.StringVarP(&nodeName, "node", "n", "", "Node name") f.StringVarP(&nodeName, "node", "n", "", "Node name")
f.StringVarP(&factName, "fact", "F", "", "Fact name") f.StringVarP(&factName, "fact", "F", "", "Fact name (comma-separated for several, e.g. -F ipaddress,enc_role)")
f.BoolVarP(&showRole, "role", "R", false, "Show role fact ("+defaultRoleFact+" by default)") f.BoolVarP(&showRole, "role", "R", false, "Show role fact ("+defaultRoleFact+" by default)")
f.StringVarP(&match, "match", "m", "", "Exact value match") f.StringVarP(&match, "match", "m", "", "Value to match (use with -p and/or -i)")
f.StringVar(&partialMatch, "pm", "", "Partial match on value") f.BoolVarP(&partial, "partial", "p", false, "Partial/regex match modifier (combine with -m)")
f.BoolVarP(&inverse, "inverse", "i", false, "Inverse match modifier (combine with -m)")
f.BoolVarP(&nodeOnly, "nodeonly", "1", false, "Show only the node name") f.BoolVarP(&nodeOnly, "nodeonly", "1", false, "Show only the node name")
f.BoolVarP(&valueOnly, "valueonly", "2", false, "Show only the value") f.BoolVarP(&valueOnly, "valueonly", "2", false, "Show only the value")
f.BoolVarP(&count, "count", "C", false, "Count fact occurrences") f.BoolVarP(&count, "count", "C", false, "Count fact occurrences")
f.BoolVarP(&ansible, "ansible", "A", false, "Output as Ansible inventory") f.BoolVarP(&ansible, "ansible", "A", false, "Output as Ansible inventory")
f.BoolVarP(&jsonMode, "json", "j", false, "Emit valid JSON for all output") f.BoolVarP(&jsonMode, "json", "j", false, "Emit valid JSON for all output")
f.BoolVarP(&allFacts, "all", "a", false, "Show all facts for a node (requires -n)") f.BoolVarP(&allFacts, "all", "a", false, "Show all facts for a node (requires -n)")
f.StringVar(&domain, "domain", cfg.Domain, "Domain appended to short (dotless) -n node names (overrides config and NODE_LOOKUP_DOMAIN)")
rootCmd.PersistentFlags().StringVar(&puppetDBURL, "url", cfg.PuppetDBURL, "PuppetDB facts URL (overrides config and NODE_LOOKUP_URL)") rootCmd.PersistentFlags().StringVar(&puppetDBURL, "url", cfg.PuppetDBURL, "PuppetDB facts URL (overrides config and NODE_LOOKUP_URL)")
configCmd := &cobra.Command{ configCmd := &cobra.Command{
@@ -413,6 +534,8 @@ func main() {
fmt.Printf("config file : %s\n", configPath()) fmt.Printf("config file : %s\n", configPath())
fmt.Printf("puppetdb_url : %s\n", cfg.PuppetDBURL) fmt.Printf("puppetdb_url : %s\n", cfg.PuppetDBURL)
fmt.Printf("role_fact : %s\n", cfg.RoleFact) fmt.Printf("role_fact : %s\n", cfg.RoleFact)
fmt.Printf("puppetboard_url: %s\n", cfg.PuppetboardURL)
fmt.Printf("domain : %s\n", cfg.Domain)
return nil return nil
}, },
SilenceUsage: true, SilenceUsage: true,
+523 -80
View File
@@ -9,6 +9,8 @@ import (
"path/filepath" "path/filepath"
"strings" "strings"
"testing" "testing"
"gopkg.in/yaml.v3"
) )
// ---- helpers ---------------------------------------------------------------- // ---- helpers ----------------------------------------------------------------
@@ -26,74 +28,285 @@ func rawJSON(v interface{}) json.RawMessage {
return json.RawMessage(b) return json.RawMessage(b)
} }
// captureStdout redirects os.Stdout for the duration of fn and returns whatever
// was written. run() writes directly to os.Stdout, so the output modes are
// exercised end-to-end this way.
func captureStdout(t *testing.T, fn func()) string {
t.Helper()
old := os.Stdout
r, w, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
os.Stdout = w
defer func() { os.Stdout = old }()
fn()
_ = w.Close()
var buf strings.Builder
_, _ = io.Copy(&buf, r)
return buf.String()
}
// ---- buildQuery ------------------------------------------------------------- // ---- buildQuery -------------------------------------------------------------
func TestBuildQuery_NoFilters(t *testing.T) { func TestBuildQuery_NoFilters(t *testing.T) {
q := buildQuery("", "", "", "", "enc_role", false) // With no node/fact/match, buildQuery falls back to a role-fact query.
if !strings.Contains(q, "enc_role") { q := buildQuery("", "", "", "enc_role", false, false, false)
if !strings.Contains(q, "enc_role") || !strings.Contains(q, "name") {
t.Fatalf("expected default role query, got %s", q) t.Fatalf("expected default role query, got %s", q)
} }
} }
func TestBuildQuery_Node(t *testing.T) { func TestBuildQuery_Node(t *testing.T) {
q := buildQuery("host1", "", "", "", "enc_role", false) q := buildQuery("host1", "", "", "enc_role", false, false, false)
if !strings.Contains(q, "certname") || !strings.Contains(q, "host1") { if !strings.Contains(q, "certname") || !strings.Contains(q, "host1") {
t.Fatalf("unexpected query: %s", q) t.Fatalf("unexpected query: %s", q)
} }
} }
func TestBuildQuery_FactAndMatch(t *testing.T) { func TestBuildQuery_FactAndMatch(t *testing.T) {
q := buildQuery("", "region", "syd1", "", "enc_role", false) q := buildQuery("", "region", "syd1", "enc_role", false, false, false)
if !strings.Contains(q, "region") || !strings.Contains(q, "syd1") { if !strings.Contains(q, "region") || !strings.Contains(q, "syd1") {
t.Fatalf("unexpected query: %s", q) t.Fatalf("unexpected query: %s", q)
} }
// exact match uses the "=" operator, never "~".
if strings.Contains(q, `"~"`) {
t.Fatalf("exact match should not use ~ operator: %s", q)
}
} }
func TestBuildQuery_PartialMatch(t *testing.T) { func TestBuildQuery_PartialMatch(t *testing.T) {
q := buildQuery("", "enc_role", "", "dns", "enc_role", false) // -p turns the value match into a regex ("~") comparison.
if !strings.Contains(q, "~") || !strings.Contains(q, "dns") { q := buildQuery("", "enc_role", "dns", "enc_role", false, true, false)
t.Fatalf("expected partial match query, got %s", q) if !strings.Contains(q, `"~"`) || !strings.Contains(q, "dns") {
t.Fatalf("expected partial (~) match query, got %s", q)
}
}
func TestBuildQuery_InverseMatch(t *testing.T) {
// -i wraps the value comparison in a "not".
q := buildQuery("", "enc_role", "dns", "enc_role", false, false, true)
if !strings.Contains(q, `"not"`) {
t.Fatalf("expected inverse (not) match query, got %s", q)
}
}
func TestBuildQuery_InversePartialMatch(t *testing.T) {
// -i and -p compose: a negated regex match.
q := buildQuery("", "enc_role", "dns", "enc_role", false, true, true)
if !strings.Contains(q, `"not"`) || !strings.Contains(q, `"~"`) {
t.Fatalf("expected inverse partial match query, got %s", q)
}
// The regex operator must sit inside the "not" wrapper, not beside it.
notIdx := strings.Index(q, `"not"`)
tildeIdx := strings.Index(q, `"~"`)
if notIdx < 0 || tildeIdx < 0 || tildeIdx < notIdx {
t.Fatalf("expected ~ nested inside not, got %s", q)
} }
} }
func TestBuildQuery_ShowRole(t *testing.T) { func TestBuildQuery_ShowRole(t *testing.T) {
q := buildQuery("", "", "", "", "my_role_fact", true) q := buildQuery("", "", "", "my_role_fact", true, false, false)
if !strings.Contains(q, "my_role_fact") { if !strings.Contains(q, "my_role_fact") {
t.Fatalf("expected role fact in query, got %s", q) t.Fatalf("expected role fact in query, got %s", q)
} }
} }
func TestBuildQuery_CustomRoleFact(t *testing.T) { func TestBuildQuery_CustomRoleFact(t *testing.T) {
q := buildQuery("", "", "", "", "custom_role", true) q := buildQuery("", "", "", "custom_role", true, false, false)
if !strings.Contains(q, "custom_role") { if !strings.Contains(q, "custom_role") {
t.Fatalf("expected custom role fact, got %s", q) t.Fatalf("expected custom role fact, got %s", q)
} }
} }
// ---- valueString ------------------------------------------------------------ func TestBuildQuery_FactBeatsRole(t *testing.T) {
// An explicit -F fact name takes precedence over the role fact.
q := buildQuery("", "region", "", "enc_role", true, false, false)
if !strings.Contains(q, "region") {
t.Fatalf("expected explicit fact name, got %s", q)
}
if strings.Contains(q, "enc_role") {
t.Fatalf("role fact should not appear when -F is set: %s", q)
}
}
func TestBuildQuery_ValidJSON(t *testing.T) {
// Whatever the flag combination, the query must be a valid JSON array.
q := buildQuery("host1", "region", "syd1", "enc_role", false, true, true)
var v []interface{}
if err := json.Unmarshal([]byte(q), &v); err != nil {
t.Fatalf("query is not valid JSON: %v (%s)", err, q)
}
if v[0] != "and" {
t.Fatalf("expected combined query to start with 'and', got %v", v[0])
}
}
// ---- multi-fact -F (comma-separated) ----------------------------------------
func TestSplitFactNames(t *testing.T) {
cases := map[string][]string{
"ipaddress": {"ipaddress"},
"ipaddress,enc_role": {"ipaddress", "enc_role"},
"ipaddress, enc_role ": {"ipaddress", "enc_role"}, // trims spaces
"a,,b,": {"a", "b"}, // drops empties
"": nil,
}
for in, want := range cases {
got := splitFactNames(in)
if len(got) != len(want) {
t.Fatalf("splitFactNames(%q) = %v, want %v", in, got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("splitFactNames(%q) = %v, want %v", in, got, want)
}
}
}
}
func TestQualifyNode(t *testing.T) {
const domain = "main.unkin.net"
cases := []struct {
name string
in string
want string
}{
{"short name appends domain", "ausyd1nxvm2120", "ausyd1nxvm2120.main.unkin.net"},
{"fqdn in default domain unchanged", "ausyd1nxvm2120.main.unkin.net", "ausyd1nxvm2120.main.unkin.net"},
{"multi-label fqdn other domain unchanged", "foo.k8s.syd1.au.unkin.net", "foo.k8s.syd1.au.unkin.net"},
{"short name with trailing dot qualified", "ausyd1nxvm2120.", "ausyd1nxvm2120.main.unkin.net"},
{"fqdn with trailing dot stripped", "foo.main.unkin.net.", "foo.main.unkin.net"},
{"empty unchanged", "", ""},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if got := qualifyNode(tc.in, domain); got != tc.want {
t.Fatalf("qualifyNode(%q, %q) = %q, want %q", tc.in, domain, got, tc.want)
}
})
}
}
func TestQualifyNode_CustomDomain(t *testing.T) {
if got := qualifyNode("host1", "example.com"); got != "host1.example.com" {
t.Fatalf("qualifyNode with custom domain = %q, want host1.example.com", got)
}
}
func TestBuildQuery_SingleFact_NoOr(t *testing.T) {
q := buildQuery("", "ipaddress", "", "enc_role", false, false, false)
if strings.Contains(q, `"or"`) {
t.Fatalf("single fact should not use 'or': %s", q)
}
if !strings.Contains(q, "ipaddress") {
t.Fatalf("expected fact name in query: %s", q)
}
}
func TestBuildQuery_MultiFact_UsesOr(t *testing.T) {
q := buildQuery("host1", "ipaddress,enc_role", "", "enc_role", false, false, false)
if !strings.Contains(q, `"or"`) {
t.Fatalf("expected 'or' over fact names: %s", q)
}
if !strings.Contains(q, "ipaddress") || !strings.Contains(q, "enc_role") {
t.Fatalf("expected both fact names: %s", q)
}
// Must remain valid PQL JSON, combined under "and" with the certname filter.
var v []interface{}
if err := json.Unmarshal([]byte(q), &v); err != nil {
t.Fatalf("query is not valid JSON: %v (%s)", err, q)
}
if v[0] != "and" {
t.Fatalf("expected top-level 'and', got %v", v[0])
}
}
func TestRun_JSON_MultipleFacts(t *testing.T) {
// Two facts returned for one host must both appear, keyed by their real name.
facts := []fact{
{Certname: "hosta", Name: "ipaddress", Value: rawJSON("198.18.0.1")},
{Certname: "hosta", Name: "enc_role", Value: rawJSON("roles::dns")},
}
out := runToString(t, facts, func(a *runArgs) {
a.showRole = false
a.factName = "ipaddress,enc_role"
a.jsonMode = true
})
var parsed map[string]map[string]interface{}
if err := json.Unmarshal([]byte(out), &parsed); err != nil {
t.Fatalf("output is not valid JSON: %v (%s)", err, out)
}
host := parsed["hosta"]
if host["ipaddress"] != "198.18.0.1" || host["enc_role"] != "roles::dns" {
t.Fatalf("expected both facts under host, got: %v", host)
}
}
// ---- matchValue (positional fallback for `-pm value`) -----------------------
func TestMatchValue_FlagWins(t *testing.T) {
// An explicit -m value takes precedence over any positional arg.
if got := matchValue("flagval", []string{"posval"}); got != "flagval" {
t.Fatalf("expected flag value to win, got %q", got)
}
}
func TestMatchValue_PositionalFallback(t *testing.T) {
// This is the `-pm k8s` case: pflag leaves k8s as a positional because the
// grouped -m flag does not attach the space-separated value.
if got := matchValue("", []string{"k8s"}); got != "k8s" {
t.Fatalf("expected positional fallback, got %q", got)
}
}
func TestMatchValue_NoneGiven(t *testing.T) {
if got := matchValue("", nil); got != "" {
t.Fatalf("expected empty, got %q", got)
}
}
// ---- valueString / valueAny -------------------------------------------------
func TestValueString_String(t *testing.T) { func TestValueString_String(t *testing.T) {
raw := rawJSON("hello") if got := valueString(rawJSON("hello")); got != "hello" {
if got := valueString(raw); got != "hello" {
t.Fatalf("expected hello, got %s", got) t.Fatalf("expected hello, got %s", got)
} }
} }
func TestValueString_Number(t *testing.T) { func TestValueString_Number(t *testing.T) {
raw := rawJSON(42) if got := valueString(rawJSON(42)); got != "42" {
if got := valueString(raw); got != "42" {
t.Fatalf("expected 42, got %s", got) t.Fatalf("expected 42, got %s", got)
} }
} }
func TestValueString_Bool(t *testing.T) {
if got := valueString(rawJSON(true)); got != "true" {
t.Fatalf("expected true, got %s", got)
}
}
func TestValueString_Object(t *testing.T) { func TestValueString_Object(t *testing.T) {
raw := json.RawMessage(`{"a":1}`) raw := json.RawMessage(`{"a":1}`)
got := valueString(raw) if got := valueString(raw); got != `{"a":1}` {
if got != `{"a":1}` {
t.Fatalf("expected compact JSON, got %s", got) t.Fatalf("expected compact JSON, got %s", got)
} }
} }
func TestValueAny_TypesPreserved(t *testing.T) {
if v := valueAny(rawJSON("s")); v != "s" {
t.Fatalf("expected string, got %v", v)
}
if v := valueAny(rawJSON(3)); v != float64(3) {
t.Fatalf("expected float64(3), got %T %v", v, v)
}
if v := valueAny(rawJSON(true)); v != true {
t.Fatalf("expected bool true, got %v", v)
}
}
// ---- processResults --------------------------------------------------------- // ---- processResults ---------------------------------------------------------
func TestProcessResults_Sorted(t *testing.T) { func TestProcessResults_Sorted(t *testing.T) {
@@ -108,8 +321,7 @@ func TestProcessResults_Sorted(t *testing.T) {
} }
func TestProcessResults_Empty(t *testing.T) { func TestProcessResults_Empty(t *testing.T) {
out := processResults(nil) if out := processResults(nil); len(out) != 0 {
if len(out) != 0 {
t.Fatalf("expected empty, got %v", out) t.Fatalf("expected empty, got %v", out)
} }
} }
@@ -117,11 +329,7 @@ func TestProcessResults_Empty(t *testing.T) {
// ---- countResults ----------------------------------------------------------- // ---- countResults -----------------------------------------------------------
func TestCountResults_Basic(t *testing.T) { func TestCountResults_Basic(t *testing.T) {
lines := []string{ lines := []string{"host1 syd1", "host2 syd1", "host3 mel1"}
"host1 syd1",
"host2 syd1",
"host3 mel1",
}
out := countResults(lines) out := countResults(lines)
found := map[string]bool{} found := map[string]bool{}
for _, l := range out { for _, l := range out {
@@ -136,9 +344,8 @@ func TestCountResults_Basic(t *testing.T) {
} }
func TestCountResults_Sorted(t *testing.T) { func TestCountResults_Sorted(t *testing.T) {
lines := []string{"h1 z", "h2 a", "h3 a"} // Lexicographic sort of the "N: value" strings: "1: z" < "2: a".
out := countResults(lines) out := countResults([]string{"h1 z", "h2 a", "h3 a"})
// lexicographic sort: "1: z" < "2: a"
if out[0] != "1: z" || out[1] != "2: a" { if out[0] != "1: z" || out[1] != "2: a" {
t.Fatalf("unexpected order: %v", out) t.Fatalf("unexpected order: %v", out)
} }
@@ -147,13 +354,11 @@ func TestCountResults_Sorted(t *testing.T) {
// ---- queryPuppetDB ---------------------------------------------------------- // ---- queryPuppetDB ----------------------------------------------------------
func TestQueryPuppetDB_Success(t *testing.T) { func TestQueryPuppetDB_Success(t *testing.T) {
want := []fact{ want := []fact{{Certname: "node1", Name: "enc_role", Value: rawJSON("roles::dns")}}
{Certname: "node1", Name: "enc_role", Value: rawJSON("roles::dns")},
}
srv := newTestServer(t, want) srv := newTestServer(t, want)
defer srv.Close() defer srv.Close()
got, err := queryPuppetDB(srv.URL+"/pdb/query/v4/facts", `["=","name","enc_role"]`) got, err := queryPuppetDB(srv.URL, `["=","name","enc_role"]`)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -162,14 +367,27 @@ func TestQueryPuppetDB_Success(t *testing.T) {
} }
} }
func TestQueryPuppetDB_SendsQueryParam(t *testing.T) {
var got string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
got = r.URL.Query().Get("query")
_ = json.NewEncoder(w).Encode([]fact{})
}))
defer srv.Close()
_, _ = queryPuppetDB(srv.URL, `["=","name","enc_role"]`)
if got != `["=","name","enc_role"]` {
t.Fatalf("query param not forwarded correctly, got %q", got)
}
}
func TestQueryPuppetDB_HTTPError(t *testing.T) { func TestQueryPuppetDB_HTTPError(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Error(w, "not found", http.StatusNotFound) http.Error(w, "not found", http.StatusNotFound)
})) }))
defer srv.Close() defer srv.Close()
_, err := queryPuppetDB(srv.URL+"/pdb/query/v4/facts", `["=","name","enc_role"]`) if _, err := queryPuppetDB(srv.URL, `[]`); err == nil {
if err == nil {
t.Fatal("expected error for 404") t.Fatal("expected error for 404")
} }
} }
@@ -180,15 +398,13 @@ func TestQueryPuppetDB_BadJSON(t *testing.T) {
})) }))
defer srv.Close() defer srv.Close()
_, err := queryPuppetDB(srv.URL+"/pdb/query/v4/facts", `["=","name","enc_role"]`) if _, err := queryPuppetDB(srv.URL, `[]`); err == nil {
if err == nil {
t.Fatal("expected decode error") t.Fatal("expected decode error")
} }
} }
func TestQueryPuppetDB_ConnectionRefused(t *testing.T) { func TestQueryPuppetDB_ConnectionRefused(t *testing.T) {
_, err := queryPuppetDB("http://127.0.0.1:1/facts", `["=","name","enc_role"]`) if _, err := queryPuppetDB("http://127.0.0.1:1/facts", `[]`); err == nil {
if err == nil {
t.Fatal("expected connection error") t.Fatal("expected connection error")
} }
} }
@@ -210,6 +426,22 @@ func TestLoadConfig_Defaults(t *testing.T) {
if cfg.RoleFact != defaultRoleFact { if cfg.RoleFact != defaultRoleFact {
t.Fatalf("expected default role fact, got %s", cfg.RoleFact) t.Fatalf("expected default role fact, got %s", cfg.RoleFact)
} }
if cfg.Domain != defaultDomain {
t.Fatalf("expected default domain, got %s", cfg.Domain)
}
}
func TestLoadConfig_DomainEnvOverride(t *testing.T) {
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
t.Setenv("NODE_LOOKUP_DOMAIN", "example.com")
cfg, err := loadConfig()
if err != nil {
t.Fatal(err)
}
if cfg.Domain != "example.com" {
t.Fatalf("domain env override failed: %s", cfg.Domain)
}
} }
func TestLoadConfig_EnvOverride(t *testing.T) { func TestLoadConfig_EnvOverride(t *testing.T) {
@@ -292,8 +524,7 @@ func TestLoadConfig_InvalidYAML(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
_, err := loadConfig() if _, err := loadConfig(); err == nil {
if err == nil {
t.Fatal("expected error for invalid YAML") t.Fatal("expected error for invalid YAML")
} }
} }
@@ -305,9 +536,7 @@ func TestWriteDefaultConfig(t *testing.T) {
if err := writeDefaultConfig(); err != nil { if err := writeDefaultConfig(); err != nil {
t.Fatal(err) t.Fatal(err)
} }
data, err := os.ReadFile(filepath.Join(dir, appName, configFileName))
path := filepath.Join(dir, appName, configFileName)
data, err := os.ReadFile(path)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -323,15 +552,14 @@ func TestWriteDefaultConfig_AlreadyExists(t *testing.T) {
if err := writeDefaultConfig(); err != nil { if err := writeDefaultConfig(); err != nil {
t.Fatal(err) t.Fatal(err)
} }
err := writeDefaultConfig() if err := writeDefaultConfig(); err == nil {
if err == nil {
t.Fatal("expected error when config already exists") t.Fatal("expected error when config already exists")
} }
} }
// ---- allFactsForNode -------------------------------------------------------- // ---- allFactsForNode --------------------------------------------------------
func TestAllFactsForNode_ReturnsSortedFacts(t *testing.T) { func TestAllFactsForNode_ReturnsFacts(t *testing.T) {
facts := []fact{ facts := []fact{
{Certname: "node1", Name: "zebra", Value: rawJSON("z-val")}, {Certname: "node1", Name: "zebra", Value: rawJSON("z-val")},
{Certname: "node1", Name: "alpha", Value: rawJSON("a-val")}, {Certname: "node1", Name: "alpha", Value: rawJSON("a-val")},
@@ -340,7 +568,7 @@ func TestAllFactsForNode_ReturnsSortedFacts(t *testing.T) {
srv := newTestServer(t, facts) srv := newTestServer(t, facts)
defer srv.Close() defer srv.Close()
got, err := allFactsForNode(srv.URL+"/pdb/query/v4/facts", "node1") got, err := allFactsForNode(srv.URL, "node1")
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -353,12 +581,11 @@ func TestAllFactsForNode_QueryContainsCertname(t *testing.T) {
var receivedQuery string var receivedQuery string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
receivedQuery = r.URL.Query().Get("query") receivedQuery = r.URL.Query().Get("query")
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode([]fact{}) _ = json.NewEncoder(w).Encode([]fact{})
})) }))
defer srv.Close() defer srv.Close()
_, _ = allFactsForNode(srv.URL+"/pdb/query/v4/facts", "mynode.example.com") _, _ = allFactsForNode(srv.URL, "mynode.example.com")
if !strings.Contains(receivedQuery, "mynode.example.com") { if !strings.Contains(receivedQuery, "mynode.example.com") {
t.Fatalf("expected certname in query, got: %s", receivedQuery) t.Fatalf("expected certname in query, got: %s", receivedQuery)
} }
@@ -373,59 +600,275 @@ func TestAllFactsForNode_HTTPError(t *testing.T) {
})) }))
defer srv.Close() defer srv.Close()
_, err := allFactsForNode(srv.URL+"/pdb/query/v4/facts", "node1") if _, err := allFactsForNode(srv.URL, "node1"); err == nil {
if err == nil {
t.Fatal("expected error for HTTP 500") t.Fatal("expected error for HTTP 500")
} }
} }
// ---- run with -a flag ------------------------------------------------------- // ---- stdinReader (no-TTY behavior) ------------------------------------------
func TestStdinReader_CharDeviceFallsThrough(t *testing.T) {
// /dev/null is a character device — the same shape stdin has when the tool
// is invoked by an agent/CI without a TTY. It must NOT be treated as input.
f, err := os.Open(os.DevNull)
if err != nil {
t.Fatal(err)
}
defer func() { _ = f.Close() }()
if _, ok := stdinReader(f); ok {
t.Fatal("expected /dev/null to be treated as no stdin data")
}
}
func TestStdinReader_EmptyPipeFallsThrough(t *testing.T) {
// A closed, empty pipe (e.g. `true | node-lookup`) must fall through to a
// normal query rather than consuming empty input and printing nothing.
r, w, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
_ = w.Close() // no data written; reader sees immediate EOF
defer func() { _ = r.Close() }()
if _, ok := stdinReader(r); ok {
t.Fatal("expected empty pipe to be treated as no stdin data")
}
}
func TestStdinReader_PipeWithDataIsRead(t *testing.T) {
r, w, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
go func() {
_, _ = io.WriteString(w, "node1\nnode2\n")
_ = w.Close()
}()
defer func() { _ = r.Close() }()
reader, ok := stdinReader(r)
if !ok {
t.Fatal("expected pipe with data to be treated as stdin input")
}
got, _ := reader.ReadString('\n')
if strings.TrimSpace(got) != "node1" {
t.Fatalf("expected first line 'node1', got %q", got)
}
}
func TestStdinReader_RegularFileWithData(t *testing.T) {
path := filepath.Join(t.TempDir(), "nodes.txt")
if err := os.WriteFile(path, []byte("host1\nhost2\n"), 0o644); err != nil {
t.Fatal(err)
}
f, err := os.Open(path)
if err != nil {
t.Fatal(err)
}
defer func() { _ = f.Close() }()
if _, ok := stdinReader(f); !ok {
t.Fatal("expected redirected file with data to be treated as stdin input")
}
}
func TestStdinReader_EmptyFileFallsThrough(t *testing.T) {
path := filepath.Join(t.TempDir(), "empty.txt")
if err := os.WriteFile(path, nil, 0o644); err != nil {
t.Fatal(err)
}
f, err := os.Open(path)
if err != nil {
t.Fatal(err)
}
defer func() { _ = f.Close() }()
if _, ok := stdinReader(f); ok {
t.Fatal("expected empty file to be treated as no stdin data")
}
}
// ---- run: validation --------------------------------------------------------
func TestRun_AllFacts_RequiresNode(t *testing.T) { func TestRun_AllFacts_RequiresNode(t *testing.T) {
cfg := config{PuppetDBURL: "http://unused", RoleFact: "enc_role"} cfg := config{PuppetDBURL: "http://unused", RoleFact: "enc_role"}
err := run(cfg, "", "", "", "", false, false, false, false, false, false, true) err := run(cfg, "", "", "", false, false, false, false, false, false, false, false, true)
if err == nil || !strings.Contains(err.Error(), "-a requires -n") { if err == nil || !strings.Contains(err.Error(), "-a requires -n") {
t.Fatalf("expected -a requires -n error, got: %v", err) t.Fatalf("expected -a requires -n error, got: %v", err)
} }
} }
func TestRun_OutputFlagsRequireRoleOrFact(t *testing.T) {
cfg := config{PuppetDBURL: "http://unused", RoleFact: "enc_role"}
// -1 (nodeOnly) with neither -R nor -F must error before any HTTP call.
err := run(cfg, "", "", "", false, false, false, true, false, false, false, false, false)
if err == nil || !strings.Contains(err.Error(), "-R or -F") {
t.Fatalf("expected -R/-F requirement error, got: %v", err)
}
}
func TestRun_MatchRequiresRoleOrFact(t *testing.T) {
cfg := config{PuppetDBURL: "http://unused", RoleFact: "enc_role"}
// -m with neither -R nor -F must error before any HTTP call.
err := run(cfg, "", "", "someval", false, false, false, false, false, false, false, false, false)
if err == nil || !strings.Contains(err.Error(), "-R or -F") {
t.Fatalf("expected -R/-F requirement error, got: %v", err)
}
}
// ---- run: output modes ------------------------------------------------------
type runArgs struct {
cfg config
nodeName, factName, match string
showRole, partial, inverse, nodeOnly, valueOnly, count bool
ansible, jsonMode, allFacts bool
}
// runToString invokes run against a mock PuppetDB returning facts, capturing
// stdout. nodeName defaults to "n1" so the stdin path is skipped.
func runToString(t *testing.T, facts []fact, mutate func(*runArgs)) string {
t.Helper()
srv := newTestServer(t, facts)
t.Cleanup(srv.Close)
a := runArgs{
cfg: config{PuppetDBURL: srv.URL, RoleFact: "enc_role"},
nodeName: "n1",
showRole: true,
}
if mutate != nil {
mutate(&a)
}
return captureStdout(t, func() {
if err := run(a.cfg, a.nodeName, a.factName, a.match, a.showRole, a.partial, a.inverse, a.nodeOnly, a.valueOnly, a.count, a.ansible, a.jsonMode, a.allFacts); err != nil {
t.Fatalf("run returned error: %v", err)
}
})
}
func TestRun_DefaultOutput(t *testing.T) {
facts := []fact{
{Certname: "hostb", Name: "enc_role", Value: rawJSON("roles::web")},
{Certname: "hosta", Name: "enc_role", Value: rawJSON("roles::db")},
}
out := strings.TrimSpace(runToString(t, facts, nil))
lines := strings.Split(out, "\n")
if len(lines) != 2 || lines[0] != "hosta roles::db" || lines[1] != "hostb roles::web" {
t.Fatalf("unexpected default output: %q", out)
}
}
func TestRun_NodeOnly(t *testing.T) {
facts := []fact{
{Certname: "hosta", Name: "enc_role", Value: rawJSON("roles::db")},
{Certname: "hostb", Name: "enc_role", Value: rawJSON("roles::web")},
}
out := strings.TrimSpace(runToString(t, facts, func(a *runArgs) { a.nodeOnly = true }))
if out != "hosta\nhostb" {
t.Fatalf("unexpected -1 output: %q", out)
}
}
func TestRun_ValueOnly(t *testing.T) {
facts := []fact{
{Certname: "hosta", Name: "enc_role", Value: rawJSON("roles::db")},
{Certname: "hostb", Name: "enc_role", Value: rawJSON("roles::web")},
}
out := strings.TrimSpace(runToString(t, facts, func(a *runArgs) { a.valueOnly = true }))
if out != "roles::db\nroles::web" {
t.Fatalf("unexpected -2 output: %q", out)
}
}
func TestRun_Count(t *testing.T) {
facts := []fact{
{Certname: "hosta", Name: "enc_role", Value: rawJSON("roles::web")},
{Certname: "hostb", Name: "enc_role", Value: rawJSON("roles::web")},
{Certname: "hostc", Name: "enc_role", Value: rawJSON("roles::db")},
}
out := strings.TrimSpace(runToString(t, facts, func(a *runArgs) { a.count = true }))
if !strings.Contains(out, "2: roles::web") || !strings.Contains(out, "1: roles::db") {
t.Fatalf("unexpected -C output: %q", out)
}
}
func TestRun_JSON(t *testing.T) {
facts := []fact{{Certname: "hosta", Name: "enc_role", Value: rawJSON("roles::db")}}
out := runToString(t, facts, func(a *runArgs) { a.jsonMode = true })
var parsed map[string]map[string]interface{}
if err := json.Unmarshal([]byte(out), &parsed); err != nil {
t.Fatalf("output is not valid JSON: %v (%s)", err, out)
}
if parsed["hosta"]["enc_role"] != "roles::db" {
t.Fatalf("unexpected JSON structure: %v", parsed)
}
}
func TestRun_Ansible(t *testing.T) {
facts := []fact{
{Certname: "hosta", Name: "enc_role", Value: rawJSON("roles::db")},
{Certname: "hostb", Name: "enc_role", Value: rawJSON("roles::web")},
}
out := runToString(t, facts, func(a *runArgs) { a.ansible = true })
if !strings.Contains(out, "all:") || !strings.Contains(out, "hosts:") {
t.Fatalf("expected Ansible inventory structure, got: %q", out)
}
if !strings.Contains(out, "hosta:") || !strings.Contains(out, "hostb:") {
t.Fatalf("expected both hosts in inventory, got: %q", out)
}
// The queried fact is attached as a host var.
if !strings.Contains(out, "enc_role: roles::db") || !strings.Contains(out, "enc_role: roles::web") {
t.Fatalf("expected fact host vars in inventory, got: %q", out)
}
}
func TestRun_Ansible_MultipleFacts(t *testing.T) {
// -F ipaddress,enc_role -A must include both facts as host vars.
facts := []fact{
{Certname: "hosta", Name: "ipaddress", Value: rawJSON("198.18.0.1")},
{Certname: "hosta", Name: "enc_role", Value: rawJSON("roles::dns")},
}
out := runToString(t, facts, func(a *runArgs) {
a.showRole = false
a.factName = "ipaddress,enc_role"
a.ansible = true
})
// Parse it back as YAML and assert the structure precisely.
var inv struct {
All struct {
Hosts map[string]map[string]interface{} `yaml:"hosts"`
} `yaml:"all"`
}
if err := yaml.Unmarshal([]byte(out), &inv); err != nil {
t.Fatalf("inventory is not valid YAML: %v (%s)", err, out)
}
host := inv.All.Hosts["hosta"]
if host["ipaddress"] != "198.18.0.1" || host["enc_role"] != "roles::dns" {
t.Fatalf("expected both facts as host vars, got: %v", host)
}
}
func TestRun_AllFacts_PrintsSortedByName(t *testing.T) { func TestRun_AllFacts_PrintsSortedByName(t *testing.T) {
facts := []fact{ facts := []fact{
{Certname: "node1", Name: "zzz_fact", Value: rawJSON("last")}, {Certname: "node1", Name: "zzz_fact", Value: rawJSON("last")},
{Certname: "node1", Name: "aaa_fact", Value: rawJSON("first")}, {Certname: "node1", Name: "aaa_fact", Value: rawJSON("first")},
{Certname: "node1", Name: "mmm_fact", Value: rawJSON(true)}, {Certname: "node1", Name: "mmm_fact", Value: rawJSON(true)},
} }
srv := newTestServer(t, facts) out := strings.TrimSpace(runToString(t, facts, func(a *runArgs) {
defer srv.Close() a.showRole = false
a.allFacts = true
old := os.Stdout }))
r, w, _ := os.Pipe() lines := strings.Split(out, "\n")
os.Stdout = w
cfg := config{PuppetDBURL: srv.URL + "/pdb/query/v4/facts", RoleFact: "enc_role"}
err := run(cfg, "node1", "", "", "", false, false, false, false, false, false, true)
_ = w.Close()
os.Stdout = old
var buf strings.Builder
_, _ = io.Copy(&buf, r)
out := buf.String()
if err != nil {
t.Fatal(err)
}
lines := strings.Split(strings.TrimSpace(out), "\n")
if len(lines) != 3 { if len(lines) != 3 {
t.Fatalf("expected 3 lines, got %d: %q", len(lines), out) t.Fatalf("expected 3 lines, got %d: %q", len(lines), out)
} }
if !strings.HasPrefix(lines[0], "aaa_fact") { if !strings.HasPrefix(lines[0], "aaa_fact") ||
t.Errorf("first line should be aaa_fact, got: %s", lines[0]) !strings.HasPrefix(lines[1], "mmm_fact") ||
} !strings.HasPrefix(lines[2], "zzz_fact") {
if !strings.HasPrefix(lines[1], "mmm_fact") { t.Fatalf("facts not sorted by name: %v", lines)
t.Errorf("second line should be mmm_fact, got: %s", lines[1])
}
if !strings.HasPrefix(lines[2], "zzz_fact") {
t.Errorf("third line should be zzz_fact, got: %s", lines[2])
} }
} }
+82
View File
@@ -0,0 +1,82 @@
---
# nfpm config for building the node-lookup RPM.
# Rendered through envsubst (see scripts/build-rpm.sh) then fed to `nfpm pkg`.
name: ${PACKAGE_NAME}
version: ${PACKAGE_VERSION}
release: ${PACKAGE_RELEASE}
arch: ${PACKAGE_ARCH}
platform: ${PACKAGE_PLATFORM}
section: default
priority: extra
description: "${PACKAGE_DESCRIPTION}"
maintainer: ${PACKAGE_MAINTAINER}
homepage: ${PACKAGE_HOMEPAGE}
license: ${PACKAGE_LICENSE}
disable_globbing: false
replaces:
- node-lookup
provides:
- node-lookup
contents:
# The CLI binaries: node-lookup and its companion tools.
- src: dist/node-lookup
dst: /usr/bin/node-lookup
file_info:
mode: 0755
owner: root
group: root
- src: dist/pburl
dst: /usr/bin/pburl
file_info:
mode: 0755
owner: root
group: root
- src: dist/pblastreport
dst: /usr/bin/pblastreport
file_info:
mode: 0755
owner: root
group: root
# Shell completions (generated by scripts/build-rpm.sh before packaging).
- src: dist/completions/node-lookup.bash
dst: /usr/share/bash-completion/completions/node-lookup
file_info:
mode: 0644
- src: dist/completions/_node-lookup
dst: /usr/share/zsh/site-functions/_node-lookup
file_info:
mode: 0644
- src: dist/completions/node-lookup.fish
dst: /usr/share/fish/vendor_completions.d/node-lookup.fish
file_info:
mode: 0644
- src: dist/completions/pburl.bash
dst: /usr/share/bash-completion/completions/pburl
file_info:
mode: 0644
- src: dist/completions/_pburl
dst: /usr/share/zsh/site-functions/_pburl
file_info:
mode: 0644
- src: dist/completions/pburl.fish
dst: /usr/share/fish/vendor_completions.d/pburl.fish
file_info:
mode: 0644
- src: dist/completions/pblastreport.bash
dst: /usr/share/bash-completion/completions/pblastreport
file_info:
mode: 0644
- src: dist/completions/_pblastreport
dst: /usr/share/zsh/site-functions/_pblastreport
file_info:
mode: 0644
- src: dist/completions/pblastreport.fish
dst: /usr/share/fish/vendor_completions.d/pblastreport.fish
file_info:
mode: 0644
+49
View File
@@ -0,0 +1,49 @@
#!/usr/bin/env bash
#
# Package the (already built) node-lookup, pburl and pblastreport binaries into
# an RPM with nfpm, bundling generated bash/zsh/fish shell completions.
# Usage: scripts/build-rpm.sh [version] (version defaults to $CI_COMMIT_TAG)
#
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "${ROOT_DIR}"
VERSION="${1:-${CI_COMMIT_TAG:-0.0.0-dev}}"
VERSION="${VERSION#v}" # strip a leading v
BINARY="node-lookup"
BINARIES=(node-lookup pburl pblastreport)
DIST="dist"
for b in "${BINARIES[@]}"; do
if [ ! -f "${DIST}/${b}" ]; then
echo "ERROR: ${DIST}/${b} not found; run 'make build' first" >&2
exit 1
fi
done
# Generate shell completions from the freshly built binaries so they always
# match the shipped flags/subcommands.
COMP_DIR="${DIST}/completions"
mkdir -p "${COMP_DIR}"
for b in "${BINARIES[@]}"; do
"./${DIST}/${b}" completion bash >"${COMP_DIR}/${b}.bash"
"./${DIST}/${b}" completion zsh >"${COMP_DIR}/_${b}"
"./${DIST}/${b}" completion fish >"${COMP_DIR}/${b}.fish"
done
export PACKAGE_NAME="${BINARY}"
export PACKAGE_VERSION="${VERSION}"
export PACKAGE_RELEASE="1"
export PACKAGE_ARCH="amd64"
export PACKAGE_PLATFORM="linux"
export PACKAGE_DESCRIPTION="CLI tools for PuppetDB: node-lookup (fact lookup/filtering) plus pburl and pblastreport (Puppetboard URLs and last-report times)"
export PACKAGE_MAINTAINER="Ben Vincent <ben@unkin.net>"
export PACKAGE_HOMEPAGE="https://git.unkin.net/unkin/node-lookup"
export PACKAGE_LICENSE="MIT"
envsubst <packaging/nfpm.yaml >"${DIST}/nfpm.yaml"
nfpm pkg --config "${DIST}/nfpm.yaml" --target "${DIST}" --packager rpm
echo "Built:"
ls -1 "${DIST}"/*.rpm