15 Commits

Author SHA1 Message Date
benvin 5c0eb1e899 Merge pull request 'Abort watchpr when a poll can no longer see the PR' (#12) from benvin/watchpr-terminal-errors into main
ci/woodpecker/tag/release Pipeline was successful
Reviewed-on: #12
2026-09-09 23:13:17 +10:00
unkin-agent d04c5aa58d Scope watchpr's terminal 404 to the PR lookup
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Only a 404 from GetPR means the PR is gone. A 404 from any other call
can be a proxy or ingress blip, so it now warns and counts against the
consecutive-failure cap instead of killing the watch on first sight.
2026-09-09 22:55:56 +10:00
benvin 7c6ec361ae Merge pull request 'Accept a bare integer as seconds for watchpr --interval' (#11) from benvin/watchpr-interval-units into main
Reviewed-on: #11
2026-09-09 22:42:24 +10:00
unkin-agent 46dfe48adc Abort watchpr when a poll can no longer see the PR
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
- treat a mid-run 404 on a tracked PR as terminal
- cap consecutive transient poll failures at 20 per PR
- reset the failure count on a successful poll
- export IsNotFound for callers to classify the abort
2026-09-09 22:41:27 +10:00
unkin-agent 71e42811fb Accept a bare integer as seconds for watchpr --interval
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
- add agent.ParseDurationFlag: bare integer means seconds, duration strings still parse
- take --interval as a string and parse it in the command
- reject unparseable and non-positive values with an error naming the flag
2026-09-09 22:38:09 +10:00
benvin 985b58c406 Merge pull request 'Re-mint watchpr's Gitea token when it expires' (#10) from benvin/watchpr-auth-expiry into main
Reviewed-on: #10
2026-09-09 22:34:15 +10:00
unkin-agent 7ef0e28e96 Re-mint watchpr's Gitea token when it expires
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Vault-minted Gitea tokens last ~1h, far less than a watch, and every poll
past expiry 401'd into a warning while watchpr looked healthy.

- retry a rejected request once with a freshly minted token
- abort the watch when the fresh token is rejected too
- poll anonymously when no token can be minted, mint only on a real 401/403
2026-09-09 21:15:46 +10:00
benvin d9645ec5e4 Merge pull request 'Add agentvault seed-oauth for oauth2-proxy credentials' (#9) from benvin/agentvault-seed-oauth into main
ci/woodpecker/tag/release Pipeline was successful
Reviewed-on: #9
2026-08-30 15:41:13 +10:00
unkin-agent 155392a809 Add agentvault seed-oauth for oauth2-proxy credentials
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Seeding an oauth2-proxy secret by hand means an agent shell-plumbing a
client secret and a cookie secret, which the classifier blocks. seed-oauth
does it in one self-contained invocation: it reads the KV path, fills in
only the keys that are missing, preserves everything else and prints key
names and the new version, never a value.

- Add SeedOAuth in internal/agent: read-modify-write of the client_id,
  client_secret and cookie_secret keys with per-key created/kept/rotated
  actions and a no-op when nothing changed.
- Generate secrets from 32 crypto/rand bytes; cookie_secret is base64url so
  it decodes to exactly the 32 bytes oauth2-proxy requires.
- Add ReadKVOptional (missing secret = empty) and WriteKVAny (non-string
  fields survive a round trip) to the KV-v2 client.
- Wire the seed-oauth subcommand and document it in README and AGENTS.md.
- Cover fresh create, patch-preserves-client_secret, other-key
  preservation, --rotate, idempotence, denial errors and secret leakage.
2026-08-30 15:31:25 +10:00
benvin 47118215b4 Merge pull request 'Make agentpr's Vault gitea creds path selectable' (#8) from benvin/agentpr-creds-path into main
ci/woodpecker/tag/release Pipeline was successful
Reviewed-on: #8
2026-08-30 09:36:22 +10:00
unkin-agent 26cd05e961 Make the Vault gitea creds path selectable
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
agentpr always read gitea/creds/unkin-agent from a bare const, so a service
like repospawner could not run it as its own Gitea identity.

- Replace the GiteaCredsPath const with a function: GITEA_CREDS_PATH when set,
  otherwise gitea/creds/<AGENT_LOGIN>. Unset env still resolves to
  gitea/creds/unkin-agent, so existing callers are unchanged.
- Thread the creds path through fetchGiteaToken/readGiteaCreds instead of
  reading a package-level const, and report it in the error messages.
- Make agentpr's help text login-agnostic and document both variables.
2026-08-30 00:50:35 +10:00
benvin 68805a8cde Merge pull request 'Add agentvault with a seed-outpost subcommand' (#7) from benvin/agentvault-seed-outpost into main
ci/woodpecker/tag/release Pipeline was successful
Reviewed-on: #7
2026-08-29 21:58:01 +10:00
unkin-agent 61bb464e32 Add agentvault with a seed-outpost subcommand
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Interactive agents are classifier-blocked from plumbing credentials through
a shell, so seeding an Authentik outpost token into Vault KV needs to happen
inside one binary invocation that never exposes the secret.

- Add cmd/agentvault, a fourth CLI sharing the agentpr Vault AppRole login
  (role_id only, VAULT_ADDR/AGENT_APPROLE_ROLE_ID defaults unchanged).
- Add `agentvault seed-outpost`: read the Authentik API token from
  kv/service/authentik/agent-api-token (field `token`, falling back to
  `api_token`), exact-match the outpost by name via the instances search,
  fetch its key from /api/v3/core/tokens/<identifier>/view_key/ and write it
  to --dest-path under --dest-key.
- Print only the outpost name, token identifier, dest path and new KV
  version; keep secret material out of results, errors and logs.
- Distinguish the failure stages (login, KV read denied, outpost missing,
  view_key, KV write denied) with ErrVaultDenied/ErrVaultNotFound/
  ErrOutpostNotFound sentinels and actionable messages.
- Add internal/agent vaultkv.go (AppRole-authenticated KV-v2 client) and
  authentik.go (outpost search + view_key) for reuse by future flows.
- Cover the happy path, idempotent re-run, field fallback and every failure
  mode with httptest servers, including a leak check on error strings.
- Wire agentvault into the Makefile, build-rpm.sh, nfpm contents, release
  cross-builds/assets, README and AGENTS.md.
2026-08-29 21:03:31 +10:00
benvin 60b08f1198 Merge pull request 'watchpr: fix watch mode never detecting changes' (#5) from benvin/watchpr-watchloop-fix into main
ci/woodpecker/tag/release Pipeline was successful
Reviewed-on: #5
2026-08-15 15:00:01 +10:00
unkin-agent fda3761ead watchpr: fix watch mode hanging when a merged PR's head commit is gone
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
FetchState fetched the PR (merged=true) but then failed the whole state
fetch when CommitStatus 404'd for a head commit that no longer existed
(the branch was deleted after a squash/rebase merge). The merge signal was
discarded, so the watch loop treated every post-merge poll as a transient
error and never exited -- the 37-minute hang seen in production.

- Add a typed APIError carrying the HTTP status so callers can detect a 404
  without parsing error strings.
- FetchState now tolerates a 404 from CommitStatus (commit gone => no CI
  status) and returns the authoritative merged/closed PR state.
- Regression tests: FetchState survives a 404 status; the full watch loop,
  driven through a real client, detects a merge whose head commit is gone
  (both fail/hang before the fix).
2026-08-15 13:23:25 +10:00
26 changed files with 3132 additions and 92 deletions
+3 -3
View File
@@ -17,7 +17,7 @@ steps:
memory: 2Gi memory: 2Gi
cpu: 2 cpu: 2
# Build both binaries into dist/ (consumed by the RPM step) plus the # Build every binary into dist/ (consumed by the RPM step) plus the
# cross-platform binaries attached to the Gitea release. Each tool is a # cross-platform binaries attached to the Gitea release. Each tool is a
# separate main package, so they are built individually per os/arch. # separate main package, so they are built individually per os/arch.
- name: build - name: build
@@ -28,7 +28,7 @@ steps:
# for the shell instead of substituting them (as pipeline vars) at parse # for the shell instead of substituting them (as pipeline vars) at parse
# time. ${CI_COMMIT_TAG} is a real Woodpecker var and stays single-$. # time. ${CI_COMMIT_TAG} is a real Woodpecker var and stays single-$.
- | - |
for entry in "agentpr:./cmd/agentpr" "watchpr:./cmd/watchpr" "agentws:./cmd/agentws"; do for entry in "agentpr:./cmd/agentpr" "watchpr:./cmd/watchpr" "agentws:./cmd/agentws" "agentvault:./cmd/agentvault"; do
name="$${entry%%:*}"; pkg="$${entry##*:}" name="$${entry%%:*}"; pkg="$${entry##*:}"
for osarch in linux/amd64 linux/arm64 darwin/amd64 darwin/arm64; do for osarch in linux/amd64 linux/arm64 darwin/amd64 darwin/arm64; do
os="$${osarch%/*}"; arch="$${osarch#*/}" os="$${osarch%/*}"; arch="$${osarch#*/}"
@@ -135,7 +135,7 @@ steps:
# root; the package step writes the RPM to dist/. Generate a checksums # root; the package step writes the RPM to dist/. Generate a checksums
# manifest over everything we attach so downloads can be verified. # manifest over everything we attach so downloads can be verified.
RPM=$$(ls dist/*.rpm 2>/dev/null | head -1) RPM=$$(ls dist/*.rpm 2>/dev/null | head -1)
ASSETS="agentpr-linux-amd64 agentpr-linux-arm64 agentpr-darwin-amd64 agentpr-darwin-arm64 watchpr-linux-amd64 watchpr-linux-arm64 watchpr-darwin-amd64 watchpr-darwin-arm64 agentws-linux-amd64 agentws-linux-arm64 agentws-darwin-amd64 agentws-darwin-arm64" ASSETS="agentpr-linux-amd64 agentpr-linux-arm64 agentpr-darwin-amd64 agentpr-darwin-arm64 watchpr-linux-amd64 watchpr-linux-arm64 watchpr-darwin-amd64 watchpr-darwin-arm64 agentws-linux-amd64 agentws-linux-arm64 agentws-darwin-amd64 agentws-darwin-arm64 agentvault-linux-amd64 agentvault-linux-arm64 agentvault-darwin-amd64 agentvault-darwin-arm64"
[ -n "$$RPM" ] && ASSETS="$$ASSETS $$RPM" [ -n "$$RPM" ] && ASSETS="$$ASSETS $$RPM"
sha256sum $$ASSETS > sha256sums.txt sha256sum $$ASSETS > sha256sums.txt
tea releases assets create "${CI_COMMIT_TAG}" $$ASSETS sha256sums.txt \ tea releases assets create "${CI_COMMIT_TAG}" $$ASSETS sha256sums.txt \
+72 -11
View File
@@ -3,8 +3,10 @@
## Project Overview ## Project Overview
This repo ships several Gitea-automation CLIs in one RPM (`agent-tools`). They This repo ships several Gitea-automation CLIs in one RPM (`agent-tools`). They
act as the `unkin-agent` user by minting a scoped Gitea token from Vault, so act as an agent user (`unkin-agent` by default) by minting a scoped Gitea token
actions are attributed to the agent rather than to whoever runs the tool. from Vault, so actions are attributed to the agent rather than to whoever runs
the tool. Setting `AGENT_LOGIN` selects a different agent identity, so a service
like repospawner can run these tools as itself.
- **`agentpr`** — create pull requests and post PR comments as `unkin-agent` - **`agentpr`** — create pull requests and post PR comments as `unkin-agent`
(fixes the "tea posts as Ben" attribution problem). Subcommands: (fixes the "tea posts as Ben" attribution problem). Subcommands:
@@ -29,13 +31,18 @@ parsing, watch-state comparison, git worktree helpers).
cmd/agentpr/main.go # agentpr CLI (pr create / pr comment / whoami) cmd/agentpr/main.go # agentpr CLI (pr create / pr comment / whoami)
cmd/watchpr/main.go # watchpr CLI (poll + meaningful-change exit) cmd/watchpr/main.go # watchpr CLI (poll + meaningful-change exit)
cmd/agentws/main.go # agentws CLI (new / list / rm / clean / token / credential) cmd/agentws/main.go # agentws CLI (new / list / rm / clean / token / credential)
cmd/agentvault/main.go # agentvault CLI (seed-outpost / seed-oauth)
internal/agent/ # shared plumbing: internal/agent/ # shared plumbing:
token.go # env config + in-process Gitea-token cache token.go # env config + in-process Gitea-token cache
vault.go # AppRole login + read gitea/creds/unkin-agent vault.go # AppRole login + read the gitea creds path
gitea.go # Gitea REST client (PR create/get, comments, status, whoami) gitea.go # Gitea REST client (PR create/get, comments, status, whoami)
parse.go # owner/repo#N and owner/repo parsing parse.go # owner/repo#N and owner/repo parsing
watch.go # PRState snapshot + MeaningfulChange comparison watch.go # PRState snapshot + MeaningfulChange comparison
git.go # git worktree/clone/fetch helpers (os/exec, no go-git) git.go # git worktree/clone/fetch helpers (os/exec, no go-git)
vaultkv.go # AppRole-authenticated Vault client + KV-v2 read/write
authentik.go # Authentik REST client (outpost search, token view_key)
seedoutpost.go # seed-outpost flow (Authentik token -> Vault KV)
seedoauth.go # seed-oauth flow (oauth2-proxy credential set in Vault KV)
go.mod # module git.unkin.net/unkin/agent-tools go.mod # module git.unkin.net/unkin/agent-tools
Makefile # build / test / lint / completions / rpm / version-bump Makefile # build / test / lint / completions / rpm / version-bump
packaging/nfpm.yaml # nfpm spec (envsubst-templated) for the RPM (all binaries) packaging/nfpm.yaml # nfpm spec (envsubst-templated) for the RPM (all binaries)
@@ -53,7 +60,9 @@ All tools call `agent.GiteaToken()`, which (once per process):
1. AppRole login: `POST $VAULT_ADDR/v1/auth/approle/login` with `role_id` only 1. AppRole login: `POST $VAULT_ADDR/v1/auth/approle/login` with `role_id` only
(no `secret_id`) → `client_token`. (no `secret_id`) → `client_token`.
2. `GET $VAULT_ADDR/v1/gitea/creds/unkin-agent` with `X-Vault-Token``.data.token`. 2. `GET $VAULT_ADDR/v1/<creds path>` with `X-Vault-Token``.data.token`, where
the creds path is `GITEA_CREDS_PATH` if set, else `gitea/creds/$AGENT_LOGIN`
(so unset env still reads `gitea/creds/unkin-agent`).
Config via env (all have defaults): Config via env (all have defaults):
@@ -62,17 +71,19 @@ Config via env (all have defaults):
| `VAULT_ADDR` | `https://vault.service.consul:8200` | Vault/OpenBao address | | `VAULT_ADDR` | `https://vault.service.consul:8200` | Vault/OpenBao address |
| `AGENT_APPROLE_ROLE_ID` | built-in default | AppRole role_id (overridable) | | `AGENT_APPROLE_ROLE_ID` | built-in default | AppRole role_id (overridable) |
| `GITEA_URL` | `https://git.unkin.net` | Gitea base URL | | `GITEA_URL` | `https://git.unkin.net` | Gitea base URL |
| `AGENT_LOGIN` | `unkin-agent` | login whose comments watchpr ignores; agentws git identity | | `AGENT_LOGIN` | `unkin-agent` | agent identity: selects `gitea/creds/<login>`; login whose comments watchpr ignores; agentws git identity |
| `GITEA_CREDS_PATH` | `gitea/creds/$AGENT_LOGIN` | Vault path minting the Gitea token (wins over `AGENT_LOGIN`) |
| `AGENTWS_SRC_ROOT` | `~/src/prodenv` | agentws source-of-truth checkout root | | `AGENTWS_SRC_ROOT` | `~/src/prodenv` | agentws source-of-truth checkout root |
| `AGENTWS_ROOT` | `~/.cache/agentws` | agentws worktree root | | `AGENTWS_ROOT` | `~/.cache/agentws` | agentws worktree root |
| `AGENTWS_OWNER` | `unkin` | Gitea org that owns agentws-managed repos | | `AGENTWS_OWNER` | `unkin` | Gitea org that owns agentws-managed repos |
| `AUTHENTIK_URL` | `https://identity.k8s.syd1.au.unkin.net` | Authentik base URL (`agentvault`) |
### agentws git auth (ephemeral credential helper) ### agentws git auth (ephemeral credential helper)
Gitea tokens are ~1h ephemeral, so `agentws` never bakes one into a remote URL Gitea tokens are ~1h ephemeral, so `agentws` never bakes one into a remote URL
or config. `agentws token` prints a fresh token; `agentws credential get` or config. `agentws token` prints a fresh token; `agentws credential get`
implements the git credential protocol (reads the key=value request on stdin, implements the git credential protocol (reads the key=value request on stdin,
and for the configured Gitea host only emits `username=unkin-agent` + and for the configured Gitea host only emits `username=$AGENT_LOGIN` +
`password=<fresh token>`). `agentws new` wires this per worktree — it enables `password=<fresh token>`). `agentws new` wires this per worktree — it enables
`extensions.worktreeConfig` on the repo once, then writes `user.name`, `extensions.worktreeConfig` on the repo once, then writes `user.name`,
`user.email` and `credential.helper = !<agentws> credential` to the `user.email` and `credential.helper = !<agentws> credential` to the
@@ -85,7 +96,7 @@ to keep the default branch current.
## Build ## Build
```bash ```bash
make build # -> dist/agentpr, dist/watchpr, dist/agentws (CGO disabled, static) make build # -> dist/agentpr, dist/watchpr, dist/agentws, dist/agentvault (CGO disabled, static)
``` ```
Requires Go 1.21+. Dependency: `github.com/spf13/cobra` (CLI). Requires Go 1.21+. Dependency: `github.com/spf13/cobra` (CLI).
@@ -97,8 +108,8 @@ make rpm # build all binaries + package into dist/*.rpm via nfpm
``` ```
`scripts/build-rpm.sh` generates bash/zsh/fish completions from the built `scripts/build-rpm.sh` generates bash/zsh/fish completions from the built
binaries and bundles them alongside `/usr/bin/agentpr`, `/usr/bin/watchpr` and binaries and bundles them alongside `/usr/bin/agentpr`, `/usr/bin/watchpr`,
`/usr/bin/agentws`. `/usr/bin/agentws` and `/usr/bin/agentvault`.
On a `v*` tag the release pipeline builds the RPM and `PUT`s it to the On a `v*` tag the release pipeline builds the RPM and `PUT`s it to the
artifactapi `rpm-internal` repo, then cuts a Gitea release. artifactapi `rpm-internal` repo, then cuts a Gitea release.
@@ -120,10 +131,60 @@ alerting transitions), request-body construction, and the Vault+Gitea client
against `httptest` servers (fake AppRole login + gitea creds + PR create / against `httptest` servers (fake AppRole login + gitea creds + PR create /
comment / whoami / status). No live Vault/Gitea access is required for tests. comment / whoami / status). No live Vault/Gitea access is required for tests.
## agentvault seed-outpost
`agentvault seed-outpost --outpost <name> --dest-path <kv/path>` does the whole
flow in-process:
1. AppRole login (shared `approleLogin`), then KV-v2 read of
`kv/service/authentik/agent-api-token` (field `token`, falling back to
`api_token`).
2. `GET /api/v3/outposts/instances/?search=<name>` — Authentik's `search` is a
substring match, so the exact `name` is re-checked client-side.
3. `GET /api/v3/core/tokens/<token_identifier>/view_key/` for the key.
4. KV-v2 write to `--dest-path` under `--dest-key` (default `token`).
Only the outpost name, token identifier, dest path and new KV version are
printed. Errors are wrapped per stage (login / read denied / outpost missing /
view_key / write denied) via the `ErrVaultDenied`, `ErrVaultNotFound` and
`ErrOutpostNotFound` sentinels.
## agentvault seed-oauth
`agentvault seed-oauth --path <kv/path> --client-id <id>` makes a KV-v2 path
hold a complete oauth2-proxy credential set, in-process:
1. AppRole login (shared `approleLogin`), then a KV-v2 read via
`ReadKVOptional` — a 404 or a deleted version means "empty", not an error,
so the first seed of a path works.
2. Desired keys are computed over the existing map: `client_id` from the flag
(`kept`/`created`/`updated`), `client_secret` and `cookie_secret` generated
from 32 `crypto/rand` bytes only when absent or when `--rotate` is set
(`kept`/`created`/`rotated`). `cookie_secret` is base64url so it decodes to
exactly the 32 bytes oauth2-proxy demands; `client_secret` is standard
base64.
3. Any other key on the path is carried through unchanged (`preserved`), which
is why the write goes through `WriteKVAny` rather than `WriteKV`.
4. The write is skipped entirely when nothing changed; the command then prints
`version: unchanged`.
Only key names, per-key actions and the new KV version are printed. Errors are
wrapped per stage (login / read denied / write denied) via `ErrVaultDenied`.
## Gotchas ## Gotchas
- `watchpr` exits 0 with no output changes on `--once` (just prints state). - `watchpr` exits 0 with no output changes on `--once` (just prints state).
- The token cache is process-wide (`sync.Once`); tests call the unexported - Gitea tokens expire in ~1h, shorter than a watch: the client re-mints once on a
`fetchGiteaToken` to avoid it. 401/403 and replays the request. If the fresh token is rejected too, `watchpr`
exits non-zero rather than polling blind.
- `watchpr` polls anonymously when no token can be minted (public repos work
fine); only a real 401/403 reaches for Vault.
- The token cache is process-wide (mutex-guarded); `RefreshGiteaToken` replaces
it. Tests call the unexported `fetchGiteaToken` to avoid the cache.
- `agentvault` never puts a secret in an error string: Vault decode failures and
Authentik `view_key` responses are reported without their bodies, and
`seed-oauth` reports key names only.
- `--rotate` regenerates the `client_secret` too, which then no longer matches
the IdP provider unless that is rotated alongside.
- CI "combined status" comes from `/commits/{sha}/status`; an empty head SHA - CI "combined status" comes from `/commits/{sha}/status`; an empty head SHA
yields an empty state without an API call. yields an empty state without an API call.
+1 -1
View File
@@ -1,6 +1,6 @@
# All shipped binaries and the package path each is built from. Both tools live # All shipped binaries and the package path each is built from. Both tools live
# under cmd/; the module root ships no binary of its own. # under cmd/; the module root ships no binary of its own.
BINARIES := agentpr watchpr agentws BINARIES := agentpr watchpr agentws agentvault
DIST := dist DIST := dist
VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo dev) VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo dev)
GOFLAGS := -ldflags="-s -w -X main.version=$(VERSION)" GOFLAGS := -ldflags="-s -w -X main.version=$(VERSION)"
+91 -11
View File
@@ -1,21 +1,25 @@
# agent-tools # agent-tools
Small Gitea-automation CLIs, shipped together in one RPM (`agent-tools`). They Small Gitea-automation CLIs, shipped together in one RPM (`agent-tools`). They
act as the **`unkin-agent`** user by minting a scoped Gitea token from Vault, so act as an agent user (**`unkin-agent`** by default) by minting a scoped Gitea
automated PRs, comments and pushes are attributed to the agent — not to whoever token from Vault, so automated PRs, comments and pushes are attributed to the
happens to run the command. agent — not to whoever happens to run the command. Set `AGENT_LOGIN` to act as a
different agent identity.
- **`agentpr`** — create pull requests and post PR comments as `unkin-agent`. - **`agentpr`** — create pull requests and post PR comments as the agent user.
- **`watchpr`** — poll one or more PRs and exit when one changes in a way worth - **`watchpr`** — poll one or more PRs and exit when one changes in a way worth
acting on. acting on.
- **`agentws`** — manage per-branch git worktrees for `unkin-agent`, cloning - **`agentws`** — manage per-branch git worktrees for `unkin-agent`, cloning
into Ben's source checkout and isolating agent work under the XDG cache. into Ben's source checkout and isolating agent work under the XDG cache.
- **`agentvault`** — run deterministic Vault flows in one invocation, so agents
never plumb secret material through a shell.
## How it gets a token ## How it gets a token
On first use each tool performs a Vault AppRole login (`role_id` only, no On first use each tool performs a Vault AppRole login (`role_id` only, no
`secret_id`), then reads `gitea/creds/unkin-agent` to obtain a short-lived Gitea `secret_id`), then reads `gitea/creds/$AGENT_LOGIN` — or `GITEA_CREDS_PATH` when
token, cached in-process for the run. set — to obtain a short-lived Gitea token, cached in-process for the run. With
neither variable set that is `gitea/creds/unkin-agent`, as before.
Everything is configured by environment variables, all with defaults: Everything is configured by environment variables, all with defaults:
@@ -24,15 +28,17 @@ Everything is configured by environment variables, all with defaults:
| `VAULT_ADDR` | `https://vault.service.consul:8200` | Vault/OpenBao address | | `VAULT_ADDR` | `https://vault.service.consul:8200` | Vault/OpenBao address |
| `AGENT_APPROLE_ROLE_ID` | built-in default | AppRole role_id (overridable) | | `AGENT_APPROLE_ROLE_ID` | built-in default | AppRole role_id (overridable) |
| `GITEA_URL` | `https://git.unkin.net` | Gitea base URL | | `GITEA_URL` | `https://git.unkin.net` | Gitea base URL |
| `AGENT_LOGIN` | `unkin-agent` | login whose comments `watchpr` ignores | | `AGENT_LOGIN` | `unkin-agent` | agent identity: selects `gitea/creds/<login>`, and the login whose comments `watchpr` ignores |
| `GITEA_CREDS_PATH` | `gitea/creds/$AGENT_LOGIN` | Vault path minting the Gitea token (wins over `AGENT_LOGIN`) |
| `AGENTWS_SRC_ROOT` | `~/src/prodenv` | source-of-truth checkout root (`agentws`) | | `AGENTWS_SRC_ROOT` | `~/src/prodenv` | source-of-truth checkout root (`agentws`) |
| `AGENTWS_ROOT` | `~/.cache/agentws` | worktree root (`agentws`) | | `AGENTWS_ROOT` | `~/.cache/agentws` | worktree root (`agentws`) |
| `AGENTWS_OWNER` | `unkin` | Gitea org that owns the repos (`agentws`) | | `AGENTWS_OWNER` | `unkin` | Gitea org that owns the repos (`agentws`) |
| `AUTHENTIK_URL` | `https://identity.k8s.syd1.au.unkin.net` | Authentik base URL (`agentvault`) |
## agentpr ## agentpr
```bash ```bash
# Verify identity (should print: unkin-agent) # Verify identity (prints the agent login, unkin-agent by default)
agentpr whoami agentpr whoami
# Open a PR # Open a PR
@@ -64,6 +70,9 @@ watchpr unkin/argocd-apps#42
# Multiple PRs, custom interval; refs accept #N or :N # Multiple PRs, custom interval; refs accept #N or :N
watchpr --interval 30s unkin/argocd-apps#42 unkin/terraform-vault:98 watchpr --interval 30s unkin/argocd-apps#42 unkin/terraform-vault:98
# --interval takes a duration (30s, 2m, 1h30m) or a bare number of seconds
watchpr --interval 30 unkin/argocd-apps#42
# One-shot: print current state and exit 0 (great for scripts) # One-shot: print current state and exit 0 (great for scripts)
watchpr --once unkin/argocd-apps#42 watchpr --once unkin/argocd-apps#42
watchpr --once --json unkin/argocd-apps#42 watchpr --once --json unkin/argocd-apps#42
@@ -98,7 +107,7 @@ agentws rm ~/.cache/agentws/argocd-apps__benvin-my-change --delete-branch
# Remove every managed worktree and prune each source repo # Remove every managed worktree and prune each source repo
agentws clean agentws clean
# Print a fresh unkin-agent Gitea token # Print a fresh Gitea token for the agent login
agentws token agentws token
``` ```
@@ -110,7 +119,7 @@ persists one in a remote URL or in git config. Instead it wires itself as an
- `agentws token` prints a fresh token to stdout (handy for scripts). - `agentws token` prints a fresh token to stdout (handy for scripts).
- `agentws credential get` speaks the git credential protocol on stdin and, for - `agentws credential get` speaks the git credential protocol on stdin and, for
the configured Gitea host only, emits `username=unkin-agent` + the configured Gitea host only, emits `username=$AGENT_LOGIN` +
`password=<fresh token>`. `password=<fresh token>`.
`agentws new` sets this up per worktree without touching the shared checkout: it `agentws new` sets this up per worktree without touching the shared checkout: it
@@ -121,10 +130,81 @@ the **per-worktree** config. Clone/fetch use the same helper via a transient
removal `agentws` fetches in `~/src/prodenv/<repo>` so its default branch stays removal `agentws` fetches in `~/src/prodenv/<repo>` so its default branch stays
current. current.
## agentvault
Deterministic Vault flows, each a single self-contained invocation: the tool
reads and writes the secrets itself, and prints only identifiers.
### seed-outpost
Copy an Authentik outpost's token into Vault KV-v2. `agentvault` reads the
Authentik API token from `kv/service/authentik/agent-api-token`, resolves the
named outpost's `token_identifier`, fetches its key via
`/api/v3/core/tokens/<identifier>/view_key/` and writes it to the destination
KV path. The token value is never printed or logged.
```bash
agentvault seed-outpost \
--outpost k8s-outpost \
--dest-path kubernetes/namespace/authentik/default/outpost-token
```
```
outpost: k8s-outpost
token_identifier: ak-outpost-k8s-outpost
dest: kv/kubernetes/namespace/authentik/default/outpost-token
version: 3
```
Re-running is safe: it writes a new KV version. Flags: `--outpost` and
`--dest-path` are required; `--dest-key` (default `token`), `--kv-mount`
(default `kv`), `--token-path` (default `service/authentik/agent-api-token`) and
`--authentik-url` override the rest.
Errors name the failing stage: AppRole login, KV read denied (policy not
applied), outpost not found (terraform not applied), `view_key` failure, or KV
write denied.
### seed-oauth
Make a Vault KV-v2 path hold a complete oauth2-proxy credential set. It is a
read-modify-write: `client_id` is set from the flag, `client_secret` and
`cookie_secret` are generated (32 bytes from `crypto/rand`) only when missing,
every other key on the path is written back untouched, and nothing is written
at all when the secret is already correct. `cookie_secret` is base64url so it
decodes to exactly the 32 bytes oauth2-proxy requires.
```bash
agentvault seed-oauth \
--path kubernetes/namespace/repospawner/default/oauth-credentials \
--client-id 4f1c…
```
```
path: kv/kubernetes/namespace/repospawner/default/oauth-credentials
keys: client_id, client_secret, cookie_secret
client_id: created
client_secret: kept
cookie_secret: created
version: 4
```
That is the common case: the provider's `client_secret` already lives on the
path, so only the missing keys are added. A run with nothing to do prints
`version: unchanged` and issues no write.
Flags: `--path` and `--client-id` are required; `--kv-mount` (default `kv`) and
`--rotate` override the rest. `--rotate` regenerates both secrets — only use it
when the IdP provider's secret is being rotated alongside, since a rotated
`client_secret` no longer matches the provider.
Errors name the failing stage: AppRole login, KV read denied, or KV write
denied. Only key names, actions and the KV version are printed.
## Build & package ## Build & package
```bash ```bash
make build # -> dist/agentpr, dist/watchpr, dist/agentws make build # -> dist/agentpr, dist/watchpr, dist/agentws, dist/agentvault
make test # go test -race ./... make test # go test -race ./...
make rpm # build + package dist/agent-tools-<version>-1.x86_64.rpm make rpm # build + package dist/agent-tools-<version>-1.x86_64.rpm
``` ```
+7 -7
View File
@@ -1,7 +1,7 @@
// Command agentpr manages Gitea pull requests and comments as the unkin-agent // Command agentpr manages Gitea pull requests and comments as an agent user. It
// user. It obtains a scoped Gitea token from Vault (AppRole login, then reads // obtains a scoped Gitea token from Vault (AppRole login, then reads
// gitea/creds/unkin-agent) so actions are attributed to the agent rather than // gitea/creds/<AGENT_LOGIN>, or GITEA_CREDS_PATH when set) so actions are
// to whoever runs the tool. // attributed to that agent rather than to whoever runs the tool.
// //
// agentpr pr create --repo owner/repo --base main --head feature --title T --body B // agentpr pr create --repo owner/repo --base main --head feature --title T --body B
// agentpr pr comment --repo owner/repo --pr 12 --body "..." // agentpr pr comment --repo owner/repo --pr 12 --body "..."
@@ -33,8 +33,8 @@ func main() {
func newRootCmd() *cobra.Command { func newRootCmd() *cobra.Command {
root := &cobra.Command{ root := &cobra.Command{
Use: "agentpr", Use: "agentpr",
Short: "Manage Gitea PRs and comments as the unkin-agent user.", Short: "Manage Gitea PRs and comments as an agent user.",
Long: "agentpr manages Gitea pull requests and comments as unkin-agent, using a\nGitea token minted from Vault (AppRole login + gitea/creds/unkin-agent).", Long: "agentpr manages Gitea pull requests and comments as an agent user, using a\nGitea token minted from Vault (AppRole login + gitea/creds/<AGENT_LOGIN>).\nSet AGENT_LOGIN to act as another agent identity, or GITEA_CREDS_PATH to name\nthe Vault creds path outright.",
Version: version, Version: version,
SilenceUsage: true, SilenceUsage: true,
} }
@@ -146,7 +146,7 @@ func newPRCommentCmd() *cobra.Command {
func newWhoamiCmd() *cobra.Command { func newWhoamiCmd() *cobra.Command {
return &cobra.Command{ return &cobra.Command{
Use: "whoami", Use: "whoami",
Short: "Print the authenticated Gitea login (should be unkin-agent)", Short: "Print the authenticated Gitea login (the identity PRs are opened as)",
SilenceUsage: true, SilenceUsage: true,
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
c, err := client() c, err := client()
+127
View File
@@ -0,0 +1,127 @@
// Command agentvault runs deterministic Vault flows for agents in a single
// invocation, so credentials are never plumbed through a shell. It authenticates
// with the same Vault AppRole as agentpr (role_id only, no secret_id).
//
// agentvault seed-outpost --outpost <name> --dest-path <kv/path>
// agentvault seed-oauth --path <kv/path> --client-id <id>
package main
import (
"fmt"
"os"
"strings"
"git.unkin.net/unkin/agent-tools/internal/agent"
"github.com/spf13/cobra"
)
var version = "dev"
func main() {
if err := newRootCmd().Execute(); err != nil {
os.Exit(1)
}
}
// newRootCmd builds the agentvault command tree. Separated from main so tests
// can execute it against httptest servers.
func newRootCmd() *cobra.Command {
root := &cobra.Command{
Use: "agentvault",
Short: "Run deterministic Vault flows as the agent AppRole.",
Long: "agentvault performs self-contained Vault flows for agents: it logs in with the\nagent AppRole and moves secret material between systems without ever printing it.",
Version: version,
SilenceUsage: true,
}
root.SetVersionTemplate("{{.Version}}\n")
root.AddCommand(newSeedOutpostCmd(), newSeedOAuthCmd(), newVersionCmd())
return root
}
func newSeedOutpostCmd() *cobra.Command {
opts := agent.SeedOutpostOptions{}
cmd := &cobra.Command{
Use: "seed-outpost",
Short: "Copy an Authentik outpost token into Vault KV",
Long: "Read the Authentik API token from Vault KV, resolve the named outpost's\n" +
"token_identifier, fetch its key and write it to a Vault KV path. Re-running\n" +
"writes a new KV version. The token value is never printed or logged.",
SilenceUsage: true,
RunE: func(cmd *cobra.Command, args []string) error {
opts.VaultAddr = agent.VaultAddr()
opts.RoleID = agent.RoleID()
res, err := agent.SeedOutpost(opts)
if err != nil {
return err
}
out := cmd.OutOrStdout()
_, _ = fmt.Fprintf(out, "outpost: %s\n", res.Outpost)
_, _ = fmt.Fprintf(out, "token_identifier: %s\n", res.TokenIdentifier)
_, _ = fmt.Fprintf(out, "dest: %s/%s\n", res.KVMount, res.DestPath)
_, _ = fmt.Fprintf(out, "version: %d\n", res.Version)
return nil
},
}
f := cmd.Flags()
f.StringVar(&opts.Outpost, "outpost", "", "Authentik outpost name (required)")
f.StringVar(&opts.DestPath, "dest-path", "", "KV-v2 path to write the token to, e.g. kubernetes/namespace/authentik/default/outpost-token (required)")
f.StringVar(&opts.DestKey, "dest-key", agent.DefaultDestKey, "Field to write the token under")
f.StringVar(&opts.KVMount, "kv-mount", agent.DefaultKVMount, "KV-v2 mount holding both the API token and the destination")
f.StringVar(&opts.TokenPath, "token-path", agent.DefaultOutpostTokenPath, "KV-v2 path of the Authentik API token")
f.StringVar(&opts.AuthentikURL, "authentik-url", agent.AuthentikURL(), "Authentik base URL")
_ = cmd.MarkFlagRequired("outpost")
_ = cmd.MarkFlagRequired("dest-path")
return cmd
}
func newSeedOAuthCmd() *cobra.Command {
opts := agent.SeedOAuthOptions{}
cmd := &cobra.Command{
Use: "seed-oauth",
Short: "Seed an oauth2-proxy credential set into Vault KV",
Long: "Make a Vault KV-v2 path hold a complete oauth2-proxy credential set: the\n" +
"given client_id, plus a client_secret and a 32-byte cookie_secret that are\n" +
"generated only when missing (or with --rotate). Existing keys are preserved\n" +
"and nothing is written when the secret is already correct. Secret values are\n" +
"never printed or logged.",
SilenceUsage: true,
RunE: func(cmd *cobra.Command, args []string) error {
opts.VaultAddr = agent.VaultAddr()
opts.RoleID = agent.RoleID()
res, err := agent.SeedOAuth(opts)
if err != nil {
return err
}
out := cmd.OutOrStdout()
_, _ = fmt.Fprintf(out, "path: %s/%s\n", res.KVMount, res.Path)
_, _ = fmt.Fprintf(out, "keys: %s\n", strings.Join(res.KeyNames(), ", "))
for _, k := range res.Keys {
_, _ = fmt.Fprintf(out, " %-14s %s\n", k.Name+":", k.Action)
}
if res.Changed {
_, _ = fmt.Fprintf(out, "version: %d\n", res.Version)
} else {
_, _ = fmt.Fprintf(out, "version: unchanged\n")
}
return nil
},
}
f := cmd.Flags()
f.StringVar(&opts.Path, "path", "", "KV-v2 path holding the credentials, e.g. kubernetes/namespace/repospawner/default/oauth-credentials (required)")
f.StringVar(&opts.ClientID, "client-id", "", "OIDC client id to store (required)")
f.StringVar(&opts.KVMount, "kv-mount", agent.DefaultKVMount, "KV-v2 mount holding the path")
f.BoolVar(&opts.Rotate, "rotate", false, "Regenerate client_secret and cookie_secret even when they already exist")
_ = cmd.MarkFlagRequired("path")
_ = cmd.MarkFlagRequired("client-id")
return cmd
}
func newVersionCmd() *cobra.Command {
return &cobra.Command{
Use: "version",
Short: "Print the version",
Run: func(cmd *cobra.Command, args []string) { fmt.Println(version) },
SilenceUsage: true,
}
}
+180
View File
@@ -0,0 +1,180 @@
package main
import (
"bytes"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
const (
apiToken = "ak-api-token-secret"
outpostKey = "outpost-key-secret"
destPath = "kubernetes/namespace/authentik/default/outpost-token"
)
// fakeEstate serves the Vault (approle + KV-v2 read/write) and Authentik
// (outpost search + view_key) endpoints the seed flow needs.
func fakeEstate(t *testing.T) (vaultURL, authentikURL string) {
t.Helper()
vmux := http.NewServeMux()
vmux.HandleFunc("/v1/auth/approle/login", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `{"auth":{"client_token":"s.vaulttoken"}}`)
})
vmux.HandleFunc("/v1/kv/data/service/authentik/agent-api-token", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `{"data":{"data":{"token":"`+apiToken+`"}}}`)
})
vmux.HandleFunc("/v1/kv/data/"+destPath, func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `{"data":{"version":7}}`)
})
vs := httptest.NewServer(vmux)
t.Cleanup(vs.Close)
amux := http.NewServeMux()
amux.HandleFunc("/api/v3/outposts/instances/", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `{"results":[{"pk":"1","name":"k8s-outpost","token_identifier":"ak-outpost-k8s"}]}`)
})
amux.HandleFunc("/api/v3/core/tokens/ak-outpost-k8s/view_key/", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `{"key":"`+outpostKey+`"}`)
})
as := httptest.NewServer(amux)
t.Cleanup(as.Close)
return vs.URL, as.URL
}
// The command prints identifiers and the KV version only — never a secret.
func TestSeedOutpostOutputHasNoSecrets(t *testing.T) {
vaultURL, authentikURL := fakeEstate(t)
t.Setenv("VAULT_ADDR", vaultURL)
t.Setenv("AGENT_APPROLE_ROLE_ID", "role-xyz")
var out bytes.Buffer
cmd := newRootCmd()
cmd.SetOut(&out)
cmd.SetErr(&out)
cmd.SetArgs([]string{
"seed-outpost",
"--outpost", "k8s-outpost",
"--dest-path", destPath,
"--authentik-url", authentikURL,
})
if err := cmd.Execute(); err != nil {
t.Fatalf("Execute: %v", err)
}
got := out.String()
for _, want := range []string{"k8s-outpost", "ak-outpost-k8s", "kv/" + destPath, "version: 7"} {
if !strings.Contains(got, want) {
t.Errorf("output missing %q:\n%s", want, got)
}
}
for _, secret := range []string{apiToken, outpostKey} {
if strings.Contains(got, secret) {
t.Fatalf("output leaks a secret:\n%s", got)
}
}
}
const oauthPath = "kubernetes/namespace/repospawner/default/oauth-credentials"
// fakeOAuthVault serves approle login plus a KV-v2 path that already holds a
// client_secret, and records what gets written back.
func fakeOAuthVault(t *testing.T, existing map[string]string) (vaultURL string, written *map[string]string) {
t.Helper()
writes := map[string]string{}
mux := http.NewServeMux()
mux.HandleFunc("/v1/auth/approle/login", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `{"auth":{"client_token":"s.vaulttoken"}}`)
})
mux.HandleFunc("/v1/kv/data/"+oauthPath, func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodPost {
var body struct {
Data map[string]string `json:"data"`
}
_ = json.NewDecoder(r.Body).Decode(&body)
for k, v := range body.Data {
writes[k] = v
}
_, _ = io.WriteString(w, `{"data":{"version":4}}`)
return
}
payload, _ := json.Marshal(map[string]any{"data": map[string]any{"data": existing}})
_, _ = w.Write(payload)
})
srv := httptest.NewServer(mux)
t.Cleanup(srv.Close)
return srv.URL, &writes
}
// The command prints key names and the KV version only — never a value.
func TestSeedOAuthOutputHasNoSecrets(t *testing.T) {
const existingSecret = "existing-client-secret-value"
vaultURL, written := fakeOAuthVault(t, map[string]string{"client_secret": existingSecret})
t.Setenv("VAULT_ADDR", vaultURL)
t.Setenv("AGENT_APPROLE_ROLE_ID", "role-xyz")
var out bytes.Buffer
cmd := newRootCmd()
cmd.SetOut(&out)
cmd.SetErr(&out)
cmd.SetArgs([]string{"seed-oauth", "--path", oauthPath, "--client-id", "mediamark-client-id"})
if err := cmd.Execute(); err != nil {
t.Fatalf("Execute: %v", err)
}
got := out.String()
for _, want := range []string{"kv/" + oauthPath, "client_id, client_secret, cookie_secret", "client_secret: kept", "cookie_secret: created", "version: 4"} {
if !strings.Contains(got, want) {
t.Errorf("output missing %q:\n%s", want, got)
}
}
for key, value := range *written {
if key == "client_id" {
continue
}
if strings.Contains(got, value) {
t.Fatalf("output leaks the %s value:\n%s", key, got)
}
}
if strings.Contains(got, existingSecret) {
t.Fatalf("output leaks the existing client_secret:\n%s", got)
}
}
func TestSeedOAuthRequiresFlags(t *testing.T) {
for name, args := range map[string][]string{
"no path": {"seed-oauth", "--client-id", "mediamark-client-id"},
"no client-id": {"seed-oauth", "--path", oauthPath},
} {
t.Run(name, func(t *testing.T) {
cmd := newRootCmd()
cmd.SetOut(io.Discard)
cmd.SetErr(io.Discard)
cmd.SetArgs(args)
if err := cmd.Execute(); err == nil {
t.Fatal("Execute() = nil, want a missing-required-flag error")
}
})
}
}
func TestSeedOutpostRequiresFlags(t *testing.T) {
for name, args := range map[string][]string{
"no outpost": {"seed-outpost", "--dest-path", destPath},
"no dest-path": {"seed-outpost", "--outpost", "k8s-outpost"},
} {
t.Run(name, func(t *testing.T) {
cmd := newRootCmd()
cmd.SetOut(io.Discard)
cmd.SetErr(io.Discard)
cmd.SetArgs(args)
if err := cmd.Execute(); err == nil {
t.Fatal("Execute() = nil, want a missing-required-flag error")
}
})
}
}
+21 -9
View File
@@ -7,6 +7,7 @@
// watchpr owner/repo#12 owner/repo:15 // watchpr owner/repo#12 owner/repo:15
// watchpr --once --json owner/repo#12 // watchpr --once --json owner/repo#12
// watchpr --interval 30s owner/repo#12 // watchpr --interval 30s owner/repo#12
// watchpr --interval 30 owner/repo#12
package main package main
import ( import (
@@ -34,7 +35,7 @@ func main() {
// tests can invoke Execute and assert the exit behaviour without spawning a // tests can invoke Execute and assert the exit behaviour without spawning a
// process. // process.
func newRootCmd() *cobra.Command { func newRootCmd() *cobra.Command {
var interval time.Duration var intervalFlag string
var once, jsonMode bool var once, jsonMode bool
root := &cobra.Command{ root := &cobra.Command{
@@ -50,6 +51,10 @@ func newRootCmd() *cobra.Command {
if len(args) == 0 { if len(args) == 0 {
return fmt.Errorf("no PR references given (e.g. owner/repo#12)") return fmt.Errorf("no PR references given (e.g. owner/repo#12)")
} }
interval, err := agent.ParseDurationFlag("interval", intervalFlag)
if err != nil {
return err
}
refs := make([]agent.PRRef, 0, len(args)) refs := make([]agent.PRRef, 0, len(args))
for _, a := range args { for _, a := range args {
ref, err := agent.ParsePRRef(a) ref, err := agent.ParsePRRef(a)
@@ -58,10 +63,7 @@ func newRootCmd() *cobra.Command {
} }
refs = append(refs, ref) refs = append(refs, ref)
} }
c, err := clientFor() c := clientFor()
if err != nil {
return err
}
if once { if once {
return runOnce(c, refs, jsonMode) return runOnce(c, refs, jsonMode)
} }
@@ -71,7 +73,7 @@ func newRootCmd() *cobra.Command {
root.SetVersionTemplate("{{.Version}}\n") root.SetVersionTemplate("{{.Version}}\n")
f := root.Flags() f := root.Flags()
f.DurationVar(&interval, "interval", 60*time.Second, "Polling interval") f.StringVar(&intervalFlag, "interval", "60s", "Polling interval: a duration (30s, 2m, 1h30m) or a bare number of seconds")
f.BoolVar(&once, "once", false, "Check once, print current state, and exit") f.BoolVar(&once, "once", false, "Check once, print current state, and exit")
f.BoolVar(&jsonMode, "json", false, "Emit JSON") f.BoolVar(&jsonMode, "json", false, "Emit JSON")
@@ -84,12 +86,16 @@ func newRootCmd() *cobra.Command {
return root return root
} }
func clientFor() (*agent.GiteaClient, error) { // clientFor builds the Gitea client. Watching public repos works anonymously,
// so an unavailable token is a warning, not a failure; a poll that is actually
// rejected re-mints then.
func clientFor() *agent.GiteaClient {
token, err := agent.GiteaToken() token, err := agent.GiteaToken()
if err != nil { if err != nil {
return nil, err fmt.Fprintf(os.Stderr, "warning: no Gitea token (%v); polling anonymously\n", err)
token = ""
} }
return agent.NewGiteaClient(token), nil return agent.NewGiteaClient(token)
} }
// runOnce fetches and prints the current state of each PR, then exits 0. // runOnce fetches and prints the current state of each PR, then exits 0.
@@ -131,6 +137,12 @@ func runWatch(c *agent.GiteaClient, refs []agent.PRRef, interval time.Duration,
res, err := agent.Watch(c, refs, login, ticker.C, onBaseline, onError) res, err := agent.Watch(c, refs, login, ticker.C, onBaseline, onError)
if err != nil { if err != nil {
if agent.IsAuthError(err) {
return fmt.Errorf("gitea authentication failed after re-minting the token, watch aborted: %w", err)
}
if agent.IsPRGone(err) {
return fmt.Errorf("PR no longer visible (repo deleted, renamed, or made private), watch aborted: %w", err)
}
return err return err
} }
report(res.Ref.String(), res.Reason, res.State, jsonMode) report(res.Ref.String(), res.Reason, res.State, jsonMode)
+77
View File
@@ -2,6 +2,9 @@ package main
import ( import (
"io" "io"
"net/http"
"net/http/httptest"
"strings"
"testing" "testing"
) )
@@ -28,3 +31,77 @@ func TestExecuteNoArgsErrors(t *testing.T) {
t.Fatal("Execute() = nil, want error when no PR references are given") t.Fatal("Execute() = nil, want error when no PR references are given")
} }
} }
// Watching a public repo with no credentials available must work: the failed
// mint is a warning, the poll goes out unauthenticated, and the command exits 0.
func TestOnceRunsAnonymouslyWhenNoTokenIsAvailable(t *testing.T) {
vault := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusForbidden)
}))
defer vault.Close()
authHeaders := 0
mux := http.NewServeMux()
mux.HandleFunc("/api/v1/repos/unkin/repo/pulls/7", func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") != "" {
authHeaders++
}
_, _ = io.WriteString(w, `{"number":7,"state":"open","mergeable":true,"head":{"sha":"cafebabe"}}`)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/commits/cafebabe/status", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `{"state":"success"}`)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/issues/7/comments", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `[]`)
})
gitea := httptest.NewServer(mux)
defer gitea.Close()
t.Setenv("VAULT_ADDR", vault.URL)
t.Setenv("GITEA_URL", gitea.URL)
cmd := newRootCmd()
cmd.SetArgs([]string{"--once", "unkin/repo#7"})
cmd.SetOut(io.Discard)
cmd.SetErr(io.Discard)
if err := cmd.Execute(); err != nil {
t.Fatalf("anonymous --once should succeed without a token: %v", err)
}
if authHeaders != 0 {
t.Errorf("sent %d Authorization headers, want none", authHeaders)
}
}
// A bare integer interval means seconds and must survive flag parsing: the
// command should fail on the missing PR reference, not on the flag value.
func TestExecuteBareIntervalIsSeconds(t *testing.T) {
cmd := newRootCmd()
cmd.SetArgs([]string{"--interval", "15"})
cmd.SetOut(io.Discard)
cmd.SetErr(io.Discard)
err := cmd.Execute()
if err == nil {
t.Fatal("Execute() = nil, want the no-references error")
}
if !strings.Contains(err.Error(), "no PR references given") {
t.Fatalf("Execute() error = %v, want the no-references error", err)
}
}
// An unparseable interval is rejected before any Vault/Gitea call, with an
// error naming the flag and showing valid forms.
func TestExecuteBadIntervalErrors(t *testing.T) {
cmd := newRootCmd()
cmd.SetArgs([]string{"--interval", "soon", "unkin/repo#1"})
cmd.SetOut(io.Discard)
cmd.SetErr(io.Discard)
err := cmd.Execute()
if err == nil {
t.Fatal("Execute() = nil, want error for an unparseable --interval")
}
for _, want := range []string{"--interval", "30s"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("Execute() error %q does not mention %q", err, want)
}
}
}
+93
View File
@@ -0,0 +1,93 @@
package agent
import (
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strings"
)
// ErrOutpostNotFound marks a search that returned no exactly-named outpost.
var ErrOutpostNotFound = errors.New("outpost not found")
// AuthentikClient talks to the Authentik REST API with a bearer API token. The
// internal CA is in the OS trust store, so the default transport suffices.
type AuthentikClient struct {
BaseURL string
Token string
HTTP *http.Client
}
// NewAuthentikClient builds a client for the given Authentik base URL.
func NewAuthentikClient(baseURL, token string) *AuthentikClient {
return &AuthentikClient{BaseURL: strings.TrimRight(baseURL, "/"), Token: token, HTTP: httpClient}
}
// Outpost is the subset of Authentik's outpost object we need.
type Outpost struct {
PK string `json:"pk"`
Name string `json:"name"`
TokenIdentifier string `json:"token_identifier"`
}
// get issues an authenticated GET and decodes into out. Error text never
// includes a successful response body, which may carry key material.
func (c *AuthentikClient) get(path string, out any) error {
req, err := http.NewRequest(http.MethodGet, c.BaseURL+path, nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+c.Token)
req.Header.Set("Accept", "application/json")
resp, err := c.HTTP.Do(req)
if err != nil {
return fmt.Errorf("authentik GET %s: %w", path, err)
}
defer func() { _ = resp.Body.Close() }()
data, _ := io.ReadAll(resp.Body)
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return fmt.Errorf("authentik GET %s: HTTP %d: %s", path, resp.StatusCode, strings.TrimSpace(string(data)))
}
if err := json.Unmarshal(data, out); err != nil {
return fmt.Errorf("authentik GET %s: decoding response: %w", path, err)
}
return nil
}
// FindOutpost searches outpost instances and returns the one whose name matches
// exactly (search is a substring match, so the exact name is re-checked here).
func (c *AuthentikClient) FindOutpost(name string) (Outpost, error) {
var out struct {
Results []Outpost `json:"results"`
}
path := "/api/v3/outposts/instances/?search=" + url.QueryEscape(name)
if err := c.get(path, &out); err != nil {
return Outpost{}, err
}
for _, o := range out.Results {
if o.Name == name {
return o, nil
}
}
return Outpost{}, fmt.Errorf("authentik outpost %q: %w (searched %d result(s))", name, ErrOutpostNotFound, len(out.Results))
}
// TokenKey returns the key behind a token identifier
// (GET /api/v3/core/tokens/<identifier>/view_key/).
func (c *AuthentikClient) TokenKey(identifier string) (string, error) {
var out struct {
Key string `json:"key"`
}
path := "/api/v3/core/tokens/" + url.PathEscape(identifier) + "/view_key/"
if err := c.get(path, &out); err != nil {
return "", err
}
if out.Key == "" {
return "", fmt.Errorf("authentik view_key for %q: response has no key field", identifier)
}
return out.Key, nil
}
+278 -6
View File
@@ -2,14 +2,17 @@ package agent
import ( import (
"encoding/json" "encoding/json"
"errors"
"io" "io"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"strings"
"testing" "testing"
) )
// fakeVault serves the AppRole login and gitea creds endpoints. // fakeVault serves the AppRole login and the gitea creds secret at credsPath
func fakeVault(t *testing.T, wantRoleID, giteaToken string) *httptest.Server { // only, so a read of any other path 404s.
func fakeVault(t *testing.T, wantRoleID, credsPath, giteaToken string) *httptest.Server {
t.Helper() t.Helper()
mux := http.NewServeMux() mux := http.NewServeMux()
mux.HandleFunc("/v1/auth/approle/login", func(w http.ResponseWriter, r *http.Request) { mux.HandleFunc("/v1/auth/approle/login", func(w http.ResponseWriter, r *http.Request) {
@@ -26,7 +29,7 @@ func fakeVault(t *testing.T, wantRoleID, giteaToken string) *httptest.Server {
} }
_, _ = io.WriteString(w, `{"auth":{"client_token":"s.vaulttoken"}}`) _, _ = io.WriteString(w, `{"auth":{"client_token":"s.vaulttoken"}}`)
}) })
mux.HandleFunc("/v1/"+GiteaCredsPath, func(w http.ResponseWriter, r *http.Request) { mux.HandleFunc("/v1/"+credsPath, func(w http.ResponseWriter, r *http.Request) {
if got := r.Header.Get("X-Vault-Token"); got != "s.vaulttoken" { if got := r.Header.Get("X-Vault-Token"); got != "s.vaulttoken" {
t.Errorf("X-Vault-Token = %q, want s.vaulttoken", got) t.Errorf("X-Vault-Token = %q, want s.vaulttoken", got)
} }
@@ -36,10 +39,10 @@ func fakeVault(t *testing.T, wantRoleID, giteaToken string) *httptest.Server {
} }
func TestFetchGiteaToken(t *testing.T) { func TestFetchGiteaToken(t *testing.T) {
srv := fakeVault(t, "role-xyz", "gitea-abc") srv := fakeVault(t, "role-xyz", "gitea/creds/unkin-agent", "gitea-abc")
defer srv.Close() defer srv.Close()
tok, err := fetchGiteaToken(srv.URL, "role-xyz") tok, err := fetchGiteaToken(srv.URL, "role-xyz", "gitea/creds/unkin-agent")
if err != nil { if err != nil {
t.Fatalf("fetchGiteaToken: %v", err) t.Fatalf("fetchGiteaToken: %v", err)
} }
@@ -57,7 +60,7 @@ func TestFetchGiteaTokenLoginError(t *testing.T) {
srv := httptest.NewServer(mux) srv := httptest.NewServer(mux)
defer srv.Close() defer srv.Close()
if _, err := fetchGiteaToken(srv.URL, "role-xyz"); err == nil { if _, err := fetchGiteaToken(srv.URL, "role-xyz", "gitea/creds/unkin-agent"); err == nil {
t.Fatal("expected error on 403 login") t.Fatal("expected error on 403 login")
} }
} }
@@ -163,6 +166,59 @@ func TestFetchState(t *testing.T) {
} }
} }
// A merged PR whose branch was deleted leaves its head commit unreachable, so
// the commit-status endpoint 404s. FetchState must still return the (merged)
// state rather than failing, otherwise the watch loop never sees the merge.
func TestFetchStateToleratesMissingCommit(t *testing.T) {
mux := http.NewServeMux()
mux.HandleFunc("/api/v1/repos/unkin/repo/pulls/7", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `{"number":7,"state":"closed","merged":true,"mergeable":true,"title":"feat","html_url":"u","head":{"sha":"cafebabecafebabe"}}`)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/commits/cafebabecafebabe/status", func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusNotFound)
_, _ = io.WriteString(w, `{"message":"not found"}`)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/issues/7/comments", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `[]`)
})
srv := httptest.NewServer(mux)
defer srv.Close()
c := &GiteaClient{BaseURL: srv.URL, Token: "t", HTTP: srv.Client()}
ref := PRRef{Owner: "unkin", Repo: "repo", Number: 7}
st, err := FetchState(c, ref, "unkin-agent")
if err != nil {
t.Fatalf("FetchState must tolerate a 404 status for a gone commit: %v", err)
}
if !st.Merged || st.State != "closed" {
t.Errorf("merged/state = %t/%q, want true/closed", st.Merged, st.State)
}
if st.CIStatus != "" {
t.Errorf("CIStatus = %q, want empty (no status for a gone commit)", st.CIStatus)
}
}
// A non-404 error from the status endpoint is still fatal: only "commit gone" is
// tolerated, not, say, an auth or server failure.
func TestFetchStateFailsOnNon404StatusError(t *testing.T) {
mux := http.NewServeMux()
mux.HandleFunc("/api/v1/repos/unkin/repo/pulls/7", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `{"number":7,"state":"open","merged":false,"mergeable":true,"head":{"sha":"cafebabecafebabe"}}`)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/commits/cafebabecafebabe/status", func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusInternalServerError)
_, _ = io.WriteString(w, `{"message":"boom"}`)
})
srv := httptest.NewServer(mux)
defer srv.Close()
c := &GiteaClient{BaseURL: srv.URL, Token: "t", HTTP: srv.Client()}
ref := PRRef{Owner: "unkin", Repo: "repo", Number: 7}
if _, err := FetchState(c, ref, "unkin-agent"); err == nil {
t.Fatal("FetchState should surface a 500 from the status endpoint")
}
}
func TestGiteaAPIError(t *testing.T) { func TestGiteaAPIError(t *testing.T) {
mux := http.NewServeMux() mux := http.NewServeMux()
mux.HandleFunc("/api/v1/repos/unkin/repo/pulls", func(w http.ResponseWriter, r *http.Request) { mux.HandleFunc("/api/v1/repos/unkin/repo/pulls", func(w http.ResponseWriter, r *http.Request) {
@@ -177,3 +233,219 @@ func TestGiteaAPIError(t *testing.T) {
t.Fatal("expected error on 422") t.Fatal("expected error on 422")
} }
} }
// expiringGitea serves the PR endpoint, rejecting every token other than
// wantToken with a 401 exactly as Gitea does once a Vault-minted token expires.
// It records the tokens it saw, newest last.
func expiringGitea(t *testing.T, wantToken string, seen *[]string) *httptest.Server {
t.Helper()
mux := http.NewServeMux()
mux.HandleFunc("/api/v1/repos/unkin/repo/pulls/7", func(w http.ResponseWriter, r *http.Request) {
tok := strings.TrimPrefix(r.Header.Get("Authorization"), "token ")
*seen = append(*seen, tok)
if tok != wantToken {
w.WriteHeader(http.StatusUnauthorized)
_, _ = io.WriteString(w, `{"message":"invalid username, password or token"}`)
return
}
_, _ = io.WriteString(w, `{"number":7,"state":"open","mergeable":true,"head":{"sha":"cafebabe"}}`)
})
return httptest.NewServer(mux)
}
// The production failure: the token expired mid-run. The client must re-mint
// once and replay the request with the fresh token.
func TestExpiredTokenIsRemintedAndRetried(t *testing.T) {
var seen []string
srv := expiringGitea(t, "fresh", &seen)
defer srv.Close()
refreshes := 0
c := &GiteaClient{BaseURL: srv.URL, Token: "stale", HTTP: srv.Client(),
Refresh: func() (string, error) { refreshes++; return "fresh", nil }}
pr, err := c.GetPR("unkin/repo", 7)
if err != nil {
t.Fatalf("GetPR after re-mint: %v", err)
}
if pr.Number != 7 {
t.Errorf("PR number = %d, want 7", pr.Number)
}
if refreshes != 1 {
t.Errorf("refreshes = %d, want 1", refreshes)
}
if len(seen) != 2 || seen[0] != "stale" || seen[1] != "fresh" {
t.Errorf("tokens seen = %v, want [stale fresh]", seen)
}
if c.Token != "fresh" {
t.Errorf("client token = %q, want the refreshed token", c.Token)
}
}
// A fresh token that is also rejected is a real auth failure: report it as one
// rather than re-minting forever.
func TestAuthFailureSurvivesRemint(t *testing.T) {
var seen []string
srv := expiringGitea(t, "never-issued", &seen)
defer srv.Close()
refreshes := 0
c := &GiteaClient{BaseURL: srv.URL, Token: "stale", HTTP: srv.Client(),
Refresh: func() (string, error) { refreshes++; return "still-bad", nil }}
_, err := c.GetPR("unkin/repo", 7)
if err == nil {
t.Fatal("GetPR should fail when the fresh token is rejected too")
}
if !IsAuthError(err) {
t.Errorf("IsAuthError(%v) = false, want true", err)
}
if refreshes != 1 {
t.Errorf("refreshes = %d, want 1 (re-mint exactly once)", refreshes)
}
if len(seen) != 2 {
t.Errorf("requests = %d, want 2", len(seen))
}
}
// A refresh that itself fails must surface as an auth error, not as a silent
// success or a bare Vault error.
func TestRemintErrorIsReportedAsAuthFailure(t *testing.T) {
var seen []string
srv := expiringGitea(t, "fresh", &seen)
defer srv.Close()
c := &GiteaClient{BaseURL: srv.URL, Token: "stale", HTTP: srv.Client(),
Refresh: func() (string, error) { return "", errors.New("vault approle login: HTTP 503") }}
_, err := c.GetPR("unkin/repo", 7)
if err == nil || !IsAuthError(err) {
t.Fatalf("GetPR error = %v, want an auth error", err)
}
if !strings.Contains(err.Error(), "vault approle login") {
t.Errorf("error %q should name the re-mint failure", err)
}
if len(seen) != 1 {
t.Errorf("requests = %d, want 1 (no replay without a token)", len(seen))
}
}
// A 5xx is transient, not an auth problem: no re-mint, no retry, and the caller
// keeps its existing retry behaviour.
func TestServerErrorDoesNotRemint(t *testing.T) {
requests := 0
mux := http.NewServeMux()
mux.HandleFunc("/api/v1/repos/unkin/repo/pulls/7", func(w http.ResponseWriter, r *http.Request) {
requests++
w.WriteHeader(http.StatusBadGateway)
})
srv := httptest.NewServer(mux)
defer srv.Close()
refreshes := 0
c := &GiteaClient{BaseURL: srv.URL, Token: "t", HTTP: srv.Client(),
Refresh: func() (string, error) { refreshes++; return "fresh", nil }}
_, err := c.GetPR("unkin/repo", 7)
if err == nil {
t.Fatal("expected error on 502")
}
if IsAuthError(err) {
t.Errorf("502 must not be an auth error")
}
if refreshes != 0 || requests != 1 {
t.Errorf("refreshes = %d, requests = %d, want 0 and 1", refreshes, requests)
}
}
// The replayed request must carry the original body, not an empty one.
func TestRemintReplaysRequestBody(t *testing.T) {
var bodies []CreatePROptions
mux := http.NewServeMux()
mux.HandleFunc("/api/v1/repos/unkin/repo/pulls", func(w http.ResponseWriter, r *http.Request) {
var body CreatePROptions
_ = json.NewDecoder(r.Body).Decode(&body)
bodies = append(bodies, body)
if strings.TrimPrefix(r.Header.Get("Authorization"), "token ") != "fresh" {
w.WriteHeader(http.StatusUnauthorized)
return
}
_, _ = io.WriteString(w, `{"number":7}`)
})
srv := httptest.NewServer(mux)
defer srv.Close()
c := &GiteaClient{BaseURL: srv.URL, Token: "stale", HTTP: srv.Client(),
Refresh: func() (string, error) { return "fresh", nil }}
if _, err := c.CreatePR("unkin/repo", CreatePROptions{Base: "main", Head: "feature", Title: "T", Body: "B"}); err != nil {
t.Fatalf("CreatePR: %v", err)
}
if len(bodies) != 2 {
t.Fatalf("requests = %d, want 2", len(bodies))
}
if bodies[1] != bodies[0] {
t.Errorf("replayed body = %+v, want %+v", bodies[1], bodies[0])
}
}
func TestIsAuthError(t *testing.T) {
tests := []struct {
status int
want bool
}{
{http.StatusUnauthorized, true},
{http.StatusForbidden, true},
{http.StatusNotFound, false},
{http.StatusUnprocessableEntity, false},
{http.StatusBadGateway, false},
}
for _, tt := range tests {
err := error(&APIError{Method: "GET", Path: "/p", StatusCode: tt.status})
if got := IsAuthError(err); got != tt.want {
t.Errorf("IsAuthError(HTTP %d) = %v, want %v", tt.status, got, tt.want)
}
}
if IsAuthError(errors.New("dial tcp: timeout")) {
t.Errorf("a network error is not an auth error")
}
}
// Anonymous polling of a public repo is a supported mode: with no token the
// client must send no Authorization header, and must never reach for Vault.
func TestAnonymousPollingNeverMints(t *testing.T) {
authHeaders := 0
mux := http.NewServeMux()
mux.HandleFunc("/api/v1/repos/unkin/repo/pulls/7", func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") != "" {
authHeaders++
}
_, _ = io.WriteString(w, `{"number":7,"state":"open","mergeable":true,"head":{"sha":"cafebabe"}}`)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/commits/cafebabe/status", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `{"state":"success"}`)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/issues/7/comments", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `[]`)
})
srv := httptest.NewServer(mux)
defer srv.Close()
refreshes := 0
c := &GiteaClient{BaseURL: srv.URL, HTTP: srv.Client(),
Refresh: func() (string, error) { refreshes++; return "", errors.New("vault unreachable") }}
st, err := FetchState(c, PRRef{Owner: "unkin", Repo: "repo", Number: 7}, "unkin-agent")
if err != nil {
t.Fatalf("anonymous FetchState: %v", err)
}
if st.State != "open" || st.CIStatus != "success" || st.HeadSHA != "cafebabe" {
t.Errorf("state = %+v", st)
}
if refreshes != 0 {
t.Errorf("refreshes = %d, want 0 (a 200 must never trigger a mint)", refreshes)
}
if authHeaders != 0 {
t.Errorf("sent %d Authorization headers, want none", authHeaders)
}
}
+62 -6
View File
@@ -3,40 +3,96 @@ package agent
import ( import (
"bytes" "bytes"
"encoding/json" "encoding/json"
"errors"
"fmt" "fmt"
"io" "io"
"net/http" "net/http"
"strings" "strings"
) )
// APIError is a non-2xx response from the Gitea API. It carries the status code
// so callers can react to specific failures (e.g. tolerate a 404 for a commit
// whose branch was deleted after a merge) instead of parsing error strings.
type APIError struct {
Method string
Path string
StatusCode int
Body string
}
func (e *APIError) Error() string {
return fmt.Sprintf("gitea %s %s: HTTP %d: %s", e.Method, e.Path, e.StatusCode, e.Body)
}
// IsNotFound reports whether err is a Gitea 404. Gitea hides repositories a
// caller may not see behind a 404 rather than a 403, so this also covers a repo
// that was renamed, deleted, or made private.
func IsNotFound(err error) bool {
var apiErr *APIError
return errors.As(err, &apiErr) && apiErr.StatusCode == http.StatusNotFound
}
// IsAuthError reports whether err is a Gitea 401/403: the token is expired or
// unauthorised, which retrying the same request cannot fix.
func IsAuthError(err error) bool {
var apiErr *APIError
return errors.As(err, &apiErr) &&
(apiErr.StatusCode == http.StatusUnauthorized || apiErr.StatusCode == http.StatusForbidden)
}
// GiteaClient talks to the Gitea REST API as the agent user. // GiteaClient talks to the Gitea REST API as the agent user.
type GiteaClient struct { type GiteaClient struct {
BaseURL string BaseURL string
Token string Token string
HTTP *http.Client HTTP *http.Client
// Refresh mints a replacement token when the current one is rejected; Vault's
// Gitea tokens expire in ~1h, far short of a watchpr run.
Refresh func() (string, error)
} }
// NewGiteaClient builds a client from the configured base URL and a Vault-minted // NewGiteaClient builds a client from the configured base URL and a Vault-minted
// token. // token, re-minting from Vault when that token expires.
func NewGiteaClient(token string) *GiteaClient { func NewGiteaClient(token string) *GiteaClient {
return &GiteaClient{BaseURL: GiteaURL(), Token: token, HTTP: httpClient} return &GiteaClient{BaseURL: GiteaURL(), Token: token, HTTP: httpClient, Refresh: RefreshGiteaToken}
} }
// do sends the request and, if the token was rejected, re-mints it once and
// replays the request with the fresh token.
func (c *GiteaClient) do(method, path string, body any, out any) error { func (c *GiteaClient) do(method, path string, body any, out any) error {
var reader io.Reader var payload []byte
if body != nil { if body != nil {
b, err := json.Marshal(body) b, err := json.Marshal(body)
if err != nil { if err != nil {
return err return err
} }
reader = bytes.NewReader(b) payload = b
}
err := c.attempt(method, path, payload, out)
if !IsAuthError(err) || c.Refresh == nil {
return err
}
token, refreshErr := c.Refresh()
if refreshErr != nil {
return fmt.Errorf("%w; re-minting token: %v", err, refreshErr)
}
c.Token = token
return c.attempt(method, path, payload, out)
}
func (c *GiteaClient) attempt(method, path string, body []byte, out any) error {
var reader io.Reader
if body != nil {
reader = bytes.NewReader(body)
} }
url := strings.TrimRight(c.BaseURL, "/") + path url := strings.TrimRight(c.BaseURL, "/") + path
req, err := http.NewRequest(method, url, reader) req, err := http.NewRequest(method, url, reader)
if err != nil { if err != nil {
return err return err
} }
req.Header.Set("Authorization", "token "+c.Token) // An empty token means anonymous access, which public repos serve fine.
if c.Token != "" {
req.Header.Set("Authorization", "token "+c.Token)
}
req.Header.Set("Accept", "application/json") req.Header.Set("Accept", "application/json")
if body != nil { if body != nil {
req.Header.Set("Content-Type", "application/json") req.Header.Set("Content-Type", "application/json")
@@ -49,7 +105,7 @@ func (c *GiteaClient) do(method, path string, body any, out any) error {
defer func() { _ = resp.Body.Close() }() defer func() { _ = resp.Body.Close() }()
data, _ := io.ReadAll(resp.Body) data, _ := io.ReadAll(resp.Body)
if resp.StatusCode < 200 || resp.StatusCode >= 300 { if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return fmt.Errorf("gitea %s %s: HTTP %d: %s", method, path, resp.StatusCode, strings.TrimSpace(string(data))) return &APIError{Method: method, Path: path, StatusCode: resp.StatusCode, Body: strings.TrimSpace(string(data))}
} }
if out != nil && len(data) > 0 { if out != nil && len(data) > 0 {
if err := json.Unmarshal(data, out); err != nil { if err := json.Unmarshal(data, out); err != nil {
+20
View File
@@ -4,6 +4,7 @@ import (
"fmt" "fmt"
"strconv" "strconv"
"strings" "strings"
"time"
) )
// PRRef identifies a single pull request by repository and number. // PRRef identifies a single pull request by repository and number.
@@ -48,6 +49,25 @@ func ParsePRRef(s string) (PRRef, error) {
return PRRef{Owner: owner, Repo: repo, Number: n}, nil return PRRef{Owner: owner, Repo: repo, Number: n}, nil
} }
// ParseDurationFlag parses a duration flag value, accepting either a Go
// duration string ("30s", "1h30m") or a bare integer read as seconds ("15").
// flag names the flag so the error says which value was rejected.
func ParseDurationFlag(flag, value string) (time.Duration, error) {
s := strings.TrimSpace(value)
d, err := time.ParseDuration(s)
if err != nil {
n, nerr := strconv.Atoi(s)
if nerr != nil {
return 0, fmt.Errorf("invalid --%s value %q: want a duration such as 30s, 2m or 1h30m, or a bare number of seconds such as 15", flag, value)
}
d = time.Duration(n) * time.Second
}
if d <= 0 {
return 0, fmt.Errorf("invalid --%s value %q: must be greater than zero", flag, value)
}
return d, nil
}
// ParseRepo validates and splits an "owner/repo" string. // ParseRepo validates and splits an "owner/repo" string.
func ParseRepo(s string) (owner, repo string, err error) { func ParseRepo(s string) (owner, repo string, err error) {
s = strings.TrimSpace(s) s = strings.TrimSpace(s)
+56 -1
View File
@@ -1,6 +1,10 @@
package agent package agent
import "testing" import (
"strings"
"testing"
"time"
)
func TestParsePRRef(t *testing.T) { func TestParsePRRef(t *testing.T) {
tests := []struct { tests := []struct {
@@ -77,3 +81,54 @@ func TestParseRepo(t *testing.T) {
} }
} }
} }
func TestParseDurationFlag(t *testing.T) {
tests := []struct {
in string
want time.Duration
wantErr bool
}{
{"15", 15 * time.Second, false},
{"15s", 15 * time.Second, false},
{"2m", 2 * time.Minute, false},
{"1h30m", 90 * time.Minute, false},
{"500ms", 500 * time.Millisecond, false},
{" 45 ", 45 * time.Second, false},
{"0", 0, true},
{"0s", 0, true},
{"-5", 0, true},
{"-5s", 0, true},
{"15x", 0, true},
{"", 0, true},
}
for _, tt := range tests {
got, err := ParseDurationFlag("interval", tt.in)
if tt.wantErr {
if err == nil {
t.Errorf("ParseDurationFlag(%q): expected error, got %v", tt.in, got)
}
continue
}
if err != nil {
t.Errorf("ParseDurationFlag(%q): unexpected error: %v", tt.in, err)
continue
}
if got != tt.want {
t.Errorf("ParseDurationFlag(%q) = %v, want %v", tt.in, got, tt.want)
}
}
}
// The error must name the flag and show valid forms instead of surfacing
// time.ParseDuration's "missing unit" wording.
func TestParseDurationFlagErrorMessage(t *testing.T) {
_, err := ParseDurationFlag("interval", "soon")
if err == nil {
t.Fatal("ParseDurationFlag(\"soon\"): expected error")
}
for _, want := range []string{"--interval", `"soon"`, "30s", "seconds"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("error %q does not mention %q", err, want)
}
}
}
+183
View File
@@ -0,0 +1,183 @@
package agent
import (
"crypto/rand"
"encoding/base64"
"errors"
"fmt"
"sort"
)
const (
// OAuthClientIDKey, OAuthClientSecretKey and OAuthCookieSecretKey are the
// KV fields oauth2-proxy deployments read their credentials from.
OAuthClientIDKey = "client_id"
OAuthClientSecretKey = "client_secret"
OAuthCookieSecretKey = "cookie_secret"
// oauthSecretBytes is the raw length of generated secrets. oauth2-proxy
// rejects a cookie secret that does not decode to exactly 32 bytes.
oauthSecretBytes = 32
)
// Per-key outcomes reported by SeedOAuth.
const (
ActionCreated = "created"
ActionKept = "kept"
ActionRotated = "rotated"
ActionUpdated = "updated"
ActionPreserved = "preserved"
)
// SeedOAuthOptions configures SeedOAuth. The CLI supplies the defaults.
type SeedOAuthOptions struct {
VaultAddr string
RoleID string
KVMount string
Path string
ClientID string
Rotate bool
}
// SeedOAuthKey names a key present in the secret and what happened to it.
type SeedOAuthKey struct {
Name string
Action string
}
// SeedOAuthResult is the non-secret summary of a seed run. Version is zero when
// nothing changed and no write was issued.
type SeedOAuthResult struct {
KVMount string
Path string
Keys []SeedOAuthKey
Version int
Changed bool
}
// KeyNames returns the key names present in the secret after the run.
func (r SeedOAuthResult) KeyNames() []string {
names := make([]string, 0, len(r.Keys))
for _, k := range r.Keys {
names = append(names, k.Name)
}
return names
}
// SeedOAuth makes a Vault KV-v2 path hold a complete oauth2-proxy credential
// set: client_id from the caller, plus a client_secret and cookie_secret that
// are generated only when absent (or when Rotate is set). It is a
// read-modify-write, so keys it does not own survive untouched, and it writes
// nothing when the secret is already correct. Secret material never leaves this
// function: results and errors carry only key names.
func SeedOAuth(o SeedOAuthOptions) (SeedOAuthResult, error) {
var res SeedOAuthResult
if o.Path == "" {
return res, errors.New("a KV-v2 path is required")
}
if o.ClientID == "" {
return res, errors.New("a client id is required")
}
vc, err := NewVaultClient(o.VaultAddr, o.RoleID)
if err != nil {
return res, fmt.Errorf("vault approle login failed against %s (check VAULT_ADDR and AGENT_APPROLE_ROLE_ID): %w", o.VaultAddr, err)
}
existing, err := vc.ReadKVOptional(o.KVMount, o.Path)
if err != nil {
if errors.Is(err, ErrVaultDenied) {
return res, fmt.Errorf("reading %s/%s denied: the agent AppRole policy does not grant read on this path (apply the terraform-vault policy change): %w", o.KVMount, o.Path, err)
}
return res, fmt.Errorf("reading %s/%s: %w", o.KVMount, o.Path, err)
}
data := make(map[string]any, len(existing)+3)
for k, v := range existing {
data[k] = v
}
var (
keys []SeedOAuthKey
changed bool
)
current, _ := existing[OAuthClientIDKey].(string)
switch current {
case o.ClientID:
keys = append(keys, SeedOAuthKey{OAuthClientIDKey, ActionKept})
case "":
keys = append(keys, SeedOAuthKey{OAuthClientIDKey, ActionCreated})
changed = true
default:
keys = append(keys, SeedOAuthKey{OAuthClientIDKey, ActionUpdated})
changed = true
}
data[OAuthClientIDKey] = o.ClientID
for _, gen := range []struct {
name string
enc *base64.Encoding
}{
// oauth2-proxy accepts a standard-base64 client secret, but the cookie
// secret goes into a cookie and must be URL-safe.
{OAuthClientSecretKey, base64.StdEncoding},
{OAuthCookieSecretKey, base64.RawURLEncoding},
} {
current, _ := existing[gen.name].(string)
if current != "" && !o.Rotate {
keys = append(keys, SeedOAuthKey{gen.name, ActionKept})
continue
}
value, err := randomSecret(gen.enc)
if err != nil {
return res, fmt.Errorf("generating %s: %w", gen.name, err)
}
action := ActionCreated
if current != "" {
action = ActionRotated
}
data[gen.name] = value
keys = append(keys, SeedOAuthKey{gen.name, action})
changed = true
}
var others []string
for k := range existing {
switch k {
case OAuthClientIDKey, OAuthClientSecretKey, OAuthCookieSecretKey:
default:
others = append(others, k)
}
}
sort.Strings(others)
for _, k := range others {
keys = append(keys, SeedOAuthKey{k, ActionPreserved})
}
res = SeedOAuthResult{KVMount: o.KVMount, Path: o.Path, Keys: keys}
if !changed {
return res, nil
}
version, err := vc.WriteKVAny(o.KVMount, o.Path, data)
if err != nil {
if errors.Is(err, ErrVaultDenied) {
return SeedOAuthResult{}, fmt.Errorf("writing %s/%s denied: the agent AppRole policy does not grant create/update on this path (apply the terraform-vault policy change): %w", o.KVMount, o.Path, err)
}
return SeedOAuthResult{}, fmt.Errorf("writing %s/%s: %w", o.KVMount, o.Path, err)
}
res.Version = version
res.Changed = true
return res, nil
}
// randomSecret returns oauthSecretBytes of crypto/rand entropy in the given
// base64 encoding.
func randomSecret(enc *base64.Encoding) (string, error) {
buf := make([]byte, oauthSecretBytes)
if _, err := rand.Read(buf); err != nil {
return "", err
}
return enc.EncodeToString(buf), nil
}
+421
View File
@@ -0,0 +1,421 @@
package agent
import (
"encoding/base64"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strconv"
"strings"
"testing"
)
const (
oauthPath = "kubernetes/namespace/repospawner/default/oauth-credentials"
oauthClientID = "mediamark-client-id"
existingClientSec = "existing-client-secret-from-authentik"
existingCookieSec = "existing-cookie-secret-value-abcdefghij"
oauthExtraKeyValue = "extra-key-secret-value"
oauthVaultClientTok = "s.vaulttoken"
)
// oauthVaultStub is a KV-v2 stand-in that actually stores what is written, so
// read-modify-write behaviour can be asserted end to end.
type oauthVaultStub struct {
data map[string]any
exists bool
version int
readStatus int
writeStatus int
writes []map[string]any
}
func newOAuthVaultStub() *oauthVaultStub {
return &oauthVaultStub{readStatus: http.StatusOK, writeStatus: http.StatusOK}
}
// seed makes the path exist with the given fields at version 1.
func (v *oauthVaultStub) seed(data map[string]any) *oauthVaultStub {
v.data = data
v.exists = true
v.version = 1
return v
}
func (v *oauthVaultStub) server(t *testing.T) *httptest.Server {
t.Helper()
mux := http.NewServeMux()
mux.HandleFunc("/v1/auth/approle/login", func(w http.ResponseWriter, r *http.Request) {
var body map[string]string
_ = json.NewDecoder(r.Body).Decode(&body)
if _, ok := body["secret_id"]; ok {
t.Errorf("secret_id must not be sent")
}
_, _ = io.WriteString(w, `{"auth":{"client_token":"`+oauthVaultClientTok+`"}}`)
})
mux.HandleFunc("/v1/kv/data/"+oauthPath, func(w http.ResponseWriter, r *http.Request) {
if got := r.Header.Get("X-Vault-Token"); got != oauthVaultClientTok {
t.Errorf("X-Vault-Token = %q, want %q", got, oauthVaultClientTok)
}
switch r.Method {
case http.MethodGet:
if v.readStatus != http.StatusOK {
w.WriteHeader(v.readStatus)
_, _ = io.WriteString(w, `{"errors":["permission denied"]}`)
return
}
if !v.exists {
w.WriteHeader(http.StatusNotFound)
_, _ = io.WriteString(w, `{"errors":[]}`)
return
}
payload, _ := json.Marshal(map[string]any{
"data": map[string]any{"data": v.data, "metadata": map[string]any{"version": v.version}},
})
_, _ = w.Write(payload)
case http.MethodPost:
if v.writeStatus != http.StatusOK {
w.WriteHeader(v.writeStatus)
_, _ = io.WriteString(w, `{"errors":["permission denied"]}`)
return
}
var body struct {
Data map[string]any `json:"data"`
}
_ = json.NewDecoder(r.Body).Decode(&body)
v.writes = append(v.writes, body.Data)
v.data = body.Data
v.exists = true
v.version++
_, _ = io.WriteString(w, `{"data":{"version":`+strconv.Itoa(v.version)+`}}`)
default:
t.Errorf("unexpected method %s", r.Method)
}
})
srv := httptest.NewServer(mux)
t.Cleanup(srv.Close)
return srv
}
func oauthOpts(vaultURL string) SeedOAuthOptions {
return SeedOAuthOptions{
VaultAddr: vaultURL,
RoleID: "role-xyz",
KVMount: DefaultKVMount,
Path: oauthPath,
ClientID: oauthClientID,
}
}
// actions flattens a result into key -> action for order-independent asserts.
func actions(res SeedOAuthResult) map[string]string {
m := make(map[string]string, len(res.Keys))
for _, k := range res.Keys {
m[k.Name] = k.Action
}
return m
}
func stringField(t *testing.T, data map[string]any, key string) string {
t.Helper()
s, ok := data[key].(string)
if !ok {
t.Fatalf("written %s = %v, want a string", key, data[key])
}
return s
}
// assertDecodesTo32 fails unless the value is base64 of exactly 32 bytes, which
// is what oauth2-proxy requires of a cookie secret.
func assertDecodesTo32(t *testing.T, enc *base64.Encoding, value, name string) {
t.Helper()
raw, err := enc.DecodeString(value)
if err != nil {
t.Fatalf("%s is not valid base64: %v", name, err)
}
if len(raw) != oauthSecretBytes {
t.Errorf("%s decodes to %d bytes, want %d", name, len(raw), oauthSecretBytes)
}
}
func TestSeedOAuthFreshCreate(t *testing.T) {
v := newOAuthVaultStub()
res, err := SeedOAuth(oauthOpts(v.server(t).URL))
if err != nil {
t.Fatalf("SeedOAuth: %v", err)
}
if !res.Changed || res.Version != 1 {
t.Errorf("Changed=%v Version=%d, want a first write at version 1", res.Changed, res.Version)
}
for key, want := range map[string]string{
OAuthClientIDKey: ActionCreated,
OAuthClientSecretKey: ActionCreated,
OAuthCookieSecretKey: ActionCreated,
} {
if got := actions(res)[key]; got != want {
t.Errorf("%s action = %q, want %q", key, got, want)
}
}
if len(v.writes) != 1 {
t.Fatalf("%d writes, want exactly 1", len(v.writes))
}
w := v.writes[0]
if got := stringField(t, w, OAuthClientIDKey); got != oauthClientID {
t.Errorf("written client_id = %q, want %q", got, oauthClientID)
}
assertDecodesTo32(t, base64.StdEncoding, stringField(t, w, OAuthClientSecretKey), OAuthClientSecretKey)
assertDecodesTo32(t, base64.RawURLEncoding, stringField(t, w, OAuthCookieSecretKey), OAuthCookieSecretKey)
}
// The mediamark case: a client_secret already issued by Authentik must survive
// while the missing keys are filled in.
func TestSeedOAuthPreservesExistingClientSecret(t *testing.T) {
v := newOAuthVaultStub().seed(map[string]any{OAuthClientSecretKey: existingClientSec})
res, err := SeedOAuth(oauthOpts(v.server(t).URL))
if err != nil {
t.Fatalf("SeedOAuth: %v", err)
}
got := actions(res)
for key, want := range map[string]string{
OAuthClientIDKey: ActionCreated,
OAuthClientSecretKey: ActionKept,
OAuthCookieSecretKey: ActionCreated,
} {
if got[key] != want {
t.Errorf("%s action = %q, want %q", key, got[key], want)
}
}
if len(v.writes) != 1 {
t.Fatalf("%d writes, want exactly 1", len(v.writes))
}
if s := stringField(t, v.writes[0], OAuthClientSecretKey); s != existingClientSec {
t.Errorf("client_secret was replaced, want the existing value kept")
}
}
func TestSeedOAuthPreservesOtherKeys(t *testing.T) {
v := newOAuthVaultStub().seed(map[string]any{
OAuthClientIDKey: oauthClientID,
OAuthClientSecretKey: existingClientSec,
"redirect_url": "https://mediamark.unkin.net/oauth2/callback",
"extra": oauthExtraKeyValue,
})
res, err := SeedOAuth(oauthOpts(v.server(t).URL))
if err != nil {
t.Fatalf("SeedOAuth: %v", err)
}
got := actions(res)
for _, key := range []string{"redirect_url", "extra"} {
if got[key] != ActionPreserved {
t.Errorf("%s action = %q, want %q", key, got[key], ActionPreserved)
}
}
if len(v.writes) != 1 {
t.Fatalf("%d writes, want exactly 1", len(v.writes))
}
w := v.writes[0]
if stringField(t, w, "extra") != oauthExtraKeyValue {
t.Errorf("extra key was not written back unchanged")
}
if stringField(t, w, "redirect_url") != "https://mediamark.unkin.net/oauth2/callback" {
t.Errorf("redirect_url was not written back unchanged")
}
}
func TestSeedOAuthRotateRegenerates(t *testing.T) {
v := newOAuthVaultStub().seed(map[string]any{
OAuthClientIDKey: oauthClientID,
OAuthClientSecretKey: existingClientSec,
OAuthCookieSecretKey: existingCookieSec,
})
o := oauthOpts(v.server(t).URL)
o.Rotate = true
res, err := SeedOAuth(o)
if err != nil {
t.Fatalf("SeedOAuth: %v", err)
}
got := actions(res)
for key, want := range map[string]string{
OAuthClientIDKey: ActionKept,
OAuthClientSecretKey: ActionRotated,
OAuthCookieSecretKey: ActionRotated,
} {
if got[key] != want {
t.Errorf("%s action = %q, want %q", key, got[key], want)
}
}
if len(v.writes) != 1 {
t.Fatalf("%d writes, want exactly 1", len(v.writes))
}
w := v.writes[0]
if stringField(t, w, OAuthClientSecretKey) == existingClientSec {
t.Errorf("client_secret unchanged under --rotate")
}
if stringField(t, w, OAuthCookieSecretKey) == existingCookieSec {
t.Errorf("cookie_secret unchanged under --rotate")
}
assertDecodesTo32(t, base64.RawURLEncoding, stringField(t, w, OAuthCookieSecretKey), OAuthCookieSecretKey)
}
// A complete, correct secret must not produce a new KV version.
func TestSeedOAuthIdempotentWritesNothing(t *testing.T) {
v := newOAuthVaultStub()
url := v.server(t).URL
if _, err := SeedOAuth(oauthOpts(url)); err != nil {
t.Fatalf("first SeedOAuth: %v", err)
}
res, err := SeedOAuth(oauthOpts(url))
if err != nil {
t.Fatalf("second SeedOAuth: %v", err)
}
if res.Changed || res.Version != 0 {
t.Errorf("Changed=%v Version=%d, want an unchanged result", res.Changed, res.Version)
}
if len(v.writes) != 1 {
t.Errorf("%d writes, want the second run to write nothing", len(v.writes))
}
for _, k := range res.Keys {
if k.Action != ActionKept {
t.Errorf("%s action = %q, want %q", k.Name, k.Action, ActionKept)
}
}
}
func TestSeedOAuthClientIDUpdated(t *testing.T) {
v := newOAuthVaultStub().seed(map[string]any{
OAuthClientIDKey: "stale-client-id",
OAuthClientSecretKey: existingClientSec,
OAuthCookieSecretKey: existingCookieSec,
})
res, err := SeedOAuth(oauthOpts(v.server(t).URL))
if err != nil {
t.Fatalf("SeedOAuth: %v", err)
}
if got := actions(res)[OAuthClientIDKey]; got != ActionUpdated {
t.Errorf("client_id action = %q, want %q", got, ActionUpdated)
}
if len(v.writes) != 1 || stringField(t, v.writes[0], OAuthClientIDKey) != oauthClientID {
t.Errorf("writes = %v, want the new client_id written", v.writes)
}
}
func TestSeedOAuthLoginFailure(t *testing.T) {
mux := http.NewServeMux()
mux.HandleFunc("/v1/auth/approle/login", func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusBadRequest)
_, _ = io.WriteString(w, `{"errors":["invalid role ID"]}`)
})
vs := httptest.NewServer(mux)
defer vs.Close()
_, err := SeedOAuth(oauthOpts(vs.URL))
if err == nil {
t.Fatal("SeedOAuth() = nil, want an approle login error")
}
if !strings.Contains(err.Error(), "approle login failed") {
t.Errorf("error = %v, want it to name the approle login", err)
}
}
func TestSeedOAuthReadDenied(t *testing.T) {
v := newOAuthVaultStub()
v.readStatus = http.StatusForbidden
_, err := SeedOAuth(oauthOpts(v.server(t).URL))
if err == nil {
t.Fatal("SeedOAuth() = nil, want a KV read error")
}
msg := err.Error()
if !strings.Contains(msg, oauthPath) || !strings.Contains(msg, "policy") {
t.Errorf("error = %v, want it to name the path and point at the policy", err)
}
if len(v.writes) != 0 {
t.Errorf("wrote %v, want no write when the read is denied", v.writes)
}
}
func TestSeedOAuthWriteDenied(t *testing.T) {
v := newOAuthVaultStub()
v.writeStatus = http.StatusForbidden
_, err := SeedOAuth(oauthOpts(v.server(t).URL))
if err == nil {
t.Fatal("SeedOAuth() = nil, want a KV write error")
}
msg := err.Error()
if !strings.Contains(msg, oauthPath) || !strings.Contains(msg, "create/update") {
t.Errorf("error = %v, want it to name the path and the missing capability", err)
}
}
// A missing path is normal (first seed), not a not-found error.
func TestSeedOAuthMissingPathIsNotAnError(t *testing.T) {
v := newOAuthVaultStub()
if _, err := SeedOAuth(oauthOpts(v.server(t).URL)); err != nil {
t.Fatalf("SeedOAuth on a missing path: %v", err)
}
}
func TestSeedOAuthRequiresPathAndClientID(t *testing.T) {
v := newOAuthVaultStub()
url := v.server(t).URL
for name, mutate := range map[string]func(*SeedOAuthOptions){
"no path": func(o *SeedOAuthOptions) { o.Path = "" },
"no client id": func(o *SeedOAuthOptions) { o.ClientID = "" },
} {
t.Run(name, func(t *testing.T) {
o := oauthOpts(url)
mutate(&o)
if _, err := SeedOAuth(o); err == nil {
t.Fatal("SeedOAuth() = nil, want a required-input error")
}
})
}
}
// No failure path may leak stored or generated secret material.
func TestSeedOAuthErrorsNeverLeakSecrets(t *testing.T) {
cases := map[string]func(*oauthVaultStub){
"read denied": func(v *oauthVaultStub) { v.readStatus = http.StatusForbidden },
"write denied": func(v *oauthVaultStub) { v.writeStatus = http.StatusForbidden },
"read error": func(v *oauthVaultStub) { v.readStatus = http.StatusInternalServerError },
"write error": func(v *oauthVaultStub) { v.writeStatus = http.StatusInternalServerError },
}
for name, mutate := range cases {
t.Run(name, func(t *testing.T) {
v := newOAuthVaultStub().seed(map[string]any{
OAuthClientSecretKey: existingClientSec,
OAuthCookieSecretKey: existingCookieSec,
"extra": oauthExtraKeyValue,
})
mutate(v)
_, err := SeedOAuth(oauthOpts(v.server(t).URL))
if err == nil {
t.Fatal("SeedOAuth() = nil, want an error")
}
for _, secret := range []string{existingClientSec, existingCookieSec, oauthExtraKeyValue} {
if strings.Contains(err.Error(), secret) {
t.Errorf("error %q leaks a secret", err)
}
}
})
}
}
// The successful result carries key names and a version, never values.
func TestSeedOAuthResultNeverCarriesSecrets(t *testing.T) {
v := newOAuthVaultStub().seed(map[string]any{OAuthClientSecretKey: existingClientSec})
res, err := SeedOAuth(oauthOpts(v.server(t).URL))
if err != nil {
t.Fatalf("SeedOAuth: %v", err)
}
rendered := strings.Join(append(res.KeyNames(), res.Path, res.KVMount), " ")
written := v.writes[0]
for _, key := range []string{OAuthClientSecretKey, OAuthCookieSecretKey} {
if value := stringField(t, written, key); strings.Contains(rendered, value) {
t.Errorf("result leaks the %s value", key)
}
}
}
+98
View File
@@ -0,0 +1,98 @@
package agent
import (
"errors"
"fmt"
)
const (
// DefaultOutpostTokenPath is the KV-v2 path holding the Authentik API token
// the agent uses to read outpost tokens.
DefaultOutpostTokenPath = "service/authentik/agent-api-token"
// DefaultDestKey is the KV field the outpost token is written to.
DefaultDestKey = "token"
)
// SeedOutpostOptions configures SeedOutpost. Every field is required; the CLI
// supplies the defaults.
type SeedOutpostOptions struct {
VaultAddr string
RoleID string
AuthentikURL string
Outpost string
KVMount string
TokenPath string
DestPath string
DestKey string
}
// SeedOutpostResult is the non-secret summary of a successful seed.
type SeedOutpostResult struct {
Outpost string
TokenIdentifier string
KVMount string
DestPath string
Version int
}
// SeedOutpost copies an Authentik outpost's token into Vault KV-v2. It reads an
// Authentik API token from Vault, resolves the outpost's token identifier,
// fetches the key and writes it to the destination path. The token value never
// leaves this function: results and errors carry only identifiers.
func SeedOutpost(o SeedOutpostOptions) (SeedOutpostResult, error) {
var res SeedOutpostResult
vc, err := NewVaultClient(o.VaultAddr, o.RoleID)
if err != nil {
return res, fmt.Errorf("vault approle login failed against %s (check VAULT_ADDR and AGENT_APPROLE_ROLE_ID): %w", o.VaultAddr, err)
}
secret, err := vc.ReadKV(o.KVMount, o.TokenPath)
if err != nil {
switch {
case errors.Is(err, ErrVaultDenied):
return res, fmt.Errorf("reading %s/%s denied: the agent AppRole policy does not grant read on this path (apply the terraform-vault policy change): %w", o.KVMount, o.TokenPath, err)
case errors.Is(err, ErrVaultNotFound):
return res, fmt.Errorf("secret %s/%s does not exist: seed the Authentik API token there first: %w", o.KVMount, o.TokenPath, err)
}
return res, fmt.Errorf("reading %s/%s: %w", o.KVMount, o.TokenPath, err)
}
apiToken := StringField(secret, "token", "api_token")
if apiToken == "" {
return res, fmt.Errorf("secret %s/%s has neither a 'token' nor an 'api_token' field", o.KVMount, o.TokenPath)
}
ac := NewAuthentikClient(o.AuthentikURL, apiToken)
outpost, err := ac.FindOutpost(o.Outpost)
if err != nil {
if errors.Is(err, ErrOutpostNotFound) {
return res, fmt.Errorf("no outpost named %q at %s: has the terraform-authentik outpost been applied?: %w", o.Outpost, o.AuthentikURL, err)
}
return res, fmt.Errorf("looking up outpost %q: %w", o.Outpost, err)
}
if outpost.TokenIdentifier == "" {
return res, fmt.Errorf("outpost %q has an empty token_identifier", outpost.Name)
}
key, err := ac.TokenKey(outpost.TokenIdentifier)
if err != nil {
return res, fmt.Errorf("fetching the key for token identifier %q (the API token needs view_key on it): %w", outpost.TokenIdentifier, err)
}
version, err := vc.WriteKV(o.KVMount, o.DestPath, map[string]string{o.DestKey: key})
if err != nil {
if errors.Is(err, ErrVaultDenied) {
return res, fmt.Errorf("writing %s/%s denied: the agent AppRole policy does not grant create/update on this path (apply the terraform-vault policy change): %w", o.KVMount, o.DestPath, err)
}
return res, fmt.Errorf("writing %s/%s: %w", o.KVMount, o.DestPath, err)
}
return SeedOutpostResult{
Outpost: outpost.Name,
TokenIdentifier: outpost.TokenIdentifier,
KVMount: o.KVMount,
DestPath: o.DestPath,
Version: version,
}, nil
}
+369
View File
@@ -0,0 +1,369 @@
package agent
import (
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strconv"
"strings"
"testing"
)
const (
testAPIToken = "ak-api-token-secret"
testOutpostKey = "outpost-key-secret"
testTokenPath = "service/authentik/agent-api-token"
testDestPath = "kubernetes/namespace/authentik/default/outpost-token"
testOutpostName = "k8s-outpost"
testTokenIdent = "ak-outpost-k8s-outpost"
testVaultClientT = "s.vaulttoken"
)
// vaultStub is a KV-v2 stand-in whose per-path behaviour tests can override.
type vaultStub struct {
readStatus int
writeStatus int
tokenField string // field name the API token is stored under
writes []map[string]string
writeCount int
}
func newVaultStub() *vaultStub {
return &vaultStub{readStatus: http.StatusOK, writeStatus: http.StatusOK, tokenField: "token"}
}
func (v *vaultStub) server(t *testing.T) *httptest.Server {
t.Helper()
mux := http.NewServeMux()
mux.HandleFunc("/v1/auth/approle/login", func(w http.ResponseWriter, r *http.Request) {
var body map[string]string
_ = json.NewDecoder(r.Body).Decode(&body)
if _, ok := body["secret_id"]; ok {
t.Errorf("secret_id must not be sent")
}
_, _ = io.WriteString(w, `{"auth":{"client_token":"`+testVaultClientT+`"}}`)
})
mux.HandleFunc("/v1/kv/data/"+testTokenPath, func(w http.ResponseWriter, r *http.Request) {
if got := r.Header.Get("X-Vault-Token"); got != testVaultClientT {
t.Errorf("X-Vault-Token = %q, want %q", got, testVaultClientT)
}
if v.readStatus != http.StatusOK {
w.WriteHeader(v.readStatus)
_, _ = io.WriteString(w, `{"errors":["permission denied"]}`)
return
}
_, _ = io.WriteString(w, `{"data":{"data":{"`+v.tokenField+`":"`+testAPIToken+`"},"metadata":{"version":1}}}`)
})
mux.HandleFunc("/v1/kv/data/"+testDestPath, func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
t.Errorf("write method = %s, want POST", r.Method)
}
if v.writeStatus != http.StatusOK {
w.WriteHeader(v.writeStatus)
_, _ = io.WriteString(w, `{"errors":["permission denied"]}`)
return
}
var body struct {
Data map[string]string `json:"data"`
}
_ = json.NewDecoder(r.Body).Decode(&body)
v.writes = append(v.writes, body.Data)
v.writeCount++
_, _ = io.WriteString(w, `{"data":{"version":`+strconv.Itoa(v.writeCount+2)+`}}`)
})
srv := httptest.NewServer(mux)
t.Cleanup(srv.Close)
return srv
}
// authentikStub serves the outpost search and view_key endpoints.
type authentikStub struct {
results string // JSON array body for .results
viewKeyStatus int
viewKeyBody string
sawBearer string
sawSearchQuery string
}
func newAuthentikStub() *authentikStub {
return &authentikStub{
results: `{"pk":"1","name":"` + testOutpostName + `","token_identifier":"` + testTokenIdent + `"}`,
viewKeyStatus: http.StatusOK,
viewKeyBody: `{"key":"` + testOutpostKey + `"}`,
}
}
func (a *authentikStub) server(t *testing.T) *httptest.Server {
t.Helper()
mux := http.NewServeMux()
mux.HandleFunc("/api/v3/outposts/instances/", func(w http.ResponseWriter, r *http.Request) {
a.sawBearer = r.Header.Get("Authorization")
a.sawSearchQuery = r.URL.Query().Get("search")
_, _ = io.WriteString(w, `{"results":[`+a.results+`]}`)
})
mux.HandleFunc("/api/v3/core/tokens/"+testTokenIdent+"/view_key/", func(w http.ResponseWriter, r *http.Request) {
if a.viewKeyStatus != http.StatusOK {
w.WriteHeader(a.viewKeyStatus)
_, _ = io.WriteString(w, `{"detail":"boom"}`)
return
}
_, _ = io.WriteString(w, a.viewKeyBody)
})
srv := httptest.NewServer(mux)
t.Cleanup(srv.Close)
return srv
}
func opts(vaultURL, authentikURL string) SeedOutpostOptions {
return SeedOutpostOptions{
VaultAddr: vaultURL,
RoleID: "role-xyz",
AuthentikURL: authentikURL,
Outpost: testOutpostName,
KVMount: DefaultKVMount,
TokenPath: testTokenPath,
DestPath: testDestPath,
DestKey: DefaultDestKey,
}
}
func TestSeedOutpostHappyPath(t *testing.T) {
v := newVaultStub()
a := newAuthentikStub()
vs, as := v.server(t), a.server(t)
res, err := SeedOutpost(opts(vs.URL, as.URL))
if err != nil {
t.Fatalf("SeedOutpost: %v", err)
}
if res.Outpost != testOutpostName {
t.Errorf("Outpost = %q, want %q", res.Outpost, testOutpostName)
}
if res.TokenIdentifier != testTokenIdent {
t.Errorf("TokenIdentifier = %q, want %q", res.TokenIdentifier, testTokenIdent)
}
if res.DestPath != testDestPath || res.KVMount != DefaultKVMount {
t.Errorf("dest = %s/%s, want kv/%s", res.KVMount, res.DestPath, testDestPath)
}
if res.Version != 3 {
t.Errorf("Version = %d, want 3", res.Version)
}
if a.sawBearer != "Bearer "+testAPIToken {
t.Errorf("Authorization = %q, want the API token as a bearer", a.sawBearer)
}
if a.sawSearchQuery != testOutpostName {
t.Errorf("search = %q, want %q", a.sawSearchQuery, testOutpostName)
}
if len(v.writes) != 1 || v.writes[0][DefaultDestKey] != testOutpostKey {
t.Fatalf("written data = %v, want {%s: outpost key}", v.writes, DefaultDestKey)
}
}
// Re-running writes a new KV version rather than failing.
func TestSeedOutpostIdempotentNewVersion(t *testing.T) {
v := newVaultStub()
a := newAuthentikStub()
vs, as := v.server(t), a.server(t)
first, err := SeedOutpost(opts(vs.URL, as.URL))
if err != nil {
t.Fatalf("first SeedOutpost: %v", err)
}
second, err := SeedOutpost(opts(vs.URL, as.URL))
if err != nil {
t.Fatalf("second SeedOutpost: %v", err)
}
if second.Version != first.Version+1 {
t.Errorf("versions = %d then %d, want consecutive", first.Version, second.Version)
}
}
// The API token may be stored under api_token instead of token.
func TestSeedOutpostAPITokenFallbackField(t *testing.T) {
v := newVaultStub()
v.tokenField = "api_token"
a := newAuthentikStub()
vs, as := v.server(t), a.server(t)
if _, err := SeedOutpost(opts(vs.URL, as.URL)); err != nil {
t.Fatalf("SeedOutpost with api_token field: %v", err)
}
if a.sawBearer != "Bearer "+testAPIToken {
t.Errorf("Authorization = %q, want the api_token value", a.sawBearer)
}
}
func TestSeedOutpostCustomDestKey(t *testing.T) {
v := newVaultStub()
a := newAuthentikStub()
vs, as := v.server(t), a.server(t)
o := opts(vs.URL, as.URL)
o.DestKey = "outpost-token"
if _, err := SeedOutpost(o); err != nil {
t.Fatalf("SeedOutpost: %v", err)
}
if len(v.writes) != 1 || v.writes[0]["outpost-token"] != testOutpostKey {
t.Errorf("written data = %v, want the key under outpost-token", v.writes)
}
}
func TestSeedOutpostLoginFailure(t *testing.T) {
mux := http.NewServeMux()
mux.HandleFunc("/v1/auth/approle/login", func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusBadRequest)
_, _ = io.WriteString(w, `{"errors":["invalid role ID"]}`)
})
vs := httptest.NewServer(mux)
defer vs.Close()
a := newAuthentikStub()
_, err := SeedOutpost(opts(vs.URL, a.server(t).URL))
if err == nil {
t.Fatal("SeedOutpost() = nil, want an approle login error")
}
if !strings.Contains(err.Error(), "approle login failed") {
t.Errorf("error = %v, want it to name the approle login", err)
}
}
func TestSeedOutpostKVReadDenied(t *testing.T) {
v := newVaultStub()
v.readStatus = http.StatusForbidden
a := newAuthentikStub()
vs, as := v.server(t), a.server(t)
_, err := SeedOutpost(opts(vs.URL, as.URL))
if err == nil {
t.Fatal("SeedOutpost() = nil, want a KV read error")
}
msg := err.Error()
if !strings.Contains(msg, testTokenPath) || !strings.Contains(msg, "policy") {
t.Errorf("error = %v, want it to name the token path and point at the policy", err)
}
if strings.Contains(msg, "does not exist") {
t.Errorf("error = %v, denied must not be reported as missing", err)
}
}
func TestSeedOutpostKVReadNotFound(t *testing.T) {
v := newVaultStub()
v.readStatus = http.StatusNotFound
a := newAuthentikStub()
vs, as := v.server(t), a.server(t)
_, err := SeedOutpost(opts(vs.URL, as.URL))
if err == nil {
t.Fatal("SeedOutpost() = nil, want a missing-secret error")
}
if !strings.Contains(err.Error(), "does not exist") {
t.Errorf("error = %v, want it to say the secret does not exist", err)
}
}
func TestSeedOutpostMissingTokenField(t *testing.T) {
v := newVaultStub()
v.tokenField = "password"
a := newAuthentikStub()
vs, as := v.server(t), a.server(t)
_, err := SeedOutpost(opts(vs.URL, as.URL))
if err == nil {
t.Fatal("SeedOutpost() = nil, want an error for a secret with no token field")
}
if !strings.Contains(err.Error(), "api_token") {
t.Errorf("error = %v, want it to name the accepted fields", err)
}
}
// A substring hit that is not the exact name must not be accepted.
func TestSeedOutpostNotFound(t *testing.T) {
v := newVaultStub()
a := newAuthentikStub()
a.results = `{"pk":"1","name":"` + testOutpostName + `-staging","token_identifier":"other"}`
vs, as := v.server(t), a.server(t)
_, err := SeedOutpost(opts(vs.URL, as.URL))
if err == nil {
t.Fatal("SeedOutpost() = nil, want an outpost-not-found error")
}
msg := err.Error()
if !strings.Contains(msg, "no outpost named") || !strings.Contains(msg, "terraform") {
t.Errorf("error = %v, want it to report the missing outpost and mention terraform", err)
}
if len(v.writes) != 0 {
t.Errorf("wrote %v, want no KV write when the outpost is missing", v.writes)
}
}
func TestSeedOutpostViewKeyFailure(t *testing.T) {
v := newVaultStub()
a := newAuthentikStub()
a.viewKeyStatus = http.StatusForbidden
vs, as := v.server(t), a.server(t)
_, err := SeedOutpost(opts(vs.URL, as.URL))
if err == nil {
t.Fatal("SeedOutpost() = nil, want a view_key error")
}
if !strings.Contains(err.Error(), testTokenIdent) {
t.Errorf("error = %v, want it to name the token identifier", err)
}
if len(v.writes) != 0 {
t.Errorf("wrote %v, want no KV write when view_key fails", v.writes)
}
}
func TestSeedOutpostViewKeyEmpty(t *testing.T) {
v := newVaultStub()
a := newAuthentikStub()
a.viewKeyBody = `{}`
vs, as := v.server(t), a.server(t)
if _, err := SeedOutpost(opts(vs.URL, as.URL)); err == nil {
t.Fatal("SeedOutpost() = nil, want an error when view_key returns no key")
}
}
func TestSeedOutpostKVWriteDenied(t *testing.T) {
v := newVaultStub()
v.writeStatus = http.StatusForbidden
a := newAuthentikStub()
vs, as := v.server(t), a.server(t)
_, err := SeedOutpost(opts(vs.URL, as.URL))
if err == nil {
t.Fatal("SeedOutpost() = nil, want a KV write error")
}
msg := err.Error()
if !strings.Contains(msg, testDestPath) || !strings.Contains(msg, "create/update") {
t.Errorf("error = %v, want it to name the dest path and the missing capability", err)
}
}
// No failure path may leak the API token or the outpost key into the error.
func TestSeedOutpostErrorsNeverLeakSecrets(t *testing.T) {
cases := map[string]func(*vaultStub, *authentikStub){
"read denied": func(v *vaultStub, a *authentikStub) { v.readStatus = http.StatusForbidden },
"write denied": func(v *vaultStub, a *authentikStub) { v.writeStatus = http.StatusForbidden },
"view_key fail": func(v *vaultStub, a *authentikStub) { a.viewKeyStatus = http.StatusInternalServerError },
"outpost gone": func(v *vaultStub, a *authentikStub) { a.results = "" },
}
for name, mutate := range cases {
t.Run(name, func(t *testing.T) {
v, a := newVaultStub(), newAuthentikStub()
mutate(v, a)
vs, as := v.server(t), a.server(t)
_, err := SeedOutpost(opts(vs.URL, as.URL))
if err == nil {
t.Fatal("SeedOutpost() = nil, want an error")
}
for _, secret := range []string{testAPIToken, testOutpostKey} {
if strings.Contains(err.Error(), secret) {
t.Errorf("error %q leaks a secret", err)
}
}
})
}
}
+62 -22
View File
@@ -1,12 +1,13 @@
// Package agent holds the plumbing shared by the agent-tools CLIs (agentpr and // Package agent holds the plumbing shared by the agent-tools CLIs (agentpr,
// watchpr): obtaining a Gitea token via Vault AppRole, talking to the Gitea // watchpr, agentws and agentvault): the Vault AppRole login and its KV-v2
// API, parsing PR references, and deciding when a watched PR changed // client, talking to the Gitea and Authentik APIs, parsing PR references, and
// meaningfully. Both tools acquire their Gitea token the same way, so that // deciding when a watched PR changed meaningfully. Every tool authenticates to
// logic lives here once. // Vault the same way, so that logic lives here once.
package agent package agent
import ( import (
"os" "os"
"strings"
"sync" "sync"
) )
@@ -16,13 +17,18 @@ const (
// DefaultRoleID is the agent AppRole role_id used when AGENT_APPROLE_ROLE_ID // DefaultRoleID is the agent AppRole role_id used when AGENT_APPROLE_ROLE_ID
// is unset. Login uses role_id only (no secret_id). // is unset. Login uses role_id only (no secret_id).
DefaultRoleID = "ababbcd3-9c77-5c6a-be2d-287fce9214a6" DefaultRoleID = "ababbcd3-9c77-5c6a-be2d-287fce9214a6"
// GiteaCredsPath is the Vault path that mints a scoped Gitea token. // GiteaCredsPrefix is the Vault gitea secrets-engine creds prefix; the agent
GiteaCredsPath = "gitea/creds/unkin-agent" // login is appended to it to form the path that mints a scoped Gitea token.
GiteaCredsPrefix = "gitea/creds/"
// DefaultGiteaURL is the Gitea base URL used when GITEA_URL is unset. // DefaultGiteaURL is the Gitea base URL used when GITEA_URL is unset.
DefaultGiteaURL = "https://git.unkin.net" DefaultGiteaURL = "https://git.unkin.net"
// DefaultAgentLogin is the Gitea login of the agent whose own comments are // DefaultAgentLogin is the Gitea login the tools act as: it selects the Vault
// ignored by watchpr. Overridable via AGENT_LOGIN. // creds path, sets the agentws git identity and is the login whose own
// comments watchpr ignores. Overridable via AGENT_LOGIN.
DefaultAgentLogin = "unkin-agent" DefaultAgentLogin = "unkin-agent"
// DefaultAuthentikURL is the Authentik base URL used when AUTHENTIK_URL is
// unset. identity.unkin.net has no DNS record; the k8s name is the real one.
DefaultAuthentikURL = "https://identity.k8s.syd1.au.unkin.net"
) )
// VaultAddr returns the configured Vault address (env VAULT_ADDR or the default). // VaultAddr returns the configured Vault address (env VAULT_ADDR or the default).
@@ -50,8 +56,8 @@ func GiteaURL() string {
return DefaultGiteaURL return DefaultGiteaURL
} }
// AgentLogin returns the login whose comments watchpr ignores (env AGENT_LOGIN // AgentLogin returns the Gitea login the tools act as (env AGENT_LOGIN or the
// or the default). // default).
func AgentLogin() string { func AgentLogin() string {
if v := os.Getenv("AGENT_LOGIN"); v != "" { if v := os.Getenv("AGENT_LOGIN"); v != "" {
return v return v
@@ -59,28 +65,62 @@ func AgentLogin() string {
return DefaultAgentLogin return DefaultAgentLogin
} }
// GiteaCredsPath returns the Vault path that mints a scoped Gitea token:
// GITEA_CREDS_PATH when set, otherwise gitea/creds/<AgentLogin>. So a service
// running as its own identity only has to set AGENT_LOGIN.
func GiteaCredsPath() string {
// Trimmed because callers join this onto ".../v1/".
if v := strings.Trim(strings.TrimSpace(os.Getenv("GITEA_CREDS_PATH")), "/"); v != "" {
return v
}
return GiteaCredsPrefix + AgentLogin()
}
// AuthentikURL returns the configured Authentik base URL (env AUTHENTIK_URL or
// the default).
func AuthentikURL() string {
if v := os.Getenv("AUTHENTIK_URL"); v != "" {
return v
}
return DefaultAuthentikURL
}
var ( var (
tokenOnce sync.Once tokenMu sync.Mutex
tokenValue string tokenMinted bool
tokenErr error tokenValue string
tokenErr error
) )
// GiteaToken returns a Gitea token, minting it via Vault AppRole on first call // GiteaToken returns a Gitea token, minting it via Vault AppRole on first call
// and caching it in-process for the lifetime of the command. // and caching it in-process for the lifetime of the command.
func GiteaToken() (string, error) { func GiteaToken() (string, error) {
tokenOnce.Do(func() { tokenMu.Lock()
tokenValue, tokenErr = fetchGiteaToken(VaultAddr(), RoleID()) defer tokenMu.Unlock()
}) if !tokenMinted {
tokenValue, tokenErr = fetchGiteaToken(VaultAddr(), RoleID(), GiteaCredsPath())
tokenMinted = true
}
return tokenValue, tokenErr return tokenValue, tokenErr
} }
// fetchGiteaToken performs the AppRole login and reads the Gitea creds. It is // RefreshGiteaToken mints a fresh Gitea token and replaces the cached one, for
// separated from GiteaToken so tests can exercise it directly against an // callers that outlive the ~1h token TTL.
// httptest server without touching the process-wide cache. func RefreshGiteaToken() (string, error) {
func fetchGiteaToken(vaultAddr, roleID string) (string, error) { tokenMu.Lock()
defer tokenMu.Unlock()
tokenValue, tokenErr = fetchGiteaToken(VaultAddr(), RoleID(), GiteaCredsPath())
tokenMinted = true
return tokenValue, tokenErr
}
// fetchGiteaToken performs the AppRole login and reads the Gitea creds at
// credsPath. It is separated from GiteaToken so tests can exercise it directly
// against an httptest server without touching the process-wide cache.
func fetchGiteaToken(vaultAddr, roleID, credsPath string) (string, error) {
clientToken, err := approleLogin(vaultAddr, roleID) clientToken, err := approleLogin(vaultAddr, roleID)
if err != nil { if err != nil {
return "", err return "", err
} }
return readGiteaCreds(vaultAddr, clientToken) return readGiteaCreds(vaultAddr, clientToken, credsPath)
} }
+155
View File
@@ -0,0 +1,155 @@
package agent
import (
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
)
func TestGiteaCredsPath(t *testing.T) {
tests := []struct {
name string
agentLogin string
credsPath string
want string
}{
{"no env keeps the historical path", "", "", "gitea/creds/unkin-agent"},
{"derived from AGENT_LOGIN", "repospawner", "", "gitea/creds/repospawner"},
{"GITEA_CREDS_PATH beats AGENT_LOGIN", "repospawner", "gitea/creds/someone-else", "gitea/creds/someone-else"},
{"GITEA_CREDS_PATH beats the default", "", "other-gitea/creds/bot", "other-gitea/creds/bot"},
{"override is trimmed for joining onto /v1/", "", " /gitea/creds/bot/ ", "gitea/creds/bot"},
{"blank override falls back to the login", "repospawner", " ", "gitea/creds/repospawner"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Setenv("AGENT_LOGIN", tt.agentLogin)
t.Setenv("GITEA_CREDS_PATH", tt.credsPath)
if got := GiteaCredsPath(); got != tt.want {
t.Errorf("GiteaCredsPath() = %q, want %q", got, tt.want)
}
})
}
}
// recordingVault serves the AppRole login plus any creds path under /v1/,
// recording which one was read so tests can assert the selected path.
type recordingVault struct {
mu sync.Mutex
credsPath string
}
func (v *recordingVault) path() string {
v.mu.Lock()
defer v.mu.Unlock()
return v.credsPath
}
// fakeEstate serves both Vault (/v1/...) and Gitea (/api/v1/...) from one
// server, so a test can drive the whole token-then-API flow.
func fakeEstate(t *testing.T, giteaToken, login string) (*httptest.Server, *recordingVault) {
t.Helper()
rec := &recordingVault{}
mux := http.NewServeMux()
mux.HandleFunc("/v1/auth/approle/login", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `{"auth":{"client_token":"s.vaulttoken"}}`)
})
mux.HandleFunc("/v1/", func(w http.ResponseWriter, r *http.Request) {
rec.mu.Lock()
rec.credsPath = strings.TrimPrefix(r.URL.Path, "/v1/")
rec.mu.Unlock()
_, _ = io.WriteString(w, `{"data":{"token":"`+giteaToken+`"}}`)
})
mux.HandleFunc("/api/v1/user", func(w http.ResponseWriter, r *http.Request) {
if got := r.Header.Get("Authorization"); got != "token "+giteaToken {
t.Errorf("whoami auth header = %q, want token %s", got, giteaToken)
}
_, _ = io.WriteString(w, `{"login":"`+login+`","id":7}`)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/pulls", func(w http.ResponseWriter, r *http.Request) {
if got := r.Header.Get("Authorization"); got != "token "+giteaToken {
t.Errorf("create PR auth header = %q, want token %s", got, giteaToken)
}
var body CreatePROptions
_ = json.NewDecoder(r.Body).Decode(&body)
_, _ = io.WriteString(w, `{"number":12,"state":"open","html_url":"`+r.Host+`/pulls/12"}`)
})
return httptest.NewServer(mux), rec
}
// A service that sets AGENT_LOGIN must read its own creds path and act as its
// own Gitea identity for both whoami and PR creation.
func TestWhoamiAndPRUseSelectedCredsPath(t *testing.T) {
srv, rec := fakeEstate(t, "gitea-repospawner", "repospawner")
defer srv.Close()
t.Setenv("AGENT_LOGIN", "repospawner")
t.Setenv("GITEA_CREDS_PATH", "")
t.Setenv("VAULT_ADDR", srv.URL)
t.Setenv("GITEA_URL", srv.URL)
tok, err := fetchGiteaToken(VaultAddr(), RoleID(), GiteaCredsPath())
if err != nil {
t.Fatalf("fetchGiteaToken: %v", err)
}
if got := rec.path(); got != "gitea/creds/repospawner" {
t.Errorf("vault read path = %q, want gitea/creds/repospawner", got)
}
if tok != "gitea-repospawner" {
t.Fatalf("token = %q, want gitea-repospawner", tok)
}
c := NewGiteaClient(tok)
u, err := c.Whoami()
if err != nil {
t.Fatalf("Whoami: %v", err)
}
if u.Login != "repospawner" {
t.Errorf("whoami login = %q, want repospawner", u.Login)
}
pr, err := c.CreatePR("unkin/repo", CreatePROptions{Base: "main", Head: "feature", Title: "T"})
if err != nil {
t.Fatalf("CreatePR: %v", err)
}
if pr.Number != 12 {
t.Errorf("PR number = %d, want 12", pr.Number)
}
}
// GITEA_CREDS_PATH must win even when AGENT_LOGIN names a different identity.
func TestCredsPathOverrideBeatsAgentLogin(t *testing.T) {
srv, rec := fakeEstate(t, "gitea-override", "someone-else")
defer srv.Close()
t.Setenv("AGENT_LOGIN", "repospawner")
t.Setenv("GITEA_CREDS_PATH", "gitea/creds/someone-else")
t.Setenv("VAULT_ADDR", srv.URL)
if _, err := fetchGiteaToken(VaultAddr(), RoleID(), GiteaCredsPath()); err != nil {
t.Fatalf("fetchGiteaToken: %v", err)
}
if got := rec.path(); got != "gitea/creds/someone-else" {
t.Errorf("vault read path = %q, want gitea/creds/someone-else", got)
}
}
// With no env set the tools must still read the exact path they always did.
func TestCredsPathDefaultIsBackwardCompatible(t *testing.T) {
srv, rec := fakeEstate(t, "gitea-abc", "unkin-agent")
defer srv.Close()
t.Setenv("AGENT_LOGIN", "")
t.Setenv("GITEA_CREDS_PATH", "")
t.Setenv("VAULT_ADDR", srv.URL)
if _, err := fetchGiteaToken(VaultAddr(), RoleID(), GiteaCredsPath()); err != nil {
t.Fatalf("fetchGiteaToken: %v", err)
}
if got := rec.path(); got != "gitea/creds/unkin-agent" {
t.Errorf("vault read path = %q, want gitea/creds/unkin-agent", got)
}
}
+8 -7
View File
@@ -49,9 +49,10 @@ func approleLogin(vaultAddr, roleID string) (string, error) {
return out.Auth.ClientToken, nil return out.Auth.ClientToken, nil
} }
// readGiteaCreds reads the Gitea creds secret and returns the token field. // readGiteaCreds reads the Gitea creds secret at credsPath and returns the
func readGiteaCreds(vaultAddr, clientToken string) (string, error) { // token field.
url := strings.TrimRight(vaultAddr, "/") + "/v1/" + GiteaCredsPath func readGiteaCreds(vaultAddr, clientToken, credsPath string) (string, error) {
url := strings.TrimRight(vaultAddr, "/") + "/v1/" + credsPath
req, err := http.NewRequest(http.MethodGet, url, nil) req, err := http.NewRequest(http.MethodGet, url, nil)
if err != nil { if err != nil {
return "", err return "", err
@@ -60,12 +61,12 @@ func readGiteaCreds(vaultAddr, clientToken string) (string, error) {
resp, err := httpClient.Do(req) resp, err := httpClient.Do(req)
if err != nil { if err != nil {
return "", fmt.Errorf("vault read %s: %w", GiteaCredsPath, err) return "", fmt.Errorf("vault read %s: %w", credsPath, err)
} }
defer func() { _ = resp.Body.Close() }() defer func() { _ = resp.Body.Close() }()
data, _ := io.ReadAll(resp.Body) data, _ := io.ReadAll(resp.Body)
if resp.StatusCode != http.StatusOK { if resp.StatusCode != http.StatusOK {
return "", fmt.Errorf("vault read %s: HTTP %d: %s", GiteaCredsPath, resp.StatusCode, strings.TrimSpace(string(data))) return "", fmt.Errorf("vault read %s: HTTP %d: %s", credsPath, resp.StatusCode, strings.TrimSpace(string(data)))
} }
var out struct { var out struct {
@@ -74,10 +75,10 @@ func readGiteaCreds(vaultAddr, clientToken string) (string, error) {
} `json:"data"` } `json:"data"`
} }
if err := json.Unmarshal(data, &out); err != nil { if err := json.Unmarshal(data, &out); err != nil {
return "", fmt.Errorf("vault read %s: decoding response: %w", GiteaCredsPath, err) return "", fmt.Errorf("vault read %s: decoding response: %w", credsPath, err)
} }
if out.Data.Token == "" { if out.Data.Token == "" {
return "", fmt.Errorf("vault read %s: no token field in secret", GiteaCredsPath) return "", fmt.Errorf("vault read %s: no token field in secret", credsPath)
} }
return out.Data.Token, nil return out.Data.Token, nil
} }
+166
View File
@@ -0,0 +1,166 @@
package agent
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"strings"
)
const (
// DefaultKVMount is the KV-v2 mount holding agent-facing secrets.
DefaultKVMount = "kv"
)
var (
// ErrVaultDenied marks a 403 from Vault (the AppRole policy lacks the capability).
ErrVaultDenied = errors.New("permission denied")
// ErrVaultNotFound marks a 404 from Vault (mount or secret does not exist).
ErrVaultNotFound = errors.New("not found")
)
// VaultClient issues authenticated requests against Vault/OpenBao using a token
// obtained from the agent AppRole.
type VaultClient struct {
Addr string
Token string
HTTP *http.Client
}
// NewVaultClient performs the AppRole login (role_id only, no secret_id) and
// returns a client bound to the resulting client_token.
func NewVaultClient(addr, roleID string) (*VaultClient, error) {
token, err := approleLogin(addr, roleID)
if err != nil {
return nil, err
}
return &VaultClient{Addr: addr, Token: token, HTTP: httpClient}, nil
}
func (c *VaultClient) do(method, path string, body any, out any) error {
var reader io.Reader
if body != nil {
b, err := json.Marshal(body)
if err != nil {
return err
}
reader = bytes.NewReader(b)
}
url := strings.TrimRight(c.Addr, "/") + path
req, err := http.NewRequest(method, url, reader)
if err != nil {
return err
}
req.Header.Set("X-Vault-Token", c.Token)
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
resp, err := c.HTTP.Do(req)
if err != nil {
return fmt.Errorf("vault %s %s: %w", method, path, err)
}
defer func() { _ = resp.Body.Close() }()
data, _ := io.ReadAll(resp.Body)
switch {
case resp.StatusCode == http.StatusForbidden:
return fmt.Errorf("vault %s %s: %w", method, path, ErrVaultDenied)
case resp.StatusCode == http.StatusNotFound:
return fmt.Errorf("vault %s %s: %w", method, path, ErrVaultNotFound)
case resp.StatusCode < 200 || resp.StatusCode >= 300:
return fmt.Errorf("vault %s %s: HTTP %d: %s", method, path, resp.StatusCode, strings.TrimSpace(string(data)))
}
if out != nil && len(data) > 0 {
// Response bodies here carry secret material, so decode failures never
// echo the body.
if err := json.Unmarshal(data, out); err != nil {
return fmt.Errorf("vault %s %s: decoding response: %w", method, path, err)
}
}
return nil
}
// kvDataPath builds the KV-v2 data path for a mount and secret path.
func kvDataPath(mount, path string) string {
return "/v1/" + strings.Trim(mount, "/") + "/data/" + strings.Trim(path, "/")
}
// ReadKV returns the data map of a KV-v2 secret.
func (c *VaultClient) ReadKV(mount, path string) (map[string]any, error) {
var out struct {
Data struct {
Data map[string]any `json:"data"`
} `json:"data"`
}
if err := c.do(http.MethodGet, kvDataPath(mount, path), nil, &out); err != nil {
return nil, err
}
if out.Data.Data == nil {
return nil, fmt.Errorf("vault read %s/%s: secret has no data", mount, path)
}
return out.Data.Data, nil
}
// ReadKVOptional is ReadKV but treats a missing secret as empty: a 404 or a
// deleted version (data: null) yields an empty map, not an error. Callers that
// read-modify-write a path that may not exist yet use this.
func (c *VaultClient) ReadKVOptional(mount, path string) (map[string]any, error) {
var out struct {
Data struct {
Data map[string]any `json:"data"`
} `json:"data"`
}
if err := c.do(http.MethodGet, kvDataPath(mount, path), nil, &out); err != nil {
if errors.Is(err, ErrVaultNotFound) {
return map[string]any{}, nil
}
return nil, err
}
if out.Data.Data == nil {
return map[string]any{}, nil
}
return out.Data.Data, nil
}
// WriteKV writes a KV-v2 secret and returns the version it created.
func (c *VaultClient) WriteKV(mount, path string, data map[string]string) (int, error) {
fields := make(map[string]any, len(data))
for k, v := range data {
fields[k] = v
}
return c.WriteKVAny(mount, path, fields)
}
// WriteKVAny writes a KV-v2 secret whose fields are not all strings (so a
// read-modify-write can put back values it did not author) and returns the
// version it created.
func (c *VaultClient) WriteKVAny(mount, path string, data map[string]any) (int, error) {
var out struct {
Data struct {
Version int `json:"version"`
} `json:"data"`
}
body := map[string]any{"data": data}
if err := c.do(http.MethodPost, kvDataPath(mount, path), body, &out); err != nil {
return 0, err
}
if out.Data.Version == 0 {
return 0, fmt.Errorf("vault write %s/%s: no version in response", mount, path)
}
return out.Data.Version, nil
}
// StringField returns the first non-empty string value among the given keys.
func StringField(data map[string]any, keys ...string) string {
for _, k := range keys {
if s, ok := data[k].(string); ok && s != "" {
return s
}
}
return ""
}
+44 -5
View File
@@ -1,6 +1,20 @@
package agent package agent
import "time" import (
"errors"
"fmt"
"time"
)
// errPRGone marks a 404 from the PR lookup itself. A 404 from any other endpoint
// can be a proxy or ingress blip and is left to the ordinary failure cap.
var errPRGone = errors.New("PR no longer visible")
// IsPRGone reports whether err is a 404 from the PR lookup, meaning the PR is no
// longer visible rather than one endpoint being briefly unreachable.
func IsPRGone(err error) bool {
return errors.Is(err, errPRGone)
}
// PRState is a point-in-time snapshot of the PR attributes watchpr tracks. // PRState is a point-in-time snapshot of the PR attributes watchpr tracks.
type PRState struct { type PRState struct {
@@ -20,10 +34,16 @@ type PRState struct {
func FetchState(c *GiteaClient, ref PRRef, agentLogin string) (PRState, error) { func FetchState(c *GiteaClient, ref PRRef, agentLogin string) (PRState, error) {
pr, err := c.GetPR(ref.RepoPath(), ref.Number) pr, err := c.GetPR(ref.RepoPath(), ref.Number)
if err != nil { if err != nil {
if IsNotFound(err) {
return PRState{}, fmt.Errorf("%w: %w", errPRGone, err)
}
return PRState{}, err return PRState{}, err
} }
// A 404 here means the head commit is gone (branch deleted after a squash/
// rebase merge); the PR object is still authoritative, so treat CI as absent
// rather than discarding the merge signal and hanging the watch loop.
ci, err := c.CommitStatus(ref.RepoPath(), pr.Head.Sha) ci, err := c.CommitStatus(ref.RepoPath(), pr.Head.Sha)
if err != nil { if err != nil && !IsNotFound(err) {
return PRState{}, err return PRState{}, err
} }
comments, err := c.ListComments(ref.RepoPath(), ref.Number) comments, err := c.ListComments(ref.RepoPath(), ref.Number)
@@ -75,12 +95,22 @@ func terminalState(st PRState) (bool, string) {
return false, "" return false, ""
} }
// MaxPollFailures is how many consecutive failed polls of the same PR are
// tolerated before Watch gives up. The abort fires on the 20th failed tick, so
// at watchpr's default 60s interval a watch rides out ~19 minutes of failure.
const MaxPollFailures = 20
// Watch establishes a baseline for each ref, then polls on every tick until a // Watch establishes a baseline for each ref, then polls on every tick until a
// tracked PR changes meaningfully, returning the first such change. A PR that is // tracked PR changes meaningfully, returning the first such change. A PR that is
// already terminal (merged/closed) at baseline is reported immediately rather // already terminal (merged/closed) at baseline is reported immediately rather
// than polled forever. Poll errors are handed to onError and never stop the // than polled forever. Transient poll errors are handed to onError and the loop
// loop; only a baseline fetch error aborts. onBaseline, if set, fires once after // continues, but never blindly: a baseline fetch error, an authentication
// all baselines are captured and before the first tick. // failure surviving a token re-mint, a 404 from the PR lookup itself (the repo
// is gone, renamed, or no longer visible), and MaxPollFailures consecutive
// failures of one PR all abort, because a watcher that sees nothing must not
// look healthy.
// onBaseline, if set, fires once after all baselines are captured and before the
// first tick.
func Watch(f StateFetcher, refs []PRRef, agentLogin string, ticks <-chan time.Time, onBaseline func(), onError func(PRRef, error)) (WatchResult, error) { func Watch(f StateFetcher, refs []PRRef, agentLogin string, ticks <-chan time.Time, onBaseline func(), onError func(PRRef, error)) (WatchResult, error) {
prev := make(map[string]PRState, len(refs)) prev := make(map[string]PRState, len(refs))
for _, ref := range refs { for _, ref := range refs {
@@ -96,16 +126,25 @@ func Watch(f StateFetcher, refs []PRRef, agentLogin string, ticks <-chan time.Ti
if onBaseline != nil { if onBaseline != nil {
onBaseline() onBaseline()
} }
fails := make(map[string]int, len(refs))
for range ticks { for range ticks {
for _, ref := range refs { for _, ref := range refs {
key := ref.String() key := ref.String()
cur, err := f.FetchState(ref, agentLogin) cur, err := f.FetchState(ref, agentLogin)
if err != nil { if err != nil {
if IsAuthError(err) || IsPRGone(err) {
return WatchResult{}, fmt.Errorf("polling %s: %w", key, err)
}
fails[key]++
if onError != nil { if onError != nil {
onError(ref, err) onError(ref, err)
} }
if fails[key] >= MaxPollFailures {
return WatchResult{}, fmt.Errorf("polling %s: giving up after %d consecutive failures: %w", key, fails[key], err)
}
continue continue
} }
fails[key] = 0
if changed, reason := MeaningfulChange(prev[key], cur); changed { if changed, reason := MeaningfulChange(prev[key], cur); changed {
return WatchResult{Ref: ref, Reason: reason, State: cur}, nil return WatchResult{Ref: ref, Reason: reason, State: cur}, nil
} }
+517
View File
@@ -2,6 +2,11 @@ package agent
import ( import (
"errors" "errors"
"fmt"
"net/http"
"net/http/httptest"
"strings"
"sync/atomic"
"testing" "testing"
"time" "time"
) )
@@ -273,6 +278,78 @@ func TestWatchBaselineErrorAborts(t *testing.T) {
} }
} }
// The production hang, end to end: a watched PR stays open across several polls,
// then is squash-merged and its branch deleted, so the commit-status endpoint
// 404s. Driven through a real *GiteaClient, the watch loop must still detect the
// merge on the poll it happens. Before the fix, FetchState returned an error on
// that poll (the 404 masked the merge), so the loop reported only poll errors
// and never exited -- exactly the 37-minute hang seen in production.
func TestWatchDetectsMergeWhenCommitGone(t *testing.T) {
const sha = "cafebabecafebabe"
var polls atomic.Int32 // number of PR fetches so far
mux := http.NewServeMux()
mux.HandleFunc("/api/v1/repos/unkin/repo/pulls/7", func(w http.ResponseWriter, r *http.Request) {
n := polls.Add(1)
if n >= 4 { // baseline + two unchanged polls, then merged
_, _ = fmt.Fprintf(w, `{"number":7,"state":"closed","merged":true,"mergeable":true,"head":{"sha":%q}}`, sha)
return
}
_, _ = fmt.Fprintf(w, `{"number":7,"state":"open","merged":false,"mergeable":true,"head":{"sha":%q}}`, sha)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/commits/"+sha+"/status", func(w http.ResponseWriter, r *http.Request) {
if polls.Load() >= 4 { // branch deleted post-merge: commit is gone
w.WriteHeader(http.StatusNotFound)
_, _ = fmt.Fprint(w, `{"message":"not found"}`)
return
}
_, _ = fmt.Fprint(w, `{"state":"success"}`)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/issues/7/comments", func(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprint(w, `[]`)
})
srv := httptest.NewServer(mux)
defer srv.Close()
c := &GiteaClient{BaseURL: srv.URL, Token: "t", HTTP: srv.Client()}
ref := PRRef{Owner: "unkin", Repo: "repo", Number: 7}
// A real ticker so the loop advances on its own; a hang (the bug) is caught
// by the timeout below instead of blocking the suite.
tk := time.NewTicker(5 * time.Millisecond)
defer tk.Stop()
var pollErr atomic.Pointer[error]
type outcome struct {
res WatchResult
err error
}
done := make(chan outcome, 1)
go func() {
res, err := Watch(c, []PRRef{ref}, "unkin-agent", tk.C, nil,
func(_ PRRef, e error) { pollErr.Store(&e) })
done <- outcome{res, err}
}()
select {
case o := <-done:
if o.err != nil {
t.Fatalf("Watch: %v", o.err)
}
if o.res.Reason != "PR merged" {
t.Errorf("reason = %q, want %q", o.res.Reason, "PR merged")
}
if p := pollErr.Load(); p != nil {
t.Errorf("no poll error expected once a 404 status is tolerated, got: %v", *p)
}
case <-time.After(3 * time.Second):
var got error
if p := pollErr.Load(); p != nil {
got = *p
}
t.Fatalf("Watch hung: a merge with a gone head commit was never detected (last poll error: %v)", got)
}
}
func TestCountNonAgentComments(t *testing.T) { func TestCountNonAgentComments(t *testing.T) {
comments := []Comment{ comments := []Comment{
{User: User{Login: "unkin-agent"}}, {User: User{Login: "unkin-agent"}},
@@ -284,3 +361,443 @@ func TestCountNonAgentComments(t *testing.T) {
t.Errorf("countNonAgentComments = %d, want 2", n) t.Errorf("countNonAgentComments = %d, want 2", n)
} }
} }
// The production failure: the Vault-minted token expired mid-watch and every
// poll 401'd, which the loop logged as a warning and polled past forever. An
// auth error that survived the client's re-mint must end the watch with an
// error so watchpr exits non-zero instead of watching blind.
func TestWatchAbortsOnAuthError(t *testing.T) {
open := base()
merged := base()
merged.State = "closed"
merged.Merged = true
f := &fakeFetcher{
states: []PRState{open, open, merged},
errs: []error{nil, &APIError{Method: "GET", Path: "/p", StatusCode: 401, Body: "invalid token"}, nil},
}
warned := 0
ticks := make(chan time.Time, 2)
ticks <- time.Now()
ticks <- time.Now()
_, err := Watch(f, []PRRef{open.Ref}, "unkin-agent", ticks, nil, func(PRRef, error) { warned++ })
if err == nil {
t.Fatal("Watch should return the auth failure, not keep polling")
}
if !IsAuthError(err) {
t.Errorf("Watch error = %v, want an auth error", err)
}
if warned != 0 {
t.Errorf("auth failure was logged as a warning %d time(s); it must abort", warned)
}
if f.calls != 2 {
t.Errorf("fetch calls = %d, want 2 (baseline + the failing poll)", f.calls)
}
}
// A 5xx keeps its retry behaviour: warn and poll on.
func TestWatchContinuesPastServerError(t *testing.T) {
open := base()
merged := base()
merged.State = "closed"
merged.Merged = true
f := &fakeFetcher{
states: []PRState{open, open, merged},
errs: []error{nil, &APIError{Method: "GET", Path: "/p", StatusCode: 502, Body: "bad gateway"}, nil},
}
warned := 0
ticks := make(chan time.Time, 2)
ticks <- time.Now()
ticks <- time.Now()
res, err := Watch(f, []PRRef{open.Ref}, "unkin-agent", ticks, nil, func(PRRef, error) { warned++ })
if err != nil {
t.Fatalf("Watch: %v", err)
}
if warned != 1 {
t.Errorf("warnings = %d, want 1", warned)
}
if res.Reason != "PR merged" {
t.Errorf("reason = %q, want %q", res.Reason, "PR merged")
}
}
// The production failure: a watched repo was renamed mid-watch, so every poll
// 404'd (Gitea hides a repo the caller may not see rather than 403ing) and the
// loop warned past it forever while reporting nothing. A 404 on a tracked PR
// must end the watch with an error naming that PR.
func TestWatchAbortsOnMidRunNotFound(t *testing.T) {
const sha = "deadbeefdeadbeef"
var polls atomic.Int32
mux := http.NewServeMux()
mux.HandleFunc("/api/v1/repos/unkin/repo/pulls/7", func(w http.ResponseWriter, r *http.Request) {
if polls.Add(1) > 1 { // repo renamed/made private after the baseline
w.WriteHeader(http.StatusNotFound)
_, _ = fmt.Fprint(w, `{"message":"Not Found"}`)
return
}
_, _ = fmt.Fprintf(w, `{"number":7,"state":"open","merged":false,"mergeable":true,"head":{"sha":%q}}`, sha)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/commits/"+sha+"/status", func(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprint(w, `{"state":"success"}`)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/issues/7/comments", func(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprint(w, `[]`)
})
srv := httptest.NewServer(mux)
defer srv.Close()
c := &GiteaClient{BaseURL: srv.URL, Token: "t", HTTP: srv.Client()}
ref := PRRef{Owner: "unkin", Repo: "repo", Number: 7}
tk := time.NewTicker(5 * time.Millisecond)
defer tk.Stop()
var warned atomic.Int32
done := make(chan error, 1)
go func() {
_, err := Watch(c, []PRRef{ref}, "unkin-agent", tk.C, nil,
func(PRRef, error) { warned.Add(1) })
done <- err
}()
select {
case err := <-done:
if err == nil {
t.Fatal("Watch should abort on a mid-run 404, not keep polling")
}
if !IsNotFound(err) {
t.Errorf("Watch error = %v, want a 404", err)
}
if !strings.Contains(err.Error(), ref.String()) {
t.Errorf("Watch error = %v, want it to name %s", err, ref.String())
}
if n := warned.Load(); n != 0 {
t.Errorf("404 was logged as a warning %d time(s); it must abort", n)
}
case <-time.After(3 * time.Second):
t.Fatal("Watch hung: a vanished repo was warned past instead of aborting")
}
}
// A 404 from a sub-resource is not proof the PR is gone: an ingress can serve
// one during a Gitea rolling restart. Only the PR lookup itself is authoritative,
// so a comments 404 must warn and keep polling like any other transient failure,
// and still catch the merge that lands afterwards.
func TestWatchSurvivesCommentsNotFound(t *testing.T) {
const sha = "0badc0de0badc0de"
var polls atomic.Int32
mux := http.NewServeMux()
mux.HandleFunc("/api/v1/repos/unkin/repo/pulls/7", func(w http.ResponseWriter, r *http.Request) {
if polls.Add(1) >= 4 {
_, _ = fmt.Fprintf(w, `{"number":7,"state":"closed","merged":true,"mergeable":true,"head":{"sha":%q}}`, sha)
return
}
_, _ = fmt.Fprintf(w, `{"number":7,"state":"open","merged":false,"mergeable":true,"head":{"sha":%q}}`, sha)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/commits/"+sha+"/status", func(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprint(w, `{"state":"success"}`)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/issues/7/comments", func(w http.ResponseWriter, r *http.Request) {
if n := polls.Load(); n == 2 || n == 3 { // proxy blip across two polls
w.WriteHeader(http.StatusNotFound)
_, _ = fmt.Fprint(w, `{"message":"Not Found"}`)
return
}
_, _ = fmt.Fprint(w, `[]`)
})
srv := httptest.NewServer(mux)
defer srv.Close()
c := &GiteaClient{BaseURL: srv.URL, Token: "t", HTTP: srv.Client()}
ref := PRRef{Owner: "unkin", Repo: "repo", Number: 7}
tk := time.NewTicker(5 * time.Millisecond)
defer tk.Stop()
var warned atomic.Int32
type outcome struct {
res WatchResult
err error
}
done := make(chan outcome, 1)
go func() {
res, err := Watch(c, []PRRef{ref}, "unkin-agent", tk.C, nil,
func(PRRef, error) { warned.Add(1) })
done <- outcome{res, err}
}()
select {
case o := <-done:
if o.err != nil {
t.Fatalf("Watch: %v (a comments 404 must not be terminal)", o.err)
}
if o.res.Reason != "PR merged" {
t.Errorf("reason = %q, want %q", o.res.Reason, "PR merged")
}
if n := warned.Load(); n != 2 {
t.Errorf("warnings = %d, want 2", n)
}
case <-time.After(3 * time.Second):
t.Fatal("Watch hung: a comments 404 must warn and keep polling")
}
}
// A comments 404 costs a poll from the same budget as any other failure: it must
// not be free, and a permanently 404ing sub-resource must still end the watch.
func TestWatchCommentsNotFoundCountsTowardCap(t *testing.T) {
const sha = "1badc0de1badc0de"
mux := http.NewServeMux()
mux.HandleFunc("/api/v1/repos/unkin/repo/pulls/7", func(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprintf(w, `{"number":7,"state":"open","merged":false,"mergeable":true,"head":{"sha":%q}}`, sha)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/commits/"+sha+"/status", func(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprint(w, `{"state":"success"}`)
})
var comments atomic.Int32
mux.HandleFunc("/api/v1/repos/unkin/repo/issues/7/comments", func(w http.ResponseWriter, r *http.Request) {
if comments.Add(1) > 1 { // healthy at baseline, gone from the first poll on
w.WriteHeader(http.StatusNotFound)
_, _ = fmt.Fprint(w, `{"message":"Not Found"}`)
return
}
_, _ = fmt.Fprint(w, `[]`)
})
srv := httptest.NewServer(mux)
defer srv.Close()
c := &GiteaClient{BaseURL: srv.URL, Token: "t", HTTP: srv.Client()}
ref := PRRef{Owner: "unkin", Repo: "repo", Number: 7}
tk := time.NewTicker(time.Millisecond)
defer tk.Stop()
var warned atomic.Int32
done := make(chan error, 1)
go func() {
_, err := Watch(c, []PRRef{ref}, "unkin-agent", tk.C, nil,
func(PRRef, error) { warned.Add(1) })
done <- err
}()
select {
case err := <-done:
if err == nil {
t.Fatal("Watch should give up once the comments 404 stops being transient")
}
if !strings.Contains(err.Error(), "consecutive failures") {
t.Errorf("Watch error = %v, want it to report the failure cap", err)
}
if n := warned.Load(); n != MaxPollFailures {
t.Errorf("warnings = %d, want %d", n, MaxPollFailures)
}
case <-time.After(3 * time.Second):
t.Fatal("Watch hung: a permanently 404ing comments endpoint must hit the cap")
}
}
// The PR lookup is the call whose 404 means the PR is gone, so it aborts on the
// very first occurrence rather than spending the failure budget.
func TestWatchAbortsOnFirstPRLookupNotFound(t *testing.T) {
const sha = "2badc0de2badc0de"
var polls atomic.Int32
mux := http.NewServeMux()
mux.HandleFunc("/api/v1/repos/unkin/repo/pulls/7", func(w http.ResponseWriter, r *http.Request) {
if polls.Add(1) > 1 {
w.WriteHeader(http.StatusNotFound)
_, _ = fmt.Fprint(w, `{"message":"Not Found"}`)
return
}
_, _ = fmt.Fprintf(w, `{"number":7,"state":"open","merged":false,"mergeable":true,"head":{"sha":%q}}`, sha)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/commits/"+sha+"/status", func(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprint(w, `{"state":"success"}`)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/issues/7/comments", func(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprint(w, `[]`)
})
srv := httptest.NewServer(mux)
defer srv.Close()
c := &GiteaClient{BaseURL: srv.URL, Token: "t", HTTP: srv.Client()}
ref := PRRef{Owner: "unkin", Repo: "repo", Number: 7}
tk := time.NewTicker(5 * time.Millisecond)
defer tk.Stop()
done := make(chan error, 1)
go func() {
_, err := Watch(c, []PRRef{ref}, "unkin-agent", tk.C, nil,
func(PRRef, error) { t.Errorf("a PR-lookup 404 must abort, not warn") })
done <- err
}()
select {
case err := <-done:
if !IsNotFound(err) {
t.Fatalf("Watch error = %v, want a 404", err)
}
if n := polls.Load(); n != 2 {
t.Errorf("PR fetches = %d, want 2 (baseline + the 404 that aborts)", n)
}
case <-time.After(3 * time.Second):
t.Fatal("Watch hung: a vanished PR must abort")
}
}
// A 5xx blip must not kill a long watch: it warns, keeps polling, and still
// catches the merge that lands afterwards.
func TestWatchSurvivesTransientServerError(t *testing.T) {
const sha = "feedfacefeedface"
var polls atomic.Int32
mux := http.NewServeMux()
mux.HandleFunc("/api/v1/repos/unkin/repo/pulls/7", func(w http.ResponseWriter, r *http.Request) {
switch n := polls.Add(1); {
case n == 2 || n == 3: // gateway blip across two polls
w.WriteHeader(http.StatusBadGateway)
_, _ = fmt.Fprint(w, `bad gateway`)
case n >= 4:
_, _ = fmt.Fprintf(w, `{"number":7,"state":"closed","merged":true,"mergeable":true,"head":{"sha":%q}}`, sha)
default:
_, _ = fmt.Fprintf(w, `{"number":7,"state":"open","merged":false,"mergeable":true,"head":{"sha":%q}}`, sha)
}
})
mux.HandleFunc("/api/v1/repos/unkin/repo/commits/"+sha+"/status", func(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprint(w, `{"state":"success"}`)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/issues/7/comments", func(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprint(w, `[]`)
})
srv := httptest.NewServer(mux)
defer srv.Close()
c := &GiteaClient{BaseURL: srv.URL, Token: "t", HTTP: srv.Client()}
ref := PRRef{Owner: "unkin", Repo: "repo", Number: 7}
tk := time.NewTicker(5 * time.Millisecond)
defer tk.Stop()
var warned atomic.Int32
type outcome struct {
res WatchResult
err error
}
done := make(chan outcome, 1)
go func() {
res, err := Watch(c, []PRRef{ref}, "unkin-agent", tk.C, nil,
func(PRRef, error) { warned.Add(1) })
done <- outcome{res, err}
}()
select {
case o := <-done:
if o.err != nil {
t.Fatalf("Watch: %v", o.err)
}
if o.res.Reason != "PR merged" {
t.Errorf("reason = %q, want %q", o.res.Reason, "PR merged")
}
if n := warned.Load(); n != 2 {
t.Errorf("warnings = %d, want 2", n)
}
case <-time.After(3 * time.Second):
t.Fatal("Watch hung: a transient 5xx must not stop the watch")
}
}
// pollFailures scripts a fetcher whose polls fail with a 502 at the given call
// indexes (0 is the baseline); the final call returns merged.
func pollFailures(calls int, failAt map[int]bool) *fakeFetcher {
open, merged := base(), base()
merged.State = "closed"
merged.Merged = true
f := &fakeFetcher{states: make([]PRState, calls), errs: make([]error, calls)}
for i := range calls {
f.states[i] = open
if failAt[i] {
f.errs[i] = &APIError{Method: "GET", Path: "/p", StatusCode: 502, Body: "bad gateway"}
}
}
f.states[calls-1] = merged
return f
}
// A permanently wedged endpoint (5xx forever) must eventually give up instead of
// warning on every tick for the life of the process.
func TestWatchAbortsAfterConsecutiveFailures(t *testing.T) {
failAt := map[int]bool{}
for i := 1; i <= MaxPollFailures; i++ {
failAt[i] = true
}
f := pollFailures(MaxPollFailures+1, failAt)
warned := 0
ticks := make(chan time.Time, MaxPollFailures)
for range MaxPollFailures {
ticks <- time.Now()
}
close(ticks)
_, err := Watch(f, []PRRef{base().Ref}, "unkin-agent", ticks, nil, func(PRRef, error) { warned++ })
if err == nil {
t.Fatal("Watch should give up once the failures stop being transient")
}
if !strings.Contains(err.Error(), "consecutive failures") {
t.Errorf("Watch error = %v, want it to report the failure cap", err)
}
if warned != MaxPollFailures {
t.Errorf("warnings = %d, want %d", warned, MaxPollFailures)
}
if f.calls != MaxPollFailures+1 {
t.Errorf("fetch calls = %d, want %d", f.calls, MaxPollFailures+1)
}
}
// The cap counts consecutive failures only: a single successful poll clears it,
// so an intermittent endpoint is watched indefinitely and the merge is caught.
func TestWatchFailureCountResetsOnSuccess(t *testing.T) {
const runs = MaxPollFailures - 1
failAt := map[int]bool{}
for i := 1; i <= runs; i++ { // first run of failures
failAt[i] = true
}
for i := runs + 2; i <= 2*runs+1; i++ { // second run, after one good poll
failAt[i] = true
}
f := pollFailures(2*runs+3, failAt)
ticks := make(chan time.Time, 2*runs+2)
for range 2*runs + 2 {
ticks <- time.Now()
}
close(ticks)
res, err := Watch(f, []PRRef{base().Ref}, "unkin-agent", ticks, nil, func(PRRef, error) {})
if err != nil {
t.Fatalf("Watch: %v (a successful poll must reset the failure count)", err)
}
if res.Reason != "PR merged" {
t.Errorf("reason = %q, want %q", res.Reason, "PR merged")
}
}
// Anonymous watching of a public repo must poll on without a credential in
// sight: no token, no mint, no exit until something actually changes.
func TestWatchAnonymousKeepsPolling(t *testing.T) {
open := base()
f := &fakeFetcher{states: []PRState{open}}
ticks := make(chan time.Time, 2)
ticks <- time.Now()
ticks <- time.Now()
close(ticks)
res, err := Watch(f, []PRRef{open.Ref}, "unkin-agent", ticks, nil,
func(_ PRRef, e error) { t.Errorf("unexpected poll error: %v", e) })
if err != nil {
t.Fatalf("Watch: %v", err)
}
if res.Reason != "" {
t.Errorf("reason = %q, want no change reported", res.Reason)
}
if f.calls != 3 {
t.Errorf("fetch calls = %d, want 3 (baseline + two polls)", f.calls)
}
}
+18
View File
@@ -42,6 +42,12 @@ contents:
mode: 0755 mode: 0755
owner: root owner: root
group: root group: root
- src: dist/agentvault
dst: /usr/bin/agentvault
file_info:
mode: 0755
owner: root
group: root
# Shell completions (generated by scripts/build-rpm.sh before packaging). # Shell completions (generated by scripts/build-rpm.sh before packaging).
- src: dist/completions/agentpr.bash - src: dist/completions/agentpr.bash
@@ -80,3 +86,15 @@ contents:
dst: /usr/share/fish/vendor_completions.d/agentws.fish dst: /usr/share/fish/vendor_completions.d/agentws.fish
file_info: file_info:
mode: 0644 mode: 0644
- src: dist/completions/agentvault.bash
dst: /usr/share/bash-completion/completions/agentvault
file_info:
mode: 0644
- src: dist/completions/_agentvault
dst: /usr/share/zsh/site-functions/_agentvault
file_info:
mode: 0644
- src: dist/completions/agentvault.fish
dst: /usr/share/fish/vendor_completions.d/agentvault.fish
file_info:
mode: 0644
+3 -3
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# #
# Package the (already built) agentpr, watchpr and agentws binaries into an RPM # Package the (already built) agentpr, watchpr, agentws and agentvault binaries into an RPM
# with nfpm, bundling generated bash/zsh/fish shell completions. # with nfpm, bundling generated bash/zsh/fish shell completions.
# Usage: scripts/build-rpm.sh [version] (version defaults to $CI_COMMIT_TAG) # Usage: scripts/build-rpm.sh [version] (version defaults to $CI_COMMIT_TAG)
# #
@@ -12,7 +12,7 @@ cd "${ROOT_DIR}"
VERSION="${1:-${CI_COMMIT_TAG:-0.0.0-dev}}" VERSION="${1:-${CI_COMMIT_TAG:-0.0.0-dev}}"
VERSION="${VERSION#v}" # strip a leading v VERSION="${VERSION#v}" # strip a leading v
PACKAGE="agent-tools" PACKAGE="agent-tools"
BINARIES=(agentpr watchpr agentws) BINARIES=(agentpr watchpr agentws agentvault)
DIST="dist" DIST="dist"
for b in "${BINARIES[@]}"; do for b in "${BINARIES[@]}"; do
@@ -37,7 +37,7 @@ export PACKAGE_VERSION="${VERSION}"
export PACKAGE_RELEASE="1" export PACKAGE_RELEASE="1"
export PACKAGE_ARCH="amd64" export PACKAGE_ARCH="amd64"
export PACKAGE_PLATFORM="linux" export PACKAGE_PLATFORM="linux"
export PACKAGE_DESCRIPTION="CLI tools for Gitea automation as the unkin-agent user: agentpr (create PRs/comments), watchpr (poll PRs and alert on meaningful change) and agentws (manage per-branch git worktrees)" export PACKAGE_DESCRIPTION="CLI tools for automation as the unkin-agent user: agentpr (create PRs/comments), watchpr (poll PRs and alert on meaningful change), agentws (manage per-branch git worktrees) and agentvault (deterministic Vault flows)"
export PACKAGE_MAINTAINER="Ben Vincent <ben@unkin.net>" export PACKAGE_MAINTAINER="Ben Vincent <ben@unkin.net>"
export PACKAGE_HOMEPAGE="https://git.unkin.net/unkin/agent-tools" export PACKAGE_HOMEPAGE="https://git.unkin.net/unkin/agent-tools"
export PACKAGE_LICENSE="MIT" export PACKAGE_LICENSE="MIT"