8 Commits

Author SHA1 Message Date
benvin d9645ec5e4 Merge pull request 'Add agentvault seed-oauth for oauth2-proxy credentials' (#9) from benvin/agentvault-seed-oauth into main
ci/woodpecker/tag/release Pipeline was successful
Reviewed-on: #9
2026-08-30 15:41:13 +10:00
unkin-agent 155392a809 Add agentvault seed-oauth for oauth2-proxy credentials
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Seeding an oauth2-proxy secret by hand means an agent shell-plumbing a
client secret and a cookie secret, which the classifier blocks. seed-oauth
does it in one self-contained invocation: it reads the KV path, fills in
only the keys that are missing, preserves everything else and prints key
names and the new version, never a value.

- Add SeedOAuth in internal/agent: read-modify-write of the client_id,
  client_secret and cookie_secret keys with per-key created/kept/rotated
  actions and a no-op when nothing changed.
- Generate secrets from 32 crypto/rand bytes; cookie_secret is base64url so
  it decodes to exactly the 32 bytes oauth2-proxy requires.
- Add ReadKVOptional (missing secret = empty) and WriteKVAny (non-string
  fields survive a round trip) to the KV-v2 client.
- Wire the seed-oauth subcommand and document it in README and AGENTS.md.
- Cover fresh create, patch-preserves-client_secret, other-key
  preservation, --rotate, idempotence, denial errors and secret leakage.
2026-08-30 15:31:25 +10:00
benvin 47118215b4 Merge pull request 'Make agentpr's Vault gitea creds path selectable' (#8) from benvin/agentpr-creds-path into main
ci/woodpecker/tag/release Pipeline was successful
Reviewed-on: #8
2026-08-30 09:36:22 +10:00
unkin-agent 26cd05e961 Make the Vault gitea creds path selectable
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
agentpr always read gitea/creds/unkin-agent from a bare const, so a service
like repospawner could not run it as its own Gitea identity.

- Replace the GiteaCredsPath const with a function: GITEA_CREDS_PATH when set,
  otherwise gitea/creds/<AGENT_LOGIN>. Unset env still resolves to
  gitea/creds/unkin-agent, so existing callers are unchanged.
- Thread the creds path through fetchGiteaToken/readGiteaCreds instead of
  reading a package-level const, and report it in the error messages.
- Make agentpr's help text login-agnostic and document both variables.
2026-08-30 00:50:35 +10:00
benvin 68805a8cde Merge pull request 'Add agentvault with a seed-outpost subcommand' (#7) from benvin/agentvault-seed-outpost into main
ci/woodpecker/tag/release Pipeline was successful
Reviewed-on: #7
2026-08-29 21:58:01 +10:00
unkin-agent 61bb464e32 Add agentvault with a seed-outpost subcommand
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Interactive agents are classifier-blocked from plumbing credentials through
a shell, so seeding an Authentik outpost token into Vault KV needs to happen
inside one binary invocation that never exposes the secret.

- Add cmd/agentvault, a fourth CLI sharing the agentpr Vault AppRole login
  (role_id only, VAULT_ADDR/AGENT_APPROLE_ROLE_ID defaults unchanged).
- Add `agentvault seed-outpost`: read the Authentik API token from
  kv/service/authentik/agent-api-token (field `token`, falling back to
  `api_token`), exact-match the outpost by name via the instances search,
  fetch its key from /api/v3/core/tokens/<identifier>/view_key/ and write it
  to --dest-path under --dest-key.
- Print only the outpost name, token identifier, dest path and new KV
  version; keep secret material out of results, errors and logs.
- Distinguish the failure stages (login, KV read denied, outpost missing,
  view_key, KV write denied) with ErrVaultDenied/ErrVaultNotFound/
  ErrOutpostNotFound sentinels and actionable messages.
- Add internal/agent vaultkv.go (AppRole-authenticated KV-v2 client) and
  authentik.go (outpost search + view_key) for reuse by future flows.
- Cover the happy path, idempotent re-run, field fallback and every failure
  mode with httptest servers, including a leak check on error strings.
- Wire agentvault into the Makefile, build-rpm.sh, nfpm contents, release
  cross-builds/assets, README and AGENTS.md.
2026-08-29 21:03:31 +10:00
benvin 60b08f1198 Merge pull request 'watchpr: fix watch mode never detecting changes' (#5) from benvin/watchpr-watchloop-fix into main
ci/woodpecker/tag/release Pipeline was successful
Reviewed-on: #5
2026-08-15 15:00:01 +10:00
unkin-agent fda3761ead watchpr: fix watch mode hanging when a merged PR's head commit is gone
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
FetchState fetched the PR (merged=true) but then failed the whole state
fetch when CommitStatus 404'd for a head commit that no longer existed
(the branch was deleted after a squash/rebase merge). The merge signal was
discarded, so the watch loop treated every post-merge poll as a transient
error and never exited -- the 37-minute hang seen in production.

- Add a typed APIError carrying the HTTP status so callers can detect a 404
  without parsing error strings.
- FetchState now tolerates a 404 from CommitStatus (commit gone => no CI
  status) and returns the authoritative merged/closed PR state.
- Regression tests: FetchState survives a 404 status; the full watch loop,
  driven through a real client, detects a merge whose head commit is gone
  (both fail/hang before the fix).
2026-08-15 13:23:25 +10:00
22 changed files with 2190 additions and 66 deletions
+3 -3
View File
@@ -17,7 +17,7 @@ steps:
memory: 2Gi memory: 2Gi
cpu: 2 cpu: 2
# Build both binaries into dist/ (consumed by the RPM step) plus the # Build every binary into dist/ (consumed by the RPM step) plus the
# cross-platform binaries attached to the Gitea release. Each tool is a # cross-platform binaries attached to the Gitea release. Each tool is a
# separate main package, so they are built individually per os/arch. # separate main package, so they are built individually per os/arch.
- name: build - name: build
@@ -28,7 +28,7 @@ steps:
# for the shell instead of substituting them (as pipeline vars) at parse # for the shell instead of substituting them (as pipeline vars) at parse
# time. ${CI_COMMIT_TAG} is a real Woodpecker var and stays single-$. # time. ${CI_COMMIT_TAG} is a real Woodpecker var and stays single-$.
- | - |
for entry in "agentpr:./cmd/agentpr" "watchpr:./cmd/watchpr" "agentws:./cmd/agentws"; do for entry in "agentpr:./cmd/agentpr" "watchpr:./cmd/watchpr" "agentws:./cmd/agentws" "agentvault:./cmd/agentvault"; do
name="$${entry%%:*}"; pkg="$${entry##*:}" name="$${entry%%:*}"; pkg="$${entry##*:}"
for osarch in linux/amd64 linux/arm64 darwin/amd64 darwin/arm64; do for osarch in linux/amd64 linux/arm64 darwin/amd64 darwin/arm64; do
os="$${osarch%/*}"; arch="$${osarch#*/}" os="$${osarch%/*}"; arch="$${osarch#*/}"
@@ -135,7 +135,7 @@ steps:
# root; the package step writes the RPM to dist/. Generate a checksums # root; the package step writes the RPM to dist/. Generate a checksums
# manifest over everything we attach so downloads can be verified. # manifest over everything we attach so downloads can be verified.
RPM=$$(ls dist/*.rpm 2>/dev/null | head -1) RPM=$$(ls dist/*.rpm 2>/dev/null | head -1)
ASSETS="agentpr-linux-amd64 agentpr-linux-arm64 agentpr-darwin-amd64 agentpr-darwin-arm64 watchpr-linux-amd64 watchpr-linux-arm64 watchpr-darwin-amd64 watchpr-darwin-arm64 agentws-linux-amd64 agentws-linux-arm64 agentws-darwin-amd64 agentws-darwin-arm64" ASSETS="agentpr-linux-amd64 agentpr-linux-arm64 agentpr-darwin-amd64 agentpr-darwin-arm64 watchpr-linux-amd64 watchpr-linux-arm64 watchpr-darwin-amd64 watchpr-darwin-arm64 agentws-linux-amd64 agentws-linux-arm64 agentws-darwin-amd64 agentws-darwin-arm64 agentvault-linux-amd64 agentvault-linux-arm64 agentvault-darwin-amd64 agentvault-darwin-arm64"
[ -n "$$RPM" ] && ASSETS="$$ASSETS $$RPM" [ -n "$$RPM" ] && ASSETS="$$ASSETS $$RPM"
sha256sum $$ASSETS > sha256sums.txt sha256sum $$ASSETS > sha256sums.txt
tea releases assets create "${CI_COMMIT_TAG}" $$ASSETS sha256sums.txt \ tea releases assets create "${CI_COMMIT_TAG}" $$ASSETS sha256sums.txt \
+65 -9
View File
@@ -3,8 +3,10 @@
## Project Overview ## Project Overview
This repo ships several Gitea-automation CLIs in one RPM (`agent-tools`). They This repo ships several Gitea-automation CLIs in one RPM (`agent-tools`). They
act as the `unkin-agent` user by minting a scoped Gitea token from Vault, so act as an agent user (`unkin-agent` by default) by minting a scoped Gitea token
actions are attributed to the agent rather than to whoever runs the tool. from Vault, so actions are attributed to the agent rather than to whoever runs
the tool. Setting `AGENT_LOGIN` selects a different agent identity, so a service
like repospawner can run these tools as itself.
- **`agentpr`** — create pull requests and post PR comments as `unkin-agent` - **`agentpr`** — create pull requests and post PR comments as `unkin-agent`
(fixes the "tea posts as Ben" attribution problem). Subcommands: (fixes the "tea posts as Ben" attribution problem). Subcommands:
@@ -29,13 +31,18 @@ parsing, watch-state comparison, git worktree helpers).
cmd/agentpr/main.go # agentpr CLI (pr create / pr comment / whoami) cmd/agentpr/main.go # agentpr CLI (pr create / pr comment / whoami)
cmd/watchpr/main.go # watchpr CLI (poll + meaningful-change exit) cmd/watchpr/main.go # watchpr CLI (poll + meaningful-change exit)
cmd/agentws/main.go # agentws CLI (new / list / rm / clean / token / credential) cmd/agentws/main.go # agentws CLI (new / list / rm / clean / token / credential)
cmd/agentvault/main.go # agentvault CLI (seed-outpost / seed-oauth)
internal/agent/ # shared plumbing: internal/agent/ # shared plumbing:
token.go # env config + in-process Gitea-token cache token.go # env config + in-process Gitea-token cache
vault.go # AppRole login + read gitea/creds/unkin-agent vault.go # AppRole login + read the gitea creds path
gitea.go # Gitea REST client (PR create/get, comments, status, whoami) gitea.go # Gitea REST client (PR create/get, comments, status, whoami)
parse.go # owner/repo#N and owner/repo parsing parse.go # owner/repo#N and owner/repo parsing
watch.go # PRState snapshot + MeaningfulChange comparison watch.go # PRState snapshot + MeaningfulChange comparison
git.go # git worktree/clone/fetch helpers (os/exec, no go-git) git.go # git worktree/clone/fetch helpers (os/exec, no go-git)
vaultkv.go # AppRole-authenticated Vault client + KV-v2 read/write
authentik.go # Authentik REST client (outpost search, token view_key)
seedoutpost.go # seed-outpost flow (Authentik token -> Vault KV)
seedoauth.go # seed-oauth flow (oauth2-proxy credential set in Vault KV)
go.mod # module git.unkin.net/unkin/agent-tools go.mod # module git.unkin.net/unkin/agent-tools
Makefile # build / test / lint / completions / rpm / version-bump Makefile # build / test / lint / completions / rpm / version-bump
packaging/nfpm.yaml # nfpm spec (envsubst-templated) for the RPM (all binaries) packaging/nfpm.yaml # nfpm spec (envsubst-templated) for the RPM (all binaries)
@@ -53,7 +60,9 @@ All tools call `agent.GiteaToken()`, which (once per process):
1. AppRole login: `POST $VAULT_ADDR/v1/auth/approle/login` with `role_id` only 1. AppRole login: `POST $VAULT_ADDR/v1/auth/approle/login` with `role_id` only
(no `secret_id`) → `client_token`. (no `secret_id`) → `client_token`.
2. `GET $VAULT_ADDR/v1/gitea/creds/unkin-agent` with `X-Vault-Token``.data.token`. 2. `GET $VAULT_ADDR/v1/<creds path>` with `X-Vault-Token``.data.token`, where
the creds path is `GITEA_CREDS_PATH` if set, else `gitea/creds/$AGENT_LOGIN`
(so unset env still reads `gitea/creds/unkin-agent`).
Config via env (all have defaults): Config via env (all have defaults):
@@ -62,17 +71,19 @@ Config via env (all have defaults):
| `VAULT_ADDR` | `https://vault.service.consul:8200` | Vault/OpenBao address | | `VAULT_ADDR` | `https://vault.service.consul:8200` | Vault/OpenBao address |
| `AGENT_APPROLE_ROLE_ID` | built-in default | AppRole role_id (overridable) | | `AGENT_APPROLE_ROLE_ID` | built-in default | AppRole role_id (overridable) |
| `GITEA_URL` | `https://git.unkin.net` | Gitea base URL | | `GITEA_URL` | `https://git.unkin.net` | Gitea base URL |
| `AGENT_LOGIN` | `unkin-agent` | login whose comments watchpr ignores; agentws git identity | | `AGENT_LOGIN` | `unkin-agent` | agent identity: selects `gitea/creds/<login>`; login whose comments watchpr ignores; agentws git identity |
| `GITEA_CREDS_PATH` | `gitea/creds/$AGENT_LOGIN` | Vault path minting the Gitea token (wins over `AGENT_LOGIN`) |
| `AGENTWS_SRC_ROOT` | `~/src/prodenv` | agentws source-of-truth checkout root | | `AGENTWS_SRC_ROOT` | `~/src/prodenv` | agentws source-of-truth checkout root |
| `AGENTWS_ROOT` | `~/.cache/agentws` | agentws worktree root | | `AGENTWS_ROOT` | `~/.cache/agentws` | agentws worktree root |
| `AGENTWS_OWNER` | `unkin` | Gitea org that owns agentws-managed repos | | `AGENTWS_OWNER` | `unkin` | Gitea org that owns agentws-managed repos |
| `AUTHENTIK_URL` | `https://identity.k8s.syd1.au.unkin.net` | Authentik base URL (`agentvault`) |
### agentws git auth (ephemeral credential helper) ### agentws git auth (ephemeral credential helper)
Gitea tokens are ~1h ephemeral, so `agentws` never bakes one into a remote URL Gitea tokens are ~1h ephemeral, so `agentws` never bakes one into a remote URL
or config. `agentws token` prints a fresh token; `agentws credential get` or config. `agentws token` prints a fresh token; `agentws credential get`
implements the git credential protocol (reads the key=value request on stdin, implements the git credential protocol (reads the key=value request on stdin,
and for the configured Gitea host only emits `username=unkin-agent` + and for the configured Gitea host only emits `username=$AGENT_LOGIN` +
`password=<fresh token>`). `agentws new` wires this per worktree — it enables `password=<fresh token>`). `agentws new` wires this per worktree — it enables
`extensions.worktreeConfig` on the repo once, then writes `user.name`, `extensions.worktreeConfig` on the repo once, then writes `user.name`,
`user.email` and `credential.helper = !<agentws> credential` to the `user.email` and `credential.helper = !<agentws> credential` to the
@@ -85,7 +96,7 @@ to keep the default branch current.
## Build ## Build
```bash ```bash
make build # -> dist/agentpr, dist/watchpr, dist/agentws (CGO disabled, static) make build # -> dist/agentpr, dist/watchpr, dist/agentws, dist/agentvault (CGO disabled, static)
``` ```
Requires Go 1.21+. Dependency: `github.com/spf13/cobra` (CLI). Requires Go 1.21+. Dependency: `github.com/spf13/cobra` (CLI).
@@ -97,8 +108,8 @@ make rpm # build all binaries + package into dist/*.rpm via nfpm
``` ```
`scripts/build-rpm.sh` generates bash/zsh/fish completions from the built `scripts/build-rpm.sh` generates bash/zsh/fish completions from the built
binaries and bundles them alongside `/usr/bin/agentpr`, `/usr/bin/watchpr` and binaries and bundles them alongside `/usr/bin/agentpr`, `/usr/bin/watchpr`,
`/usr/bin/agentws`. `/usr/bin/agentws` and `/usr/bin/agentvault`.
On a `v*` tag the release pipeline builds the RPM and `PUT`s it to the On a `v*` tag the release pipeline builds the RPM and `PUT`s it to the
artifactapi `rpm-internal` repo, then cuts a Gitea release. artifactapi `rpm-internal` repo, then cuts a Gitea release.
@@ -120,10 +131,55 @@ alerting transitions), request-body construction, and the Vault+Gitea client
against `httptest` servers (fake AppRole login + gitea creds + PR create / against `httptest` servers (fake AppRole login + gitea creds + PR create /
comment / whoami / status). No live Vault/Gitea access is required for tests. comment / whoami / status). No live Vault/Gitea access is required for tests.
## agentvault seed-outpost
`agentvault seed-outpost --outpost <name> --dest-path <kv/path>` does the whole
flow in-process:
1. AppRole login (shared `approleLogin`), then KV-v2 read of
`kv/service/authentik/agent-api-token` (field `token`, falling back to
`api_token`).
2. `GET /api/v3/outposts/instances/?search=<name>` — Authentik's `search` is a
substring match, so the exact `name` is re-checked client-side.
3. `GET /api/v3/core/tokens/<token_identifier>/view_key/` for the key.
4. KV-v2 write to `--dest-path` under `--dest-key` (default `token`).
Only the outpost name, token identifier, dest path and new KV version are
printed. Errors are wrapped per stage (login / read denied / outpost missing /
view_key / write denied) via the `ErrVaultDenied`, `ErrVaultNotFound` and
`ErrOutpostNotFound` sentinels.
## agentvault seed-oauth
`agentvault seed-oauth --path <kv/path> --client-id <id>` makes a KV-v2 path
hold a complete oauth2-proxy credential set, in-process:
1. AppRole login (shared `approleLogin`), then a KV-v2 read via
`ReadKVOptional` — a 404 or a deleted version means "empty", not an error,
so the first seed of a path works.
2. Desired keys are computed over the existing map: `client_id` from the flag
(`kept`/`created`/`updated`), `client_secret` and `cookie_secret` generated
from 32 `crypto/rand` bytes only when absent or when `--rotate` is set
(`kept`/`created`/`rotated`). `cookie_secret` is base64url so it decodes to
exactly the 32 bytes oauth2-proxy demands; `client_secret` is standard
base64.
3. Any other key on the path is carried through unchanged (`preserved`), which
is why the write goes through `WriteKVAny` rather than `WriteKV`.
4. The write is skipped entirely when nothing changed; the command then prints
`version: unchanged`.
Only key names, per-key actions and the new KV version are printed. Errors are
wrapped per stage (login / read denied / write denied) via `ErrVaultDenied`.
## Gotchas ## Gotchas
- `watchpr` exits 0 with no output changes on `--once` (just prints state). - `watchpr` exits 0 with no output changes on `--once` (just prints state).
- The token cache is process-wide (`sync.Once`); tests call the unexported - The token cache is process-wide (`sync.Once`); tests call the unexported
`fetchGiteaToken` to avoid it. `fetchGiteaToken` to avoid it.
- `agentvault` never puts a secret in an error string: Vault decode failures and
Authentik `view_key` responses are reported without their bodies, and
`seed-oauth` reports key names only.
- `--rotate` regenerates the `client_secret` too, which then no longer matches
the IdP provider unless that is rotated alongside.
- CI "combined status" comes from `/commits/{sha}/status`; an empty head SHA - CI "combined status" comes from `/commits/{sha}/status`; an empty head SHA
yields an empty state without an API call. yields an empty state without an API call.
+1 -1
View File
@@ -1,6 +1,6 @@
# All shipped binaries and the package path each is built from. Both tools live # All shipped binaries and the package path each is built from. Both tools live
# under cmd/; the module root ships no binary of its own. # under cmd/; the module root ships no binary of its own.
BINARIES := agentpr watchpr agentws BINARIES := agentpr watchpr agentws agentvault
DIST := dist DIST := dist
VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo dev) VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo dev)
GOFLAGS := -ldflags="-s -w -X main.version=$(VERSION)" GOFLAGS := -ldflags="-s -w -X main.version=$(VERSION)"
+88 -11
View File
@@ -1,21 +1,25 @@
# agent-tools # agent-tools
Small Gitea-automation CLIs, shipped together in one RPM (`agent-tools`). They Small Gitea-automation CLIs, shipped together in one RPM (`agent-tools`). They
act as the **`unkin-agent`** user by minting a scoped Gitea token from Vault, so act as an agent user (**`unkin-agent`** by default) by minting a scoped Gitea
automated PRs, comments and pushes are attributed to the agent — not to whoever token from Vault, so automated PRs, comments and pushes are attributed to the
happens to run the command. agent — not to whoever happens to run the command. Set `AGENT_LOGIN` to act as a
different agent identity.
- **`agentpr`** — create pull requests and post PR comments as `unkin-agent`. - **`agentpr`** — create pull requests and post PR comments as the agent user.
- **`watchpr`** — poll one or more PRs and exit when one changes in a way worth - **`watchpr`** — poll one or more PRs and exit when one changes in a way worth
acting on. acting on.
- **`agentws`** — manage per-branch git worktrees for `unkin-agent`, cloning - **`agentws`** — manage per-branch git worktrees for `unkin-agent`, cloning
into Ben's source checkout and isolating agent work under the XDG cache. into Ben's source checkout and isolating agent work under the XDG cache.
- **`agentvault`** — run deterministic Vault flows in one invocation, so agents
never plumb secret material through a shell.
## How it gets a token ## How it gets a token
On first use each tool performs a Vault AppRole login (`role_id` only, no On first use each tool performs a Vault AppRole login (`role_id` only, no
`secret_id`), then reads `gitea/creds/unkin-agent` to obtain a short-lived Gitea `secret_id`), then reads `gitea/creds/$AGENT_LOGIN` — or `GITEA_CREDS_PATH` when
token, cached in-process for the run. set — to obtain a short-lived Gitea token, cached in-process for the run. With
neither variable set that is `gitea/creds/unkin-agent`, as before.
Everything is configured by environment variables, all with defaults: Everything is configured by environment variables, all with defaults:
@@ -24,15 +28,17 @@ Everything is configured by environment variables, all with defaults:
| `VAULT_ADDR` | `https://vault.service.consul:8200` | Vault/OpenBao address | | `VAULT_ADDR` | `https://vault.service.consul:8200` | Vault/OpenBao address |
| `AGENT_APPROLE_ROLE_ID` | built-in default | AppRole role_id (overridable) | | `AGENT_APPROLE_ROLE_ID` | built-in default | AppRole role_id (overridable) |
| `GITEA_URL` | `https://git.unkin.net` | Gitea base URL | | `GITEA_URL` | `https://git.unkin.net` | Gitea base URL |
| `AGENT_LOGIN` | `unkin-agent` | login whose comments `watchpr` ignores | | `AGENT_LOGIN` | `unkin-agent` | agent identity: selects `gitea/creds/<login>`, and the login whose comments `watchpr` ignores |
| `GITEA_CREDS_PATH` | `gitea/creds/$AGENT_LOGIN` | Vault path minting the Gitea token (wins over `AGENT_LOGIN`) |
| `AGENTWS_SRC_ROOT` | `~/src/prodenv` | source-of-truth checkout root (`agentws`) | | `AGENTWS_SRC_ROOT` | `~/src/prodenv` | source-of-truth checkout root (`agentws`) |
| `AGENTWS_ROOT` | `~/.cache/agentws` | worktree root (`agentws`) | | `AGENTWS_ROOT` | `~/.cache/agentws` | worktree root (`agentws`) |
| `AGENTWS_OWNER` | `unkin` | Gitea org that owns the repos (`agentws`) | | `AGENTWS_OWNER` | `unkin` | Gitea org that owns the repos (`agentws`) |
| `AUTHENTIK_URL` | `https://identity.k8s.syd1.au.unkin.net` | Authentik base URL (`agentvault`) |
## agentpr ## agentpr
```bash ```bash
# Verify identity (should print: unkin-agent) # Verify identity (prints the agent login, unkin-agent by default)
agentpr whoami agentpr whoami
# Open a PR # Open a PR
@@ -98,7 +104,7 @@ agentws rm ~/.cache/agentws/argocd-apps__benvin-my-change --delete-branch
# Remove every managed worktree and prune each source repo # Remove every managed worktree and prune each source repo
agentws clean agentws clean
# Print a fresh unkin-agent Gitea token # Print a fresh Gitea token for the agent login
agentws token agentws token
``` ```
@@ -110,7 +116,7 @@ persists one in a remote URL or in git config. Instead it wires itself as an
- `agentws token` prints a fresh token to stdout (handy for scripts). - `agentws token` prints a fresh token to stdout (handy for scripts).
- `agentws credential get` speaks the git credential protocol on stdin and, for - `agentws credential get` speaks the git credential protocol on stdin and, for
the configured Gitea host only, emits `username=unkin-agent` + the configured Gitea host only, emits `username=$AGENT_LOGIN` +
`password=<fresh token>`. `password=<fresh token>`.
`agentws new` sets this up per worktree without touching the shared checkout: it `agentws new` sets this up per worktree without touching the shared checkout: it
@@ -121,10 +127,81 @@ the **per-worktree** config. Clone/fetch use the same helper via a transient
removal `agentws` fetches in `~/src/prodenv/<repo>` so its default branch stays removal `agentws` fetches in `~/src/prodenv/<repo>` so its default branch stays
current. current.
## agentvault
Deterministic Vault flows, each a single self-contained invocation: the tool
reads and writes the secrets itself, and prints only identifiers.
### seed-outpost
Copy an Authentik outpost's token into Vault KV-v2. `agentvault` reads the
Authentik API token from `kv/service/authentik/agent-api-token`, resolves the
named outpost's `token_identifier`, fetches its key via
`/api/v3/core/tokens/<identifier>/view_key/` and writes it to the destination
KV path. The token value is never printed or logged.
```bash
agentvault seed-outpost \
--outpost k8s-outpost \
--dest-path kubernetes/namespace/authentik/default/outpost-token
```
```
outpost: k8s-outpost
token_identifier: ak-outpost-k8s-outpost
dest: kv/kubernetes/namespace/authentik/default/outpost-token
version: 3
```
Re-running is safe: it writes a new KV version. Flags: `--outpost` and
`--dest-path` are required; `--dest-key` (default `token`), `--kv-mount`
(default `kv`), `--token-path` (default `service/authentik/agent-api-token`) and
`--authentik-url` override the rest.
Errors name the failing stage: AppRole login, KV read denied (policy not
applied), outpost not found (terraform not applied), `view_key` failure, or KV
write denied.
### seed-oauth
Make a Vault KV-v2 path hold a complete oauth2-proxy credential set. It is a
read-modify-write: `client_id` is set from the flag, `client_secret` and
`cookie_secret` are generated (32 bytes from `crypto/rand`) only when missing,
every other key on the path is written back untouched, and nothing is written
at all when the secret is already correct. `cookie_secret` is base64url so it
decodes to exactly the 32 bytes oauth2-proxy requires.
```bash
agentvault seed-oauth \
--path kubernetes/namespace/repospawner/default/oauth-credentials \
--client-id 4f1c…
```
```
path: kv/kubernetes/namespace/repospawner/default/oauth-credentials
keys: client_id, client_secret, cookie_secret
client_id: created
client_secret: kept
cookie_secret: created
version: 4
```
That is the common case: the provider's `client_secret` already lives on the
path, so only the missing keys are added. A run with nothing to do prints
`version: unchanged` and issues no write.
Flags: `--path` and `--client-id` are required; `--kv-mount` (default `kv`) and
`--rotate` override the rest. `--rotate` regenerates both secrets — only use it
when the IdP provider's secret is being rotated alongside, since a rotated
`client_secret` no longer matches the provider.
Errors name the failing stage: AppRole login, KV read denied, or KV write
denied. Only key names, actions and the KV version are printed.
## Build & package ## Build & package
```bash ```bash
make build # -> dist/agentpr, dist/watchpr, dist/agentws make build # -> dist/agentpr, dist/watchpr, dist/agentws, dist/agentvault
make test # go test -race ./... make test # go test -race ./...
make rpm # build + package dist/agent-tools-<version>-1.x86_64.rpm make rpm # build + package dist/agent-tools-<version>-1.x86_64.rpm
``` ```
+7 -7
View File
@@ -1,7 +1,7 @@
// Command agentpr manages Gitea pull requests and comments as the unkin-agent // Command agentpr manages Gitea pull requests and comments as an agent user. It
// user. It obtains a scoped Gitea token from Vault (AppRole login, then reads // obtains a scoped Gitea token from Vault (AppRole login, then reads
// gitea/creds/unkin-agent) so actions are attributed to the agent rather than // gitea/creds/<AGENT_LOGIN>, or GITEA_CREDS_PATH when set) so actions are
// to whoever runs the tool. // attributed to that agent rather than to whoever runs the tool.
// //
// agentpr pr create --repo owner/repo --base main --head feature --title T --body B // agentpr pr create --repo owner/repo --base main --head feature --title T --body B
// agentpr pr comment --repo owner/repo --pr 12 --body "..." // agentpr pr comment --repo owner/repo --pr 12 --body "..."
@@ -33,8 +33,8 @@ func main() {
func newRootCmd() *cobra.Command { func newRootCmd() *cobra.Command {
root := &cobra.Command{ root := &cobra.Command{
Use: "agentpr", Use: "agentpr",
Short: "Manage Gitea PRs and comments as the unkin-agent user.", Short: "Manage Gitea PRs and comments as an agent user.",
Long: "agentpr manages Gitea pull requests and comments as unkin-agent, using a\nGitea token minted from Vault (AppRole login + gitea/creds/unkin-agent).", Long: "agentpr manages Gitea pull requests and comments as an agent user, using a\nGitea token minted from Vault (AppRole login + gitea/creds/<AGENT_LOGIN>).\nSet AGENT_LOGIN to act as another agent identity, or GITEA_CREDS_PATH to name\nthe Vault creds path outright.",
Version: version, Version: version,
SilenceUsage: true, SilenceUsage: true,
} }
@@ -146,7 +146,7 @@ func newPRCommentCmd() *cobra.Command {
func newWhoamiCmd() *cobra.Command { func newWhoamiCmd() *cobra.Command {
return &cobra.Command{ return &cobra.Command{
Use: "whoami", Use: "whoami",
Short: "Print the authenticated Gitea login (should be unkin-agent)", Short: "Print the authenticated Gitea login (the identity PRs are opened as)",
SilenceUsage: true, SilenceUsage: true,
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
c, err := client() c, err := client()
+127
View File
@@ -0,0 +1,127 @@
// Command agentvault runs deterministic Vault flows for agents in a single
// invocation, so credentials are never plumbed through a shell. It authenticates
// with the same Vault AppRole as agentpr (role_id only, no secret_id).
//
// agentvault seed-outpost --outpost <name> --dest-path <kv/path>
// agentvault seed-oauth --path <kv/path> --client-id <id>
package main
import (
"fmt"
"os"
"strings"
"git.unkin.net/unkin/agent-tools/internal/agent"
"github.com/spf13/cobra"
)
var version = "dev"
func main() {
if err := newRootCmd().Execute(); err != nil {
os.Exit(1)
}
}
// newRootCmd builds the agentvault command tree. Separated from main so tests
// can execute it against httptest servers.
func newRootCmd() *cobra.Command {
root := &cobra.Command{
Use: "agentvault",
Short: "Run deterministic Vault flows as the agent AppRole.",
Long: "agentvault performs self-contained Vault flows for agents: it logs in with the\nagent AppRole and moves secret material between systems without ever printing it.",
Version: version,
SilenceUsage: true,
}
root.SetVersionTemplate("{{.Version}}\n")
root.AddCommand(newSeedOutpostCmd(), newSeedOAuthCmd(), newVersionCmd())
return root
}
func newSeedOutpostCmd() *cobra.Command {
opts := agent.SeedOutpostOptions{}
cmd := &cobra.Command{
Use: "seed-outpost",
Short: "Copy an Authentik outpost token into Vault KV",
Long: "Read the Authentik API token from Vault KV, resolve the named outpost's\n" +
"token_identifier, fetch its key and write it to a Vault KV path. Re-running\n" +
"writes a new KV version. The token value is never printed or logged.",
SilenceUsage: true,
RunE: func(cmd *cobra.Command, args []string) error {
opts.VaultAddr = agent.VaultAddr()
opts.RoleID = agent.RoleID()
res, err := agent.SeedOutpost(opts)
if err != nil {
return err
}
out := cmd.OutOrStdout()
_, _ = fmt.Fprintf(out, "outpost: %s\n", res.Outpost)
_, _ = fmt.Fprintf(out, "token_identifier: %s\n", res.TokenIdentifier)
_, _ = fmt.Fprintf(out, "dest: %s/%s\n", res.KVMount, res.DestPath)
_, _ = fmt.Fprintf(out, "version: %d\n", res.Version)
return nil
},
}
f := cmd.Flags()
f.StringVar(&opts.Outpost, "outpost", "", "Authentik outpost name (required)")
f.StringVar(&opts.DestPath, "dest-path", "", "KV-v2 path to write the token to, e.g. kubernetes/namespace/authentik/default/outpost-token (required)")
f.StringVar(&opts.DestKey, "dest-key", agent.DefaultDestKey, "Field to write the token under")
f.StringVar(&opts.KVMount, "kv-mount", agent.DefaultKVMount, "KV-v2 mount holding both the API token and the destination")
f.StringVar(&opts.TokenPath, "token-path", agent.DefaultOutpostTokenPath, "KV-v2 path of the Authentik API token")
f.StringVar(&opts.AuthentikURL, "authentik-url", agent.AuthentikURL(), "Authentik base URL")
_ = cmd.MarkFlagRequired("outpost")
_ = cmd.MarkFlagRequired("dest-path")
return cmd
}
func newSeedOAuthCmd() *cobra.Command {
opts := agent.SeedOAuthOptions{}
cmd := &cobra.Command{
Use: "seed-oauth",
Short: "Seed an oauth2-proxy credential set into Vault KV",
Long: "Make a Vault KV-v2 path hold a complete oauth2-proxy credential set: the\n" +
"given client_id, plus a client_secret and a 32-byte cookie_secret that are\n" +
"generated only when missing (or with --rotate). Existing keys are preserved\n" +
"and nothing is written when the secret is already correct. Secret values are\n" +
"never printed or logged.",
SilenceUsage: true,
RunE: func(cmd *cobra.Command, args []string) error {
opts.VaultAddr = agent.VaultAddr()
opts.RoleID = agent.RoleID()
res, err := agent.SeedOAuth(opts)
if err != nil {
return err
}
out := cmd.OutOrStdout()
_, _ = fmt.Fprintf(out, "path: %s/%s\n", res.KVMount, res.Path)
_, _ = fmt.Fprintf(out, "keys: %s\n", strings.Join(res.KeyNames(), ", "))
for _, k := range res.Keys {
_, _ = fmt.Fprintf(out, " %-14s %s\n", k.Name+":", k.Action)
}
if res.Changed {
_, _ = fmt.Fprintf(out, "version: %d\n", res.Version)
} else {
_, _ = fmt.Fprintf(out, "version: unchanged\n")
}
return nil
},
}
f := cmd.Flags()
f.StringVar(&opts.Path, "path", "", "KV-v2 path holding the credentials, e.g. kubernetes/namespace/repospawner/default/oauth-credentials (required)")
f.StringVar(&opts.ClientID, "client-id", "", "OIDC client id to store (required)")
f.StringVar(&opts.KVMount, "kv-mount", agent.DefaultKVMount, "KV-v2 mount holding the path")
f.BoolVar(&opts.Rotate, "rotate", false, "Regenerate client_secret and cookie_secret even when they already exist")
_ = cmd.MarkFlagRequired("path")
_ = cmd.MarkFlagRequired("client-id")
return cmd
}
func newVersionCmd() *cobra.Command {
return &cobra.Command{
Use: "version",
Short: "Print the version",
Run: func(cmd *cobra.Command, args []string) { fmt.Println(version) },
SilenceUsage: true,
}
}
+180
View File
@@ -0,0 +1,180 @@
package main
import (
"bytes"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
const (
apiToken = "ak-api-token-secret"
outpostKey = "outpost-key-secret"
destPath = "kubernetes/namespace/authentik/default/outpost-token"
)
// fakeEstate serves the Vault (approle + KV-v2 read/write) and Authentik
// (outpost search + view_key) endpoints the seed flow needs.
func fakeEstate(t *testing.T) (vaultURL, authentikURL string) {
t.Helper()
vmux := http.NewServeMux()
vmux.HandleFunc("/v1/auth/approle/login", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `{"auth":{"client_token":"s.vaulttoken"}}`)
})
vmux.HandleFunc("/v1/kv/data/service/authentik/agent-api-token", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `{"data":{"data":{"token":"`+apiToken+`"}}}`)
})
vmux.HandleFunc("/v1/kv/data/"+destPath, func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `{"data":{"version":7}}`)
})
vs := httptest.NewServer(vmux)
t.Cleanup(vs.Close)
amux := http.NewServeMux()
amux.HandleFunc("/api/v3/outposts/instances/", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `{"results":[{"pk":"1","name":"k8s-outpost","token_identifier":"ak-outpost-k8s"}]}`)
})
amux.HandleFunc("/api/v3/core/tokens/ak-outpost-k8s/view_key/", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `{"key":"`+outpostKey+`"}`)
})
as := httptest.NewServer(amux)
t.Cleanup(as.Close)
return vs.URL, as.URL
}
// The command prints identifiers and the KV version only — never a secret.
func TestSeedOutpostOutputHasNoSecrets(t *testing.T) {
vaultURL, authentikURL := fakeEstate(t)
t.Setenv("VAULT_ADDR", vaultURL)
t.Setenv("AGENT_APPROLE_ROLE_ID", "role-xyz")
var out bytes.Buffer
cmd := newRootCmd()
cmd.SetOut(&out)
cmd.SetErr(&out)
cmd.SetArgs([]string{
"seed-outpost",
"--outpost", "k8s-outpost",
"--dest-path", destPath,
"--authentik-url", authentikURL,
})
if err := cmd.Execute(); err != nil {
t.Fatalf("Execute: %v", err)
}
got := out.String()
for _, want := range []string{"k8s-outpost", "ak-outpost-k8s", "kv/" + destPath, "version: 7"} {
if !strings.Contains(got, want) {
t.Errorf("output missing %q:\n%s", want, got)
}
}
for _, secret := range []string{apiToken, outpostKey} {
if strings.Contains(got, secret) {
t.Fatalf("output leaks a secret:\n%s", got)
}
}
}
const oauthPath = "kubernetes/namespace/repospawner/default/oauth-credentials"
// fakeOAuthVault serves approle login plus a KV-v2 path that already holds a
// client_secret, and records what gets written back.
func fakeOAuthVault(t *testing.T, existing map[string]string) (vaultURL string, written *map[string]string) {
t.Helper()
writes := map[string]string{}
mux := http.NewServeMux()
mux.HandleFunc("/v1/auth/approle/login", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `{"auth":{"client_token":"s.vaulttoken"}}`)
})
mux.HandleFunc("/v1/kv/data/"+oauthPath, func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodPost {
var body struct {
Data map[string]string `json:"data"`
}
_ = json.NewDecoder(r.Body).Decode(&body)
for k, v := range body.Data {
writes[k] = v
}
_, _ = io.WriteString(w, `{"data":{"version":4}}`)
return
}
payload, _ := json.Marshal(map[string]any{"data": map[string]any{"data": existing}})
_, _ = w.Write(payload)
})
srv := httptest.NewServer(mux)
t.Cleanup(srv.Close)
return srv.URL, &writes
}
// The command prints key names and the KV version only — never a value.
func TestSeedOAuthOutputHasNoSecrets(t *testing.T) {
const existingSecret = "existing-client-secret-value"
vaultURL, written := fakeOAuthVault(t, map[string]string{"client_secret": existingSecret})
t.Setenv("VAULT_ADDR", vaultURL)
t.Setenv("AGENT_APPROLE_ROLE_ID", "role-xyz")
var out bytes.Buffer
cmd := newRootCmd()
cmd.SetOut(&out)
cmd.SetErr(&out)
cmd.SetArgs([]string{"seed-oauth", "--path", oauthPath, "--client-id", "mediamark-client-id"})
if err := cmd.Execute(); err != nil {
t.Fatalf("Execute: %v", err)
}
got := out.String()
for _, want := range []string{"kv/" + oauthPath, "client_id, client_secret, cookie_secret", "client_secret: kept", "cookie_secret: created", "version: 4"} {
if !strings.Contains(got, want) {
t.Errorf("output missing %q:\n%s", want, got)
}
}
for key, value := range *written {
if key == "client_id" {
continue
}
if strings.Contains(got, value) {
t.Fatalf("output leaks the %s value:\n%s", key, got)
}
}
if strings.Contains(got, existingSecret) {
t.Fatalf("output leaks the existing client_secret:\n%s", got)
}
}
func TestSeedOAuthRequiresFlags(t *testing.T) {
for name, args := range map[string][]string{
"no path": {"seed-oauth", "--client-id", "mediamark-client-id"},
"no client-id": {"seed-oauth", "--path", oauthPath},
} {
t.Run(name, func(t *testing.T) {
cmd := newRootCmd()
cmd.SetOut(io.Discard)
cmd.SetErr(io.Discard)
cmd.SetArgs(args)
if err := cmd.Execute(); err == nil {
t.Fatal("Execute() = nil, want a missing-required-flag error")
}
})
}
}
func TestSeedOutpostRequiresFlags(t *testing.T) {
for name, args := range map[string][]string{
"no outpost": {"seed-outpost", "--dest-path", destPath},
"no dest-path": {"seed-outpost", "--outpost", "k8s-outpost"},
} {
t.Run(name, func(t *testing.T) {
cmd := newRootCmd()
cmd.SetOut(io.Discard)
cmd.SetErr(io.Discard)
cmd.SetArgs(args)
if err := cmd.Execute(); err == nil {
t.Fatal("Execute() = nil, want a missing-required-flag error")
}
})
}
}
+93
View File
@@ -0,0 +1,93 @@
package agent
import (
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strings"
)
// ErrOutpostNotFound marks a search that returned no exactly-named outpost.
var ErrOutpostNotFound = errors.New("outpost not found")
// AuthentikClient talks to the Authentik REST API with a bearer API token. The
// internal CA is in the OS trust store, so the default transport suffices.
type AuthentikClient struct {
BaseURL string
Token string
HTTP *http.Client
}
// NewAuthentikClient builds a client for the given Authentik base URL.
func NewAuthentikClient(baseURL, token string) *AuthentikClient {
return &AuthentikClient{BaseURL: strings.TrimRight(baseURL, "/"), Token: token, HTTP: httpClient}
}
// Outpost is the subset of Authentik's outpost object we need.
type Outpost struct {
PK string `json:"pk"`
Name string `json:"name"`
TokenIdentifier string `json:"token_identifier"`
}
// get issues an authenticated GET and decodes into out. Error text never
// includes a successful response body, which may carry key material.
func (c *AuthentikClient) get(path string, out any) error {
req, err := http.NewRequest(http.MethodGet, c.BaseURL+path, nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+c.Token)
req.Header.Set("Accept", "application/json")
resp, err := c.HTTP.Do(req)
if err != nil {
return fmt.Errorf("authentik GET %s: %w", path, err)
}
defer func() { _ = resp.Body.Close() }()
data, _ := io.ReadAll(resp.Body)
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return fmt.Errorf("authentik GET %s: HTTP %d: %s", path, resp.StatusCode, strings.TrimSpace(string(data)))
}
if err := json.Unmarshal(data, out); err != nil {
return fmt.Errorf("authentik GET %s: decoding response: %w", path, err)
}
return nil
}
// FindOutpost searches outpost instances and returns the one whose name matches
// exactly (search is a substring match, so the exact name is re-checked here).
func (c *AuthentikClient) FindOutpost(name string) (Outpost, error) {
var out struct {
Results []Outpost `json:"results"`
}
path := "/api/v3/outposts/instances/?search=" + url.QueryEscape(name)
if err := c.get(path, &out); err != nil {
return Outpost{}, err
}
for _, o := range out.Results {
if o.Name == name {
return o, nil
}
}
return Outpost{}, fmt.Errorf("authentik outpost %q: %w (searched %d result(s))", name, ErrOutpostNotFound, len(out.Results))
}
// TokenKey returns the key behind a token identifier
// (GET /api/v3/core/tokens/<identifier>/view_key/).
func (c *AuthentikClient) TokenKey(identifier string) (string, error) {
var out struct {
Key string `json:"key"`
}
path := "/api/v3/core/tokens/" + url.PathEscape(identifier) + "/view_key/"
if err := c.get(path, &out); err != nil {
return "", err
}
if out.Key == "" {
return "", fmt.Errorf("authentik view_key for %q: response has no key field", identifier)
}
return out.Key, nil
}
+60 -6
View File
@@ -8,8 +8,9 @@ import (
"testing" "testing"
) )
// fakeVault serves the AppRole login and gitea creds endpoints. // fakeVault serves the AppRole login and the gitea creds secret at credsPath
func fakeVault(t *testing.T, wantRoleID, giteaToken string) *httptest.Server { // only, so a read of any other path 404s.
func fakeVault(t *testing.T, wantRoleID, credsPath, giteaToken string) *httptest.Server {
t.Helper() t.Helper()
mux := http.NewServeMux() mux := http.NewServeMux()
mux.HandleFunc("/v1/auth/approle/login", func(w http.ResponseWriter, r *http.Request) { mux.HandleFunc("/v1/auth/approle/login", func(w http.ResponseWriter, r *http.Request) {
@@ -26,7 +27,7 @@ func fakeVault(t *testing.T, wantRoleID, giteaToken string) *httptest.Server {
} }
_, _ = io.WriteString(w, `{"auth":{"client_token":"s.vaulttoken"}}`) _, _ = io.WriteString(w, `{"auth":{"client_token":"s.vaulttoken"}}`)
}) })
mux.HandleFunc("/v1/"+GiteaCredsPath, func(w http.ResponseWriter, r *http.Request) { mux.HandleFunc("/v1/"+credsPath, func(w http.ResponseWriter, r *http.Request) {
if got := r.Header.Get("X-Vault-Token"); got != "s.vaulttoken" { if got := r.Header.Get("X-Vault-Token"); got != "s.vaulttoken" {
t.Errorf("X-Vault-Token = %q, want s.vaulttoken", got) t.Errorf("X-Vault-Token = %q, want s.vaulttoken", got)
} }
@@ -36,10 +37,10 @@ func fakeVault(t *testing.T, wantRoleID, giteaToken string) *httptest.Server {
} }
func TestFetchGiteaToken(t *testing.T) { func TestFetchGiteaToken(t *testing.T) {
srv := fakeVault(t, "role-xyz", "gitea-abc") srv := fakeVault(t, "role-xyz", "gitea/creds/unkin-agent", "gitea-abc")
defer srv.Close() defer srv.Close()
tok, err := fetchGiteaToken(srv.URL, "role-xyz") tok, err := fetchGiteaToken(srv.URL, "role-xyz", "gitea/creds/unkin-agent")
if err != nil { if err != nil {
t.Fatalf("fetchGiteaToken: %v", err) t.Fatalf("fetchGiteaToken: %v", err)
} }
@@ -57,7 +58,7 @@ func TestFetchGiteaTokenLoginError(t *testing.T) {
srv := httptest.NewServer(mux) srv := httptest.NewServer(mux)
defer srv.Close() defer srv.Close()
if _, err := fetchGiteaToken(srv.URL, "role-xyz"); err == nil { if _, err := fetchGiteaToken(srv.URL, "role-xyz", "gitea/creds/unkin-agent"); err == nil {
t.Fatal("expected error on 403 login") t.Fatal("expected error on 403 login")
} }
} }
@@ -163,6 +164,59 @@ func TestFetchState(t *testing.T) {
} }
} }
// A merged PR whose branch was deleted leaves its head commit unreachable, so
// the commit-status endpoint 404s. FetchState must still return the (merged)
// state rather than failing, otherwise the watch loop never sees the merge.
func TestFetchStateToleratesMissingCommit(t *testing.T) {
mux := http.NewServeMux()
mux.HandleFunc("/api/v1/repos/unkin/repo/pulls/7", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `{"number":7,"state":"closed","merged":true,"mergeable":true,"title":"feat","html_url":"u","head":{"sha":"cafebabecafebabe"}}`)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/commits/cafebabecafebabe/status", func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusNotFound)
_, _ = io.WriteString(w, `{"message":"not found"}`)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/issues/7/comments", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `[]`)
})
srv := httptest.NewServer(mux)
defer srv.Close()
c := &GiteaClient{BaseURL: srv.URL, Token: "t", HTTP: srv.Client()}
ref := PRRef{Owner: "unkin", Repo: "repo", Number: 7}
st, err := FetchState(c, ref, "unkin-agent")
if err != nil {
t.Fatalf("FetchState must tolerate a 404 status for a gone commit: %v", err)
}
if !st.Merged || st.State != "closed" {
t.Errorf("merged/state = %t/%q, want true/closed", st.Merged, st.State)
}
if st.CIStatus != "" {
t.Errorf("CIStatus = %q, want empty (no status for a gone commit)", st.CIStatus)
}
}
// A non-404 error from the status endpoint is still fatal: only "commit gone" is
// tolerated, not, say, an auth or server failure.
func TestFetchStateFailsOnNon404StatusError(t *testing.T) {
mux := http.NewServeMux()
mux.HandleFunc("/api/v1/repos/unkin/repo/pulls/7", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `{"number":7,"state":"open","merged":false,"mergeable":true,"head":{"sha":"cafebabecafebabe"}}`)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/commits/cafebabecafebabe/status", func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusInternalServerError)
_, _ = io.WriteString(w, `{"message":"boom"}`)
})
srv := httptest.NewServer(mux)
defer srv.Close()
c := &GiteaClient{BaseURL: srv.URL, Token: "t", HTTP: srv.Client()}
ref := PRRef{Owner: "unkin", Repo: "repo", Number: 7}
if _, err := FetchState(c, ref, "unkin-agent"); err == nil {
t.Fatal("FetchState should surface a 500 from the status endpoint")
}
}
func TestGiteaAPIError(t *testing.T) { func TestGiteaAPIError(t *testing.T) {
mux := http.NewServeMux() mux := http.NewServeMux()
mux.HandleFunc("/api/v1/repos/unkin/repo/pulls", func(w http.ResponseWriter, r *http.Request) { mux.HandleFunc("/api/v1/repos/unkin/repo/pulls", func(w http.ResponseWriter, r *http.Request) {
+22 -1
View File
@@ -3,12 +3,33 @@ package agent
import ( import (
"bytes" "bytes"
"encoding/json" "encoding/json"
"errors"
"fmt" "fmt"
"io" "io"
"net/http" "net/http"
"strings" "strings"
) )
// APIError is a non-2xx response from the Gitea API. It carries the status code
// so callers can react to specific failures (e.g. tolerate a 404 for a commit
// whose branch was deleted after a merge) instead of parsing error strings.
type APIError struct {
Method string
Path string
StatusCode int
Body string
}
func (e *APIError) Error() string {
return fmt.Sprintf("gitea %s %s: HTTP %d: %s", e.Method, e.Path, e.StatusCode, e.Body)
}
// isNotFound reports whether err is a Gitea 404.
func isNotFound(err error) bool {
var apiErr *APIError
return errors.As(err, &apiErr) && apiErr.StatusCode == http.StatusNotFound
}
// GiteaClient talks to the Gitea REST API as the agent user. // GiteaClient talks to the Gitea REST API as the agent user.
type GiteaClient struct { type GiteaClient struct {
BaseURL string BaseURL string
@@ -49,7 +70,7 @@ func (c *GiteaClient) do(method, path string, body any, out any) error {
defer func() { _ = resp.Body.Close() }() defer func() { _ = resp.Body.Close() }()
data, _ := io.ReadAll(resp.Body) data, _ := io.ReadAll(resp.Body)
if resp.StatusCode < 200 || resp.StatusCode >= 300 { if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return fmt.Errorf("gitea %s %s: HTTP %d: %s", method, path, resp.StatusCode, strings.TrimSpace(string(data))) return &APIError{Method: method, Path: path, StatusCode: resp.StatusCode, Body: strings.TrimSpace(string(data))}
} }
if out != nil && len(data) > 0 { if out != nil && len(data) > 0 {
if err := json.Unmarshal(data, out); err != nil { if err := json.Unmarshal(data, out); err != nil {
+183
View File
@@ -0,0 +1,183 @@
package agent
import (
"crypto/rand"
"encoding/base64"
"errors"
"fmt"
"sort"
)
const (
// OAuthClientIDKey, OAuthClientSecretKey and OAuthCookieSecretKey are the
// KV fields oauth2-proxy deployments read their credentials from.
OAuthClientIDKey = "client_id"
OAuthClientSecretKey = "client_secret"
OAuthCookieSecretKey = "cookie_secret"
// oauthSecretBytes is the raw length of generated secrets. oauth2-proxy
// rejects a cookie secret that does not decode to exactly 32 bytes.
oauthSecretBytes = 32
)
// Per-key outcomes reported by SeedOAuth.
const (
ActionCreated = "created"
ActionKept = "kept"
ActionRotated = "rotated"
ActionUpdated = "updated"
ActionPreserved = "preserved"
)
// SeedOAuthOptions configures SeedOAuth. The CLI supplies the defaults.
type SeedOAuthOptions struct {
VaultAddr string
RoleID string
KVMount string
Path string
ClientID string
Rotate bool
}
// SeedOAuthKey names a key present in the secret and what happened to it.
type SeedOAuthKey struct {
Name string
Action string
}
// SeedOAuthResult is the non-secret summary of a seed run. Version is zero when
// nothing changed and no write was issued.
type SeedOAuthResult struct {
KVMount string
Path string
Keys []SeedOAuthKey
Version int
Changed bool
}
// KeyNames returns the key names present in the secret after the run.
func (r SeedOAuthResult) KeyNames() []string {
names := make([]string, 0, len(r.Keys))
for _, k := range r.Keys {
names = append(names, k.Name)
}
return names
}
// SeedOAuth makes a Vault KV-v2 path hold a complete oauth2-proxy credential
// set: client_id from the caller, plus a client_secret and cookie_secret that
// are generated only when absent (or when Rotate is set). It is a
// read-modify-write, so keys it does not own survive untouched, and it writes
// nothing when the secret is already correct. Secret material never leaves this
// function: results and errors carry only key names.
func SeedOAuth(o SeedOAuthOptions) (SeedOAuthResult, error) {
var res SeedOAuthResult
if o.Path == "" {
return res, errors.New("a KV-v2 path is required")
}
if o.ClientID == "" {
return res, errors.New("a client id is required")
}
vc, err := NewVaultClient(o.VaultAddr, o.RoleID)
if err != nil {
return res, fmt.Errorf("vault approle login failed against %s (check VAULT_ADDR and AGENT_APPROLE_ROLE_ID): %w", o.VaultAddr, err)
}
existing, err := vc.ReadKVOptional(o.KVMount, o.Path)
if err != nil {
if errors.Is(err, ErrVaultDenied) {
return res, fmt.Errorf("reading %s/%s denied: the agent AppRole policy does not grant read on this path (apply the terraform-vault policy change): %w", o.KVMount, o.Path, err)
}
return res, fmt.Errorf("reading %s/%s: %w", o.KVMount, o.Path, err)
}
data := make(map[string]any, len(existing)+3)
for k, v := range existing {
data[k] = v
}
var (
keys []SeedOAuthKey
changed bool
)
current, _ := existing[OAuthClientIDKey].(string)
switch current {
case o.ClientID:
keys = append(keys, SeedOAuthKey{OAuthClientIDKey, ActionKept})
case "":
keys = append(keys, SeedOAuthKey{OAuthClientIDKey, ActionCreated})
changed = true
default:
keys = append(keys, SeedOAuthKey{OAuthClientIDKey, ActionUpdated})
changed = true
}
data[OAuthClientIDKey] = o.ClientID
for _, gen := range []struct {
name string
enc *base64.Encoding
}{
// oauth2-proxy accepts a standard-base64 client secret, but the cookie
// secret goes into a cookie and must be URL-safe.
{OAuthClientSecretKey, base64.StdEncoding},
{OAuthCookieSecretKey, base64.RawURLEncoding},
} {
current, _ := existing[gen.name].(string)
if current != "" && !o.Rotate {
keys = append(keys, SeedOAuthKey{gen.name, ActionKept})
continue
}
value, err := randomSecret(gen.enc)
if err != nil {
return res, fmt.Errorf("generating %s: %w", gen.name, err)
}
action := ActionCreated
if current != "" {
action = ActionRotated
}
data[gen.name] = value
keys = append(keys, SeedOAuthKey{gen.name, action})
changed = true
}
var others []string
for k := range existing {
switch k {
case OAuthClientIDKey, OAuthClientSecretKey, OAuthCookieSecretKey:
default:
others = append(others, k)
}
}
sort.Strings(others)
for _, k := range others {
keys = append(keys, SeedOAuthKey{k, ActionPreserved})
}
res = SeedOAuthResult{KVMount: o.KVMount, Path: o.Path, Keys: keys}
if !changed {
return res, nil
}
version, err := vc.WriteKVAny(o.KVMount, o.Path, data)
if err != nil {
if errors.Is(err, ErrVaultDenied) {
return SeedOAuthResult{}, fmt.Errorf("writing %s/%s denied: the agent AppRole policy does not grant create/update on this path (apply the terraform-vault policy change): %w", o.KVMount, o.Path, err)
}
return SeedOAuthResult{}, fmt.Errorf("writing %s/%s: %w", o.KVMount, o.Path, err)
}
res.Version = version
res.Changed = true
return res, nil
}
// randomSecret returns oauthSecretBytes of crypto/rand entropy in the given
// base64 encoding.
func randomSecret(enc *base64.Encoding) (string, error) {
buf := make([]byte, oauthSecretBytes)
if _, err := rand.Read(buf); err != nil {
return "", err
}
return enc.EncodeToString(buf), nil
}
+421
View File
@@ -0,0 +1,421 @@
package agent
import (
"encoding/base64"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strconv"
"strings"
"testing"
)
const (
oauthPath = "kubernetes/namespace/repospawner/default/oauth-credentials"
oauthClientID = "mediamark-client-id"
existingClientSec = "existing-client-secret-from-authentik"
existingCookieSec = "existing-cookie-secret-value-abcdefghij"
oauthExtraKeyValue = "extra-key-secret-value"
oauthVaultClientTok = "s.vaulttoken"
)
// oauthVaultStub is a KV-v2 stand-in that actually stores what is written, so
// read-modify-write behaviour can be asserted end to end.
type oauthVaultStub struct {
data map[string]any
exists bool
version int
readStatus int
writeStatus int
writes []map[string]any
}
func newOAuthVaultStub() *oauthVaultStub {
return &oauthVaultStub{readStatus: http.StatusOK, writeStatus: http.StatusOK}
}
// seed makes the path exist with the given fields at version 1.
func (v *oauthVaultStub) seed(data map[string]any) *oauthVaultStub {
v.data = data
v.exists = true
v.version = 1
return v
}
func (v *oauthVaultStub) server(t *testing.T) *httptest.Server {
t.Helper()
mux := http.NewServeMux()
mux.HandleFunc("/v1/auth/approle/login", func(w http.ResponseWriter, r *http.Request) {
var body map[string]string
_ = json.NewDecoder(r.Body).Decode(&body)
if _, ok := body["secret_id"]; ok {
t.Errorf("secret_id must not be sent")
}
_, _ = io.WriteString(w, `{"auth":{"client_token":"`+oauthVaultClientTok+`"}}`)
})
mux.HandleFunc("/v1/kv/data/"+oauthPath, func(w http.ResponseWriter, r *http.Request) {
if got := r.Header.Get("X-Vault-Token"); got != oauthVaultClientTok {
t.Errorf("X-Vault-Token = %q, want %q", got, oauthVaultClientTok)
}
switch r.Method {
case http.MethodGet:
if v.readStatus != http.StatusOK {
w.WriteHeader(v.readStatus)
_, _ = io.WriteString(w, `{"errors":["permission denied"]}`)
return
}
if !v.exists {
w.WriteHeader(http.StatusNotFound)
_, _ = io.WriteString(w, `{"errors":[]}`)
return
}
payload, _ := json.Marshal(map[string]any{
"data": map[string]any{"data": v.data, "metadata": map[string]any{"version": v.version}},
})
_, _ = w.Write(payload)
case http.MethodPost:
if v.writeStatus != http.StatusOK {
w.WriteHeader(v.writeStatus)
_, _ = io.WriteString(w, `{"errors":["permission denied"]}`)
return
}
var body struct {
Data map[string]any `json:"data"`
}
_ = json.NewDecoder(r.Body).Decode(&body)
v.writes = append(v.writes, body.Data)
v.data = body.Data
v.exists = true
v.version++
_, _ = io.WriteString(w, `{"data":{"version":`+strconv.Itoa(v.version)+`}}`)
default:
t.Errorf("unexpected method %s", r.Method)
}
})
srv := httptest.NewServer(mux)
t.Cleanup(srv.Close)
return srv
}
func oauthOpts(vaultURL string) SeedOAuthOptions {
return SeedOAuthOptions{
VaultAddr: vaultURL,
RoleID: "role-xyz",
KVMount: DefaultKVMount,
Path: oauthPath,
ClientID: oauthClientID,
}
}
// actions flattens a result into key -> action for order-independent asserts.
func actions(res SeedOAuthResult) map[string]string {
m := make(map[string]string, len(res.Keys))
for _, k := range res.Keys {
m[k.Name] = k.Action
}
return m
}
func stringField(t *testing.T, data map[string]any, key string) string {
t.Helper()
s, ok := data[key].(string)
if !ok {
t.Fatalf("written %s = %v, want a string", key, data[key])
}
return s
}
// assertDecodesTo32 fails unless the value is base64 of exactly 32 bytes, which
// is what oauth2-proxy requires of a cookie secret.
func assertDecodesTo32(t *testing.T, enc *base64.Encoding, value, name string) {
t.Helper()
raw, err := enc.DecodeString(value)
if err != nil {
t.Fatalf("%s is not valid base64: %v", name, err)
}
if len(raw) != oauthSecretBytes {
t.Errorf("%s decodes to %d bytes, want %d", name, len(raw), oauthSecretBytes)
}
}
func TestSeedOAuthFreshCreate(t *testing.T) {
v := newOAuthVaultStub()
res, err := SeedOAuth(oauthOpts(v.server(t).URL))
if err != nil {
t.Fatalf("SeedOAuth: %v", err)
}
if !res.Changed || res.Version != 1 {
t.Errorf("Changed=%v Version=%d, want a first write at version 1", res.Changed, res.Version)
}
for key, want := range map[string]string{
OAuthClientIDKey: ActionCreated,
OAuthClientSecretKey: ActionCreated,
OAuthCookieSecretKey: ActionCreated,
} {
if got := actions(res)[key]; got != want {
t.Errorf("%s action = %q, want %q", key, got, want)
}
}
if len(v.writes) != 1 {
t.Fatalf("%d writes, want exactly 1", len(v.writes))
}
w := v.writes[0]
if got := stringField(t, w, OAuthClientIDKey); got != oauthClientID {
t.Errorf("written client_id = %q, want %q", got, oauthClientID)
}
assertDecodesTo32(t, base64.StdEncoding, stringField(t, w, OAuthClientSecretKey), OAuthClientSecretKey)
assertDecodesTo32(t, base64.RawURLEncoding, stringField(t, w, OAuthCookieSecretKey), OAuthCookieSecretKey)
}
// The mediamark case: a client_secret already issued by Authentik must survive
// while the missing keys are filled in.
func TestSeedOAuthPreservesExistingClientSecret(t *testing.T) {
v := newOAuthVaultStub().seed(map[string]any{OAuthClientSecretKey: existingClientSec})
res, err := SeedOAuth(oauthOpts(v.server(t).URL))
if err != nil {
t.Fatalf("SeedOAuth: %v", err)
}
got := actions(res)
for key, want := range map[string]string{
OAuthClientIDKey: ActionCreated,
OAuthClientSecretKey: ActionKept,
OAuthCookieSecretKey: ActionCreated,
} {
if got[key] != want {
t.Errorf("%s action = %q, want %q", key, got[key], want)
}
}
if len(v.writes) != 1 {
t.Fatalf("%d writes, want exactly 1", len(v.writes))
}
if s := stringField(t, v.writes[0], OAuthClientSecretKey); s != existingClientSec {
t.Errorf("client_secret was replaced, want the existing value kept")
}
}
func TestSeedOAuthPreservesOtherKeys(t *testing.T) {
v := newOAuthVaultStub().seed(map[string]any{
OAuthClientIDKey: oauthClientID,
OAuthClientSecretKey: existingClientSec,
"redirect_url": "https://mediamark.unkin.net/oauth2/callback",
"extra": oauthExtraKeyValue,
})
res, err := SeedOAuth(oauthOpts(v.server(t).URL))
if err != nil {
t.Fatalf("SeedOAuth: %v", err)
}
got := actions(res)
for _, key := range []string{"redirect_url", "extra"} {
if got[key] != ActionPreserved {
t.Errorf("%s action = %q, want %q", key, got[key], ActionPreserved)
}
}
if len(v.writes) != 1 {
t.Fatalf("%d writes, want exactly 1", len(v.writes))
}
w := v.writes[0]
if stringField(t, w, "extra") != oauthExtraKeyValue {
t.Errorf("extra key was not written back unchanged")
}
if stringField(t, w, "redirect_url") != "https://mediamark.unkin.net/oauth2/callback" {
t.Errorf("redirect_url was not written back unchanged")
}
}
func TestSeedOAuthRotateRegenerates(t *testing.T) {
v := newOAuthVaultStub().seed(map[string]any{
OAuthClientIDKey: oauthClientID,
OAuthClientSecretKey: existingClientSec,
OAuthCookieSecretKey: existingCookieSec,
})
o := oauthOpts(v.server(t).URL)
o.Rotate = true
res, err := SeedOAuth(o)
if err != nil {
t.Fatalf("SeedOAuth: %v", err)
}
got := actions(res)
for key, want := range map[string]string{
OAuthClientIDKey: ActionKept,
OAuthClientSecretKey: ActionRotated,
OAuthCookieSecretKey: ActionRotated,
} {
if got[key] != want {
t.Errorf("%s action = %q, want %q", key, got[key], want)
}
}
if len(v.writes) != 1 {
t.Fatalf("%d writes, want exactly 1", len(v.writes))
}
w := v.writes[0]
if stringField(t, w, OAuthClientSecretKey) == existingClientSec {
t.Errorf("client_secret unchanged under --rotate")
}
if stringField(t, w, OAuthCookieSecretKey) == existingCookieSec {
t.Errorf("cookie_secret unchanged under --rotate")
}
assertDecodesTo32(t, base64.RawURLEncoding, stringField(t, w, OAuthCookieSecretKey), OAuthCookieSecretKey)
}
// A complete, correct secret must not produce a new KV version.
func TestSeedOAuthIdempotentWritesNothing(t *testing.T) {
v := newOAuthVaultStub()
url := v.server(t).URL
if _, err := SeedOAuth(oauthOpts(url)); err != nil {
t.Fatalf("first SeedOAuth: %v", err)
}
res, err := SeedOAuth(oauthOpts(url))
if err != nil {
t.Fatalf("second SeedOAuth: %v", err)
}
if res.Changed || res.Version != 0 {
t.Errorf("Changed=%v Version=%d, want an unchanged result", res.Changed, res.Version)
}
if len(v.writes) != 1 {
t.Errorf("%d writes, want the second run to write nothing", len(v.writes))
}
for _, k := range res.Keys {
if k.Action != ActionKept {
t.Errorf("%s action = %q, want %q", k.Name, k.Action, ActionKept)
}
}
}
func TestSeedOAuthClientIDUpdated(t *testing.T) {
v := newOAuthVaultStub().seed(map[string]any{
OAuthClientIDKey: "stale-client-id",
OAuthClientSecretKey: existingClientSec,
OAuthCookieSecretKey: existingCookieSec,
})
res, err := SeedOAuth(oauthOpts(v.server(t).URL))
if err != nil {
t.Fatalf("SeedOAuth: %v", err)
}
if got := actions(res)[OAuthClientIDKey]; got != ActionUpdated {
t.Errorf("client_id action = %q, want %q", got, ActionUpdated)
}
if len(v.writes) != 1 || stringField(t, v.writes[0], OAuthClientIDKey) != oauthClientID {
t.Errorf("writes = %v, want the new client_id written", v.writes)
}
}
func TestSeedOAuthLoginFailure(t *testing.T) {
mux := http.NewServeMux()
mux.HandleFunc("/v1/auth/approle/login", func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusBadRequest)
_, _ = io.WriteString(w, `{"errors":["invalid role ID"]}`)
})
vs := httptest.NewServer(mux)
defer vs.Close()
_, err := SeedOAuth(oauthOpts(vs.URL))
if err == nil {
t.Fatal("SeedOAuth() = nil, want an approle login error")
}
if !strings.Contains(err.Error(), "approle login failed") {
t.Errorf("error = %v, want it to name the approle login", err)
}
}
func TestSeedOAuthReadDenied(t *testing.T) {
v := newOAuthVaultStub()
v.readStatus = http.StatusForbidden
_, err := SeedOAuth(oauthOpts(v.server(t).URL))
if err == nil {
t.Fatal("SeedOAuth() = nil, want a KV read error")
}
msg := err.Error()
if !strings.Contains(msg, oauthPath) || !strings.Contains(msg, "policy") {
t.Errorf("error = %v, want it to name the path and point at the policy", err)
}
if len(v.writes) != 0 {
t.Errorf("wrote %v, want no write when the read is denied", v.writes)
}
}
func TestSeedOAuthWriteDenied(t *testing.T) {
v := newOAuthVaultStub()
v.writeStatus = http.StatusForbidden
_, err := SeedOAuth(oauthOpts(v.server(t).URL))
if err == nil {
t.Fatal("SeedOAuth() = nil, want a KV write error")
}
msg := err.Error()
if !strings.Contains(msg, oauthPath) || !strings.Contains(msg, "create/update") {
t.Errorf("error = %v, want it to name the path and the missing capability", err)
}
}
// A missing path is normal (first seed), not a not-found error.
func TestSeedOAuthMissingPathIsNotAnError(t *testing.T) {
v := newOAuthVaultStub()
if _, err := SeedOAuth(oauthOpts(v.server(t).URL)); err != nil {
t.Fatalf("SeedOAuth on a missing path: %v", err)
}
}
func TestSeedOAuthRequiresPathAndClientID(t *testing.T) {
v := newOAuthVaultStub()
url := v.server(t).URL
for name, mutate := range map[string]func(*SeedOAuthOptions){
"no path": func(o *SeedOAuthOptions) { o.Path = "" },
"no client id": func(o *SeedOAuthOptions) { o.ClientID = "" },
} {
t.Run(name, func(t *testing.T) {
o := oauthOpts(url)
mutate(&o)
if _, err := SeedOAuth(o); err == nil {
t.Fatal("SeedOAuth() = nil, want a required-input error")
}
})
}
}
// No failure path may leak stored or generated secret material.
func TestSeedOAuthErrorsNeverLeakSecrets(t *testing.T) {
cases := map[string]func(*oauthVaultStub){
"read denied": func(v *oauthVaultStub) { v.readStatus = http.StatusForbidden },
"write denied": func(v *oauthVaultStub) { v.writeStatus = http.StatusForbidden },
"read error": func(v *oauthVaultStub) { v.readStatus = http.StatusInternalServerError },
"write error": func(v *oauthVaultStub) { v.writeStatus = http.StatusInternalServerError },
}
for name, mutate := range cases {
t.Run(name, func(t *testing.T) {
v := newOAuthVaultStub().seed(map[string]any{
OAuthClientSecretKey: existingClientSec,
OAuthCookieSecretKey: existingCookieSec,
"extra": oauthExtraKeyValue,
})
mutate(v)
_, err := SeedOAuth(oauthOpts(v.server(t).URL))
if err == nil {
t.Fatal("SeedOAuth() = nil, want an error")
}
for _, secret := range []string{existingClientSec, existingCookieSec, oauthExtraKeyValue} {
if strings.Contains(err.Error(), secret) {
t.Errorf("error %q leaks a secret", err)
}
}
})
}
}
// The successful result carries key names and a version, never values.
func TestSeedOAuthResultNeverCarriesSecrets(t *testing.T) {
v := newOAuthVaultStub().seed(map[string]any{OAuthClientSecretKey: existingClientSec})
res, err := SeedOAuth(oauthOpts(v.server(t).URL))
if err != nil {
t.Fatalf("SeedOAuth: %v", err)
}
rendered := strings.Join(append(res.KeyNames(), res.Path, res.KVMount), " ")
written := v.writes[0]
for _, key := range []string{OAuthClientSecretKey, OAuthCookieSecretKey} {
if value := stringField(t, written, key); strings.Contains(rendered, value) {
t.Errorf("result leaks the %s value", key)
}
}
}
+98
View File
@@ -0,0 +1,98 @@
package agent
import (
"errors"
"fmt"
)
const (
// DefaultOutpostTokenPath is the KV-v2 path holding the Authentik API token
// the agent uses to read outpost tokens.
DefaultOutpostTokenPath = "service/authentik/agent-api-token"
// DefaultDestKey is the KV field the outpost token is written to.
DefaultDestKey = "token"
)
// SeedOutpostOptions configures SeedOutpost. Every field is required; the CLI
// supplies the defaults.
type SeedOutpostOptions struct {
VaultAddr string
RoleID string
AuthentikURL string
Outpost string
KVMount string
TokenPath string
DestPath string
DestKey string
}
// SeedOutpostResult is the non-secret summary of a successful seed.
type SeedOutpostResult struct {
Outpost string
TokenIdentifier string
KVMount string
DestPath string
Version int
}
// SeedOutpost copies an Authentik outpost's token into Vault KV-v2. It reads an
// Authentik API token from Vault, resolves the outpost's token identifier,
// fetches the key and writes it to the destination path. The token value never
// leaves this function: results and errors carry only identifiers.
func SeedOutpost(o SeedOutpostOptions) (SeedOutpostResult, error) {
var res SeedOutpostResult
vc, err := NewVaultClient(o.VaultAddr, o.RoleID)
if err != nil {
return res, fmt.Errorf("vault approle login failed against %s (check VAULT_ADDR and AGENT_APPROLE_ROLE_ID): %w", o.VaultAddr, err)
}
secret, err := vc.ReadKV(o.KVMount, o.TokenPath)
if err != nil {
switch {
case errors.Is(err, ErrVaultDenied):
return res, fmt.Errorf("reading %s/%s denied: the agent AppRole policy does not grant read on this path (apply the terraform-vault policy change): %w", o.KVMount, o.TokenPath, err)
case errors.Is(err, ErrVaultNotFound):
return res, fmt.Errorf("secret %s/%s does not exist: seed the Authentik API token there first: %w", o.KVMount, o.TokenPath, err)
}
return res, fmt.Errorf("reading %s/%s: %w", o.KVMount, o.TokenPath, err)
}
apiToken := StringField(secret, "token", "api_token")
if apiToken == "" {
return res, fmt.Errorf("secret %s/%s has neither a 'token' nor an 'api_token' field", o.KVMount, o.TokenPath)
}
ac := NewAuthentikClient(o.AuthentikURL, apiToken)
outpost, err := ac.FindOutpost(o.Outpost)
if err != nil {
if errors.Is(err, ErrOutpostNotFound) {
return res, fmt.Errorf("no outpost named %q at %s: has the terraform-authentik outpost been applied?: %w", o.Outpost, o.AuthentikURL, err)
}
return res, fmt.Errorf("looking up outpost %q: %w", o.Outpost, err)
}
if outpost.TokenIdentifier == "" {
return res, fmt.Errorf("outpost %q has an empty token_identifier", outpost.Name)
}
key, err := ac.TokenKey(outpost.TokenIdentifier)
if err != nil {
return res, fmt.Errorf("fetching the key for token identifier %q (the API token needs view_key on it): %w", outpost.TokenIdentifier, err)
}
version, err := vc.WriteKV(o.KVMount, o.DestPath, map[string]string{o.DestKey: key})
if err != nil {
if errors.Is(err, ErrVaultDenied) {
return res, fmt.Errorf("writing %s/%s denied: the agent AppRole policy does not grant create/update on this path (apply the terraform-vault policy change): %w", o.KVMount, o.DestPath, err)
}
return res, fmt.Errorf("writing %s/%s: %w", o.KVMount, o.DestPath, err)
}
return SeedOutpostResult{
Outpost: outpost.Name,
TokenIdentifier: outpost.TokenIdentifier,
KVMount: o.KVMount,
DestPath: o.DestPath,
Version: version,
}, nil
}
+369
View File
@@ -0,0 +1,369 @@
package agent
import (
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strconv"
"strings"
"testing"
)
const (
testAPIToken = "ak-api-token-secret"
testOutpostKey = "outpost-key-secret"
testTokenPath = "service/authentik/agent-api-token"
testDestPath = "kubernetes/namespace/authentik/default/outpost-token"
testOutpostName = "k8s-outpost"
testTokenIdent = "ak-outpost-k8s-outpost"
testVaultClientT = "s.vaulttoken"
)
// vaultStub is a KV-v2 stand-in whose per-path behaviour tests can override.
type vaultStub struct {
readStatus int
writeStatus int
tokenField string // field name the API token is stored under
writes []map[string]string
writeCount int
}
func newVaultStub() *vaultStub {
return &vaultStub{readStatus: http.StatusOK, writeStatus: http.StatusOK, tokenField: "token"}
}
func (v *vaultStub) server(t *testing.T) *httptest.Server {
t.Helper()
mux := http.NewServeMux()
mux.HandleFunc("/v1/auth/approle/login", func(w http.ResponseWriter, r *http.Request) {
var body map[string]string
_ = json.NewDecoder(r.Body).Decode(&body)
if _, ok := body["secret_id"]; ok {
t.Errorf("secret_id must not be sent")
}
_, _ = io.WriteString(w, `{"auth":{"client_token":"`+testVaultClientT+`"}}`)
})
mux.HandleFunc("/v1/kv/data/"+testTokenPath, func(w http.ResponseWriter, r *http.Request) {
if got := r.Header.Get("X-Vault-Token"); got != testVaultClientT {
t.Errorf("X-Vault-Token = %q, want %q", got, testVaultClientT)
}
if v.readStatus != http.StatusOK {
w.WriteHeader(v.readStatus)
_, _ = io.WriteString(w, `{"errors":["permission denied"]}`)
return
}
_, _ = io.WriteString(w, `{"data":{"data":{"`+v.tokenField+`":"`+testAPIToken+`"},"metadata":{"version":1}}}`)
})
mux.HandleFunc("/v1/kv/data/"+testDestPath, func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
t.Errorf("write method = %s, want POST", r.Method)
}
if v.writeStatus != http.StatusOK {
w.WriteHeader(v.writeStatus)
_, _ = io.WriteString(w, `{"errors":["permission denied"]}`)
return
}
var body struct {
Data map[string]string `json:"data"`
}
_ = json.NewDecoder(r.Body).Decode(&body)
v.writes = append(v.writes, body.Data)
v.writeCount++
_, _ = io.WriteString(w, `{"data":{"version":`+strconv.Itoa(v.writeCount+2)+`}}`)
})
srv := httptest.NewServer(mux)
t.Cleanup(srv.Close)
return srv
}
// authentikStub serves the outpost search and view_key endpoints.
type authentikStub struct {
results string // JSON array body for .results
viewKeyStatus int
viewKeyBody string
sawBearer string
sawSearchQuery string
}
func newAuthentikStub() *authentikStub {
return &authentikStub{
results: `{"pk":"1","name":"` + testOutpostName + `","token_identifier":"` + testTokenIdent + `"}`,
viewKeyStatus: http.StatusOK,
viewKeyBody: `{"key":"` + testOutpostKey + `"}`,
}
}
func (a *authentikStub) server(t *testing.T) *httptest.Server {
t.Helper()
mux := http.NewServeMux()
mux.HandleFunc("/api/v3/outposts/instances/", func(w http.ResponseWriter, r *http.Request) {
a.sawBearer = r.Header.Get("Authorization")
a.sawSearchQuery = r.URL.Query().Get("search")
_, _ = io.WriteString(w, `{"results":[`+a.results+`]}`)
})
mux.HandleFunc("/api/v3/core/tokens/"+testTokenIdent+"/view_key/", func(w http.ResponseWriter, r *http.Request) {
if a.viewKeyStatus != http.StatusOK {
w.WriteHeader(a.viewKeyStatus)
_, _ = io.WriteString(w, `{"detail":"boom"}`)
return
}
_, _ = io.WriteString(w, a.viewKeyBody)
})
srv := httptest.NewServer(mux)
t.Cleanup(srv.Close)
return srv
}
func opts(vaultURL, authentikURL string) SeedOutpostOptions {
return SeedOutpostOptions{
VaultAddr: vaultURL,
RoleID: "role-xyz",
AuthentikURL: authentikURL,
Outpost: testOutpostName,
KVMount: DefaultKVMount,
TokenPath: testTokenPath,
DestPath: testDestPath,
DestKey: DefaultDestKey,
}
}
func TestSeedOutpostHappyPath(t *testing.T) {
v := newVaultStub()
a := newAuthentikStub()
vs, as := v.server(t), a.server(t)
res, err := SeedOutpost(opts(vs.URL, as.URL))
if err != nil {
t.Fatalf("SeedOutpost: %v", err)
}
if res.Outpost != testOutpostName {
t.Errorf("Outpost = %q, want %q", res.Outpost, testOutpostName)
}
if res.TokenIdentifier != testTokenIdent {
t.Errorf("TokenIdentifier = %q, want %q", res.TokenIdentifier, testTokenIdent)
}
if res.DestPath != testDestPath || res.KVMount != DefaultKVMount {
t.Errorf("dest = %s/%s, want kv/%s", res.KVMount, res.DestPath, testDestPath)
}
if res.Version != 3 {
t.Errorf("Version = %d, want 3", res.Version)
}
if a.sawBearer != "Bearer "+testAPIToken {
t.Errorf("Authorization = %q, want the API token as a bearer", a.sawBearer)
}
if a.sawSearchQuery != testOutpostName {
t.Errorf("search = %q, want %q", a.sawSearchQuery, testOutpostName)
}
if len(v.writes) != 1 || v.writes[0][DefaultDestKey] != testOutpostKey {
t.Fatalf("written data = %v, want {%s: outpost key}", v.writes, DefaultDestKey)
}
}
// Re-running writes a new KV version rather than failing.
func TestSeedOutpostIdempotentNewVersion(t *testing.T) {
v := newVaultStub()
a := newAuthentikStub()
vs, as := v.server(t), a.server(t)
first, err := SeedOutpost(opts(vs.URL, as.URL))
if err != nil {
t.Fatalf("first SeedOutpost: %v", err)
}
second, err := SeedOutpost(opts(vs.URL, as.URL))
if err != nil {
t.Fatalf("second SeedOutpost: %v", err)
}
if second.Version != first.Version+1 {
t.Errorf("versions = %d then %d, want consecutive", first.Version, second.Version)
}
}
// The API token may be stored under api_token instead of token.
func TestSeedOutpostAPITokenFallbackField(t *testing.T) {
v := newVaultStub()
v.tokenField = "api_token"
a := newAuthentikStub()
vs, as := v.server(t), a.server(t)
if _, err := SeedOutpost(opts(vs.URL, as.URL)); err != nil {
t.Fatalf("SeedOutpost with api_token field: %v", err)
}
if a.sawBearer != "Bearer "+testAPIToken {
t.Errorf("Authorization = %q, want the api_token value", a.sawBearer)
}
}
func TestSeedOutpostCustomDestKey(t *testing.T) {
v := newVaultStub()
a := newAuthentikStub()
vs, as := v.server(t), a.server(t)
o := opts(vs.URL, as.URL)
o.DestKey = "outpost-token"
if _, err := SeedOutpost(o); err != nil {
t.Fatalf("SeedOutpost: %v", err)
}
if len(v.writes) != 1 || v.writes[0]["outpost-token"] != testOutpostKey {
t.Errorf("written data = %v, want the key under outpost-token", v.writes)
}
}
func TestSeedOutpostLoginFailure(t *testing.T) {
mux := http.NewServeMux()
mux.HandleFunc("/v1/auth/approle/login", func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusBadRequest)
_, _ = io.WriteString(w, `{"errors":["invalid role ID"]}`)
})
vs := httptest.NewServer(mux)
defer vs.Close()
a := newAuthentikStub()
_, err := SeedOutpost(opts(vs.URL, a.server(t).URL))
if err == nil {
t.Fatal("SeedOutpost() = nil, want an approle login error")
}
if !strings.Contains(err.Error(), "approle login failed") {
t.Errorf("error = %v, want it to name the approle login", err)
}
}
func TestSeedOutpostKVReadDenied(t *testing.T) {
v := newVaultStub()
v.readStatus = http.StatusForbidden
a := newAuthentikStub()
vs, as := v.server(t), a.server(t)
_, err := SeedOutpost(opts(vs.URL, as.URL))
if err == nil {
t.Fatal("SeedOutpost() = nil, want a KV read error")
}
msg := err.Error()
if !strings.Contains(msg, testTokenPath) || !strings.Contains(msg, "policy") {
t.Errorf("error = %v, want it to name the token path and point at the policy", err)
}
if strings.Contains(msg, "does not exist") {
t.Errorf("error = %v, denied must not be reported as missing", err)
}
}
func TestSeedOutpostKVReadNotFound(t *testing.T) {
v := newVaultStub()
v.readStatus = http.StatusNotFound
a := newAuthentikStub()
vs, as := v.server(t), a.server(t)
_, err := SeedOutpost(opts(vs.URL, as.URL))
if err == nil {
t.Fatal("SeedOutpost() = nil, want a missing-secret error")
}
if !strings.Contains(err.Error(), "does not exist") {
t.Errorf("error = %v, want it to say the secret does not exist", err)
}
}
func TestSeedOutpostMissingTokenField(t *testing.T) {
v := newVaultStub()
v.tokenField = "password"
a := newAuthentikStub()
vs, as := v.server(t), a.server(t)
_, err := SeedOutpost(opts(vs.URL, as.URL))
if err == nil {
t.Fatal("SeedOutpost() = nil, want an error for a secret with no token field")
}
if !strings.Contains(err.Error(), "api_token") {
t.Errorf("error = %v, want it to name the accepted fields", err)
}
}
// A substring hit that is not the exact name must not be accepted.
func TestSeedOutpostNotFound(t *testing.T) {
v := newVaultStub()
a := newAuthentikStub()
a.results = `{"pk":"1","name":"` + testOutpostName + `-staging","token_identifier":"other"}`
vs, as := v.server(t), a.server(t)
_, err := SeedOutpost(opts(vs.URL, as.URL))
if err == nil {
t.Fatal("SeedOutpost() = nil, want an outpost-not-found error")
}
msg := err.Error()
if !strings.Contains(msg, "no outpost named") || !strings.Contains(msg, "terraform") {
t.Errorf("error = %v, want it to report the missing outpost and mention terraform", err)
}
if len(v.writes) != 0 {
t.Errorf("wrote %v, want no KV write when the outpost is missing", v.writes)
}
}
func TestSeedOutpostViewKeyFailure(t *testing.T) {
v := newVaultStub()
a := newAuthentikStub()
a.viewKeyStatus = http.StatusForbidden
vs, as := v.server(t), a.server(t)
_, err := SeedOutpost(opts(vs.URL, as.URL))
if err == nil {
t.Fatal("SeedOutpost() = nil, want a view_key error")
}
if !strings.Contains(err.Error(), testTokenIdent) {
t.Errorf("error = %v, want it to name the token identifier", err)
}
if len(v.writes) != 0 {
t.Errorf("wrote %v, want no KV write when view_key fails", v.writes)
}
}
func TestSeedOutpostViewKeyEmpty(t *testing.T) {
v := newVaultStub()
a := newAuthentikStub()
a.viewKeyBody = `{}`
vs, as := v.server(t), a.server(t)
if _, err := SeedOutpost(opts(vs.URL, as.URL)); err == nil {
t.Fatal("SeedOutpost() = nil, want an error when view_key returns no key")
}
}
func TestSeedOutpostKVWriteDenied(t *testing.T) {
v := newVaultStub()
v.writeStatus = http.StatusForbidden
a := newAuthentikStub()
vs, as := v.server(t), a.server(t)
_, err := SeedOutpost(opts(vs.URL, as.URL))
if err == nil {
t.Fatal("SeedOutpost() = nil, want a KV write error")
}
msg := err.Error()
if !strings.Contains(msg, testDestPath) || !strings.Contains(msg, "create/update") {
t.Errorf("error = %v, want it to name the dest path and the missing capability", err)
}
}
// No failure path may leak the API token or the outpost key into the error.
func TestSeedOutpostErrorsNeverLeakSecrets(t *testing.T) {
cases := map[string]func(*vaultStub, *authentikStub){
"read denied": func(v *vaultStub, a *authentikStub) { v.readStatus = http.StatusForbidden },
"write denied": func(v *vaultStub, a *authentikStub) { v.writeStatus = http.StatusForbidden },
"view_key fail": func(v *vaultStub, a *authentikStub) { a.viewKeyStatus = http.StatusInternalServerError },
"outpost gone": func(v *vaultStub, a *authentikStub) { a.results = "" },
}
for name, mutate := range cases {
t.Run(name, func(t *testing.T) {
v, a := newVaultStub(), newAuthentikStub()
mutate(v, a)
vs, as := v.server(t), a.server(t)
_, err := SeedOutpost(opts(vs.URL, as.URL))
if err == nil {
t.Fatal("SeedOutpost() = nil, want an error")
}
for _, secret := range []string{testAPIToken, testOutpostKey} {
if strings.Contains(err.Error(), secret) {
t.Errorf("error %q leaks a secret", err)
}
}
})
}
}
+43 -17
View File
@@ -1,12 +1,13 @@
// Package agent holds the plumbing shared by the agent-tools CLIs (agentpr and // Package agent holds the plumbing shared by the agent-tools CLIs (agentpr,
// watchpr): obtaining a Gitea token via Vault AppRole, talking to the Gitea // watchpr, agentws and agentvault): the Vault AppRole login and its KV-v2
// API, parsing PR references, and deciding when a watched PR changed // client, talking to the Gitea and Authentik APIs, parsing PR references, and
// meaningfully. Both tools acquire their Gitea token the same way, so that // deciding when a watched PR changed meaningfully. Every tool authenticates to
// logic lives here once. // Vault the same way, so that logic lives here once.
package agent package agent
import ( import (
"os" "os"
"strings"
"sync" "sync"
) )
@@ -16,13 +17,18 @@ const (
// DefaultRoleID is the agent AppRole role_id used when AGENT_APPROLE_ROLE_ID // DefaultRoleID is the agent AppRole role_id used when AGENT_APPROLE_ROLE_ID
// is unset. Login uses role_id only (no secret_id). // is unset. Login uses role_id only (no secret_id).
DefaultRoleID = "ababbcd3-9c77-5c6a-be2d-287fce9214a6" DefaultRoleID = "ababbcd3-9c77-5c6a-be2d-287fce9214a6"
// GiteaCredsPath is the Vault path that mints a scoped Gitea token. // GiteaCredsPrefix is the Vault gitea secrets-engine creds prefix; the agent
GiteaCredsPath = "gitea/creds/unkin-agent" // login is appended to it to form the path that mints a scoped Gitea token.
GiteaCredsPrefix = "gitea/creds/"
// DefaultGiteaURL is the Gitea base URL used when GITEA_URL is unset. // DefaultGiteaURL is the Gitea base URL used when GITEA_URL is unset.
DefaultGiteaURL = "https://git.unkin.net" DefaultGiteaURL = "https://git.unkin.net"
// DefaultAgentLogin is the Gitea login of the agent whose own comments are // DefaultAgentLogin is the Gitea login the tools act as: it selects the Vault
// ignored by watchpr. Overridable via AGENT_LOGIN. // creds path, sets the agentws git identity and is the login whose own
// comments watchpr ignores. Overridable via AGENT_LOGIN.
DefaultAgentLogin = "unkin-agent" DefaultAgentLogin = "unkin-agent"
// DefaultAuthentikURL is the Authentik base URL used when AUTHENTIK_URL is
// unset. identity.unkin.net has no DNS record; the k8s name is the real one.
DefaultAuthentikURL = "https://identity.k8s.syd1.au.unkin.net"
) )
// VaultAddr returns the configured Vault address (env VAULT_ADDR or the default). // VaultAddr returns the configured Vault address (env VAULT_ADDR or the default).
@@ -50,8 +56,8 @@ func GiteaURL() string {
return DefaultGiteaURL return DefaultGiteaURL
} }
// AgentLogin returns the login whose comments watchpr ignores (env AGENT_LOGIN // AgentLogin returns the Gitea login the tools act as (env AGENT_LOGIN or the
// or the default). // default).
func AgentLogin() string { func AgentLogin() string {
if v := os.Getenv("AGENT_LOGIN"); v != "" { if v := os.Getenv("AGENT_LOGIN"); v != "" {
return v return v
@@ -59,6 +65,26 @@ func AgentLogin() string {
return DefaultAgentLogin return DefaultAgentLogin
} }
// GiteaCredsPath returns the Vault path that mints a scoped Gitea token:
// GITEA_CREDS_PATH when set, otherwise gitea/creds/<AgentLogin>. So a service
// running as its own identity only has to set AGENT_LOGIN.
func GiteaCredsPath() string {
// Trimmed because callers join this onto ".../v1/".
if v := strings.Trim(strings.TrimSpace(os.Getenv("GITEA_CREDS_PATH")), "/"); v != "" {
return v
}
return GiteaCredsPrefix + AgentLogin()
}
// AuthentikURL returns the configured Authentik base URL (env AUTHENTIK_URL or
// the default).
func AuthentikURL() string {
if v := os.Getenv("AUTHENTIK_URL"); v != "" {
return v
}
return DefaultAuthentikURL
}
var ( var (
tokenOnce sync.Once tokenOnce sync.Once
tokenValue string tokenValue string
@@ -69,18 +95,18 @@ var (
// and caching it in-process for the lifetime of the command. // and caching it in-process for the lifetime of the command.
func GiteaToken() (string, error) { func GiteaToken() (string, error) {
tokenOnce.Do(func() { tokenOnce.Do(func() {
tokenValue, tokenErr = fetchGiteaToken(VaultAddr(), RoleID()) tokenValue, tokenErr = fetchGiteaToken(VaultAddr(), RoleID(), GiteaCredsPath())
}) })
return tokenValue, tokenErr return tokenValue, tokenErr
} }
// fetchGiteaToken performs the AppRole login and reads the Gitea creds. It is // fetchGiteaToken performs the AppRole login and reads the Gitea creds at
// separated from GiteaToken so tests can exercise it directly against an // credsPath. It is separated from GiteaToken so tests can exercise it directly
// httptest server without touching the process-wide cache. // against an httptest server without touching the process-wide cache.
func fetchGiteaToken(vaultAddr, roleID string) (string, error) { func fetchGiteaToken(vaultAddr, roleID, credsPath string) (string, error) {
clientToken, err := approleLogin(vaultAddr, roleID) clientToken, err := approleLogin(vaultAddr, roleID)
if err != nil { if err != nil {
return "", err return "", err
} }
return readGiteaCreds(vaultAddr, clientToken) return readGiteaCreds(vaultAddr, clientToken, credsPath)
} }
+155
View File
@@ -0,0 +1,155 @@
package agent
import (
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
)
func TestGiteaCredsPath(t *testing.T) {
tests := []struct {
name string
agentLogin string
credsPath string
want string
}{
{"no env keeps the historical path", "", "", "gitea/creds/unkin-agent"},
{"derived from AGENT_LOGIN", "repospawner", "", "gitea/creds/repospawner"},
{"GITEA_CREDS_PATH beats AGENT_LOGIN", "repospawner", "gitea/creds/someone-else", "gitea/creds/someone-else"},
{"GITEA_CREDS_PATH beats the default", "", "other-gitea/creds/bot", "other-gitea/creds/bot"},
{"override is trimmed for joining onto /v1/", "", " /gitea/creds/bot/ ", "gitea/creds/bot"},
{"blank override falls back to the login", "repospawner", " ", "gitea/creds/repospawner"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Setenv("AGENT_LOGIN", tt.agentLogin)
t.Setenv("GITEA_CREDS_PATH", tt.credsPath)
if got := GiteaCredsPath(); got != tt.want {
t.Errorf("GiteaCredsPath() = %q, want %q", got, tt.want)
}
})
}
}
// recordingVault serves the AppRole login plus any creds path under /v1/,
// recording which one was read so tests can assert the selected path.
type recordingVault struct {
mu sync.Mutex
credsPath string
}
func (v *recordingVault) path() string {
v.mu.Lock()
defer v.mu.Unlock()
return v.credsPath
}
// fakeEstate serves both Vault (/v1/...) and Gitea (/api/v1/...) from one
// server, so a test can drive the whole token-then-API flow.
func fakeEstate(t *testing.T, giteaToken, login string) (*httptest.Server, *recordingVault) {
t.Helper()
rec := &recordingVault{}
mux := http.NewServeMux()
mux.HandleFunc("/v1/auth/approle/login", func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, `{"auth":{"client_token":"s.vaulttoken"}}`)
})
mux.HandleFunc("/v1/", func(w http.ResponseWriter, r *http.Request) {
rec.mu.Lock()
rec.credsPath = strings.TrimPrefix(r.URL.Path, "/v1/")
rec.mu.Unlock()
_, _ = io.WriteString(w, `{"data":{"token":"`+giteaToken+`"}}`)
})
mux.HandleFunc("/api/v1/user", func(w http.ResponseWriter, r *http.Request) {
if got := r.Header.Get("Authorization"); got != "token "+giteaToken {
t.Errorf("whoami auth header = %q, want token %s", got, giteaToken)
}
_, _ = io.WriteString(w, `{"login":"`+login+`","id":7}`)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/pulls", func(w http.ResponseWriter, r *http.Request) {
if got := r.Header.Get("Authorization"); got != "token "+giteaToken {
t.Errorf("create PR auth header = %q, want token %s", got, giteaToken)
}
var body CreatePROptions
_ = json.NewDecoder(r.Body).Decode(&body)
_, _ = io.WriteString(w, `{"number":12,"state":"open","html_url":"`+r.Host+`/pulls/12"}`)
})
return httptest.NewServer(mux), rec
}
// A service that sets AGENT_LOGIN must read its own creds path and act as its
// own Gitea identity for both whoami and PR creation.
func TestWhoamiAndPRUseSelectedCredsPath(t *testing.T) {
srv, rec := fakeEstate(t, "gitea-repospawner", "repospawner")
defer srv.Close()
t.Setenv("AGENT_LOGIN", "repospawner")
t.Setenv("GITEA_CREDS_PATH", "")
t.Setenv("VAULT_ADDR", srv.URL)
t.Setenv("GITEA_URL", srv.URL)
tok, err := fetchGiteaToken(VaultAddr(), RoleID(), GiteaCredsPath())
if err != nil {
t.Fatalf("fetchGiteaToken: %v", err)
}
if got := rec.path(); got != "gitea/creds/repospawner" {
t.Errorf("vault read path = %q, want gitea/creds/repospawner", got)
}
if tok != "gitea-repospawner" {
t.Fatalf("token = %q, want gitea-repospawner", tok)
}
c := NewGiteaClient(tok)
u, err := c.Whoami()
if err != nil {
t.Fatalf("Whoami: %v", err)
}
if u.Login != "repospawner" {
t.Errorf("whoami login = %q, want repospawner", u.Login)
}
pr, err := c.CreatePR("unkin/repo", CreatePROptions{Base: "main", Head: "feature", Title: "T"})
if err != nil {
t.Fatalf("CreatePR: %v", err)
}
if pr.Number != 12 {
t.Errorf("PR number = %d, want 12", pr.Number)
}
}
// GITEA_CREDS_PATH must win even when AGENT_LOGIN names a different identity.
func TestCredsPathOverrideBeatsAgentLogin(t *testing.T) {
srv, rec := fakeEstate(t, "gitea-override", "someone-else")
defer srv.Close()
t.Setenv("AGENT_LOGIN", "repospawner")
t.Setenv("GITEA_CREDS_PATH", "gitea/creds/someone-else")
t.Setenv("VAULT_ADDR", srv.URL)
if _, err := fetchGiteaToken(VaultAddr(), RoleID(), GiteaCredsPath()); err != nil {
t.Fatalf("fetchGiteaToken: %v", err)
}
if got := rec.path(); got != "gitea/creds/someone-else" {
t.Errorf("vault read path = %q, want gitea/creds/someone-else", got)
}
}
// With no env set the tools must still read the exact path they always did.
func TestCredsPathDefaultIsBackwardCompatible(t *testing.T) {
srv, rec := fakeEstate(t, "gitea-abc", "unkin-agent")
defer srv.Close()
t.Setenv("AGENT_LOGIN", "")
t.Setenv("GITEA_CREDS_PATH", "")
t.Setenv("VAULT_ADDR", srv.URL)
if _, err := fetchGiteaToken(VaultAddr(), RoleID(), GiteaCredsPath()); err != nil {
t.Fatalf("fetchGiteaToken: %v", err)
}
if got := rec.path(); got != "gitea/creds/unkin-agent" {
t.Errorf("vault read path = %q, want gitea/creds/unkin-agent", got)
}
}
+8 -7
View File
@@ -49,9 +49,10 @@ func approleLogin(vaultAddr, roleID string) (string, error) {
return out.Auth.ClientToken, nil return out.Auth.ClientToken, nil
} }
// readGiteaCreds reads the Gitea creds secret and returns the token field. // readGiteaCreds reads the Gitea creds secret at credsPath and returns the
func readGiteaCreds(vaultAddr, clientToken string) (string, error) { // token field.
url := strings.TrimRight(vaultAddr, "/") + "/v1/" + GiteaCredsPath func readGiteaCreds(vaultAddr, clientToken, credsPath string) (string, error) {
url := strings.TrimRight(vaultAddr, "/") + "/v1/" + credsPath
req, err := http.NewRequest(http.MethodGet, url, nil) req, err := http.NewRequest(http.MethodGet, url, nil)
if err != nil { if err != nil {
return "", err return "", err
@@ -60,12 +61,12 @@ func readGiteaCreds(vaultAddr, clientToken string) (string, error) {
resp, err := httpClient.Do(req) resp, err := httpClient.Do(req)
if err != nil { if err != nil {
return "", fmt.Errorf("vault read %s: %w", GiteaCredsPath, err) return "", fmt.Errorf("vault read %s: %w", credsPath, err)
} }
defer func() { _ = resp.Body.Close() }() defer func() { _ = resp.Body.Close() }()
data, _ := io.ReadAll(resp.Body) data, _ := io.ReadAll(resp.Body)
if resp.StatusCode != http.StatusOK { if resp.StatusCode != http.StatusOK {
return "", fmt.Errorf("vault read %s: HTTP %d: %s", GiteaCredsPath, resp.StatusCode, strings.TrimSpace(string(data))) return "", fmt.Errorf("vault read %s: HTTP %d: %s", credsPath, resp.StatusCode, strings.TrimSpace(string(data)))
} }
var out struct { var out struct {
@@ -74,10 +75,10 @@ func readGiteaCreds(vaultAddr, clientToken string) (string, error) {
} `json:"data"` } `json:"data"`
} }
if err := json.Unmarshal(data, &out); err != nil { if err := json.Unmarshal(data, &out); err != nil {
return "", fmt.Errorf("vault read %s: decoding response: %w", GiteaCredsPath, err) return "", fmt.Errorf("vault read %s: decoding response: %w", credsPath, err)
} }
if out.Data.Token == "" { if out.Data.Token == "" {
return "", fmt.Errorf("vault read %s: no token field in secret", GiteaCredsPath) return "", fmt.Errorf("vault read %s: no token field in secret", credsPath)
} }
return out.Data.Token, nil return out.Data.Token, nil
} }
+166
View File
@@ -0,0 +1,166 @@
package agent
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"strings"
)
const (
// DefaultKVMount is the KV-v2 mount holding agent-facing secrets.
DefaultKVMount = "kv"
)
var (
// ErrVaultDenied marks a 403 from Vault (the AppRole policy lacks the capability).
ErrVaultDenied = errors.New("permission denied")
// ErrVaultNotFound marks a 404 from Vault (mount or secret does not exist).
ErrVaultNotFound = errors.New("not found")
)
// VaultClient issues authenticated requests against Vault/OpenBao using a token
// obtained from the agent AppRole.
type VaultClient struct {
Addr string
Token string
HTTP *http.Client
}
// NewVaultClient performs the AppRole login (role_id only, no secret_id) and
// returns a client bound to the resulting client_token.
func NewVaultClient(addr, roleID string) (*VaultClient, error) {
token, err := approleLogin(addr, roleID)
if err != nil {
return nil, err
}
return &VaultClient{Addr: addr, Token: token, HTTP: httpClient}, nil
}
func (c *VaultClient) do(method, path string, body any, out any) error {
var reader io.Reader
if body != nil {
b, err := json.Marshal(body)
if err != nil {
return err
}
reader = bytes.NewReader(b)
}
url := strings.TrimRight(c.Addr, "/") + path
req, err := http.NewRequest(method, url, reader)
if err != nil {
return err
}
req.Header.Set("X-Vault-Token", c.Token)
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
resp, err := c.HTTP.Do(req)
if err != nil {
return fmt.Errorf("vault %s %s: %w", method, path, err)
}
defer func() { _ = resp.Body.Close() }()
data, _ := io.ReadAll(resp.Body)
switch {
case resp.StatusCode == http.StatusForbidden:
return fmt.Errorf("vault %s %s: %w", method, path, ErrVaultDenied)
case resp.StatusCode == http.StatusNotFound:
return fmt.Errorf("vault %s %s: %w", method, path, ErrVaultNotFound)
case resp.StatusCode < 200 || resp.StatusCode >= 300:
return fmt.Errorf("vault %s %s: HTTP %d: %s", method, path, resp.StatusCode, strings.TrimSpace(string(data)))
}
if out != nil && len(data) > 0 {
// Response bodies here carry secret material, so decode failures never
// echo the body.
if err := json.Unmarshal(data, out); err != nil {
return fmt.Errorf("vault %s %s: decoding response: %w", method, path, err)
}
}
return nil
}
// kvDataPath builds the KV-v2 data path for a mount and secret path.
func kvDataPath(mount, path string) string {
return "/v1/" + strings.Trim(mount, "/") + "/data/" + strings.Trim(path, "/")
}
// ReadKV returns the data map of a KV-v2 secret.
func (c *VaultClient) ReadKV(mount, path string) (map[string]any, error) {
var out struct {
Data struct {
Data map[string]any `json:"data"`
} `json:"data"`
}
if err := c.do(http.MethodGet, kvDataPath(mount, path), nil, &out); err != nil {
return nil, err
}
if out.Data.Data == nil {
return nil, fmt.Errorf("vault read %s/%s: secret has no data", mount, path)
}
return out.Data.Data, nil
}
// ReadKVOptional is ReadKV but treats a missing secret as empty: a 404 or a
// deleted version (data: null) yields an empty map, not an error. Callers that
// read-modify-write a path that may not exist yet use this.
func (c *VaultClient) ReadKVOptional(mount, path string) (map[string]any, error) {
var out struct {
Data struct {
Data map[string]any `json:"data"`
} `json:"data"`
}
if err := c.do(http.MethodGet, kvDataPath(mount, path), nil, &out); err != nil {
if errors.Is(err, ErrVaultNotFound) {
return map[string]any{}, nil
}
return nil, err
}
if out.Data.Data == nil {
return map[string]any{}, nil
}
return out.Data.Data, nil
}
// WriteKV writes a KV-v2 secret and returns the version it created.
func (c *VaultClient) WriteKV(mount, path string, data map[string]string) (int, error) {
fields := make(map[string]any, len(data))
for k, v := range data {
fields[k] = v
}
return c.WriteKVAny(mount, path, fields)
}
// WriteKVAny writes a KV-v2 secret whose fields are not all strings (so a
// read-modify-write can put back values it did not author) and returns the
// version it created.
func (c *VaultClient) WriteKVAny(mount, path string, data map[string]any) (int, error) {
var out struct {
Data struct {
Version int `json:"version"`
} `json:"data"`
}
body := map[string]any{"data": data}
if err := c.do(http.MethodPost, kvDataPath(mount, path), body, &out); err != nil {
return 0, err
}
if out.Data.Version == 0 {
return 0, fmt.Errorf("vault write %s/%s: no version in response", mount, path)
}
return out.Data.Version, nil
}
// StringField returns the first non-empty string value among the given keys.
func StringField(data map[string]any, keys ...string) string {
for _, k := range keys {
if s, ok := data[k].(string); ok && s != "" {
return s
}
}
return ""
}
+4 -1
View File
@@ -22,8 +22,11 @@ func FetchState(c *GiteaClient, ref PRRef, agentLogin string) (PRState, error) {
if err != nil { if err != nil {
return PRState{}, err return PRState{}, err
} }
// A 404 here means the head commit is gone (branch deleted after a squash/
// rebase merge); the PR object is still authoritative, so treat CI as absent
// rather than discarding the merge signal and hanging the watch loop.
ci, err := c.CommitStatus(ref.RepoPath(), pr.Head.Sha) ci, err := c.CommitStatus(ref.RepoPath(), pr.Head.Sha)
if err != nil { if err != nil && !isNotFound(err) {
return PRState{}, err return PRState{}, err
} }
comments, err := c.ListComments(ref.RepoPath(), ref.Number) comments, err := c.ListComments(ref.RepoPath(), ref.Number)
+76
View File
@@ -2,6 +2,10 @@ package agent
import ( import (
"errors" "errors"
"fmt"
"net/http"
"net/http/httptest"
"sync/atomic"
"testing" "testing"
"time" "time"
) )
@@ -273,6 +277,78 @@ func TestWatchBaselineErrorAborts(t *testing.T) {
} }
} }
// The production hang, end to end: a watched PR stays open across several polls,
// then is squash-merged and its branch deleted, so the commit-status endpoint
// 404s. Driven through a real *GiteaClient, the watch loop must still detect the
// merge on the poll it happens. Before the fix, FetchState returned an error on
// that poll (the 404 masked the merge), so the loop reported only poll errors
// and never exited -- exactly the 37-minute hang seen in production.
func TestWatchDetectsMergeWhenCommitGone(t *testing.T) {
const sha = "cafebabecafebabe"
var polls atomic.Int32 // number of PR fetches so far
mux := http.NewServeMux()
mux.HandleFunc("/api/v1/repos/unkin/repo/pulls/7", func(w http.ResponseWriter, r *http.Request) {
n := polls.Add(1)
if n >= 4 { // baseline + two unchanged polls, then merged
_, _ = fmt.Fprintf(w, `{"number":7,"state":"closed","merged":true,"mergeable":true,"head":{"sha":%q}}`, sha)
return
}
_, _ = fmt.Fprintf(w, `{"number":7,"state":"open","merged":false,"mergeable":true,"head":{"sha":%q}}`, sha)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/commits/"+sha+"/status", func(w http.ResponseWriter, r *http.Request) {
if polls.Load() >= 4 { // branch deleted post-merge: commit is gone
w.WriteHeader(http.StatusNotFound)
_, _ = fmt.Fprint(w, `{"message":"not found"}`)
return
}
_, _ = fmt.Fprint(w, `{"state":"success"}`)
})
mux.HandleFunc("/api/v1/repos/unkin/repo/issues/7/comments", func(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprint(w, `[]`)
})
srv := httptest.NewServer(mux)
defer srv.Close()
c := &GiteaClient{BaseURL: srv.URL, Token: "t", HTTP: srv.Client()}
ref := PRRef{Owner: "unkin", Repo: "repo", Number: 7}
// A real ticker so the loop advances on its own; a hang (the bug) is caught
// by the timeout below instead of blocking the suite.
tk := time.NewTicker(5 * time.Millisecond)
defer tk.Stop()
var pollErr atomic.Pointer[error]
type outcome struct {
res WatchResult
err error
}
done := make(chan outcome, 1)
go func() {
res, err := Watch(c, []PRRef{ref}, "unkin-agent", tk.C, nil,
func(_ PRRef, e error) { pollErr.Store(&e) })
done <- outcome{res, err}
}()
select {
case o := <-done:
if o.err != nil {
t.Fatalf("Watch: %v", o.err)
}
if o.res.Reason != "PR merged" {
t.Errorf("reason = %q, want %q", o.res.Reason, "PR merged")
}
if p := pollErr.Load(); p != nil {
t.Errorf("no poll error expected once a 404 status is tolerated, got: %v", *p)
}
case <-time.After(3 * time.Second):
var got error
if p := pollErr.Load(); p != nil {
got = *p
}
t.Fatalf("Watch hung: a merge with a gone head commit was never detected (last poll error: %v)", got)
}
}
func TestCountNonAgentComments(t *testing.T) { func TestCountNonAgentComments(t *testing.T) {
comments := []Comment{ comments := []Comment{
{User: User{Login: "unkin-agent"}}, {User: User{Login: "unkin-agent"}},
+18
View File
@@ -42,6 +42,12 @@ contents:
mode: 0755 mode: 0755
owner: root owner: root
group: root group: root
- src: dist/agentvault
dst: /usr/bin/agentvault
file_info:
mode: 0755
owner: root
group: root
# Shell completions (generated by scripts/build-rpm.sh before packaging). # Shell completions (generated by scripts/build-rpm.sh before packaging).
- src: dist/completions/agentpr.bash - src: dist/completions/agentpr.bash
@@ -80,3 +86,15 @@ contents:
dst: /usr/share/fish/vendor_completions.d/agentws.fish dst: /usr/share/fish/vendor_completions.d/agentws.fish
file_info: file_info:
mode: 0644 mode: 0644
- src: dist/completions/agentvault.bash
dst: /usr/share/bash-completion/completions/agentvault
file_info:
mode: 0644
- src: dist/completions/_agentvault
dst: /usr/share/zsh/site-functions/_agentvault
file_info:
mode: 0644
- src: dist/completions/agentvault.fish
dst: /usr/share/fish/vendor_completions.d/agentvault.fish
file_info:
mode: 0644
+3 -3
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# #
# Package the (already built) agentpr, watchpr and agentws binaries into an RPM # Package the (already built) agentpr, watchpr, agentws and agentvault binaries into an RPM
# with nfpm, bundling generated bash/zsh/fish shell completions. # with nfpm, bundling generated bash/zsh/fish shell completions.
# Usage: scripts/build-rpm.sh [version] (version defaults to $CI_COMMIT_TAG) # Usage: scripts/build-rpm.sh [version] (version defaults to $CI_COMMIT_TAG)
# #
@@ -12,7 +12,7 @@ cd "${ROOT_DIR}"
VERSION="${1:-${CI_COMMIT_TAG:-0.0.0-dev}}" VERSION="${1:-${CI_COMMIT_TAG:-0.0.0-dev}}"
VERSION="${VERSION#v}" # strip a leading v VERSION="${VERSION#v}" # strip a leading v
PACKAGE="agent-tools" PACKAGE="agent-tools"
BINARIES=(agentpr watchpr agentws) BINARIES=(agentpr watchpr agentws agentvault)
DIST="dist" DIST="dist"
for b in "${BINARIES[@]}"; do for b in "${BINARIES[@]}"; do
@@ -37,7 +37,7 @@ export PACKAGE_VERSION="${VERSION}"
export PACKAGE_RELEASE="1" export PACKAGE_RELEASE="1"
export PACKAGE_ARCH="amd64" export PACKAGE_ARCH="amd64"
export PACKAGE_PLATFORM="linux" export PACKAGE_PLATFORM="linux"
export PACKAGE_DESCRIPTION="CLI tools for Gitea automation as the unkin-agent user: agentpr (create PRs/comments), watchpr (poll PRs and alert on meaningful change) and agentws (manage per-branch git worktrees)" export PACKAGE_DESCRIPTION="CLI tools for automation as the unkin-agent user: agentpr (create PRs/comments), watchpr (poll PRs and alert on meaningful change), agentws (manage per-branch git worktrees) and agentvault (deterministic Vault flows)"
export PACKAGE_MAINTAINER="Ben Vincent <ben@unkin.net>" export PACKAGE_MAINTAINER="Ben Vincent <ben@unkin.net>"
export PACKAGE_HOMEPAGE="https://git.unkin.net/unkin/agent-tools" export PACKAGE_HOMEPAGE="https://git.unkin.net/unkin/agent-tools"
export PACKAGE_LICENSE="MIT" export PACKAGE_LICENSE="MIT"