unkin-agent
8786636f7c
Four issues from the review of the initial repospawner service, none of which change the shape of a request or the file terraform-git receives. - Encode status checks as one --check flag per context on the server-to-job hop, so a separator inside a context can no longer turn one context into several; ban commas (and cap lengths) in Validate as well, since a real context never holds one. - Fail a merged request that has waited five minutes for a Woodpecker token that vanished after acceptance, surfacing "woodpecker token unavailable" through the API, instead of warning in the log forever from enabling-ci. Advance now leaves a terminal request alone so the failure sticks. - Hold a per-name lock from the duplicate checks through the store write, so two concurrent submissions of one name cannot both be accepted. - Cap the description at 500 characters and the status checks at 20 contexts of 100 characters each, and mirror the first two caps in the form.
216 lines
9.2 KiB
Markdown
216 lines
9.2 KiB
Markdown
# repospawner
|
|
|
|
repospawner turns a JSON "I want a new repository" request into a terraform-git
|
|
pull request, follows that pull request to its merge, and optionally activates
|
|
the new repository in Woodpecker. New repositories in the estate are created by
|
|
Terraform from `config/git.unkin.net/unkin/repository/<name>.yaml`; repospawner
|
|
writes that file and opens the PR so the review gate stays exactly where it is.
|
|
|
|
It is one Go binary carrying its own UI. The server runs as a Deployment and
|
|
launches the work as Kubernetes Jobs running the *same image* with a different
|
|
argv, so there is never a second image to keep in step.
|
|
|
|
## Flow
|
|
|
|
```
|
|
browser / curl
|
|
|
|
|
| POST /api/requests {name, description, woodpecker, status_checks}
|
|
v
|
|
+-------------+ name in use? +----------------+
|
|
| repospawner |----------------->| terraform-git | GET contents/<name>.yaml
|
|
| server |<-----------------| (Gitea) | 404 = free, 200 = 409
|
|
+-------------+ +----------------+
|
|
| 202 {id, status_url}
|
|
|
|
|
| creates Job repospawner-pr-<id>
|
|
v
|
|
+-------------------------------------------------+
|
|
| job pr branch repospawner/<name> |
|
|
| write config/.../<name>.yaml |
|
|
| open the pull request |
|
|
| -> /dev/termination-log |
|
|
| {"pr_number":N,"pr_url":"..."} |
|
|
+-------------------------------------------------+
|
|
| server reads the termination message; state = pr-open
|
|
|
|
|
| creates Job repospawner-watch-<id>
|
|
v
|
|
+-------------------------------------------------+
|
|
| job watch poll the PR every 10s (7d deadline) |
|
|
| -> {"merged":true} / {"closed":true}|
|
|
+-------------------------------------------------+
|
|
| merged -> state = merged
|
|
|
|
|
| creates Job repospawner-woodpecker-<id> (only if woodpecker:true)
|
|
v
|
|
+-------------------------------------------------+
|
|
| job woodpecker-enable |
|
|
| GET gitea repo id -> POST /api/repos?forge_ |
|
|
| remote_id=<id> -> verify -> {"enabled":true} |
|
|
+-------------------------------------------------+
|
|
|
|
|
v
|
|
state = ready
|
|
```
|
|
|
|
Every Job is labelled `repospawner.unkin.net/request=<id>` and
|
|
`repospawner.unkin.net/type=pr|watch|woodpecker`, carries the request payload in
|
|
annotations, sets `ttlSecondsAfterFinished: 3600` and `backoffLimit: 2`.
|
|
|
|
Request state lives in memory and is **reconstructed on startup** from those Job
|
|
labels and annotations, so the Deployment must run **one replica with the
|
|
`Recreate` strategy**. A request whose Jobs have all aged out of the cluster is
|
|
gone from the list; the terraform-git PR it produced is not.
|
|
|
|
## API
|
|
|
|
Every route below the health probes is gated on the oauth2-proxy group header
|
|
(`REPOSPAWNER_GROUPS_HEADER` / `REPOSPAWNER_ALLOWED_GROUPS`) as well as by the
|
|
front door.
|
|
|
|
Submit a request:
|
|
|
|
```console
|
|
$ curl -sS -X POST https://repospawner.k8s.syd1.au.unkin.net/api/requests \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{
|
|
"name": "widget",
|
|
"description": "Widget service",
|
|
"woodpecker": true,
|
|
"status_checks": [
|
|
"ci/woodpecker/pr/build",
|
|
"ci/woodpecker/pr/test",
|
|
"ci/woodpecker/pr/pre-commit"
|
|
]
|
|
}'
|
|
{"id":"9f2c1ab4d0e7","state":"opening-pr","status_url":"/api/requests/9f2c1ab4d0e7"}
|
|
```
|
|
|
|
The response is `202 Accepted`: opening the pull request needs a Vault-minted
|
|
Gitea token and two forge round-trips, which happen in the Job. Poll the status
|
|
URL for the pull request URL:
|
|
|
|
```console
|
|
$ curl -sS https://repospawner.k8s.syd1.au.unkin.net/api/requests/9f2c1ab4d0e7
|
|
{"id":"9f2c1ab4d0e7","name":"widget","state":"pr-open","pr_number":312,
|
|
"pr_url":"https://git.unkin.net/unkin/terraform-git/pulls/312", ...}
|
|
```
|
|
|
|
List every request, newest first (this is what the UI table renders):
|
|
|
|
```console
|
|
$ curl -sS https://repospawner.k8s.syd1.au.unkin.net/api/requests
|
|
```
|
|
|
|
| Route | Meaning |
|
|
| --- | --- |
|
|
| `POST /api/requests` | Submit a request. `202` with `{id, status_url, state}`; `400` with `{error, fields}` on validation failure; `409` if the name is taken (in terraform-git, by an in-flight request, or by a request submitted concurrently); `503` if `woodpecker:true` but no Woodpecker token is mounted. |
|
|
| `GET /api/requests` | Every request, most recent first. |
|
|
| `GET /api/requests/{id}` | One request: `state`, `pr_url`, `error`. |
|
|
| `GET /api/capabilities` | Whether Woodpecker enablement is available. |
|
|
| `GET /livez` | Always `ok` while the process is up. |
|
|
| `GET /readyz` | `ok` when the Kubernetes API is reachable. |
|
|
|
|
### States
|
|
|
|
`opening-pr` -> `pr-open` -> `merged` -> (`enabling-ci` ->) `ready`, with
|
|
`closed` (PR closed unmerged) and `failed` (a Job failed; `error` says why) as
|
|
the other terminal states. A request stuck in `enabling-ci` because the
|
|
Woodpecker token was unmounted after it was accepted fails with
|
|
`woodpecker token unavailable` after five minutes of waiting, rather than
|
|
waiting forever.
|
|
|
|
### Limits
|
|
|
|
`name` is at most 40 characters of `[a-z0-9-]`, `description` at most 500, and a
|
|
request carries at most 20 status check contexts of at most 100 characters each.
|
|
A context may not contain a quote, a newline or a comma.
|
|
|
|
### Generated config
|
|
|
|
Only the description and the status check contexts come from the request;
|
|
everything else is fixed estate policy:
|
|
|
|
```yaml
|
|
description: "Widget service"
|
|
private: false
|
|
default_branch: "main"
|
|
default_delete_branch_after_merge: true
|
|
default_merge_style: "squash"
|
|
branch_protection:
|
|
- rule_name: "main"
|
|
merge_whitelist_teams:
|
|
- "Owners"
|
|
enable_push: false
|
|
status_check_contexts:
|
|
- "ci/woodpecker/pr/build"
|
|
- "ci/woodpecker/pr/test"
|
|
- "ci/woodpecker/pr/pre-commit"
|
|
approval_whitelist_users:
|
|
- "benvin"
|
|
```
|
|
|
|
## Configuration
|
|
|
|
| Variable | Default | Meaning |
|
|
| --- | --- | --- |
|
|
| `REPOSPAWNER_LISTEN` | `:8080` | HTTP listen address. |
|
|
| `REPOSPAWNER_NAMESPACE` | `repospawner` | Namespace the Jobs are created in. |
|
|
| `REPOSPAWNER_IMAGE` | *(required)* | This deployment's own image reference; the Jobs run it. |
|
|
| `REPOSPAWNER_JOB_SERVICE_ACCOUNT` | `repospawner` | Service account the Jobs run as. |
|
|
| `GITEA_URL` | `https://git.unkin.net` | Forge base URL. |
|
|
| `REPOSPAWNER_TFGIT_REPO` | `unkin/terraform-git` | Repository owning the repo config tree. |
|
|
| `VAULT_ADDR` | `https://vault.service.consul:8200` | Vault address. |
|
|
| `REPOSPAWNER_VAULT_K8S_MOUNT` | `k8s/au/syd1` | Kubernetes auth mount. |
|
|
| `REPOSPAWNER_VAULT_K8S_ROLE` | `repospawner` | Kubernetes auth role. |
|
|
| `REPOSPAWNER_VAULT_SA_TOKEN_PATH` | `/var/run/secrets/vault/token` | Projected SA token with audience `vault`. |
|
|
| `REPOSPAWNER_GITEA_CREDS_PATH` | `gitea/creds/repospawner` | Vault path of the dynamic Gitea credential. |
|
|
| `WOODPECKER_SERVER` | `https://ci.k8s.syd1.au.unkin.net` | Woodpecker API base URL. |
|
|
| `REPOSPAWNER_WOODPECKER_TOKEN_FILE` | `/etc/repospawner/woodpecker/token` | Mounted Woodpecker API token. Absent means enablement is unavailable and `woodpecker:true` is refused with `503`. |
|
|
| `REPOSPAWNER_WOODPECKER_SECRET` | `repospawner-woodpecker` | Secret the enablement Job mounts to obtain that file. |
|
|
| `REPOSPAWNER_GROUPS_HEADER` | `X-Forwarded-Groups` | oauth2-proxy group header. |
|
|
| `REPOSPAWNER_ALLOWED_GROUPS` | `akP-repospawner-user` | Allow-list; an empty list is a startup error. |
|
|
|
|
## Credentials
|
|
|
|
repospawner logs into Vault **natively** with the pod's projected service
|
|
account token (audience `vault`) and reads a short-lived Gitea credential from
|
|
`gitea/creds/repospawner`. It does not shell out to `agentpr`: that path
|
|
authenticates with an AppRole whose CIDR binding excludes in-cluster addresses.
|
|
Tokens are minted per operation, never logged, and re-minted when the forge
|
|
answers `401` — the watch Job routinely outlives a one-hour credential.
|
|
|
|
## Deploy prerequisites
|
|
|
|
Provided by the argocd-apps deployment, not by this repository:
|
|
|
|
- ServiceAccount `repospawner` in namespace `repospawner`, bound to a Role
|
|
granting `jobs` `create/get/list/watch/delete`, `pods` `get/list/watch` and
|
|
`pods/log` `get`.
|
|
- A projected `serviceAccountToken` volume with `audience: vault` mounted at
|
|
`/var/run/secrets/vault`, on the Deployment. The Jobs declare their own.
|
|
- Vault kubernetes auth role `repospawner` bound to that service account, with a
|
|
policy allowing `read` on `gitea/creds/repospawner`. *(Already applied.)*
|
|
- Secret `repospawner-woodpecker` with key `token`, seeded from
|
|
`kv/kubernetes/namespace/repospawner/default/woodpecker` (key `token`) via a
|
|
VaultStaticSecret. Optional: without it, Woodpecker enablement is refused
|
|
rather than the service failing to start.
|
|
- One replica, `Recreate` strategy — request state is rebuilt from Jobs.
|
|
- ServiceAccount `repospawner-ci` for the Woodpecker pipelines.
|
|
|
|
## Development
|
|
|
|
```console
|
|
$ make build # dist/repospawner
|
|
$ make test # go test -race
|
|
$ make pre-commit # gofmt, go vet, golangci-lint, pre-commit hooks
|
|
```
|
|
|
|
`repospawner --help` lists the subcommands; `repospawner` with no arguments
|
|
serves the API and UI.
|
|
|
|
Releases are cut with `make patch|minor|major`, which tags and pushes; the tag
|
|
pipeline builds and pushes
|
|
`artifactapi.k8s.syd1.au.unkin.net/docker-internal/repospawner`.
|