# repospawner repospawner turns a JSON "I want a new repository" request into a terraform-git pull request, follows that pull request to its merge, and optionally activates the new repository in Woodpecker. New repositories in the estate are created by Terraform from `config/git.unkin.net/unkin/repository/.yaml`; repospawner writes that file and opens the PR so the review gate stays exactly where it is. It is one Go binary carrying its own UI. The server runs as a Deployment and launches the work as Kubernetes Jobs running the *same image* with a different argv, so there is never a second image to keep in step. ## Flow ``` browser / curl | | POST /api/requests {name, description, woodpecker, status_checks} v +-------------+ name in use? +----------------+ | repospawner |----------------->| terraform-git | GET contents/.yaml | server |<-----------------| (Gitea) | 404 = free, 200 = 409 +-------------+ +----------------+ | 202 {id, status_url} | | creates Job repospawner-pr- v +-------------------------------------------------+ | job pr branch repospawner/ | | write config/.../.yaml | | open the pull request | | -> /dev/termination-log | | {"pr_number":N,"pr_url":"..."} | +-------------------------------------------------+ | server reads the termination message; state = pr-open | | creates Job repospawner-watch- v +-------------------------------------------------+ | job watch poll the PR every 10s (7d deadline) | | -> {"merged":true} / {"closed":true}| +-------------------------------------------------+ | merged -> state = merged | | creates Job repospawner-woodpecker- (only if woodpecker:true) v +-------------------------------------------------+ | job woodpecker-enable | | GET gitea repo id -> POST /api/repos?forge_ | | remote_id= -> verify -> {"enabled":true} | +-------------------------------------------------+ | v state = ready ``` Every Job is labelled `repospawner.unkin.net/request=` and `repospawner.unkin.net/type=pr|watch|woodpecker`, carries the request payload in annotations, sets `ttlSecondsAfterFinished: 3600` and `backoffLimit: 2`. Request state lives in memory and is **reconstructed on startup** from those Job labels and annotations, so the Deployment must run **one replica with the `Recreate` strategy**. A request whose Jobs have all aged out of the cluster is gone from the list; the terraform-git PR it produced is not. ## API Every route below the health probes is gated on the oauth2-proxy group header (`REPOSPAWNER_GROUPS_HEADER` / `REPOSPAWNER_ALLOWED_GROUPS`) as well as by the front door. Submit a request: ```console $ curl -sS -X POST https://repospawner.k8s.syd1.au.unkin.net/api/requests \ -H 'Content-Type: application/json' \ -d '{ "name": "widget", "description": "Widget service", "woodpecker": true, "status_checks": [ "ci/woodpecker/pr/build", "ci/woodpecker/pr/test", "ci/woodpecker/pr/pre-commit" ] }' {"id":"9f2c1ab4d0e7","state":"opening-pr","status_url":"/api/requests/9f2c1ab4d0e7"} ``` The response is `202 Accepted`: opening the pull request needs a Vault-minted Gitea token and two forge round-trips, which happen in the Job. Poll the status URL for the pull request URL: ```console $ curl -sS https://repospawner.k8s.syd1.au.unkin.net/api/requests/9f2c1ab4d0e7 {"id":"9f2c1ab4d0e7","name":"widget","state":"pr-open","pr_number":312, "pr_url":"https://git.unkin.net/unkin/terraform-git/pulls/312", ...} ``` List every request, newest first (this is what the UI table renders): ```console $ curl -sS https://repospawner.k8s.syd1.au.unkin.net/api/requests ``` | Route | Meaning | | --- | --- | | `POST /api/requests` | Submit a request. `202` with `{id, status_url, state}`; `400` with `{error, fields}` on validation failure; `409` if the name is taken (in terraform-git, by an in-flight request, or by a request submitted concurrently); `503` if `woodpecker:true` but no Woodpecker token is mounted. | | `GET /api/requests` | Every request, most recent first. | | `GET /api/requests/{id}` | One request: `state`, `pr_url`, `error`. | | `GET /api/capabilities` | Whether Woodpecker enablement is available. | | `GET /livez` | Always `ok` while the process is up. | | `GET /readyz` | `ok` when the Kubernetes API is reachable. | ### States `opening-pr` -> `pr-open` -> `merged` -> (`enabling-ci` ->) `ready`, with `closed` (PR closed unmerged) and `failed` (a Job failed; `error` says why) as the other terminal states. A request stuck in `enabling-ci` because the Woodpecker token was unmounted after it was accepted fails with `woodpecker token unavailable` after five minutes of waiting, rather than waiting forever. ### Limits `name` is at most 40 characters of `[a-z0-9-]`, `description` at most 500, and a request carries at most 20 status check contexts of at most 100 characters each. A context may not contain a quote, a newline or a comma. ### Generated config Only the description and the status check contexts come from the request; everything else is fixed estate policy: ```yaml description: "Widget service" private: false default_branch: "main" default_delete_branch_after_merge: true default_merge_style: "squash" branch_protection: - rule_name: "main" merge_whitelist_teams: - "Owners" enable_push: false status_check_contexts: - "ci/woodpecker/pr/build" - "ci/woodpecker/pr/test" - "ci/woodpecker/pr/pre-commit" approval_whitelist_users: - "benvin" ``` ## Configuration | Variable | Default | Meaning | | --- | --- | --- | | `REPOSPAWNER_LISTEN` | `:8080` | HTTP listen address. | | `REPOSPAWNER_NAMESPACE` | `repospawner` | Namespace the Jobs are created in. | | `REPOSPAWNER_IMAGE` | *(required)* | This deployment's own image reference; the Jobs run it. | | `REPOSPAWNER_JOB_SERVICE_ACCOUNT` | `repospawner` | Service account the Jobs run as. | | `GITEA_URL` | `https://git.unkin.net` | Forge base URL. | | `REPOSPAWNER_TFGIT_REPO` | `unkin/terraform-git` | Repository owning the repo config tree. | | `VAULT_ADDR` | `https://vault.service.consul:8200` | Vault address. | | `REPOSPAWNER_VAULT_K8S_MOUNT` | `k8s/au/syd1` | Kubernetes auth mount. | | `REPOSPAWNER_VAULT_K8S_ROLE` | `repospawner` | Kubernetes auth role. | | `REPOSPAWNER_VAULT_SA_TOKEN_PATH` | `/var/run/secrets/vault/token` | Projected SA token with audience `vault`. | | `REPOSPAWNER_GITEA_CREDS_PATH` | `gitea/creds/repospawner` | Vault path of the dynamic Gitea credential. | | `WOODPECKER_SERVER` | `https://ci.k8s.syd1.au.unkin.net` | Woodpecker API base URL. | | `REPOSPAWNER_WOODPECKER_TOKEN_FILE` | `/etc/repospawner/woodpecker/token` | Mounted Woodpecker API token. Absent means enablement is unavailable and `woodpecker:true` is refused with `503`. | | `REPOSPAWNER_WOODPECKER_SECRET` | `repospawner-woodpecker` | Secret the enablement Job mounts to obtain that file. | | `REPOSPAWNER_GROUPS_HEADER` | `X-Forwarded-Groups` | oauth2-proxy group header. | | `REPOSPAWNER_ALLOWED_GROUPS` | `akP-repospawner-user` | Allow-list; an empty list is a startup error. | ## Credentials repospawner logs into Vault **natively** with the pod's projected service account token (audience `vault`) and reads a short-lived Gitea credential from `gitea/creds/repospawner`. It does not shell out to `agentpr`: that path authenticates with an AppRole whose CIDR binding excludes in-cluster addresses. Tokens are minted per operation, never logged, and re-minted when the forge answers `401` — the watch Job routinely outlives a one-hour credential. ## Deploy prerequisites Provided by the argocd-apps deployment, not by this repository: - ServiceAccount `repospawner` in namespace `repospawner`, bound to a Role granting `jobs` `create/get/list/watch/delete`, `pods` `get/list/watch` and `pods/log` `get`. - A projected `serviceAccountToken` volume with `audience: vault` mounted at `/var/run/secrets/vault`, on the Deployment. The Jobs declare their own. - Vault kubernetes auth role `repospawner` bound to that service account, with a policy allowing `read` on `gitea/creds/repospawner`. *(Already applied.)* - Secret `repospawner-woodpecker` with key `token`, seeded from `kv/kubernetes/namespace/repospawner/default/woodpecker` (key `token`) via a VaultStaticSecret. Optional: without it, Woodpecker enablement is refused rather than the service failing to start. - One replica, `Recreate` strategy — request state is rebuilt from Jobs. - ServiceAccount `repospawner-ci` for the Woodpecker pipelines. ## Development ```console $ make build # dist/repospawner $ make test # go test -race $ make pre-commit # gofmt, go vet, golangci-lint, pre-commit hooks ``` `repospawner --help` lists the subcommands; `repospawner` with no arguments serves the API and UI. Releases are cut with `make patch|minor|major`, which tags and pushes; the tag pipeline builds and pushes `artifactapi.k8s.syd1.au.unkin.net/docker-internal/repospawner`.