repospawner/README.md

repospawner

repospawner turns a JSON "I want a new repository" request into a terraform-git pull request, follows that pull request to its merge, and optionally activates the new repository in Woodpecker. New repositories in the estate are created by Terraform from config/git.unkin.net/unkin/repository/<name>.yaml; repospawner writes that file and opens the PR so the review gate stays exactly where it is.

It is one Go binary carrying its own UI. The server runs as a Deployment and launches the work as Kubernetes Jobs running the same image with a different argv, so there is never a second image to keep in step.

Flow

  browser / curl
        |
        |  POST /api/requests  {name, description, woodpecker, status_checks}
        v
  +-------------+   name in use?   +----------------+
  | repospawner |----------------->| terraform-git  |  GET contents/<name>.yaml
  |   server    |<-----------------|     (Gitea)    |  404 = free, 200 = 409
  +-------------+                  +----------------+
        |  202 {id, status_url}
        |
        |  creates Job  repospawner-pr-<id>
        v
  +-------------------------------------------------+
  | job pr      branch repospawner/<name>           |
  |             write config/.../<name>.yaml        |
  |             open the pull request               |
  |             -> /dev/termination-log             |
  |                {"pr_number":N,"pr_url":"..."}   |
  +-------------------------------------------------+
        |  server reads the termination message; state = pr-open
        |
        |  creates Job  repospawner-watch-<id>
        v
  +-------------------------------------------------+
  | job watch   poll the PR every 10s (7d deadline) |
  |             -> {"merged":true} / {"closed":true}|
  +-------------------------------------------------+
        |  merged -> state = merged
        |
        |  creates Job  repospawner-woodpecker-<id>   (only if woodpecker:true)
        v
  +-------------------------------------------------+
  | job woodpecker-enable                           |
  |   GET gitea repo id -> POST /api/repos?forge_   |
  |   remote_id=<id> -> verify -> {"enabled":true}  |
  +-------------------------------------------------+
        |
        v
     state = ready

Every Job is labelled repospawner.unkin.net/request=<id> and repospawner.unkin.net/type=pr|watch|woodpecker, carries the request payload in annotations, sets ttlSecondsAfterFinished: 3600 and backoffLimit: 2.

Request state lives in memory and is reconstructed on startup from those Job labels and annotations, so the Deployment must run one replica with the Recreate strategy. A request whose Jobs have all aged out of the cluster is gone from the list; the terraform-git PR it produced is not.

API

Every route below the health probes is gated on the oauth2-proxy group header (REPOSPAWNER_GROUPS_HEADER / REPOSPAWNER_ALLOWED_GROUPS) as well as by the front door.

Submit a request:

$ curl -sS -X POST https://repospawner.k8s.syd1.au.unkin.net/api/requests \
    -H 'Content-Type: application/json' \
    -d '{
          "name": "widget",
          "description": "Widget service",
          "woodpecker": true,
          "status_checks": [
            "ci/woodpecker/pr/build",
            "ci/woodpecker/pr/test",
            "ci/woodpecker/pr/pre-commit"
          ]
        }'
{"id":"9f2c1ab4d0e7","state":"opening-pr","status_url":"/api/requests/9f2c1ab4d0e7"}

The response is 202 Accepted: opening the pull request needs a Vault-minted Gitea token and two forge round-trips, which happen in the Job. Poll the status URL for the pull request URL:

$ curl -sS https://repospawner.k8s.syd1.au.unkin.net/api/requests/9f2c1ab4d0e7
{"id":"9f2c1ab4d0e7","name":"widget","state":"pr-open","pr_number":312,
 "pr_url":"https://git.unkin.net/unkin/terraform-git/pulls/312", ...}

List every request, newest first (this is what the UI table renders):

$ curl -sS https://repospawner.k8s.syd1.au.unkin.net/api/requests

Route	Meaning
POST /api/requests	Submit a request. 202 with {id, status_url, state}; 400 with {error, fields} on validation failure; 409 if the name is taken (in terraform-git, by an in-flight request, or by a request submitted concurrently); 503 if woodpecker:true but no Woodpecker token is mounted.
GET /api/requests	Every request, most recent first.
GET /api/requests/{id}	One request: state, pr_url, error.
GET /api/capabilities	Whether Woodpecker enablement is available.
GET /livez	Always ok while the process is up.
GET /readyz	ok when the Kubernetes API is reachable.

States

opening-pr -> pr-open -> merged -> (enabling-ci ->) ready, with closed (PR closed unmerged) and failed (a Job failed; error says why) as the other terminal states. A request stuck in enabling-ci because the Woodpecker token was unmounted after it was accepted fails with woodpecker token unavailable after five minutes of waiting, rather than waiting forever.

Limits

name is at most 40 characters of [a-z0-9-], description at most 500, and a request carries at most 20 status check contexts of at most 100 characters each. A context may not contain a quote, a newline or a comma.

Generated config

Only the description and the status check contexts come from the request; everything else is fixed estate policy:

description: "Widget service"
private: false
default_branch: "main"
default_delete_branch_after_merge: true
default_merge_style: "squash"
branch_protection:
  - rule_name: "main"
    merge_whitelist_teams:
      - "Owners"
    enable_push: false
    status_check_contexts:
      - "ci/woodpecker/pr/build"
      - "ci/woodpecker/pr/test"
      - "ci/woodpecker/pr/pre-commit"
    approval_whitelist_users:
      - "benvin"

Configuration

Variable	Default	Meaning
REPOSPAWNER_LISTEN	:8080	HTTP listen address.
REPOSPAWNER_NAMESPACE	repospawner	Namespace the Jobs are created in.
REPOSPAWNER_IMAGE	(required)	This deployment's own image reference; the Jobs run it.
REPOSPAWNER_JOB_SERVICE_ACCOUNT	repospawner	Service account the Jobs run as.
GITEA_URL	https://git.unkin.net	Forge base URL.
REPOSPAWNER_TFGIT_REPO	unkin/terraform-git	Repository owning the repo config tree.
VAULT_ADDR	https://vault.service.consul:8200	Vault address.
REPOSPAWNER_VAULT_K8S_MOUNT	k8s/au/syd1	Kubernetes auth mount.
REPOSPAWNER_VAULT_K8S_ROLE	repospawner	Kubernetes auth role.
REPOSPAWNER_VAULT_SA_TOKEN_PATH	/var/run/secrets/vault/token	Projected SA token with audience vault.
REPOSPAWNER_GITEA_CREDS_PATH	gitea/creds/repospawner	Vault path of the dynamic Gitea credential.
WOODPECKER_SERVER	https://ci.k8s.syd1.au.unkin.net	Woodpecker API base URL.
REPOSPAWNER_WOODPECKER_TOKEN_FILE	/etc/repospawner/woodpecker/token	Mounted Woodpecker API token. Absent means enablement is unavailable and woodpecker:true is refused with 503.
REPOSPAWNER_WOODPECKER_SECRET	repospawner-woodpecker	Secret the enablement Job mounts to obtain that file.
REPOSPAWNER_GROUPS_HEADER	X-Forwarded-Groups	oauth2-proxy group header.
REPOSPAWNER_ALLOWED_GROUPS	akP-repospawner-user	Allow-list; an empty list is a startup error.

Credentials

repospawner logs into Vault natively with the pod's projected service account token (audience vault) and reads a short-lived Gitea credential from gitea/creds/repospawner. It does not shell out to agentpr: that path authenticates with an AppRole whose CIDR binding excludes in-cluster addresses. Tokens are minted per operation, never logged, and re-minted when the forge answers 401 — the watch Job routinely outlives a one-hour credential.

Deploy prerequisites

Provided by the argocd-apps deployment, not by this repository:

• ServiceAccount repospawner in namespace repospawner, bound to a Role granting jobs create/get/list/watch/delete, pods get/list/watch and pods/log get.
• A projected serviceAccountToken volume with audience: vault mounted at /var/run/secrets/vault, on the Deployment. The Jobs declare their own.
• Vault kubernetes auth role repospawner bound to that service account, with a policy allowing read on gitea/creds/repospawner. (Already applied.)
• Secret repospawner-woodpecker with key token, seeded from kv/kubernetes/namespace/repospawner/default/woodpecker (key token) via a VaultStaticSecret. Optional: without it, Woodpecker enablement is refused rather than the service failing to start.
• One replica, Recreate strategy — request state is rebuilt from Jobs.
• ServiceAccount repospawner-ci for the Woodpecker pipelines.

Development

$ make build       # dist/repospawner
$ make test        # go test -race
$ make pre-commit  # gofmt, go vet, golangci-lint, pre-commit hooks

repospawner --help lists the subcommands; repospawner with no arguments serves the API and UI.

Releases are cut with make patch|minor|major, which tags and pushes; the tag pipeline builds and pushes artifactapi.k8s.syd1.au.unkin.net/docker-internal/repospawner.