repospawner/README.md

repospawner

repospawner turns a JSON "I want a new repository" request into a terraform-git pull request, follows that pull request to its merge, and optionally activates the new repository in Woodpecker. New repositories in the estate are created by Terraform from config/git.unkin.net/unkin/repository/<name>.yaml; repospawner writes that file and opens the PR so the review gate stays exactly where it is.

It is one Go binary carrying its own UI. The server runs as a Deployment and launches the work as Kubernetes Jobs running the same image with a different argv, so there is never a second image to keep in step.

Flow

  browser / curl
        |
        |  POST /api/requests  {name, description, woodpecker, status_checks}
        v
  +-------------+   name in use?   +----------------+
  | repospawner |----------------->| terraform-git  |  GET contents/<name>.yaml
  |   server    |<-----------------|     (Gitea)    |  404 = free, 200 = 409
  +-------------+                  +----------------+
        |  202 {id, status_url}
        |
        |  creates Job  repospawner-pr-<id>
        v
  +-------------------------------------------------+
  | job pr      branch repospawner/<name>           |
  |             write config/.../<name>.yaml        |
  |             open the pull request               |
  |             -> /dev/termination-log             |
  |                {"pr_number":N,"pr_url":"..."}   |
  +-------------------------------------------------+
        |  server reads the termination message; state = pr-open
        |
        |  creates Job  repospawner-watch-<id>
        v
  +-------------------------------------------------+
  | job watch   poll the PR every 10s (7d deadline) |
  |             -> {"merged":true} / {"closed":true}|
  +-------------------------------------------------+
        |  merged -> state = merged
        |
        |  creates Job  repospawner-woodpecker-<id>   (only if woodpecker:true)
        v
  +-------------------------------------------------+
  | job woodpecker-enable                           |
  |   GET gitea repo id -> POST /api/repos?forge_   |
  |   remote_id=<id> -> verify -> {"enabled":true}  |
  +-------------------------------------------------+
        |
        v
     state = ready

Every Job is labelled repospawner.unkin.net/request=<id> and repospawner.unkin.net/type=pr|watch|woodpecker, carries the request payload in annotations, sets ttlSecondsAfterFinished: 3600 and backoffLimit: 2.

Request state lives in memory and is reconstructed on startup from those Job labels and annotations, so the Deployment must run one replica with the Recreate strategy. A request whose Jobs have all aged out of the cluster is gone from the list; the terraform-git PR it produced is not.

API

Every route below the health probes is gated on the oauth2-proxy group header (REPOSPAWNER_GROUPS_HEADER / REPOSPAWNER_ALLOWED_GROUPS) as well as by the front door.

Submit a request:

$ curl -sS -X POST https://repospawner.k8s.syd1.au.unkin.net/api/requests \
    -H 'Content-Type: application/json' \
    -d '{
          "name": "widget",
          "description": "Widget service",
          "woodpecker": true,
          "status_checks": [
            "ci/woodpecker/pr/build",
            "ci/woodpecker/pr/test",
            "ci/woodpecker/pr/pre-commit"
          ]
        }'
{"id":"9f2c1ab4d0e7","state":"opening-pr","status_url":"/api/requests/9f2c1ab4d0e7"}

The response is 202 Accepted: opening the pull request needs a Vault-minted Gitea token and two forge round-trips, which happen in the Job. Poll the status URL for the pull request URL:

$ curl -sS https://repospawner.k8s.syd1.au.unkin.net/api/requests/9f2c1ab4d0e7
{"id":"9f2c1ab4d0e7","name":"widget","state":"pr-open","pr_number":312,
 "pr_url":"https://git.unkin.net/unkin/terraform-git/pulls/312", ...}

List every request, newest first (this is what the UI table renders):

$ curl -sS https://repospawner.k8s.syd1.au.unkin.net/api/requests

Route	Meaning
POST /api/requests	Submit a request. 202 with {id, status_url, state}; 400 with {error, fields} on validation failure; 409 if the name is taken (in terraform-git or by an in-flight request); 503 if woodpecker:true but no Woodpecker token is mounted.
GET /api/requests	Every request, most recent first.
GET /api/requests/{id}	One request: state, pr_url, error.
GET /api/capabilities	Whether Woodpecker enablement is available.
GET /livez	Always ok while the process is up.
GET /readyz	ok when the Kubernetes API is reachable.

States

opening-pr -> pr-open -> merged -> (enabling-ci ->) ready, with closed (PR closed unmerged) and failed (a Job failed; error says why) as the other terminal states.

Generated config

Only the description and the status check contexts come from the request; everything else is fixed estate policy:

description: "Widget service"
private: false
default_branch: "main"
default_delete_branch_after_merge: true
default_merge_style: "squash"
branch_protection:
  - rule_name: "main"
    merge_whitelist_teams:
      - "Owners"
    enable_push: false
    status_check_contexts:
      - "ci/woodpecker/pr/build"
      - "ci/woodpecker/pr/test"
      - "ci/woodpecker/pr/pre-commit"
    approval_whitelist_users:
      - "benvin"

Configuration

Variable	Default	Meaning
REPOSPAWNER_LISTEN	:8080	HTTP listen address.
REPOSPAWNER_NAMESPACE	repospawner	Namespace the Jobs are created in.
REPOSPAWNER_IMAGE	(required)	This deployment's own image reference; the Jobs run it.
REPOSPAWNER_JOB_SERVICE_ACCOUNT	repospawner	Service account the Jobs run as.
GITEA_URL	https://git.unkin.net	Forge base URL.
REPOSPAWNER_TFGIT_REPO	unkin/terraform-git	Repository owning the repo config tree.
VAULT_ADDR	https://vault.service.consul:8200	Vault address.
REPOSPAWNER_VAULT_K8S_MOUNT	k8s/au/syd1	Kubernetes auth mount.
REPOSPAWNER_VAULT_K8S_ROLE	repospawner	Kubernetes auth role.
REPOSPAWNER_VAULT_SA_TOKEN_PATH	/var/run/secrets/vault/token	Projected SA token with audience vault.
REPOSPAWNER_GITEA_CREDS_PATH	gitea/creds/repospawner	Vault path of the dynamic Gitea credential.
WOODPECKER_SERVER	https://ci.k8s.syd1.au.unkin.net	Woodpecker API base URL.
REPOSPAWNER_WOODPECKER_TOKEN_FILE	/etc/repospawner/woodpecker/token	Mounted Woodpecker API token. Absent means enablement is unavailable and woodpecker:true is refused with 503.
REPOSPAWNER_WOODPECKER_SECRET	repospawner-woodpecker	Secret the enablement Job mounts to obtain that file.
REPOSPAWNER_GROUPS_HEADER	X-Forwarded-Groups	oauth2-proxy group header.
REPOSPAWNER_ALLOWED_GROUPS	akP-repospawner-user	Allow-list; an empty list is a startup error.

Credentials

repospawner logs into Vault natively with the pod's projected service account token (audience vault) and reads a short-lived Gitea credential from gitea/creds/repospawner. It does not shell out to agentpr: that path authenticates with an AppRole whose CIDR binding excludes in-cluster addresses. Tokens are minted per operation, never logged, and re-minted when the forge answers 401 — the watch Job routinely outlives a one-hour credential.

Deploy prerequisites

Provided by the argocd-apps deployment, not by this repository:

• ServiceAccount repospawner in namespace repospawner, bound to a Role granting jobs create/get/list/watch/delete, pods get/list/watch and pods/log get.
• A projected serviceAccountToken volume with audience: vault mounted at /var/run/secrets/vault, on the Deployment. The Jobs declare their own.
• Vault kubernetes auth role repospawner bound to that service account, with a policy allowing read on gitea/creds/repospawner. (Already applied.)
• Secret repospawner-woodpecker with key token, seeded from kv/kubernetes/namespace/repospawner/default/woodpecker (key token) via a VaultStaticSecret. Optional: without it, Woodpecker enablement is refused rather than the service failing to start.
• One replica, Recreate strategy — request state is rebuilt from Jobs.
• ServiceAccount repospawner-ci for the Woodpecker pipelines.

Development

$ make build       # dist/repospawner
$ make test        # go test -race
$ make pre-commit  # gofmt, go vet, golangci-lint, pre-commit hooks

repospawner --help lists the subcommands; repospawner with no arguments serves the API and UI.

Releases are cut with make patch|minor|major, which tags and pushes; the tag pipeline builds and pushes artifactapi.k8s.syd1.au.unkin.net/docker-internal/repospawner.