Commit Graph

167 Commits

Author SHA1 Message Date
Ben Vincent f887489193 Mount the rancher secrets engine + seed a service account + roles
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/plan Pipeline was successful
Deploys the Rancher token secrets engine into the cluster, mirroring the litellm
and gpg wiring. The plugin is registered separately (config/plugins) and the
deployer policy is granted separately; this mounts and configures the engine.

- Add rancher_secret_backend module (mount + config via the ranchervaultsecret
  provider): rancher_url https://rancher.k8s.syd1.au.unkin.net.
- Add rancher_secret_backend_service_account module: seeds an auto-rotated token
  (90d TTL / 45d rotation), reading the seed token from KV at
  secret_backend/rancher/service_account/<name> (keys: token, token_name).
- Add rancher_secret_backend_role module + a 'ci' role (1h/8h scoped creds).
- Wire config.hcl discovery, module variables, main.tf module blocks, the
  terragrunt inputs, and the rancher provider in root.hcl.

Prereq: populate kv/service/vault/au/syd1/secret_backend/rancher/service_account/admin
with a live Rancher admin token before apply.
2026-07-17 23:45:48 +10:00
unkinben 933de177fa Register + mount the GPG secrets engine at gpg/ (#87)
ci/woodpecker/push/apply Pipeline was successful
Complete the deploy of the [vault-plugin-secrets-gpg](https://git.unkin.net/unkin/vault-plugin-secrets-gpg) engine. Puppet ([puppet-prod #480](unkin/puppet-prod#480)) installs the `openbao-plugin-secrets-gpg` RPM onto the OpenBao nodes; this registers that binary in the plugin catalog and enables the secrets engine so `gpg/` is actually usable.

- Add a `gpg_secret_backend` module using the standard `hashicorp/vault` provider (already required at 5.6.0): `vault_plugin` (catalog register with a pinned sha256) + `vault_mount` (enable at the mount path).
- Wire it through `vault_cluster` (new `gpg_secret_backend` variable + module block) and the config discovery (`config.hcl` group + syd1 terragrunt input), mirroring `litellm_secret_backend`.
- Add `config/gpg_secret_backend/gpg.yaml` mounting at `gpg/` and pinning the released v0.1.0 binary sha256 (`0e92d740…a7b20`, extracted from the published RPM). Puppet installs the RPM floating, so this sha must be bumped in lockstep on any plugin upgrade or OpenBao rejects the binary.

Validated locally with `tofu validate` + `tofu fmt`. Granting non-root access to `gpg/*` (auth roles + policies) is a follow-up scoped to whoever consumes the engine (e.g. passv from CI).

Reviewed-on: #87
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-17 23:19:38 +10:00
unkinben ce1185deba Grant vault deployer access to import + manage the gpg engine (#88)
ci/woodpecker/push/apply Pipeline was successful
## Why
Applying the gpg mount (#87) needs two grants the deployer (`tf_vault` approle / `woodpecker_terraform_vault` k8s role) doesn't have. terraform-vault **registers the plugin itself** (`vault_plugin` → `sys/plugins/catalog`, a sudo-protected path) and **manages keys** via the gpgvaultsecret provider (`gpg/keys/*`). The deployer already has `sys/mounts/*` but neither of these, so apply would 403 on the plugin registration and on `gpg/keys` writes — the same failure mode as #84.

## Changes
- Add `policies/gpg/admin.yaml` granting:
  - `create/read/update/delete/sudo` on `sys/plugins/catalog/secret/vault-plugin-secrets-gpg` — to **import** (register/deregister) the plugin.
  - full management of `gpg/keys/*` (+ `gpg/keys` list) — to **manage keys**.
  - assigned to `tf_vault` (approle) + `woodpecker_terraform_vault` (k8s/au/syd1), mirroring `policies/litellm/admin.yaml` (#84).

Should merge/apply **before** #87 so the deployer can register the plugin and create the `pass` key.

Reviewed-on: #88
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-17 23:09:31 +10:00
unkinben 3d59758324 Add a plugin-import module + config/plugins for catalog registration (#89)
ci/woodpecker/push/apply Pipeline was canceled
Split plugin catalog registration out of the per-engine backend modules into its own concern (previously bundled into #87's gpg_secret_backend).

## Changes
- New generic `plugin` module (`vault_plugin`: type/name/command/sha256/plugin_version) that imports a binary into the catalog.
- New `config/plugins/` discovery group (filename = catalog name = mount type), wired through `vault_cluster` (`plugins` variable + module) and the syd1 environment.
- `config/plugins/vault-plugin-secrets-gpg.yaml` pins the released v0.1.0 binary sha256 (`0e92d740…a7b20`, from the published RPM). Puppet installs the RPM floating, so bump this in lockstep on upgrade.

Any engine now registers its plugin by dropping a file in `config/plugins/`; its `*_secret_backend` module just mounts the registered type.

Needs the deployer's plugin-catalog access (#88). Merge order: **#88 → this → #87**.

Reviewed-on: #89
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-17 23:08:04 +10:00
unkinben 8bb071ae46 Add auth and state access for terraform-rancher (#86)
ci/woodpecker/push/apply Pipeline was successful
## Why

The new `terraform-rancher` repo (manages Rancher's Authentik OIDC auth via the rancher2 provider) needs Vault auth + Consul state, mirroring the terraform-authentik runner (#78/#81/#82).

## Change

- `AppRole/terraform_rancher` + k8s auth role `woodpecker_terraform_rancher` (SA terraform-rancher in the woodpecker ns).
- Consul secret-backend role + ACL policy (`resources/secret_backend/consul_root/au/syd1/terraform-rancher.hcl`) granting write to the `infra/terraform/rancher/` state prefix.
- Vault policies: read the Rancher admin API token (`kv/service/terraform/rancher`) and the keycloakoidc client secret (`kv/kubernetes/namespace/cattle-system/default/oauth-credentials`), plus the consul_root state creds.

Scoped the OAuth read to the `cattle-system` path specifically (rather than the `+` wildcard the authentik policy uses) since the Rancher runner only needs its own app's secret.

## Validation

pre-commit (terragrunt-hcl-fmt + yamllint) passed. CI plan will confirm.

Reviewed-on: #86
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-15 21:37:45 +10:00
benvin 0dba5e00a6 chore: update litellm address (#85)
ci/woodpecker/push/apply Pipeline was successful
- update litellm address
- add a test role

---------

Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #85
2026-07-09 23:47:32 +10:00
unkinben a400e5dc7e fix: grant vault deployer access to manage the litellm engine (#84)
ci/woodpecker/push/apply Pipeline was successful
## Why
Applying the newly-merged litellm mount (#83) failed at apply time with:

```
Error: failed to write litellm config
URL: PUT https://vault.service.consul:8200/v1/litellm/config
Code: 403. * permission denied
```

The deployer identity (`tf_vault` approle / `woodpecker_terraform_vault` k8s role) can enable the mount via `sys/mounts/admin`, but no policy grants it access to the engine's own data paths, so writing the config and roles is denied.

## Changes
- Add `policies/litellm/admin.yaml` granting `create`/`read`/`update`/`delete` on `litellm/config` and `litellm/roles/*` (plus `read`/`list` on `litellm/roles`), assigned to the same auth roles as the other secret-engine admin policies (`tf_vault`, `woodpecker_terraform_vault`).

## Note
The policy attaches to the deployer's auth roles, so it takes effect on the next token issuance — a re-run of the apply (fresh Vault login) will have the permission and can write `litellm/config` and the roles.

Reviewed-on: #84
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-07 20:10:43 +10:00
unkinben 95e7a81b2e feat: manage litellm secrets engine via terraform-provider-litellmvaultsecret (#83)
ci/woodpecker/push/apply Pipeline failed
## Why
The `vault-plugin-secrets-litellm` engine (mints LiteLLM virtual keys) is registered in Vault, but nothing in this repo declared its mount, config, or roles. This wires in the companion `litellm` provider (`git.unkin.net/unkin/litellmvaultsecret`) so the mount is managed as code alongside the other secret backends.

## Changes
- Add `litellm_secret_backend` module that mounts the engine and writes its config (`base_url`, `request_timeout_seconds`); reads the sensitive `master_key` from KV at `kv/service/vault/<country>/<region>/secret_backend/<path>`, matching the consul/kubernetes backend convention.
- Add `litellm_secret_backend_role` module that manages roles (`models`, `max_budget`, `key_alias_prefix`, `ttl`/`max_ttl` in seconds, `metadata`).
- Register both modules in `vault_cluster` `main.tf` and add typed variables in `variables.tf`.
- Discover `litellm_secret_backend[_role]` YAML in `config.hcl` and pass the maps through the terragrunt inputs.
- Declare the `litellm` provider (pinned `0.1.0`) and a `provider "litellm"` block in the generated root `backend.tf`.
- Add example config for the `litellm` mount and a sample `team-a` role.

## Notes
- Requires the `master_key` KV secret to exist at `kv/service/vault/au/syd1/secret_backend/litellm` before apply (the module reads it, does not create it).
- Assumes provider `git.unkin.net/unkin/litellmvaultsecret` `0.1.0` is published to the artifactapi `terraform-unkin` registry.

Reviewed-on: #83
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-07 00:18:29 +10:00
unkinben bbde79d2a6 policies: let terraform-authentik read its provider API token (#82)
ci/woodpecker/push/apply Pipeline was successful
## Why
terraform-authentik's provider needs an Authentik API token (`TF_VAR_authentik_token`), now sourced from Vault at `kv/service/terraform/authentik` (Makefile wiring in terraform-authentik #2). The CI role needs read access to that path.

## Change
Extend `policies/kv/service/terraform/authentik.yaml` to also grant read on `kv/data/service/terraform/authentik` for the `terraform_authentik` approle + `woodpecker_terraform_authentik` k8s role.

Reviewed-on: #82
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-06 23:41:03 +10:00
unkinben f2c54888de policies: let terraform-authentik read oauth client secrets from kv (#81)
ci/woodpecker/push/apply Pipeline was successful
## Why
terraform-authentik now reads OAuth2 client secrets from Vault (`data.vault_kv_secret_v2`) rather than committing them (terraform-authentik #2). But the `terraform_authentik` approle / `woodpecker_terraform_authentik` k8s role only had the consul-creds policy, so `plan` fails with permission denied on the grafana oauth path.

## Change
Add `policies/kv/service/terraform/authentik.yaml` granting read on `kv/data/kubernetes/namespace/+/default/oauth-credentials` for both the approle and the woodpecker k8s role.

Reviewed-on: #81
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-06 23:05:20 +10:00
unkinben 36d7afbb65 feat: add vault/consul config for media terraform repos (#79)
ci/woodpecker/push/apply Pipeline was successful
Add Kubernetes auth roles, AppRole configs, Consul secret backend roles, Consul ACL policies, and Vault kv read policies for terraform-sonarr, terraform-radarr, and terraform-prowlarr.

Reviewed-on: #79
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-28 22:03:25 +10:00
unkinben c33dcdc447 Add auth and state access for terraform-authentik (#78)
ci/woodpecker/push/apply Pipeline was successful
## Summary
- K8s auth role for Woodpecker CI (`terraform-authentik` SA in `woodpecker` namespace)
- AppRole for local terraform runs
- Consul secret backend role (`terraform-authentik`, TTL 120/300)
- Consul ACL policy for `infra/terraform/authentik/` key prefix
- Vault policy granting both auth methods access to Consul creds

Reviewed-on: #78
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-28 01:17:51 +10:00
benvin be9bd96cf3 feat: enable consul state store for artifactapi (#77)
ci/woodpecker/push/apply Pipeline was successful
enable the terraform-artifactapi system to manage its state in consul
using dynamic credentials from kubernetes ci jobs in woodpecker

---------

Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #77
2026-06-17 21:42:25 +10:00
unkinben bb5f6922fa feat: add vault policy for terraform-git webhook secrets (#75)
ci/woodpecker/push/apply Pipeline was successful
## Summary
- Add read policy for kv/data/service/gitea/webhook/* path
- Assigned to terraform_git approle and woodpecker_terraform_git k8s auth role
- Webhook URLs are stored in Vault KV and read at plan/apply time

## Test plan
- [ ] Verify terragrunt plan succeeds for terraform-git after merge

Reviewed-on: #75
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-08 22:56:30 +10:00
benvin 346cf9fa43 feat: manage gitadmin token (#74)
ci/woodpecker/push/apply Pipeline was successful
- add approle for terraform-git
- add policy to read gitadmin token
- update access to the terraform-git consul token

---------

Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #74
2026-06-08 15:17:58 +10:00
unkinben 1288057b81 feat: add vault and consul roles for terraform-git (#73)
ci/woodpecker/push/apply Pipeline was successful
## Summary
- Add K8s auth role woodpecker_terraform_git for CI pipeline authentication
- Add consul secret backend role terraform-git for consul state storage tokens
- Add consul ACL policy granting write access to infra/terraform/git/ key prefix
- Add vault policy for reading consul creds at consul_root/au/syd1/creds/terraform-git

## Test plan
- [ ] Verify terragrunt plan succeeds
- [ ] Verify consul ACL policy is created correctly
- [ ] Verify K8s auth role can authenticate from woodpecker namespace

Reviewed-on: #73
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-07 20:36:35 +10:00
unkinben 3876fa818d chore: bump almalinux9 image tags (#72)
ci/woodpecker/push/apply Pipeline was successful
Bump almalinux9 image tags to 20260606

Reviewed-on: #72
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-07 00:35:30 +10:00
unkinben a548bf1cb1 fix: apply requires plan (#71)
ci/woodpecker/push/apply Pipeline was successful
- ensure make plan runs before make apply when deploying

Reviewed-on: #71
2026-05-22 00:03:08 +10:00
unkinben 93ba86baf3 feat: add apply workflow (#70)
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #70
2026-05-21 23:57:25 +10:00
unkinben 098830c10b Merge pull request 'feat: add plan workflow' (#69) from benvin/make-plan-buildwq into master
Reviewed-on: #69
2026-05-21 23:54:07 +10:00
unkinben 9cbac6d3ef feat: add plan workflow
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/plan Pipeline was successful
- update makefile to enable kubernetes auth or roleid auth
- add plan workflow
- update all policies to allow the terraform-vault kubernetes role
2026-05-21 23:52:30 +10:00
unkinben 73aaaaeb99 Merge pull request 'chore: enable access to gateway.networking.k8s.io' (#68) from benvin/gatewayapi into master
Reviewed-on: #68
2026-05-21 22:42:28 +10:00
unkinben 7c60a5fd53 chore: enable access to gateway.networking.k8s.io
ci/woodpecker/pr/pre-commit Pipeline was successful
2026-05-21 22:39:57 +10:00
unkinben 27f12f183e Merge pull request 'chore: change to specific ci image' (#67) from benvin/ci_image into master
Reviewed-on: #67
2026-03-09 01:16:59 +11:00
unkinben c61434b692 chore: change to specific ci image
ci/woodpecker/pr/pre-commit Pipeline was successful
- almalinux9-opentofu image contains all required tools
2026-03-09 01:14:41 +11:00
unkinben 172ceac2fc Merge pull request 'feat: add templated policies for kubernetes' (#66) from benvin/kubernetes_structured_paths into master
Reviewed-on: #66
2026-03-08 12:57:58 +11:00
unkinben 48a4fd0dd1 feat: add templated policies for kubernetes
ci/woodpecker/pr/pre-commit Pipeline was successful
- add default kubernetes auth role
- add templated access kv/kubernetes/*
2026-03-08 12:48:08 +11:00
unkinben 4dc09547ef Merge pull request 'fix: update audience for rpmbuilder' (#65) from benvin/default_aud into master
Reviewed-on: #65
2026-03-08 12:29:43 +11:00
unkinben 546a9efe44 fix: update audience for rpmbuilder
ci/woodpecker/pr/pre-commit Pipeline was successful
when using using the service account jwt directly, the default audience
is the api servers url
2026-03-07 11:31:36 +11:00
unkinben 679cec4bc1 Merge pull request 'feat: add rpmbuilder k8s role' (#64) from benvin/rpmbuilder-in-k8s into master
Reviewed-on: #64
2026-03-07 11:11:23 +11:00
unkinben 71789f9f32 feat: add rpmbuilder k8s role
ci/woodpecker/pr/pre-commit Pipeline was successful
- create rpmbuilder role
- enable access to gitea/github ro-tokens
- enable access to rpmbuilder role from woodpeckerci
2026-03-07 11:06:27 +11:00
unkinben 4cbcec58d3 Merge pull request 'feat: enable woodpecker access to ro tokens' (#63) from benvin/woodpecker_task_access into master
Reviewed-on: #63
2026-03-07 10:52:38 +11:00
unkinben 9c93e185f8 feat: enable woodpecker access to ro tokens
ci/woodpecker/pr/pre-commit Pipeline was successful
- enable woodpecker tasks to access gitea/github read-only tokens
2026-03-07 10:49:39 +11:00
unkinben d6c8474bd3 Merge pull request 'chore: move pgsql password to vault' (#62) from benvin/artifactapi_postgrespassword into master
Reviewed-on: #62
2026-03-06 19:51:25 +11:00
unkinben 42351000ee chore: move pgsql password to vault
ci/woodpecker/pr/pre-commit Pipeline was successful
- no more storing secrets in configmaps
2026-03-06 19:39:36 +11:00
unkinben f7d1330c37 Merge pull request 'chore: add artifactapi k8s role' (#61) from benvin/artifactapi into master
Reviewed-on: #61
2026-03-06 18:57:05 +11:00
unkinben d9e07e432e chore: add artifactapi k8s role
ci/woodpecker/pr/pre-commit Pipeline was successful
- enable access to read artifactapi secrets
2026-03-06 18:53:42 +11:00
unkinben 14a258de7d Merge pull request 'chore: enable access woodpecker-agent-secret' (#60) from benvin/woodpecker_agent_secret into master
Reviewed-on: #60
2026-03-03 23:34:32 +11:00
unkinben be8bcc3743 chore: enable access woodpecker-agent-secret
ci/woodpecker/pr/pre-commit Pipeline was successful
- add policy to access woodpecker-agent-secret
2026-03-03 23:30:49 +11:00
unkinben dc257b1bcd Merge pull request 'feat: add pre-commit check in ci' (#59) from benvin/woodpecker_integration into master
Reviewed-on: #59
2026-02-28 22:28:21 +11:00
unkinben 66119e5207 feat: add pre-commit check in ci
ci/woodpecker/pr/pre-commit Pipeline was successful
- add a ci workflow to verify pre-commit passes
- fix pre-commit errors/warnings:
  - missing required_version
  - missing required_providers
  - fixed terraform_deprecated_interpolation
  - removed terraform_unused_declarations
2026-02-28 21:42:47 +11:00
unkinben 9e6de4dc32 Merge pull request 'feat: set max token life for auth_kubernetes_role' (#58) from benvin/token_max_ttl into master
Reviewed-on: #58
2026-02-22 22:30:18 +11:00
unkinben 7cafafd483 feat: set max token life for auth_kubernetes_role
found kubernetes vaultauth resources never picking up new policies,
because they would infinitely renew their token.

- set default max token length for roles to 1 day
- changed all existing role token_max_ttl to match their token_ttl
2026-02-22 22:28:21 +11:00
unkinben c94b2af196 Merge pull request 'feat: add woodpecker secrets' (#57) from benvin/woodpecker into master
Reviewed-on: #57
2026-02-22 22:27:50 +11:00
unkinben dd44146d88 feat: add woodpecker secrets
- add secrets required to integrate woodpecker into gitea/pgsql
2026-02-22 22:27:30 +11:00
unkinben 18a62332f6 Merge pull request 'chore: enable access to openldap admin creds' (#56) from benvin/ldap_admin_pass_terraform_ldap into master
Reviewed-on: #56
2026-02-15 20:17:35 +11:00
unkinben 8fa68e2670 chore: enable access to openldap admin creds
- ensure terraform_ldap can read ldap admin credentials
2026-02-15 20:16:58 +11:00
unkinben 4cad39989f Merge pull request 'chore: add default_user_password credentials policy' (#55) from benvin/openldap_default_pass into master
Reviewed-on: #55
2026-02-15 13:45:45 +11:00
unkinben c825962490 chore: add default_user_password credentials policy
- fix the comment for ldap_admin_password
- add policy to read default_user_password
2026-02-15 13:43:02 +11:00
unkinben 51bc3fffc0 Merge pull request 'feat: add terraform-ldap service' (#54) from benvin/terraform-ldap into master
Reviewed-on: #54
2026-02-15 13:40:32 +11:00