Commit Graph

166 Commits

Author SHA1 Message Date
unkinben f7c738fbc2 Manage the litellm plugin via config/plugins (import existing registration)
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/plan Pipeline was successful
Bring the litellm plugin under terraform management like the gpg one. It was
registered manually before terraform owned the catalog, so its state must be
imported before applying, otherwise apply fails creating an entry that already
exists.

- Add config/plugins/vault-plugin-secrets-litellm.yaml (sha256 = released v0.1.1
  openbao binary; verify against the live catalog on import).

Manual pre-step before apply:
  cd environments/au/syd1
  terragrunt import \
    'module.plugin["vault-plugin-secrets-litellm"].vault_plugin.this' \
    secret/vault-plugin-secrets-litellm
2026-07-17 23:16:32 +10:00
unkinben ce1185deba Grant vault deployer access to import + manage the gpg engine (#88)
ci/woodpecker/push/apply Pipeline was successful
## Why
Applying the gpg mount (#87) needs two grants the deployer (`tf_vault` approle / `woodpecker_terraform_vault` k8s role) doesn't have. terraform-vault **registers the plugin itself** (`vault_plugin` → `sys/plugins/catalog`, a sudo-protected path) and **manages keys** via the gpgvaultsecret provider (`gpg/keys/*`). The deployer already has `sys/mounts/*` but neither of these, so apply would 403 on the plugin registration and on `gpg/keys` writes — the same failure mode as #84.

## Changes
- Add `policies/gpg/admin.yaml` granting:
  - `create/read/update/delete/sudo` on `sys/plugins/catalog/secret/vault-plugin-secrets-gpg` — to **import** (register/deregister) the plugin.
  - full management of `gpg/keys/*` (+ `gpg/keys` list) — to **manage keys**.
  - assigned to `tf_vault` (approle) + `woodpecker_terraform_vault` (k8s/au/syd1), mirroring `policies/litellm/admin.yaml` (#84).

Should merge/apply **before** #87 so the deployer can register the plugin and create the `pass` key.

Reviewed-on: #88
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-17 23:09:31 +10:00
unkinben 3d59758324 Add a plugin-import module + config/plugins for catalog registration (#89)
ci/woodpecker/push/apply Pipeline was canceled
Split plugin catalog registration out of the per-engine backend modules into its own concern (previously bundled into #87's gpg_secret_backend).

## Changes
- New generic `plugin` module (`vault_plugin`: type/name/command/sha256/plugin_version) that imports a binary into the catalog.
- New `config/plugins/` discovery group (filename = catalog name = mount type), wired through `vault_cluster` (`plugins` variable + module) and the syd1 environment.
- `config/plugins/vault-plugin-secrets-gpg.yaml` pins the released v0.1.0 binary sha256 (`0e92d740…a7b20`, from the published RPM). Puppet installs the RPM floating, so bump this in lockstep on upgrade.

Any engine now registers its plugin by dropping a file in `config/plugins/`; its `*_secret_backend` module just mounts the registered type.

Needs the deployer's plugin-catalog access (#88). Merge order: **#88 → this → #87**.

Reviewed-on: #89
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-17 23:08:04 +10:00
unkinben 8bb071ae46 Add auth and state access for terraform-rancher (#86)
ci/woodpecker/push/apply Pipeline was successful
## Why

The new `terraform-rancher` repo (manages Rancher's Authentik OIDC auth via the rancher2 provider) needs Vault auth + Consul state, mirroring the terraform-authentik runner (#78/#81/#82).

## Change

- `AppRole/terraform_rancher` + k8s auth role `woodpecker_terraform_rancher` (SA terraform-rancher in the woodpecker ns).
- Consul secret-backend role + ACL policy (`resources/secret_backend/consul_root/au/syd1/terraform-rancher.hcl`) granting write to the `infra/terraform/rancher/` state prefix.
- Vault policies: read the Rancher admin API token (`kv/service/terraform/rancher`) and the keycloakoidc client secret (`kv/kubernetes/namespace/cattle-system/default/oauth-credentials`), plus the consul_root state creds.

Scoped the OAuth read to the `cattle-system` path specifically (rather than the `+` wildcard the authentik policy uses) since the Rancher runner only needs its own app's secret.

## Validation

pre-commit (terragrunt-hcl-fmt + yamllint) passed. CI plan will confirm.

Reviewed-on: #86
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-15 21:37:45 +10:00
benvin 0dba5e00a6 chore: update litellm address (#85)
ci/woodpecker/push/apply Pipeline was successful
- update litellm address
- add a test role

---------

Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #85
2026-07-09 23:47:32 +10:00
unkinben a400e5dc7e fix: grant vault deployer access to manage the litellm engine (#84)
ci/woodpecker/push/apply Pipeline was successful
## Why
Applying the newly-merged litellm mount (#83) failed at apply time with:

```
Error: failed to write litellm config
URL: PUT https://vault.service.consul:8200/v1/litellm/config
Code: 403. * permission denied
```

The deployer identity (`tf_vault` approle / `woodpecker_terraform_vault` k8s role) can enable the mount via `sys/mounts/admin`, but no policy grants it access to the engine's own data paths, so writing the config and roles is denied.

## Changes
- Add `policies/litellm/admin.yaml` granting `create`/`read`/`update`/`delete` on `litellm/config` and `litellm/roles/*` (plus `read`/`list` on `litellm/roles`), assigned to the same auth roles as the other secret-engine admin policies (`tf_vault`, `woodpecker_terraform_vault`).

## Note
The policy attaches to the deployer's auth roles, so it takes effect on the next token issuance — a re-run of the apply (fresh Vault login) will have the permission and can write `litellm/config` and the roles.

Reviewed-on: #84
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-07 20:10:43 +10:00
unkinben 95e7a81b2e feat: manage litellm secrets engine via terraform-provider-litellmvaultsecret (#83)
ci/woodpecker/push/apply Pipeline failed
## Why
The `vault-plugin-secrets-litellm` engine (mints LiteLLM virtual keys) is registered in Vault, but nothing in this repo declared its mount, config, or roles. This wires in the companion `litellm` provider (`git.unkin.net/unkin/litellmvaultsecret`) so the mount is managed as code alongside the other secret backends.

## Changes
- Add `litellm_secret_backend` module that mounts the engine and writes its config (`base_url`, `request_timeout_seconds`); reads the sensitive `master_key` from KV at `kv/service/vault/<country>/<region>/secret_backend/<path>`, matching the consul/kubernetes backend convention.
- Add `litellm_secret_backend_role` module that manages roles (`models`, `max_budget`, `key_alias_prefix`, `ttl`/`max_ttl` in seconds, `metadata`).
- Register both modules in `vault_cluster` `main.tf` and add typed variables in `variables.tf`.
- Discover `litellm_secret_backend[_role]` YAML in `config.hcl` and pass the maps through the terragrunt inputs.
- Declare the `litellm` provider (pinned `0.1.0`) and a `provider "litellm"` block in the generated root `backend.tf`.
- Add example config for the `litellm` mount and a sample `team-a` role.

## Notes
- Requires the `master_key` KV secret to exist at `kv/service/vault/au/syd1/secret_backend/litellm` before apply (the module reads it, does not create it).
- Assumes provider `git.unkin.net/unkin/litellmvaultsecret` `0.1.0` is published to the artifactapi `terraform-unkin` registry.

Reviewed-on: #83
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-07 00:18:29 +10:00
unkinben bbde79d2a6 policies: let terraform-authentik read its provider API token (#82)
ci/woodpecker/push/apply Pipeline was successful
## Why
terraform-authentik's provider needs an Authentik API token (`TF_VAR_authentik_token`), now sourced from Vault at `kv/service/terraform/authentik` (Makefile wiring in terraform-authentik #2). The CI role needs read access to that path.

## Change
Extend `policies/kv/service/terraform/authentik.yaml` to also grant read on `kv/data/service/terraform/authentik` for the `terraform_authentik` approle + `woodpecker_terraform_authentik` k8s role.

Reviewed-on: #82
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-06 23:41:03 +10:00
unkinben f2c54888de policies: let terraform-authentik read oauth client secrets from kv (#81)
ci/woodpecker/push/apply Pipeline was successful
## Why
terraform-authentik now reads OAuth2 client secrets from Vault (`data.vault_kv_secret_v2`) rather than committing them (terraform-authentik #2). But the `terraform_authentik` approle / `woodpecker_terraform_authentik` k8s role only had the consul-creds policy, so `plan` fails with permission denied on the grafana oauth path.

## Change
Add `policies/kv/service/terraform/authentik.yaml` granting read on `kv/data/kubernetes/namespace/+/default/oauth-credentials` for both the approle and the woodpecker k8s role.

Reviewed-on: #81
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-06 23:05:20 +10:00
unkinben 36d7afbb65 feat: add vault/consul config for media terraform repos (#79)
ci/woodpecker/push/apply Pipeline was successful
Add Kubernetes auth roles, AppRole configs, Consul secret backend roles, Consul ACL policies, and Vault kv read policies for terraform-sonarr, terraform-radarr, and terraform-prowlarr.

Reviewed-on: #79
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-28 22:03:25 +10:00
unkinben c33dcdc447 Add auth and state access for terraform-authentik (#78)
ci/woodpecker/push/apply Pipeline was successful
## Summary
- K8s auth role for Woodpecker CI (`terraform-authentik` SA in `woodpecker` namespace)
- AppRole for local terraform runs
- Consul secret backend role (`terraform-authentik`, TTL 120/300)
- Consul ACL policy for `infra/terraform/authentik/` key prefix
- Vault policy granting both auth methods access to Consul creds

Reviewed-on: #78
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-28 01:17:51 +10:00
benvin be9bd96cf3 feat: enable consul state store for artifactapi (#77)
ci/woodpecker/push/apply Pipeline was successful
enable the terraform-artifactapi system to manage its state in consul
using dynamic credentials from kubernetes ci jobs in woodpecker

---------

Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #77
2026-06-17 21:42:25 +10:00
unkinben bb5f6922fa feat: add vault policy for terraform-git webhook secrets (#75)
ci/woodpecker/push/apply Pipeline was successful
## Summary
- Add read policy for kv/data/service/gitea/webhook/* path
- Assigned to terraform_git approle and woodpecker_terraform_git k8s auth role
- Webhook URLs are stored in Vault KV and read at plan/apply time

## Test plan
- [ ] Verify terragrunt plan succeeds for terraform-git after merge

Reviewed-on: #75
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-08 22:56:30 +10:00
benvin 346cf9fa43 feat: manage gitadmin token (#74)
ci/woodpecker/push/apply Pipeline was successful
- add approle for terraform-git
- add policy to read gitadmin token
- update access to the terraform-git consul token

---------

Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #74
2026-06-08 15:17:58 +10:00
unkinben 1288057b81 feat: add vault and consul roles for terraform-git (#73)
ci/woodpecker/push/apply Pipeline was successful
## Summary
- Add K8s auth role woodpecker_terraform_git for CI pipeline authentication
- Add consul secret backend role terraform-git for consul state storage tokens
- Add consul ACL policy granting write access to infra/terraform/git/ key prefix
- Add vault policy for reading consul creds at consul_root/au/syd1/creds/terraform-git

## Test plan
- [ ] Verify terragrunt plan succeeds
- [ ] Verify consul ACL policy is created correctly
- [ ] Verify K8s auth role can authenticate from woodpecker namespace

Reviewed-on: #73
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-07 20:36:35 +10:00
unkinben 3876fa818d chore: bump almalinux9 image tags (#72)
ci/woodpecker/push/apply Pipeline was successful
Bump almalinux9 image tags to 20260606

Reviewed-on: #72
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-07 00:35:30 +10:00
unkinben a548bf1cb1 fix: apply requires plan (#71)
ci/woodpecker/push/apply Pipeline was successful
- ensure make plan runs before make apply when deploying

Reviewed-on: #71
2026-05-22 00:03:08 +10:00
unkinben 93ba86baf3 feat: add apply workflow (#70)
ci/woodpecker/push/apply Pipeline was successful
Reviewed-on: #70
2026-05-21 23:57:25 +10:00
unkinben 098830c10b Merge pull request 'feat: add plan workflow' (#69) from benvin/make-plan-buildwq into master
Reviewed-on: #69
2026-05-21 23:54:07 +10:00
unkinben 9cbac6d3ef feat: add plan workflow
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/plan Pipeline was successful
- update makefile to enable kubernetes auth or roleid auth
- add plan workflow
- update all policies to allow the terraform-vault kubernetes role
2026-05-21 23:52:30 +10:00
unkinben 73aaaaeb99 Merge pull request 'chore: enable access to gateway.networking.k8s.io' (#68) from benvin/gatewayapi into master
Reviewed-on: #68
2026-05-21 22:42:28 +10:00
unkinben 7c60a5fd53 chore: enable access to gateway.networking.k8s.io
ci/woodpecker/pr/pre-commit Pipeline was successful
2026-05-21 22:39:57 +10:00
unkinben 27f12f183e Merge pull request 'chore: change to specific ci image' (#67) from benvin/ci_image into master
Reviewed-on: #67
2026-03-09 01:16:59 +11:00
unkinben c61434b692 chore: change to specific ci image
ci/woodpecker/pr/pre-commit Pipeline was successful
- almalinux9-opentofu image contains all required tools
2026-03-09 01:14:41 +11:00
unkinben 172ceac2fc Merge pull request 'feat: add templated policies for kubernetes' (#66) from benvin/kubernetes_structured_paths into master
Reviewed-on: #66
2026-03-08 12:57:58 +11:00
unkinben 48a4fd0dd1 feat: add templated policies for kubernetes
ci/woodpecker/pr/pre-commit Pipeline was successful
- add default kubernetes auth role
- add templated access kv/kubernetes/*
2026-03-08 12:48:08 +11:00
unkinben 4dc09547ef Merge pull request 'fix: update audience for rpmbuilder' (#65) from benvin/default_aud into master
Reviewed-on: #65
2026-03-08 12:29:43 +11:00
unkinben 546a9efe44 fix: update audience for rpmbuilder
ci/woodpecker/pr/pre-commit Pipeline was successful
when using using the service account jwt directly, the default audience
is the api servers url
2026-03-07 11:31:36 +11:00
unkinben 679cec4bc1 Merge pull request 'feat: add rpmbuilder k8s role' (#64) from benvin/rpmbuilder-in-k8s into master
Reviewed-on: #64
2026-03-07 11:11:23 +11:00
unkinben 71789f9f32 feat: add rpmbuilder k8s role
ci/woodpecker/pr/pre-commit Pipeline was successful
- create rpmbuilder role
- enable access to gitea/github ro-tokens
- enable access to rpmbuilder role from woodpeckerci
2026-03-07 11:06:27 +11:00
unkinben 4cbcec58d3 Merge pull request 'feat: enable woodpecker access to ro tokens' (#63) from benvin/woodpecker_task_access into master
Reviewed-on: #63
2026-03-07 10:52:38 +11:00
unkinben 9c93e185f8 feat: enable woodpecker access to ro tokens
ci/woodpecker/pr/pre-commit Pipeline was successful
- enable woodpecker tasks to access gitea/github read-only tokens
2026-03-07 10:49:39 +11:00
unkinben d6c8474bd3 Merge pull request 'chore: move pgsql password to vault' (#62) from benvin/artifactapi_postgrespassword into master
Reviewed-on: #62
2026-03-06 19:51:25 +11:00
unkinben 42351000ee chore: move pgsql password to vault
ci/woodpecker/pr/pre-commit Pipeline was successful
- no more storing secrets in configmaps
2026-03-06 19:39:36 +11:00
unkinben f7d1330c37 Merge pull request 'chore: add artifactapi k8s role' (#61) from benvin/artifactapi into master
Reviewed-on: #61
2026-03-06 18:57:05 +11:00
unkinben d9e07e432e chore: add artifactapi k8s role
ci/woodpecker/pr/pre-commit Pipeline was successful
- enable access to read artifactapi secrets
2026-03-06 18:53:42 +11:00
unkinben 14a258de7d Merge pull request 'chore: enable access woodpecker-agent-secret' (#60) from benvin/woodpecker_agent_secret into master
Reviewed-on: #60
2026-03-03 23:34:32 +11:00
unkinben be8bcc3743 chore: enable access woodpecker-agent-secret
ci/woodpecker/pr/pre-commit Pipeline was successful
- add policy to access woodpecker-agent-secret
2026-03-03 23:30:49 +11:00
unkinben dc257b1bcd Merge pull request 'feat: add pre-commit check in ci' (#59) from benvin/woodpecker_integration into master
Reviewed-on: #59
2026-02-28 22:28:21 +11:00
unkinben 66119e5207 feat: add pre-commit check in ci
ci/woodpecker/pr/pre-commit Pipeline was successful
- add a ci workflow to verify pre-commit passes
- fix pre-commit errors/warnings:
  - missing required_version
  - missing required_providers
  - fixed terraform_deprecated_interpolation
  - removed terraform_unused_declarations
2026-02-28 21:42:47 +11:00
unkinben 9e6de4dc32 Merge pull request 'feat: set max token life for auth_kubernetes_role' (#58) from benvin/token_max_ttl into master
Reviewed-on: #58
2026-02-22 22:30:18 +11:00
unkinben 7cafafd483 feat: set max token life for auth_kubernetes_role
found kubernetes vaultauth resources never picking up new policies,
because they would infinitely renew their token.

- set default max token length for roles to 1 day
- changed all existing role token_max_ttl to match their token_ttl
2026-02-22 22:28:21 +11:00
unkinben c94b2af196 Merge pull request 'feat: add woodpecker secrets' (#57) from benvin/woodpecker into master
Reviewed-on: #57
2026-02-22 22:27:50 +11:00
unkinben dd44146d88 feat: add woodpecker secrets
- add secrets required to integrate woodpecker into gitea/pgsql
2026-02-22 22:27:30 +11:00
unkinben 18a62332f6 Merge pull request 'chore: enable access to openldap admin creds' (#56) from benvin/ldap_admin_pass_terraform_ldap into master
Reviewed-on: #56
2026-02-15 20:17:35 +11:00
unkinben 8fa68e2670 chore: enable access to openldap admin creds
- ensure terraform_ldap can read ldap admin credentials
2026-02-15 20:16:58 +11:00
unkinben 4cad39989f Merge pull request 'chore: add default_user_password credentials policy' (#55) from benvin/openldap_default_pass into master
Reviewed-on: #55
2026-02-15 13:45:45 +11:00
unkinben c825962490 chore: add default_user_password credentials policy
- fix the comment for ldap_admin_password
- add policy to read default_user_password
2026-02-15 13:43:02 +11:00
unkinben 51bc3fffc0 Merge pull request 'feat: add terraform-ldap service' (#54) from benvin/terraform-ldap into master
Reviewed-on: #54
2026-02-15 13:40:32 +11:00
unkinben dca26029c0 feat: add terraform-ldap service
- add consul role/policy/acls to allow terraform-ldap state management
- add approle to generate tokens for consul
2026-02-15 13:38:31 +11:00