17 Commits

Author SHA1 Message Date
unkin-agent 6c68057e73 operator RO creds: read operator CRD API groups (#135)
ci/woodpecker/push/apply Pipeline was successful
## Why

The `operator` kube context is a Vault-minted, read-only credential (Kubernetes
secret engine role `cluster-operator`, bound to a `get/list/watch`-only
ClusterRole). It is currently RBAC-forbidden from listing operator-owned CRDs —
the immediate breakage is `valkeyclusters.valkey.io` — and likewise every other
operator CRD group deployed via `argocd-apps`. This extends the RO ruleset so the
context can read those CRDs. Still strictly read-only: no create/update/delete.

## Change

- Extend the `cluster-operator` generated_role_rules
  (`resources/secret_backend/kubernetes/au/syd1/roles/cluster-operator.yaml`)
  with `get/list/watch` on the CRD API groups of the operators deployed via
  `argocd-apps` (verbs and `resources: "*"` unchanged; same single rule block).

## API groups added

- `valkey.io` (valkey-operator — immediate need)
- `ceph.unkin.net` (cephrgw-operator)
- `bind.unkin.net` (bind-operator)
- `kea.unkin.net` (kea/dhcp operator)
- `k8up.io` (k8up)
- `grafana.integreatly.org` (grafana-operator)
- `operator.victoriametrics.com` (VictoriaMetrics operator)
- `clickhouse.altinity.com`, `clickhouse-keeper.altinity.com` (altinity clickhouse-operator)
- `acme.cert-manager.io` (cert-manager companion CRD group)
- `deviceplugin.intel.com`, `fpga.intel.com` (intel device plugins operator)
- `autoscaling.k8s.io` (VPA)
- `apm.k8s.elastic.co`, `beat.k8s.elastic.co`, `agent.k8s.elastic.co`,
  `maps.k8s.elastic.co`, `enterprisesearch.k8s.elastic.co`,
  `autoscaling.k8s.elastic.co`, `stackconfigpolicy.k8s.elastic.co` (ECK — the
  `elasticsearch`/`kibana`/`logstash` ECK groups were already granted)
- `snapshot.storage.k8s.io`, `groupsnapshot.storage.k8s.io` (CSI external-snapshotter, deployed via csi-cephfs/csi-cephrbd)

Groups already present (`postgresql.cnpg.io`, `cert-manager.io`,
`externaldns.k8s.io`, `secrets.hashicorp.com`, `purelb.io`, `nfd.k8s-sigs.io`,
`elasticsearch/kibana/logstash.k8s.elastic.co`, `gateway.networking.k8s.io`,
etc.) are unchanged. Rancher/RKE/Calico/cluster-api/fleet management-layer CRD
groups are intentionally excluded — they are not `argocd-apps` operators.

---------

Co-authored-by: unkin-agent <agent@unkin.net>
Reviewed-on: #135
Co-authored-by: Unkin Agent <unkin-agent@unkin.net>
Co-committed-by: Unkin Agent <unkin-agent@unkin.net>
2026-08-23 12:29:02 +10:00
unkinben 95927202ba Rename terraform-ipam CI Vault access -> terraform-infra (#111)
ci/woodpecker/push/apply Pipeline was successful
Follows the `terraform-ipam` -> `terraform-infra` repo rename. Renames the k8s auth role (`woodpecker_terraform_infra`), consul secret-backend role + ACL policy (`terraform-infra`, state path `infra/terraform/infra/*`), consul creds read policy, and kv read policy (`kv/service/terraform/infra`).

https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT
Reviewed-on: #111
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-08-06 22:25:38 +10:00
unkinben d61e985ef2 Add terraform-ipam CI Vault access (#110)
ci/woodpecker/push/apply Pipeline was successful
Grants the `terraform-ipam` Woodpecker pipeline its Vault footprint, mirroring the `terraform-artifactapi` set plus a KV read for provider tokens.

- k8s auth role `woodpecker_terraform_ipam` (SA `terraform-ipam`, ns `woodpecker`)
- consul secret backend role `terraform-ipam` + ACL policy writing `infra/terraform/ipam/*`
- consul creds read policy
- kv read policy for `kv/service/terraform/ipam` (`netbox_token` + `kea_token`)

The KV secret itself still needs seeding before the pipeline can auth to NetBox/Kea.

https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT
Reviewed-on: #110
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-08-03 00:16:34 +10:00
unkinben 0e6da5cdd3 Add Vault-scoped agent kubernetes roles + agents AppRole (#109)
ci/woodpecker/push/apply Pipeline was successful
## Why
Agentic workloads currently need cluster-admin/root kubeconfig contexts to do routine per-domain work. This adds domain-scoped, Vault-issued kubernetes credentials plus an `agents` AppRole so agents get least-privilege access instead of escalating.

## Changes
- Add `kubernetes_secret_backend_role` configs `agent-dhcp`, `agent-dns`, `agent-certs`, `agent-storage` (au/syd1):
  - **agent-dhcp** (Role, ns `dhcp-system`): full verbs on `kea.unkin.net` CRDs; get/list/watch pods/services/configmaps/events + pods/log.
  - **agent-dns** (`service_account_name` mode): mints tokens for the static `agent-dns` SA (argocd-apps#332) whose per-namespace RoleBindings confine access to bind-system/bind-internal/bind-external/externaldns. `allowed_kubernetes_namespaces: [bind-system]` (the SA's namespace).
  - **agent-certs** (Role, ns `cert-manager`): full verbs on `cert-manager.io` + `acme.cert-manager.io` (closes the orders/challenges debugging gap); get/list/watch/delete secrets; get/list/watch pods + pods/log. Secret delete confined to `cert-manager`.
  - **agent-storage** (Role, ns `cephrgw-system`): full verbs on `ceph.unkin.net` CRDs (buckets/bucketaccesses/objectstoreusers); get/list/watch pods + pods/log.
- Extend the `kubernetes_secret_backend_role` module with an optional `service_account_name`; when set, `generated_role_rules`/`kubernetes_role_type` are omitted (the SA's own bindings supply RBAC).
- Add creds policies for each role, bound to the `kubernetes_au_syd1_cluster_operator` ldap group (human kubectl use) and the `agents` AppRole (programmatic use).
- Add the `agents` AppRole (mirrors the certmanager approle schema): `bind_secret_id: false` (role_id-only login), `token_bound_cidrs: [10.10.12.200/32]` (agent workstation wg0 addr), deterministic role_id, 1h/4h TTLs.
- Add `policies/kv/kubernetes/agents.yaml` granting the AppRole create/read/update/list on `kv/data/kubernetes/*` + read/list on `kv/metadata/kubernetes/*` (no delete).

## Ordering
argocd-apps#332 (the `agent-dns` SA + ClusterRole + per-namespace RoleBindings) must sync **before** the `agent-dns` creds here are usable — Vault mints tokens for a service account that must already exist.

https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT
Reviewed-on: #109
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-08-02 21:55:05 +10:00
unkinben 1fa5900787 Add terraform-enc Vault/Consul plumbing + encapi token grant (#98)
ci/woodpecker/push/apply Pipeline was successful
The new **terragrunt-enc** repo manages all encapi ENC data (statuses, roles, node classifications) via Terraform/Terragrunt and needs its own Vault/Consul plumbing, mirroring terraform-git and terraform-incus. This supersedes the dual-write approach in terraform-incus PR #39; the equivalent terraform-incus grant (PR #97) is being closed, so the encapi-token grant is created fresh here for the new approle.

Changes:
- Add approle role `terraform_enc` and k8s auth role `woodpecker_terraform_enc` (bound to the `terraform-enc` ServiceAccount in the `woodpecker` namespace) for CI auth.
- Add consul secret backend role `terraform-enc` plus its ACL rules granting `write` on `infra/terraform/enc/` (its terragrunt state prefix), and a policy letting both auth roles read `consul_root/au/syd1/creds/terraform-enc`.
- Grant both auth roles read on `kv/data/kubernetes/namespace/encapi/default/environment` (the ENCAPI_WRITE_TOKEN) so `make apply` can write to encapi via the encapi provider.

Reviewed-on: #98
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-24 23:18:56 +10:00
unkinben 8bb071ae46 Add auth and state access for terraform-rancher (#86)
ci/woodpecker/push/apply Pipeline was successful
## Why

The new `terraform-rancher` repo (manages Rancher's Authentik OIDC auth via the rancher2 provider) needs Vault auth + Consul state, mirroring the terraform-authentik runner (#78/#81/#82).

## Change

- `AppRole/terraform_rancher` + k8s auth role `woodpecker_terraform_rancher` (SA terraform-rancher in the woodpecker ns).
- Consul secret-backend role + ACL policy (`resources/secret_backend/consul_root/au/syd1/terraform-rancher.hcl`) granting write to the `infra/terraform/rancher/` state prefix.
- Vault policies: read the Rancher admin API token (`kv/service/terraform/rancher`) and the keycloakoidc client secret (`kv/kubernetes/namespace/cattle-system/default/oauth-credentials`), plus the consul_root state creds.

Scoped the OAuth read to the `cattle-system` path specifically (rather than the `+` wildcard the authentik policy uses) since the Rancher runner only needs its own app's secret.

## Validation

pre-commit (terragrunt-hcl-fmt + yamllint) passed. CI plan will confirm.

Reviewed-on: #86
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-15 21:37:45 +10:00
unkinben 36d7afbb65 feat: add vault/consul config for media terraform repos (#79)
ci/woodpecker/push/apply Pipeline was successful
Add Kubernetes auth roles, AppRole configs, Consul secret backend roles, Consul ACL policies, and Vault kv read policies for terraform-sonarr, terraform-radarr, and terraform-prowlarr.

Reviewed-on: #79
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-28 22:03:25 +10:00
unkinben c33dcdc447 Add auth and state access for terraform-authentik (#78)
ci/woodpecker/push/apply Pipeline was successful
## Summary
- K8s auth role for Woodpecker CI (`terraform-authentik` SA in `woodpecker` namespace)
- AppRole for local terraform runs
- Consul secret backend role (`terraform-authentik`, TTL 120/300)
- Consul ACL policy for `infra/terraform/authentik/` key prefix
- Vault policy granting both auth methods access to Consul creds

Reviewed-on: #78
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-28 01:17:51 +10:00
benvin be9bd96cf3 feat: enable consul state store for artifactapi (#77)
ci/woodpecker/push/apply Pipeline was successful
enable the terraform-artifactapi system to manage its state in consul
using dynamic credentials from kubernetes ci jobs in woodpecker

---------

Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #77
2026-06-17 21:42:25 +10:00
unkinben 1288057b81 feat: add vault and consul roles for terraform-git (#73)
ci/woodpecker/push/apply Pipeline was successful
## Summary
- Add K8s auth role woodpecker_terraform_git for CI pipeline authentication
- Add consul secret backend role terraform-git for consul state storage tokens
- Add consul ACL policy granting write access to infra/terraform/git/ key prefix
- Add vault policy for reading consul creds at consul_root/au/syd1/creds/terraform-git

## Test plan
- [ ] Verify terragrunt plan succeeds
- [ ] Verify consul ACL policy is created correctly
- [ ] Verify K8s auth role can authenticate from woodpecker namespace

Reviewed-on: #73
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-07 20:36:35 +10:00
unkinben 7c60a5fd53 chore: enable access to gateway.networking.k8s.io
ci/woodpecker/pr/pre-commit Pipeline was successful
2026-05-21 22:39:57 +10:00
unkinben dca26029c0 feat: add terraform-ldap service
- add consul role/policy/acls to allow terraform-ldap state management
- add approle to generate tokens for consul
2026-02-15 13:38:31 +11:00
unkinben 5536869a38 feat: implement consul ACL management with provider aliases
This commit message captures the major architectural change of implementing Consul ACL management
with proper provider aliasing, along with the supporting configuration files and policy definitions
for various terraform services.

- add consul_acl_management module to manage consul acl policies and roles
- add consul backend roles and policies for terraform services (incus, k8s, nomad, repoflow, vault)
- add consul provider configuration to root.hcl
- add policies to generate credentials for each role
- simplify consul_secret_backend_role module to reference acl-managed roles
- switch to opentofu for provider foreach support
- update terragrunt configuration to support consul backend aliases
- update pre-commit hooks to use opentofu instead of terraform
- configure tflint exceptions for consul acl management module
2026-02-14 18:13:50 +11:00
unkinben 75e9db1aa6 chore: add puppet k8s role
- add role and policies
2026-02-01 14:54:23 +11:00
unkinben 8070b6f66b feat: major restructuring in migration to terragrunt
- migrate from individual terraform files to config-driven terragrunt module structure
- add vault_cluster module with config discovery system
- replace individual .tf files with centralized config.hcl
- restructure auth and secret backends as configurable modules
- move auth roles and secret backends to yaml-based configuration
- convert policies from .hcl to .yaml format, add rules/auth definition
- add pre-commit hooks for yaml formatting and file cleanup
- add terragrunt cache to gitignore
- update makefile with terragrunt commands and format target
2026-01-26 23:02:44 +11:00
unkinben c88b19a216 feat: label kubernetes ephemeral serviceaccounts
- ensure all service accounts are labelled with role/cluster
- add additional api endpoints to cluster roles
2025-12-07 12:41:37 +11:00
unkinben 6624f7aed1 feat: add kubernetes secrets engine with RBAC roles for au-syd1 cluster
- Add Kubernetes secrets engine at kubernetes/au/syd1 path
  - Create four RBAC roles with external YAML configuration:
    * media-apps-operator: namespaced role for media-apps with selective permissions
    * cluster-operator: cluster-wide read-only access to specific API groups
    * cluster-admin: cluster-wide full access to specific API groups
    * cluster-root: cluster-wide superuser access to all resources
  - Add Vault policies for credential generation for each role
  - Add admin policies for kubernetes auth backend configuration and role management
  - Refactor kubernetes auth backend to use shared locals for CA certificate
  - Update terraform-vault approle with required kubernetes policies
2025-11-27 23:22:13 +11:00