## Why
The re-keyed internal `unkin.net` intermediate broke CA consumers (CNPG->RGW backups, subPath/startup-cached CA mounts) and needed manual pod restarts, because Reloader was deployed with `ignoreSecrets: true` and could not restart on the `vault-ca-cert` Secret. Enabling secret watching naively is unsafe: many workloads carry the generic `reloader.stakater.com/auto`, and the estate rotates numerous Secrets via Vault/VSO — those would restart on every rotation. This enables secret watching but scopes existing `auto` to ConfigMaps, making secret-reload opt-in per Secret.
## Changes
- Set `reloader.ignoreSecrets: false` (au-syd1 reloader-system values) so Secrets are watched.
- Convert every generic `reloader.stakater.com/auto: "true"` to the ConfigMap-only `configmap.reloader.stakater.com/auto: "true"` — 22 annotations across 19 files. Existing ConfigMap-reload behaviour is preserved; Vault/VSO Secret rotations no longer restart these workloads.
- Add explicit `secret.reloader.stakater.com/reload: "vault-ca-cert"` to the CA consumers that mount the CA and carry a Reloader annotation: `artifactapi/api`, `cephrgw-operator`, `puppetserver-master`, `puppetserver-compiler`, `litellm`, `logarchiver`.
- Add `secret.reloader.stakater.com/reload: "kanidm-tls"` so kanidm rolls when cert-manager renews its leaf.
- Add `docs/ca-rotation.md` runbook (indexed in `docs/README.md`).
## Safety review (secret-only / CA workloads)
`vault-ca-cert` is a plain reflected Secret that bootstraps Vault trust (not VSO-rotated; changes only on intermediate re-key). `kanidm-tls` is a cert-manager leaf. Everything else mounted (`environment`, `*-credentials`, `eyaml-keys`, `puppetboard-secrets`, `s3-credentials`, `nats-auth`, `clickhouse-credentials`, `woodpecker-*`) is VSO/CNPG Vault-rotated and deliberately excluded.
- `cephrgw-operator` — mounts only Secrets (`cephrgw-credentials` VSO + `vault-ca-cert`), no ConfigMap. Its old comment said "restart when the credentials Secret rotates"; `cephrgw-credentials` is VSO so that is now excluded, and reload is scoped to `vault-ca-cert` only. Comment updated.
- `nats` (logging) — old comment "Roll the StatefulSet when nats-auth changes"; `nats-auth` is VSO, so this is now ConfigMap-only (deliberately no roll on rotation). Comment updated. Same for the vector agent/aggregator/vm-ingest (VSO `nats-auth`/`clickhouse-credentials`).
- `artifactapi/ui` — mounts neither a ConfigMap nor a Secret; its `auto` was already a no-op. Left as ConfigMap-only.
- `puppetdb` / `puppetboard` — mount a ConfigMap plus VSO Secrets (postgres creds / puppetboard-secrets); ConfigMap-only is correct, no secret reload added.
CA consumers that mount `vault-ca-cert` but have **no** Reloader annotation (CRD-managed or startup-cached) are documented in `docs/ca-rotation.md` for manual restart rather than annotated here: `grafana`, `observability/vmagent`, `paperclip`, `argocd-repo-server`, plus CNPG clusters (`kubectl cnpg restart`).
## Notes / coordination
- Annotations left in their existing location (some sit on the pod template, e.g. `litellm`, `puppetdb`; Reloader reads controller-level metadata — placement unchanged from before, no regression).
- Touches `apps/overlays/au-syd1/logging/values-vector-*.yaml`, which overlap open PR #320 (Tier-2 Vector pipelines) — only the one-line reloader annotation is changed here.
## Validation
- `make kubeconform` — touched overlays (reloader-system, logging, woodpecker, authentik) valid; only the known-unrelated cattle-system rancher chart kubeVersion failure remains.
- `uvx pre-commit run --all-files` — all hooks pass.
Closes#326
---------
Co-authored-by: Ben Vincent <neotheo@gmail.com>
Reviewed-on: #339
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Move the estate's own service/operator image pulls off the Gitea container registry (git.unkin.net/unkin) to the artifactapi local docker registry (docker-internal), ahead of the git.unkin.net forge migration which disables Gitea's container registry. The images were copied digest-for-digest into docker-internal and pulls verified before this repoint.
- repoint age-api, bind-operator, bind-tsig-api, cephrgw-operator, encapi, logarchiver, pdbmux image pulls to artifactapi.k8s.syd1.au.unkin.net/docker-internal
Deliberately not repointed here: artifactapi's own api/ui images (circular — it can't pull itself from itself), and the almalinux9-* base/CI images (huge, and their per-repo Woodpecker push targets move in a batched follow-up). Forge raw-CRD URLs and git clone sources are unaffected (those stay on the forge).
Reviewed-on: #310
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Why
Turns on right-sizing telemetry for the whole estate. Adds a `VerticalPodAutoscaler` with `updateMode: "Off"` (recommendation-only, advise mode) for every Deployment and StatefulSet in `apps/base`. Off mode never evicts or mutates pods, so this is purely observational: the VPA recommender (added in the vpa-system PR) publishes suggested requests/limits in each VPA's status, and nothing acts on them until someone deliberately flips a mode.
## Changes
- Add one `vpa.yaml` per app under `apps/base/<app>/` containing a `<workload>-vpa` VerticalPodAutoscaler for each workload, and register it in that app's `kustomization.yaml`.
- Coverage: 17 workloads across 11 apps.
- age-api (age-api), artifactapi (api, redis, ui), authentik (redis), bind-system (bind-operator), cephrgw-system (cephrgw-operator), encapi (encapi), kanidm (kanidm StatefulSet), litellm (litellm, redis), paperclip (paperclip), pdbmux (pdbmux), puppet (puppetboard, puppetdb, puppetserver-compiler, puppetserver-master).
## Skipped (intentionally)
- **CNPG `Cluster` objects** (artifactapi, authentik, encapi, grafana, litellm, paperclip, puppet, woodpecker) — Postgres is managed by CloudNativePG, not a VPA target.
- **CronJobs** (puppet g10k/generate-types, reposync x4) — not VPA-able.
## HPA / VPA caveat
api, ui (artifactapi), litellm, and all four puppet deployments also carry an HPA. With `updateMode: "Off"` there is no conflict today (VPA only recommends). VPA objects targeting these carry an inline comment: do **not** flip to `Auto`/`Initial` while the HPA still autoscales on CPU/memory, or the two controllers will fight over the same resource. Move the HPA to a custom/non-resource metric first.
## Verification
- `kubectl kustomize` over every touched overlay: 9/11 overlays PASS rendering all their VPAs; 2 failures are pre-existing and unrelated to this change — `authentik` fails on a local helm-tooling flag error identically on origin/main, and `paperclip` has no `apps/overlays/au-syd1/paperclip` directory yet. Both apps' base kustomizations build clean and render their VPA.
## Merge gate
- **Requires the vpa-system CRDs PR (argocd-apps #281) to merge first.** These manifests use `autoscaling.k8s.io/v1 VerticalPodAutoscaler`; without the CRDs installed, ArgoCD sync fails on an unknown kind.
Reviewed-on: #283
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Summary
- Deploy age-api to the au-syd1 cluster
- Uses configMapGenerator for people config with jaidi, ben, and sudaporn
- Includes gateway, httproute, service, and deployment
- Image: git.unkin.net/unkin/age-api:v0.1.0
Reviewed-on: #210
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
unkinben