Files

T

unkinben 1d87c42fc1 Add advise-mode VPA for every workload (#283 )

## Why

Turns on right-sizing telemetry for the whole estate. Adds a `VerticalPodAutoscaler` with `updateMode: "Off"` (recommendation-only, advise mode) for every Deployment and StatefulSet in `apps/base`. Off mode never evicts or mutates pods, so this is purely observational: the VPA recommender (added in the vpa-system PR) publishes suggested requests/limits in each VPA's status, and nothing acts on them until someone deliberately flips a mode.

## Changes

- Add one `vpa.yaml` per app under `apps/base/<app>/` containing a `<workload>-vpa` VerticalPodAutoscaler for each workload, and register it in that app's `kustomization.yaml`.
- Coverage: 17 workloads across 11 apps.
  - age-api (age-api), artifactapi (api, redis, ui), authentik (redis), bind-system (bind-operator), cephrgw-system (cephrgw-operator), encapi (encapi), kanidm (kanidm StatefulSet), litellm (litellm, redis), paperclip (paperclip), pdbmux (pdbmux), puppet (puppetboard, puppetdb, puppetserver-compiler, puppetserver-master).

## Skipped (intentionally)

- **CNPG `Cluster` objects** (artifactapi, authentik, encapi, grafana, litellm, paperclip, puppet, woodpecker) — Postgres is managed by CloudNativePG, not a VPA target.
- **CronJobs** (puppet g10k/generate-types, reposync x4) — not VPA-able.

## HPA / VPA caveat

api, ui (artifactapi), litellm, and all four puppet deployments also carry an HPA. With `updateMode: "Off"` there is no conflict today (VPA only recommends). VPA objects targeting these carry an inline comment: do **not** flip to `Auto`/`Initial` while the HPA still autoscales on CPU/memory, or the two controllers will fight over the same resource. Move the HPA to a custom/non-resource metric first.

## Verification

- `kubectl kustomize` over every touched overlay: 9/11 overlays PASS rendering all their VPAs; 2 failures are pre-existing and unrelated to this change — `authentik` fails on a local helm-tooling flag error identically on origin/main, and `paperclip` has no `apps/overlays/au-syd1/paperclip` directory yet. Both apps' base kustomizations build clean and render their VPA.

## Merge gate

- **Requires the vpa-system CRDs PR (argocd-apps #281) to merge first.** These manifests use `autoscaling.k8s.io/v1 VerticalPodAutoscaler`; without the CRDs installed, ArgoCD sync fails on an unknown kind.

Reviewed-on: #283
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>

2026-07-25 18:30:23 +10:00

resources

feat(kanidm): scale down to single replica, remove replication (#185 )

2026-06-02 22:41:28 +10:00

certificate.yaml

benvin/kanidm (#159 )

2026-05-24 19:55:22 +10:00

gateway.yaml

benvin/kanidm (#159 )

2026-05-24 19:55:22 +10:00

httproute.yaml

fix(gateways): add explicit group/kind/weight to all HTTPRoute refs (#162 )

2026-05-24 20:32:37 +10:00

kustomization.yaml

Add advise-mode VPA for every workload (#283 )

2026-07-25 18:30:23 +10:00

namespace.yaml

benvin/kanidm (#159 )

2026-05-24 19:55:22 +10:00

poddisruptionbudget.yaml

benvin/kanidm (#159 )

2026-05-24 19:55:22 +10:00

rbac.yaml

benvin/kanidm (#159 )

2026-05-24 19:55:22 +10:00

README.md

feat(kanidm): vault-managed replication certs with auto-restart (#176 )

2026-05-30 23:00:46 +10:00

service.yaml

feat(kanidm): scale down to single replica, remove replication (#185 )

2026-06-02 22:41:28 +10:00

serviceaccount.yaml

benvin/kanidm (#159 )

2026-05-24 19:55:22 +10:00

statefulset.yaml

feat(kanidm): scale down to single replica, remove replication (#185 )

2026-06-02 22:41:28 +10:00

tlsroute.yaml

fix(kanidm): add explicit group/kind/weight to TLSRoute refs (#165 )

2026-05-25 22:43:52 +10:00

vaultauth.yaml

Update apps/base/kanidm/vaultauth.yaml (#177 )

2026-05-30 23:11:38 +10:00

vaultstaticsecret.yaml

feat: autorestart statefulset (#180 )

2026-05-30 23:40:07 +10:00

vpa.yaml

Add advise-mode VPA for every workload (#283 )

2026-07-25 18:30:23 +10:00

README.md

kanidm

Three-replica kanidm identity server with Vault-managed replication certificates.

Architecture

Per-pod server-N.toml in resources/ — each has its own replication origin hardcoded
config-init busybox init container copies the right config and injects peer certs from the vault-synced kanidm-repl-certs Secret at pod startup
reloader.stakater.com/auto: "true" triggers a rolling restart when the ConfigMap or Secret changes
Vault path: kv/kubernetes/namespace/kanidm/default/repl-certs
- Keys: kanidm-0, kanidm-1, kanidm-2 — each holds that pod's replication certificate

Initial setup

After the first pod starts, generate the admin credentials:

kubectl exec -n kanidm kanidm-0 -- /sbin/kanidmd recover-account -c /config/server.toml admin
kubectl exec -n kanidm kanidm-0 -- /sbin/kanidmd recover-account -c /config/server.toml idm_admin

Replication certificate rotation

When certs need to be renewed, update vault and reloader will roll the StatefulSet:

# Get new cert from a pod
kubectl exec -it -n kanidm kanidm-N -- /sbin/kanidmd renew-replication-certificate -c /config/server.toml

# Write updated cert to vault (reloader triggers restart automatically)
vault kv patch kv/kubernetes/namespace/kanidm/default/repl-certs "kanidm-N=<cert>"

Resolving domain UUID mismatch

If pods initialized independently (each with a different domain UUID), replication will fail with Consumer Domain UUID does not match. Fix by resetting kanidm-1 and kanidm-2 to sync from kanidm-0 (the authoritative node):

# Scale down to avoid split-brain during reset
kubectl scale statefulset -n kanidm kanidm --replicas=1

# Delete the stale PVCs for the replica pods
kubectl delete pvc -n kanidm data-kanidm-1 data-kanidm-2

# Scale back up — replicas start with empty DBs and automatic_refresh=true
# will trigger a full sync from kanidm-0 once TLS peer certs are verified
kubectl scale statefulset -n kanidm kanidm --replicas=3