The internal intermediate CA re-key broke CA consumers because Reloader
ignored Secrets and could not restart on the vault-ca-cert Secret. Enabling
generic secret watching naively would restart every workload on each
Vault/VSO secret rotation, so this scopes existing auto annotations to
ConfigMaps and makes secret reload opt-in per Secret.
- set reloader ignoreSecrets: false so Secrets are watched
- convert every generic reloader.stakater.com/auto to the configmap-only
configmap.reloader.stakater.com/auto form (22 annotations, 19 files)
- add explicit secret.reloader.stakater.com/reload: "vault-ca-cert" to the CA
consumers (artifactapi api, cephrgw-operator, puppetserver master+compiler,
litellm, logarchiver) so CA rotation restarts them
- add secret.reloader.stakater.com/reload: "kanidm-tls" so the cert-manager
leaf renewal rolls kanidm
- add docs/ca-rotation.md runbook and index it
Closes#326
Claude-Session: https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT
Authentik is canonical at https://identity.unkin.net, served by the internal
unkin.net CA. Grafana, LiteLLM and NetBox failed OIDC discovery because their
images don't trust that CA (x509: unknown authority); NetBox also still pointed
at the secondary admin host.
- grafana: mount the reflected vault-ca-cert; set generic_oauth `tls_client_ca`.
- litellm: `combine-certs` init builds public+internal CA bundle; `SSL_CERT_FILE`
+ `REQUESTS_CA_BUNDLE` point at it.
- netbox: flip OIDC issuer to identity.unkin.net; same combine bundle for
python-social-auth (`requests`).
- docs: record the Rancher manual runtime step (issuer + CA in the auth config).
Validated: kustomize build + kubeconform + pre-commit.
https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
Reviewed-on: #314
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
App-side of the LiteLLM Authentik onboarding (terraform-authentik#8). Configures
LiteLLM's generic OIDC SSO against Authentik.
- VaultStaticSecret oauth-credentials: surfaces the OIDC client secret (same
secret Authentik sets on the provider) as a k8s Secret.
- Deployment: GENERIC_CLIENT_SECRET from that Secret.
- litellm-env: GENERIC_CLIENT_ID, authorization/token/userinfo endpoints, scope,
and PROXY_BASE_URL (required for SSO). reloader restarts on secret/config change.
finding litellm performance has dropped, crashed in multiple cases, and
then it had scaled to the maximum level using the majority of memory in
cluster.
- reduce the rate at which litellm autoscales
- increase the requests/limits to match usage
Reviewed-on: #144
Deploys LiteLLM proxy with CNPG PostgreSQL (3-instance HA), PgBouncer
pooler, and Redis cache. Introduces a dedicated aitooling AppProject and
ApplicationSet to keep AI tooling services separate from platform infra.
Reviewed-on: #94
Ben Vincent
unkinben