unkin/argocd-apps
3
0
Fork 0
Code Issues 7 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
406 Commits 22 Branches 0 Tags
8eb57f9c19bd4fd277d8dc2359024f49e100ab05
Commit Graph

5 Commits

Author SHA1 Message Date
unkin-agent 8eb57f9c19 authentik: deploy LDAP outpost for in-cluster app-password binds (#422) 
## Why

Jellyfin authenticates users against Authentik over LDAP (app-password binds). The `authentik-ldap` Service, Gateway, and TLSRoute already exist in the authentik base, but nothing backed the Service — no LDAP outpost was ever deployed. This adds the missing Deployment so in-cluster clients (Jellyfin) can bind.

Deployed as a separate Deployment inside the existing `authentik` namespace rather than a new app/namespace: the authentik overlay is already wired into the `platform` ApplicationSet and the LDAP Service/Gateway/TLSRoute already live here, so this needs no new AppProject/ApplicationSet wiring and keeps the outpost next to the core it serves.

## How

- `ldap-outpost-deployment.yaml`: 2-replica (stateless) Deployment, image `ghcr.io/goauthentik/ldap:2026.5.3` (canonical upstream name; matches the deployed authentik chart version; containerd mirrors route via artifactapi). Pod labels match the existing `authentik-ldap` Service selector.
  - `AUTHENTIK_HOST=https://identity.k8s.syd1.au.unkin.net`, `AUTHENTIK_INSECURE=false`.
  - Trusts the internal CA via the established combine-certs initContainer pattern (concatenates the base roots with the reflected `vault-ca-cert`) + `SSL_CERT_FILE`.
  - `AUTHENTIK_TOKEN` sourced from secret `authentik-ldap-outpost-token`.
  - Resources set; TCP probes on the LDAP port; reloader annotation for token + CA rotation.
- `ldap-outpost-vaultstaticsecret.yaml`: VSS (reuses the namespace `default` VaultAuth) materialising the token from KV `kubernetes/namespace/authentik/default/outpost-token` (key `token`).
- `ldap-service.yaml`: adds the plaintext `ldap` port 3389 alongside the existing `ldaps` 6636 (ClusterIP only, no public exposure — external reach is via the pre-existing internal Gateway/TLSRoute).
- `ldap-outpost-vmpodscrape.yaml`: VMPodScrape for the outpost metrics endpoint (:9300).
- No sync-wave annotations.

## Dependency — token seed (blocking)

The outpost pods CrashLoop until the token exists. After merge, the Authentik LDAP outpost token must be seeded into Vault KV at `kv/kubernetes/namespace/authentik/default/outpost-token` with key `token`. This path is provided by the corresponding terraform-authentik PR; the value must match the outpost token authentik issues.

## Validation

`kustomize build --enable-helm apps/overlays/au-syd1/authentik` + repo kubeconform: 38 resources, Valid: 38, Invalid: 0, Errors: 0.

Reviewed-on: #422
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
 2026-08-26 23:19:24 +10:00
unkin-agent 13a04a6ebc Export and scrape metrics from all redis/valkey instances (#398) 
## Why

No redis/valkey instance managed in this repo exports metrics today — the six caches (artifactapi, authentik, litellm, gitea, netbox, jellyfin) are blind spots in VictoriaMetrics.

## What

- Enables the valkey-operator exporter sidecar on the jellyfin ValkeyCluster (`exporter.enabled: true`); overrides the operator's default bare-dockerhub image (`oliver006/redis_exporter:v1.80.0`) with the artifactapi-proxied pin. The operator manages a dedicated `_exporter` ACL user for it.
- Adds a `metrics-exporter` sidecar (`artifactapi.k8s.syd1.au.unkin.net/dockerhub/oliver006/redis_exporter:v1.89.0`, :9121, port name `metrics`, req 32Mi/25m lim 64Mi/100m) to the five raw Deployments: artifactapi/authentik/litellm redis and gitea/netbox valkey. gitea/netbox sidecars get the same restrictive securityContext as their valkey container.
- Adds the `monitoring.unkin.net/redis-exporter: "true"` opt-in pod label to all five pod templates.
- Adds `apps/base/observability/vmpodscrape-redis.yaml` with two VMPodScrapes (any namespace, port `metrics`): `redis-exporters` selecting the opt-in label, and `valkey-operator-clusters` selecting `app.kubernetes.io/managed-by: valkey-operator` + `app.kubernetes.io/component: valkey-node` — the ValkeyCluster CR has no pod-label passthrough (verified against the v0.5.0 CRD/source), so operator pods cannot carry the opt-in label.

## Notes

- No instance runs with `requirepass`/auth, so no REDIS_PASSWORD wiring is needed on the raw Deployments; the jellyfin exporter auth is operator-managed.
- Validated: `kustomize build` on every touched base + au-syd1 overlay (helm overlays with `--enable-helm`) and `kubeconform -strict -ignore-missing-schemas` (68 valid, 0 invalid).

Reviewed-on: #398
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
 2026-08-23 19:19:16 +10:00
unkinben 3c2bdf307a Add S3 backups to all CNPG Postgres clusters (#298) 
## Why

None of the 8 CNPG Postgres clusters in this repo had **any** backup configured. A
lost PVC, a fat-fingered migration, or a bad app deploy meant permanent, unrecoverable
data loss for authentik, litellm, artifactapi, woodpecker, puppet, encapi, paperclip
and grafana. This adds continuous WAL archiving + a nightly base backup to Ceph RGW for
every cluster, plus code-forward restore docs.

## What

- **`spec.backup.barmanObjectStore`** on each `cnpg_cluster.yaml` — turns on continuous
  WAL archiving to `s3://cnpg-<app>`, WAL compressed with zstd, base backups with bzip2,
  30-day retention. TLS to `s3.ceph.unkin.net` is trusted via the reflected
  `vault-ca-cert` (`endpointCA`).
- **`cnpg_backup.yaml`** per app — a cephrgw `ObjectStoreUser` + `Bucket` (operator
  provisions the bucket and mints the S3 key into `cnpg-<app>-backup-s3`; **nothing is
  hardcoded**) and a staggered nightly `ScheduledBackup`.
- **`schemas/ceph.unkin.net/*.json`** — the three cephrgw CRD schemas so kubeconform can
  validate the new CRs.
- **`docs/`** — new docs folder (README index + `cnpg-backups.md` + `cnpg-restore.md`).

## Design decisions (answers to the open questions)

**One bucket for all, or per-database?** → **Per-database (one bucket + owner user per
cluster).** The cephrgw CRDs are namespace-scoped (`BucketRef`/`OwnerRef` resolve only
*within the same namespace*), and CNPG reads its S3 credential Secret from its *own*
namespace. A single shared bucket would require either cross-namespace bucket refs
(unsupported) or hand-copying the S3 secret into all 8 namespaces (defeats "operator
mints the keys"). Per-namespace `s3://cnpg-<app>` with a dedicated owner user is the
simplest correct topology and needs zero manual seeding. Each user owns exactly one
bucket, so owner-level (full) access is already tightly scoped — no extra `BucketAccess`
grant needed.

**Backup mechanism.** The deployed CNPG operator is **v1.28** (helm chart
`cloudnative-pg-0.27.0`, appVersion 1.28.0). 1.26+ deprecates the in-tree
`barmanObjectStore` in favour of the Barman Cloud Plugin, but the plugin is **not
deployed**, and `barmanObjectStore` is still fully functional on 1.28. So this uses the
in-tree mechanism. Migrating to the plugin is a follow-up (noted in `docs/cnpg-backups.md`).

## Schedule / retention (defaults — Ben to adjust)

| App | Cluster | Bucket | Nightly base backup |
| --- | --- | --- | --- |
| authentik | postgres | cnpg-authentik | 01:00 |
| litellm | litellm-postgres | cnpg-litellm | 01:20 |
| artifactapi | postgres | cnpg-artifactapi | 01:40 |
| woodpecker | woodpecker-postgres | cnpg-woodpecker | 02:00 |
| puppet | puppet-postgres | cnpg-puppet | 02:20 |
| encapi | postgres | cnpg-encapi | 02:40 |
| paperclip | paperclip-postgres | cnpg-paperclip | 03:00 |
| grafana | postgres | cnpg-grafana | 03:20 |

Retention is **30d** across the board — flagged as a default to tune per cluster.
Schedules are staggered 20 min apart so 8 base backups don't hit RGW at once.

## Validation

- `kustomize build --enable-helm` + `kubeconform` (repo CI args, incl. the new ceph
  schemas) pass on all 8 affected overlays (paperclip validated at base — it has no
  overlay yet). ceph CRs resolve their schemas (`Skipped: 0`).
- `pre-commit run` passes on all changed files (yamllint, no-plain-secrets, etc.).
- Note: a full `ci/validate-apps.sh` run aborts locally on the unrelated
  `cattle-system` overlay (`chart requires kubeVersion < 1.35 vs host helm v1.36.0`) —
  pre-existing, reproduces on `origin/main`, unrelated to this change.

## Notes / caveats

- No overlap with the woodpecker chart-bump PR (#297, overlay files only) or the logging
  PR (#296) beyond the three **identical** generated `schemas/ceph.unkin.net/*.json`
  files, which merge cleanly whichever lands first.
- Credentials: no manual seeding — the cephrgw-operator mints the RGW user + keys. The
  only prerequisite is the operator being healthy (it is, in `cephrgw-system`).

## Follow-ups

- Barman Cloud Plugin migration (deploy plugin, move clusters to `ObjectStore` CRs).
- Tune per-cluster retention / schedule if the defaults don't fit.

https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
Reviewed-on: #298
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
 2026-07-27 23:57:08 +10:00
unkinben 1d87c42fc1 Add advise-mode VPA for every workload (#283) 
## Why

Turns on right-sizing telemetry for the whole estate. Adds a `VerticalPodAutoscaler` with `updateMode: "Off"` (recommendation-only, advise mode) for every Deployment and StatefulSet in `apps/base`. Off mode never evicts or mutates pods, so this is purely observational: the VPA recommender (added in the vpa-system PR) publishes suggested requests/limits in each VPA's status, and nothing acts on them until someone deliberately flips a mode.

## Changes

- Add one `vpa.yaml` per app under `apps/base/<app>/` containing a `<workload>-vpa` VerticalPodAutoscaler for each workload, and register it in that app's `kustomization.yaml`.
- Coverage: 17 workloads across 11 apps.
  - age-api (age-api), artifactapi (api, redis, ui), authentik (redis), bind-system (bind-operator), cephrgw-system (cephrgw-operator), encapi (encapi), kanidm (kanidm StatefulSet), litellm (litellm, redis), paperclip (paperclip), pdbmux (pdbmux), puppet (puppetboard, puppetdb, puppetserver-compiler, puppetserver-master).

## Skipped (intentionally)

- **CNPG `Cluster` objects** (artifactapi, authentik, encapi, grafana, litellm, paperclip, puppet, woodpecker) — Postgres is managed by CloudNativePG, not a VPA target.
- **CronJobs** (puppet g10k/generate-types, reposync x4) — not VPA-able.

## HPA / VPA caveat

api, ui (artifactapi), litellm, and all four puppet deployments also carry an HPA. With `updateMode: "Off"` there is no conflict today (VPA only recommends). VPA objects targeting these carry an inline comment: do **not** flip to `Auto`/`Initial` while the HPA still autoscales on CPU/memory, or the two controllers will fight over the same resource. Move the HPA to a custom/non-resource metric first.

## Verification

- `kubectl kustomize` over every touched overlay: 9/11 overlays PASS rendering all their VPAs; 2 failures are pre-existing and unrelated to this change — `authentik` fails on a local helm-tooling flag error identically on origin/main, and `paperclip` has no `apps/overlays/au-syd1/paperclip` directory yet. Both apps' base kustomizations build clean and render their VPA.

## Merge gate

- **Requires the vpa-system CRDs PR (argocd-apps #281) to merge first.** These manifests use `autoscaling.k8s.io/v1 VerticalPodAutoscaler`; without the CRDs installed, ArgoCD sync fails on an unknown kind.

Reviewed-on: #283
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
 2026-07-25 18:30:23 +10:00
unkinben 7f1444fb38 Add Authentik identity provider deployment (#211) 
## Summary
- Deploy Authentik (identity.unkin.net) via Helm chart 2026.5.3
- CNPG PostgreSQL cluster (3 instances) with separate rw/ro poolers (2 instances each)
- Redis with 5Gi persistent storage
- Gateway API for HTTPS (identity.unkin.net) and LDAPS (ldap.k8s.syd1.au.unkin.net, ldap.main.unkin.net)
- TLSRoute for LDAPS passthrough, HTTPRoute for external-dns record creation
- Vault secrets for postgres credentials, authentik secret key, and S3 storage credentials
- S3 storage via RadosGW (bucket: authentik)
- 3 server replicas, 2 worker replicas
- Woodpecker ServiceAccount for terraform-authentik CI
- Platform applicationset and project updated

## Dependencies
- terraform-git #15 (merged) — repo definition
- terraform-vault #78 (merged) — auth roles and Consul ACL

## Vault secrets needed before deploy
Write to `kv/kubernetes/namespace/authentik/default/`:
- `postgres-credentials`: username + password
- `authentik-credentials`: AUTHENTIK_SECRET_KEY
- `s3-credentials`: S3 access key + secret key

Reviewed-on: #211
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
 2026-06-28 17:42:49 +10:00