Deploy NetBox (CNPG Postgres, Valkey, Authentik SSO) (#299)

## Why NetBox becomes the source of truth for host/interface/IPAM data as part of retiring Cobbler. The ENC role already moved to encapi; a kickstart-generation microservice that consumes NetBox comes later and is intentionally **out of scope** here. ## Change Adds the `netbox` app (namespace `netbox`, platform project) using the netbox-community helm chart via the OCI helm-through-kustomize pattern, plus estate-native data stores: - **NetBox** — chart `8.3.40` (appVersion `v4.6.5`), `oci://ghcr.io/netbox-community/netbox-chart`. 2 web replicas, 1 worker; bundled Postgres/Valkey subcharts disabled. Media on RWX CephFS so replicas share uploads. Chart `helm.sh/hook: test` Pod dropped via a kustomize delete patch (we deploy through ArgoCD, not `helm test`). - **CNPG Postgres** — `netbox-postgres`, 2 instances, PG 18, `cephrbd-fast-delete`. Standard per-cluster S3 backup: `barmanObjectStore -> s3://cnpg-netbox`, cephrgw `ObjectStoreUser` + `Bucket`, nightly `ScheduledBackup` at **03:40** (`0 40 3 * * *` — next free slot after grafana's 03:20), 30d retention. A pgbouncer `Pooler` (session mode) fronts it; NetBox connects via `netbox-postgres-pooler-rw`. - **Valkey** — standalone Deployment (`valkey/valkey:8-alpine`), AOF-persistent PVC on `cephrbd-fast-delete`. One instance: DB 0 = RQ task queue, DB 1 = cache. No auth (in-cluster, namespace-isolated). Chosen over the bundled Bitnami subchart to keep image control in-estate and avoid Bitnami's legacy-image churn; mirrors the litellm standalone-cache pattern. - **Ingress** — `Gateway` + `HTTPRoute` at `netbox.k8s.syd1.au.unkin.net` (`traefik-internal`, `vault-issuer` cert into `netbox-tls`, external-dns to the internal VIP), HTTP->HTTPS 301. - **Secrets** — all via VSO `VaultStaticSecret` (`postgres-credentials`, `netbox-secret-key`, `netbox-superuser`, `oauth-credentials`); no plain Secrets committed. The shared `default` k8s-auth role already binds `*` namespaces with a namespace-templated KV policy, so **no terraform-vault change is needed**. - **Authentik OIDC SSO** — `remoteAuth` wires `REMOTE_AUTH_ENABLED` + the `OpenIdConnectAuth` backend via chart values; `SOCIAL_AUTH_OIDC_*` via `extraConfig` (the chart's config loader globs `/run/config/extra/*/*.yaml`). The client secret is injected as a YAML fragment mounted from the Vault-synced `oauth-credentials` secret. New users auto-provision on first login. - Registers `netbox` in the platform ApplicationSet and AppProject (destination namespace + chart sourceRepo). ## Image table (source -> artifactapi -> DHI decision) All images flow through the estate's containerd registry mirrors; the allowlist patterns gate them. DHI (Docker Hardened Images) require authenticated pulls from a Docker Hub `dhi/` org and are **not** reachable through the anonymous mirror, so upstream official is used throughout. | Image | Upstream | Mirror / allowlist | Allowlisted? | DHI decision | |---|---|---|---|---| | NetBox app/worker/housekeeping | `ghcr.io/netbox-community/netbox:v4.6.5` | ghcr remote, `^netbox-community/` | **added in terraform-artifactapi #17** | No DHI published; upstream official | | CNPG Postgres | `ghcr.io/cloudnative-pg/postgresql:18.1-system-trixie` | ghcr, `^cloudnative-pg/` | already | estate-standard CNPG image | | Valkey | `docker.io/valkey/valkey:8-alpine` | dockerhub, `^valkey/valkey` | already | DHI not anon-pullable; upstream official | | NetBox init (perms) | `docker.io/busybox:1.38.0` | dockerhub, `^library/busybox` | already | upstream official | | Worker wait-for-backend | `docker.io/rancher/kubectl:v1.36.2` | dockerhub, `^rancher/` | already | upstream official | ## Cross-repo PRs (merge order) 1. **terraform-artifactapi #17** (`^netbox-community/` ghcr allowlist) — **merge before** this PR so the NetBox image pulls on first sync. 2. **terraform-authentik #11** (OIDC provider/application) — independent; SSO works once applied + the secret is seeded. 3. This PR. > Note: the CNPG S3-backup stanza pattern here is identical to PR #298 (already in main); it merges cleanly regardless of ordering. ## One-time Vault seeds (before/at first sync) ``` # App DB user (CNPG bootstrap + NetBox both consume this) vault kv put kv/kubernetes/namespace/netbox/default/postgres-credentials \ username=netbox password="$(openssl rand -base64 30)" # Django SECRET_KEY vault kv put kv/kubernetes/namespace/netbox/default/netbox-secret-key \ secret_key="$(python3 -c 'import secrets;print(secrets.token_urlsafe(60))')" # Bootstrap superuser vault kv put kv/kubernetes/namespace/netbox/default/netbox-superuser \ username=admin email=admin@unkin.net \ password="$(openssl rand -base64 24)" api_token="$(openssl rand -hex 20)" # OIDC client secret — ONE value stored two ways (raw for Authentik, YAML for NetBox) CS="$(openssl rand -base64 30)" vault kv put kv/kubernetes/namespace/netbox/default/oauth-credentials \ client_secret="$CS" oidc.yaml="SOCIAL_AUTH_OIDC_SECRET: \"$CS\"" ``` Seed `oauth-credentials` **before** applying terraform-authentik #11 (that apply reads `client_secret`). ## Validation - `kustomize build --enable-helm apps/overlays/au-syd1/netbox` — clean. - `kubeconform` (CI args, k8s 1.33.7): **27/27 valid, 0 invalid**. - `pre-commit` (yamllint + no-plain-secrets) on all changed files — pass. ## Out of scope The PXE/kickstart microservice that will consume NetBox — not scaffolded here. https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv Reviewed-on: #299 Co-authored-by: Ben Vincent <ben@unkin.net> Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-28 18:12:25 +10:00
parent 3c2bdf307a
commit 2ee4c9518c
16 changed files with 642 additions and 0 deletions
@@ -0,0 +1,28 @@
+---
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+resources:
+  - ../../../base/netbox
+
+helmCharts:
+  - name: netbox
+    repo: oci://ghcr.io/netbox-community/netbox-chart
+    version: "8.3.40"
+    releaseName: netbox
+    namespace: netbox
+    valuesFile: values.yaml
+
+# The chart always renders a `helm.sh/hook: test` connection Pod. We deploy via
+# kustomize+ArgoCD (not `helm test`), so drop it rather than leave an orphan Pod.
+patches:
+  - target:
+      version: v1
+      kind: Pod
+      name: netbox-test-connection
+    patch: |-
+      $patch: delete
+      apiVersion: v1
+      kind: Pod
+      metadata:
+        name: netbox-test-connection
@@ -0,0 +1,101 @@
+---
+# NetBox (netbox-community chart 8.3.40, appVersion v4.6.5).
+# Postgres = external CNPG cluster, Valkey = standalone Deployment, both defined
+# in apps/base/netbox. Secrets come from Vault via VSO (see base/vaultstaticsecret.yaml).
+
+image:
+  # ghcr.io is mirrored through artifactapi at the containerd level; the pull is
+  # gated by the ^netbox-community/ allowlist added in terraform-artifactapi.
+  # tag defaults to the chart appVersion (v4.6.5).
+  registry: ghcr.io
+  repository: netbox-community/netbox
+
+# Two web replicas for zero-downtime rollouts (media is RWX, see below).
+replicaCount: 2
+
+# Django SECRET_KEY — existingSecret must contain key: secret_key.
+existingSecret: netbox-secret-key
+
+# Bootstrap superuser — existingSecret keys: username, password, email, api_token.
+superuser:
+  existingSecret: netbox-superuser
+
+# Disable the bundled Bitnami subcharts; we bring our own Postgres and Valkey.
+postgresql:
+  enabled: false
+valkey:
+  enabled: false
+
+# External Postgres = the CNPG cluster's pgbouncer pooler (session mode).
+externalDatabase:
+  host: netbox-postgres-pooler-rw
+  port: 5432
+  database: netbox
+  username: netbox
+  existingSecretName: postgres-credentials
+  existingSecretKey: password
+  disableServerSideCursors: false
+
+# External Valkey — one instance, DB 0 = RQ task queue, DB 1 = cache. No auth
+# (in-cluster, namespace-isolated).
+tasksDatabase:
+  host: netbox-valkey
+  port: 6379
+  database: 0
+cachingDatabase:
+  host: netbox-valkey
+  port: 6379
+  database: 1
+
+# Uploaded media/attachments, shared across web replicas (RWX CephFS).
+persistence:
+  enabled: true
+  storageClass: cephfs-raid6-delete
+  accessMode: ReadWriteMany
+  size: 5Gi
+
+allowedHosts:
+  - netbox.k8s.syd1.au.unkin.net
+
+serviceAccount:
+  create: true
+  automountServiceAccountToken: false
+
+resources:
+  requests:
+    cpu: 200m
+    memory: 512Mi
+  limits:
+    cpu: "1"
+    memory: 2Gi
+
+worker:
+  replicaCount: 1
+  resources:
+    requests:
+      cpu: 100m
+      memory: 256Mi
+    limits:
+      cpu: "1"
+      memory: 1Gi
+
+# --- Authentik OIDC SSO via python-social-auth ---
+# remoteAuth wires REMOTE_AUTH_ENABLED + REMOTE_AUTH_BACKEND; the SOCIAL_AUTH_*
+# settings are supplied via extraConfig (YAML fragments the chart loads into the
+# NetBox config). New OIDC users are auto-provisioned on first login.
+remoteAuth:
+  enabled: true
+  backends:
+    - social_core.backends.open_id_connect.OpenIdConnectAuth
+  autoCreateUser: true
+extraConfig:
+  # index 0 -> /run/config/extra/0/extra-0.yaml (non-secret OIDC config)
+  - values:
+      SOCIAL_AUTH_OIDC_OIDC_ENDPOINT: https://identity.k8s.syd1.au.unkin.net/application/o/netbox/
+      SOCIAL_AUTH_OIDC_KEY: netbox
+  # index 1 -> /run/config/extra/1/oidc.yaml (client secret, from Vault via VSO)
+  - secret:
+      secretName: oauth-credentials
+      items:
+        - key: oidc.yaml
+          path: oidc.yaml