unkin/argocd-apps
3
0
Fork 0
Code Issues 7 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

Deploy NetBox (CNPG Postgres, Valkey, Authentik SSO) #299

Merged
benvin merged 1 commits from benvin/netbox into main 2026-07-28 18:12:26 +10:00
Conversation 1 Commits 1 Files Changed 16
+642
unkinben commented 2026-07-28 16:46:13 +10:00
Owner
Copy Link
Copy Source

Why

NetBox becomes the source of truth for host/interface/IPAM data as part of retiring Cobbler. The ENC role already moved to encapi; a kickstart-generation microservice that consumes NetBox comes later and is intentionally out of scope here.

Change

Adds the netbox app (namespace netbox, platform project) using the netbox-community helm chart via the OCI helm-through-kustomize pattern, plus estate-native data stores:

  • NetBox — chart 8.3.40 (appVersion v4.6.5), oci://ghcr.io/netbox-community/netbox-chart. 2 web replicas, 1 worker; bundled Postgres/Valkey subcharts disabled. Media on RWX CephFS so replicas share uploads. Chart helm.sh/hook: test Pod dropped via a kustomize delete patch (we deploy through ArgoCD, not helm test).
  • CNPG Postgres — netbox-postgres, 2 instances, PG 18, cephrbd-fast-delete. Standard per-cluster S3 backup: barmanObjectStore -> s3://cnpg-netbox, cephrgw ObjectStoreUser + Bucket, nightly ScheduledBackup at 03:40 (0 40 3 * * * — next free slot after grafana's 03:20), 30d retention. A pgbouncer Pooler (session mode) fronts it; NetBox connects via netbox-postgres-pooler-rw.
  • Valkey — standalone Deployment (valkey/valkey:8-alpine), AOF-persistent PVC on cephrbd-fast-delete. One instance: DB 0 = RQ task queue, DB 1 = cache. No auth (in-cluster, namespace-isolated). Chosen over the bundled Bitnami subchart to keep image control in-estate and avoid Bitnami's legacy-image churn; mirrors the litellm standalone-cache pattern.
  • Ingress — Gateway + HTTPRoute at netbox.k8s.syd1.au.unkin.net (traefik-internal, vault-issuer cert into netbox-tls, external-dns to the internal VIP), HTTP->HTTPS 301.
  • Secrets — all via VSO VaultStaticSecret (postgres-credentials, netbox-secret-key, netbox-superuser, oauth-credentials); no plain Secrets committed. The shared default k8s-auth role already binds * namespaces with a namespace-templated KV policy, so no terraform-vault change is needed.
  • Authentik OIDC SSO — remoteAuth wires REMOTE_AUTH_ENABLED + the OpenIdConnectAuth backend via chart values; SOCIAL_AUTH_OIDC_* via extraConfig (the chart's config loader globs /run/config/extra/*/*.yaml). The client secret is injected as a YAML fragment mounted from the Vault-synced oauth-credentials secret. New users auto-provision on first login.
  • Registers netbox in the platform ApplicationSet and AppProject (destination namespace + chart sourceRepo).

Image table (source -> artifactapi -> DHI decision)

All images flow through the estate's containerd registry mirrors; the allowlist patterns gate them. DHI (Docker Hardened Images) require authenticated pulls from a Docker Hub dhi/ org and are not reachable through the anonymous mirror, so upstream official is used throughout.

Image Upstream Mirror / allowlist Allowlisted? DHI decision
NetBox app/worker/housekeeping ghcr.io/netbox-community/netbox:v4.6.5 ghcr remote, ^netbox-community/ added in terraform-artifactapi #17 No DHI published; upstream official
CNPG Postgres ghcr.io/cloudnative-pg/postgresql:18.1-system-trixie ghcr, ^cloudnative-pg/ already estate-standard CNPG image
Valkey docker.io/valkey/valkey:8-alpine dockerhub, ^valkey/valkey already DHI not anon-pullable; upstream official
NetBox init (perms) docker.io/busybox:1.38.0 dockerhub, ^library/busybox already upstream official
Worker wait-for-backend docker.io/rancher/kubectl:v1.36.2 dockerhub, ^rancher/ already upstream official

Cross-repo PRs (merge order)

  1. terraform-artifactapi #17 (^netbox-community/ ghcr allowlist) — merge before this PR so the NetBox image pulls on first sync.
  2. terraform-authentik #11 (OIDC provider/application) — independent; SSO works once applied + the secret is seeded.
  3. This PR.

Note: the CNPG S3-backup stanza pattern here is identical to PR #298 (already in main); it merges cleanly regardless of ordering.

One-time Vault seeds (before/at first sync)

# App DB user (CNPG bootstrap + NetBox both consume this)
vault kv put kv/kubernetes/namespace/netbox/default/postgres-credentials \
  username=netbox password="$(openssl rand -base64 30)"

# Django SECRET_KEY
vault kv put kv/kubernetes/namespace/netbox/default/netbox-secret-key \
  secret_key="$(python3 -c 'import secrets;print(secrets.token_urlsafe(60))')"

# Bootstrap superuser
vault kv put kv/kubernetes/namespace/netbox/default/netbox-superuser \
  username=admin email=admin@unkin.net \
  password="$(openssl rand -base64 24)" api_token="$(openssl rand -hex 20)"

# OIDC client secret — ONE value stored two ways (raw for Authentik, YAML for NetBox)
CS="$(openssl rand -base64 30)"
vault kv put kv/kubernetes/namespace/netbox/default/oauth-credentials \
  client_secret="$CS" oidc.yaml="SOCIAL_AUTH_OIDC_SECRET: \"$CS\""

Seed oauth-credentials before applying terraform-authentik #11 (that apply reads client_secret).

Validation

  • kustomize build --enable-helm apps/overlays/au-syd1/netbox — clean.
  • kubeconform (CI args, k8s 1.33.7): 27/27 valid, 0 invalid.
  • pre-commit (yamllint + no-plain-secrets) on all changed files — pass.

Out of scope

The PXE/kickstart microservice that will consume NetBox — not scaffolded here.

https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv

## Why NetBox becomes the source of truth for host/interface/IPAM data as part of retiring Cobbler. The ENC role already moved to encapi; a kickstart-generation microservice that consumes NetBox comes later and is intentionally **out of scope** here. ## Change Adds the `netbox` app (namespace `netbox`, platform project) using the netbox-community helm chart via the OCI helm-through-kustomize pattern, plus estate-native data stores: - **NetBox** — chart `8.3.40` (appVersion `v4.6.5`), `oci://ghcr.io/netbox-community/netbox-chart`. 2 web replicas, 1 worker; bundled Postgres/Valkey subcharts disabled. Media on RWX CephFS so replicas share uploads. Chart `helm.sh/hook: test` Pod dropped via a kustomize delete patch (we deploy through ArgoCD, not `helm test`). - **CNPG Postgres** — `netbox-postgres`, 2 instances, PG 18, `cephrbd-fast-delete`. Standard per-cluster S3 backup: `barmanObjectStore -> s3://cnpg-netbox`, cephrgw `ObjectStoreUser` + `Bucket`, nightly `ScheduledBackup` at **03:40** (`0 40 3 * * *` — next free slot after grafana's 03:20), 30d retention. A pgbouncer `Pooler` (session mode) fronts it; NetBox connects via `netbox-postgres-pooler-rw`. - **Valkey** — standalone Deployment (`valkey/valkey:8-alpine`), AOF-persistent PVC on `cephrbd-fast-delete`. One instance: DB 0 = RQ task queue, DB 1 = cache. No auth (in-cluster, namespace-isolated). Chosen over the bundled Bitnami subchart to keep image control in-estate and avoid Bitnami's legacy-image churn; mirrors the litellm standalone-cache pattern. - **Ingress** — `Gateway` + `HTTPRoute` at `netbox.k8s.syd1.au.unkin.net` (`traefik-internal`, `vault-issuer` cert into `netbox-tls`, external-dns to the internal VIP), HTTP->HTTPS 301. - **Secrets** — all via VSO `VaultStaticSecret` (`postgres-credentials`, `netbox-secret-key`, `netbox-superuser`, `oauth-credentials`); no plain Secrets committed. The shared `default` k8s-auth role already binds `*` namespaces with a namespace-templated KV policy, so **no terraform-vault change is needed**. - **Authentik OIDC SSO** — `remoteAuth` wires `REMOTE_AUTH_ENABLED` + the `OpenIdConnectAuth` backend via chart values; `SOCIAL_AUTH_OIDC_*` via `extraConfig` (the chart's config loader globs `/run/config/extra/*/*.yaml`). The client secret is injected as a YAML fragment mounted from the Vault-synced `oauth-credentials` secret. New users auto-provision on first login. - Registers `netbox` in the platform ApplicationSet and AppProject (destination namespace + chart sourceRepo). ## Image table (source -> artifactapi -> DHI decision) All images flow through the estate's containerd registry mirrors; the allowlist patterns gate them. DHI (Docker Hardened Images) require authenticated pulls from a Docker Hub `dhi/` org and are **not** reachable through the anonymous mirror, so upstream official is used throughout. | Image | Upstream | Mirror / allowlist | Allowlisted? | DHI decision | |---|---|---|---|---| | NetBox app/worker/housekeeping | `ghcr.io/netbox-community/netbox:v4.6.5` | ghcr remote, `^netbox-community/` | **added in terraform-artifactapi #17** | No DHI published; upstream official | | CNPG Postgres | `ghcr.io/cloudnative-pg/postgresql:18.1-system-trixie` | ghcr, `^cloudnative-pg/` | already | estate-standard CNPG image | | Valkey | `docker.io/valkey/valkey:8-alpine` | dockerhub, `^valkey/valkey` | already | DHI not anon-pullable; upstream official | | NetBox init (perms) | `docker.io/busybox:1.38.0` | dockerhub, `^library/busybox` | already | upstream official | | Worker wait-for-backend | `docker.io/rancher/kubectl:v1.36.2` | dockerhub, `^rancher/` | already | upstream official | ## Cross-repo PRs (merge order) 1. **terraform-artifactapi #17** (`^netbox-community/` ghcr allowlist) — **merge before** this PR so the NetBox image pulls on first sync. 2. **terraform-authentik #11** (OIDC provider/application) — independent; SSO works once applied + the secret is seeded. 3. This PR. > Note: the CNPG S3-backup stanza pattern here is identical to PR #298 (already in main); it merges cleanly regardless of ordering. ## One-time Vault seeds (before/at first sync) ``` # App DB user (CNPG bootstrap + NetBox both consume this) vault kv put kv/kubernetes/namespace/netbox/default/postgres-credentials \ username=netbox password="$(openssl rand -base64 30)" # Django SECRET_KEY vault kv put kv/kubernetes/namespace/netbox/default/netbox-secret-key \ secret_key="$(python3 -c 'import secrets;print(secrets.token_urlsafe(60))')" # Bootstrap superuser vault kv put kv/kubernetes/namespace/netbox/default/netbox-superuser \ username=admin email=admin@unkin.net \ password="$(openssl rand -base64 24)" api_token="$(openssl rand -hex 20)" # OIDC client secret — ONE value stored two ways (raw for Authentik, YAML for NetBox) CS="$(openssl rand -base64 30)" vault kv put kv/kubernetes/namespace/netbox/default/oauth-credentials \ client_secret="$CS" oidc.yaml="SOCIAL_AUTH_OIDC_SECRET: \"$CS\"" ``` Seed `oauth-credentials` **before** applying terraform-authentik #11 (that apply reads `client_secret`). ## Validation - `kustomize build --enable-helm apps/overlays/au-syd1/netbox` — clean. - `kubeconform` (CI args, k8s 1.33.7): **27/27 valid, 0 invalid**. - `pre-commit` (yamllint + no-plain-secrets) on all changed files — pass. ## Out of scope The PXE/kickstart microservice that will consume NetBox — not scaffolded here. https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
unkinben added 1 commit 2026-07-28 16:46:14 +10:00
Deploy NetBox with CNPG Postgres, Valkey and Authentik SSO
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/pr/kubeconform Pipeline was successful
Details
a28aabae78 
NetBox becomes the source of truth for host/interface/IPAM data as part of
retiring Cobbler (the ENC already moved to encapi; a kickstart-generation
service consuming NetBox comes later and is out of scope here).

Adds the netbox app in namespace netbox:
- netbox-community helm chart 8.3.40 (appVersion v4.6.5) via the OCI
  helm-through-kustomize pattern; bundled Postgres and Valkey subcharts
  disabled in favour of estate-native components.
- CNPG Postgres cluster (2 instances) with the standard per-cluster S3 backup
  stanza (barmanObjectStore -> s3://cnpg-netbox, cephrgw ObjectStoreUser +
  Bucket, nightly ScheduledBackup at 03:40, 30d retention) and a pgbouncer
  Pooler (session mode) that NetBox connects through.
- Standalone Valkey Deployment (valkey/valkey:8-alpine, AOF-persistent PVC)
  serving DB 0 = RQ task queue, DB 1 = cache; matches the litellm standalone
  cache pattern and keeps image control in-estate.
- Gateway + HTTPRoute at netbox.k8s.syd1.au.unkin.net (traefik-internal,
  vault-issuer cert, external-dns), HTTP->HTTPS redirect.
- All credentials via VSO VaultStaticSecret (postgres-credentials,
  netbox-secret-key, netbox-superuser, oauth-credentials); no plain Secrets.
- Authentik OIDC SSO (python-social-auth): REMOTE_AUTH wired via chart values,
  SOCIAL_AUTH_OIDC_* via extraConfig, client secret injected as a YAML fragment
  from Vault; auto-provisions users on first login.

Registers netbox in the platform ApplicationSet and AppProject (destination
namespace + chart sourceRepo).

Claude-Session: https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
unkinben commented 2026-07-28 17:32:27 +10:00
Author
Owner
Copy Link
Copy Source

Note: the four Vault seeds listed as manual prerequisites (postgres-credentials, netbox-secret-key, netbox-superuser, oauth-credentials under kv/kubernetes/namespace/netbox/default/) are already written and verified (all present at v1). No manual Vault steps remain for this PR — merge order is just terraform-artifactapi #17 first for the image allowlist.

Note: the four Vault seeds listed as manual prerequisites (`postgres-credentials`, `netbox-secret-key`, `netbox-superuser`, `oauth-credentials` under `kv/kubernetes/namespace/netbox/default/`) are **already written and verified** (all present at v1). No manual Vault steps remain for this PR — merge order is just terraform-artifactapi #17 first for the image allowlist.
benvin merged commit 2ee4c9518c into main 2026-07-28 18:12:26 +10:00
benvin deleted branch benvin/netbox 2026-07-28 18:12:26 +10:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
benvin (BenVincent) unkin-agent (Unkin Agent) unkinben (Ben Vincent)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: unkin/argocd-apps#299
Delete Branch "benvin/netbox"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?