Merge branch 'main' into fix-kanidm-replication-config

fix(kanidm): remove invalid automatic_refresh from replication config (#179 )
Reviewed-on: #179
2026-05-30 23:33:36 +10:00 · 2026-05-30 23:20:48 +10:00 · 2026-05-30 23:15:41 +10:00 · 2026-05-30 23:11:38 +10:00 · 2026-05-30 23:00:46 +10:00 · 2026-05-26 23:58:57 +10:00
40 changed files with 731 additions and 35 deletions
@@ -8,7 +8,9 @@ spec:
  hostnames:
    - artifactapi.k8s.syd1.au.unkin.net
  parentRefs:
-    - name: artifactapi
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: artifactapi
      sectionName: http
  rules:
    - filters:
@@ -30,12 +32,17 @@ spec:
  hostnames:
    - artifactapi.k8s.syd1.au.unkin.net
  parentRefs:
-    - name: artifactapi
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: artifactapi
      sectionName: https
  rules:
    - backendRefs:
-        - name: artifactapi-api
+        - group: ""
+          kind: Service
+          name: artifactapi-api
          port: 80
+          weight: 1
      matches:
        - path:
            type: PathPrefix
@@ -6,9 +6,10 @@ remotes:
    immutable_patterns:
      - "^cloudnative-pg/cloudnative-pg"
      - "^emberstack/helm-charts"
-      - "^kanidm/"
+      - "^open-webui/open-webui"
      - "^openvoxproject/"
      - "^stakater/reloader"
+      - "^stalwartlabs/stalwart"
      - "^voxpupuli/puppetboard"
      - "^woodpecker-ci/helm"
    cache:
@@ -34,8 +35,12 @@ remotes:
      - "^hashicorp/consul"
      - "^hashicorp/vault"
      - "^jfrog/"
+      - "^kanidm/"
      - "^rancher/"
+      - "^rspamd/rspamd"
+      - "^tozd/postfix"
      - "^traefik/"
+      - "^valkey/valkey"
      - "^ubi9/ubi-minimal"
      - "^victoriametrics/"
      - "^woodpeckerci/"
@@ -5,6 +5,7 @@ remotes:
    description: "GitHub releases and files"
    mutable_patterns:
      - ".*/archive/refs/heads/.*.tar.gz$"
+      - "stalwartlabs/webadmin/releases/latest/download/webadmin.zip$"
    immutable_patterns:
      - ".*/archive/refs/tags/.*.tar.gz$"
      - "ahmetb/kubectx/.*/kubectx_.*_linux_x86_64.tar.gz$"
@@ -35,6 +36,7 @@ remotes:
      - "neovim/neovim/.*/nvim-linux-x86_64.tar.gz$"
      - "nzbgetcom/nzbget/.*/nzbget-.*.x86_64.rpm$"
      - "onedr0p/exportarr/.*/exportarr_.*_linux_amd64.tar.gz$"
+      - "open-policy-agent/conftest/.*/conftest_.*_Linux_x86_64.tar.gz$"
      - "openbao/openbao-plugins/.*/openbao-plugin-secrets-consul_linux_amd64_.*.tar.gz$"
      - "openbao/openbao-plugins/.*/openbao-plugin-secrets-nomad_linux_amd64_.*.tar.gz$"
      - "prometheus-community/bind_exporter/.*/bind_exporter-.*.linux-amd64.tar.gz$"
@@ -8,7 +8,9 @@ spec:
  hostnames:
    - rancher.k8s.syd1.au.unkin.net
  parentRefs:
-    - name: rancher
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: rancher
      sectionName: http
  rules:
    - filters:
@@ -30,12 +32,17 @@ spec:
  hostnames:
    - rancher.k8s.syd1.au.unkin.net
  parentRefs:
-    - name: rancher
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: rancher
      sectionName: https
  rules:
    - backendRefs:
-        - name: rancher
+        - group: ""
+          kind: Service
+          name: rancher
          port: 80
+          weight: 1
      matches:
        - path:
            type: PathPrefix
@@ -11,7 +11,9 @@ spec:
  hostnames:
    - consul.k8s.syd1.au.unkin.net
  parentRefs:
-    - name: consul
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: consul
      sectionName: http
  rules:
    - filters:
@@ -36,12 +38,17 @@ spec:
  hostnames:
    - consul.k8s.syd1.au.unkin.net
  parentRefs:
-    - name: consul
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: consul
      sectionName: https
  rules:
    - backendRefs:
-        - name: consul-ui
+        - group: ""
+          kind: Service
+          name: consul-ui
          port: 80
+          weight: 1
      matches:
        - path:
            type: PathPrefix
@@ -59,12 +66,17 @@ spec:
  hostnames:
    - consul.service.consul
  parentRefs:
-    - name: consul
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: consul
      sectionName: consul-svc
  rules:
    - backendRefs:
-        - name: consul-ui
+        - group: ""
+          kind: Service
+          name: consul-ui
          port: 80
+          weight: 1
      matches:
        - path:
            type: PathPrefix
@@ -0,0 +1,51 @@
+# kanidm
+
+Three-replica kanidm identity server with Vault-managed replication certificates.
+
+## Architecture
+
+- Per-pod `server-N.toml` in `resources/` — each has its own replication origin hardcoded
+- `config-init` busybox init container copies the right config and injects peer certs from the
+  vault-synced `kanidm-repl-certs` Secret at pod startup
+- `reloader.stakater.com/auto: "true"` triggers a rolling restart when the ConfigMap or Secret changes
+- Vault path: `kv/kubernetes/namespace/kanidm/default/repl-certs`
+  - Keys: `kanidm-0`, `kanidm-1`, `kanidm-2` — each holds that pod's replication certificate
+
+## Initial setup
+
+After the first pod starts, generate the admin credentials:
+
+```bash
+kubectl exec -n kanidm kanidm-0 -- /sbin/kanidmd recover-account -c /config/server.toml admin
+kubectl exec -n kanidm kanidm-0 -- /sbin/kanidmd recover-account -c /config/server.toml idm_admin
+```
+
+## Replication certificate rotation
+
+When certs need to be renewed, update vault and reloader will roll the StatefulSet:
+
+```bash
+# Get new cert from a pod
+kubectl exec -it -n kanidm kanidm-N -- /sbin/kanidmd renew-replication-certificate -c /config/server.toml
+
+# Write updated cert to vault (reloader triggers restart automatically)
+vault kv patch kv/kubernetes/namespace/kanidm/default/repl-certs "kanidm-N=<cert>"
+```
+
+## Resolving domain UUID mismatch
+
+If pods initialized independently (each with a different domain UUID), replication will fail with
+`Consumer Domain UUID does not match`. Fix by resetting kanidm-1 and kanidm-2 to sync from
+kanidm-0 (the authoritative node):
+
+```bash
+# Scale down to avoid split-brain during reset
+kubectl scale statefulset -n kanidm kanidm --replicas=1
+
+# Delete the stale PVCs for the replica pods
+kubectl delete pvc -n kanidm data-kanidm-1 data-kanidm-2
+
+# Scale back up — replicas start with empty DBs and automatic_refresh=true
+# will trigger a full sync from kanidm-0 once TLS peer certs are verified
+kubectl scale statefulset -n kanidm kanidm --replicas=3
+```
@@ -0,0 +1,26 @@
+---
+apiVersion: cert-manager.io/v1
+kind: Certificate
+metadata:
+  name: kanidm-tls
+  namespace: kanidm
+  labels:
+    app.kubernetes.io/name: kanidm
+    app.kubernetes.io/instance: kanidm
+spec:
+  secretName: kanidm-tls
+  issuerRef:
+    kind: ClusterIssuer
+    name: vault-issuer
+  commonName: auth.unkin.net
+  dnsNames:
+    - auth.unkin.net
+    - au.auth.unkin.net
+    - kanidm.k8s.syd1.au.unkin.net
+    - kanidm.kanidm.svc.cluster.local
+    - kanidm-0.kanidm-headless.kanidm.svc.cluster.local
+    - kanidm-1.kanidm-headless.kanidm.svc.cluster.local
+    - kanidm-2.kanidm-headless.kanidm.svc.cluster.local
+  privateKey:
+    algorithm: RSA
+    size: 4096
@@ -0,0 +1,30 @@
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: Gateway
+metadata:
+  name: kanidm
+  namespace: kanidm
+  labels:
+    app.kubernetes.io/name: kanidm
+    app.kubernetes.io/instance: kanidm
+    traefik.io/instance: internal
+  annotations:
+    external-dns.alpha.kubernetes.io/hostname: kanidm.k8s.syd1.au.unkin.net
+    external-dns.alpha.kubernetes.io/target: 198.18.200.4
+spec:
+  gatewayClassName: traefik-internal
+  listeners:
+    - name: http
+      port: 80
+      protocol: HTTP
+      allowedRoutes:
+        namespaces:
+          from: Same
+    - name: https-passthrough
+      port: 443
+      protocol: TLS
+      tls:
+        mode: Passthrough
+      allowedRoutes:
+        namespaces:
+          from: Same
@@ -0,0 +1,29 @@
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: kanidm-http-redirect
+  namespace: kanidm
+  labels:
+    app.kubernetes.io/name: kanidm
+    app.kubernetes.io/instance: kanidm
+spec:
+  hostnames:
+    - kanidm.k8s.syd1.au.unkin.net
+    - auth.unkin.net
+    - au.auth.unkin.net
+  parentRefs:
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: kanidm
+      sectionName: http
+  rules:
+    - filters:
+        - type: RequestRedirect
+          requestRedirect:
+            scheme: https
+            statusCode: 301
+      matches:
+        - path:
+            type: PathPrefix
+            value: /
@@ -0,0 +1,29 @@
+---
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+resources:
+  - namespace.yaml
+  - serviceaccount.yaml
+  - vaultauth.yaml
+  - vaultstaticsecret.yaml
+  - certificate.yaml
+  - service.yaml
+  - statefulset.yaml
+  - poddisruptionbudget.yaml
+  - gateway.yaml
+  - httproute.yaml
+  - tlsroute.yaml
+
+configMapGenerator:
+  - name: kanidm-config
+    namespace: kanidm
+    options:
+      disableNameSuffixHash: true
+      labels:
+        app.kubernetes.io/name: kanidm
+        app.kubernetes.io/instance: kanidm
+    files:
+      - server-0.toml=resources/server-0.toml
+      - server-1.toml=resources/server-1.toml
+      - server-2.toml=resources/server-2.toml
@@ -0,0 +1,5 @@
+---
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: kanidm
@@ -0,0 +1,15 @@
+---
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+  name: kanidm
+  namespace: kanidm
+  labels:
+    app.kubernetes.io/name: kanidm
+    app.kubernetes.io/instance: kanidm
+spec:
+  maxUnavailable: 1
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: kanidm
+      app.kubernetes.io/instance: kanidm
@@ -0,0 +1,37 @@
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  name: kanidm-repl
+  namespace: kanidm
+  labels:
+    app.kubernetes.io/name: kanidm
+    app.kubernetes.io/instance: kanidm
+rules:
+  - apiGroups: [""]
+    resources: ["pods"]
+    verbs: ["get", "list"]
+  - apiGroups: [""]
+    resources: ["pods/exec"]
+    verbs: ["create"]
+  - apiGroups: [""]
+    resources: ["configmaps"]
+    resourceNames: ["kanidm-repl-certs"]
+    verbs: ["get", "patch"]
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: kanidm-repl
+  namespace: kanidm
+  labels:
+    app.kubernetes.io/name: kanidm
+    app.kubernetes.io/instance: kanidm
+subjects:
+  - kind: ServiceAccount
+    name: kanidm
+    namespace: kanidm
+roleRef:
+  kind: Role
+  name: kanidm-repl
+  apiGroup: rbac.authorization.k8s.io
@@ -0,0 +1,19 @@
+version = "2"
+
+domain = "auth.unkin.net"
+origin = "https://auth.unkin.net"
+bindaddress = "[::]:8443"
+db_path = "/data/kanidm.db"
+db_arc_size = 2048
+tls_chain = "/data/tls/tls.crt"
+tls_key = "/data/tls/tls.key"
+log_level = "info"
+
+[online_backup]
+path = "/data/backups/"
+schedule = "0 22 * * *"
+versions = 7
+
+[replication]
+origin = "repl://kanidm-0.kanidm-headless.kanidm.svc.cluster.local:8444"
+bindaddress = "[::]:8444"
@@ -0,0 +1,19 @@
+version = "2"
+
+domain = "auth.unkin.net"
+origin = "https://auth.unkin.net"
+bindaddress = "[::]:8443"
+db_path = "/data/kanidm.db"
+db_arc_size = 2048
+tls_chain = "/data/tls/tls.crt"
+tls_key = "/data/tls/tls.key"
+log_level = "info"
+
+[online_backup]
+path = "/data/backups/"
+schedule = "0 22 * * *"
+versions = 7
+
+[replication]
+origin = "repl://kanidm-1.kanidm-headless.kanidm.svc.cluster.local:8444"
+bindaddress = "[::]:8444"
@@ -0,0 +1,19 @@
+version = "2"
+
+domain = "auth.unkin.net"
+origin = "https://auth.unkin.net"
+bindaddress = "[::]:8443"
+db_path = "/data/kanidm.db"
+db_arc_size = 2048
+tls_chain = "/data/tls/tls.crt"
+tls_key = "/data/tls/tls.key"
+log_level = "info"
+
+[online_backup]
+path = "/data/backups/"
+schedule = "0 22 * * *"
+versions = 7
+
+[replication]
+origin = "repl://kanidm-2.kanidm-headless.kanidm.svc.cluster.local:8444"
+bindaddress = "[::]:8444"
@@ -0,0 +1,43 @@
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: kanidm
+  namespace: kanidm
+  labels:
+    app.kubernetes.io/name: kanidm
+    app.kubernetes.io/instance: kanidm
+spec:
+  type: ClusterIP
+  ports:
+    - name: https
+      port: 8443
+      targetPort: https
+      protocol: TCP
+  selector:
+    app.kubernetes.io/name: kanidm
+    app.kubernetes.io/instance: kanidm
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: kanidm-headless
+  namespace: kanidm
+  labels:
+    app.kubernetes.io/name: kanidm
+    app.kubernetes.io/instance: kanidm
+spec:
+  type: ClusterIP
+  clusterIP: None
+  ports:
+    - name: https
+      port: 8443
+      targetPort: https
+      protocol: TCP
+    - name: replication
+      port: 8444
+      targetPort: replication
+      protocol: TCP
+  selector:
+    app.kubernetes.io/name: kanidm
+    app.kubernetes.io/instance: kanidm
@@ -0,0 +1,9 @@
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: kanidm
+  namespace: kanidm
+  labels:
+    app.kubernetes.io/name: kanidm
+    app.kubernetes.io/instance: kanidm
@@ -0,0 +1,133 @@
+---
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+  name: kanidm
+  namespace: kanidm
+  annotations:
+    reloader.stakater.com/auto: "true"
+  labels:
+    app.kubernetes.io/name: kanidm
+    app.kubernetes.io/instance: kanidm
+spec:
+  serviceName: kanidm-headless
+  replicas: 3
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: kanidm
+      app.kubernetes.io/instance: kanidm
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: kanidm
+        app.kubernetes.io/instance: kanidm
+    spec:
+      serviceAccountName: kanidm
+      affinity:
+        podAntiAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+            - labelSelector:
+                matchLabels:
+                  app.kubernetes.io/name: kanidm
+                  app.kubernetes.io/instance: kanidm
+              topologyKey: kubernetes.io/hostname
+      securityContext:
+        runAsUser: 1000
+        runAsGroup: 1000
+        runAsNonRoot: true
+        fsGroup: 1000
+      initContainers:
+        - name: config-init
+          image: busybox:1.36
+          command: ["/bin/sh", "-c"]
+          args:
+            - |
+              set -e
+              cp "/config-template/server-${POD_NAME##*-}.toml" /config/server.toml
+              for peer in kanidm-0 kanidm-1 kanidm-2; do
+                [ "${peer}" = "${POD_NAME}" ] && continue
+                cert_file="/repl-certs/${peer}"
+                [ -s "${cert_file}" ] || continue
+                fqdn="${peer}.kanidm-headless.kanidm.svc.cluster.local"
+                printf '\n[replication."repl://%s:8444"]\ntype = "mutual-pull"\npartner_cert = "%s"\n' \
+                  "${fqdn}" "$(cat ${cert_file})" >> /config/server.toml
+              done
+          env:
+            - name: POD_NAME
+              valueFrom:
+                fieldRef:
+                  fieldPath: metadata.name
+          volumeMounts:
+            - name: config-template
+              mountPath: /config-template
+              readOnly: true
+            - name: config
+              mountPath: /config
+            - name: repl-certs
+              mountPath: /repl-certs
+              readOnly: true
+          securityContext:
+            allowPrivilegeEscalation: false
+            readOnlyRootFilesystem: true
+      containers:
+        - name: kanidm
+          image: kanidm/server:1.10.3
+          command: ["/sbin/kanidmd"]
+          args: ["server", "-c", "/config/server.toml"]
+          ports:
+            - name: https
+              containerPort: 8443
+              protocol: TCP
+            - name: replication
+              containerPort: 8444
+              protocol: TCP
+          volumeMounts:
+            - name: data
+              mountPath: /data
+            - name: config
+              mountPath: /config
+              readOnly: true
+            - name: tls
+              mountPath: /data/tls
+              readOnly: true
+          securityContext:
+            allowPrivilegeEscalation: false
+            readOnlyRootFilesystem: false
+          resources:
+            requests:
+              memory: 256Mi
+              cpu: 100m
+            limits:
+              memory: 1Gi
+              cpu: 500m
+          readinessProbe:
+            tcpSocket:
+              port: 8443
+            initialDelaySeconds: 15
+            periodSeconds: 10
+          livenessProbe:
+            tcpSocket:
+              port: 8443
+            initialDelaySeconds: 30
+            periodSeconds: 30
+      volumes:
+        - name: config-template
+          configMap:
+            name: kanidm-config
+        - name: config
+          emptyDir: {}
+        - name: repl-certs
+          secret:
+            secretName: kanidm-repl-certs
+        - name: tls
+          secret:
+            secretName: kanidm-tls
+  volumeClaimTemplates:
+    - metadata:
+        name: data
+      spec:
+        accessModes: [ReadWriteOnce]
+        storageClassName: cephrbd-fast-delete
+        resources:
+          requests:
+            storage: 10Gi
@@ -0,0 +1,26 @@
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: TLSRoute
+metadata:
+  name: kanidm
+  namespace: kanidm
+  labels:
+    app.kubernetes.io/name: kanidm
+    app.kubernetes.io/instance: kanidm
+spec:
+  hostnames:
+    - kanidm.k8s.syd1.au.unkin.net
+    - auth.unkin.net
+    - au.auth.unkin.net
+  parentRefs:
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: kanidm
+      sectionName: https-passthrough
+  rules:
+    - backendRefs:
+        - group: ""
+          kind: Service
+          name: kanidm
+          port: 8443
+          weight: 1
@@ -0,0 +1,18 @@
+---
+apiVersion: secrets.hashicorp.com/v1beta1
+kind: VaultAuth
+metadata:
+  name: default
+  namespace: kanidm
+spec:
+  allowedNamespaces:
+    - kanidm
+  kubernetes:
+    audiences:
+      - vault
+    role: default
+    serviceAccount: default
+    tokenExpirationSeconds: 600
+  method: kubernetes
+  mount: k8s/au/syd1
+  vaultConnectionRef: vso-system/default
@@ -0,0 +1,20 @@
+---
+apiVersion: secrets.hashicorp.com/v1beta1
+kind: VaultStaticSecret
+metadata:
+  name: repl-certs
+  namespace: kanidm
+  labels:
+    app.kubernetes.io/name: kanidm
+    app.kubernetes.io/instance: kanidm
+spec:
+  vaultAuthRef: default
+  mount: kv
+  type: kv-v2
+  path: kubernetes/namespace/kanidm/default/repl-certs
+  refreshAfter: 5m
+  destination:
+    name: kanidm-repl-certs
+    create: true
+    overwrite: true
+  hmacSecretData: true
@@ -76,8 +76,8 @@ spec:
    updateInterval: 30
  resources:
    limits:
-      cpu: 1
-      memory: 1024Mi
+      cpu: "1"
+      memory: 1Gi
    requests:
      cpu: 250m
      memory: 512Mi
@@ -8,7 +8,9 @@ spec:
  hostnames:
    - litellm.k8s.syd1.au.unkin.net
  parentRefs:
-    - name: litellm
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: litellm
      sectionName: http
  rules:
    - filters:
@@ -30,12 +32,17 @@ spec:
  hostnames:
    - litellm.k8s.syd1.au.unkin.net
  parentRefs:
-    - name: litellm
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: litellm
      sectionName: https
  rules:
    - backendRefs:
-        - name: litellm
+        - group: ""
+          kind: Service
+          name: litellm
          port: 4000
+          weight: 1
      matches:
        - path:
            type: PathPrefix
@@ -8,7 +8,9 @@ spec:
  hostnames:
    - paperclip.k8s.syd1.au.unkin.net
  parentRefs:
-    - name: paperclip
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: paperclip
      sectionName: http
  rules:
    - filters:
@@ -30,12 +32,17 @@ spec:
  hostnames:
    - paperclip.k8s.syd1.au.unkin.net
  parentRefs:
-    - name: paperclip
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: paperclip
      sectionName: https
  rules:
    - backendRefs:
-        - name: paperclip
+        - group: ""
+          kind: Service
+          name: paperclip
          port: 3100
+          weight: 1
      matches:
        - path:
            type: PathPrefix
@@ -0,0 +1,6 @@
+---
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+resources:
+  - priorityclasses.yaml
@@ -0,0 +1,36 @@
+---
+apiVersion: scheduling.k8s.io/v1
+kind: PriorityClass
+metadata:
+  name: low
+value: 100
+preemptionPolicy: Never
+globalDefault: false
+description: "Low-importance workloads. Can be evicted under pressure but will not preempt other pods."
+---
+apiVersion: scheduling.k8s.io/v1
+kind: PriorityClass
+metadata:
+  name: power
+value: 100
+preemptionPolicy: Never
+globalDefault: false
+description: "Compute-heavy workloads with low scheduling importance. Evictable under pressure."
+---
+apiVersion: scheduling.k8s.io/v1
+kind: PriorityClass
+metadata:
+  name: medium
+value: 10000
+preemptionPolicy: PreemptLowerPriority
+globalDefault: false
+description: "Standard workloads. Will preempt low-priority pods if the cluster is under pressure."
+---
+apiVersion: scheduling.k8s.io/v1
+kind: PriorityClass
+metadata:
+  name: high
+value: 100000
+preemptionPolicy: PreemptLowerPriority
+globalDefault: false
+description: "High-importance services. Will preempt medium- and low-priority pods if necessary."
@@ -13,7 +13,9 @@ spec:
  hostnames:
    - puppetboard.k8s.syd1.au.unkin.net
  parentRefs:
-    - name: puppetboard
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: puppetboard
      sectionName: http
  rules:
    - filters:
@@ -40,12 +42,17 @@ spec:
  hostnames:
    - puppetboard.k8s.syd1.au.unkin.net
  parentRefs:
-    - name: puppetboard
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: puppetboard
      sectionName: https
  rules:
    - backendRefs:
-        - name: puppetboard
+        - group: ""
+          kind: Service
+          name: puppetboard
          port: 80
+          weight: 1
      matches:
        - path:
            type: PathPrefix
@@ -13,12 +13,17 @@ spec:
  hostnames:
    - puppetdb.k8s.syd1.au.unkin.net
  parentRefs:
-    - name: puppetdb
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: puppetdb
      sectionName: https
  rules:
    - backendRefs:
-        - name: puppetdb
+        - group: ""
+          kind: Service
+          name: puppetdb
          port: 8080
+          weight: 1
      matches:
        - path:
            type: PathPrefix
@@ -9,7 +9,6 @@ metadata:
  name: puppetdb
  namespace: puppet
 spec:
-  clusterIP: null
  ports:
  - name: pdb-http
    port: 8080
@@ -11,7 +11,9 @@ spec:
  hostnames:
    - vault.k8s.syd1.au.unkin.net
  parentRefs:
-    - name: vault
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: vault
      sectionName: http
  rules:
    - filters:
@@ -36,12 +38,17 @@ spec:
  hostnames:
    - vault.k8s.syd1.au.unkin.net
  parentRefs:
-    - name: vault
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: vault
      sectionName: https
  rules:
    - backendRefs:
-        - name: vault
+        - group: ""
+          kind: Service
+          name: vault
          port: 8200
+          weight: 1
      matches:
        - path:
            type: PathPrefix
@@ -60,12 +67,17 @@ spec:
    - vault.service.consul
    - vault.query.consul
  parentRefs:
-    - name: vault
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: vault
      sectionName: vault-direct
  rules:
    - backendRefs:
-        - name: vault
+        - group: ""
+          kind: Service
+          name: vault
          port: 8200
+          weight: 1
      matches:
        - path:
            type: PathPrefix
@@ -6,3 +6,4 @@ resources:
  - namespace.yaml
  - gateway.yaml
  - httproute.yaml
+  - role_k8s-service-registration.yaml
@@ -0,0 +1,24 @@
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  name: vault-k8s-service-registration
+  namespace: vault
+rules:
+  - apiGroups: [""]
+    resources: ["pods"]
+    verbs: ["get", "update", "patch"]
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: vault-k8s-service-registration
+  namespace: vault
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: vault-k8s-service-registration
+subjects:
+  - kind: ServiceAccount
+    name: vault
+    namespace: vault
@@ -37,7 +37,7 @@ server:
      cpu: 100m
    limits:
      memory: 2Gi
-      cpu: 1000m
+      cpu: "1"

 client:
  enabled: false
@@ -0,0 +1,6 @@
+---
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+resources:
+  - ../../../base/kanidm
@@ -0,0 +1,6 @@
+---
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+resources:
+  - ../../../base/priority-classes
@@ -40,9 +40,7 @@ server:
          }
        }

-        service_registration "consul" {
-          address = "consul-server.consul.svc.cluster.local:8500"
-        }
+        service_registration "kubernetes" {}

  dataStorage:
    enabled: true
@@ -50,6 +48,14 @@ server:
    storageClass: cephrbd-fast-delete
    accessMode: ReadWriteOnce

+  extraEnv:
+    - name: VAULT_K8S_NAMESPACE
+      value: vault
+    - name: VAULT_K8S_POD_NAME
+      valueFrom:
+        fieldRef:
+          fieldPath: metadata.name
+
  statefulSet:
    securityContext:
      container:
@@ -2,6 +2,7 @@ agent:
  replicaCount: 3
  env:
    WOODPECKER_MAX_WORKFLOWS: "8"
+    WOODPECKER_BACKEND_K8S_PRIORITY_CLASS: power
    WOODPECKER_BACKEND_K8S_STORAGE_CLASS: cephrbd-fast-delete
    WOODPECKER_BACKEND_K8S_VOLUME_SIZE: 10G
    WOODPECKER_BACKEND_K8S_STORAGE_RWX: false
@@ -20,7 +20,9 @@ spec:
      - path: apps/overlays/*/externaldns
      - path: apps/overlays/*/inteldeviceplugins-system
      - path: apps/overlays/*/jfrog
+      - path: apps/overlays/*/kanidm
      - path: apps/overlays/*/node-feature-discovery
+      - path: apps/overlays/*/priority-classes
      - path: apps/overlays/*/puppet
      - path: apps/overlays/*/purelb
      - path: apps/overlays/*/reflector-system
@@ -43,6 +45,12 @@ spec:
      destination:
        server: https://kubernetes.default.svc
        namespace: '{{path[3]}}'  # Use directory name as namespace
+      ignoreDifferences:
+      - group: ""
+        kind: ConfigMap
+        name: kanidm-repl-certs
+        jsonPointers:
+        - /data
      syncPolicy:
        automated:
          prune: true
@@ -27,8 +27,12 @@ spec:
      server: https://kubernetes.default.svc
    - namespace: 'jfrog'
      server: https://kubernetes.default.svc
+    - namespace: 'kanidm'
+      server: https://kubernetes.default.svc
    - namespace: 'node-feature-discovery'
      server: https://kubernetes.default.svc
+    - namespace: 'priority-classes'
+      server: https://kubernetes.default.svc
    - namespace: 'purelb'
      server: https://kubernetes.default.svc
    - namespace: 'puppet'
Author	SHA1	Message	Date
unkinben	0199a422a0	Merge branch 'main' into fix-kanidm-replication-config ci/woodpecker/pr/pre-commit Pipeline was successful Details ci/woodpecker/pr/kubeconform Pipeline was successful Details	2026-05-30 23:33:36 +10:00
unkinben	f11ec1056d	fix(kanidm): remove invalid automatic_refresh from replication config (#179 ) Reviewed-on: #179	2026-05-30 23:20:48 +10:00
unkinben	1b4b22cad8	fix(kanidm): remove invalid automatic_refresh from replication config ci/woodpecker/pr/pre-commit Pipeline was successful Details ci/woodpecker/pr/kubeconform Pipeline was successful Details	2026-05-30 23:15:41 +10:00
unkinben	ed7feaf19a	Update apps/base/kanidm/vaultauth.yaml (#177 ) Fix the VaultAuth object Reviewed-on: #177	2026-05-30 23:11:38 +10:00
unkinben	4d594fbde7	feat(kanidm): vault-managed replication certs with auto-restart (#176 ) - Store per-pod replication certs in Vault (kv/kubernetes/namespace/kanidm/default/repl-certs) - VaultAuth + VaultStaticSecret sync certs to kanidm-repl-certs Secret - busybox config-init init container injects peer certs from Secret into server.toml at startup - Remove hardcoded partner_cert entries from per-pod server.toml templates - Add automatic_refresh = true to all replication configs - Add reloader.stakater.com/auto annotation to trigger rolling restart on ConfigMap/Secret changes - Document domain UUID mismatch resolution and cert rotation in README Reviewed-on: #176	2026-05-30 23:00:46 +10:00
unkinben	1b781e0885	feat(woodpecker): set workflow pod priority class to power (#175 ) ## Summary Sets `WOODPECKER_BACKEND_K8S_PRIORITY_CLASS: power` on the Woodpecker agent so all CI pipeline pods are scheduled with the `power` PriorityClass (value 100, preemptionPolicy: Never). This means pipeline pods can be evicted when the cluster is under pressure but won't preempt other workloads. ## Dependency Requires the `power` PriorityClass to exist on the cluster — deploy PR #174 (priority-classes app) first. ## Test plan - Trigger a pipeline run and confirm pods are created with `priorityClassName: power` - `kubectl get pod -n woodpecker -o jsonpath='{.items[*].spec.priorityClassName}'` Reviewed-on: #175	2026-05-26 23:58:57 +10:00
unkinben	ede25a3858	feat(platform): add priority-classes app with low/power/medium/high classes (#174 ) ## Summary - New `apps/base/priority-classes/` app with four `PriorityClass` objects managed via the `platform` ArgoCD project - Adds `apps/overlays/*/priority-classes` to the platform ApplicationSet generator - Adds `priority-classes` namespace to platform AppProject destinations (required even for cluster-scoped resources) \| Class \| Value \| PreemptionPolicy \| Intent \| \|---\|---\|---\|---\| \| `low` \| 100 \| Never \| Background work; evictable, won't preempt others \| \| `power` \| 100 \| Never \| Compute-heavy but expendable (e.g. AI/ML workloads) \| \| `medium` \| 10000 \| PreemptLowerPriority \| Standard services \| \| `high` \| 100000 \| PreemptLowerPriority \| Critical services; preempts lower-priority pods \| `PriorityClass` is already in the platform project's `clusterResourceWhitelist` so no project policy changes were needed. ## Test plan - ArgoCD syncs `platform-priority-classes` successfully - `kubectl get priorityclasses low power medium high` shows all four classes Reviewed-on: #174	2026-05-26 23:41:54 +10:00
unkinben	f5f713fe86	feat(artifactapi): add open-webui/open-webui to ghcr immutable patterns (#173 ) Part of #155 (prerequisite for open-webui deployment PR #172). ## Summary - Adds `^open-webui/open-webui` to the `ghcr` remote's `immutable_patterns` in `remote-docker.yaml` so version-pinned open-webui image pulls are cached indefinitely through artifactapi ## Test plan - artifactapi serves `ghcr.io/open-webui/open-webui:<version>` with `X-Artifact-Source: cache` on second fetch Reviewed-on: #173	2026-05-26 23:28:27 +10:00
unkinben	3990fbfe06	feat(vault): switch to Kubernetes service registration (#171 ) Replaces Consul service registration with the native Kubernetes provider so Vault labels its own pods with active/standby/perf-standby status without requiring a Consul dependency. ## Changes - `values.yaml`: swap `service_registration "consul"` for `service_registration "kubernetes" {}`, add `VAULT_K8S_NAMESPACE` and `VAULT_K8S_POD_NAME` env vars via downward API - `role_k8s-service-registration.yaml`: Role + RoleBinding granting the `vault` service account `get`/`update`/`patch` on pods - `kustomization.yaml`: include new RBAC file Reviewed-on: #171	2026-05-26 00:06:56 +10:00
unkinben	d358098fff	chore: update replication certs (#170 ) - add replication certs for kanidm-0, kanidm-1 and kanidm-2 Reviewed-on: #170	2026-05-25 23:52:06 +10:00
unkinben	201e601737	feat: update kanidm replicaiton (#169 ) - split to per-server configs - remove init containers that attempted to automate the replication config - add README.md Reviewed-on: #169	2026-05-25 23:25:48 +10:00
unkinben	d230d87ec9	feat(artifactapi): add conftest to GitHub generic remote cache (#168 ) ## Summary - Adds `open-policy-agent/conftest/./conftest_._Linux_x86_64.tar.gz$` to the `github` remote immutable patterns in artifactapi ## Why conftest v0.68.2 (https://github.com/open-policy-agent/conftest/releases/tag/v0.68.2) is now used for OPA policy checks in CI (see #167). Caching the release tarball in artifactapi reduces external dependency on GitHub during builds. Reviewed-on: #168	2026-05-25 22:44:57 +10:00
unkinben	6497dab25e	fix(puppet): remove explicit clusterIP: null from puppetdb Service (#166 ) ## Summary - Removes `clusterIP: null` from the `puppetdb` Service spec ## Why Setting `clusterIP: null` makes ArgoCD's desired state explicit about the field being null. Kubernetes assigns a real IP on creation and the field is immutable afterward. The null vs assigned-IP mismatch causes permanent OutOfSync on the puppetdb Service. Removing the field means ArgoCD no longer claims ownership of `clusterIP`, so the API server's value is authoritative. Reviewed-on: #166	2026-05-25 22:44:24 +10:00
unkinben	f403c6b05d	fix(kanidm): add explicit group/kind/weight to TLSRoute refs (#165 ) ## Summary - Adds `group: gateway.networking.k8s.io` and `kind: Gateway` to `parentRefs` - Adds `group: ""`, `kind: Service`, and `weight: 1` to `backendRefs` ## Why The Gateway API controller defaults these fields when creating/updating TLSRoute objects, so the live state always has them. ArgoCD diffs desired vs live by string comparison, causing the `kanidm` TLSRoute to show permanent OutOfSync. Same root cause as #162 (HTTPRoutes). Reviewed-on: #165	2026-05-25 22:43:52 +10:00
unkinben	ac8b8212bd	fix(consul): normalize cpu limit to canonical string form (#164 ) ## Summary - Changes `server.resources.limits.cpu` from `1000m` to `"1"` in consul Helm values ## Why `1000m` (1000 milliCPU) is equivalent to `1` CPU, but Kubernetes normalizes the value to `"1"` when storing. ArgoCD diffs desired vs live by string comparison, so the mismatch causes a permanent OutOfSync on the `consul-server` StatefulSet. Same root cause as #163. Reviewed-on: #164	2026-05-25 22:43:35 +10:00
unkinben	dd282f59fb	fix(litellm): normalize postgres cluster resource values (#163 ) ## Summary - Changes `limits.memory` from `1024Mi` to `1Gi` (same value, canonical form) - Changes `limits.cpu` from `1` (integer) to `"1"` (string, canonical form) ## Why Kubernetes normalizes resource quantities on write — `1024Mi` becomes `1Gi` and integer `1` becomes string `"1"`. ArgoCD diffs by string comparison, so these equivalent values cause a permanent OutOfSync on the `litellm-postgres` Cluster. Reviewed-on: #163	2026-05-24 23:30:10 +10:00
unkinben	1890dd4bda	fix(gateways): add explicit group/kind/weight to all HTTPRoute refs (#162 ) ## Summary - Adds `group: gateway.networking.k8s.io` and `kind: Gateway` to all `parentRefs` entries - Adds `group: ""`, `kind: Service`, and `weight: 1` to all `backendRefs` entries - Affects 9 HTTPRoute files across artifactapi, cattle-system, consul, kanidm, litellm, paperclip, puppet, and vault ## Why ArgoCD diffs the desired manifest against the live Kubernetes object. The Gateway API controller defaults these fields when creating/updating objects, so the live state always has them — causing persistent OutOfSync for every HTTPRoute. Same root cause as #153 (certificateRefs). ## Test plan - [ ] All affected ArgoCD applications show Synced after merge Reviewed-on: #162	2026-05-24 20:32:37 +10:00
unkinben	6815b66010	fix(kanidm): use dockerhub image instead of ghcr.io (#161 ) ## Summary - Changes both `config-init` init container and `kanidm` container images from `ghcr.io/kanidm/server:1.10.3` to `kanidm/server:1.10.3` ## Why `kanidm/server` is published on Docker Hub, not ghcr.io. RKE2 rewrites dockerhub pulls through the artifactapi mirror automatically. ## Test plan - [ ] Pods roll successfully after ArgoCD sync - [ ] Verify kanidm cluster replication still healthy Reviewed-on: #161	2026-05-24 20:27:21 +10:00
unkinben	7cbec33588	fix(artifactapi): move kanidm to dockerhub remote (#160 ) ## Summary - Removes `^kanidm/` from the `ghcr` remote immutable_patterns - Adds `^kanidm/` to the `dockerhub` remote immutable_patterns ## Why `kanidm/server` is published on Docker Hub, not ghcr.io. Pulling via the `ghcr` cache was failing with 403 on anonymous token fetch → 502 Bad Gateway. ## Test plan - [ ] `docker pull artifactapi.k8s.syd1.au.unkin.net/dockerhub/kanidm/server:1.10.3` succeeds after artifactapi redeploys Reviewed-on: #160	2026-05-24 20:24:33 +10:00
unkinben	3756208ccd	benvin/kanidm (#159 ) Reviewed-on: #159	2026-05-24 19:55:22 +10:00
unkinben	6ce92e8ead	benvin/artifactapi-mail-images (#158 ) Reviewed-on: #158	2026-05-24 14:44:38 +10:00
unkinben	af79d86db6	feat(artifactapi): cache stalwart webadmin zip (#157 ) ## Summary - Adds \`stalwartlabs/webadmin/releases/latest/download/webadmin.zip\` to \`mutable_patterns\` in the \`github\` generic remote so the stalwart webadmin UI can be fetched through artifactapi rather than directly from GitHub. ## Notes - Uses \`mutable_patterns\` (not \`immutable\`) because \`releases/latest\` resolves to whichever release is current and changes over time. - Access URL: \`https://artifactapi.k8s.syd1.au.unkin.net/generic/github/stalwartlabs/webadmin/releases/latest/download/webadmin.zip\` Reviewed-on: #157	2026-05-24 12:55:16 +10:00
unkinben	5f4c9225bb	feat(artifactapi): add mail stack images to docker registry cache (#156 ) - ghcr: stalwartlabs/stalwart (Stalwart mail server) - dockerhub: rspamd/rspamd (spam filter), tozd/postfix (MTA gateway) Reviewed-on: #156	2026-05-24 12:42:27 +10:00