feat(open-webui): HA deployment with CNPG, PDB, and session persistence

- Switch from SQLite/PVC to CNPG PostgreSQL (3 instances, low-resource) with a transaction-mode PgBouncer pooler (2 instances) - Raise open-webui replicas to 3 with priorityClassName: power - Add PodDisruptionBudget (minAvailable: 1) - Add Gateway API sessionPersistence (cookie) on the HTTPS HTTPRoute so WebSocket connections stick to the same backend pod - Add postgres-credentials VaultStaticSecret; DATABASE_URL must be added to kv/kubernetes/namespace/open-webui/default/open-webui-credentials
fix(open-webui): use traefik-internal gateway for chat hostname
2026-05-26 23:37:10 +10:00 · 2026-05-26 23:26:01 +10:00 · 2026-05-26 23:25:26 +10:00 · 2026-05-26 00:11:25 +10:00
30 changed files with 409 additions and 162 deletions
@@ -6,7 +6,6 @@ remotes:
    immutable_patterns:
      - "^cloudnative-pg/cloudnative-pg"
      - "^emberstack/helm-charts"
-      - "^open-webui/open-webui"
      - "^openvoxproject/"
      - "^stakater/reloader"
      - "^stalwartlabs/stalwart"
@@ -1,51 +1,32 @@
 # kanidm

-Three-replica kanidm identity server with Vault-managed replication certificates.
-
-## Architecture
-
- Per-pod `server-N.toml` in `resources/` — each has its own replication origin hardcoded
- `config-init` busybox init container copies the right config and injects peer certs from the
-  vault-synced `kanidm-repl-certs` Secret at pod startup
- `reloader.stakater.com/auto: "true"` triggers a rolling restart when the ConfigMap or Secret changes
- Vault path: `kv/kubernetes/namespace/kanidm/default/repl-certs`
-  - Keys: `kanidm-0`, `kanidm-1`, `kanidm-2` — each holds that pod's replication certificate
+Single-replica kanidm identity server deployment.

 ## Initial setup

-After the first pod starts, generate the admin credentials:
+After the pod starts for the first time, generate the admin and idm_admin credentials:

 ```bash
-kubectl exec -n kanidm kanidm-0 -- /sbin/kanidmd recover-account -c /config/server.toml admin
-kubectl exec -n kanidm kanidm-0 -- /sbin/kanidmd recover-account -c /config/server.toml idm_admin
+kubectl exec -n kanidm kanidm-0 -- /sbin/kanidmd recover-account admin
+kubectl exec -n kanidm kanidm-0 -- /sbin/kanidmd recover-account idm_admin
 ```

-## Replication certificate rotation
+## Adding replication

-When certs need to be renewed, update vault and reloader will roll the StatefulSet:
+If replication is needed in the future:
+
+1. Scale the StatefulSet to 3 replicas and add `podAntiAffinity` to spread across nodes.
+2. Add a `[replication]` section to `configmap.yaml` per pod (origin is pod-specific:
+   `repl://kanidm-N.kanidm-headless.kanidm.svc.cluster.local:8444`).
+3. Add the replication port (8444) back to the StatefulSet container ports and headless service.
+4. Restore `rbac.yaml` for the cert-publisher sidecar, or exchange certificates manually:

 ```bash
-# Get new cert from a pod
-kubectl exec -it -n kanidm kanidm-N -- /sbin/kanidmd renew-replication-certificate -c /config/server.toml
+# On each pod, get its replication certificate
+kubectl exec -n kanidm kanidm-0 -- /sbin/kanidmd renew-replication-certificate

-# Write updated cert to vault (reloader triggers restart automatically)
-vault kv patch kv/kubernetes/namespace/kanidm/default/repl-certs "kanidm-N=<cert>"
-```
-
-## Resolving domain UUID mismatch
-
-If pods initialized independently (each with a different domain UUID), replication will fail with
-`Consumer Domain UUID does not match`. Fix by resetting kanidm-1 and kanidm-2 to sync from
-kanidm-0 (the authoritative node):
-
-```bash
-# Scale down to avoid split-brain during reset
-kubectl scale statefulset -n kanidm kanidm --replicas=1
-
-# Delete the stale PVCs for the replica pods
-kubectl delete pvc -n kanidm data-kanidm-1 data-kanidm-2
-
-# Scale back up — replicas start with empty DBs and automatic_refresh=true
-# will trigger a full sync from kanidm-0 once TLS peer certs are verified
-kubectl scale statefulset -n kanidm kanidm --replicas=3
+# Add each peer's certificate to the other pods' configs under:
+# [replication."repl://<peer-fqdn>:8444"]
+# type = "mutual-pull"
+# partner_cert = "<cert>"
 ```
@@ -5,8 +5,6 @@ kind: Kustomization
 resources:
  - namespace.yaml
  - serviceaccount.yaml
-  - vaultauth.yaml
-  - vaultstaticsecret.yaml
  - certificate.yaml
  - service.yaml
  - statefulset.yaml
@@ -17,3 +17,11 @@ versions = 7
 [replication]
 origin = "repl://kanidm-0.kanidm-headless.kanidm.svc.cluster.local:8444"
 bindaddress = "[::]:8444"
+
+[replication."repl://kanidm-1.kanidm-headless.kanidm.svc.cluster.local:8444"]
+type = "mutual-pull"
+partner_cert = "MIIB-TCCAZ-gAwIBAgIRASqOpORz60wiv7wF_7oBOxQwCgYIKoZIzj0EAwIwTDEtMCsGA1UEAwwkMmE4ZWE0ZTQtNzNlYi00YzIyLWJmYmMtMDVmZmJhMDEzYjE0MRswGQYDVQQKDBJLYW5pZG0gUmVwbGljYXRpb24wHhcNMjYwNTI1MTMyODM5WhcNMzAwNTI1MTMyODM5WjBMMS0wKwYDVQQDDCQyYThlYTRlNC03M2ViLTRjMjItYmZiYy0wNWZmYmEwMTNiMTQxGzAZBgNVBAoMEkthbmlkbSBSZXBsaWNhdGlvbjBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABFQP3zpFRt7TCOhzrUpOJBojn-sC2LmqZUub8P2ymVdIQbmoAyh4Q8Me0hNWJFyuFDnnqO06dt5I2iv0910-X6KjYjBgMCAGA1UdJQEB_wQWMBQGCCsGAQUFBwMCBggrBgEFBQcDATA8BgNVHREENTAzgjFrYW5pZG0tMS5rYW5pZG0taGVhZGxlc3Mua2FuaWRtLnN2Yy5jbHVzdGVyLmxvY2FsMAoGCCqGSM49BAMCA0gAMEUCIGjl58U6apcDjMEPIca8Wwg_JMfuMvV-uVcJI49Gl_9GAiEA2tFdb9rnFeBI7mwysScf5UsmY3ZziMD3UVm1vWN2IKs"
+
+[replication."repl://kanidm-2.kanidm-headless.kanidm.svc.cluster.local:8444"]
+type = "mutual-pull"
+partner_cert = "MIIB-TCCAZ-gAwIBAgIRAeFGUAJbCkJ2vzf_Vv4qjeUwCgYIKoZIzj0EAwIwTDEtMCsGA1UEAwwkZTE0NjUwMDItNWIwYS00Mjc2LWJmMzctZmY1NmZlMmE4ZGU1MRswGQYDVQQKDBJLYW5pZG0gUmVwbGljYXRpb24wHhcNMjYwNTI1MTMyOTEwWhcNMzAwNTI1MTMyOTEwWjBMMS0wKwYDVQQDDCRlMTQ2NTAwMi01YjBhLTQyNzYtYmYzNy1mZjU2ZmUyYThkZTUxGzAZBgNVBAoMEkthbmlkbSBSZXBsaWNhdGlvbjBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABCrncHSbDNSV3_aOSZ14plbVfrvSXQQL9MOqvrDKlf_Q6WbcA8OrTUjs3Jt0Q2beWjC3Z5-5c9fGu8M_k2iVWf-jYjBgMCAGA1UdJQEB_wQWMBQGCCsGAQUFBwMCBggrBgEFBQcDATA8BgNVHREENTAzgjFrYW5pZG0tMi5rYW5pZG0taGVhZGxlc3Mua2FuaWRtLnN2Yy5jbHVzdGVyLmxvY2FsMAoGCCqGSM49BAMCA0gAMEUCIQDHY5Yl-bhDTuJaYnHSMSiSAEWPrDcRVzvfmOJukuJ1QQIgSwgyeSG3K0MY87DI1RDYAdZlpP1YOK3Yatj7-YSXPC0"
@@ -17,3 +17,11 @@ versions = 7
 [replication]
 origin = "repl://kanidm-1.kanidm-headless.kanidm.svc.cluster.local:8444"
 bindaddress = "[::]:8444"
+
+[replication."repl://kanidm-0.kanidm-headless.kanidm.svc.cluster.local:8444"]
+type = "mutual-pull"
+partner_cert = "MIIB-jCCAZ-gAwIBAgIRAVKuoPDpF0IBnvFjCwdK41EwCgYIKoZIzj0EAwIwTDEtMCsGA1UEAwwkNTJhZWEwZjAtZTkxNy00MjAxLTllZjEtNjMwYjA3NGFlMzUxMRswGQYDVQQKDBJLYW5pZG0gUmVwbGljYXRpb24wHhcNMjYwNTI1MTMzNzQ5WhcNMzAwNTI1MTMzNzQ5WjBMMS0wKwYDVQQDDCQ1MmFlYTBmMC1lOTE3LTQyMDEtOWVmMS02MzBiMDc0YWUzNTExGzAZBgNVBAoMEkthbmlkbSBSZXBsaWNhdGlvbjBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABGejqjk0Eet-RILHI236wHYqISdnPlebqnkuUTh4W2mCzkmqKibyjxGIUOs8LBrUeTR2DxVR1VV6H2rYQk2wdROjYjBgMCAGA1UdJQEB_wQWMBQGCCsGAQUFBwMCBggrBgEFBQcDATA8BgNVHREENTAzgjFrYW5pZG0tMC5rYW5pZG0taGVhZGxlc3Mua2FuaWRtLnN2Yy5jbHVzdGVyLmxvY2FsMAoGCCqGSM49BAMCA0kAMEYCIQCSkFj2A-KVWv2tKJLFzb18J5eWWKtsvWewZTn-FVnRnQIhAKJbt84IoZ9oXxgfp0VOLyVZiAgUgwMFS6JOfno3D-Nw"
+
+[replication."repl://kanidm-2.kanidm-headless.kanidm.svc.cluster.local:8444"]
+type = "mutual-pull"
+partner_cert = "MIIB-TCCAZ-gAwIBAgIRAeFGUAJbCkJ2vzf_Vv4qjeUwCgYIKoZIzj0EAwIwTDEtMCsGA1UEAwwkZTE0NjUwMDItNWIwYS00Mjc2LWJmMzctZmY1NmZlMmE4ZGU1MRswGQYDVQQKDBJLYW5pZG0gUmVwbGljYXRpb24wHhcNMjYwNTI1MTMyOTEwWhcNMzAwNTI1MTMyOTEwWjBMMS0wKwYDVQQDDCRlMTQ2NTAwMi01YjBhLTQyNzYtYmYzNy1mZjU2ZmUyYThkZTUxGzAZBgNVBAoMEkthbmlkbSBSZXBsaWNhdGlvbjBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABCrncHSbDNSV3_aOSZ14plbVfrvSXQQL9MOqvrDKlf_Q6WbcA8OrTUjs3Jt0Q2beWjC3Z5-5c9fGu8M_k2iVWf-jYjBgMCAGA1UdJQEB_wQWMBQGCCsGAQUFBwMCBggrBgEFBQcDATA8BgNVHREENTAzgjFrYW5pZG0tMi5rYW5pZG0taGVhZGxlc3Mua2FuaWRtLnN2Yy5jbHVzdGVyLmxvY2FsMAoGCCqGSM49BAMCA0gAMEUCIQDHY5Yl-bhDTuJaYnHSMSiSAEWPrDcRVzvfmOJukuJ1QQIgSwgyeSG3K0MY87DI1RDYAdZlpP1YOK3Yatj7-YSXPC0"
@@ -17,3 +17,11 @@ versions = 7
 [replication]
 origin = "repl://kanidm-2.kanidm-headless.kanidm.svc.cluster.local:8444"
 bindaddress = "[::]:8444"
+
+[replication."repl://kanidm-0.kanidm-headless.kanidm.svc.cluster.local:8444"]
+type = "mutual-pull"
+partner_cert = "MIIB-jCCAZ-gAwIBAgIRAVKuoPDpF0IBnvFjCwdK41EwCgYIKoZIzj0EAwIwTDEtMCsGA1UEAwwkNTJhZWEwZjAtZTkxNy00MjAxLTllZjEtNjMwYjA3NGFlMzUxMRswGQYDVQQKDBJLYW5pZG0gUmVwbGljYXRpb24wHhcNMjYwNTI1MTMzNzQ5WhcNMzAwNTI1MTMzNzQ5WjBMMS0wKwYDVQQDDCQ1MmFlYTBmMC1lOTE3LTQyMDEtOWVmMS02MzBiMDc0YWUzNTExGzAZBgNVBAoMEkthbmlkbSBSZXBsaWNhdGlvbjBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABGejqjk0Eet-RILHI236wHYqISdnPlebqnkuUTh4W2mCzkmqKibyjxGIUOs8LBrUeTR2DxVR1VV6H2rYQk2wdROjYjBgMCAGA1UdJQEB_wQWMBQGCCsGAQUFBwMCBggrBgEFBQcDATA8BgNVHREENTAzgjFrYW5pZG0tMC5rYW5pZG0taGVhZGxlc3Mua2FuaWRtLnN2Yy5jbHVzdGVyLmxvY2FsMAoGCCqGSM49BAMCA0kAMEYCIQCSkFj2A-KVWv2tKJLFzb18J5eWWKtsvWewZTn-FVnRnQIhAKJbt84IoZ9oXxgfp0VOLyVZiAgUgwMFS6JOfno3D-Nw"
+
+[replication."repl://kanidm-1.kanidm-headless.kanidm.svc.cluster.local:8444"]
+type = "mutual-pull"
+partner_cert = "MIIB-TCCAZ-gAwIBAgIRASqOpORz60wiv7wF_7oBOxQwCgYIKoZIzj0EAwIwTDEtMCsGA1UEAwwkMmE4ZWE0ZTQtNzNlYi00YzIyLWJmYmMtMDVmZmJhMDEzYjE0MRswGQYDVQQKDBJLYW5pZG0gUmVwbGljYXRpb24wHhcNMjYwNTI1MTMyODM5WhcNMzAwNTI1MTMyODM5WjBMMS0wKwYDVQQDDCQyYThlYTRlNC03M2ViLTRjMjItYmZiYy0wNWZmYmEwMTNiMTQxGzAZBgNVBAoMEkthbmlkbSBSZXBsaWNhdGlvbjBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABFQP3zpFRt7TCOhzrUpOJBojn-sC2LmqZUub8P2ymVdIQbmoAyh4Q8Me0hNWJFyuFDnnqO06dt5I2iv0910-X6KjYjBgMCAGA1UdJQEB_wQWMBQGCCsGAQUFBwMCBggrBgEFBQcDATA8BgNVHREENTAzgjFrYW5pZG0tMS5rYW5pZG0taGVhZGxlc3Mua2FuaWRtLnN2Yy5jbHVzdGVyLmxvY2FsMAoGCCqGSM49BAMCA0gAMEUCIGjl58U6apcDjMEPIca8Wwg_JMfuMvV-uVcJI49Gl_9GAiEA2tFdb9rnFeBI7mwysScf5UsmY3ZziMD3UVm1vWN2IKs"
@@ -4,8 +4,6 @@ kind: StatefulSet
 metadata:
  name: kanidm
  namespace: kanidm
-  annotations:
-    reloader.stakater.com/auto: "true"
  labels:
    app.kubernetes.io/name: kanidm
    app.kubernetes.io/instance: kanidm
@@ -41,17 +39,7 @@ spec:
          image: busybox:1.36
          command: ["/bin/sh", "-c"]
          args:
-            - |
-              set -e
-              cp "/config-template/server-${POD_NAME##*-}.toml" /config/server.toml
-              for peer in kanidm-0 kanidm-1 kanidm-2; do
-                [ "${peer}" = "${POD_NAME}" ] && continue
-                cert_file="/repl-certs/${peer}"
-                [ -s "${cert_file}" ] || continue
-                fqdn="${peer}.kanidm-headless.kanidm.svc.cluster.local"
-                printf '\n[replication."repl://%s:8444"]\ntype = "mutual-pull"\npartner_cert = "%s"\n' \
-                  "${fqdn}" "$(cat ${cert_file})" >> /config/server.toml
-              done
+            - cp "/config-template/server-${POD_NAME##*-}.toml" /config/server.toml
          env:
            - name: POD_NAME
              valueFrom:
@@ -63,9 +51,6 @@ spec:
              readOnly: true
            - name: config
              mountPath: /config
-            - name: repl-certs
-              mountPath: /repl-certs
-              readOnly: true
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
@@ -116,9 +101,6 @@ spec:
            name: kanidm-config
        - name: config
          emptyDir: {}
-        - name: repl-certs
-          secret:
-            secretName: kanidm-repl-certs
        - name: tls
          secret:
            secretName: kanidm-tls
@@ -1,20 +0,0 @@
---
-apiVersion: secrets.hashicorp.com/v1beta1
-kind: VaultStaticSecret
-metadata:
-  name: repl-certs
-  namespace: kanidm
-  labels:
-    app.kubernetes.io/name: kanidm
-    app.kubernetes.io/instance: kanidm
-spec:
-  vaultAuthRef: default
-  mount: kv
-  type: kv-v2
-  path: kubernetes/namespace/kanidm/default/repl-certs
-  refreshAfter: 5m
-  destination:
-    name: kanidm-repl-certs
-    create: true
-    overwrite: true
-  hmacSecretData: true
@@ -0,0 +1,91 @@
+---
+apiVersion: postgresql.cnpg.io/v1
+kind: Cluster
+metadata:
+  name: open-webui-postgres
+  namespace: open-webui
+spec:
+  affinity:
+    podAntiAffinityType: preferred
+  bootstrap:
+    initdb:
+      database: open-webui
+      encoding: UTF8
+      localeCType: C
+      localeCollate: C
+      owner: open-webui
+      secret:
+        name: postgres-credentials
+  enablePDB: true
+  enableSuperuserAccess: false
+  failoverDelay: 0
+  imageName: ghcr.io/cloudnative-pg/postgresql:17-minimal-trixie
+  instances: 3
+  logLevel: info
+  maxSyncReplicas: 0
+  minSyncReplicas: 0
+  monitoring:
+    customQueriesConfigMap:
+      - key: queries
+        name: cnpg-default-monitoring
+    disableDefaultQueries: false
+    enablePodMonitor: false
+  postgresql:
+    parameters:
+      archive_mode: "on"
+      archive_timeout: 5min
+      dynamic_shared_memory_type: posix
+      effective_cache_size: 128MB
+      full_page_writes: "on"
+      log_destination: csvlog
+      log_directory: /controller/log
+      log_filename: postgres
+      log_rotation_age: "0"
+      log_rotation_size: "0"
+      log_truncate_on_rotation: "false"
+      logging_collector: "on"
+      max_connections: "100"
+      max_parallel_workers: "4"
+      max_replication_slots: "16"
+      max_worker_processes: "4"
+      shared_buffers: 64MB
+      shared_memory_type: mmap
+      ssl_max_protocol_version: TLSv1.3
+      ssl_min_protocol_version: TLSv1.3
+      wal_keep_size: 128MB
+      wal_level: logical
+      wal_log_hints: "on"
+      wal_receiver_timeout: 5s
+      wal_sender_timeout: 5s
+    syncReplicaElectionConstraint:
+      enabled: false
+  primaryUpdateMethod: restart
+  primaryUpdateStrategy: unsupervised
+  probes:
+    liveness:
+      isolationCheck:
+        connectionTimeout: 1000
+        enabled: true
+        requestTimeout: 1000
+  replicationSlots:
+    highAvailability:
+      enabled: true
+      slotPrefix: _cnpg_
+    synchronizeReplicas:
+      enabled: true
+    updateInterval: 30
+  resources:
+    limits:
+      cpu: 500m
+      memory: 512Mi
+    requests:
+      cpu: 100m
+      memory: 256Mi
+  smartShutdownTimeout: 180
+  startDelay: 3600
+  stopDelay: 1800
+  storage:
+    resizeInUseVolumes: true
+    size: 5Gi
+    storageClass: cephrbd-fast-delete
+  switchoverDelay: 3600
@@ -0,0 +1,33 @@
+---
+apiVersion: postgresql.cnpg.io/v1
+kind: Pooler
+metadata:
+  name: open-webui-postgres-pooler
+  namespace: open-webui
+spec:
+  cluster:
+    name: open-webui-postgres
+  instances: 2
+  pgbouncer:
+    parameters:
+      default_pool_size: "50"
+      max_client_conn: "200"
+    paused: false
+    poolMode: transaction
+  template:
+    metadata:
+      labels:
+        app: pooler
+    spec:
+      affinity:
+        podAntiAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+            - labelSelector:
+                matchExpressions:
+                  - key: app
+                    operator: In
+                    values:
+                      - pooler
+              topologyKey: kubernetes.io/hostname
+      containers: []
+  type: rw
@@ -0,0 +1,61 @@
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: open-webui
+  namespace: open-webui
+spec:
+  replicas: 3
+  selector:
+    matchLabels:
+      app: open-webui
+  template:
+    metadata:
+      annotations:
+        reloader.stakater.com/auto: "true"
+      labels:
+        app: open-webui
+    spec:
+      priorityClassName: power
+      containers:
+        - name: open-webui
+          image: ghcr.io/open-webui/open-webui:main
+          imagePullPolicy: Always
+          ports:
+            - containerPort: 8080
+              name: http
+              protocol: TCP
+          env:
+            - name: OPENAI_API_BASE_URL
+              value: https://litellm.k8s.syd1.au.unkin.net
+            - name: WEBUI_URL
+              value: https://chat.k8s.syd1.au.unkin.net
+          envFrom:
+            - secretRef:
+                name: open-webui-credentials
+          livenessProbe:
+            httpGet:
+              path: /health
+              port: 8080
+            failureThreshold: 3
+            initialDelaySeconds: 30
+            periodSeconds: 30
+            successThreshold: 1
+            timeoutSeconds: 5
+          readinessProbe:
+            httpGet:
+              path: /health
+              port: 8080
+            failureThreshold: 3
+            initialDelaySeconds: 10
+            periodSeconds: 10
+            successThreshold: 1
+            timeoutSeconds: 5
+          resources:
+            limits:
+              cpu: "2"
+              memory: 4Gi
+            requests:
+              cpu: 250m
+              memory: 512Mi
+      restartPolicy: Always
@@ -0,0 +1,37 @@
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: Gateway
+metadata:
+  labels:
+    traefik.io/instance: internal
+  annotations:
+    cert-manager.io/cluster-issuer: vault-issuer
+    cert-manager.io/common-name: chat.k8s.syd1.au.unkin.net
+    cert-manager.io/private-key-size: "4096"
+    external-dns.alpha.kubernetes.io/hostname: chat.k8s.syd1.au.unkin.net
+    external-dns.alpha.kubernetes.io/target: 198.18.200.4
+  name: open-webui
+  namespace: open-webui
+spec:
+  gatewayClassName: traefik-internal
+  listeners:
+    - allowedRoutes:
+        namespaces:
+          from: Same
+      hostname: chat.k8s.syd1.au.unkin.net
+      name: http
+      port: 80
+      protocol: HTTP
+    - allowedRoutes:
+        namespaces:
+          from: Same
+      hostname: chat.k8s.syd1.au.unkin.net
+      name: https
+      port: 443
+      protocol: HTTPS
+      tls:
+        certificateRefs:
+          - group: ""
+            kind: Secret
+            name: open-webui-tls
+        mode: Terminate
@@ -0,0 +1,53 @@
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: open-webui-http-redirect
+  namespace: open-webui
+spec:
+  hostnames:
+    - chat.k8s.syd1.au.unkin.net
+  parentRefs:
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: open-webui
+      sectionName: http
+  rules:
+    - filters:
+        - type: RequestRedirect
+          requestRedirect:
+            scheme: https
+            statusCode: 301
+      matches:
+        - path:
+            type: PathPrefix
+            value: /
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: open-webui
+  namespace: open-webui
+spec:
+  hostnames:
+    - chat.k8s.syd1.au.unkin.net
+  parentRefs:
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: open-webui
+      sectionName: https
+  rules:
+    - backendRefs:
+        - group: ""
+          kind: Service
+          name: open-webui
+          port: 8080
+          weight: 1
+      matches:
+        - path:
+            type: PathPrefix
+            value: /
+      sessionPersistence:
+        type: Cookie
+        cookieName: open-webui-backend
+        absoluteTimeout: 24h0m0s
@@ -0,0 +1,15 @@
+---
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+resources:
+  - namespace.yaml
+  - cnpg_cluster.yaml
+  - cnpg_pooler.yaml
+  - deployment.yaml
+  - pdb.yaml
+  - service.yaml
+  - gateway.yaml
+  - httproute.yaml
+  - vaultauth.yaml
+  - vaultstaticsecret.yaml
@@ -0,0 +1,5 @@
+---
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: open-webui
@@ -0,0 +1,11 @@
+---
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+  name: open-webui
+  namespace: open-webui
+spec:
+  minAvailable: 1
+  selector:
+    matchLabels:
+      app: open-webui
@@ -0,0 +1,17 @@
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: open-webui
+  namespace: open-webui
+spec:
+  internalTrafficPolicy: Cluster
+  ports:
+    - name: http
+      port: 8080
+      protocol: TCP
+      targetPort: http
+  selector:
+    app: open-webui
+  sessionAffinity: None
+  type: ClusterIP
@@ -3,10 +3,10 @@ apiVersion: secrets.hashicorp.com/v1beta1
 kind: VaultAuth
 metadata:
  name: default
-  namespace: kanidm
+  namespace: open-webui
 spec:
  allowedNamespaces:
-    - kanidm
+    - open-webui
  kubernetes:
    audiences:
      - vault
@@ -0,0 +1,34 @@
+---
+apiVersion: secrets.hashicorp.com/v1beta1
+kind: VaultStaticSecret
+metadata:
+  name: postgres-credentials
+  namespace: open-webui
+spec:
+  destination:
+    create: true
+    name: postgres-credentials
+    overwrite: true
+  hmacSecretData: true
+  mount: kv
+  path: kubernetes/namespace/open-webui/default/postgres-credentials
+  refreshAfter: 5m
+  type: kv-v2
+  vaultAuthRef: default
+---
+apiVersion: secrets.hashicorp.com/v1beta1
+kind: VaultStaticSecret
+metadata:
+  name: open-webui-credentials
+  namespace: open-webui
+spec:
+  destination:
+    create: true
+    name: open-webui-credentials
+    overwrite: true
+  hmacSecretData: true
+  mount: kv
+  path: kubernetes/namespace/open-webui/default/open-webui-credentials
+  refreshAfter: 5m
+  type: kv-v2
+  vaultAuthRef: default
@@ -1,36 +0,0 @@
---
-apiVersion: scheduling.k8s.io/v1
-kind: PriorityClass
-metadata:
-  name: low
-value: 100
-preemptionPolicy: Never
-globalDefault: false
-description: "Low-importance workloads. Can be evicted under pressure but will not preempt other pods."
---
-apiVersion: scheduling.k8s.io/v1
-kind: PriorityClass
-metadata:
-  name: power
-value: 100
-preemptionPolicy: Never
-globalDefault: false
-description: "Compute-heavy workloads with low scheduling importance. Evictable under pressure."
---
-apiVersion: scheduling.k8s.io/v1
-kind: PriorityClass
-metadata:
-  name: medium
-value: 10000
-preemptionPolicy: PreemptLowerPriority
-globalDefault: false
-description: "Standard workloads. Will preempt low-priority pods if the cluster is under pressure."
---
-apiVersion: scheduling.k8s.io/v1
-kind: PriorityClass
-metadata:
-  name: high
-value: 100000
-preemptionPolicy: PreemptLowerPriority
-globalDefault: false
-description: "High-importance services. Will preempt medium- and low-priority pods if necessary."
@@ -6,4 +6,3 @@ resources:
  - namespace.yaml
  - gateway.yaml
  - httproute.yaml
-  - role_k8s-service-registration.yaml
@@ -1,24 +0,0 @@
---
-apiVersion: rbac.authorization.k8s.io/v1
-kind: Role
-metadata:
-  name: vault-k8s-service-registration
-  namespace: vault
-rules:
-  - apiGroups: [""]
-    resources: ["pods"]
-    verbs: ["get", "update", "patch"]
---
-apiVersion: rbac.authorization.k8s.io/v1
-kind: RoleBinding
-metadata:
-  name: vault-k8s-service-registration
-  namespace: vault
-roleRef:
-  apiGroup: rbac.authorization.k8s.io
-  kind: Role
-  name: vault-k8s-service-registration
-subjects:
-  - kind: ServiceAccount
-    name: vault
-    namespace: vault
@@ -3,4 +3,4 @@ apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization

 resources:
-  - priorityclasses.yaml
+  - ../../../base/open-webui
@@ -1,6 +0,0 @@
---
-apiVersion: kustomize.config.k8s.io/v1beta1
-kind: Kustomization
-
-resources:
-  - ../../../base/priority-classes
@@ -40,7 +40,9 @@ server:
          }
        }

-        service_registration "kubernetes" {}
+        service_registration "consul" {
+          address = "consul-server.consul.svc.cluster.local:8500"
+        }

  dataStorage:
    enabled: true
@@ -48,14 +50,6 @@ server:
    storageClass: cephrbd-fast-delete
    accessMode: ReadWriteOnce

-  extraEnv:
-    - name: VAULT_K8S_NAMESPACE
-      value: vault
-    - name: VAULT_K8S_POD_NAME
-      valueFrom:
-        fieldRef:
-          fieldPath: metadata.name
-
  statefulSet:
    securityContext:
      container:
@@ -2,7 +2,6 @@ agent:
  replicaCount: 3
  env:
    WOODPECKER_MAX_WORKFLOWS: "8"
-    WOODPECKER_BACKEND_K8S_PRIORITY_CLASS: power
    WOODPECKER_BACKEND_K8S_STORAGE_CLASS: cephrbd-fast-delete
    WOODPECKER_BACKEND_K8S_VOLUME_SIZE: 10G
    WOODPECKER_BACKEND_K8S_STORAGE_RWX: false
@@ -11,6 +11,7 @@ spec:
      revision: HEAD
      directories:
      - path: apps/overlays/*/litellm
+      - path: apps/overlays/*/open-webui
      - path: apps/overlays/*/paperclip
  template:
    metadata:
@@ -22,7 +22,6 @@ spec:
      - path: apps/overlays/*/jfrog
      - path: apps/overlays/*/kanidm
      - path: apps/overlays/*/node-feature-discovery
-      - path: apps/overlays/*/priority-classes
      - path: apps/overlays/*/puppet
      - path: apps/overlays/*/purelb
      - path: apps/overlays/*/reflector-system
@@ -11,6 +11,8 @@ spec:
  destinations:
    - namespace: 'litellm'
      server: https://kubernetes.default.svc
+    - namespace: 'open-webui'
+      server: https://kubernetes.default.svc
    - namespace: 'paperclip'
      server: https://kubernetes.default.svc
  clusterResourceWhitelist:
@@ -31,8 +31,6 @@ spec:
      server: https://kubernetes.default.svc
    - namespace: 'node-feature-discovery'
      server: https://kubernetes.default.svc
-    - namespace: 'priority-classes'
-      server: https://kubernetes.default.svc
    - namespace: 'purelb'
      server: https://kubernetes.default.svc
    - namespace: 'puppet'
Author	SHA1	Message	Date
unkinben	3d85105afd	feat(open-webui): HA deployment with CNPG, PDB, and session persistence ci/woodpecker/pr/pre-commit Pipeline was successful Details ci/woodpecker/pr/kubeconform Pipeline failed Details - Switch from SQLite/PVC to CNPG PostgreSQL (3 instances, low-resource) with a transaction-mode PgBouncer pooler (2 instances) - Raise open-webui replicas to 3 with priorityClassName: power - Add PodDisruptionBudget (minAvailable: 1) - Add Gateway API sessionPersistence (cookie) on the HTTPS HTTPRoute so WebSocket connections stick to the same backend pod - Add postgres-credentials VaultStaticSecret; DATABASE_URL must be added to kv/kubernetes/namespace/open-webui/default/open-webui-credentials	2026-05-26 23:37:10 +10:00
unkinben	85a8cfe47d	fix(open-webui): use traefik-internal gateway for chat hostname ci/woodpecker/pr/pre-commit Pipeline was successful Details ci/woodpecker/pr/kubeconform Pipeline was successful Details	2026-05-26 23:26:01 +10:00
unkinben	16dabbbf8d	fix(open-webui): use litellm external hostname as OPENAI_API_BASE_URL ci/woodpecker/pr/pre-commit Pipeline was canceled Details ci/woodpecker/pr/kubeconform Pipeline was canceled Details	2026-05-26 23:25:26 +10:00
unkinben	1bcb88d3dd	feat(open-webui): deploy Open WebUI with litellm backend ci/woodpecker/pr/pre-commit Pipeline was successful Details ci/woodpecker/pr/kubeconform Pipeline was successful Details Deploys Open WebUI (chat.k8s.syd1.au.unkin.net) into the open-webui namespace via the aitooling ArgoCD project. Uses SQLite with a 10Gi cephrbd PVC for persistence, routes model requests to the existing litellm deployment, and exposes the UI through the traefik-external gateway. Credentials (OPENAI_API_KEY, WEBUI_SECRET_KEY) are injected via VaultStaticSecret from kv/kubernetes/namespace/open-webui/default. Closes #155	2026-05-26 00:11:25 +10:00