feat(open-webui): HA deployment with CNPG, PDB, and session persistence

- Switch from SQLite/PVC to CNPG PostgreSQL (3 instances, low-resource)
  with a transaction-mode PgBouncer pooler (2 instances)
- Raise open-webui replicas to 3 with priorityClassName: power
- Add PodDisruptionBudget (minAvailable: 1)
- Add Gateway API sessionPersistence (cookie) on the HTTPS HTTPRoute
  so WebSocket connections stick to the same backend pod
- Add postgres-credentials VaultStaticSecret; DATABASE_URL must be
  added to kv/kubernetes/namespace/open-webui/default/open-webui-credentials

.claude/commands/solve-issue.md

@@ -1,273 +0,0 @@
----
-description: Pull master, read open issues, pick one, branch, implement, test, commit, PR, and comment.
----
-
-# Solve a Gitea Issue
-
-## Current repo state
-
-```!
-git status --short
-echo "Current branch: $(git branch --show-current)"
-echo "Remote: $(git remote get-url origin 2>/dev/null || echo 'none')"
-```
-
-## Open issues (with full body)
-
-```!
-echo "Fetching open issues..."
-issue_ids=$(tea issues list --output simple 2>/dev/null | awk 'NF && $1 ~ /^[0-9]+$/ {print $1}')
-if [ -z "$issue_ids" ]; then
-  echo "No open issues found (or tea is not logged in)."
-else
-  for id in $issue_ids; do
-    echo ""
-    echo "══════════════════════════════════════"
-    tea issues view "$id" --fields index,title,body 2>/dev/null \
-      || tea issue "$id" 2>/dev/null \
-      || echo "  (could not read issue #$id)"
-    echo "══════════════════════════════════════"
-  done
-fi
-```
-
----
-
-## Your task
-
-Follow these steps **in order**. Do not skip steps.
-
-### 1 — Choose an issue
-
-Present the issues above to the user as a numbered list (index, one-line title). Ask which one to work on. Wait for the answer before continuing.
-
-### 2 — Sync master
-
-```bash
-git checkout master
-git pull
-```
-
-Confirm you are on master and up to date.
-
-### 3 — Create a branch
-
-Name the branch `benvin/issue-<N>-<short-slug>` where `<short-slug>` is 2–4 kebab-case words from the issue title.
-
-```bash
-git checkout -b benvin/issue-<N>-<slug>
-```
-
-### 4 — Read the issue in full
-
-Re-read the full issue body shown above. If any part is ambiguous, state your interpretation before coding.
-
-**If you discover other problems while working:** do NOT solve them inline. Create a new Gitea issue with `tea issues create --title "..." --description "..."` and stay focused on the assigned issue.
-
-### 5 — Implement the solution
-
-Make the code changes needed to resolve the issue. Follow the conventions already in the repo:
-- `main.py` route handlers each contain a single function call; logic lives in submodules.
-- No comments unless the WHY is non-obvious.
-- No new files unless the issue or architecture requires it.
-- Security: no command injection, XSS, SQL injection, or secrets in code.
-- **For performance improvements:** implement at the most generic call site possible so the fix applies to all current and future implementations, not just the one being tested.
-
-### 6 — Update tests
-
-Add or update tests that cover the new behaviour. Tests live in `tests/`. Check existing test structure before writing new ones — mirror the style and fixture patterns already in use.
-
-### 7 — Update README
-
-If the feature introduces new config keys, endpoints, or user-facing behaviour, document it in `README.md`. Keep additions concise — follow the existing section style.
-
-### 8 — Run the full test suite
-
-```bash
-make test
-```
-
-All tests must pass. If any fail, fix them before proceeding. Do not skip or suppress failing tests.
-
-### 9 — Live Docker test (new package type only)
-
-**Skip this step if the issue does not add a new remote package type.**
-
-If the issue adds a new package type (e.g. `deb`, `conda`, `cargo`, `rubygems`, or any type not already in `remotes.yaml`), do the following before committing.
-
-#### 9a — Add a real test remote to remotes.yaml
-
-Append a valid, publicly accessible remote of the new type to `remotes.yaml`. Use a real upstream URL and patterns that cover both an immutable file (versioned artifact) and a mutable file (index/metadata). Add a comment explaining which URLs to use for manual testing.
-
-#### 9b — Start the stack
-
-```bash
-make docker-up
-```
-
-Wait until `curl -s http://localhost:8000/health` returns `{"status":"healthy"}`.
-
-#### 9c — Test a mutable file (first fetch — cache miss)
-
-Download the index or metadata file for the new remote. Confirm:
-- HTTP 200
-- `X-Artifact-Source: remote` header (or equivalent log line confirming a cache miss)
-- Content looks correct (not empty, not an error page)
-
-```bash
-curl -sv "http://localhost:8000/api/v1/remote/<new-remote>/<mutable-path>" 2>&1 | grep -E "< HTTP|X-Artifact"
-```
-
-#### 9d — Test a mutable file (second fetch — cache hit)
-
-Repeat the exact same request. Confirm:
-- HTTP 200
-- `X-Artifact-Source: cache`
-
-```bash
-curl -sv "http://localhost:8000/api/v1/remote/<new-remote>/<mutable-path>" 2>&1 | grep -E "< HTTP|X-Artifact"
-```
-
-#### 9e — Test an immutable file (first fetch — cache miss)
-
-Download a versioned/immutable artifact. Confirm HTTP 200 and a cache-miss log line.
-
-```bash
-curl -sv "http://localhost:8000/api/v1/remote/<new-remote>/<immutable-path>" 2>&1 | grep -E "< HTTP|X-Artifact"
-```
-
-#### 9f — Test an immutable file (second fetch — cache hit)
-
-Repeat. Confirm `X-Artifact-Source: cache`.
-
-#### 9g — Check container logs
-
-```bash
-make docker-logs
-```
-
-Scan for:
-- `Cache MISS` on first fetches, `Cache HIT` on second fetches
-- `Cache ADD SUCCESS` with correct sizes
-- No unhandled exceptions or ERROR lines
-
-#### 9h — Exercise package-type tooling against the proxy
-
-Use the native tooling for this package type to verify end-to-end behaviour. Examples:
-
-| Package type | Command |
-|---|---|
-| `pypi` | `uv run --index-url http://localhost:8000/api/v1/remote/<remote>/simple <tool>` |
-| `npm` | `npm install --registry http://localhost:8000/api/v1/remote/<remote>/ <pkg>` |
-| `helm` | `helm repo add test http://localhost:8000/api/v1/remote/<remote> && helm search repo test && helm template test/<chart>` |
-| `alpine` | `apk fetch --repository http://localhost:8000/api/v1/remote/<remote>/<branch>/<arch> <pkg>` |
-| `rpm` | `dnf install --repofrompath ... <pkg>` or `repoquery` |
-| `generic` | `curl` / `wget` as appropriate |
-
-Confirm the tool resolves and downloads correctly through the proxy.
-
-#### 9i — Tear down
-
-```bash
-make docker-down
-```
-
-Fix any failures found during 9b–9h before moving on.
-
-### 9.5 — Performance issues: measure before/after and gate the PR
-
-**Skip this step if the issue is not a performance improvement.**
-
-For performance issues, a PR is only warranted if there is a measurable gain. Use the Docker stack to compare before and after.
-
-#### 9.5a — Baseline measurement (before)
-
-Start the stack with the **unmodified** code (temporarily revert your change):
-
-```bash
-make docker-up
-```
-
-Warm or clear the cache as appropriate, then measure the relevant metric — e.g. concurrent request latency during a slow operation, response time for a specific endpoint, or throughput. Record the numbers.
-
-#### 9.5b — Apply your change and rebuild
-
-```bash
-make docker-up   # rebuilds the image
-```
-
-Repeat exactly the same measurement. Record the numbers.
-
-#### 9.5c — Decide
-
-If the improvement is not clearly measurable, **do not open a PR**. Instead:
-1. Update the issue with your findings.
-2. Note any conditions under which the improvement would be observable.
-3. Skip steps 11–14.
-
-If the improvement is clear, proceed with the commit and PR. Include the before/after numbers in the PR description and the issue comment.
-
-#### 9.5d — Tear down
-
-```bash
-make docker-down
-```
-
-### 10 — Build the wheel (smoke check)
-
-```bash
-uv build --wheel
-```
-
-Confirm the build succeeds.
-
-### 11 — Stage and commit
-
-Stage only the files you changed. Do not use `git add -A` or `git add .` — list files explicitly. Run:
-
-```bash
-git add <file1> <file2> ...
-git commit
-```
-
-The commit message must:
-- Start with a conventional-commit prefix (`feat:`, `fix:`, `refactor:`, `chore:`, etc.)
-- Summarise the change in ≤ 72 characters on the first line
-- Optionally include a short body explaining *why* (not *what*)
-
-If the pre-commit hook auto-fixes files, re-stage the fixed files and commit again.
-
-### 12 — Push the branch
-
-```bash
-git push origin <branch-name>
-```
-
-### 13 — Open a pull request
-
-```bash
-tea pulls create \
-  --base master \
-  --head <branch-name> \
-  --title "<same as commit subject>" \
-  --description "Closes #<N>\n\n## Summary\n<bullet points>\n\n## Test plan\n<what was verified>"
-```
-
-### 14 — Comment on the issue
-
-```bash
-tea comment <N> "<resolution comment>"
-```
-
-The comment must cover:
-- **How it was resolved** — what changed and why
-- **Issues encountered** — any non-obvious problems hit during implementation
-- **Potential future improvements** — what could be done next
-
-### 15 — Return to master
-
-```bash
-git checkout master
-```
-
-Report the PR URL and a one-sentence summary to the user.

.woodpecker/kubeconform.yaml

@@ -3,7 +3,7 @@ when:
 
 steps:
   - name: kubeconform
-    image: git.unkin.net/unkin/almalinux9-kubetest:20260606
+    image: git.unkin.net/unkin/almalinux9-kubetest:20260319
     commands:
       - make kubeconform
     backend_options:

.woodpecker/pre-commit.yaml

@@ -3,7 +3,7 @@ when:
 
 steps:
   - name: pre-commit
-    image: git.unkin.net/unkin/almalinux9-base:20260606
+    image: git.unkin.net/unkin/almalinux9-base:20260308
     commands:
       - uvx pre-commit run --all-files
     backend_options:

AGENTS.md

@@ -1,261 +0,0 @@
-# AGENTS.md
-
-## Project Overview
-
-This is an **ArgoCD GitOps repository** that manages Kubernetes applications for the `au-syd1` cluster using a Kustomize + Helm pattern. Applications are deployed via ArgoCD ApplicationSets that watch directory patterns in this repo.
-
-The migration pattern for this repo is: **Terragrunt/Terraform → ArgoCD** (see `migration.md` for full guide).
-
----
-
-## Essential Commands
-
-```bash
-# Build and render manifests for a path (outputs to manifests/<path>/)
-make build apps/overlays/au-syd1/<app-name>
-make build clusters/au-syd1/bootstrap
-
-# Validate all apps and clusters with kubeconform
-make kubeconform
-
-# Clean generated manifests
-make clean
-
-# Quick build + inspect without persisting output
-kustomize build --enable-helm apps/overlays/au-syd1/<app-name>
-
-# Check all resource kinds produced by an overlay
-kustomize build --enable-helm apps/overlays/au-syd1/<app-name> | grep "^kind:" | sort | uniq -c
-
-# Run pre-commit checks against all files
-uvx pre-commit run --all-files
-```
-
----
-
-## Directory Structure
-
-```
-argocd-apps/
-├── argocd/
-│   ├── applicationsets/     # ArgoCD ApplicationSet definitions (platform.yaml, storage.yaml)
-│   └── projects/            # ArgoCD AppProject definitions (platform.yaml, storage.yaml)
-├── apps/
-│   ├── base/                # Base Kustomize resources per app (no cluster-specific config)
-│   │   └── <app-name>/
-│   │       ├── kustomization.yaml
-│   │       ├── namespace.yaml
-│   │       ├── vaultauth.yaml        # (if Vault-managed secrets)
-│   │       └── vaultstaticsecret.yaml
-│   └── overlays/
-│       └── au-syd1/         # Cluster-specific overlays
-│           └── <app-name>/
-│               ├── kustomization.yaml  # references base + helmCharts
-│               └── values.yaml         # Helm values for this cluster
-├── clusters/
-│   └── au-syd1/
-│       ├── apps/            # Entry point: references apps/base (ArgoCD app-of-apps)
-│       └── bootstrap/       # ArgoCD install + initial Application manifest
-├── ci/
-│   ├── validate-apps.sh     # kubeconform over apps/overlays/*/kustomization.yaml
-│   ├── validate-clusters.sh # kubeconform over clusters/*/kustomization.yaml
-│   └── validate-no-secrets.sh # pre-commit hook: blocks plain Kubernetes Secrets
-└── sources/                 # Reference sources (Terraform configs, upstream charts, etc.)
-    └── terraform-k8s/       # Original Terraform configs — reference when migrating
-```
-
----
-
-## Adding a New Application
-
-Follow these 10 steps (detailed in `migration.md`):
-
-### 1. Create base resources
-```
-apps/base/<app-name>/
-├── kustomization.yaml
-├── namespace.yaml
-├── vaultauth.yaml          # if needed
-└── vaultstaticsecret.yaml  # if needed
-```
-
-### 2. Create cluster overlay
-```
-apps/overlays/au-syd1/<app-name>/
-├── kustomization.yaml
-└── values.yaml
-```
-
-**Overlay kustomization.yaml pattern:**
-```yaml
----
-apiVersion: kustomize.config.k8s.io/v1beta1
-kind: Kustomization
-
-resources:
-  - ../../../base/<app-name>
-
-helmCharts:
-  - name: <chart-name>
-    repo: <helm-repo-url>
-    version: "<version>"
-    releaseName: <release-name>
-    namespace: <namespace>
-    valuesFile: values.yaml
-```
-
-### 3. Register in ApplicationSet
-Add a directory entry to `argocd/applicationsets/platform.yaml` (or `storage.yaml` for `csi-*` apps):
-```yaml
-- path: apps/overlays/*/<app-name>
-```
-
-### 4. Update AppProject
-In `argocd/projects/platform.yaml` (or `storage.yaml`):
-- Add the Helm repo URL to `sourceRepos`
-- Add the namespace to `destinations`
-- Add any required cluster-scoped resource types to `clusterResourceWhitelist`
-
-### 5. Validate
-```bash
-kustomize build --enable-helm apps/overlays/au-syd1/<app-name>
-make kubeconform
-```
-
----
-
-## Secret Management
-
-**Plain Kubernetes `Secret` objects are blocked** by the pre-commit hook. Use Vault Operator CRDs instead:
-
-### VaultAuth template
-```yaml
-apiVersion: secrets.hashicorp.com/v1beta1
-kind: VaultAuth
-metadata:
-  name: default
-  namespace: <namespace>
-spec:
-  method: kubernetes
-  mount: k8s/au/syd1
-  vaultConnectionRef: vso-system/default
-  allowedNamespaces:
-    - <namespace>
-  kubernetes:
-    role: <role>
-    serviceAccount: <service-account>
-    audiences:
-      - vault
-    tokenExpirationSeconds: 600
-```
-
-### VaultStaticSecret template
-```yaml
-apiVersion: secrets.hashicorp.com/v1beta1
-kind: VaultStaticSecret
-metadata:
-  name: <secret-name>
-  namespace: <namespace>
-spec:
-  vaultAuthRef: default
-  mount: kv
-  type: kv-v2
-  path: kubernetes/namespace/<namespace>/default/<secret-name>
-  refreshAfter: 5m
-  destination:
-    name: <k8s-secret-name>
-    create: true
-    overwrite: true
-  hmacSecretData: true
-```
-
----
-
-## YAML Conventions
-
-- **2-space indentation** (enforced by yamllint)
-- All files must end with a newline (`end-of-file-fixer`)
-- No trailing whitespace
-- YAML linting uses relaxed rules with `line-length: disable` (long base64/URLs are fine)
-- yamllint ignores `chart` directories (vendored Helm charts)
-- `---` document separator at top of every YAML file
-- Multiple documents in one file are allowed (e.g., `vaultstaticsecret.yaml` often contains multiple secrets)
-
----
-
-## Kubernetes Labels Pattern
-
-Use standard `app.kubernetes.io/*` labels consistently:
-```yaml
-labels:
-  app.kubernetes.io/component: <component>
-  app.kubernetes.io/instance: <release-name>
-  app.kubernetes.io/name: <app-name>
-  app.kubernetes.io/version: <version>
-```
-
----
-
-## Resource Naming Conventions
-
-Files in `apps/base/<app-name>/` follow the pattern:
-```
-<kind>_<name>.yaml
-```
-Examples:
-- `deployment_puppetserver-master.yaml`
-- `cronjob_g10k-code.yaml`
-- `configmap_puppetboard-config.yaml`
-- `horizontalpodautoscaler_puppetserver-compilers-autoscaler.yaml`
-- `service_puppet-headless.yaml`
-
----
-
-## Helm Chart Vendoring
-
-Some overlays vendor Helm charts locally under `apps/overlays/au-syd1/<app-name>/charts/<chart-name>/`. When a chart is vendored, the overlay's `kustomization.yaml` references the local path. When not vendored, it references the OCI or HTTP repo directly.
-
-Current Kubernetes target version: **1.33.7** (used by kubeconform in CI).
-
----
-
-## Project Boundaries
-
-| Project    | ApplicationSet            | App pattern              |
-|------------|---------------------------|--------------------------|
-| `platform` | `argocd/applicationsets/platform.yaml` | Named apps (cert-manager, puppet, woodpecker, etc.) |
-| `storage`  | `argocd/applicationsets/storage.yaml`  | `csi-*` apps             |
-
-The `clusters/au-syd1/apps/` entry-point is deployed as a standalone ArgoCD `Application` (not an ApplicationSet) called `au-syd1-apps`.
-
----
-
-## CI / Pre-commit Hooks
-
-Runs on every PR via Woodpecker CI (`.woodpecker/`):
-
-| Check | Tool | Trigger |
-|---|---|---|
-| YAML lint + general file checks | `pre-commit` (yamllint + pre-commit-hooks) | PR |
-| No plain Secrets | `ci/validate-no-secrets.sh` | PR (staged files) |
-| Kubernetes manifest validation | `kubeconform` via `make kubeconform` | PR |
-
-kubeconform skips: `CustomResourceDefinition`, `GpuDevicePlugin` (for apps validation).
-
----
-
-## Git Workflow
-
-- Branch naming: `benvin/<app-name>` (user prefix)
-- **Never `git add .`** — add only relevant files explicitly
-- If pre-commit modifies files, `git add -u` then `git commit --amend --no-edit`
-- Use `git push --force-with-lease` after amending
-
----
-
-## Security Policies
-
-- `reloader.stakater.com/auto: "true"` annotation triggers rolling restarts on ConfigMap/Secret changes
-- Security contexts follow least-privilege: `drop: [all]` then add only required capabilities
-- `fsGroup: 999` on pod security context for Puppet workloads
-- `runAsUser: 0` is used only for init containers that need to set file permissions, then regular containers run as non-root

apps/base/artifactapi/api-deployment.yaml

@@ -1,91 +0,0 @@
----
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: api
-  namespace: artifactapi
-  annotations:
-    reloader.stakater.com/auto: "true"
-spec:
-  selector:
-    matchLabels:
-      app: api
-  strategy:
-    rollingUpdate:
-      maxUnavailable: 1
-    type: RollingUpdate
-  template:
-    metadata:
-      labels:
-        app: api
-    spec:
-      automountServiceAccountToken: true
-      initContainers:
-        - name: combine-certs
-          image: alpine:3
-          command:
-            - sh
-            - -c
-            - cat /etc/ssl/certs/ca-certificates.crt /custom-ca/ca.crt > /combined-certs/ca-certificates.crt
-          volumeMounts:
-            - name: vault-ca-cert
-              mountPath: /custom-ca
-              readOnly: true
-            - name: combined-certs
-              mountPath: /combined-certs
-      containers:
-        - name: api
-          image: git.unkin.net/unkin/artifactapi:v3.5.0
-          imagePullPolicy: IfNotPresent
-          ports:
-            - containerPort: 8000
-              name: http
-              protocol: TCP
-          envFrom:
-            - configMapRef:
-                name: api-env
-                optional: false
-            - secretRef:
-                name: environment2
-                optional: false
-          volumeMounts:
-            - name: combined-certs
-              mountPath: /etc/ssl/combined
-              readOnly: true
-          livenessProbe:
-            failureThreshold: 3
-            httpGet:
-              path: /health
-              port: http
-              scheme: HTTP
-            initialDelaySeconds: 30
-            periodSeconds: 30
-            successThreshold: 1
-            timeoutSeconds: 5
-          readinessProbe:
-            failureThreshold: 3
-            httpGet:
-              path: /health
-              port: http
-              scheme: HTTP
-            initialDelaySeconds: 10
-            periodSeconds: 5
-            successThreshold: 1
-            timeoutSeconds: 5
-          resources:
-            limits:
-              cpu: "1"
-              memory: 4Gi
-            requests:
-              cpu: 100m
-              memory: 256Mi
-      volumes:
-        - name: vault-ca-cert
-          secret:
-            secretName: vault-ca-cert
-            items:
-              - key: ca.crt
-                path: ca.crt
-        - name: combined-certs
-          emptyDir: {}
-      restartPolicy: Always

apps/base/artifactapi/api-hpa.yaml

@@ -1,41 +0,0 @@
----
-apiVersion: autoscaling/v2
-kind: HorizontalPodAutoscaler
-metadata:
-  name: api-hpa
-  namespace: artifactapi
-spec:
-  scaleTargetRef:
-    apiVersion: apps/v1
-    kind: Deployment
-    name: api
-  minReplicas: 2
-  maxReplicas: 10
-  metrics:
-    - type: Resource
-      resource:
-        name: cpu
-        target:
-          type: Utilization
-          averageUtilization: 60
-  behavior:
-    scaleUp:
-      stabilizationWindowSeconds: 0
-      selectPolicy: Max
-      policies:
-        - type: Percent
-          value: 100
-          periodSeconds: 30
-        - type: Pods
-          value: 4
-          periodSeconds: 30
-    scaleDown:
-      stabilizationWindowSeconds: 300
-      selectPolicy: Min
-      policies:
-        - type: Percent
-          value: 10
-          periodSeconds: 60
-        - type: Pods
-          value: 2
-          periodSeconds: 60

apps/base/artifactapi/configmap.yaml

@@ -25,20 +25,3 @@ metadata:
 data:
   POSTGRES_DB: artifacts
   POSTGRES_USER: artifacts
-###
----
-apiVersion: v1
-kind: ConfigMap
-metadata:
-  name: api-env
-  namespace: artifactapi
-data:
-  DBHOST: postgres-pooler
-  DBNAME: artifacts
-  DBPORT: "5432"
-  DBUSER: artifacts
-  MINIO_BUCKET: artifactapi-prod-k8s-syd1-au
-  MINIO_ENDPOINT: radosgw.service.consul
-  MINIO_SECURE: "true"
-  REDIS_URL: redis://redis:6379
-  SSL_CERT_FILE: /etc/ssl/combined/ca-certificates.crt

apps/base/artifactapi/kustomization.yaml

@@ -5,28 +5,16 @@ kind: Kustomization
 resources:
   - artifactapi-deployment.yaml
   - artifactapi-hpa.yaml
-  - httproute.yaml
-  - postgres-deployment.yaml
-  - redis-deployment.yaml
-  - pvc.yaml
-  # shared
   - configmap.yaml
-  - services.yaml
   - gateway.yaml
+  - httproute.yaml
   - namespace.yaml
+  - postgres-deployment.yaml
+  - pvc.yaml
+  - redis-deployment.yaml
+  - services.yaml
   - vaultauth.yaml
   - vaultstaticsecret.yaml
-  # new
-  - api-deployment.yaml
-  - api-hpa.yaml
-  - cnpg_cluster.yaml
-  - cnpg_pooler.yaml
-  - ui-deployment.yaml
-  - ui-hpa.yaml
-  # to copy still
-  # - httproute.yaml
-  # - redis-deployment.yaml
-
 
 configMapGenerator:
   - name: remotes-config

apps/base/artifactapi/resources/conf.d/remote-docker.yaml

@@ -6,7 +6,6 @@ remotes:
     immutable_patterns:
       - "^cloudnative-pg/cloudnative-pg"
       - "^emberstack/helm-charts"
-      - "^open-webui/open-webui"
       - "^openvoxproject/"
       - "^stakater/reloader"
       - "^stalwartlabs/stalwart"

apps/base/artifactapi/resources/conf.d/remote-generic.yaml

@@ -81,8 +81,6 @@ remotes:
     description: "Gitea download site"
     immutable_patterns:
       - "act_runner/.*/act_runner-.*-linux-amd64$"
-      - "gitea/.*/gitea-.*-linux-amd64$"
-      - "gitea/.*/gitea-.*-linux-amd64.xz$"
       - "tea/.*/tea-.*-linux-amd64$"
     cache:
       immutable_ttl: 0

apps/base/artifactapi/resources/conf.d/remote-rpm.yaml

@@ -17,24 +17,6 @@ remotes:
       immutable_ttl: 0
       mutable_ttl: 7200
 
-  almalinux-vault:
-    base_url: "https://vault.almalinux.org"
-    package: "rpm"
-    description: "AlmaLinux Vault RPM package repository"
-    immutable_patterns:
-      - ".*/x86_64/.*\\.rpm$"
-      - ".*/noarch/.*\\.rpm$"
-      - ".*/repodata/.*\\.sqlite.*$"
-      - ".*/repodata/.*\\.xml.*$"
-      - ".*/repodata/.*\\.yaml.*$"
-      - ".*/install.img"
-      - ".*/squashfs.img"
-      - ".*/updates.img"
-      - ".*/RPM-GPG-KEY-.*$"
-    cache:
-      immutable_ttl: 0
-      mutable_ttl: 7200
-
   ceph-reef:
     base_url: "https://download.ceph.com/rpm-reef/"
     package: "rpm"

apps/base/artifactapi/services.yaml

@@ -49,55 +49,3 @@ spec:
     app: redis
   sessionAffinity: None
   type: ClusterIP
-####
----
-apiVersion: v1
-kind: Service
-metadata:
-  name: artifactapi
-  namespace: artifactapi
-spec:
-  internalTrafficPolicy: Cluster
-  ports:
-    - name: http
-      port: 80
-      protocol: TCP
-      targetPort: http
-  selector:
-    app: api
-  sessionAffinity: None
-  type: ClusterIP
----
-apiVersion: v1
-kind: Service
-metadata:
-  name: ui
-  namespace: artifactapi
-spec:
-  internalTrafficPolicy: Cluster
-  ports:
-    - name: http
-      port: 80
-      protocol: TCP
-      targetPort: http
-  selector:
-    app: ui
-  sessionAffinity: None
-  type: ClusterIP
----
-apiVersion: v1
-kind: Service
-metadata:
-  name: redis
-  namespace: artifactapi
-spec:
-  internalTrafficPolicy: Cluster
-  ports:
-    - name: redis
-      port: 6379
-      protocol: TCP
-      targetPort: redis
-  selector:
-    app: redis
-  sessionAffinity: None
-  type: ClusterIP

apps/base/artifactapi/ui-deployment.yaml

@@ -1,58 +0,0 @@
----
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: ui
-  namespace: artifactapi
-  annotations:
-    reloader.stakater.com/auto: "true"
-spec:
-  selector:
-    matchLabels:
-      app: ui
-  strategy:
-    rollingUpdate:
-      maxUnavailable: 1
-    type: RollingUpdate
-  template:
-    metadata:
-      labels:
-        app: ui
-    spec:
-      automountServiceAccountToken: true
-      containers:
-        - name: ui
-          image: git.unkin.net/unkin/artifactapi-ui:v3.5.0
-          imagePullPolicy: IfNotPresent
-          ports:
-            - containerPort: 80
-              name: http
-              protocol: TCP
-          livenessProbe:
-            failureThreshold: 3
-            httpGet:
-              path: /
-              port: http
-              scheme: HTTP
-            initialDelaySeconds: 15
-            periodSeconds: 30
-            successThreshold: 1
-            timeoutSeconds: 5
-          readinessProbe:
-            failureThreshold: 3
-            httpGet:
-              path: /
-              port: http
-              scheme: HTTP
-            initialDelaySeconds: 5
-            periodSeconds: 5
-            successThreshold: 1
-            timeoutSeconds: 5
-          resources:
-            limits:
-              cpu: 500m
-              memory: 512Mi
-            requests:
-              cpu: 50m
-              memory: 128Mi
-      restartPolicy: Always

apps/base/artifactapi/ui-hpa.yaml

@@ -1,41 +0,0 @@
----
-apiVersion: autoscaling/v2
-kind: HorizontalPodAutoscaler
-metadata:
-  name: ui-hpa
-  namespace: artifactapi
-spec:
-  scaleTargetRef:
-    apiVersion: apps/v1
-    kind: Deployment
-    name: ui
-  minReplicas: 2
-  maxReplicas: 10
-  metrics:
-    - type: Resource
-      resource:
-        name: cpu
-        target:
-          type: Utilization
-          averageUtilization: 60
-  behavior:
-    scaleUp:
-      stabilizationWindowSeconds: 0
-      selectPolicy: Max
-      policies:
-        - type: Percent
-          value: 100
-          periodSeconds: 30
-        - type: Pods
-          value: 4
-          periodSeconds: 30
-    scaleDown:
-      stabilizationWindowSeconds: 300
-      selectPolicy: Min
-      policies:
-        - type: Percent
-          value: 10
-          periodSeconds: 60
-        - type: Pods
-          value: 2
-          periodSeconds: 60

apps/base/artifactapi/vaultauth.yaml

@@ -16,22 +16,3 @@ spec:
   method: kubernetes
   mount: k8s/au/syd1
   vaultConnectionRef: vso-system/default
-### change to default from default1
----
-apiVersion: secrets.hashicorp.com/v1beta1
-kind: VaultAuth
-metadata:
-  name: default1
-  namespace: artifactapi
-spec:
-  allowedNamespaces:
-    - artifactapi
-  kubernetes:
-    audiences:
-      - vault
-    role: default
-    serviceAccount: default
-    tokenExpirationSeconds: 600
-  method: kubernetes
-  mount: k8s/au/syd1
-  vaultConnectionRef: vso-system/default

apps/base/artifactapi/vaultstaticsecret.yaml

@@ -32,38 +32,3 @@ spec:
   refreshAfter: 5m
   type: kv-v2
   vaultAuthRef: default
-### change auth ref to default from default1
----
-apiVersion: secrets.hashicorp.com/v1beta1
-kind: VaultStaticSecret
-metadata:
-  name: postgres-credentials
-  namespace: artifactapi
-spec:
-  destination:
-    create: true
-    name: postgres-credentials
-    overwrite: true
-  hmacSecretData: true
-  mount: kv
-  path: kubernetes/namespace/artifactapi/default/postgres-credentials
-  refreshAfter: 5m
-  type: kv-v2
-  vaultAuthRef: default1
----
-apiVersion: secrets.hashicorp.com/v1beta1
-kind: VaultStaticSecret
-metadata:
-  name: environment2
-  namespace: artifactapi
-spec:
-  destination:
-    create: true
-    name: environment
-    overwrite: true
-  hmacSecretData: true
-  mount: kv
-  path: kubernetes/namespace/artifactapi/default/environment
-  refreshAfter: 5m
-  type: kv-v2
-  vaultAuthRef: default1

apps/base/kanidm/README.md

@@ -1,51 +1,32 @@
 # kanidm
 
-Three-replica kanidm identity server with Vault-managed replication certificates.
+Single-replica kanidm identity server deployment.
-
-## Architecture
-
-- Per-pod `server-N.toml` in `resources/` — each has its own replication origin hardcoded
-- `config-init` busybox init container copies the right config and injects peer certs from the
-  vault-synced `kanidm-repl-certs` Secret at pod startup
-- `reloader.stakater.com/auto: "true"` triggers a rolling restart when the ConfigMap or Secret changes
-- Vault path: `kv/kubernetes/namespace/kanidm/default/repl-certs`
-  - Keys: `kanidm-0`, `kanidm-1`, `kanidm-2` — each holds that pod's replication certificate
 
 ## Initial setup
 
-After the first pod starts, generate the admin credentials:
+After the pod starts for the first time, generate the admin and idm_admin credentials:
 
 ```bash
-kubectl exec -n kanidm kanidm-0 -- /sbin/kanidmd recover-account -c /config/server.toml admin
+kubectl exec -n kanidm kanidm-0 -- /sbin/kanidmd recover-account admin
-kubectl exec -n kanidm kanidm-0 -- /sbin/kanidmd recover-account -c /config/server.toml idm_admin
+kubectl exec -n kanidm kanidm-0 -- /sbin/kanidmd recover-account idm_admin
 ```
 
-## Replication certificate rotation
+## Adding replication
 
-When certs need to be renewed, update vault and reloader will roll the StatefulSet:
+If replication is needed in the future:
+
+1. Scale the StatefulSet to 3 replicas and add `podAntiAffinity` to spread across nodes.
+2. Add a `[replication]` section to `configmap.yaml` per pod (origin is pod-specific:
+   `repl://kanidm-N.kanidm-headless.kanidm.svc.cluster.local:8444`).
+3. Add the replication port (8444) back to the StatefulSet container ports and headless service.
+4. Restore `rbac.yaml` for the cert-publisher sidecar, or exchange certificates manually:
 
 ```bash
-# Get new cert from a pod
+# On each pod, get its replication certificate
-kubectl exec -it -n kanidm kanidm-N -- /sbin/kanidmd renew-replication-certificate -c /config/server.toml
+kubectl exec -n kanidm kanidm-0 -- /sbin/kanidmd renew-replication-certificate
 
-# Write updated cert to vault (reloader triggers restart automatically)
+# Add each peer's certificate to the other pods' configs under:
-vault kv patch kv/kubernetes/namespace/kanidm/default/repl-certs "kanidm-N=<cert>"
+# [replication."repl://<peer-fqdn>:8444"]
-```
+# type = "mutual-pull"
-
+# partner_cert = "<cert>"
-## Resolving domain UUID mismatch
-
-If pods initialized independently (each with a different domain UUID), replication will fail with
-`Consumer Domain UUID does not match`. Fix by resetting kanidm-1 and kanidm-2 to sync from
-kanidm-0 (the authoritative node):
-
-```bash
-# Scale down to avoid split-brain during reset
-kubectl scale statefulset -n kanidm kanidm --replicas=1
-
-# Delete the stale PVCs for the replica pods
-kubectl delete pvc -n kanidm data-kanidm-1 data-kanidm-2
-
-# Scale back up — replicas start with empty DBs and automatic_refresh=true
-# will trigger a full sync from kanidm-0 once TLS peer certs are verified
-kubectl scale statefulset -n kanidm kanidm --replicas=3
 ```

apps/base/kanidm/kustomization.yaml

@@ -5,8 +5,6 @@ kind: Kustomization
 resources:
   - namespace.yaml
   - serviceaccount.yaml
-  - vaultauth.yaml
-  - vaultstaticsecret.yaml
   - certificate.yaml
   - service.yaml
   - statefulset.yaml
@@ -25,3 +23,5 @@ configMapGenerator:
         app.kubernetes.io/instance: kanidm
     files:
       - server-0.toml=resources/server-0.toml
+      - server-1.toml=resources/server-1.toml
+      - server-2.toml=resources/server-2.toml

apps/base/kanidm/resources/server-0.toml

@@ -13,3 +13,15 @@ log_level = "info"
 path = "/data/backups/"
 schedule = "0 22 * * *"
 versions = 7
+
+[replication]
+origin = "repl://kanidm-0.kanidm-headless.kanidm.svc.cluster.local:8444"
+bindaddress = "[::]:8444"
+
+[replication."repl://kanidm-1.kanidm-headless.kanidm.svc.cluster.local:8444"]
+type = "mutual-pull"
+partner_cert = "MIIB-TCCAZ-gAwIBAgIRASqOpORz60wiv7wF_7oBOxQwCgYIKoZIzj0EAwIwTDEtMCsGA1UEAwwkMmE4ZWE0ZTQtNzNlYi00YzIyLWJmYmMtMDVmZmJhMDEzYjE0MRswGQYDVQQKDBJLYW5pZG0gUmVwbGljYXRpb24wHhcNMjYwNTI1MTMyODM5WhcNMzAwNTI1MTMyODM5WjBMMS0wKwYDVQQDDCQyYThlYTRlNC03M2ViLTRjMjItYmZiYy0wNWZmYmEwMTNiMTQxGzAZBgNVBAoMEkthbmlkbSBSZXBsaWNhdGlvbjBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABFQP3zpFRt7TCOhzrUpOJBojn-sC2LmqZUub8P2ymVdIQbmoAyh4Q8Me0hNWJFyuFDnnqO06dt5I2iv0910-X6KjYjBgMCAGA1UdJQEB_wQWMBQGCCsGAQUFBwMCBggrBgEFBQcDATA8BgNVHREENTAzgjFrYW5pZG0tMS5rYW5pZG0taGVhZGxlc3Mua2FuaWRtLnN2Yy5jbHVzdGVyLmxvY2FsMAoGCCqGSM49BAMCA0gAMEUCIGjl58U6apcDjMEPIca8Wwg_JMfuMvV-uVcJI49Gl_9GAiEA2tFdb9rnFeBI7mwysScf5UsmY3ZziMD3UVm1vWN2IKs"
+
+[replication."repl://kanidm-2.kanidm-headless.kanidm.svc.cluster.local:8444"]
+type = "mutual-pull"
+partner_cert = "MIIB-TCCAZ-gAwIBAgIRAeFGUAJbCkJ2vzf_Vv4qjeUwCgYIKoZIzj0EAwIwTDEtMCsGA1UEAwwkZTE0NjUwMDItNWIwYS00Mjc2LWJmMzctZmY1NmZlMmE4ZGU1MRswGQYDVQQKDBJLYW5pZG0gUmVwbGljYXRpb24wHhcNMjYwNTI1MTMyOTEwWhcNMzAwNTI1MTMyOTEwWjBMMS0wKwYDVQQDDCRlMTQ2NTAwMi01YjBhLTQyNzYtYmYzNy1mZjU2ZmUyYThkZTUxGzAZBgNVBAoMEkthbmlkbSBSZXBsaWNhdGlvbjBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABCrncHSbDNSV3_aOSZ14plbVfrvSXQQL9MOqvrDKlf_Q6WbcA8OrTUjs3Jt0Q2beWjC3Z5-5c9fGu8M_k2iVWf-jYjBgMCAGA1UdJQEB_wQWMBQGCCsGAQUFBwMCBggrBgEFBQcDATA8BgNVHREENTAzgjFrYW5pZG0tMi5rYW5pZG0taGVhZGxlc3Mua2FuaWRtLnN2Yy5jbHVzdGVyLmxvY2FsMAoGCCqGSM49BAMCA0gAMEUCIQDHY5Yl-bhDTuJaYnHSMSiSAEWPrDcRVzvfmOJukuJ1QQIgSwgyeSG3K0MY87DI1RDYAdZlpP1YOK3Yatj7-YSXPC0"

apps/base/kanidm/resources/server-1.toml

@@ -0,0 +1,27 @@
+version = "2"
+
+domain = "auth.unkin.net"
+origin = "https://auth.unkin.net"
+bindaddress = "[::]:8443"
+db_path = "/data/kanidm.db"
+db_arc_size = 2048
+tls_chain = "/data/tls/tls.crt"
+tls_key = "/data/tls/tls.key"
+log_level = "info"
+
+[online_backup]
+path = "/data/backups/"
+schedule = "0 22 * * *"
+versions = 7
+
+[replication]
+origin = "repl://kanidm-1.kanidm-headless.kanidm.svc.cluster.local:8444"
+bindaddress = "[::]:8444"
+
+[replication."repl://kanidm-0.kanidm-headless.kanidm.svc.cluster.local:8444"]
+type = "mutual-pull"
+partner_cert = "MIIB-jCCAZ-gAwIBAgIRAVKuoPDpF0IBnvFjCwdK41EwCgYIKoZIzj0EAwIwTDEtMCsGA1UEAwwkNTJhZWEwZjAtZTkxNy00MjAxLTllZjEtNjMwYjA3NGFlMzUxMRswGQYDVQQKDBJLYW5pZG0gUmVwbGljYXRpb24wHhcNMjYwNTI1MTMzNzQ5WhcNMzAwNTI1MTMzNzQ5WjBMMS0wKwYDVQQDDCQ1MmFlYTBmMC1lOTE3LTQyMDEtOWVmMS02MzBiMDc0YWUzNTExGzAZBgNVBAoMEkthbmlkbSBSZXBsaWNhdGlvbjBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABGejqjk0Eet-RILHI236wHYqISdnPlebqnkuUTh4W2mCzkmqKibyjxGIUOs8LBrUeTR2DxVR1VV6H2rYQk2wdROjYjBgMCAGA1UdJQEB_wQWMBQGCCsGAQUFBwMCBggrBgEFBQcDATA8BgNVHREENTAzgjFrYW5pZG0tMC5rYW5pZG0taGVhZGxlc3Mua2FuaWRtLnN2Yy5jbHVzdGVyLmxvY2FsMAoGCCqGSM49BAMCA0kAMEYCIQCSkFj2A-KVWv2tKJLFzb18J5eWWKtsvWewZTn-FVnRnQIhAKJbt84IoZ9oXxgfp0VOLyVZiAgUgwMFS6JOfno3D-Nw"
+
+[replication."repl://kanidm-2.kanidm-headless.kanidm.svc.cluster.local:8444"]
+type = "mutual-pull"
+partner_cert = "MIIB-TCCAZ-gAwIBAgIRAeFGUAJbCkJ2vzf_Vv4qjeUwCgYIKoZIzj0EAwIwTDEtMCsGA1UEAwwkZTE0NjUwMDItNWIwYS00Mjc2LWJmMzctZmY1NmZlMmE4ZGU1MRswGQYDVQQKDBJLYW5pZG0gUmVwbGljYXRpb24wHhcNMjYwNTI1MTMyOTEwWhcNMzAwNTI1MTMyOTEwWjBMMS0wKwYDVQQDDCRlMTQ2NTAwMi01YjBhLTQyNzYtYmYzNy1mZjU2ZmUyYThkZTUxGzAZBgNVBAoMEkthbmlkbSBSZXBsaWNhdGlvbjBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABCrncHSbDNSV3_aOSZ14plbVfrvSXQQL9MOqvrDKlf_Q6WbcA8OrTUjs3Jt0Q2beWjC3Z5-5c9fGu8M_k2iVWf-jYjBgMCAGA1UdJQEB_wQWMBQGCCsGAQUFBwMCBggrBgEFBQcDATA8BgNVHREENTAzgjFrYW5pZG0tMi5rYW5pZG0taGVhZGxlc3Mua2FuaWRtLnN2Yy5jbHVzdGVyLmxvY2FsMAoGCCqGSM49BAMCA0gAMEUCIQDHY5Yl-bhDTuJaYnHSMSiSAEWPrDcRVzvfmOJukuJ1QQIgSwgyeSG3K0MY87DI1RDYAdZlpP1YOK3Yatj7-YSXPC0"

apps/base/kanidm/resources/server-2.toml

@@ -0,0 +1,27 @@
+version = "2"
+
+domain = "auth.unkin.net"
+origin = "https://auth.unkin.net"
+bindaddress = "[::]:8443"
+db_path = "/data/kanidm.db"
+db_arc_size = 2048
+tls_chain = "/data/tls/tls.crt"
+tls_key = "/data/tls/tls.key"
+log_level = "info"
+
+[online_backup]
+path = "/data/backups/"
+schedule = "0 22 * * *"
+versions = 7
+
+[replication]
+origin = "repl://kanidm-2.kanidm-headless.kanidm.svc.cluster.local:8444"
+bindaddress = "[::]:8444"
+
+[replication."repl://kanidm-0.kanidm-headless.kanidm.svc.cluster.local:8444"]
+type = "mutual-pull"
+partner_cert = "MIIB-jCCAZ-gAwIBAgIRAVKuoPDpF0IBnvFjCwdK41EwCgYIKoZIzj0EAwIwTDEtMCsGA1UEAwwkNTJhZWEwZjAtZTkxNy00MjAxLTllZjEtNjMwYjA3NGFlMzUxMRswGQYDVQQKDBJLYW5pZG0gUmVwbGljYXRpb24wHhcNMjYwNTI1MTMzNzQ5WhcNMzAwNTI1MTMzNzQ5WjBMMS0wKwYDVQQDDCQ1MmFlYTBmMC1lOTE3LTQyMDEtOWVmMS02MzBiMDc0YWUzNTExGzAZBgNVBAoMEkthbmlkbSBSZXBsaWNhdGlvbjBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABGejqjk0Eet-RILHI236wHYqISdnPlebqnkuUTh4W2mCzkmqKibyjxGIUOs8LBrUeTR2DxVR1VV6H2rYQk2wdROjYjBgMCAGA1UdJQEB_wQWMBQGCCsGAQUFBwMCBggrBgEFBQcDATA8BgNVHREENTAzgjFrYW5pZG0tMC5rYW5pZG0taGVhZGxlc3Mua2FuaWRtLnN2Yy5jbHVzdGVyLmxvY2FsMAoGCCqGSM49BAMCA0kAMEYCIQCSkFj2A-KVWv2tKJLFzb18J5eWWKtsvWewZTn-FVnRnQIhAKJbt84IoZ9oXxgfp0VOLyVZiAgUgwMFS6JOfno3D-Nw"
+
+[replication."repl://kanidm-1.kanidm-headless.kanidm.svc.cluster.local:8444"]
+type = "mutual-pull"
+partner_cert = "MIIB-TCCAZ-gAwIBAgIRASqOpORz60wiv7wF_7oBOxQwCgYIKoZIzj0EAwIwTDEtMCsGA1UEAwwkMmE4ZWE0ZTQtNzNlYi00YzIyLWJmYmMtMDVmZmJhMDEzYjE0MRswGQYDVQQKDBJLYW5pZG0gUmVwbGljYXRpb24wHhcNMjYwNTI1MTMyODM5WhcNMzAwNTI1MTMyODM5WjBMMS0wKwYDVQQDDCQyYThlYTRlNC03M2ViLTRjMjItYmZiYy0wNWZmYmEwMTNiMTQxGzAZBgNVBAoMEkthbmlkbSBSZXBsaWNhdGlvbjBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABFQP3zpFRt7TCOhzrUpOJBojn-sC2LmqZUub8P2ymVdIQbmoAyh4Q8Me0hNWJFyuFDnnqO06dt5I2iv0910-X6KjYjBgMCAGA1UdJQEB_wQWMBQGCCsGAQUFBwMCBggrBgEFBQcDATA8BgNVHREENTAzgjFrYW5pZG0tMS5rYW5pZG0taGVhZGxlc3Mua2FuaWRtLnN2Yy5jbHVzdGVyLmxvY2FsMAoGCCqGSM49BAMCA0gAMEUCIGjl58U6apcDjMEPIca8Wwg_JMfuMvV-uVcJI49Gl_9GAiEA2tFdb9rnFeBI7mwysScf5UsmY3ZziMD3UVm1vWN2IKs"

apps/base/kanidm/service.yaml

@@ -9,10 +9,6 @@ metadata:
     app.kubernetes.io/instance: kanidm
 spec:
   type: ClusterIP
-  sessionAffinity: ClientIP
-  sessionAffinityConfig:
-    clientIP:
-      timeoutSeconds: 10800
   ports:
     - name: https
       port: 8443
@@ -38,6 +34,10 @@ spec:
       port: 8443
       targetPort: https
       protocol: TCP
+    - name: replication
+      port: 8444
+      targetPort: replication
+      protocol: TCP
   selector:
     app.kubernetes.io/name: kanidm
     app.kubernetes.io/instance: kanidm

apps/base/kanidm/statefulset.yaml

@@ -4,14 +4,12 @@ kind: StatefulSet
 metadata:
   name: kanidm
   namespace: kanidm
-  annotations:
-    reloader.stakater.com/auto: "true"
   labels:
     app.kubernetes.io/name: kanidm
     app.kubernetes.io/instance: kanidm
 spec:
   serviceName: kanidm-headless
-  replicas: 1
+  replicas: 3
   selector:
     matchLabels:
       app.kubernetes.io/name: kanidm
@@ -23,11 +21,39 @@ spec:
         app.kubernetes.io/instance: kanidm
     spec:
       serviceAccountName: kanidm
+      affinity:
+        podAntiAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+            - labelSelector:
+                matchLabels:
+                  app.kubernetes.io/name: kanidm
+                  app.kubernetes.io/instance: kanidm
+              topologyKey: kubernetes.io/hostname
       securityContext:
         runAsUser: 1000
         runAsGroup: 1000
         runAsNonRoot: true
         fsGroup: 1000
+      initContainers:
+        - name: config-init
+          image: busybox:1.36
+          command: ["/bin/sh", "-c"]
+          args:
+            - cp "/config-template/server-${POD_NAME##*-}.toml" /config/server.toml
+          env:
+            - name: POD_NAME
+              valueFrom:
+                fieldRef:
+                  fieldPath: metadata.name
+          volumeMounts:
+            - name: config-template
+              mountPath: /config-template
+              readOnly: true
+            - name: config
+              mountPath: /config
+          securityContext:
+            allowPrivilegeEscalation: false
+            readOnlyRootFilesystem: true
       containers:
         - name: kanidm
           image: kanidm/server:1.10.3
@@ -37,12 +63,14 @@ spec:
             - name: https
               containerPort: 8443
               protocol: TCP
+            - name: replication
+              containerPort: 8444
+              protocol: TCP
           volumeMounts:
             - name: data
               mountPath: /data
             - name: config
-              mountPath: /config/server.toml
+              mountPath: /config
-              subPath: server-0.toml
               readOnly: true
             - name: tls
               mountPath: /data/tls
@@ -68,9 +96,11 @@ spec:
             initialDelaySeconds: 30
             periodSeconds: 30
       volumes:
-        - name: config
+        - name: config-template
           configMap:
             name: kanidm-config
+        - name: config
+          emptyDir: {}
         - name: tls
           secret:
             secretName: kanidm-tls

apps/base/kanidm/vaultstaticsecret.yaml

@@ -1,23 +0,0 @@
----
-apiVersion: secrets.hashicorp.com/v1beta1
-kind: VaultStaticSecret
-metadata:
-  name: repl-certs
-  namespace: kanidm
-  labels:
-    app.kubernetes.io/name: kanidm
-    app.kubernetes.io/instance: kanidm
-spec:
-  vaultAuthRef: default
-  mount: kv
-  type: kv-v2
-  path: kubernetes/namespace/kanidm/default/repl-certs
-  refreshAfter: 5m
-  destination:
-    name: kanidm-repl-certs
-    create: true
-    overwrite: true
-  hmacSecretData: true
-  rolloutRestartTargets:
-    - kind: StatefulSet
-      name: kanidm

apps/base/open-webui/cnpg_cluster.yaml

@@ -2,24 +2,24 @@
 apiVersion: postgresql.cnpg.io/v1
 kind: Cluster
 metadata:
-  name: postgres
+  name: open-webui-postgres
-  namespace: artifactapi
+  namespace: open-webui
 spec:
   affinity:
     podAntiAffinityType: preferred
   bootstrap:
     initdb:
-      database: artifacts
+      database: open-webui
       encoding: UTF8
       localeCType: C
       localeCollate: C
-      owner: artifacts
+      owner: open-webui
       secret:
         name: postgres-credentials
   enablePDB: true
   enableSuperuserAccess: false
   failoverDelay: 0
-  imageName: ghcr.io/cloudnative-pg/postgresql:18.1-system-trixie
+  imageName: ghcr.io/cloudnative-pg/postgresql:17-minimal-trixie
   instances: 3
   logLevel: info
   maxSyncReplicas: 0
@@ -35,7 +35,7 @@ spec:
       archive_mode: "on"
       archive_timeout: 5min
       dynamic_shared_memory_type: posix
-      effective_cache_size: 256MB
+      effective_cache_size: 128MB
       full_page_writes: "on"
       log_destination: csvlog
       log_directory: /controller/log
@@ -44,15 +44,15 @@ spec:
       log_rotation_size: "0"
       log_truncate_on_rotation: "false"
       logging_collector: "on"
-      max_connections: "200"
+      max_connections: "100"
-      max_parallel_workers: "16"
+      max_parallel_workers: "4"
       max_replication_slots: "16"
-      max_worker_processes: "16"
+      max_worker_processes: "4"
-      shared_buffers: 128MB
+      shared_buffers: 64MB
       shared_memory_type: mmap
       ssl_max_protocol_version: TLSv1.3
       ssl_min_protocol_version: TLSv1.3
-      wal_keep_size: 256MB
+      wal_keep_size: 128MB
       wal_level: logical
       wal_log_hints: "on"
       wal_receiver_timeout: 5s
@@ -79,13 +79,13 @@ spec:
       cpu: 500m
       memory: 512Mi
     requests:
-      cpu: 250m
+      cpu: 100m
       memory: 256Mi
   smartShutdownTimeout: 180
   startDelay: 3600
   stopDelay: 1800
   storage:
     resizeInUseVolumes: true
-    size: 20Gi
+    size: 5Gi
     storageClass: cephrbd-fast-delete
   switchoverDelay: 3600

apps/base/open-webui/cnpg_pooler.yaml

@@ -2,18 +2,18 @@
 apiVersion: postgresql.cnpg.io/v1
 kind: Pooler
 metadata:
-  name: postgres-pooler
+  name: open-webui-postgres-pooler
-  namespace: artifactapi
+  namespace: open-webui
 spec:
   cluster:
-    name: postgres
+    name: open-webui-postgres
   instances: 2
   pgbouncer:
     parameters:
-      default_pool_size: "100"
+      default_pool_size: "50"
-      max_client_conn: "400"
+      max_client_conn: "200"
     paused: false
-    poolMode: session
+    poolMode: transaction
   template:
     metadata:
       labels:

apps/base/open-webui/deployment.yaml

@@ -0,0 +1,61 @@
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: open-webui
+  namespace: open-webui
+spec:
+  replicas: 3
+  selector:
+    matchLabels:
+      app: open-webui
+  template:
+    metadata:
+      annotations:
+        reloader.stakater.com/auto: "true"
+      labels:
+        app: open-webui
+    spec:
+      priorityClassName: power
+      containers:
+        - name: open-webui
+          image: ghcr.io/open-webui/open-webui:main
+          imagePullPolicy: Always
+          ports:
+            - containerPort: 8080
+              name: http
+              protocol: TCP
+          env:
+            - name: OPENAI_API_BASE_URL
+              value: https://litellm.k8s.syd1.au.unkin.net
+            - name: WEBUI_URL
+              value: https://chat.k8s.syd1.au.unkin.net
+          envFrom:
+            - secretRef:
+                name: open-webui-credentials
+          livenessProbe:
+            httpGet:
+              path: /health
+              port: 8080
+            failureThreshold: 3
+            initialDelaySeconds: 30
+            periodSeconds: 30
+            successThreshold: 1
+            timeoutSeconds: 5
+          readinessProbe:
+            httpGet:
+              path: /health
+              port: 8080
+            failureThreshold: 3
+            initialDelaySeconds: 10
+            periodSeconds: 10
+            successThreshold: 1
+            timeoutSeconds: 5
+          resources:
+            limits:
+              cpu: "2"
+              memory: 4Gi
+            requests:
+              cpu: 250m
+              memory: 512Mi
+      restartPolicy: Always

apps/base/open-webui/gateway.yaml

@@ -0,0 +1,37 @@
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: Gateway
+metadata:
+  labels:
+    traefik.io/instance: internal
+  annotations:
+    cert-manager.io/cluster-issuer: vault-issuer
+    cert-manager.io/common-name: chat.k8s.syd1.au.unkin.net
+    cert-manager.io/private-key-size: "4096"
+    external-dns.alpha.kubernetes.io/hostname: chat.k8s.syd1.au.unkin.net
+    external-dns.alpha.kubernetes.io/target: 198.18.200.4
+  name: open-webui
+  namespace: open-webui
+spec:
+  gatewayClassName: traefik-internal
+  listeners:
+    - allowedRoutes:
+        namespaces:
+          from: Same
+      hostname: chat.k8s.syd1.au.unkin.net
+      name: http
+      port: 80
+      protocol: HTTP
+    - allowedRoutes:
+        namespaces:
+          from: Same
+      hostname: chat.k8s.syd1.au.unkin.net
+      name: https
+      port: 443
+      protocol: HTTPS
+      tls:
+        certificateRefs:
+          - group: ""
+            kind: Secret
+            name: open-webui-tls
+        mode: Terminate

apps/base/open-webui/httproute.yaml

@@ -0,0 +1,53 @@
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: open-webui-http-redirect
+  namespace: open-webui
+spec:
+  hostnames:
+    - chat.k8s.syd1.au.unkin.net
+  parentRefs:
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: open-webui
+      sectionName: http
+  rules:
+    - filters:
+        - type: RequestRedirect
+          requestRedirect:
+            scheme: https
+            statusCode: 301
+      matches:
+        - path:
+            type: PathPrefix
+            value: /
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: open-webui
+  namespace: open-webui
+spec:
+  hostnames:
+    - chat.k8s.syd1.au.unkin.net
+  parentRefs:
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: open-webui
+      sectionName: https
+  rules:
+    - backendRefs:
+        - group: ""
+          kind: Service
+          name: open-webui
+          port: 8080
+          weight: 1
+      matches:
+        - path:
+            type: PathPrefix
+            value: /
+      sessionPersistence:
+        type: Cookie
+        cookieName: open-webui-backend
+        absoluteTimeout: 24h0m0s

apps/base/open-webui/kustomization.yaml

@@ -0,0 +1,15 @@
+---
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+resources:
+  - namespace.yaml
+  - cnpg_cluster.yaml
+  - cnpg_pooler.yaml
+  - deployment.yaml
+  - pdb.yaml
+  - service.yaml
+  - gateway.yaml
+  - httproute.yaml
+  - vaultauth.yaml
+  - vaultstaticsecret.yaml

apps/base/open-webui/namespace.yaml

@@ -0,0 +1,5 @@
+---
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: open-webui

apps/base/open-webui/pdb.yaml

@@ -0,0 +1,11 @@
+---
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+  name: open-webui
+  namespace: open-webui
+spec:
+  minAvailable: 1
+  selector:
+    matchLabels:
+      app: open-webui

apps/base/open-webui/service.yaml

@@ -0,0 +1,17 @@
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: open-webui
+  namespace: open-webui
+spec:
+  internalTrafficPolicy: Cluster
+  ports:
+    - name: http
+      port: 8080
+      protocol: TCP
+      targetPort: http
+  selector:
+    app: open-webui
+  sessionAffinity: None
+  type: ClusterIP

apps/base/open-webui/vaultauth.yaml

@@ -3,10 +3,10 @@ apiVersion: secrets.hashicorp.com/v1beta1
 kind: VaultAuth
 metadata:
   name: default
-  namespace: kanidm
+  namespace: open-webui
 spec:
   allowedNamespaces:
-    - kanidm
+    - open-webui
   kubernetes:
     audiences:
       - vault

apps/base/open-webui/vaultstaticsecret.yaml

@@ -0,0 +1,34 @@
+---
+apiVersion: secrets.hashicorp.com/v1beta1
+kind: VaultStaticSecret
+metadata:
+  name: postgres-credentials
+  namespace: open-webui
+spec:
+  destination:
+    create: true
+    name: postgres-credentials
+    overwrite: true
+  hmacSecretData: true
+  mount: kv
+  path: kubernetes/namespace/open-webui/default/postgres-credentials
+  refreshAfter: 5m
+  type: kv-v2
+  vaultAuthRef: default
+---
+apiVersion: secrets.hashicorp.com/v1beta1
+kind: VaultStaticSecret
+metadata:
+  name: open-webui-credentials
+  namespace: open-webui
+spec:
+  destination:
+    create: true
+    name: open-webui-credentials
+    overwrite: true
+  hmacSecretData: true
+  mount: kv
+  path: kubernetes/namespace/open-webui/default/open-webui-credentials
+  refreshAfter: 5m
+  type: kv-v2
+  vaultAuthRef: default

apps/base/priority-classes/priorityclasses.yaml

@@ -1,36 +0,0 @@
----
-apiVersion: scheduling.k8s.io/v1
-kind: PriorityClass
-metadata:
-  name: low
-value: 100
-preemptionPolicy: Never
-globalDefault: false
-description: "Low-importance workloads. Can be evicted under pressure but will not preempt other pods."
----
-apiVersion: scheduling.k8s.io/v1
-kind: PriorityClass
-metadata:
-  name: power
-value: 100
-preemptionPolicy: Never
-globalDefault: false
-description: "Compute-heavy workloads with low scheduling importance. Evictable under pressure."
----
-apiVersion: scheduling.k8s.io/v1
-kind: PriorityClass
-metadata:
-  name: medium
-value: 10000
-preemptionPolicy: PreemptLowerPriority
-globalDefault: false
-description: "Standard workloads. Will preempt low-priority pods if the cluster is under pressure."
----
-apiVersion: scheduling.k8s.io/v1
-kind: PriorityClass
-metadata:
-  name: high
-value: 100000
-preemptionPolicy: PreemptLowerPriority
-globalDefault: false
-description: "High-importance services. Will preempt medium- and low-priority pods if necessary."

apps/base/puppet/cronjob_g10k-code.yaml

@@ -28,7 +28,7 @@ spec:
           imagePullSecrets: null
           containers:
             - name: g10k-code
-              image: git.unkin.net/unkin/almalinux9-g10k:20260606
+              image: git.unkin.net/unkin/almalinux9-g10k:20260308
               imagePullPolicy: IfNotPresent
               resources:
                 requests:

apps/base/puppet/deployment_puppetboard.yaml

@@ -50,7 +50,7 @@ spec:
               cpu: 20m
               memory: 32Mi
         - name: cert-generator
-          image: git.unkin.net/unkin/almalinux9-base:20260606
+          image: git.unkin.net/unkin/almalinux9-base:20260308
           imagePullPolicy: IfNotPresent
           command:
             - sh

apps/base/puppet/deployment_puppetserver-compiler.yaml

@@ -181,7 +181,7 @@ spec:
           name: puppet-puppet-volume
 
       - name: setup-shared-bins
-        image: git.unkin.net/unkin/almalinux9-base:20260606
+        image: git.unkin.net/unkin/almalinux9-base:20260308
         command:
         - sh
         - -c

apps/base/vault/kustomization.yaml

@@ -6,4 +6,3 @@ resources:
   - namespace.yaml
   - gateway.yaml
   - httproute.yaml
-  - role_k8s-service-registration.yaml

apps/base/vault/role_k8s-service-registration.yaml

@@ -1,24 +0,0 @@
----
-apiVersion: rbac.authorization.k8s.io/v1
-kind: Role
-metadata:
-  name: vault-k8s-service-registration
-  namespace: vault
-rules:
-  - apiGroups: [""]
-    resources: ["pods"]
-    verbs: ["get", "update", "patch"]
----
-apiVersion: rbac.authorization.k8s.io/v1
-kind: RoleBinding
-metadata:
-  name: vault-k8s-service-registration
-  namespace: vault
-roleRef:
-  apiGroup: rbac.authorization.k8s.io
-  kind: Role
-  name: vault-k8s-service-registration
-subjects:
-  - kind: ServiceAccount
-    name: vault
-    namespace: vault

apps/base/woodpecker/kustomization.yaml

@@ -6,8 +6,6 @@ resources:
   - namespace.yaml
   - cnpg_cluster.yaml
   - cnpg_pooler.yaml
-  - serviceaccount_terraform_artifactapi.yaml
-  - serviceaccount_terraform_git.yaml
   - serviceaccount_terraform_vault.yaml
   - vaultauth.yaml
   - vaultstaticsecret.yaml

apps/base/woodpecker/serviceaccount_terraform_artifactapi.yaml

@@ -1,6 +0,0 @@
----
-apiVersion: v1
-kind: ServiceAccount
-metadata:
-  name: terraform-artifactapi
-  namespace: woodpecker

apps/base/woodpecker/serviceaccount_terraform_git.yaml

@@ -1,6 +0,0 @@
----
-apiVersion: v1
-kind: ServiceAccount
-metadata:
-  name: terraform-git
-  namespace: woodpecker

apps/overlays/au-syd1/open-webui/kustomization.yaml

@@ -3,4 +3,4 @@ apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 
 resources:
-  - priorityclasses.yaml
+  - ../../../base/open-webui

apps/overlays/au-syd1/priority-classes/kustomization.yaml

@@ -1,6 +0,0 @@
----
-apiVersion: kustomize.config.k8s.io/v1beta1
-kind: Kustomization
-
-resources:
-  - ../../../base/priority-classes

apps/overlays/au-syd1/vault/values.yaml

@@ -40,7 +40,9 @@ server:
           }
         }
 
-        service_registration "kubernetes" {}
+        service_registration "consul" {
+          address = "consul-server.consul.svc.cluster.local:8500"
+        }
 
   dataStorage:
     enabled: true
@@ -48,14 +50,6 @@ server:
     storageClass: cephrbd-fast-delete
     accessMode: ReadWriteOnce
 
-  extraEnv:
-    - name: VAULT_K8S_NAMESPACE
-      value: vault
-    - name: VAULT_K8S_POD_NAME
-      valueFrom:
-        fieldRef:
-          fieldPath: metadata.name
-
   statefulSet:
     securityContext:
       container:

apps/overlays/au-syd1/woodpecker/values.yaml

@@ -2,7 +2,6 @@ agent:
   replicaCount: 3
   env:
     WOODPECKER_MAX_WORKFLOWS: "8"
-    WOODPECKER_BACKEND_K8S_PRIORITY_CLASS: power
     WOODPECKER_BACKEND_K8S_STORAGE_CLASS: cephrbd-fast-delete
     WOODPECKER_BACKEND_K8S_VOLUME_SIZE: 10G
     WOODPECKER_BACKEND_K8S_STORAGE_RWX: false

argocd/applicationsets/aitooling.yaml

@@ -11,6 +11,7 @@ spec:
       revision: HEAD
       directories:
       - path: apps/overlays/*/litellm
+      - path: apps/overlays/*/open-webui
       - path: apps/overlays/*/paperclip
   template:
     metadata:

argocd/applicationsets/platform.yaml

@@ -11,7 +11,6 @@ spec:
       revision: HEAD
       directories:
       - path: apps/overlays/*/artifactapi
-      - path: apps/overlays/*/artifactapi3
       - path: apps/overlays/*/cattle-system
       - path: apps/overlays/*/cert-manager
       - path: apps/overlays/*/certificates
@@ -23,7 +22,6 @@ spec:
       - path: apps/overlays/*/jfrog
       - path: apps/overlays/*/kanidm
       - path: apps/overlays/*/node-feature-discovery
-      - path: apps/overlays/*/priority-classes
       - path: apps/overlays/*/puppet
       - path: apps/overlays/*/purelb
       - path: apps/overlays/*/reflector-system

argocd/projects/aitooling.yaml

@@ -11,6 +11,8 @@ spec:
   destinations:
     - namespace: 'litellm'
       server: https://kubernetes.default.svc
+    - namespace: 'open-webui'
+      server: https://kubernetes.default.svc
     - namespace: 'paperclip'
       server: https://kubernetes.default.svc
   clusterResourceWhitelist:

argocd/projects/platform.yaml

@@ -17,8 +17,6 @@ spec:
       server: https://kubernetes.default.svc
     - namespace: 'artifactapi'
       server: https://kubernetes.default.svc
-    - namespace: 'artifactapi3'
-      server: https://kubernetes.default.svc
     - namespace: 'cert-manager'
       server: https://kubernetes.default.svc
     - namespace: 'certificates'
@@ -33,8 +31,6 @@ spec:
       server: https://kubernetes.default.svc
     - namespace: 'node-feature-discovery'
       server: https://kubernetes.default.svc
-    - namespace: 'priority-classes'
-      server: https://kubernetes.default.svc
     - namespace: 'purelb'
       server: https://kubernetes.default.svc
     - namespace: 'puppet'

migration.md

@@ -1,145 +0,0 @@
-# Migration Guide: Terragrunt to ArgoCD
-
-## Prerequisites
-- Examine existing Terraform configuration in `sources/terraform-k8s/config/<app-name>/`
-- Identify Helm charts, values, storage classes, secrets, and other resources
-
-## Migration Steps
-
-### 1. Branch Creation
-- Change back to main branch: `git checkout main`
-- Create new branch: `git checkout -b benvin/<app-name>`
-
-### 2. Create Base Application Structure
-- Create directory: `apps/base/<app-name>/`
-- Create `apps/base/<app-name>/kustomization.yaml` with resources:
-  - `namespace.yaml`
-  - `vaultauth.yaml` (if needed)
-  - `vaultstaticsecret.yaml` (if needed)
-  - Additional resources as required (storageclass.yaml, etc.)
-
-### 3. Create Namespace
-- Create `apps/base/<app-name>/namespace.yaml` with app namespace
-
-### 4. Create Vault Integration (if needed)
-- Create `apps/base/<app-name>/vaultauth.yaml` from Terraform `vault_auth.yaml`
-  - Convert snake_case to camelCase for Kubernetes
-  - Map Terraform fields to VaultAuth spec
-- Create `apps/base/<app-name>/vaultstaticsecret.yaml` from `vault_static_secret.yaml`
-  - Convert to VaultStaticSecret spec format
-
-### 5. Create Additional Resources
-- Create any additional resources (StorageClass, etc.) from Terraform config
-- Maintain exact parameter parity with Terraform
-
-### 6. Create Overlay Structure
-- Create directory: `apps/overlays/au-syd1/<app-name>/`
-- Create `apps/overlays/au-syd1/<app-name>/kustomization.yaml`:
-  - Reference base: `../../../base/<app-name>`
-  - Add helmCharts section with repo, version, valuesFile
-- Create `apps/overlays/au-syd1/<app-name>/values.yaml` from Terraform helm_release values
-
-### 7. Update Project Configuration
-- Add Helm repository to appropriate project in `argocd/projects/`
-- Add namespace to project destinations (if needed)
-- Add required cluster resource permissions
-
-### 8. Update ApplicationSet
-- Add directory pattern to appropriate ApplicationSet in `argocd/applicationsets/`
-- Use existing patterns like `apps/overlays/*/csi-*` or `apps/overlays/*/<app-name>`
-
-### 9. Validation
-- Run `kustomize build --enable-helm apps/overlays/au-syd1/<app-name>` to generate all resources
-- Check resource types: `kustomize build --enable-helm apps/overlays/au-syd1/<app-name> | grep "^kind:" | sort | uniq -c`
-- Verify all resource types are permitted in the target project's `clusterResourceWhitelist` and `namespaceResourceWhitelist`
-- Run `make kubeconform` to validate all resources
-- Fix any validation errors
-
-### 10. Git Workflow
-- Add only created/modified files: `git add apps/base/<app-name>/ apps/overlays/au-syd1/<app-name>/ argocd/projects/<project>.yaml argocd/applicationsets/<project>.yaml`
-- **Never use `git add .`**
-- Create commit with descriptive message following existing patterns
-- Push branch: `git push -u origin benvin/<app-name>`
-
-## Project Organization
-
-### Platform Project
-- Core infrastructure and system components
-- Examples: cert-manager, external-dns, cnpg-system, reflector-system, etc.
-
-### Storage Project
-- Storage-related components
-- Pattern: `csi-*` applications
-- Examples: csi-cephfs, csi-cephrbd
-
-### Application-Specific Projects
-- Create new projects for logical groupings as needed
-
-## Common Patterns
-
-### Helm Chart Integration
-```yaml
-helmCharts:
-  - name: <chart-name>
-    repo: <helm-repo-url>
-    version: "<version>"
-    releaseName: <release-name>
-    namespace: <namespace>
-    valuesFile: values.yaml
-```
-
-### VaultAuth Template
-```yaml
-apiVersion: secrets.hashicorp.com/v1beta1
-kind: VaultAuth
-metadata:
-  name: <auth-name>
-  namespace: <namespace>
-spec:
-  method: kubernetes
-  mount: k8s/au/syd1
-  vaultConnectionRef: vso-system/default
-  allowedNamespaces:
-    - <namespace>
-  kubernetes:
-    role: <role>
-    serviceAccount: <service-account>
-    audiences:
-      - vault
-    tokenExpirationSeconds: 600
-```
-
-### VaultStaticSecret Template
-```yaml
-apiVersion: secrets.hashicorp.com/v1beta1
-kind: VaultStaticSecret
-metadata:
-  name: <secret-name>
-  namespace: <namespace>
-spec:
-  vaultAuthRef: <auth-ref>
-  mount: kv
-  type: kv-v2
-  path: <vault-path>
-  refreshAfter: 5m
-  destination:
-    name: <k8s-secret-name>
-    create: true
-```
-
-## Troubleshooting
-
-### Pre-commit Hook Issues
-- If hooks modify files, add changes: `git add -u`
-- Amend commit if safe: `git commit --amend --no-edit`
-- Use `git push --force-with-lease` for amended commits
-
-### Validation Failures
-- Check Helm chart compatibility with Kubernetes version
-- Verify all required fields are present in resources
-- Ensure proper YAML formatting
-
-### Missing Permissions
-- Add required cluster resources to project clusterResourceWhitelist
-- Add namespaces to project destinations
-- Verify Helm repository is in project sourceRepos