Add k8s Gitea deployment (migration target for git.unkin.net) (#309)

Stand up the git.unkin.net forge on k8s to replace the Puppet VM. Deployed HA-shaped to match what the VM already runs (multi-replica on shared storage + external DB/cache), so this is genuine multi-replica HA rather than single-replica failover. Serves a temporary git2.k8s.syd1.au.unkin.net host; the git.unkin.net cutover is staged in docs/gitea-migration.md.

- add apps/base/gitea: namespace, CNPG gitea-postgres (2 instances, S3 backup bucket cnpg-gitea, nightly 04:00/30d), pgbouncer pooler, standalone Valkey (session/cache/queue, AOF), VaultAuth + VaultStaticSecrets, Gateway + HTTPRoute
- add apps/overlays/au-syd1/gitea: official Gitea chart 12.6.0 (app 1.26.2, rootless, 2 replicas) via helm-through-kustomize; RWX CephFS repo storage, external CNPG + Valkey, Actions disabled, container registry disabled (moved to artifactapi), Authentik OIDC with auto-register/account-linking; SSH via LoadBalancer VIP 198.18.200.10:2222
- register gitea in the platform ApplicationSet + AppProject
- add docs/gitea-migration.md staged cutover plan (VM Postgres->CNPG dump/restore, DNS in main.unkin.net zone, consumer checklist, rollback)

Depends on: terraform-authentik gitea OIDC app, and terraform-artifactapi ^gitea/ dockerhub allowlist (both separate PRs). One-time Vault seeds are listed in the migration doc.

Reviewed-on: #309
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>

apps/base/bind-internal/authoritative/records.yaml

@@ -20,6 +20,32 @@ spec:
     # identity.unkin.net hostname there.
     - 198.18.200.4
 ---
+# PRODUCTION CUTOVER RECORD — intentionally commented out.
+# git.unkin.net currently resolves to the LIVE VM forge (HAProxy VRRP VIP
+# 198.18.19.17), which holds every repo the estate depends on. Uncommenting this
+# repoints the whole org's git.unkin.net at the new k8s Gitea gateway VIP, so it
+# is the FINAL step of the forge migration — gated on the data migration (gitea
+# dump/restore + SECRET_KEY copy) in argocd-apps docs/gitea-migration.md.
+# NOTE: the live git.unkin.net answer is served by the puppet DNS master today
+# (profiles::dns::master, records from PuppetDB); this k8s apex zone holds only
+# SOA+NS + a few DNSRecords so far. Confirm the k8s bind cluster is the live
+# authority for unkin.net (or update the puppet record instead) before relying
+# on this CR at cutover.
+# ---
+# apiVersion: bind.unkin.net/v1alpha1
+# kind: DNSRecord
+# metadata:
+#   name: git-dns-internal
+#   namespace: bind-internal
+# spec:
+#   zoneRef: unkin-net
+#   name: git
+#   type: A
+#   ttl: 600
+#   values:
+#     # traefik-internal gateway VIP; the gitea Gateway serves git.unkin.net there.
+#     - 198.18.200.4
+---
 apiVersion: bind.unkin.net/v1alpha1
 kind: DNSRecord
 metadata:

apps/base/gitea/cnpg_backup.yaml

@@ -0,0 +1,46 @@
+---
+# Ceph RGW (S3) backup target for the gitea CNPG cluster, provisioned by the
+# in-estate cephrgw-operator. One dedicated bucket + owner user per cluster.
+apiVersion: ceph.unkin.net/v1alpha1
+kind: ObjectStoreUser
+metadata:
+  name: cnpg-gitea-backup
+  namespace: gitea
+spec:
+  displayName: "CNPG backup owner (gitea)"
+  # RGW users are global; keep the uid namespace-qualified so it never collides.
+  uid: cnpg-gitea-backup
+  maxBuckets: 5
+  # Operator writes AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY (+ RGW_UID,
+  # S3_ENDPOINT) into this Secret; the Cluster's barmanObjectStore consumes it.
+  secretName: cnpg-gitea-backup-s3
+  retainOnDelete: true
+---
+apiVersion: ceph.unkin.net/v1alpha1
+kind: Bucket
+metadata:
+  name: cnpg-gitea
+  namespace: gitea
+spec:
+  bucketName: cnpg-gitea
+  ownerRef: cnpg-gitea-backup
+  versioning: false
+  tags:
+    app: gitea
+    purpose: cnpg-backup
+  retainOnDelete: true
+---
+apiVersion: postgresql.cnpg.io/v1
+kind: ScheduledBackup
+metadata:
+  name: cnpg-gitea-nightly
+  namespace: gitea
+spec:
+  # 6-field CNPG cron (seconds first). 04:00 — next free slot after netbox
+  # (03:40), keeping the estate's 20-minute stagger.
+  schedule: "0 0 4 * * *"
+  immediate: false
+  backupOwnerReference: self
+  method: barmanObjectStore
+  cluster:
+    name: gitea-postgres

apps/base/gitea/cnpg_cluster.yaml

@@ -0,0 +1,90 @@
+---
+# Postgres for the k8s Gitea (replaces the Patroni-shared DB the VM uses). Gitea
+# already runs on Postgres, so cutover is a plain pg dump/restore (no engine
+# conversion). App-user creds come from the postgres-credentials Vault secret.
+apiVersion: postgresql.cnpg.io/v1
+kind: Cluster
+metadata:
+  name: gitea-postgres
+  namespace: gitea
+spec:
+  affinity:
+    podAntiAffinityType: preferred
+  backup:
+    # 30-day retention. Enforced by CNPG against the object store on each
+    # successful base backup.
+    retentionPolicy: 30d
+    barmanObjectStore:
+      # Dedicated per-cluster Ceph RGW bucket (cephrgw-operator provisions it).
+      destinationPath: s3://cnpg-gitea
+      endpointURL: https://s3.ceph.unkin.net
+      # radosgw serves a Vault-PKI cert; trust the internal CA (reflected into
+      # every namespace as the vault-ca-cert Secret).
+      endpointCA:
+        name: vault-ca-cert
+        key: ca.crt
+      # Keys minted by the ObjectStoreUser in cnpg_backup.yaml; never hardcoded.
+      s3Credentials:
+        accessKeyId:
+          name: cnpg-gitea-backup-s3
+          key: AWS_ACCESS_KEY_ID
+        secretAccessKey:
+          name: cnpg-gitea-backup-s3
+          key: AWS_SECRET_ACCESS_KEY
+      # Path prefix within the bucket; keep stable across restores (see docs).
+      serverName: gitea
+      data:
+        compression: bzip2
+        jobs: 2
+      wal:
+        compression: zstd
+        maxParallel: 2
+  bootstrap:
+    initdb:
+      database: gitea
+      encoding: UTF8
+      localeCType: C
+      localeCollate: C
+      owner: gitea
+      secret:
+        name: postgres-credentials
+  enablePDB: true
+  enableSuperuserAccess: false
+  failoverDelay: 0
+  imageName: ghcr.io/cloudnative-pg/postgresql:18.1-system-trixie
+  instances: 2
+  logLevel: info
+  monitoring:
+    customQueriesConfigMap:
+      - key: queries
+        name: cnpg-default-monitoring
+    disableDefaultQueries: false
+    enablePodMonitor: false
+  postgresql:
+    parameters:
+      max_connections: "200"
+      shared_buffers: 256MB
+  primaryUpdateMethod: restart
+  primaryUpdateStrategy: unsupervised
+  replicationSlots:
+    highAvailability:
+      enabled: true
+      slotPrefix: _cnpg_
+    synchronizeReplicas:
+      enabled: true
+    updateInterval: 30
+  resources:
+    limits:
+      cpu: "2"
+      memory: 2Gi
+    requests:
+      cpu: 250m
+      memory: 512Mi
+  smartShutdownTimeout: 180
+  startDelay: 3600
+  stopDelay: 1800
+  storage:
+    resizeInUseVolumes: true
+    size: 20Gi
+    storageClass: cephrbd-fast-delete
+  switchoverDelay: 3600

apps/base/gitea/cnpg_pooler.yaml

@@ -0,0 +1,36 @@
+---
+# pgbouncer in front of the primary. Gitea opens a connection per request and
+# benefits from pooling under multiple app replicas. Session mode keeps Gitea's
+# occasional session-scoped state (advisory locks, LISTEN/NOTIFY) working.
+apiVersion: postgresql.cnpg.io/v1
+kind: Pooler
+metadata:
+  name: gitea-postgres-pooler-rw
+  namespace: gitea
+spec:
+  cluster:
+    name: gitea-postgres
+  instances: 2
+  pgbouncer:
+    parameters:
+      default_pool_size: "50"
+      max_client_conn: "200"
+    paused: false
+    poolMode: session
+  template:
+    metadata:
+      labels:
+        app: pooler-rw
+    spec:
+      affinity:
+        podAntiAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+            - labelSelector:
+                matchExpressions:
+                  - key: app
+                    operator: In
+                    values:
+                      - pooler-rw
+              topologyKey: kubernetes.io/hostname
+      containers: []
+  type: rw

apps/base/gitea/gateway.yaml

@@ -0,0 +1,69 @@
+---
+# HTTPS front for the k8s Gitea, served on two names:
+#   git.unkin.net              — canonical/production (apex, bind-operator zone;
+#                                DNS flip is the gated cutover step, see the doc)
+#   git.k8s.syd1.au.unkin.net  — admin/backup route (external-dns k8s.syd1 zone),
+#                                same dual-name pattern as identity.unkin.net.
+# The cert-manager Certificate (vault-issuer) takes CN git.unkin.net and gets a
+# DNS SAN for each TLS listener hostname automatically.
+apiVersion: gateway.networking.k8s.io/v1
+kind: Gateway
+metadata:
+  name: gitea
+  namespace: gitea
+  labels:
+    app.kubernetes.io/name: gitea
+    app.kubernetes.io/instance: gitea
+    traefik.io/instance: internal
+  annotations:
+    cert-manager.io/cluster-issuer: vault-issuer
+    cert-manager.io/common-name: git.unkin.net
+    cert-manager.io/private-key-size: "4096"
+    # Only the k8s admin route is published by external-dns (it owns just the
+    # k8s.syd1.au.unkin.net zone). git.unkin.net lives in the apex zone and is
+    # flipped at cutover — NOT managed here.
+    external-dns.alpha.kubernetes.io/hostname: git.k8s.syd1.au.unkin.net
+    external-dns.alpha.kubernetes.io/target: 198.18.200.4
+spec:
+  gatewayClassName: traefik-internal
+  listeners:
+    - name: http-primary
+      port: 80
+      protocol: HTTP
+      hostname: git.unkin.net
+      allowedRoutes:
+        namespaces:
+          from: Same
+    - name: https-primary
+      port: 443
+      protocol: HTTPS
+      hostname: git.unkin.net
+      allowedRoutes:
+        namespaces:
+          from: Same
+      tls:
+        mode: Terminate
+        certificateRefs:
+          - group: ""
+            kind: Secret
+            name: gitea-tls
+    - name: http-admin
+      port: 80
+      protocol: HTTP
+      hostname: git.k8s.syd1.au.unkin.net
+      allowedRoutes:
+        namespaces:
+          from: Same
+    - name: https-admin
+      port: 443
+      protocol: HTTPS
+      hostname: git.k8s.syd1.au.unkin.net
+      allowedRoutes:
+        namespaces:
+          from: Same
+      tls:
+        mode: Terminate
+        certificateRefs:
+          - group: ""
+            kind: Secret
+            name: gitea-tls

apps/base/gitea/httproute.yaml

@@ -0,0 +1,65 @@
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: gitea-http-redirect
+  namespace: gitea
+  labels:
+    app.kubernetes.io/name: gitea
+    app.kubernetes.io/instance: gitea
+spec:
+  hostnames:
+    - git.unkin.net
+    - git.k8s.syd1.au.unkin.net
+  parentRefs:
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: gitea
+      sectionName: http-primary
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: gitea
+      sectionName: http-admin
+  rules:
+    - filters:
+        - type: RequestRedirect
+          requestRedirect:
+            scheme: https
+            statusCode: 301
+      matches:
+        - path:
+            type: PathPrefix
+            value: /
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: gitea
+  namespace: gitea
+  labels:
+    app.kubernetes.io/name: gitea
+    app.kubernetes.io/instance: gitea
+spec:
+  hostnames:
+    - git.unkin.net
+    - git.k8s.syd1.au.unkin.net
+  parentRefs:
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: gitea
+      sectionName: https-primary
+    - group: gateway.networking.k8s.io
+      kind: Gateway
+      name: gitea
+      sectionName: https-admin
+  rules:
+    - backendRefs:
+        - group: ""
+          kind: Service
+          name: gitea-http
+          port: 3000
+          weight: 1
+      matches:
+        - path:
+            type: PathPrefix
+            value: /

apps/base/gitea/kustomization.yaml

@@ -0,0 +1,16 @@
+---
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+resources:
+  - namespace.yaml
+  - cnpg_cluster.yaml
+  - cnpg_backup.yaml
+  - cnpg_pooler.yaml
+  - valkey-deployment.yaml
+  - valkey-pvc.yaml
+  - valkey-service.yaml
+  - vaultauth.yaml
+  - vaultstaticsecret.yaml
+  - gateway.yaml
+  - httproute.yaml

apps/base/gitea/namespace.yaml

@@ -0,0 +1,7 @@
+---
+apiVersion: v1
+kind: Namespace
+metadata:
+  labels:
+    app.kubernetes.io/name: gitea
+  name: gitea

apps/base/gitea/valkey-deployment.yaml

@@ -0,0 +1,89 @@
+---
+# Standalone Valkey (Redis-compatible) for Gitea's session store, cache and
+# queue. The Gitea chart bundles a redis-cluster subchart, but we run our own
+# standalone Valkey here: it keeps image control in-estate (valkey/valkey,
+# already allowlisted through the artifactapi dockerhub mirror) and matches the
+# standalone-cache pattern used by litellm/netbox. One instance serves three
+# logical DBs: DB 0 = session, DB 1 = cache, DB 2 = queue. AOF persistence is
+# enabled so queued actions/webhook deliveries survive a restart.
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: gitea-valkey
+  namespace: gitea
+  labels:
+    app.kubernetes.io/name: gitea
+    app.kubernetes.io/component: valkey
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: gitea-valkey
+  strategy:
+    type: Recreate
+  template:
+    metadata:
+      labels:
+        app: gitea-valkey
+        app.kubernetes.io/name: gitea
+        app.kubernetes.io/component: valkey
+    spec:
+      securityContext:
+        fsGroup: 999
+      containers:
+        - name: valkey
+          image: valkey/valkey:8-alpine
+          imagePullPolicy: IfNotPresent
+          command:
+            - valkey-server
+            - --appendonly
+            - "yes"
+            - --save
+            - "60"
+            - "1"
+          ports:
+            - containerPort: 6379
+              name: valkey
+              protocol: TCP
+          securityContext:
+            allowPrivilegeEscalation: false
+            runAsNonRoot: true
+            runAsUser: 999
+            capabilities:
+              drop:
+                - ALL
+          livenessProbe:
+            exec:
+              command:
+                - valkey-cli
+                - ping
+            failureThreshold: 3
+            initialDelaySeconds: 30
+            periodSeconds: 30
+            successThreshold: 1
+            timeoutSeconds: 5
+          readinessProbe:
+            exec:
+              command:
+                - valkey-cli
+                - ping
+            failureThreshold: 3
+            initialDelaySeconds: 5
+            periodSeconds: 10
+            successThreshold: 1
+            timeoutSeconds: 5
+          resources:
+            limits:
+              cpu: 500m
+              memory: 512Mi
+            requests:
+              cpu: 50m
+              memory: 128Mi
+          volumeMounts:
+            - mountPath: /data
+              name: data
+      restartPolicy: Always
+      volumes:
+        - name: data
+          persistentVolumeClaim:
+            claimName: gitea-valkey-data

apps/base/gitea/valkey-pvc.yaml

@@ -0,0 +1,14 @@
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: gitea-valkey-data
+  namespace: gitea
+spec:
+  accessModes:
+    - ReadWriteOnce
+  resources:
+    requests:
+      storage: 5Gi
+  storageClassName: cephrbd-fast-delete
+  volumeMode: Filesystem

apps/base/gitea/valkey-service.yaml

@@ -0,0 +1,20 @@
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: gitea-valkey
+  namespace: gitea
+  labels:
+    app.kubernetes.io/name: gitea
+    app.kubernetes.io/component: valkey
+spec:
+  internalTrafficPolicy: Cluster
+  ports:
+    - name: valkey
+      port: 6379
+      protocol: TCP
+      targetPort: valkey
+  selector:
+    app: gitea-valkey
+  sessionAffinity: None
+  type: ClusterIP

apps/base/gitea/vaultauth.yaml

@@ -0,0 +1,18 @@
+---
+apiVersion: secrets.hashicorp.com/v1beta1
+kind: VaultAuth
+metadata:
+  name: default
+  namespace: gitea
+spec:
+  allowedNamespaces:
+    - gitea
+  kubernetes:
+    audiences:
+      - vault
+    role: default
+    serviceAccount: default
+    tokenExpirationSeconds: 600
+  method: kubernetes
+  mount: k8s/au/syd1
+  vaultConnectionRef: vso-system/default

apps/base/gitea/vaultstaticsecret.yaml

@@ -0,0 +1,83 @@
+---
+# CNPG app-user credentials (keys: username, password). Consumed by the Cluster
+# bootstrap (initdb.secret) AND by Gitea (gitea.config.database.PASSWD via the
+# chart's existingSecret wiring). One-time Vault seed — see the PR description.
+apiVersion: secrets.hashicorp.com/v1beta1
+kind: VaultStaticSecret
+metadata:
+  name: postgres-credentials
+  namespace: gitea
+spec:
+  destination:
+    create: true
+    name: postgres-credentials
+    overwrite: true
+  hmacSecretData: true
+  mount: kv
+  path: kubernetes/namespace/gitea/default/postgres-credentials
+  refreshAfter: 5m
+  type: kv-v2
+  vaultAuthRef: default
+---
+# Initial Gitea admin (keys: username, password, email). Applied by the chart's
+# init job on first boot (gitea.admin.existingSecret). Local fallback account
+# that survives the Authentik OIDC cutover. One-time Vault seed.
+apiVersion: secrets.hashicorp.com/v1beta1
+kind: VaultStaticSecret
+metadata:
+  name: gitea-admin
+  namespace: gitea
+spec:
+  destination:
+    create: true
+    name: gitea-admin
+    overwrite: true
+  hmacSecretData: true
+  mount: kv
+  path: kubernetes/namespace/gitea/default/gitea-admin
+  refreshAfter: 5m
+  type: kv-v2
+  vaultAuthRef: default
+---
+# Gitea internal secrets (keys: SECRET_KEY, INTERNAL_TOKEN). Pinned here rather
+# than chart-generated so all replicas share identical values AND so the data
+# cutover can replace them with the VM's app.ini values (SECRET_KEY encrypts
+# 2FA/mirror/oauth secrets in the DB — it MUST match the restored database).
+# One-time Vault seed.
+apiVersion: secrets.hashicorp.com/v1beta1
+kind: VaultStaticSecret
+metadata:
+  name: gitea-inner
+  namespace: gitea
+spec:
+  destination:
+    create: true
+    name: gitea-inner
+    overwrite: true
+  hmacSecretData: true
+  mount: kv
+  path: kubernetes/namespace/gitea/default/gitea-inner
+  refreshAfter: 5m
+  type: kv-v2
+  vaultAuthRef: default
+---
+# Authentik OIDC client secret (key: client_secret). Read by the
+# terraform-authentik provider runner (policy already grants
+# kv/.../namespace/+/default/oauth-credentials) AND mounted into Gitea to
+# register the OIDC login source. One-time Vault seed.
+apiVersion: secrets.hashicorp.com/v1beta1
+kind: VaultStaticSecret
+metadata:
+  name: oauth-credentials
+  namespace: gitea
+spec:
+  destination:
+    create: true
+    name: oauth-credentials
+    overwrite: true
+  hmacSecretData: true
+  mount: kv
+  path: kubernetes/namespace/gitea/default/oauth-credentials
+  refreshAfter: 5m
+  type: kv-v2
+  vaultAuthRef: default

apps/overlays/au-syd1/gitea/kustomization.yaml

@@ -0,0 +1,51 @@
+---
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+resources:
+  - ../../../base/gitea
+
+helmCharts:
+  - name: gitea
+    repo: oci://docker.gitea.com/charts
+    version: "12.6.0"
+    releaseName: gitea
+    namespace: gitea
+    valuesFile: values.yaml
+
+# The chart renders a `helm.sh/hook: test` connection Pod (busybox). We deploy
+# via kustomize+ArgoCD (not `helm test`), so drop it rather than leave an orphan
+# Pod pulling an unallowlisted image.
+patches:
+  - target:
+      version: v1
+      kind: Pod
+      name: gitea-test-connection
+    patch: |-
+      $patch: delete
+      apiVersion: v1
+      kind: Pod
+      metadata:
+        name: gitea-test-connection
+  # The chart renders the http Service with `targetPort: null` (it defaults the
+  # target to the port name, which kustomize drops); pin it to 3000 so the
+  # manifest is valid and the HTTPRoute backend resolves.
+  - target:
+      version: v1
+      kind: Service
+      name: gitea-http
+    patch: |-
+      - op: replace
+        path: /spec/ports/0/targetPort
+        value: 3000
+  # SSH is disabled (DISABLE_SSH); drop the ssh Service the chart still renders.
+  - target:
+      version: v1
+      kind: Service
+      name: gitea-ssh
+    patch: |-
+      $patch: delete
+      apiVersion: v1
+      kind: Service
+      metadata:
+        name: gitea-ssh

apps/overlays/au-syd1/gitea/values.yaml

@@ -0,0 +1,184 @@
+# Gitea helm-gitea chart 12.6.0 (appVersion 1.26.1), app pinned to 1.26.2 to
+# match the VM being replaced. HA-shaped: 2 replicas on shared RWX CephFS, with
+# Postgres (CNPG), cache/session/queue (Valkey), and OIDC all externalised —
+# exactly the shape the chart's docs/ha-setup.md requires for replicaCount > 1.
+replicaCount: 2
+
+image:
+  registry: docker.io
+  repository: gitea/gitea
+  tag: "1.26.2"       # chart appends "-rootless" because rootless: true below
+  rootless: true
+  pullPolicy: IfNotPresent
+
+# All bundled stateful subcharts OFF — we run CNPG Postgres + standalone Valkey.
+postgresql-ha:
+  enabled: false
+postgresql:
+  enabled: false
+valkey-cluster:
+  enabled: false
+valkey:
+  enabled: false
+
+# Shared repo/LFS/attachment storage. RWX is mandatory for >1 replica so every
+# pod sees the same /data (repos, lfs, avatars, attachments).
+persistence:
+  enabled: true
+  create: true
+  mount: true
+  accessModes:
+    - ReadWriteMany
+  storageClass: cephfs-raid6-delete
+  size: 20Gi
+
+service:
+  http:
+    type: ClusterIP
+    port: 3000
+  # git-over-ssh is disabled — HTTPS clones only (the estate norm). DISABLE_SSH
+  # below stops the in-pod SSH server; the chart still renders a gitea-ssh
+  # Service, which the overlay kustomization $patch-deletes.
+  ssh:
+    type: ClusterIP
+    port: 22
+
+# Gateway API (HTTPRoute in the base) fronts HTTP; the chart Ingress is unused.
+ingress:
+  enabled: false
+
+# Rolling replicas on slow shared storage: keep one old pod up and give new pods
+# generous startup headroom so migrations on boot don't trip the probes.
+strategy:
+  type: RollingUpdate
+  rollingUpdate:
+    maxSurge: "100%"
+    maxUnavailable: 0
+
+resources:
+  requests:
+    cpu: "500m"
+    memory: 512Mi
+  limits:
+    cpu: "2"
+    memory: 2Gi
+
+# Trust the internal unkin.net CA that signs identity.unkin.net's cert. Gitea is
+# Go, so SSL_CERT_DIR adds the mounted vault-ca-cert to the system trust pool
+# (additive — public roots stay via the default bundle). Needed for the OIDC
+# discovery/JWKS fetch at login.
+deployment:
+  env:
+    - name: SSL_CERT_DIR
+      value: /etc/gitea/tls-ca
+extraVolumes:
+  - name: vault-ca-cert
+    secret:
+      secretName: vault-ca-cert
+      items:
+        - key: ca.crt
+          path: ca.crt
+extraContainerVolumeMounts:
+  - name: vault-ca-cert
+    mountPath: /etc/gitea/tls-ca
+    readOnly: true
+extraInitVolumeMounts:
+  - name: vault-ca-cert
+    mountPath: /etc/gitea/tls-ca
+    readOnly: true
+
+gitea:
+  # Local admin fallback (survives the OIDC cutover). Secret keys: username,
+  # password. Seeded in Vault -> synced to the gitea-admin Secret by VSO.
+  admin:
+    existingSecret: gitea-admin
+    email: "benvin@unkin.net"
+    passwordMode: keepUpdated
+
+  # DB password (and Gitea internal secrets) injected as env from VSO Secrets,
+  # never rendered into app.ini in git. GITEA__<section>__<KEY> maps to app.ini.
+  additionalConfigFromEnvs:
+    - name: GITEA__database__PASSWD
+      valueFrom:
+        secretKeyRef:
+          name: postgres-credentials
+          key: password
+    - name: GITEA__security__SECRET_KEY
+      valueFrom:
+        secretKeyRef:
+          name: gitea-inner
+          key: SECRET_KEY
+    - name: GITEA__security__INTERNAL_TOKEN
+      valueFrom:
+        secretKeyRef:
+          name: gitea-inner
+          key: INTERNAL_TOKEN
+
+  # Authentik OIDC login source, registered declaratively on boot. existingSecret
+  # must carry keys `key` (client id) and `secret` (client secret) — seeded at the
+  # same Vault path terraform-authentik reads client_secret from.
+  oauth:
+    - name: "authentik"
+      provider: "openidConnect"
+      existingSecret: oauth-credentials
+      autoDiscoverUrl: "https://identity.unkin.net/application/o/gitea/.well-known/openid-configuration"
+
+  config:
+    server:
+      DOMAIN: git.unkin.net
+      ROOT_URL: https://git.unkin.net/
+      SSH_DOMAIN: git.unkin.net
+      DISABLE_SSH: true
+      START_SSH_SERVER: false
+      LFS_START_SERVER: true
+
+    database:
+      DB_TYPE: postgres
+      HOST: gitea-postgres-pooler-rw:5432
+      NAME: gitea
+      USER: gitea
+      SSL_MODE: disable
+
+    # Standalone Valkey: session on db 0, cache on db 1, queue on db 2.
+    session:
+      PROVIDER: redis
+      PROVIDER_CONFIG: "redis://gitea-valkey:6379/0"
+    cache:
+      ENABLED: true
+      ADAPTER: redis
+      HOST: "redis://gitea-valkey:6379/1"
+    queue:
+      TYPE: redis
+      CONN_STR: "redis://gitea-valkey:6379/2"
+
+    # DB-backed issue indexer is replica-safe on shared storage (matches the VM);
+    # avoids the on-disk bleve indexer that HA can't share.
+    indexer:
+      ISSUE_INDEXER_TYPE: db
+      REPO_INDEXER_ENABLED: false
+
+    # CI stays on Woodpecker — Gitea Actions disabled.
+    actions:
+      ENABLED: false
+
+    repository:
+      DEFAULT_BRANCH: main
+      DEFAULT_PRIVATE: last
+
+    # OIDC auto-registration + account linking so existing local users (benvin,
+    # bots) link to their Authentik identity by matching username on first login.
+    service:
+      ENABLE_AUTO_REGISTRATION: true
+    oauth2_client:
+      ENABLE_AUTO_REGISTRATION: true
+      ACCOUNT_LINKING: auto
+      USERNAME: preferred_username
+      UPDATE_AVATAR: true
+
+  # Startup headroom for migrations on RWX storage across a rolling update.
+  startupProbe:
+    enabled: true
+  livenessProbe:
+    enabled: true
+  readinessProbe:
+    enabled: true

argocd/applicationsets/platform.yaml

@@ -24,6 +24,7 @@ spec:
       - path: apps/overlays/*/elastic-system
       - path: apps/overlays/*/encapi
       - path: apps/overlays/*/externaldns
+      - path: apps/overlays/*/gitea
       - path: apps/overlays/*/grafana-system
       - path: apps/overlays/*/inteldeviceplugins-system
       - path: apps/overlays/*/jfrog

argocd/projects/platform.yaml

@@ -13,6 +13,7 @@ spec:
     - oci://gcr.io/k8s-staging-nfd/charts
     - oci://ghcr.io/woodpecker-ci/helm/woodpecker
     - oci://ghcr.io/netbox-community/netbox-chart/netbox
+    - oci://docker.gitea.com/charts
   destinations:
     - namespace: '*-system'
       server: https://kubernetes.default.svc
@@ -34,6 +35,8 @@ spec:
       server: https://kubernetes.default.svc
     - namespace: 'externaldns'
       server: https://kubernetes.default.svc
+    - namespace: 'gitea'
+      server: https://kubernetes.default.svc
     - namespace: 'jfrog'
       server: https://kubernetes.default.svc
     - namespace: 'kanidm'

docs/README.md

@@ -7,3 +7,4 @@ Operational notes for the manifests in this repo.
 | [cnpg-backups.md](cnpg-backups.md) | How CNPG Postgres backups (WAL archiving + nightly base backups) to Ceph RGW are configured. |
 | [cnpg-restore.md](cnpg-restore.md) | Restoring a CNPG cluster: full recovery, point-in-time recovery, cutover, and gotchas. |
 | [authentik-rancher-sso.md](authentik-rancher-sso.md) | Manual runtime step to point Rancher's OIDC auth at the canonical `identity.unkin.net` issuer and trust the internal CA. |
+| [gitea-migration.md](gitea-migration.md) | Staged cutover of the git.unkin.net forge from the Puppet VM to the `gitea` namespace. |

docs/gitea-migration.md

@@ -0,0 +1,130 @@
+# Gitea VM → k8s migration
+
+Replaces the Puppet-managed `git.unkin.net` VM (`roles::infra::git::server`, 3 nodes
+on shared `/shared/apps/gitea` + Patroni Postgres + Redis) with the `gitea`
+namespace here. The forge is critical infra (Woodpecker, tea, terraform-git,
+teabot, ArgoCD repo sources, r10k), so cutover is staged with the VM left intact
+for rollback.
+
+## What's deployed
+
+| Piece | Here | VM today |
+|---|---|---|
+| App | gitea chart 12.6.0, Gitea **1.26.2** rootless, **2 replicas** | binary 1.26.2, 3 nodes |
+| Repo/LFS storage | RWX PVC `gitea-shared-storage` (`cephfs-raid6-delete`, 20Gi) | `/shared/apps/gitea` |
+| DB | CNPG `gitea-postgres` (2 instances) + pgbouncer, S3 backup `cnpg-gitea` 04:00/30d | Patroni-shared Postgres |
+| Cache/session/queue | standalone Valkey (db 0/1/2, AOF) | Redis sentinel |
+| SSH | **disabled** (`DISABLE_SSH`) — HTTPS clones only | host sshd, port 2222 |
+| Ingress | Gateway API → `git.unkin.net` + `git.k8s.syd1.au.unkin.net` | HAProxy `be_gitea` → git.unkin.net |
+| Actions | **disabled** (CI stays Woodpecker) | runner deployed separately |
+| Container registry | **disabled** (moved to artifactapi `docker-internal`) | built-in, in use |
+| Other package types | enabled (generic archives for *arr etc. still used) | built-in |
+
+HA note: this is the **same shape the VM already runs** (multiple app nodes on
+shared storage + external DB + external Redis), which the Gitea chart's
+`docs/ha-setup.md` endorses for `replicaCount > 1`. Genuine multi-replica HA, not
+single-replica failover.
+
+## One-time Vault seeds (before first sync)
+
+```sh
+# app DB user (CNPG bootstrap + Gitea)
+vault kv put kv/kubernetes/namespace/gitea/default/postgres-credentials \
+  username=gitea password="$(openssl rand -hex 24)"
+
+# local admin fallback (survives OIDC cutover)
+vault kv put kv/kubernetes/namespace/gitea/default/gitea-admin \
+  username=benvin password="$(openssl rand -hex 24)" email=benvin@unkin.net
+
+# Gitea internal secrets — pinned so all replicas match. At data cutover (stage 1/2)
+# REPLACE these with the VM's app.ini [security] SECRET_KEY + INTERNAL_TOKEN so the
+# restored DB's encrypted columns (2FA, mirrors, oauth) stay decryptable.
+vault kv put kv/kubernetes/namespace/gitea/default/gitea-inner \
+  SECRET_KEY="$(gitea generate secret SECRET_KEY)" \
+  INTERNAL_TOKEN="$(gitea generate secret INTERNAL_TOKEN)"
+
+# Authentik OIDC client — `client_secret` read by terraform-authentik; `key`/`secret`
+# consumed by the chart's oauth login-source (existingSecret needs both keys).
+S=$(openssl rand -hex 40)
+vault kv put kv/kubernetes/namespace/gitea/default/oauth-credentials \
+  client_secret="$S" key=gitea secret="$S"
+```
+
+## Stages
+
+**0 — Validate on the admin route.** Merge this PR + the terraform-authentik PR
+(gitea OIDC app). ArgoCD syncs the `gitea` app; external-dns publishes
+`git.k8s.syd1.au.unkin.net` → the gateway VIP (safe — new name, empty instance).
+`git.unkin.net` stays on the live VM until stage 2. Confirm on the admin route:
+local-admin login, OIDC login (Authentik → auto-register/link), create a test
+repo, HTTPS push/clone (SSH is disabled — HTTPS only), and a Woodpecker webhook
+fires against `git.k8s.syd1.au.unkin.net`.
+
+**1 — Data migration rehearsal.** Gitea is already Postgres→Postgres, so **no DB
+engine conversion** — a plain SQL dump/restore plus a filesystem copy of the data
+dir. On a VM node:
+
+```sh
+# repos + LFS + attachments + avatars (the /shared/apps/gitea tree, minus the DB)
+gitea dump -c /etc/gitea/app.ini --skip-db --file /tmp/gitea-data.zip     # or rsync /shared/apps/gitea
+pg_dump -h master.patroni-shared-prod.service.au-syd1.consul -U gitea gitea > /tmp/gitea.sql
+```
+
+Restore into k8s:
+
+```sh
+# DB
+kubectl -n gitea exec -i gitea-postgres-1 -- psql -U gitea gitea < /tmp/gitea.sql
+# data dir into the shared PVC (via a throwaway pod mounting gitea-shared-storage)
+kubectl -n gitea cp /shared/apps/gitea/repos <helper-pod>:/data/git/repositories
+kubectl -n gitea cp /shared/apps/gitea/lfs   <helper-pod>:/data/git/lfs
+```
+
+Then set `gitea-inner` to the VM's real `SECRET_KEY`/`INTERNAL_TOKEN` (above) and
+restart the deployment. Verify repos/issues/LFS/users. Rehearse fully before the
+freeze so the freeze window is just a delta.
+
+**2 — Freeze + cutover (the `git.unkin.net` DNS flip).** Put the VM read-only (stop
+Woodpecker/webhooks), take a final dump/restore of the delta. The Gateway already
+serves `git.unkin.net` (listener + cert SAN) and the app already advertises
+`ROOT_URL https://git.unkin.net/`, so cutover is **only the DNS flip**:
+- Repoint `git.unkin.net` → the internal Traefik VIP `198.18.200.4`.
+- **Where the record lives (reconciled):** `git.unkin.net` resolves TODAY via the
+  **Puppet DNS master** (`profiles::dns::master`, records from PuppetDB exported
+  resources) to the HAProxy VRRP VIP `198.18.19.17` → the VM. The GitOps home for
+  the `unkin.net` apex is the **k8s bind-operator** `unkin-net` BindZone
+  (`apps/base/bind-internal/authoritative/`), where `identity.unkin.net` already
+  lives as a DNSRecord — but that zone currently holds only SOA+NS + a few records
+  (the bulk apex migration off Puppet is a pending follow-up). So at cutover, flip
+  **whichever is the live authority for `unkin.net` at that moment**: either
+  uncomment the `git-dns-internal` DNSRecord in `records.yaml` (prepared, gated) if
+  the k8s bind cluster is live, or change the Puppet-side record. **Not**
+  external-dns — it owns only the `k8s.syd1.au.unkin.net` zone.
+
+**3 — Consumer verification.** After DNS/cert flips:
+
+| Consumer | Check / knob |
+|---|---|
+| Woodpecker | forge URL (`WOODPECKER_GITEA_URL`) still `https://git.unkin.net`; OAuth app client id/secret unchanged; **`forge_remote_id` per-repo is stable** only if the Gitea repo IDs are preserved by the DB restore — a full SQL restore keeps them, a re-migration would not. Verify a webhook + a build. |
+| tea | `~/.config/tea/config.yml` host unchanged; token still valid (tokens are in the restored DB). |
+| terraform-git provider | `base_url https://git.unkin.net`, admin token from restored DB works. |
+| teabot / vault-plugin-secrets-gitea | admin user+password (Basic-Auth API) — the rotate-root password is in Vault, and the admin **local** user must exist post-restore. Re-run rotate-root if needed. |
+| ArgoCD repo creds | `repoURL https://git.unkin.net/unkin/argocd-apps` unchanged; the repo credential secret still valid. |
+| r10k / puppet | `r10k_repo https://git.unkin.net/unkin/puppet-r10k.git` unchanged. |
+| Container registry | pulls already moved to artifactapi `docker-internal` (see the registry-migration PR). Nothing should still pull `git.unkin.net/unkin/<image>`. |
+
+**4 — Rollback.** DNS `git.unkin.net` back to `198.18.19.17`, un-freeze the VM. The
+VM was never modified, so rollback is a single record change.
+
+## Open decisions / hard problems
+
+- **SSH is disabled** (`DISABLE_SSH`) — HTTPS clones only, the estate norm. No SSH
+  VIP, no SSH DNS, nothing to cut over.
+- **artifactapi's own image is circular** — it currently pulls `git.unkin.net/unkin/
+  artifactapi`. It can't pull itself from `docker-internal`. Keep it on an
+  external/bootstrap-safe registry (or node-preloaded) before the VM registry goes
+  away; not repointed in the registry-migration PR.
+- **`forge_remote_id` stability** (Woodpecker) hinges on preserving Gitea repo IDs —
+  guaranteed by a full SQL restore, broken by any re-create/re-migrate path.
+- **Cutover window**: freeze = final delta dump/restore + DNS TTL. Lower the
+  `git.unkin.net` record TTL (600s in both zones) a day ahead so the flip is quick.