Add k8s Gitea deployment (HA-shaped, migration target for git.unkin.net)
Stand up the forge on k8s to replace the Puppet VM. Deployed HA-shaped to match what the VM already runs (multi-replica on shared storage + external DB/cache): official Gitea chart 12.6.0 (app 1.26.2) at 2 replicas on RWX CephFS, CNPG Postgres with S3 backup, standalone Valkey for cache/session/queue, Authentik OIDC, Actions disabled and the container registry moved to artifactapi. Serves a temporary git2.k8s.syd1.au.unkin.net host; cutover is staged (see the doc). - add apps/base/gitea (namespace, CNPG cluster+backup+pooler, Valkey, VaultAuth, VaultStaticSecrets, Gateway, HTTPRoute) - add apps/overlays/au-syd1/gitea (chart 12.6.0 via helm-through-kustomize + values, drop the chart test Pod) - register gitea in the platform ApplicationSet and AppProject - add docs/gitea-migration.md staged cutover plan Claude-Session: https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
This commit is contained in:
@@ -0,0 +1,46 @@
|
||||
---
|
||||
# Ceph RGW (S3) backup target for the gitea CNPG cluster, provisioned by the
|
||||
# in-estate cephrgw-operator. One dedicated bucket + owner user per cluster.
|
||||
apiVersion: ceph.unkin.net/v1alpha1
|
||||
kind: ObjectStoreUser
|
||||
metadata:
|
||||
name: cnpg-gitea-backup
|
||||
namespace: gitea
|
||||
spec:
|
||||
displayName: "CNPG backup owner (gitea)"
|
||||
# RGW users are global; keep the uid namespace-qualified so it never collides.
|
||||
uid: cnpg-gitea-backup
|
||||
maxBuckets: 5
|
||||
# Operator writes AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY (+ RGW_UID,
|
||||
# S3_ENDPOINT) into this Secret; the Cluster's barmanObjectStore consumes it.
|
||||
secretName: cnpg-gitea-backup-s3
|
||||
retainOnDelete: true
|
||||
---
|
||||
apiVersion: ceph.unkin.net/v1alpha1
|
||||
kind: Bucket
|
||||
metadata:
|
||||
name: cnpg-gitea
|
||||
namespace: gitea
|
||||
spec:
|
||||
bucketName: cnpg-gitea
|
||||
ownerRef: cnpg-gitea-backup
|
||||
versioning: false
|
||||
tags:
|
||||
app: gitea
|
||||
purpose: cnpg-backup
|
||||
retainOnDelete: true
|
||||
---
|
||||
apiVersion: postgresql.cnpg.io/v1
|
||||
kind: ScheduledBackup
|
||||
metadata:
|
||||
name: cnpg-gitea-nightly
|
||||
namespace: gitea
|
||||
spec:
|
||||
# 6-field CNPG cron (seconds first). 04:00 — next free slot after netbox
|
||||
# (03:40), keeping the estate's 20-minute stagger.
|
||||
schedule: "0 0 4 * * *"
|
||||
immediate: false
|
||||
backupOwnerReference: self
|
||||
method: barmanObjectStore
|
||||
cluster:
|
||||
name: gitea-postgres
|
||||
@@ -0,0 +1,90 @@
|
||||
---
|
||||
# Postgres for the k8s Gitea (replaces the Patroni-shared DB the VM uses). Gitea
|
||||
# already runs on Postgres, so cutover is a plain pg dump/restore (no engine
|
||||
# conversion). App-user creds come from the postgres-credentials Vault secret.
|
||||
apiVersion: postgresql.cnpg.io/v1
|
||||
kind: Cluster
|
||||
metadata:
|
||||
name: gitea-postgres
|
||||
namespace: gitea
|
||||
spec:
|
||||
affinity:
|
||||
podAntiAffinityType: preferred
|
||||
backup:
|
||||
# 30-day retention. Enforced by CNPG against the object store on each
|
||||
# successful base backup.
|
||||
retentionPolicy: 30d
|
||||
barmanObjectStore:
|
||||
# Dedicated per-cluster Ceph RGW bucket (cephrgw-operator provisions it).
|
||||
destinationPath: s3://cnpg-gitea
|
||||
endpointURL: https://s3.ceph.unkin.net
|
||||
# radosgw serves a Vault-PKI cert; trust the internal CA (reflected into
|
||||
# every namespace as the vault-ca-cert Secret).
|
||||
endpointCA:
|
||||
name: vault-ca-cert
|
||||
key: ca.crt
|
||||
# Keys minted by the ObjectStoreUser in cnpg_backup.yaml; never hardcoded.
|
||||
s3Credentials:
|
||||
accessKeyId:
|
||||
name: cnpg-gitea-backup-s3
|
||||
key: AWS_ACCESS_KEY_ID
|
||||
secretAccessKey:
|
||||
name: cnpg-gitea-backup-s3
|
||||
key: AWS_SECRET_ACCESS_KEY
|
||||
# Path prefix within the bucket; keep stable across restores (see docs).
|
||||
serverName: gitea
|
||||
data:
|
||||
compression: bzip2
|
||||
jobs: 2
|
||||
wal:
|
||||
compression: zstd
|
||||
maxParallel: 2
|
||||
bootstrap:
|
||||
initdb:
|
||||
database: gitea
|
||||
encoding: UTF8
|
||||
localeCType: C
|
||||
localeCollate: C
|
||||
owner: gitea
|
||||
secret:
|
||||
name: postgres-credentials
|
||||
enablePDB: true
|
||||
enableSuperuserAccess: false
|
||||
failoverDelay: 0
|
||||
imageName: ghcr.io/cloudnative-pg/postgresql:18.1-system-trixie
|
||||
instances: 2
|
||||
logLevel: info
|
||||
monitoring:
|
||||
customQueriesConfigMap:
|
||||
- key: queries
|
||||
name: cnpg-default-monitoring
|
||||
disableDefaultQueries: false
|
||||
enablePodMonitor: false
|
||||
postgresql:
|
||||
parameters:
|
||||
max_connections: "200"
|
||||
shared_buffers: 256MB
|
||||
primaryUpdateMethod: restart
|
||||
primaryUpdateStrategy: unsupervised
|
||||
replicationSlots:
|
||||
highAvailability:
|
||||
enabled: true
|
||||
slotPrefix: _cnpg_
|
||||
synchronizeReplicas:
|
||||
enabled: true
|
||||
updateInterval: 30
|
||||
resources:
|
||||
limits:
|
||||
cpu: "2"
|
||||
memory: 2Gi
|
||||
requests:
|
||||
cpu: 250m
|
||||
memory: 512Mi
|
||||
smartShutdownTimeout: 180
|
||||
startDelay: 3600
|
||||
stopDelay: 1800
|
||||
storage:
|
||||
resizeInUseVolumes: true
|
||||
size: 20Gi
|
||||
storageClass: cephrbd-fast-delete
|
||||
switchoverDelay: 3600
|
||||
@@ -0,0 +1,36 @@
|
||||
---
|
||||
# pgbouncer in front of the primary. Gitea opens a connection per request and
|
||||
# benefits from pooling under multiple app replicas. Session mode keeps Gitea's
|
||||
# occasional session-scoped state (advisory locks, LISTEN/NOTIFY) working.
|
||||
apiVersion: postgresql.cnpg.io/v1
|
||||
kind: Pooler
|
||||
metadata:
|
||||
name: gitea-postgres-pooler-rw
|
||||
namespace: gitea
|
||||
spec:
|
||||
cluster:
|
||||
name: gitea-postgres
|
||||
instances: 2
|
||||
pgbouncer:
|
||||
parameters:
|
||||
default_pool_size: "50"
|
||||
max_client_conn: "200"
|
||||
paused: false
|
||||
poolMode: session
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: pooler-rw
|
||||
spec:
|
||||
affinity:
|
||||
podAntiAffinity:
|
||||
requiredDuringSchedulingIgnoredDuringExecution:
|
||||
- labelSelector:
|
||||
matchExpressions:
|
||||
- key: app
|
||||
operator: In
|
||||
values:
|
||||
- pooler-rw
|
||||
topologyKey: kubernetes.io/hostname
|
||||
containers: []
|
||||
type: rw
|
||||
@@ -0,0 +1,43 @@
|
||||
---
|
||||
# HTTPS front for the k8s Gitea. Temporary validation hostname
|
||||
# git2.k8s.syd1.au.unkin.net; the git.unkin.net SAN is added at cutover (see
|
||||
# docs/gitea-migration.md). SSH is exposed separately as an L4 LoadBalancer
|
||||
# (ssh-service.yaml) since Gateway API here terminates HTTP(S) only.
|
||||
apiVersion: gateway.networking.k8s.io/v1
|
||||
kind: Gateway
|
||||
metadata:
|
||||
name: gitea
|
||||
namespace: gitea
|
||||
labels:
|
||||
app.kubernetes.io/name: gitea
|
||||
app.kubernetes.io/instance: gitea
|
||||
traefik.io/instance: internal
|
||||
annotations:
|
||||
cert-manager.io/cluster-issuer: vault-issuer
|
||||
cert-manager.io/common-name: git2.k8s.syd1.au.unkin.net
|
||||
cert-manager.io/private-key-size: "4096"
|
||||
external-dns.alpha.kubernetes.io/hostname: git2.k8s.syd1.au.unkin.net
|
||||
external-dns.alpha.kubernetes.io/target: 198.18.200.4
|
||||
spec:
|
||||
gatewayClassName: traefik-internal
|
||||
listeners:
|
||||
- name: http
|
||||
port: 80
|
||||
protocol: HTTP
|
||||
hostname: git2.k8s.syd1.au.unkin.net
|
||||
allowedRoutes:
|
||||
namespaces:
|
||||
from: Same
|
||||
- name: https
|
||||
port: 443
|
||||
protocol: HTTPS
|
||||
hostname: git2.k8s.syd1.au.unkin.net
|
||||
allowedRoutes:
|
||||
namespaces:
|
||||
from: Same
|
||||
tls:
|
||||
mode: Terminate
|
||||
certificateRefs:
|
||||
- group: ""
|
||||
kind: Secret
|
||||
name: gitea-tls
|
||||
@@ -0,0 +1,55 @@
|
||||
---
|
||||
apiVersion: gateway.networking.k8s.io/v1
|
||||
kind: HTTPRoute
|
||||
metadata:
|
||||
name: gitea-http-redirect
|
||||
namespace: gitea
|
||||
labels:
|
||||
app.kubernetes.io/name: gitea
|
||||
app.kubernetes.io/instance: gitea
|
||||
spec:
|
||||
hostnames:
|
||||
- git2.k8s.syd1.au.unkin.net
|
||||
parentRefs:
|
||||
- group: gateway.networking.k8s.io
|
||||
kind: Gateway
|
||||
name: gitea
|
||||
sectionName: http
|
||||
rules:
|
||||
- filters:
|
||||
- type: RequestRedirect
|
||||
requestRedirect:
|
||||
scheme: https
|
||||
statusCode: 301
|
||||
matches:
|
||||
- path:
|
||||
type: PathPrefix
|
||||
value: /
|
||||
---
|
||||
apiVersion: gateway.networking.k8s.io/v1
|
||||
kind: HTTPRoute
|
||||
metadata:
|
||||
name: gitea
|
||||
namespace: gitea
|
||||
labels:
|
||||
app.kubernetes.io/name: gitea
|
||||
app.kubernetes.io/instance: gitea
|
||||
spec:
|
||||
hostnames:
|
||||
- git2.k8s.syd1.au.unkin.net
|
||||
parentRefs:
|
||||
- group: gateway.networking.k8s.io
|
||||
kind: Gateway
|
||||
name: gitea
|
||||
sectionName: https
|
||||
rules:
|
||||
- backendRefs:
|
||||
- group: ""
|
||||
kind: Service
|
||||
name: gitea-http
|
||||
port: 3000
|
||||
weight: 1
|
||||
matches:
|
||||
- path:
|
||||
type: PathPrefix
|
||||
value: /
|
||||
@@ -0,0 +1,16 @@
|
||||
---
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- namespace.yaml
|
||||
- cnpg_cluster.yaml
|
||||
- cnpg_backup.yaml
|
||||
- cnpg_pooler.yaml
|
||||
- valkey-deployment.yaml
|
||||
- valkey-pvc.yaml
|
||||
- valkey-service.yaml
|
||||
- vaultauth.yaml
|
||||
- vaultstaticsecret.yaml
|
||||
- gateway.yaml
|
||||
- httproute.yaml
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: gitea
|
||||
name: gitea
|
||||
@@ -0,0 +1,89 @@
|
||||
---
|
||||
# Standalone Valkey (Redis-compatible) for Gitea's session store, cache and
|
||||
# queue. The Gitea chart bundles a redis-cluster subchart, but we run our own
|
||||
# standalone Valkey here: it keeps image control in-estate (valkey/valkey,
|
||||
# already allowlisted through the artifactapi dockerhub mirror) and matches the
|
||||
# standalone-cache pattern used by litellm/netbox. One instance serves three
|
||||
# logical DBs: DB 0 = session, DB 1 = cache, DB 2 = queue. AOF persistence is
|
||||
# enabled so queued actions/webhook deliveries survive a restart.
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: gitea-valkey
|
||||
namespace: gitea
|
||||
labels:
|
||||
app.kubernetes.io/name: gitea
|
||||
app.kubernetes.io/component: valkey
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: gitea-valkey
|
||||
strategy:
|
||||
type: Recreate
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: gitea-valkey
|
||||
app.kubernetes.io/name: gitea
|
||||
app.kubernetes.io/component: valkey
|
||||
spec:
|
||||
securityContext:
|
||||
fsGroup: 999
|
||||
containers:
|
||||
- name: valkey
|
||||
image: valkey/valkey:8-alpine
|
||||
imagePullPolicy: IfNotPresent
|
||||
command:
|
||||
- valkey-server
|
||||
- --appendonly
|
||||
- "yes"
|
||||
- --save
|
||||
- "60"
|
||||
- "1"
|
||||
ports:
|
||||
- containerPort: 6379
|
||||
name: valkey
|
||||
protocol: TCP
|
||||
securityContext:
|
||||
allowPrivilegeEscalation: false
|
||||
runAsNonRoot: true
|
||||
runAsUser: 999
|
||||
capabilities:
|
||||
drop:
|
||||
- ALL
|
||||
livenessProbe:
|
||||
exec:
|
||||
command:
|
||||
- valkey-cli
|
||||
- ping
|
||||
failureThreshold: 3
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 30
|
||||
successThreshold: 1
|
||||
timeoutSeconds: 5
|
||||
readinessProbe:
|
||||
exec:
|
||||
command:
|
||||
- valkey-cli
|
||||
- ping
|
||||
failureThreshold: 3
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 10
|
||||
successThreshold: 1
|
||||
timeoutSeconds: 5
|
||||
resources:
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
requests:
|
||||
cpu: 50m
|
||||
memory: 128Mi
|
||||
volumeMounts:
|
||||
- mountPath: /data
|
||||
name: data
|
||||
restartPolicy: Always
|
||||
volumes:
|
||||
- name: data
|
||||
persistentVolumeClaim:
|
||||
claimName: gitea-valkey-data
|
||||
@@ -0,0 +1,14 @@
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: gitea-valkey-data
|
||||
namespace: gitea
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 5Gi
|
||||
storageClassName: cephrbd-fast-delete
|
||||
volumeMode: Filesystem
|
||||
@@ -0,0 +1,20 @@
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: gitea-valkey
|
||||
namespace: gitea
|
||||
labels:
|
||||
app.kubernetes.io/name: gitea
|
||||
app.kubernetes.io/component: valkey
|
||||
spec:
|
||||
internalTrafficPolicy: Cluster
|
||||
ports:
|
||||
- name: valkey
|
||||
port: 6379
|
||||
protocol: TCP
|
||||
targetPort: valkey
|
||||
selector:
|
||||
app: gitea-valkey
|
||||
sessionAffinity: None
|
||||
type: ClusterIP
|
||||
@@ -0,0 +1,18 @@
|
||||
---
|
||||
apiVersion: secrets.hashicorp.com/v1beta1
|
||||
kind: VaultAuth
|
||||
metadata:
|
||||
name: default
|
||||
namespace: gitea
|
||||
spec:
|
||||
allowedNamespaces:
|
||||
- gitea
|
||||
kubernetes:
|
||||
audiences:
|
||||
- vault
|
||||
role: default
|
||||
serviceAccount: default
|
||||
tokenExpirationSeconds: 600
|
||||
method: kubernetes
|
||||
mount: k8s/au/syd1
|
||||
vaultConnectionRef: vso-system/default
|
||||
@@ -0,0 +1,83 @@
|
||||
---
|
||||
# CNPG app-user credentials (keys: username, password). Consumed by the Cluster
|
||||
# bootstrap (initdb.secret) AND by Gitea (gitea.config.database.PASSWD via the
|
||||
# chart's existingSecret wiring). One-time Vault seed — see the PR description.
|
||||
apiVersion: secrets.hashicorp.com/v1beta1
|
||||
kind: VaultStaticSecret
|
||||
metadata:
|
||||
name: postgres-credentials
|
||||
namespace: gitea
|
||||
spec:
|
||||
destination:
|
||||
create: true
|
||||
name: postgres-credentials
|
||||
overwrite: true
|
||||
hmacSecretData: true
|
||||
mount: kv
|
||||
path: kubernetes/namespace/gitea/default/postgres-credentials
|
||||
refreshAfter: 5m
|
||||
type: kv-v2
|
||||
vaultAuthRef: default
|
||||
---
|
||||
# Initial Gitea admin (keys: username, password, email). Applied by the chart's
|
||||
# init job on first boot (gitea.admin.existingSecret). Local fallback account
|
||||
# that survives the Authentik OIDC cutover. One-time Vault seed.
|
||||
apiVersion: secrets.hashicorp.com/v1beta1
|
||||
kind: VaultStaticSecret
|
||||
metadata:
|
||||
name: gitea-admin
|
||||
namespace: gitea
|
||||
spec:
|
||||
destination:
|
||||
create: true
|
||||
name: gitea-admin
|
||||
overwrite: true
|
||||
hmacSecretData: true
|
||||
mount: kv
|
||||
path: kubernetes/namespace/gitea/default/gitea-admin
|
||||
refreshAfter: 5m
|
||||
type: kv-v2
|
||||
vaultAuthRef: default
|
||||
---
|
||||
# Gitea internal secrets (keys: SECRET_KEY, INTERNAL_TOKEN). Pinned here rather
|
||||
# than chart-generated so all replicas share identical values AND so the data
|
||||
# cutover can replace them with the VM's app.ini values (SECRET_KEY encrypts
|
||||
# 2FA/mirror/oauth secrets in the DB — it MUST match the restored database).
|
||||
# One-time Vault seed.
|
||||
apiVersion: secrets.hashicorp.com/v1beta1
|
||||
kind: VaultStaticSecret
|
||||
metadata:
|
||||
name: gitea-inner
|
||||
namespace: gitea
|
||||
spec:
|
||||
destination:
|
||||
create: true
|
||||
name: gitea-inner
|
||||
overwrite: true
|
||||
hmacSecretData: true
|
||||
mount: kv
|
||||
path: kubernetes/namespace/gitea/default/gitea-inner
|
||||
refreshAfter: 5m
|
||||
type: kv-v2
|
||||
vaultAuthRef: default
|
||||
---
|
||||
# Authentik OIDC client secret (key: client_secret). Read by the
|
||||
# terraform-authentik provider runner (policy already grants
|
||||
# kv/.../namespace/+/default/oauth-credentials) AND mounted into Gitea to
|
||||
# register the OIDC login source. One-time Vault seed.
|
||||
apiVersion: secrets.hashicorp.com/v1beta1
|
||||
kind: VaultStaticSecret
|
||||
metadata:
|
||||
name: oauth-credentials
|
||||
namespace: gitea
|
||||
spec:
|
||||
destination:
|
||||
create: true
|
||||
name: oauth-credentials
|
||||
overwrite: true
|
||||
hmacSecretData: true
|
||||
mount: kv
|
||||
path: kubernetes/namespace/gitea/default/oauth-credentials
|
||||
refreshAfter: 5m
|
||||
type: kv-v2
|
||||
vaultAuthRef: default
|
||||
@@ -0,0 +1,29 @@
|
||||
---
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- ../../../base/gitea
|
||||
|
||||
helmCharts:
|
||||
- name: gitea
|
||||
repo: oci://docker.gitea.com/charts
|
||||
version: "12.6.0"
|
||||
releaseName: gitea
|
||||
namespace: gitea
|
||||
valuesFile: values.yaml
|
||||
|
||||
# The chart renders a `helm.sh/hook: test` connection Pod (busybox). We deploy
|
||||
# via kustomize+ArgoCD (not `helm test`), so drop it rather than leave an orphan
|
||||
# Pod pulling an unallowlisted image.
|
||||
patches:
|
||||
- target:
|
||||
version: v1
|
||||
kind: Pod
|
||||
name: gitea-test-connection
|
||||
patch: |-
|
||||
$patch: delete
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: gitea-test-connection
|
||||
@@ -0,0 +1,169 @@
|
||||
# Gitea helm-gitea chart 12.6.0 (appVersion 1.26.1), app pinned to 1.26.2 to
|
||||
# match the VM being replaced. HA-shaped: 2 replicas on shared RWX CephFS, with
|
||||
# Postgres (CNPG), cache/session/queue (Valkey), and OIDC all externalised —
|
||||
# exactly the shape the chart's docs/ha-setup.md requires for replicaCount > 1.
|
||||
replicaCount: 2
|
||||
|
||||
image:
|
||||
registry: docker.io
|
||||
repository: gitea/gitea
|
||||
tag: "1.26.2" # chart appends "-rootless" because rootless: true below
|
||||
rootless: true
|
||||
pullPolicy: IfNotPresent
|
||||
|
||||
# All bundled stateful subcharts OFF — we run CNPG Postgres + standalone Valkey.
|
||||
postgresql-ha:
|
||||
enabled: false
|
||||
postgresql:
|
||||
enabled: false
|
||||
valkey-cluster:
|
||||
enabled: false
|
||||
valkey:
|
||||
enabled: false
|
||||
|
||||
# Shared repo/LFS/attachment storage. RWX is mandatory for >1 replica so every
|
||||
# pod sees the same /data (repos, lfs, avatars, attachments).
|
||||
persistence:
|
||||
enabled: true
|
||||
create: true
|
||||
mount: true
|
||||
accessModes:
|
||||
- ReadWriteMany
|
||||
storageClass: cephfs-raid6-delete
|
||||
size: 20Gi
|
||||
|
||||
service:
|
||||
http:
|
||||
type: ClusterIP
|
||||
port: 3000
|
||||
# git-over-ssh. Each replica runs Gitea's built-in Go SSH server (rootless,
|
||||
# listens 2222); the LoadBalancer fans connections across pods. Dedicated
|
||||
# PureLB VIP + its own DNS name for the validation phase (see the migration
|
||||
# doc for the git.unkin.net:2222 unification decision at cutover).
|
||||
ssh:
|
||||
type: LoadBalancer
|
||||
port: 2222
|
||||
loadBalancerIP: 198.18.200.10
|
||||
externalTrafficPolicy: Local
|
||||
annotations:
|
||||
purelb.io/addresses: 198.18.200.10
|
||||
purelb.io/service-group: common
|
||||
external-dns.alpha.kubernetes.io/hostname: git2-ssh.k8s.syd1.au.unkin.net
|
||||
external-dns.alpha.kubernetes.io/target: 198.18.200.10
|
||||
|
||||
# Gateway API (HTTPRoute in the base) fronts HTTP; the chart Ingress is unused.
|
||||
ingress:
|
||||
enabled: false
|
||||
|
||||
# Rolling replicas on slow shared storage: keep one old pod up and give new pods
|
||||
# generous startup headroom so migrations on boot don't trip the probes.
|
||||
strategy:
|
||||
type: RollingUpdate
|
||||
rollingUpdate:
|
||||
maxSurge: "100%"
|
||||
maxUnavailable: 0
|
||||
|
||||
resources:
|
||||
requests:
|
||||
cpu: "500m"
|
||||
memory: 512Mi
|
||||
limits:
|
||||
cpu: "2"
|
||||
memory: 2Gi
|
||||
|
||||
gitea:
|
||||
# Local admin fallback (survives the OIDC cutover). Secret keys: username,
|
||||
# password. Seeded in Vault -> synced to the gitea-admin Secret by VSO.
|
||||
admin:
|
||||
existingSecret: gitea-admin
|
||||
email: "benvin@unkin.net"
|
||||
passwordMode: keepUpdated
|
||||
|
||||
# DB password (and Gitea internal secrets) injected as env from VSO Secrets,
|
||||
# never rendered into app.ini in git. GITEA__<section>__<KEY> maps to app.ini.
|
||||
additionalConfigFromEnvs:
|
||||
- name: GITEA__database__PASSWD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-credentials
|
||||
key: password
|
||||
- name: GITEA__security__SECRET_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: gitea-inner
|
||||
key: SECRET_KEY
|
||||
- name: GITEA__security__INTERNAL_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: gitea-inner
|
||||
key: INTERNAL_TOKEN
|
||||
|
||||
# Authentik OIDC login source, registered declaratively on boot. existingSecret
|
||||
# must carry keys `key` (client id) and `secret` (client secret) — seeded at the
|
||||
# same Vault path terraform-authentik reads client_secret from.
|
||||
oauth:
|
||||
- name: "authentik"
|
||||
provider: "openidConnect"
|
||||
existingSecret: oauth-credentials
|
||||
autoDiscoverUrl: "https://identity.k8s.syd1.au.unkin.net/application/o/gitea/.well-known/openid-configuration"
|
||||
|
||||
config:
|
||||
server:
|
||||
DOMAIN: git2.k8s.syd1.au.unkin.net
|
||||
ROOT_URL: https://git2.k8s.syd1.au.unkin.net/
|
||||
SSH_DOMAIN: git2-ssh.k8s.syd1.au.unkin.net
|
||||
SSH_PORT: 2222 # advertised in clone URLs
|
||||
SSH_LISTEN_PORT: 2222 # rootless in-container listener
|
||||
START_SSH_SERVER: true
|
||||
LFS_START_SERVER: true
|
||||
|
||||
database:
|
||||
DB_TYPE: postgres
|
||||
HOST: gitea-postgres-pooler-rw:5432
|
||||
NAME: gitea
|
||||
USER: gitea
|
||||
SSL_MODE: disable
|
||||
|
||||
# Standalone Valkey: session on db 0, cache on db 1, queue on db 2.
|
||||
session:
|
||||
PROVIDER: redis
|
||||
PROVIDER_CONFIG: "redis://gitea-valkey:6379/0"
|
||||
cache:
|
||||
ENABLED: true
|
||||
ADAPTER: redis
|
||||
HOST: "redis://gitea-valkey:6379/1"
|
||||
queue:
|
||||
TYPE: redis
|
||||
CONN_STR: "redis://gitea-valkey:6379/2"
|
||||
|
||||
# DB-backed issue indexer is replica-safe on shared storage (matches the VM);
|
||||
# avoids the on-disk bleve indexer that HA can't share.
|
||||
indexer:
|
||||
ISSUE_INDEXER_TYPE: db
|
||||
REPO_INDEXER_ENABLED: false
|
||||
|
||||
# CI stays on Woodpecker — Gitea Actions disabled.
|
||||
actions:
|
||||
ENABLED: false
|
||||
|
||||
repository:
|
||||
DEFAULT_BRANCH: main
|
||||
DEFAULT_PRIVATE: last
|
||||
|
||||
# OIDC auto-registration + account linking so existing local users (benvin,
|
||||
# bots) link to their Authentik identity by matching username on first login.
|
||||
service:
|
||||
ENABLE_AUTO_REGISTRATION: true
|
||||
oauth2_client:
|
||||
ENABLE_AUTO_REGISTRATION: true
|
||||
ACCOUNT_LINKING: auto
|
||||
USERNAME: preferred_username
|
||||
UPDATE_AVATAR: true
|
||||
|
||||
# Startup headroom for migrations on RWX storage across a rolling update.
|
||||
startupProbe:
|
||||
enabled: true
|
||||
livenessProbe:
|
||||
enabled: true
|
||||
readinessProbe:
|
||||
enabled: true
|
||||
@@ -24,6 +24,7 @@ spec:
|
||||
- path: apps/overlays/*/elastic-system
|
||||
- path: apps/overlays/*/encapi
|
||||
- path: apps/overlays/*/externaldns
|
||||
- path: apps/overlays/*/gitea
|
||||
- path: apps/overlays/*/grafana-system
|
||||
- path: apps/overlays/*/inteldeviceplugins-system
|
||||
- path: apps/overlays/*/jfrog
|
||||
|
||||
@@ -13,6 +13,7 @@ spec:
|
||||
- oci://gcr.io/k8s-staging-nfd/charts
|
||||
- oci://ghcr.io/woodpecker-ci/helm/woodpecker
|
||||
- oci://ghcr.io/netbox-community/netbox-chart/netbox
|
||||
- oci://docker.gitea.com/charts
|
||||
destinations:
|
||||
- namespace: '*-system'
|
||||
server: https://kubernetes.default.svc
|
||||
@@ -34,6 +35,8 @@ spec:
|
||||
server: https://kubernetes.default.svc
|
||||
- namespace: 'externaldns'
|
||||
server: https://kubernetes.default.svc
|
||||
- namespace: 'gitea'
|
||||
server: https://kubernetes.default.svc
|
||||
- namespace: 'jfrog'
|
||||
server: https://kubernetes.default.svc
|
||||
- namespace: 'kanidm'
|
||||
|
||||
@@ -7,3 +7,4 @@ Operational notes for the manifests in this repo.
|
||||
| [cnpg-backups.md](cnpg-backups.md) | How CNPG Postgres backups (WAL archiving + nightly base backups) to Ceph RGW are configured. |
|
||||
| [cnpg-restore.md](cnpg-restore.md) | Restoring a CNPG cluster: full recovery, point-in-time recovery, cutover, and gotchas. |
|
||||
| [authentik-rancher-sso.md](authentik-rancher-sso.md) | Manual runtime step to point Rancher's OIDC auth at the canonical `identity.unkin.net` issuer and trust the internal CA. |
|
||||
| [gitea-migration.md](gitea-migration.md) | Staged cutover of the git.unkin.net forge from the Puppet VM to the `gitea` namespace. |
|
||||
|
||||
@@ -0,0 +1,127 @@
|
||||
# Gitea VM → k8s migration
|
||||
|
||||
Replaces the Puppet-managed `git.unkin.net` VM (`roles::infra::git::server`, 3 nodes
|
||||
on shared `/shared/apps/gitea` + Patroni Postgres + Redis) with the `gitea`
|
||||
namespace here. The forge is critical infra (Woodpecker, tea, terraform-git,
|
||||
teabot, ArgoCD repo sources, r10k), so cutover is staged with the VM left intact
|
||||
for rollback.
|
||||
|
||||
## What's deployed
|
||||
|
||||
| Piece | Here | VM today |
|
||||
|---|---|---|
|
||||
| App | gitea chart 12.6.0, Gitea **1.26.2** rootless, **2 replicas** | binary 1.26.2, 3 nodes |
|
||||
| Repo/LFS storage | RWX PVC `gitea-shared-storage` (`cephfs-raid6-delete`, 20Gi) | `/shared/apps/gitea` |
|
||||
| DB | CNPG `gitea-postgres` (2 instances) + pgbouncer, S3 backup `cnpg-gitea` 04:00/30d | Patroni-shared Postgres |
|
||||
| Cache/session/queue | standalone Valkey (db 0/1/2, AOF) | Redis sentinel |
|
||||
| SSH | LoadBalancer VIP `198.18.200.10:2222` (built-in per-pod SSH server) | host sshd, port 2222 |
|
||||
| Ingress | Gateway API → `git2.k8s.syd1.au.unkin.net` (temp) | HAProxy `be_gitea` → git.unkin.net |
|
||||
| Actions | **disabled** (CI stays Woodpecker) | runner deployed separately |
|
||||
| Container registry | **disabled** (moved to artifactapi `docker-internal`) | built-in, in use |
|
||||
| Other package types | enabled (generic archives for *arr etc. still used) | built-in |
|
||||
|
||||
HA note: this is the **same shape the VM already runs** (multiple app nodes on
|
||||
shared storage + external DB + external Redis), which the Gitea chart's
|
||||
`docs/ha-setup.md` endorses for `replicaCount > 1`. Genuine multi-replica HA, not
|
||||
single-replica failover.
|
||||
|
||||
## One-time Vault seeds (before first sync)
|
||||
|
||||
```sh
|
||||
# app DB user (CNPG bootstrap + Gitea)
|
||||
vault kv put kv/kubernetes/namespace/gitea/default/postgres-credentials \
|
||||
username=gitea password="$(openssl rand -hex 24)"
|
||||
|
||||
# local admin fallback (survives OIDC cutover)
|
||||
vault kv put kv/kubernetes/namespace/gitea/default/gitea-admin \
|
||||
username=benvin password="$(openssl rand -hex 24)" email=benvin@unkin.net
|
||||
|
||||
# Gitea internal secrets — pinned so all replicas match. At data cutover (stage 1/2)
|
||||
# REPLACE these with the VM's app.ini [security] SECRET_KEY + INTERNAL_TOKEN so the
|
||||
# restored DB's encrypted columns (2FA, mirrors, oauth) stay decryptable.
|
||||
vault kv put kv/kubernetes/namespace/gitea/default/gitea-inner \
|
||||
SECRET_KEY="$(gitea generate secret SECRET_KEY)" \
|
||||
INTERNAL_TOKEN="$(gitea generate secret INTERNAL_TOKEN)"
|
||||
|
||||
# Authentik OIDC client — `client_secret` read by terraform-authentik; `key`/`secret`
|
||||
# consumed by the chart's oauth login-source (existingSecret needs both keys).
|
||||
S=$(openssl rand -hex 40)
|
||||
vault kv put kv/kubernetes/namespace/gitea/default/oauth-credentials \
|
||||
client_secret="$S" key=gitea secret="$S"
|
||||
```
|
||||
|
||||
## Stages
|
||||
|
||||
**0 — Validate on the temp host.** Merge this PR + the terraform-authentik PR
|
||||
(gitea OIDC app). ArgoCD syncs the `gitea` app. Confirm: `git2.k8s.syd1.au.unkin.net`
|
||||
serves, local-admin login, OIDC login (Authentik → auto-register/link), create a
|
||||
test repo, HTTPS push/clone, SSH push/clone via `198.18.200.10:2222`, and a
|
||||
Woodpecker webhook fires against the temp host.
|
||||
|
||||
**1 — Data migration rehearsal.** Gitea is already Postgres→Postgres, so **no DB
|
||||
engine conversion** — a plain SQL dump/restore plus a filesystem copy of the data
|
||||
dir. On a VM node:
|
||||
|
||||
```sh
|
||||
# repos + LFS + attachments + avatars (the /shared/apps/gitea tree, minus the DB)
|
||||
gitea dump -c /etc/gitea/app.ini --skip-db --file /tmp/gitea-data.zip # or rsync /shared/apps/gitea
|
||||
pg_dump -h master.patroni-shared-prod.service.au-syd1.consul -U gitea gitea > /tmp/gitea.sql
|
||||
```
|
||||
|
||||
Restore into k8s:
|
||||
|
||||
```sh
|
||||
# DB
|
||||
kubectl -n gitea exec -i gitea-postgres-1 -- psql -U gitea gitea < /tmp/gitea.sql
|
||||
# data dir into the shared PVC (via a throwaway pod mounting gitea-shared-storage)
|
||||
kubectl -n gitea cp /shared/apps/gitea/repos <helper-pod>:/data/git/repositories
|
||||
kubectl -n gitea cp /shared/apps/gitea/lfs <helper-pod>:/data/git/lfs
|
||||
```
|
||||
|
||||
Then set `gitea-inner` to the VM's real `SECRET_KEY`/`INTERNAL_TOKEN` (above) and
|
||||
restart the deployment. Verify repos/issues/LFS/users. Rehearse fully before the
|
||||
freeze so the freeze window is just a delta.
|
||||
|
||||
**2 — Freeze + cutover.** Put the VM read-only (stop Woodpecker/webhooks), take a
|
||||
final dump/restore of the delta. Then:
|
||||
- Add `git.unkin.net` to the Gateway: append it as a second listener hostname +
|
||||
cert SAN (`cert-manager.io/common-name` stays, add the host to the listeners and
|
||||
a matching HTTPRoute), and set `gitea.config.server.{DOMAIN,ROOT_URL,SSH_DOMAIN}`
|
||||
back to `git.unkin.net`.
|
||||
- Repoint DNS `git.unkin.net` → the internal Traefik VIP `198.18.200.4`. **Record
|
||||
owner:** `git.unkin.net` currently resolves to the HAProxy VRRP anycast VIP
|
||||
`198.18.19.17` (`roles::infra::halb::haproxy2 anycast_ip`); the A record lives in
|
||||
the `main.unkin.net` zone (Puppet DNS `roles::infra::dns::master`). Update it
|
||||
there (or via the bind estate) — **not** external-dns, which only owns the
|
||||
`*.k8s.…` records.
|
||||
|
||||
**3 — Consumer verification.** After DNS/cert flips:
|
||||
|
||||
| Consumer | Check / knob |
|
||||
|---|---|
|
||||
| Woodpecker | forge URL (`WOODPECKER_GITEA_URL`) still `https://git.unkin.net`; OAuth app client id/secret unchanged; **`forge_remote_id` per-repo is stable** only if the Gitea repo IDs are preserved by the DB restore — a full SQL restore keeps them, a re-migration would not. Verify a webhook + a build. |
|
||||
| tea | `~/.config/tea/config.yml` host unchanged; token still valid (tokens are in the restored DB). |
|
||||
| terraform-git provider | `base_url https://git.unkin.net`, admin token from restored DB works. |
|
||||
| teabot / vault-plugin-secrets-gitea | admin user+password (Basic-Auth API) — the rotate-root password is in Vault, and the admin **local** user must exist post-restore. Re-run rotate-root if needed. |
|
||||
| ArgoCD repo creds | `repoURL https://git.unkin.net/unkin/argocd-apps` unchanged; the repo credential secret still valid. |
|
||||
| r10k / puppet | `r10k_repo https://git.unkin.net/unkin/puppet-r10k.git` unchanged. |
|
||||
| Container registry | pulls already moved to artifactapi `docker-internal` (see the registry-migration PR). Nothing should still pull `git.unkin.net/unkin/<image>`. |
|
||||
|
||||
**4 — Rollback.** DNS `git.unkin.net` back to `198.18.19.17`, un-freeze the VM. The
|
||||
VM was never modified, so rollback is a single record change.
|
||||
|
||||
## Open decisions / hard problems
|
||||
|
||||
- **SSH on git.unkin.net:2222.** One A record can't point at both the HTTPS gateway
|
||||
VIP (`.4`) and a separate SSH VIP. Options: (a) HTTPS-only clones (estate norm,
|
||||
SSH dropped); (b) a Traefik `TCPRoute` so `.4:2222` reaches the SSH service
|
||||
(share one VIP); (c) keep SSH on a dedicated host/VIP. Validation uses (c) via
|
||||
`git2-ssh.k8s.…:2222`.
|
||||
- **artifactapi's own image is circular** — it currently pulls `git.unkin.net/unkin/
|
||||
artifactapi`. It can't pull itself from `docker-internal`. Keep it on an
|
||||
external/bootstrap-safe registry (or node-preloaded) before the VM registry goes
|
||||
away; not repointed in the registry-migration PR.
|
||||
- **`forge_remote_id` stability** (Woodpecker) hinges on preserving Gitea repo IDs —
|
||||
guaranteed by a full SQL restore, broken by any re-create/re-migrate path.
|
||||
- **Cutover window**: freeze = final delta dump/restore + DNS TTL. Keep the
|
||||
`main.unkin.net` record TTL low a day ahead.
|
||||
Reference in New Issue
Block a user