Disable SSH; serve git.unkin.net + git.k8s.syd1 admin route
ci/woodpecker/pr/vector-test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/kubeconform Pipeline was successful

Per the forge design: drop git-over-ssh (HTTPS clones only, estate norm) and
serve the new instance on both the canonical git.unkin.net and a
git.k8s.syd1.au.unkin.net admin/backup route (same dual-name pattern as
identity). Only the k8s admin name goes live now (external-dns); the
git.unkin.net DNS flip stays the gated final cutover step.

- values: DISABLE_SSH + START_SSH_SERVER false; DOMAIN/ROOT_URL/SSH_DOMAIN ->
  git.unkin.net; drop the SSH LoadBalancer service
- overlay: $patch-delete the chart's leftover gitea-ssh Service
- gateway/httproute: listeners + routes for git.unkin.net and
  git.k8s.syd1.au.unkin.net; cert CN git.unkin.net (both as SANs); external-dns
  publishes only git.k8s.syd1.au.unkin.net -> 198.18.200.4
- bind-internal: prepared (commented) git-dns-internal DNSRecord as the gated
  apex cutover step
- docs: SSH removed, dual hostnames, git.unkin.net flip as the final step

Claude-Session: https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
This commit is contained in:
2026-07-31 00:00:23 +10:00
parent 1ea0dc920c
commit 34c2994cd3
6 changed files with 126 additions and 59 deletions
@@ -20,6 +20,32 @@ spec:
# identity.unkin.net hostname there.
- 198.18.200.4
---
# PRODUCTION CUTOVER RECORD — intentionally commented out.
# git.unkin.net currently resolves to the LIVE VM forge (HAProxy VRRP VIP
# 198.18.19.17), which holds every repo the estate depends on. Uncommenting this
# repoints the whole org's git.unkin.net at the new k8s Gitea gateway VIP, so it
# is the FINAL step of the forge migration — gated on the data migration (gitea
# dump/restore + SECRET_KEY copy) in argocd-apps docs/gitea-migration.md.
# NOTE: the live git.unkin.net answer is served by the puppet DNS master today
# (profiles::dns::master, records from PuppetDB); this k8s apex zone holds only
# SOA+NS + a few DNSRecords so far. Confirm the k8s bind cluster is the live
# authority for unkin.net (or update the puppet record instead) before relying
# on this CR at cutover.
# ---
# apiVersion: bind.unkin.net/v1alpha1
# kind: DNSRecord
# metadata:
# name: git-dns-internal
# namespace: bind-internal
# spec:
# zoneRef: unkin-net
# name: git
# type: A
# ttl: 600
# values:
# # traefik-internal gateway VIP; the gitea Gateway serves git.unkin.net there.
# - 198.18.200.4
---
apiVersion: bind.unkin.net/v1alpha1
kind: DNSRecord
metadata:
+36 -10
View File
@@ -1,8 +1,11 @@
---
# HTTPS front for the k8s Gitea. Temporary validation hostname
# git2.k8s.syd1.au.unkin.net; the git.unkin.net SAN is added at cutover (see
# docs/gitea-migration.md). SSH is exposed separately as an L4 LoadBalancer
# (ssh-service.yaml) since Gateway API here terminates HTTP(S) only.
# HTTPS front for the k8s Gitea, served on two names:
# git.unkin.net — canonical/production (apex, bind-operator zone;
# DNS flip is the gated cutover step, see the doc)
# git.k8s.syd1.au.unkin.net — admin/backup route (external-dns k8s.syd1 zone),
# same dual-name pattern as identity.unkin.net.
# The cert-manager Certificate (vault-issuer) takes CN git.unkin.net and gets a
# DNS SAN for each TLS listener hostname automatically.
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
@@ -14,24 +17,47 @@ metadata:
traefik.io/instance: internal
annotations:
cert-manager.io/cluster-issuer: vault-issuer
cert-manager.io/common-name: git2.k8s.syd1.au.unkin.net
cert-manager.io/common-name: git.unkin.net
cert-manager.io/private-key-size: "4096"
external-dns.alpha.kubernetes.io/hostname: git2.k8s.syd1.au.unkin.net
# Only the k8s admin route is published by external-dns (it owns just the
# k8s.syd1.au.unkin.net zone). git.unkin.net lives in the apex zone and is
# flipped at cutover — NOT managed here.
external-dns.alpha.kubernetes.io/hostname: git.k8s.syd1.au.unkin.net
external-dns.alpha.kubernetes.io/target: 198.18.200.4
spec:
gatewayClassName: traefik-internal
listeners:
- name: http
- name: http-primary
port: 80
protocol: HTTP
hostname: git2.k8s.syd1.au.unkin.net
hostname: git.unkin.net
allowedRoutes:
namespaces:
from: Same
- name: https
- name: https-primary
port: 443
protocol: HTTPS
hostname: git2.k8s.syd1.au.unkin.net
hostname: git.unkin.net
allowedRoutes:
namespaces:
from: Same
tls:
mode: Terminate
certificateRefs:
- group: ""
kind: Secret
name: gitea-tls
- name: http-admin
port: 80
protocol: HTTP
hostname: git.k8s.syd1.au.unkin.net
allowedRoutes:
namespaces:
from: Same
- name: https-admin
port: 443
protocol: HTTPS
hostname: git.k8s.syd1.au.unkin.net
allowedRoutes:
namespaces:
from: Same
+14 -4
View File
@@ -9,12 +9,17 @@ metadata:
app.kubernetes.io/instance: gitea
spec:
hostnames:
- git2.k8s.syd1.au.unkin.net
- git.unkin.net
- git.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: gitea
sectionName: http
sectionName: http-primary
- group: gateway.networking.k8s.io
kind: Gateway
name: gitea
sectionName: http-admin
rules:
- filters:
- type: RequestRedirect
@@ -36,12 +41,17 @@ metadata:
app.kubernetes.io/instance: gitea
spec:
hostnames:
- git2.k8s.syd1.au.unkin.net
- git.unkin.net
- git.k8s.syd1.au.unkin.net
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: gitea
sectionName: https
sectionName: https-primary
- group: gateway.networking.k8s.io
kind: Gateway
name: gitea
sectionName: https-admin
rules:
- backendRefs:
- group: ""
@@ -38,3 +38,14 @@ patches:
- op: replace
path: /spec/ports/0/targetPort
value: 3000
# SSH is disabled (DISABLE_SSH); drop the ssh Service the chart still renders.
- target:
version: v1
kind: Service
name: gitea-ssh
patch: |-
$patch: delete
apiVersion: v1
kind: Service
metadata:
name: gitea-ssh
+10 -19
View File
@@ -36,20 +36,12 @@ service:
http:
type: ClusterIP
port: 3000
# git-over-ssh. Each replica runs Gitea's built-in Go SSH server (rootless,
# listens 2222); the LoadBalancer fans connections across pods. Dedicated
# PureLB VIP + its own DNS name for the validation phase (see the migration
# doc for the git.unkin.net:2222 unification decision at cutover).
# git-over-ssh is disabled — HTTPS clones only (the estate norm). DISABLE_SSH
# below stops the in-pod SSH server; the chart still renders a gitea-ssh
# Service, which the overlay kustomization $patch-deletes.
ssh:
type: LoadBalancer
port: 2222
loadBalancerIP: 198.18.200.10
externalTrafficPolicy: Local
annotations:
purelb.io/addresses: 198.18.200.10
purelb.io/service-group: common
external-dns.alpha.kubernetes.io/hostname: git2-ssh.k8s.syd1.au.unkin.net
external-dns.alpha.kubernetes.io/target: 198.18.200.10
type: ClusterIP
port: 22
# Gateway API (HTTPRoute in the base) fronts HTTP; the chart Ingress is unused.
ingress:
@@ -133,12 +125,11 @@ gitea:
config:
server:
DOMAIN: git2.k8s.syd1.au.unkin.net
ROOT_URL: https://git2.k8s.syd1.au.unkin.net/
SSH_DOMAIN: git2-ssh.k8s.syd1.au.unkin.net
SSH_PORT: 2222 # advertised in clone URLs
SSH_LISTEN_PORT: 2222 # rootless in-container listener
START_SSH_SERVER: true
DOMAIN: git.unkin.net
ROOT_URL: https://git.unkin.net/
SSH_DOMAIN: git.unkin.net
DISABLE_SSH: true
START_SSH_SERVER: false
LFS_START_SERVER: true
database:
+29 -26
View File
@@ -14,8 +14,8 @@ for rollback.
| Repo/LFS storage | RWX PVC `gitea-shared-storage` (`cephfs-raid6-delete`, 20Gi) | `/shared/apps/gitea` |
| DB | CNPG `gitea-postgres` (2 instances) + pgbouncer, S3 backup `cnpg-gitea` 04:00/30d | Patroni-shared Postgres |
| Cache/session/queue | standalone Valkey (db 0/1/2, AOF) | Redis sentinel |
| SSH | LoadBalancer VIP `198.18.200.10:2222` (built-in per-pod SSH server) | host sshd, port 2222 |
| Ingress | Gateway API → `git2.k8s.syd1.au.unkin.net` (temp) | HAProxy `be_gitea` → git.unkin.net |
| SSH | **disabled** (`DISABLE_SSH`) — HTTPS clones only | host sshd, port 2222 |
| Ingress | Gateway API → `git.unkin.net` + `git.k8s.syd1.au.unkin.net` | HAProxy `be_gitea` → git.unkin.net |
| Actions | **disabled** (CI stays Woodpecker) | runner deployed separately |
| Container registry | **disabled** (moved to artifactapi `docker-internal`) | built-in, in use |
| Other package types | enabled (generic archives for *arr etc. still used) | built-in |
@@ -52,11 +52,13 @@ vault kv put kv/kubernetes/namespace/gitea/default/oauth-credentials \
## Stages
**0 — Validate on the temp host.** Merge this PR + the terraform-authentik PR
(gitea OIDC app). ArgoCD syncs the `gitea` app. Confirm: `git2.k8s.syd1.au.unkin.net`
serves, local-admin login, OIDC login (Authentik → auto-register/link), create a
test repo, HTTPS push/clone, SSH push/clone via `198.18.200.10:2222`, and a
Woodpecker webhook fires against the temp host.
**0 — Validate on the admin route.** Merge this PR + the terraform-authentik PR
(gitea OIDC app). ArgoCD syncs the `gitea` app; external-dns publishes
`git.k8s.syd1.au.unkin.net` → the gateway VIP (safe — new name, empty instance).
`git.unkin.net` stays on the live VM until stage 2. Confirm on the admin route:
local-admin login, OIDC login (Authentik → auto-register/link), create a test
repo, HTTPS push/clone (SSH is disabled — HTTPS only), and a Woodpecker webhook
fires against `git.k8s.syd1.au.unkin.net`.
**1 — Data migration rehearsal.** Gitea is already Postgres→Postgres, so **no DB
engine conversion** — a plain SQL dump/restore plus a filesystem copy of the data
@@ -82,18 +84,22 @@ Then set `gitea-inner` to the VM's real `SECRET_KEY`/`INTERNAL_TOKEN` (above) an
restart the deployment. Verify repos/issues/LFS/users. Rehearse fully before the
freeze so the freeze window is just a delta.
**2 — Freeze + cutover.** Put the VM read-only (stop Woodpecker/webhooks), take a
final dump/restore of the delta. Then:
- Add `git.unkin.net` to the Gateway: append it as a second listener hostname +
cert SAN (`cert-manager.io/common-name` stays, add the host to the listeners and
a matching HTTPRoute), and set `gitea.config.server.{DOMAIN,ROOT_URL,SSH_DOMAIN}`
back to `git.unkin.net`.
- Repoint DNS `git.unkin.net` → the internal Traefik VIP `198.18.200.4`. **Record
owner:** `git.unkin.net` currently resolves to the HAProxy VRRP anycast VIP
`198.18.19.17` (`roles::infra::halb::haproxy2 anycast_ip`); the A record lives in
the `main.unkin.net` zone (Puppet DNS `roles::infra::dns::master`). Update it
there (or via the bind estate) — **not** external-dns, which only owns the
`*.k8s.…` records.
**2 — Freeze + cutover (the `git.unkin.net` DNS flip).** Put the VM read-only (stop
Woodpecker/webhooks), take a final dump/restore of the delta. The Gateway already
serves `git.unkin.net` (listener + cert SAN) and the app already advertises
`ROOT_URL https://git.unkin.net/`, so cutover is **only the DNS flip**:
- Repoint `git.unkin.net` → the internal Traefik VIP `198.18.200.4`.
- **Where the record lives (reconciled):** `git.unkin.net` resolves TODAY via the
**Puppet DNS master** (`profiles::dns::master`, records from PuppetDB exported
resources) to the HAProxy VRRP VIP `198.18.19.17` → the VM. The GitOps home for
the `unkin.net` apex is the **k8s bind-operator** `unkin-net` BindZone
(`apps/base/bind-internal/authoritative/`), where `identity.unkin.net` already
lives as a DNSRecord — but that zone currently holds only SOA+NS + a few records
(the bulk apex migration off Puppet is a pending follow-up). So at cutover, flip
**whichever is the live authority for `unkin.net` at that moment**: either
uncomment the `git-dns-internal` DNSRecord in `records.yaml` (prepared, gated) if
the k8s bind cluster is live, or change the Puppet-side record. **Not**
external-dns — it owns only the `k8s.syd1.au.unkin.net` zone.
**3 — Consumer verification.** After DNS/cert flips:
@@ -112,16 +118,13 @@ VM was never modified, so rollback is a single record change.
## Open decisions / hard problems
- **SSH on git.unkin.net:2222.** One A record can't point at both the HTTPS gateway
VIP (`.4`) and a separate SSH VIP. Options: (a) HTTPS-only clones (estate norm,
SSH dropped); (b) a Traefik `TCPRoute` so `.4:2222` reaches the SSH service
(share one VIP); (c) keep SSH on a dedicated host/VIP. Validation uses (c) via
`git2-ssh.k8s.…:2222`.
- **SSH is disabled** (`DISABLE_SSH`) — HTTPS clones only, the estate norm. No SSH
VIP, no SSH DNS, nothing to cut over.
- **artifactapi's own image is circular** — it currently pulls `git.unkin.net/unkin/
artifactapi`. It can't pull itself from `docker-internal`. Keep it on an
external/bootstrap-safe registry (or node-preloaded) before the VM registry goes
away; not repointed in the registry-migration PR.
- **`forge_remote_id` stability** (Woodpecker) hinges on preserving Gitea repo IDs —
guaranteed by a full SQL restore, broken by any re-create/re-migrate path.
- **Cutover window**: freeze = final delta dump/restore + DNS TTL. Keep the
`main.unkin.net` record TTL low a day ahead.
- **Cutover window**: freeze = final delta dump/restore + DNS TTL. Lower the
`git.unkin.net` record TTL (600s in both zones) a day ahead so the flip is quick.