Compare commits

..

8 Commits

Author SHA1 Message Date
unkin-agent 6e0ac5fdbf Set VictoriaLogs cluster image version to v1.52.0
ci/woodpecker/pr/vector-test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/kubeconform Pipeline was successful
2026-09-26 23:43:56 +10:00
unkin-agent 74c092040a Pin VictoriaLogs cluster image version to v1.50.0
ci/woodpecker/pr/vector-test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/kubeconform Pipeline was successful
2026-09-26 23:39:52 +10:00
unkin-agent acf074ca3c Size vlstorage at 250Gi per node on cephrbd-fast-delete
ci/woodpecker/pr/vector-test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/kubeconform Pipeline was successful
2026-09-26 23:35:28 +10:00
unkin-agent f193c7e75a Size vlstorage for a 180d log store
ci/woodpecker/pr/vector-test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/kubeconform Pipeline was successful
- raise the vlstorage PVCs to 500Gi and the disk cap to 440GiB per node
- move vlstorage onto cephrbd-fast-retain
2026-09-26 19:02:34 +10:00
unkin-agent efdfd3eb4b Add a VictoriaLogs cluster and point logs-ingest at it
ci/woodpecker/pr/vector-test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/kubeconform Pipeline was successful
- add VLCluster logs (2 vlinsert, 2 vlselect, 3 vlstorage, 180d)
- repoint the logs-ingest HTTPRoute at vlinsert-logs:9481
- add a VictoriaLogs Grafana datasource and its plugin
2026-09-26 18:37:47 +10:00
unkin-agent 4190785389 Move the au-syd1 haproxy edge into Kubernetes (#485)
The au-syd1 edge proxy runs on a hand-managed LXD container outside the cluster, with no HA and no shared config source.

- Add `apps/base/haproxy/`: 3 replicas behind the DMZ LoadBalancer 198.18.199.1, config from a ConfigMap, wildcard certs from reflected secrets.
- Keep source IPs via `externalTrafficPolicy: Local`; `sessionAffinity: ClientIP` stands in for the stick-table peers a Deployment cannot name.
- Drain on shutdown: `hard-stop-after 2m`, a preStop SIGUSR1 soft-stop, 150s grace.
- Bind the stats listener to 127.0.0.1 so it is port-forward only.
- Register the app in the platform project and ApplicationSet.

Reviewed-on: #485
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-09-26 18:22:57 +10:00
unkin-agent 812a9a2f2b Publish real delegation records for acme.unkin.net (#486)
The acme.unkin.net zone still serves only the bind-operator seed apex: NS ns1.acme.unkin.net glued to A 10.42.6.38, a pod IP no pod holds. The parent delegates to acme-ns1.unkin.net, but public resolvers have already promoted the child NS RRset, so when the cached address expires DNS-01 fails for every unkin.net wildcard at once.

- Add apex NS acme-ns1.unkin.net., matching the parent delegation (out of zone, no glue needed).
- Point ns1.acme.unkin.net at 103.216.191.185 so resolvers holding the seeded NS name still reach the zone.
- The operator seed placeholder itself is tracked separately in bind-operator.

---------

Co-authored-by: unkin-agent <agent@unkin.net>
Reviewed-on: #486
Co-authored-by: Unkin Agent <unkin-agent@unkin.net>
Co-committed-by: Unkin Agent <unkin-agent@unkin.net>
2026-09-26 16:42:02 +10:00
unkin-agent f37749523d Add *.main and *.ceph wildcard certificates for haproxy (#484)
The haproxy edge terminates TLS for hosts under `main.unkin.net` and `ceph.unkin.net`, which the single `*.unkin.net` wildcard does not cover.

- Add cert-manager Certificates for both wildcards from the `letsencrypt` ClusterIssuer.
- Reflect the minted secrets into the `haproxy` namespace.

Needs these records in the public unkin.net zone first:
`_acme-challenge.main.unkin.net. CNAME _acme-challenge.main.acme.unkin.net.`
`_acme-challenge.ceph.unkin.net. CNAME _acme-challenge.ceph.acme.unkin.net.`

Reviewed-on: #484
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-09-26 16:38:31 +10:00
28 changed files with 899 additions and 28 deletions
+4 -4
View File
@@ -22,10 +22,10 @@ make kubeconform
make clean
# Quick build + inspect without persisting output
kustomize build --enable-helm --helm-kube-version "$(cat ci/kube-version)" apps/overlays/au-syd1/<app-name>
kustomize build --enable-helm apps/overlays/au-syd1/<app-name>
# Check all resource kinds produced by an overlay
kustomize build --enable-helm --helm-kube-version "$(cat ci/kube-version)" apps/overlays/au-syd1/<app-name> | grep "^kind:" | sort | uniq -c
kustomize build --enable-helm apps/overlays/au-syd1/<app-name> | grep "^kind:" | sort | uniq -c
# Run pre-commit checks against all files
uvx pre-commit run --all-files
@@ -118,7 +118,7 @@ In `argocd/projects/platform.yaml` (or `storage.yaml`):
### 5. Validate
```bash
kustomize build --enable-helm --helm-kube-version "$(cat ci/kube-version)" apps/overlays/au-syd1/<app-name>
kustomize build --enable-helm apps/overlays/au-syd1/<app-name>
make kubeconform
```
@@ -215,7 +215,7 @@ Examples:
Some overlays vendor Helm charts locally under `apps/overlays/au-syd1/<app-name>/charts/<chart-name>/`. When a chart is vendored, the overlay's `kustomization.yaml` references the local path. When not vendored, it references the OCI or HTTP repo directly.
The Kubernetes target version lives in `ci/kube-version`; the Makefile and CI scripts read it for both the Helm capabilities version and kubeconform.
Current Kubernetes target version: **1.33.7** (used by kubeconform in CI).
---
+2 -5
View File
@@ -3,11 +3,8 @@
# Build a kustomization path to manifests directory
# Usage: make build clusters/au-syd1/bootstrap
build:
@kube_version="$$(cat ci/kube-version)"; \
if [ -z "$$kube_version" ]; then echo "cannot read ci/kube-version" >&2; exit 1; fi; \
target="$(filter-out $@,$(MAKECMDGOALS))"; \
mkdir -p "manifests/$$target"; \
kustomize build --enable-helm --helm-kube-version "$$kube_version" "$$target" --output "manifests/$$target"
@mkdir -p manifests/$(filter-out $@,$(MAKECMDGOALS))
@kustomize build --enable-helm $(filter-out $@,$(MAKECMDGOALS)) --output manifests/$(filter-out $@,$(MAKECMDGOALS))
# Generate JSON schemas from CRDs and Kubernetes swagger spec (run manually, results committed)
schemas:
@@ -7,4 +7,5 @@ resources:
- cluster.yaml
- tsigkey.yaml
- zones.yaml
- records.yaml
- agent-dns-rolebinding.yaml
+36
View File
@@ -0,0 +1,36 @@
# Authoritative delegation records for acme.unkin.net. Without these the zone
# only holds the operator's seed apex (NS ns1.acme.unkin.net glued to the
# primary pod IP), which is unroutable off-cluster and goes stale on
# reschedule. DNSRecords must live in the same namespace as their BindZone.
---
apiVersion: bind.unkin.net/v1alpha1
kind: DNSRecord
metadata:
name: acme-apex-ns
namespace: bind-external
spec:
zoneRef: acme-unkin-net
# "@" is the zone apex.
name: "@"
type: NS
ttl: 3600
values:
# Matches the parent delegation in Google Cloud DNS. Out of zone, so the
# child needs no glue of its own.
- acme-ns1.unkin.net.
---
apiVersion: bind.unkin.net/v1alpha1
kind: DNSRecord
metadata:
name: acme-ns1-a
namespace: bind-external
spec:
zoneRef: acme-unkin-net
name: ns1
type: A
ttl: 3600
values:
# Public address of this cluster's external BIND, same target as
# acme-ns1.unkin.net. Resolvers that cached the seeded ns1.acme.unkin.net
# NS name must still reach the zone.
- 103.216.191.185
@@ -0,0 +1,26 @@
---
# Let's Encrypt *.ceph.unkin.net wildcard for the haproxy edge (ceph dashboard).
# DNS-01 needs the delegated _acme-challenge.ceph.unkin.net CNAME in the public
# unkin.net zone.
# _acme-challenge.ceph.unkin.net. CNAME _acme-challenge.ceph.acme.unkin.net.
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: wildcard-ceph-unkin-net
namespace: cert-manager
spec:
secretName: wildcard-ceph-unkin-net-tls
secretTemplate:
annotations:
reflector.v1.k8s.emberstack.com/reflection-allowed: "true"
reflector.v1.k8s.emberstack.com/reflection-allowed-namespaces: "haproxy"
reflector.v1.k8s.emberstack.com/reflection-auto-enabled: "true"
reflector.v1.k8s.emberstack.com/reflection-auto-namespaces: "haproxy"
privateKey:
size: 4096
dnsNames:
- "*.ceph.unkin.net"
issuerRef:
name: letsencrypt
kind: ClusterIssuer
group: cert-manager.io
@@ -0,0 +1,26 @@
---
# Let's Encrypt *.main.unkin.net wildcard for the haproxy edge (pve, arr stack,
# jellyfin, stalwart webadmin/autoconfig). DNS-01 needs the delegated
# _acme-challenge.main.unkin.net CNAME in the public unkin.net zone.
# _acme-challenge.main.unkin.net. CNAME _acme-challenge.main.acme.unkin.net.
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: wildcard-main-unkin-net
namespace: cert-manager
spec:
secretName: wildcard-main-unkin-net-tls
secretTemplate:
annotations:
reflector.v1.k8s.emberstack.com/reflection-allowed: "true"
reflector.v1.k8s.emberstack.com/reflection-allowed-namespaces: "haproxy"
reflector.v1.k8s.emberstack.com/reflection-auto-enabled: "true"
reflector.v1.k8s.emberstack.com/reflection-auto-namespaces: "haproxy"
privateKey:
size: 4096
dnsNames:
- "*.main.unkin.net"
issuerRef:
name: letsencrypt
kind: ClusterIssuer
group: cert-manager.io
@@ -14,9 +14,9 @@ spec:
secretTemplate:
annotations:
reflector.v1.k8s.emberstack.com/reflection-allowed: "true"
reflector.v1.k8s.emberstack.com/reflection-allowed-namespaces: "cheeztv,arrstack,authentik,gitea,watchstate,mediamark,repospawner"
reflector.v1.k8s.emberstack.com/reflection-allowed-namespaces: "cheeztv,arrstack,authentik,gitea,watchstate,mediamark,repospawner,haproxy"
reflector.v1.k8s.emberstack.com/reflection-auto-enabled: "true"
reflector.v1.k8s.emberstack.com/reflection-auto-namespaces: "cheeztv,arrstack,authentik,gitea,watchstate,mediamark,repospawner"
reflector.v1.k8s.emberstack.com/reflection-auto-namespaces: "cheeztv,arrstack,authentik,gitea,watchstate,mediamark,repospawner,haproxy"
privateKey:
size: 4096
dnsNames:
@@ -12,3 +12,5 @@ resources:
- clusterissuer_letsencrypt.yaml
- clusterissuer_letsencrypt-staging.yaml
- certificate_wildcard-unkin-net.yaml
- certificate_wildcard-main-unkin-net.yaml
- certificate_wildcard-ceph-unkin-net.yaml
+19
View File
@@ -20,3 +20,22 @@ spec:
jsonData:
timeInterval: "15s"
httpMethod: "POST"
---
apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaDatasource
metadata:
name: victorialogs
namespace: grafana
spec:
instanceSelector:
matchLabels:
dashboards: "grafana"
plugins:
- name: victoriametrics-logs-datasource
version: 0.32.0
datasource:
name: "VictoriaLogs"
type: "victoriametrics-logs-datasource"
uid: "victorialogs"
access: "proxy"
url: "http://vlselect-logs.logging.svc.cluster.local:9471"
+452
View File
@@ -0,0 +1,452 @@
---
apiVersion: v1
kind: ConfigMap
metadata:
name: haproxy-config
namespace: haproxy
data:
certificate.list: |
# First entry is the default cert for non-matching SNI.
/etc/haproxy/certs/unkin-net/tls.crt
/etc/haproxy/certs/main-unkin-net/tls.crt
/etc/haproxy/certs/ceph-unkin-net/tls.crt
fe_http.map: |
sonarr.main.unkin.net be_sonarr
radarr.main.unkin.net be_radarr
lidarr.main.unkin.net be_lidarr
readarr.main.unkin.net be_readarr
prowlarr.main.unkin.net be_prowlarr
nzbget.main.unkin.net be_nzbget
jellyfin.main.unkin.net be_jellyfin
fafflix.unkin.net be_jellyfin
git.unkin.net be_gitea
grafana.unkin.net be_grafana
dashboard.ceph.unkin.net be_ceph_dashboard
mail-webadmin.main.unkin.net be_stalwart_webadmin
autoconfig.main.unkin.net be_stalwart_webadmin
autodiscovery.main.unkin.net be_stalwart_webadmin
auth.unkin.net be_k8s_kanidm
fe_https.map: |
sonarr.main.unkin.net be_sonarr
radarr.main.unkin.net be_radarr
lidarr.main.unkin.net be_lidarr
readarr.main.unkin.net be_readarr
prowlarr.main.unkin.net be_prowlarr
nzbget.main.unkin.net be_nzbget
jellyfin.main.unkin.net be_jellyfin
fafflix.unkin.net be_jellyfin
git.unkin.net be_gitea
grafana.unkin.net be_grafana
dashboard.ceph.unkin.net be_ceph_dashboard
mail-webadmin.main.unkin.net be_stalwart_webadmin
autoconfig.main.unkin.net be_stalwart_webadmin
autodiscovery.main.unkin.net be_stalwart_webadmin
auth.unkin.net be_k8s_kanidm
haproxy.cfg: |
global
log stdout format raw local0
log stdout format raw local1 notice
maxconn 4000
hard-stop-after 2m
ssl-default-bind-ciphers EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH
ssl-default-bind-options ssl-min-ver TLSv1.2 ssl-max-ver TLSv1.3
ssl-default-server-ciphers kEECDH+aRSA+AES:kRSA+AES:+AES256:RC4-SHA:!kEDH:!LOW:!EXP:!MD5:!aNULL:!eNULL
ssl-default-server-options no-sslv3
stats timeout 30s
stats socket /var/lib/haproxy/stats
stats socket /var/lib/haproxy/admin.sock mode 660 level admin
tune.ssl.default-dh-param 2048
defaults
log global
maxconn 5000
mode http
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
stats enable
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 5m
timeout server 5m
timeout http-keep-alive 10s
timeout check 10s
frontend fe_http
bind 0.0.0.0:80
mode http
description Global HTTP Frontend
acl acl-letsencrypt path_beg /.well-known/acme-challenge/
http-request set-header X-Forwarded-Proto https
http-request set-header X-Real-IP %[src]
use_backend be_letsencrypt if acl-letsencrypt
use_backend %[req.hdr(host),lower,map(/usr/local/etc/haproxy/fe_http.map,be_default)]
frontend fe_https
bind 0.0.0.0:443 ssl crt-list /usr/local/etc/haproxy/certificate.list ciphers EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH force-tlsv12
mode http
description Global HTTPS Frontend
acl acl-letsencrypt path_beg /.well-known/acme-challenge/
acl acl_sonarr req.hdr(host) -i sonarr.main.unkin.net
acl acl_radarr req.hdr(host) -i radarr.main.unkin.net
acl acl_lidarr req.hdr(host) -i lidarr.main.unkin.net
acl acl_readarr req.hdr(host) -i readarr.main.unkin.net
acl acl_prowlarr req.hdr(host) -i prowlarr.main.unkin.net
acl acl_nzbget req.hdr(host) -i nzbget.main.unkin.net
acl acl_jellyfin req.hdr(host) -i jellyfin.main.unkin.net
acl acl_fafflix req.hdr(host) -i fafflix.unkin.net
acl acl_gitea req.hdr(host) -i git.unkin.net
acl acl_grafana req.hdr(host) -i grafana.unkin.net
acl acl_ceph_dashboard req.hdr(host) -i dashboard.ceph.unkin.net
acl acl_stalwart_webadmin req.hdr(host) -i mail-webadmin.main.unkin.net
acl acl_stalwart_webadmin req.hdr(host) -i autoconfig.main.unkin.net
acl acl_stalwart_webadmin req.hdr(host) -i autodiscovery.main.unkin.net
acl acl_kanidm req.hdr(host) -i auth.unkin.net
http-request set-header X-Forwarded-Proto https
http-request set-header X-Real-IP %[src]
http-response set-header X-Frame-Options DENY if acl_sonarr
http-response set-header X-Frame-Options DENY if acl_radarr
http-response set-header X-Frame-Options DENY if acl_lidarr
http-response set-header X-Frame-Options DENY if acl_readarr
http-response set-header X-Frame-Options DENY if acl_prowlarr
http-response set-header X-Frame-Options DENY if acl_nzbget
http-response set-header X-Frame-Options DENY if acl_jellyfin
http-response set-header X-Frame-Options DENY if acl_fafflix
http-response set-header X-Frame-Options DENY if acl_gitea
http-response set-header X-Frame-Options DENY if acl_grafana
http-response set-header X-Frame-Options DENY if acl_ceph_dashboard
http-response set-header X-Frame-Options DENY if acl_stalwart_webadmin
http-response set-header X-Frame-Options DENY if acl_kanidm
http-response set-header X-Content-Type-Options nosniff
http-response set-header X-XSS-Protection 1;mode=block
use_backend be_letsencrypt if acl-letsencrypt
use_backend %[req.hdr(host),lower,map(/usr/local/etc/haproxy/fe_https.map,be_default)]
frontend fe_imap
bind 0.0.0.0:143
mode tcp
description Frontend for Stalwart IMAP (STARTTLS)
default_backend be_stalwart_imap
log global
option tcplog
tcp-request inspect-delay 5s
tcp-request content accept if { req_len 0 }
frontend fe_imaps
bind 0.0.0.0:993
mode tcp
description Frontend for Stalwart IMAPS (implicit TLS)
default_backend be_stalwart_imaps
log global
option tcplog
tcp-request inspect-delay 5s
tcp-request content accept if { req_len 0 }
frontend fe_metrics
bind 0.0.0.0:8405
mode http
description Metrics Frontend
http-request set-header X-Forwarded-Proto https
http-request set-header X-Real-IP %[src]
http-request use-service prometheus-exporter if { path /metrics }
frontend fe_smtp
bind 0.0.0.0:25
mode tcp
description Frontend for Stalwart SMTP
default_backend be_stalwart_smtp
log global
option tcplog
tcp-request inspect-delay 5s
tcp-request content accept if { req_len 0 }
frontend fe_submission
bind 0.0.0.0:587
mode tcp
description Frontend for Stalwart SMTP Submission
default_backend be_stalwart_submission
log global
option tcplog
tcp-request inspect-delay 5s
tcp-request content accept if { req_len 0 }
backend be_ceph_dashboard
description Backend for Ceph Dashboard from Mgr instances
balance roundrobin
cookie SRVNAME insert indirect nocache
http-check expect status 200
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { dst_port 9443 }
http-reuse always
option httpchk GET /
option forwardfor
option http-keep-alive
option prefer-last-server
redirect scheme https if !{ ssl_fc }
stick-table type ip size 200k expire 30m
server prodnxsr0009 198.18.23.9:9443 check cookie prodnxsr0009 fall 2 inter 2s rise 3 ssl verify none
server prodnxsr0010 198.18.23.10:9443 check cookie prodnxsr0010 fall 2 inter 2s rise 3 ssl verify none
server prodnxsr0011 198.18.23.11:9443 check cookie prodnxsr0011 fall 2 inter 2s rise 3 ssl verify none
server prodnxsr0012 198.18.23.12:9443 check cookie prodnxsr0012 fall 2 inter 2s rise 3 ssl verify none
server prodnxsr0013 198.18.23.13:9443 check cookie prodnxsr0013 fall 2 inter 2s rise 3 ssl verify none
backend be_default
description Backend for unmatched HTTP traffic
balance roundrobin
cookie SRVNAME insert
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { dst_port 443 }
option httpchk GET /
option forwardfor
backend be_gitea
description Backend for gitea cluster
balance roundrobin
cookie SRVNAME insert indirect nocache
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { dst_port 443 }
http-reuse always
option httpchk GET /
option forwardfor
option http-keep-alive
option prefer-last-server
redirect scheme https if !{ ssl_fc }
stick on src
stick-table type ip size 200k expire 30m
server ausyd1nxvm2080 198.18.26.18:443 check cookie ausyd1nxvm2080 fall 2 inter 2s rise 3 ssl verify none
server ausyd1nxvm2081 198.18.27.117:443 check cookie ausyd1nxvm2081 fall 2 inter 2s rise 3 ssl verify none
server ausyd1nxvm2082 198.18.28.71:443 check cookie ausyd1nxvm2082 fall 2 inter 2s rise 3 ssl verify none
backend be_grafana
description Backend for grafana nodes
balance roundrobin
cookie SRVNAME insert indirect nocache
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { dst_port 443 }
http-reuse always
option httpchk GET /
option forwardfor
option http-keep-alive
option prefer-last-server
redirect scheme https if !{ ssl_fc }
stick on src
stick-table type ip size 200k expire 30m
server ausyd1nxvm2015 198.18.27.2:443 check cookie ausyd1nxvm2015 fall 2 inter 2s rise 3 ssl verify none
server ausyd1nxvm2016 198.18.28.189:443 check cookie ausyd1nxvm2016 fall 2 inter 2s rise 3 ssl verify none
backend be_jellyfin
description Backend for au-syd1 jellyfin
balance roundrobin
cookie SRVNAME insert indirect nocache
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { dst_port 443 }
http-reuse always
option httpchk GET /
option forwardfor
option http-keep-alive
option prefer-last-server
redirect scheme https if !{ ssl_fc }
server ausyd1nxvm2051 198.18.25.164:443 check cookie ausyd1nxvm2051 fall 2 inter 2s rise 3 ssl verify none
backend be_k8s_kanidm
description Backend for Kanidm (auth.unkin.net via Kubernetes internal Traefik)
balance roundrobin
http-reuse always
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { dst_port 443 }
redirect scheme https if !{ ssl_fc }
option httpchk
option forwardfor
option http-keep-alive
option prefer-last-server
http-check connect ssl sni auth.unkin.net
http-check send meth GET uri /status ver HTTP/1.1 hdr Host auth.unkin.net
http-check expect status 200
server k8s-traefik-internal 198.18.200.4:443 ssl verify none check inter 2s rise 3 fall 2 sni str(auth.unkin.net)
backend be_letsencrypt
description Backend for LetsEncrypt Verifications
balance roundrobin
server ausyd1nxvm2057 198.18.25.3:8888
backend be_lidarr
description Backend for au-syd1 lidarr
balance roundrobin
cookie SRVNAME insert indirect nocache
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { dst_port 443 }
http-reuse always
option httpchk GET /consul/health
option forwardfor
option http-keep-alive
option prefer-last-server
redirect scheme https if !{ ssl_fc }
server ausyd1nxvm2048 198.18.28.165:443 check cookie ausyd1nxvm2048 fall 2 inter 2s rise 3 ssl verify none
backend be_nzbget
description Backend for au-syd1 nzbget
balance roundrobin
cookie SRVNAME insert indirect nocache
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { dst_port 443 }
http-reuse always
option httpchk GET /consul/health
option forwardfor
option http-keep-alive
option prefer-last-server
redirect scheme https if !{ ssl_fc }
server ausyd1nxvm2045 198.18.25.44:443 check cookie ausyd1nxvm2045 fall 2 inter 2s rise 3 ssl verify none
backend be_prowlarr
description Backend for au-syd1 prowlarr
balance roundrobin
cookie SRVNAME insert indirect nocache
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { dst_port 443 }
http-reuse always
option httpchk GET /consul/health
option forwardfor
option http-keep-alive
option prefer-last-server
redirect scheme https if !{ ssl_fc }
server ausyd1nxvm2050 198.18.25.66:443 check cookie ausyd1nxvm2050 fall 2 inter 2s rise 3 ssl verify none
backend be_radarr
description Backend for au-syd1 radarr
balance roundrobin
cookie SRVNAME insert indirect nocache
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { dst_port 443 }
http-reuse always
option httpchk GET /consul/health
option forwardfor
option http-keep-alive
option prefer-last-server
redirect scheme https if !{ ssl_fc }
server ausyd1nxvm2047 198.18.27.131:443 check cookie ausyd1nxvm2047 fall 2 inter 2s rise 3 ssl verify none
backend be_readarr
description Backend for au-syd1 readarr
balance roundrobin
cookie SRVNAME insert indirect nocache
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { dst_port 443 }
http-reuse always
option httpchk GET /consul/health
option forwardfor
option http-keep-alive
option prefer-last-server
redirect scheme https if !{ ssl_fc }
server ausyd1nxvm2049 198.18.29.32:443 check cookie ausyd1nxvm2049 fall 2 inter 2s rise 3 ssl verify none
backend be_sonarr
description Backend for au-syd1 sonarr
balance roundrobin
cookie SRVNAME insert indirect nocache
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { dst_port 443 }
http-reuse always
option httpchk GET /consul/health
option forwardfor
option http-keep-alive
option prefer-last-server
redirect scheme https if !{ ssl_fc }
server ausyd1nxvm2046 198.18.26.161:443 check cookie ausyd1nxvm2046 fall 2 inter 2s rise 3 ssl verify none
backend be_stalwart_imap
description Backend for Stalwart IMAP (STARTTLS)
balance roundrobin
mode tcp
option tcp-check
option prefer-last-server
stick on src
stick-table type ip size 200k expire 30m
tcp-check connect port 143 send-proxy
tcp-check expect string "* OK"
tcp-check send "A001 STARTTLS\r\n"
tcp-check expect rstring "A001 (OK|2.0.0)"
server ausyd1nxvm2124 198.18.28.76:143 check fall 3 inter 3s rise 2 send-proxy-v2
server ausyd1nxvm2125 198.18.29.44:143 check fall 3 inter 3s rise 2 send-proxy-v2
server ausyd1nxvm2126 198.18.25.160:143 check fall 3 inter 3s rise 2 send-proxy-v2
backend be_stalwart_imaps
description Backend for Stalwart IMAPS (implicit TLS)
balance roundrobin
mode tcp
option tcp-check
option prefer-last-server
stick on src
stick-table type ip size 200k expire 30m
tcp-check connect ssl send-proxy
tcp-check expect string "* OK"
server ausyd1nxvm2124 198.18.28.76:993 check fall 3 inter 3s rise 2 send-proxy-v2 ssl verify none
server ausyd1nxvm2125 198.18.29.44:993 check fall 3 inter 3s rise 2 send-proxy-v2 ssl verify none
server ausyd1nxvm2126 198.18.25.160:993 check fall 3 inter 3s rise 2 send-proxy-v2 ssl verify none
backend be_stalwart_smtp
description Backend for Stalwart SMTP
balance roundrobin
mode tcp
option tcp-check
option prefer-last-server
stick on src
stick-table type ip size 200k expire 30m
tcp-check connect port 25 send-proxy
tcp-check expect string "220 "
server ausyd1nxvm2124 198.18.28.76:25 check fall 3 inter 3s rise 2 send-proxy-v2
server ausyd1nxvm2125 198.18.29.44:25 check fall 3 inter 3s rise 2 send-proxy-v2
server ausyd1nxvm2126 198.18.25.160:25 check fall 3 inter 3s rise 2 send-proxy-v2
backend be_stalwart_submission
description Backend for Stalwart SMTP Submission
balance roundrobin
mode tcp
option tcp-check
option prefer-last-server
stick on src
stick-table type ip size 200k expire 30m
tcp-check connect port 587 send-proxy
tcp-check expect string "220 "
server ausyd1nxvm2124 198.18.28.76:587 check fall 3 inter 3s rise 2 send-proxy-v2
server ausyd1nxvm2125 198.18.29.44:587 check fall 3 inter 3s rise 2 send-proxy-v2
server ausyd1nxvm2126 198.18.25.160:587 check fall 3 inter 3s rise 2 send-proxy-v2
backend be_stalwart_webadmin
description Backend for Stalwart Webadmin
balance roundrobin
cookie SRVNAME insert indirect nocache
http-check expect status 200
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { dst_port 9443 }
http-reuse always
option httpchk GET /
option forwardfor
option http-keep-alive
option prefer-last-server
redirect scheme https if !{ ssl_fc }
stick-table type ip size 200k expire 30m
server ausyd1nxvm2124 198.18.28.76:443 check cookie ausyd1nxvm2124 fall 2 inter 2s rise 3 send-proxy-v2 ssl verify none
server ausyd1nxvm2125 198.18.29.44:443 check cookie ausyd1nxvm2125 fall 2 inter 2s rise 3 send-proxy-v2 ssl verify none
server ausyd1nxvm2126 198.18.25.160:443 check cookie ausyd1nxvm2126 fall 2 inter 2s rise 3 send-proxy-v2 ssl verify none
# The `peers au-syd1-prod` section is dropped: peer names must be static and a
# Deployment cannot provide them. Service sessionAffinity: ClientIP pins a
# client to one replica so the per-replica stick-tables behave as before.
listen health
bind 0.0.0.0:8404
mode http
monitor-uri /healthz
listen stats
bind 127.0.0.1:9090
mode http
stats uri /
stats auth admin:admin
+162
View File
@@ -0,0 +1,162 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: haproxy
namespace: haproxy
annotations:
reloader.stakater.com/auto: "true"
spec:
replicas: 3
selector:
matchLabels:
app: haproxy
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
template:
metadata:
labels:
app: haproxy
spec:
automountServiceAccountToken: false
terminationGracePeriodSeconds: 150
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchLabels:
app: haproxy
topologyKey: kubernetes.io/hostname
securityContext:
runAsNonRoot: true
runAsUser: 99
runAsGroup: 99
seccompProfile:
type: RuntimeDefault
containers:
- name: haproxy
image: haproxy:3.2.24-alpine
imagePullPolicy: IfNotPresent
command:
- haproxy
- -W
- -db
- -f
- /usr/local/etc/haproxy/haproxy.cfg
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: [ALL]
# Frontends bind 25/80/143/443/587; the dst_port ACLs need the real ports.
add: [NET_BIND_SERVICE]
ports:
- name: http
containerPort: 80
protocol: TCP
- name: https
containerPort: 443
protocol: TCP
- name: smtp
containerPort: 25
protocol: TCP
- name: imap
containerPort: 143
protocol: TCP
- name: submission
containerPort: 587
protocol: TCP
- name: imaps
containerPort: 993
protocol: TCP
- name: health
containerPort: 8404
protocol: TCP
- name: metrics
containerPort: 8405
protocol: TCP
- name: stats
containerPort: 9090
protocol: TCP
lifecycle:
preStop:
exec:
# SIGUSR1 to the master soft-stops the workers; hard-stop-after
# caps the drain. Wait so kubelet holds SIGTERM until it is done.
command:
- /bin/sh
- -c
- kill -s USR1 1; while kill -0 1 2>/dev/null; do sleep 1; done
livenessProbe:
httpGet:
path: /healthz
port: health
initialDelaySeconds: 15
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
httpGet:
path: /healthz
port: health
initialDelaySeconds: 5
periodSeconds: 5
timeoutSeconds: 5
failureThreshold: 3
resources:
requests:
cpu: 200m
memory: 256Mi
limits:
cpu: 2
memory: 1Gi
volumeMounts:
- name: config
mountPath: /usr/local/etc/haproxy
readOnly: true
- name: cert-unkin-net
mountPath: /etc/haproxy/certs/unkin-net
readOnly: true
- name: cert-main-unkin-net
mountPath: /etc/haproxy/certs/main-unkin-net
readOnly: true
- name: cert-ceph-unkin-net
mountPath: /etc/haproxy/certs/ceph-unkin-net
readOnly: true
- name: run
mountPath: /var/lib/haproxy
volumes:
- name: config
configMap:
name: haproxy-config
# ssl-load-extra-files loads <crtfile>.key by default, so the key is
# projected next to the cert as tls.crt.key.
- name: cert-unkin-net
secret:
secretName: wildcard-unkin-net-tls
items:
- key: tls.crt
path: tls.crt
- key: tls.key
path: tls.crt.key
- name: cert-main-unkin-net
secret:
secretName: wildcard-main-unkin-net-tls
items:
- key: tls.crt
path: tls.crt
- key: tls.key
path: tls.crt.key
- name: cert-ceph-unkin-net
secret:
secretName: wildcard-ceph-unkin-net-tls
items:
- key: tls.crt
path: tls.crt
- key: tls.key
path: tls.crt.key
- name: run
emptyDir: {}
restartPolicy: Always
+12
View File
@@ -0,0 +1,12 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- namespace.yaml
- configmap.yaml
- deployment.yaml
- service.yaml
- pdb.yaml
- vpa.yaml
- vmpodscrape.yaml
+5
View File
@@ -0,0 +1,5 @@
---
apiVersion: v1
kind: Namespace
metadata:
name: haproxy
+11
View File
@@ -0,0 +1,11 @@
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: haproxy
namespace: haproxy
spec:
maxUnavailable: 1
selector:
matchLabels:
app: haproxy
+43
View File
@@ -0,0 +1,43 @@
---
apiVersion: v1
kind: Service
metadata:
name: haproxy
namespace: haproxy
annotations:
purelb.io/service-group: dmz
purelb.io/addresses: 198.18.199.1
spec:
type: LoadBalancer
loadBalancerIP: "198.18.199.1"
# Source IP must survive for acl_internalsubnets, X-Real-IP and SMTP.
externalTrafficPolicy: Local
# Pins a client to one replica, standing in for the dropped stick-table peers.
sessionAffinity: ClientIP
selector:
app: haproxy
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
- name: https
port: 443
protocol: TCP
targetPort: https
- name: smtp
port: 25
protocol: TCP
targetPort: smtp
- name: imap
port: 143
protocol: TCP
targetPort: imap
- name: submission
port: 587
protocol: TCP
targetPort: submission
- name: imaps
port: 993
protocol: TCP
targetPort: imaps
+13
View File
@@ -0,0 +1,13 @@
---
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMPodScrape
metadata:
name: haproxy
namespace: haproxy
spec:
selector:
matchLabels:
app: haproxy
podMetricsEndpoints:
- port: metrics
path: /metrics
+13
View File
@@ -0,0 +1,13 @@
---
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: haproxy-vpa
namespace: haproxy
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: haproxy
updatePolicy:
updateMode: "Off"
+3 -6
View File
@@ -1,16 +1,13 @@
---
# Log ingestion endpoint for puppet-managed VMs (and any non-k8s client).
# Reuses the internal Traefik gateway + cert-manager + external-dns pattern so
# VMs reach the Vector aggregator's HTTP source over TLS at a DNS name they can
# resolve. The puppet-side Vector rollout ships NDJSON to
# https://logs-ingest.k8s.syd1.au.unkin.net/ (a later task).
# Log ingestion endpoint for puppet-managed VMs (and any non-k8s client):
# fronts the VLCluster vlinsert service over TLS at a name VMs can resolve.
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: logs-ingest
namespace: logging
labels:
app.kubernetes.io/name: vector-aggregator
app.kubernetes.io/name: victorialogs
app.kubernetes.io/component: ingest
traefik.io/instance: internal
annotations:
+4 -4
View File
@@ -5,7 +5,7 @@ metadata:
name: logs-ingest-http-redirect
namespace: logging
labels:
app.kubernetes.io/name: vector-aggregator
app.kubernetes.io/name: victorialogs
app.kubernetes.io/component: ingest
spec:
hostnames:
@@ -32,7 +32,7 @@ metadata:
name: logs-ingest
namespace: logging
labels:
app.kubernetes.io/name: vector-aggregator
app.kubernetes.io/name: victorialogs
app.kubernetes.io/component: ingest
spec:
hostnames:
@@ -46,8 +46,8 @@ spec:
- backendRefs:
- group: ""
kind: Service
name: vector-vm-ingest
port: 8080
name: vlinsert-logs
port: 9481
weight: 1
matches:
- path:
+1
View File
@@ -10,6 +10,7 @@ resources:
- job_clickhouse-schema.yaml
- nats-bootstrap-job.yaml
- cephrgw.yaml
- vlcluster.yaml
- gateway.yaml
- httproute.yaml
- serviceaccount_logarchiver.yaml
+47
View File
@@ -0,0 +1,47 @@
---
apiVersion: operator.victoriametrics.com/v1
kind: VLCluster
metadata:
name: logs
namespace: logging
spec:
clusterVersion: v1.52.0
vlinsert:
replicaCount: 2
resources:
requests:
cpu: 500m
memory: 1Gi
limits:
cpu: "2"
memory: 4Gi
vlselect:
replicaCount: 2
resources:
requests:
cpu: 500m
memory: 1Gi
limits:
cpu: "2"
memory: 4Gi
vlstorage:
replicaCount: 3
retentionPeriod: 180d
# ~3 GiB/day measured; 220GiB/node cap keeps 180d time-based, not disk-bound
retentionMaxDiskSpaceUsageBytes: 220GiB
storage:
volumeClaimTemplate:
spec:
accessModes:
- ReadWriteOnce
storageClassName: cephrbd-fast-delete
resources:
requests:
storage: 250Gi
resources:
requests:
cpu: "1"
memory: 2Gi
limits:
cpu: "4"
memory: 8Gi
@@ -0,0 +1,6 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../../base/haproxy
+1
View File
@@ -29,6 +29,7 @@ spec:
- path: apps/overlays/*/ghp
- path: apps/overlays/*/gitea
- path: apps/overlays/*/grafana-system
- path: apps/overlays/*/haproxy
- path: apps/overlays/*/inteldeviceplugins-system
- path: apps/overlays/*/jfrog
- path: apps/overlays/*/k8up-system
+2
View File
@@ -43,6 +43,8 @@ spec:
server: https://kubernetes.default.svc
- namespace: 'gitea'
server: https://kubernetes.default.svc
- namespace: 'haproxy'
server: https://kubernetes.default.svc
- namespace: 'jfrog'
server: https://kubernetes.default.svc
- namespace: 'kanidm'
-1
View File
@@ -1 +0,0 @@
1.33.13
+2 -2
View File
@@ -1,7 +1,7 @@
#!/usr/bin/env bash
set -euo pipefail
KUBE_VERSION="$(cat ci/kube-version)"
KUBE_VERSION="1.33.7"
SCHEMA_DIR="${SCHEMA_DIR:-schemas}"
@@ -15,7 +15,7 @@ while IFS= read -r -d "" k; do
dir="$(dirname "$k")"
echo "==> kubeconform: $dir" >&2
kustomize build --enable-helm --helm-kube-version "$KUBE_VERSION" "$dir" \
kustomize build --enable-helm "$dir" \
| kubeconform \
-kubernetes-version "$KUBE_VERSION" \
-summary \
+2 -2
View File
@@ -1,7 +1,7 @@
#!/usr/bin/env bash
set -euo pipefail
KUBE_VERSION="$(cat ci/kube-version)"
KUBE_VERSION="1.33.7"
SCHEMA_DIR="${SCHEMA_DIR:-schemas}"
@@ -15,7 +15,7 @@ while IFS= read -r -d "" k; do
dir="$(dirname "$k")"
echo "==> kubeconform: $dir" >&2
kustomize build --enable-helm --helm-kube-version "$KUBE_VERSION" "$dir" \
kustomize build --enable-helm "$dir" \
| kubeconform \
-kubernetes-version "$KUBE_VERSION" \
-summary \
+2 -2
View File
@@ -49,8 +49,8 @@
- Use existing patterns like `apps/overlays/*/csi-*` or `apps/overlays/*/<app-name>`
### 9. Validation
- Run `kustomize build --enable-helm --helm-kube-version "$(cat ci/kube-version)" apps/overlays/au-syd1/<app-name>` to generate all resources
- Check resource types: `kustomize build --enable-helm --helm-kube-version "$(cat ci/kube-version)" apps/overlays/au-syd1/<app-name> | grep "^kind:" | sort | uniq -c`
- Run `kustomize build --enable-helm apps/overlays/au-syd1/<app-name>` to generate all resources
- Check resource types: `kustomize build --enable-helm apps/overlays/au-syd1/<app-name> | grep "^kind:" | sort | uniq -c`
- Verify all resource types are permitted in the target project's `clusterResourceWhitelist` and `namespaceResourceWhitelist`
- Run `make kubeconform` to validate all resources
- Fix any validation errors