Compare commits

...

23 Commits

Author SHA1 Message Date
unkin-agent d1565b3026 Require explicit RUNNER_MODE in runner job-started hook
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/kubeconform Pipeline was successful
2026-11-16 23:08:36 +11:00
unkin-agent dcc7057897 add iuclabs-release runner scale set gated to v* tag pushes
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/kubeconform Pipeline was successful
2026-11-16 23:03:34 +11:00
unkin-agent e9a27d3377 grafana: add router/firewall dashboard for prodnxsr0020 (#588)
prodnxsr0020 has no single view of its interface bandwidth, conntrack/softnet health, shorewall drop/reject logs and OSPF state.

- add `router-firewall` GrafanaDashboard: overview stats, per-interface bandwidth (Internet/Lab/Home), system, firewall (VictoriaLogs) and routing rows
- firewall panels count rate-limited kernel LOG lines (logged events, not packets)
- host set via a hidden `host` constant

Reviewed-on: #588
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-11-16 21:31:06 +11:00
unkin-agent e63b85d5a5 vlogs: accept Vault identity JWTs and run oauth2-proxy HA (#587)
Agents need programmatic vlogs access with short-lived Vault identity OIDC tokens, and the oauth2-proxy runs as a single replica.

- accept Vault identity JWT bearer tokens (audience `victorialogs`) via extra JWT issuer
- run oauth2-proxy with 2 replicas spread across nodes
- add a PodDisruptionBudget (minAvailable 1)

requires terraform-vault #179 applied first

Reviewed-on: #587
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-11-16 21:30:39 +11:00
unkin-agent 04f9d37a7f ceph-system: serve rgw over TLS on a PureLB IP (#584)
The in-cluster RGW only listens on plain HTTP via a ClusterIP, so S3 clients outside the cluster cannot reach it. TLS terminates in beast to avoid proxy body-size and timeout limits on large uploads.

- add a vault-issuer Certificate covering the existing radosgw SANs, rgw.k8s.syd1.au.unkin.net and 198.18.200.11
- mount the cert and enable beast ssl_port 7443, keeping 7480 for probes
- add rgw-external LoadBalancer on 198.18.200.11:443 with externalTrafficPolicy Local

Part of #582

Reviewed-on: #584
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-11-15 22:19:06 +11:00
unkin-agent 26e53ee35c ceph-system: run radosgw as a deployment (#583)
Part of #582. RGW runs on VMs (ausyd1nxvm2115-2119) and needs to move into k8s under GitOps; this adds the `ceph-system` namespace with an in-cluster RGW behind a ClusterIP service. Cutover comes later.

- add `ceph-system` app to the platform applicationset
- add ceph.conf configmap with a shared `client.rgw.syd1` section and explicit keyring path
- add rgw keyring from vault `kubernetes/namespace/ceph-system/default/rgw-keyring` (`keyring`)
- add 3-replica `quay.io/ceph/ceph:v18.2.8` radosgw deployment, non-root, read-only root, startup probe, hard per-node spread, PDB + VPA
- add ClusterIP service `rgw` on 7480

Requires unkin/terraform-artifactapi#78 merged first.

Reviewed-on: #583
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-11-15 21:51:59 +11:00
unkin-agent 166605d3f1 iuclabs-runners: trust Vault CA in runner pods (#581)
Workflows on the iuclabs-k8s scale set log into https://vault.service.consul:8200, which serves a cert from the internal unkin.net CA that the runner image does not trust, so vault-action and curl fail TLS verification.

- Mount the reflected `vault-ca-cert` Secret read-only at `/etc/vault-ca` in the runner container.
- Set `NODE_EXTRA_CA_CERTS=/etc/vault-ca/ca.crt` so Node actions (vault-action) trust it alongside public CAs.
- curl callers pass `--cacert /etc/vault-ca/ca.crt`; `SSL_CERT_FILE` stays untouched to keep public TLS working.

Reviewed-on: #581
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-11-14 22:02:55 +11:00
unkin-agent 283ccde24f Enable ARC metrics with VictoriaMetrics scrapes and dashboard (#580)
ARC runs with metrics disabled, so scale-set behaviour (assigned jobs, desired/idle runners, job startup time) and controller health are invisible.

- enable controller and listener metrics on :8080 via chart `metrics:` values
- add VMPodScrapes for the controller and listener pods in arc-system
- keep runner-namespace labels on listener/ephemeral-runner series so the dashboard filters match
- add the upstream ARC Autoscaling Runner Set dashboard (gha-runner-scale-set-0.15.1) resolved to VictoriaMetrics

Reviewed-on: #580
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-11-11 00:17:16 +11:00
unkin-agent fd3446ae8a Add ARC self-hosted runners for IUCLabs (#579)
IUCLabs public repos need self-hosted CI capacity, but anyone can open PRs there, so runners must only execute trusted code and must not reach internal services.

- deploy the ARC controller chart in arc-system, pulled through artifactapi
- keep shared runner resources (namespace, manager Role/RoleBinding, VSO secret, hooks) in the iuclabs-runners base
- package each scale set as a kustomize component (AutoscalingRunnerSet + its HTTPS/Vault egress policy), starting with iuclabs-k8s
- allow only push/dispatch/schedule/merge_group/release and trusted pull_request jobs via a job-started hook, tested in pre-commit
- default-deny the namespace and allow DNS

Depends on the github-app secret existing in Vault.

Reviewed-on: #579
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-11-10 23:27:40 +11:00
unkin-agent 19f7919851 puppet: bump compiler encapic to v0.4.0 (#578)
Keep the k8s puppetserver compilers on the current encapic release, matching the VM masters.

- Bump `ENCAPIC_VERSION` in the compiler initContainer from v0.1.0 to v0.4.0.

Reviewed-on: #578
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-11-08 22:14:48 +11:00
unkin-agent 7c4a01785b encapi: bump to v0.3.0 (#577)
Ships the encapi testing restore endpoint (restore a node from testing to its recorded previous environment).

- Bump encapi image pin v0.2.0 -> v0.3.0 in apps/base/encapi/deployment.yaml
- v0.3.0 adds migration 0002 (previous_environment column), applied by encapi on startup

Reviewed-on: #577
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-11-08 17:53:48 +11:00
unkin-agent 592b2ec9e2 Mount per-pod emptyDir for puppetdb logs (#576)
Both puppetdb replicas write logback logs to the same logs/ dir on the shared RWX PVC. Concurrent rollovers race on compression and leave orphaned .tmp files that maxHistory never removes, eventually filling the volume quota and failing /pdb/cmd/v1 with "Disk quota exceeded".

- mount a per-pod emptyDir (sizeLimit 512Mi) at /opt/puppetlabs/server/data/puppetdb/logs

Reviewed-on: #576
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-11-08 17:53:23 +11:00
unkin-agent dadada0274 Bump artifactapi to v3.25.2 (#575)
v3.25.2 preserves repo_type on remote update (with a repair migration), fixing 404s on virtual repo redirects, and lets registry /v2/token accept OAuth2 POST form credentials, fixing 401s on buildkit/helm OCI pushes.

- bump artifactapi API image to v3.25.2
- bump artifactapi-ui image to v3.25.2
- bump image-keeper api and ui images to v3.25.2

Reviewed-on: #575
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-11-08 11:56:51 +11:00
unkin-agent a4efca8ee5 Require tokens for artifactapi uploads (#574)
Woodpecker uploaders now mint Vault OIDC tokens, so anonymous local-repo writes can be rejected.

- set `ARTIFACTAPI_AUTH_UPLOADS: "true"` in the artifactapi `api-env` configmap

Merge after unkin/vault-plugin-secrets-litellm#6.

Reviewed-on: #574
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-10-26 20:33:29 +11:00
unkin-agent 0552be6028 Bump artifactapi to v3.25.0 (#573)
Artifactapi v3.25.0 adds github_rpm cache_repodata toggle and general fixes.

- Bump artifactapi and artifactapi-ui images to v3.25.0

Reviewed-on: #573
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-10-25 23:45:54 +11:00
unkin-agent 4b5d14cf52 Bump artifactapi to v3.24.0 (#572)
Pull in artifactapi v3.24.0 images (per-remote sync_interval and GitHub sync spreading).

- Update artifactapi image from v3.23.0 to v3.24.0
- Update artifactapi-ui image from v3.23.0 to v3.24.0

Reviewed-on: #572
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-10-25 20:25:21 +11:00
unkin-agent ec35ef9830 bump artifactapi to v3.23.0 (#571)
v3.23.0 adds virtual alpine repositories and drops local files and metadata when a repo is deleted.

- pin api, ui and image-keeper to v3.23.0

Reviewed-on: #571
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-10-24 20:40:34 +11:00
unkin-agent bb0cc51e0a bump artifactapi to v3.22.0 (#570)
v3.22.0 adds the vcs remote type for git provider source archives and release assets.

- pin api, ui and image-keeper to v3.22.0

Reviewed-on: #570
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-10-24 17:28:37 +11:00
unkin-agent bcde44ffab bind-resolvers: RBL zones as BindZones (#569)
bind-operator v0.4.0 renders a forward BindZone without forwarders as `forwarders { };`, so the RBL direct-resolution zones no longer need raw zone clauses in the openforwarder view extraOptions.

- add rbl-zones.yaml with one forwarder-less forward BindZone per RBL domain
- remove the raw RBL zone lines from the openforwarder view

Reviewed-on: #569
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-10-24 17:06:09 +11:00
unkin-agent 90600e3840 bump artifactapi to v3.21.0 (#568)
v3.21.0 adds virtual deb repositories.

- pin api, ui and image-keeper to v3.21.0

Reviewed-on: #568
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-10-24 16:41:34 +11:00
unkin-agent 2dad204428 Deploy encapi v0.2.0 with Vault OIDC (#567)
encapi v0.2.0 authenticates writes with Vault identity OIDC tokens; it must validate the issuer, which is served with the internal Vault CA.

- bump encapi image to v0.2.0
- set ENCAPI_OIDC_ISSUER to the Vault identity OIDC issuer
- trust the Vault CA via a combined bundle and SSL_CERT_FILE, matching artifactapi

Reviewed-on: #567
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-10-23 23:02:00 +11:00
unkin-agent f4cd6160ab Raise puppet-postgres memory (#566)
The puppet-postgres primary is repeatedly OOMKilled at its 512Mi limit, causing repeated failovers; one left a replica diverged and serving stale PuppetDB reads.

- raise memory request 256Mi -> 1Gi and limit 512Mi -> 2Gi
- raise shared_buffers 128MB -> 256MB
- raise effective_cache_size 256MB -> 1GB

Reviewed-on: #566
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-10-23 21:25:31 +11:00
unkin-agent 45aa23942f bump artifactapi to v3.20.0 (#565)
v3.20.0 adds virtual puppet repositories and filtered remote Forge release listings.

- pin api, ui and image-keeper to v3.20.0

Reviewed-on: #565
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-10-20 22:25:18 +11:00
66 changed files with 1262 additions and 22 deletions
+6
View File
@@ -42,3 +42,9 @@ repos:
entry: ci/validate-no-secrets.sh
language: system
pass_filenames: false
- id: runner_hooks
name: test arc runner job-started hook
entry: ci/test-runner-hooks.sh
language: system
pass_filenames: false
files: ^(apps/base/iuclabs-runners/job-started\.sh|ci/test-runner-hooks\.sh|ci/fixtures/runner-hooks/)
+8
View File
@@ -0,0 +1,8 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- namespace.yaml
- pdb.yaml
- vmpodscrape.yaml
+5
View File
@@ -0,0 +1,5 @@
---
apiVersion: v1
kind: Namespace
metadata:
name: arc-system
+12
View File
@@ -0,0 +1,12 @@
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: arc-controller
namespace: arc-system
spec:
minAvailable: 1
selector:
matchLabels:
app.kubernetes.io/name: gha-rs-controller
app.kubernetes.io/instance: arc
+40
View File
@@ -0,0 +1,40 @@
---
# Controller metrics. gha_controller_*_ephemeral_runners carry the runner
# namespace, which the dashboard filters on, so restore it over the target's.
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMPodScrape
metadata:
name: arc-controller
namespace: arc-system
spec:
selector:
matchLabels:
app.kubernetes.io/name: gha-rs-controller
app.kubernetes.io/instance: arc
podMetricsEndpoints:
- port: metrics
path: /metrics
metricRelabelConfigs:
- sourceLabels: [__name__, exported_namespace]
regex: gha_controller_.+_ephemeral_runners;(.+)
targetLabel: namespace
---
# Listener metrics carry their own name/namespace (the runner scale set), and
# the dashboard also expects the scale-set pod labels.
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMPodScrape
metadata:
name: arc-listener
namespace: arc-system
spec:
selector:
matchLabels:
app.kubernetes.io/component: runner-scale-set-listener
app.kubernetes.io/part-of: gha-runner-scale-set
podMetricsEndpoints:
- port: metrics
path: /metrics
honorLabels: true
relabelConfigs:
- action: labelmap
regex: __meta_kubernetes_pod_label_(actions_github_com_scale_set_.+)
+1 -1
View File
@@ -36,7 +36,7 @@ spec:
mountPath: /combined-certs
containers:
- name: api
image: artifactapi.k8s.syd1.au.unkin.net/docker-internal/artifactapi:v3.19.0
image: artifactapi.k8s.syd1.au.unkin.net/docker-internal/artifactapi:v3.25.2
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8000
+1
View File
@@ -5,6 +5,7 @@ metadata:
name: api-env
namespace: artifactapi
data:
ARTIFACTAPI_AUTH_UPLOADS: "true"
ARTIFACTAPI_OIDC_AUDIENCE: artifactapi
ARTIFACTAPI_OIDC_ISSUER: https://vault.service.consul:8200/v1/identity/oidc
DBHOST: postgres-pooler
+2 -2
View File
@@ -49,7 +49,7 @@ spec:
capabilities:
drop: ["ALL"]
- name: api
image: artifactapi.k8s.syd1.au.unkin.net/docker-internal/artifactapi:v3.19.0
image: artifactapi.k8s.syd1.au.unkin.net/docker-internal/artifactapi:v3.25.2
imagePullPolicy: IfNotPresent
command: ["/keeper/true"]
volumeMounts:
@@ -69,7 +69,7 @@ spec:
capabilities:
drop: ["ALL"]
- name: ui
image: artifactapi.k8s.syd1.au.unkin.net/docker-internal/artifactapi-ui:v3.19.0
image: artifactapi.k8s.syd1.au.unkin.net/docker-internal/artifactapi-ui:v3.25.2
imagePullPolicy: IfNotPresent
command: ["true"]
resources:
+1 -1
View File
@@ -22,7 +22,7 @@ spec:
automountServiceAccountToken: true
containers:
- name: ui
image: artifactapi.k8s.syd1.au.unkin.net/docker-internal/artifactapi-ui:v3.19.0
image: artifactapi.k8s.syd1.au.unkin.net/docker-internal/artifactapi-ui:v3.25.2
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
@@ -7,3 +7,4 @@ resources:
- acls.yaml
- view.yaml
- forward-zones.yaml
- rbl-zones.yaml
@@ -0,0 +1,121 @@
# DNSBLs refuse queries relayed via public resolvers, so these zones resolve directly.
---
apiVersion: bind.unkin.net/v1alpha1
kind: BindZone
metadata:
name: rbl-spamhaus-org
namespace: bind-internal
spec:
clusterRef: bind-resolvers
viewRef: openforwarder
zoneName: spamhaus.org
type: forward
catalog: false
---
apiVersion: bind.unkin.net/v1alpha1
kind: BindZone
metadata:
name: rbl-mailspike-net
namespace: bind-internal
spec:
clusterRef: bind-resolvers
viewRef: openforwarder
zoneName: mailspike.net
type: forward
catalog: false
---
apiVersion: bind.unkin.net/v1alpha1
kind: BindZone
metadata:
name: rbl-spameatingmonkey-net
namespace: bind-internal
spec:
clusterRef: bind-resolvers
viewRef: openforwarder
zoneName: spameatingmonkey.net
type: forward
catalog: false
---
apiVersion: bind.unkin.net/v1alpha1
kind: BindZone
metadata:
name: rbl-dnswl-org
namespace: bind-internal
spec:
clusterRef: bind-resolvers
viewRef: openforwarder
zoneName: dnswl.org
type: forward
catalog: false
---
apiVersion: bind.unkin.net/v1alpha1
kind: BindZone
metadata:
name: rbl-virusfree-cz
namespace: bind-internal
spec:
clusterRef: bind-resolvers
viewRef: openforwarder
zoneName: virusfree.cz
type: forward
catalog: false
---
apiVersion: bind.unkin.net/v1alpha1
kind: BindZone
metadata:
name: rbl-blocklist-de
namespace: bind-internal
spec:
clusterRef: bind-resolvers
viewRef: openforwarder
zoneName: blocklist.de
type: forward
catalog: false
---
apiVersion: bind.unkin.net/v1alpha1
kind: BindZone
metadata:
name: rbl-msbl-org
namespace: bind-internal
spec:
clusterRef: bind-resolvers
viewRef: openforwarder
zoneName: msbl.org
type: forward
catalog: false
---
apiVersion: bind.unkin.net/v1alpha1
kind: BindZone
metadata:
name: rbl-surbl-org
namespace: bind-internal
spec:
clusterRef: bind-resolvers
viewRef: openforwarder
zoneName: surbl.org
type: forward
catalog: false
---
apiVersion: bind.unkin.net/v1alpha1
kind: BindZone
metadata:
name: rbl-uribl-com
namespace: bind-internal
spec:
clusterRef: bind-resolvers
viewRef: openforwarder
zoneName: uribl.com
type: forward
catalog: false
---
apiVersion: bind.unkin.net/v1alpha1
kind: BindZone
metadata:
name: rbl-rspamd-com
namespace: bind-internal
spec:
clusterRef: bind-resolvers
viewRef: openforwarder
zoneName: rspamd.com
type: forward
catalog: false
@@ -23,14 +23,3 @@ spec:
- "allow-query-cache { any; }"
# pods have no IPv6 route; v6 NS attempts burn the recursion-query budget
- "server ::/0 { bogus yes; }"
# RBL zones resolve directly: DNSBLs refuse queries relayed via public resolvers.
- 'zone "spamhaus.org" { type forward; forwarders { }; }'
- 'zone "mailspike.net" { type forward; forwarders { }; }'
- 'zone "spameatingmonkey.net" { type forward; forwarders { }; }'
- 'zone "dnswl.org" { type forward; forwarders { }; }'
- 'zone "virusfree.cz" { type forward; forwarders { }; }'
- 'zone "blocklist.de" { type forward; forwarders { }; }'
- 'zone "msbl.org" { type forward; forwarders { }; }'
- 'zone "surbl.org" { type forward; forwarders { }; }'
- 'zone "uribl.com" { type forward; forwarders { }; }'
- 'zone "rspamd.com" { type forward; forwarders { }; }'
+26
View File
@@ -0,0 +1,26 @@
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: rgw-tls
namespace: ceph-system
labels:
app.kubernetes.io/name: rgw
spec:
secretName: rgw-tls
issuerRef:
kind: ClusterIssuer
name: vault-issuer
commonName: s3.ceph.unkin.net
dnsNames:
- s3.ceph.unkin.net
- rgw.k8s.syd1.au.unkin.net
- radosgw.service.consul
- radosgw.query.consul
- radosgw.service.au-syd1.consul
- radosgw.main.unkin.net
ipAddresses:
- 198.18.200.11
privateKey:
algorithm: RSA
size: 4096
+26
View File
@@ -0,0 +1,26 @@
---
apiVersion: v1
kind: ConfigMap
metadata:
name: ceph-conf
namespace: ceph-system
data:
ceph.conf: |
[global]
fsid = de96a98f-3d23-465a-a899-86d3d67edab8
mon_host = 198.18.23.9,198.18.23.10,198.18.23.11,198.18.23.12,198.18.23.13
auth_client_required = cephx
auth_cluster_required = cephx
auth_service_required = cephx
ms_bind_ipv4 = true
ms_bind_ipv6 = false
[client.rgw.syd1]
rgw_realm = unkin
rgw_zonegroup = au
rgw_zone = syd1
rgw_frontends = beast port=7480 ssl_port=7443 ssl_certificate=/etc/rgw/tls/tls.crt ssl_private_key=/etc/rgw/tls/tls.key
keyring = /var/lib/ceph/radosgw/ceph-rgw.syd1/keyring
log_to_file = false
log_to_stderr = true
err_to_stderr = true
+133
View File
@@ -0,0 +1,133 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: rgw
namespace: ceph-system
labels:
app.kubernetes.io/name: rgw
app.kubernetes.io/component: rgw
app.kubernetes.io/version: v18.2.8
annotations:
reloader.stakater.com/auto: "true"
spec:
replicas: 3
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 0
selector:
matchLabels:
app.kubernetes.io/name: rgw
template:
metadata:
labels:
app.kubernetes.io/name: rgw
app.kubernetes.io/component: rgw
spec:
automountServiceAccountToken: false
securityContext:
runAsNonRoot: true
runAsUser: 167
runAsGroup: 167
fsGroup: 167
seccompProfile:
type: RuntimeDefault
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
topologyKey: kubernetes.io/hostname
labelSelector:
matchLabels:
app.kubernetes.io/name: rgw
topologySpreadConstraints:
- maxSkew: 1
topologyKey: kubernetes.io/hostname
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app.kubernetes.io/name: rgw
containers:
- name: rgw
image: quay.io/ceph/ceph:v18.2.8
imagePullPolicy: IfNotPresent
command:
- radosgw
- -f
- --cluster
- ceph
- --name
- client.rgw.syd1
ports:
- name: http
containerPort: 7480
- name: https
containerPort: 7443
startupProbe:
httpGet:
path: /
port: http
periodSeconds: 5
failureThreshold: 30
readinessProbe:
httpGet:
path: /
port: http
initialDelaySeconds: 5
periodSeconds: 10
livenessProbe:
httpGet:
path: /
port: http
periodSeconds: 20
failureThreshold: 6
resources:
requests:
cpu: "1"
memory: 2Gi
limits:
cpu: "2"
memory: 4Gi
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: ["ALL"]
volumeMounts:
- name: ceph-conf
mountPath: /etc/ceph
readOnly: true
- name: tls
mountPath: /etc/rgw/tls
readOnly: true
- name: keyring
mountPath: /var/lib/ceph/radosgw/ceph-rgw.syd1
readOnly: true
- name: run
mountPath: /var/run/ceph
- name: log
mountPath: /var/log/ceph
- name: tmp
mountPath: /tmp
volumes:
- name: ceph-conf
configMap:
name: ceph-conf
- name: tls
secret:
secretName: rgw-tls
- name: keyring
secret:
secretName: rgw-keyring
items:
- key: keyring
path: keyring
- name: run
emptyDir: {}
- name: log
emptyDir: {}
- name: tmp
emptyDir: {}
+15
View File
@@ -0,0 +1,15 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- namespace.yaml
- configmap.yaml
- vaultauth.yaml
- vaultstaticsecret.yaml
- deployment.yaml
- service.yaml
- service-external.yaml
- certificate.yaml
- pdb.yaml
- vpa.yaml
+5
View File
@@ -0,0 +1,5 @@
---
apiVersion: v1
kind: Namespace
metadata:
name: ceph-system
+11
View File
@@ -0,0 +1,11 @@
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: rgw
namespace: ceph-system
spec:
maxUnavailable: 1
selector:
matchLabels:
app.kubernetes.io/name: rgw
@@ -0,0 +1,21 @@
---
apiVersion: v1
kind: Service
metadata:
name: rgw-external
namespace: ceph-system
labels:
app.kubernetes.io/name: rgw
annotations:
purelb.io/service-group: common
purelb.io/addresses: 198.18.200.11
external-dns.alpha.kubernetes.io/hostname: rgw.k8s.syd1.au.unkin.net
spec:
type: LoadBalancer
externalTrafficPolicy: Local
selector:
app.kubernetes.io/name: rgw
ports:
- name: https
port: 443
targetPort: https
+16
View File
@@ -0,0 +1,16 @@
---
apiVersion: v1
kind: Service
metadata:
name: rgw
namespace: ceph-system
labels:
app.kubernetes.io/name: rgw
spec:
type: ClusterIP
selector:
app.kubernetes.io/name: rgw
ports:
- name: http
port: 7480
targetPort: http
+18
View File
@@ -0,0 +1,18 @@
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
name: default
namespace: ceph-system
spec:
method: kubernetes
mount: k8s/au/syd1
vaultConnectionRef: vso-system/default
allowedNamespaces:
- ceph-system
kubernetes:
role: default
serviceAccount: default
audiences:
- vault
tokenExpirationSeconds: 600
@@ -0,0 +1,18 @@
---
# Seed kv/kubernetes/namespace/ceph-system/default/rgw-keyring field `keyring` with the client.rgw.syd1 keyring.
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: rgw-keyring
namespace: ceph-system
spec:
vaultAuthRef: default
mount: kv
type: kv-v2
path: kubernetes/namespace/ceph-system/default/rgw-keyring
refreshAfter: 5m
hmacSecretData: true
destination:
name: rgw-keyring
create: true
overwrite: true
+13
View File
@@ -0,0 +1,13 @@
---
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: rgw-vpa
namespace: ceph-system
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: rgw
updatePolicy:
updateMode: "Off"
+2
View File
@@ -11,3 +11,5 @@ data:
DBPORT: "5432"
DBUSER: encapi
DBSSL: require
ENCAPI_OIDC_ISSUER: https://vault.service.consul:8200/v1/identity/oidc
SSL_CERT_FILE: /etc/ssl/combined/ca-certificates.crt
+28 -1
View File
@@ -6,6 +6,7 @@ metadata:
namespace: encapi
annotations:
configmap.reloader.stakater.com/auto: "true"
secret.reloader.stakater.com/reload: "vault-ca-cert"
spec:
replicas: 2
selector:
@@ -21,9 +22,22 @@ spec:
app: encapi
spec:
automountServiceAccountToken: true
initContainers:
- name: combine-certs
image: alpine:3
command:
- sh
- -c
- cat /etc/ssl/certs/ca-certificates.crt /custom-ca/ca.crt > /combined-certs/ca-certificates.crt
volumeMounts:
- name: vault-ca-cert
mountPath: /custom-ca
readOnly: true
- name: combined-certs
mountPath: /combined-certs
containers:
- name: encapi
image: artifactapi.k8s.syd1.au.unkin.net/docker-internal/encapi:v0.1.2
image: artifactapi.k8s.syd1.au.unkin.net/docker-internal/encapi:v0.3.0
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8000
@@ -38,6 +52,10 @@ spec:
- secretRef:
name: environment
optional: false
volumeMounts:
- name: combined-certs
mountPath: /etc/ssl/combined
readOnly: true
livenessProbe:
httpGet:
path: /healthz
@@ -61,4 +79,13 @@ spec:
limits:
cpu: 500m
memory: 256Mi
volumes:
- name: vault-ca-cert
secret:
secretName: vault-ca-cert
items:
- key: ca.crt
path: ca.crt
- name: combined-certs
emptyDir: {}
restartPolicy: Always
File diff suppressed because one or more lines are too long
@@ -0,0 +1,13 @@
---
apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaDashboard
metadata:
name: router-firewall
namespace: grafana
spec:
instanceSelector:
matchLabels:
dashboards: "grafana"
resyncPeriod: 5m
allowCrossNamespaceImport: false
gzipJson: H4sIAAAAAAAC/+1ce3PbuBH/KijnpiP1JEeSbflxSWYS27m4TWyP7eYmk7gciIQkJCTBAqBlxdFNP0Q/YT9JdwGSoqyHZccPnU/5I7ZJvBb47Q+7iwUvHRpFQlPNRaSc7Usn4Eo7258unVbCA70fOdv1iuNTTZVIpMewiO7H8NPpSNqmEXUqTsJ9+LtaJb/aR6RadQYVh0W0FUBJLRNWcbrcz3/nnoh2RCAkVJOdFi3VKqRRr8N/6+sVUi9DmxENsZNXw9GRv5JXAZNawdt0CD5V3Zag0ncGZ9ihz3WxyzZXHg0+MipPNJX6vYh019muVXDocfdUiEDz2AgY8OgriP/prOLENGKBMlOQ9iJFD7vkGlt2Ds+ZPOcMH3kiCGisGEjfpoFCwXzTXEdy/0iY+eyaBz1nu7FWcS5M7334f1DoCIae96VA2EJnO0f/JK1E9R3bdGPKUsRShEx3WaLy1fCZbvTX133qrX356uFywBR0mLaS3b4VdhHjqtVrNfI3UqqTKqHnnZKkmpUi4TPXixNXMVhfX7kali645BEIFXnsxWfnp65QensLKn92KiEUh2fcD9hnZ/DpJ9fFVlweaZhgGpyVy4iDgHVY5L8RMoSJ2XZclyZawHPJ2vs4wlf4O4066aIPzq7O/pqZ/eLk1wcIDRb4OyJq8w6W81mbJoE2dZKIY08xg9mJzGJ0JVNdEfjmNQ4bXtOWEkGiGbxXmsV2Wr0U0x3JWARvQIwECkdJEOAaZ6+FHXD+fqNWfCsBT/mrrRogG8YrAHUgFkvx4og411hT770dla2VAjx9RiVDJf2SKM3b/eyhnUXNLvToE+g98dhhoXkaeNirE1ClD4Q+AGGcs3QG4YWTDValWjAYTMfzO0F9QE1YTgG9ujCANuANYHj1aYAd3B0aVw0a1+ZB492ADyHjM4+HFBmn8XQA9Z6FQvZJoozS4BKvLSJHGnSFZqwuDPnVOeUB7lRuq6+Zmoo48uxqzVOk1Gtqle8aqBuLSZubfxrahBmPtKTe1yLQ1xcM6AapUdv1ssG6gAPJ54D3pEpuwEOuH4CLrWVQX5oGj4zx050jwhTa71x1c5A3F8tAiKA5GLZ76sXuTiLlHg74wQyG+oNaDE8HWocnR29IxHgHnMVEqhRaGwsDLZWEpbaUrlBx203HKSfR5upaY/L+PiLcNGxV7muoLvW/UGBEbzLZTx11GxaVFOoWxv76RnqxsdSL23ljnQ7ziS9FrMjPRLIvzNOZemxOUY9z7mkhOYRnYJ/2VDUQHVUtlMwglJXD93NpzG0bTpHpvj08OT149X5vOwXeZ4ecfDx5d/iru7+7d3C6/2Z/73j7K5MQdSFuqDrbv392/vWJVr/VqltnP1fz30hp9/jw6Pvx3t/3dk7LBJr5TnDiFPFEEulSmXS5maV/J0z2T4czi88wBJWvv4c/ivo42wZp1JYwng5jdGCVJ7lpBEq84ZL1KBAIrtYzu1oEMEEglscU4REBSiOKBYBpALkhkxXyD7v+gIq0HAhCMO5UNfYmFGz1CUxv2nap/kxVIAInlSZgIhKw/IiFI9k/Km9bRNhGAqtM7BwsQ6gDYUsSgz0LkF9xCio4Gkx8TSO/x30IS06LJm7NEU1cnxZN1DxkiklLrVmf+xhgA0OGlN5RGD0LyuR///kvYVFcV7VU+dEcXpDNcRhahDH3hPzqSuYxfp76ztcEGX0GdIFPrXiTA43gvGyO707y4p730nHJwAOKFMDwvkXTFzfZajcN4jYLgGvO5Sa1YpwEDyhChKaIpL0T3TcgROVzsJEgOAQ14bpvQYfPfzMKgUAHRewdCZBGGRsH6AoJDqCOfFGkhiKTASlS7XWhLPRpVqDVP8DThALB4Qyg0sDOBycKPK1nCtvxrpilaMOEDYkTRtCBo4hzVv2I5w1XGNNOsZGTqzigGSmaAxXoKgArJ0SvEcYjNE6Jle9dWs/CytHZqUTO7SFMK8fCQmoDJwUgGqPDY6sUhLZgLgz10QsOPJRBirRYIHojVDSRHd7RlqGDFkTwayuNWj1jhPpTY4RcwidJCreS7va8sLnkhafNC28Bwpmd0BjaCY0naCc0nq6d0HgoO6HeXBLC0yYE40a0YawEZhTCVRBAMIGEjBlW/wDMACOfU3t+z8zs72Zj/Qz/cHP9PkunxvXp8tI2NxgQeYHzph6POh5Rdj0m++sHkT1bdsTp4y079l4Qfedhl/0RZddjsu9et6U0x4Md9bX5zugeZ095XK6fGmc66UNYMJyesrY2R5SpUbtRmMmkrvVJaGOgppdFO7GGwwwcYgnHWCa3TmL7y+wsNjjlxjREKks2fvxDaXLliSqGZQaDmxwrWlsNzffh+t708Ht+5WreTrkwmA0hVB51CsiPUOzAhKNFEtuwOgw85JFNq1xIbZysIXlWR5rzkOnJoh163ySnYxyeQ+Hu09K5u0QSU/7mfk+qS/hjfmXCkwb9IPvUFA2ZyweyMzLNDTKnTLnvk+tEm1+YLA7zc6eQwGL0ouhBoThW5EIrKHSa4e1U7I/tTyB8vXY21kBxgvImao/uec2v/21IB0zkkAA2FtF3KuqWSYJBY26qXs1lHF4xB+/ZAxoTAC4GBP07EGPYzsO5NGPCgASgmi5Cifk/Js9IUw/nqkwQCQbIf1gY08hNvI6NCcbR1lx8Lv7kXsdEnjsRbY3nzS1YWDglz0huc6Hyr4Y4VHa4Rp9j5s82zyda/JOJLh7RpftKzhqXw6yJqyBbhX27I3Gyxm5iJW1MspKWWjV33DaFEHmR6RHB9LlfSLYW8OLg1dE+JKv4oBmEXXQpTBg873HdJRgEIgFr6xnJKHlGTWkkl6U8PWwwT3LKavNGYYNiTtqzNCMNnfRvImKQUMNldhWutsxPG+anQcYVbJsa/niOc/SSPIe/ADcvyf7Bi0IGG0Y7sECF2Pfla3PajtN89NmJbRNiEtgNBP0uL21PEJ4Y54QtywnNIlw27sBxalETXR4hhc37jkEsNHuc5CltsQi41ydel0KKXElJr+pDchuqV9mkt6W6X8iqW4CsuVlUgSMl54rYwWb0UF/Sw+3o4d6J4Vo+GDlBvhM++NMYCY+vqOb+feFWj4hJKxBQ0Cc8Szy1XWTRjkZjqaqZqualYZqqV1T35HjnxXPg65fkisbCs1F1xbcAE/PS/I0gKcNTAwJY6TtKXt8a95PXZmd8zFYeVIS3jPoYjUwV4couZoFDRHvCNkXa4CqZhAbEGG5oj7VxAQSzpJDswxpZtBRu/YIhfCre2ET3EfHDbOKDJIQnwzBnyGRnJCJ7OXwnZIdG/Nvoa0CqFyQ+e21jufDglIf5ZzFcFz+u4br5JPPIZxfDsoAns5wfbCAVI9mSYZVhkfSVMzILaSjADAvx97o/OihLXZ8uLT7GatvFztzHweBsTloBD1kLmDQ4Z4vtJmBYZXXJKpMMgKPjw9PDF8/NpL28+nL36PTFcz/WYxRjilcIvFoApkmDB/dLNUvmuAVzGJgY7gCkGJsqJYrGAnCI0bchhWAeXKRJOwt65FeHMgJZWxJISiBXlNjGDvUt1Bg1d+zy0voPqnERoD24ogZu4XumFMVQZObCA2oOpVF2ZxcAAqY2OvTZJ6qgxi7TcNqhsnZ8uLAWn2jU907fOPzgQExPrzkWibYNTg6UNdbnCJQ1N2Z9FIpV0QdOHZnJF5sNX2fJNo3FSSIYvy9sBjr1pvAkpzGrChGl0uUlx7zTgblSPHfGC/p7V63V5uZs5zKkcQzrOvI1sOw2ZAGFdTMxsI0ivEQvsjgofLHB0KQ9fG8Uyu5HJtFg7NsPaXE0vFYLxRvV32i/UL5lB5KWRopdK5TeuzAfPSuU77MgMLBNa6xCjfWRGhAUsiOYVgVz75qFKvg9pxz4E2ugdm8UarzBm56Vsbupaekm0P9Z/jbf2Ap3Xge3+eiG6ftmF2CRWT7k6dw2FmB20w/ptVRLFBLNCIAmiFZbgcAFHDV2onxng3h7Duas0czTz9Iyxl19BTMasAnevQEA4YrY+CNkauseTB86QaCKkkAwEjxs9QvBSSYiCvo2+L97/Oz17vH1AbUr30kAg3r0cr7hleVnE/7In01IWbBZNKSvY8FFyVVa7FNutAGGWrK5ULuvIYapJ7/Td13a5jdLKx2F1+YSXvPD68xMM+5smJ9sWoO8vJB+AEY39L9WR2QYY8SxTG8mxLovtvP8d1h0/JwC5HzDOHV6eJV/2jVFEWTbZIZ8+rVVxAPiAF3N4Z/2y62N1Amw8POjCyVrtUZtJYRzpJUkghOyFYi7mQWU0sxLvufPLp9dmppVqrg0IED2VYdsF767fmzOoObWRcWIorH+e9UmpiGic4t/OWmZmIPfLq2Xgn9j0BHxAFaBMsszQg4SNtThGXtxGOVcje3CVgvLep6tf33wfwjdds+tVwAA
+2
View File
@@ -25,8 +25,10 @@ resources:
- dashboards/postgresql-database.yaml
- dashboards/puppet-report.yaml
- dashboards/redis.yaml
- dashboards/router-firewall.yaml
- dashboards/traefik.yaml
- dashboards/victorialogs-cluster.yaml
- dashboards/victoriametrics-cluster.yaml
- dashboards/victoriametrics-vmagent.yaml
- dashboards/cloudnativepg.yaml
- dashboards/arc-runner-scale-set.yaml
@@ -0,0 +1,7 @@
---
apiVersion: kustomize.config.k8s.io/v1alpha1
kind: Component
resources:
- scaleset.yaml
- networkpolicy.yaml
@@ -0,0 +1,45 @@
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: egress-iuclabs-k8s
namespace: iuclabs-runners
spec:
podSelector:
matchLabels:
runners.unkin.net/scale-set: iuclabs-k8s
policyTypes:
- Egress
egress:
- to:
- ipBlock:
cidr: 0.0.0.0/0
except:
- 10.0.0.0/8
- 172.16.0.0/12
- 192.168.0.0/16
- 198.18.0.0/15
- 0.0.0.0/8
- 100.64.0.0/10
- 127.0.0.0/8
- 169.254.0.0/16
- 224.0.0.0/4
- 240.0.0.0/4
ports:
- protocol: TCP
port: 443
# vault.service.consul servers, from `dig vault.service.consul`
- to:
- ipBlock:
cidr: 198.18.25.78/32
- ipBlock:
cidr: 198.18.26.197/32
- ipBlock:
cidr: 198.18.27.116/32
- ipBlock:
cidr: 198.18.28.248/32
- ipBlock:
cidr: 198.18.29.92/32
ports:
- protocol: TCP
port: 8200
@@ -0,0 +1,76 @@
---
apiVersion: actions.github.com/v1alpha1
kind: AutoscalingRunnerSet
metadata:
name: iuclabs-k8s
namespace: iuclabs-runners
labels:
app.kubernetes.io/version: "0.15.1"
annotations:
actions.github.com/cleanup-manager-role-name: arc-manager
actions.github.com/cleanup-manager-role-binding: arc-manager
spec:
githubConfigUrl: https://github.com/IUCLabs
githubConfigSecret: github-app
runnerScaleSetName: iuclabs-k8s
minRunners: 0
maxRunners: 4
template:
metadata:
labels:
runners.unkin.net/scale-set: iuclabs-k8s
spec:
serviceAccountName: runner
automountServiceAccountToken: false
restartPolicy: Never
securityContext:
runAsNonRoot: true
runAsUser: 1001
runAsGroup: 1001
seccompProfile:
type: RuntimeDefault
topologySpreadConstraints:
- maxSkew: 1
topologyKey: kubernetes.io/hostname
whenUnsatisfiable: ScheduleAnyway
labelSelector:
matchLabels:
runners.unkin.net/scale-set: iuclabs-k8s
containers:
- name: runner
image: ghcr.io/actions/actions-runner:2.338.2
command: ["/home/runner/run.sh"]
env:
- name: ACTIONS_RUNNER_HOOK_JOB_STARTED
value: /hooks/job-started.sh
- name: NODE_EXTRA_CA_CERTS
value: /etc/vault-ca/ca.crt
- name: RUNNER_MODE
value: default
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
resources:
requests:
cpu: 1
memory: 512Mi
limits:
cpu: 2
memory: 2Gi
volumeMounts:
- name: hooks
mountPath: /hooks
readOnly: true
- name: vault-ca-cert
mountPath: /etc/vault-ca
readOnly: true
volumes:
- name: hooks
configMap:
name: runner-hooks
defaultMode: 0555
- name: vault-ca-cert
secret:
secretName: vault-ca-cert
@@ -0,0 +1,7 @@
---
apiVersion: kustomize.config.k8s.io/v1alpha1
kind: Component
resources:
- scaleset.yaml
- networkpolicy.yaml
@@ -0,0 +1,56 @@
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: egress-iuclabs-release
namespace: iuclabs-runners
spec:
podSelector:
matchLabels:
runners.unkin.net/scale-set: iuclabs-release
policyTypes:
- Egress
egress:
- to:
- ipBlock:
cidr: 0.0.0.0/0
except:
- 10.0.0.0/8
- 172.16.0.0/12
- 192.168.0.0/16
- 198.18.0.0/15
- 0.0.0.0/8
- 100.64.0.0/10
- 127.0.0.0/8
- 169.254.0.0/16
- 224.0.0.0/4
- 240.0.0.0/4
ports:
- protocol: TCP
port: 443
# vault.service.consul servers, from `dig vault.service.consul`
- to:
- ipBlock:
cidr: 198.18.25.78/32
- ipBlock:
cidr: 198.18.26.197/32
- ipBlock:
cidr: 198.18.27.116/32
- ipBlock:
cidr: 198.18.28.248/32
- ipBlock:
cidr: 198.18.29.92/32
ports:
- protocol: TCP
port: 8200
# artifactapi is only exposed with TLS via traefik-internal; kube-proxy DNATs its LB IP to these pods
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: traefik-system
podSelector:
matchLabels:
app.kubernetes.io/instance: traefik-internal-traefik-system
ports:
- protocol: TCP
port: 443
@@ -0,0 +1,76 @@
---
apiVersion: actions.github.com/v1alpha1
kind: AutoscalingRunnerSet
metadata:
name: iuclabs-release
namespace: iuclabs-runners
labels:
app.kubernetes.io/version: "0.15.1"
annotations:
actions.github.com/cleanup-manager-role-name: arc-manager
actions.github.com/cleanup-manager-role-binding: arc-manager
spec:
githubConfigUrl: https://github.com/IUCLabs
githubConfigSecret: github-app
runnerScaleSetName: iuclabs-release
minRunners: 0
maxRunners: 2
template:
metadata:
labels:
runners.unkin.net/scale-set: iuclabs-release
spec:
serviceAccountName: runner
automountServiceAccountToken: false
restartPolicy: Never
securityContext:
runAsNonRoot: true
runAsUser: 1001
runAsGroup: 1001
seccompProfile:
type: RuntimeDefault
topologySpreadConstraints:
- maxSkew: 1
topologyKey: kubernetes.io/hostname
whenUnsatisfiable: ScheduleAnyway
labelSelector:
matchLabels:
runners.unkin.net/scale-set: iuclabs-release
containers:
- name: runner
image: ghcr.io/actions/actions-runner:2.338.2
command: ["/home/runner/run.sh"]
env:
- name: ACTIONS_RUNNER_HOOK_JOB_STARTED
value: /hooks/job-started.sh
- name: NODE_EXTRA_CA_CERTS
value: /etc/vault-ca/ca.crt
- name: RUNNER_MODE
value: release
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
resources:
requests:
cpu: 1
memory: 512Mi
limits:
cpu: 2
memory: 2Gi
volumeMounts:
- name: hooks
mountPath: /hooks
readOnly: true
- name: vault-ca-cert
mountPath: /etc/vault-ca
readOnly: true
volumes:
- name: hooks
configMap:
name: runner-hooks
defaultMode: 0555
- name: vault-ca-cert
secret:
secretName: vault-ca-cert
+39
View File
@@ -0,0 +1,39 @@
#!/bin/sh
set -eu
event="${GITHUB_EVENT_NAME:-}"
refuse() {
echo "job-started hook: refusing '${event}' job: $1" >&2
exit 1
}
[ -r "${GITHUB_EVENT_PATH:-}" ] || refuse "event payload missing"
jq -e 'type == "object"' "${GITHUB_EVENT_PATH}" >/dev/null 2>&1 || refuse "event payload unparseable"
case "${RUNNER_MODE:-}" in
default) ;;
release)
[ "${event}" = push ] || refuse "release runners only run tag pushes"
ref=$(jq -r '.ref' "${GITHUB_EVENT_PATH}") || refuse "event payload unreadable"
case "${ref}" in
refs/tags/v[0-9]*) exit 0 ;;
*) refuse "release runners only run v<digit>* tag pushes, got '${ref}'" ;;
esac
;;
*) refuse "RUNNER_MODE unset or invalid" ;;
esac
case "${event}" in
push | workflow_dispatch | schedule | merge_group | release) ;;
pull_request)
trusted=$(jq -r '
.pull_request.head.repo.full_name as $head
| if $head == null then false
elif $head == .repository.full_name then true
else (.pull_request.author_association | IN("OWNER", "MEMBER", "COLLABORATOR"))
end' "${GITHUB_EVENT_PATH}") || refuse "event payload unreadable"
[ "${trusted}" = true ] || refuse "pull request from untrusted fork"
;;
*) refuse "event not allowed on self-hosted runners" ;;
esac
@@ -0,0 +1,23 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- namespace.yaml
- vaultauth.yaml
- vaultstaticsecret.yaml
- networkpolicy.yaml
- rbac.yaml
components:
- components/iuclabs-k8s
- components/iuclabs-release
generatorOptions:
disableNameSuffixHash: true
configMapGenerator:
- name: runner-hooks
namespace: iuclabs-runners
files:
- job-started.sh
+5
View File
@@ -0,0 +1,5 @@
---
apiVersion: v1
kind: Namespace
metadata:
name: iuclabs-runners
@@ -0,0 +1,34 @@
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
namespace: iuclabs-runners
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-dns
namespace: iuclabs-runners
spec:
podSelector: {}
policyTypes:
- Egress
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
k8s-app: kube-dns
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
+44
View File
@@ -0,0 +1,44 @@
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: runner
namespace: iuclabs-runners
automountServiceAccountToken: false
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: arc-manager
namespace: iuclabs-runners
finalizers:
- actions.github.com/cleanup-protection
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["create", "delete", "get"]
- apiGroups: [""]
resources: ["pods/status"]
verbs: ["get"]
- apiGroups: [""]
resources: ["secrets", "serviceaccounts"]
verbs: ["create", "delete", "get", "list", "patch", "update"]
- apiGroups: ["rbac.authorization.k8s.io"]
resources: ["roles", "rolebindings"]
verbs: ["create", "delete", "get", "patch", "update"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: arc-manager
namespace: iuclabs-runners
finalizers:
- actions.github.com/cleanup-protection
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: arc-manager
subjects:
- kind: ServiceAccount
name: arc-gha-rs-controller
namespace: arc-system
+18
View File
@@ -0,0 +1,18 @@
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
name: default
namespace: iuclabs-runners
spec:
allowedNamespaces:
- iuclabs-runners
kubernetes:
audiences:
- vault
role: default
serviceAccount: default
tokenExpirationSeconds: 600
method: kubernetes
mount: k8s/au/syd1
vaultConnectionRef: vso-system/default
@@ -0,0 +1,17 @@
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
name: github-app
namespace: iuclabs-runners
spec:
destination:
create: true
name: github-app
overwrite: true
hmacSecretData: true
mount: kv
path: kubernetes/namespace/iuclabs-runners/default/github-app
refreshAfter: 5m
type: kv-v2
vaultAuthRef: default
+4 -4
View File
@@ -85,7 +85,7 @@ spec:
archive_mode: "on"
archive_timeout: 5min
dynamic_shared_memory_type: posix
effective_cache_size: 256MB
effective_cache_size: 1GB
full_page_writes: "on"
log_destination: csvlog
log_directory: /controller/log
@@ -98,7 +98,7 @@ spec:
max_parallel_workers: "16"
max_replication_slots: "16"
max_worker_processes: "16"
shared_buffers: 128MB
shared_buffers: 256MB
shared_memory_type: mmap
ssl_max_protocol_version: TLSv1.3
ssl_min_protocol_version: TLSv1.3
@@ -127,10 +127,10 @@ spec:
resources:
limits:
cpu: 500m
memory: 512Mi
memory: 2Gi
requests:
cpu: 50m
memory: 256Mi
memory: 1Gi
smartShutdownTimeout: 180
startDelay: 3600
stopDelay: 1800
@@ -93,6 +93,8 @@ spec:
- mountPath: /opt/puppetlabs/server/data/puppetdb/stockpile
name: puppetdb-storage
subPathExpr: stockpile/$(POD_NAME)
- mountPath: /opt/puppetlabs/server/data/puppetdb/logs
name: puppetdb-logs
- mountPath: /etc/puppetlabs/puppetdb/conf.d/read-database.conf
name: puppetdb-read-database-conf
subPath: read-database.conf
@@ -191,6 +193,9 @@ spec:
- name: puppetdb-storage
persistentVolumeClaim:
claimName: puppetserver-puppetdb-claim
- name: puppetdb-logs
emptyDir:
sizeLimit: 512Mi
- name: puppetdb-read-database-conf
configMap:
name: puppetdb-read-database-conf
@@ -202,7 +202,7 @@ spec:
# Install the encapic ENC client (stdlib-only Go binary) to the shared
# bin volume. It replaces the uv/python ENC script, whose
# first-invocation dependency resolution failed on fresh compiler pods.
ENCAPIC_VERSION=v0.1.0
ENCAPIC_VERSION=v0.4.0
BASE=https://git.unkin.net/unkin/encapic/releases/download/$ENCAPIC_VERSION
cd /tmp
curl -fsSL -o encapic "$BASE/encapic_linux_amd64"
+1
View File
@@ -8,6 +8,7 @@ resources:
- vaultstaticsecret.yaml
- oauth2-proxy-configmap.yaml
- oauth2-proxy-deployment.yaml
- pdb.yaml
- service.yaml
- gateway.yaml
- httproute.yaml
@@ -26,3 +26,5 @@ data:
OAUTH2_PROXY_PROVIDER_CA_FILES: "/etc/ssl/combined/ca-certificates.crt"
OAUTH2_PROXY_CODE_CHALLENGE_METHOD: "S256"
OAUTH2_PROXY_SKIP_PROVIDER_BUTTON: "true"
OAUTH2_PROXY_SKIP_JWT_BEARER_TOKENS: "true"
OAUTH2_PROXY_EXTRA_JWT_ISSUERS: "https://vault.service.consul:8200/v1/identity/oidc=victorialogs"
+8 -1
View File
@@ -8,7 +8,7 @@ metadata:
configmap.reloader.stakater.com/auto: "true"
secret.reloader.stakater.com/reload: "vlogs-oauth-credentials,vault-ca-cert"
spec:
replicas: 1
replicas: 2
selector:
matchLabels:
app: vlogs-oauth2
@@ -23,6 +23,13 @@ spec:
spec:
serviceAccountName: default
automountServiceAccountToken: false
topologySpreadConstraints:
- maxSkew: 1
topologyKey: kubernetes.io/hostname
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: vlogs-oauth2
securityContext:
runAsNonRoot: true
runAsUser: 65532
+11
View File
@@ -0,0 +1,11 @@
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: vlogs-oauth2
namespace: vlogs
spec:
minAvailable: 1
selector:
matchLabels:
app: vlogs-oauth2
@@ -0,0 +1,15 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../../base/arc-system
helmCharts:
- name: gha-runner-scale-set-controller
repo: oci://artifactapi.k8s.syd1.au.unkin.net/ghcr/actions/actions-runner-controller-charts
version: "0.15.1"
releaseName: arc
namespace: arc-system
valuesFile: values.yaml
includeCRDs: true
@@ -0,0 +1,39 @@
---
replicaCount: 2
podSecurityContext:
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 500m
memory: 512Mi
topologySpreadConstraints:
- maxSkew: 1
topologyKey: kubernetes.io/hostname
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app.kubernetes.io/name: gha-rs-controller
app.kubernetes.io/instance: arc
flags:
logLevel: info
logFormat: json
metrics:
controllerManagerAddr: ":8080"
listenerAddr: ":8080"
listenerEndpoint: "/metrics"
@@ -0,0 +1,6 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../../base/ceph-system
@@ -0,0 +1,6 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../../base/iuclabs-runners
+3
View File
@@ -16,9 +16,11 @@ spec:
- path: apps/overlays/*/bind-internal
- path: apps/overlays/*/bind-external
- path: apps/overlays/*/age-api
- path: apps/overlays/*/arc-system
- path: apps/overlays/*/cattle-system
- path: apps/overlays/*/cert-manager
- path: apps/overlays/*/certificates
- path: apps/overlays/*/ceph-system
- path: apps/overlays/*/cephrgw-system
- path: apps/overlays/*/cnpg-system
- path: apps/overlays/*/consul
@@ -31,6 +33,7 @@ spec:
- path: apps/overlays/*/grafana-system
- path: apps/overlays/*/haproxy
- path: apps/overlays/*/inteldeviceplugins-system
- path: apps/overlays/*/iuclabs-runners
- path: apps/overlays/*/jfrog
- path: apps/overlays/*/k8up-system
- path: apps/overlays/*/kanidm
+3
View File
@@ -14,6 +14,7 @@ spec:
- oci://ghcr.io/woodpecker-ci/helm/woodpecker
- oci://ghcr.io/netbox-community/netbox-chart/netbox
- oci://docker.gitea.com/charts
- oci://artifactapi.k8s.syd1.au.unkin.net/ghcr/actions/actions-runner-controller-charts/gha-runner-scale-set-controller
destinations:
- namespace: '*-system'
server: https://kubernetes.default.svc
@@ -45,6 +46,8 @@ spec:
server: https://kubernetes.default.svc
- namespace: 'haproxy'
server: https://kubernetes.default.svc
- namespace: 'iuclabs-runners'
server: https://kubernetes.default.svc
- namespace: 'jfrog'
server: https://kubernetes.default.svc
- namespace: 'kanidm'
+1
View File
@@ -0,0 +1 @@
not json {
@@ -0,0 +1,11 @@
{
"comment": {
"author_association": "NONE"
},
"issue": {
"number": 1
},
"repository": {
"full_name": "IUCLabs/app"
}
}
@@ -0,0 +1,11 @@
{
"pull_request": {
"author_association": "MEMBER",
"head": {
"repo": null
}
},
"repository": {
"full_name": "IUCLabs/app"
}
}
@@ -0,0 +1,14 @@
{
"pull_request": {
"author_association": "NONE",
"head": {
"repo": {
"fork": true,
"full_name": "evil/app"
}
}
},
"repository": {
"full_name": "IUCLabs/app"
}
}
@@ -0,0 +1,14 @@
{
"pull_request": {
"author_association": "NONE",
"head": {
"repo": {
"fork": false,
"full_name": "evil/app"
}
}
},
"repository": {
"full_name": "IUCLabs/app"
}
}
+14
View File
@@ -0,0 +1,14 @@
{
"pull_request": {
"author_association": "MEMBER",
"head": {
"repo": {
"fork": true,
"full_name": "someone/app"
}
}
},
"repository": {
"full_name": "IUCLabs/app"
}
}
@@ -0,0 +1,14 @@
{
"pull_request": {
"author_association": "CONTRIBUTOR",
"head": {
"repo": {
"fork": false,
"full_name": "IUCLabs/app"
}
}
},
"repository": {
"full_name": "IUCLabs/app"
}
}
+3
View File
@@ -0,0 +1,3 @@
{
"ref": "refs/heads/main"
}
@@ -0,0 +1,3 @@
{
"ref": "refs/tags/release-1"
}
@@ -0,0 +1,3 @@
{
"ref": "refs/tags/vagrant"
}
+3
View File
@@ -0,0 +1,3 @@
{
"ref": "refs/tags/v1.2.3"
}
+60
View File
@@ -0,0 +1,60 @@
#!/usr/bin/env bash
set -euo pipefail
hook=apps/base/iuclabs-runners/job-started.sh
fixtures=ci/fixtures/runner-hooks
fail=0
# check <want> <mode|unset> <event|-> <fixture|->
check() {
local want=$1 mode=$2 event=$3 fixture=$4 got=0
local -a env=(-u GITHUB_EVENT_NAME -u GITHUB_EVENT_PATH -u RUNNER_MODE)
[ "$mode" = unset ] || env+=("RUNNER_MODE=$mode")
[ "$event" = - ] || env+=("GITHUB_EVENT_NAME=$event")
[ "$fixture" = - ] || env+=("GITHUB_EVENT_PATH=$fixtures/$fixture")
env "${env[@]}" "$hook" >/dev/null 2>&1 || got=1
if [ "$got" -ne "$want" ]; then
echo "FAIL: mode=$mode $event $fixture exit=$got want=$want" >&2
fail=1
fi
}
for m in unset "" bogus true Default; do
check 1 "$m" push push.json
check 1 "$m" push tag-push.json
check 1 "$m" pull_request pr-member.json
done
for e in push workflow_dispatch schedule merge_group release; do
check 0 default "$e" push.json
done
check 0 default pull_request pr-member.json
check 0 default pull_request pr-same-repo.json
check 1 default pull_request pr-fork-none.json
check 1 default pull_request pr-fork-spoofed.json
check 1 default pull_request pr-deleted-fork.json
check 1 default pull_request push.json
for e in pull_request_target workflow_run pull_request_review pull_request_review_comment issue_comment issues discussion discussion_comment create delete unknown_event; do
check 1 default "$e" pr-member.json
done
check 1 default pull_request_review pr-fork-none.json
check 1 default issue_comment issue-comment.json
check 1 default push -
check 1 default push missing.json
check 1 default push invalid.txt
check 1 default - push.json
check 0 default push tag-push.json
check 0 release push tag-push.json
check 1 release push tag-push-vagrant.json
check 1 release push tag-push-nov.json
check 1 release push push.json
for e in pull_request pull_request_target workflow_dispatch schedule merge_group release workflow_run issue_comment create unknown_event; do
check 1 release "$e" tag-push.json
done
check 1 release pull_request pr-same-repo.json
check 1 release - tag-push.json
check 1 release push -
check 1 release push invalid.txt
exit "$fail"