Compare commits

..

6 Commits

Author SHA1 Message Date
unkinben d0b3c26223 feat(opa): add conftest OPA policies and pre-commit hook
ci/woodpecker/pr/pre-commit Pipeline failed
ci/woodpecker/pr/kubeconform Pipeline was successful
Adds three policy files under policy/ plus a pre-commit hook that
runs conftest against all staged YAML manifests (excluding chart
templates).

Policies:
  no_ingress.rego
    Deny Ingress resources — cluster uses Gateway API only.

  gateway_api.rego
    HTTPRoute/TLSRoute: require explicit group/kind on parentRefs and
    group/kind/weight on backendRefs (PR #162, #165).
    Gateway: require explicit group on certificateRefs (PR #153).
    All fields are defaulted by the controller; omitting them causes
    permanent ArgoCD OutOfSync.

  resource_normalization.rego
    CPU integer: deny unquoted integer cpu values (PR #163).
    CPU milliCPU: deny values like 1000m/2000m that normalise to "1"/"2" (PR #164).
    Memory Mi→Gi: deny 1024Mi/2048Mi etc. that normalise to 1Gi/2Gi (PR #163).
    clusterIP null: deny Service with explicit null clusterIP (PR #166).

Also fixes all existing violations found by the new policies across
puppet deployments and reposync cronjobs (resource normalization).
kanidm/tlsroute.yaml and puppet/service_puppetdb.yaml are excluded
from this commit as they are addressed in PRs #165 and #166.
2026-05-25 00:00:37 +10:00
unkinben dd282f59fb fix(litellm): normalize postgres cluster resource values (#163)
## Summary

- Changes `limits.memory` from `1024Mi` to `1Gi` (same value, canonical form)
- Changes `limits.cpu` from `1` (integer) to `"1"` (string, canonical form)

## Why

Kubernetes normalizes resource quantities on write — `1024Mi` becomes `1Gi` and integer `1` becomes string `"1"`. ArgoCD diffs by string comparison, so these equivalent values cause a permanent OutOfSync on the `litellm-postgres` Cluster.

Reviewed-on: #163
2026-05-24 23:30:10 +10:00
unkinben 1890dd4bda fix(gateways): add explicit group/kind/weight to all HTTPRoute refs (#162)
## Summary

- Adds `group: gateway.networking.k8s.io` and `kind: Gateway` to all `parentRefs` entries
- Adds `group: ""`, `kind: Service`, and `weight: 1` to all `backendRefs` entries
- Affects 9 HTTPRoute files across artifactapi, cattle-system, consul, kanidm, litellm, paperclip, puppet, and vault

## Why

ArgoCD diffs the desired manifest against the live Kubernetes object. The Gateway API controller defaults these fields when creating/updating objects, so the live state always has them — causing persistent OutOfSync for every HTTPRoute. Same root cause as #153 (certificateRefs).

## Test plan

- [ ] All affected ArgoCD applications show Synced after merge

Reviewed-on: #162
2026-05-24 20:32:37 +10:00
unkinben 6815b66010 fix(kanidm): use dockerhub image instead of ghcr.io (#161)
## Summary

- Changes both `config-init` init container and `kanidm` container images from `ghcr.io/kanidm/server:1.10.3` to `kanidm/server:1.10.3`

## Why

`kanidm/server` is published on Docker Hub, not ghcr.io. RKE2 rewrites dockerhub pulls through the artifactapi mirror automatically.

## Test plan

- [ ] Pods roll successfully after ArgoCD sync
- [ ] Verify kanidm cluster replication still healthy

Reviewed-on: #161
2026-05-24 20:27:21 +10:00
unkinben 7cbec33588 fix(artifactapi): move kanidm to dockerhub remote (#160)
## Summary

- Removes `^kanidm/` from the `ghcr` remote immutable_patterns
- Adds `^kanidm/` to the `dockerhub` remote immutable_patterns

## Why

`kanidm/server` is published on Docker Hub, not ghcr.io. Pulling via the `ghcr` cache was failing with 403 on anonymous token fetch → 502 Bad Gateway.

## Test plan

- [ ] `docker pull artifactapi.k8s.syd1.au.unkin.net/dockerhub/kanidm/server:1.10.3` succeeds after artifactapi redeploys

Reviewed-on: #160
2026-05-24 20:24:33 +10:00
unkinben 3756208ccd benvin/kanidm (#159)
Reviewed-on: #159
2026-05-24 19:55:22 +10:00
24 changed files with 397 additions and 44 deletions
+7
View File
@@ -40,3 +40,10 @@ repos:
entry: ci/validate-no-secrets.sh
language: system
pass_filenames: false
- id: conftest_policies
name: OPA policy checks (conftest)
entry: conftest test --policy policy/
language: system
types: [yaml]
exclude: ".*/charts/.*|.*/templates/.*|\\.woodpecker/.*"
pass_filenames: true
+10 -3
View File
@@ -8,7 +8,9 @@ spec:
hostnames:
- artifactapi.k8s.syd1.au.unkin.net
parentRefs:
- name: artifactapi
- group: gateway.networking.k8s.io
kind: Gateway
name: artifactapi
sectionName: http
rules:
- filters:
@@ -30,12 +32,17 @@ spec:
hostnames:
- artifactapi.k8s.syd1.au.unkin.net
parentRefs:
- name: artifactapi
- group: gateway.networking.k8s.io
kind: Gateway
name: artifactapi
sectionName: https
rules:
- backendRefs:
- name: artifactapi-api
- group: ""
kind: Service
name: artifactapi-api
port: 80
weight: 1
matches:
- path:
type: PathPrefix
@@ -6,7 +6,6 @@ remotes:
immutable_patterns:
- "^cloudnative-pg/cloudnative-pg"
- "^emberstack/helm-charts"
- "^kanidm/"
- "^openvoxproject/"
- "^stakater/reloader"
- "^stalwartlabs/stalwart"
@@ -35,6 +34,7 @@ remotes:
- "^hashicorp/consul"
- "^hashicorp/vault"
- "^jfrog/"
- "^kanidm/"
- "^rancher/"
- "^rspamd/rspamd"
- "^tozd/postfix"
+10 -3
View File
@@ -8,7 +8,9 @@ spec:
hostnames:
- rancher.k8s.syd1.au.unkin.net
parentRefs:
- name: rancher
- group: gateway.networking.k8s.io
kind: Gateway
name: rancher
sectionName: http
rules:
- filters:
@@ -30,12 +32,17 @@ spec:
hostnames:
- rancher.k8s.syd1.au.unkin.net
parentRefs:
- name: rancher
- group: gateway.networking.k8s.io
kind: Gateway
name: rancher
sectionName: https
rules:
- backendRefs:
- name: rancher
- group: ""
kind: Service
name: rancher
port: 80
weight: 1
matches:
- path:
type: PathPrefix
+17 -5
View File
@@ -11,7 +11,9 @@ spec:
hostnames:
- consul.k8s.syd1.au.unkin.net
parentRefs:
- name: consul
- group: gateway.networking.k8s.io
kind: Gateway
name: consul
sectionName: http
rules:
- filters:
@@ -36,12 +38,17 @@ spec:
hostnames:
- consul.k8s.syd1.au.unkin.net
parentRefs:
- name: consul
- group: gateway.networking.k8s.io
kind: Gateway
name: consul
sectionName: https
rules:
- backendRefs:
- name: consul-ui
- group: ""
kind: Service
name: consul-ui
port: 80
weight: 1
matches:
- path:
type: PathPrefix
@@ -59,12 +66,17 @@ spec:
hostnames:
- consul.service.consul
parentRefs:
- name: consul
- group: gateway.networking.k8s.io
kind: Gateway
name: consul
sectionName: consul-svc
rules:
- backendRefs:
- name: consul-ui
- group: ""
kind: Service
name: consul-ui
port: 80
weight: 1
matches:
- path:
type: PathPrefix
+3 -1
View File
@@ -13,7 +13,9 @@ spec:
- auth.unkin.net
- au.auth.unkin.net
parentRefs:
- name: kanidm
- group: gateway.networking.k8s.io
kind: Gateway
name: kanidm
sectionName: http
rules:
- filters:
+2 -2
View File
@@ -36,7 +36,7 @@ spec:
fsGroup: 1000
initContainers:
- name: config-init
image: ghcr.io/kanidm/server:1.10.3
image: kanidm/server:1.10.3
command: ["/bin/sh", "-c"]
args:
- |
@@ -99,7 +99,7 @@ spec:
readOnlyRootFilesystem: false
containers:
- name: kanidm
image: ghcr.io/kanidm/server:1.10.3
image: kanidm/server:1.10.3
command: ["/sbin/kanidmd"]
args: ["server", "-c", "/config/server.toml"]
ports:
+2 -2
View File
@@ -76,8 +76,8 @@ spec:
updateInterval: 30
resources:
limits:
cpu: 1
memory: 1024Mi
cpu: "1"
memory: 1Gi
requests:
cpu: 250m
memory: 512Mi
+10 -3
View File
@@ -8,7 +8,9 @@ spec:
hostnames:
- litellm.k8s.syd1.au.unkin.net
parentRefs:
- name: litellm
- group: gateway.networking.k8s.io
kind: Gateway
name: litellm
sectionName: http
rules:
- filters:
@@ -30,12 +32,17 @@ spec:
hostnames:
- litellm.k8s.syd1.au.unkin.net
parentRefs:
- name: litellm
- group: gateway.networking.k8s.io
kind: Gateway
name: litellm
sectionName: https
rules:
- backendRefs:
- name: litellm
- group: ""
kind: Service
name: litellm
port: 4000
weight: 1
matches:
- path:
type: PathPrefix
+10 -3
View File
@@ -8,7 +8,9 @@ spec:
hostnames:
- paperclip.k8s.syd1.au.unkin.net
parentRefs:
- name: paperclip
- group: gateway.networking.k8s.io
kind: Gateway
name: paperclip
sectionName: http
rules:
- filters:
@@ -30,12 +32,17 @@ spec:
hostnames:
- paperclip.k8s.syd1.au.unkin.net
parentRefs:
- name: paperclip
- group: gateway.networking.k8s.io
kind: Gateway
name: paperclip
sectionName: https
rules:
- backendRefs:
- name: paperclip
- group: ""
kind: Service
name: paperclip
port: 3100
weight: 1
matches:
- path:
type: PathPrefix
+1 -1
View File
@@ -150,7 +150,7 @@ spec:
memory: 350Mi
cpu: 100m
limits:
memory: 1024Mi
memory: 1Gi
cpu: 500m
securityContext:
runAsNonRoot: true
+1 -1
View File
@@ -35,7 +35,7 @@ spec:
imagePullPolicy: IfNotPresent
resources:
limits:
cpu: 1
cpu: "1"
memory: 1536Mi
requests:
cpu: 250m
@@ -31,11 +31,11 @@ spec:
imagePullPolicy: IfNotPresent
resources:
limits:
cpu: 2
memory: 3072Mi
cpu: "2"
memory: 3Gi
requests:
cpu: 500m
memory: 1024Mi
memory: 1Gi
ports:
- containerPort: 8140
name: puppetserver
@@ -35,11 +35,11 @@ spec:
imagePullPolicy: IfNotPresent
resources:
limits:
cpu: 2
cpu: "2"
memory: 3500Mi
requests:
cpu: 250m
memory: 1024Mi
memory: 1Gi
ports:
- containerPort: 8140
name: puppetserver
+10 -3
View File
@@ -13,7 +13,9 @@ spec:
hostnames:
- puppetboard.k8s.syd1.au.unkin.net
parentRefs:
- name: puppetboard
- group: gateway.networking.k8s.io
kind: Gateway
name: puppetboard
sectionName: http
rules:
- filters:
@@ -40,12 +42,17 @@ spec:
hostnames:
- puppetboard.k8s.syd1.au.unkin.net
parentRefs:
- name: puppetboard
- group: gateway.networking.k8s.io
kind: Gateway
name: puppetboard
sectionName: https
rules:
- backendRefs:
- name: puppetboard
- group: ""
kind: Service
name: puppetboard
port: 80
weight: 1
matches:
- path:
type: PathPrefix
+7 -2
View File
@@ -13,12 +13,17 @@ spec:
hostnames:
- puppetdb.k8s.syd1.au.unkin.net
parentRefs:
- name: puppetdb
- group: gateway.networking.k8s.io
kind: Gateway
name: puppetdb
sectionName: https
rules:
- backendRefs:
- name: puppetdb
- group: ""
kind: Service
name: puppetdb
port: 8080
weight: 1
matches:
- path:
type: PathPrefix
@@ -53,7 +53,7 @@ spec:
cpu: 500m
memory: 1Gi
limits:
cpu: 2000m
cpu: "2"
memory: 4Gi
volumeMounts:
- name: repodata
@@ -56,7 +56,7 @@ spec:
cpu: 500m
memory: 1Gi
limits:
cpu: 2000m
cpu: "2"
memory: 4Gi
volumeMounts:
- name: repodata
@@ -53,7 +53,7 @@ spec:
cpu: 500m
memory: 1Gi
limits:
cpu: 2000m
cpu: "2"
memory: 4Gi
volumeMounts:
- name: repodata
@@ -52,7 +52,7 @@ spec:
cpu: 500m
memory: 1Gi
limits:
cpu: 2000m
cpu: "2"
memory: 4Gi
volumeMounts:
- name: repodata
+17 -5
View File
@@ -11,7 +11,9 @@ spec:
hostnames:
- vault.k8s.syd1.au.unkin.net
parentRefs:
- name: vault
- group: gateway.networking.k8s.io
kind: Gateway
name: vault
sectionName: http
rules:
- filters:
@@ -36,12 +38,17 @@ spec:
hostnames:
- vault.k8s.syd1.au.unkin.net
parentRefs:
- name: vault
- group: gateway.networking.k8s.io
kind: Gateway
name: vault
sectionName: https
rules:
- backendRefs:
- name: vault
- group: ""
kind: Service
name: vault
port: 8200
weight: 1
matches:
- path:
type: PathPrefix
@@ -60,12 +67,17 @@ spec:
- vault.service.consul
- vault.query.consul
parentRefs:
- name: vault
- group: gateway.networking.k8s.io
kind: Gateway
name: vault
sectionName: vault-direct
rules:
- backendRefs:
- name: vault
- group: ""
kind: Service
name: vault
port: 8200
weight: 1
matches:
- path:
type: PathPrefix
+94
View File
@@ -0,0 +1,94 @@
package main
# Gateway API resources require several fields to be set explicitly even though
# the Gateway API controller defaults them. ArgoCD diffs desired vs live by
# string comparison, so any field the controller defaults that is absent from
# the git manifest causes a permanent OutOfSync.
#
# Affected resources:
# HTTPRoute / TLSRoute — parentRefs and backendRefs (see PR #162, #165)
# Gateway — listeners[*].tls.certificateRefs (see PR #153)
_route_kinds := {"HTTPRoute", "TLSRoute"}
# ---- parentRefs: group must be "gateway.networking.k8s.io" ----
deny contains msg if {
_route_kinds[input.kind]
ref := input.spec.parentRefs[i]
object.get(ref, "group", null) != "gateway.networking.k8s.io"
msg := sprintf(
"%s %s/%s parentRefs[%d]: add 'group: gateway.networking.k8s.io' — controller defaults this field, causing ArgoCD OutOfSync when omitted",
[input.kind, input.metadata.namespace, input.metadata.name, i],
)
}
# ---- parentRefs: kind must be "Gateway" ----
deny contains msg if {
_route_kinds[input.kind]
ref := input.spec.parentRefs[i]
object.get(ref, "kind", null) != "Gateway"
msg := sprintf(
"%s %s/%s parentRefs[%d]: add 'kind: Gateway' — controller defaults this field, causing ArgoCD OutOfSync when omitted",
[input.kind, input.metadata.namespace, input.metadata.name, i],
)
}
# ---- backendRefs: group must be present (may be empty string "") ----
deny contains msg if {
_route_kinds[input.kind]
rule := input.spec.rules[ri]
ref := rule.backendRefs[bi]
not _has_key(ref, "group")
msg := sprintf(
"%s %s/%s rules[%d].backendRefs[%d]: add 'group: \"\"' — controller defaults this field, causing ArgoCD OutOfSync when omitted",
[input.kind, input.metadata.namespace, input.metadata.name, ri, bi],
)
}
# ---- backendRefs: kind must be "Service" ----
deny contains msg if {
_route_kinds[input.kind]
rule := input.spec.rules[ri]
ref := rule.backendRefs[bi]
object.get(ref, "kind", null) != "Service"
msg := sprintf(
"%s %s/%s rules[%d].backendRefs[%d]: add 'kind: Service' — controller defaults this field, causing ArgoCD OutOfSync when omitted",
[input.kind, input.metadata.namespace, input.metadata.name, ri, bi],
)
}
# ---- backendRefs: weight must be present ----
deny contains msg if {
_route_kinds[input.kind]
rule := input.spec.rules[ri]
ref := rule.backendRefs[bi]
not _has_key(ref, "weight")
msg := sprintf(
"%s %s/%s rules[%d].backendRefs[%d]: add 'weight: 1' — controller defaults this field, causing ArgoCD OutOfSync when omitted",
[input.kind, input.metadata.namespace, input.metadata.name, ri, bi],
)
}
# ---- Gateway certificateRefs: group must be present (may be empty string "") ----
deny contains msg if {
input.kind == "Gateway"
listener := input.spec.listeners[li]
ref := listener.tls.certificateRefs[ci]
not _has_key(ref, "group")
msg := sprintf(
"Gateway %s/%s listeners[%d].tls.certificateRefs[%d]: add 'group: \"\"' — admission webhook defaults this field, causing ArgoCD OutOfSync when omitted",
[input.metadata.namespace, input.metadata.name, li, ci],
)
}
# ---- Helper: key presence check (works for null, "", and any defined value) ----
_has_key(obj, key) if {
_ = obj[key]
}
+13
View File
@@ -0,0 +1,13 @@
package main
# Deny all Kubernetes Ingress resources.
# This cluster uses Gateway API (HTTPRoute + Gateway) for ingress routing.
# Ingress is the legacy API and must not be added.
deny contains msg if {
input.kind == "Ingress"
msg := sprintf(
"%s/%s: Ingress resources are forbidden — use Gateway API HTTPRoute instead",
[input.metadata.namespace, input.metadata.name],
)
}
+173
View File
@@ -0,0 +1,173 @@
package main
# Kubernetes normalizes resource quantity values on write. ArgoCD diffs by
# string comparison, so a non-canonical value in git will always differ from
# the live object, causing permanent OutOfSync.
#
# Rules enforced here:
# CPU integers — k8s converts integer 1 to string "1" (see PR #163)
# CPU milliCPU — k8s converts 1000m → "1", 2000m → "2", etc. (PR #164)
# Memory Mi→Gi — k8s converts 1024Mi → 1Gi, 2048Mi → 2Gi, etc. (PR #163)
# clusterIP null — k8s assigns a real IP; null in git always differs
# from the live assigned value (see PR #166)
# ---- Container helpers ----
# Extracts containers from Deployment/StatefulSet/DaemonSet/Job/CronJob/Pod
_containers contains c if { c := input.spec.template.spec.containers[_] }
_containers contains c if { c := input.spec.template.spec.initContainers[_] }
_containers contains c if { c := input.spec.containers[_] }
_containers contains c if { c := input.spec.initContainers[_] }
_containers contains c if { c := input.spec.jobTemplate.spec.template.spec.containers[_] }
# ---- CPU: must not be an integer ----
# YAML `cpu: 1` (unquoted) parses as JSON integer; k8s stores as string "1".
deny contains msg if {
c := _containers[_]
cpu := c.resources.limits.cpu
is_number(cpu)
msg := sprintf(
"%s container %q: cpu limit %v is an unquoted integer — use \"%v\" (string) to prevent ArgoCD OutOfSync",
[input.kind, c.name, cpu, cpu],
)
}
deny contains msg if {
c := _containers[_]
cpu := c.resources.requests.cpu
is_number(cpu)
msg := sprintf(
"%s container %q: cpu request %v is an unquoted integer — use \"%v\" (string) to prevent ArgoCD OutOfSync",
[input.kind, c.name, cpu, cpu],
)
}
deny contains msg if {
input.kind == "Cluster"
cpu := input.spec.resources.limits.cpu
is_number(cpu)
msg := sprintf(
"Cluster %s/%s: cpu limit %v is an unquoted integer — use \"%v\" (string) to prevent ArgoCD OutOfSync",
[input.metadata.namespace, input.metadata.name, cpu, cpu],
)
}
deny contains msg if {
input.kind == "Cluster"
cpu := input.spec.resources.requests.cpu
is_number(cpu)
msg := sprintf(
"Cluster %s/%s: cpu request %v is an unquoted integer — use \"%v\" (string) to prevent ArgoCD OutOfSync",
[input.metadata.namespace, input.metadata.name, cpu, cpu],
)
}
# ---- CPU: milliCPU divisible by 1000 normalizes to a whole number ----
# k8s converts 1000m → "1", 2000m → "2". Use the canonical whole-number form.
_milli_whole(cpu) := n if {
is_string(cpu)
endswith(cpu, "m")
val := to_number(substring(cpu, 0, count(cpu) - 1))
val % 1000 == 0
val > 0
n := val / 1000
}
deny contains msg if {
c := _containers[_]
n := _milli_whole(c.resources.limits.cpu)
msg := sprintf(
"%s container %q: cpu limit %q normalizes to \"%v\" — use canonical form to prevent ArgoCD OutOfSync",
[input.kind, c.name, c.resources.limits.cpu, n],
)
}
deny contains msg if {
c := _containers[_]
n := _milli_whole(c.resources.requests.cpu)
msg := sprintf(
"%s container %q: cpu request %q normalizes to \"%v\" — use canonical form to prevent ArgoCD OutOfSync",
[input.kind, c.name, c.resources.requests.cpu, n],
)
}
deny contains msg if {
input.kind == "Cluster"
n := _milli_whole(input.spec.resources.limits.cpu)
msg := sprintf(
"Cluster %s/%s: cpu limit %q normalizes to \"%v\" — use canonical form to prevent ArgoCD OutOfSync",
[input.metadata.namespace, input.metadata.name, input.spec.resources.limits.cpu, n],
)
}
deny contains msg if {
input.kind == "Cluster"
n := _milli_whole(input.spec.resources.requests.cpu)
msg := sprintf(
"Cluster %s/%s: cpu request %q normalizes to \"%v\" — use canonical form to prevent ArgoCD OutOfSync",
[input.metadata.namespace, input.metadata.name, input.spec.resources.requests.cpu, n],
)
}
# ---- Memory: Mi values divisible by 1024 normalize to Gi ----
# k8s converts 1024Mi → 1Gi, 2048Mi → 2Gi, etc.
_mi_canonical(mem) := canonical if {
endswith(mem, "Mi")
val := to_number(substring(mem, 0, count(mem) - 2))
val % 1024 == 0
val > 0
canonical := sprintf("%vGi", [val / 1024])
}
deny contains msg if {
c := _containers[_]
canonical := _mi_canonical(c.resources.limits.memory)
msg := sprintf(
"%s container %q: memory limit %q normalizes to %q — use canonical form to prevent ArgoCD OutOfSync",
[input.kind, c.name, c.resources.limits.memory, canonical],
)
}
deny contains msg if {
c := _containers[_]
canonical := _mi_canonical(c.resources.requests.memory)
msg := sprintf(
"%s container %q: memory request %q normalizes to %q — use canonical form to prevent ArgoCD OutOfSync",
[input.kind, c.name, c.resources.requests.memory, canonical],
)
}
deny contains msg if {
input.kind == "Cluster"
canonical := _mi_canonical(input.spec.resources.limits.memory)
msg := sprintf(
"Cluster %s/%s: memory limit %q normalizes to %q use canonical form to prevent ArgoCD OutOfSync",
[input.metadata.namespace, input.metadata.name, input.spec.resources.limits.memory, canonical],
)
}
deny contains msg if {
input.kind == "Cluster"
canonical := _mi_canonical(input.spec.resources.requests.memory)
msg := sprintf(
"Cluster %s/%s: memory request %q normalizes to %q — use canonical form to prevent ArgoCD OutOfSync",
[input.metadata.namespace, input.metadata.name, input.spec.resources.requests.memory, canonical],
)
}
# ---- Service: clusterIP must not be null ----
# k8s assigns a real IP on creation; clusterIP is immutable afterward.
# Setting null in git means desired=null vs live=10.x.x.x → permanent OutOfSync.
# Remove the field and let k8s own it.
deny contains msg if {
input.kind == "Service"
input.spec.clusterIP == null
msg := sprintf(
"Service %s/%s has 'clusterIP: null' remove this field; Kubernetes assigns the IP on creation and it causes ArgoCD OutOfSync",
[input.metadata.namespace, input.metadata.name],
)
}