The Vault KV layout is kubernetes/namespace/<ns>/<sa>/<secret>, so vlogs and logviewer both running as SA default in the shared logging namespace would collide on one oauth-credentials entry. Splitting vlogs out resolves it without widening any Vault policy.
- Move apps/base/logging/vlogs to apps/base/vlogs, namespace vlogs
- Add namespace.yaml and a vlogs-scoped VaultAuth (role default)
- Point the VaultStaticSecret at kubernetes/namespace/vlogs/default/oauth-credentials
- Add the au-syd1 overlay, platform ApplicationSet path and project destination
- Move the wildcard-unkin-net-tls reflection from logging to vlogs; vlogs was its only consumer there
Secret is already seeded at the new Vault path.
Reviewed-on: #507
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
The vector -> JetStream -> ClickHouse chain duplicated storage the in-cluster VictoriaLogs already provides, and its NATS StatefulSet keeps the logging Application from syncing.
- collect pod logs with a VLAgent DaemonSet remote-writing to vlinsert-logs
- delete vector, NATS, ClickHouse, logarchiver and logviewer
- drop clickhouse-system from the logging ApplicationSet
---------
Co-authored-by: BenVincent <benvin@main.unkin.net>
Reviewed-on: #506
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Vector is being removed from the logging stack in favour of a VictoriaLogs VLAgent, so every path this pipeline tests is going away and the pipeline becomes dead weight.
- Delete `.woodpecker/vector-test.yaml`.
Companion PR removes the vector manifests under `apps/base/logging/vector/`.
Reviewed-on: #505
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
bind-operator v0.3.1 fixes the apex NS bug two ways: the operator no longer glues apex NS records to pod IPs, and a new CEL rule on `DNSRecordSpec` rejects apex NS DNSRecords at admission. The CRDs are pinned by tag URL, so bumping only the image would leave the admission half inert.
- Bump the bind-operator image to `v0.3.1`
- Bump the CRD install URL to the matching `v0.3.1` tag
---------
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Reviewed-on: #503
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Visiting https://vlogs.unkin.net/ currently lands on the bare VictoriaLogs index page with relative links, not the query UI.
- Add an Exact `/` rule to the `vlogs` HTTPRoute that 302s to `/select/vmui/`
- Keep the PathPrefix `/` catch-all pointed at `vlogs-oauth2:80`, so auth is unchanged
Reviewed-on: #504
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
records.yaml had grown to ten DNSRecord documents across two zones, so finding or reviewing a single record meant scanning the whole file and every change touched it.
- move each record to authoritative/<zone>/<type>/<record>.yaml
- add a kustomization.yaml per zone and per type directory
- keep the git.unkin.net cutover record commented out, alongside a commented kustomization entry
- move the DNSRecord-namespace rationale onto the authoritative kustomization
- delete records.yaml
Rendered output is unchanged.
Reviewed-on: #501
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
A single records.yaml holding every DNSRecord means any record change touches a shared file, and deleting one record is a hunk edit rather than a file removal. One file per record under <zone>/<type>/<record>.yaml makes each record independently editable and removable.
- move acme-apex-ns to acme-unkin-net/ns/apex.yaml and acme-ns1-a to acme-unkin-net/a/ns1.yaml
- add kustomization.yaml per zone and type directory, referencing directories from the parent
- carry the delegation rationale onto the zone kustomization
- drop records.yaml and reference the zone directory from the app kustomization
Rendered output is unchanged.
Reviewed-on: #500
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
The vlselect query UI is only reachable in-cluster, so every log search needs a port-forward. Publish it at vlogs.unkin.net behind oauth2-proxy on the external (DMZ) Traefik.
- Add `apps/base/logging/vlogs`: external Gateway, http->https redirect and main HTTPRoute
- Terminate TLS with the reflected Let's Encrypt `*.unkin.net` wildcard; reflect it into `logging`
- Publish the `vlogs` A record at the DMZ gateway VIP from the bind-operator `unkin.net` zone
- Route all traffic through the `vlogs-oauth2` Service, upstreaming to `vlselect-logs:9471`
- Gate on the Authentik `vlogs` application, group `akP-vlogs-admin`
- Read OIDC credentials from `kv/kubernetes/namespace/logging/default/vlogs-oauth-credentials`
Depends on unkin/terraform-authentik#40.
Reviewed-on: #498
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
`connect` mode gives the client no local cache, so every cache operation is a network round trip -- about 2.7x slower than pointing the plugin straight at S3. Developers and CI run the plugin in direct mode instead.
- Remove the `go-cache-plugin serve` Deployment and its nginx stream-proxy ConfigMap
- Remove the PureLB Service, freeing `198.18.200.11`
- Keep the `gocache` bucket and `gocache-s3` Secret for direct-mode clients
Reviewed-on: #499
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
The k8s log pipeline stores to ClickHouse via NATS+vector, while the VM estate ships journald to a separate puppet-managed VictoriaLogs cluster. Consolidating on VictoriaLogs in-cluster collapses the two paths, and the logs-ingest gateway has no clients yet so it can be repointed now, ahead of the puppet change.
- add VLCluster `logs` at v1.52.0 (2 vlinsert, 2 vlselect, 3 vlstorage, 180d retention, 250Gi each on cephrbd-fast-delete)
- cap vlstorage disk use at 220GiB per node so 180d stays time-based rather than disk-bound
- repoint the logs-ingest HTTPRoute at `vlinsert-logs:9481`
- add a VictoriaLogs Grafana datasource and install its plugin
Nothing is removed here; NATS, ClickHouse, vector, logarchiver and logviewer keep running until a follow-up drops them.
Reviewed-on: #488
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
`go-cache-plugin serve` binds `127.0.0.1` only, so nothing outside the pod can reach it and laptops have no way to use the S3-backed Go cache without holding RGW credentials.
- Run `go-cache-plugin serve` against the `gocache` bucket, path-style, explicit region to skip the GetBucketLocation probe
- Add an nginx sidecar stream-proxying `:9090` to the loopback plugin port, `proxy_timeout 2h`
- Publish it on PureLB `198.18.200.11`, `externalTrafficPolicy: Local` so the client IP reaches the allow rules
- Restrict to workstation + pod CIDRs: GOCACHEPROG is unauthenticated and a poisoned entry runs in every consuming build
Merge only after `docker-internal/go-cache-plugin:v0.1.0` is published.
Reviewed-on: #496
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
The zone was seeded with an apex `NS ns1.acme.unkin.net` glued to the primary pod IP. Both were later corrected by hand, so the live RRset and the ns1 address exist only in the zone journal -- a reseed republishes the pod IP and breaks DNS-01 for every `*.unkin.net` cert. Declaring them makes git the source of truth.
- declare the two published apex NS names
- declare the in-zone ns1 address, which a seed would otherwise glue to the pod IP
Matches what the zone serves today, so applying it changes no records. Requires bind-operator v0.3.0.
Reviewed-on: #495
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
v0.3.0 converges a zone apex NS onto its declared nameservers instead of leaving the seed placeholder, which publishes a primary pod IP. The CRD moves with the image because the operator reads the new `spec.nameservers` field.
- pin the bind-operator image to v0.3.0
- pull the CRDs from the v0.3.0 tag
Reviewed-on: #494
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
The haproxy edge holds its own DMZ VIP, a second public entry point alongside
traefik-external that must be firewalled and DNS'd separately. Traefik can
front it with TLS passthrough, leaving haproxy's certs and backends untouched.
- Add a `traefik-external` Gateway: HTTP :80 plus Passthrough TLS :443.
- TLSRoute the 12 `fe_https.map` hostnames to haproxy:443; HTTPRoute 301s :80.
- Make the Service ClusterIP on 443 only, releasing 198.18.199.1.
- Drop `fe_http`, `be_letsencrypt` and `fe_http.map`; certs are DNS-01 only.
Client IP now reads as a Traefik pod — the Gateway provider cannot emit PROXY protocol to a TLSRoute backend. `sessionAffinity` goes too (it would pin Traefik pods, not clients); SRVNAME cookies keep persistence.
Reviewed-on: #492
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Stalwart was only ever a test deployment. The daemon is dead on all three
backend VMs and nothing public depends on it — `unkin.net` MX points at Google —
so the edge is proxying mail to nowhere and the tcp frontends make `defaults`
emit 20 spurious HTTP-mode warnings.
- Drop the `fe_smtp`, `fe_submission`, `fe_imap` and `fe_imaps` frontends.
- Drop the five `be_stalwart_*` backends and their map entries in `fe_http.map`/`fe_https.map`.
- Drop the now-unused 25/143/587/993 Service and container ports.
`haproxy -c` on the rendered config: exit 0, 0 warnings (was 20), 0 alerts.
Reviewed-on: #491
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Go builds on CI and laptops each rebuild the same packages from scratch. A
GOCACHEPROG backend needs an S3 bucket plus credentials before anything can
point at it, so provision those first. The bucket lives in the woodpecker
namespace because CI is the primary consumer and reads the Secret there.
- add Bucket and ObjectStoreUser for the shared Go build cache
- use default (replicated) placement rather than the ec target, since a build
cache is millions of small objects
- purge and drop the bucket and user on delete; the cache is disposable
Nothing consumes the bucket yet.
Reviewed-on: #489
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
A g10k-code job wedged in ContainerCreating on a failed CephFS mount and never reached a terminal condition, so it stayed in the CronJob active list and `concurrencyPolicy: Forbid` skipped every following minute. No Puppet code reached the estate for 6 days, and the piled-up missed slots crossed the controller 100-slot cap into `TooManyMissedTimes`. The CronJob carried no deadlines at all.
- Cap a job at `activeDeadlineSeconds: 300` on the Job spec, so a hang is failed as `DeadlineExceeded` and drops out of the active list (healthy runs take 16-18s).
- Set `startingDeadlineSeconds: 200`, bounding missed-schedule look-back to ~3 slots so the count cannot reach 100.
Reviewed-on: #490
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
The 13 `X-Frame-Options DENY if acl_<host>` rules in `fe_https` have never fired:
their ACLs use `req.hdr(host)`, a request-direction fetch that is invalid in a
response ruleset, so HAProxy rejects them at config-check time. Carried over
verbatim from the Puppet LXD config during the k8s move.
- Remove the 13 dead `http-response set-header X-Frame-Options` rules.
- Remove the 15 now-orphaned `acl acl_*` definition lines.
Not switching the header on: it has never been live, and Grafana/Gitea send their
own. `haproxy -c` warnings drop 33 -> 20; the two working response headers stay.
Reviewed-on: #487
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
The au-syd1 edge proxy runs on a hand-managed LXD container outside the cluster, with no HA and no shared config source.
- Add `apps/base/haproxy/`: 3 replicas behind the DMZ LoadBalancer 198.18.199.1, config from a ConfigMap, wildcard certs from reflected secrets.
- Keep source IPs via `externalTrafficPolicy: Local`; `sessionAffinity: ClientIP` stands in for the stick-table peers a Deployment cannot name.
- Drain on shutdown: `hard-stop-after 2m`, a preStop SIGUSR1 soft-stop, 150s grace.
- Bind the stats listener to 127.0.0.1 so it is port-forward only.
- Register the app in the platform project and ApplicationSet.
Reviewed-on: #485
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
The acme.unkin.net zone still serves only the bind-operator seed apex: NS ns1.acme.unkin.net glued to A 10.42.6.38, a pod IP no pod holds. The parent delegates to acme-ns1.unkin.net, but public resolvers have already promoted the child NS RRset, so when the cached address expires DNS-01 fails for every unkin.net wildcard at once.
- Add apex NS acme-ns1.unkin.net., matching the parent delegation (out of zone, no glue needed).
- Point ns1.acme.unkin.net at 103.216.191.185 so resolvers holding the seeded NS name still reach the zone.
- The operator seed placeholder itself is tracked separately in bind-operator.
---------
Co-authored-by: unkin-agent <agent@unkin.net>
Reviewed-on: #486
Co-authored-by: Unkin Agent <unkin-agent@unkin.net>
Co-committed-by: Unkin Agent <unkin-agent@unkin.net>
The haproxy edge terminates TLS for hosts under `main.unkin.net` and `ceph.unkin.net`, which the single `*.unkin.net` wildcard does not cover.
- Add cert-manager Certificates for both wildcards from the `letsencrypt` ClusterIssuer.
- Reflect the minted secrets into the `haproxy` namespace.
Needs these records in the public unkin.net zone first:
`_acme-challenge.main.unkin.net. CNAME _acme-challenge.main.acme.unkin.net.`
`_acme-challenge.ceph.unkin.net. CNAME _acme-challenge.ceph.acme.unkin.net.`
Reviewed-on: #484
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
profiles::pki::vault and profiles::ssh::sign shell out to
/usr/local/bin/certmanager and /usr/local/bin/sshsignhost from generate()
during catalog compilation. Neither binary exists in the compiler image, so
every node using them fails to compile.
- install certmanager v0.2.0 and sshsignhost v0.1.0 onto the shared bin volume with sha256 verification
- wrap both at /usr/local/bin from a pre-default entrypoint hook, failing startup loudly if either is missing
- mount read-only Vault configs for both: kubernetes auth on k8s/au/syd1, internal CA verified rather than skipped
Reviewed-on: #482
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
The deployed v0.3.3 build returns 500 from /Shows/NextUp on PostgreSQL, breaking the home screen, and lets replicas diverge: library-visibility and shared-config changes never propagate, user data (resume, played state, favourites, ratings) is overwritten between pods, and eight scheduled tasks run on every replica instead of only the scan leader. v0.4.0 carries the fixes.
- Pin cheeztv and fafflix to jellyfin-ha:v0.4.0.
No config change needed: cross-pod invalidation reuses the transcode-store Redis connection string both apps already set.
Reviewed-on: #483
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Deploy bind-operator v0.2.7. The operator seeded a fresh skeleton zone file at serial 1 over zones whose BIND journal was still on disk at a higher serial; BIND rejected the inconsistent pair (`addzone failed: out of range`) and, with a PVC per replica, the stale journal outlived restarts while every reconcile rewrote the skeleton, so it never converged. That SERVFAILed roughly 1 in 3 authoritative answers for k8s.syd1.au.unkin.net and resolvers cached the failures.
- Bumps the operator image to v0.2.7
- Bumps the CRD install pin to the v0.2.7 tag, which changes the CRDs
Expect one rolling restart of the operator Deployment as the new image lands.
Reviewed-on: #480
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
The new `unkin/jellyfin-plugin-sso` fork is getting a Woodpecker pipeline, and its build step will set `serviceAccountName: jellyfin-plugin-sso`. Without the SA declared here the pipeline pods fail to schedule.
- add a bare ServiceAccount `jellyfin-plugin-sso` in the `woodpecker` namespace
- register it in the woodpecker base kustomization
The step only builds .NET code, so no Vault kube-auth role or RBAC is needed.
Reviewed-on: #479
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Authentik advertises the token endpoint with a trailing slash, but some OIDC clients (the ArgoCD iOS app) POST to /application/o/token without one; Django's APPEND_SLASH will not redirect a POST, so the token exchange gets 405 and login fails.
- Add an exact-match rule on /application/o/token to the authentik and authentik-internal HTTPRoutes.
- Rewrite it to /application/o/token/ with a URLRewrite ReplaceFullPath filter, preserving the method and the authentik-server backend.
- Leave the catch-all PathPrefix rule untouched; exact matches outrank it in Gateway API precedence.
Reviewed-on: #478
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
The Authentik client for ArgoCD is now public (the iOS app can't hold
a secret), so Authentik no longer enforces client_secret on token
exchange. PKCE replaces that as the protection against
authorization-code interception.
- Add `enablePKCEAuthentication: true` to the `oidc.config` block in
`argocd-cm-patch.yaml`
- Note why PKCE is needed now that the client is public
Reviewed-on: #477
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Authentik server pods expose django_prometheus metrics on :9300, but only ldap-outpost and redis-exporter are scraped in this namespace. Add the missing per-app scrape.
- add apps/base/authentik/server-vmpodscrape.yaml selecting app.kubernetes.io/name=authentik, component=server on the metrics port
- wire it into apps/base/authentik/kustomization.yaml
Reviewed-on: #476
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
The cluster preloads no statement-statistics library, so there is no per-query cost attribution in postgres and slow query paths have to be inferred from application-side metrics instead of read straight out of the database.
- preload `pg_stat_statements`
- set `pg_stat_statements.max` and `.track`, which is what makes CNPG manage the extension and create it in every database
Requires a postgres restart. Stacked on `benvin/authentik-cnpg-resources`.
Reviewed-on: #475
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
The 500m CPU limit is a 50ms CFS quota per 100ms period, so the postgres pods are throttled on every burst even at ~0.01 cores average and each query pays that latency. 128MB of `shared_buffers` and a 256MB cache estimate also under-serve the planner on the joins authentik issues on its hot read paths.
- raise resources to requests `500m`/`1Gi`, limits `2`/`2Gi`
- raise `shared_buffers` to 512MB and `effective_cache_size` to 1536MB
- hold the post-incident memory headroom multiple over `shared_buffers`
Rolling restart with switchover. Stacked on `benvin/authentik-hot-standby-feedback`.
Reviewed-on: #474
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Authentik serves multi-second API reads from the CNPG hot standbys. Those reads outlive `max_standby_streaming_delay`, so recovery cancels them with `canceling statement due to conflict with recovery`, which authentik surfaces as HTTP 500 — enough to break a terraform apply mid-run.
- set `hot_standby_feedback` on so replicas report their oldest xmin to the primary and long reads stop being cancelled
- SIGHUP reload only, no restart or switchover
- retained-dead-tuple cost is negligible on a ~155MB database
Reviewed-on: #473
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Edits to the cheeztv/fafflix plugin ConfigMaps only reach the pods via the inject-plugin-config initContainer, so a config change sat inert until someone manually rolled the StatefulSet. Reloader is deployed cluster-wide with autoReloadAll disabled, so each workload has to opt in.
- annotate both StatefulSets with configmap.reloader.stakater.com/auto: "true"
Reviewed-on: #471
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
The internal-CA identity.k8s.syd1.au.unkin.net host has no CA bundle mounted in the jellyfin pods, so the OIDC discovery fetch fails TLS handshake (PartialChain). Authentik's discovery response is host-relative, so the browser-facing hostname must be used, not the internal one.
- Change OidEndpoint to identity.unkin.net in fafflix plugin config
- Change OidEndpoint to identity.unkin.net in cheeztv plugin config
Reviewed-on: #470
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
v0.3.0 and v0.3.1 crash-looped on Postgres migration/reader bugs and were reverted. v0.3.2 fixes both and was validated end to end against production-baseline Postgres and valkey: full migration chain completes, all previously-500 endpoints return 200, RedisTranscodeSessionStore and scan-leader gating confirmed active.
- Bump jellyfin-ha image tag v0.2.0 -> v0.3.2 in cheeztv and fafflix statefulsets
Depends on a pre-sync duplicate-username check and fresh pg_dump of both databases.
Reviewed-on: #468
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Catalog compiles fail with `LoadError: no such file to load -- toml`: server-side functions run in the puppetserver JRuby, whose gem path is separate from the agent CRuby path this hook installs into. puppet-prod's `profiles::puppet::gems` covers both; the hook only did the agent half.
- Install toml via `puppetserver gem`, mirroring the `puppetserver_gem` resource in puppet-prod
- Note in a comment that under `set -e` a failed install takes down an already-serving compiler
Reviewed-on: #467
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
v0.3.0 still serves facts from cache and collapses non-4xx upstream rejections into a 502, so clients see stale facts and lose the real status.
- Pin the pdbmux image to v0.4.0
Reviewed-on: #466
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
The deployed pin sits on v0.2.0, so pdbmux still answers malformed queries with `502 all backends failed` and resolves per-certname routes by configured backend order rather than by which backend actually owns the node.
Bump the pdbmux image pin to v0.3.0:
- Replay a unanimous upstream rejection (PuppetDB's real 400 + parse message) instead of a 502.
- Resolve per-certname routes to the node's owning backend by report freshness.
- Match backend addresses case-insensitively when redacting.
Reviewed-on: #464
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
catalog-diff compiles a host's catalog in two environments and diffs them to validate puppet-prod changes before merge, which means compiling catalogs on behalf of other nodes via POST /puppet/v4/catalog. The compilers run the image default auth.conf, where that endpoint is denied.
- add a compiler auth.conf allowing catalog-diff.main.unkin.net to POST /puppet/v4/catalog
- add a pre-default entrypoint script seeding it into conf.d, failing hard if the source is absent
- mount both onto the compiler deployment via configMapGenerator
Reviewed-on: #462
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Woodpecker's k8s backend provisions a per-pipeline workspace PVC sized by
WOODPECKER_BACKEND_K8S_VOLUME_SIZE. At 10G, large builds (e.g. .NET clone +
build output) leave too little free space for tests that hard-require free
disk headroom, failing purely on disk exhaustion.
- raise WOODPECKER_BACKEND_K8S_VOLUME_SIZE from 10G to 20Gi
Reviewed-on: #459
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
**Fix-forward companion to the #457 rollback. This is NOT the current outage fix — see below.**
## The actual outage
The UI is 503 because the Authentik application slug `artifactapi` **does not exist**. OIDC discovery 404s, so oauth2-proxy exits at startup, the Service has no ready endpoints, and Traefik answers `no available server`.
```
identity.unkin.net /application/o/artifactapi/… 404
identity.k8s.syd1.au.unkin.net /application/o/artifactapi/… 404
identity.unkin.net /application/o/repospawner/… 200
identity.unkin.net /application/o/argocd/… 200
```
Root cause is upstream in **terraform-authentik**: `ci/woodpecker/push/apply` on main HEAD `4e16401` **failed**. That apply has to succeed before any argocd-apps change can help. **This PR does not fix that.**
## What this PR does fix
#456 dropped the `combine-certs` initContainer and `OAUTH2_PROXY_PROVIDER_CA_FILES`, reasoning that `identity.unkin.net` serves a publicly trusted Let's Encrypt cert and so needs no internal CA. That holds for the browser redirect but not for oauth2-proxy's own back-channel discovery/token calls.
artifactapi is the **only one of six** oauth2-proxies in the estate without it:
| app | issuer host | `PROVIDER_CA_FILES` |
|---|---|---|
| arrproxy | identity.unkin.net | yes |
| logviewer | identity.unkin.net | yes |
| mediamark | identity.unkin.net | yes |
| repospawner | identity.unkin.net | yes |
| watchstate | identity.k8s… | yes |
| **artifactapi** | identity.unkin.net | **no** |
repospawner uses the **same public `identity.unkin.net` issuer** and still needs the internal bundle, which falsifies the removal reasoning. The existing comment on that initContainer states it plainly: *"The Authentik issuer is served behind the internal unkin.net CA."*
## Changes
- Add the `combine-certs` initContainer — byte-identical to repospawner's.
- Mount the combined bundle and set `OAUTH2_PROXY_PROVIDER_CA_FILES`.
- Reload the Deployment when `vault-ca-cert` rotates.
`vault-ca-cert` already exists in the `artifactapi` namespace (`api-deployment.yaml` uses it). `kustomize build apps/base/artifactapi` succeeds.
## Risk
Trust-only and strictly additive — it appends the internal CA to the system roots. Harmless if the back channel turns out to reach a publicly trusted endpoint after all. Expected to remove the *next* blocker, surfacing as x509, once the terraform-authentik apply lands.
## Sequencing
1. Fix and re-run terraform-authentik `push/apply` so the `artifactapi` application exists.
2. Merge this.
3. Confirm `/ui/` returns 200, then close#457 unmerged.
Only merge #457 instead if the UI must come back before step 1 can be done.
Reviewed-on: #458
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
The artifactapi web UI is open to anyone who can reach the host. Front it with Authentik SSO gated on akP-artifactapi-admin, while leaving the package-manager surfaces (/api/v1, /api/v2, /v2 docker registry, /terraform, /.well-known) untouched — dnf, containerd mirrors, buildah, terraform and CI publish steps cannot do a browser flow.
- Add the oauth2-proxy ConfigMap, Deployment, Service and VMPodScrape.
- Add the oauth-credentials VaultStaticSecret.
- Point the api-route /ui rule at oauth2-proxy and add a /oauth2 rule; the catch-all / rule still goes straight to the api Service on both listeners.
Requires terraform-authentik #34 applied and the Vault kv seed first.
Reviewed-on: #456
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
v0.2.0 ships the reports/events union, aggregate summing across backends, config-file support, and the removal of the primary/prefer ranking.
## How
- Pin the pdbmux Deployment image to `v0.2.0`.
- Leave `pdbmux-env` unchanged: `PDBMUX_LISTEN`, `PDBMUX_BACKENDS`, `PDBMUX_MERGE` are the only keys v0.2.0 reads from this ConfigMap, and `PDBMUX_PRIMARY`/`PDBMUX_PREFER` are already gone (#452).
Reviewed-on: #455
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Why: pick up v3.11.2, which moves DB migrations onto golib/pg with no behavior change.
- Bump the artifactapi API and UI image pins from v3.11.1 to v3.11.2.
Reviewed-on: #453
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
pdbmux#9 drops primary/prefer and makes configured backend order the only tie-break, so the current `old`-first list would silently reverse which PuppetDB wins.
## How
- Order `PDBMUX_BACKENDS` with `new=http://puppetdb.puppet.svc.cluster.local:8080` first and `old=http://puppetdbapi.service.consul:8080` second, URLs unchanged.
- Drop `PDBMUX_PRIMARY` and `PDBMUX_PREFER`; both already resolve to `new`, on the deployed v0.1.0 image (estate defaults) and on pdbmux main (first-backend fallback), so the rendered behaviour is unchanged today.
- Refresh the configmap and deployment comments to describe order-based precedence.
Merge this before pdbmux#9.
Reviewed-on: #452
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Why: the new vimpack repo's woodpecker pipelines set `serviceAccountName: vimpack-ci`, which does not exist in the woodpecker namespace yet.
- Add bare `vimpack-ci` ServiceAccount in `apps/base/woodpecker/` and register it in the kustomization.
Reviewed-on: #454
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
The `ghcr.io/openvoxproject/openvoxserver` image reads `OPENVOXSERVER_JAVA_ARGS` (`/etc/default/puppetserver`: `JAVA_ARGS=$OPENVOXSERVER_JAVA_ARGS`) and ships no `PUPPETSERVER_*` alias, so our heap/JMX flags have been inert since the fork switch — live masters and compilers run the image default `-Xms1024m -Xmx1024m` with no JMX.
## How
- Rename `PUPPETSERVER_JAVA_ARGS` to `OPENVOXSERVER_JAVA_ARGS` in `puppetserver-master-config`, `puppetserver-compiler-config` and `puppetserver-init-config`.
- Rename the same var on the `perms-and-dirs` init container in `deployment_puppetserver-compiler.yaml`.
- Flag values are unchanged (`-Xms1024m -Xmx3072m` plus the JMX flags); heap tuning is a separate call.
Reviewed-on: #451
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Why: pick up encapi v0.1.2, which moves DB migrations onto golib/pg with no behavior change (release pipeline green).
- Bump the encapi deployment container image from v0.1.1 to v0.1.2
Reviewed-on: #450
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
An unlimited environment timeout serves stale catalogs after code changes.
## How
- Set `OPENVOXSERVER_ENVIRONMENT_TIMEOUT: "0"` in the puppetserver master and compiler ConfigMaps.
Reviewed-on: #449
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
The new `golib` repo's Woodpecker pipeline needs a dedicated ServiceAccount to run its CI steps under.
## How
- Add bare ServiceAccount `golib-ci` in the `woodpecker` namespace and wire it into the base kustomization.
Reviewed-on: #447
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Why: repospawner v0.1.1 stops spawned job pods from automounting the API token.
- Bump the deployment image and the matching REPOSPAWNER_IMAGE env value to v0.1.1
Reviewed-on: #446
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
repospawner v0.1.0 is built and its Vault kubernetes auth role is applied, but nothing deploys it. It turns a "I want a new repository" request into a terraform-git pull request, follows that PR to merge, and optionally activates the repo in Woodpecker, so the review gate stays where it is instead of moving into an agent's hands.
## How
- Add `apps/base/repospawner/`: namespace, ServiceAccount `repospawner`, `default` VaultAuth for VSO, and a namespaced Role/RoleBinding granting jobs create/get/list/watch/delete plus pods and pods/log reads (mirrors mediamover).
- Deployment pinned to `artifactapi.k8s.syd1.au.unkin.net/docker-internal/repospawner:v0.1.0`, one replica with the `Recreate` strategy because request state is in memory and rebuilt from Job labels; the same image reference is passed down as `REPOSPAWNER_IMAGE` so the spawned Jobs stay in step.
- Mount a projected `audience: vault` service account token at `/var/run/secrets/vault` — the app logs into Vault natively rather than through VSO — and the `repospawner-woodpecker` Secret at `/etc/repospawner/woodpecker`, optional so the server still starts and refuses `woodpecker: true` with 503 when it is absent.
- Two VaultStaticSecrets: `oauth-credentials` from `kv/kubernetes/namespace/repospawner/default/oauth-credentials` and `repospawner-woodpecker` (key `token`) from `.../default/woodpecker`, with reloader annotations on both consumers.
- oauth2-proxy front door on the watchstate/mediamark pattern, gated on `akP-repospawner-admin` via the `ak_groups` claim and re-checked by the app from `X-Forwarded-Groups`; public `repospawner.unkin.net` on the reflected wildcard and internal `repospawner.k8s.syd1.au.unkin.net` on `vault-issuer`, both routed to the oauth2 Service.
- Register the overlay in the platform ApplicationSet and AppProject, and append `repospawner` to the wildcard Certificate's two reflector namespace lists.
Depends on the terraform-authentik `repospawner` client being applied and `kv/kubernetes/namespace/repospawner/default/oauth-credentials` + `.../woodpecker` being seeded.
Reviewed-on: #445
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
arrproxy v0.6.0 applies its own schema at startup under a Postgres advisory lock and holds `/readyz` until the schema is current, so every replica is safe to roll without an external gate. The wave-1 psql `arrproxy-migrate` Job and its SQL ConfigMap now only re-run idempotent statements the app already owns — dead weight, a second source of truth for the schema, and a standing drift trap whenever the app's embedded migrations move ahead of the manifests.
## How
- Bump `arrproxy-api` and `arrproxy-ui` to `v0.6.0`.
- Delete `migrate-job.yaml` and `migrations-configmap.yaml` and drop both from the arrproxy kustomization.
- Keep the wave-0/wave-2 split: wave 2 still orders the api behind the wave-0 CNPG Cluster and VSO-synced Secrets, which is independent of the migrate Job; the stale "serve only after the wave-1 migrate Job" comment is corrected.
- Rendered diff vs `main` is exactly the two image bumps plus the `arrproxy-migrate` Job and `arrproxy-migrations` ConfigMap disappearing; `kustomize build --enable-helm apps/overlays/au-syd1/arrstack` and pre-commit both clean.
Reviewed-on: #444
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
mediamark needs a home in the cluster: it marks/organises media on the shared mediastore tree and drives the adult-tier sonarr/radarr through arrproxy's hash routes. This adds the full app to the `media` project, mirroring the watchstate two-host oauth2-proxy pattern and the arrstack static-CephFS + projected-API-keys patterns.
## How
- Adds `apps/base/mediamark/`: namespace, VaultAuth (`k8s/au/syd1`, role `default`), three VaultStaticSecrets, static mediastore PV/PVC, the app Deployment, oauth2-proxy ConfigMap/Deployment, two Services, and internal + external Gateway/HTTPRoute pairs.
- Binds a dedicated static PV (`mediamark-mediastore`, own `volumeHandle`, `claimRef`-pinned) to the same CephFS mediastore subvolume arrstack/fafflix/cheeztv use, RWX 10Ti Retain, mounted at `/media`.
- Runs the app as 1000:1000 (deliberately not 65532) so it owns files on the shared media tree and hardlink/rename moves stay valid; read-only root filesystem, all caps dropped, no service-account token, `/livez` + `/readyz` probes.
- Projects the sonarr/radarr API keys as one file per app under `/etc/mediamark/keys`, mirroring arrproxy's keys projection, with reloader annotations on both secrets.
- Fronts both `mediamark.unkin.net` (traefik-external, reflected Let's Encrypt wildcard, no cert-manager annotations) and `mediamark.k8s.syd1.au.unkin.net` (traefik-internal, vault-issuer) with a single oauth2-proxy using a relative `/oauth2/callback` redirect; gated on `akP-mediamark-user` and passing identity to the app as `X-Forwarded-Groups` via `PASS_USER_HEADERS`.
- Appends `mediamark` to the `wildcard-unkin-net` Certificate's two reflector namespace lists, and registers the app in `argocd/applicationsets/media.yaml` + `argocd/projects/media.yaml` with a passthrough `apps/overlays/au-syd1/mediamark` overlay.
## Prerequisite seeds (Ben, before pods go Ready)
These KV paths must exist under `kv/kubernetes/namespace/mediamark/default/` — the `mediamark/default` templated policy already grants read, so no terraform-vault change is needed:
- `oauth-credentials` — needs `client_id` and `cookie_secret` added alongside the existing `client_secret` (Authentik mediamark provider; both absolute callback URIs registered there).
- `sonarr` — key `apitoken`.
- `radarr` — key `apitoken`.
## Validation
- `kustomize build --enable-helm apps/overlays/au-syd1/mediamark` (18 resources) and `.../cert-manager` both build.
- kubeconform clean on both touched overlays.
- `pre-commit run --all-files` passes.
Reviewed-on: #441
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
arrproxy v0.5.0 ships per-token HTTP method scoping for machine tokens, so a minted token can be limited to e.g. `GET` only. Zero-downtime: the mint-API field is additive and existing tokens get an empty methods list, which means unrestricted — they behave exactly as before.
## How
- Bump `arrproxy-api` and `arrproxy-ui` pins from v0.4.0 to v0.5.0.
- Mirror repo migrations `0002_tier_tokens.sql` and `0003_token_methods.sql` into the migrations ConfigMap. It had drifted at 0001 while v0.4.0 already queried `tier`/`read_only`, and every v0.5.0 token query selects `methods` — without this the new API errors on every token read.
- Have the wave-1 migrate Job apply all three files in order. Every statement is `IF NOT EXISTS`, so a resync over an already-migrated database is a no-op.
Rendered `kustomize build --enable-helm apps/overlays/au-syd1/arrstack` diff vs main is exactly the two image tags, the two added ConfigMap keys, and the two added `-f` args.
Reviewed-on: #443
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
The new repospawner repo needs a Woodpecker CI pipeline, and every pipeline step must run under its own ServiceAccount in the woodpecker namespace.
- Add `apps/base/woodpecker/serviceaccount_repospawner_ci.yaml` (bare SA `repospawner-ci`, ns woodpecker), mirroring the existing mediamark-ci SA.
- Register it in the woodpecker kustomization resources list.
Reviewed-on: #442
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
rke2's `registries.yaml` already rewrites upstream image names to the artifactapi mirror, so manifests must carry canonical upstream names. Only in-house `artifactapi.k8s.syd1.au.unkin.net/docker-internal/...` images stay explicit (logarchiver/logviewer are untouched).
Changes:
- Point the ClickHouseInstallation and the clickhouse-schema job at `docker.io/clickhouse/clickhouse-server:24.8`.
- Point the logviewer oauth2-proxy cert-combine init container at `docker.io/library/alpine:3`.
- Point the NATS bootstrap job at `docker.io/natsio/nats-box:0.18.0`.
- Point the NATS chart values at `docker.io/library/nats` and `docker.io/natsio/nats-server-config-reloader`.
- Point all three Vector values files (agent, aggregator, vm-ingest) at `docker.io/timberio/vector`.
- Drop the now-wrong "pulled through the artifactapi dockerhub remote" comments in the NATS and vector-agent values.
Tags/digests unchanged and the `repository`/`tag` split is preserved. `kustomize build --enable-helm apps/overlays/au-syd1/logging` differs from main only in those nine image strings.
Extra found, not changed here: `.woodpecker/vector-test.yaml` still pins its CI step image to `artifactapi.k8s.syd1.au.unkin.net/dockerhub/timberio/vector:0.57.0-debian`. That is a Woodpecker step image rather than a namespace manifest, so it is left out to keep this PR to the logging namespace — say the word and I will fix it separately.
Reviewed-on: #433
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
rke2's `registries.yaml` already rewrites upstream image names to the artifactapi mirror, so manifests must carry canonical upstream names. Only in-house `artifactapi.k8s.syd1.au.unkin.net/docker-internal/...` images stay explicit.
Changes:
- Set the valkey-operator chart `image.registry` to `ghcr.io`.
The `registry`/`repository`/`tag` split is untouched otherwise, so the rendered image is `ghcr.io/valkey-io/valkey-operator:v0.5.0`. `kustomize build --enable-helm apps/overlays/au-syd1/valkey-operator-system` differs from main only in that image string. No other proxied image refs in the file (the `helmCharts[].repo` entry in kustomization.yaml is a Helm chart repo, not a container registry, so it stays on artifactapi).
Reviewed-on: #437
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
`identity.unkin.net` moved from an internal `unkin.net` CA-issued cert to the LetsEncrypt `*.unkin.net` wildcard. `auth.generic_oauth`'s `tls_client_ca` pointed Grafana at the internal root only, so the OAuth handshake to the LE-issued cert now fails. Grafana's image trust store already contains the public roots.
## Changes
- Remove `tls_client_ca: /etc/grafana/vault-ca/ca.crt` (and its stale comment) from `auth.generic_oauth`.
- Remove the now-unused `vault-ca-cert` volume and volumeMount from the Grafana pod spec — nothing else in the pod referenced it (the CNPG `endpointCA` reference to `vault-ca-cert` for `s3.ceph.unkin.net` is a separate resource and stays).
- Leave the auth/token/api URLs, scopes and `role_attribute_path` untouched.
Reviewed-on: #440
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
`identity.unkin.net` moved from an internal `unkin.net` CA-issued cert to the LetsEncrypt `*.unkin.net` wildcard. The `oidc.config` block pinned the internal root as the *only* trust anchor, so argocd-server now rejects OIDC discovery with `x509: certificate signed by unknown authority` and SSO login is broken. The stock image trust store already carries the public roots.
## Changes
- Remove the `rootCA:` block from `argocd-cm`'s `oidc.config` for the `https://identity.unkin.net/application/o/argocd/` issuer.
- Replace the now-false internal-CA rationale comment with a one-liner noting the LE-issued cert needs no pin.
- Leave issuer, clientID, clientSecret ref, `requestedScopes` (incl. `ak_groups`) and `requestedIDTokenClaims` untouched.
Reviewed-on: #439
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
The mediamark Woodpecker docker step needs a dedicated ServiceAccount so it can push to the trusted in-cluster registry, mirroring the existing arrproxy-ci setup.
- Add bare ServiceAccount `mediamark-ci` in namespace `woodpecker` and register it in the woodpecker base kustomization.
Reviewed-on: #438
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
rke2's `registries.yaml` already rewrites upstream image names to the artifactapi mirror, so manifests must carry canonical upstream names. Only in-house `artifactapi.k8s.syd1.au.unkin.net/docker-internal/...` images stay explicit.
Changes:
- Point the Altinity chart crdHook at `docker.io/bitnami/kubectl`.
- Point the operator at `docker.io/altinity/clickhouse-operator`.
- Point the metrics exporter at `docker.io/altinity/metrics-exporter`.
- Reword the header comment that claimed all images are pulled through the artifactapi dockerhub remote.
Only the `repository` keys change; the chart still supplies the tags (0.27.2 / latest), so rendered tags are identical. `kustomize build --enable-helm apps/overlays/au-syd1/clickhouse-system` differs from main only in those three image strings. No other proxied refs in the file.
Reviewed-on: #436
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
rke2's `registries.yaml` already rewrites upstream image names to the artifactapi mirror, so manifests must carry canonical upstream names. Only in-house `artifactapi.k8s.syd1.au.unkin.net/docker-internal/...` images stay explicit.
Changes:
- Point the watchstate oauth2-proxy cert-combine init container at `docker.io/library/alpine:3`.
Tag unchanged. `kustomize build --enable-helm apps/overlays/au-syd1/watchstate` differs from main only in that image string. No extra proxied refs in the file (the oauth2-proxy image itself is already canonical `quay.io/...`).
Reviewed-on: #435
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
rke2's `registries.yaml` already rewrites upstream image names to the artifactapi mirror, so manifests must carry canonical upstream names. Only in-house `artifactapi.k8s.syd1.au.unkin.net/docker-internal/...` images stay explicit.
Changes:
- Point the netbox valkey metrics sidecar at `docker.io/oliver006/redis_exporter:v1.89.0`.
Tag unchanged. `kustomize build --enable-helm apps/overlays/au-syd1/netbox` differs from main only in that image string. No extra proxied refs in the file (the `valkey/valkey:8-alpine` container is already a bare upstream name).
Reviewed-on: #434
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
rke2's `registries.yaml` already rewrites upstream image names to the artifactapi mirror, so manifests must carry canonical upstream names. Only in-house `artifactapi.k8s.syd1.au.unkin.net/docker-internal/...` images stay explicit.
Changes:
- Point the litellm redis metrics sidecar at `docker.io/oliver006/redis_exporter:v1.89.0`.
Tag unchanged. `kustomize build --enable-helm apps/overlays/au-syd1/litellm` differs from main only in that image string. No extra proxied refs in the file (the `redis:7-alpine` container is already a bare upstream name).
Reviewed-on: #432
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
rke2's `registries.yaml` already rewrites upstream image names to the artifactapi mirror, so manifests must carry canonical upstream names. Only in-house `artifactapi.k8s.syd1.au.unkin.net/docker-internal/...` images stay explicit.
Changes:
- Point the gitea valkey metrics sidecar at `docker.io/oliver006/redis_exporter:v1.89.0`.
Tag unchanged. `kustomize build --enable-helm apps/overlays/au-syd1/gitea` differs from main only in that image string. No extra proxied refs in the file (the `valkey/valkey:8-alpine` container is already a bare upstream name).
Reviewed-on: #431
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
rke2's `registries.yaml` already rewrites upstream image names to the artifactapi mirror, so manifests must carry canonical upstream names. Only in-house `artifactapi.k8s.syd1.au.unkin.net/docker-internal/...` images stay explicit.
Changes:
- Point the authentik redis metrics sidecar at `docker.io/oliver006/redis_exporter:v1.89.0`.
Tag unchanged. `kustomize build --enable-helm apps/overlays/au-syd1/authentik` differs from main only in that image string. No extra proxied refs in the file (the `redis:7-alpine` container is already a bare upstream name).
Reviewed-on: #430
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
rke2's `registries.yaml` already rewrites upstream image names to the artifactapi mirror, so manifests must carry canonical upstream names. Only in-house `artifactapi.k8s.syd1.au.unkin.net/docker-internal/...` images stay explicit.
Changes:
- Point the artifactapi redis metrics sidecar at `docker.io/oliver006/redis_exporter:v1.89.0`.
Tag unchanged. `kustomize build --enable-helm apps/overlays/au-syd1/artifactapi` differs from main only in that image string. No extra proxied refs in the file (the `redis:7-alpine` container is already a bare upstream name).
Reviewed-on: #429
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
The media estate is currently split across two 1Ti CephFS subvolumes (`media-tv`
and `media-movies`), so a download that lands in one and imports to the other is
a cross-filesystem copy, not a hardlink move. A single 10Ti `mediastore`
subvolume (already created on `cephfs_csi_ssd_ec_4_1`) collapses that: one
filesystem holding `/{fafflix,cheeztv}/{tvseries,movies}` plus
`/nzbget/downloads/complete`, so every arr import is an atomic same-fs hardlink.
This PR only provisions the volume and seeds its tree. No existing PV, PVC or
workload is touched, and nothing mounts the new claims yet — the cutover of the
arrs and the two jellyfins is a separate change.
## How
- Add static PVs `arrstack-mediastore`, `fafflix-mediastore` and
`cheeztv-mediastore`, all pointing at the same rootPath
(`/volumes/csi_ssd_ec_4_1/mediastore/a0152dac-…`) with unique names and
volumeHandles pinned by `claimRef` — the established pattern for the shared
media subvolumes.
- Add the matching RWX 10Ti PVCs (`mediastore` in arrstack,
`fafflix-mediastore`, `cheeztv-mediastore`), annotated
`k8up.io/backup: "false"` and statically bound via `volumeName` +
`storageClassName: ""`.
- Add `mediastore-bootstrap`, a one-shot ArgoCD Sync-hook Job
(`hook-delete-policy: BeforeHookCreation`, no sync-wave needed) that mounts
the arrstack claim and `mkdir -p`s the directory tree as uid/gid 1000 —
the uid the arrstack media pods run as. Idempotent, so it self-heals on
every sync.
- Wire the new manifests into the arrstack, fafflix and cheeztv bases.
## Validation
- `kustomize build` clean on `apps/overlays/au-syd1/{arrstack,fafflix,cheeztv}`
- `kubeconform` clean on all three overlays (91 / 29 / 32 resources valid)
- pre-commit (yamllint, check-yaml, no-plain-secrets) passed
Reviewed-on: #428
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
rke2's `registries.yaml` already rewrites upstream image names to the artifactapi mirror, so manifests must carry canonical upstream names. Only in-house `artifactapi.k8s.syd1.au.unkin.net/docker-internal/...` images stay explicit.
Changes:
- Point the arrproxy migrate job at `docker.io/library/postgres:18-alpine`.
- Point the arrproxy oauth2-proxy cert-combine init container at `docker.io/library/alpine:3`.
- Point the arrstack ValkeyCluster at `docker.io/valkey/valkey:9.0.0`.
Tags are unchanged. `kustomize build --enable-helm apps/overlays/au-syd1/arrstack` differs from main only in those three image strings. No extra proxied refs found in these files (the oauth2-proxy image itself is already canonical `quay.io/...`).
Reviewed-on: #427
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Summary
- Adds SSO-Auth.xml, LDAP-Auth.xml, and branding.xml plugin config (rendered by an initContainer into /config) to fafflix and cheeztv
- Adds a per-namespace oauth-credentials VaultStaticSecret to each instance
- Adds a "Sign in with SSO" login link via jellyfin branding config
## Why
Phase-1 jellyfin auth: browser SSO via the 9p4 SSO plugin (Authentik OIDC, ak_groups claim) and phone/TV app-passwords via the LDAP plugin against the Authentik LDAP outpost. Config is declarative — restarts overwrite UI edits. Activates when the plugin-baked jellyfin-ha image lands (separate PR).
Reviewed-on: #424
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Summary
- Bumps the jellyfin-ha image to v0.2.0 on fafflix and cheeztv
## Why
v0.2.0 bakes the LDAP and SSO auth plugins into the image; with the plugin config PR this enables browser SSO and app-password logins.
Reviewed-on: #426
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Summary
- Bumps kea-operator image pin to v0.1.5
## Why
v0.1.4 fixed HA peer URLs but the HA hook then bound its dedicated listener to the ClusterIP and crashed; v0.1.5 disables the dedicated listener so HA flows via the ctrl-agent.
Reviewed-on: #425
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
why: production kea-dhcp4 is in a permanent crashloop — Kea's HA hook rejects DNS hostnames in peer URLs and v0.1.3 renders headless-DNS peer URLs; v0.1.4 ships the merged fix (per-peer ClusterIP Services + IP-literal URLs).
- bump kea-operator image pin to v0.1.4
Reviewed-on: #423
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
Jellyfin authenticates users against Authentik over LDAP (app-password binds). The `authentik-ldap` Service, Gateway, and TLSRoute already exist in the authentik base, but nothing backed the Service — no LDAP outpost was ever deployed. This adds the missing Deployment so in-cluster clients (Jellyfin) can bind.
Deployed as a separate Deployment inside the existing `authentik` namespace rather than a new app/namespace: the authentik overlay is already wired into the `platform` ApplicationSet and the LDAP Service/Gateway/TLSRoute already live here, so this needs no new AppProject/ApplicationSet wiring and keeps the outpost next to the core it serves.
## How
- `ldap-outpost-deployment.yaml`: 2-replica (stateless) Deployment, image `ghcr.io/goauthentik/ldap:2026.5.3` (canonical upstream name; matches the deployed authentik chart version; containerd mirrors route via artifactapi). Pod labels match the existing `authentik-ldap` Service selector.
- `AUTHENTIK_HOST=https://identity.k8s.syd1.au.unkin.net`, `AUTHENTIK_INSECURE=false`.
- Trusts the internal CA via the established combine-certs initContainer pattern (concatenates the base roots with the reflected `vault-ca-cert`) + `SSL_CERT_FILE`.
- `AUTHENTIK_TOKEN` sourced from secret `authentik-ldap-outpost-token`.
- Resources set; TCP probes on the LDAP port; reloader annotation for token + CA rotation.
- `ldap-outpost-vaultstaticsecret.yaml`: VSS (reuses the namespace `default` VaultAuth) materialising the token from KV `kubernetes/namespace/authentik/default/outpost-token` (key `token`).
- `ldap-service.yaml`: adds the plaintext `ldap` port 3389 alongside the existing `ldaps` 6636 (ClusterIP only, no public exposure — external reach is via the pre-existing internal Gateway/TLSRoute).
- `ldap-outpost-vmpodscrape.yaml`: VMPodScrape for the outpost metrics endpoint (:9300).
- No sync-wave annotations.
## Dependency — token seed (blocking)
The outpost pods CrashLoop until the token exists. After merge, the Authentik LDAP outpost token must be seeded into Vault KV at `kv/kubernetes/namespace/authentik/default/outpost-token` with key `token`. This path is provided by the corresponding terraform-authentik PR; the value must match the outpost token authentik issues.
## Validation
`kustomize build --enable-helm apps/overlays/au-syd1/authentik` + repo kubeconform: 38 resources, Valid: 38, Invalid: 0, Errors: 0.
Reviewed-on: #422
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
WatchState currently only has an internal front door (watchstate.k8s.syd1.au.unkin.net, PR #419, merged). This adds a public front door at **watchstate.unkin.net** via the external (DMZ) Traefik so the admin UI is reachable off-cluster, still fully gated by oauth2-proxy/Authentik. The internal gateway and routes are untouched; oauth2-proxy now fronts BOTH hostnames.
Stacked base note: the intended base (benvin/watchstate-deploy, PR #419) merged into main before this PR opened, so this targets `main`.
## How
- **DNS**: `watchstate-dns-internal` DNSRecord in `apps/base/bind-internal/authoritative/records.yaml` (unkin.net zone, bind-internal) -> `A 198.18.199.0`, the traefik-external VIP. Mirrors the `arrstack-dns-internal` precedent (external front door via bind, not external-dns). Kept in the central bind-internal location, so no AppProject destination widening is needed.
- **Gateway**: new `watchstate-external` Gateway (`gatewayClassName: traefik-external`), http + https listeners on hostname `watchstate.unkin.net`. TLS terminated with the Let's Encrypt `*.unkin.net` wildcard secret `wildcard-unkin-net-tls` — so **no cert-manager and no external-dns annotations**.
- **Routes**: `watchstate-external-http-redirect` (http->https 301) and `watchstate-external` (https -> `watchstate-oauth2:80`), mirroring the arrproxy/cheeztv external patterns.
- **oauth2-proxy dual-host**: `OAUTH2_PROXY_REDIRECT_URL` changed from the pinned cluster callback to the **relative** `/oauth2/callback`. With `OAUTH2_PROXY_REVERSE_PROXY=true` (already set), oauth2-proxy derives scheme+host per request from `X-Forwarded-Proto/Host` (verified in oauth2-proxy v7.15.3 `getOAuthRedirectURI`: host-less redirectURL falls through to request-derived), so the callback works on both `watchstate.unkin.net` and `watchstate.k8s.syd1.au.unkin.net`. `COOKIE_DOMAINS` and `WHITELIST_DOMAINS` now list both hosts (one cookie per host; a single parent-domain cookie can't cleanly span unkin.net vs k8s.syd1.au.unkin.net).
- Dropped the no-op `argocd.argoproj.io/sync-wave: "0"` annotation on the `default` VaultAuth. New resources intentionally carry no sync-wave annotations.
## Validation
- `kustomize build --enable-helm` on `apps/overlays/au-syd1/watchstate` and `.../bind-internal`: OK.
- kubeconform (repo CI config, k8s 1.33.7): watchstate overlay 15/15 valid (2 Gateways, 4 HTTPRoutes); bind-internal overlay 67/67 valid (incl. `watchstate-dns-internal`).
## Dependencies
- **Reflector allow-list (argocd-apps PR #418)**: reflects `wildcard-unkin-net-tls` into the `watchstate` namespace. This PR references that secret as if present.
- **Authentik (terraform-authentik, separate PR)**: register both `https://watchstate.unkin.net/oauth2/callback` and `https://watchstate.k8s.syd1.au.unkin.net/oauth2/callback` as redirect URIs on the watchstate provider.
Reviewed-on: #420
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
#417 stood up the central Let's Encrypt `*.unkin.net` wildcard (Certificate `wildcard-unkin-net` in `cert-manager`, secret `wildcard-unkin-net-tls`, emberstack-reflected). This wires the remaining single-label public hosts onto that publicly-trusted leaf instead of the internal Vault-PKI CA.
A `*.unkin.net` wildcard matches **single-label** subdomains only, so cluster hostnames under `*.k8s.syd1.au.unkin.net` are deliberately left on `vault-issuer`.
> Note: PR #417 already merged to `main`, so this is no longer stacked — it targets `main` directly.
## How
- **cert-manager**: reflect `wildcard-unkin-net-tls` into `arrstack`, `authentik`, `gitea` (appended to both reflector namespace lists → `cheeztv,arrstack,authentik,gitea`).
- **arrstack**: repoint the public `arrproxy` gateway listener (`arrstack.unkin.net`) `certificateRefs` → `wildcard-unkin-net-tls`; drop its `cert-manager.io/*` vault-issuer annotations.
- **authentik**: repoint the `identity.unkin.net` https listener → `wildcard-unkin-net-tls`. The `identity.k8s.syd1.au.unkin.net` listeners keep `authentik-tls` (vault-issuer), so the gateway `common-name` is retargeted to that cluster host. `authentik-ldap` gateway untouched.
- **gitea**: secret **reflected only**; `git.unkin.net` cutover deferred (no manifest change), same deferred pattern as fafflix.
## Validation
- `kustomize build` OK for cert-manager / arrstack / authentik / gitea overlays.
- `kubeconform` all valid (0 invalid, 0 errors) across the four overlays.
- Verified no `*.k8s.syd1.au.unkin.net` listener lost its vault-issuer wiring.
Reviewed-on: #418
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
Deploy WatchState (arabcoders/watchstate), the Jellyfin/Plex/Emby watch-state
sync tool, as an internal admin tool. It gets an admin-only web UI/API gated the
same way as logviewer: an oauth2-proxy front backed by Authentik OIDC.
## What
- New `watchstate` namespace + media-project app; base at `apps/base/watchstate`,
overlay at `apps/overlays/au-syd1/watchstate`.
- Image `ghcr.io/arabcoders/watchstate:v1.10.3` (current release; canonical
upstream name, containerd mirrors route ghcr via artifactapi).
- `replicas: 1`, `strategy: Recreate`, single `5Gi` `cephrbd-fast-delete` RWO PVC
at `/config` — sqlite + the in-container cron/redis are single-writer.
- `runAsUser/runAsGroup/fsGroup: 1000` (image's rootless user); liveness/readiness
`GET /v1/api/system/healthcheck` on 8080 (route confirmed in upstream
`src/API/System/HealthCheck.php`, no auth guard).
- oauth2-proxy (mirrors logviewer 1:1) fronts every path. Authentik issuer
`identity.k8s.syd1.au.unkin.net`, redirect
`https://watchstate.k8s.syd1.au.unkin.net/oauth2/callback`. Authorization is
enforced Authentik-side (akR-global-admin only), so no oauth2-proxy group
allowlist is configured.
- Internal-only Gateway (`traefik-internal`) for
`watchstate.k8s.syd1.au.unkin.net`, `vault-issuer` TLS leaf, external-dns to
`198.18.200.4`. HTTP -> HTTPS redirect.
- `VaultStaticSecret` pulls OIDC creds from
`kv/kubernetes/namespace/watchstate/default/oauth-credentials`; `vault-ca-cert`
auto-reflects into the namespace.
- Registered in the media `ApplicationSet` + `AppProject`.
## Scrape decision
No `VMPodScrape`: WatchState exposes no Prometheus/`/metrics` endpoint.
## Follow-ups
- **Seed check:** the VaultStaticSecret expects `client_id`, `client_secret`, and
`cookie_secret` keys at the kv path. Only `client_secret` was confirmed seeded;
`client_id` and a generated `cookie_secret` must also be present or the
oauth2-proxy pod will not start.
- **Webhook ingestion:** the Jellyfin webhook endpoint (`/v1/api/webhook`)
currently sits behind oauth2-proxy like everything else. When sync is wired up,
the jellyfins pushing webhooks will need an auth-bypass or an apikey route for
that path.
Reviewed-on: #419
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
why: cheeztv.unkin.net needs a publicly-trusted cert; DNS-01 via the delegated acme.unkin.net zone is now working (staging wildcard verified end-to-end in ~70s). A single centrally-managed wildcard avoids per-host CNAMEs and LE duplicate-order rate limits.
- add Certificate wildcard-unkin-net (cert-manager ns, ClusterIssuer letsencrypt prod, secret wildcard-unkin-net-tls) with emberstack reflector secretTemplate annotations
- reflect the secret into the cheeztv namespace (append future consumers to the two reflector namespace lists)
- point the public cheeztv gateway listener at wildcard-unkin-net-tls and drop its vault-issuer annotations (cheeztv-internal stays on vault-issuer)
Reviewed-on: #417
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
The old bare `sonarr`/`radarr` Services are gone, but arrproxy still ran the v0.3.x built-in default topology pointing at non-existent `sonarr`/`radarr` upstreams and mounted dead `sonarr-apikey`/`radarr-apikey` Secrets. This cuts the front door over to arrproxy v0.4.0 tier routing so the adult (fafflix, `/3aa168`) and kids (cheeztv, `/3df803`) *arr instances are reachable again, with per-tier group authorization.
## Changes
- Bump `arrproxy-api` and `arrproxy-ui` images to `v0.4.0`.
- Add `arrproxy-tiers` ConfigMap (`ARRPROXY_TIERS_FILE`): `fafflix` + `cheeztv` tiers, each with sonarr/radarr upstreams, route hash, key subdir and group grants. Adults (`akP-media-fafflix`) reach both tiers all methods; kids (`akP-media-cheeztv`) reach only cheeztv, GET/HEAD. `legacyRoutes:false` retires the unprefixed routes.
- Because arrproxy strips `/<hash>` and proxies `/<app>/...`, each upstream URL carries the `/<hash>` path segment so the joined upstream path (`/3aa168/sonarr/...`) matches the *arr's own UrlBase.
- Mount the tiers file and rework the projected keys volume to the real per-instance `{sonarr,radarr}-{adult,kids}-apikey` Secrets under `<keyDir>/<app>` (`adult/*`, `kids/*`); refresh reloader annotations.
- oauth2-proxy: route `/3aa168` and `/3df803` to arrproxy-api (drop dead `/sonarr` `/radarr` `/prowlarr`; prowlarr is served directly), and widen `SKIP_AUTH_REGEX` to `^/[^/]+/[^/]+/api` for the hash-prefixed tier api paths.
Group headers keep flowing via the existing Authentik `ak_groups` claim -> oauth2-proxy `X-Forwarded-Groups` mechanism. Validated with `kustomize build` + repo kubeconform (88/88 valid). No new Vault seeds required — the per-instance apikey Secrets already exist.
Reviewed-on: #416
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
Rebuild the adult media instance as `fafflix`, mirroring the kids instance (`cheeztv`) HA pattern (Postgres-backed jellyfin-ha fork, Valkey transcode-lease store, shared-RWX transcode, CNPG + k8up/restic backups, static CephFS media PVs). Ben: "replace the adult jellyfin with the same pattern as kids." The adult instance holds no data currently, so the wipe/replace is sanctioned.
Stacked on top of `benvin/cheeztv` (#413) — base is that branch, not main.
## How
- Add `apps/base/fafflix/` — cheeztv pattern with `s/cheeztv/fafflix` on names/namespace/labels/secrets/buckets and the Vault kv path `kubernetes/namespace/fafflix/default/k8up-restic`.
- Media: mount the shared movies/tv CephFS subvolumes' `adult` subtree at `/media/movies` and `/media/tv`, plus the `kids` subtree at `/media/movies-kids` and `/media/tv-kids` (same two static PVs, new unique volumeHandles `fafflix-media-{movies,tv}-static`) so fafflix can resume kids content started on cheeztv. All media mounts readOnly, matching cheeztv.
- **Hostname preserved:** fafflix keeps serving the legacy jellyfin host `jellyfin.k8s.syd1.au.unkin.net` (TLS secret `fafflix-tls`, cert-manager re-mints). The dedicated fafflix domain switch is explicitly deferred — no `fafflix.unkin.net` created.
- config PVC on `cephfs-raid5-delete` (delete variant).
- Remove `apps/base/jellyfin/` + its overlay; swap `jellyfin`->`fafflix` in the `media-apps` ApplicationSet directory glob and the `media` AppProject namespace destination.
- Third-party images stay canonical upstream; the in-house `jellyfin-ha` image ref is unchanged.
## Verification
- `kustomize build` + `kubeconform -strict` clean on the fafflix overlay and the whole media appset scope (fafflix/cheeztv/arrstack): 0 invalid, 0 errors.
- No dangling `apps/base/jellyfin` references; `jellyfin.k8s.syd1.au.unkin.net` now served by exactly one app (fafflix); no `fafflix.unkin.net` anywhere.
Reviewed-on: #415
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
Kids need their own safe Jellyfin front (cheeztv.unkin.net) with a library that only ever exposes kids titles, while those same titles stay resumable in the existing adult instance (fafflix). The fafflix.unkin.net domain switch is deliberately out of scope.
## How
- **New `cheeztv` instance** (`apps/base/cheeztv`): mirrors the fafflix HA Jellyfin stack 1:1 - same `jellyfin-ha:v0.1.3` fork image, 2-replica StatefulSet, RWX transcode PVC + HA takeover, Intel iGPU transcode, CNPG Postgres + PgBouncer, Valkey transcode store, k8up config backup, VMPodScrape.
- **Kids-only media**: its own static PVs on the *same* shared movies/tv CephFS subvolumes, mounted with `subPath: kids` so it sees only `movies/kids` and `tvshows/kids`.
- **Fully separate state**: own namespace, config PVC, `cheeztv-postgres` cluster, `cheeztv-valkey`, and `cnpg-cheeztv` / `cheeztv-config-backup` RGW buckets - nothing shared with fafflix.
- **DNS/cert**: `cheeztv.unkin.net` via internal Traefik + external-dns (198.18.200.4) + cert-manager `cheeztv-tls`, matching logviewer.unkin.net; plus the cluster hostname `cheeztv.k8s.syd1.au.unkin.net` mirroring fafflix.
- **fafflix change**: ADDS `movies/kids` + `tvshows/kids` subPath mounts alongside its current media mounts so kids libraries are resumable there. Existing mounts, hostname and ingress untouched.
- Registers cheeztv in the media ApplicationSet generator and AppProject destinations.
## Validation
kubeconform clean: cheeztv 28/28, jellyfin 25/25 valid. No arr/watchstate/domain changes.
Note: the k8up restic repo password must be seeded at `kv/kubernetes/namespace/cheeztv/default/k8up-restic` (same manual step as fafflix); templated Vault policy already grants read, no terraform-vault change needed.
Reviewed-on: #413
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Splits the arrstack media pipeline into an adult and a kids tier so kids content lives in its own libraries, databases, and downloader while reusing the shared prowlarr, CNPG cluster, Valkey, media PVCs, and S3 buckets. Backend workloads only — no routing/ingress, no arrproxy, no adult-instance changes in this PR.
How:
- sonarr-kids / radarr-kids: mirror the adult -unkin fork Deployments (same images/tags, 3 active-active replicas) on their own Postgres DBs (sonarr-kids-main / radarr-kids-main), UrlBase /3df803/sonarr and /3df803/radarr, and kids MediaCover/Backup S3 prefixes.
- Media mounts use the tvshows/kids and movies/kids subPaths of the shared media-tv / media-movies PVCs, so kids libraries are isolated on the same CephFS subvolumes.
- nzbget-kids: second downloader on its own RWO config PVC, completed downloads land in the kids media subtrees for same-filesystem hardlink imports.
- postgres: two managed roles + Database CRDs and their VSO-synced <app>-kids-db credential secrets.
- per-app VMPodScrape for each kids exportarr sidecar; new apps registered in the base kustomization.
Validated: kustomize build + kubeconform (96/96 valid), yamllint + pre-commit clean.
Reviewed-on: #412
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Deploy the new space-pirate UI background released in arrproxy v0.3.1.
- Bump arrproxy-api image to v0.3.1
- Bump arrproxy-ui image to v0.3.1
Reviewed-on: #411
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
Tofu applies against authentik hit a cross-replica read-back race: a create lands on one of the 3 server pods and the provider's immediate read-back hits another pod that returns stale data, failing the run with "inconsistent result after apply" (upstream bug goauthentik/terraform-provider-authentik#954, still open). ClientIP session affinity keeps a whole apply run on a single pod, sidestepping the race until the provider is fixed.
## Changes
- Sets `server.service.sessionAffinity: ClientIP` with `sessionAffinityConfig.clientIP.timeoutSeconds: 60` in the au-syd1 authentik overlay values (rendered by the vendored chart's `templates/server/service.yaml`)
Reviewed-on: #410
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-08-23 23:13:51 +10:00
299 changed files with 7205 additions and 3974 deletions
# traefik-internal gateway VIP; the ghp Gateway serves ghp.unkin.net there.
- 198.18.200.4
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
unkin-agent
unkinben