The in-cluster RGW only listens on plain HTTP via a ClusterIP, so S3 clients outside the cluster cannot reach it. TLS terminates in beast to avoid proxy body-size and timeout limits on large uploads.
- add a vault-issuer Certificate covering the existing radosgw SANs, rgw.k8s.syd1.au.unkin.net and 198.18.200.11
- mount the cert and enable beast ssl_port 7443, keeping 7480 for probes
- add rgw-external LoadBalancer on 198.18.200.11:443 with externalTrafficPolicy Local
Part of #582
Reviewed-on: #584
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Part of #582. RGW runs on VMs (ausyd1nxvm2115-2119) and needs to move into k8s under GitOps; this adds the `ceph-system` namespace with an in-cluster RGW behind a ClusterIP service. Cutover comes later.
- add `ceph-system` app to the platform applicationset
- add ceph.conf configmap with a shared `client.rgw.syd1` section and explicit keyring path
- add rgw keyring from vault `kubernetes/namespace/ceph-system/default/rgw-keyring` (`keyring`)
- add 3-replica `quay.io/ceph/ceph:v18.2.8` radosgw deployment, non-root, read-only root, startup probe, hard per-node spread, PDB + VPA
- add ClusterIP service `rgw` on 7480
Requires unkin/terraform-artifactapi#78 merged first.
Reviewed-on: #583
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>