Initial cephrgw-operator: Ceph RGW buckets & keys via dashboard API

Adds a Kubernetes operator that provisions Ceph RGW (S3) buckets and
access keys declaratively through the Ceph manager dashboard REST API.

Three CRDs in group ceph.unkin.net/v1alpha1:
- ObjectStoreUser: creates an RGW user, delivers its key pair to a Secret
- Bucket: creates an S3 bucket owned by an ObjectStoreUser; owns the
  bucket's aggregate S3 policy (union of all BucketAccess grants)
- BucketAccess: grants read-only/read-write/full access, provisioning a
  dedicated user (or reusing a referenced one) and delivering RW/RO keys

The internal/ceph client wraps the dashboard /api/auth, /api/rgw/user and
/api/rgw/bucket endpoints with lazy token auth and re-auth on 401. Bucket
policies are rendered deterministically and applied via the bucket
policy API (Reef 18.2+). Credentials come from the cephrgw-credentials
Secret via env. Includes generated CRDs/RBAC, samples, kind manifests,
Woodpecker CI, and docs/ceph-setup.md covering the required Ceph
dashboard account, RGW wiring and permissions.
This commit is contained in:
benvin
2026-07-18 00:07:22 +10:00
commit 1ea1713d6e
41 changed files with 4455 additions and 0 deletions
+13
View File
@@ -0,0 +1,13 @@
# The owner of a bucket. The operator creates an RGW user and writes its
# access/secret key pair into the Secret "app-owner-rgw".
apiVersion: ceph.unkin.net/v1alpha1
kind: ObjectStoreUser
metadata:
name: app-owner
namespace: default
spec:
displayName: "Application bucket owner"
maxBuckets: 50
quota:
enabled: true
maxSizeBytes: 107374182400 # 100 GiB
+16
View File
@@ -0,0 +1,16 @@
# A bucket owned by the app-owner user, with versioning enabled.
apiVersion: ceph.unkin.net/v1alpha1
kind: Bucket
metadata:
name: app-data
namespace: default
spec:
bucketName: app-data
ownerRef: app-owner
versioning: true
tags:
team: platform
env: prod
# By default the operator deletes the (empty) bucket when this object is
# removed. Set retainOnDelete: true to keep it, or purgeOnDelete: true to
# delete it together with all objects.
+13
View File
@@ -0,0 +1,13 @@
# Read-only credentials for the bucket. Because no userRef is given, the
# operator provisions a dedicated RGW user for this grant and writes its keys
# into the Secret "app-data-ro-rgw". The Bucket's S3 policy is updated to grant
# this user GetObject/ListBucket only.
apiVersion: ceph.unkin.net/v1alpha1
kind: BucketAccess
metadata:
name: app-data-ro
namespace: default
spec:
bucketRef: app-data
level: read-only
secretName: app-data-ro-rgw
+34
View File
@@ -0,0 +1,34 @@
# Read-write credentials for the bucket, delivered to a dedicated user and
# Secret "app-data-rw-rgw". The owner (app-owner) always retains full control;
# this grant is for a separate workload that needs to read and write objects
# but must not manage the bucket itself.
apiVersion: ceph.unkin.net/v1alpha1
kind: BucketAccess
metadata:
name: app-data-rw
namespace: default
spec:
bucketRef: app-data
level: read-write
secretName: app-data-rw-rgw
---
# Alternatively, grant an *existing* ObjectStoreUser access to the bucket by
# name. Here no user or Secret is created; the shared user's own credential
# Secret is used, and the bucket policy is extended to include it.
apiVersion: ceph.unkin.net/v1alpha1
kind: ObjectStoreUser
metadata:
name: analytics
namespace: default
spec:
displayName: "Analytics pipeline"
---
apiVersion: ceph.unkin.net/v1alpha1
kind: BucketAccess
metadata:
name: app-data-analytics
namespace: default
spec:
bucketRef: app-data
level: read-only
userRef: analytics