Buckets could not choose which RGW placement target (and thus durability
profile) backs them, so all data landed on the cluster default. The estate's
radosgw exposes two targets - default-placement (3x replicated) and ec (4+1
erasure-coded) - and archival workloads want ec.
- Validate spec.placementTarget: a DNS-ish pattern, 63-char cap, and a CEL
self==oldSelf immutability rule (RGW fixes placement at bucket creation and
cannot move a bucket between targets); make spec.zonegroup immutable too.
- Thread the target into the S3 CreateBucket LocationConstraint via the existing
helper; an empty zonegroup yields ":<target>", selecting the local zonegroup
so callers need not name the zonegroup api-name.
- Read the live placement_rule and zonegroup back from the Admin Ops bucket
stats and surface them: status.placementTarget plus a Placement print column.
- Guard the controller: if a live bucket's placement differs from spec, set an
Error phase with a PlacementImmutable reason instead of deleting/recreating.
- Cover locationConstraint construction, placement readback (httptest), and the
placementConflict guard with tests; document targets and immutability in the
README and add config/samples/06-bucket-ec.yaml.
Claude-Session: https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
The operator strict-decodes CR specs, so when the in-cluster CRDs lag the
running operator (e.g. CRDs pinned to v0.1.0 while the operator ran v0.3.0),
new spec fields fail to decode with no operator-side signal. This adds an
advisory startup check so the mismatch is visible in the operator logs.
- Add CheckCRDVersions: GET each owned CRD and verify it carries a
version-sentinel spec field, logging a distinct WARNING per problem
(missing CRD vs. present-but-stale schema); advisory only, never exits.
- Keep the sentinel list (buckets/managePolicy, objectstoreusers/
retainOnDelete, bucketaccesses/rawStatements) in one place.
- Wire the check into operator startup after the manager config is available.
- Add apiextensions customresourcedefinitions get;list RBAC marker and
regenerate config/rbac/role.yaml.
- Promote k8s.io/apiextensions-apiserver to a direct dependency.
Claude-Session: https://claude.ai/code/session_016CEncETbf8cvy1PhsHfFHM
The operator previously assumed it created every user and bucket it managed:
reconciling an existing resource could overwrite its user attributes or wipe its
bucket policy, and deleting a CRD always deleted the underlying RGW object (only
Bucket had retainOnDelete). That made taking over pre-existing radosgw state
unsafe. Make adoption first-class.
- add retainOnDelete to ObjectStoreUser and BucketAccess (dedicated users), so
deleting the CRD orphans the RGW user instead of deleting it (symmetric with
Bucket)
- merge bucket policy instead of replacing it: the operator marks its own
statements with a cephrgwop* Sid and preserves any statement it does not own,
so adopting a bucket with a hand-written policy keeps it; add Bucket
managePolicy (default true) to opt out of policy management entirely
- only reconcile user attributes the spec sets: DisplayName when non-empty and
Suspended is now an optional *bool, so adopting a user does not reset them
- record adoption: ObjectStoreUser/Bucket status.adopted (+ printcolumn) is true
when the RGW object already existed on first reconcile
- add GetBucketPolicy + MergeBucketPolicy; keyed adoption detection off the
status identity field so a Pending owner wait does not mislabel it
- regenerate CRDs/deepcopy; add docs/adoption.md and
config/samples/05-adoption.yaml; cover the merge in policy_test.go
Claude-Session: https://claude.ai/code/session_016CEncETbf8cvy1PhsHfFHM
The BucketAccess model only offered three coarse levels (read-only/read-write/
full) applied to the whole bucket. Real grants often need to be scoped to a key
prefix, limited to a source network or TLS, restricted to specific actions, or
expressed as an arbitrary S3 statement. RGW (Reef 18.2+/Squid) honours the S3
bucket-policy features to do all of this; expose them on BucketAccess while
keeping the level as the ergonomic default.
- add BucketAccess spec fields: paths (key-prefix scoping), actions (action
override), conditions (sourceIPs + secureTransportOnly), rawStatements
(arbitrary S3 statements with the principal injected)
- extend ceph.Grant + BuildBucketPolicy to render prefixed object resources,
custom-action statements, S3 condition blocks, and raw statements, keeping
output deterministic (sorted, stable sids)
- translate the new spec fields into grants in the Bucket controller and
fingerprint grants so distinct fine-grained BucketAccess objects no longer
collapse on UID+level alone
- regenerate deepcopy + CRDs; add config/samples/04-access-fine-grained.yaml
- cover paths, action override, conditions, raw statements and determinism in
policy_test.go; document the fields in the README
Claude-Session: https://claude.ai/code/session_016CEncETbf8cvy1PhsHfFHM
Adds a Kubernetes operator that provisions Ceph RGW (S3) buckets and
access keys declaratively through the Ceph manager dashboard REST API.
Three CRDs in group ceph.unkin.net/v1alpha1:
- ObjectStoreUser: creates an RGW user, delivers its key pair to a Secret
- Bucket: creates an S3 bucket owned by an ObjectStoreUser; owns the
bucket's aggregate S3 policy (union of all BucketAccess grants)
- BucketAccess: grants read-only/read-write/full access, provisioning a
dedicated user (or reusing a referenced one) and delivering RW/RO keys
The internal/ceph client wraps the dashboard /api/auth, /api/rgw/user and
/api/rgw/bucket endpoints with lazy token auth and re-auth on 401. Bucket
policies are rendered deterministically and applied via the bucket
policy API (Reef 18.2+). Credentials come from the cephrgw-credentials
Secret via env. Includes generated CRDs/RBAC, samples, kind manifests,
Woodpecker CI, and docs/ceph-setup.md covering the required Ceph
dashboard account, RGW wiring and permissions.
unkinben
benvin