Files
unkinben c1b3ba1c34
ci/woodpecker/pr/build Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
Add immutable placement-target selection to Bucket
Buckets could not choose which RGW placement target (and thus durability
profile) backs them, so all data landed on the cluster default. The estate's
radosgw exposes two targets - default-placement (3x replicated) and ec (4+1
erasure-coded) - and archival workloads want ec.

- Validate spec.placementTarget: a DNS-ish pattern, 63-char cap, and a CEL
  self==oldSelf immutability rule (RGW fixes placement at bucket creation and
  cannot move a bucket between targets); make spec.zonegroup immutable too.
- Thread the target into the S3 CreateBucket LocationConstraint via the existing
  helper; an empty zonegroup yields ":<target>", selecting the local zonegroup
  so callers need not name the zonegroup api-name.
- Read the live placement_rule and zonegroup back from the Admin Ops bucket
  stats and surface them: status.placementTarget plus a Placement print column.
- Guard the controller: if a live bucket's placement differs from spec, set an
  Error phase with a PlacementImmutable reason instead of deleting/recreating.
- Cover locationConstraint construction, placement readback (httptest), and the
  placementConflict guard with tests; document targets and immutability in the
  README and add config/samples/06-bucket-ec.yaml.

Claude-Session: https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
2026-07-29 00:16:31 +10:00

756 lines
33 KiB
YAML

# Generated by "make generate". DO NOT EDIT.
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: bucketaccesses.ceph.unkin.net
spec:
group: ceph.unkin.net
names:
kind: BucketAccess
listKind: BucketAccessList
plural: bucketaccesses
shortNames:
- ba
singular: bucketaccess
scope: Namespaced
versions:
- additionalPrinterColumns:
- jsonPath: .spec.bucketRef
name: Bucket
type: string
- jsonPath: .spec.level
name: Level
type: string
- jsonPath: .status.uid
name: UID
type: string
- jsonPath: .status.phase
name: Phase
type: string
name: v1alpha1
schema:
openAPIV3Schema:
description: |-
BucketAccess grants an RGW user read-only, read-write or full access to a
Bucket via the bucket's S3 policy.
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: |-
BucketAccessSpec grants an RGW user a level of access to a Bucket by
maintaining a statement in the bucket's S3 policy. If UserRef is empty the
operator provisions a dedicated user for this grant and writes its keys into
a Secret; otherwise it grants an existing ObjectStoreUser.
properties:
actions:
description: |-
Actions optionally overrides the S3 actions granted by Level. When set,
exactly these actions are granted, on the bucket and its (optionally
prefixed) objects. Ignored when RawStatements is set.
items:
type: string
type: array
bucketRef:
description: BucketRef names the Bucket (in this namespace) to grant
access to.
type: string
conditions:
description: |-
Conditions optionally restricts when the grant applies (e.g. source IPs,
TLS required). Ignored when RawStatements is set.
properties:
secureTransportOnly:
description: |-
SecureTransportOnly requires the request to use TLS, via the S3
aws:SecureTransport condition.
type: boolean
sourceIPs:
description: |-
SourceIPs restricts the grant to requests from these CIDRs (or single
addresses), via the S3 aws:SourceIp condition.
items:
type: string
type: array
type: object
level:
description: Level is the access level to grant.
enum:
- read-only
- read-write
- full
type: string
paths:
description: |-
Paths optionally scopes object-level access to these key prefixes within
the bucket; each becomes the resource "<bucket>/<prefix>*". Empty grants
the whole bucket. The bucket-level ListBucket action always applies to the
whole bucket. Ignored when RawStatements is set.
items:
type: string
type: array
rawStatements:
description: |-
RawStatements is an escape hatch for arbitrary S3 policy statements, merged
into the bucket policy for this grant's principal. When set, Level,
Actions, Paths and Conditions on this object are ignored; the operator only
fills in the Principal (this grant's user) when a statement omits one.
items:
description: |-
PolicyStatement is a raw S3 bucket-policy statement, exposed for grants that
need control beyond Level/Actions/Paths/Conditions.
properties:
actions:
description: Actions are the S3 actions the statement covers
(e.g. s3:GetObject).
items:
type: string
type: array
conditions:
additionalProperties:
additionalProperties:
items:
type: string
type: array
type: object
description: |-
Conditions is the raw S3 condition block: operator -> condition key ->
values, e.g. {"IpAddress": {"aws:SourceIp": ["10.0.0.0/8"]}}.
type: object
effect:
default: Allow
description: Effect is Allow or Deny. Defaults to Allow.
enum:
- Allow
- Deny
type: string
resources:
description: |-
Resources are S3 resource ARNs, or bucket-relative key prefixes when they
do not start with "arn:". Empty means the whole bucket and its objects.
items:
type: string
type: array
sid:
description: Sid is an optional statement id. The operator derives
one when empty.
type: string
required:
- actions
type: object
type: array
retainOnDelete:
description: |-
RetainOnDelete keeps the dedicated RGW user (created when UserRef is empty)
instead of deleting it when this BucketAccess is removed. Ignored when
UserRef is set (that user is never managed here). Defaults to false.
type: boolean
secretName:
description: |-
SecretName is the Secret the operator writes credentials into for the
dedicated user it creates (UserRef empty). Defaults to "<name>-rgw".
type: string
uid:
description: |-
UID overrides the id of the dedicated user created when UserRef is empty.
Defaults to "<bucket>-<name>". Ignored when UserRef is set.
type: string
userRef:
description: |-
UserRef optionally names an existing ObjectStoreUser (in this namespace)
to grant. When set, the operator does not create or delete a user and
SecretName is ignored (that user already owns its own credential Secret).
type: string
required:
- bucketRef
- level
type: object
status:
description: BucketAccessStatus reports observed grant state.
properties:
bound:
description: Bound reports whether the grant is reflected in the bucket
policy.
type: boolean
conditions:
items:
description: Condition contains details for one aspect of the current
state of this API Resource.
properties:
lastTransitionTime:
description: |-
lastTransitionTime is the last time the condition transitioned from one status to another.
This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.
format: date-time
type: string
message:
description: |-
message is a human readable message indicating details about the transition.
This may be an empty string.
maxLength: 32768
type: string
observedGeneration:
description: |-
observedGeneration represents the .metadata.generation that the condition was set based upon.
For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
with respect to the current state of the instance.
format: int64
minimum: 0
type: integer
reason:
description: |-
reason contains a programmatic identifier indicating the reason for the condition's last transition.
Producers of specific condition types may define expected values and meanings for this field,
and whether the values are considered a guaranteed API.
The value should be a CamelCase string.
This field may not be empty.
maxLength: 1024
minLength: 1
pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
type: string
status:
description: status of the condition, one of True, False, Unknown.
enum:
- "True"
- "False"
- Unknown
type: string
type:
description: type of condition in CamelCase or in foo.example.com/CamelCase.
maxLength: 316
pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
type: string
required:
- lastTransitionTime
- message
- reason
- status
- type
type: object
type: array
x-kubernetes-list-map-keys:
- type
x-kubernetes-list-type: map
observedGeneration:
format: int64
type: integer
phase:
description: Phase is a coarse lifecycle summary (Pending/Ready/Error).
type: string
secretName:
description: SecretName is the Secret holding the dedicated user's
credentials, if any.
type: string
uid:
description: UID is the RGW user id that was granted access.
type: string
type: object
type: object
served: true
storage: true
subresources:
status: {}
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: buckets.ceph.unkin.net
spec:
group: ceph.unkin.net
names:
kind: Bucket
listKind: BucketList
plural: buckets
shortNames:
- bkt
singular: bucket
scope: Namespaced
versions:
- additionalPrinterColumns:
- jsonPath: .status.bucketName
name: Bucket
type: string
- jsonPath: .status.owner
name: Owner
type: string
- jsonPath: .status.placementTarget
name: Placement
type: string
- jsonPath: .status.policyPrincipals
name: Grants
type: integer
- jsonPath: .status.adopted
name: Adopted
type: boolean
- jsonPath: .status.phase
name: Phase
type: string
name: v1alpha1
schema:
openAPIV3Schema:
description: Bucket is a Ceph RGW S3 bucket.
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: BucketSpec defines a Ceph RGW (S3) bucket owned by an ObjectStoreUser.
properties:
bucketName:
description: BucketName is the S3 bucket name. Defaults to metadata.name.
Immutable.
type: string
managePolicy:
default: true
description: |-
ManagePolicy controls whether the operator manages the bucket's S3 policy
from BucketAccess grants. When true (the default) the operator reconciles
its own statements while preserving any statements it does not own, so it
is safe to adopt a bucket that already has a policy. Set to false to leave
the bucket policy entirely untouched (BucketAccess grants then have no
effect on this bucket).
type: boolean
objectLock:
description: ObjectLock configures S3 object lock. Enabling it forces
versioning on.
properties:
days:
description: Days is the default retention period in days. Mutually
exclusive with Years.
format: int32
type: integer
enabled:
description: Enabled turns on object lock for the bucket.
type: boolean
mode:
description: Mode is the default retention mode applied to new
objects.
enum:
- GOVERNANCE
- COMPLIANCE
type: string
years:
description: Years is the default retention period in years. Mutually
exclusive with Days.
format: int32
type: integer
required:
- enabled
type: object
ownerRef:
description: |-
OwnerRef names the ObjectStoreUser (in this namespace) that owns the
bucket. The owner always has full control; grant additional principals
with BucketAccess objects.
type: string
placementTarget:
description: |-
PlacementTarget optionally selects the RGW placement target that backs the
bucket, choosing which pools (and thus replication/erasure profile) store
its data. Empty (the default) uses the owning user's default_placement, or
the zonegroup default. The valid values are cluster configuration, not a
fixed set; on this estate the two configured targets are
"default-placement" (3x replicated) and "ec" (4+1 erasure-coded).
Immutable: RGW chooses the placement at bucket creation (from the S3
LocationConstraint) and cannot move an existing bucket between placement
targets. Set it on a fresh Bucket; changing it later is rejected, and if a
pre-existing bucket is on a different placement the operator reports an
error instead of recreating it.
maxLength: 63
pattern: ^[a-zA-Z0-9]([a-zA-Z0-9._-]*[a-zA-Z0-9])?$
type: string
x-kubernetes-validations:
- message: placementTarget is immutable; RGW cannot move a bucket
between placement targets
rule: self == oldSelf
purgeOnDelete:
description: |-
PurgeOnDelete deletes the bucket together with all objects it contains
when the Bucket resource is removed. Dangerous; defaults to false.
type: boolean
quota:
description: Quota optionally applies a bucket-level quota.
properties:
enabled:
default: true
description: |-
Enabled turns the quota on. When false the other fields are ignored and
the quota is disabled on the target.
type: boolean
maxObjects:
description: MaxObjects caps the number of objects. Nil or negative
means unlimited.
format: int64
type: integer
maxSizeBytes:
description: MaxSizeBytes caps the total size in bytes. Nil or
negative means unlimited.
format: int64
type: integer
type: object
retainOnDelete:
description: |-
RetainOnDelete keeps the RGW bucket (and its objects) when the Bucket
resource is deleted. By default the operator removes the empty bucket;
it never purges objects unless PurgeOnDelete is also set.
type: boolean
tags:
additionalProperties:
type: string
description: Tags are bucket tags (key/value) applied to the bucket.
type: object
versioning:
description: Versioning enables S3 object versioning on the bucket.
type: boolean
zonegroup:
description: |-
Zonegroup optionally pins the bucket to a specific RGW zonegroup by its
api-name. Empty (the default) uses the cluster's local/master zonegroup, so
PlacementTarget selection works without naming the zonegroup. Immutable:
RGW resolves the zonegroup at bucket creation and cannot move it afterwards.
type: string
x-kubernetes-validations:
- message: zonegroup is immutable; RGW fixes it at bucket creation
rule: self == oldSelf
required:
- ownerRef
type: object
status:
description: BucketStatus reports observed bucket state.
properties:
adopted:
description: |-
Adopted reports that the RGW bucket already existed when the operator
first reconciled this resource (it was taken over, not created).
type: boolean
bucketID:
description: BucketID is the RGW internal bucket instance id.
type: string
bucketName:
description: BucketName is the provisioned S3 bucket name.
type: string
conditions:
items:
description: Condition contains details for one aspect of the current
state of this API Resource.
properties:
lastTransitionTime:
description: |-
lastTransitionTime is the last time the condition transitioned from one status to another.
This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.
format: date-time
type: string
message:
description: |-
message is a human readable message indicating details about the transition.
This may be an empty string.
maxLength: 32768
type: string
observedGeneration:
description: |-
observedGeneration represents the .metadata.generation that the condition was set based upon.
For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
with respect to the current state of the instance.
format: int64
minimum: 0
type: integer
reason:
description: |-
reason contains a programmatic identifier indicating the reason for the condition's last transition.
Producers of specific condition types may define expected values and meanings for this field,
and whether the values are considered a guaranteed API.
The value should be a CamelCase string.
This field may not be empty.
maxLength: 1024
minLength: 1
pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
type: string
status:
description: status of the condition, one of True, False, Unknown.
enum:
- "True"
- "False"
- Unknown
type: string
type:
description: type of condition in CamelCase or in foo.example.com/CamelCase.
maxLength: 316
pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
type: string
required:
- lastTransitionTime
- message
- reason
- status
- type
type: object
type: array
x-kubernetes-list-map-keys:
- type
x-kubernetes-list-type: map
observedGeneration:
format: int64
type: integer
owner:
description: Owner is the RGW uid that owns the bucket.
type: string
phase:
description: Phase is a coarse lifecycle summary (Pending/Ready/Error).
type: string
placementTarget:
description: |-
PlacementTarget is the placement target RGW actually stores the bucket on,
read back from the live bucket. It makes placement drift (a bucket landing
on a different target than spec requested) visible.
type: string
policyPrincipals:
description: |-
PolicyPrincipals is the number of extra principals granted via
BucketAccess and reflected in the bucket policy.
format: int32
type: integer
type: object
type: object
served: true
storage: true
subresources:
status: {}
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: objectstoreusers.ceph.unkin.net
spec:
group: ceph.unkin.net
names:
kind: ObjectStoreUser
listKind: ObjectStoreUserList
plural: objectstoreusers
shortNames:
- osu
singular: objectstoreuser
scope: Namespaced
versions:
- additionalPrinterColumns:
- jsonPath: .status.uid
name: UID
type: string
- jsonPath: .status.secretName
name: Secret
type: string
- jsonPath: .status.adopted
name: Adopted
type: boolean
- jsonPath: .status.phase
name: Phase
type: string
name: v1alpha1
schema:
openAPIV3Schema:
description: ObjectStoreUser is a Ceph RGW S3 user whose keys are delivered
into a Secret.
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: |-
ObjectStoreUserSpec defines a Ceph RGW (S3) user. The operator creates the
user through the radosgw Admin Ops API and writes its generated access/secret
key pair into a Kubernetes Secret. The key material is never stored on the
resource itself.
properties:
displayName:
description: DisplayName is the human-readable name for the user.
Defaults to the UID.
type: string
email:
description: Email is an optional email address recorded on the user.
type: string
maxBuckets:
default: 1000
description: |-
MaxBuckets caps how many buckets the user may own. A negative value
disables bucket creation; 0 leaves the RGW default. Defaults to 1000.
format: int32
type: integer
quota:
description: Quota optionally applies a user-level quota.
properties:
enabled:
default: true
description: |-
Enabled turns the quota on. When false the other fields are ignored and
the quota is disabled on the target.
type: boolean
maxObjects:
description: MaxObjects caps the number of objects. Nil or negative
means unlimited.
format: int64
type: integer
maxSizeBytes:
description: MaxSizeBytes caps the total size in bytes. Nil or
negative means unlimited.
format: int64
type: integer
type: object
retainOnDelete:
description: |-
RetainOnDelete keeps the RGW user (and its keys) when the ObjectStoreUser
resource is deleted, instead of removing it. Set this before adopting an
existing user you may later want to hand back. Defaults to false.
type: boolean
secretName:
description: |-
SecretName is the Secret the operator writes the access/secret key into.
Defaults to "<name>-rgw". The Secret holds AWS_ACCESS_KEY_ID,
AWS_SECRET_ACCESS_KEY, BUCKET_HOST and the RGW uid.
type: string
suspended:
description: |-
Suspended manages the user's suspended state: true suspends the user so
its keys stop working, false resumes it. When unset the operator does not
touch the suspended state (useful when adopting an existing user).
type: boolean
uid:
description: UID is the RGW user id. Defaults to metadata.name. Immutable
once created.
type: string
type: object
status:
description: ObjectStoreUserStatus reports observed user state.
properties:
adopted:
description: |-
Adopted reports that the RGW user already existed when the operator first
reconciled this resource (it was taken over, not created).
type: boolean
conditions:
items:
description: Condition contains details for one aspect of the current
state of this API Resource.
properties:
lastTransitionTime:
description: |-
lastTransitionTime is the last time the condition transitioned from one status to another.
This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.
format: date-time
type: string
message:
description: |-
message is a human readable message indicating details about the transition.
This may be an empty string.
maxLength: 32768
type: string
observedGeneration:
description: |-
observedGeneration represents the .metadata.generation that the condition was set based upon.
For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
with respect to the current state of the instance.
format: int64
minimum: 0
type: integer
reason:
description: |-
reason contains a programmatic identifier indicating the reason for the condition's last transition.
Producers of specific condition types may define expected values and meanings for this field,
and whether the values are considered a guaranteed API.
The value should be a CamelCase string.
This field may not be empty.
maxLength: 1024
minLength: 1
pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
type: string
status:
description: status of the condition, one of True, False, Unknown.
enum:
- "True"
- "False"
- Unknown
type: string
type:
description: type of condition in CamelCase or in foo.example.com/CamelCase.
maxLength: 316
pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
type: string
required:
- lastTransitionTime
- message
- reason
- status
- type
type: object
type: array
x-kubernetes-list-map-keys:
- type
x-kubernetes-list-type: map
observedGeneration:
format: int64
type: integer
phase:
description: Phase is a coarse lifecycle summary (Pending/Ready/Error).
type: string
secretName:
description: SecretName is the Secret holding the user's credentials.
type: string
uid:
description: UID is the RGW user id that was provisioned.
type: string
type: object
type: object
served: true
storage: true
subresources:
status: {}