Commit Graph

15 Commits

Author SHA1 Message Date
unkinben 10854b6501 fix: preload correct rke2 airgap images to break CNI bootstrap deadlock (#512)
## Why

Four newly-provisioned el9_8 compute nodes (prodnxsr0014/0015/0018/0019) hang with canal's kube-flannel container in `ImagePullBackOff`: the flannel VXLAN overlay never comes up, so the node can't reach any in-cluster `198.18.200.x` LoadBalancer VIP.

Root cause is a stale airgap-preload version. The nodes boot rke2 **v1.33.11+rke2r1** from the rolling `latest/1.33` repo, whose canal manifest requests `hardened-flannel:v0.28.4` / `hardened-calico:v3.31.5` (build20260415). But `rke2::install` pinned the preloaded bundle to **v1.33.4+rke2r1** (flannel v0.27.2 / calico v3.30.2), so those images were never on disk. containerd then falls back to the `docker.io` mirror (artifactapi, `disable-default-registry-endpoint: true`), reachable only via the pod-overlay VIP that requires the very flannel image being pulled — a bootstrap deadlock. Older nodes (0001-0008) are versionlocked at 1.33.4 and still match their original bundle, so they were unaffected.

## Changes

- Bump pinned `rke2_version` `1.33.4` -> `1.33.11` so the versionlock, RPM `ensure`, and preloaded bundle all line up with the canal image tags the running binary requests. The default `rke2-images.linux-amd64.tar.zst` bundle already contains the canal CNI images (it is RKE2's default CNI), so no extra tarball is needed.
- Wire the airgap archive `source` to the `container_archive_source` class parameter (previously declared in the module but never consumed). The module keeps its generic upstream default; the artifactapi override (the pre-CNI-reachable source, same BGP/physical path the rke2 yum repos already use) lives in the k8s role hiera as `rke2::container_archive_source`.

Applies to servers and agents alike (`rke2::install` runs for both) and preloads `before => Service`, so the bundle lands before rke2 starts.

Notes:
- The `latest/1.33` repo is rolling, so the pinned version must be maintained as the repo advances; a follow-up to pin the yum channel to a fixed patch would remove the drift entirely.
- No terraform-artifactapi change is required. (If a canal-only preload were ever wanted, the github generic remote allowlist would need `rancher/rke2/.*/rke2-images-canal.linux-amd64.tar.zst$` added — but the default bundle already carries those images, so it is unnecessary.)

https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT
Reviewed-on: #512
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-08-08 18:18:55 +10:00
unkinben aeae26711f Convert RKE2 registries to template, disable default endpoints (#474)
## Summary
- Replace static `registries.yaml` with EPP template driven by `rke2::registries` hash
- Add `disable-default-registry-endpoint: true` to all mirrors — RKE2 will only use artifactapi and never fall back to upstream registries
- Registry configuration now fully managed via hiera data (`roles/infra/k8s.yaml`)

Reviewed-on: #474
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-29 22:30:48 +10:00
benvin 97d21c81c5 feat: make rke2 registries.yaml conditional on manage_registries (#472)
Add/Remove the registries.yaml file based on the manage_registries
boolean. We are leaving it on default=false now as the artifactapi
server was broken.

---------

Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #472
2026-06-27 07:50:31 +10:00
unkinben ceacfc85ae feat: restart rke2 when registries.yaml is deployed (#465)
- ensure we restart rke2 to pickup registries.yaml changes
- add a comment to registries.yaml to force a restart

Reviewed-on: #465
2026-05-06 23:11:20 +10:00
unkinben 0d412aebdb chore: deploy rke2 registries.yaml (#462)
ensure all new docker pulls are actioned through artifactapi

Reviewed-on: #462
2026-05-06 22:17:59 +10:00
unkinben c4d28d52bc chore: remove helm deploys from puppet (#444)
- migrate helm deployments to terraform

Reviewed-on: #444
2026-01-30 20:52:51 +11:00
unkinben 1077bdcbc1 chore: update ceph gpgkey (#438)
- stop checking ceph gpgkey (fixme)
- use artifactapi for retrieving large rke image bundle

Reviewed-on: #438
2026-01-16 23:51:11 +11:00
unkinben efbbb6bcb1 feat: moderate the k8s install (#403)
- only install a base config
- wait for 3 masters before deploying helm charts
- remove cluster-domain
- manage nginx ingres via rke2 helmconfig

Reviewed-on: #403
2025-10-12 17:50:24 +11:00
unkinben a9c959d924 fix: remove unicode from ceph-csi-yaml (#400)
Reviewed-on: #400
2025-09-21 00:41:06 +10:00
unkinben b224cfb516 fix: cattle-system namespace (#399)
- cattle-system namespace is created earlier than helm
- leave namespaces.yaml to manage cattle-system namespace (required
  before installing helm/rancher)

Reviewed-on: #399
2025-09-21 00:21:41 +10:00
unkinben 4c9204858e feat: define node-token from puppet (#398)
- define the token on the bootstrap node too, so node-token is defined
  for new clusters

Reviewed-on: #398
2025-09-20 22:25:56 +10:00
unkinben 571a9b25a7 fix: resolve rke2-server errors (#397)
- kubectl yaml files must not use underscores
- replace unicode hyphen with ascii hyphen

Reviewed-on: #397
2025-09-20 18:40:18 +10:00
unkinben 762f415d2d feat: k8s helm rework (#396)
- remove helm-generated-yaml, replace with helm execs
- template/parameterise ceph csi

Reviewed-on: #396
2025-09-20 17:40:41 +10:00
unkinben 4e77fb7ee7 feat: manage rancher, purelb, cert-manager (#395)
This change will install rancher, purelb and cert-manager, then
configure a dmz and common ip pool to be used by loadbalancers. The
nginx ingres controller is configured to use 198.18.200.0 (common) and
announce the ip from all nodes so that it becomes an anycast ip in ospf.

- manage the install of rancher, purelb and cert-manager
- add rancher ingress routes
- add nginx externalip/loadBalancer

Reviewed-on: #395
2025-09-14 20:59:39 +10:00
unkinben 6e4bc9fbc7 feat: adding rke2 (#394)
- manage rke2 repos
- add rke2 module (init, params, install, config, service)
- split roles::infra::k8s::node -> control/compute roles
- moved common k8s config into k8s.yaml
- add bootstrap_node, manage server and token fields in rke2 config
- manage install of helm
- manage node attributes (from puppet facts)
- manage frr exclusions for service/cluster network

Reviewed-on: #394
2025-09-14 13:27:49 +10:00