Commit Graph

12 Commits

Author SHA1 Message Date
unkinben 10854b6501 fix: preload correct rke2 airgap images to break CNI bootstrap deadlock (#512)
## Why

Four newly-provisioned el9_8 compute nodes (prodnxsr0014/0015/0018/0019) hang with canal's kube-flannel container in `ImagePullBackOff`: the flannel VXLAN overlay never comes up, so the node can't reach any in-cluster `198.18.200.x` LoadBalancer VIP.

Root cause is a stale airgap-preload version. The nodes boot rke2 **v1.33.11+rke2r1** from the rolling `latest/1.33` repo, whose canal manifest requests `hardened-flannel:v0.28.4` / `hardened-calico:v3.31.5` (build20260415). But `rke2::install` pinned the preloaded bundle to **v1.33.4+rke2r1** (flannel v0.27.2 / calico v3.30.2), so those images were never on disk. containerd then falls back to the `docker.io` mirror (artifactapi, `disable-default-registry-endpoint: true`), reachable only via the pod-overlay VIP that requires the very flannel image being pulled — a bootstrap deadlock. Older nodes (0001-0008) are versionlocked at 1.33.4 and still match their original bundle, so they were unaffected.

## Changes

- Bump pinned `rke2_version` `1.33.4` -> `1.33.11` so the versionlock, RPM `ensure`, and preloaded bundle all line up with the canal image tags the running binary requests. The default `rke2-images.linux-amd64.tar.zst` bundle already contains the canal CNI images (it is RKE2's default CNI), so no extra tarball is needed.
- Wire the airgap archive `source` to the `container_archive_source` class parameter (previously declared in the module but never consumed). The module keeps its generic upstream default; the artifactapi override (the pre-CNI-reachable source, same BGP/physical path the rke2 yum repos already use) lives in the k8s role hiera as `rke2::container_archive_source`.

Applies to servers and agents alike (`rke2::install` runs for both) and preloads `before => Service`, so the bundle lands before rke2 starts.

Notes:
- The `latest/1.33` repo is rolling, so the pinned version must be maintained as the repo advances; a follow-up to pin the yum channel to a fixed patch would remove the drift entirely.
- No terraform-artifactapi change is required. (If a canal-only preload were ever wanted, the github generic remote allowlist would need `rancher/rke2/.*/rke2-images-canal.linux-amd64.tar.zst$` added — but the default bundle already carries those images, so it is unnecessary.)

https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT
Reviewed-on: #512
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-08-08 18:18:55 +10:00
unkinben aeae26711f Convert RKE2 registries to template, disable default endpoints (#474)
## Summary
- Replace static `registries.yaml` with EPP template driven by `rke2::registries` hash
- Add `disable-default-registry-endpoint: true` to all mirrors — RKE2 will only use artifactapi and never fall back to upstream registries
- Registry configuration now fully managed via hiera data (`roles/infra/k8s.yaml`)

Reviewed-on: #474
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-06-29 22:30:48 +10:00
benvin 97d21c81c5 feat: make rke2 registries.yaml conditional on manage_registries (#472)
Add/Remove the registries.yaml file based on the manage_registries
boolean. We are leaving it on default=false now as the artifactapi
server was broken.

---------

Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #472
2026-06-27 07:50:31 +10:00
unkinben ceacfc85ae feat: restart rke2 when registries.yaml is deployed (#465)
- ensure we restart rke2 to pickup registries.yaml changes
- add a comment to registries.yaml to force a restart

Reviewed-on: #465
2026-05-06 23:11:20 +10:00
unkinben 0d412aebdb chore: deploy rke2 registries.yaml (#462)
ensure all new docker pulls are actioned through artifactapi

Reviewed-on: #462
2026-05-06 22:17:59 +10:00
unkinben c4d28d52bc chore: remove helm deploys from puppet (#444)
- migrate helm deployments to terraform

Reviewed-on: #444
2026-01-30 20:52:51 +11:00
unkinben 1077bdcbc1 chore: update ceph gpgkey (#438)
- stop checking ceph gpgkey (fixme)
- use artifactapi for retrieving large rke image bundle

Reviewed-on: #438
2026-01-16 23:51:11 +11:00
unkinben efbbb6bcb1 feat: moderate the k8s install (#403)
- only install a base config
- wait for 3 masters before deploying helm charts
- remove cluster-domain
- manage nginx ingres via rke2 helmconfig

Reviewed-on: #403
2025-10-12 17:50:24 +11:00
unkinben b224cfb516 fix: cattle-system namespace (#399)
- cattle-system namespace is created earlier than helm
- leave namespaces.yaml to manage cattle-system namespace (required
  before installing helm/rancher)

Reviewed-on: #399
2025-09-21 00:21:41 +10:00
unkinben 4c9204858e feat: define node-token from puppet (#398)
- define the token on the bootstrap node too, so node-token is defined
  for new clusters

Reviewed-on: #398
2025-09-20 22:25:56 +10:00
unkinben 762f415d2d feat: k8s helm rework (#396)
- remove helm-generated-yaml, replace with helm execs
- template/parameterise ceph csi

Reviewed-on: #396
2025-09-20 17:40:41 +10:00
unkinben 6e4bc9fbc7 feat: adding rke2 (#394)
- manage rke2 repos
- add rke2 module (init, params, install, config, service)
- split roles::infra::k8s::node -> control/compute roles
- moved common k8s config into k8s.yaml
- add bootstrap_node, manage server and token fields in rke2 config
- manage install of helm
- manage node attributes (from puppet facts)
- manage frr exclusions for service/cluster network

Reviewed-on: #394
2025-09-14 13:27:49 +10:00