Commit Graph

5 Commits

Author SHA1 Message Date
unkinben ff1a30823c fix: lock rke2-common and pin rke2 to 1.33.13~rke2r2 to unblock puppet (#516)
## Why

Every puppet run on k8s nodes (e.g. prodnxsr0002) fails on `Package[rke2-server]` and stops applying the rest of the catalog, so the nodes stop receiving all further package/config updates:

```
change from '1.33.4~rke2r1-1.el9' to '1.33.11~rke2r1' failed: Could not update:
dnf upgrade rke2-server-1.33.11~rke2r1 returned 1:
 Problem: problem with installed package rke2-common-1.33.13~rke2r2-0.el9.x86_64
  - package rke2-server-1.33.11~rke2r1 requires rke2-common = 1.33.11~rke2r1, but none of the providers can be installed
  - cannot install the best update candidate for package rke2-server
```

`rke2::install` versionlocks only `rke2-server`/`rke2-agent`, never their strict (`= version`) `rke2-common` dependency. `rke2-common` is served from the rolling `rancher-rke2-1.33-latest` channel, whose head is now `1.33.13~rke2r2`, so `rke2-common` drifted up to `1.33.13~rke2r2` while the pin (`#512`) sat at `1.33.11~rke2r1`. `dnf upgrade` cannot downgrade the newer `rke2-common` to satisfy the older server, so the transaction fails. This is the rolling-channel drift `#512` flagged as needing follow-up.

## How

- Versionlock `rke2-common` to the same `${rke2_version}~${rke2_release}` as the server/agent, so the rolling channel can no longer drift the dependency ahead of the pin.
- Bump `rke2_version` `1.33.11` -> `1.33.13` and `rke2_release` `rke2r1` -> `rke2r2` to match the current channel head and the already-drifted installed `rke2-common`, so the pinned server/agent, both versionlocks, and the preloaded airgap bundle resolve in one transaction.

Verified against the live artifactapi rke2 remote: `rke2-server-1.33.13~rke2r2-0.el9.x86_64.rpm` and the `v1.33.13+rke2r2` `rke2-images.linux-amd64.tar.zst` airgap bundle both serve HTTP 200.

Reviewed-on: #516
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-08-08 23:13:40 +10:00
unkinben 0fdc7c97d8 rke2,puppet: enforce versionlock before package install (#514)
## Why

A package versionlock is only effective if it is applied **before** the package resource is evaluated. When the package is upgraded/installed first and locked afterwards, dnf can pull an unpinned version and the lock then pins whatever landed. This is the exact drift that let new RKE2 nodes boot 1.33.11 instead of the pinned version, and it also matters because a global `dnf update -y` autoupdate timer (`profiles::yum::autoupdater`) relies on the versionlock plugin already having the lock written.

## Changes

- `rke2::install` orders the versionlock ahead of the package: adds `before => Package["rke2-${node_type}"]` so the lock exists before install/upgrade.
- `profiles::puppet::agent` (RedHat branch) adds `before => Package[$use_package]` on the versionlock for the same reason.
- Leaves `profiles::gitea::runner` and `profiles::vault::server` unchanged: both already order the versionlock ahead of the package (`before => Package` / `before => Class['vault']`).
- Changes ordering only; locked versions (including the RKE2 pin) are untouched. No dependency cycle is introduced (packages hold no back-reference to the versionlock).

https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT
Reviewed-on: #514
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-08-08 20:00:42 +10:00
unkinben 10854b6501 fix: preload correct rke2 airgap images to break CNI bootstrap deadlock (#512)
## Why

Four newly-provisioned el9_8 compute nodes (prodnxsr0014/0015/0018/0019) hang with canal's kube-flannel container in `ImagePullBackOff`: the flannel VXLAN overlay never comes up, so the node can't reach any in-cluster `198.18.200.x` LoadBalancer VIP.

Root cause is a stale airgap-preload version. The nodes boot rke2 **v1.33.11+rke2r1** from the rolling `latest/1.33` repo, whose canal manifest requests `hardened-flannel:v0.28.4` / `hardened-calico:v3.31.5` (build20260415). But `rke2::install` pinned the preloaded bundle to **v1.33.4+rke2r1** (flannel v0.27.2 / calico v3.30.2), so those images were never on disk. containerd then falls back to the `docker.io` mirror (artifactapi, `disable-default-registry-endpoint: true`), reachable only via the pod-overlay VIP that requires the very flannel image being pulled — a bootstrap deadlock. Older nodes (0001-0008) are versionlocked at 1.33.4 and still match their original bundle, so they were unaffected.

## Changes

- Bump pinned `rke2_version` `1.33.4` -> `1.33.11` so the versionlock, RPM `ensure`, and preloaded bundle all line up with the canal image tags the running binary requests. The default `rke2-images.linux-amd64.tar.zst` bundle already contains the canal CNI images (it is RKE2's default CNI), so no extra tarball is needed.
- Wire the airgap archive `source` to the `container_archive_source` class parameter (previously declared in the module but never consumed). The module keeps its generic upstream default; the artifactapi override (the pre-CNI-reachable source, same BGP/physical path the rke2 yum repos already use) lives in the k8s role hiera as `rke2::container_archive_source`.

Applies to servers and agents alike (`rke2::install` runs for both) and preloads `before => Service`, so the bundle lands before rke2 starts.

Notes:
- The `latest/1.33` repo is rolling, so the pinned version must be maintained as the repo advances; a follow-up to pin the yum channel to a fixed patch would remove the drift entirely.
- No terraform-artifactapi change is required. (If a canal-only preload were ever wanted, the github generic remote allowlist would need `rancher/rke2/.*/rke2-images-canal.linux-amd64.tar.zst$` added — but the default bundle already carries those images, so it is unnecessary.)

https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT
Reviewed-on: #512
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-08-08 18:18:55 +10:00
unkinben 1077bdcbc1 chore: update ceph gpgkey (#438)
- stop checking ceph gpgkey (fixme)
- use artifactapi for retrieving large rke image bundle

Reviewed-on: #438
2026-01-16 23:51:11 +11:00
unkinben 6e4bc9fbc7 feat: adding rke2 (#394)
- manage rke2 repos
- add rke2 module (init, params, install, config, service)
- split roles::infra::k8s::node -> control/compute roles
- moved common k8s config into k8s.yaml
- add bootstrap_node, manage server and token fields in rke2 config
- manage install of helm
- manage node attributes (from puppet facts)
- manage frr exclusions for service/cluster network

Reviewed-on: #394
2025-09-14 13:27:49 +10:00