fix: preload correct rke2 airgap images to break CNI bootstrap deadlock (#512)
## Why Four newly-provisioned el9_8 compute nodes (prodnxsr0014/0015/0018/0019) hang with canal's kube-flannel container in `ImagePullBackOff`: the flannel VXLAN overlay never comes up, so the node can't reach any in-cluster `198.18.200.x` LoadBalancer VIP. Root cause is a stale airgap-preload version. The nodes boot rke2 **v1.33.11+rke2r1** from the rolling `latest/1.33` repo, whose canal manifest requests `hardened-flannel:v0.28.4` / `hardened-calico:v3.31.5` (build20260415). But `rke2::install` pinned the preloaded bundle to **v1.33.4+rke2r1** (flannel v0.27.2 / calico v3.30.2), so those images were never on disk. containerd then falls back to the `docker.io` mirror (artifactapi, `disable-default-registry-endpoint: true`), reachable only via the pod-overlay VIP that requires the very flannel image being pulled — a bootstrap deadlock. Older nodes (0001-0008) are versionlocked at 1.33.4 and still match their original bundle, so they were unaffected. ## Changes - Bump pinned `rke2_version` `1.33.4` -> `1.33.11` so the versionlock, RPM `ensure`, and preloaded bundle all line up with the canal image tags the running binary requests. The default `rke2-images.linux-amd64.tar.zst` bundle already contains the canal CNI images (it is RKE2's default CNI), so no extra tarball is needed. - Wire the airgap archive `source` to the `container_archive_source` class parameter (previously declared in the module but never consumed). The module keeps its generic upstream default; the artifactapi override (the pre-CNI-reachable source, same BGP/physical path the rke2 yum repos already use) lives in the k8s role hiera as `rke2::container_archive_source`. Applies to servers and agents alike (`rke2::install` runs for both) and preloads `before => Service`, so the bundle lands before rke2 starts. Notes: - The `latest/1.33` repo is rolling, so the pinned version must be maintained as the repo advances; a follow-up to pin the yum channel to a fixed patch would remove the drift entirely. - No terraform-artifactapi change is required. (If a canal-only preload were ever wanted, the github generic remote allowlist would need `rancher/rke2/.*/rke2-images-canal.linux-amd64.tar.zst$` added — but the default bundle already carries those images, so it is unnecessary.) https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT Reviewed-on: #512 Co-authored-by: Ben Vincent <ben@unkin.net> Co-committed-by: Ben Vincent <ben@unkin.net>
This commit was merged in pull request #512.
This commit is contained in:
@@ -11,6 +11,8 @@ hiera_include:
|
|||||||
# manage rke2
|
# manage rke2
|
||||||
rke2::bootstrap_node: prodnxsr0001.main.unkin.net
|
rke2::bootstrap_node: prodnxsr0001.main.unkin.net
|
||||||
rke2::join_url: https://join-k8s.service.consul:9345
|
rke2::join_url: https://join-k8s.service.consul:9345
|
||||||
|
# pull the airgap image bundle via artifactapi (host-reachable pre-CNI), not github directly
|
||||||
|
rke2::container_archive_source: 'https://artifactapi.k8s.syd1.au.unkin.net/api/v1/remote/github/rancher/rke2/releases/download'
|
||||||
rke2::manage_registries: true
|
rke2::manage_registries: true
|
||||||
rke2::registries:
|
rke2::registries:
|
||||||
docker.io:
|
docker.io:
|
||||||
|
|||||||
@@ -27,10 +27,10 @@ class rke2::install (
|
|||||||
before => Service["rke2-${node_type}"],
|
before => Service["rke2-${node_type}"],
|
||||||
}
|
}
|
||||||
|
|
||||||
# download required archive of containers
|
# preload the airgap bundle (has the default canal CNI images) so canal starts from disk, not the mirror VIP that needs flannel first
|
||||||
archive { '/var/lib/rancher/rke2/agent/images/rke2-images.linux-amd64.tar.zst':
|
archive { '/var/lib/rancher/rke2/agent/images/rke2-images.linux-amd64.tar.zst':
|
||||||
ensure => present,
|
ensure => present,
|
||||||
source => "https://artifactapi.k8s.syd1.au.unkin.net/api/v1/remote/github/rancher/rke2/releases/download/v${rke2_version}%2B${rke2_release}/rke2-images.linux-amd64.tar.zst",
|
source => "${container_archive_source}/v${rke2_version}%2B${rke2_release}/rke2-images.linux-amd64.tar.zst",
|
||||||
require => [
|
require => [
|
||||||
Package["rke2-${node_type}"],
|
Package["rke2-${node_type}"],
|
||||||
File['/var/lib/rancher/rke2/agent/images'],
|
File['/var/lib/rancher/rke2/agent/images'],
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
# rke2 params
|
# rke2 params
|
||||||
class rke2::params (
|
class rke2::params (
|
||||||
Enum['server', 'agent'] $node_type = 'agent',
|
Enum['server', 'agent'] $node_type = 'agent',
|
||||||
String $rke2_version = '1.33.4',
|
String $rke2_version = '1.33.11',
|
||||||
String $rke2_release = 'rke2r1',
|
String $rke2_release = 'rke2r1',
|
||||||
Stdlib::Absolutepath $config_file = '/etc/rancher/rke2/config.yaml',
|
Stdlib::Absolutepath $config_file = '/etc/rancher/rke2/config.yaml',
|
||||||
Hash $config_hash = {},
|
Hash $config_hash = {},
|
||||||
|
|||||||
Reference in New Issue
Block a user