## Why
The new `openbao-plugin-secrets-arrstack` OpenBao secrets engine mints dynamic per-service arrproxy machine tokens for the arrstack (Sonarr/Radarr/Prowlarr) fleet. Its v0.1.0 RPM is now published in artifactapi (rpm-internal), so the OpenBao VM nodes can install the plugin binary into `/opt/openbao-plugins`.
## Changes
- Add `openbao-plugin-secrets-arrstack` pinned to `0.1.0` to `profiles::packages::include` in `hieradata/roles/infra/storage/vault.yaml`, matching the existing pinned sibling plugins (litellm, gpg, rancher, gitea, apptoken, netbox, ghp).
This PR only installs the RPM binary on the nodes. Catalog registration is handled separately by terraform-vault (`config/plugins/*.yaml`).
## Reference
Plugin binary sha256 (used by the terraform-vault catalog registration, separate PR):
```
f8ee60ca7ba14819976acb7dc4cfb6799e3e8da8f871d0bb2bd18d1d9e537972
```
Reviewed-on: #521
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
Why: step 1 of adding the ghp Vault secrets engine — the plugin binary must be on the OpenBao nodes before terraform-vault registers it in the catalog (the catalog sha256 pins this exact build).
How: pin `openbao-plugin-secrets-ghp` ensure 0.1.0 in hieradata/roles/infra/storage/vault.yaml profiles::packages::include, alongside the other secrets plugins.
Ordering: this merges + puppet-applies first; then the config-write policy; then the terraform-vault engine resources (terraform-vault#121).
Reviewed-on: #520
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
## Why
terraform-vault #115 registers `vault-plugin-secrets-netbox` in the OpenBao plugin catalog pinned to the released v0.1.0 binary's sha256. OpenBao only registers/mounts a plugin whose binary is physically present in the plugin_directory (`/opt/openbao-plugins`); the `openbao-plugin-secrets-netbox` RPM is published on rpm-internal but not yet installed on the bao nodes, so #115 fails to register until Puppet lays the binary down. The `openbao-plugin-secrets-apptoken` binary is likewise published but missing from the bao package list, deferred from the apptoken engine's earlier deploy follow-up.
## Changes
- Add `openbao-plugin-secrets-netbox` (pinned `0.1.0`) to the bao node `profiles::packages::include` list
- Add `openbao-plugin-secrets-apptoken` (pinned `0.1.0`) to the same list, alongside the existing gpg/litellm/rancher/gitea plugin RPMs
- Both install to `/opt/openbao-plugins`, matching the plugin_directory OpenBao reads and the sha256 registered in terraform-vault
## Ordering
This must apply (Puppet converge on the `roles::infra::storage::vault` nodes) BEFORE terraform-vault #115 is applied, otherwise #115's catalog registration fails on a missing binary.
Reviewed-on: #519
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Why
`/etc/ceph/ceph.conf` on the prodnxsr ceph cluster (fsid `de96a98f`) is currently unmanaged and hand-maintained via the runbook (`profiles::ceph::client::manage_ceph_conf` is `false` on the k8s and incus roles). This brings it under Puppet, drives it from a single source of truth in hiera, and normalizes `public_network` to every ceph host's `/32`.
## How hosts are identified (osd-only vs mon/mgr/mds)
The authoritative, declarative signal is the role's `hiera_include` list (ENC `role` -> role hiera):
- **osd-only hosts** = `roles::infra::k8s::*` (prodnxsr0001-0008 and prodnxsr0014-0019): include `profiles::ceph::osd` + `::client`, NOT `::mds` -> `[global]` only.
- **mon/mgr/mds hosts** = `roles::infra::incus::node` (prodnxsr0009-0013): also include `profiles::ceph::mds` -> `[global]` + `[mds]` + `[mds.*]`.
The template keys on `profiles::ceph::client::render_mds_config`, set `true` only in the incus node role hiera (the single role that includes `profiles::ceph::mds`). The `is_ceph_mds` fact is a runtime fallback, not used here.
## Changes
- Add cluster topology as a single source of truth in `hieradata/common.yaml`: `cluster_public_ips` (all 19 ceph host `/32`s, 198.18.23.1-19), `mon_initial_members` (the 5 mons), and the `mds_instances` map (two mds daemons per mon/mgr/mds host).
- Rework `profiles/ceph/client.conf.erb` to render `[global]` always and the `[mds]`/`[mds.*]` sections only when `render_mds_config` is true, preserving the live two-space indent, exact key order, and blank-line separators between mds sections.
- Extend `profiles::ceph::client` with topology + tunable params (sample values as defaults) and gate `[mds]` on `render_mds_config`.
- Drop the hard `Package[ceph-common]` dependency when the class does not manage the package (cephadm / `profiles::packages` deliver it on the k8s and incus hosts; k8s hosts do not declare `ceph-common`, so an unconditional require would fail to compile).
- Enable `manage_ceph_conf: true` on `roles/infra/k8s.yaml`, `roles/infra/k8s/node.yaml`, and `roles/infra/incus/node.yaml`; add `render_mds_config: true` to the incus node role. prodnxsr0014-0019 run `roles::infra::k8s::compute` (confirmed live via PuppetDB; `is_ceph_osd` true, `is_ceph_mds` false), so they are already covered by `k8s.yaml` and get the `[global]`-only variant with no mds sections.
## Live-cluster safety review (please read before merge)
This touches a running storage cluster's config. The rendered content is byte-equivalent to the live file **except** for the one deliberate `public_network` normalization below.
- **Intended change:** `public_network` is normalized to all 19 ceph host `/32`s on every host (198.18.23.1-19). Adoption is now a **single `public_network`-line change on every cluster host**, no longer a no-op on the osd hosts:
- osd hosts prodnxsr0001-0008 currently list `.1-.13` live -> first convergence adds `.14-.19`.
- mon/mgr/mds hosts prodnxsr0009-0013 currently list only `.9-.13` live -> first convergence adds `.1-.8` and `.14-.19`.
- prodnxsr0014-0019 (newly joined OSD hosts) get the file created with the full `[global]` variant.
- **Verified byte diff against the live files** (`/etc/ceph/ceph.conf` pulled from prodnxsr0008 and prodnxsr0009): each differs by **exactly one line** (the expanded `public_network`); on prodnxsr0009 the `[mds]` + `[mds.*]` sections and their blank-line separators are byte-identical to live. Rendering prodnxsr0009 with its current live `.9-.13` value reproduces the live file byte-for-byte, confirming the format match.
- RGW hosts (`roles::infra::ceph::rgw`, ausyd1nxvm2115-2119) include only `profiles::ceph::rgw` -> `profiles::ceph::conf`; they do not include `profiles::ceph::client`, so their variant is untouched and there is no duplicate `File[/etc/ceph/ceph.conf]`.
Rendered samples:
- osd-only (prodnxsr0008): `[global]` with all 19 `/32`s in `public_network`, nothing else.
- mon/mgr/mds (prodnxsr0009): the same `[global]` plus `[mds]` and `[mds.<host>-1|2]` for prodnxsr0009-0013, blank-line separated.
Reviewed-on: #515
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Why
LLDP is the only source of physical switch/port topology in the estate: which switch and which port each machine is cabled to exists nowhere else (not in intent YAML, not in any other fact, not in DHCP/DNS). NetBox needs it to record device interface -> switch/port connections. This installs and enables lldpd on physical hosts and exposes its neighbour data as a structured `lldp` fact for the terraform-infra pdbmux backfill to consume (linked issue below).
## Changes
- Add `profiles::lldpd`: install the `lldpd` package and enable+start the service (it ships disabled), the service subscribing to the package.
- Assign it physicals-only via `hiera_include` in `hieradata/virtual/physical.yaml` (`facts.virtual == 'physical'`), merged `unique` with the common `hiera_include`. VMs never receive it.
- Add the `lldp` custom fact (`modules/libs/lib/facter/lldp.rb`): parse `lldpctl -f json0` into a per-interface map of `{neighbor_chassis_name, neighbor_chassis_mac, neighbor_chassis_descr, neighbor_port_id, neighbor_port_descr, vlan_id, vlan_name}`, skipping interfaces with no neighbour. Confined to physical Linux hosts that have `lldpctl` and a live lldpd socket; returns an empty hash on any error so it can never break a puppet run.
json0 is used because it array-wraps every node regardless of cardinality, so one neighbour and many neighbours parse identically (plain `keyvalue` folds the neighbour SysName into the key path; plain `json` collapses single-element arrays into objects).
## Sample fact output (prodnxsr0019, enp1s0)
```json
{
"enp1s0": {
"neighbor_chassis_name": "sg3429x-m2-02",
"neighbor_chassis_mac": "b8:fb:b3:a7:f9:5d",
"neighbor_chassis_descr": "48-Port Gigabit Smart Managed Pro Switch",
"neighbor_port_id": "24",
"neighbor_port_descr": "two-gigabitEthernet 1/0/11",
"vlan_id": "201",
"vlan_name": "fabric_common_02"
}
}
```
## Consumer
NetBox population of the switch/port cabling is tracked in terraform-infra: unkin/terraform-infra#7 (extend the pdbmux backfill / PR #6 to read this fact and emit the connection).
https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT
Reviewed-on: #513
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Why
Four newly-provisioned el9_8 compute nodes (prodnxsr0014/0015/0018/0019) hang with canal's kube-flannel container in `ImagePullBackOff`: the flannel VXLAN overlay never comes up, so the node can't reach any in-cluster `198.18.200.x` LoadBalancer VIP.
Root cause is a stale airgap-preload version. The nodes boot rke2 **v1.33.11+rke2r1** from the rolling `latest/1.33` repo, whose canal manifest requests `hardened-flannel:v0.28.4` / `hardened-calico:v3.31.5` (build20260415). But `rke2::install` pinned the preloaded bundle to **v1.33.4+rke2r1** (flannel v0.27.2 / calico v3.30.2), so those images were never on disk. containerd then falls back to the `docker.io` mirror (artifactapi, `disable-default-registry-endpoint: true`), reachable only via the pod-overlay VIP that requires the very flannel image being pulled — a bootstrap deadlock. Older nodes (0001-0008) are versionlocked at 1.33.4 and still match their original bundle, so they were unaffected.
## Changes
- Bump pinned `rke2_version` `1.33.4` -> `1.33.11` so the versionlock, RPM `ensure`, and preloaded bundle all line up with the canal image tags the running binary requests. The default `rke2-images.linux-amd64.tar.zst` bundle already contains the canal CNI images (it is RKE2's default CNI), so no extra tarball is needed.
- Wire the airgap archive `source` to the `container_archive_source` class parameter (previously declared in the module but never consumed). The module keeps its generic upstream default; the artifactapi override (the pre-CNI-reachable source, same BGP/physical path the rke2 yum repos already use) lives in the k8s role hiera as `rke2::container_archive_source`.
Applies to servers and agents alike (`rke2::install` runs for both) and preloads `before => Service`, so the bundle lands before rke2 starts.
Notes:
- The `latest/1.33` repo is rolling, so the pinned version must be maintained as the repo advances; a follow-up to pin the yum channel to a fixed patch would remove the drift entirely.
- No terraform-artifactapi change is required. (If a canal-only preload were ever wanted, the github generic remote allowlist would need `rancher/rke2/.*/rke2-images-canal.linux-amd64.tar.zst$` added — but the default bundle already carries those images, so it is unnecessary.)
https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT
Reviewed-on: #512
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
must change management interfaces for new nodes to start from 21, since
.14 to .20 were used for various anycast services
really need ipam
---------
Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #511
AlmaLinux hosts still pulled in-house RPMs from the legacy `unkin` repo hosted on Gitea (git.query.consul), and a second legacy Gitea repo `unkinben` was enabled fleet-wide. This moves AlmaLinux hosts fully onto the artifactapi local rpm repos so all hosts get the in-house/vendored set for their EL release, matching the artifactapi-is-the-real-registry model already used for baseos/epel/etc., and removes both legacy Gitea yum repos.
Depends on unkin/terraform-artifactapi#15 (creates the general `rpm-vendor` local repo); that should apply before this reaches hosts.
### RPM source migration
- retarget `rpm-vendor` to the distribution-agnostic `local/rpm-vendor` repo
- add `rpm-internal-release`/`rpm-vendor-release`, resolved from `os.release.major` so el8 hosts pull `-el8`, el9 hosts `-el9`, etc.
- remove the legacy `unkin` yumrepo definition (its content now comes from rpm-internal/rpm-vendor)
### unkinben removal
- remove the `unkinben` yumrepo definition from `AlmaLinux/all_releases.yaml`
- remove the `unkinben` `ensure: present` override from the jellyfin media role (its only consumer)
- `purge => true` on the yum profile then cleans `/etc/yum.repos.d/unkin.repo` from hosts
**Heads-up for reviewer:** the jellyfin role installs `jellyfin-web`, `jellyfin-server`, and `jellyfin-ffmpeg-bin` from unkinben. These packages exist **only** in unkinben, so once it is removed they will remain installed but **stop receiving updates** until they are republished into artifactapi (e.g. `rpm-internal-elN`). Jellyfin's other deps (`SDL2`, `ffmpeg`, `ffmpeg-devel`) come from rpmfusion and are unaffected. The `unkinben` *generic* archive downloads used by the *arr modules (sonarr/radarr/prowlarr/lidarr/readarr) and glauth are direct HTTP fetches, not the yum repo, and are untouched.
https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
Reviewed-on: #496
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Why
We want to manage Fedora 44 hosts with this Puppet codebase, starting with the `base` role only. Fedora reuses the RedHat osfamily code paths (dnf/yum, crypto-policies, firewalld, openvox), so this adds the Fedora-specific hieradata, repositories, and gates needed for `base` to compile and apply, mirroring how AlmaLinux is wired and consuming the artifactapi `rpm-internal-f<major>` / `rpm-vendor-f<major>` local repos. Everything is keyed off `facts.os.release.major` so a future Fedora release only needs its artifactapi local repos created, not new hieradata.
## How
- Add `hieradata/os/Fedora/all_releases.yaml`:
- Define dnf repos via `profiles::yum::global::repos`: `fedora` and `updates` proxied through artifactapi's `fedora` remote, plus the artifactapi locals `rpm-internal-f%{major}` and `rpm-vendor-f%{major}`; GPG via the on-disk `fedora-gpg-keys`; metalink/mirrorlist cleared so only the artifactapi baseurl is used.
- Set `crypto_policies::policy: DEFAULT`, `profiles::puppet::agent::openvox_enable: true`, and `lm-sensors:📦 lm_sensors`.
- Fix up the base package set for Fedora: exclude `p7zip`/`dstat`/`iotop` (absent on Fedora) and include `7zip` and `iotop-c`, plus the same `crypto-policies-scripts`/`lzo`/`policycoreutils`/`unar`/`xz` additions AlmaLinux carries.
- Wire `profiles::fedora::base` via `hiera_include`.
- Add `profiles::fedora::base` (ensures NetworkManager enabled) as the Fedora analogue of `profiles::almalinux::base`; deliberately minimal so it can grow into workstation/laptop use later.
- Make `profiles::puppet::agent` select the OpenVox distribution path (`fedora/<major>` on Fedora, `el/<major>` elsewhere); AlmaLinux/Debian behaviour is unchanged.
## Validation
`puppet-lint`, puppet manifest validate, and `yamllint` all pass via the repo's pre-commit hooks on the changed files.
## Note for reviewer
OpenVox does not yet publish a Fedora 44 build (`openvox7/fedora/` currently has only 36/40/41), so `openvox-agent` will 404 until upstream publishes f44 or a build is placed in `rpm-internal-f44`. The Puppet code produces the correct path for when that exists; installing the agent is a prerequisite for a Fedora 44 host to actually run.
https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
Reviewed-on: #497
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Why
The new Gitea token secrets engine (`vault-plugin-secrets-gitea`, released v0.1.0) needs its plugin binary present on the OpenBao servers before terraform-vault can register it in the plugin catalog and mount it. This is the same install step already done for the rancher, gpg, and litellm secrets plugins.
## Change
- Add `openbao-plugin-secrets-gitea` to `profiles::packages::include` on the vault storage role, pinned to `0.1.0` so the on-disk binary stays in lockstep with the sha256 that terraform-vault pins in its plugin catalog entry (`config/plugins/vault-plugin-secrets-gitea.yaml`).
## Order
This is the first step. It must merge (and Puppet must run on the vault nodes to lay down the binary) before the terraform-vault engine PR is applied, or OpenBao cannot launch the plugin.
https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
Reviewed-on: #498
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Why
During the k8s DNS cutover the `dns` module dual-writes every host's records two
ways: (a) LEGACY exported `@@concat::fragment`s collected onto the VM puppet DNS
master into `/var/named/sources/<zone>.conf`, and (b) NEW nsupdate/RFC2136 to the
k8s bind-authoritative cluster (`manage_nsupdate`). Both were gated on so the
legacy path stayed as a safety net.
The legacy path is no longer in any client's resolution path:
- **#490** repointed `profiles::dns::base::nameservers` from the VM resolver
anycast `198.18.19.16` to the k8s **bind-resolvers** LB `198.18.200.7`. Every
puppet client now resolves there.
- The k8s bind-resolvers forward `unkin.net`, `main.unkin.net`, and all
`18.198.in-addr.arpa` reverse zones to the k8s **bind-authoritative** cluster
(`198.18.200.6`), which is populated purely by the nsupdate path — the same
`profiles::dns::record` defines, `manage_nsupdate` branch.
- The VM master's assembled `/var/named/sources` files are therefore orphaned:
nothing resolves against them anymore.
This flips `profiles::dns::updater::manage_export: false` to stop nodes exporting
the legacy fragments. `manage_nsupdate` stays on.
## What changes
- `hieradata/common.yaml`: `profiles::dns::updater::manage_export` `true` -> `false`.
## Verification evidence
Pulled all **498** exported `Concat::Fragment` resources from PuppetDB (239 A,
232 PTR, 27 CNAME; 490 unique fqdn/type/value) and `dig`'d each against the k8s
authoritative read endpoint `198.18.200.7` (the dynamic path's serving side):
| Result | Count |
|---|---|
| MATCH | 447 |
| MISSING-from-dynamic | 38 |
| DIFFERENT-value | 5 |
Both non-match buckets are accounted for and are **not coverage gaps**:
- **38 MISSING** are all records the dynamic path deliberately filters via
`profiles::dns::updater::deny_ranges`
(`198.18.200.0/24`, `10.42.0.0/16`, ...) plus reverse zones the authoritative
cluster isn't even authoritative for: `*-kube-lb0` (198.18.200.x),
`*-flannel.1` (10.42.x.x pod CIDR), and `*-nomad`/`*-brcom1`
(172.26.64.x / 10.255.255.x). These are exactly the k8s/LB noise the deny
ranges exist to keep out of the authoritative zones. Verified every one falls
in a denied/non-authoritative range.
- **5 DIFFERENT** are stale legacy duplicates that the dynamic path holds *more*
correctly:
- 4x `au-syd1-prod-halb.main.unkin.net` A: four ceph nodes (2069-2073) each
self-register the shared name with their own primary IP (haproxy::dns
exports `networking.ip` per host, a legacy collision). Dynamic resolves it to
the single VIP `198.18.24.60`.
- 1x `dashboard.ceph.unkin.net` CNAME: the ceph node self-publishes this to
`lb1.unkin.net.` via its own nsupdate (per the k8s zones.yaml note); the
exported VRRP-name fragment is stale.
Conclusion: the dynamic path fully covers the served record set, and the serving
path no longer depends on the exported files.
## Collector-behavior note (what the flip actually does)
`profiles::dns::zone` realizes the fragments with an exported-resource collector
(`Concat::Fragment <<| target == ... |>>`). Turning off export purges the
fragments from PuppetDB, so on the VM master concat rebuilds each
`/var/named/sources/<zone>.conf` down to just the header (SOA+NS+glue) as nodes
converge. That is **cosmetic**: no client resolves against the VM master anymore
(#490), and the k8s authoritative zones are unaffected (fed by nsupdate, not
these files).
## Rollback
Set `profiles::dns::updater::manage_export: true` again and let puppet converge;
nodes re-export their fragments and the VM-master zone files repopulate.
## Follow-up (after soak)
Deleting the `@@concat::fragment` / `Concat::Fragment <<||>>` collector code in
`profiles::dns::record` / `profiles::dns::zone` (and the VM `profiles::dns::master`
zone assembly) is a separate cleanup PR once this has soaked and the VM master is
confirmed decommissionable.
Reviewed-on: #495
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
First live agent migration to the k8s puppetserver, using the single-boolean mechanism from #493 on the lowest-risk host (roles::base test host).
- Adds hieradata/nodes/ausyd1nxvm2120.main.unkin.net.yaml with profiles::puppet::migrate::enabled: true
On merge: the node's next run (from the old master) rewrites puppet.conf to the k8s server/ca_server with a fresh ssl-k8s ssldir; the run after enrols against the k8s CA (autosigned) and compiles from the k8s compilers (ENC=encapi). Verify: node appears in the k8s PuppetDB with a fresh report. Rollback: revert this file — legacy ssldir/cert remain intact.
Reviewed-on: #494
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Why
`ensure: present`/`latest` lets the plugin binaries drift from the sha256 pinned in the terraform-vault catalog (`config/plugins/*.yaml`). On the next OpenBao restart, a drifted binary fails the sha check and the plugin won't launch — a latent footgun (hit exactly this with rancher on `ensure: latest`).
## Changes
Pin each secrets plugin to the version whose binary matches its registered catalog sha (all verified against the RPMs in rpm-internal):
- `openbao-plugin-secrets-litellm`: **0.1.1** (sha 2263ebcb…)
- `openbao-plugin-secrets-gpg`: **0.1.0** (sha 0e92d740…)
- `openbao-plugin-secrets-rancher`: **0.1.1** (sha 9e597cd9…; was `ensure: latest`)
All three are no-op on the binary (installed versions already match) — this just locks them so a future release can't silently upgrade the binary out of lockstep with the catalog.
`openbao-plugins` (base bundle) left unpinned — its version couldn't be verified from the tooling side and it tracks the openbao package, not a catalog sha.
## Note
To upgrade a plugin in future: bump the RPM version here **and** the catalog sha256 in terraform-vault in the same change, then `vault write sys/plugins/reload/backend plugin=<name>`.
---------
Co-authored-by: Ben Vincent <neotheo@gmail.com>
Reviewed-on: #492
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
- change the default anycast nameserver to bind-internal resolvers in
kubernetes
---------
Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #490
## Why
`s3.ceph.unkin.net` is the RGW S3 endpoint name used by cephrgw-operator consumers. A companion argocd-apps PR adds it as a CNAME to `radosgw.service.consul`. The radosgw hosts front RGW with nginx (443 -> radosgw:7480) using the vault-issued cert, so **both** the certificate and nginx must recognise the name.
## Changes (hieradata/roles/infra/ceph/rgw.yaml)
- Add `s3.ceph.unkin.net` to `profiles::pki::vault::alt_names` — puts the name in the radosgw cert SAN (renewal re-triggers on the alt-names change).
- Add `s3.ceph.unkin.net` to `profiles::nginx::simpleproxy::nginx_aliases` — nginx answers for that server_name.
### nginx: yes, needed
radosgw sits behind `profiles::nginx::simpleproxy` (listens 443, proxies to :7480) sharing the same vault cert, so the alias is required for nginx to serve the hostname — not just the cert SAN.
---------
Co-authored-by: benvin <neotheo@gmail.com>
Reviewed-on: #487
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Why
The vault nodes have rancher plugin v0.1.0, which mints non-functional tokens (returns `status.value` not `status.bearerToken`; every cred/rotation 401s). v0.1.1 fixes it. `ensure_packages` defaults to `present`, which won't upgrade an already-installed package, so pin to `latest` to pull v0.1.1.
## Changes
- `openbao-plugin-secrets-rancher: { ensure: latest }` on the vault storage role.
## Note
`latest` will also pull future releases automatically — keep the terraform-vault catalog sha256 (`config/plugins/vault-plugin-secrets-rancher.yaml`) bumped in lockstep, or a vault restart after a future upgrade will fail the sha check. Happy to pin to an exact version instead if you'd prefer lockstep-by-pin.
---------
Co-authored-by: Ben Vincent <neotheo@gmail.com>
Reviewed-on: #488
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Repoints the `rpmfusion-free` and `rpmfusion-nonfree` yum repos from the retired `packagerepo.service.consul` mirror to the artifactapi RPM Fusion remotes.
dnf on media nodes (jellyfin, nzbget) failed with `Cannot download repomd.xml: All mirrors were tried`, which blocked package installs such as `dns-updater`.
Changes:
- `rpmfusion-free` baseurl → `.../api/v1/remote/rpmfusion-free/el/updates/$major/$arch/`, gpgkey → `.../rpmfusion-free/el/RPM-GPG-KEY-rpmfusion-free-el-$major`
- `rpmfusion-nonfree` baseurl → `.../api/v1/remote/rpmfusion-nonfree/el/updates/$major/$arch/`, gpgkey → `.../rpmfusion-nonfree/el/RPM-GPG-KEY-rpmfusion-nonfree-el-$major`
Depends on terraform-artifactapi PR #7 (adds the rpmfusion remotes) being applied first.
https://claude.ai/code/session_01VZ81u2JtjvaPq8Z3EDTSMb
Reviewed-on: #486
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Why: rpmbuilder's vendored third-party RPMs are moving from Gitea's RPM registry to the artifactapi rpm-vendor local repos. Managed AlmaLinux hosts need the matching yum repo to install them; this mirrors the existing rpm-internal repo and selects the per-EL-release repo via os.release.major.
Changes:
- Add rpm-vendor to profiles::yum::global::repos in hieradata/os/AlmaLinux/all_releases.yaml, pointing at .../api/v1/local/rpm-vendor-el<major>/ (el8/el9), gpgcheck disabled and mirrorlist absent, matching the rpm-internal repo definition.
Depends on terraform-artifactapi PR #6 (creates the rpm-vendor-el8/el9 repos).
Reviewed-on: #484
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Why
The Rancher token secrets engine needs its plugin binary present in the OpenBao plugin directory before terraform-vault can register/mount it. Mirrors #480 (gpg).
## Changes
- Add `openbao-plugin-secrets-rancher` to `profiles::packages::include` on the vault storage role. Installs the RPM (from rpm-internal) to `/opt/openbao-plugins/vault-plugin-secrets-rancher`.
## Merge order
Part 3 of 4 — merge first (binary must exist on disk before the terraform-vault plugin-import #92 registers it with a matching sha256).
---------
Co-authored-by: Ben Vincent <neotheo@gmail.com>
Reviewed-on: #483
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Deploy the GPG/OpenPGP secrets engine to the OpenBao (vault-role) cluster by installing its plugin RPM into `/opt/openbao-plugins`, mirroring the existing `openbao-plugin-secrets-litellm` deployment (#479).
- Add `openbao-plugin-secrets-gpg` to `profiles::packages::include` in the vault role hiera. The RPM ships from artifactapi `rpm-internal` (built on the [vault-plugin-secrets-gpg](https://git.unkin.net/unkin/vault-plugin-secrets-gpg) v0.1.0 tag) and lands the plugin binary in the node's configured `plugin_directory`.
Registering + enabling the secrets backend (`plugin register` / `secrets enable`) is a follow-up terraform-vault change, matching how litellm is wired.
Reviewed-on: #480
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Replaces the exported-resources → puppet DNS master zone-file flow with per-host RFC2136 dynamic updates against the k8s **bind-authoritative** write endpoint (198.18.200.9). The master no longer manages zone files.
## Design
Each node assembles its DNS records into a local concat file; a systemd `.path` unit watches it and runs `dns-update` (nsupdate) on change — exactly the watch-a-file model requested.
## Changes
- **profiles::dns::updater** (new): concat records file + TSIG key file + `dns-update` script + `dns-update.service` (oneshot) + `dns-update.path` (watcher). The script sends only the delta since last run and deletes removed records, grouped per zone.
- **profiles::dns::record**: writes a local concat fragment (`zone|name|type|ttl|value`) instead of exporting `@@concat::fragment` to the master.
- **profiles::dns::base**: includes `profiles::dns::updater` (all nodes).
- **hiera**: `profiles::dns::updater` server/key_name/algorithm in common.yaml.
## Inert until keyed
The updater does nothing until `profiles::dns::updater::key_secret` (TSIG) is set in eyaml — records are assembled but not applied, so nodes are safe before the key exists.
## Prerequisites (k8s side, separate)
1. The `bind-authoritative` zones must set `dynamicUpdate: true` + an `updateKeyRef` (a client-update BindTSIGKey) so they accept these updates.
2. The TSIG key must be shared: the operator-generated key value goes into eyaml here (or the planned Vault-sync feature bridges it).
## Validated
puppet parser/epp validate, puppet-lint, and a functional test of the generated per-zone nsupdate message (replace + delete-removed).
Reviewed-on: #475
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
enable installing locally packaged rpms with no specific linux release
- add the rpm-internal repo from artifactapi
---------
Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #476
## Summary
- Replace static `registries.yaml` with EPP template driven by `rke2::registries` hash
- Add `disable-default-registry-endpoint: true` to all mirrors — RKE2 will only use artifactapi and never fall back to upstream registries
- Registry configuration now fully managed via hiera data (`roles/infra/k8s.yaml`)
Reviewed-on: #474
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Pin grafana package version to 13.0.2 via a new version parameter on
profiles::metrics::grafana, wired through to the puppet-grafana class.
---------
Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #470
- update release to install to 1.26.2
- change base_url to artifactapi
- update releases/checksums
---------
Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #469
## Summary
- Adds `Unkin::Ceph::Utils` facter module detecting ceph service instances via `systemctl list-units`, exposing `is_ceph_mon`, `is_ceph_mgr`, `is_ceph_mds`, `is_ceph_osd` booleans and a `ceph_services` hash of unit names
- Adds `profiles::ceph::mon`, `mgr`, `mds`, `osd` — each with `Boolean $ensure_running` that iterates discovered service instances and manages them as running and enabled
- Works across incus nodes (mon/mgr/mds/osd) and k8s compute/control nodes (osd only); verified on prodnxsr0001 which correctly reports `is_ceph_osd: true` and `ceph_services: {osd: [ceph-osd@5]}`
## Test plan
- [x] Noop deploy against prodnxsr0001.main.unkin.net passed cleanly
- [x] `ceph_services` fact returns correct service map
- [x] `is_ceph_osd` returns `True`, `is_ceph_mon` returns `False` as expected
- [x] Test on an incus/ceph node with mon/mgr/mds services
Reviewed-on: #459
rebuilding router, taking the chance to not mess up ip ranges. I did
have 198.18.21.0/24 and 198.18.21.160/27 and 198.18.21.192/27 all on
differnt interfaces.
- update IP's that can reach bind view for main.unkin.net
- keep both for intermediate period
Reviewed-on: #460
need to separate the permissions inside vault into different groups, one
per-permission.
- add group for each kubernetes role in vault
Reviewed-on: #449
unkin-agent
unkinben
benvin