Commit Graph

31 Commits

Author SHA1 Message Date
unkin-agent 734fcb8cf4 Trust the estate CA when fetching ENC facts (#529)
Facter runs under Puppet's vendored Ruby, which reads its own bundled CA file and never the system trust store. The ENC fact now fetches over HTTPS, so every node fails certificate verification and falls back to its cached value.

- set ca_file on the request to the vaultca anchor bundle
- keep VERIFY_PEER on, and fall through to the existing cache path when the anchor is absent

Reviewed-on: #529
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-09-20 23:32:16 +10:00
unkin-agent f933660d3b Fetch agent ENC facts from encapi (#528)
The enc_role and enc_env facts still resolve against Cobbler on every agent, the last Cobbler dependency in the classification path now that the master-side ENC runs encapic-enc.

- Point the fact at https://encapi.k8s.syd1.au.unkin.net
- Rename the module and its messages from Cobbler to encapi

Cache file, TTL and fallback-to-cache failure behaviour are unchanged.

Reviewed-on: #528
Co-authored-by: unkin-agent <unkin-agent@unkin.net>
Co-committed-by: unkin-agent <unkin-agent@unkin.net>
2026-09-20 23:08:20 +10:00
unkinben 499251575d lldpd: run on physicals and expose neighbour topology as the lldp fact (#513)
## Why

LLDP is the only source of physical switch/port topology in the estate: which switch and which port each machine is cabled to exists nowhere else (not in intent YAML, not in any other fact, not in DHCP/DNS). NetBox needs it to record device interface -> switch/port connections. This installs and enables lldpd on physical hosts and exposes its neighbour data as a structured `lldp` fact for the terraform-infra pdbmux backfill to consume (linked issue below).

## Changes

- Add `profiles::lldpd`: install the `lldpd` package and enable+start the service (it ships disabled), the service subscribing to the package.
- Assign it physicals-only via `hiera_include` in `hieradata/virtual/physical.yaml` (`facts.virtual == 'physical'`), merged `unique` with the common `hiera_include`. VMs never receive it.
- Add the `lldp` custom fact (`modules/libs/lib/facter/lldp.rb`): parse `lldpctl -f json0` into a per-interface map of `{neighbor_chassis_name, neighbor_chassis_mac, neighbor_chassis_descr, neighbor_port_id, neighbor_port_descr, vlan_id, vlan_name}`, skipping interfaces with no neighbour. Confined to physical Linux hosts that have `lldpctl` and a live lldpd socket; returns an empty hash on any error so it can never break a puppet run.

json0 is used because it array-wraps every node regardless of cardinality, so one neighbour and many neighbours parse identically (plain `keyvalue` folds the neighbour SysName into the key path; plain `json` collapses single-element arrays into objects).

## Sample fact output (prodnxsr0019, enp1s0)

```json
{
  "enp1s0": {
    "neighbor_chassis_name":  "sg3429x-m2-02",
    "neighbor_chassis_mac":   "b8:fb:b3:a7:f9:5d",
    "neighbor_chassis_descr": "48-Port Gigabit Smart Managed Pro Switch",
    "neighbor_port_id":       "24",
    "neighbor_port_descr":    "two-gigabitEthernet 1/0/11",
    "vlan_id":                "201",
    "vlan_name":              "fabric_common_02"
  }
}
```

## Consumer

NetBox population of the switch/port cabling is tracked in terraform-infra: unkin/terraform-infra#7 (extend the pdbmux backfill / PR #6 to read this fact and emit the connection).

https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT
Reviewed-on: #513
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-08-08 18:41:50 +10:00
unkinben d5cfaed7d4 libs: add ceph_osd_devices fact listing ceph OSD block devices (#504)
A forthcoming profile needs to set the I/O scheduler to noop/none on the disks backing ceph OSDs, so Puppet must know which block devices those are. This fact surfaces them by inspecting LVM PVs whose volume group belongs to ceph.

- add ceph_osd_devices fact returning the sorted array of PV device paths whose VG name starts with ceph-
- confine the fact to physical Linux hosts with pvs installed so it never resolves on VMs

https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT
Reviewed-on: #504
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-08-02 17:25:18 +10:00
unkinben 88fcb97ad1 dns: nsupdate host records to the authoritative server (#475)
Replaces the exported-resources → puppet DNS master zone-file flow with per-host RFC2136 dynamic updates against the k8s **bind-authoritative** write endpoint (198.18.200.9). The master no longer manages zone files.

## Design
Each node assembles its DNS records into a local concat file; a systemd `.path` unit watches it and runs `dns-update` (nsupdate) on change — exactly the watch-a-file model requested.

## Changes
- **profiles::dns::updater** (new): concat records file + TSIG key file + `dns-update` script + `dns-update.service` (oneshot) + `dns-update.path` (watcher). The script sends only the delta since last run and deletes removed records, grouped per zone.
- **profiles::dns::record**: writes a local concat fragment (`zone|name|type|ttl|value`) instead of exporting `@@concat::fragment` to the master.
- **profiles::dns::base**: includes `profiles::dns::updater` (all nodes).
- **hiera**: `profiles::dns::updater` server/key_name/algorithm in common.yaml.

## Inert until keyed
The updater does nothing until `profiles::dns::updater::key_secret` (TSIG) is set in eyaml — records are assembled but not applied, so nodes are safe before the key exists.

## Prerequisites (k8s side, separate)
1. The `bind-authoritative` zones must set `dynamicUpdate: true` + an `updateKeyRef` (a client-update BindTSIGKey) so they accept these updates.
2. The TSIG key must be shared: the operator-generated key value goes into eyaml here (or the planned Vault-sync feature bridges it).

## Validated
puppet parser/epp validate, puppet-lint, and a functional test of the generated per-zone nsupdate message (replace + delete-removed).

Reviewed-on: #475
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
2026-07-12 22:23:18 +10:00
unkinben 0451894b48 feat: add ceph service management profiles and facts (#459)
## Summary

- Adds `Unkin::Ceph::Utils` facter module detecting ceph service instances via `systemctl list-units`, exposing `is_ceph_mon`, `is_ceph_mgr`, `is_ceph_mds`, `is_ceph_osd` booleans and a `ceph_services` hash of unit names
- Adds `profiles::ceph::mon`, `mgr`, `mds`, `osd` — each with `Boolean $ensure_running` that iterates discovered service instances and manages them as running and enabled
- Works across incus nodes (mon/mgr/mds/osd) and k8s compute/control nodes (osd only); verified on prodnxsr0001 which correctly reports `is_ceph_osd: true` and `ceph_services: {osd: [ceph-osd@5]}`

## Test plan

- [x] Noop deploy against prodnxsr0001.main.unkin.net passed cleanly
- [x] `ceph_services` fact returns correct service map
- [x] `is_ceph_osd` returns `True`, `is_ceph_mon` returns `False` as expected
- [x] Test on an incus/ceph node with mon/mgr/mds services

Reviewed-on: #459
2026-04-07 19:02:17 +10:00
unkinben d8b354558d feat: add incus auto-client certificate trust (#406)
- add fact to export vault public cert from agents
- add fact to export list of trusted incus client certs
- add method for incus clients to export their client cert to be trusted

Reviewed-on: #406
2025-10-17 22:46:26 +11:00
unkinben 0e64c9855a feat: add vmcluster module (#363)
- manage vmstorage package, service and environment file
- manage vmselect package, service and environment file
- manage vminsert package, service and environment file
- manage vmagent package, service and environment file
- manage options for vmstorage, vmselect, vminsert, vmagent role

Reviewed-on: #363
2025-07-26 18:17:20 +10:00
unkinben 40c57ede59 feat: add ci build task (#342)
- a ci workflow for build tests
- run pre-commit against all files

Reviewed-on: #342
2025-07-08 20:19:36 +10:00
unkinben a9faa098ee benvin/grafana_postgres (#334)
Reviewed-on: https://git.query.consul/unkinben/puppet-prod/pulls/334
2025-07-01 19:07:24 +10:00
unkinben 62837bb22d feat: add zone to subnet facts (#320)
- add common and dmz zone fact information

Reviewed-on: https://git.query.consul/unkinben/puppet-prod/pulls/320
2025-06-21 15:42:37 +10:00
unkinben 2321186ad5 neoloc/mpls_ldp_frr (#255)
Reviewed-on: https://git.query.consul/unkinben/puppet-prod/pulls/255
2025-04-24 16:51:31 +10:00
unkinben c225564bdb feat: continue incus implementation (#245)
- migrate to systemd-networkd
- setup dummy, bridge and static/ethernet interfaces
- manage sshd.service droping to start ssh after networking is online
- enable ip forewarding
- add fastpool/data/incus dataset
- enable ospf and frr
- add loopback0 as ssh listenaddress
- add loopback1/2 for ceph cluster/public traffic

Reviewed-on: https://git.query.consul/unkinben/puppet-prod/pulls/245
2025-04-06 16:38:04 +10:00
unkinben 8eb751e22f feat: change enc_* fact to read direct from cobbler (#219)
- change enc_role and enc_env to read direct from cobbler
- cleanup profiles::base::facts

Reviewed-on: https://git.query.consul/unkinben/puppet-prod/pulls/219
2025-03-12 23:09:15 +11:00
unkinben 265400db91 feat: confine fact to patroni 2024-09-03 22:18:53 +10:00
unkinben afda425fab feat: psql changes on master only
- add fact to detect if a psql host is a slave
- only import users/db/grants on master
2024-09-03 22:13:50 +10:00
unkinben f3046f8fbb feat: manage jellyfin data migration_flag 2024-07-03 22:49:54 +10:00
unkinben 94aed2df9c feat: add pveceph consul services
- refacter the pveceph facts
- define consul services for osd, mgr, mds and mons
2024-06-18 21:14:57 +10:00
unkinben b468f67103 feat: sign ssh host keys
- manage python script/venv to sign ssh host certificates
- add approle_id to puppetmaster eyaml files
- add class to sign ssh-rsa host keys
- add facts to check if the current principals match the desired principals
2024-06-01 22:51:42 +10:00
unkinben dde8d5978d feat: firstrun improvements
- add fact to detect firstrun
- run a limited subset of classes on firstrun
- firstrun: includes:
  - vault ca certificates
  - yum/apt repositories
  - fast-install packages with an exec
2024-05-19 21:28:14 +10:00
unkinben 6f9a606549 feat: configure edgecache for postgresql
- add fact to record system resolvers
- add resolvers feature in /etc/nginx/conf.d/resolvers.conf
- add rewrite rules for postgres/yum/repodata
2024-05-19 16:56:36 +10:00
unkinben fee0bde604 feat: complete cobbler automation
- add facts to manage the /var/www/cobbler and /data/cobbler directories
- move /var/www/cobbler -> /data/cobbler
- create symlink from /var/www/cobbler -> /data/cobbler
- ensure that cobbler nodes are set to permissive selinux mode
2024-05-09 22:44:55 +10:00
unkinben 3ed433fb97 Merge pull request 'feat: moved enc_role and enc_env to ruby facts' (#171) from neoloc/enc_role_facts into develop
Reviewed-on: unkinben/puppet-prod#171
2024-04-24 23:01:57 +09:30
unkinben e0dbecbfa0 feat: moved enc_role and enc_env to ruby facts 2024-04-24 23:30:27 +10:00
unkinben bc4246dd05 feat: add new syd1 prod networks 2024-04-21 22:55:06 +10:00
unkinben f04c74bd4d feat: manage proxmox nodes
- change /etc/hosts to meet proxmox requirements
- add proxmox node role
- add init, params, repo, install, clusterjoin classes
2024-04-21 15:08:28 +10:00
unkinben f2cdcb8c8e feat: add sydney subnets 2024-03-21 22:02:25 +11:00
unkinben df05be21f6 Merge pull request 'feat: merge subnet facts' (#133) from neoloc/env_fact into develop
Reviewed-on: unkinben/puppet-prod#133
2024-03-10 14:13:46 +09:30
unkinben 5dff24d9b9 feat: merge subnet facts
- add fact for environment
- define 198.18.18.0/24 subnet
2024-03-10 15:42:14 +11:00
unkinben 8009b59514 feat: automatically generate vault certs
- certificate will be generated for:
  - fqdn
  - hostname
  - primary ip address
  - localhost
  - 127.0.0.1
- update base profile to generate vault certificate for all
- create facts for use with vault_certs
2024-03-03 13:38:52 +11:00
unkinben 1030ba460e refacter: renamed facts to libs 2024-02-17 23:03:54 +11:00