The cobbler master exports `cobbler.main.unkin.net` as a CNAME whose value is the bare hostname (`ausyd1nxvm2098`). The legacy puppet DNS master loaded this into a zone file under `$ORIGIN main.unkin.net.`, so BIND completed the bare label to `ausyd1nxvm2098.main.unkin.net.` and it resolved. `dns-updater` renders RFC2136 updates with no `$ORIGIN`, so the same bare label becomes root-absolute (`ausyd1nxvm2098.`) and dead-ends in NXDOMAIN — the record exists on the k8s authoritative servers (and via the resolvers at 198.18.200.7) but has no reachable address. This is the same failure already documented and fixed for haproxy in `profiles::haproxy::dns`.
Changes:
- Set the cobbler CNAME `value` to the fully-qualified `${facts.networking.fqdn}.` (trailing dot) so the target is origin-independent and resolves identically on both the legacy zone-file export and the dns-updater nsupdate path.
- Add an explanatory comment mirroring the one in `profiles::haproxy::dns` to prevent reintroducing a bare label.
Reviewed-on: #491
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Why
The halb publishes service CNAMEs (`git.unkin.net`, `dashboard.ceph.unkin.net`, and every other halb-fronted name) with a **bare target label**, e.g.:
```
git.unkin.net. 300 IN CNAME au-syd1-prod-halb-vrrp.
```
`dns-updater` builds the RR with `dns.NewRR` and **no $ORIGIN**, so a bare value becomes root-absolute (`au-syd1-prod-halb-vrrp.`) and dead-ends in NXDOMAIN. Once the k8s bind became authoritative for these zones, that broke resolution of every halb-fronted service (git, the Ceph dashboard, ...). `au-syd1-prod-halb-vrrp.unkin.net` / `.main.unkin.net` resolve fine (198.18.19.17) — only the CNAME target was truncated.
## Change (`site/profiles/manifests/haproxy/dns.pp`)
Emit **fully-qualified** CNAME targets (trailing dot), the FQDN form `dns-updater` expects (per its own test fixture `au-syd1-prod-halb.main.unkin.net.`):
- vrrp cnames: `${location_environment}-halb-vrrp.${domain}.`
- non-vrrp cnames: `${location_environment}-halb.${domain}.`
The matching A records are already published in `main.unkin.net`/`unkin.net` just above, so the targets resolve to the VIP.
---------
Co-authored-by: benvin <neotheo@gmail.com>
Reviewed-on: #489
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Configures the dns-updater record filter (dns-updater#3) so k8s/LB/internal addresses stay out of the authoritative zones.
## Why
Hosts publish records the authoritative server should not hold — `*-flannel.1 A 10.42.x`, `*-kube-lb0 A 198.18.200.x`, and the matching reverse PTRs (`200.18.198`, `2.42.10`) — which the daemon WARNs `NOTAUTH`/`bad authentication` on.
## Changes
- `DNS_UPDATER_DENY_RANGES=198.18.199.0/24,198.18.200.0/24,10.42.0.0/16,10.43.0.0/16` (pod CIDR, service CIDR, LB VIP pool, .199).
- New class params: `deny_ranges` / `allow_ranges` / `deny_domains` / `allow_domains`.
- `package_ensure` → `latest` so hosts pick up the filter release (rpm-internal `metadata_expire` is 1h, so no thrash).
## Sequencing
Needs dns-updater **#3** merged + a new tag (v0.2.0) so the filter env vars are honored. The env keys are ignored by v0.1.0, so this is safe to merge first — it takes effect when hosts upgrade to v0.2.0.
Reviewed-on: #485
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Why
Replace the `profiles::dns::updater` shell mechanism (`dns-update.sh` + `dns-update.path`/`.service` + the in-run `exec`) with the packaged **dns-updater** daemon. The daemon watches the records file (inotify) and network interfaces and pushes TSIG-signed RFC2136 updates to BIND natively — with structured per-zone RCODEs and a status API/facter fact, so failures like the recent `invalid owner name: empty label` / NOTZONE surface directly instead of as opaque nsupdate stderr.
## Changes
- Install the `dns-updater` package; manage `/etc/dns-updater/env`.
- Run the packaged `dns-updater.service`, restarting **only** on env/key change — records-file edits are picked up by the daemon`s own inotify watch, so no service churn on record changes.
- Keep the `concat` records file and the TSIG key file unchanged (same paths/format).
- Ensure the old `/usr/local/bin/dns-update` + `dns-update.path`/`.service` units are absent.
- Drop the now-dead `dns-update.{sh,service,path}.epp` templates.
## Sequencing — HOLD
Do not merge until the `dns-updater` RPM is published to artifactapi `rpm-internal` (needs terraform-git#33 to create the repo, then the daemon code pushed + tagged so Woodpecker builds the RPM). Merging before the package exists makes `package { dns-updater }` fail on every host.
Supersedes the interim shell fix in #481 (which stays valid until this rolls out). Keeps the file as the desired-state interface (puppet owns desired records; the daemon reconciles + reports).
Reviewed-on: #482
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
## Why
6 hosts (ausyd1nxvm2069-2073, 2098) ended up with a reverse PTR in bind-authoritative but **no forward A record**, and the `unkin.net` service records (git/grafana/auth/fafflix, all published by the halb host 2069) never landed at all.
VictoriaLogs (`dns-update-apply` on 2069 & 2070) shows the cause:
```
dns-update: nsupdate to 198.18.200.9 failed
invalid owner name: empty label
syntax error
```
`profiles::dns::record` publishes some records whose name is already fully-qualified (trailing dot) — e.g. `au-syd1-pve.main.unkin.net.`, `cobbler.main.unkin.net.`, `dashboard.ceph.unkin.net.`, and the halb CNAMEs. The `dns-update` script `fqdn()` unconditionally appended the zone, producing `…net..main.unkin.net.` — the `..` is an empty label, which nsupdate rejects, failing the entire per-zone `send`. The reverse-PTR send is sorted first and its name is always relative, so it still applied — hence "PTR but no A".
## Change
`fqdn()` now handles three cases:
- `@`/empty → zone apex (unchanged)
- name ending in `.` → already FQDN, used verbatim (**the fix**)
- otherwise → relative, append `.zone.` (unchanged)
Verified against all record shapes (relative host, apex, FQDN CNAME, reverse label) — no more `..`.
## After merge
Once puppet re-runs on the affected hosts their `main.unkin.net`/`unkin.net` updates succeed, filling in the missing A records and the `unkin.net` service zone. Pairs with argocd-apps#260 (adds the `ceph.unkin.net` zone so `dashboard.ceph.unkin.net` does not then hit NOTZONE).
Reviewed-on: #481
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Replaces the exported-resources → puppet DNS master zone-file flow with per-host RFC2136 dynamic updates against the k8s **bind-authoritative** write endpoint (198.18.200.9). The master no longer manages zone files.
## Design
Each node assembles its DNS records into a local concat file; a systemd `.path` unit watches it and runs `dns-update` (nsupdate) on change — exactly the watch-a-file model requested.
## Changes
- **profiles::dns::updater** (new): concat records file + TSIG key file + `dns-update` script + `dns-update.service` (oneshot) + `dns-update.path` (watcher). The script sends only the delta since last run and deletes removed records, grouped per zone.
- **profiles::dns::record**: writes a local concat fragment (`zone|name|type|ttl|value`) instead of exporting `@@concat::fragment` to the master.
- **profiles::dns::base**: includes `profiles::dns::updater` (all nodes).
- **hiera**: `profiles::dns::updater` server/key_name/algorithm in common.yaml.
## Inert until keyed
The updater does nothing until `profiles::dns::updater::key_secret` (TSIG) is set in eyaml — records are assembled but not applied, so nodes are safe before the key exists.
## Prerequisites (k8s side, separate)
1. The `bind-authoritative` zones must set `dynamicUpdate: true` + an `updateKeyRef` (a client-update BindTSIGKey) so they accept these updates.
2. The TSIG key must be shared: the operator-generated key value goes into eyaml here (or the planned Vault-sync feature bridges it).
## Validated
puppet parser/epp validate, puppet-lint, and a functional test of the generated per-zone nsupdate message (replace + delete-removed).
Reviewed-on: #475
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
Pin grafana package version to 13.0.2 via a new version parameter on
profiles::metrics::grafana, wired through to the puppet-grafana class.
---------
Co-authored-by: Ben Vincent <ben@unkin.net>
Reviewed-on: #470
## Summary
- Adds `Unkin::Ceph::Utils` facter module detecting ceph service instances via `systemctl list-units`, exposing `is_ceph_mon`, `is_ceph_mgr`, `is_ceph_mds`, `is_ceph_osd` booleans and a `ceph_services` hash of unit names
- Adds `profiles::ceph::mon`, `mgr`, `mds`, `osd` — each with `Boolean $ensure_running` that iterates discovered service instances and manages them as running and enabled
- Works across incus nodes (mon/mgr/mds/osd) and k8s compute/control nodes (osd only); verified on prodnxsr0001 which correctly reports `is_ceph_osd: true` and `ceph_services: {osd: [ceph-osd@5]}`
## Test plan
- [x] Noop deploy against prodnxsr0001.main.unkin.net passed cleanly
- [x] `ceph_services` fact returns correct service map
- [x] `is_ceph_osd` returns `True`, `is_ceph_mon` returns `False` as expected
- [x] Test on an incus/ceph node with mon/mgr/mds services
Reviewed-on: #459
Replace deprecated dalen-puppetdbquery module with native puppetdb_query
function using PQL syntax to resolve URI.escape compatibility issues.
This is required to migrated to Puppet 8 (and kubernetes).
Changes:
- Remove dalen-puppetdbquery dependency from Puppetfile
- Replace query_nodes() calls with puppetdb_query() using PQL syntax
- Update 27 function calls across 18 Puppet manifests
- Maintain equivalent functionality with improved compatibility
Reviewed-on: #457
saving artifacts are breaking in some actions as the runner will switch
between different git hosts. using haproxy will ensure the same backend
is always hit via stick-tables and cookies
- ensure runners use haproxy to reach git
we now package act_runner now, lets use the rpm
- change installation method to rpm instead of curl + untar
- add capability to versionlock act_runner
- fix paths to act_runner
- remove manually installed act_runner
Reviewed-on: #432
- update root password in common.eyaml
- add missing param to the accounts::root manifest
- remove if block as undef sshkeys has same effect
Reviewed-on: #429
- add module to manage externaldns bind for k8s
- add infra::dns::externaldns role
- add 198.18.19.20 as anycast for k8s external-dns service
Reviewed-on: #428
- add SMTP submission listener on port 587 with TLS requirement
- configure HAProxy frontend/backend for submission with send-proxy-v2 support
- add send-proxy-v2 support to all listeners
- add dynamic HAProxy node discovery for proxy trusted networks
- use service hostname instead of node FQDN for autoconfig/autodiscover
- remove redundant IMAP/IMAPS/SMTP alt-names from TLS certificates
- update VRRP CNAME configuration to use mail.main.unkin.net
Reviewed-on: #425
- refactor profiles::postfix::gateway as parameterized class
- move base postfix parameters, transports, and virtuals to hiera for flexibility
- convert SMTP restrictions to arrays for better readability using join()
- add postscreen enable/disable boolean with conditional master.cf configuration
- add per-domain TLS policy maps (smtp_tls_policy_maps)
- convert alias_maps to array parameter for flexibility
- convert all postfix map files to ERB templates with parameter hashes
- add map parameters: sender_canonical_maps, sender_access_maps, relay_recipients_maps,
relay_domains_maps, recipient_canonical_maps, recipient_access_maps, postscreen_access_maps, helo_access_maps
- move default map data to hiera while keeping parameters as empty hashes by default
This approach balances flexibility with data-driven configuration, allowing
easy customization through parameters while keeping transport/virtual maps
and default map data in hiera for role-specific overrides.
Reviewed-on: #416
- add voxpupuli-postfix module to Puppetfile
- create profiles::postfix::gateway class with config based on efa5
- add master.cf entries for postscreen, smtpd, dnsblog, and tlsproxy services
- create postfix hash files: aliases, access controls, canonical maps
- configure TLS with system PKI certificates and strong cipher suites
- add transport and virtual alias mappings for mail routing
Reviewed-on: #414
- enable openvox repo
- ensure puppetdb-termini and puppetserver are purged
- set openvox-server as the package to install
- set termini package to openvoxdb-termini
Reviewed-on: #412
- change from puppet-agent to openvox-agent
- upgrade version from 7.34 to 7.36
- ensure workflow of: Yumrepo -> dnf-makecache -> Package
Reviewed-on: #408
Add hasrestart => true to nginx service in simpleproxy profile to ensure
nginx performs a full restart (not reload) when certificate files change.
This is required because nginx reload does not pick up SSL certificate
changes from disk.
Reviewed-on: #402
- ensure the autopromoter removes hardlinks/replicas for repos older
than the current promoted monthly
- this is to reduce MDS load for ceph, as hardlinks require memory
Reviewed-on: #393
- start managing ceph configuration file
- manage ceph-radosgw
- merge the ceph::conf and ceph::node profiles
- ensure the ceph repos exist
- mange nginx frontend and consul service
Reviewed-on: #380
- create module class for journald clients
- ensure module class it used on all hosts
- use consul service address for insert/journald
Reviewed-on: #377
unkinben
benvin