From 82a5b2ee1a256711e971339bbe474b92f83d91ec Mon Sep 17 00:00:00 2001 From: Ben Vincent Date: Sat, 25 Jul 2026 12:29:21 +1000 Subject: [PATCH] dns: stop exporting legacy DNS fragments (manage_export false) (#495) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Why During the k8s DNS cutover the `dns` module dual-writes every host's records two ways: (a) LEGACY exported `@@concat::fragment`s collected onto the VM puppet DNS master into `/var/named/sources/.conf`, and (b) NEW nsupdate/RFC2136 to the k8s bind-authoritative cluster (`manage_nsupdate`). Both were gated on so the legacy path stayed as a safety net. The legacy path is no longer in any client's resolution path: - **#490** repointed `profiles::dns::base::nameservers` from the VM resolver anycast `198.18.19.16` to the k8s **bind-resolvers** LB `198.18.200.7`. Every puppet client now resolves there. - The k8s bind-resolvers forward `unkin.net`, `main.unkin.net`, and all `18.198.in-addr.arpa` reverse zones to the k8s **bind-authoritative** cluster (`198.18.200.6`), which is populated purely by the nsupdate path — the same `profiles::dns::record` defines, `manage_nsupdate` branch. - The VM master's assembled `/var/named/sources` files are therefore orphaned: nothing resolves against them anymore. This flips `profiles::dns::updater::manage_export: false` to stop nodes exporting the legacy fragments. `manage_nsupdate` stays on. ## What changes - `hieradata/common.yaml`: `profiles::dns::updater::manage_export` `true` -> `false`. ## Verification evidence Pulled all **498** exported `Concat::Fragment` resources from PuppetDB (239 A, 232 PTR, 27 CNAME; 490 unique fqdn/type/value) and `dig`'d each against the k8s authoritative read endpoint `198.18.200.7` (the dynamic path's serving side): | Result | Count | |---|---| | MATCH | 447 | | MISSING-from-dynamic | 38 | | DIFFERENT-value | 5 | Both non-match buckets are accounted for and are **not coverage gaps**: - **38 MISSING** are all records the dynamic path deliberately filters via `profiles::dns::updater::deny_ranges` (`198.18.200.0/24`, `10.42.0.0/16`, ...) plus reverse zones the authoritative cluster isn't even authoritative for: `*-kube-lb0` (198.18.200.x), `*-flannel.1` (10.42.x.x pod CIDR), and `*-nomad`/`*-brcom1` (172.26.64.x / 10.255.255.x). These are exactly the k8s/LB noise the deny ranges exist to keep out of the authoritative zones. Verified every one falls in a denied/non-authoritative range. - **5 DIFFERENT** are stale legacy duplicates that the dynamic path holds *more* correctly: - 4x `au-syd1-prod-halb.main.unkin.net` A: four ceph nodes (2069-2073) each self-register the shared name with their own primary IP (haproxy::dns exports `networking.ip` per host, a legacy collision). Dynamic resolves it to the single VIP `198.18.24.60`. - 1x `dashboard.ceph.unkin.net` CNAME: the ceph node self-publishes this to `lb1.unkin.net.` via its own nsupdate (per the k8s zones.yaml note); the exported VRRP-name fragment is stale. Conclusion: the dynamic path fully covers the served record set, and the serving path no longer depends on the exported files. ## Collector-behavior note (what the flip actually does) `profiles::dns::zone` realizes the fragments with an exported-resource collector (`Concat::Fragment <<| target == ... |>>`). Turning off export purges the fragments from PuppetDB, so on the VM master concat rebuilds each `/var/named/sources/.conf` down to just the header (SOA+NS+glue) as nodes converge. That is **cosmetic**: no client resolves against the VM master anymore (#490), and the k8s authoritative zones are unaffected (fed by nsupdate, not these files). ## Rollback Set `profiles::dns::updater::manage_export: true` again and let puppet converge; nodes re-export their fragments and the VM-master zone files repopulate. ## Follow-up (after soak) Deleting the `@@concat::fragment` / `Concat::Fragment <<||>>` collector code in `profiles::dns::record` / `profiles::dns::zone` (and the VM `profiles::dns::master` zone assembly) is a separate cleanup PR once this has soaked and the VM master is confirmed decommissionable. Reviewed-on: https://git.unkin.net/unkin/puppet-prod/pulls/495 Co-authored-by: Ben Vincent Co-committed-by: Ben Vincent --- hieradata/common.yaml | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/hieradata/common.yaml b/hieradata/common.yaml index 0b9a6fe..0524318 100644 --- a/hieradata/common.yaml +++ b/hieradata/common.yaml @@ -219,7 +219,13 @@ profiles::dns::master::basedir: '/var/named/sources' # profiles::dns::updater::key_secret: ENC[...] # (must match the key the bind-authoritative zones allow-update # with; algorithm hmac-sha256) -profiles::dns::updater::manage_export: true +# k8s is now authoritative: clients resolve via the k8s bind-resolvers +# (profiles::dns::base::nameservers 198.18.200.7, since #490), which forward the +# unkin.net/main.unkin.net/*.18.198.in-addr.arpa zones to the k8s +# bind-authoritative cluster fed by the nsupdate path below. The legacy +# exported-resources -> VM-master /var/named/sources files are no longer in any +# client's resolution path, so stop exporting them. Rollback: set back to true. +profiles::dns::updater::manage_export: false profiles::dns::updater::manage_nsupdate: true profiles::dns::updater::server: '198.18.200.9' profiles::dns::updater::key_name: 'client-update'