Notify secondaries immediately on primary zone changes #13

2026-07-21T00:08:17+10:00

unkinben commented

2026-07-21 00:08:17 +10:00

The k8s authoritative cluster is a 3-pod set (pod-0 primary + 2 secondaries) behind one serve VIP. Records are written by RFC2136 dynamic update onto the primary only. They were reaching the secondaries only on the hardcoded 1-hour SOA refresh, because the operator configured no NOTIFY and a seed zone's only apex NS is the primary itself (so BIND's default notify yes had no secondary to notify). Meanwhile the serve VIP kept load-balancing queries onto the stale secondaries, which returned NXDOMAIN; downstream resolvers negatively cached that for the 300s SOA minimum. Net effect: a record (e.g. cobbler.main.unkin.net) flapped between correct and NXDOMAIN for up to an hour after every update to the zone.

Changes:

Add notify explicit + also-notify { <secondary pod IPs> } to primary zone stanzas for catalog-member zones, so a change (in particular a dynamic update) NOTIFYs the secondaries for an immediate IXFR instead of waiting for the SOA refresh. notify explicit scopes NOTIFY to the pod IPs and keeps it off the query-serving VIP.
Add a secondaryPodIPs helper that lists the cluster's non-primary pods with an IP (sorted for stable, churn-free config). Pod-list RBAC already exists on the manager role.
Apply via rndc modzone, so existing zones (main.unkin.net et al.) pick this up on the next reconcile without recreation. The zone controller's periodic requeue refreshes the also-notify set if a secondary pod IP changes.
Shorten the seed SOA refresh (3600->300), retry (900->60) and minimum/negative-TTL (300->60) as a fallback for a missed NOTIFY and to shrink stale-NXDOMAIN caching. Affects newly seeded zones.

Tests: added unit tests asserting the also-notify rendering (with and without secondaries); go build, go vet, go test ./internal/... all pass.

The k8s authoritative cluster is a 3-pod set (pod-0 primary + 2 secondaries) behind one serve VIP. Records are written by RFC2136 dynamic update onto the primary only. They were reaching the secondaries only on the hardcoded 1-hour SOA refresh, because the operator configured no NOTIFY and a seed zone's only apex NS is the primary itself (so BIND's default `notify yes` had no secondary to notify). Meanwhile the serve VIP kept load-balancing queries onto the stale secondaries, which returned NXDOMAIN; downstream resolvers negatively cached that for the 300s SOA minimum. Net effect: a record (e.g. `cobbler.main.unkin.net`) flapped between correct and NXDOMAIN for up to an hour after every update to the zone. Changes: - Add `notify explicit` + `also-notify { <secondary pod IPs> }` to primary zone stanzas for catalog-member zones, so a change (in particular a dynamic update) NOTIFYs the secondaries for an immediate IXFR instead of waiting for the SOA refresh. `notify explicit` scopes NOTIFY to the pod IPs and keeps it off the query-serving VIP. - Add a `secondaryPodIPs` helper that lists the cluster's non-primary pods with an IP (sorted for stable, churn-free config). Pod-list RBAC already exists on the manager role. - Apply via `rndc modzone`, so existing zones (main.unkin.net et al.) pick this up on the next reconcile without recreation. The zone controller's periodic requeue refreshes the also-notify set if a secondary pod IP changes. - Shorten the seed SOA refresh (3600->300), retry (900->60) and minimum/negative-TTL (300->60) as a fallback for a missed NOTIFY and to shrink stale-NXDOMAIN caching. Affects newly seeded zones. Tests: added unit tests asserting the also-notify rendering (with and without secondaries); `go build`, `go vet`, `go test ./internal/...` all pass.

unkinben added 1 commit 2026-07-21 00:15:47 +10:00

Notify secondaries immediately on primary zone changes

ci/woodpecker/pr/pre-commit Pipeline was successful

Details

ci/woodpecker/pr/test Pipeline was successful

Details

ci/woodpecker/pr/build Pipeline was successful

Details

6a07f91ea1

Dynamically-updated primary zones were only reaching the secondary pods on
the hardcoded 1h SOA refresh: the operator emitted no NOTIFY, and a zone's
only apex NS is the primary itself, so default 'notify yes' reached no one.
Queries load-balanced across the serve VIP hit stale secondaries and
returned NXDOMAIN (negatively cached downstream for the 300s SOA minimum),
so records flapped for up to an hour after every update.

Add 'notify explicit' + 'also-notify' with the secondary pod IPs to primary
zone stanzas so an update NOTIFYs the secondaries for an immediate IXFR.
Applied via modzone, so existing zones pick it up on the next reconcile.
Also shorten the seed SOA refresh/retry/minimum as a fallback for missed
NOTIFYs and to shrink stale-NXDOMAIN negative caching.

unkinben force-pushed benvin/notify-secondaries from d5e08607c1 to 6a07f91ea1

2026-07-21 00:15:47 +10:00

Compare

benvin merged commit 439aa9ea6b into main

2026-07-21 00:23:14 +10:00

benvin deleted branch benvin/notify-secondaries

2026-07-21 00:23:14 +10:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unkin/bind-operator#13