Make intra-cluster NOTIFY loop-free (TSIG-keyed, no pod IPs in restart config)
v0.2.5 (PR #14) added an options-scope allow-notify enumerating the primary pod IP on secondaries. Options-scope config feeds the config-hash annotation that rolls the StatefulSet, so any config change rolled the pods, the primary came back on a new pod IP, the operator re-rendered with the new IP, the hash changed, the pods rolled again — an infinite roll loop across every BindCluster. The prod deployment was reverted to v0.2.4. Replace the pod-IP allow-notify with TSIG-authenticated NOTIFY: - Secondaries render `allow-notify { key "<name>"; };` — a static key element with NO IPs. It depends only on the key name, so pod-IP churn can never change the render, the config-hash, or trigger a restart. - The primary signs its outgoing NOTIFYs: the zone-scope also-notify entries (already enumerating replica pod IPs, applied via rndc addzone/modzone with NO restart) now carry `key "<name>"`. - Key choice: reuse the cluster's catalog transfer TSIG key (TransferKeyRef). Secondaries already present it for AXFR and it is in keys.conf on every pod, so no new key plumbing is needed. Add a permanent regression guard for the loop class: - controller: reconcile the ConfigMap with the primary pod on two different IPs and assert the config-hash is byte-identical. - render: render restart-scoped input and assert no pod IP appears in allow-notify; RenderInput no longer has any pod-IP field. Zone-scope also-notify (rndc, no restart) legitimately still lists pod IPs; only restart-scoped config must be pod-IP-independent.
This commit is contained in:
+40
-38
@@ -22,15 +22,20 @@ type RenderInput struct {
|
||||
Forwards []bindv1alpha1.BindZone
|
||||
// PrimaryAddress is the in-cluster address secondaries transfer from.
|
||||
PrimaryAddress string
|
||||
// PrimaryPodAddresses are the primary pod's own IP(s). Secondaries transfer
|
||||
// from PrimaryAddress (the stable primary Service ClusterIP) but the primary
|
||||
// pod's NOTIFYs egress with its *pod* IP as the source — k8s Services only
|
||||
// NAT the inbound direction — so BIND, whose implicit allow-notify is the
|
||||
// zone's primaries list (the ClusterIP), REFUSES them as "non-primary" and
|
||||
// replication falls back to the SOA refresh timer. Secondaries render these
|
||||
// into an options-scope allow-notify so intra-cluster NOTIFYs are accepted
|
||||
// immediately. Empty leaves BIND's default behaviour unchanged.
|
||||
PrimaryPodAddresses []string
|
||||
// NotifyKeyName is the TSIG key name secondaries accept intra-cluster NOTIFYs
|
||||
// under. The primary pod's NOTIFYs egress with its *pod* IP as the source — k8s
|
||||
// Services only NAT the inbound direction — so BIND, whose implicit
|
||||
// allow-notify is the zone's primaries list (the stable ClusterIP), REFUSES
|
||||
// them as "non-primary" and replication falls back to the SOA refresh timer.
|
||||
//
|
||||
// The primary signs its outgoing NOTIFYs with this key (the zone-scope
|
||||
// also-notify entries, applied via rndc, carry `key "<name>"`), and
|
||||
// secondaries render `allow-notify { key "<name>"; }` — an address-match-list
|
||||
// with a *key* element and NO IPs. Because it names only the key, this clause
|
||||
// is fully static: it never changes when a pod IP churns, so it cannot feed a
|
||||
// changed config-hash and cannot roll the StatefulSet (the v0.2.5 loop). Empty
|
||||
// leaves BIND's default behaviour unchanged.
|
||||
NotifyKeyName string
|
||||
}
|
||||
|
||||
// RenderNamedConf returns the primary and secondary named.conf contents for a
|
||||
@@ -95,11 +100,13 @@ func render(in RenderInput, isPrimary bool) string {
|
||||
b.WriteString(" allow-new-zones yes;\n")
|
||||
}
|
||||
b.WriteString(" dnssec-validation auto;\n")
|
||||
// Secondaries accept NOTIFY from the primary's pod IP(s). Catalog member and
|
||||
// plain secondary zones take their implicit allow-notify from their primaries
|
||||
// (the primary Service ClusterIP), but the primary's NOTIFYs are sourced from
|
||||
// its pod IP, so an options-scope allow-notify covering the pod IP(s) is
|
||||
// needed or every NOTIFY is refused and replication waits for the SOA refresh.
|
||||
// Secondaries accept NOTIFY authenticated by the cluster's NOTIFY TSIG key.
|
||||
// Catalog member and plain secondary zones take their implicit allow-notify
|
||||
// from their primaries (the primary Service ClusterIP), but the primary's
|
||||
// NOTIFYs are sourced from its pod IP, so they are refused as "non-primary"
|
||||
// unless an explicit allow-notify covers them. Rather than enumerate pod IPs
|
||||
// (restart-scoped config that depends on pod IPs — the v0.2.5 roll loop), we
|
||||
// accept any NOTIFY signed with the cluster key: `allow-notify { key "X"; }`.
|
||||
b.WriteString(allowNotifyClause(in, isPrimary, " "))
|
||||
for _, o := range c.Spec.ExtraOptions {
|
||||
b.WriteString(" " + strings.TrimRight(o, ";") + ";\n")
|
||||
@@ -293,36 +300,31 @@ func transferPrimaries(in RenderInput) []string {
|
||||
}
|
||||
|
||||
// allowNotifyClause renders an options-scope allow-notify on secondaries that
|
||||
// permits the primary pod IP(s). Zones (catalog members and plain secondaries)
|
||||
// point their primaries at the primary Service ClusterIP for stable AXFR, which
|
||||
// also becomes their implicit allow-notify — but NOTIFYs leave the primary pod
|
||||
// with its pod IP as source, so without this they are refused as "non-primary".
|
||||
// Emitted only on secondaries and only when the primary pod IP(s) are known.
|
||||
// accepts intra-cluster NOTIFYs authenticated by the cluster's NOTIFY TSIG key.
|
||||
// Zones (catalog members and plain secondaries) point their primaries at the
|
||||
// primary Service ClusterIP for stable AXFR, which also becomes their implicit
|
||||
// allow-notify — but NOTIFYs leave the primary pod with its *pod* IP as source,
|
||||
// so without this they are refused as "non-primary". The primary signs those
|
||||
// NOTIFYs with NotifyKeyName (see the also-notify entries applied via rndc), and
|
||||
// this clause admits them by key.
|
||||
//
|
||||
// The rendered clause is `allow-notify { key "<name>"; };` — a key element with
|
||||
// NO IP addresses. It is therefore fully static: it depends only on the key
|
||||
// name, never on any pod IP, so it can never change the config-hash and can
|
||||
// never trigger a rolling restart. This is the fix for the v0.2.5 loop, where an
|
||||
// options-scope allow-notify enumerating the primary pod IP re-rendered on every
|
||||
// pod churn, flipped the hash, rolled the pods, changed the IP, and looped.
|
||||
//
|
||||
// Emitted only on secondaries and only when the NOTIFY key name is known.
|
||||
func allowNotifyClause(in RenderInput, isPrimary bool, indent string) string {
|
||||
if isPrimary {
|
||||
return ""
|
||||
}
|
||||
addrs := make([]string, 0, len(in.PrimaryPodAddresses)+1)
|
||||
seen := map[string]bool{}
|
||||
for _, a := range in.PrimaryPodAddresses {
|
||||
a = strings.TrimSpace(strings.TrimRight(a, ";"))
|
||||
if a == "" || seen[a] {
|
||||
continue
|
||||
}
|
||||
seen[a] = true
|
||||
addrs = append(addrs, a)
|
||||
}
|
||||
// Keep the transfer address (the Service ClusterIP, or the pod IP when no
|
||||
// primary Service exists) in the set: an explicit allow-notify replaces the
|
||||
// implicit primaries-derived default, so it must still cover that source.
|
||||
if a := strings.TrimSpace(strings.TrimRight(in.PrimaryAddress, ";")); a != "" && !seen[a] {
|
||||
addrs = append(addrs, a)
|
||||
}
|
||||
if len(addrs) == 0 {
|
||||
key := strings.TrimSpace(in.NotifyKeyName)
|
||||
if key == "" {
|
||||
return ""
|
||||
}
|
||||
sort.Strings(addrs)
|
||||
return fmt.Sprintf("%sallow-notify { %s };\n", indent, terminate(addrs))
|
||||
return fmt.Sprintf("%sallow-notify { key \"%s\"; };\n", indent, key)
|
||||
}
|
||||
|
||||
func catalogZonesClause(in RenderInput, isPrimary bool, indent string) string {
|
||||
|
||||
Reference in New Issue
Block a user