Apply config changes via rndc reconfig instead of restarting pods #26

2026-10-16T23:31:00+11:00

unkin-agent commented

2026-10-16 23:31:00 +11:00

Rendered config changes restart every BIND pod because the config hash sits on the pod template. Apply them in place with rndc reconfig instead.

symlink named.conf to the projected ConfigMap so kubelet updates reach named
wait until a pod's mounted config hash matches, then reconfigure it once
record per-pod applied hashes in status.appliedConfig; reconfigure unrecorded pods once
set ConfigApplied: Propagating (incl. not-ready pods), ExecFailed or ReconfigFailed with named-checkconf output, backing off 2m on errors
roll pods only when the entrypoint or pod spec changes

Rendered config changes restart every BIND pod because the config hash sits on the pod template. Apply them in place with `rndc reconfig` instead. - symlink named.conf to the projected ConfigMap so kubelet updates reach named - wait until a pod's mounted config hash matches, then reconfigure it once - record per-pod applied hashes in `status.appliedConfig`; reconfigure unrecorded pods once - set `ConfigApplied`: `Propagating` (incl. not-ready pods), `ExecFailed` or `ReconfigFailed` with named-checkconf output, backing off 2m on errors - roll pods only when the entrypoint or pod spec changes

unkin-agent added 1 commit 2026-10-16 23:31:00 +11:00

apply config changes via rndc reconfig instead of restarting pods

ci/woodpecker/pr/pre-commit Pipeline was successful

Details

ci/woodpecker/pr/test Pipeline was successful

Details

ci/woodpecker/pr/build Pipeline was successful

Details

70e5b0ed55

- symlink named.conf to the projected file so kubelet updates reach named
- drop the config hash from the pod template; hash only the entrypoint
- reconfig each pod once its mounted config matches, tracked in status
- surface reconfig failures with checkconf detail on ConfigApplied

unkin-agent commented

2026-10-16 23:37:43 +11:00

internal/controller/bindcluster_controller.go:496 — detail, _ := Exec(named-checkconf) drops the error; Exec returns only stdout and puts stderr in the error, and named-checkconf writes its file:line errors to stderr, so ConfigApplied carries no detail in production (the fake returns it on stdout, hiding this) → use the error text (stderr) when stdout is empty, and make fakeExec return it via the error.
internal/controller/bindcluster_controller.go:486 — a new ready pod has no appliedConfig entry, and its mount already equals want, so it gets a needless rndc reconfig (every rolled pod, and every pod once on upgrade) → seed the entry without reconfig when the pod started after the config was last written, or otherwise skip pods whose named loaded the current file.
internal/controller/bindcluster_controller.go:489 — an Exec error on the cat (RBAC, exec failure) is treated as "waiting" and reported as Propagating forever, requeueing every 15s → on err (not hash mismatch) put the pod in failed with the error.
internal/controller/reconfig_test.go:100 — no test for pod restart/new pod, cat Exec error, or Reconcile persisting appliedConfig via Status().Update → add them.
nit: internal/controller/bindcluster_controller.go:486 — a not-ready pod with stale config is skipped without being counted, so ConfigApplied=True while it is unconverged → count it as waiting.

- internal/controller/bindcluster_controller.go:496 — `detail, _ := Exec(named-checkconf)` drops the error; Exec returns only stdout and puts stderr in the error, and named-checkconf writes its file:line errors to stderr, so ConfigApplied carries no detail in production (the fake returns it on stdout, hiding this) → use the error text (stderr) when stdout is empty, and make fakeExec return it via the error. - internal/controller/bindcluster_controller.go:486 — a new ready pod has no appliedConfig entry, and its mount already equals `want`, so it gets a needless rndc reconfig (every rolled pod, and every pod once on upgrade) → seed the entry without reconfig when the pod started after the config was last written, or otherwise skip pods whose named loaded the current file. - internal/controller/bindcluster_controller.go:489 — an Exec error on the `cat` (RBAC, exec failure) is treated as "waiting" and reported as Propagating forever, requeueing every 15s → on err (not hash mismatch) put the pod in `failed` with the error. - internal/controller/reconfig_test.go:100 — no test for pod restart/new pod, `cat` Exec error, or Reconcile persisting appliedConfig via Status().Update → add them. - nit: internal/controller/bindcluster_controller.go:486 — a not-ready pod with stale config is skipped without being counted, so ConfigApplied=True while it is unconverged → count it as waiting.

unkin-agent added 1 commit 2026-10-16 23:39:47 +11:00

surface exec and checkconf errors on ConfigApplied, count not-ready stale pods as waiting

ci/woodpecker/pr/test Pipeline was successful

Details

ci/woodpecker/pr/pre-commit Pipeline was successful

Details

ci/woodpecker/pr/build Pipeline was canceled

Details

60b04e5eb1

unkin-agent commented

2026-10-16 23:41:45 +11:00

internal/controller/bindcluster_controller.go:101,531 — requeueShort overwrites requeueLong whenever any pod is waiting or the StatefulSet is not Ready, so a pod with a failing rndc reconfig/exec is retried (reconfig + checkconf exec) every 15s for as long as a rollout or a stale pod exists → let failures keep requeueLong (only set short when requeue == 0), and add a test with one failed pod plus one waiting pod asserting requeueLong.
internal/controller/reconfig_test.go — nothing covers the entrypoint-hash annotation changing when entrypointScript() changes, so "roll only on entrypoint/pod-spec change" is only half tested → add a case that changes the entrypoint hash input (or asserts annotation == sha256Hex(entrypointScript())) and expects the template to differ.

- internal/controller/bindcluster_controller.go:101,531 — `requeueShort` overwrites `requeueLong` whenever any pod is waiting or the StatefulSet is not Ready, so a pod with a failing `rndc reconfig`/exec is retried (reconfig + checkconf exec) every 15s for as long as a rollout or a stale pod exists → let failures keep `requeueLong` (only set short when `requeue == 0`), and add a test with one failed pod plus one waiting pod asserting `requeueLong`. - internal/controller/reconfig_test.go — nothing covers the entrypoint-hash annotation changing when `entrypointScript()` changes, so "roll only on entrypoint/pod-spec change" is only half tested → add a case that changes the entrypoint hash input (or asserts annotation == `sha256Hex(entrypointScript())`) and expects the template to differ.

unkin-agent added 1 commit 2026-10-16 23:43:06 +11:00

keep the 2m backoff on config apply failures while pods are waiting or the StatefulSet is not ready

ci/woodpecker/pr/pre-commit Pipeline was successful

Details

ci/woodpecker/pr/test Pipeline was successful

Details

ci/woodpecker/pr/build Pipeline was successful

Details

2ea750dffa

unkin-agent commented

2026-10-16 23:44:41 +11:00

nit: internal/controller/bindcluster_controller.go:473 — pod List failure in reconfigPods returns requeueShort and is swallowed (no log, no condition) → log it and return requeueLong like the other failures.

- nit: internal/controller/bindcluster_controller.go:473 — pod List failure in reconfigPods returns requeueShort and is swallowed (no log, no condition) → log it and return requeueLong like the other failures.

benvin merged commit 66582eb93b into main

2026-10-17 12:37:11 +11:00

benvin deleted branch benvin/reconfig-without-restart

2026-10-17 12:37:12 +11:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unkin/bind-operator#26