unkin/argocd-apps
3
0
Fork 0
Code Issues 7 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

Pin puppet master to a single Recreate replica #341

Merged
benvin merged 1 commits from benvin/puppet-master-single into main 2026-08-08 20:02:39 +10:00
Conversation 0 Commits 1 Files Changed 4
+5 -42
unkinben commented 2026-08-08 19:58:57 +10:00
Owner
Copy Link
Copy Source

Why

The puppet MASTER is the singleton CA/master. A second master, even transiently during a rolling update, races on CA/cert signing and shared state (the CA lives on a shared PVC mounted by every master pod). The master was previously driven by an HPA with minReplicas: 2, maxReplicas: 5 and a RollingUpdate strategy, so 2-5 masters could coexist normally and a rollout would briefly run old+new masters against the same CA data — a latent CA-corruption/split-brain bug. Recreate guarantees the old pod terminates before the new one starts, so two masters never coexist.

Changes

  • Set puppetserver-master spec.replicas: 1 and spec.strategy.type: Recreate (drops RollingUpdate).
  • Remove the puppetserver-masters-autoscaler HPA and its kustomization entry, which forced 2-5 master replicas and would otherwise override replicas: 1.
  • Refresh the puppetserver-master-vpa note to reflect the pinned-singleton, no-HPA state (VPA stays updateMode: Off, recommendation-only).

The compiler (puppetserver-compiler) remains the horizontally-scalable tier with its own HPA — untouched. puppetdb/puppetboard untouched.

https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT

## Why The puppet MASTER is the singleton CA/master. A second master, even transiently during a rolling update, races on CA/cert signing and shared state (the CA lives on a shared PVC mounted by every master pod). The master was previously driven by an HPA with `minReplicas: 2`, `maxReplicas: 5` and a `RollingUpdate` strategy, so 2-5 masters could coexist normally and a rollout would briefly run old+new masters against the same CA data — a latent CA-corruption/split-brain bug. Recreate guarantees the old pod terminates before the new one starts, so two masters never coexist. ## Changes - Set `puppetserver-master` `spec.replicas: 1` and `spec.strategy.type: Recreate` (drops RollingUpdate). - Remove the `puppetserver-masters-autoscaler` HPA and its kustomization entry, which forced 2-5 master replicas and would otherwise override `replicas: 1`. - Refresh the `puppetserver-master-vpa` note to reflect the pinned-singleton, no-HPA state (VPA stays `updateMode: Off`, recommendation-only). The compiler (`puppetserver-compiler`) remains the horizontally-scalable tier with its own HPA — untouched. puppetdb/puppetboard untouched. https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT
unkinben added 1 commit 2026-08-08 19:58:57 +10:00
Pin puppet master to a single Recreate replica
ci/woodpecker/pr/vector-test Pipeline was successful
Details
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/pr/kubeconform Pipeline was successful
Details
a5f3b18e86 
The puppet MASTER is the singleton CA/master: a second master, even
transiently during a rolling update, races on CA/cert signing and shared
state. It was previously driven by an HPA (minReplicas 2, maxReplicas 5)
with a RollingUpdate strategy, so 2-5 masters could coexist and a rollout
would briefly run old+new masters against the same shared CA PVC.

- Set puppetserver-master replicas to 1 and strategy to Recreate so the
  old master always terminates before a new one starts.
- Remove the puppetserver-masters-autoscaler HPA (and its kustomization
  entry) that forced 2-5 master replicas.
- Refresh the master VPA note to reflect the pinned-singleton, no-HPA state.

Claude-Session: https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT
benvin merged commit 5d1cc10588 into main 2026-08-08 20:02:39 +10:00
benvin deleted branch benvin/puppet-master-single 2026-08-08 20:02:39 +10:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
benvin (BenVincent) unkin-agent (Unkin Agent) unkinben (Ben Vincent)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: unkin/argocd-apps#341
Delete Branch "benvin/puppet-master-single"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?