fix: thundering hurd #435

Merged

unkinben merged 1 commits from benvin/spread_puppet_agents into develop 2026-01-12 20:21:40 +11:00

Conversation 1 Commits 1 Files Changed 2 +4

unkinben commented 2026-01-12 20:15:48 +11:00

Owner

Copy Link

• started all puppet clients at the same time, resulting in thundering herd
• add a randomness timer of 10 minutes

- started all puppet clients at the same time, resulting in thundering herd - add a randomness timer of 10 minutes

unkinben added 1 commit 2026-01-12 20:15:49 +11:00

fix: thundering hurd ... 49f142a6a3

- started all puppet clients at the same time, resulting in thundering herd
- add a randomness timer of 10 minutes

unkinben commented 2026-01-12 20:16:03 +11:00

Author

Owner

Copy Link

tested:

ben@metabox ~/s/p/puppet-prod> ./deploy.py ausyd1nxvm2043                                                                                                         benvin/spread_puppet_agents?
Reading inline script metadata from `./deploy.py`
[2026-01-12 20:15:02] Current branch: benvin/spread_puppet_agents
[2026-01-12 20:15:02] Restarting puppet-g10k.service on puppetca.query.consul...
[2026-01-12 20:15:02] Running puppet agent on ausyd1nxvm2043 with environment benvin/spread_puppet_agents...
[2026-01-12 20:15:03] Info: Using environment 'benvin_spread_puppet_agents'
[2026-01-12 20:15:03] Info: Retrieving pluginfacts
[2026-01-12 20:15:03] Info: Retrieving plugin
[2026-01-12 20:15:04] Info: Loading facts
[2026-01-12 20:15:07] Notice: Requesting catalog from puppetca.query.consul:8140 (198.18.29.239)
[2026-01-12 20:15:10] Notice: Catalog compiled by ausyd1nxvm2097.main.unkin.net
[2026-01-12 20:15:10] Info: Caching catalog for ausyd1nxvm2043.main.unkin.net
[2026-01-12 20:15:10] Info: Applying configuration version '49f142a6a3c2e88236dc6211c1f22c61b1aa3764'
[2026-01-12 20:15:13] Notice: /Stage[main]/Profiles::Puppet::Client/File[/etc/puppetlabs/puppet/puppet.conf]/content:
[2026-01-12 20:15:13] --- /etc/puppetlabs/puppet/puppet.conf    2025-05-24 18:46:17.409313312 +1000
[2026-01-12 20:15:13] +++ /tmp/puppet-file20260112-86834-13d72kb        2026-01-12 20:15:13.581788210 +1100
[2026-01-12 20:15:13] @@ -12,3 +12,5 @@
[2026-01-12 20:15:13]  show_diff = true
[2026-01-12 20:15:13]  usecacheonfailure = false
[2026-01-12 20:15:13]  number_of_facts_soft_limit = 4096
[2026-01-12 20:15:13] +splay = true
[2026-01-12 20:15:13] +splaylimit = 600
[2026-01-12 20:15:13] 
[2026-01-12 20:15:13] Notice: /Stage[main]/Profiles::Puppet::Client/File[/etc/puppetlabs/puppet/puppet.conf]/content: content changed '{sha256}1428271dc21b653be282c2ba25732c65bba312161fd67b39
64c99f6dac3a8d4b' to '{sha256}2aec9aa68f82592531bedb67fa9c6bd624045a3b42659a8b0d450cb5dcab790a'
[2026-01-12 20:15:13] Info: /Stage[main]/Profiles::Puppet::Client/File[/etc/puppetlabs/puppet/puppet.conf]: Scheduling refresh of Service[puppet]
[2026-01-12 20:15:13] Notice: /Stage[main]/Profiles::Puppet::Agent/Service[puppet]: Triggered 'refresh' from 1 event
[2026-01-12 20:15:14] Notice: Applied catalog in 3.76 seconds

tested: ``` ben@metabox ~/s/p/puppet-prod> ./deploy.py ausyd1nxvm2043 benvin/spread_puppet_agents? Reading inline script metadata from `./deploy.py` [2026-01-12 20:15:02] Current branch: benvin/spread_puppet_agents [2026-01-12 20:15:02] Restarting puppet-g10k.service on puppetca.query.consul... [2026-01-12 20:15:02] Running puppet agent on ausyd1nxvm2043 with environment benvin/spread_puppet_agents... [2026-01-12 20:15:03] Info: Using environment 'benvin_spread_puppet_agents' [2026-01-12 20:15:03] Info: Retrieving pluginfacts [2026-01-12 20:15:03] Info: Retrieving plugin [2026-01-12 20:15:04] Info: Loading facts [2026-01-12 20:15:07] Notice: Requesting catalog from puppetca.query.consul:8140 (198.18.29.239) [2026-01-12 20:15:10] Notice: Catalog compiled by ausyd1nxvm2097.main.unkin.net [2026-01-12 20:15:10] Info: Caching catalog for ausyd1nxvm2043.main.unkin.net [2026-01-12 20:15:10] Info: Applying configuration version '49f142a6a3c2e88236dc6211c1f22c61b1aa3764' [2026-01-12 20:15:13] Notice: /Stage[main]/Profiles::Puppet::Client/File[/etc/puppetlabs/puppet/puppet.conf]/content: [2026-01-12 20:15:13] --- /etc/puppetlabs/puppet/puppet.conf 2025-05-24 18:46:17.409313312 +1000 [2026-01-12 20:15:13] +++ /tmp/puppet-file20260112-86834-13d72kb 2026-01-12 20:15:13.581788210 +1100 [2026-01-12 20:15:13] @@ -12,3 +12,5 @@ [2026-01-12 20:15:13] show_diff = true [2026-01-12 20:15:13] usecacheonfailure = false [2026-01-12 20:15:13] number_of_facts_soft_limit = 4096 [2026-01-12 20:15:13] +splay = true [2026-01-12 20:15:13] +splaylimit = 600 [2026-01-12 20:15:13] [2026-01-12 20:15:13] Notice: /Stage[main]/Profiles::Puppet::Client/File[/etc/puppetlabs/puppet/puppet.conf]/content: content changed '{sha256}1428271dc21b653be282c2ba25732c65bba312161fd67b39 64c99f6dac3a8d4b' to '{sha256}2aec9aa68f82592531bedb67fa9c6bd624045a3b42659a8b0d450cb5dcab790a' [2026-01-12 20:15:13] Info: /Stage[main]/Profiles::Puppet::Client/File[/etc/puppetlabs/puppet/puppet.conf]: Scheduling refresh of Service[puppet] [2026-01-12 20:15:13] Notice: /Stage[main]/Profiles::Puppet::Agent/Service[puppet]: Triggered 'refresh' from 1 event [2026-01-12 20:15:14] Notice: Applied catalog in 3.76 seconds ```

unkinben merged commit 9f5b1cec82 into develop 2026-01-12 20:21:40 +11:00

unkinben deleted branch benvin/spread_puppet_agents 2026-01-12 20:21:40 +11:00

unkinben referenced this issue from a commit 2026-01-12 20:21:40 +11:00

fix: thundering hurd (#435)

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No Label

Milestone

Clear milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

No Assignees

1 Participants

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: unkin/puppet-prod#435

No description provided.