Build / precommit (pull_request) Successful in 5m2s

Details

- k8s / metallb / cilium created chaos
- broke puppet agent and servers
- adding issue/resolution here

2025-09-12 23:24:22 +10:00

1.7 KiB

Raw Blame History

add additional master

these steps are required when adding additional puppet masters, as the subject alternative names on the certificate will need to be changed. this requires the old certificate be revoked, cleaned up, and for a new certificate to be generated and signed.

prepare a new node

deploy a new now, or identify a space with the base role
change the hosts class to roles::infra::puppet::master
apply puppet until there are no more changes

revoke the current certificate on the puppet master

sudo puppetserver ca clean --certname ausyd1nxvm1023.main.unkin.net

stop the new puppetserver and cleanup revoked certificates

sudo systemctl stop puppetserver
sudo rm -f /etc/puppetlabs/puppet/ssl/certs/$(hostname -f).pem
sudo rm -f /etc/puppetlabs/puppet/ssl/private_keys/$(hostname -f).pem

copy the current crl.pem, as puppetserver will overwrite it when starting

sudo cp /etc/puppetlabs/puppet/ssl/crl.pem /root/current_crl.pem

request new puppet agent certificate

sudo puppet ssl bootstrap

start the puppetserver service and move the crl.pem back in place

sudo systemctl start puppetserver
sudo cp /root/current_crl.pem /etc/puppetlabs/puppet/ssl/crl.pem

troubleshooting

Issue 1:

[sysadmin@ausyd1nxvm2056 ~]$ sudo puppet agent -t
Error: The CRL issued by 'CN=Puppet CA: prodinf01n01.main.unkin.net' is missing

Find another puppetserver that IS working, copy the /etc/puppetlabs/puppet/ssl/crl.pem to this host, run puppet again.

Issue 2:

[sysadmin@ausyd1nxvm2097 ~]$ sudo puppet agent -t
Error: Failed to parse CA certificates as PEM

The puppet-agents CA cert /etc/puppetlabs/puppet/ssl/certs/ca.pem is empty or missing. Grab it from any other host. Run puppet again.

1.7 KiB Raw Blame History