vault: move openbao plugin sockets off /tmp onto /run (#509)
## Why The litellm secrets engine on the OpenBao cluster died with `rpc Unavailable / dial unix /tmp/pluginNNN: no such file` (terraform-vault#112), fixed only by a manual `sys/plugins/reload/backend`. Root cause (post-incident log audit): go-plugin puts each plugin's control socket under the process TMPDIR — `/tmp/pluginNNN` — and `vault.service` runs without PrivateTmp. The daily `systemd-tmpfiles-clean` reaps aged `/tmp` files; bao ran a single systemd invocation for 3+ weeks, so the socket long outlived the `/tmp` cleanup age and got deleted out from under the still-running plugin process (no panic/OOM/signal/exit in the bao journal — the process was healthy, just unreachable). The risk is shared by every OpenBao plugin (gpg, rancher, gitea, ...), not just litellm. Supersedes the earlier tmpfiles-exclude approach (#508, closed) with the permanent fix: move the sockets off `/tmp` entirely. ## Change - Add a `vault.service` drop-in (`systemd::manage_dropin`) that sets `Environment=TMPDIR=/run/vault-plugins` and `RuntimeDirectory=vault-plugins` (mode 0700). - Point plugin sockets at `/run` (tmpfs, no age-based cleanup); `RuntimeDirectory` creates/owns the dir per service start. - Notify a vault service restart (module handles daemon-reload) so the new TMPDIR takes effect and plugins respawn with sockets under `/run`. ## Heads-up Puppet rolls a **bao restart per node** when this lands (the drop-in notifies `Service['vault']`). With auto-unseal (this cluster runs `profiles::vault::unseal`) it is a rolling non-event; if any node relies on manual unseal it will come back **sealed** and need unsealing. Merge consciously / stagger if needed. https://claude.ai/code/session_01JUoARVdmhxKQHyyyp1pxeT Reviewed-on: #509 Co-authored-by: Ben Vincent <ben@unkin.net> Co-committed-by: Ben Vincent <ben@unkin.net>
This commit was merged in pull request #509.
This commit is contained in:
@@ -130,6 +130,20 @@ class profiles::vault::server (
|
||||
mode => '0600',
|
||||
}
|
||||
|
||||
# go-plugin creates each secrets plugin's control socket under TMPDIR
|
||||
# (/tmp/pluginNNN by default); systemd-tmpfiles-clean reaps aged /tmp files
|
||||
# and severs the socket of a long-lived plugin, orphaning the process. Point
|
||||
# TMPDIR at a per-start RuntimeDirectory on /run (tmpfs, no age cleanup).
|
||||
systemd::manage_dropin { 'plugin-tmpdir.conf':
|
||||
unit => 'vault.service',
|
||||
service_entry => {
|
||||
'RuntimeDirectory' => 'vault-plugins',
|
||||
'RuntimeDirectoryMode' => '0700',
|
||||
'Environment' => 'TMPDIR=/run/vault-plugins',
|
||||
},
|
||||
notify => Service['vault'],
|
||||
}
|
||||
|
||||
service { 'vault':
|
||||
ensure => true,
|
||||
enable => true,
|
||||
|
||||
Reference in New Issue
Block a user