From e0eeeb6b04d644e010abba68ece514b11ba47483 Mon Sep 17 00:00:00 2001
From: Ben Vincent <ben@unkin.net>
Date: Tue, 28 Jul 2026 23:11:30 +1000
Subject: [PATCH] Use full postgres image on minimal CNPG clusters so WAL
 archiving works (#304)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

## Why

The CNPG buckets are empty after PR #298. Live diagnosis found **two** independent
causes; this PR fixes one of them.

`litellm`, `puppet` and `paperclip` run the CNPG `17-minimal-trixie` image, which
omits the `barman-cloud` CLI. In-tree `barmanObjectStore` archiving shells out to
`barman-cloud-wal-archive`, so their archiver dies immediately:

```
ContinuousArchiving=False :: unexpected failure invoking barman-cloud-wal-archive:
exec: "barman-cloud-wal-archive": executable file not found in $PATH
```

(verified on the live `puppet-postgres` primary: `which barman-cloud-wal-archive` →
not found; on a `-system` cluster it resolves to `/usr/local/bin/barman-cloud-wal-archive`).

## How

Switch those three clusters from `17-minimal-trixie` to `17-system-trixie` — the
`-system` variant already used by the other six clusters, which bundles the
barman-cloud tools. Tag confirmed present upstream (ghcr manifest HTTP 200).

```
- imageName: ghcr.io/cloudnative-pg/postgresql:17-minimal-trixie
+ imageName: ghcr.io/cloudnative-pg/postgresql:17-system-trixie
```

CNPG applies this as a rolling image update (switchover, no data change).

## Not fixed here (separate, primary blocker)

The other six clusters (full image, barman present) fail with a **TLS trust**
error — the reflected `vault-ca-cert` bundle carries a **stale intermediate CA**,
so barman can't verify `s3.ceph.unkin.net`:

```
SSL: CERTIFICATE_VERIFY_FAILED ... certificate signature failure
```

That is a shared trust-anchor refresh (likely owned by terraform-k8s /
`config/certificates/secret.yaml`, `managed-by: terragrunt`), handled separately —
it also gates litellm/puppet once they have barman. See the investigation report.

## Validation

- `kustomize build --enable-helm` + `kubeconform` pass on `litellm`, `puppet`
  overlays and the `paperclip` base (paperclip has no overlay yet).
- `pre-commit run` passes on all changed files.

## Follow-ups

- Longer term, the Barman Cloud Plugin (sidecar) would let minimal images keep
  their size while still archiving — track with the plugin migration.

https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
Reviewed-on: https://git.unkin.net/unkin/argocd-apps/pulls/304
Co-authored-by: Ben Vincent <ben@unkin.net>
Co-committed-by: Ben Vincent <ben@unkin.net>
---
 apps/base/litellm/cnpg_cluster.yaml   | 2 +-
 apps/base/paperclip/cnpg_cluster.yaml | 2 +-
 apps/base/puppet/cnpg_cluster.yaml    | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/apps/base/litellm/cnpg_cluster.yaml b/apps/base/litellm/cnpg_cluster.yaml
index 1148308..5a16b00 100644
--- a/apps/base/litellm/cnpg_cluster.yaml
+++ b/apps/base/litellm/cnpg_cluster.yaml
@@ -48,7 +48,7 @@ spec:
   enablePDB: true
   enableSuperuserAccess: false
   failoverDelay: 0
-  imageName: ghcr.io/cloudnative-pg/postgresql:17-minimal-trixie
+  imageName: ghcr.io/cloudnative-pg/postgresql:17-system-trixie
   instances: 3
   logLevel: info
   maxSyncReplicas: 0
diff --git a/apps/base/paperclip/cnpg_cluster.yaml b/apps/base/paperclip/cnpg_cluster.yaml
index 0703eae..221620e 100644
--- a/apps/base/paperclip/cnpg_cluster.yaml
+++ b/apps/base/paperclip/cnpg_cluster.yaml
@@ -48,7 +48,7 @@ spec:
   enablePDB: true
   enableSuperuserAccess: false
   failoverDelay: 0
-  imageName: ghcr.io/cloudnative-pg/postgresql:17-minimal-trixie
+  imageName: ghcr.io/cloudnative-pg/postgresql:17-system-trixie
   instances: 3
   logLevel: info
   maxSyncReplicas: 0
diff --git a/apps/base/puppet/cnpg_cluster.yaml b/apps/base/puppet/cnpg_cluster.yaml
index 7087ef1..c3099ab 100644
--- a/apps/base/puppet/cnpg_cluster.yaml
+++ b/apps/base/puppet/cnpg_cluster.yaml
@@ -69,7 +69,7 @@ spec:
   enablePDB: true
   enableSuperuserAccess: false
   failoverDelay: 0
-  imageName: ghcr.io/cloudnative-pg/postgresql:17-minimal-trixie
+  imageName: ghcr.io/cloudnative-pg/postgresql:17-system-trixie
   instances: 3
   logLevel: info
   maxSyncReplicas: 0