certificates: restore the validly-signed intermediate in vault-ca-cert #305

Merged
benvin merged 1 commits from benvin/vault-ca-cert-refresh into main 2026-07-28 23:14:15 +10:00
Owner

CNPG WAL archiving to Ceph RGW fails with CERTIFICATE_VERIFY_FAILED on six clusters because the reflected vault-ca-cert bundle carries a corrupt intermediate: the genuinely-signed cert has a typo'd AIA URL (vault.servuce.consul), and the committed copy was text-edited at some point to fix the typo — flipping one byte of signed data and invalidating the signature (openssl verify: error 7 certificate signature failure). Only radosgw surfaces it because it serves a bare leaf, forcing clients to verify the stored intermediate against the root; services presenting their own intermediate never exercised the corrupt copy. terraform-k8s's copy is defunct per Ben — this file is the authoritative source.

  • restore the original signed intermediate (one base64 character; sha256 E0:13:1B..., verified against the root, and the resulting bundle validates the live s3.ceph.unkin.net leaf)
  • add an explicit allow-plain-secret marker mechanism to ci/validate-no-secrets.sh for public-data bootstrap secrets, and mark vault-ca-cert.yaml with it (a CA bundle is public and cannot be Vault-sourced since it establishes Vault trust)

After merge+sync the reflector propagates to all namespaces and barman's next retry (~1min) succeeds; base backups run on tonight's schedule. Follow-ups worth considering: re-issue the intermediate in Vault with a corrected AIA URL, and/or configure radosgw to serve its intermediate.

CNPG WAL archiving to Ceph RGW fails with CERTIFICATE_VERIFY_FAILED on six clusters because the reflected vault-ca-cert bundle carries a corrupt intermediate: the genuinely-signed cert has a typo'd AIA URL (vault.servuce.consul), and the committed copy was text-edited at some point to fix the typo — flipping one byte of signed data and invalidating the signature (openssl verify: error 7 certificate signature failure). Only radosgw surfaces it because it serves a bare leaf, forcing clients to verify the stored intermediate against the root; services presenting their own intermediate never exercised the corrupt copy. terraform-k8s's copy is defunct per Ben — this file is the authoritative source. - restore the original signed intermediate (one base64 character; sha256 E0:13:1B..., verified against the root, and the resulting bundle validates the live s3.ceph.unkin.net leaf) - add an explicit allow-plain-secret marker mechanism to ci/validate-no-secrets.sh for public-data bootstrap secrets, and mark vault-ca-cert.yaml with it (a CA bundle is public and cannot be Vault-sourced since it establishes Vault trust) After merge+sync the reflector propagates to all namespaces and barman's next retry (~1min) succeeds; base backups run on tonight's schedule. Follow-ups worth considering: re-issue the intermediate in Vault with a corrected AIA URL, and/or configure radosgw to serve its intermediate.
unkinben added 1 commit 2026-07-28 23:02:11 +10:00
certificates: restore the validly-signed intermediate in vault-ca-cert
ci/woodpecker/pr/vector-test Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/pr/kubeconform Pipeline was successful
08a75ea01a
Claude-Session: https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
benvin merged commit 57691ef1d5 into main 2026-07-28 23:14:15 +10:00
benvin deleted branch benvin/vault-ca-cert-refresh 2026-07-28 23:14:15 +10:00
Sign in to join this conversation.
No Reviewers
No Label
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: unkin/argocd-apps#305