unkinben
6ac73a25e3
Live diagnosis of the still-broken stack found the real NATS auth bug. The server was rejecting ALL clients (bootstrap admin, and every vector tier) with "authorization violation" even though the pod env and the Vault secret matched: authenticating as log-admin with the LITERAL string "$NATS_ADMIN_PASSWORD" succeeded, proving the server stored the passwords un-expanded. The nats chart renders config.merge as JSON, so a plain `$VAR` becomes the quoted literal "$VAR" in nats.conf, and NATS does not expand variables inside quoted strings. Per the chart README, env vars must be wrapped in `<< $VAR >>` so they render UNQUOTED and NATS expands them. Wrap all three user passwords. This is the server-side half; #301 fixed the client side (Vector 0.57 needs VECTOR_DANGEROUSLY_ALLOW_ENV_VAR_INTERPOLATION to send the real password). Both are required: with both, server-expanded == vector-interpolated. Verified end-to-end (nats-server with unquoted $VAR + env, vector with the flag): admin auth ok, stream/consumer created, vector consumer connects. Once this syncs, the config-reloader reloads NATS with the real passwords, the nats-bootstrap PostSync hook succeeds (creating the LOGS stream + consumers), the sync completes and applies #301's vector env, and the aggregator/archiver recover on their next restart. Claude-Session: https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
argocd-apps docs
Operational notes for the manifests in this repo.
| Doc | What it covers |
|---|---|
| cnpg-backups.md | How CNPG Postgres backups (WAL archiving + nightly base backups) to Ceph RGW are configured. |
| cnpg-restore.md | Restoring a CNPG cluster: full recovery, point-in-time recovery, cutover, and gotchas. |