unkinben
ab76fb8420
Two root causes found by diagnosing the live cluster after #296 deployed. 1. Vector 0.57 disables ${VAR} config interpolation by default (it now requires --dangerously-allow-env-var-interpolation). So every ${...PASSWORD} in the pipelines was sent as a literal string, breaking NATS auth for ALL vector tiers: the aggregator/archiver crash-looped with "authorization violation", and the agent/vm-ingest producers silently failed to publish (ClickHouse creds would have failed the same way). Enable interpolation via VECTOR_DANGEROUSLY_ALLOW_ENV_VAR_INTERPOLATION=true on all four vector deployments. Verified end-to-end: without the flag -> authorization violation; with it -> connects and authenticates (narrow ACL unchanged, so the earlier ACL theory was wrong and nothing there needed changing). 2. The Altinity operator's watchNamespaces defaulted to [] which, per the chart, makes it watch ONLY its own namespace (clickhouse-system). The ClickHouseInstallation lives in `logging`, so it was never reconciled (no finalizer, empty status) -> the logging-logging Argo sync blocked forever on "waiting for healthy CHI" -> the PostSync hooks (nats-bootstrap stream/ consumers, clickhouse-schema) never ran. Set watchNamespaces: [logging] so the operator reconciles the CHI, the sync completes, and the hooks run. Claude-Session: https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
argocd-apps docs
Operational notes for the manifests in this repo.
| Doc | What it covers |
|---|---|
| cnpg-backups.md | How CNPG Postgres backups (WAL archiving + nightly base backups) to Ceph RGW are configured. |
| cnpg-restore.md | Restoring a CNPG cluster: full recovery, point-in-time recovery, cutover, and gotchas. |