Two root causes found by diagnosing the live cluster after #296 deployed.
1. Vector 0.57 disables ${VAR} config interpolation by default (it now requires
--dangerously-allow-env-var-interpolation). So every ${...PASSWORD} in the
pipelines was sent as a literal string, breaking NATS auth for ALL vector
tiers: the aggregator/archiver crash-looped with "authorization violation",
and the agent/vm-ingest producers silently failed to publish (ClickHouse
creds would have failed the same way). Enable interpolation via
VECTOR_DANGEROUSLY_ALLOW_ENV_VAR_INTERPOLATION=true on all four vector
deployments. Verified end-to-end: without the flag -> authorization
violation; with it -> connects and authenticates (narrow ACL unchanged, so
the earlier ACL theory was wrong and nothing there needed changing).
2. The Altinity operator's watchNamespaces defaulted to [] which, per the chart,
makes it watch ONLY its own namespace (clickhouse-system). The
ClickHouseInstallation lives in `logging`, so it was never reconciled (no
finalizer, empty status) -> the logging-logging Argo sync blocked forever on
"waiting for healthy CHI" -> the PostSync hooks (nats-bootstrap stream/
consumers, clickhouse-schema) never ran. Set watchNamespaces: [logging] so
the operator reconciles the CHI, the sync completes, and the hooks run.
Claude-Session: https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
unkinben