unkin-agent
d1085f0ae2
rke2's `registries.yaml` already rewrites upstream image names to the artifactapi mirror, so manifests must carry canonical upstream names. Only in-house `artifactapi.k8s.syd1.au.unkin.net/docker-internal/...` images stay explicit (logarchiver/logviewer are untouched). Changes: - Point the ClickHouseInstallation and the clickhouse-schema job at `docker.io/clickhouse/clickhouse-server:24.8`. - Point the logviewer oauth2-proxy cert-combine init container at `docker.io/library/alpine:3`. - Point the NATS bootstrap job at `docker.io/natsio/nats-box:0.18.0`. - Point the NATS chart values at `docker.io/library/nats` and `docker.io/natsio/nats-server-config-reloader`. - Point all three Vector values files (agent, aggregator, vm-ingest) at `docker.io/timberio/vector`. - Drop the now-wrong "pulled through the artifactapi dockerhub remote" comments in the NATS and vector-agent values. Tags/digests unchanged and the `repository`/`tag` split is preserved. `kustomize build --enable-helm apps/overlays/au-syd1/logging` differs from main only in those nine image strings. Extra found, not changed here: `.woodpecker/vector-test.yaml` still pins its CI step image to `artifactapi.k8s.syd1.au.unkin.net/dockerhub/timberio/vector:0.57.0-debian`. That is a Woodpecker step image rather than a namespace manifest, so it is left out to keep this PR to the logging namespace — say the word and I will fix it separately. Reviewed-on: #433 Co-authored-by: unkin-agent <unkin-agent@unkin.net> Co-committed-by: unkin-agent <unkin-agent@unkin.net>
85 lines
2.6 KiB
YAML
85 lines
2.6 KiB
YAML
# Vector TRANSFORM tier (STATELESS Deployment) — the "brain": sole ClickHouse
|
|
# writer, owns all transforms, holds the only ClickHouse + NATS-consumer creds.
|
|
#
|
|
# Stateless by design: a JetStream pull consumer with NO PVC and NO disk buffer.
|
|
# JetStream is the sole durability layer. On a ClickHouse outage the clickhouse
|
|
# sink blocks (buffer when_full=block), back-pressure stops the source pulling,
|
|
# and unpulled messages stay in JetStream for redelivery. Because Vector's NATS
|
|
# source does NOT support end-to-end acknowledgements (it acks on receipt, not
|
|
# after the sink), the only at-risk window is the in-memory buffer's worth of
|
|
# already-pulled events if a pod is killed mid-outage — the accepted trade for a
|
|
# horizontally-autoscalable stateless tier. Multiple replicas share the one
|
|
# durable consumer `transform` (JetStream pull consumers distribute work), so
|
|
# HPA is safe.
|
|
role: Stateless-Aggregator
|
|
fullnameOverride: vector-aggregator
|
|
|
|
image:
|
|
repository: docker.io/timberio/vector
|
|
tag: 0.57.0-distroless-libc
|
|
|
|
# Horizontal autoscaling on CPU — safe with N replicas on one durable consumer.
|
|
autoscaling:
|
|
enabled: true
|
|
minReplicas: 2
|
|
maxReplicas: 8
|
|
targetCPUUtilizationPercentage: 70
|
|
|
|
workloadResourceAnnotations:
|
|
configmap.reloader.stakater.com/auto: "true"
|
|
|
|
podLabels:
|
|
vector.dev/exclude: "true"
|
|
|
|
# Pipeline is the single source of truth in apps/base/logging/vector/
|
|
# aggregator.yaml (unit-tested by `vector test` in CI), mounted via
|
|
# existingConfigMaps. No persistence — stateless.
|
|
dataDir: /vector-data-dir
|
|
existingConfigMaps:
|
|
- vector-aggregator-config
|
|
|
|
# The ONLY place ClickHouse + NATS-consumer creds are consumed.
|
|
env:
|
|
# Vector 0.57 disables ${VAR} config interpolation by default; the pipeline's
|
|
# auth (${CLICKHOUSE_*}, ${NATS_CONSUMER_PASSWORD}) needs it enabled.
|
|
- name: VECTOR_DANGEROUSLY_ALLOW_ENV_VAR_INTERPOLATION
|
|
value: "true"
|
|
- name: CLICKHOUSE_USER
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: clickhouse-credentials
|
|
key: username
|
|
- name: CLICKHOUSE_PASSWORD
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: clickhouse-credentials
|
|
key: password
|
|
- name: NATS_CONSUMER_PASSWORD
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: nats-auth
|
|
key: consumer_password
|
|
|
|
# Pure consumer: expose only the Vector API for debugging.
|
|
containerPorts:
|
|
- name: api
|
|
containerPort: 8686
|
|
protocol: TCP
|
|
|
|
service:
|
|
enabled: true
|
|
type: ClusterIP
|
|
ports:
|
|
- name: api
|
|
port: 8686
|
|
targetPort: 8686
|
|
protocol: TCP
|
|
|
|
resources:
|
|
requests:
|
|
cpu: 250m
|
|
memory: 512Mi
|
|
limits:
|
|
cpu: "2"
|
|
memory: 2Gi
|