unkinben
1202aae06f
Three changes from review: 1. Pull every container image through the artifactapi dockerhub remote instead of direct upstream: clickhouse-server, altinity operator + metrics-exporter, bitnami/kubectl (crdHook), nats + nats-server-config-reloader, nats-box (bootstrap Job), and vector (all tiers + the CI image). Requires terraform-artifactapi#16 (dockerhub allowlist patterns) merged first. 2. Keep upstream official images (no Docker Hardened Images). DHI exists for clickhouse-server and vector but is subscription-gated and served from a private org namespace not reachable via the anonymous artifactapi dockerhub proxy; its shell-less images would also break the bash bootstrap Jobs and the shell-based vector-test CI step. Use vector's distroless-libc for runtime pods (near-hardened) and the debian variant only for CI. 3. Make the transform tier a stateless Deployment (was a StatefulSet): no PVC, no disk buffer — JetStream is the sole durability layer. The ClickHouse sink uses an in-memory block buffer so a ClickHouse outage back-pressures the JetStream pull source (unpulled messages are retained/redelivered). Add a CPU HPA (2-8) — safe because JetStream pull consumers distribute work across N replicas on the one durable consumer. Caveat documented: vector's NATS source has no end-to-end acks (acks on receipt), so a pod killed mid-outage can lose its in-memory buffer window; accepted trade for a stateless autoscaling tier. Claude-Session: https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
51 lines
1.3 KiB
YAML
51 lines
1.3 KiB
YAML
# Vector EDGE agent (DaemonSet) — thin publisher. Tails every node's pod logs
|
|
# (incl. control-plane via the blanket toleration) and publishes them into
|
|
# JetStream over the Vector NATS sink. No parsing; only a routing subject token
|
|
# is attached. Shaping happens in the transform tier after JetStream.
|
|
role: Agent
|
|
fullnameOverride: vector-agent
|
|
|
|
# Pulled through the artifactapi dockerhub remote; distroless-libc (no DHI —
|
|
# subscription-gated/private-namespace, not reachable via the anon proxy).
|
|
image:
|
|
repository: artifactapi.k8s.syd1.au.unkin.net/dockerhub/timberio/vector
|
|
tag: 0.57.0-distroless-libc
|
|
|
|
rbac:
|
|
create: true
|
|
serviceAccount:
|
|
create: true
|
|
|
|
podLabels:
|
|
vector.dev/exclude: "true"
|
|
|
|
tolerations:
|
|
- operator: Exists
|
|
|
|
env:
|
|
- name: NATS_PRODUCER_PASSWORD
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: nats-auth
|
|
key: producer_password
|
|
|
|
resources:
|
|
requests:
|
|
cpu: 100m
|
|
memory: 256Mi
|
|
limits:
|
|
cpu: "1"
|
|
memory: 1Gi
|
|
|
|
service:
|
|
enabled: false
|
|
|
|
# Pipeline is the single source of truth in apps/base/logging/vector/agent.yaml,
|
|
# mounted via existingConfigMaps (avoids the chart's customConfig Helm-tpl pass).
|
|
dataDir: /vector-data-dir
|
|
existingConfigMaps:
|
|
- vector-agent-config
|
|
|
|
workloadResourceAnnotations:
|
|
reloader.stakater.com/auto: "true"
|