unkinben
1202aae06f
Three changes from review: 1. Pull every container image through the artifactapi dockerhub remote instead of direct upstream: clickhouse-server, altinity operator + metrics-exporter, bitnami/kubectl (crdHook), nats + nats-server-config-reloader, nats-box (bootstrap Job), and vector (all tiers + the CI image). Requires terraform-artifactapi#16 (dockerhub allowlist patterns) merged first. 2. Keep upstream official images (no Docker Hardened Images). DHI exists for clickhouse-server and vector but is subscription-gated and served from a private org namespace not reachable via the anonymous artifactapi dockerhub proxy; its shell-less images would also break the bash bootstrap Jobs and the shell-based vector-test CI step. Use vector's distroless-libc for runtime pods (near-hardened) and the debian variant only for CI. 3. Make the transform tier a stateless Deployment (was a StatefulSet): no PVC, no disk buffer — JetStream is the sole durability layer. The ClickHouse sink uses an in-memory block buffer so a ClickHouse outage back-pressures the JetStream pull source (unpulled messages are retained/redelivered). Add a CPU HPA (2-8) — safe because JetStream pull consumers distribute work across N replicas on the one durable consumer. Caveat documented: vector's NATS source has no end-to-end acks (acks on receipt), so a pod killed mid-outage can lose its in-memory buffer window; accepted trade for a stateless autoscaling tier. Claude-Session: https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv
57 lines
1.1 KiB
YAML
57 lines
1.1 KiB
YAML
# Vector VM-INGEST tier (Deployment) — VM front door. HTTP NDJSON in (behind the
|
|
# logs-ingest Gateway), publishes into JetStream. Stateless publisher.
|
|
# Pipeline: apps/base/logging/vector/vm-ingest.yaml (unit-tested in CI).
|
|
role: Stateless-Aggregator
|
|
fullnameOverride: vector-vm-ingest
|
|
replicas: 2
|
|
|
|
image:
|
|
repository: artifactapi.k8s.syd1.au.unkin.net/dockerhub/timberio/vector
|
|
tag: 0.57.0-distroless-libc
|
|
|
|
workloadResourceAnnotations:
|
|
reloader.stakater.com/auto: "true"
|
|
|
|
podLabels:
|
|
vector.dev/exclude: "true"
|
|
|
|
dataDir: /vector-data-dir
|
|
existingConfigMaps:
|
|
- vector-vm-ingest-config
|
|
|
|
env:
|
|
- name: NATS_PRODUCER_PASSWORD
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: nats-auth
|
|
key: producer_password
|
|
|
|
containerPorts:
|
|
- name: http-ingest
|
|
containerPort: 8080
|
|
protocol: TCP
|
|
- name: api
|
|
containerPort: 8686
|
|
protocol: TCP
|
|
|
|
service:
|
|
enabled: true
|
|
type: ClusterIP
|
|
ports:
|
|
- name: http-ingest
|
|
port: 8080
|
|
targetPort: 8080
|
|
protocol: TCP
|
|
- name: api
|
|
port: 8686
|
|
targetPort: 8686
|
|
protocol: TCP
|
|
|
|
resources:
|
|
requests:
|
|
cpu: 100m
|
|
memory: 256Mi
|
|
limits:
|
|
cpu: "1"
|
|
memory: 1Gi
|