unkinben
da4a66046a
Why: extend the Tier-1 survey with 7 more high-value log sources so they parse into logs.raw columns/.fields for real querying instead of the generic catch-all. **Stacks on #318 — merge after it.** How: - 7 mutually-exclusive app_route conditions + parse transforms into the ClickHouse sink: **bind_query** (k8s bind-* + VM named), **rancher_audit** (cattle-system sidecar JSON), **cnpg_pg** (ONE transform for all 10 CNPG clusters via the `.postgres` container), **gitea** (router+access, k8s+VM), **puppet** (openvoxserver/openvoxdb logback + access), **litellm** (JSON request logs), **postfix** (per-line maillog). - Carve `.postgres` out of the Tier-1 authentik route + new puppet/gitea/litellm routes so the single cnpg_pg route claims every CNPG pod without double-insert (keeps app_route mutually exclusive). Catch-all intact. - Companion k8s flips in this PR: litellm `JSON_LOGS=True`; bind `querylog yes` on both bind-internal BindClusters; gitea router+access logging to stdout. Rancher auditLog was already on. - 15 new `vector test` cases (routing + field extraction + authentik-postgres→cnpg exclusivity proof); all 35 green (vector 0.57). Fields go into the existing `fields Map(String,String)` — no DDL change. Puppet-side follow-ups (out of scope for argocd): enable named query logging (profiles/dns/server.pp); ship the VM vector rollout with `.file`/`.SYSLOG_IDENTIFIER` tags for named/gitea/puppetserver(+multiline logback join)/postfix maillog. https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv --------- Co-authored-by: Ben Vincent <neotheo@gmail.com> Reviewed-on: #320 Co-authored-by: Ben Vincent <ben@unkin.net> Co-committed-by: Ben Vincent <ben@unkin.net>
51 lines
1.8 KiB
YAML
51 lines
1.8 KiB
YAML
---
|
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
kind: Kustomization
|
|
|
|
resources:
|
|
- cnpg_cluster.yaml
|
|
- cnpg_backup.yaml
|
|
- cnpg_pooler.yaml
|
|
- deployment.yaml
|
|
- hpa.yaml
|
|
- gateway.yaml
|
|
- httproute.yaml
|
|
- namespace.yaml
|
|
- redis-deployment.yaml
|
|
- redis-pvc.yaml
|
|
- services.yaml
|
|
- vaultauth.yaml
|
|
- vaultstaticsecret.yaml
|
|
- vpa.yaml
|
|
|
|
configMapGenerator:
|
|
- name: litellm-config
|
|
files:
|
|
- config.yaml=resources/config.yaml
|
|
options:
|
|
disableNameSuffixHash: true
|
|
- name: litellm-env
|
|
literals:
|
|
- STORE_MODEL_IN_DB=True
|
|
# Emit structured JSON logs so the Tier-2 vector litellm pipeline can parse
|
|
# model/tokens/latency/key/status (logs.k8s.litellm.*).
|
|
- JSON_LOGS=True
|
|
# Authentik OIDC SSO (generic). Client secret is injected from the
|
|
# oauth-credentials Secret in the Deployment; endpoints match the other
|
|
# apps (identity.unkin.net). PROXY_BASE_URL is required for SSO.
|
|
- GENERIC_CLIENT_ID=litellm
|
|
- GENERIC_AUTHORIZATION_ENDPOINT=https://identity.unkin.net/application/o/authorize/
|
|
- GENERIC_TOKEN_ENDPOINT=https://identity.unkin.net/application/o/token/
|
|
- GENERIC_USERINFO_ENDPOINT=https://identity.unkin.net/application/o/userinfo/
|
|
# litellm_role scope carries the Authentik-computed role claim; LiteLLM
|
|
# reads it via GENERIC_USER_ROLE_ATTRIBUTE and maps to proxy_admin/etc.
|
|
- GENERIC_SCOPE=openid email profile litellm_role
|
|
- GENERIC_USER_ROLE_ATTRIBUTE=litellm_role
|
|
- PROXY_BASE_URL=https://litellm.k8s.syd1.au.unkin.net
|
|
# Trust the internal unkin.net CA (identity.unkin.net) via the combined
|
|
# bundle assembled by the combine-certs init container.
|
|
- SSL_CERT_FILE=/etc/ssl/combined/ca-certificates.crt
|
|
- REQUESTS_CA_BUNDLE=/etc/ssl/combined/ca-certificates.crt
|
|
options:
|
|
disableNameSuffixHash: true
|