unkinben
da4a66046a
Why: extend the Tier-1 survey with 7 more high-value log sources so they parse into logs.raw columns/.fields for real querying instead of the generic catch-all. **Stacks on #318 — merge after it.** How: - 7 mutually-exclusive app_route conditions + parse transforms into the ClickHouse sink: **bind_query** (k8s bind-* + VM named), **rancher_audit** (cattle-system sidecar JSON), **cnpg_pg** (ONE transform for all 10 CNPG clusters via the `.postgres` container), **gitea** (router+access, k8s+VM), **puppet** (openvoxserver/openvoxdb logback + access), **litellm** (JSON request logs), **postfix** (per-line maillog). - Carve `.postgres` out of the Tier-1 authentik route + new puppet/gitea/litellm routes so the single cnpg_pg route claims every CNPG pod without double-insert (keeps app_route mutually exclusive). Catch-all intact. - Companion k8s flips in this PR: litellm `JSON_LOGS=True`; bind `querylog yes` on both bind-internal BindClusters; gitea router+access logging to stdout. Rancher auditLog was already on. - 15 new `vector test` cases (routing + field extraction + authentik-postgres→cnpg exclusivity proof); all 35 green (vector 0.57). Fields go into the existing `fields Map(String,String)` — no DDL change. Puppet-side follow-ups (out of scope for argocd): enable named query logging (profiles/dns/server.pp); ship the VM vector rollout with `.file`/`.SYSLOG_IDENTIFIER` tags for named/gitea/puppetserver(+multiline logback join)/postfix maillog. https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv --------- Co-authored-by: Ben Vincent <neotheo@gmail.com> Reviewed-on: #320 Co-authored-by: Ben Vincent <ben@unkin.net> Co-committed-by: Ben Vincent <ben@unkin.net>
45 lines
1.7 KiB
YAML
45 lines
1.7 KiB
YAML
---
|
|
# Recursive resolvers (replaces the 3x Puppet only-resolver servers).
|
|
# Three identical recursive servers; no zone replication.
|
|
apiVersion: bind.unkin.net/v1alpha1
|
|
kind: BindCluster
|
|
metadata:
|
|
name: bind-resolvers
|
|
namespace: bind-internal
|
|
spec:
|
|
mode: resolver
|
|
replicas: 3
|
|
storageClassName: cephrbd-fast-delete
|
|
storageSize: 1Gi
|
|
service:
|
|
type: LoadBalancer
|
|
externalTrafficPolicy: Local
|
|
annotations:
|
|
purelb.io/service-group: common
|
|
purelb.io/addresses: 198.18.200.7
|
|
external-dns.alpha.kubernetes.io/hostname: bind-resolvers.k8s.syd1.au.unkin.net
|
|
forwarders:
|
|
- 8.8.8.8
|
|
- 1.1.1.1
|
|
# The internal split-horizon zones are served UNSIGNED by the in-cluster
|
|
# authoritative, but their public parents publish DS records (e.g. unkin.net
|
|
# is DNSSEC-signed on the Internet). With dnssec-validation on, the validator
|
|
# sees "parent indicates secure" but gets an insecure answer and returns
|
|
# SERVFAIL (broken trust chain). Treat the forwarded internal domains as
|
|
# insecure so they are not validated. unkin.net covers all *.unkin.net
|
|
# (incl. k8s.syd1.au.unkin.net); 18.198.in-addr.arpa covers every reverse zone.
|
|
extraOptions:
|
|
- "validate-except { unkin.net; 18.198.in-addr.arpa; consul; }"
|
|
# Enable query logging so the Tier-2 vector bind_query pipeline can parse
|
|
# client/qname/qtype. Routes to the `queries` category which, with no explicit
|
|
# logging{} clause, follows the default category to the named foreground
|
|
# stderr channel -> pod stdout -> vector (subject logs.k8s.bind-internal.*).
|
|
- "querylog yes"
|
|
resources:
|
|
requests:
|
|
cpu: 20m
|
|
memory: 128Mi
|
|
limits:
|
|
cpu: "1"
|
|
memory: 512Mi
|