unkinben
da4a66046a
Why: extend the Tier-1 survey with 7 more high-value log sources so they parse into logs.raw columns/.fields for real querying instead of the generic catch-all. **Stacks on #318 — merge after it.** How: - 7 mutually-exclusive app_route conditions + parse transforms into the ClickHouse sink: **bind_query** (k8s bind-* + VM named), **rancher_audit** (cattle-system sidecar JSON), **cnpg_pg** (ONE transform for all 10 CNPG clusters via the `.postgres` container), **gitea** (router+access, k8s+VM), **puppet** (openvoxserver/openvoxdb logback + access), **litellm** (JSON request logs), **postfix** (per-line maillog). - Carve `.postgres` out of the Tier-1 authentik route + new puppet/gitea/litellm routes so the single cnpg_pg route claims every CNPG pod without double-insert (keeps app_route mutually exclusive). Catch-all intact. - Companion k8s flips in this PR: litellm `JSON_LOGS=True`; bind `querylog yes` on both bind-internal BindClusters; gitea router+access logging to stdout. Rancher auditLog was already on. - 15 new `vector test` cases (routing + field extraction + authentik-postgres→cnpg exclusivity proof); all 35 green (vector 0.57). Fields go into the existing `fields Map(String,String)` — no DDL change. Puppet-side follow-ups (out of scope for argocd): enable named query logging (profiles/dns/server.pp); ship the VM vector rollout with `.file`/`.SYSLOG_IDENTIFIER` tags for named/gitea/puppetserver(+multiline logback join)/postfix maillog. https://claude.ai/code/session_015ur3i7D2azsMAWTSVABApv --------- Co-authored-by: Ben Vincent <neotheo@gmail.com> Reviewed-on: #320 Co-authored-by: Ben Vincent <ben@unkin.net> Co-committed-by: Ben Vincent <ben@unkin.net>
55 lines
1.8 KiB
YAML
55 lines
1.8 KiB
YAML
---
|
|
# Authoritative masters (replaces the 3x Puppet authoritative servers).
|
|
# pod-0 is the primary; pods 1-2 replicate via the catalog zone + AXFR/IXFR.
|
|
apiVersion: bind.unkin.net/v1alpha1
|
|
kind: BindCluster
|
|
metadata:
|
|
name: bind-authoritative
|
|
namespace: bind-internal
|
|
spec:
|
|
mode: authoritative
|
|
replicas: 3
|
|
storageClassName: cephrbd-fast-delete
|
|
storageSize: 2Gi
|
|
# Restrict queries to internal networks (puppet acl-main.unkin.net).
|
|
# 10.42.0.0/16 (pod net) is required so secondaries can SOA-refresh
|
|
# from the primary during catalog replication. localhost is required so the
|
|
# operator's in-pod `nsupdate` (sent to 127.0.0.1) passes query-authorization;
|
|
# without it every dynamic update is "denied due to allow-query".
|
|
extraOptions:
|
|
- "allow-query { localhost; auth-acl-main; 10.42.0.0/16; }"
|
|
# Enable query logging for the Tier-2 vector bind_query pipeline (see the
|
|
# resolvers cluster for the routing rationale).
|
|
- "querylog yes"
|
|
service:
|
|
type: LoadBalancer
|
|
externalTrafficPolicy: Local
|
|
annotations:
|
|
purelb.io/service-group: common
|
|
purelb.io/addresses: 198.18.200.6
|
|
external-dns.alpha.kubernetes.io/hostname: bind-authoritative.k8s.syd1.au.unkin.net
|
|
primaryService:
|
|
type: LoadBalancer
|
|
annotations:
|
|
purelb.io/service-group: common
|
|
purelb.io/addresses: 198.18.200.9
|
|
external-dns.alpha.kubernetes.io/hostname: bind-authoritative-primary.k8s.syd1.au.unkin.net
|
|
resources:
|
|
requests:
|
|
cpu: 20m
|
|
memory: 128Mi
|
|
limits:
|
|
cpu: "1"
|
|
memory: 512Mi
|
|
---
|
|
# Catalog zone so new BindZones auto-provision onto the secondaries.
|
|
apiVersion: bind.unkin.net/v1alpha1
|
|
kind: BindCatalogZone
|
|
metadata:
|
|
name: bind-authoritative-catalog
|
|
namespace: bind-internal
|
|
spec:
|
|
clusterRef: bind-authoritative
|
|
zoneName: catalog.internal
|
|
transferKeyRef: transfer-key
|