unkin-agent
757ae5b240
The k8s log pipeline stores to ClickHouse via NATS+vector, while the VM estate ships journald to a separate puppet-managed VictoriaLogs cluster. Consolidating on VictoriaLogs in-cluster collapses the two paths, and the logs-ingest gateway has no clients yet so it can be repointed now, ahead of the puppet change. - add VLCluster `logs` at v1.52.0 (2 vlinsert, 2 vlselect, 3 vlstorage, 180d retention, 250Gi each on cephrbd-fast-delete) - cap vlstorage disk use at 220GiB per node so 180d stays time-based rather than disk-bound - repoint the logs-ingest HTTPRoute at `vlinsert-logs:9481` - add a VictoriaLogs Grafana datasource and install its plugin Nothing is removed here; NATS, ClickHouse, vector, logarchiver and logviewer keep running until a follow-up drops them. Reviewed-on: #488 Co-authored-by: unkin-agent <unkin-agent@unkin.net> Co-committed-by: unkin-agent <unkin-agent@unkin.net>
42 lines
1.2 KiB
YAML
42 lines
1.2 KiB
YAML
---
|
|
# Log ingestion endpoint for puppet-managed VMs (and any non-k8s client):
|
|
# fronts the VLCluster vlinsert service over TLS at a name VMs can resolve.
|
|
apiVersion: gateway.networking.k8s.io/v1
|
|
kind: Gateway
|
|
metadata:
|
|
name: logs-ingest
|
|
namespace: logging
|
|
labels:
|
|
app.kubernetes.io/name: victorialogs
|
|
app.kubernetes.io/component: ingest
|
|
traefik.io/instance: internal
|
|
annotations:
|
|
cert-manager.io/cluster-issuer: vault-issuer
|
|
cert-manager.io/common-name: logs-ingest.k8s.syd1.au.unkin.net
|
|
cert-manager.io/private-key-size: "4096"
|
|
external-dns.alpha.kubernetes.io/hostname: logs-ingest.k8s.syd1.au.unkin.net
|
|
external-dns.alpha.kubernetes.io/target: 198.18.200.4
|
|
spec:
|
|
gatewayClassName: traefik-internal
|
|
listeners:
|
|
- name: http
|
|
port: 80
|
|
protocol: HTTP
|
|
hostname: logs-ingest.k8s.syd1.au.unkin.net
|
|
allowedRoutes:
|
|
namespaces:
|
|
from: Same
|
|
- name: https
|
|
port: 443
|
|
protocol: HTTPS
|
|
hostname: logs-ingest.k8s.syd1.au.unkin.net
|
|
allowedRoutes:
|
|
namespaces:
|
|
from: Same
|
|
tls:
|
|
mode: Terminate
|
|
certificateRefs:
|
|
- group: ""
|
|
kind: Secret
|
|
name: logs-ingest-tls
|