unkin/puppet-prod
4
0
Fork 0
Code Issues 15 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

VM-side Vector rollout: ship host logs to the central logging stack #502

New Issue
Open
opened 2026-08-02 16:24:31 +10:00 by unkinben · 0 comments
unkinben commented 2026-08-02 16:24:31 +10:00
Owner
Copy Link
Copy Source

Problem
The centralized logging platform (Vector->JetStream->ClickHouse) is live for k8s, and Vector transforms for VM apps (nginx, HAProxy, glauth, BIND, puppetserver, postfix, Vault file-audit) are implemented but dormant — no VM ships logs yet, and the transforms match on .file/.SYSLOG_IDENTIFIER source tags the VMs must set.

Proposal

  • add a puppet profile deploying edge Vector on VMs, shipping journald + tailed files to https://logs-ingest.k8s.syd1.au.unkin.net, publishing logs.vm. subjects
  • set .file (file sources) and .SYSLOG_IDENTIFIER (journald) tags for named/nginx/haproxy/glauth/postfix/puppetserver/vault_audit
  • puppetserver multiline logback stacktrace join at the edge; tail puppetserver-access.log

Acceptance

  • VM logs land in logs.raw under logs.vm.*; the dormant transforms populate structured fields; Vault file-audit reaches the logarchiver archive subject.
**Problem** The centralized logging platform (Vector->JetStream->ClickHouse) is live for k8s, and Vector transforms for VM apps (nginx, HAProxy, glauth, BIND, puppetserver, postfix, Vault file-audit) are implemented but dormant — no VM ships logs yet, and the transforms match on .file/.SYSLOG_IDENTIFIER source tags the VMs must set. **Proposal** - add a puppet profile deploying edge Vector on VMs, shipping journald + tailed files to https://logs-ingest.k8s.syd1.au.unkin.net, publishing logs.vm.<host> subjects - set .file (file sources) and .SYSLOG_IDENTIFIER (journald) tags for named/nginx/haproxy/glauth/postfix/puppetserver/vault_audit - puppetserver multiline logback stacktrace join at the edge; tail puppetserver-access.log **Acceptance** - VM logs land in logs.raw under logs.vm.*; the dormant transforms populate structured fields; Vault file-audit reaches the logarchiver archive subject.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
benvin (BenVincent) unkin-agent (Unkin Agent) unkinben (Ben Vincent)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: unkin/puppet-prod#502
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?