observability: scrape existing app metrics (audit + ceph-csi batch) (#239)
## Why Audit of existing helm deployments for (a) metrics collection and (b) bundled Grafana dashboards, per request. This PR enables the first, cleanest batch (ceph-csi) and documents the rest for follow-up review. ## Key constraint The cluster has **no Prometheus-Operator CRDs** (`servicemonitors.monitoring.coreos.com` absent) — only the VM-native `VMServiceScrape`/`VMPodScrape`. So flipping charts' `serviceMonitor: true` toggles would fail. The correct mechanism here is native **VMServiceScrape** CRs, which the observability VMAgent picks up (`selectAllByDefault`, added in #234). Bundled chart dashboards ship as ConfigMaps that grafana-operator v5 ignores (it uses CRs), so "enabling" them means adding `GrafanaDashboard` CRs. ## This PR - `VMServiceScrape` for **csi-cephfs** and **csi-cephrbd** (their `*-http-metrics` services already exist). ## Audit — metrics (apps already exposing a metrics service; enable via VMServiceScrape) | App | Metrics service / port | Note | |-----|------------------------|------| | csi-cephfs / csi-cephrbd | `*-http-metrics` :8080 | **done here** | | cert-manager | webhook :9402 `metrics`, cainjector :9402 `http-metrics` | multi-port; easy follow-up | | argocd | `argocd-*-metrics` :8082/8083/8084/9001 | many components | | media-apps | `prowlarr/radarr/sonarr-metrics` :9707 (exportarr) | follow-up | | vm-system / grafana-operator | operator metrics | low value | ## Audit — charts with metrics *toggles* (need endpoint enabled + VMServiceScrape) authentik, vault, purelb, externaldns, cnpg-system (`enablePodMonitor`), reloader, traefik. These expose `serviceMonitor`/`podMonitor` toggles that assume Prometheus CRDs — recommend enabling the metrics endpoint + a VMServiceScrape rather than the toggle. ## Audit — bundled Grafana dashboards - **CNPG**: ships a dashboard (grafana.com id 20417) — add as `GrafanaDashboard` (grafanaCom). - **traefik**: `dashboard: true` option + community dashboards — add as `GrafanaDashboard`. - Others: none bundled. ## Review ask Confirm the VMServiceScrape approach; I'll extend to the remaining apps (cert-manager/argocd/media-apps + CNPG/traefik dashboards) in follow-up batches. Reviewed-on: #239 Co-authored-by: Ben Vincent <ben@unkin.net> Co-committed-by: Ben Vincent <ben@unkin.net>
This commit was merged in pull request #239.
This commit is contained in:
@@ -7,3 +7,4 @@ resources:
|
||||
- vaultauth.yaml
|
||||
- vaultstaticsecret.yaml
|
||||
- storageclass.yaml
|
||||
- vmservicescrape.yaml
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
---
|
||||
# Scrape the ceph-csi-cephfs nodeplugin + provisioner http-metrics endpoints.
|
||||
# Picked up by the observability VMAgent (selectAllByDefault).
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMServiceScrape
|
||||
metadata:
|
||||
name: ceph-csi-cephfs
|
||||
namespace: csi-cephfs
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
app: ceph-csi-cephfs
|
||||
endpoints:
|
||||
- port: http-metrics
|
||||
path: /metrics
|
||||
Reference in New Issue
Block a user