unkinben
6c8c0dd9e4
## Why Deploy the jellyfin-ha fork as a genuine high-availability service rather than a single replica, so its two headline capabilities can actually be exercised: the Redis-coordinated distributed transcoding (a surviving pod resumes the HLS segments of a pod that dies mid-stream) and the experimental PostgreSQL main database (which is what lets more than one replica share the same library). It lands in its own `jellyfin` namespace under a new `media` ArgoCD project. ## How **Workload — StatefulSet, 2 replicas.** The Deployment becomes a StatefulSet so each pod has a stable name. The fork's Redis transcode lease keys ownership on `JELLYFIN_INSTANCE_ID`, which is set from the downward-API pod name, giving each replica a unique, stable lease identity. Soft `podAntiAffinity` spreads the two pods across nodes and a `PodDisruptionBudget` keeps `minAvailable: 1` through drains and rollouts. **Main database — CloudNativePG.** A CNPG trio in-namespace mirrors the litellm pattern: a 3-instance `Cluster`, a PgBouncer `Pooler`, and Ceph RGW (barman) S3 backups to a dedicated `cnpg-jellyfin` bucket owned by a `cephrgw` `ObjectStoreUser`. An init container writes `/config/config/database.xml` selecting the fork's `Jellyfin-PostgreSQL` provider, and the connection string is composed from the CNPG-generated `jellyfin-postgres-app` secret (username / password / dbname) pointed at the pooler service — the password is never rendered into the manifest. Library-item metadata therefore moves off SQLite; metadata images, plugins, subtitles and config XML stay on `/config`. **Storage.** - `/config` is now a shared `ReadWriteMany` cephfs PVC (raid5, retain) so every replica reads/writes the same metadata and config. - `/config/transcodes` — the fork's real transcode temp path — is a shared RWX PVC (raid5, delete). This is the load-bearing fix: takeover reads the dead pod's in-flight `.ts`/`.m3u8` segments off shared storage, so per-pod scratch would silently break it. - `/cache` is per-pod via a `volumeClaimTemplate` (RWO). - The media library stays a fresh, empty RWX PVC mounted read-only; populating it is out of scope. **Hardware transcoding.** The container requests the `gpu.intel.com/i915` Intel device-plugin resource (which pins the pod to a GPU-labelled node and injects the DRI render node — no `/dev/dri` hostPath or privileged container) plus the render/video supplemental groups. VA-API hardware acceleration is now on by default: the `inject-config` init container seeds `/config/config/encoding.xml` with `HardwareAccelerationType` `vaapi`, `EnableHardwareEncoding`, the injected render node (`/dev/dri/renderD128`) and h264/hevc hardware decode, so transcodes use the iGPU on first boot with no manual admin-UI step. Both seed files (`database.xml`, `encoding.xml`) are written only when absent, so later admin changes persisted to the shared RWX `/config` are never clobbered on restart. **Networking.** The Gateway/HTTPRoute (traefik-internal, `jellyfin.k8s.syd1.au.unkin.net`) is unchanged; the Service gains `sessionAffinity: ClientIP` to keep a client pinned to one replica and reduce transcode-session churn. **Redis.** The in-namespace single-replica Redis stays as the transcode lease store. ## Follow-up UDP auto-discovery is disabled, but scheduled library scans still run on every replica (redundant scans). Single-scanner leader election is a planned follow-up pending a fork feature, tracked separately. --------- Co-authored-by: unkin-agent <unkin-agent@unkin.net> Co-authored-by: Ben Vincent <neotheo@gmail.com> Co-authored-by: Ben Vin <neotheo@gmail.com> Reviewed-on: #237 Co-authored-by: Ben Vincent <ben@unkin.net> Co-committed-by: Ben Vincent <ben@unkin.net>
127 lines
3.6 KiB
YAML
127 lines
3.6 KiB
YAML
---
|
|
# Main Jellyfin database. The jellyfin-ha fork's experimental EF Core provider
|
|
# moves the entire Jellyfin DB (incl. library items) off SQLite into PostgreSQL,
|
|
# which is what makes a shared-nothing multi-replica deployment possible. No
|
|
# bootstrap secret is given, so CNPG generates the jellyfin-postgres-app secret
|
|
# (username/password/dbname) that the StatefulSet composes its DSN from.
|
|
apiVersion: postgresql.cnpg.io/v1
|
|
kind: Cluster
|
|
metadata:
|
|
name: jellyfin-postgres
|
|
namespace: jellyfin
|
|
spec:
|
|
# Exclude the operator-managed data PVCs (jellyfin-postgres-N) from the
|
|
# jellyfin-config k8up Schedule (skipWithoutAnnotation is false cluster-wide,
|
|
# so unannotated PVCs are swept in). Postgres has its own barmanObjectStore
|
|
# backup below; restic must not touch the raw RWO data volumes.
|
|
inheritedMetadata:
|
|
annotations:
|
|
k8up.io/backup: "false"
|
|
affinity:
|
|
podAntiAffinityType: preferred
|
|
backup:
|
|
retentionPolicy: 30d
|
|
barmanObjectStore:
|
|
# Dedicated per-cluster Ceph RGW bucket (cephrgw-operator provisions it).
|
|
destinationPath: s3://cnpg-jellyfin
|
|
endpointURL: https://s3.ceph.unkin.net
|
|
endpointCA:
|
|
name: vault-ca-cert
|
|
key: ca.crt
|
|
s3Credentials:
|
|
accessKeyId:
|
|
name: cnpg-jellyfin-backup-s3
|
|
key: AWS_ACCESS_KEY_ID
|
|
secretAccessKey:
|
|
name: cnpg-jellyfin-backup-s3
|
|
key: AWS_SECRET_ACCESS_KEY
|
|
serverName: jellyfin
|
|
data:
|
|
compression: bzip2
|
|
jobs: 2
|
|
wal:
|
|
compression: zstd
|
|
maxParallel: 2
|
|
bootstrap:
|
|
initdb:
|
|
database: jellyfin
|
|
encoding: UTF8
|
|
localeCType: C
|
|
localeCollate: C
|
|
owner: jellyfin
|
|
enablePDB: true
|
|
enableSuperuserAccess: false
|
|
failoverDelay: 0
|
|
# PG 17 — accepted by the fork's Npgsql/EF Core provider (needs PG14+); the
|
|
# provider generates its own migrations on first start.
|
|
imageName: ghcr.io/cloudnative-pg/postgresql:17-system-trixie
|
|
instances: 3
|
|
logLevel: info
|
|
maxSyncReplicas: 0
|
|
minSyncReplicas: 0
|
|
monitoring:
|
|
customQueriesConfigMap:
|
|
- key: queries
|
|
name: cnpg-default-monitoring
|
|
disableDefaultQueries: false
|
|
enablePodMonitor: false
|
|
postgresql:
|
|
parameters:
|
|
archive_mode: "on"
|
|
archive_timeout: 5min
|
|
dynamic_shared_memory_type: posix
|
|
effective_cache_size: 256MB
|
|
full_page_writes: "on"
|
|
log_destination: csvlog
|
|
log_directory: /controller/log
|
|
log_filename: postgres
|
|
log_rotation_age: "0"
|
|
log_rotation_size: "0"
|
|
log_truncate_on_rotation: "false"
|
|
logging_collector: "on"
|
|
max_connections: "200"
|
|
max_parallel_workers: "16"
|
|
max_replication_slots: "16"
|
|
max_worker_processes: "16"
|
|
shared_buffers: 128MB
|
|
shared_memory_type: mmap
|
|
ssl_max_protocol_version: TLSv1.3
|
|
ssl_min_protocol_version: TLSv1.3
|
|
wal_keep_size: 256MB
|
|
wal_level: logical
|
|
wal_log_hints: "on"
|
|
wal_receiver_timeout: 5s
|
|
wal_sender_timeout: 5s
|
|
syncReplicaElectionConstraint:
|
|
enabled: false
|
|
primaryUpdateMethod: restart
|
|
primaryUpdateStrategy: unsupervised
|
|
probes:
|
|
liveness:
|
|
isolationCheck:
|
|
connectionTimeout: 1000
|
|
enabled: true
|
|
requestTimeout: 1000
|
|
replicationSlots:
|
|
highAvailability:
|
|
enabled: true
|
|
slotPrefix: _cnpg_
|
|
synchronizeReplicas:
|
|
enabled: true
|
|
updateInterval: 30
|
|
resources:
|
|
limits:
|
|
cpu: "1"
|
|
memory: 1Gi
|
|
requests:
|
|
cpu: 50m
|
|
memory: 512Mi
|
|
smartShutdownTimeout: 180
|
|
startDelay: 3600
|
|
stopDelay: 1800
|
|
storage:
|
|
resizeInUseVolumes: true
|
|
size: 10Gi
|
|
storageClass: cephrbd-fast-delete
|
|
switchoverDelay: 3600
|