Commit Graph

30307 Commits

Author SHA1 Message Date
unkin-agent 086fdb8257 fix(session): parse the owner key without assuming the pod id
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
2026-09-25 00:04:54 +10:00
unkin-agent 51261b0128 test(session): sample ownership after each non-owner request
ci/woodpecker/push/ci Pipeline was canceled
ci/woodpecker/pr/ci Pipeline was canceled
2026-09-25 00:01:42 +10:00
unkin-agent 9e66708d87 fix(session): tie directory ownership to the live connection
Ownership is claimed with a Lua check-and-set keyed on the instance holding
the websocket, routing prefers a live controller over a local copy, the
session list deduplicates by owner, removal is ownership-checked, undelivered
routed messages surface, single-session lookups stop scanning the keyspace and
directory writes leave the request path bounded by a timeout.
2026-09-24 23:50:01 +10:00
unkin-agent 6f362c33c9 fix(session): satisfy StyleCop member ordering and indentation
ci/woodpecker/pr/ci Pipeline was successful
ci/woodpecker/push/ci Pipeline was successful
2026-09-24 23:08:52 +10:00
unkin-agent 00d0765152 feat(session): share the session directory between instances
ci/woodpecker/pr/ci Pipeline failed
ci/woodpecker/push/ci Pipeline failed
Publish every session to valkey with the instance holding it, and route a
command for a non-local session to its owner over a per-instance pub/sub
channel. Entries expire, so a dead instance leaves the directory.
2026-09-24 22:48:48 +10:00
benvin 6691b785c3 Merge pull request #32 from benvin/userdata-cache
ci/woodpecker/push/ci Pipeline was successful
fix(userdata): read user data through to the database
2026-09-21 23:13:37 +10:00
benvin 2adb13f50f Merge pull request #29 from benvin/config-propagation
ci/woodpecker/push/ci Pipeline was successful
fix(ha): propagate shared-config and library-visibility changes between replicas
2026-09-21 21:44:34 +10:00
benvin 1c98f4a074 Merge pull request #31 from benvin/pg-tests-in-ci
ci/woodpecker/push/ci Pipeline was successful
test(db): run the PostgreSQL provider tests in CI
2026-09-21 07:45:29 +10:00
benvin 393994a454 Merge pull request #30 from benvin/nextup-datetime-kind
ci/woodpecker/push/ci Pipeline was canceled
fix(api): normalise the Next Up cutoff to UTC
2026-09-21 07:42:33 +10:00
benvin ad50c4e433 Merge pull request #28 from benvin/gated-task-keys
ci/woodpecker/push/ci Pipeline was canceled
fix(ha): gate the remaining timer-driven scheduled tasks
2026-09-21 07:40:32 +10:00
unkin-agent 483c739fb1 report a peer's port change locally without rewriting it
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
Keep IConfigurationManager source-compatible for plugin implementers.
2026-09-21 00:46:16 +10:00
unkin-agent 1c59e6afcb perf(nextup): batch the user data reads the next episode selection makes
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
Next Up walked every series in an unpaginated loop and read user data one
episode at a time, so the Home screen row cost two queries per series once the
cache was gone.

- read the played state of every candidate episode in one query
- read the versions the resume check and the last played date need in one query each
- default PrefetchedUserData on IUserBaseItemComparer so plugin comparers still compile
- cover the bounded query count and the plugin comparer with tests
- share one database across the replica tests
2026-09-21 00:36:39 +10:00
unkin-agent d39ec60e2c test(db): link PostgreSqlTestServer instead of referencing Jellyfin.Server.Tests
ci/woodpecker/pr/ci Pipeline was successful
ci/woodpecker/push/ci Pipeline was successful
2026-09-21 00:29:11 +10:00
unkin-agent a9d6c749fb keep an applied invalidation from inducing a write that publishes back
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
2026-09-21 00:24:01 +10:00
unkin-agent 44b62dcc64 test(ha): pin the default gated task key set
ci/woodpecker/pr/ci Pipeline was successful
ci/woodpecker/push/ci Pipeline was successful
2026-09-20 23:55:17 +10:00
unkin-agent a7919b9bac fix(userdata): read user data through to the database
ci/woodpecker/pr/ci Pipeline was successful
ci/woodpecker/push/ci Pipeline was successful
The user data cache and the item's in-memory rows were both filled once per
replica and never invalidated, so a pod serving a playback tick read its own
stale resume position and wrote it back over the position another pod had just
saved, silently losing resume points, played state, favourites and ratings.

- drop the user item data cache
- read single and batched user data from the database on every query
- prefetch user data for in-memory sorts and filters so each stays one query
- cover the lost update against real PostgreSQL with two manager instances
2026-09-20 23:51:29 +10:00
unkin-agent a025655b4d test(db): run the PostgreSQL provider tests in CI
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
Point Jellyfin.Database.Tests.PostgreSQL at the server postgres-migration-chain
already runs, give every test its own database, and fix the two tests that only
ever failed against real PostgreSQL.
2026-09-20 23:48:43 +10:00
unkin-agent face8ac653 fix(api): normalise the Next Up cutoff to UTC
ci/woodpecker/pr/ci Pipeline was successful
ci/woodpecker/push/ci Pipeline was successful
The query-string cutoff binds as DateTimeKind.Unspecified, which Npgsql
refuses to write to timestamp with time zone, so /Shows/NextUp 500s.
2026-09-20 23:47:23 +10:00
unkin-agent b662ffa48f propagate shared-config and library-option changes between instances
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
Add a Redis pub/sub invalidation bus, publish from the configuration and library-option write paths, and drop the matching local cache entry on the instances that did not write. No-op without a Redis connection string, and fails open when it is unreachable.
2026-09-20 23:47:21 +10:00
unkin-agent 1965c68a76 fix(ha): gate the remaining timer-driven scheduled tasks
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
Add the eight provider, live TV and plugin-update task keys to
ScanLeaderOptions.GatedTaskKeys and widen the test's task-key discovery to
every assembly that declares an IScheduledTask.
2026-09-20 23:36:34 +10:00
benvin ec581b5e5f Merge pull request 'fix(ha): gate library tasks on the scan leader by default' (#11) from benvin/scan-leader-default into main
ci/woodpecker/push/ci Pipeline was successful
Reviewed-on: #11
2026-09-13 15:04:38 +10:00
unkin-agent b77702c851 merge: resolve CoreAppHost conflict with main, drop dead redisConnectionString local
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
2026-09-13 14:48:26 +10:00
benvin 2370787471 Merge pull request 'fix(db): collapse presentation-key groups without min(uuid)' (#10) from benvin/fix-guid-aggregate-pg into main
ci/woodpecker/push/ci Pipeline was successful
Reviewed-on: #10
2026-09-13 14:25:19 +10:00
benvin a652d718ca Merge pull request 'fix(ha): read transcode store config from the variable form deployments set' (#9) from benvin/ha-config-wiring into main
ci/woodpecker/push/ci Pipeline was canceled
Reviewed-on: #9
2026-09-13 14:24:11 +10:00
benvin 0c707c7989 Merge pull request 'fix(devices): read devices and device options through to the database' (#8) from benvin/fix-device-cache into main
ci/woodpecker/push/ci Pipeline was canceled
Reviewed-on: #8
2026-09-13 14:23:17 +10:00
unkin-agent 91655fcb0a fix(ha): gate library tasks on the scan leader by default
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
Jellyfin:ScanLeader:Enabled defaults to false and nothing sets it, so
leader election never runs and every replica executes the timer-driven
library tasks concurrently - the exact behaviour the lease prevents.

- Enable gating by default when a Redis connection string is configured
- Honour an explicit Enabled setting either way
- Carry the effective decision onto the bound options the task worker reads
- Log at startup whether gating is active
- Warn when gating is enabled but no Redis connection string is configured
2026-09-13 13:19:48 +10:00
unkin-agent 6e405af1dc fix(db): collapse presentation-key groups without min(uuid)
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
PostgreSQL has no min(uuid) aggregate, so every query that picked a group
representative with MIN over the item id failed with 42883: library browse,
search, Recently Added, the by-name endpoints and Upcoming.

- Pick the representative with an anti-join on (primary version, id)
- Cover the collapse with a repository test against a real PostgreSQL
2026-09-13 13:18:11 +10:00
unkin-agent 95220e2ed6 fix(ha): read transcode store config from the variable form deployments set
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
The startup configuration only reads JELLYFIN_ prefixed environment
variables, so the bare Jellyfin__TranscodeStore__* form used by the chart,
the manifests and the README is dropped and the Redis store is never
registered. Nothing logs the selected store, so the fallback is invisible.

- Read bare Jellyfin__* variables into the Jellyfin:* configuration root
- Keep an explicit JELLYFIN_ variable winning over the bare form
- Log the selected transcode session store at startup
- Ping Redis once at startup and log an unreachable store at Error
- Log endpoints only, never the connection string
2026-09-13 13:10:52 +10:00
unkin-agent 56919b9565 fix(devices): read devices and device options through to the database
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
The device cache was filled once at construction, so a token minted by one
replica was unknown to every other replica already running and a token revoked
on one replica stayed valid on the others until they restarted.

- drop the eager device and device options dictionaries
- read devices and device options from the database on every query
- push the device query filters and ordering into SQL
- open the request's database context only for the api key fallback
- cover both directions against real PostgreSQL with two manager instances
2026-09-13 13:08:47 +10:00
benvin 25269263ce Merge pull request 'test(db): run the startup migration chain against PostgreSQL in CI' (#7) from benvin/ci-postgres-migration-chain into main
ci/woodpecker/push/ci Pipeline was successful
Reviewed-on: #7
2026-09-12 22:22:12 +10:00
benvin b183bec94b Merge pull request 'fix(db): materialise the rating list before updating' (#6) from benvin/fix-rating-levels-pg into main
ci/woodpecker/push/ci Pipeline was canceled
Reviewed-on: #6
2026-09-12 22:21:30 +10:00
unkin-agent bc42ff4b28 test(db): drop a leftover test database before recreating it
ci/woodpecker/pr/ci Pipeline was successful
ci/woodpecker/push/ci Pipeline was successful
A server handed in through JELLYFIN_TEST_POSTGRES outlives the run, so a second
run finds the databases the first one created. Also name the failures in
Jellyfin.Database.Tests.PostgreSQL the CI step steps around.
2026-09-12 21:36:09 +10:00
unkin-agent b501ba9620 test(db): run the startup migration chain against PostgreSQL in CI
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
Every PostgreSQL test carries Category=RequiresDocker and the pipeline filters
that category out, so the provider production runs on was never exercised and
two upgrade-path bugs reached it in a row. Nothing covered the whole startup
sequence either - code and schema migrations interleaved the way the migration
service orders them.

- run both database stages through JellyfinMigrationService against a seeded
  library and against a fresh install
- take the server from JELLYFIN_TEST_POSTGRES when it is set, else start a
  container
- add a pipeline step that runs the PostgreSQL tests on every push and pull
  request, with the server inside the step: the kubernetes backend has no
  docker daemon, and a service container deadlocks on the workspace volume
2026-09-12 21:28:17 +10:00
unkin-agent acab85ada3 merge the rating levels fix so the chain test has it 2026-09-12 21:20:27 +10:00
unkin-agent ad834167d5 ci: probe service containers
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/push/probe Pipeline was canceled
2026-09-12 21:03:15 +10:00
unkin-agent f59ab11c43 fix(db): materialise the rating list before updating
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
MigrateRatingLevels issued an ExecuteUpdate while the SELECT DISTINCT reader
was still open on the same connection. SQLite tolerates that, Npgsql does not,
so the AppInitialisation stage aborted and every PostgreSQL instance
crash-looped on startup.

- read the distinct ratings into a list before the update loop
- cover the migration against a real PostgreSQL, with NULL and empty ratings
2026-09-12 21:00:18 +10:00
benvin 10fed9409d Merge pull request 'fix(db): restore the PostgreSQL upgrade path' (#4) from benvin/fix-postgres-upgrade into main
ci/woodpecker/push/ci Pipeline was successful
Reviewed-on: #4
2026-09-12 17:55:56 +10:00
benvin f2166df463 Merge pull request 'fix(config): read an unusable encoding.xml EncoderPreset as the default' (#5) from benvin/fix-encoder-preset-fallback into main
ci/woodpecker/push/ci Pipeline was successful
Reviewed-on: #5
2026-09-12 17:43:53 +10:00
unkin-agent 3b60289502 fix(config): read an unusable encoding.xml EncoderPreset as the default
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
An element the enum cannot parse throws, and the configuration manager
catches that and returns defaults, so one bad preset discarded every other
encoding setting.

- serialize EncoderPreset through a string surrogate in XML
- fall back to auto for an empty, unknown or out-of-range value
2026-09-12 16:26:01 +10:00
unkin-agent 2f9b8888e9 revert: move the encoding.xml EncoderPreset fix to its own branch
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
It is an unrelated subsystem and ships as a separate change.
2026-09-12 16:21:47 +10:00
unkin-agent 65af2bcbd7 fix(db): harden the PostgreSQL upgrade against odd legacy data
- accept only the two id shapes the uuid cast parses, so misplaced hyphens
  null out instead of aborting the migration
- clear an OwnerId that already is the detached placeholder, matching the
  SQLite chain, so CleanupOrphanedExtras cannot delete the item
- recreate the placeholder item before the repoint if it is missing
- compare the migrated schema against the model, constraints included
2026-09-12 16:21:42 +10:00
unkin-agent 2e3d24cc05 fix(db): restore the PostgreSQL upgrade path
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
The v12.0 rebase replaced the PostgreSQL provider's initial migration instead of
adding to it, so an existing database kept the pre-12.0 schema while the code
migrations ran against it and startup aborted on a missing NormalizedUsername.

- restore 20260305010333_InitialPostgreSql as the baseline
- add 20260306000000_UpgradeToServer12Schema carrying it to the 12.0 model
- convert OwnerId and PrimaryVersionId to uuid with explicit casts, clear
  unparseable ids and repoint dangling owners at the placeholder item
- drop orphaned permissions and preferences before UserId becomes non-nullable
- add 20260524120336_AddUniqueNormalizedUsernameIndex after the code migration
  that fills the column in
- read an empty or unknown encoding.xml EncoderPreset as the default
- cover both paths against a real PostgreSQL and guard the migration ordering
2026-09-12 15:32:12 +10:00
benvin dcb6ae396c Merge pull request 'Re-apply HA fork patches on upstream v12.0' (#3) from benvin/rebase-v12 into main
ci/woodpecker/push/ci Pipeline was successful
Reviewed-on: #3
2026-09-12 12:48:35 +10:00
unkin-agent c438ec17ed feat(helm): expose the transcode session retention window
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
The chart sets the lease duration but not the retention window that keeps an
orphaned session record available for takeover.

- add ha.transcodeStore.sessionRetentionSeconds and pass it to the statefulset
2026-09-12 12:43:46 +10:00
unkin-agent f54bc76767 chore: merge main history into the v12.0 rebase
The branch is a fresh rebase onto the v12.0 tag, so it shares no history with
main and cannot be merged normally.

- record main as a parent without changing a single file
2026-09-12 10:19:36 +10:00
unkin-agent baa16b6586 fix: make transcode leases ownership-checked and cleanup-aware
Cleanup never matched a live session because the controller registered empty
manifest and segment paths, renewal was a read-modify-write that could revert a
takeover, and the takeover script compared an ISO date to a number, so it errored.

- populate the session record's manifest and segment paths from the playlist path
- renew the lease via a Lua compare-and-set on the owning pod
- store the lease expiry as unix milliseconds so the scripts can compare it
- retain the session record past its lease so an orphan can still be taken over
- test the Redis store against a real Redis, including the renew-vs-takeover race
- drop the live stream record nothing ever read back
2026-09-12 10:19:29 +10:00
unkin-agent d825f8ac81 ci: keep the package cache off the workspace volume
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/pr/ci Pipeline was successful
The workspace volume is 9.8G and the clone plus build output already fill 5.7G,
so a shared package cache there leaves too little free space to test against.

- restore, build and test in one step with the cache on ephemeral storage
- exclude BackupServiceTests, which requires 5GiB free on the workspace
- print workspace free space before the test run
2026-09-12 01:03:59 +10:00
unkin-agent 0148f374ee ci: grant ephemeral storage and harden the package install
ci/woodpecker/push/ci Pipeline failed
ci/woodpecker/pr/ci Pipeline failed
The backup service test needs 5GiB free on the workspace and the apt mirror
occasionally serves a half-synced index, so both fail the pipeline at random.

- request and limit ephemeral storage on every step
- retry apt-get update and verify fontconfig is loadable before testing
- report workspace free space before the test run
- run the full test filter again
2026-09-12 00:57:51 +10:00
unkin-agent d8c7eb5d3d ci: skip the backup service test on the pipeline workspace
ci/woodpecker/pr/ci Pipeline was successful
ci/woodpecker/push/ci Pipeline failed
BackupService refuses to write a backup with less than 5GiB free, which the
pipeline workspace volume does not have, so the suite fails on disk size alone.

- exclude BackupServiceTests from the test filter
- keep restore, build and test as separate steps
- install fontconfig for the Skia tests before running them
2026-09-12 00:51:47 +10:00
unkin-agent 03ac3bbc00 ci: run each test suite as its own step
ci/woodpecker/push/ci Pipeline failed
ci/woodpecker/pr/ci Pipeline failed
A solution-wide test run reports one exit code, so a failing suite cannot be
identified without the pipeline logs.

- run the suites carrying HA tests as separate named steps
- run the remaining upstream suites in one step after installing fontconfig
2026-09-12 00:44:47 +10:00